Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Updated 702.25.23 for sprint 58 (→ v0.2)01.2024

Detailed TTA Tools Sprint Feedback Page

...

Table of Contents
outlinetrue

Summary of Available Solicitations, Facilities, and Capabilities

...


Section
Column


Solicitations for review

SolicitationCurrent Test Solicitation?Notes
test solicitationno; retired Sprint 37
a realistic-looking solicitation

Sprint 37

Gone after Sprint 37

VLA: 4, P, L, S, C, X, Ku, K, Ka, Q

GBT: 4, P, L, S, C, X, Ku, K, Ka, Q (even if some are nonsensical for the GBT)

20D

Sprint 37

Gone after Sprint 37

Specific Hardware Constraints:

GBT: P- and K-band (400 MHz, 22 GHz)

VLA: Ka- and L-band (33 GHz, 1.5 GHz)

Sem_25ASprint 38 - current, prod v0.1.2

VLA and GBT have L – Q band available

Production Solicitation for v0.1

SC_GBT_24BSprint 38 - current, prod v0.1.2

GBT only, L, S, and C band

Production Solicitation for v0.1

Note: The Solicitation Json lists the basebands that are included in the solicitation.

Column
Solicitation Specific Facility/Capability for testing


a realistic-looking solicitationCapability
Facility VLA ContinuumGBT Spectral Line
real VLASprint ?? >--
GBT--Sprint 28 >
VLA (replaced real VLA)Sprint 37.2

Prototype FE stories:

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1142

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1007

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1060

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1167

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1088




Helpful Files for Testing


FileUpdatedIntended CapabilityDescription
Sprint 38GBT Spectral Line50 Field Sources from the Megamaser use case; the use case specifics a  rms line sensitivity of 6 mJy
Sprint 38GBT Spectral Line1 Spectral Spec from Megamaser use case; Units are GHz, km/s, km/s (extra spectral spec but can delete)
Sprint 38GBT Spectral Line

General GBT Spectral Line SS.

There are 2 SS in this file: KFPA and Q band

Sprint 38VLA Continuum2 Field Sources from the polarization/continuum use case (W4); use case specifies a rms sensitivity of 100 microJy/bm. RA DEC in hms/dms.
Sprint 38VLA Continuum

2 SS (C-, X-band); Units in GHz, GHz, GHz

VLA_SS_allbands.csv

GBT_SS_allbands.csv

FieldSources_3_VLA.csv

FieldSources_3_GBT.csv

Sprint 39

v0.1.2

VLA Continuum

GBT Spectral Line

General Test Files


Easy Navigation to Jira Epics and Sprints

NameJiraDescription

Sprint 48

Jira
serverDMS JIRA
jqlQuerysprint = 154
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Prototype Consensus, Finish out ISRs and hook up BE/FE ISRs + Conflicts

Feb 28

Sprint 49

Jira
serverDMS JIRA
jqlQuerySprint = 158
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Due March 14: Finish hooking up ISRs (Completed/Finalized), Initiate Consensus, move Review Configuration into Solicitation tab, make views possible in UI for different roles.
Sprint 50

Jira
serverDMS JIRA
jqlQuerySprint = 160
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Initiate Consensus, see (SRP) scores as Conflicted and Non Conflicted SRP member, notification on finalized ISRs and less than 3 non conflicted reviewers.
Sprint 51

Jira
serverDMS JIRA
jqlQuerysprint = 161
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Consensus Comments and consensus related Scores, ability to export a file for ISR comments and score

Start of UI overhaul

Sprint 52

Jira
serverDMS JIRA
jqlQuerysprint = 162
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Start UI/DM/CA reconciliation

Utility functions: upload/export ISRs, print proposals

BE work on finalizing consensus and generating NRL

Sprint 53

Jira
serverDMS JIRA
jqlQuerysprint = 175
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Refactoring 1/?
Sprint 54

Jira
serverDMS JIRA
jqlQuerysprint = 176
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Refactoring 2/?
Sprint 55

Jira
serverDMS JIRA
jqlQuerysprint = 180
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Refactoring 3/?
Sprint 56

Jira
serverDMS JIRA
jqlQuerysprint = 183
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Refactoring 4/?
Sprint 57

Sprint 58

Jira
serverDMS JIRA
jqlQuerysprint = 191
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

connecting FE/BE consensus. System fully connected
Sprint 59

Jira
serverDMS JIRA
jqlQuerysprint = 199
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

bug hunting and final feature push
SRP ProcessSprint 60

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-930

EpicSRP Configuration

jqlQueryissueKey in (STT-1202,STT-1323,STT-1324,STT-1325,STT-1330,STT-1336,STT-1338,STT-1341,STT-1351,STT-1357,STT-1358,STT-1360,STT-1369,STT-1376,STT-1378,STT-1379,STT-1381,STT-1382,STT-1384,STT-1387,STT-1388,STT-1394,STT-1395,STT-1398,STT-1404,STT-1406,STT-1205,STT-1349,STT-1355,STT-1393,STT-1397,STT-1401,STT-1403,STT-1407,STT-1156,STT-1399)
counttrue
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-929

EpicOSRbug hunting and final feature push
Sprint 61

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
jqlQuerysprint = 214
counttruecolumnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-438

EpicProposal Submissionbug hunting; final official sprint of version 0.2
Sprint 62

Jira
serverDMS JIRA
jqlQuerysprint = 217
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

prototype v0.2.1 interface; RIDs for internal view
SRP Process

Jira
serverDMS JIRA
columnIdscolumnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-434930

EpicFeasibility
ReviewsSRP Configuration

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1035929

Epic
V0.2 BugsOSR

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1059438

EpicSolicitation
ConfigurationProposal Submission

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-932434

Epic

Proposal
BugsFeasibility Reviews

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-10341035

EpicLabel: NotificationRelated
V0.2 Bugs

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (NotificationRelated)
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolutioncounttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label

keySTT-1059

Epic
Solicitation ConfigurationLabel: ConsensusReview

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (ConsensusReview)
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolutioncounttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label

keySTT-932

Epic
Proposal BugsLabel: Conflicts

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (Conflicts)
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolutioncounttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1034

EpicLabel
Label: ConfigurabilityNotificationRelated

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (ConfigurabilityNotificationRelated)
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label
Label: ISRConsensusReview

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (ISRConsensusReview)
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label

Label: TTAMemberConflicts

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (TTAMemberConflicts)
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label
Label: SRPChairConfigurability

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (SRPChairConfigurability)
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label

...

Label: ISR

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (ISR)
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label
Label: TTAMember

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (TTAMember)
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label
Label: SRPChair

Jira
serverDMS JIRA
jqlQueryproject = "SSA TTA Tools" and labels in (SRPChair)
counttrue
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

Label


Current Report

Landing Page

...

DescriptionJIRAImplementedLast WorkingTestedStatus
  • Message of the Day on landing page

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-754


Sprint 34Sprint 37Sprint 37

Status
subtleTrue
colorgreen
titleworking

but! it would be nice if there was more feedback when the save successful.



Notifications

DescriptionJIRAImplementedLast WorkingTestedStatus

Notification for a Submitted Proposal

  • emails are sent to the specified emails informing the TTA member, PIs/coPIs of a successfull submission

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-924



Sprint 47

Sprint

48

68

V0.2

Sprint

48

68

V0.2

Status
subtletrue
colourGreen
colorgreen
titleworks

  • upon a successful submission, a notification is sent to a mailing list (tta-notifications-test)
  • this should be true on every resubmission
  • this should work for OSRs and PPRs

Status
subtleTrue
colourBlue
colorgreen
titlenot implemented

  • notifications to PI/Co-Is are blocked until Identity management system is up


Notification when Review Configuration is Finalized

  • emails are sent to SRP members and chairs when the panels have been configured.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-882

Sprint 47

Sprint 68

V0.2

Sprint 68

V0.2

Status
subtletrue
colourYellowGreen
colorgreen
titleto testWorking

  • when the Finalize button is used by a TTA member in the Review Configuration, a notification is sent to a mailing list (tta-notifications-test)Allie, yes; Dana, no emailed received
  • one email is sent per panel per reviewer
    • Seems to be only per panel

Status
subtleTrue
colourBlue
colorgreen
titlenot implemented

  • notifications per panel to SRP members/ Chairs are blocked until Identity management system is up
  • did not send an email to feasibility reveiwers reviewers (on hold)


Notification when TTA Member changes Conflict State of Reviewer

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1145

Sprint 47

Sprint

48

68

V0.2

Sprint

48

68

V0.2

Status
subtletrue
colourGreen
colorgreen
titleWorking

  • one email per reviewer per change is sent out to the mailing list (tta-notifications-test)

Status
subtleTrue
colourBlue
colorgreen
titlenot implemented

  • notifications to tta member/ srp chair are blocked until Identity management system is up

Status
subtleTrue
colourYellow
colorgreen
titleReview

  • The ticket says that "Conflict Reason, and updated Conflict State" should be included in email but it is currently not
  • right now, theres there is a bulk update on conflict cerification certification on the front end, so there are emails for every proposal instead of just the ones that were changed.


Notification when Consensus Review is finalized

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-914




Status
subtletrue
colourYellowGreen
colorgreen
titleto testWorking

  • email is sent to TTA member per panel that all consensus review have been finalized by chair


Notification when all ISRs are finalized for a panel

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1036

Sprint 50

Sprint 68

V0.2

Sprint 68

V0.2

Status
subtletrue
colourGreen
colorgreen
titleWorking

Notification when there are less than 3 N available reviews for a proposal.

  • When a reviewer certifies conflicts, all of the proposals that do not have at least
3
  • N available or unknown reviewers per proposal are sent in a single email.
  • The list in the email will have all of the proposals affected. It is triggered when SRP members certify. It does not trigger when the auto conflicts are managed.
  • N is a configurable number

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1037

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1182

Sprint 50

Solicitation Creation

Sprint 61

V0.2

Sprint 61

V0.2

Status
subtletrue
colourGreen
colorgreen
titleWorking

  • tta member can specifiy per panel the minimum number of reviewers. the default is three
  • email is sent to TTA list if a proposals has M number of available or unknown reviewers if M < N

Solicitation Creation

...

DescriptionJIRAImplementedLast WorkingTestedDescriptionJIRAImplementedLast WorkingTestedStatus

Create a Simple Solicitation from JSON for

  • VLA
  • GBT

not implemented/requirements not in a story:

  • modify parameters such as frequency ranges on bands

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-342

Solicitation Config Files


Sprint 16Sprint 16Sprint 30

Status
subtleTrue
colourGrey
colorgreen
titlehack for release

Configure Field Source, SS, PP, CP





Status
subtleTrue
colourBlue
colorgreen
titlenot implemented

Configure Advance Tab options





Status
subtleTrue
colourBlue
colorgreen
titlenot implemented

Update/Test a solicitation when a different one is Open and has submitted proposals



 

Status
subtleTrue
colourBlue
colorgreen
titlenot implemented
Solicitation Tab

Sprint

37

61

V0.

3

2

Sprint

37

61

V0.

3

2

Only available to TTA Member

A Proposal Prefix can be specified per solicitation

 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-866

Sprint 40Sprint 40Sprint 40

Status
subtleTrue
colourBlue
colorgreen
titleWorking

Review Configuration

Status
color
subtleTrue
colourYellow
green
titleTesting

Review Configuration Tab is with Solicitation creation now

Create a Test Solicitation+Test Proposals
Sprint 46


Create a simple solicitation with hardcoded Science Categories and Hardware. The Proposal Process can be specified. The number of proposals it creates with 1 of 3 Science categories can be specified. The names and information on the proposal is randomly generated.

Close Solicitation

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1141

Sprint 45

Sprint

47

61

V0.2

Sprint

47

61

V0.2

Force close a Solicitation, sending all proposals into In Review (if PPR).



Navigation

...


DescriptionJIRA/BehaviorImplementedLast WorkingTestedStatus

As an Author, I would like to view my proposals.

  • in a list
  • in a detailed view (Card)

<Sprint 29Sprint 37Sprint 37

Status
subtletrue
colourGreen
colorgreen
titleworking

Status
subtleTrue
colourYellow
colorgreen
titleReview

  • would like to filter on proposals by Science Category


Saving within a CR

Capability Requests have their own save button. A user is warned if they will lose their changes from navigating.

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-976





Status
subtleTrue
colourGreen
colorgreen
titleworking

Saving and AR

A user is warned if they will lose their changes from navigating away.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-976




Status
subtleTrue
colourGreen
colorgreen
titleworking

Observation Specification Generation

Prompt warns that user is overwriting their OS.

Pops up even if no OS is currently generated or if one cannot be generated (validation needed?)

If OS cannot be generated, tab is available but empty.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-972

<Sprint 35Sprint 38Sprint 38

Status
subtletrue
colourGreen
colorgreen
titleworking

Navigate and Edit OS on fresh AR



Not Implemented

Status
subtleTrue
colourGrey
colorgreen
titleignored for release

Review Tab
Sprint 43

Sprint 61

V0.2

Sprint 61

V0.2

43

Status
subtleTrue
colourYellowBlue
colorgreen
titleTestingWorking

Can see panels once Review Configuration is Finalized

Can see Review Panel as SRP Member for panel which they are assigned to

Can see Review Panel as SRP Chair for panel which they are assigned to

TTA Member can see both SRP and Chair views

Submission

DescriptionJIRAImplementedLast WorkingTestedStatus
  • Submit a Proposal
  • Proposal ID and timestamp generated
  • Light Validation on Submission
  • Submission affects Proposal State
    • Submitted for PPR
    • In Review for OSR


Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-850

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-860

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-851


Sprint 41Sprint 43Sprint 43

 

Status
subtleTrue
colourGreen
colorgreen
titleworking

timestamp is in MST, not local? – sprint 43 seems to be local now

Observatory Copy and Author Copy

  • As an Author:
    • Author Copy is viewable and modifiable when in Draft State and Submitted if the Solicitation is Open
    • Observatory Copy is generated when Proposal is Submitted
    • Author and Observatory Copies are both viewable if
      • the Solicitation is open
      • the Proposal State is Submitted
    • Can edit Author Copy after a submit if solicitation open
      • Can resubmit if Solicitation is open
    • Cannot edit Observatory Copy ever
    • Cannot edit an Author Copy when proposal is In Review
    • Can only see Observatory Copy after Close of Call
    • A Proposal with a Proposal State of Submitted should not be able to be deleted.
  • As TTA member:
    • Can edit Author Copy when the Proposal has any State (only up to In Review possible to test though).
      • Can Save and it persists. Should not affect Obs Copy if one exists.
      • Can Submit it (when call is open?)
    • Can edit the Observatory Copy, if one exists. Save persists and affects downstream.
    • A Proposal with a Proposal State of Hidden can be submitted if the Solictation is Open or Closed.
    • A Proposal with a Proposal State of Hidden cannot be Withdrawn
    • A Proposal with a Proposal State of Withdrawn should not appear in the Review Phase.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-863

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-861

Sprint 42Sprint 42Sprint 42

Status
subtleTrue
colourGreen
colorgreen
titleworking

  • As an Author:
    • Author Copy is viewable and modifiable when in Draft State and Submitted if the Solicitation is Open
    • Observatory Copy is generated when Proposal is Submitted
    • Author and Observatory Copies are both viewable if
      • the Solicitation is Open
      • the Proposal State is Submitted
    • Can edit Author Copy after a submit if solicitation open
      • Can resubmit if Solicitation is open and PPR
    • Cannot edit Observatory Copy ever
    • Cannot edit or view an Author Copy when proposal is In Review
    • Can view only Observatory Copy when Proposal State is In Review
    • A Proposal with a Proposal State of Submitted should not be able to be deleted.
  • As TTA member:
    • Can edit Author Copy when the Proposal has any State (only up to In Review possible to test though).
      • Can Save and it persists. Should not affect Obs Copy if one exists.
      • Can Submit it only when call is open.

Status
subtleTrue
colourRed
colorgreen
titlebug

    • Can edit the Observatory Copy, if one exists but save applies to Author Copy only. There is no way to submit, so changes do not affect downstream (review phase) information.
    • Can withdraw a proposal in the Hidden State.
    • Withdrawn proposal still shows up in Review phase.
    • A Hidden proposal cannot be submitted, regardless of the Solicitation being open or not. It should be allowed to be submitted at anytime.

Status
subtleTrue
colourYellow
colorgreen
titlereview

  • navigating between author and observatory copy is not intuitive
  • Information in Observatory Copy is not displaying correctly or ever in OS case

Status
subtleTrue
colourRed
colorgreen
titlebug

  • Saving changes to author copy shows a warning on the list of proposals and allows resubmission  only after navigating away and then back again.  The button is not always available when it should be.
  • Submitting a proposal works but the view of the proposal is missing afterwards and until a refresh is done.


Proposal State Changes

Allowed:

  • Draft → Submitted
  • Draft → Hidden
  • Submitted → Withdrawn
  • Submitted → In Review
  • In Review → Withdrawn
  • In Review → Completed
  • Hidden → Submitted (TTA only)

Not Allowed:

  • (Submitted, In Review, Completed, Closed) → Hidden
  • Hidden → Withdrawn

Sprint 42SprintSprint 42

Status
subtleTrue
colourGreen
colorgreen
titleworking

  • submission either goes to submitted or in review (DDT); Proposal State change to Completed is out of scope for v0.2
  • Withdrawing a proposal works: the author can see the Observatory Copy but not edit either it or the author copy. Cannot even access the author copy. Tooltip shows the action is not available.

Status
subtleTrue
colourRed
colorgreen
titlebug

  • can delete a proposal which causes the state to change to hidden; however, author can see the Observatory Copy!
  • Draft → Withdrawn is currently allowed but it should not be.

Status
subtleTrue
colourYellow
colorgreen
titleReview

  • Withdraw button is only available in the edit mode of a proposal for the TTA member; it should be higher.

Print a single or multiple proposals

  • A user can print a proposal or more
  • the science justification is included in the view
  • all of the components of the proposal are concluded in the view (need definition)

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-915

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-916

Sprint 52

Status
subtleTrue
colourGreen
colorgreen
titleworking

Back end supports printable view

Status
subtleTrue
colourYellow
colorgreen
titleReview

Navigation to this functionality missing in UI

Status
subtleTrue
colourRed
colorgreen
titlebug

Print view is missing Observation Specifications (need to tackle larger DM issues maybe)

Print view is missing Science Justification (might be a trade off with other issues)

PPRs

PPRs - Vet Proposals

...

  • Drop down menu on available Science Categories should either be linked to Solicitations or the drop down menu on the available science categories on the vetting should be linked to the master list.
  • How saving affectings the persisted changed
  • There is an interface with which to vet Science Categories
  • Can see the Title, Abstract, and Requested Science Category per proposal
  • the Vetted SC = the Requested SC at first, but it can be changed.
  • There is a place to record notes; they persist.
  • The proposal can be opened for viewing
  • There is a check box per proposal to indicate it has been vetted; there is a "vet all" button.
  • Changes persist
  • proposals can be assigned to panels
  • Hidden proposals are not shown in Vetting; i.e., only those with Proposal States of In Review

DescriptionJIRA/RequirementsImplementedLast WorkingTestedStatus
  • There is an interface with which to vet Science Categories
  • Can see the Title, Abstract, and Requested Science Category per proposal
  • the Vetted SC = the Requested SC at first, but it can be changed.
  • There is a place to record notes; they persist.
  • The proposal can be opened for viewing
  • There is a check box per proposal to indicate it has been vetted; there is a "vet all" button.
  • There is a save button
  • the vetting/panel assignment configuration persists
  • proposals can be assigned to a panel
  • Hidden proposals are not shown in Vetting; i.e., only those with Proposal States of In Review
  • Changing a SC and back again reverts the badge appropriately
  • Guidance Text to tell TTA member missing components necessary to Finalize (vet all, assign panels, create panels per SC, chair assigned to panel)


Jira
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-874

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-890

Sprint 44

Sprint 61

Sprint 48

V0.2

Sprint 61

V0.2

 

Status
subtleTrue
colourBlue
colorgreen
titleWorking

  • changing a SC and back again reverts the icon
Status
subtleTrue
colourRed
colorgreen
titleto fix
Status
subtleTrue
colourRed
colorgreen
titleto fix

  • Drop down menu on available Science Categories should either be linked to Solicitations or the drop down menu on the available science categories on the vetting should be linked to the master list.


TTA Member can access vetting comments after Configuration in Finalized and forever more.




PPRs - Review Configuration

...

  • Automatic proposal to panel assignment based on Science Category
  • saving
  • finalizing
  • Delete reviewers
  • SRP members can only belong to one panel
  • Assign multiple chairs

 

DescriptionJIRAImplementedLast WorkingTestedStatus

Panels

  • Create SRP panels with unique names
  • Delete SRP Panels
  • Associate panels with 1 or more Science Categories (edge cases)
  • Assign 1 or more SRP Chairs
  • Assign multiple unique SRP Members to panel
  • SRP members can only belong to one panel
  • Auto bulk assign proposals to panels based on SC (in vetting UI)
  • Manually assign proposals to panels
  • Save
  • (in vetting UI)
  • Changes to panel persist
  • Feasibility groups (back burner)Assign independent science review to a member of a different SRP
  • Upload and use a configuration file
  • Filtering on Panels/proposals
  • Can delete reviewers
  • Guidance Text to tell TTA member missing components necessary to Finalize (vet all, assign panels, create panels per SC, chair assigned to panel)Should not be able to assign the same person to multiple panels


Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-867

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-868

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-869

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-870

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-872

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-883

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1066

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-998

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1077

________________________

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-896

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-897

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-898

Sprint 43

Sprint 61

V0.2

Sprint 61

V0.2

Sprint 43

Status
subtleTrue
colourBlue
colorgreen
titleWorking

  • can create new SRP panels
  • can select from a pre-set list of SRP panels
  • can assign SRP members/chair
  • can assign proposals to SRP
Status
subtleTrue
colourYellow
colorgreen
titleTesting
Status
subtleTrue
colourYellow
colorgreen
titleNot implemented

  • feasibility groups on placeholder

Status
subtleTrue
colourRed
colorgreen
titleto fix

  • Should be able to select multiple chairs (currently can only do one)
  • Want a reviewer focused display
  • Icon on number of proposals assigned to panel needs updating.
  • Drop down menu on available Science Categories should either be linked to Solicitations or the drop down menu on the available science categories on the vetting should be linked to the master list.
  • Should not be able to assign the same person to multiple panels.



Finalize Review Configuration

  • Finalize can occur when
    • all proposals are vetted
    • the Solicitation is Closed
  • Pop-up asking In line prompt to confirm Finalization
  • Finalizing should prevent future edits to configuration.


Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-884

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-889

Jira
showSummaryfalse
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1412

Sprint 45

Sprint 61

V0.2

Sprint

47The Finalize button is available when all proposals are vetted and SC are assigned but I can use it before the close of a Solicitation.

61

V0.2

Status
subtleTrue
colourRedBlue
colorgreen
titleto fixWorking

  • Finalize can occur when
    • all proposals are vetted
    • the Solicitation is Closed
  • Finalizing should
  • Finalize button gives an error but seems to work
  • The finalize button is available before the Solicitation is Closed. It would be nice it were there but grayed out. Eventually with a tooltip that states the condition necessary for it to work.
    • If the Solicitation isn't Closed, I can see the Configured Review Panels in the Reviews tab, but the Assignment page does not populate.
  • No confirmation on Finalization
  • Error messaging should be more robust here.
  • Finalizing should prevent future edits to configuration.
Configuration and Vetting is available prior to the close of a Solicitation
  • Pop-up asking to confirm Finalization

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1168




Configuration and Vetting is part of the Solicitation UI nowProposal Process Tab



PPRs - Conflict Declaration

...

  • Must complete Conflicts before progressing to ISRs
  • Cannot change Auto Conflict
  • Default Conflict state is Unknown
  • Reviewer must change to either Conflict or Available
  • Reviewer may enter comments; persists
  • Can view Conflict State per ISR
  • Can see Proposal ID, Abstract, and Title
  • Bulk Action of Certification (cannot certify some and not others)
  • A valid certifications is one without a state of unknown and with comments if conflicted.
  • Certifying conflicts updates UI widgets that show "Certified" status
    • Individual Review list
    • Summary UI

 

DescriptionJIRAImplementedLast WorkingTestedStatus
  • Auto Conflicts based on authorship
  • Instructions present
  • Submit & Certify button is always there but not active until necessary
  • Highlights when option selected to keep track of progress


Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-884

Sprint 45

Sprint

48

61

V0.2

Sprint

48

61

V0.2

 

Status
subtleTrue
colourBlue
colorgreen
titleWorking

 

SRP Reviewer

  • Must complete Conflicts before progressing to ISRs
  • Cannot change Auto Conflict
  • Default Conflict state is Unknown
  • Reviewer must change to either Conflict or Available
  • Reviewer may must enter comments; persists
  • Can view Conflict State per ISR
  • Can see Proposal ID, Abstract, and Title
  • Bulk Action of Certification (cannot certify some and not others)
  • A valid certifications is one without a state of unknown and with comments if conflicted.
  • Certifying conflicts updates UI widgets that show "Certified" status
    • Individual Review list
    • Summary UI


Jira
serverDMS JIRA
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-885

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-875

Sprint 46Sprint 48Sprint 48

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1413

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1414

Sprint 46

Sprint 61

V0.2

Sprint 61

V0.2

Status
subtleTrue
colourBlue
colorgreen
titleWorking

  • Must complete Conflicts before progressing to ISRs
  • Cannot change Auto Conflict
  • Default Conflict state is Unknown
  • Reviewer
Status
subtleTrue
colourBlue
colorgreen
titleWorking
Status
subtleTrue
colourYellow
colorgreen
titlenot implemented
Status
subtleTrue
colourRed
colorgreen
titleto fix

SRP Chair

  • must complete conflicts like a reviewer
  • can see a "matrix" of Reviewers in their panel and their Conflict State
  • can assign Review Type per ISR
    • Only 1 Primary, 1 Secondary, * Tertiary, * None per Proposal
    • Default is None
    • Can change Review Types unless ISR is Finalized
  • Should not be able to assign types of Conflicted reviewers

Jira
server
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-887

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-871

Sprint 46/47

Sprint 61

V0.2

Sprint 61

V0.2

Status
subtleTrue
colourBlue
colorgreen
titleWorking

  • must complete conflicts like a reviewer
  • can see a "matrix" of Reviewers in their panel and their Conflict State
  • Only one Primary and one Secondary can be saved to a proposal.
  • Any number of None or Teritary can be saved to a proposal.
StatussubtleTruecolourYellowcolorgreentitlenot implemented StatussubtleTruecolourRedcolorgreentitleto fix
  • should not be able to assign Review Types to a conflicted reviewer


TTA Member

  • Can change Conflict State of reviewers
  • Valid Conflict State transitions for a TTA Member:
    • Available → Available
    • Available → Conflict
    • Conflict → Available
    • Conflict → Conflict
    • AutoConflict → Available
  • Proposed Consequences:
    • ISRs

TTA Member

  • Can change Conflict State of reviewers
  • Valid Conflict State transitions for a TTA Member:
    • Available → Available
    • Available → Conflict
    • Conflict → Available
    • Conflict → Conflict
    • AutoConflict → Available
  • Proposed Consequences:
    • ISRs either become available or unavailable as necessary for the SRP Member.
    • ISRs that are now Conflict State of Conflict that had Review Types other than None should automatically have Review Types changed to None.
  • Notifications sent
  • Conflicted ISRs do not go to Consensus

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-886

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1146

Sprint 45/47

Sprint 61

V0.2

Sprint 61

V0.2

Status
subtleTrue
colourBlue
colorgreen
titleWorking

  • Can update Conflict state and Certify (preferrable preferable that it is done after user certifies)
StatussubtleTruecolourRedcolorgreentitleto fix
  • ISRs are not affected downstream yet.