Prototype Consensus, Finish out ISRs and hook up BE/FE ISRs + Conflicts
Feb 28
Sprint 49
Getting issues...
Due March 14: Finish hooking up ISRs (Completed/Finalized), Initiate Consensus, move Review Configuration into Solicitation tab, make views possible in UI for different roles.
Sprint 50
Getting issues...
Initiate Consensus, see (SRP) scores as Conflicted and Non Conflicted SRP member, notification on finalized ISRs and less than 3 non conflicted reviewers.
Sprint 51
Getting issues...
Consensus Comments and consensus related Scores, ability to export a file for ISR comments and score
one email per reviewer per change is sent out to the mailing list (tta-notifications-test)
NOT IMPLEMENTED
notifications to tta member/ srp chair are blocked until Identity management system is up
REVIEW
The ticket says that "Conflict Reason, and updated Conflict State" should be included in email but it is currently not
right now, theres is a bulk update on conflict cerification on the front end, so there are emails for every proposal instead of just the ones that were changed.
Notification when there are less than 3 available reviews for a proposal.
When a reviewer certifies conflicts, all of the proposals that do not have at least 3 available or unknown reviewers per proposal are sent in a single email.
The list in the email will have all of the proposals affected. It is triggered when SRP members certify. It does not trigger when the auto conflicts are managed.
Review Configuration Tab is with Solicitation creation now
Create a Test Solicitation+Test Proposals
Sprint 46
Create a simple solicitation with hardcoded Science Categories and Hardware. The Proposal Process can be specified. The number of proposals it creates with 1 of 3 Science categories can be specified. The names and information on the proposal is randomly generated.
Author Copy is viewable and modifiable when in Draft State and Submitted if the Solicitation is Open
Observatory Copy is generated when Proposal is Submitted
Author and Observatory Copies are both viewable if
the Solicitation is Open
the Proposal State is Submitted
Can edit Author Copy after a submit if solicitation open
Can resubmit if Solicitation is open and PPR
Cannot edit Observatory Copy ever
Cannot edit or view an Author Copy when proposal is In Review
Can view only Observatory Copy when Proposal State is In Review
A Proposal with a Proposal State of Submitted should not be able to be deleted.
As TTA member:
Can edit Author Copy when the Proposal has any State (only up to In Review possible to test though).
Can Save and it persists. Should not affect Obs Copy if one exists.
Can Submit it only when call is open.
BUG
Can edit the Observatory Copy, if one exists but save applies to Author Copy only. There is no way to submit, so changes do not affect downstream (review phase) information.
Can withdraw a proposal in the Hidden State.
Withdrawn proposal still shows up in Review phase.
A Hidden proposal cannot be submitted, regardless of the Solicitation being open or not. It should be allowed to be submitted at anytime.
REVIEW
navigating between author and observatory copy is not intuitive
Information in Observatory Copy is not displaying correctly or ever in OS case
BUG
Saving changes to author copy shows a warning on the list of proposals and allows resubmission only after navigating away and then back again. The button is not always available when it should be.
Submitting a proposal works but the view of the proposal is missing afterwards and until a refresh is done.
Proposal State Changes
Allowed:
Draft → Submitted
Draft → Hidden
Submitted → Withdrawn
Submitted → In Review
In Review → Withdrawn
In Review → Completed
Hidden → Submitted (TTA only)
Not Allowed:
(Submitted, In Review, Completed, Closed) → Hidden
Hidden → Withdrawn
Sprint 42
Sprint
Sprint 42
WORKING
submission either goes to submitted or in review (DDT); Proposal State change to Completed is out of scope for v0.2
Withdrawing a proposal works: the author can see the Observatory Copy but not edit either it or the author copy. Cannot even access the author copy. Tooltip shows the action is not available.
BUG
can delete a proposal which causes the state to change to hidden; however, author can see the Observatory Copy!
Draft → Withdrawn is currently allowed but it should not be.
REVIEW
Withdraw button is only available in the edit mode of a proposal for the TTA member; it should be higher.
Print a single or multiple proposals
A user can print a proposal or more
the science justification is included in the view
all of the components of the proposal are concluded in the view (need definition)
Drop down menu on available Science Categories should either be linked to Solicitations or the drop down menu on the available science categories on the vetting should be linked to the master list.
How saving affectings the persisted changed
TTA Member can access vetting comments after Configuration in Finalized and forever more.
PPRs - Review Configuration
Description
JIRA
Implemented
Last Working
Tested
Status
Panels
Create SRP panels with unique names
Delete SRP Panels
Associate panels with 1 or more Science Categories (edge cases)
Assign 1 or more SRP Chairs
Assign multiple SRP Members to panel
Auto bulk assign proposals to panels based on SC
Manually assign proposals to panels
Save
Feasibility groups (back burner)
Assign independent science review to a member of a different SRP
Upload and use a configuration file
Filtering on Panels/proposals
Should not be able to assign the same person to multiple panels
Automatic proposal to panel assignment based on Science Category
saving
finalizing
NOT IMPLEMENTED
feasibility groups on placeholder
TO FIX
Should be able to select multiple chairs (currently can only do one)
Want a reviewer focused display
Icon on number of proposals assigned to panel needs updating.
Drop down menu on available Science Categories should either be linked to Solicitations or the drop down menu on the available science categories on the vetting should be linked to the master list.
Should not be able to assign the same person to multiple panels.
The Finalize button is available when all proposals are vetted and SC are assigned but I can use it before the close of a Solicitation.
TO FIX
Finalize button gives an error but seems to work
The finalize button is available before the Solicitation is Closed. It would be nice it were there but grayed out. Eventually with a tooltip that states the condition necessary for it to work.
If the Solicitation isn't Closed, I can see the Configured Review Panels in the Reviews tab, but the Assignment page does not populate.
No confirmation on Finalization
Error messaging should be more robust here.
Finalizing should prevent future edits to configuration.
Configuration and Vetting is available prior to the close of a Solicitation
Can update Conflict state and Certify (preferrable that it is done after user certifies)
TO FIX
ISRs are not affected downstream yet.
PPRs - Review Type Assignments
Description
JIRA
Implemented
Last Working
Tested
Status
SRP Chair
can see a "matrix" of Reviewers in their panel, their Conflict States, the Review States of their ISRs
can assign Review Type per ISR
Only 1 Primary, 1 Secondary, * Tertiary, * None per Proposal
Default is None
Can change Review Types unless ISR is Finalized
Cannot be able to assign types of Conflicted reviewers
Cannot change Review Type of External Reviewer
External Reviewer's name is redacted but can see the Review Type and Review State for the external ISR
Sprint 46/47
Sprint 59
Sprint 59
WORKING
can see a "matrix" of Reviewers in their panel and their Conflict State
Only one Primary and one Secondary can be saved to a proposal.
Any number of None or Teritary can be saved to a proposal.
Cannot assign Review Types to a conflicted reviewer
Cannot change Review Type of External Reviewer
External Reviewer's name is redacted but can see the Review Type and Review State for the external ISR
TTA Member
Can change any Review Type even if the Review State is Finalized until Consensus starts
Can change the Review Type of external ISRs until Consensus of the hosting panel begins
Sprint 45/47
Sprint 59
Sprint 59
WORKING
Can change RT of external ISR as long as Consensus for the hosting panel has not started
Can change RT of any ISR even in Finalized State until the start of Consensus
PPRs - ISRs
Description
JIRA
Implemented
Last Working
Tested
Status
SRP Reviewer
See list of proposals (after conflict certification)
Review Type is displayed
Default is None; otherwise P,S,T
Review State is displayed
default is Blank; otherwise Saved, Completed, Finalized, Closed
Conflict State is displayed
Conflict/Available
If not Conflicted, have an ISR in which to enter comments and a score and see the proposal
can Save (Review State → Saved) per ISR
validate if score not equal to 0; otherwise it cannot save
does not validate comments
can Complete (Review State → Completed) per ISR
can Finalize (bulk action) if any ISRs with Review not equal to None are not Closed or Finalized
If Conflicted, only see title, abstract, proposal ID and cannot enter any information
All comments and scores are saved (though not propagated to consensus; see later)
Normalized Individual Scores are calculated on Finalize
Edge cases to test:
If an external proposal is added after reviewer has finalized, they should be able to review and finalize that one proposal (finalize is still bulk in theory)
If consensus has started for their panel, the Finalized Normalized Individual Score in that panel may be inconsistent with their new Normalized Individual Score (since it has the extra proposal). This is acceptable.
External Reviewer is someone who is on any one panel associated with the Solicitation in question
An external reviewer can be assigned an external ISR for any proposal as long as the Panel associated with that proposal has not started Consensus
External Reviewers do not have to declare conflicts on external ISRs. When assigned, the conflict state is automatically set to Available
ER do not have access to the Consensus UI or any responsibility at Consensus for the panel associated with the external ISR.
TO FIX
External Reviewers can see the external ISR in their list of ISRs.
ER can provide comments, scores, save, complete, and finalize their external ISRs
Import/Export ISRs
a CSV file is expected but it can use as a delimiter any of the following:
, ; | tab
putting double quotes around the comment will keep the commas within the comment safe
if reviews exist when asked to export, it will export the reviews
cannot upload a file with any blank fields (fails and warning issued)
if a user changes the name of the reviewer within the file, it gives an error
protects against Bobby Drop Tables
on a successfull upload, the Review State of the ISR will changed to Saved (not sure what it does for completed or a re-upload)
TTA Member
can change Review State to Closed
Auto changes Review Type to None
If a TTA member Finalizes a SRP member's ISRs (for the reviewer), it will finalize the ISRs that have a valid comment and score (Review State of Saved, Completed) and Review Type not equal to None. If a comment+score is not valid, regardless of the Review Type, it will Close the ISR (unsure if it still ignores review types of None)
can change Review Type until Consensus is initated for the SRP member's panel.
can change Review State utnil Consensus is initiated for the SRP member's panel.
When TTA member uses Finalize, as a bulk action for all ISRs for that reviewer
Review Types of Primary, Secondary, Tertiary but have Review State of Blank are set to Close
Review Type of None is ignored, regardless of Review State
Review Types of P,S,T that have Review States in (Saved, Completed) are transitioned to Finalized
A separate Close functionality will be per ISR
Adding exernal reviewers as TTA member
If an external reviewer is added, the Conflict state is automatically Available.
PPRs - Consensus
Description
JIRA
Implemented
Last Working
Tested
Status
SRP Member
As a conflicted SRP member, only see the SRP scope, proposal ID, title, abstract (BE only)
As a non-conflicted SRP member, I want to see the comments, scores, and proposals for which I am not conflicted (BE only)
Comments are not connected to reviewers
Normalized Individual Score
Finalized Normalized Individual Score
Mean Normalized Score
Standard Deviation of the Mean Normalized Score
SRP Score
As a non-conflicted SRP member and with a Review Type of Primary or Secondary, I want to enter consensus comments. (BE only)
All members that are not conflicted can view Consensus comments. (BE only)
Be able to see the updated SRP score, if it is changed by the chair or via changes to the Finalized Normalized Individual Score. Push or refresh. (BE only)
See Feasibility Reviews (on hold)
As a Primary or Secondary, I want to complete a consensus comment
Can enter comments for PI, Technical, Data Management + an internal for each
Score
Needs to be Binary
Can Save
Can Finalize which "dangles" but next step is out of scope. It does change the review state to Finalized, as expected. Proposal State doesn't change yet (not in scope).
NOT IMPLEMENTED
Hard to tell which proposals have been reviewed.
TO FIX
Score needs to be binary
Version 0.3 Outline
Description
JIRA
Implemented
Last Working
Tested
Status
Scheduler (during ISRs)
Source Conflicts
Scheduler (post Consensus)
Construct Preliminary Allocation Dispositions (priorities and time)
Update/Test a solicitation when a different one is Open and has submitted proposals
NOT IMPLEMENTED
Solicitation Tab
Sprint 37.3
Sprint 37.3
Only available to TTA Member
Navigation
Description
JIRA/Behavior
Implemented
Last Working
Tested
Status
As an Author, I would like to view my proposals.
in a list
in a detailed view (Card)
<Sprint 29
Sprint 37
Sprint 37
WORKING
Saving within a CR
Capability Requests have their own save button. Though information looks like it is kept on the frontend, it is not in the backend without hitting save.
Change CR1 → don't save → navigate to CR2, change CR2 → save
Only changes to CR2 are kept
CR1 changes are not saved
Change CR1 → don't save → navigate to CR2, change CR2 → don't save → navigate to CR2 (changes are still present in front end) → save
Changes to CR1 are persisted
Changes to CR2 are not persisted
A refresh without hitting save does not persist work
