Page History

• There is an interface with which to vet Science Categories
• Can see the Title, Abstract, and Requested Science Category per proposal
• the Vetted SC = the Requested SC at first, but it can be changed.
• There is a place to record notes; they persist.
• The proposal can be opened for viewing
• There is a check box per proposal to indicate it has been vetted; there is a "vet all" button.
• the vetting/panel assignment configuration persists
• proposals can be assigned to a panel
• Hidden proposals are not shown in Vetting; i.e., only those with Proposal States of In Review
• Changing a SC and back again reverts the badge appropiratelyappropriately
  
• Guidance Text to tell TTA member missing components necessary to Finalize (vet all, assign panels, create panels per SC, chair assigned to panel)

Sprint 61

V0.2

Sprint 61

V0.2

Panels

• Create SRP panels with unique names
• Delete SRP Panels
• Associate panels with 1 or more Science Categories (edge cases)
• Assign 1 or more SRP Chairs
• Assign multiple unique SRP Members to panel
• SRP members can only belong to one panel
• Auto bulk assign proposals to panels based on SC (in vetting UI)
• Manually assign proposals to panels (in vetting UI)
• Changes to panel persist
• Feasibility groups (back burner)
• Upload and use a configuration file
• Filtering on Panels/proposals
• Can delete reviewers
• Guidance Text to tell TTA member missing components necessary to Finalize (vet all, assign panels, create panels per SC, chair assigned to panel)

Sprint 61

V0.2

Sprint 61

V0.2

Finalize Review Configuration

• Finalize can occur when
  • all proposals are vetted
  • the Solicitation is Closed
• Pop-up asking In line prompt to confirm Finalization
• Finalizing should prevent future edits to configuration.

Sprint 61

V0.2

Sprint 61

V0.2

• Auto Conflicts based on authorship
• Instructions present
• Submit & Certify button is always there but not active until necessary
• Highlights when option selected to keep track of progress

Sprint 61

V0.2

Sprint 61

V0.2

SRP Reviewer

• Must complete Conflicts before progressing to ISRs
• Cannot change Auto Conflict
• Default Conflict state is Unknown
• Reviewer must change to either Conflict or Available
• Reviewer must enter comments; persists
• Can view Conflict State per ISR
• Can see Proposal ID, Abstract, and Title
• Bulk Action of Certification (cannot certify some and not others)
• A valid certifications is one without a state of unknown and with comments if conflicted.
• Certifying conflicts updates UI widgets that show "Certified" status
  • Individual Review list
  • Summary UI

Sprint 61

V0.2

Sprint 61

V0.2

SRP Chair

• must complete conflicts like a reviewer

Sprint 61

V0.2

Sprint 61

V0.2

TTA Member

• Can change Conflict State of reviewers
• Valid Conflict State transitions for a TTA Member:
  • Available → Available
  • Available → Conflict
  • Conflict → Available
  • Conflict → Conflict
  • AutoConflict → Available
• Proposed Consequences:
  • ISRs either become available or unavailable as necessary for the SRP Member.
  • ISRs that are now Conflict State of Conflict that had Review Types other than None should automatically have Review Types changed to None.
• Notifications sent
• Conflicted ISRs do not go to Consensus

Sprint 61

V0.2

Sprint 61

V0.2

SRP Chair

• can see a "matrix" of Reviewers in their panel, their Conflict States, the Review States of their ISRs
• can assign Review Type per ISR
  • Only 1 Primary, 1 Secondary, * Tertiary, * None per Proposal
  • Default is None
  • Can change Review Types unless ISR is Finalized
• Cannot be able to assign types of Conflicted reviewers
• Can change Review Type of External Reviewer
• External Reviewer's name is redacted but can see the Review Type and Review State for the external ISR
• Can upload a configuration file
• Proposal ID, Proposal Title, Pop-out to Proposal

Sprint 61

V0.2

Sprint 61

V0.2

• Can upload a configuration file

TTA Member

• Can change any Review Type even if the Review State is Finalized until Consensus starts
• Can change the Review Type of external ISRs until Consensus of the hosting panel begins
• Can upload a configuration file

Sprint 61

V0.2

Sprint 61

V0.2

SRP Reviewer

• See list of proposals (after conflict certification)
• Review Type is displayed
  • Default is None; otherwise P,S,T
• Review State is displayed
  • default is Blank; otherwise Saved, Completed, Finalized, Closed
• Conflict State is displayed
  • Conflict/Available
• If not Conflicted, have an ISR in which to enter comments and a score and see the proposal
  • can Save (Review State → Saved) per ISR
    • validate if score not equal to 0; otherwise it cannot save
    • does not validate comments
  • can Complete (Review State → Completed) per ISR
  • can Finalize (bulk action) if any ISRs with Review not equal to None are not Closed or Finalized
• If Conflicted, only see title, abstract, proposal ID and cannot enter any information
• All comments and scores are saved (though not propagated to consensus; see later)
• Normalized Individual Scores are calculated on Finalize
• Cannot provide comments or scores for ISRs with a Review State of Closed
• Edge cases:
  • If an external proposal is added after reviewer has finalized, they should be able to review and finalize that one proposal (finalize is still bulk in theory)
  • If consensus has started for their panel, the Finalized Normalized Individual Score in that panel may be inconsistent with their new Normalized Individual Score (since it has the extra proposal). This is acceptable.
• Instructions present
• Finalize Reviews button always there but not active

Sprint 61

V0.2

Sprint 61

V0.2

• If an external proposal is added after reviewer has finalized, they should be able to review and finalize that one proposal (finalize is still bulk in theory)
• If consensus has started for their panel, the Finalized Normalized Individual Score in that panel may be inconsistent with their new Normalized Individual Score (since it has the extra proposal). This is acceptable.
• Cannot provide comments or scores for ISRs with a Review State of Closed
• Instructions present
• Finalize Reviews button always there but not active

External Reviewer

• External Reviewer is someone who is on any one panel associated with the Solicitation in question
• An external reviewer can be assigned an external ISR for any proposal as long as the Panel associated with that proposal has not started Consensus
• An external reviewer can be assigned an ISR even if their host panel is in Consensus.
• External Reviewers do not have to declare conflicts on external ISRs. When assigned, the conflict state is automatically set to Available
• External Reviewers can see the external ISR in their list of ISRs.
• ER can provide comments, scores, save, complete, and finalize their external ISRs
• ER do not have access to the Consensus UI or any responsibility at Consensus for the panel associated with the external ISR.
• ER scores and comment are available in Consensus and corretly integrated

Sprint 61

V0.2

Sprint 61

V0.2

Import/Export ISRs

• a CSV file is expected but it can use as a delimiter any of the following:
  • , ; | tab
• putting double quotes around the comment will keep the commas within the comment safe
• if reviews exist when asked to export, it will export the reviews
• cannot upload a file with any blank fields (fails and warning issued)
• if a user changes the name of the reviewer within the file, it gives an error
• protects against Bobby Drop Tables
• on a successful upload, the Review State of the ISR will changed to Saved
• reuploading will update the comments/score
• Don't need every ISR in file to work

Sprint 61

V0.2

Sprint 61

V0.2

TTA Member

• can change Review State to Closed
  • Auto changes Review Type to None
• If a TTA member Finalizes a SRP member's ISRs (for the reviewer), it will finalize the ISRs that have a valid comment and score (Review State of Saved, Completed) and Review Type not equal to None. If a comment+score is not valid, regardless of the Review Type, it will Close the ISR (unsure if it still ignores review types of None)
• can change Review Type until Consensus is initated for the SRP member's panel.
• can change Review State until Consensus is initiated for the SRP member's panel.

Sprint 61

V0.2

Sprint 61

V0.2

Adding external reviewers as TTA member

• If an external reviewer is added, the Conflict state is automatically Available.
• TTA member can change Review Type until Consensus starts

Sprint 61

V0.2

Sprint 61

V0.2

SRP Member

• As a conflicted SRP member, only see the SRP scope, proposal ID, title, abstract, scores, and Rank
• As a non-conflicted SRP member, I want to see the comments, scores, and proposals for which I am not conflicted
  
  • Comments are not connected to reviewers
  • Normalized Individual Score
  • Finalized Normalized Individual Score
  • Mean Normalized Score
  • Standard Deviation of the Mean Normalized Score
  • SRP Score
  • Rank
• As a non-conflicted SRP member and with a Review Type of Primary or Secondary, I want to enter consensus comments.
• All members that are not conflicted can view Consensus comments.
• Be able to see the updated SRP score, if it is changed by the chair or via changes to the Finalized Normalized Individual Score. Push only for now.
• See Feasibility Reviews (on hold)
• As a Primary or Secondary, I want to complete a consensus comment
  • Members can no longer edit, but chair can
• Warning them Completing Consensus
• Badges display Review Types/ Conflicts
• Import/Export Consensus Comments
• Badges display all reviewer roles
• Does not display other conflicts

Sprint 61

V0.2

Sprint 61

V0.2

SRP Chair

• Overview of consensus comment state (Completed, Saved)
• Overview of Review Types and Conflict State
• Ability to change SRP Score directly
• Update Finalized Normalized Individual Scores
  • Forces update to MNS, STD_MNS, and SRP if the SRP score was not manually changed already
• Have visibility to past disposition letters
• Review Consensus comments, modify comments, and complete and  finalize Consensus Reviews
  • Note, there is not a bookkeeping equivalent to "completed" from ISRs for SRP Chairs in this process
• Warning on Finalizing and Completing Comments
• Cannot see Conflicted comments and FNIS
• History of Changes of Consensus Reviews to be kept
• Can see all Reviewers by Review Type (badges with P,S, T, all T) and all Conflicted
• Scores shown to 100ths place

Sprint 61

V0.2

Sprint 61

V0.2

External ISRs

• Comments and Scores from External ISRs are shown and properly integrated into FNIS
• External reviewers names are redacted for all but the TTA member. Instead, any reference to them is given as "External Reviewer"

Sprint 61

V0.2

Sprint 61

V0.2

• Comments and scores properly integrated

TTA Member

• Initiate Consensus per panel if all ISRs are Finalized
• Generate Normalized Linear Rank when all consensus reviews are finalized per panel.

Sprint 61

V0.2

Sprint 61

V0.2

Uploading PPR Proposal Reviews (Removed)

As SRP

• download contains all review comments for the panel
• uploading must only contain proposals that are RT = P/S (and not conflicted).
• Cannot modify SRP Score in an upload
• Fails if any others are included.