Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Agenda:



  • Is this actually automatic or does the TTA member initiate this?
    Jira
    serverDMS JIRA
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
    keySTT-904
  • Currently, do we use the Review State of Finalized/Closed to prevent edits to ISRs?
    Jira
    serverDMS JIRA
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
    keySTT-879
    • (How) Do we prevent modification of ISRs with Review Type = None currently?
    • Suggestion: At initiation of Consensus sub-phase, all ISRs are immutable for the SRP member


  • What do we mean by "Reviews" in the UI currently? (Allie)
    • Proposal Process
    • "Review Process" 


  • Here is a link to some GBO DSS documentation to clarify the distinction between DSS and PHT.


Further discussion on

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-907

  • TTA member can modify ISRs (Review State, comments and score) up until Consensus starts for a panel
    • Note, this means they cannot Close an ISR once Consensus starts.
  • TTA member can change Review Types until the NLR is generated
  • TTA member can modify Consensus reviews (Review State, comments, score) up until Disposition Letters are sent.

Consequences of changing ISR Review States and Review Types

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1201

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySTT-1200

Recall:

  • Finalizing ISRs is a per reviewer action. It changes the Review State to Finalized for To Finalize, the ISRs with Review type Type of  PrimaryPrimary, Secondary, or Tertiary must be in Review States of Saved or Completed.
    • To Finalize, the For ISRs with Review Type Types of Primary, Secondary, Tertiary must be in the Saved/Completed state.P, S, or T, the Finalize action changes the Review State to Finalized. 
    • If the Review Type is None, the Review State does not change.
  • Finalizing calculates the Normalized Individual Score (NIS) using any ISR all ISRs with a Review State of Finalized and a Review Type of PrimaryP, SecondaryS, or TertiaryT. Note, this condition is different that than what the ticket tickets explicitly stated stated
    Jira
    serverDMS JIRA
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
    keySTT-1166
    and 
    Jira
    serverDMS JIRA
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
    keySTT-879
    . This modified behavior is more useful though.
    • ISRs with review state Review States of Finalized or Closed cannot be modified by SRP members.
    • There is currently nothing stopping an SRP member from editing ISRs that have a Review State other than Finalized or Closed.
  • To start Consensus, all ISRs in a panel with Review Types of P,S, or T must have Review States State of Finalized.

Use Case

A SRP member has Finalized their ISRs. It is prior to Consensus startingReviews.

  • If the Review Type is None, then the Review State can only be Blank/Saved/Completed/Closed.
    • If the Review Type is not changed and the Review State is not Closed/Finalized, then the SRP member are is (as of Sprint 50) able to continue editing the review their ISR and save edits into perpetuity.
      • (1) Even though the comments and score won't go to Consensus, it is best to have a hard stop to their edits at some point.
      • Proposed Fix: Initiating Consensus should prevent edits by SRP members to any ISRs associated with that panel.
    • If a TTA member changes the Review Type to P, S, or T and Review State is not Closed/Finalized, then the SRP member can modify their review as needed. The Finalize button will trigger.
      • (2) The correct NIS's will propagate into Consensus and the TTA member cannot start Consensus until the SRP member finalizes (again). This is the outcome/system behavior we desire.
    • If the TTA member changes the Review State is changed from Closed to Blank/Saved/Completed, then 1 or 2 applies.
    • If the Review State is changed to Closed, then no further action is needed.
      • This is the outcome/system behavior we desire.
    • The TTA member should never change the a single ISR's Review State to Finalized per ISR in this case. It should always be done via the bulk Finalize action (UI button), which acts on all ISRs per reviewer. Otherwise, the NIS won't be recalculated correctly.
  • If the Review Type is P, S, or T, then the Review State is Finalized.
    • If a TTA member changes the Review State to Blank/Saved/Completed and does not change the Review Type to None, Consensus cannot be initiated until the SRP member elects to Finalize their ISRs again. Note, it does not matter if the Review Type is changed between P, S, or T.The correct NIS's will propagate into Consensus and the TTA member cannot start Consensus until the SRP member finalizes (again; same as 2 above) 
      • See (2) above.
    • If a TTA member changes the Review Type between P, S, and T, no changes occur that affect ISRs, the NIS, or the ISR comments that go to Consensus. It only changes access the ability to modify to Consensus Reviews.
      • This is the outcome/system behavior we desire.
    • If a TTA member changes the Review State to Blank/Saved/Completed and changes the Review Type to None:
      • (3) The previously calculated NIS does not reflect this change as there is currently not an automatic recalculation of the NIS. Furthermore, the Finalize button will never trigger for the SRP member, so there is no way for the SRP member to force the recalculation.
      • They also will be able to edit ISRs with Review Type of None into perpetuity (see 1 above).
      • (4) The TTA member could use the Finalize button for the SRP member, which would solve resolve this issue.
        • Recall that if the TTA member uses the Finalize button for an SRP member:
          • all of their ISRs with Review Type of P,S, or T with review states of Blank → Closed
          • ISRs with Review Types of P, S, T and states of Saved/Completed → Finalized
          • ISRs with Review Types of None have no state change
          • ISRs with Review States of Finalized and Closed have no state change
          • Only ISRs with Review States of Finalized and Review Types of P, S, or T are used to calculate NIS.
        • So, the correct NIS would be calculated in this event.
    • If a TTA member changes the Review State to Closed, the Review Type is always forced to None. Further edits are prevented because of the Closed state but the NIS is out of sync now:
      • See (3).
      • See (4).
    • If a TTA member does not change the Review State of Finalized but changes the Review Type to None:
      • See (3).
      • See
      • Should there be a recalculation of the NIS?
        Jira
        serverDMS JIRA
        columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
        columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
        serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
        keySTT-1166
      • If so, see (3) and (4).

In Consensus, an ISR review state, comments, and score cannot be modified by anyone, but a TTA member is able to modify the Review Type.  At the start of Consensus, the NIS per reviewer is copied into Finalized NIS.

Changes to the Review Type really should only determine the SRP member's ability to enter Consensus Comments

the FNIS.

  • Edge casescase 1:
    • SRP member A for Panel 1 finalizes their ISRs. Their Consensus Meeting starts for Panel 1.For a good reason, the TTA member changes one of SRP member A's ISR's ISRs to Review Type = to None, which previously had a Review Type = Secondary.The ISR Review State equals Finalized and now the Review Type = None.  
    • This change only means that SRP member A cannot enter or complete Consensus Comments for that proposal. This change cannot affect the NIS or FNIS or any other score or available availability of ISR comments because all of those objects are defined at the start of Consensus.
  • Edge case 2:
    • The Then, SRP member A is subsequently assigned an external ISR after edge case 1 occurs. They do their new ISR and hit finalize.
      • The ISR with a Review State of Finalized and a Review Type of None will be excluded from the calculation of the new NIS. Their The new NIS will be discrepant from the NIS shown in Panel A but this is acceptable.

Summary:

  • The TTA member should always have access to the bulk Finalize button, regardless of the triggering criteria, in order to "fix" the NIS if an out of process action is taken.
  • A TTA member should, by good practice, never change the individual Review State of an ISR to Finalized. Finalizing should only be done via the bulk action to ensure the NIS is properly calculated.
  • Changing the Review Type to None during Consensus will not change the NISs or ISR comments available to the Consensus panel.

Open Actions:

Task report
pages68321335