The objectives of this review are to ensure that:

• The L1 capabilities projected for delivery to Pilot Operations are ready for verification and validation.
• That the software system to be tested is under configuration control and is available in the software test environment.
• That any items not yet ready for test are identified with a plan to advance them to a state of completion and test prior to delivery to Pilot Operations.
• That a suite of tests sufficient for verification and validation have been defined.

Review Plan

• This is an internal SRDP Project Office review, the Review Plan is completely described within this Confluence page
• The Review is scheduled for May 3, 2019

Participants

• Project Office: Jeff, John, Mark, Bob
• DMS: Mark Whitehead, Stephan, Kana, Morgan, Andy, David, James

Artifacts Under Review

Pilot TRR Agenda

1. SRDP Project Director: Opening Remarks

2. DMS: Test Environment Ready (including configuration list)

3. SRDP PM: Present Verification and Validation RVTMs; review process to track V&V against the requirements
4. DMS:  Present L2 verification and validation completion report, based on Jira ticket status
5. DMS:  Present L1 Verification Plan and Test Procedures, defined within the set of Jira Test Tickets
6. SRDP Project Scientist: Present L1 Validation Plan and Test procedures defined within the set of Jira Test Tickets
7. SRDP Operations Mgr.:  Present Operations Process Test Plan defined within the set of Jira Test Tickets: SRDP-336; SRDP-337; SRDP-338, SRDP-340
8. SRDP Project Director: Preliminary assessment of Test Readiness, itemize any key blockers or major risk to resolve before proceeding
   

Outcome of Pilot Ops TRR

Report prepared by SRDP Project Director to include:

• Outcome of review to be one of the following:

  Initial outcome May 3, was a Fail.  Significant issues were identified in both the test configuration control, and feature completion.  Delta TRR scheduled for May 17 to review progress and assess.

Pilot (Delta) TRR Agenda - May 17, 2019

1. SRDP Project Director: Opening Remarks
2. SRDP PM: Review Open Actions from TRR
   • SRDP-364 - Configuration profiles for Test and Production environments
   • SRDP-363 - Scripts for workspace manager (see  & discussion on Requirements for SSA work-around scripts for pilot)
     

3. SRDP PM: Review issues since TRR:

   Issue Number	Description	References	Impact	Reported by	Date	Owner/Assigned in Jira
   1	Outstanding requirement to allow selection of pre-release CASA versions within the Archive menu pull down described in SSA-5592 is not in the L1 Doc or model, should this be aded?	SSA-5592	Revision to Requirements and all downstream artifacts (model, V&V Test, etc), added work to DMS	Jeff Kern	5/13/2019	Richard Falardeau
   2	Spectral cutouts within a spectral window (ALMA Cube Imaging) can only be exported to a single file, unsure if additional exports are overwriting a previously generated file.  ALMA exports a single file since cutouts are not supported.	PIPE-345	Significant inconvenience, workarounds are in discussion	Joe Masters / John Tobin	5/13/2019	Kana
   3	The export of calibration products from the VLA pipeline run is addressed in PIPE-47 is not working as expected	Pipe-47	Potential blocker to Pilot Ops	John Tobin	5/13/2019	Kana
   4	SRDP-251 (ALMA Metadata) missing EBs although the DB was reported as copied from production	SRDP-251 (ALMA), SRDP-253 (VLA) & SSA-5583	Blocker to Pilot Ops due to partial validation	John Tobin	5/13/2019	John Tobin
   5	Permissions within archive to download to local directories	SRDP-209, SRDP-218, SSA-5268	Significant inconvenience, no progress since Dec 2018	John Tobin	5/13/2019	John Tobin

4. Any Other issues?
   

   
   

   1. Pass: All inputs are satisfactory, no additional risks found, proceed as scheduled.
   2. Risk Identified: One or more risks were identified requiring additional attention.  Risk is not deemed sufficient to affect the schedule.  Risk management plan shall be updated.
   3. Partial Pass: One or more risks are likely to impact schedule of the release.  Re-planning to address risk required.
   4. Fail: Sufficient risk identified to halt the release.  Re-planning or rescoping required.
      
      
5. Outcome less than an unconditional "Pass" shall include recommended steps and dates to rectify all discrepancies

6. SRDP Project Director: Closing Statement / next steps

Mid-Pilot TRR Agenda - July 9, 2019

Previous date for Mid-Pilot Go/No-Go Decision:  July 19, 2019

Revised date for Mid-Pilot Go/No-Go Decision: July 26, 2019

Participants

SRDP Project Office: John Tobin, Mark lacy, Bob Treacy

DMS: Morgan Griffith, Mark Whitehead, Stephan Witz, Kana Sugimoto, Joe Masters

1. SRDP PM: Review Open Actions required to deliver Pilot Operations Capability to Validation
   

• 
  
  1. PIPE-389 - Template recipe: creation of 'mfs' and 'cont' images
     • Status: Ready to validate
       
  2. SSA-5113 - Epic ticket: Implement ALMA optimized imaging cube capability
     • Review Issues under Epic for Blockers and completion target dates
     • Status: Actions captured to table below
       
  3. SSA-5330 Epic Ticket: Science Products, Ancillary Products and Associations
     
     • Review Issues under Epic for completion target dates
     • Status: Actions captured to table below
  4. (ML) Status of SSA-5599 Epic ticket - workaround scripts for Optimized Imaging workflow corresponding to requirements on Requirements for SSA work-around scripts for pilot, including SSA-5604, SSA-5632, SSA-5605.
  5. Identify Configuration changes (i.e. versions reflecting changes from these tickets):  CASA 5.4.2-7; Archive TBD
     
  6. Any Other issues?
     1. Account permissions etc for DAs and AoD (ML) to run jobs as almapipe. (Status: Captured as M7 in the table below)
        
  7. Review and Assign Action Items (Status: Actions captured to table below)
  8. Outcome:

  • • Not ready for full testing yet, though some tests can be conducted.

    • Pipeline is ready, with a follow up SSA item to incorporate pipeline changes in the template (item M1).

    • SSA is in an all-hands effort to finish the integration work for the product system (SSA-5330), with an estimate of 1-2 days before this is complete.  At that point (July 11) attention can be turned to wrapping up other action items listed below.

    • Changes, particularly to the Request Handler, will require extensive retesting of items previously tested for Pilot.  John is confident that he knows what needs to be tested.

    • Action items in the table below are needed to close the Mid-Pilot TRR and will be reviewed again on July 16.

Issue Number	Description	References	Impact	Reported by	Date	Owner/Assigned in Jira	Status
M1	Apply PL changes to template	PIPE-389	No CLI E2E testing until we do this, works with bespoke PPRs	John Tobin	7/9	SW/Jim Sheckard	7/30 - Under validation. PIPE-389 is done, but John has held off on fully calling it validated until he can throw things through the workflow using its features.
M2	SSA-5175 - CLI	SSA-5175 SSA-5645	CLI functional, but changes needed to make it useful for users; need about half a day to do, once product system working	John Tobin	7/9	Stephan Witz	7/11 - memory management issues? 8/6 - SSA-5645 (Critical) looks like the only unresolved ticket in this epic, Ready to Verify
M3	Adding continuum image (switch)	SSA-5532	Generating continuum by default, on/off switch not a blocker	John Tobin	7/9	Stephan Witz	8/6 - Closed
M4	GUI Interface for Archive	Stephan will provide ticket number SSA-5392 SSA-5173	Not implemented on test yet, will be addressed after product system integration	John Tobin	7/9	Stephan Witz/ Reid Givens	8/6 - SSA-5392 (design/prototype) Resolved  SSA-5173 - Ready to Validate, LSRK question
M5	Need time (day or two) to integrate, then will have status	SSA-5330	Integration/delivery of the product system items is an all-hands effort on the part of the SSA team which blocks work on the other tickets.	Stephan	7/9	Stephan Witz	8/6 - 20 tix. 2 completed/resolved, 0 validation, 7 verifying/ready, 10 scheduled, 1 open
M6	SSA-5599, VLA implemented, ALMA in progress, 1) Script to notify Operations manager and DAs when imaging pipeline job is completed and ready for review: status - already exists 2) Script to rerun the imaging pipeline after edits to the PPR and/or extra flagging and/or changes to continuum subtraction: status - not yet done, but could be adapted from existing VLA recaldirs script. 3) Script to ingest image products into the archive and clean up the pipeline directory: status - not yet done (needs product system)	SSA-5599 dependent on SSA-5330	Scripts needed for ALMA, will address when product system items completed.	Mark Lacy	7/9	Stephan Witz	7/29 - Are there critical tasks in this epic, or do they all need to be done? 8/6 - 14 tix, 3 closed/resolved, 4 ready for validation, 2 ready/under verification, 4 scheduled
M7	Run jobs as almapipe in CV, as they run as vlapipe, need to check/configure ACLs	SSA-5689	Operational efficiency, jobs should run the same way in both environments	Mark Lacy	7/9	Stephan Witz

Mid-Pilot delta TRR Agenda - August 6, 2019

9:30 MT/ 11:30 ET SOC-280, ER-331

VideoHubIP: 192.33.117.12##6519
AudioHub: 434-817-6286

0) General status - Stephan

1) Review List from last TRR (above)

2) Scripts & script tickets - need to separate mid-Pilot script tickets from others: (SSA-5999, SSA-5621)

3) Any new things?

4) Next steps

Key Dates (Tentative)

Aug 6 Mid-Pilot Delta TRR (this meeting)

Aug 20 Ends validation window

Aug 22 Operations Readiness Review

Aug 27 Operations Go/NoGo

Participants

SRDP Project Office: John Tobin, Mark Lacy, Bob Treacy

DMS: Morgan Griffith, Mark Whitehead, Stephan Witz, Kana Sugimoto, Joe Masters

Review Open Issues from July 9

Agenda