The objectives of this review are to ensure that:

• The L1 capabilities projected for delivery to Pilot Operations are ready for verification and validation.
• That the software system to be tested is under configuration control and is available in the software test environment.
• That any items not yet ready for test are identified with a plan to advance them to a state of completion and test prior to delivery to Pilot Operations.
• That a suite of tests sufficient for verification and validation have been defined.

Review Plan

• This is an internal SRDP Project Office review, the Review Plan is completely described within this Confluence page
• The Review is scheduled for May 3, 2019

Participants

• Project Office: Jeff, John, Mark, Bob
• DMS: Mark Whitehead, Stephan, Kana, Morgan, Andy, David, James

Artifacts Under Review

Pilot TRR Agenda

1. SRDP Project Director: Opening Remarks

2. DMS: Test Environment Ready (including configuration list)

3. SRDP PM: Present Verification and Validation RVTMs; review process to track V&V against the requirements
4. DMS:  Present L2 verification and validation completion report, based on Jira ticket status
5. DMS:  Present L1 Verification Plan and Test Procedures, defined within the set of Jira Test Tickets
6. SRDP Project Scientist: Present L1 Validation Plan and Test procedures defined within the set of Jira Test Tickets
7. SRDP Operations Mgr.:  Present Operations Process Test Plan defined within the set of Jira Test Tickets: SRDP-336; SRDP-337; SRDP-338, SRDP-340
8. SRDP Project Director: Preliminary assessment of Test Readiness, itemize any key blockers or major risk to resolve before proceeding
   

Outcome of Pilot Ops TRR

Report prepared by SRDP Project Director to include:

• Outcome of review to be one of the following:

  Initial outcome May 3, was a Fail.  Significant issues were identified in both the test configuration control, and feature completion.  Delta TRR scheduled for May 17 to review progress and assess.

Pilot (Delta) TRR Agenda - May 17, 2019

1. SRDP Project Director: Opening Remarks
2. SRDP PM: Review Open Actions from TRR
   • SRDP-364 - Configuration profiles for Test and Production environments
   • SRDP-363 - Scripts for workspace manager (see SSA-5599 - Getting issue details... STATUS  & discussion on Requirements for SSA work-around scripts for pilot)
     

3. SRDP PM: Review issues since TRR:

   Issue Number	Description	References	Impact	Reported by	Date	Owner/Assigned in Jira
   1	Outstanding requirement to allow selection of pre-release CASA versions within the Archive menu pull down described in SSA-5592 is not in the L1 Doc or model, should this be aded?	SSA-5592	Revision to Requirements and all downstream artifacts (model, V&V Test, etc), added work to DMS	Jeff Kern	5/13/2019	Richard Falardeau
   2	Spectral cutouts within a spectral window (ALMA Cube Imaging) can only be exported to a single file, unsure if additional exports are overwriting a previously generated file.  ALMA exports a single file since cutouts are not supported.	PIPE-345	Significant inconvenience, workarounds are in discussion	Joe Masters / John Tobin	5/13/2019	Kana
   3	The export of calibration products from the VLA pipeline run is addressed in PIPE-47 is not working as expected	Pipe-47	Potential blocker to Pilot Ops	John Tobin	5/13/2019	Kana
   4	SRDP-251 (ALMA Metadata) missing EBs although the DB was reported as copied from production	SRDP-251 (ALMA), SRDP-253 (VLA) & SSA-5583	Blocker to Pilot Ops due to partial validation	John Tobin	5/13/2019	John Tobin
   5	Permissions within archive to download to local directories	SRDP-209, SRDP-218, SSA-5268	Significant inconvenience, no progress since Dec 2018	John Tobin	5/13/2019	John Tobin

4. Any Other issues?
   

   
   

   1. Pass: All inputs are satisfactory, no additional risks found, proceed as scheduled.
   2. Risk Identified: One or more risks were identified requiring additional attention.  Risk is not deemed sufficient to affect the schedule.  Risk management plan shall be updated.
   3. Partial Pass: One or more risks are likely to impact schedule of the release.  Re-planning to address risk required.
   4. Fail: Sufficient risk identified to halt the release.  Re-planning or rescoping required.
      
      
5. Outcome less than an unconditional "Pass" shall include recommended steps and dates to rectify all discrepancies

6. SRDP Project Director: Closing Statement / next steps

Mid-Pilot TRR Agenda - July 9, 2019

Previous date for Mid-Pilot Go/No-Go Decision:  July 19, 2019

Revised date for Mid-Pilot Go/No-Go Decision: July 26, 2019

Participants

SRDP Project Office: John Tobin, Mark lacy, Bob Treacy

DMS: Morgan Griffith, Mark Whitehead, Stephan Witz, Kana Sugimoto, Joe Masters

1. SRDP PM: Review Open Actions required to deliver Pilot Operations Capability to Validation
   

• 
  
  1. PIPE-389 - Template recipe: creation of 'mfs' and 'cont' images
     • Status: Ready to validate
       
  2. SSA-5113 - Epic ticket: Implement ALMA optimized imaging cube capability
     • Review Issues under Epic for Blockers and completion target dates
     • Status: Actions captured to table below
       
  3. SSA-5330 Epic Ticket: Science Products, Ancillary Products and Associations
     
     • Review Issues under Epic for completion target dates
     • Status: Actions captured to table below
  4. (ML) Status of SSA-5599 Epic ticket - workaround scripts for Optimized Imaging workflow corresponding to requirements on Requirements for SSA work-around scripts for pilot, including SSA-5604, SSA-5632, SSA-5605.
  5. Identify Configuration changes (i.e. versions reflecting changes from these tickets):  CASA 5.4.2-7; Archive TBD
     
  6. Any Other issues?
     1. Account permissions etc for DAs and AoD (ML) to run jobs as almapipe. (Status: Captured as M7 in the table below)
        
  7. Review and Assign Action Items (Status: Actions captured to table below)
  8. Outcome:

  • • Not ready for full testing yet, though some tests can be conducted.

    • Pipeline is ready, with a follow up SSA item to incorporate pipeline changes in the template (item M1).

    • SSA is in an all-hands effort to finish the integration work for the product system (SSA-5330), with an estimate of 1-2 days before this is complete.  At that point (July 11) attention can be turned to wrapping up other action items listed below.

    • Changes, particularly to the Request Handler, will require extensive retesting of items previously tested for Pilot.  John is confident that he knows what needs to be tested.

    • Action items in the table below are needed to close the Mid-Pilot TRR and will be reviewed again on July 16.

Issue Number	Description	References	Impact	Reported by	Date	Owner/Assigned in Jira	Status	Due Date
M1	Apply PL changes to template	PIPE-389	No CLI E2E testing until we do this, works with bespoke PPRs	John Tobin	7/9	SW/Jim Sheckard	7/30 - Under validation. PIPE-389 is done, but John has held off on fully calling it validated until he can throw things through the workflow using its features. Ready to to integration testing, need CLI back up -
M2	SSA-5175 - CLI	SSA-5175 SSA-5645	CLI functional, but changes needed to make it useful for users; need about half a day to do, once product system working	John Tobin	7/9	Stephan Witz	7/11 - memory management issues? 8/6 - SSA-5645 (Critical) looks like the only unresolved ticket in this epic, Ready to Verify + Jim to test today, verify that CLI is working	8/13 - Jim 8/13 - should be viable after release
M3	Adding continuum image (switch)	SSA-5532	Generating continuum by default, on/off switch not a blocker	John Tobin	7/9	Stephan Witz	8/6 - Closed    Enabled by default, deferred the GUI switch	Done
M4	GUI Interface for Archive	Stephan will provide ticket number SSA-5392 SSA-5173	Not implemented on test yet, will be addressed after product system integration	John Tobin	7/9	Stephan Witz/ French	8/6 - SSA-5392 (design/prototype) Resolved  SSA-5173 - Ready to Validate, LSRK question Will get to test when "Rest Frequency" in GUI, won't wait for LSRK + John to send "must haves" for the GUI	8/12 - Deployment Blockers to be addressed in conjunction with testing 8/12 - already deployed
M5	Need time (day or two) to integrate, then will have status	SSA-5330	Integration/delivery of the product system items is an all-hands effort on the part of the SSA team which blocks work on the other tickets.	Stephan	7/9	Stephan Witz	8/6 - 20 tix. 2 completed/resolved, 0 validation, 7 verifying/ready, 10 scheduled, 1 open Integration of new DF into product system, end of day + Stephan will let Morgan know if it makes it by the EOD Operational workflows still being revived all user initiated workflows operational.	8/12 - Deployment 8/13 Deployment to Test
M6	SSA-5599, VLA implemented, ALMA in progress, 1) Script to notify Operations manager and DAs when imaging pipeline job is completed and ready for review: status - already exists 2) Script to rerun the imaging pipeline after edits to the PPR and/or extra flagging and/or changes to continuum subtraction: status - not yet done, but could be adapted from existing VLA recaldirs script. 3) Script to ingest image products into the archive and clean up the pipeline directory: status - not yet done (needs product system)	SSA-5599 dependent on SSA-5330	Scripts needed for ALMA, will address when product system items completed.	Mark Lacy	7/9	Stephan Witz	7/29 - Are there critical tasks in this epic, or do they all need to be done? 8/6 - 14 tix, 3 closed/resolved, 4 ready for validation, 2 ready/under verification, 4 scheduled Cube has ancillary prod, need to ingest cube with these products, have to hook this together and ingest a cube, check that DF works,    notification - 8/8    re-run - 8/8    ingest - later Stephan/Jim	8/8 - Stephan/Jim 8/12 - status the same: done/on the way/future 8/16 - Deployed, ready to test
M7	Run jobs as almapipe in CV, as they run as vlapipe, need to check/configure ACLs	SSA-5689	Operational efficiency, jobs should run the same way in both environments	Mark Lacy	7/9	Stephan Witz	8/6 - still an issue? (ticket last updated 7/11) ACL's for directory, let analysts into almapipe account	Done

Mid-Pilot Delta TRR Agenda - August 6, 2019

9:30 MT/ 11:30 ET SOC-280, ER-331

VideoHubIP: 192.33.117.12##6519
AudioHub: 434-817-6286

0) a) Review outcome of last TRR - Morgan, b) General status - Stephan

Products working, PL issue that we're working on, ingestion in progress (Rick & Jim), Rudumentary IF for triggering cube generation, working on feature requests (LSRK)

1) Review List from last TRR (above)

2) Scripts & script tickets - need to separate mid-Pilot script tickets from others: (SSA-5599 - see above, SSA-5621) - this epic has no tickets)

3) Any new things?

   What needs to be re-ingested - VLASS (for ancillary data), old, 6 wks+  ALMA cals run a one-off script, a couple of days

4) Next steps

   Sched a Friday tag up (Stephan, John, Jeff, 10 MT/12 ET)

   Reset key dates on Friday (Jeff)

   Test can go to 3.7

   5.4.2-8 released as of yesterday, Stephan make sure symlink is modified to pick it up (can't ready Cycle 7 ASDMs, OK for test

Key Dates (Tentative)

Aug 6 Mid-Pilot Delta TRR (this meeting)

Aug 20 Ends validation window

Aug 22 Operations Readiness Review

Aug 27 Operations Go/NoGo

Participants

SRDP Project Office: John Tobin, Mark Lacy, Bob Treacy

DMS: Morgan Griffith, Mark Whitehead, Stephan Witz, Kana Sugimoto, Joe Masters

Review Open Issues from July 9

Mid-Pilot Delta TRR Agenda - August 15, 2019

Outcome:  TRR is closed

Most critical items have been addressed so that partial testing can move forward.  AAT/PPI version 3.7 has been deployed to the test environment using CASA 5.4.2-8, some cleanup continues with scripts as a pacing item for Ops validation