Page History

...

• Pipeline rerun script:
  • DA needs to be able to rerun the pipeline in the correct environment with added flag template file(s) and possibly a modified PPR.
    • Input: execution block ID,  (optional)  flagtemplate modified PPR, flagtemplate.txt
    • Results: new pipeline run run using modified PPR and/or flagtemplate.txt (if supplied)
• Archive ingest script: 
  • needs to enable the DA to add a QA report file (text) to the weblog, set an SRDP QA state (pass/fail/null) and ingest the calibration products and the flagtemplate.txt and flagtargetstemplate.txt file (if created by the DA) for a given EB into the archive.
    • Input: Pipeline run ID,  QAFLag=[Pass/Fail/Null], qa_report.txt (optional) flagtemplate.txt flagtargetstemplate.txt
    • Results: content of products directory for the pipeline run are ingested into the archive with the correct metadata corresponding to the relevant EB, QAFlag is registered in the archive database, QA report (text file) is added to the weblog and the weblog is archived.  If added, flagtemplate.txt and flagtargetstemplate.txt are also ingested.
• Cleanup script:
  • run after archive ingest. Checks data successfully ingested into the archive, deletes pipeline run from disk.
    • Input: Pipeline run ID
    • Result: if EB is was correctly ingested, proceed to delete the pipeline directory. If not, kick out with an error: "The data products are not in the archive, please wait and try again. If failures are repeated, please file an SSA ticket."

...

• Pipeline completion notification:
  • Notification should be sent to the SRDP operations manager upon completion of a pipeline job, so that they may assign the QA to a DA.
    • Input: none
    • Output: email notification to SRDP operations manager upon completion of a user-initiated imaging job, containing pipeline run ID.
• Pipeline rerun script:
  • DA needs to be able to rerun the pipeline in the correct environment with optional added flag target template file and possibly a modified PPR.
    • Input: MOUS UID, (optional) modified PPR, flagtargetstemplate.txt
    • Results: new pipeline run using modified PPR and/or flagtargetstemplate.txt (if supplied)
• Archive ingest script: 
  • needs to enable the DA to add a QA report file (text) to the weblog  and ingest the imaging products for a given MOUS into the archive.
    • Input: Pipeline run ID,  qa_report.txt (optional) flagtargetstemplate.txt
    • Results: content of products directory for the pipeline run are ingested into the archive with the correct metadata corresponding to the relevant MOUS, QAFlag is registered in the archive database as "Pass", QA report (text file) is added to the weblog and the weblog is archived. If added, flagtargetstemplate.txt is also ingested.
• Cleanup script:
  • run after archive ingest. Checks data successfully ingested into the archive, deletes pipeline run from disk.
    • Input: Pipeline run ID
    • Result: if MOUS was correctly ingested, proceed to delete the pipeline directory. If not, kick out with an error: "The data products are not in the archive, please wait and try again. If failures are repeated, please file an SSA ticket."

The work to put all of this in place is being tracked via: