You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 22 Next »


This guide is intended for DAs and AODs using the workspaces capabilities for VLA calibration and imaging.

Step-by-step guide for WS 2.5 (see also slides from Mark's presentation linked here)

  1. Go to https://ws.nrao.edu/workspaces/active-requests (do NOT log in, this is not required and will break your session) and enter the EB to be processed in the "SDM ID" box at the top, hit the "Calibrate" button. Make sure the Auto-calibration button at the top right is set to run (this has nothing yet to do with the automatic start of jobs, which is controlled by the CAPO profile*).
  2. The job should appear in the job list with a status of "executing". Select the job by clicking the request on the left-hand column, this should open a request page.
  3. You should be able to see the working directory in lustre for the request in the Version Parameters section of the request page. You can click on the text to copy it.
  4. At any point prior to QA you should set the SRDP status to T/F with the checkbox and assign a DA (yourself) and an AOD to the job. 
  5. When complete, the status will change to "Awaiting QA". Click the link to the request page in the "Request" column, or the link the email to Workapsaces-Analysts, which will take you to the request page.
  6. Follow the link to the weblog and perform QA (if you need to you can go to the working directory on spool using the directory name from step 4).
  7. On the bottom right is a button to edit the DA notes, these notes persist between versions and are not seen by the user.
  8. If you need to add flags, click "Create new version" on the bottom right of the page, and upload a flagtemplate.txt file. You can either submit one with the commands in, or submit a blank file (touch <FSID>.flagtemplate.txt) and edit in the editor.
  9. If you need to edit the PPR, this is also accessible to edit at this stage.
  10. Select the latest version on the drop-down menu under "Versions", then click "Submit new version". The old version remains and you can toggle between them using the dropdown menu. 
  11. When you are satisfied, edit the QA notes and inform the AOD e.g. via mattermost  that a job is ready for QA. 
  12. AOD role: check the calibration, the QA notes and that the SRDP status is set correctly. If satisfied, hit QAPass. QAPass will pass the current selected version and fail all the others. If not, you can return to the DA after adding comments in the DA notes. The execution state should change to "Ingesting", then "Complete".
  13. AOD role: if the data are SRDP (always true in pilot) wait for the calibration ingest to complete, then hit the "Create Image request" button. Go back to the active-requests page, select std_cms_imaging from the top-left menu. Set the DA and AOD to the same people as the calibration request, and status to SRDP.
  14. AOD role: Submit the image request  (bottom right button on the image request page).
  15. Once the imaging is complete, the state will change to "Awaiting QA". The DA should watch the image queue for their datasets to check for new job completions.
  16. Weblog review can be carried out in the same way as for calibration. Note the "Launch CARTA session" capability that can be used to inspect the image data. Flagging can be added with a <FSID>.flagtargetstemplate.txt (after the Fall pipeline release) or a <FSID>.flagtemplate.txt file (before that).
  17. Once satisfied, edit the QA notes and let the AOD know that the image is ready for review via Mattermost.
  18. AOD role:  If satisfied, check the QA notes and click "QA Pass" and an email will be sent to Workspaces-Analysts to say that the data were ingested.

*need to set edu.nrao.workspaces.StandardCalibrationSettings.runState = STOP in /home/casa/capo/dsoc-prod.properties for the pilot; setting to PAUSE will create jobs automatically but not submit, GO will create and submit jobs)



Command line tools 

There are several command line utilities that SSA have written to help with e.g. diagnosing issues. They are described on https://ssa.gitlab-pages.nrao.edu/workspaces/source/tools.html

They are located in /lustre/aoc/cluster/pipeline/dsoc-prod/workspaces/sbin and need to be run as vlapipe on mcilroy in the dsoc-prod environment (activate_profile dsoc-prod) or on hamilton on dsoc-test (activate_profile dsoc-test).

Checking on jobs:

./wf_inspector <req number> -v <version number> will log into the running job.

Metrics: 

Write out the operations report as a csv:

./ws_report -o <date1> <date2> --report  

Number of executions of a capability between date1 and date2:

./ws_report -c <capability> <date1> <date2>

Dates in format YYYY-MM-DD. Note that the report is not actually filtered on the dates yet. 

Changing DAs/AODs in the menus:

./mod_analyst e.g. to add an AOD:

./mod_analyst -name "Nathan Bockisch" -aod -email "nbockisc@nrao.edu"

or to remove a DA:

./mod_analyst -name "Nathan Bockisch" -rm

Email templates:

The operations manager should check the email templates that are sent to the users after every deployment and edit if needed. For example, the filesizes are included as a keyword, but not actually looked up, so those should be removed.

Pipeline recipes

Note that, currently, the pipeline recipes used by workspaces are not automatically updated when the pipeline is updated, they need to be edited separately (easy to forget this step)!

  • No labels