You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 78 Next »

Checklist for SRDP calibrations of VLA data, based heavily on the VLASS one.

Also relevant:

Checklist (in order of pipeline stage), to be done after the initial pipeline run that is performed for all EBs:

  1. Overview of MS: check if EB as executed is SRDP compliant (many of these problems may have been caught at the SB submission stage, but some observers may choose to ignore recommendations anyway). If any of the conditions below are violated the pipeline should be run only once and no SRDP QA should be set (jump to step 15):
    1. The EB uses only X-band or above for science, and has only one science band, with a single integration time, and no frequency averaging.
    2. intents for amplitude (flux) and gain calibration (and bandpass if needed, fallback is amplitude) all present.
    3. the amplitude (flux) calibrator is appropriate for the frequency and configuration (i.e. a good model exists). Particularly, SBs using 3C48 may need to be excluded given its recent bad behaviour. See  https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/fdscale
    4. all science spw are calibrated.
    5. gain calibrations are at appropriate intervals for the frequency being observed i.e. <10-60min <12GHz, <2-20min >12GHz, depending on configuration (see https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/calibration#CalibrationCycle for details), and bracket the science observations.
    6. appropriate setup scans were performed: see https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/set-up
    7. antenna reference pointing calibrations were performed for high frequency (>15GHz) observations.
    8. check that the observation has continuum windows (at least 64 MHz bandwidth).
    9. check for missing scans due to system issues (or data fetcher errors)
  2. hifv_flagdata - online flags. Check the flagging plot for systematic issues and for the fraction of zeros in the data. Notify scientist if >30% flagged.
  3. hifv_priorcals - check for gain compression due to severe RFI.
  4. hifv_testBPdcals - check for Data Transmission System (DTS) problems.
  5. hifv_flagbaddef - check that any DTS issues have been flagged.
  6. hifv_semiFinalBPdcals - check for any remaining DTS issues.
  7. hifv_solint - check solints for calibrators are reasonable (< a few mins for the long interval, > 0.05s for the short interval. Note if the short interval is > the integration time) - if in doubt, e.g. long solint > scan duration, check with a scientist.
  8. hifv_fluxboot:
    1. check the input tables look clean 
    2. check that  the model is reasonable and covers all science spws.
    3. make a note of the phase calibrator flux densities.
  9. hifv_finalcals:
    1. check for delay outliers
    2. check for unusual bandpass shapes
    3. check for outlying bandpass phases
    4. phase (short) gain solution: check for large phase scatter
    5. final amp/time: check for outliers >+/-50%
    6. final amp/freq: check that outliers fall into known bad spw
    7. Final phase - check that mean phase is close to zero on the calibrators and not too scattered.
  10. hifv_applycals: check end of CASA log and note final % flagged. If over 50%, check with a scientist
  11. hifv_plotsummary:
    1. phase vs time plots: check all are centered on zero
    2. amp vs uvwave:
      1. check calibrators are point-like (constant flux). If not (and they have no model):
        1. If the calibrator is <10% resolved (i.e. the longest baselines contain >90% of the total flux) ignore and continue.
        2. If the calibrator is >10% resolved, but >~30% of the baselines (at the short end) contain all the flux, set a uvrange in flux.csv and rerun.
        3. If < 30% of the baselines have all the flux, and/or the object is clearly resolved (double etc), then this is an SRDP FAIL. 
      2. check there is no evidence of decorrelation (strong "waterfall")
      3. check RFI is fully flagged
    3. amp vs frequency:
      1. check for residual RFI on calibrators
      2. check for residual RFI on targets, flag if obvious.
  12. Update google spreadsheet with run details
  13. Rerun with additional flagging if needed using the script /users/vlapipe/workflows/dsoc-test/bin/recalDirectories. Can we say that only whole spw if >50% of channels affected, and/or whole antennas will be flagged? If so, DAs may not need to run plotms. Maybe look at wt*amplitude given we are running statwt? Baseline-based flagging too intensive.
  14. Once satisfied, write a short overall evaluation suitable for archiving and transmission to the PI using the attached template qa_notes.html, and add this to the weblog html directory (as qa_notes.html). Make sure this file is either created by owner vlapipe, or it is chmod'ed to 777 to prevent ingestion problems. In the comments section:
    • If the EB fails SRDP QA, briefly explain why.
    • Note any significant problems with data acquisition and telescope hardware.
    • Note any manually added flags (at the level of an antenna/spw).
    • Add any other comments.
  15. Trigger archive ingest with  the script /users/vlapipe/workflows/dsoc-test/bin/qaPass
  16. Run cleanup script (to be supplied by SSA) to remove pipeline run after data ingested successfully into the archive.
  17. Send user an email via the helpdesk with the summary from step (14) and instructions on how to obtain their calibrated data.


  • No labels