Page History

This page is now obsolete. Please see the equivalent page in the POPS Confluence Space: QA Checklist - VLA Calibration

Checklist for SRDP calibrations of VLA data, based heavily on the VLASS one.

Step-by-step guide

Also relevant:

• requirements for the AQUAReport 

  Jira
  server DMS JIRA serverId eb2e750b-a83a-387e-8345-36eee8a98f01 key SRDP-194

• Data analysts scripts page https://open-confluence.nrao.edu/pages/viewpage.action?spaceKey=KBSAN1&title=Data+Analyst+Scripts

Checklist (in order of pipeline stage), to be done after the initial pipeline run that is performed for all EBs:

1. Overview of MS: check if EB as executed is SRDP compliant (many of these problems may have been caught at the SB submission stage, but some observers may choose to ignore recommendations anyway). If any of the conditions below are violated the pipeline should be run only once and no SRDP QA should be set (jump to step 15):
   1. The EB uses only C-band and above for science, with a single integration time per band, and no frequency averaging. If no clearly specline spws (narrow/high resolution/channel numbers), see https://open-confluence.nrao.edu/pages/viewpage.action?pageId=135725580 for details on bandwidths if not obviously a spectral line obs.
      
   2. intents for amplitude (flux) and gain calibration (and bandpass if needed, fallback is amplitude) all present.
      
   3. the amplitude (flux) calibrator is appropriate for the frequency and configuration (i.e. a good model exists). Particularly, SBs using 3C48 or 3C138 may need to be excluded given its recent bad behavior. See https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/fdscale
      
   4. all science spw are calibrated.
      
   5. cycle times are appropriate for the frequencies being observed i.e. <8-30min <12GHz, <2-10min >12GHz, depending on configuration (see https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/calibration#CalibrationCycle for details), and bracket the science observations.
      
   6. appropriate setup scans were performed: see https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/set-up
      
   7. antenna reference pointing calibrations were performed for high frequency (>15GHz) observations.
      
   8. check that the observation has continuum windows (at least 64 MHz bandwidth).
   9. check for missing scans due to system issues (or data fetcher errors)
      
2. hifvhifi_flagdata - online flags. Check the flagging plot for systematic issues and for the fraction of zeros in the data. Notify scientist if >20% >30% flagged.
   
3. hifv_priorcals/hifv_syspower - 8-BIT DATA ONLY!! check for gain compression due to severe RFI. We can running the compression fix for Ku and lower frequency bands but only in 8-bit and no multi-band data. (AmyK updated 2023-09-13)
   
4. hifv_testBDdcals testBPdcals - check for Data Transmission System (DTS) problems.
5. hifv_flagbaddef - check that any DTS issues have been flagged.
6. hifv_semiFinalBPdcals - check for any remaining DTS issues.
7. hifv_solint - check solints for calibrators are reasonable (< a few mins for the long interval, > 0.05s for the short interval. Note if the short interval is > the integration time) - if in doubt, e.g. long solint > scan duration, check with a scientist.
8. hifv_fluxboot:
   1. check the input tables look clean 
   2. check that the model is reasonable and covers all science spws.
   3. make a mental note of the phase calibrator flux density and spectral index.
      
9. hifv_finalcals:
   
   1. check for delay outliers: whenever there is a very high delay (e.g., >  ~100 nanoseconds, such as a single spw), the offending spw (for that antenna) should be flagged
   2. check for unusual bandpass shapes
   3. check for outlying bandpass phases
   4. phase (short) gain solution: check for large phase scatter (AEK notes: band/config/weather dependent; learn with experience)
   5. final amp/time: check for outliers >+/-50%
   6. final amp/freq: check that outliers fall into known bad spw (e.g. RFI, or a priori info from Operator log)
   7. Final phase - check that mean phase is close to zero on the calibrators and not too scattered.
10. hifv_applycals: check flagging table and note final % flagged. If over 50%, check with a scientist  (AEK notes: band/config/weather dependent; learn with experience)
    
11. hifv_plotsummary:
    1. phase vs time plots: check all are centered on zero
    2. amp vs uvwave:
       1. check calibrators are point-like (constant flux). If not (and they have no model):
          1. If the calibrator is <10% resolved (i.e. the longest baselines contain >90% of the total flux) ignore and continue.
          2. If the calibrator is >10% resolved, but >~30% of the baselines (at the short end) contain all the flux, set a uvrange in flux.csv and rerun.
          3. If < 30% of the baselines have all the flux, and/or the object is clearly resolved (double etc), then this is an SRDP FAIL. 
       2. check there is no evidence of decorrelation (strong "waterfall")    (AEK notes: learn with experience)
       3. check RFI is well-flagged  (AEK notes: can be a bit subjective; learn with experience) 
    3. amp vs frequency:
       
       1. check for residual RFI on calibrators
       2. check for residual RFI on targets, flag if obvious.
          
12. Update google spreadsheet with run details
    
13. Extra flagging if needed
    1. pre-workspaces: Rerun with additional flagging if needed using the script /users/vlapipe/workflows/dsoc-test/bin/recalDirectories. Can we say that only whole spw if >50% of channels affected, and/or whole antennas will be flagged? If so, DAs may not need to run plotms. Maybe look at wt*amplitude given we are running statwt? Baseline-based flagging too intensive.
       
    2. workspaces:  use flagging template, create/edit as appropriate (will be inherited by new execution version). Submit new execution.
14. Once satisfied that a version can be QA passed:
    1. pre-workspaces:
       
       1. write a short overall evaluation suitable for archiving and transmission to the PI using the attached template qa_notes.html, and add this to the weblog html directory (as qa_notes.html). Make sure this file is either created by owner vlapipe, or it is chmod'ed to 777 to prevent ingestion problems. In the comments section:
          
       2. If the EB fails SRDP QA, briefly explain why.
          
       3. Note any significant problems with data acquisition and telescope hardware.
          
       4. Note any manually added flags (at the level of an antenna/spw).
          
       5. Add any other comments.
          
    2. workspaces:
       1. follow the above "pre-workspaces" steps, but within the workspaces interface
15. Execute qaPass:
    1. pre-workspaces:
       1. Trigger archive ingest with the qaPass script after activating the dsoc-prod environment.
          
    2. workspaces:
       1. Send to AoD ; AoD will complete or return to DA
          
16. If pre-workspaces: Send user an email with the summary from step (14) and instructions on how to obtain their calibrated data.
    

Related articles

...