You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 36 Next »

Summary of development requests, decisions and actions from SRDP NM Operations meetings July 2019 onwards:

Development requests:

  1. The archive search results should indicate the QA standard (SRDP or non-SRDP) as well as whether it is a pass/fail
  2. The qa_notes.html file from the weblog should be directly linked in the search results so users can click on it and look at the QA notes before deciding whether to download a dataset or not
  3. For QA fails that are so bad that no usable calibration products are produced, the qa_fail script (or workspace manager, once that is done) should still ingest the weblog into the archive and make the qa_notes.html file available so users can see why it failed.
  4. It would be good if the users could obtain the weblogs from the initial pipeline run, so they can see more easily what was flagged.
  5. Ability to trigger restores on more than one dataset at a time.

    Items 1-3 sent to John T on 2nd August 2019

Decisions:

  • 8/29/19: expand SRDP pilot to include X-band (in addition to Ku-band and higher).
  • 10/23/19: use the NM User Support Calendar to record vacations etc.
  • 11/20/19: flag outlying A-config antennas in wrong configuration for SRDP purposes once we are in the "official" D-config (otherwise risk trying to calibrate resolved calibrators).
  • 12/11/19: AoD review for non-problematic datasets is performed only at the final stage, before archiving. DAs should flag and rerun as they see fit beforehand (but should also feel free to set a dataset to "Ready for AOD" if they have any questions prior to that).
  • 2/26/20: add informational note to qa_notes.html if flagging percentage >50%. Decision of SRDP Pass/fail for heavily flagged datasets is at the discretion of the AoD on a case-by-case basis.
  • 5/27/20: On the deployment of 3.8.2 with the MS flagging fix, Drew will update the email sent to non-SRDP PIs to include a suggestion that they can try archive-new for their restores/downloads. The ability of non-staff users to download directly to AOC lustre will be discussed at the next SRDP-DMS meeting. We will attapet to ingest calibration files for datasets that can only be run manually due to memory issues (only ~10/yr).
  • 8/26/20 For runs that fail the pipeline (both regular CIPL and SRDP), if the reason for a pipeline fail is clear, the email sent to the PI will state this. This is especially important in cases relating to setup (poor calibrator choice etc). For SRDP, if the reason for the pipeline failure is not clear, a short investigation will be performed and the results communicated in the email to the PI.  
  • 8/26/20 Plan for VLA imaging testing and implementation - at the end of November after VLASS 2.1 ends, try to send feedback to get any changes to the pipeline into a spring pipeline patch. Aim to start production when array moves to C-config (June 2021)
  • 1/27/21 3C138 variability:
    • datasets on-hold to be delivered as-is with 3C48-style warning: "The flux density scale calibrator 3C 138 has been undergoing a flare since at least mid-December 2020.  At K and Ka-bands the magnitude of the flare is of order 40-50% compared to the Perley-Butler 2017 flux scale.  The effect is smaller at lower frequencies (10-20% at C and X-band),  and is larger at higher frequencies (more than a factor of two at Q-band).  If you care about the flux density scale of your observations at that level, you may want to re-calibrate your data once new time-variable calibrator models have been put into CASA." 
    • For both 3C48 and 3C138, once the calibrator group have a page indicating how these calibrators have varied since Perley-Butler17 we will include data with these flux calibrators in SRDP, either linking to the flux variation page from the email, or applying the flux corrections via a script and probably remove the statement about time-variable models in CASA as these are still a ways off.

Ramp-up schedule FY21

  • 2020 Nov 20 (start of A-config): add C-band calibration to SRDP.
  • 2021 Jan 18: add Multiband calibration to SRDP (down to C-band, including S and L later (for Eps unaffected by gain compression); not including P-band).
  • 2021 Mar 5 (start of D-config): begin testing imaging pipeline and workflows (*dependency on SSA to provide scripts to process and ingest).
  • 2021 Jun 10 (start of C-config): add S-band calibration to SRDP.
  • 2021 Aug 15: begin running imaging as pilot on a small fraction of SRDP datasets.
  • (2022 Mar 4 (start of A-config): add L-band calibration to SRDP.)

SSA tickets important for NM Ops:

SSA-6129 - Getting issue details... STATUS SDM files no longer have .loading appended - Fix version: unassigned; write requirement, design solution, determine priority

SSA-6495 - Getting issue details... STATUS


NMS software page

https://staff.nrao.edu/wiki/bin/view/DMS/NMSSoftwareReleases









  • No labels