Decision log

1. "compression fix" should never be applied on 3-bit data regardless of Pdiff behavior
2. If compression evident (< ~ 20%) include qa note about possible error in flux scaling
3. If scatter < 20% in (amplitude) calibration tables, leave be.
4. If scatter > 20%, compare to Pdiff/compression levels → if tracking compression, leave be. If not tracking compression, flagging likely needed.
5. If scatter is outside of 0.5–1.5 range (i.e. > 50%), flag affected data.

• MCAF
  Rich M was informed of the MCAF bug and it has been fixed the week of 2022-10-03.
  
  CASA
  Steve and George M discussed and would like to change the way gencal caltype='rq' behaves. CASA ticket has been created:
  https://open-jira.nrao.edu/browse/CAS-13941
  
  22A-126
  We decided to qaFail all EBs. The PI has been informed of the issue through a helpdesk ticket:
  https://help.nrao.edu/agent/go/ticket/35000
  
  We would like to note the issue in the archive for future users. Comment added here:
  https://open-confluence.nrao.edu/display/SRDP/Archive+Data+Annotation+Tools
  
  Emmanuel provided the scripted pipeline hack and Brian K provided the integrated hack. We gave the scripted hack but not the integrated hack to the 22A-126 PI, because they are mostly using it to compliment manual data reduction and did not press the issue. Official policy may need more discussion. (Personal note from Edward: if I have an easy solution and it could help a user, it feels weird to sit on the info and not share it with them.)
  
  20A-160
  This project has at least one EB with a completely empty SysPower.bin. We gave the integrated hack to Michael Rugel since he is a Jansky Fellow, and had already processed MANY large data sets with the integrated pipeline.
  
  Scripted Pipeline Hack
  
  In EVLA_pipe_priorcals.py:
  
  if startdate >= 55616.6:
  default(gencal)
  vis=ms_active
  caltable='requantizergains.g'
  caltype='rq'
  spw=''
  antenna=''
  pol=''
  parameter=[]
  # gencal()
  
  if os.path.exists('requantizergains.g'):
  priorcals=['gain_curves.g','opacities.g','requantizergains.g']
  else:
  priorcals=['gain_curves.g','opacities.g']
  
  Integrated Pipeline Hack
  
  In the file:
  ./pipeline/pipeline/hifv/tasks/priorcals/priorcals.py
  
  Change the line:
  
  rq_result = self._do_rqcal()
  to
  rq_result = None
  
  In the file:
  ./pipeline/pipeline/hifv/templates/priorcals.mako
  
  Add a conditional "if" statement around the rq table loop:
  
  %if single_result.rq_result:
  
      %for single_result in result:
       <p><b>${os.path.basename(single_result.rq_result.inputs['caltable'])}
       </b></p>
      %endfor
  
  %endif
  
  [WARNING - indentation matters and may not be correct in this format]

Pipeline results with missing fluxboot solutions are qa pass with noted below added.

QA Note:

"In the fluxboot stage, gain solutions are not present for some basebands/spws of 3C138. Additionally, because its CASA model is not used, the resulting amp vs uvwave plots for 3C138 are inaccurate. As 3C138 is used only for polarization angle calibration, the lack of fluxboot gain solutions has no effect on the overall calibration for Stokes I. However, before performing any polarization calibration we recommend setting the 3C138 model image with setjy, then running gaincal on 3C138 with calmode='p'."

QA Note for instances where basebands/spw are not flagged:

"Because 3C138 has only scan intents for polarization calibration, the CASA model is not used: the resulting amp vs uvwave plot for 3C138 is inaccurate. As 3C138 is used only for polarization angle calibration, the lack of fluxboot gain solutions has no effect on the overall calibration for Stokes I. However, before performing any polarization calibration we recommend setting the 3C138 model image with setjy, then running gaincal on 3C138 with calmode='p'."

We will look into improving the model.

Mark says to continue including affected data in SRDP and do not flag the data. In comparison to the recent 3C147 model issue, we do not see ramps in amp vs freq solutions. Include a QA note:

"The 3C286 Ka-band model used for pipeline processing and packaged with CASA does not represent the source in A-configuration very well, causing amplitude errors. A new, more accurate model will be released with CASA in the future."

"The current calibrator models for resolved flux density calibrators in A-configuration in CASA are created

with narrow band data. At the high frequencies of a broad band observation, this can mean that the calibrator

model does not represent the structure of the calibrator very well, leading to amplitude errors that reduce

the dynamic range of the calibrated data. This will be remedied in a future version of CASA and the pipeline."

srdp > flag affected band
non-srdp > leave in

Include qa note in both cases. Example:
** S band long sol int = 950s due to a pipeline bug. Manual calibration is advised. **

srdp > review by AoD, likely leave in and pass
non-srdp > leave in

In both cases include a qa note. Example:
The current calibrator models for resolved flux density calibrators in A-configuration in CASA are created
with narrow band data. At the high frequencies of a broad band observation, this can mean that the calibrator
model does not represent the structure of the calibrator very well, leading to amplitude errors that reduce
the dynamic range of the calibrated data. This will be remedied in a future version of CASA and the pipeline.
In the meantime, if high image fidelity is required we have found that recalibration using a model for a higher
frequency band scaled to the band of interest can work well.

srdp > review by AoD, likely SRDP Fail
non-srdp > leave in

In both cases, include qa note. Example:
SRDP Failure due to strong gain compression in A2C2 in several antennas.