Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Why Do We Care?

What is a Restore, and why is it 'A Thing(tm)'?

After an observation (or a set of observations, in the case of ALMA) are complete, the data is then calibrated using the CASA Pipeline.  Unfortunately, due to space constraints, the entire Calibrated Measurement Set (or CMS for short) is far to large to store in the archive due to the details of how CASA lays out and uses the data.  Instead, NRAO does the next best thing:  We archive what are called the 'Calibration Tables', these are far smaller sets of data which describe the corrections applied to the data by CASA to achieve the CMS. 

The application of an already calculated calibration solution is significantly faster (by a factor of ~10 or so) than performing a full re-calibration of the data.  In addition, the restored CMS is either identical (or nearly so) to the initial result.  This provides a measure of reproduce-ability which is important for the purposes of the Science Ready Data Products initiative.

The reason that a CMS (and thus a restore) are important is that it is the starting point for data analysis.  What this provides is freedom from some grunt-work which underlies the real scientific analysis.  Most critically, any imaging or statistical analysis of the data starts with a CMS.  Once the restore process is stable and validated, it can be used as the precursor to much more scientifically interesting analysis.  

Why were restores ALMA developed so much later than those for the EVLA?

Unlike the process for the EVLA, performing a restore for ALMA is slightly more complicated.  In many cases, ALMA processing involves more than one Execution Block (EB) at a time, and thus all the original EBs must be acquired along with the calibration products.  In addition, the PPR for handling ALMA data needs to meet the requirements of a much more stringent schema than is used for EVLA data processing. 

...