This document discusses the preliminary work done to potentially provide the images from the ALFALFA project via the AAT/WS.

The initial metadata ingestion was successful attaching the images to the Operations project and some other assumed metadata values.  

This document is the result of the following research request:

SSA-7716 - Getting issue details... STATUS

It is also in furtherance of the following project:

ALFALFA: SRAO and Collection Support

Work involved in the preliminary ingestions

A basic proof-of-concept was done with multiple simplifications.

Header Refinements

For consistency, we requested the information regarding the data units be changed from COMMENTs to the BTYPE/BUNIT header elements. 

Ingestion Refinements

Image ingestion worked upon the assumption of -SIN projection on the NAXIS types.   This was relaxed to a requirement of RA and DEC as the units provided (CTYPE keywords)

The BMAJ and BMIN keywords are missing from single dish data.  In place of the existing calculations, we replace the axis ratio with 1.0, and the resolution with the geometric mean of the spatial axis step values (CDELT keywords)

Added flexibility to whichever of RESTFREQ and RESTFRQ header elements exist.

The additional metadata file was easily worked up with available information for the limited elements we populate.

Required Steps to Facilitate True Support

The initial test ingestions were performed with several workarounds.  Proper support of for the images will require additional work.

Preliminary preparation

Creation of ALFALFA project

We'd require the Title, Abstract, and Author list for the ALFALFA project.  Additional elements (start and end MJD) would be good to have, if only as approximations.  The author list might need to be truncated if any members of the project don't have an NRAO or ALMA account. 

Additionally, we'll want to add a new external source (Arecibo Processing) to the allowed list. 

Answer: 

Creation of ALFALFA collection

After definition of any collection metadata, a method for finding or extracting that information will be required.  If (like ELWA) this collection.  The scope of this is dependent upon the data desired. 

Addition of Arecibo as a recognized telescope in the AAT/PPI/WS system

The RequestHandler and OODT workflow system need a new telescope value.  The majority of the code which interrogates the ENUM will need no changes, but the logic around the proprietary check might need some expansion. 

The Workspaces delivery system bases the layout partially on the telescope to handle specific nuances.  There will need to be definition and implementation of this for Arecibo images.

Further Ingestion Refinements

Manifest Generation and Bulk Ingestion Infrastructure

The hand-generated ingestion manifest and additional metadata file sufficed for the proof of concept.  In practice the additional metadata file will be consistent for these images, but a new manifest will be required for each set of images.  The file naming scheme makes it possible to use a template and simple SED substitutions to generate the manifests for a one-time ingestion project.   The existing scripts used for bulk ingestion of VLASS images could be reworked to provide the framework for this process. 

Alternatively, a limited-lifetime Workspaces workflow could be created, especially if it turns out that the ALFALFA collection has significant collection metadata required.

Projection Effect Management

it is likely that some of the calculations done as part of metadata extraction (mostly notably the IRCS BOX definition for the image) is going to be effected by the difference between -SIN and -CAR projections of the data.  These effects will need to be mitigated, but the WCS tools provided by Astropy should assist in that process.

Spectral Axis Handling

Further investigation of the extracted metadata has shown that the CRTYPE keyword for the frequency axis is being mishandled.  The addition of the HEL specifier is causing zeros to be filled in for those columns.

Any metadata error identified by stakeholders

A review of the metadata displayed has found a few issues which will be addressed by a combination of methods.

Header Additions

The workarounds noted above regarding the missing BMAJ and BMIN keywords will be removed.   Instead those keywords will be added such that the standard formulate can be applied.
These keywords have been added in the test data cubes located at /lustre/aoc/sciops/bkent/grids/ as of May 12, 2023

Location Errors

Brian noted that the displayed location for the demonstration image was incorrect (and obviously off compared to the filename).  This is due to using the reference point in the header (a corner of the image), rather than its center.   There also might be some projection effects (see above regarding changes to handle those transparently).

Work on the Path Forward:

This plan is the guildance for the following Epic Ticket:

SSA-7766 - Getting issue details... STATUS