A starter page for discussion of the SRDP Alma Optimized Imaging capability for the archive. It will start with some basics about the parameters involved, and how to use the starter command line tool to launch an imaging workflow () for testing purposes. This capability will operate only upon a single Virtual Spectral Window of a single restorable MOUS for the early stages of SRDP, and it is assumed to be a tool for expert use (necessitating the selection of parameters without guidance).
Also discussed somewhat are the imaging parameters, and what information we have about their limits for contemplation of the prototype UI for the user-facing version of this capability ().
Much like the almaRestore command line interface, this is simply a wrapper around a direct launch of the 'basic' workflow (i.e. a version of the OUS processing workflow which stops after CASA completes, instead of preparing the results for external delivery), and provides the location of the working directory once it is created. It has the same options for overriding the default CAPO value for the CASA to use (currently /home/casa/packages/RHEL6/prerelease/casa-prerelease-5.5.0-140) and the option to have an email sent when the workflow completes.
Note: In the versions of AAT/PPI after 3.6, it is no longer required to specify the nmtest/dsoc-test profiles with the -P option anymore. The limitation of the underlying messaging software has been accounted for in our software after that point.
Unlike the restore workflow, you need to provide the cube's imaging parameters in JSON a file, as well as the MOUS UID. Because of the flexibility provided by the pipeline's template, the file should contain a JSON Array, with each element containing the parameters for a single cube. All requested cubes are then written into the PPR processing procedure.
For instance:
[
{
"field": "NGC1333_IRAS_4A",
"spw": 25,
"restfreq": "140839502000.0Hz",
"start": "140827765374.83334Hz",
"width": "117366.25166666666Hz",
"nchan": 160
}
]
This is the content of the json file used for initial testing of the workflow. It requests/defines a single cube in the array. JSON was chosen to make the passage of a complex set of parameters easier on the AAT/PPI UI when we come to full integration of the workflow .
The frequency units could be given in various units (GHz/MHz/kHz as appropriate), and the start & width parameters can be given in terms of velocities if desired. Be sure to provide the units in the parameter, however, as these values are being placed straight into the PPR. The unit flexibility is possible because the workflow has been relieved of the necessity of looking up the spectral window value. Many of the convenience features (automatic look-up of the spectral window #, parameter limit checking, etc) are not part of the workflow itself, and will be integrated into the UI.
In most instances, imaging testing will be taking place in the 'test' environment, and thus the relevant command line will be:
/lustre/naasc/web/almapipe/workflows/naasc-test/almaImageCube /path/to/my/parameters.json my_mous_uid
will get the workflow proceeding, and report back the working directory location after it is created. The default version of CASA can be changed by editing the value of the parameter "edu.nrao.archive.workflow.config.CasaVersions.homeForAlmaRestore" in the naasc-test.properties file.
When it is necessary to perform testing on the 'development' environment, change the path used for the command to:
/lustre/naasc/web/almapipe/workflows/vatest/almaImageCube /path/to/my/parameters.json my_mous_uid
The fallback CASA version is then the one in the vatest.properties file (as of 5/24/19 set to the custom CASA installation I did per Morgan's request).
It is non-trivial to figure out appropriate parameters for a given test run, and the AAT/PPI's metadata database does not (yet) contain any of the information required to perform validation of parameter values. Due to these issues, it is recommended that you make use of the example uses cases which can be found on John's test sheet.
Also available is a workflow & initating command to re-run the CASA pipeline using the same parameters, but will use any PPR modifications or extra files added (much like recalDirectories for CIPL). Note: This will only work is the original spool directory (containing the metadata for the initial run) hasn't been automatically deleted. It takes the name of a directory in the image-qa or spool directories.
The Optimized Imaging system will work with the existing spectral information displayed by the ALMA AQ in the Frequency Support column. This data seems to be derived from the ASA_ENERGY table of the ALMA metadata database, and we will have to harvest information from that table in order to provide appropriate limitations upon user requests.
Each MOUS contains a set of of 'fields' or sources, and a set of 'Virtual Spectral Windows' (or 'Science Spectral Windows'). Each of the Spectral Windows has some frequency ranges associated with it (both Sky and Rest frequency minimum and maximum values are available), as well as an identifying number which the CASA pipeline requires.
There are six parameters available for customizing the cube:
The field parameter looks to be entirely independent from the spectral information. Each target appears to share the same set of spectral windows for a MOUS, which means that the user simply selects which of the targets are of interest. The rest of the parameters are strongly dependent upon one another.
Because many of the Virtual Spectral Windows are disjoint and the initial requirements from SRDP limit cube imaging to a single Virtual Spectral window, I suggest that we generate a type of 'display name' for the user to select the appropriate window (We could use the existing frequency range strings used by the AQ, for instance). This is consistent with ALMA's practices because the spectral window's identifying number (required by the pipeline) is typically hidden from the users. Once a particular window is selected, the details of that spectral window can provide boundaries for the UI to check or enforce on the frequency/velocity ranges for all the spectral parameters (i.e. the cube's range of [start, start+(width*nchan)] should fall between the [min, max] range of the spectral, window, for instance).
So far, the expected additional metadata required for imaging will contain the set of field names, and a set of spectral window information (number and sky frequency min/max to start) associated to an MOUS. Further required data can be refined as the prototype UI starts to take shape.