We propose a plan to allow users of the AAT/PPI to pick which release of CASA is used to process their request, and further, to allow the Data Analysts of the NAASC and DSOC to each specify the default CASA version for ALMA and VLA requests respectively.

Note: this specifically only covers user requests, it does not cover system requests, such as calibration or imaging pipelines. For system requests the version of CASA the AAT/PPI uses by default is a configuration property that can be changed by editing a properties file:

  • /home/casa/capo/$PROFILE.properties

Where $PROFILE is nmtest or nmprod for the DSOC (VLA) and vatest or vaprod for NAASC (ALMA). For the DSOC, the value of this property is currently the path to a symbolic link the DAs can change at will.

Also, this does not address the need to provide users with a restore process that uses the same version of CASA to do the restore that was used to do the calibration. What we will initially provide users is the choice of which CASA release to use, one of which is marked as the default. We will address the version dependency for restores once we start tracking which CASA release produced the calibration products for a given observation.

What we need to offer users this capability is:

  1. a directory of CASA releases that is common to both sites, and
  2. a mechanism for the DAs to specify which release is the default for their domain for the production AAT/PPI and which is the default for the test version of the AAT/PPI

For 1 we recommend /home/casa/packages/$OS/release, this directory is mirrored from CV to NM and thus is the same for both, and it is free from pre-releases, which we don't want to expose to end users.

For 2 we suggest the DAs create symbolic links under /home/casa/packages/$OS/release: nmprod, nmtest, vaprod and vatest, that point to the DSOC's production and test CASAes and the NAASC's production and test CASAes. 

The AAT/PPI will look in /home/casa/packages/$OS/release, and every directory it finds there will be added to the list of available CASA versions, then the AAT/PPI will mark which option is the default by following the appropriate symbolic link. In the absence of these links the default CASA version will be the most recent.

We should investigate how often this directory is mirrored, and if it is infrequently, we should ask the helpdesk to provide a way for the DAs to trigger the mirroring.

  • No labels

6 Comments

  1. Claire Chandler

    This looks fine. I have a question about the statement "We will address the version dependency for restores once we start tracking which CASA release produced the calibration products for a given observation." though. This is currently stored in the pipeline weblog, where else are you planning to track it? Is it to be a field in the archive metadata database?

    1. Stephan Witz

      Yes. To implement a system such that by default  a restore uses the same version of CASA that generated the calibration products we'd have to track that in the database and expose it to the front end. We're not there yet though.

  2. Mark Lacy

    Looks good to me, but as Claire pointed out we need to track the CASA version, which is not currently done in the ALMA archive (at least not as a separate metadata field), so you might have to harvest that from the weblogs when you ingest the ALMA metadata.

  3. Kana Sugimoto

    Jim drew my attention to this topic and asked me to comment about casa releases.

    A couple of points to consider:

    • not all casa packages have pipeline in it.
      So, you may want to filter CASA versions which have pipeline in your list if your goal is to run pipeline
    • not all casa packages with pipeline is verified to run VLA, ALMA, and VLASS data properly.
      For example, the current VLA pipeline release is CASA 5.1.2 while ALMA one is CASA 5.1.1. CASA 5.1.2 is released because 5.1.1 had issues for VLA (I forgot the details). So, it is not appropriate to use CASA 5.1.1 for VLA data. Furthermore, ALMA has clear acceptance process to accept pipeline for each observation cycle. No other CASA+pipeline version is verified for ALMA data.

    Depending on the policy of AAT/PPI and that of each observatory you are going to support, you may end up with thinking of having separate lists of casa + pipeline releases per pipeline, i.e., VLA, VLASS, and ALMA.

    FYI, the past CASA and pipeline versions for ALMA can be found at science portal:

    https://almascience.nrao.edu/processing/science-pipeline#version

    1. Stephan Witz

      The symlink part of the scheme should get us past the immediate problems, so that at least the recommended versions have pipelines and are functional per-telescope. I need to know who curates that directory of releases and get those links made.

      1. Kana Sugimoto

        You may want to contact the build and test team of casa (Darrell Schiebel and Ville Suoranta at NRAO) for packages put into release directory.

        I am not sure who exactly lists pipeline and casa versions for ALMA on the webpage I put the link above, but it may be worth contacting Theodoros Nakos (Theodoros.Nakos@alma.cl), the head of data management group in Joint ALMA Observatory.