...
The AAT/PPI has 3 parallel 'environments'. Each of these environments consists of a set of software which is deployed to dedicated servers in both NM and CV, and a set of configuration parameters (also called CAPO Profiles) for each deployment location for that environment. The 3 environments (and their associated profiles) are:
- Production (nmprod, vaproddsoc-prod, naasc-prod)
- The current release of the AAT/PPI
- No redeployments without an announcement to the archive_issues list
- Web interface: archive-new.nrao.edu
- Test (dsoc-test, naasc-test)
- Used for integration testing & validation of changes (features or bug fixes)
- Also the home for pre-release candidates for release to production
- Typically stable on day+ timescales
- Major redeployments are announced on the archive_test list
- Web interface: archive-test.nrao.edu
- Development (nmtest, vatestdsoc-dev, naasc-dev)
- Developers' basic verification testing ground
- Often subject to sudden redeployments of subsystems
- Can have components from multiple branches of development deployed at one time
- Web interface: webtest.aoc.nrao.edu/portal/#/
- (Yes, I know, that name is unfortunate)
...
As a convenience feature, these two accounts have had the activate_profile & deactivate_profile commands added to their Bash configuration. When given the name of a CAPO profile (nmproddsoc-prod, dsoc-test, etc) activate_profile sets up a few environment variables and sets up your shell so that the corresponding set of utilities (and the correct supporting libraries) in your $PATH, so that they're easy to use.
...
When running just a few commands, it might be easier to simply directly call the appropriate utility without resorting to the relevant 'pipe' account. In that case, pay particular attention to the directory from which you run the command (either by changing directory there or typing out the path). The differences between /users/vlapipe/workflows/nmproddsoc-prod/bin/myCommand and /users/vlapipe/workflows/nmtestdsoc-test/bin/myCommand can (potentially) be not just the details of the software itself, but also any differences in the profile's parameters.
...
In addition, some of the services which facilitate processing are deployed only to NM, so that information requests (like those made for image ingestion) may be slower.
...
{needs more details}Self-contained sets of python libraries & associated commands. This is how we deploy the utilities for the AAT/PPI.
CAPO Files
The configuration details for the AAT/PPI are kept in text files in /home/casa/capo. This directory is replicated to CV via rsync on a cron job, so it is not immediately updated after an edit. There are a set of properties for each site (NM vs CV) and environment (Development, Test, Production). Included in that directory are a set of files for the VLASS project, containing configuration specific to their processing needs.
These properties define information like server names, where to process data, and what version of CASA to use as a default. They are editable by the vlapipe account, so those values which are of interest can be readily modified by the Data Analysts. It is possible (even likely) that some of those variable names may change as the system evolves. It is important to remember that the copy of the files in NM take precedence and will periodically overwrite their counterparts in CV.
The Renaming Of The Profiles
If you don't have access to the vlapipe account, contact one of your fellow DAs or the SSA team.
What is that source ~pipeaccount/workflows/myprofile/bin/activate command doing?
In order to function properly, the AAT/PPI utilities (both internal and external) require a stable, defined group of python libraries. In order to provide that, these tools are deployed within Python Virtual Environments. These environments evolve over time both as new tools are added, but also as the interactions (either internally or with external libraries) change. There will be times when two separate environments will be entirely incompatible, which is why we try to provide simplifications for accessing a coherent set of utilities. {needs more details}There are already symlinks, no official timeline, but be prepared for the replacement of nmprod with dsoc-prod, and nmtest with dsoc-dev the like.