Overview
Ingestion of the IDIFTS files from eLWA observations is a new functionality for the AAT/PPI. The process is, in a nutshell:
...
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Using the System
Assumptions:
- eLWA team has access to the AAT/PPI command line installation areas (/users/vlapipe/workflows/)
- Vlapipe user (and group) have access (Read, Write, Execute) access.
- Everyone has (Read) access (to facilitate the AAT/PPI services access)
- The defined staging area (see below) is on the same filesystem as the data to be ingested
...
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# # ELWA Collection Settings # edu.nrao.archive.workflow.config.collection.ElwaSettings.ingestNgas = false edu.nrao.archive.workflow.config.collection.ElwaSettings.elwaSourcePath = /lustre/aoc/cluster/pipeline/jls_test/elwa edu.nrao.archive.workflow.config.collection.ElwaSettings.elwaServiceEndpoint = elwa_science_source?path= |
Under The Hood
It should be noted that the elwaIngest
command isn't doing any processing itself. It only prepares the basic metadata and initiates the workflow. It is possible to provide some limited feedback (a working directory name where some log files are kept, and a success/fail email) with a bit of additional work.
...