...
- Ingest the SDM for the EB as normal
- Fill in the RealFast collection metadata table
- Ingest the associated PNG candidate file(s)
- Link the new EB to its donor EB
...
The two path arguments are provided for flexibility, but it is assumed that the default values in the CAPO profiles (dsoc-test, dsoc-prod/nmprod) are the typical location. The If those paths are correct, the command can then be invoked with (for example): :
Testing:
As vlapipe:
activate_profile dsoc-test
realfastIngest realfast_18B-320.sb38241161.eb38244520.59002.47251115741_1591099113820
Not as vlapipe:
/users/vlapipe/workflows/dsoc-test/bin/realfastIngest realfast_18B-320.sb38241161.eb38244520.59002.47251115741_1591099113820
Production:
As vlapipe:
activate_profile dsoc-prod
realfastIngest realfast_18B-320.sb38241161.eb38244520.59002.47251115741_1591099113820
Not as vlapipe:
/users/vlapipe/workflows/dsoc-prod/bin/realfastIngest realfast_18B-320.sb38241161.eb38244520.59002.47251115741_1591099113820
This will initiate the process, and the SDM will shortly be available in the UI (a matter of ~10 minutes).
...
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
edu.nrao.archive.workflow.config.collection.RealfastSettings.serviceUrl = https://webtest.aoc.nrao.edu/archiveServices/ # edu.nrao.archive.workflow.config.collection.RealfastSettings.pngNameArgument = realfast_ancillaries?path= edu.nrao.archive.workflow.config.collection.RealfastSettings.donorLocatorArgument = realfast_associate?path= edu.nrao.archive.workflow.config.collection.RealfastSettings.collectionMetadataArgument = realfast_collection?path= # edu.nrao.archive.workflow.config.collection.RealfastSettings.ingestNGAS = false edu.nrao.archive.workflow.config.collection.RealfastSettings.realfastStagePath = /lustre/aoc/cluster/pipeline/nmtest/stage_products # edu.nrao.archive.workflow.config.collection.RealfastSettings.realfastSdmPath = /lustre/aoc/sciops/pdemores/realfast_sdms edu.nrao.archive.workflow.config.collection.RealfastSettings.realfastPngPath = /lustre/aoc/sciops/pdemores/realfast_sdms |
Under The Hood
It should be noted that the realfastIngest
command isn't doing any processing itself. It only prepares the metadata and initiates the workflow. It is possible to provide some limited feedback (a working directory name where some log files are kept, and a success/fail email) with a bit of additional work.
What the workflow does in more detail:
...