...
The NRAO Archive contains data from as far back as 1976, and currently contains just over 3 petabytes of information. Preserving this data and providing tools for the efficient utilization of the data products by the full multi-messenger community is a key responsibility of the DMS department. Starting in FY2023, SSR working collaboratively with DMS will launch a new project to expand the use of the NRAO Archive as a Research Instrument. This project will focus on user specified reprocessing, interoperability with other science archives, and increasing the supported type of projects. We expect this project to extend at least into 2024 and perhaps to 2025.
Scope Statement
The archive project will improve the user experience with the NRAO archive for the raw (visibility or otherwise) and image data from the VLA, VLBA, ALMA, and other telescopes, while not accruing significant new technical debt and retiring technical debt where necessary to improve the user experience. Interdependencies between the Workspaces system and the archive will be address within this project, but significant changes to the Workspaces software is out of scope, though limited enhancements are possible.
Key Goals
- Scriptability of the archive query interface via VO-compliant Table Access Protocol (TAP).
- Download improvements (inclusion of wget commands to retrieve data).
- Search improvements.
- Position search fixes.
- Search based on integration time.
- Scanlist improvements
- VLA-specific
- Metadata display improvements (i.e., scan list, sources, frequency setup(s), total integration times).
- VLBA-specific
- Completion of VLBA data collection: Fixing corrupted Mark IV files, and adding a few missing GMVA, HSA, and Global VLBI data.
- Downloading large numbers (>500) of correlation files for VLBA projects in bulk via the 'Select All' option.
- Fixing the missing or incomplete metadata of some VLBA+EVLA (i.e. +Y1, +Y27) projects, without which download is impossible.
- Enable the retirement of the legacy AAT/PPI
- Improvements to AUDI (ALMA User-Defined Imaging) and reimplementation into Workspaces
- Add VLA User Defined Imaging
- Increased robustness of ALMA MOUS ingestion of EBs and calibrations (ALMA Butler)
- IDM? (Identity Management; might be mostly done?)
- Support ngVLA archive workspaces
...