You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Introduction

The AAT/PPI (Archive Access Tool/Pipeline Processing Interface) is designed to be a replacement for the NRAO's current archive tool (https://archive.nrao.edu) that gives astronomers the ability to utilize the NRAO's processing clusters to manipulate observation data using CASA. This preliminary release provides the capability to run EVLA and ALMA observations through their respective calibration pipelines; additional capabilities will be forthcoming.

Additional features include:

  • Responsive interface that works on mobile devices.
  • Natural language (text based) searches of the observation's title, abstract, source names, authors, and more; e.g., search on 'nova'.
  • Search results displayed as an Observations View (similar to the legacy archive) or a Projects View, which aggregates observations by project and also displays the title, abstract and other assorted details missing from the Observations View.
  • Display a scans report for a given execution block, including the scan intents and subscans.
  • The 'my data' feature: log in, press a button, and see a list of your observations.
  • If you link your ALMA and NRAO accounts, you can use either to access your observations.
  • For ALMA and VLA you can download your SDM as a measurement set and filter by scan intent.
  • For ALMA and VLA you can download a calibrated measurement set, which re-runs the calibration pipeline on your SDM.
  • For VLA you can download a calibrated measurement set restored from a previously run calibration pipeline.

New Features

  • support for images, starting with the VLASS 1.1 'quicklook' images; these have been loaded into the system and are available for download
  • support for VLBA IDIFITs and UVFITs data sets, which we are currently loading into the system
  • multi-product requests: users can now retrieve multiple products in a single request, with the following limitations:
    • requested products must be from the same project and must be of the same type, e.g. six execution blocks from 17B-197, or 12 images from VLASS1.1
    • users are limited by disk space and number of products. The current settings for EBs are 10 EBs or 10TB; for images 20 images or 10TB, but we will adjust these to fit the usage and our storage space

Known Issues

  • The search interface for this release of the AAT/PPI is built on a completely new database of metadata, redesigned around the needs of VLASS and SRDP (ingestion of calibration products, observations with thousands of sub-scans and so on). We have ingested metadata from ALMA and the Jansky VLA into it, but it lacks metadata for the GBT and legacy VLA. Work is underway to address these issues, but until it is complete users are advised to use the legacy archive instead.  VLBA ingestions  of UVFITs and IDIFITs have started and will be complete in several months time.

Release Notes - 3.6.0 (Testing) - 2019-03-12

Epic

  • [SSA-4493] - Image Ingestion & Retrieval
  • [SSA-4793] - Implement easier retrieval of log information
  • [SSA-4973] - Initial Implementation of Alma MOUS Restores

Research Request

  • [SSA-4800] - Recreate Potential SHA Design
  • [SSA-4811] - First fringes: OUS-level restore

Engineering Task

  • [SSA-4822] - Update Request Handler to Use ID values in logging.
  • [SSA-4911] - Image Group Naming
  • [SSA-4919] - Set up Liquibase for the archive project
  • [SSA-4928] - Extract components for download selections
  • [SSA-4987] - An Alma-specific restore-setup task.
  • [SSA-4988] - Restore PPR creator for Alma
  • [SSA-4990] - Simplify the usage of multiple pipeline scripts to run CASA.
  • [SSA-5017] - BDF ngas_file_id Sanitization
  • [SSA-5043] - Check status of NGAS ARCHIVE operation
  • [SSA-5073] - Separated ingestion staging areas for EVLA SDMs
  • [SSA-5091] - Make all download types functional

Feature

  • [SSA-4494] - Baseline Image Retrieval Workflow
  • [SSA-4541] - Ingest Quicklook Images
  • [SSA-4788] - Vlass Messaging Post-Image ingestion
  • [SSA-4868] - Design/Implement Metadata Extraction for VLBA, IDIFITs
  • [SSA-4912] - Multi-Product Requests
  • [SSA-4913] - Allow specification of equinox in position searches
  • [SSA-4989] - CLI for Alma Restores
  • [SSA-5001] - Upgrade transfer of measurement sets for delivery.
  • [SSA-5022] - Design/Implement Metadata Extraction for VLBA, UVFITS
  • [SSA-5044] - Create Workflow for ingestion of VLBA data
  • [SSA-5051] - Reindex Images core after a successful ingestion
  • [SSA-5066] - VLBA File Retrieval
  • [SSA-5085] - hide the catalogs tab
  • [SSA-5086] - Don't show the catalog tab.

Bug

  • [SSA-4840] - Ingestion of Particularly Long Filenames
  • [SSA-4983] - Fix reindexing of calibration information upon sucessful ingestion.
  • [SSA-4984] - Telescope Screening preventing VLASS 'finalized' events.
  • [SSA-5012] - VLBA Authorization
  • [SSA-5028] - VLBA Subscans / Frequencies
  • [SSA-5029] - UVFITS data can have multiple UV_DATA tables
  • [SSA-5037] - RH permissions error, VLBA
  • [SSA-5039] - Archive-new tarballs do not have correct permissions
  • [SSA-5041] - CIPL Calibration status not updating in AAT
  • [SSA-5042] - Insure the stripping of extraneous spaces in PPRs
  • [SSA-5046] - CIPL MS cache naming incorrect and in wrong location
  • [SSA-5050] - Teach the Request Handler How to Cope with Image Requests
  • [SSA-5055] - Image table entries created with null image product id
  • [SSA-5056] - NGAS error handling cleanup
  • [SSA-5082] - polarizations for images not in polarization pull-down
  • [SSA-5088] - Incorred proposal lookup query
  • [SSA-5089] - Re-ingestion deletion

Sub-Task

  • [SSA-4926] - Implement new spool directory structure
  • [SSA-4927] - Update Server Tasks to handle new directory structure
  • [SSA-4970] - Make Request Handler handle multiple-item requests


  • No labels