Introduction

The AAT/PPI (Archive Access Tool/Pipeline Processing Interface) is designed to be a replacement for the NRAO's current archive tool (https://archive.nrao.edu) that gives astronomers the ability to utilize the NRAO's processing clusters to manipulate observation data using CASA. This preliminary release provides the capability to run EVLA and ALMA observations through their respective calibration pipelines; additional capabilities will be forthcoming.

Additional features include:

• Responsive interface that works on mobile devices.
• Natural language (text based) searches of the observation's title, abstract, source names, authors, and more; e.g., search on 'nova'.
• Search results displayed as an Observations View (similar to the legacy archive) or a Projects View, which aggregates observations by project and also displays the title, abstract and other assorted details missing from the Observations View.
• Display a scans report for a given execution block, including the scan intents and subscans.
• The 'my data' feature: log in, press a button, and see a list of your observations.
• If you link your ALMA and NRAO accounts, you can use either to access your observations.
• For ALMA and VLA you can download your SDM as a measurement set and filter by scan intent.
• For VLA you can download a calibrated measurement set restored from a previously run calibration pipeline.

New Features

• support for a wider variety of legacy VLBA IDIFITs and UVFITs data sets, due to improvements in the ingestion algorithms, ingestion of legacy VLBA data will continue
• support for VLASS quicklook image and VLBA IDIFITs and UVFITs reingestion, in order to improve metadata quality
• faster availability of restore functionality, as the search index updates are more complete upon calibration ingestion
• support for customization of cluster job limits (both time and memory) to allow greater flexibility of CIPL processing

Known Issues

• With the current upgraded functionality comes the need to reingest a large fraction of the ALMA observations (from Cycle 5 forward), and the ingestion of ALMA calibration information.  This process will take several weeks, and thus the new capabilities will not be readily apparent to start.
• Due to the separation of the UI search index from the true database of metadata, some observations do not show up right away.  A fix is currently being developed to remedy this problem.
  
  

Release Notes - 3.6.0 (Testing) - 2019-03-12

Epic

• [SSA-5111] - implement OUS Restores

Engineering Task

• [SSA-4786] - Configurable Workflow Memory Requests
• [SSA-4905] - Replace log with LOG
• [SSA-5030] - Refactoriing of VLBA ingestion
• [SSA-5045] - Get rid of "unavailable entity" in dumplogs
• [SSA-5049] - Handle ALMA Database Problems Gracefully
• [SSA-5160] - Write developer documentation
• [SSA-5246] - Block tests of RH classes not in use
• [SSA-5278] - RH Third-Party Code Cleanup
• [SSA-5286] - Remove unused link-creation code from workflow
• [SSA-5294] - De-complicate unwieldy RequestResource.represent()
• [SSA-5300] - speed up VLBA reingestion
• [SSA-5367] - setup pipeline areas for the new profiles
• [SSA-5376] - test cluster job submission
• [SSA-5396] - acceptance testing for OUS restores
• [SSA-5445] - add a Builder to SimpleProjectData
• [SSA-5459] - Cascade delete needed for subscans_data_descriptions
• [SSA-5460] - Name long-lived rabbitmq queues

Feature

• [SSA-4485] - VLASS workflow needs to update indexer in archive
• [SSA-4761] - Configurable Cluster Timeouts
• [SSA-4890] - per-workflow max time to live
• [SSA-5077] - Expand Session mapping algorithm to handle fail & semi-pass QA0 EBs.
• [SSA-5125] - modify ingestion to support OUS structure
• [SSA-5126] - modify Ingestion to support OUS calibrations
• [SSA-5128] - implement OUS restore workflow
• [SSA-5244] - Fix and restore direct-download link in result email
• [SSA-5304] - Update Execution Blocks Core Upon Calibration Ingestion
• [SSA-5346] - Add flux.csv files to the calibration tar files
• [SSA-5356] - Modification to accomodate 'Project View' date sorts that make sense
• [SSA-5358] - Modify Alma reingestion to include relevant calibrations.
• [SSA-5465] - track and display SB name associated with an MOUS

Bug

• [SSA-5216] - VLBA ingestion error mode
• [SSA-5241] - Unable to create file download links for local delivery
• [SSA-5247] - Link Generation Error for Multi-SDM tar local Use Case
• [SSA-5259] - Double MS Local Delivery Failure
• [SSA-5264] - Image Reingestion Process Bug
• [SSA-5270] - "Download Complete" email is sent thrice
• [SSA-5313] - Email delivery fails silently
• [SSA-5321] - Calibrations table entries for EVLA entries
• [SSA-5353] - Greater Granularity for Reindexing CAPO settings
• [SSA-5360] - Last Addition field on project not updating correctly on ingestion
• [SSA-5380] - Polarization session problem
• [SSA-5416] - calibration failure, DF error
• [SSA-5431] - fix 'verbose' flag
• [SSA-5438] - CMS request fails when datafetcher can't find CAL file
• [SSA-5468] - Apply Flags (and potentially other settings) mistranslated in RH
• [SSA-5470] - QA Fail on EBs from a STOP period
• [SSA-5481] - ngas_id error for Alma Calibrations