Versions Compared

Old Version 28

changes.mady.by.user John Tobin

Saved on

compared with

New Version Current

changes.mady.by.user John Tobin

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page captures the high level planning for each wave of SRAO development.  Objectives are captured as Epics in Jira and a brief description is below.

Table of Contents
maxLevel2


Wave 2

Epics

VLA Calibration Improvement: Improvements to the flagging heuristics for system and RFI Issues.
  • RFI heuristics for VLA PI pipeline will be modified - moving toward VLASS or VLASS-derived
  • bad deformatter flagging should gracefully recover from bad deformatter on refant
  • Heuristic changes for bad deformatter identification
  • proper multi-band support
    • allow different min/max solints per band
    • allow different bandpass calibrators per band
  • API plot should appear in weblog
  • gaintable flagging of final gaintables based on statistics (do within final cals?)
    • heuristics need to be defined
  • Fluxboot 2 display refinements (PIPE-319, PIPE-426)
  • Fluxboot fitorder determined when different gain calibrators used for different bands
  • Antenna pointing solutions in weblog
  • observing log in weblog
  • flagging summary per stage (SRDP-411)
    • AQUA Report needs to be finished (SRDP-194, CASR-263, CASR-358)
  • VLA Spectral Line Calibration: Modifications to the system to support Spectral Line science with the VLA.
    • per-baseband delay capability should be implemented
    • ability to use spw-mapping per baseband for final gain calibration in low S/N regime (mixed spw setups)
      • pipeline should use fluxboot2 results to determine if gain calibration on individual spws will have too low S/N
    • switched power weights (CASR-476)
    • mstransform hanning smooth pass through (CASR-357)
    • SPW intents and processing options need to be passed from RCT/OPT to pipeline
      • SDM option
      • PPR option
        • Intents : Spectral Line or Continuum
        • Processing Options: Hanning smooth, RFI Flagging, Statwt
    • VLA Imaging Pipeline: Initial version of VLA Imaging pipeline targeting Ku-band and above
  • establish imaging benchmark suite using existing SRDP-QA'd SBs
  • continuum-only
  • Use nsigma threshhold
    • test automasking or not
    • X-band and higher frequency
  • ALMA PI Driven Imaging: Additional options in the pipeline and interface
    • Pipeline
      • Selectable resolution (several options), higher resolution, max sensitivity, max sensitivity + larger scale
    • Interface
      • rest frequencies for commonly used transitions selectable (interface with splatalogue)
      • support the multiple resolution modes
      • multiple sources per request
      • multiple cube requests
      • Scrape hifa_imageprecheck weblog output to beam sizes for different resolution selections
      • Enable user to manually edit their continuum regions
        • Use cont.dat from archive as starting point (if available)
        • show plot of continuum regions
          • when possible use CARTA to extract spectra from archive datacubes within masks
          • plot spectra in window
        • regions defined as start and end in LSRK frequency
          • organize in rows
          • add/delete rows
          • expand manually with mouse
    • Infrastructure
      • Split workflow for Restore and Imaging
        • Enables Restore in same CASA version as reduction (e.g., 5.1.1) and newest for imaging (5.6.1)
  • Improved Download Support: Cutouts for images and data selection for MS downloads
    • Archive restore workflows should use XML template recipes for VLA and ALMA delivery
      • templates provided in CASA 5.6.1/2
      • enable intent selection, target selection, frequency averaging, and time averaging
    • batch request capability
      • upload formatted catalog, obtain cutouts against selected image collections (e.g., ALMA, VLASS)
  • Metadata Improvement: Improved meta-data for products in the archive.
    • Migrate to a refreshed archive-new interface
      • Ability add/hide presentation of some metadata associated with project/observation/image
    • Thumbnails for archived image products
    • QA State for different systems
      • ALMA
        • QA2 per MOUS
        • QA0 per EB
        • SRDP Imaging
      • VLA/SRDP QA
    • Weblog from Calibrations linked and viewable through archive
      • VLA
      • ALMA Calibration
      • ALMA Imaging (If separate from Calibration run)
      • ALMA SRDP Imaging
    • Proprietary state properly reported
      • ALMA must be correct
      • VLA
    • VLA Receiver Band Metadata correction
      • Fix older EVLA data (listed as subband...)
    • Angular resolution
      • Displayed per MOUS for ALMA
      • Tabulated approximately per VLA band/configuration
    • Conform to standard results table from System Requirements document
  • CARTA- Archive Integration: Integration of the CARTA viewer with the Archive.
    • Support starting CARTA backend (require to be logged-in) and user views through new web browser tab
    • User should be able to select an image product and open it in CARTA
      • User should see a progress bar so they know what is happening
        • important for large cubes that are not available within seconds
    • Ability to open second image from archive within CARTA and overlay
    • Regions support is required
      • region information should be available to archive interface for cutout or search functionality
    • Ability to upload their own image
    • Overlay catalogs  (available in archive or external catalog available in CARTA)
      • those in archive (e.g., NVSS, VLASS, etc)
      • external catalogs available in CARTA (e.g., 2MASS, IRAS, GAIA)
      • Upload user's catalog
  • Catalog Support: Support for searchable scientific catalogs in the Archive
    • Catalogs should be searchable from a subset of search filters
      • Return results able to be used as query for project/observation/image searches
    • Ability to hide columns
    • Interface for projects to upload catalogs for ingestion
  • Image Ingestion: If not already complete the ingestion and service of VLASS and ALMA images.
    • ALMA
      • ALMA cubes and continuum images associated with each MOUS need to be ingested
      • ALMA image products must be displayed to users and downloadable
      • Thumbnails should be present for ALMA image products
      • Viewable in a CARTA session
    • VLASS
      • VLASS QL images from the entire first epoch must be ingested
      • VLASS images must be displayed and downloadable
      • VLASS images require thumbnails and high-resolution png images
      • Viewable in CARTA session
    • VLA Images
      • When VLA imaging pipeline comes on line products must be ingested
      • Image products have same requirements as ALMA image products
  • Large Project Support: Support for the ingest and serving of products from Large projects as collections.
    • VLA Sky Survey
      • Epochs 1.1 and 1.2 images ingested and served
      • proper position search within the images and SDMs for VLASS
    • Projects should be viewable as a 'collection' to see only the data from that project.
    • project specific metadata should be searchable
    • Interface to support upload and ingestion of large project data
    • Observatory Calibrator Database: Development of a centralized calibrator database for the VLA.
  • Calibrator Flux densities
    • Per VLA Band
      • Central frequency
      • bandwidth
    • spectral index and curvature
    • Date
  • Calibrator models
    • Time-dependent models
    • Polarization models
    • models with spatial and spectral information
    • Antenna position database
  • Performance Improvement: Efforts to make the system more responsive and decrease product latency.
    • Optimized imaging
      • Multiple sources per MOUS grouped into single request
      • Multiple cubes specified per source(s) grouped into single request
      • Reduce restore load if user comes back to specifc MOUS several times for different jobs
      • Skip continuum finding step when possible
          • Data from current ALMA cycle should use the archived cont.dat when possible
          • Older data should always make use of current continuum finding routines
          • Must check that source(s)/spw selected for imaging has continuum defined in cont.dat
            • Data that are size mitigated might not have a region defined for all sources/spws
      • Restore
        • Reduce latency for popular data sets
        • Restore datasets with same CASA version as calibrated with
    • Column
    width67%

    Wave 2

    Wave 3

    Column
    width33%

    Relevant Documents

    Column
    width33%

    Relevant Documents

      • SRDP Wave 2 Conceptual Requirements