This page captures the high level planning for each wave of SRAO development.  Objectives are captured as Epics in Jira and a brief description is below.

Wave 2

Epics

  • VLA Calibration Improvement: Improvements to the flagging heuristics for system and RFI Issues.
    • RFI heuristics for VLA PI pipeline will be modified - moving toward VLASS or VLASS-derived
    • bad deformatter flagging should gracefully recover from bad deformatter on refant
    • Heuristic changes for bad deformatter identification
    • proper multi-band support
      • allow different min/max solints per band
      • allow different bandpass calibrators per band
    • API plot should appear in weblog
    • gaintable flagging of final gaintables based on statistics (do within final cals?)
      • heuristics need to be defined
    • Fluxboot 2 display refinements (PIPE-319, PIPE-426)
    • Fluxboot fitorder determined when different gain calibrators used for different bands
    • Antenna pointing solutions in weblog
    • observing log in weblog
    • flagging summary per stage (SRDP-411)
    • AQUA Report needs to be finished (SRDP-194, CASR-263, CASR-358)

  • VLA Spectral Line Calibration: Modifications to the system to support Spectral Line science with the VLA.
    • per-baseband delay capability should be implemented
    • ability to use spw-mapping per baseband for final gain calibration in low S/N regime (mixed spw setups)
      • pipeline should use fluxboot2 results to determine if gain calibration on individual spws will have too low S/N
    • switched power weights (CASR-476)
    • mstransform hanning smooth pass through (CASR-357)
    • SPW intents and processing options need to be passed from RCT/OPT to pipeline
      • SDM option
      • PPR option
        • Intents : Spectral Line or Continuum
        • Processing Options: Hanning smooth, RFI Flagging, Statwt
  • VLA Imaging Pipeline: Initial version of VLA Imaging pipeline targeting Ku-band and above
    • establish imaging benchmark suite using existing SRDP-QA'd SBs
    • continuum-only
    • Use nsigma threshhold
      • test automasking or not
    • X-band and higher frequency

  • Authentication & Authorization Improvements
    • Lock/Unlock icons of user interface should be accurate
    • Release status of ALMA files should be shown per-MOUS, not per-Project
    • ALMA MOUSes can be public, unreleased or released, not just public or private
    • Thumbnails/weblogs should only be shown if they are public, or if the user is authorized to see them

  • ALMA User-Defined Imaging: Additional options in the pipeline and interface
    • Pipeline
      • Selectable resolution
        • higher resolution (robust=0.0 for mosaics, robust=-0.5 single pointing)
          • possibly useful for ACA and 12m
        • max sensitivity (12 m only)
          • robust=2
        • max sensitivity + larger scale (12 m only)
          • robust=2
          • tapering of some percentile of uv distances
      • Self-calibration
        • requirements and suggested implementation under discussion
      • Add additional call to MFS imaging on continuum-subtraced data
        • Characterization of continuum subtraction quality/reliability
        • must not overwrite mfs images of continuum generated earlier in PL recipe
    • Interface
      • rest frequencies for commonly used transitions selectable (interface with splatalogue)
      • support the multiple resolution modes
      • multiple sources per request
      • multiple cube requests
      • Scrape hifa_imageprecheck weblog output to beam sizes for different resolution selections
      • Enable user to manually edit their continuum regions
        • Use cont.dat from archive as starting point (if available)
        • show plot of continuum regions
        • Ability to Pan and Zoom in plot
          • when possible use CARTA to extract spectra from archive datacubes within masks
          • plot spectra in window
        • regions defined as start and end in LSRK frequency
          • organize in rows
          • add/delete rows
          • expand manually with mouse
      • Enable user to request delivery of calibrated MS with imaging request
    • Infrastructure
      • Split workflow for Restore and Imaging
        • Enables Restore in same CASA version as reduction (e.g., 5.1.1) and newest for imaging (5.6.1)
      • Add option to deliver calibrated MS in addition to images to user
  • Improved Download Support: Cutouts for images and data selection for MS downloads
    • Archive restore workflows should use XML template recipes for VLA and ALMA delivery
      • templates provided in CASA 5.6.1/2
      • enable intent selection, target selection, frequency averaging, and time averaging
    • batch request capability
      • upload formatted catalog, obtain cutouts against selected image collections (e.g., ALMA, VLASS)
  • Metadata Improvement: Improved meta-data for products in the archive.
    • Migrate to a refreshed archive-new interface
      • Ability add/hide presentation of some metadata associated with project/observation/image
    • Thumbnails for archived image products
      • Clickable to obtain large thumbnail
      • Determine thumbnail generation mode
        • crawl and process
        • scrape weblogs
    • QA State for different telescopes
        • ALMA
        • QA2 per MOUS
        • QA0 per EB
        • SRDP Imaging
        • VLA/SRDP QA
      • Weblog from Calibrations linked and viewable through archive
        • VLA
        • ALMA Calibration
        • ALMA Imaging (If separate from Calibration run)
        • ALMA SRDP Imaging
      • VLA Receiver Band Metadata correction
        • Fix older EVLA data (listed as subband...)
      • Angular resolution
        • Displayed per MOUS for ALMA
        • Tabulated approximately per VLA band/configuration
      • Conform to standard results table from System Requirements document
      • Search operates properly on images using their field of view
      • Search operates properly for mosaic observations (EBs and MOUSes)
        • Results should be returned if search position is within mosaic
    • CARTA- Archive Integration: Integration of the CARTA viewer with the Archive.
      • Support starting CARTA backend (require to be logged-in) and user views through new web browser tab
      • User should be able to select an image product in the archive and open it in CARTA
      • User should see a progress bar so they know what is happening
        • important for large cubes that are not available within seconds
      • Ability to open second image from archive within CARTA and view or overlay
        • Multiple image selection to make available in CARTA session
      • Regions support is required
        • Upload to CARTA
        • Download from CARTA to personal computer
      • region information should be available to archive interface for cutout or search functionality
      • Ability to upload their own image
      • Overlay catalogs  (available in archive e.g., NVSS, VLASS, etc or external catalog available in CARTA)
      • external catalogs available in CARTA (e.g., 2MASS, IRAS, GAIA, FIRST, NVSS, IRAS)
      • external image servers available to CARTA (e.g., DSS, SDSS, 2MASS, IRAS, WISE)
      • Upload user's catalog
      • Enable CARTA to be used behind the scenes to extract data from Cubes (e.g., spectra) for use in interface
      • Enable user to download fits or other image format from CARTA view
    • Catalog Support: Support for searchable scientific catalogs in the Archive
      • Catalogs should be searchable from a subset of search filters
      • Return results able to be used as query for project/observation/image searches
      • Ability to hide columns
      • Interface for projects to upload catalogs for ingestion
    • Image Ingestion: If not already complete the ingestion and service of VLASS and ALMA images.
      • ALMA
      • ALMA cubes and continuum images associated with each MOUS need to be ingested
      • ALMA image products must be displayed to users and downloadable
      • Thumbnails should be present for ALMA image products
      • Viewable in a CARTA session
      • VLASS
      • VLASS QL images from the entire first epoch must be ingested
      • VLASS images must be displayed and downloadable
      • VLASS images require thumbnails and high-resolution png images
      • Viewable in CARTA session
      • VLA Images
      • When VLA imaging pipeline comes on line products must be ingested
      • Image products have same requirements as ALMA image products
      • Ancillary products for all imaging products should be made available for download
    • Large Project Support: Support for the ingest and serving of products from Large projects as collections.
      • VLA Sky Survey
      • Epochs 1.1 and 1.2 images ingested and served
      • proper position search within the images and SDMs for VLASS
      • Projects should be viewable as a 'collection' to see only the data from that project.
      • project specific metadata should be searchable
      • Interface to support upload and ingestion of large project data
    • Observatory Calibrator Database: Development of a centralized calibrator database for the VLA.
      • Calibrator Flux densities
      • Per VLA Band
        • Central frequency
        • bandwidth
      • spectral index and curvature
      • Date
      • Calibrator models
      • Time-dependent models
      • Polarization models
      • models with spatial and spectral information
      • Antenna position database

    • Performance Improvement: Efforts to make the system more responsive and decrease product latency.
      • Optimized imaging
      • Multiple sources per MOUS grouped into single request
      • Multiple cubes specified per source(s) grouped into single request
      • Reduce restore load if user comes back to specifc MOUS several times for different jobs
      • Skip continuum finding step when possible
          • Data from current ALMA cycle should use the archived cont.dat when possible
          • Older data should always make use of current continuum finding routines
          • Must check that source(s)/spw selected for imaging has continuum defined in cont.dat
            • Data that are size mitigated might not have a region defined for all sources/spws
      • Restore
        • Reduce latency for popular data sets
        • Restore datasets with same CASA version as calibrated with

Relevant Documents

  • No labels