Versions Compared

Old Version 43

changes.mady.by.user Morgan Griffith [X]

Saved on

compared with

New Version Current

changes.mady.by.user John Tobin

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page captures the high level planning for each wave of SRAO development.  Objectives are captured as Epics in Jira and a brief description is below.

Table of Contents
maxLevel2


Wave 2

Column
width67%

Wave 2

Wave 3

Epics

VLA Calibration Improvement: Improvements to the flagging heuristics for system and RFI Issues.
  • RFI heuristics for VLA PI pipeline will be modified - moving toward VLASS or VLASS-derived
  • bad deformatter flagging should gracefully recover from bad deformatter on refant
  • Heuristic changes for bad deformatter identification
  • proper multi-band support
    • allow different min/max solints per band
    • allow different bandpass calibrators per band
  • API plot should appear in weblog
  • gaintable flagging of final gaintables based on statistics (do within final cals?)
    • heuristics need to be defined
  • Fluxboot 2 display refinements (PIPE-319, PIPE-426)
  • Fluxboot fitorder determined when different gain calibrators used for different bands
  • Antenna pointing solutions in weblog
  • observing log in weblog
  • flagging summary per stage (SRDP-411)
    • AQUA Report needs to be finished (SRDP-194, CASR-263, CASR-358)
  • VLA Spectral Line Calibration: Modifications to the system to support Spectral Line science with the VLA.
    • per-baseband delay capability should be implemented
    • ability to use spw-mapping per baseband for final gain calibration in low S/N regime (mixed spw setups)
      • pipeline should use fluxboot2 results to determine if gain calibration on individual spws will have too low S/N
    • switched power weights (CASR-476)
    • mstransform hanning smooth pass through (CASR-357)
    • SPW intents and processing options need to be passed from RCT/OPT to pipeline
      • SDM option
      • PPR option
        • Intents : Spectral Line or Continuum
        • Processing Options: Hanning smooth, RFI Flagging, Statwt
    • VLA Imaging Pipeline: Initial version of VLA Imaging pipeline targeting Ku-band and above
  • establish imaging benchmark suite using existing SRDP-QA'd SBs
  • continuum-only
  • Use nsigma threshhold
    • test automasking or not
    • X-band and higher frequency
  • ALMA User-Defined Imaging: Additional options in the pipeline and interface
    • Pipeline
      • Selectable resolution
        • higher resolution (robust=0.0 for mosaics, robust=-0.5 single pointing)
          • possibly useful for ACA and 12m
        • max sensitivity (12 m only)
          • robust=2
        • max sensitivity + larger scale (12 m only)
          • robust=2
          • tapering of some percentile of uv distances
      • Self-calibration
        • requirements and suggested implementation under discussion
      • Add additional call to MFS imaging on continuum-subtraced data
        • Characterization of continuum subtraction quality/reliability
        • must not overwrite mfs images of continuum generated earlier in PL recipe
    • Interface
      • rest frequencies for commonly used transitions selectable (interface with splatalogue)
      • support the multiple resolution modes
      • multiple sources per request
      • multiple cube requests
      • Scrape hifa_imageprecheck weblog output to beam sizes for different resolution selections
      • Enable user to manually edit their continuum regions
        • Use cont.dat from archive as starting point (if available)
        • show plot of continuum regions
        • Ability to Pan and Zoom in plot
          • when possible use CARTA to extract spectra from archive datacubes within masks
          • plot spectra in window
        • regions defined as start and end in LSRK frequency
          • organize in rows
          • add/delete rows
          • expand manually with mouse
      • Enable user to request delivery of calibrated MS with imaging request
    • Infrastructure
      • Split workflow for Restore and Imaging
        • Enables Restore in same CASA version as reduction (e.g., 5.1.1) and newest for imaging (5.6.1)
      • Add option to deliver calibrated MS in addition to images to user
  • Improved Download Support: Cutouts for images and data selection for MS downloads
    • Archive restore workflows should use XML template recipes for VLA and ALMA delivery
      • templates provided in CASA 5.6.1/2
      • enable intent selection, target selection, frequency averaging, and time averaging
    • batch request capability
      • upload formatted catalog, obtain cutouts against selected image collections (e.g., ALMA, VLASS)
    • Metadata Improvement: Improved meta-data for products in the archive.
  • Migrate to a refreshed archive-new interface
    • Ability add/hide presentation of some metadata associated with project/observation/image
  • Thumbnails for archived image products
    • Clickable to obtain large thumbnail
    • Determine thumbnail generation mode
      • crawl and process
      • scrape weblogs
    • Must be proprietary-state aware
      • not show thumbnails for
  • QA State for different telescopes
      • ALMA
      • QA2 per MOUS
      • QA0 per EB
      • SRDP Imaging
      • VLA/SRDP QA
    • Weblog from Calibrations linked and viewable through archive
      • VLA
      • ALMA Calibration
      • ALMA Imaging (If separate from Calibration run)
      • ALMA SRDP Imaging
    • Proprietary state properly reported
      • ALMA must be correct
      • VLA
    • VLA Receiver Band Metadata correction
      • Fix older EVLA data (listed as subband...)
    • Angular resolution
      • Displayed per MOUS for ALMA
      • Tabulated approximately per VLA band/configuration
    • Conform to standard results table from System Requirements document
    • Search operates properly on images using their field of view
    • Search operates properly for mosaic observations (EBs and MOUSes)
      • Results should be returned if search position is within mosaic
  • CARTA- Archive Integration: Integration of the CARTA viewer with the Archive.
    • Support starting CARTA backend (require to be logged-in) and user views through new web browser tab
    • User should be able to select an image product and open it in CARTA
    • User should see a progress bar so they know what is happening
      • important for large cubes that are not available within seconds
    • Ability to open second image from archive within CARTA and overlay
    • Regions support is required
    • region information should be available to archive interface for cutout or search functionality
    • Ability to upload their own image
    • Overlay catalogs  (available in archive or external catalog available in CARTA)
    • those in archive (e.g., NVSS, VLASS, etc)
    • external catalogs available in CARTA (e.g., 2MASS, IRAS, GAIA, FIRST, NVSS, IRAS)
    • external image servers available to CARTA (e.g., DSS, SDSS, 2MASS, IRAS, WISE)
    • Upload user's catalog
    • Enable CARTA to be used behind the scenes to extract data from Cubes (e.g., spectra) for use in interface
  • Catalog Support: Support for searchable scientific catalogs in the Archive
    • Catalogs should be searchable from a subset of search filters
    • Return results able to be used as query for project/observation/image searches
    • Ability to hide columns
    • Interface for projects to upload catalogs for ingestion
  • Image Ingestion: If not already complete the ingestion and service of VLASS and ALMA images.
    • ALMA
    • ALMA cubes and continuum images associated with each MOUS need to be ingested
    • ALMA image products must be displayed to users and downloadable
    • Thumbnails should be present for ALMA image products
    • Viewable in a CARTA session
    • VLASS
    • VLASS QL images from the entire first epoch must be ingested
    • VLASS images must be displayed and downloadable
    • VLASS images require thumbnails and high-resolution png images
    • Viewable in CARTA session
    • VLA Images
    • When VLA imaging pipeline comes on line products must be ingested
    • Image products have same requirements as ALMA image products
    • Ancillary products for all imaging products should be made available for download
  • Large Project Support: Support for the ingest and serving of products from Large projects as collections.
    • VLA Sky Survey
    • Epochs 1.1 and 1.2 images ingested and served
    • proper position search within the images and SDMs for VLASS
    • Projects should be viewable as a 'collection' to see only the data from that project.
    • project specific metadata should be searchable
    • Interface to support upload and ingestion of large project data
    • Observatory Calibrator Database: Development of a centralized calibrator database for the VLA.
  • Calibrator Flux densities
  • Per VLA Band
    • Central frequency
    • bandwidth
  • spectral index and curvature
  • Date
  • Calibrator models
  • Time-dependent models
  • Polarization models
  • models with spatial and spectral information
    • Antenna position database
    Performance Improvement: Efforts to make the system more responsive and decrease product latency.
  • Optimized imaging
  • Multiple sources per MOUS grouped into single request
  • Multiple cubes specified per source(s) grouped into single request
  • Reduce restore load if user comes back to specifc MOUS several times for different jobs
  • Skip continuum finding step when possible
      • Data from current ALMA cycle should use the archived cont.dat when possible
      • Older data should always make use of current continuum finding routines
      • Must check that source(s)/spw selected for imaging has continuum defined in cont.dat
        • Data that are size mitigated might not have a region defined for all sources/spws
    • Restore
  • Reduce latency for popular data sets
    • Restore datasets with same CASA version as calibrated with

    Column
    width33%

    Relevant Documents