VLASS SE Imaging Operations Meeting 7-21-20

Date and Time

21 Jul 202016:00 EDT / 14:00 MDT

Video: https://zoom.us/j/5646363780?pwd=VGxrUGFrTTUrUmZHRGQ1L3poMTN6dz09 

Main page: VLASS SE Imaging operations

Roadmap: SE Imaging Operations Roadmap

External processing page: VLASS SE External Processing

HTCondor area: High Throughput Computing with HTCondor

VIP Script timing info: https://docs.google.com/spreadsheets/d/1yGpNNexL9YdEST3-_zyqtNqK2UxSNvmzKFOeRyanOHQ/edit#gid=0

All tickets pertaining to VLASS SE operations should have the VLASS-SE-OPS label.  Add VLASS-SE-DISCUSS label if it should be brought up in this meeting

Open Issues

• Need to discuss loop control for splitting MSes into relevant sq dg.  E.g. a single cluster job that walks through the N phase centers or N jobs submitted per phase center.
  • Note the pipeline currently works on single sq dg.  
• There is a need to split out per sq dg or all sq dg,  is it necessary to support splitting by tile.  Is a 'tile' an sufficient object within the VLASS manager that can be used to define that split ?
• How to handle Continuum products that may be of use for Cube imaging

Decisions

• Plan is to manage loop control as N unique jobs, one per phase center
• For now SSA plans to support single or multiple phase centers,  per tile and per MS splits.  Implementation may initially only support single phase center and entire MS but ultimately all 4 will be supported
• No conclusion on products,  ultimately they will probably be ingested as regular products but Jeff wants to investigate more before making a decision.
• No meeting next week,  next meeting is August 4th in the lead up to the SSA planning meeting.

Agenda (Primarily focused on roadmap)

• Coordination/Logistics
  • <nothing>
• Schedule (broad strokes, need to flesh out details)
  • <nothing>
• SSA
  • Discuss requirements capture and workflow
  • Need list of existing tickets tied to SE imaging (I can go in and add labels)
  • HTCondor effort, when to start, how to coordinate, how to set up test environment (want to start in June, need to discuss what it should look like)
  • How to track SE-continuum products and their possible inclusion in SE-cube imaging (relevant in the short term,  inputs to Cube need to be known so they can drive continuum products  (IDs, name formats etc)
• SCG
  • Working on drawing, settle on Google Draw,  simplest.
    • https://docs.google.com/drawings/d/1JtN7BcDl_X-dab11b40GMNwhmxxC9vxHFUQTiopFn8g/edit?usp=sharing
• Pipeline
  • <nothing>

Action items

•  James to restructure page and add a Open Issues section.  Loop control for spliting MSes first on list