Time: 2pm MT, 4pm ET
https://zoom.us/j/5646363780?pwd=VGxrUGFrTTUrUmZHRGQ1L3poMTN6dz09 (full info at bottom of page)
Main page: VLASS SE Imaging operations
External processing page: VLASS SE External Processing
HTCondor area: High Throughput Computing with HTCondor
Attendees: James,
Agenda (Primarily a getting organized meeting)
- Purpose of group meeting
- Coordinate all DMS effort to support VLASS SE imaging internally and externally
- Coordination/Logistics
- Track everything here or just major targets and let groups track their own deliverables deliverables and JIRA tickets (I'd prefer the latter)
- Answer: Track effort globally for confluence and then link into JIRA via label
- Use of tags to make clearing house pages easy (ie select for all tickets in SSA/SCG/CASA/CASA-pipeline/ARDG with tag 'XXXX'
- Answer: need a label james will come up with.
- Track everything here or just major targets and let groups track their own deliverables deliverables and JIRA tickets (I'd prefer the latter)
- Schedule (broad strokes, need to flesh out details)
- Demonstrate 100 sq degrees via HTCondor external to pipleine pipeline by October 1st.
- VIP script implemented as pipeline by ? (would like October) (Likely Q2 FY21)
- SSA natively uses HTCondor for SE job submissions by ? (would like September or October) (Likely Q1FY21)
- Pipeline
- Is the plan to make a separate task or leverage existing task
- how to manage pybdsf and other external python apps
- Split pipeline HTCondor
- apps (largest concern), could use sequence manager class to iterate over tclean in different ways, may be able to use
- Can each tclean() call be checkpointed and restarted.
- Packaging question w.r.t. pipeline for external packages
- cfcache tracking is a workflow issue not a pipeline issue
- Can some external scripts be reimplemented in CASA pipeline or always externally called scripts
- Weblog completely uncertain
- SSA
- Split pipeline between data selection and mstransform and imaging (should this be done in the pipeline recipe, ie hifv_splitms() or some such)
- HTCondor effort, when to start, how to coordinate, how to set up test environment (want to start in June, need to discuss what it should look like)
- How to incorporate into workflow manager vs vlass manager
- How to track SE-continuum products and their possible inclusion in SE-cube imaging (relevant in the short term, inputs to Cube need to be known so they can drive continuum products (IDs, name formats etc)
- SCG
- Return to effort to create an external imaging approach with decomposition along the major/minor cycle and SPWs. Imaging with HTCondor
- Assist other groups, set up test environments
- Coordinate with CHTC NRAO-CHTC HTCondor collaboration
- Can we split the pipeline in to the PSF and MASK steps and the final imaging steps, has utility for the
Full meeting connection info
Join Zoom Meeting
https://zoom.us/j/5646363780?pwd=VGxrUGFrTTUrUmZHRGQ1L3poMTN6dz09
Meeting ID: 564 636 3780
Password: 3780
One tap mobile
+12532158782,,5646363780#,,1#,3780# US (Tacoma)
+13462487799,,5646363780#,,1#,3780# US (Houston)
Dial by your location
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 669 900 9128 US (San Jose)
+1 301 715 8592 US (Germantown)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
Meeting ID: 564 636 3780
Password: 3780
Find your local number: https://zoom.us/u/ab9WnMz3mT