Date and Time
Sep 202016:00 EDT / 14:00 MDT
Video: https://zoom.us/j/5646363780?pwd=VGxrUGFrTTUrUmZHRGQ1L3poMTN6dz09
...
External processing page: VLASS SE External Processing
Pipeline version of PI script: https://gitlab.nrao.edu/nrao-pl-devs/vlass-imaging-project
HTCondor area: High Throughput Computing with HTCondor
Image Comparison Tool: Image Comparison
VIP Script timing info: https://docs.google.com/spreadsheets/d/1yGpNNexL9YdEST3-_zyqtNqK2UxSNvmzKFOeRyanOHQ/edit#gid=0
All tickets pertaining to VLASS SE operations should have the VLASS-SE-OPS label. Add VLASS-SE-DISCUSS label if it should be brought up in this meeting
News
Open Issues
- Need to determine whether pybdsf can be run from within CASA, possibly it will work with CASA6 derivative.
- Possibly a problem with the smaller Josh provided data set with CASA6
- Need to have smaller data set's cfcache's preloaded on VLASS nodes
Decisions
- Josh Marvil's test data set includes requisite CFCaches. It reduces flops and memory via fewer channels and smaller image size, completes with 8-way parallelzation through VIP workflow in ~24hours.
Agenda
- Pipeline
- Initial CASA6 based tests show pronounced speed up, SCG to help investigate.
- SSA
Is the proposed directory structure (/lustre/aoc/cluster/pipeline/vlass_auto/cache/pims/EB/CAL/DEC/RA) acceptable to everyone
What people want to see as directory components for EBs, calibrations, RA and dec for the phase center. Less essential since it will be mostly managed by the tool
what syntax do people want to use on the CLI to input this information.
- SCG
- Processing 47 + 7 repeat square degrees in AWS
Action items
- Brian Kentor Laszlo Szucs [X] to point to failed execution for Josh's smaller data set
- James Robnett [X] to provide qsub call for 2x8 parallelization
- SE Imaging Workflow: Is the component list generation step in the workflow or does it only run the pipeline
- How are SE calibration products handled in the VLASS Manager (I'll explain)
Decision on whether to produce a Mosaic implementation (we will produce one)Pipeline available for testing (available for AWP implementation, Mosaic in progress)- SSA PIMS workflow validated (validated)
- SSA SE Imaging workflow ready for testing (In progress ?)
Pipeline validated (6/10)Pipeline released (6/10)- SSA SE Imaging workflow validated
- Begin operations (shooting for June
1st25th) - Migrate above to htcondor (Mid/late August)
- Accept and archive implemented
Decisions
Agenda
- Pipeline
- SSA
- PIMS testing in progress
- SE Imaging workflow testing state
- HTCondor design
- SCG
- Verification of pipeline vs VIP script in mosaic and AWP (AWP is consistent). Stalled for now
- Investigation of GPU gridder vs CASA underway
Action items
Ticket Discussions
| Jira | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|