...
- DONE: write .log, .out and .png files one level up so they are not in the working directory and therefore not copied to execute hosts.
- DONE: add rm -f *.last to the sh script?
- DONE: Task24 and Task25 swapped with 8cores so they need to run with fewer cores. So I may need to make another variable to pass to the sh script for this.
- DONE: re-create my DOT graph after finishing task01-25-parallel-dag4. Also make a PDF instead of a PS.
- DONE: Craft parallel-dag5 with the concept of running tclean at CHTC and everything else locally.
- DONE: I don't like using the name Task as that has meaning to CASA. A better term might be Step as in DAG Step.
- DONE: Figure out how to not copy SYSPOWER in the MS. Presumably we can just cp /dev/null SYSPOWER/table.f0 and cp /dev/null SYSPOWER/table.f0i I am doing this with my dag5 test now.
- Task19 needs to be unwraveled from NRAO filesystems. Actually maybe not. This can be a task we always run here. But it does need to move into a VLASS area instead of Josh's home acocunt.
- DONE: Setup testpost-serv-1 with Lustre access over IB so we can start submitting to CHTC.
- DONE: Need to make the MS in my DAG script a variable. Right now I specify data/VLASS.../table.f23_TSM1
Possible Improvements
- Task19 needs to be unwraveled from NRAO filesystems. Actually maybe not. This can be a task we always run here. But it does need to move into a VLASS area instead of Josh's home acocunt.
- Task06 has parallel=false and I am using 8core mpicasa which is a waste of cores.
- Update this document to reflect the changes needed for CASA-6.
Possible Improvements
- Task03 could perhaps be run concurrently with Task01 as Task03 could perhaps be run concurrently with Task01 as long as Task04 is run after both.
- Task13 could perhaps be run concurrently with Task12.
- Task13 could perhaps be run concurrently with Taks11 and Task12 as long as Task14 is run after both Task11 and Task13.
- Task21 could perhaps be run concurrently with Task11 or later as long as Task22 is run after both Task21 and Task12.
...
- input: ../data
- input: VIP_iter1.*, VIP_QLcatmask.mask
- output: VIP_iter1.*
Task06 - Step06
Alters the MS
Note that this sets parallel=False which means running mpicasa may be a waste of cores. Hopefully this step will not be necessary with CASA-6.Alters the MS
run_tclean( 'iter1', calcres=False, calcpsf=False, savemodel='modelcolumn', parallel=False )
- input: ../data
- input: VIP_iter1.*
- output: ../data
VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.f23_TSM1VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/SOURCE/table.lock
Task07 - Step07
Alters the MS
...