...
- DONE: write .log, .out and .png files one level up so they are not in the working directory and therefore not copied to execute hosts.
- DONE: add rm -f *.last to the sh script?
- DONE: Task24 and Task25 swapped with 8cores so they need to run with fewer cores. So I may need to make another variable to pass to the sh script for this.
- DONE: re-create my DOT graph after finishing task01-25-parallel-dag4. Also make a PDF instead of a PS.
- DONE: Craft parallel-dag5 with the concept of running tclean at CHTC and everything else locally.
- DONE: I don't like using the name Task as that has meaning to CASA. A better term might be Step as in DAG Step.
- DONE: Figure out how to not copy SYSPOWER in the MS. Presumably we can just cp /dev/null SYSPOWER/table.f0 and cp /dev/null SYSPOWER/table.f0i I am doing this with my dag5 test now.
- Task19 needs to be unwraveled from NRAO filesystems. Actually maybe not. This can be a task we always run here. But it does need to move into a VLASS area instead of Josh's home acocunt.
- DONE: Setup testpost-serv-1 with Lustre access over IB so we can start submitting to CHTC.
- DONE: Need to make the MS in my DAG script a variable. Right now I specify data/VLASS.../table.f23_TSM1
Possible Improvements
- Task19 needs to be unwraveled from NRAO filesystems. Actually maybe not. This can be a task we always run here. But it does need to move into a VLASS area instead of Josh's home acocunt.
- Task06 has parallel=false and I am using 8core mpicasa which is a waste of cores.
- Update this document to reflect the changes needed for CASA-6.
Possible Improvements
- Task03 could perhaps be run concurrently with Task01 as long as Task04 Task04 could perhaps be run concurrently with Task01 as long as Task05 is run after both.
- Task13 could perhaps be run concurrently with Task12.
- Task14 Task13 could perhaps be run concurrently with Taks12 Taks11 and Task13 Task12 as long as Task15 Task14 is run after both Task12 Task11 and Task14Task13.
- Task22 Task21 could perhaps be run concurrently with Task12 Task11 or later as long as Task23 Task22 is run after both Task22 Task21 and Task13Task12.
Task01 - Step01
Doesn't alter the MS
...
This task could possibly run at the same time as Task01 except that I have combined this with Task05 Task04 which requires both Task01 and Task04Task34.
run_tclean( 'iter1', robust=-2.0, uvtaper="3arcsec" )
...
This is just some python that deletes VIP_iter1.psf.* [tt0|tt1|tt2] and copies VIP_iter0.psf.* .[tt0|tt1|tt2] to VIP_iter1.psf.*[tt0|tt1|tt2]. It is would be inefficient to ever make this task be its own DAG step because the job would have to transfer iter0 and iter1 to the scratch area just to make the copy. I suggest it always be in the same DAG step as Task04Task03. Will It will produce an error because *.workdirectory doesn't exist but that error is ignorable.
...
- input: ../data
- input: VIP_iter1.*, VIP_QLcatmask.mask
- output: VIP_iter1.*
Task06 - Step06
Alters the MS
- VIP_iter1.*
Task06 - Step06
Alters the MS
Note that this sets parallel=False which means running mpicasa may be a waste of cores. Hopefully this step will not be necessary with CASA-6.
run_tclean( 'iter1', calcres=False, calcpsf=False, savemodel='modelcolumn', parallel=False )
- input: ../data
- input: VIP_iter1.*
- output: ../data
- input: ../data
- input: VIP_iter1.*
- output: ../data
VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.f23_TSM1VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/SOURCE/table.lock
Task07 - Step07
Alters the MS
...
Could this run in parallel with one or more previous run_tclean calls like Task12Task11?
run_tclean( 'iter0d', datacolumn='corrected', cfcache=cfcache_nowb, calcres=False )
...
This task could possibly run at the same time as Task11 and/or Task12 or Task13 except that I have combined this with Task15 Task14 which requires both Task14 Task13 and Task12Task11.
run_tclean( 'iter1b', datacolumn='corrected', robust=-2.0, uvtaper="3arcsec" )
...
This needs iter0.psf.tt0 which is set to the variable inext. VIP_iter0 was copied to VIP_iter1 back in Task05Task04
Doesn't alter the MS
edit_pybdsf_islands(catalog_fits_file=imagename_base+'iter1b.image.smooth5.cat.fits')
...
As far as I can tell at this point, ../data has not changed since Task11 Task10 (applycal).
Could this run in parallel with one or more previous run_tclean calls like Task16Task15?
run_tclean( 'iter2', datacolumn='corrected' )
...
This is just some python that deletes VIP_iter2.psf.* and copies VIP_iter0d.psf.* to VIP_iter2.psf.*. It is inefficient to ever make this task be its own DAG. I suggest it always be in the same DAG as Task22Task21.
- input: VIP_iter2.psf.*, VIP_iter0d.psf.*
- output: VIP_iter2.psf.*
...