Page History

...

• DONE: write .log, .out and .png files one level up so they are not in the working directory and therefore not copied to execute hosts.
• DONE: add rm -f *.last to the sh script?
• DONE: Task24 and Task25 swapped with 8cores so they need to run with fewer cores.  So I may need to make another variable to pass to the sh script for this.
• Task19 needs to be unwraveled from NRAO filesystems.
• I don't like using the name Task as that has meaning to CASA.  A better term might be Step as in DAG Step.
• Look into running Task22 at the same time as another Task.  If it is possible, perhaps this can be done for some other tasks as well.  Note, I specificly avoided using the work parallel here so as not to confuse this idea with using mpicasa.
• Figure out how to not copy SYSPOWER in the MS.  Presumably we can just cp /dev/null SYSPOWER/table.f0 and cp /dev/null SYSPOWER/table.f0i
• De-link besf_1.8.15-py27-rh7-env.  I can't just tar it up because it links to /usr/include and /usr/lib and I can't trust those are on CHTC.  This is for Task17.
• re-create my DOT graph after finishing task01-25-parallel-dag4. Also make a PDF instead of a PS.
• Craft parallel-dag5 with the concept of running tclean at CHTC and everything else locally.

Possible Improvements

• Task04 could perhaps be run concurrently with Task01 as long as Task05 is run after both.
• Task13 could perhaps be run concurrently with Task12.
• Task14 could perhaps be run concurrently with Taks12 and Task13 as long as Task15 is run after both Task12 and Task14.
• Task22 could perhaps be run concurrently with Task12 or later as long as Task23 is run after both Task22 and Task13.

Task01 - Step01

Doesn't alter the MS

run_tclean( 'iter0', cfcache=cfcache_nowb, robust=-2.0, uvtaper='3arcsec', calcres=False  )

...

• input: ../data
• input: ../VLASS1Q.fits, VIP_iter0.psf.tt0
• output: mask_from_cat.crtf, VIP_QLcatmask.mask

Task04 - Step04

Doesn't alter the MS

This task could possibly run at the same time as Task01 except that I have combined this with Task05 which requires both Task01 and Task04.

...

• input: VIP_iter0.psf.*, VIP_iter1.psf.*
• output: VIP_iter1.psf.*

Task06 - Step06

Doesn't alter the MS

run_tclean( 'iter1', robust=-2.0, uvtaper="3arcsec", niter=20000, nsigma=5.0, mask="QLcatmask.mask", calcres=False, calcpsf=False  )

• input: ../data
• input: VIP_iter1.*, VIP_QLcatmask.mask
• output: VIP_iter1.*

Task07 - Step07

Alters the MS

run_tclean( 'iter1', calcres=False, calcpsf=False, savemodel='modelcolumn', parallel=False  )

• input: ../data
• input: VIP_iter1.*
• output: ../data
  • VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.f23_TSM1
  • VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/SOURCE/table.lock
  • VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.lock

Task08 - Step08

Alters the MS

Tasks 08, 09, 10 and 11 take only minutes to run so could be combined into one DAG step.

...

• input: ../data
• input: g.0
• output: ../data
  • VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/HISTORY/table.f0
  • VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/HISTORY/table.lock
  • VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.f24_TSM1
  • VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.lock

Task12 - Step12

Doesn't alter the MS

run_tclean( 'iter0c', datacolumn='corrected', cfcache=cfcache_nowb, robust=-2.0, uvtaper='3arcsec', calcres=False  )

• input: ../data
• output: VIP_iter0c.*

Task13 - Step13

Doesn't alter the MS

Could this run in parallel with one or more previous run_tclean calls like Task12?

...

• input: ../data
• output: VIP_iter0d.*

Task14 - Step14

Doesn't alter the MS

This task could possibly run at the same time as Task12 or Task13 except that I have combined this with Task15 which requires both Task14 and Task12.

...

• input: VIP_iter1b.psf.*, VIP_iter0c.psf.*
• output: VIP_iter1b.psf.*

Task16 - Step16

Doesn't alter the MS

run_tclean( 'iter1b', datacolumn='corrected', robust=-2.0, uvtaper="3arcsec", niter=20000, nsigma=5.0, mask="QLcatmask.mask", calcres=False, calcpsf=False  )

• input: ../data
• input: VIP_iter1b.*, VIP_QLcatmask.mask
• output: VIP_iter1b.*

Task17 - Step17

Doesn't alter the MS

imsmooth(imagename=imagename_base+"iter1b.image.tt0", major='5arcsec', minor='5arcsec', pa='0deg', outfile=imagename_base+"iter1b.image.smooth5.tt0")

...

im.mask(image=imagename_base+'sum_of_masks.mask',mask=imagename_base+'combined.mask',threshold=0.5)

• input: secondmask.mask, VIP_QLcatmask.mask
• output: sum_of_masks.mask
• input: sum_of_masks.mask
• output: VIP_combined.mask

Task22 - Step22

Doesn't alter the MS

As far as I can tell at this point, ../data has not changed since Task11 (applycal).

...

• input: ../data
• input: VIP_iter2.*, VIP_QLcatmask.mask
• output: VIP_iter2.*

...

• input: ../data
• input: VIP_iter2.*, VIP_combined.mask
• output: VIP_iter2.*