...
input: ../datainput: cfcache_nowb='/mnt/scratch/cfcache/cfcache_spw2-17_imsize16384_cell0.6arcsec_w32_conjT_psf_wbawp_False.cf'output: VIP_iter0b.*
...
Task02
This task doesn't parallelize and only takes tens of seconds to run. Should this be stuck on the end of task01?
...
- input: ../data
- input: ../VLASS1Q.fits, VIP_iter0.psf.tt0
- output: mask_from_cat.crtf, VIP_QLcatmask.mask
...
Task03
Doesn't alter the MS
This task could possibly run at the same time as Task01 except that I have combined this with Task05 which requires both Task01 and Task04.
...
- input: ../data
- output: VIP_iter1.*
...
Task04
Doesn't alter the MS
replace_psf('iter1','iter0')
...
- input: VIP_iter0.psf.*, VIP_iter1.psf.*
- output: VIP_iter1.psf.*
...
Task05 -
...
Step05
Doesn't alter the MS
run_tclean( 'iter1', robust=-2.0, uvtaper="3arcsec", niter=20000, nsigma=5.0, mask="QLcatmask.mask", calcres=False, calcpsf=False )
- input: ../data
- input: VIP_iter1.*, VIP_QLcatmask.mask
- output: VIP_iter1.*
...
Task06 -
...
Step06
Alters the MS
run_tclean( 'iter1', calcres=False, calcpsf=False, savemodel='modelcolumn', parallel=False )
- input: ../data
- input: VIP_iter1.*
- output: ../data
...
Task07 -
...
Step07
Alters the MS
Tasks 08, 09, 10 and 11 take only minutes to run so could be combined into one DAG step.
...
- input: ../data
- output: tdev.txt,. fdev.txt
- ouput: ../data
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/./table.f6
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/./HISTORY/table.f0
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/./HISTORY/table.lock
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/./table.f20_TSM1
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/./table.lock
- output: ../data/VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions
...
Task08
Alters the MS
statwt(vis=vis,combine='field,scan,state,corr',chanbin=1,timebin='1yr', datacolumn='residual_data' )
- input: ../data
- output: ../data
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions/FLAG_VERSION_LIST
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions/flags.statwt_1/table.f1
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions/flags.statwt_1/table.dat
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions/flags.statwt_1/table.f0
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions/flags.statwt_1/table.lock
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions/flags.statwt_1/table.info
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms.flagversions/flags.statwt_1/table.f0_TSM1
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.f25_TSM1
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.f22_TSM1
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.f6
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/HISTORY/table.f0
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/HISTORY/table.lock
- VLASS1.2.sb36491855.eb36574404.58585.53016267361_split.ms/table.lock
...
Task09
Doesn't alter the MS
gaincal(vis=vis,caltable='g.0',gaintype='T',calmode='p',refant='0',combine='field,spw',minsnr=5)
- input: ../data
- output: g.0
...
Task10
Alters the MS
applycal(vis=vis,calwt=False,applymode='calonly',gaintable='g.0',spwmap=18*[2], interp='nearest')
- input: ../data
- input: g.0
- output: ../data
...
Task11
Doesn't alter the MS
run_tclean( 'iter0c', datacolumn='corrected', cfcache=cfcache_nowb, robust=-2.0, uvtaper='3arcsec', calcres=False )
- input: ../data
- output: VIP_iter0c.*
...
Task12
Doesn't alter the MS
Could this run in parallel with one or more previous run_tclean calls like Task12?
...
- input: ../data
- output: VIP_iter0d.*
...
Task13
Doesn't alter the MS
This task could possibly run at the same time as Task12 or Task13 except that I have combined this with Task15 which requires both Task14 and Task12.
...
- input: ../data
- output: VIP_iter1b.*
...
Task14
Doesn't alter the MS
replace_psf('iter1b','iter0c')
...
- input: VIP_iter1b.psf.*, VIP_iter0c.psf.*
- output: VIP_iter1b.psf.*
...
Task15 -
...
Step15
Doesn't alter the MS
run_tclean( 'iter1b', datacolumn='corrected', robust=-2.0, uvtaper="3arcsec", niter=20000, nsigma=5.0, mask="QLcatmask.mask", calcres=False, calcpsf=False )
- input: ../data
- input: VIP_iter1b.*, VIP_QLcatmask.mask
- output: VIP_iter1b.*
...
Task16 -
...
Step16
Doesn't alter the MS
imsmooth(imagename=imagename_base+"iter1b.image.tt0", major='5arcsec', minor='5arcsec', pa='0deg', outfile=imagename_base+"iter1b.image.smooth5.tt0")
- input: ../data
- input: VIP_iter1b.image.tt0
- output: VIP_iter1b.image.smooth5.tt0
...
Task17
Doesn't alter the MS
exportfits(imagename=imagename_base+"iter1b.image.smooth5.tt0", fitsimage=imagename_base+"iter1b.image.smooth5.fits")
- input: ../data
- input: VIP_iter1b.image.smooth5.tt0
- output: VIP_iter1b.image.smooth5.fits
...
Task18
This needs some modification. It calls a script from Josh's homedir and runs bdsf out of /lustre. Also, I have been unable to run this task by itself. I get the following errors. I am going to combine this with tasks17, 18, 20 and 21 so it isn't an issue right now.
...
- input: ../data
- input: VIP_iter1b.image.smooth5.fits
- input: ??
- output:
- VIP_iter1b.image.smooth5.cat.ds9.reg
- VIP_iter1b.image.smooth5.cat.fits
- VIP_iter1b.image.smooth5.fits.island.mask
- VIP_iter1b.image.smooth5.fits.pybdsf.log
- VIP_iter1b.image.smooth5.fits.rms
- VIP_iter1b.image.smooth5.fits
...
Task19
This needs iter0.psf.tt0 which is set to the variable inext. VIP_iter0 was copied to VIP_iter1 back in Task05
...
- input: VIP_iter1b.image.smooth5.cat.fits
- input: VIP_iter1b.image.smooth5.cat.edited.fits, VIP_iter0.psf.tt0
- output: secondmask.mask
...
Task20
Doesn't alter the MS
immath(imagename=[imagename_base+'secondmask.mask',imagename_base+'QLcatmask.mask'],expr='IM0+IM1',outfile=imagename_base+'sum_of_masks.mask')
...
- input: secondmask.mask, VIP_QLcatmask.mask
- output: sum_of_masks.mask
- input: sum_of_masks.mask
- output: VIP_combined.mask
...
Task21
Doesn't alter the MS
As far as I can tell at this point, ../data has not changed since Task11 (applycal).
...
- input: ../data
- output: VIP_iter2.*
...
Task22
Doens't alter the MS
replace_psf('iter2', 'iter0d')
...
- input: VIP_iter2.psf.*, VIP_iter0d.psf.*
- output: VIP_iter2.psf.*
...
Task23 - Step23
Doesn't alter the MS
At this point I think we are using iter2's image with iter0d's psf.
...
- input: ../data
- input: VIP_iter2.*, VIP_QLcatmask.mask
- output: VIP_iter2.*
...
Task24 - Step24
os.system('rm -rf *.workdirectory')
...