You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Page for presenting results; description of the tests is presented in the parent page:

Calibration pipeline

Goal: Test the behavior of Tier0 parallelization of calibrator imaging in the calibration pipelline


Table: Runtime of the calibration pipeline vs. CASA version, 8 way parallel runs (unless stated otherwise) and 256 GB memory limit.

Projectcasa-pipeline-release-5.6.1-8.el7casa-6.1.0-63casa-CAS-9386-53casa-CAS-9386-53_2MPI
2017.1.00717.S28h51m11s32h14m43s31h37m52s33h09m28s
2017.1.00750.T05h17m12s04h52m57s04h50m23s04h42m11s
2017.1.00884.S09h15m26s08h46m16s08h43m33s08h36m50s
2017.1.00983.S55h41m06s51h29m49s51h27m05s58h05m08s
2017.1.01214.S28h54m29s22h41m17s22h39m23s20h49m49s
E2E6.1.00080.S14h01m18s13h14m39s13h18m04s14h25m03s
E2E6.1.00092.S61h26m59s65h46m20s66h13m38s70h14m20s


No obvious issues in calibration, slight increase in runtime which is not unexpected given tclean() runtime change. Some things could be investigated but no current plans given competing time demands.


Imaging pipeline

Goal: Demonstrate that the refactored code has the desired memory footprint effect.


Table: Memory footprint of entire pipeline run vs. CASA version, 8 way parallel runs with 128 GB memory limit,


Memory footprint (GB)
Projectcasa-pipeline-release-5.6.1-8.el7casa-6.1.0-63casa-CAS-9386-53
2017.1.00717.S15.9602851915.3538589517.16140747
2017.1.00750.T5.3002357483.9925956734.630950928
2017.1.00884.S48.8861808853.224071579.35606766
2017.1.00983.S50.8941841150.6911315959.88224411
2017.1.01214.S20.7219772320.3068275524.82666397
E2E6.1.00080.S47.1487808245.9478797949.49978256
E2E6.1.00092.S23.1033630422.399372157.35647202


Memory footprint is systematically higher for the refactor case. Which is opposite of expected. Potentially this is as an efficiency improvement in chanchunk estimation and not the actual natural unconstrained limit which would suggest that the memory usage would increase as available memory increased.


Goal: Demonstrate the runtime cost of the refactored code and whether it's a fixed overhead so it's contribution goes to zero for larger data sets or whether the overhead scales with image complexity


Table: Runtime of imaging pipeline vs. CASA version, 8 way parallel runs with 128 GB memory limit.

Project

casa-pipeline-release-5.6.1-8.el7

casa-6.1.0-63casa-CAS-9386-53
2017.1.00717.S18h36m15s17h21m13s15h43m25s
2017.1.00750.T06h41m32s05h30m07s02h55m08s
2017.1.00884.S07h17m28s06h45m21s05h50m07s
2017.1.00983.S110h33m48s117h53m42s276h24m43s
2017.1.01214.S06h46m09s06h11m58s05h31m04s
E2E6.1.00080.S35h22m22s35h16m56s29h31m27s
E2E6.1.00092.S126h30m37s130h18m17s53h35m03s


  • No labels