Page for presenting results; description of the tests is presented in the parent page:
Calibration pipeline
Goal: Test the behavior of Tier0 parallelization of calibrator imaging in the calibration pipelline
Table: Runtime of the calibration pipeline vs. CASA version, 8 way parallel runs (unless stated otherwise) and 256 GB memory limit.
Project | casa-pipeline-release-5.6.1-8.el7 | casa-6.1.0-63 | casa-CAS-9386-53 | casa-CAS-9386-53_2MPI |
2017.1.00717.S | 28h51m11s | 32h14m43s | 31h37m52s | 33h09m28s |
2017.1.00750.T | 05h17m12s | 04h52m57s | 04h50m23s | 04h42m11s |
2017.1.00884.S | 09h15m26s | 08h46m16s | 08h43m33s | 08h36m50s |
2017.1.00983.S | 55h41m06s | 51h29m49s | 51h27m05s | 58h05m08s |
2017.1.01214.S | 28h54m29s | 22h41m17s | 22h39m23s | 20h49m49s |
E2E6.1.00080.S | 14h01m18s | 13h14m39s | 13h18m04s | 14h25m03s |
E2E6.1.00092.S | 61h26m59s | 65h46m20s | 66h13m38s | 70h14m20s |
No obvious issues in calibration, slight increase in runtime which is not unexpected given tclean() runtime change. Some things could be investigated but no current plans given competing time demands.
Imaging pipeline
Goal: Demonstrate that the refactored code has the desired memory footprint effect.
Table: Memory footprint of entire pipeline run vs. CASA version, 8 way parallel runs with 128 GB memory limit,
Memory footprint (GB) | |||
Project | casa-pipeline-release-5.6.1-8.el7 | casa-6.1.0-63 | casa-CAS-9386-53 |
2017.1.00717.S | 15.96028519 | 15.35385895 | 17.16140747 |
2017.1.00750.T | 5.300235748 | 3.992595673 | 4.630950928 |
2017.1.00884.S | 48.88618088 | 53.2240715 | 79.35606766 |
2017.1.00983.S | 50.89418411 | 50.69113159 | 59.88224411 |
2017.1.01214.S | 20.72197723 | 20.30682755 | 24.82666397 |
E2E6.1.00080.S | 47.14878082 | 45.94787979 | 49.49978256 |
E2E6.1.00092.S | 23.10336304 | 22.3993721 | 57.35647202 |
Memory footprint is systematically higher for the refactor case. Which is opposite of expected. Potentially this is as an efficiency improvement in chanchunk estimation and not the actual natural unconstrained limit which would suggest that the memory usage would increase as available memory increased.
Goal: Demonstrate the runtime cost of the refactored code and whether it's a fixed overhead so it's contribution goes to zero for larger data sets or whether the overhead scales with image complexity
Table: Runtime of imaging pipeline vs. CASA version, 8 way parallel runs with 128 GB memory limit.
Project | casa-pipeline-release-5.6.1-8.el7 | casa-6.1.0-63 | casa-CAS-9386-53 |
2017.1.00717.S | 18h36m15s | 17h21m13s | 15h43m25s |
2017.1.00750.T | 06h41m32s | 05h30m07s | 02h55m08s |
2017.1.00884.S | 07h17m28s | 06h45m21s | 05h50m07s |
2017.1.00983.S | 110h33m48s | 117h53m42s | 276h24m43s |
2017.1.01214.S | 06h46m09s | 06h11m58s | 05h31m04s |
E2E6.1.00080.S | 35h22m22s | 35h16m56s | 29h31m27s |
E2E6.1.00092.S | 126h30m37s | 130h18m17s | 53h35m03s |