Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Demonstrate that the refactored code has the desired memory footprint effect.  We'll start with the referenced data set and then expand to larger data sets.  
  2. Demonstrate the runtime cost of the refactored code and whether it's a fixed overhead so it's contribution goes to zero for larger data sets or whether the overhead scales with image complexity

For all tests below Record tclean parameters and telemetry data for each of the 3 tclean calls.

Run each standard ALMA imaging pipeline generated data set through the following 3 casa revs.  Record tclean parameters and telemetry data for each of the 3 tclean callsAll tests run with 8 way parallelization and 128GB memory limit.  All tests run within AWS.

ALMA dataset (project)

casa-pipeline-release-5.6.1-8.el7

pipeline rev. 42866

Casa6 version and pipe rev TBD

casa-CAS-9386-45 (CASA 6.1.0.54a9386.dev45)

Pipeline master-v0.1-18-g2de4d78-dirty

2017.1.00717.Scompletenot started
complete
testing (local run only)
2017.1.01214.Scompletenot started
not started
2017.1.00884.Scompletenot startednot started
E2E6.1.00080.Scompletenot startednot started
2017.1.00983.Scompletenot startednot started


The following tests vary memory environment for each data set, all tests using casa-CAS-9386-45 (CASA 6.1.0.54a9386.dev45) refactor code. All tests run within AWS

ALMA dataset (project)

128 GB memory 8 way parallelization

256 GB memory 8 way parallelization

512 GB memory 8 way parallelization


2017.1.00717.Snot startednot startednot started
2017.1.01214.Snot startednot started
not started
2017.1.00884.Snot startednot startednot started
E2E6.1.00080.Snot startednot startednot started
2017.1.00983.Snot startednot startednot started

As a control the following two data sets will be run on NRAO clusters as a check against AWS runs.  

ALMA dataset (project)

128 GB memory 8 way parallelization

256 GB memory 8 way parallelization

512 GB memory 8 way parallelization


E2E6.1.00080.Snot startednot startednot started
2017.1.00983.Snot startednot startednot started