...
The target (zen.2458098.44615.HH.uvh5.EXTRACT_AUTOS.log) was changed from the .out file to the .log file. This is common in makefiles where the target depends on the result of that target. The .out file is created by the wrapper_*.sh script in the recipe. Honestly, using either the .log or .out file is ok.
The prerequisites contain all the files we want HTCondor to copy to the execution host. hera_calibration_packages.tar.gz is about 250MB and zen.2458098.44615.HH.autos.uvh5 is about 5GB. They are both too big for HTCondor's transfer mechanism and while it works in testing, we will need to either make them smaller or transfer them some other way. http://chtc.cs.wisc.edu/file-avail-largedata
I replaced the absolute path to wrapper_zen.2458098.44615.HH.uvh5.EXTRACT_AUTOS.sh with ./ to cause makeflow to tell HTCondor to transfer this file to the scratch area on the execution host and execute it from that scratch area.
I removed the absolute path from zen.2458098.44615.HH.uvh5.EXTRACT_AUTOS.log so it can be written to the scratch area and HTCondor will copy it back from the execution host.
...
Build python-packages tarball.
...
Build python-packages tarball.
If over 100MB have it installed in CHTC’s squid proxy. Using xz instead of gz compression reduces the file from 240MB to 178MB but increases the time to uncompress it from 65s to 215s
.