...
Can SSA use this with their containers? Have I already looked at this? No I have not tried this because it doesn't do what I first thought. It spools all input files on the schedd in the spool directory (/var/lib/condor/spool). For SSA, the schedd is mcilroy so this option would be disasterous for them.
checkpointing
This is a crazy idea but what about using checkpointing with SSA's workflow. Right now they have a three-step process: download, process, upload. all of which use lustre. But what if we ran checkpointing after each step? Would this allow the data to be downloaded directlyi to local storage instead of lustre, then processeed, then uploaded. Now that I write it out, I don't see how this is much better than the current process of copying from archive to lustre to local to lustre to local to lustre. Have to think about it more.
...