...
Can SSA use this with their containers? Have I already looked at this?
checkpointing
This is a crazy idea but what about using checkpointing with SSA's workflow. Right now they have a three-step process: download, process, upload. all of which use lustre. But what if we ran checkpointing after each step? Would this allow the data to be downloaded directlyi to local storage instead of lustre, then processeed, then uploaded. Now that I write it out, I don't see how this is much better than the current process of copying from archive to lustre to local to lustre to local to lustre. Have to think about it more.