Open questions:
- Transfer mechanism: Documenation Documentation implies that only files with an mtime newer than when the transfer_input_files finished will be transferred back to the submit host. While running a dag, the files in my working directory (which is in both transfer_input_files and transfer_output_files) seem to always have an mtime around the most recent step in the DAG suggesting that the entire working directory is copied from the execution host to the submit host at the end of each DAG step. Perhaps this means the transfer mechanism only looks at the mtime of the files/dirs specified in transfer_output_files and doesn't descend into the directories.
- Flocking: When we flock to CHTC what are the two points that transfer_input_files? Is it our submit host and CHTC's execution host?
...