...
- What limits are there to transfer_input_files? I would sometimes get Failed to transfer files when the number of files was around 10,000
- Is there a way to generate the dag.dot file without having to submit the job?
- Is there a way to start a dag at a given point? E.g. if there are 5 steps in the dag, can you start the job at step 3?
- Can transfer mechanism accept manifest files? E.g. a file that is a list of files?
- Maybe with some tricks. Greg is going to look into it.
- Perhaps a plugin
- Perhaps reading a file into a variable somehow
- What other options are there than holding a job? I find myself not noticing, sometimes for hours, that a job is on hold. Is there a way to make jobs fail instead of get held? I assume others will make this mistake like me.
- I see I can set periodic_remove = (JobStatus == 5) but HTCondor doesn't seem to think that is an error so if I have notification = Error I don't get any email.
- Greg will look into adding a Hold option to notification
- The HTCondor idea of held jobs is that you submitted a large DAG of jobs, one step is missing a file and you would like to put that file in place and continue the job instead of the whole DAG failing and having to be resubmitted. This makes sense but it would be nice to be notified when a job gets held.
...