...
- jobs ran 3 to 4 times faster when we copied cfcache from /staging to local disk. I ran a small data set test with full parameters at CHTC that copied cfcache from /staging to local disk and step05 took only 16.7 hours instead of the 56.8 hours it had taken using cfcache on /staging.
- I had a job killed because it exceeded 72 hours even though I set +LongJobs = true in the submit file
- 2385.0 krowe 9/22 20:43 Error from slot1_1@e2008.chtc.wisc.edu: Job failed to complete in 72 hrs
- What are the options to setting up HTCondor to both flock to CHTC and annex to AWS? Multiple submit hosts? Multiple CMs? etc.
- What are the clever solutions to submitting N different DAG jobs with each having different parmeters?
- T10t34
- J220200-003000
- bin, working, data
- J220600-003000
- bin, working, data
- ...
- J220200-003000
- T10t35
- J170743-393000
- bin, working, data
- J171241-383000
- bin, working, data
- ...
- J170743-393000
- ANSWERS:
- INCLUDE syntax for DAGs
- include syntax for submit files
- make a template of files
- use a PRE script that populates things
- usedagdir
- T10t34
...