...
- What are the clever solutions to submitting N different DAG jobs with each having different parmeters?
- T10t34
- J220200-003000
- bin, working, data
- J220600-003000
- bin, working, data
- ...
- J220200-003000
- T10t35
- J170743-393000
- bin, working, data
- J171241-383000
- bin, working, data
- ...
- J170743-393000
- T10t34
- It seems that when using DAGs the recommended method is to define variables in the DAG script instead of submit scripts. This makes sense as it allows for only one file, the DAG script, that needs to be edited to make changes. But, is there a way to get variables into the DAG script from the command line or environment or an include_file or something?
- Setting ANNEX_DEFAULT_ODI_IMAGE_ID in user_config didn't seem to work but the -aws-spot-fleet-config-file <json> argument does.
- We are starting to run 10s of jobs in CHTC requiring 40GB as part of a local DAG. Are there any options we can set to improve their execution chance. What memory footprint (32, 20, 16, 8GB) would significantly improve their chances. Ask Lauren.
- How can we set AWS Tags with condor_annex? We'd like this to track jobs and set billing tags.
- Possibly with Launch Templates
- Possibly use aws-user-data to condor_annex
...