Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Open questions:

  • It seems that when using DAGs the recommended method is to define variables in the DAG script instead of submit scripts.  This makes sense as it allows for only one file, the DAG script, that needs to be edited to make changes.  But, is there a way to get variables into the DAG script from the command line or environment or an include_file or something?
  • How can I use my own AMIs with condor_annex?  Setting ANNEX_DEFAULT_ODI_IMAGE_ID in user_config didn't seem to work.  Using the -aws-on-demand-ami-id command-line option starts the instance with my AMI but the instance is never added to the pool.
  • We are starting to run 10s of jobs in CHTC requiring 40GB as part of a local DAG.  Are there any options we can set to improve their execution chance.  What memory footprint (32, 20, 16, 8GB) would significantly improve their chances.  Ask Lauren.
  • Do you have any examples of how to launch instances in the spot marked with condor_annex?  I have read the docs and am still lost.
    • James knows how to make this json blob
  • How can we set AWS Tags with condor_annex?  We'd like this to track jobs and set billing tags.
    • Possibly with Launch Templates
    • Possibly use aws-user-data to condor_annex

...