...

I don't like setting not-slurm in the dedicated HTCondor nodes.  I would rather set something like "glidein=true" or "glidein=1000" in the default 99-nrao config file and then remove it for the 99-nrao config in snapshots for dedicated HTCondor nodes.  But that assumes that the base 99-nrao is for NM.  Since we are sharing an image with CV we can't assume that.  Therefore every node, weather dedicated HTCondor or not, will need a 99-nrao in its snapshot area.

SOLUTION

This seems to work.  If I set NRAOGLIDEIN = True on a node, then that node will be chosen last.  You may ask why not just add 10000000 ASTERISK (NRAOGLIDEIN == True).  If I did that I would have to also set it to false on all the other nodes otherwise the negotiator would fail to parse NEGOTIATOR_PRE_JOB_RANK into a float.  So I check if it isn't undefined then check if it is true.  This way you could set NRAOGLIDEIN to False if you wanted.

NEGOTIATOR_PRE_JOB_RANK = (10000000 * Target.Rank) + (1000000 * (RemoteOwner =?= UNDEFINED)) - (100000 * Cpus) - Memory - 10000000 * ((NRAOGLIDEIN =!= UNDEFINED) && (NRAOGLIDEIN == True))

...

Can I have the configured our pilot.sh script to add the NRAOGLIDEIN = True key/value pair to a node when it glides in to HTCondor?  I think that would be sexyist.  That is the simplest and best place to set this I think.

...

In progress

condor_remote_cluster

...