Page History

...

• DONE: MPI: We have some users that use MPI across multiple nodes.  It would be nice to keep that as an option.

  • Slurm
    • mpich2
      • PATH=${PATH}:/usr/lib64/mpich/bin salloc --ntasks=8 mpiexec mpiexec.sh
      • PATH=${PATH}:/usr/lib64/mpich/bin salloc --nodes=2 mpiexec mpiexec.sh
    • OpenMPI
      • Use #SBATCH to request a number of tasks (cores) and then run mpiexec or mpicasa as normal.
  • HTCondor
    
    • Single-node MPI jobs do work in the Vanilla universe.
    • Multi-node MPI jobs require the creation of a Parallel universe.  But it might be best to tell users that want multi-node MPI to use Slurm and not HTCondor.
      
  • OpenPBS

• DONE: While preemption can be useful in some circumstances I expect we will want it disabled for the foreseeable future.

  • Slurm

    • The default is PreemptType=preempt/none which means Slurm will not preempt jobs.

  • HTCondor
    • Setting a Machine Rank will cause jobs to be preempted https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToConfigPrioritiesForUsers
  • OpenPBS
    • Defaults to preemtion.  Disable it by setting preemptive_sched: false ALL in /var/spool/pbs/sched_priv on the central server.

...