How do I forward resource limits from my shell to my batch job on Big Red II at IU?

On Big Red II at Indiana University, user resource limits set in the shell (with ulimit in bash, or limit in tcsh) are not automatically forwarded to batch jobs on the Extreme Scalability Mode (ESM) or Cluster Compatibility Mode (CCM) compute nodes.

To enable the forwarding of user resource limits, set the APRUN_XFER_LIMITS runtime environment variable to 1 (one) by adding one of the following lines (depending on the shell) to your job script:

  • In bash, add:
      export APRUN_XFER_LIMITS=1
  • In tcsh, add:
      setenv APRUN_XFER_LIMITS 1

For example:

ESM job (bash shell) CCM job (tcsh shell)
  #PBS -l nodes=1:ppn=32
  #PBS -l walltime=00:30:00
  #PBS -q cpu
  #PBS -j oe
  export APRUN_XFER_LIMITS=1  
  aprun -n 32 ~/my_code
  #PBS -l nodes=1:ppn=16
  #PBS -l walltime=00:30:00
  #PBS -l gres=ccm
  #PBS -j oe  

  module load ccm
  ccmrun ~/my_code

For more about the APRUN_XFER_LIMITS runtime environment variable, see its entries in the aprun and ccmrun manual pages:

  man aprun | less -p"APRUN_XFER_LIMITS"
  man ccmrum | less -p"APRUN_XFER_LIMITS"

For more on running batch jobs on Big Red II, see:

Support for IU research computing systems, software, and services is provided by the Research Technologies division of UITS. To ask a question or get help, contact UITS Research Technologies.

This is document bdvg in the Knowledge Base.
Last modified on 2018-10-10 18:25:38.

Contact us

For help or to comment, email the UITS Support Center.