How do I run interactive jobs on Big Red II at IU?

On this page:


Overview

On Big Red II at Indiana University, you can run applications interactively on the login node from the command line, but if your interactive session will require more than 20 minutes of processor time, you must submit it to the TORQUE resource manager as an interactive job.

Big Red II runs the Cray Linux Environment (CLE), which provides two separate execution environments for running jobs: Extreme Scalability Mode (ESM) and Cluster Compatibility Mode (CCM). Both execution environment require the use of unique commands to properly launch applications on Big Red II's compute nodes. For details, see Execution environments on Big Red II at IU: Extreme Scalability Mode (ESM) and Cluster Compatibility Mode (CCM).

Running interactive ESM jobs

To launch an interactive job in the ESM execution environment, use the TORQUE qsub command with the -I (interactive) flag and other options to indicate the resources your job needs. When the required resources are free, your job will start, and you'll be placed on an aprun node. From here, use the aprun command to start your application on the compute nodes.

Note: Submitting your interactive job with qsub places you on one of Big Red II's aprun service nodes. These nodes have limited resources shared between all users on the system and are not intended for computational use. You must invoke the aprun command from the aprun node command line to launch your application on a compute node in the ESM execution environment.

The following example session shows the user lcalriss running four instances of the Hello_Bespin application on one Big Red II compute node:

  lcalriss@login1:~> qsub -I -l walltime=00:30:00 -l nodes=1:ppn=32 -q cpu
  qsub: waiting for job 9758 to start 
  qsub: job 94758 ready
  Directory: /N/u/lcalriss/BigRed2
  Mon Sep 16 11:03:32 EDT 2013
  lcalriss@aprun2:~>aprun -n 4 Hello_Bespin
  [NID 00888] 2013-09-16 11:27:08 Apid 946943: initiated application termination
  Hello, Bespin! Hello, Bespin! Hello, Bespin! Hello, Bespin! 
  Application 946943 exit codes: 16
  Application 946943 resources: utime ~0s, stime ~0s
  lcalriss@aprun2:~>exit
  logout
  
  qsub: job 94758 completed
  lcalriss@login1:~>

Running interactive CCM jobs

To launch an interactive job in the CCM execution environment, use the TORQUE qsub command with the -I (interactive) option, the -l gres=ccm flag, and other options to indicate the resources your job needs; for example:

  qsub -I -l walltime=00:30:00 -l nodes=1:ppn=32 -l gres=ccm -q cpu

Note: To run interactive X11 applications, you must use the CCM execution environment. To do so, add the -X (X11) option, as well as the -I option and -l gres=ccm flag, to your qsub command. For more, see On my personal computer, how do I use X forwarding to securely run graphical applications installed on IU's research computing systems?

When the required resources are free, your job will start, and you'll be placed on an aprun node. From here, load the ccm module (if your ~/.modules file doesn't already contain the module load ccm command). With the ccm module loaded, enter the ccmlogin command.

Note: Submitting your interactive job with qsub places your job on one of Big Red II's aprun service nodes. These nodes have limited resources shared between all users on the system and are not intended for computational use. The ccmlogin command places you on a compute node (e.g., nid00998), from which you can launch applications to run on the emulated cluster (i.e., CCM execution environment).

The following example session shows the user dartmaul launching the graphical version of matlab on a Big Red II compute node in the CCM execution environment:

  dartmaul@login1:~> qsub -I -X -l walltime=00:30:00 -l nodes=1:ppn=32 -l gres=ccm -q cpu
  qsub: waiting for job 72255 to start 
  qsub: job 72255 ready 
  In CCM JOB: 72255 JID 72255 USER dartmaul GROUP uits 
  Initializing CCM environment, Please Wait 
  CCM Start success, 1 of 1 responses  
  Directory: /N/u/dartmaul/BigRed2 
  Thu Sep 5 10:40:19 EDT 2013
  dartmaul@aprun1:~> module load ccm
  dartmaul@aprun1:~> ccmlogin 
  Warning: Permanently added '[nid00998]:203' (RSA) to the list of known hosts.
  dartmaul@nid00998:~>matlab

Note: To run MATLAB, you first must add module load matlab to your ~/.modules file.

Getting help

Support for IU research computing systems, software, and services is provided by various UITS Research Technologies units. For help, see Research computing support at IU.

Back to top

This is document bdsi in the Knowledge Base.
Last modified on 2017-01-25 09:06:23.

  • Fill out this form to submit your issue to the UITS Support Center.
  • Please note that you must be affiliated with Indiana University to receive support.
  • All fields are required.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.

  • Fill out this form to submit your comment to the IU Knowledge Base.
  • If you are affiliated with Indiana University and need help with a computing problem, please use the I need help with a computing problem section above, or contact your campus Support Center.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.