ARCHIVED: Migration between IBM LoadLeveler and TORQUE

This content has been archived, and is no longer maintained by Indiana University. Information here may no longer be accurate, and links may no longer be available or reliable.

To help users migrate between the TORQUE job submission system (also called PBS, and used by Indiana University's Quarry cluster) and IBM LoadLeveler (used by IU's Big Red cluster), the following tables compare commands, environment variables, and resource specifications for both batch systems.

If you're not familiar with either LoadLeveler or TORQUE and new to Big Red, see ARCHIVED: At IU, what was Big Red?.

Commonly used commands

  LoadLeveler TORQUE
Job submission llsubmit <scriptfile> qsub <scriptfile>
Job deletion llcancel <job_id> qdel <job_id>
Job status (for user) llq -u <username> qstat -u <username>
Extended job status llq -l <job_id> qstat -f <job_id>
Hold a job temporarily llhold <job_id> qhold <job_id>
Resume job on hold llhold -r <job_id> qrls <job_id>
List usable queues llclass qstat -Q
GUI for batch system xloadl xpbs

Environment variables

  LoadLeveler TORQUE
Processor list

Note: For jobs using more than 128 tasks (processors/cores), the value of $LOADL_PROCESSOR_LIST will be set to NULL. If your job uses more than 128 tasks, run:

  /opt/ibmll/LoadL/full/bin/ll_get_machine_list > /tmp/machinelist.$LOADL_STEP_ID

Then use the temporary file (/tmp/machinelist.$LOADL_STEP_ID) created to get the list of machines assigned to your job.

Resource specifications

  LoadLeveler TORQUE
Queue #@ class=<queue> #PBS -q <queue>
Nodes #@ node=<#> #PBS -l nodes=<#>
Processors #@ tasks_per_node=<#> #PBS -l ppn=<#>
Wall clock limit #@ wall_clock_limit=<hh:mm:ss> #PBS -l walltime=<hh:mm:ss>
Standard output file #@ output=<file> #PBS -o <file>
Standard error #@ error=<file> #PBS -e <file>
Copy environment #@ environment=COPY_ALL #PBS -V
Notification event #@ notification=start|error|complete|never|always #PBS -m abe
Email address #@ notify_user=<email> #PBS -M <email>
Job name #@ job_name=<name> #PBS -N <name>
Job restart #@ restart=<yes|no> #PBS -r <y|n>
Job type #@ job_type=<type> N/A
Initial directory #@ initialdir=<directory> N/A
Node usage #@ node_usage=not_shared #PBS -l naccesspolicy=singlejob
Memory requirement #@ requirements=(Memory >= NumKiloBytes) #PBS -l mem=NumBytes

For more, see the LLGuide documents for Big Red and ARCHIVED: Use TORQUE to submit and manage jobs on high performance computing systems

This is document axpz in the Knowledge Base.
Last modified on 2018-01-18 16:06:03.