ARCHIVED: Monitoring LoadLeveler jobs on Big Red at IU

This content has been archived, and is no longer maintained by Indiana University. Information here may no longer be accurate, and links may no longer be available or reliable.

Note: Big Red, originally commissioned in 2006, was retired from service on September 30, 2013. Its replacement, Big Red II, is a hybrid CPU/GPU Cray XE6/XK7 system capable of achieving a theoretical peak performance (Rpeak) of 1 petaFLOPS, making it one of the two fastest university-owned supercomputers in the US. For more, see ARCHIVED: Get started on Big Red II. If you have questions or concerns about the retirement of Big Red, contact the High Performance Systems group.

On this page:

Useful commands for checking job status
Common status codes
Cancelling jobs

Useful commands for checking job status

Following are some useful commands for checking the status of LoadLeveler jobs you're running on Big Red at Indiana University. Most of the examples use the llq command, and listed are only a few of the most useful options for llq. For a complete explanation of all llq options, enter:

  man llq

To determine the current status of all submitted jobs, enter:
```
  llq
```
To limit the listing to only those jobs in a specific queue (or class), enter:
```
  llq -c queuename
```
Replace queuename with the appropriate queue name. To get a description of all Big Red queues, enter:
```
  llclass
```
To see the status of only your jobs, enter:
```
  llq -u username
```
Replace username with your username. To add multiple usernames, separate them with spaces.
To see the status of all your jobs in a certain queue, enter:
```
  llq -c queuename -u username
```
To see brief information about a particular job, enter:
```
  llq jobid
```
Replace jobid with the appropriate job ID. To add multiple job IDs, separate them with spaces.
For detailed information about a job, enter:
```
  llq -l -s jobid
```
If the job status is Idle, Queued, or Deferred, a detailed job evaluation will appear in the last section of the output, which should help you determine why your job is not running.
For further diagnostic information, use the Maui command checkjob -v jobid.
If your job isn't running, find out when it's scheduled using the Maui command showstart jobid.
For the most detailed information about a job, enter:
```
  llq -l -x jobid
```
Note: The -x and -s options are incompatible.
For a detailed list of the statuses for all your jobs, enter:
```
  llq -username -l
```
Note: This will produce a long output if you have many jobs.
To determine the status of all nodes on Big Red, enter:
```
  llstatus
```
This will produce about one thousand lines of output, one for each node on Big Red. For a description of the fields displayed, enter:
```
  man llstatus
```

Common status codes

Following are the more common status codes:

Status code	Meaning
I	Queued, waiting for free nodes
R	Running
C	Completed
CA	Cancelled by user
H	Put on hold by user, or on hold due to invalid requirements

Cancelling jobs

To cancel jobs you have submitted to LoadLeveler, whether they're waiting to run or already running, use the llcancel command:

  llcancel jobid

Replace jobid with the appropriate job ID.

If you want to keep a job in its place in the queue, but prevent it from running, you can put it on hold. Enter:

  llhold jobid

To release it from hold, enter:

  llhold -r jobid