What is Moab?

On this page:

• Introduction
• Moab at IU
• Fairshare scheduling
• When to expect your job to start
• Display jobs eligible for scheduling

Introduction

Moab is an advanced job scheduler for use on clusters and supercomputers. It is a highly optimized and configurable tool capable of supporting a large array of scheduling and fairness policies, dynamic priorities, and extensive reservations. Acknowledged by many as one of the most advanced schedulers available, Moab is currently in use at hundreds of leading government, academic, and commercial sites throughout the world. Moab improves the manageability and efficiency of machines ranging from clusters of a few processors to multi-teraflop supercomputers.

Moab at IU

On the Quarry system at Indiana University, Moab serves as the job scheduler for the TORQUE resource manager (also called PBS). TORQUE is based on OpenPBS; if you are familiar with PBS Pro, you'll find much of the syntax the same. On the Big Red system at IU, Moab serves as the job scheduler for the LoadLeveler resource manager.

Once a job has been submitted to one of the TORQUE/Loadleveler queues, it may become eligible for dispatch by Moab. The following commands provide useful information on the status of a queued or running job:

For more about these commands as well as other Moab utilities, see the Moab Workload Manager User's Manual.

Fairshare scheduling

Fairshare scheduling allows historical resource usage to affect job priority decisions. Administrators can set target utilization goals for each user, group, class, or service group. When these utilization goals are exceeded by one usage class, jobs from other usage classes will take precedence over jobs from the offending class.

Currently, the fairshare policy on Quarry and Big Red records usage over the last seven days and decays at a rate of 80% per day. Each usage class (usually a username) has a goal of 5% usage. Anything above that will cause that user's jobs to have a lower scheduling priority.

Use the diagnose -f command to display the fairshare scheduling usage table. The following example shows that users baikgrp and dsheen have exceeded their "fair share" and will be given lower priorities over the next week:

When to expect your job to start

Moab uses the fairshare tables to determine which job will be assigned to the next open processors. The showq command shows the state of submitted jobs. Following is sample output:

The jobs at the top of the "eligible jobs" list will run next if resources are available. Various reservations can prevent jobs from running if they have blocked off resources that waiting jobs would need. You can use the command showres to examine the list of reservations.

To find the estimated start time of a particular job, try:

If a job already has a node or nodes reserved, showstart returns the start time of the reservation. In all other cases, showstart returns the earliest possible start time, assuming that the job in question is the highest priority job. In most cases a job will not be the highest priority, so showstart is only an estimate of the start time.

Display jobs eligible for scheduling

Use the showq -i command to view the priority of all jobs eligible for scheduling. Note that any job with an asterisk ( * ) appended to the jobid already has a reservation. Thus, the output of showstart for these jobs will be as accurate as possible.

Comments/Questions/Corrections