What is a batch job?

A batch job is a computer program or set of programs processed in batch mode. This means that a sequence of commands to be executed by the operating system is listed in a file (often called a batch file, command file, or shell script) and submitted for execution as a single unit. The opposite of a batch job is interactive processing, in which a user enters individual commands to be processed immediately.

In many cases, batch jobs accumulate during working hours, and are then executed during the evening or another time the computer is idle. This is often the best way to run programs that place heavy demands on the computer.

On high-performance compute clusters, users typically submit batch jobs to queues, which are classes of compute nodes, managed by a resource manager, such as LoadLeveler and TORQUE (also known as Portable Batch System). Frequently, clusters employ separate job schedulers, such as Moab, to dispatch batch jobs based on the availability of compute resources, job requirements specified by users, and usage policies set by cluster administrators.

At Indiana University, the Mason and Quarry clusters use TORQUE for job submission; Big Red uses LoadLeveler. All three use Moab for scheduling jobs. Rockhopper uses Sun Grid Engine (SGE) for both job submission and scheduling. For more, see Common commands in TORQUE, LoadLeveler, and SGE

To see which resource managers and job schedulers are used on various Extreme Science and Engineering Discovery Environment (XSEDE) clusters, see On XSEDE compute systems, what applications are used for scheduling jobs?

FutureGrid high-performance computing resources use TORQUE for resource management and Moab for job scheduling. For more, see In FutureGrid, how do I work with HPC job services?

This document was developed with support from National Science Foundation (NSF) grant OCI-1053575. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.

This document is based upon work supported in part by National Science Foundation (NSF) grant 0910812 to Indiana University for "FutureGrid: An Experimental, High-Performance Grid Test-bed." FutureGrid project partners include the University of California - San Diego and the San Diego Supercomputer Center (SDSC), the University of Chicago/Argonne National Labs, the University of Florida, Purdue University, the University of Southern California, the University of Texas - Austin, and the Center for Information Services and High Performance Computing at Technische Universität Dresden.