ARCHIVED: At IU, how do I use the serialjob script to submit batch jobs on Quarry?

On Quarry at Indiana University, the serialjob script provides a convenient method for submitting serial (single-processor) programs to the TORQUE resource manager. For complete documentation, see the serialjob manual page (man serialjob) on Quarry.

• Setting up your user environment
• serialjob options
• Using serialjob
• Limits of serialjob

Setting up your user environment

To add the serialjob script to your Quarry user environment, load the local-utils/20130301 module; on the command line, enter:

  module load local-utils/20130301

Options

When you submit a job with serialjob, you may specify how long the job should be allowed to run. The default is to launch one process for up to two hours on Quarry without claiming the name of the queue. The general form of the command is:

  serialjob program [ program-options ] [ -- -softkey softkey -wallclocklimit hh:mm:ss -notify n|a|b|e -jobname name ]

Items in brackets are optional. Replace the example text above as follows:

• For program, substitute the name of the program to submit.
• For program-options, substitute the command-line options you want to pass to the program.
• -- is required to separate the program options from the other options.
• You need to specify the softkey option if you are using certain packages. For example, if you are using meme-4.1.0, the softkey is +meme-4.1.0.
• For wallclocklimit, use the hh:mm:ss format to specify the number of hours the job should be allowed to run.
• For notify, you can choose to get email when the job begins (b), aborts (a), or ends (e), or to get no email (n).
• For jobname, specify a name for your job.

No queue option is needed for the serialjob command on Quarry, because TORQUE automatically selects a queue based on the hours you request to run the job.

Using serialjob

For example, suppose you've written a program called speedster that takes options that specify speed and the name of the file to be processed. To run the program for up to two hours, you would enter:

  serialjob speedster -speed super mydata.dat

To run for up to 48 hours, you would enter:

  serialjob speedster -speed super mydata.dat -- -wallclocklimit 48:00:00

To use the IMa program in the IM package, which has a softkey of +IM-4-21-2008, and run for up to 10 hours, you would enter:

  serialjob IMa -i IMa_test.u -o testwithpath.out -q1 10 -m1 4 -m2 4 -t 10 -u 1 -b 100000 -L 0.5 -p 123456 
  -n 20 -k 20 -f g -g1 0.6 -g2 0.8 .s 31910246 -- -softkey +IM-4-21-2008 -wallclocklimit 1:00:00 .jobname test

If the program that you wish to run is not in your default path, be sure to use the fully qualified path name of the program. When your job runs, the current working directory of your program is the directory from which you ran the serialjob command.

Limits of serialjob

• The serialjob script works only for serial applications.
• serialjob will not work for programs that use a -- (double-dash) argument, because serialjob uses that argument to separate options to your program from options to TORQUE.
• If you need to quote arguments, serialjob handles only double quotes. It cannot provide the protection that is usually afforded by single quotes because the Bourne shell provides no mechanism for escaping characters within strings in single quotes. Thus, serialjob treats single quotes as double quotes.