Quarry usage policies

On this page:

• Accounts
• Home directories
• Computational resources, CPU limits, and batch jobs
• Mail usage
• Scheduled down time
• Questions and comments

Accounts

Quarry serves multiple communities as a research cluster and as a general purpose Linux environment. Accounts are open to all Indiana University students, faculty, and staff, as well as affiliated researchers.

Home directories

Quarry home directory disk space is allocated on the IBM N5500 NAS storage device. New accounts have a 10GB disk quota which is shared with Big Red, Libra, and the Research Database Complex (RDC), if you have accounts on those systems; see At IU, how much disk space is available to me on the research systems? If you need additional permanent storage space, apply for an account on IU's Massive Data Storage Service (MDSS).

Computational resources, CPU limits, and batch jobs

Effective October 2, 2007, the batch queues are named on Big Red and Quarry to align with TeraGrid partners. This should make it easier for users to transition between TeraGrid resources.

On Quarry:

• defaultq will be known as long.
• fastq will be known as debug.
• osgq will be known as osg.

Wall time limits and node limits per queue will not change. Aliases for the old names will be in place for job submissions.

Please send any comments or suggestions to  researchtechnologies@iu.edu .

Each queue has properties defined in order to maximize job throughput and minimize time spent waiting in queue. Quarry utilizes a PBS routing queue to funnel jobs from the default batch queue into the serial, normal, and long queues. This sorting is done according to the size of the job's resource requests as documented below. By default, your jobs will run in the batch queue unless you specifically submit your jobs to one of the special queues such as debug or osg.

Note: Cluster-wide maximum number of tasks is 336.

SERIAL queue properties

• Nodes: 5 serial + 33 normal + 42 long = 80 blades
• Maximum walltime: 12 hours
• Maximum nodes per job: 1 node
• Maximum cores per job: 8 cores
• Maximum number of jobs per queue: 3500
• Maximum number of jobs per user: 1500
• Direct submission: No

NORMAL queue properties

• Nodes: 33 normal + 42 long = 75 blades
• Maximum walltime: 7 days
• Maximum nodes per job: 2 nodes
• Maximum cores per job: 16 cores
• Maximum number of jobs per queue: None
• Maximum number of jobs per user: None
• Direct submission: No

LONG queue properties

• Nodes: 42 long = 42 blades
• Maximum walltime: 14 days
• Maximum nodes per job: 42 nodes
• Maximum cores per job: 336 cores
• Maximum number of jobs per queue: None
• Maximum number of jobs per user: None
• Direct submission: No

DEBUG queue properties

• Nodes: 4 blades dedicated
• Maximum walltime: 30 min
• Maximum nodes per job: 2 nodes
• Maximum cores per job: 16 cores
• Maximum number of jobs per queue: 1
• Maximum number of jobs per user: 1
• Direct submission: Yes

OSG queue properties (group restricted access)

• Nodes: 16 blades dedicated
• Maximum walltime: 14 days
• Maximum nodes per job: None
• Maximum cores per job: None
• Maximum number of jobs per queue: None
• Maximum number of jobs per user: None
• Direct submission: Yes

Note: Access to the osg queue is restricted and jobs are routed to this queue via Globus. The debug queue is for debugging only; once code has been debugged, you may submit it to the long queue. If you have questions about the job queues, email High Performance Systems.

Mail usage

UITS does not provide a production mail service on the Quarry cluster; however, TORQUE communicates via email. Quarry will send email about your jobs to the address specified in the ~/.forward file in your home directory. (Note the period [ . ] preceding the filename.) By default, this is the email address you provided when you requested your account.

If you'd like to change this email address, enter a command similar to the following, replacing username@host.com with your email address:

Be sure to use a valid email address; if you do not, you will not be notified about the status of your jobs. For help, see How do I forward my mail from a Unix account?

Scheduled down time

The Quarry maintenance window is scheduled for the first Tuesday of each month. A window will be used only if necessary; an announcement regarding the status of maintenance will be posted the preceding Friday morning on both the Quarry Message of the Day and the Scheduled Downtime page.

Questions and comments

The policies described in this document were established with the goal of providing a stable, powerful, and efficient research computing environment for IU faculty, graduate students, and staff. If you have any questions or comments about these policies, UITS welcomes your input; email High Performance Systems.