Indiana University
University Information Technology Services
  
What are archived documents?
Login>>
Login

Login is for authorized groups (e.g., UITS, OVPIT, and TCC) that need access to specialized Knowledge Base documents. Otherwise, simply use the Knowledge Base without logging in.

Close

At IU, what is Quarry?

On this page:

Introduction

Quarry provides a general-purpose Unix computing environment for academic and instructional use. IU's newest supercomputer, Quarry is an IBM HS21 Bladeserver cluster running Red Hat Linux, with TORQUE (also called PBS) and Moab for job management and SoftEnv to simplify application and environment configuration. Quarry is a seven teraflop system built from Intel processors.

The user environment is generally similar to AVIDD's, except that Quarry uses SoftEnv. When you log into Quarry for the first time, a file named .soft will be created for you, defining system defaults, such as Intel's C/C++ and Fortran compilers. You can add more packages using the Nano editor (similar to Pico); for a list of the possibilities, enter the command softenv .

If you have questions about the system, email the High Performance Systems group. For help with migrating applications or optimizing code, email the High Performance Applications team.

Hardware configuration

Quarry consists of 140 IBM HS21 Blade servers, each containing two Intel Xeon 5335 quad-core processors, 8 or 16GB of memory, a 36 or 73GB locally attached SAS disk for local scratch space, and gigabit Ethernet for system interconnects. The Blade configuration provides electrical power and cooling advantages that make Quarry much more cost effective than AVIDD, while occupying much less space.

Following is information about Quarry's cluster nodes:

  • Compute Nodes (140, HS21 Bladeserver):

    • 2 x 2.0GHz quad-core Intel Xeon 5335 processors
    • 8 or 16GB 667MHz DDR2 ECC
    • 36 or 73GB locally attached SAS disk
    • 250TB GPFS global scratch
    • 340TB Lustre global scratch (Data Capacitor)
    • 340TB Lustre global scratch (Data Capacitor - WAN)
    • 2 x 1GB Ethernet card

  • Image Servers (2, E1350 X3650):

    • 2 x 2.5GHz dual-core PowerPC 970MP processors
    • 8GB 533MHz DDR2 SDRAM
    • 73GB SAS disk (67GB in /scratch)
    • 1 x Myricom M3S-PCIXD-2-I (Lanai XP)

  • Management nodes (3, E1350 X3650):

    • 2 x 1.65GHz dual-core Power5+ processor
    • 8GB 533MHz DDR2 SDRAM
    • 73GB SAS disk
    • 2 x Emulex LP10000 PCI-X/133MHz FC adapters

Networking

Each Blade chassis has six gigabit-Ethernet connections to a Force10 E1200 switch that connects to the other research systems. Fourteen HS21 Blades, each with gigabit-Ethernet adapters, share those six connections.

Storage

The disk for Quarry's 266TB GPFS volume is hosted on 10 DataDirect Networks S2A 9500 storage controllers, each dual-pathed to 5 SAF 4248 chassis. Physical disks are aggregated in an 8+2 RAID configuration; combined with the dual-pathed controllers and active/active GPFS storage hosts, this provides multiple levels of redundancy for Quarry's storage space. Home directories are available via NFSv3 over gigabit Ethernet.

Quarry currently shares the GPFS volume with Big Red and the Research Database Complex (RDC).

Home directory disk space is allocated on the IBM N5500 NAS storage device. The default 10GB disk quota is shared with Big Red and the RDC if you have accounts on those systems.

More information

For more information about Quarry, see:

This is document avju in domain all.
Last modified on November 04, 2009.

Comments/Questions/Corrections

Use this form to offer suggestions, corrections, and additions to the Knowledge Base. We welcome your input!

If you are affiliated with Indiana University and would like assistance with a specific computing problem, please use the Ask a Consultant form, or contact your campus Support Center.

Contact Information

Note: We will reply to your comment at this address. If your message concerns a problem receiving email, please enter an alternate email address.