Mason at Indiana University

Note:
Mason, Indiana University's large memory computer cluster, will be retired on January 1, 2018. For more, see About the Mason retirement.

On this page:


System overview

Mason (mason.indiana.edu) at Indiana University is a large memory computer cluster configured to support data-intensive, high-performance computing tasks for researchers using genome assembly software (particularly software suitable for assembly of data from next-generation sequencers), large-scale phylogenetic software, or other genome analysis applications that require large amounts of computer memory. At IU, Mason accounts are available to IU faculty, postdoctoral fellows, research staff, and students involved in genome research. IU educators providing instruction on genome analysis software, and developers of such software, are also welcome to use Mason. IU has also made Mason available to genome researchers from the National Science Foundation's Extreme Science and Engineering Discovery Environment (XSEDE) project.

Mason consists of 18 Hewlett-Packard (HP) DL580 servers, each containing four Intel Xeon L7555 8-core processors and 512 GB of RAM, and two HP DL360 login nodes, each containing two Intel Xeon E5-2600 processors and 24 GB of RAM. The total RAM in the system is 9 TB. Each server chassis has a 10-gigabit Ethernet connection to the other research systems at IU and the XSEDE network (XSEDENet).

Mason nodes run Red Hat Enterprise Linux (RHEL 6.x). The system uses TORQUE integrated with Moab Workload Manager to coordinate resource management and job scheduling. The Data Capacitor II and Data Capacitor Wide Area Network 2 (DC-WAN2) parallel file systems are mounted for temporary storage of research data. The Modules environment management package on Mason allows users to dynamically customize their shell environments.

Back to top

System access

Although Mason is an IU resource dedicated to genome analysis research, accounts on the cluster are not restricted to IU researchers:

For information about your responsibilities as a user of this resource, see:

Once your IU or NCGAS account is created, you can connect to mason.uits.iu.edu with any SSH2 client. Public key authentication also is permitted, see How do I set up SSH public-key authentication to connect to a remote system?.

XSEDE users should connect to mason.iu.xsede.org via the XSEDE Single Sign-on Login Hub. Alternatively, you can connect using public-key authentication or a MyProxy certificate from your account on another XSEDE digital service.

Back to top

Available software

The National Center for Genome Analysis Support (NCGAS) supports genome analysis software on Mason.

To view all of the packages available on Mason, log into your account, and then, on the command line, enter:

   module avail

Mason users are free to install software in their home directories. Faculty and staff users on Mason can request installation of applications for use by all users on the system by submitting the NCGAS Mason software request form. If students require software packages on Mason, their advisors must request them. For more, see At IU, what is the policy about installing software on Carbonate and Mason?

Back to top

Setting up your user environment

On the research computing resources at Indiana University, the Modules environment management system provides a convenient method for dynamically customizing your software environment.

For more about using Modules to configure your user environment, see On the research computing systems at IU, how do I use Modules to manage my software environment?

Back to top

File systems (storage for IU users)

Note:
Before storing data on this system, make sure you understand the information in the Working with data containing PHI section (below).

For file storage information, see On IU's research systems, how much allocated and short-term storage capacity is available to me?

Back to top

Working with data containing PHI

The Health Insurance Portability and Accountability Act of 1996 (HIPAA) established rules protecting the privacy and security of individually identifiable health information. The HIPAA Privacy Rule and Security Rule set national standards requiring organizations and individuals to implement certain administrative, physical, and technical safeguards to maintain the confidentiality, integrity, and availability of protected health information (PHI).

This system meets certain requirements established in the HIPAA Security Rule that enable its use for research involving data that contain protected health information (PHI). You may use this resource for research involving data that contain PHI only if you institute additional administrative, physical, and technical safeguards that complement those UITS already has in place. For more, see When using UITS Research Technologies systems and services, what are my legal responsibilities for protecting the privacy and security of data containing protected health information (PHI)? If you need help or have questions, contact UITS HIPAA Consulting.

Note:
Although PHI is one type of institutional data classified as Critical at IU, other types of institutional data classified as Critical are not permitted on Research Technologies systems. Except for PHI, the most sensitive classification of institutional data allowed on Research Technologies resources is Restricted. For help determining which institutional data elements classified as Critical are considered PHI, see Which data elements in the classifications of institutional data are considered protected health information (PHI)?
Note:
In accordance with standards for access control mandated by the HIPAA Security Rule, you are not permitted to access data containing protected health information (PHI) using a group (or departmental) account. To ensure accountability and permit access to authorized users only, IU researchers must use their personal IU username and passphrase for all work involving PHI.

Back to top

Queue information

The BATCH queue is the default, general-purpose queue on Mason. The default walltime is one hour; the maximum limit is two weeks. If your job requires more than two weeks of walltime, email the High Performance Systems group for assistance.

Note:
To best meet the needs of all research projects affiliated with Indiana University, the High Performance Systems (HPS) team administers the batch job queues on UITS Research Technologies supercomputers using resource management and job scheduling policies that optimize the overall efficiency and performance of workloads on those systems. If the structure or configuration of the batch queues on any of IU's supercomputing systems does not meet the needs of your research project, fill out and submit the Research Technologies Ask RT for Help form (for "Select a group to contact", select High Performance Systems).

Back to top

Running jobs on Mason

IU's research computing clusters use the TORQUE resource manager (based on OpenPBS) and the Moab Workload Manager to manage and schedule batch jobs. Moab uses fairshare scheduling to track usage and prioritize jobs.

CPU/Memory limits and batch jobs

User processes on the login nodes are limited to 20 minutes of CPU time. Processes on the login nodes that run longer than 20 minutes are terminated automatically (without warning). If your application requires more than 20 minutes of CPU time, submit a batch job or an interactive session using the TORQUE qsub command.

Because of this limit:

  • When running Java programs, add the -Xmx parameter (values must be multiples of 1,024 greater than 2 MB) on the command line to specify the Java Virtual Machine (JVM) maximum heap size. For example, to run a Java program (e.g., Hello_DeathStar) with a maximum heap size of 640 MB , on the command line, enter:
  •   java -Xmx640m Hello_DeathStar
    
  • Memory-intensive jobs started on the login nodes will be killed almost immediately. Debugging and testing on Mason should be done by submitting a request for an interactive job via the batch system, for example:
      qsub -I -q shared -l nodes=1:ppn=4,vmem=10gb,walltime=4:00:00

    The interactive session will start as soon as the requested resources are available.

Submitting jobs

To submit a job to run on Mason, use the TORQUE qsub command. If the command exits successfully, it will return a job ID, for example:

    [jdoe@Mason]$ qsub job.script
    123456.m1.mason
    [jdoe@Mason]$
    

If you need attribute values different from the defaults, but less than the maximum allowed, specify these either in the job script using TORQUE directives, or on the command line with the -l switch. For example, to submit a job that needs more than the default 60 minutes of walltime, use:

    qsub -l walltime=10:00:00 job.script
    

Jobs on Mason default to a per-job virtual memory resource of 8 MB. So, for example, to submit a job that needs 100 GB of virtual memory, use:

    qsub -l nodes=1:ppn=4,vmem=100gb job.script
    
Note:
Command-line arguments override directives in the job script, and you may specify many attributes on the command line, either as comma-separated options following the -l switch, or each with its own -l switch. The following two commands are equivalent:
    qsub -l nodes=1:ppn=16,vmem=1024mb job.script
    qsub -l nodes=1:ppn=16 -l vmem=1024mb job.script
    

Useful qsub options include:

Option Action
-a <date_time> Execute the job only after specified date and time.
-I Run the job interactively. (Interactive jobs are forced to not re-runnable.)
-m e Mail a job summary report when the job terminates.
-q <queue name> Specify the destination queue for the job. (Not applicable on Mason.)
-r [y|n] Declare whether the job is re-runnable. Use the argument n if the job is not re-runnable. The default value is y (re-runnable).
-V Export all environment variables in your current environment to the job.

For more, see the qsub manual page.

Monitoring jobs

To monitor the status of a queued or running job, use the TORQUE qstat command. Useful qstat options include:

Option Action
-a Display all jobs.
-f Write a full status display to standard output.
-n List the nodes allocated to a job.
-r Display jobs that are running.
-u user1@host,user2@host Display jobs owned by specified users.

For more, see the qstat manual page.

Deleting jobs

To delete a queued or running job, use the qdel command.

Occasionally, a node will become unresponsive and unable to respond to the TORQUE server's requests to kill a job. In such cases, try using qdel -W <delay> to override the delay between SIGTERM and SIGKILL signals (for <delay>, specify a value in seconds).

For more, see the qdel manual page.

Back to top

Requesting single user time

Although UITS Research Technologies cannot provide dedicated access to an entire compute system during the course of normal operations, "single user time" is made available by request one day a month during each system's regularly scheduled maintenance window to accommodate IU researchers with tasks requiring dedicated access to an entire compute system. To request such single user time, complete and submit the Research Technologies Ask RT for Help form, requesting to run jobs in single user time on HPS systems. If you have questions, email the HPS team.

Back to top

Acknowledging grant support

Mason itself is not grant-funded, but if your project made use of software or consulting provided by NCGAS, include the following acknowledgment statement in any related scholarly printed works, web pages, talks, online publications, and other presentations:

This research is based upon work supported by the National Science Foundation under Grant Nos. DBI-1458641 and ABI-1062432 to Indiana University. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation, the National Center for Genome Analysis Support, or Indiana University.

Back to top

Support

Support for IU and NCGAS users is provided by the UITS High Performance Systems (HPS) and Scientific Applications and Performance Tuning (SciAPT) teams, and by the National Center for Genome Analysis Support (NCGAS) :

  • If you have a system-specific question about Mason, email the HPS team.
  • If you need help or have a question about programming tools and scientific applications on Mason, email the SciAPT team.
  • If you need help or have a question about bioinformatics or genome analysis software on Mason, or need help installing software packages in your home directory on Mason, email NCGAS.

XSEDE users needing help on Mason should contact the XSEDE Help Desk.

Back to top

This document was developed with support from National Science Foundation (NSF) grants 1053575 and 1548562. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.

This is document bbhh in the Knowledge Base.
Last modified on 2017-08-07 16:44:35.

  • Fill out this form to submit your issue to the UITS Support Center.
  • Please note that you must be affiliated with Indiana University to receive support.
  • All fields are required.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.

  • Fill out this form to submit your comment to the IU Knowledge Base.
  • If you are affiliated with Indiana University and need help with a computing problem, please use the I need help with a computing problem section above, or contact your campus Support Center.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.