At IU, how much disk space is available to me on the research systems?

On this page:

• General information
  • Home directories on the research systems
  • Checking your quota usage
  • Scratch space
• Big Red
• Libra
• Quarry
• Research Database Complex
• Mail spool disk space usage
• MDSS on Big Red, Libra, and Quarry
  • About MDSS
  • Accounts
  • Access
  • Security

General information

Home directories on the research systems

Following is the naming convention for home directory paths on the UITS research systems:

When you log into any of these systems, you can access your home directory from any of the other systems by changing to the parent directory (/N/u/username), and then moving down into the desired system home directory, for example:

User home directories reside on the IBM N5500 NAS storage device with disk quotas of 10GB per user. Note that this quota will be shared by your Big Red, Libra, Quarry, and RDC accounts, if you have accounts on those systems. If you need additional disk space, email the High Performance Systems group. The Massive Data Storage Service (MDSS) is also available if you need more storage.

Back to top

Checking your quota usage

To determine your current disk quota, type quota. You will see output similar to the following:

"Filesystem" is the name of the filesystem where your home directory resides, and "blocks" is the number of 1KB blocks you are currently using. In this example, over 36MB have been used out of a quota of 50MB. The limit signifies the absolute hard limit of disk space you can use during the default grace period (five days). If you exceed your quota and do not reduce your disk space usage to within your quota, after the grace period you won't be able to use more than the quota. If you reach your quota while a job is running, the job will continue to run but no output files will be appended to or created.

Back to top

Scratch space

Scratch disk space is available locally on each node in /tmp and on an additional large scratch file system shared by all nodes within the system.

Scratch file systems are not backed up to tape; data purged from the scratch file systems cannot be recovered. No disk quotas are enforced on these file systems. You are responsible for making your own copies of the data in case of a disk failure or the need to free space.

• Local scratch space: /tmp is a relatively small local file system, and many programs use /tmp by default for temporary work space. You can redirect files from /tmp to other scratch disk space by setting the environment variable TMPDIR. Files in /tmp are automatically deleted once they are 24 hours old.

  If your batch jobs encounter an error when trying to write to or read from the scratch file systems, a subdirectory with your username probably does not exist. Before reading from or writing to scratch file systems, check that your scratch subdirectory still exists, and create the subdirectory if necessary.

• Common scratch space: Common scratch disk space refers to disk space that is available to all users and accessible to multiple systems. It is intended for short-term storage (working space for running programs and temporary storage for program output), not online archival space. Files there are purged automatically based on age to free up space, and are not backed up. For details, see the information below for specific systems.

Back to top

Big Red

• Home directory: Your Big Red home directory disk space is allocated on the IBM N5500 NAS storage device. You have a 10GB disk quota which is shared with Libra, Quarry, and the RDC if you have accounts on those systems.
  
  
• Local scratch: Scratch disk space is available locally on each node in /scratch (67GB). Files in /scratch are automatically deleted once they are 14 days old.
  
  
• Shared scratch: Shared scratch space is accessible in one 266TB GPFS file system, /N/gpfs. The path to your scratch space is /N/gpfs/username. Files older than 60 days are purged automatically to free disk space.
  
  
• Data Capacitor: The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated and its total capacity will fluctuate based on project space requirements.

  The Data Capacitor is mounted on Big Red as /N/dc/... and behaves like any other disk device on that machine. If you have an account on Big Red, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see What is the Data Capacitor? Files in scratch space may be purged after 14 days.

Back to top

Libra

• Home directory: Your Libra home directory disk space is allocated on the IBM N5500 NAS storage device. You have a 10GB disk quota which is shared with Big Red, Quarry, and the RDC if you have accounts on those systems.
  
  
• Local scratch: Scratch disk space is available locally on each node in /tmp (1GB) and an additional large scratch file system named /scr (10GB). Files in /tmp are automatically deleted once they are 24 hours old. Files in /scr are deleted when they are 30 days old. No disk quotas are enforced on /tmp or /scr.
  
  
• Shared scratch: Shared scratch space is accessible in one 266TB GPFS file system, /N/gpfs. The path to your scratch space is /N/gpfs/username. Files older than 60 days are purged automatically to free disk space.

Back to top

Quarry

• Home directory: Your Quarry home directory disk space is allocated on the IBM N5500 NAS storage device. You have a 10GB disk quota which is shared with Big Red, Libra, and the RDC if you have accounts on those systems.
  
  
• Local scratch: Scratch disk space is available locally on each node in /scratch (19GB). Files in /scratch are automatically deleted once they are 14 days old.
  
  
• Shared scratch: Shared scratch space is accessible in one 266TB GPFS file system, /N/gpfs. The path to your scratch space is /N/gpfs/username. Files older than 60 days are purged automatically to free disk space.
  
  
• Data Capacitor: The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated and its total capacity will fluctuate based on project space requirements.

  The Data Capacitor is mounted on Quarry as /N/dc/... and behaves like any other disk device on that machine. If you have an account on Quarry, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see What is the Data Capacitor? Files in scratch space may be purged after 14 days.

Back to top

Research Database Complex

• Home directory: Your RDC home directory disk space is allocated on the IBM N5500 NAS storage device. You have a 10GB disk quota which is shared with Big Red, Libra, and Quarry if you have accounts on those systems.
  
  
• Local scratch: Shared scratch space is accessible in one 266TB GPFS file system, /N/gpfs. The path to your scratch space is /N/gpfs/username. Files older than 60 days are purged automatically to free disk space.

Back to top

Mail spool disk space usage

UITS does not provide a production mail service on any of the research systems. However, PBS and LoadLeveler communicate via email. Mail forwarding should be configured during account creation, but if you are receiving mail on any of the research systems, you may need to set up a .forward file for your account. For help, see How do I forward my mail from a Unix account?

If you do leave email on the research systems, it may be purged. Twice a month /var/spool/mail is checked for mail files larger than 100KB. Files larger than that size are moved to /scr/mail. You may have up to two old mail files in /scr/mail. After a mail file is 30 days old, it will be purged from /scr/mail.

Back to top

MDSS on Big Red, Libra, and Quarry

About MDSS

IU's MDSS provides additional storage for those who need more than 10GB of permanent storage space.

The Massive Data Storage Service (MDSS) is a UITS service that provides extensive capacity (4.2 petabytes) for Indiana University researchers to store and access huge amounts of data. It uses the High Performance Storage System (HPSS), a hierarchical storage management software package. The system is located at both IU Bloomington and IUPUI, providing automatic off-site copies of data for disaster recovery.

For more information, visit the MDSS home page.

Back to top

Accounts

To obtain an MDSS account, see At IU, how can I apply for an account on the MDSS or RFS?

Quotas on the MDSS start at 1TB. If you are a researcher at IU, you can request additional storage if necessary.

Back to top

Access

For information about accessing your data on the MDSS, see:

• At IU, what is MDSSWeb and how do I use it?
• At IU, how do I use FTP to transfer files to and from MDSS?
• At IU, how do I use HSI to access my MDSS account?
• At IU, how do I use SFTP or SCP to access my MDSS account?

To use HSI from Big Red to access MDSS, add the +hpss keyword to your ~/.soft file and run the resoft command. For more about SoftEnv, see On Big Red and Quarry at IU, how can I use SoftEnv to customize my software environment?

Back to top

Security

MDSS security is based on Kerberos. All MDSS services are fully Kerberized.

Neither FTP nor pftp_client encrypt data during transmission. If you plan to store confidential data, e.g., medical records or similar material, on the MDSS, encrypt the data before transferring it to the MDSS. This will have the additional benefit that not even MDSS administrators will be able to read the data, an important consideration if data confidentiality is an issue.

Back to top