Indiana University
University Information Technology Services
  
What are archived documents?
Login>>
Login

Login is for authorized groups (e.g., UITS, OVPIT, and TCC) that need access to specialized Knowledge Base documents. Otherwise, simply use the Knowledge Base without logging in.

Close

XSEDE data storage overview

Extreme Science and Engineering Discovery Environment (XSEDE) users can store their data in their home directories on individual compute services, in temporary scratch space or parallel file systems, and in archival mass storage.

Choose where to store your data depending on your needs (e.g., speed, visibility, quotas, backup and purge policies):

  • Home directories: Storage is permanent and backed up regularly, but quotas are relatively small. Home directories are visible to all nodes in the cluster, including the login nodes. Best practices dictates that you move large data sets to mass storage as soon as possible to conserve space on individual compute resources.

  • Scratch space: Use scratch space for temporary storage of very large data sets. Scratch has more space than home directories, but data is purged regularly and not backed up. The amount of storage space available at any time depends on the level of concurrent use by others. Scratch space is visible by all nodes in a cluster, including the login nodes.

  • Parallel file systems: These provide fast access to large sets of data, but data is purged regularly and not backed up. The amount of storage space available at any time depends on the level of concurrent use by others. This space is visible to all nodes in a cluster, including the login nodes.

  • Archival (mass) storage: For long-term storage of large data sets, use archival storage. Access times are normally slower than for other storage options, but a GridFTP front end can increase transfer speeds. This space is accessible from all sites, but backups are the responsibility of the user.

Note: To determine the amount of available space on a scratch or parallel file system, use the df command.

For descriptions of allocable storage systems on the Extreme Science and Engineering Discovery Environment (XSEDE), see the XSEDE Storage page. For availability, see the Storage Systems tab on the Resource Monitor.

This document was developed with support from National Science Foundation (NSF) grant OCI-1053575. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.

This is document avyq in domains all and xsede-all.
Last modified on February 06, 2012.

Comments/Questions/Corrections

Use this form to offer suggestions, corrections, and additions to the Knowledge Base. We welcome your input!

If you are affiliated with Indiana University and would like assistance with a specific computing problem, please use the Ask a Consultant form, or contact your campus Support Center.

Contact Information

Note: We will reply to your comment at this address. If your message concerns a problem receiving email, please enter an alternate email address.