ARCHIVED: On XSEDE, what is the PSC Data Supercell?

This content has been archived, and is no longer maintained by Indiana University. Information here may no longer be accurate, and links may no longer be available or reliable.

On this page:

Introduction
Transfer methods
Allocations
More information

Introduction

The Data Supercell is a high-capacity, disk-based archival file repository hosted by Pittsburgh Supercomputing Center (PSC) for use on the Extreme Science and Engineering Discovery Environment (XSEDE). Designed to be highly reliable and secure, the Data Supercell provides very high transfer rates, data transfer verification and validation, and low latency.

The Data Supercell is intended for file storage and retrieval only. The file system can accommodate files of any size, but it is especially well suited for storing extremely large (multi-terabyte) datasets. The initial system capacity is 4 PB, but its architecture enables it to scale well beyond this initial deployment.

Transfer methods

The Data Supercell supports a variety of transfer methods, including SFTP and SCP, GridFTP, GSIFTP and GSISCP, and Globus Online:

To use SFTP/SCP or GSIFTP/GSISCP, connect to the PSC high-speed data conduit:

  data.psc.xsede.org

For instructions, see sftp and scp in the PSC Data Supercell User Guide.

To use GridFTP, connect to the Data Supercell's GridFTP server:

  gsiftp://gridftp.psc.xsede.org

For instructions, see globus-url-copy in the PSC Data Supercell User Guide.

To use Globus Online, connect to the Data Supercell's endpoint:

  xsede#pscdata

For instructions, see Globus in the PSC Data Supercell User Guide.

Note: If you need to store a file that is 2 TB or larger, or if you need to store more than 500 GB of data in a single day, email remarks@psc.edu in advance so PSC can make special arrangements to accommodate your transfers.

Allocations

The minimum storage allocation is 10 TB. XSEDE researchers with allocations on a PSC-hosted compute service are automatically given accounts on the Data Supercell. Projects with Research allocations on other XSEDE digital services also may request storage space on the Data Supercell.

To request an allocation on an XSEDE dedicated storage service, follow the process for submitting a Research allocation request through the XSEDE User Portal. On the Resource Request page, select the desired system and indicate the storage capacity (in gigabytes) your project requires. For more, see ARCHIVED: Apply for a new XSEDE allocation

Once you have an account, the path to your home directory on the Data Supercell will be (replace username with your XSEDE username):

  /arc/users/username

More information

For complete user documentation, see PSC Data Supercell User Guide on the XSEDE User Portal. For an explanation of the Data Supercell's advantages over traditional tape-based archives, see the PSC Data Supercell page. If you have questions or need help, contact the XSEDE Help Desk.

For an overview of storage resources available to Extreme Science and Engineering Discovery Environment (XSEDE) researchers, see the Storage page on the XSEDE website. To check the current availability of allocatable storage space, consult the Systems Monitor in the XSEDE User Portal (XUP). For specific system details, including file transfer methods, quotas, and backup and purge policies, access the user guides available on the XSEDE Resources User Guides page in the XUP.

This document was developed with support from National Science Foundation (NSF) grants 1053575 and 1548562. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.