Indiana University
University Information Technology Services
  
What are archived documents?
Login>>
Login

Login is for authorized groups (e.g., UITS, OVPIT, and TCC) that need access to specialized Knowledge Base documents. Otherwise, simply use the Knowledge Base without logging in.

Close

ARCHIVED: How do I use GridFTP clients with IU's SDA?

Note: For information about other methods of accessing the SDA, see At IU, how do I access the SDA?

On this page:


Connecting to the SDA

To use a GridFTP client with the Indiana University Scholarly Data Archive (SDA) (formerly known as the MDSS), connect to:

gridftp.archive.iu.teragrid.org

This address supports TeraGrid credential certificates only. For more, see ARCHIVED: How do I access IU HPSS from my TeraGrid account?

For more information on GridFTP clients, see Data: How to Initiate Transfers: GridFTP Clients page in the TeraGrid User Support documentation.

Note: The IU SDA is not supported by the tgcp tool.

Specifying a class of service

To specify a class of service when creating a file in the SDA, append two commas, followed by the numeric class of service. For example, to copy big.tar from your local machine using uberftp, enter:

put big.tar "big.tar,,3"

This creates a file called big.tar in class of service 3. Quotes are required with recent versions of UberFTP; the comma character is also used to separate commands.

Using parallel transfers

The SDA supports high-performance, parallel transfers using class of service 4. This class of service is designed for large files (many gigabytes), high-speed networks (TeraGrid), and parallel transfers (four-way, or multiples of four-way).

The GridFTP server on port 2814 is a four-way GridFTP server, optimized to take advantage of class of service 4. It performs well only with 8 MB blocks, and four-way or a multiple of four-way striping. For maximum performance, make transfers between the IU GridFTP server and other similarly configured GridFTP servers using third-party transfers. Following is an example:

globus-url-copy -rp -stripe -bs 8388608 \ gsiftp://my.local.site/bigfile.tar \ gsiftp://gridftp.archive.iu.teragrid.org:2814/bigfile.tar,,4 globus-url-copy -rp -stripe \ gsiftp://gridftp.archive.iu.teragrid.org:2814/bigfile.tar \ gsiftp://my.local.site/ uberftp> open -P 2814 gridftp.archive.iu.teragrid.org uberftp> lopen my.local.site uberftp> parallel 4 uberftp> blksize 8388608 uberftp> put bigfile.tar "bigfile.tar,,4"

Regular transfers

To perform a regular transfer from Big Red to the SDA, follow this example (for username jdoe):

globus-url-copy -vb gsiftp://gridftp.bigred.iu.teragrid.org/path/to/myfile \ gsiftp://gridftp.mdss.iu.edu/hpss/j/d/jdoe/myfile

This document was developed with support from the National Science Foundation (NSF) under Grant No. 0503697 to the University of Chicago and subcontracted to Indiana University. Additional support was provided by IU through its participation in the TeraGrid, which is supported by the NSF under Grants No. 0833618, SCI451237, SCI535258, and SCI504075. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.

This is document auxj in domains all and tgrid-all.
Last modified on July 28, 2011.

Comments/Questions/Corrections

Use this form to offer suggestions, corrections, and additions to the Knowledge Base. We welcome your input!

If you are affiliated with Indiana University and would like assistance with a specific computing problem, please use the Ask a Consultant form, or contact your campus Support Center.

Contact Information

Note: We will reply to your comment at this address. If your message concerns a problem receiving email, please enter an alternate email address.