Indiana University
University Information Technology Services
  
What are archived documents?
Login>>
Login

Login is for authorized groups (e.g., UITS, OVPIT, and TCC) that need access to specialized Knowledge Base documents. Otherwise, simply use the Knowledge Base without logging in.

Close

How do I use GridFTP clients with IU's MDSS?

Note: For information about other methods of accessing the MDSS, see At IU, how do I access the MDSS?

On this page:


Connecting to the MDSS

To use a GridFTP client with the Massive Data Storage Service (MDSS) at Indiana University, connect to the following address:

gridftp.archive.iu.teragrid.org

This supports TeraGrid credential certificates only. For more information, see How do I access IU's MDSS/HPSS from my TeraGrid account?

For more information on GridFTP clients, visit the TeraGrid's Data: How to Initiate Transfers: GridFTP Clients page.

Note: IU's MDSS system is not supported by the tgcp tool.

Specifying a class of service

To specify a class of service when creating a file in the MDSS, append two commas, followed by the numeric class of service. For example, to copy big.tar from your local machine into IU's MDSS using uberftp, you would enter:

put big.tar "big.tar,,3"

This will create a file called big.tar in class of service 3. Note that quotes are required with recent versions of UberFTP, as the comma character is also used to separate commands.

Using parallel transfers

IU's MDSS supports high-performance, parallel transfers using class of service 4. This class of service is designed for large files (many gigabytes), high-speed networks (TeraGrid), and parallel transfers (four-way, or multiples of four-way).

The GridFTP server on port 2814 is a four-way GridFTP server, optimized to take advantage of class of service 4. It performs well only with 8MB blocks and four-way or a multiple of four-way striping. For maximum performance, make transfers between the MDSS GridFTP and another, similarly configured GridFTP server using third-party transfers.

Following is an example:

globus-url-copy -rp -stripe -bs 8388608 \ gsiftp://my.local.site/bigfile.tar \ gsiftp://gridftp.archive.iu.teragrid.org:2814/bigfile.tar,,4 globus-url-copy -rp -stripe \ gsiftp://gridftp.archive.iu.teragrid.org:2814/bigfile.tar \ gsiftp://my.local.site/ uberftp> open -P 2814 gridftp.archive.iu.teragrid.org uberftp> lopen my.local.site uberftp> parallel 4 uberftp> blksize 8388608 uberftp> put bigfile.tar "bigfile.tar,,4"

Regular transfers

To perform a regular transfer from Big Red to the MDSS, follow this example (for username jdoe):

globus-url-copy -vb gsiftp://gridftp.bigred.iu.teragrid.org/path/to/myfile \ gsiftp://gridftp.mdss.iu.edu/hpss/j/d/jdoe/myfile

This document was developed with support from the National Science Foundation (NSF) under Grant No. 0503697 to the University of Chicago and subcontracted to Indiana University. Additional support was provided by IU through its participation in the TeraGrid, which is supported by the NSF under Grants No. 0833618, SCI451237, SCI535258, and SCI504075. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.

This is document auxj in domains all and tgrid-all.
Last modified on June 23, 2009.

Comments/Questions/Corrections

Use this form to offer suggestions, corrections, and additions to the Knowledge Base. We welcome your input!

If you are affiliated with Indiana University and would like assistance with a specific computing problem, please use the Ask a Consultant form, or contact your campus Support Center.

Contact Information

Note: We will reply to your comment at this address. If your message concerns a problem receiving email, please enter an alternate email address.