Use the IU Globus Web App for SDA and DC2 file transfers

• Overview
• Access IU Globus Web App
• Set up a transfer between two endpoints
  • Endpoint names
  • Activate endpoints
• Transfer files between endpoints
• Add your personal computer as an endpoint
• Getting help

Overview

You can use the IU Globus Web App to transfer data between your accounts on IU research computing and storage systems. You can also use it to move data to an IU research system from a personal computer (or vice versa). You can also use the IU Globus Web App to transfer data to and from other networked hosts, such as digital services on the Extreme Science and Engineering Discovery Environment (XSEDE). For example, you can use the IU Globus Web App to:

• Transfer output data from an application running on an IU research supercomputer to your archival storage space on the Scholarly Data Archive (SDA).
  Important:

  Before storing data on the SDA, make sure you understand the information in Types of sensitive institutional data appropriate for the Scholarly Data Archive at IU.
• Transfer data from your laptop to your Data Capacitor II (DC2) scratch space or DC-WAN2 project space for use with an application running on an IU research supercomputer.
• Transfer your application's output data from your SDA account to your account on an XSEDE visualization system.

The IU Globus Web App is based on Globus Online, a fast, reliable, high-performance Globus service designed to provide researchers a tool for secure data movement between networked hosts (endpoints). Although the IU Globus Web App uses the GridFTP protocol, it automates many of the tasks associated with managing large data transfers, saving you time and delivering better performance than other transfer methods (e.g., SCP).

Access the IU Globus Web App

Set up a transfer between two endpoints

The IU Globus Web App allows you to set up transfers between networked endpoints. To set up a data transfer, you must select and activate the desired endpoints.

Endpoint names

The IU Globus Web App uses Globus Online's naming convention to assign short names to registered endpoints (so users don't have to remember fully qualified domain names). Each endpoint's name includes a hash mark ( # ) and text signifying the name of the host institution, department, group, or individual, and a hostname or a personal computer's nickname. IU resources have endpoint names that start with iu# :

XSEDE digital services have endpoint names that begin with xsede#. To transfer files to or from an XSEDE endpoint, you need an XSEDE allocation.

Activate endpoints

To set up a transfer between two endpoints, on the Transfer Files page, use either transfer window for the source or destination endpoint; when both endpoints are activated, you'll be able to move data from one to the other in either direction.

To select and activate endpoints:

1. In one of the Endpoint fields, begin typing an endpoint name, or to the right of the Endpoint field, click the ... icon. This will reveal a drop-down list of available endpoints.
2. Browse to find the desired endpoint, click its name, and then click Go to activate it.
   Note:

   Activating the iu#dc2 endpoint will place you in the system's top-level directory ( /). To access your personal scratch space, in the "Path" field, enter /~/N/dc2/scratch/username (replace username with your IU username), and then click Go.
3. When activating an IU endpoint (e.g., IU - Scholarly Data Archive), you won't be prompted to log in; the IU Globus Web App will automatically pass your IU credentials to the endpoint. When activating an XSEDE endpoint, you will be prompted to authenticate with your XSEDE-wide login credentials.

Activating the IU - Scholarly Data Archive endpoint places you in your home directory on the SDA. From there, you should be able to see, browse, and manipulate all of your files and subdirectories.

When you have activated endpoints in both transfer windows, you can start transferring files between them.

Transfer files

To transfer files between endpoints:

1. In the source transfer window, select one or more files or folders to transfer.
2. To reveal optional transfer settings, near the bottom, click more options. Check the boxes to select the settings you prefer:
   • only transfer new or changed files
   • delete files on destination that do not exist on source
   • preserve source file modification times
   • verify file integrity after transfer
   • encrypt transfer
   Note:

   The verify file integrity after transfer setting is selected by default.
3. Optionally, in the "Label This Transfer" field, enter a short text description, which will be displayed in the log of transfer activity.
4. To start the transfer, near the top of the destination transfer window, click the highlighted arrow.

To monitor the progress of your transfer, near the top of the "Start Transfer" page, click View Transfer Activity to open the "View Transfers" page. Alternatively, from the top menu bar (or the touch-friendly drop-down menu), select Manage Data, and then View Transfers.

On the "View Transfers" page, you'll see a history of your data transfers. By default, 25 records are displayed per page, with the most recent transfer requests at the top:

• To change the sort order, click the gear-shaped icon next to right of Completion Time or Status.
• To change how many records are displayed per page, use the View Records drop-down list to select 5, 10, 50, or 100.
• To navigate through the list of records, use the arrows at the bottom of the page. Click the single arrows to go one page forward or backward; click the double arrows to skip to the end or beginning of the list.

The "Task Progress" column will display a progress bar for each transfer. While a transfer is in progress, its progress bar will indicate how many files have been successfully transferred (e.g., 0/5, 2/5). When a transfer is complete, its progress bar will turn green and indicate that all requested files have been transferred (e.g., 5/5).

To cancel one or more transfers that are in progress, click the checkbox(es) to select the transfer(s) you want to cancel, and then click Cancel.

To return to the "Start Transfer" page, from the top menu bar (or the touch-friendly drop-down menu), select Manage Data, and then Start Transfer.

Add your personal computer as an endpoint

To use your personal computer as an endpoint:

1. Go to Add Globus Connect Personal Endpoint.
2. Next to "Display Name", enter a human-friendly name for your personal endpoint. UITS recommends combining your IU username with an informal name for your computer, for example:

     username#my_laptop
   

3. Click Generate Setup Key, and then copy the setup key that's generated.
4. Under "Download & Install Globus Connect Personal", click to download the installer for your operating system (macOS, Linux, or Windows).
5. Run the downloaded installer (you'll need to be logged into your computer as an administrator).
6. When prompted, paste in your Globus Connect Personal setup key, and then click OK.
7. When prompted to modify local folder access settings, click Modify Settings to select which local folders will be accessible and/or writable by Globus Connect Personal; then, when you are finished selecting accessible folders, click Save.

Getting help