Use the IU Globus Web App to transfer data between your accounts on IU's research computing and storage systems

On this page:


Overview

You can use the IU Globus Web App to transfer data between your accounts on Indiana University's research supercomputers and research storage systems. You also can use it to transfer data between an IU research supercomputer and your personal computer, or use it to transfer data to and from other networked hosts, such as digital services on the Extreme Science and Engineering Discovery Environment (XSEDE).

For example, you can use the IU Globus Web App to:

The IU Globus Web App is based on Globus Online, a high-performance GridFTP service designed to provide researchers a tool for secure data movement between networked hosts (endpoints). The IU Globus Web App automates many of the tasks associated with managing large data transfers, saving you time and delivering better performance than other transfer methods, such as SCP.

Access the IU Globus Web App

Note:
To access the SDA from off campus, UITS recommends protocols such as Globus and SFTP, which provide IU Login and Duo authentication, as well as encryption in transit. Because the HSI/HTAR protocols don't provide the same protections, you will need apply for an off-campus exemption to remotely access the SDA with a local HSI/HTAR client.

To access the IU Globus Web App:

  1. In your web browser, go to the IU Globus Web App .
  2. Under "Use your existing organizational login", use the drop-down list to find and select Indiana University.
  3. Click Continue. Log in with your IU username and passphrase. Confirm your identity with Duo two-step login.

Activate the source and destination endpoints

To set up a data transfer in the IU Globus Web App, select and activate the source and destination endpoints.

Note:
  1. From File Manager, click in the text field next to "Collection" and begin typing an endpoint name; for example, enter IU to automatically reveal a list of available IU endpoints.
  2. Scroll through the list to find the desired endpoint; for example:
    IU resource Endpoint name
    Scholarly Data Archive (SDA) IURT - Scholarly Data Archive
    Data Capacitor II IURT - DC2
    Slate or Slate-Project IURT - Slate
    Note:

    As of October 11, 2020, some IU endpoints have new names:

    • iu#hpfs is now IURT - Slate.
    • iu#dc2 is now IURT - DC2 (You can still use the old name to locate it.)
    • IU - Scholarly Data Archive is now IURT - Scholarly Data Archive. (You can still use the old name to locate it.)
    • IU - Geode is now IURT-Geode. (You can still use the old name to locate it.)
  3. When you find the desired endpoint, click its name to activate it:
    • Activating the IURT - Scholarly Data Archive endpoint places you in the home directory of your account on the SDA.
    • Activating the IURT - DC2 or IURT - Slate endpoint may produce a "permission" denied" error at first. The error will resolve when you click in the "Path" field, enter the appropriate path, as follows, and press Enter or Return:
      • Slate: To activate your Slate account, enter the following path (replace username with the IU username associated with your Slate account):
        /N/slate/username
        
      • Slate-Project: To activate your Slate-Project space, enter the following path (replace project_name with the project name associated with your Slate-Project space):
        /N/project/project_name
        
      • DC2 scratch: To activate your DC2 scratch space, enter the following path (replace username with your IU username):
        /N/dc2/scratch/username
        
      • DC2 project space: To activate your DC2 project space, enter the following path (replace project_name with the name of your project space):
        /N/dc2/projects/project_name
        
      Note:
      The Data Capacitor II (DC2) high-performance file system will be retired from service on December 11, 2020. As of October 11, 2020, the file system is mounted read-only on the IU research supercomputers. Users with DC2 project data should migrate their data to Slate or Slate-Project; for instructions, see Move DC2 project data to Slate or Slate-Project at IU. If you have questions or need help, contact the UITS High Performance File Systems (HPFS) team.
  4. Click in the Transfer or sync to... field, and then repeat the previous steps to find and activate your second endpoint.

With two endpoints activated, you can start transferring files between them.

Transfer files

Important:

Files containing personal health information (PHI) must be encrypted when they are stored (at rest) and when they are transferred between networked systems (in transit). To ensure that files containing PHI are encrypted when they are stored, encrypt them before transferring them. For more, see Recommended tools for encrypting data containing HIPAA-regulated PHI.

Before transferring files containing PHI to an IU endpoint, make sure you understand the information in Your legal responsibilities for protecting data containing protected health information (PHI) when using UITS Research Technologies systems and services.

To transfer files between endpoints:

  1. Use the source endpoint's file browser to find and select one or more files and/or folders to transfer, and then click Transfer or Sync to....
  2. To reveal optional transfer settings, near the bottom of the screen, click Transfer & Sync Options, and then:
    • Optionally, click in the text field next to "Label This Transfer" and enter a short text description, which will be displayed in the log of transfer activity.
    • Check the boxes to select preferred settings:
      • sync - only transfer new or changed files
      • delete files on destination that do not exist on source
      • preserve source file modification times
      • verify file integrity after transfer (This setting is selected by default.)
      • encrypt transfer (If you are transferring files containing PHI, make sure encrypt transfer is selected to ensure that your files remain encrypted during transit. If the IURT - Scholarly Data Archive endpoint is activated, this setting will be selected and cannot be changed.)
  3. Use the destination endpoint's file browser to select the destination directory for the selected files and/or folders.
  4. To start the transfer, below the source file browser, click Start. Above the two endpoints, you should see a "Transfer request submitted successfully" message and a corresponding "Task id" number. You can click the task ID number to see an overview of your transfer and monitor its progress.

Add your personal computer as an endpoint

To use your personal computer as an endpoint:

  1. In the IU Globus Web App , click File Manager, and then, near the top of the screen, click inside the "Collection" field.
  2. Under the "Collection" field, on the Recent tab, click Install Globus Connect Personal.
  3. On the "Create New GCP Endpoint" screen:
    1. Under "Step 1 Name Your Endpoint":
      • Next to "Endpoint Display Name", enter a human-friendly name for your Globus Connect Personal endpoint. UITS recommends combining your IU username with an informal name for your computer, for example:
          username#my_laptop
        
      • If you plan to use the IU Globus Web App to transfer data containing HIPAA-protected personal health information (PHI), select this will be a high assurance endpoint.
        Important:

        At Indiana University, never store files containing sensitive institutional data, especially protected health information (PHI) regulated by the Health Insurance Portability and Accountability Act of 1996 (HIPAA), on your desktop workstation, laptop, USB flash drive, tablet, smartphone, or other mobile device unless the files are properly encrypted on the device, and your senior executive officer or the IU Institutional Review Board (IRB) has given prior written approval. Because PHI must remain encrypted at rest, make sure you are using full-disk encryption on any device that has research data containing PHI on it. If you are not sure, ask your department or school IT Pro for help.

        If you have questions about securing HIPAA-regulated research data at IU, email securemyresearch@iu.edu. SecureMyResearch provides self-service resources and one-on-one consulting to help IU researchers, faculty, and staff meet cybersecurity and compliance requirements for processing, storing, and sharing regulated and unregulated research data; for more, see About SecureMyResearch. To learn more about properly ensuring the safe handling of PHI on UITS systems, see the UITS IT Training video Securing HIPAA Workflows on UITS Systems. To learn about division of responsibilities for securing PHI, see Shared responsibility model for securing PHI on UITS systems.

      • Next to "Associated Identity", make sure your username@iu.edu identity is selected.
    2. Under "Step 2 Generate a Setup Key for Installation", click Generate Setup Key, and then copy the setup key that's generated.
    3. Under "Step 3 Download & Install Globus Connect Personal", click to download the installer for your operating system (macOS, Linux, or Windows).
  4. Run the downloaded installer (you'll need to be logged into your computer as an administrator). When the installation is complete, make sure Run Globus Connect Personal is selected before closing the installer.
  5. When prompted, paste in your Globus Connect Personal setup key, and then click OK.
  6. When prompted to modify local folder access settings, click Yes, Modify Settings to select which local folders will be accessible and/or writable by Globus Connect Personal; then, when you are finished selecting accessible folders, click Save.

Get help

Support for IU research supercomputers, software, and services is provided by the Research Technologies division of UITS. To ask a question or get help, contact UITS Research Technologies.

This is document bdqp in the Knowledge Base.
Last modified on 2020-10-11 07:02:36.

Contact us

For help or to comment, email the UITS Support Center.