Use Big Red II data transfer nodes to improve the efficiency of data transfers

On this page:


Overview

Big Red II users can improve the efficiency and performance of their data transfers by using the system's dedicated data transfer nodes (DTNs). The DTNs feature hardware components that are tuned to optimize I/O throughput, enabling high-speed data transfers between Research Technologies resources at Indiana University and other remote hosts.

Note:
The DTNs are intended only for running data transfer commands (e.g., cp, mv, scp, sftp, and hsi), or data compression and archiving commands (e.g., tar, htar, gzip, and cpio). Any other use of a DTN will cause your process to be terminated.

Access the DTNs

Important:
To proceed, you must be connected to either the IU network (on campus) or the IU SSL VPN (off campus).

Big Red II users can use SSH to access the DTNs (dataxfer.bigred2.uits.iu.edu); for example (replace username with your IU username):

  ssh username@dataxfer.bigred2.uits.iu.edu

If prompted, enter your IU passphrase.

After authenticating, you will be placed on the DTN command line; for example:

  username@dataxfer1:~>

However, if you check the present working directory (pwd), you will see that you are in your Big Red II home directory; for example (where username is your IU username):

  username@dataxfer1:~> pwd
  /N/u/username/BigRed2

Users with accounts on other IU research computing systems and/or the Data Capacitor II (DC2) high-performance file system can use cd to navigate to those systems while logged into the DTNs; for example:

  • To navigate to your Carbonate account, on the DTN command line, enter (replace username with your IU username):
      cd /N/u/username/Carbonate
    
  • To navigate to your DC2 scratch or project space, enter one of the following (replace username or project_name with your IU username or DC2 project name, whichever is applicable):
      cd /N/dc2/scratch/username
      cd /N/dc2/projects/project_name
    

Transfer data

To transfer data to or from another IU research computing system or your DC2 scratch or project space, use the cp or mv command.

To transfer data transfer to or from your account on the IU Scholarly Data Archive (SDA), use HSI, HTAR, or SFTP/SCP; for instructions, see:

Important:
Before storing data on the SDA, make sure you understand the information in Types of sensitive institutional data appropriate for the Scholarly Data Archive at IU.

To transfer data to or from a remote host, use SCP or SFTP; for instructions, see:

Get help

If you need help, or have a question about the DTNs, contact the UITS High Performance Systems (HPS) team.

Support for IU research computing systems, software, and services is provided by various UITS Research Technologies units. For help, see Research computing support at IU.

This is document bfvm in the Knowledge Base.
Last modified on 2018-02-22 16:09:13.

  • Fill out this form to submit your issue to the UITS Support Center.
  • Please note that you must be affiliated with Indiana University to receive support.
  • All fields are required.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.

  • Fill out this form to submit your comment to the IU Knowledge Base.
  • If you are affiliated with Indiana University and need help with a computing problem, please use the I need help with a computing problem section above, or contact your campus Support Center.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.