ARCHIVED: Use Big Red II data transfer nodes to improve the efficiency of data transfers

This content has been archived, and is no longer maintained by Indiana University. Information here may no longer be accurate, and links may no longer be available or reliable.

On this page:

Note:

Big Red II was retired from service on December 15, 2019; for more, see ARCHIVED: About Big Red II at Indiana University (Retired).

Overview

Big Red II users can improve the efficiency and performance of their data transfers by using the system's dedicated data transfer nodes (DTNs). The DTNs feature hardware components that are tuned to optimize I/O throughput, enabling high-speed data transfers between Research Technologies resources at Indiana University and other remote hosts.

Note:
The DTNs are intended only for running data transfer commands (for example, cp, mv, scp, sftp, and hsi), or data compression and archiving commands (for example, tar, htar, gzip, and cpio). Any other use of a DTN will cause your process to be terminated.

Access the DTNs

Important:
To proceed, you must be connected to either the IU network (on campus) or the IU SSL VPN (off campus).

Big Red II users can use SSH to access the DTNs (dataxfer.bigred2.uits.iu.edu); for example (replace username with your IU username):

  ssh username@dataxfer.bigred2.uits.iu.edu

If prompted, enter your IU passphrase.

After authenticating, you will be placed on the DTN command line; for example:

  username@dataxfer1:~>

However, if you check the present working directory (pwd), you will see that you are in your Big Red II home directory; for example (where username is your IU username):

  username@dataxfer1:~> pwd
  /N/u/username/BigRed2

When logged into the DTNs, you can navigate (cd) to your home directory space on another IU research supercomputer, or your space on the Data Capacitor II (DC2), Slate, or Slate-Project high-performance file system; for example:

  • To navigate to your Carbonate account, on the DTN command line, enter (replace username with your IU username): 
      cd /N/u/username/Carbonate
  • To navigate to your space on DC2, Slate, or Slate-Project, enter the following (replace username with your IU username; replace project_name with your DC2 or Slate-Project project name, whichever is applicable):
    DC2 scratch: cd /N/dc2/scratch/username
    DC2 project: cd /N/dc2/project/project_name
    Slate: cd /N/slate/username
    Slate-Project: cd /N/project/project_name

Transfer data

To transfer data to or from your home directory space on another IU research computing system, or your space on the DC2, Slate, or Slate-Project file system, use the cp or mv command.

To transfer data transfer to or from your account on the IU Scholarly Data Archive (SDA), use HSI, HTAR, or SFTP/SCP; for instructions, see:

Important:
Before storing data on the SDA, make sure you understand the information in Types of sensitive institutional data appropriate for UITS Research Technologies services.

To transfer data to or from a remote host, use SCP or SFTP; for instructions, see:

Get help

Support for IU research computing systems, software, and services is provided by various teams within the Research Technologies division of UITS.

For general questions about research computing at IU, contact UITS Research Technologies.

For more options, see Research computing support at IU.

This is document bfvm in the Knowledge Base.
Last modified on 2019-12-15 07:02:45.