Use HSI to access your SDA account at IU

On this page:


HSI at IU

Important:

Files containing PHI must be encrypted when they are stored (at rest) and when they are transferred between networked systems (in transit). To ensure that files containing PHI are encrypted when they are stored, encrypt them before transferring them to storage. To ensure that files containing PHI remain encrypted during transit, use SFTP/SCP or the IU Globus Web App. For more, see Recommended tools for encrypting data containing HIPAA-regulated PHI.

The Hierarchical Storage Interface (HSI) application provides a convenient command-line interface for securely transferring files to and from High Performance Storage System (HPSS) resources, such as Indiana University's Scholarly Data Archive (SDA).

HSI is available on the IU research supercomputers. To use HSI, add it to your user environment by loading the hpss module; on the command line, enter:

module load hpss

You can save your customized user environment so that it loads every time you start a new session; for instructions, see Use modules to manage your software environment on IU research supercomputers.

For use on personal workstations, IU SDA users can download and install HSI version 8.3.3 (bundled with its companion program, HTAR) from the UITS Research Technologies HSI folder in Google at IU My Drive. (You must be signed into your Google at IU account to access this folder; see Access Google at IU.) Bundles are available for 32- and 64-bit Windows, macOS, and Red Hat Enterprise Linux, and for 64-bit Ubuntu Linux.

Note:
  • To connect to the SDA with a local HSI/HTAR client, make sure you have version 8.3.3 installed.
  • To connect to the SDA with a local HSI/HTAR client from an off-campus network location, you must first email store-admin@iu.edu to request an off-campus exemption.

HSI commands

HSI commands are similar to SFTP commands and should seem familiar to Linux users. For details about HSI commands and command-line options, see the HSI Reference Manual.

An HSI session might look similar to the following, for example (% is the shell prompt, and ? is the HSI prompt):

% hsi
Principal: jdoe
[jdoe]Password:
Username: jdoe  UID: 11021  CC: 11021 Copies: 1 [hsi.6.2.0.p3 Fri Jul 26 09:49:16 EDT 2019]

? ls
/hpss/j/d/jdoe/:
NPB-ppcc.tar   foobar/        movies/

? du -k
861309  /hpss/j/d/jdoe/
-----------------------
861309  total 1024-byte blocks, 6 Files (881,979,719 bytes)

? put myfile1.dat
put myfile1.dat : /hpss/j/d/jdoe/myfile.dat ( 10485760 bytes,
12283.4 KBS (cos=3))

? cd movies
? get myfile2.mov
Scheduler: retrieving file(s)
get myfile2.mov : /hpss/j/d/jdoe/movies/myfile2.dat
(2010/09/29 08:49:03 10485760 bytes, 16842.8 KBS )

? exit
Note:
HSI's companion program, HTAR, can simplify aggregation of many files into one large file, which is the preferred method of storage in HPSS. For more, see Use HTAR with your SDA account.

Alternative authentication methods

By default, HSI will prompt for login information (known in HSI as the "combo" authentication method). Alternatively, if your binaries are built with the appropriate method, you can enable authentication based on either existing Kerberos credentials (known as the "Kerberos" method) or Kerberos keytabs (known as the "keytab" method) using one of the following options.

Note:
You also can use the following options to set the "combo" authentication method explicitly (this is a common remedy for authentication problems in HSI).
  • On the command line, invoke hsi with the -A option; for example:
    • To enable the "kerberos" authentication method, enter:
      hsi -A kerberos
    • To explicitly enable the "combo" authentication method, enter:
      hsi -A combo
    • To enable the "keytab" authentication method, you also must include the -k option to specify the path to your keytab file (for example, /path/to/my_keytab) and the -l option to specify the appropriate login name (for example, username):
      hsi -A keytab -k /path/to/my_keytab -l username
  • Create a private HSI startup file (~/.hsirc) that contains the authmethod parameter. Create this file in your home directory on the system you use to connect to the SDA; do not create or store this file in your SDA home directory. Add the authmethod parameter line as follows:
    • To enable the "kerberos" authentication method, add:
      authmethod = kerberos
    • To explicitly enable the "combo" authentication method, add:
      authmethod = combo
    • To enable the "keytab" method, your private HSI startup file also must contain the keytab parameter to specify the path to your keytab file (for example, /path/to/my_keytab) and the principal parameter to specify the appropriate login name (for example, username):
      authmethod = keytab
      principal = username
      keytab = /path/to/my_keytab
  • Define the HPSS_AUTH_METHOD environment variable; for example:
    • In the csh or tcsh shell:
      • To enable the "kerberos" authentication method, enter:
        setenv HPSS_AUTH_METHOD kerberos
      • To explicitly enable the "combo" authentication method, enter:
        setenv HPSS_AUTH_METHOD combo
      • To enable the "keytab" authentication method, you also must define the HPSS_KEYTAB_PATH environment variable with the path to your keytab file (for example, path/to/my_keytab) and the HPSS_PRINCIPAL environment variable with the appropriate login name (for example, username):
        setenv HPSS_PRINCIPAL username
        setenv HPSS_AUTH_METHOD keytab
        setenv HPSS_KEYTAB_PATH /path/to/my_keytab
    • In the ksh or bash shell:
      • To enable the "kerberos" authentication method, enter:
        export HPSS_AUTH_METHOD=kerberos
      • To explicitly enable the "combo" authentication method, enter:
        export HPSS_AUTH_METHOD=combo
      • To enable the "keytab" authentication method, you also must define the HPSS_KEYTAB_PATH environment variable with the path to your keytab file (for example, path/to/my_keytab) and the HPSS_PRINCIPAL environment variable with the appropriate login name (for example, username):
        export HPSS_PRINCIPAL=username
        export HPSS_AUTH_METHOD=keytab
        export HPSS_KEYTAB_PATH=/path/to/my_keytab

Get help

If you encounter a problem or have questions about using HSI to access your SDA account, email the UITS Research Storage team (store-admin@iu.edu).

Note:

Research Storage currently provides the following HSI/HTAR clients:

  • Clients with vendor support for HPSS 8.3 (see HPSS Collaboration), including:
    • Red Hat Enterprise Linux (RHEL) 7.7
    • Ubuntu 17.10
    • OpenSUSE Tumbleweed
  • IU research supercomputers:
    • Quartz (RHEL 8, best effort)
    • Big Red 200 (SUSE Enterprise Linux Server 15/SUSE 15, best effort)
  • Desktop clients (all under best effort support):
    • CentOS 8 (best effort)
    • Ubuntu 18.04 LTS (best effort)
    • Ubuntu 20.04.3 LTS (best effort)
    • macOS 10.15 (best effort)
    • Windows 10 running HSI for Cygwin (best effort)

This is document avdb in the Knowledge Base.
Last modified on 2023-12-17 07:03:40.