Use HSI to access your SDA account at IU

On this page:


HSI at IU

Important:
Files containing PHI must be encrypted when they are stored (at rest) and when they are transferred between networked systems (in transit). To ensure that files containing PHI are encrypted when they are stored, encrypt them before transferring them to storage. To ensure that files containing PHI remain encrypted during transit, use SFTP/SCP or the IU Globus Web App. For more, see Recommended tools for encrypting data containing HIPAA-regulated PHI.

The Hierarchical Storage Interface (HSI) application provides a convenient command-line interface for securely transferring files to and from High Performance Storage System (HPSS) resources, such as Indiana University's Scholarly Data Archive (SDA).

At IU, HSI is available on:

  • IU research supercomputers: To use HSI on an IU research supercomputer, you must add the HPSS package to your software environment; on the command line, enter:
    module load hpss
    

    You can save your customized user environment so that it loads every time you start a new session; for instructions, see Use modules to manage your software environment on IU research supercomputers.

  • Research Database Complex (RDC): On the RDC, HSI is installed in /usr/local/bin, so you don't have to add it to your user environment.

For use on personal workstations, IU SDA users can download and install HSI version 6.2.0 (bundled with its companion program, HTAR) from the UITS Research Technologies HSI folder in Google at IU My Drive. (You must be signed into your Google at IU account to access this folder; see Access Google at IU.) Bundles are available for 32- and 64-bit Windows, macOS, and Red Hat Enterprise Linux, and for 64-bit Ubuntu Linux.

Note:
  • To connect to the SDA with a local HSI/HTAR client, make sure you have at least version 6.2.0 installed.
  • An SDA Remote Access Exemption is required to connect to the SDA with a local HSI/HTAR client from an off-campus location.

If you encounter a problem or have questions about using HSI to access your SDA account, email the UITS Research Storage team.

HSI commands

HSI commands are similar to SFTP commands and should seem familiar to Linux users. For details about HSI commands and command-line options, see the HSI Reference Manual.

An HSI session might look similar to the following, for example (% is the shell prompt, and ? is the HSI prompt):

% hsi
Principal: jdoe
[jdoe]Password:
Username: jdoe  UID: 11021  CC: 11021 Copies: 1 [hsi.6.2.0.p3 Fri Jul 26 09:49:16 EDT 2019]

? ls
/hpss/j/d/jdoe/:
NPB-ppcc.tar   foobar/        movies/

? du -k
861309  /hpss/j/d/jdoe/
-----------------------
861309  total 1024-byte blocks, 6 Files (881,979,719 bytes)

? put myfile1.dat
put myfile1.dat : /hpss/j/d/jdoe/myfile.dat ( 10485760 bytes,
12283.4 KBS (cos=3))

? cd movies
? get myfile2.mov
Scheduler: retrieving file(s)
get myfile2.mov : /hpss/j/d/jdoe/movies/myfile2.dat
(2010/09/29 08:49:03 10485760 bytes, 16842.8 KBS )

? exit

Note:
HSI's companion program, HTAR, can simplify aggregation of many files into one large file, which is the preferred method of storage in HPSS. For more, see Use HTAR with your SDA account.

Alternative authentication methods

By default, HSI will prompt for login information (known in HSI as the "combo" authentication method). Alternatively, if your binaries are built with the appropriate method, you can enable authentication based on either existing Kerberos credentials (known as the "Kerberos" method) or Kerberos keytabs (known as the "keytab" method) using one of the following options.

Note:
You also can use the following options to set the "combo" authentication method explicitly (this is a common remedy for authentication problems in HSI).
  • On the command line, invoke hsi with the -A option; for example:
    • To enable the "kerberos" authentication method, enter:
      hsi -A kerberos
      
    • To explicitly enable the "combo" authentication method, enter:
      hsi -A combo
      
    • To enable the "keytab" authentication method, you also must include the -k option to specify the path to your keytab file (e.g., /path/to/my_keytab) and the -l option to specify the appropriate login name (e.g., username):
      hsi -A keytab -k /path/to/my_keytab -l username
      
  • Create a private HSI startup file (~/.hsirc) that contains the authmethod parameter. Create this file in your home directory on the system you use to connect to the SDA; do not create or store this file in your SDA home directory. Add the authmethod parameter line as follows:
    • To enable the "kerberos" authentication method, add:
      authmethod = kerberos
      
    • To explicitly enable the "combo" authentication method, add:
      authmethod = combo
      
    • To enable the "keytab" method, your private HSI startup file also must contain the keytab parameter to specify the path to your keytab file (e.g., /path/to/my_keytab) and the principal parameter to specify the appropriate login name (e.g., username):
      authmethod = keytab
      principal = username
      keytab = /path/to/my_keytab 
      
  • Define the HPSS_AUTH_METHOD environment variable; for example:
    • In the csh or tcsh shell:
      • To enable the "kerberos" authentication method, enter:
        setenv HPSS_AUTH_METHOD kerberos
        
      • To explicitly enable the "combo" authentication method, enter:
        setenv HPSS_AUTH_METHOD combo
        
      • To enable the "keytab" authentication method, you also must define the HPSS_KEYTAB_PATH environment variable with the path to your keytab file (e.g., path/to/my_keytab) and the HPSS_PRINCIPAL environment variable with the appropriate login name (e.g., username):
        setenv HPSS_PRINCIPAL username
        setenv HPSS_AUTH_METHOD keytab
        setenv HPSS_KEYTAB_PATH /path/to/my_keytab
        
    • In the ksh or bash shell:
      • To enable the "kerberos" authentication method, enter:
        export HPSS_AUTH_METHOD=kerberos
        
      • To explicitly enable the "combo" authentication method, enter:
        export HPSS_AUTH_METHOD=combo
        
      • To enable the "keytab" authentication method, you also must define the HPSS_KEYTAB_PATH environment variable with the path to your keytab file (for example, path/to/my_keytab) and the HPSS_PRINCIPAL environment variable with the appropriate login name (for example, username):
        export HPSS_PRINCIPAL=username
        export HPSS_AUTH_METHOD=keytab
        export HPSS_KEYTAB_PATH=/path/to/my_keytab
        

This is document avdb in the Knowledge Base.
Last modified on 2021-05-05 15:57:15.

Contact us

For help or to comment, email the UITS Support Center.