Use classes of service on the Scholarly Data Archive at IU

Important:

Files containing PHI must be encrypted when they are stored (at rest) and when they are transferred between networked systems (in transit). To ensure that files containing PHI are encrypted when they are stored, encrypt them before transferring them to storage. To ensure that files containing PHI remain encrypted during transit, use SFTP/SCP or the IU Globus Web App. For more, see Recommended tools for encrypting data containing HIPAA-regulated PHI.

Indiana University's HPSS archival storage system, the Scholarly Data Archive (SDA), uses classes of service to determine how data are cached and stored. The class of service (COS) assigned depends on the size of the file, how frequently it is accesses, and how many copies are stored, among other considerations.

SDA classes of service are organized by file size and the number of copies stored. Each COS has its own disk cache and tape pool. Other classes of service may be reserved for internal use. The SDA uses the following classes of service:

COS	Description	Number of copies	Optimal file size	Maximum file size
1	Small files	2	1 - 4 MB	10 GB
2	Medium files	2	4 - 64 MB	40 GB
3	Large files	2	64 MB - 1 TB	10 TB
Contact Research Storage team for guidance	Large files (parallel transfers)	2	500 GB - 50 TB	50 TB
13	Large files	1	64 MB - 1 TB	10 TB

Classes 1, 2, and 3 are dual-copy, meaning files are stored on two tapes in a separate tape libraries located at IU Bloomington and IU Indianapolis (storing copies at two geographically distant locations provides additional protection from disasters). If the primary tape is damaged and can no longer be read, data can be recovered from copies on the second tape.

By default, SDA accounts are assigned "class 0", and the SDA assigns an appropriate COS for each file depending on its size. Because the SDA picks classes from the beginning of the list, most automatically set classes are dual-copy. If the SDA cannot determine the size when creating a file (for example, when data are piped to HSI for transfer to SDA), it sets the COS to 1.

Make sure to use a dual-copy COS for important files. If your files are easily regenerated or have a limited lifetime, or you have master copies stored elsewhere, use a single-copy COS; for example, use COS 13 for intermediate data files, workstation backups, and copies of tapes or CD-ROMs.

The method you use to set a non-default COS depends on the application you are using; however, you must specify the COS before transferring your files to the SDA (the COS normally does not change after that).

Application	Set COS to `N`
HSI	`cos=N` for all subsequent transfers, or `put cos=N ...` for just the current transfer
`sftp` or `SCP`	Change directory to current directory, prefixed with `/cosN`, as described in Use SFTP or SCP to access your SDA account at IU
GridFTP	Append `,,N` to the target filename

To see the COS assigned to a file on the SDA

Access your SDA account using HSI; for help, see Use HSI to access your SDA account at IU.
In HSI, change to the directory that contains the file; at the HSI prompt (?), enter (replace archive_dir with the appropriate directory in your SDA space):
```
cd archive_dir
```
At the HSI prompt (?), enter (replace filename with the name of your file):
```
ls -U filename
```
You should see output similar to the following:
```
-rw-------  1 username  hpss   1  36834 DISK   17408 Mar 31 16:07 filename
```
Alternatively, to see information about all the files in the directory, omit the filename; for example:
```
ls -U
```
You should see output similar to the following:
```
/hpss/u/s/username/archive_dir:
-rw-------  1 username  hpss   1  36834 DISK   17408 Mar 31 16:07 filename
-rw-------  1 username  hpss   1  36834 DISK    2336 Mar 31 16:07 filename.idx
```
In the above examples, the files are assigned COS 1 (the number displayed to the right of hpss).

If you have questions about classes of service on the SDA, or need help, email the UITS Research Storage team (store-admin@iu.edu).