On the Scholarly Data Archive at IU, what are classes of service, and how do I use them?
Indiana University's HPSS archival storage system, the Scholarly Data Archive (SDA), uses classes of service to determine how data are cached and stored. The class of service (COS) assigned depends on the size of the file, how frequently it is accesses, and how many copies are stored, among other considerations.
SDA classes of service are organized by file size and the number of copies stored. Each COS has its own disk cache and tape pool. Other classes of service may be reserved for internal use. The SDA uses the following classes of service:
|COS||Description||Number of copies||Optimal file size||Maximum file size|
|1||Small files||2||1 - 4 MB||10 GB|
|2||Medium files||2||4 - 64 MB||40 GB|
|3||Large files||2||64 MB - 1 TB||10 TB|
|4||Large files (parallel transfers)||2||500 GB - 50 TB||50 TB|
|13||Large files||1||64 MB - 1 TB||10 TB|
Classes 1, 2, 3, and 4 are dual-copy, meaning files are stored on two tapes in a separate tape libraries located at IU Bloomington and IUPUI (storing copies at two geographically distant locations provides additional protection from disasters). If the primary tape is damaged and can no longer be read, data can be recovered from copies on the second tape.
By default, SDA accounts are assigned "class 0", and the SDA assigns an appropriate COS for each file depending on its size. Because the SDA picks classes from the beginning of the list, most automatically set classes are dual-copy. If the SDA cannot determine the size when creating a file (e.g., when data are piped to HSI for transfer to SDA), it sets the COS to 1.
Make sure to use a dual-copy COS for important files. If your files
are easily regenerated or have a limited lifetime, or you have master
copies stored elsewhere, use a single-copy COS (e.g.,
13). For example, use COS 13 for intermediate data files,
workstation backups, and copies of tapes or CD-ROMs.
The method you use to set a non-default COS depends on the application you are using; however, you must specify the COS before transferring your files to the SDA (the COS normally does not change after that).
|Application||Set COS to
||Change directory to current directory, prefixed with
|CIFS/Samba||Connect to the small (COS 1), medium (COS 2), or large (COS 3) share, as described in At IU, how do I access the SDA via Samba?|
|SDA web interface||Not possible (always COS 1)|
To see the COS assigned to a file on the SDA, you must access your account using HSI. Once you are connected, change to the directory that contains the file, and then, at the HSI prompt, enter:
ls -U filename
In the above example, replace
filename with the name
of your file, or omit it to see information about all the files in the
directory. For example:
? cd archive ? ls -U r2.tar -rw------- 1 leorgana hpss 1 36834 DISK 17408 Mar 31 16:07 r2.tar ? ls -U /hpss/p/a/pamidala/archive: -rw------- 1 leorgana hpss 1 36834 DISK 17408 Mar 31 16:07 r2.tar -rw------- 1 leorgana hpss 1 36834 DISK 2336 Mar 31 16:07 r2.tar.idx ?
In the above example, the files are assigned COS 1 (the number
displayed to the right of
If you have questions about classes of service on the SDA, or need help, email the UITS Research Storage team.
This is document auli in the Knowledge Base.
Last modified on 2015-09-30 00:00:00.
- Fill out this form to submit your issue to the UITS Support Center.
- Please note that you must be affiliated with Indiana University to receive support.
- All fields are required.