At IU, how do I transfer files larger than 500 GB to the SDA?

If you need to transfer a single file larger than 500 GB, you can use HSI, FTP, or GridFTP as usual without any special options. If you need to transfer a large amount of data using a pipe (for example, when using tar to aggregate the data), you must inform the system that the resulting file may be very large. To do this, specify copies=2 coslist=large instead of cos=3 in order to change the class of service to 3 (for large files), for example:

This not only sets the class of service to 3, it also gives a hint to the system that it should allocate as much disk cache space as possible for the file. Please do not use this method unless you are certain that you will be creating files with sizes in the hundreds of gigabytes.

Comments/Questions/Corrections