On XSEDE, how are compute jobs charged?
Computational services on the Extreme Science and Engineering Discovery Environment (XSEDE) are allocated and charged in service units (SUs). SUs are defined locally on each system, with conversion factors among systems based on High-Performance LINPACK (HPL) benchmark results. To calculate the conversion of SUs from one XSEDE digital service to the equivalent amount of SUs on another, use the XSEDE SU Conversion Calculator.
Current XSEDE supercomputers have complex multi-core and memory hierarchies. Each resource has a specific configuration that determines the number (N) of cores that can be dedicated to a job without slowing the code (and other user and system codes). Each Service Provider (SP) defines for its system the minimum number of SUs charged for a job running in the default batch queue, calculated as wall-clock runtime multiplied by N. For the minimum charges on XSEDE resources that use this calculation, consult the following table:
| Digital service | Minimum charge in SUs |
|---|---|
| Kraken (NICS) | wall-clock time * 12 cores |
| Blacklight (PSC) | wall-clock time * 16 cores (see additional information below) |
| Stampede (TACC) | normal = wall-clock * cores largemem = wall-clock * cores * 2 |
| Steele (Purdue) | wall-clock time * number of CPUs
requested (see additional information below) |
| Trestles (SDSC) | normal queue = wall-clock time *
32 cores shared queue = wall-clock time (see additional information below) |
| Lonestar (TACC) | wall-clock time * 12 cores (see additional information below) |
Note: The actual charge will depend on the specific requirements of your job (e.g., the mapping of the cores across the machine, or the priority you wish to obtain). Consult each system's user guide for details. If you have questions, contact the XSEDE Help Desk.
Additional information
-
Blacklight (PSC): Usage of
SUs is accounted using a granularity in seconds. For example, a
300-core job that finishes in 30 minutes would be charged 150
units.
Note: To ensure smooth operation on Blacklight, only 14 16-core jobs can run concurrently.
-
Steele (Purdue): Charges are
calculated at [wall-clock time * CPUs requested] unless exclusive access is
requested, in which case the calculation is [wall-clock time used * nodes
requested * 8] (eight being the number of CPUs in Steele's nodes).
Note: Purdue University is retiring its Steele cluster and Condor Pool from service on the Extreme Science and Engineering Discovery Environment (XSEDE). Both systems will be decommissioned July 31, 2013. If your XSEDE allocation on Steele or the Condor Pool is expiring after July 31, contact the XSEDE Help Desk as soon as possible so administrators can help transfer your project to another XSEDE digital service.
-
Trestles (SDSC): For the
normal queue, you are charged for a full node even if you use less
than 32 cores. For the shared queue, you are not charged for a full
node, and the node is shared with other users/jobs.
- Lonestar (TACC): You are charged for the entire blade, even if you use less than 12 cores.
Note: Not all XSEDE allocations are requested in SUs. For example, requests for storage are made directly in terabytes, rather than converted to SUs.
For more about XSEDE compute, advanced visualization, storage, and special purpose systems, see the Resources Overview, Systems Monitor, and User Guides. For scheduled maintenance windows, outages, and other announcements related to XSEDE digital services, see User News.
This document was developed with support from National Science Foundation (NSF) grant OCI-1053575. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.
Last modified on February 19, 2013.
