Compiling programs on Karst, Carbonate, or Mason at IU

At Indiana University, the Karst, Carbonate, and Mason research computing clusters are designed to support codes that have extremely large memory requirements. Because such codes typically do not implement a distributed memory model, Karst, Carbonate, and Mason are geared toward a serial or shared-memory (OpenMP) parallel programming paradigm. However, each system also supports distributed memory (MPI) parallelism.

Following is information about compiling serial and parallel C, C++, and Fortran programs using the GNU Compiler Collection (GCC), the Intel Compiler Suite, Portland Group (PGI) compilers, and MPI (Open MPI and MPICH2) wrapper compilers available on Karst, Carbonate, and Mason.

On this page:

Note:
Mason, Indiana University's large memory computer cluster, will be retired on January 1, 2018. For more, see About the Mason retirement.

GNU Compiler Collection (GCC)

The GNU Compiler Collection (GCC) is added to the default user environment on each system. The GCC includes compilers for C, C++, and Fortran codes:

GNU Compiler Collection (GCC)
Language Compile command Manual page
C gcc man gcc
C++ g++ man g++
Fortran gfortran man gfortran

Optimizing your code

Adding the -O option to your compile command enables various optimization flags that tell the compiler to try to improve your code's performance. A higher -O level indicates higher optimization, but can add compilation time and potentially alter the semantics of your program:

UITS recommends using -O2 or -03 for production runs:

Optimization level Description
-O0
No optimization (this is the default)
-O or -O1
These levels are identical; the compiler tries to reduce code size and execution time, but doesn't perform optimizations that take a lot of compilation time.
-O2
This level turns on all the optimization flags specified by -O1, plus some others that increase both performance and compilation time.
-O3
This turns on additional optimization flags, along with those specified by -O2.

Examples

On Karst, Carbonate, and Mason, UITS recommends using the mtune=native and march=native options with the GCC compilers to generate instructions for the machine and CPU type. Compile by default in 64-bit. Add the -m32 switch to your compile command to compile in 32-bit; add the -m64 switch to return to 64-bit.

Following are example commands for compiling optimized serial programs with the GCC compilers:

  • To compile a serial C program (e.g., my_program.c) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   gcc -O2 -mtune=native -march=native my_program.c -o my_program.exe
    
  • To compile a serial C++ program (e.g., my_program.cpp) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   g++ -O2 -mtune=native -march=native my_program.cpp -o my_program.exe
    
  • To compile a serial Fortran program (e.g., my_program.f) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   gfortran -O2 -mtune=native -march=native program.f -o my_program.exe
    

To enable support for shared-memory multi-threaded (OpenMP) parallel programs, you must pass the -fopenmp flag to the compiler; for example:

  gcc -fopenmp my_omp_program.c -o my_omp_program.exe

For distributed-memory examples, see the MPI wrapper compiler section.

Available libraries and routines

The GSL (GNU Scientific Library) numerical library for C and C++ programs is also available on Karst, Carbonate, and Mason. To add it to your user environment, load the appropriate module; on the command line, enter:

  module load gsl

GNU versions of the FFTW (Fastest Fourier Transforms in the West) routines are also available. To add them to your user environment; on the command line, enter:

  module load fftw/gnu

To make permanent changes to your environment, edit your ~/.modules file. For more, see In Modules, how do I save my environment with a .modules file?

Documentation

For more, see the GCC website and Using the GNU Compiler Collection.

Intel Compiler Suite

The Intel Compiler Suite is available on Carbonate and Mason; it includes compilers for C, C++, and Fortran codes:

Intel Compiler Suite
Language Compile command Manual page
C icc man icc
C++ icpc man icpc
Fortran ifort man ifort

Setting up your user environment

To see which versions are currently available, on the command line, enter:

  module avail intel

To add the default version to your user environment, load the intel module; on the command line, enter:

  module load intel

To add a non-default version instead, on the command line, enter (replacing version_number with the desired version number (e.g., 12.1.5):

  module load intel/version_number

To make permanent changes to your environment, edit your ~/.modules file. For more, see In Modules, how do I save my environment with a .modules file?

Optimizing your code

Adding the -O option to your compile command enables various optimization flags that tell the compiler to try to improve your code's performance. A higher -O level indicates higher optimization, but can add compilation time and potentially alter the semantics of your program:

Optimization level Description
-O1
Optimize for maximum speed, but disable some optimizations that increase code size for a small speed benefit.
-O or -O2
These levels are identical. Optimize for maximum speed. This is the default.
-O3
Optimize for maximum speed and enable more aggressive optimizations that may not improve performance on some programs.
-O0 Disable optimizations.

UITS recommends using -O2 for production runs. You can also try the -fast switch to see if this improves your code's performance.

Examples

Following are example commands for compiling optimized serial programs with the Intel compilers (compile by default in 64-bit):

  • To compile a serial C program (e.g., my_program.c) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   icc -O my_program.c -o my_program.exe
    
  • To compile a serial C++ program (e.g., my_program.cpp) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   icpc -O my_program.cpp -o my_program.exe
    
  • To compile a serial Fortran program (e.g., my_program.f) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   ifort -O my_program.f -o my_program.exe
    

To enable support for shared-memory multi-threaded (OpenMP) parallel programs, you must pass the -openmp flag to the compiler; for example:

  icc -O2 -openmp my_omp_program.c -o my_omp_program.exe

For distributed-memory examples, see the MPI wrapper compiler section.

Available libraries and routines

Loading the intel module sets environment variables allowing you to link applications the following Intel performance libraries:

  • Integrated Performance Primitives (IPP)
  • Math Kernel Library (MKL)
  • Threaded Building Blocks (TBB)

Documentation

For more, see the Intel Compilers website.

Portland Group (PGI) compilers

The Portland Group (PGI) compilers are available on Carbonate and Mason. The PGI package includes compilers for C, C++, and Fortran codes:

Portland Group (PGI) compilers
Language Compile command Manual page
C pgcc man pgcc
C++ pgCC man pgCC
Fortran 77
Fortran 90/95
pgf77
pgfortran
man pgf77
man pgfortran

Setting up your user environment

To see which versions are currently available, on the command line, enter:

  module avail pgi

To add the default version to your user environment, on the command line, enter:

  module load pgi

To load a non-default version instead, on the command line, enter (replacing version_number with the desired version number (e.g., 13.4):

  module load pgi/version_number

To make permanent changes to your environment, edit your ~/.modules file. For more, see In Modules, how do I save my environment with a .modules file?

Optimizing your code

Adding the -O option to your compile command enables various optimization flags that tell the compiler to try to improve your code's performance. A higher -O level indicates higher optimization, but can add compilation time and potentially alter the semantics of your program:

  • Using the -O switch without specifying an optimization level is equivalent to specifying -O2 (global optimization).
  • Using -O0 compiles and links your code without optimization, which is useful for debugging.

    Alternatively, to set the optimization level to -O0 and generate symbolic debug information, add the -g switch to your compile command.

  • Excluding the -O and -g switches sets optimization to -O1 (local optimization).

Examples

Compile by default in 64-bit. For 32-bit compilations, add the -tp p7 switch to your compiler command.

Following are example commands for compiling optimized serial programs with the PGI compilers:

  • To compile a serial C program (e.g., my_program.c) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   pgcc -O my_program.c -o my_program.exe
    
  • To compile a serial C++ program (e.g., my_program.C) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   pgCC -O my_program.C -o my_program.exe
    
  • To compile a serial Fortran program (e.g., my_program.f) and create an optimized executable (e.g., my_program.exe), on the command line, enter:
  •   pgfortran -O my_program.f -o my_program.exe
    

To enable support for shared-memory multi-threaded (OpenMP) parallel programs, you must pass the -mp flag to the compiler; for example:

  pgcc -O -mp my_omp_program.c -o my_omp_program.exe

For distributed-memory examples, see the MPI wrapper compiler section.

Documentation

For more, see The Portland Group's PGI Compiler User's Guide (in PDF format) and PGI Compiler Reference Manual.

MPI wrapper compilers

Open MPI and MPICH wrapper compilers are available on Karst, Carbonate, and Mason. The wrapper compilers add the necessary flags for compiling and linking MPI programs, and then invoke the respective back-end compilers to perform the actual commands.

Setting up your user environment

To see which Open MPI wrapper compilers are available, on the command line, enter:

  module avail openmpi

To see which MPICH wrapper compilers are available, on the command line, enter:

  module avail mpich

To add the wrapper compilers to your user environment, load the appropriate module along with any required prerequisite packages.

To make permanent changes to your environment, edit your ~/.modules file. For more, see In Modules, how do I save my environment with a .modules file?

Examples

MPI wrapper compilers are available for C, C++, and Fortran codes:

MPI/MPICH wrapper compilers
Language Wrapper compiler Manual page
C mpicc man mpicc
C++ mpic++ man mpic++
Fortran mpifort man mpifort

Once you've added the appropriate modules to your environment:

  • To compile and link an MPI program written in C (e.g., my_mpi_program.c), on the command line, enter:
  •   mpicc my_mpi_program.c -o my_mpi_program.exe
    
  • To compile and link an MPI program written in C++ (e.g., my_mpi_program.cxx), on the command line, enter:
  •   mpicxx my_mpi_program.cxx -o my_mpi_program.exe
    
  • To compile and link an MPI program written in Fortran (e.g., my_mpi_program.f), on the command line, enter:
  •   mpiffort my_mpi_program.f -o my_mpi_program.exe
    

Documentation

For more, see the Open MPI Project's Compiling MPI applications FAQ.

Getting help

Support for this system is provided by the UITS High Performance Systems (HPS) and Scientific Applications and Performance Tuning (SciAPT) groups. If you have system-specific questions, contact the HPS group. If you have questions about compilers, programming, scientific/numerical libraries, or debuggers on this system, contact the SciAPT group.

This is document beys in the Knowledge Base.
Last modified on 2017-07-24 17:25:18.

  • Fill out this form to submit your issue to the UITS Support Center.
  • Please note that you must be affiliated with Indiana University to receive support.
  • All fields are required.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.

  • Fill out this form to submit your comment to the IU Knowledge Base.
  • If you are affiliated with Indiana University and need help with a computing problem, please use the I need help with a computing problem section above, or contact your campus Support Center.

Please provide your IU email address. If you currently have a problem receiving email at your IU account, enter an alternate email address.