ARCHIVED: Compiling programs on Quarry at IU

This content has been archived, and is no longer maintained by Indiana University. Information here may no longer be accurate, and links may no longer be available or reliable.

Note: After seven years of production service, Indiana University's Quarry research computing cluster was decommissioned January 30, 2015. IU students, faculty, and staff can request accounts on Karst, IU's newest high-throughput computing cluster; for instructions, see ARCHIVED: Requesting an account. User data on Quarry has not be removed; you can access your old home directory data on Quarry from your account on any other IU research computing resource (see Available access to allocated and short-term storage capacity on IU's research systems). All software modules that were available on Quarry are available on Karst. If you have questions or concerns about Quarry's retirement, or need help, contact the High Performance Systems group.

Following is information about compiling C, C++, Fortran, and MPI programs using the compiler collections available on the Quarry research computing cluster at Indiana University:

On this page:


GNU Compiler Collection (GCC)

Setting up your user environment

The GNU Compiler Collection (GCC) is available on Quarry. To see which versions are currently available, on the command line, enter:

  module avail gcc

To add the default version of the GNU Compiler Collection (GCC) to your user environment on Quarry, enter:

  module load gcc

Alternatively, to add another version (if available), enter the following, replacing version_number with the version you want to use (e.g., 4.7.2):

  module load gcc/version_number

Examples

The GCC package includes compilers for C, C++, and Fortran code:

GNU Compiler Collection
Language Compile command Manual page
C gcc man gcc
C++ g++ man g++
Fortran gfortran man gfortran

Once you've added the appropriate module to your environment on Quarry:

  • To compile the C program myprogram.c to create the executable myprogram.exe, use:
      gcc -o myprogram.exe myprogram.c
  • To compile the C++ program myprogram.cpp to create the executable myprogram.exe, use:
      g++ -o myprogram.exe myprogram.cpp
  • To compile the Fortran program myprogram.f to create the executable myprogram.exe, use:
      gfortran -o myprogram.exe program.f

Compile by default in 64-bit. Add the -m32 switch to your compile command to compile in 32-bit; add the -m64 switch to return to 64-bit.

Optimizing your code

Adding the -O option to your compile command enables various optimization flags that tell the compiler to try to improve your code's performance. A higher -O level indicates higher optimization, but can add compilation time and potentially alter the semantics of your program:

Optimization level Description
-O0
No optimization (this is the default)
-O or -O1
These levels are identical; the compiler tries to reduce code size and execution time, but doesn't perform optimizations that take a lot of compilation time.
-O2
This level turns on all the optimization flags specified by -O1, plus some others that increase both performance and compilation time.
-O3
This turns on additional optimization flags, along with those specified by -O2.

UITS recommends using -O2 or -O3 for production runs. For example, to compile the C program myprogram.c using optimization level 2 to create the executable myprogram.exe, use:

  gcc -O2 -o myprogram.exe myprogram.c

Available libraries and routines

The GSL (GNU Scientific Library) numerical library for C and C++ programs (version 1.15) is also available on Quarry. To add it to your environment, enter:

  module load gsl

The following GNU versions of the FFTW (Fastest Fourier Transforms in the West) routines are also available:

  fftw/gnu/double/3.3.3
  fftw/gnu/quad/3.3.3

To add either version to your environment, you must first load the GNU compilers. Then, use the appropriate module load command:

  module load fftw/gnu/double
  module load fftw/gnu/quad

Documentation

For a complete list of options for gcc and g++, see the gcc manual page:

  man gcc

Note: The g++ compiler accepts mostly the same options as gcc; if an option is useful only with C++, the manual page explanation will explicitly say so.

For gfortran options, see the gfortran manual page:

  man gfortran

GCC development is part of the GNU Project; for documentation, see the GCC website and Using the GNU Compiler Collection.

Intel Compiler Suite

Setting up your user environment

The Intel Compiler Suite is available on Quarry. To see which versions are currently available, on the command line, enter:

  module avail intel

To add the default version to your user environment on Quarry, enter:

  module load intel

Alternatively, to add another version (if available), enter the following, replacing version_number with the specific version number you want to use (e.g., 13.0.1):

  module load intel/version_number

Examples

The intel package includes compilers for C, C++, and Fortran code:

Intel Compiler Suite
Language Compile command Manual page
C icc man icc
C++ icpc man icpc
Fortran ifort man ifort

Once you've added the appropriate module to your environment on Quarry:

  • To compile the C program myprogram.c to create the executable myprogram.exe, enter:
      icc -o myprogram.exe myprogram.c
  • To compile the C++ program myprogram.cpp to create the executable myprogram.exe, enter:
      icpc -o myprogram.exe myprogram.cpp
  • To compile the Fortran program myprogram.f to create the executable myprogram.exe, enter:
      ifort -o myprogram.exe program.f

Compile by default in 64-bit.

Optimizing your code

Adding the -O option to your compile command enables various optimization flags that tell the compiler to try to improve your code's performance. A higher -O level indicates higher optimization, but can add compilation time and potentially alter the semantics of your program:

Optimization level Description
-O1
Optimize for maximum speed, but disable some optimizations that increase code size for a small speed benefit.
-O or -O2
These levels are identical. Optimize for maximum speed. This is the default.
-O3
Optimize for maximum speed and enable more aggressive optimizations that may not improve performance on some programs.
-O0 Disable optimizations.

UITS recommends using -O2 for production runs. For example, to compile the C program myprogram.c using optimization level 2 to create the executable myprogram.exe, enter:

  icc -O2 -o myprogram.exe myprogram.c

You can also try the -fast switch to see if this improves your code's performance.

Available libraries and routines

The following Intel versions of the FFTW (Fastest Fourier Transforms in the West) routines are available on Quarry:

  • fftw/intel/double/3.3.3: Includes routines for shared-memory parallelism with pthreads (libfftw3_threads) or OpenMP threads (libfftw3_omp)
  • fftw/intel/double/2.1.5: Includes MPI routines (libfftw_mpi) for distributed-memory parallelism

To add either version to your environment:

  1. Use the module load command to add your preferred version of the Intel Compiler Suite:
      module load intel/version_number

    Note: To compile MPI code, you also must load the Intel Open MPI wrapper compilers:

      module load openmpi/intel/1.6.3
  2. Use the module load command to add your preferred version of the FFTW routines (replace version_number with the version you want to use):
      module load fftw/intel/double/version_number

For example:

  • To compile an OpenMP application that links the appropriate FFTW routines, add the default versions of the Intel Compiler Suite and FFTW routines:
      [davader@q0141 ~]$ module load intel
      Intel compiler suite version 14.0.2 loaded.
      [davader@q0141 ~]$ module load fftw 
      FFTW (Intel, Double precision) version 3.3.3 loaded.

    Then, to compile your code (e.g., myprogram.f), enter:

      ifort -openmp -o myprogram.exe -lfftw3_omp -lfftw3 -lm myprogram.f
  • To compile an MPI application that links the appropriate FFTW routines, add the Intel Compiler Suite, the Intel MPI wrapper compilers, and the MPI FFTW routines:
      [qgjinn@q0141 ~]$ module load intel
      Intel compiler suite version 14.0.2 loaded.
      [qgjinn@q0141 ~]$ module load openmpi/intel
      OpenMPI libraries (Intel) version 1.6.3 loaded.
      [qgjinn@q0141 ~]$ module load fftw/intel/double/2.1.5
      FFTW (Intel, Double precision) version 2.1.5 loaded.
      This version is for use with legacy packages which require it.
      Please use FFTW3 for current development

    Then, to compile your code (e.g., myprogram.f), enter:

      mpif90 -o myprogram.exe -lfftw3_mpi -lfftw3 -lm myprogram.f

    For more about MPI wrapper compilers on Quarry, see the MPI wrapper compilers section below.

The Numerical Algorithms Group (NAG) Fortran Library and the IMSL Fortran Numerical Library are also available on Quarry. For instructions on adding them to your user environment, see ARCHIVED: On Quarry at IU, how do I add the NAG or IMSL libraries to my user environment?

Documentation

To view a complete list of options:

  • For icc and icpc, see the icc manual page:
      man icc

    Alternatively, enter:

      icc -help | less
  • For ifort options, see the ifort manual page:
      man ifort

    Alternatively, enter:

      ifort -help | less

Development of the Intel Compiler Suite is by Intel Corporation; for further documentation, see the Intel Compilers website.

The Portland Group (PGI) compilers

Setting up your user environment

The Portland Group (PGI) compilers are available on Quarry. To see which versions are currently available, on the command line, enter:

  module avail pgi

To add the default version to your user environment on Quarry, enter:

  module load pgi

Alternatively, to load another version (if available), enter the following, replacing version_number with the specific version number you want to use (e.g., 12.9):

  module load pgi/version_number

Examples

The PGI package includes compilers for C, C++, and Fortran code:

Portland Group (PGI) compilers
Language Compile command Manual page
C pgcc man pgcc
C++ pgCC man pgCC
Fortran 77
Fortran 90/95
pgf77
pgfortran
man pgf77
man pgfortran

Once you've added the appropriate module to your environment on Quarry:

  • To compile the C program myprogram.c to create the executable myprogram.exe, enter:
      pgcc -o myprogram.exe  myprogram.c
  • To compile the C++ program myprogram.C to create the executable myprogram.exe, enter:
      pgCC -o myprogram.exe myprogram.C
  • To compile the Fortran program myprogram.f to create the executable myprogram.exe, enter:
      pgfortran -o myprogram.exe myprogram.f

Compile by default in 64-bit. For 32-bit compilations, add the -tp p7 switch to your compiler command.

Optimizing your code

To compile and link your code without optimization (useful for debugging), add either the -O0 or -g switch (minimal optimization) to your compile command. Using the -O switch without specifying an optimization level is equivalent to specifying -O2 (global optimization). If you exclude the -O<level> and -g switches from your compile command, optimization is set to the default, -O1 (local optimization).

Higher values for -O indicate higher optimization, but have greater potential to alter the semantics of a program.

For example, to compile the C program myprogram.c using optimization level 2 to create the executable myprogram.exe, enter:

  pgcc -O2 -o myprogram.exe  myprogram.c

Documentation

Manual pages are located at:

  /N/soft/rhel6/pgi/12.9/linux86-64/12.9/man/cat1
  /N/soft/rhel6/pgi/13.4/linux86-64/13.4/man/cat1

For online documentation, see The Portland Group's PGI Compiler User's Guide (in PDF format) and PGI Compiler Reference Manual.

MPI wrapper compilers

Open MPI wrapper compilers and libraries for the GNU Compiler Collection (GCC) and the Intel Compiler Suite are available on Quarry. MPICH wrapper compilers and libraries for the Intel compilers are also available. The wrapper compilers add the necessary flags for compiling and linking MPI programs, and then invoke the respective back-end compilers to perform the actual commands.

Setting up your user environment

The MPI libraries are not part of your default user environment on Quarry; you must add the library you want by loading its module:

  • Adding the GCC Open MPI library: To see which versions of the GCC Open MPI library are available on Quarry, use the module avail command; for example:
      [davader@q0141 ~]$ module avail openmpi/gnu
    
      ------- /N/soft/rhel6/modules/quarry/DEVELOPMENT ------
      openmpi/gnu/1.6.3

    To add the GCC Open MPI library to your user environment, on the command line, enter:

      module load openmpi/gnu
  • Adding the Intel Open MPI or MPICH library to your environment: You must add the Intel Compiler Collection to your user environment before adding the corresponding MPICH or Open MPI library; to load the Intel compiler module, enter:
      module load intel

    Once the intel module is loaded, you can add the MPICH or Open MPI library to your user environment. To see which versions are available, use module avail from the command line; for example:

    • MPICH:
        [pamidala@q0141 ~]$ module avail mpich/intel
      
        ------- /N/soft/rhel6/modules/quarry/DEVELOPMENT -------
        mpich/intel/1.2.7p1   mpich/intel/3.0.1
    • Open MPI:
        [pamidala@q0141 ~]$ module avail openmpi/intel
      
        ------- /N/soft/rhel6/modules/quarry/DEVELOPMENT ------
        openmpi/intel/1.6.3

    To load the desired module, from the command line, enter:

    • For Open MPI:
        module load openmpi/intel
    • For MPICH (the default version):
        module load mpich
    • For MPICH1:
        module load mpich/intel/1.2.7p1

Examples

MPI wrapper compilers are available for C, C++, and Fortran codes:

MPI/MPICH wrapper compilers
Language Wrapper compiler Manual page
C mpicc man mpicc
C++ mpic++ man mpic++
Fortran mpifort man mpifort

Once you've added the appropriate modules to your environment on Quarry:

  • To compile and link an MPI program written in C, on the command line, enter:
      mpicc -o myprog.exe myprog.c
  • To compile and link an MPI program written in C++, on the command line, enter:
      mpicxx -o myprog.exe myprog.cxx
  • To compile and link an MPI program written in Fortran 90, on the command line, enter:
      mpif90 -o myprog.exe myprog.f

Documentation

For more on compiling MPI applications, see the Open MPI Project's Compiling MPI applications FAQ. Additionally, see the manual pages for:

Getting help

Support for this system is provided by the UITS High Performance Systems (HPS) and Scientific Applications and Performance Tuning (SciAPT) groups. If you have system-specific questions, contact the HPS group. If you have questions about compilers, programming, scientific/numerical libraries, or debuggers on this system, contact the SciAPT group.

Back to top

This is document avpx in the Knowledge Base.
Last modified on 2023-04-21 16:56:57.