Use Apptainer on Quartz or Big Red 200 at IU

On this page:

Overview
Apptainer at IU
Apptainer commands
Use Apptainer with Docker containers
Bind mount Slate, Slate-Project, and Slate-Scratch directories
Useful options
Get help

Overview

Apptainer is a container solution that promotes reproducible science by giving researchers the ability to run their scientific applications and workflows in a variety of different Linux-based operating system (OS) environments.

An Apptainer container is an encapsulation of an application and its dependencies (the libraries, packages, and data files it needs for execution), which is saved as a single, distributable image file. The image file can be copied, shared, and launched without modification in most Linux-based OS environments (as long as the destination host has Apptainer installed). When the image launches, the container daemon executes on the host, virtualizes the contained application's OS environment, and then runs the application inside the virtualized environment, allowing you (as a non-root user on a shared HPC resource), to run your application even when it cannot be natively installed or supported on the host due to OS and/or library incompatibilities or conflicts.

Because the Apptainer container runs on the host's kernel, it's able to leverage the host's physical hardware (for example, GPUs and accelerators), interconnects, and file systems, giving the contained application the same performance characteristics as native applications. Apptainer also natively supports HPC resource managers and job schedulers, and features built-in Open MPI support.

Although some aspects of container execution require escalated privileges, all escalated privileges are dropped once the container environment is instantiated. Apptainer also prevents user context escalation within the container. Although root privileges are not needed to run them, Apptainer images can be built, configured, and/or modified only on hosts for which the user has root privileges.

Apptainer was developed under the leadership of Gregory Kurtzer, Linux Cluster Technical Architect for the Berkeley Research Computing High Performance Computing service. For more, see the Apptainer website.

Apptainer at IU

At Indiana University, Apptainer is available on Quartz and Big Red 200. To get starting using Apptainer, add it to your user environment with the following command:

module load apptainer

You can save your customized user environment so that it loads every time you start a new session; for instructions, see Use modules to manage your software environment on IU research supercomputers.

Although Apptainer containers will run on Quartz and Big Red 200, you first should contact the UITS Research Applications and Deep Learning team to check whether they can install your application natively.

If you have an Apptainer container image that you want to use (after ruling out native installation), contact the UITS Research Applications and Deep Learning team for help migrating it to your space on Quartz or Big Red 200. Most likely, you will need to modify your image to add appropriate mount points; you can modify an Apptainer container only on systems for which you have root privileges (not on IU's research supercomputers).

Apptainer commands

Apptainer uses a primary command wrapper called apptainer, several sub-commands, and global and command-level options. To see a list of Apptainer sub-commands and options, on the command line, enter apptainer (without any options).

The general syntax for the apptainer command wrapper is:

apptainer [global options] <sub-command> [sub-command options] <container_path>

Following are some sample Apptainer commands (replace /N/slate/username/my_image.img with the path to your Apptainer container):

To spawn a shell within your container, on the command line, enter:
```
apptainer shell /N/slate/username/my_image.img
```
To see options available for the shell sub-command, on the command line, enter apptainer shell -h. For more, see apptainer shell in the Apptainer User Guide.
To execute a program inside your Apptainer container (for example, /path/inside/container/go_fish), on the command line, enter:
```
apptainer exec /N/slate/username/my_image.img /path/inside/container/go_fish
```
To see options available for the exec sub-command, on the command line, enter apptainer exec -h. For more, see apptainer exec in the Apptainer User Guide.

As mentioned previously, you can modify Apptainer containers only on systems for which you have root privileges (not on Quartz or Big Red 200). Furthermore, by default, all Apptainer containers are read-only. To make your container's file system accessible in writable mode, add the -w option to your Apptainer sub-commands. For example, to spawn a writable shell in your container (for example, my_image.img, on the command line (of the system on which you are root), enter:

apptainer shell -w my_image.img

Use Apptainer with Docker containers

You can use Apptainer's pull sub-command to import a container image directly from Docker Hub without having root or superuser privileges (or Docker) on your host system. For example, to use Apptainer to import the image of the latest long-term support (LTS) version of Ubuntu into the present working directory on your host system, use the following command:

apptainer pull docker://ubuntu:latest

Alternatively, you can use Apptainer's shell sub-command to spawn an interactive shell within a Docker container "on the fly". For example, to shell into the latest LTS Ubuntu image available on Docker Hub, use the following command:

apptainer shell docker://ubuntu:latest

For more about using Apptainer to work with Docker images, see Support for Docker and OCI and apptainer pull in the Apptainer User Guide.

Bind mount Slate, Slate-Project, Slate-Scratch directories

To make directories on Slate, Slate-Project, and Slate-Scratch accessible from within your Apptainer container, you must manually bind mount them by adding the -B (--bind) option to your shell or exec command. The -B option lets you specify the source and destination bind paths outside and inside the container, respectively. If you do not specify the destination path, the mount point inside the container is set automatically to equal the source path. To bind mount multiple directories in a single command, you can specify the -B option multiple times, or use a comma-delimited string of bind path specifications.

For example:

To make your space on Slate available within your container, start the container with the following command (replace username with your IU username and my_container.sif with the path to your container):
```
apptainer shell -B /N/slate/username my_container.sif
```
The above command binds Slate directory /N/slate/username to a mount point of the same name (/N/slate/username) inside the container.
To make your Slate-Scratch directory available within your container from a mount point named, for example, /mnt, start the container with the following command (/mnt does not have to already exist in the container; replace username with your IU username and my_container.sif with the path to your container):
```
apptainer shell -B /N/scratch/username:/mnt my_container.sif
```
The above command binds Slate directory /N/slate/username to the /mnt mount point inside the container (/mnt does not have to already exist in the container).
To bind mount your Slate, Slate-Project, and Slate-Scratch spaces, start the container with the following command replace username with your IU username, projectname with your Slate-Project project name, and my_container.sif with the path to your container):
```
apptainer shell -B /N/slate/username,/N/project/projectname,/N/scratch/username my_container.sif
```
The above command binds Slate directory /N/slate/username, Slate-Project directory /N/project/projectname, and Slate-Scratch directory /N/scratch/username to mount points of the same name inside the container.

Useful options

Following are some Apptainer options you may find useful:

Option Description

--nv If you want to use GPUs, you need to use this option, which enables NVIDIA support. You must be running on a compute node that has GPUs available for this to work.

--cleanenv Create a clean environment for the container. Normally, environment variables are inherited inside the container. This can be convenient but may cause issues if those variables don't work as desired within the container's environment. Particularly, the PATH and LD_LIBRARY_PATH variables may cause incompatible programs and libraries to override those in a container. To disable this behavior, use the --cleanenv option.

Option	Description
`--nv`	If you want to use GPUs, you need to use this option, which enables NVIDIA support. You must be running on a compute node that has GPUs available for this to work.
`--cleanenv`	Create a clean environment for the container. Normally, environment variables are inherited inside the container. This can be convenient but may cause issues if those variables don't work as desired within the container's environment. Particularly, the PATH and LD_LIBRARY_PATH variables may cause incompatible programs and libraries to override those in a container. To disable this behavior, use the `--cleanenv` option.

Get help

If you have questions or need help using Apptainer on an IU research supercomputer, contact the UITS Research Applications and Deep Learning team.