Custom Images

Published

April 2, 2020

Modified

November 13, 2023

Abstract

This section demonstrates how to build custom container images. Users can build containers on the submit nodes and store the images on shared storage. However container images can be build anywhere include the local computer, as is illustrated in the second part.

All users can build container images directly on the cluster submit nodes. This capability is provided by the Apptainer Fakeroot Feature ¹ which enables an unprivileged user to run a container with the appearance of running as root. Effectively this mechanism makes it possible for users to install any software within the container, which includes the use of a package management systems from the Linux distribute used for the container image.

Lets consider following simple example which builds a container image for GNU Hello ² using a container definition file ³ show below. Simply speaking the definition uses the official Fedora container images, and installs development tools to build GNU hello from source before Installing it:

BootStrap: docker
From: quay.io/fedora/fedora:latest

%labels
Maintainer John Snow <j.snow@example.org>

%post
version=2.12.1
archive=hello-$version.tar.gz

dnf install -y wget tar gzip gcc make
dnf clean all

wget https://ftp.gnu.org/gnu/hello/$archive
tar xvzf $archive -C /opt
rm $archive
cd /opt/hello-$version

./configure
make
make install

%runscript
/usr/local/bin/hello

Option	Description
`-f`,`--fakeroot`	Work as a fake root user inside a container. The option is automatically implied when doing a build as an unprivileged user.

The definition file is passed as argument to the apptainer build command after providing the name for the container image. Note that by convention Apptainer container images us .sif as suffix to indicate the file-type.

# ...login to a cluster submit-node (non VAE environment)
ssh -J lxlogin.gsi.de virgo.hpc.gsi.de

# create a user specific directory in /tmp
tmp=$(mktemp -d /tmp/$USER-apptainer-XXXXXX)
export APPTAINER_TMPDIR=$tmp
export APPTAINER_CACHEDIR=$tmp

# ...create the container defintiton file
$EDITOR gnu-hello.def

# ...set the path to the container image (on shared storage)
export APPTAINER_CONTAINER=$LUSTRE_HOME/containers/hello.sif
# ...build the container image
apptainer build $APPTAINER_CONTAINER gnu-hello.def

# ...test the functionality of the container image
>>> apptainer run hello-2.12.1.sif                
Hello, world!
>>> apptainer exec hello-2.12.1.sif hello --version | head -n1
hello (GNU Hello) 2.12.1

A much more involved instruction to build containers is available in the Apptainer User Guide ⁴.

Footnotes

Apptainer Fakeroot Feature, Apptainer User Guide
https://apptainer.org/docs/user/main/fakeroot.html ↩︎
GNU Hello Website
https://www.gnu.org/software/hello ↩︎
Definition Files, Apptainer User Guide
https://apptainer.org/docs/user/main/definition_files.html ↩︎
Build a Container, Apptainer User Guide
https://apptainer.org/docs/user/main/build_a_container.html ↩︎

--- title: Custom Images date: 2020/04/02 date-modified: 2023/11/13 abstract: > This section demonstrates how to build custom container images. Users can build containers on the submit nodes and store the images on shared storage. However container images can be build anywhere include the local computer, as is illustrated in the second part. --- All users can build container images directly on the cluster [submit nodes][vQWA1]. This capability is provided by the Apptainer Fakeroot Feature [^iEXIb] which enables an unprivileged user to run a container with the appearance of running as root. Effectively this mechanism makes it possible for users to install any software within the container, which includes the use of a package management systems from the Linux distribute used for the container image. [^iEXIb]: Apptainer Fakeroot Feature, Apptainer User Guide <https://apptainer.org/docs/user/main/fakeroot.html> [vQWA1]: run-time.html#submit-node Lets consider following simple example which builds a container image for GNU Hello [^c6Hhz] using a container definition file [^xHg3l] show below. Simply speaking the definition uses the official Fedora container images, and installs development tools to build GNU hello from source before Installing it: [^c6Hhz]: GNU Hello Website <https://www.gnu.org/software/hello> [^xHg3l]: Definition Files, Apptainer User Guide <https://apptainer.org/docs/user/main/definition_files.html> ```sh BootStrap: docker From: quay.io/fedora/fedora:latest %labels Maintainer John Snow <j.snow@example.org> %post version=2.12.1 archive=hello-$version.tar.gz dnf install -y wget tar gzip gcc make dnf clean all wget https://ftp.gnu.org/gnu/hello/$archive tar xvzf $archive -C /opt rm $archive cd /opt/hello-$version ./configure make make install %runscript /usr/local/bin/hello ``` Option | Description -------|-------------------------- `-f`,`--fakeroot` | Work as a fake root user inside a container. The option is automatically implied when doing a build as an unprivileged user. The definition file is passed as argument to the `apptainer build` command after providing the name for the container image. Note that by convention Apptainer container images us `.sif` as suffix to indicate the file-type. ```sh # ...login to a cluster submit-node (non VAE environment) ssh -J lxlogin.gsi.de virgo.hpc.gsi.de # create a user specific directory in /tmp tmp=$(mktemp -d /tmp/$USER-apptainer-XXXXXX) export APPTAINER_TMPDIR=$tmp export APPTAINER_CACHEDIR=$tmp # ...create the container defintiton file $EDITOR gnu-hello.def # ...set the path to the container image (on shared storage) export APPTAINER_CONTAINER=$LUSTRE_HOME/containers/hello.sif # ...build the container image apptainer build $APPTAINER_CONTAINER gnu-hello.def # ...test the functionality of the container image >>> apptainer run hello-2.12.1.sif Hello, world! >>> apptainer exec hello-2.12.1.sif hello --version | head -n1 hello (GNU Hello) 2.12.1 ``` A much more involved instruction to build containers is available in the Apptainer User Guide [^yrg0w]. [^yrg0w]: Build a Container, Apptainer User Guide <https://apptainer.org/docs/user/main/build_a_container.html>