Software Environment

Modified

August 12, 2024

Abstract

The cluster group provides multiple so called Virtual Application Environments aka VAEs tailored for many use-cases among the GSI user community. The following section provides an overview to this infrastructure and illustrates how to use the Spack HPC package manager to load required application software.

Prerequisites

The following material assumes your familiarity with the Linux command-line interface (CLI) ¹ and a basic understanding of a command line shell ² environment like Bash or Zsh. If this is new to you, we recommend to read the very beginner friendly introduction to this topic Effective Shell ³ by Dave Kerr and The Linux Command Line ⁴ by William Shotts. Shotts book is freely available under the Creative Commons license and has been translated into multiple languages. Furthermore it is recommended to know the basic usage of a text-editor like Vim or Emacs.

Besides the already mentioned prerequisites there are more widely used tools for scientific computing environments. Your selection of these tools may vary depending on your use-case. On common denominator for most software projects in this environment is the use of the Git version control system ⁵. Most scientific software frameworks and libraries require the user to have a understanding of a programming language. In scientific computing Python and C++ are very popular. An excellent introduction into the methodologies and basic tools for scientific computing are available from the Software Carpentry Foundation ⁶. Depending on your scientific field you may find more specific introduction material from The Carpentries Community ⁷.

Virtual Application Environment

The Virgo cluster is build with the incentive to align its software stack with the cluster computing infrastructure build by a majority of scientific research institutions. The operating system platform is build with RockyLinux ⁸. Slurm is used as workload management system and supports the allocation of containers. All containerized applications use Apptainer ⁹ as container run-time engine. Multiple containerized application environments called Virtual Application Environment (VAE) provide a pre-build selection of software. Users access a specific VAE by login to a corresponding submit node:

Virtual Environment	Support Until
VAE’23	2025
VAE’24	2026

It is required to use SSH key-authentication in order to logon to a submit node of the Virgo cluster. Make sure to use a supported default-shell, otherwise SSH login will not work.

» ssh vae23.hpc.gsi.de
Slurm Cluster – Virgo 3.0 Submit Node
Container launched: /cvmfs/vae.gsi.de/vae23/containers/vae23-user_container_20230308T0124.sif

Containers will be transparently started for your login sessions as well as for your jobs executed on the cluster. Note that each login runs in a dedicated container. If you need multiple terminal sessions, which are also known as shells, it is recommend to use a terminal-multiplexer.

VAEs allow decoupling of the user application environments from the underling system software stack used to operate on the compute hardware. This allows great options for cluster administrators to utilize the most optimal software components to support the hardware and to update/upgrade the infrastructure as required. Furthermore VAEs do enable multiple distinct application environments tailored to specific use-cases to coexist on the same infrastructure. These VAEs can be continuously developed with minimal dependencies to the underlying systems software stack. Environment variables describing the VAE container:

Variable	Description
`APPTAINER_CONTAINER`	Absolute path to the container image
`APPTAINER_NAME`	Name of the container image

Spack Packages

Software is made available on the Virgo compute cluster using the Spack ¹⁰ package manager. Spack builds software packages with optimisation for multiple computer microarchitectures ¹¹ like AMD Zen or Intel Broadwell. List available software in a virtual application environment (VAE) using following command:

# list packages
spack find

# details infromation about a package (i.e openmpi)
spack find -v openmpi

This prints all packages with their respective microarchitecture and compiler ¹², e.g. GCC from the GNU Compiler Collection ¹³. Print the location’s absolute path of a package:

spack location -i openmpi target=$(spack arch -t)

# note that following command evaluates to the
# microarchitecture of your current host
spack arch -t

Load a package into the current environment. If multiple versions of a specific package are available, it is required to provide a version for loading the package:

spack load openmpi target=$(spack arch -t)

# load a package including all dependencies
spack load -r openmpi target=$(spack arch -t)

# load a specific version
spack load openmpi@3.1.0 target=$(spack arch -t)

Remove a package from your environment:

# list loaded packages
spack find --loaded

# remove a package
spack unload openmpi

# remove all packages
spack unload --all

Spack unload --all removes the command-line interface to Slurm from the environment also. This is required to interact with the cluster controller using the sinfo, sbatch and similar commands. Make sure to load the Spack slurm package again if required.

Footnotes

Command-line Interface, Wikipedia
https://en.wikipedia.org/wiki/Command-line_interface ↩︎
Shell (Computing), Wikipedia
https://en.wikipedia.org/wiki/Shell_(computing)↩︎
Effective Shell, Dave Kerr
https://effective-shell.com
https://github.com/dwmkerr/effective-shell ↩︎
The Linux Command Line, William Shotts
http://www.linuxcommand.org/tlcl.php ↩︎
Version Control, Wikipedia
https://en.wikipedia.org/wiki/Version_control ↩︎
Software Carpentry Lessons
Software Carpentry Foundation
https://software-carpentry.org/lessons ↩︎
The Carpentries Community Developed Lessons
Carpentries Communit
https://carpentries.org/community-lessons ↩︎
Rocky Linux, RESF Project
https://rockylinux.org ↩︎
Apptainer Documentation, Linux Foundation
https://apptainer.org/docs/user/latest/↩︎
Basic Usage, Spack Documentation
https://spack.readthedocs.io/en/latest/basic_usage.html ↩︎
Computer microarchitecture, Wikipedia
https://en.wikipedia.org/wiki/Microarchitecture ↩︎
Compiler, Wikipedia
https://en.wikipedia.org/wiki/Compiler ↩︎
GNU Compiler Collection, GNU Project
https://gcc.gnu.org/↩︎

--- title: Software Environment date-modified: 2024/08/12 abstract: > The cluster group provides multiple so called Virtual Application Environments aka VAEs tailored for many use-cases among the GSI user community. The following section provides an overview to this infrastructure and illustrates how to use the Spack HPC package manager to load required application software. --- ### Prerequisites The following material assumes your familiarity with the Linux command-line interface (CLI) [^wpcl] and a basic understanding of a command line shell [^wpsh] environment like Bash or Zsh. If this is new to you, we recommend to read the very beginner friendly introduction to this topic _Effective Shell_ [^effsh] by Dave Kerr and _The Linux Command Line_ [^tlcsw] by William Shotts. Shotts book is freely available under the Creative Commons license and has been translated into multiple languages. Furthermore it is recommended to know the basic usage of a text-editor like Vim or Emacs. Besides the already mentioned prerequisites there are more widely used tools for scientific computing environments. Your selection of these tools may vary depending on your use-case. On common denominator for most software projects in this environment is the use of the Git version control system [^wpvc]. Most scientific software frameworks and libraries require the user to have a understanding of a programming language. In scientific computing Python and C++ are very popular. An excellent introduction into the methodologies and basic tools for scientific computing are available from the _Software Carpentry Foundation_ [^scles]. Depending on your scientific field you may find more specific introduction material from _The Carpentries Community_ [^sccdl]. [^wpcl]: Command-line Interface, Wikipedia <https://en.wikipedia.org/wiki/Command-line_interface> [^wpsh]: Shell (Computing), Wikipedia <https://en.wikipedia.org/wiki/Shell_(computing)> [^wpvc]: Version Control, Wikipedia <https://en.wikipedia.org/wiki/Version_control> [^tlcsw]: The Linux Command Line, William Shotts <http://www.linuxcommand.org/tlcl.php> [^scles]: Software Carpentry Lessons Software Carpentry Foundation <https://software-carpentry.org/lessons> [^sccdl]: The Carpentries Community Developed Lessons Carpentries Communit <https://carpentries.org/community-lessons> [^effsh]: Effective Shell, Dave Kerr <https://effective-shell.com> <https://github.com/dwmkerr/effective-shell> ### Virtual Application Environment {#vae} The Virgo cluster is build with the incentive to align its software stack with the cluster computing infrastructure build by a majority of scientific research institutions. The operating system platform is build with RockyLinux [^rocky]. Slurm is used as workload management system and supports the allocation of containers. All containerized applications use Apptainer [^apptainer] as container run-time engine. Multiple containerized application environments called **Virtual Application Environment** (VAE) provide a pre-build selection of software. Users access a specific VAE by login to a corresponding [submit node][LrAob]: [LrAob]: ../access/submit-nodes.html [^rocky]: Rocky Linux, RESF Project <https://rockylinux.org> [^apptainer]: Apptainer Documentation, Linux Foundation <https://apptainer.org/docs/user/latest/> Virtual Environment | Support Until ------|------------------------ VAE'23 | 2025 VAE'24 | 2026 ::: {.callout-note appearance="simple"} It is **required to use SSH [key-authentication][GPrap]** in order to logon to a submit node of the Virgo cluster. Make sure to **use a supported [default-shell][5dSmn]**, otherwise SSH login will not work. [GPrap]: ../access/key-authentication.html [5dSmn]: ../access/submit-nodes.html#default-shell ::: Login to an VAE will print the absolute path to the container launched for reference. ```sh » ssh vae23.hpc.gsi.de Slurm Cluster – Virgo 3.0 Submit Node Container launched: /cvmfs/vae.gsi.de/vae23/containers/vae23-user_container_20230308T0124.sif ``` Containers will be **transparently started for your login sessions as well as for your jobs executed on the cluster**. Note that **each login runs in a dedicated container**. If you need multiple terminal sessions, which are also known as shells, it is recommend to use a [terminal-multiplexer][v1usr]. [v1usr]: ../appendix/configuration_tools.html#terminal-multiplexer VAEs allow **decoupling of the user application environments from the underling system software stack** used to operate on the compute hardware. This allows great options for cluster administrators to utilize the most optimal software components to support the hardware and to update/upgrade the infrastructure as required. Furthermore VAEs do enable multiple distinct application environments tailored to specific use-cases to coexist on the same infrastructure. These VAEs can be continuously developed with minimal dependencies to the underlying systems software stack. Environment variables describing the VAE container: Variable | Description -------------------------|---------------------------------- `APPTAINER_CONTAINER` | Absolute path to the container image `APPTAINER_NAME` | Name of the container image ### Spack Packages {#spack} Software is made available on the Virgo compute cluster using the Spack [^spack] package manager. Spack builds software packages with optimisation for multiple computer microarchitectures [^micro] like AMD Zen or Intel Broadwell. List available software in a virtual application environment (VAE) using following command: ```sh # list packages spack find # details infromation about a package (i.e openmpi) spack find -v openmpi ``` This prints all packages with their respective microarchitecture and compiler [^comp], e.g. GCC from the GNU Compiler Collection [^gcc]. Print the location's absolute path of a package: ```sh spack location -i openmpi target=$(spack arch -t) # note that following command evaluates to the # microarchitecture of your current host spack arch -t ``` **Load a package** into the current environment. If multiple versions of a specific package are available, it is required to provide a version for loading the package: ```sh spack load openmpi target=$(spack arch -t) # load a package including all dependencies spack load -r openmpi target=$(spack arch -t) # load a specific version spack load openmpi@3.1.0 target=$(spack arch -t) ``` **Remove a package** from your environment: ```sh # list loaded packages spack find --loaded # remove a package spack unload openmpi # remove all packages spack unload --all ``` Spack `unload --all` removes the command-line interface to Slurm from the environment also. This is required to interact with the cluster controller using the `sinfo`, `sbatch` and similar commands. Make sure to load the Spack `slurm` package again if required. [^spack]: Basic Usage, Spack Documentation <https://spack.readthedocs.io/en/latest/basic_usage.html> [^micro]: Computer microarchitecture, Wikipedia <https://en.wikipedia.org/wiki/Microarchitecture> [^comp]: Compiler, Wikipedia <https://en.wikipedia.org/wiki/Compiler> [^gcc]: GNU Compiler Collection, GNU Project <https://gcc.gnu.org/>