Configuration & Tools

Modified

November 14, 2023

Abstract

This section of the appendix introduces multiple tools commonly used for scientific computing. Furthermore, it addresses common issues and how to overcome these by configuring the user environment.

Terminal Multiplexer

A terminal multiplexer (Tmux) ¹ is used to multiplex several separate shells inside a single terminal window. It is particularly useful to use with a single SSH login session for remote access to the HPC infrastructure. The terminal multiplexer Tmux ² supports:

Multiplex several shells with a single terminal session
Organises shells with split panes and tabs
Persistence keeps terminal sessions during logout until reconnect

Keys	Description
`C-b ?`	Show key binding
`C-b d`	Detach from running sessions

# start tmux
tmux

# at some point use `C-b d` to detach
# close your SSH session, and eventully login again later
# list all sessions managed by the server
tmux list-sessions

# attach to most recently used session
tmux attach-session

We recommend to read the book The Tao of tmux ³, which gives an excellent introduction to get started with Tmux.

Git Protocol Proxy

Git uses several protocols for client-server communication:

Protocol	Example Connection Address
https	`https://example.com/repository.git`
ssh	`git@example.com/repository.git`
git	`git://example.com/repository.git`

In case you want to learn more about the different protocols used to connect with a Git server read Git on the Server - The Protocols ⁴. The https protocol is recommended, since it uses the existing configuration for HTTP proxies on the Virgo cluster:

# print the pre-configured HTTP proxies
» env | grep HTTP
HTTPS_PROXY=http://proxy-ib2.gsi.de:3128/
HTTP_PROXY=http://proxy-ib2.gsi.de:3128/

Proxy a connection over SSH using the environment variable GIT_SSH_COMMAND. Modify the ssh command executed by Git to include an option for a to configure login nodes as jump host:

GIT_SSH_COMMAND="ssh -J lxpool.gsi.de " git ...

However if access to a repository is required over the git protocol it is necessary to configure a Git proxy command. Use the login nodes as network proxy to bypass the firewall for an outgoing connection:

# create a helper script with the proxy command
cat > gitproxy <<'EOF'
#!/bin/bash
exec ssh lxpool.gsi.de nc "$@"
EOF
# make sure that is is executable
chmod +x gitproxy
# set an environment variable to use a proxy command
export GIT_PROXY_COMMAND=$PWD/gitproxy

Connection with a Git server using the Git protocol:

# the connection uses git:// as protocol prefix
git clone git://...

Footnotes

Terminal multiplexer, Wikipedia
https://en.wikipedia.org/wiki/Terminal_multiplexer ↩︎
tmux project, GitHub
https://tmux.github.io ↩︎
The Tao of tmux, Leanpub
https://leanpub.com/the-tao-of-tmux/read ↩︎
Git on the Server - The Protocols
https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#The-Git-Protocol ↩︎

--- title: Configuration & Tools date-modified: 2023/11/14 abstract: > This section of the appendix introduces multiple tools commonly used for scientific computing. Furthermore, it addresses common issues and how to overcome these by configuring the user environment. --- ### Terminal Multiplexer A terminal multiplexer (Tmux) [^trmlx] is used to multiplex several separate shells inside a single terminal window. It is particularly useful to use with a single SSH login session for remote access to the HPC infrastructure. The terminal multiplexer **Tmux** [^tmux] supports: [^trmlx]: Terminal multiplexer, Wikipedia <https://en.wikipedia.org/wiki/Terminal_multiplexer> [^tmux]: tmux project, GitHub <https://tmux.github.io> - Multiplex several shells with a single terminal session - Organises shells with **split panes** and **tabs** - **Persistence** keeps terminal sessions during logout until reconnect Keys | Description -----|------------- `C-b ?` | Show key binding `C-b d` | Detach from running sessions ```sh # start tmux tmux # at some point use `C-b d` to detach # close your SSH session, and eventully login again later # list all sessions managed by the server tmux list-sessions # attach to most recently used session tmux attach-session ``` We recommend to read the book _The Tao of tmux_ [^txtao], which gives an excellent introduction to get started with Tmux. [^txtao]: The Tao of tmux, Leanpub <https://leanpub.com/the-tao-of-tmux/read> ### Git Protocol Proxy Git uses several protocols for client-server communication: Protocol | Example Connection Address ---------|---------------------------- https | `https://example.com/repository.git` ssh | `git@example.com/repository.git` git | `git://example.com/repository.git` In case you want to learn more about the different protocols used to connect with a Git server read _Git on the Server - The Protocols_ [^gitgp]. The `https` protocol is recommended, since it uses the existing configuration for HTTP proxies on the Virgo cluster: [^gitgp]: Git on the Server - The Protocols <https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#The-Git-Protocol> ```sh # print the pre-configured HTTP proxies » env | grep HTTP HTTPS_PROXY=http://proxy-ib2.gsi.de:3128/ HTTP_PROXY=http://proxy-ib2.gsi.de:3128/ ``` Proxy a connection over SSH using the environment variable `GIT_SSH_COMMAND`. Modify the `ssh` command executed by Git to include an option for a to configure [login nodes][XsJAm] as jump host: [XsJAm]: access/submit-nodes.html#login-nodes ```sh GIT_SSH_COMMAND="ssh -J lxpool.gsi.de " git ... ``` However if access to a repository is required over the `git` protocol it is necessary to configure a **Git proxy command**. Use the login nodes as network proxy to bypass the firewall for an outgoing connection: ```bash # create a helper script with the proxy command cat > gitproxy <<'EOF' #!/bin/bash exec ssh lxpool.gsi.de nc "$@" EOF # make sure that is is executable chmod +x gitproxy # set an environment variable to use a proxy command export GIT_PROXY_COMMAND=$PWD/gitproxy ``` Connection with a Git server using the Git protocol: ```sh # the connection uses git:// as protocol prefix git clone git://... ```