Connecting to the Ra Cluster for Data Analysis

Getting on the Ra cluster

From the PSI network (when on site), one can use any Linux console and use ssh <user>@ra.

From outside, the easiest way is to use NoMachine. Go to nx.psi.ch and open ra_nx virtual desktop on the cluster. Here you can open a terminal and by default land in your home directory on the cluster. (If you prefer a standalone client, download it and follow these instructions).

First-time setting up environments and paths

If you're logging in for the first time, copy the commonly used bash scripts to your home folder: cp /sf/cristallina/applications/it/cluster_tools/ra_cluster_cristallina.sh ~

Run it from your home to set up Cristallina environments and create a symbolic link to your experiment sh ~/ra_cluster_cristallina.sh <pgroup> <folder_name_you_like>

Running jupyter via spawner session

From the PSI network simply open https://jupytera.psi.ch in your browser, externally this can be done via Nomachine virtual desktop, just open a browser there.

Running own jupyter session

The control over own the default jupyter on spawner is limited, so sometimes it might be needed to run an own jupyter server. This can be done as follows.

Step 1: Secure Shell Access

Connect to ra login node as usual (ssh <user>@ra or via Nomachine). Copy the following scripts to your home directory: cp /sf/cristallina/applications/it/cluster_tools/jupyter_on_ra.sh ~ cp /sf/cristallina/applications/it/cluster_tools/start_jupyter.sh ~ cp /sf/cristallina/applications/it/cluster_tools/print_jupyter_link.sh ~ cp /sf/cristallina/applications/it/cluster_tools/ensure_jupyter.sh ~

Step 2: Check for an Assigned Node

Once connected, verify if a compute node has already been assigned to your account. This step is crucial for determining whether you can start a new session or join an existing one. Use the squeue command combined with grep to search for your username in the queue list:

squeue | grep <user>

Replace <user> with your username. If your username appears in the output, a node has been assigned to you.

Step 3: Start Jupyter Notebook

If no node is currently assigned (i.e., your username does not appear in the queue), you'll need to start a Jupyter Notebook session. This is typically done by executing a script provided in the cluster environment:

start_jupyter.sh

This script initialises a Jupyter Notebook server. Note any output from this command, especially the node name and port number, as you will need them for accessing the notebook.

Step 4: Access Jupyter Notebook (from PSI network)

Once the Jupyter Notebook server is running, open Google Chrome (google-chrome) or any web browser of your choice. To access your notebook, you'll need to enter the node name and port number into the browser's address bar in the following format:

http://<name_of_the_node>:<port>

<name_of_the_node>: Replace this with the actual name of the node assigned to your Jupyter session.
<port>: This is typically 8888, but if that port is occupied, the script might start the notebook on an alternative port like 8889. Use the port number provided by the start_jupyter.sh script output.

For example, if your node is named ra-node01 and the notebook is running on port 8888, you would enter:

http://ra-node01:8888

You can check where your jupyter session is running also with: sh ~/print_jupyter_link.sh

Step 5: Access Jupyter Notebook (from outside PSI)

First, you need to have the hop session open in a separate terminal window (ssh <username>@hop.psi.ch, authenticate yourself with 2FA). Don't close this terminal window, it must stay open for any tunneling to work. If you get disconnected for any reason, you need connect to hop first again.

This can be done with the scripts from this git repo. Clone the repo or download the script tunnel2jupyera to create an ssh tunnel to your node or to jupytera.psi.ch jupyterhub. Use the powershell .ps1 version for Windows and bash .sh for MacOS/Linux.

Run the script from your local machine and it will guide you through what you need to enter. Don't forget that your hop session must be running in a separate terminal already.