Documentation/computing.md

Computing

Computing environments

Standard Linux environment:

Linux saresc-cons-04.psi.ch 3.10.0-1160.45.1.el7.x86_64 #1 SMP Fri Sep 24 10:17:16 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"

Some basic commands:

• module list : lists currently loaded modules in the environment
• module avail: lists available modules
• module load <modulename> : loads the given module into the current shell (it is directly active)

Space

There is a large number of directories, default locations and quotas for the different computing tasks at SwissFEL.

For large experimental data the allocated space for the respective p-group should be used. E.g. under /gpfs/photonics/swissfel/res/cristallina-staff/pxxxxx, here ra.psi.ch does not have write access (regardless of the user) but can read the data.

To check the currently available space: quota -h and df -h.

File permsissions

We should all be part of the unx-cristall group. Please make sure permissions are set at least chmod g+rw, so that everyone within the group can access and modify documents. Only set more restrictive permissions when necessary. (Alternatively chmod g=u to set the permissions equal to user permissions).

Anaconda

For all python-based applications and analysis scripts we use use virtual environments administrated using conda/mamba.

Activate base environment

To bootstrap the base environment as a starting point use source /sf/cristallina/applications/conda/envs/miniconda/bin/activate. With a base environment now loaded we can initialize our shell using conda init so this environment is available at the next login.

Cristallina environments

All conda environments for cristallina are located at /sf/cristallina/applications/conda/envs. Their respective descriptions in /sf/cristallina/applications/conda/env_specs, which is also tracked in git.psi.ch.

The main environments are:

• slic (/sf/cristallina/applications/conda/envs/slic)
• analysis (/sf/cristallina/applications/conda/envs/analysis)

for instrument control and data analysis. We are trying to track recent upstream packages but please only update if you ensure a working state afterwards and document the changes.

To be able to use the shortcut for activation, e.g. conda activate slic the environment directory needs to be added to your .condarc file as follows:

envs_dirs:
  - /sf/cristallina/applications/conda/envs

A backup environment is currently located at:

/gpfs/photonics/swissfel/res/cristallina-staff/p19739/backup/cristallina2

if all else fails.

Updating environments

The cleanest way to install and update environments is via their .yml specification. After editing the specification run for example:

mamba env update --file cristallina_diagnostics.yml --prune

to bring the environment to the most up-to-date state according to .yml file.

This has the advantage that we can always reproduce this environment. If manually installed packages become uninstalled their dependencies remain. Additionally we lose track of these manual installations.

Getting git

The git installed on the cristallina consoles is from 2015, so we can use conda to provide us with an up-to-date version. E.g. with miniconda activated simply install using conda install git.

sf_datafiles directly from github

• pip:
  • "git+https://github.com/paulscherrerinstitute/sf_datafiles@master"

sf_data is installed directly from github (using python ensures that the activated environment site-packages is used):

python -m pip install git+https://github.com/paulscherrerinstitute/sf_datafiles@master

Git repository settings

To allow shared access to a git repository folder:

git config --get core.sharedRepository

Remote access (how can users get to RA)

Through hop.psi.ch (nicer way)

Connect to PSI network for data access ssh -L user_name@hop.psi.ch user_name@ra.psi.ch -L5000:jupytera.psi.ch:443

after that, just open the browser and type: https://localhost:5000/hub/login

Through NoMachine

see https://gitlab.psi.ch/maloja/docs/-/wikis/python/Getting-started

Setting up the cluster:

This is a short summary of the steps to be followed to access the experimental data and the compute resources available for Cristallina users.

Set up the enviroments paths

Once on ra (ssh user_name@ra.psi.ch as described above), create a symbolic link to the bash file: ln -s /sf/cristallina/bin/setting_up_ra.sh ~

and run it sh setting_up_ra.sh

This should set up the working python enviroments. For the analysis, please use the 'analysis' enviroment, which is set as a default one.

Running own jupyter instance

Once on ra, copy the bash file: cp /sf/cristallina/bin/jupyter_on_ra.sh ~ and run it.