117 lines
4.7 KiB
Markdown
Executable File
117 lines
4.7 KiB
Markdown
Executable File
# Computing
|
|
|
|
## Computing environments
|
|
|
|
Standard Linux environment:
|
|
|
|
````
|
|
Linux saresc-cons-04.psi.ch 3.10.0-1160.45.1.el7.x86_64 #1 SMP Fri Sep 24 10:17:16 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
|
|
|
|
NAME="Red Hat Enterprise Linux Server"
|
|
VERSION="7.9 (Maipo)"
|
|
````
|
|
|
|
Some basic commands:
|
|
* `module list` : lists currently loaded modules in the environment
|
|
* `module avail`: lists available modules
|
|
* `module load <modulename>` : loads the given module into the current shell (it is directly active)
|
|
|
|
## Space
|
|
There is a large number of directories, default locations and quotas for the different computing tasks at SwissFEL.
|
|
|
|
For large experimental data the allocated space for the respective p-group should be used. E.g. under `/gpfs/photonics/swissfel/res/cristallina-staff/pxxxxx`, here ra.psi.ch does not have write access (regardless of the user) but can read the data.
|
|
|
|
To check the currently available space:
|
|
`quota -h` and `df -h`.
|
|
|
|
|
|
## File permsissions
|
|
We should all be part of the unx-cristall group. Please make sure permissions are set at least `chmod g+rw,` so that everyone within the group can access and modify documents. Only set more restrictive permissions when necessary. (Alternatively `chmod g=u` to set the permissions equal to user permissions).
|
|
|
|
|
|
## Anaconda
|
|
For all python-based applications and analysis scripts we use use virtual environments administrated using conda/mamba.
|
|
|
|
### Activate base environment
|
|
To bootstrap the base environment as a starting point use `source /sf/cristallina/applications/conda/envs/miniconda/bin/activate`. With a base environment now loaded we can initialize our shell using `conda init` so this environment is available at the next login.
|
|
|
|
### Cristallina environments
|
|
All conda environments for cristallina are located at `/sf/cristallina/applications/conda/envs`. Their respective descriptions in `/sf/cristallina/applications/conda/env_specs`, which is also tracked in git.psi.ch.
|
|
|
|
The main environments are:
|
|
* slic (`/sf/cristallina/applications/conda/envs/slic`)
|
|
* analysis (`/sf/cristallina/applications/conda/envs/analysis`)
|
|
|
|
for instrument control and data analysis. We are trying to track recent upstream packages but please only update if you ensure a working state afterwards and document the changes.
|
|
|
|
To be able to use the shortcut for activation, e.g. `conda activate slic` the environment directory needs to be added to your `.condarc` file as follows:
|
|
|
|
````
|
|
envs_dirs:
|
|
- /sf/cristallina/applications/conda/envs
|
|
````
|
|
|
|
A backup environment is currently located at:
|
|
````
|
|
/gpfs/photonics/swissfel/res/cristallina-staff/p19739/backup/cristallina2
|
|
````
|
|
if all else fails.
|
|
|
|
|
|
### Updating environments
|
|
|
|
The cleanest way to install and update environments is via their .yml specification. After editing the specification run for example:
|
|
|
|
`mamba env update --file cristallina_diagnostics.yml --prune`
|
|
|
|
to bring the environment to the most up-to-date state according to .yml file.
|
|
|
|
This has the advantage that we can always reproduce this environment. If manually installed packages become uninstalled their dependencies remain. Additionally we lose track of these manual installations.
|
|
|
|
## Getting git
|
|
The `git` installed on the cristallina consoles is from 2015, so we can use conda to provide us with an up-to-date version. E.g. with `miniconda` activated simply install using `conda install git`.
|
|
|
|
# sf_datafiles directly from github
|
|
- pip:
|
|
- "git+https://github.com/paulscherrerinstitute/sf_datafiles@master"
|
|
|
|
sf_data is installed directly from github (using python ensures that the activated environment site-packages is used):
|
|
|
|
python -m pip install git+https://github.com/paulscherrerinstitute/sf_datafiles@master
|
|
|
|
## Git repository settings
|
|
|
|
To allow shared access to a git repository folder:
|
|
````
|
|
git config --get core.sharedRepository
|
|
````
|
|
|
|
# Remote access (how can users get to RA)
|
|
## Through hop.psi.ch (nicer way)
|
|
Connect to PSI network for data access
|
|
`ssh -L user_name@hop.psi.ch user_name@ra.psi.ch -L5000:jupytera.psi.ch:443`
|
|
|
|
after that, just open the browser and type:
|
|
`https://localhost:5000/hub/login`
|
|
|
|
## Through NoMachine
|
|
see https://gitlab.psi.ch/maloja/docs/-/wikis/python/Getting-started
|
|
|
|
# Setting up the cluster:
|
|
This is a short summary of the steps to be followed to access the experimental data and the compute resources available for Cristallina users.
|
|
|
|
### Set up the enviroments paths
|
|
Once on ra (ssh user_name@ra.psi.ch as described above), create a symbolic link to the bash file:
|
|
`ln -s /sf/cristallina/bin/setting_up_ra.sh ~`
|
|
|
|
and run it
|
|
`sh setting_up_ra.sh`
|
|
|
|
This should set up the working python enviroments. For the analysis, please use the 'analysis' enviroment, which is set as a default one.
|
|
|
|
### Running own jupyter instance
|
|
Once on ra, copy the bash file:
|
|
`cp /sf/cristallina/bin/jupyter_on_ra.sh ~`
|
|
and run it.
|
|
|