70 lines
3.1 KiB
Markdown
70 lines
3.1 KiB
Markdown
# Computing
|
|
|
|
## Computing environments
|
|
|
|
Standard Linux environment:
|
|
|
|
````
|
|
Linux saresc-cons-04.psi.ch 3.10.0-1160.45.1.el7.x86_64 #1 SMP Fri Sep 24 10:17:16 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
|
|
|
|
NAME="Red Hat Enterprise Linux Server"
|
|
VERSION="7.9 (Maipo)"
|
|
````
|
|
|
|
Some basic commands:
|
|
* `module list` : lists currently loaded modules in the environment
|
|
* `module avail`: lists available modules
|
|
* `module load <modulename>` : loads the given module into the current shell (it is directly active)
|
|
|
|
## Space
|
|
There is a large number of directories, default locations and quotas for the different computing tasks at SwissFEL.
|
|
|
|
For large experimental data the allocated space for the respective p-group should be used. E.g. under `/gpfs/photonics/swissfel/res/cristallina-staff/pxxxxx`, here ra.psi.ch does not have write access (regardless of the user) but can read the data.
|
|
|
|
To check the currently available space:
|
|
`quota -h` and `df -h`.
|
|
|
|
|
|
## File permsissions
|
|
We should all be part of the unx-cristall group. Please make sure permissions are set at least `chmod g+rw,` so that everyone within the group can access and modify documents. Only set more restrictive permissions when necessary. (Alternatively `chmod g=u` to set the permissions equal to user permissions).
|
|
|
|
|
|
## Anaconda
|
|
For all python-based applications and analysis scripts we use use virtual environments administrated using conda/mamba.
|
|
|
|
### Activate base environment
|
|
To bootstrap the base environment as a starting point use `source /sf/cristallina/applications/conda/envs/miniconda/bin/activate`. With a base environment now loaded we can initialize our shell using `conda init` so this environment is available at the next login.
|
|
|
|
### Cristallina environments
|
|
All conda environments for cristallina are located at `/sf/cristallina/applications/conda/envs`. Their respective descriptions in `/sf/cristallina/applications/conda/env_specs`, which is also tracked in git.psi.ch.
|
|
|
|
The main environments are:
|
|
* slic (`/sf/cristallina/applications/conda/envs/slic`)
|
|
* analysis (`/sf/cristallina/applications/conda/envs/analysis`)
|
|
|
|
for instrument control and data analysis. We are trying to track recent upstream packages but please only update if you ensure a working state afterwards and document the changes.
|
|
|
|
To be able to use the shortcut for activation, e.g. `conda activate slic` the environment directory needs to be added to your `.condarc` file as follows:
|
|
|
|
````
|
|
envs_dirs:
|
|
- /sf/cristallina/applications/conda/envs
|
|
````
|
|
|
|
A backup environment is currently located at:
|
|
````
|
|
/gpfs/photonics/swissfel/res/cristallina-staff/p19739/backup/cristallina2
|
|
````
|
|
if all else fails.
|
|
|
|
## Getting git
|
|
The `git` installed on the cristallina consoles is from 2015, so we can use conda to provide us with an up-to-date version. E.g. with `miniconda` activated simply install using `conda install git`.
|
|
|
|
# sf_datafiles directly from github
|
|
- pip:
|
|
- "git+https://github.com/paulscherrerinstitute/sf_datafiles@master"
|
|
|
|
sf_data is installed directly from github (using python ensures that the activated environment site-packages is used):
|
|
|
|
python -m pip install git+https://github.com/paulscherrerinstitute/sf_datafiles@master
|