diff --git a/docs/cls/data_transfer/emf.md b/docs/cls/data_transfer/emf.md new file mode 100644 index 00000000..41085cc9 --- /dev/null +++ b/docs/cls/data_transfer/emf.md @@ -0,0 +1,88 @@ +# EMF PSI + +The storage systems used by the Electron Microscopy Facility at PSI (EMF) are mounted on the Merlin 7 login nodes. You can copy files from the EMF storage to your home or project directory on Merlin 7, or transfer them from Merlin 7 to your local computer. + +## Within Merlin 7 + +The EMF files are accessible through the `/emf` mount on the Merlin 7 cluster. However, the files are **not stored on Merlin 7**; the mount provides access to files that remain on the EMF storage system. + +Before working with the files, copy them to your home directory or project directory on Merlin 7. + +### `rsync` + +We recommend using `rsync` to copy files because it preserves file attributes and provides useful information about the transfer. + +Run: + +```bash +rsync -avAHXS sourcedir/ destdir/ +``` + +If you need to change the group ownership of the copied files, use: + +```bash +rsync -avAHXS --chown=:$GROUP_NAME sourcedir/ destdir/ +``` + +Replace `$GROUP_NAME` with the name of the group that should own the files. + +!!! note + The trailing `/` in `sourcedir/` is significant. It means that the **contents** of `sourcedir` are copied to `destdir`, rather than the directory itself. + +### `cp` + +Alternatively, you can use `cp`: + +```bash +cp -r sourcedir/ destdir/ +``` + +## From Merlin 7 to your local machine + +You can transfer files from Merlin 7 to your local computer using `rsync`, `scp`, or a graphical application such as FileZilla. + +### `rsync` + +`rsync` is recommended for transferring large amounts of data or for transfers that may need to be resumed. + +For example: + +```bash +rsync -avAHXS $USER@login001.merlin7.psi.ch:/data/project/general/myproject/ ~/localdata/ +``` + +### `scp` + +For a simple recursive copy, you can use `scp`: + +```bash +scp -r $USER@login001.merlin7.psi.ch:/data/project/general/myproject/ ~/localdata/ +``` + +### FileZilla + +You can also use [FileZilla](https://filezilla-project.org/) to transfer files between your local computer and Merlin 7. + +Open FileZilla and create a connection to Merlin 7 using the following settings: + +| Setting | Value | +| ------------ | ------------------------------------------------------ | +| **Host** | `login001.merlin7.psi.ch` or `login002.merlin7.psi.ch` | +| **Username** | Your Merlin 7 username | +| **Password** | Your Merlin 7 password | +| **Port** | `22` | +| **Protocol** | `SFTP - SSH File Transfer Protocol` | + +Once connected, the files on Merlin 7 will be displayed in the **Remote site** panel. + +You can transfer files between your computer and Merlin 7 by dragging them between the **Remote site** and **Local site** panels. Alternatively, right-click a file or directory and select **Download** or **Upload**. + +!!! tip + To improve file transfer speeds, increase the maximum number of simultaneous transfers in FileZilla: + + 1. Open **Edit > Settings**. + 2. Select **Transfers**. + 3. Under **Maximum simultaneous transfers**, set **Maximum simultaneous transfers** to `10`. + 4. Click **OK** to save the changes. + + ![FileZilla transfer settings](../../images/CLS/data_transfer/data_transfer_tips.png) diff --git a/docs/cls/data_transfer/scopem.md b/docs/cls/data_transfer/scopem.md new file mode 100644 index 00000000..4a6c5e94 --- /dev/null +++ b/docs/cls/data_transfer/scopem.md @@ -0,0 +1,139 @@ +# ScopeM ETH Zurich + +This article explains how to transfer data remotely between the [ETH Zurich ScopeM facility](https://scopem.ethz.ch/) and Merlin 7 using the **SFTP (SSH File Transfer Protocol)**. + +## Using NoMachine + +One of the easiest ways to transfer files using SFTP is through **NoMachine** and **FileZilla**. + +!!! tip + To improve file transfer speeds, increase the maximum number of simultaneous transfers in FileZilla: + + 1. Open **Edit > Settings**. + 2. Select **Transfers**. + 3. Under **Maximum simultaneous transfers**, set the value to **10**. + 4. Click **OK** to save the changes. + + ![FileZilla transfer settings](../../images/CLS/data_transfer/data_transfer_tips.png) + +### 1. Connect to Merlin 7 + +Connect to Merlin 7 using NoMachine. + +### 2. Open FileZilla + +Open **FileZilla** either by clicking the FileZilla icon or by running the following command in a terminal: `filezilla` + +![Open FileZilla](../../images/CLS/data_transfer/data_transfer_scopem_1.webp) + +### 3. Connect to the ScopeM endpoint + +In FileZilla, enter the ScopeM SFTP endpoint in the connection fields: + +* **Host:** `sftp://scopem-krios1data.ethz.ch` +* **Username:** Your ScopeM username +* **Password:** Your ScopeM password +* **Port:** `5007` + +Click **Quickconnect** to establish the connection. + +![FileZilla connection settings](../../images/CLS/data_transfer/data_transfer_scopem_2.webp) + +Once connected, your files on the ScopeM endpoint will be displayed in the **Remote site** panel. + +![FileZilla remote files](../../images/CLS/data_transfer/data_transfer_scopem_3.webp) + +You can transfer files between ScopeM and Merlin 7 by dragging them between the **Remote site** and **Local site** panels. Alternatively, right-click a file or directory and select **Download** or **Upload**. + +!!! note "NoMachine session" + File transfers can continue after you close the NoMachine window, as long as the NoMachine session itself remains active. Do not terminate the NoMachine session until the transfer has completed. + +## Using the terminal + +ScopeM endpoints can also be accessed directly from a Merlin 7 login node using `sftp`. + +### **Interactive SFTP session** + +Connect to a ScopeM endpoint with: + +```bash +sftp -P 5007 @scopem-krios1data.ethz.ch +``` + +If you encounter an error such as **"Too many authentication failures"**, force SFTP to use password authentication: + +```bash +sftp -o PubkeyAuthentication=no -o PreferredAuthentications=password -P 5007 @scopem-krios1data.ethz.ch +``` + +Enter your password when prompted. + +Once connected, use the `get` command to download files or directories. For example: + +```text +sftp> get -r home/P1 + +Fetching home/P1/ to P1 +Retrieving home/P1 +Retrieving home/P1/J1 +``` + +The `-r` option recursively downloads the specified directory and its contents. + +### **Download a directory directly** + +You can also specify the remote path directly on the command line: + +```bash +sftp -r -P 5007 @scopem-krios1data.ethz.ch:home/P1 ./ +``` + +You will be prompted for your password: + +```text +@scopem-krios1data.ethz.ch's password: + +Connected to scopem-krios1data.ethz.ch. + +Fetching /home/P1/ to ./P1 +Retrieving /home/P1 +Retrieving /home/P1/J1 +``` + +### **Configure SSH for easier access** + +If you frequently transfer data from ScopeM, you can simplify the connection by adding the ScopeM endpoints to your SSH configuration file: + +```text +$HOME/.ssh/config +``` + +For example: + +```text +Host krios1 krios2 krios3 + HostName scopem-%hdata.ethz.ch + +Host scopem-krios*data.ethz.ch + Port 5007 + +Host *.ethz.ch + User +``` + +With this configuration, the hostname, port, and username do not need to be specified for every connection. + +For example, instead of: + +```bash +sftp -r -P 5007 @scopem-krios1data.ethz.ch:home/P1 ./ +``` + +you can use: + +```bash +sftp -r krios1:home/P1 ./ +``` + +!!! note "SSH configuration" + Adjust the `User` entry in `$HOME/.ssh/config` to match your username. diff --git a/docs/cls/data_transfer/uni_basel.md b/docs/cls/data_transfer/uni_basel.md new file mode 100644 index 00000000..fbf692cd --- /dev/null +++ b/docs/cls/data_transfer/uni_basel.md @@ -0,0 +1,115 @@ +# Uni Basel + +The Uni Basel data-transfer server is directly accessible from the Merlin 7 login nodes using **SFTP**. + +The examples below show how to automate file listing and data transfer using Bash and `expect`. The scripts were originally provided by P. Filipcik. + +!!! warning "Password security" + The examples below use `expect` to provide the SFTP password automatically. **Do not commit scripts containing your password to version control or share them with others.** For regular or automated transfers, consider using SSH keys or passing the password through a protected environment variable or other secure mechanism. + +In the examples, the username on the Uni Basel SFTP server is `sftp`. Replace the example password with your own credentials and adjust the paths as needed. + +## 1. Listing files and folders + +The following script lists the directories available in the `share` directory on the Uni Basel server. + +```bash +#!/bin/bash + +# Usage: ./bioem-list.sh + +echo "Listing directories in BioEM share" + +expect <" +send "ls share/\r" + +expect "sftp>" +send "bye\r" +EOF +``` + +Save the script as `bioem-list.sh` and make it executable: + +```bash +chmod +x bioem-list.sh +``` + +Then run: + +```bash +./bioem-list.sh +``` + +## 2. Retrieving data + +The following script downloads a directory from the Uni Basel server. + +The directory name should be specified **without** the `share/` prefix. Run the script from the directory where you want the downloaded data to be stored. + +```bash +#!/bin/bash + +# Usage: ./bioem-get.sh +# +# Example: +# ./bioem-get.sh 2024-09-01_mydata + +if [ "$#" -lt 1 ]; then + echo "Illegal number of parameters" + echo "Usage: ./bioem-get.sh " + echo "Example: ./bioem-get.sh 2024-09-01_mydata" + exit 1 +fi + +folder="$1" + +expect <" +send "get -R share/$folder\r" + +expect "sftp>" +send "bye\r" +EOF +``` + +Make the script executable: + +```bash +chmod +x bioem-get.sh +``` + +For example, to download the directory `2024-09-01_mydata`: + +```bash +./bioem-get.sh 2024-09-01_mydata +``` + +The data will be downloaded to the current working directory. + +!!! note "Recursive download" + The `-R` option in the SFTP `get` command recursively downloads the specified directory and all of its contents. + +!!! note "Run from the target directory" + Run `bioem-get.sh` from the directory where you want the downloaded data to be stored. + +## Password handling + +The examples above contain a placeholder for the password: `your_password` + +Replace this with your actual password if you use the scripts as written. + +!!! warning "Avoid hardcoding passwords" + Hardcoding a password directly in a script is not recommended because anyone who can read the script can also read the password. For repeated or automated transfers, use a more secure authentication method, such as **SSH keys**, where supported. + + If password-based authentication is required, consider using a protected `expect` wrapper or another mechanism that keeps the password outside the script. diff --git a/docs/cls/data_transfer/unil_s3_bucket.md b/docs/cls/data_transfer/unil_s3_bucket.md new file mode 100644 index 00000000..86331e84 --- /dev/null +++ b/docs/cls/data_transfer/unil_s3_bucket.md @@ -0,0 +1,50 @@ +# UNIL S3 Bucket + +Data transfer from the **University of Lausanne (UNIL)** is now performed using **Simple Storage Service (S3)**. + +Before using the service, the **principal investigator (PI)** of the project must complete the account registration process. UNIL will provide the necessary information and instructions by email. + +Once the account has been registered, data can be transferred using [`rclone`](https://rclone.org/), which is available on Merlin 7. To verify that `rclone` is installed, run: + +```bash +rclone --version +``` + +UNIL will provide the specific `rclone` configuration and commands required to access the S3 storage. The Merlin 7 login nodes are already configured to support this service. + +## Example commands + +After configuring `rclone` with the credentials provided by UNIL, the configuration file should be located at: + +```text +~/.config/rclone/rclone.conf +``` + +You can then use the following commands. + +### List files in the bucket + +To check the connection and list the directories available in the bucket: + +```bash +rclone -vvP lsd S3-UNIL-RW: +``` + +### Copy data from the S3 bucket + +To copy data from the S3 bucket to a local directory: + +```bash +rclone -vvP copy S3-UNIL-RW:Test/ IntoMyFolder/IntoMySubFolder/ --s3-nocheck-bucket +``` + +!!! note "Command-line options" + - `-vv` enables verbose logging, which is useful for troubleshooting. + - `-P` displays transfer progress. + - `--s3-nocheck-bucket` disables the bucket existence check. This may be required for some S3 configurations. + +!!! warning "Access credentials" + Make sure that your `rclone` configuration contains the correct access credentials provided by UNIL. **Do not share your `rclone.conf` file**, as it may contain sensitive credentials. + +!!! tip "Test the connection first" + Before starting a large data transfer, use `rclone lsd` to verify that your configuration is working and that you can access the expected bucket. diff --git a/docs/cls/software/EM/cryosparc/benchmarks.md b/docs/cls/software/EM/cryosparc/benchmarks.md new file mode 100644 index 00000000..aae0800b --- /dev/null +++ b/docs/cls/software/EM/cryosparc/benchmarks.md @@ -0,0 +1,13 @@ +The results of these benchmarks are influenced by node, filesystem, and network utilization at the time they were executed. In addition, the benchmarks were run using different software versions, which may also affect performance. Therefore, the results should be considered indicative rather than a controlled hardware-only comparison. + +Nevertheless, the benchmarks provide a useful indication of the performance difference between the **NVIDIA A100-SXM4-80GB** and **NVIDIA GH200 120GB** GPUs. + +## **Benchmark Job** + +The benchmark was run using a single GPU on both the A100 and GH200 nodes. The job on the A100 node was executed with **version 4.7.1**, while the job on the Grace Hopper node was executed with **version 5.1.0-privatebeta.2**. + +Because the software versions differ, some of the observed performance difference may be attributable to software changes rather than GPU hardware alone. A direct hardware comparison would ideally use the **same software version and configuration** on both nodes. + +![Overall benchmark times for A100 and GH200 GPUs](../../../../images/CLS/cryosparc/cryosparc_benchmarks_overall_times.png) + +![Speedup between A100 and GH200 GPUs](../../../../images/CLS/cryosparc/cryosparc_benchmarks_speedup_common.png) diff --git a/docs/cls/software/EM/cryosparc/installation.md b/docs/cls/software/EM/cryosparc/installation.md new file mode 100644 index 00000000..762af922 --- /dev/null +++ b/docs/cls/software/EM/cryosparc/installation.md @@ -0,0 +1,93 @@ +Users can install multiple versions of CryoSPARC. However, each version must have its own separate database. Since **version 5**, a single CryoSPARC version can also support multiple worker-node architectures simultaneously. For example, the same version can have lanes for `x86_64` workers (CPU and A100 nodes on Merlin 7) and `aarch64` workers (Grace Hopper nodes on Merlin 7). When submitting a job, the user can then select the appropriate worker architecture. + +!!! warning "Mixed Worker Architectures" + Currently, a single CryoSPARC installation cannot have both `x86_64` and `aarch64` workers installed simultaneously on Merlin 7. If you need to use both architectures, separate CryoSPARC installations must be used. + + **Support for running both architectures under the same CryoSPARC version will soon become available.** + +**Before installing CryoSPARC, obtain a license key from the [CryoSPARC download page](https://cryosparc.com/download/).** + +## Installation on CPU and A100 nodes + +The installation can be performed from one of the Merlin 7 login nodes. By default, the CryoSPARC installation path is `/data/user/$USER/cryosparc`. This path can be changed but you will need to add the flag `-d [path to CryoSPARC installation]` in the `cryosparc-merlin` commands when using a non-default path. + +To install CryoSPARC version `4.7.1` in the default path (`/data/user/$USER/cryosparc`): + +```bash +cryosparc-merlin install complete \ + --license [CryoSPARC License] \ + --version 4.7.1 +``` + +!!! info "CryoSPARC base port" + The installer also automatically selects a new base port based on availability in the remote node. There is no need to specify one manually during the installation. + +Follow the instructions displayed by the installation script. + +If an existing installation is overwritten, the installer automatically backs up the previous database. + +When the master-node installation starts, the installer will ask: + +> Are the above settings correct? + +Enter `1` and press **Enter** to proceed. + +Later, it will ask: + +> Add bin directory to your ~/.bashrc ? + +Enter `2` and press **Enter**. + +You can then create your CryoSPARC user account. **Do not use the same password as your PSI password.** + +## Installation on Grace Hopper nodes + +Installing CryoSPARC for the Grace Hopper nodes is similar to the standard installation described above. The main differences are: + +* The worker-node architecture is `aarch64`. +* A different CryoSPARC version is used. +* A separate installation path must be specified to avoid conflicts with an existing installation. + +In this example, the installation path is `/data/user/$USER/cryosparc_beta`. + +The installation command is: + +```bash +cryosparc-merlin install complete \ + --license [CryoSPARC License] \ + --version 5.1.0-privatebeta.2 \ + -d /data/user/$USER/cryosparc_beta \ + --arch-worker aarch64 +``` + +!!! info "Grace Hopper Node Availability" + The installation requires an interactive job on a Grace Hopper node to compile the `aarch64` worker components. If all Grace Hopper nodes are busy, the job may take some time to start. The installation will continue automatically once a suitable node becomes available. + +The master node remains `x86_64` because the CryoSPARC instance runs on a node with an `x86_64` CPU architecture. Only the worker architecture is changed to `aarch64`. + +As with the standard installation, when the installer asks: + +> Are the above settings correct? + +Enter `1` and press **Enter** to proceed. + +When it asks: + +> Add bin directory to your ~/.bashrc ? + +Enter `2` and press **Enter**. + +## Possible Issues + +The installation may complete without installing the default lanes. If this happens, run: + +```bash +cryosparc-merlin lanes install --all +``` + +If CryoSPARC is installed in a location other than `/data/user/$USER/cryosparc`, specify the installation path with `-d`: + +```bash +cryosparc-merlin lanes install -d [CryoSPARC installation path] --all +``` + diff --git a/docs/cls/software/EM/cryosparc/issues.md b/docs/cls/software/EM/cryosparc/issues.md new file mode 100644 index 00000000..f17f787a --- /dev/null +++ b/docs/cls/software/EM/cryosparc/issues.md @@ -0,0 +1,125 @@ +## CryoSPARC project needs to be moved + +The easiest way to continue working with a CryoSPARC project after moving it to a different directory is to: + +1. **Detach the project** from CryoSPARC. +2. **Move the project directory** to its new location. +3. **Reattach the project** in CryoSPARC. + +!!! warning "Recommended procedure" + Always detach the project from CryoSPARC **before** moving its directory. This ensures that the project is properly removed from the CryoSPARC database before it is moved. + +### 1. Detach the project + +In CryoSPARC, open the project and select **Detach Project**. + +![Detach project](../../../../images/CLS/cryosparc/cryosparc_issues_correct_symlinks_6.webp) + +### 2. Move the project directory + +Move the project folder to its new location. For example: + +```bash +mv /data/project/bio//CS-project /data/project/cls//CS-project +``` + +### 3. Reattach the project + +In CryoSPARC, use **Attach Project** to add the project from its new location. + +![Attach project](../../../../images/CLS/cryosparc/cryosparc_issues_correct_symlinks_7.webp) + +## CryoSPARC project was moved + +If you moved the project directory **before detaching it from CryoSPARC**, the project may still be registered in the CryoSPARC database under its old location. + +In this case, you can mark the project as deleted in the database and then attach it again from its new location. + +!!! warning "Use with caution" + The following procedure directly modifies the CryoSPARC database. Make sure that you specify the correct project ID and that the project directory has already been moved to its new location. + +On a Merlin 7 login node, start the CryoSPARC interactive Python session: + +```bash +/data/user/$USER/cryosparc/cryosparc_master/bin/cryosparcm icli +``` + +!!! info "CryoSPARC path" + If your CryoSPARC installation is in a different directory, correct the path to `cryosparc_master` in the command above. For example, `/data/user/$USER/cryosparc_beta/cryosparc_master/bin/cryosparcm icli` + +Once inside the interactive Python shell, specify the project ID you want to remove from the database. For example, for project `P10`: + +```python +db.projects.update_one({'uid': 'P10'}, {'$set': {'deleted': True}}) +``` + +!!! note + Replace `P10` with the ID of the project you want to remove from the database. + +After running the command, refresh your CryoSPARC browser session. + +Next, navigate to the **new project directory** and remove the `cs.lock` file: + +```bash +cd /path/to/CryoSPARC/project +rm cs.lock +``` + +You can then use **Attach Project** in CryoSPARC to attach the project from its new location. + +!!! warning "Check the project path" + Before attaching the project, verify that you are selecting the correct project directory and that the project files are intact. + +## Correct symlinks + +When raw data is imported into a CryoSPARC project, the data itself is not copied into the project directory. Instead, CryoSPARC creates symbolic links (symlinks) in the import job directory that point to the original data files. + +If you move the data after importing it with an **Import Particles**, **Import Micrographs**, or **Import Movies** job, the original paths stored in these symlinks may no longer exist. As a result, the symlinks become broken and CryoSPARC can no longer access the imported data. + +To repair these symlinks, use the `replace_cryosparc_symlinks.py` script provided on Merlin 7. + +### Start the CryoSPARC interactive Python session + +Start a CryoSPARC interactive Python session by running: + +```bash +/data/user/$USER/cryosparc/cryosparc_master/bin/cryosparcm icli +``` + +!!! info "CryoSPARC path" + If your CryoSPARC installation is in a different directory, correct the path to `cryosparc_master` in the command above. For example, `/data/user/$USER/cryosparc_beta/cryosparc_master/bin/cryosparcm icli` + +Once inside the interactive Python shell, indicated by the `>>>` prompt, execute the symlink-replacement script: + +```python +exec(open("/data/project/cls/shared/software/cryosparc/merlin-cryosparc-installation/scripts/replace_cryosparc_symlinks.py").read()) +``` + +Follow the instructions displayed by the script to identify the affected project and jobs and to replace the outdated paths. + +### Example + +Suppose you want to update all symlinks in every job within the project with ID `P6`. + +![Select project](../../../../images/CLS/cryosparc/cryosparc_issues_correct_symlinks_1.webp) + +Next, select all jobs that should be updated: + +![Select jobs](../../../../images/CLS/cryosparc/cryosparc_issues_correct_symlinks_2.webp) + +Specify the path that should be replaced. In this example, replace: + +`/data/project/bio` with `/data/project/cls` + +![Replace path](../../../../images/CLS/cryosparc/cryosparc_issues_correct_symlinks_3.webp) + +Review the proposed changes and confirm them to apply the replacements: + +![Confirm changes](../../../../images/CLS/cryosparc/cryosparc_issues_correct_symlinks_4.webp) + +After the operation completes, the script displays a confirmation that the changes were applied successfully and that no errors occurred. + +![Successful replacement](../../../../images/CLS/cryosparc/cryosparc_issues_correct_symlinks_5.webp) + +!!! warning "Check the replacement paths carefully" + Make sure that the old and new paths are correct before confirming the changes. The script modifies the symlinks for the selected jobs, so an incorrect path replacement may cause the symlinks to point to the wrong files. diff --git a/docs/cls/software/EM/cryosparc/lanes.md b/docs/cls/software/EM/cryosparc/lanes.md new file mode 100644 index 00000000..a164dac3 --- /dev/null +++ b/docs/cls/software/EM/cryosparc/lanes.md @@ -0,0 +1,91 @@ +A standard CryoSPARC `complete` installation creates a set of default lanes configured for the Merlin 7 cluster. + +The lanes available by default are listed below. + +| **Lanes** | **Memory** | **CPUs** | **Time** | **Queue** | +| ------------------------- | ------------------ | ------------------ | ------------------- | ------------ | +| **cpu-hourly** | *Set by CryoSPARC* | *Set by CryoSPARC* | 1 hour | hourly | +| **cpu-daily** | *Set by CryoSPARC* | *Set by CryoSPARC* | 24 hours *(1 day)* | daily | +| | | | | | +| **a100-hourly** | *Set by CryoSPARC* | 16 per GPU | 1 hour | a100-hourly | +| **a100-daily-4h** | *Set by CryoSPARC* | 16 per GPU | 4 hours | a100-daily | +| **a100-daily** | *Set by CryoSPARC* | 16 per GPU | 24 hours *(1 day)* | a100-daily | +| **a100-general-2d** | *Set by CryoSPARC* | 16 per GPU | 48 hours *(2 days)* | a100-general | +| **a100-general-2d-100Gb** | 100 Gb | 16 per GPU | 48 hours *(2 days)* | a100-general | +| | | | | | +| **gh-hourly** | *Set by CryoSPARC* | 72 per GPU | 1 hour | gh-hourly | +| **gh-daily-4h** | *Set by CryoSPARC* | 72 per GPU | 4 hours | gh-daily | +| **gh-daily** | *Set by CryoSPARC* | 72 per GPU | 24 hours *(1 day)* | gh-daily | +| **gh-general-2d** | *Set by CryoSPARC* | 72 per GPU | 48 hours *(2 days)* | gh-general | +| **gh-general-2d-100Gb** | 100 Gb | 72 per GPU | 48 hours *(2 days)* | gh-general | + +During installation, lanes are created according to the architecture of the available worker nodes. + +## Default lanes + +The default lanes can also be installed in an existing CryoSPARC installation using the cryosparc-merlin command. + +For **x86_64** worker nodes (CPU and A100 nodes on Merlin 7): + +```bash +cryosparc-merlin lanes create default --arch-worker x86_64 +``` + +For **aarch64** worker nodes (Grace Hopper nodes on Merlin 7): + +```bash +cryosparc-merlin lanes create default --arch-worker aarch64 +``` + +!!! info "The -d / --cryosparc-path option" + If the CryoSPARC installation is not located in the default directory `/data/user/$USER/cryosparc`, specify the installation path using the `-d` option. + +The lanes are created in the CryoSPARC installation directory under `lanes/`. For example, `/data/user/$USER/cryosparc/lanes/`. + +To install all lanes stored in this directory, run: + +```bash +cryosparc-merlin lanes install --all +``` + +## Create lanes + +You can create custom CryoSPARC lanes using the `cryosparc-merlin lanes create` command: + +```bash +cryosparc-merlin lanes create \ + --name [Lane name] \ + --cache-path [Cache directory] \ + --cluster ['merlin7' or 'gmerlin7'] \ + --memory [Memory in GB] \ + --time [dd-hh:mm:ss] \ + --partition [Partition] \ + --gpus [Number of GPUs] \ + --cpus-per-task [Number of CPUs] \ + --arch-worker ['x86_64' or 'aarch64'] +``` + +The lane configuration is saved in the CryoSPARC installation directory under `lanes/`. + +!!! info "Node CPU architecture" + On Merlin 7, CPU and A100 nodes have a `x86_64` CPU architecture while the Grace-Hopper nodes have a `aarch64` CPU architecture. + +## Install lanes + +To install a previously created lane into the CryoSPARC database, run: + +```bash +cryosparc-merlin lanes install --name [Lane name] +``` + +## Remove lanes + +To remove a lane from the CryoSPARC database, run: + +```bash +cryosparc-merlin lanes remove --name [Lane name] +``` + +!!! note + Removing a lane from the CryoSPARC database does not remove its configuration file from the `lanes/` directory. + diff --git a/docs/cls/software/EM/cryosparc/patch.md b/docs/cls/software/EM/cryosparc/patch.md new file mode 100644 index 00000000..1730a2fe --- /dev/null +++ b/docs/cls/software/EM/cryosparc/patch.md @@ -0,0 +1,54 @@ +CryoSPARC periodically releases new patches, which you can view on the [CryoSPARC Updates page](https://cryosparc.com/updates). + +### 1. Ensure your CryoSPARC instance is running +If it is not running, start it with: + +```bash +cryosparc-merlin cryosparcm start +``` + +!!! info "The -d / --cryosparc-path option" + If the CryoSPARC installation is not located in the default directory `/data/user/$USER/cryosparc`, specify the installation path using the `-d` option. + +### 2. Download the patches + +```bash +cryosparc-merlin cryosparcm patch --download +``` + +### 3. Install the master node patch + +```bash +cryosparc-merlin cryosparcm patch --install +``` + +### 4. Install the worker node patch + +```bash +CRYOSPARC_PATH=/data/user/$USER/cryosparc + +mv $CRYOSPARC_PATH/cryosparc_master/cryosparc_worker_patch.tar.gz \ +$CRYOSPARC_PATH/cryosparc_worker +``` + +```bash +cryosparc-merlin cryosparcw patch +``` + +!!! warning "Worker Architecture" + Make sure you access the correct worker installation. For installations with **version 5** or above, the path to the worker installation has the architecture in the directory path. + + ```bash + /data/user/$USER/cryosparc/cryosparc_worker_x86_64 + ``` + or + + ```bash + /data/user/$USER/cryosparc/cryosparc_worker_aarch64 + ``` + +### 5. Restart CryoSPARC to finalize the update + +```bash +cryosparc-merlin cryosparcm restart +``` diff --git a/docs/cls/software/EM/cryosparc/update.md b/docs/cls/software/EM/cryosparc/update.md new file mode 100644 index 00000000..eb27de5b --- /dev/null +++ b/docs/cls/software/EM/cryosparc/update.md @@ -0,0 +1,79 @@ +Follow the steps below to update an existing CryoSPARC installation. Before starting the update, ensure that all running jobs have finished or have been terminated. Updating an active installation may interrupt running jobs. + +### 1. Shut down the CryoSPARC instance + +Before updating, fully shut down the CryoSPARC instance: + +```bash +cryosparc-merlin cryosparcm stop +``` + +!!! info "The -d / --cryosparc-path option" + If the CryoSPARC installation is not located in the default directory `/data/user/$USER/cryosparc`, specify the installation path using the `-d` option. + +### 2. Check for available updates + +Check whether a new CryoSPARC version is available: + +```bash +cryosparc-merlin cryosparcm update --check +``` + +This command checks for available updates without installing them. + +### 3. Update the CryoSPARC master + +Start the master update with: + +```bash +cryosparc-merlin cryosparcm update +``` + +The update process will: + +1. Stop the CryoSPARC instance if it is still running. +2. Download the required software updates. +3. Install the update on the CryoSPARC master. +4. Restart the CryoSPARC instance automatically if the update completes successfully. + +After the update, verify that the CryoSPARC instance is running correctly: + +```bash +cryosparc-merlin cryosparcm status +``` + +### 4. Update the CryoSPARC worker + +After updating the master, the worker installation must also be updated. + +First, change to the worker installation directory, for example: + +```bash +cd /data/user/$USER/cryosparc/cryosparc_worker/ +``` + +!!! warning "Worker Architecture" + Make sure you access the correct worker installation. For installations with **version 5** or above, the path to the worker installation has the architecture in the directory path. + + ```bash + /data/user/$USER/cryosparc/cryosparc_worker_x86_64 + ``` + or + + ```bash + /data/user/$USER/cryosparc/cryosparc_worker_aarch64 + ``` + +Copy the updated worker package from the master installation: + +```bash +cp ../cryosparc_master/cryosparc_worker.tar.gz ./ +``` + +Then run the worker update command: + +```bash +cryosparc-merlin cryosparcw update +``` + + diff --git a/docs/cls/software/EM/cryosparc/usage.md b/docs/cls/software/EM/cryosparc/usage.md new file mode 100644 index 00000000..41f53b02 --- /dev/null +++ b/docs/cls/software/EM/cryosparc/usage.md @@ -0,0 +1,93 @@ +## Connect to CryoSPARC + +Once the CryoSPARC installation is complete and the instance is running, open a web browser on your local machine and navigate to: + +`http://service03.merlin7.psi.ch:` + +The URL is displayed at the end of a successful installation. You can also list the running CryoSPARC instances with: + +```bash +cryosparc-merlin instances +``` + +The URL will open the CryoSPARC web interface, where you can log in with your CryoSPARC credentials. + +## Port conflicts + +If a CryoSPARC instance is stopped and the ports assigned to it are subsequently occupied by another process, restarting the instance may fail. + +To restart the instance, run: + +```bash +cryosparc-merlin cryosparcm restart +``` + +If the restart fails because of a port conflict, you can assign a new base port with: + +```bash +cryosparc-merlin changeport +``` + +This command automatically searches for available ports on the remote host and updates the CryoSPARC configuration accordingly. + +## SSD particle caching + +For classification, refinement, and reconstruction jobs that process large particle datasets, using local SSD storage on worker nodes can improve performance. Many cryo-EM algorithms perform random data access and multiple passes over particle data, making fast local storage particularly beneficial compared with repeated access to shared storage. + +When **Cache particle images on SSD** is enabled for a job, CryoSPARC copies the required particle data to the configured SSD cache and subsequently reads the data from the cache. If multiple jobs within the same project use the same particles, the cached data can be reused, avoiding redundant data transfers. + +When the available cache space is exhausted, CryoSPARC automatically removes older cached data to make room for new data. + +### Configure the SSD cache path + +To allow multiple jobs across worker nodes to access a shared cache, configure a shared SSD cache directory. + +Add the following line to the worker configuration file: + +`/data/user/$USER/cryosparc/cryosparc_worker/config.sh` + +```bash +export CRYOSPARC_SSD_PATH=/data/scratch/shared/$USER +``` + +!!! warning "Worker Architecture" + Make sure you access the correct worker installation. For installations with **version 5** or above, the path to the worker installation has the architecture in the directory path. + + ```bash + /data/user/$USER/cryosparc/cryosparc_worker_x86_64 + ``` + or + + ```bash + /data/user/$USER/cryosparc/cryosparc_worker_aarch64 + ``` + +### Adjust the cache copy threads + +Starting with CryoSPARC v4.3.0, multiple threads are used to copy particle data from the project directory to the SSD cache. The default is **2 threads**. + +To increase or decrease the number of copy threads, add the following line to: + +`/data/user/$USER/cryosparc/cryosparc_worker/config.sh` + +For example, to use 8 threads: + +```bash +export CRYOSPARC_CACHE_NUM_THREADS=8 +``` + +### Configure the cache lifetime + +CryoSPARC automatically removes cached particle data that has not been accessed for a specified period. The default cache lifetime is **30 days**. + +You can customize this value by setting the number of days after which unused cached data is removed. + +Add the following line to the master configuration file: + +`/data/user/$USER/cryosparc/cryosparc_master/config.sh` + +For example, to retain cached data for 15 days: + +```bash +export CRYOSPARC_SSD_CACHE_LIFETIME_DAYS=15 +``` diff --git a/docs/cls/software/EM/relion.md b/docs/cls/software/EM/relion.md new file mode 100644 index 00000000..ce4fc5c8 --- /dev/null +++ b/docs/cls/software/EM/relion.md @@ -0,0 +1,221 @@ +# RELION + +RELION (for **RE**gularised **LI**kelihood **O**ptimisatio**N**, pronounced *rely-on*) is a software package for structure determination using electron cryo-microscopy (cryo-EM). It employs an empirical Bayesian approach to perform image processing and 3D reconstruction. + +RELION is developed by the group of Sjors Scheres at the MRC Laboratory of Molecular Biology. + +For more information, see the [official RELION documentation](https://relion.readthedocs.io/en/latest/). + +## Package availability + +| Compute node type | Available | +| ----------------------- | :-------: | +| CPU | ✓ | +| NVIDIA A100 GPU | ✓ | +| NVIDIA Grace Hopper GPU | ✓ | + +## Run RELION + +### Load the RELION module + +Load the RELION module before starting the application: + +```bash +module load relion/5.0.1 +``` + +### Launch the RELION GUI + +To start the RELION GUI in either a new project directory or an existing RELION project directory, run: + +```bash +relion & +``` + +For tomographic processing, use: + +```bash +relion --tomo & +``` + +![RELION GUI 1](../../../images/CLS/relion/relion_gui_1.webp) + +### Batch submission + +RELION can submit jobs to the cluster's batch system using `sbatch`. The following settings are recommended. + +#### `Compute` tab settings + +Keep the following default values: + +| Setting | Value | +| ----------------------------------- | ---------- | +| Use parallel disk I/O? | **Yes** | +| Pre-read all particles into RAM? | **No** | +| Copy particles to scratch directory | `/scratch` | +| Combine iterations through disk? | **No** | + +##### CPU jobs + +For CPU-only jobs: + +| Setting | Value | +| --------------------- | ------ | +| Use GPU acceleration? | **No** | + +![RELION GUI 2](../../../images/CLS/relion/relion_gui_2.webp) + +##### GPU jobs + +For GPU-accelerated jobs: + +| Setting | Value | +| --------------------- | ----------------------- | +| Use GPU acceleration? | **Yes** | +| Which GPUs to use | Leave blank | +| ModelAngelo | `$CUDA_VISIBLE_DEVICES` | + +![RELION GUI 3](../../../images/CLS/relion/relion_gui_3.webp) + +#### `Running` tab settings + +Configure the following settings: + +| Setting | Value | +| -------------------------------- | ------------------------------------------- | +| Number of MPI procs | 1–16 | +| Number of threads | 1–36 | +| Submit to queue? | **Yes** | +| Queue submit command | `sbatch` | +| Time limit | `dd-hh:mm:ss` | +| Additional `sbatch` arguments | Leave blank or specify additional arguments | +| Number of nodes | `1` | +| Total memory per node | 0–818 GB | +| Number of GPUs per node | 0–4 | +| Minimum dedicated cores per node | `1` | +| Additional arguments | Leave blank | + +!!! warning "Additional `sbatch` arguments" + Additional `sbatch` arguments override the corresponding settings above. If there is a conflict, the values specified in the additional arguments take precedence. + +##### Queue names + +For **CPU jobs**, use one of the following queues: + +```text +hourly +daily +general +``` + +For **GPU jobs**, use one of the following queues: + +```text +a100-hourly +a100-daily +a100-general + +gh-hourly +gh-daily +gh-general +``` + +##### Standard submission scripts + +For **CPU jobs**: + +```text +/opt/psi/overlays/Alps/EM/relion/5.0.1/scripts/CPU.sh +``` + +For **GPU jobs**: + +```text +/opt/psi/overlays/Alps/EM/relion/5.0.1/scripts/GPU.sh +``` + +#### CPU jobs + +The following settings illustrate a typical CPU job configuration: + +![RELION GUI 4](../../../images/CLS/relion/relion_gui_4.webp) + +#### GPU jobs + +The following settings illustrate a typical GPU job configuration: + +![RELION GUI 5](../../../images/CLS/relion/relion_gui_5.webp) + +The main differences from a CPU job are: + +* **Queue name:** `a100-daily` +* **Standard submission script:** `/opt/psi/overlays/Alps/EM/relion/5.0.1/scripts/GPU.sh` +* **Use GPU acceleration:** `Yes` + +By default, the `GPU.sh` submission script requests **one GPU** per job. To request additional GPUs, specify the desired number in **Number of GPUs per node**. + +!!! tip "Resource allocation" + Request resources according to the requirements of your job. Requesting more resources than necessary may increase queue wait times and reduce cluster efficiency. + +### Benchmark (Tomography) + +#### Motion Correction +It is recommended to have the number of MPI proc has multiple of 2 or the job might fail. There is a near linear increase in performance by increasing the number of MPI procs while maintaining the same number of threads. + +| Job | Version | GPUs | MPI procs | CPUs (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ------------------ | --------- | ----------- | --------------- | ----------------- | ------------ | --------------------- | +| Motion Correction | 5.0.1 | - | 1 | 16 | 59 | 00:52:15 | +| Motion Correction | 5.0.1 | - | 2 | 16 | 118 | 00:24:51 | +| Motion Correction | 5.0.1 | - | 4 | 16 | 236 | 00:15:40 | +| Motion Correction | 5.0.1 | - | 8 | 16 | 472 | **00:08:47** | + +#### CTF estimation +Increasing the number of MPI processes seems to significantly speed up the job. + +| Job | Version | GPUs | MPI procs | CPUs (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ------------------ | --------- | ----------- | --------------- | ----------------- | ------------ | --------------------- | +| CTF estimation | 5.0.1 | - | 1 | 1 | 59 | 00:33:47 | +| CTF estimation | 5.0.1 | - | 2 | 1 | 59 | 00:16:01 | +| CTF estimation | 5.0.1 | - | 16 | 1 | 59 | **00:02:09** | + +#### Align tilt-series + +For GPU jobs, set $CUDA_VISIBLE_DEVICES + +| Job | Version | GPUs | MPI procs | CPUs (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ---------------------------- | --------- | ----------- | --------------- | ----------------- | ------------ | --------------------- | +| Align tilt-series (IMOD) | 5.0.1 | - | 1 | 1 | 59 | > 01:00:00 | +| Align tilt-series (IMOD) | 5.0.1 | - | 16 | 1 | 59 | 00:11:37 | +| Align tilt-series (AreTomo2) | 5.0.1 | 1 (A100) | 1 | 1 | 118 | > 01:00:00 | +| Align tilt-series (AreTomo2) | 5.0.1 | 1 (GH) | 1 | 1 | 204 | **00:05:59** | + +#### Reconstruct tomograms + +| Job | Version | GPUs | MPI procs | CPUs (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ------------------ | --------- | ----------- | --------------- | ----------------- | ------------ | --------------------- | +| Reconstruct tomograms | 5.0.1 | - | 1 | 16 | 59 | 00:13:38 | +| Reconstruct tomograms | 5.0.1 | - | 2 | 16 | 118 | 00:07:04 | +| Reconstruct tomograms | 5.0.1 | - | 4 | 16 | 236 | **00:03:25** | +| Reconstruct tomograms | 5.0.1 | - | 8 | 16 | 472 | 00:02:05 | + +#### Denoise tomograms + +/opt/psi/overlays/Alps/EM/cryocare/0.3.0/miniforge/envs/cryocare_0.3.0/bin +$CUDA_VISIBLE_DEVICES +Only available for A100 GPUs + +#### Extract subtomos + +| Job | Version | GPUs | MPI procs | CPUs (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ------------------ | --------- | ----------- | --------------- | ----------------- | ------------ | --------------------- | +| Extract subtomos | 5.0.1 | - | 1 | 16 | 59 | 00:09:24 | +| Extract subtomos | 5.0.1 | - | 2 | 16 | 118 | **00:02:26** | +| Extract subtomos | 5.0.1 | - | 4 | 16 | 236 | 00:02:12 | +| Extract subtomos | 5.0.1 | - | 8 | 16 | 472 | 00:01:14 | + +#### 3D initial reference + +| Job | Version | GPUs | MPI procs | CPUs (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ------------------ | --------- | ----------- | --------------- | ----------------- | ------------ | --------------------- | +| 3D initial reference | 5.0.1 | 1 (A100) | 1 | 16 | 118 | | +| 3D initial reference | 5.0.1 | 1 (GH) | 1 | 36 | 118 | | diff --git a/docs/cls/software/modules.md b/docs/cls/software/modules.md new file mode 100644 index 00000000..a11f6ef0 --- /dev/null +++ b/docs/cls/software/modules.md @@ -0,0 +1,114 @@ +# Modules available + +This page provides a list of the modules available on the cluster. The list may differ from the modules currently installed on the **Merlin 7** cluster. + +!!! tip "Finding the right module" + If you need a specific software package or version, use `module avail` or `module spider` to check which versions are available before loading the module. + +To load a module, run: `module load [module_name]` + +## Electron Microscopy + +| Software | Module name | CPU/A100 nodes | GH nodes | Run type | +| ------------- | ---------------- | :-------------: | :-------------------: | ---------------------- | +| CryoSPARC | - | ✓ | ✓ | Batch job | +| RELION | `relion` | ✓ | ✓ | Interactive, Batch job | +| LipIDens | `lipidens` | ✓ | ✓ | Interactive | +| ModelAngelo | `model_angelo` | ✓ | ✓ | Batch job | +| BindCraft | `BindCraft`[^1] | ✓ | ✓ | Batch job | +| CryoDECO | `cryodeco` | ✓ | - | Batch job | +| CryoRanker | `cryo_ief` | ✓ | - | Batch job | +| Cryo-IEF | `cryo_ief` | ✓ | - | Batch job | +| CryoSamba | `cryosamba`[^2] | ✓ | - | Interactive | +| crYOLO | `crYOLO` | ✓ | - | Interactive, Batch job | +| CryoDRGN | `cryodrgn` | ✓ | - | Batch job | +| pyEM | `pyem` | ✓ | - | Batch job | +| Phenix | `phenix`[^3] | ✓ | - | Interactive | +| MotionCor3 | `MotionCor3` | ✓ | ✓ | Batch job | +| AreTomo3 | `AreTomo3` | ✓ | - | Batch job | +| AreTomo2 | `AreTomo2` | ✓ | ✓ | Batch job | +| spIsoNet | `spIsoNet` | ✓ | - | Batch job | +| IsoNet | `IsoNet` | ✓ | - | Interactive, Batch job | +| IsoNet2 | `IsoNet2` | ✓ | - | Batch job | +| Warp | `warp` | ✓ | - | Interactive | +| ctffind4 | `ctffind4` | ✓ | ✓ | Batch job | +| ResMap | `ResMap` | ✓ | ✓ | Interactive, Batch job | +| pytom-match-pick | `pytom-match-pick` | ✓ | ✓ | Batch job | +| topaz | `topaz` | ✓ | ✓ | Batch job | + +## Protein Structure Prediction and Design + +| Software | Module name | CPU/A100 nodes | GH nodes | Run type | +| ----------------- | ------------------------------------ | :-------------: | :-------------------: | ---------------------- | +| AlphaFold 3 | `alphafold3` | ✓ | ✓ | Batch job | +| AlphaFold 2 | `alphafold`[^4] | ✓ | - | Batch job | +| dl_binder_design | dl_binder_design_module.sh [^5],[^6] | ✓ | - | Batch job | +| ROCKET | rocket_module.sh [^7] | ✓ | - | Batch job | +| GRASP-JAX | GRASP_module.sh [^8] | ✓ | - | Batch job | +| Rosetta | `Rosetta`[^9] | ✓ | ✓ | Batch job | +| Germinal | germinal_module.sh[^10] | ✓ | - | Batch job | +| Chai-1 | `chai-1` | ✓ | ✓ | Batch job | +| Boltz-2 | `boltz` | ✓ | ✓ | Batch job | +| BoltzDesign1 | BoltzDesign1_module.sh[^11] | ✓ | - | Interactive, Batch job | +| RFdiffusion | `RFdiffusion` | ✓ | - | Batch job | +| ProteinMPNN | `ProteinMPNN` | ✓ | - | Batch job | +| prosculpt | prosculpt_module.sh[^12] | ✓ | - | Interactive, Batch job | +| ColabFold | `colabfold` | ✓ | - | Batch job | +| AF3x | `af3x` | ✓ | - | Batch job | + +## Molecular and Volumetric Visualization + +| Software | Module name | CPU/A100 nodes | GH nodes | Run type | +| ----------------- | ------------------- | :-------------: | :-------------------: | ---------------------- | +| ChimeraX | `chimerax` | ✓ | - | Interactive | +| IMOD | `IMOD` | ✓ | - | Interactive, Batch job | +| MemBrain | `membrain` | ✓ | - | Interactive, Batch job | +| PyMOL | `pymol` | ✓ | - | Interactive | +| napari | `napari` | ✓ | - | Interactive | +| napari-boxmanager | `napari-boxmanager` | ✓ | - | Interactive | +| napari-tomoslice | `napari-tomoslice` | ✓ | - | Interactive | + +## Molecular Dynamics + +| Software | Module name | CPU/A100 nodes | GH nodes | Run type | +| ------------- | ------------------- | :-------------: | :-------------------: | ---------------------- | +| DNet | dnet_module.sh[^13] | ✓ | - | Interactive, Batch job | +| NAMD | `NAMD` | ✓ | ✓ | Batch job | +| VMD | `vmd` | ✓ | - | Interactive | +| GROMACS | `gromacs` | ✓ | ✓ | Batch job | +| Amber26 | `amber` | ✓ | ✓ | Batch job | +| AmberTools26 | `ambertools` | ✓ | - | Batch job | +| PLUMED | `plumed` | ✓ | ✓ | Batch job | +| LAMMPS | `lammps` | ✓ | ✓ | Batch job | +| SevenNet | `sevennet` | ✓ | ✓ | Batch job | +| dssp | `dssp` | ✓ | ✓ | Batch job | + +## Bioinformatics + +| Software | Module name | CPU/A100 nodes | GH nodes | Run type | +| ------------- | ---------------- | :-------------: | :-------------------: | ---------------------- | +| FastQC | `fastqc` | ✓ | - | Batch job | +| SRA Toolkit | `sratoolkit` | ✓ | - | Batch job | +| Entrez Direct | `edirect` | ✓ | - | Batch job | +| Cell Ranger | `cellranger` | ✓ | - | Batch job | +| kalign | `kalign` | ✓ | ✓ | Batch job | +| Refgenie | `refgenie` | ✓ | ✓ | Interactive | +| Nextflow | `nextflow` | ✓ | ✓ | Interactive, Batch job | +| nf-core | `nf-core` | ✓ | ✓ | Interactive, Batch job | +| hh-suite | `hhsuite` | ✓ | ✓ | Batch job | +| hmmer | `hmmer` | ✓ | ✓ | Batch job | +| SeqKit | `seqkit` | ✓ | - | Batch job | + +[^1]: `module use Alps_A100; module load BindCraft` (A100 GPU nodes) +[^2]: `source /data/project/cls/shared/software/cryosamba/dev_2025_11/cryosamba_module.sh load` +[^3]: `module use MX; module load phenix/phenix-1.20-4459` +[^4]: `module use MX; module load alphafold` +[^5]: `source /data/project/cls/shared/software/dl_binder_design/1.0.0/dl_binder_design_module.sh proteinmpnn_binder_design` +[^6]: `source /data/project/cls/shared/software/dl_binder_design/1.0.0/dl_binder_design_module.sh af2_binder_design` +[^7]: `source /data/project/cls/shared/software/rocket/0.2.0/rocket_module.sh load` +[^8]: `source /data/project/cls/shared/software/GRASP-JAX/dev_2025_11/GRASP_module.sh load` +[^9]: `module use MX; module load Rosetta` +[^10]: `source /data/project/cls/shared/software/germinal/dev_2025_10/germinal/germinal_module.sh load` +[^11]: `source /data/project/cls/shared/software/BoltzDesign1/dev_2025_07/BoltzDesign1_module.sh load` +[^12]: `source /data/project/cls/shared/software/prosculpt/1.1.4/prosculpt_module.sh load` +[^13]: `source /data/project/cls/shared/software/DNet/dnet_module.sh load` diff --git a/docs/cls/software/molecular_dynamics/gromacs.md b/docs/cls/software/molecular_dynamics/gromacs.md new file mode 100644 index 00000000..3ef69a74 --- /dev/null +++ b/docs/cls/software/molecular_dynamics/gromacs.md @@ -0,0 +1,129 @@ +# GROMACS + +[GROMACS](https://manual.gromacs.org/documentation/current/) is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles and is a community-driven project. + +## Package availability + +| Compute node type | Available | +| ----------------------- | :-------: | +| CPU | ✓ | +| NVIDIA A100 GPU | ✓ | +| NVIDIA Grace Hopper GPU | ✓ | + +## Run GROMACS + +GROMACS is available in both MPI and non-MPI versions. In single-GPU-node tests, both versions produced comparable results. For multi-node runs, use the MPI version. + +To load the non-MPI version: + +```bash +module load gcc/13.4.0 gromacs/2026.4 +``` + +To load the MPI version: + +```bash +module load gcc/13.4.0 mpich/5.0.1 gromacs/2026.4 +``` + +For the examples and benchmarks in this documentation, we use the non-MPI version. + +### NVIDIA A100 GPU + +```bash +#!/bin/bash +#SBATCH --time=01:00:00 +#SBATCH --output=gromacs_stdout.txt +#SBATCH --error=gromacs_stderr.txt +#SBATCH --nodes=1 +#SBATCH --ntasks-per-node=8 +#SBATCH --cpus-per-task=8 +#SBATCH --cluster=gmerlin7 +#SBATCH --partition=a100-hourly +#SBATCH --gpus=4 +#SBATCH --mem=0 +#SBATCH --exclusive + +module purge +module use Alps_A100 +module load gcc/13.4.0 gromacs/2026.4 + +gmx mdrun \ + -s /path/to/input/topol.tpr \ + -ntmpi $SLURM_NTASKS_PER_NODE \ + -ntomp $SLURM_CPUS_PER_TASK \ + -nb gpu \ + -pme gpu \ + -npme 1 \ + -update gpu \ + -bonded gpu \ + -pin on \ + -v \ + -gpu_id $CUDA_VISIBLE_DEVICES +``` + +### Grace Hopper GPU + +```bash +#!/bin/bash +#SBATCH --time=01:00:00 +#SBATCH --output=gromacs_stdout.txt +#SBATCH --error=gromacs_stderr.txt +#SBATCH --nodes=1 +#SBATCH --ntasks-per-node=8 +#SBATCH --cpus-per-task=36 +#SBATCH --cluster=gmerlin7 +#SBATCH --partition=gh-hourly +#SBATCH --gpus=4 +#SBATCH --mem=0 +#SBATCH --exclusive + +module purge +module load gcc/13.4.0 gromacs/2026.4 + +export GMX_DISABLE_DIRECT_GPU_COMM=1 + +gmx mdrun \ + -s /path/to/input/topol.tpr \ + -ntmpi $SLURM_NTASKS_PER_NODE \ + -ntomp $SLURM_CPUS_PER_TASK \ + -nb gpu \ + -pme gpu \ + -npme 1 \ + -update gpu \ + -bonded gpu \ + -pin on \ + -v \ + -gpu_id $CUDA_VISIBLE_DEVICES +``` + +!!! tip "Resource allocation" + Request resources according to the requirements of your job. Requesting more resources than necessary may increase queue wait times and reduce cluster efficiency. + +## Benchmark results +We performed a single-node performance benchmark using the **Satellite Tobacco Mosaic Virus (STMV)** benchmark, comparing performance on **A100** and **Grace Hopper** nodes. + +For most of the tested configurations, **8 tasks per node** provided the best performance and appears to be the optimal configuration for single-node runs. For a single GPU, **4 tasks per node** has a slight advantage in performance. + +| Run | Version | GPUs | CPU (Threads) | N tasks | Memory (Gb) | Performance (ns/day) | +| ---- | ----------- | --------- | ------------- | -------- | ----------------- | --------------------- | +| STMV | 2026.3 | 1 A100 | 8 per task | 2 | 118 | 21.9 | +| STMV | 2026.3 | 1 A100 | 4 per task | 4 | 118 | **23.0** | +| STMV | 2026.3 | 1 A100 | 2 per task | 8 | 118 | 22.8 | +| STMV | 2026.3 | 1 GH | 36 per task | 2 | 118 | 44.1 | +| STMV | 2026.3 | 1 GH | 18 per task | 4 | 118 | **45.9** | +| STMV | 2026.3 | 1 GH | 9 per task | 8 | 118 | 44.7 | +| | | | | | | | +| STMV | 2026.3 | 2 A100 | 16 per task | 2 | 236 | 26.0 | +| STMV | 2026.3 | 2 A100 | 8 per task | 4 | 236 | 39.3 | +| STMV | 2026.3 | 2 A100 | 4 per task | 8 | 236 | **44.8** | +| STMV | 2026.3 | 2 A100 | 2 per task | 16 | 236 | 41.3 | +| STMV | 2026.3 | 2 GH | 36 per task | 4 | 236 | 75.4 | +| STMV | 2026.3 | 2 GH | 18 per task | 8 | 236 | **85.2** | +| | | | | | | | +| STMV | 2026.3 | 4 A100 | 16 per task | 4 | 472 | 73.8 | +| STMV | 2026.3 | 4 A100 | 8 per task | 8 | 472 | **85.7** | +| STMV | 2026.3 | 4 A100 | 4 per task | 16 | 472 | 74.6 | +| STMV | 2026.3 | 4 GH | 72 per task | 4 | 472 | 137.1 | +| STMV | 2026.3 | 4 GH | 36 per task | 8 | 472 | **153.4** | +| STMV | 2026.3 | 4 GH | 18 per task | 16 | 472 | 128.2 | diff --git a/docs/cls/software/molecular_dynamics/namd.md b/docs/cls/software/molecular_dynamics/namd.md new file mode 100644 index 00000000..ab556a39 --- /dev/null +++ b/docs/cls/software/molecular_dynamics/namd.md @@ -0,0 +1,116 @@ +# NAMD + +NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. + +For more information, consult the official [NAMD website](https://www.ks.uiuc.edu/Research/namd/). + +## Package availability + +| Compute node type | Available | +| ----------------------- | :-------: | +| CPU | ✓ | +| NVIDIA A100 GPU | ✓ | +| NVIDIA Grace Hopper GPU | ✓ | + +## Run NAMD + +Below are some examples of how to run NAMD’s GPU-resident mode, which executes entire molecular dynamics simulations directly on the GPU. This approach can deliver significant performance improvements by minimizing data transfers between the GPU and CPU. + +### NVIDIA A100 GPU + +```bash +#!/bin/bash -e + +#SBATCH --job-name=namd +#SBATCH --cluster=gmerlin7 +#SBATCH --partition=a100-hourly +#SBATCH --nodes=1 +#SBATCH --hint=nomultithread +#SBATCH --cpus-per-task=8 +#SBATCH --gres=gpu:1 +#SBATCH --time=01:00:00 +#SBATCH --output=namd_%j.log + +module purge +module use Alps_A100 +module load NAMD + +namd3 +p8 +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +### Grace Hopper GPU + +```bash +#!/bin/bash -e + +#SBATCH --job-name=namd +#SBATCH --cluster=gmerlin7 +#SBATCH --partition=gh-hourly +#SBATCH --nodes=1 +#SBATCH --cpus-per-task=8 +#SBATCH --gres=gpu:1 +#SBATCH --time=01:00:00 +#SBATCH --output=namd_%j.log + +module purge +module load NAMD + +namd3 +p8 +setcpuaffinity +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +!!! tip "Resource allocation" + Request resources according to the requirements of your job. Requesting more resources than necessary may increase queue wait times and reduce cluster efficiency. + +## Benchmark results + +We performed a performance benchmark using the Satellite Tobacco Mosaic Virus (STMV), comparing GPU-resident runs on **1, 2, and 4 A100 and GH GPUs** with a **CPU-only** execution. All simulations were performed in the **NVE (constant-energy) ensemble** using the following commands. + +### CPU-only + +The CPU-only benchmark used **128 cores per node, 4 processes per node, and 3 nodes**: + +```bash +srun --cpu-bind=cores namd3 +p4 stmv.namd +``` + +### A100 GPUs + +#### 1 GPU + +```bash +namd3 +p8 +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +#### 2 GPUs + +```bash +namd3 +p15 +pmepes 7 +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +#### 4 GPUs + +```bash +namd3 +p29 +pmepes 5 +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +### GH GPUs + +#### 1 GPU + +```bash +namd3 +p8 +setcpuaffinity +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +#### 2 GPUs + +```bash +namd3 +p15 +pmepes 7 +setcpuaffinity +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +#### 4 GPUs + +```bash +namd3 +p29 +pmepes 5 +setcpuaffinity +devices ${CUDA_VISIBLE_DEVICES} stmv.namd +``` + +![NAMD performance results](../../../images/CLS/namd/namd_performance.png.webp) diff --git a/docs/cls/software/protein_structure_prediction/alphafold3.md b/docs/cls/software/protein_structure_prediction/alphafold3.md new file mode 100644 index 00000000..236b9ad5 --- /dev/null +++ b/docs/cls/software/protein_structure_prediction/alphafold3.md @@ -0,0 +1,227 @@ +AlphaFold 3, introduced in May 2024 by Google DeepMind in collaboration with Isomorphic Labs, marks a significant advance in computational biology. Building on the groundbreaking success of AlphaFold 2 in protein structure prediction, AlphaFold 3 expands the scope of structure prediction to model the three-dimensional structures and interactions of a broader range of biomolecules, including proteins, DNA, RNA, ligands, and ions. + +[AlphaFold 3 GitHub repository](https://github.com/google-deepmind/alphafold3) + +## Package availability + +| Compute node type | Available | +|-------------------------|:---------:| +| CPU | ✓ | +| NVIDIA A100 GPU | ✓ | +| NVIDIA Grace Hopper GPU | ✓ | + +## Request model parameters + +To run AlphaFold 3, you must obtain the model parameters separately. + +To request access, please complete the [AlphaFold 3 model parameters request form](https://forms.gle/svvpY4u2jsHEwWYS6) provided by Google DeepMind. Access is granted at the sole discretion of Google DeepMind, and responses to requests typically take **2–3 business days**. + +You may use the AlphaFold 3 model parameters only if they are provided directly by Google DeepMind. Use of the model parameters is subject to the applicable [terms of use](https://github.com/google-deepmind/alphafold3/blob/main/WEIGHTS_TERMS_OF_USE.md). + +## Run AlphaFold 3 + +AlphaFold 3 can be executed in multiple stages to optimise resource utilisation. In particular, the CPU-only data pipeline can be separated from model inference, which requires a GPU. This allows each stage to run on the most appropriate and cost-effective compute resources. + +The genetic database searches performed during the data pipeline can also be significantly accelerated by splitting the databases into multiple shards and searching these shards in parallel. For this reason, we recommend separating an AlphaFold 3 analysis into the following Slurm jobs: + +1. **Data pipeline** — CPU-only; run on a full CPU node. +2. **Model inference** — GPU-accelerated; run on a GPU node. + +!!! note "Model parameters" + You must provide the path to the AlphaFold 3 model parameters directory supplied to you or your group by Google DeepMind using the `--model_dir` flag. + +### 1. Data pipeline (CPU only) + +The following example runs the data pipeline on a full CPU node. The database searches are distributed across multiple shards to maximise CPU utilisation. + +```bash +#!/bin/bash -e + +#SBATCH --job-name=alphafold3_data_pipeline +#SBATCH --cluster=merlin7 +#SBATCH --partition=hourly +#SBATCH --ntasks=1 +#SBATCH --cpus-per-task=256 +#SBATCH --time=01:00:00 +#SBATCH --output=alphafold3_data_pipeline_%j.log +#SBATCH --mem=0 +#SBATCH --exclusive + +module purge +module load alphafold3 + +srun --cpu-bind=threads alphafold3_sharded \ + --json_path /path/to/json/input.json \ + --model_dir /path/to/model_parameters \ + --output_dir /path/to/output \ + --run_inference=false \ + --run_data_pipeline=true \ + --jackhmmer_n_cpu=1 \ + --jackhmmer_max_parallel_shards=$((SLURM_CPUS_PER_TASK / 4)) \ + --nhmmer_n_cpu=1 \ + --nhmmer_max_parallel_shards=$((SLURM_CPUS_PER_TASK / 4)) +``` + +#### Required parameters + +The following parameters must be specified for every AlphaFold 3 run: + +| Parameter | Description | +|----------------|--------------------------------------------------------------------| +| `--json_path` | Path to the input JSON file. | +| `--model_dir` | Path to the directory containing the AlphaFold 3 model parameters. | +| `--output_dir` | Path to the directory where the results will be saved. | + +#### Performance parameters + +The following parameters control the number of database shards searched in parallel: + +| Parameter | Description | +|-----------------------------------|----------------------------------------------------------------------| +| `--jackhmmer_max_parallel_shards` | Maximum number of database shards searched in parallel by JackHMMER. | +| `--nhmmer_max_parallel_shards` | Maximum number of database shards searched in parallel by NHMMER. | + + +!!! note "If the job takes a long time to start" + In the example above, each database search uses one CPU per process, with up to 64 shards searched in parallel. Four protein databases are searched concurrently, resulting in: + + 1 CPU × 64 parallel shards × 4 concurrent database searches = 256 CPUs + + This configuration is designed to utilise the full 256-CPU node. + + If the job is slow to start due to limited cluster availability, you can reduce the number of parallel shards to request fewer resources by reducing **--cpus-per-task** and **--mem**. + +#### Non-sharded databases run + +You can also run the data pipeline on non-sharded databases. The benchmark results between the two option are displayed below. + +```bash +#!/bin/bash -e + +#SBATCH --job-name=alphafold3_data_pipeline +#SBATCH --cluster=merlin7 +#SBATCH --partition=hourly +#SBATCH --ntasks=1 +#SBATCH --cpus-per-task=256 +#SBATCH --time=01:00:00 +#SBATCH --output=alphafold3_data_pipeline_%j.log +#SBATCH --mem=0 +#SBATCH --exclusive + +module purge +module load alphafold3 + +srun --cpu-bind=threads alphafold3 \ + --json_path /path/to/json/input.json \ + --model_dir /path/to/model_parameters \ + --output_dir /path/to/output \ + --run_inference=false \ + --run_data_pipeline=true \ + --jackhmmer_n_cpu=$((SLURM_CPUS_PER_TASK / 4)) \ + --nhmmer_n_cpu=$((SLURM_CPUS_PER_TASK / 4)) +``` + +#### Benchmark results + +These results are representative of a large, heterogeneous protein–protein complex prediction workload using AlphaFold 3. The input consists of two protein chains, with chain A being an exceptionally long, highly repetitive and compositionally complex protein (approximately 3,000 residues) and chain B being a long, predominantly helical protein (approximately 600 residues). Thus, the benchmark represents a large protein–protein complex prediction with substantial sequence length and structural complexity, rather than a typical small single-protein prediction. + +They also compare running with [sharded databases](https://github.com/google-deepmind/alphafold3/blob/main/docs/performance.md#sharded-genetic-databases) and non-sharded. + +| Run | Version | CPU (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ------------------ | --------------- | ------------- | ----------------- | ---------------------- | +| alphafold3_sharded | 3.0.4 | 256 | 472 | **00:12:10** | +| alphafold3 | 3.0.4 | 256 | 472 | 00:29:05 | +| | | | | | +| alphafold3_sharded | 3.0.4 | 128 | 236 | **00:24:23** | +| alphafold3 | 3.0.4 | 128 | 236 | 00:29:58 | +| | | | | | +| alphafold3_sharded | 3.0.4 | 64 | 118 | **00:32:46** | +| alphafold3 | 3.0.4 | 64 | 118 | 00:37:08 | +| | | | | | +| alphafold3_sharded | 3.0.4 | 32 | 59 | **01:02:00** | +| alphafold3 | 3.0.4 | 32 | 59 | 01:06:52 | + +### 2. Inference (GPU only) + +The inference stage requires a GPU and should be run separately from the CPU-only data pipeline. The following examples show how to run AlphaFold 3 on an NVIDIA A100 GPU node and a Grace Hopper GPU node. + +### NVIDIA A100 GPU + +```bash +#!/bin/bash -e + +#SBATCH --job-name=alphafold3_inference +#SBATCH --cluster=gmerlin7 +#SBATCH --partition=a100-hourly +#SBATCH --ntasks=1 +#SBATCH --cpus-per-task=32 +#SBATCH --time=01:00:00 +#SBATCH --output=alphafold3_inference_%j.log +#SBATCH --mem=100G +#SBATCH --gres=gpu:1 + +module purge +module load alphafold3 + +srun alphafold3 \ + --json_path /path/to/input.json \ + --model_dir /path/to/model_params \ + --output_dir /path/to/outputs \ + --run_inference=true \ + --run_data_pipeline=false +``` + +### Grace Hopper GPU + +```bash +#!/bin/bash -e + +#SBATCH --job-name=alphafold3_inference +#SBATCH --cluster=gmerlin7 +#SBATCH --partition=gh-hourly +#SBATCH --ntasks=1 +#SBATCH --cpus-per-task=72 +#SBATCH --time=01:00:00 +#SBATCH --output=alphafold3_inference_%j.log +#SBATCH --mem=100G +#SBATCH --gres=gpu:1 + +module purge +module load alphafold3 + +srun alphafold3 \ + --json_path /path/to/input.json \ + --model_dir /path/to/model_params \ + --output_dir /path/to/outputs \ + --run_inference=true \ + --run_data_pipeline=false +``` + +!!! note "Resource allocation" + Adapt the `sbatch` script to the resources required by your job. For example, you may need to increase the number of GPUs or the amount of memory allocated. + + +#### Required parameters + +| Parameter | Description | +| -------------- | -------------------------------------------------------------------| +| `--json_path` | Path to the input JSON file. | +| `--model_dir` | Path to the directory containing the AlphaFold 3 model parameters. | +| `--output_dir` | Path to the directory where the results will be saved. | + +#### Optional parameters + +To use the JAX persistent compilation cache and avoid unnecessary recompilation between runs, specify: + +```bash +--jax_compilation_cache_dir /path/to/jax_cache +``` + +#### Benchmark results + +| Run | Version | GPUs | CPU (Threads) | Memory (Gb) | Total time (HH:MM:SS) | +| ------------------ | --------- | ----------- | ----------------- | ------------ | --------------------- | +| GH | 3.0.4 | 1 | 72 | 100 | **00:26:16** | +| A100 | 3.0.3[^1] | 1 | 32 | 100 | 00:41:47 | + +[^1]: Running with version 3.0.4 resulted in CUDA OOM errors. diff --git a/docs/images/CLS/cryosparc/cryosparc_benchmarks_overall_times.png b/docs/images/CLS/cryosparc/cryosparc_benchmarks_overall_times.png new file mode 100644 index 00000000..9a0a1391 Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_benchmarks_overall_times.png differ diff --git a/docs/images/CLS/cryosparc/cryosparc_benchmarks_speedup_common.png b/docs/images/CLS/cryosparc/cryosparc_benchmarks_speedup_common.png new file mode 100644 index 00000000..d03420bd Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_benchmarks_speedup_common.png differ diff --git a/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_1.webp b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_1.webp new file mode 100644 index 00000000..e13fbb82 Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_1.webp differ diff --git a/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_2.webp b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_2.webp new file mode 100644 index 00000000..32cfadc6 Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_2.webp differ diff --git a/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_3.webp b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_3.webp new file mode 100644 index 00000000..118fcf55 Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_3.webp differ diff --git a/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_4.webp b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_4.webp new file mode 100644 index 00000000..02fb205b Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_4.webp differ diff --git a/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_5.webp b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_5.webp new file mode 100644 index 00000000..ff04c5a6 Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_5.webp differ diff --git a/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_6.webp b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_6.webp new file mode 100644 index 00000000..d91e3acb Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_6.webp differ diff --git a/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_7.webp b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_7.webp new file mode 100644 index 00000000..dc1af006 Binary files /dev/null and b/docs/images/CLS/cryosparc/cryosparc_issues_correct_symlinks_7.webp differ diff --git a/docs/images/CLS/data_transfer/data_transfer_scopem_1.webp b/docs/images/CLS/data_transfer/data_transfer_scopem_1.webp new file mode 100644 index 00000000..0fd087a1 Binary files /dev/null and b/docs/images/CLS/data_transfer/data_transfer_scopem_1.webp differ diff --git a/docs/images/CLS/data_transfer/data_transfer_scopem_2.webp b/docs/images/CLS/data_transfer/data_transfer_scopem_2.webp new file mode 100644 index 00000000..cdd10c3c Binary files /dev/null and b/docs/images/CLS/data_transfer/data_transfer_scopem_2.webp differ diff --git a/docs/images/CLS/data_transfer/data_transfer_scopem_3.webp b/docs/images/CLS/data_transfer/data_transfer_scopem_3.webp new file mode 100644 index 00000000..fc191163 Binary files /dev/null and b/docs/images/CLS/data_transfer/data_transfer_scopem_3.webp differ diff --git a/docs/images/CLS/data_transfer/data_transfer_tips.png b/docs/images/CLS/data_transfer/data_transfer_tips.png new file mode 100644 index 00000000..8e4ce5cc Binary files /dev/null and b/docs/images/CLS/data_transfer/data_transfer_tips.png differ diff --git a/docs/images/CLS/namd/namd_performance.png.webp b/docs/images/CLS/namd/namd_performance.png.webp new file mode 100644 index 00000000..e6742b65 Binary files /dev/null and b/docs/images/CLS/namd/namd_performance.png.webp differ diff --git a/docs/images/CLS/relion/relion_gui_1.webp b/docs/images/CLS/relion/relion_gui_1.webp new file mode 100644 index 00000000..1d8e627e Binary files /dev/null and b/docs/images/CLS/relion/relion_gui_1.webp differ diff --git a/docs/images/CLS/relion/relion_gui_2.webp b/docs/images/CLS/relion/relion_gui_2.webp new file mode 100644 index 00000000..9f234bd8 Binary files /dev/null and b/docs/images/CLS/relion/relion_gui_2.webp differ diff --git a/docs/images/CLS/relion/relion_gui_3.webp b/docs/images/CLS/relion/relion_gui_3.webp new file mode 100644 index 00000000..030e090a Binary files /dev/null and b/docs/images/CLS/relion/relion_gui_3.webp differ diff --git a/docs/images/CLS/relion/relion_gui_4.webp b/docs/images/CLS/relion/relion_gui_4.webp new file mode 100644 index 00000000..56c40210 Binary files /dev/null and b/docs/images/CLS/relion/relion_gui_4.webp differ diff --git a/docs/images/CLS/relion/relion_gui_5.webp b/docs/images/CLS/relion/relion_gui_5.webp new file mode 100644 index 00000000..56ada582 Binary files /dev/null and b/docs/images/CLS/relion/relion_gui_5.webp differ diff --git a/mkdocs.yml b/mkdocs.yml index 7d209fb3..76d5d85e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -219,7 +219,33 @@ nav: - MeG: - meg/introduction.md - meg/contact.md - - meg/migration-to-merlin7.md + - meg/migration-to-merlin7.md + - CLS: + # - Archiving: + - Data Transfer: + - cls/data_transfer/emf.md + - cls/data_transfer/scopem.md + - cls/data_transfer/unil_s3_bucket.md + - cls/data_transfer/uni_basel.md + - Software: + - cls/software/modules.md + - Electron Microscopy: + - cls/software/EM/relion.md + - CryoSPARC: + - cls/software/EM/cryosparc/installation.md + - cls/software/EM/cryosparc/usage.md + - cls/software/EM/cryosparc/lanes.md + - cls/software/EM/cryosparc/update.md + - cls/software/EM/cryosparc/patch.md + - cls/software/EM/cryosparc/issues.md + - cls/software/EM/cryosparc/benchmarks.md + - Protein Structure Prediction and Design: + - cls/software/protein_structure_prediction/alphafold3.md + # - Molecular and Volumetric Visualization: + - Molecular Dynamics: + - cls/software/molecular_dynamics/namd.md + - cls/software/molecular_dynamics/gromacs.md + # - Bioinformatics: - PSI@CSCS: - cscs-userlab/introduction.md - cscs-userlab/transfer-data.md