Add scopem and alphafold3 documentation

This commit is contained in:
2026-09-02 17:58:00 +02:00
parent f3dacdcb38
commit e4e5e0e9dc
6 changed files with 236 additions and 94 deletions
+88
View File
@@ -0,0 +1,88 @@
# EMF PSI
The storage systems used by the Electron Microscopy Facility at PSI (EMF) are mounted on the Merlin 7 login nodes. You can copy files from the EMF storage to your home or project directory on Merlin 7, or transfer them from Merlin 7 to your local computer.
## Within Merlin 7
The EMF files are accessible through the `/emf` mount on the Merlin 7 cluster. However, the files are **not stored on Merlin 7**; the mount provides access to files that remain on the EMF storage system.
Before working with the files, copy them to your home directory or project directory on Merlin 7.
### `rsync`
We recommend using `rsync` to copy files because it preserves file attributes and provides useful information about the transfer.
Run:
```bash
rsync -avAHXS sourcedir/ destdir/
```
If you need to change the group ownership of the copied files, use:
```bash
rsync -avAHXS --chown=:$GROUP_NAME sourcedir/ destdir/
```
Replace `$GROUP_NAME` with the name of the group that should own the files.
!!! note
The trailing `/` in `sourcedir/` is significant. It means that the **contents** of `sourcedir` are copied to `destdir`, rather than the directory itself.
### `cp`
Alternatively, you can use `cp`:
```bash
cp -r sourcedir/ destdir/
```
## From Merlin 7 to your local machine
You can transfer files from Merlin 7 to your local computer using `rsync`, `scp`, or a graphical application such as FileZilla.
### `rsync`
`rsync` is recommended for transferring large amounts of data or for transfers that may need to be resumed.
For example:
```bash
rsync -avAHXS $USER@login001.merlin7.psi.ch:/data/project/general/myproject/ ~/localdata/
```
### `scp`
For a simple recursive copy, you can use `scp`:
```bash
scp -r $USER@login001.merlin7.psi.ch:/data/project/general/myproject/ ~/localdata/
```
### FileZilla
You can also use [FileZilla](https://filezilla-project.org/) to transfer files between your local computer and Merlin 7.
Open FileZilla and create a connection to Merlin 7 using the following settings:
| Setting | Value |
| ------------ | ------------------------------------------------------ |
| **Host** | `login001.merlin7.psi.ch` or `login002.merlin7.psi.ch` |
| **Username** | Your Merlin 7 username |
| **Password** | Your Merlin 7 password |
| **Port** | `22` |
| **Protocol** | `SFTP - SSH File Transfer Protocol` |
Once connected, the files on Merlin 7 will be displayed in the **Remote site** panel.
You can transfer files between your computer and Merlin 7 by dragging them between the **Remote site** and **Local site** panels. Alternatively, right-click a file or directory and select **Download** or **Upload**.
!!! tip
To improve file transfer speeds, increase the maximum number of simultaneous transfers in FileZilla:
1. Open **Edit > Settings**.
2. Select **Transfers**.
3. Under **Maximum simultaneous transfers**, set **Maximum simultaneous transfers** to `10`.
4. Click **OK** to save the changes.
![FileZilla transfer settings](../../images/CLS/data_transfer/data_transfer_tips.png)
+34 -14
View File
@@ -1,4 +1,4 @@
# ETH Zurich ScopeM
# ScopeM ETH Zurich
This article explains how to transfer data remotely between the [ETH Zurich ScopeM facility](https://scopem.ethz.ch/) and Merlin 7 using the **SFTP (SSH File Transfer Protocol)**.
@@ -6,6 +6,16 @@ This article explains how to transfer data remotely between the [ETH Zurich Scop
One of the easiest ways to transfer files using SFTP is through **NoMachine** and **FileZilla**.
!!! tip
To improve file transfer speeds, increase the maximum number of simultaneous transfers in FileZilla:
1. Open **Edit > Settings**.
2. Select **Transfers**.
3. Under **Maximum simultaneous transfers**, set the value to **10**.
4. Click **OK** to save the changes.
![FileZilla transfer settings](../../images/CLS/data_transfer/data_transfer_tips.png)
### 1. Connect to Merlin 7
Connect to Merlin 7 using NoMachine.
@@ -40,16 +50,24 @@ You can transfer files between ScopeM and Merlin 7 by dragging them between the
## Using the terminal
ScopeM endpoints can also be accessed directly from a Merlin 7 terminal using `sftp`.
ScopeM endpoints can also be accessed directly from a Merlin 7 login node using `sftp`.
### Interactive SFTP session
### **Interactive SFTP session**
Connect to a ScopeM endpoint with:
```bash
sftp -P 5007 cryoem1@scopem-krios1data.ethz.ch
sftp -P 5007 <username>@scopem-krios1data.ethz.ch
```
If you encounter an error such as **"Too many authentication failures"**, force SFTP to use password authentication:
```bash
sftp -o PubkeyAuthentication=no -o PreferredAuthentications=password -P 5007 <username>@scopem-krios1data.ethz.ch
```
Enter your password when prompted.
Once connected, use the `get` command to download files or directories. For example:
```text
@@ -62,29 +80,31 @@ Retrieving home/P1/J1
The `-r` option recursively downloads the specified directory and its contents.
### Download a directory directly
### **Download a directory directly**
You can also specify the remote path directly from the command line:
You can also specify the remote path directly on the command line:
```bash
sftp -r -P 5007 cryoem1@scopem-krios1data.ethz.ch:home/P1 ./
sftp -r -P 5007 <username>@scopem-krios1data.ethz.ch:home/P1 ./
```
You will be prompted for your password:
```text
cryoem1@scopem-krios1data.ethz.ch's password:
<username>@scopem-krios1data.ethz.ch's password:
Connected to scopem-krios1data.ethz.ch.
Fetching /home/P1/ to ./P1
Retrieving /home/P1
Retrieving /home/P1/J1
```
### Configure SSH for easier access
### **Configure SSH for easier access**
If you frequently transfer data from ScopeM, you can simplify the connection by adding the ScopeM endpoints to your SSH configuration file:
```bash
```text
$HOME/.ssh/config
```
@@ -92,13 +112,13 @@ For example:
```text
Host krios1 krios2 krios3
Hostname scopem-%hdata.ethz.ch
HostName scopem-%hdata.ethz.ch
Host scopem-krios*data.ethz.ch
Port 5007
Host *.ethz.ch
User cryoem1
User <username>
```
With this configuration, the hostname, port, and username do not need to be specified for every connection.
@@ -106,7 +126,7 @@ With this configuration, the hostname, port, and username do not need to be spec
For example, instead of:
```bash
sftp -r -P 5007 cryoem1@scopem-krios1data.ethz.ch:home/P1 ./
sftp -r -P 5007 <username>@scopem-krios1data.ethz.ch:home/P1 ./
```
you can use:
@@ -116,4 +136,4 @@ sftp -r krios1:home/P1 ./
```
!!! note "SSH configuration"
Adjust the `User` entry in `$HOME/.ssh/config` if your ScopeM username differs from `cryoem1`.
Adjust the `User` entry in `$HOME/.ssh/config` to match your username.
+46 -33
View File
@@ -2,7 +2,6 @@
This page provides a list of the modules available on the cluster. The list may differ from the modules currently installed on the **Merlin 7** cluster.
!!! tip "Finding the right module"
If you need a specific software package or version, use `module avail` or `module spider` to check which versions are available before loading the module.
@@ -10,34 +9,39 @@ To load a module, run: `module load [module_name]`
## Electron Microscopy
| Software | Module name | CPU/A100 nodes | GH nodes | Run type |
| ------------- | ---------------- | :-------------: | :-------------------: | ---------------------- |
| CryoSPARC | - | ✓ | ✓ | Batch job |
| RELION | `relion` | ✓ | ✓ (IMOD not available) | Interactive, Batch job |
| LipIDens | `lipidens` | ✓ | ✓ | Interactive |
| ModelAngelo | `model_angelo` | ✓ | ✓ | Batch job |
| BindCraft | `BindCraft`[^1] | ✓ | ✓ | Batch job |
| CryoDECO | `cryodeco` | ✓ | - | Batch job |
| CryoRanker | `cryo_ief` | ✓ | - | Batch job |
| Cryo-IEF | `cryo_ief` | ✓ | - | Batch job |
| CryoSamba | `cryosamba`[^2] | ✓ | - | Interactive |
| crYOLO | `crYOLO` | ✓ | - | Interactive, Batch job |
| CryoDRGN | `cryodrgn` | ✓ | - | Batch job |
| pyEM | `pyem` | ✓ | - | Batch job |
| Phenix | `phenix`[^3] | ✓ | - | Interactive |
| MotionCor3 | `MotionCor3` | ✓ | - | Batch job |
| AreTomo3 | `AreTomo3` | ✓ | - | Batch job |
| AreTomo2 | `AreTomo2` | ✓ | - | Batch job |
| spIsoNet | `spIsoNet` | ✓ | - | Batch job |
| IsoNet | `IsoNet` | ✓ | - | Interactive, Batch job |
| Warp | `warp` | ✓ | - | Interactive |
| Software | Module name | CPU/A100 nodes | GH nodes | Run type |
| ------------- | ---------------- | :-------------: | :-------------------: | ---------------------- |
| CryoSPARC | - | ✓ | ✓ | Batch job |
| RELION | `relion` | ✓ | ✓ (IMOD not available) | Interactive, Batch job |
| LipIDens | `lipidens` | ✓ | ✓ | Interactive |
| ModelAngelo | `model_angelo` | ✓ | ✓ | Batch job |
| BindCraft | `BindCraft`[^1] | ✓ | ✓ | Batch job |
| CryoDECO | `cryodeco` | ✓ | - | Batch job |
| CryoRanker | `cryo_ief` | ✓ | - | Batch job |
| Cryo-IEF | `cryo_ief` | ✓ | - | Batch job |
| CryoSamba | `cryosamba`[^2] | ✓ | - | Interactive |
| crYOLO | `crYOLO` | ✓ | - | Interactive, Batch job |
| CryoDRGN | `cryodrgn` | ✓ | - | Batch job |
| pyEM | `pyem` | ✓ | - | Batch job |
| Phenix | `phenix`[^3] | ✓ | - | Interactive |
| MotionCor3 | `MotionCor3` | ✓ | | Batch job |
| AreTomo3 | `AreTomo3` | ✓ | - | Batch job |
| AreTomo2 | `AreTomo2` | ✓ | | Batch job |
| spIsoNet | `spIsoNet` | ✓ | - | Batch job |
| IsoNet | `IsoNet` | ✓ | - | Interactive, Batch job |
| IsoNet2 | `IsoNet2` | ✓ | - | Batch job |
| Warp | `warp` | ✓ | - | Interactive |
| ctffind4 | `ctffind4` | ✓ | ✓ | Batch job |
| ResMap | `ResMap` | ✓ | ✓ | Interactive, Batch job |
| pytom-match-pick | `pytom-match-pick` | ✓ | ✓ | Batch job |
| topaz | `topaz` | ✓ | ✓ | Batch job |
## Protein Structure Prediction and Design
| Software | Module name | CPU/A100 nodes | GH nodes | Run type |
| ----------------- | ------------------------------------ | :-------------: | :-------------------: | ---------------------- |
| AlphaFold 3 | `alphafold3` | ✓ | ✓ | Batch job |
| AlphaFold 2 | `alphafold2`[^4] | ✓ | - | Batch job |
| AlphaFold 2 | `alphafold`[^4] | ✓ | - | Batch job |
| dl_binder_design | dl_binder_design_module.sh [^5],[^6] | ✓ | - | Batch job |
| ROCKET | rocket_module.sh [^7] | ✓ | - | Batch job |
| GRASP-JAX | GRASP_module.sh [^8] | ✓ | - | Batch job |
@@ -59,7 +63,8 @@ To load a module, run: `module load [module_name]`
| ChimeraX | `chimerax` | ✓ | - | Interactive |
| IMOD | `IMOD` | ✓ | - | Interactive, Batch job |
| MemBrain | `membrain` | ✓ | - | Interactive, Batch job |
| PyMOL | Spack[^13] | ✓ | - | Interactive |
| PyMOL | `pymol` | ✓ | - | Interactive |
| napari | `napari` | ✓ | - | Interactive |
| napari-boxmanager | `napari-boxmanager` | ✓ | - | Interactive |
| napari-tomoslice | `napari-tomoslice` | ✓ | - | Interactive |
@@ -67,25 +72,34 @@ To load a module, run: `module load [module_name]`
| Software | Module name | CPU/A100 nodes | GH nodes | Run type |
| ------------- | ------------------- | :-------------: | :-------------------: | ---------------------- |
| DNet | dnet_module.sh[^14] | ✓ | - | Interactive, Batch job |
| DNet | dnet_module.sh[^13] | ✓ | - | Interactive, Batch job |
| NAMD | `NAMD` | ✓ | ✓ | Batch job |
| VMD | `VMD` | ✓ | | Interactive |
| VMD | `vmd` | ✓ | - | Interactive |
| GROMACS | `gromacs` | ✓ | ✓ | Batch job |
| Amber26 | `amber` | ✓ | ✓ | Batch job |
| AmberTools26 | `ambertools` | ✓ | - | Batch job |
| PLUMED | `plumed` | ✓ | ✓ | Batch job |
| LAMMPS | `lammps` | ✓ | ✓ | Batch job |
| SevenNet | `sevennet` | ✓ | ✓ | Batch job |
| dssp | `dssp` | ✓ | ✓ | Batch job |
## Bioinformatics
| Software | Module name | CPU/A100 nodes | GH nodes | Run type |
| ------------- | ---------------- | :-------------: | :-------------------: | ---------------------- |
| FastQC | `fastqc` | ✓ | | Batch job |
| SRA Toolkit | `sratoolkit` | ✓ | | Batch job |
| Entrez Direct | `edirect` | ✓ | | Batch job |
| Cell Ranger | `cellranger` | ✓ | | Batch job |
| FastQC | `fastqc` | ✓ | - | Batch job |
| SRA Toolkit | `sratoolkit` | ✓ | - | Batch job |
| Entrez Direct | `edirect` | ✓ | - | Batch job |
| Cell Ranger | `cellranger` | ✓ | - | Batch job |
| kalign | `kalign` | ✓ | ✓ | Batch job |
| Refgenie | `refgenie` | ✓ | ✓ | Interactive |
| Nextflow | `nextflow` | ✓ | ✓ | Interactive, Batch job |
| nf-core | `nf-core` | ✓ | ✓ | Interactive, Batch job |
| hh-suite | `hhsuite` | ✓ | ✓ | Batch job |
| hmmer | `hmmer` | ✓ | ✓ | Batch job |
| SeqKit | `seqkit` | ✓ | - | Batch job |
[^1]: `module use Alps_A100; module load BindCraft`
[^1]: `module use Alps_A100; module load BindCraft` (A100 GPU nodes)
[^2]: `source /data/project/cls/shared/software/cryosamba/dev_2025_11/cryosamba_module.sh load`
[^3]: `module use MX; module load phenix/phenix-1.20-4459`
[^4]: `module use MX; module load alphafold`
@@ -97,5 +111,4 @@ To load a module, run: `module load [module_name]`
[^10]: `source /data/project/cls/shared/software/germinal/dev_2025_10/germinal/germinal_module.sh load`
[^11]: `source /data/project/cls/shared/software/BoltzDesign1/dev_2025_07/BoltzDesign1_module.sh load`
[^12]: `source /data/project/cls/shared/software/prosculpt/1.1.4/prosculpt_module.sh load`
[^13]: `module use Spack unstable; module load gcc/12.3 openmpi/5.0.7-qhmk-A100-gpu py-pymol/3.1.0-5qcf`
[^14]: `source /data/project/cls/shared/software/DNet/dnet_module.sh load`
[^13]: `source /data/project/cls/shared/software/DNet/dnet_module.sh load`
@@ -47,54 +47,15 @@ The following example runs the data pipeline on a full CPU node. The database se
#SBATCH --mem=0
#SBATCH --exclusive
# ---------------------------------------------------------------------------- #
# Databases
# ---------------------------------------------------------------------------- #
# Public databases
# $PUBLIC_DATABASES_DIR is automatically set by the AlphaFold 3 environment.
# PUBLIC_DATABASES_DIR=/data/project/cls/shared/databases/alphafold3/public_databases
# Sharded databases
SHARDED_DATABASES_DIR=/data/project/cls/shared/databases/alphafold3/sharded_public_databases
RFAM_SPLIT_DIR=${SHARDED_DATABASES_DIR}/rfam_14_9_clust_seq_id_90_cov_80_rep_seq.fasta.split
MGY_SPLIT_DIR=${SHARDED_DATABASES_DIR}/mgy_clusters_2022_05.fa.split
BFD_SPLIT_DIR=${SHARDED_DATABASES_DIR}/bfd-first_non_consensus_sequences.fasta.split
UNIPROT_SPLIT_DIR=${SHARDED_DATABASES_DIR}/uniprot_all_2021_04.fa.split
UNIREF90_SPLIT_DIR=${SHARDED_DATABASES_DIR}/uniref90_2022_05.fa.split
NT_RNA_SPLIT_DIR=${SHARDED_DATABASES_DIR}/nt_rna_2023_02_23_clust_seq_id_90_cov_80_rep_seq.fasta.split
RNACENTRAL_SPLIT_DIR=${SHARDED_DATABASES_DIR}/rnacentral_active_seq_id_90_cov_80_linclust.fasta.split
# ---------------------------------------------------------------------------- #
# AlphaFold 3
# ---------------------------------------------------------------------------- #
module purge
module load alphafold3
/usr/bin/time -v srun --cpu-bind=threads alphafold3 \
srun --cpu-bind=threads alphafold3_sharded \
--json_path /path/to/json/input.json \
--model_dir /path/to/model_parameters \
--output_dir /path/to/output \
--run_inference=false \
--run_data_pipeline=true \
--seqres_database_path ${PUBLIC_DATABASES_DIR}/pdb_seqres_2022_09_28.fasta \
--pdb_database_path ${PUBLIC_DATABASES_DIR}/mmcif_files \
--rfam_database_path="${RFAM_SPLIT_DIR}/rfam_14_9_clust_seq_id_90_cov_80_rep_seq.fasta@32" \
--rfam_z_value=138.115553 \
--rna_central_database_path="${RNACENTRAL_SPLIT_DIR}/rnacentral_active_seq_id_90_cov_80_linclust.fasta@128" \
--rna_central_z_value=13271.415730 \
--small_bfd_database_path="${BFD_SPLIT_DIR}/bfd-first_non_consensus_sequences.fasta@128" \
--small_bfd_z_value=65984053 \
--uniref90_database_path="${UNIREF90_SPLIT_DIR}/uniref90_2022_05.fa@256" \
--uniref90_z_value=153742194 \
--uniprot_cluster_annot_database_path="${UNIPROT_SPLIT_DIR}/uniprot_all_2021_04.fa@512" \
--uniprot_cluster_annot_z_value=225619586 \
--ntrna_database_path="${NT_RNA_SPLIT_DIR}/nt_rna_2023_02_23_clust_seq_id_90_cov_80_rep_seq.fasta@512" \
--ntrna_z_value=76752.808514 \
--mgnify_database_path="${MGY_SPLIT_DIR}/mgy_clusters_2022_05.fa@1024" \
--mgnify_z_value=623796864 \
--jackhmmer_n_cpu=1 \
--jackhmmer_max_parallel_shards=$((SLURM_CPUS_PER_TASK / 4)) \
--nhmmer_n_cpu=1 \
@@ -105,20 +66,20 @@ module load alphafold3
The following parameters must be specified for every AlphaFold 3 run:
| Parameter | Description |
|----------------|---|
| `--json_path` | Path to the input JSON file. |
| Parameter | Description |
|----------------|--------------------------------------------------------------------|
| `--json_path` | Path to the input JSON file. |
| `--model_dir` | Path to the directory containing the AlphaFold 3 model parameters. |
| `--output_dir` | Path to the directory where the results will be saved. |
| `--output_dir` | Path to the directory where the results will be saved. |
#### Performance parameters
The following parameters control the number of database shards searched in parallel:
| Parameter | Description |
|---|---|
| Parameter | Description |
|-----------------------------------|----------------------------------------------------------------------|
| `--jackhmmer_max_parallel_shards` | Maximum number of database shards searched in parallel by JackHMMER. |
| `--nhmmer_max_parallel_shards` | Maximum number of database shards searched in parallel by NHMMER. |
| `--nhmmer_max_parallel_shards` | Maximum number of database shards searched in parallel by NHMMER. |
!!! note "If the job takes a long time to start"
@@ -130,6 +91,56 @@ The following parameters control the number of database shards searched in paral
If the job is slow to start due to limited cluster availability, you can reduce the number of parallel shards to request fewer resources by reducing **--cpus-per-task** and **--mem**.
#### Non-sharded databases run
You can also run the data pipeline on non-sharded databases. The benchmark results between the two option are displayed below.
```bash
#!/bin/bash -e
#SBATCH --job-name=alphafold3_data_pipeline
#SBATCH --cluster=merlin7
#SBATCH --partition=hourly
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=256
#SBATCH --time=01:00:00
#SBATCH --output=alphafold3_data_pipeline_%j.log
#SBATCH --mem=0
#SBATCH --exclusive
module purge
module load alphafold3
srun --cpu-bind=threads alphafold3 \
--json_path /path/to/json/input.json \
--model_dir /path/to/model_parameters \
--output_dir /path/to/output \
--run_inference=false \
--run_data_pipeline=true \
--jackhmmer_n_cpu=$((SLURM_CPUS_PER_TASK / 4)) \
--nhmmer_n_cpu=$((SLURM_CPUS_PER_TASK / 4))
```
#### Benchmark results
These results are representative of a large, heterogeneous proteinprotein complex prediction workload using AlphaFold 3. The input consists of two protein chains, with chain A being an exceptionally long, highly repetitive and compositionally complex protein (approximately 3,000 residues) and chain B being a long, predominantly helical protein (approximately 600 residues). Thus, the benchmark represents a large proteinprotein complex prediction with substantial sequence length and structural complexity, rather than a typical small single-protein prediction.
They also compare running with [sharded databases](https://github.com/google-deepmind/alphafold3/blob/main/docs/performance.md#sharded-genetic-databases) and non-sharded.
| Run | Version | CPU (Threads) | Memory (Gb) | Total time (HH:MM:SS) |
| ------------------ | --------------- | ------------- | ----------------- | ---------------------- |
| alphafold3_sharded | 3.0.4 | 256 | 472 | **00:12:10** |
| alphafold3 | 3.0.4 | 256 | 472 | 00:29:05 |
| | | | | |
| alphafold3_sharded | 3.0.4 | 128 | 236 | **00:24:23** |
| alphafold3 | 3.0.4 | 128 | 236 | 00:29:58 |
| | | | | |
| alphafold3_sharded | 3.0.4 | 64 | 118 | **00:32:46** |
| alphafold3 | 3.0.4 | 64 | 118 | 00:37:08 |
| | | | | |
| alphafold3_sharded | 3.0.4 | 32 | 59 | **01:02:00** |
| alphafold3 | 3.0.4 | 32 | 59 | 01:06:52 |
### 2. Inference (GPU only)
The inference stage requires a GPU and should be run separately from the CPU-only data pipeline. The following examples show how to run AlphaFold 3 on an NVIDIA A100 GPU node and a Grace Hopper GPU node.
@@ -205,3 +216,12 @@ To use the JAX persistent compilation cache and avoid unnecessary recompilation
```bash
--jax_compilation_cache_dir /path/to/jax_cache
```
#### Benchmark results
| Run | Version | GPUs | CPU (Threads) | Memory (Gb) | Total time (HH:MM:SS) |
| ------------------ | --------- | ----------- | ----------------- | ------------ | --------------------- |
| GH | 3.0.4 | 1 | 72 | 100 | **00:26:16** |
| A100 | 3.0.3[^1] | 1 | 32 | 100 | 00:41:47 |
[^1]: Running with version 3.0.4 resulted in CUDA OOM errors.
Binary file not shown.

After

Width:  |  Height:  |  Size: 110 KiB

+1
View File
@@ -220,6 +220,7 @@ nav:
- CLS:
# - Archiving:
- Data Transfer:
- cls/data_transfer/emf.md
- cls/data_transfer/scopem.md
- cls/data_transfer/unil_s3_bucket.md
- cls/data_transfer/uni_basel.md