initial formatting changes complete

This commit is contained in:
2026-01-06 16:40:15 +01:00
parent f58c1f57b8
commit 7db5d0fd05
81 changed files with 805 additions and 1112 deletions

View File

@@ -1,12 +1,3 @@
---
#tags:
keywords: merlin6, merlin7, migration, fpsync, rsync
#summary: ""
sidebar: merlin7_sidebar
last_updated: 28 May 2025
permalink: /merlin7/migrating.html
---
# Merlin6 to Merlin7 Migration Guide
Welcome to the official documentation for migrating your data from **Merlin6** to **Merlin7**. Please follow the instructions carefully to ensure a smooth and secure transition.
@@ -15,7 +6,7 @@ Welcome to the official documentation for migrating your data from **Merlin6** t
### Phase 1: Users without Projects — **Deadline: July 11**
If you **do not belong to any Merlin project**, i.e for
If you **do not belong to any Merlin project**, i.e for
* Users not in any group project (`/data/projects/general`)
* Users not in BIO, MEG, Mu3e
@@ -59,8 +50,8 @@ for further information.
* The **home directory and user data directory have been merged** into the single new home directory`/data/user/$USER`.
* The **experiments directory has been integrated into `/data/project/`**:
* `/data/project/general` contains general Merlin7 projects.
* Other subdirectories are used for large-scale projects such as CLS division, Mu3e, and MeG.
* `/data/project/general` contains general Merlin7 projects.
* Other subdirectories are used for large-scale projects such as CLS division, Mu3e, and MeG.
---
@@ -70,13 +61,15 @@ Before starting the migration, make sure you:
* are **registered on Merlin7**.
* If not yet registered, please do so following [these instructions](../01-Quick-Start-Guide/requesting-accounts.md)
* If not yet registered, please do so following [these instructions](../01-Quick-Start-Guide/requesting-accounts.md)
* **have cleaned up your data to reduce migration time and space usage**.
* **For the user data migration**, ensure your total usage on Merlin6 (`/psi/home`+`/data/user`) is **well below the 1TB quota** (use the `merlin_quotas` command). Remember:
* **Merlin7 also has a 1TB quota on your home directory**, and you might already have data there.
* If your usage exceeds this during the transfer, the process might fail.
* **Merlin7 also has a 1TB quota on your home directory**, and you might already have data there.
* If your usage exceeds this during the transfer, the process might fail.
* No activity should be running / performed on Merlin6 when the transfer process is ongoing.
### Recommended Cleanup Actions
@@ -85,13 +78,13 @@ Before starting the migration, make sure you:
* Archive large, inactive data sets.
* Delete or clean up unused `conda` or `virtualenv` Python environments:
* These are often large and may not work as-is on Merlin7.
* You can export your conda environment description to a file with:
* These are often large and may not work as-is on Merlin7.
* You can export your conda environment description to a file with:
```bash
conda env export -n myenv > $HOME/myenv.yml
```
* Then recreate them later on Merlin7 from these files.
* Then recreate them later on Merlin7 from these files.
> 🧹 For the **user data**, you can always remove more old data **after** migration — it will be copied into `~/merlin6data` and `~/merlin6home` on Merlin7.
@@ -113,10 +106,11 @@ This script will:
* Configure and check that your environment is ready for transferring files via Slurm job.
* **Create two directories:**
* `~/merlin6data` → copy of your old /data/user
* `~/merlin6home` → copy of your old home
* `~/merlin6data` → copy of your old /data/user
* `~/merlin6home` → copy of your old home
> ⚠️ **Important:** If `~/merlin6home` or `~/merlin6data` already exist on Merlin7, the script will exit.
> ⚠️ **Important:** If `~/merlin6home` or `~/merlin6data` already exist on Merlin7, the script will exit.
> **Please remove them or contact support**.
If there are issues, the script will:
@@ -159,9 +153,9 @@ If a problem occurs during the migration process:
* 🔍 **Check the job log files** mentioned in the script output. They contain detailed messages that explain what failed and why.
* 🛠️ **Fix the root cause** on the source system. Common issues include:
* Files with incorrect permissions
* Ownership mismatches
* Disk quota exceeded on Merlin7
* Files with incorrect permissions
* Ownership mismatches
* Disk quota exceeded on Merlin7
* 📚 Refer to the [⚠️ Common rsync/fpsync Migration Issues](#common-rsyncfpsync-migration-issues) section below for detailed explanations and solutions.
> **Important:** If `migrate_merlin6data.batch` fails, the migration process will automatically cancel `migrate_merlin6home.batch` to avoid ending in an inconsistent state.
@@ -200,10 +194,10 @@ merlin7_migration.setup
*Expected output:*
```bash
✅ login002.merlin7.psi.ch
✅ login002.merlin7.psi.ch
✅ `$USER` is a member of svc-cluster_merlin7
✅ Skipping key generation
✅ SSH key already added to agent.
✅ Skipping key generation
✅ SSH key already added to agent.
✅ SSH ID successfully copied to login00[1|2].merlin7.psi.ch.
✅ Test successful.
✅ /data/software/xfer_logs/caubet_m created.
@@ -287,7 +281,7 @@ Further instructions will be sent via email once the owning team is contacted by
* **Cause**: Source files are owned by another user (e.g. root or a collaborator).
* **Solution**:
* Change ownership before migration:
* Change ownership before migration:
```bash
chown -R $USER /path/to/file