first stab at mkdocs migration
refactor CSCS and Meg content add merlin6 quick start update merlin6 nomachine docs give the userdoc its own color scheme we use the Materials default one refactored slurm general docs merlin6 add merlin6 JB docs add software support m6 docs add all files to nav vibed changes #1 add missing pages further vibing #2 vibe #3 further fixes
This commit is contained in:
50
docs/merlin7/99-support/contact.md
Normal file
50
docs/merlin7/99-support/contact.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: Contact
|
||||
#tags:
|
||||
keywords: contact, support, snow, service now, mailing list, mailing, email, mail, merlin-admins@lists.psi.ch, merlin-users@lists.psi.ch, merlin users
|
||||
last_updated: 15. Jan 2025
|
||||
#summary: ""
|
||||
sidebar: merlin7_sidebar
|
||||
permalink: /merlin7/contact.html
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
Support can be asked through:
|
||||
* [PSI Service Now](https://psi.service-now.com/psisp)
|
||||
* E-Mail: <merlin-admins@lists.psi.ch>
|
||||
|
||||
Basic contact information is also displayed on every shell login to the system using the *Message of the Day* mechanism.
|
||||
|
||||
|
||||
### PSI Service Now
|
||||
|
||||
**[PSI Service Now](https://psi.service-now.com/psisp)**: is the official tool for opening incident requests.
|
||||
* PSI HelpDesk will redirect the incident to the corresponding department, or
|
||||
* you can always assign it directly by checking the box `I know which service is affected` and providing the service name `Local HPC Resources (e.g. Merlin) [CF]` (just type in `Local` and you should get the valid completions).
|
||||
|
||||
### Contact Merlin Administrators
|
||||
|
||||
**E-Mail <merlin-admins@lists.psi.ch>**
|
||||
* This is the official way to contact Merlin Administrators for discussions which do not fit well into the incident category.
|
||||
Do not hesitate to contact us for such cases.
|
||||
|
||||
---
|
||||
|
||||
## Get updated through the Merlin User list!
|
||||
|
||||
Is strongly recommended that users subscribe to the Merlin Users mailing list: **<merlin-users@lists.psi.ch>**
|
||||
|
||||
This mailing list is the official channel used by Merlin administrators to inform users about downtimes,
|
||||
interventions or problems. Users can be subscribed in two ways:
|
||||
|
||||
* *(Preferred way)* Self-registration through **[Sympa](https://psilists.ethz.ch/sympa/info/merlin-users)**
|
||||
* If you need to subscribe many people (e.g. your whole group) by sending a request to the admin list **<merlin-admins@lists.psi.ch>**
|
||||
and providing a list of email addresses.
|
||||
|
||||
---
|
||||
|
||||
## The Merlin Cluster Team
|
||||
|
||||
The PSI Merlin clusters are managed by the **[High Performance Computing and Emerging technologies Group](https://www.psi.ch/de/lsm/hpce-group)**, which
|
||||
is part of the [Science IT Infrastructure, and Services department (AWI)](https://www.psi.ch/en/awi) in PSI's [Center for Scientific Computing, Theory and Data (SCD)](https://www.psi.ch/en/csd).
|
||||
328
docs/merlin7/99-support/migration-from-merlin6.md
Normal file
328
docs/merlin7/99-support/migration-from-merlin6.md
Normal file
@@ -0,0 +1,328 @@
|
||||
---
|
||||
#tags:
|
||||
keywords: merlin6, merlin7, migration, fpsync, rsync
|
||||
#summary: ""
|
||||
sidebar: merlin7_sidebar
|
||||
last_updated: 28 May 2025
|
||||
permalink: /merlin7/migrating.html
|
||||
---
|
||||
|
||||
# Merlin6 to Merlin7 Migration Guide
|
||||
|
||||
Welcome to the official documentation for migrating your data from **Merlin6** to **Merlin7**. Please follow the instructions carefully to ensure a smooth and secure transition.
|
||||
|
||||
## 📅 Migration Schedule
|
||||
|
||||
### Phase 1: Users without Projects — **Deadline: July 11**
|
||||
|
||||
If you **do not belong to any Merlin project**, i.e for
|
||||
|
||||
* Users not in any group project (`/data/projects/general`)
|
||||
* Users not in BIO, MEG, Mu3e
|
||||
* Users not part of PSI-owned private Merlin nodes (ASA, MEG, Mu3e)
|
||||
|
||||
You must complete your migration **before July 11**. You just need to migrate your personal */data/user/$USER* and */home/psi/$USER* directories.
|
||||
|
||||
Users are responsible for initiating and completing the migration process as lined out below.
|
||||
Contact the Merlin support team [merlin-admins@lists.psi.ch](mailto:merlin-admins@lists.psi.ch) if you need help.
|
||||
|
||||
> ⚠️ In this phase, **it's important that you don't belong to any project**.
|
||||
> Once the migration is finished, **access to Merlin6 will be no longer possible.**
|
||||
|
||||
Please refer to the [Phase 1: Step-by-Step Migration Instructions](#phase-1-step-by-step-migration-instructions) section
|
||||
for detailed information about user data migration.
|
||||
|
||||
### Phase 2: Project Members and Owners — **Start Before August 1**
|
||||
|
||||
For users in active projects:
|
||||
|
||||
* Project **owners and members will be contacted by the Merlin admins**.
|
||||
* Migration will be **scheduled individually per project**.
|
||||
* Expect contact **before August 1**.
|
||||
|
||||
> ⚠️ In this phase, **data and home directories of group owners and members will be also requested to be migrated in parallel.**
|
||||
|
||||
Please refer to the [Phase 2: Migration Instructions](#phase-2-migration-instructions) section
|
||||
for further information.
|
||||
|
||||
---
|
||||
|
||||
## Directory Structure Changes
|
||||
|
||||
### Merlin6 vs Merlin7
|
||||
|
||||
| Cluster | Home Directory | User Data Directory | Projects | Experiments |
|
||||
| ------- | :----------------- | :------------------ | -------------- | ----------------- |
|
||||
| merlin6 | /psi/home/`$USER` | /data/user/`$USER` | /data/project/ | /data/experiments |
|
||||
| merlin7 | /data/user/`$USER` | /data/user/`$USER` | /data/project/ | /data/project/ |
|
||||
|
||||
* The **home directory and user data directory have been merged** into the single new home directory`/data/user/$USER`.
|
||||
* The **experiments directory has been integrated into `/data/project/`**:
|
||||
|
||||
* `/data/project/general` contains general Merlin7 projects.
|
||||
* Other subdirectories are used for large-scale projects such as CLS division, Mu3e, and MeG.
|
||||
|
||||
---
|
||||
|
||||
## 📋 Prerequisites and Preparation
|
||||
|
||||
Before starting the migration, make sure you:
|
||||
|
||||
* are **registered on Merlin7**.
|
||||
|
||||
* If not yet registered, please do so following [these instructions](../01-Quick-Start-Guide/requesting-accounts.md)
|
||||
|
||||
* **have cleaned up your data to reduce migration time and space usage**.
|
||||
* **For the user data migration**, ensure your total usage on Merlin6 (`/psi/home`+`/data/user`) is **well below the 1 TB quota** (use the `merlin_quotas` command). Remember:
|
||||
|
||||
* **Merlin7 also has a 1 TB quota on your home directory**, and you might already have data there.
|
||||
* If your usage exceeds this during the transfer, the process might fail.
|
||||
* No activity should be running / performed on Merlin6 when the transfer process is ongoing.
|
||||
|
||||
### Recommended Cleanup Actions
|
||||
|
||||
* Remove unused files and datasets.
|
||||
* Archive large, inactive data sets.
|
||||
* Delete or clean up unused `conda` or `virtualenv` Python environments:
|
||||
|
||||
* These are often large and may not work as-is on Merlin7.
|
||||
* You can export your conda environment description to a file with:
|
||||
|
||||
```bash
|
||||
conda env export -n myenv > $HOME/myenv.yml
|
||||
```
|
||||
* Then recreate them later on Merlin7 from these files.
|
||||
|
||||
> 🧹 For the **user data**, you can always remove more old data **after** migration — it will be copied into `~/merlin6data` and `~/merlin6home` on Merlin7.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Step-by-Step Migration Instructions
|
||||
|
||||
### Step 1: Run `merlin7_migration.setup`
|
||||
|
||||
Log into any **Merlin6 login node** (`merlin-l-001.psi.ch`, `merlin-l-002.psi.ch`, `merlin-l-01.psi.ch`) and run:
|
||||
|
||||
```bash
|
||||
merlin7_migration.setup
|
||||
```
|
||||
|
||||
This script will:
|
||||
|
||||
* Check that you have an account on Merlin7.
|
||||
* Configure and check that your environment is ready for transferring files via Slurm job.
|
||||
* **Create two directories:**
|
||||
|
||||
* `~/merlin6data` → copy of your old /data/user
|
||||
* `~/merlin6home` → copy of your old home
|
||||
|
||||
> ⚠️ **Important:** If `~/merlin6home` or `~/merlin6data` already exist on Merlin7, the script will exit.
|
||||
> **Please remove them or contact support**.
|
||||
|
||||
If there are issues, the script will:
|
||||
|
||||
* Print clear diagnostic output
|
||||
* Give you some hints to resolve the issue
|
||||
|
||||
If you are stuck, email: [merlin-admins@lists.psi.ch](mailto:merlin-admins@lists.psi.ch)
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Run `merlin7_migration.start`
|
||||
|
||||
After setup completes, start the migration by running:
|
||||
|
||||
```bash
|
||||
merlin7_migration.start
|
||||
```
|
||||
|
||||
This script will:
|
||||
|
||||
* Check the status of your quota on Merlin6.
|
||||
* Submit **SLURM batch jobs** to the **`xfer`** partition
|
||||
* Queue two jobs:
|
||||
|
||||
* `migrate_merlin6data.batch` (data dir)
|
||||
* `migrate_merlin6home.batch` (home dir)
|
||||
* This job will only start if `migrate_merlin6data.batch` has successfully
|
||||
finished.
|
||||
* Automatically track the job IDs
|
||||
* Print log file locations for the different jobs
|
||||
|
||||
> ⚠️ **Once both transfers succeed, your access to Merlin6 will be revoked.**
|
||||
> Do **not** attempt to reconnect to Merlin6 after this.
|
||||
|
||||
#### ❗ If Something Goes Wrong
|
||||
|
||||
If a problem occurs during the migration process:
|
||||
|
||||
* 🔍 **Check the job log files** mentioned in the script output. They contain detailed messages that explain what failed and why.
|
||||
* 🛠️ **Fix the root cause** on the source system. Common issues include:
|
||||
|
||||
* Files with incorrect permissions
|
||||
* Ownership mismatches
|
||||
* Disk quota exceeded on Merlin7
|
||||
* 📚 Refer to the [⚠️ Common rsync/fpsync Migration Issues](#common-rsyncfpsync-migration-issues) section below for detailed explanations and solutions.
|
||||
|
||||
> ℹ️ **Important:** If `migrate_merlin6data.batch` fails, the migration process will automatically cancel `migrate_merlin6home.batch` to avoid ending in an inconsistent state.
|
||||
|
||||
Once the problem is resolved, simply re-run the `merlin7_migration.start` script to resume the migration.
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Monitor Transfer Jobs
|
||||
|
||||
To monitor your transfer jobs, run:
|
||||
|
||||
```bash
|
||||
squeue -M merlin6 -u $USER -p xfer
|
||||
```
|
||||
|
||||
Check the output to ensure your jobs are:
|
||||
|
||||
* Running (`R`) or completed (`CG` or removed from queue)
|
||||
* Not failed (`F`, `TO`, or stuck)
|
||||
|
||||
You can also check logs (as printed by the script) to verify job completion.
|
||||
|
||||
> ✅ When `/data/user/$USER` and `/psi/home/$USER` on Merlin6 are no longer accessible, migration is complete.
|
||||
|
||||
---
|
||||
|
||||
### Examples
|
||||
|
||||
#### Setup the Migration
|
||||
|
||||
```bash
|
||||
merlin7_migration.setup
|
||||
```
|
||||
|
||||
*Expected output:*
|
||||
|
||||
```bash
|
||||
✅ login002.merlin7.psi.ch
|
||||
✅ `$USER` is a member of svc-cluster_merlin7
|
||||
✅ Skipping key generation
|
||||
✅ SSH key already added to agent.
|
||||
✅ SSH ID successfully copied to login00[1|2].merlin7.psi.ch.
|
||||
✅ Test successful.
|
||||
✅ /data/software/xfer_logs/caubet_m created.
|
||||
✅ ~/merlin6data directory created.
|
||||
✅ ~/merlin6home directory created.
|
||||
```
|
||||
|
||||
#### Start the Migration
|
||||
|
||||
```bash
|
||||
merlin7_migration.start
|
||||
```
|
||||
|
||||
*Expected output:*
|
||||
|
||||
```bash
|
||||
(base) ❄ [caubet_m@merlin-l-001:/data/software/admin/scripts/merlin-user-tools/alps(master)]# ./merlin7_migration.start
|
||||
✅ Quota check passed.
|
||||
Used: 512 GB, 234001 files
|
||||
|
||||
###################################################
|
||||
Submitting transfer jobs to Slurm
|
||||
|
||||
Job logs can be found here:
|
||||
➡️ Directory '/data/user/caubet_m' does NOT have 000 permissions. Transfer pending, continuing...
|
||||
✅ Submitted DATA_MIGRATION job: 24688554. Sleeping 3 seconds...
|
||||
- /data/user transfer logs:
|
||||
- /data/software/xfer_logs/caubet_m/data-24688554.out
|
||||
- /data/software/xfer_logs/caubet_m/data-24688554.err
|
||||
➡️ Directory '/psi/home/caubet_m' does NOT have 000 permissions. Transfer pending, continuing...
|
||||
✅ Submitted HOME_MIGRATION job with dependency on 24688554: 24688555. Sleeping 3 seconds...
|
||||
- /psi/home transfer logs:
|
||||
- /data/software/xfer_logs/caubet_m/home-24688555.out
|
||||
- /data/software/xfer_logs/caubet_m/home-24688555.err
|
||||
|
||||
✅ You can start manually a monitoring window with:
|
||||
tmux new-session -d -s "xfersession" "watch 'squeue -M merlin6 -u caubet_m -p xfer'"
|
||||
tmux attach -t "xfersession"
|
||||
|
||||
✅ FINISHED - PLEASE CHECK JOB TRANSFER PROGRESS
|
||||
```
|
||||
|
||||
#### Monitor Progress
|
||||
|
||||
```bash
|
||||
squeue -M merlin6 -u $USER -p xfer
|
||||
```
|
||||
|
||||
*Output:*
|
||||
|
||||
```bash
|
||||
$ squeue -M merlin6 -u $USER -p xfer
|
||||
CLUSTER: merlin6
|
||||
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
|
||||
24688581 xfer HOME_MIG caubet_m PD 0:00 1 (Dependency)
|
||||
24688580 xfer DATA_MIG caubet_m R 0:22 1 merlin-c-017
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Migration Instructions
|
||||
|
||||
Please refer to the [Prerequisites and Preparation](#prerequisites-and-preparation) section for initial setup steps.
|
||||
Further instructions will be sent via email once the owning team is contacted by the Merlin administrators.
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Common `rsync`/`fpsync` Migration Issues
|
||||
|
||||
### File Permission Denied
|
||||
|
||||
* **Cause**: Files or directories are not readable by the user running the transfer.
|
||||
* **Solution**: Fix source-side permissions:
|
||||
|
||||
```bash
|
||||
chmod -R u+rX /path/to/file_or_dir
|
||||
```
|
||||
|
||||
### Ownership Mismatches
|
||||
|
||||
* **Cause**: Source files are owned by another user (e.g. root or a collaborator).
|
||||
* **Solution**:
|
||||
|
||||
* Change ownership before migration:
|
||||
|
||||
```bash
|
||||
chown -R $USER /path/to/file
|
||||
```
|
||||
|
||||
### Special Files (e.g. device files, sockets)
|
||||
|
||||
* **Cause**: `rsync` tries to copy UNIX sockets, device files, or FIFOs.
|
||||
* **Effect**: Errors or incomplete copies.
|
||||
* **Solution**: Avoid transferring such files entirely (by deleting them).
|
||||
|
||||
### Exceeded Disk Quota
|
||||
|
||||
* **Cause**: Combined size of existing + incoming data exceeds 1 TB quota on Merlin7.
|
||||
* **Effect**: Transfer stops abruptly.
|
||||
* **Solution**: Clean up or archive non-essential data before migration.
|
||||
|
||||
### Very Small Files or Large Trees → Many Small rsync Calls
|
||||
|
||||
* **Cause**: Directory with thousands/millions of small files.
|
||||
* **Effect**: Transfer is slow or hits process limits.
|
||||
* **Solution**: Consider archiving to `.tar.gz` before transferring:
|
||||
|
||||
```bash
|
||||
tar -czf myenv.tar.gz myenv/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Need Help?
|
||||
|
||||
If something doesn't work:
|
||||
|
||||
* Re-run the scripts and check the logs carefully.
|
||||
* Use `less`, `cat`, or `tail -f` to view your job logs.
|
||||
* Contact the Merlin support team: 📧 [merlin-admins@lists.psi.ch](mailto:merlin-admins@lists.psi.ch)
|
||||
|
||||
> We are here to help you migrate safely and efficiently.
|
||||
Reference in New Issue
Block a user