Add meg
This commit is contained in:
@@ -19,6 +19,9 @@ entries:
|
||||
- title: Merlin6 HPC Cluster
|
||||
url: /merlin6/introduction.html
|
||||
output: web
|
||||
- title: MEG HPC Cluster
|
||||
url: /meg/introduction.html
|
||||
output: web
|
||||
- title: PSI HPC@CSCS
|
||||
url: /CSCS/index.html
|
||||
output: web
|
||||
|
||||
18
pages/meg/01-Quick-Start-Guide/introduction.md
Normal file
18
pages/meg/01-Quick-Start-Guide/introduction.md
Normal file
@@ -0,0 +1,18 @@
|
||||
---
|
||||
title: Introduction
|
||||
#tags:
|
||||
keywords: introduction, home, welcome, architecture, design
|
||||
last_updated: 07 September 2022
|
||||
#summary: "MeG cluster overview"
|
||||
sidebar: meg_sidebar
|
||||
permalink: /meg/introduction.html
|
||||
redirect_from:
|
||||
- /meg
|
||||
- /meg/index.html
|
||||
---
|
||||
|
||||
## The Meg local HPC cluster
|
||||
|
||||
"The MEG II collaboration includes almost 70 physicists from research institutions from five countries. Researchers and technicians from PSI have played a leading role, particularly with providing the high-quality beam, technical support in the detector integration, and in the design, construction, and operation of the detector readout electronics." [Source](https://www.psi.ch/en/cnm/news/in-search-of-new-physics-new-result-from-the-meg-ii-collaboration)
|
||||
|
||||
The MEG data analysis cluster is a cluster tightly coupled to Merlin and dedicated to the analysis of data from the MEG experiment. Operated for the Muon Physics group.
|
||||
50
pages/meg/99-support/contact.md
Normal file
50
pages/meg/99-support/contact.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: Contact
|
||||
#tags:
|
||||
keywords: contact, support, snow, service now, mailing list, mailing, email, mail, meg-admins@lists.psi.ch, merlin users
|
||||
last_updated: 15. Jan 2025
|
||||
#summary: ""
|
||||
sidebar: meg_sidebar
|
||||
permalink: /meg/contact.html
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
Support can be asked through:
|
||||
* [PSI Service Now](https://psi.service-now.com/psisp)
|
||||
* E-Mail: <meg-admins@lists.psi.ch>
|
||||
|
||||
Basic contact information is also displayed on every shell login to the system using the *Message of the Day* mechanism.
|
||||
|
||||
|
||||
### PSI Service Now
|
||||
|
||||
**[PSI Service Now](https://psi.service-now.com/psisp)**: is the official PSI tool for opening incident requests. However, contact via email (see below) is preferred.
|
||||
* PSI HelpDesk will redirect the incident to the corresponding department, or
|
||||
* you can always assign it directly by checking the box `I know which service is affected` and providing the service name `Local HPC Resources (e.g. MEG) [CF]` (just type in `Local` and you should get the valid completions).
|
||||
|
||||
### Contact Meg Administrators
|
||||
|
||||
**E-Mail <meg-admins@lists.psi.ch>** or **<merlin-admins@lists.psi.ch>**
|
||||
* This is the preferred way to contact MEG Administrators.
|
||||
Do not hesitate to contact us for such cases.
|
||||
|
||||
---
|
||||
|
||||
## Get updated through the Merlin User list!
|
||||
|
||||
Is strongly recommended that users subscribe to the Merlin Users mailing list: **<merlin-users@lists.psi.ch>**
|
||||
|
||||
This mailing list is the official channel used by Merlin administrators to inform users about downtimes,
|
||||
interventions or problems. Users can be subscribed in two ways:
|
||||
|
||||
* *(Preferred way)* Self-registration through **[Sympa](https://psilists.ethz.ch/sympa/info/merlin-users)**
|
||||
* If you need to subscribe many people (e.g. your whole group) by sending a request to the admin list **<merlin-admins@lists.psi.ch>**
|
||||
and providing a list of email addresses.
|
||||
|
||||
---
|
||||
|
||||
## The MEG Cluster Team
|
||||
|
||||
The PSI Merlin and MEG clusters are managed by the **[High Performance Computing and Emerging technologies Group](https://www.psi.ch/de/lsm/hpce-group)**, which
|
||||
is part of the [Science IT Infrastructure, and Services department (AWI)](https://www.psi.ch/en/awi) in PSI's [Center for Scientific Computing, Theory and Data (SCD)](https://www.psi.ch/en/csd).
|
||||
328
pages/meg/99-support/migration-from-merlin6.md
Normal file
328
pages/meg/99-support/migration-from-merlin6.md
Normal file
@@ -0,0 +1,328 @@
|
||||
---
|
||||
#tags:
|
||||
keywords: meg, merlin6, merlin7, migration, fpsync, rsync
|
||||
#summary: ""
|
||||
sidebar: meg_sidebar
|
||||
last_updated: 28 May 2025
|
||||
permalink: /meg/migrating.html
|
||||
---
|
||||
|
||||
# Merlin6 to Merlin7 Migration Guide
|
||||
|
||||
Welcome to the official documentation for migrating your data from **Merlin6** to **Merlin7**. Please follow the instructions carefully to ensure a smooth and secure transition.
|
||||
|
||||
## 📅 Migration Schedule
|
||||
|
||||
### Phase 1: Users without Projects — **Deadline: July 11**
|
||||
|
||||
If you **do not belong to any Merlin project**, i.e for
|
||||
|
||||
* Users not in any group project (`/data/projects/general`)
|
||||
* Users not in BIO, MEG, Mu3e
|
||||
* Users not part of PSI-owned private Merlin nodes (ASA, MEG, Mu3e)
|
||||
|
||||
You must complete your migration **before July 11**. You just need to migrate your personal */data/user/$USER* and */home/psi/$USER* directories.
|
||||
|
||||
Users are responsible for initiating and completing the migration process as lined out below.
|
||||
Contact the Merlin support team [merlin-admins@lists.psi.ch](mailto:merlin-admins@lists.psi.ch) if you need help.
|
||||
|
||||
> ⚠️ In this phase, **it's important that you don't belong to any project**.
|
||||
> Once the migration is finished, **access to Merlin6 will be no longer possible.**
|
||||
|
||||
Please refer to the [Phase 1: Step-by-Step Migration Instructions](/merlin7/migrating.html#phase-1-step-by-step-migration-instructions) section
|
||||
for detailed information about user data migration.
|
||||
|
||||
### Phase 2: Project Members and Owners — **Start Before August 1**
|
||||
|
||||
For users in active projects:
|
||||
|
||||
* Project **owners and members will be contacted by the Merlin admins**.
|
||||
* Migration will be **scheduled individually per project**.
|
||||
* Expect contact **before August 1**.
|
||||
|
||||
> ⚠️ In this phase, **data and home directories of group owners and members will be also requested to be migrated in parallel.**
|
||||
|
||||
Please refer to the [Phase 2: Migration Instructions](/merlin7/migrating.html#phase-2-migration-instructions) section
|
||||
for further information.
|
||||
|
||||
---
|
||||
|
||||
## Directory Structure Changes
|
||||
|
||||
### Merlin6 vs Merlin7
|
||||
|
||||
| Cluster | Home Directory | User Data Directory | Projects | Experiments |
|
||||
| ------- | :----------------- | :------------------ | -------------- | ----------------- |
|
||||
| merlin6 | /psi/home/`$USER` | /data/user/`$USER` | /data/project/ | /data/experiments |
|
||||
| merlin7 | /data/user/`$USER` | /data/user/`$USER` | /data/project/ | /data/project/ |
|
||||
|
||||
* The **home directory and user data directory have been merged** into the single new home directory`/data/user/$USER`.
|
||||
* The **experiments directory has been integrated into `/data/project/`**:
|
||||
|
||||
* `/data/project/general` contains general Merlin7 projects.
|
||||
* Other subdirectories are used for large-scale projects such as CLS division, Mu3e, and MeG.
|
||||
|
||||
---
|
||||
|
||||
## 📋 Prerequisites and Preparation
|
||||
|
||||
Before starting the migration, make sure you:
|
||||
|
||||
* are **registered on Merlin7**.
|
||||
|
||||
* If not yet registered, please do so following [these instructions](../merlin7/request-account.html)
|
||||
|
||||
* **have cleaned up your data to reduce migration time and space usage**.
|
||||
* **For the user data migration**, ensure your total usage on Merlin6 (`/psi/home`+`/data/user`) is **well below the 1 TB quota** (use the `merlin_quotas` command). Remember:
|
||||
|
||||
* **Merlin7 also has a 1 TB quota on your home directory**, and you might already have data there.
|
||||
* If your usage exceeds this during the transfer, the process might fail.
|
||||
* No activity should be running / performed on Merlin6 when the transfer process is ongoing.
|
||||
|
||||
### Recommended Cleanup Actions
|
||||
|
||||
* Remove unused files and datasets.
|
||||
* Archive large, inactive data sets.
|
||||
* Delete or clean up unused `conda` or `virtualenv` Python environments:
|
||||
|
||||
* These are often large and may not work as-is on Merlin7.
|
||||
* You can export your conda environment description to a file with:
|
||||
|
||||
```bash
|
||||
conda env export -n myenv > $HOME/myenv.yml
|
||||
```
|
||||
* Then recreate them later on Merlin7 from these files.
|
||||
|
||||
> 🧹 For the **user data**, you can always remove more old data **after** migration — it will be copied into `~/merlin6data` and `~/merlin6home` on Merlin7.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Step-by-Step Migration Instructions
|
||||
|
||||
### Step 1: Run `merlin7_migration.setup`
|
||||
|
||||
Log into any **Merlin6 login node** (`merlin-l-001.psi.ch`, `merlin-l-002.psi.ch`, `merlin-l-01.psi.ch`) and run:
|
||||
|
||||
```bash
|
||||
merlin7_migration.setup
|
||||
```
|
||||
|
||||
This script will:
|
||||
|
||||
* Check that you have an account on Merlin7.
|
||||
* Configure and check that your environment is ready for transferring files via Slurm job.
|
||||
* **Create two directories:**
|
||||
|
||||
* `~/merlin6data` → copy of your old /data/user
|
||||
* `~/merlin6home` → copy of your old home
|
||||
|
||||
> ⚠️ **Important:** If `~/merlin6home` or `~/merlin6data` already exist on Merlin7, the script will exit.
|
||||
> **Please remove them or contact support**.
|
||||
|
||||
If there are issues, the script will:
|
||||
|
||||
* Print clear diagnostic output
|
||||
* Give you some hints to resolve the issue
|
||||
|
||||
If you are stuck, email: [merlin-admins@lists.psi.ch](mailto:merlin-admins@lists.psi.ch)
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Run `merlin7_migration.start`
|
||||
|
||||
After setup completes, start the migration by running:
|
||||
|
||||
```bash
|
||||
merlin7_migration.start
|
||||
```
|
||||
|
||||
This script will:
|
||||
|
||||
* Check the status of your quota on Merlin6.
|
||||
* Submit **SLURM batch jobs** to the **`xfer`** partition
|
||||
* Queue two jobs:
|
||||
|
||||
* `migrate_merlin6data.batch` (data dir)
|
||||
* `migrate_merlin6home.batch` (home dir)
|
||||
* This job will only start if `migrate_merlin6data.batch` has successfully
|
||||
finished.
|
||||
* Automatically track the job IDs
|
||||
* Print log file locations for the different jobs
|
||||
|
||||
> ⚠️ **Once both transfers succeed, your access to Merlin6 will be revoked.**
|
||||
> Do **not** attempt to reconnect to Merlin6 after this.
|
||||
|
||||
#### ❗ If Something Goes Wrong
|
||||
|
||||
If a problem occurs during the migration process:
|
||||
|
||||
* 🔍 **Check the job log files** mentioned in the script output. They contain detailed messages that explain what failed and why.
|
||||
* 🛠️ **Fix the root cause** on the source system. Common issues include:
|
||||
|
||||
* Files with incorrect permissions
|
||||
* Ownership mismatches
|
||||
* Disk quota exceeded on Merlin7
|
||||
* 📚 Refer to the [⚠️ Common rsync/fpsync Migration Issues](/merlin7/migrating.html#%EF%B8%8F--common-rsyncfpsync-migration-issues) section below for detailed explanations and solutions.
|
||||
|
||||
> ℹ️ **Important:** If `migrate_merlin6data.batch` fails, the migration process will automatically cancel `migrate_merlin6home.batch` to avoid ending in an inconsistent state.
|
||||
|
||||
Once the problem is resolved, simply re-run the `merlin7_migration.start` script to resume the migration.
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Monitor Transfer Jobs
|
||||
|
||||
To monitor your transfer jobs, run:
|
||||
|
||||
```bash
|
||||
squeue -M merlin6 -u $USER -p xfer
|
||||
```
|
||||
|
||||
Check the output to ensure your jobs are:
|
||||
|
||||
* Running (`R`) or completed (`CG` or removed from queue)
|
||||
* Not failed (`F`, `TO`, or stuck)
|
||||
|
||||
You can also check logs (as printed by the script) to verify job completion.
|
||||
|
||||
> ✅ When `/data/user/$USER` and `/psi/home/$USER` on Merlin6 are no longer accessible, migration is complete.
|
||||
|
||||
---
|
||||
|
||||
### Examples
|
||||
|
||||
#### Setup the Migration
|
||||
|
||||
```bash
|
||||
merlin7_migration.setup
|
||||
```
|
||||
|
||||
*Expected output:*
|
||||
|
||||
```bash
|
||||
✅ login002.merlin7.psi.ch
|
||||
✅ `$USER` is a member of svc-cluster_merlin7
|
||||
✅ Skipping key generation
|
||||
✅ SSH key already added to agent.
|
||||
✅ SSH ID successfully copied to login00[1|2].merlin7.psi.ch.
|
||||
✅ Test successful.
|
||||
✅ /data/software/xfer_logs/caubet_m created.
|
||||
✅ ~/merlin6data directory created.
|
||||
✅ ~/merlin6home directory created.
|
||||
```
|
||||
|
||||
#### Start the Migration
|
||||
|
||||
```bash
|
||||
merlin7_migration.start
|
||||
```
|
||||
|
||||
*Expected output:*
|
||||
|
||||
```bash
|
||||
(base) ❄ [caubet_m@merlin-l-001:/data/software/admin/scripts/merlin-user-tools/alps(master)]# ./merlin7_migration.start
|
||||
✅ Quota check passed.
|
||||
Used: 512 GB, 234001 files
|
||||
|
||||
###################################################
|
||||
Submitting transfer jobs to Slurm
|
||||
|
||||
Job logs can be found here:
|
||||
➡️ Directory '/data/user/caubet_m' does NOT have 000 permissions. Transfer pending, continuing...
|
||||
✅ Submitted DATA_MIGRATION job: 24688554. Sleeping 3 seconds...
|
||||
- /data/user transfer logs:
|
||||
- /data/software/xfer_logs/caubet_m/data-24688554.out
|
||||
- /data/software/xfer_logs/caubet_m/data-24688554.err
|
||||
➡️ Directory '/psi/home/caubet_m' does NOT have 000 permissions. Transfer pending, continuing...
|
||||
✅ Submitted HOME_MIGRATION job with dependency on 24688554: 24688555. Sleeping 3 seconds...
|
||||
- /psi/home transfer logs:
|
||||
- /data/software/xfer_logs/caubet_m/home-24688555.out
|
||||
- /data/software/xfer_logs/caubet_m/home-24688555.err
|
||||
|
||||
✅ You can start manually a monitoring window with:
|
||||
tmux new-session -d -s "xfersession" "watch 'squeue -M merlin6 -u caubet_m -p xfer'"
|
||||
tmux attach -t "xfersession"
|
||||
|
||||
✅ FINISHED - PLEASE CHECK JOB TRANSFER PROGRESS
|
||||
```
|
||||
|
||||
#### Monitor Progress
|
||||
|
||||
```bash
|
||||
squeue -M merlin6 -u $USER -p xfer
|
||||
```
|
||||
|
||||
*Output:*
|
||||
|
||||
```bash
|
||||
$ squeue -M merlin6 -u $USER -p xfer
|
||||
CLUSTER: merlin6
|
||||
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
|
||||
24688581 xfer HOME_MIG caubet_m PD 0:00 1 (Dependency)
|
||||
24688580 xfer DATA_MIG caubet_m R 0:22 1 merlin-c-017
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Migration Instructions
|
||||
|
||||
Please refer to the [Prerequisites and Preparation](/merlin7/migrating.html#-prerequisites-and-preparation) section for initial setup steps.
|
||||
Further instructions will be sent via email once the owning team is contacted by the Merlin administrators.
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Common `rsync`/`fpsync` Migration Issues
|
||||
|
||||
### File Permission Denied
|
||||
|
||||
* **Cause**: Files or directories are not readable by the user running the transfer.
|
||||
* **Solution**: Fix source-side permissions:
|
||||
|
||||
```bash
|
||||
chmod -R u+rX /path/to/file_or_dir
|
||||
```
|
||||
|
||||
### Ownership Mismatches
|
||||
|
||||
* **Cause**: Source files are owned by another user (e.g. root or a collaborator).
|
||||
* **Solution**:
|
||||
|
||||
* Change ownership before migration:
|
||||
|
||||
```bash
|
||||
chown -R $USER /path/to/file
|
||||
```
|
||||
|
||||
### Special Files (e.g. device files, sockets)
|
||||
|
||||
* **Cause**: `rsync` tries to copy UNIX sockets, device files, or FIFOs.
|
||||
* **Effect**: Errors or incomplete copies.
|
||||
* **Solution**: Avoid transferring such files entirely (by deleting them).
|
||||
|
||||
### Exceeded Disk Quota
|
||||
|
||||
* **Cause**: Combined size of existing + incoming data exceeds 1 TB quota on Merlin7.
|
||||
* **Effect**: Transfer stops abruptly.
|
||||
* **Solution**: Clean up or archive non-essential data before migration.
|
||||
|
||||
### Very Small Files or Large Trees → Many Small rsync Calls
|
||||
|
||||
* **Cause**: Directory with thousands/millions of small files.
|
||||
* **Effect**: Transfer is slow or hits process limits.
|
||||
* **Solution**: Consider archiving to `.tar.gz` before transferring:
|
||||
|
||||
```bash
|
||||
tar -czf myenv.tar.gz myenv/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Need Help?
|
||||
|
||||
If something doesn't work:
|
||||
|
||||
* Re-run the scripts and check the logs carefully.
|
||||
* Use `less`, `cat`, or `tail -f` to view your job logs.
|
||||
* Contact the Merlin support team: 📧 [merlin-admins@lists.psi.ch](mailto:merlin-admins@lists.psi.ch)
|
||||
|
||||
> We are here to help you migrate safely and efficiently.
|
||||
@@ -32,7 +32,7 @@ Support can be asked through:
|
||||
|
||||
## Get updated through the Merlin User list!
|
||||
|
||||
Is strictly recommended that users subscribe to the Merlin Users mailing list: **<merlin-users@lists.psi.ch>**
|
||||
Is strongly recommended that users subscribe to the Merlin Users mailing list: **<merlin-users@lists.psi.ch>**
|
||||
|
||||
This mailing list is the official channel used by Merlin6 administrators to inform users about downtimes,
|
||||
interventions or problems. Users can be subscribed in two ways:
|
||||
|
||||
@@ -33,7 +33,7 @@ Basic contact information is also displayed on every shell login to the system u
|
||||
|
||||
## Get updated through the Merlin User list!
|
||||
|
||||
Is strictly recommended that users subscribe to the Merlin Users mailing list: **<merlin-users@lists.psi.ch>**
|
||||
Is strongly recommended that users subscribe to the Merlin Users mailing list: **<merlin-users@lists.psi.ch>**
|
||||
|
||||
This mailing list is the official channel used by Merlin administrators to inform users about downtimes,
|
||||
interventions or problems. Users can be subscribed in two ways:
|
||||
|
||||
Reference in New Issue
Block a user