Update meg docs
All checks were successful
Build and Deploy Documentation / build-and-deploy (push) Successful in 6s
All checks were successful
Build and Deploy Documentation / build-and-deploy (push) Successful in 6s
This commit is contained in:
@@ -111,18 +111,21 @@ This script is primarily a **wrapper** around `fpsync`, providing additional log
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
[root@meg-s-001 ~]# experiment_migration.bash --help
|
[root@meg-s-001 ~]# experiment_migration.bash --help
|
||||||
Usage: ./experiment_migration.bash [options] -p <project_name>
|
Usage: /usr/local/bin/experiment_migration.bash [options] -p <project_name>
|
||||||
|
|
||||||
Options:
|
Options:
|
||||||
-t | --threads N Number of parallel threads (default: 10). Recommended 12 as max.
|
-t | --threads N Number of parallel threads (default: 10). Recommended 12 as max.
|
||||||
-b | --experiment-src-basedir DIR Experiment base directory (default: /data/experiment/meg)
|
-b | --experiment-src-basedir DIR Experiment base directory (default: /meg)
|
||||||
-B | --experiment-dst-basedir DIR Experiment base directory (default: /data/experiment/meg)
|
|
||||||
-S | --space-source SPACE Source project space name (default: data1)
|
-S | --space-source SPACE Source project space name (default: data1)
|
||||||
|
-B | --experiment-dst-basedir DIR Experiment base directory (default: /data/project/meg)
|
||||||
-D | --space-destination SPACE Destination project space name (default: data1)
|
-D | --space-destination SPACE Destination project space name (default: data1)
|
||||||
-p | --project-name PRJ_NAME Mantadory field. MeG project name. Examples:
|
-p | --project-name PRJ_NAME Mantadory field. MeG project name. Examples:
|
||||||
- 'online'
|
- 'online'
|
||||||
- 'offline'
|
- 'offline'
|
||||||
- 'shared'
|
- 'shared
|
||||||
|
-F | --force-destination-mkdir Create the destination parent directory (default: false)
|
||||||
|
Example: mkdir -p $(dirname /data/project/meg/data1/PROJECT_NAME)
|
||||||
|
Result: mkdir -p /data/project/meg/data1
|
||||||
-s | --split N Number of files per split (default: 20000)
|
-s | --split N Number of files per split (default: 20000)
|
||||||
-f | --filesize SIZE File size threshold (default: 100G)
|
-f | --filesize SIZE File size threshold (default: 100G)
|
||||||
-r | --runid ID Reuse an existing runid session
|
-r | --runid ID Reuse an existing runid session
|
||||||
@@ -154,14 +157,14 @@ However, migrating full directories is generally <b>simpler</b> and <b>less erro
|
|||||||
{{site.data.alerts.end}}
|
{{site.data.alerts.end}}
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
[root@meg-s-001 bin]# experiment_migration.bash -b /meg -B /data/project/meg -S data1 -D data1 -p "online"
|
[root@meg-s-001 ~]# experiment_migration.bash -S data1 -D data1 -p "online"
|
||||||
🔄 Transferring project:
|
🔄 Transferring project:
|
||||||
From: /meg/data1/online
|
From: /meg/data1/online
|
||||||
To: login002.merlin7.psi.ch:/data/project/meg/data1/online
|
To: login001.merlin7.psi.ch:/data/project/meg/data1/online
|
||||||
Threads: 10 | Split: 20000 files | Max size: 100G
|
Threads: 10 | Split: 20000 files | Max size: 100G
|
||||||
RunID:
|
RunID:
|
||||||
|
|
||||||
Please confirm to start (y/N): N
|
Please confirm to start (y/N):
|
||||||
❌ Transfer cancelled by user.
|
❌ Transfer cancelled by user.
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -174,7 +177,7 @@ The following example demonstrates how to migrate **only a subdirectory**.
|
|||||||
- For example, avoid running migrations with `-p "shared"` while simultaneously migrating `-p "shared/subprojects"`.
|
- For example, avoid running migrations with `-p "shared"` while simultaneously migrating `-p "shared/subprojects"`.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
[root@meg-s-001 bin]# experiment_migration.bash -b /meg -B /data/project/meg -S data1 -D data1 -p "shared/subprojects/meg1"
|
[root@meg-s-001 ~]# experiment_migration.bash -p "shared/subprojects/meg1" -F
|
||||||
🔄 Transferring project:
|
🔄 Transferring project:
|
||||||
From: /meg/data1/shared/subprojects/meg1
|
From: /meg/data1/shared/subprojects/meg1
|
||||||
To: login002.merlin7.psi.ch:/data/project/meg/data1/shared/subprojects/meg1
|
To: login002.merlin7.psi.ch:/data/project/meg/data1/shared/subprojects/meg1
|
||||||
@@ -182,6 +185,15 @@ The following example demonstrates how to migrate **only a subdirectory**.
|
|||||||
RunID:
|
RunID:
|
||||||
|
|
||||||
Please confirm to start (y/N): N
|
Please confirm to start (y/N): N
|
||||||
|
❌ Transfer cancelled by user.
|
||||||
```
|
```
|
||||||
|
|
||||||
This command initiates the migration of the directory: `/meg/data1/shared/subprojects/meg1` → `login002.merlin7.psi.ch:/data/project/meg/data1/shared/subprojects/meg1`
|
This command initiates the migration of the directory, by creating the destination parant directory (`-F` option):
|
||||||
|
* Creates the destination directory as follows:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh login002.merlin.psi.ch mkdir -p /data/project/meg/data1/shared/subprojects
|
||||||
|
```
|
||||||
|
* Runs FPSYNC with 10 threads and N parts of max 20000 files or 100G files:
|
||||||
|
* Source: `/meg/data1/shared/subprojects/meg1`
|
||||||
|
* Destination: `login002.merlin7.psi.ch:/data/project/meg/data1/shared/subprojects/meg1`
|
||||||
|
|||||||
Reference in New Issue
Block a user