diff --git a/docs/merlin7/05-Software-Support/dirac.md b/docs/merlin7/05-Software-Support/dirac.md new file mode 100644 index 0000000..72ff629 --- /dev/null +++ b/docs/merlin7/05-Software-Support/dirac.md @@ -0,0 +1,210 @@ +# DIRAC for Mu3e + +## User Introduction & Setup Guide + +This document introduces DIRAC and Mu3e and describes the initial client setup. + +!!! Warning + **Status:** Work in progress + + Further sections (catalog access, job submission, data management) **are intentionally incomplete and will be extended**. + +### What is DIRAC? + +**DIRAC** (Distributed Infrastructure with Remote Agent Control) is a distributed computing framework widely used in high-energy physics and other scientific communities. It provides: + +* Secure authentication using X.509 certificates +* Workload and data management across multiple computing centers +* A unified interface for grid, cloud, and HPC resources + +DIRAC uses a **central Configuration Service**, a **Proxy Manager**, and multiple backend services to allow users to submit jobs and manage data securely. + +### What is Mu3e? + +**[Mu3e](https://www.psi.ch/de/mu3e)** is a particle physics experiment at the Paul Scherrer Institute (PSI) searching for the rare decay: +``` +μ⁺ → e⁺ e⁻ e⁺ +``` + +Mu3e relies on distributed computing resources for simulation, reconstruction, and data analysis. +DIRAC is used as the Distributed Data Management and Workload Management System that can be used by the Mu3e experiment. + +## Configuring DIRAC for the Mu3e experiment + +### Prerequisites + +Before starting, you need: + +* A valid X.509 user certificate (and private key). +* Access to the Mu3e DIRAC VO. + +### Install User Certificates + +DIRAC relies on standard Grid certificates. This often comes in `.p12` format, and the private key and user certificate +should be retreived from it, as follows: + +```bash +openssl pkcs12 -in my_personal_CERN_cert.p12 -clcerts -nokeys -out usercert.pem +openssl pkcs12 -in my_personal_CERN_cert.p12 -nocerts -nodes -out userkey.pem +``` + +Once retrieved, place your personal certificate and private key in: +```bash +~/.globus +``` + +!!! tip + `userkey.pem` must be `chmod 600` + +### Install DIRAC Client Environment (DIRACOS) + +**DIRAC** provides a self-contained runtime called **DIRACOS**. +```bash +mkdir ~/dirac_ui +cd ~/dirac_ui + +curl -LO https://github.com/DIRACGrid/DIRACOS2/releases/latest/download/DIRACOS-Linux-$(uname -m).sh +bash DIRACOS-Linux-$(uname -m).sh + +rm DIRACOS-Linux-$(uname -m).sh +``` + +Active the environment +```bash +source ~/dirac_ui/diracos/diracosrc +``` + +Install the DIRAC Python client: +```bash +pip install DIRAC # Alternative DIRAC==$VERSION, e.g. DIRAC==8.0.74 +``` + +### Install Certificate Authorities (CAs) + +**DIRAC** requires trusted CA certificates to validate remote services. Merlin7 does not provide CA certificates +and users are responsible for managing and updating them. + +!!! warning + Users are responsible to maintain the CAs up to date + +Download the required CA bundles: +```bash +mkdir -p ~/Downloads +cd ~/Downloads + +wget https://repository.egi.eu/sw/production/cas/1/current/tgz/ca_UKeScienceCA-2B-1.138.tar.gz +wget https://repository.egi.eu/sw/production/cas/1/current/tgz/ca_UKeScienceRoot-2007-1.138.tar.gz +``` + +Install them into DIRACOS +```bash +cd ~/dirac_ui + +tar xvzf ~/Downloads/ca_UKeScienceRoot-2007-1.138.tar.gz \ + --strip-components=1 \ + -C diracos/etc/grid-security/certificates/ + +tar xvzf ~/Downloads/ca_UKeScienceCA-2B-1.138.tar.gz \ + --strip-components=1 \ + -C diracos/etc/grid-security/certificates/ +``` + +### Fetch Certificate Revocation Lists (CRLs) + +CRLs are required to ensure certificates have not been revoked. + +!!! warning + Users are responsible to maintain this list up to date. + +Load fetch-crl and update CRLs: +```bash +module use unstable +module load fetch-crl + +fetch-crl -lv $HOME/dirac_ui/diracos/etc/grid-security/certificates/ +``` + +### Initialize a Local Proxy (No Configuration Service) + +Create a local proxy without contacting the Configuration Service (useful for first setup and debugging): +```bash +dirac-proxy-init --nocs +``` + +### Configure DIRAC for Mu3e + +Configure the DIRAC client to use the GridPP setup and configuration server: +```bash +dirac-configure -F \ + -S GridPP \ + -C dips://diracweb.grid.hep.ph.ic.ac.uk:9135/Configuration/Server \ + -I +``` + +Set the certificate directory explicitly (important when using DIRACOS): +```bash +export X509_CERT_DIR=$HOME/dirac_ui/diracos/etc/grid-security/certificates +``` + +### Initialize a Mu3e DIRAC Proxy + +Create and upload a proxy for the Mu3e VO: +```bash +dirac-proxy-init -g mu3e.org_user -M +``` + +Verify: +```bash +dirac-proxy-info +``` + +## Next Steps (Work in Progress) + +!!! warning + **WORK IN PROGRESS**, not fully documented yet + + **As of today, no services are available.** What's possible and how to set it up accordingly + needs to be clarified with the experiment. + +Once the proxy and configuration are in place, users can start interacting with Mu3e's DIRAC services. + +### Accessing the File Catalog (WIP) + +DIRAC provides a File Catalog that tracks all Mu3e datasets and their replicas across storage elements. + +Examples of operations that will be documented: + +* Listing directories and files +* Querying logical file names (LFNs) +* Checking replicas and storage locations +* Uploading and downloading files + +Tools involved include: + +* `dirac-dms-filecatalog-cli` +* `dirac-dms-lfn-replicas` +* `dirac-dms-get-file` + +### Submitting Jobs (WIP) + +DIRAC supports job submission to distributed resources using: + +* Job Description Language (JDL) +* Python-based DIRAC job APIs +* Command-line submission tools + +Planned topics: + +* Writing a minimal JDL +* Submitting and monitoring jobs +* Retrieving job outputs +* Debugging failed jobs + +### Other DIRAC Tools (WIP) + +Further DIRAC components to be documented: + +* Data Management System (DMS) +* Workload Management System (WMS) +* Production system +* Monitoring and accounting tools diff --git a/mkdocs.yml b/mkdocs.yml index fafcbfe..072ba30 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -118,18 +118,19 @@ nav: - Jupyterhub: - merlin7/04-Jupyterhub/jupyterhub.md - Software Support: - - merlin7/05-Software-Support/pmodules.md - - merlin7/05-Software-Support/spack.md - - merlin7/05-Software-Support/openmpi.md - merlin7/05-Software-Support/ansys.md - merlin7/05-Software-Support/ansys-rsm.md - - merlin7/05-Software-Support/cp2k.md - - merlin7/05-Software-Support/quantum-espresso.md - - merlin7/05-Software-Support/lammps.md - - merlin7/05-Software-Support/gromacs.md - - merlin7/05-Software-Support/opal-x.md - - merlin7/05-Software-Support/ippl.md - merlin7/05-Software-Support/cray-module.env.md + - merlin7/05-Software-Support/cp2k.md + - merlin7/05-Software-Support/dirac.md + - merlin7/05-Software-Support/gromacs.md + - merlin7/05-Software-Support/ippl.md + - merlin7/05-Software-Support/lammps.md + - merlin7/05-Software-Support/opal-x.md + - merlin7/05-Software-Support/openmpi.md + - merlin7/05-Software-Support/pmodules.md + - merlin7/05-Software-Support/quantum-espresso.md + - merlin7/05-Software-Support/spack.md - Merlin 6: - Quick Start Guide: - merlin6/introduction.md