diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml index 3a742db..554e591 100644 --- a/.gitea/workflows/ci.yml +++ b/.gitea/workflows/ci.yml @@ -50,7 +50,11 @@ jobs: uses: actions/checkout@v4 with: repository: bec/csaxs_bec - ref: "${{ inputs.BEC_PLUGIN_REPO_BRANCH || github.head_ref || github.sha }}" + # For pull_request events github.sha is the test-merge commit, which + # lives in the base repo, so this works for fork PRs too. Using + # github.head_ref here would try to fetch the fork's branch name from + # bec/csaxs_bec, where it does not exist -> checkout fails (exit 1). + ref: "${{ inputs.BEC_PLUGIN_REPO_BRANCH || github.sha }}" path: ./csaxs_bec - name: Lint for merge conflicts from template updates diff --git a/.gitea/workflows/create_update_pr.yml b/.gitea/workflows/create_update_pr.yml index 8992624..3481bf3 100644 --- a/.gitea/workflows/create_update_pr.yml +++ b/.gitea/workflows/create_update_pr.yml @@ -16,14 +16,14 @@ jobs: - name: Setup Python uses: actions/setup-python@v5 with: - python-version: '3.12' + python-version: "3.12" - name: Checkout uses: actions/checkout@v4 - name: Create virtualenv - run: | - python -m virtualenv .venv + run: | + python -m virtualenv .venv - name: Install tools run: | diff --git a/docs/assets/programming.svg b/docs/assets/programming.svg new file mode 100644 index 0000000..bc9c155 --- /dev/null +++ b/docs/assets/programming.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/safety-at-csaxs.pdf b/docs/assets/safety-at-csaxs.pdf new file mode 100644 index 0000000..ca413b8 Binary files /dev/null and b/docs/assets/safety-at-csaxs.pdf differ diff --git a/docs/developer/editing_docs.md b/docs/developer/editing_docs.md index 6481740..7b10a2c 100644 --- a/docs/developer/editing_docs.md +++ b/docs/developer/editing_docs.md @@ -38,11 +38,11 @@ Every content page starts with a **cross-reference label** followed by a single top-level heading: ```md -(user.saxs.data_analysis)= -# Data analysis +(user.saxs.online_integration)= +# Setup online radial integration on Jungfraujoch ``` -- The label `(user.saxs.data_analysis)=` is what other pages, the section landing +- The label `(user.saxs.online_integration)=` is what other pages, the section landing pages, and the `{ref}` role link to. **Do not delete or rename it** without updating every reference, or links silently break. - The `# Title` is the page's H1. Sphinx needs exactly one; section pages do not @@ -81,7 +81,7 @@ Available types include `note`, `tip`, `important`, `warning`, `caution`, `dange - **To another page in this documentation**, link by its label, not by file path: ```md - See {ref}`the data analysis guide `. + See {ref}`the Jungfraujoch setup guide `. ``` - **External links** use ordinary Markdown: `[pyFAI](https://pyfai.readthedocs.io)`. @@ -114,7 +114,7 @@ Available types include `note`, `tip`, `important`, `warning`, `caution`, `dange hidden: true --- - data_analysis + online_integration ``` ```` diff --git a/docs/index.md b/docs/index.md index b3503c9..85dd535 100644 --- a/docs/index.md +++ b/docs/index.md @@ -44,6 +44,7 @@ hidden: true --- introduction/introduction +safety/safety user/user developer/developer api_reference/api_reference diff --git a/docs/safety/safety.md b/docs/safety/safety.md new file mode 100644 index 0000000..a846d1f --- /dev/null +++ b/docs/safety/safety.md @@ -0,0 +1,48 @@ +(safety)= + +# Safety at cSAXS + +Working safely at the beamline is everyone's responsibility. All activities must +be conducted such that they are safe for anyone present — even people not familiar +with the ongoing work. Please read this page and the full checklist before +starting your experiment, and discuss any safety-related question with your local +contact. + +```{important} +**No experiment may be conducted before the signed Safety Declaration form has +been handed to your local contact.** Every person working at the beamline must be +named on that form. Violations result in the immediate termination of the +experiment. + +📄 **[Open the Safety Declaration form (PDF)](https://www.psi.ch/en/media/58834/download)** +``` + +```{admonition} In an emergency +:class: danger + +- **Call 3333** — be ready to state *Who? What? When? Where?* +- Know the location of the nearest **emergency exit**, **fire extinguisher**, + **emergency shower / eye wash**, and **first-aid kit**. +- Never hesitate to ask your local contact for advice or help. +``` + +## Safety checklist (printable PDF) + +Your local contact will go through the *Safety at cSAXS* checklist with you, covering emergencies, general behaviour, +working inside the hutch, and experiment safety. It is available as a printable PDF. + +{download}`⬇ Download the "Safety at cSAXS" checklist (PDF) <../assets/safety-at-csaxs.pdf>` + +## Key safety links + +Note that some of the links below point to the PSI **intranet** and are only reachable from within the PSI network or via your local contact. + +| Resource | Link | +| --- | --- | +| **Safety Declaration form** (sign and return before the experiment) | | +| User Office — Safety Guidelines | | +| Safety@PSI portal *(intranet)* | | +| PSI Safety Specialists — contact via your local contact *(intranet)* | | +| Behaviour during a gas / fire alarm *(intranet)* | | + +If anything is unclear, discuss it with your local contact, who can in turn seek advice from the PSI safety specialists. diff --git a/docs/user/saxs/data_analysis.md b/docs/user/data_analysis/computing.md similarity index 55% rename from docs/user/saxs/data_analysis.md rename to docs/user/data_analysis/computing.md index d50caf3..54283d4 100644 --- a/docs/user/saxs/data_analysis.md +++ b/docs/user/data_analysis/computing.md @@ -1,5 +1,5 @@ -(user.saxs.data_analysis)= -# Data analysis +(user.data_analysis.computing)= +# Online data analysis We can access the online computing resources from the beamline consoles when logged in as the e-account. @@ -16,12 +16,23 @@ ssh -Y [ra-c-110] Or for a GPU node: ```bash -salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003] +salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003] --mem=100G -n 8 ssh -Y [ra-gpu-003] ``` (note that we need to ask them to explicitly use one of our GPU nodes there, and this needs a couple of days to set up) +### Specifications of the computing nodes + +A few examples of the available nodes: + +| Node | CPU | Cores | Memory | GPUs | Partition | +|---|---|---|---|---|---| +| `ra-c-110` | Intel Xeon 6152 | 44 | 377 GB | — | `csaxs` | +| `ra-gpu-003` | AMD EPYC 7452 | 64 | 1007 GB | 4x NVIDIA A100 (40 GB) | `gpu-csaxs` | +| `ra-gpu-007` | Intel Xeon 6530 | 64 | 1007 GB | 4x NVIDIA L40S (46 GB)| `gpu-csaxs` | + + **The home directory is the p group.** To go to the raw directory we need to navigate there: ```bash @@ -81,7 +92,7 @@ cd /sls/x12sa/data/[p-group]/raw/python-env source .venv/bin/activate ``` -### Troubleshooting +#### Troubleshooting If you encounter the following error when launching GUI applications (e.g. `pyFAI-calib2`): @@ -97,7 +108,7 @@ ssh -X [ra-c-110] and then reactivate the virtual environment before launching the application. -## Use Jupyter Notebook / Lab +### Use Jupyter Notebook / Lab Create a virtual environment following the instructions above, then: @@ -110,40 +121,15 @@ To run Jupyter Lab: ```bash jupyter lab --port 6006 --ip $(hostname) --no-browser ``` +## cSAXS matlab scripts -## Setup online radial integration on Jungfraujoch +To use the cSAXS matlab scripts we can go to the `cxs_software` package in the raw folder. -1. **Calibrate the detector using pyFAI.** - - Analyze a calibration scan carefully with `pyFAI` and save the resulting **PONI** file and **pixel mask**. Also determine the beam center position. - - :::{note} - As of 2026-06-18, Jungfraujoch only accepts `uint32` TIFF mask files. If the mask generated by `pyFAI` is stored as `uint8`, it needs to be converted to the required format before uploading. - To do this (from the home directory `/sls/x12sa/data/[p-group]/raw/`): - - - Activate the python environment: `source mask/.mask/bin/activate` - - Run `python convert_mask_format.py -i -o ` - ::: - -2. **Update the BEC configuration.** - - Update the detector distance and beam center coordinates in the BEC configuration files using the calibration results. - -3. **Configure Jungfraujoch.** - - Open the **Jungfraujoch Expert Configuration** panel in the frontend interface, for example: - - ```text - http://sls-jfjoch-001:8080/frontend - ``` - - - Upload the pixel mask under **Pixel Mask**. - - Fill in the required parameters under **Azimuthal Integration Settings**. - - Make sure to enable **Force CPU calculation in FPGA workflow**. - - After uploading a new configuration, do not press "initialize", as this will remove all the previously uploaded configuration. - -4. **Detector rotation (if required).** - - :::{note} - Adding rotation of the detector still needs to be documented (likely configured in the BEC configuration). - ::: +In base we can do: +```bash +source setup-environment.sh +``` +And then we can use Matlab interactively as usual. +```bash +matlab & +``` diff --git a/docs/user/data_analysis/data_analysis.md b/docs/user/data_analysis/data_analysis.md new file mode 100644 index 0000000..c145f37 --- /dev/null +++ b/docs/user/data_analysis/data_analysis.md @@ -0,0 +1,34 @@ +(user.data_analysis)= +# Data analysis +Welcome to the data analysis section of the cSAXS beamline. + +```{toctree} +--- +maxdepth: 2 +hidden: true +--- + +computing + +``` + + +*** + +````{grid} 2 +:gutter: 5 + +```{grid-item-card} +:link: user.data_analysis.computing +:link-type: ref +:img-top: /assets/index_user_guide.svg +:text-align: center +:class-item: index-card + +## Online data analysis + +Allocating compute nodes, programming using Python and Matlab during the beamtime. +``` + +```` +For offline analysis, more detailed documentation on use of slurm on Ra, description of NoMachine setup is available at https://dari.pages.psi.ch/ra/compute.html/. diff --git a/docs/user/saxs/online_integration.md b/docs/user/saxs/online_integration.md new file mode 100644 index 0000000..6ff984f --- /dev/null +++ b/docs/user/saxs/online_integration.md @@ -0,0 +1,37 @@ +(user.saxs.online_integration)= +# Setup online radial integration on Jungfraujoch + +1. **Calibrate the detector using pyFAI.** + + Analyze a calibration scan carefully with `pyFAI` and save the resulting **PONI** file and **pixel mask**. Also determine the beam center position. + + :::{note} + As of 2026-06-18, Jungfraujoch only accepts `uint32` TIFF mask files. If the mask generated by `pyFAI` is stored as `uint8`, it needs to be converted to the required format before uploading. + To do this (from the home directory `/sls/x12sa/data/[p-group]/raw/`): + + - Activate the python environment: `source mask/.mask/bin/activate` + - Run `python convert_mask_format.py -i -o ` + ::: + +2. **Update the BEC configuration.** + + Update the detector distance and beam center coordinates in the BEC configuration files using the calibration results. + +3. **Configure Jungfraujoch.** + + Open the **Jungfraujoch Expert Configuration** panel in the frontend interface, for example: + + ```text + http://sls-jfjoch-001:8080/frontend + ``` + + - Upload the pixel mask under **Pixel Mask**. + - Fill in the required parameters under **Azimuthal Integration Settings**. + - Make sure to enable **Force CPU calculation in FPGA workflow**. + - After uploading a new configuration, do not press "initialize", as this will remove all the previously uploaded configuration. + +4. **Detector rotation (if required).** + + :::{note} + Adding rotation of the detector still needs to be documented (likely configured in the BEC configuration). + ::: diff --git a/docs/user/saxs/saxs.md b/docs/user/saxs/saxs.md index 5bef4d6..abaa235 100644 --- a/docs/user/saxs/saxs.md +++ b/docs/user/saxs/saxs.md @@ -8,7 +8,7 @@ maxdepth: 2 hidden: true --- -data_analysis +online_integration ``` @@ -19,15 +19,15 @@ data_analysis :gutter: 5 ```{grid-item-card} -:link: user.saxs.data_analysis +:link: user.saxs.online_integration :link-type: ref :img-top: /assets/biotech.svg :text-align: center :class-item: index-card -## Data analysis +## Online integration -Allocating compute nodes, pyFAI installation and usage, and online radial integration on Jungfraujoch. +Setup of online radial integration on Jungfraujoch. ``` ```` diff --git a/docs/user/user.md b/docs/user/user.md index 11081e7..763d47d 100644 --- a/docs/user/user.md +++ b/docs/user/user.md @@ -11,6 +11,7 @@ hidden: true before_you_arrive/before_you_arrive.md ptychography/ptychography.md saxs/saxs.md +data_analysis/data_analysis.md ``` @@ -55,5 +56,17 @@ All about ptychography measurements and reconstructions. All about SAXS measurements and analysis. ``` +```{grid-item-card} +:link: user.data_analysis +:link-type: ref +:img-top: /assets/programming.svg +:text-align: center +:class-item: index-card + +## Data analysis + +All about data analysis during the beamtime. +``` + ```` \ No newline at end of file