diff --git a/docs/assets/programming.svg b/docs/assets/programming.svg new file mode 100644 index 0000000..bc9c155 --- /dev/null +++ b/docs/assets/programming.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/developer/editing_docs.md b/docs/developer/editing_docs.md index 6481740..7b10a2c 100644 --- a/docs/developer/editing_docs.md +++ b/docs/developer/editing_docs.md @@ -38,11 +38,11 @@ Every content page starts with a **cross-reference label** followed by a single top-level heading: ```md -(user.saxs.data_analysis)= -# Data analysis +(user.saxs.online_integration)= +# Setup online radial integration on Jungfraujoch ``` -- The label `(user.saxs.data_analysis)=` is what other pages, the section landing +- The label `(user.saxs.online_integration)=` is what other pages, the section landing pages, and the `{ref}` role link to. **Do not delete or rename it** without updating every reference, or links silently break. - The `# Title` is the page's H1. Sphinx needs exactly one; section pages do not @@ -81,7 +81,7 @@ Available types include `note`, `tip`, `important`, `warning`, `caution`, `dange - **To another page in this documentation**, link by its label, not by file path: ```md - See {ref}`the data analysis guide `. + See {ref}`the Jungfraujoch setup guide `. ``` - **External links** use ordinary Markdown: `[pyFAI](https://pyfai.readthedocs.io)`. @@ -114,7 +114,7 @@ Available types include `note`, `tip`, `important`, `warning`, `caution`, `dange hidden: true --- - data_analysis + online_integration ``` ```` diff --git a/docs/user/saxs/data_analysis.md b/docs/user/data_analysis/computing.md similarity index 55% rename from docs/user/saxs/data_analysis.md rename to docs/user/data_analysis/computing.md index d50caf3..54283d4 100644 --- a/docs/user/saxs/data_analysis.md +++ b/docs/user/data_analysis/computing.md @@ -1,5 +1,5 @@ -(user.saxs.data_analysis)= -# Data analysis +(user.data_analysis.computing)= +# Online data analysis We can access the online computing resources from the beamline consoles when logged in as the e-account. @@ -16,12 +16,23 @@ ssh -Y [ra-c-110] Or for a GPU node: ```bash -salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003] +salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003] --mem=100G -n 8 ssh -Y [ra-gpu-003] ``` (note that we need to ask them to explicitly use one of our GPU nodes there, and this needs a couple of days to set up) +### Specifications of the computing nodes + +A few examples of the available nodes: + +| Node | CPU | Cores | Memory | GPUs | Partition | +|---|---|---|---|---|---| +| `ra-c-110` | Intel Xeon 6152 | 44 | 377 GB | — | `csaxs` | +| `ra-gpu-003` | AMD EPYC 7452 | 64 | 1007 GB | 4x NVIDIA A100 (40 GB) | `gpu-csaxs` | +| `ra-gpu-007` | Intel Xeon 6530 | 64 | 1007 GB | 4x NVIDIA L40S (46 GB)| `gpu-csaxs` | + + **The home directory is the p group.** To go to the raw directory we need to navigate there: ```bash @@ -81,7 +92,7 @@ cd /sls/x12sa/data/[p-group]/raw/python-env source .venv/bin/activate ``` -### Troubleshooting +#### Troubleshooting If you encounter the following error when launching GUI applications (e.g. `pyFAI-calib2`): @@ -97,7 +108,7 @@ ssh -X [ra-c-110] and then reactivate the virtual environment before launching the application. -## Use Jupyter Notebook / Lab +### Use Jupyter Notebook / Lab Create a virtual environment following the instructions above, then: @@ -110,40 +121,15 @@ To run Jupyter Lab: ```bash jupyter lab --port 6006 --ip $(hostname) --no-browser ``` +## cSAXS matlab scripts -## Setup online radial integration on Jungfraujoch +To use the cSAXS matlab scripts we can go to the `cxs_software` package in the raw folder. -1. **Calibrate the detector using pyFAI.** - - Analyze a calibration scan carefully with `pyFAI` and save the resulting **PONI** file and **pixel mask**. Also determine the beam center position. - - :::{note} - As of 2026-06-18, Jungfraujoch only accepts `uint32` TIFF mask files. If the mask generated by `pyFAI` is stored as `uint8`, it needs to be converted to the required format before uploading. - To do this (from the home directory `/sls/x12sa/data/[p-group]/raw/`): - - - Activate the python environment: `source mask/.mask/bin/activate` - - Run `python convert_mask_format.py -i -o ` - ::: - -2. **Update the BEC configuration.** - - Update the detector distance and beam center coordinates in the BEC configuration files using the calibration results. - -3. **Configure Jungfraujoch.** - - Open the **Jungfraujoch Expert Configuration** panel in the frontend interface, for example: - - ```text - http://sls-jfjoch-001:8080/frontend - ``` - - - Upload the pixel mask under **Pixel Mask**. - - Fill in the required parameters under **Azimuthal Integration Settings**. - - Make sure to enable **Force CPU calculation in FPGA workflow**. - - After uploading a new configuration, do not press "initialize", as this will remove all the previously uploaded configuration. - -4. **Detector rotation (if required).** - - :::{note} - Adding rotation of the detector still needs to be documented (likely configured in the BEC configuration). - ::: +In base we can do: +```bash +source setup-environment.sh +``` +And then we can use Matlab interactively as usual. +```bash +matlab & +``` diff --git a/docs/user/data_analysis/data_analysis.md b/docs/user/data_analysis/data_analysis.md new file mode 100644 index 0000000..c145f37 --- /dev/null +++ b/docs/user/data_analysis/data_analysis.md @@ -0,0 +1,34 @@ +(user.data_analysis)= +# Data analysis +Welcome to the data analysis section of the cSAXS beamline. + +```{toctree} +--- +maxdepth: 2 +hidden: true +--- + +computing + +``` + + +*** + +````{grid} 2 +:gutter: 5 + +```{grid-item-card} +:link: user.data_analysis.computing +:link-type: ref +:img-top: /assets/index_user_guide.svg +:text-align: center +:class-item: index-card + +## Online data analysis + +Allocating compute nodes, programming using Python and Matlab during the beamtime. +``` + +```` +For offline analysis, more detailed documentation on use of slurm on Ra, description of NoMachine setup is available at https://dari.pages.psi.ch/ra/compute.html/. diff --git a/docs/user/saxs/online_integration.md b/docs/user/saxs/online_integration.md new file mode 100644 index 0000000..6ff984f --- /dev/null +++ b/docs/user/saxs/online_integration.md @@ -0,0 +1,37 @@ +(user.saxs.online_integration)= +# Setup online radial integration on Jungfraujoch + +1. **Calibrate the detector using pyFAI.** + + Analyze a calibration scan carefully with `pyFAI` and save the resulting **PONI** file and **pixel mask**. Also determine the beam center position. + + :::{note} + As of 2026-06-18, Jungfraujoch only accepts `uint32` TIFF mask files. If the mask generated by `pyFAI` is stored as `uint8`, it needs to be converted to the required format before uploading. + To do this (from the home directory `/sls/x12sa/data/[p-group]/raw/`): + + - Activate the python environment: `source mask/.mask/bin/activate` + - Run `python convert_mask_format.py -i -o ` + ::: + +2. **Update the BEC configuration.** + + Update the detector distance and beam center coordinates in the BEC configuration files using the calibration results. + +3. **Configure Jungfraujoch.** + + Open the **Jungfraujoch Expert Configuration** panel in the frontend interface, for example: + + ```text + http://sls-jfjoch-001:8080/frontend + ``` + + - Upload the pixel mask under **Pixel Mask**. + - Fill in the required parameters under **Azimuthal Integration Settings**. + - Make sure to enable **Force CPU calculation in FPGA workflow**. + - After uploading a new configuration, do not press "initialize", as this will remove all the previously uploaded configuration. + +4. **Detector rotation (if required).** + + :::{note} + Adding rotation of the detector still needs to be documented (likely configured in the BEC configuration). + ::: diff --git a/docs/user/saxs/saxs.md b/docs/user/saxs/saxs.md index 5bef4d6..abaa235 100644 --- a/docs/user/saxs/saxs.md +++ b/docs/user/saxs/saxs.md @@ -8,7 +8,7 @@ maxdepth: 2 hidden: true --- -data_analysis +online_integration ``` @@ -19,15 +19,15 @@ data_analysis :gutter: 5 ```{grid-item-card} -:link: user.saxs.data_analysis +:link: user.saxs.online_integration :link-type: ref :img-top: /assets/biotech.svg :text-align: center :class-item: index-card -## Data analysis +## Online integration -Allocating compute nodes, pyFAI installation and usage, and online radial integration on Jungfraujoch. +Setup of online radial integration on Jungfraujoch. ``` ```` diff --git a/docs/user/user.md b/docs/user/user.md index 11081e7..763d47d 100644 --- a/docs/user/user.md +++ b/docs/user/user.md @@ -11,6 +11,7 @@ hidden: true before_you_arrive/before_you_arrive.md ptychography/ptychography.md saxs/saxs.md +data_analysis/data_analysis.md ``` @@ -55,5 +56,17 @@ All about ptychography measurements and reconstructions. All about SAXS measurements and analysis. ``` +```{grid-item-card} +:link: user.data_analysis +:link-type: ref +:img-top: /assets/programming.svg +:text-align: center +:class-item: index-card + +## Data analysis + +All about data analysis during the beamtime. +``` + ```` \ No newline at end of file