menzelandClaude Opus 4.8
19aa25c7a7
Migrate the SAXS data-analysis content into the RTD/Sphinx docs as user/saxs/data_analysis.md, and turn the empty saxs.md stub into a landing page linking to it. Source: cSAXS documentation wiki, page "SAXS-data-analysis" https://gitea.psi.ch/cSAXS/documentation/wiki/SAXS-data-analysis (git@gitea.psi.ch:cSAXS/documentation.wiki.git, as of 408e535) Converted Gitea-flavored Markdown to MyST: added cross-reference labels, bash code fences, and {note} admonitions. Also bundle the SAXS standards plate datasheet (SAXS_standards_plate.pdf), previously at cSAXS/documentation/attachments/, linked via the MyST {download} role so Sphinx copies it into the build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
150 lines
3.7 KiB
Markdown
150 lines
3.7 KiB
Markdown
(user.saxs.data_analysis)=
|
|
# Data analysis
|
|
|
|
We can access the online computing resources from the beamline consoles when logged in as the e-account.
|
|
|
|
## Allocation of nodes
|
|
|
|
Check available nodes and allocate one:
|
|
|
|
```bash
|
|
sinfo
|
|
salloc -A csaxs -p csaxs -w [ra-c-110]
|
|
ssh -Y [ra-c-110]
|
|
```
|
|
|
|
Or for a GPU node:
|
|
|
|
```bash
|
|
salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003]
|
|
ssh -Y [ra-gpu-003]
|
|
```
|
|
|
|
(note that we need to ask them to explicitly use one of our GPU nodes there, and this needs a couple of days to set up)
|
|
|
|
**The home directory is the p group.** To go to the raw directory we need to navigate there:
|
|
|
|
```bash
|
|
cd /sls/x12sa/data/[p22598]/raw
|
|
```
|
|
|
|
## Python tools
|
|
|
|
### Python package installation
|
|
|
|
Anaconda is no longer supported. Packages should now be installed directly using `pip`.
|
|
|
|
For example, `pyFAI` can be installed as follows:
|
|
|
|
```bash
|
|
module load Python/3.11.11
|
|
pip install "pyFAI[gui]"
|
|
```
|
|
|
|
To launch the `pyFAI` applications:
|
|
|
|
```bash
|
|
# Detector calibration
|
|
python -m pyFAI.app.calib2
|
|
|
|
# Batch integration
|
|
python -m pyFAI.app.integrate
|
|
```
|
|
|
|
### Recommended: using a Python virtual environment
|
|
|
|
For long-term maintainability and reproducibility, it is recommended to use a dedicated Python virtual environment.
|
|
|
|
#### Create a virtual environment
|
|
|
|
```bash
|
|
cd /sls/x12sa/data/[p-group]/raw
|
|
|
|
module load Python/3.11.11
|
|
mkdir python-env
|
|
cd python-env
|
|
|
|
# Create a virtual environment named ".venv"
|
|
python3 -m venv .venv
|
|
|
|
# Activate the environment
|
|
source .venv/bin/activate
|
|
|
|
# Install required packages
|
|
pip install "pyFAI[gui]"
|
|
```
|
|
|
|
#### Use an existing virtual environment
|
|
|
|
```bash
|
|
cd /sls/x12sa/data/[p-group]/raw/python-env
|
|
source .venv/bin/activate
|
|
```
|
|
|
|
### Troubleshooting
|
|
|
|
If you encounter the following error when launching GUI applications (e.g. `pyFAI-calib2`):
|
|
|
|
```text
|
|
X11 connection rejected
|
|
```
|
|
|
|
reconnect to the analysis node with X11 forwarding enabled:
|
|
|
|
```bash
|
|
ssh -X [ra-c-110]
|
|
```
|
|
|
|
and then reactivate the virtual environment before launching the application.
|
|
|
|
## Use Jupyter Notebook / Lab
|
|
|
|
Create a virtual environment following the instructions above, then:
|
|
|
|
```bash
|
|
pip install jupyter
|
|
```
|
|
|
|
To run Jupyter Lab:
|
|
|
|
```bash
|
|
jupyter lab --port 6006 --ip $(hostname) --no-browser
|
|
```
|
|
|
|
## Setup online radial integration on Jungfraujoch
|
|
|
|
1. **Calibrate the detector using pyFAI.**
|
|
|
|
Analyze a calibration scan carefully with `pyFAI` and save the resulting **PONI** file and **pixel mask**. Also determine the beam center position.
|
|
|
|
:::{note}
|
|
As of 2026-06-18, Jungfraujoch only accepts `uint32` TIFF mask files. If the mask generated by `pyFAI` is stored as `uint8`, it needs to be converted to the required format before uploading.
|
|
To do this (from the home directory `/sls/x12sa/data/[p-group]/raw/`):
|
|
|
|
- Activate the python environment: `source mask/.mask/bin/activate`
|
|
- Run `python convert_mask_format.py -i <input .tif file path> -o <output .tiff file path>`
|
|
:::
|
|
|
|
2. **Update the BEC configuration.**
|
|
|
|
Update the detector distance and beam center coordinates in the BEC configuration files using the calibration results.
|
|
|
|
3. **Configure Jungfraujoch.**
|
|
|
|
Open the **Jungfraujoch Expert Configuration** panel in the frontend interface, for example:
|
|
|
|
```text
|
|
http://sls-jfjoch-001:8080/frontend
|
|
```
|
|
|
|
- Upload the pixel mask under **Pixel Mask**.
|
|
- Fill in the required parameters under **Azimuthal Integration Settings**.
|
|
- Make sure to enable **Force CPU calculation in FPGA workflow**.
|
|
- After uploading a new configuration, do not press "initialize", as this will remove all the previously uploaded configuration.
|
|
|
|
4. **Detector rotation (if required).**
|
|
|
|
:::{note}
|
|
Adding rotation of the detector still needs to be documented (likely configured in the BEC configuration).
|
|
:::
|