csaxs_bec/docs/user/saxs/data_analysis.md

(user.saxs.data_analysis)=

Data analysis

We can access the online computing resources from the beamline consoles when logged in as the e-account.

Allocation of nodes

Check available nodes and allocate one:

sinfo
salloc -A csaxs -p csaxs -w [ra-c-110]
ssh -Y [ra-c-110]

Or for a GPU node:

salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003]
ssh -Y [ra-gpu-003]

(note that we need to ask them to explicitly use one of our GPU nodes there, and this needs a couple of days to set up)

The home directory is the p group. To go to the raw directory we need to navigate there:

cd /sls/x12sa/data/[p22598]/raw

Python tools

Python package installation

Anaconda is no longer supported. Packages should now be installed directly using pip.

For example, pyFAI can be installed as follows:

module load Python/3.11.11
pip install "pyFAI[gui]"

To launch the pyFAI applications:

# Detector calibration
python -m pyFAI.app.calib2

# Batch integration
python -m pyFAI.app.integrate

Recommended: using a Python virtual environment

For long-term maintainability and reproducibility, it is recommended to use a dedicated Python virtual environment.

Create a virtual environment

cd /sls/x12sa/data/[p-group]/raw

module load Python/3.11.11
mkdir python-env
cd python-env

# Create a virtual environment named ".venv"
python3 -m venv .venv

# Activate the environment
source .venv/bin/activate

# Install required packages
pip install "pyFAI[gui]"

Use an existing virtual environment

cd /sls/x12sa/data/[p-group]/raw/python-env
source .venv/bin/activate

Troubleshooting

If you encounter the following error when launching GUI applications (e.g. pyFAI-calib2):

X11 connection rejected

reconnect to the analysis node with X11 forwarding enabled:

ssh -X [ra-c-110]

and then reactivate the virtual environment before launching the application.

Use Jupyter Notebook / Lab

Create a virtual environment following the instructions above, then:

pip install jupyter

To run Jupyter Lab:

jupyter lab --port 6006 --ip $(hostname) --no-browser

Setup online radial integration on Jungfraujoch

1. Calibrate the detector using pyFAI.

   Analyze a calibration scan carefully with pyFAI and save the resulting PONI file and pixel mask. Also determine the beam center position.

   :::{note} As of 2026-06-18, Jungfraujoch only accepts uint32 TIFF mask files. If the mask generated by pyFAI is stored as uint8, it needs to be converted to the required format before uploading. To do this (from the home directory /sls/x12sa/data/[p-group]/raw/):

   • Activate the python environment: source mask/.mask/bin/activate
   • Run python convert_mask_format.py -i <input .tif file path> -o <output .tiff file path> :::

2. Update the BEC configuration.

   Update the detector distance and beam center coordinates in the BEC configuration files using the calibration results.

3. Configure Jungfraujoch.

   Open the Jungfraujoch Expert Configuration panel in the frontend interface, for example:

   http://sls-jfjoch-001:8080/frontend
   

   • Upload the pixel mask under Pixel Mask.
   • Fill in the required parameters under Azimuthal Integration Settings.
   • Make sure to enable Force CPU calculation in FPGA workflow.
   • After uploading a new configuration, do not press "initialize", as this will remove all the previously uploaded configuration.

4. Detector rotation (if required).

   :::{note} Adding rotation of the detector still needs to be documented (likely configured in the BEC configuration). :::