Update data analysis section

2026-07-03 10:12:21 +02:00
parent eecc6cf37b
commit 8cfc82dfe6
6 changed files with 192 additions and 122 deletions
@@ -38,11 +38,11 @@ Every content page starts with a **cross-reference label** followed by a single
 top-level heading:

 ```md
-(user.saxs.data_analysis)=
-# Data analysis
+(user.saxs.online_integration)=
+# Setup online radial integration on Jungfraujoch
 ```

- The label `(user.saxs.data_analysis)=` is what other pages, the section landing
+- The label `(user.saxs.online_integration)=` is what other pages, the section landing
  pages, and the `{ref}` role link to. **Do not delete or rename it** without
  updating every reference, or links silently break.
 - The `# Title` is the page's H1. Sphinx needs exactly one; section pages do not
@@ -81,7 +81,7 @@ Available types include `note`, `tip`, `important`, `warning`, `caution`, `dange
 - **To another page in this documentation**, link by its label, not by file path:

  ```md
-  See {ref}`the data analysis guide <user.saxs.data_analysis>`.
+  See {ref}`the Jungfraujoch setup guide <user.saxs.online_integration>`.
  ```

 - **External links** use ordinary Markdown: `[pyFAI](https://pyfai.readthedocs.io)`.
@@ -114,7 +114,7 @@ Available types include `note`, `tip`, `important`, `warning`, `caution`, `dange
   hidden: true
   ---

-   data_analysis
+   online_integration
   <name>
   ```
   ````
@@ -0,0 +1,135 @@
+(user.data_analysis.computing)=
+# Online data analysis
+
+We can access the online computing resources from the beamline consoles when logged in as the e-account.
+
+## Allocation of nodes
+
+Check available nodes and allocate one:
+
+```bash
+sinfo
+salloc -A csaxs -p csaxs -w [ra-c-110]
+ssh -Y [ra-c-110]
+```
+
+Or for a GPU node:
+
+```bash
+salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003] --mem=100G -n 8
+ssh -Y [ra-gpu-003]
+```
+
+(note that we need to ask them to explicitly use one of our GPU nodes there, and this needs a couple of days to set up)
+
+### Specifications of the computing nodes
+
+A few examples of the available nodes:
+
+| Node | CPU | Cores | Memory | GPUs | Partition |
+|---|---|---|---|---|---|
+| `ra-c-110` | Intel Xeon 6152 | 44 | 377 GB | — | `csaxs` |
+| `ra-gpu-003` | AMD EPYC 7452 | 64 | 1007 GB | 4x NVIDIA A100 (40 GB) | `gpu-csaxs` |
+| `ra-gpu-007` | Intel Xeon 6530 | 64 | 1007 GB | 4x NVIDIA L40S (46 GB)| `gpu-csaxs` |
+
+
+**The home directory is the p group.** To go to the raw directory we need to navigate there:
+
+```bash
+cd /sls/x12sa/data/[p22598]/raw
+```
+
+## Python tools
+
+### Python package installation
+
+Anaconda is no longer supported. Packages should now be installed directly using `pip`.
+
+For example, `pyFAI` can be installed as follows:
+
+```bash
+module load Python/3.11.11
+pip install "pyFAI[gui]"
+```
+
+To launch the `pyFAI` applications:
+
+```bash
+# Detector calibration
+python -m pyFAI.app.calib2
+
+# Batch integration
+python -m pyFAI.app.integrate
+```
+
+### Recommended: using a Python virtual environment
+
+For long-term maintainability and reproducibility, it is recommended to use a dedicated Python virtual environment.
+
+#### Create a virtual environment
+
+```bash
+cd /sls/x12sa/data/[p-group]/raw
+
+module load Python/3.11.11
+mkdir python-env
+cd python-env
+
+# Create a virtual environment named ".venv"
+python3 -m venv .venv
+
+# Activate the environment
+source .venv/bin/activate
+
+# Install required packages
+pip install "pyFAI[gui]"
+```
+
+#### Use an existing virtual environment
+
+```bash
+cd /sls/x12sa/data/[p-group]/raw/python-env
+source .venv/bin/activate
+```
+
+#### Troubleshooting
+
+If you encounter the following error when launching GUI applications (e.g. `pyFAI-calib2`):
+
+```text
+X11 connection rejected
+```
+
+reconnect to the analysis node with X11 forwarding enabled:
+
+```bash
+ssh -X [ra-c-110]
+```
+
+and then reactivate the virtual environment before launching the application.
+
+### Use Jupyter Notebook / Lab
+
+Create a virtual environment following the instructions above, then:
+
+```bash
+pip install jupyter
+```
+
+To run Jupyter Lab:
+
+```bash
+jupyter lab --port 6006 --ip $(hostname) --no-browser
+```
+## cSAXS matlab scripts
+
+To use the cSAXS matlab scripts we can go to the `cxs_software` package in the raw folder. 
+
+In base we can do:
+```bash
+source setup-environment.sh 
+```
+And then we can use Matlab interactively as usual.
+```bash
+matlab &
+```
@@ -0,0 +1,34 @@
+(user.data_analysis)=
+# Data analysis
+Welcome to the data analysis section of the cSAXS beamline. 
+
+```{toctree}
+---
+maxdepth: 2
+hidden: true
+---
+
+computing
+
+```
+
+
+***
+
+````{grid} 2
+:gutter: 5
+
+```{grid-item-card}
+:link: user.data_analysis.computing
+:link-type: ref
+:img-top: /assets/index_user_guide.svg
+:text-align: center
+:class-item: index-card
+
+## Online data analysis
+
+Allocating compute nodes, programming using Python and Matlab during the beamtime.
+```
+
+````
+For offline analysis, more detailed documentation on use of slurm on Ra, description of NoMachine setup is available at https://dari.pages.psi.ch/ra/compute.html/.
@@ -1,117 +1,5 @@
-(user.saxs.data_analysis)=
-# Data analysis
-
-We can access the online computing resources from the beamline consoles when logged in as the e-account.
-
-## Allocation of nodes
-
-Check available nodes and allocate one:
-
-```bash
-sinfo
-salloc -A csaxs -p csaxs -w [ra-c-110]
-ssh -Y [ra-c-110]
-```
-
-Or for a GPU node:
-
-```bash
-salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003]
-ssh -Y [ra-gpu-003]
-```
-
-(note that we need to ask them to explicitly use one of our GPU nodes there, and this needs a couple of days to set up)
-
-**The home directory is the p group.** To go to the raw directory we need to navigate there:
-
-```bash
-cd /sls/x12sa/data/[p22598]/raw
-```
-
-## Python tools
-
-### Python package installation
-
-Anaconda is no longer supported. Packages should now be installed directly using `pip`.
-
-For example, `pyFAI` can be installed as follows:
-
-```bash
-module load Python/3.11.11
-pip install "pyFAI[gui]"
-```
-
-To launch the `pyFAI` applications:
-
-```bash
-# Detector calibration
-python -m pyFAI.app.calib2
-
-# Batch integration
-python -m pyFAI.app.integrate
-```
-
-### Recommended: using a Python virtual environment
-
-For long-term maintainability and reproducibility, it is recommended to use a dedicated Python virtual environment.
-
-#### Create a virtual environment
-
-```bash
-cd /sls/x12sa/data/[p-group]/raw
-
-module load Python/3.11.11
-mkdir python-env
-cd python-env
-
-# Create a virtual environment named ".venv"
-python3 -m venv .venv
-
-# Activate the environment
-source .venv/bin/activate
-
-# Install required packages
-pip install "pyFAI[gui]"
-```
-
-#### Use an existing virtual environment
-
-```bash
-cd /sls/x12sa/data/[p-group]/raw/python-env
-source .venv/bin/activate
-```
-
-### Troubleshooting
-
-If you encounter the following error when launching GUI applications (e.g. `pyFAI-calib2`):
-
-```text
-X11 connection rejected
-```
-
-reconnect to the analysis node with X11 forwarding enabled:
-
-```bash
-ssh -X [ra-c-110]
-```
-
-and then reactivate the virtual environment before launching the application.
-
-## Use Jupyter Notebook / Lab
-
-Create a virtual environment following the instructions above, then:
-
-```bash
-pip install jupyter
-```
-
-To run Jupyter Lab:
-
-```bash
-jupyter lab --port 6006 --ip $(hostname) --no-browser
-```
-
-## Setup online radial integration on Jungfraujoch
+(user.saxs.online_integration)=
+# Setup online radial integration on Jungfraujoch

 1. **Calibrate the detector using pyFAI.**

@@ -8,7 +8,7 @@ maxdepth: 2
 hidden: true
 ---

-data_analysis
+online_integration

 ```

@@ -19,7 +19,7 @@ data_analysis
 :gutter: 5

 ```{grid-item-card}
-:link: user.saxs.data_analysis
+:link: user.saxs.online_integration
 :link-type: ref
 :img-top: /assets/biotech.svg
 :text-align: center
@@ -27,7 +27,7 @@ data_analysis

 ## Data analysis

-Allocating compute nodes, pyFAI installation and usage, and online radial integration on Jungfraujoch.
+Setup of online radial integration on Jungfraujoch.
 ```

 ````
@@ -11,6 +11,7 @@ hidden: true
 before_you_arrive/before_you_arrive.md
 ptychography/ptychography.md
 saxs/saxs.md
+data_analysis/data_analysis.md

 ```

@@ -55,5 +56,17 @@ All about ptychography measurements and reconstructions.

 All about SAXS measurements and analysis.
 ```
+```{grid-item-card}
+:link: user.data_analysis
+:link-type: ref
+:img-top: /assets/biotech.svg
+:text-align: center
+:class-item: index-card
+
+## Data analysis
+
+All about data analysis during the beamtime.
+```
+

 ````