(user.data_analysis.computing)=
# Online data analysis

We can access the online computing resources from the beamline consoles when logged in as the e-account.

## Allocation of nodes

Check available nodes and allocate one:

```bash
sinfo
salloc -A csaxs -p csaxs -w [ra-c-110]
ssh -Y [ra-c-110]
```

Or for a GPU node:

```bash
salloc -A csaxs -p gpu-csaxs --gpus=4 [-w ra-gpu-003] --mem=100G -n 8
ssh -Y [ra-gpu-003]
```

(note that we need to ask them to explicitly use one of our GPU nodes there, and this needs a couple of days to set up)

### Specifications of the computing nodes

A few examples of the available nodes:

| Node | CPU | Cores | Memory | GPUs | Partition |
|---|---|---|---|---|---|
| `ra-c-110` | Intel Xeon 6152 | 44 | 377 GB | — | `csaxs` |
| `ra-gpu-003` | AMD EPYC 7452 | 64 | 1007 GB | 4x NVIDIA A100 (40 GB) | `gpu-csaxs` |
| `ra-gpu-007` | Intel Xeon 6530 | 64 | 1007 GB | 4x NVIDIA L40S (46 GB)| `gpu-csaxs` |


**The home directory is the p group.** To go to the raw directory we need to navigate there:

```bash
cd /sls/x12sa/data/[p22598]/raw
```

## Python tools

### Python package installation

Anaconda is no longer supported. Packages should now be installed directly using `pip`.

For example, `pyFAI` can be installed as follows:

```bash
module load Python/3.11.11
pip install "pyFAI[gui]"
```

To launch the `pyFAI` applications:

```bash
# Detector calibration
python -m pyFAI.app.calib2

# Batch integration
python -m pyFAI.app.integrate
```

### Recommended: using a Python virtual environment

For long-term maintainability and reproducibility, it is recommended to use a dedicated Python virtual environment.

#### Create a virtual environment

```bash
cd /sls/x12sa/data/[p-group]/raw

module load Python/3.11.11
mkdir python-env
cd python-env

# Create a virtual environment named ".venv"
python3 -m venv .venv

# Activate the environment
source .venv/bin/activate

# Install required packages
pip install "pyFAI[gui]"
```

#### Use an existing virtual environment

```bash
cd /sls/x12sa/data/[p-group]/raw/python-env
source .venv/bin/activate
```

#### Troubleshooting

If you encounter the following error when launching GUI applications (e.g. `pyFAI-calib2`):

```text
X11 connection rejected
```

reconnect to the analysis node with X11 forwarding enabled:

```bash
ssh -X [ra-c-110]
```

and then reactivate the virtual environment before launching the application.

### Use Jupyter Notebook / Lab

Create a virtual environment following the instructions above, then:

```bash
pip install jupyter
```

To run Jupyter Lab:

```bash
jupyter lab --port 6006 --ip $(hostname) --no-browser
```
## cSAXS matlab scripts

To use the cSAXS matlab scripts we can go to the `cxs_software` package in the raw folder. 

In base we can do:
```bash
source setup-environment.sh 
```
And then we can use Matlab interactively as usual.
```bash
matlab &
```