Jungfraujoch/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

Jungfraujoch is the data-acquisition and analysis system for the PSI JUNGFRAU and EIGER
X-ray detectors. It receives detector data, runs it through an FPGA-accelerated pipeline
(spot finding, azimuthal/ROI integration, compression), streams images out over ZeroMQ for
writing to HDF5, and runs crystallographic analysis (indexing, integration, scaling/merging).
Most authoritative documentation lives in `docs/` and on Read The Docs
(https://jungfraujoch.readthedocs.io). When changing CLI behaviour, the program's own
usage message is the source of truth, not the docs.

## Build

Out-of-source CMake build, C++20, heavy use of `FetchContent` (spdlog, zstd, HDF5,
slsDetectorPackage, Catch2, cpp-httplib, libzmq, Ceres, fast-feedback-indexer are downloaded
and statically linked — the **first configure needs network access** and is slow).

```
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j$(nproc) jfjoch_broker      # the main service; build other targets by name
```

Key CMake options:
- `JFJOCH_USE_CUDA` (default ON) — GPU path. Needs CUDA ≥ 12.8. Provides the `ffbidx` and `fft`
  GPU indexers; without it only the CPU `fftw` indexer is available (requires FFTW at configure
  time, auto-detected). CUDA absence is not a build error.
- `JFJOCH_WRITER_ONLY` (default OFF) — builds only the HDF5 writer; skips broker, FPGA,
  receiver, analysis, tests, frontend.
- `JFJOCH_VIEWER_BUILD` (default OFF) — builds the Qt6 `jfjoch_viewer` desktop app.
- `SLS9` (default OFF) — build against slsDetectorPackage 9.2.0 instead of 8.0.2.

The frontend is a separate custom target: `make frontend` (runs `npm ci && npm run build` in
`frontend/`). It is not built by default unless installing.

## Test

Tests use **Catch2** and are collected into a single binary `tests/jfjoch_test`.

```
make -j$(nproc) jfjoch_test
cd tests
./jfjoch_test                       # all tests
./jfjoch_test "<test name>"         # one test case (exact name in TEST_CASE)
./jfjoch_test "[tag]"               # by tag
./jfjoch_test -r junit -o report.xml
```

`make jfjoch_hdf5_test` builds the HDF5 write-speed benchmark, also used by CI to produce files
that are validated against XDS (Durin/Neggia) and CrystFEL.

Lint config is `.clang-tidy` (broad `*` check set with many exceptions; namespaces lower_case,
classes CamelCase, global constants UPPER_CASE).

## Code style

The overriding principle is **simple, readable code** — favour the smallest, most direct
implementation that a reader can verify at a glance. Extra abstraction, speculative guards, and
clever-but-dense constructs are treated as actively harmful, not as polish. When torn between a
tidy abstraction and a flat, obvious version, pick the obvious one.

This matters most in `image_analysis/pixel_refinement/` (e.g. `PixelRefine.cpp`), which is an
experimental prototype where readability is how the physics gets verified — keep it especially
plain. **Do not add defensive/unrequested code** (extra validation, rejection heuristics, "just in
case" branches) without asking first; if a guard isn't clearly needed, leave it out.

Match the surrounding code's idiom, naming, and comment density rather than importing a different
style.

## Local end-to-end run (no detector / no FPGA)

The FPGA HLS logic can be simulated on the CPU (`HLSSimulatedDevice`), so the full software
stack runs without hardware (slowly — fixed-point math on CPU). See
`docs/JFJOCH_BROKER.md` for the canonical walkthrough.

```
cd build/broker
./jfjoch_broker ../../etc/broker_local.json 5232      # config JSON + HTTP port
# then, separately:
cd tests/test_data && python jfjoch_broker_test.py     # feeds a test image, starts collection
# observe at http://localhost:5232 ; HDF5 is written under build/broker
```

`etc/broker_local.json`, `broker_eiger.json`, `broker_crmx.json` are example broker configs
(schema = `jfjoch_settings` in `broker/jfjoch_api.yaml`).

## Architecture

**Data flow (online):** detector → FPGA acquisition (`fpga/`, `acquisition_device/`) →
`receiver/` builds full images from per-module FPGA output → `image_pusher/` streams CBOR-encoded
images over ZeroMQ → `jfjoch_writer` (`writer/`) consumes the stream and writes NXmx HDF5. The
broker also emits a low-rate preview stream and a metadata stream (`preview/`).

**Writer file split:** one acquisition produces one `_master.h5` plus many `_data_NNNNNN.h5`
files. Dataset-wide metadata (geometry, detector config, ROI/azimuthal definitions — anything
fixed for the whole run) is written to the **master** file in `writer/HDF5NXmx.cpp` (the `NXmx`
class). Per-image arrays (one entry per frame) are written to the **data** files by the
`HDF5DataFilePlugin` subclasses in `writer/`. Put shared metadata in `NXmx`, not in a data-file
plugin.

The HDF5 master/data layout is one of three `FileWriterFormat`s (`common/JFJochMessages.h`),
all NXmx: **`NXmxLegacy`** (master + `_data_NNNNNN.h5` joined by external links),
**`NXmxVDS`** (master + data joined by HDF5 virtual datasets — the default), and
**`NXmxIntegrated`** (a single self-contained file, no separate data files). Per-image plugins
must work for all three; with `NXmxIntegrated` "master" and "data" are the same file.

**Two acquisition workflows:** the FPGA-accelerated path (JUNGFRAU at PSI; FPGA does masking,
summation, spot finding, ROI/azimuthal integration, compression) and the DECTRIS SIMPLON path
(EIGER), which has no FPGA — masking/ROI/azimuthal analysis then runs on CPU through the shared
`image_analysis/` library. Treat ROI and azimuthal features as available in **both** workflows,
not FPGA-only.

**`jfjoch_broker`** (`broker/`) is the central online service: HTTP/REST + OpenAPI control plane,
FPGA configuration, image building, ZeroMQ output. `JFJochStateMachine` drives acquisition state;
`JFJochServices` wires the pieces; `OpenAPIConvert`/`JFJochBrokerParser` translate between the
generated API model and internal types.

**Three analysis frontends share one analysis library** (`image_analysis/`, built as
`JFJochImageAnalysis`):
- `jfjoch_broker` — online, real-time (FPGA + GPU).
- `jfjoch_viewer` — interactive Qt desktop (`viewer/`), results not persisted.
- `jfjoch_process` (`tools/jfjoch_process.cpp`) — offline batch over a stored HDF5; writes
  `_process.h5` and `.mtz`/`.cif`/`.hkl`. `jfjoch_scale` re-scales/merges already-integrated data.

**`image_analysis/` pipeline** (subdirs): `spot_finding`, `indexing` (`ffbidx`/`fft` GPU,
`fftw` CPU), `lattice_search`, `geom_refinement`, `pixel_refinement`, `bragg_prediction`,
`bragg_integration`, `rotation_indexer`, `azint`, `roi`, `scale_merge`. Least-squares refinement
uses **Ceres** (fetched, built with miniglog, no MKL, CXX_THREADS). `ffbidx` needs a known cell
(`-C`) and suits sparse serial stills; `fft`/`fftw` index de novo and suit strong rotation data.

**FPGA** (`fpga/`): `hls/` is the Vitis HLS source (image-analysis kernels), `hls_simulation/`
runs that same HLS on CPU for hardware-free testing, `host_library/` is the host-side driver,
`pcie_driver/` is the kernel module. The HLS algorithms are documented in
`docs/FPGA_DATA_ANALYSIS.md`.

**Detector control** (`detector_control/`): wrappers for SLS (JUNGFRAU) and DECTRIS SIMPLON
(EIGER). **`jungfrau/`**: JUNGFRAU ADU→energy gain/pedestal calibration.

**Other libs:** `common/` (geometry, diffraction experiment, image buffer, CUDA wrappers — the
shared core, linked nearly everywhere), `compression/` (zstd + bitshuffle + sqrt lossy),
`frame_serialize/` (CBOR stream codec), `gemmi_gph/` (vendored GEMMI for MTZ/XDS_ASCII I/O),
`xds-plugin/` (XDS HDF5 read plugin).

## Portability (jfjoch_viewer)

Cross-platform support is a goal **only for `jfjoch_viewer` and its dependency tree** — keep that
code, and any shared library it transitively links (`common/`, `image_analysis/`, `reader/`,
`gemmi_gph/`, etc.), **MSVC-compatible** so the viewer can build on Windows. The rest of the
project (broker, receiver, FPGA host, detector control, …) is Linux-only and does not need to be
portable; don't constrain it for portability's sake.

- **Windows/MSVC** is the primary portability target. The end goal is a Windows viewer built
  with **MSVC *and* CUDA** (`JFJOCH_USE_CUDA=ON`): GPU processing is a wanted feature, not
  optional, so the intended Windows config is the full GPU path (`ffbidx`, GPU `fft`), not a
  CPU-only fallback. MSVC is required regardless, because CUDA on Windows requires it. Avoid
  GCC/Clang-only extensions, POSIX-only APIs, and other non-MSVC constructs in viewer-reachable
  code, and keep CUDA-reachable viewer code (`ffbidx`, GPU indexers) MSVC-buildable too.
- **macOS** is a nice-to-have for the viewer. It rules out CUDA, so anything the viewer depends on
  must also have a working CPU-only / non-CUDA path (the `JFJOCH_USE_CUDA=OFF`, `fftw`-indexer
  configuration). This non-CUDA path must keep working, but it is the macOS fallback — *not* the
  intended Windows configuration.

A self-contained Windows build still needs a few dependencies that the Linux build picks up from
the system and that are **not** auto-provided via `FetchContent` (besides Qt, supplied
externally): **ZLIB** (pulled by hdf5/libtiff/gemmi/libzmq — a bundled zlib-ng in `ZLIB_COMPAT`
mode is the intended fix, presented as the `ZLIB::ZLIB` target so the scattered
`find_package(ZLIB)` calls resolve), **libjpeg-turbo** (used by `preview/`; upstream discourages
`add_subdirectory`, so bring it in via `ExternalProject`), and **Eigen** (header-only; needed by
Ceres, by the analysis libs directly, and by `ffbidx` under CUDA — fetch it and point Ceres'
internal `find_package(Eigen3)` at it). Wire each guarded so the Linux build, which finds these on
the system, stays unchanged.

## OpenAPI is the single source of truth

`broker/jfjoch_api.yaml` defines the entire REST API **and the shared data schemas**. From it,
`update_version.sh` regenerates three clients — **do not hand-edit generated code**:
- C++ server model → `broker/gen/` (cpp-pistache-server generator; compiled as `JFJochAPI`).
- Python client → `python-client/` (and `gen_python_client.sh`, published as PyPI `jfjoch-client`).
- TypeScript frontend client → `frontend/src/client/` (hey-api `openapi-ts`, `npm run openapi`).

When you change `jfjoch_api.yaml`, regenerate the relevant client(s); for a version bump run
`update_version.sh` (also rewrites `VERSION`, `frontend/src/version.ts`, and the Redoc html).

## Frontend

React 19 + TypeScript + MUI + Vite (`frontend/`). Data layer is generated from the OpenAPI spec
(`@hey-api/openapi-ts` → fetch client + TanStack Query hooks + zod schemas). Scripts:
`npm start` (dev server), `npm run build` (tsc + vite), `npm run openapi` (regen client),
`npm run redocly4broker` (regen `broker/redoc-static.html`).