Files
Jungfraujoch/docs/JFJOCH_VIEWER.md
T
leonarski_fandjungfrau 4dc2534dbf
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
v1.0.0.rc-162 (#72)
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-25 08:21:39 +02:00

162 lines
10 KiB
Markdown

# jfjoch_viewer
`jfjoch_viewer` is the **interactive** desktop application of Jungfraujoch. It opens diffraction
datasets, displays each image together with the analysis overlay (spots, predictions, azimuthal
integration, per-image statistics), and can follow a live data collection by syncing with a
running [`jfjoch_broker`](JFJOCH_BROKER.md) over its HTTP interface.
It is a standalone Qt 6 application, distributed pre-built for **Linux and Windows** on the Gitea
release page and in the Jungfraujoch RPM/APT repositories — see [Release contents](RELEASE_CONTENTS.md)
for what each package contains and what it requires, and [Deployment](DEPLOYMENT.md) for how to
install it.
## Where it fits among the three analysis tools
| Tool | Mode | Driven by | Output |
| --- | --- | --- | --- |
| [`jfjoch_broker`](JFJOCH_BROKER.md) | Online, real-time streaming analysis on FPGA + GPU | HTTP/REST + ZeroMQ | Live results and statistics, images streamed to [`jfjoch_writer`](JFJOCH_WRITER.md) |
| **`jfjoch_viewer`** | **Interactive, on-screen exploration** | **Qt desktop application** | **On screen; a processing job can write the same files as `rugnux`** |
| [`rugnux`](RUGNUX.md) | Offline batch processing of a stored dataset | Command-line interface | `_process.h5`, and `.mtz`/`.cif`/`.hkl` when merging |
## Functionality
- Opens HDF5 files written by [`jfjoch_writer`](JFJOCH_WRITER.md) (`*_master.h5`) and the
`*_process.h5` files produced by [`rugnux`](RUGNUX.md). It also opens NXmx files
written by DECTRIS detectors, though that path has had only limited testing.
- Runs an **embedded data-processing pipeline** — the same analysis code as the rest of
Jungfraujoch — performing spot finding, indexing and integration on the displayed image, with the
result drawn over it. This interactive analysis is not written anywhere.
- Runs **full processing jobs** on the open dataset with *Analyze dataset*, on the same
[`rugnux`](RUGNUX.md) engine and off the GUI thread. The settings panel's **MX / AzInt / Calib**
toggle decides what a run does — full analysis, azimuthal integration only, or a detector
calibration — over a chosen image range, optionally writing `_process.h5` and the merged
`.mtz`/`.cif`. A finished run becomes a selectable view of the dataset, so several processing runs
can be compared against each other, and its merging statistics (or, for a calibration, its fitted
geometry) open in their own window; the *Processing* panel lists the runs and reopens those
results. The equivalent `rugnux` command line can also be copied out to run the same job on a
cluster instead.
- **Detector calibration** against a powder standard, on the *Calib* page: pick the calibrant
(`LaB6`, `AgBh`, `CeO2`, `Si`, `ice`, or the open dataset's own unit cell) and fit either the image
on screen (*Guess* / *Refine detector calibration*) or the whole dataset (*Analyze dataset*, which
writes a pyFAI `<output prefix>.poni`). The whole-dataset fit measures the rings either from the
azimuthally-binned profile summed over the run (*Rings*, the default) or from the pooled spot lists
(*Spots*), and reports PONI x/y, the two tilts and the distance against the header values. Judge it
by the **radial rms**, not the beam-centre sigma: the sigma shrinks with the number of ring points,
so a fit that sits a couple of pixels off every ring can still report a small one. *Rings* needs
the run to be integrated in azimuthal sectors — with the AzInt page's *Azimuthal bins* below 4 the
calibration run raises it to 32, as `rugnux --mode calibration` does, and says so.
- **Settings** panel for the geometry, unit cell, spot finding, indexing, azimuthal integration,
Bragg integration, scaling, powder calibration and a reference dataset — the same settings the
CLI takes.
- Auxiliary windows: image list, dataset metadata, spot list, reflection list, reciprocal-space
viewer, 2D azimuthal-integration image, calibration-image viewer and a magnifier; plus the
*Inspector* (per-image statistics, image features, resolution rings, ROI statistics), the
*Image strip* thumbnail feed and dataset-info charts.
- User-mask editing: build a user mask interactively, load one from TIFF (replacing or adding to the
current one), save it as TIFF, clear it, or upload it to a connected server.
- Layout presets (*View ▸ Image layout / Processing layout / Reset layout*) rearrange the docks for
looking at images or at processing results.
## Hardware
As with the rest of Jungfraujoch, **serious performance requires an NVIDIA GPU**. On systems with a
GPU, use the CUDA build (a separate package variant everywhere: RPM/APT repository, `.tgz` and
Windows installer) for the embedded indexing and integration; the non-CUDA build runs the same
pipeline on the CPU at much lower throughput. The CUDA build also runs on a machine without a GPU —
see [Release contents ▸ CUDA and non-CUDA builds](RELEASE_CONTENTS.md#cuda-and-non-cuda-builds).
The CUDA build needs an NVIDIA **driver** on the host but no CUDA toolkit — 525.60.13 or newer for
the CUDA 12 artefacts (RHEL 8 packages, portable Linux `.tgz`), 580.65.06 or newer on Linux and an
R580 driver on Windows for the CUDA 13 ones (RHEL 9, Ubuntu, Windows installer). The Windows
installer and the `.tgz` are CUDA 13 and CUDA 12 respectively, which also decides the oldest GPU
they run on — a V100 needs the CUDA 12 `.tgz`. See
[Release contents ▸ GPU generations and the NVIDIA driver](RELEASE_CONTENTS.md#gpu-generations-and-the-nvidia-driver).
## Opening data
- **File ▸ Open** (`Ctrl+O`) — open a local HDF5 file.
- **File ▸ Open HTTP** (`Ctrl+H`) — connect to a `jfjoch_broker` HTTP endpoint to follow a live
collection. The dialog defaults to host `localhost` and port `8080`; these defaults can be
overridden with the environment variables `JUNGFRAUJOCH_HTTP_HOST` and `JUNGFRAUJOCH_HTTP_PORT`.
- **Command line** — `jfjoch_viewer <file.h5>` opens a file (or an `http://host:port` URL) on
start-up. `--dbus <true|false>` (`-d`) enables or disables the D-Bus interface (default: enabled);
`--help` and `--version` behave as usual.
## D-Bus interface
When enabled, the viewer registers the D-Bus interface `ch.psi.jfjoch_viewer`, so other processes
can drive it:
- `LoadFile(filename, image_number=0, summation=1)` — open a file (or an `http://host:port` URL)
and display the given image.
- `LoadImage(image_number, summation=1)` — navigate to an image in the already-open dataset.
`summation` sums that many consecutive images before display.
## Building from source on Windows
`jfjoch_viewer` is the one Jungfraujoch component that is cross-platform: it builds on Windows 11
with MSVC and the full CUDA GPU path. (The rest of Jungfraujoch — broker, receiver, FPGA host — is
Linux-only.) A pre-built installer is published with every release, so building from source is only
needed to develop or to change the build options. On Windows the build is automatically restricted
to the viewer and the libraries it needs (`JFJOCH_VIEWER_ONLY` is forced on), and the remaining
dependencies are fetched and built automatically (the first configure needs network access).
Verified toolchain — the same one the released installer is built with:
- Windows 11
- Visual Studio 2026 with the C++ (MSVC) toolset — required; CUDA on Windows builds through MSVC
- CUDA Toolkit 13.3 (12.8 or newer is required) — for the GPU indexing/integration path
- Qt 6.11 for MSVC (`msvc2022_64`), including the **Qt Charts** module — e.g. `C:\Qt\6.11.1\msvc2022_64`
- CMake plus Ninja. The CMake that ships with Visual Studio is the simplest choice and works out of
the box — it comes with the C++ workload, so there is nothing extra to install. Any recent
standalone CMake (from cmake.org, or the one bundled with Qt in `C:\Qt\Tools\CMake_64`) works too.
- zlib and Eigen — the two libraries not auto-fetched on Windows. Build/install both into one prefix
(here `C:\deps`) and point CMake at it:
```
:: static zlib
git clone --branch v1.3.1 https://github.com/madler/zlib
cmake -G Ninja -S zlib -B zlib-build -DCMAKE_INSTALL_PREFIX=C:/deps
cmake --build zlib-build --target install
:: Eigen 3.4 (header-only) -- install just the headers with `cmake --install`; the BLAS/LAPACK/test
:: targets are disabled since they are not needed (and fail to build under MSVC). Use the 3.4 series:
:: the project requests find_package(Eigen3 3.4), which Eigen's same-major rule rejects for 5.x.
git clone --branch 3.4.0 https://gitlab.com/libeigen/eigen.git
cmake -G Ninja -S eigen -B eigen-build -DCMAKE_INSTALL_PREFIX=C:/deps ^
-DEIGEN_BUILD_BLAS=OFF -DEIGEN_BUILD_LAPACK=OFF -DEIGEN_BUILD_DOC=OFF -DBUILD_TESTING=OFF
cmake --install eigen-build
```
- Optional: [NSIS](https://nsis.sourceforge.io) to build the `.exe` installer.
Configure and build from an **x64 Native Tools Command Prompt for VS 2026** (so `cl`, `nvcc` and
`ninja` are on `PATH`):
```
cmake -G Ninja -B build-win -DCMAKE_BUILD_TYPE=Release ^
-DCMAKE_PREFIX_PATH="C:/deps;C:/Qt/6.11.1/msvc2022_64"
cmake --build build-win --target jfjoch_viewer
```
Notes:
- `CMAKE_PREFIX_PATH` (the `C:/deps` prefix plus Qt) is the only required flag — CMake finds zlib and
Eigen from the prefix, so no separate `-DZLIB_ROOT` is needed.
- The CUDA toolchain is located automatically from the `CUDA_PATH` environment variable that the
CUDA installer sets (or from `nvcc` on `PATH`). Pass `-DCMAKE_CUDA_COMPILER=".../bin/nvcc.exe"`
only if `nvcc` is installed in a nonstandard location and is not found.
- For a machine without an NVIDIA GPU, add `-DJFJOCH_USE_CUDA=OFF`: the viewer then runs the same
pipeline on the CPU (FFTW indexer) at lower throughput.
To produce a self-contained installer (bundles the Qt runtime via `windeployqt`, the analysis CLIs,
and — on the CUDA build — the cuFFT runtime DLL, so the target host needs neither Qt nor a CUDA
toolkit), with NSIS installed:
```
cd build-win
cpack
```
The NSIS generator is selected automatically on Windows (no `-G` needed). What comes out, and how
the CUDA and CPU variants are named and told apart, is described in
[Release contents ▸ Windows installer](RELEASE_CONTENTS.md#windows-installer).