Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one. * HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected. * HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it. * HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset. * HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts. * HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout. * A grid scan and a goniometer axis can both be set; they are no longer alternatives. * `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000. * The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned. * The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer. * rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it. * rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`. * rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way. * rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound. * rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached. * rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use. * rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own. * rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement. * rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged. * rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged. * Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged. * A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses. * rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given. * CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer. * The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2. * Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact. **Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`): * `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file. * `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them. --------- Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch> Reviewed-on: #72 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
162 lines
10 KiB
Markdown
162 lines
10 KiB
Markdown
# jfjoch_viewer
|
|
|
|
`jfjoch_viewer` is the **interactive** desktop application of Jungfraujoch. It opens diffraction
|
|
datasets, displays each image together with the analysis overlay (spots, predictions, azimuthal
|
|
integration, per-image statistics), and can follow a live data collection by syncing with a
|
|
running [`jfjoch_broker`](JFJOCH_BROKER.md) over its HTTP interface.
|
|
|
|
It is a standalone Qt 6 application, distributed pre-built for **Linux and Windows** on the Gitea
|
|
release page and in the Jungfraujoch RPM/APT repositories — see [Release contents](RELEASE_CONTENTS.md)
|
|
for what each package contains and what it requires, and [Deployment](DEPLOYMENT.md) for how to
|
|
install it.
|
|
|
|
## Where it fits among the three analysis tools
|
|
|
|
| Tool | Mode | Driven by | Output |
|
|
| --- | --- | --- | --- |
|
|
| [`jfjoch_broker`](JFJOCH_BROKER.md) | Online, real-time streaming analysis on FPGA + GPU | HTTP/REST + ZeroMQ | Live results and statistics, images streamed to [`jfjoch_writer`](JFJOCH_WRITER.md) |
|
|
| **`jfjoch_viewer`** | **Interactive, on-screen exploration** | **Qt desktop application** | **On screen; a processing job can write the same files as `rugnux`** |
|
|
| [`rugnux`](RUGNUX.md) | Offline batch processing of a stored dataset | Command-line interface | `_process.h5`, and `.mtz`/`.cif`/`.hkl` when merging |
|
|
|
|
## Functionality
|
|
|
|
- Opens HDF5 files written by [`jfjoch_writer`](JFJOCH_WRITER.md) (`*_master.h5`) and the
|
|
`*_process.h5` files produced by [`rugnux`](RUGNUX.md). It also opens NXmx files
|
|
written by DECTRIS detectors, though that path has had only limited testing.
|
|
- Runs an **embedded data-processing pipeline** — the same analysis code as the rest of
|
|
Jungfraujoch — performing spot finding, indexing and integration on the displayed image, with the
|
|
result drawn over it. This interactive analysis is not written anywhere.
|
|
- Runs **full processing jobs** on the open dataset with *Analyze dataset*, on the same
|
|
[`rugnux`](RUGNUX.md) engine and off the GUI thread. The settings panel's **MX / AzInt / Calib**
|
|
toggle decides what a run does — full analysis, azimuthal integration only, or a detector
|
|
calibration — over a chosen image range, optionally writing `_process.h5` and the merged
|
|
`.mtz`/`.cif`. A finished run becomes a selectable view of the dataset, so several processing runs
|
|
can be compared against each other, and its merging statistics (or, for a calibration, its fitted
|
|
geometry) open in their own window; the *Processing* panel lists the runs and reopens those
|
|
results. The equivalent `rugnux` command line can also be copied out to run the same job on a
|
|
cluster instead.
|
|
- **Detector calibration** against a powder standard, on the *Calib* page: pick the calibrant
|
|
(`LaB6`, `AgBh`, `CeO2`, `Si`, `ice`, or the open dataset's own unit cell) and fit either the image
|
|
on screen (*Guess* / *Refine detector calibration*) or the whole dataset (*Analyze dataset*, which
|
|
writes a pyFAI `<output prefix>.poni`). The whole-dataset fit measures the rings either from the
|
|
azimuthally-binned profile summed over the run (*Rings*, the default) or from the pooled spot lists
|
|
(*Spots*), and reports PONI x/y, the two tilts and the distance against the header values. Judge it
|
|
by the **radial rms**, not the beam-centre sigma: the sigma shrinks with the number of ring points,
|
|
so a fit that sits a couple of pixels off every ring can still report a small one. *Rings* needs
|
|
the run to be integrated in azimuthal sectors — with the AzInt page's *Azimuthal bins* below 4 the
|
|
calibration run raises it to 32, as `rugnux --mode calibration` does, and says so.
|
|
- **Settings** panel for the geometry, unit cell, spot finding, indexing, azimuthal integration,
|
|
Bragg integration, scaling, powder calibration and a reference dataset — the same settings the
|
|
CLI takes.
|
|
- Auxiliary windows: image list, dataset metadata, spot list, reflection list, reciprocal-space
|
|
viewer, 2D azimuthal-integration image, calibration-image viewer and a magnifier; plus the
|
|
*Inspector* (per-image statistics, image features, resolution rings, ROI statistics), the
|
|
*Image strip* thumbnail feed and dataset-info charts.
|
|
- User-mask editing: build a user mask interactively, load one from TIFF (replacing or adding to the
|
|
current one), save it as TIFF, clear it, or upload it to a connected server.
|
|
- Layout presets (*View ▸ Image layout / Processing layout / Reset layout*) rearrange the docks for
|
|
looking at images or at processing results.
|
|
|
|
## Hardware
|
|
|
|
As with the rest of Jungfraujoch, **serious performance requires an NVIDIA GPU**. On systems with a
|
|
GPU, use the CUDA build (a separate package variant everywhere: RPM/APT repository, `.tgz` and
|
|
Windows installer) for the embedded indexing and integration; the non-CUDA build runs the same
|
|
pipeline on the CPU at much lower throughput. The CUDA build also runs on a machine without a GPU —
|
|
see [Release contents ▸ CUDA and non-CUDA builds](RELEASE_CONTENTS.md#cuda-and-non-cuda-builds).
|
|
|
|
The CUDA build needs an NVIDIA **driver** on the host but no CUDA toolkit — 525.60.13 or newer for
|
|
the CUDA 12 artefacts (RHEL 8 packages, portable Linux `.tgz`), 580.65.06 or newer on Linux and an
|
|
R580 driver on Windows for the CUDA 13 ones (RHEL 9, Ubuntu, Windows installer). The Windows
|
|
installer and the `.tgz` are CUDA 13 and CUDA 12 respectively, which also decides the oldest GPU
|
|
they run on — a V100 needs the CUDA 12 `.tgz`. See
|
|
[Release contents ▸ GPU generations and the NVIDIA driver](RELEASE_CONTENTS.md#gpu-generations-and-the-nvidia-driver).
|
|
|
|
## Opening data
|
|
|
|
- **File ▸ Open** (`Ctrl+O`) — open a local HDF5 file.
|
|
- **File ▸ Open HTTP** (`Ctrl+H`) — connect to a `jfjoch_broker` HTTP endpoint to follow a live
|
|
collection. The dialog defaults to host `localhost` and port `8080`; these defaults can be
|
|
overridden with the environment variables `JUNGFRAUJOCH_HTTP_HOST` and `JUNGFRAUJOCH_HTTP_PORT`.
|
|
- **Command line** — `jfjoch_viewer <file.h5>` opens a file (or an `http://host:port` URL) on
|
|
start-up. `--dbus <true|false>` (`-d`) enables or disables the D-Bus interface (default: enabled);
|
|
`--help` and `--version` behave as usual.
|
|
|
|
## D-Bus interface
|
|
|
|
When enabled, the viewer registers the D-Bus interface `ch.psi.jfjoch_viewer`, so other processes
|
|
can drive it:
|
|
|
|
- `LoadFile(filename, image_number=0, summation=1)` — open a file (or an `http://host:port` URL)
|
|
and display the given image.
|
|
- `LoadImage(image_number, summation=1)` — navigate to an image in the already-open dataset.
|
|
|
|
`summation` sums that many consecutive images before display.
|
|
|
|
## Building from source on Windows
|
|
|
|
`jfjoch_viewer` is the one Jungfraujoch component that is cross-platform: it builds on Windows 11
|
|
with MSVC and the full CUDA GPU path. (The rest of Jungfraujoch — broker, receiver, FPGA host — is
|
|
Linux-only.) A pre-built installer is published with every release, so building from source is only
|
|
needed to develop or to change the build options. On Windows the build is automatically restricted
|
|
to the viewer and the libraries it needs (`JFJOCH_VIEWER_ONLY` is forced on), and the remaining
|
|
dependencies are fetched and built automatically (the first configure needs network access).
|
|
|
|
Verified toolchain — the same one the released installer is built with:
|
|
|
|
- Windows 11
|
|
- Visual Studio 2026 with the C++ (MSVC) toolset — required; CUDA on Windows builds through MSVC
|
|
- CUDA Toolkit 13.3 (12.8 or newer is required) — for the GPU indexing/integration path
|
|
- Qt 6.11 for MSVC (`msvc2022_64`), including the **Qt Charts** module — e.g. `C:\Qt\6.11.1\msvc2022_64`
|
|
- CMake plus Ninja. The CMake that ships with Visual Studio is the simplest choice and works out of
|
|
the box — it comes with the C++ workload, so there is nothing extra to install. Any recent
|
|
standalone CMake (from cmake.org, or the one bundled with Qt in `C:\Qt\Tools\CMake_64`) works too.
|
|
- zlib and Eigen — the two libraries not auto-fetched on Windows. Build/install both into one prefix
|
|
(here `C:\deps`) and point CMake at it:
|
|
```
|
|
:: static zlib
|
|
git clone --branch v1.3.1 https://github.com/madler/zlib
|
|
cmake -G Ninja -S zlib -B zlib-build -DCMAKE_INSTALL_PREFIX=C:/deps
|
|
cmake --build zlib-build --target install
|
|
:: Eigen 3.4 (header-only) -- install just the headers with `cmake --install`; the BLAS/LAPACK/test
|
|
:: targets are disabled since they are not needed (and fail to build under MSVC). Use the 3.4 series:
|
|
:: the project requests find_package(Eigen3 3.4), which Eigen's same-major rule rejects for 5.x.
|
|
git clone --branch 3.4.0 https://gitlab.com/libeigen/eigen.git
|
|
cmake -G Ninja -S eigen -B eigen-build -DCMAKE_INSTALL_PREFIX=C:/deps ^
|
|
-DEIGEN_BUILD_BLAS=OFF -DEIGEN_BUILD_LAPACK=OFF -DEIGEN_BUILD_DOC=OFF -DBUILD_TESTING=OFF
|
|
cmake --install eigen-build
|
|
```
|
|
- Optional: [NSIS](https://nsis.sourceforge.io) to build the `.exe` installer.
|
|
|
|
Configure and build from an **x64 Native Tools Command Prompt for VS 2026** (so `cl`, `nvcc` and
|
|
`ninja` are on `PATH`):
|
|
|
|
```
|
|
cmake -G Ninja -B build-win -DCMAKE_BUILD_TYPE=Release ^
|
|
-DCMAKE_PREFIX_PATH="C:/deps;C:/Qt/6.11.1/msvc2022_64"
|
|
cmake --build build-win --target jfjoch_viewer
|
|
```
|
|
|
|
Notes:
|
|
|
|
- `CMAKE_PREFIX_PATH` (the `C:/deps` prefix plus Qt) is the only required flag — CMake finds zlib and
|
|
Eigen from the prefix, so no separate `-DZLIB_ROOT` is needed.
|
|
- The CUDA toolchain is located automatically from the `CUDA_PATH` environment variable that the
|
|
CUDA installer sets (or from `nvcc` on `PATH`). Pass `-DCMAKE_CUDA_COMPILER=".../bin/nvcc.exe"`
|
|
only if `nvcc` is installed in a nonstandard location and is not found.
|
|
- For a machine without an NVIDIA GPU, add `-DJFJOCH_USE_CUDA=OFF`: the viewer then runs the same
|
|
pipeline on the CPU (FFTW indexer) at lower throughput.
|
|
|
|
To produce a self-contained installer (bundles the Qt runtime via `windeployqt`, the analysis CLIs,
|
|
and — on the CUDA build — the cuFFT runtime DLL, so the target host needs neither Qt nor a CUDA
|
|
toolkit), with NSIS installed:
|
|
|
|
```
|
|
cd build-win
|
|
cpack
|
|
```
|
|
|
|
The NSIS generator is selected automatically on Windows (no `-G` needed). What comes out, and how
|
|
the CUDA and CPU variants are named and told apart, is described in
|
|
[Release contents ▸ Windows installer](RELEASE_CONTENTS.md#windows-installer).
|