leonarski_f
749db470ca
Build Packages / build:rpm (rocky9) (push) Successful in 19m56s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m57s
Build Packages / build:windows:cuda (push) Successful in 19m18s
Build Packages / build:viewer-tgz:cpu (push) Successful in 14m48s
Build Packages / build:viewer-tgz:cuda (push) Successful in 16m18s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 14m19s
Build Packages / build:rugnux:windows (push) Successful in 10m34s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 8m49s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m55s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 17m4s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 20m48s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 19m15s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 24m26s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 20m32s
Build Packages / build:rpm (rocky8) (push) Successful in 23m39s
Build Packages / Generate python client (push) Successful in 46s
Build Packages / Build documentation (push) Successful in 1m45s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 11m3s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m30s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m10s
Build Packages / XDS test (neggia plugin) (push) Successful in 10m17s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 23m12s
Build Packages / DIALS test (push) Successful in 20m12s
* rugnux now tells you whether a crystal diffracts anisotropically and how far it reaches in each direction, without a second program: a new `9. DIFFRACTION ANISOTROPY` section in `<prefix>_report.txt` and matching `_reflns.pdbx_aniso_B_tensor_*` / `_reflns.jfjoch_aniso_*` items in the merged mmCIF report the anisotropic deltaB, the diffraction limit along each principal direction, and a `NOT DETECTED` / `DETECTED` / `CANNOT DETERMINE` verdict measured against the data set's own systematic error. It is a description only - no intensity is corrected, no reflection is removed, and the merged data do not depend on direction.
* rugnux can hand its integrated observations to another scaling program: `--export-unmerged` writes `<prefix>_unmerged.mtz`, an unmerged MTZ readable by aimless, pointless, careless and `iotbx.merging_statistics`, in `--mode mx` and `--mode scale` alike. Each rotation reflection's partials are summed into one full; `--export-unmerged-partials` writes one row per image instead. Intensities carry the Lorentz-polarization factor and nothing else, since those programs scale the data themselves. Lattice-centring absences are not written; screw and glide absences are.
* rugnux integrates crystals with broad spots better - where it changes anything, per-shell mean I/sigma improves by up to 31% and R_meas by up to 24% - because on rotation data the integration signal radius is now taken from the crystal's own measured spot width instead of a fixed 4 px. `--adaptive-integration-radius=off` restores the fixed radius and an explicit `--integration-radius` still overrides both. The widened radius applies to the final integration pass only, and a pattern too dense for it is re-integrated at 4 px with a note in the log.
* rugnux discards fewer stills reflections for want of a background ring, improving per-shell R_meas over most of the signal-bearing range: the stills background ring now runs to 14 px instead of 12. The gain reverses in shells below a mean I/sigma of about 4.
* rugnux determines the space group with thresholds that mean the same thing on a weak crystal as on a strong one: symmetry operators are scored on resolution-normalised intensities (E squared) instead of raw merged intensities, and a reflection counts as genuinely present on its counting significance instead of on the merged I/sigma, which saturates at the merge's own ISa. The search resolution cut is no longer able to move the answer, and the twin-law H bound moves from 1.70 to 1.85, which stops one class of correct high-symmetry assignment being refused as twinning.
* rugnux says what the space-group search tested and what it could not: the twin-law disagreement H is printed for every operator together with the adopted point group's H ratio and its bound; alternatives that are not on the reported lattice are named with how their cell differs; and a lattice centring the data could not test - the crystal having been integrated on the primitive sub-cell, so the reflections it extinguishes were never measured - is marked `UNTESTED` and warned about where it is adopted, as coming from the lattice metric rather than from the intensities.
* rugnux `--mode scale` re-merges a `_process.h5` in the right symmetry without being told it: the file now records the space group on every run - a two-pass rotation run wrote none before, so re-merging defaulted to P1 - together with the change of basis under `/entry/MX/reindexMatrix` where the lattice was re-seated, and `--mode scale` also reports the Wilson B-factor estimate instead of `WILSON_B= nan`. A file written before this stops with a message naming the two cells and the override to use, instead of failing inside the merge. A third-party reader of a `_process.h5` must apply `reindexMatrix` where it is present.
* rugnux installs on its own, as a package called `rugnux` - `dnf install rugnux` or `apt install rugnux` - instead of arriving inside `jfjoch-viewer`. It pulls in none of the acquisition stack, so a machine that only processes data no longer has to carry the broker, the detector libraries or Qt to get it. Installing it over a `jfjoch-viewer` from rc.163 or earlier, which still owns `/usr/bin/rugnux`, upgrades cleanly rather than failing on the duplicate file.
* rugnux is also a standalone download, built for arm64 as well as x86_64: `rugnux-<version>-linux-{x86_64|aarch64}-cuda<major>.tgz` and `rugnux-<version>-win64-cuda<major>.zip` on the release page, for machines that are not managed by a package manager. The aarch64 build targets GH200 and DGX Spark, and is untested on hardware.
* Every portable Linux binary is now a single self-contained file: cuFFT is linked statically instead of being shipped beside the executable and found through an rpath, so `rugnux` and `jfjoch_viewer` need nothing but an NVIDIA driver, and only to use the GPU. The `.rpm`/`.deb` continue to take cuFFT from the distribution. The developer utilities `jfjoch_extract_hkl` and `jfjoch_recompress` are no longer packaged anywhere.
* Jungfraujoch needs six fewer shared libraries on the machine - libopenblas and libmetis, and libgfortran, libquadmath, libgomp and libz behind them - because the Ceres LAPACK, METIS and SuiteSparse back-ends are no longer built. Nothing in the code ever selected them, and results are unchanged.
* The PCIe driver DKMS package builds for the kernel it is being installed for instead of the running one, so a module built while a kernel update is being applied loads after the reboot.
* The PCIe driver builds on RHEL 9.5 and later, and on their CentOS Stream, Rocky and AlmaLinux equivalents, where the `vm_flags` kernel interface was backported into the 5.14 kernel.
* A data collection started with `async_start` that fails to start - a writer refusing to overwrite an existing file, for instance - is reported as an error by `/wait_until_running` and `/wait_till_done` instead of as a timeout and a successful collection respectively. The error message is the one the writer gave.
* A calibration that is cancelled or that fails to collect its pedestals is no longer reported as a successful one. The broker goes to `Inactive` with an error message and has to be initialized again, instead of sitting in `Idle` looking ready to measure while holding partial pedestals - data collected in that state was silently mis-converted.
* A failed `/initialize` is reported to `/wait_until_running` and `/wait_till_done` as soon as it happens, instead of when their timeout expires.
* `space_group_number` accepts space groups up to 230 in the API schema, so cubic space groups can be recorded. The broker always accepted them; the generated clients rejected them before the request was sent.
* The results report's `REPORT_VERSION` is 3, two sections having been added. Existing key names and table columns are unchanged.
* The merged statistics table has **9** resolution shells instead of 10, which is what XDS reports. The bins were already XDS's - equal steps in 1/d^2 between the lowest- and the highest-resolution reflection the merge kept - so at the same resolution limits the two tables now have the same shell boundaries and can be read row for row. `--resolution-shells` sets a different count.
* `rugnux --model` now settles the frame the merged reflections are written in, not only the frame the R-factors and the maps are computed in: the `.mtz`/`.cif`/`.hkl` come out in the model's indexing, and where the data were merged in the model's enantiomorph they take the model's hand and space group - which on anomalous data puts I(+) and I(-) the right way round. The indexing choice is logged with the winning R-free and the runner-up, so a decision made within noise is visible.
* `rugnux --model` can resolve the indexing ambiguity of a **serial stills** run, which a model could not do before: structure factors computed from the model become the per-image reference, the same role a reference MTZ plays. It needs the cell and space group up front (`-C` / `-S`). Without one or the other, a merohedral serial run still merges both hands together and says so.
* The rugnux documentation opens with a quick start - the default run, and runs with a reference MTZ, with a model, or with the space group and cell pinned - and explains the indexing ambiguity: what it costs on rotation and on serial data, and which of `-z` / `--model` resolves it in each case. The long reference pages now carry a table of contents.
Reviewed-on: #74
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
161 lines
11 KiB
Markdown
161 lines
11 KiB
Markdown
# Release contents
|
|
|
|
This page describes **what a Jungfraujoch release ships and what each artefact needs on the target
|
|
machine** — which CPU instruction set the binaries were compiled for, which CUDA toolkit they were
|
|
built against, and which runtime libraries are bundled rather than expected from the host.
|
|
|
|
The artefacts in the table below are built and published by the continuous-integration pipeline
|
|
(`.gitea/workflows/build_and_test.yml`) when a tag is pushed. For *how* to install and configure the
|
|
result see [Deployment](DEPLOYMENT.md); for the package-repository URLs see
|
|
[Linux package repositories](REPOSITORIES.md).
|
|
|
|
## Artefacts
|
|
|
|
| Artefact | Distributed via | Contains |
|
|
| --- | --- | --- |
|
|
| `.rpm` / `.deb` packages | [package repositories](REPOSITORIES.md) | The full server stack: `jfjoch` (broker, frontend, FPGA and detector tools), `jfjoch-writer`, `jfjoch-viewer` (incl. the XDS plugin), `jfjoch-driver-dkms`, and `rugnux` (offline analysis, independent of the rest) |
|
|
| `jfjoch_viewer-<version>-linux-cuda<major>.tgz`, `...-linux-cpu.tgz` | Gitea release page | Portable Linux viewer: `jfjoch_viewer`, its desktop entry, icon and D-Bus service, and the license notices |
|
|
| `jfjoch-viewer-<version>-win64-cuda<major>.exe`, `...-win64-cpu.exe` | Gitea release page | Windows installer for `jfjoch_viewer`, plus the Qt runtime |
|
|
| `rugnux-<version>-linux-x86_64-cuda<major>.tgz` | Gitea release page | Portable Linux [`rugnux`](RUGNUX.md), the offline analysis CLI, and the license notices. One executable |
|
|
| `rugnux-<version>-linux-aarch64-cuda<major>.tgz` | Gitea release page | The same, cross-built for 64-bit Arm — NVIDIA GH200 and DGX Spark |
|
|
| `rugnux-<version>-win64-cuda<major>.zip` | Gitea release page | The same for Windows, plus the cuFFT DLL |
|
|
| `jfjoch-writer` `.rpm` / `.deb` | Gitea release page | The writer alone, for a file-writing machine without the rest of the stack |
|
|
| `libjfjoch_xds_plugin.so.<version>` | Gitea release page | XDS HDF5 read plugin (built on RHEL 8); see [Integration with MX software](SOFTWARE_INTEGRATION.md) |
|
|
| `jfjoch-client` | [PyPI](https://pypi.org/project/jfjoch-client/) and the Gitea PyPI index | Generated Python OpenAPI client |
|
|
| Documentation | [Read the Docs](https://jungfraujoch.readthedocs.io) and the `gitea-pages` branch | This documentation set |
|
|
|
|
The FPGA firmware (`.mcs`) images are attached to the release as well. The firmware is stable and is
|
|
carried from version to version, and is rebuilt with Vivado (see [FPGA smartNIC](FPGA.md)) when it
|
|
needs to change — so a card keeps its image across a software upgrade unless the release notes say
|
|
otherwise.
|
|
|
|
## CPU instruction set
|
|
|
|
The architecture flags live in the CI configuration rather than in `CMakeLists.txt`, so a site
|
|
building from source picks its own (`x86-64-v4` on an AVX-512 cluster, `-march=native`, or the plain
|
|
baseline the compiler defaults to). The released binaries are compiled to a fixed floor:
|
|
|
|
| Release | Flags | Minimum CPU |
|
|
| --- | --- | --- |
|
|
| Linux (all packages, and the portable `.tgz`) | `-march=x86-64-v3 -flto=auto` | AVX2 + FMA + BMI2 — Intel Haswell (2013) / AMD Zen (2017) and newer |
|
|
| Windows installer | `/arch:AVX` | AVX — Intel Sandy Bridge (2011) / AMD Bulldozer and newer |
|
|
|
|
The Windows floor is lower because MSVC has no spelling for the `x86-64-v2` level; `/arch:AVX` is
|
|
the nearest one and implies SSE4.1/4.2, which is what actually matters — without it Eigen has no
|
|
vectorised `round` and falls back to a libm call per element. Link-time optimisation is applied on
|
|
Linux only.
|
|
|
|
A binary will fault with an illegal instruction on a CPU below its floor. If you must run on older
|
|
hardware, build from source without the flags.
|
|
|
|
## Operating-system floor
|
|
|
|
The `.rpm` / `.deb` packages are built per distribution (RHEL/Rocky 8 and 9, Ubuntu 22.04 and 24.04)
|
|
and are tied to it. The portable viewer `.tgz` and the **x86_64** `rugnux` `.tgz` are built on
|
|
RHEL 8, the oldest supported distribution, so their glibc floor is low enough to run on any newer
|
|
Linux — that is what they are for, and why they replace the per-distro packaging of those programs
|
|
on the release page. The **aarch64** `rugnux` `.tgz` is the exception: it is cross-built against
|
|
Ubuntu 24.04, so it needs glibc 2.39 or newer (which DGX OS 7 and any current Arm server distribution
|
|
have). The Windows installer is built and verified on Windows 11.
|
|
|
|
**The portable archives have no top-level directory.** They unpack straight into `bin/` and
|
|
`share/`, so always extract them into a directory of their own (`tar xzf … -C /opt/rugnux-<version>`)
|
|
rather than into a working directory.
|
|
|
|
## CUDA and non-CUDA builds
|
|
|
|
Every binary artefact is released in **two variants**, `cuda<major>` and `cpu`. The CUDA variant adds
|
|
the GPU fast-feedback indexer (`ffbidx`), the GPU FFT indexer and GPU image processing; the CPU-only
|
|
variant runs the same pipeline on the CPU with the FFTW indexer, at much lower throughput.
|
|
|
|
The CUDA toolkit used is the one on the corresponding build machine: **CUDA 12** for the RHEL 8
|
|
packages, **CUDA 13** for RHEL 9, Ubuntu and Windows. The major version is part of the artefact and
|
|
repository name, so a download is self-identifying. Building from source needs CUDA 12.8 or newer.
|
|
|
|
**A CUDA build does not require a CUDA machine.** Jungfraujoch asks how many CUDA devices are
|
|
present at start-up and treats "none" (including "no driver installed") as zero GPUs, falling back
|
|
to the CPU path. So a CUDA build starts and runs correctly on a machine with no NVIDIA GPU at all.
|
|
What each artefact has to find at run time differs:
|
|
|
|
- **Portable Linux `.tgz`** — nothing. The CUDA runtime, the fast-feedback indexer **and cuFFT** are
|
|
all linked statically, so each archive is a single executable that depends on nothing but the C
|
|
and C++ runtimes. On a GPU machine the NVIDIA driver is the only NVIDIA component needed.
|
|
- **Windows installer and `.zip`** — the CUDA toolkit ships no static cuFFT for Windows, so the
|
|
cuFFT DLL is **part of the distribution**, next to the executable. No CUDA toolkit is needed.
|
|
- **`.rpm` / `.deb`** — these deliberately keep cuFFT dynamic, so that one dependency is managed
|
|
centrally with the rest of CUDA. Install the cuFFT package alongside, or use the `nocuda`
|
|
repositories on a machine where CUDA is not wanted.
|
|
|
|
Static CUDA linkage makes the Linux CUDA artefacts substantially bigger than the CPU ones, and the
|
|
Windows cuFFT DLL is ~256 MB — which is the other reason for shipping both variants.
|
|
|
|
On a machine with an NVIDIA GPU, take the CUDA variant: only that one uses the GPU.
|
|
|
|
## GPU generations and the NVIDIA driver
|
|
|
|
A CUDA variant carries compiled device code for a fixed set of GPU generations, and which
|
|
generations those are follows from the CUDA toolkit it was built with. The CUDA runtime is linked
|
|
statically, so the only NVIDIA component the target machine has to supply is the **driver** — there
|
|
is no CUDA-toolkit version requirement on the host.
|
|
|
|
| Artefact | CUDA toolkit | GPU generations | Minimum driver |
|
|
| --- | --- | --- | --- |
|
|
| RHEL 8 packages, portable viewer `.tgz`, x86_64 `rugnux` `.tgz` | 12.9 | Volta (V100) through Blackwell: `sm_70`, `75`, `80`, `86`, `89`, `90`, `100`, `120`, `121` | 525.60.13 |
|
|
| RHEL 9 and Ubuntu packages, Windows installer, Windows `rugnux` `.zip` | 13.x | Turing (T4) through Blackwell: the same list **without** `sm_70` | 580.65.06 (Linux), R580 (Windows) |
|
|
| aarch64 `rugnux` `.tgz` | 13.x | `sm_90` (GH200) and `sm_121` (DGX Spark) only | 580.65.06 |
|
|
| any `cpu` / `nocuda` variant | — | — | none |
|
|
|
|
The aarch64 build is cross-compiled and verified in CI to be Arm, self-contained and to carry both
|
|
GPU targets, but it is **not exercised on hardware** — CI has no GH200 or Spark runner.
|
|
|
|
**A V100 needs the CUDA 12 build.** CUDA 13 dropped offline compilation for Volta, and the PTX a
|
|
fatbin also carries only ever JIT-compiles *forwards*, so a CUDA 13 artefact contains nothing a V100
|
|
can execute: every kernel launch fails with *no kernel image is available for execution on the
|
|
device*. On a V100 host take the RHEL 8 packages or the portable Linux `.tgz`. Nothing older than
|
|
Volta is supported.
|
|
|
|
Newer GPUs never need a newer build — the highest generation in the list ships PTX as well as SASS,
|
|
which the driver JIT-compiles for a GPU that came out after the release.
|
|
|
|
The minimum driver above is the floor for the whole CUDA *major* version, which is what applies here
|
|
because the CUDA runtime is statically linked
|
|
([CUDA minor version compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/minor-version-compatibility.html)).
|
|
Newer drivers are always fine; they are backward compatible. A driver from the same release as the
|
|
build toolkit (575.57.08 for the CUDA 12.9 build, 610.43.02 for a CUDA 13.3 one) additionally rules
|
|
out the single caveat of minor version compatibility — a call into a driver API newer than the
|
|
installed driver, which fails with `cudaErrorCallRequiresNewerDriver`.
|
|
|
|
## Windows installer
|
|
|
|
The Windows artefacts are the `jfjoch_viewer` installer and the separate `rugnux` `.zip`; the rest
|
|
of Jungfraujoch (broker, receiver, FPGA host, detector control) is Linux-only.
|
|
|
|
The toolchain bounds of the released installer are:
|
|
|
|
- **Visual Studio 2026** with the C++ (MSVC) toolset. MSVC is not optional — CUDA on Windows builds
|
|
through it — and it is what the release is compiled with.
|
|
- **CUDA Toolkit 13.3** for the `cuda13` variant.
|
|
- **Qt 6.11** for MSVC (`msvc2022_64`), including Qt Charts.
|
|
- Ninja as the generator; zlib and Eigen 3.4 supplied from a build prefix.
|
|
|
|
The installer is generated with NSIS and **bundles the Qt runtime** (via `windeployqt`) and, on the
|
|
CUDA variant, the cuFFT DLL — so the end user installs neither Qt nor a CUDA toolkit. The two
|
|
variants share an install directory and Start Menu group and replace each other (CUDA is a strict
|
|
superset); they are told apart by the installer filename and the Add/Remove Programs entry:
|
|
|
|
| Build | Installer file | Add/Remove Programs |
|
|
| --- | --- | --- |
|
|
| CUDA (default) | `jfjoch-viewer-<version>-win64-cuda<major>.exe` | `Jungfraujoch (CUDA)` |
|
|
| CPU-only | `jfjoch-viewer-<version>-win64-cpu.exe` | `Jungfraujoch (CPU)` |
|
|
|
|
To build the viewer yourself on Windows, see
|
|
[jfjoch_viewer ▸ Building from source on Windows](JFJOCH_VIEWER.md#building-from-source-on-windows).
|
|
|
|
## Licenses
|
|
|
|
Every package variant carries the project license, the third-party manifest and the verbatim
|
|
license texts of the bundled dependencies, each package under a directory of its own —
|
|
`share/doc/jfjoch_broker`, `jfjoch_writer`, `jfjoch_viewer`, `jfjoch_rugnux`, `jfjoch_driver_dkms` —
|
|
so that no two packages claim the same path and they can be upgraded independently. See
|
|
[Third-party software notices](THIRD_PARTY_NOTICES.md).
|