Files
Jungfraujoch/tests/CalibrationTest.cpp
T
leonarski_fandjungfrau 4dc2534dbf
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
v1.0.0.rc-162 (#72)
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-25 08:21:39 +02:00

131 lines
7.0 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#include <cstdio>
#include <fstream>
#include <map>
#include <sstream>
#include "../common/Definitions.h"
#include "../common/JFJochMath.h"
#include "../image_analysis/geom_refinement/Calibrants.h"
#include "../rugnux/RugnuxCalibration.h"
TEST_CASE("Calibrants_LookupIsCaseInsensitive", "[DetGeomCalib]") {
CHECK(CalibrantRings("LaB6") == CalibrantRings("lab6"));
CHECK(CalibrantRings("AgBh") == CalibrantRings("agbh"));
CHECK(CalibrantRings("nonsense").empty());
}
// GuessInitialGeometry pairs the innermost OBSERVED ring with the first entry of this list to fix the
// detector distance, so the first entry has to be a reflection that is really there. Only the primitive
// standard starts at (100): both face-centred ones extinguish it and start at (111), i.e. sqrt(3) times
// further out. Listing a forbidden ring first would scale every calibration by that ratio.
TEST_CASE("Calibrants_FirstRingIsThePresentOne", "[DetGeomCalib]") {
struct Standard { std::string name; double a_A; double first_hkl_norm; };
// sqrt(h^2+k^2+l^2) of the innermost present reflection: LaB6 Pm-3m -> 100, CeO2 Fm-3m and Si Fd-3m -> 111.
const std::vector<Standard> standards = {
{"lab6", LAB6_CELL_A, 1.0},
{"ceo2", 5.4115, std::sqrt(3.0)},
{"si", 5.43102, std::sqrt(3.0)}
};
for (const auto &s : standards) {
const auto q = CalibrantRings(s.name);
REQUIRE(!q.empty());
CHECK(q.front() == Catch::Approx(2.0 * PI * s.first_hkl_norm / s.a_A).epsilon(1e-5));
}
}
// The centring conditions themselves, checked ring by ring rather than only on the first one: an
// extinct reflection anywhere in the list mis-assigns the observed rings around it.
TEST_CASE("Calibrants_CentredStandardsOmitTheExtinctRings", "[DetGeomCalib]") {
auto has_ring = [](const std::vector<float> &q, double a_A, double hkl_norm) {
const auto want = static_cast<float>(2.0 * PI * hkl_norm / a_A);
return std::any_of(q.begin(), q.end(),
[&](float v) { return std::fabs(v - want) < 1e-3f; });
};
const auto ceo2 = CalibrantRings("ceo2");
CHECK(has_ring(ceo2, 5.4115, std::sqrt(3.0))); // 111 - all odd
CHECK(has_ring(ceo2, 5.4115, std::sqrt(4.0))); // 200 - all even
CHECK(has_ring(ceo2, 5.4115, std::sqrt(12.0))); // 222 - all even, present without a glide plane
CHECK_FALSE(has_ring(ceo2, 5.4115, 1.0)); // 100 - mixed parity
CHECK_FALSE(has_ring(ceo2, 5.4115, std::sqrt(2.0))); // 110 - mixed parity
const auto si = CalibrantRings("si");
CHECK(has_ring(si, 5.43102, std::sqrt(3.0))); // 111 - all odd
CHECK(has_ring(si, 5.43102, std::sqrt(8.0))); // 220 - all even, h+k+l = 4n
CHECK_FALSE(has_ring(si, 5.43102, 1.0)); // 100 - mixed parity
CHECK_FALSE(has_ring(si, 5.43102, std::sqrt(4.0))); // 200 - all even, h+k+l = 2
CHECK_FALSE(has_ring(si, 5.43102, std::sqrt(12.0))); // 222 - the diamond glide takes it out
}
// Ice is the reason the calibrant abstraction is a ring list and not a UnitCell: its entries are
// measured ring positions, and enumerating hkl from the hexagonal cell would add rings that are
// systematically absent in P6_3/mmc.
TEST_CASE("Calibrants_IceIsTheMeasuredRingList", "[DetGeomCalib]") {
const auto q = CalibrantRings("ice");
REQUIRE(q.size() == ICE_RING_RES_A.size());
CHECK(std::is_sorted(q.begin(), q.end()));
CHECK(q.front() == Catch::Approx(2.0 * PI / ICE_RING_RES_A[0]).epsilon(1e-5)); // 3.895 A, the widest
}
// pyFAI's Poni1 is the SLOW axis (rows, our y) and Poni2 the FAST axis (columns, our x), both in
// metres. Transposing them produces a file that is silently wrong, so pin the mapping with a geometry
// whose two axes differ.
TEST_CASE("Calibration_PoniFileAxisConvention", "[DetGeomCalib]") {
DiffractionExperiment x(DetJF4M());
x.BeamX_pxl(1000.0f).BeamY_pxl(1275.0f).DetectorDistance_mm(150.0f);
DiffractionGeometry geom = x.GetDiffractionGeometry();
geom.PoniRot1_rad(0.01f).PoniRot2_rad(-0.02f).PoniRot3_rad(0.03f);
const std::string path = "poni_test.poni";
WritePoniFile(path, x, geom);
std::map<std::string, std::string> keys;
std::ifstream f(path);
std::string line;
while (std::getline(f, line)) {
const auto colon = line.find(':');
if (line.empty() || line[0] == '#' || colon == std::string::npos)
continue;
keys[line.substr(0, colon)] = line.substr(colon + 2);
}
f.close();
std::remove(path.c_str());
const double pixel_m = geom.GetPixelSize_mm() * 1e-3;
CHECK(keys["poni_version"] == "2.1");
// orientation 2 = "top left seen from the sample", the MX convention we assemble to. Without it
// pyFAI applies its own default (3, bottom left) and gets the azimuth sense backwards.
CHECK(keys["Detector_config"].find("\"orientation\": 2") != std::string::npos);
// The half pixel is the origin convention (docs/DETECTOR_GEOMETRY.md): our beam centre is
// pixel-centred, pyFAI measures from the edge of the sensor and puts the centre of pixel i at
// (i + 0.5) * pixel size.
// Declaring orientation 2 anchors Poni1 at the top edge, so the same physical point is
// (height - 1 - beam_y) rows down from it.
CHECK(std::stod(keys["Poni1"])
== Catch::Approx((x.GetYPixelsNumConv() - 1 - 1275 + 0.5) * pixel_m)); // slow axis = y
CHECK(std::stod(keys["Poni2"]) == Catch::Approx(1000.5 * pixel_m)); // fast axis = x
CHECK(std::stod(keys["Distance"]) == Catch::Approx(0.150));
// With orientation declared, (Rot1, Rot2, Rot3) = (+rot1, +rot2, -rot3 + pi): a row flip is
// improper, so it reverses rotations about x and about the beam and leaves the one about the
// vertical, and the half turn sets the azimuthal reference - pyFAI's in-plane axes are the
// negatives of ours, so without it every chi is 180 degrees out. Being a rotation about the
// beam it leaves 2theta alone, which is why radial integration was right while the azimuth was
// not. Pinned against pyFAI 2026.5.0 on a tilted detector, against the lab positions of the
// NXmx chain: 2theta to 3.6e-15 deg and chi to 2.8e-14 deg. Do not "fix" these without
// repeating that check - a powder-ring test cannot see rot3, which moves only the azimuth.
CHECK(std::stod(keys["Rot1"]) == Catch::Approx(0.01));
CHECK(std::stod(keys["Rot2"]) == Catch::Approx(-0.02));
CHECK(std::stod(keys["Rot3"]) == Catch::Approx(-0.03 + PI));
CHECK(std::stod(keys["Wavelength"]) == Catch::Approx(geom.GetWavelength_A() * 1e-10));
// max_shape is [rows, cols] - the same slow-then-fast order as Poni1/Poni2.
const std::string shape = "[" + std::to_string(x.GetYPixelsNumConv()) + ", "
+ std::to_string(x.GetXPixelsNumConv()) + "]";
CHECK(keys["Detector_config"].find(shape) != std::string::npos);
}