v1.0.0.rc-162 (#72)

**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>

tests/JFJochReaderTest.cpp

@@ -9,6 +9,8 @@
 #include "../reader/JFJochHDF5Reader.h"
 #include "../compression/JFJochCompressor.h"
 
+#include <future>
+
 TEST_CASE("HDF5DataType_Sign","[HDF5]") {
     HDF5DataType type_u8((uint8_t)0), type_fl(0.0f), type_i32((int32_t) 0), type_u32((uint32_t) 0);
 
@@ -395,6 +397,56 @@ TEST_CASE("JFJochReader_Goniometer", "[HDF5][Full]") {
     REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
 }
 
+// The axis name is free-form in the API, on the wire and in the writer - tests/CBORTest.cpp round
+// trips one literally called "z". The reader used to look only for "omega", so a sweep recorded
+// under any other name came back as stills, with nothing to indicate it. This is that case.
+TEST_CASE("JFJochReader_Goniometer_NonOmegaName", "[HDF5][Full]") {
+    DiffractionExperiment x(DetJF(1));
+
+    x.FilePrefix("test17b").ImagesPerTrigger(950).OverwriteExistingFiles(true);
+    x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
+            .IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
+            .FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
+    x.Goniometer(GoniometerAxis("phi", 12, 0.2f, Coord(-1,0,0),{}));
+
+    RegisterHDF5Filter();
+
+    std::vector<uint16_t> image(x.GetPixelsNum(), 0);
+    {
+        StartMessage start_message;
+        x.FillMessage(start_message);
+
+        FileWriter file_set(start_message);
+
+        DataMessage message{};
+        for (int i = 0; i < 5; i++) {
+            message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
+            message.number = i;
+            REQUIRE_NOTHROW(file_set.WriteHDF5(message));
+        }
+
+        EndMessage end_message;
+        end_message.max_image_number = 5;
+        file_set.WriteHDF5(end_message);
+        file_set.Finalize();
+    }
+    {
+        JFJochHDF5Reader reader;
+        reader.ReadFile("test17b_master.h5");
+        auto dataset = reader.GetDataset();
+
+        REQUIRE(dataset->experiment.GetGoniometer().has_value());
+        CHECK(dataset->experiment.GetGoniometer()->GetName() == "phi");
+        CHECK(dataset->experiment.GetGoniometer()->GetStart_deg() == 12.0);
+        CHECK(dataset->experiment.GetGoniometer()->GetIncrement_deg() == Catch::Approx(0.2f).margin(0.00001f));
+        CHECK(dataset->experiment.GetGoniometer()->IsScanning());
+        CHECK(dataset->experiment.GetGoniometer()->GetAxis().x == -1);
+    }
+    remove("test17b_master.h5");
+
+    REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
+}
+
 TEST_CASE("JFJochReader_GridScan", "[HDF5][Full]") {
     DiffractionExperiment x(DetJF(1));
 
@@ -440,7 +492,11 @@ TEST_CASE("JFJochReader_GridScan", "[HDF5][Full]") {
         auto dataset = reader.GetDataset();
 
 
-        REQUIRE(!dataset->experiment.GetGoniometer().has_value());
+        // A grid scan carries a stationary spindle: NXmx cannot say "no rotation", and a chain of
+        // translations alone is not readable (dxtbx raises on it). It must not read back as a sweep.
+        REQUIRE(dataset->experiment.GetGoniometer().has_value());
+        CHECK(!dataset->experiment.GetGoniometer()->IsScanning());
+        CHECK(dataset->experiment.GetGoniometer()->GetIncrement_deg() == 0.0f);
         REQUIRE(dataset->experiment.GetGridScan().has_value());
 
         CHECK(dataset->experiment.GetGridScan()->IsSnakeScan());
@@ -451,6 +507,24 @@ TEST_CASE("JFJochReader_GridScan", "[HDF5][Full]") {
         CHECK(dataset->experiment.GetGridScan()->GetGridStepX_um() == Catch::Approx(-7.5));
         CHECK(dataset->experiment.GetGridScan()->GetGridStepY_um() == Catch::Approx(8.0));
     }
+    {
+        // That placeholder spindle must carry one entry per image, not a scalar. Our own reader
+        // copes with either, so the check has to be on the stored shape: a third-party reader takes
+        // the image count from the innermost axis of the sample chain, and grid_scan_x/y are
+        // translations and are passed over - so with a scalar here the whole scan reads as one image.
+        hid_t file = H5Fopen("test_reader_grid_scan_master.h5", H5F_ACC_RDONLY, H5P_DEFAULT);
+        REQUIRE(file >= 0);
+        hid_t omega = H5Dopen2(file, "/entry/sample/transformations/omega", H5P_DEFAULT);
+        REQUIRE(omega >= 0);
+        hid_t space = H5Dget_space(omega);
+        CHECK(H5Sget_simple_extent_ndims(space) == 1);
+        hsize_t dim = 0;
+        H5Sget_simple_extent_dims(space, &dim, nullptr);
+        CHECK(dim == 5);
+        H5Sclose(space);
+        H5Dclose(omega);
+        H5Fclose(file);
+    }
     remove("test_reader_grid_scan_master.h5");
 
     REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
@@ -1682,6 +1756,67 @@ TEST_CASE("JFJochReader_GetRawImage_NXmxLegacy", "[HDF5][Full]") {
     REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
 }
 
+// GetRawImage takes the chunk address under the HDF5 lock, then reads the bytes outside it, so this
+// is the one path where several workers are inside the reader at once - which is how rugnux uses it.
+// The per-image cases above are all single-threaded and would not notice the file being pulled from
+// under a read, nor a cache entry racing its own creation.
+TEST_CASE("JFJochReader_GetRawImage_Concurrent", "[HDF5][Full]") {
+    DiffractionExperiment x(DetJF(1));
+    x.FilePrefix("test_raw_concurrent").ImagesPerTrigger(16).OverwriteExistingFiles(true);
+    x.BitDepthImage(16).ImagesPerFile(4).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true);
+    x.Compression(CompressionAlgorithm::BSHUF_ZSTD);
+
+    std::vector<int16_t> image(x.GetPixelsNum());
+    for (size_t i = 0; i < image.size(); i++)
+        image[i] = static_cast<int16_t>((i * 11 + 5) % UINT16_MAX);
+
+    RegisterHDF5Filter();
+    JFJochBitShuffleCompressor compressor(CompressionAlgorithm::BSHUF_ZSTD);
+    const auto compressed_image = compressor.Compress(image);
+    {
+        StartMessage start_message;
+        x.FillMessage(start_message);
+        FileWriter file_set(start_message);
+        for (int i = 0; i < x.GetImageNum(); i++) {
+            DataMessage message{};
+            message.image = CompressedImage(compressed_image, x.GetXPixelsNum(), x.GetYPixelsNum(),
+                                            CompressedImageMode::Int16, CompressionAlgorithm::BSHUF_ZSTD);
+            message.number = i;
+            REQUIRE_NOTHROW(file_set.WriteHDF5(message));
+        }
+        EndMessage end_message;
+        end_message.max_image_number = x.GetImageNum();
+        file_set.WriteHDF5(end_message);
+        file_set.Finalize();
+    }
+    {
+        JFJochHDF5Reader reader;
+        REQUIRE_NOTHROW(reader.ReadFile("test_raw_concurrent_master.h5"));
+
+        std::vector<std::future<bool>> workers;
+        workers.reserve(8);
+        for (int w = 0; w < 8; w++) {
+            workers.push_back(std::async(std::launch::async, [&reader, &compressed_image, &x]() {
+                for (int i = 0; i < x.GetImageNum(); i++) {
+                    auto raw = reader.GetRawImage(i);
+                    if (raw->image_buffer.size() != compressed_image.size())
+                        return false;
+                    if (memcmp(raw->image_buffer.data(), compressed_image.data(),
+                               compressed_image.size()) != 0)
+                        return false;
+                }
+                return true;
+            }));
+        }
+        for (auto &worker: workers)
+            CHECK(worker.get());
+    }
+    remove("test_raw_concurrent_master.h5");
+    for (int f = 1; f <= 4; f++)
+        remove(("test_raw_concurrent_data_00000" + std::to_string(f) + ".h5").c_str());
+    REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
+}
+
 TEST_CASE("JFJochReader_GetRawImage_VDS", "[HDF5][Full]") {
     DiffractionExperiment x(DetJF(1));
 
@@ -2871,4 +3006,394 @@ TEST_CASE("JFJochReader_Snapshots", "[HDF5][Full]") {
     remove("test_snap_proc_master.h5");
 
     REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
-}
+}
+// The chain may be sent in the END message or left to the writer to build. Both must produce the
+// same file - the sent one is written verbatim, which is what will later allow measured positions to
+// be reported, and the built one is what a producer that does not send it gets.
+TEST_CASE("JFJochReader_TransformationChain_SentAndBuilt", "[HDF5][Full]") {
+    DiffractionExperiment x(DetJF(1));
+    x.ImagesPerTrigger(5).OverwriteExistingFiles(true);
+    x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
+            .IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
+            .FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
+    x.Goniometer(GoniometerAxis("omega", 95, 0.1f, Coord(0,-1,0), {}));
+    x.Smargon(SmargonPosition{.phi_deg = -7.25f, .chi_deg = 12.5f});
+
+    RegisterHDF5Filter();
+    std::vector<uint16_t> image(x.GetPixelsNum(), 0);
+
+    const auto write = [&](const std::string &prefix, bool send_chain) {
+        DiffractionExperiment local = x;
+        local.FilePrefix(prefix);
+        StartMessage start_message;
+        local.FillMessage(start_message);
+
+        FileWriter file_set(start_message);
+        DataMessage message{};
+        for (int i = 0; i < local.GetImageNum(); i++) {
+            message.image = CompressedImage(image, local.GetXPixelsNum(), local.GetYPixelsNum());
+            message.number = i;
+            REQUIRE_NOTHROW(file_set.WriteHDF5(message));
+        }
+
+        EndMessage end_message;
+        end_message.max_image_number = local.GetImageNum();
+        if (send_chain)
+            end_message.transformations = local.BuildTransformationChain(local.GetImageNum());
+        file_set.WriteHDF5(end_message);
+        file_set.Finalize();
+    };
+
+    write("test_chain_built", false);
+    write("test_chain_sent", true);
+
+    const auto read = [](const std::string &prefix) {
+        JFJochHDF5Reader reader;
+        reader.ReadFile(prefix + "_master.h5");
+        return reader.GetDataset()->experiment;
+    };
+
+    const auto built = read("test_chain_built");
+    const auto sent = read("test_chain_sent");
+
+    REQUIRE(built.GetGoniometer().has_value());
+    REQUIRE(sent.GetGoniometer().has_value());
+    CHECK(sent.GetGoniometer()->GetName() == built.GetGoniometer()->GetName());
+    CHECK(sent.GetGoniometer()->GetStart_deg()
+          == Catch::Approx(built.GetGoniometer()->GetStart_deg()).margin(1e-3));
+    CHECK(sent.GetGoniometer()->GetIncrement_deg()
+          == Catch::Approx(built.GetGoniometer()->GetIncrement_deg()).margin(1e-4));
+
+    // chi/phi survive both routes, which they did not before they became ordinary axes.
+    REQUIRE(built.GetDatasetSettings().GetSmargonPosition().has_value());
+    REQUIRE(sent.GetDatasetSettings().GetSmargonPosition().has_value());
+    CHECK(sent.GetDatasetSettings().GetSmargonPosition()->chi_deg
+          == Catch::Approx(12.5f).margin(1e-3));
+    CHECK(sent.GetDatasetSettings().GetSmargonPosition()->phi_deg
+          == Catch::Approx(-7.25f).margin(1e-3));
+
+    // Compared on the files, not through the reader: the reader reads neither AXISNAME_end nor the
+    // rotation width, so it cannot see the two routes diverge - and it did, until the writer started
+    // deriving them for a chain it was handed.
+    {
+        HDF5ReadOnlyFile built_file("test_chain_built_master.h5");
+        HDF5ReadOnlyFile sent_file("test_chain_sent_master.h5");
+        CHECK(built_file.FindLeafs("/entry/sample/transformations")
+              == sent_file.FindLeafs("/entry/sample/transformations"));
+        CHECK(sent_file.ReadVector<double>("/entry/sample/transformations/omega_end").size()
+              == static_cast<size_t>(x.GetImageNum()));
+        CHECK(sent_file.ReadVector<double>("/entry/sample/transformations/omega_range_average").at(0)
+              == Catch::Approx(0.1).margin(1e-4));
+    }
+
+    remove("test_chain_built_master.h5");
+    remove("test_chain_sent_master.h5");
+    REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
+}
+
+// Recovering the sample axes from a written file, across the configurations the writer produces.
+// The reader searches every leaf of /entry/sample/transformations, so on the way it meets the
+// writer's own AXISNAME_end and rotation-width datasets, which are not axes and carry no
+// transformation_type. It used to throw on them: a master whose axis did not turn never stopped the
+// search early, walked into omega_end and could not be opened at all - which took out rugnux's own
+// output for a grid scan.
+TEST_CASE("JFJochReader_AxisRecovery", "[HDF5][Full]") {
+    RegisterHDF5Filter();
+
+    const auto round_trip = [](DiffractionExperiment x, const std::string &prefix) {
+        x.FilePrefix(prefix).OverwriteExistingFiles(true)
+                .BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
+                .IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
+                .FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
+        std::vector<uint16_t> image(x.GetPixelsNum(), 0);
+        StartMessage start_message;
+        x.FillMessage(start_message);
+        {
+            FileWriter file_set(start_message);
+            DataMessage message{};
+            for (int i = 0; i < x.GetImageNum(); i++) {
+                message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
+                message.number = i;
+                REQUIRE_NOTHROW(file_set.WriteHDF5(message));
+            }
+            EndMessage end_message;
+            end_message.max_image_number = x.GetImageNum();
+            file_set.WriteHDF5(end_message);
+            file_set.Finalize();
+        }
+        JFJochHDF5Reader reader;
+        REQUIRE_NOTHROW(reader.ReadFile(prefix + "_master.h5"));
+        return reader.GetDataset()->experiment;
+    };
+    const auto cleanup = [](const std::string &prefix) {
+        remove((prefix + "_master.h5").c_str());
+        remove((prefix + "_data_000001.h5").c_str());
+    };
+
+    SECTION("a sweep") {
+        DiffractionExperiment x(DetJF(1));
+        x.ImagesPerTrigger(5).Goniometer(GoniometerAxis("omega", 95, 0.1f, Coord(0,-1,0), {}));
+        const auto out = round_trip(x, "test_ax_sweep");
+        REQUIRE(out.GetGoniometer().has_value());
+        CHECK(out.GetGoniometer()->GetName() == "omega");
+        CHECK(out.GetGoniometer()->IsScanning());
+        CHECK(out.GetGoniometer()->GetStart_deg() == Catch::Approx(95).margin(1e-3));
+        CHECK(out.GetGoniometer()->GetIncrement_deg() == Catch::Approx(0.1).margin(1e-4));
+        CHECK(out.GetGoniometer()->GetAxis() == Coord(0,-1,0));
+        cleanup("test_ax_sweep");
+    }
+
+    SECTION("a sweep about an axis that is not called omega") {
+        DiffractionExperiment x(DetJF(1));
+        x.ImagesPerTrigger(5).Goniometer(GoniometerAxis("kappa", 10, 0.5f, Coord(0,-1,0), {}));
+        const auto out = round_trip(x, "test_ax_kappa");
+        REQUIRE(out.GetGoniometer().has_value());
+        CHECK(out.GetGoniometer()->GetName() == "kappa");
+        CHECK(out.GetGoniometer()->IsScanning());
+        CHECK(out.GetGoniometer()->GetIncrement_deg() == Catch::Approx(0.5).margin(1e-4));
+        cleanup("test_ax_kappa");
+    }
+
+    SECTION("a spindle that does not turn") {
+        DiffractionExperiment x(DetJF(1));
+        x.ImagesPerTrigger(5).Goniometer(GoniometerAxis("omega", 12.5f, 0.0f, Coord(0,-1,0), {}));
+        const auto out = round_trip(x, "test_ax_still");
+        REQUIRE(out.GetGoniometer().has_value());
+        CHECK(out.GetGoniometer()->GetName() == "omega");
+        CHECK(!out.GetGoniometer()->IsScanning());
+        CHECK(out.GetGoniometer()->GetStart_deg() == Catch::Approx(12.5).margin(1e-3));
+        cleanup("test_ax_still");
+    }
+
+    SECTION("a grid scan, which sits on a spindle that does not turn") {
+        DiffractionExperiment x(DetJF(1));
+        x.ImagesPerTrigger(6).GridScan(GridScanSettings(3, 10.0f, 20.0f, false, false).ImageNum(6));
+        const auto out = round_trip(x, "test_ax_grid");
+        REQUIRE(out.GetGridScan().has_value());
+        CHECK(out.GetGridScan()->GetNFast() == 3);
+        CHECK(out.GetGridScan()->GetGridStepX_um() == Catch::Approx(10.0).margin(1e-3));
+        CHECK(out.GetGridScan()->GetGridStepY_um() == Catch::Approx(20.0).margin(1e-3));
+        // NXmx cannot say "no rotation", so the writer records the spindle standing still.
+        REQUIRE(out.GetGoniometer().has_value());
+        CHECK(!out.GetGoniometer()->IsScanning());
+        cleanup("test_ax_grid");
+    }
+
+    SECTION("a grid scan under a turning spindle") {
+        DiffractionExperiment x(DetJF(1));
+        x.ImagesPerTrigger(6).GridScan(GridScanSettings(3, 10.0f, 20.0f, false, false).ImageNum(6))
+                .Goniometer(GoniometerAxis("omega", 0, 0.2f, Coord(0,-1,0), {}));
+        const auto out = round_trip(x, "test_ax_gridsweep");
+        REQUIRE(out.GetGridScan().has_value());
+        CHECK(out.GetGridScan()->GetNFast() == 3);
+        REQUIRE(out.GetGoniometer().has_value());
+        CHECK(out.GetGoniometer()->IsScanning());
+        CHECK(out.GetGoniometer()->GetIncrement_deg() == Catch::Approx(0.2).margin(1e-4));
+        cleanup("test_ax_gridsweep");
+    }
+
+    SECTION("a sweep with the head at a Smargon position") {
+        DiffractionExperiment x(DetJF(1));
+        x.ImagesPerTrigger(5).Goniometer(GoniometerAxis("omega", 95, 0.1f, Coord(0,-1,0), {}))
+                .Smargon(SmargonPosition{.phi_deg = -7.25f, .chi_deg = 12.5f});
+        const auto out = round_trip(x, "test_ax_smargon");
+        REQUIRE(out.GetGoniometer().has_value());
+        CHECK(out.GetGoniometer()->GetName() == "omega");
+        CHECK(out.GetGoniometer()->IsScanning());
+        REQUIRE(out.GetDatasetSettings().GetSmargonPosition().has_value());
+        CHECK(out.GetDatasetSettings().GetSmargonPosition()->chi_deg == Catch::Approx(12.5).margin(1e-3));
+        CHECK(out.GetDatasetSettings().GetSmargonPosition()->phi_deg == Catch::Approx(-7.25).margin(1e-3));
+        cleanup("test_ax_smargon");
+    }
+
+    REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
+}
+
+// saturation_value is written inclusive and used exclusive, so a read has to add the count back. It
+// did not, and the value fell by one on every write-read-write cycle - unbounded, and compounding
+// whenever a _process.h5 was reprocessed. Nothing caught it: no test asserted the read-back limit.
+TEST_CASE("JFJochReader_SaturationSurvivesRoundTrip", "[HDF5][Full]") {
+    DiffractionExperiment x(DetJF(1));
+    x.ImagesPerTrigger(2).OverwriteExistingFiles(true).FilePrefix("test_satrt");
+    x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
+            .IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
+            .FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
+
+    const int64_t original_limit = x.GetSaturationLimit();
+    RegisterHDF5Filter();
+    std::vector<uint16_t> image(x.GetPixelsNum(), 0);
+
+    // The second pass writes metadata only: reading pins the experiment to signed 32-bit, the
+    // container images are handed out in, so feeding it the uint16 frames again would - rightly -
+    // be refused by the writer's pixel-format check.
+    const auto write = [&](const DiffractionExperiment &src, const std::string &prefix,
+                           bool with_images) {
+        DiffractionExperiment local = src;
+        local.FilePrefix(prefix).OverwriteExistingFiles(true);
+        StartMessage start_message;
+        local.FillMessage(start_message);
+        FileWriter file_set(start_message);
+        DataMessage message{};
+        if (with_images) {
+            for (int i = 0; i < 2; i++) {
+                message.image = CompressedImage(image, local.GetXPixelsNum(), local.GetYPixelsNum());
+                message.number = i;
+                REQUIRE_NOTHROW(file_set.WriteHDF5(message));
+            }
+        }
+        EndMessage end_message;
+        end_message.max_image_number = 2;
+        file_set.WriteHDF5(end_message);
+        file_set.Finalize();
+        return start_message.saturation_value;
+    };
+    const auto read = [](const std::string &prefix) {
+        JFJochHDF5Reader reader;
+        reader.ReadFile(prefix + "_master.h5");
+        return reader.GetDataset()->experiment;
+    };
+
+    const int64_t declared_once = write(x, "test_satrt", true);
+    CHECK(declared_once == SaturationValueFromLimit(original_limit));
+
+    const auto once = read("test_satrt");
+    CHECK(once.GetSaturationLimit() == original_limit);
+
+    // The cycle that used to lose a count: read a file, write what was read, read it again.
+    const int64_t declared_twice = write(once, "test_satrt2", false);
+    CHECK(declared_twice == declared_once);
+    CHECK(read("test_satrt2").GetSaturationLimit() == original_limit);
+
+    remove("test_satrt_master.h5");
+    remove("test_satrt_data_000001.h5");
+    remove("test_satrt2_master.h5");
+    remove("test_satrt2_data_000001.h5");
+    REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
+}
+
+// A head position is not a sweep, and it is not the spindle either. Both properties are carried by
+// the file itself - the axis length says how many images there are, the equipment_component tag says
+// what the axis is - so both are checked here on the file, not through the reader: the reader alone
+// cannot see a shape or an attribute it never looks at.
+TEST_CASE("JFJochReader_Smargon_StillIsNotOneImage", "[HDF5][Full]") {
+    DiffractionExperiment x(DetJF(1));
+    x.ImagesPerTrigger(5).OverwriteExistingFiles(true).FilePrefix("test_smargon");
+    x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
+            .IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
+            .FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
+    x.Smargon(SmargonPosition{.phi_deg = -7.25f, .chi_deg = 12.5f});
+    REQUIRE(!x.GetGoniometer().has_value());
+
+    RegisterHDF5Filter();
+    std::vector<uint16_t> image(x.GetPixelsNum(), 0);
+
+    StartMessage start_message;
+    x.FillMessage(start_message);
+    FileWriter file_set(start_message);
+    DataMessage message{};
+    for (int i = 0; i < x.GetImageNum(); i++) {
+        message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
+        message.number = i;
+        REQUIRE_NOTHROW(file_set.WriteHDF5(message));
+    }
+    EndMessage end_message;
+    end_message.max_image_number = x.GetImageNum();
+    file_set.WriteHDF5(end_message);
+    file_set.Finalize();
+
+    {
+        HDF5ReadOnlyFile master("test_smargon_master.h5");
+
+        // One entry per image. A reader takes the image count from the innermost axis of the sample
+        // chain when no axis varies; as scalars these read back as a single image.
+        CHECK(master.GetDimension("/entry/sample/transformations/chi")
+              == std::vector<hsize_t>{static_cast<hsize_t>(x.GetImageNum())});
+        CHECK(master.GetDimension("/entry/sample/transformations/phi")
+              == std::vector<hsize_t>{static_cast<hsize_t>(x.GetImageNum())});
+        CHECK(master.ReadVector<double>("/entry/sample/transformations/phi")
+              == std::vector<double>(x.GetImageNum(), -7.25));
+
+        // Tagged, so neither is mistaken for the spindle - and so a phi from anywhere else is not
+        // mistaken for a head position.
+        HDF5DataSet chi(master, "/entry/sample/transformations/chi");
+        HDF5DataSet phi(master, "/entry/sample/transformations/phi");
+        REQUIRE(chi.AttrExists("equipment_component"));
+        REQUIRE(phi.AttrExists("equipment_component"));
+        CHECK(chi.ReadAttrStr("equipment_component") == "smargon");
+        CHECK(phi.ReadAttrStr("equipment_component") == "smargon");
+    }
+
+    const auto read = [](const std::string &prefix) {
+        JFJochHDF5Reader reader;
+        reader.ReadFile(prefix + "_master.h5");
+        return reader.GetDataset()->experiment;
+    };
+    const auto read_back = read("test_smargon");
+
+    // chi is the alphabetically first stationary axis in the file; it must not become the spindle.
+    CHECK(!read_back.GetGoniometer().has_value());
+    REQUIRE(read_back.GetDatasetSettings().GetSmargonPosition().has_value());
+    CHECK(read_back.GetDatasetSettings().GetSmargonPosition()->chi_deg == Catch::Approx(12.5f).margin(1e-3));
+    CHECK(read_back.GetDatasetSettings().GetSmargonPosition()->phi_deg == Catch::Approx(-7.25f).margin(1e-3));
+
+    remove("test_smargon_master.h5");
+    remove("test_smargon_data_000001.h5");
+    REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
+}
+
+// phi is an ordinary spindle name in MX. A file whose rotation axis is called phi carries no
+// equipment_component, so it stays the spindle and no head position is invented from it - which also
+// means the file can be written back out, instead of colliding on a second dataset called phi.
+TEST_CASE("JFJochReader_Goniometer_NamedPhiIsNotSmargon", "[HDF5][Full]") {
+    DiffractionExperiment x(DetJF(1));
+    x.ImagesPerTrigger(5).OverwriteExistingFiles(true).FilePrefix("test_phispindle");
+    x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
+            .IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
+            .FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
+    x.Goniometer(GoniometerAxis("phi", 30, 0.2f, Coord(0,-1,0), {}));
+
+    RegisterHDF5Filter();
+    std::vector<uint16_t> image(x.GetPixelsNum(), 0);
+
+    StartMessage start_message;
+    x.FillMessage(start_message);
+    FileWriter file_set(start_message);
+    DataMessage message{};
+    for (int i = 0; i < x.GetImageNum(); i++) {
+        message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
+        message.number = i;
+        REQUIRE_NOTHROW(file_set.WriteHDF5(message));
+    }
+    EndMessage end_message;
+    end_message.max_image_number = x.GetImageNum();
+    file_set.WriteHDF5(end_message);
+    file_set.Finalize();
+
+    const auto read = [](const std::string &prefix) {
+        JFJochHDF5Reader reader;
+        reader.ReadFile(prefix + "_master.h5");
+        return reader.GetDataset()->experiment;
+    };
+    const auto read_back = read("test_phispindle");
+
+    REQUIRE(read_back.GetGoniometer().has_value());
+    CHECK(read_back.GetGoniometer()->GetName() == "phi");
+    CHECK(read_back.GetGoniometer()->GetStart_deg() == Catch::Approx(30).margin(1e-3));
+    CHECK(!read_back.GetDatasetSettings().GetSmargonPosition().has_value());
+
+    // Writing what was read must not try to create phi a second time.
+    DiffractionExperiment rewrite = read_back;
+    rewrite.FilePrefix("test_phispindle_out").OverwriteExistingFiles(true);
+    StartMessage out_start;
+    rewrite.FillMessage(out_start);
+    FileWriter out(out_start);
+    EndMessage out_end;
+    out_end.max_image_number = rewrite.GetImageNum();
+    REQUIRE_NOTHROW(out.WriteHDF5(out_end));
+    REQUIRE_NOTHROW(out.Finalize());
+
+    remove("test_phispindle_master.h5");
+    remove("test_phispindle_data_000001.h5");
+    remove("test_phispindle_out_master.h5");
+    REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
+}