v1.0.0-rc.166 (#76)

* `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands.
* `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion.
* Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants.
* `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing.
* A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed.
* `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing.
* Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences.
* The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to.
* The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after.
* The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution.
* `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have.
* Twinning is no longer reported when the L-test contradicts it.
* The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's.
* `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots.
* The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area.

Reviewed-on: #76
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>

reader/HDF5MetadataSource.cpp

@@ -2,6 +2,7 @@
 // SPDX-License-Identifier: GPL-3.0-only
 
 #include <cmath>
+#include <cstring>
 #include <set>
 
 #include "HDF5MetadataSource.h"
@@ -10,8 +11,98 @@
 #include "../image_analysis/spot_finding/SpotUtils.h"
 #include "../common/GridScanSettings.h"
 #include "../common/JFJochMath.h"
+#include "../common/Logger.h"
 #include "../common/ROIDefinition.h"
 
+// A McStas direction in the internal frame. The two differ by a 180 degree turn about z, which is a
+// rotation and not a mirror - so an axis carried through it turns the same way by the same angle.
+static Coord McStasToInternal(const std::vector<double> &v) {
+    return {static_cast<float>(-v[0]), static_cast<float>(-v[1]), static_cast<float>(v[2])};
+}
+
+// The image orientation the file itself states, in its NXdetector_module pixel directions. NXmx gives
+// those in the McStas frame.
+//
+// Only an exact match against one of the eight discrete orientations is taken. Anything else is a
+// continuous rotation of the detector in its own plane, which belongs in rot1/rot2/rot3 and cannot be
+// separated from the tilt by looking at the module alone - so it is left as it is rather than
+// approximated. Every real file examined here is exactly discrete.
+static std::optional<DetectorOrientation> ReadModuleOrientation(HDF5Object *file) {
+    const std::string base = "/entry/instrument/detector/module/";
+    if (!file->IsDataSet(base + "fast_pixel_direction") || !file->IsDataSet(base + "slow_pixel_direction"))
+        return {};
+
+    HDF5DataSet fast_dataset(*file, base + "fast_pixel_direction");
+    HDF5DataSet slow_dataset(*file, base + "slow_pixel_direction");
+    if (!fast_dataset.AttrExists("vector") || !slow_dataset.AttrExists("vector"))
+        return {};
+
+    const auto f = fast_dataset.ReadAttrVec("vector");
+    const auto s = slow_dataset.ReadAttrVec("vector");
+    if ((f.size() != 3) || (s.size() != 3))
+        return {};
+
+    return DetectorOrientation::Match(McStasToInternal(f), McStasToInternal(s));
+}
+
+// Where the detector stands, from the chain of transformations the file says it depends on.
+//
+// NXmx has no field for a detector swung out on a 2theta arm. It states the detector's position as a
+// depends_on chain and the arm is one rotation in that chain, so following the chain is the only way
+// to find it: "two_theta" is one beamline's name for that dataset and the next spells it otherwise.
+//
+// Only the rotations are taken, composed from the detector outwards. Each transformation states its
+// vector in the frame of the one it depends on, so the product is the rotation that carries a
+// detector square to the beam to where this one stands. The translations in the chain are the
+// detector distance and the beam centre, which the file states separately in that square-on frame -
+// the arm turns the detector about the sample and moves neither, and a Diamond master writes the same
+// beam_center_x/y for a swung sweep as for the square-on one beside it. Nothing comes back when no
+// rotation in the chain turns, which is every detector square to the beam.
+static std::optional<RotMatrix> ReadDetectorRotationChain(HDF5Object *file) {
+    std::string node = file->GetString("/entry/instrument/detector/depends_on");
+    if (node.empty() && file->IsDataSet("/entry/instrument/detector/module/module_offset")) {
+        HDF5DataSet module_offset(*file, "/entry/instrument/detector/module/module_offset");
+        if (module_offset.AttrExists("depends_on"))
+            node = module_offset.ReadAttrStr("depends_on");
+    }
+
+    // A file this system wrote states its PONI angles in the chain as well, and they are read from
+    // these three paths just before this is called. Taking them here too would apply the tilt twice.
+    static const std::set<std::string> poni_angles = {"/entry/instrument/detector/transformations/rot1",
+                                                      "/entry/instrument/detector/transformations/rot2",
+                                                      "/entry/instrument/detector/transformations/rot3"};
+
+    RotMatrix chain;
+    bool turns = false;
+    std::set<std::string> seen;
+    while ((node != ".") && !node.empty() && file->IsDataSet(node) && seen.insert(node).second) {
+        HDF5DataSet axis(*file, node);
+        const std::string current = node;
+        node = axis.AttrExists("depends_on") ? axis.ReadAttrStr("depends_on") : ".";
+
+        if (poni_angles.contains(current) || !axis.AttrExists("transformation_type") || !axis.AttrExists("vector")
+                || (axis.ReadAttrStr("transformation_type") != "rotation"))
+            continue;
+
+        std::vector<double> value;
+        axis.ReadVector(value);
+        const auto vec = axis.ReadAttrVec("vector");
+        if (value.empty() || (value[0] == 0.0) || (vec.size() != 3))
+            continue;
+
+        // NXmx states a rotation in degrees unless it says otherwise.
+        const bool radians = axis.AttrExists("units") && (axis.ReadAttrStr("units") == "rad");
+        const auto angle_rad = static_cast<float>(radians ? value[0] : value[0] * PI / 180.0);
+        chain = RotMatrix(angle_rad, McStasToInternal(vec)) * chain;
+        turns = true;
+    }
+
+    if (!turns)
+        return {};
+    return chain;
+}
+
+
 inline std::pair<gemmi::CrystalSystem, char> parse_bravais_lattice(const std::string &val) {
     if (val.empty())
         return {gemmi::CrystalSystem::Triclinic, 'P'};
@@ -65,7 +156,7 @@ inline std::pair<gemmi::CrystalSystem, char> parse_bravais_lattice(const std::st
 std::vector<hsize_t> GetDimension(HDF5Object &object, const std::string &path) {
     const auto dim = object.GetDimension(path);
     if (dim.size() != 3)
-        throw JFJochException(JFJochExceptionCategory::HDF5, "Wrong dimension of /entry/data/data");
+        throw JFJochException(JFJochExceptionCategory::HDF5, "Wrong dimension of " + path);
     return dim;
 }
 
@@ -100,6 +191,74 @@ std::string ResolveRelativeToMaster(const std::string &directory,
     return (std::filesystem::path(directory) / path).string();
 }
 
+// A length in NXmx says which unit it is in, and a master written outside the DECTRIS toolchain
+// uses that freedom: a Diamond-written one states pixel size, sensor thickness and detector
+// distance in millimetres, which is correct NXmx. Reading those as metres is not a failure but a
+// silent factor of a thousand, so the unit is read rather than assumed. An undeclared unit means
+// metres - what every DECTRIS master and everything this system writes means by one. An unknown
+// unit is refused rather than guessed at, for the same reason.
+float LengthUnitFactor(HDF5DataSet &dataset, const std::string &name) {
+    if (!dataset.AttrExists("units"))
+        return 1.0f;
+
+    const std::string units = dataset.ReadAttrStr("units");
+    if (units == "m")
+        return 1.0f;
+    if (units == "mm")
+        return 1e-3f;
+    if (units == "um")
+        return 1e-6f;
+    throw JFJochException(JFJochExceptionCategory::HDF5, name + ": unknown length unit " + units);
+}
+
+float ReadLength_m(HDF5Object &file, const std::string &name) {
+    HDF5DataSet dataset(file, name);
+    return dataset.ReadScalar<float>() * LengthUnitFactor(dataset, name);
+}
+
+// The same value under different names. DECTRIS Eiger firmware 1.x writes detector_distance where
+// NXmx says distance, and a file from that era is still what a repository hands you; a Diamond
+// master puts the distance one level up, in NXinstrument rather than in NXdetector. The NXmx
+// spelling is tried first and the rest are pure fallbacks, which is safe because every current
+// Eiger master carries BOTH DECTRIS names (measured on thirteen masters from ten facilities,
+// firmware release-2020.2.1 through release-2024.1.1 - all of them write detector_distance and
+// countrate_correction_count_cutoff beside the NXmx names). So this can never change what a
+// modern file reads. ReadIntWithLegacyFallback below is the same arrangement for saturation_value.
+float ReadLengthWithFallback_m(HDF5Object &file, const std::vector<std::string> &names) {
+    for (const auto &name: names)
+        if (file.Exists(name))
+            return ReadLength_m(file, name);
+    throw JFJochException(JFJochExceptionCategory::HDF5, "Cannot find " + names.front());
+}
+
+int64_t ReadIntWithLegacyFallback(HDF5Object &file, const std::string &nxmx, const std::string &legacy) {
+    if (file.Exists(nxmx))
+        return file.GetInt(nxmx);
+    if (file.Exists(legacy))
+        return file.GetInt(legacy);
+    throw JFJochException(JFJochExceptionCategory::HDF5, "Cannot find " + nxmx + " (nor " + legacy + ")");
+}
+
+// Where the goniometer axes live. NXmx puts them in /entry/sample/transformations; firmware 1.x put
+// them in /entry/sample/goniometer and wrote no transformation_type and no vector on them. Getting
+// this one wrong is not a missing value but a WRONG ANSWER: a goniometer is only ever set from this
+// group, so a file whose axes are somewhere else is read as stills, silently.
+//
+// A hybrid file has both: an NXmx transformations group holding one empty subgroup per axis, which
+// states the direction and nothing else, beside a legacy goniometer group holding all the angles.
+// So present is not the same as usable - transformations is the angle source only if it holds an
+// axis dataset, and an axis is always a dataset.
+std::string GoniometerGroup(HDF5Object &file) {
+    if (file.Exists("/entry/sample/transformations")) {
+        for (const auto &name: file.FindLeafs("/entry/sample/transformations"))
+            if (file.IsDataSet("/entry/sample/transformations/" + name))
+                return "/entry/sample/transformations";
+    }
+    if (file.Exists("/entry/sample/goniometer"))
+        return "/entry/sample/goniometer";
+    return {};
+}
+
 template<class T>
 void ReadVector(std::vector<T> &v,
                 HDF5Object &file,
@@ -335,7 +494,7 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         // at the end. format stays NoFile if the master carries no image data.
         FileWriterFormat format = FileWriterFormat::NoFile;
         HDF5DataSetLayout data_layout = HDF5DataSetLayout::CONTIGUOUS;
-        std::vector<std::string> legacy_format_files;
+        std::vector<HDF5ImageLocator::LegacyFile> legacy_format_files;
         std::vector<HDF5VirtualDatasetMapping> vds_data_mappings;
         size_t images_per_file = 1;
 
@@ -430,11 +589,6 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
 
             legacy_format_files.clear();
 
-            image_size_x = master_file->GetInt("/entry/instrument/detector/detectorSpecific/x_pixels_in_detector");
-            image_size_y = master_file->GetInt("/entry/instrument/detector/detectorSpecific/y_pixels_in_detector");
-
-            //size_t expected_images = master_file->GetInt("/entry/instrument/detector/detectorSpecific/nimages");
-
             images_per_file = 0;
             number_of_images = 0;
             uint32_t nfiles = 0;
@@ -450,14 +604,26 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                 size_t fimages = 0;
 
                 try {
-                    auto fname = ResolveRelativeToMaster(directory.string(),
-                                                         master_file->GetLinkedFileName(dname));
+                    // The link names both the file and the dataset inside it. DECTRIS and this
+                    // system always call that dataset /entry/data/data, but a Diamond-written
+                    // master links to a plain /data - so looking up a fixed name found nothing,
+                    // and a full sweep read back as zero images.
+                    const auto [linked_file, data_path] = master_file->GetLinkedTarget(dname);
+                    const auto fname = ResolveRelativeToMaster(directory.string(), linked_file);
 
                     HDF5ReadOnlyFile data_file(fname);
 
-                    fimages = GetDimension(data_file, "/entry/data/data")[0];
+                    // The image size comes from the array itself, as it does for a VDS master.
+                    // detectorSpecific/x_pixels_in_detector is a DECTRIS extension rather than
+                    // NXmx, so a third-party writer has no reason to emit it.
+                    const auto dim = GetDimension(data_file, data_path);
+                    fimages = dim[0];
+                    if (nfiles == 0) {
+                        image_size_y = dim[1];
+                        image_size_x = dim[2];
+                    }
 
-                    legacy_format_files.push_back(fname);
+                    legacy_format_files.push_back({fname, data_path});
 
                     if (nfiles == 0 && data_file.Exists("/entry/roi"))
                         dataset->roi = data_file.FindLeafs("/entry/roi");
@@ -565,6 +731,10 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                                    number_of_images, fimages);
                     }
                 } catch (JFJochException &e) {
+                    // Say why. Everything read here is optional per-image metadata except the
+                    // image array itself, and losing that silently leaves an empty dataset that
+                    // reads as "nothing to process" rather than as a broken file.
+                    Logger("HDF5Reader").Warning("{}: {}", dname, e.what());
                 }
 
                 if (nfiles == 0)
@@ -572,6 +742,13 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                 number_of_images += fimages;
                 nfiles++;
             }
+
+            // The master says where its images are. If none of them could be read, that is a
+            // broken dataset, not an empty one - processing it would end in a "no images"
+            // warning and a successful exit over a sweep that is sitting right there.
+            if (number_of_images == 0)
+                throw JFJochException(JFJochExceptionCategory::HDF5,
+                                      "Master file links to data files, but no images could be read from them");
         } else {
             image_size_x = master_file->GetInt("/entry/instrument/detector/detectorSpecific/x_pixels_in_detector");
             image_size_y = master_file->GetInt("/entry/instrument/detector/detectorSpecific/y_pixels_in_detector");
@@ -598,12 +775,30 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             master_file->GetOptFloat("/entry/instrument/detector/transformations/rot2").value_or(0.0));
         dataset->experiment.PoniRot3_rad(
             master_file->GetOptFloat("/entry/instrument/detector/transformations/rot3").value_or(0.0));
+        // A detector swung out on a 2theta arm - routine in chemical crystallography - and any other
+        // rotation the file puts in the detector's chain. It turns the detector about the sample, so
+        // it carries the whole square-on geometry with it and composes on the left of the PONI
+        // rotation the file states directly.
+        if (const auto chain = ReadDetectorRotationChain(master_file.get())) {
+            float rot1 = 0, rot2 = 0, rot3 = 0;
+            PoniAnglesFromMatrix(chain.value()
+                                     * PoniRotMatrix(dataset->experiment.GetPoniRot1_rad(),
+                                                     dataset->experiment.GetPoniRot2_rad(),
+                                                     dataset->experiment.GetPoniRot3_rad()),
+                                 rot1, rot2, rot3);
+            dataset->experiment.PoniRot1_rad(rot1).PoniRot2_rad(rot2).PoniRot3_rad(rot3);
+            Logger("HDF5Reader").Info("Detector placed by its NXmx transformation chain: "
+                                      "rot1 {:.5f} rot2 {:.5f} rot3 {:.5f} rad", rot1, rot2, rot3);
+        }
         dataset->experiment.SampleTemperature_K(master_file->GetOptFloat("/entry/sample/temperature"));
 
         dataset->experiment.BeamX_pxl(master_file->GetFloat("/entry/instrument/detector/beam_center_x"));
         dataset->experiment.BeamY_pxl(master_file->GetFloat("/entry/instrument/detector/beam_center_y"));
 
-        float det_distance = master_file->GetFloat("/entry/instrument/detector/distance");
+        float det_distance = ReadLengthWithFallback_m(*master_file,
+                                                      {"/entry/instrument/detector/distance",
+                                                       "/entry/instrument/detector/detector_distance",
+                                                       "/entry/instrument/detector_distance"});
         if (det_distance < 0.001)
             det_distance = 0.1; // Set to 100 mm, if det distance is less than 1 mm
         dataset->experiment.DetectorDistance_mm(det_distance * 1000.0);
@@ -630,13 +825,14 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         // the CBOR stream and the writer, so looking only for "omega" - as this did - read a sweep
         // recorded as "phi" back as stills, silently. Prefer an axis that actually turns; fall back
         // to a stationary one, which still says where the head was.
-        if (master_file->Exists("/entry/sample/transformations")) {
+        const std::string gonio_group = GoniometerGroup(*master_file);
+        if (!gonio_group.empty()) {
             // A Smargon chi/phi is tagged with equipment_component - it is a head position, not the
             // spindle. Recognised by that tag and not by name: phi is an ordinary spindle name in MX,
             // so a file from anywhere else must not have its rotation axis read back as a head
             // position, nor its spindle mistaken for one here.
-            auto is_smargon_axis = [this](const std::string &name) {
-                const std::string dname = "/entry/sample/transformations/" + name;
+            auto is_smargon_axis = [this, &gonio_group](const std::string &name) {
+                const std::string dname = gonio_group + "/" + name;
                 if (!master_file->Exists(dname))
                     return false;
                 HDF5DataSet axis(*master_file, dname);
@@ -645,10 +841,10 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             };
 
             std::optional<GoniometerAxis> stationary;
-            for (const auto &name: master_file->FindLeafs("/entry/sample/transformations")) {
+            for (const auto &name: master_file->FindLeafs(gonio_group)) {
                 if (is_smargon_axis(name))
                     continue;
-                auto axis = ReadAxis(master_file.get(), name);
+                auto axis = ReadAxis(master_file.get(), name, gonio_group);
                 if (!axis.has_value())
                     continue;
                 if (axis->IsScanning()) {
@@ -667,9 +863,9 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             // the head position entirely - nothing in reader/ read it.
             std::optional<GoniometerAxis> chi, phi;
             if (is_smargon_axis("chi"))
-                chi = ReadAxis(master_file.get(), "chi");
+                chi = ReadAxis(master_file.get(), "chi", gonio_group);
             if (is_smargon_axis("phi"))
-                phi = ReadAxis(master_file.get(), "phi");
+                phi = ReadAxis(master_file.get(), "phi", gonio_group);
             if (chi.has_value() || phi.has_value()) {
                 SmargonPosition smargon;
                 if (chi.has_value()) {
@@ -708,7 +904,12 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                 .beta = tmp[4],
                 .gamma = tmp[5]
             });
-        dataset->experiment.SpaceGroupNumber(master_file->GetOptInt("/entry/sample/space_group_number"));
+        // The name carries the setting, the number cannot ("R 3:R" reads back as "R 3:H"), so the
+        // name is preferred; the number is the fallback for a file written before it was recorded.
+        if (const auto *sg = gemmi::find_spacegroup_by_name(master_file->GetString("/entry/sample/space_group")))
+            dataset->experiment.SetSpaceGroup(*sg);
+        else
+            dataset->experiment.SpaceGroupNumber(master_file->GetOptInt("/entry/sample/space_group_number"));
         // The setting the cell and space group just read are in, relative to the setting the per-image
         // reflections and lattices were written in. Absent on every file written before the offline
         // analysis started recording it, and on every run that never re-seated its lattice - both mean
@@ -727,6 +928,19 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             total_flux.reset();   // negative value is an "unknown flux" sentinel; treat as absent
         dataset->experiment.TotalFlux(total_flux);
 
+        // NXmx incident_beam_size is one two-element array in the order x, y; the settings hold
+        // it as two lengths in micrometres.
+        if (master_file->Exists("/entry/instrument/beam/incident_beam_size")) {
+            HDF5DataSet beam_size(*master_file, "/entry/instrument/beam/incident_beam_size");
+            std::vector<float> size;
+            beam_size.ReadVector(size);
+            if (size.size() == 2) {
+                const float to_um = LengthUnitFactor(beam_size, "incident_beam_size") * 1e6f;
+                dataset->experiment.BeamSizeX_um(size[0] * to_um);
+                dataset->experiment.BeamSizeY_um(size[1] * to_um);
+            }
+        }
+
         if (master_file->Exists("/entry/azint") && master_file->Exists("/entry/azint/bin_to_q")) {
             HDF5DataSet bin_to_q_dataset(*master_file, "/entry/azint/bin_to_q");
             HDF5DataSpace bin_to_q_dataspace(bin_to_q_dataset);
@@ -765,20 +979,53 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         auto detector_name = master_file->GetString("/entry/instrument/detector/description");
 
         DetectorSetup detector = DetDECTRIS(image_size_x, image_size_y, detector_name, {});
-        detector.PixelSize_um(master_file->GetFloat("/entry/instrument/detector/x_pixel_size") * 1e6);
+        detector.PixelSize_um(ReadLength_m(*master_file, "/entry/instrument/detector/x_pixel_size") * 1e6);
         // Whether the stored image is mirrored in Y. A file written before this was recorded is
         // mirrored - that is the only thing Jungfraujoch has ever produced - so absence means true.
         detector.MirrorY(master_file
                          ->GetOptBool("/entry/instrument/detector/detectorSpecific/mirror_y")
                          .value_or(true));
+        // How the stored image sits in the detector plane. A different setting from mirror_y above,
+        // recorded separately by the writer; absence means the identity, which is what a file written
+        // before it existed - or by anything else - describes.
+        // NXmx states this in the module's pixel directions, which is where it is read from first;
+        // detectorSpecific carries the same setting for a file this system wrote, and is the fallback
+        // for one whose module group says nothing usable. Absence of both means the identity.
+        detector.ImageOrientation(ReadModuleOrientation(master_file.get()).value_or(
+                DetectorOrientation(
+                        master_file->GetOptBool(
+                                "/entry/instrument/detector/detectorSpecific/detector_orientation_mirror_y")
+                                .value_or(false),
+                        master_file->GetOptInt(
+                                "/entry/instrument/detector/detectorSpecific/detector_orientation_quarter_turns")
+                                .value_or(0))));
         // Sensor thickness/material drive the parallax/absorption model, so take them from the file
-        // rather than the DetectorSetup default (NXmx stores thickness in metres).
+        // rather than the DetectorSetup default.
         if (master_file->Exists("/entry/instrument/detector/sensor_thickness"))
-            detector.SensorThickness_um(master_file->GetFloat("/entry/instrument/detector/sensor_thickness") * 1e6);
+            detector.SensorThickness_um(
+                ReadLength_m(*master_file, "/entry/instrument/detector/sensor_thickness") * 1e6);
         if (master_file->Exists("/entry/instrument/detector/sensor_material"))
             detector.SensorMaterial(master_file->GetString("/entry/instrument/detector/sensor_material"));
-        detector.SaturationLimit(SaturationLimitFromValue(
-                master_file->GetInt("/entry/instrument/detector/saturation_value")));
+        // Optional, because a file that states no saturation value anywhere is a real and common
+        // thing: an Eiger master links saturation_value into a companion _meta.h5, and a deposited
+        // dataset frequently does not include that file, leaving neither the NXmx name nor the
+        // DECTRIS one readable. Left unset, DiffractionExperiment::GetSaturationLimit() falls back
+        // to the container's own overflow, which is the safe direction - it can only fail to call a
+        // pixel saturated, where too LOW a value drops the whole reflection and silently removes the
+        // strongest data (see BitDepthImage below). Refusing the file outright is the one option that
+        // helps nobody.
+        if (master_file->Exists("/entry/instrument/detector/saturation_value")
+            || master_file->Exists("/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff"))
+            detector.SaturationLimit(SaturationLimitFromValue(
+                    ReadIntWithLegacyFallback(*master_file,
+                            "/entry/instrument/detector/saturation_value",
+                            "/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff")));
+        else
+            Logger("HDF5Reader").Warning("The file states no saturation value - neither NXmx saturation_value nor "
+                           "the DECTRIS countrate_correction_count_cutoff is readable, which is what "
+                           "an Eiger master looks like when its companion _meta.h5 was not kept. No "
+                           "pixel will be called saturated; if this detector overloads, its strongest "
+                           "reflections will be integrated as if they were valid.");
         // The reader hands every image out as signed int32 whatever the file stored (see PixelSigned
         // below), so that is the container depth the rest of the code has to see. DetectorSetup defaults
         // DECTRIS to 16 bits and GetByteDepthImage() prefers the detector's value over the image
@@ -795,13 +1042,19 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         detector.ReadOutTime(std::chrono::nanoseconds(0));
         dataset->experiment.Detector(detector);
 
+        // frame_time is the period between frames, count_time the exposure within one. NXmx requires
+        // neither, and a master written outside the DECTRIS toolchain often carries only count_time;
+        // falling back to it says "no dead time", which is the honest reading of a file that does not
+        // state one. What is read here is metadata - the one place frame time is divided by is the
+        // JUNGFRAU summation, which a dataset read from a DECTRIS-style file never reaches.
+        const float count_time_s = master_file->GetFloat("/entry/instrument/detector/count_time");
         dataset->experiment.FrameTime(
             std::chrono::duration_cast<std::chrono::nanoseconds>(
                 std::chrono::duration<float>(
-                    master_file->GetFloat("/entry/instrument/detector/frame_time"))),
+                    master_file->GetOptFloat("/entry/instrument/detector/frame_time")
+                              .value_or(count_time_s))),
             std::chrono::duration_cast<std::chrono::nanoseconds>(
-                std::chrono::duration<float>(
-                    master_file->GetFloat("/entry/instrument/detector/count_time")))
+                std::chrono::duration<float>(count_time_s))
         );
 
         if (master_file->Exists("/entry/instrument/detector/calibration")) {
@@ -810,17 +1063,15 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         }
 
         if (image_size_x * image_size_y > 0) {
-            auto mask_tmp = master_file->ReadOptVector<uint32_t>(
-                "/entry/instrument/detector/pixel_mask",
-                {0, 0},
-                {image_size_y, image_size_x}
-            );
-            if (mask_tmp.empty())
-                mask_tmp = master_file->ReadOptVector<uint32_t>(
-                    "/entry/instrument/detector/detectorSpecific/pixel_mask",
-                    {0, 0},
-                    {image_size_y, image_size_x}
-                );
+            // IsDataSet, not Exists: a mask can be an external link into a file that does not hold
+            // it, which a deposition shipping no mask at all leaves behind. The link is there, so
+            // the name exists; only opening it says whether the array does.
+            std::vector<uint32_t> mask_tmp;
+            for (const char *name: {"/entry/instrument/detector/pixel_mask",
+                                    "/entry/instrument/detector/detectorSpecific/pixel_mask"}) {
+                if (mask_tmp.empty() && master_file->IsDataSet(name))
+                    mask_tmp = master_file->ReadVector<uint32_t>(name, {0, 0}, {image_size_y, image_size_x});
+            }
             if (mask_tmp.empty())
                 mask_tmp = std::vector<uint32_t>(image_size_x * image_size_y);
             dataset->pixel_mask = std::make_shared<const PixelMask>(mask_tmp);
@@ -829,9 +1080,14 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         ReadROIMetadata(*master_file, *dataset);
 
         // Resolve VDS mapping filenames to absolute paths so the image source's locator only ever
-        // deals with real paths, then report the layout to the caller.
+        // deals with real paths, then report the layout to the caller. "." is HDF5's spelling for
+        // "the file this dataset is in", not a relative path - a master is allowed to compose its
+        // VDS over datasets in ITSELF, which are then external links to the data files. Resolved as
+        // a path it became <dir>/. and no image could be opened at all.
         for (auto &m : vds_data_mappings)
-            m.filename = ResolveRelativeToMaster(master_file_directory, m.filename);
+            m.filename = (m.filename == ".")
+                         ? master_filename
+                         : ResolveRelativeToMaster(master_file_directory, m.filename);
 
         dataset->experiment.ImagesPerTrigger(number_of_images);
         cached_geom = dataset->experiment.GetDiffractionGeometry();
@@ -1141,10 +1397,12 @@ void HDF5MetadataSource::FillPerImage(DataMessage &message, int64_t requested_im
     }
 }
 
-std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, const std::string &name) {
-    std::string dname = "/entry/sample/transformations/" + name;
+std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, const std::string &name,
+                                                           const std::string &group) {
+    std::string dname = group + "/" + name;
 
-    if (!file->Exists(dname))
+    // Not a dataset, not an axis: a hybrid file keeps a bare subgroup here for the direction alone.
+    if (!file->IsDataSet(dname))
         return {};
 
 
@@ -1160,9 +1418,29 @@ std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, con
     // Missing attribute means "not a transformation", so skip it rather than throwing: the search
     // for the goniometer walks every leaf and only stops early on an axis that turns, so a master
     // whose axis was stationary reached omega_end and could not be opened at all.
-    if (!dataset.AttrExists("transformation_type")
-            || (dataset.ReadAttrStr("transformation_type") != "rotation"))
+    // NXmx tags every axis; DECTRIS firmware 1.x tagged none of them, so absence has to mean two
+    // different things depending on the layout. In a transformations group it means "not an axis"
+    // (the writer's own AXISNAME_end and the rotation-width scalars live there and carry only units),
+    // and skipping is right. In the legacy goniometer group EVERY leaf is an axis and none is tagged,
+    // so skipping there would find no goniometer at all and the sweep would be read as stills.
+    const bool legacy_group = (group != "/entry/sample/transformations");
+    if (dataset.AttrExists("transformation_type")) {
+        if (dataset.ReadAttrStr("transformation_type") != "rotation")
+            return {};
+    } else if (!legacy_group) {
         return {};
+    } else {
+        // The same companion datasets, recognised by name because there is no tag to go on. In the
+        // legacy layout each axis NAME carries five of them - AXIS_end, _start, _increment,
+        // _range_average, _range_total - and only the bare name is the axis itself. Matching the
+        // suffix rather than "contains an underscore" keeps a genuine two_theta axis readable.
+        static const char *const companions[] = {"_end", "_start", "_increment",
+                                                 "_range_average", "_range_total"};
+        for (const char *suffix: companions)
+            if (name.size() > strlen(suffix)
+                    && name.compare(name.size() - strlen(suffix), strlen(suffix), suffix) == 0)
+                return {};
+    }
 
     std::vector<double> end = file->ReadOptVector<double>(dname + "_end");
 
@@ -1172,7 +1450,32 @@ std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, con
     double start = angle[0];
     double incr = (angle.size() < 2) ? 0.0 : angle[1] - angle[0];
 
-    std::vector<double> axis_vec = dataset.ReadAttrVec("vector");
+    std::vector<double> axis_vec;
+    if (dataset.AttrExists("vector")) {
+        axis_vec = dataset.ReadAttrVec("vector");
+    } else if (legacy_group) {
+        // The angles carry no direction here, but a hybrid file still states one next door: an
+        // NXmx-shaped subgroup /entry/sample/transformations/AXIS, holding the vector attribute
+        // and no angles. That is the file speaking, so it beats the assumption below.
+        const std::string nxmx_axis = "/entry/sample/transformations/" + name;
+        if (file->Exists(nxmx_axis) && !file->IsDataSet(nxmx_axis)) {
+            HDF5Group nxmx_group(*file, nxmx_axis);
+            if (nxmx_group.AttrExists("vector"))
+                axis_vec = nxmx_group.ReadAttrVec("vector");
+        }
+        if (axis_vec.empty()) {
+            // Firmware 1.x stored no direction at all. Assume the one every DECTRIS master since has
+            // written, and say so - a wrong guess here does not index, so it is visible rather than
+            // silent, and the rotation first pass will try the opposite sign anyway.
+            axis_vec = {-1.0, 0.0, 0.0};
+            Logger("HDF5Reader").Warning("{} carries no axis direction (pre-NXmx layout); assuming "
+                                         "(-1,0,0), the direction current DECTRIS masters write", dname);
+        }
+    } else {
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              dname + " has no vector attribute");
+    }
+
     if (axis_vec.size() != 3)
         throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
                               dname + " Vector must have 3 elements");