v1.0.0-rc.166 (#76)

* `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands.
* `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion.
* Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants.
* `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing.
* A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed.
* `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing.
* Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences.
* The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to.
* The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after.
* The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution.
* `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have.
* Twinning is no longer reported when the L-test contradicts it.
* The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's.
* `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots.
* The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area.

Reviewed-on: #76
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>

reader/CMakeLists.txt

@@ -2,6 +2,10 @@ ADD_LIBRARY(JFJochReader STATIC
         JFJochReader.cpp JFJochReader.h
         JFJochHDF5Reader.cpp
         JFJochHDF5Reader.h
+        MiniCBF.cpp
+        MiniCBF.h
+        JFJochCBFReader.cpp
+        JFJochCBFReader.h
         HDF5ImageLocator.cpp
         HDF5ImageLocator.h
         HDF5ImageSource.cpp

reader/HDF5ImageLocator.cpp

@@ -63,8 +63,8 @@ HDF5ImageLocator::Location HDF5ImageLocator::Resolve(int64_t global_image) const
     if (layout_.format == FileWriterFormat::NXmxLegacy) {
         const uint32_t file_id = global_image / layout_.images_per_file;
         const uint32_t local_index = global_image % layout_.images_per_file;
-        const auto &path = layout_.legacy_files.at(file_id);
-        return {OpenCached(path), local_index, path};
+        const auto &data_file = layout_.legacy_files.at(file_id);
+        return {OpenCached(data_file.path), local_index, data_file.path, data_file.dataset};
     }
 
     if (layout_.format == FileWriterFormat::NXmxVDS
@@ -74,7 +74,7 @@ HDF5ImageLocator::Location HDF5ImageLocator::Resolve(int64_t global_image) const
             if (!mapping.ContainsVirtualImage(image))
                 continue;
             return {OpenCached(mapping.filename), static_cast<uint32_t>(mapping.SourceImage(image)),
-                    mapping.filename};
+                    mapping.filename, mapping.dataset};
         }
         throw JFJochException(JFJochExceptionCategory::HDF5,
                               "Image not covered by /entry/data/data VDS mappings");
@@ -161,7 +161,8 @@ std::vector<HDF5DataSourceMessage> HDF5ImageLocator::GetSourceMapping(uint64_t f
                 throw JFJochException(JFJochExceptionCategory::HDF5,
                                       "Legacy image source file missing");
 
-            AppendOrExtendSourceMapping(ret, layout_.legacy_files.at(file_id), "/entry/data/data",
+            const auto &data_file = layout_.legacy_files.at(file_id);
+            AppendOrExtendSourceMapping(ret, data_file.path, data_file.dataset,
                                         source_image, local_image, 1);
         }
 

reader/HDF5ImageLocator.h

@@ -28,6 +28,16 @@ public:
         // Path the file was opened from. Needed to open it a second time as a plain file, for the
         // positional reads HDF5ImageSource does outside the mutex.
         std::string path;
+        // Where the images sit INSIDE that file. /entry/data/data everywhere DECTRIS writes, but a
+        // VDS names its source dataset and is free to name another one, so take it at its word.
+        std::string dataset = "/entry/data/data";
+    };
+
+    // One data file of a legacy multi-file dataset, with the dataset the master's link names
+    // inside it - the same "take the link at its word" the VDS branch already does.
+    struct LegacyFile {
+        std::string path;
+        std::string dataset;
     };
 
     // Layout description, filled by the reader once the master file has been parsed. All paths
@@ -38,7 +48,7 @@ public:
         HDF5DataSetLayout data_layout = HDF5DataSetLayout::CONTIGUOUS;
         std::shared_ptr<HDF5ReadOnlyFile> master_file;
         std::string master_filename;
-        std::vector<std::string> legacy_files;
+        std::vector<LegacyFile> legacy_files;
         size_t images_per_file = 1;
         std::vector<HDF5VirtualDatasetMapping> vds_mappings;
     };

reader/HDF5ImageSource.cpp

@@ -71,7 +71,7 @@ HDF5ImageLocator::Location HDF5ImageSource::Resolve(int64_t global) const {
 
 StoredPixelFormat HDF5ImageSource::GetStoredPixelFormat() const {
     auto loc = locator_.Resolve(0);
-    HDF5DataSet dataset(*loc.file, "/entry/data/data");
+    HDF5DataSet dataset(*loc.file, loc.dataset);
     HDF5DataType datatype(dataset);
     return {static_cast<int64_t>(datatype.GetElemSize()) * 8, datatype.IsSigned()};
 }
@@ -85,12 +85,13 @@ std::vector<HDF5DataSourceMessage> HDF5ImageSource::GetSourceMapping(uint64_t fi
 
 const HDF5ImageSource::OpenDataset &
 HDF5ImageSource::GetDataset(const HDF5ImageLocator::Location &loc) const {
-    if (auto it = dataset_cache_.find(loc.file.get()); it != dataset_cache_.end())
+    auto key = std::make_pair(loc.file.get(), loc.dataset);
+    if (auto it = dataset_cache_.find(key); it != dataset_cache_.end())
         return it->second;
 
     OpenDataset entry;
     entry.file = loc.file;
-    entry.dataset = std::make_unique<HDF5DataSet>(*loc.file, "/entry/data/data");
+    entry.dataset = std::make_unique<HDF5DataSet>(*loc.file, loc.dataset);
 
     HDF5DataSpace dataspace(*entry.dataset);
     HDF5DataType datatype(*entry.dataset);
@@ -98,7 +99,7 @@ HDF5ImageSource::GetDataset(const HDF5ImageLocator::Location &loc) const {
 
     if (dataspace.GetNumOfDimensions() != 3)
         throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
-                              "/entry/data/data dataset must be 3D");
+                              loc.dataset + " dataset must be 3D");
 
     if (datatype.IsFloat())
         throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
@@ -115,20 +116,37 @@ HDF5ImageSource::GetDataset(const HDF5ImageLocator::Location &loc) const {
     if (entry.direct_chunk)
         entry.algorithm = dcpl.GetCompression();
 
-    if (entry.direct_chunk && !loc.path.empty()) {
-        entry.raw = std::make_shared<RawFile>(loc.path);
-        if (!entry.raw->IsOpen())
-            entry.raw.reset();
-        hid_t fcpl = H5Fget_create_plist(loc.file->GetID());
-        if (fcpl >= 0) {
-            hsize_t user_block = 0;
-            if (H5Pget_userblock(fcpl, &user_block) >= 0)
-                entry.user_block = user_block;
-            H5Pclose(fcpl);
+    if (entry.direct_chunk) {
+        // A positional read needs the file the PIXELS are in, and that is not always the file that
+        // was opened: a dataset reached through an external link lives elsewhere, and a chunk
+        // address is an offset into ITS file - as is the user block the address counts from. Ask
+        // HDF5 which file the dataset ended up in rather than assuming it is the one we asked.
+        std::string path = loc.path;
+        const hid_t owner = H5Iget_file_id(entry.dataset->GetID());
+        if (owner >= 0) {
+            const ssize_t name_size = H5Fget_name(owner, nullptr, 0);
+            if (name_size > 0) {
+                std::string name(static_cast<size_t>(name_size), '\0');
+                if (H5Fget_name(owner, name.data(), name.size() + 1) >= 0)
+                    path = name;
+            }
+            const hid_t fcpl = H5Fget_create_plist(owner);
+            if (fcpl >= 0) {
+                hsize_t user_block = 0;
+                if (H5Pget_userblock(fcpl, &user_block) >= 0)
+                    entry.user_block = user_block;
+                H5Pclose(fcpl);
+            }
+            H5Fclose(owner);
+        }
+        if (!path.empty()) {
+            entry.raw = std::make_shared<RawFile>(path);
+            if (!entry.raw->IsOpen())
+                entry.raw.reset();
         }
     }
 
-    return dataset_cache_.emplace(loc.file.get(), std::move(entry)).first->second;
+    return dataset_cache_.emplace(std::move(key), std::move(entry)).first->second;
 }
 
 std::optional<HDF5ImageSource::DirectChunk>

reader/HDF5ImageSource.h

@@ -7,6 +7,8 @@
 #include <map>
 #include <memory>
 #include <optional>
+#include <string>
+#include <utility>
 #include <vector>
 
 #include "HDF5ImageLocator.h"
@@ -126,6 +128,8 @@ private:
         CompressionAlgorithm algorithm = CompressionAlgorithm::NO_COMPRESSION;
         bool direct_chunk = false;
     };
-    mutable std::map<const HDF5ReadOnlyFile *, OpenDataset> dataset_cache_;
+    // Keyed by file AND dataset path: a master whose VDS sources are datasets in itself serves
+    // several of them out of one file, and keying by file alone would hand back the wrong one.
+    mutable std::map<std::pair<const HDF5ReadOnlyFile *, std::string>, OpenDataset> dataset_cache_;
     const OpenDataset &GetDataset(const HDF5ImageLocator::Location &loc) const;
 };

reader/HDF5MetadataSource.cpp

@@ -2,6 +2,7 @@
 // SPDX-License-Identifier: GPL-3.0-only
 
 #include <cmath>
+#include <cstring>
 #include <set>
 
 #include "HDF5MetadataSource.h"
@@ -10,8 +11,98 @@
 #include "../image_analysis/spot_finding/SpotUtils.h"
 #include "../common/GridScanSettings.h"
 #include "../common/JFJochMath.h"
+#include "../common/Logger.h"
 #include "../common/ROIDefinition.h"
 
+// A McStas direction in the internal frame. The two differ by a 180 degree turn about z, which is a
+// rotation and not a mirror - so an axis carried through it turns the same way by the same angle.
+static Coord McStasToInternal(const std::vector<double> &v) {
+    return {static_cast<float>(-v[0]), static_cast<float>(-v[1]), static_cast<float>(v[2])};
+}
+
+// The image orientation the file itself states, in its NXdetector_module pixel directions. NXmx gives
+// those in the McStas frame.
+//
+// Only an exact match against one of the eight discrete orientations is taken. Anything else is a
+// continuous rotation of the detector in its own plane, which belongs in rot1/rot2/rot3 and cannot be
+// separated from the tilt by looking at the module alone - so it is left as it is rather than
+// approximated. Every real file examined here is exactly discrete.
+static std::optional<DetectorOrientation> ReadModuleOrientation(HDF5Object *file) {
+    const std::string base = "/entry/instrument/detector/module/";
+    if (!file->IsDataSet(base + "fast_pixel_direction") || !file->IsDataSet(base + "slow_pixel_direction"))
+        return {};
+
+    HDF5DataSet fast_dataset(*file, base + "fast_pixel_direction");
+    HDF5DataSet slow_dataset(*file, base + "slow_pixel_direction");
+    if (!fast_dataset.AttrExists("vector") || !slow_dataset.AttrExists("vector"))
+        return {};
+
+    const auto f = fast_dataset.ReadAttrVec("vector");
+    const auto s = slow_dataset.ReadAttrVec("vector");
+    if ((f.size() != 3) || (s.size() != 3))
+        return {};
+
+    return DetectorOrientation::Match(McStasToInternal(f), McStasToInternal(s));
+}
+
+// Where the detector stands, from the chain of transformations the file says it depends on.
+//
+// NXmx has no field for a detector swung out on a 2theta arm. It states the detector's position as a
+// depends_on chain and the arm is one rotation in that chain, so following the chain is the only way
+// to find it: "two_theta" is one beamline's name for that dataset and the next spells it otherwise.
+//
+// Only the rotations are taken, composed from the detector outwards. Each transformation states its
+// vector in the frame of the one it depends on, so the product is the rotation that carries a
+// detector square to the beam to where this one stands. The translations in the chain are the
+// detector distance and the beam centre, which the file states separately in that square-on frame -
+// the arm turns the detector about the sample and moves neither, and a Diamond master writes the same
+// beam_center_x/y for a swung sweep as for the square-on one beside it. Nothing comes back when no
+// rotation in the chain turns, which is every detector square to the beam.
+static std::optional<RotMatrix> ReadDetectorRotationChain(HDF5Object *file) {
+    std::string node = file->GetString("/entry/instrument/detector/depends_on");
+    if (node.empty() && file->IsDataSet("/entry/instrument/detector/module/module_offset")) {
+        HDF5DataSet module_offset(*file, "/entry/instrument/detector/module/module_offset");
+        if (module_offset.AttrExists("depends_on"))
+            node = module_offset.ReadAttrStr("depends_on");
+    }
+
+    // A file this system wrote states its PONI angles in the chain as well, and they are read from
+    // these three paths just before this is called. Taking them here too would apply the tilt twice.
+    static const std::set<std::string> poni_angles = {"/entry/instrument/detector/transformations/rot1",
+                                                      "/entry/instrument/detector/transformations/rot2",
+                                                      "/entry/instrument/detector/transformations/rot3"};
+
+    RotMatrix chain;
+    bool turns = false;
+    std::set<std::string> seen;
+    while ((node != ".") && !node.empty() && file->IsDataSet(node) && seen.insert(node).second) {
+        HDF5DataSet axis(*file, node);
+        const std::string current = node;
+        node = axis.AttrExists("depends_on") ? axis.ReadAttrStr("depends_on") : ".";
+
+        if (poni_angles.contains(current) || !axis.AttrExists("transformation_type") || !axis.AttrExists("vector")
+                || (axis.ReadAttrStr("transformation_type") != "rotation"))
+            continue;
+
+        std::vector<double> value;
+        axis.ReadVector(value);
+        const auto vec = axis.ReadAttrVec("vector");
+        if (value.empty() || (value[0] == 0.0) || (vec.size() != 3))
+            continue;
+
+        // NXmx states a rotation in degrees unless it says otherwise.
+        const bool radians = axis.AttrExists("units") && (axis.ReadAttrStr("units") == "rad");
+        const auto angle_rad = static_cast<float>(radians ? value[0] : value[0] * PI / 180.0);
+        chain = RotMatrix(angle_rad, McStasToInternal(vec)) * chain;
+        turns = true;
+    }
+
+    if (!turns)
+        return {};
+    return chain;
+}
+
+
 inline std::pair<gemmi::CrystalSystem, char> parse_bravais_lattice(const std::string &val) {
     if (val.empty())
         return {gemmi::CrystalSystem::Triclinic, 'P'};
@@ -65,7 +156,7 @@ inline std::pair<gemmi::CrystalSystem, char> parse_bravais_lattice(const std::st
 std::vector<hsize_t> GetDimension(HDF5Object &object, const std::string &path) {
     const auto dim = object.GetDimension(path);
     if (dim.size() != 3)
-        throw JFJochException(JFJochExceptionCategory::HDF5, "Wrong dimension of /entry/data/data");
+        throw JFJochException(JFJochExceptionCategory::HDF5, "Wrong dimension of " + path);
     return dim;
 }
 
@@ -100,6 +191,74 @@ std::string ResolveRelativeToMaster(const std::string &directory,
     return (std::filesystem::path(directory) / path).string();
 }
 
+// A length in NXmx says which unit it is in, and a master written outside the DECTRIS toolchain
+// uses that freedom: a Diamond-written one states pixel size, sensor thickness and detector
+// distance in millimetres, which is correct NXmx. Reading those as metres is not a failure but a
+// silent factor of a thousand, so the unit is read rather than assumed. An undeclared unit means
+// metres - what every DECTRIS master and everything this system writes means by one. An unknown
+// unit is refused rather than guessed at, for the same reason.
+float LengthUnitFactor(HDF5DataSet &dataset, const std::string &name) {
+    if (!dataset.AttrExists("units"))
+        return 1.0f;
+
+    const std::string units = dataset.ReadAttrStr("units");
+    if (units == "m")
+        return 1.0f;
+    if (units == "mm")
+        return 1e-3f;
+    if (units == "um")
+        return 1e-6f;
+    throw JFJochException(JFJochExceptionCategory::HDF5, name + ": unknown length unit " + units);
+}
+
+float ReadLength_m(HDF5Object &file, const std::string &name) {
+    HDF5DataSet dataset(file, name);
+    return dataset.ReadScalar<float>() * LengthUnitFactor(dataset, name);
+}
+
+// The same value under different names. DECTRIS Eiger firmware 1.x writes detector_distance where
+// NXmx says distance, and a file from that era is still what a repository hands you; a Diamond
+// master puts the distance one level up, in NXinstrument rather than in NXdetector. The NXmx
+// spelling is tried first and the rest are pure fallbacks, which is safe because every current
+// Eiger master carries BOTH DECTRIS names (measured on thirteen masters from ten facilities,
+// firmware release-2020.2.1 through release-2024.1.1 - all of them write detector_distance and
+// countrate_correction_count_cutoff beside the NXmx names). So this can never change what a
+// modern file reads. ReadIntWithLegacyFallback below is the same arrangement for saturation_value.
+float ReadLengthWithFallback_m(HDF5Object &file, const std::vector<std::string> &names) {
+    for (const auto &name: names)
+        if (file.Exists(name))
+            return ReadLength_m(file, name);
+    throw JFJochException(JFJochExceptionCategory::HDF5, "Cannot find " + names.front());
+}
+
+int64_t ReadIntWithLegacyFallback(HDF5Object &file, const std::string &nxmx, const std::string &legacy) {
+    if (file.Exists(nxmx))
+        return file.GetInt(nxmx);
+    if (file.Exists(legacy))
+        return file.GetInt(legacy);
+    throw JFJochException(JFJochExceptionCategory::HDF5, "Cannot find " + nxmx + " (nor " + legacy + ")");
+}
+
+// Where the goniometer axes live. NXmx puts them in /entry/sample/transformations; firmware 1.x put
+// them in /entry/sample/goniometer and wrote no transformation_type and no vector on them. Getting
+// this one wrong is not a missing value but a WRONG ANSWER: a goniometer is only ever set from this
+// group, so a file whose axes are somewhere else is read as stills, silently.
+//
+// A hybrid file has both: an NXmx transformations group holding one empty subgroup per axis, which
+// states the direction and nothing else, beside a legacy goniometer group holding all the angles.
+// So present is not the same as usable - transformations is the angle source only if it holds an
+// axis dataset, and an axis is always a dataset.
+std::string GoniometerGroup(HDF5Object &file) {
+    if (file.Exists("/entry/sample/transformations")) {
+        for (const auto &name: file.FindLeafs("/entry/sample/transformations"))
+            if (file.IsDataSet("/entry/sample/transformations/" + name))
+                return "/entry/sample/transformations";
+    }
+    if (file.Exists("/entry/sample/goniometer"))
+        return "/entry/sample/goniometer";
+    return {};
+}
+
 template<class T>
 void ReadVector(std::vector<T> &v,
                 HDF5Object &file,
@@ -335,7 +494,7 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         // at the end. format stays NoFile if the master carries no image data.
         FileWriterFormat format = FileWriterFormat::NoFile;
         HDF5DataSetLayout data_layout = HDF5DataSetLayout::CONTIGUOUS;
-        std::vector<std::string> legacy_format_files;
+        std::vector<HDF5ImageLocator::LegacyFile> legacy_format_files;
         std::vector<HDF5VirtualDatasetMapping> vds_data_mappings;
         size_t images_per_file = 1;
 
@@ -430,11 +589,6 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
 
             legacy_format_files.clear();
 
-            image_size_x = master_file->GetInt("/entry/instrument/detector/detectorSpecific/x_pixels_in_detector");
-            image_size_y = master_file->GetInt("/entry/instrument/detector/detectorSpecific/y_pixels_in_detector");
-
-            //size_t expected_images = master_file->GetInt("/entry/instrument/detector/detectorSpecific/nimages");
-
             images_per_file = 0;
             number_of_images = 0;
             uint32_t nfiles = 0;
@@ -450,14 +604,26 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                 size_t fimages = 0;
 
                 try {
-                    auto fname = ResolveRelativeToMaster(directory.string(),
-                                                         master_file->GetLinkedFileName(dname));
+                    // The link names both the file and the dataset inside it. DECTRIS and this
+                    // system always call that dataset /entry/data/data, but a Diamond-written
+                    // master links to a plain /data - so looking up a fixed name found nothing,
+                    // and a full sweep read back as zero images.
+                    const auto [linked_file, data_path] = master_file->GetLinkedTarget(dname);
+                    const auto fname = ResolveRelativeToMaster(directory.string(), linked_file);
 
                     HDF5ReadOnlyFile data_file(fname);
 
-                    fimages = GetDimension(data_file, "/entry/data/data")[0];
+                    // The image size comes from the array itself, as it does for a VDS master.
+                    // detectorSpecific/x_pixels_in_detector is a DECTRIS extension rather than
+                    // NXmx, so a third-party writer has no reason to emit it.
+                    const auto dim = GetDimension(data_file, data_path);
+                    fimages = dim[0];
+                    if (nfiles == 0) {
+                        image_size_y = dim[1];
+                        image_size_x = dim[2];
+                    }
 
-                    legacy_format_files.push_back(fname);
+                    legacy_format_files.push_back({fname, data_path});
 
                     if (nfiles == 0 && data_file.Exists("/entry/roi"))
                         dataset->roi = data_file.FindLeafs("/entry/roi");
@@ -565,6 +731,10 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                                    number_of_images, fimages);
                     }
                 } catch (JFJochException &e) {
+                    // Say why. Everything read here is optional per-image metadata except the
+                    // image array itself, and losing that silently leaves an empty dataset that
+                    // reads as "nothing to process" rather than as a broken file.
+                    Logger("HDF5Reader").Warning("{}: {}", dname, e.what());
                 }
 
                 if (nfiles == 0)
@@ -572,6 +742,13 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                 number_of_images += fimages;
                 nfiles++;
             }
+
+            // The master says where its images are. If none of them could be read, that is a
+            // broken dataset, not an empty one - processing it would end in a "no images"
+            // warning and a successful exit over a sweep that is sitting right there.
+            if (number_of_images == 0)
+                throw JFJochException(JFJochExceptionCategory::HDF5,
+                                      "Master file links to data files, but no images could be read from them");
         } else {
             image_size_x = master_file->GetInt("/entry/instrument/detector/detectorSpecific/x_pixels_in_detector");
             image_size_y = master_file->GetInt("/entry/instrument/detector/detectorSpecific/y_pixels_in_detector");
@@ -598,12 +775,30 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             master_file->GetOptFloat("/entry/instrument/detector/transformations/rot2").value_or(0.0));
         dataset->experiment.PoniRot3_rad(
             master_file->GetOptFloat("/entry/instrument/detector/transformations/rot3").value_or(0.0));
+        // A detector swung out on a 2theta arm - routine in chemical crystallography - and any other
+        // rotation the file puts in the detector's chain. It turns the detector about the sample, so
+        // it carries the whole square-on geometry with it and composes on the left of the PONI
+        // rotation the file states directly.
+        if (const auto chain = ReadDetectorRotationChain(master_file.get())) {
+            float rot1 = 0, rot2 = 0, rot3 = 0;
+            PoniAnglesFromMatrix(chain.value()
+                                     * PoniRotMatrix(dataset->experiment.GetPoniRot1_rad(),
+                                                     dataset->experiment.GetPoniRot2_rad(),
+                                                     dataset->experiment.GetPoniRot3_rad()),
+                                 rot1, rot2, rot3);
+            dataset->experiment.PoniRot1_rad(rot1).PoniRot2_rad(rot2).PoniRot3_rad(rot3);
+            Logger("HDF5Reader").Info("Detector placed by its NXmx transformation chain: "
+                                      "rot1 {:.5f} rot2 {:.5f} rot3 {:.5f} rad", rot1, rot2, rot3);
+        }
         dataset->experiment.SampleTemperature_K(master_file->GetOptFloat("/entry/sample/temperature"));
 
         dataset->experiment.BeamX_pxl(master_file->GetFloat("/entry/instrument/detector/beam_center_x"));
         dataset->experiment.BeamY_pxl(master_file->GetFloat("/entry/instrument/detector/beam_center_y"));
 
-        float det_distance = master_file->GetFloat("/entry/instrument/detector/distance");
+        float det_distance = ReadLengthWithFallback_m(*master_file,
+                                                      {"/entry/instrument/detector/distance",
+                                                       "/entry/instrument/detector/detector_distance",
+                                                       "/entry/instrument/detector_distance"});
         if (det_distance < 0.001)
             det_distance = 0.1; // Set to 100 mm, if det distance is less than 1 mm
         dataset->experiment.DetectorDistance_mm(det_distance * 1000.0);
@@ -630,13 +825,14 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         // the CBOR stream and the writer, so looking only for "omega" - as this did - read a sweep
         // recorded as "phi" back as stills, silently. Prefer an axis that actually turns; fall back
         // to a stationary one, which still says where the head was.
-        if (master_file->Exists("/entry/sample/transformations")) {
+        const std::string gonio_group = GoniometerGroup(*master_file);
+        if (!gonio_group.empty()) {
             // A Smargon chi/phi is tagged with equipment_component - it is a head position, not the
             // spindle. Recognised by that tag and not by name: phi is an ordinary spindle name in MX,
             // so a file from anywhere else must not have its rotation axis read back as a head
             // position, nor its spindle mistaken for one here.
-            auto is_smargon_axis = [this](const std::string &name) {
-                const std::string dname = "/entry/sample/transformations/" + name;
+            auto is_smargon_axis = [this, &gonio_group](const std::string &name) {
+                const std::string dname = gonio_group + "/" + name;
                 if (!master_file->Exists(dname))
                     return false;
                 HDF5DataSet axis(*master_file, dname);
@@ -645,10 +841,10 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             };
 
             std::optional<GoniometerAxis> stationary;
-            for (const auto &name: master_file->FindLeafs("/entry/sample/transformations")) {
+            for (const auto &name: master_file->FindLeafs(gonio_group)) {
                 if (is_smargon_axis(name))
                     continue;
-                auto axis = ReadAxis(master_file.get(), name);
+                auto axis = ReadAxis(master_file.get(), name, gonio_group);
                 if (!axis.has_value())
                     continue;
                 if (axis->IsScanning()) {
@@ -667,9 +863,9 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             // the head position entirely - nothing in reader/ read it.
             std::optional<GoniometerAxis> chi, phi;
             if (is_smargon_axis("chi"))
-                chi = ReadAxis(master_file.get(), "chi");
+                chi = ReadAxis(master_file.get(), "chi", gonio_group);
             if (is_smargon_axis("phi"))
-                phi = ReadAxis(master_file.get(), "phi");
+                phi = ReadAxis(master_file.get(), "phi", gonio_group);
             if (chi.has_value() || phi.has_value()) {
                 SmargonPosition smargon;
                 if (chi.has_value()) {
@@ -708,7 +904,12 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
                 .beta = tmp[4],
                 .gamma = tmp[5]
             });
-        dataset->experiment.SpaceGroupNumber(master_file->GetOptInt("/entry/sample/space_group_number"));
+        // The name carries the setting, the number cannot ("R 3:R" reads back as "R 3:H"), so the
+        // name is preferred; the number is the fallback for a file written before it was recorded.
+        if (const auto *sg = gemmi::find_spacegroup_by_name(master_file->GetString("/entry/sample/space_group")))
+            dataset->experiment.SetSpaceGroup(*sg);
+        else
+            dataset->experiment.SpaceGroupNumber(master_file->GetOptInt("/entry/sample/space_group_number"));
         // The setting the cell and space group just read are in, relative to the setting the per-image
         // reflections and lattices were written in. Absent on every file written before the offline
         // analysis started recording it, and on every run that never re-seated its lattice - both mean
@@ -727,6 +928,19 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
             total_flux.reset();   // negative value is an "unknown flux" sentinel; treat as absent
         dataset->experiment.TotalFlux(total_flux);
 
+        // NXmx incident_beam_size is one two-element array in the order x, y; the settings hold
+        // it as two lengths in micrometres.
+        if (master_file->Exists("/entry/instrument/beam/incident_beam_size")) {
+            HDF5DataSet beam_size(*master_file, "/entry/instrument/beam/incident_beam_size");
+            std::vector<float> size;
+            beam_size.ReadVector(size);
+            if (size.size() == 2) {
+                const float to_um = LengthUnitFactor(beam_size, "incident_beam_size") * 1e6f;
+                dataset->experiment.BeamSizeX_um(size[0] * to_um);
+                dataset->experiment.BeamSizeY_um(size[1] * to_um);
+            }
+        }
+
         if (master_file->Exists("/entry/azint") && master_file->Exists("/entry/azint/bin_to_q")) {
             HDF5DataSet bin_to_q_dataset(*master_file, "/entry/azint/bin_to_q");
             HDF5DataSpace bin_to_q_dataspace(bin_to_q_dataset);
@@ -765,20 +979,53 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         auto detector_name = master_file->GetString("/entry/instrument/detector/description");
 
         DetectorSetup detector = DetDECTRIS(image_size_x, image_size_y, detector_name, {});
-        detector.PixelSize_um(master_file->GetFloat("/entry/instrument/detector/x_pixel_size") * 1e6);
+        detector.PixelSize_um(ReadLength_m(*master_file, "/entry/instrument/detector/x_pixel_size") * 1e6);
         // Whether the stored image is mirrored in Y. A file written before this was recorded is
         // mirrored - that is the only thing Jungfraujoch has ever produced - so absence means true.
         detector.MirrorY(master_file
                          ->GetOptBool("/entry/instrument/detector/detectorSpecific/mirror_y")
                          .value_or(true));
+        // How the stored image sits in the detector plane. A different setting from mirror_y above,
+        // recorded separately by the writer; absence means the identity, which is what a file written
+        // before it existed - or by anything else - describes.
+        // NXmx states this in the module's pixel directions, which is where it is read from first;
+        // detectorSpecific carries the same setting for a file this system wrote, and is the fallback
+        // for one whose module group says nothing usable. Absence of both means the identity.
+        detector.ImageOrientation(ReadModuleOrientation(master_file.get()).value_or(
+                DetectorOrientation(
+                        master_file->GetOptBool(
+                                "/entry/instrument/detector/detectorSpecific/detector_orientation_mirror_y")
+                                .value_or(false),
+                        master_file->GetOptInt(
+                                "/entry/instrument/detector/detectorSpecific/detector_orientation_quarter_turns")
+                                .value_or(0))));
         // Sensor thickness/material drive the parallax/absorption model, so take them from the file
-        // rather than the DetectorSetup default (NXmx stores thickness in metres).
+        // rather than the DetectorSetup default.
         if (master_file->Exists("/entry/instrument/detector/sensor_thickness"))
-            detector.SensorThickness_um(master_file->GetFloat("/entry/instrument/detector/sensor_thickness") * 1e6);
+            detector.SensorThickness_um(
+                ReadLength_m(*master_file, "/entry/instrument/detector/sensor_thickness") * 1e6);
         if (master_file->Exists("/entry/instrument/detector/sensor_material"))
             detector.SensorMaterial(master_file->GetString("/entry/instrument/detector/sensor_material"));
-        detector.SaturationLimit(SaturationLimitFromValue(
-                master_file->GetInt("/entry/instrument/detector/saturation_value")));
+        // Optional, because a file that states no saturation value anywhere is a real and common
+        // thing: an Eiger master links saturation_value into a companion _meta.h5, and a deposited
+        // dataset frequently does not include that file, leaving neither the NXmx name nor the
+        // DECTRIS one readable. Left unset, DiffractionExperiment::GetSaturationLimit() falls back
+        // to the container's own overflow, which is the safe direction - it can only fail to call a
+        // pixel saturated, where too LOW a value drops the whole reflection and silently removes the
+        // strongest data (see BitDepthImage below). Refusing the file outright is the one option that
+        // helps nobody.
+        if (master_file->Exists("/entry/instrument/detector/saturation_value")
+            || master_file->Exists("/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff"))
+            detector.SaturationLimit(SaturationLimitFromValue(
+                    ReadIntWithLegacyFallback(*master_file,
+                            "/entry/instrument/detector/saturation_value",
+                            "/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff")));
+        else
+            Logger("HDF5Reader").Warning("The file states no saturation value - neither NXmx saturation_value nor "
+                           "the DECTRIS countrate_correction_count_cutoff is readable, which is what "
+                           "an Eiger master looks like when its companion _meta.h5 was not kept. No "
+                           "pixel will be called saturated; if this detector overloads, its strongest "
+                           "reflections will be integrated as if they were valid.");
         // The reader hands every image out as signed int32 whatever the file stored (see PixelSigned
         // below), so that is the container depth the rest of the code has to see. DetectorSetup defaults
         // DECTRIS to 16 bits and GetByteDepthImage() prefers the detector's value over the image
@@ -795,13 +1042,19 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         detector.ReadOutTime(std::chrono::nanoseconds(0));
         dataset->experiment.Detector(detector);
 
+        // frame_time is the period between frames, count_time the exposure within one. NXmx requires
+        // neither, and a master written outside the DECTRIS toolchain often carries only count_time;
+        // falling back to it says "no dead time", which is the honest reading of a file that does not
+        // state one. What is read here is metadata - the one place frame time is divided by is the
+        // JUNGFRAU summation, which a dataset read from a DECTRIS-style file never reaches.
+        const float count_time_s = master_file->GetFloat("/entry/instrument/detector/count_time");
         dataset->experiment.FrameTime(
             std::chrono::duration_cast<std::chrono::nanoseconds>(
                 std::chrono::duration<float>(
-                    master_file->GetFloat("/entry/instrument/detector/frame_time"))),
+                    master_file->GetOptFloat("/entry/instrument/detector/frame_time")
+                              .value_or(count_time_s))),
             std::chrono::duration_cast<std::chrono::nanoseconds>(
-                std::chrono::duration<float>(
-                    master_file->GetFloat("/entry/instrument/detector/count_time")))
+                std::chrono::duration<float>(count_time_s))
         );
 
         if (master_file->Exists("/entry/instrument/detector/calibration")) {
@@ -810,17 +1063,15 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         }
 
         if (image_size_x * image_size_y > 0) {
-            auto mask_tmp = master_file->ReadOptVector<uint32_t>(
-                "/entry/instrument/detector/pixel_mask",
-                {0, 0},
-                {image_size_y, image_size_x}
-            );
-            if (mask_tmp.empty())
-                mask_tmp = master_file->ReadOptVector<uint32_t>(
-                    "/entry/instrument/detector/detectorSpecific/pixel_mask",
-                    {0, 0},
-                    {image_size_y, image_size_x}
-                );
+            // IsDataSet, not Exists: a mask can be an external link into a file that does not hold
+            // it, which a deposition shipping no mask at all leaves behind. The link is there, so
+            // the name exists; only opening it says whether the array does.
+            std::vector<uint32_t> mask_tmp;
+            for (const char *name: {"/entry/instrument/detector/pixel_mask",
+                                    "/entry/instrument/detector/detectorSpecific/pixel_mask"}) {
+                if (mask_tmp.empty() && master_file->IsDataSet(name))
+                    mask_tmp = master_file->ReadVector<uint32_t>(name, {0, 0}, {image_size_y, image_size_x});
+            }
             if (mask_tmp.empty())
                 mask_tmp = std::vector<uint32_t>(image_size_x * image_size_y);
             dataset->pixel_mask = std::make_shared<const PixelMask>(mask_tmp);
@@ -829,9 +1080,14 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
         ReadROIMetadata(*master_file, *dataset);
 
         // Resolve VDS mapping filenames to absolute paths so the image source's locator only ever
-        // deals with real paths, then report the layout to the caller.
+        // deals with real paths, then report the layout to the caller. "." is HDF5's spelling for
+        // "the file this dataset is in", not a relative path - a master is allowed to compose its
+        // VDS over datasets in ITSELF, which are then external links to the data files. Resolved as
+        // a path it became <dir>/. and no image could be opened at all.
         for (auto &m : vds_data_mappings)
-            m.filename = ResolveRelativeToMaster(master_file_directory, m.filename);
+            m.filename = (m.filename == ".")
+                         ? master_filename
+                         : ResolveRelativeToMaster(master_file_directory, m.filename);
 
         dataset->experiment.ImagesPerTrigger(number_of_images);
         cached_geom = dataset->experiment.GetDiffractionGeometry();
@@ -1141,10 +1397,12 @@ void HDF5MetadataSource::FillPerImage(DataMessage &message, int64_t requested_im
     }
 }
 
-std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, const std::string &name) {
-    std::string dname = "/entry/sample/transformations/" + name;
+std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, const std::string &name,
+                                                           const std::string &group) {
+    std::string dname = group + "/" + name;
 
-    if (!file->Exists(dname))
+    // Not a dataset, not an axis: a hybrid file keeps a bare subgroup here for the direction alone.
+    if (!file->IsDataSet(dname))
         return {};
 
 
@@ -1160,9 +1418,29 @@ std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, con
     // Missing attribute means "not a transformation", so skip it rather than throwing: the search
     // for the goniometer walks every leaf and only stops early on an axis that turns, so a master
     // whose axis was stationary reached omega_end and could not be opened at all.
-    if (!dataset.AttrExists("transformation_type")
-            || (dataset.ReadAttrStr("transformation_type") != "rotation"))
+    // NXmx tags every axis; DECTRIS firmware 1.x tagged none of them, so absence has to mean two
+    // different things depending on the layout. In a transformations group it means "not an axis"
+    // (the writer's own AXISNAME_end and the rotation-width scalars live there and carry only units),
+    // and skipping is right. In the legacy goniometer group EVERY leaf is an axis and none is tagged,
+    // so skipping there would find no goniometer at all and the sweep would be read as stills.
+    const bool legacy_group = (group != "/entry/sample/transformations");
+    if (dataset.AttrExists("transformation_type")) {
+        if (dataset.ReadAttrStr("transformation_type") != "rotation")
+            return {};
+    } else if (!legacy_group) {
         return {};
+    } else {
+        // The same companion datasets, recognised by name because there is no tag to go on. In the
+        // legacy layout each axis NAME carries five of them - AXIS_end, _start, _increment,
+        // _range_average, _range_total - and only the bare name is the axis itself. Matching the
+        // suffix rather than "contains an underscore" keeps a genuine two_theta axis readable.
+        static const char *const companions[] = {"_end", "_start", "_increment",
+                                                 "_range_average", "_range_total"};
+        for (const char *suffix: companions)
+            if (name.size() > strlen(suffix)
+                    && name.compare(name.size() - strlen(suffix), strlen(suffix), suffix) == 0)
+                return {};
+    }
 
     std::vector<double> end = file->ReadOptVector<double>(dname + "_end");
 
@@ -1172,7 +1450,32 @@ std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, con
     double start = angle[0];
     double incr = (angle.size() < 2) ? 0.0 : angle[1] - angle[0];
 
-    std::vector<double> axis_vec = dataset.ReadAttrVec("vector");
+    std::vector<double> axis_vec;
+    if (dataset.AttrExists("vector")) {
+        axis_vec = dataset.ReadAttrVec("vector");
+    } else if (legacy_group) {
+        // The angles carry no direction here, but a hybrid file still states one next door: an
+        // NXmx-shaped subgroup /entry/sample/transformations/AXIS, holding the vector attribute
+        // and no angles. That is the file speaking, so it beats the assumption below.
+        const std::string nxmx_axis = "/entry/sample/transformations/" + name;
+        if (file->Exists(nxmx_axis) && !file->IsDataSet(nxmx_axis)) {
+            HDF5Group nxmx_group(*file, nxmx_axis);
+            if (nxmx_group.AttrExists("vector"))
+                axis_vec = nxmx_group.ReadAttrVec("vector");
+        }
+        if (axis_vec.empty()) {
+            // Firmware 1.x stored no direction at all. Assume the one every DECTRIS master since has
+            // written, and say so - a wrong guess here does not index, so it is visible rather than
+            // silent, and the rotation first pass will try the opposite sign anyway.
+            axis_vec = {-1.0, 0.0, 0.0};
+            Logger("HDF5Reader").Warning("{} carries no axis direction (pre-NXmx layout); assuming "
+                                         "(-1,0,0), the direction current DECTRIS masters write", dname);
+        }
+    } else {
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              dname + " has no vector attribute");
+    }
+
     if (axis_vec.size() != 3)
         throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
                               dname + " Vector must have 3 elements");

reader/HDF5MetadataSource.h

@@ -74,6 +74,9 @@ private:
     std::optional<int64_t> ToLocalIndex(int64_t image_number) const;
 
     HDF5ImageLocator::Location ResolveMeta(int64_t global) const;
-    std::optional<GoniometerAxis> ReadAxis(HDF5Object *file, const std::string &name);
+    // group is where the axes live: /entry/sample/transformations (NXmx) or
+    // /entry/sample/goniometer (DECTRIS firmware 1.x, which tags no axis and gives no vector).
+    std::optional<GoniometerAxis> ReadAxis(HDF5Object *file, const std::string &name,
+                                           const std::string &group);
     void ReadROIMetadata(HDF5ReadOnlyFile &file, JFJochReaderDataset &dataset) const;
 };

reader/JFJochCBFReader.cpp

@@ -0,0 +1,349 @@
+// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
+// SPDX-License-Identifier: GPL-3.0-only
+
+#include "JFJochCBFReader.h"
+
+#include <algorithm>
+#include <cctype>
+#include <cstring>
+#include <filesystem>
+#include <array>
+#include <map>
+#include <optional>
+
+#include "../common/JFJochException.h"
+#include "../common/Logger.h"
+#include "../common/JFJochMath.h"
+
+namespace {
+
+bool HasCBFExtension(const std::filesystem::path &p) {
+    std::string ext = p.extension().string();
+    std::transform(ext.begin(), ext.end(), ext.begin(), [](unsigned char c) { return std::tolower(c); });
+    return ext == ".cbf";
+}
+
+// The sweep a file belongs to, as a template: everything before the trailing run of digits, the
+// number of digits, and the extension. "o8_1_0042.cbf" -> {"o8_1_", 4}. A directory can hold several
+// sweeps ("o8_1_*" beside "o8_2_*"), so collecting every .cbf in it would silently splice two
+// crystals together; matching the template is what makes "point at any frame" safe.
+struct Template {
+    std::string prefix;
+    size_t digits = 0;
+
+    bool Matches(const std::string &name) const {
+        if (name.size() != prefix.size() + digits + 4)   // + ".cbf"
+            return false;
+        if (name.compare(0, prefix.size(), prefix) != 0)
+            return false;
+        for (size_t i = 0; i < digits; i++)
+            if (!std::isdigit(static_cast<unsigned char>(name[prefix.size() + i])))
+                return false;
+        return true;
+    }
+};
+
+std::optional<Template> TemplateOf(const std::string &filename) {
+    const std::filesystem::path p(filename);
+    if (!HasCBFExtension(p))
+        return {};
+    const std::string stem = p.stem().string();
+    size_t end = stem.size();
+    while (end > 0 && std::isdigit(static_cast<unsigned char>(stem[end - 1])))
+        end--;
+    if (end == stem.size())
+        return {};                                   // no trailing number: not part of a numbered sweep
+    return Template{stem.substr(0, end), stem.size() - end};
+}
+
+std::vector<std::string> CollectSweep(const std::string &path) {
+    std::filesystem::path p(path);
+    const bool is_dir = std::filesystem::is_directory(p);
+    // A frame named with no directory at all is in this one - parent_path() of a bare filename is
+    // empty, and iterating an empty path finds nothing, so naming a frame from inside its own
+    // directory found no sweep.
+    std::filesystem::path dir = is_dir ? p : p.parent_path();
+    if (dir.empty())
+        dir = ".";
+
+    // Naming a frame selects ITS sweep. Naming a directory selects the sweep with the most frames in
+    // it, which is the one a user pointing at a data directory means.
+    std::optional<Template> want;
+    if (!is_dir)
+        want = TemplateOf(p.filename().string());
+
+    std::map<std::pair<std::string, size_t>, std::vector<std::string>> sweeps;
+    std::error_code ec;
+    for (const auto &e : std::filesystem::directory_iterator(dir, ec)) {
+        if (!e.is_regular_file() || !HasCBFExtension(e.path()))
+            continue;
+        const std::string name = e.path().filename().string();
+        const auto t = TemplateOf(name);
+        if (!t.has_value())
+            continue;
+        if (want.has_value() && !want->Matches(name))
+            continue;
+        sweeps[{t->prefix, t->digits}].push_back(e.path().string());
+    }
+
+    std::vector<std::string> out;
+    for (auto &[key, files] : sweeps)
+        if (files.size() > out.size())
+            out = std::move(files);
+
+    // The frame number is zero-padded in every PILATUS naming scheme in use, so within one template a
+    // plain sort is the collection order.
+    std::sort(out.begin(), out.end());
+    return out;
+}
+
+// An imgCIF laboratory direction in the internal frame. imgCIF puts Z from the sample towards the
+// source and Y opposite gravity, while the internal frame has z along the beam and y along increasing
+// row, so the two differ by a half turn about x - a rotation and not a mirror, so an axis carried
+// through it turns the same way by the same angle. (writer/HDF5NXmx.cpp states the same relation from
+// the other side, where it separates this from the McStas one, which is a half turn about z.)
+Coord ImgCIFToInternal(const std::array<double, 3> &v) {
+    return {static_cast<float>(v[0]), static_cast<float>(-v[1]), static_cast<float>(-v[2])};
+}
+
+// The base rotation axis where a header states nothing about it, in the internal frame (x along
+// increasing detector column, y along increasing row, z along the beam). It is a convention: such a
+// header names its rotation axis but gives no direction, so this is the sign an NXmx master writes
+// for the same instruments, and a file that needs the other one is settled from the data by the run's
+// axis-sign rescue.
+const Coord ASSUMED_BASE_AXIS(-1.0f, 0.0f, 0.0f);
+
+// The base rotation axis of the instrument, in the internal frame.
+//
+// Two headers in every corpus examined here state it and were being overruled by the assumption. One
+// beamline's PILATUS turns about the VERTICAL: its axis table says so outright, and its "# Oscillation
+// _axis" line says so a second way, by naming the image direction the spindle runs along rather than a
+// vector. Assuming the horizontal axis put the spindle 90 degrees out - which no amount of refinement
+// recovers, and which a sign rescue cannot reach either, since it is not a sign - and the run indexed
+// nothing. The sign is still the rescue's business; the DIRECTION is the file's.
+Coord BaseAxis(const minicbf::Header &h) {
+    if (h.spindle_axis.has_value())
+        return ImgCIFToInternal(*h.spindle_axis);
+    if (h.spindle_along_slow)
+        return {0.0f, -1.0f, 0.0f};     // minus the slow direction, as the default is minus the fast
+    return ASSUMED_BASE_AXIS;
+}
+
+// How the stored image sits in the detector plane, where the header's axis table states it - the same
+// thing the NXmx module directions say, in the form this format says it. Nothing comes back when the
+// header carries no table, or when what it states is not one of the eight discrete orientations.
+std::optional<DetectorOrientation> ImageOrientation(const minicbf::Header &h) {
+    if (!h.fast_direction.has_value() || !h.slow_direction.has_value())
+        return {};
+    return DetectorOrientation::Match(ImgCIFToInternal(*h.fast_direction),
+                                      ImgCIFToInternal(*h.slow_direction));
+}
+
+// The axis a miniCBF sweep turns about, in the internal frame (x along increasing detector column,
+// y along increasing row, z along the beam).
+//
+// The base axis comes from the header where it states one (BaseAxis above) and is otherwise assumed.
+// The sense of the omega rotation below follows the base axis, so a sweep parked at a non-zero omega
+// inherits whichever direction it turns out to have.
+//
+// The head is base -> chi -> phi, so only the axes OUTSIDE the scanned one can tilt it. An omega
+// scan turns about the base axis however the cradle is set - which is why a header carrying a large
+// fixed chi still comes out as the base axis here - and only a phi scan is carried by chi and by
+// omega. Chi turns about the beam, as the imgCIF axis convention has it, pointing back at the
+// source; internal z points the other way, hence the minus. A kappa arm cannot be expressed at all:
+// its inclination is a property of the hardware that no miniCBF header states.
+Coord RotationAxis(const minicbf::Header &h) {
+    const Coord base = BaseAxis(h);
+    if (!minicbf::ScansPhi(h))
+        return base;
+
+    const auto rad = [](double deg) { return static_cast<float>(deg * PI / 180.0); };
+    const Coord chi_axis(0.0f, 0.0f, -1.0f);
+    const Coord tilted = RotMatrix(rad(h.chi_deg.value_or(0.0)), chi_axis) * base;
+    return RotMatrix(rad(h.omega_deg.value_or(0.0)), base) * tilted;
+}
+
+}  // namespace
+
+bool JFJochCBFReader::CanRead(const std::string &path) {
+    std::error_code ec;
+    if (std::filesystem::is_directory(path, ec))
+        return !CollectSweep(path).empty();
+    if (!HasCBFExtension(std::filesystem::path(path)))
+        return false;
+    try {
+        // A pixel size as well as the compression: a byte-offset CBF with no "# Pixel_size" line has
+        // no geometry, and XDS writes its correction files in exactly that shape. Claiming one here
+        // would open it with a pixel size of zero.
+        const auto h = minicbf::ReadHeader(path);
+        return h.byte_offset && h.pixel_x_m > 0.0 && h.pixel_y_m > 0.0;
+    } catch (const JFJochException &) {
+        return false;
+    }
+}
+
+void JFJochCBFReader::ReadFiles(const std::string &path) {
+    files_ = CollectSweep(path);
+    if (files_.empty())
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "No CBF images found for " + path);
+
+    header0_ = minicbf::ReadHeader(files_[0]);
+    if (!header0_.byte_offset)
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Unsupported CBF compression (only x-CBF_BYTE_OFFSET)");
+    // Beside the compression, because a pixel size is what makes the file an IMAGE: every resolution,
+    // every scattering vector and the beam centre in millimetres scale by it, and the default of 0
+    // collapses all of them without a word. A byte-offset CBF with no "# Pixel_size" line is not a
+    // detector image from this family - XDS writes its correction files in exactly that shape.
+    if (!(header0_.pixel_x_m > 0.0) || !(header0_.pixel_y_m > 0.0))
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              files_[0] + " has no pixel size in its header (no '# Pixel_size' line); "
+                              "it carries a CBF binary section but is not a detector image");
+
+    dataset_ = std::make_shared<JFJochReaderDataset>();
+    dataset_->experiment = default_experiment;
+
+    DetectorSetup detector = DetDECTRIS(header0_.nx, header0_.ny, header0_.detector, {});
+    detector.PixelSize_um(static_cast<int64_t>(std::lround(header0_.pixel_x_m * 1e6)));
+    detector.SensorThickness_um(static_cast<int64_t>(std::lround(header0_.thickness_m * 1e6)));
+    detector.SensorMaterial(header0_.material);
+    // Only when the header states one. Count_cutoff defaults to 0 and SaturationLimitFromValue(0) is
+    // 1, so an absent line marked EVERY pixel at or above one count as saturated - the integration
+    // accept gate then drops the whole reflection and the run comes out empty for a reason nothing
+    // reports. Left unset, DiffractionExperiment::GetSaturationLimit() falls back to the container's
+    // own overflow, which is the safe direction: it can only fail to call a pixel saturated.
+    if (header0_.count_cutoff > 0)
+        detector.SaturationLimit(SaturationLimitFromValue(header0_.count_cutoff));
+    else
+        Logger("CBFReader").Warning("{} states no Count_cutoff, so no pixel will be called "
+                                    "saturated; if this detector overloads, its strongest "
+                                    "reflections will be integrated as if they were valid.",
+                                    files_[0]);
+    // Images are handed out as signed 32-bit whatever the file stored, so that is the depth the rest
+    // of the code must see; the real overflow is the header's Count_cutoff, set above.
+    detector.BitDepthImage(32);
+    if (const auto orientation = ImageOrientation(header0_))
+        detector.ImageOrientation(orientation.value());
+    detector.MinFrameTime(std::chrono::microseconds(0));
+    detector.MinCountTime(std::chrono::microseconds(0));
+    detector.ReadOutTime(std::chrono::nanoseconds(0));
+    dataset_->experiment.Detector(detector);
+
+    dataset_->experiment.BeamX_pxl(static_cast<float>(header0_.beam_x_px));
+    dataset_->experiment.BeamY_pxl(static_cast<float>(header0_.beam_y_px));
+    dataset_->experiment.DetectorDistance_mm(static_cast<float>(header0_.distance_m * 1000.0));
+
+    // A detector swung out on a 2theta arm, which small-molecule collection uses routinely. The arm
+    // turns the detector about the sample, so it carries the square-on geometry with it: the header's
+    // Detector_distance stays the distance along the detector normal and Beam_xy stays the point of
+    // normal incidence, neither of which the swing moves - which is exactly what the PONI convention
+    // wants, so the swing is a PONI rotation and nothing else in the header changes. It turns about
+    // the axis the header's table gives the arm, and otherwise about the base spindle axis: on the
+    // four-circle geometry these headers describe the arm and the spindle are one axis, and the one
+    // header here that states both states them with the same vector.
+    if (header0_.two_theta_deg != 0.0) {
+        const Coord axis = header0_.detector_axis.has_value()
+                               ? ImgCIFToInternal(*header0_.detector_axis) : BaseAxis(header0_);
+        float rot1 = 0, rot2 = 0, rot3 = 0;
+        PoniAnglesFromMatrix(RotMatrix(static_cast<float>(header0_.two_theta_deg * PI / 180.0), axis),
+                             rot1, rot2, rot3);
+        dataset_->experiment.PoniRot1_rad(rot1).PoniRot2_rad(rot2).PoniRot3_rad(rot3);
+    }
+
+    dataset_->experiment.IncidentEnergy_keV(WVL_1A_IN_KEV / static_cast<float>(header0_.wavelength_A));
+    dataset_->experiment.FrameTime(
+        std::chrono::duration_cast<std::chrono::nanoseconds>(
+            std::chrono::duration<double>(header0_.period_s)),
+        std::chrono::duration_cast<std::chrono::nanoseconds>(
+            std::chrono::duration<double>(header0_.exposure_s)));
+
+    // The rotation angle of every image, from its own header.
+    std::vector<double> angles(files_.size());
+    for (size_t i = 0; i < files_.size(); i++)
+        angles[i] = minicbf::ReadHeader(files_[i]).start_angle_deg;
+
+    double increment = header0_.angle_increment_deg;
+    if (files_.size() > 1) {
+        // Prefer the measured step over the header's nominal one, and unwrap a sweep that passes 360.
+        double d = angles[1] - angles[0];
+        if (d < -180.0) d += 360.0;
+        if (std::abs(d) > 1e-6) increment = d;
+    }
+    dataset_->experiment.Goniometer(GoniometerAxis(header0_.axis_name,
+                                                   static_cast<float>(angles.front()),
+                                                   static_cast<float>(increment),
+                                                   RotationAxis(header0_), {}));
+
+    dataset_->error_value = -1;
+    dataset_->experiment.ImagesPerTrigger(static_cast<int64_t>(files_.size()));
+
+    // The untrusted pixels a PILATUS marks with a negative value: module gaps and the bad-pixel map.
+    // They are the same on every frame of a sweep, so frame 0 defines the mask.
+    std::vector<int32_t> first;
+    minicbf::Read(files_[0], first);
+    std::vector<uint32_t> mask(first.size(), 0);
+    for (size_t i = 0; i < first.size(); i++)
+        if (first[i] < 0)
+            mask[i] = 1;
+    dataset_->pixel_mask = std::make_shared<const PixelMask>(mask);
+
+    SetStartMessage(dataset_);
+}
+
+uint64_t JFJochCBFReader::GetNumberOfImages() const {
+    return files_.size();
+}
+
+void JFJochCBFReader::Close() {
+    files_.clear();
+    dataset_.reset();
+}
+
+template <class Buffer>
+CompressedImage JFJochCBFReader::DecodeInto(int64_t image_number, Buffer &buffer) const {
+    if (image_number < 0 || static_cast<size_t>(image_number) >= files_.size())
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Image number out of range");
+
+    const size_t npixel = static_cast<size_t>(header0_.nx) * static_cast<size_t>(header0_.ny);
+    buffer.resize(npixel * sizeof(int32_t));
+
+    // Decode straight into the caller's bytes: the pixels are plain int32 and nothing downstream has
+    // to decompress them, so NO_COMPRESSION over that buffer is the whole image.
+    const auto h = minicbf::ReadInto(files_[image_number],
+                                     reinterpret_cast<int32_t *>(buffer.data()), npixel);
+    if (static_cast<size_t>(h.nelem) != npixel)
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "CBF image size differs from the first image of the sweep");
+
+    return CompressedImage(buffer.data(), buffer.size(),
+                           static_cast<size_t>(header0_.nx), static_cast<size_t>(header0_.ny),
+                           CompressedImageMode::Int32, CompressionAlgorithm::NO_COMPRESSION);
+}
+
+bool JFJochCBFReader::LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
+                                  DataMessage &message,
+                                  std::vector<uint8_t> &buffer,
+                                  int64_t image_number,
+                                  bool update_dataset) {
+    (void) update_dataset;
+    if (!dataset)
+        return false;
+
+    // The image must outlive this call, so it is decoded straight into the caller's buffer - the same
+    // thing the argument is for on the HDF5 path - and message.image only points at it.
+    message.image = DecodeInto(image_number, buffer);
+    message.number = image_number;
+    return true;
+}
+
+std::shared_ptr<JFJochReaderRawImage> JFJochCBFReader::GetRawImage(int64_t image_number) {
+    auto ret = std::make_shared<JFJochReaderRawImage>();
+    ret->image = DecodeInto(image_number, ret->image_buffer);
+    return ret;
+}
+
+std::vector<SpotToSave> JFJochCBFReader::ReadSpots(int64_t) const {
+    return {};   // a raw CBF stores no analysis results
+}

reader/JFJochCBFReader.h

@@ -0,0 +1,54 @@
+// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
+// SPDX-License-Identifier: GPL-3.0-only
+
+#pragma once
+
+#include <string>
+#include <vector>
+
+#include "JFJochReader.h"
+#include "MiniCBF.h"
+
+// Reads a rotation sweep straight from a directory of PILATUS miniCBF files - the form most
+// facilities still archive - with no conversion step and no libcbf.
+//
+// The sweep's geometry comes from the first file's header; the rotation angle of each image comes
+// from its own header, which costs only the bytes up to the binary separator. Images are decoded on
+// demand, one per call, so any number of workers can read at once - there is no global lock as there
+// is on the HDF5 path, HDF5 not being thread-safe.
+//
+// A raw CBF carries no analysis results, so this reader has no spots, no reflections and no snapshots;
+// the dataset it builds is the geometry, the mask and nothing else, exactly as a plain DECTRIS file
+// with no /entry/MX would give.
+class JFJochCBFReader : public JFJochReader {
+    std::vector<std::string> files_;
+    std::shared_ptr<JFJochReaderDataset> dataset_;
+    minicbf::Header header0_;
+
+    bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
+                     DataMessage &message,
+                     std::vector<uint8_t> &buffer,
+                     int64_t image_number,
+                     bool update_dataset) override;
+
+    // Decodes one image into the caller's byte buffer (RawByteBuffer or std::vector<uint8_t>) and
+    // returns the image that points at it.
+    template <class Buffer>
+    CompressedImage DecodeInto(int64_t image_number, Buffer &buffer) const;
+
+public:
+    ~JFJochCBFReader() override = default;
+
+    // True if the path names something this reader can open: a miniCBF file, or a directory holding
+    // at least one. Cheap - it reads a few kB at most.
+    static bool CanRead(const std::string &path);
+
+    // path is a directory of *.cbf, or one *.cbf inside the sweep to take the whole directory from.
+    void ReadFiles(const std::string &path);
+
+    [[nodiscard]] uint64_t GetNumberOfImages() const override;
+    void Close() override;
+
+    std::shared_ptr<JFJochReaderRawImage> GetRawImage(int64_t image_number) override;
+    [[nodiscard]] std::vector<SpotToSave> ReadSpots(int64_t image) const override;
+};

reader/JFJochReader.cpp

@@ -83,7 +83,7 @@ void JFJochReader::UpdateGeomMetadata(const DiffractionExperiment &experiment) {
     new_dataset->experiment.PoniRot2_rad(experiment.GetDatasetSettings().GetPoniRot2_rad());
     new_dataset->experiment.PoniRot3_rad(experiment.GetDatasetSettings().GetPoniRot3_rad());
     new_dataset->experiment.SetUnitCell(experiment.GetUnitCell());
-    new_dataset->experiment.SpaceGroupNumber(experiment.GetSpaceGroupNumber());
+    new_dataset->experiment.SetSpaceGroup(experiment.GetGemmiSpaceGroup());
     new_dataset->experiment.PolarizationFactor(experiment.GetPolarizationFactor());
     new_dataset->experiment.Goniometer(experiment.GetGoniometer());
     new_dataset->experiment.GridScan(experiment.GetGridScan());

reader/MiniCBF.cpp

@@ -0,0 +1,427 @@
+// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
+// SPDX-License-Identifier: GPL-3.0-only
+
+#include "MiniCBF.h"
+
+#include <algorithm>
+#include <cctype>
+#include <cstring>
+#include <fstream>
+#include <limits>
+#include <map>
+#include <mutex>
+#include <regex>
+#include <stdexcept>
+
+#include "../common/JFJochException.h"
+
+namespace minicbf {
+
+namespace {
+
+// One capture group, first match, or nothing. The headers are a few kB, so a regex per field is
+// cheap and keeps each rule next to the thing it reads - but COMPILING one is not: measured at ~60 us
+// here, and a header parse runs 26 of them. Parsing happens once per image read across a whole sweep,
+// so the same two dozen patterns were being recompiled thousands of times. They are compiled once and
+// kept; std::map never invalidates a reference, so the pointer outlives the lock.
+std::optional<std::string> Match(const std::string &text, const char *pattern) {
+    static std::mutex cache_mutex;
+    static std::map<std::string, std::regex> cache;
+    const std::regex *re;
+    {
+        const std::lock_guard<std::mutex> lock(cache_mutex);
+        auto it = cache.find(pattern);
+        if (it == cache.end())
+            it = cache.emplace(pattern, std::regex(pattern)).first;
+        re = &it->second;
+    }
+    std::smatch m;
+    if (!std::regex_search(text, m, *re) || m.size() < 2)
+        return {};
+    return m[1].str();
+}
+
+// The captures are character classes, not number grammars: "[\d.eE+-]+" matches "." and "+-", and
+// "(\d+)" matches a digit string too long for int64. std::stod and std::stoll answer both with a raw
+// std:: exception, which would leave the format probe below - CanRead catches JFJochException only -
+// and reach the caller as an unhandled throw from merely LOOKING at a file. A header field that does
+// not parse is a malformed header, so say that.
+double Num(const std::string &text, const char *pattern, double fallback = 0.0) {
+    const auto s = Match(text, pattern);
+    if (!s.has_value())
+        return fallback;
+    try {
+        return std::stod(*s);
+    } catch (const std::exception &) {
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Malformed number in CBF header: '" + *s + "'");
+    }
+}
+
+int64_t Int(const std::string &text, const char *pattern, int64_t fallback = 0) {
+    const auto s = Match(text, pattern);
+    if (!s.has_value())
+        return fallback;
+    try {
+        return std::stoll(*s);
+    } catch (const std::exception &) {
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Malformed integer in CBF header: '" + *s + "'");
+    }
+}
+
+// A goniometer angle, or nothing where the head has no such axis. Writers spell that -9999, and
+// taking a sentinel for an angle would put the head somewhere it never was.
+std::optional<double> Angle(const std::string &text, const char *pattern) {
+    const auto s = Match(text, pattern);
+    if (!s.has_value())
+        return {};
+    double v;
+    try {
+        v = std::stod(*s);
+    } catch (const std::exception &) {
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Malformed angle in CBF header: '" + *s + "'");
+    }
+    if (v < -9998.0)
+        return {};
+    return v;
+}
+
+// One "loop_" of the imgCIF template block these headers carry, as rows keyed by tag. Tags and values
+// are both read as whitespace-separated tokens rather than by line, because a template packs several
+// tags onto one line ("_axis.vector[1] _axis.vector[2] _axis.vector[3]") and the rows that follow are
+// laid out to match the tags, not the lines.
+std::vector<std::map<std::string, std::string>> ParseLoop(const std::string &text, const std::string &tag) {
+    // The loop_ that introduces the tag, not the tag's own position: everything before it is another
+    // loop's data.
+    const size_t tag_at = text.find("\n" + tag);
+    if (tag_at == std::string::npos)
+        return {};
+    const size_t loop_at = text.rfind("loop_", tag_at);
+    if (loop_at == std::string::npos)
+        return {};
+
+    // Bare tokens, and the quoted ones a CIF value may be - a quoted value holding spaces would
+    // otherwise be counted as several columns and shift every row after it.
+    std::vector<std::string> tokens;
+    for (size_t i = loop_at + 5; i < text.size();) {
+        while ((i < text.size()) && std::isspace(static_cast<unsigned char>(text[i])))
+            i++;
+        if (i >= text.size())
+            break;
+        size_t end;
+        if ((text[i] == '\'') || (text[i] == '"')) {
+            end = text.find(text[i], i + 1);
+            if (end == std::string::npos)
+                break;
+            tokens.push_back(text.substr(i + 1, end - i - 1));
+            end++;
+        } else {
+            end = i;
+            while ((end < text.size()) && !std::isspace(static_cast<unsigned char>(text[end])))
+                end++;
+            tokens.push_back(text.substr(i, end - i));
+        }
+        // A second loop_, or a tag belonging to another category, ends this one.
+        if ((tokens.back() == "loop_")
+                || (tokens.back().starts_with("_") && !tokens.back().starts_with(tag.substr(0, tag.find('.') + 1)))) {
+            tokens.pop_back();
+            break;
+        }
+        i = end;
+    }
+
+    std::vector<std::string> names;
+    size_t first_value = 0;
+    while ((first_value < tokens.size()) && tokens[first_value].starts_with("_"))
+        names.push_back(tokens[first_value++]);
+    if (names.empty())
+        return {};
+
+    std::vector<std::map<std::string, std::string>> rows;
+    for (size_t i = first_value; i + names.size() <= tokens.size(); i += names.size()) {
+        std::map<std::string, std::string> row;
+        for (size_t j = 0; j < names.size(); j++)
+            row[names[j]] = tokens[i + j];
+        rows.push_back(std::move(row));
+    }
+    return rows;
+}
+
+std::string Field(const std::map<std::string, std::string> &row, const std::string &name) {
+    const auto it = row.find(name);
+    return (it == row.end()) ? std::string() : it->second;
+}
+
+// The imgCIF axis table: which axis turns or translates in which laboratory direction, and which two
+// axes the image's columns and rows run along. Absent from most headers, which say nothing about any
+// of this and are left exactly as they were read before.
+//
+// The element vectors are stated in the frame of the axis they depend on. Between them and the
+// detector's own rotation every header seen has translations only, so they describe the image in the
+// unswung detector frame - which is where the image orientation belongs, with the arm applied on top.
+// Following Hammersley, Bernstein & Westbrook (2006) Int. Tables Cryst. G, 444-458
+void ParseAxisTable(const std::string &text, Header &h) {
+    const auto axes = ParseLoop(text, "_axis.id");
+    if (axes.empty())
+        return;
+
+    std::map<std::string, std::array<double, 3>> vector_of;
+    for (const auto &row: axes) {
+        const std::string id = Field(row, "_axis.id");
+        const std::string type = Field(row, "_axis.type");
+        const std::string equipment = Field(row, "_axis.equipment");
+        const std::string depends_on = Field(row, "_axis.depends_on");
+        std::array<double, 3> v{};
+        try {
+            for (int i = 0; i < 3; i++)
+                v[i] = std::stod(Field(row, "_axis.vector[" + std::to_string(i + 1) + "]"));
+        } catch (const std::exception &) {
+            continue;   // "." for a vector: the table states no direction for this axis
+        }
+        vector_of[id] = v;
+
+        // The base spindle and the detector arm are the rotations that hang off nothing: everything
+        // further in is carried by them. Naming neither, so a beamline is free to call them anything.
+        if ((type == "rotation") && (depends_on == ".")) {
+            if (equipment == "goniometer")
+                h.spindle_axis = v;
+            else if (equipment == "detector")
+                h.detector_axis = v;
+        }
+    }
+
+    // Which axis the fast index runs along, and which the slow, through the two tables that say so.
+    std::map<std::string, std::string> axis_of_set;
+    for (const auto &row: ParseLoop(text, "_array_structure_list_axis.axis_set_id"))
+        axis_of_set[Field(row, "_array_structure_list_axis.axis_set_id")]
+                = Field(row, "_array_structure_list_axis.axis_id");
+
+    for (const auto &row: ParseLoop(text, "_array_structure_list.array_id")) {
+        const std::string set = Field(row, "_array_structure_list.axis_set_id");
+        const auto id = axis_of_set.contains(set) ? axis_of_set[set] : set;
+        const auto it = vector_of.find(id);
+        if (it == vector_of.end())
+            continue;
+        std::array<double, 3> v = it->second;
+        if (Field(row, "_array_structure_list.direction") == "decreasing")
+            for (double &c: v)
+                c = -c;
+        const std::string index = Field(row, "_array_structure_list.index");
+        if (index == "1")
+            h.fast_direction = v;
+        else if (index == "2")
+            h.slow_direction = v;
+    }
+}
+
+int16_t ReadI16(const uint8_t *p) { int16_t v; std::memcpy(&v, p, 2); return v; }
+int32_t ReadI32(const uint8_t *p) { int32_t v; std::memcpy(&v, p, 4); return v; }
+int64_t ReadI64(const uint8_t *p) { int64_t v; std::memcpy(&v, p, 8); return v; }
+
+}  // namespace
+
+bool ScansPhi(const Header &h) {
+    if (h.phi_increment_deg != 0.0 || h.omega_increment_deg != 0.0 || h.chi_increment_deg != 0.0)
+        return h.phi_increment_deg != 0.0;
+    std::string name = h.axis_name;
+    std::transform(name.begin(), name.end(), name.begin(),
+                   [](unsigned char c) { return std::tolower(c); });
+    return name.starts_with("phi");
+}
+
+std::optional<size_t> FindBinarySection(const uint8_t *data, size_t size) {
+    if (size < sizeof(BINARY_SEPARATOR))
+        return {};
+    const auto *end = data + size;
+    const auto *hit = std::search(data, end, std::begin(BINARY_SEPARATOR), std::end(BINARY_SEPARATOR));
+    if (hit == end)
+        return {};
+    return static_cast<size_t>(hit - data) + sizeof(BINARY_SEPARATOR);
+}
+
+Header ParseHeader(const char *data, size_t size) {
+    const std::string t(data, size);
+    Header h;
+
+    h.detector       = Match(t, R"(#\s*Detector:\s*([^\r\n]+))").value_or("PILATUS");
+    h.pixel_x_m      = Num(t, R"(#\s*Pixel_size\s+([\d.eE+-]+)\s*m)");
+    h.pixel_y_m      = Num(t, R"(#\s*Pixel_size\s+[\d.eE+-]+\s*m\s*x\s*([\d.eE+-]+)\s*m)");
+    // The unit is optional: one ALBA set writes "thickness 0.001000" with no " m" after it.
+    h.thickness_m    = Num(t, R"(sensor,\s*thickness\s+([\d.eE+-]+))");
+    h.distance_m     = Num(t, R"(#\s*Detector_distance\s+([\d.eE+-]+))");
+    h.beam_x_px      = Num(t, R"(#\s*Beam_xy\s*\(\s*([\d.eE+-]+))");
+    h.beam_y_px      = Num(t, R"(#\s*Beam_xy\s*\([^,]+,\s*([\d.eE+-]+))");
+    h.wavelength_A   = Num(t, R"(#\s*Wavelength\s+([\d.eE+-]+))");
+    h.start_angle_deg     = Num(t, R"(#\s*Start_angle\s+([\d.eE+-]+))");
+    h.angle_increment_deg = Num(t, R"(#\s*Angle_increment\s+([\d.eE+-]+))");
+    h.two_theta_deg  = Num(t, R"(#\s*Detector_2theta\s+([\d.eE+-]+))");
+    // The whitespace after each name is what keeps "Chi_increment" out of "Chi".
+    h.chi_deg        = Angle(t, R"(#\s*Chi\s+([\d.eE+-]+))");
+    h.omega_deg      = Angle(t, R"(#\s*Omega\s+([\d.eE+-]+))");
+    h.chi_increment_deg   = Angle(t, R"(#\s*Chi_increment\s+([\d.eE+-]+))").value_or(0.0);
+    h.phi_increment_deg   = Angle(t, R"(#\s*Phi_increment\s+([\d.eE+-]+))").value_or(0.0);
+    h.omega_increment_deg = Angle(t, R"(#\s*Omega_increment\s+([\d.eE+-]+))").value_or(0.0);
+    h.exposure_s     = Num(t, R"(#\s*Exposure_time\s+([\d.eE+-]+))");
+    h.period_s       = Num(t, R"(#\s*Exposure_period\s+([\d.eE+-]+))");
+    h.count_cutoff   = Int(t, R"(#\s*Count_cutoff\s+(\d+))");
+    h.axis_name      = Match(t, R"(#\s*Oscillation_axis\s+(\S+))").value_or("omega");
+    // "+SLOW" / "+FAST" on that same line: which of the image's two directions the spindle runs
+    // along. Some writers state that instead of an axis name, and it is the only thing a header with
+    // no axis table says about the spindle's direction at all.
+    h.spindle_along_slow = Match(t, R"(#\s*Oscillation_axis[^\r\n]*\+(SLOW|slow))").has_value();
+    ParseAxisTable(t, h);
+
+    // "# Silicon sensor, ..." / "# CdTe sensor, ...". The rest of the code compares the material
+    // against "CdTe" (BraggIntegrationEngine), so an unnormalised "Silicon" would silently give a
+    // CdTe sensor silicon's attenuation length.
+    if (const auto m = Match(t, R"(#\s*(\w+)\s+sensor,)")) {
+        std::string s = *m;
+        std::transform(s.begin(), s.end(), s.begin(), [](unsigned char c) { return std::tolower(c); });
+        h.material = (s == "cdte") ? "CdTe" : "Si";
+    }
+
+    h.nx    = Int(t, R"(X-Binary-Size-Fastest-Dimension:\s*(\d+))");
+    h.ny    = Int(t, R"(X-Binary-Size-Second-Dimension:\s*(\d+))");
+    h.nelem = Int(t, R"(X-Binary-Number-of-Elements:\s*(\d+))");
+
+    const auto conv = Match(t, R"RE(conversions\s*=\s*"([^"]+)")RE").value_or("");
+    h.byte_offset = conv.find("x-CBF_BYTE_OFFSET") != std::string::npos;
+
+    if (h.period_s <= 0.0)
+        h.period_s = h.exposure_s;
+
+    return h;
+}
+
+// The x-CBF_BYTE_OFFSET scheme: a running value, each pixel stored as a delta in the smallest
+// container that holds it, escaping to the next size with that container's most negative value.
+// Following Bernstein & Hammersley (2006) Int. Tables Cryst. G, 37-43
+void DecodeByteOffset(const uint8_t *data, size_t size, int32_t *out, size_t n_pixels) {
+    int64_t value = 0;
+    size_t pos = 0;
+    size_t written = 0;
+
+    while (written < n_pixels) {
+        if (pos >= size)
+            throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                                  "miniCBF byte-offset stream ended after " + std::to_string(written)
+                                  + " of " + std::to_string(n_pixels) + " pixels");
+
+        int64_t delta = static_cast<int8_t>(data[pos]);
+        pos += 1;
+
+        if (delta == -128) {
+            if (pos + 2 > size) break;
+            delta = ReadI16(data + pos);
+            pos += 2;
+            if (delta == -32768) {
+                if (pos + 4 > size) break;
+                delta = ReadI32(data + pos);
+                pos += 4;
+                if (delta == INT32_MIN) {
+                    if (pos + 8 > size) break;
+                    delta = ReadI64(data + pos);
+                    pos += 8;
+                }
+            }
+        }
+
+        value += delta;
+        out[written++] = static_cast<int32_t>(value);
+    }
+
+    if (written != n_pixels)
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "miniCBF byte-offset stream ended after " + std::to_string(written)
+                              + " of " + std::to_string(n_pixels) + " pixels");
+}
+
+namespace {
+
+std::vector<uint8_t> Slurp(const std::string &path, size_t max_bytes) {
+    std::ifstream f(path, std::ios::binary);
+    if (!f)
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Cannot open CBF file " + path);
+    f.seekg(0, std::ios::end);
+    const auto file_size = static_cast<size_t>(f.tellg());
+    f.seekg(0, std::ios::beg);
+    std::vector<uint8_t> buf(std::min(file_size, max_bytes));
+    f.read(reinterpret_cast<char *>(buf.data()), static_cast<std::streamsize>(buf.size()));
+    buf.resize(static_cast<size_t>(f.gcount()));
+    return buf;
+}
+
+// Enough to reach the separator on any header seen in the wild (the longest measured is ~6.3 kB).
+constexpr size_t HEADER_PROBE_BYTES = 256 * 1024;
+
+}  // namespace
+
+Header ReadHeader(const std::string &path) {
+    const auto buf = Slurp(path, HEADER_PROBE_BYTES);
+    const auto start = FindBinarySection(buf.data(), buf.size());
+    if (!start.has_value())
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "No CBF binary section in " + path);
+    return ParseHeader(reinterpret_cast<const char *>(buf.data()),
+                       *start - sizeof(BINARY_SEPARATOR));
+}
+
+namespace {
+
+// One read of the file, header parsed, dimensions checked. The caller supplies where the pixels go.
+Header ReadCommon(const std::string &path, std::vector<uint8_t> &buf, size_t &binary_start) {
+    buf = Slurp(path, std::numeric_limits<size_t>::max());
+    const auto start = FindBinarySection(buf.data(), buf.size());
+    if (!start.has_value())
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "No CBF binary section in " + path);
+
+    const Header h = ParseHeader(reinterpret_cast<const char *>(buf.data()),
+                                 *start - sizeof(BINARY_SEPARATOR));
+    if (!h.byte_offset)
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Unsupported CBF compression in " + path + " (only x-CBF_BYTE_OFFSET)");
+    if (h.nelem <= 0 || h.nx <= 0 || h.ny <= 0 || h.nelem != h.nx * h.ny)
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "Inconsistent image dimensions in " + path);
+    // Without a pixel size there is no geometry at all - every resolution, every scattering vector
+    // and the beam centre in millimetres all scale by it - and the default of 0 collapses all of
+    // them silently. A byte-offset CBF carrying no "# Pixel_size" line is not a detector image from
+    // this family at all; XDS writes correction files in exactly that shape. Refuse it here rather
+    // than let it through with a geometry of zero.
+    if (!(h.pixel_x_m > 0.0) || !(h.pixel_y_m > 0.0))
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              path + " has no pixel size in its header (no '# Pixel_size' line); "
+                              "it carries a CBF binary section but is not a detector image");
+
+    binary_start = *start;
+    return h;
+}
+
+}  // namespace
+
+Header Read(const std::string &path, std::vector<int32_t> &out) {
+    std::vector<uint8_t> buf;
+    size_t start = 0;
+    const Header h = ReadCommon(path, buf, start);
+    out.resize(static_cast<size_t>(h.nelem));
+    DecodeByteOffset(buf.data() + start, buf.size() - start, out.data(),
+                     static_cast<size_t>(h.nelem));
+    return h;
+}
+
+Header ReadInto(const std::string &path, int32_t *out, size_t capacity) {
+    std::vector<uint8_t> buf;
+    size_t start = 0;
+    const Header h = ReadCommon(path, buf, start);
+    if (static_cast<size_t>(h.nelem) > capacity)
+        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
+                              "CBF image does not fit the supplied buffer: " + path);
+    DecodeByteOffset(buf.data() + start, buf.size() - start, out, static_cast<size_t>(h.nelem));
+    return h;
+}
+
+}  // namespace minicbf

reader/MiniCBF.h

@@ -0,0 +1,94 @@
+// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
+// SPDX-License-Identifier: GPL-3.0-only
+
+#pragma once
+
+#include <array>
+#include <cstdint>
+#include <optional>
+#include <string>
+#include <vector>
+
+// PILATUS miniCBF: an ASCII header, four separator bytes, then one byte-offset compressed image.
+// No CIF parser and no libcbf are needed - every value is on a "# " comment line or a MIME line.
+namespace minicbf {
+
+// The four bytes that end the MIME header and begin the binary section.
+inline constexpr unsigned char BINARY_SEPARATOR[4] = {0x0c, 0x1a, 0x04, 0xd5};
+
+struct Header {
+    std::string detector;          // "PILATUS3 6M, S/N 60-0136"
+    int64_t nx = 0;                // fast dimension (columns)
+    int64_t ny = 0;                // slow dimension (rows)
+    int64_t nelem = 0;             // pixel count declared by the MIME header
+    double pixel_x_m = 0;
+    double pixel_y_m = 0;
+    double thickness_m = 0;
+    std::string material = "Si";   // NORMALISED: the file says "Silicon", the rest of the code wants "Si"
+    double distance_m = 0;
+    double beam_x_px = 0;
+    double beam_y_px = 0;
+    double wavelength_A = 0;
+    double start_angle_deg = 0;
+    double angle_increment_deg = 0;
+    double two_theta_deg = 0;
+    // Goniometer head angles and the per-image increment of each, in degrees. A writer spells "this
+    // head has no such axis" as -9999, which is a sentinel and not an angle, so an axis the header
+    // does not really carry is absent here rather than 9999 degrees away from zero.
+    std::optional<double> chi_deg;
+    std::optional<double> omega_deg;
+    double chi_increment_deg = 0;
+    double phi_increment_deg = 0;
+    double omega_increment_deg = 0;
+    double exposure_s = 0;
+    double period_s = 0;
+    int64_t count_cutoff = 0;      // saturation
+    std::string axis_name = "omega";
+    bool byte_offset = false;      // the only conversion supported
+
+    // The imgCIF axis table the CBF template block carries after the "# " lines, as the file states
+    // it: vectors in the imgCIF laboratory frame (Z from the sample towards the source, Y opposite
+    // gravity, X completing a right-handed set). Turning them into any other frame is the caller's
+    // business, not this format reader's. Most headers carry no such table and leave all of these
+    // empty, which is not the same as their stating the usual mounting.
+    std::optional<std::array<double, 3>> fast_direction;   // laboratory direction of a +1 column step
+    std::optional<std::array<double, 3>> slow_direction;   // ... of a +1 row step
+    std::optional<std::array<double, 3>> spindle_axis;     // the base goniometer rotation axis
+    std::optional<std::array<double, 3>> detector_axis;    // the detector's own rotation axis: a 2theta arm
+    // "# Oscillation_axis X.CW +SLOW". Some writers say which of the two image directions the spindle
+    // runs along instead of stating a vector. The axis NAME on that line is a poor discriminator (see
+    // ScansPhi, and note that the one header carrying both a name and a table says "X" where its table
+    // says Y) but this token is not: where both are present they agree.
+    bool spindle_along_slow = false;
+};
+
+// Whether the sweep turns PHI rather than the base (omega) axis. What moved is the axis with a
+// non-zero increment - a header routinely carries a "# Phi" angle for a phi that stands still - so
+// the name is consulted only when no increment is stated at all. It is a poor discriminator: some
+// writers put a direction convention there ("X.CW") instead of an axis name.
+bool ScansPhi(const Header &h);
+
+// Byte offset of the binary section (just past the separator), or nothing if there is none.
+std::optional<size_t> FindBinarySection(const uint8_t *data, size_t size);
+
+// Parse the ASCII header. Pass the bytes BEFORE the separator; headers are not a fixed size (one
+// Diamond I24 set carries 6335 bytes, well past a 4 kB guess), so never parse a fixed prefix.
+Header ParseHeader(const char *data, size_t size);
+
+// x-CBF_BYTE_OFFSET -> int32. Deltas against a running value, smallest container first: int8,
+// escaping to int16 via -128, to int32 via -32768, to int64 via INT32_MIN. Little-endian, packed.
+// Throws if the stream ends before n_pixels are produced. out must hold n_pixels.
+void DecodeByteOffset(const uint8_t *data, size_t size, int32_t *out, size_t n_pixels);
+
+// Header + pixels of one file, read from disk.
+Header Read(const std::string &path, std::vector<int32_t> &out);
+
+// The same, decoding into memory the caller already has (one read of the file, no extra copy).
+// Throws if the image does not fit in capacity pixels.
+Header ReadInto(const std::string &path, int32_t *out, size_t capacity);
+
+// Header only - reads just enough of the file to reach the separator. Cheap enough to call per
+// frame for the rotation angles.
+Header ReadHeader(const std::string &path);
+
+}  // namespace minicbf