reader: take a miniCBF's mounting from the imgCIF axis table its header states

A miniCBF header states three things about how the instrument is put together
that the reader was assuming instead: which laboratory direction the image's
columns run along, which its rows run along, and which the spindle turns about.
Some beamlines append a CBF template block holding the full imgCIF axis table,
which says all three outright.

Two instruments in the corpus are not what was assumed, in two different ways.
One mounts its detector a quarter turn round, so the image's columns run
vertically. Another turns its spindle about the VERTICAL, with the image mounted
the usual way; its table says so, and its "# Oscillation_axis" line says so a
second way, by naming the image direction the spindle runs along rather than a
vector. Either error leaves the spindle 90 degrees from the image. That is not a
sign, so the run's axis-sign rescue cannot reach it, and no refinement recovers
it: all three affected sweeps indexed nothing usable.

So the table is read. The element axes give the image orientation, matched against
the eight discrete mountings exactly as the NXmx module directions already are -
the match itself moves to DetectorOrientation, so both readers share one
definition rather than two copies. The goniometer axis with no parent gives the
spindle DIRECTION; its sign stays the rescue's business, which is the part a
convention can legitimately differ on. The detector axis with no parent gives the
2theta arm, replacing the assumption that the arm shares the spindle's axis - the
one header stating both states them with the same vector, so this changes no
answer, only what it rests on. imgCIF's frame differs from the internal one by a
half turn about x, a rotation and not a mirror, as writer/HDF5NXmx.cpp already
records from the other side.

Where a header carries no table, a "+SLOW" on the Oscillation_axis line still
says the spindle runs along the image's slow direction. That is the only thing one
of the three affected sets says about it. The axis NAME on that line stays
unusable - the header that carries both says "X.CW" where its own table says Y -
but the direction token is not: where both are present they agree, which is what
makes reading it evidence rather than a guess.

Also: naming a frame with no directory at all now finds its sweep. parent_path()
of a bare filename is empty and iterating an empty path finds nothing, so running
from inside the data directory reported that no images were found.

Measured, with nothing on the command line. The vertical-spindle protein set goes
from no usable lattice to 100% indexed, P 6(3) 2 2 with a cell 0.43% from
deposited, 87846 reflections at 86.3% completeness and CC(1/2) 0.995. Its
companion from the same detector, which has no table and only the +SLOW token,
goes from a spurious monoclinic cell at 2.3% completeness and I/sigma 0.21 to the
right orthorhombic lattice, 97.7% indexed, 59.7% complete, CC(1/2) 0.996. The
quarter-turned set's three sweeps, at three arm positions, now all index without
the hand-passed quarter turn they needed and agree on one cell to 0.03 A. Six
miniCBF sets that state no table and no +SLOW - including one whose
Oscillation_axis line names an axis in a third dialect - are byte-identical in
.hkl, .mtz, .cif and the image statistics, as are two NXmx sets, which is the
shared orientation matcher moving nothing on that path either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

common/DetectorOrientation.cpp

@@ -46,6 +46,18 @@ RotMatrix DetectorOrientation::Matrix() const {
     return {rz_x, mirror_y ? -rz_y : rz_y, {0, 0, 1}};
 }
 
+std::optional<DetectorOrientation> DetectorOrientation::Match(const Coord &fast, const Coord &slow) {
+    for (int64_t quarter_turns = 0; quarter_turns < 4; quarter_turns++) {
+        for (const bool mirror_y: {false, true}) {
+            const DetectorOrientation orientation(mirror_y, quarter_turns);
+            const RotMatrix m = orientation.Matrix();
+            if (((m.Column(0) - fast).Length() < 1e-3f) && ((m.Column(1) - slow).Length() < 1e-3f))
+                return orientation;
+        }
+    }
+    return {};
+}
+
 bool DetectorOrientation::operator==(const DetectorOrientation &other) const {
     return (mirror_y == other.mirror_y) && (quarter_turns == other.quarter_turns);
 }

common/DetectorOrientation.h

@@ -3,6 +3,8 @@
 
 #pragma once
 
+#include <optional>
+
 #include "Coord.h"
 
 // How the stored image is laid out in the detector plane: mirrored in Y, and/or turned by a multiple
@@ -38,5 +40,11 @@ public:
     // Rz(quarter_turns * 90 deg) * diag(1,-1,1)^mirror_y. Entries are exactly 0 and +-1.
     [[nodiscard]] RotMatrix Matrix() const;
 
+    // The orientation whose fast and slow axes are the two directions given, or nothing when they are
+    // not one of the eight. Nothing means the image is turned in its own plane by something that is
+    // not a multiple of 90 degrees, which is a continuous rotation of the detector and belongs in the
+    // PONI angles - it cannot be told apart from the tilt by looking at the two directions alone.
+    [[nodiscard]] static std::optional<DetectorOrientation> Match(const Coord &fast, const Coord &slow);
+
     bool operator==(const DetectorOrientation &other) const;
 };

docs/CHANGELOG.md

@@ -1,6 +1,8 @@
 # Changelog
 ## 1.0.0
 ### 1.0.0-rc.166
+* A miniCBF sweep takes the mounting from the imgCIF axis table its header carries: which laboratory direction the image's columns and rows run along, and which the spindle turns about. Where there is no table, a `+SLOW` on the `Oscillation_axis` line still says the spindle runs along the image's slow direction. All three were previously assumed, and two instruments are not what was assumed.
+* Naming a miniCBF frame with no directory - a frame in the working directory - finds its sweep instead of reporting that no images were found.
 * A detector swung out on a 2theta arm is placed where it stands, from the depends_on transformation chain of an NXmx master or the `Detector_2theta` line of a miniCBF header; both were previously read and then ignored.
 * `jfjoch_viewer` opens PILATUS miniCBF sweeps - naming any frame opens the whole sweep - and can run a processing job on one.
 * A detector whose stored image is mirrored in Y or mounted at a multiple of 90 degrees can be described as such, in the detector configuration or with `--detector-mirror-y` / `--detector-quarter-turns`, rather than having to be expressed as a detector rotation.

docs/DETECTOR_GEOMETRY.md

@@ -59,6 +59,30 @@ rot2 = asin(-slow.z)         rot1 = atan2(-fast.z, normal.z)         rot3 = atan
 with `rot2` in [-90°, 90°]. The angles are what is stored and what is written out, so a geometry
 given as angles comes back exactly as it was given.
 
+## What a miniCBF header states about the mounting
+
+A PILATUS miniCBF gives the geometry twice. The `# ` lines every writer produces carry the distance, the
+beam centre and the angles; some beamlines then append a CBF template block holding a full **imgCIF axis
+table**, which states the laboratory direction of the image's fast and slow pixel directions, of the base
+goniometer axis, and of a 2theta arm where there is one. Where that table is present it is read, in
+preference to any assumption - it is the same information NXmx puts in `fast_pixel_direction` /
+`slow_pixel_direction` and the goniometer `vector`, in the form this format states it.
+
+imgCIF's laboratory frame has Z from the sample towards the source and Y opposite gravity, so it differs
+from the internal frame by a half turn about x - a rotation, not a mirror, so an axis carried through it
+turns the same way by the same angle.
+
+Two things a header can state that an assumption gets wrong by 90 degrees, which no refinement recovers
+and which the run's axis-sign rescue cannot reach either, a quarter turn not being a sign:
+
+* the image mounted a quarter turn round, so its columns run vertically;
+* a spindle that turns about the **vertical** rather than the horizontal.
+
+Where a header carries no axis table, a `+SLOW` on its `# Oscillation_axis` line still says the spindle
+runs along the image's slow direction rather than its fast one. The axis *name* on that line is not
+usable - one header says `X.CW +SLOW` where its own table says the axis is Y - but the direction token is,
+and on the header that states both they agree.
+
 ## A detector swung out on a 2theta arm
 
 Chemical crystallography reaches high angle by swinging the detector out on a 2theta arm rather than by

reader/HDF5MetadataSource.cpp

@@ -42,18 +42,7 @@ static std::optional<DetectorOrientation> ReadModuleOrientation(HDF5Object *file
     if ((f.size() != 3) || (s.size() != 3))
         return {};
 
-    const Coord fast = McStasToInternal(f);
-    const Coord slow = McStasToInternal(s);
-
-    for (int64_t quarter_turns = 0; quarter_turns < 4; quarter_turns++) {
-        for (bool mirror_y: {false, true}) {
-            const DetectorOrientation orientation(mirror_y, quarter_turns);
-            const RotMatrix m = orientation.Matrix();
-            if (((m.Column(0) - fast).Length() < 1e-3f) && ((m.Column(1) - slow).Length() < 1e-3f))
-                return orientation;
-        }
-    }
-    return {};
+    return DetectorOrientation::Match(McStasToInternal(f), McStasToInternal(s));
 }
 
 // Where the detector stands, from the chain of transformations the file says it depends on.

reader/JFJochCBFReader.cpp

@@ -7,6 +7,7 @@
 #include <cctype>
 #include <cstring>
 #include <filesystem>
+#include <array>
 #include <map>
 #include <optional>
 
@@ -57,7 +58,12 @@ std::optional<Template> TemplateOf(const std::string &filename) {
 std::vector<std::string> CollectSweep(const std::string &path) {
     std::filesystem::path p(path);
     const bool is_dir = std::filesystem::is_directory(p);
-    const std::filesystem::path dir = is_dir ? p : p.parent_path();
+    // A frame named with no directory at all is in this one - parent_path() of a bare filename is
+    // empty, and iterating an empty path finds nothing, so naming a frame from inside its own
+    // directory found no sweep.
+    std::filesystem::path dir = is_dir ? p : p.parent_path();
+    if (dir.empty())
+        dir = ".";
 
     // Naming a frame selects ITS sweep. Naming a directory selects the sweep with the most frames in
     // it, which is the one a user pointing at a data directory means.
@@ -90,19 +96,54 @@ std::vector<std::string> CollectSweep(const std::string &path) {
     return out;
 }
 
-// The base rotation axis of the instrument, in the internal frame (x along increasing detector
-// column, y along increasing row, z along the beam). Both the spindle and the detector arm turn
-// about it: the spindle in RotationAxis below, and the 2theta arm in ReadFiles.
-const Coord BASE_AXIS(-1.0f, 0.0f, 0.0f);
+// An imgCIF laboratory direction in the internal frame. imgCIF puts Z from the sample towards the
+// source and Y opposite gravity, while the internal frame has z along the beam and y along increasing
+// row, so the two differ by a half turn about x - a rotation and not a mirror, so an axis carried
+// through it turns the same way by the same angle. (writer/HDF5NXmx.cpp states the same relation from
+// the other side, where it separates this from the McStas one, which is a half turn about z.)
+Coord ImgCIFToInternal(const std::array<double, 3> &v) {
+    return {static_cast<float>(v[0]), static_cast<float>(-v[1]), static_cast<float>(-v[2])};
+}
+
+// The base rotation axis where a header states nothing about it, in the internal frame (x along
+// increasing detector column, y along increasing row, z along the beam). It is a convention: such a
+// header names its rotation axis but gives no direction, so this is the sign an NXmx master writes
+// for the same instruments, and a file that needs the other one is settled from the data by the run's
+// axis-sign rescue.
+const Coord ASSUMED_BASE_AXIS(-1.0f, 0.0f, 0.0f);
+
+// The base rotation axis of the instrument, in the internal frame.
+//
+// Two headers in every corpus examined here state it and were being overruled by the assumption. One
+// beamline's PILATUS turns about the VERTICAL: its axis table says so outright, and its "# Oscillation
+// _axis" line says so a second way, by naming the image direction the spindle runs along rather than a
+// vector. Assuming the horizontal axis put the spindle 90 degrees out - which no amount of refinement
+// recovers, and which a sign rescue cannot reach either, since it is not a sign - and the run indexed
+// nothing. The sign is still the rescue's business; the DIRECTION is the file's.
+Coord BaseAxis(const minicbf::Header &h) {
+    if (h.spindle_axis.has_value())
+        return ImgCIFToInternal(*h.spindle_axis);
+    if (h.spindle_along_slow)
+        return {0.0f, -1.0f, 0.0f};     // minus the slow direction, as the default is minus the fast
+    return ASSUMED_BASE_AXIS;
+}
+
+// How the stored image sits in the detector plane, where the header's axis table states it - the same
+// thing the NXmx module directions say, in the form this format says it. Nothing comes back when the
+// header carries no table, or when what it states is not one of the eight discrete orientations.
+std::optional<DetectorOrientation> ImageOrientation(const minicbf::Header &h) {
+    if (!h.fast_direction.has_value() || !h.slow_direction.has_value())
+        return {};
+    return DetectorOrientation::Match(ImgCIFToInternal(*h.fast_direction),
+                                      ImgCIFToInternal(*h.slow_direction));
+}
 
 // The axis a miniCBF sweep turns about, in the internal frame (x along increasing detector column,
 // y along increasing row, z along the beam).
 //
-// The base axis is a convention: a miniCBF names its rotation axis but never states a direction, so
-// this is the sign an NXmx master writes for the same instruments, and a file that needs the other
-// one is settled from the data by the run's axis-sign rescue. The sense of the omega rotation below
-// follows that same convention, so a sweep parked at a non-zero omega inherits whichever sign the
-// base axis turns out to have.
+// The base axis comes from the header where it states one (BaseAxis above) and is otherwise assumed.
+// The sense of the omega rotation below follows the base axis, so a sweep parked at a non-zero omega
+// inherits whichever direction it turns out to have.
 //
 // The head is base -> chi -> phi, so only the axes OUTSIDE the scanned one can tilt it. An omega
 // scan turns about the base axis however the cradle is set - which is why a header carrying a large
@@ -111,7 +152,7 @@ const Coord BASE_AXIS(-1.0f, 0.0f, 0.0f);
 // source; internal z points the other way, hence the minus. A kappa arm cannot be expressed at all:
 // its inclination is a property of the hardware that no miniCBF header states.
 Coord RotationAxis(const minicbf::Header &h) {
-    const Coord base = BASE_AXIS;
+    const Coord base = BaseAxis(h);
     if (!minicbf::ScansPhi(h))
         return base;
 
@@ -158,6 +199,8 @@ void JFJochCBFReader::ReadFiles(const std::string &path) {
     // Images are handed out as signed 32-bit whatever the file stored, so that is the depth the rest
     // of the code must see; the real overflow is the header's Count_cutoff, set above.
     detector.BitDepthImage(32);
+    if (const auto orientation = ImageOrientation(header0_))
+        detector.ImageOrientation(orientation.value());
     detector.MinFrameTime(std::chrono::microseconds(0));
     detector.MinCountTime(std::chrono::microseconds(0));
     detector.ReadOutTime(std::chrono::nanoseconds(0));
@@ -172,12 +215,14 @@ void JFJochCBFReader::ReadFiles(const std::string &path) {
     // Detector_distance stays the distance along the detector normal and Beam_xy stays the point of
     // normal incidence, neither of which the swing moves - which is exactly what the PONI convention
     // wants, so the swing is a PONI rotation and nothing else in the header changes. It turns about
-    // the base spindle axis, the four-circle geometry these headers describe having the arm and the
-    // spindle on one axis; the imgCIF axis table such a header carries states the two with the same
-    // vector.
+    // the axis the header's table gives the arm, and otherwise about the base spindle axis: on the
+    // four-circle geometry these headers describe the arm and the spindle are one axis, and the one
+    // header here that states both states them with the same vector.
     if (header0_.two_theta_deg != 0.0) {
+        const Coord axis = header0_.detector_axis.has_value()
+                               ? ImgCIFToInternal(*header0_.detector_axis) : BaseAxis(header0_);
         float rot1 = 0, rot2 = 0, rot3 = 0;
-        PoniAnglesFromMatrix(RotMatrix(static_cast<float>(header0_.two_theta_deg * PI / 180.0), BASE_AXIS),
+        PoniAnglesFromMatrix(RotMatrix(static_cast<float>(header0_.two_theta_deg * PI / 180.0), axis),
                              rot1, rot2, rot3);
         dataset_->experiment.PoniRot1_rad(rot1).PoniRot2_rad(rot2).PoniRot3_rad(rot3);
     }

reader/MiniCBF.cpp

@@ -8,7 +8,9 @@
 #include <cstring>
 #include <fstream>
 #include <limits>
+#include <map>
 #include <regex>
+#include <stdexcept>
 
 #include "../common/JFJochException.h"
 
@@ -48,6 +50,133 @@ std::optional<double> Angle(const std::string &text, const char *pattern) {
     return v;
 }
 
+// One "loop_" of the imgCIF template block these headers carry, as rows keyed by tag. Tags and values
+// are both read as whitespace-separated tokens rather than by line, because a template packs several
+// tags onto one line ("_axis.vector[1] _axis.vector[2] _axis.vector[3]") and the rows that follow are
+// laid out to match the tags, not the lines.
+std::vector<std::map<std::string, std::string>> ParseLoop(const std::string &text, const std::string &tag) {
+    // The loop_ that introduces the tag, not the tag's own position: everything before it is another
+    // loop's data.
+    const size_t tag_at = text.find("\n" + tag);
+    if (tag_at == std::string::npos)
+        return {};
+    const size_t loop_at = text.rfind("loop_", tag_at);
+    if (loop_at == std::string::npos)
+        return {};
+
+    // Bare tokens, and the quoted ones a CIF value may be - a quoted value holding spaces would
+    // otherwise be counted as several columns and shift every row after it.
+    std::vector<std::string> tokens;
+    for (size_t i = loop_at + 5; i < text.size();) {
+        while ((i < text.size()) && std::isspace(static_cast<unsigned char>(text[i])))
+            i++;
+        if (i >= text.size())
+            break;
+        size_t end;
+        if ((text[i] == '\'') || (text[i] == '"')) {
+            end = text.find(text[i], i + 1);
+            if (end == std::string::npos)
+                break;
+            tokens.push_back(text.substr(i + 1, end - i - 1));
+            end++;
+        } else {
+            end = i;
+            while ((end < text.size()) && !std::isspace(static_cast<unsigned char>(text[end])))
+                end++;
+            tokens.push_back(text.substr(i, end - i));
+        }
+        // A second loop_, or a tag belonging to another category, ends this one.
+        if ((tokens.back() == "loop_")
+                || (tokens.back().starts_with("_") && !tokens.back().starts_with(tag.substr(0, tag.find('.') + 1)))) {
+            tokens.pop_back();
+            break;
+        }
+        i = end;
+    }
+
+    std::vector<std::string> names;
+    size_t first_value = 0;
+    while ((first_value < tokens.size()) && tokens[first_value].starts_with("_"))
+        names.push_back(tokens[first_value++]);
+    if (names.empty())
+        return {};
+
+    std::vector<std::map<std::string, std::string>> rows;
+    for (size_t i = first_value; i + names.size() <= tokens.size(); i += names.size()) {
+        std::map<std::string, std::string> row;
+        for (size_t j = 0; j < names.size(); j++)
+            row[names[j]] = tokens[i + j];
+        rows.push_back(std::move(row));
+    }
+    return rows;
+}
+
+std::string Field(const std::map<std::string, std::string> &row, const std::string &name) {
+    const auto it = row.find(name);
+    return (it == row.end()) ? std::string() : it->second;
+}
+
+// The imgCIF axis table: which axis turns or translates in which laboratory direction, and which two
+// axes the image's columns and rows run along. Absent from most headers, which say nothing about any
+// of this and are left exactly as they were read before.
+//
+// The element vectors are stated in the frame of the axis they depend on. Between them and the
+// detector's own rotation every header seen has translations only, so they describe the image in the
+// unswung detector frame - which is where the image orientation belongs, with the arm applied on top.
+void ParseAxisTable(const std::string &text, Header &h) {
+    const auto axes = ParseLoop(text, "_axis.id");
+    if (axes.empty())
+        return;
+
+    std::map<std::string, std::array<double, 3>> vector_of;
+    for (const auto &row: axes) {
+        const std::string id = Field(row, "_axis.id");
+        const std::string type = Field(row, "_axis.type");
+        const std::string equipment = Field(row, "_axis.equipment");
+        const std::string depends_on = Field(row, "_axis.depends_on");
+        std::array<double, 3> v{};
+        try {
+            for (int i = 0; i < 3; i++)
+                v[i] = std::stod(Field(row, "_axis.vector[" + std::to_string(i + 1) + "]"));
+        } catch (const std::exception &) {
+            continue;   // "." for a vector: the table states no direction for this axis
+        }
+        vector_of[id] = v;
+
+        // The base spindle and the detector arm are the rotations that hang off nothing: everything
+        // further in is carried by them. Naming neither, so a beamline is free to call them anything.
+        if ((type == "rotation") && (depends_on == ".")) {
+            if (equipment == "goniometer")
+                h.spindle_axis = v;
+            else if (equipment == "detector")
+                h.detector_axis = v;
+        }
+    }
+
+    // Which axis the fast index runs along, and which the slow, through the two tables that say so.
+    std::map<std::string, std::string> axis_of_set;
+    for (const auto &row: ParseLoop(text, "_array_structure_list_axis.axis_set_id"))
+        axis_of_set[Field(row, "_array_structure_list_axis.axis_set_id")]
+                = Field(row, "_array_structure_list_axis.axis_id");
+
+    for (const auto &row: ParseLoop(text, "_array_structure_list.array_id")) {
+        const std::string set = Field(row, "_array_structure_list.axis_set_id");
+        const auto id = axis_of_set.contains(set) ? axis_of_set[set] : set;
+        const auto it = vector_of.find(id);
+        if (it == vector_of.end())
+            continue;
+        std::array<double, 3> v = it->second;
+        if (Field(row, "_array_structure_list.direction") == "decreasing")
+            for (double &c: v)
+                c = -c;
+        const std::string index = Field(row, "_array_structure_list.index");
+        if (index == "1")
+            h.fast_direction = v;
+        else if (index == "2")
+            h.slow_direction = v;
+    }
+}
+
 int16_t ReadI16(const uint8_t *p) { int16_t v; std::memcpy(&v, p, 2); return v; }
 int32_t ReadI32(const uint8_t *p) { int32_t v; std::memcpy(&v, p, 4); return v; }
 int64_t ReadI64(const uint8_t *p) { int64_t v; std::memcpy(&v, p, 8); return v; }
@@ -99,6 +228,11 @@ Header ParseHeader(const char *data, size_t size) {
     h.period_s       = Num(t, R"(#\s*Exposure_period\s+([\d.eE+-]+))");
     h.count_cutoff   = Int(t, R"(#\s*Count_cutoff\s+(\d+))");
     h.axis_name      = Match(t, R"(#\s*Oscillation_axis\s+(\S+))").value_or("omega");
+    // "+SLOW" / "+FAST" on that same line: which of the image's two directions the spindle runs
+    // along. Some writers state that instead of an axis name, and it is the only thing a header with
+    // no axis table says about the spindle's direction at all.
+    h.spindle_along_slow = Match(t, R"(#\s*Oscillation_axis[^\r\n]*\+(SLOW|slow))").has_value();
+    ParseAxisTable(t, h);
 
     // "# Silicon sensor, ..." / "# CdTe sensor, ...". The rest of the code compares the material
     // against "CdTe" (BraggIntegrationEngine), so an unnormalised "Silicon" would silently give a

reader/MiniCBF.h

@@ -3,6 +3,7 @@
 
 #pragma once
 
+#include <array>
 #include <cstdint>
 #include <optional>
 #include <string>
@@ -44,6 +45,21 @@ struct Header {
     int64_t count_cutoff = 0;      // saturation
     std::string axis_name = "omega";
     bool byte_offset = false;      // the only conversion supported
+
+    // The imgCIF axis table the CBF template block carries after the "# " lines, as the file states
+    // it: vectors in the imgCIF laboratory frame (Z from the sample towards the source, Y opposite
+    // gravity, X completing a right-handed set). Turning them into any other frame is the caller's
+    // business, not this format reader's. Most headers carry no such table and leave all of these
+    // empty, which is not the same as their stating the usual mounting.
+    std::optional<std::array<double, 3>> fast_direction;   // laboratory direction of a +1 column step
+    std::optional<std::array<double, 3>> slow_direction;   // ... of a +1 row step
+    std::optional<std::array<double, 3>> spindle_axis;     // the base goniometer rotation axis
+    std::optional<std::array<double, 3>> detector_axis;    // the detector's own rotation axis: a 2theta arm
+    // "# Oscillation_axis X.CW +SLOW". Some writers say which of the two image directions the spindle
+    // runs along instead of stating a vector. The axis NAME on that line is a poor discriminator (see
+    // ScansPhi, and note that the one header carrying both a name and a table says "X" where its table
+    // says Y) but this token is not: where both are present they agree.
+    bool spindle_along_slow = false;
 };
 
 // Whether the sweep turns PHI rather than the base (omega) axis. What moved is the axis with a

tests/JFJochReaderTest.cpp

@@ -7,9 +7,14 @@
 #include "../common/ScanResultGenerator.h"
 #include "../writer/FileWriter.h"
 #include "../reader/JFJochHDF5Reader.h"
+#include "../reader/JFJochCBFReader.h"
+#include "../reader/MiniCBF.h"
 #include "../compression/JFJochCompressor.h"
 
+#include <fstream>
 #include <future>
+#include <iomanip>
+#include <sstream>
 
 TEST_CASE("HDF5DataType_Sign","[HDF5]") {
     HDF5DataType type_u8((uint8_t)0), type_fl(0.0f), type_i32((int32_t) 0), type_u32((uint32_t) 0);
@@ -3773,3 +3778,160 @@ TEST_CASE("JFJochReader_DetectorChainDoesNotDoubleTheTilt", "[HDF5][Full]") {
     remove("test_ponichain_data_000001.h5");
     REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
 }
+
+// A miniCBF header states more about the instrument than the "# " lines do: the CBF template block
+// some beamlines write carries a full imgCIF axis table, saying which laboratory direction the image's
+// columns and rows run along and which the spindle turns about. The reader assumed all three, and two
+// instruments in the corpus are not what it assumed - one stores its image a quarter turn round, the
+// other turns about the VERTICAL. Either way the spindle ends up 90 degrees from the image, which is
+// not a sign and so is beyond the axis-sign rescue; both indexed nothing at all.
+namespace {
+    // Two frames of a sweep whose pixels are all zero. Every delta of a zero image is zero, so the
+    // byte-offset stream is one 0x00 per pixel - which is a valid stream and enough to open a sweep.
+    void WriteMiniCBFSweep(const std::string &prefix, const std::string &header_body,
+                           int64_t nx, int64_t ny) {
+        for (int frame = 1; frame <= 2; frame++) {
+            std::ostringstream head;
+            head << "###CBF: VERSION 1.5\n_array_data.header_convention \"PILATUS_1.2\"\n"
+                 << "_array_data.header_contents\n"
+                 << "# Detector: PILATUS3 6M, S/N 60-0119\n"
+                 << "# Pixel_size 172e-6 m x 172e-6 m\n"
+                 << "# Silicon sensor, thickness 0.000450 m\n"
+                 << "# Exposure_time 0.1 s\n# Exposure_period 0.1 s\n# Count_cutoff 768595 counts\n"
+                 << "# Wavelength 0.96864 A\n# Detector_distance 0.33161 m\n"
+                 << "# Beam_xy (12.00, 8.00) pixels\n"
+                 << "# Start_angle " << (frame - 1) * 0.1 << " deg.\n# Angle_increment 0.1000 deg.\n"
+                 << "# Omega " << (frame - 1) * 0.1 << " deg.\n# Omega_increment 0.1000 deg.\n"
+                 << "# Phi 0.0000 deg.\n# Phi_increment 0.0000 deg.\n"
+                 << "# Chi 0.0000 deg.\n# Chi_increment 0.0000 deg.\n"
+                 << header_body
+                 << "_array_data.data\n--CIF-BINARY-FORMAT-SECTION--\n"
+                 << "Content-Type: application/octet-stream;\n"
+                 << "     conversions=\"x-CBF_BYTE_OFFSET\"\n"
+                 << "Content-Transfer-Encoding: BINARY\n"
+                 << "X-Binary-Size: " << nx * ny << "\n"
+                 << "X-Binary-Element-Type: \"signed 32-bit integer\"\n"
+                 << "X-Binary-Number-of-Elements: " << nx * ny << "\n"
+                 << "X-Binary-Size-Fastest-Dimension: " << nx << "\n"
+                 << "X-Binary-Size-Second-Dimension: " << ny << "\n\n";
+
+            std::ostringstream name;
+            name << prefix << "_" << std::setfill('0') << std::setw(4) << frame << ".cbf";
+            std::ofstream f(name.str(), std::ios::binary);
+            const std::string text = head.str();
+            f.write(text.data(), static_cast<std::streamsize>(text.size()));
+            f.write(reinterpret_cast<const char *>(minicbf::BINARY_SEPARATOR),
+                    sizeof(minicbf::BINARY_SEPARATOR));
+            const std::vector<char> zeros(static_cast<size_t>(nx * ny), 0);
+            f.write(zeros.data(), static_cast<std::streamsize>(zeros.size()));
+        }
+    }
+
+    void RemoveMiniCBFSweep(const std::string &prefix) {
+        for (int frame = 1; frame <= 2; frame++) {
+            std::ostringstream name;
+            name << prefix << "_" << std::setfill('0') << std::setw(4) << frame << ".cbf";
+            remove(name.str().c_str());
+        }
+    }
+
+    // The axis table in the form these headers write it, several tags to a line.
+    std::string AxisTable(const std::string &rows, int64_t nx, int64_t ny) {
+        return "loop_\n_axis.id\n_axis.type\n_axis.equipment\n_axis.depends_on\n"
+               "_axis.vector[1] _axis.vector[2] _axis.vector[3]\n"
+               "_axis.offset[1] _axis.offset[2] _axis.offset[3]\n"
+             + rows +
+               "loop_\n_array_structure_list.array_id\n_array_structure_list.index\n"
+               "_array_structure_list.dimension\n_array_structure_list.precedence\n"
+               "_array_structure_list.direction\n_array_structure_list.axis_set_id\n"
+               "ARRAY1 1 " + std::to_string(nx) + " 1 increasing ELEMENT_X\n"
+               "ARRAY1 2 " + std::to_string(ny) + " 2 increasing ELEMENT_Y\n"
+               "loop_\n_array_structure_list_axis.axis_set_id\n_array_structure_list_axis.axis_id\n"
+               "_array_structure_list_axis.displacement\n_array_structure_list_axis.displacement_increment\n"
+               "ELEMENT_X ELEMENT_X 0.0 0.1720\nELEMENT_Y ELEMENT_Y 0.0 0.1720\n";
+    }
+}
+
+TEST_CASE("JFJochCBFReader_AxisTableStatesTheMounting", "[HDF5][Full]") {
+    const int64_t nx = 24, ny = 16;
+
+    // A header that states nothing: the assumption, and the behaviour of nearly every file there is.
+    SECTION("no table, no hint - the assumption stands") {
+        WriteMiniCBFSweep("cbfaxis_plain", "# Detector_2theta 0.0000 deg.\n# Oscillation_axis OMEGA\n",
+                          nx, ny);
+        JFJochCBFReader reader;
+        REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_plain_0001.cbf"));
+        const auto x = reader.GetDataset()->experiment;
+        REQUIRE(x.GetGoniometer().has_value());
+        CHECK((x.GetGoniometer()->GetAxis() - Coord(-1, 0, 0)).Length() < 1e-6f);
+        CHECK(x.GetDetectorSetup().GetImageOrientation().IsIdentity());
+        CHECK(x.GetDiffractionGeometry().GetPoniRot2_rad() == 0.0f);
+        reader.Close();
+        RemoveMiniCBFSweep("cbfaxis_plain");
+    }
+
+    // A spindle that turns about the VERTICAL, with the image mounted the usual way round. imgCIF Y is
+    // up and the internal frame's y is down, so the stated (0,1,0) is internal (0,-1,0) - and NOT the
+    // (-1,0,0) that was assumed, which is 90 degrees away and indexes nothing.
+    SECTION("vertical spindle, standard image") {
+        WriteMiniCBFSweep("cbfaxis_vert",
+                          "# Detector_2theta 0.0000 deg.\n# Oscillation_axis X.CW +SLOW\n"
+                          + AxisTable("GON_OMEGA rotation goniometer . 0 1 0 . . .\n"
+                                      "DET_Z translation detector . 0 0 -1 0 0 0\n"
+                                      "ELEMENT_X translation detector DET_Z 1 0 0 -1 1 0\n"
+                                      "ELEMENT_Y translation detector ELEMENT_X 0 -1 0 0 0 0\n", nx, ny),
+                          nx, ny);
+        JFJochCBFReader reader;
+        REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_vert_0001.cbf"));
+        const auto x = reader.GetDataset()->experiment;
+        REQUIRE(x.GetGoniometer().has_value());
+        CHECK((x.GetGoniometer()->GetAxis() - Coord(0, -1, 0)).Length() < 1e-6f);
+        // The image itself is standard, so nothing about it is turned - the axis was the whole error.
+        CHECK(x.GetDetectorSetup().GetImageOrientation().IsIdentity());
+        reader.Close();
+        RemoveMiniCBFSweep("cbfaxis_vert");
+    }
+
+    // The same vertical spindle, stated only by the "+SLOW" token, which is all a header with no axis
+    // table says. Two datasets from that instrument are in this state.
+    SECTION("vertical spindle from the +SLOW token alone") {
+        WriteMiniCBFSweep("cbfaxis_slow", "# Detector_2theta 0.0000 deg.\n# Oscillation_axis X.CW +SLOW\n",
+                          nx, ny);
+        JFJochCBFReader reader;
+        REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_slow_0001.cbf"));
+        const auto x = reader.GetDataset()->experiment;
+        REQUIRE(x.GetGoniometer().has_value());
+        CHECK((x.GetGoniometer()->GetAxis() - Coord(0, -1, 0)).Length() < 1e-6f);
+        reader.Close();
+        RemoveMiniCBFSweep("cbfaxis_slow");
+    }
+
+    // An image stored a quarter turn round, on a detector swung out to 30 degrees. The two are read
+    // together or not at all: the arm turns about a laboratory axis, and which way that runs across
+    // the stored image is exactly what the mounting says.
+    SECTION("quarter-turned image on a swung arm") {
+        WriteMiniCBFSweep("cbfaxis_turn",
+                          "# Detector_2theta 30.0000 deg.\n# Oscillation_axis OMEGA\n"
+                          + AxisTable("GON_OMEGA rotation goniometer . 1 0 0 . . .\n"
+                                      "DET_2THETA rotation detector . 1 0 0 . . .\n"
+                                      "DET_Z translation detector DET_2THETA 0 0 -1 0 0 0\n"
+                                      "ELEMENT_X translation detector DET_Z 0 1 0 -1 1 0\n"
+                                      "ELEMENT_Y translation detector ELEMENT_X 1 0 0 0 0 0\n", nx, ny),
+                          nx, ny);
+        JFJochCBFReader reader;
+        REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_turn_0001.cbf"));
+        const auto x = reader.GetDataset()->experiment;
+        REQUIRE(x.GetGoniometer().has_value());
+        CHECK((x.GetGoniometer()->GetAxis() - Coord(1, 0, 0)).Length() < 1e-6f);
+        // fast = imgCIF (0,1,0) = internal (0,-1,0), slow = imgCIF (1,0,0) = internal (1,0,0)
+        CHECK(x.GetDetectorSetup().GetImageOrientation() == DetectorOrientation(false, 3));
+        // and the arm turns about its own stated axis, internal +x, by the stated 30 degrees
+        const auto geom = x.GetDiffractionGeometry();
+        const RotMatrix expected = RotMatrix(static_cast<float>(30.0 * PI / 180.0), {1, 0, 0})
+                                 * DetectorOrientation(false, 3).Matrix();
+        for (int64_t column = 0; column < 3; column++)
+            CHECK((geom.GetDetectorMatrix().Column(column) - expected.Column(column)).Length() < 1e-5f);
+        reader.Close();
+        RemoveMiniCBFSweep("cbfaxis_turn");
+    }
+}