grid scan: say how the stationary spindle angle is stated, and pin it

A grid scan is a set of stills at a stationary spindle, and the angle it stood
at is what relates one grid to another taken elsewhere on the circle. Users were
finding an all-zero omega in the file and concluding the angle could not be
recorded at all.

It already can, and has since the goniometer and the grid scan stopped being
alternatives: send the axis with step 0 and its start angle, and that angle is
written per image into the NXmx sample chain, read back by reader/, and taken by
dials.import as a set of stills. Measured on a generated 12-image grid: the
placeholder file carries omega = 0 x 12, the same file with the axis sent at
step 0 carries omega = 90 x 12, and dials.import reports "still: 1, sweep: 0"
for both. Nothing in the code needed changing, so nothing was; what was missing
was that nobody could tell, and that no test held the behaviour down.

So: the API and the HDF5 documentation now say it in as many words, and three
tests pin the three legs the value crosses - the OpenAPI request (which used to
drop the grid scan whenever an axis was present, unpinned until now), the CBOR
start message, and the file round trip.

Also corrects a claim two comments and the HDF5 page were making. NXmx can
express "no rotation" perfectly well - a sample may depend_on "." - so the
placeholder is not there for the standard's sake. It is there because dxtbx
cannot read a sample chain of translations alone: strip the rotation axis from a
grid scan master and dials.import dies in get_dxtbx_goniometer with a matmul
dimension mismatch. Recorded so nobody removes the placeholder on the strength
of the standard.

One thing the change does not fix, because it cannot: a stationary angle is
invisible to DIALS when a grid scan is present. dxtbx picks the first varying
axis as the scan axis, which is a grid translation, so the oscillation reads
(0, 0); and with exactly one rotation axis in the chain it builds a single-axis
goniometer whose fixed rotation is the identity, never consulting the angle. The
same angle IS visible when it is the only candidate (oscillation reads (90, 0))
or when a Smargon head puts a second rotation axis in the chain (the setting
rotation then carries it). The value is in the file and correct either way.

broker/gen/model/Grid_scan.h

@@ -12,7 +12,7 @@
 /*
  * Grid_scan.h
  *
- * Definition of a grid scan. May be combined with a goniometer axis: a grid is often collected at a particular head position, and a stationary axis records where that was. 
+ * Definition of a grid scan. Combine it with a goniometer axis to state the angle the spindle stood at: send `goniometer` with `step` 0 and the `start` angle of the grid scan, and that angle is written per image into the NXmx sample transformation chain. Without one the spindle is recorded at 0, which says that nobody stated an angle rather than that the spindle stood at 0. 
  */
 
 #ifndef Grid_scan_H_
@@ -25,7 +25,7 @@ namespace org::openapitools::server::model
 {
 
 /// <summary>
-/// Definition of a grid scan. May be combined with a goniometer axis: a grid is often collected at a particular head position, and a stationary axis records where that was. 
+/// Definition of a grid scan. Combine it with a goniometer axis to state the angle the spindle stood at: send &#x60;goniometer&#x60; with &#x60;step&#x60; 0 and the &#x60;start&#x60; angle of the grid scan, and that angle is written per image into the NXmx sample transformation chain. Without one the spindle is recorded at 0, which says that nobody stated an angle rather than that the spindle stood at 0. 
 /// </summary>
 class  Grid_scan
 {

broker/gen/model/Rotation_axis.h

@@ -67,7 +67,7 @@ public:
     bool nameIsSet() const;
     void unsetName();
     /// <summary>
-    /// Angle step (per image) in degrees
+    /// Angle step (per image) in degrees. 0 for an axis that does not turn: the axis then records the angle the spindle stood at, which is how a grid scan or a set of stills states its head position. 
     /// </summary>
     float getStep() const;
     void setStep(float const value);

broker/jfjoch_api.yaml

@@ -241,8 +241,11 @@ components:
   schemas:
     grid_scan:
       description: |
-        Definition of a grid scan. May be combined with a goniometer axis: a grid is often collected
-        at a particular head position, and a stationary axis records where that was.
+        Definition of a grid scan. Combine it with a goniometer axis to state the angle the spindle
+        stood at: send `goniometer` with `step` 0 and the `start` angle of the grid scan, and that
+        angle is written per image into the NXmx sample transformation chain. Without one the
+        spindle is recorded at 0, which says that nobody stated an angle rather than that the
+        spindle stood at 0.
       type: object
       required:
         - n_fast
@@ -298,7 +301,10 @@ components:
           type: number
           format: float
           example: 0.1
-          description: Angle step (per image) in degrees
+          description: |
+            Angle step (per image) in degrees. 0 for an axis that does not turn: the axis then
+            records the angle the spindle stood at, which is how a grid scan or a set of stills
+            states its head position.
         start:
           type: number
           format: float

common/DiffractionExperiment.cpp

@@ -1535,8 +1535,10 @@ std::vector<DetectorTransformation> DiffractionExperiment::BuildTransformationCh
             axis.Value(goniometer->GetStart_deg());
         add(std::move(axis));
     } else if (GetGridScan().has_value()) {
-        // A grid scan still sits on a spindle that simply does not turn. NXmx cannot say "there is no
-        // rotation", and a sample chain of translations alone is not something readers accept.
+        // A grid scan still sits on a spindle that simply does not turn, and 0 is a placeholder for
+        // an angle nobody told us: send the axis with step 0 and the position it really stood at is
+        // written instead. NXmx does allow a sample with no goniometer at all (depends_on "."), but
+        // a sample chain of translations alone is not something readers accept - dxtbx raises on it.
         add(DetectorTransformation("omega", TransformationType::Rotation, {-1, 0, 0}).Value(0.0f));
     }
 

docs/CBOR.md

@@ -59,7 +59,7 @@ There are minor differences at the moment:
 | - - axis                         | Array(float)         | Vector for the rotation axis                                                                                                                   |                           |
 | - - helical_step                 | Array(float)         | Translation for helical scan for 1 image \[m\]                                                                                                 |                           |
 | - - screening_wedge              | Array(float)         | Wedge for screening \[deg\] (increment would correspond to difference between screening points)                                                |                           |
-| grid_scan                        | object               | Grid scan definition (optional). May be sent together with `goniometer`: a grid is often collected at a given head position, recorded as a stationary axis (step 0)                |                           |
+| grid_scan                        | object               | Grid scan definition (optional). Send `goniometer` with it, `increment` 0, to state the angle the spindle stood at; without one the spindle is recorded at 0, meaning "nobody said"                |                           |
 | - n_fast                         | uint64               | Number of elements along fast axis                                                                                                             |                           |
 | - n_slow                         | uint64               | Number of elements along slow axis                                                                                                             |                           |
 | - step_x_axis                    | float                | Step along X axis, can be negative \[m\]                                                                                                       |                           |

docs/CHANGELOG.md

@@ -30,6 +30,7 @@
 * `dataset_settings` takes `beam_size_x_um`/`beam_size_y_um`, the size of the X-ray beam at the sample, and `jfjoch_writer` writes them as `incident_beam_size` in the HDF5 master.
 * `dataset_settings` accepts any `smargon.chi_deg`, which was restricted to 0-90 degrees.
 * `jfjoch_viewer` reads PILATUS miniCBF sweeps natively, and draws grid scan cells in the proportion of the scan steps.
+* The HDF5 and API documentation say how a grid scan records the angle its spindle stood at: send the goniometer axis with a step of 0.
 * The rugnux manual is reorganised into task pages with a run overview, and gains worked phenix / REFMAC5 / POINTLESS-AIMLESS / careless examples.
 * `jfjoch_viewer` labels the merge-statistics plot over the range the axis is drawn on, so the CC1/2 curve is no longer read against tick labels covering only part of it.
 

docs/HDF5.md

@@ -275,6 +275,14 @@ the one nearest the sample. It may hold, in that order: the grid-scan translatio
 `grid_scan_y`, the spindle, and a Smargon head's `chi` and `phi`. A grid scan and a goniometer axis
 are **not** alternatives; both can be present.
 
+A **grid scan is collected at a stationary spindle, and the angle it stood at is stated by sending
+the goniometer axis with a step of 0** — in `dataset_settings.goniometer`, or as the `goniometer`
+map of the CBOR start message. The angle is then written per image as `omega` (or whatever the axis
+is named) in the chain above, and read back by `reader/`. Send no axis and the spindle is still
+recorded, at 0: NXmx would allow a sample with no goniometer at all (`depends_on = "."`), but a
+sample chain of translations alone is not something readers accept, so the placeholder is written
+whether or not the angle is known. **A 0 there means "nobody said", not "the spindle was at 0".**
+
 A **Smargon head position is told apart from the spindle** by the `equipment_component` attribute,
 which is `"smargon"` on `chi` and `phi` and absent on the spindle. This is load-bearing: a spindle can
 itself be named `phi`, and without the attribute a reader would take a head position for the scan

docs/python_client/docs/GridScan.md

@@ -1,6 +1,6 @@
 # GridScan
 
-Definition of a grid scan. May be combined with a goniometer axis: a grid is often collected at a particular head position, and a stationary axis records where that was. 
+Definition of a grid scan. Combine it with a goniometer axis to state the angle the spindle stood at: send `goniometer` with `step` 0 and the `start` angle of the grid scan, and that angle is written per image into the NXmx sample transformation chain. Without one the spindle is recorded at 0, which says that nobody stated an angle rather than that the spindle stood at 0. 
 
 ## Properties
 

docs/python_client/docs/RotationAxis.md

@@ -7,7 +7,7 @@ Definition of a crystal rotation axis
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **name** | **str** | Name of rotation axis (e.g., omega, phi) | [optional] [default to 'omega']
-**step** | **float** | Angle step (per image) in degrees | 
+**step** | **float** | Angle step (per image) in degrees. 0 for an axis that does not turn: the axis then records the angle the spindle stood at, which is how a grid scan or a set of stills states its head position.  | 
 **start** | **float** | Start angle in degrees | [optional] [default to 0]
 **vector** | **List[float]** | Rotation axis | 
 **helical_step_um** | **List[float]** | Translation (per image) for helical scan | [optional] 

frontend/src/client/types.gen.ts

@@ -5,8 +5,11 @@ export type ClientOptions = {
 };
 
 /**
- * Definition of a grid scan. May be combined with a goniometer axis: a grid is often collected
- * at a particular head position, and a stationary axis records where that was.
+ * Definition of a grid scan. Combine it with a goniometer axis to state the angle the spindle
+ * stood at: send `goniometer` with `step` 0 and the `start` angle of the grid scan, and that
+ * angle is written per image into the NXmx sample transformation chain. Without one the
+ * spindle is recorded at 0, which says that nobody stated an angle rather than that the
+ * spindle stood at 0.
  *
  */
 export type grid_scan = {
@@ -50,7 +53,10 @@ export type rotation_axis = {
      */
     name?: string;
     /**
-     * Angle step (per image) in degrees
+     * Angle step (per image) in degrees. 0 for an axis that does not turn: the axis then
+     * records the angle the spindle stood at, which is how a grid scan or a set of stills
+     * states its head position.
+     *
      */
     step: number;
     /**

frontend/src/client/zod.gen.ts

@@ -3,8 +3,11 @@
 import * as z from 'zod';
 
 /**
- * Definition of a grid scan. May be combined with a goniometer axis: a grid is often collected
- * at a particular head position, and a stationary axis records where that was.
+ * Definition of a grid scan. Combine it with a goniometer axis to state the angle the spindle
+ * stood at: send `goniometer` with `step` 0 and the `start` angle of the grid scan, and that
+ * angle is written per image into the NXmx sample transformation chain. Without one the
+ * spindle is recorded at 0, which says that nobody stated an angle rather than that the
+ * spindle stood at 0.
  *
  */
 export const zGridScan = z.object({

tests/CBORTest.cpp

@@ -233,6 +233,32 @@ TEST_CASE("CBORSerialize_Start_GridScan", "[CBOR]") {
     CHECK(output_message.grid_scan->GetGridElemSlow_um() == Catch::Approx(grid.GetGridElemSlow_um()));
 }
 
+// A grid scan and a goniometer axis travel together on the wire: the axis with increment 0 is how
+// the angle the spindle stood at reaches the file.
+TEST_CASE("CBORSerialize_Start_GridScanAtStationaryAxis", "[CBOR]") {
+    GridScanSettings grid(20, 5.0, 5.0, false, false);
+    GoniometerAxis axis("omega", 90.0f, 0.0f, Coord(-1, 0, 0), {});
+    StartMessage message{
+        .grid_scan = grid,
+        .goniometer = axis
+    };
+    std::vector<uint8_t> buffer(8 * 1024 * 1024);
+    CBORStream2Serializer serializer(buffer.data(), buffer.size());
+    REQUIRE_NOTHROW(serializer.SerializeSequenceStart(message));
+
+    auto deserialized = CBORStream2Deserialize(buffer.data(), serializer.GetBufferSize());
+    REQUIRE(deserialized);
+    REQUIRE(deserialized->start_message);
+    StartMessage &output_message = *deserialized->start_message;
+
+    REQUIRE(output_message.grid_scan.has_value());
+    CHECK(output_message.grid_scan->GetNFast() == grid.GetNFast());
+    REQUIRE(output_message.goniometer.has_value());
+    CHECK(output_message.goniometer->GetName() == "omega");
+    CHECK(!output_message.goniometer->IsScanning());
+    CHECK(output_message.goniometer->GetStart_deg() == Catch::Approx(90.0f));
+}
+
 TEST_CASE("CBORSerialize_Start_ThresholdEnergy", "[CBOR]") {
     std::vector<uint8_t> buffer(8 * 1024 * 1024);
     CBORStream2Serializer serializer(buffer.data(), buffer.size());

tests/DiffractionExperimentTest.cpp

@@ -8,7 +8,9 @@
 #include "../common/NetworkAddressConvert.h"
 
 #include <nlohmann/json.hpp>
+#include "Dataset_settings.h"
 #include "Dataset_settings_smargon.h"
+#include "../broker/OpenAPIConvert.h"
 
 using namespace std::literals::chrono_literals;
 
@@ -1238,3 +1240,25 @@ TEST_CASE("DiffractionExperiment_SmargonChiReadoutNoise", "[DiffractionExperimen
     REQUIRE(noisy.GetValues().size() == zero.GetValues().size());
     CHECK(noisy.GetValues().at(0) == Catch::Approx(zero.GetValues().at(0)).margin(1e-6));
 }
+
+// A grid scan and a goniometer axis are not alternatives: a grid is collected at a stationary
+// spindle, and the axis sent with step 0 is what states the angle it stood at. The request used to
+// drop the grid scan whenever an axis was present, and nothing pinned it.
+TEST_CASE("DiffractionExperiment_GridScanAtStationaryAxis", "[DiffractionExperiment]") {
+    org::openapitools::server::model::Dataset_settings model;
+    from_json(nlohmann::json::parse(R"({
+        "images_per_trigger": 12,
+        "beam_x_pxl": 512.0, "beam_y_pxl": 256.0,
+        "detector_distance_mm": 150.0, "incident_energy_keV": 12.4,
+        "grid_scan": {"n_fast": 4, "step_x_um": 20.0, "step_y_um": 20.0},
+        "goniometer": {"name": "omega", "start": 90.0, "step": 0.0, "vector": [-1, 0, 0]}
+    })"), model);
+
+    const auto settings = Convert(model);
+    REQUIRE(settings.GetGridScan().has_value());
+    CHECK(settings.GetGridScan()->GetNFast() == 4);
+    REQUIRE(settings.GetGoniometer().has_value());
+    CHECK(settings.GetGoniometer()->GetName() == "omega");
+    CHECK(!settings.GetGoniometer()->IsScanning());
+    CHECK(settings.GetGoniometer()->GetStart_deg() == Catch::Approx(90.0f));
+}

tests/JFJochReaderTest.cpp

@@ -3365,6 +3365,24 @@ TEST_CASE("JFJochReader_AxisRecovery", "[HDF5][Full]") {
         cleanup("test_ax_grid");
     }
 
+    // A grid scan is taken at a stationary spindle, and the angle it stood at is what relates one
+    // grid to another taken elsewhere on the circle. It is stated by sending the axis with step 0;
+    // send nothing and the spindle is recorded at 0, which says only that nobody told us.
+    SECTION("a grid scan at a stationary head position") {
+        DiffractionExperiment x(DetJF(1));
+        x.ImagesPerTrigger(6).GridScan(GridScanSettings(3, 10.0f, 20.0f, false, false).ImageNum(6))
+                .Goniometer(GoniometerAxis("omega", 90.0f, 0.0f, Coord(-1,0,0), {}));
+        const auto out = round_trip(x, "test_ax_gridstill");
+        REQUIRE(out.GetGridScan().has_value());
+        CHECK(out.GetGridScan()->GetNFast() == 3);
+        REQUIRE(out.GetGoniometer().has_value());
+        CHECK(out.GetGoniometer()->GetName() == "omega");
+        CHECK(!out.GetGoniometer()->IsScanning());
+        CHECK(out.GetGoniometer()->GetStart_deg() == Catch::Approx(90).margin(1e-3));
+        CHECK(out.GetGoniometer()->GetAxis() == Coord(-1,0,0));
+        cleanup("test_ax_gridstill");
+    }
+
     SECTION("a grid scan under a turning spindle") {
         DiffractionExperiment x(DetJF(1));
         x.ImagesPerTrigger(6).GridScan(GridScanSettings(3, 10.0f, 20.0f, false, false).ImageNum(6))

writer/HDF5NXmx.cpp

@@ -959,10 +959,13 @@ void NXmx::Sample(const StartMessage &start, const EndMessage &end) {
             depends_on = "/entry/sample/transformations/" + start.goniometer->GetName();
         } else if (write_grid_scan) {
             // No axis was given, but the sample still sits on a spindle - it simply does not turn.
-            // Say so: NXmx cannot express "no rotation", and a chain of translations alone is not
-            // something readers accept (dxtbx raises on it outright). At 0 degrees the rotation is
-            // the identity whatever the axis points along, so the conventional vector below carries
-            // no geometric claim.
+            // Say so: a chain of translations alone is not something readers accept (dxtbx raises on
+            // it outright, in get_dxtbx_goniometer). NXmx itself would allow a sample with no
+            // goniometer, depends_on ".", so this is a reader constraint and not a standard one. At
+            // 0 degrees the rotation is the identity whatever the axis points along, so the
+            // conventional vector below carries no geometric claim - but 0 is a placeholder for an
+            // angle nobody told us. Send the axis with step 0 and the angle the spindle really stood
+            // at is written instead, which is how a grid scan records its head position.
             //
             // One entry per image, for the reason write_smargon gives: a reader takes the image
             // count from the innermost axis of the chain, and with no goniometer and no Smargon head