mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 73 Wiki Activity

reader: take a miniCBF's mounting from the imgCIF axis table its header states

Browse Source 
A miniCBF header states three things about how the instrument is put together
that the reader was assuming instead: which laboratory direction the image's
columns run along, which its rows run along, and which the spindle turns about.
Some beamlines append a CBF template block holding the full imgCIF axis table,
which says all three outright.

Two instruments in the corpus are not what was assumed, in two different ways.
One mounts its detector a quarter turn round, so the image's columns run
vertically. Another turns its spindle about the VERTICAL, with the image mounted
the usual way; its table says so, and its "# Oscillation_axis" line says so a
second way, by naming the image direction the spindle runs along rather than a
vector. Either error leaves the spindle 90 degrees from the image. That is not a
sign, so the run's axis-sign rescue cannot reach it, and no refinement recovers
it: all three affected sweeps indexed nothing usable.

So the table is read. The element axes give the image orientation, matched against
the eight discrete mountings exactly as the NXmx module directions already are -
the match itself moves to DetectorOrientation, so both readers share one
definition rather than two copies. The goniometer axis with no parent gives the
spindle DIRECTION; its sign stays the rescue's business, which is the part a
convention can legitimately differ on. The detector axis with no parent gives the
2theta arm, replacing the assumption that the arm shares the spindle's axis - the
one header stating both states them with the same vector, so this changes no
answer, only what it rests on. imgCIF's frame differs from the internal one by a
half turn about x, a rotation and not a mirror, as writer/HDF5NXmx.cpp already
records from the other side.

Where a header carries no table, a "+SLOW" on the Oscillation_axis line still
says the spindle runs along the image's slow direction. That is the only thing one
of the three affected sets says about it. The axis NAME on that line stays
unusable - the header that carries both says "X.CW" where its own table says Y -
but the direction token is not: where both are present they agree, which is what
makes reading it evidence rather than a guess.

Also: naming a frame with no directory at all now finds its sweep. parent_path()
of a bare filename is empty and iterating an empty path finds nothing, so running
from inside the data directory reported that no images were found.

Measured, with nothing on the command line. The vertical-spindle protein set goes
from no usable lattice to 100% indexed, P 6(3) 2 2 with a cell 0.43% from
deposited, 87846 reflections at 86.3% completeness and CC(1/2) 0.995. Its
companion from the same detector, which has no table and only the +SLOW token,
goes from a spurious monoclinic cell at 2.3% completeness and I/sigma 0.21 to the
right orthorhombic lattice, 97.7% indexed, 59.7% complete, CC(1/2) 0.996. The
quarter-turned set's three sweeps, at three arm positions, now all index without
the hand-passed quarter turn they needed and agree on one cell to 0.03 A. Six
miniCBF sets that state no table and no +SLOW - including one whose
Oscillation_axis line names an axis in a third dialect - are byte-identical in
.hkl, .mtz, .cif and the image statistics, as are two NXmx sets, which is the
shared orientation matcher moving nothing on that path either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f
This commit is contained in:
leonarski_f
2026-08-30 14:03:32 +02:00
co-authored by Claude Opus 5
parent 5c44e544dc
commit a27c4cf26f
9 changed files with 419 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/JFJochReaderTest.cpp
+162
View File
@@ -7,9 +7,14 @@
#include "../common/ScanResultGenerator.h"
#include "../writer/FileWriter.h"
#include "../reader/JFJochHDF5Reader.h"
#include "../reader/JFJochCBFReader.h"
#include "../reader/MiniCBF.h"
#include "../compression/JFJochCompressor.h"
#include <fstream>
#include <future>
#include <iomanip>
#include <sstream>
TEST_CASE("HDF5DataType_Sign","[HDF5]") {
HDF5DataType type_u8((uint8_t)0), type_fl(0.0f), type_i32((int32_t) 0), type_u32((uint32_t) 0);
@@ -3773,3 +3778,160 @@ TEST_CASE("JFJochReader_DetectorChainDoesNotDoubleTheTilt", "[HDF5][Full]") {
remove("test_ponichain_data_000001.h5");
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
}
// A miniCBF header states more about the instrument than the "# " lines do: the CBF template block
// some beamlines write carries a full imgCIF axis table, saying which laboratory direction the image's
// columns and rows run along and which the spindle turns about. The reader assumed all three, and two
// instruments in the corpus are not what it assumed - one stores its image a quarter turn round, the
// other turns about the VERTICAL. Either way the spindle ends up 90 degrees from the image, which is
// not a sign and so is beyond the axis-sign rescue; both indexed nothing at all.
namespace {
// Two frames of a sweep whose pixels are all zero. Every delta of a zero image is zero, so the
// byte-offset stream is one 0x00 per pixel - which is a valid stream and enough to open a sweep.
void WriteMiniCBFSweep(const std::string &prefix, const std::string &header_body,
int64_t nx, int64_t ny) {
for (int frame = 1; frame <= 2; frame++) {
std::ostringstream head;
head << "###CBF: VERSION 1.5\n_array_data.header_convention \"PILATUS_1.2\"\n"
<< "_array_data.header_contents\n"
<< "# Detector: PILATUS3 6M, S/N 60-0119\n"
<< "# Pixel_size 172e-6 m x 172e-6 m\n"
<< "# Silicon sensor, thickness 0.000450 m\n"
<< "# Exposure_time 0.1 s\n# Exposure_period 0.1 s\n# Count_cutoff 768595 counts\n"
<< "# Wavelength 0.96864 A\n# Detector_distance 0.33161 m\n"
<< "# Beam_xy (12.00, 8.00) pixels\n"
<< "# Start_angle " << (frame - 1) * 0.1 << " deg.\n# Angle_increment 0.1000 deg.\n"
<< "# Omega " << (frame - 1) * 0.1 << " deg.\n# Omega_increment 0.1000 deg.\n"
<< "# Phi 0.0000 deg.\n# Phi_increment 0.0000 deg.\n"
<< "# Chi 0.0000 deg.\n# Chi_increment 0.0000 deg.\n"
<< header_body
<< "_array_data.data\n--CIF-BINARY-FORMAT-SECTION--\n"
<< "Content-Type: application/octet-stream;\n"
<< " conversions=\"x-CBF_BYTE_OFFSET\"\n"
<< "Content-Transfer-Encoding: BINARY\n"
<< "X-Binary-Size: " << nx * ny << "\n"
<< "X-Binary-Element-Type: \"signed 32-bit integer\"\n"
<< "X-Binary-Number-of-Elements: " << nx * ny << "\n"
<< "X-Binary-Size-Fastest-Dimension: " << nx << "\n"
<< "X-Binary-Size-Second-Dimension: " << ny << "\n\n";
std::ostringstream name;
name << prefix << "_" << std::setfill('0') << std::setw(4) << frame << ".cbf";
std::ofstream f(name.str(), std::ios::binary);
const std::string text = head.str();
f.write(text.data(), static_cast<std::streamsize>(text.size()));
f.write(reinterpret_cast<const char *>(minicbf::BINARY_SEPARATOR),
sizeof(minicbf::BINARY_SEPARATOR));
const std::vector<char> zeros(static_cast<size_t>(nx * ny), 0);
f.write(zeros.data(), static_cast<std::streamsize>(zeros.size()));
}
}
void RemoveMiniCBFSweep(const std::string &prefix) {
for (int frame = 1; frame <= 2; frame++) {
std::ostringstream name;
name << prefix << "_" << std::setfill('0') << std::setw(4) << frame << ".cbf";
remove(name.str().c_str());
}
}
// The axis table in the form these headers write it, several tags to a line.
std::string AxisTable(const std::string &rows, int64_t nx, int64_t ny) {
return "loop_\n_axis.id\n_axis.type\n_axis.equipment\n_axis.depends_on\n"
"_axis.vector[1] _axis.vector[2] _axis.vector[3]\n"
"_axis.offset[1] _axis.offset[2] _axis.offset[3]\n"
+ rows +
"loop_\n_array_structure_list.array_id\n_array_structure_list.index\n"
"_array_structure_list.dimension\n_array_structure_list.precedence\n"
"_array_structure_list.direction\n_array_structure_list.axis_set_id\n"
"ARRAY1 1 " + std::to_string(nx) + " 1 increasing ELEMENT_X\n"
"ARRAY1 2 " + std::to_string(ny) + " 2 increasing ELEMENT_Y\n"
"loop_\n_array_structure_list_axis.axis_set_id\n_array_structure_list_axis.axis_id\n"
"_array_structure_list_axis.displacement\n_array_structure_list_axis.displacement_increment\n"
"ELEMENT_X ELEMENT_X 0.0 0.1720\nELEMENT_Y ELEMENT_Y 0.0 0.1720\n";
}
}
TEST_CASE("JFJochCBFReader_AxisTableStatesTheMounting", "[HDF5][Full]") {
const int64_t nx = 24, ny = 16;
// A header that states nothing: the assumption, and the behaviour of nearly every file there is.
SECTION("no table, no hint - the assumption stands") {
WriteMiniCBFSweep("cbfaxis_plain", "# Detector_2theta 0.0000 deg.\n# Oscillation_axis OMEGA\n",
nx, ny);
JFJochCBFReader reader;
REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_plain_0001.cbf"));
const auto x = reader.GetDataset()->experiment;
REQUIRE(x.GetGoniometer().has_value());
CHECK((x.GetGoniometer()->GetAxis() - Coord(-1, 0, 0)).Length() < 1e-6f);
CHECK(x.GetDetectorSetup().GetImageOrientation().IsIdentity());
CHECK(x.GetDiffractionGeometry().GetPoniRot2_rad() == 0.0f);
reader.Close();
RemoveMiniCBFSweep("cbfaxis_plain");
}
// A spindle that turns about the VERTICAL, with the image mounted the usual way round. imgCIF Y is
// up and the internal frame's y is down, so the stated (0,1,0) is internal (0,-1,0) - and NOT the
// (-1,0,0) that was assumed, which is 90 degrees away and indexes nothing.
SECTION("vertical spindle, standard image") {
WriteMiniCBFSweep("cbfaxis_vert",
"# Detector_2theta 0.0000 deg.\n# Oscillation_axis X.CW +SLOW\n"
+ AxisTable("GON_OMEGA rotation goniometer . 0 1 0 . . .\n"
"DET_Z translation detector . 0 0 -1 0 0 0\n"
"ELEMENT_X translation detector DET_Z 1 0 0 -1 1 0\n"
"ELEMENT_Y translation detector ELEMENT_X 0 -1 0 0 0 0\n", nx, ny),
nx, ny);
JFJochCBFReader reader;
REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_vert_0001.cbf"));
const auto x = reader.GetDataset()->experiment;
REQUIRE(x.GetGoniometer().has_value());
CHECK((x.GetGoniometer()->GetAxis() - Coord(0, -1, 0)).Length() < 1e-6f);
// The image itself is standard, so nothing about it is turned - the axis was the whole error.
CHECK(x.GetDetectorSetup().GetImageOrientation().IsIdentity());
reader.Close();
RemoveMiniCBFSweep("cbfaxis_vert");
}
// The same vertical spindle, stated only by the "+SLOW" token, which is all a header with no axis
// table says. Two datasets from that instrument are in this state.
SECTION("vertical spindle from the +SLOW token alone") {
WriteMiniCBFSweep("cbfaxis_slow", "# Detector_2theta 0.0000 deg.\n# Oscillation_axis X.CW +SLOW\n",
nx, ny);
JFJochCBFReader reader;
REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_slow_0001.cbf"));
const auto x = reader.GetDataset()->experiment;
REQUIRE(x.GetGoniometer().has_value());
CHECK((x.GetGoniometer()->GetAxis() - Coord(0, -1, 0)).Length() < 1e-6f);
reader.Close();
RemoveMiniCBFSweep("cbfaxis_slow");
}
// An image stored a quarter turn round, on a detector swung out to 30 degrees. The two are read
// together or not at all: the arm turns about a laboratory axis, and which way that runs across
// the stored image is exactly what the mounting says.
SECTION("quarter-turned image on a swung arm") {
WriteMiniCBFSweep("cbfaxis_turn",
"# Detector_2theta 30.0000 deg.\n# Oscillation_axis OMEGA\n"
+ AxisTable("GON_OMEGA rotation goniometer . 1 0 0 . . .\n"
"DET_2THETA rotation detector . 1 0 0 . . .\n"
"DET_Z translation detector DET_2THETA 0 0 -1 0 0 0\n"
"ELEMENT_X translation detector DET_Z 0 1 0 -1 1 0\n"
"ELEMENT_Y translation detector ELEMENT_X 1 0 0 0 0 0\n", nx, ny),
nx, ny);
JFJochCBFReader reader;
REQUIRE_NOTHROW(reader.ReadFiles("cbfaxis_turn_0001.cbf"));
const auto x = reader.GetDataset()->experiment;
REQUIRE(x.GetGoniometer().has_value());
CHECK((x.GetGoniometer()->GetAxis() - Coord(1, 0, 0)).Length() < 1e-6f);
// fast = imgCIF (0,1,0) = internal (0,-1,0), slow = imgCIF (1,0,0) = internal (1,0,0)
CHECK(x.GetDetectorSetup().GetImageOrientation() == DetectorOrientation(false, 3));
// and the arm turns about its own stated axis, internal +x, by the stated 30 degrees
const auto geom = x.GetDiffractionGeometry();
const RotMatrix expected = RotMatrix(static_cast<float>(30.0 * PI / 180.0), {1, 0, 0})
* DetectorOrientation(false, 3).Matrix();
for (int64_t column = 0; column < 3; column++)
CHECK((geom.GetDetectorMatrix().Column(column) - expected.Column(column)).Length() < 1e-5f);
reader.Close();
RemoveMiniCBFSweep("cbfaxis_turn");
}
}