mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests 1 Actions 2 Packages Projects Releases 72 Wiki Activity

reader: fall back to the pre-NXmx key names, so an Eiger 1.x master opens

Browse Source 
Firmware 1.x writes the same three values under different names. Try the NXmx spelling first and
the old one only if it is absent:

    detector/distance         <- detector/detector_distance
    detector/saturation_value <- detectorSpecific/countrate_correction_count_cutoff
    sample/transformations    <- sample/goniometer

This cannot change what a current file reads: every modern Eiger master carries BOTH spellings.
Measured on thirteen masters from ten facilities, firmware release-2020.2.1 through
release-2024.1.1 - all of them write detector_distance and countrate_correction_count_cutoff beside
the NXmx names, and a goniometer group beside the transformations one.

The goniometer is the one that matters. A goniometer is only ever set from that one group, so a
file whose axes are somewhere else was not an error - it was read as STILLS, silently, and the run
completed with the wrong answer. The old layout also tags no axis with transformation_type and
gives no vector, both of which ReadAxis required, so absence now means two different things by
layout: in a transformations group it still means "not an axis" (that is how AXISNAME_end and the
width scalars are skipped), while in the legacy group every leaf IS an axis and the companions are
recognised by name instead. A missing direction defaults to the one every DECTRIS master since has
written and says so in a warning rather than assuming it silently; a wrong guess there does not
index, so it is visible, and the rotation first pass tries the opposite sign anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
leonarski_f
2026-08-28 21:56:12 +02:00
co-authored by Claude Opus 5
parent a7615c8778
commit df3696e6e2
2 changed files with 85 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
reader/HDF5MetadataSource.cpp
+81 -16
View File
@@ -10,6 +10,7 @@
#include "../image_analysis/spot_finding/SpotUtils.h"
#include "../common/GridScanSettings.h"
#include "../common/JFJochMath.h"
#include "../common/Logger.h"
#include "../common/ROIDefinition.h"
inline std::pair<gemmi::CrystalSystem, char> parse_bravais_lattice(const std::string &val) {
@@ -100,6 +101,40 @@ std::string ResolveRelativeToMaster(const std::string &directory,
return (std::filesystem::path(directory) / path).string();
}
// DECTRIS Eiger firmware 1.x writes the same values under different names, and a file from that era
// is still what a repository hands you. The modern spelling is tried first; the legacy one is a pure
// fallback, and it is safe because every current Eiger master carries BOTH (measured on thirteen
// masters from ten facilities, firmware release-2020.2.1 through release-2024.1.1 - all of them
// write detector_distance and countrate_correction_count_cutoff beside the NXmx names). So this can
// never change what a modern file reads.
float ReadWithLegacyFallback(HDF5Object &file, const std::string &nxmx, const std::string &legacy) {
if (file.Exists(nxmx))
return file.GetFloat(nxmx);
if (file.Exists(legacy))
return file.GetFloat(legacy);
throw JFJochException(JFJochExceptionCategory::HDF5, "Cannot find " + nxmx + " (nor " + legacy + ")");
}
int64_t ReadIntWithLegacyFallback(HDF5Object &file, const std::string &nxmx, const std::string &legacy) {
if (file.Exists(nxmx))
return file.GetInt(nxmx);
if (file.Exists(legacy))
return file.GetInt(legacy);
throw JFJochException(JFJochExceptionCategory::HDF5, "Cannot find " + nxmx + " (nor " + legacy + ")");
}
// Where the goniometer axes live. NXmx puts them in /entry/sample/transformations; firmware 1.x put
// them in /entry/sample/goniometer and wrote no transformation_type and no vector on them. Getting
// this one wrong is not a missing value but a WRONG ANSWER: a goniometer is only ever set from this
// group, so a file whose axes are somewhere else is read as stills, silently.
std::string GoniometerGroup(HDF5Object &file) {
if (file.Exists("/entry/sample/transformations"))
return "/entry/sample/transformations";
if (file.Exists("/entry/sample/goniometer"))
return "/entry/sample/goniometer";
return {};
}
template<class T>
void ReadVector(std::vector<T> &v,
HDF5Object &file,
@@ -603,7 +638,9 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
dataset->experiment.BeamX_pxl(master_file->GetFloat("/entry/instrument/detector/beam_center_x"));
dataset->experiment.BeamY_pxl(master_file->GetFloat("/entry/instrument/detector/beam_center_y"));
float det_distance = master_file->GetFloat("/entry/instrument/detector/distance");
float det_distance = ReadWithLegacyFallback(*master_file,
"/entry/instrument/detector/distance",
"/entry/instrument/detector/detector_distance");
if (det_distance < 0.001)
det_distance = 0.1; // Set to 100 mm, if det distance is less than 1 mm
dataset->experiment.DetectorDistance_mm(det_distance * 1000.0);
@@ -630,13 +667,14 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
// the CBOR stream and the writer, so looking only for "omega" - as this did - read a sweep
// recorded as "phi" back as stills, silently. Prefer an axis that actually turns; fall back
// to a stationary one, which still says where the head was.
if (master_file->Exists("/entry/sample/transformations")) {
const std::string gonio_group = GoniometerGroup(*master_file);
if (!gonio_group.empty()) {
// A Smargon chi/phi is tagged with equipment_component - it is a head position, not the
// spindle. Recognised by that tag and not by name: phi is an ordinary spindle name in MX,
// so a file from anywhere else must not have its rotation axis read back as a head
// position, nor its spindle mistaken for one here.
auto is_smargon_axis = [this](const std::string &name) {
const std::string dname = "/entry/sample/transformations/" + name;
auto is_smargon_axis = [this, &gonio_group](const std::string &name) {
const std::string dname = gonio_group + "/" + name;
if (!master_file->Exists(dname))
return false;
HDF5DataSet axis(*master_file, dname);
@@ -645,10 +683,10 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
};
std::optional<GoniometerAxis> stationary;
for (const auto &name: master_file->FindLeafs("/entry/sample/transformations")) {
for (const auto &name: master_file->FindLeafs(gonio_group)) {
if (is_smargon_axis(name))
continue;
auto axis = ReadAxis(master_file.get(), name);
auto axis = ReadAxis(master_file.get(), name, gonio_group);
if (!axis.has_value())
continue;
if (axis->IsScanning()) {
@@ -667,9 +705,9 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
// the head position entirely - nothing in reader/ read it.
std::optional<GoniometerAxis> chi, phi;
if (is_smargon_axis("chi"))
chi = ReadAxis(master_file.get(), "chi");
chi = ReadAxis(master_file.get(), "chi", gonio_group);
if (is_smargon_axis("phi"))
phi = ReadAxis(master_file.get(), "phi");
phi = ReadAxis(master_file.get(), "phi", gonio_group);
if (chi.has_value() || phi.has_value()) {
SmargonPosition smargon;
if (chi.has_value()) {
@@ -778,7 +816,9 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
if (master_file->Exists("/entry/instrument/detector/sensor_material"))
detector.SensorMaterial(master_file->GetString("/entry/instrument/detector/sensor_material"));
detector.SaturationLimit(SaturationLimitFromValue(
master_file->GetInt("/entry/instrument/detector/saturation_value")));
ReadIntWithLegacyFallback(*master_file,
"/entry/instrument/detector/saturation_value",
"/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff")));
// The reader hands every image out as signed int32 whatever the file stored (see PixelSigned
// below), so that is the container depth the rest of the code has to see. DetectorSetup defaults
// DECTRIS to 16 bits and GetByteDepthImage() prefers the detector's value over the image
@@ -1141,8 +1181,9 @@ void HDF5MetadataSource::FillPerImage(DataMessage &message, int64_t requested_im
}
}
std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, const std::string &name) {
std::string dname = "/entry/sample/transformations/" + name;
std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, const std::string &name,
const std::string &group) {
std::string dname = group + "/" + name;
if (!file->Exists(dname))
return {};
@@ -1160,9 +1201,20 @@ std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, con
// Missing attribute means "not a transformation", so skip it rather than throwing: the search
// for the goniometer walks every leaf and only stops early on an axis that turns, so a master
// whose axis was stationary reached omega_end and could not be opened at all.
if (!dataset.AttrExists("transformation_type")
|| (dataset.ReadAttrStr("transformation_type") != "rotation"))
// NXmx tags every axis; DECTRIS firmware 1.x tagged none of them, so absence has to mean two
// different things depending on the layout. In a transformations group it means "not an axis"
// (the writer's own AXISNAME_end and the rotation-width scalars live there and carry only units),
// and skipping is right. In the legacy goniometer group EVERY leaf is an axis and none is tagged,
// so skipping there would find no goniometer at all and the sweep would be read as stills.
const bool legacy_group = (group != "/entry/sample/transformations");
if (dataset.AttrExists("transformation_type")) {
if (dataset.ReadAttrStr("transformation_type") != "rotation")
return {};
} else if (!legacy_group) {
return {};
} else if (name.find("_end") != std::string::npos || name.find("_range") != std::string::npos) {
return {}; // the same companion datasets, by name, since there is no tag to go on
}
std::vector<double> end = file->ReadOptVector<double>(dname + "_end");
@@ -1172,10 +1224,23 @@ std::optional<GoniometerAxis> HDF5MetadataSource::ReadAxis(HDF5Object *file, con
double start = angle[0];
double incr = (angle.size() < 2) ? 0.0 : angle[1] - angle[0];
std::vector<double> axis_vec = dataset.ReadAttrVec("vector");
if (axis_vec.size() != 3)
std::vector<double> axis_vec;
if (dataset.AttrExists("vector")) {
axis_vec = dataset.ReadAttrVec("vector");
if (axis_vec.size() != 3)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
dname + " Vector must have 3 elements");
} else if (legacy_group) {
// Firmware 1.x stored no direction at all. Assume the one every DECTRIS master since has
// written, and say so - a wrong guess here does not index, so it is visible rather than
// silent, and the rotation first pass will try the opposite sign anyway.
axis_vec = {-1.0, 0.0, 0.0};
Logger("HDF5Reader").Warning("{} carries no axis direction (pre-NXmx layout); assuming "
"(-1,0,0), the direction current DECTRIS masters write", dname);
} else {
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
dname + " Vector must have 3 elements");
dname + " has no vector attribute");
}
Coord axis(axis_vec[0], axis_vec[1], axis_vec[2]);
GoniometerAxis g_axis(name, start, incr, axis, {});
reader/HDF5MetadataSource.h
+4 -1
View File
@@ -74,6 +74,9 @@ private:
std::optional<int64_t> ToLocalIndex(int64_t image_number) const;
HDF5ImageLocator::Location ResolveMeta(int64_t global) const;
std::optional<GoniometerAxis> ReadAxis(HDF5Object *file, const std::string &name);
// group is where the axes live: /entry/sample/transformations (NXmx) or
// /entry/sample/goniometer (DECTRIS firmware 1.x, which tags no axis and gives no vector).
std::optional<GoniometerAxis> ReadAxis(HDF5Object *file, const std::string &name,
const std::string &group);
void ReadROIMetadata(HDF5ReadOnlyFile &file, JFJochReaderDataset &dataset) const;
};