mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 74 Wiki Activity

Merge rc167 into the detection-score branch

Browse Source 
Both lanes added a per-image scalar to the same eleven files, so every conflict was two
additions competing for one line. All were resolved by keeping both, with three that needed
more than that:

- ScanResultGenerator: rc167 changed the per-image float vectors to resize(n, NAN) so a frame
  that never arrived does not read back as a real 0. v_protein_score and v_ice_score are exactly
  that case - 0 is a real answer ("nothing detected here") - so they take the NAN default too.
- HDF5MetadataSource: rc167 established that NaN in a stored per-image array means "no value" and
  the optional must come back absent. The two detection scores now follow it, which they did not
  before the merge; without the guard a missing score would come back as a NaN that a threshold
  would silently compare against.
- CBORTest: designated initialisers must follow member declaration order, so spindle_blind_fraction
  precedes the two scores in the DataMessage aggregate.

Verified after the merge that every CBOR key that is encoded is also decoded (198 encoded keys,
one intentional exception: the "type" discriminator), that both lanes' fields survive in the
writer, the reader, the plots and the API, and that a stored file still round-trips.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
This commit is contained in:
leonarski_f
2026-09-08 00:13:51 +02:00
co-authored by Claude Opus 5
parent 62ca2b24f6 b650cda037
commit 14e2bf3c0f
283 changed files with 10689 additions and 1732 deletions
Download Patch File Download Diff File Expand all files Collapse all files
reader/HDF5MetadataSource.cpp
+48 -3
View File
@@ -325,6 +325,13 @@ bool ReadReflectionsFromGroup(HDF5Object &file,
// Written since the merge stopped back-deriving it; older _process.h5 do not carry it.
auto var_bkg = file.ReadOptVector<float>(image_group_name + "/background_variance");
auto lp = file.ReadOptVector<float>(image_group_name + "/lp");
// The sensor-efficiency part of lp, written since the correction existed. Absent in every
// older file, and there the whole of lp is taken for Lorentz-polarization - which is what
// those files mean, since nothing had corrected for the sensor when they were written.
auto qe = file.ReadOptVector<float>(image_group_name + "/qe");
// The flight path, in the same reciprocal convention. Absent in every older file, and there
// it is taken as 1: nothing had corrected for the flight path when those files were written.
auto flight = file.ReadOptVector<float>(image_group_name + "/flight");
auto partiality = file.ReadOptVector<float>(image_group_name + "/partiality");
auto phi = file.ReadOptVector<float>(image_group_name + "/delta_phi");
auto zeta = file.ReadOptVector<float>(image_group_name + "/zeta");
@@ -350,6 +357,14 @@ bool ReadReflectionsFromGroup(HDF5Object &file,
if (lp.size() > i && lp[i] != 0.0f)
lp_val = 1.0f / lp[i];
float qe_val = 1.0f;
if (qe.size() > i && qe[i] != 0.0f)
qe_val = 1.0f / qe[i];
float flight_val = 1.0f;
if (flight.size() > i && flight[i] != 0.0f)
flight_val = 1.0f / flight[i];
float partiality_val = -1.0f;
if (partiality.size() > i && partiality[i] >= 0.0f)
partiality_val = partiality[i];
@@ -396,7 +411,9 @@ bool ReadReflectionsFromGroup(HDF5Object &file,
.bkg = bkg.at(i),
.var_bkg = var_bkg_val,
.sigma = int_err.at(i),
.rlp = lp_val,
.prescaling_corr = lp_val,
.qe_corr = qe_val,
.flight_corr = flight_val,
.partiality = partiality_val,
.zeta = zeta_val,
.image_scale_corr = image_scale_corr_val
@@ -557,6 +574,7 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
dataset->indexing_result = master_file->ReadOptVector<float>("/entry/MX/imageIndexed");
dataset->bkg_estimate = master_file->ReadOptVector<float>("/entry/MX/bkgEstimate");
dataset->spindle_blind_fraction = master_file->ReadOptVector<float>("/entry/MX/spindleBlindFraction");
dataset->ice_ring_score = master_file->ReadOptVector<float>("/entry/MX/iceRingScore");
dataset->protein_score = master_file->ReadOptVector<float>("/entry/MX/proteinScore");
dataset->ice_score = master_file->ReadOptVector<float>("/entry/MX/iceScore");
@@ -704,6 +722,10 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
data_file, "/entry/MX/bkgEstimate",
number_of_images, fimages);
ReadVector(dataset->spindle_blind_fraction,
data_file, "/entry/MX/spindleBlindFraction",
number_of_images, fimages);
ReadVector(dataset->ice_ring_score,
data_file, "/entry/MX/iceRingScore",
number_of_images, fimages);
@@ -1013,11 +1035,25 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
.value_or(0))));
// Sensor thickness/material drive the parallax/absorption model, so take them from the file
// rather than the DetectorSetup default.
// Say so when they are absent rather than let the DetectorSetup default stand in silently.
// The default (320 um Si) is right for every JUNGFRAU and wrong for the 450 um Si and 750 um
// CdTe sensors this code also reads, and both the angle-dependent efficiency correction and
// the parallax variance term are computed from these two numbers - so a file that omits them
// gets a correction derived from an assumption, which the user should be told about. The
// correction is still applied: a 320 um Si assumption is closer to every real sensor than
// switching the physics off, and declining would silently disagree with the same data read
// from a file that does state its sensor.
if (master_file->Exists("/entry/instrument/detector/sensor_thickness"))
detector.SensorThickness_um(
ReadLength_m(*master_file, "/entry/instrument/detector/sensor_thickness") * 1e6);
else
Logger("HDF5Reader").Warning("No sensor_thickness in the file; assuming {:.0f} um for the "
"sensor absorption model", detector.GetSensorThickness_um());
if (master_file->Exists("/entry/instrument/detector/sensor_material"))
detector.SensorMaterial(master_file->GetString("/entry/instrument/detector/sensor_material"));
else
Logger("HDF5Reader").Warning("No sensor_material in the file; assuming {} for the sensor "
"absorption model", detector.GetSensorMaterial());
// Optional, because a file that states no saturation value anywhere is a real and common
// thing: an Eiger master links saturation_value into a companion _meta.h5, and a deposited
// dataset frequently does not include that file, leaving neither the NXmx name nor the
@@ -1339,11 +1375,20 @@ void HDF5MetadataSource::FillPerImage(DataMessage &message, int64_t requested_im
message.indexing_lattice_count = dataset->indexing_lattice_count[image_number];
if (dataset->bkg_estimate.size() > image_number)
message.bkg_estimate = dataset->bkg_estimate[image_number];
// NaN is how the file stores a frame with no value; the optional must come back absent, not
// carrying a NaN, because absence is the CANNOT-SAY trigger state and a value is not.
if (dataset->spindle_blind_fraction.size() > image_number
&& std::isfinite(dataset->spindle_blind_fraction[image_number]))
message.spindle_blind_fraction = dataset->spindle_blind_fraction[image_number];
if (dataset->ice_ring_score.size() > image_number)
message.ice_ring_score = dataset->ice_ring_score[image_number];
if (dataset->protein_score.size() > image_number)
// Same convention as spindle_blind_fraction above: a frame that produced no score is stored as
// NaN, and 0 is a real answer ("nothing detected here"), so a missing one has to come back absent
// rather than as a NaN that a threshold would then compare against.
if (dataset->protein_score.size() > image_number
&& std::isfinite(dataset->protein_score[image_number]))
message.protein_score = dataset->protein_score[image_number];
if (dataset->ice_score.size() > image_number)
if (dataset->ice_score.size() > image_number && std::isfinite(dataset->ice_score[image_number]))
message.ice_score = dataset->ice_score[image_number];
if (dataset->efficiency.size() > image_number)
message.image_collection_efficiency = dataset->efficiency[image_number];