mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 74 Wiki Activity

integration: the sensor efficiency is carried as its own quantity, not folded into the Lorentz-polarization factor

Browse Source 
It was multiplied into the per-reflection factor at prediction, so that factor held
Lorentz, polarization and efficiency at once and the two spellings that reach a file
- the wire key and the reflection dataset - meant something different from what
they had meant the day before. The unmerged MTZ had to divide the two apart again
at write time to fill its own columns, which is a good sign the wrong thing was
being carried.

Carry them separately. The prescaling factor is Lorentz and polarization again, what
its name and both reference implementations mean by it, and the efficiency is its own
field through prediction, integration, serialization and storage. Fifteen sites that
want the total now multiply the two - once per reflection, not once per pixel.

The efficiency is stored rather than recomputed on read, because the writer has no
geometry to recompute it from, and because a file written before the correction
existed would have had a radial trend invented for it. Sixty stored files were
checked for the one combination that would be ambiguous - the old meaning of the
factor beside a stored efficiency - and none carries it.

Output does not move. Re-scaling a file written before the efficiency existed is
byte-identical, which is a proof rather than a sample, since the stored factor is
exactly one there. Where the efficiency is live, one product is reassociated -
(L*Q)/P becomes (L/P)*Q - and about a third of the values differ in the last bit or
two: every structural column is identical, so no reflection is gained, lost or
reindexed, and no intensity in 1.4 million observations moves by as much as 1e-4 of
its own sigma.

The parity tests now compare the efficiency as well, and their non-vacuity guard
watches it rather than the factor it left - which is the same guard that went blind
when the efficiency was added to a field it was not watching.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
This commit is contained in:
leonarski_f
2026-09-05 16:28:15 +02:00
co-authored by Claude Opus 5
parent e44ce069a2
commit 7c10d62dab
20 changed files with 90 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files
image_analysis/WriteReflections.cpp
+21 -16
View File
@@ -671,11 +671,12 @@ std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &
for (size_t m = i; m < j; ++m) {
const Reflection &r = *parts[m].r;
const double p = r.partiality;
const float pc = r.prescaling_corr * r.qe_corr; // the whole correction, LP x QE
sum_p += p;
sum_I += static_cast<double>(r.I) * r.prescaling_corr;
sum_var += static_cast<double>(r.sigma) * r.sigma * r.prescaling_corr * r.prescaling_corr;
sum_var_bkg += static_cast<double>(r.var_bkg) * r.prescaling_corr * r.prescaling_corr;
p_corr += p * r.prescaling_corr;
sum_I += static_cast<double>(r.I) * pc;
sum_var += static_cast<double>(r.sigma) * r.sigma * pc * pc;
sum_var_bkg += static_cast<double>(r.var_bkg) * pc * pc;
p_corr += p * pc;
p_qe += p * r.qe_corr;
p_frame += p * r.image_number;
p_x += p * DetectorX(r);
@@ -689,14 +690,17 @@ std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &
if (sum_p < min_partiality || sum_p < min_captured_fraction)
continue;
// The prescaling factor is applied by the writer, which multiplies I by prescaling_corr, so divide the
// event's own factor back out of the sums here. It is the same geometry for every part of
// one event to a median 2e-4, so the file's I / LP * QE is still the raw count sum.
full.prescaling_corr = static_cast<float>(p_corr / sum_p);
// The prescaling factor is applied by the writer, which multiplies I by the whole correction, so
// divide the event's own factor back out of the sums here. It is the same geometry for every part
// of one event to a median 2e-4, so the file's I / LP * QE is still the raw count sum. The event's
// mean is taken on the whole correction and on its QE part; the Lorentz-polarization half is what
// is left when the second is divided out of the first.
const float mean_corr = static_cast<float>(p_corr / sum_p);
full.qe_corr = static_cast<float>(p_qe / sum_p);
full.I = static_cast<float>(sum_I / full.prescaling_corr);
full.sigma = static_cast<float>(std::sqrt(sum_var) / full.prescaling_corr);
full.var_bkg = static_cast<float>(sum_var_bkg / (static_cast<double>(full.prescaling_corr) * full.prescaling_corr));
full.prescaling_corr = mean_corr / full.qe_corr;
full.I = static_cast<float>(sum_I / mean_corr);
full.sigma = static_cast<float>(std::sqrt(sum_var) / mean_corr);
full.var_bkg = static_cast<float>(sum_var_bkg / (static_cast<double>(mean_corr) * mean_corr));
full.partiality = static_cast<float>(sum_p);
full.image_number = static_cast<float>(p_frame / sum_p);
full.observed_x = static_cast<float>(p_x / sum_p);
@@ -819,15 +823,16 @@ void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes
mtz.data.push_back(static_cast<float>(hkl[2]));
mtz.data.push_back(static_cast<float>((partials ? 256 : 0) + isym));
mtz.data.push_back(static_cast<float>(batch));
mtz.data.push_back(r.I * r.prescaling_corr);
mtz.data.push_back(r.sigma * r.prescaling_corr);
const float corr = r.prescaling_corr * r.qe_corr;
mtz.data.push_back(r.I * corr);
mtz.data.push_back(r.sigma * corr);
mtz.data.push_back(r.partiality);
mtz.data.push_back(DetectorX(r));
mtz.data.push_back(DetectorY(r));
mtz.data.push_back(phi_start_deg(r.image_number) + wedge_deg / 2.0f);
// prescaling_corr is the whole prescaling product; qe_corr is its sensor-efficiency part. LP is what
// is left when that part is divided out, and QE is that part in the divide-by direction.
mtz.data.push_back(r.prescaling_corr / r.qe_corr);
// prescaling_corr is Lorentz x polarization, which is what LP means; qe_corr is the sensor
// efficiency beside it, written in the divide-by direction.
mtz.data.push_back(r.prescaling_corr);
mtz.data.push_back(1.0f / r.qe_corr);
mtz.data.push_back(0.0f); // FLAG: nothing here is a rejected observation
mtz.data.push_back(r.delta_phi_deg);