unmerged MTZ: export the events the merge would keep, not every event

The export declared its rows FULL - LDTYPE=2 in the batch header, M=0 in
M/ISYM - while filtering them on --min-partiality alone (0.02), where the 3D
combine also applies --min-captured-fraction (0.7 on rotation) to the same
summed event. AIMLESS reads the FULL declaration, reports "all runs have only
fulls" and ignores FRACTIONCALC, so an event that caught a twentieth of its
rocking curve entered the merge whole, with a small sigma, and was weighted
heavily. There was no cut for the reader to make: the column that would let it
make one is the one the reading program has been told to ignore. Dropping those
events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from
0.985 to 0.993.

FRACTIONCALC itself is unchanged, values above 1 included. Each part's
partiality is the erf pair the predictor computed on that frame, from that
frame's own refined lattice and mosaicity, so the parts of one event do not tile
the rocking curve exactly - the offset steps by the wedge to within 12% of it,
and the sums land in a peak at 1.000 whose 95th percentile is 1.09. It is an
honest estimate of a captured fraction, and it is not the number rugnux scales
on, so clamping it would only hide the spread.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
This commit is contained in:
2026-09-02 09:18:22 +02:00
co-authored by Claude Opus 5
parent b33cae9068
commit 762c093b10
3 changed files with 24 additions and 11 deletions
+22 -10
View File
@@ -602,15 +602,26 @@ float DetectorY(const Reflection &r) { return std::isfinite(r.observed_y) ? r.ob
// 8-41% of events have parts summing to zero or less and 10-21% of the intensity-weighted centroids
// fall outside the event's own frame range. Partialities are the rocking curve's own weights and are
// positive by construction.
// min_partiality is the combine's own floor on the assembled full (--min-partiality, default 0.02,
// 0 = off). An event that caught a thousandth of its rocking curve is not a measurement of that
// reflection - rugnux does not merge one either - and writing it as a full hands the reading program
// a whole observation whose intensity is noise and whose sigma is small, so it is weighted heavily.
// On a crystal whose rocking curves span twenty frames those events are 7% of the file and cost
// POINTLESS the point group. Everything above the floor is written with its honest FRACTIONCALC,
// truncated edge-of-sweep events included, for the reader to cut where it wants.
// The floor is the SAME pair the 3D combine applies to the same event - --min-partiality (default
// 0.02) and --min-captured-fraction (default 0.7 on rotation) - so what is exported as a full is
// exactly what rugnux merges as a full. The captured-fraction floor is what makes the FULL
// declaration true: the batch header says LDTYPE=2 and M/ISYM carries M=0, AIMLESS reads that as
// "all runs have only fulls" and ignores FRACTIONCALC entirely, so an event that caught a twentieth
// of its rocking curve does not enter the merge at a twentieth of its intensity - it enters whole,
// with a small sigma, and is weighted heavily. There is no cut for the reader to make: the column
// that would let it make one is one the reading program has been told to ignore. Measured, dropping
// those events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from 0.985 to 0.993.
//
// FRACTIONCALC itself is written as summed, above 1 included. Each part's partiality is the erf pair
// the predictor computed on THAT frame, from that frame's own refined lattice and its own mosaicity
// estimate, so the parts of one event do not tile the rocking curve exactly - measured, the offset
// from one part to the next steps by the wedge to within 12% of it, and the sums land in a peak at
// 1.000 whose 95th percentile is 1.09. It is the honest estimate of a captured fraction and it is
// not the number rugnux scales on (the merge recomputes every partiality from a frame-order-smoothed
// mosaicity, RotationScaleMerge::SmoothMosaicity), so clamping it to 1 would hide the spread and buy
// a reading program nothing.
std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &outcomes,
double min_partiality) {
double min_partiality, double min_captured_fraction) {
constexpr float MAX_FRAME_GAP = 2.0f; // == RotationScaleMerge's: what makes one rocking event
// The sort key travels with the part instead of being read back through the pointer, the way the
@@ -664,7 +675,7 @@ std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &
}
Reflection full = *parts[i].r;
i = j;
if (sum_p < min_partiality)
if (sum_p < min_partiality || sum_p < min_captured_fraction)
continue;
// The Lorentz-polarization factor is applied by the writer, which multiplies I by rlp, so
@@ -794,7 +805,8 @@ void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes
};
if (scanning && sum_partials) {
for (const auto &r : SumRockingEvents(outcomes,
experiment.GetScalingSettings().GetMinPartiality()))
experiment.GetScalingSettings().GetMinPartiality(),
experiment.GetScalingSettings().GetMinCapturedFraction()))
add_row(r);
} else {
for (const auto &outcome : outcomes)