mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests 1 Actions 3 Packages Projects Releases 72 Wiki Activity

unmerged MTZ: export the events the merge would keep, not every event

Browse Source 
The export declared its rows FULL - LDTYPE=2 in the batch header, M=0 in
M/ISYM - while filtering them on --min-partiality alone (0.02), where the 3D
combine also applies --min-captured-fraction (0.7 on rotation) to the same
summed event. AIMLESS reads the FULL declaration, reports "all runs have only
fulls" and ignores FRACTIONCALC, so an event that caught a twentieth of its
rocking curve entered the merge whole, with a small sigma, and was weighted
heavily. There was no cut for the reader to make: the column that would let it
make one is the one the reading program has been told to ignore. Dropping those
events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from
0.985 to 0.993.

FRACTIONCALC itself is unchanged, values above 1 included. Each part's
partiality is the erf pair the predictor computed on that frame, from that
frame's own refined lattice and mosaicity, so the parts of one event do not tile
the rocking curve exactly - the offset steps by the wedge to within 12% of it,
and the sums land in a peak at 1.000 whose 95th percentile is 1.09. It is an
honest estimate of a captured fraction, and it is not the number rugnux scales
on, so clamping it would only hide the spread.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
This commit is contained in:
leonarski_f
2026-09-02 09:18:22 +02:00
co-authored by Claude Opus 5
parent b33cae9068
commit 762c093b10
3 changed files with 24 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CHANGELOG.md
+1
View File
@@ -8,6 +8,7 @@
* `rugnux` places a detector swung out on a 2theta arm where the file says it stands.
* `rugnux` writes the unmerged MTZ by default, with a P1 merge beside it, so a wrong space group can be re-merged without reprocessing.
* `rugnux` writes a batch header in the unmerged MTZ for every image the observations span, not only for the images that produced one, so a scaling program reads one run per sweep.
* `rugnux` leaves an event out of the unmerged MTZ when less of its rocking curve was captured than `--min-captured-fraction`, as the merge does, since the exported rows are declared full.
* `rugnux` handles symmetry better: the lattice, the point group, the setting and the systematic absences.
* `rugnux` fits the direction of the goniometer axis and not its length, so the cell chosen by the first pass is the one its own refinement scored.
* `rugnux` reports the detector geometry it measured - the direct beam, the tilt and the beam centre - and what a single sweep can and cannot determine.
image_analysis/WriteReflections.cpp
+22 -10
View File
@@ -602,15 +602,26 @@ float DetectorY(const Reflection &r) { return std::isfinite(r.observed_y) ? r.ob
// 8-41% of events have parts summing to zero or less and 10-21% of the intensity-weighted centroids
// fall outside the event's own frame range. Partialities are the rocking curve's own weights and are
// positive by construction.
// min_partiality is the combine's own floor on the assembled full (--min-partiality, default 0.02,
// 0 = off). An event that caught a thousandth of its rocking curve is not a measurement of that
// reflection - rugnux does not merge one either - and writing it as a full hands the reading program
// a whole observation whose intensity is noise and whose sigma is small, so it is weighted heavily.
// On a crystal whose rocking curves span twenty frames those events are 7% of the file and cost
// POINTLESS the point group. Everything above the floor is written with its honest FRACTIONCALC,
// truncated edge-of-sweep events included, for the reader to cut where it wants.
// The floor is the SAME pair the 3D combine applies to the same event - --min-partiality (default
// 0.02) and --min-captured-fraction (default 0.7 on rotation) - so what is exported as a full is
// exactly what rugnux merges as a full. The captured-fraction floor is what makes the FULL
// declaration true: the batch header says LDTYPE=2 and M/ISYM carries M=0, AIMLESS reads that as
// "all runs have only fulls" and ignores FRACTIONCALC entirely, so an event that caught a twentieth
// of its rocking curve does not enter the merge at a twentieth of its intensity - it enters whole,
// with a small sigma, and is weighted heavily. There is no cut for the reader to make: the column
// that would let it make one is one the reading program has been told to ignore. Measured, dropping
// those events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from 0.985 to 0.993.
//
// FRACTIONCALC itself is written as summed, above 1 included. Each part's partiality is the erf pair
// the predictor computed on THAT frame, from that frame's own refined lattice and its own mosaicity
// estimate, so the parts of one event do not tile the rocking curve exactly - measured, the offset
// from one part to the next steps by the wedge to within 12% of it, and the sums land in a peak at
// 1.000 whose 95th percentile is 1.09. It is the honest estimate of a captured fraction and it is
// not the number rugnux scales on (the merge recomputes every partiality from a frame-order-smoothed
// mosaicity, RotationScaleMerge::SmoothMosaicity), so clamping it to 1 would hide the spread and buy
// a reading program nothing.
std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &outcomes,
double min_partiality) {
double min_partiality, double min_captured_fraction) {
constexpr float MAX_FRAME_GAP = 2.0f; // == RotationScaleMerge's: what makes one rocking event
// The sort key travels with the part instead of being read back through the pointer, the way the
@@ -664,7 +675,7 @@ std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &
}
Reflection full = *parts[i].r;
i = j;
if (sum_p < min_partiality)
if (sum_p < min_partiality || sum_p < min_captured_fraction)
continue;
// The Lorentz-polarization factor is applied by the writer, which multiplies I by rlp, so
@@ -794,7 +805,8 @@ void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes
};
if (scanning && sum_partials) {
for (const auto &r : SumRockingEvents(outcomes,
experiment.GetScalingSettings().GetMinPartiality()))
experiment.GetScalingSettings().GetMinPartiality(),
experiment.GetScalingSettings().GetMinCapturedFraction()))
add_row(r);
} else {
for (const auto &outcome : outcomes)
rugnux/rugnux_cli.cpp
+1 -1
View File
@@ -171,7 +171,7 @@ void print_usage() {
std::cout << " -z, --reference-mtz <file> Reference MTZ file" << std::endl;
std::cout << " --reference-column <label> Reference MTZ column to use (default: auto - F-model, else IMEAN/I, else FP/FOBS/F)" << std::endl;
std::cout << " --model <file> After merging, validate vs this atomic model: R-free, 2Fo-Fc/Fo-Fc maps, and - where the merge kept the Bijvoet split - an anomalous difference map with the strongest anomalous scatterers named by the atom they sit on. PDB or mmCIF, gzipped or not, recognised by content rather than by extension" << std::endl;
std::cout << " --export-unmerged Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On by default, whenever there is an output prefix. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-partiality is not written, as in the merge. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
std::cout << " --export-unmerged Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On by default, whenever there is an output prefix. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-captured-fraction (or --min-partiality) is not written, exactly as in the merge, since a full is what the reading program is told these rows are. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
std::cout << " --no-export-unmerged Do not write <prefix>_unmerged.mtz. It is the largest file a run produces, so a run whose observations are not going to another scaling program - a regression battery, say - can skip writing it" << std::endl;
std::cout << " --export-unmerged-partials Write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summed here. Off by default and independent of --export-unmerged; both can be written in one run" << std::endl;
std::cout << " --no-p1-crosscheck Do not write <prefix>_P1.mtz. That file is the same observations merged in P1 instead of the space group the run determined, so a wrong space group can be re-merged, re-solved or re-refined without processing the images again. Every rotation run that determines its own space group writes it - including one that determined P1, where it repeats the merged output - so it is there to be harvested whatever the run decided. Not written when -S fixed the space group: that run never integrated the group's centring absences, so its P1 merge would be missing whole classes of reflections. It is a full merge, not the degraded one the search runs on, and it costs one extra merge" << std::endl;