From ef103d2ea2920d1642fa4afe1ae2716528b3f3ec Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 10 Jul 2026 22:11:30 +0200 Subject: [PATCH] rugnux: raise default min-captured-fraction to 0.7 (recover weak F-cubic space groups) The de-novo space-group search runs its point-group correlation only on reflections with I/sigma >= 3, but the Diederichs error model floors merged I/sigma at the asymptotic ISa = 1/b. On a weakly-diffracting crystal a few edge-of-sweep truncated fulls (rocking curve captured far below 1) inflate the error-model b term, dropping ISa below 3 so no reflection clears the gate: every operator gets zero pairs and the point group collapses to P1 (or a low subgroup). The chi^2 self-consistency guard is likewise corrupted by those zingers. Raising the rotation default of --min-captured-fraction from 0.5 to 0.7 drops those partiality-extrapolated fulls, lifting ISa back above the gate and cleaning the intensity statistics, so the search recovers the true symmetry. On the F-cubic Benas crystals: Benas_3 P4 -> F432 (CC1/2 78% -> 98%) and Benas_7 P1 -> P6122 (CC1/2 40% -> 99%). Benas_7's data supports hexagonal P6122, not cubic F432 - the cubic 3-fold/4-folds fail the merge self-consistency test - matching XDS. Full /data/rotation_test battery: every other crystal keeps its exact space group; the only cost is a little multiplicity from the extra dropped edge-fulls on a few weak/ narrow-sweep crystals (worst completeness -7% on EP_cs_01-24, whose CC1/2 nonetheless rises), while several crystals' CC1/2 improve. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/CHANGELOG.md | 2 +- tools/rugnux_cli.cpp | 13 ++++++++----- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 05ec6473..d776eba3 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,7 +7,7 @@ This is an UNSTABLE release. It includes many experimental features, as well as * rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling. * rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics. * rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below = 1. -* rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.5 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). Merged intensities, ISa and CC1/2 are unchanged. +* rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group. * rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix. * jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation. * jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning. diff --git a/tools/rugnux_cli.cpp b/tools/rugnux_cli.cpp index 658d4ac4..187ea612 100644 --- a/tools/rugnux_cli.cpp +++ b/tools/rugnux_cli.cpp @@ -87,7 +87,7 @@ void print_usage() { std::cout << " --resolution-shells Number of resolution shells in the reported statistics table (default: 10)" << std::endl; std::cout << " --min-partiality Minimum partiality to accept reflection (default: 0.02)" << std::endl; std::cout << " --capture-uncertainty rot3d: systematic sigma ~num*(1-captured_fraction)*I on under-captured fulls (default: 1.0 for rot3d, 0 otherwise)" << std::endl; - std::cout << " --min-captured-fraction rot3d: drop a combined full whose rocking curve was captured below this fraction (edge-of-sweep truncated fulls) (default: 0.5 for rotation, 0 otherwise; 0 = off)" << std::endl; + std::cout << " --min-captured-fraction rot3d: drop a combined full whose rocking curve was captured below this fraction (edge-of-sweep truncated fulls) (default: 0.7 for rotation, 0 otherwise; 0 = off)" << std::endl; std::cout << " --mosaicity Diagnostic: fix the scaling mosaicity (deg) instead of the per-image seed" << std::endl; std::cout << " --reject-outliers Per-observation merge outlier rejection, N sigma from the per-reflection median (default: 6 for rot3d, XDS/DIALS-style; 0 = off)" << std::endl; std::cout << " --reject-delta-cchalf Per-crystal CC1/2-delta rejection: drop images with deltaCChalf below mean - N*stddev (default: off; e.g. 2.5)" << std::endl; @@ -931,7 +931,7 @@ int main(int argc, char **argv) { scaling_settings.RefineB(refine_bfactor); scaling_settings.MinPartiality(min_partiality); scaling_settings.MinCapturedFraction(min_captured_fraction_arg.value_or( - (experiment.GetGoniometer().has_value() && !force_still) ? 0.5 : 0.0)); + (experiment.GetGoniometer().has_value() && !force_still) ? 0.7 : 0.0)); scaling_settings.MinCCForImage(min_image_cc / 100.0); // --min-image-cc is percent; the setting is a fraction if (intensity_format) scaling_settings.FileFormat(intensity_format.value()); @@ -1200,9 +1200,12 @@ int main(int argc, char **argv) { scaling_settings.RefineB(refine_bfactor); scaling_settings.MinPartiality(min_partiality); // Drop edge-of-sweep truncated fulls (rocking curve captured < this fraction) from the rot3d combine. - // Defaults ON (0.5) for rotation - removes the low-capture fulls that inflate low-res R-meas and - // slightly bias accuracy; off for non-rot3d (no combine). An explicit --min-captured-fraction wins. - scaling_settings.MinCapturedFraction(min_captured_fraction_arg.value_or(rotation_indexing ? 0.5 : 0.0)); + // Defaults ON (0.7) for rotation - removes the low-capture fulls that inflate low-res R-meas and + // slightly bias accuracy; off for non-rot3d (no combine). 0.7 (rather than 0.5) also strips the + // partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting + // crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor + // (e.g. F-cubic Benas_3 -> F432, Benas_7 -> P6122 instead of P1). An explicit --min-captured-fraction wins. + scaling_settings.MinCapturedFraction(min_captured_fraction_arg.value_or(rotation_indexing ? 0.7 : 0.0)); // Capture-aware systematic sigma defaults ON (1.0) for the rot3d combine - it down-weights the // over-extrapolated under-captured fulls and, with the mosaicity fix, lifts rotation ISa/anomalous // substantially. Off for non-rot3d (no combine). An explicit --capture-uncertainty always wins.