leonarski_fandClaude Opus 5
3171b071e6
The point-group decision moved with the AMOUNT of data at fixed physics: a partially twinned trigonal crystal was promoted into the twin's holohedry whenever the search happened to see a larger first-pass merge, and kept its true subgroup when it saw a smaller one. Simulation over 6 noise draws with only the merge multiplicity varying: the twin is promoted 0/6 at multiplicity 2 and 6/6 at 18, while the genuine control is promoted 6/6 throughout. The cause is that every existing gate is a ratio to the merge error model - b_parent grows toward the true systematic scatter as sigma shrinks with 1/sqrt(N), while b_cand is already saturated by the twin's disagreement, so the ratio slides down through a fixed veto. The parent statistic moves with data amount and the candidate statistic does not. Gate promotions on the operator disagreement H = <|I1-I2|/(I1+I2)> instead, as the ratio of the operators a promotion ADDS to the parent group's own operators on the same reflections. There is no sigma in it, so it cannot drift with the error model, and the parent normalisation cancels data quality. Measured over 27 runs, 5 promotion types and 450-1800 images: genuine symmetry 0.862-1.219, merohedral twins 1.270-2.084. On the synthetic grid it is flat across a 9x change in multiplicity - genuine pinned at 1.00, twins 3-12x the bound - which is precisely the property the old gates lacked. chi^2 and the systematic-b stay as secondary vetoes; they protect against non-crystallographic pseudo-symmetry, which is where correlation-based scoring is weak. Pick the parent carefully: 422 has two maximal subgroups of order 4, and on a tetragonal crystal twinned by 2[100] the rival (222) is CC-confirmed too and CONTAINS the twin laws, so normalising against it hides the twin among the promotion's own real operators (ratio 8.19 against the true parent, 0.78 against the rival). Where several parents tie, judge on the most damning. Also: - Report a refused promotion instead of silently processing lower. Merging a twin in the twin's holohedry averages non-equivalent reflections into each other and cannot be undone from the output; keeping the subgroup costs only redundancy. The refusal names the group and the number that caused it. - Stop the twinning report from arguing in a circle. It ran after adoption and conditioned on the adopted group, so a promotion into a holohedral Laue class made it print "no merohedral twin law exists" - the test was conditioned on the decision it should audit. Twinning is now also measured on the subgroup merge before adoption, and the post-adoption text says when its own conclusion is not authoritative. - Compare PRIMITIVE cell volumes in the first-pass scheme tie-break. A centred setting's cell is an exact integer multiple of its primitive one (a rhombohedral lattice in hexagonal axes is exactly 3x), so the integer-supercell test fired on a pure setting difference and demoted a good scheme to a threefold-smaller merge - which is what let the twin see the small merge to begin with. Rotation battery, 33 crystals: point-group agreement 30/33 -> 29/33, one crystal moved. That crystal (P422 -> P222) is the one with the known unresolved integration defect where reflections near the rotation-axis plane are wildly mis-integrated; its symmetry mates genuinely disagree, and its lower-symmetry merge is measurably better (ISa 2.72 -> 3.63, high-shell CC 75.4 -> 86.0). The threshold was not moved to accommodate it: 1.25 sits inside the measured gap and widening it would admit real twins. Separately the tie-break improved one crystal's CC1/2 from 77.7 to 84.0. Tests: a synthetic twin-fraction x multiplicity grid, which is what the search had never had - the existing tests are noise-free and exercise only Stage B absences. A NOTE ON WHAT WAS TRIED AND REJECTED, so it is not rebuilt: the obvious "physics-anchored" statistic is the disattenuated cross-validated correlation rho = corr(I_half0(h), I_half1(Rh)) / corr(I_half0, I_half1), which is 1 for real symmetry at any data quality and 2a(1-a)/((1-a)^2+a^2) for a twin. It passes the synthetic grid perfectly and FAILS ON REAL DATA IN BOTH DIRECTIONS - five false refusals of genuine symmetry on the battery, and it waves through a twin (rho 0.998) that H refuses. The reason is that cc_half correlates the two halves of the SAME reflection and so measures only random error, while cc_cross compares DIFFERENT reflections carrying different systematic error; dividing by cc_half removes the noise and leaves a systematic floor that varies by crystal AND by operator. Genuine rho measures 0.9987 on strong data and 0.73 on weak. A synthetic generator validates a statistic's arithmetic, never its premise, and this premise - that the only departure from exact symmetry is noise - is false for every real crystal. Any per-operator agreement statistic needs a same-crystal reference; an absolute threshold on one cannot be made to work by tuning. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
54 lines
2.7 KiB
C++
54 lines
2.7 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "../../common/Reflection.h"
|
|
#include "gemmi/symmetry.hpp"
|
|
|
|
// Intensity-statistics test for merohedral twinning, in the spirit of phenix.xtriage. Two robust,
|
|
// normalisation-free indicators on the merged intensities:
|
|
//
|
|
// - Padilla & Yeates L-test (Acta Cryst D59, 2003): for pairs of nearby but symmetry-independent
|
|
// reflections, L = (I1 - I2)/(I1 + I2). Untwinned acentric data give <|L|> = 0.5, <L^2> = 1/3;
|
|
// a perfect merohedral twin narrows this to <|L|> = 0.375, <L^2> = 0.2. Insensitive to
|
|
// anisotropy and overall scale because it is a local ratio.
|
|
//
|
|
// - Second moment <I^2>/<I>^2 of acentric intensities, normalised per resolution shell so it is
|
|
// 2.0 for untwinned data and 1.5 for a perfect twin (= 2(1 - a + a^2) for twin fraction a).
|
|
//
|
|
// These flag the *presence* of twinning; they do not by themselves identify the twin law.
|
|
|
|
struct TwinningAnalysisResult {
|
|
int l_test_pairs = 0;
|
|
double mean_abs_l = 0.0; // <|L|>: 0.5 untwinned, 0.375 perfect twin
|
|
double mean_l_squared = 0.0; // <L^2>: 0.333 untwinned, 0.2 perfect twin
|
|
|
|
int moment_reflections = 0;
|
|
double second_moment = 0.0; // <I^2>/<I>^2 (shell-normalised): 2.0 untwinned, 1.5 perfect twin
|
|
|
|
// Twin fraction estimated from the second moment, a = (1 - sqrt(2M-3))/2, clamped to [0, 0.5].
|
|
double estimated_twin_fraction = 0.0;
|
|
bool twinning_suspected = false;
|
|
|
|
// False when the Laue class is holohedral (4/mmm, 6/mmm, m-3m, rhombohedral -3m): no merohedral
|
|
// twin law can exist, so a low <|L|> / second moment there is a statistical artefact, not twinning.
|
|
bool merohedral_twinning_possible = true;
|
|
|
|
// Set when this analysis ran on a merge whose space group the pipeline CHOSE by promoting past a
|
|
// subgroup. The "holohedral Laue class, so no twin law exists" conclusion is then circular - the
|
|
// promotion is exactly what a twin would have caused - so the report must not state it as a fact.
|
|
bool laue_class_was_chosen_by_promotion = false;
|
|
};
|
|
|
|
// The space group (when known) is used to drop centric reflections, which follow different
|
|
// statistics (<|L|>, <I^2>/<I>^2 = 3 not 2); pass nullptr for P1 data, where none are centric.
|
|
TwinningAnalysisResult AnalyzeTwinning(const std::vector<MergedReflection>& merged,
|
|
const gemmi::SpaceGroup* space_group = nullptr,
|
|
int resolution_shells = 20);
|
|
|
|
std::string TwinningAnalysisToText(const TwinningAnalysisResult& result);
|