From 63bf7359028f81aa5bc404dec0d2ee8e634f6607 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Thu, 12 Nov 2026 07:13:53 +0100 Subject: [PATCH] RESULT-CHANGING: LatticeSearch takes Le Page's class over the table's only for a length-equality class 3c001eb7a let the metric two-folds (Le Page, obliquity up to 1 deg) override the Niggli character table wherever they found a richer class. The motivating case - an F-cubic lattice whose degenerate shortest vectors give the table a reduced cell it reads as I-tetragonal - is a class that equates lengths. For a class carried by its angles alone (monoclinic, orthorhombic) the override instead promoted pseudo-symmetric metrics the table had refused: - a monoclinic small-molecule set (twin-composite pass): mC/mP -> oC at 0.3-0.9 deg obliquity; ISa 46.4 -> 16.7, R_meas 4.2 -> 4.9 %, SHELXL R1 0.049 -> 0.066 - a monoclinic protein, two in-house sets: mP -> oC at 0.2-1.0 deg; the beam-centre check then compared different classes at the two centres; ISa 3.6 -> 2.4, CC1/2 0.98 -> 0.45 on one, R_meas 68 -> 135 % on the other - a triclinic crystal with a pseudo-I-orthorhombic metric: aP -> oI at 0.5 deg at the file's centre, and the run merged at a markedly lower ISa and resolution Obliquity does not separate these from the F-cubic case (its unrefined cells promote at up to 1 deg too), so the line is drawn on the class: Le Page's answer is taken only where its class is tetragonal, trigonal, hexagonal or cubic. The F-cubic set keeps F432 (ISa 7.7, R_meas 13.2 %, unchanged); the three regressions return to their rc184 numbers. Guard sets (open + in-house, 26) against rc186-merge: 20 identical; cytidine, 2wnn and myob_x10sa return to their rc184 numbers (differences in the third digit); 2wnq returns to rc184's C2221 (R-free 0.547) from P21212 (0.444) - the deposition is P21, so both fail. The synthetic gains 3c001eb7a measured for oC/oI (monoclinic under-calls of centred orthorhombic lattices) are given back; cF, hP, hR, tI are kept. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK --- image_analysis/lattice_search/LatticeSearch.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/image_analysis/lattice_search/LatticeSearch.cpp b/image_analysis/lattice_search/LatticeSearch.cpp index b60c3c388..de3e127c3 100644 --- a/image_analysis/lattice_search/LatticeSearch.cpp +++ b/image_analysis/lattice_search/LatticeSearch.cpp @@ -480,10 +480,19 @@ std::optional SearchCharacters(const CrystalLattice &L, dou // class than the table, take it - held to this call's own tolerances, so a search that grants no // length equality still grants none. // + // Only for a class that equates lengths (tetragonal, trigonal, hexagonal, cubic): those are the + // lattices whose degenerate shortest vectors give the table more than one reduced cell. A class + // carried by its angles alone the table reads on any of them, and where it refuses one the + // two-folds would grant, the angle is not there: measured on monoclinic crystals with + // pseudo-orthorhombic metrics (and a triclinic one with a pseudo-I-orthorhombic one), Le Page + // promoted at 0.2-1 deg obliquity, the constrained cell was integrated and every one merged + // worse (ISa 46 -> 17 on one, CC1/2 0.98 -> 0.45 on another). + // // A tetragonal or cubic cell refused only on its length equality is, on its angles alone, the // orthorhombic lattice of the same centring in the same basis - the class a search granting no // length equality has to return, as the table did for the settings it could read. - if (auto lp = LePageLattice(L, std::min(angle_tolerance, LATTICE_MAX_OBLIQUITY_DEG))) { + if (auto lp = LePageLattice(L, std::min(angle_tolerance, LATTICE_MAX_OBLIQUITY_DEG)); + lp && ClassImposesLengthEquality(lp->system)) { const auto [angle, length] = ClassMetricDeparture(lp->conventional.GetUnitCell(), lp->system); if (length > dist_tolerance && (lp->system == gemmi::CrystalSystem::Tetragonal || lp->system == gemmi::CrystalSystem::Cubic))