From 6081b6bc433db8a65ef8b418cfdaa804e5aef7cc Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Wed, 2 Sep 2026 08:52:21 +0200 Subject: [PATCH] docs: credit the L test, FFT indexing, TORO, Niggli, peakfinder8 and SparseCCL Six methods the pages name or describe carried no citation: Padilla & Yeates (the L test), Steller, Bolotovsky & Rossmann (the projection/FFT autoindexing MOSFLM implements), TORO (what ffbidx implements), Krivy & Gruber and the ITA lattice-character table (the reduction and Bravais assignment), Cheetah's peakfinder8 (the per-ring background statistics of the adaptive finder) and Hennequin et al.'s SparseCCL (already credited to traccc, now also to its authors). Each gets its ACKNOWLEDGEMENT.md paragraph, a References entry in CPU_DATA_ANALYSIS.md, and a one-line credit at the algorithm. The Sheriff & Hendrickson / Popov & Bourenkov entry is re-scoped so each claim sits on the paper that supports it - P&B 2003 is titled, and credited for the sigma-aware anisotropy estimation its statistic modelling contains, not for the tensor and its constraints. All DOIs verified against the publishers; the SparseCCL DOI resolves to IEEE document 9049184 (IEEE blocks content scraping, so verified by the resolved document id plus two independent sources). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N --- docs/ACKNOWLEDGEMENT.md | 40 ++++++++++++++++++- docs/CPU_DATA_ANALYSIS.md | 10 ++++- image_analysis/indexing/FFBIDXIndexer.cpp | 3 ++ image_analysis/indexing/FFTIndexer.cpp | 1 + .../lattice_search/LatticeSearch.cpp | 2 + .../scale_merge/TwinningAnalysis.cpp | 1 + .../spot_finding/AdaptiveSpotFinderCPU.cpp | 2 + .../spot_finding/StrongPixelSet.cpp | 1 + 8 files changed, 56 insertions(+), 4 deletions(-) diff --git a/docs/ACKNOWLEDGEMENT.md b/docs/ACKNOWLEDGEMENT.md index 6e4029888..68501f226 100644 --- a/docs/ACKNOWLEDGEMENT.md +++ b/docs/ACKNOWLEDGEMENT.md @@ -19,7 +19,10 @@ experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822); [traccc](https://github.com/acts-project/traccc). The CPU spot extractor adapts its SparseCCL source, and the CUDA spot extractor follows the design of its GPU counterpart - a backward-neighbour graph over a sorted hit list, resolved by a parallel union-find. traccc is MPL-2.0; see -[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). +[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). The SparseCCL algorithm itself is A. Hennequin, +B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: Connected Components Labeling and +Analysis for sparse images" (2019), DASIP 2019, 65-70 +[doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184). This software uses Viridis, Magma and Inferno colormaps from Matplotlib under its BSD-compatible license @@ -129,7 +132,12 @@ crystallography" (2023), Acta Cryst. D79, 449-461 **[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/)** — the Rossmann FFT autoindexing algorithm and post-refinement practice, including which parameters are safe to refine per image and which must be -refined over a wedge. A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM" +refined over a wedge. The autoindexing algorithm itself — projecting the reciprocal-space points +onto many directions and Fourier-transforming the 1D projection histograms — is I. Steller, +R. Bolotovsky and M. G. Rossmann, "An algorithm for automatic indexing of oscillation images using +Fourier analysis" (1997), J. Appl. Cryst. 30, 1036-1040 +[doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777); MOSFLM is the +implementation whose practice is followed. A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM" (2007), in *Evolving Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245, 41-51 [doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4); T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new @@ -144,12 +152,33 @@ region, the serial/stills processing model, and the per-frame indexing acceptanc (`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass, A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012), J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312). +The self-calibrating spot finder's per-resolution-ring background statistics, with the Bragg peaks +excluded by iterated clipping, follow Cheetah's peakfinder8: A. Barty, R. A. Kirian, +F. R. N. C. Maia, M. Hantke, C. H. Yoon, T. A. White and H. N. Chapman, "Cheetah: software for +high-throughput reduction and analysis of serial femtosecond X-ray diffraction data" (2014), +J. Appl. Cryst. 47, 1118-1131 +[doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626). + +**[fast-feedback-indexer](https://github.com/paulscherrerinstitute/fast-feedback-indexer)** — the +known-cell indexer for serial stills (`-X ffbidx`) is PSI's fast-feedback-indexer library, linked at +build time (BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), which implements +the TORO algorithm: P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton, +M. Janousch, F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for +kilohertz serial crystallography" (2024), J. Appl. Cryst. 57, 931-944 +[doi:10.1107/S1600576724003182](https://doi.org/10.1107/S1600576724003182). **[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and structure-factor machinery, and MTZ / XDS_ASCII I/O. Vendored in `gemmi_gph/`, so it also carries a licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). +**Křivý & Gruber's Niggli reduction, and the lattice-character table** — the reduction that puts +every candidate cell in a comparable form is I. Křivý and B. Gruber, "A unified algorithm for +determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298 +[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636), used through GEMMI's +implementation; the table of lattice characters that maps a reduced cell to Bravais lattices and +centrings follows International Tables for Crystallography Vol. A, Table 9.2.5.1. + **Grosse-Kunstleve, Sauter & Adams's numerically stable cell reduction** - the magnitude-scaled tolerance that decides the sign of a structurally-zero scalar product, and with it the Niggli type a reduced cell is presented in. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically @@ -176,6 +205,13 @@ Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, 644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). +**The twinning L test** is Padilla and Yeates's: pairing each acentric reflection with a +symmetry-independent neighbour and reading the first and second moments of +L = (I1−I2)/(I1+I2) against their untwinned and perfect-twin values. J. E. Padilla and +T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and +pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, 1124-1130 +[doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). + **Diffraction anisotropy** — the description of the overall fall-off by a single anisotropic displacement tensor, its symmetry constraints, and the fact that only its deviatoric part is determined (the isotropic part being degenerate with the overall scale) are Sheriff and Hendrickson's. diff --git a/docs/CPU_DATA_ANALYSIS.md b/docs/CPU_DATA_ANALYSIS.md index 504209489..5f3f66be1 100644 --- a/docs/CPU_DATA_ANALYSIS.md +++ b/docs/CPU_DATA_ANALYSIS.md @@ -32,7 +32,13 @@ The methods draw on, and in places reimplement, solutions from: - W. Kabsch, “Integration, scaling, space-group assignment and post-refinement”, *Acta Cryst.* **D66** (2010), 133–144 (mosaicity/partiality likelihood treatment; notation such as ζ and rotation factors). - T. A. White et al., CrystFEL method papers (spot finding, three‑ring integration, serial/still diffraction processing concepts). - J. Kieffer & J. P. Wright, "PyFAI: a Python library for high performance azimuthal integration on GPU", *Powder Diffraction* **28** (2013), S339-S350 (detector geometry definition, azimuthal integration) -- H. Powell, "The Rossmann Fourier autoindexing algorithm in MOSFLM", *Acta Cryst.* **D55** (1999), 1690-1695 (FFT indexing) +- I. Steller, R. Bolotovsky & M. G. Rossmann, "An algorithm for automatic indexing of oscillation images using Fourier analysis", *J. Appl. Cryst.* **30** (1997), 1036-1040 (the projection/1D-FFT autoindexing algorithm of §5). +- H. Powell, "The Rossmann Fourier autoindexing algorithm in MOSFLM", *Acta Cryst.* **D55** (1999), 1690-1695 (the MOSFLM implementation of it, whose practice is followed) +- P. Gasparotto, L. Barba, H.-C. Stadler et al., "TORO Indexer: a PyTorch-based indexing algorithm for kilohertz serial crystallography", *J. Appl. Cryst.* **57** (2024), 931-944 (the algorithm of the `ffbidx` fast-feedback indexer, §4). +- I. Křivý & B. Gruber, "A unified algorithm for determining the reduced (Niggli) cell", *Acta Cryst.* **A32** (1976), 297-298, and International Tables for Crystallography Vol. A, Table 9.2.5.1 (the Niggli reduction and the lattice-character table of §5.3/§6). +- J. E. Padilla & T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and pseudo-centering and utility for detecting twinning", *Acta Cryst.* **D59** (2003), 1124-1130 (the L test, §13.2). +- A. Barty, R. A. Kirian, F. R. N. C. Maia et al., "Cheetah: software for high-throughput reduction and analysis of serial femtosecond X-ray diffraction data", *J. Appl. Cryst.* **47** (2014), 1118-1131 (peakfinder8: the per-resolution-ring background statistics of §3.2). +- A. Hennequin, B. Couturier, V. V. Gligorov & L. Lacassagne, "SparseCCL: Connected Components Labeling and Analysis for sparse images", DASIP 2019, 65-70 (the connected-component labelling of §3.4, used via ACTS/traccc). - S. French & K. Wilson, "On the treatment of negative intensity observations", *Acta Cryst.* **A34** (1978), 517-525 (Bayesian amplitude estimation from intensities). - A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures", *Nature* **355** (1992), 472-475 (R-free cross-validation). - M. Wojdyr, "GEMMI: A library for structural biology", *J. Open Source Softw.* **7** (2022), 4200 (model / structure-factor / map machinery used in §14). @@ -44,7 +50,7 @@ The methods draw on, and in places reimplement, solutions from: - A. G. W. Leslie & H. R. Powell, "Processing diffraction data with MOSFLM" (2007), NATO Science Series II **245**, 41-51 (post-refinement practice: what is refined per image and what over a wedge). - D. W. Moreau, H. Atakisi & R. E. Thorne, "Ice in biomolecular cryocrystallography", *Acta Cryst.* **D77** (2021), 540-554 (measured hexagonal-ice ring positions, used by the ice-ring score, the ice flagging and the ice calibrant). - K. Röttger, A. Endriss, J. Ihringer, S. Doyle & W. F. Kuhs, "Lattice constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K", *Acta Cryst.* **B50** (1994), 644-648 (the ice Ih cell the ring positions below 1.522 Å are calculated from). -- S. Sheriff & W. A. Hendrickson, "Description of overall anisotropy in diffraction from macromolecular crystals", *Acta Cryst.* **A43** (1987), 118-121, and A. N. Popov & G. P. Bourenkov, *Acta Cryst.* **D59** (2003), 1145-1153 (the overall anisotropic B tensor, its symmetry constraints, and its estimation from the observed intensities). +- S. Sheriff & W. A. Hendrickson, "Description of overall anisotropy in diffraction from macromolecular crystals", *Acta Cryst.* **A43** (1987), 118-121 (the overall anisotropic B tensor and its symmetry constraints), and A. N. Popov & G. P. Bourenkov, "Choice of data-collection parameters based on statistic modelling", *Acta Cryst.* **D59** (2003), 1145-1153 (the sigma-aware estimation of the anisotropy of the observed intensity distribution, part of that paper's statistic modelling). - P. R. Evans & G. N. Murshudov, "How good are my data and what is the resolution?", *Acta Cryst.* **D69** (2013), 1204-1214 (AIMLESS: the anisotropic deltaB as the range of the principal components, and diffraction limits from a cone about each principal direction). - K. Diederichs & P. A. Karplus, *Nat. Struct. Biol.* **4** (1997), 269-275, and P. A. Karplus & K. Diederichs, *Science* **336** (2012), 1030-1033 (R_meas / R_pim, CC1/2 and CC\*). - IUCr Commission on Crystallographic Nomenclature, "Statistical descriptors in crystallography", *Acta Cryst.* **A45** (1989), 63-75, and *Acta Cryst.* **A51** (1995), 565-569 (uncertainty conventions). diff --git a/image_analysis/indexing/FFBIDXIndexer.cpp b/image_analysis/indexing/FFBIDXIndexer.cpp index 6079826cb..fb62a7040 100644 --- a/image_analysis/indexing/FFBIDXIndexer.cpp +++ b/image_analysis/indexing/FFBIDXIndexer.cpp @@ -4,6 +4,9 @@ #include "FFBIDXIndexer.h" #include "PostIndexingRefinement.h" +// The ffbidx library implements the TORO algorithm: +// Gasparotto et al. (2024) J. Appl. Cryst. 57, 931-944 + void FFBIDXIndexer::SetupUnitCell(const std::optional &cell) { if (!cell.has_value()) throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, diff --git a/image_analysis/indexing/FFTIndexer.cpp b/image_analysis/indexing/FFTIndexer.cpp index e994c6a94..3405b8784 100644 --- a/image_analysis/indexing/FFTIndexer.cpp +++ b/image_analysis/indexing/FFTIndexer.cpp @@ -6,6 +6,7 @@ #include #include "PostIndexingRefinement.h" +// Following Steller, Bolotovsky & Rossmann (1997) J. Appl. Cryst. 30, 1036-1040 FFTIndexer::FFTIndexer(const IndexingSettings &settings) : max_length_A(settings.GetFFT_MaxUnitCell_A()), min_length_A(settings.GetFFT_MinUnitCell_A()), diff --git a/image_analysis/lattice_search/LatticeSearch.cpp b/image_analysis/lattice_search/LatticeSearch.cpp index 0a1544d87..fc8951821 100644 --- a/image_analysis/lattice_search/LatticeSearch.cpp +++ b/image_analysis/lattice_search/LatticeSearch.cpp @@ -42,6 +42,7 @@ std::optional SearchCharacters(const CrystalLattice &L, dou const std::pair *only_class) { UnitCell uc = L.GetUnitCell(); gemmi::UnitCell g_uc(uc.a, uc.b, uc.c, uc.alpha, uc.beta, uc.gamma); + // Niggli reduction following Krivy & Gruber (1976) Acta Cryst. A32, 297-298, via gemmi gemmi::GruberVector g_vec(g_uc, 'P', true); // The reduction decides the Niggli TYPE from the signs of the three scalar products, and gemmi's // default epsilon is 1e-9 ABSOLUTE while those products are 10^3 to 10^5 A^2 on a cell held in @@ -71,6 +72,7 @@ std::optional SearchCharacters(const CrystalLattice &L, dou double F = g_vec.zeta / 2; // D, E, F are parameters so the table can also be built for the type-flipped setting below. + // Lattice characters following International Tables for Crystallography Vol. A, Table 9.2.5.1 auto make_classes = [&](double D, double E, double F) { return std::vector{ { diff --git a/image_analysis/scale_merge/TwinningAnalysis.cpp b/image_analysis/scale_merge/TwinningAnalysis.cpp index 26b365e6d..8ae183ace 100644 --- a/image_analysis/scale_merge/TwinningAnalysis.cpp +++ b/image_analysis/scale_merge/TwinningAnalysis.cpp @@ -59,6 +59,7 @@ TwinningAnalysisResult AnalyzeTwinning(const std::vector& merg }; // --- L-test --- + // Following Padilla & Yeates (2003) Acta Cryst. D59, 1124-1130 // Pair each reflection with a symmetry-independent neighbour two steps away along an axis (the // step of 2 keeps the partner local in resolution while avoiding the reflection itself). The // merged reflections are unique in the asymmetric unit, so any other merged reflection is diff --git a/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp b/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp index 9f7cd16ad..2171912e4 100644 --- a/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp +++ b/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp @@ -23,6 +23,8 @@ AdaptiveSpotFinderCPU::AdaptiveSpotFinderCPU(const AzimuthalIntegrationMapping & ring_bits.assign(OutputSize(), 0); } +// Per-ring background statistics with iterated peak exclusion following peakfinder8: +// Barty et al. (2014) J. Appl. Cryst. 47, 1118-1131 // Accumulate per-ring mean/variance from the raw (photon) image. clip_k <= 0 -> use every valid // pixel (first pass); clip_k > 0 -> keep only pixels within clip_k sigma of the current ring mean, // which removes the Bragg peaks from the background estimate. diff --git a/image_analysis/spot_finding/StrongPixelSet.cpp b/image_analysis/spot_finding/StrongPixelSet.cpp index 0c12cc241..b0f567a65 100644 --- a/image_analysis/spot_finding/StrongPixelSet.cpp +++ b/image_analysis/spot_finding/StrongPixelSet.cpp @@ -4,6 +4,7 @@ // SparseCCL code taken from https://github.com/acts-project/traccc/blob/main/core/include/traccc/clusterization/detail/sparse_ccl.hpp // (c) 2021-2022 CERN for the benefit of the ACTS project // Mozilla Public License Version 2.0 +// The algorithm: Hennequin, Couturier, Gligorov & Lacassagne (2019) DASIP 2019, 65-70 (SparseCCL) // // The union-find and the two-scan structure are theirs. How a pixel's earlier neighbours are FOUND // is not: see sparseccl below.