diff --git a/common/ColorScale.h b/common/ColorScale.h index 0daac58b2..501f7ab53 100644 --- a/common/ColorScale.h +++ b/common/ColorScale.h @@ -34,6 +34,7 @@ enum class ColorScaleSpecial { }; class ColorScale { + // Viridis, Magma and Inferno are Matplotlib's colormaps, sampled. Following Hunter (2007) Comput. Sci. Eng. 9, 90-95 const std::vector viridis_colormap = { {68, 1, 84}, {71, 44, 123}, {59, 81, 139}, {44, 113, 142}, {33, 144, 141}, {39, 173, 129}, {92, 200, 99}, {170, 220, 50}, {253, 231, 37} diff --git a/docs/ACKNOWLEDGEMENT.md b/docs/ACKNOWLEDGEMENT.md index ba9f14c24..0a8ddab27 100644 --- a/docs/ACKNOWLEDGEMENT.md +++ b/docs/ACKNOWLEDGEMENT.md @@ -22,47 +22,53 @@ The project is supported by: * ETH Domain via Open Research Data Contribute project (Jan - Dec 2023). * AMD University Program with donation of licenses of Ethernet IP cores and Vivado software. -## Automatic processing as an expert system - -**Global Phasing Ltd** — much of what rugnux tries to do was inspired by the work of Gérard Bricogne -and his colleagues at [Global Phasing](https://www.globalphasing.com/): the Fourier-transform -indexing its de-novo indexers descend from, which Bricogne proposed in 1986 (see -[Indexing](#indexing) below), the view of data processing as an expert system, the distrust of -image-header metadata, and the attention paid to diffraction anisotropy, each described below. - -**[autoPROC](https://www.globalphasing.com/autoproc/)** — rugnux's goal is a fully automatic -pipeline that takes the decisions an experienced crystallographer would take while processing a -dataset, and reports each decision together with the evidence it rests on, so that a bare -invocation gives the best answer available. autoPROC is the inspiration and the target for that -behaviour, and two of rugnux's working principles come from it. The first, and the one that has -guided rugnux most, we owe to Clemens Vonrhein: a processing pipeline should not take the -experimental metadata it is given - the beam centre, the detector geometry, the goniometer and its -rotation axis - on trust, but verify it against the images and, where it is wrong, correct it. We -are grateful to him for arguing this so clearly and for so long. The second follows from it: the -beam centre should be determined from the images themselves; rugnux does this in its own way, -described in §1.4 of the [data-analysis reference](CPU_DATA_ANALYSIS_IMAGE.md). The -reasons rugnux gives for a weak stretch of a sweep also use autoPROC's wording ("loss of centring -during crystal rotation"). C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, -T. Womack and G. Bricogne, "Data processing and analysis with the autoPROC toolbox" (2011), -Acta Cryst. D67, 293-302 -[doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773). - -**[STARANISO](https://staraniso.globalphasing.org/)** — the understanding that diffraction -anisotropy matters, and that every run should analyse and report it, was inspired by STARANISO. -The estimators rugnux uses for it are those credited under -[Scaling, merging and data quality](#scaling-merging-and-data-quality) below. I. J. Tickle, -C. Flensburg, P. Keller, W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO -([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)), -Cambridge, United Kingdom: Global Phasing Ltd. - ## Crystallographic methods adopted from other packages The analysis pipeline builds on methods first published, and in most cases first implemented, by -other crystallographic software. The code below is Jungfraujoch's own; the methods are theirs, and -are acknowledged here. Where a package's source was consulted this is said explicitly. Most of these -packages are neither linked nor vendored; the three that are - GEMMI, traccc and -fast-feedback-indexer - also carry a licence obligation, recorded in -[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). +other crystallographic software. The code is Jungfraujoch's own; the methods are theirs, and are +acknowledged here, entry by entry: what was taken, from whom, and the citation. How rugnux uses each +method is described in the [data-analysis reference](CPU_DATA_ANALYSIS.md), not here. Where a +package's source was consulted this is said explicitly. Most of these packages are neither linked nor +vendored; the three that are - GEMMI, traccc and fast-feedback-indexer - also carry a licence +obligation, recorded in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). + +### Experimental geometry and physical corrections + +**[autoPROC](https://www.globalphasing.com/autoproc/)** — the principle, long argued for by Clemens +Vonrhein, that a pipeline does not take the experimental metadata it is given (beam centre, detector +geometry, goniometer axis) on trust but verifies it against the images and corrects it where it is +wrong, and with it that the beam centre is found from the images; the aim of a pipeline that takes the +decisions an expert would take and reports the evidence for each. C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, "Data +processing and analysis with the autoPROC toolbox" (2011), Acta Cryst. D67, 293-302 +[doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773). + +**Sensor absorption and air path** — the attenuation coefficients of silicon, CdTe, dry air and +helium behind the oblique-incidence sensor efficiency, the parallax variance and the air-path +attenuation are the NIST tabulation. J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray +Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for +Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004), +NIST Standard Reference Database 126 +[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F). + +**Polarization correction** — the azimuthal polarization factor for a partially polarized +synchrotron source. R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular +crystallography with synchrotron radiation: photographic data collection and polarization +correction" (1982), J. Appl. Cryst. 15, 330-337 +[doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060). + +**Hexagonal-ice ring positions** — the eleven measured ring spacings from 3.895 to 1.522 Å. +D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in biomolecular cryocrystallography" (2021), Acta +Cryst. D77, 540-554 [doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170). +The bands below 1.522 Å are calculated here from ice Ih structure factors on the lattice constants +of K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice constants and thermal +expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, 644-648 +[doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). + +**Powder calibrants** — the silver behenate cell is T. C. Huang, H. Toraya, T. N. Blanton and Y. Wu, +"X-ray powder diffraction analysis of silver behenate, a possible low-angle diffraction standard" +(1993), J. Appl. Cryst. 26, 180-184 +[doi:10.1107/S0021889892009762](https://doi.org/10.1107/S0021889892009762); the CeO2 and silicon +cells are those of the NIST standard reference materials SRM 674b and SRM 640 (certificates, no DOI). ### Spot finding @@ -71,63 +77,57 @@ region, the serial/stills processing model, and the per-frame indexing acceptanc (`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass, A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012), J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312). -The self-calibrating spot finder's per-resolution-ring background statistics, with the Bragg peaks -excluded by iterated clipping, follow Cheetah's peakfinder8: A. Barty, R. A. Kirian, -F. R. N. C. Maia, M. Hantke, C. H. Yoon, T. A. White and H. N. Chapman, "Cheetah: software for -high-throughput reduction and analysis of serial femtosecond X-ray diffraction data" (2014), -J. Appl. Cryst. 47, 1118-1131 + +**Cheetah** — the per-resolution-ring background statistics with the Bragg peaks excluded by +iterated clipping, from peakfinder8. A. Barty, R. A. Kirian, F. R. N. C. Maia, M. Hantke, C. H. Yoon, +T. A. White and H. N. Chapman, "Cheetah: software for high-throughput reduction and analysis of +serial femtosecond X-ray diffraction data" (2014), J. Appl. Cryst. 47, 1118-1131 [doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626). -**Per-crystal post-refinement of serial stills** - refining, for each still, the partiality -parameters the spot positions cannot pin (a small reorientation, the crystal's own rocking width and the -radial scale of its reciprocal lattice, which a shot's photon energy or a crystal's cell moves) against -the merged intensities follows the post-refinement of PRIME and the mosaic-block model of Sauter et al.: -M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, A. T. Brunger -and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging biological systems -from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter, J. Hattne, A. S. Brewster, -N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and physical properties from -single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not verified here, so not given). +**Poisson significance threshold** — beyond the summation limit the Poisson quantile of the adaptive +spot-finding threshold is taken from the Cornish-Fisher expansion. E. A. Cornish and R. A. Fisher, +"Moments and cumulants in the specification of distributions" (1938), Rev. Int. Stat. Inst. 5, +307-320 [doi:10.2307/1400905](https://doi.org/10.2307/1400905). -Spot extraction groups strong pixels into spots with the sparse connected-component labelling of the -ACTS traccc project: P. Gessinger, H. M. Gray, A. Krasznahorkay, C. Leggett, J. Niermann, -A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction library for HEP -experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822); -[traccc](https://github.com/acts-project/traccc). The CPU spot extractor adapts its SparseCCL source, -and the CUDA spot extractor follows the design of its GPU counterpart - a backward-neighbour graph -over a sorted hit list, resolved by a parallel union-find. traccc is MPL-2.0; see -[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). The SparseCCL algorithm itself is A. Hennequin, -B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: Connected Components Labeling and -Analysis for sparse images" (2019), DASIP 2019, 65-70 +**GPU spot finder** — the CUDA spot finder was developed by Hans-Christian Stadler (PSI). + +**[traccc](https://github.com/acts-project/traccc)** — the sparse connected-component labelling +that groups strong pixels into spots: the CPU extractor adapts its SparseCCL source, the CUDA +extractor follows the design of its GPU counterpart (MPL-2.0; see +[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). P. Gessinger, H. M. Gray, A. Krasznahorkay, +C. Leggett, J. Niermann, A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction +library for HEP experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822). The +SparseCCL algorithm itself: A. Hennequin, B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: +Connected Components Labeling and Analysis for sparse images" (2019), DASIP 2019, 65-70 [doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184). -**XDS and DISTL** — a spot is bounded by its shape (how much of its bounding box it fills) rather than -by its size alone, as XDS (cited under [Integration and rotation geometry](#integration-and-rotation-geometry)) and DISTL bound theirs; the statistic -is rugnux's own, the choice of shape over size is theirs. Z. Zhang, N. K. Sauter, H. van den Bedem, -G. Snell and A. M. Deacon, "Automated diffraction image analysis and spot searching for high-throughput -crystal screening" (2006), J. Appl. Cryst. 39, 112-119 +**XDS and DISTL** — bounding a spot by its shape (how much of its bounding box it fills) rather than +by its size; the statistic is rugnux's own. XDS is cited under +[Integration and rotation geometry](#integration-and-rotation-geometry); Z. Zhang, N. K. Sauter, +H. van den Bedem, G. Snell and A. M. Deacon, "Automated diffraction image analysis and spot searching +for high-throughput crystal screening" (2006), J. Appl. Cryst. 39, 112-119 [doi:10.1107/S0021889805040677](https://doi.org/10.1107/S0021889805040677). ### Indexing -**Fourier-transform indexing** — indexing a diffraction pattern, with no prior knowledge of the -cell, by Fourier-transforming the reciprocal-space positions of its spots was proposed by Gérard -Bricogne as a three-dimensional transform; the de-novo FFT indexers (`-X fft`, `-X fftw`) use the -one-dimensional projection form of that idea described in the next paragraph. G. Bricogne (1986), -in *Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software -(Phase III)*, LURE, Paris, p. 28 (a workshop proceedings without a DOI). +**Fourier-transform indexing** — indexing with no prior knowledge of the cell by Fourier-transforming +the reciprocal-space positions of the spots, proposed as a three-dimensional transform; the de-novo +indexers (`-X fft`, `-X fftw`) use its one-dimensional projection form. G. Bricogne (1986), in +*Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software (Phase III)*, +LURE, Paris, p. 28 (a workshop proceedings without a DOI). -**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/mosflm/)** — the Rossmann FFT autoindexing algorithm and -post-refinement practice, including which parameters are safe to refine per image and which must be -refined over a wedge. The autoindexing algorithm itself — projecting the reciprocal-space points -onto many directions and Fourier-transforming the 1D projection histograms — is I. Steller, -R. Bolotovsky and M. G. Rossmann, "An algorithm for automatic indexing of oscillation images using -Fourier analysis" (1997), J. Appl. Cryst. 30, 1036-1040 -[doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777); MOSFLM is the -implementation whose practice is followed, described in H. R. Powell, "The Rossmann Fourier -autoindexing algorithm in MOSFLM" (1999), Acta Cryst. D55, 1690-1695 -[doi:10.1107/S0907444999009506](https://doi.org/10.1107/S0907444999009506). A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM" -(2007), in *Evolving Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245, -41-51 [doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4); +**Rossmann FFT autoindexing** — projecting the reciprocal-space points onto many directions and +Fourier-transforming the 1D histograms. I. Steller, R. Bolotovsky and M. G. Rossmann, "An algorithm +for automatic indexing of oscillation images using Fourier analysis" (1997), J. Appl. Cryst. 30, +1036-1040 [doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777). + +**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/mosflm/)** — the practice of that algorithm, and +the post-refinement practice of which parameters are refined per image and which over a wedge. +H. R. Powell, "The Rossmann Fourier autoindexing algorithm in MOSFLM" (1999), Acta Cryst. D55, +1690-1695 [doi:10.1107/S0907444999009506](https://doi.org/10.1107/S0907444999009506); +A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM" (2007), in *Evolving +Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245, 41-51 +[doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4); T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new graphical interface for diffraction-image processing with MOSFLM" (2011), Acta Cryst. D67, 271-281 [doi:10.1107/S0907444910048675](https://doi.org/10.1107/S0907444910048675); H. R. Powell, @@ -136,186 +136,170 @@ diffraction data with the graphical user interface iMosflm" (2017), Nat. Protoc. [doi:10.1038/nprot.2017.037](https://doi.org/10.1038/nprot.2017.037). **[fast-feedback-indexer](https://github.com/paulscherrerinstitute/fast-feedback-indexer)** — the -known-cell indexer for serial stills (`-X ffbidx`) is PSI's fast-feedback-indexer library, linked at -build time (BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), which implements -the TORO algorithm: P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton, -M. Janousch, F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for -kilohertz serial crystallography" (2024), J. Appl. Cryst. 57, 931-944 +known-cell indexer for serial stills (`-X ffbidx`), PSI's library linked at build time +(BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), implementing TORO. +P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton, M. Janousch, +F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for kilohertz serial +crystallography" (2024), J. Appl. Cryst. 57, 931-944 [doi:10.1107/S1600576724003182](https://doi.org/10.1107/S1600576724003182). The CPU implementation -used without a GPU (`image_analysis/indexing/FFBIDXIndexerCPU.cpp`) is a port of the library's CUDA -kernels by H.-C. Stadler (PSI): the same samples and scores, with the loops reorganised for a CPU. +used without a GPU is a port of the library's CUDA kernels by H.-C. Stadler (PSI). ### Cell reduction and lattice symmetry **[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and -structure-factor machinery, and MTZ / XDS_ASCII I/O. Vendored in `gemmi_gph/`, so it also carries a +structure-factor machinery, and MTZ / XDS_ASCII I/O; vendored in `gemmi_gph/`, so it also carries a licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). -**Křivý & Gruber's Niggli reduction, and the lattice-character table** — the reduction that puts -every candidate cell in a comparable form is I. Křivý and B. Gruber, "A unified algorithm for -determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298 -[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636), used through GEMMI's -implementation; the table of lattice characters that maps a reduced cell to Bravais lattices and -centrings follows International Tables for Crystallography Vol. A, Table 9.2.5.1. +**Niggli reduction** — used through GEMMI's implementation. I. Křivý and B. Gruber, "A unified +algorithm for determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298 +[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636). The table of lattice +characters that maps a reduced cell to Bravais lattices and centrings follows International Tables +for Crystallography Vol. A, Table 9.2.5.1. -**Grosse-Kunstleve, Sauter & Adams's numerically stable cell reduction** - the magnitude-scaled -tolerance that decides the sign of a structurally-zero scalar product, and with it the Niggli type a -reduced cell is presented in. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically +**Numerically stable cell reduction** — the magnitude-scaled tolerance that decides the sign of a +structurally-zero scalar product. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically stable algorithms for the computation of reduced unit cells" (2004), Acta Cryst. A60, 1-6 [doi:10.1107/S010876730302186X](https://doi.org/10.1107/S010876730302186X). -**Le Page's metric-symmetry search** - the obliquity of each of the 81 candidate two-folds of a -reduced cell, which is what tells a run that its lattice metric hosts more rotational symmetry than -the group its intensities supported, and the derivation of the conventional axes from that -rotation group, which is what the run then offers to the space-group search as a second lattice -candidate. The two-fold search is used through GEMMI's implementation of it. Y. Le Page, "The -derivation of the axes of the conventional unit cell from the dimensions of the Buerger-reduced -cell" (1982), J. Appl. Cryst. 15, 255-259 -[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959). +**Metric-symmetry search** — the obliquity of the 81 candidate two-folds of a reduced cell, and the +conventional axes derived from the rotation group found; the two-fold search is used through GEMMI. +Y. Le Page, "The derivation of the axes of the conventional unit cell from the dimensions of the +Buerger-reduced cell" (1982), J. Appl. Cryst. 15, 255-259 +[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959). The step from the +two-folds found to a conventional cell, a centring and an integral change of basis follows +R. W. Grosse-Kunstleve, "Algorithms for deriving crystallographic space-group information" (1999), +Acta Cryst. A55, 383-395 [doi:10.1107/S0108767398010186](https://doi.org/10.1107/S0108767398010186). ### Integration and rotation geometry **[XDS](https://xds.mr.mpg.de/)** — rotation geometry and notation, the reciprocal Lorentz and -partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for rejecting -a reflection whose predicted profile is not cleanly its own, the intensity-based test for a -centred lattice, the recognition of shaded detector regions by comparing a pixel's background -against the background at its own resolution (`DEFPIX`), and the scaling correction surfaces +partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for a +reflection whose predicted profile is not cleanly its own, the intensity-based test for a centred +lattice, the recognition of shaded detector regions (`DEFPIX`), and the scaling correction surfaces indexed by image number and detector region. W. Kabsch, "XDS" (2010), Acta Cryst. D66, 125-132 [doi:10.1107/S0907444909047337](https://doi.org/10.1107/S0907444909047337); W. Kabsch, "Integration, scaling, space-group assignment and post-refinement" (2010), Acta Cryst. D66, 133-144 [doi:10.1107/S0907444909047374](https://doi.org/10.1107/S0907444909047374). -**Profile fitting** with reweighted, de-biased variances is the Kabsch/Otwinowski iteration, from the +**Profile fitting** — the Kabsch/Otwinowski iteration with reweighted, de-biased variances, from the second XDS paper above and from Z. Otwinowski and W. Minor, "Processing of X-ray diffraction data collected in oscillation mode" (1997), Methods Enzymol. 276, 307-326 [doi:10.1016/S0076-6879(97)76066-X](https://doi.org/10.1016/S0076-6879%2897%2976066-X). -**The two-dimensional integration architecture** — integrating each image in the detector plane -and only afterwards assembling a reflection's partials into a full across images, as against -three-dimensional profile fitting through the image stack — is the architecture of DENZO/SCALEPACK -and MOSFLM, and it is the one Rugnux's rotation pipeline follows (per-image profile-fitted -integration, then partials combined into fulls; §9 and §10.6 of the [data-analysis reference](CPU_DATA_ANALYSIS.md)). The -Otwinowski & Minor citation above and the MOSFLM citations below carry the credit for the paradigm -as well as for the specifics taken from each. +**Two-dimensional integration** — integrating each image in the detector plane and only afterwards +combining a reflection's partials into a full, as against three-dimensional profile fitting through +the image stack, is the architecture of DENZO/SCALEPACK (Otwinowski and Minor, above) and MOSFLM +(cited under [Indexing](#indexing)). ### Space group, twinning and pseudo-symmetry -**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search. Stage A scores each -candidate rotation operator by the correlation of I(h) with I(Rh) on **resolution-normalised** -intensities (E²), as POINTLESS does — both arms of a symmetry pair sit at the same |s|, so on raw -intensities the resolution fall-off is variance shared between them and lifts a false operator's -correlation as much as a true one's; the screw-axis test scores a -predicted-absent class against the rest of its own axial row rather than against a global mean or a -fixed cut, and lets confidence fall away with the number of axial reflections instead of refusing -below a count; the glide-plane test is that same test applied to a zone, scoring the extinguished -class against the rest of its own plane, as POINTLESS scores zonal absences. P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82 -[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693); P. R. Evans, "An -introduction to data reduction: space-group determination, scaling and intensity statistics" (2011), -Acta Cryst. D67, 282-292 [doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X); -P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta -Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061); -J. Agirre, M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular +**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search: scoring each candidate +rotation operator by the correlation of I(h) with I(Rh) on resolution-normalised intensities (E²); +the screw-axis test that scores a predicted-absent class against the rest of its own axial row and +lets confidence fall with the number of axial reflections instead of refusing below a count; and the +same test on a zone for a glide plane. P. Evans, "Scaling and assessment of data quality" (2006), +Acta Cryst. D62, 72-82 [doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693); +P. R. Evans, "An introduction to data reduction: space-group determination, scaling and intensity +statistics" (2011), Acta Cryst. D67, 282-292 +[doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X); P. R. Evans and +G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta Cryst. D69, +1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061); J. Agirre, +M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular crystallography" (2023), Acta Cryst. D79, 449-461 [doi:10.1107/S2059798323003595](https://doi.org/10.1107/S2059798323003595). -**Baur and Kassner** — the convention for which of two absence-equivalent glide groups the space-group -search writes (P2/c over Pc, C2/c over Cc): the centrosymmetric one, because a missed inversion centre is -the common error among published small-molecule space-group assignments. W. H. Baur and D. Kassner, -"The perils of Cc: comparing the frequencies of falsely assigned space groups with their general -population" (1992), Acta Cryst. B48, 356-369 +**Centrosymmetric glide groups** — of two absence-equivalent glide groups the search writes the +centrosymmetric one (P2/c over Pc, C2/c over Cc), a missed inversion centre being the common error. +W. H. Baur and D. Kassner, "The perils of Cc: comparing the frequencies of falsely assigned space +groups with their general population" (1992), Acta Cryst. B48, 356-369 [doi:10.1107/S0108768191014726](https://doi.org/10.1107/S0108768191014726). -**The centre-of-symmetry statistics** are Wilson's and Howells, Phillips & Rogers's: the intensity -distributions of acentric and centric structures and the cumulative N(z) test built on them, read here -on the general reflections of the Laue class beside <|E^2-1|> and Padilla and Yeates's L test (taken at -its centric value, 2/pi). A. J. C. Wilson, "The probability distribution of X-ray intensities" (1949), -Acta Cryst. 2, 318-321 [doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); -E. R. Howells, D. C. Phillips and D. Rogers, "The probability distribution of X-ray intensities. II. -Experimental investigation and the X-ray detection of centres of symmetry" (1950), Acta Cryst. 3, -210-214 [doi:10.1107/S0365110X50000513](https://doi.org/10.1107/S0365110X50000513). +**Centre-of-symmetry statistics** — the acentric and centric intensity distributions and the +cumulative N(z) test built on them, read beside <|E^2-1|> and the L test. A. J. C. Wilson, "The +probability distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321 +[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); E. R. Howells, +D. C. Phillips and D. Rogers, "The probability distribution of X-ray intensities. II. Experimental +investigation and the X-ray detection of centres of symmetry" (1950), Acta Cryst. 3, 210-214 +[doi:10.1107/S0365110X50000513](https://doi.org/10.1107/S0365110X50000513). -**The twinning L test** is Padilla and Yeates's: pairing each acentric reflection with a -symmetry-independent neighbour and reading the first and second moments of -L = (I1−I2)/(I1+I2) against their untwinned and perfect-twin values. J. E. Padilla and -T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and -pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, 1124-1130 -[doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). Their title claims -robustness to pseudo-centering, and this program's partner steps deliver it: a step of 2 along an -axis preserves the class of a half-integer pseudo-translation, which is what a pseudo-centering is. -That robustness does not extend to a pseudo-translation which is not half-integer, and the partner -steps are restricted when one is detected - see the translational-pseudo-symmetry note below. +**Twinning L test** — the moments of L = (I1−I2)/(I1+I2) over pairs of neighbouring acentric +reflections against their untwinned and perfect-twin values; its robustness to pseudo-centering is +kept by the partner steps, which are restricted when a non-half-integer pseudo-translation is found. +J. E. Padilla and T. O. Yeates, "A statistic for local intensity differences: robustness to +anisotropy and pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, +1124-1130 [doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). The +test's design - indicators on shell-normalised merged intensities, a twin law admitted within +3 degrees of obliquity - follows phenix.xtriage: Zwart, Grosse-Kunstleve and Adams (2005, cited under +[Data quality and anisotropy](#data-quality-and-anisotropy)); D. Liebschner, P. V. Afonine, +M. L. Baker et al., "Macromolecular structure determination using X-rays, neutrons and electrons: +recent developments in Phenix" (2019), Acta Cryst. D75, 861-877 +[doi:10.1107/S2059798319011471](https://doi.org/10.1107/S2059798319011471). -**The twin fraction of the twin hypothesis, and the twinned model it is scored with**, follow Yeates: -the intensity a twinned crystal records is (1−α) I(h) + α I(Th), and the disagreement -H = |I(h) − I(Th)|/(I(h) + I(Th)) of the pairs a law relates is uniform on [0, 1−2α], so the median -gives the fraction. T. O. Yeates, "Detecting and overcoming crystal twinning" (1997), Methods Enzymol. -276, 344-358 [doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879%2897%2976068-3). +**Twin fraction and twinned intensity** — the model (1−α) I(h) + α I(Th), and the fraction read +from the median of H = |I(h) − I(Th)|/(I(h) + I(Th)), uniform on [0, 1−2α]. T. O. Yeates, "Detecting +and overcoming crystal twinning" (1997), Methods Enzymol. 276, 344-358 +[doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879%2897%2976068-3). -**Twin refinement with more than two domains, and the free set closed under the twin domains**, follow -REFMAC5's treatment - one fraction per domain operator of the lattice, refined with the model, small domains -dropped - and its practice of keeping twin-related reflections in one set: -G. N. Murshudov, P. Skubák, A. A. Lebedev, N. S. Pannu, R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and -A. A. Vagin, "REFMAC5 for the refinement of macromolecular crystal structures" (2011), Acta Cryst. D67, 355-367 -[doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314); P. Roversi, E. Blanc, S. Johnson and -S. M. Lea, "Tetartohedral twinning could happen to you too" (2012), Acta Cryst. D68, 418-424 -[doi:10.1107/S0907444912006737](https://doi.org/10.1107/S0907444912006737). +**Twin refinement** — one fraction per domain operator of the lattice refined with the model, small +domains dropped, and twin-related reflections kept in one free set, as REFMAC5 does; the fractions +re-fitted by least squares at every cycle as REFMAC5 refines them, against an amplitude likelihood +with the twinned model amplitude in it (a simplification of REFMAC5's twinned-intensity target). +G. N. Murshudov, P. Skubák, A. A. Lebedev, N. S. Pannu, R. A. Steiner, R. A. Nicholls, M. D. Winn, +F. Long and A. A. Vagin, "REFMAC5 for the refinement of macromolecular crystal structures" (2011), +Acta Cryst. D67, 355-367 [doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314); +P. Roversi, E. Blanc, S. Johnson and S. M. Lea, "Tetartohedral twinning could happen to you too" +(2012), Acta Cryst. D68, 418-424 +[doi:10.1107/S0907444912006737](https://doi.org/10.1107/S0907444912006737); the six-domain case of +a monoclinic crystal on a hexagonal metric is K. A. Sultana, I. Alexeev, I. Kursula, P. Mäntsälä, +J. Niemi and G. Schneider, "Structure determination by multiwavelength anomalous diffraction of +aclacinomycin oxidoreductase: indications of multidomain pseudomerohedral twinning" (2007), Acta +Cryst. D63, 149-159 [doi:10.1107/S0907444906044271](https://doi.org/10.1107/S0907444906044271). +A twin fraction as a +parameter refined beside the model is SHELXL's BASF (rugnux re-fits it between cycles rather than in +the same least squares): R. Herbst-Irmer and G. M. Sheldrick, "Refinement of twinned structures with +SHELXL97" (1998), Acta Cryst. B54, 443-449 +[doi:10.1107/S0108768197018454](https://doi.org/10.1107/S0108768197018454). The twin laws come from +the lattice's metric: A. A. Lebedev, A. A. Vagin and G. N. Murshudov, "Intensity statistics in +twinned crystals with examples from the PDB" (2006), Acta Cryst. D62, 83-95 +[doi:10.1107/S0907444905036759](https://doi.org/10.1107/S0907444905036759). -**The symmetry hypotheses a refused promotion leaves** - the adopted group and every other index-2 subgroup -of the refused group perfectly twinned by a law of its coset, all merging alike, carried to the model and -decided by placing and refining it in each - follow the advice to refine in the candidate groups and compare, -rather than decide from intensity statistics, that pseudo-symmetry and twinning call for: -P. H. Zwart, R. W. Grosse-Kunstleve, A. A. Lebedev, G. N. Murshudov and P. D. Adams, "Surprises and pitfalls +**Deciding pseudo-symmetry by refinement** — carrying the adopted group and the index-2 subgroups a +refused promotion leaves, each perfectly twinned by a law of its coset, to the model and deciding by +placing and refining it in each, rather than from intensity statistics. P. H. Zwart, +R. W. Grosse-Kunstleve, A. A. Lebedev, G. N. Murshudov and P. D. Adams, "Surprises and pitfalls arising from (pseudo)symmetry" (2008), Acta Cryst. D64, 99-107 -[doi:10.1107/S090744490705531X](https://doi.org/10.1107/S090744490705531X); A. A. Lebedev and M. N. Isupov, -"Space-group and origin ambiguity in macromolecular structures with pseudo-symmetry and its treatment with the -program Zanuda" (2014), Acta Cryst. D70, 2430-2443 +[doi:10.1107/S090744490705531X](https://doi.org/10.1107/S090744490705531X); A. A. Lebedev and +M. N. Isupov, "Space-group and origin ambiguity in macromolecular structures with pseudo-symmetry +and its treatment with the program Zanuda" (2014), Acta Cryst. D70, 2430-2443 [doi:10.1107/S1399004714014795](https://doi.org/10.1107/S1399004714014795). -**Twin refinement** (§15.12) refines the model against Yeates's twinned intensity, (1−α) I(h) + α I(Th), with the -twin fraction refined beside it as SHELXL refines BASF: R. Herbst-Irmer and G. M. Sheldrick, "Refinement of twinned -structures with SHELXL97" (1998), Acta Cryst. B54, 443-449 -[doi:10.1107/S0108768197018454](https://doi.org/10.1107/S0108768197018454). The twin laws come from the lattice's -metric as A. A. Lebedev, A. A. Vagin and G. N. Murshudov describe, "Intensity statistics in twinned crystals with -examples from the PDB" (2006), Acta Cryst. D62, 83-95 -[doi:10.1107/S0907444905036759](https://doi.org/10.1107/S0907444905036759), and the fractions are refined by -least squares at every cycle, as REFMAC5 refines them (Murshudov et al. 2011, Acta Cryst. D67, 355-367, cited -above). REFMAC5's target is a twinned-intensity likelihood; ours keeps the amplitude likelihood with the twinned -model amplitude in it, a simplification of that. - -**A first-pass cell that is the coincidence lattice of two twin domains** (twinning by reticular -merohedry of index 2 or 3, e.g. obverse/reverse rhombohedral domains or a cubic crystal twinned about -[111]) is recognised by the nodes that belong to neither domain being empty. The twin index is +**Twinning by reticular merohedry** — a first-pass cell that is the coincidence lattice of two twin +domains (reticular merohedry or pseudo-merohedry of index up to 10, e.g. obverse/reverse +rhombohedral domains) is recognised by the nodes of neither domain being empty. The twin index is Friedel's (G. Friedel, *Leçons de cristallographie*, 1926; no DOI); the split of the reflections into -those of one domain, of the other, of both and of neither is the one Herbst-Irmer and Sheldrick use -for obverse/reverse twins: R. Herbst-Irmer and G. M. Sheldrick, "Refinement of obverse/reverse twins" -(2002), Acta Cryst. B58, 477-481 -[doi:10.1107/S0108768102001039](https://doi.org/10.1107/S0108768102001039). +those of one domain, of the other, of both and of neither, and the treatment of reticular +pseudo-merohedry (Herbst-Irmer and Sheldrick 1998, above), are Herbst-Irmer and Sheldrick's: +R. Herbst-Irmer and G. M. Sheldrick, "Refinement of obverse/reverse twins" (2002), Acta Cryst. B58, +477-481 [doi:10.1107/S0108768102001039](https://doi.org/10.1107/S0108768102001039). -**Translational pseudo-symmetry** is detected from the native Patterson computed from the merged -intensities, and the interpretation of an off-origin peak as a pseudo-translation between copies of -the contents of the asymmetric unit - together with the modulation it puts on the intensities, which -is the second half of the test here - is Read, Adams and McCoy's. Their fitted peak-height table is -not used: the peak is scored against a per-dataset within-shell permutation null instead, because the -noise floor of the statistic depends strongly on how many reflections a dataset has. The same -modulation is what the axial systematic-absence test scores against, so that a reflection class a -pseudo-translation merely suppresses is not read as extinct and does not buy a screw axis; the -estimate of its depth there is our own, measured per axial row from the merged intensities rather -than from the Patterson vector. Where a symmetry operator or a twin law carries the vector to one -that is not ±itself, the modulation is also read on classes averaged over the lattice's rotations - -their modulation model summed over the copy pairs each rotation relates, as the measured intensity -sums them; that averaging is our own construction. R. J. Read, -P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of translational -noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183 -[doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374). +**Translational pseudo-symmetry** — the off-origin native-Patterson peak read as a pseudo-translation +between copies of the asymmetric unit, and the modulation it puts on the intensities, which the +axial systematic-absence test also scores against. The per-dataset permutation null for the peak, +the per-row estimate of the modulation depth, and its averaging over the lattice's rotations are +rugnux's own. R. J. Read, P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of +translational noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183 +[doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374). A pseudo-translation +is declared at a Patterson peak of a fifth of the origin or more, Phaser's default (McCoy et al. +2007, cited under [Model placement and molecular replacement](#model-placement-and-molecular-replacement)). -### Scaling, merging and data quality +### Scaling and merging -**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, per-observation -outlier rejection at merge, the scaling error model, and the treatment of a reflection whose -background is contaminated. Its published behaviour, and in places its source, settled several -choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS: implementation and -evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97 +**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, +per-observation outlier rejection at merge, the scaling error model, and the treatment of a +reflection whose background is contaminated; its published behaviour, and in places its source, +settled several choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS: +implementation and evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97 [doi:10.1107/S2059798317017235](https://doi.org/10.1107/S2059798317017235); D. G. Waterman, G. Winter, R. J. Gildea et al., "Diffraction-geometry refinement in the DIALS framework" (2016), Acta Cryst. D72, 558-575 [doi:10.1107/S2059798316002187](https://doi.org/10.1107/S2059798316002187); @@ -325,113 +309,136 @@ package: algorithms and new approaches for multi-crystal scaling" (2020), Acta C G. Winter, D. G. Waterman et al., "Robust background modelling in DIALS" (2016), J. Appl. Cryst. 49, 1912-1921 [doi:10.1107/S1600576716013595](https://doi.org/10.1107/S1600576716013595). -**Wilson outlier test** — judging an observation that has no symmetry mates against the acentric and -centric intensity distributions of its resolution shell, with the symmetry enhancement factor, is -Wilson's statistics; rejecting only observations that are also significant, and keeping a reflection -whose observations are all large, follows AIMLESS's EMAX test. A. J. C. Wilson, "The probability -distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321 -[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); -P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82 +**Wilson outlier test** — judging an observation without symmetry mates against the acentric and +centric distributions of its shell, with the symmetry enhancement factor; rejecting only significant +observations, and keeping a reflection whose observations are all large, follows AIMLESS's EMAX test. +A. J. C. Wilson, "The probability distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321 +[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); P. Evans, "Scaling and +assessment of data quality" (2006), Acta Cryst. D62, 72-82 [doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693). -**Amplitudes from intensities** — the posterior-mean amplitude of each merged intensity under the -acentric and centric Wilson priors is French and Wilson's; giving no amplitude to an intensity more -than 3.7 sigma below zero, and leaving such intensities out of the prior, follows CCP4's -[ctruncate](https://www.ccp4.ac.uk/) (C. Ballard and N. Stein). So does scaling each reflection's -Wilson prior by the anisotropy tensor along its direction, which ctruncate has done by default since -its version 1.7 ("use anisotropy in prior for truncate procedure"); rugnux uses its own tensor for it -(see Diffraction anisotropy below). S. French and K. Wilson, "On the treatment of negative intensity -observations" (1978), Acta Cryst. A34, 517-525 -[doi:10.1107/S0567739478001114](https://doi.org/10.1107/S0567739478001114); ctruncate is cited through -the CCP4 suite: M. D. Winn, C. C. Ballard, K. D. Cowtan et al., -"Overview of the CCP4 suite and current developments" (2011), Acta Cryst. D67, 235-242 +**Amplitudes from intensities** — the posterior-mean amplitude under the acentric and centric Wilson +priors. S. French and K. Wilson, "On the treatment of negative intensity observations" (1978), Acta +Cryst. A34, 517-525 [doi:10.1107/S0567739478001114](https://doi.org/10.1107/S0567739478001114). +Giving no amplitude to an intensity more than 3.7 sigma below zero, and scaling each reflection's +prior by the anisotropy tensor along its direction, follow CCP4's +[ctruncate](https://www.ccp4.ac.uk/) (C. Ballard and N. Stein), cited through the suite: +M. D. Winn, C. C. Ballard, K. D. Cowtan et al., "Overview of the CCP4 suite and current developments" +(2011), Acta Cryst. D67, 235-242 [doi:10.1107/S0907444910045749](https://doi.org/10.1107/S0907444910045749). -**Absorption as spherical harmonics** — describing an empirical absorption correction as a series of -real spherical harmonics of the beam directions in the crystal frame is Blessing's; its use as a +**Absorption as spherical harmonics** — the empirical absorption correction as a series of real +spherical harmonics of the beam directions in the crystal frame is Blessing's; its use as a restrained scaling surface of the diffracted-beam direction follows SCALA and AIMLESS. R. H. Blessing, "An empirical correction for absorption anisotropy" (1995), Acta Cryst. A51, 33-38 [doi:10.1107/S0108767394005726](https://doi.org/10.1107/S0108767394005726); P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82 [doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693). -**Diffraction anisotropy** — the description of the overall fall-off by a single anisotropic -displacement tensor, its symmetry constraints, and the fact that only its deviatoric part is -determined (the isotropic part being degenerate with the overall scale) are Sheriff and Hendrickson's. -The estimator fits that tensor to the observed intensity distribution, taking sigma(I) into account, -in the sense of Popov and Bourenkov. The directional diffraction limits - in a cone about -each principal direction, and the reporting of the anisotropic deltaB as the range of the principal -components - follow AIMLESS. Rugnux reports these; it corrects no intensity and removes no reflection -on a directional criterion. S. Sheriff and W. A. Hendrickson, "Description of overall anisotropy in -diffraction from macromolecular crystals" (1987), Acta Cryst. A43, 118-121 -[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X); A. N. Popov and -G. P. Bourenkov, "Choice of data-collection parameters based on statistic modelling" (2003), Acta -Cryst. D59, 1145-1153 [doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163); -P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta -Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061). - -**Data-quality statistics** follow the established conventions rather than any one program: R_meas -and R_pim, CC1/2 and CC\*, the per-shell CC(model, data) between F^2_calc and F^2_obs, and the -reporting of I/sigma(I). K. Diederichs and P. A. Karplus, "Improved -R-factors for diffraction data analysis in macromolecular crystallography" (1997), Nat. Struct. Biol. -4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269); P. A. Karplus and -K. Diederichs, "Linking crystallographic model and data quality" (2012), Science 336, 1030-1033 -[doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231) (CC\* is also what a joint merge of -several sweeps tests their isomorphism against: CC_AB / (CC\*_A CC\*_B) is the cosine of the angle between -two data sets, their systematic difference apart from their noise, as K. Diederichs, "Dissecting random and -systematic differences between noisy composite data sets" (2017), Acta Cryst. D73, 286-293 -[doi:10.1107/S2059798317000699](https://doi.org/10.1107/S2059798317000699) shows - which is why the join is -decided on the size of that ratio, and a doubtful sweep by the merge with and without it, the idea of -delta-CC1/2 by Assmann, Brehm and Diederichs (2016), below); M. S. Weiss, "Global -indicators of X-ray data quality" (2001), J. Appl. Cryst. 34, 130-135 -[doi:10.1107/S0021889800018227](https://doi.org/10.1107/S0021889800018227) (R_pim beside R_meas and -R_merge); K. Diederichs and -P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222 -[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121). - -**The Whittaker smoother** — the per-frame scale of the rotation fulls is a penalised least-squares -curve (a second-difference penalty, the smoothness chosen by cross-validation) -in the form P. H. C. Eilers gave Whittaker's graduation: P. H. C. Eilers, "A perfect smoother" (2003), +**Per-frame scale smoothing** — the Whittaker smoother (a second-difference penalty, the smoothness +chosen by cross-validation) in Eilers's form. P. H. C. Eilers, "A perfect smoother" (2003), Anal. Chem. 75, 3631-3636 [doi:10.1021/ac034173t](https://doi.org/10.1021/ac034173t); E. T. Whittaker, "On a new method of graduation" (1923), Proc. Edinburgh Math. Soc. 41, 63-75 [doi:10.1017/S0013091500077853](https://doi.org/10.1017/S0013091500077853). -**Fisher's z-transformation** — the cross-validation of the scaling correction surfaces averages the -change of the half-set CC1/2 over resolution shells on atanh(CC), so that the shells near CC = 1, where -a multiplicative error shows, are not outweighed by the noise of the shells without signal. R. A. -Fisher, "Frequency distribution of the values of the correlation coefficient in samples from an -indefinitely large population" (1915), Biometrika 10, 507-521 -[doi:10.2307/2331838](https://doi.org/10.2307/2331838). +**Fisher's z-transformation** — averaging the change of the half-set CC1/2 over resolution shells on +atanh(CC) when the correction surfaces are cross-validated. R. A. Fisher, "Frequency distribution of +the values of the correlation coefficient in samples from an indefinitely large population" (1915), +Biometrika 10, 507-521 [doi:10.2307/2331838](https://doi.org/10.2307/2331838). -**The frame disposition** - which stretches of a rotation sweep are kept, carried at reduced weight or -dropped from the merge - decides every exclusion on delta-CC1/2, the change in the overall CC1/2 when a -group of images is left out, measured in the sigma-tau form so that no random half-dataset split is -involved. The statistic, the Fisher transformation used to compare it across CC1/2 values, its standard -error going as the inverse square root of the reflection count, and the rejection discipline (never -remove a group whose delta-CC1/2 is positive or near zero; remove a little, re-form the reference and -repeat) are all taken from its authors, whose XDSCC12 is the reference implementation. -G. Assmann, W. Brehm and K. Diederichs, "Identification of rogue datasets in serial crystallography" -(2016), J. Appl. Cryst. 49, 1021-1028 -[doi:10.1107/S1600576716005471](https://doi.org/10.1107/S1600576716005471); G. M. Assmann, M. Wang and -K. Diederichs, "Making a difference in multi-data-set crystallography: simple and deterministic +**Frame disposition by delta-CC1/2** — the statistic in its sigma-tau form, the Fisher transformation +used to compare it, its standard error, and the rejection discipline (never remove a group whose +delta-CC1/2 is positive or near zero; remove a little, re-form the reference and repeat), whose +reference implementation is XDSCC12. G. Assmann, W. Brehm and K. Diederichs, "Identification of +rogue datasets in serial crystallography" (2016), J. Appl. Cryst. 49, 1021-1028 +[doi:10.1107/S1600576716005471](https://doi.org/10.1107/S1600576716005471); G. M. Assmann, M. Wang +and K. Diederichs, "Making a difference in multi-data-set crystallography: simple and deterministic data-scaling/selection methods" (2020), Acta Cryst. D76, 636-652 -[doi:10.1107/S2059798320006348](https://doi.org/10.1107/S2059798320006348). That the same statistic -belongs at scaling, applied to groups of images rather than to whole datasets, follows -[DIALS](https://dials.github.io/) (`dials.scale`, delta-CC1/2 image-group filtering): +[doi:10.1107/S2059798320006348](https://doi.org/10.1107/S2059798320006348). Applying it at scaling, +to groups of images rather than whole datasets, follows DIALS (`dials.scale`): J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399 [doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198). -**Several sweeps merged as one** - each sweep's partials combined into fulls on their own, and the -fulls of all the sweeps then scaled together, every sweep with its own frame scales and a relative B -against the joint merge, is how XSCALE merges several data sets (W. Kabsch, "Integration, scaling, -space-group assignment and post-refinement" (2010), above). Reading the radiation dose a sweep was taken -at from its relative B, which orders the sweeps by dose from the data rather than from the order they -were given in, follows K. Diederichs, "Some aspects of quantitative analysis and correction of radiation -damage" (2006), Acta Cryst. D62, 96-101 +**Reasons for a weak stretch of a sweep** — named in the field's own words: "loss of centring" after +autoPROC's "loss of centring during crystal rotation" (cited under +[Experimental geometry and physical corrections](#experimental-geometry-and-physical-corrections)), +and "crystal rotating out of the beam" after the HKL-2000 manual (HKL: Otwinowski and Minor 1997, +cited under [Integration and rotation geometry](#integration-and-rotation-geometry)). + +**Several sweeps merged as one** — each sweep's partials combined into fulls on their own, then the +fulls of all sweeps scaled together with per-sweep frame scales and a relative B, as XSCALE merges +data sets (W. Kabsch, "Integration, scaling, space-group assignment and post-refinement" (2010), +above). Ordering the sweeps by dose from their relative B follows K. Diederichs, "Some aspects of +quantitative analysis and correction of radiation damage" (2006), Acta Cryst. D62, 96-101 [doi:10.1107/S0907444905031537](https://doi.org/10.1107/S0907444905031537). -**Uncertainty conventions** follow the IUCr Commission on Crystallographic Nomenclature: +**Reference sweep of a joint merge** — the sweep whose own merge reaches the finest resolution, as +KAMO takes the data set of the smallest Wilson B. K. Yamashita, K. Hirata and M. Yamamoto, "KAMO: +towards automated data processing for microcrystals" (2018), Acta Cryst. D74, 441-449 +[doi:10.1107/S2059798318004576](https://doi.org/10.1107/S2059798318004576). + +### Serial data + +**Per-crystal post-refinement of stills** — refining per still the partiality parameters the spot +positions cannot pin (a small reorientation, the rocking width, the radial scale of the reciprocal +lattice) against the merged intensities, after PRIME and the mosaic-block model of Sauter et al. +M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, +A. T. Brunger and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging +biological systems from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter, +J. Hattne, A. S. Brewster, N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and +physical properties from single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not +verified here, so not given). + +**Indexing ambiguity without a reference** — putting each crystal into the indexing in which its +intensities agree best with all the others, iterated to convergence, the method CrystFEL's +`ambigator` also implements; rugnux measures the agreement as a product of standardised amplitudes +rather than a correlation. W. Brehm and K. Diederichs, "Breaking the indexing ambiguity in serial +crystallography" (2014), Acta Cryst. D70, 101-109 +[doi:10.1107/S1399004713025431](https://doi.org/10.1107/S1399004713025431). The same objective +settles the indexing of each sweep of a joint merge, decided over all pairs at once as dials.cosym +does: R. J. Gildea and G. Winter, "Determination of Patterson group symmetry from sparse +multi-crystal data sets in the presence of an indexing ambiguity" (2018), Acta Cryst. D74, 405-410 +[doi:10.1107/S2059798318002978](https://doi.org/10.1107/S2059798318002978). + +### Data quality and anisotropy + +**Data-quality statistics** — R_meas and R_pim, CC1/2 and CC\*, the per-shell CC(model, data), and +the reporting of I/sigma(I) follow the established conventions. K. Diederichs and P. A. Karplus, +"Improved R-factors for diffraction data analysis in macromolecular crystallography" (1997), +Nat. Struct. Biol. 4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269); +P. A. Karplus and K. Diederichs, "Linking crystallographic model and data quality" (2012), Science +336, 1030-1033 [doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231); M. S. Weiss, +"Global indicators of X-ray data quality" (2001), J. Appl. Cryst. 34, 130-135 +[doi:10.1107/S0021889800018227](https://doi.org/10.1107/S0021889800018227); K. Diederichs and +P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222 +[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121). + +**Isomorphism of sweeps** — CC_AB / (CC\*_A CC\*_B) as the cosine of the angle between two data +sets, on which a joint merge decides whether a sweep joins. K. Diederichs, "Dissecting random and +systematic differences between noisy composite data sets" (2017), Acta Cryst. D73, 286-293 +[doi:10.1107/S2059798317000699](https://doi.org/10.1107/S2059798317000699). + +**Diffraction anisotropy** — the overall fall-off as a single anisotropic displacement tensor, its +symmetry constraints and the determinacy of its deviatoric part only: S. Sheriff and +W. A. Hendrickson, "Description of overall anisotropy in diffraction from macromolecular crystals" +(1987), Acta Cryst. A43, 118-121 +[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X). Fitting it to the +intensity distribution with sigma(I) taken into account: A. N. Popov and G. P. Bourenkov, "Choice of +data-collection parameters based on statistic modelling" (2003), Acta Cryst. D59, 1145-1153 +[doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163). The directional +diffraction limits and the anisotropic deltaB reported as the range of the principal components +follow AIMLESS: P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" +(2013), Acta Cryst. D69, 1204-1214 +[doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061). That every run should +analyse and report its anisotropy is inspired by +[STARANISO](https://staraniso.globalphasing.org/): I. J. Tickle, C. Flensburg, P. Keller, +W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO +([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)), +Cambridge, United Kingdom: Global Phasing Ltd. Rugnux reports the anisotropy; it corrects no +intensity and removes no reflection on a directional criterion. + +**Uncertainty conventions** — the IUCr Commission on Crystallographic Nomenclature's. D. Schwarzenbach, S. C. Abrahams, H. D. Flack et al., "Statistical descriptors in crystallography: Report of the IUCr Subcommittee on Statistical Descriptors" (1989), Acta Cryst. A45, 63-75 [doi:10.1107/S0108767388009596](https://doi.org/10.1107/S0108767388009596); D. Schwarzenbach, @@ -439,425 +446,389 @@ S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descrip crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995), Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340). -**Unit-cell e.s.d.s** — that the e.s.d.s of a least-squares cell fit are too small, seeing only the -scatter of the centroids, is Herbstein's survey; rugnux therefore also fits the cell on blocks of -consecutive frames and writes the larger of the two estimates: F. H. Herbstein, "How precise are -measurements of unit-cell dimensions from single crystals?" (2000), Acta Cryst. B56, 547-557 -[doi:10.1107/S010876810000269X](https://doi.org/10.1107/S010876810000269X). +**Unit-cell e.s.d.s** — that the e.s.d.s of a least-squares cell fit are too small is Herbstein's +survey, which is why the cell is also fitted on blocks of consecutive frames. F. H. Herbstein, "How +precise are measurements of unit-cell dimensions from single crystals?" (2000), Acta Cryst. B56, +547-557 [doi:10.1107/S010876810000269X](https://doi.org/10.1107/S010876810000269X). -**Indexing ambiguity of serial data without a reference** — where a serial run's lattice is more -symmetric than its crystals and no reference or model is given, each crystal is put into the indexing in -which its intensities agree best with those of all the other crystals, crystal by crystal until none -changes. The objective - the summed agreement between every pair of crystals - and the iteration that -maximises it are taken from Brehm and Diederichs, whose method CrystFEL's `ambigator` also implements; -rugnux measures the agreement as a product of standardised amplitudes (square roots of the intensities) -rather than a correlation, which lets the sum over all partners be taken against one running sum instead -of pair by pair. -W. Brehm and K. Diederichs, "Breaking the indexing ambiguity in serial crystallography" (2014), -Acta Cryst. D70, 101-109 [doi:10.1107/S1399004713025431](https://doi.org/10.1107/S1399004713025431). -The same objective settles which twin law puts each sweep of a joint merge of several sweeps on the -reference's indexing - decided over all pairs of sweeps at once, as dials.cosym does for its data sets: -R. J. Gildea and G. Winter, "Determination of Patterson group symmetry from sparse multi-crystal data sets -in the presence of an indexing ambiguity" (2018), Acta Cryst. D74, 405-410 -[doi:10.1107/S2059798318002978](https://doi.org/10.1107/S2059798318002978). - -**Reference sweep of a joint merge** — the sweep the others are put on the axes of is the one whose own -merge reaches the finest resolution (among the sweeps with a measured ISa), as KAMO takes the data set of -the smallest Wilson B as its reference: -K. Yamashita, K. Hirata and M. Yamamoto, "KAMO: towards automated data processing for microcrystals" -(2018), Acta Cryst. D74, 441-449 [doi:10.1107/S2059798318004576](https://doi.org/10.1107/S2059798318004576). - -**Measurability** — the fraction of the acentric Bijvoet pairs with |I(+)-I(-)| at three sigma and both -hands at I/sigma 3, reported beside SigAno and CCanom — is phenix.xtriage's: P. H. Zwart, "Anomalous signal -indicators in protein crystallography" (2005), Acta Cryst. D61, 1437-1448 +**Measurability** — the fraction of acentric Bijvoet pairs significant at three sigma, reported +beside SigAno and CCanom, is phenix.xtriage's. P. H. Zwart, "Anomalous signal indicators in protein +crystallography" (2005), Acta Cryst. D61, 1437-1448 [doi:10.1107/S0907444905023589](https://doi.org/10.1107/S0907444905023589). -**Ice in the merged intensities** — the merged reflections on the hexagonal-ice rings are compared with -the off-ring reflections beside them in resolution, and the rings' excesses are fitted together in -proportion to hexagonal ice's line strengths; the table of those strengths, and the idea of reading ice -off a merge's own intensities at the ring positions, are phenix.xtriage's: P. H. Zwart, R. W. -Grosse-Kunstleve and P. D. Adams, "Xtriage and Fest: automatic assessment of X-ray data and substructure -structure factor estimation" (2005), CCP4 Newsletter on Protein Crystallography 43 (no DOI). rugnux's -comparison is local in resolution, where xtriage's is a z-score of a shell against the whole Wilson curve. +**Ice in the merged intensities** — reading ice off a merge's own intensities at the hexagonal-ice +ring positions, and the table of the rings' line strengths, are phenix.xtriage's; rugnux's comparison +is local in resolution where xtriage's is against the whole Wilson curve. P. H. Zwart, +R. W. Grosse-Kunstleve and P. D. Adams, "Xtriage and Fest: automatic assessment of X-ray data and +substructure structure factor estimation" (2005), CCP4 Newsletter on Protein Crystallography 43 (no +DOI). -**Advice on the next collection** - the rotation a sweep needs and the blind region about the spindle -that a second, tilted sweep recovers are the data-collection geometry set out by Z. Dauter, -"Data-collection strategies" (1999), Acta Cryst. D55, 1703-1717 (DOI not verified here). +**Advice on the next collection** — the rotation a sweep needs and the blind region about the +spindle that a tilted second sweep recovers. Z. Dauter, "Data-collection strategies" (1999), Acta +Cryst. D55, 1703-1717 (DOI not verified here). -### Physical corrections and calibration +### Model placement and molecular replacement -**Sensor absorption at oblique incidence, and the flight path** — the angle-dependent quantum -efficiency of a flat sensor, the radial parallax variance that comes from the same integral, and the -attenuation of a reflection in the air between the sample and its pixel are all the Beer-Lambert law -taken along a ray that crosses t/cos(alpha) of sensor, or D/cos(alpha) of air, and converts at a -random depth. The attenuation coefficients, for silicon, CdTe, dry air and helium alike, are the -NIST tabulation: J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray -Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for -Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004), -NIST Standard Reference Database 126 -[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F). - -**Polarization correction** — the azimuthal polarization factor applied to the azimuthally -integrated profile, to the integrated Bragg intensities and to the ring background the beam-stop -shadow test compares a pixel against is the one derived for a partially polarized -synchrotron source by R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular -crystallography with synchrotron radiation: photographic data collection and polarization -correction" (1982), J. Appl. Cryst. 15, 330-337 -[doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060). - -**Hexagonal-ice ring positions** — the eleven ring $d$ spacings from 3.895 to 1.522 Å that the -ice-ring score, the ice-ring flagging and the ice calibrant are all built on are taken from the -measurements of Moreau and co-workers, not enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in -biomolecular cryocrystallography" (2021), Acta Cryst. D77, 540-554 -[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170). - -That list ends at 1.522 Å by its own scope, so the eight bands below it are calculated here rather -than taken from anyone: ice Ih structure factors on the oxygen sublattice, kept where they reach 3% of -the strongest line, which reproduces the eleven measured positions exactly. The lattice constants are -Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice -constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, -644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). - -### Model-based analysis and maps - -**Bulk-solvent correction and overall scaling** — the model's structure factors are put on the -observed scale with an overall factor, an anisotropic B and a flat bulk-solvent term, the flat-mask -model of A. Fokine and A. Urzhumtsev, "Flat bulk-solvent model: obtaining optimal parameters" -(2002), Acta Cryst. D58, 1387-1392 -[doi:10.1107/S0907444902010284](https://doi.org/10.1107/S0907444902010284), which is also the source -of the starting values and of the range those two parameters are physically meaningful over. The mask itself - -the region a probe cannot reach from the atoms, shrunk back - is J.-S. Jiang and A. T. Brünger's, "Protein -hydration observed by X-ray diffraction: solvation properties of penicillopepsin and neuraminidase crystal -structures" (1994), J. Mol. Biol. 243, 100-115 -[doi:10.1006/jmbi.1994.1633](https://doi.org/10.1006/jmbi.1994.1633), drawn with the radii REFMAC5 uses. The -procedure that fits them — a grid search over that range for the solvent pair, with the overall -scale and the anisotropic B refitted at every grid point — follows P. V. Afonine, +**Bulk-solvent correction and overall scaling** — the flat bulk-solvent model, its starting values +and the physically meaningful range of its two parameters: A. Fokine and A. Urzhumtsev, "Flat +bulk-solvent model: obtaining optimal parameters" (2002), Acta Cryst. D58, 1387-1392 +[doi:10.1107/S0907444902010284](https://doi.org/10.1107/S0907444902010284). The mask (the region a +probe cannot reach, shrunk back), drawn with the radii REFMAC5 uses: J.-S. Jiang and A. T. Brünger, +"Protein hydration observed by X-ray diffraction: solvation properties of penicillopepsin and +neuraminidase crystal structures" (1994), J. Mol. Biol. 243, 100-115 +[doi:10.1006/jmbi.1994.1633](https://doi.org/10.1006/jmbi.1994.1633). The grid search over the +solvent pair with scale and anisotropic B refitted at each point: P. V. Afonine, R. W. Grosse-Kunstleve and P. D. Adams, "A robust bulk-solvent correction and anisotropic scaling procedure" (2005), Acta Cryst. D61, 850-855 [doi:10.1107/S0907444905007894](https://doi.org/10.1107/S0907444905007894). The fit is unweighted, -as in both that procedure and REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu, -R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, "REFMAC5 for the refinement of -macromolecular crystal structures" (2011), Acta Cryst. D67, 355-367 +as there and in REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu, R. A. Steiner, +R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, "REFMAC5 for the refinement of macromolecular +crystal structures" (2011), Acta Cryst. D67, 355-367 [doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314). -**sigma_A map coefficients** — the maps written by `--model` are weighted by a maximum-likelihood -sigma_A estimated per resolution shell, giving 2mFo-DFc and mFo-DFc rather than 2Fo-Fc and Fo-Fc. -What is taken is the formalism itself: the Rice and Woolfson likelihoods of |Fo| given |Fc| and -sigma_A, the figure of merit m and the scale D that follow from it, and the result that the -bias-corrected coefficient is 2mFo-DFc for an acentric reflection and mFo for a centric one. -R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures with -errors" (1986), Acta Cryst. A42, 140-149 +**sigma_A map coefficients** — the Rice and Woolfson likelihoods of |Fo| given |Fc| and sigma_A, the +figure of merit m and the scale D, and the bias-corrected coefficients 2mFo-DFc (acentric) and mFo +(centric). R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures +with errors" (1986), Acta Cryst. A42, 140-149 [doi:10.1107/S0108767386099622](https://doi.org/10.1107/S0108767386099622). **[ANODE](https://doi.org/10.1107/S0021889811041768)** — reading the anomalous difference map at the atoms of a supplied model and reporting the strongest sites by name, instead of searching the map for -blobs. The map itself is the textbook anomalous difference Fourier; what is taken from ANODE is that -reading: A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density calculation" (2011), +blobs. A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density calculation" (2011), J. Appl. Cryst. 44, 1285-1287 [doi:10.1107/S0021889811041768](https://doi.org/10.1107/S0021889811041768). -**Uniform random rotations** — the null a supplied model is scored against re-orients that model at -random about its own centroid, and the rotations are drawn uniformly from SO(3) through a uniform -random unit quaternion. K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk, -Academic Press (1992), 124-132 (no DOI). +**Uniform random rotations** — the null a supplied model, and a molecular-replacement search, are +scored against draws rotations uniformly from SO(3) through a uniform random unit quaternion. +K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk, Academic Press (1992), +124-132 (no DOI). -**Variable projection** — the rigid-body placement of a supplied model re-fits the overall scale at -every step, and its Jacobian folds that re-fit in by projecting the scale's own derivatives out of the -placement's, in Kaufman's simplified form of Golub and Pereyra's derivative of the reduced problem. +**Variable projection** — the rigid-body placement re-fits the overall scale at every step and folds +that re-fit into its Jacobian, in Kaufman's simplified form of Golub and Pereyra's derivative. G. H. Golub and V. Pereyra, "The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate" (1973), SIAM J. Numer. Anal. 10, 413-432 -[doi:10.1137/0710036](https://doi.org/10.1137/0710036). L. Kaufman, "A variable projection method +[doi:10.1137/0710036](https://doi.org/10.1137/0710036); L. Kaufman, "A variable projection method for solving separable nonlinear least squares problems" (1975), BIT 15, 49-57 [doi:10.1007/BF01932995](https://doi.org/10.1007/BF01932995). -**Predicted models** — `--model` (and `--mr`) place an AlphaFold prediction (or any model the rigid -body does not place) by molecular replacement. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction -with AlphaFold" (2021), Nature 596, 583-589 +**Predicted models** — `--model` (and `--mr`) place an AlphaFold prediction by molecular replacement. +J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction with +AlphaFold" (2021), Nature 596, 583-589 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); the models are those of the AlphaFold Protein Structure Database: M. Varadi, S. Anyango, M. Deshpande et al., "AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res. 50, D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061). The model is prepared as -phenix.process_predicted_model prepares one - residues below pLDDT 70 removed, B-factors from the -coordinate error the pLDDT predicts: R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold -models to work with phenix.process_predicted_model and ISOLDE" (2022), Acta Cryst. D78, 1303-1314 -[doi:10.1107/S2059798322010026](https://doi.org/10.1107/S2059798322010026). - -**Molecular replacement** — the search scores each orientation of the model against the data by -interpolating its molecular transform, computed once by FFT, as Navaza's AMoRe does: J. Navaza, -"AMoRe: an automated package for molecular replacement" (1994), Acta Cryst. A50, 157-163 -[doi:10.1107/S0108767393007597](https://doi.org/10.1107/S0108767393007597). The translation -function is the full-symmetry one, all pairs of symmetry copies on one Fourier synthesis: -R. A. Crowther and D. M. Blow, "A method of positioning a known molecule in an unknown crystal -structure" (1967), Acta Cryst. 23, 544-548 -[doi:10.1107/S0365110X67003172](https://doi.org/10.1107/S0365110X67003172). A placement is called -a solution, as Phaser calls one, where its translation-function Z-score reaches 8: A. J. McCoy, -R. W. Grosse-Kunstleve, P. D. Adams, M. D. Winn, L. C. Storoni and R. J. Read, "Phaser -crystallographic software" (2007), J. Appl. Cryst. 40, 658-674 -[doi:10.1107/S0021889807021206](https://doi.org/10.1107/S0021889807021206). Whether one copy fits -the cell is read off the Matthews coefficient: B. W. Matthews, "Solvent content of protein -crystals" (1968), J. Mol. Biol. 33, 491-497 -[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836%2868%2990205-2). How many copies -to search for is weighed by a distribution of the Matthews coefficient over protein crystals made from -the mode and mean of Kantardjieff and Rupp's: K. A. Kantardjieff and B. Rupp, "Matthews coefficient -probabilities: Improved estimates for unit cell contents of proteins, DNA, and protein-nucleic acid -complex crystals" (2003), Protein Sci. 12, 1865-1871 -[doi:10.1110/ps.0350503](https://doi.org/10.1110/ps.0350503). A copy refined beside the others has the -others' placements read off by least-squares superposition: W. Kabsch, "A solution for the best rotation -to relate two sets of vectors" (1976), Acta Cryst. A32, 922-923 -[doi:10.1107/S0567739476001873](https://doi.org/10.1107/S0567739476001873). The placements are judged by -their log-likelihood gain, from the Rice and Woolfson distributions of the observed normalised amplitudes -given the model's, with sigma_A from the model's expected coordinate error: R. J. Read, "Pushing the -boundaries of molecular replacement with maximum likelihood" (2001), Acta Cryst. D57, 1373-1382 -[doi:10.1107/S0907444901012471](https://doi.org/10.1107/S0907444901012471); and the gain is read as a -Z-score over the same orientation at other positions, as the likelihood-enhanced translation function's -is: A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni and R. J. Read, "Likelihood-enhanced fast -translation functions" (2005), Acta Cryst. D61, 458-464 -[doi:10.1107/S0907444905001617](https://doi.org/10.1107/S0907444905001617). A model whose expected -log-likelihood gain - the sum of sigma_A^4 / 2 over the reflections - is too small for a placement to stand -out is not searched for: A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason, -B. Lohkamp and R. J. Read, "Ab initio solution of macromolecular crystal structures without direct -methods" (2017), Proc. Natl. Acad. Sci. USA 114, 3637-3641 -[doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114). Translational NCS is looked -for in the native Patterson before the search, an off-origin peak read as Read, Adams and McCoy read -it (see -[Translational pseudo-symmetry](#space-group-twinning-and-pseudo-symmetry) above). The logarithm of the Bessel -function I0 in it is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun, -"Handbook of Mathematical Functions" (1964), National Bureau of Standards, 9.8.1-9.8.2. A predicted -model is cut into rigid domains by its predicted aligned error, as Oeffner et al. (above) do before -molecular replacement; the error itself is AlphaFold's: J. Jumper, R. Evans, A. Pritzel et al., "Highly -accurate protein structure prediction with AlphaFold" (2021), Nature 596, 583-589 +phenix.process_predicted_model prepares one (residues below pLDDT 70 removed, B factors from the +predicted coordinate error) and cut into rigid domains by its predicted aligned error: +R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold models to work with +phenix.process_predicted_model and ISOLDE" (2022), Acta Cryst. D78, 1303-1314 +[doi:10.1107/S2059798322010026](https://doi.org/10.1107/S2059798322010026); the error itself is +AlphaFold's: J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction +with AlphaFold" (2021), Nature 596, 583-589 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2). -**Structure refinement** (of a placed model) — the placed model is refined against the maximum-likelihood -amplitude target, the Rice distribution of an observed amplitude given the model's (its centric -counterpart for a centric reflection): N. S. Pannu and R. J. Read, "Improved structure refinement -through maximum likelihood" (1996), Acta Cryst. A52, 659-668 +**Molecular replacement search** — scoring each orientation of the model by interpolating its +molecular transform, computed once by FFT, as AMoRe does: J. Navaza, "AMoRe: an automated package for +molecular replacement" (1994), Acta Cryst. A50, 157-163 +[doi:10.1107/S0108767393007597](https://doi.org/10.1107/S0108767393007597). The full-symmetry +translation function, all pairs of symmetry copies on one Fourier synthesis: R. A. Crowther and +D. M. Blow, "A method of positioning a known molecule in an unknown crystal structure" (1967), Acta +Cryst. 23, 544-548 [doi:10.1107/S0365110X67003172](https://doi.org/10.1107/S0365110X67003172). +A placement is a solution, as Phaser calls one, where its translation-function Z-score reaches 8: +A. J. McCoy, R. W. Grosse-Kunstleve, P. D. Adams, M. D. Winn, L. C. Storoni and R. J. Read, "Phaser +crystallographic software" (2007), J. Appl. Cryst. 40, 658-674 +[doi:10.1107/S0021889807021206](https://doi.org/10.1107/S0021889807021206). + +**Cell contents** — whether one copy fits the cell is read off the Matthews coefficient: +B. W. Matthews, "Solvent content of protein crystals" (1968), J. Mol. Biol. 33, 491-497 +[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836%2868%2990205-2); how many +copies to search for is weighed by its distribution over protein crystals: K. A. Kantardjieff and +B. Rupp, "Matthews coefficient probabilities: Improved estimates for unit cell contents of proteins, +DNA, and protein-nucleic acid complex crystals" (2003), Protein Sci. 12, 1865-1871 +[doi:10.1110/ps.0350503](https://doi.org/10.1110/ps.0350503). The placements of several copies are +compared by least-squares superposition: W. Kabsch, "A solution for the best rotation to relate two +sets of vectors" (1976), Acta Cryst. A32, 922-923 +[doi:10.1107/S0567739476001873](https://doi.org/10.1107/S0567739476001873). + +**Likelihood scoring of placements** — the log-likelihood gain from the Rice and Woolfson +distributions of the normalised amplitudes given the model's, with sigma_A falling off with resolution +for the model's expected coordinate error as Luzzati's D: V. Luzzati, "Traitement statistique des +erreurs dans la détermination des structures cristallines" (1952), Acta Cryst. 5, 802-810 +[doi:10.1107/S0365110X52002161](https://doi.org/10.1107/S0365110X52002161); R. J. Read, "Pushing the boundaries of molecular replacement with maximum +likelihood" (2001), Acta Cryst. D57, 1373-1382 +[doi:10.1107/S0907444901012471](https://doi.org/10.1107/S0907444901012471); the gain read as a +Z-score over the same orientation at other positions, as the likelihood-enhanced translation +function's is: A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni and R. J. Read, +"Likelihood-enhanced fast translation functions" (2005), Acta Cryst. D61, 458-464 +[doi:10.1107/S0907444905001617](https://doi.org/10.1107/S0907444905001617); and a model whose +expected log-likelihood gain is too small for a placement to stand out is not searched for: +A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason, B. Lohkamp and R. J. Read, +"Ab initio solution of macromolecular crystal structures without direct methods" (2017), Proc. Natl. +Acad. Sci. USA 114, 3637-3641 [doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114). +Translational NCS is looked for in the native Patterson before the search, read as under +[Translational pseudo-symmetry](#space-group-twinning-and-pseudo-symmetry). The logarithm of the +Bessel function I0 is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun, +"Handbook of Mathematical Functions" (1964), National Bureau of Standards, 9.8.1-9.8.2. + +### Structure refinement + +**Likelihood target** — the Rice distribution of an observed amplitude given the model's, and its +centric counterpart. N. S. Pannu and R. J. Read, "Improved structure refinement through maximum +likelihood" (1996), Acta Cryst. A52, 659-668 [doi:10.1107/S0108767396004370](https://doi.org/10.1107/S0108767396004370); G. N. Murshudov, -A. A. Vagin and E. J. Dodson, "Refinement of macromolecular structures by the maximum-likelihood method" -(1997), Acta Cryst. D53, 240-255 [doi:10.1107/S0907444996012255](https://doi.org/10.1107/S0907444996012255). -Its two parameters are fitted by maximum likelihood in resolution bins - the variance on the free -reflections, D on the working reflections with the measurement error added to that variance, which is how -REFMAC5 divides them (Murshudov et al. 2011, above); the refined model's maps are weighted by the same D -and figure of merit. Estimating the likelihood's parameters on free reflections follows V. Y. Lunin and T. P. Skovoroda, "R-free likelihood-based estimates of errors for phases -calculated from atomic models" (1995), Acta Cryst. A51, 880-887 -[doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin, P. V. Afonine and -A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors" (2002), Acta Cryst. A58, -270-282 [doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). The gradient with -respect to every atom is taken by the FFT method - the derivatives of the target with respect to the -structure factors transformed into a map and summed against each atom's density derivatives: -R. C. Agarwal, "A new least-squares refinement technique based on the fast Fourier transform algorithm" -(1978), Acta Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618). -The positions and B factors are minimised by L-BFGS (rugnux's own, its memory kept from one macro-cycle to -the next, the B factors' steps scaled by their estimated curvature): D. C. Liu and J. Nocedal, "On the limited +A. A. Vagin and E. J. Dodson, "Refinement of macromolecular structures by the maximum-likelihood +method" (1997), Acta Cryst. D53, 240-255 +[doi:10.1107/S0907444996012255](https://doi.org/10.1107/S0907444996012255). Its parameters are fitted +in resolution bins, the variance on the free reflections and D on the working ones, as REFMAC5 +divides them (Murshudov et al. 2011, above); estimating them on free reflections follows +V. Y. Lunin and T. P. Skovoroda, "R-free likelihood-based estimates of errors for phases calculated +from atomic models" (1995), Acta Cryst. A51, 880-887 +[doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin, +P. V. Afonine and A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors" +(2002), Acta Cryst. A58, 270-282 +[doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). + +**Gradients by FFT** — the derivatives of the target with respect to the structure factors +transformed into a map and summed against each atom's density derivatives. R. C. Agarwal, "A new +least-squares refinement technique based on the fast Fourier transform algorithm" (1978), Acta +Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618). + +**Minimiser** — L-BFGS (rugnux's own implementation). D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization" (1989), Math. Program. 45, 503-528 -[doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116), whose two-loop recursion is J. Nocedal's, "Updating -quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782 -[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). The geometry restraints are those of the CCP4 -Monomer Library, a subset of which rugnux carries (see THIRD_PARTY_NOTICES.md): A. A. Vagin, -R. A. Steiner, A. A. Lebedev, L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5 -dictionary: organization of prior chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60, -2184-2195 [doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510); applied to the model -through GEMMI's topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, "GEMMI: A -library for structural biology" (2022), J. Open Source Softw. 7, 4200 +[doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116); its two-loop recursion: J. Nocedal, +"Updating quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782 +[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). The +initial inverse Hessian is rescaled at each iteration by the latest pair, the self-scaling of +S. S. Oren and D. G. Luenberger, "Self-scaling variable metric (SSVM) algorithms. Part I" (1974), +Management Sci. 20, 845-862 [doi:10.1287/mnsc.20.5.845](https://doi.org/10.1287/mnsc.20.5.845), and +D. F. Shanno and K.-H. Phua, "Matrix conditioning and nonlinear optimization" (1978), Math. Program. +14, 149-160 [doi:10.1007/BF01588962](https://doi.org/10.1007/BF01588962). + +**Geometry restraints** — those of the CCP4 Monomer Library, a subset of which rugnux carries (see +[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). A. A. Vagin, R. A. Steiner, A. A. Lebedev, +L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5 dictionary: organization of prior +chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60, 2184-2195 +[doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510). They are applied +through GEMMI's topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, "GEMMI: +A library for structural biology" (2022), J. Open Source Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). The torsion derivatives follow -A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion angles -in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141 +A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion +angles in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141 [doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T](https://doi.org/10.1002/(SICI)1096-987X(19960715)17:9%3C1132::AID-JCC5%3E3.0.CO;2-T). -The B factors of bonded atoms, and of atoms in contact, are restrained to each other in the -Kullback-Leibler form REFMAC5 uses, whose tolerance grows with B (Murshudov et al. 2011, above; the divergence itself -is S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22, 79-86 -[doi:10.1214/aoms/1177729694](https://doi.org/10.1214/aoms/1177729694)), and the -overall isotropic B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012, -below). The weights of the X-ray term against the geometry and against the B restraints start from the -ratios of the gradients, as X-PLOR's crystallographic weight was first set: A. T. Brünger, M. Karplus and -G. A. Petsko, "Crystallographic refinement by simulated annealing: application to crambin" (1989), Acta Cryst. -A45, 50-61 [doi:10.1107/S0108767388009195](https://doi.org/10.1107/S0108767388009195); and they are chosen -among short refinements by R-free with the geometry held within bounds, as phenix.refine chooses them: P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd, -N. W. Moriarty, M. Mustyakimov, T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams, -"Towards automated crystallographic structure refinement with phenix.refine" (2012), Acta Cryst. D68, -352-367 [doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself: + +**Riding hydrogens** — placed on their parent atoms by GEMMI's riding-hydrogen code, which follows +D. Liebschner, P. V. Afonine, A. G. Urzhumtsev and P. D. Adams, "Implementation of the riding +hydrogen model in CCTBX to support the next generation of X-ray and neutron joint refinement in +Phenix" (2020), Methods Enzymol. 634, 177-199 +[doi:10.1016/bs.mie.2020.01.007](https://doi.org/10.1016/bs.mie.2020.01.007). + +**B-factor restraints** — bonded atoms, and atoms in contact, restrained to each other in the +Kullback-Leibler form REFMAC5 uses (Murshudov et al. 2011, above); the divergence itself is +S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22, +79-86 [doi:10.1214/aoms/1177729694](https://doi.org/10.1214/aoms/1177729694). The overall isotropic +B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012, below). + +**Weights and R-free** — the weights of the X-ray term against the geometry and B restraints start +from the ratios of the gradients, as X-PLOR's crystallographic weight was first set: A. T. Brünger, +M. Karplus and G. A. Petsko, "Crystallographic refinement by simulated annealing: application to +crambin" (1989), Acta Cryst. A45, 50-61 +[doi:10.1107/S0108767388009195](https://doi.org/10.1107/S0108767388009195); and are chosen among +short refinements by R-free with the geometry held within bounds, as phenix.refine chooses them: +P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd, N. W. Moriarty, M. Mustyakimov, +T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams, "Towards automated crystallographic +structure refinement with phenix.refine" (2012), Acta Cryst. D68, 352-367 +[doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself: A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures" (1992), Nature 355, 472-475 [doi:10.1038/355472a0](https://doi.org/10.1038/355472a0). -The model the refinement scores is scaled to the data with the overall scale and anisotropic B fitted by -robust least squares - iteratively reweighted, with the pseudo-Huber weight of each reflection's residual - -so that a few reflections the model cannot explain do not set the scale of the rest: P. J. Huber, "Robust -estimation of a location parameter" (1964), Ann. Math. Statist. 35, 73-101 -[doi:10.1214/aoms/1177703732](https://doi.org/10.1214/aoms/1177703732), in the smooth (pseudo-Huber) form of -P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, "Deterministic edge-preserving regularization in -computed imaging" (1997), IEEE Trans. Image Process. 6, 298-311 + +**Robust scaling of the model** — the overall scale and anisotropic B are fitted by iteratively +reweighted least squares with the pseudo-Huber weight. P. J. Huber, "Robust estimation of a location +parameter" (1964), Ann. Math. Statist. 35, 73-101 +[doi:10.1214/aoms/1177703732](https://doi.org/10.1214/aoms/1177703732), in the smooth form of +P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, "Deterministic edge-preserving +regularization in computed imaging" (1997), IEEE Trans. Image Process. 6, 298-311 [doi:10.1109/83.551699](https://doi.org/10.1109/83.551699). -The bonds of metal ions to their donor atoms that the model's LINK records name are restrained at the -distances of the Monomer Library's own statistics of metal sites (its `metals.json`, embedded as -`monomers/metals.cif`; per metal, donor element and coordination number), in line with the surveys of -M. M. Harding, "Geometry of metal-ligand interactions in proteins" (2001), Acta Cryst. D57, 401-411 + +**Metal-site restraints** — the bonds of metal ions to the donor atoms the model's LINK records name +are restrained at the distances of the Monomer Library's own metal-site statistics (`metals.json`, +embedded as `monomers/metals.cif`), in line with the surveys of M. M. Harding, "Geometry of +metal-ligand interactions in proteins" (2001), Acta Cryst. D57, 401-411 [doi:10.1107/S0907444900019168](https://doi.org/10.1107/S0907444900019168) and "Small revisions to predicted distances around metal sites in proteins" (2006), Acta Cryst. D62, 678-682 -[doi:10.1107/S0907444906014594](https://doi.org/10.1107/S0907444906014594), and, for Mg²⁺, Na⁺ and K⁺ at -nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to nucleobase -nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004 [doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175). -The anisotropic displacements: a TLS group's U is that of a rigid body's translation, libration and their -correlation, U = T + A L A^T + A S + S^T A^T, of V. Schomaker and K. N. Trueblood, "On the rigid-body motion of -molecules in crystals" (1968), Acta Cryst. B24, 63-76 -[doi:10.1107/S0567740868001718](https://doi.org/10.1107/S0567740868001718), refined against the likelihood with a -residual B per atom, and a model's B column read as the total or as the residual to its TLS groups, as in -M. D. Winn, M. N. Isupov and G. N. Murshudov, "Use of TLS parameters to model anisotropic displacements in +[doi:10.1107/S0907444906014594](https://doi.org/10.1107/S0907444906014594), and, for Mg²⁺, Na⁺ and +K⁺ at nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to +nucleobase nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004 +[doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175). + +**Anisotropic displacements** — the TLS model of a rigid body's translation, libration and their +correlation: V. Schomaker and K. N. Trueblood, "On the rigid-body motion of molecules in crystals" +(1968), Acta Cryst. B24, 63-76 +[doi:10.1107/S0567740868001718](https://doi.org/10.1107/S0567740868001718); refined against the +likelihood with a residual B per atom, the B column read as total or residual, as in M. D. Winn, +M. N. Isupov and G. N. Murshudov, "Use of TLS parameters to model anisotropic displacements in macromolecular refinement" (2001), Acta Cryst. D57, 122-133 -[doi:10.1107/S0907444900014736](https://doi.org/10.1107/S0907444900014736). The chains are cut into TLS groups as -TLSMD cuts them - contiguous segments, by dynamic programming over the fit of an isotropic TLS model to the -refined B factors: J. Painter and E. A. Merritt, "Optimal description of a protein structure in terms of multiple -groups undergoing TLS motion" (2006), Acta Cryst. D62, 439-450 -[doi:10.1107/S0907444906005270](https://doi.org/10.1107/S0907444906005270). Atoms refined with a U of their own are -restrained by the rigid-bond postulate of F. L. Hirshfeld, "Can X-ray data distinguish bonding effects from -vibrational smearing?" (1976), Acta Cryst. A32, 239-244 -[doi:10.1107/S0567739476000533](https://doi.org/10.1107/S0567739476000533), in the form of SHELXL's RIGU - the -components of U along the bond and across it in the bond's plane: A. Thorn, B. Dittrich and G. M. Sheldrick, -"Enhanced rigid-bond restraints" (2012), Acta Cryst. A68, 448-451 -[doi:10.1107/S0108767312014535](https://doi.org/10.1107/S0108767312014535) - with SHELXL's similarity (SIMU) and -approximate-isotropy (ISOR) restraints as G. M. Sheldrick, "Crystal structure refinement with SHELXL" (2015), Acta -Cryst. C71, 3-8 [doi:10.1107/S2053229614024218](https://doi.org/10.1107/S2053229614024218) documents them. Which -displacement model a refinement keeps - isotropic, TLS, or a U per atom - is decided on the free set, as a test of -what the data support rather than a rule by resolution: E. A. Merritt, "To B or not to B: a question of -resolution?" (2012), Acta Cryst. D68, 468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320). +[doi:10.1107/S0907444900014736](https://doi.org/10.1107/S0907444900014736). The chains are cut into +TLS groups as TLSMD cuts them: J. Painter and E. A. Merritt, "Optimal description of a protein +structure in terms of multiple groups undergoing TLS motion" (2006), Acta Cryst. D62, 439-450 +[doi:10.1107/S0907444906005270](https://doi.org/10.1107/S0907444906005270). Atoms with a U of their +own are restrained by the rigid-bond postulate of F. L. Hirshfeld, "Can X-ray data distinguish +bonding effects from vibrational smearing?" (1976), Acta Cryst. A32, 239-244 +[doi:10.1107/S0567739476000533](https://doi.org/10.1107/S0567739476000533), in the form of SHELXL's +RIGU: A. Thorn, B. Dittrich and G. M. Sheldrick, "Enhanced rigid-bond restraints" (2012), Acta +Cryst. A68, 448-451 [doi:10.1107/S0108767312014535](https://doi.org/10.1107/S0108767312014535), +with SHELXL's SIMU and ISOR restraints as G. M. Sheldrick, "Crystal structure refinement with +SHELXL" (2015), Acta Cryst. C71, 3-8 +[doi:10.1107/S2053229614024218](https://doi.org/10.1107/S2053229614024218) documents them. Which +displacement model a refinement keeps is decided on the free set rather than by a rule of +resolution: E. A. Merritt, "To B or not to B: a question of resolution?" (2012), Acta Cryst. D68, +468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320). -**NCS restraints** — copies of a molecule in the model are found by sequence alignment with REFMAC5's criterion -(more than 15 residues aligned, 80% identity - here of the longer chain - and 2.5 Å rms after superposition; -Murshudov et al. 2011, above) and restrained locally, on pairs of -corresponding interatomic distances under the Geman-McClure robust loss, as REFMAC5's local NCS restraints are, with -the distance range of R. A. Nicholls, F. Long and G. N. Murshudov, "Low-resolution refinement tools in REFMAC5" -(2012), Acta Cryst. D68, 404-417 [doi:10.1107/S090744491105606X](https://doi.org/10.1107/S090744491105606X), and -the bonds and angles left out as they leave them out. The case for local restraints over a superposition, and for -deciding them by R-free, is G. J. Kleywegt's, "Use of non-crystallographic symmetry in protein structure refinement" -(1996), Acta Cryst. D52, 842-857 [doi:10.1107/S0907444995016477](https://doi.org/10.1107/S0907444995016477); I. Usón, -E. Pohl, T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz and G. M. Sheldrick, "1.7 Å structure of the stabilized -REIv mutant T39K. Application of local NCS restraints" (1999), Acta Cryst. D55, 1158-1167 -[doi:10.1107/S0907444999003972](https://doi.org/10.1107/S0907444999003972); and J. J. Headd, N. Echols, -P. V. Afonine, N. W. Moriarty, R. J. Gildea and P. D. Adams, "Flexible torsion-angle noncrystallographic symmetry -restraints for improved macromolecular structure refinement" (2014), Acta Cryst. D70, 1346-1356 -[doi:10.1107/S1399004714003277](https://doi.org/10.1107/S1399004714003277), whose comparison of global, torsion and -no NCS is why the copies' B factors are refined independently here. +**NCS restraints** — copies found by sequence alignment with REFMAC5's criterion (Murshudov et al. +2011, above) and restrained locally on pairs of corresponding interatomic distances under the +Geman-McClure loss, as REFMAC5's local NCS restraints are, with the distance range of +R. A. Nicholls, F. Long and G. N. Murshudov, "Low-resolution refinement tools in REFMAC5" (2012), +Acta Cryst. D68, 404-417 [doi:10.1107/S090744491105606X](https://doi.org/10.1107/S090744491105606X). +The superposition behind the rms criterion is GEMMI's QCP: D. L. Theobald, "Rapid calculation of +RMSDs using a quaternion-based characteristic polynomial" (2005), Acta Cryst. A61, 478-480 +[doi:10.1107/S0108767305015266](https://doi.org/10.1107/S0108767305015266); P. Liu, D. K. Agrafiotis +and D. L. Theobald, "Fast determination of the optimal rotational matrix for macromolecular +superpositions" (2010), J. Comput. Chem. 31, 1561-1563 +[doi:10.1002/jcc.21439](https://doi.org/10.1002/jcc.21439). +The case for local restraints over a superposition, decided by R-free: G. J. Kleywegt, "Use of +non-crystallographic symmetry in protein structure refinement" (1996), Acta Cryst. D52, 842-857 +[doi:10.1107/S0907444995016477](https://doi.org/10.1107/S0907444995016477); I. Usón, E. Pohl, +T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz and G. M. Sheldrick, "1.7 Å structure of the +stabilized REIv mutant T39K. Application of local NCS restraints" (1999), Acta Cryst. D55, +1158-1167 [doi:10.1107/S0907444999003972](https://doi.org/10.1107/S0907444999003972); and +J. J. Headd, N. Echols, P. V. Afonine, N. W. Moriarty, R. J. Gildea and P. D. Adams, "Flexible +torsion-angle noncrystallographic symmetry restraints for improved macromolecular structure +refinement" (2014), Acta Cryst. D70, 1346-1356 +[doi:10.1107/S1399004714003277](https://doi.org/10.1107/S1399004714003277), whose comparison is why +the copies' B factors are refined independently. -**Unmodelled density** — the blobs the refinement lists are the connected regions of the difference map above -a cutoff, found by flood fill as Coot's "unmodelled blobs" finds them, in GEMMI's implementation -(`find_blobs_by_flood_fill`): P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, "Features and development -of Coot" (2010), Acta Cryst. D66, 486-501 [doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493). +**Special positions** — an atom on a special position is kept on it, its shifts confined to the +subspace its site symmetry leaves unchanged. R. W. Grosse-Kunstleve and P. D. Adams, "Algorithms for +deriving crystallographic space-group information. II. Treatment of special positions" (2002), Acta +Cryst. A58, 60-65 [doi:10.1107/S0108767301016658](https://doi.org/10.1107/S0108767301016658). -**Special positions** — an atom on a crystallographic special position is placed on it and its shifts are -confined to the subspace its site symmetry leaves unchanged, as R. W. Grosse-Kunstleve and P. D. Adams treat -special positions, "Algorithms for deriving crystallographic space-group information. II. Treatment of special -positions" (2002), Acta Cryst. A58, 60-65 [doi:10.1107/S0108767301016658](https://doi.org/10.1107/S0108767301016658). +### Solvent, ions and unmodelled density -**Water build** — the waters the refinement adds and removes follow the documented ordered-solvent defaults of -phenix.refine: difference-map peaks from 3 σ, hydrogen-bond partners to 3.2 Å and picking only to 2.8 Å, with -the waters added, filtered and refined between macro-cycles: -Afonine et al. (2012, above); -the distances to the model that Coot's Find Waters keeps (2.4-3.2 Å), from the Coot manual and Emsley et al. -(2010, above); and the removal of a water in negative difference density at -3 σ, the cut Echols et al. (2014, -below) use to exclude a site. +**Unmodelled density** — the connected regions of the difference map above a cutoff, found by flood +fill as Coot's "unmodelled blobs" are, in GEMMI's implementation. P. Emsley, B. Lohkamp, W. G. Scott +and K. Cowtan, "Features and development of Coot" (2010), Acta Cryst. D66, 486-501 +[doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493). -**Solvent audit** — the waters and ions of the refined model are scored by their surroundings with the -stereochemical criteria of the surveys of ions at nucleic acids by P. Auffinger's group: the coordination bands -1.9-2.3 / 2.3-2.6 / 2.6-3.2 Å, the Mg²⁺ exclusion zone and the clash shells of N. Naleem, A. Henning-Knechtel, -S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and -annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120 -[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); the Mg-O and Mg-N distances and the -2.3-3.4 Å exclusion zone of F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar and P. Auffinger, -"Principles of ion binding to RNA inferred from the analysis of a 1.55 Å resolution bacterial ribosome -structure – Part I: Mg2+" (2025), Nucleic Acids Res. 53, gkae1148 -[doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); the Na⁺, K⁺ and water distances and -coordination numbers of F. Leonarski, L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor -Mg2+ inner-sphere binders but excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192 -[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118), and the distance bands of their -2017 survey (above). The ligand sphere of the bond-valence sums and the reading of halides by the donors -around them follow N. Echols, N. Morshed, P. V. Afonine, A. J. McCoy, M. D. Miller, R. J. Read, -J. S. Richardson, T. C. Terwilliger and P. D. Adams, "Automated identification of elemental ions in -macromolecular crystal structures" (2014), Acta Cryst. D70, 1104-1114 -[doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308), and the halide's positive and -negative neighbours N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental -ion identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158 -[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241); the chloride's shell of donors -at up to 3.4 Å is O. Carugo's, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC -Struct. Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8). The -bond-valence sums are I. D. Brown and D. Altermatt's, "Bond-valence parameters obtained from a systematic -analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247 -[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of N. E. Brese and -M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197 -[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by H. Zheng, -K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, "Data mining of -iron(II) and iron(III) bond-valence parameters, and their relevance for macromolecular crystallography" -(2017), Acta Cryst. D73, 316-325 [doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584), -weighted by occupancy as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method -able to identify metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37 -[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); their normalised vector sum and -its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski, I. G. Shabalin, K. B. Handing -and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation tool" (2017), Acta Cryst. D73, 223-233 -[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061). The score that combines them is -our own. The divalent ion's place on a purine N7 - in the base's plane, 3.5-3.9 Å from its O6 or N6 - is the test of -Leonarski et al. (2017, above). A light ion is told from a water by a density above the waters', as Echols et al. -(2014, above) flag one; a halide's few or no donors are allowed as Z. Dauter and M. Dauter describe halides binding -also at hydrophobic surfaces, "Entering a new phase: using solvent halide ions in protein structure determination" -(2001), Structure 9, R21-R26 [doi:10.1016/S0969-2126(01)00565-2](https://doi.org/10.1016/S0969-2126(01)00565-2). -The anomalous scattering expected of each species is f″ at the data's wavelength by D. T. Cromer and D. A. -Liberman, "Anomalous dispersion calculations near to and on the long-wavelength side of an absorption edge" -(1981), Acta Cryst. A37, 267-268 [doi:10.1107/S0567739481000600](https://doi.org/10.1107/S0567739481000600), -with the corrections of L. Kissel and R. H. Pratt, "Corrections to tabulated anomalous-scattering factors" -(1990), Acta Cryst. A46, 170-175 [doi:10.1107/S0108767389010718](https://doi.org/10.1107/S0108767389010718), as -GEMMI computes them (its port of the Larch routines). +**Water build** — the documented ordered-solvent defaults of phenix.refine (difference-map peaks +from 3 σ, picking to 2.8 Å, waters added, filtered and refined between macro-cycles; Afonine et al. +2012, above); the distance band to the model, widened a little from the 2.4-3.2 Å Coot's Find Waters +keeps (Emsley et al. 2010, above); and the removal of a water in negative difference density at +-3 σ (Echols et al. 2014, below). + +**Ion stereochemistry** — the coordination bands, the Mg²⁺ exclusion zone and the clash shells of +the surveys of ions at nucleic acids by P. Auffinger's group. N. Naleem, A. Henning-Knechtel, +S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, +diagnosing, and annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120 +[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); F. Leonarski, A. Henning-Knechtel, +S. Kirmizialtin, E. Ennifar and P. Auffinger, "Principles of ion binding to RNA inferred from the +analysis of a 1.55 Å resolution bacterial ribosome structure – Part I: Mg2+" (2025), Nucleic Acids +Res. 53, gkae1148 [doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); F. Leonarski, +L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor Mg2+ inner-sphere binders but +excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192 +[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118); and their 2017 survey (above). +The score that combines the criteria is rugnux's own. + +**Ion identification** — the ligand sphere of the bond-valence sums, the reading of halides by their +donors, and telling a light ion from a water by its density: N. Echols, N. Morshed, P. V. Afonine, +A. J. McCoy, M. D. Miller, R. J. Read, J. S. Richardson, T. C. Terwilliger and P. D. Adams, +"Automated identification of elemental ions in macromolecular crystal structures" (2014), Acta Cryst. +D70, 1104-1114 [doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308); +N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental ion +identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158 +[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241). The chloride's shell of +donors: O. Carugo, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC Struct. +Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8); halides at +hydrophobic surfaces: Z. Dauter and M. Dauter, "Entering a new phase: using solvent halide ions in +protein structure determination" (2001), Structure 9, R21-R26 +[doi:10.1016/S0969-2126(01)00565-2](https://doi.org/10.1016/S0969-2126(01)00565-2). + +**Bond-valence sums** — I. D. Brown and D. Altermatt, "Bond-valence parameters obtained from a +systematic analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247 +[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of +N. E. Brese and M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197 +[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by +H. Zheng, K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, +"Data mining of iron(II) and iron(III) bond-valence parameters, and their relevance for +macromolecular crystallography" (2017), Acta Cryst. D73, 316-325 +[doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584); weighted by occupancy +as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method able to identify +metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37 +[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); the normalised vector +sum and its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski, +I. G. Shabalin, K. B. Handing and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation +tool" (2017), Acta Cryst. D73, 223-233 +[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061). + +**Anomalous scattering factors** — f″ at the data's wavelength, as GEMMI computes it (its port of +the Larch routines). D. T. Cromer and D. A. Liberman, "Anomalous dispersion calculations near to +and on the long-wavelength side of an absorption edge" (1981), Acta Cryst. A37, 267-268 +[doi:10.1107/S0567739481000600](https://doi.org/10.1107/S0567739481000600), with the corrections of +L. Kissel and R. H. Pratt, "Corrections to tabulated anomalous-scattering factors" (1990), Acta +Cryst. A46, 170-175 [doi:10.1107/S0108767389010718](https://doi.org/10.1107/S0108767389010718). ## Software and computing methods -Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading -the result, follows Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4 -data", HDF5 User Group meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders). -The CUDA kernels in Jungfraujoch are its own, but the approach is his. +**GPU decompression** — decoding bitshuffle+LZ4 images on the GPU rather than on the host follows +Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4 data", HDF5 User Group +meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders). The CUDA kernels +are Jungfraujoch's own; the approach is his. -Removing the small islands of solvent from the bulk-solvent mask on the GPU labels the connected -components with a parallel union-find, following D. P. Playne and K. Hawick, "A New Algorithm for -Parallel Connected-Component Labelling on GPUs" (2018), IEEE Trans. Parallel Distrib. Syst. 29, -1217-1230 [doi:10.1109/TPDS.2018.2799216](https://doi.org/10.1109/TPDS.2018.2799216). +**Parallel connected components** — removing the islands of the bulk-solvent mask on the GPU with a +parallel union-find. D. P. Playne and K. Hawick, "A New Algorithm for Parallel Connected-Component +Labelling on GPUs" (2018), IEEE Trans. Parallel Distrib. Syst. 29, 1217-1230 +[doi:10.1109/TPDS.2018.2799216](https://doi.org/10.1109/TPDS.2018.2799216). -The FFT indexer's GPU peak search divides by a fixed window width with a reciprocal multiplication -corrected once by the exact remainder, which returns the correctly rounded quotient, following -P. W. Markstein, "Computation of elementary functions on the IBM RISC System/6000 processor" -(1990), IBM J. Res. Dev. 34, 111-119. +**Correctly rounded integer division** — the FFT indexer's GPU peak search divides by a reciprocal +multiplication corrected once by the exact remainder. P. W. Markstein, "Computation of elementary +functions on the IBM RISC System/6000 processor" (1990), IBM J. Res. Dev. 34, 111-119. -The per-frame scaling loops of the rotation merge are accelerated by Anderson acceleration in the -type-II form analysed by Walker and Ni: H. F. Walker and P. Ni, "Anderson acceleration for fixed-point -iterations" (2011), SIAM J. Numer. Anal. 49, 1715-1735 -[doi:10.1137/10078356X](https://doi.org/10.1137/10078356X), after D. G. Anderson, "Iterative -procedures for nonlinear integral equations" (1965), J. ACM 12, 547-560 +**Anderson acceleration** — of the per-frame scaling loops of the rotation merge, in the type-II +form. H. F. Walker and P. Ni, "Anderson acceleration for fixed-point iterations" (2011), SIAM J. +Numer. Anal. 49, 1715-1735 [doi:10.1137/10078356X](https://doi.org/10.1137/10078356X), after +D. G. Anderson, "Iterative procedures for nonlinear integral equations" (1965), J. ACM 12, 547-560 [doi:10.1145/321296.321305](https://doi.org/10.1145/321296.321305). -The interval the report gives for ISa comes from a bootstrap over the unique reflections: -B. Efron, "Bootstrap methods: another look at the jackknife" (1979), Ann. Statist. 7, 1-26 +**Bootstrap** — the interval reported for ISa. B. Efron, "Bootstrap methods: another look at the +jackknife" (1979), Ann. Statist. 7, 1-26 [doi:10.1214/aos/1176344552](https://doi.org/10.1214/aos/1176344552). Its draws are Poisson weights -hashed from the replicate and the reflection with the SplitMix64 finaliser of G. L. Steele Jr., -D. Lea and C. H. Flood, "Fast splittable pseudorandom number generators" (2014), Proc. OOPSLA 2014, -453-472 [doi:10.1145/2660193.2660195](https://doi.org/10.1145/2660193.2660195). +hashed with the SplitMix64 finaliser of G. L. Steele Jr., D. Lea and C. H. Flood, "Fast splittable +pseudorandom number generators" (2014), Proc. OOPSLA 2014, 453-472 +[doi:10.1145/2660193.2660195](https://doi.org/10.1145/2660193.2660195). -The single-precision exp the Bragg integration's profile fit shares between the CPU and an Apple GPU -(`BraggExp.h`) is built from textbook error-free floating-point arithmetic: the argument reduction of -W. J. Cody and W. Waite, "Software Manual for the Elementary Functions" (1980), Prentice-Hall; the -exact product (with Veltkamp's split) and Fast2Sum of T. J. Dekker, "A floating-point technique for -extending the available precision" (1971), Numer. Math. 18, 224-242; and the 2Sum of D. E. Knuth, -"The Art of Computer Programming, Vol. 2: Seminumerical Algorithms", section 4.2.2. +**Error-free floating-point arithmetic** — the single-precision exp shared between the CPU and an +Apple GPU (`BraggExp.h`): the argument reduction of W. J. Cody and W. Waite, "Software Manual for the +Elementary Functions" (1980), Prentice-Hall; the exact product (with Veltkamp's split) and Fast2Sum +of T. J. Dekker, "A floating-point technique for extending the available precision" (1971), Numer. +Math. 18, 224-242; and the 2Sum of D. E. Knuth, "The Art of Computer Programming, Vol. 2: +Seminumerical Algorithms", section 4.2.2. -This software uses the Viridis, Magma and Inferno colormaps from Matplotlib under its -BSD-compatible license. J. D. Hunter, "Matplotlib: A 2D graphics environment" (2007), Comput. Sci. -Eng. 9, 90-95 [doi:10.1109/MCSE.2007.55](https://doi.org/10.1109/MCSE.2007.55). +**Colormaps** — Viridis, Magma and Inferno from Matplotlib, under its BSD-compatible license. +J. D. Hunter, "Matplotlib: A 2D graphics environment" (2007), Comput. Sci. Eng. 9, 90-95 +[doi:10.1109/MCSE.2007.55](https://doi.org/10.1109/MCSE.2007.55). -The Mol* structure viewer of the web frontend follows the setup Guillaume Gotthard (PSI) tuned for -[AareDB](https://gitea.psi.ch/mx/aaredb), the sample-management and data-pipeline system of the PSI MX -beamlines: its Coot-style mouse controls and trackball, the model representations and colours, and the -display of the density maps (carved wireframe meshes at absolute levels from the whole cell's -statistics, contour-level defaults and colours). +**[Mol\*](https://molstar.org/)** — the web frontend's structure viewer. D. Sehnal, S. Bittrich, +M. Deshpande, R. Svobodová, K. Berka, V. Bazgier, S. Velankar, S. K. Burley, J. Koča and A. S. Rose, +"Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures" +(2021), Nucleic Acids Res. 49, W431-W437 [doi:10.1093/nar/gkab314](https://doi.org/10.1093/nar/gkab314). +Its setup follows the one Guillaume Gotthard (PSI) tuned for [AareDB](https://gitea.psi.ch/mx/aaredb), +the sample-management and data-pipeline system of the PSI MX beamlines: the Coot-style mouse +controls, the model representations and colours, and the display of the density maps. The virtual +trackball is a port of Coot's, itself a port of SGI's `trackball.c` by Gavin Bell. ## File formats read from a published specification -**CBF / imgCIF** - the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression scheme -and reads the imgCIF `_axis` table (the laboratory directions of the image's fast and slow pixel -directions, of the goniometer axes and of a 2theta arm) from the specification alone; no CBFlib or -other CBF code is used, so there is no licence obligation, only this credit. +**CBF / imgCIF** — the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression and the +imgCIF `_axis` table from the specification alone; no CBFlib or other CBF code is used. H. J. Bernstein and A. P. Hammersley, "Specification of the Crystallographic Binary File (CBF/imgCIF)" (2006), International Tables for Crystallography Vol. G, 37-43 [doi:10.1107/97809553602060000729](https://doi.org/10.1107/97809553602060000729); @@ -865,12 +836,16 @@ A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, "Image dictionary (imgCIF International Tables for Crystallography Vol. G, 444-458 [doi:10.1107/97809553602060000746](https://doi.org/10.1107/97809553602060000746). -**d\*TREK SMV** - the SMV reader reads the d\*TREK header vocabulary written by Rigaku's -CrystalClear (Saturn and R-AXIS detectors: detector and spatial-distortion vectors, detector -circles, 2theta arm, encoded overflows) from the headers themselves, interpreting the detector -vectors as dxtbx does for these detectors; no d\*TREK or dxtbx code is used. -J. W. Pflugrath, "The finer things in X-ray diffraction data collection" (1999), Acta Cryst. D55, -1718-1725 [doi:10.1107/S090744499900935X](https://doi.org/10.1107/S090744499900935X); +**PILATUS count-rate clipping** — that the count-rate correction clips an over-range pixel to the +cutoff rather than flagging it, which is how the miniCBF reader tells a saturated pixel. P. Trueb, +C. Dejoie, M. Kobas, P. Pattison, D. J. Peake, V. Radicci, B. A. Sobott, D. A. Walko and +C. Broennimann, "Bunch mode specific rate corrections for PILATUS3 detectors" (2015), J. Synchrotron +Rad. 22, 701-707 [doi:10.1107/S1600577515003288](https://doi.org/10.1107/S1600577515003288). + +**d\*TREK SMV** — the SMV reader reads the d\*TREK header vocabulary of Rigaku's CrystalClear from +the headers themselves, interpreting the detector vectors as dxtbx does; no d\*TREK or dxtbx code is +used. J. W. Pflugrath, "The finer things in X-ray diffraction data collection" (1999), Acta Cryst. +D55, 1718-1725 [doi:10.1107/S090744499900935X](https://doi.org/10.1107/S090744499900935X); [dxtbx](https://github.com/cctbx/dxtbx): J. M. Parkhurst, A. S. Brewster, L. Fuentes-Montero, D. G. Waterman et al., "dxtbx: the diffraction experiment toolbox" (2014), J. Appl. Cryst. 47, 1459-1465 [doi:10.1107/S1600576714011996](https://doi.org/10.1107/S1600576714011996). @@ -879,18 +854,17 @@ D. G. Waterman et al., "dxtbx: the diffraction experiment toolbox" (2014), J. Ap In addition to in-house datasets collected at SLS 2.0, Jungfraujoch is tested against public diffraction data collected on other people's beamlines, on detectors and in file formats we do not -produce ourselves - most of it at other facilities, a few sets at the Swiss Light Source but not by -this system. That data was collected and published by other people. Every dataset used, the DOI to -cite for it, and the deposition it belongs to are listed in -[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md); we thank the depositors, and the repositories that make -the data findable and citable. +produce ourselves. Every dataset used, the DOI to cite for it, and the deposition it belongs to are +listed in [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) and +[EXTERNAL_TEST_DATA_STILLS](EXTERNAL_TEST_DATA_STILLS.md); we thank the depositors, and the +repositories that make the data findable and citable. **[IRRMC](https://proteindiffraction.org/)**, the Integrated Resource for Reproducibility in -Macromolecular Crystallography (Minor lab, University of Virginia), is the source of most of them. -IRRMC releases its data under CC0 and asks that the DOI of the dataset be -cited. M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng, -D. R. Cooper, M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of -macromolecular diffraction experiments" (2016), Acta Cryst. D72, 1181-1193 +Macromolecular Crystallography (Minor lab, University of Virginia), is the source of the largest +share of them; it releases its data under CC0 and asks that the DOI of the dataset be cited. +M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng, D. R. Cooper, +M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of macromolecular +diffraction experiments" (2016), Acta Cryst. D72, 1181-1193 [doi:10.1107/S2059798316014716](https://doi.org/10.1107/S2059798316014716); M. Grabowski, M. Cymborowski, P. J. Porebski, T. Osinski, I. G. Shabalin, D. R. Cooper and W. Minor, "The Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first @@ -898,61 +872,69 @@ four years" (2019), Struct. Dyn. 6, 064301 [doi:10.1063/1.5128672](https://doi.org/10.1063/1.5128672). **[SBGrid Data Bank](https://data.sbgrid.org/)**, the structural biology community's data -publication service (SBGrid Consortium, Harvard Medical School). P. A. Meyer, -S. Socias, J. Key, E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the -structural biology data grid supports live analysis" (2016), Nat. Commun. 7, 10882 +publication service (SBGrid Consortium, Harvard Medical School). P. A. Meyer, S. Socias, J. Key, +E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the structural biology data grid +supports live analysis" (2016), Nat. Commun. 7, 10882 [doi:10.1038/ncomms10882](https://doi.org/10.1038/ncomms10882). **[Zenodo](https://zenodo.org/)**, CERN's open repository, hosts datasets deposited there directly by the groups that collected them. European Organization for Nuclear Research and OpenAIRE, -"Zenodo" (2013), CERN [doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). Some of those -deposits are described in IUCrData Raw Data Letters; the letters are cited on the -[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page, beside the datasets they describe. +"Zenodo" (2013), CERN [doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). The IUCrData +Raw Data Letters describing some of those deposits are cited on the +[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page. **[CXIDB](https://cxidb.org/)**, the Coherent X-ray Imaging Data Bank, is the source of the serial -(XFEL stills) datasets of the still battery, listed with their DOIs on -[EXTERNAL_TEST_DATA_STILLS](EXTERNAL_TEST_DATA_STILLS.md). F. R. N. C. Maia, "The Coherent X-ray -Imaging Data Bank" (2012), Nat. Methods 9, 854-855 -[doi:10.1038/nmeth.2110](https://doi.org/10.1038/nmeth.2110). +(XFEL stills) datasets. F. R. N. C. Maia, "The Coherent X-ray Imaging Data Bank" (2012), Nat. +Methods 9, 854-855 [doi:10.1038/nmeth.2110](https://doi.org/10.1038/nmeth.2110). **[MXRDR](https://mxrdr.icm.edu.pl/)**, the Macromolecular Xtallography Raw Data Repository (ICM, University of Warsaw), releases its data under CC0 and asks that the DOI of the dataset be cited. -**[XRDa](https://xrda.pdbj.org/)**, the Xtal Raw Data Archive (Protein Data Bank Japan), which -publishes raw diffraction images - X-ray, electron and neutron - and mints a DOI for each; it asks -that the DOI of the dataset be cited. It has no canonical citation paper. +**[XRDa](https://xrda.pdbj.org/)**, the Xtal Raw Data Archive (Protein Data Bank Japan), publishes +raw diffraction images and mints a DOI for each, which it asks to be cited. It has no canonical +citation paper. -**The [ESRF data portal](https://data.esrf.fr/)**, through which the European Synchrotron -publishes raw data under its data policy (CC BY 4.0, with the dataset DOI to be cited). -R. Dimper, A. Götz, A. De Maria, V. A. Solé, M. Chaillet and B. Lebayle, "ESRF Data Policy, -Storage, and Services" (2019), Synchrotron Rad. News 32, 7-12 +**The [ESRF data portal](https://data.esrf.fr/)**, through which the European Synchrotron publishes +raw data under its data policy (CC BY 4.0, with the dataset DOI to be cited). R. Dimper, A. Götz, +A. De Maria, V. A. Solé, M. Chaillet and B. Lebayle, "ESRF Data Policy, Storage, and Services" +(2019), Synchrotron Rad. News 32, 7-12 [doi:10.1080/08940886.2019.1608119](https://doi.org/10.1080/08940886.2019.1608119). **The [Keele University research data repository](https://researchdata.keele.ac.uk/)** and -**[UQ eSpace](https://espace.library.uq.edu.au/)** (The University of Queensland), which host the -raw images of datasets deposited there by the groups that collected them; the dataset DOIs are -cited on the [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page. +**[UQ eSpace](https://espace.library.uq.edu.au/)** (The University of Queensland) host the raw +images of datasets deposited there by the groups that collected them; the dataset DOIs are cited on +the [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page. -The beamline, resolution, space group and unit cell quoted for each dataset are the values -deposited with the corresponding PDB entry, read from the RCSB PDB data API. H. M. Berman, -J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and P. E. Bourne, -"The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242 +**The [Protein Data Bank](https://www.rcsb.org/)** supplies, through the RCSB PDB data API, the +beamline, resolution, space group and unit cell quoted for each dataset, and the deposited +coordinates and structure factors of the public entries the refinement battery starts from. +H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and +P. E. Bourne, "The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242 [doi:10.1093/nar/28.1.235](https://doi.org/10.1093/nar/28.1.235). +**The [Crystallography Open Database](https://www.crystallography.net/cod/)** supplies the reference +cells of the small-molecule sets. S. Gražulis, D. Chateigner, R. T. Downs, A. F. T. Yokochi, +M. Quirós, L. Lutterotti, E. Manakova, J. Butkus, P. Moeck and A. Le Bail, "Crystallography Open +Database – an open-access collection of crystal structures" (2009), J. Appl. Cryst. 42, 726-729 +[doi:10.1107/S0021889809016690](https://doi.org/10.1107/S0021889809016690); S. Gražulis, +A. Daškevič, A. Merkys et al., "Crystallography Open Database (COD): an open-access collection of +crystal structures and platform for world-wide collaboration" (2012), Nucleic Acids Res. 40, +D420-D427 [doi:10.1093/nar/gkr900](https://doi.org/10.1093/nar/gkr900). + **The [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/)** (EMBL-EBI and Google DeepMind) supplies the start models of the battery's timing benchmark: the predictions of the deposited entities of 9ZLO (AF-P17090-F1), 9S02 (AF-P32322-F1) and 9GJX (AF-Q65MG6-F1) and, for the in-house benchmark, of horse myoglobin (AF-P68082-F1) and bovine cytochrome c (AF-P62894-F1), model version 6, with their predicted aligned error, kept unmodified in `tools/battery/models/` under -CC BY 4.0. J. Jumper, R. Evans, -A. Pritzel et al., "Highly accurate protein structure prediction with AlphaFold" (2021), Nature 596, -583-589 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); M. Varadi, +CC BY 4.0. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction +with AlphaFold" (2021), Nature 596, 583-589 +[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); M. Varadi, S. Anyango, M. Deshpande et al., "AlphaFold Protein Structure Database: massively expanding the -structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res. 50, -D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061); M. Varadi, D. Bertoni, -P. Magana et al., "AlphaFold Protein Structure Database in 2024: providing structure coverage for over -214 million protein sequences" (2024), Nucleic Acids Res. 52, D368-D375 +structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res. +50, D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061); M. Varadi, +D. Bertoni, P. Magana et al., "AlphaFold Protein Structure Database in 2024: providing structure +coverage for over 214 million protein sequences" (2024), Nucleic Acids Res. 52, D368-D375 [doi:10.1093/nar/gkad1011](https://doi.org/10.1093/nar/gkad1011). The large case of both benchmarks is the deposited model of PDB entry 4NWV, moved away from its place. diff --git a/docs/CPU_DATA_ANALYSIS.md b/docs/CPU_DATA_ANALYSIS.md index 8df2e38fd..bfc7217c1 100644 --- a/docs/CPU_DATA_ANALYSIS.md +++ b/docs/CPU_DATA_ANALYSIS.md @@ -112,5 +112,15 @@ The methods draw on, and in places reimplement, solutions from: - R. J. Read, "Pushing the boundaries of molecular replacement with maximum likelihood", *Acta Cryst.* **D57** (2001), 1373-1382; A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni & R. J. Read, "Likelihood-enhanced fast translation functions", *Acta Cryst.* **D61** (2005), 458-464; A. J. McCoy et al., "Phaser crystallographic software", *J. Appl. Cryst.* **40** (2007), 658-674; A. J. McCoy et al., *Proc. Natl. Acad. Sci. USA* **114** (2017), 3637-3641 (the likelihood target, the solution criterion and the expected gain of molecular replacement, §14.10). - R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold models to work with phenix.process_predicted_model and ISOLDE", *Acta Cryst.* **D78** (2022), 1303-1314, and J. Jumper et al., "Highly accurate protein structure prediction with AlphaFold", *Nature* **596** (2021), 583-589 (preparing a predicted model for molecular replacement, §14.10). +- E. A. Cornish & R. A. Fisher, "Moments and cumulants in the specification of distributions", *Rev. Int. Stat. Inst.* **5** (1938), 307-320 (the Poisson quantile of the adaptive spot-finding threshold beyond the summation limit). +- R. W. Grosse-Kunstleve, "Algorithms for deriving crystallographic space-group information", *Acta Cryst.* **A55** (1999), 383-395 (from Le Page's two-folds to a conventional cell, centring and change of basis, §13.1). +- T. C. Huang, H. Toraya, T. N. Blanton & Y. Wu, "X-ray powder diffraction analysis of silver behenate, a possible low-angle diffraction standard", *J. Appl. Cryst.* **26** (1993), 180-184 (the silver behenate calibrant cell, §7.6). +- P. Trueb, C. Dejoie, M. Kobas et al., "Bunch mode specific rate corrections for PILATUS3 detectors", *J. Synchrotron Rad.* **22** (2015), 701-707 (the count-rate correction clips an over-range pixel to the cutoff, §1.7). +- D. Liebschner, P. V. Afonine, M. L. Baker et al., "Macromolecular structure determination using X-rays, neutrons and electrons: recent developments in Phenix", *Acta Cryst.* **D75** (2019), 861-877 (phenix.xtriage, the model of the twinning test, §13.2). +- V. Luzzati, "Traitement statistique des erreurs dans la détermination des structures cristallines", *Acta Cryst.* **5** (1952), 802-810 (the fall-off of $\sigma_A$ with resolution for a model's coordinate error, Luzzati's $D$, in the likelihood of a placement, §14.10). +- D. Liebschner, P. V. Afonine, A. G. Urzhumtsev & P. D. Adams, "Implementation of the riding hydrogen model in CCTBX to support the next generation of X-ray and neutron joint refinement in Phenix", *Methods Enzymol.* **634** (2020), 177-199 (the riding hydrogens, placed by GEMMI's code, §15.2). +- D. L. Theobald, "Rapid calculation of RMSDs using a quaternion-based characteristic polynomial", *Acta Cryst.* **A61** (2005), 478-480, and P. Liu, D. K. Agrafiotis & D. L. Theobald, *J. Comput. Chem.* **31** (2010), 1561-1563 (the QCP superposition behind the NCS rms criterion, via GEMMI, §15.5). +- S. S. Oren & D. G. Luenberger, "Self-scaling variable metric (SSVM) algorithms. Part I", *Management Sci.* **20** (1974), 845-862, and D. F. Shanno & K.-H. Phua, "Matrix conditioning and nonlinear optimization", *Math. Program.* **14** (1978), 149-160 (the rescaling of the L-BFGS initial inverse Hessian by the latest pair, §15.6). + (list is not exhaustive; the full citations, with DOIs, are in [ACKNOWLEDGEMENT.md](ACKNOWLEDGEMENT.md)) diff --git a/image_analysis/SensorAbsorption.h b/image_analysis/SensorAbsorption.h index 7ff580b7b..4fe3eeb6e 100644 --- a/image_analysis/SensorAbsorption.h +++ b/image_analysis/SensorAbsorption.h @@ -32,7 +32,8 @@ // untilted detector. On a tilted one the correction acquires an azimuthal dependence at fixed // resolution, which is the only part of it that is not degenerate with an overall Wilson B. // -// Attenuation coefficients: NIST X-Ray Mass Attenuation Coefficients (Hubbell & Seltzer), total +// Attenuation coefficients: NIST X-Ray Mass Attenuation Coefficients, Hubbell & Seltzer (1995) NIST +// Standard Reference Database 126, total // mu/rho with coherent scattering. Photoelectric absorption dominates over the whole range used // here, and because the correction is a RATIO of two absorbed fractions the photoelectric branching // ratio cancels exactly; only the attenuation length enters. Coherent scattering, which attenuates diff --git a/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp b/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp index e10a3a6aa..bf844c160 100644 --- a/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp +++ b/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp @@ -246,7 +246,8 @@ std::vector BraggIntegrationEngineCPU::RunImpl(const Sampler &img, const auto &r = predicted[i]; const BraggPanel &pn = panels[i]; // This reflection's stencil: the r1 signal disk, and the r2..r3 ring elongated radially by - // the analytic smear. + // the analytic smear. The three-ring region follows CrystFEL, White et al. (2012) J. Appl. + // Cryst. 45, 335-341 const BraggStencil st = MakeBraggStencil(pn, stencil); const Box box = box_of(r, st); const int x0 = box.x0, x1 = box.x1, y0 = box.y0, y1 = box.y1; diff --git a/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu b/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu index 41f8d3cef..6b9eb3178 100644 --- a/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu +++ b/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu @@ -241,7 +241,8 @@ __global__ void boxsum(const float *px_x, const float *px_y, const BraggPanel *p const BraggStencilDist d = BraggStencilDistances(st, (float) x - cx, (float) y - cy); // The window is the bounding box of the outer ellipse, so nearly half of it is neither the // signal disk nor the background ring. Reading the pixel only once it is known to be wanted - // keeps those slots from fetching a cache line for nothing. + // keeps those slots from fetching a cache line for nothing. The r1 disk / r2..r3 ring region + // follows CrystFEL, White et al. (2012) J. Appl. Cryst. 45, 335-341 if (BraggStencilInSignal(st, (float) x - cx, (float) y - cy, d.signal, p.r1_sq)) { // A pixel a nearer neighbour owns carries that neighbour's flux; see the CPU engine. ++l_nd; diff --git a/image_analysis/geom_refinement/PostRefine.h b/image_analysis/geom_refinement/PostRefine.h index 2f7d8c360..6ae1b3e74 100644 --- a/image_analysis/geom_refinement/PostRefine.h +++ b/image_analysis/geom_refinement/PostRefine.h @@ -30,6 +30,8 @@ // * the positional detector<->reciprocal residual at each partial's observed spot -> pins the distance. // Where there are more reflections than the fit's caps, the strongest by I/sigma are the ones kept; // inside the fit every reflection carries the same weight. +// Following MOSFLM's post-refinement, Powell (1999) Acta Cryst. D55, 1690-1695, and Leslie & Powell (2007) +// Evolving Methods for Macromolecular Crystallography, NATO Sci. Ser. II 245, 41-51 // How far one step of the joint fit may move the distance or a cell length, relative (see STEP_BOUND in // PostRefine.cpp, where it is explained). diff --git a/image_analysis/scale_merge/CentreOfSymmetry.cpp b/image_analysis/scale_merge/CentreOfSymmetry.cpp index a7faebf74..be42bfc6f 100644 --- a/image_analysis/scale_merge/CentreOfSymmetry.cpp +++ b/image_analysis/scale_merge/CentreOfSymmetry.cpp @@ -98,7 +98,8 @@ namespace { } } -// Following Howells, Phillips & Rogers (1950) Acta Cryst. 3, 210-214 (N(z)), and Padilla & Yeates +// Following Wilson (1949) Acta Cryst. 2, 318-321 (the acentric and centric intensity distributions), +// Howells, Phillips & Rogers (1950) Acta Cryst. 3, 210-214 (N(z)), and Padilla & Yeates // (2003) Acta Cryst. D59, 1124-1130 (the L-test, read here against its centric value 2/pi). CentreOfSymmetryResult AnalyzeCentreOfSymmetry(const std::vector& merged, const gemmi::SpaceGroup& space_group, size_t nthreads) { diff --git a/image_analysis/scale_merge/StillsPartialityRefine.h b/image_analysis/scale_merge/StillsPartialityRefine.h index 11af1d26f..1a0fa081b 100644 --- a/image_analysis/scale_merge/StillsPartialityRefine.h +++ b/image_analysis/scale_merge/StillsPartialityRefine.h @@ -39,6 +39,7 @@ // `image_scale_corr` in place; the existing MergeOnTheFly then consumes the improved corrections. The // width of the Gaussian partiality is the run's, measured once from the intensity-weighted excitation // errors (MeasureRunWidth). +// Following Uervirojnangkoorn et al. (2015) eLife 4, e05421 (PRIME) and Sauter et al. (2014) Acta Cryst. D70, 3299-3309 class StillsPartialityRefine { public: struct Settings { diff --git a/image_analysis/structure_refinement/MolecularReplacement.h b/image_analysis/structure_refinement/MolecularReplacement.h index f645099af..109f3026a 100644 --- a/image_analysis/structure_refinement/MolecularReplacement.h +++ b/image_analysis/structure_refinement/MolecularReplacement.h @@ -63,6 +63,8 @@ SearchModel PrepareSearchModel(const RugnuxModel &model); // linkage on the symmetrised PAE until every two clusters are a mean PAE_DOMAIN_A apart; clusters of at least // DOMAIN_MIN_RESIDUES, largest first, each a list of positions in the prediction. Empty where the matrix does // not match the model. +// The models, their pLDDT and PAE following Jumper et al. (2021) Nature 596, 583-589, and Varadi et al. (2022) +// Nucleic Acids Res. 50, D439-D444 (the AlphaFold DB) std::vector> PaeDomains(const std::vector> &pae, const SearchModel &model); // The search model of one domain: the residues of `model` at those positions, recentred. SearchModel DomainModel(const SearchModel &model, const std::vector &residues); diff --git a/image_analysis/structure_refinement/Refinement.cpp b/image_analysis/structure_refinement/Refinement.cpp index bae38e302..07f0e089c 100644 --- a/image_analysis/structure_refinement/Refinement.cpp +++ b/image_analysis/structure_refinement/Refinement.cpp @@ -2653,6 +2653,9 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector TlsBodiesToGroups(const gemmi::Model &model, const // U23 of the difference in a frame whose z is n, as SHELXL's RIGU restrains them; // - similarity, for the same pairs: |U_i - U_j|^2 (Frobenius) / sigma^2, sigma 0.04 A^2 (SHELXL's SIMU); // - near isotropy, for an atom in no pair (a water, an ion): |U - tr(U)/3 I|^2 / sigma^2, sigma 0.1 A^2 (ISOR). -// Following Hirshfeld (1976) Acta Cryst. A32, 239-244, and Thorn, Dittrich & Sheldrick (2012) Acta Cryst. A68, 448-451 +// Following Hirshfeld (1976) Acta Cryst. A32, 239-244, Thorn, Dittrich & Sheldrick (2012) Acta Cryst. A68, 448-451 +// (RIGU), and Sheldrick (2015) Acta Cryst. C71, 3-8 (SIMU and ISOR) struct URestraints { static constexpr double SIGMA_RIGID = 0.004, SIGMA_SIMILAR = 0.04, SIGMA_ISOTROPIC = 0.1; std::vector> pairs; diff --git a/image_analysis/structure_refinement/RefinementXray.h b/image_analysis/structure_refinement/RefinementXray.h index 789e888c9..3b8640d6e 100644 --- a/image_analysis/structure_refinement/RefinementXray.h +++ b/image_analysis/structure_refinement/RefinementXray.h @@ -133,7 +133,8 @@ private: // beta fitted by maximum likelihood to the free reflections as the whole spread of Fo about D Fc, D then fitted to // the working reflections with the variance epsilon beta + 2 sigma^2 (acentric; + sigma^2 centric), which is also // the variance of a reflection in the target. `fmodel` is k_total (F + fmask), without D. -// Following Lunin & Skovoroda (1995) Acta Cryst. A51, 880-887, and Murshudov et al. (2011) Acta Cryst. D67, 355-367 +// Following Lunin & Skovoroda (1995) Acta Cryst. A51, 880-887, Lunin, Afonine & Urzhumtsev (2002) Acta Cryst. A58, +// 270-282 (the parameters estimated on the free reflections), and Murshudov et al. (2011) Acta Cryst. D67, 355-367 void EstimateLikelihoodParameters(const RefinementReflections &refl, const std::vector> &fmodel, XrayRowModel &rows, size_t nthreads = 1); @@ -151,7 +152,8 @@ std::vector> TwinnedModel(const RefinementReflections &refl, // twinned amplitudes took a fraction of 0.16 for unrelated partners of one model at R-free 0.31, the average of two // predictions being the better prediction. Reflections that are their own mate under every law say nothing and are // left out. With one law the fraction is at most 1/2. `f` is F at every row, `rows` gives k_total and fmask. -// Following Yeates (1997) Methods Enzymol. 276, 344-358 (the twinned intensity), refined as Murshudov et al. (2011) +// Following Yeates (1997) Methods Enzymol. 276, 344-358 (the twinned intensity), refined beside the model as +// Herbst-Irmer & Sheldrick (1998) Acta Cryst. B54, 443-449 (SHELXL's BASF) and Murshudov et al. (2011) // Acta Cryst. D67, 355-367 refine the fractions, by least squares constexpr size_t TWIN_FRACTION_SHELL = 500; std::vector FitTwinFractions(const RefinementReflections &refl, const XrayRowModel &rows,