From 291386e525727b8a2fb768495e4f6cf9f601ebd6 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 13 Nov 2026 11:10:03 +0100 Subject: [PATCH 1/3] docs: ACKNOWLEDGEMENT.md credits each work under its topic, no group singled out The "Automatic processing as an expert system" section that opened the methods part was given over to one company - a paragraph on Global Phasing Ltd, one on autoPROC and one on STARANISO, ahead of every other package - which made that group look far more prominent than what was taken from it. The section is gone and its three pieces now sit where their subject is: autoPROC under a new first subsection, "Experimental geometry and physical corrections", that also takes over the sensor-absorption, polarization and ice-ring entries of the former "Physical corrections and calibration" section at the end (they are image-level physics, so they belong before spot finding in the pipeline order the page follows); STARANISO in the "Diffraction anisotropy" entry it inspired; the Bricogne (1986) credit was already under Indexing and stays there alone. The umbrella paragraph only summarised those three and is dropped; the autoPROC entry keeps every statement it made (header metadata verified against the images, the beam centre found from them, the expert-system aim, the "loss of centring" wording) in the same tone and length as the entries around it. Two further moves for balance: the per-crystal post-refinement of stills (PRIME, Sauter et al.) moves from Spot finding to the serial entries of the scaling section, and the single ninety-line "Structure refinement" paragraph is split into entries (gradient, minimiser, geometry restraints, B restraints, weights, model scaling, metal bonds, anisotropic displacements) with no change of wording. No acknowledged work, citation or statement of what was taken is dropped: all 151 DOIs appear exactly as before, and every sentence outside the three rewritten Global Phasing entries is verbatim. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK --- docs/ACKNOWLEDGEMENT.md | 173 +++++++++++++++++++--------------------- 1 file changed, 84 insertions(+), 89 deletions(-) diff --git a/docs/ACKNOWLEDGEMENT.md b/docs/ACKNOWLEDGEMENT.md index ba9f14c24..dc748edc1 100644 --- a/docs/ACKNOWLEDGEMENT.md +++ b/docs/ACKNOWLEDGEMENT.md @@ -22,39 +22,6 @@ The project is supported by: * ETH Domain via Open Research Data Contribute project (Jan - Dec 2023). * AMD University Program with donation of licenses of Ethernet IP cores and Vivado software. -## Automatic processing as an expert system - -**Global Phasing Ltd** — much of what rugnux tries to do was inspired by the work of Gérard Bricogne -and his colleagues at [Global Phasing](https://www.globalphasing.com/): the Fourier-transform -indexing its de-novo indexers descend from, which Bricogne proposed in 1986 (see -[Indexing](#indexing) below), the view of data processing as an expert system, the distrust of -image-header metadata, and the attention paid to diffraction anisotropy, each described below. - -**[autoPROC](https://www.globalphasing.com/autoproc/)** — rugnux's goal is a fully automatic -pipeline that takes the decisions an experienced crystallographer would take while processing a -dataset, and reports each decision together with the evidence it rests on, so that a bare -invocation gives the best answer available. autoPROC is the inspiration and the target for that -behaviour, and two of rugnux's working principles come from it. The first, and the one that has -guided rugnux most, we owe to Clemens Vonrhein: a processing pipeline should not take the -experimental metadata it is given - the beam centre, the detector geometry, the goniometer and its -rotation axis - on trust, but verify it against the images and, where it is wrong, correct it. We -are grateful to him for arguing this so clearly and for so long. The second follows from it: the -beam centre should be determined from the images themselves; rugnux does this in its own way, -described in §1.4 of the [data-analysis reference](CPU_DATA_ANALYSIS_IMAGE.md). The -reasons rugnux gives for a weak stretch of a sweep also use autoPROC's wording ("loss of centring -during crystal rotation"). C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, -T. Womack and G. Bricogne, "Data processing and analysis with the autoPROC toolbox" (2011), -Acta Cryst. D67, 293-302 -[doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773). - -**[STARANISO](https://staraniso.globalphasing.org/)** — the understanding that diffraction -anisotropy matters, and that every run should analyse and report it, was inspired by STARANISO. -The estimators rugnux uses for it are those credited under -[Scaling, merging and data quality](#scaling-merging-and-data-quality) below. I. J. Tickle, -C. Flensburg, P. Keller, W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO -([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)), -Cambridge, United Kingdom: Global Phasing Ltd. - ## Crystallographic methods adopted from other packages The analysis pipeline builds on methods first published, and in most cases first implemented, by @@ -64,6 +31,54 @@ packages are neither linked nor vendored; the three that are - GEMMI, traccc and fast-feedback-indexer - also carry a licence obligation, recorded in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). +### Experimental geometry and physical corrections + +**[autoPROC](https://www.globalphasing.com/autoproc/)** — the principle that a processing pipeline +does not take the experimental metadata it is given - the beam centre, the detector geometry, the +goniometer and its rotation axis - on trust, but verifies it against the images and, where it is +wrong, corrects it, which Clemens Vonrhein has long argued for; and, following from it, that the beam +centre is determined from the images themselves (rugnux does this in its own way, described in §1.4 +of the [data-analysis reference](CPU_DATA_ANALYSIS_IMAGE.md)). autoPROC is also the model for the +behaviour rugnux aims at as a whole: a pipeline that takes the decisions an experienced +crystallographer would take while processing a dataset and reports each decision together with the +evidence it rests on, so that a bare invocation gives the best answer available. The reasons rugnux +gives for a weak stretch of a sweep use its wording ("loss of centring during crystal rotation"). +C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, +"Data processing and analysis with the autoPROC toolbox" (2011), Acta Cryst. D67, 293-302 +[doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773). + +**Sensor absorption at oblique incidence, and the flight path** — the angle-dependent quantum +efficiency of a flat sensor, the radial parallax variance that comes from the same integral, and the +attenuation of a reflection in the air between the sample and its pixel are all the Beer-Lambert law +taken along a ray that crosses t/cos(alpha) of sensor, or D/cos(alpha) of air, and converts at a +random depth. The attenuation coefficients, for silicon, CdTe, dry air and helium alike, are the +NIST tabulation: J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray +Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for +Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004), +NIST Standard Reference Database 126 +[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F). + +**Polarization correction** — the azimuthal polarization factor applied to the azimuthally +integrated profile, to the integrated Bragg intensities and to the ring background the beam-stop +shadow test compares a pixel against is the one derived for a partially polarized +synchrotron source by R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular +crystallography with synchrotron radiation: photographic data collection and polarization +correction" (1982), J. Appl. Cryst. 15, 330-337 +[doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060). + +**Hexagonal-ice ring positions** — the eleven ring $d$ spacings from 3.895 to 1.522 Å that the +ice-ring score, the ice-ring flagging and the ice calibrant are all built on are taken from the +measurements of Moreau and co-workers, not enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in +biomolecular cryocrystallography" (2021), Acta Cryst. D77, 540-554 +[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170). + +That list ends at 1.522 Å by its own scope, so the eight bands below it are calculated here rather +than taken from anyone: ice Ih structure factors on the oxygen sublattice, kept where they reach 3% of +the strongest line, which reproduces the eleven measured positions exactly. The lattice constants are +Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice +constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, +644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). + ### Spot finding **[CrystFEL](https://www.desy.de/~twhite/crystfel/)** — spot finding, the three-ring integration @@ -78,16 +93,6 @@ high-throughput reduction and analysis of serial femtosecond X-ray diffraction d J. Appl. Cryst. 47, 1118-1131 [doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626). -**Per-crystal post-refinement of serial stills** - refining, for each still, the partiality -parameters the spot positions cannot pin (a small reorientation, the crystal's own rocking width and the -radial scale of its reciprocal lattice, which a shot's photon energy or a crystal's cell moves) against -the merged intensities follows the post-refinement of PRIME and the mosaic-block model of Sauter et al.: -M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, A. T. Brunger -and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging biological systems -from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter, J. Hattne, A. S. Brewster, -N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and physical properties from -single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not verified here, so not given). - Spot extraction groups strong pixels into spots with the sparse connected-component labelling of the ACTS traccc project: P. Gessinger, H. M. Gray, A. Krasznahorkay, C. Leggett, J. Niermann, A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction library for HEP @@ -362,13 +367,18 @@ The estimator fits that tensor to the observed intensity distribution, taking si in the sense of Popov and Bourenkov. The directional diffraction limits - in a cone about each principal direction, and the reporting of the anisotropic deltaB as the range of the principal components - follow AIMLESS. Rugnux reports these; it corrects no intensity and removes no reflection -on a directional criterion. S. Sheriff and W. A. Hendrickson, "Description of overall anisotropy in +on a directional criterion. That diffraction anisotropy matters, and that every run should analyse +and report it, is inspired by STARANISO. S. Sheriff and W. A. Hendrickson, "Description of overall anisotropy in diffraction from macromolecular crystals" (1987), Acta Cryst. A43, 118-121 [doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X); A. N. Popov and G. P. Bourenkov, "Choice of data-collection parameters based on statistic modelling" (2003), Acta Cryst. D59, 1145-1153 [doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163); P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta -Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061). +Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061); +[STARANISO](https://staraniso.globalphasing.org/): I. J. Tickle, C. Flensburg, P. Keller, +W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO +([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)), +Cambridge, United Kingdom: Global Phasing Ltd. **Data-quality statistics** follow the established conventions rather than any one program: R_meas and R_pim, CC1/2 and CC\*, the per-shell CC(model, data) between F^2_calc and F^2_obs, and the @@ -445,6 +455,16 @@ consecutive frames and writes the larger of the two estimates: F. H. Herbstein, measurements of unit-cell dimensions from single crystals?" (2000), Acta Cryst. B56, 547-557 [doi:10.1107/S010876810000269X](https://doi.org/10.1107/S010876810000269X). +**Per-crystal post-refinement of serial stills** - refining, for each still, the partiality +parameters the spot positions cannot pin (a small reorientation, the crystal's own rocking width and the +radial scale of its reciprocal lattice, which a shot's photon energy or a crystal's cell moves) against +the merged intensities follows the post-refinement of PRIME and the mosaic-block model of Sauter et al.: +M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, A. T. Brunger +and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging biological systems +from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter, J. Hattne, A. S. Brewster, +N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and physical properties from +single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not verified here, so not given). + **Indexing ambiguity of serial data without a reference** — where a serial run's lattice is more symmetric than its crystals and no reference or model is given, each crystal is put into the indexing in which its intensities agree best with those of all the other crystals, crystal by crystal until none @@ -484,40 +504,6 @@ comparison is local in resolution, where xtriage's is a z-score of a shell again that a second, tilted sweep recovers are the data-collection geometry set out by Z. Dauter, "Data-collection strategies" (1999), Acta Cryst. D55, 1703-1717 (DOI not verified here). -### Physical corrections and calibration - -**Sensor absorption at oblique incidence, and the flight path** — the angle-dependent quantum -efficiency of a flat sensor, the radial parallax variance that comes from the same integral, and the -attenuation of a reflection in the air between the sample and its pixel are all the Beer-Lambert law -taken along a ray that crosses t/cos(alpha) of sensor, or D/cos(alpha) of air, and converts at a -random depth. The attenuation coefficients, for silicon, CdTe, dry air and helium alike, are the -NIST tabulation: J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray -Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for -Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004), -NIST Standard Reference Database 126 -[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F). - -**Polarization correction** — the azimuthal polarization factor applied to the azimuthally -integrated profile, to the integrated Bragg intensities and to the ring background the beam-stop -shadow test compares a pixel against is the one derived for a partially polarized -synchrotron source by R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular -crystallography with synchrotron radiation: photographic data collection and polarization -correction" (1982), J. Appl. Cryst. 15, 330-337 -[doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060). - -**Hexagonal-ice ring positions** — the eleven ring $d$ spacings from 3.895 to 1.522 Å that the -ice-ring score, the ice-ring flagging and the ice calibrant are all built on are taken from the -measurements of Moreau and co-workers, not enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in -biomolecular cryocrystallography" (2021), Acta Cryst. D77, 540-554 -[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170). - -That list ends at 1.522 Å by its own scope, so the eight bands below it are calculated here rather -than taken from anyone: ice Ih structure factors on the oxygen sublattice, kept where they reach 3% of -the strongest line, which reproduces the eleven measured positions exactly. The lattice constants are -Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice -constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, -644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). - ### Model-based analysis and maps **Bulk-solvent correction and overall scaling** — the model's structure factors are put on the @@ -643,18 +629,21 @@ and figure of merit. Estimating the likelihood's parameters on free reflections calculated from atomic models" (1995), Acta Cryst. A51, 880-887 [doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin, P. V. Afonine and A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors" (2002), Acta Cryst. A58, -270-282 [doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). The gradient with -respect to every atom is taken by the FFT method - the derivatives of the target with respect to the +270-282 [doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). + +**The gradient** with respect to every atom is taken by the FFT method - the derivatives of the target with respect to the structure factors transformed into a map and summed against each atom's density derivatives: R. C. Agarwal, "A new least-squares refinement technique based on the fast Fourier transform algorithm" (1978), Acta Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618). -The positions and B factors are minimised by L-BFGS (rugnux's own, its memory kept from one macro-cycle to + +**The positions and B factors** are minimised by L-BFGS (rugnux's own, its memory kept from one macro-cycle to the next, the B factors' steps scaled by their estimated curvature): D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization" (1989), Math. Program. 45, 503-528 [doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116), whose two-loop recursion is J. Nocedal's, "Updating quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782 -[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). The geometry restraints are those of the CCP4 -Monomer Library, a subset of which rugnux carries (see THIRD_PARTY_NOTICES.md): A. A. Vagin, +[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). + +**The geometry restraints** are those of the CCP4 Monomer Library, a subset of which rugnux carries (see THIRD_PARTY_NOTICES.md): A. A. Vagin, R. A. Steiner, A. A. Lebedev, L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5 dictionary: organization of prior chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60, 2184-2195 [doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510); applied to the model @@ -664,12 +653,15 @@ library for structural biology" (2022), J. Open Source Softw. 7, 4200 A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion angles in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141 [doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T](https://doi.org/10.1002/(SICI)1096-987X(19960715)17:9%3C1132::AID-JCC5%3E3.0.CO;2-T). -The B factors of bonded atoms, and of atoms in contact, are restrained to each other in the + +**The B factors of bonded atoms**, and of atoms in contact, are restrained to each other in the Kullback-Leibler form REFMAC5 uses, whose tolerance grows with B (Murshudov et al. 2011, above; the divergence itself is S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22, 79-86 [doi:10.1214/aoms/1177729694](https://doi.org/10.1214/aoms/1177729694)), and the overall isotropic B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012, -below). The weights of the X-ray term against the geometry and against the B restraints start from the +below). + +**The weights of the X-ray term** against the geometry and against the B restraints start from the ratios of the gradients, as X-PLOR's crystallographic weight was first set: A. T. Brünger, M. Karplus and G. A. Petsko, "Crystallographic refinement by simulated annealing: application to crambin" (1989), Acta Cryst. A45, 50-61 [doi:10.1107/S0108767388009195](https://doi.org/10.1107/S0108767388009195); and they are chosen @@ -679,7 +671,8 @@ N. W. Moriarty, M. Mustyakimov, T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart an 352-367 [doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself: A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures" (1992), Nature 355, 472-475 [doi:10.1038/355472a0](https://doi.org/10.1038/355472a0). -The model the refinement scores is scaled to the data with the overall scale and anisotropic B fitted by + +**The model the refinement scores** is scaled to the data with the overall scale and anisotropic B fitted by robust least squares - iteratively reweighted, with the pseudo-Huber weight of each reflection's residual - so that a few reflections the model cannot explain do not set the scale of the rest: P. J. Huber, "Robust estimation of a location parameter" (1964), Ann. Math. Statist. 35, 73-101 @@ -687,7 +680,8 @@ estimation of a location parameter" (1964), Ann. Math. Statist. 35, 73-101 P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, "Deterministic edge-preserving regularization in computed imaging" (1997), IEEE Trans. Image Process. 6, 298-311 [doi:10.1109/83.551699](https://doi.org/10.1109/83.551699). -The bonds of metal ions to their donor atoms that the model's LINK records name are restrained at the + +**The bonds of metal ions** to their donor atoms that the model's LINK records name are restrained at the distances of the Monomer Library's own statistics of metal sites (its `metals.json`, embedded as `monomers/metals.cif`; per metal, donor element and coordination number), in line with the surveys of M. M. Harding, "Geometry of metal-ligand interactions in proteins" (2001), Acta Cryst. D57, 401-411 @@ -696,7 +690,8 @@ predicted distances around metal sites in proteins" (2006), Acta Cryst. D62, 678 [doi:10.1107/S0907444906014594](https://doi.org/10.1107/S0907444906014594), and, for Mg²⁺, Na⁺ and K⁺ at nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to nucleobase nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004 [doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175). -The anisotropic displacements: a TLS group's U is that of a rigid body's translation, libration and their + +**The anisotropic displacements**: a TLS group's U is that of a rigid body's translation, libration and their correlation, U = T + A L A^T + A S + S^T A^T, of V. Schomaker and K. N. Trueblood, "On the rigid-body motion of molecules in crystals" (1968), Acta Cryst. B24, 63-76 [doi:10.1107/S0567740868001718](https://doi.org/10.1107/S0567740868001718), refined against the likelihood with a From a0b39e1e11c16e54f2b22f08f96773a7ab33ce7a Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 13 Nov 2026 11:36:04 +0100 Subject: [PATCH 2/3] docs: ACKNOWLEDGEMENT.md as short entries by topic, checked against the code Second pass on the acknowledgements for readability. Every entry now says what was taken, from whom, and gives the citation; the detail of how rugnux uses a method, which belongs in CPU_DATA_ANALYSIS.md, is gone. The two long sections are split: "Scaling, merging and data quality" (174 lines) into "Scaling and merging", "Serial data" and "Data quality and anisotropy"; "Model-based analysis and maps" (285 lines) into "Model placement and molecular replacement", "Structure refinement" and "Solvent, ions and unmodelled density", with the former single refinement paragraph and the solvent-audit paragraph broken into one entry per method. The longest section is now 123 lines, the longest entry 22. Descriptions corrected where the code differs: rugnux prints "loss of centring", not autoPROC's full phrase; the twin fraction is re-fitted between cycles, not refined in the same least squares as SHELXL's BASF; the coincidence-lattice test covers reticular (pseudo-)merohedry up to index 10 and also follows Herbst-Irmer & Sheldrick (1998); the water build widens Coot's 2.4-3.2 A band rather than using it as is; the random rotations also serve the molecular-replacement null; IRRMC supplies the largest share of the public sets, not most of them; the PDB also supplies the refinement battery's coordinates and structure factors. No citation lost: all 151 DOIs appear with the same counts as before, and every DOI-less work is kept. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK --- docs/ACKNOWLEDGEMENT.md | 1280 ++++++++++++++++++--------------------- 1 file changed, 598 insertions(+), 682 deletions(-) diff --git a/docs/ACKNOWLEDGEMENT.md b/docs/ACKNOWLEDGEMENT.md index dc748edc1..e6b87380b 100644 --- a/docs/ACKNOWLEDGEMENT.md +++ b/docs/ACKNOWLEDGEMENT.md @@ -25,59 +25,46 @@ The project is supported by: ## Crystallographic methods adopted from other packages The analysis pipeline builds on methods first published, and in most cases first implemented, by -other crystallographic software. The code below is Jungfraujoch's own; the methods are theirs, and -are acknowledged here. Where a package's source was consulted this is said explicitly. Most of these -packages are neither linked nor vendored; the three that are - GEMMI, traccc and -fast-feedback-indexer - also carry a licence obligation, recorded in -[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). +other crystallographic software. The code is Jungfraujoch's own; the methods are theirs, and are +acknowledged here, entry by entry: what was taken, from whom, and the citation. How rugnux uses each +method is described in the [data-analysis reference](CPU_DATA_ANALYSIS.md), not here. Where a +package's source was consulted this is said explicitly. Most of these packages are neither linked nor +vendored; the three that are - GEMMI, traccc and fast-feedback-indexer - also carry a licence +obligation, recorded in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). ### Experimental geometry and physical corrections -**[autoPROC](https://www.globalphasing.com/autoproc/)** — the principle that a processing pipeline -does not take the experimental metadata it is given - the beam centre, the detector geometry, the -goniometer and its rotation axis - on trust, but verifies it against the images and, where it is -wrong, corrects it, which Clemens Vonrhein has long argued for; and, following from it, that the beam -centre is determined from the images themselves (rugnux does this in its own way, described in §1.4 -of the [data-analysis reference](CPU_DATA_ANALYSIS_IMAGE.md)). autoPROC is also the model for the -behaviour rugnux aims at as a whole: a pipeline that takes the decisions an experienced -crystallographer would take while processing a dataset and reports each decision together with the -evidence it rests on, so that a bare invocation gives the best answer available. The reasons rugnux -gives for a weak stretch of a sweep use its wording ("loss of centring during crystal rotation"). -C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, -"Data processing and analysis with the autoPROC toolbox" (2011), Acta Cryst. D67, 293-302 +**[autoPROC](https://www.globalphasing.com/autoproc/)** — the principle, long argued for by Clemens +Vonrhein, that a pipeline does not take the experimental metadata it is given (beam centre, detector +geometry, goniometer axis) on trust but verifies it against the images and corrects it where it is +wrong, and with it that the beam centre is found from the images; the aim of a pipeline that takes the +decisions an expert would take and reports the evidence for each; and the wording "loss of centring" +(autoPROC's "loss of centring during crystal rotation") for a weak stretch of a sweep. C. Vonrhein, +C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, "Data +processing and analysis with the autoPROC toolbox" (2011), Acta Cryst. D67, 293-302 [doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773). -**Sensor absorption at oblique incidence, and the flight path** — the angle-dependent quantum -efficiency of a flat sensor, the radial parallax variance that comes from the same integral, and the -attenuation of a reflection in the air between the sample and its pixel are all the Beer-Lambert law -taken along a ray that crosses t/cos(alpha) of sensor, or D/cos(alpha) of air, and converts at a -random depth. The attenuation coefficients, for silicon, CdTe, dry air and helium alike, are the -NIST tabulation: J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray +**Sensor absorption and air path** — the attenuation coefficients of silicon, CdTe, dry air and +helium behind the oblique-incidence sensor efficiency, the parallax variance and the air-path +attenuation are the NIST tabulation. J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004), NIST Standard Reference Database 126 [doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F). -**Polarization correction** — the azimuthal polarization factor applied to the azimuthally -integrated profile, to the integrated Bragg intensities and to the ring background the beam-stop -shadow test compares a pixel against is the one derived for a partially polarized -synchrotron source by R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular +**Polarization correction** — the azimuthal polarization factor for a partially polarized +synchrotron source. R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular crystallography with synchrotron radiation: photographic data collection and polarization correction" (1982), J. Appl. Cryst. 15, 330-337 [doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060). -**Hexagonal-ice ring positions** — the eleven ring $d$ spacings from 3.895 to 1.522 Å that the -ice-ring score, the ice-ring flagging and the ice calibrant are all built on are taken from the -measurements of Moreau and co-workers, not enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in -biomolecular cryocrystallography" (2021), Acta Cryst. D77, 540-554 -[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170). - -That list ends at 1.522 Å by its own scope, so the eight bands below it are calculated here rather -than taken from anyone: ice Ih structure factors on the oxygen sublattice, kept where they reach 3% of -the strongest line, which reproduces the eleven measured positions exactly. The lattice constants are -Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice -constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, -644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). +**Hexagonal-ice ring positions** — the eleven measured ring spacings from 3.895 to 1.522 Å. +D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in biomolecular cryocrystallography" (2021), Acta +Cryst. D77, 540-554 [doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170). +The bands below 1.522 Å are calculated here from ice Ih structure factors on the lattice constants +of K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice constants and thermal +expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, 644-648 +[doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). ### Spot finding @@ -86,53 +73,50 @@ region, the serial/stills processing model, and the per-frame indexing acceptanc (`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass, A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012), J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312). -The self-calibrating spot finder's per-resolution-ring background statistics, with the Bragg peaks -excluded by iterated clipping, follow Cheetah's peakfinder8: A. Barty, R. A. Kirian, -F. R. N. C. Maia, M. Hantke, C. H. Yoon, T. A. White and H. N. Chapman, "Cheetah: software for -high-throughput reduction and analysis of serial femtosecond X-ray diffraction data" (2014), -J. Appl. Cryst. 47, 1118-1131 + +**Cheetah** — the per-resolution-ring background statistics with the Bragg peaks excluded by +iterated clipping, from peakfinder8. A. Barty, R. A. Kirian, F. R. N. C. Maia, M. Hantke, C. H. Yoon, +T. A. White and H. N. Chapman, "Cheetah: software for high-throughput reduction and analysis of +serial femtosecond X-ray diffraction data" (2014), J. Appl. Cryst. 47, 1118-1131 [doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626). -Spot extraction groups strong pixels into spots with the sparse connected-component labelling of the -ACTS traccc project: P. Gessinger, H. M. Gray, A. Krasznahorkay, C. Leggett, J. Niermann, -A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction library for HEP -experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822); -[traccc](https://github.com/acts-project/traccc). The CPU spot extractor adapts its SparseCCL source, -and the CUDA spot extractor follows the design of its GPU counterpart - a backward-neighbour graph -over a sorted hit list, resolved by a parallel union-find. traccc is MPL-2.0; see -[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). The SparseCCL algorithm itself is A. Hennequin, -B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: Connected Components Labeling and -Analysis for sparse images" (2019), DASIP 2019, 65-70 +**[traccc](https://github.com/acts-project/traccc)** — the sparse connected-component labelling +that groups strong pixels into spots: the CPU extractor adapts its SparseCCL source, the CUDA +extractor follows the design of its GPU counterpart (MPL-2.0; see +[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). P. Gessinger, H. M. Gray, A. Krasznahorkay, +C. Leggett, J. Niermann, A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction +library for HEP experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822). The +SparseCCL algorithm itself: A. Hennequin, B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: +Connected Components Labeling and Analysis for sparse images" (2019), DASIP 2019, 65-70 [doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184). -**XDS and DISTL** — a spot is bounded by its shape (how much of its bounding box it fills) rather than -by its size alone, as XDS (cited under [Integration and rotation geometry](#integration-and-rotation-geometry)) and DISTL bound theirs; the statistic -is rugnux's own, the choice of shape over size is theirs. Z. Zhang, N. K. Sauter, H. van den Bedem, -G. Snell and A. M. Deacon, "Automated diffraction image analysis and spot searching for high-throughput -crystal screening" (2006), J. Appl. Cryst. 39, 112-119 +**XDS and DISTL** — bounding a spot by its shape (how much of its bounding box it fills) rather than +by its size; the statistic is rugnux's own. XDS is cited under +[Integration and rotation geometry](#integration-and-rotation-geometry); Z. Zhang, N. K. Sauter, +H. van den Bedem, G. Snell and A. M. Deacon, "Automated diffraction image analysis and spot searching +for high-throughput crystal screening" (2006), J. Appl. Cryst. 39, 112-119 [doi:10.1107/S0021889805040677](https://doi.org/10.1107/S0021889805040677). ### Indexing -**Fourier-transform indexing** — indexing a diffraction pattern, with no prior knowledge of the -cell, by Fourier-transforming the reciprocal-space positions of its spots was proposed by Gérard -Bricogne as a three-dimensional transform; the de-novo FFT indexers (`-X fft`, `-X fftw`) use the -one-dimensional projection form of that idea described in the next paragraph. G. Bricogne (1986), -in *Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software -(Phase III)*, LURE, Paris, p. 28 (a workshop proceedings without a DOI). +**Fourier-transform indexing** — indexing with no prior knowledge of the cell by Fourier-transforming +the reciprocal-space positions of the spots, proposed as a three-dimensional transform; the de-novo +indexers (`-X fft`, `-X fftw`) use its one-dimensional projection form. G. Bricogne (1986), in +*Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software (Phase III)*, +LURE, Paris, p. 28 (a workshop proceedings without a DOI). -**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/mosflm/)** — the Rossmann FFT autoindexing algorithm and -post-refinement practice, including which parameters are safe to refine per image and which must be -refined over a wedge. The autoindexing algorithm itself — projecting the reciprocal-space points -onto many directions and Fourier-transforming the 1D projection histograms — is I. Steller, -R. Bolotovsky and M. G. Rossmann, "An algorithm for automatic indexing of oscillation images using -Fourier analysis" (1997), J. Appl. Cryst. 30, 1036-1040 -[doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777); MOSFLM is the -implementation whose practice is followed, described in H. R. Powell, "The Rossmann Fourier -autoindexing algorithm in MOSFLM" (1999), Acta Cryst. D55, 1690-1695 -[doi:10.1107/S0907444999009506](https://doi.org/10.1107/S0907444999009506). A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM" -(2007), in *Evolving Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245, -41-51 [doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4); +**Rossmann FFT autoindexing** — projecting the reciprocal-space points onto many directions and +Fourier-transforming the 1D histograms. I. Steller, R. Bolotovsky and M. G. Rossmann, "An algorithm +for automatic indexing of oscillation images using Fourier analysis" (1997), J. Appl. Cryst. 30, +1036-1040 [doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777). + +**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/mosflm/)** — the practice of that algorithm, and +the post-refinement practice of which parameters are refined per image and which over a wedge. +H. R. Powell, "The Rossmann Fourier autoindexing algorithm in MOSFLM" (1999), Acta Cryst. D55, +1690-1695 [doi:10.1107/S0907444999009506](https://doi.org/10.1107/S0907444999009506); +A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM" (2007), in *Evolving +Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245, 41-51 +[doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4); T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new graphical interface for diffraction-image processing with MOSFLM" (2011), Acta Cryst. D67, 271-281 [doi:10.1107/S0907444910048675](https://doi.org/10.1107/S0907444910048675); H. R. Powell, @@ -141,186 +125,154 @@ diffraction data with the graphical user interface iMosflm" (2017), Nat. Protoc. [doi:10.1038/nprot.2017.037](https://doi.org/10.1038/nprot.2017.037). **[fast-feedback-indexer](https://github.com/paulscherrerinstitute/fast-feedback-indexer)** — the -known-cell indexer for serial stills (`-X ffbidx`) is PSI's fast-feedback-indexer library, linked at -build time (BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), which implements -the TORO algorithm: P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton, -M. Janousch, F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for -kilohertz serial crystallography" (2024), J. Appl. Cryst. 57, 931-944 +known-cell indexer for serial stills (`-X ffbidx`), PSI's library linked at build time +(BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), implementing TORO. +P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton, M. Janousch, +F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for kilohertz serial +crystallography" (2024), J. Appl. Cryst. 57, 931-944 [doi:10.1107/S1600576724003182](https://doi.org/10.1107/S1600576724003182). The CPU implementation -used without a GPU (`image_analysis/indexing/FFBIDXIndexerCPU.cpp`) is a port of the library's CUDA -kernels by H.-C. Stadler (PSI): the same samples and scores, with the loops reorganised for a CPU. +used without a GPU is a port of the library's CUDA kernels by H.-C. Stadler (PSI). ### Cell reduction and lattice symmetry **[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and -structure-factor machinery, and MTZ / XDS_ASCII I/O. Vendored in `gemmi_gph/`, so it also carries a +structure-factor machinery, and MTZ / XDS_ASCII I/O; vendored in `gemmi_gph/`, so it also carries a licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). -**Křivý & Gruber's Niggli reduction, and the lattice-character table** — the reduction that puts -every candidate cell in a comparable form is I. Křivý and B. Gruber, "A unified algorithm for -determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298 -[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636), used through GEMMI's -implementation; the table of lattice characters that maps a reduced cell to Bravais lattices and -centrings follows International Tables for Crystallography Vol. A, Table 9.2.5.1. +**Niggli reduction** — used through GEMMI's implementation. I. Křivý and B. Gruber, "A unified +algorithm for determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298 +[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636). The table of lattice +characters that maps a reduced cell to Bravais lattices and centrings follows International Tables +for Crystallography Vol. A, Table 9.2.5.1. -**Grosse-Kunstleve, Sauter & Adams's numerically stable cell reduction** - the magnitude-scaled -tolerance that decides the sign of a structurally-zero scalar product, and with it the Niggli type a -reduced cell is presented in. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically +**Numerically stable cell reduction** — the magnitude-scaled tolerance that decides the sign of a +structurally-zero scalar product. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically stable algorithms for the computation of reduced unit cells" (2004), Acta Cryst. A60, 1-6 [doi:10.1107/S010876730302186X](https://doi.org/10.1107/S010876730302186X). -**Le Page's metric-symmetry search** - the obliquity of each of the 81 candidate two-folds of a -reduced cell, which is what tells a run that its lattice metric hosts more rotational symmetry than -the group its intensities supported, and the derivation of the conventional axes from that -rotation group, which is what the run then offers to the space-group search as a second lattice -candidate. The two-fold search is used through GEMMI's implementation of it. Y. Le Page, "The -derivation of the axes of the conventional unit cell from the dimensions of the Buerger-reduced -cell" (1982), J. Appl. Cryst. 15, 255-259 +**Metric-symmetry search** — the obliquity of the 81 candidate two-folds of a reduced cell, and the +conventional axes derived from the rotation group found; the two-fold search is used through GEMMI. +Y. Le Page, "The derivation of the axes of the conventional unit cell from the dimensions of the +Buerger-reduced cell" (1982), J. Appl. Cryst. 15, 255-259 [doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959). ### Integration and rotation geometry **[XDS](https://xds.mr.mpg.de/)** — rotation geometry and notation, the reciprocal Lorentz and -partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for rejecting -a reflection whose predicted profile is not cleanly its own, the intensity-based test for a -centred lattice, the recognition of shaded detector regions by comparing a pixel's background -against the background at its own resolution (`DEFPIX`), and the scaling correction surfaces +partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for a +reflection whose predicted profile is not cleanly its own, the intensity-based test for a centred +lattice, the recognition of shaded detector regions (`DEFPIX`), and the scaling correction surfaces indexed by image number and detector region. W. Kabsch, "XDS" (2010), Acta Cryst. D66, 125-132 [doi:10.1107/S0907444909047337](https://doi.org/10.1107/S0907444909047337); W. Kabsch, "Integration, scaling, space-group assignment and post-refinement" (2010), Acta Cryst. D66, 133-144 [doi:10.1107/S0907444909047374](https://doi.org/10.1107/S0907444909047374). -**Profile fitting** with reweighted, de-biased variances is the Kabsch/Otwinowski iteration, from the +**Profile fitting** — the Kabsch/Otwinowski iteration with reweighted, de-biased variances, from the second XDS paper above and from Z. Otwinowski and W. Minor, "Processing of X-ray diffraction data collected in oscillation mode" (1997), Methods Enzymol. 276, 307-326 [doi:10.1016/S0076-6879(97)76066-X](https://doi.org/10.1016/S0076-6879%2897%2976066-X). -**The two-dimensional integration architecture** — integrating each image in the detector plane -and only afterwards assembling a reflection's partials into a full across images, as against -three-dimensional profile fitting through the image stack — is the architecture of DENZO/SCALEPACK -and MOSFLM, and it is the one Rugnux's rotation pipeline follows (per-image profile-fitted -integration, then partials combined into fulls; §9 and §10.6 of the [data-analysis reference](CPU_DATA_ANALYSIS.md)). The -Otwinowski & Minor citation above and the MOSFLM citations below carry the credit for the paradigm -as well as for the specifics taken from each. +**Two-dimensional integration** — integrating each image in the detector plane and only afterwards +combining a reflection's partials into a full, as against three-dimensional profile fitting through +the image stack, is the architecture of DENZO/SCALEPACK (Otwinowski and Minor, above) and MOSFLM +(cited under [Indexing](#indexing)). ### Space group, twinning and pseudo-symmetry -**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search. Stage A scores each -candidate rotation operator by the correlation of I(h) with I(Rh) on **resolution-normalised** -intensities (E²), as POINTLESS does — both arms of a symmetry pair sit at the same |s|, so on raw -intensities the resolution fall-off is variance shared between them and lifts a false operator's -correlation as much as a true one's; the screw-axis test scores a -predicted-absent class against the rest of its own axial row rather than against a global mean or a -fixed cut, and lets confidence fall away with the number of axial reflections instead of refusing -below a count; the glide-plane test is that same test applied to a zone, scoring the extinguished -class against the rest of its own plane, as POINTLESS scores zonal absences. P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82 -[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693); P. R. Evans, "An -introduction to data reduction: space-group determination, scaling and intensity statistics" (2011), -Acta Cryst. D67, 282-292 [doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X); -P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta -Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061); -J. Agirre, M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular +**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search: scoring each candidate +rotation operator by the correlation of I(h) with I(Rh) on resolution-normalised intensities (E²); +the screw-axis test that scores a predicted-absent class against the rest of its own axial row and +lets confidence fall with the number of axial reflections instead of refusing below a count; and the +same test on a zone for a glide plane. P. Evans, "Scaling and assessment of data quality" (2006), +Acta Cryst. D62, 72-82 [doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693); +P. R. Evans, "An introduction to data reduction: space-group determination, scaling and intensity +statistics" (2011), Acta Cryst. D67, 282-292 +[doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X); P. R. Evans and +G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta Cryst. D69, +1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061); J. Agirre, +M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular crystallography" (2023), Acta Cryst. D79, 449-461 [doi:10.1107/S2059798323003595](https://doi.org/10.1107/S2059798323003595). -**Baur and Kassner** — the convention for which of two absence-equivalent glide groups the space-group -search writes (P2/c over Pc, C2/c over Cc): the centrosymmetric one, because a missed inversion centre is -the common error among published small-molecule space-group assignments. W. H. Baur and D. Kassner, -"The perils of Cc: comparing the frequencies of falsely assigned space groups with their general -population" (1992), Acta Cryst. B48, 356-369 +**Centrosymmetric glide groups** — of two absence-equivalent glide groups the search writes the +centrosymmetric one (P2/c over Pc, C2/c over Cc), a missed inversion centre being the common error. +W. H. Baur and D. Kassner, "The perils of Cc: comparing the frequencies of falsely assigned space +groups with their general population" (1992), Acta Cryst. B48, 356-369 [doi:10.1107/S0108768191014726](https://doi.org/10.1107/S0108768191014726). -**The centre-of-symmetry statistics** are Wilson's and Howells, Phillips & Rogers's: the intensity -distributions of acentric and centric structures and the cumulative N(z) test built on them, read here -on the general reflections of the Laue class beside <|E^2-1|> and Padilla and Yeates's L test (taken at -its centric value, 2/pi). A. J. C. Wilson, "The probability distribution of X-ray intensities" (1949), -Acta Cryst. 2, 318-321 [doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); -E. R. Howells, D. C. Phillips and D. Rogers, "The probability distribution of X-ray intensities. II. -Experimental investigation and the X-ray detection of centres of symmetry" (1950), Acta Cryst. 3, -210-214 [doi:10.1107/S0365110X50000513](https://doi.org/10.1107/S0365110X50000513). +**Centre-of-symmetry statistics** — the acentric and centric intensity distributions and the +cumulative N(z) test built on them, read beside <|E^2-1|> and the L test. A. J. C. Wilson, "The +probability distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321 +[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); E. R. Howells, +D. C. Phillips and D. Rogers, "The probability distribution of X-ray intensities. II. Experimental +investigation and the X-ray detection of centres of symmetry" (1950), Acta Cryst. 3, 210-214 +[doi:10.1107/S0365110X50000513](https://doi.org/10.1107/S0365110X50000513). -**The twinning L test** is Padilla and Yeates's: pairing each acentric reflection with a -symmetry-independent neighbour and reading the first and second moments of -L = (I1−I2)/(I1+I2) against their untwinned and perfect-twin values. J. E. Padilla and -T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and -pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, 1124-1130 -[doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). Their title claims -robustness to pseudo-centering, and this program's partner steps deliver it: a step of 2 along an -axis preserves the class of a half-integer pseudo-translation, which is what a pseudo-centering is. -That robustness does not extend to a pseudo-translation which is not half-integer, and the partner -steps are restricted when one is detected - see the translational-pseudo-symmetry note below. +**Twinning L test** — the moments of L = (I1−I2)/(I1+I2) over pairs of neighbouring acentric +reflections against their untwinned and perfect-twin values; its robustness to pseudo-centering is +kept by the partner steps, which are restricted when a non-half-integer pseudo-translation is found. +J. E. Padilla and T. O. Yeates, "A statistic for local intensity differences: robustness to +anisotropy and pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, +1124-1130 [doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). -**The twin fraction of the twin hypothesis, and the twinned model it is scored with**, follow Yeates: -the intensity a twinned crystal records is (1−α) I(h) + α I(Th), and the disagreement -H = |I(h) − I(Th)|/(I(h) + I(Th)) of the pairs a law relates is uniform on [0, 1−2α], so the median -gives the fraction. T. O. Yeates, "Detecting and overcoming crystal twinning" (1997), Methods Enzymol. -276, 344-358 [doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879%2897%2976068-3). +**Twin fraction and twinned intensity** — the model (1−α) I(h) + α I(Th), and the fraction read +from the median of H = |I(h) − I(Th)|/(I(h) + I(Th)), uniform on [0, 1−2α]. T. O. Yeates, "Detecting +and overcoming crystal twinning" (1997), Methods Enzymol. 276, 344-358 +[doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879%2897%2976068-3). -**Twin refinement with more than two domains, and the free set closed under the twin domains**, follow -REFMAC5's treatment - one fraction per domain operator of the lattice, refined with the model, small domains -dropped - and its practice of keeping twin-related reflections in one set: -G. N. Murshudov, P. Skubák, A. A. Lebedev, N. S. Pannu, R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and -A. A. Vagin, "REFMAC5 for the refinement of macromolecular crystal structures" (2011), Acta Cryst. D67, 355-367 -[doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314); P. Roversi, E. Blanc, S. Johnson and -S. M. Lea, "Tetartohedral twinning could happen to you too" (2012), Acta Cryst. D68, 418-424 -[doi:10.1107/S0907444912006737](https://doi.org/10.1107/S0907444912006737). +**Twin refinement** — one fraction per domain operator of the lattice refined with the model, small +domains dropped, and twin-related reflections kept in one free set, as REFMAC5 does; the fractions +re-fitted by least squares at every cycle as REFMAC5 refines them, against an amplitude likelihood +with the twinned model amplitude in it (a simplification of REFMAC5's twinned-intensity target). +G. N. Murshudov, P. Skubák, A. A. Lebedev, N. S. Pannu, R. A. Steiner, R. A. Nicholls, M. D. Winn, +F. Long and A. A. Vagin, "REFMAC5 for the refinement of macromolecular crystal structures" (2011), +Acta Cryst. D67, 355-367 [doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314); +P. Roversi, E. Blanc, S. Johnson and S. M. Lea, "Tetartohedral twinning could happen to you too" +(2012), Acta Cryst. D68, 418-424 +[doi:10.1107/S0907444912006737](https://doi.org/10.1107/S0907444912006737). A twin fraction as a +parameter refined beside the model is SHELXL's BASF (rugnux re-fits it between cycles rather than in +the same least squares): R. Herbst-Irmer and G. M. Sheldrick, "Refinement of twinned structures with +SHELXL97" (1998), Acta Cryst. B54, 443-449 +[doi:10.1107/S0108768197018454](https://doi.org/10.1107/S0108768197018454). The twin laws come from +the lattice's metric: A. A. Lebedev, A. A. Vagin and G. N. Murshudov, "Intensity statistics in +twinned crystals with examples from the PDB" (2006), Acta Cryst. D62, 83-95 +[doi:10.1107/S0907444905036759](https://doi.org/10.1107/S0907444905036759). -**The symmetry hypotheses a refused promotion leaves** - the adopted group and every other index-2 subgroup -of the refused group perfectly twinned by a law of its coset, all merging alike, carried to the model and -decided by placing and refining it in each - follow the advice to refine in the candidate groups and compare, -rather than decide from intensity statistics, that pseudo-symmetry and twinning call for: -P. H. Zwart, R. W. Grosse-Kunstleve, A. A. Lebedev, G. N. Murshudov and P. D. Adams, "Surprises and pitfalls +**Deciding pseudo-symmetry by refinement** — carrying the adopted group and the index-2 subgroups a +refused promotion leaves, each perfectly twinned by a law of its coset, to the model and deciding by +placing and refining it in each, rather than from intensity statistics. P. H. Zwart, +R. W. Grosse-Kunstleve, A. A. Lebedev, G. N. Murshudov and P. D. Adams, "Surprises and pitfalls arising from (pseudo)symmetry" (2008), Acta Cryst. D64, 99-107 -[doi:10.1107/S090744490705531X](https://doi.org/10.1107/S090744490705531X); A. A. Lebedev and M. N. Isupov, -"Space-group and origin ambiguity in macromolecular structures with pseudo-symmetry and its treatment with the -program Zanuda" (2014), Acta Cryst. D70, 2430-2443 +[doi:10.1107/S090744490705531X](https://doi.org/10.1107/S090744490705531X); A. A. Lebedev and +M. N. Isupov, "Space-group and origin ambiguity in macromolecular structures with pseudo-symmetry +and its treatment with the program Zanuda" (2014), Acta Cryst. D70, 2430-2443 [doi:10.1107/S1399004714014795](https://doi.org/10.1107/S1399004714014795). -**Twin refinement** (§15.12) refines the model against Yeates's twinned intensity, (1−α) I(h) + α I(Th), with the -twin fraction refined beside it as SHELXL refines BASF: R. Herbst-Irmer and G. M. Sheldrick, "Refinement of twinned -structures with SHELXL97" (1998), Acta Cryst. B54, 443-449 -[doi:10.1107/S0108768197018454](https://doi.org/10.1107/S0108768197018454). The twin laws come from the lattice's -metric as A. A. Lebedev, A. A. Vagin and G. N. Murshudov describe, "Intensity statistics in twinned crystals with -examples from the PDB" (2006), Acta Cryst. D62, 83-95 -[doi:10.1107/S0907444905036759](https://doi.org/10.1107/S0907444905036759), and the fractions are refined by -least squares at every cycle, as REFMAC5 refines them (Murshudov et al. 2011, Acta Cryst. D67, 355-367, cited -above). REFMAC5's target is a twinned-intensity likelihood; ours keeps the amplitude likelihood with the twinned -model amplitude in it, a simplification of that. - -**A first-pass cell that is the coincidence lattice of two twin domains** (twinning by reticular -merohedry of index 2 or 3, e.g. obverse/reverse rhombohedral domains or a cubic crystal twinned about -[111]) is recognised by the nodes that belong to neither domain being empty. The twin index is +**Twinning by reticular merohedry** — a first-pass cell that is the coincidence lattice of two twin +domains (reticular merohedry or pseudo-merohedry of index up to 10, e.g. obverse/reverse +rhombohedral domains) is recognised by the nodes of neither domain being empty. The twin index is Friedel's (G. Friedel, *Leçons de cristallographie*, 1926; no DOI); the split of the reflections into -those of one domain, of the other, of both and of neither is the one Herbst-Irmer and Sheldrick use -for obverse/reverse twins: R. Herbst-Irmer and G. M. Sheldrick, "Refinement of obverse/reverse twins" -(2002), Acta Cryst. B58, 477-481 -[doi:10.1107/S0108768102001039](https://doi.org/10.1107/S0108768102001039). +those of one domain, of the other, of both and of neither, and the treatment of reticular +pseudo-merohedry (Herbst-Irmer and Sheldrick 1998, above), are Herbst-Irmer and Sheldrick's: +R. Herbst-Irmer and G. M. Sheldrick, "Refinement of obverse/reverse twins" (2002), Acta Cryst. B58, +477-481 [doi:10.1107/S0108768102001039](https://doi.org/10.1107/S0108768102001039). -**Translational pseudo-symmetry** is detected from the native Patterson computed from the merged -intensities, and the interpretation of an off-origin peak as a pseudo-translation between copies of -the contents of the asymmetric unit - together with the modulation it puts on the intensities, which -is the second half of the test here - is Read, Adams and McCoy's. Their fitted peak-height table is -not used: the peak is scored against a per-dataset within-shell permutation null instead, because the -noise floor of the statistic depends strongly on how many reflections a dataset has. The same -modulation is what the axial systematic-absence test scores against, so that a reflection class a -pseudo-translation merely suppresses is not read as extinct and does not buy a screw axis; the -estimate of its depth there is our own, measured per axial row from the merged intensities rather -than from the Patterson vector. Where a symmetry operator or a twin law carries the vector to one -that is not ±itself, the modulation is also read on classes averaged over the lattice's rotations - -their modulation model summed over the copy pairs each rotation relates, as the measured intensity -sums them; that averaging is our own construction. R. J. Read, -P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of translational -noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183 +**Translational pseudo-symmetry** — the off-origin native-Patterson peak read as a pseudo-translation +between copies of the asymmetric unit, and the modulation it puts on the intensities, which the +axial systematic-absence test also scores against. The per-dataset permutation null for the peak, +the per-row estimate of the modulation depth, and its averaging over the lattice's rotations are +rugnux's own. R. J. Read, P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of +translational noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183 [doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374). -### Scaling, merging and data quality +### Scaling and merging -**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, per-observation -outlier rejection at merge, the scaling error model, and the treatment of a reflection whose -background is contaminated. Its published behaviour, and in places its source, settled several -choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS: implementation and -evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97 +**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, +per-observation outlier rejection at merge, the scaling error model, and the treatment of a +reflection whose background is contaminated; its published behaviour, and in places its source, +settled several choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS: +implementation and evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97 [doi:10.1107/S2059798317017235](https://doi.org/10.1107/S2059798317017235); D. G. Waterman, G. Winter, R. J. Gildea et al., "Diffraction-geometry refinement in the DIALS framework" (2016), Acta Cryst. D72, 558-575 [doi:10.1107/S2059798316002187](https://doi.org/10.1107/S2059798316002187); @@ -330,118 +282,130 @@ package: algorithms and new approaches for multi-crystal scaling" (2020), Acta C G. Winter, D. G. Waterman et al., "Robust background modelling in DIALS" (2016), J. Appl. Cryst. 49, 1912-1921 [doi:10.1107/S1600576716013595](https://doi.org/10.1107/S1600576716013595). -**Wilson outlier test** — judging an observation that has no symmetry mates against the acentric and -centric intensity distributions of its resolution shell, with the symmetry enhancement factor, is -Wilson's statistics; rejecting only observations that are also significant, and keeping a reflection -whose observations are all large, follows AIMLESS's EMAX test. A. J. C. Wilson, "The probability -distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321 -[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); -P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82 +**Wilson outlier test** — judging an observation without symmetry mates against the acentric and +centric distributions of its shell, with the symmetry enhancement factor; rejecting only significant +observations, and keeping a reflection whose observations are all large, follows AIMLESS's EMAX test. +A. J. C. Wilson, "The probability distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321 +[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); P. Evans, "Scaling and +assessment of data quality" (2006), Acta Cryst. D62, 72-82 [doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693). -**Amplitudes from intensities** — the posterior-mean amplitude of each merged intensity under the -acentric and centric Wilson priors is French and Wilson's; giving no amplitude to an intensity more -than 3.7 sigma below zero, and leaving such intensities out of the prior, follows CCP4's -[ctruncate](https://www.ccp4.ac.uk/) (C. Ballard and N. Stein). So does scaling each reflection's -Wilson prior by the anisotropy tensor along its direction, which ctruncate has done by default since -its version 1.7 ("use anisotropy in prior for truncate procedure"); rugnux uses its own tensor for it -(see Diffraction anisotropy below). S. French and K. Wilson, "On the treatment of negative intensity -observations" (1978), Acta Cryst. A34, 517-525 -[doi:10.1107/S0567739478001114](https://doi.org/10.1107/S0567739478001114); ctruncate is cited through -the CCP4 suite: M. D. Winn, C. C. Ballard, K. D. Cowtan et al., -"Overview of the CCP4 suite and current developments" (2011), Acta Cryst. D67, 235-242 +**Amplitudes from intensities** — the posterior-mean amplitude under the acentric and centric Wilson +priors. S. French and K. Wilson, "On the treatment of negative intensity observations" (1978), Acta +Cryst. A34, 517-525 [doi:10.1107/S0567739478001114](https://doi.org/10.1107/S0567739478001114). +Giving no amplitude to an intensity more than 3.7 sigma below zero, and scaling each reflection's +prior by the anisotropy tensor along its direction, follow CCP4's +[ctruncate](https://www.ccp4.ac.uk/) (C. Ballard and N. Stein), cited through the suite: +M. D. Winn, C. C. Ballard, K. D. Cowtan et al., "Overview of the CCP4 suite and current developments" +(2011), Acta Cryst. D67, 235-242 [doi:10.1107/S0907444910045749](https://doi.org/10.1107/S0907444910045749). -**Absorption as spherical harmonics** — describing an empirical absorption correction as a series of -real spherical harmonics of the beam directions in the crystal frame is Blessing's; its use as a +**Absorption as spherical harmonics** — the empirical absorption correction as a series of real +spherical harmonics of the beam directions in the crystal frame is Blessing's; its use as a restrained scaling surface of the diffracted-beam direction follows SCALA and AIMLESS. R. H. Blessing, "An empirical correction for absorption anisotropy" (1995), Acta Cryst. A51, 33-38 [doi:10.1107/S0108767394005726](https://doi.org/10.1107/S0108767394005726); P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82 [doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693). -**Diffraction anisotropy** — the description of the overall fall-off by a single anisotropic -displacement tensor, its symmetry constraints, and the fact that only its deviatoric part is -determined (the isotropic part being degenerate with the overall scale) are Sheriff and Hendrickson's. -The estimator fits that tensor to the observed intensity distribution, taking sigma(I) into account, -in the sense of Popov and Bourenkov. The directional diffraction limits - in a cone about -each principal direction, and the reporting of the anisotropic deltaB as the range of the principal -components - follow AIMLESS. Rugnux reports these; it corrects no intensity and removes no reflection -on a directional criterion. That diffraction anisotropy matters, and that every run should analyse -and report it, is inspired by STARANISO. S. Sheriff and W. A. Hendrickson, "Description of overall anisotropy in -diffraction from macromolecular crystals" (1987), Acta Cryst. A43, 118-121 -[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X); A. N. Popov and -G. P. Bourenkov, "Choice of data-collection parameters based on statistic modelling" (2003), Acta -Cryst. D59, 1145-1153 [doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163); -P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta -Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061); -[STARANISO](https://staraniso.globalphasing.org/): I. J. Tickle, C. Flensburg, P. Keller, -W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO -([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)), -Cambridge, United Kingdom: Global Phasing Ltd. - -**Data-quality statistics** follow the established conventions rather than any one program: R_meas -and R_pim, CC1/2 and CC\*, the per-shell CC(model, data) between F^2_calc and F^2_obs, and the -reporting of I/sigma(I). K. Diederichs and P. A. Karplus, "Improved -R-factors for diffraction data analysis in macromolecular crystallography" (1997), Nat. Struct. Biol. -4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269); P. A. Karplus and -K. Diederichs, "Linking crystallographic model and data quality" (2012), Science 336, 1030-1033 -[doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231) (CC\* is also what a joint merge of -several sweeps tests their isomorphism against: CC_AB / (CC\*_A CC\*_B) is the cosine of the angle between -two data sets, their systematic difference apart from their noise, as K. Diederichs, "Dissecting random and -systematic differences between noisy composite data sets" (2017), Acta Cryst. D73, 286-293 -[doi:10.1107/S2059798317000699](https://doi.org/10.1107/S2059798317000699) shows - which is why the join is -decided on the size of that ratio, and a doubtful sweep by the merge with and without it, the idea of -delta-CC1/2 by Assmann, Brehm and Diederichs (2016), below); M. S. Weiss, "Global -indicators of X-ray data quality" (2001), J. Appl. Cryst. 34, 130-135 -[doi:10.1107/S0021889800018227](https://doi.org/10.1107/S0021889800018227) (R_pim beside R_meas and -R_merge); K. Diederichs and -P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222 -[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121). - -**The Whittaker smoother** — the per-frame scale of the rotation fulls is a penalised least-squares -curve (a second-difference penalty, the smoothness chosen by cross-validation) -in the form P. H. C. Eilers gave Whittaker's graduation: P. H. C. Eilers, "A perfect smoother" (2003), +**Per-frame scale smoothing** — the Whittaker smoother (a second-difference penalty, the smoothness +chosen by cross-validation) in Eilers's form. P. H. C. Eilers, "A perfect smoother" (2003), Anal. Chem. 75, 3631-3636 [doi:10.1021/ac034173t](https://doi.org/10.1021/ac034173t); E. T. Whittaker, "On a new method of graduation" (1923), Proc. Edinburgh Math. Soc. 41, 63-75 [doi:10.1017/S0013091500077853](https://doi.org/10.1017/S0013091500077853). -**Fisher's z-transformation** — the cross-validation of the scaling correction surfaces averages the -change of the half-set CC1/2 over resolution shells on atanh(CC), so that the shells near CC = 1, where -a multiplicative error shows, are not outweighed by the noise of the shells without signal. R. A. -Fisher, "Frequency distribution of the values of the correlation coefficient in samples from an -indefinitely large population" (1915), Biometrika 10, 507-521 -[doi:10.2307/2331838](https://doi.org/10.2307/2331838). +**Fisher's z-transformation** — averaging the change of the half-set CC1/2 over resolution shells on +atanh(CC) when the correction surfaces are cross-validated. R. A. Fisher, "Frequency distribution of +the values of the correlation coefficient in samples from an indefinitely large population" (1915), +Biometrika 10, 507-521 [doi:10.2307/2331838](https://doi.org/10.2307/2331838). -**The frame disposition** - which stretches of a rotation sweep are kept, carried at reduced weight or -dropped from the merge - decides every exclusion on delta-CC1/2, the change in the overall CC1/2 when a -group of images is left out, measured in the sigma-tau form so that no random half-dataset split is -involved. The statistic, the Fisher transformation used to compare it across CC1/2 values, its standard -error going as the inverse square root of the reflection count, and the rejection discipline (never -remove a group whose delta-CC1/2 is positive or near zero; remove a little, re-form the reference and -repeat) are all taken from its authors, whose XDSCC12 is the reference implementation. -G. Assmann, W. Brehm and K. Diederichs, "Identification of rogue datasets in serial crystallography" -(2016), J. Appl. Cryst. 49, 1021-1028 -[doi:10.1107/S1600576716005471](https://doi.org/10.1107/S1600576716005471); G. M. Assmann, M. Wang and -K. Diederichs, "Making a difference in multi-data-set crystallography: simple and deterministic +**Frame disposition by delta-CC1/2** — the statistic in its sigma-tau form, the Fisher transformation +used to compare it, its standard error, and the rejection discipline (never remove a group whose +delta-CC1/2 is positive or near zero; remove a little, re-form the reference and repeat), whose +reference implementation is XDSCC12. G. Assmann, W. Brehm and K. Diederichs, "Identification of +rogue datasets in serial crystallography" (2016), J. Appl. Cryst. 49, 1021-1028 +[doi:10.1107/S1600576716005471](https://doi.org/10.1107/S1600576716005471); G. M. Assmann, M. Wang +and K. Diederichs, "Making a difference in multi-data-set crystallography: simple and deterministic data-scaling/selection methods" (2020), Acta Cryst. D76, 636-652 -[doi:10.1107/S2059798320006348](https://doi.org/10.1107/S2059798320006348). That the same statistic -belongs at scaling, applied to groups of images rather than to whole datasets, follows -[DIALS](https://dials.github.io/) (`dials.scale`, delta-CC1/2 image-group filtering): +[doi:10.1107/S2059798320006348](https://doi.org/10.1107/S2059798320006348). Applying it at scaling, +to groups of images rather than whole datasets, follows DIALS (`dials.scale`): J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399 [doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198). -**Several sweeps merged as one** - each sweep's partials combined into fulls on their own, and the -fulls of all the sweeps then scaled together, every sweep with its own frame scales and a relative B -against the joint merge, is how XSCALE merges several data sets (W. Kabsch, "Integration, scaling, -space-group assignment and post-refinement" (2010), above). Reading the radiation dose a sweep was taken -at from its relative B, which orders the sweeps by dose from the data rather than from the order they -were given in, follows K. Diederichs, "Some aspects of quantitative analysis and correction of radiation -damage" (2006), Acta Cryst. D62, 96-101 +**Several sweeps merged as one** — each sweep's partials combined into fulls on their own, then the +fulls of all sweeps scaled together with per-sweep frame scales and a relative B, as XSCALE merges +data sets (W. Kabsch, "Integration, scaling, space-group assignment and post-refinement" (2010), +above). Ordering the sweeps by dose from their relative B follows K. Diederichs, "Some aspects of +quantitative analysis and correction of radiation damage" (2006), Acta Cryst. D62, 96-101 [doi:10.1107/S0907444905031537](https://doi.org/10.1107/S0907444905031537). -**Uncertainty conventions** follow the IUCr Commission on Crystallographic Nomenclature: +**Reference sweep of a joint merge** — the sweep whose own merge reaches the finest resolution, as +KAMO takes the data set of the smallest Wilson B. K. Yamashita, K. Hirata and M. Yamamoto, "KAMO: +towards automated data processing for microcrystals" (2018), Acta Cryst. D74, 441-449 +[doi:10.1107/S2059798318004576](https://doi.org/10.1107/S2059798318004576). + +### Serial data + +**Per-crystal post-refinement of stills** — refining per still the partiality parameters the spot +positions cannot pin (a small reorientation, the rocking width, the radial scale of the reciprocal +lattice) against the merged intensities, after PRIME and the mosaic-block model of Sauter et al. +M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, +A. T. Brunger and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging +biological systems from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter, +J. Hattne, A. S. Brewster, N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and +physical properties from single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not +verified here, so not given). + +**Indexing ambiguity without a reference** — putting each crystal into the indexing in which its +intensities agree best with all the others, iterated to convergence, the method CrystFEL's +`ambigator` also implements; rugnux measures the agreement as a product of standardised amplitudes +rather than a correlation. W. Brehm and K. Diederichs, "Breaking the indexing ambiguity in serial +crystallography" (2014), Acta Cryst. D70, 101-109 +[doi:10.1107/S1399004713025431](https://doi.org/10.1107/S1399004713025431). The same objective +settles the indexing of each sweep of a joint merge, decided over all pairs at once as dials.cosym +does: R. J. Gildea and G. Winter, "Determination of Patterson group symmetry from sparse +multi-crystal data sets in the presence of an indexing ambiguity" (2018), Acta Cryst. D74, 405-410 +[doi:10.1107/S2059798318002978](https://doi.org/10.1107/S2059798318002978). + +### Data quality and anisotropy + +**Data-quality statistics** — R_meas and R_pim, CC1/2 and CC\*, the per-shell CC(model, data), and +the reporting of I/sigma(I) follow the established conventions. K. Diederichs and P. A. Karplus, +"Improved R-factors for diffraction data analysis in macromolecular crystallography" (1997), +Nat. Struct. Biol. 4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269); +P. A. Karplus and K. Diederichs, "Linking crystallographic model and data quality" (2012), Science +336, 1030-1033 [doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231); M. S. Weiss, +"Global indicators of X-ray data quality" (2001), J. Appl. Cryst. 34, 130-135 +[doi:10.1107/S0021889800018227](https://doi.org/10.1107/S0021889800018227); K. Diederichs and +P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222 +[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121). + +**Isomorphism of sweeps** — CC_AB / (CC\*_A CC\*_B) as the cosine of the angle between two data +sets, on which a joint merge decides whether a sweep joins. K. Diederichs, "Dissecting random and +systematic differences between noisy composite data sets" (2017), Acta Cryst. D73, 286-293 +[doi:10.1107/S2059798317000699](https://doi.org/10.1107/S2059798317000699). + +**Diffraction anisotropy** — the overall fall-off as a single anisotropic displacement tensor, its +symmetry constraints and the determinacy of its deviatoric part only: S. Sheriff and +W. A. Hendrickson, "Description of overall anisotropy in diffraction from macromolecular crystals" +(1987), Acta Cryst. A43, 118-121 +[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X). Fitting it to the +intensity distribution with sigma(I) taken into account: A. N. Popov and G. P. Bourenkov, "Choice of +data-collection parameters based on statistic modelling" (2003), Acta Cryst. D59, 1145-1153 +[doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163). The directional +diffraction limits and the anisotropic deltaB reported as the range of the principal components +follow AIMLESS: P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" +(2013), Acta Cryst. D69, 1204-1214 +[doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061). That every run should +analyse and report its anisotropy is inspired by +[STARANISO](https://staraniso.globalphasing.org/): I. J. Tickle, C. Flensburg, P. Keller, +W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO +([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)), +Cambridge, United Kingdom: Global Phasing Ltd. Rugnux reports the anisotropy; it corrects no +intensity and removes no reflection on a directional criterion. + +**Uncertainty conventions** — the IUCr Commission on Crystallographic Nomenclature's. D. Schwarzenbach, S. C. Abrahams, H. D. Flack et al., "Statistical descriptors in crystallography: Report of the IUCr Subcommittee on Statistical Descriptors" (1989), Acta Cryst. A45, 63-75 [doi:10.1107/S0108767388009596](https://doi.org/10.1107/S0108767388009596); D. Schwarzenbach, @@ -449,410 +413,366 @@ S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descrip crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995), Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340). -**Unit-cell e.s.d.s** — that the e.s.d.s of a least-squares cell fit are too small, seeing only the -scatter of the centroids, is Herbstein's survey; rugnux therefore also fits the cell on blocks of -consecutive frames and writes the larger of the two estimates: F. H. Herbstein, "How precise are -measurements of unit-cell dimensions from single crystals?" (2000), Acta Cryst. B56, 547-557 -[doi:10.1107/S010876810000269X](https://doi.org/10.1107/S010876810000269X). +**Unit-cell e.s.d.s** — that the e.s.d.s of a least-squares cell fit are too small is Herbstein's +survey, which is why the cell is also fitted on blocks of consecutive frames. F. H. Herbstein, "How +precise are measurements of unit-cell dimensions from single crystals?" (2000), Acta Cryst. B56, +547-557 [doi:10.1107/S010876810000269X](https://doi.org/10.1107/S010876810000269X). -**Per-crystal post-refinement of serial stills** - refining, for each still, the partiality -parameters the spot positions cannot pin (a small reorientation, the crystal's own rocking width and the -radial scale of its reciprocal lattice, which a shot's photon energy or a crystal's cell moves) against -the merged intensities follows the post-refinement of PRIME and the mosaic-block model of Sauter et al.: -M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, A. T. Brunger -and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging biological systems -from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter, J. Hattne, A. S. Brewster, -N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and physical properties from -single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not verified here, so not given). - -**Indexing ambiguity of serial data without a reference** — where a serial run's lattice is more -symmetric than its crystals and no reference or model is given, each crystal is put into the indexing in -which its intensities agree best with those of all the other crystals, crystal by crystal until none -changes. The objective - the summed agreement between every pair of crystals - and the iteration that -maximises it are taken from Brehm and Diederichs, whose method CrystFEL's `ambigator` also implements; -rugnux measures the agreement as a product of standardised amplitudes (square roots of the intensities) -rather than a correlation, which lets the sum over all partners be taken against one running sum instead -of pair by pair. -W. Brehm and K. Diederichs, "Breaking the indexing ambiguity in serial crystallography" (2014), -Acta Cryst. D70, 101-109 [doi:10.1107/S1399004713025431](https://doi.org/10.1107/S1399004713025431). -The same objective settles which twin law puts each sweep of a joint merge of several sweeps on the -reference's indexing - decided over all pairs of sweeps at once, as dials.cosym does for its data sets: -R. J. Gildea and G. Winter, "Determination of Patterson group symmetry from sparse multi-crystal data sets -in the presence of an indexing ambiguity" (2018), Acta Cryst. D74, 405-410 -[doi:10.1107/S2059798318002978](https://doi.org/10.1107/S2059798318002978). - -**Reference sweep of a joint merge** — the sweep the others are put on the axes of is the one whose own -merge reaches the finest resolution (among the sweeps with a measured ISa), as KAMO takes the data set of -the smallest Wilson B as its reference: -K. Yamashita, K. Hirata and M. Yamamoto, "KAMO: towards automated data processing for microcrystals" -(2018), Acta Cryst. D74, 441-449 [doi:10.1107/S2059798318004576](https://doi.org/10.1107/S2059798318004576). - -**Measurability** — the fraction of the acentric Bijvoet pairs with |I(+)-I(-)| at three sigma and both -hands at I/sigma 3, reported beside SigAno and CCanom — is phenix.xtriage's: P. H. Zwart, "Anomalous signal -indicators in protein crystallography" (2005), Acta Cryst. D61, 1437-1448 +**Measurability** — the fraction of acentric Bijvoet pairs significant at three sigma, reported +beside SigAno and CCanom, is phenix.xtriage's. P. H. Zwart, "Anomalous signal indicators in protein +crystallography" (2005), Acta Cryst. D61, 1437-1448 [doi:10.1107/S0907444905023589](https://doi.org/10.1107/S0907444905023589). -**Ice in the merged intensities** — the merged reflections on the hexagonal-ice rings are compared with -the off-ring reflections beside them in resolution, and the rings' excesses are fitted together in -proportion to hexagonal ice's line strengths; the table of those strengths, and the idea of reading ice -off a merge's own intensities at the ring positions, are phenix.xtriage's: P. H. Zwart, R. W. -Grosse-Kunstleve and P. D. Adams, "Xtriage and Fest: automatic assessment of X-ray data and substructure -structure factor estimation" (2005), CCP4 Newsletter on Protein Crystallography 43 (no DOI). rugnux's -comparison is local in resolution, where xtriage's is a z-score of a shell against the whole Wilson curve. +**Ice in the merged intensities** — reading ice off a merge's own intensities at the hexagonal-ice +ring positions, and the table of the rings' line strengths, are phenix.xtriage's; rugnux's comparison +is local in resolution where xtriage's is against the whole Wilson curve. P. H. Zwart, +R. W. Grosse-Kunstleve and P. D. Adams, "Xtriage and Fest: automatic assessment of X-ray data and +substructure structure factor estimation" (2005), CCP4 Newsletter on Protein Crystallography 43 (no +DOI). -**Advice on the next collection** - the rotation a sweep needs and the blind region about the spindle -that a second, tilted sweep recovers are the data-collection geometry set out by Z. Dauter, -"Data-collection strategies" (1999), Acta Cryst. D55, 1703-1717 (DOI not verified here). +**Advice on the next collection** — the rotation a sweep needs and the blind region about the +spindle that a tilted second sweep recovers. Z. Dauter, "Data-collection strategies" (1999), Acta +Cryst. D55, 1703-1717 (DOI not verified here). -### Model-based analysis and maps +### Model placement and molecular replacement -**Bulk-solvent correction and overall scaling** — the model's structure factors are put on the -observed scale with an overall factor, an anisotropic B and a flat bulk-solvent term, the flat-mask -model of A. Fokine and A. Urzhumtsev, "Flat bulk-solvent model: obtaining optimal parameters" -(2002), Acta Cryst. D58, 1387-1392 -[doi:10.1107/S0907444902010284](https://doi.org/10.1107/S0907444902010284), which is also the source -of the starting values and of the range those two parameters are physically meaningful over. The mask itself - -the region a probe cannot reach from the atoms, shrunk back - is J.-S. Jiang and A. T. Brünger's, "Protein -hydration observed by X-ray diffraction: solvation properties of penicillopepsin and neuraminidase crystal -structures" (1994), J. Mol. Biol. 243, 100-115 -[doi:10.1006/jmbi.1994.1633](https://doi.org/10.1006/jmbi.1994.1633), drawn with the radii REFMAC5 uses. The -procedure that fits them — a grid search over that range for the solvent pair, with the overall -scale and the anisotropic B refitted at every grid point — follows P. V. Afonine, +**Bulk-solvent correction and overall scaling** — the flat bulk-solvent model, its starting values +and the physically meaningful range of its two parameters: A. Fokine and A. Urzhumtsev, "Flat +bulk-solvent model: obtaining optimal parameters" (2002), Acta Cryst. D58, 1387-1392 +[doi:10.1107/S0907444902010284](https://doi.org/10.1107/S0907444902010284). The mask (the region a +probe cannot reach, shrunk back), drawn with the radii REFMAC5 uses: J.-S. Jiang and A. T. Brünger, +"Protein hydration observed by X-ray diffraction: solvation properties of penicillopepsin and +neuraminidase crystal structures" (1994), J. Mol. Biol. 243, 100-115 +[doi:10.1006/jmbi.1994.1633](https://doi.org/10.1006/jmbi.1994.1633). The grid search over the +solvent pair with scale and anisotropic B refitted at each point: P. V. Afonine, R. W. Grosse-Kunstleve and P. D. Adams, "A robust bulk-solvent correction and anisotropic scaling procedure" (2005), Acta Cryst. D61, 850-855 [doi:10.1107/S0907444905007894](https://doi.org/10.1107/S0907444905007894). The fit is unweighted, -as in both that procedure and REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu, -R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, "REFMAC5 for the refinement of -macromolecular crystal structures" (2011), Acta Cryst. D67, 355-367 +as there and in REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu, R. A. Steiner, +R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, "REFMAC5 for the refinement of macromolecular +crystal structures" (2011), Acta Cryst. D67, 355-367 [doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314). -**sigma_A map coefficients** — the maps written by `--model` are weighted by a maximum-likelihood -sigma_A estimated per resolution shell, giving 2mFo-DFc and mFo-DFc rather than 2Fo-Fc and Fo-Fc. -What is taken is the formalism itself: the Rice and Woolfson likelihoods of |Fo| given |Fc| and -sigma_A, the figure of merit m and the scale D that follow from it, and the result that the -bias-corrected coefficient is 2mFo-DFc for an acentric reflection and mFo for a centric one. -R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures with -errors" (1986), Acta Cryst. A42, 140-149 +**sigma_A map coefficients** — the Rice and Woolfson likelihoods of |Fo| given |Fc| and sigma_A, the +figure of merit m and the scale D, and the bias-corrected coefficients 2mFo-DFc (acentric) and mFo +(centric). R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures +with errors" (1986), Acta Cryst. A42, 140-149 [doi:10.1107/S0108767386099622](https://doi.org/10.1107/S0108767386099622). **[ANODE](https://doi.org/10.1107/S0021889811041768)** — reading the anomalous difference map at the atoms of a supplied model and reporting the strongest sites by name, instead of searching the map for -blobs. The map itself is the textbook anomalous difference Fourier; what is taken from ANODE is that -reading: A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density calculation" (2011), +blobs. A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density calculation" (2011), J. Appl. Cryst. 44, 1285-1287 [doi:10.1107/S0021889811041768](https://doi.org/10.1107/S0021889811041768). -**Uniform random rotations** — the null a supplied model is scored against re-orients that model at -random about its own centroid, and the rotations are drawn uniformly from SO(3) through a uniform -random unit quaternion. K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk, -Academic Press (1992), 124-132 (no DOI). +**Uniform random rotations** — the null a supplied model, and a molecular-replacement search, are +scored against draws rotations uniformly from SO(3) through a uniform random unit quaternion. +K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk, Academic Press (1992), +124-132 (no DOI). -**Variable projection** — the rigid-body placement of a supplied model re-fits the overall scale at -every step, and its Jacobian folds that re-fit in by projecting the scale's own derivatives out of the -placement's, in Kaufman's simplified form of Golub and Pereyra's derivative of the reduced problem. +**Variable projection** — the rigid-body placement re-fits the overall scale at every step and folds +that re-fit into its Jacobian, in Kaufman's simplified form of Golub and Pereyra's derivative. G. H. Golub and V. Pereyra, "The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate" (1973), SIAM J. Numer. Anal. 10, 413-432 -[doi:10.1137/0710036](https://doi.org/10.1137/0710036). L. Kaufman, "A variable projection method +[doi:10.1137/0710036](https://doi.org/10.1137/0710036); L. Kaufman, "A variable projection method for solving separable nonlinear least squares problems" (1975), BIT 15, 49-57 [doi:10.1007/BF01932995](https://doi.org/10.1007/BF01932995). -**Predicted models** — `--model` (and `--mr`) place an AlphaFold prediction (or any model the rigid -body does not place) by molecular replacement. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction -with AlphaFold" (2021), Nature 596, 583-589 +**Predicted models** — `--model` (and `--mr`) place an AlphaFold prediction by molecular replacement. +J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction with +AlphaFold" (2021), Nature 596, 583-589 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); the models are those of the AlphaFold Protein Structure Database: M. Varadi, S. Anyango, M. Deshpande et al., "AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res. 50, D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061). The model is prepared as -phenix.process_predicted_model prepares one - residues below pLDDT 70 removed, B-factors from the -coordinate error the pLDDT predicts: R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold -models to work with phenix.process_predicted_model and ISOLDE" (2022), Acta Cryst. D78, 1303-1314 -[doi:10.1107/S2059798322010026](https://doi.org/10.1107/S2059798322010026). - -**Molecular replacement** — the search scores each orientation of the model against the data by -interpolating its molecular transform, computed once by FFT, as Navaza's AMoRe does: J. Navaza, -"AMoRe: an automated package for molecular replacement" (1994), Acta Cryst. A50, 157-163 -[doi:10.1107/S0108767393007597](https://doi.org/10.1107/S0108767393007597). The translation -function is the full-symmetry one, all pairs of symmetry copies on one Fourier synthesis: -R. A. Crowther and D. M. Blow, "A method of positioning a known molecule in an unknown crystal -structure" (1967), Acta Cryst. 23, 544-548 -[doi:10.1107/S0365110X67003172](https://doi.org/10.1107/S0365110X67003172). A placement is called -a solution, as Phaser calls one, where its translation-function Z-score reaches 8: A. J. McCoy, -R. W. Grosse-Kunstleve, P. D. Adams, M. D. Winn, L. C. Storoni and R. J. Read, "Phaser -crystallographic software" (2007), J. Appl. Cryst. 40, 658-674 -[doi:10.1107/S0021889807021206](https://doi.org/10.1107/S0021889807021206). Whether one copy fits -the cell is read off the Matthews coefficient: B. W. Matthews, "Solvent content of protein -crystals" (1968), J. Mol. Biol. 33, 491-497 -[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836%2868%2990205-2). How many copies -to search for is weighed by a distribution of the Matthews coefficient over protein crystals made from -the mode and mean of Kantardjieff and Rupp's: K. A. Kantardjieff and B. Rupp, "Matthews coefficient -probabilities: Improved estimates for unit cell contents of proteins, DNA, and protein-nucleic acid -complex crystals" (2003), Protein Sci. 12, 1865-1871 -[doi:10.1110/ps.0350503](https://doi.org/10.1110/ps.0350503). A copy refined beside the others has the -others' placements read off by least-squares superposition: W. Kabsch, "A solution for the best rotation -to relate two sets of vectors" (1976), Acta Cryst. A32, 922-923 -[doi:10.1107/S0567739476001873](https://doi.org/10.1107/S0567739476001873). The placements are judged by -their log-likelihood gain, from the Rice and Woolfson distributions of the observed normalised amplitudes -given the model's, with sigma_A from the model's expected coordinate error: R. J. Read, "Pushing the -boundaries of molecular replacement with maximum likelihood" (2001), Acta Cryst. D57, 1373-1382 -[doi:10.1107/S0907444901012471](https://doi.org/10.1107/S0907444901012471); and the gain is read as a -Z-score over the same orientation at other positions, as the likelihood-enhanced translation function's -is: A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni and R. J. Read, "Likelihood-enhanced fast -translation functions" (2005), Acta Cryst. D61, 458-464 -[doi:10.1107/S0907444905001617](https://doi.org/10.1107/S0907444905001617). A model whose expected -log-likelihood gain - the sum of sigma_A^4 / 2 over the reflections - is too small for a placement to stand -out is not searched for: A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason, -B. Lohkamp and R. J. Read, "Ab initio solution of macromolecular crystal structures without direct -methods" (2017), Proc. Natl. Acad. Sci. USA 114, 3637-3641 -[doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114). Translational NCS is looked -for in the native Patterson before the search, an off-origin peak read as Read, Adams and McCoy read -it (see -[Translational pseudo-symmetry](#space-group-twinning-and-pseudo-symmetry) above). The logarithm of the Bessel -function I0 in it is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun, -"Handbook of Mathematical Functions" (1964), National Bureau of Standards, 9.8.1-9.8.2. A predicted -model is cut into rigid domains by its predicted aligned error, as Oeffner et al. (above) do before -molecular replacement; the error itself is AlphaFold's: J. Jumper, R. Evans, A. Pritzel et al., "Highly -accurate protein structure prediction with AlphaFold" (2021), Nature 596, 583-589 +phenix.process_predicted_model prepares one (residues below pLDDT 70 removed, B factors from the +predicted coordinate error) and cut into rigid domains by its predicted aligned error: +R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold models to work with +phenix.process_predicted_model and ISOLDE" (2022), Acta Cryst. D78, 1303-1314 +[doi:10.1107/S2059798322010026](https://doi.org/10.1107/S2059798322010026); the error itself is +AlphaFold's: J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction +with AlphaFold" (2021), Nature 596, 583-589 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2). -**Structure refinement** (of a placed model) — the placed model is refined against the maximum-likelihood -amplitude target, the Rice distribution of an observed amplitude given the model's (its centric -counterpart for a centric reflection): N. S. Pannu and R. J. Read, "Improved structure refinement -through maximum likelihood" (1996), Acta Cryst. A52, 659-668 +**Molecular replacement search** — scoring each orientation of the model by interpolating its +molecular transform, computed once by FFT, as AMoRe does: J. Navaza, "AMoRe: an automated package for +molecular replacement" (1994), Acta Cryst. A50, 157-163 +[doi:10.1107/S0108767393007597](https://doi.org/10.1107/S0108767393007597). The full-symmetry +translation function, all pairs of symmetry copies on one Fourier synthesis: R. A. Crowther and +D. M. Blow, "A method of positioning a known molecule in an unknown crystal structure" (1967), Acta +Cryst. 23, 544-548 [doi:10.1107/S0365110X67003172](https://doi.org/10.1107/S0365110X67003172). +A placement is a solution, as Phaser calls one, where its translation-function Z-score reaches 8: +A. J. McCoy, R. W. Grosse-Kunstleve, P. D. Adams, M. D. Winn, L. C. Storoni and R. J. Read, "Phaser +crystallographic software" (2007), J. Appl. Cryst. 40, 658-674 +[doi:10.1107/S0021889807021206](https://doi.org/10.1107/S0021889807021206). + +**Cell contents** — whether one copy fits the cell is read off the Matthews coefficient: +B. W. Matthews, "Solvent content of protein crystals" (1968), J. Mol. Biol. 33, 491-497 +[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836%2868%2990205-2); how many +copies to search for is weighed by its distribution over protein crystals: K. A. Kantardjieff and +B. Rupp, "Matthews coefficient probabilities: Improved estimates for unit cell contents of proteins, +DNA, and protein-nucleic acid complex crystals" (2003), Protein Sci. 12, 1865-1871 +[doi:10.1110/ps.0350503](https://doi.org/10.1110/ps.0350503). The placements of several copies are +compared by least-squares superposition: W. Kabsch, "A solution for the best rotation to relate two +sets of vectors" (1976), Acta Cryst. A32, 922-923 +[doi:10.1107/S0567739476001873](https://doi.org/10.1107/S0567739476001873). + +**Likelihood scoring of placements** — the log-likelihood gain from the Rice and Woolfson +distributions of the normalised amplitudes given the model's, with sigma_A from the model's expected +coordinate error: R. J. Read, "Pushing the boundaries of molecular replacement with maximum +likelihood" (2001), Acta Cryst. D57, 1373-1382 +[doi:10.1107/S0907444901012471](https://doi.org/10.1107/S0907444901012471); the gain read as a +Z-score over the same orientation at other positions, as the likelihood-enhanced translation +function's is: A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni and R. J. Read, +"Likelihood-enhanced fast translation functions" (2005), Acta Cryst. D61, 458-464 +[doi:10.1107/S0907444905001617](https://doi.org/10.1107/S0907444905001617); and a model whose +expected log-likelihood gain is too small for a placement to stand out is not searched for: +A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason, B. Lohkamp and R. J. Read, +"Ab initio solution of macromolecular crystal structures without direct methods" (2017), Proc. Natl. +Acad. Sci. USA 114, 3637-3641 [doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114). +Translational NCS is looked for in the native Patterson before the search, read as under +[Translational pseudo-symmetry](#space-group-twinning-and-pseudo-symmetry). The logarithm of the +Bessel function I0 is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun, +"Handbook of Mathematical Functions" (1964), National Bureau of Standards, 9.8.1-9.8.2. + +### Structure refinement + +**Likelihood target** — the Rice distribution of an observed amplitude given the model's, and its +centric counterpart. N. S. Pannu and R. J. Read, "Improved structure refinement through maximum +likelihood" (1996), Acta Cryst. A52, 659-668 [doi:10.1107/S0108767396004370](https://doi.org/10.1107/S0108767396004370); G. N. Murshudov, -A. A. Vagin and E. J. Dodson, "Refinement of macromolecular structures by the maximum-likelihood method" -(1997), Acta Cryst. D53, 240-255 [doi:10.1107/S0907444996012255](https://doi.org/10.1107/S0907444996012255). -Its two parameters are fitted by maximum likelihood in resolution bins - the variance on the free -reflections, D on the working reflections with the measurement error added to that variance, which is how -REFMAC5 divides them (Murshudov et al. 2011, above); the refined model's maps are weighted by the same D -and figure of merit. Estimating the likelihood's parameters on free reflections follows V. Y. Lunin and T. P. Skovoroda, "R-free likelihood-based estimates of errors for phases -calculated from atomic models" (1995), Acta Cryst. A51, 880-887 -[doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin, P. V. Afonine and -A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors" (2002), Acta Cryst. A58, -270-282 [doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). +A. A. Vagin and E. J. Dodson, "Refinement of macromolecular structures by the maximum-likelihood +method" (1997), Acta Cryst. D53, 240-255 +[doi:10.1107/S0907444996012255](https://doi.org/10.1107/S0907444996012255). Its parameters are fitted +in resolution bins, the variance on the free reflections and D on the working ones, as REFMAC5 +divides them (Murshudov et al. 2011, above); estimating them on free reflections follows +V. Y. Lunin and T. P. Skovoroda, "R-free likelihood-based estimates of errors for phases calculated +from atomic models" (1995), Acta Cryst. A51, 880-887 +[doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin, +P. V. Afonine and A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors" +(2002), Acta Cryst. A58, 270-282 +[doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). -**The gradient** with respect to every atom is taken by the FFT method - the derivatives of the target with respect to the -structure factors transformed into a map and summed against each atom's density derivatives: -R. C. Agarwal, "A new least-squares refinement technique based on the fast Fourier transform algorithm" -(1978), Acta Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618). +**Gradients by FFT** — the derivatives of the target with respect to the structure factors +transformed into a map and summed against each atom's density derivatives. R. C. Agarwal, "A new +least-squares refinement technique based on the fast Fourier transform algorithm" (1978), Acta +Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618). -**The positions and B factors** are minimised by L-BFGS (rugnux's own, its memory kept from one macro-cycle to -the next, the B factors' steps scaled by their estimated curvature): D. C. Liu and J. Nocedal, "On the limited +**Minimiser** — L-BFGS (rugnux's own implementation). D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization" (1989), Math. Program. 45, 503-528 -[doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116), whose two-loop recursion is J. Nocedal's, "Updating -quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782 +[doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116); its two-loop recursion: J. Nocedal, +"Updating quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782 [doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). -**The geometry restraints** are those of the CCP4 Monomer Library, a subset of which rugnux carries (see THIRD_PARTY_NOTICES.md): A. A. Vagin, -R. A. Steiner, A. A. Lebedev, L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5 -dictionary: organization of prior chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60, -2184-2195 [doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510); applied to the model -through GEMMI's topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, "GEMMI: A -library for structural biology" (2022), J. Open Source Softw. 7, 4200 +**Geometry restraints** — those of the CCP4 Monomer Library, a subset of which rugnux carries (see +[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). A. A. Vagin, R. A. Steiner, A. A. Lebedev, +L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5 dictionary: organization of prior +chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60, 2184-2195 +[doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510). They are applied +through GEMMI's topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, "GEMMI: +A library for structural biology" (2022), J. Open Source Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). The torsion derivatives follow -A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion angles -in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141 +A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion +angles in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141 [doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T](https://doi.org/10.1002/(SICI)1096-987X(19960715)17:9%3C1132::AID-JCC5%3E3.0.CO;2-T). -**The B factors of bonded atoms**, and of atoms in contact, are restrained to each other in the -Kullback-Leibler form REFMAC5 uses, whose tolerance grows with B (Murshudov et al. 2011, above; the divergence itself -is S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22, 79-86 -[doi:10.1214/aoms/1177729694](https://doi.org/10.1214/aoms/1177729694)), and the -overall isotropic B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012, -below). +**B-factor restraints** — bonded atoms, and atoms in contact, restrained to each other in the +Kullback-Leibler form REFMAC5 uses (Murshudov et al. 2011, above); the divergence itself is +S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22, +79-86 [doi:10.1214/aoms/1177729694](https://doi.org/10.1214/aoms/1177729694). The overall isotropic +B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012, below). -**The weights of the X-ray term** against the geometry and against the B restraints start from the -ratios of the gradients, as X-PLOR's crystallographic weight was first set: A. T. Brünger, M. Karplus and -G. A. Petsko, "Crystallographic refinement by simulated annealing: application to crambin" (1989), Acta Cryst. -A45, 50-61 [doi:10.1107/S0108767388009195](https://doi.org/10.1107/S0108767388009195); and they are chosen -among short refinements by R-free with the geometry held within bounds, as phenix.refine chooses them: P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd, -N. W. Moriarty, M. Mustyakimov, T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams, -"Towards automated crystallographic structure refinement with phenix.refine" (2012), Acta Cryst. D68, -352-367 [doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself: +**Weights and R-free** — the weights of the X-ray term against the geometry and B restraints start +from the ratios of the gradients, as X-PLOR's crystallographic weight was first set: A. T. Brünger, +M. Karplus and G. A. Petsko, "Crystallographic refinement by simulated annealing: application to +crambin" (1989), Acta Cryst. A45, 50-61 +[doi:10.1107/S0108767388009195](https://doi.org/10.1107/S0108767388009195); and are chosen among +short refinements by R-free with the geometry held within bounds, as phenix.refine chooses them: +P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd, N. W. Moriarty, M. Mustyakimov, +T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams, "Towards automated crystallographic +structure refinement with phenix.refine" (2012), Acta Cryst. D68, 352-367 +[doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself: A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures" (1992), Nature 355, 472-475 [doi:10.1038/355472a0](https://doi.org/10.1038/355472a0). -**The model the refinement scores** is scaled to the data with the overall scale and anisotropic B fitted by -robust least squares - iteratively reweighted, with the pseudo-Huber weight of each reflection's residual - -so that a few reflections the model cannot explain do not set the scale of the rest: P. J. Huber, "Robust -estimation of a location parameter" (1964), Ann. Math. Statist. 35, 73-101 -[doi:10.1214/aoms/1177703732](https://doi.org/10.1214/aoms/1177703732), in the smooth (pseudo-Huber) form of -P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, "Deterministic edge-preserving regularization in -computed imaging" (1997), IEEE Trans. Image Process. 6, 298-311 +**Robust scaling of the model** — the overall scale and anisotropic B are fitted by iteratively +reweighted least squares with the pseudo-Huber weight. P. J. Huber, "Robust estimation of a location +parameter" (1964), Ann. Math. Statist. 35, 73-101 +[doi:10.1214/aoms/1177703732](https://doi.org/10.1214/aoms/1177703732), in the smooth form of +P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, "Deterministic edge-preserving +regularization in computed imaging" (1997), IEEE Trans. Image Process. 6, 298-311 [doi:10.1109/83.551699](https://doi.org/10.1109/83.551699). -**The bonds of metal ions** to their donor atoms that the model's LINK records name are restrained at the -distances of the Monomer Library's own statistics of metal sites (its `metals.json`, embedded as -`monomers/metals.cif`; per metal, donor element and coordination number), in line with the surveys of -M. M. Harding, "Geometry of metal-ligand interactions in proteins" (2001), Acta Cryst. D57, 401-411 +**Metal-site restraints** — the bonds of metal ions to the donor atoms the model's LINK records name +are restrained at the distances of the Monomer Library's own metal-site statistics (`metals.json`, +embedded as `monomers/metals.cif`), in line with the surveys of M. M. Harding, "Geometry of +metal-ligand interactions in proteins" (2001), Acta Cryst. D57, 401-411 [doi:10.1107/S0907444900019168](https://doi.org/10.1107/S0907444900019168) and "Small revisions to predicted distances around metal sites in proteins" (2006), Acta Cryst. D62, 678-682 -[doi:10.1107/S0907444906014594](https://doi.org/10.1107/S0907444906014594), and, for Mg²⁺, Na⁺ and K⁺ at -nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to nucleobase -nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004 [doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175). +[doi:10.1107/S0907444906014594](https://doi.org/10.1107/S0907444906014594), and, for Mg²⁺, Na⁺ and +K⁺ at nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to +nucleobase nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004 +[doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175). -**The anisotropic displacements**: a TLS group's U is that of a rigid body's translation, libration and their -correlation, U = T + A L A^T + A S + S^T A^T, of V. Schomaker and K. N. Trueblood, "On the rigid-body motion of -molecules in crystals" (1968), Acta Cryst. B24, 63-76 -[doi:10.1107/S0567740868001718](https://doi.org/10.1107/S0567740868001718), refined against the likelihood with a -residual B per atom, and a model's B column read as the total or as the residual to its TLS groups, as in -M. D. Winn, M. N. Isupov and G. N. Murshudov, "Use of TLS parameters to model anisotropic displacements in +**Anisotropic displacements** — the TLS model of a rigid body's translation, libration and their +correlation: V. Schomaker and K. N. Trueblood, "On the rigid-body motion of molecules in crystals" +(1968), Acta Cryst. B24, 63-76 +[doi:10.1107/S0567740868001718](https://doi.org/10.1107/S0567740868001718); refined against the +likelihood with a residual B per atom, the B column read as total or residual, as in M. D. Winn, +M. N. Isupov and G. N. Murshudov, "Use of TLS parameters to model anisotropic displacements in macromolecular refinement" (2001), Acta Cryst. D57, 122-133 -[doi:10.1107/S0907444900014736](https://doi.org/10.1107/S0907444900014736). The chains are cut into TLS groups as -TLSMD cuts them - contiguous segments, by dynamic programming over the fit of an isotropic TLS model to the -refined B factors: J. Painter and E. A. Merritt, "Optimal description of a protein structure in terms of multiple -groups undergoing TLS motion" (2006), Acta Cryst. D62, 439-450 -[doi:10.1107/S0907444906005270](https://doi.org/10.1107/S0907444906005270). Atoms refined with a U of their own are -restrained by the rigid-bond postulate of F. L. Hirshfeld, "Can X-ray data distinguish bonding effects from -vibrational smearing?" (1976), Acta Cryst. A32, 239-244 -[doi:10.1107/S0567739476000533](https://doi.org/10.1107/S0567739476000533), in the form of SHELXL's RIGU - the -components of U along the bond and across it in the bond's plane: A. Thorn, B. Dittrich and G. M. Sheldrick, -"Enhanced rigid-bond restraints" (2012), Acta Cryst. A68, 448-451 -[doi:10.1107/S0108767312014535](https://doi.org/10.1107/S0108767312014535) - with SHELXL's similarity (SIMU) and -approximate-isotropy (ISOR) restraints as G. M. Sheldrick, "Crystal structure refinement with SHELXL" (2015), Acta -Cryst. C71, 3-8 [doi:10.1107/S2053229614024218](https://doi.org/10.1107/S2053229614024218) documents them. Which -displacement model a refinement keeps - isotropic, TLS, or a U per atom - is decided on the free set, as a test of -what the data support rather than a rule by resolution: E. A. Merritt, "To B or not to B: a question of -resolution?" (2012), Acta Cryst. D68, 468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320). +[doi:10.1107/S0907444900014736](https://doi.org/10.1107/S0907444900014736). The chains are cut into +TLS groups as TLSMD cuts them: J. Painter and E. A. Merritt, "Optimal description of a protein +structure in terms of multiple groups undergoing TLS motion" (2006), Acta Cryst. D62, 439-450 +[doi:10.1107/S0907444906005270](https://doi.org/10.1107/S0907444906005270). Atoms with a U of their +own are restrained by the rigid-bond postulate of F. L. Hirshfeld, "Can X-ray data distinguish +bonding effects from vibrational smearing?" (1976), Acta Cryst. A32, 239-244 +[doi:10.1107/S0567739476000533](https://doi.org/10.1107/S0567739476000533), in the form of SHELXL's +RIGU: A. Thorn, B. Dittrich and G. M. Sheldrick, "Enhanced rigid-bond restraints" (2012), Acta +Cryst. A68, 448-451 [doi:10.1107/S0108767312014535](https://doi.org/10.1107/S0108767312014535), +with SHELXL's SIMU and ISOR restraints as G. M. Sheldrick, "Crystal structure refinement with +SHELXL" (2015), Acta Cryst. C71, 3-8 +[doi:10.1107/S2053229614024218](https://doi.org/10.1107/S2053229614024218) documents them. Which +displacement model a refinement keeps is decided on the free set rather than by a rule of +resolution: E. A. Merritt, "To B or not to B: a question of resolution?" (2012), Acta Cryst. D68, +468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320). -**NCS restraints** — copies of a molecule in the model are found by sequence alignment with REFMAC5's criterion -(more than 15 residues aligned, 80% identity - here of the longer chain - and 2.5 Å rms after superposition; -Murshudov et al. 2011, above) and restrained locally, on pairs of -corresponding interatomic distances under the Geman-McClure robust loss, as REFMAC5's local NCS restraints are, with -the distance range of R. A. Nicholls, F. Long and G. N. Murshudov, "Low-resolution refinement tools in REFMAC5" -(2012), Acta Cryst. D68, 404-417 [doi:10.1107/S090744491105606X](https://doi.org/10.1107/S090744491105606X), and -the bonds and angles left out as they leave them out. The case for local restraints over a superposition, and for -deciding them by R-free, is G. J. Kleywegt's, "Use of non-crystallographic symmetry in protein structure refinement" -(1996), Acta Cryst. D52, 842-857 [doi:10.1107/S0907444995016477](https://doi.org/10.1107/S0907444995016477); I. Usón, -E. Pohl, T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz and G. M. Sheldrick, "1.7 Å structure of the stabilized -REIv mutant T39K. Application of local NCS restraints" (1999), Acta Cryst. D55, 1158-1167 -[doi:10.1107/S0907444999003972](https://doi.org/10.1107/S0907444999003972); and J. J. Headd, N. Echols, -P. V. Afonine, N. W. Moriarty, R. J. Gildea and P. D. Adams, "Flexible torsion-angle noncrystallographic symmetry -restraints for improved macromolecular structure refinement" (2014), Acta Cryst. D70, 1346-1356 -[doi:10.1107/S1399004714003277](https://doi.org/10.1107/S1399004714003277), whose comparison of global, torsion and -no NCS is why the copies' B factors are refined independently here. +**NCS restraints** — copies found by sequence alignment with REFMAC5's criterion (Murshudov et al. +2011, above) and restrained locally on pairs of corresponding interatomic distances under the +Geman-McClure loss, as REFMAC5's local NCS restraints are, with the distance range of +R. A. Nicholls, F. Long and G. N. Murshudov, "Low-resolution refinement tools in REFMAC5" (2012), +Acta Cryst. D68, 404-417 [doi:10.1107/S090744491105606X](https://doi.org/10.1107/S090744491105606X). +The case for local restraints over a superposition, decided by R-free: G. J. Kleywegt, "Use of +non-crystallographic symmetry in protein structure refinement" (1996), Acta Cryst. D52, 842-857 +[doi:10.1107/S0907444995016477](https://doi.org/10.1107/S0907444995016477); I. Usón, E. Pohl, +T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz and G. M. Sheldrick, "1.7 Å structure of the +stabilized REIv mutant T39K. Application of local NCS restraints" (1999), Acta Cryst. D55, +1158-1167 [doi:10.1107/S0907444999003972](https://doi.org/10.1107/S0907444999003972); and +J. J. Headd, N. Echols, P. V. Afonine, N. W. Moriarty, R. J. Gildea and P. D. Adams, "Flexible +torsion-angle noncrystallographic symmetry restraints for improved macromolecular structure +refinement" (2014), Acta Cryst. D70, 1346-1356 +[doi:10.1107/S1399004714003277](https://doi.org/10.1107/S1399004714003277), whose comparison is why +the copies' B factors are refined independently. -**Unmodelled density** — the blobs the refinement lists are the connected regions of the difference map above -a cutoff, found by flood fill as Coot's "unmodelled blobs" finds them, in GEMMI's implementation -(`find_blobs_by_flood_fill`): P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, "Features and development -of Coot" (2010), Acta Cryst. D66, 486-501 [doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493). +**Special positions** — an atom on a special position is kept on it, its shifts confined to the +subspace its site symmetry leaves unchanged. R. W. Grosse-Kunstleve and P. D. Adams, "Algorithms for +deriving crystallographic space-group information. II. Treatment of special positions" (2002), Acta +Cryst. A58, 60-65 [doi:10.1107/S0108767301016658](https://doi.org/10.1107/S0108767301016658). -**Special positions** — an atom on a crystallographic special position is placed on it and its shifts are -confined to the subspace its site symmetry leaves unchanged, as R. W. Grosse-Kunstleve and P. D. Adams treat -special positions, "Algorithms for deriving crystallographic space-group information. II. Treatment of special -positions" (2002), Acta Cryst. A58, 60-65 [doi:10.1107/S0108767301016658](https://doi.org/10.1107/S0108767301016658). +### Solvent, ions and unmodelled density -**Water build** — the waters the refinement adds and removes follow the documented ordered-solvent defaults of -phenix.refine: difference-map peaks from 3 σ, hydrogen-bond partners to 3.2 Å and picking only to 2.8 Å, with -the waters added, filtered and refined between macro-cycles: -Afonine et al. (2012, above); -the distances to the model that Coot's Find Waters keeps (2.4-3.2 Å), from the Coot manual and Emsley et al. -(2010, above); and the removal of a water in negative difference density at -3 σ, the cut Echols et al. (2014, -below) use to exclude a site. +**Unmodelled density** — the connected regions of the difference map above a cutoff, found by flood +fill as Coot's "unmodelled blobs" are, in GEMMI's implementation. P. Emsley, B. Lohkamp, W. G. Scott +and K. Cowtan, "Features and development of Coot" (2010), Acta Cryst. D66, 486-501 +[doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493). -**Solvent audit** — the waters and ions of the refined model are scored by their surroundings with the -stereochemical criteria of the surveys of ions at nucleic acids by P. Auffinger's group: the coordination bands -1.9-2.3 / 2.3-2.6 / 2.6-3.2 Å, the Mg²⁺ exclusion zone and the clash shells of N. Naleem, A. Henning-Knechtel, -S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and -annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120 -[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); the Mg-O and Mg-N distances and the -2.3-3.4 Å exclusion zone of F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar and P. Auffinger, -"Principles of ion binding to RNA inferred from the analysis of a 1.55 Å resolution bacterial ribosome -structure – Part I: Mg2+" (2025), Nucleic Acids Res. 53, gkae1148 -[doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); the Na⁺, K⁺ and water distances and -coordination numbers of F. Leonarski, L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor -Mg2+ inner-sphere binders but excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192 -[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118), and the distance bands of their -2017 survey (above). The ligand sphere of the bond-valence sums and the reading of halides by the donors -around them follow N. Echols, N. Morshed, P. V. Afonine, A. J. McCoy, M. D. Miller, R. J. Read, -J. S. Richardson, T. C. Terwilliger and P. D. Adams, "Automated identification of elemental ions in -macromolecular crystal structures" (2014), Acta Cryst. D70, 1104-1114 -[doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308), and the halide's positive and -negative neighbours N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental -ion identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158 -[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241); the chloride's shell of donors -at up to 3.4 Å is O. Carugo's, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC -Struct. Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8). The -bond-valence sums are I. D. Brown and D. Altermatt's, "Bond-valence parameters obtained from a systematic -analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247 -[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of N. E. Brese and -M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197 -[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by H. Zheng, -K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, "Data mining of -iron(II) and iron(III) bond-valence parameters, and their relevance for macromolecular crystallography" -(2017), Acta Cryst. D73, 316-325 [doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584), -weighted by occupancy as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method -able to identify metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37 -[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); their normalised vector sum and -its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski, I. G. Shabalin, K. B. Handing -and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation tool" (2017), Acta Cryst. D73, 223-233 -[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061). The score that combines them is -our own. The divalent ion's place on a purine N7 - in the base's plane, 3.5-3.9 Å from its O6 or N6 - is the test of -Leonarski et al. (2017, above). A light ion is told from a water by a density above the waters', as Echols et al. -(2014, above) flag one; a halide's few or no donors are allowed as Z. Dauter and M. Dauter describe halides binding -also at hydrophobic surfaces, "Entering a new phase: using solvent halide ions in protein structure determination" -(2001), Structure 9, R21-R26 [doi:10.1016/S0969-2126(01)00565-2](https://doi.org/10.1016/S0969-2126(01)00565-2). -The anomalous scattering expected of each species is f″ at the data's wavelength by D. T. Cromer and D. A. -Liberman, "Anomalous dispersion calculations near to and on the long-wavelength side of an absorption edge" -(1981), Acta Cryst. A37, 267-268 [doi:10.1107/S0567739481000600](https://doi.org/10.1107/S0567739481000600), -with the corrections of L. Kissel and R. H. Pratt, "Corrections to tabulated anomalous-scattering factors" -(1990), Acta Cryst. A46, 170-175 [doi:10.1107/S0108767389010718](https://doi.org/10.1107/S0108767389010718), as -GEMMI computes them (its port of the Larch routines). +**Water build** — the documented ordered-solvent defaults of phenix.refine (difference-map peaks +from 3 σ, picking to 2.8 Å, waters added, filtered and refined between macro-cycles; Afonine et al. +2012, above); the distance band to the model, widened a little from the 2.4-3.2 Å Coot's Find Waters +keeps (Emsley et al. 2010, above); and the removal of a water in negative difference density at +-3 σ (Echols et al. 2014, below). + +**Ion stereochemistry** — the coordination bands, the Mg²⁺ exclusion zone and the clash shells of +the surveys of ions at nucleic acids by P. Auffinger's group. N. Naleem, A. Henning-Knechtel, +S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, +diagnosing, and annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120 +[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); F. Leonarski, A. Henning-Knechtel, +S. Kirmizialtin, E. Ennifar and P. Auffinger, "Principles of ion binding to RNA inferred from the +analysis of a 1.55 Å resolution bacterial ribosome structure – Part I: Mg2+" (2025), Nucleic Acids +Res. 53, gkae1148 [doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); F. Leonarski, +L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor Mg2+ inner-sphere binders but +excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192 +[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118); and their 2017 survey (above). +The score that combines the criteria is rugnux's own. + +**Ion identification** — the ligand sphere of the bond-valence sums, the reading of halides by their +donors, and telling a light ion from a water by its density: N. Echols, N. Morshed, P. V. Afonine, +A. J. McCoy, M. D. Miller, R. J. Read, J. S. Richardson, T. C. Terwilliger and P. D. Adams, +"Automated identification of elemental ions in macromolecular crystal structures" (2014), Acta Cryst. +D70, 1104-1114 [doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308); +N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental ion +identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158 +[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241). The chloride's shell of +donors: O. Carugo, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC Struct. +Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8); halides at +hydrophobic surfaces: Z. Dauter and M. Dauter, "Entering a new phase: using solvent halide ions in +protein structure determination" (2001), Structure 9, R21-R26 +[doi:10.1016/S0969-2126(01)00565-2](https://doi.org/10.1016/S0969-2126(01)00565-2). + +**Bond-valence sums** — I. D. Brown and D. Altermatt, "Bond-valence parameters obtained from a +systematic analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247 +[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of +N. E. Brese and M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197 +[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by +H. Zheng, K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, +"Data mining of iron(II) and iron(III) bond-valence parameters, and their relevance for +macromolecular crystallography" (2017), Acta Cryst. D73, 316-325 +[doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584); weighted by occupancy +as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method able to identify +metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37 +[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); the normalised vector +sum and its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski, +I. G. Shabalin, K. B. Handing and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation +tool" (2017), Acta Cryst. D73, 223-233 +[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061). + +**Anomalous scattering factors** — f″ at the data's wavelength, as GEMMI computes it (its port of +the Larch routines). D. T. Cromer and D. A. Liberman, "Anomalous dispersion calculations near to +and on the long-wavelength side of an absorption edge" (1981), Acta Cryst. A37, 267-268 +[doi:10.1107/S0567739481000600](https://doi.org/10.1107/S0567739481000600), with the corrections of +L. Kissel and R. H. Pratt, "Corrections to tabulated anomalous-scattering factors" (1990), Acta +Cryst. A46, 170-175 [doi:10.1107/S0108767389010718](https://doi.org/10.1107/S0108767389010718). ## Software and computing methods -Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading -the result, follows Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4 -data", HDF5 User Group meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders). -The CUDA kernels in Jungfraujoch are its own, but the approach is his. +**GPU decompression** — decoding bitshuffle+LZ4 images on the GPU rather than on the host follows +Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4 data", HDF5 User Group +meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders). The CUDA kernels +are Jungfraujoch's own; the approach is his. -Removing the small islands of solvent from the bulk-solvent mask on the GPU labels the connected -components with a parallel union-find, following D. P. Playne and K. Hawick, "A New Algorithm for -Parallel Connected-Component Labelling on GPUs" (2018), IEEE Trans. Parallel Distrib. Syst. 29, -1217-1230 [doi:10.1109/TPDS.2018.2799216](https://doi.org/10.1109/TPDS.2018.2799216). +**Parallel connected components** — removing the islands of the bulk-solvent mask on the GPU with a +parallel union-find. D. P. Playne and K. Hawick, "A New Algorithm for Parallel Connected-Component +Labelling on GPUs" (2018), IEEE Trans. Parallel Distrib. Syst. 29, 1217-1230 +[doi:10.1109/TPDS.2018.2799216](https://doi.org/10.1109/TPDS.2018.2799216). -The FFT indexer's GPU peak search divides by a fixed window width with a reciprocal multiplication -corrected once by the exact remainder, which returns the correctly rounded quotient, following -P. W. Markstein, "Computation of elementary functions on the IBM RISC System/6000 processor" -(1990), IBM J. Res. Dev. 34, 111-119. +**Correctly rounded integer division** — the FFT indexer's GPU peak search divides by a reciprocal +multiplication corrected once by the exact remainder. P. W. Markstein, "Computation of elementary +functions on the IBM RISC System/6000 processor" (1990), IBM J. Res. Dev. 34, 111-119. -The per-frame scaling loops of the rotation merge are accelerated by Anderson acceleration in the -type-II form analysed by Walker and Ni: H. F. Walker and P. Ni, "Anderson acceleration for fixed-point -iterations" (2011), SIAM J. Numer. Anal. 49, 1715-1735 -[doi:10.1137/10078356X](https://doi.org/10.1137/10078356X), after D. G. Anderson, "Iterative -procedures for nonlinear integral equations" (1965), J. ACM 12, 547-560 +**Anderson acceleration** — of the per-frame scaling loops of the rotation merge, in the type-II +form. H. F. Walker and P. Ni, "Anderson acceleration for fixed-point iterations" (2011), SIAM J. +Numer. Anal. 49, 1715-1735 [doi:10.1137/10078356X](https://doi.org/10.1137/10078356X), after +D. G. Anderson, "Iterative procedures for nonlinear integral equations" (1965), J. ACM 12, 547-560 [doi:10.1145/321296.321305](https://doi.org/10.1145/321296.321305). -The interval the report gives for ISa comes from a bootstrap over the unique reflections: -B. Efron, "Bootstrap methods: another look at the jackknife" (1979), Ann. Statist. 7, 1-26 +**Bootstrap** — the interval reported for ISa. B. Efron, "Bootstrap methods: another look at the +jackknife" (1979), Ann. Statist. 7, 1-26 [doi:10.1214/aos/1176344552](https://doi.org/10.1214/aos/1176344552). Its draws are Poisson weights -hashed from the replicate and the reflection with the SplitMix64 finaliser of G. L. Steele Jr., -D. Lea and C. H. Flood, "Fast splittable pseudorandom number generators" (2014), Proc. OOPSLA 2014, -453-472 [doi:10.1145/2660193.2660195](https://doi.org/10.1145/2660193.2660195). +hashed with the SplitMix64 finaliser of G. L. Steele Jr., D. Lea and C. H. Flood, "Fast splittable +pseudorandom number generators" (2014), Proc. OOPSLA 2014, 453-472 +[doi:10.1145/2660193.2660195](https://doi.org/10.1145/2660193.2660195). -The single-precision exp the Bragg integration's profile fit shares between the CPU and an Apple GPU -(`BraggExp.h`) is built from textbook error-free floating-point arithmetic: the argument reduction of -W. J. Cody and W. Waite, "Software Manual for the Elementary Functions" (1980), Prentice-Hall; the -exact product (with Veltkamp's split) and Fast2Sum of T. J. Dekker, "A floating-point technique for -extending the available precision" (1971), Numer. Math. 18, 224-242; and the 2Sum of D. E. Knuth, -"The Art of Computer Programming, Vol. 2: Seminumerical Algorithms", section 4.2.2. +**Error-free floating-point arithmetic** — the single-precision exp shared between the CPU and an +Apple GPU (`BraggExp.h`): the argument reduction of W. J. Cody and W. Waite, "Software Manual for the +Elementary Functions" (1980), Prentice-Hall; the exact product (with Veltkamp's split) and Fast2Sum +of T. J. Dekker, "A floating-point technique for extending the available precision" (1971), Numer. +Math. 18, 224-242; and the 2Sum of D. E. Knuth, "The Art of Computer Programming, Vol. 2: +Seminumerical Algorithms", section 4.2.2. -This software uses the Viridis, Magma and Inferno colormaps from Matplotlib under its -BSD-compatible license. J. D. Hunter, "Matplotlib: A 2D graphics environment" (2007), Comput. Sci. -Eng. 9, 90-95 [doi:10.1109/MCSE.2007.55](https://doi.org/10.1109/MCSE.2007.55). +**Colormaps** — Viridis, Magma and Inferno from Matplotlib, under its BSD-compatible license. +J. D. Hunter, "Matplotlib: A 2D graphics environment" (2007), Comput. Sci. Eng. 9, 90-95 +[doi:10.1109/MCSE.2007.55](https://doi.org/10.1109/MCSE.2007.55). -The Mol* structure viewer of the web frontend follows the setup Guillaume Gotthard (PSI) tuned for -[AareDB](https://gitea.psi.ch/mx/aaredb), the sample-management and data-pipeline system of the PSI MX -beamlines: its Coot-style mouse controls and trackball, the model representations and colours, and the -display of the density maps (carved wireframe meshes at absolute levels from the whole cell's -statistics, contour-level defaults and colours). +**Mol\* viewer setup** — the web frontend's structure viewer follows the setup Guillaume Gotthard +(PSI) tuned for [AareDB](https://gitea.psi.ch/mx/aaredb), the sample-management and data-pipeline +system of the PSI MX beamlines: the Coot-style mouse controls, the model representations and +colours, and the display of the density maps. ## File formats read from a published specification -**CBF / imgCIF** - the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression scheme -and reads the imgCIF `_axis` table (the laboratory directions of the image's fast and slow pixel -directions, of the goniometer axes and of a 2theta arm) from the specification alone; no CBFlib or -other CBF code is used, so there is no licence obligation, only this credit. +**CBF / imgCIF** — the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression and the +imgCIF `_axis` table from the specification alone; no CBFlib or other CBF code is used. H. J. Bernstein and A. P. Hammersley, "Specification of the Crystallographic Binary File (CBF/imgCIF)" (2006), International Tables for Crystallography Vol. G, 37-43 [doi:10.1107/97809553602060000729](https://doi.org/10.1107/97809553602060000729); @@ -860,12 +780,10 @@ A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, "Image dictionary (imgCIF International Tables for Crystallography Vol. G, 444-458 [doi:10.1107/97809553602060000746](https://doi.org/10.1107/97809553602060000746). -**d\*TREK SMV** - the SMV reader reads the d\*TREK header vocabulary written by Rigaku's -CrystalClear (Saturn and R-AXIS detectors: detector and spatial-distortion vectors, detector -circles, 2theta arm, encoded overflows) from the headers themselves, interpreting the detector -vectors as dxtbx does for these detectors; no d\*TREK or dxtbx code is used. -J. W. Pflugrath, "The finer things in X-ray diffraction data collection" (1999), Acta Cryst. D55, -1718-1725 [doi:10.1107/S090744499900935X](https://doi.org/10.1107/S090744499900935X); +**d\*TREK SMV** — the SMV reader reads the d\*TREK header vocabulary of Rigaku's CrystalClear from +the headers themselves, interpreting the detector vectors as dxtbx does; no d\*TREK or dxtbx code is +used. J. W. Pflugrath, "The finer things in X-ray diffraction data collection" (1999), Acta Cryst. +D55, 1718-1725 [doi:10.1107/S090744499900935X](https://doi.org/10.1107/S090744499900935X); [dxtbx](https://github.com/cctbx/dxtbx): J. M. Parkhurst, A. S. Brewster, L. Fuentes-Montero, D. G. Waterman et al., "dxtbx: the diffraction experiment toolbox" (2014), J. Appl. Cryst. 47, 1459-1465 [doi:10.1107/S1600576714011996](https://doi.org/10.1107/S1600576714011996). @@ -874,18 +792,17 @@ D. G. Waterman et al., "dxtbx: the diffraction experiment toolbox" (2014), J. Ap In addition to in-house datasets collected at SLS 2.0, Jungfraujoch is tested against public diffraction data collected on other people's beamlines, on detectors and in file formats we do not -produce ourselves - most of it at other facilities, a few sets at the Swiss Light Source but not by -this system. That data was collected and published by other people. Every dataset used, the DOI to -cite for it, and the deposition it belongs to are listed in -[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md); we thank the depositors, and the repositories that make -the data findable and citable. +produce ourselves. Every dataset used, the DOI to cite for it, and the deposition it belongs to are +listed in [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) and +[EXTERNAL_TEST_DATA_STILLS](EXTERNAL_TEST_DATA_STILLS.md); we thank the depositors, and the +repositories that make the data findable and citable. **[IRRMC](https://proteindiffraction.org/)**, the Integrated Resource for Reproducibility in -Macromolecular Crystallography (Minor lab, University of Virginia), is the source of most of them. -IRRMC releases its data under CC0 and asks that the DOI of the dataset be -cited. M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng, -D. R. Cooper, M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of -macromolecular diffraction experiments" (2016), Acta Cryst. D72, 1181-1193 +Macromolecular Crystallography (Minor lab, University of Virginia), is the source of the largest +share of them; it releases its data under CC0 and asks that the DOI of the dataset be cited. +M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng, D. R. Cooper, +M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of macromolecular +diffraction experiments" (2016), Acta Cryst. D72, 1181-1193 [doi:10.1107/S2059798316014716](https://doi.org/10.1107/S2059798316014716); M. Grabowski, M. Cymborowski, P. J. Porebski, T. Osinski, I. G. Shabalin, D. R. Cooper and W. Minor, "The Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first @@ -893,46 +810,45 @@ four years" (2019), Struct. Dyn. 6, 064301 [doi:10.1063/1.5128672](https://doi.org/10.1063/1.5128672). **[SBGrid Data Bank](https://data.sbgrid.org/)**, the structural biology community's data -publication service (SBGrid Consortium, Harvard Medical School). P. A. Meyer, -S. Socias, J. Key, E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the -structural biology data grid supports live analysis" (2016), Nat. Commun. 7, 10882 +publication service (SBGrid Consortium, Harvard Medical School). P. A. Meyer, S. Socias, J. Key, +E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the structural biology data grid +supports live analysis" (2016), Nat. Commun. 7, 10882 [doi:10.1038/ncomms10882](https://doi.org/10.1038/ncomms10882). **[Zenodo](https://zenodo.org/)**, CERN's open repository, hosts datasets deposited there directly by the groups that collected them. European Organization for Nuclear Research and OpenAIRE, -"Zenodo" (2013), CERN [doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). Some of those -deposits are described in IUCrData Raw Data Letters; the letters are cited on the -[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page, beside the datasets they describe. +"Zenodo" (2013), CERN [doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). The IUCrData +Raw Data Letters describing some of those deposits are cited on the +[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page. **[CXIDB](https://cxidb.org/)**, the Coherent X-ray Imaging Data Bank, is the source of the serial -(XFEL stills) datasets of the still battery, listed with their DOIs on -[EXTERNAL_TEST_DATA_STILLS](EXTERNAL_TEST_DATA_STILLS.md). F. R. N. C. Maia, "The Coherent X-ray -Imaging Data Bank" (2012), Nat. Methods 9, 854-855 -[doi:10.1038/nmeth.2110](https://doi.org/10.1038/nmeth.2110). +(XFEL stills) datasets. F. R. N. C. Maia, "The Coherent X-ray Imaging Data Bank" (2012), Nat. +Methods 9, 854-855 [doi:10.1038/nmeth.2110](https://doi.org/10.1038/nmeth.2110). **[MXRDR](https://mxrdr.icm.edu.pl/)**, the Macromolecular Xtallography Raw Data Repository (ICM, University of Warsaw), releases its data under CC0 and asks that the DOI of the dataset be cited. -**[XRDa](https://xrda.pdbj.org/)**, the Xtal Raw Data Archive (Protein Data Bank Japan), which -publishes raw diffraction images - X-ray, electron and neutron - and mints a DOI for each; it asks -that the DOI of the dataset be cited. It has no canonical citation paper. +**[XRDa](https://xrda.pdbj.org/)**, the Xtal Raw Data Archive (Protein Data Bank Japan), publishes +raw diffraction images and mints a DOI for each, which it asks to be cited. It has no canonical +citation paper. -**The [ESRF data portal](https://data.esrf.fr/)**, through which the European Synchrotron -publishes raw data under its data policy (CC BY 4.0, with the dataset DOI to be cited). -R. Dimper, A. Götz, A. De Maria, V. A. Solé, M. Chaillet and B. Lebayle, "ESRF Data Policy, -Storage, and Services" (2019), Synchrotron Rad. News 32, 7-12 +**The [ESRF data portal](https://data.esrf.fr/)**, through which the European Synchrotron publishes +raw data under its data policy (CC BY 4.0, with the dataset DOI to be cited). R. Dimper, A. Götz, +A. De Maria, V. A. Solé, M. Chaillet and B. Lebayle, "ESRF Data Policy, Storage, and Services" +(2019), Synchrotron Rad. News 32, 7-12 [doi:10.1080/08940886.2019.1608119](https://doi.org/10.1080/08940886.2019.1608119). **The [Keele University research data repository](https://researchdata.keele.ac.uk/)** and -**[UQ eSpace](https://espace.library.uq.edu.au/)** (The University of Queensland), which host the -raw images of datasets deposited there by the groups that collected them; the dataset DOIs are -cited on the [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page. +**[UQ eSpace](https://espace.library.uq.edu.au/)** (The University of Queensland) host the raw +images of datasets deposited there by the groups that collected them; the dataset DOIs are cited on +the [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page. -The beamline, resolution, space group and unit cell quoted for each dataset are the values -deposited with the corresponding PDB entry, read from the RCSB PDB data API. H. M. Berman, -J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and P. E. Bourne, -"The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242 +**The [Protein Data Bank](https://www.rcsb.org/)** supplies, through the RCSB PDB data API, the +beamline, resolution, space group and unit cell quoted for each dataset, and the deposited +coordinates and structure factors of the public entries the refinement battery starts from. +H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and +P. E. Bourne, "The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242 [doi:10.1093/nar/28.1.235](https://doi.org/10.1093/nar/28.1.235). **The [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/)** (EMBL-EBI and Google @@ -940,14 +856,14 @@ DeepMind) supplies the start models of the battery's timing benchmark: the predi deposited entities of 9ZLO (AF-P17090-F1), 9S02 (AF-P32322-F1) and 9GJX (AF-Q65MG6-F1) and, for the in-house benchmark, of horse myoglobin (AF-P68082-F1) and bovine cytochrome c (AF-P62894-F1), model version 6, with their predicted aligned error, kept unmodified in `tools/battery/models/` under -CC BY 4.0. J. Jumper, R. Evans, -A. Pritzel et al., "Highly accurate protein structure prediction with AlphaFold" (2021), Nature 596, -583-589 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); M. Varadi, +CC BY 4.0. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction +with AlphaFold" (2021), Nature 596, 583-589 +[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); M. Varadi, S. Anyango, M. Deshpande et al., "AlphaFold Protein Structure Database: massively expanding the -structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res. 50, -D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061); M. Varadi, D. Bertoni, -P. Magana et al., "AlphaFold Protein Structure Database in 2024: providing structure coverage for over -214 million protein sequences" (2024), Nucleic Acids Res. 52, D368-D375 +structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res. +50, D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061); M. Varadi, +D. Bertoni, P. Magana et al., "AlphaFold Protein Structure Database in 2024: providing structure +coverage for over 214 million protein sequences" (2024), Nucleic Acids Res. 52, D368-D375 [doi:10.1093/nar/gkad1011](https://doi.org/10.1093/nar/gkad1011). The large case of both benchmarks is the deposited model of PDB entry 4NWV, moved away from its place. From 575846349ee2790661295a576f54673db88a9df6 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 13 Nov 2026 11:46:28 +0100 Subject: [PATCH 3/3] docs: every credit found in the code is on the acknowledgement page, and the reverse MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third pass on docs/ACKNOWLEDGEMENT.md. Entries added for the works the code credits but the page did not: the Cornish-Fisher expansion behind the Poisson spot threshold; Grosse-Kunstleve (1999) for the step from Le Page's two-folds to a conventional cell; the silver behenate cell (Huang et al. 1993) and the NIST SRM 674b / SRM 640 calibrants; the GPU spot finder by H.-C. Stadler; phenix.xtriage as the model of the twinning test (with the 2019 Phenix paper); Phaser's default for declaring a pseudo-translation; Sultana et al. (2007) for the six-domain twin; Luzzati's D in the likelihood of a placement; the Oren-Luenberger / Shanno-Phua self-scaling of the L-BFGS initial Hessian; the riding-hydrogen model (Liebschner et al. 2020) and the QCP superposition (Theobald 2005; Liu et al. 2010), both through GEMMI; the PILATUS count-rate clipping (Trueb et al. 2015) the CBF reader relies on; Mol* itself (Sehnal et al. 2021) and the SGI trackball port behind the viewer; the Crystallography Open Database (Gražulis et al. 2009, 2012) as the source of the small-molecule reference cells; and the HKL-2000 manual's wording for a weak stretch of a sweep, now in an entry of its own beside autoPROC's. Every DOI on the page, the 151 that were there and the 15 added, was checked against Crossref (title, journal and year read back) or, for the two DataCite ones, doi.org; all resolve. The same papers go into the References list of docs/CPU_DATA_ANALYSIS.md where that page describes the method. The reverse direction: works the page credits that had no comment at the algorithm now carry the one-line credit there - Wilson (1949) at the centre-of-symmetry statistics; Sheldrick (2015) for SIMU/ISOR; Lunin, Afonine & Urzhumtsev (2002) at the likelihood parameters; CrystFEL's three-ring region at the CPU and GPU stencils; MOSFLM's post-refinement practice; Kleywegt / Uson / Headd at the NCS decision; Herbst-Irmer & Sheldrick (1998) at the twin-fraction fit; Jumper / Varadi at the PAE domains; Hubbell & Seltzer's database name; Matplotlib at the colormaps; PRIME / Sauter at the stills post-refinement. Comments only, no code change. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK --- common/ColorScale.h | 1 + docs/ACKNOWLEDGEMENT.md | 99 ++++++++++++++++--- docs/CPU_DATA_ANALYSIS.md | 10 ++ image_analysis/SensorAbsorption.h | 3 +- .../BraggIntegrationEngineCPU.cpp | 3 +- .../BraggIntegrationEngineGPU.cu | 3 +- image_analysis/geom_refinement/PostRefine.h | 2 + .../scale_merge/CentreOfSymmetry.cpp | 3 +- .../scale_merge/StillsPartialityRefine.h | 1 + .../MolecularReplacement.h | 2 + .../structure_refinement/Refinement.cpp | 3 + .../structure_refinement/RefinementADP.h | 3 +- .../structure_refinement/RefinementXray.h | 6 +- 13 files changed, 118 insertions(+), 21 deletions(-) diff --git a/common/ColorScale.h b/common/ColorScale.h index 0daac58b2..501f7ab53 100644 --- a/common/ColorScale.h +++ b/common/ColorScale.h @@ -34,6 +34,7 @@ enum class ColorScaleSpecial { }; class ColorScale { + // Viridis, Magma and Inferno are Matplotlib's colormaps, sampled. Following Hunter (2007) Comput. Sci. Eng. 9, 90-95 const std::vector viridis_colormap = { {68, 1, 84}, {71, 44, 123}, {59, 81, 139}, {44, 113, 142}, {33, 144, 141}, {39, 173, 129}, {92, 200, 99}, {170, 220, 50}, {253, 231, 37} diff --git a/docs/ACKNOWLEDGEMENT.md b/docs/ACKNOWLEDGEMENT.md index e6b87380b..0a8ddab27 100644 --- a/docs/ACKNOWLEDGEMENT.md +++ b/docs/ACKNOWLEDGEMENT.md @@ -38,9 +38,7 @@ obligation, recorded in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). Vonrhein, that a pipeline does not take the experimental metadata it is given (beam centre, detector geometry, goniometer axis) on trust but verifies it against the images and corrects it where it is wrong, and with it that the beam centre is found from the images; the aim of a pipeline that takes the -decisions an expert would take and reports the evidence for each; and the wording "loss of centring" -(autoPROC's "loss of centring during crystal rotation") for a weak stretch of a sweep. C. Vonrhein, -C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, "Data +decisions an expert would take and reports the evidence for each. C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, "Data processing and analysis with the autoPROC toolbox" (2011), Acta Cryst. D67, 293-302 [doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773). @@ -66,6 +64,12 @@ of K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice const expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, 644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933). +**Powder calibrants** — the silver behenate cell is T. C. Huang, H. Toraya, T. N. Blanton and Y. Wu, +"X-ray powder diffraction analysis of silver behenate, a possible low-angle diffraction standard" +(1993), J. Appl. Cryst. 26, 180-184 +[doi:10.1107/S0021889892009762](https://doi.org/10.1107/S0021889892009762); the CeO2 and silicon +cells are those of the NIST standard reference materials SRM 674b and SRM 640 (certificates, no DOI). + ### Spot finding **[CrystFEL](https://www.desy.de/~twhite/crystfel/)** — spot finding, the three-ring integration @@ -80,6 +84,13 @@ T. A. White and H. N. Chapman, "Cheetah: software for high-throughput reduction serial femtosecond X-ray diffraction data" (2014), J. Appl. Cryst. 47, 1118-1131 [doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626). +**Poisson significance threshold** — beyond the summation limit the Poisson quantile of the adaptive +spot-finding threshold is taken from the Cornish-Fisher expansion. E. A. Cornish and R. A. Fisher, +"Moments and cumulants in the specification of distributions" (1938), Rev. Int. Stat. Inst. 5, +307-320 [doi:10.2307/1400905](https://doi.org/10.2307/1400905). + +**GPU spot finder** — the CUDA spot finder was developed by Hans-Christian Stadler (PSI). + **[traccc](https://github.com/acts-project/traccc)** — the sparse connected-component labelling that groups strong pixels into spots: the CPU extractor adapts its SparseCCL source, the CUDA extractor follows the design of its GPU counterpart (MPL-2.0; see @@ -155,7 +166,10 @@ stable algorithms for the computation of reduced unit cells" (2004), Acta Cryst. conventional axes derived from the rotation group found; the two-fold search is used through GEMMI. Y. Le Page, "The derivation of the axes of the conventional unit cell from the dimensions of the Buerger-reduced cell" (1982), J. Appl. Cryst. 15, 255-259 -[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959). +[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959). The step from the +two-folds found to a conventional cell, a centring and an integral change of basis follows +R. W. Grosse-Kunstleve, "Algorithms for deriving crystallographic space-group information" (1999), +Acta Cryst. A55, 383-395 [doi:10.1107/S0108767398010186](https://doi.org/10.1107/S0108767398010186). ### Integration and rotation geometry @@ -214,7 +228,13 @@ reflections against their untwinned and perfect-twin values; its robustness to p kept by the partner steps, which are restricted when a non-half-integer pseudo-translation is found. J. E. Padilla and T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, -1124-1130 [doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). +1124-1130 [doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). The +test's design - indicators on shell-normalised merged intensities, a twin law admitted within +3 degrees of obliquity - follows phenix.xtriage: Zwart, Grosse-Kunstleve and Adams (2005, cited under +[Data quality and anisotropy](#data-quality-and-anisotropy)); D. Liebschner, P. V. Afonine, +M. L. Baker et al., "Macromolecular structure determination using X-rays, neutrons and electrons: +recent developments in Phenix" (2019), Acta Cryst. D75, 861-877 +[doi:10.1107/S2059798319011471](https://doi.org/10.1107/S2059798319011471). **Twin fraction and twinned intensity** — the model (1−α) I(h) + α I(Th), and the fraction read from the median of H = |I(h) − I(Th)|/(I(h) + I(Th)), uniform on [0, 1−2α]. T. O. Yeates, "Detecting @@ -230,7 +250,12 @@ F. Long and A. A. Vagin, "REFMAC5 for the refinement of macromolecular crystal s Acta Cryst. D67, 355-367 [doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314); P. Roversi, E. Blanc, S. Johnson and S. M. Lea, "Tetartohedral twinning could happen to you too" (2012), Acta Cryst. D68, 418-424 -[doi:10.1107/S0907444912006737](https://doi.org/10.1107/S0907444912006737). A twin fraction as a +[doi:10.1107/S0907444912006737](https://doi.org/10.1107/S0907444912006737); the six-domain case of +a monoclinic crystal on a hexagonal metric is K. A. Sultana, I. Alexeev, I. Kursula, P. Mäntsälä, +J. Niemi and G. Schneider, "Structure determination by multiwavelength anomalous diffraction of +aclacinomycin oxidoreductase: indications of multidomain pseudomerohedral twinning" (2007), Acta +Cryst. D63, 149-159 [doi:10.1107/S0907444906044271](https://doi.org/10.1107/S0907444906044271). +A twin fraction as a parameter refined beside the model is SHELXL's BASF (rugnux re-fits it between cycles rather than in the same least squares): R. Herbst-Irmer and G. M. Sheldrick, "Refinement of twinned structures with SHELXL97" (1998), Acta Cryst. B54, 443-449 @@ -264,7 +289,9 @@ axial systematic-absence test also scores against. The per-dataset permutation n the per-row estimate of the modulation depth, and its averaging over the lattice's rotations are rugnux's own. R. J. Read, P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of translational noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183 -[doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374). +[doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374). A pseudo-translation +is declared at a Patterson peak of a fifth of the origin or more, Phaser's default (McCoy et al. +2007, cited under [Model placement and molecular replacement](#model-placement-and-molecular-replacement)). ### Scaling and merging @@ -333,6 +360,12 @@ J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in t package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399 [doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198). +**Reasons for a weak stretch of a sweep** — named in the field's own words: "loss of centring" after +autoPROC's "loss of centring during crystal rotation" (cited under +[Experimental geometry and physical corrections](#experimental-geometry-and-physical-corrections)), +and "crystal rotating out of the beam" after the HKL-2000 manual (HKL: Otwinowski and Minor 1997, +cited under [Integration and rotation geometry](#integration-and-rotation-geometry)). + **Several sweeps merged as one** — each sweep's partials combined into fulls on their own, then the fulls of all sweeps scaled together with per-sweep frame scales and a relative B, as XSCALE merges data sets (W. Kabsch, "Integration, scaling, space-group assignment and post-refinement" (2010), @@ -519,8 +552,10 @@ sets of vectors" (1976), Acta Cryst. A32, 922-923 [doi:10.1107/S0567739476001873](https://doi.org/10.1107/S0567739476001873). **Likelihood scoring of placements** — the log-likelihood gain from the Rice and Woolfson -distributions of the normalised amplitudes given the model's, with sigma_A from the model's expected -coordinate error: R. J. Read, "Pushing the boundaries of molecular replacement with maximum +distributions of the normalised amplitudes given the model's, with sigma_A falling off with resolution +for the model's expected coordinate error as Luzzati's D: V. Luzzati, "Traitement statistique des +erreurs dans la détermination des structures cristallines" (1952), Acta Cryst. 5, 802-810 +[doi:10.1107/S0365110X52002161](https://doi.org/10.1107/S0365110X52002161); R. J. Read, "Pushing the boundaries of molecular replacement with maximum likelihood" (2001), Acta Cryst. D57, 1373-1382 [doi:10.1107/S0907444901012471](https://doi.org/10.1107/S0907444901012471); the gain read as a Z-score over the same orientation at other positions, as the likelihood-enhanced translation @@ -563,7 +598,12 @@ Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S056 memory BFGS method for large scale optimization" (1989), Math. Program. 45, 503-528 [doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116); its two-loop recursion: J. Nocedal, "Updating quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782 -[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). +[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). The +initial inverse Hessian is rescaled at each iteration by the latest pair, the self-scaling of +S. S. Oren and D. G. Luenberger, "Self-scaling variable metric (SSVM) algorithms. Part I" (1974), +Management Sci. 20, 845-862 [doi:10.1287/mnsc.20.5.845](https://doi.org/10.1287/mnsc.20.5.845), and +D. F. Shanno and K.-H. Phua, "Matrix conditioning and nonlinear optimization" (1978), Math. Program. +14, 149-160 [doi:10.1007/BF01588962](https://doi.org/10.1007/BF01588962). **Geometry restraints** — those of the CCP4 Monomer Library, a subset of which rugnux carries (see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). A. A. Vagin, R. A. Steiner, A. A. Lebedev, @@ -577,6 +617,12 @@ A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles an angles in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141 [doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T](https://doi.org/10.1002/(SICI)1096-987X(19960715)17:9%3C1132::AID-JCC5%3E3.0.CO;2-T). +**Riding hydrogens** — placed on their parent atoms by GEMMI's riding-hydrogen code, which follows +D. Liebschner, P. V. Afonine, A. G. Urzhumtsev and P. D. Adams, "Implementation of the riding +hydrogen model in CCTBX to support the next generation of X-ray and neutron joint refinement in +Phenix" (2020), Methods Enzymol. 634, 177-199 +[doi:10.1016/bs.mie.2020.01.007](https://doi.org/10.1016/bs.mie.2020.01.007). + **B-factor restraints** — bonded atoms, and atoms in contact, restrained to each other in the Kullback-Leibler form REFMAC5 uses (Murshudov et al. 2011, above); the divergence itself is S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22, @@ -643,6 +689,12 @@ resolution: E. A. Merritt, "To B or not to B: a question of resolution?" (2012), Geman-McClure loss, as REFMAC5's local NCS restraints are, with the distance range of R. A. Nicholls, F. Long and G. N. Murshudov, "Low-resolution refinement tools in REFMAC5" (2012), Acta Cryst. D68, 404-417 [doi:10.1107/S090744491105606X](https://doi.org/10.1107/S090744491105606X). +The superposition behind the rms criterion is GEMMI's QCP: D. L. Theobald, "Rapid calculation of +RMSDs using a quaternion-based characteristic polynomial" (2005), Acta Cryst. A61, 478-480 +[doi:10.1107/S0108767305015266](https://doi.org/10.1107/S0108767305015266); P. Liu, D. K. Agrafiotis +and D. L. Theobald, "Fast determination of the optimal rotational matrix for macromolecular +superpositions" (2010), J. Comput. Chem. 31, 1561-1563 +[doi:10.1002/jcc.21439](https://doi.org/10.1002/jcc.21439). The case for local restraints over a superposition, decided by R-free: G. J. Kleywegt, "Use of non-crystallographic symmetry in protein structure refinement" (1996), Acta Cryst. D52, 842-857 [doi:10.1107/S0907444995016477](https://doi.org/10.1107/S0907444995016477); I. Usón, E. Pohl, @@ -764,10 +816,14 @@ Seminumerical Algorithms", section 4.2.2. J. D. Hunter, "Matplotlib: A 2D graphics environment" (2007), Comput. Sci. Eng. 9, 90-95 [doi:10.1109/MCSE.2007.55](https://doi.org/10.1109/MCSE.2007.55). -**Mol\* viewer setup** — the web frontend's structure viewer follows the setup Guillaume Gotthard -(PSI) tuned for [AareDB](https://gitea.psi.ch/mx/aaredb), the sample-management and data-pipeline -system of the PSI MX beamlines: the Coot-style mouse controls, the model representations and -colours, and the display of the density maps. +**[Mol\*](https://molstar.org/)** — the web frontend's structure viewer. D. Sehnal, S. Bittrich, +M. Deshpande, R. Svobodová, K. Berka, V. Bazgier, S. Velankar, S. K. Burley, J. Koča and A. S. Rose, +"Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures" +(2021), Nucleic Acids Res. 49, W431-W437 [doi:10.1093/nar/gkab314](https://doi.org/10.1093/nar/gkab314). +Its setup follows the one Guillaume Gotthard (PSI) tuned for [AareDB](https://gitea.psi.ch/mx/aaredb), +the sample-management and data-pipeline system of the PSI MX beamlines: the Coot-style mouse +controls, the model representations and colours, and the display of the density maps. The virtual +trackball is a port of Coot's, itself a port of SGI's `trackball.c` by Gavin Bell. ## File formats read from a published specification @@ -780,6 +836,12 @@ A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, "Image dictionary (imgCIF International Tables for Crystallography Vol. G, 444-458 [doi:10.1107/97809553602060000746](https://doi.org/10.1107/97809553602060000746). +**PILATUS count-rate clipping** — that the count-rate correction clips an over-range pixel to the +cutoff rather than flagging it, which is how the miniCBF reader tells a saturated pixel. P. Trueb, +C. Dejoie, M. Kobas, P. Pattison, D. J. Peake, V. Radicci, B. A. Sobott, D. A. Walko and +C. Broennimann, "Bunch mode specific rate corrections for PILATUS3 detectors" (2015), J. Synchrotron +Rad. 22, 701-707 [doi:10.1107/S1600577515003288](https://doi.org/10.1107/S1600577515003288). + **d\*TREK SMV** — the SMV reader reads the d\*TREK header vocabulary of Rigaku's CrystalClear from the headers themselves, interpreting the detector vectors as dxtbx does; no d\*TREK or dxtbx code is used. J. W. Pflugrath, "The finer things in X-ray diffraction data collection" (1999), Acta Cryst. @@ -851,6 +913,15 @@ H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. P. E. Bourne, "The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242 [doi:10.1093/nar/28.1.235](https://doi.org/10.1093/nar/28.1.235). +**The [Crystallography Open Database](https://www.crystallography.net/cod/)** supplies the reference +cells of the small-molecule sets. S. Gražulis, D. Chateigner, R. T. Downs, A. F. T. Yokochi, +M. Quirós, L. Lutterotti, E. Manakova, J. Butkus, P. Moeck and A. Le Bail, "Crystallography Open +Database – an open-access collection of crystal structures" (2009), J. Appl. Cryst. 42, 726-729 +[doi:10.1107/S0021889809016690](https://doi.org/10.1107/S0021889809016690); S. Gražulis, +A. Daškevič, A. Merkys et al., "Crystallography Open Database (COD): an open-access collection of +crystal structures and platform for world-wide collaboration" (2012), Nucleic Acids Res. 40, +D420-D427 [doi:10.1093/nar/gkr900](https://doi.org/10.1093/nar/gkr900). + **The [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/)** (EMBL-EBI and Google DeepMind) supplies the start models of the battery's timing benchmark: the predictions of the deposited entities of 9ZLO (AF-P17090-F1), 9S02 (AF-P32322-F1) and 9GJX (AF-Q65MG6-F1) and, for the diff --git a/docs/CPU_DATA_ANALYSIS.md b/docs/CPU_DATA_ANALYSIS.md index 8df2e38fd..bfc7217c1 100644 --- a/docs/CPU_DATA_ANALYSIS.md +++ b/docs/CPU_DATA_ANALYSIS.md @@ -112,5 +112,15 @@ The methods draw on, and in places reimplement, solutions from: - R. J. Read, "Pushing the boundaries of molecular replacement with maximum likelihood", *Acta Cryst.* **D57** (2001), 1373-1382; A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni & R. J. Read, "Likelihood-enhanced fast translation functions", *Acta Cryst.* **D61** (2005), 458-464; A. J. McCoy et al., "Phaser crystallographic software", *J. Appl. Cryst.* **40** (2007), 658-674; A. J. McCoy et al., *Proc. Natl. Acad. Sci. USA* **114** (2017), 3637-3641 (the likelihood target, the solution criterion and the expected gain of molecular replacement, §14.10). - R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold models to work with phenix.process_predicted_model and ISOLDE", *Acta Cryst.* **D78** (2022), 1303-1314, and J. Jumper et al., "Highly accurate protein structure prediction with AlphaFold", *Nature* **596** (2021), 583-589 (preparing a predicted model for molecular replacement, §14.10). +- E. A. Cornish & R. A. Fisher, "Moments and cumulants in the specification of distributions", *Rev. Int. Stat. Inst.* **5** (1938), 307-320 (the Poisson quantile of the adaptive spot-finding threshold beyond the summation limit). +- R. W. Grosse-Kunstleve, "Algorithms for deriving crystallographic space-group information", *Acta Cryst.* **A55** (1999), 383-395 (from Le Page's two-folds to a conventional cell, centring and change of basis, §13.1). +- T. C. Huang, H. Toraya, T. N. Blanton & Y. Wu, "X-ray powder diffraction analysis of silver behenate, a possible low-angle diffraction standard", *J. Appl. Cryst.* **26** (1993), 180-184 (the silver behenate calibrant cell, §7.6). +- P. Trueb, C. Dejoie, M. Kobas et al., "Bunch mode specific rate corrections for PILATUS3 detectors", *J. Synchrotron Rad.* **22** (2015), 701-707 (the count-rate correction clips an over-range pixel to the cutoff, §1.7). +- D. Liebschner, P. V. Afonine, M. L. Baker et al., "Macromolecular structure determination using X-rays, neutrons and electrons: recent developments in Phenix", *Acta Cryst.* **D75** (2019), 861-877 (phenix.xtriage, the model of the twinning test, §13.2). +- V. Luzzati, "Traitement statistique des erreurs dans la détermination des structures cristallines", *Acta Cryst.* **5** (1952), 802-810 (the fall-off of $\sigma_A$ with resolution for a model's coordinate error, Luzzati's $D$, in the likelihood of a placement, §14.10). +- D. Liebschner, P. V. Afonine, A. G. Urzhumtsev & P. D. Adams, "Implementation of the riding hydrogen model in CCTBX to support the next generation of X-ray and neutron joint refinement in Phenix", *Methods Enzymol.* **634** (2020), 177-199 (the riding hydrogens, placed by GEMMI's code, §15.2). +- D. L. Theobald, "Rapid calculation of RMSDs using a quaternion-based characteristic polynomial", *Acta Cryst.* **A61** (2005), 478-480, and P. Liu, D. K. Agrafiotis & D. L. Theobald, *J. Comput. Chem.* **31** (2010), 1561-1563 (the QCP superposition behind the NCS rms criterion, via GEMMI, §15.5). +- S. S. Oren & D. G. Luenberger, "Self-scaling variable metric (SSVM) algorithms. Part I", *Management Sci.* **20** (1974), 845-862, and D. F. Shanno & K.-H. Phua, "Matrix conditioning and nonlinear optimization", *Math. Program.* **14** (1978), 149-160 (the rescaling of the L-BFGS initial inverse Hessian by the latest pair, §15.6). + (list is not exhaustive; the full citations, with DOIs, are in [ACKNOWLEDGEMENT.md](ACKNOWLEDGEMENT.md)) diff --git a/image_analysis/SensorAbsorption.h b/image_analysis/SensorAbsorption.h index 7ff580b7b..4fe3eeb6e 100644 --- a/image_analysis/SensorAbsorption.h +++ b/image_analysis/SensorAbsorption.h @@ -32,7 +32,8 @@ // untilted detector. On a tilted one the correction acquires an azimuthal dependence at fixed // resolution, which is the only part of it that is not degenerate with an overall Wilson B. // -// Attenuation coefficients: NIST X-Ray Mass Attenuation Coefficients (Hubbell & Seltzer), total +// Attenuation coefficients: NIST X-Ray Mass Attenuation Coefficients, Hubbell & Seltzer (1995) NIST +// Standard Reference Database 126, total // mu/rho with coherent scattering. Photoelectric absorption dominates over the whole range used // here, and because the correction is a RATIO of two absorbed fractions the photoelectric branching // ratio cancels exactly; only the attenuation length enters. Coherent scattering, which attenuates diff --git a/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp b/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp index e10a3a6aa..bf844c160 100644 --- a/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp +++ b/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp @@ -246,7 +246,8 @@ std::vector BraggIntegrationEngineCPU::RunImpl(const Sampler &img, const auto &r = predicted[i]; const BraggPanel &pn = panels[i]; // This reflection's stencil: the r1 signal disk, and the r2..r3 ring elongated radially by - // the analytic smear. + // the analytic smear. The three-ring region follows CrystFEL, White et al. (2012) J. Appl. + // Cryst. 45, 335-341 const BraggStencil st = MakeBraggStencil(pn, stencil); const Box box = box_of(r, st); const int x0 = box.x0, x1 = box.x1, y0 = box.y0, y1 = box.y1; diff --git a/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu b/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu index 41f8d3cef..6b9eb3178 100644 --- a/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu +++ b/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu @@ -241,7 +241,8 @@ __global__ void boxsum(const float *px_x, const float *px_y, const BraggPanel *p const BraggStencilDist d = BraggStencilDistances(st, (float) x - cx, (float) y - cy); // The window is the bounding box of the outer ellipse, so nearly half of it is neither the // signal disk nor the background ring. Reading the pixel only once it is known to be wanted - // keeps those slots from fetching a cache line for nothing. + // keeps those slots from fetching a cache line for nothing. The r1 disk / r2..r3 ring region + // follows CrystFEL, White et al. (2012) J. Appl. Cryst. 45, 335-341 if (BraggStencilInSignal(st, (float) x - cx, (float) y - cy, d.signal, p.r1_sq)) { // A pixel a nearer neighbour owns carries that neighbour's flux; see the CPU engine. ++l_nd; diff --git a/image_analysis/geom_refinement/PostRefine.h b/image_analysis/geom_refinement/PostRefine.h index 2f7d8c360..6ae1b3e74 100644 --- a/image_analysis/geom_refinement/PostRefine.h +++ b/image_analysis/geom_refinement/PostRefine.h @@ -30,6 +30,8 @@ // * the positional detector<->reciprocal residual at each partial's observed spot -> pins the distance. // Where there are more reflections than the fit's caps, the strongest by I/sigma are the ones kept; // inside the fit every reflection carries the same weight. +// Following MOSFLM's post-refinement, Powell (1999) Acta Cryst. D55, 1690-1695, and Leslie & Powell (2007) +// Evolving Methods for Macromolecular Crystallography, NATO Sci. Ser. II 245, 41-51 // How far one step of the joint fit may move the distance or a cell length, relative (see STEP_BOUND in // PostRefine.cpp, where it is explained). diff --git a/image_analysis/scale_merge/CentreOfSymmetry.cpp b/image_analysis/scale_merge/CentreOfSymmetry.cpp index a7faebf74..be42bfc6f 100644 --- a/image_analysis/scale_merge/CentreOfSymmetry.cpp +++ b/image_analysis/scale_merge/CentreOfSymmetry.cpp @@ -98,7 +98,8 @@ namespace { } } -// Following Howells, Phillips & Rogers (1950) Acta Cryst. 3, 210-214 (N(z)), and Padilla & Yeates +// Following Wilson (1949) Acta Cryst. 2, 318-321 (the acentric and centric intensity distributions), +// Howells, Phillips & Rogers (1950) Acta Cryst. 3, 210-214 (N(z)), and Padilla & Yeates // (2003) Acta Cryst. D59, 1124-1130 (the L-test, read here against its centric value 2/pi). CentreOfSymmetryResult AnalyzeCentreOfSymmetry(const std::vector& merged, const gemmi::SpaceGroup& space_group, size_t nthreads) { diff --git a/image_analysis/scale_merge/StillsPartialityRefine.h b/image_analysis/scale_merge/StillsPartialityRefine.h index 11af1d26f..1a0fa081b 100644 --- a/image_analysis/scale_merge/StillsPartialityRefine.h +++ b/image_analysis/scale_merge/StillsPartialityRefine.h @@ -39,6 +39,7 @@ // `image_scale_corr` in place; the existing MergeOnTheFly then consumes the improved corrections. The // width of the Gaussian partiality is the run's, measured once from the intensity-weighted excitation // errors (MeasureRunWidth). +// Following Uervirojnangkoorn et al. (2015) eLife 4, e05421 (PRIME) and Sauter et al. (2014) Acta Cryst. D70, 3299-3309 class StillsPartialityRefine { public: struct Settings { diff --git a/image_analysis/structure_refinement/MolecularReplacement.h b/image_analysis/structure_refinement/MolecularReplacement.h index 1ef05b244..668b017e6 100644 --- a/image_analysis/structure_refinement/MolecularReplacement.h +++ b/image_analysis/structure_refinement/MolecularReplacement.h @@ -63,6 +63,8 @@ SearchModel PrepareSearchModel(const RugnuxModel &model); // linkage on the symmetrised PAE until every two clusters are a mean PAE_DOMAIN_A apart; clusters of at least // DOMAIN_MIN_RESIDUES, largest first, each a list of positions in the prediction. Empty where the matrix does // not match the model. +// The models, their pLDDT and PAE following Jumper et al. (2021) Nature 596, 583-589, and Varadi et al. (2022) +// Nucleic Acids Res. 50, D439-D444 (the AlphaFold DB) std::vector> PaeDomains(const std::vector> &pae, const SearchModel &model); // The search model of one domain: the residues of `model` at those positions, recentred. SearchModel DomainModel(const SearchModel &model, const std::vector &residues); diff --git a/image_analysis/structure_refinement/Refinement.cpp b/image_analysis/structure_refinement/Refinement.cpp index bae38e302..07f0e089c 100644 --- a/image_analysis/structure_refinement/Refinement.cpp +++ b/image_analysis/structure_refinement/Refinement.cpp @@ -2653,6 +2653,9 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector TlsBodiesToGroups(const gemmi::Model &model, const // U23 of the difference in a frame whose z is n, as SHELXL's RIGU restrains them; // - similarity, for the same pairs: |U_i - U_j|^2 (Frobenius) / sigma^2, sigma 0.04 A^2 (SHELXL's SIMU); // - near isotropy, for an atom in no pair (a water, an ion): |U - tr(U)/3 I|^2 / sigma^2, sigma 0.1 A^2 (ISOR). -// Following Hirshfeld (1976) Acta Cryst. A32, 239-244, and Thorn, Dittrich & Sheldrick (2012) Acta Cryst. A68, 448-451 +// Following Hirshfeld (1976) Acta Cryst. A32, 239-244, Thorn, Dittrich & Sheldrick (2012) Acta Cryst. A68, 448-451 +// (RIGU), and Sheldrick (2015) Acta Cryst. C71, 3-8 (SIMU and ISOR) struct URestraints { static constexpr double SIGMA_RIGID = 0.004, SIGMA_SIMILAR = 0.04, SIGMA_ISOTROPIC = 0.1; std::vector> pairs; diff --git a/image_analysis/structure_refinement/RefinementXray.h b/image_analysis/structure_refinement/RefinementXray.h index 789e888c9..3b8640d6e 100644 --- a/image_analysis/structure_refinement/RefinementXray.h +++ b/image_analysis/structure_refinement/RefinementXray.h @@ -133,7 +133,8 @@ private: // beta fitted by maximum likelihood to the free reflections as the whole spread of Fo about D Fc, D then fitted to // the working reflections with the variance epsilon beta + 2 sigma^2 (acentric; + sigma^2 centric), which is also // the variance of a reflection in the target. `fmodel` is k_total (F + fmask), without D. -// Following Lunin & Skovoroda (1995) Acta Cryst. A51, 880-887, and Murshudov et al. (2011) Acta Cryst. D67, 355-367 +// Following Lunin & Skovoroda (1995) Acta Cryst. A51, 880-887, Lunin, Afonine & Urzhumtsev (2002) Acta Cryst. A58, +// 270-282 (the parameters estimated on the free reflections), and Murshudov et al. (2011) Acta Cryst. D67, 355-367 void EstimateLikelihoodParameters(const RefinementReflections &refl, const std::vector> &fmodel, XrayRowModel &rows, size_t nthreads = 1); @@ -151,7 +152,8 @@ std::vector> TwinnedModel(const RefinementReflections &refl, // twinned amplitudes took a fraction of 0.16 for unrelated partners of one model at R-free 0.31, the average of two // predictions being the better prediction. Reflections that are their own mate under every law say nothing and are // left out. With one law the fraction is at most 1/2. `f` is F at every row, `rows` gives k_total and fmask. -// Following Yeates (1997) Methods Enzymol. 276, 344-358 (the twinned intensity), refined as Murshudov et al. (2011) +// Following Yeates (1997) Methods Enzymol. 276, 344-358 (the twinned intensity), refined beside the model as +// Herbst-Irmer & Sheldrick (1998) Acta Cryst. B54, 443-449 (SHELXL's BASF) and Murshudov et al. (2011) // Acta Cryst. D67, 355-367 refine the fractions, by least squares constexpr size_t TWIN_FRACTION_SHELL = 500; std::vector FitTwinFractions(const RefinementReflections &refl, const XrayRowModel &rows,