Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m43s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 28m46s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 32m5s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 39m44s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m7s
Build Packages / Create release (push) Successful in 8s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 22m30s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 31m54s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m29s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 23m51s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 26m7s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 29m38s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 28m6s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 29m44s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 33m22s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 34m6s
Build Packages / Unit tests (push) Successful in 1h2m47s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 35m59s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 34m8s
Build Packages / Validation Battery (push) Successful in 38m31s
* Rugnux: Refinement several times faster, one protocol on the CPU and the GPU, parts of it on the Apple GPU; its random draws are the same on every platform. * Rugnux: Refinement follows common practice: the free set decides only the restraint weights and the displacement model (isotropic, TLS or anisotropic, by Hamilton's test), everything else is decided on the working set; lower R-free for placed and predicted models. * Rugnux: Restraint dictionaries via `--restraints` (or among the inputs) and more built-in ligands; a model whose space group is a subgroup of the data's is refined in its own group; reflections plus a model select `--mode refine` on their own. * Rugnux: `-C` fixes the crystal system on the axes as typed; `-S` alone fixes the space group only, the cell is still found and reduced; a space group or cell stored in the input file is ignored, with a line in the log. * Rugnux: More reliable multi-sweep merging; `--mode scale` merges several stills runs as one; `_plot.txt` and the report name each input; a second crystal form is reported for stills given a cell. * jfjoch_broker (experimental): Live Rugnux on FPGA detectors (JUNGFRAU) takes the same path as a DECTRIS stream, rotation and serial alike, model step included, with analysis threads per GPU; frames waiting for the analysis can be viewed; no host stage drops a frame - a source faster than the analysis is held at the card. * jfjoch_broker: Summation of more than 256 frames per image is split between the card and the host (`summation_mode` is then `hybrid`), and live Rugnux runs on such collections; pedestal runs take a receiver of their own. * jfjoch_broker (experimental): `stills_downsample` analyses 1 of every N serial images (chosen by a hash; the first 2000 all; the others written without analysis; reset to 1 by `/initialize`); `max_spot_count` left out means 250 for per-image indexing and up to 1000 for live stills; the live status gives the merge before the model work, the fitted resolution and the anomalous peak height of each unexplained blob. * jfjoch_broker: Without a `pcie` entry the broker runs on a mock of the card instead of the HLS simulation; its spot, ROI and azimuthal results are zero then. * Web frontend: Redesigned Rugnux live page; Vite 8, TypeScript 6, Mol* 5.13 and dependency security updates. * Tools: `jfjoch_fpga_stills_test` measures live stills on the cards with a stored dataset; `jfjoch_fpga_test` loses the HLS-simulation option. * Other: FFTW no longer fetched or linked (the CPU FFT indexer keeps the name `FFTW`); PCIe driver now GPL-2.0-or-later and third-party notices complete; several crash fixes. Reviewed-on: #98 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
1003 lines
76 KiB
Markdown
1003 lines
76 KiB
Markdown
# Acknowledgements
|
|
|
|
How to cite Jungfraujoch and Rugnux, and the papers that describe them, is on the
|
|
[Publications](PUBLICATIONS.md) page. In short:
|
|
|
|
- **Rugnux**: F. Leonarski and M. Wang, "Rugnux: from diffraction images to structure at synchrotrons
|
|
and XFELs in one program" (2026), bioRxiv
|
|
[doi:10.64898/2026.10.24.761522](https://doi.org/10.64898/2026.10.24.761522).
|
|
- **Jungfraujoch**: F. Leonarski, M. Brückner, C. Lopez-Cuenca, A. Mozzanica, H.-C. Stadler,
|
|
Z. Matěj, A. Castellane, B. Mesnet, J. Wojdyla, B. Schmitt and M. Wang, "Jungfraujoch:
|
|
hardware-accelerated data-acquisition system for kilohertz pixel-array X-ray detectors" (2023),
|
|
J. Synchrotron Rad. 30, 227-234 [doi:10.1107/S1600577522010268](https://doi.org/10.1107/S1600577522010268);
|
|
for real-time serial crystallography and live indexing feedback also J. Duan, H.-C. Stadler,
|
|
J. Nan et al., "Jungfraujoch: 38 GB s⁻¹ real-time serial crystallography – data acquisition and
|
|
live indexing feedback" (2026), J. Appl. Cryst. 59, 1668-1677
|
|
[doi:10.1107/S1600576726008083](https://doi.org/10.1107/S1600576726008083).
|
|
|
|
## Funding and support
|
|
|
|
The project is supported by:
|
|
* Innosuisse via Innovation Project "NextGenDCU high data rate acquisition system for X-ray detectors in structural biology applications" (101.535.1 IP-ENG; Apr 2023 - Sep 2025).
|
|
* ETH Domain via Open Research Data Contribute project (Jan - Dec 2023).
|
|
* AMD University Program with donation of licenses of Ethernet IP cores and Vivado software.
|
|
|
|
## Crystallographic methods adopted from other packages
|
|
|
|
The analysis pipeline builds on methods first published, and in most cases first implemented, by
|
|
other crystallographic software. The code is Jungfraujoch's own; the methods are theirs, and are
|
|
acknowledged here, entry by entry: what was taken, from whom, and the citation. How rugnux uses each
|
|
method is described in the [data-analysis reference](CPU_DATA_ANALYSIS.md), not here. Where a
|
|
package's source was consulted this is said explicitly. Most of these packages are neither linked nor
|
|
vendored; the three that are - GEMMI, traccc and fast-feedback-indexer - also carry a licence
|
|
obligation, recorded in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
|
|
|
|
### Experimental geometry and physical corrections
|
|
|
|
**[autoPROC](https://www.globalphasing.com/autoproc/)** — the principle, long argued for by Clemens
|
|
Vonrhein, that a pipeline does not take the experimental metadata it is given (beam centre, detector
|
|
geometry, goniometer axis) on trust but verifies it against the images and corrects it where it is
|
|
wrong, and with it that the beam centre is found from the images; the aim of a pipeline that takes the
|
|
decisions an expert would take and reports the evidence for each. C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, "Data
|
|
processing and analysis with the autoPROC toolbox" (2011), Acta Cryst. D67, 293-302
|
|
[doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773).
|
|
|
|
**Sensor absorption and air path** — the attenuation coefficients of silicon, CdTe, dry air and
|
|
helium behind the oblique-incidence sensor efficiency, the parallax variance and the air-path
|
|
attenuation are the NIST tabulation. J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray
|
|
Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for
|
|
Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004),
|
|
NIST Standard Reference Database 126
|
|
[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F).
|
|
|
|
**Polarization correction** — the azimuthal polarization factor for a partially polarized
|
|
synchrotron source. R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular
|
|
crystallography with synchrotron radiation: photographic data collection and polarization
|
|
correction" (1982), J. Appl. Cryst. 15, 330-337
|
|
[doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060).
|
|
|
|
**Hexagonal-ice ring positions** — the eleven measured ring spacings from 3.895 to 1.522 Å.
|
|
D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in biomolecular cryocrystallography" (2021), Acta
|
|
Cryst. D77, 540-554 [doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170).
|
|
The bands below 1.522 Å are calculated here from ice Ih structure factors on the lattice constants
|
|
of K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice constants and thermal
|
|
expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50, 644-648
|
|
[doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933).
|
|
|
|
**Powder calibrants** — the silver behenate cell is T. C. Huang, H. Toraya, T. N. Blanton and Y. Wu,
|
|
"X-ray powder diffraction analysis of silver behenate, a possible low-angle diffraction standard"
|
|
(1993), J. Appl. Cryst. 26, 180-184
|
|
[doi:10.1107/S0021889892009762](https://doi.org/10.1107/S0021889892009762); the CeO2 and silicon
|
|
cells are those of the NIST standard reference materials SRM 674b and SRM 640 (certificates, no DOI).
|
|
|
|
### Spot finding
|
|
|
|
**[CrystFEL](https://www.desy.de/~twhite/crystfel/)** — spot finding, the three-ring integration
|
|
region, the serial/stills processing model, and the per-frame indexing acceptance test
|
|
(`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass,
|
|
A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012),
|
|
J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312).
|
|
|
|
**Cheetah** — the per-resolution-ring background statistics with the Bragg peaks excluded by
|
|
iterated clipping, from peakfinder8. A. Barty, R. A. Kirian, F. R. N. C. Maia, M. Hantke, C. H. Yoon,
|
|
T. A. White and H. N. Chapman, "Cheetah: software for high-throughput reduction and analysis of
|
|
serial femtosecond X-ray diffraction data" (2014), J. Appl. Cryst. 47, 1118-1131
|
|
[doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626).
|
|
|
|
**Poisson significance threshold** — beyond the summation limit the Poisson quantile of the adaptive
|
|
spot-finding threshold is taken from the Cornish-Fisher expansion. E. A. Cornish and R. A. Fisher,
|
|
"Moments and cumulants in the specification of distributions" (1938), Rev. Int. Stat. Inst. 5,
|
|
307-320 [doi:10.2307/1400905](https://doi.org/10.2307/1400905).
|
|
|
|
**GPU spot finder** — the CUDA spot finder was developed by Hans-Christian Stadler (PSI).
|
|
|
|
**[traccc](https://github.com/acts-project/traccc)** — the sparse connected-component labelling
|
|
that groups strong pixels into spots: the CPU extractor adapts its SparseCCL source, the CUDA
|
|
extractor follows the design of its GPU counterpart (MPL-2.0; see
|
|
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). P. Gessinger, H. M. Gray, A. Krasznahorkay,
|
|
C. Leggett, J. Niermann, A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction
|
|
library for HEP experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822). The
|
|
SparseCCL algorithm itself: A. Hennequin, B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL:
|
|
Connected Components Labeling and Analysis for sparse images" (2019), DASIP 2019, 65-70
|
|
[doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184).
|
|
|
|
**XDS and DISTL** — bounding a spot by its shape (how much of its bounding box it fills) rather than
|
|
by its size; the statistic is rugnux's own. XDS is cited under
|
|
[Integration and rotation geometry](#integration-and-rotation-geometry); Z. Zhang, N. K. Sauter,
|
|
H. van den Bedem, G. Snell and A. M. Deacon, "Automated diffraction image analysis and spot searching
|
|
for high-throughput crystal screening" (2006), J. Appl. Cryst. 39, 112-119
|
|
[doi:10.1107/S0021889805040677](https://doi.org/10.1107/S0021889805040677).
|
|
|
|
### Indexing
|
|
|
|
**Fourier-transform indexing** — indexing with no prior knowledge of the cell by Fourier-transforming
|
|
the reciprocal-space positions of the spots, proposed as a three-dimensional transform; the de-novo
|
|
indexers (`-X fft`, `-X fftw`) use its one-dimensional projection form. G. Bricogne (1986), in
|
|
*Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software (Phase III)*,
|
|
LURE, Paris, p. 28 (a workshop proceedings without a DOI).
|
|
|
|
**Rossmann FFT autoindexing** — projecting the reciprocal-space points onto many directions and
|
|
Fourier-transforming the 1D histograms. I. Steller, R. Bolotovsky and M. G. Rossmann, "An algorithm
|
|
for automatic indexing of oscillation images using Fourier analysis" (1997), J. Appl. Cryst. 30,
|
|
1036-1040 [doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777).
|
|
|
|
**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/mosflm/)** — the practice of that algorithm, and
|
|
the post-refinement practice of which parameters are refined per image and which over a wedge.
|
|
H. R. Powell, "The Rossmann Fourier autoindexing algorithm in MOSFLM" (1999), Acta Cryst. D55,
|
|
1690-1695 [doi:10.1107/S0907444999009506](https://doi.org/10.1107/S0907444999009506);
|
|
A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM" (2007), in *Evolving
|
|
Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245, 41-51
|
|
[doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4);
|
|
T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new
|
|
graphical interface for diffraction-image processing with MOSFLM" (2011), Acta Cryst. D67, 271-281
|
|
[doi:10.1107/S0907444910048675](https://doi.org/10.1107/S0907444910048675); H. R. Powell,
|
|
T. G. G. Battye, L. Kontogiannis, O. Johnson and A. G. W. Leslie, "Integrating macromolecular X-ray
|
|
diffraction data with the graphical user interface iMosflm" (2017), Nat. Protoc. 12, 1310-1325
|
|
[doi:10.1038/nprot.2017.037](https://doi.org/10.1038/nprot.2017.037).
|
|
|
|
**[fast-feedback-indexer](https://github.com/paulscherrerinstitute/fast-feedback-indexer)** — the
|
|
known-cell indexer for serial stills (`-X ffbidx`), PSI's library linked at build time
|
|
(BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), implementing TORO.
|
|
P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton, M. Janousch,
|
|
F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for kilohertz serial
|
|
crystallography" (2024), J. Appl. Cryst. 57, 931-944
|
|
[doi:10.1107/S1600576724003182](https://doi.org/10.1107/S1600576724003182). The CPU implementation
|
|
used without a GPU is a port of the library's CUDA kernels by H.-C. Stadler (PSI).
|
|
|
|
### Cell reduction and lattice symmetry
|
|
|
|
**[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and
|
|
structure-factor machinery, and MTZ / XDS_ASCII I/O; vendored in `gemmi_gph/`, so it also carries a
|
|
licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source
|
|
Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200).
|
|
|
|
**Niggli reduction** — used through GEMMI's implementation. I. Křivý and B. Gruber, "A unified
|
|
algorithm for determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298
|
|
[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636). The table of lattice
|
|
characters that maps a reduced cell to Bravais lattices and centrings follows International Tables
|
|
for Crystallography Vol. A, Table 9.2.5.1.
|
|
|
|
**Numerically stable cell reduction** — the magnitude-scaled tolerance that decides the sign of a
|
|
structurally-zero scalar product. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically
|
|
stable algorithms for the computation of reduced unit cells" (2004), Acta Cryst. A60, 1-6
|
|
[doi:10.1107/S010876730302186X](https://doi.org/10.1107/S010876730302186X).
|
|
|
|
**Metric-symmetry search** — the obliquity of the 81 candidate two-folds of a reduced cell, and the
|
|
conventional axes derived from the rotation group found; the two-fold search is used through GEMMI.
|
|
Y. Le Page, "The derivation of the axes of the conventional unit cell from the dimensions of the
|
|
Buerger-reduced cell" (1982), J. Appl. Cryst. 15, 255-259
|
|
[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959). The step from the
|
|
two-folds found to a conventional cell, a centring and an integral change of basis follows
|
|
R. W. Grosse-Kunstleve, "Algorithms for deriving crystallographic space-group information" (1999),
|
|
Acta Cryst. A55, 383-395 [doi:10.1107/S0108767398010186](https://doi.org/10.1107/S0108767398010186).
|
|
|
|
### Integration and rotation geometry
|
|
|
|
**[XDS](https://xds.mr.mpg.de/)** — rotation geometry and notation, the reciprocal Lorentz and
|
|
partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for a
|
|
reflection whose predicted profile is not cleanly its own, the intensity-based test for a centred
|
|
lattice, the recognition of shaded detector regions (`DEFPIX`), and the scaling correction surfaces
|
|
indexed by image number and detector region. W. Kabsch, "XDS" (2010), Acta Cryst. D66, 125-132
|
|
[doi:10.1107/S0907444909047337](https://doi.org/10.1107/S0907444909047337); W. Kabsch, "Integration,
|
|
scaling, space-group assignment and post-refinement" (2010), Acta Cryst. D66, 133-144
|
|
[doi:10.1107/S0907444909047374](https://doi.org/10.1107/S0907444909047374).
|
|
|
|
**Profile fitting** — the Kabsch/Otwinowski iteration with reweighted, de-biased variances, from the
|
|
second XDS paper above and from Z. Otwinowski and W. Minor, "Processing of X-ray diffraction data
|
|
collected in oscillation mode" (1997), Methods Enzymol. 276, 307-326
|
|
[doi:10.1016/S0076-6879(97)76066-X](https://doi.org/10.1016/S0076-6879%2897%2976066-X).
|
|
|
|
**Two-dimensional integration** — integrating each image in the detector plane and only afterwards
|
|
combining a reflection's partials into a full, as against three-dimensional profile fitting through
|
|
the image stack, is the architecture of DENZO/SCALEPACK (Otwinowski and Minor, above) and MOSFLM
|
|
(cited under [Indexing](#indexing)).
|
|
|
|
### Space group, twinning and pseudo-symmetry
|
|
|
|
**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search: scoring each candidate
|
|
rotation operator by the correlation of I(h) with I(Rh) on resolution-normalised intensities (E²);
|
|
the screw-axis test that scores a predicted-absent class against the rest of its own axial row and
|
|
lets confidence fall with the number of axial reflections instead of refusing below a count; and the
|
|
same test on a zone for a glide plane. P. Evans, "Scaling and assessment of data quality" (2006),
|
|
Acta Cryst. D62, 72-82 [doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693);
|
|
P. R. Evans, "An introduction to data reduction: space-group determination, scaling and intensity
|
|
statistics" (2011), Acta Cryst. D67, 282-292
|
|
[doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X); P. R. Evans and
|
|
G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta Cryst. D69,
|
|
1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061); J. Agirre,
|
|
M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular
|
|
crystallography" (2023), Acta Cryst. D79, 449-461
|
|
[doi:10.1107/S2059798323003595](https://doi.org/10.1107/S2059798323003595).
|
|
|
|
**Centrosymmetric glide groups** — of two absence-equivalent glide groups the search writes the
|
|
centrosymmetric one (P2/c over Pc, C2/c over Cc), a missed inversion centre being the common error.
|
|
W. H. Baur and D. Kassner, "The perils of Cc: comparing the frequencies of falsely assigned space
|
|
groups with their general population" (1992), Acta Cryst. B48, 356-369
|
|
[doi:10.1107/S0108768191014726](https://doi.org/10.1107/S0108768191014726).
|
|
|
|
**Centre-of-symmetry statistics** — the acentric and centric intensity distributions and the
|
|
cumulative N(z) test built on them, read beside <|E^2-1|> and the L test. A. J. C. Wilson, "The
|
|
probability distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321
|
|
[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); E. R. Howells,
|
|
D. C. Phillips and D. Rogers, "The probability distribution of X-ray intensities. II. Experimental
|
|
investigation and the X-ray detection of centres of symmetry" (1950), Acta Cryst. 3, 210-214
|
|
[doi:10.1107/S0365110X50000513](https://doi.org/10.1107/S0365110X50000513).
|
|
|
|
**Twinning L test** — the moments of L = (I1−I2)/(I1+I2) over pairs of neighbouring acentric
|
|
reflections against their untwinned and perfect-twin values; its robustness to pseudo-centering is
|
|
kept by the partner steps, which are restricted when a non-half-integer pseudo-translation is found.
|
|
J. E. Padilla and T. O. Yeates, "A statistic for local intensity differences: robustness to
|
|
anisotropy and pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59,
|
|
1124-1130 [doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). The
|
|
test's design - indicators on shell-normalised merged intensities, a twin law admitted within
|
|
3 degrees of obliquity - follows phenix.xtriage: Zwart, Grosse-Kunstleve and Adams (2005, cited under
|
|
[Data quality and anisotropy](#data-quality-and-anisotropy)); D. Liebschner, P. V. Afonine,
|
|
M. L. Baker et al., "Macromolecular structure determination using X-rays, neutrons and electrons:
|
|
recent developments in Phenix" (2019), Acta Cryst. D75, 861-877
|
|
[doi:10.1107/S2059798319011471](https://doi.org/10.1107/S2059798319011471).
|
|
|
|
**Twin fraction and twinned intensity** — the model (1−α) I(h) + α I(Th), and the fraction read
|
|
from the median of H = |I(h) − I(Th)|/(I(h) + I(Th)), uniform on [0, 1−2α]. T. O. Yeates, "Detecting
|
|
and overcoming crystal twinning" (1997), Methods Enzymol. 276, 344-358
|
|
[doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879%2897%2976068-3).
|
|
|
|
**Twin refinement** — one fraction per domain operator of the lattice refined with the model, small
|
|
domains dropped, and twin-related reflections kept in one free set, as REFMAC5 does; the fractions
|
|
re-fitted by least squares at every cycle as REFMAC5 refines them, against an amplitude likelihood
|
|
with the twinned model amplitude in it (a simplification of REFMAC5's twinned-intensity target).
|
|
G. N. Murshudov, P. Skubák, A. A. Lebedev, N. S. Pannu, R. A. Steiner, R. A. Nicholls, M. D. Winn,
|
|
F. Long and A. A. Vagin, "REFMAC5 for the refinement of macromolecular crystal structures" (2011),
|
|
Acta Cryst. D67, 355-367 [doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314);
|
|
P. Roversi, E. Blanc, S. Johnson and S. M. Lea, "Tetartohedral twinning could happen to you too"
|
|
(2012), Acta Cryst. D68, 418-424
|
|
[doi:10.1107/S0907444912006737](https://doi.org/10.1107/S0907444912006737); the six-domain case of
|
|
a monoclinic crystal on a hexagonal metric is K. A. Sultana, I. Alexeev, I. Kursula, P. Mäntsälä,
|
|
J. Niemi and G. Schneider, "Structure determination by multiwavelength anomalous diffraction of
|
|
aclacinomycin oxidoreductase: indications of multidomain pseudomerohedral twinning" (2007), Acta
|
|
Cryst. D63, 149-159 [doi:10.1107/S0907444906044271](https://doi.org/10.1107/S0907444906044271).
|
|
A twin fraction as a
|
|
parameter refined beside the model is SHELXL's BASF (rugnux re-fits it between cycles rather than in
|
|
the same least squares): R. Herbst-Irmer and G. M. Sheldrick, "Refinement of twinned structures with
|
|
SHELXL97" (1998), Acta Cryst. B54, 443-449
|
|
[doi:10.1107/S0108768197018454](https://doi.org/10.1107/S0108768197018454). The twin laws come from
|
|
the lattice's metric: A. A. Lebedev, A. A. Vagin and G. N. Murshudov, "Intensity statistics in
|
|
twinned crystals with examples from the PDB" (2006), Acta Cryst. D62, 83-95
|
|
[doi:10.1107/S0907444905036759](https://doi.org/10.1107/S0907444905036759).
|
|
|
|
**Deciding pseudo-symmetry by refinement** — carrying the adopted group and the index-2 subgroups a
|
|
refused promotion leaves, each perfectly twinned by a law of its coset, to the model and deciding by
|
|
placing and refining it in each, rather than from intensity statistics. P. H. Zwart,
|
|
R. W. Grosse-Kunstleve, A. A. Lebedev, G. N. Murshudov and P. D. Adams, "Surprises and pitfalls
|
|
arising from (pseudo)symmetry" (2008), Acta Cryst. D64, 99-107
|
|
[doi:10.1107/S090744490705531X](https://doi.org/10.1107/S090744490705531X); A. A. Lebedev and
|
|
M. N. Isupov, "Space-group and origin ambiguity in macromolecular structures with pseudo-symmetry
|
|
and its treatment with the program Zanuda" (2014), Acta Cryst. D70, 2430-2443
|
|
[doi:10.1107/S1399004714014795](https://doi.org/10.1107/S1399004714014795).
|
|
|
|
**Twinning by reticular merohedry** — a first-pass cell that is the coincidence lattice of two twin
|
|
domains (reticular merohedry or pseudo-merohedry of index up to 10, e.g. obverse/reverse
|
|
rhombohedral domains) is recognised by the nodes of neither domain being empty. The twin index is
|
|
Friedel's (G. Friedel, *Leçons de cristallographie*, 1926; no DOI); the split of the reflections into
|
|
those of one domain, of the other, of both and of neither, and the treatment of reticular
|
|
pseudo-merohedry (Herbst-Irmer and Sheldrick 1998, above), are Herbst-Irmer and Sheldrick's:
|
|
R. Herbst-Irmer and G. M. Sheldrick, "Refinement of obverse/reverse twins" (2002), Acta Cryst. B58,
|
|
477-481 [doi:10.1107/S0108768102001039](https://doi.org/10.1107/S0108768102001039).
|
|
|
|
**Translational pseudo-symmetry** — the off-origin native-Patterson peak read as a pseudo-translation
|
|
between copies of the asymmetric unit, and the modulation it puts on the intensities, which the
|
|
axial systematic-absence test also scores against. The per-dataset permutation null for the peak,
|
|
the per-row estimate of the modulation depth, and its averaging over the lattice's rotations are
|
|
rugnux's own. R. J. Read, P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of
|
|
translational noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183
|
|
[doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374). A pseudo-translation
|
|
is declared at a Patterson peak of a fifth of the origin or more, Phaser's default (McCoy et al.
|
|
2007, cited under [Model placement and molecular replacement](#model-placement-and-molecular-replacement)).
|
|
|
|
### Scaling and merging
|
|
|
|
**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off,
|
|
per-observation outlier rejection at merge, the scaling error model, and the treatment of a
|
|
reflection whose background is contaminated; its published behaviour, and in places its source,
|
|
settled several choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS:
|
|
implementation and evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97
|
|
[doi:10.1107/S2059798317017235](https://doi.org/10.1107/S2059798317017235); D. G. Waterman,
|
|
G. Winter, R. J. Gildea et al., "Diffraction-geometry refinement in the DIALS framework" (2016),
|
|
Acta Cryst. D72, 558-575 [doi:10.1107/S2059798316002187](https://doi.org/10.1107/S2059798316002187);
|
|
J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software
|
|
package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399
|
|
[doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198); J. M. Parkhurst,
|
|
G. Winter, D. G. Waterman et al., "Robust background modelling in DIALS" (2016), J. Appl. Cryst. 49,
|
|
1912-1921 [doi:10.1107/S1600576716013595](https://doi.org/10.1107/S1600576716013595).
|
|
|
|
**Wilson outlier test** — judging an observation without symmetry mates against the acentric and
|
|
centric distributions of its shell, with the symmetry enhancement factor; rejecting only significant
|
|
observations, and keeping a reflection whose observations are all large, follows AIMLESS's EMAX test.
|
|
A. J. C. Wilson, "The probability distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321
|
|
[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813); P. Evans, "Scaling and
|
|
assessment of data quality" (2006), Acta Cryst. D62, 72-82
|
|
[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693).
|
|
|
|
**Amplitudes from intensities** — the posterior-mean amplitude under the acentric and centric Wilson
|
|
priors. S. French and K. Wilson, "On the treatment of negative intensity observations" (1978), Acta
|
|
Cryst. A34, 517-525 [doi:10.1107/S0567739478001114](https://doi.org/10.1107/S0567739478001114).
|
|
Giving no amplitude to an intensity more than 3.7 sigma below zero, and scaling each reflection's
|
|
prior by the anisotropy tensor along its direction, follow CCP4's
|
|
[ctruncate](https://www.ccp4.ac.uk/) (C. Ballard and N. Stein), cited through the suite:
|
|
M. D. Winn, C. C. Ballard, K. D. Cowtan et al., "Overview of the CCP4 suite and current developments"
|
|
(2011), Acta Cryst. D67, 235-242
|
|
[doi:10.1107/S0907444910045749](https://doi.org/10.1107/S0907444910045749).
|
|
|
|
**Absorption as spherical harmonics** — the empirical absorption correction as a series of real
|
|
spherical harmonics of the beam directions in the crystal frame is Blessing's; its use as a
|
|
restrained scaling surface of the diffracted-beam direction follows SCALA and AIMLESS. R. H. Blessing,
|
|
"An empirical correction for absorption anisotropy" (1995), Acta Cryst. A51, 33-38
|
|
[doi:10.1107/S0108767394005726](https://doi.org/10.1107/S0108767394005726); P. Evans, "Scaling and
|
|
assessment of data quality" (2006), Acta Cryst. D62, 72-82
|
|
[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693).
|
|
|
|
**Per-frame scale smoothing** — the Whittaker smoother (a second-difference penalty, the smoothness
|
|
chosen by cross-validation) in Eilers's form. P. H. C. Eilers, "A perfect smoother" (2003),
|
|
Anal. Chem. 75, 3631-3636 [doi:10.1021/ac034173t](https://doi.org/10.1021/ac034173t); E. T. Whittaker,
|
|
"On a new method of graduation" (1923), Proc. Edinburgh Math. Soc. 41, 63-75
|
|
[doi:10.1017/S0013091500077853](https://doi.org/10.1017/S0013091500077853).
|
|
|
|
**Fisher's z-transformation** — averaging the change of the half-set CC1/2 over resolution shells on
|
|
atanh(CC) when the correction surfaces are cross-validated. R. A. Fisher, "Frequency distribution of
|
|
the values of the correlation coefficient in samples from an indefinitely large population" (1915),
|
|
Biometrika 10, 507-521 [doi:10.2307/2331838](https://doi.org/10.2307/2331838).
|
|
|
|
**Frame disposition by delta-CC1/2** — the statistic in its sigma-tau form, the Fisher transformation
|
|
used to compare it, its standard error, and the rejection discipline (never remove a group whose
|
|
delta-CC1/2 is positive or near zero; remove a little, re-form the reference and repeat), whose
|
|
reference implementation is XDSCC12. G. Assmann, W. Brehm and K. Diederichs, "Identification of
|
|
rogue datasets in serial crystallography" (2016), J. Appl. Cryst. 49, 1021-1028
|
|
[doi:10.1107/S1600576716005471](https://doi.org/10.1107/S1600576716005471); G. M. Assmann, M. Wang
|
|
and K. Diederichs, "Making a difference in multi-data-set crystallography: simple and deterministic
|
|
data-scaling/selection methods" (2020), Acta Cryst. D76, 636-652
|
|
[doi:10.1107/S2059798320006348](https://doi.org/10.1107/S2059798320006348). Applying it at scaling,
|
|
to groups of images rather than whole datasets, follows DIALS (`dials.scale`):
|
|
J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software
|
|
package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399
|
|
[doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198).
|
|
|
|
**Reasons for a weak stretch of a sweep** — named in the field's own words: "loss of centring" after
|
|
autoPROC's "loss of centring during crystal rotation" (cited under
|
|
[Experimental geometry and physical corrections](#experimental-geometry-and-physical-corrections)),
|
|
and "crystal rotating out of the beam" after the HKL-2000 manual (HKL: Otwinowski and Minor 1997,
|
|
cited under [Integration and rotation geometry](#integration-and-rotation-geometry)).
|
|
|
|
**Several sweeps merged as one** — each sweep's partials combined into fulls on their own, then the
|
|
fulls of all sweeps scaled together with per-sweep frame scales and a relative B, as XSCALE merges
|
|
data sets (W. Kabsch, "Integration, scaling, space-group assignment and post-refinement" (2010),
|
|
above). Ordering the sweeps by dose from their relative B follows K. Diederichs, "Some aspects of
|
|
quantitative analysis and correction of radiation damage" (2006), Acta Cryst. D62, 96-101
|
|
[doi:10.1107/S0907444905031537](https://doi.org/10.1107/S0907444905031537).
|
|
|
|
**Reference sweep of a joint merge** — the sweep whose own merge reaches the finest resolution, as
|
|
KAMO takes the data set of the smallest Wilson B. K. Yamashita, K. Hirata and M. Yamamoto, "KAMO:
|
|
towards automated data processing for microcrystals" (2018), Acta Cryst. D74, 441-449
|
|
[doi:10.1107/S2059798318004576](https://doi.org/10.1107/S2059798318004576).
|
|
|
|
### Serial data
|
|
|
|
**Per-crystal post-refinement of stills** — refining per still the partiality parameters the spot
|
|
positions cannot pin (a small reorientation, the rocking width, the radial scale of the reciprocal
|
|
lattice) against the merged intensities, after PRIME and the mosaic-block model of Sauter et al.
|
|
M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter,
|
|
A. T. Brunger and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging
|
|
biological systems from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter,
|
|
J. Hattne, A. S. Brewster, N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and
|
|
physical properties from single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not
|
|
verified here, so not given).
|
|
|
|
**Indexing ambiguity without a reference** — putting each crystal into the indexing in which its
|
|
intensities agree best with all the others, iterated to convergence, the method CrystFEL's
|
|
`ambigator` also implements; rugnux measures the agreement as a product of standardised amplitudes
|
|
rather than a correlation. W. Brehm and K. Diederichs, "Breaking the indexing ambiguity in serial
|
|
crystallography" (2014), Acta Cryst. D70, 101-109
|
|
[doi:10.1107/S1399004713025431](https://doi.org/10.1107/S1399004713025431). The same objective
|
|
settles the indexing of each sweep of a joint merge, decided over all pairs at once as dials.cosym
|
|
does: R. J. Gildea and G. Winter, "Determination of Patterson group symmetry from sparse
|
|
multi-crystal data sets in the presence of an indexing ambiguity" (2018), Acta Cryst. D74, 405-410
|
|
[doi:10.1107/S2059798318002978](https://doi.org/10.1107/S2059798318002978).
|
|
|
|
### Data quality and anisotropy
|
|
|
|
**Data-quality statistics** — R_meas and R_pim, CC1/2 and CC\*, the per-shell CC(model, data), and
|
|
the reporting of I/sigma(I) follow the established conventions. K. Diederichs and P. A. Karplus,
|
|
"Improved R-factors for diffraction data analysis in macromolecular crystallography" (1997),
|
|
Nat. Struct. Biol. 4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269);
|
|
P. A. Karplus and K. Diederichs, "Linking crystallographic model and data quality" (2012), Science
|
|
336, 1030-1033 [doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231); M. S. Weiss,
|
|
"Global indicators of X-ray data quality" (2001), J. Appl. Cryst. 34, 130-135
|
|
[doi:10.1107/S0021889800018227](https://doi.org/10.1107/S0021889800018227); K. Diederichs and
|
|
P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222
|
|
[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121).
|
|
|
|
**Isomorphism of sweeps** — CC_AB / (CC\*_A CC\*_B) as the cosine of the angle between two data
|
|
sets, on which a joint merge decides whether a sweep joins. K. Diederichs, "Dissecting random and
|
|
systematic differences between noisy composite data sets" (2017), Acta Cryst. D73, 286-293
|
|
[doi:10.1107/S2059798317000699](https://doi.org/10.1107/S2059798317000699).
|
|
|
|
**Diffraction anisotropy** — the overall fall-off as a single anisotropic displacement tensor, its
|
|
symmetry constraints and the determinacy of its deviatoric part only: S. Sheriff and
|
|
W. A. Hendrickson, "Description of overall anisotropy in diffraction from macromolecular crystals"
|
|
(1987), Acta Cryst. A43, 118-121
|
|
[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X). Fitting it to the
|
|
intensity distribution with sigma(I) taken into account: A. N. Popov and G. P. Bourenkov, "Choice of
|
|
data-collection parameters based on statistic modelling" (2003), Acta Cryst. D59, 1145-1153
|
|
[doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163). The directional
|
|
diffraction limits and the anisotropic deltaB reported as the range of the principal components
|
|
follow AIMLESS: P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?"
|
|
(2013), Acta Cryst. D69, 1204-1214
|
|
[doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061). That every run should
|
|
analyse and report its anisotropy is inspired by
|
|
[STARANISO](https://staraniso.globalphasing.org/): I. J. Tickle, C. Flensburg, P. Keller,
|
|
W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO
|
|
([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)),
|
|
Cambridge, United Kingdom: Global Phasing Ltd. Rugnux reports the anisotropy; it corrects no
|
|
intensity and removes no reflection on a directional criterion.
|
|
|
|
**Uncertainty conventions** — the IUCr Commission on Crystallographic Nomenclature's.
|
|
D. Schwarzenbach, S. C. Abrahams, H. D. Flack et al., "Statistical descriptors in crystallography:
|
|
Report of the IUCr Subcommittee on Statistical Descriptors" (1989), Acta Cryst. A45, 63-75
|
|
[doi:10.1107/S0108767388009596](https://doi.org/10.1107/S0108767388009596); D. Schwarzenbach,
|
|
S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descriptors in
|
|
crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995),
|
|
Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340).
|
|
|
|
**Unit-cell e.s.d.s** — that the e.s.d.s of a least-squares cell fit are too small is Herbstein's
|
|
survey, which is why the cell is also fitted on blocks of consecutive frames. F. H. Herbstein, "How
|
|
precise are measurements of unit-cell dimensions from single crystals?" (2000), Acta Cryst. B56,
|
|
547-557 [doi:10.1107/S010876810000269X](https://doi.org/10.1107/S010876810000269X).
|
|
|
|
**Measurability** — the fraction of acentric Bijvoet pairs significant at three sigma, reported
|
|
beside SigAno and CCanom, is phenix.xtriage's. P. H. Zwart, "Anomalous signal indicators in protein
|
|
crystallography" (2005), Acta Cryst. D61, 1437-1448
|
|
[doi:10.1107/S0907444905023589](https://doi.org/10.1107/S0907444905023589).
|
|
|
|
**Ice in the merged intensities** — reading ice off a merge's own intensities at the hexagonal-ice
|
|
ring positions, and the table of the rings' line strengths, are phenix.xtriage's; rugnux's comparison
|
|
is local in resolution where xtriage's is against the whole Wilson curve. P. H. Zwart,
|
|
R. W. Grosse-Kunstleve and P. D. Adams, "Xtriage and Fest: automatic assessment of X-ray data and
|
|
substructure structure factor estimation" (2005), CCP4 Newsletter on Protein Crystallography 43 (no
|
|
DOI).
|
|
|
|
**Advice on the next collection** — the rotation a sweep needs and the blind region about the
|
|
spindle that a tilted second sweep recovers. Z. Dauter, "Data-collection strategies" (1999), Acta
|
|
Cryst. D55, 1703-1717 (DOI not verified here).
|
|
|
|
### Model placement and molecular replacement
|
|
|
|
**Bulk-solvent correction and overall scaling** — the flat bulk-solvent model, its starting values
|
|
and the physically meaningful range of its two parameters: A. Fokine and A. Urzhumtsev, "Flat
|
|
bulk-solvent model: obtaining optimal parameters" (2002), Acta Cryst. D58, 1387-1392
|
|
[doi:10.1107/S0907444902010284](https://doi.org/10.1107/S0907444902010284). The mask (the region a
|
|
probe cannot reach, shrunk back), drawn with the radii REFMAC5 uses: J.-S. Jiang and A. T. Brünger,
|
|
"Protein hydration observed by X-ray diffraction: solvation properties of penicillopepsin and
|
|
neuraminidase crystal structures" (1994), J. Mol. Biol. 243, 100-115
|
|
[doi:10.1006/jmbi.1994.1633](https://doi.org/10.1006/jmbi.1994.1633). The grid search over the
|
|
solvent pair with scale and anisotropic B refitted at each point: P. V. Afonine,
|
|
R. W. Grosse-Kunstleve and P. D. Adams, "A robust bulk-solvent correction and anisotropic scaling
|
|
procedure" (2005), Acta Cryst. D61, 850-855
|
|
[doi:10.1107/S0907444905007894](https://doi.org/10.1107/S0907444905007894). The fit is unweighted,
|
|
as there and in REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu, R. A. Steiner,
|
|
R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, "REFMAC5 for the refinement of macromolecular
|
|
crystal structures" (2011), Acta Cryst. D67, 355-367
|
|
[doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314).
|
|
|
|
**sigma_A map coefficients** — the Rice and Woolfson likelihoods of |Fo| given |Fc| and sigma_A, the
|
|
figure of merit m and the scale D, and the bias-corrected coefficients 2mFo-DFc (acentric) and mFo
|
|
(centric). R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures
|
|
with errors" (1986), Acta Cryst. A42, 140-149
|
|
[doi:10.1107/S0108767386099622](https://doi.org/10.1107/S0108767386099622).
|
|
|
|
**[ANODE](https://doi.org/10.1107/S0021889811041768)** — reading the anomalous difference map at the
|
|
atoms of a supplied model and reporting the strongest sites by name, instead of searching the map for
|
|
blobs. A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density calculation" (2011),
|
|
J. Appl. Cryst. 44, 1285-1287
|
|
[doi:10.1107/S0021889811041768](https://doi.org/10.1107/S0021889811041768).
|
|
|
|
**Uniform random rotations** — the null a supplied model, and a molecular-replacement search, are
|
|
scored against draws rotations uniformly from SO(3) through a uniform random unit quaternion.
|
|
K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk, Academic Press (1992),
|
|
124-132 (no DOI).
|
|
|
|
**Variable projection** — the rigid-body placement re-fits the overall scale at every step and folds
|
|
that re-fit into its Jacobian, in Kaufman's simplified form of Golub and Pereyra's derivative.
|
|
G. H. Golub and V. Pereyra, "The Differentiation of Pseudo-Inverses and Nonlinear Least Squares
|
|
Problems Whose Variables Separate" (1973), SIAM J. Numer. Anal. 10, 413-432
|
|
[doi:10.1137/0710036](https://doi.org/10.1137/0710036); L. Kaufman, "A variable projection method
|
|
for solving separable nonlinear least squares problems" (1975), BIT 15, 49-57
|
|
[doi:10.1007/BF01932995](https://doi.org/10.1007/BF01932995).
|
|
|
|
**Predicted models** — `--model` (and `--mr`) place an AlphaFold prediction by molecular replacement.
|
|
J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction with
|
|
AlphaFold" (2021), Nature 596, 583-589
|
|
[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); the models are those of
|
|
the AlphaFold Protein Structure Database: M. Varadi, S. Anyango, M. Deshpande et al., "AlphaFold
|
|
Protein Structure Database: massively expanding the structural coverage of protein-sequence space
|
|
with high-accuracy models" (2022), Nucleic Acids Res. 50, D439-D444
|
|
[doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061). The model is prepared as
|
|
phenix.process_predicted_model prepares one (residues below pLDDT 70 removed, B factors from the
|
|
predicted coordinate error) and cut into rigid domains by its predicted aligned error:
|
|
R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold models to work with
|
|
phenix.process_predicted_model and ISOLDE" (2022), Acta Cryst. D78, 1303-1314
|
|
[doi:10.1107/S2059798322010026](https://doi.org/10.1107/S2059798322010026); the error itself is
|
|
AlphaFold's: J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction
|
|
with AlphaFold" (2021), Nature 596, 583-589
|
|
[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2).
|
|
|
|
**Molecular replacement search** — scoring each orientation of the model by interpolating its
|
|
molecular transform, computed once by FFT, as AMoRe does: J. Navaza, "AMoRe: an automated package for
|
|
molecular replacement" (1994), Acta Cryst. A50, 157-163
|
|
[doi:10.1107/S0108767393007597](https://doi.org/10.1107/S0108767393007597). The full-symmetry
|
|
translation function, all pairs of symmetry copies on one Fourier synthesis: R. A. Crowther and
|
|
D. M. Blow, "A method of positioning a known molecule in an unknown crystal structure" (1967), Acta
|
|
Cryst. 23, 544-548 [doi:10.1107/S0365110X67003172](https://doi.org/10.1107/S0365110X67003172).
|
|
A placement is a solution, as Phaser calls one, where its translation-function Z-score reaches 8:
|
|
A. J. McCoy, R. W. Grosse-Kunstleve, P. D. Adams, M. D. Winn, L. C. Storoni and R. J. Read, "Phaser
|
|
crystallographic software" (2007), J. Appl. Cryst. 40, 658-674
|
|
[doi:10.1107/S0021889807021206](https://doi.org/10.1107/S0021889807021206).
|
|
|
|
**Cell contents** — whether one copy fits the cell is read off the Matthews coefficient:
|
|
B. W. Matthews, "Solvent content of protein crystals" (1968), J. Mol. Biol. 33, 491-497
|
|
[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836%2868%2990205-2); how many
|
|
copies to search for is weighed by its distribution over protein crystals: K. A. Kantardjieff and
|
|
B. Rupp, "Matthews coefficient probabilities: Improved estimates for unit cell contents of proteins,
|
|
DNA, and protein-nucleic acid complex crystals" (2003), Protein Sci. 12, 1865-1871
|
|
[doi:10.1110/ps.0350503](https://doi.org/10.1110/ps.0350503). The placements of several copies are
|
|
compared by least-squares superposition: W. Kabsch, "A solution for the best rotation to relate two
|
|
sets of vectors" (1976), Acta Cryst. A32, 922-923
|
|
[doi:10.1107/S0567739476001873](https://doi.org/10.1107/S0567739476001873).
|
|
|
|
**Likelihood scoring of placements** — the log-likelihood gain from the Rice and Woolfson
|
|
distributions of the normalised amplitudes given the model's, with sigma_A falling off with resolution
|
|
for the model's expected coordinate error as Luzzati's D: V. Luzzati, "Traitement statistique des
|
|
erreurs dans la détermination des structures cristallines" (1952), Acta Cryst. 5, 802-810
|
|
[doi:10.1107/S0365110X52002161](https://doi.org/10.1107/S0365110X52002161); R. J. Read, "Pushing the boundaries of molecular replacement with maximum
|
|
likelihood" (2001), Acta Cryst. D57, 1373-1382
|
|
[doi:10.1107/S0907444901012471](https://doi.org/10.1107/S0907444901012471); the gain read as a
|
|
Z-score over the same orientation at other positions, as the likelihood-enhanced translation
|
|
function's is: A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni and R. J. Read,
|
|
"Likelihood-enhanced fast translation functions" (2005), Acta Cryst. D61, 458-464
|
|
[doi:10.1107/S0907444905001617](https://doi.org/10.1107/S0907444905001617); and a model whose
|
|
expected log-likelihood gain is too small for a placement to stand out is not searched for:
|
|
A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason, B. Lohkamp and R. J. Read,
|
|
"Ab initio solution of macromolecular crystal structures without direct methods" (2017), Proc. Natl.
|
|
Acad. Sci. USA 114, 3637-3641 [doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114).
|
|
Translational NCS is looked for in the native Patterson before the search, read as under
|
|
[Translational pseudo-symmetry](#space-group-twinning-and-pseudo-symmetry). The logarithm of the
|
|
Bessel function I0 is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun,
|
|
"Handbook of Mathematical Functions" (1964), National Bureau of Standards, 9.8.1-9.8.2.
|
|
|
|
### Structure refinement
|
|
|
|
**Likelihood target** — the Rice distribution of an observed amplitude given the model's, and its
|
|
centric counterpart. N. S. Pannu and R. J. Read, "Improved structure refinement through maximum
|
|
likelihood" (1996), Acta Cryst. A52, 659-668
|
|
[doi:10.1107/S0108767396004370](https://doi.org/10.1107/S0108767396004370); G. N. Murshudov,
|
|
A. A. Vagin and E. J. Dodson, "Refinement of macromolecular structures by the maximum-likelihood
|
|
method" (1997), Acta Cryst. D53, 240-255
|
|
[doi:10.1107/S0907444996012255](https://doi.org/10.1107/S0907444996012255). Its parameters are fitted
|
|
in resolution bins, the variance on the free reflections and D on the working ones, as REFMAC5
|
|
divides them (Murshudov et al. 2011, above); estimating them on free reflections follows
|
|
V. Y. Lunin and T. P. Skovoroda, "R-free likelihood-based estimates of errors for phases calculated
|
|
from atomic models" (1995), Acta Cryst. A51, 880-887
|
|
[doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin,
|
|
P. V. Afonine and A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors"
|
|
(2002), Acta Cryst. A58, 270-282
|
|
[doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046).
|
|
|
|
**Gradients by FFT** — the derivatives of the target with respect to the structure factors
|
|
transformed into a map and summed against each atom's density derivatives. R. C. Agarwal, "A new
|
|
least-squares refinement technique based on the fast Fourier transform algorithm" (1978), Acta
|
|
Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618).
|
|
|
|
**Minimiser** — L-BFGS (rugnux's own implementation). D. C. Liu and J. Nocedal, "On the limited
|
|
memory BFGS method for large scale optimization" (1989), Math. Program. 45, 503-528
|
|
[doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116); its two-loop recursion: J. Nocedal,
|
|
"Updating quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782
|
|
[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). Its
|
|
initial inverse Hessian is the target's sparse normal matrix - the X-ray target's curvature per atom
|
|
on the diagonal and the restraints' Gauss-Newton matrix in full - as REFMAC refines on its normal
|
|
matrix (Murshudov, Vagin & Dodson 1997, above), solved by conjugate gradients: M. R. Hestenes and
|
|
E. Stiefel, "Methods of conjugate gradients for solving linear systems" (1952), J. Res. Natl. Bur.
|
|
Stand. 49, 409-436 [doi:10.6028/jres.049.044](https://doi.org/10.6028/jres.049.044), each residual
|
|
made orthogonal again to the earlier ones against the loss of orthogonality in rounding described by
|
|
C. C. Paige, "Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix" (1976),
|
|
J. Inst. Maths Applics 18, 341-349 (DOI not verified here), and A. Greenbaum, "Behavior of
|
|
slightly perturbed Lanczos and conjugate-gradient recurrences" (1989), Linear Algebra Appl. 113, 7-63
|
|
[doi:10.1016/0024-3795(89)90285-1](https://doi.org/10.1016/0024-3795(89)90285-1). Each atom's
|
|
diagonal is raised to hold its step within the step bound, a trust region per atom after the damping
|
|
of K. Levenberg, "A method for the solution of certain non-linear problems in least squares" (1944),
|
|
Q. Appl. Math. 2, 164-168 [doi:10.1090/qam/10666](https://doi.org/10.1090/qam/10666), and D. W.
|
|
Marquardt, "An algorithm for least-squares estimation of nonlinear parameters" (1963), J. Soc. Ind.
|
|
Appl. Math. 11, 431-441 [doi:10.1137/0111030](https://doi.org/10.1137/0111030).
|
|
|
|
**Geometry restraints** — those of the CCP4 Monomer Library, a subset of which rugnux carries (see
|
|
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). A. A. Vagin, R. A. Steiner, A. A. Lebedev,
|
|
L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5 dictionary: organization of prior
|
|
chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60, 2184-2195
|
|
[doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510). They are applied
|
|
through GEMMI's topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, "GEMMI:
|
|
A library for structural biology" (2022), J. Open Source Softw. 7, 4200
|
|
[doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). The torsion derivatives follow
|
|
A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion
|
|
angles in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141
|
|
[doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T](https://doi.org/10.1002/(SICI)1096-987X(19960715)17:9%3C1132::AID-JCC5%3E3.0.CO;2-T).
|
|
|
|
**Riding hydrogens** — placed on their parent atoms by GEMMI's riding-hydrogen code, which follows
|
|
D. Liebschner, P. V. Afonine, A. G. Urzhumtsev and P. D. Adams, "Implementation of the riding
|
|
hydrogen model in CCTBX to support the next generation of X-ray and neutron joint refinement in
|
|
Phenix" (2020), Methods Enzymol. 634, 177-199
|
|
[doi:10.1016/bs.mie.2020.01.007](https://doi.org/10.1016/bs.mie.2020.01.007).
|
|
|
|
**B-factor restraints** — bonded atoms, and atoms in contact, restrained to each other in the
|
|
Kullback-Leibler form REFMAC5 uses (Murshudov et al. 2011, above); the divergence itself is
|
|
S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22,
|
|
79-86 [doi:10.1214/aoms/1177729694](https://doi.org/10.1214/aoms/1177729694). The overall isotropic
|
|
B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012, below).
|
|
|
|
**Weights and R-free** — the weights of the X-ray term against the geometry and B restraints start
|
|
from the ratios of the gradients, as X-PLOR's crystallographic weight was first set: A. T. Brünger,
|
|
M. Karplus and G. A. Petsko, "Crystallographic refinement by simulated annealing: application to
|
|
crambin" (1989), Acta Cryst. A45, 50-61
|
|
[doi:10.1107/S0108767388009195](https://doi.org/10.1107/S0108767388009195); and are chosen among
|
|
short refinements by R-free with the geometry held within bounds, as phenix.refine chooses them:
|
|
P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd, N. W. Moriarty, M. Mustyakimov,
|
|
T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams, "Towards automated crystallographic
|
|
structure refinement with phenix.refine" (2012), Acta Cryst. D68, 352-367
|
|
[doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself:
|
|
A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal
|
|
structures" (1992), Nature 355, 472-475 [doi:10.1038/355472a0](https://doi.org/10.1038/355472a0).
|
|
|
|
**What the free set decides** — the free set decides the weights, the displacement model and whether the model
|
|
as given is kept, and only where its two measures agree: a candidate is taken where its free $-\ln L$ is lower and its R-free not higher
|
|
than the incumbent's by more than the paired standard error of the difference; every other step of the
|
|
refinement is taken on the work-set target. The approach of deciding on more than one free measure, and of keeping the
|
|
free set for few choices, is PDB-REDO's: R. P. Joosten, K. Joosten, G. N. Murshudov and A. Perrakis,
|
|
"PDB_REDO: constructive validation, more than just looking for errors" (2012), Acta Cryst. D68, 484-496
|
|
[doi:10.1107/S0907444911054515](https://doi.org/10.1107/S0907444911054515), and R. P. Joosten, F. Long,
|
|
G. N. Murshudov and A. Perrakis, "The PDB_REDO server for macromolecular structure model optimization" (2014),
|
|
IUCrJ 1, 213-220 [doi:10.1107/S2052252514009324](https://doi.org/10.1107/S2052252514009324).
|
|
|
|
**Robust scaling of the model** — the overall scale and anisotropic B are fitted by iteratively
|
|
reweighted least squares with the pseudo-Huber weight. P. J. Huber, "Robust estimation of a location
|
|
parameter" (1964), Ann. Math. Statist. 35, 73-101
|
|
[doi:10.1214/aoms/1177703732](https://doi.org/10.1214/aoms/1177703732), in the smooth form of
|
|
P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, "Deterministic edge-preserving
|
|
regularization in computed imaging" (1997), IEEE Trans. Image Process. 6, 298-311
|
|
[doi:10.1109/83.551699](https://doi.org/10.1109/83.551699).
|
|
|
|
**Metal-site restraints** — the bonds of metal ions to the donor atoms the model's LINK records name
|
|
are restrained at the distances of the Monomer Library's own metal-site statistics (`metals.json`,
|
|
embedded as `monomers/metals.cif`), in line with the surveys of M. M. Harding, "Geometry of
|
|
metal-ligand interactions in proteins" (2001), Acta Cryst. D57, 401-411
|
|
[doi:10.1107/S0907444900019168](https://doi.org/10.1107/S0907444900019168) and "Small revisions to
|
|
predicted distances around metal sites in proteins" (2006), Acta Cryst. D62, 678-682
|
|
[doi:10.1107/S0907444906014594](https://doi.org/10.1107/S0907444906014594), and, for Mg²⁺, Na⁺ and
|
|
K⁺ at nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to
|
|
nucleobase nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004
|
|
[doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175).
|
|
|
|
**Anisotropic displacements** — the TLS model of a rigid body's translation, libration and their
|
|
correlation: V. Schomaker and K. N. Trueblood, "On the rigid-body motion of molecules in crystals"
|
|
(1968), Acta Cryst. B24, 63-76
|
|
[doi:10.1107/S0567740868001718](https://doi.org/10.1107/S0567740868001718); refined against the
|
|
likelihood with a residual B per atom, the B column read as total or residual, as in M. D. Winn,
|
|
M. N. Isupov and G. N. Murshudov, "Use of TLS parameters to model anisotropic displacements in
|
|
macromolecular refinement" (2001), Acta Cryst. D57, 122-133
|
|
[doi:10.1107/S0907444900014736](https://doi.org/10.1107/S0907444900014736). The chains are cut into
|
|
TLS groups as TLSMD cuts them: J. Painter and E. A. Merritt, "Optimal description of a protein
|
|
structure in terms of multiple groups undergoing TLS motion" (2006), Acta Cryst. D62, 439-450
|
|
[doi:10.1107/S0907444906005270](https://doi.org/10.1107/S0907444906005270). Atoms with a U of their
|
|
own are restrained by the rigid-bond postulate of F. L. Hirshfeld, "Can X-ray data distinguish
|
|
bonding effects from vibrational smearing?" (1976), Acta Cryst. A32, 239-244
|
|
[doi:10.1107/S0567739476000533](https://doi.org/10.1107/S0567739476000533), in the form of SHELXL's
|
|
RIGU: A. Thorn, B. Dittrich and G. M. Sheldrick, "Enhanced rigid-bond restraints" (2012), Acta
|
|
Cryst. A68, 448-451 [doi:10.1107/S0108767312014535](https://doi.org/10.1107/S0108767312014535),
|
|
with SHELXL's SIMU and ISOR restraints as G. M. Sheldrick, "Crystal structure refinement with
|
|
SHELXL" (2015), Acta Cryst. C71, 3-8
|
|
[doi:10.1107/S2053229614024218](https://doi.org/10.1107/S2053229614024218) documents them. Which
|
|
displacement model a refinement keeps is decided on the free set rather than by a rule of
|
|
resolution: E. A. Merritt, "To B or not to B: a question of resolution?" (2012), Acta Cryst. D68,
|
|
468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320). Between a model
|
|
with more displacement parameters and one with fewer, the R-factor ratio test of W. C. Hamilton, "Significance
|
|
tests on the crystallographic R factor" (1965), Acta Cryst. 18, 502-510 (DOI not verified), decides first,
|
|
over a range of restraint weights as Merritt (2012) applies it, with PDB-REDO's (Joosten et al. 2012, above)
|
|
95%/30% rule; its F point is from E. Paulson,
|
|
"An approximate normalization of the analysis of variance distribution" (1942), Ann. Math. Statist. 13,
|
|
233-235 [doi:10.1214/aoms/1177731608](https://doi.org/10.1214/aoms/1177731608).
|
|
|
|
**Population and crossover** — the members' refinement on blurred data is a continuation (homotopy) on the
|
|
resolution, the deformation of the landscape of L. Piela, J. Kostrowicki and H. A. Scheraga, "On the
|
|
multiple-minima problem in the conformational analysis of molecules: deformation of the potential energy
|
|
hypersurface by the diffusion equation method" (1989), J. Phys. Chem. 93, 3339-3346
|
|
[doi:10.1021/j100345a090](https://doi.org/10.1021/j100345a090), applied as the "low resolution first" of
|
|
A. Jack and M. Levitt, "Refinement of large structures by simultaneous minimization of energy and R factor"
|
|
(1978), Acta Cryst. A34, 931-935 [doi:10.1107/S0567739478001904](https://doi.org/10.1107/S0567739478001904).
|
|
The kicked members and the map averaged over them follow J. Praznikar, P. V. Afonine, G. Guncar, P. D. Adams
|
|
and D. Turk, "Averaged kick maps: less noise, more signal... and probably less bias" (2009), Acta Cryst. D65,
|
|
921-931 [doi:10.1107/S0907444909021933](https://doi.org/10.1107/S0907444909021933). Taking each residue from
|
|
the member that fits best is the combination of J. C. Nwachukwu, M. R. Southern, J. R. Kiefer, P. V. Afonine,
|
|
P. D. Adams, T. C. Terwilliger and K. W. Nettles, "Improved crystallographic
|
|
structures using extensive combinatorial refinement" (2013), Structure 21, 1923-1930
|
|
[doi:10.1016/j.str.2013.07.025](https://doi.org/10.1016/j.str.2013.07.025), made on one map inside the
|
|
refinement rather than offline on whole models.
|
|
|
|
**NCS restraints** — copies found by sequence alignment with REFMAC5's criterion (Murshudov et al.
|
|
2011, above) and restrained locally on pairs of corresponding interatomic distances under the
|
|
Geman-McClure loss, as REFMAC5's local NCS restraints are, with the distance range of
|
|
R. A. Nicholls, F. Long and G. N. Murshudov, "Low-resolution refinement tools in REFMAC5" (2012),
|
|
Acta Cryst. D68, 404-417 [doi:10.1107/S090744491105606X](https://doi.org/10.1107/S090744491105606X).
|
|
The superposition behind the rms criterion is GEMMI's QCP: D. L. Theobald, "Rapid calculation of
|
|
RMSDs using a quaternion-based characteristic polynomial" (2005), Acta Cryst. A61, 478-480
|
|
[doi:10.1107/S0108767305015266](https://doi.org/10.1107/S0108767305015266); P. Liu, D. K. Agrafiotis
|
|
and D. L. Theobald, "Fast determination of the optimal rotational matrix for macromolecular
|
|
superpositions" (2010), J. Comput. Chem. 31, 1561-1563
|
|
[doi:10.1002/jcc.21439](https://doi.org/10.1002/jcc.21439).
|
|
The case for local restraints over a superposition, decided by R-free: G. J. Kleywegt, "Use of
|
|
non-crystallographic symmetry in protein structure refinement" (1996), Acta Cryst. D52, 842-857
|
|
[doi:10.1107/S0907444995016477](https://doi.org/10.1107/S0907444995016477); I. Usón, E. Pohl,
|
|
T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz and G. M. Sheldrick, "1.7 Å structure of the
|
|
stabilized REIv mutant T39K. Application of local NCS restraints" (1999), Acta Cryst. D55,
|
|
1158-1167 [doi:10.1107/S0907444999003972](https://doi.org/10.1107/S0907444999003972); and
|
|
J. J. Headd, N. Echols, P. V. Afonine, N. W. Moriarty, R. J. Gildea and P. D. Adams, "Flexible
|
|
torsion-angle noncrystallographic symmetry restraints for improved macromolecular structure
|
|
refinement" (2014), Acta Cryst. D70, 1346-1356
|
|
[doi:10.1107/S1399004714003277](https://doi.org/10.1107/S1399004714003277), whose comparison is why
|
|
the copies' B factors are refined independently.
|
|
|
|
**Special positions** — an atom on a special position is kept on it, its shifts confined to the
|
|
subspace its site symmetry leaves unchanged. R. W. Grosse-Kunstleve and P. D. Adams, "Algorithms for
|
|
deriving crystallographic space-group information. II. Treatment of special positions" (2002), Acta
|
|
Cryst. A58, 60-65 [doi:10.1107/S0108767301016658](https://doi.org/10.1107/S0108767301016658).
|
|
|
|
**Side chains as a discrete field** — the rotamers of the side chains searched as a combinatorial problem rather
|
|
than by a gradient, with the likelihood of the working reflections as the energy and iterated conditional modes
|
|
(each residue in turn put in its best state given the others) as the solver; small residues first and arginine
|
|
last, as PDB-REDO rebuilds. J. Desmet, M. De Maeyer, B. Hazes and I. Lasters, "The dead-end elimination theorem
|
|
and its use in protein side-chain positioning" (1992), Nature 356, 539-542
|
|
[doi:10.1038/356539a0](https://doi.org/10.1038/356539a0); J. Besag, "On the statistical analysis of dirty
|
|
pictures" (1986), J. R. Statist. Soc. B48, 259-279
|
|
[doi:10.1111/j.2517-6161.1986.tb01412.x](https://doi.org/10.1111/j.2517-6161.1986.tb01412.x); R. P. Joosten,
|
|
K. Joosten, S. X. Cohen, G. Vriend and A. Perrakis, "Automatic rebuilding and optimization of crystallographic
|
|
structures in the Protein Data Bank" (2011), Bioinformatics 27, 3392-3398
|
|
[doi:10.1093/bioinformatics/btr590](https://doi.org/10.1093/bioinformatics/btr590).
|
|
|
|
### Solvent, ions and unmodelled density
|
|
|
|
**Unmodelled density** — the connected regions of the difference map above a cutoff, found by flood
|
|
fill as Coot's "unmodelled blobs" are, in GEMMI's implementation. P. Emsley, B. Lohkamp, W. G. Scott
|
|
and K. Cowtan, "Features and development of Coot" (2010), Acta Cryst. D66, 486-501
|
|
[doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493).
|
|
|
|
**Water build** — the documented ordered-solvent defaults of phenix.refine (difference-map peaks
|
|
from 3 σ, picking to 2.8 Å, waters added, filtered and refined between macro-cycles; Afonine et al.
|
|
2012, above); the distance band to the model, widened a little from the 2.4-3.2 Å Coot's Find Waters
|
|
keeps (Emsley et al. 2010, above); and the removal of a water in negative difference density at
|
|
-3 σ (Echols et al. 2014, below).
|
|
|
|
**Ion stereochemistry** — the coordination bands, the Mg²⁺ exclusion zone and the clash shells of
|
|
the surveys of ions at nucleic acids by P. Auffinger's group. N. Naleem, A. Henning-Knechtel,
|
|
S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating,
|
|
diagnosing, and annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120
|
|
[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); F. Leonarski, A. Henning-Knechtel,
|
|
S. Kirmizialtin, E. Ennifar and P. Auffinger, "Principles of ion binding to RNA inferred from the
|
|
analysis of a 1.55 Å resolution bacterial ribosome structure – Part I: Mg2+" (2025), Nucleic Acids
|
|
Res. 53, gkae1148 [doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); F. Leonarski,
|
|
L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor Mg2+ inner-sphere binders but
|
|
excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192
|
|
[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118); and their 2017 survey (above).
|
|
The score that combines the criteria is rugnux's own.
|
|
|
|
**Ion identification** — the ligand sphere of the bond-valence sums, the reading of halides by their
|
|
donors, and telling a light ion from a water by its density: N. Echols, N. Morshed, P. V. Afonine,
|
|
A. J. McCoy, M. D. Miller, R. J. Read, J. S. Richardson, T. C. Terwilliger and P. D. Adams,
|
|
"Automated identification of elemental ions in macromolecular crystal structures" (2014), Acta Cryst.
|
|
D70, 1104-1114 [doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308);
|
|
N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental ion
|
|
identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158
|
|
[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241). The chloride's shell of
|
|
donors: O. Carugo, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC Struct.
|
|
Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8); halides at
|
|
hydrophobic surfaces: Z. Dauter and M. Dauter, "Entering a new phase: using solvent halide ions in
|
|
protein structure determination" (2001), Structure 9, R21-R26
|
|
[doi:10.1016/S0969-2126(01)00565-2](https://doi.org/10.1016/S0969-2126(01)00565-2).
|
|
|
|
**Bond-valence sums** — I. D. Brown and D. Altermatt, "Bond-valence parameters obtained from a
|
|
systematic analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247
|
|
[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of
|
|
N. E. Brese and M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197
|
|
[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by
|
|
H. Zheng, K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor,
|
|
"Data mining of iron(II) and iron(III) bond-valence parameters, and their relevance for
|
|
macromolecular crystallography" (2017), Acta Cryst. D73, 316-325
|
|
[doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584); weighted by occupancy
|
|
as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method able to identify
|
|
metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37
|
|
[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); the normalised vector
|
|
sum and its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski,
|
|
I. G. Shabalin, K. B. Handing and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation
|
|
tool" (2017), Acta Cryst. D73, 223-233
|
|
[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061).
|
|
|
|
**Anomalous scattering factors** — f″ at the data's wavelength, as GEMMI computes it (its port of
|
|
the Larch routines). D. T. Cromer and D. A. Liberman, "Anomalous dispersion calculations near to
|
|
and on the long-wavelength side of an absorption edge" (1981), Acta Cryst. A37, 267-268
|
|
[doi:10.1107/S0567739481000600](https://doi.org/10.1107/S0567739481000600), with the corrections of
|
|
L. Kissel and R. H. Pratt, "Corrections to tabulated anomalous-scattering factors" (1990), Acta
|
|
Cryst. A46, 170-175 [doi:10.1107/S0108767389010718](https://doi.org/10.1107/S0108767389010718).
|
|
|
|
## Software and computing methods
|
|
|
|
**GPU decompression** — decoding bitshuffle+LZ4 images on the GPU rather than on the host follows
|
|
Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4 data", HDF5 User Group
|
|
meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders). The CUDA kernels
|
|
are Jungfraujoch's own; the approach is his.
|
|
|
|
**Parallel connected components** — removing the islands of the bulk-solvent mask on the GPU with a
|
|
parallel union-find. D. P. Playne and K. Hawick, "A New Algorithm for Parallel Connected-Component
|
|
Labelling on GPUs" (2018), IEEE Trans. Parallel Distrib. Syst. 29, 1217-1230
|
|
[doi:10.1109/TPDS.2018.2799216](https://doi.org/10.1109/TPDS.2018.2799216).
|
|
|
|
**Correctly rounded integer division** — the FFT indexer's GPU peak search divides by a reciprocal
|
|
multiplication corrected once by the exact remainder. P. W. Markstein, "Computation of elementary
|
|
functions on the IBM RISC System/6000 processor" (1990), IBM J. Res. Dev. 34, 111-119.
|
|
|
|
**Anderson acceleration** — of the per-frame scaling loops of the rotation merge, in the type-II
|
|
form. H. F. Walker and P. Ni, "Anderson acceleration for fixed-point iterations" (2011), SIAM J.
|
|
Numer. Anal. 49, 1715-1735 [doi:10.1137/10078356X](https://doi.org/10.1137/10078356X), after
|
|
D. G. Anderson, "Iterative procedures for nonlinear integral equations" (1965), J. ACM 12, 547-560
|
|
[doi:10.1145/321296.321305](https://doi.org/10.1145/321296.321305).
|
|
|
|
**Bootstrap** — the interval reported for ISa. B. Efron, "Bootstrap methods: another look at the
|
|
jackknife" (1979), Ann. Statist. 7, 1-26
|
|
[doi:10.1214/aos/1176344552](https://doi.org/10.1214/aos/1176344552). Its draws are Poisson weights
|
|
hashed with the SplitMix64 finaliser of G. L. Steele Jr., D. Lea and C. H. Flood, "Fast splittable
|
|
pseudorandom number generators" (2014), Proc. OOPSLA 2014, 453-472
|
|
[doi:10.1145/2660193.2660195](https://doi.org/10.1145/2660193.2660195). The same finaliser chooses the images rugnux live
|
|
stills analyses with `stills_downsample`.
|
|
|
|
**Error-free floating-point arithmetic** — the single-precision exp shared between the CPU and an
|
|
Apple GPU (`BraggExp.h`) and the double-float sums of the refinement's Gauss-Newton products on the GPU
|
|
(`RefinementNewtonGPU.cu`): the argument reduction of W. J. Cody and W. Waite, "Software Manual for the
|
|
Elementary Functions" (1980), Prentice-Hall; the exact product (with Veltkamp's split) and Fast2Sum
|
|
of T. J. Dekker, "A floating-point technique for extending the available precision" (1971), Numer.
|
|
Math. 18, 224-242; and the 2Sum of D. E. Knuth, "The Art of Computer Programming, Vol. 2:
|
|
Seminumerical Algorithms", section 4.2.2.
|
|
|
|
**Colormaps** — Viridis, Magma and Inferno from Matplotlib, under its BSD-compatible license.
|
|
J. D. Hunter, "Matplotlib: A 2D graphics environment" (2007), Comput. Sci. Eng. 9, 90-95
|
|
[doi:10.1109/MCSE.2007.55](https://doi.org/10.1109/MCSE.2007.55).
|
|
|
|
**[Mol\*](https://molstar.org/)** — the web frontend's structure viewer. D. Sehnal, S. Bittrich,
|
|
M. Deshpande, R. Svobodová, K. Berka, V. Bazgier, S. Velankar, S. K. Burley, J. Koča and A. S. Rose,
|
|
"Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures"
|
|
(2021), Nucleic Acids Res. 49, W431-W437 [doi:10.1093/nar/gkab314](https://doi.org/10.1093/nar/gkab314).
|
|
Its setup follows the one Guillaume Gotthard (PSI) tuned for [AareDB](https://gitea.psi.ch/mx/aaredb),
|
|
the sample-management and data-pipeline system of the PSI MX beamlines: the Coot-style mouse
|
|
controls, the model representations and colours, and the display of the density maps. The virtual
|
|
trackball is a port of Coot's, itself a port of SGI's `trackball.c` by Gavin Bell.
|
|
|
|
## File formats read from a published specification
|
|
|
|
**CBF / imgCIF** — the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression and the
|
|
imgCIF `_axis` table from the specification alone; no CBFlib or other CBF code is used.
|
|
H. J. Bernstein and A. P. Hammersley, "Specification of the Crystallographic Binary File
|
|
(CBF/imgCIF)" (2006), International Tables for Crystallography Vol. G, 37-43
|
|
[doi:10.1107/97809553602060000729](https://doi.org/10.1107/97809553602060000729);
|
|
A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, "Image dictionary (imgCIF)" (2006),
|
|
International Tables for Crystallography Vol. G, 444-458
|
|
[doi:10.1107/97809553602060000746](https://doi.org/10.1107/97809553602060000746).
|
|
|
|
**PILATUS count-rate clipping** — that the count-rate correction clips an over-range pixel to the
|
|
cutoff rather than flagging it, which is how the miniCBF reader tells a saturated pixel. P. Trueb,
|
|
C. Dejoie, M. Kobas, P. Pattison, D. J. Peake, V. Radicci, B. A. Sobott, D. A. Walko and
|
|
C. Broennimann, "Bunch mode specific rate corrections for PILATUS3 detectors" (2015), J. Synchrotron
|
|
Rad. 22, 701-707 [doi:10.1107/S1600577515003288](https://doi.org/10.1107/S1600577515003288).
|
|
|
|
**d\*TREK SMV** — the SMV reader reads the d\*TREK header vocabulary of Rigaku's CrystalClear from
|
|
the headers themselves, interpreting the detector vectors as dxtbx does; no d\*TREK or dxtbx code is
|
|
used. J. W. Pflugrath, "The finer things in X-ray diffraction data collection" (1999), Acta Cryst.
|
|
D55, 1718-1725 [doi:10.1107/S090744499900935X](https://doi.org/10.1107/S090744499900935X);
|
|
[dxtbx](https://github.com/cctbx/dxtbx): J. M. Parkhurst, A. S. Brewster, L. Fuentes-Montero,
|
|
D. G. Waterman et al., "dxtbx: the diffraction experiment toolbox" (2014), J. Appl. Cryst. 47,
|
|
1459-1465 [doi:10.1107/S1600576714011996](https://doi.org/10.1107/S1600576714011996).
|
|
|
|
## Public diffraction data used for testing
|
|
|
|
In addition to in-house datasets collected at SLS 2.0, Jungfraujoch is tested against public
|
|
diffraction data collected on other people's beamlines, on detectors and in file formats we do not
|
|
produce ourselves. Every dataset used, the DOI to cite for it, and the deposition it belongs to are
|
|
listed in [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) and
|
|
[EXTERNAL_TEST_DATA_STILLS](EXTERNAL_TEST_DATA_STILLS.md); we thank the depositors, and the
|
|
repositories that make the data findable and citable.
|
|
|
|
**[IRRMC](https://proteindiffraction.org/)**, the Integrated Resource for Reproducibility in
|
|
Macromolecular Crystallography (Minor lab, University of Virginia), is the source of the largest
|
|
share of them; it releases its data under CC0 and asks that the DOI of the dataset be cited.
|
|
M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng, D. R. Cooper,
|
|
M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of macromolecular
|
|
diffraction experiments" (2016), Acta Cryst. D72, 1181-1193
|
|
[doi:10.1107/S2059798316014716](https://doi.org/10.1107/S2059798316014716); M. Grabowski,
|
|
M. Cymborowski, P. J. Porebski, T. Osinski, I. G. Shabalin, D. R. Cooper and W. Minor, "The
|
|
Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first
|
|
four years" (2019), Struct. Dyn. 6, 064301
|
|
[doi:10.1063/1.5128672](https://doi.org/10.1063/1.5128672).
|
|
|
|
**[SBGrid Data Bank](https://data.sbgrid.org/)**, the structural biology community's data
|
|
publication service (SBGrid Consortium, Harvard Medical School). P. A. Meyer, S. Socias, J. Key,
|
|
E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the structural biology data grid
|
|
supports live analysis" (2016), Nat. Commun. 7, 10882
|
|
[doi:10.1038/ncomms10882](https://doi.org/10.1038/ncomms10882).
|
|
|
|
**[Zenodo](https://zenodo.org/)**, CERN's open repository, hosts datasets deposited there directly
|
|
by the groups that collected them. European Organization for Nuclear Research and OpenAIRE,
|
|
"Zenodo" (2013), CERN [doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). The IUCrData
|
|
Raw Data Letters describing some of those deposits are cited on the
|
|
[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page.
|
|
|
|
**[CXIDB](https://cxidb.org/)**, the Coherent X-ray Imaging Data Bank, is the source of the serial
|
|
(XFEL stills) datasets. F. R. N. C. Maia, "The Coherent X-ray Imaging Data Bank" (2012), Nat.
|
|
Methods 9, 854-855 [doi:10.1038/nmeth.2110](https://doi.org/10.1038/nmeth.2110).
|
|
|
|
**[MXRDR](https://mxrdr.icm.edu.pl/)**, the Macromolecular Xtallography Raw Data Repository
|
|
(ICM, University of Warsaw), releases its data under CC0 and asks that the DOI of the dataset be
|
|
cited.
|
|
|
|
**[XRDa](https://xrda.pdbj.org/)**, the Xtal Raw Data Archive (Protein Data Bank Japan), publishes
|
|
raw diffraction images and mints a DOI for each, which it asks to be cited. It has no canonical
|
|
citation paper.
|
|
|
|
**The [ESRF data portal](https://data.esrf.fr/)**, through which the European Synchrotron publishes
|
|
raw data under its data policy (CC BY 4.0, with the dataset DOI to be cited). R. Dimper, A. Götz,
|
|
A. De Maria, V. A. Solé, M. Chaillet and B. Lebayle, "ESRF Data Policy, Storage, and Services"
|
|
(2019), Synchrotron Rad. News 32, 7-12
|
|
[doi:10.1080/08940886.2019.1608119](https://doi.org/10.1080/08940886.2019.1608119).
|
|
|
|
**The [Keele University research data repository](https://researchdata.keele.ac.uk/)** and
|
|
**[UQ eSpace](https://espace.library.uq.edu.au/)** (The University of Queensland) host the raw
|
|
images of datasets deposited there by the groups that collected them; the dataset DOIs are cited on
|
|
the [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page.
|
|
|
|
**The [Protein Data Bank](https://www.rcsb.org/)** supplies, through the RCSB PDB data API, the
|
|
beamline, resolution, space group and unit cell quoted for each dataset, and the deposited
|
|
coordinates and structure factors of the public entries the refinement battery starts from.
|
|
H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and
|
|
P. E. Bourne, "The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242
|
|
[doi:10.1093/nar/28.1.235](https://doi.org/10.1093/nar/28.1.235).
|
|
|
|
**The [Crystallography Open Database](https://www.crystallography.net/cod/)** supplies the reference
|
|
cells of the small-molecule sets. S. Gražulis, D. Chateigner, R. T. Downs, A. F. T. Yokochi,
|
|
M. Quirós, L. Lutterotti, E. Manakova, J. Butkus, P. Moeck and A. Le Bail, "Crystallography Open
|
|
Database – an open-access collection of crystal structures" (2009), J. Appl. Cryst. 42, 726-729
|
|
[doi:10.1107/S0021889809016690](https://doi.org/10.1107/S0021889809016690); S. Gražulis,
|
|
A. Daškevič, A. Merkys et al., "Crystallography Open Database (COD): an open-access collection of
|
|
crystal structures and platform for world-wide collaboration" (2012), Nucleic Acids Res. 40,
|
|
D420-D427 [doi:10.1093/nar/gkr900](https://doi.org/10.1093/nar/gkr900).
|
|
|
|
**The [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/)** (EMBL-EBI and Google
|
|
DeepMind) supplies the start models of the battery's timing benchmark: the predictions of the
|
|
deposited entities of 9ZLO (AF-P17090-F1), 9S02 (AF-P32322-F1) and 9GJX (AF-Q65MG6-F1) and, for the
|
|
in-house benchmark, of horse myoglobin (AF-P68082-F1) and bovine cytochrome c (AF-P62894-F1), model
|
|
version 6, with their predicted aligned error, kept unmodified in `tools/battery/models/` under
|
|
CC BY 4.0. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction
|
|
with AlphaFold" (2021), Nature 596, 583-589
|
|
[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); M. Varadi,
|
|
S. Anyango, M. Deshpande et al., "AlphaFold Protein Structure Database: massively expanding the
|
|
structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res.
|
|
50, D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061); M. Varadi,
|
|
D. Bertoni, P. Magana et al., "AlphaFold Protein Structure Database in 2024: providing structure
|
|
coverage for over 214 million protein sequences" (2024), Nucleic Acids Res. 52, D368-D375
|
|
[doi:10.1093/nar/gkad1011](https://doi.org/10.1093/nar/gkad1011). The large case of both benchmarks
|
|
is the deposited model of PDB entry 4NWV, moved away from its place.
|
|
|
|
## Generative AI usage declaration
|
|
|
|
Large language models were used extensively in developing this code. Jungfraujoch development was
|
|
supported with JetBrains AI (mostly GPT models) to refactor and verify particular code fragments.
|
|
Rugnux was developed with the assistance of Claude Code (mostly the Opus model). This documentation
|
|
was written with the assistance of Claude Opus and Fable models.
|