Build Packages / Create release (push) Successful in 8s
Build Packages / Build documentation (push) Successful in 58s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m20s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m18s
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m15s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 22m57s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 24m42s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 26m35s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 28m29s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 29m35s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 30m23s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 30m10s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 25m27s
Build Packages / Generate python client (push) Successful in 30s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 28m55s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 25m12s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 31m44s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 29m43s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 28m23s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 31m29s
Build Packages / Unit tests (push) Successful in 1h35m1s
Build Packages / Validation Battery (push) Successful in 53m4s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVeDQqrcrEZy6XBUCMZvyT
936 lines
75 KiB
Markdown
936 lines
75 KiB
Markdown
# Acknowledgements
|
|
|
|
How to cite Jungfraujoch and Rugnux, and the papers that describe them, is on the
|
|
[Publications](PUBLICATIONS.md) page. In short:
|
|
|
|
- **Rugnux**: F. Leonarski and M. Wang, "Rugnux: from diffraction images to structure at synchrotrons
|
|
and XFELs in one program" (2026), bioRxiv
|
|
[doi:10.64898/2026.10.24.761522](https://doi.org/10.64898/2026.10.24.761522).
|
|
- **Jungfraujoch**: F. Leonarski, M. Brückner, C. Lopez-Cuenca, A. Mozzanica, H.-C. Stadler,
|
|
Z. Matěj, A. Castellane, B. Mesnet, J. Wojdyla, B. Schmitt and M. Wang, "Jungfraujoch:
|
|
hardware-accelerated data-acquisition system for kilohertz pixel-array X-ray detectors" (2023),
|
|
J. Synchrotron Rad. 30, 227-234 [doi:10.1107/S1600577522010268](https://doi.org/10.1107/S1600577522010268);
|
|
for real-time serial crystallography and live indexing feedback also J. Duan, H.-C. Stadler,
|
|
J. Nan et al., "Jungfraujoch: 38 GB s⁻¹ real-time serial crystallography – data acquisition and
|
|
live indexing feedback" (2026), J. Appl. Cryst. 59, 1668-1677
|
|
[doi:10.1107/S1600576726008083](https://doi.org/10.1107/S1600576726008083).
|
|
|
|
## Funding and support
|
|
|
|
The project is supported by:
|
|
* Innosuisse via Innovation Project "NextGenDCU high data rate acquisition system for X-ray detectors in structural biology applications" (101.535.1 IP-ENG; Apr 2023 - Sep 2025).
|
|
* ETH Domain via Open Research Data Contribute project (Jan - Dec 2023).
|
|
* AMD University Program with donation of licenses of Ethernet IP cores and Vivado software.
|
|
|
|
## Automatic processing as an expert system
|
|
|
|
**Global Phasing Ltd** — much of what rugnux tries to do was inspired by the work of Gérard Bricogne
|
|
and his colleagues at [Global Phasing](https://www.globalphasing.com/): the Fourier-transform
|
|
indexing its de-novo indexers descend from, which Bricogne proposed in 1986 (see
|
|
[Indexing](#indexing) below), the view of data processing as an expert system, the distrust of
|
|
image-header metadata, and the attention paid to diffraction anisotropy, each described below.
|
|
|
|
**[autoPROC](https://www.globalphasing.com/autoproc/)** — rugnux's goal is a fully automatic
|
|
pipeline that takes the decisions an experienced crystallographer would take while processing a
|
|
dataset, and reports each decision together with the evidence it rests on, so that a bare
|
|
invocation gives the best answer available. autoPROC is the inspiration and the target for that
|
|
behaviour, and two of rugnux's working principles come from it. The first, and the one that has
|
|
guided rugnux most, we owe to Clemens Vonrhein: a processing pipeline should not take the
|
|
experimental metadata it is given - the beam centre, the detector geometry, the goniometer and its
|
|
rotation axis - on trust, but verify it against the images and, where it is wrong, correct it. We
|
|
are grateful to him for arguing this so clearly and for so long. The second follows from it: the
|
|
beam centre should be determined from the images themselves; rugnux does this in its own way,
|
|
described in §1.4 of the [data-analysis reference](CPU_DATA_ANALYSIS_IMAGE.md). The
|
|
reasons rugnux gives for a weak stretch of a sweep also use autoPROC's wording ("loss of centring
|
|
during crystal rotation"). C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek,
|
|
T. Womack and G. Bricogne, "Data processing and analysis with the autoPROC toolbox" (2011),
|
|
Acta Cryst. D67, 293-302
|
|
[doi:10.1107/S0907444911007773](https://doi.org/10.1107/S0907444911007773).
|
|
|
|
**[STARANISO](https://staraniso.globalphasing.org/)** — the understanding that diffraction
|
|
anisotropy matters, and that every run should analyse and report it, was inspired by STARANISO.
|
|
The estimators rugnux uses for it are those credited under
|
|
[Scaling, merging and data quality](#scaling-merging-and-data-quality) below. I. J. Tickle,
|
|
C. Flensburg, P. Keller, W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO
|
|
([staraniso.globalphasing.org/cgi-bin/staraniso.cgi](https://staraniso.globalphasing.org/cgi-bin/staraniso.cgi)),
|
|
Cambridge, United Kingdom: Global Phasing Ltd.
|
|
|
|
## Crystallographic methods adopted from other packages
|
|
|
|
The analysis pipeline builds on methods first published, and in most cases first implemented, by
|
|
other crystallographic software. The code below is Jungfraujoch's own; the methods are theirs, and
|
|
are acknowledged here. Where a package's source was consulted this is said explicitly. Most of these
|
|
packages are neither linked nor vendored; the three that are - GEMMI, traccc and
|
|
fast-feedback-indexer - also carry a licence obligation, recorded in
|
|
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
|
|
|
|
### Spot finding
|
|
|
|
**[CrystFEL](https://www.desy.de/~twhite/crystfel/)** — spot finding, the three-ring integration
|
|
region, the serial/stills processing model, and the per-frame indexing acceptance test
|
|
(`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass,
|
|
A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012),
|
|
J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312).
|
|
The self-calibrating spot finder's per-resolution-ring background statistics, with the Bragg peaks
|
|
excluded by iterated clipping, follow Cheetah's peakfinder8: A. Barty, R. A. Kirian,
|
|
F. R. N. C. Maia, M. Hantke, C. H. Yoon, T. A. White and H. N. Chapman, "Cheetah: software for
|
|
high-throughput reduction and analysis of serial femtosecond X-ray diffraction data" (2014),
|
|
J. Appl. Cryst. 47, 1118-1131
|
|
[doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626).
|
|
|
|
**Per-crystal post-refinement of serial stills** - refining, for each still, the partiality
|
|
parameters the spot positions cannot pin (a small reorientation, the crystal's own rocking width and the
|
|
radial scale of its reciprocal lattice, which a shot's photon energy or a crystal's cell moves) against
|
|
the merged intensities follows the post-refinement of PRIME and the mosaic-block model of Sauter et al.:
|
|
M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, A. T. Brunger
|
|
and W. I. Weis, "Enabling X-ray free electron laser crystallography for challenging biological systems
|
|
from a limited number of crystals" (2015), eLife 4, e05421; N. K. Sauter, J. Hattne, A. S. Brewster,
|
|
N. Echols, P. H. Zwart and P. D. Adams, "Improved crystal orientation and physical properties from
|
|
single-shot XFEL stills" (2014), Acta Cryst. D70, 3299-3309 (DOIs not verified here, so not given).
|
|
|
|
Spot extraction groups strong pixels into spots with the sparse connected-component labelling of the
|
|
ACTS traccc project: P. Gessinger, H. M. Gray, A. Krasznahorkay, C. Leggett, J. Niermann,
|
|
A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction library for HEP
|
|
experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822);
|
|
[traccc](https://github.com/acts-project/traccc). The CPU spot extractor adapts its SparseCCL source,
|
|
and the CUDA spot extractor follows the design of its GPU counterpart - a backward-neighbour graph
|
|
over a sorted hit list, resolved by a parallel union-find. traccc is MPL-2.0; see
|
|
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). The SparseCCL algorithm itself is A. Hennequin,
|
|
B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: Connected Components Labeling and
|
|
Analysis for sparse images" (2019), DASIP 2019, 65-70
|
|
[doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184).
|
|
|
|
**XDS and DISTL** — a spot is bounded by its shape (how much of its bounding box it fills) rather than
|
|
by its size alone, as XDS (cited under [Integration and rotation geometry](#integration-and-rotation-geometry)) and DISTL bound theirs; the statistic
|
|
is rugnux's own, the choice of shape over size is theirs. Z. Zhang, N. K. Sauter, H. van den Bedem,
|
|
G. Snell and A. M. Deacon, "Automated diffraction image analysis and spot searching for high-throughput
|
|
crystal screening" (2006), J. Appl. Cryst. 39, 112-119
|
|
[doi:10.1107/S0021889805040677](https://doi.org/10.1107/S0021889805040677).
|
|
|
|
### Indexing
|
|
|
|
**Fourier-transform indexing** — indexing a diffraction pattern, with no prior knowledge of the
|
|
cell, by Fourier-transforming the reciprocal-space positions of its spots was proposed by Gérard
|
|
Bricogne as a three-dimensional transform; the de-novo FFT indexers (`-X fft`, `-X fftw`) use the
|
|
one-dimensional projection form of that idea described in the next paragraph. G. Bricogne (1986),
|
|
in *Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software
|
|
(Phase III)*, LURE, Paris, p. 28 (a workshop proceedings without a DOI).
|
|
|
|
**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/mosflm/)** — the Rossmann FFT autoindexing algorithm and
|
|
post-refinement practice, including which parameters are safe to refine per image and which must be
|
|
refined over a wedge. The autoindexing algorithm itself — projecting the reciprocal-space points
|
|
onto many directions and Fourier-transforming the 1D projection histograms — is I. Steller,
|
|
R. Bolotovsky and M. G. Rossmann, "An algorithm for automatic indexing of oscillation images using
|
|
Fourier analysis" (1997), J. Appl. Cryst. 30, 1036-1040
|
|
[doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777); MOSFLM is the
|
|
implementation whose practice is followed, described in H. R. Powell, "The Rossmann Fourier
|
|
autoindexing algorithm in MOSFLM" (1999), Acta Cryst. D55, 1690-1695
|
|
[doi:10.1107/S0907444999009506](https://doi.org/10.1107/S0907444999009506). A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM"
|
|
(2007), in *Evolving Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245,
|
|
41-51 [doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4);
|
|
T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new
|
|
graphical interface for diffraction-image processing with MOSFLM" (2011), Acta Cryst. D67, 271-281
|
|
[doi:10.1107/S0907444910048675](https://doi.org/10.1107/S0907444910048675); H. R. Powell,
|
|
T. G. G. Battye, L. Kontogiannis, O. Johnson and A. G. W. Leslie, "Integrating macromolecular X-ray
|
|
diffraction data with the graphical user interface iMosflm" (2017), Nat. Protoc. 12, 1310-1325
|
|
[doi:10.1038/nprot.2017.037](https://doi.org/10.1038/nprot.2017.037).
|
|
|
|
**[fast-feedback-indexer](https://github.com/paulscherrerinstitute/fast-feedback-indexer)** — the
|
|
known-cell indexer for serial stills (`-X ffbidx`) is PSI's fast-feedback-indexer library, linked at
|
|
build time (BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), which implements
|
|
the TORO algorithm: P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton,
|
|
M. Janousch, F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for
|
|
kilohertz serial crystallography" (2024), J. Appl. Cryst. 57, 931-944
|
|
[doi:10.1107/S1600576724003182](https://doi.org/10.1107/S1600576724003182). The CPU implementation
|
|
used without a GPU (`image_analysis/indexing/FFBIDXIndexerCPU.cpp`) is a port of the library's CUDA
|
|
kernels by H.-C. Stadler (PSI): the same samples and scores, with the loops reorganised for a CPU.
|
|
|
|
### Cell reduction and lattice symmetry
|
|
|
|
**[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and
|
|
structure-factor machinery, and MTZ / XDS_ASCII I/O. Vendored in `gemmi_gph/`, so it also carries a
|
|
licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source
|
|
Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200).
|
|
|
|
**Křivý & Gruber's Niggli reduction, and the lattice-character table** — the reduction that puts
|
|
every candidate cell in a comparable form is I. Křivý and B. Gruber, "A unified algorithm for
|
|
determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298
|
|
[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636), used through GEMMI's
|
|
implementation; the table of lattice characters that maps a reduced cell to Bravais lattices and
|
|
centrings follows International Tables for Crystallography Vol. A, Table 9.2.5.1.
|
|
|
|
**Grosse-Kunstleve, Sauter & Adams's numerically stable cell reduction** - the magnitude-scaled
|
|
tolerance that decides the sign of a structurally-zero scalar product, and with it the Niggli type a
|
|
reduced cell is presented in. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically
|
|
stable algorithms for the computation of reduced unit cells" (2004), Acta Cryst. A60, 1-6
|
|
[doi:10.1107/S010876730302186X](https://doi.org/10.1107/S010876730302186X).
|
|
|
|
**Le Page's metric-symmetry search** - the obliquity of each of the 81 candidate two-folds of a
|
|
reduced cell, which is what tells a run that its lattice metric hosts more rotational symmetry than
|
|
the group its intensities supported, and the derivation of the conventional axes from that
|
|
rotation group, which is what the run then offers to the space-group search as a second lattice
|
|
candidate. The two-fold search is used through GEMMI's implementation of it. Y. Le Page, "The
|
|
derivation of the axes of the conventional unit cell from the dimensions of the Buerger-reduced
|
|
cell" (1982), J. Appl. Cryst. 15, 255-259
|
|
[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959).
|
|
|
|
### Integration and rotation geometry
|
|
|
|
**[XDS](https://xds.mr.mpg.de/)** — rotation geometry and notation, the reciprocal Lorentz and
|
|
partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for rejecting
|
|
a reflection whose predicted profile is not cleanly its own, the intensity-based test for a
|
|
centred lattice, the recognition of shaded detector regions by comparing a pixel's background
|
|
against the background at its own resolution (`DEFPIX`), and the scaling correction surfaces
|
|
indexed by image number and detector region. W. Kabsch, "XDS" (2010), Acta Cryst. D66, 125-132
|
|
[doi:10.1107/S0907444909047337](https://doi.org/10.1107/S0907444909047337); W. Kabsch, "Integration,
|
|
scaling, space-group assignment and post-refinement" (2010), Acta Cryst. D66, 133-144
|
|
[doi:10.1107/S0907444909047374](https://doi.org/10.1107/S0907444909047374).
|
|
|
|
**Profile fitting** with reweighted, de-biased variances is the Kabsch/Otwinowski iteration, from the
|
|
second XDS paper above and from Z. Otwinowski and W. Minor, "Processing of X-ray diffraction data
|
|
collected in oscillation mode" (1997), Methods Enzymol. 276, 307-326
|
|
[doi:10.1016/S0076-6879(97)76066-X](https://doi.org/10.1016/S0076-6879%2897%2976066-X).
|
|
|
|
**The two-dimensional integration architecture** — integrating each image in the detector plane
|
|
and only afterwards assembling a reflection's partials into a full across images, as against
|
|
three-dimensional profile fitting through the image stack — is the architecture of DENZO/SCALEPACK
|
|
and MOSFLM, and it is the one Rugnux's rotation pipeline follows (per-image profile-fitted
|
|
integration, then partials combined into fulls; §9 and §10.6 of the [data-analysis reference](CPU_DATA_ANALYSIS.md)). The
|
|
Otwinowski & Minor citation above and the MOSFLM citations below carry the credit for the paradigm
|
|
as well as for the specifics taken from each.
|
|
|
|
### Space group, twinning and pseudo-symmetry
|
|
|
|
**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search. Stage A scores each
|
|
candidate rotation operator by the correlation of I(h) with I(Rh) on **resolution-normalised**
|
|
intensities (E²), as POINTLESS does — both arms of a symmetry pair sit at the same |s|, so on raw
|
|
intensities the resolution fall-off is variance shared between them and lifts a false operator's
|
|
correlation as much as a true one's; the screw-axis test scores a
|
|
predicted-absent class against the rest of its own axial row rather than against a global mean or a
|
|
fixed cut, and lets confidence fall away with the number of axial reflections instead of refusing
|
|
below a count; the glide-plane test is that same test applied to a zone, scoring the extinguished
|
|
class against the rest of its own plane, as POINTLESS scores zonal absences. P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82
|
|
[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693); P. R. Evans, "An
|
|
introduction to data reduction: space-group determination, scaling and intensity statistics" (2011),
|
|
Acta Cryst. D67, 282-292 [doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X);
|
|
P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta
|
|
Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061);
|
|
J. Agirre, M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular
|
|
crystallography" (2023), Acta Cryst. D79, 449-461
|
|
[doi:10.1107/S2059798323003595](https://doi.org/10.1107/S2059798323003595).
|
|
|
|
**Baur and Kassner** — the convention for which of two absence-equivalent glide groups the space-group
|
|
search writes (P2/c over Pc, C2/c over Cc): the centrosymmetric one, because a missed inversion centre is
|
|
the common error among published small-molecule space-group assignments. W. H. Baur and D. Kassner,
|
|
"The perils of Cc: comparing the frequencies of falsely assigned space groups with their general
|
|
population" (1992), Acta Cryst. B48, 356-369
|
|
[doi:10.1107/S0108768191014726](https://doi.org/10.1107/S0108768191014726).
|
|
|
|
**The centre-of-symmetry statistics** are Wilson's and Howells, Phillips & Rogers's: the intensity
|
|
distributions of acentric and centric structures and the cumulative N(z) test built on them, read here
|
|
on the general reflections of the Laue class beside <|E^2-1|> and Padilla and Yeates's L test (taken at
|
|
its centric value, 2/pi). A. J. C. Wilson, "The probability distribution of X-ray intensities" (1949),
|
|
Acta Cryst. 2, 318-321 [doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813);
|
|
E. R. Howells, D. C. Phillips and D. Rogers, "The probability distribution of X-ray intensities. II.
|
|
Experimental investigation and the X-ray detection of centres of symmetry" (1950), Acta Cryst. 3,
|
|
210-214 [doi:10.1107/S0365110X50000513](https://doi.org/10.1107/S0365110X50000513).
|
|
|
|
**The twinning L test** is Padilla and Yeates's: pairing each acentric reflection with a
|
|
symmetry-independent neighbour and reading the first and second moments of
|
|
L = (I1−I2)/(I1+I2) against their untwinned and perfect-twin values. J. E. Padilla and
|
|
T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and
|
|
pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, 1124-1130
|
|
[doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). Their title claims
|
|
robustness to pseudo-centering, and this program's partner steps deliver it: a step of 2 along an
|
|
axis preserves the class of a half-integer pseudo-translation, which is what a pseudo-centering is.
|
|
That robustness does not extend to a pseudo-translation which is not half-integer, and the partner
|
|
steps are restricted when one is detected - see the translational-pseudo-symmetry note below.
|
|
|
|
**The twin fraction of the twin hypothesis, and the twinned model it is scored with**, follow Yeates:
|
|
the intensity a twinned crystal records is (1−α) I(h) + α I(Th), and the disagreement
|
|
H = |I(h) − I(Th)|/(I(h) + I(Th)) of the pairs a law relates is uniform on [0, 1−2α], so the median
|
|
gives the fraction. T. O. Yeates, "Detecting and overcoming crystal twinning" (1997), Methods Enzymol.
|
|
276, 344-358 [doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879%2897%2976068-3).
|
|
|
|
**The symmetry hypotheses a refused promotion leaves** - the adopted group and every other index-2 subgroup
|
|
of the refused group perfectly twinned by a law of its coset, all merging alike, carried to the model and
|
|
decided by placing and refining it in each - follow the advice to refine in the candidate groups and compare,
|
|
rather than decide from intensity statistics, that pseudo-symmetry and twinning call for:
|
|
P. H. Zwart, R. W. Grosse-Kunstleve, A. A. Lebedev, G. N. Murshudov and P. D. Adams, "Surprises and pitfalls
|
|
arising from (pseudo)symmetry" (2008), Acta Cryst. D64, 99-107
|
|
[doi:10.1107/S090744490705531X](https://doi.org/10.1107/S090744490705531X); A. A. Lebedev and M. N. Isupov,
|
|
"Space-group and origin ambiguity in macromolecular structures with pseudo-symmetry and its treatment with the
|
|
program Zanuda" (2014), Acta Cryst. D70, 2430-2443
|
|
[doi:10.1107/S1399004714014795](https://doi.org/10.1107/S1399004714014795).
|
|
|
|
**Twin refinement** (§15.12) refines the model against Yeates's twinned intensity, (1−α) I(h) + α I(Th), with the
|
|
twin fraction refined beside it as SHELXL refines BASF: R. Herbst-Irmer and G. M. Sheldrick, "Refinement of twinned
|
|
structures with SHELXL97" (1998), Acta Cryst. B54, 443-449
|
|
[doi:10.1107/S0108768197018454](https://doi.org/10.1107/S0108768197018454). The twin laws come from the lattice's
|
|
metric as A. A. Lebedev, A. A. Vagin and G. N. Murshudov describe, "Intensity statistics in twinned crystals with
|
|
examples from the PDB" (2006), Acta Cryst. D62, 83-95
|
|
[doi:10.1107/S0907444905036759](https://doi.org/10.1107/S0907444905036759), and the fractions are refined by
|
|
least squares at every cycle, as REFMAC5 refines them (Murshudov et al. 2011, Acta Cryst. D67, 355-367, cited
|
|
above). REFMAC5's target is a twinned-intensity likelihood; ours keeps the amplitude likelihood with the twinned
|
|
model amplitude in it, a simplification of that.
|
|
|
|
**A first-pass cell that is the coincidence lattice of two twin domains** (twinning by reticular
|
|
merohedry of index 2 or 3, e.g. obverse/reverse rhombohedral domains or a cubic crystal twinned about
|
|
[111]) is recognised by the nodes that belong to neither domain being empty. The twin index is
|
|
Friedel's (G. Friedel, *Leçons de cristallographie*, 1926; no DOI); the split of the reflections into
|
|
those of one domain, of the other, of both and of neither is the one Herbst-Irmer and Sheldrick use
|
|
for obverse/reverse twins: R. Herbst-Irmer and G. M. Sheldrick, "Refinement of obverse/reverse twins"
|
|
(2002), Acta Cryst. B58, 477-481
|
|
[doi:10.1107/S0108768102001039](https://doi.org/10.1107/S0108768102001039).
|
|
|
|
**Translational pseudo-symmetry** is detected from the native Patterson computed from the merged
|
|
intensities, and the interpretation of an off-origin peak as a pseudo-translation between copies of
|
|
the contents of the asymmetric unit - together with the modulation it puts on the intensities, which
|
|
is the second half of the test here - is Read, Adams and McCoy's. Their fitted peak-height table is
|
|
not used: the peak is scored against a per-dataset within-shell permutation null instead, because the
|
|
noise floor of the statistic depends strongly on how many reflections a dataset has. The same
|
|
modulation is what the axial systematic-absence test scores against, so that a reflection class a
|
|
pseudo-translation merely suppresses is not read as extinct and does not buy a screw axis; the
|
|
estimate of its depth there is our own, measured per axial row from the merged intensities rather
|
|
than from the Patterson vector. Where a symmetry operator or a twin law carries the vector to one
|
|
that is not ±itself, the modulation is also read on classes averaged over the lattice's rotations -
|
|
their modulation model summed over the copy pairs each rotation relates, as the measured intensity
|
|
sums them; that averaging is our own construction. R. J. Read,
|
|
P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of translational
|
|
noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183
|
|
[doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374).
|
|
|
|
### Scaling, merging and data quality
|
|
|
|
**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, per-observation
|
|
outlier rejection at merge, the scaling error model, and the treatment of a reflection whose
|
|
background is contaminated. Its published behaviour, and in places its source, settled several
|
|
choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS: implementation and
|
|
evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97
|
|
[doi:10.1107/S2059798317017235](https://doi.org/10.1107/S2059798317017235); D. G. Waterman,
|
|
G. Winter, R. J. Gildea et al., "Diffraction-geometry refinement in the DIALS framework" (2016),
|
|
Acta Cryst. D72, 558-575 [doi:10.1107/S2059798316002187](https://doi.org/10.1107/S2059798316002187);
|
|
J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software
|
|
package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399
|
|
[doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198); J. M. Parkhurst,
|
|
G. Winter, D. G. Waterman et al., "Robust background modelling in DIALS" (2016), J. Appl. Cryst. 49,
|
|
1912-1921 [doi:10.1107/S1600576716013595](https://doi.org/10.1107/S1600576716013595).
|
|
|
|
**Wilson outlier test** — judging an observation that has no symmetry mates against the acentric and
|
|
centric intensity distributions of its resolution shell, with the symmetry enhancement factor, is
|
|
Wilson's statistics; rejecting only observations that are also significant, and keeping a reflection
|
|
whose observations are all large, follows AIMLESS's EMAX test. A. J. C. Wilson, "The probability
|
|
distribution of X-ray intensities" (1949), Acta Cryst. 2, 318-321
|
|
[doi:10.1107/S0365110X49000813](https://doi.org/10.1107/S0365110X49000813);
|
|
P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82
|
|
[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693).
|
|
|
|
**Amplitudes from intensities** — the posterior-mean amplitude of each merged intensity under the
|
|
acentric and centric Wilson priors is French and Wilson's; giving no amplitude to an intensity more
|
|
than 3.7 sigma below zero, and leaving such intensities out of the prior, follows CCP4's
|
|
[ctruncate](https://www.ccp4.ac.uk/) (C. Ballard and N. Stein). So does scaling each reflection's
|
|
Wilson prior by the anisotropy tensor along its direction, which ctruncate has done by default since
|
|
its version 1.7 ("use anisotropy in prior for truncate procedure"); rugnux uses its own tensor for it
|
|
(see Diffraction anisotropy below). S. French and K. Wilson, "On the treatment of negative intensity
|
|
observations" (1978), Acta Cryst. A34, 517-525
|
|
[doi:10.1107/S0567739478001114](https://doi.org/10.1107/S0567739478001114); ctruncate is cited through
|
|
the CCP4 suite: M. D. Winn, C. C. Ballard, K. D. Cowtan et al.,
|
|
"Overview of the CCP4 suite and current developments" (2011), Acta Cryst. D67, 235-242
|
|
[doi:10.1107/S0907444910045749](https://doi.org/10.1107/S0907444910045749).
|
|
|
|
**Absorption as spherical harmonics** — describing an empirical absorption correction as a series of
|
|
real spherical harmonics of the beam directions in the crystal frame is Blessing's; its use as a
|
|
restrained scaling surface of the diffracted-beam direction follows SCALA and AIMLESS. R. H. Blessing,
|
|
"An empirical correction for absorption anisotropy" (1995), Acta Cryst. A51, 33-38
|
|
[doi:10.1107/S0108767394005726](https://doi.org/10.1107/S0108767394005726); P. Evans, "Scaling and
|
|
assessment of data quality" (2006), Acta Cryst. D62, 72-82
|
|
[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693).
|
|
|
|
**Diffraction anisotropy** — the description of the overall fall-off by a single anisotropic
|
|
displacement tensor, its symmetry constraints, and the fact that only its deviatoric part is
|
|
determined (the isotropic part being degenerate with the overall scale) are Sheriff and Hendrickson's.
|
|
The estimator fits that tensor to the observed intensity distribution, taking sigma(I) into account,
|
|
in the sense of Popov and Bourenkov. The directional diffraction limits - <I/sigma(I)> in a cone about
|
|
each principal direction, and the reporting of the anisotropic deltaB as the range of the principal
|
|
components - follow AIMLESS. Rugnux reports these; it corrects no intensity and removes no reflection
|
|
on a directional criterion. S. Sheriff and W. A. Hendrickson, "Description of overall anisotropy in
|
|
diffraction from macromolecular crystals" (1987), Acta Cryst. A43, 118-121
|
|
[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X); A. N. Popov and
|
|
G. P. Bourenkov, "Choice of data-collection parameters based on statistic modelling" (2003), Acta
|
|
Cryst. D59, 1145-1153 [doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163);
|
|
P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta
|
|
Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061).
|
|
|
|
**Data-quality statistics** follow the established conventions rather than any one program: R_meas
|
|
and R_pim, CC1/2 and CC\*, the per-shell CC(model, data) between F^2_calc and F^2_obs, and the
|
|
reporting of I/sigma(I). K. Diederichs and P. A. Karplus, "Improved
|
|
R-factors for diffraction data analysis in macromolecular crystallography" (1997), Nat. Struct. Biol.
|
|
4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269); P. A. Karplus and
|
|
K. Diederichs, "Linking crystallographic model and data quality" (2012), Science 336, 1030-1033
|
|
[doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231) (CC\* is also what a joint merge of
|
|
several sweeps tests their isomorphism against: CC_AB / (CC\*_A CC\*_B) is the cosine of the angle between
|
|
two data sets, their systematic difference apart from their noise, as K. Diederichs, "Dissecting random and
|
|
systematic differences between noisy composite data sets" (2017), Acta Cryst. D73, 286-293
|
|
[doi:10.1107/S2059798317000699](https://doi.org/10.1107/S2059798317000699) shows - which is why the join is
|
|
decided on the size of that ratio, and a doubtful sweep by the merge with and without it, the idea of
|
|
delta-CC1/2 by Assmann, Brehm and Diederichs (2016), below); M. S. Weiss, "Global
|
|
indicators of X-ray data quality" (2001), J. Appl. Cryst. 34, 130-135
|
|
[doi:10.1107/S0021889800018227](https://doi.org/10.1107/S0021889800018227) (R_pim beside R_meas and
|
|
R_merge); K. Diederichs and
|
|
P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222
|
|
[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121).
|
|
|
|
**The Whittaker smoother** — the per-frame scale of the rotation fulls is a penalised least-squares
|
|
curve (a second-difference penalty, the smoothness chosen by cross-validation)
|
|
in the form P. H. C. Eilers gave Whittaker's graduation: P. H. C. Eilers, "A perfect smoother" (2003),
|
|
Anal. Chem. 75, 3631-3636 [doi:10.1021/ac034173t](https://doi.org/10.1021/ac034173t); E. T. Whittaker,
|
|
"On a new method of graduation" (1923), Proc. Edinburgh Math. Soc. 41, 63-75
|
|
[doi:10.1017/S0013091500077853](https://doi.org/10.1017/S0013091500077853).
|
|
|
|
**Fisher's z-transformation** — the cross-validation of the scaling correction surfaces averages the
|
|
change of the half-set CC1/2 over resolution shells on atanh(CC), so that the shells near CC = 1, where
|
|
a multiplicative error shows, are not outweighed by the noise of the shells without signal. R. A.
|
|
Fisher, "Frequency distribution of the values of the correlation coefficient in samples from an
|
|
indefinitely large population" (1915), Biometrika 10, 507-521
|
|
[doi:10.2307/2331838](https://doi.org/10.2307/2331838).
|
|
|
|
**The frame disposition** - which stretches of a rotation sweep are kept, carried at reduced weight or
|
|
dropped from the merge - decides every exclusion on delta-CC1/2, the change in the overall CC1/2 when a
|
|
group of images is left out, measured in the sigma-tau form so that no random half-dataset split is
|
|
involved. The statistic, the Fisher transformation used to compare it across CC1/2 values, its standard
|
|
error going as the inverse square root of the reflection count, and the rejection discipline (never
|
|
remove a group whose delta-CC1/2 is positive or near zero; remove a little, re-form the reference and
|
|
repeat) are all taken from its authors, whose XDSCC12 is the reference implementation.
|
|
G. Assmann, W. Brehm and K. Diederichs, "Identification of rogue datasets in serial crystallography"
|
|
(2016), J. Appl. Cryst. 49, 1021-1028
|
|
[doi:10.1107/S1600576716005471](https://doi.org/10.1107/S1600576716005471); G. M. Assmann, M. Wang and
|
|
K. Diederichs, "Making a difference in multi-data-set crystallography: simple and deterministic
|
|
data-scaling/selection methods" (2020), Acta Cryst. D76, 636-652
|
|
[doi:10.1107/S2059798320006348](https://doi.org/10.1107/S2059798320006348). That the same statistic
|
|
belongs at scaling, applied to groups of images rather than to whole datasets, follows
|
|
[DIALS](https://dials.github.io/) (`dials.scale`, delta-CC1/2 image-group filtering):
|
|
J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software
|
|
package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399
|
|
[doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198).
|
|
|
|
**Several sweeps merged as one** - each sweep's partials combined into fulls on their own, and the
|
|
fulls of all the sweeps then scaled together, every sweep with its own frame scales and a relative B
|
|
against the joint merge, is how XSCALE merges several data sets (W. Kabsch, "Integration, scaling,
|
|
space-group assignment and post-refinement" (2010), above). Reading the radiation dose a sweep was taken
|
|
at from its relative B, which orders the sweeps by dose from the data rather than from the order they
|
|
were given in, follows K. Diederichs, "Some aspects of quantitative analysis and correction of radiation
|
|
damage" (2006), Acta Cryst. D62, 96-101
|
|
[doi:10.1107/S0907444905031537](https://doi.org/10.1107/S0907444905031537).
|
|
|
|
**Uncertainty conventions** follow the IUCr Commission on Crystallographic Nomenclature:
|
|
D. Schwarzenbach, S. C. Abrahams, H. D. Flack et al., "Statistical descriptors in crystallography:
|
|
Report of the IUCr Subcommittee on Statistical Descriptors" (1989), Acta Cryst. A45, 63-75
|
|
[doi:10.1107/S0108767388009596](https://doi.org/10.1107/S0108767388009596); D. Schwarzenbach,
|
|
S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descriptors in
|
|
crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995),
|
|
Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340).
|
|
|
|
**Unit-cell e.s.d.s** — that the e.s.d.s of a least-squares cell fit are too small, seeing only the
|
|
scatter of the centroids, is Herbstein's survey; rugnux therefore also fits the cell on blocks of
|
|
consecutive frames and writes the larger of the two estimates: F. H. Herbstein, "How precise are
|
|
measurements of unit-cell dimensions from single crystals?" (2000), Acta Cryst. B56, 547-557
|
|
[doi:10.1107/S010876810000269X](https://doi.org/10.1107/S010876810000269X).
|
|
|
|
**Indexing ambiguity of serial data without a reference** — where a serial run's lattice is more
|
|
symmetric than its crystals and no reference or model is given, each crystal is put into the indexing in
|
|
which its intensities agree best with those of all the other crystals, crystal by crystal until none
|
|
changes. The objective - the summed agreement between every pair of crystals - and the iteration that
|
|
maximises it are taken from Brehm and Diederichs, whose method CrystFEL's `ambigator` also implements;
|
|
rugnux measures the agreement as a product of standardised amplitudes (square roots of the intensities)
|
|
rather than a correlation, which lets the sum over all partners be taken against one running sum instead
|
|
of pair by pair.
|
|
W. Brehm and K. Diederichs, "Breaking the indexing ambiguity in serial crystallography" (2014),
|
|
Acta Cryst. D70, 101-109 [doi:10.1107/S1399004713025431](https://doi.org/10.1107/S1399004713025431).
|
|
The same objective settles which twin law puts each sweep of a joint merge of several sweeps on the
|
|
reference's indexing - decided over all pairs of sweeps at once, as dials.cosym does for its data sets:
|
|
R. J. Gildea and G. Winter, "Determination of Patterson group symmetry from sparse multi-crystal data sets
|
|
in the presence of an indexing ambiguity" (2018), Acta Cryst. D74, 405-410
|
|
[doi:10.1107/S2059798318002978](https://doi.org/10.1107/S2059798318002978).
|
|
|
|
**Reference sweep of a joint merge** — the sweep the others are put on the axes of is the one whose own
|
|
merge reaches the finest resolution, as KAMO takes the data set of the smallest Wilson B as its reference:
|
|
K. Yamashita, K. Hirata and M. Yamamoto, "KAMO: towards automated data processing for microcrystals"
|
|
(2018), Acta Cryst. D74, 441-449 [doi:10.1107/S2059798318004576](https://doi.org/10.1107/S2059798318004576).
|
|
|
|
### Physical corrections and calibration
|
|
|
|
**Sensor absorption at oblique incidence, and the flight path** — the angle-dependent quantum
|
|
efficiency of a flat sensor, the radial parallax variance that comes from the same integral, and the
|
|
attenuation of a reflection in the air between the sample and its pixel are all the Beer-Lambert law
|
|
taken along a ray that crosses t/cos(alpha) of sensor, or D/cos(alpha) of air, and converts at a
|
|
random depth. The attenuation coefficients, for silicon, CdTe, dry air and helium alike, are the
|
|
NIST tabulation: J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray
|
|
Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for
|
|
Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004),
|
|
NIST Standard Reference Database 126
|
|
[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F).
|
|
|
|
**Polarization correction** — the azimuthal polarization factor applied to the azimuthally
|
|
integrated profile, to the integrated Bragg intensities and to the ring background the beam-stop
|
|
shadow test compares a pixel against is the one derived for a partially polarized
|
|
synchrotron source by R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular
|
|
crystallography with synchrotron radiation: photographic data collection and polarization
|
|
correction" (1982), J. Appl. Cryst. 15, 330-337
|
|
[doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060).
|
|
|
|
**Hexagonal-ice ring positions** — the eleven ring $d$ spacings from 3.895 to 1.522 Å that the
|
|
ice-ring score, the ice-ring flagging and the ice calibrant are all built on are taken from the
|
|
measurements of Moreau and co-workers, not enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in
|
|
biomolecular cryocrystallography" (2021), Acta Cryst. D77, 540-554
|
|
[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170).
|
|
|
|
That list ends at 1.522 Å by its own scope, so the eight bands below it are calculated here rather
|
|
than taken from anyone: ice Ih structure factors on the oxygen sublattice, kept where they reach 3% of
|
|
the strongest line, which reproduces the eleven measured positions exactly. The lattice constants are
|
|
Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice
|
|
constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50,
|
|
644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933).
|
|
|
|
### Model-based analysis and maps
|
|
|
|
**Bulk-solvent correction and overall scaling** — the model's structure factors are put on the
|
|
observed scale with an overall factor, an anisotropic B and a flat bulk-solvent term, the flat-mask
|
|
model of A. Fokine and A. Urzhumtsev, "Flat bulk-solvent model: obtaining optimal parameters"
|
|
(2002), Acta Cryst. D58, 1387-1392
|
|
[doi:10.1107/S0907444902010284](https://doi.org/10.1107/S0907444902010284), which is also the source
|
|
of the starting values and of the range those two parameters are physically meaningful over. The mask itself -
|
|
the region a probe cannot reach from the atoms, shrunk back - is J.-S. Jiang and A. T. Brünger's, "Protein
|
|
hydration observed by X-ray diffraction: solvation properties of penicillopepsin and neuraminidase crystal
|
|
structures" (1994), J. Mol. Biol. 243, 100-115
|
|
[doi:10.1006/jmbi.1994.1633](https://doi.org/10.1006/jmbi.1994.1633), drawn with the radii REFMAC5 uses. The
|
|
procedure that fits them — a grid search over that range for the solvent pair, with the overall
|
|
scale and the anisotropic B refitted at every grid point — follows P. V. Afonine,
|
|
R. W. Grosse-Kunstleve and P. D. Adams, "A robust bulk-solvent correction and anisotropic scaling
|
|
procedure" (2005), Acta Cryst. D61, 850-855
|
|
[doi:10.1107/S0907444905007894](https://doi.org/10.1107/S0907444905007894). The fit is unweighted,
|
|
as in both that procedure and REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu,
|
|
R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, "REFMAC5 for the refinement of
|
|
macromolecular crystal structures" (2011), Acta Cryst. D67, 355-367
|
|
[doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314).
|
|
|
|
**sigma_A map coefficients** — the maps written by `--model` are weighted by a maximum-likelihood
|
|
sigma_A estimated per resolution shell, giving 2mFo-DFc and mFo-DFc rather than 2Fo-Fc and Fo-Fc.
|
|
What is taken is the formalism itself: the Rice and Woolfson likelihoods of |Fo| given |Fc| and
|
|
sigma_A, the figure of merit m and the scale D that follow from it, and the result that the
|
|
bias-corrected coefficient is 2mFo-DFc for an acentric reflection and mFo for a centric one.
|
|
R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures with
|
|
errors" (1986), Acta Cryst. A42, 140-149
|
|
[doi:10.1107/S0108767386099622](https://doi.org/10.1107/S0108767386099622).
|
|
|
|
**[ANODE](https://doi.org/10.1107/S0021889811041768)** — reading the anomalous difference map at the
|
|
atoms of a supplied model and reporting the strongest sites by name, instead of searching the map for
|
|
blobs. The map itself is the textbook anomalous difference Fourier; what is taken from ANODE is that
|
|
reading: A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density calculation" (2011),
|
|
J. Appl. Cryst. 44, 1285-1287
|
|
[doi:10.1107/S0021889811041768](https://doi.org/10.1107/S0021889811041768).
|
|
|
|
**Uniform random rotations** — the null a supplied model is scored against re-orients that model at
|
|
random about its own centroid, and the rotations are drawn uniformly from SO(3) through a uniform
|
|
random unit quaternion. K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk,
|
|
Academic Press (1992), 124-132 (no DOI).
|
|
|
|
**Variable projection** — the rigid-body placement of a supplied model re-fits the overall scale at
|
|
every step, and its Jacobian folds that re-fit in by projecting the scale's own derivatives out of the
|
|
placement's, in Kaufman's simplified form of Golub and Pereyra's derivative of the reduced problem.
|
|
G. H. Golub and V. Pereyra, "The Differentiation of Pseudo-Inverses and Nonlinear Least Squares
|
|
Problems Whose Variables Separate" (1973), SIAM J. Numer. Anal. 10, 413-432
|
|
[doi:10.1137/0710036](https://doi.org/10.1137/0710036). L. Kaufman, "A variable projection method
|
|
for solving separable nonlinear least squares problems" (1975), BIT 15, 49-57
|
|
[doi:10.1007/BF01932995](https://doi.org/10.1007/BF01932995).
|
|
|
|
**Predicted models** — `--model` (and `--mr`) place an AlphaFold prediction (or any model the rigid
|
|
body does not place) by molecular replacement. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction
|
|
with AlphaFold" (2021), Nature 596, 583-589
|
|
[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); the models are those of
|
|
the AlphaFold Protein Structure Database: M. Varadi, S. Anyango, M. Deshpande et al., "AlphaFold
|
|
Protein Structure Database: massively expanding the structural coverage of protein-sequence space
|
|
with high-accuracy models" (2022), Nucleic Acids Res. 50, D439-D444
|
|
[doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061). The model is prepared as
|
|
phenix.process_predicted_model prepares one - residues below pLDDT 70 removed, B-factors from the
|
|
coordinate error the pLDDT predicts: R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold
|
|
models to work with phenix.process_predicted_model and ISOLDE" (2022), Acta Cryst. D78, 1303-1314
|
|
[doi:10.1107/S2059798322010026](https://doi.org/10.1107/S2059798322010026).
|
|
|
|
**Molecular replacement** — the search scores each orientation of the model against the data by
|
|
interpolating its molecular transform, computed once by FFT, as Navaza's AMoRe does: J. Navaza,
|
|
"AMoRe: an automated package for molecular replacement" (1994), Acta Cryst. A50, 157-163
|
|
[doi:10.1107/S0108767393007597](https://doi.org/10.1107/S0108767393007597). The translation
|
|
function is the full-symmetry one, all pairs of symmetry copies on one Fourier synthesis:
|
|
R. A. Crowther and D. M. Blow, "A method of positioning a known molecule in an unknown crystal
|
|
structure" (1967), Acta Cryst. 23, 544-548
|
|
[doi:10.1107/S0365110X67003172](https://doi.org/10.1107/S0365110X67003172). A placement is called
|
|
a solution, as Phaser calls one, where its translation-function Z-score reaches 8: A. J. McCoy,
|
|
R. W. Grosse-Kunstleve, P. D. Adams, M. D. Winn, L. C. Storoni and R. J. Read, "Phaser
|
|
crystallographic software" (2007), J. Appl. Cryst. 40, 658-674
|
|
[doi:10.1107/S0021889807021206](https://doi.org/10.1107/S0021889807021206). Whether one copy fits
|
|
the cell is read off the Matthews coefficient: B. W. Matthews, "Solvent content of protein
|
|
crystals" (1968), J. Mol. Biol. 33, 491-497
|
|
[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836%2868%2990205-2). How many copies
|
|
to search for is weighed by a distribution of the Matthews coefficient over protein crystals made from
|
|
the mode and mean of Kantardjieff and Rupp's: K. A. Kantardjieff and B. Rupp, "Matthews coefficient
|
|
probabilities: Improved estimates for unit cell contents of proteins, DNA, and protein-nucleic acid
|
|
complex crystals" (2003), Protein Sci. 12, 1865-1871
|
|
[doi:10.1110/ps.0350503](https://doi.org/10.1110/ps.0350503). A copy refined beside the others has the
|
|
others' placements read off by least-squares superposition: W. Kabsch, "A solution for the best rotation
|
|
to relate two sets of vectors" (1976), Acta Cryst. A32, 922-923
|
|
[doi:10.1107/S0567739476001873](https://doi.org/10.1107/S0567739476001873). The placements are judged by
|
|
their log-likelihood gain, from the Rice and Woolfson distributions of the observed normalised amplitudes
|
|
given the model's, with sigma_A from the model's expected coordinate error: R. J. Read, "Pushing the
|
|
boundaries of molecular replacement with maximum likelihood" (2001), Acta Cryst. D57, 1373-1382
|
|
[doi:10.1107/S0907444901012471](https://doi.org/10.1107/S0907444901012471); and the gain is read as a
|
|
Z-score over the same orientation at other positions, as the likelihood-enhanced translation function's
|
|
is: A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni and R. J. Read, "Likelihood-enhanced fast
|
|
translation functions" (2005), Acta Cryst. D61, 458-464
|
|
[doi:10.1107/S0907444905001617](https://doi.org/10.1107/S0907444905001617). A model whose expected
|
|
log-likelihood gain - the sum of sigma_A^4 / 2 over the reflections - is too small for a placement to stand
|
|
out is not searched for: A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason,
|
|
B. Lohkamp and R. J. Read, "Ab initio solution of macromolecular crystal structures without direct
|
|
methods" (2017), Proc. Natl. Acad. Sci. USA 114, 3637-3641
|
|
[doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114). Translational NCS is looked
|
|
for in the native Patterson before the search, an off-origin peak read as Read, Adams and McCoy read
|
|
it (see
|
|
[Translational pseudo-symmetry](#space-group-twinning-and-pseudo-symmetry) above). The logarithm of the Bessel
|
|
function I0 in it is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun,
|
|
"Handbook of Mathematical Functions" (1964), National Bureau of Standards, 9.8.1-9.8.2. A predicted
|
|
model is cut into rigid domains by its predicted aligned error, as Oeffner et al. (above) do before
|
|
molecular replacement; the error itself is AlphaFold's: J. Jumper, R. Evans, A. Pritzel et al., "Highly
|
|
accurate protein structure prediction with AlphaFold" (2021), Nature 596, 583-589
|
|
[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2).
|
|
|
|
**Structure refinement** (of a placed model) — the placed model is refined against the maximum-likelihood
|
|
amplitude target, the Rice distribution of an observed amplitude given the model's (its centric
|
|
counterpart for a centric reflection): N. S. Pannu and R. J. Read, "Improved structure refinement
|
|
through maximum likelihood" (1996), Acta Cryst. A52, 659-668
|
|
[doi:10.1107/S0108767396004370](https://doi.org/10.1107/S0108767396004370); G. N. Murshudov,
|
|
A. A. Vagin and E. J. Dodson, "Refinement of macromolecular structures by the maximum-likelihood method"
|
|
(1997), Acta Cryst. D53, 240-255 [doi:10.1107/S0907444996012255](https://doi.org/10.1107/S0907444996012255).
|
|
Its two parameters are fitted by maximum likelihood in resolution bins - the variance on the free
|
|
reflections, D on the working reflections with the measurement error added to that variance, which is how
|
|
REFMAC5 divides them (Murshudov et al. 2011, above); the refined model's maps are weighted by the same D
|
|
and figure of merit. Estimating the likelihood's parameters on free reflections follows V. Y. Lunin and T. P. Skovoroda, "R-free likelihood-based estimates of errors for phases
|
|
calculated from atomic models" (1995), Acta Cryst. A51, 880-887
|
|
[doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin, P. V. Afonine and
|
|
A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors" (2002), Acta Cryst. A58,
|
|
270-282 [doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). The gradient with
|
|
respect to every atom is taken by the FFT method - the derivatives of the target with respect to the
|
|
structure factors transformed into a map and summed against each atom's density derivatives:
|
|
R. C. Agarwal, "A new least-squares refinement technique based on the fast Fourier transform algorithm"
|
|
(1978), Acta Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618).
|
|
The positions and B factors are minimised by L-BFGS (rugnux's own, its memory kept from one macro-cycle to
|
|
the next, the B factors' steps scaled by their estimated curvature): D. C. Liu and J. Nocedal, "On the limited
|
|
memory BFGS method for large scale optimization" (1989), Math. Program. 45, 503-528
|
|
[doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116), whose two-loop recursion is J. Nocedal's, "Updating
|
|
quasi-Newton matrices with limited storage" (1980), Math. Comp. 35, 773-782
|
|
[doi:10.1090/S0025-5718-1980-0572855-7](https://doi.org/10.1090/S0025-5718-1980-0572855-7). The geometry restraints are those of the CCP4
|
|
Monomer Library, a subset of which rugnux carries (see THIRD_PARTY_NOTICES.md): A. A. Vagin,
|
|
R. A. Steiner, A. A. Lebedev, L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5
|
|
dictionary: organization of prior chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60,
|
|
2184-2195 [doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510); applied to the model
|
|
through GEMMI's topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, "GEMMI: A
|
|
library for structural biology" (2022), J. Open Source Softw. 7, 4200
|
|
[doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). The torsion derivatives follow
|
|
A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion angles
|
|
in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141
|
|
[doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T](https://doi.org/10.1002/(SICI)1096-987X(19960715)17:9%3C1132::AID-JCC5%3E3.0.CO;2-T).
|
|
The B factors of bonded atoms, and of atoms in contact, are restrained to each other in the
|
|
Kullback-Leibler form REFMAC5 uses, whose tolerance grows with B (Murshudov et al. 2011, above; the divergence itself
|
|
is S. Kullback and R. A. Leibler's, "On information and sufficiency" (1951), Ann. Math. Statist. 22, 79-86
|
|
[doi:10.1214/aoms/1177729694](https://doi.org/10.1214/aoms/1177729694)), and the
|
|
overall isotropic B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012,
|
|
below). The weights of the X-ray term against the geometry and against the B restraints start from the
|
|
ratios of the gradients, as X-PLOR's crystallographic weight was first set: A. T. Brünger, M. Karplus and
|
|
G. A. Petsko, "Crystallographic refinement by simulated annealing: application to crambin" (1989), Acta Cryst.
|
|
A45, 50-61 [doi:10.1107/S0108767388009195](https://doi.org/10.1107/S0108767388009195); and they are chosen
|
|
among short refinements by R-free with the geometry held within bounds, as phenix.refine chooses them: P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd,
|
|
N. W. Moriarty, M. Mustyakimov, T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams,
|
|
"Towards automated crystallographic structure refinement with phenix.refine" (2012), Acta Cryst. D68,
|
|
352-367 [doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself:
|
|
A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal
|
|
structures" (1992), Nature 355, 472-475 [doi:10.1038/355472a0](https://doi.org/10.1038/355472a0).
|
|
The model the refinement scores is scaled to the data with the overall scale and anisotropic B fitted by
|
|
robust least squares - iteratively reweighted, with the pseudo-Huber weight of each reflection's residual -
|
|
so that a few reflections the model cannot explain do not set the scale of the rest: P. J. Huber, "Robust
|
|
estimation of a location parameter" (1964), Ann. Math. Statist. 35, 73-101
|
|
[doi:10.1214/aoms/1177703732](https://doi.org/10.1214/aoms/1177703732), in the smooth (pseudo-Huber) form of
|
|
P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, "Deterministic edge-preserving regularization in
|
|
computed imaging" (1997), IEEE Trans. Image Process. 6, 298-311
|
|
[doi:10.1109/83.551699](https://doi.org/10.1109/83.551699).
|
|
The bonds of metal ions to their donor atoms that the model's LINK records name are restrained at the
|
|
distances of the Monomer Library's own statistics of metal sites (its `metals.json`, embedded as
|
|
`monomers/metals.cif`; per metal, donor element and coordination number), in line with the surveys of
|
|
M. M. Harding, "Geometry of metal-ligand interactions in proteins" (2001), Acta Cryst. D57, 401-411
|
|
[doi:10.1107/S0907444900019168](https://doi.org/10.1107/S0907444900019168) and "Small revisions to
|
|
predicted distances around metal sites in proteins" (2006), Acta Cryst. D62, 678-682
|
|
[doi:10.1107/S0907444906014594](https://doi.org/10.1107/S0907444906014594), and, for Mg²⁺, Na⁺ and K⁺ at
|
|
nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to nucleobase
|
|
nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004 [doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175).
|
|
The anisotropic displacements: a TLS group's U is that of a rigid body's translation, libration and their
|
|
correlation, U = T + A L A^T + A S + S^T A^T, of V. Schomaker and K. N. Trueblood, "On the rigid-body motion of
|
|
molecules in crystals" (1968), Acta Cryst. B24, 63-76
|
|
[doi:10.1107/S0567740868001718](https://doi.org/10.1107/S0567740868001718), refined against the likelihood with a
|
|
residual B per atom, and a model's B column read as the total or as the residual to its TLS groups, as in
|
|
M. D. Winn, M. N. Isupov and G. N. Murshudov, "Use of TLS parameters to model anisotropic displacements in
|
|
macromolecular refinement" (2001), Acta Cryst. D57, 122-133
|
|
[doi:10.1107/S0907444900014736](https://doi.org/10.1107/S0907444900014736). The chains are cut into TLS groups as
|
|
TLSMD cuts them - contiguous segments, by dynamic programming over the fit of an isotropic TLS model to the
|
|
refined B factors: J. Painter and E. A. Merritt, "Optimal description of a protein structure in terms of multiple
|
|
groups undergoing TLS motion" (2006), Acta Cryst. D62, 439-450
|
|
[doi:10.1107/S0907444906005270](https://doi.org/10.1107/S0907444906005270). Atoms refined with a U of their own are
|
|
restrained by the rigid-bond postulate of F. L. Hirshfeld, "Can X-ray data distinguish bonding effects from
|
|
vibrational smearing?" (1976), Acta Cryst. A32, 239-244
|
|
[doi:10.1107/S0567739476000533](https://doi.org/10.1107/S0567739476000533), in the form of SHELXL's RIGU - the
|
|
components of U along the bond and across it in the bond's plane: A. Thorn, B. Dittrich and G. M. Sheldrick,
|
|
"Enhanced rigid-bond restraints" (2012), Acta Cryst. A68, 448-451
|
|
[doi:10.1107/S0108767312014535](https://doi.org/10.1107/S0108767312014535) - with SHELXL's similarity (SIMU) and
|
|
approximate-isotropy (ISOR) restraints as G. M. Sheldrick, "Crystal structure refinement with SHELXL" (2015), Acta
|
|
Cryst. C71, 3-8 [doi:10.1107/S2053229614024218](https://doi.org/10.1107/S2053229614024218) documents them. Which
|
|
displacement model a refinement keeps - isotropic, TLS, or a U per atom - is decided on the free set, as a test of
|
|
what the data support rather than a rule by resolution: E. A. Merritt, "To B or not to B: a question of
|
|
resolution?" (2012), Acta Cryst. D68, 468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320).
|
|
|
|
**NCS restraints** — copies of a molecule in the model are found by sequence alignment with REFMAC5's criterion
|
|
(more than 15 residues aligned, 80% identity - here of the longer chain - and 2.5 Å rms after superposition;
|
|
Murshudov et al. 2011, above) and restrained locally, on pairs of
|
|
corresponding interatomic distances under the Geman-McClure robust loss, as REFMAC5's local NCS restraints are, with
|
|
the distance range of R. A. Nicholls, F. Long and G. N. Murshudov, "Low-resolution refinement tools in REFMAC5"
|
|
(2012), Acta Cryst. D68, 404-417 [doi:10.1107/S090744491105606X](https://doi.org/10.1107/S090744491105606X), and
|
|
the bonds and angles left out as they leave them out. The case for local restraints over a superposition, and for
|
|
deciding them by R-free, is G. J. Kleywegt's, "Use of non-crystallographic symmetry in protein structure refinement"
|
|
(1996), Acta Cryst. D52, 842-857 [doi:10.1107/S0907444995016477](https://doi.org/10.1107/S0907444995016477); I. Usón,
|
|
E. Pohl, T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz and G. M. Sheldrick, "1.7 Å structure of the stabilized
|
|
REIv mutant T39K. Application of local NCS restraints" (1999), Acta Cryst. D55, 1158-1167
|
|
[doi:10.1107/S0907444999003972](https://doi.org/10.1107/S0907444999003972); and J. J. Headd, N. Echols,
|
|
P. V. Afonine, N. W. Moriarty, R. J. Gildea and P. D. Adams, "Flexible torsion-angle noncrystallographic symmetry
|
|
restraints for improved macromolecular structure refinement" (2014), Acta Cryst. D70, 1346-1356
|
|
[doi:10.1107/S1399004714003277](https://doi.org/10.1107/S1399004714003277), whose comparison of global, torsion and
|
|
no NCS is why the copies' B factors are refined independently here.
|
|
|
|
**Unmodelled density** — the blobs the refinement lists are the connected regions of the difference map above
|
|
a cutoff, found by flood fill as Coot's "unmodelled blobs" finds them, in GEMMI's implementation
|
|
(`find_blobs_by_flood_fill`): P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, "Features and development
|
|
of Coot" (2010), Acta Cryst. D66, 486-501 [doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493).
|
|
|
|
**Special positions** — an atom on a crystallographic special position is placed on it and its shifts are
|
|
confined to the subspace its site symmetry leaves unchanged, as R. W. Grosse-Kunstleve and P. D. Adams treat
|
|
special positions, "Algorithms for deriving crystallographic space-group information. II. Treatment of special
|
|
positions" (2002), Acta Cryst. A58, 60-65 [doi:10.1107/S0108767301016658](https://doi.org/10.1107/S0108767301016658).
|
|
|
|
**Water build** — the waters the refinement adds and removes follow the documented ordered-solvent defaults of
|
|
phenix.refine: difference-map peaks from 3 σ, hydrogen-bond partners to 3.2 Å and picking only to 2.8 Å, with
|
|
the waters added, filtered and refined between macro-cycles:
|
|
Afonine et al. (2012, above);
|
|
the distances to the model that Coot's Find Waters keeps (2.4-3.2 Å), from the Coot manual and Emsley et al.
|
|
(2010, above); and the removal of a water in negative difference density at -3 σ, the cut Echols et al. (2014,
|
|
below) use to exclude a site.
|
|
|
|
**Solvent audit** — the waters and ions of the refined model are scored by their surroundings with the
|
|
stereochemical criteria of the surveys of ions at nucleic acids by P. Auffinger's group: the coordination bands
|
|
1.9-2.3 / 2.3-2.6 / 2.6-3.2 Å, the Mg²⁺ exclusion zone and the clash shells of N. Naleem, A. Henning-Knechtel,
|
|
S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and
|
|
annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120
|
|
[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); the Mg-O and Mg-N distances and the
|
|
2.3-3.4 Å exclusion zone of F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar and P. Auffinger,
|
|
"Principles of ion binding to RNA inferred from the analysis of a 1.55 Å resolution bacterial ribosome
|
|
structure – Part I: Mg2+" (2025), Nucleic Acids Res. 53, gkae1148
|
|
[doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); the Na⁺, K⁺ and water distances and
|
|
coordination numbers of F. Leonarski, L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor
|
|
Mg2+ inner-sphere binders but excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192
|
|
[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118), and the distance bands of their
|
|
2017 survey (above). The ligand sphere of the bond-valence sums and the reading of halides by the donors
|
|
around them follow N. Echols, N. Morshed, P. V. Afonine, A. J. McCoy, M. D. Miller, R. J. Read,
|
|
J. S. Richardson, T. C. Terwilliger and P. D. Adams, "Automated identification of elemental ions in
|
|
macromolecular crystal structures" (2014), Acta Cryst. D70, 1104-1114
|
|
[doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308), and the halide's positive and
|
|
negative neighbours N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental
|
|
ion identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158
|
|
[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241); the chloride's shell of donors
|
|
at up to 3.4 Å is O. Carugo's, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC
|
|
Struct. Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8). The
|
|
bond-valence sums are I. D. Brown and D. Altermatt's, "Bond-valence parameters obtained from a systematic
|
|
analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247
|
|
[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of N. E. Brese and
|
|
M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197
|
|
[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by H. Zheng,
|
|
K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, "Data mining of
|
|
iron(II) and iron(III) bond-valence parameters, and their relevance for macromolecular crystallography"
|
|
(2017), Acta Cryst. D73, 316-325 [doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584),
|
|
weighted by occupancy as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method
|
|
able to identify metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37
|
|
[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); their normalised vector sum and
|
|
its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski, I. G. Shabalin, K. B. Handing
|
|
and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation tool" (2017), Acta Cryst. D73, 223-233
|
|
[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061). The score that combines them is
|
|
our own. The divalent ion's place on a purine N7 - in the base's plane, 3.5-3.9 Å from its O6 or N6 - is the test of
|
|
Leonarski et al. (2017, above). A light ion is told from a water by a density above the waters', as Echols et al.
|
|
(2014, above) flag one; a halide's few or no donors are allowed as Z. Dauter and M. Dauter describe halides binding
|
|
also at hydrophobic surfaces, "Entering a new phase: using solvent halide ions in protein structure determination"
|
|
(2001), Structure 9, R21-R26 [doi:10.1016/S0969-2126(01)00565-2](https://doi.org/10.1016/S0969-2126(01)00565-2).
|
|
The anomalous scattering expected of each species is f″ at the data's wavelength by D. T. Cromer and D. A.
|
|
Liberman, "Anomalous dispersion calculations near to and on the long-wavelength side of an absorption edge"
|
|
(1981), Acta Cryst. A37, 267-268 [doi:10.1107/S0567739481000600](https://doi.org/10.1107/S0567739481000600),
|
|
with the corrections of L. Kissel and R. H. Pratt, "Corrections to tabulated anomalous-scattering factors"
|
|
(1990), Acta Cryst. A46, 170-175 [doi:10.1107/S0108767389010718](https://doi.org/10.1107/S0108767389010718), as
|
|
GEMMI computes them (its port of the Larch routines).
|
|
|
|
## Software and computing methods
|
|
|
|
Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading
|
|
the result, follows Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4
|
|
data", HDF5 User Group meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders).
|
|
The CUDA kernels in Jungfraujoch are its own, but the approach is his.
|
|
|
|
Removing the small islands of solvent from the bulk-solvent mask on the GPU labels the connected
|
|
components with a parallel union-find, following D. P. Playne and K. Hawick, "A New Algorithm for
|
|
Parallel Connected-Component Labelling on GPUs" (2018), IEEE Trans. Parallel Distrib. Syst. 29,
|
|
1217-1230 [doi:10.1109/TPDS.2018.2799216](https://doi.org/10.1109/TPDS.2018.2799216).
|
|
|
|
The FFT indexer's GPU peak search divides by a fixed window width with a reciprocal multiplication
|
|
corrected once by the exact remainder, which returns the correctly rounded quotient, following
|
|
P. W. Markstein, "Computation of elementary functions on the IBM RISC System/6000 processor"
|
|
(1990), IBM J. Res. Dev. 34, 111-119.
|
|
|
|
The per-frame scaling loops of the rotation merge are accelerated by Anderson acceleration in the
|
|
type-II form analysed by Walker and Ni: H. F. Walker and P. Ni, "Anderson acceleration for fixed-point
|
|
iterations" (2011), SIAM J. Numer. Anal. 49, 1715-1735
|
|
[doi:10.1137/10078356X](https://doi.org/10.1137/10078356X), after D. G. Anderson, "Iterative
|
|
procedures for nonlinear integral equations" (1965), J. ACM 12, 547-560
|
|
[doi:10.1145/321296.321305](https://doi.org/10.1145/321296.321305).
|
|
|
|
The interval the report gives for ISa comes from a bootstrap over the unique reflections:
|
|
B. Efron, "Bootstrap methods: another look at the jackknife" (1979), Ann. Statist. 7, 1-26
|
|
[doi:10.1214/aos/1176344552](https://doi.org/10.1214/aos/1176344552). Its draws are Poisson weights
|
|
hashed from the replicate and the reflection with the SplitMix64 finaliser of G. L. Steele Jr.,
|
|
D. Lea and C. H. Flood, "Fast splittable pseudorandom number generators" (2014), Proc. OOPSLA 2014,
|
|
453-472 [doi:10.1145/2660193.2660195](https://doi.org/10.1145/2660193.2660195).
|
|
|
|
The single-precision exp the Bragg integration's profile fit shares between the CPU and an Apple GPU
|
|
(`BraggExp.h`) is built from textbook error-free floating-point arithmetic: the argument reduction of
|
|
W. J. Cody and W. Waite, "Software Manual for the Elementary Functions" (1980), Prentice-Hall; the
|
|
exact product (with Veltkamp's split) and Fast2Sum of T. J. Dekker, "A floating-point technique for
|
|
extending the available precision" (1971), Numer. Math. 18, 224-242; and the 2Sum of D. E. Knuth,
|
|
"The Art of Computer Programming, Vol. 2: Seminumerical Algorithms", section 4.2.2.
|
|
|
|
This software uses the Viridis, Magma and Inferno colormaps from Matplotlib under its
|
|
BSD-compatible license. J. D. Hunter, "Matplotlib: A 2D graphics environment" (2007), Comput. Sci.
|
|
Eng. 9, 90-95 [doi:10.1109/MCSE.2007.55](https://doi.org/10.1109/MCSE.2007.55).
|
|
|
|
The Mol* structure viewer of the web frontend follows the setup Guillaume Gotthard (PSI) tuned for
|
|
[AareDB](https://gitea.psi.ch/mx/aaredb), the sample-management and data-pipeline system of the PSI MX
|
|
beamlines: its Coot-style mouse controls and trackball, the model representations and colours, and the
|
|
display of the density maps (carved wireframe meshes at absolute levels from the whole cell's
|
|
statistics, contour-level defaults and colours).
|
|
|
|
## File formats read from a published specification
|
|
|
|
**CBF / imgCIF** - the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression scheme
|
|
and reads the imgCIF `_axis` table (the laboratory directions of the image's fast and slow pixel
|
|
directions, of the goniometer axes and of a 2theta arm) from the specification alone; no CBFlib or
|
|
other CBF code is used, so there is no licence obligation, only this credit.
|
|
H. J. Bernstein and A. P. Hammersley, "Specification of the Crystallographic Binary File
|
|
(CBF/imgCIF)" (2006), International Tables for Crystallography Vol. G, 37-43
|
|
[doi:10.1107/97809553602060000729](https://doi.org/10.1107/97809553602060000729);
|
|
A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, "Image dictionary (imgCIF)" (2006),
|
|
International Tables for Crystallography Vol. G, 444-458
|
|
[doi:10.1107/97809553602060000746](https://doi.org/10.1107/97809553602060000746).
|
|
|
|
**d\*TREK SMV** - the SMV reader reads the d\*TREK header vocabulary written by Rigaku's
|
|
CrystalClear (Saturn and R-AXIS detectors: detector and spatial-distortion vectors, detector
|
|
circles, 2theta arm, encoded overflows) from the headers themselves, interpreting the detector
|
|
vectors as dxtbx does for these detectors; no d\*TREK or dxtbx code is used.
|
|
J. W. Pflugrath, "The finer things in X-ray diffraction data collection" (1999), Acta Cryst. D55,
|
|
1718-1725 [doi:10.1107/S090744499900935X](https://doi.org/10.1107/S090744499900935X);
|
|
[dxtbx](https://github.com/cctbx/dxtbx): J. M. Parkhurst, A. S. Brewster, L. Fuentes-Montero,
|
|
D. G. Waterman et al., "dxtbx: the diffraction experiment toolbox" (2014), J. Appl. Cryst. 47,
|
|
1459-1465 [doi:10.1107/S1600576714011996](https://doi.org/10.1107/S1600576714011996).
|
|
|
|
## Public diffraction data used for testing
|
|
|
|
In addition to in-house datasets collected at SLS 2.0, Jungfraujoch is tested against public
|
|
diffraction data collected on other people's beamlines, on detectors and in file formats we do not
|
|
produce ourselves - most of it at other facilities, a few sets at the Swiss Light Source but not by
|
|
this system. That data was collected and published by other people. Every dataset used, the DOI to
|
|
cite for it, and the deposition it belongs to are listed in
|
|
[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md); we thank the depositors, and the repositories that make
|
|
the data findable and citable.
|
|
|
|
**[IRRMC](https://proteindiffraction.org/)**, the Integrated Resource for Reproducibility in
|
|
Macromolecular Crystallography (Minor lab, University of Virginia), is the source of most of them.
|
|
IRRMC releases its data under CC0 and asks that the DOI of the dataset be
|
|
cited. M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng,
|
|
D. R. Cooper, M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of
|
|
macromolecular diffraction experiments" (2016), Acta Cryst. D72, 1181-1193
|
|
[doi:10.1107/S2059798316014716](https://doi.org/10.1107/S2059798316014716); M. Grabowski,
|
|
M. Cymborowski, P. J. Porebski, T. Osinski, I. G. Shabalin, D. R. Cooper and W. Minor, "The
|
|
Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first
|
|
four years" (2019), Struct. Dyn. 6, 064301
|
|
[doi:10.1063/1.5128672](https://doi.org/10.1063/1.5128672).
|
|
|
|
**[SBGrid Data Bank](https://data.sbgrid.org/)**, the structural biology community's data
|
|
publication service (SBGrid Consortium, Harvard Medical School). P. A. Meyer,
|
|
S. Socias, J. Key, E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the
|
|
structural biology data grid supports live analysis" (2016), Nat. Commun. 7, 10882
|
|
[doi:10.1038/ncomms10882](https://doi.org/10.1038/ncomms10882).
|
|
|
|
**[Zenodo](https://zenodo.org/)**, CERN's open repository, hosts datasets deposited there directly
|
|
by the groups that collected them. European Organization for Nuclear Research and OpenAIRE,
|
|
"Zenodo" (2013), CERN [doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). Some of those
|
|
deposits are described in IUCrData Raw Data Letters; the letters are cited on the
|
|
[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page, beside the datasets they describe.
|
|
|
|
**[CXIDB](https://cxidb.org/)**, the Coherent X-ray Imaging Data Bank, is the source of the serial
|
|
(XFEL stills) datasets of the still battery, listed with their DOIs on
|
|
[EXTERNAL_TEST_DATA_STILLS](EXTERNAL_TEST_DATA_STILLS.md). F. R. N. C. Maia, "The Coherent X-ray
|
|
Imaging Data Bank" (2012), Nat. Methods 9, 854-855
|
|
[doi:10.1038/nmeth.2110](https://doi.org/10.1038/nmeth.2110).
|
|
|
|
**[MXRDR](https://mxrdr.icm.edu.pl/)**, the Macromolecular Xtallography Raw Data Repository
|
|
(ICM, University of Warsaw), releases its data under CC0 and asks that the DOI of the dataset be
|
|
cited.
|
|
|
|
**[XRDa](https://xrda.pdbj.org/)**, the Xtal Raw Data Archive (Protein Data Bank Japan), which
|
|
publishes raw diffraction images - X-ray, electron and neutron - and mints a DOI for each; it asks
|
|
that the DOI of the dataset be cited. It has no canonical citation paper.
|
|
|
|
**The [ESRF data portal](https://data.esrf.fr/)**, through which the European Synchrotron
|
|
publishes raw data under its data policy (CC BY 4.0, with the dataset DOI to be cited).
|
|
R. Dimper, A. Götz, A. De Maria, V. A. Solé, M. Chaillet and B. Lebayle, "ESRF Data Policy,
|
|
Storage, and Services" (2019), Synchrotron Rad. News 32, 7-12
|
|
[doi:10.1080/08940886.2019.1608119](https://doi.org/10.1080/08940886.2019.1608119).
|
|
|
|
**The [Keele University research data repository](https://researchdata.keele.ac.uk/)** and
|
|
**[UQ eSpace](https://espace.library.uq.edu.au/)** (The University of Queensland), which host the
|
|
raw images of datasets deposited there by the groups that collected them; the dataset DOIs are
|
|
cited on the [EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page.
|
|
|
|
The beamline, resolution, space group and unit cell quoted for each dataset are the values
|
|
deposited with the corresponding PDB entry, read from the RCSB PDB data API. H. M. Berman,
|
|
J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and P. E. Bourne,
|
|
"The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242
|
|
[doi:10.1093/nar/28.1.235](https://doi.org/10.1093/nar/28.1.235).
|
|
|
|
**The [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/)** (EMBL-EBI and Google
|
|
DeepMind) supplies the start models of the battery's timing benchmark: the predictions of horse
|
|
myoglobin (AF-P68082-F1) and bovine cytochrome c (AF-P62894-F1), model version 6, with their predicted
|
|
aligned error, kept unmodified in `tools/battery/models/` under CC BY 4.0. J. Jumper, R. Evans,
|
|
A. Pritzel et al., "Highly accurate protein structure prediction with AlphaFold" (2021), Nature 596,
|
|
583-589 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); M. Varadi,
|
|
S. Anyango, M. Deshpande et al., "AlphaFold Protein Structure Database: massively expanding the
|
|
structural coverage of protein-sequence space with high-accuracy models" (2022), Nucleic Acids Res. 50,
|
|
D439-D444 [doi:10.1093/nar/gkab1061](https://doi.org/10.1093/nar/gkab1061); M. Varadi, D. Bertoni,
|
|
P. Magana et al., "AlphaFold Protein Structure Database in 2024: providing structure coverage for over
|
|
214 million protein sequences" (2024), Nucleic Acids Res. 52, D368-D375
|
|
[doi:10.1093/nar/gkad1011](https://doi.org/10.1093/nar/gkad1011). Its third start model is the
|
|
deposited model of PDB entry 4NWV, moved away from its place.
|
|
|
|
## Generative AI usage declaration
|
|
|
|
Large language models were used extensively in developing this code. Jungfraujoch development was
|
|
supported with JetBrains AI (mostly GPT models) to refactor and verify particular code fragments.
|
|
Rugnux was developed with the assistance of Claude Code (mostly the Opus model). This documentation
|
|
was written with the assistance of Claude Opus and Fable models.
|