Acknowledgements¶
How to cite Jungfraujoch and Rugnux, and the papers that describe them, is on the Publications page. In short:
Rugnux: F. Leonarski and M. Wang, “Rugnux: from diffraction images to structure at synchrotrons and XFELs in one program” (2026), bioRxiv doi:10.64898/2026.10.24.761522.
Jungfraujoch: F. Leonarski, M. Brückner, C. Lopez-Cuenca, A. Mozzanica, H.-C. Stadler, Z. Matěj, A. Castellane, B. Mesnet, J. Wojdyla, B. Schmitt and M. Wang, “Jungfraujoch: hardware-accelerated data-acquisition system for kilohertz pixel-array X-ray detectors” (2023), J. Synchrotron Rad. 30, 227-234 doi:10.1107/S1600577522010268; for real-time serial crystallography and live indexing feedback also J. Duan, H.-C. Stadler, J. Nan et al., “Jungfraujoch: 38 GB s⁻¹ real-time serial crystallography – data acquisition and live indexing feedback” (2026), J. Appl. Cryst. 59, 1668-1677 doi:10.1107/S1600576726008083.
Funding and support¶
The project is supported by:
Innosuisse via Innovation Project “NextGenDCU high data rate acquisition system for X-ray detectors in structural biology applications” (101.535.1 IP-ENG; Apr 2023 - Sep 2025).
ETH Domain via Open Research Data Contribute project (Jan - Dec 2023).
AMD University Program with donation of licenses of Ethernet IP cores and Vivado software.
Crystallographic methods adopted from other packages¶
The analysis pipeline builds on methods first published, and in most cases first implemented, by other crystallographic software. The code is Jungfraujoch’s own; the methods are theirs, and are acknowledged here, entry by entry: what was taken, from whom, and the citation. How rugnux uses each method is described in the data-analysis reference, not here. Where a package’s source was consulted this is said explicitly. Most of these packages are neither linked nor vendored; the three that are - GEMMI, traccc and fast-feedback-indexer - also carry a licence obligation, recorded in THIRD_PARTY_NOTICES.md.
Experimental geometry and physical corrections¶
autoPROC — the principle, long argued for by Clemens Vonrhein, that a pipeline does not take the experimental metadata it is given (beam centre, detector geometry, goniometer axis) on trust but verifies it against the images and corrects it where it is wrong, and with it that the beam centre is found from the images; the aim of a pipeline that takes the decisions an expert would take and reports the evidence for each. C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek, T. Womack and G. Bricogne, “Data processing and analysis with the autoPROC toolbox” (2011), Acta Cryst. D67, 293-302 doi:10.1107/S0907444911007773.
Sensor absorption and air path — the attenuation coefficients of silicon, CdTe, dry air and helium behind the oblique-incidence sensor efficiency, the parallax variance and the air-path attenuation are the NIST tabulation. J. H. Hubbell and S. M. Seltzer, “Tables of X-Ray Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest” (1995, data updated 2004), NIST Standard Reference Database 126 doi:10.18434/T4D01F.
Polarization correction — the azimuthal polarization factor for a partially polarized synchrotron source. R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, “Macromolecular crystallography with synchrotron radiation: photographic data collection and polarization correction” (1982), J. Appl. Cryst. 15, 330-337 doi:10.1107/S0021889882012060.
Hexagonal-ice ring positions — the eleven measured ring spacings from 3.895 to 1.522 Å. D. W. Moreau, H. Atakisi and R. E. Thorne, “Ice in biomolecular cryocrystallography” (2021), Acta Cryst. D77, 540-554 doi:10.1107/S2059798321001170. The bands below 1.522 Å are calculated here from ice Ih structure factors on the lattice constants of K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, “Lattice constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K” (1994), Acta Cryst. B50, 644-648 doi:10.1107/S0108768194004933.
Powder calibrants — the silver behenate cell is T. C. Huang, H. Toraya, T. N. Blanton and Y. Wu, “X-ray powder diffraction analysis of silver behenate, a possible low-angle diffraction standard” (1993), J. Appl. Cryst. 26, 180-184 doi:10.1107/S0021889892009762; the CeO2 and silicon cells are those of the NIST standard reference materials SRM 674b and SRM 640 (certificates, no DOI).
Spot finding¶
CrystFEL — spot finding, the three-ring integration region, the serial/stills processing model, and the per-frame indexing acceptance test (indexing_peak_check() in peaks.c). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass, A. Barty and H. N. Chapman, “CrystFEL: a software suite for snapshot serial crystallography” (2012), J. Appl. Cryst. 45, 335-341 doi:10.1107/S0021889812002312.
Cheetah — the per-resolution-ring background statistics with the Bragg peaks excluded by iterated clipping, from peakfinder8. A. Barty, R. A. Kirian, F. R. N. C. Maia, M. Hantke, C. H. Yoon, T. A. White and H. N. Chapman, “Cheetah: software for high-throughput reduction and analysis of serial femtosecond X-ray diffraction data” (2014), J. Appl. Cryst. 47, 1118-1131 doi:10.1107/S1600576714007626.
Poisson significance threshold — beyond the summation limit the Poisson quantile of the adaptive spot-finding threshold is taken from the Cornish-Fisher expansion. E. A. Cornish and R. A. Fisher, “Moments and cumulants in the specification of distributions” (1938), Rev. Int. Stat. Inst. 5, 307-320 doi:10.2307/1400905.
GPU spot finder — the CUDA spot finder was developed by Hans-Christian Stadler (PSI).
traccc — the sparse connected-component labelling that groups strong pixels into spots: the CPU extractor adapts its SparseCCL source, the CUDA extractor follows the design of its GPU counterpart (MPL-2.0; see THIRD_PARTY_NOTICES.md). P. Gessinger, H. M. Gray, A. Krasznahorkay, C. Leggett, J. Niermann, A. Salzburger, S. N. Swatman and B. Yeo, “traccc: GPU track reconstruction library for HEP experiments” (2025), arXiv:2505.22822. The SparseCCL algorithm itself: A. Hennequin, B. Couturier, V. V. Gligorov and L. Lacassagne, “SparseCCL: Connected Components Labeling and Analysis for sparse images” (2019), DASIP 2019, 65-70 doi:10.1109/DASIP48288.2019.9049184.
XDS and DISTL — bounding a spot by its shape (how much of its bounding box it fills) rather than by its size; the statistic is rugnux’s own. XDS is cited under Integration and rotation geometry; Z. Zhang, N. K. Sauter, H. van den Bedem, G. Snell and A. M. Deacon, “Automated diffraction image analysis and spot searching for high-throughput crystal screening” (2006), J. Appl. Cryst. 39, 112-119 doi:10.1107/S0021889805040677.
Indexing¶
Fourier-transform indexing — indexing with no prior knowledge of the cell by Fourier-transforming the reciprocal-space positions of the spots, proposed as a three-dimensional transform; the de-novo indexers (-X fft, -X fftw) use its one-dimensional projection form. G. Bricogne (1986), in Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software (Phase III), LURE, Paris, p. 28 (a workshop proceedings without a DOI).
Rossmann FFT autoindexing — projecting the reciprocal-space points onto many directions and Fourier-transforming the 1D histograms. I. Steller, R. Bolotovsky and M. G. Rossmann, “An algorithm for automatic indexing of oscillation images using Fourier analysis” (1997), J. Appl. Cryst. 30, 1036-1040 doi:10.1107/S0021889897008777.
MOSFLM — the practice of that algorithm, and the post-refinement practice of which parameters are refined per image and which over a wedge. H. R. Powell, “The Rossmann Fourier autoindexing algorithm in MOSFLM” (1999), Acta Cryst. D55, 1690-1695 doi:10.1107/S0907444999009506; A. G. W. Leslie and H. R. Powell, “Processing diffraction data with MOSFLM” (2007), in Evolving Methods for Macromolecular Crystallography, NATO Science Series II, vol. 245, 41-51 doi:10.1007/978-1-4020-6316-9_4; T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, “iMOSFLM: a new graphical interface for diffraction-image processing with MOSFLM” (2011), Acta Cryst. D67, 271-281 doi:10.1107/S0907444910048675; H. R. Powell, T. G. G. Battye, L. Kontogiannis, O. Johnson and A. G. W. Leslie, “Integrating macromolecular X-ray diffraction data with the graphical user interface iMosflm” (2017), Nat. Protoc. 12, 1310-1325 doi:10.1038/nprot.2017.037.
fast-feedback-indexer — the known-cell indexer for serial stills (-X ffbidx), PSI’s library linked at build time (BSD-3-Clause; see THIRD_PARTY_NOTICES.md), implementing TORO. P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton, M. Janousch, F. Leonarski and B. Béjar, “TORO Indexer: a PyTorch-based indexing algorithm for kilohertz serial crystallography” (2024), J. Appl. Cryst. 57, 931-944 doi:10.1107/S1600576724003182. The CPU implementation used without a GPU is a port of the library’s CUDA kernels by H.-C. Stadler (PSI).
Cell reduction and lattice symmetry¶
GEMMI — symmetry operations, unit-cell and structure-factor machinery, and MTZ / XDS_ASCII I/O; vendored in gemmi_gph/, so it also carries a licence obligation. M. Wojdyr, “GEMMI: A library for structural biology” (2022), J. Open Source Softw. 7, 4200 doi:10.21105/joss.04200.
Niggli reduction — used through GEMMI’s implementation. I. Křivý and B. Gruber, “A unified algorithm for determining the reduced (Niggli) cell” (1976), Acta Cryst. A32, 297-298 doi:10.1107/S0567739476000636. The table of lattice characters that maps a reduced cell to Bravais lattices and centrings follows International Tables for Crystallography Vol. A, Table 9.2.5.1.
Numerically stable cell reduction — the magnitude-scaled tolerance that decides the sign of a structurally-zero scalar product. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, “Numerically stable algorithms for the computation of reduced unit cells” (2004), Acta Cryst. A60, 1-6 doi:10.1107/S010876730302186X.
Metric-symmetry search — the obliquity of the 81 candidate two-folds of a reduced cell, and the conventional axes derived from the rotation group found; the two-fold search is used through GEMMI. Y. Le Page, “The derivation of the axes of the conventional unit cell from the dimensions of the Buerger-reduced cell” (1982), J. Appl. Cryst. 15, 255-259 doi:10.1107/S0021889882011959. The step from the two-folds found to a conventional cell, a centring and an integral change of basis follows R. W. Grosse-Kunstleve, “Algorithms for deriving crystallographic space-group information” (1999), Acta Cryst. A55, 383-395 doi:10.1107/S0108767398010186.
Integration and rotation geometry¶
XDS — rotation geometry and notation, the reciprocal Lorentz and partiality treatment, the maximum-likelihood mosaicity estimate, the MINPK criterion for a reflection whose predicted profile is not cleanly its own, the intensity-based test for a centred lattice, the recognition of shaded detector regions (DEFPIX), and the scaling correction surfaces indexed by image number and detector region. W. Kabsch, “XDS” (2010), Acta Cryst. D66, 125-132 doi:10.1107/S0907444909047337; W. Kabsch, “Integration, scaling, space-group assignment and post-refinement” (2010), Acta Cryst. D66, 133-144 doi:10.1107/S0907444909047374.
Profile fitting — the Kabsch/Otwinowski iteration with reweighted, de-biased variances, from the second XDS paper above and from Z. Otwinowski and W. Minor, “Processing of X-ray diffraction data collected in oscillation mode” (1997), Methods Enzymol. 276, 307-326 doi:10.1016/S0076-6879(97)76066-X.
Two-dimensional integration — integrating each image in the detector plane and only afterwards combining a reflection’s partials into a full, as against three-dimensional profile fitting through the image stack, is the architecture of DENZO/SCALEPACK (Otwinowski and Minor, above) and MOSFLM (cited under Indexing).
Space group, twinning and pseudo-symmetry¶
POINTLESS (CCP4) — the space-group search: scoring each candidate rotation operator by the correlation of I(h) with I(Rh) on resolution-normalised intensities (E²); the screw-axis test that scores a predicted-absent class against the rest of its own axial row and lets confidence fall with the number of axial reflections instead of refusing below a count; and the same test on a zone for a glide plane. P. Evans, “Scaling and assessment of data quality” (2006), Acta Cryst. D62, 72-82 doi:10.1107/S0907444905036693; P. R. Evans, “An introduction to data reduction: space-group determination, scaling and intensity statistics” (2011), Acta Cryst. D67, 282-292 doi:10.1107/S090744491003982X; P. R. Evans and G. N. Murshudov, “How good are my data and what is the resolution?” (2013), Acta Cryst. D69, 1204-1214 doi:10.1107/S0907444913000061; J. Agirre, M. Atanasova, H. Bagdonas et al., “The CCP4 suite: integrative software for macromolecular crystallography” (2023), Acta Cryst. D79, 449-461 doi:10.1107/S2059798323003595.
Centrosymmetric glide groups — of two absence-equivalent glide groups the search writes the centrosymmetric one (P2/c over Pc, C2/c over Cc), a missed inversion centre being the common error. W. H. Baur and D. Kassner, “The perils of Cc: comparing the frequencies of falsely assigned space groups with their general population” (1992), Acta Cryst. B48, 356-369 doi:10.1107/S0108768191014726.
Centre-of-symmetry statistics — the acentric and centric intensity distributions and the cumulative N(z) test built on them, read beside <|E^2-1|> and the L test. A. J. C. Wilson, “The probability distribution of X-ray intensities” (1949), Acta Cryst. 2, 318-321 doi:10.1107/S0365110X49000813; E. R. Howells, D. C. Phillips and D. Rogers, “The probability distribution of X-ray intensities. II. Experimental investigation and the X-ray detection of centres of symmetry” (1950), Acta Cryst. 3, 210-214 doi:10.1107/S0365110X50000513.
Twinning L test — the moments of L = (I1−I2)/(I1+I2) over pairs of neighbouring acentric reflections against their untwinned and perfect-twin values; its robustness to pseudo-centering is kept by the partner steps, which are restricted when a non-half-integer pseudo-translation is found. J. E. Padilla and T. O. Yeates, “A statistic for local intensity differences: robustness to anisotropy and pseudo-centering and utility for detecting twinning” (2003), Acta Cryst. D59, 1124-1130 doi:10.1107/S0907444903007947. The test’s design - indicators on shell-normalised merged intensities, a twin law admitted within 3 degrees of obliquity - follows phenix.xtriage: Zwart, Grosse-Kunstleve and Adams (2005, cited under Data quality and anisotropy); D. Liebschner, P. V. Afonine, M. L. Baker et al., “Macromolecular structure determination using X-rays, neutrons and electrons: recent developments in Phenix” (2019), Acta Cryst. D75, 861-877 doi:10.1107/S2059798319011471.
Twin fraction and twinned intensity — the model (1−α) I(h) + α I(Th), and the fraction read from the median of H = |I(h) − I(Th)|/(I(h) + I(Th)), uniform on [0, 1−2α]. T. O. Yeates, “Detecting and overcoming crystal twinning” (1997), Methods Enzymol. 276, 344-358 doi:10.1016/S0076-6879(97)76068-3.
Twin refinement — one fraction per domain operator of the lattice refined with the model, small domains dropped, and twin-related reflections kept in one free set, as REFMAC5 does; the fractions re-fitted by least squares at every cycle as REFMAC5 refines them, against an amplitude likelihood with the twinned model amplitude in it (a simplification of REFMAC5’s twinned-intensity target). G. N. Murshudov, P. Skubák, A. A. Lebedev, N. S. Pannu, R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, “REFMAC5 for the refinement of macromolecular crystal structures” (2011), Acta Cryst. D67, 355-367 doi:10.1107/S0907444911001314; P. Roversi, E. Blanc, S. Johnson and S. M. Lea, “Tetartohedral twinning could happen to you too” (2012), Acta Cryst. D68, 418-424 doi:10.1107/S0907444912006737; the six-domain case of a monoclinic crystal on a hexagonal metric is K. A. Sultana, I. Alexeev, I. Kursula, P. Mäntsälä, J. Niemi and G. Schneider, “Structure determination by multiwavelength anomalous diffraction of aclacinomycin oxidoreductase: indications of multidomain pseudomerohedral twinning” (2007), Acta Cryst. D63, 149-159 doi:10.1107/S0907444906044271. A twin fraction as a parameter refined beside the model is SHELXL’s BASF (rugnux re-fits it between cycles rather than in the same least squares): R. Herbst-Irmer and G. M. Sheldrick, “Refinement of twinned structures with SHELXL97” (1998), Acta Cryst. B54, 443-449 doi:10.1107/S0108768197018454. The twin laws come from the lattice’s metric: A. A. Lebedev, A. A. Vagin and G. N. Murshudov, “Intensity statistics in twinned crystals with examples from the PDB” (2006), Acta Cryst. D62, 83-95 doi:10.1107/S0907444905036759.
Deciding pseudo-symmetry by refinement — carrying the adopted group and the index-2 subgroups a refused promotion leaves, each perfectly twinned by a law of its coset, to the model and deciding by placing and refining it in each, rather than from intensity statistics. P. H. Zwart, R. W. Grosse-Kunstleve, A. A. Lebedev, G. N. Murshudov and P. D. Adams, “Surprises and pitfalls arising from (pseudo)symmetry” (2008), Acta Cryst. D64, 99-107 doi:10.1107/S090744490705531X; A. A. Lebedev and M. N. Isupov, “Space-group and origin ambiguity in macromolecular structures with pseudo-symmetry and its treatment with the program Zanuda” (2014), Acta Cryst. D70, 2430-2443 doi:10.1107/S1399004714014795.
Twinning by reticular merohedry — a first-pass cell that is the coincidence lattice of two twin domains (reticular merohedry or pseudo-merohedry of index up to 10, e.g. obverse/reverse rhombohedral domains) is recognised by the nodes of neither domain being empty. The twin index is Friedel’s (G. Friedel, Leçons de cristallographie, 1926; no DOI); the split of the reflections into those of one domain, of the other, of both and of neither, and the treatment of reticular pseudo-merohedry (Herbst-Irmer and Sheldrick 1998, above), are Herbst-Irmer and Sheldrick’s: R. Herbst-Irmer and G. M. Sheldrick, “Refinement of obverse/reverse twins” (2002), Acta Cryst. B58, 477-481 doi:10.1107/S0108768102001039.
Translational pseudo-symmetry — the off-origin native-Patterson peak read as a pseudo-translation between copies of the asymmetric unit, and the modulation it puts on the intensities, which the axial systematic-absence test also scores against. The per-dataset permutation null for the peak, the per-row estimate of the modulation depth, and its averaging over the lattice’s rotations are rugnux’s own. R. J. Read, P. D. Adams and A. J. McCoy, “Intensity statistics in the presence of translational noncrystallographic symmetry” (2013), Acta Cryst. D69, 176-183 doi:10.1107/S0907444912045374. A pseudo-translation is declared at a Patterson peak of a fifth of the origin or more, Phaser’s default (McCoy et al. 2007, cited under Model placement and molecular replacement).
Scaling and merging¶
DIALS — the resolution cutoff from the CC1/2 fall-off, per-observation outlier rejection at merge, the scaling error model, and the treatment of a reflection whose background is contaminated; its published behaviour, and in places its source, settled several choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., “DIALS: implementation and evaluation of a new integration package” (2018), Acta Cryst. D74, 85-97 doi:10.1107/S2059798317017235; D. G. Waterman, G. Winter, R. J. Gildea et al., “Diffraction-geometry refinement in the DIALS framework” (2016), Acta Cryst. D72, 558-575 doi:10.1107/S2059798316002187; J. Beilsten-Edmands, G. Winter, R. Gildea et al., “Scaling diffraction data in the DIALS software package: algorithms and new approaches for multi-crystal scaling” (2020), Acta Cryst. D76, 385-399 doi:10.1107/S2059798320003198; J. M. Parkhurst, G. Winter, D. G. Waterman et al., “Robust background modelling in DIALS” (2016), J. Appl. Cryst. 49, 1912-1921 doi:10.1107/S1600576716013595.
Wilson outlier test — judging an observation without symmetry mates against the acentric and centric distributions of its shell, with the symmetry enhancement factor; rejecting only significant observations, and keeping a reflection whose observations are all large, follows AIMLESS’s EMAX test. A. J. C. Wilson, “The probability distribution of X-ray intensities” (1949), Acta Cryst. 2, 318-321 doi:10.1107/S0365110X49000813; P. Evans, “Scaling and assessment of data quality” (2006), Acta Cryst. D62, 72-82 doi:10.1107/S0907444905036693.
Amplitudes from intensities — the posterior-mean amplitude under the acentric and centric Wilson priors. S. French and K. Wilson, “On the treatment of negative intensity observations” (1978), Acta Cryst. A34, 517-525 doi:10.1107/S0567739478001114. Giving no amplitude to an intensity more than 3.7 sigma below zero, and scaling each reflection’s prior by the anisotropy tensor along its direction, follow CCP4’s ctruncate (C. Ballard and N. Stein), cited through the suite: M. D. Winn, C. C. Ballard, K. D. Cowtan et al., “Overview of the CCP4 suite and current developments” (2011), Acta Cryst. D67, 235-242 doi:10.1107/S0907444910045749.
Absorption as spherical harmonics — the empirical absorption correction as a series of real spherical harmonics of the beam directions in the crystal frame is Blessing’s; its use as a restrained scaling surface of the diffracted-beam direction follows SCALA and AIMLESS. R. H. Blessing, “An empirical correction for absorption anisotropy” (1995), Acta Cryst. A51, 33-38 doi:10.1107/S0108767394005726; P. Evans, “Scaling and assessment of data quality” (2006), Acta Cryst. D62, 72-82 doi:10.1107/S0907444905036693.
Per-frame scale smoothing — the Whittaker smoother (a second-difference penalty, the smoothness chosen by cross-validation) in Eilers’s form. P. H. C. Eilers, “A perfect smoother” (2003), Anal. Chem. 75, 3631-3636 doi:10.1021/ac034173t; E. T. Whittaker, “On a new method of graduation” (1923), Proc. Edinburgh Math. Soc. 41, 63-75 doi:10.1017/S0013091500077853.
Fisher’s z-transformation — averaging the change of the half-set CC1/2 over resolution shells on atanh(CC) when the correction surfaces are cross-validated. R. A. Fisher, “Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population” (1915), Biometrika 10, 507-521 doi:10.2307/2331838.
Frame disposition by delta-CC1/2 — the statistic in its sigma-tau form, the Fisher transformation used to compare it, its standard error, and the rejection discipline (never remove a group whose delta-CC1/2 is positive or near zero; remove a little, re-form the reference and repeat), whose reference implementation is XDSCC12. G. Assmann, W. Brehm and K. Diederichs, “Identification of rogue datasets in serial crystallography” (2016), J. Appl. Cryst. 49, 1021-1028 doi:10.1107/S1600576716005471; G. M. Assmann, M. Wang and K. Diederichs, “Making a difference in multi-data-set crystallography: simple and deterministic data-scaling/selection methods” (2020), Acta Cryst. D76, 636-652 doi:10.1107/S2059798320006348. Applying it at scaling, to groups of images rather than whole datasets, follows DIALS (dials.scale): J. Beilsten-Edmands, G. Winter, R. Gildea et al., “Scaling diffraction data in the DIALS software package: algorithms and new approaches for multi-crystal scaling” (2020), Acta Cryst. D76, 385-399 doi:10.1107/S2059798320003198.
Reasons for a weak stretch of a sweep — named in the field’s own words: “loss of centring” after autoPROC’s “loss of centring during crystal rotation” (cited under Experimental geometry and physical corrections), and “crystal rotating out of the beam” after the HKL-2000 manual (HKL: Otwinowski and Minor 1997, cited under Integration and rotation geometry).
Several sweeps merged as one — each sweep’s partials combined into fulls on their own, then the fulls of all sweeps scaled together with per-sweep frame scales and a relative B, as XSCALE merges data sets (W. Kabsch, “Integration, scaling, space-group assignment and post-refinement” (2010), above). Ordering the sweeps by dose from their relative B follows K. Diederichs, “Some aspects of quantitative analysis and correction of radiation damage” (2006), Acta Cryst. D62, 96-101 doi:10.1107/S0907444905031537.
Reference sweep of a joint merge — the sweep whose own merge reaches the finest resolution, as KAMO takes the data set of the smallest Wilson B. K. Yamashita, K. Hirata and M. Yamamoto, “KAMO: towards automated data processing for microcrystals” (2018), Acta Cryst. D74, 441-449 doi:10.1107/S2059798318004576.
Serial data¶
Per-crystal post-refinement of stills — refining per still the partiality parameters the spot positions cannot pin (a small reorientation, the rocking width, the radial scale of the reciprocal lattice) against the merged intensities, after PRIME and the mosaic-block model of Sauter et al. M. Uervirojnangkoorn, O. B. Zeldin, A. Y. Lyubimov, J. Hattne, A. S. Brewster, N. K. Sauter, A. T. Brunger and W. I. Weis, “Enabling X-ray free electron laser crystallography for challenging biological systems from a limited number of crystals” (2015), eLife 4, e05421; N. K. Sauter, J. Hattne, A. S. Brewster, N. Echols, P. H. Zwart and P. D. Adams, “Improved crystal orientation and physical properties from single-shot XFEL stills” (2014), Acta Cryst. D70, 3299-3309 (DOIs not verified here, so not given).
Indexing ambiguity without a reference — putting each crystal into the indexing in which its intensities agree best with all the others, iterated to convergence, the method CrystFEL’s ambigator also implements; rugnux measures the agreement as a product of standardised amplitudes rather than a correlation. W. Brehm and K. Diederichs, “Breaking the indexing ambiguity in serial crystallography” (2014), Acta Cryst. D70, 101-109 doi:10.1107/S1399004713025431. The same objective settles the indexing of each sweep of a joint merge, decided over all pairs at once as dials.cosym does: R. J. Gildea and G. Winter, “Determination of Patterson group symmetry from sparse multi-crystal data sets in the presence of an indexing ambiguity” (2018), Acta Cryst. D74, 405-410 doi:10.1107/S2059798318002978.
Data quality and anisotropy¶
Data-quality statistics — R_meas and R_pim, CC1/2 and CC*, the per-shell CC(model, data), and the reporting of I/sigma(I) follow the established conventions. K. Diederichs and P. A. Karplus, “Improved R-factors for diffraction data analysis in macromolecular crystallography” (1997), Nat. Struct. Biol. 4, 269-275 doi:10.1038/nsb0497-269; P. A. Karplus and K. Diederichs, “Linking crystallographic model and data quality” (2012), Science 336, 1030-1033 doi:10.1126/science.1218231; M. S. Weiss, “Global indicators of X-ray data quality” (2001), J. Appl. Cryst. 34, 130-135 doi:10.1107/S0021889800018227; K. Diederichs and P. A. Karplus, “Better models by discarding data?” (2013), Acta Cryst. D69, 1215-1222 doi:10.1107/S0907444913001121.
Isomorphism of sweeps — CC_AB / (CC*_A CC*_B) as the cosine of the angle between two data sets, on which a joint merge decides whether a sweep joins. K. Diederichs, “Dissecting random and systematic differences between noisy composite data sets” (2017), Acta Cryst. D73, 286-293 doi:10.1107/S2059798317000699.
Diffraction anisotropy — the overall fall-off as a single anisotropic displacement tensor, its symmetry constraints and the determinacy of its deviatoric part only: S. Sheriff and W. A. Hendrickson, “Description of overall anisotropy in diffraction from macromolecular crystals” (1987), Acta Cryst. A43, 118-121 doi:10.1107/S010876738709977X. Fitting it to the intensity distribution with sigma(I) taken into account: A. N. Popov and G. P. Bourenkov, “Choice of data-collection parameters based on statistic modelling” (2003), Acta Cryst. D59, 1145-1153 doi:10.1107/S0907444903008163. The directional diffraction limits and the anisotropic deltaB reported as the range of the principal components follow AIMLESS: P. R. Evans and G. N. Murshudov, “How good are my data and what is the resolution?” (2013), Acta Cryst. D69, 1204-1214 doi:10.1107/S0907444913000061. That every run should analyse and report its anisotropy is inspired by STARANISO: I. J. Tickle, C. Flensburg, P. Keller, W. Paciorek, A. Sharff, C. Vonrhein and G. Bricogne (2016), STARANISO (staraniso.globalphasing.org/cgi-bin/staraniso.cgi), Cambridge, United Kingdom: Global Phasing Ltd. Rugnux reports the anisotropy; it corrects no intensity and removes no reflection on a directional criterion.
Uncertainty conventions — the IUCr Commission on Crystallographic Nomenclature’s. D. Schwarzenbach, S. C. Abrahams, H. D. Flack et al., “Statistical descriptors in crystallography: Report of the IUCr Subcommittee on Statistical Descriptors” (1989), Acta Cryst. A45, 63-75 doi:10.1107/S0108767388009596; D. Schwarzenbach, S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, “Statistical descriptors in crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement” (1995), Acta Cryst. A51, 565-569 doi:10.1107/S0108767395002340.
Unit-cell e.s.d.s — that the e.s.d.s of a least-squares cell fit are too small is Herbstein’s survey, which is why the cell is also fitted on blocks of consecutive frames. F. H. Herbstein, “How precise are measurements of unit-cell dimensions from single crystals?” (2000), Acta Cryst. B56, 547-557 doi:10.1107/S010876810000269X.
Measurability — the fraction of acentric Bijvoet pairs significant at three sigma, reported beside SigAno and CCanom, is phenix.xtriage’s. P. H. Zwart, “Anomalous signal indicators in protein crystallography” (2005), Acta Cryst. D61, 1437-1448 doi:10.1107/S0907444905023589.
Ice in the merged intensities — reading ice off a merge’s own intensities at the hexagonal-ice ring positions, and the table of the rings’ line strengths, are phenix.xtriage’s; rugnux’s comparison is local in resolution where xtriage’s is against the whole Wilson curve. P. H. Zwart, R. W. Grosse-Kunstleve and P. D. Adams, “Xtriage and Fest: automatic assessment of X-ray data and substructure structure factor estimation” (2005), CCP4 Newsletter on Protein Crystallography 43 (no DOI).
Advice on the next collection — the rotation a sweep needs and the blind region about the spindle that a tilted second sweep recovers. Z. Dauter, “Data-collection strategies” (1999), Acta Cryst. D55, 1703-1717 (DOI not verified here).
Model placement and molecular replacement¶
Bulk-solvent correction and overall scaling — the flat bulk-solvent model, its starting values and the physically meaningful range of its two parameters: A. Fokine and A. Urzhumtsev, “Flat bulk-solvent model: obtaining optimal parameters” (2002), Acta Cryst. D58, 1387-1392 doi:10.1107/S0907444902010284. The mask (the region a probe cannot reach, shrunk back), drawn with the radii REFMAC5 uses: J.-S. Jiang and A. T. Brünger, “Protein hydration observed by X-ray diffraction: solvation properties of penicillopepsin and neuraminidase crystal structures” (1994), J. Mol. Biol. 243, 100-115 doi:10.1006/jmbi.1994.1633. The grid search over the solvent pair with scale and anisotropic B refitted at each point: P. V. Afonine, R. W. Grosse-Kunstleve and P. D. Adams, “A robust bulk-solvent correction and anisotropic scaling procedure” (2005), Acta Cryst. D61, 850-855 doi:10.1107/S0907444905007894. The fit is unweighted, as there and in REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu, R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, “REFMAC5 for the refinement of macromolecular crystal structures” (2011), Acta Cryst. D67, 355-367 doi:10.1107/S0907444911001314.
sigma_A map coefficients — the Rice and Woolfson likelihoods of |Fo| given |Fc| and sigma_A, the figure of merit m and the scale D, and the bias-corrected coefficients 2mFo-DFc (acentric) and mFo (centric). R. J. Read, “Improved Fourier coefficients for maps using phases from partial structures with errors” (1986), Acta Cryst. A42, 140-149 doi:10.1107/S0108767386099622.
ANODE — reading the anomalous difference map at the atoms of a supplied model and reporting the strongest sites by name, instead of searching the map for blobs. A. Thorn and G. M. Sheldrick, “ANODE: anomalous and heavy-atom density calculation” (2011), J. Appl. Cryst. 44, 1285-1287 doi:10.1107/S0021889811041768.
Uniform random rotations — the null a supplied model, and a molecular-replacement search, are scored against draws rotations uniformly from SO(3) through a uniform random unit quaternion. K. Shoemake, “Uniform Random Rotations”, in Graphics Gems III, ed. D. Kirk, Academic Press (1992), 124-132 (no DOI).
Variable projection — the rigid-body placement re-fits the overall scale at every step and folds that re-fit into its Jacobian, in Kaufman’s simplified form of Golub and Pereyra’s derivative. G. H. Golub and V. Pereyra, “The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate” (1973), SIAM J. Numer. Anal. 10, 413-432 doi:10.1137/0710036; L. Kaufman, “A variable projection method for solving separable nonlinear least squares problems” (1975), BIT 15, 49-57 doi:10.1007/BF01932995.
Predicted models — --model (and --mr) place an AlphaFold prediction by molecular replacement. J. Jumper, R. Evans, A. Pritzel et al., “Highly accurate protein structure prediction with AlphaFold” (2021), Nature 596, 583-589 doi:10.1038/s41586-021-03819-2; the models are those of the AlphaFold Protein Structure Database: M. Varadi, S. Anyango, M. Deshpande et al., “AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models” (2022), Nucleic Acids Res. 50, D439-D444 doi:10.1093/nar/gkab1061. The model is prepared as phenix.process_predicted_model prepares one (residues below pLDDT 70 removed, B factors from the predicted coordinate error) and cut into rigid domains by its predicted aligned error: R. D. Oeffner, T. Croll, C. Millán et al., “Putting AlphaFold models to work with phenix.process_predicted_model and ISOLDE” (2022), Acta Cryst. D78, 1303-1314 doi:10.1107/S2059798322010026; the error itself is AlphaFold’s: J. Jumper, R. Evans, A. Pritzel et al., “Highly accurate protein structure prediction with AlphaFold” (2021), Nature 596, 583-589 doi:10.1038/s41586-021-03819-2.
Molecular replacement search — scoring each orientation of the model by interpolating its molecular transform, computed once by FFT, as AMoRe does: J. Navaza, “AMoRe: an automated package for molecular replacement” (1994), Acta Cryst. A50, 157-163 doi:10.1107/S0108767393007597. The full-symmetry translation function, all pairs of symmetry copies on one Fourier synthesis: R. A. Crowther and D. M. Blow, “A method of positioning a known molecule in an unknown crystal structure” (1967), Acta Cryst. 23, 544-548 doi:10.1107/S0365110X67003172. A placement is a solution, as Phaser calls one, where its translation-function Z-score reaches 8: A. J. McCoy, R. W. Grosse-Kunstleve, P. D. Adams, M. D. Winn, L. C. Storoni and R. J. Read, “Phaser crystallographic software” (2007), J. Appl. Cryst. 40, 658-674 doi:10.1107/S0021889807021206.
Cell contents — whether one copy fits the cell is read off the Matthews coefficient: B. W. Matthews, “Solvent content of protein crystals” (1968), J. Mol. Biol. 33, 491-497 doi:10.1016/0022-2836(68)90205-2; how many copies to search for is weighed by its distribution over protein crystals: K. A. Kantardjieff and B. Rupp, “Matthews coefficient probabilities: Improved estimates for unit cell contents of proteins, DNA, and protein-nucleic acid complex crystals” (2003), Protein Sci. 12, 1865-1871 doi:10.1110/ps.0350503. The placements of several copies are compared by least-squares superposition: W. Kabsch, “A solution for the best rotation to relate two sets of vectors” (1976), Acta Cryst. A32, 922-923 doi:10.1107/S0567739476001873.
Likelihood scoring of placements — the log-likelihood gain from the Rice and Woolfson distributions of the normalised amplitudes given the model’s, with sigma_A falling off with resolution for the model’s expected coordinate error as Luzzati’s D: V. Luzzati, “Traitement statistique des erreurs dans la détermination des structures cristallines” (1952), Acta Cryst. 5, 802-810 doi:10.1107/S0365110X52002161; R. J. Read, “Pushing the boundaries of molecular replacement with maximum likelihood” (2001), Acta Cryst. D57, 1373-1382 doi:10.1107/S0907444901012471; the gain read as a Z-score over the same orientation at other positions, as the likelihood-enhanced translation function’s is: A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni and R. J. Read, “Likelihood-enhanced fast translation functions” (2005), Acta Cryst. D61, 458-464 doi:10.1107/S0907444905001617; and a model whose expected log-likelihood gain is too small for a placement to stand out is not searched for: A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason, B. Lohkamp and R. J. Read, “Ab initio solution of macromolecular crystal structures without direct methods” (2017), Proc. Natl. Acad. Sci. USA 114, 3637-3641 doi:10.1073/pnas.1701640114. Translational NCS is looked for in the native Patterson before the search, read as under Translational pseudo-symmetry. The logarithm of the Bessel function I0 is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun, “Handbook of Mathematical Functions” (1964), National Bureau of Standards, 9.8.1-9.8.2.
Structure refinement¶
Likelihood target — the Rice distribution of an observed amplitude given the model’s, and its centric counterpart. N. S. Pannu and R. J. Read, “Improved structure refinement through maximum likelihood” (1996), Acta Cryst. A52, 659-668 doi:10.1107/S0108767396004370; G. N. Murshudov, A. A. Vagin and E. J. Dodson, “Refinement of macromolecular structures by the maximum-likelihood method” (1997), Acta Cryst. D53, 240-255 doi:10.1107/S0907444996012255. Its parameters are fitted in resolution bins, the variance on the free reflections and D on the working ones, as REFMAC5 divides them (Murshudov et al. 2011, above); estimating them on free reflections follows V. Y. Lunin and T. P. Skovoroda, “R-free likelihood-based estimates of errors for phases calculated from atomic models” (1995), Acta Cryst. A51, 880-887 doi:10.1107/S010876739500688X; V. Y. Lunin, P. V. Afonine and A. G. Urzhumtsev, “Likelihood-based refinement. I. Irremovable model errors” (2002), Acta Cryst. A58, 270-282 doi:10.1107/S0108767302001046.
Gradients by FFT — the derivatives of the target with respect to the structure factors transformed into a map and summed against each atom’s density derivatives. R. C. Agarwal, “A new least-squares refinement technique based on the fast Fourier transform algorithm” (1978), Acta Cryst. A34, 791-809 doi:10.1107/S0567739478001618.
Minimiser — L-BFGS (rugnux’s own implementation). D. C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization” (1989), Math. Program. 45, 503-528 doi:10.1007/BF01589116; its two-loop recursion: J. Nocedal, “Updating quasi-Newton matrices with limited storage” (1980), Math. Comp. 35, 773-782 doi:10.1090/S0025-5718-1980-0572855-7. The initial inverse Hessian is rescaled at each iteration by the latest pair, the self-scaling of S. S. Oren and D. G. Luenberger, “Self-scaling variable metric (SSVM) algorithms. Part I” (1974), Management Sci. 20, 845-862 doi:10.1287/mnsc.20.5.845, and D. F. Shanno and K.-H. Phua, “Matrix conditioning and nonlinear optimization” (1978), Math. Program. 14, 149-160 doi:10.1007/BF01588962.
Geometry restraints — those of the CCP4 Monomer Library, a subset of which rugnux carries (see THIRD_PARTY_NOTICES.md). A. A. Vagin, R. A. Steiner, A. A. Lebedev, L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, “REFMAC5 dictionary: organization of prior chemical knowledge and guidelines for its use” (2004), Acta Cryst. D60, 2184-2195 doi:10.1107/S0907444904023510. They are applied through GEMMI’s topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, “GEMMI: A library for structural biology” (2022), J. Open Source Softw. 7, 4200 doi:10.21105/joss.04200. The torsion derivatives follow A. Blondel and M. Karplus, “New formulation for derivatives of torsion angles and improper torsion angles in molecular mechanics: elimination of singularities” (1996), J. Comput. Chem. 17, 1132-1141 doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T.
Riding hydrogens — placed on their parent atoms by GEMMI’s riding-hydrogen code, which follows D. Liebschner, P. V. Afonine, A. G. Urzhumtsev and P. D. Adams, “Implementation of the riding hydrogen model in CCTBX to support the next generation of X-ray and neutron joint refinement in Phenix” (2020), Methods Enzymol. 634, 177-199 doi:10.1016/bs.mie.2020.01.007.
B-factor restraints — bonded atoms, and atoms in contact, restrained to each other in the Kullback-Leibler form REFMAC5 uses (Murshudov et al. 2011, above); the divergence itself is S. Kullback and R. A. Leibler’s, “On information and sufficiency” (1951), Ann. Math. Statist. 22, 79-86 doi:10.1214/aoms/1177729694. The overall isotropic B of the scale is moved into the atoms as phenix.refine moves it (Afonine et al. 2012, below).
Weights and R-free — the weights of the X-ray term against the geometry and B restraints start from the ratios of the gradients, as X-PLOR’s crystallographic weight was first set: A. T. Brünger, M. Karplus and G. A. Petsko, “Crystallographic refinement by simulated annealing: application to crambin” (1989), Acta Cryst. A45, 50-61 doi:10.1107/S0108767388009195; and are chosen among short refinements by R-free with the geometry held within bounds, as phenix.refine chooses them: P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd, N. W. Moriarty, M. Mustyakimov, T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams, “Towards automated crystallographic structure refinement with phenix.refine” (2012), Acta Cryst. D68, 352-367 doi:10.1107/S0907444912001308. R-free itself: A. T. Brünger, “Free R value: a novel statistical quantity for assessing the accuracy of crystal structures” (1992), Nature 355, 472-475 doi:10.1038/355472a0.
Robust scaling of the model — the overall scale and anisotropic B are fitted by iteratively reweighted least squares with the pseudo-Huber weight. P. J. Huber, “Robust estimation of a location parameter” (1964), Ann. Math. Statist. 35, 73-101 doi:10.1214/aoms/1177703732, in the smooth form of P. Charbonnier, L. Blanc-Féraud, G. Aubert and M. Barlaud, “Deterministic edge-preserving regularization in computed imaging” (1997), IEEE Trans. Image Process. 6, 298-311 doi:10.1109/83.551699.
Metal-site restraints — the bonds of metal ions to the donor atoms the model’s LINK records name are restrained at the distances of the Monomer Library’s own metal-site statistics (metals.json, embedded as monomers/metals.cif), in line with the surveys of M. M. Harding, “Geometry of metal-ligand interactions in proteins” (2001), Acta Cryst. D57, 401-411 doi:10.1107/S0907444900019168 and “Small revisions to predicted distances around metal sites in proteins” (2006), Acta Cryst. D62, 678-682 doi:10.1107/S0907444906014594, and, for Mg²⁺, Na⁺ and K⁺ at nucleic acids, F. Leonarski, L. D’Ascenzo and P. Auffinger, “Mg2+ ions: do they bind to nucleobase nitrogens?” (2017), Nucleic Acids Res. 45, 987-1004 doi:10.1093/nar/gkw1175.
Anisotropic displacements — the TLS model of a rigid body’s translation, libration and their correlation: V. Schomaker and K. N. Trueblood, “On the rigid-body motion of molecules in crystals” (1968), Acta Cryst. B24, 63-76 doi:10.1107/S0567740868001718; refined against the likelihood with a residual B per atom, the B column read as total or residual, as in M. D. Winn, M. N. Isupov and G. N. Murshudov, “Use of TLS parameters to model anisotropic displacements in macromolecular refinement” (2001), Acta Cryst. D57, 122-133 doi:10.1107/S0907444900014736. The chains are cut into TLS groups as TLSMD cuts them: J. Painter and E. A. Merritt, “Optimal description of a protein structure in terms of multiple groups undergoing TLS motion” (2006), Acta Cryst. D62, 439-450 doi:10.1107/S0907444906005270. Atoms with a U of their own are restrained by the rigid-bond postulate of F. L. Hirshfeld, “Can X-ray data distinguish bonding effects from vibrational smearing?” (1976), Acta Cryst. A32, 239-244 doi:10.1107/S0567739476000533, in the form of SHELXL’s RIGU: A. Thorn, B. Dittrich and G. M. Sheldrick, “Enhanced rigid-bond restraints” (2012), Acta Cryst. A68, 448-451 doi:10.1107/S0108767312014535, with SHELXL’s SIMU and ISOR restraints as G. M. Sheldrick, “Crystal structure refinement with SHELXL” (2015), Acta Cryst. C71, 3-8 doi:10.1107/S2053229614024218 documents them. Which displacement model a refinement keeps is decided on the free set rather than by a rule of resolution: E. A. Merritt, “To B or not to B: a question of resolution?” (2012), Acta Cryst. D68, 468-477 doi:10.1107/S0907444911028320.
NCS restraints — copies found by sequence alignment with REFMAC5’s criterion (Murshudov et al. 2011, above) and restrained locally on pairs of corresponding interatomic distances under the Geman-McClure loss, as REFMAC5’s local NCS restraints are, with the distance range of R. A. Nicholls, F. Long and G. N. Murshudov, “Low-resolution refinement tools in REFMAC5” (2012), Acta Cryst. D68, 404-417 doi:10.1107/S090744491105606X. The superposition behind the rms criterion is GEMMI’s QCP: D. L. Theobald, “Rapid calculation of RMSDs using a quaternion-based characteristic polynomial” (2005), Acta Cryst. A61, 478-480 doi:10.1107/S0108767305015266; P. Liu, D. K. Agrafiotis and D. L. Theobald, “Fast determination of the optimal rotational matrix for macromolecular superpositions” (2010), J. Comput. Chem. 31, 1561-1563 doi:10.1002/jcc.21439. The case for local restraints over a superposition, decided by R-free: G. J. Kleywegt, “Use of non-crystallographic symmetry in protein structure refinement” (1996), Acta Cryst. D52, 842-857 doi:10.1107/S0907444995016477; I. Usón, E. Pohl, T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz and G. M. Sheldrick, “1.7 Å structure of the stabilized REIv mutant T39K. Application of local NCS restraints” (1999), Acta Cryst. D55, 1158-1167 doi:10.1107/S0907444999003972; and J. J. Headd, N. Echols, P. V. Afonine, N. W. Moriarty, R. J. Gildea and P. D. Adams, “Flexible torsion-angle noncrystallographic symmetry restraints for improved macromolecular structure refinement” (2014), Acta Cryst. D70, 1346-1356 doi:10.1107/S1399004714003277, whose comparison is why the copies’ B factors are refined independently.
Special positions — an atom on a special position is kept on it, its shifts confined to the subspace its site symmetry leaves unchanged. R. W. Grosse-Kunstleve and P. D. Adams, “Algorithms for deriving crystallographic space-group information. II. Treatment of special positions” (2002), Acta Cryst. A58, 60-65 doi:10.1107/S0108767301016658.
Solvent, ions and unmodelled density¶
Unmodelled density — the connected regions of the difference map above a cutoff, found by flood fill as Coot’s “unmodelled blobs” are, in GEMMI’s implementation. P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, “Features and development of Coot” (2010), Acta Cryst. D66, 486-501 doi:10.1107/S0907444910007493.
Water build — the documented ordered-solvent defaults of phenix.refine (difference-map peaks from 3 σ, picking to 2.8 Å, waters added, filtered and refined between macro-cycles; Afonine et al. 2012, above); the distance band to the model, widened a little from the 2.4-3.2 Å Coot’s Find Waters keeps (Emsley et al. 2010, above); and the removal of a water in negative difference density at -3 σ (Echols et al. 2014, below).
Ion stereochemistry — the coordination bands, the Mg²⁺ exclusion zone and the clash shells of the surveys of ions at nucleic acids by P. Auffinger’s group. N. Naleem, A. Henning-Knechtel, S. Kirmizialtin and P. Auffinger, “Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and annotating Mg2+ ions in RNA structures” (2026), Nucleic Acids Res. 54, gkag120 doi:10.1093/nar/gkag120; F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar and P. Auffinger, “Principles of ion binding to RNA inferred from the analysis of a 1.55 Å resolution bacterial ribosome structure – Part I: Mg2+” (2025), Nucleic Acids Res. 53, gkae1148 doi:10.1093/nar/gkae1148; F. Leonarski, L. D’Ascenzo and P. Auffinger, “Nucleobase carbonyl groups are poor Mg2+ inner-sphere binders but excellent monovalent ion binders—a critical PDB survey” (2019), RNA 25, 173-192 doi:10.1261/rna.068437.118; and their 2017 survey (above). The score that combines the criteria is rugnux’s own.
Ion identification — the ligand sphere of the bond-valence sums, the reading of halides by their donors, and telling a light ion from a water by its density: N. Echols, N. Morshed, P. V. Afonine, A. J. McCoy, M. D. Miller, R. J. Read, J. S. Richardson, T. C. Terwilliger and P. D. Adams, “Automated identification of elemental ions in macromolecular crystal structures” (2014), Acta Cryst. D70, 1104-1114 doi:10.1107/S1399004714001308; N. Morshed, N. Echols and P. D. Adams, “Using support vector machines to improve elemental ion identification in macromolecular crystal structures” (2015), Acta Cryst. D71, 1147-1158 doi:10.1107/S1399004715004241. The chloride’s shell of donors: O. Carugo, “Buried chloride stereochemistry in the Protein Data Bank” (2014), BMC Struct. Biol. 14, 19 doi:10.1186/s12900-014-0019-8; halides at hydrophobic surfaces: Z. Dauter and M. Dauter, “Entering a new phase: using solvent halide ions in protein structure determination” (2001), Structure 9, R21-R26 doi:10.1016/S0969-2126(01)00565-2.
Bond-valence sums — I. D. Brown and D. Altermatt, “Bond-valence parameters obtained from a systematic analysis of the Inorganic Crystal Structure Database” (1985), Acta Cryst. B41, 244-247 doi:10.1107/S0108768185002063, with the R₀ of N. E. Brese and M. O’Keeffe, “Bond-valence parameters for solids” (1991), Acta Cryst. B47, 192-197 doi:10.1107/S0108768190011041, as tabulated by H. Zheng, K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, “Data mining of iron(II) and iron(III) bond-valence parameters, and their relevance for macromolecular crystallography” (2017), Acta Cryst. D73, 316-325 doi:10.1107/S2059798317000584; weighted by occupancy as P. Müller, S. Köpke and G. M. Sheldrick weight them, “Is the bond-valence method able to identify metal atoms in protein structures?” (2003), Acta Cryst. D59, 32-37 doi:10.1107/S0907444902018000; the normalised vector sum and its outlier bound are CheckMyMetal’s: H. Zheng, D. R. Cooper, P. J. Porebski, I. G. Shabalin, K. B. Handing and W. Minor, “CheckMyMetal: a macromolecular metal-binding validation tool” (2017), Acta Cryst. D73, 223-233 doi:10.1107/S2059798317001061.
Anomalous scattering factors — f″ at the data’s wavelength, as GEMMI computes it (its port of the Larch routines). D. T. Cromer and D. A. Liberman, “Anomalous dispersion calculations near to and on the long-wavelength side of an absorption edge” (1981), Acta Cryst. A37, 267-268 doi:10.1107/S0567739481000600, with the corrections of L. Kissel and R. H. Pratt, “Corrections to tabulated anomalous-scattering factors” (1990), Acta Cryst. A46, 170-175 doi:10.1107/S0108767389010718.
Software and computing methods¶
GPU decompression — decoding bitshuffle+LZ4 images on the GPU rather than on the host follows Jon Wright (ESRF): “Experiences with GPU decompression for bitshuffle + LZ4 data”, HDF5 User Group meeting (2021), and bslz4decoders. The CUDA kernels are Jungfraujoch’s own; the approach is his.
Parallel connected components — removing the islands of the bulk-solvent mask on the GPU with a parallel union-find. D. P. Playne and K. Hawick, “A New Algorithm for Parallel Connected-Component Labelling on GPUs” (2018), IEEE Trans. Parallel Distrib. Syst. 29, 1217-1230 doi:10.1109/TPDS.2018.2799216.
Correctly rounded integer division — the FFT indexer’s GPU peak search divides by a reciprocal multiplication corrected once by the exact remainder. P. W. Markstein, “Computation of elementary functions on the IBM RISC System/6000 processor” (1990), IBM J. Res. Dev. 34, 111-119.
Anderson acceleration — of the per-frame scaling loops of the rotation merge, in the type-II form. H. F. Walker and P. Ni, “Anderson acceleration for fixed-point iterations” (2011), SIAM J. Numer. Anal. 49, 1715-1735 doi:10.1137/10078356X, after D. G. Anderson, “Iterative procedures for nonlinear integral equations” (1965), J. ACM 12, 547-560 doi:10.1145/321296.321305.
Bootstrap — the interval reported for ISa. B. Efron, “Bootstrap methods: another look at the jackknife” (1979), Ann. Statist. 7, 1-26 doi:10.1214/aos/1176344552. Its draws are Poisson weights hashed with the SplitMix64 finaliser of G. L. Steele Jr., D. Lea and C. H. Flood, “Fast splittable pseudorandom number generators” (2014), Proc. OOPSLA 2014, 453-472 doi:10.1145/2660193.2660195.
Error-free floating-point arithmetic — the single-precision exp shared between the CPU and an Apple GPU (BraggExp.h): the argument reduction of W. J. Cody and W. Waite, “Software Manual for the Elementary Functions” (1980), Prentice-Hall; the exact product (with Veltkamp’s split) and Fast2Sum of T. J. Dekker, “A floating-point technique for extending the available precision” (1971), Numer. Math. 18, 224-242; and the 2Sum of D. E. Knuth, “The Art of Computer Programming, Vol. 2: Seminumerical Algorithms”, section 4.2.2.
Colormaps — Viridis, Magma and Inferno from Matplotlib, under its BSD-compatible license. J. D. Hunter, “Matplotlib: A 2D graphics environment” (2007), Comput. Sci. Eng. 9, 90-95 doi:10.1109/MCSE.2007.55.
Mol* — the web frontend’s structure viewer. D. Sehnal, S. Bittrich, M. Deshpande, R. Svobodová, K. Berka, V. Bazgier, S. Velankar, S. K. Burley, J. Koča and A. S. Rose, “Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures” (2021), Nucleic Acids Res. 49, W431-W437 doi:10.1093/nar/gkab314. Its setup follows the one Guillaume Gotthard (PSI) tuned for AareDB, the sample-management and data-pipeline system of the PSI MX beamlines: the Coot-style mouse controls, the model representations and colours, and the display of the density maps. The virtual trackball is a port of Coot’s, itself a port of SGI’s trackball.c by Gavin Bell.
File formats read from a published specification¶
CBF / imgCIF — the native miniCBF reader implements the x-CBF_BYTE_OFFSET compression and the imgCIF _axis table from the specification alone; no CBFlib or other CBF code is used. H. J. Bernstein and A. P. Hammersley, “Specification of the Crystallographic Binary File (CBF/imgCIF)” (2006), International Tables for Crystallography Vol. G, 37-43 doi:10.1107/97809553602060000729; A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, “Image dictionary (imgCIF)” (2006), International Tables for Crystallography Vol. G, 444-458 doi:10.1107/97809553602060000746.
PILATUS count-rate clipping — that the count-rate correction clips an over-range pixel to the cutoff rather than flagging it, which is how the miniCBF reader tells a saturated pixel. P. Trueb, C. Dejoie, M. Kobas, P. Pattison, D. J. Peake, V. Radicci, B. A. Sobott, D. A. Walko and C. Broennimann, “Bunch mode specific rate corrections for PILATUS3 detectors” (2015), J. Synchrotron Rad. 22, 701-707 doi:10.1107/S1600577515003288.
d*TREK SMV — the SMV reader reads the d*TREK header vocabulary of Rigaku’s CrystalClear from the headers themselves, interpreting the detector vectors as dxtbx does; no d*TREK or dxtbx code is used. J. W. Pflugrath, “The finer things in X-ray diffraction data collection” (1999), Acta Cryst. D55, 1718-1725 doi:10.1107/S090744499900935X; dxtbx: J. M. Parkhurst, A. S. Brewster, L. Fuentes-Montero, D. G. Waterman et al., “dxtbx: the diffraction experiment toolbox” (2014), J. Appl. Cryst. 47, 1459-1465 doi:10.1107/S1600576714011996.
Public diffraction data used for testing¶
In addition to in-house datasets collected at SLS 2.0, Jungfraujoch is tested against public diffraction data collected on other people’s beamlines, on detectors and in file formats we do not produce ourselves. Every dataset used, the DOI to cite for it, and the deposition it belongs to are listed in EXTERNAL_TEST_DATA and EXTERNAL_TEST_DATA_STILLS; we thank the depositors, and the repositories that make the data findable and citable.
IRRMC, the Integrated Resource for Reproducibility in Macromolecular Crystallography (Minor lab, University of Virginia), is the source of the largest share of them; it releases its data under CC0 and asks that the DOI of the dataset be cited. M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng, D. R. Cooper, M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, “A public database of macromolecular diffraction experiments” (2016), Acta Cryst. D72, 1181-1193 doi:10.1107/S2059798316014716; M. Grabowski, M. Cymborowski, P. J. Porebski, T. Osinski, I. G. Shabalin, D. R. Cooper and W. Minor, “The Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first four years” (2019), Struct. Dyn. 6, 064301 doi:10.1063/1.5128672.
SBGrid Data Bank, the structural biology community’s data publication service (SBGrid Consortium, Harvard Medical School). P. A. Meyer, S. Socias, J. Key, E. Ransey, E. C. Tjon, A. Buschiazzo et al., “Data publication with the structural biology data grid supports live analysis” (2016), Nat. Commun. 7, 10882 doi:10.1038/ncomms10882.
Zenodo, CERN’s open repository, hosts datasets deposited there directly by the groups that collected them. European Organization for Nuclear Research and OpenAIRE, “Zenodo” (2013), CERN doi:10.25495/7GXK-RD71. The IUCrData Raw Data Letters describing some of those deposits are cited on the EXTERNAL_TEST_DATA page.
CXIDB, the Coherent X-ray Imaging Data Bank, is the source of the serial (XFEL stills) datasets. F. R. N. C. Maia, “The Coherent X-ray Imaging Data Bank” (2012), Nat. Methods 9, 854-855 doi:10.1038/nmeth.2110.
MXRDR, the Macromolecular Xtallography Raw Data Repository (ICM, University of Warsaw), releases its data under CC0 and asks that the DOI of the dataset be cited.
XRDa, the Xtal Raw Data Archive (Protein Data Bank Japan), publishes raw diffraction images and mints a DOI for each, which it asks to be cited. It has no canonical citation paper.
The ESRF data portal, through which the European Synchrotron publishes raw data under its data policy (CC BY 4.0, with the dataset DOI to be cited). R. Dimper, A. Götz, A. De Maria, V. A. Solé, M. Chaillet and B. Lebayle, “ESRF Data Policy, Storage, and Services” (2019), Synchrotron Rad. News 32, 7-12 doi:10.1080/08940886.2019.1608119.
The Keele University research data repository and UQ eSpace (The University of Queensland) host the raw images of datasets deposited there by the groups that collected them; the dataset DOIs are cited on the EXTERNAL_TEST_DATA page.
The Protein Data Bank supplies, through the RCSB PDB data API, the beamline, resolution, space group and unit cell quoted for each dataset, and the deposited coordinates and structure factors of the public entries the refinement battery starts from. H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and P. E. Bourne, “The Protein Data Bank” (2000), Nucleic Acids Res. 28, 235-242 doi:10.1093/nar/28.1.235.
The Crystallography Open Database supplies the reference cells of the small-molecule sets. S. Gražulis, D. Chateigner, R. T. Downs, A. F. T. Yokochi, M. Quirós, L. Lutterotti, E. Manakova, J. Butkus, P. Moeck and A. Le Bail, “Crystallography Open Database – an open-access collection of crystal structures” (2009), J. Appl. Cryst. 42, 726-729 doi:10.1107/S0021889809016690; S. Gražulis, A. Daškevič, A. Merkys et al., “Crystallography Open Database (COD): an open-access collection of crystal structures and platform for world-wide collaboration” (2012), Nucleic Acids Res. 40, D420-D427 doi:10.1093/nar/gkr900.
The AlphaFold Protein Structure Database (EMBL-EBI and Google DeepMind) supplies the start models of the battery’s timing benchmark: the predictions of the deposited entities of 9ZLO (AF-P17090-F1), 9S02 (AF-P32322-F1) and 9GJX (AF-Q65MG6-F1) and, for the in-house benchmark, of horse myoglobin (AF-P68082-F1) and bovine cytochrome c (AF-P62894-F1), model version 6, with their predicted aligned error, kept unmodified in tools/battery/models/ under CC BY 4.0. J. Jumper, R. Evans, A. Pritzel et al., “Highly accurate protein structure prediction with AlphaFold” (2021), Nature 596, 583-589 doi:10.1038/s41586-021-03819-2; M. Varadi, S. Anyango, M. Deshpande et al., “AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models” (2022), Nucleic Acids Res. 50, D439-D444 doi:10.1093/nar/gkab1061; M. Varadi, D. Bertoni, P. Magana et al., “AlphaFold Protein Structure Database in 2024: providing structure coverage for over 214 million protein sequences” (2024), Nucleic Acids Res. 52, D368-D375 doi:10.1093/nar/gkad1011. The large case of both benchmarks is the deposited model of PDB entry 4NWV, moved away from its place.
Generative AI usage declaration¶
Large language models were used extensively in developing this code. Jungfraujoch development was supported with JetBrains AI (mostly GPT models) to refactor and verify particular code fragments. Rugnux was developed with the assistance of Claude Code (mostly the Opus model). This documentation was written with the assistance of Claude Opus and Fable models.