Files
Jungfraujoch/docs/ACKNOWLEDGEMENT.md
T
leonarski_f a39fd29f77
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m4s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 17m46s
Build Packages / build:windows:cuda (push) Successful in 20m20s
Build Packages / build:viewer-tgz:cpu (push) Successful in 15m56s
Build Packages / build:viewer-tgz:cuda (push) Successful in 17m57s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 14m10s
Build Packages / build:rugnux:windows (push) Successful in 11m12s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 7m14s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m13s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 19m17s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 21m21s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 17m26s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m56s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 20m48s
Build Packages / build:rpm (rocky8) (push) Successful in 23m43s
Build Packages / build:rpm (rocky9) (push) Successful in 20m38s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 24m57s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m58s
Build Packages / XDS test (durin plugin) (push) Successful in 10m43s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / Build documentation (push) Successful in 1m5s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (neggia plugin) (push) Successful in 8m57s
Build Packages / DIALS test (push) Successful in 18m40s
v1.0.0-rc.167 (#77)
* `rugnux --model` reports CC(model, data) - the correlation of the merged intensities with the placed, scaled model - by resolution shell, on the same shells as CC1/2, with the reflection count and a significance for each.
* `rugnux --model` fits the model's scale, anisotropic B and bulk-solvent parameters on the working reflections only, so the R-free it reports is measured against a model no free reflection helped scale.
* The bulk-solvent parameters of `rugnux --model` are searched over their physically meaningful range instead of being fitted without bounds, so a model is never scaled with a solvent term that has silently switched itself off.
* The rigid-body placement of `rugnux --model` uses the same bounded bulk solvent as the reported fit, so a model is no longer placed against a target carrying a solvent term with no physical meaning.
* `rugnux --model` puts the model into the data's own description of the lattice before placing it, so a model whose cell is written on other axes - I-centred where the run indexed C-centred, a different unique axis, a permuted orthorhombic cell - is placed rather than scored where it was read; `MODEL_CHANGE_OF_BASIS=` and `MODEL_SETTING_AS_READ=` report it when it happens.
* The rugnux results report opens with a summary - `VERDICT=` (`OK`, `WARNINGS`, `UNUSABLE`, `FAILED`), `VERDICT_TEXT=`, `PATHOLOGY_FLAGS=` with one closed-vocabulary code per condition that warned, and the `WARNING:` lines, which used to close the file - and the sections after it are renumbered 1-5 with no gaps.
* `rugnux --developer` writes the full results report - the pipeline-internal keys and the long explanations the default report now leaves out - and `--finalist-ledger` adds the evidence for every space group the search considered, not only the one it adopted.
* The results report warns when the merged data carry no usable signal and when too little of reciprocal space was measured inside the fitted resolution, and omits `FITTED_RESOLUTION` where the CC1/2 curve it is fitted on never falls off.
* rugnux detects translational pseudo-symmetry and reports it under the `PSEUDO_TRANSLATION` flag as `TNCS_DETECTED=` and the `TNCS_*` keys - a translation the merged data are exactly invariant under is reported as `UNDECLARED_LATTICE_TRANSLATION=` under `LATTICE_TRANSLATION` instead - and a detected pseudo-translation can no longer buy a false screw axis in the space-group search or hide a twin from the L-test (`L_TEST_VS_TNCS=`).
* The space-group search determines glide planes from zonal systematic absences, so a non-Sohncke space group such as P 2_1/c or Pbca is named where the run previously stopped at its Sohncke subgroup; `SOHNCKE_SPACE_GROUP=` carries the best Sohncke group beside it on every run that searched, and a centre of symmetry is never claimed.
* Where the cell metric carries more rotational symmetry than the Bravais class the indexer named, the extra rotations are put to the intensities and the space-group search is asked again on the metric's own cell - adopted only where the intensities confirm the higher symmetry - so a lattice that is nearly but not exactly hexagonal, or whose reduction landed in a sub-cell, still reaches its true point group.
* Systematic-absence calls rest on the evidence rather than on counts: a screw axis whose absent class the data show extinct is no longer refused because a handful of reflections in it read as present, and `SPACE_GROUP_ALTERNATIVES=` no longer drops a candidate that differs only on a zone the sweep never measured.
* A reference correlation measured on too few reflections is refused instead of scored zero, so a run given a reference MTZ is no longer reindexed on an operator that mapped almost everything outside the reference's coverage.
* A frame counts as indexed from 6 spots on its lattice rather than 9, so a weakly diffracting crystal whose frames cannot carry 9 is no longer refused the lattice it fits; `--min-indexed-spots` overrides it.
* `-C` accepts a known cell in any equivalent description - conventional or primitive, centred or not - instead of only the reduced primitive form, so a centred cell given the way it is published no longer makes the run report that it found no lattice.
* Each reflection is corrected for the sensor's quantum efficiency at the angle it meets the detector (attenuation lengths from the NIST tables, which also fixes the spot-width parallax term on CdTe) and for the attenuation of the flight path between the sample and its pixel; `--flight-path air|helium|vacuum` declares the medium - default air, since no file states it - and the report says what was assumed and what it was worth. The unmerged MTZ records the factors in new `QE` and `FLIGHT` columns beside `LP`, so raw counts are `I / LP * QE * FLIGHT`, and `_process.h5` in new optional `qe` and `flight` datasets.
* Rotation geometry post-refinement fits the crystal and the detector at once, against the observed spot positions and the observed rocking angles together, so the refined distance depends far less on how wrong the file's distance was.
* A coarsely sliced sweep integrates correctly: partials are joined into one rocking event by angle rather than by frame count, so two crossings of the Ewald sphere are no longer summed into one full, and at 0.5 degrees per image or coarser the per-frame geometry refinement accepts a spot whose miss the exposure's own rotation accounts for.
* `rugnux --mode scale` reports the detector tilt and direct beam of the geometry it re-scaled at, instead of zeros that read as a flat detector, and no longer warns that no image was indexed on a run whose lattice came from its input file.
* Every rotation run that determined a space group and merged reports what the mounting cost: `SPINDLE_LOST_UNIQUE_FRACTION=` is the fraction (0-1) of unique reflections the mounting made unmeasurable under the measured point group, also written to the master as `/entry/MX/spindleLostUniqueFraction` and what the mounting warning fires on; `SPINDLE_SYMMETRY_AXIS_ANGLE_DEG=` / `SPINDLE_SYMMETRY_AXIS_ORDER=` describe the mounting in the `--developer` report.
* Stills and grid scans carry a per-image `spindle_blind_fraction` - how much of a rotation sweep's blind cone this orientation would make unrecoverable, 0.5 and above calling for a second orientation - through the CBOR stream, HDF5 (`/entry/MX/spindleBlindFraction`), the plot and scan-result APIs, and the viewer and frontend plots; an absent value means the frame could not be assessed and is not a 0.
* The results report's `REPORT_VERSION` is 7.

Reviewed-on: #77
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-09-09 07:25:13 +02:00

366 lines
27 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Acknowledgements
Citation: F. Leonarski, M. Bruckner, C. Lopez-Cuenca, A. Mozzanica, H.-C. Stadler, Z. Matej, A. Castellane, B. Mesnet, J. Wojdyla, B. Schmitt and M. Wang "Jungfraujoch: hardware-accelerated data-acquisition system for kilohertz pixel-array X-ray detectors" (2023), J. Synchrotron Rad., 30, 227-234 [doi:10.1107/S1600577522010268](https://doi.org/10.1107/S1600577522010268).
## Funding and support
The project is supported by :
* Innosuisse via Innovation Project "NextGenDCU high data rate acquisition system for X-ray detectors in structural biology applications" (101.535.1 IP-ENG; Apr 2023 - Sep 2025).
* ETH Domain via Open Research Data Contribute project (Jan - Dec 2023)
* AMD University Program with donation of licenses of Ethernet IP cores and Vivado software
## Crystallographic methods adopted from other packages
The analysis pipeline reimplements methods first published, and in most cases first implemented, by
other crystallographic software. The code below is Jungfraujoch's own; the methods are theirs, and
are acknowledged here. Where a package's source was consulted this is said explicitly. Most of these
packages are neither linked nor vendored; the three that are - GEMMI, traccc and
fast-feedback-indexer - also carry a licence obligation, recorded in
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
### Spot finding
**[CrystFEL](https://www.desy.de/~twhite/crystfel/)** — spot finding, the three-ring integration
region, the serial/stills processing model, and the per-frame indexing acceptance test
(`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass,
A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012),
J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312).
The self-calibrating spot finder's per-resolution-ring background statistics, with the Bragg peaks
excluded by iterated clipping, follow Cheetah's peakfinder8: A. Barty, R. A. Kirian,
F. R. N. C. Maia, M. Hantke, C. H. Yoon, T. A. White and H. N. Chapman, "Cheetah: software for
high-throughput reduction and analysis of serial femtosecond X-ray diffraction data" (2014),
J. Appl. Cryst. 47, 1118-1131
[doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626).
Spot extraction groups strong pixels into spots with the sparse connected-component labelling of the
ACTS traccc project: P. Gessinger, H. M. Gray, A. Krasznahorkay, C. Leggett, J. Niermann,
A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction library for HEP
experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822);
[traccc](https://github.com/acts-project/traccc). The CPU spot extractor adapts its SparseCCL source,
and the CUDA spot extractor follows the design of its GPU counterpart - a backward-neighbour graph
over a sorted hit list, resolved by a parallel union-find. traccc is MPL-2.0; see
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). The SparseCCL algorithm itself is A. Hennequin,
B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: Connected Components Labeling and
Analysis for sparse images" (2019), DASIP 2019, 65-70
[doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184).
### Indexing
**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/)** — the Rossmann FFT autoindexing algorithm and
post-refinement practice, including which parameters are safe to refine per image and which must be
refined over a wedge. The autoindexing algorithm itself — projecting the reciprocal-space points
onto many directions and Fourier-transforming the 1D projection histograms — is I. Steller,
R. Bolotovsky and M. G. Rossmann, "An algorithm for automatic indexing of oscillation images using
Fourier analysis" (1997), J. Appl. Cryst. 30, 1036-1040
[doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777); MOSFLM is the
implementation whose practice is followed. A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM"
(2007), in *Evolving Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245,
41-51 [doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4);
T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new
graphical interface for diffraction-image processing with MOSFLM" (2011), Acta Cryst. D67, 271-281
[doi:10.1107/S0907444910048675](https://doi.org/10.1107/S0907444910048675); H. R. Powell,
T. G. G. Battye, L. Kontogiannis, O. Johnson and A. G. W. Leslie, "Integrating macromolecular X-ray
diffraction data with the graphical user interface iMosflm" (2017), Nat. Protoc. 12, 1310-1325
[doi:10.1038/nprot.2017.037](https://doi.org/10.1038/nprot.2017.037).
**[fast-feedback-indexer](https://github.com/paulscherrerinstitute/fast-feedback-indexer)** — the
known-cell indexer for serial stills (`-X ffbidx`) is PSI's fast-feedback-indexer library, linked at
build time (BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), which implements
the TORO algorithm: P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton,
M. Janousch, F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for
kilohertz serial crystallography" (2024), J. Appl. Cryst. 57, 931-944
[doi:10.1107/S1600576724003182](https://doi.org/10.1107/S1600576724003182).
### Cell reduction and lattice symmetry
**[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and
structure-factor machinery, and MTZ / XDS_ASCII I/O. Vendored in `gemmi_gph/`, so it also carries a
licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source
Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200).
**Křivý & Gruber's Niggli reduction, and the lattice-character table** — the reduction that puts
every candidate cell in a comparable form is I. Křivý and B. Gruber, "A unified algorithm for
determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298
[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636), used through GEMMI's
implementation; the table of lattice characters that maps a reduced cell to Bravais lattices and
centrings follows International Tables for Crystallography Vol. A, Table 9.2.5.1.
**Grosse-Kunstleve, Sauter & Adams's numerically stable cell reduction** - the magnitude-scaled
tolerance that decides the sign of a structurally-zero scalar product, and with it the Niggli type a
reduced cell is presented in. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically
stable algorithms for the computation of reduced unit cells" (2004), Acta Cryst. A60, 1-6
[doi:10.1107/S010876730302186X](https://doi.org/10.1107/S010876730302186X).
**Le Page's metric-symmetry search** - the obliquity of each of the 81 candidate two-folds of a
reduced cell, which is what tells a run that its lattice metric hosts more rotational symmetry than
the group its intensities supported, and the derivation of the conventional axes from that
rotation group, which is what the run then offers to the space-group search as a second lattice
candidate. The two-fold search is used through GEMMI's implementation of it. Y. Le Page, "The
derivation of the axes of the conventional unit cell from the dimensions of the Buerger-reduced
cell" (1982), J. Appl. Cryst. 15, 255-259
[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959).
### Integration and rotation geometry
**[XDS](https://xds.mr.mpg.de/)** — rotation geometry and notation, the reciprocal Lorentz and
partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for rejecting
a reflection whose predicted profile is not cleanly its own, the intensity-based test for a
centred lattice, and the scaling correction surfaces indexed by image number and detector region. W. Kabsch, "XDS" (2010), Acta Cryst. D66, 125-132
[doi:10.1107/S0907444909047337](https://doi.org/10.1107/S0907444909047337); W. Kabsch, "Integration,
scaling, space-group assignment and post-refinement" (2010), Acta Cryst. D66, 133-144
[doi:10.1107/S0907444909047374](https://doi.org/10.1107/S0907444909047374).
**Profile fitting** with reweighted, de-biased variances is the Kabsch/Otwinowski iteration, from the
second XDS paper above and from Z. Otwinowski and W. Minor, "Processing of X-ray diffraction data
collected in oscillation mode" (1997), Methods Enzymol. 276, 307-326
[doi:10.1016/S0076-6879(97)76066-X](https://doi.org/10.1016/S0076-6879%2897%2976066-X).
**The two-dimensional integration architecture** — integrating each image in the detector plane
and only afterwards assembling a reflection's partials into a full across images, as against
three-dimensional profile fitting through the image stack — is the architecture of DENZO/SCALEPACK
and MOSFLM, and it is the one rugnux's rotation pipeline follows (per-image profile-fitted
integration, then partials combined into fulls; §9 and §10.6 of the analysis reference). The
Otwinowski & Minor citation above and the MOSFLM citations below carry the credit for the paradigm
as well as for the specifics taken from each.
### Space group, twinning and pseudo-symmetry
**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search. Stage A scores each
candidate rotation operator by the correlation of I(h) with I(Rh) on **resolution-normalised**
intensities (E²), as POINTLESS does — both arms of a symmetry pair sit at the same |s|, so on raw
intensities the resolution fall-off is variance shared between them and lifts a false operator's
correlation as much as a true one's; the screw-axis test scores a
predicted-absent class against the rest of its own axial row rather than against a global mean or a
fixed cut, and lets confidence fall away with the number of axial reflections instead of refusing
below a count; the glide-plane test is that same test applied to a zone, scoring the extinguished
class against the rest of its own plane, as POINTLESS scores zonal absences. P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82
[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693); P. R. Evans, "An
introduction to data reduction: space-group determination, scaling and intensity statistics" (2011),
Acta Cryst. D67, 282-292 [doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X);
P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta
Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061);
J. Agirre, M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular
crystallography" (2023), Acta Cryst. D79, 449-461
[doi:10.1107/S2059798323003595](https://doi.org/10.1107/S2059798323003595).
**The twinning L test** is Padilla and Yeates's: pairing each acentric reflection with a
symmetry-independent neighbour and reading the first and second moments of
L = (I1I2)/(I1+I2) against their untwinned and perfect-twin values. J. E. Padilla and
T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and
pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, 1124-1130
[doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947). Their title claims
robustness to pseudo-centering, and this program's partner steps deliver it: a step of 2 along an
axis preserves the class of a half-integer pseudo-translation, which is what a pseudo-centering is.
That robustness does not extend to a pseudo-translation which is not half-integer, and the partner
steps are restricted when one is detected - see the translational-pseudo-symmetry note below.
**Translational pseudo-symmetry** is detected from the native Patterson computed from the merged
intensities, and the interpretation of an off-origin peak as a pseudo-translation between copies of
the contents of the asymmetric unit - together with the modulation it puts on the intensities, which
is the second half of the test here - is Read, Adams and McCoy's. Their fitted peak-height table is
not used: the peak is scored against a per-dataset within-shell permutation null instead, because the
noise floor of the statistic depends strongly on how many reflections a dataset has. The same
modulation is what the axial systematic-absence test scores against, so that a reflection class a
pseudo-translation merely suppresses is not read as extinct and does not buy a screw axis; the
estimate of its depth there is our own, measured per axial row from the merged intensities rather
than from the Patterson vector. R. J. Read,
P. D. Adams and A. J. McCoy, "Intensity statistics in the presence of translational
noncrystallographic symmetry" (2013), Acta Cryst. D69, 176-183
[doi:10.1107/S0907444912045374](https://doi.org/10.1107/S0907444912045374).
### Scaling, merging and data quality
**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, per-observation
outlier rejection at merge, the scaling error model, and the treatment of a reflection whose
background is contaminated. Its published behaviour, and in places its source, settled several
choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS: implementation and
evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97
[doi:10.1107/S2059798317017235](https://doi.org/10.1107/S2059798317017235); D. G. Waterman,
G. Winter, R. J. Gildea et al., "Diffraction-geometry refinement in the DIALS framework" (2016),
Acta Cryst. D72, 558-575 [doi:10.1107/S2059798316002187](https://doi.org/10.1107/S2059798316002187);
J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software
package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399
[doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198); J. M. Parkhurst,
G. Winter, D. G. Waterman et al., "Robust background modelling in DIALS" (2016), J. Appl. Cryst. 49,
1912-1921 [doi:10.1107/S1600576716013595](https://doi.org/10.1107/S1600576716013595).
**Diffraction anisotropy** — the description of the overall fall-off by a single anisotropic
displacement tensor, its symmetry constraints, and the fact that only its deviatoric part is
determined (the isotropic part being degenerate with the overall scale) are Sheriff and Hendrickson's.
The estimator fits that tensor to the observed intensity distribution, taking sigma(I) into account,
in the sense of Popov and Bourenkov. The directional diffraction limits - <I/sigma(I)> in a cone about
each principal direction, and the reporting of the anisotropic deltaB as the range of the principal
components - follow AIMLESS. rugnux reports these; it corrects no intensity and removes no reflection
on a directional criterion. S. Sheriff and W. A. Hendrickson, "Description of overall anisotropy in
diffraction from macromolecular crystals" (1987), Acta Cryst. A43, 118-121
[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X); A. N. Popov and
G. P. Bourenkov, "Choice of data-collection parameters based on statistic modelling" (2003), Acta
Cryst. D59, 1145-1153 [doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163);
P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta
Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061).
**Data-quality statistics** follow the established conventions rather than any one program: R_meas
and R_pim, CC1/2 and CC\*, the per-shell CC(model, data) between F^2_calc and F^2_obs, and the
reporting of I/sigma(I). K. Diederichs and P. A. Karplus, "Improved
R-factors for diffraction data analysis in macromolecular crystallography" (1997), Nat. Struct. Biol.
4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269); P. A. Karplus and
K. Diederichs, "Linking crystallographic model and data quality" (2012), Science 336, 1030-1033
[doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231); K. Diederichs and
P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222
[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121).
**Uncertainty conventions** follow the IUCr Commission on Crystallographic Nomenclature:
D. Schwarzenbach, S. C. Abrahams, H. D. Flack et al., "Statistical descriptors in crystallography:
Report of the IUCr Subcommittee on Statistical Descriptors" (1989), Acta Cryst. A45, 63-75
[doi:10.1107/S0108767388009596](https://doi.org/10.1107/S0108767388009596); D. Schwarzenbach,
S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descriptors in
crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995),
Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340).
### Physical corrections and calibration
**Sensor absorption at oblique incidence, and the flight path** — the angle-dependent quantum
efficiency of a flat sensor, the radial parallax variance that comes from the same integral, and the
attenuation of a reflection in the air between the sample and its pixel are all the Beer-Lambert law
taken along a ray that crosses t/cos(alpha) of sensor, or D/cos(alpha) of air, and converts at a
random depth. The attenuation coefficients, for silicon, CdTe, dry air and helium alike, are the
NIST tabulation: J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray
Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for
Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004),
NIST Standard Reference Database 126
[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F).
**Polarization correction** — the azimuthal polarization factor applied to both the azimuthally
integrated profile and the integrated Bragg intensities is the one derived for a partially polarized
synchrotron source by R. Kahn, R. Fourme, A. Gadet, J. Janin, C. Dumas and D. Andre, "Macromolecular
crystallography with synchrotron radiation: photographic data collection and polarization
correction" (1982), J. Appl. Cryst. 15, 330-337
[doi:10.1107/S0021889882012060](https://doi.org/10.1107/S0021889882012060).
**Hexagonal-ice ring positions** — the eleven ring $d$ spacings from 3.895 to 1.522 Å that the
ice-ring score, the ice-ring flagging and the ice calibrant are all built on are taken from the
measurements of, not enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in
biomolecular cryocrystallography" (2021), Acta Cryst. D77, 540-554
[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170).
That list ends at 1.522 Å by its own scope, so the eight bands below it are calculated here rather
than taken from anyone: ice Ih structure factors on the oxygen sublattice, kept where they reach 3% of
the strongest line, which reproduces the eleven measured positions exactly. The lattice constants are
Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice
constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50,
644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933).
### Model-based analysis and maps
**Bulk-solvent correction and overall scaling** — the model's structure factors are put on the
observed scale with an overall factor, an anisotropic B and a flat bulk-solvent term, the flat-mask
model of A. Fokine and A. Urzhumtsev, "Flat bulk-solvent model: obtaining optimal parameters"
(2002), Acta Cryst. D58, 1387-1392
[doi:10.1107/S0907444902010284](https://doi.org/10.1107/S0907444902010284), which is also the source
of the starting values and of the range those two parameters are physically meaningful over. The
procedure that fits them — a grid search over that range for the solvent pair, with the overall
scale and the anisotropic B refitted at every grid point — follows P. V. Afonine,
R. W. Grosse-Kunstleve and P. D. Adams, "A robust bulk-solvent correction and anisotropic scaling
procedure" (2005), Acta Cryst. D61, 850-855
[doi:10.1107/S0907444905007894](https://doi.org/10.1107/S0907444905007894). The fit is unweighted,
as in both that procedure and REFMAC5: G. N. Murshudov, P. Skubak, A. A. Lebedev, N. S. Pannu,
R. A. Steiner, R. A. Nicholls, M. D. Winn, F. Long and A. A. Vagin, "REFMAC5 for the refinement of
macromolecular crystal structures" (2011), Acta Cryst. D67, 355-367
[doi:10.1107/S0907444911001314](https://doi.org/10.1107/S0907444911001314).
**sigma_A map coefficients** — the maps written by `--model` are weighted by a maximum-likelihood
sigma_A estimated per resolution shell, giving 2mFo-DFc and mFo-DFc rather than 2Fo-Fc and Fo-Fc.
What is taken is the formalism itself: the Rice and Woolfson likelihoods of |Fo| given |Fc| and
sigma_A, the figure of merit m and the scale D that follow from it, and the result that the
bias-corrected coefficient is 2mFo-DFc for an acentric reflection and mFo for a centric one.
R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures with
errors" (1986), Acta Cryst. A42, 140-149
[doi:10.1107/S0108767386099622](https://doi.org/10.1107/S0108767386099622).
**[ANODE](https://doi.org/10.1107/S0021889811041768)** — reading the anomalous difference map at the
atoms of a supplied model and reporting the strongest sites by name, instead of searching the map for
blobs. The map itself is the textbook anomalous difference Fourier; what is taken from ANODE is that
reading: A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density calculation" (2011),
J. Appl. Cryst. 44, 1285-1287
[doi:10.1107/S0021889811041768](https://doi.org/10.1107/S0021889811041768).
**Uniform random rotations** — the null a supplied model is scored against re-orients that model at
random about its own centroid, and the rotations are drawn uniformly from SO(3) through a uniform
random unit quaternion. K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk,
Academic Press (1992), 124-132 (no DOI).
## Software and computing methods
Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading
the result, follows Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4
data", HDF5 User Group meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders).
The CUDA kernels in Jungfraujoch are its own, but the approach is his.
This software uses Viridis, Magma and Inferno colormaps from Matplotlib under its BSD-compatible license
## File formats read from a published specification
**CBF / imgCIF** - the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression scheme
and reads the imgCIF `_axis` table (the laboratory directions of the image's fast and slow pixel
directions, of the goniometer axes and of a 2theta arm) from the specification alone; no CBFlib or
other CBF code is used, so there is no licence obligation, only this credit.
H. J. Bernstein and A. P. Hammersley, "Specification of the Crystallographic Binary File
(CBF/imgCIF)" (2006), International Tables for Crystallography Vol. G, 37-43
[doi:10.1107/97809553602060000729](https://doi.org/10.1107/97809553602060000729);
A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, "Image dictionary (imgCIF)" (2006),
International Tables for Crystallography Vol. G, 444-458
[doi:10.1107/97809553602060000746](https://doi.org/10.1107/97809553602060000746).
## Public diffraction data used for testing
In addition to in-house datasets collected at SLS 2.0, Jungfraujoch is tested against public
diffraction data collected on other people's beamlines, on detectors and in file formats we do not
produce ourselves - most of it at other facilities, a few sets at the Swiss Light Source but not by
this system. That data was collected and published by other people. Every dataset used, the DOI to
cite for it, and the deposition it belongs to are listed in
[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md); we thank the depositors, and the repositories that make
the data findable and citable.
**[IRRMC](https://proteindiffraction.org/)**, the Integrated Resource for Reproducibility in
Macromolecular Crystallography (Minor lab, University of Virginia), is the source of most of them.
IRRMC releases its data under CC0 and asks that the DOI of the dataset be cited; those DOIs are in
the table. M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng,
D. R. Cooper, M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of
macromolecular diffraction experiments" (2016), Acta Cryst. D72, 1181-1193
[doi:10.1107/S2059798316014716](https://doi.org/10.1107/S2059798316014716); M. Grabowski,
M. Cymborowski, P. J. Porebski, T. Osinski, I. G. Shabalin, D. R. Cooper and W. Minor, "The
Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first
four years" (2019), Struct. Dyn. 6, 064301
[doi:10.1063/1.5128672](https://doi.org/10.1063/1.5128672).
**[SBGrid Data Bank](https://data.sbgrid.org/)** supplied eighteen of the datasets. P. A. Meyer,
S. Socias, J. Key, E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the
structural biology data grid supports live analysis" (2016), Nat. Commun. 7, 10882
[doi:10.1038/ncomms10882](https://doi.org/10.1038/ncomms10882).
**[Zenodo](https://zenodo.org/)** hosts sixteen, deposited there directly by the groups that
collected them. European Organization for Nuclear Research and OpenAIRE, "Zenodo" (2013), CERN
[doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). Three of those datasets were
published as IUCrData Raw Data Letters; the letters are cited on the
[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page, beside the datasets they describe.
**[MXRDR](https://mxrdr.icm.edu.pl/)**, the Macromolecular Xtallography Raw Data Repository
(ICM, University of Warsaw), supplied four, released under CC0 with the dataset DOI to be cited;
those DOIs (`10.18150/…`, `10.60884/…`) are in the table.
**The [ESRF data portal](https://data.esrf.fr/)** supplied one, released under CC BY 4.0 under the
ESRF data policy with the dataset DOI (`10.15151/…`) to be cited.
The beamline, resolution, space group and unit cell quoted for each dataset are the values
deposited with the corresponding PDB entry, read from the RCSB PDB data API. H. M. Berman,
J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and P. E. Bourne,
"The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242
[doi:10.1093/nar/28.1.235](https://doi.org/10.1093/nar/28.1.235).
## Generative AI usage declaration
Large language models were used extensively in developing this code. Jungfraujoch development was
supported with JetBrains AI (mostly GPT models) to refactor and verify particular code fragments.
Rugnux was developed with the assistance of Claude Code (mostly the Opus model). This documentation
was written with the assistance of Claude Opus and Fable models.