leonarski_fandClaude Opus 5
80e10f51b3
rugnux now says whether a dataset's fall-off is direction-dependent, and by how much. It corrects nothing and truncates nothing: no intensity is changed, no reflection is dropped on a directional criterion, and the written files do not depend on direction at all. Two quantities, because they are not the same thing. The anisotropic deltaB is the range of the principal components of the anisotropy tensor - a rate of fall-off. The diffraction limit along each principal direction is where <I/sigma(I)> in a 20 degree cone falls through 2 - where signal actually runs out. One battery case has only 0.28 A between its directional limits and a 58x ratio in cone <I/sigma>, so reporting either alone would miss it. The tensor is a Laue-constrained deviatoric ADP tensor fitted on INTENSITIES with no positivity cut, by weighted Gauss-Newton over 12 shells x 60 directions with a free constant per shell. Fitting amplitudes after a positivity cut, which is what xtriage and ctruncate do, destroys about 40% of the measured anisotropy - the cut keeps only the positive noise excursions in whichever direction has died, and that is the direction carrying the signal. Against the same 38 merged files rugnux reads 1.24x xtriage's eigenvalue spread and 1.61x ctruncate's; on strong near-isotropic data all three agree to a few percent, and they diverge exactly where a direction has died. The verdict is gated three ways - not detected, detected, or cannot determine - against the dataset's own systematic floor, measured in the tensor directions its Laue symmetry forbids. The floor cannot be measured on merged reflections, which have exact Laue symmetry by construction, so the floor is taken from the unmerged observations and the verdict is "cannot determine" without them. Triclinic has no forbidden subspace and always returns cannot determine. A cubic crystal returns exactly zero, because that is its symmetry and not a measurement. A second axis reports the resolution signature: a genuine Debye-Waller fall-off is linear through the origin in s^2, and a deficit that is flat is something else. Magnitude alone had promoted a crystal that is 68% not a Debye-Waller B into the top five of this battery; it now reads not detected with the caution attached. Following Sheriff & Hendrickson (1987) Acta Cryst. A43, 118-121 for the tensor and Popov & Bourenkov (2003) Acta Cryst. D59, 1145-1153 for the estimator. The directional limits are written as jfjoch_ local mmCIF items rather than _reflns.pdbx_aniso_diffraction_limit_*, whose dictionary definition is explicitly the ellipsoid fitted to a diffraction cut-off surface - a construction rugnux does not perform. The generic anisotropic B tensor items are written. Changes no existing number; only REPORT_VERSION moves, 1 to 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CHMmeM1d489zvNFT7ZMN2P
135 lines
11 KiB
Markdown
135 lines
11 KiB
Markdown
# Acknowledgements
|
|
|
|
Citation: F. Leonarski, M. Bruckner, C. Lopez-Cuenca, A. Mozzanica, H.-C. Stadler, Z. Matej, A. Castellane, B. Mesnet, J. Wojdyla, B. Schmitt and M. Wang "Jungfraujoch: hardware-accelerated data-acquisition system for kilohertz pixel-array X-ray detectors" (2023), J. Synchrotron Rad., 30, 227-234 [doi:10.1107/S1600577522010268](https://doi.org/10.1107/S1600577522010268).
|
|
|
|
The project is supported by :
|
|
* Innosuisse via Innovation Project "NextGenDCU high data rate acquisition system for X-ray detectors in structural biology applications" (101.535.1 IP-ENG; Apr 2023 - Sep 2025).
|
|
* ETH Domain via Open Research Data Contribute project (Jan - Dec 2023)
|
|
* AMD University Program with donation of licenses of Ethernet IP cores and Vivado software
|
|
|
|
Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading
|
|
the result, follows Jon Wright (ESRF): "Experiences with GPU decompression for bitshuffle + LZ4
|
|
data", HDF5 User Group meeting (2021), and [bslz4decoders](https://github.com/jonwright/bslz4decoders).
|
|
The CUDA kernels in Jungfraujoch are its own, but the approach is his.
|
|
|
|
Spot extraction groups strong pixels into spots with the sparse connected-component labelling of the
|
|
ACTS traccc project: P. Gessinger, H. M. Gray, A. Krasznahorkay, C. Leggett, J. Niermann,
|
|
A. Salzburger, S. N. Swatman and B. Yeo, "traccc: GPU track reconstruction library for HEP
|
|
experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822);
|
|
[traccc](https://github.com/acts-project/traccc). The CPU spot extractor adapts its SparseCCL source,
|
|
and the CUDA spot extractor follows the design of its GPU counterpart - a backward-neighbour graph
|
|
over a sorted hit list, resolved by a parallel union-find. traccc is MPL-2.0; see
|
|
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
|
|
|
|
This software uses Viridis, Magma and Inferno colormaps from Matplotlib under its BSD-compatible license
|
|
|
|
## Crystallographic methods adopted from other packages
|
|
|
|
The analysis pipeline reimplements methods first published, and in most cases first implemented, by
|
|
other crystallographic software. The code below is Jungfraujoch's own; the methods are theirs, and
|
|
are acknowledged here. Where a package's source was consulted this is said explicitly. None of these
|
|
packages is linked or vendored, with the single exception of GEMMI (see
|
|
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)).
|
|
|
|
**[XDS](https://xds.mr.mpg.de/)** — rotation geometry and notation, the reciprocal Lorentz and
|
|
partiality treatment, the maximum-likelihood mosaicity estimate, the `MINPK` criterion for rejecting
|
|
a reflection whose predicted profile is not cleanly its own, the intensity-based test for a
|
|
centred lattice, and the scaling correction surfaces indexed by image number and detector region. W. Kabsch, "XDS" (2010), Acta Cryst. D66, 125-132
|
|
[doi:10.1107/S0907444909047337](https://doi.org/10.1107/S0907444909047337); W. Kabsch, "Integration,
|
|
scaling, space-group assignment and post-refinement" (2010), Acta Cryst. D66, 133-144
|
|
[doi:10.1107/S0907444909047374](https://doi.org/10.1107/S0907444909047374).
|
|
|
|
**Profile fitting** with reweighted, de-biased variances is the Kabsch/Otwinowski iteration, from the
|
|
second XDS paper above and from Z. Otwinowski and W. Minor, "Processing of X-ray diffraction data
|
|
collected in oscillation mode" (1997), Methods Enzymol. 276, 307-326
|
|
[doi:10.1016/S0076-6879(97)76066-X](https://doi.org/10.1016/S0076-6879%2897%2976066-X).
|
|
|
|
**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, per-observation
|
|
outlier rejection at merge, the scaling error model, and the treatment of a reflection whose
|
|
background is contaminated. Its published behaviour, and in places its source, settled several
|
|
choices here. G. Winter, D. G. Waterman, J. M. Parkhurst et al., "DIALS: implementation and
|
|
evaluation of a new integration package" (2018), Acta Cryst. D74, 85-97
|
|
[doi:10.1107/S2059798317017235](https://doi.org/10.1107/S2059798317017235); D. G. Waterman,
|
|
G. Winter, R. J. Gildea et al., "Diffraction-geometry refinement in the DIALS framework" (2016),
|
|
Acta Cryst. D72, 558-575 [doi:10.1107/S2059798316002187](https://doi.org/10.1107/S2059798316002187);
|
|
J. Beilsten-Edmands, G. Winter, R. Gildea et al., "Scaling diffraction data in the DIALS software
|
|
package: algorithms and new approaches for multi-crystal scaling" (2020), Acta Cryst. D76, 385-399
|
|
[doi:10.1107/S2059798320003198](https://doi.org/10.1107/S2059798320003198); J. M. Parkhurst,
|
|
G. Winter, D. G. Waterman et al., "Robust background modelling in DIALS" (2016), J. Appl. Cryst. 49,
|
|
1912-1921 [doi:10.1107/S1600576716013595](https://doi.org/10.1107/S1600576716013595).
|
|
|
|
**[POINTLESS](https://www.ccp4.ac.uk/)** (CCP4) — the space-group search. Stage A scores each
|
|
candidate rotation operator by the correlation of I(h) with I(Rh); the screw-axis test scores a
|
|
predicted-absent class against the rest of its own axial row rather than against a global mean or a
|
|
fixed cut, and lets confidence fall away with the number of axial reflections instead of refusing
|
|
below a count. P. Evans, "Scaling and assessment of data quality" (2006), Acta Cryst. D62, 72-82
|
|
[doi:10.1107/S0907444905036693](https://doi.org/10.1107/S0907444905036693); P. R. Evans, "An
|
|
introduction to data reduction: space-group determination, scaling and intensity statistics" (2011),
|
|
Acta Cryst. D67, 282-292 [doi:10.1107/S090744491003982X](https://doi.org/10.1107/S090744491003982X);
|
|
P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta
|
|
Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061);
|
|
J. Agirre, M. Atanasova, H. Bagdonas et al., "The CCP4 suite: integrative software for macromolecular
|
|
crystallography" (2023), Acta Cryst. D79, 449-461
|
|
[doi:10.1107/S2059798323003595](https://doi.org/10.1107/S2059798323003595).
|
|
|
|
**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/)** — the Rossmann FFT autoindexing algorithm and
|
|
post-refinement practice, including which parameters are safe to refine per image and which must be
|
|
refined over a wedge. A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM"
|
|
(2007), in *Evolving Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245,
|
|
41-51 [doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4);
|
|
T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new
|
|
graphical interface for diffraction-image processing with MOSFLM" (2011), Acta Cryst. D67, 271-281
|
|
[doi:10.1107/S0907444910048675](https://doi.org/10.1107/S0907444910048675); H. R. Powell,
|
|
T. G. G. Battye, L. Kontogiannis, O. Johnson and A. G. W. Leslie, "Integrating macromolecular X-ray
|
|
diffraction data with the graphical user interface iMosflm" (2017), Nat. Protoc. 12, 1310-1325
|
|
[doi:10.1038/nprot.2017.037](https://doi.org/10.1038/nprot.2017.037).
|
|
|
|
**[CrystFEL](https://www.desy.de/~twhite/crystfel/)** — spot finding, the three-ring integration
|
|
region, the serial/stills processing model, and the per-frame indexing acceptance test
|
|
(`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass,
|
|
A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012),
|
|
J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312).
|
|
|
|
**[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and
|
|
structure-factor machinery, and MTZ / XDS_ASCII I/O. Vendored in `gemmi_gph/`, so it also carries a
|
|
licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source
|
|
Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200).
|
|
|
|
**Hexagonal-ice ring positions** — the eleven measured ring $d$ spacings the ice-ring score, the
|
|
ice-ring flagging and the ice calibrant are all built on are taken from the measurements of, not
|
|
enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in biomolecular
|
|
cryocrystallography" (2021), Acta Cryst. D77, 540-554
|
|
[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170).
|
|
|
|
**Diffraction anisotropy** — the description of the overall fall-off by a single anisotropic
|
|
displacement tensor, its symmetry constraints, and the fact that only its deviatoric part is
|
|
determined (the isotropic part being degenerate with the overall scale) are Sheriff and Hendrickson's.
|
|
The estimator fits that tensor to the observed intensity distribution, taking sigma(I) into account,
|
|
in the sense of Popov and Bourenkov. The directional diffraction limits - <I/sigma(I)> in a cone about
|
|
each principal direction, and the reporting of the anisotropic deltaB as the range of the principal
|
|
components - follow AIMLESS. rugnux reports these; it corrects no intensity and removes no reflection
|
|
on a directional criterion. S. Sheriff and W. A. Hendrickson, "Description of overall anisotropy in
|
|
diffraction from macromolecular crystals" (1987), Acta Cryst. A43, 118-121
|
|
[doi:10.1107/S010876738709977X](https://doi.org/10.1107/S010876738709977X); A. N. Popov and
|
|
G. P. Bourenkov, "Choice of data-collection parameters based on statistic modelling" (2003), Acta
|
|
Cryst. D59, 1145-1153 [doi:10.1107/S0907444903008163](https://doi.org/10.1107/S0907444903008163);
|
|
P. R. Evans and G. N. Murshudov, "How good are my data and what is the resolution?" (2013), Acta
|
|
Cryst. D69, 1204-1214 [doi:10.1107/S0907444913000061](https://doi.org/10.1107/S0907444913000061).
|
|
|
|
**Data-quality statistics** follow the established conventions rather than any one program: R_meas
|
|
and R_pim, CC1/2 and CC\*, and the reporting of I/sigma(I). K. Diederichs and P. A. Karplus, "Improved
|
|
R-factors for diffraction data analysis in macromolecular crystallography" (1997), Nat. Struct. Biol.
|
|
4, 269-275 [doi:10.1038/nsb0497-269](https://doi.org/10.1038/nsb0497-269); P. A. Karplus and
|
|
K. Diederichs, "Linking crystallographic model and data quality" (2012), Science 336, 1030-1033
|
|
[doi:10.1126/science.1218231](https://doi.org/10.1126/science.1218231); K. Diederichs and
|
|
P. A. Karplus, "Better models by discarding data?" (2013), Acta Cryst. D69, 1215-1222
|
|
[doi:10.1107/S0907444913001121](https://doi.org/10.1107/S0907444913001121).
|
|
|
|
**Uncertainty conventions** follow the IUCr Commission on Crystallographic Nomenclature:
|
|
D. Schwarzenbach, S. C. Abrahams, H. D. Flack et al., "Statistical descriptors in crystallography:
|
|
Report of the IUCr Subcommittee on Statistical Descriptors" (1989), Acta Cryst. A45, 63-75
|
|
[doi:10.1107/S0108767388009596](https://doi.org/10.1107/S0108767388009596); D. Schwarzenbach,
|
|
S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descriptors in
|
|
crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995),
|
|
Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340).
|