v1.0.0-rc.166 (#76)
Build Packages / Unit tests (push) Successful in 1h22m15s
Build Packages / build:windows:nocuda (push) Successful in 18m0s
Build Packages / build:windows:cuda (push) Successful in 20m30s
Build Packages / build:viewer-tgz:cpu (push) Successful in 10m32s
Build Packages / build:viewer-tgz:cuda (push) Successful in 11m39s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 8m55s
Build Packages / build:rugnux:windows (push) Successful in 11m25s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m6s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 16m27s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 20m19s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 15m34s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 20m25s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 19m36s
Build Packages / build:rpm (rocky8) (push) Successful in 17m43s
Build Packages / build:rpm (rocky9) (push) Successful in 13m34s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 21m28s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 18m19s
Build Packages / DIALS test (push) Successful in 12m36s
Build Packages / XDS test (durin plugin) (push) Successful in 6m56s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m48s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m7s
Build Packages / Generate python client (push) Successful in 11s
Build Packages / Build documentation (push) Successful in 36s
Build Packages / Create release (push) Skipped
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 5m11s
Build Packages / Unit tests (push) Successful in 1h22m15s
Build Packages / build:windows:nocuda (push) Successful in 18m0s
Build Packages / build:windows:cuda (push) Successful in 20m30s
Build Packages / build:viewer-tgz:cpu (push) Successful in 10m32s
Build Packages / build:viewer-tgz:cuda (push) Successful in 11m39s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 8m55s
Build Packages / build:rugnux:windows (push) Successful in 11m25s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m6s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 16m27s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 20m19s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 15m34s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 20m25s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 19m36s
Build Packages / build:rpm (rocky8) (push) Successful in 17m43s
Build Packages / build:rpm (rocky9) (push) Successful in 13m34s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 21m28s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 18m19s
Build Packages / DIALS test (push) Successful in 12m36s
Build Packages / XDS test (durin plugin) (push) Successful in 6m56s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m48s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m7s
Build Packages / Generate python client (push) Successful in 11s
Build Packages / Build documentation (push) Successful in 36s
Build Packages / Create release (push) Skipped
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 5m11s
* `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands. * `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion. * Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants. * `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing. * A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed. * `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing. * Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences. * The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to. * The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after. * The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution. * `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have. * Twinning is no longer reported when the L-test contradicts it. * The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's. * `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots. * The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area. Reviewed-on: #76 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
This commit was merged in pull request #76.
This commit is contained in:
1 parent
511be0c366
commit
680c36c20d
383 files changed
+20910
-3936
No files matched your search
+136
-6
@@ -19,10 +19,65 @@ experiments" (2025), [arXiv:2505.22822](https://arxiv.org/abs/2505.22822);
|
||||
[traccc](https://github.com/acts-project/traccc). The CPU spot extractor adapts its SparseCCL source,
|
||||
and the CUDA spot extractor follows the design of its GPU counterpart - a backward-neighbour graph
|
||||
over a sorted hit list, resolved by a parallel union-find. traccc is MPL-2.0; see
|
||||
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
|
||||
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). The SparseCCL algorithm itself is A. Hennequin,
|
||||
B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: Connected Components Labeling and
|
||||
Analysis for sparse images" (2019), DASIP 2019, 65-70
|
||||
[doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184).
|
||||
|
||||
This software uses Viridis, Magma and Inferno colormaps from Matplotlib under its BSD-compatible license
|
||||
|
||||
## Public diffraction data used for testing
|
||||
|
||||
In addition to in-house datasets collected at SLS 2.0, Jungfraujoch is tested against public
|
||||
diffraction data collected on other people's beamlines, on detectors and in file formats we do not
|
||||
produce ourselves - most of it at other facilities, a few sets at the Swiss Light Source but not by
|
||||
this system. That data was collected and published by other people. Every dataset used, the DOI to
|
||||
cite for it, and the deposition it belongs to are listed in
|
||||
[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md); we thank the depositors, and the repositories that make
|
||||
the data findable and citable.
|
||||
|
||||
**[IRRMC](https://proteindiffraction.org/)**, the Integrated Resource for Reproducibility in
|
||||
Macromolecular Crystallography (Minor lab, University of Virginia), is the source of most of them.
|
||||
IRRMC releases its data under CC0 and asks that the DOI of the dataset be cited; those DOIs are in
|
||||
the table. M. Grabowski, K. M. Langner, M. Cymborowski, P. J. Porebski, P. Sroka, H. Zheng,
|
||||
D. R. Cooper, M. D. Zimmerman, M.-A. Elsliger, S. K. Burley and W. Minor, "A public database of
|
||||
macromolecular diffraction experiments" (2016), Acta Cryst. D72, 1181-1193
|
||||
[doi:10.1107/S2059798316014716](https://doi.org/10.1107/S2059798316014716); M. Grabowski,
|
||||
M. Cymborowski, P. J. Porebski, T. Osinski, I. G. Shabalin, D. R. Cooper and W. Minor, "The
|
||||
Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first
|
||||
four years" (2019), Struct. Dyn. 6, 064301
|
||||
[doi:10.1063/1.5128672](https://doi.org/10.1063/1.5128672).
|
||||
|
||||
**[SBGrid Data Bank](https://data.sbgrid.org/)** supplied nine of the datasets. P. A. Meyer,
|
||||
S. Socias, J. Key, E. Ransey, E. C. Tjon, A. Buschiazzo et al., "Data publication with the
|
||||
structural biology data grid supports live analysis" (2016), Nat. Commun. 7, 10882
|
||||
[doi:10.1038/ncomms10882](https://doi.org/10.1038/ncomms10882).
|
||||
|
||||
**[Zenodo](https://zenodo.org/)** hosts eleven, deposited there directly by the groups that
|
||||
collected them. European Organization for Nuclear Research and OpenAIRE, "Zenodo" (2013), CERN
|
||||
[doi:10.25495/7GXK-RD71](https://doi.org/10.25495/7GXK-RD71). Three of those datasets were
|
||||
published as IUCrData Raw Data Letters; the letters are cited on the
|
||||
[EXTERNAL_TEST_DATA](EXTERNAL_TEST_DATA.md) page, beside the datasets they describe.
|
||||
|
||||
The beamline, resolution, space group and unit cell quoted for each dataset are the values
|
||||
deposited with the corresponding PDB entry, read from the RCSB PDB data API. H. M. Berman,
|
||||
J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov and P. E. Bourne,
|
||||
"The Protein Data Bank" (2000), Nucleic Acids Res. 28, 235-242
|
||||
[doi:10.1093/nar/28.1.235](https://doi.org/10.1093/nar/28.1.235).
|
||||
|
||||
## File formats read from a published specification
|
||||
|
||||
**CBF / imgCIF** - the native miniCBF reader implements the `x-CBF_BYTE_OFFSET` compression scheme
|
||||
and reads the imgCIF `_axis` table (the laboratory directions of the image's fast and slow pixel
|
||||
directions, of the goniometer axes and of a 2theta arm) from the specification alone; no CBFlib or
|
||||
other CBF code is used, so there is no licence obligation, only this credit.
|
||||
H. J. Bernstein and A. P. Hammersley, "Specification of the Crystallographic Binary File
|
||||
(CBF/imgCIF)" (2006), International Tables for Crystallography Vol. G, 37-43
|
||||
[doi:10.1107/97809553602060000729](https://doi.org/10.1107/97809553602060000729);
|
||||
A. P. Hammersley, H. J. Bernstein and J. D. Westbrook, "Image dictionary (imgCIF)" (2006),
|
||||
International Tables for Crystallography Vol. G, 444-458
|
||||
[doi:10.1107/97809553602060000746](https://doi.org/10.1107/97809553602060000746).
|
||||
|
||||
## Crystallographic methods adopted from other packages
|
||||
|
||||
The analysis pipeline reimplements methods first published, and in most cases first implemented, by
|
||||
@@ -44,6 +99,14 @@ second XDS paper above and from Z. Otwinowski and W. Minor, "Processing of X-ray
|
||||
collected in oscillation mode" (1997), Methods Enzymol. 276, 307-326
|
||||
[doi:10.1016/S0076-6879(97)76066-X](https://doi.org/10.1016/S0076-6879%2897%2976066-X).
|
||||
|
||||
**The two-dimensional integration architecture** — integrating each image in the detector plane
|
||||
and only afterwards assembling a reflection's partials into a full across images, as against
|
||||
three-dimensional profile fitting through the image stack — is the architecture of DENZO/SCALEPACK
|
||||
and MOSFLM, and it is the one rugnux's rotation pipeline follows (per-image profile-fitted
|
||||
integration, then partials combined into fulls; §9 and §10.6 of the analysis reference). The
|
||||
Otwinowski & Minor citation above and the MOSFLM citations below carry the credit for the paradigm
|
||||
as well as for the specifics taken from each.
|
||||
|
||||
**[DIALS](https://dials.github.io/)** — the resolution cutoff from the CC1/2 fall-off, per-observation
|
||||
outlier rejection at merge, the scaling error model, and the treatment of a reflection whose
|
||||
background is contaminated. Its published behaviour, and in places its source, settled several
|
||||
@@ -77,7 +140,12 @@ crystallography" (2023), Acta Cryst. D79, 449-461
|
||||
|
||||
**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/)** — the Rossmann FFT autoindexing algorithm and
|
||||
post-refinement practice, including which parameters are safe to refine per image and which must be
|
||||
refined over a wedge. A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM"
|
||||
refined over a wedge. The autoindexing algorithm itself — projecting the reciprocal-space points
|
||||
onto many directions and Fourier-transforming the 1D projection histograms — is I. Steller,
|
||||
R. Bolotovsky and M. G. Rossmann, "An algorithm for automatic indexing of oscillation images using
|
||||
Fourier analysis" (1997), J. Appl. Cryst. 30, 1036-1040
|
||||
[doi:10.1107/S0021889897008777](https://doi.org/10.1107/S0021889897008777); MOSFLM is the
|
||||
implementation whose practice is followed. A. G. W. Leslie and H. R. Powell, "Processing diffraction data with MOSFLM"
|
||||
(2007), in *Evolving Methods for Macromolecular Crystallography*, NATO Science Series II, vol. 245,
|
||||
41-51 [doi:10.1007/978-1-4020-6316-9_4](https://doi.org/10.1007/978-1-4020-6316-9_4);
|
||||
T. G. G. Battye, L. Kontogiannis, O. Johnson, H. R. Powell and A. G. W. Leslie, "iMOSFLM: a new
|
||||
@@ -92,18 +160,66 @@ region, the serial/stills processing model, and the per-frame indexing acceptanc
|
||||
(`indexing_peak_check()` in `peaks.c`). T. A. White, R. A. Kirian, A. V. Martin, A. Aquila, K. Nass,
|
||||
A. Barty and H. N. Chapman, "CrystFEL: a software suite for snapshot serial crystallography" (2012),
|
||||
J. Appl. Cryst. 45, 335-341 [doi:10.1107/S0021889812002312](https://doi.org/10.1107/S0021889812002312).
|
||||
The self-calibrating spot finder's per-resolution-ring background statistics, with the Bragg peaks
|
||||
excluded by iterated clipping, follow Cheetah's peakfinder8: A. Barty, R. A. Kirian,
|
||||
F. R. N. C. Maia, M. Hantke, C. H. Yoon, T. A. White and H. N. Chapman, "Cheetah: software for
|
||||
high-throughput reduction and analysis of serial femtosecond X-ray diffraction data" (2014),
|
||||
J. Appl. Cryst. 47, 1118-1131
|
||||
[doi:10.1107/S1600576714007626](https://doi.org/10.1107/S1600576714007626).
|
||||
|
||||
**[fast-feedback-indexer](https://github.com/paulscherrerinstitute/fast-feedback-indexer)** — the
|
||||
known-cell indexer for serial stills (`-X ffbidx`) is PSI's fast-feedback-indexer library, linked at
|
||||
build time (BSD-3-Clause; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)), which implements
|
||||
the TORO algorithm: P. Gasparotto, L. Barba, H.-C. Stadler, G. Assmann, H. Mendonça, A. W. Ashton,
|
||||
M. Janousch, F. Leonarski and B. Béjar, "TORO Indexer: a PyTorch-based indexing algorithm for
|
||||
kilohertz serial crystallography" (2024), J. Appl. Cryst. 57, 931-944
|
||||
[doi:10.1107/S1600576724003182](https://doi.org/10.1107/S1600576724003182).
|
||||
|
||||
**[GEMMI](https://github.com/project-gemmi/gemmi)** — symmetry operations, unit-cell and
|
||||
structure-factor machinery, and MTZ / XDS_ASCII I/O. Vendored in `gemmi_gph/`, so it also carries a
|
||||
licence obligation. M. Wojdyr, "GEMMI: A library for structural biology" (2022), J. Open Source
|
||||
Softw. 7, 4200 [doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200).
|
||||
|
||||
**Hexagonal-ice ring positions** — the eleven measured ring $d$ spacings the ice-ring score, the
|
||||
ice-ring flagging and the ice calibrant are all built on are taken from the measurements of, not
|
||||
enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in biomolecular
|
||||
cryocrystallography" (2021), Acta Cryst. D77, 540-554
|
||||
**Křivý & Gruber's Niggli reduction, and the lattice-character table** — the reduction that puts
|
||||
every candidate cell in a comparable form is I. Křivý and B. Gruber, "A unified algorithm for
|
||||
determining the reduced (Niggli) cell" (1976), Acta Cryst. A32, 297-298
|
||||
[doi:10.1107/S0567739476000636](https://doi.org/10.1107/S0567739476000636), used through GEMMI's
|
||||
implementation; the table of lattice characters that maps a reduced cell to Bravais lattices and
|
||||
centrings follows International Tables for Crystallography Vol. A, Table 9.2.5.1.
|
||||
|
||||
**Grosse-Kunstleve, Sauter & Adams's numerically stable cell reduction** - the magnitude-scaled
|
||||
tolerance that decides the sign of a structurally-zero scalar product, and with it the Niggli type a
|
||||
reduced cell is presented in. R. W. Grosse-Kunstleve, N. K. Sauter and P. D. Adams, "Numerically
|
||||
stable algorithms for the computation of reduced unit cells" (2004), Acta Cryst. A60, 1-6
|
||||
[doi:10.1107/S010876730302186X](https://doi.org/10.1107/S010876730302186X).
|
||||
|
||||
**Le Page's metric-symmetry search** - the obliquity of each of the 81 candidate two-folds of a
|
||||
reduced cell, which is what tells a run that its lattice metric hosts more rotational symmetry than
|
||||
the group its intensities supported. Used through GEMMI's implementation of it. Y. Le Page, "The
|
||||
derivation of the axes of the conventional unit cell from the dimensions of the Buerger-reduced
|
||||
cell" (1982), J. Appl. Cryst. 15, 255-259
|
||||
[doi:10.1107/S0021889882011959](https://doi.org/10.1107/S0021889882011959).
|
||||
|
||||
**Hexagonal-ice ring positions** — the eleven ring $d$ spacings from 3.895 to 1.522 Å that the
|
||||
ice-ring score, the ice-ring flagging and the ice calibrant are all built on are taken from the
|
||||
measurements of, not enumerated from a cell. D. W. Moreau, H. Atakisi and R. E. Thorne, "Ice in
|
||||
biomolecular cryocrystallography" (2021), Acta Cryst. D77, 540-554
|
||||
[doi:10.1107/S2059798321001170](https://doi.org/10.1107/S2059798321001170).
|
||||
|
||||
That list ends at 1.522 Å by its own scope, so the eight bands below it are calculated here rather
|
||||
than taken from anyone: ice Ih structure factors on the oxygen sublattice, kept where they reach 3% of
|
||||
the strongest line, which reproduces the eleven measured positions exactly. The lattice constants are
|
||||
Röttger and co-workers'. K. Röttger, A. Endriss, J. Ihringer, S. Doyle and W. F. Kuhs, "Lattice
|
||||
constants and thermal expansion of H2O and D2O ice Ih between 10 and 265 K" (1994), Acta Cryst. B50,
|
||||
644-648 [doi:10.1107/S0108768194004933](https://doi.org/10.1107/S0108768194004933).
|
||||
|
||||
**The twinning L test** is Padilla and Yeates's: pairing each acentric reflection with a
|
||||
symmetry-independent neighbour and reading the first and second moments of
|
||||
L = (I1−I2)/(I1+I2) against their untwinned and perfect-twin values. J. E. Padilla and
|
||||
T. O. Yeates, "A statistic for local intensity differences: robustness to anisotropy and
|
||||
pseudo-centering and utility for detecting twinning" (2003), Acta Cryst. D59, 1124-1130
|
||||
[doi:10.1107/S0907444903007947](https://doi.org/10.1107/S0907444903007947).
|
||||
|
||||
**Diffraction anisotropy** — the description of the overall fall-off by a single anisotropic
|
||||
displacement tensor, its symmetry constraints, and the fact that only its deviatoric part is
|
||||
determined (the isotropic part being degenerate with the overall scale) are Sheriff and Hendrickson's.
|
||||
@@ -126,6 +242,20 @@ reading: A. Thorn and G. M. Sheldrick, "ANODE: anomalous and heavy-atom density
|
||||
J. Appl. Cryst. 44, 1285-1287
|
||||
[doi:10.1107/S0021889811041768](https://doi.org/10.1107/S0021889811041768).
|
||||
|
||||
**sigma_A map coefficients** — the maps written by `--model` are weighted by a maximum-likelihood
|
||||
sigma_A estimated per resolution shell, giving 2mFo-DFc and mFo-DFc rather than 2Fo-Fc and Fo-Fc.
|
||||
What is taken is the formalism itself: the Rice and Woolfson likelihoods of |Fo| given |Fc| and
|
||||
sigma_A, the figure of merit m and the scale D that follow from it, and the result that the
|
||||
bias-corrected coefficient is 2mFo-DFc for an acentric reflection and mFo for a centric one.
|
||||
R. J. Read, "Improved Fourier coefficients for maps using phases from partial structures with
|
||||
errors" (1986), Acta Cryst. A42, 140-149
|
||||
[doi:10.1107/S0108767386099622](https://doi.org/10.1107/S0108767386099622).
|
||||
|
||||
**Uniform random rotations** — the null a supplied model is scored against re-orients that model at
|
||||
random about its own centroid, and the rotations are drawn uniformly from SO(3) through a uniform
|
||||
random unit quaternion. K. Shoemake, "Uniform Random Rotations", in *Graphics Gems III*, ed. D. Kirk,
|
||||
Academic Press (1992), 124-132 (no DOI).
|
||||
|
||||
**Data-quality statistics** follow the established conventions rather than any one program: R_meas
|
||||
and R_pim, CC1/2 and CC\*, and the reporting of I/sigma(I). K. Diederichs and P. A. Karplus, "Improved
|
||||
R-factors for diffraction data analysis in macromolecular crystallography" (1997), Nat. Struct. Biol.
|
||||
|
||||
Reference in new issue
Block a user