From ac7c6af5a81bce515415a8c781d3679ec54a4811 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Thu, 1 Oct 2026 21:33:43 +0200 Subject: [PATCH 01/97] Read Rigaku d*TREK SMV images; add two home-source sets to the open battery The SMV reader only knew the ADSC vocabulary, so a Rigaku Saturn frame was recognised as SMV and then refused for want of PIXEL_SIZE. A header naming DETECTOR_NAMES is now read as d*TREK: - pixel size and the point of normal incidence from SPATIAL_DISTORTION_INFO; - image directions from DETECTOR_VECTORS combined by SPATIAL_DISTORTION_VECTORS, matched to a DetectorOrientation (a Saturn 944+ image is mirrored and turned a quarter, so the hand is preserved); - distance and the detector circles (2theta etc.) from GONIO_*, composed into PONI angles; spindle from ROTATION_VECTOR, angles from ROTATION; - wavelength from SCAN_WAVELENGTH, or the second number of SOURCE_WAVELENGTH (the first is a count - the old NumAny fallback would have read 1.0 A); - pixels above 32767 decoded as (v - 32768) * RAXIS_COMPRESSION_RATIO. dxtbx's FormatSMVRigakuSaturnNoTS, which claims headers without DTREK_DATE_TIME, ignores the distortion vectors; on a Saturn 944+ sweep that puts the spindle 90 degrees off (DIALS: 12% indexed vs 98% with the vectors applied, at the deposited cell). The arm geometry is confirmed by the data: the refined direct beam of a 2theta = 10 deg sweep lands at x = 621.1 against 621.6 predicted. The anomalous map peaks at 11.5 sigma on the two K+ and 6-7 sigma on S and P, so the hand is right. Battery, open arm: 5cc8 (rotating anode, Saturn 944+ SMV, 2theta = 10 deg sweep) and 9jq9 (Ga K-alpha liquid-metal jet, PILATUS3 1M miniCBF, single 450 deg sweep), both IRRMC. Run 20261001-2132_84228b_dtrek-home-source: 9jq9 passes (P 21 21 21, 1.65 A); 5cc8 merges to 1.53 A at the deposited cell but is called P 21 21 21 against the deposited P 21 21 2 (screw evidence on 00l 68-78 nats, tNCS at (1/2,1/2,0.11)) - left failing, not yet adjudicated. The Saturn gain (~5 ADU/photon) is not in the header and is not modelled. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/CHANGELOG.md | 4 ++ docs/EXTERNAL_TEST_DATA.md | 18 +++--- reader/JFJochSMVReader.cpp | 34 ++++++++++- reader/SMV.cpp | 117 ++++++++++++++++++++++++++++++++++++- reader/SMV.h | 20 +++++++ tests/JFJochReaderTest.cpp | 60 +++++++++++++++++++ tools/battery/README.md | 2 +- tools/battery/open.json | 2 + 8 files changed, 247 insertions(+), 10 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 5d7fbf8c1..9a34391b6 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -1,6 +1,10 @@ # Changelog ## 1.0.0 +### 1.0.0-rc.174 + +* Rugnux reads Rigaku d*TREK SMV images (Saturn CCD), including detector 2theta, image orientation and encoded pixel overflows. + ### 1.0.0-rc.173 * jfjoch_broker: Optional per-dataset authentication - statistics, images and plots can require a bearer token, which jfjoch_viewer supports. diff --git a/docs/EXTERNAL_TEST_DATA.md b/docs/EXTERNAL_TEST_DATA.md index ea071705c..8c17899b1 100644 --- a/docs/EXTERNAL_TEST_DATA.md +++ b/docs/EXTERNAL_TEST_DATA.md @@ -4,7 +4,7 @@ Jungfraujoch is developed at the Swiss Light Source, but a data-reduction pipeli ever sees its own detectors is not tested. The datasets below were collected by other people, on detectors and in file formats we do not produce ourselves, and are used here to check that `rugnux` reads foreign files correctly and reduces them to sensible results. Most were collected -at other facilities; a few come from SLS beamlines, where the data are still written by someone +at other facilities, two on laboratory X-ray sources; a few come from SLS beamlines, where the data are still written by someone else's detector and someone else's acquisition system. Their authors published all of these for exactly this kind of reuse, and this page is where we credit them. @@ -35,6 +35,7 @@ the table below; the repositories themselves are cited in | [36GK](https://www.rcsb.org/structure/36GK) | IRRMC [10.18430/M336GK](https://doi.org/10.18430/M336GK) | CLSI 08ID-1 | 2.28 | I 2 2 2 | 120.6 189.5 199.7 90.0 90.0 90.0 | Dectris Eiger 9M | D-GlcNAc-bound structure of Vibrio vulnificus putative carbohydrate binding module and split domain | | [3INP](https://www.rcsb.org/structure/3INP) | IRRMC [10.18430/m33inp](https://doi.org/10.18430/m33inp) | APS 21-ID-F | 2.05 | F 41 3 2 | 224.1 224.1 224.1 90.0 90.0 90.0 | marCCD, 225 mm plate | 2.05 Angstrom Resolution Crystal Structure of D-ribulose-phosphate 3-epimerase from Francisella tularensis. | | [3KY7](https://www.rcsb.org/structure/3KY7) | IRRMC [10.18430/m33ky7](https://doi.org/10.18430/m33ky7) | APS 21-ID-G | 2.35 | P 43 3 2 | 125.2 125.2 125.2 90.0 90.0 90.0 | marCCD, 300 mm plate | 2.35 Angstrom resolution crystal structure of a putative tRNA (guanine-7-)-methyltransferase (trmD) from Staphylococcus aureus subsp. aureus MRSA252 | +| [5CC8](https://www.rcsb.org/structure/5CC8) | IRRMC [10.18430/M35CC8](https://doi.org/10.18430/M35CC8) | Home source, Rigaku MicroMax-007 HF | 1.75 | P 21 21 2 | 87.1 93.8 72.5 90.0 90.0 90.0 | Rigaku Saturn 944+ | Structure of thiamine-monophosphate kinase from Acinetobacter baumannii in complex with AMPPNP | | [5EBI](https://www.rcsb.org/structure/5EBI) | MXRDR [10.18150/9887707](https://doi.org/10.18150/9887707) | BESSY 14.2 | 1.09 | P 1 21 1 | 35.7 44.1 35.7 90.0 120.0 90.0 | marCCD, 225 mm plate | Crystal structure of a DNA-RNA chimera in complex with Ba2+ ions: a case of unusual multi-domain twinning | | [5EPE](https://www.rcsb.org/structure/5EPE) | IRRMC [10.18430/m3159c](https://doi.org/10.18430/m3159c) | APS 21-ID-G | 1.90 | F 2 3 | 157.5 157.5 157.5 90.0 90.0 90.0 | Rayonix MX-300 | Crystal structure of SAM-dependent methyltransferase from Thiobacillus denitrificans in complex with S-Adenosyl-L-homocysteine | | [5F6M](https://www.rcsb.org/structure/5F6M) | SBGrid [10.15785/sbgrid/201](https://doi.org/10.15785/sbgrid/201) | SSRL BL11-1 | 1.10 | P 21 21 21 | 54.8 58.5 67.4 90.0 90.0 90.0 | PILATUS 6M | Isotropic Trypsin Model for Comparison of Diffuse Scattering | @@ -170,6 +171,7 @@ the table below; the repositories themselves are cited in | [9I80](https://www.rcsb.org/structure/9I80) | Zenodo [10.5281/zenodo.14844040](https://doi.org/10.5281/zenodo.14844040) | SOLEIL PROXIMA 1 | 1.95 | P 41 | 81.2 81.2 165.0 90.0 90.0 90.0 | Dectris Eiger 16M | LecA in complex with a tolcapone derivative glycomimetic | | [9IG7](https://www.rcsb.org/structure/9IG7) | IRRMC [10.18430/M39IG7](https://doi.org/10.18430/M39IG7) | PETRA III, EMBL c/o DESY P13 (MX1) | 2.60 | P 21 21 2 | 111.5 153.5 69.0 90.0 90.0 90.0 | Dectris EIGER1 Si 16M | KOD-H4 DNA polymerase mutant in a binary complex with DNA:DNA containing two AtNA nucleotides | | [9IH9](https://www.rcsb.org/structure/9IH9) | IRRMC [10.18430/M39IH9](https://doi.org/10.18430/M39IH9) | ESRF MASSIF-3 | 1.70 | C 1 2 1 | 78.8 133.9 82.3 90.0 101.4 90.0 | Dectris EIGER1 Si 4M | KEAP1 complexed to linear peptide 6 | +| [9JQ9](https://www.rcsb.org/structure/9JQ9) | IRRMC [10.18430/M39JQ9](https://doi.org/10.18430/M39JQ9) | Home source, Excillum MetalJet D2+ | 1.90 | P 21 21 21 | 48.6 50.5 78.6 90.0 90.0 90.0 | PILATUS3 1M | Crystal structure of Plasmoredoxin from Plasmodium falciparum a disulfide oxidoreductase protein unique to Plasmodium species | | [9JZO](https://www.rcsb.org/structure/9JZO) | IRRMC [10.18430/m39jzo](https://doi.org/10.18430/m39jzo) | PAL/PLS 11C | 1.40 | P 1 | 41.6 43.1 54.2 113.0 90.1 118.2 | PILATUS3 6M | Crystal structure of PHICD111_20024_EAD. | | [9KHR](https://www.rcsb.org/structure/9KHR) | Zenodo [10.5281/zenodo.14070468](https://doi.org/10.5281/zenodo.14070468) | RRCAT INDUS-2 PX-BL21 | 2.00 | P 21 21 21 | 48.7 50.3 78.0 90.0 90.0 90.0 | marCCD, 225 mm plate | Crystal structure of Plasmoredoxin, a disulfide oxidoreductase from Plasmodium falciparum crystallized in the presence of Dithiothreitol (DTT) | | [9MH4](https://www.rcsb.org/structure/9MH4) | IRRMC [10.18430/M39MH4](https://doi.org/10.18430/M39MH4) | NSLS-II 19-ID | 3.05 | P 21 3 | 138.7 138.7 138.7 90.0 90.0 90.0 | Dectris EIGER2 Si 9M | Crystal Structure of Bifunctional protein GlmU from Klebsiella aerogenes | @@ -319,21 +321,22 @@ marCCD and SMV files name the detector differently - or not at all. A marCCD fil its instrument header states the image dimensions and the pixel size, from which the plate size follows (3072 x 73.242 um = 225 mm, 4096 x 73.242 um = 300 mm), and its comment block a serial number; the LS-CAT beamlines additionally write `detector='Rayonix MX-300 s/n 023'` into the -dataset comment. An SMV header names only a serial (`DETECTOR_SN=930`). For those rows the +dataset comment. An ADSC-style SMV header names only a serial (`DETECTOR_SN=930`); a Rigaku d*TREK one names the +model (`CCD_DETECTOR_DESCRIPTION=Saturn944+`), and that is what its row carries. For those rows the Detector column carries what the file itself establishes: the plate size (`marCCD, 225 mm plate`), the comment's name where one is present (`Rayonix MX-300`), or the serial (`SMV, S/N 930`). ## Deposited models and structure factors -164 of the 171 datasets have a released PDB entry, and RCSB +166 of the 173 datasets have a released PDB entry, and RCSB reports released structure factors (`status_code_sf = REL`) for every one of them. A merged result from this pipeline can therefore be checked against the deposited model or against the deposited intensities. ## Rows where our reduction and the deposition disagree -Six of the 171 rows are ones where `rugnux` does not reproduce the deposited space group or +Six of the 173 rows are ones where `rugnux` does not reproduce the deposited space group or cell, and where we have looked at the disagreement closely enough to change how the row is scored. They are collected here because a scoring row that silently disagrees with a published entry is not something a reader should have to discover from the code. @@ -500,15 +503,16 @@ symmetries rather than to be easy to process. The counts below describe where it like everything else on this page, they are metadata about the depositions and their files, not measurements. -- **Repository:** IRRMC 84, SBGrid 35, Zenodo 28, MXRDR 14, ESRF 3, Keele University 3, XRDa 3, +- **Repository:** IRRMC 86, SBGrid 35, Zenodo 28, MXRDR 14, ESRF 3, Keele University 3, XRDa 3, UQ eSpace 1. - **Facility** - counted from the facility part of the Facility / beamline column, the beamline ignored so that entries deposited with and without one count the same, over the 170 rows that name one: APS 26, Diamond 20, ESRF 16, NSLS-II 14, BESSY 12, PETRA III 12, SSRL 11, ALS 8, SLS 7, SOLEIL 7, SPring-8 7, SSRF 6, PAL/PLS 5, CHESS 4, CLSI 3, ALBA 2, Australian Synchrotron 2, ELETTRA 2, LNLS 2, and one each from MAX IV, NSRRC, Photon Factory and RRCAT - Indus-2 - 23 facilities. -- **Crystal system, from the deposited space group of the 164 PDB-coded rows:** orthorhombic 42, + Indus-2 - 23 facilities. The other two rows were collected on laboratory sources, a rotating + anode and a liquid-metal jet. +- **Crystal system, from the deposited space group of the 166 PDB-coded rows:** orthorhombic 44, monoclinic 40, tetragonal 22, trigonal 19, hexagonal 16, cubic 13, triclinic 12. - **Long cell axes:** eleven PDB-coded rows have a deposited cell axis longer than 320 Å - 8V4O, 9ZMU, 9Z72, 9YL4, 5NW5, 6QAJ, 7QIJ, 8T7R, 9H0Q, 6G1F and 6OEL. diff --git a/reader/JFJochSMVReader.cpp b/reader/JFJochSMVReader.cpp index 65d47b0d3..a2eeb31cd 100644 --- a/reader/JFJochSMVReader.cpp +++ b/reader/JFJochSMVReader.cpp @@ -19,6 +19,13 @@ namespace { // same arrangement JFJochCBFReader makes for a miniCBF that states no axis table. const Coord ASSUMED_BASE_AXIS(-1.0f, 0.0f, 0.0f); +// A d*TREK vector in the internal frame. d*TREK states its vectors in the imgCIF laboratory frame +// (z from the sample to the source, y up), which differs from the internal one by a half turn about +// x - the same relation JFJochCBFReader uses for a miniCBF axis table. +Coord ImgCIFToInternal(const std::array &v) { + return {static_cast(v[0]), static_cast(-v[1]), static_cast(-v[2])}; +} + } // namespace bool JFJochSMVReader::CanRead(const std::string &path) { @@ -73,6 +80,19 @@ void JFJochSMVReader::ReadFiles(const std::string &path) { // Images are handed out as signed 32-bit whatever the file stored, so that is the depth the // rest of the code must see. detector.BitDepthImage(32); + // How the stored image sits in the detector plane, where the header states it. A Saturn's image + // is mirrored and turned a quarter relative to the internal convention (fast x slow points at the + // source, and the spindle runs along the rows) - the mirror is what a rotation cannot express, + // and without it the hand of every structure would be inverted. + if (header0_.fast_direction.has_value() && header0_.slow_direction.has_value()) { + const auto orientation = DetectorOrientation::Match(ImgCIFToInternal(*header0_.fast_direction), + ImgCIFToInternal(*header0_.slow_direction)); + if (!orientation.has_value()) + throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, + files_[0] + ": the image directions its header states are not a " + "square-on layout this reader supports"); + detector.ImageOrientation(orientation.value()); + } detector.MinFrameTime(std::chrono::microseconds(0)); detector.MinCountTime(std::chrono::microseconds(0)); detector.ReadOutTime(std::chrono::nanoseconds(0)); @@ -93,6 +113,16 @@ void JFJochSMVReader::ReadFiles(const std::string &path) { ASSUMED_BASE_AXIS), rot1, rot2, rot3); dataset_->experiment.PoniRot1_rad(rot1).PoniRot2_rad(rot2).PoniRot3_rad(rot3); } + // d*TREK states every detector circle with its vector instead. They compose outermost first, as + // they carry each other: the first named circle carries all the ones after it. + if (!header0_.detector_circles.empty()) { + RotMatrix r; + for (const auto &c: header0_.detector_circles) + r = r * RotMatrix(static_cast(c.angle_deg * PI / 180.0), ImgCIFToInternal(c.axis)); + float rot1 = 0, rot2 = 0, rot3 = 0; + PoniAnglesFromMatrix(r, rot1, rot2, rot3); + dataset_->experiment.PoniRot1_rad(rot1).PoniRot2_rad(rot2).PoniRot3_rad(rot3); + } dataset_->experiment.IncidentEnergy_keV(WVL_1A_IN_KEV / static_cast(header0_.wavelength_A)); // Only when the header states a sane one. The exposure field is not always filled in: one @@ -129,7 +159,9 @@ void JFJochSMVReader::ReadFiles(const std::string &path) { dataset_->experiment.Goniometer(GoniometerAxis(header0_.axis_name, static_cast(layout.start_deg), static_cast(layout.increment_deg), - ASSUMED_BASE_AXIS, {})); + header0_.spindle_axis.has_value() + ? ImgCIFToInternal(*header0_.spindle_axis) + : ASSUMED_BASE_AXIS, {})); dataset_->error_value = -1; dataset_->experiment.ImagesPerTrigger(static_cast(files_.size())); diff --git a/reader/SMV.cpp b/reader/SMV.cpp index a7b221e97..268a59bfb 100644 --- a/reader/SMV.cpp +++ b/reader/SMV.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include "../common/JFJochException.h" @@ -86,6 +87,113 @@ double NumAny(const std::map &kv, return dflt; } +// A value holding several numbers, as d*TREK writes its vectors and circle lists. +std::vector Nums(const std::map &kv, const std::string &key) { + std::vector out; + const auto it = kv.find(key); + if (it == kv.end()) return out; + std::istringstream in(it->second); + double v; + while (in >> v) out.push_back(v); + return out; +} + +std::vector Words(const std::map &kv, const std::string &key) { + std::vector out; + const auto it = kv.find(key); + if (it == kv.end()) return out; + std::istringstream in(it->second); + std::string w; + while (in >> w) out.push_back(w); + return out; +} + +// d*TREK, as Rigaku's CrystalClear writes it for the Saturn and R-AXIS families. Read the way dxtbx's +// FormatSMVRigakuSaturn reads it: the image directions are DETECTOR_VECTORS combined by +// SPATIAL_DISTORTION_VECTORS, the point of normal incidence (pixels) and the pixel size (mm) come +// from SPATIAL_DISTORTION_INFO, and the detector circles from GONIO_*, where the distance is +// a translation along the normal applied before the rotations - so the beam position stays the PONI +// and the distance the normal distance whatever the arm does. +// +// The distortion vectors are not optional. dxtbx's variant for headers without DTREK_DATE_TIME +// leaves them out, and on a Saturn 944+ sweep that puts the spindle 90 degrees off: DIALS indexes +// 12% of the spots with it and 98% with the vectors applied, at the deposited cell. +void ParseDtrek(const std::map &kv, const std::string &path, Header &h) { + const auto detector_names = Words(kv, "DETECTOR_NAMES"); + const std::string det = detector_names.empty() ? "" : detector_names[0]; + const auto info = Nums(kv, det + "SPATIAL_DISTORTION_INFO"); + const auto vectors = Nums(kv, det + "DETECTOR_VECTORS"); + const auto names = Words(kv, det + "GONIO_NAMES"); + const auto units = Words(kv, det + "GONIO_UNITS"); + const auto values = Nums(kv, det + "GONIO_VALUES"); + const auto circle_vectors = Nums(kv, det + "GONIO_VECTORS"); + const auto rotation = Nums(kv, "ROTATION"); // start, end, increment, exposure, ... + const auto spindle = Nums(kv, "ROTATION_VECTOR"); + if (info.size() < 4 || vectors.size() < 6 || rotation.size() < 4 || spindle.size() < 3 + || names.size() != values.size() || units.size() != values.size() + || circle_vectors.size() != 3 * values.size()) + throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, + path + ": d*TREK header does not describe the detector and the rotation"); + + const auto type = kv.find("DATA_TYPE"); + if (type != kv.end() && type->second.find("short") == std::string::npos) + throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, + path + ": only 16-bit SMV images are supported (Data_type=" + type->second + ")"); + + h.beam_x_px = info[0]; + h.beam_y_px = info[1]; + h.pixel_x_m = info[2] * 1e-3; + h.pixel_y_m = info[3] * 1e-3; + // Each image direction as a combination of the two detector vectors: fast = d0 f + d1 s, + // slow = d2 f + d3 s. Absent, the detector vectors are the image directions. + auto d = Nums(kv, det + "SPATIAL_DISTORTION_VECTORS"); + if (d.size() < 4) + d = {1, 0, 0, 1}; + std::array fast{}, slow{}; + for (int i = 0; i < 3; i++) { + fast[i] = d[0] * vectors[i] + d[1] * vectors[3 + i]; + slow[i] = d[2] * vectors[i] + d[3] * vectors[3 + i]; + } + h.fast_direction = fast; + h.slow_direction = slow; + + h.distance_m = 0; + for (size_t i = 0; i < values.size(); i++) { + const std::array axis{circle_vectors[3 * i], circle_vectors[3 * i + 1], + circle_vectors[3 * i + 2]}; + if (units[i] == "deg") + h.detector_circles.push_back({axis, values[i]}); + else if (names[i] == "Distance") + h.distance_m = values[i] * 1e-3; + else if (values[i] != 0.0) + throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, + path + ": d*TREK detector translation " + names[i] + " is not supported"); + } + + h.start_angle_deg = rotation[0]; + h.angle_increment_deg = rotation[2]; + h.exposure_s = rotation[3]; + h.spindle_axis = std::array{spindle[0], spindle[1], spindle[2]}; + const auto axis_name = Words(kv, "ROTATION_AXIS_NAME"); + if (!axis_name.empty()) { + h.axis_name = axis_name[0]; + std::transform(h.axis_name.begin(), h.axis_name.end(), h.axis_name.begin(), + [](unsigned char c) { return std::tolower(c); }); + } + + // SOURCE_WAVELENGTH leads with HOW MANY wavelengths follow ("1.000000 1.541780"), so its first + // number is a count, not a wavelength. SCAN_WAVELENGTH is the one the scan used. + const auto scan_wavelength = Nums(kv, "SCAN_WAVELENGTH"); + const auto source_wavelength = Nums(kv, "SOURCE_WAVELENGTH"); + h.wavelength_A = !scan_wavelength.empty() ? scan_wavelength[0] + : (source_wavelength.size() >= 2 ? source_wavelength[1] : 0.0); + + h.overflow_ratio = static_cast(Num(kv, "RAXIS_COMPRESSION_RATIO")); + const auto id = kv.find(det + "DETECTOR_IDENTIFICATION"); + if (id != kv.end()) + h.detector = id->second; +} + std::optional> HeaderBlock(const std::string &path) { return ParseBlock(ReadPrefix(path, PROBE_BYTES)); } @@ -123,7 +231,7 @@ Header Parse(const std::map &kv, const std::string &pa h.beam_x_px = h.pixel_x_m > 0 ? bx_mm * 1e-3 / h.pixel_x_m : 0.0; h.beam_y_px = h.pixel_y_m > 0 ? by_mm * 1e-3 / h.pixel_y_m : 0.0; - h.wavelength_A = NumAny(kv, {"WAVELENGTH", "SOURCE_WAVELENGTH"}); + h.wavelength_A = NumAny(kv, {"WAVELENGTH"}); h.angle_increment_deg = NumAny(kv, {"OSC_RANGE", "OSCILLATION_RANGE"}); // OSC_START is the angle of THIS image; PHI is where the circle stands, which is the same // thing on the single-axis goniometers that write this format, and is the only value some @@ -142,6 +250,8 @@ Header Parse(const std::map &kv, const std::string &pa std::transform(a.begin(), a.end(), a.begin(), [](unsigned char c) { return std::tolower(c); }); h.axis_name = a; } + if (kv.count("DETECTOR_NAMES")) + ParseDtrek(kv, path, h); return h; } @@ -276,6 +386,11 @@ Header ReadInto(const std::string &path, int32_t *out, size_t capacity, std::vec for (size_t i = 0; i < npixel; i++) out[i] = static_cast(static_cast((src[2 * i] << 8) | src[2 * i + 1])); } + if (h.overflow_ratio > 0) { + for (size_t i = 0; i < npixel; i++) + if (out[i] > 32767) + out[i] = static_cast((out[i] - 32768) * h.overflow_ratio); + } return h; } diff --git a/reader/SMV.h b/reader/SMV.h index aadc4d090..9bbf3a14d 100644 --- a/reader/SMV.h +++ b/reader/SMV.h @@ -3,8 +3,10 @@ #pragma once +#include #include #include +#include #include #include @@ -22,6 +24,11 @@ // // Note the beam centre is in MILLIMETRES, not pixels, and its convention is the one thing writers // disagree about - see the comment on beam_x_mm in the struct. +// +// Rigaku's d*TREK writes the same container with a different vocabulary: the detector and the +// goniometer as named circles with their vectors (CCD_GONIO_VALUES, ROTATION_VECTOR, ...), the beam +// position in pixels, and its own overflow encoding for 16-bit pixels. A header naming +// DETECTOR_NAMES is read that way. namespace smv { struct Header { @@ -48,6 +55,19 @@ struct Header { int64_t saturation = 0; // CCD_IMAGE_SATURATION: the value a saturated pixel carries std::string detector; // DETECTOR_SN, where stated std::string axis_name = "phi"; + // d*TREK only (Rigaku CrystalClear): the geometry is stated as vectors rather than the ADSC + // scalars, in the imgCIF laboratory frame - z from the sample to the source, y up. Absent on an + // ADSC header, where two_theta_deg above carries the one detector circle there is. + std::optional> fast_direction; + std::optional> slow_direction; + std::optional> spindle_axis; + struct Circle { + std::array axis; + double angle_deg; + }; + std::vector detector_circles; // header order: the outermost circle first + // RAXIS_COMPRESSION_RATIO: a stored value above 32767 is (value - 32768) * ratio. 0 = not used. + int64_t overflow_ratio = 0; std::map raw; // every key, for anything not modelled above }; diff --git a/tests/JFJochReaderTest.cpp b/tests/JFJochReaderTest.cpp index 9ade44cb7..782dff797 100644 --- a/tests/JFJochReaderTest.cpp +++ b/tests/JFJochReaderTest.cpp @@ -4466,6 +4466,66 @@ TEST_CASE("JFJochSMVReader_Geometry", "[HDF5][Full][portable]") { RemoveSMVSweep("smv_", ".img"); } + SECTION("a d*TREK header: vectors, a swung arm, a counted wavelength and encoded overflows") { + // The vocabulary Rigaku's CrystalClear writes for a Saturn: the detector as named circles with + // their vectors in the imgCIF frame, the beam position in pixels, and the wavelength behind a + // count of how many follow. The arm is swung 10 degrees about +x. + const int nx = 24, ny = 16; + std::ostringstream h; + h << "{\nHEADER_BYTES= 1024;\nBYTE_ORDER=little_endian;\nDIM=2;\nSIZE1=" << nx << ";\nSIZE2=" << ny << ";\n" + << "Data_type=unsigned short int;\nDETECTOR_NAMES=CCD_;\nCCD_DETECTOR_VECTORS=1 0 0 0 1 0;\n" + << "CCD_SPATIAL_DISTORTION_VECTORS=0 -1 1 0;\n" + << "CCD_SPATIAL_DISTORTION_INFO=523.6355 515.3276 0.0900 0.0900;\n" + << "CCD_GONIO_NAMES=RotAboutBeam 2Theta RotY XShift YShift Distance;\nCCD_GONIO_NUM_VALUES=6;\n" + << "CCD_GONIO_UNITS=deg deg deg mm mm mm;\nCCD_GONIO_VALUES=0.0000 10.0000 0.0000 0.0000 0.0000 50.0000;\n" + << "CCD_GONIO_VECTORS=0 0 1 1 0 0 0 1 0 1 0 0 0 1 0 0 0 -1;\n" + << "ROTATION=-13.500 -13.000 0.500 30.000 0.000 0.000 0.000 100.000 0.000 0.000;\n" + << "ROTATION_AXIS_NAME=Omega;\nROTATION_VECTOR=1.000 0.000 0.000;\n" + << "SOURCE_WAVELENGTH=1.000000 1.541780;\nSATURATED_VALUE=4194176;\nRAXIS_COMPRESSION_RATIO=128;\n}\n"; + std::string head = h.str(); + head.resize(1024, ' '); + std::vector pixels(static_cast(nx) * ny, 100); + pixels[1] = 33024; // the smallest encoded value: (33024 - 32768) * 128 = 32768 + pixels[2] = 40000; + { + std::ofstream out("dtrek_0001.img", std::ios::binary); + out.write(head.data(), static_cast(head.size())); + out.write(reinterpret_cast(pixels.data()), + static_cast(pixels.size() * sizeof(uint16_t))); + } + + std::vector decoded; + smv::Read("dtrek_0001.img", decoded); + CHECK(decoded[0] == 100); + CHECK(decoded[1] == 32768); + CHECK(decoded[2] == (40000 - 32768) * 128); + + JFJochSMVReader reader; + REQUIRE_NOTHROW(reader.ReadFiles("dtrek_0001.img")); + const auto x = reader.GetDataset()->experiment; + CHECK(x.GetWavelength_A() == Catch::Approx(1.54178).epsilon(1e-5)); // not the count, 1.0 + CHECK(x.GetPixelSize_mm() == Catch::Approx(0.09)); + CHECK(x.GetDetectorDistance_mm() == Catch::Approx(50.0)); + CHECK(x.GetSaturationLimit() == 4194176); + const auto geom = x.GetDiffractionGeometry(); + // The distortion vectors turn the detector vectors a quarter, and fast x slow still points + // at the source: a mirrored image, which only the orientation can say. + CHECK(geom.GetOrientation().IsMirrorY()); + CHECK(geom.GetOrientation().GetQuarterTurns() == 1); + // The spindle, +x in the file, runs along increasing row once the image is laid out, and the + // arm turns about the same axis - so it moves the direct beam 50 tan(10) mm = 97.96 px off + // the point of normal incidence ALONG THE ROW, to higher columns. + const auto [beam_x, beam_y] = geom.GetDirectBeam_pxl(); + CHECK(beam_x == Catch::Approx(523.6355 + 97.96).margin(0.02)); + CHECK(beam_y == Catch::Approx(515.3276).margin(0.02)); + REQUIRE(x.GetGoniometer().has_value()); + CHECK(x.GetGoniometer()->GetStart_deg() == Catch::Approx(-13.5)); + CHECK(x.GetGoniometer()->GetIncrement_deg() == Catch::Approx(0.5)); + CHECK((x.GetGoniometer()->GetAxis() - Coord(1, 0, 0)).Length() < 1e-5f); + reader.Close(); + remove("dtrek_0001.img"); + } + SECTION("a brace block that is not an image header is refused") { SMVFields f; f.write_header = false; WriteSMVFrame("smvbare_001.img", f); diff --git a/tools/battery/README.md b/tools/battery/README.md index a5b180217..d736be19c 100644 --- a/tools/battery/README.md +++ b/tools/battery/README.md @@ -138,7 +138,7 @@ the copies. The paths inside the manifests do not change. [`docs/EXTERNAL_TEST_DATA.md`](../../docs/EXTERNAL_TEST_DATA.md). The sources are IRRMC (proteindiffraction.org), SBGrid Data Bank, Zenodo and a few others, and the page also says what each archive holds. Please cite those DOIs. At PSI the data root is `/home/data/open`, a - symlink to `/home/data/nonsls/raw`. 51 of the dataset directories in it are themselves symlinks + symlink to `/home/data/nonsls/raw`. 69 of the dataset directories in it are themselves symlinks into `/data/scout_staging2`, so **copy with symlinks followed**: `rsync -aL` or `cp -rL`. A plain `rsync -a` copies dangling links. The open arm is about 1.7 TB. - **inhouse**: these are our own measurements and cannot be downloaded publicly. Ask the diff --git a/tools/battery/open.json b/tools/battery/open.json index b7350c677..7e44a26ee 100644 --- a/tools/battery/open.json +++ b/tools/battery/open.json @@ -3,6 +3,7 @@ {"id": "36gk", "input": "36gk/CLS-0074_5-3_36GK/data/CLS-0074_5-3_master.h5", "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [120.58, 189.49, 199.69, 90.0, 90.0, 90.0], "dmin": 2.28}, "tags": ["h5", "orthorhombic"]}, {"id": "3inp", "input": "3inp/IDP02542_3inp/data/idp02542b.001", "ref": {"sg": "F 41 3 2", "sgno": 210, "cell": [224.08, 224.08, 224.08, 90.0, 90.0, 90.0], "dmin": 2.05}, "tags": ["marCCD", "cubic"]}, {"id": "3ky7", "input": "3ky7/IDP90258_3ky7/data/idp90258f.001", "ref": {"sg": "P 43 3 2", "sgno": 212, "cell": [125.176, 125.176, 125.176, 90.0, 90.0, 90.0], "dmin": 2.35}, "tags": ["marCCD", "cubic"]}, + {"id": "5cc8", "input": "5cc8/data/263060g10_x0001.img", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [87.14, 93.76, 72.49, 90.0, 90.0, 90.0], "dmin": 1.75}, "tags": ["smv", "orthorhombic", "home-source", "tncs"]}, {"id": "5ebi", "input": "5ebi/dna-rna-chimera_Ba_high_2_001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [35.72, 44.1, 35.72, 90.0, 119.98, 90.0], "dmin": 1.09}, "pinned": true, "tags": ["marCCD", "monoclinic", "twin"]}, {"id": "5epe", "input": "5epe/030805_5epe/data/E1_7_set.001", "ref": {"sg": "F 2 3", "sgno": 196, "cell": [157.536, 157.536, 157.536, 90.0, 90.0, 90.0], "dmin": 1.9}, "tags": ["marCCD", "cubic"]}, {"id": "5f6m", "input": "5f6m/crystal3_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [54.81, 58.51, 67.42, 90.0, 90.0, 90.0], "dmin": 1.1}, "tags": ["cbf", "orthorhombic", "diffuse"]}, @@ -120,6 +121,7 @@ {"id": "9i80", "input": "9i80/20230422-PX1-LecA_RO5-5315-1/LecA-RO5_1_master.h5", "ref": {"sg": "P 41", "sgno": 76, "cell": [81.214, 81.214, 165.029, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["h5", "tetragonal", "twin"]}, {"id": "9ig7", "input": "9ig7/9IG7/data/KOD-P596-G2_3_00001.cbf", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [111.472, 153.471, 69.03, 90.0, 90.0, 90.0], "dmin": 2.6}, "tags": ["cbf", "orthorhombic"]}, {"id": "9ih9", "input": "9ih9/run_01_05_datacollection_9IH9/data/design-Xinjian-P1Thro-4_1_5_1_master.h5", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [78.758, 133.933, 82.32, 90.0, 101.356, 90.0], "dmin": 1.7}, "tags": ["h5", "monoclinic"]}, + {"id": "9jq9", "input": "9jq9/pfplrxdata_9JQ9/data/lp13_001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [48.551, 50.499, 78.571, 90.0, 90.0, 90.0], "dmin": 1.9}, "tags": ["cbf", "orthorhombic", "home-source"]}, {"id": "9jzo", "input": "9jzo/JS_35P2_C3-002_diffraction_files_9JZO/data/JS_35P2_C3-002_0001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [41.63, 43.1, 54.2, 112.97, 90.11, 118.18], "dmin": 1.4}, "tags": ["cbf", "triclinic"], "tiers": {"smoke": "triclinic P1"}}, {"id": "9khr", "input": "9khr/PfPlrx-DTT-9KHR/data-PfPlrx-DTT-9KHR/ods1467-lp13D1-01001.mccd", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [48.704, 50.31, 78.018, 90.0, 90.0, 90.0], "dmin": 2.0}, "tags": ["marCCD", "orthorhombic"], "tiers": {"smoke": "marCCD reader (.mccd), fast"}}, {"id": "9mh4", "input": "9mh4/KlaeA_00150_a_B1_9mh4/data/PSL-0501_2719_master.h5", "ref": {"sg": "P 21 3", "sgno": 198, "cell": [138.65, 138.65, 138.65, 90.0, 90.0, 90.0], "dmin": 3.05}, "tags": ["h5", "cubic"]}, -- 2.54.0 From dcc710a226854bf5b83c4623a3ce03327de077bd Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 2 Oct 2026 13:57:30 +0200 Subject: [PATCH 02/97] Battery: eight more home-source sets (open arm), five small molecules (in-house arm) Open arm, all IRRMC, single Cu K-alpha sweeps on Rigaku Saturn CCDs read through the d*TREK SMV path: 3mc4 (H 3), 3meb (P 1 21 1, Saturn 944, overflow ratio 32), 3p85 (P 63 2 2), 3r6o (I 41), 5uth (P 31 2 1), 5vml (P 42 21 2), 6cee and 6v2r (P 21 21 21 / P 41 21 2, Saturn A200, unbinned 2048x2048, 0.1 mm pixels). Detector 2theta spans -10 to +10 deg. Run 20261002-1326_84228b_home-source-12: all pass except 3r6o, called I 4 2 2 against the deposited I 41 with twinning suspected (<|L|> 0.32). Two further candidates (3r6h, 3sgw) were dropped: what the repository lists as one dataset is two sweeps under one file template, split by a phi change, which the reader refuses. In-house arm: aspirin (20 and 25 keV), anhydrous citric acid, HEPES and YAG, measured at X10SA. The reference is XDS's CORRECT.LP (refs --write) with the space group taken from the literature as ref_override, because XDS reports only Sohncke groups; each override cites its COD entry and paper. Run 20261002-1354_84228b_small-molecules-inhouse: four pass, including HEPES as P b c a where XDS has P 21 21 21; YAG is called I 41 3 2 against I a -3 d (both programs merge it poorly, rugnux CC1/2 0.40, XDS ISa 3.2). EXTERNAL_TEST_DATA.md gains the eight depositions (DOIs checked on DataCite) and updated counts; the battery README lists the small-molecule standards and has the current count of symlinked open-arm directories (42). Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/EXTERNAL_TEST_DATA.md | 24 ++++++++++++++++-------- tools/battery/README.md | 4 ++-- tools/battery/inhouse.json | 7 ++++++- tools/battery/open.json | 8 ++++++++ 4 files changed, 32 insertions(+), 11 deletions(-) diff --git a/docs/EXTERNAL_TEST_DATA.md b/docs/EXTERNAL_TEST_DATA.md index 8c17899b1..dd6dfc16b 100644 --- a/docs/EXTERNAL_TEST_DATA.md +++ b/docs/EXTERNAL_TEST_DATA.md @@ -4,7 +4,7 @@ Jungfraujoch is developed at the Swiss Light Source, but a data-reduction pipeli ever sees its own detectors is not tested. The datasets below were collected by other people, on detectors and in file formats we do not produce ourselves, and are used here to check that `rugnux` reads foreign files correctly and reduces them to sensible results. Most were collected -at other facilities, two on laboratory X-ray sources; a few come from SLS beamlines, where the data are still written by someone +at other facilities, ten on laboratory X-ray sources; a few come from SLS beamlines, where the data are still written by someone else's detector and someone else's acquisition system. Their authors published all of these for exactly this kind of reuse, and this page is where we credit them. @@ -35,6 +35,10 @@ the table below; the repositories themselves are cited in | [36GK](https://www.rcsb.org/structure/36GK) | IRRMC [10.18430/M336GK](https://doi.org/10.18430/M336GK) | CLSI 08ID-1 | 2.28 | I 2 2 2 | 120.6 189.5 199.7 90.0 90.0 90.0 | Dectris Eiger 9M | D-GlcNAc-bound structure of Vibrio vulnificus putative carbohydrate binding module and split domain | | [3INP](https://www.rcsb.org/structure/3INP) | IRRMC [10.18430/m33inp](https://doi.org/10.18430/m33inp) | APS 21-ID-F | 2.05 | F 41 3 2 | 224.1 224.1 224.1 90.0 90.0 90.0 | marCCD, 225 mm plate | 2.05 Angstrom Resolution Crystal Structure of D-ribulose-phosphate 3-epimerase from Francisella tularensis. | | [3KY7](https://www.rcsb.org/structure/3KY7) | IRRMC [10.18430/m33ky7](https://doi.org/10.18430/m33ky7) | APS 21-ID-G | 2.35 | P 43 3 2 | 125.2 125.2 125.2 90.0 90.0 90.0 | marCCD, 300 mm plate | 2.35 Angstrom resolution crystal structure of a putative tRNA (guanine-7-)-methyltransferase (trmD) from Staphylococcus aureus subsp. aureus MRSA252 | +| [3MC4](https://www.rcsb.org/structure/3MC4) | IRRMC [10.18430/M33MC4](https://doi.org/10.18430/M33MC4) | Home source, Rigaku MicroMax-007 HF | 1.95 | H 3 | 104.0 104.0 105.5 90.0 90.0 120.0 | Rigaku Saturn 944+ | Crystal structure of WW/RSP5/WWP domain: bacterial transferase hexapeptide repeat: serine O-Acetyltransferase from Brucella Melitensis | +| [3MEB](https://www.rcsb.org/structure/3MEB) | IRRMC [10.18430/M33MEB](https://doi.org/10.18430/M33MEB) | Home source, Rigaku MicroMax-007 HF | 1.90 | P 1 21 1 | 58.6 101.2 81.5 90.0 90.6 90.0 | Rigaku Saturn 944 | Structure of cytoplasmic aspartate aminotransferase from giardia lamblia | +| [3P85](https://www.rcsb.org/structure/3P85) | IRRMC [10.18430/M33P85](https://doi.org/10.18430/M33P85) | Home source, Rigaku FR-E+ SuperBright | 1.90 | P 63 2 2 | 127.3 127.3 72.9 90.0 90.0 120.0 | Rigaku Saturn 944+ | Crystal structure enoyl-coa hydratase from mycobacterium avium | +| [3R6O](https://www.rcsb.org/structure/3R6O) | IRRMC [10.18430/M33R6O](https://doi.org/10.18430/M33R6O) | Home source, Rigaku FR-E+ SuperBright | 1.95 | I 41 | 90.7 90.7 76.1 90.0 90.0 90.0 | Rigaku Saturn 944+ | Crystal structure of a probable 2-hydroxyhepta-2,4-diene-1, 7-dioateisomerase from Mycobacterium abscessus | | [5CC8](https://www.rcsb.org/structure/5CC8) | IRRMC [10.18430/M35CC8](https://doi.org/10.18430/M35CC8) | Home source, Rigaku MicroMax-007 HF | 1.75 | P 21 21 2 | 87.1 93.8 72.5 90.0 90.0 90.0 | Rigaku Saturn 944+ | Structure of thiamine-monophosphate kinase from Acinetobacter baumannii in complex with AMPPNP | | [5EBI](https://www.rcsb.org/structure/5EBI) | MXRDR [10.18150/9887707](https://doi.org/10.18150/9887707) | BESSY 14.2 | 1.09 | P 1 21 1 | 35.7 44.1 35.7 90.0 120.0 90.0 | marCCD, 225 mm plate | Crystal structure of a DNA-RNA chimera in complex with Ba2+ ions: a case of unusual multi-domain twinning | | [5EPE](https://www.rcsb.org/structure/5EPE) | IRRMC [10.18430/m3159c](https://doi.org/10.18430/m3159c) | APS 21-ID-G | 1.90 | F 2 3 | 157.5 157.5 157.5 90.0 90.0 90.0 | Rayonix MX-300 | Crystal structure of SAM-dependent methyltransferase from Thiobacillus denitrificans in complex with S-Adenosyl-L-homocysteine | @@ -49,7 +53,10 @@ the table below; the repositories themselves are cited in | [5REO](https://www.rcsb.org/structure/5REO) | Zenodo [10.5281/zenodo.3730956](https://doi.org/10.5281/zenodo.3730956) | Diamond I04-1 | 1.88 | C 1 2 1 | 112.4 52.6 44.4 90.0 103.0 90.0 | PILATUS 6M-F | PanDDA analysis group deposition -- Crystal Structure of SARS-CoV-2 main protease in complex with PCM-0102578 | | [5SRC](https://www.rcsb.org/structure/5SRC) | IRRMC [10.18430/M35SRC](https://doi.org/10.18430/M35SRC) | ALS 8.3.1 | 1.05 | P 43 | 88.7 88.7 39.2 90.0 90.0 90.0 | PILATUS3 6M | PanDDA analysis group deposition -- Crystal structure of SARS-CoV-2 NSP3 macrodomain in complex with Z5198562500 - (R,R) and (R,S) isomers | | [5T39](https://www.rcsb.org/structure/5T39) | SBGrid [10.15785/sbgrid/356](https://doi.org/10.15785/sbgrid/356) | APS 21-ID-F | 1.10 | P 1 21 1 | 50.2 41.3 58.5 90.0 98.6 90.0 | Rayonix MX-300 | Crystal Structure of the N-terminal domain of EvdMO1 in the presence of SAH and D-fucose | +| [5UTH](https://www.rcsb.org/structure/5UTH) | IRRMC [10.18430/M35UTH](https://doi.org/10.18430/M35UTH) | Home source, Rigaku FR-E+ SuperBright | 1.95 | P 31 2 1 | 69.3 69.3 153.8 90.0 90.0 120.0 | Rigaku Saturn 944+ | Crystal structure of thioredoxin reductase from Mycobacterium smegmatis in complex with FAD | +| [5VML](https://www.rcsb.org/structure/5VML) | IRRMC [10.18430/M35VML](https://doi.org/10.18430/M35VML) | Home source, Rigaku FR-E+ SuperBright | 1.70 | P 42 21 2 | 66.3 66.3 115.3 90.0 90.0 90.0 | Rigaku Saturn 944+ | Crystal Structure of Acetoacetyl-CoA Reductase from Burkholderia Pseudomallei 1710b with bound NADP | | [6CDL](https://www.rcsb.org/structure/6CDL) | IRRMC [10.18430/m36cdl](https://doi.org/10.18430/m36cdl) | APS 22-ID | 1.25 | P 21 21 2 | 58.3 85.9 46.1 90.0 90.0 90.0 | marCCD, 300 mm plate | HIV-1 wild type protease with GRL-03214A, 6-5-5-ring fused umbrella-like tetrahydropyranofuran as the P2-ligand, a cyclopropylaminobenzothiazole as the P2'-ligand and 3,5-difluorophenylmethyl as the P1-ligand | +| [6CEE](https://www.rcsb.org/structure/6CEE) | IRRMC [10.18430/M36CEE](https://doi.org/10.18430/M36CEE) | Home source, Rigaku FR-E SuperBright | 1.55 | P 21 21 21 | 40.7 44.1 55.9 90.0 90.0 90.0 | Rigaku Saturn A200 | Crystal structure of fragment 3-(1-Methyl-2-oxo-1,2-dihydroquinoxalin-3-yl)propionic acid bound in the ubiquitin binding pocket of the HDAC6 zinc-finger domain | | [6F3P](https://www.rcsb.org/structure/6F3P) | IRRMC [10.18430/M36F3P](https://doi.org/10.18430/M36F3P) | APS 22-ID | 1.35 | C 1 2 1 | 142.9 85.7 112.0 90.0 122.2 90.0 | marCCD, 300 mm plate | Crystal structure of S-adenosyl-L-homocysteine hydrolase from Pseudomonas aeruginosa in complex with 3'-deoxyadenosine and K+ cation | | [6FID](https://www.rcsb.org/structure/6FID) | SBGrid [10.15785/sbgrid/541](https://doi.org/10.15785/sbgrid/541) | ESRF ID30B | 2.20 | P 21 21 21 | 59.9 64.1 69.7 90.0 90.0 90.0 | PILATUS3 6M | Bovine trypsin solved by S-SAD on ID30B | | [6FVZ](https://www.rcsb.org/structure/6FVZ) | IRRMC [10.18430/m36fvz](https://doi.org/10.18430/m36fvz) | ESRF ID23-2 | 1.80 | C 2 2 2 | 131.2 222.8 86.5 90.0 90.0 90.0 | PILATUS3 X 2M | Crystal structure of human monoamine oxidase B (MAO B) in complex with an inhibitor | @@ -83,6 +90,7 @@ the table below; the repositories themselves are cited in | [6TTN](https://www.rcsb.org/structure/6TTN) | IRRMC [10.18430/m36ttn](https://doi.org/10.18430/m36ttn) | BESSY 14.1 | 1.12 | P 21 21 21 | 39.9 79.8 104.7 90.0 90.0 90.0 | PILATUS 6M | N-terminally truncated hyoscyamine 6-hydroxylase (tH6H) in complex with N-oxalylglycine and hyoscyamine | | [6U7G](https://www.rcsb.org/structure/6U7G) | IRRMC [10.18430/m36u7g](https://doi.org/10.18430/m36u7g) | APS 23-ID-B | 2.35 | P 1 21 1 | 99.6 98.7 147.5 90.0 104.6 90.0 | Dectris Eiger 16M | HCoV-229E RBD Class V in complex with human APN | | [6UKF](https://www.rcsb.org/structure/6UKF) | IRRMC [10.18430/m36ukf](https://doi.org/10.18430/m36ukf) | APS 22-ID | 1.00 | P 1 21 1 | 61.0 37.3 69.0 90.0 109.8 90.0 | Dectris Eiger 16M | HhaI endonuclease in Complex with DNA at 1 Angstrom Resolution | +| [6V2R](https://www.rcsb.org/structure/6V2R) | IRRMC [10.18430/m36v2r](https://doi.org/10.18430/m36v2r) | Home source, Rigaku FR-E | 1.60 | P 41 21 2 | 40.2 40.2 83.1 90.0 90.0 90.0 | Rigaku Saturn A200 | Crystal Structure of chromodomain of CBX7 mutant V13A in complex with inhibitor UNC3866 | | [6VWW](https://www.rcsb.org/structure/6VWW) | IRRMC [10.18430/m36vww](https://doi.org/10.18430/m36vww) | APS 19-ID | 2.20 | P 63 | 150.5 150.5 111.3 90.0 90.0 120.0 | PILATUS3 6M | Crystal Structure of NSP15 Endoribonuclease from SARS CoV-2. | | [6W4H](https://www.rcsb.org/structure/6W4H) | IRRMC [10.18430/m36w4h](https://doi.org/10.18430/m36w4h) | APS 21-ID-F | 1.80 | P 31 2 1 | 167.7 167.7 51.9 90.0 90.0 120.0 | Rayonix MX-300 | 1.80 Angstrom Resolution Crystal Structure of NSP16 - NSP10 Complex from SARS-CoV-2 | | [6W75](https://www.rcsb.org/structure/6W75) | IRRMC [10.18430/m36w75](https://doi.org/10.18430/m36w75) | APS 21-ID-F | 1.95 | P 32 2 1 | 166.2 166.2 98.3 90.0 90.0 120.0 | Rayonix MX-300 | 1.95 Angstrom Resolution Crystal Structure of NSP10 - NSP16 Complex from SARS-CoV-2 | @@ -329,14 +337,14 @@ plate`), the comment's name where one is present (`Rayonix MX-300`), or the seri ## Deposited models and structure factors -166 of the 173 datasets have a released PDB entry, and RCSB +174 of the 181 datasets have a released PDB entry, and RCSB reports released structure factors (`status_code_sf = REL`) for every one of them. A merged result from this pipeline can therefore be checked against the deposited model or against the deposited intensities. ## Rows where our reduction and the deposition disagree -Six of the 173 rows are ones where `rugnux` does not reproduce the deposited space group or +Six of the 181 rows are ones where `rugnux` does not reproduce the deposited space group or cell, and where we have looked at the disagreement closely enough to change how the row is scored. They are collected here because a scoring row that silently disagrees with a published entry is not something a reader should have to discover from the code. @@ -503,17 +511,17 @@ symmetries rather than to be easy to process. The counts below describe where it like everything else on this page, they are metadata about the depositions and their files, not measurements. -- **Repository:** IRRMC 86, SBGrid 35, Zenodo 28, MXRDR 14, ESRF 3, Keele University 3, XRDa 3, +- **Repository:** IRRMC 94, SBGrid 35, Zenodo 28, MXRDR 14, ESRF 3, Keele University 3, XRDa 3, UQ eSpace 1. - **Facility** - counted from the facility part of the Facility / beamline column, the beamline ignored so that entries deposited with and without one count the same, over the 170 rows that name one: APS 26, Diamond 20, ESRF 16, NSLS-II 14, BESSY 12, PETRA III 12, SSRL 11, ALS 8, SLS 7, SOLEIL 7, SPring-8 7, SSRF 6, PAL/PLS 5, CHESS 4, CLSI 3, ALBA 2, Australian Synchrotron 2, ELETTRA 2, LNLS 2, and one each from MAX IV, NSRRC, Photon Factory and RRCAT - Indus-2 - 23 facilities. The other two rows were collected on laboratory sources, a rotating - anode and a liquid-metal jet. -- **Crystal system, from the deposited space group of the 166 PDB-coded rows:** orthorhombic 44, - monoclinic 40, tetragonal 22, trigonal 19, hexagonal 16, cubic 13, triclinic 12. + Indus-2 - 23 facilities. The other ten rows were collected on laboratory sources: nine on + rotating anodes and one on a liquid-metal jet. +- **Crystal system, from the deposited space group of the 174 PDB-coded rows:** orthorhombic 45, + monoclinic 41, tetragonal 25, trigonal 21, hexagonal 17, cubic 13, triclinic 12. - **Long cell axes:** eleven PDB-coded rows have a deposited cell axis longer than 320 Å - 8V4O, 9ZMU, 9Z72, 9YL4, 5NW5, 6QAJ, 7QIJ, 8T7R, 9H0Q, 6G1F and 6OEL. diff --git a/tools/battery/README.md b/tools/battery/README.md index d736be19c..5caefd8d1 100644 --- a/tools/battery/README.md +++ b/tools/battery/README.md @@ -15,7 +15,7 @@ run needed): `cd tools/battery && python3 test_score.py`. | arm | datasets | reference | manifest | |---|---|---|---| | **open** | public PDB depositions of raw diffraction data, plus a few published small-molecule sets | the deposited space group, cell and resolution | `open.json` (committed) | -| **inhouse** | standard test crystals measured at the SLS (lysozyme, thaumatin, insulin, cytochrome C, myoglobin), plus no-crystal controls | XDS, from the `CORRECT.LP` beside each dataset | `inhouse.json` (committed) | +| **inhouse** | standard test crystals measured at the SLS (lysozyme, thaumatin, insulin, cytochrome C, myoglobin), small-molecule standards (aspirin, citric acid, HEPES, YAG), plus no-crystal controls | XDS, from the `CORRECT.LP` beside each dataset; for the small molecules the space group is the literature's (`ref_override`), as XDS reports only Sohncke groups | `inhouse.json` (committed) | | **private** | user data | XDS, like inhouse | outside the repository; the local site config gives its path | Scoring checks these things in order, and the first one that fails decides the verdict: did it @@ -138,7 +138,7 @@ the copies. The paths inside the manifests do not change. [`docs/EXTERNAL_TEST_DATA.md`](../../docs/EXTERNAL_TEST_DATA.md). The sources are IRRMC (proteindiffraction.org), SBGrid Data Bank, Zenodo and a few others, and the page also says what each archive holds. Please cite those DOIs. At PSI the data root is `/home/data/open`, a - symlink to `/home/data/nonsls/raw`. 69 of the dataset directories in it are themselves symlinks + symlink to `/home/data/nonsls/raw`. 42 of the dataset directories in it are themselves symlinks into `/data/scout_staging2`, so **copy with symlinks followed**: `rsync -aL` or `cp -rL`. A plain `rsync -a` copies dangling links. The open arm is about 1.7 TB. - **inhouse**: these are our own measurements and cannot be downloaded publicly. Ask the diff --git a/tools/battery/inhouse.json b/tools/battery/inhouse.json index d893e0e06..0e6d086b8 100644 --- a/tools/battery/inhouse.json +++ b/tools/battery/inhouse.json @@ -37,5 +37,10 @@ {"id": "myob_x06da_split", "input": "myob_x06da_split/MyoB2-4_079740_master.h5", "ref": {"sgno": 3, "cell": [35.39, 28.783, 63.626, 90.0, 105.544, 90.0], "anomalous": false, "isa": 12.41, "completeness": 77.7, "r_meas": 0.577, "cc_half": 0.984, "multiplicity": 4.38, "dmin_low": 4.48, "r_meas_low": 0.091, "dmin": 1.506, "dmin_rule": "xds_range", "dmin_xds": 1.506, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, {"id": "myob_x06da_sparse", "input": "myob_x06da_sparse/MyoB2-5_7cef9c_master.h5", "ref": {"sgno": 3, "cell": [35.388, 28.752, 64.083, 90.0, 106.352, 90.0], "anomalous": false, "isa": 5.46, "completeness": 73.7, "r_meas": 0.326, "cc_half": 0.972, "multiplicity": 5.21, "dmin_low": 5.92, "r_meas_low": 0.146, "dmin": 2.0, "dmin_rule": "xds_range", "dmin_xds": 2.0, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, {"id": "nothing_2", "input": "nothing_2/test-28_3400ac_master.h5", "expect": "no_lattice", "tags": ["h5", "control"]}, - {"id": "nothing_1", "input": "nothing_1/test-28_9bf604_master.h5", "expect": "no_lattice", "tags": ["h5", "control"]} + {"id": "nothing_1", "input": "nothing_1/test-28_9bf604_master.h5", "expect": "no_lattice", "tags": ["h5", "control"]}, + {"id": "aspirin_x10sa_20keV", "input": "aspirin_x10sa_20keV/aspirin_1_002_master.h5", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.262, 6.543, 11.257, 90.0, 95.916, 90.0], "anomalous": true, "isa": 27.97, "completeness": 84.0, "r_meas": 0.032, "cc_half": 0.999, "multiplicity": 3.14, "dmin_low": 2.01, "r_meas_low": 0.03, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.55, "completeness": 86.6, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.03, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.55, "completeness": 81.0, "r_meas": 0.04, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.043, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, + {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.39, "completeness": 91.6, "r_meas": 0.03, "cc_half": 0.999, "multiplicity": 5.64, "dmin_low": 2.0, "r_meas_low": 0.035, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, + {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"} ]} diff --git a/tools/battery/open.json b/tools/battery/open.json index 7e44a26ee..374da4e60 100644 --- a/tools/battery/open.json +++ b/tools/battery/open.json @@ -3,6 +3,10 @@ {"id": "36gk", "input": "36gk/CLS-0074_5-3_36GK/data/CLS-0074_5-3_master.h5", "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [120.58, 189.49, 199.69, 90.0, 90.0, 90.0], "dmin": 2.28}, "tags": ["h5", "orthorhombic"]}, {"id": "3inp", "input": "3inp/IDP02542_3inp/data/idp02542b.001", "ref": {"sg": "F 41 3 2", "sgno": 210, "cell": [224.08, 224.08, 224.08, 90.0, 90.0, 90.0], "dmin": 2.05}, "tags": ["marCCD", "cubic"]}, {"id": "3ky7", "input": "3ky7/IDP90258_3ky7/data/idp90258f.001", "ref": {"sg": "P 43 3 2", "sgno": 212, "cell": [125.176, 125.176, 125.176, 90.0, 90.0, 90.0], "dmin": 2.35}, "tags": ["marCCD", "cubic"]}, + {"id": "3mc4", "input": "3mc4/series/206918e4_x0001.img", "ref": {"sg": "H 3", "sgno": 146, "cell": [104.03, 104.03, 105.54, 90.0, 90.0, 120.0], "dmin": 1.95}, "tags": ["smv", "trigonal", "home-source"]}, + {"id": "3meb", "input": "3meb/series/202097g3_x0001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [58.58, 101.15, 81.53, 90.0, 90.62, 90.0], "dmin": 1.9}, "tags": ["smv", "monoclinic", "home-source"]}, + {"id": "3p85", "input": "3p85/series/217175d10_x0001.img", "ref": {"sg": "P 63 2 2", "sgno": 182, "cell": [127.29, 127.29, 72.9, 90.0, 90.0, 120.0], "dmin": 1.9}, "tags": ["smv", "hexagonal", "home-source"]}, + {"id": "3r6o", "input": "3r6o/series/219594b8_x0001.img", "ref": {"sg": "I 41", "sgno": 80, "cell": [90.68, 90.68, 76.13, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["smv", "tetragonal", "home-source"]}, {"id": "5cc8", "input": "5cc8/data/263060g10_x0001.img", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [87.14, 93.76, 72.49, 90.0, 90.0, 90.0], "dmin": 1.75}, "tags": ["smv", "orthorhombic", "home-source", "tncs"]}, {"id": "5ebi", "input": "5ebi/dna-rna-chimera_Ba_high_2_001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [35.72, 44.1, 35.72, 90.0, 119.98, 90.0], "dmin": 1.09}, "pinned": true, "tags": ["marCCD", "monoclinic", "twin"]}, {"id": "5epe", "input": "5epe/030805_5epe/data/E1_7_set.001", "ref": {"sg": "F 2 3", "sgno": 196, "cell": [157.536, 157.536, 157.536, 90.0, 90.0, 90.0], "dmin": 1.9}, "tags": ["marCCD", "cubic"]}, @@ -14,6 +18,9 @@ {"id": "5nw5", "input": "5nw5/5NW5_1_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [92.14, 169.8, 390.16, 90.0, 90.0, 90.0], "dmin": 6.502}, "tags": ["cbf", "orthorhombic", "low-resolution"]}, {"id": "5reo", "input": "5reo/cbf/Mpro-x0752_1_0001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [112.39, 52.59, 44.38, 90.0, 103.04, 90.0], "dmin": 1.88}, "tags": ["cbf", "monoclinic"], "tiers": {"smoke": "fastest set (10 s): miniCBF, monoclinic"}}, {"id": "5src", "input": "5src/5src/data/FRS004_15_1_00001.cbf", "ref": {"sg": "P 43", "sgno": 78, "cell": [88.68, 88.68, 39.23, 90.0, 90.0, 90.0], "dmin": 1.05}, "tags": ["cbf", "tetragonal"]}, + {"id": "5uth", "input": "5uth/data/287007e1_0001.img", "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [69.28, 69.28, 153.8, 90.0, 90.0, 120.0], "dmin": 1.95}, "tags": ["smv", "trigonal", "home-source"]}, + {"id": "5vml", "input": "5vml/data/271705c8_x_0001.img", "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [66.31, 66.31, 115.26, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["smv", "tetragonal", "home-source"]}, + {"id": "6cee", "input": "6cee/data/Rachel_AV6_screen_0001.img", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [40.72, 44.11, 55.91, 90.0, 90.0, 90.0], "dmin": 1.55}, "tags": ["smv", "orthorhombic", "home-source"]}, {"id": "6fid", "input": "6fid/Trypsin_x1_align_2_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [59.947, 64.107, 69.694, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["cbf", "orthorhombic"]}, {"id": "6fvz", "input": "6fvz/maox215_6fvz/data/maox215_w1_1_0001.cbf", "ref": {"sg": "C 2 2 2", "sgno": 21, "cell": [131.182, 222.753, 86.482, 90.0, 90.0, 90.0], "dmin": 1.8}, "tags": ["cbf", "orthorhombic"]}, {"id": "6fwc", "input": "6fwc/maox225_6fwc/data/maox225_1_00001.cbf", "ref": {"sg": "C 2 2 2", "sgno": 21, "cell": [131.728, 222.051, 86.293, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["cbf", "orthorhombic"]}, @@ -42,6 +49,7 @@ {"id": "6ttn", "input": "6ttn/6ttn/data/H6H_33_01_hyo_full_1_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [39.894, 79.841, 104.701, 90.0, 90.0, 90.0], "dmin": 1.12}, "tags": ["cbf", "orthorhombic"]}, {"id": "6u7g", "input": "6u7g/6u7g/data/SNB02_11_8_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [99.555, 98.682, 147.525, 90.0, 104.6, 90.0], "dmin": 2.35}, "pinned": true, "tags": ["h5", "monoclinic"]}, {"id": "6ukf", "input": "6ukf/6ukf/data/XDC-7_Pn6_000001.cbf", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [61.018, 37.317, 69.027, 90.0, 109.768, 90.0], "dmin": 1.0}, "tags": ["cbf", "monoclinic"]}, + {"id": "6v2r", "input": "6v2r/data/cbx7.248483.hv6_screen_0001.img", "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [40.193, 40.193, 83.127, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["smv", "tetragonal", "home-source"]}, {"id": "6vww", "input": "6vww/IDP51000_6vww/data/m4H11g_00001.cbf", "ref": {"sg": "P 63", "sgno": 173, "cell": [150.539, 150.539, 111.31, 90.0, 90.0, 120.0], "dmin": 2.2}, "tags": ["cbf", "hexagonal", "twin"], "tiers": {"smoke": "merohedral twin, P63"}}, {"id": "6w4h", "input": "6w4h/IDP51000_6W4H/data/idp51000-201-a_1_2_3.001", "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [167.74, 167.74, 51.942, 90.0, 90.0, 120.0], "dmin": 1.8}, "tags": ["marCCD", "trigonal"]}, {"id": "6wzo", "input": "6wzo/9_1_1_000001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [43.718, 50.061, 69.337, 106.499, 90.094, 97.145], "dmin": 1.42}, "tags": ["cbf", "triclinic"]}, -- 2.54.0 From f06780bd9174d17773e8e386c639377169cbf497 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 2 Oct 2026 16:06:13 +0200 Subject: [PATCH 03/97] Battery: record each set's wavelength; add the rc173 battery report to docs Every open and in-house manifest row now carries `wavelength` (A), taken from WAVELENGTH in rugnux's own p_report.txt of the latest run of that set (the full rc173 run 20260929-2003 plus the two 10-02 runs for the sets added since). Descriptive only; the scorer ignores it. docs/BATTERY_REPORT.md is report.md of the full rc173 open + in-house run (210 sets), verbatim, to be polished. The private arm runs separately and has its own report, so nothing from it appears here. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/BATTERY_REPORT.md | 784 +++++++++++++++++++++++++++++++++++++ tools/battery/README.md | 2 + tools/battery/inhouse.json | 88 ++--- tools/battery/open.json | 362 ++++++++--------- 4 files changed, 1011 insertions(+), 225 deletions(-) create mode 100644 docs/BATTERY_REPORT.md diff --git a/docs/BATTERY_REPORT.md b/docs/BATTERY_REPORT.md new file mode 100644 index 000000000..3aa674394 --- /dev/null +++ b/docs/BATTERY_REPORT.md @@ -0,0 +1,784 @@ +# Rugnux battery - 20260929-2003_cca7bf_r12-rc173-refmac + +- rugnux 1.0.0-rc.173, sha256 51711e68a4feef9c, build flags: unknown +- runner cca7bf955, host mpc2898.psi.ch, 2026-09-29T20:03:52 -> 2026-09-30T05:02:18 +- arms: open, inhouse; 210 sets (full arm) +- timing: GPU was not shared during the run +- command options: --export-unmerged +- command, open arm: `rugnux --model ` (without --model where there is no deposited model: small molecules, unpublished sets) +- command, inhouse arm: `rugnux --report-resolution , `: the second statistics table at XDS's range is report-only; where XDS kept Friedel mates apart the scorer reads the run's own per-hand table, which needs no flag + +## Summary + +| arm | sets | pass | of which alt | fail | unscored | not run | pass rate | median res. gain | median ISa | median R_meas | median time s | total time min | +|:--|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:| +| open | 171 | 166 | 5 | 3 | 2 | 0 | 166/169 (98%) | +12.0% | 13.3 | 15.0% | 44 | 166.7 | +| inhouse | 39 | 38 | 0 | 1 | 0 | 0 | 38/39 (97%) | +7.1% | 20.3 | 12.8% | 17 | 12.9 | + +_Plot in report.html: verdicts per arm_ + +Pass rate is over the scored sets (pass + fail). `of which alt` counts the passes that are already in `pass`: rows where the answer matched an accepted alternative reference rather than the deposition, listed under Accepted alternatives below. In the bar chart they are drawn as their own segment. Every number in this table is rugnux's own run at its own resolution cut: what a user gets. Resolution gain is (reference d_min - our d_min) / reference d_min: positive = finer than the deposition, or than XDS. Medians leave out the no-crystal controls. R_meas and CC1/2 are pooled over each run's own resolution range, so they describe a run and do not rank two; the like-for-like comparison with XDS is the reference-range table below. Each set runs once, so every time includes reading its images from disk (unless they were still in the page cache from before the run). + +### By population + +| arm | population | sets | pass rate | median res. gain | +|:--|:--|--:|:--|--:| +| open | SMV | 2 | 2/2 (100%) | +11.2% | +| open | cbf | 88 | 85/87 (98%) | +11.6% | +| open | cdte | 3 | 3/3 (100%) | +4.8% | +| open | cubic | 13 | 13/13 (100%) | +14.3% | +| open | diffuse | 4 | 4/4 (100%) | +9.9% | +| open | h5 | 55 | 54/55 (98%) | +12.1% | +| open | hexagonal | 15 | 14/14 (100%) | +13.6% | +| open | long-axis | 2 | 2/2 (100%) | +16.6% | +| open | long-wavelength | 1 | 1/1 (100%) | - | +| open | low-resolution | 1 | 1/1 (100%) | -9.2% | +| open | marCCD | 14 | 14/14 (100%) | +12.2% | +| open | marccd | 8 | 8/8 (100%) | +11.1% | +| open | monoclinic | 37 | 37/37 (100%) | +12.1% | +| open | nxs | 1 | - | - | +| open | orthorhombic | 41 | 39/41 (95%) | +12.1% | +| open | small-molecule | 6 | 5/5 (100%) | -12.9% | +| open | smv | 3 | 3/3 (100%) | +7.3% | +| open | tetragonal | 17 | 17/17 (100%) | +9.1% | +| open | tncs | 4 | 3/4 (75%) | +10.6% | +| open | triclinic | 12 | 12/12 (100%) | +10.7% | +| open | trigonal | 16 | 16/16 (100%) | +13.7% | +| open | twin | 9 | 9/9 (100%) | +9.1% | +| open | two-wavelength | 2 | 2/2 (100%) | +10.8% | +| inhouse | control | 2 | 2/2 (100%) | - | +| inhouse | cytochrome-c | 3 | 3/3 (100%) | +9.2% | +| inhouse | h5 | 39 | 38/39 (97%) | +7.1% | +| inhouse | insulin | 9 | 9/9 (100%) | +4.9% | +| inhouse | iodine | 2 | 2/2 (100%) | +15.9% | +| inhouse | long-wavelength | 7 | 7/7 (100%) | +1.1% | +| inhouse | lysozyme | 12 | 12/12 (100%) | +7.5% | +| inhouse | myoglobin | 6 | 5/6 (83%) | +7.5% | +| inhouse | pink-beam | 3 | 3/3 (100%) | +13.2% | +| inhouse | thaumatin | 7 | 7/7 (100%) | +2.7% | +| inhouse | twin | 4 | 3/4 (75%) | -7.3% | + +### Distributions (sets that merged, controls left out) + +| arm | metric | n | min | q1 | median | q3 | max | +|:--|:--|--:|--:|--:|--:|--:|--:| +| open | ISa | 171 | 1.4 | 9.4 | 13.3 | 17.9 | 37.8 | +| open | R_meas | 171 | 3.2% | 9.8% | 15.0% | 22.7% | 269.6% | +| open | CC1/2 | 171 | 0.882 | 0.994 | 0.997 | 0.999 | 1.000 | +| open | res. gain % | 167 | -68.3 | +8.0 | +12.0 | +17.1 | +34.3 | +| open | low-res shell R_meas | 171 | 2.5% | 4.5% | 6.0% | 8.5% | 44.3% | +| open | R_model, shell-scaled | 165 | 0.099 | 0.179 | 0.210 | 0.247 | 0.576 | +| open | R_free (placement, within-run only) | 165 | 0.121 | 0.189 | 0.221 | 0.257 | 0.578 | +| open | radial misfit | 165 | 0.03 | 0.09 | 0.14 | 0.18 | 0.58 | +| open | R_free ratio | 165 | 0.811 | 0.968 | 1.048 | 1.142 | 3.168 | +| open | REFMAC R_free ratio | 161 | 0.840 | 0.984 | 1.002 | 1.026 | 1.903 | +| open | dCC vs depositor, all | 162 | -0.309 | -0.015 | -0.003 | +0.004 | +0.192 | +| open | dCC vs depositor, outer 2 shells | 162 | -0.625 | -0.030 | -0.005 | +0.033 | +0.307 | +| open | CC(Fc^2) past the deposited limit | 150 | -0.011 | 0.231 | 0.304 | 0.360 | 0.809 | +| open | time s | 171 | 8 | 26 | 44 | 73 | 355 | +| inhouse | ISa | 35 | 2.4 | 7.3 | 20.3 | 29.4 | 52.5 | +| inhouse | R_meas | 37 | 3.9% | 7.0% | 12.8% | 23.0% | 151.9% | +| inhouse | CC1/2 | 37 | 0.525 | 0.997 | 0.999 | 1.000 | 1.000 | +| inhouse | res. gain % | 37 | -42.2 | +2.0 | +7.1 | +10.6 | +19.9 | +| inhouse | ref-range ISa / XDS | 37 | 0.247 | 0.789 | 1.007 | 1.233 | 3.328 | +| inhouse | ref-range R_meas / XDS | 37 | 0.417 | 0.842 | 0.968 | 1.094 | 2.143 | +| inhouse | low-res shell R_meas | 37 | 2.5% | 4.0% | 5.0% | 8.9% | 48.4% | +| inhouse | ref-range low-res R_meas / XDS | 37 | 0.409 | 0.936 | 1.083 | 1.174 | 2.264 | +| inhouse | ref-range CC1/2 noise / XDS | 37 | 0.20 | 0.35 | 0.56 | 0.93 | 53.82 | +| inhouse | time s | 37 | 8 | 11 | 17 | 20 | 47 | + +### Per set, against the reference + +_Plot in report.html: open: d_min(rugnux, own cut) / d_min(reference), one point per set, sorted; below 1 = finer than the reference_ + +_Plot in report.html: inhouse: d_min(rugnux, own cut) / d_min(reference), one point per set, sorted; below 1 = finer than the reference_ + +_Plot in report.html: inhouse: ISa of the reference-range table (the error model refitted on it) / XDS's ISa; above 1 = rugnux's error model is the better one. One point per set, sorted._ + +_Plot in report.html: inhouse: R_meas over the reference range / XDS's R_meas; below 1 = rugnux's merge is the more consistent one. One point per set, sorted._ + +_Plot in report.html: inhouse: R_meas of the lowest-resolution shell of the reference-range table / of XDS's table; below 1 = rugnux's strong reflections agree better. One point per set, sorted._ + +_Plot in report.html: inhouse: half-set noise over the reference range read off CC1/2 (1 / CC1/2 - 1) / XDS's; 2 = as noisy as XDS's merge would be with half its observations (not scored). One point per set, sorted._ + +_Plot in report.html: R_free of the deposited model against our merge, as rugnux reports it with --model (a rigid-body placement, scored on rugnux's own free set: a trend number, not a refinement) / the R_free the depositor published, one point per set, sorted; below 1 = lower than published_ + +_Plot in report.html: REFMAC check (--model-check): R_free of the deposited model against our merge / against the deposited structure factors, both on the depositor's free set with the same protocol, one point per set, sorted; below 1 = our data fit the model better_ + +### Against the depositor's data + +Per resolution shell, the rank correlation of our merged intensities with |Fc|^2 minus that of the depositor's own data, on the reflections both carry (d < 4 A, eight shells of equal count), |Fc| from the deposited model as it is (no bulk solvent, no refinement; depdata_check.py). The model was refined against the depositor's data, so zero is already a good result: across a corpus the median sits near -0.01. 'Past the limit' is CC(I, |Fc|^2) of our reflections in the outermost of up to three shells beyond the deposited data's limit, which the model never saw: clearly above zero there is signal. Reported, never scored. + +| set | dep. data | our d_min | dep. d_min | common refl. | dCC all | dCC outer 2 | CC past limit | to d A | REFMAC ratio | ISa | tags | +|:--|:--|--:|--:|--:|--:|--:|--:|--:|--:|--:|:--| +| 9hnc | F | 1.65 | 1.88 | 405160 | -0.309 | -0.625 | 0.150 | 1.65 | 1.903 | 1.4 | cbf, monoclinic | +| 6zqy | F | 1.72 | 1.85 | 47652 | -0.072 | -0.307 | 0.160 | 1.72 | 1.128 | 13.8 | smv | +| 9rcs | I | 3.37 | 3.01 | 1884 | -0.241 | -0.273 | - | - | 0.996 | 6.0 | h5, monoclinic, cdte | +| 6r72 | F | 4.39 | 3.95 | 18986 | -0.102 | -0.244 | - | - | 1.048 | 15.7 | h5, monoclinic | +| 6yqf | I | 3.05 | 3.33 | 932 | -0.129 | -0.233 | 0.018 | 3.04 | 1.047 | 3.9 | cbf, orthorhombic | +| 5ky6 | F | 1.55 | 1.94 | 98442 | -0.076 | -0.148 | 0.159 | 1.55 | 1.039 | 7.2 | marccd | +| 7arr | I | 0.92 | 1.10 | 47882 | -0.044 | -0.147 | 0.234 | 0.92 | 1.021 | 17.3 | cbf, triclinic | +| 6rlr | I | 2.07 | 2.00 | 17933 | -0.027 | -0.146 | - | - | 1.035 | 13.2 | h5, triclinic, twin | +| 6zqr | F | 1.79 | 1.93 | 37169 | -0.068 | -0.143 | 0.307 | 1.78 | 1.052 | 8.7 | smv | +| 8r5r | F | 2.89 | 3.08 | 12960 | -0.087 | -0.134 | 0.220 | 2.89 | 1.052 | 15.1 | h5, orthorhombic | +| 9pbb | I | 1.83 | 2.16 | 11265 | -0.050 | -0.129 | 0.148 | 1.83 | 0.984 | 14.5 | cbf, monoclinic | +| 9b22 | I | 1.18 | 1.30 | 96597 | -0.025 | -0.122 | 0.311 | 1.18 | 1.032 | 16.3 | h5, monoclinic | +| 6u7g | I | 1.92 | 2.35 | 87246 | -0.062 | -0.121 | 0.214 | 1.92 | 0.976 | 13.2 | h5, monoclinic | +| 8qq7 | I | 3.16 | 3.62 | 5507 | -0.104 | -0.111 | -0.011 | 3.15 | 1.060 | 6.3 | cbf, hexagonal | +| 8k1g | F | 1.63 | 2.09 | 34025 | -0.077 | -0.107 | 0.283 | 1.62 | 0.975 | 11.6 | cbf, tetragonal | +| 7tcd | I | 1.72 | 1.70 | 27128 | -0.053 | -0.092 | - | - | 0.965 | 15.0 | h5, monoclinic | +| 5epe | F | 1.78 | 1.90 | 22792 | -0.019 | -0.090 | 0.372 | 1.78 | 1.010 | 10.0 | marCCD, cubic | +| 9gjx | I | 2.15 | 2.40 | 52052 | -0.014 | -0.085 | 0.235 | 2.15 | 1.054 | 35.2 | h5, monoclinic | +| 6cdl | I | 1.15 | 1.24 | 56397 | -0.018 | -0.085 | 0.316 | 1.15 | 1.072 | 11.2 | marccd | +| 9hs7 | F | 1.68 | 1.70 | 9595 | -0.132 | -0.080 | 0.066 | 1.68 | 0.979 | 13.3 | cbf, hexagonal | +| 5f6m | I | 1.09 | 1.10 | 68757 | -0.008 | -0.074 | 0.349 | 1.09 | 1.051 | 22.2 | cbf, orthorhombic, diffuse | +| 6jgh | I | 0.87 | 0.94 | 139806 | -0.018 | -0.073 | 0.338 | 0.87 | 1.111 | 6.7 | marccd | +| 6qaj | I | 2.70 | 2.63 | 4165 | -0.132 | -0.071 | - | - | 0.971 | 13.1 | cbf | +| 5t39 | I | 1.01 | 1.10 | 92596 | -0.009 | -0.064 | 0.371 | 1.01 | 1.018 | 16.0 | marccd | +| 9p7q | I | 1.81 | 2.21 | 11919 | -0.024 | -0.062 | 0.246 | 1.81 | 0.997 | 10.6 | cbf, monoclinic | +| 7brr | I | 1.27 | 1.40 | 102801 | -0.027 | -0.059 | 0.266 | 1.27 | 0.989 | 15.8 | h5, monoclinic | +| 7os3 | I | 1.96 | 2.18 | 32881 | -0.002 | -0.055 | 0.341 | 1.96 | 0.989 | 24.1 | cbf, orthorhombic | +| 9upt | F | 2.03 | 2.37 | 24478 | -0.042 | -0.051 | 0.327 | 2.03 | 1.064 | 6.4 | SMV, hexagonal | +| 6vww | F | 2.00 | 2.20 | 58979 | -0.025 | -0.051 | 0.333 | 2.00 | 1.023 | 8.6 | cbf, hexagonal, twin | +| 8y74 | F | 1.71 | 1.90 | 58040 | -0.009 | -0.048 | 0.283 | 1.71 | 0.957 | 8.8 | h5, monoclinic | +| 9fhc | F | 1.97 | 2.20 | 76515 | -0.009 | -0.047 | 0.385 | 1.97 | 1.055 | 11.6 | marCCD, cubic | +| 7atg | I | 0.60 | 0.60 | 54555 | -0.007 | -0.046 | - | - | - | 22.5 | cbf, orthorhombic | +| 8dyz | I | 1.14 | 1.27 | 24755 | -0.007 | -0.045 | 0.640 | 1.14 | 0.959 | 37.3 | cbf, tetragonal, diffuse | +| 6z8o | F | 2.23 | 2.20 | 22415 | -0.131 | -0.044 | - | - | 1.044 | 13.8 | h5, monoclinic | +| 6zr0 | F | 1.73 | 1.94 | 40650 | -0.013 | -0.044 | 0.323 | 1.72 | 1.073 | 24.3 | cbf | +| 9zlo | I | 1.58 | 2.00 | 22408 | -0.018 | -0.040 | 0.237 | 1.58 | 1.007 | 23.0 | h5, orthorhombic | +| 5m17 | I | 0.98 | 1.03 | 185105 | -0.006 | -0.036 | 0.303 | 0.98 | 1.027 | 16.0 | cbf, tetragonal | +| 5reo | F | 1.65 | 1.88 | 18392 | -0.026 | -0.036 | 0.338 | 1.65 | 0.991 | 24.0 | cbf, monoclinic | +| 7raa | I | 2.59 | 2.69 | 7523 | -0.065 | -0.035 | 0.056 | 2.59 | 0.993 | 11.0 | cbf | +| 9gdj | I | 1.40 | 1.47 | 157444 | -0.008 | -0.032 | 0.304 | 1.40 | 1.045 | 12.8 | h5, tetragonal, cdte | +| 6s1u | I | 1.75 | 1.90 | 17749 | -0.002 | -0.029 | 0.334 | 1.75 | 0.978 | 13.7 | marccd | +| 9h0q | I | 2.13 | 2.55 | 45699 | -0.010 | -0.029 | 0.328 | 2.14 | 1.043 | 18.7 | h5 | +| 8a1a | F | 1.95 | 2.05 | 101603 | -0.005 | -0.028 | 0.340 | 1.95 | 0.994 | 17.9 | h5, hexagonal | +| 6iu8 | I | 2.33 | 2.70 | 15286 | +0.003 | -0.028 | 0.175 | 2.33 | 1.013 | 8.1 | cbf, trigonal | +| 9i80 | I | 1.61 | 1.95 | 68449 | -0.009 | -0.026 | 0.312 | 1.61 | 1.025 | 6.7 | h5, tetragonal, twin | +| 7mzt | I | 3.25 | 4.05 | 8480 | -0.042 | -0.026 | 0.067 | 3.25 | 1.021 | 3.4 | cbf, orthorhombic | +| 5lzl | F | 2.86 | 3.47 | 21840 | -0.014 | -0.025 | 0.231 | 2.87 | 1.027 | 15.0 | cbf, trigonal | +| 9ih9 | F | 1.41 | 1.70 | 78426 | -0.006 | -0.025 | 0.391 | 1.41 | 1.014 | 12.6 | h5, monoclinic | +| 6ze4 | I | 1.30 | 1.60 | 136771 | -0.054 | -0.025 | 0.329 | 1.30 | 1.060 | 9.2 | cbf, orthorhombic | +| 6fid | F | 1.98 | 2.20 | 10527 | -0.009 | -0.024 | 0.796 | 1.97 | 0.982 | 13.2 | cbf, orthorhombic | +| 6oel | I | 2.85 | 3.10 | 13233 | -0.013 | -0.024 | 0.272 | 2.85 | 0.965 | 9.9 | SMV, cubic | +| 9yl4 | I | 3.61 | 3.70 | 1130 | -0.010 | -0.024 | 0.176 | 3.61 | 0.987 | 9.5 | cbf, orthorhombic | +| 6hv2 | F | 1.42 | 1.71 | 19192 | -0.035 | -0.023 | 0.092 | 1.42 | 0.840 | 13.6 | h5, hexagonal | +| 9zm0 | I | 1.81 | 2.10 | 13858 | -0.008 | -0.023 | 0.104 | 1.81 | 0.996 | 8.6 | h5, monoclinic | +| 8agq | I | 0.97 | 1.09 | 98576 | -0.013 | -0.022 | 0.298 | 0.97 | 1.148 | 14.0 | cbf, monoclinic | +| 6ttn | I | 1.08 | 1.12 | 126153 | -0.001 | -0.021 | 0.218 | 1.08 | 1.026 | 14.5 | cbf, orthorhombic | +| 9q66 | F | 2.04 | 2.01 | 87899 | -0.017 | -0.021 | - | - | 1.016 | 13.0 | h5, monoclinic | +| 6iu6 | I | 2.38 | 2.90 | 10758 | -0.001 | -0.021 | 0.254 | 2.38 | 1.045 | 7.0 | cbf, trigonal, twin | +| 7kcn | I | 1.39 | 1.46 | 43289 | -0.011 | -0.020 | 0.568 | 1.39 | 1.054 | 11.7 | cbf, tetragonal | +| 9jzo | F | 1.15 | 1.40 | 47932 | -0.003 | -0.020 | 0.725 | 1.15 | 1.015 | 8.4 | cbf, triclinic | +| 9crw | I | 2.29 | 2.49 | 53873 | -0.021 | -0.019 | 0.127 | 2.29 | 1.018 | 15.5 | h5, monoclinic | +| 6w75 | F | 1.68 | 1.95 | 99979 | -0.008 | -0.019 | 0.311 | 1.68 | 0.991 | 15.8 | marccd | +| 7n2s | I | 2.57 | 2.37 | 10754 | -0.002 | -0.019 | - | - | 0.991 | 9.6 | cbf, monoclinic | +| 6fvz | F | 1.49 | 1.80 | 105993 | -0.019 | -0.018 | 0.386 | 1.50 | 1.001 | 20.6 | cbf, orthorhombic | +| 6iu5 | I | 2.12 | 2.25 | 30820 | -0.010 | -0.018 | 0.229 | 2.12 | 1.008 | 7.9 | cbf, trigonal, twin | +| 9vyb | I | 1.71 | 2.12 | 5191 | -0.010 | -0.017 | 0.309 | 1.71 | 0.997 | 16.7 | h5, orthorhombic | +| 6h5t | F | 1.48 | 1.69 | 28184 | -0.003 | -0.015 | 0.361 | 1.48 | 1.009 | 9.2 | marCCD, tetragonal | +| 6p8p | I | 1.47 | 1.64 | 64791 | -0.012 | -0.015 | 0.306 | 1.47 | 1.047 | 15.3 | cbf, tetragonal | +| 9c18 | I | 1.76 | 1.90 | 25042 | +0.014 | -0.015 | 0.291 | 1.76 | 0.976 | 10.2 | h5, triclinic | +| 9e2t | I | 2.33 | 2.28 | 31985 | -0.095 | -0.015 | - | - | 0.990 | 6.9 | cbf, triclinic | +| 6w4h | F | 1.62 | 1.80 | 70331 | -0.003 | -0.013 | 0.418 | 1.62 | 0.994 | 18.4 | marCCD, trigonal | +| 9chw | I | 1.60 | 2.16 | 20358 | -0.006 | -0.012 | 0.641 | 1.60 | 1.005 | 20.9 | marCCD, hexagonal | +| 9z44 | I | 6.86 | 7.20 | 1893 | +0.005 | -0.009 | 0.114 | 6.94 | 1.039 | 5.9 | cbf, monoclinic | +| 8dz7 | I | 1.19 | 1.34 | 13603 | -0.002 | -0.008 | 0.809 | 1.19 | 1.015 | 33.7 | cbf, orthorhombic, diffuse | +| 6h2p_1p89A | F | 1.76 | 1.88 | 85477 | +0.000 | -0.008 | 0.481 | 1.76 | 1.059 | 22.5 | cbf, orthorhombic, long-wavelength, two-wavelength | +| 7l84 | I | 1.70 | 1.71 | 11375 | -0.001 | -0.008 | - | - | 0.979 | 11.2 | cbf, tetragonal | +| 6i3j | I | 2.32 | 2.59 | 34974 | -0.008 | -0.007 | 0.356 | 2.32 | 1.141 | 7.0 | marCCD, orthorhombic | +| 9sl0 | I | 1.38 | 1.60 | 67358 | +0.004 | -0.007 | 0.237 | 1.38 | 1.003 | 20.0 | h5, orthorhombic | +| 5mln | F | 1.26 | 1.59 | 60800 | -0.012 | -0.007 | 0.409 | 1.26 | 1.026 | 21.4 | cbf | +| 6wzo | I | 1.06 | 1.42 | 91666 | -0.004 | -0.006 | 0.303 | 1.06 | 0.976 | 16.9 | cbf, triclinic | +| 6o2h | I | 1.10 | 1.21 | 11613 | -0.003 | -0.005 | 0.747 | 1.10 | 1.162 | 20.8 | cbf, triclinic, diffuse | +| 6pxb | I | 1.39 | 1.75 | 50825 | +0.008 | -0.004 | 0.100 | 1.39 | 0.990 | 12.0 | cbf, trigonal | +| 7ris | I | 1.51 | 1.72 | 22161 | -0.005 | -0.004 | 0.220 | 1.51 | 0.984 | 30.2 | h5, trigonal | +| 8xtf | I | 1.84 | 2.13 | 27230 | +0.000 | -0.003 | 0.425 | 1.84 | 0.956 | 7.5 | h5, trigonal | +| 9qw8 | I | 1.59 | 1.80 | 38509 | -0.003 | -0.002 | 0.356 | 1.58 | 1.022 | 11.3 | h5, triclinic | +| 7l6j | F | 1.51 | 1.78 | 37485 | -0.002 | -0.001 | 0.414 | 1.51 | 1.013 | 10.5 | marCCD, cubic | +| 7bgt | I | 1.77 | 1.93 | 33462 | -0.005 | -0.000 | 0.286 | 1.78 | 0.997 | 14.7 | marCCD, triclinic | +| 8iya | F | 1.94 | 2.43 | 16262 | +0.003 | +0.001 | 0.199 | 1.94 | 0.979 | 5.8 | h5, monoclinic | +| 6f3p | F | 1.13 | 1.35 | 235840 | -0.001 | +0.001 | 0.392 | 1.13 | 1.013 | 9.4 | marccd | +| 8egn | F | 1.64 | 1.95 | 38630 | -0.005 | +0.002 | 0.269 | 1.64 | 0.999 | 22.7 | cbf, orthorhombic | +| 3ky7 | F | 1.92 | 2.35 | 11401 | +0.002 | +0.002 | 0.182 | 1.92 | 0.997 | 12.3 | marCCD, cubic | +| 8tyy | I | 1.28 | 1.68 | 44790 | -0.001 | +0.002 | 0.476 | 1.28 | 0.990 | 16.3 | cbf, cubic | +| 6nen | I | 1.72 | 2.15 | 10259 | +0.003 | +0.002 | 0.270 | 1.72 | 1.004 | 6.3 | smv | +| 3inp | I | 1.70 | 2.05 | 26251 | -0.005 | +0.003 | 0.277 | 1.70 | 0.984 | 13.3 | marCCD, cubic | +| 8sqt | I | 1.89 | 2.20 | 9296 | -0.009 | +0.003 | 0.302 | 1.89 | 1.042 | 29.6 | h5, cubic | +| 9mh4 | I | 2.78 | 3.05 | 9490 | +0.006 | +0.005 | 0.205 | 2.79 | 1.017 | 13.8 | h5, cubic | +| 11if | I | 1.38 | 1.51 | 27273 | +0.001 | +0.006 | 0.269 | 1.38 | 0.998 | 25.5 | h5, tetragonal | +| 9gqg | I | 1.82 | 2.00 | 15522 | -0.006 | +0.007 | 0.232 | 1.82 | 1.026 | 13.2 | h5, trigonal | +| 8dqb | I | 2.06 | 2.50 | 19165 | +0.001 | +0.008 | 0.381 | 2.06 | 1.014 | 22.3 | h5, cubic | +| 5ebi | F | 0.90 | 1.09 | 39152 | +0.006 | +0.008 | 0.232 | 0.90 | 0.981 | 14.4 | marCCD, monoclinic, twin | +| 6ukf | I | 0.95 | 1.00 | 143274 | +0.002 | +0.011 | 0.263 | 0.95 | 0.958 | 9.4 | cbf, monoclinic | +| 7yzx | F | 1.88 | 1.90 | 83105 | -0.030 | +0.012 | 0.324 | 1.88 | 1.024 | 10.4 | cbf, hexagonal | +| 9zmu | I | 1.71 | 1.98 | 21684 | -0.001 | +0.013 | 0.183 | 1.71 | 0.995 | 10.8 | h5, hexagonal | +| 9ig7 | F | 2.03 | 2.60 | 26626 | -0.002 | +0.014 | 0.266 | 2.03 | 1.036 | 11.0 | cbf, orthorhombic | +| 9bn8 | I | 1.20 | 1.35 | 119114 | +0.001 | +0.015 | 0.445 | 1.20 | 1.002 | 19.8 | h5, tetragonal | +| 8u0i | I | 1.40 | 1.54 | 16737 | +0.009 | +0.016 | 0.344 | 1.40 | 1.004 | 16.1 | cbf, tetragonal | +| 6g1f | I | 1.93 | 2.25 | 131738 | +0.001 | +0.016 | 0.238 | 1.93 | 1.017 | 21.2 | cbf | +| 6fwc | F | 1.41 | 1.70 | 126460 | -0.001 | +0.017 | 0.394 | 1.41 | 0.979 | 27.2 | cbf, orthorhombic | +| 8sqq | I | 1.90 | 2.25 | 8697 | +0.006 | +0.018 | 0.331 | 1.91 | 0.940 | 17.6 | h5, cubic | +| 8qaw | F | 1.30 | 1.55 | 256134 | +0.001 | +0.021 | 0.354 | 1.30 | 0.962 | 10.9 | cbf, trigonal, long-axis | +| 36gk | F | 2.09 | 2.28 | 84276 | -0.011 | +0.024 | 0.311 | 2.09 | 1.014 | 11.8 | h5, orthorhombic | +| 7dkp | F | 1.18 | 1.45 | 131355 | +0.006 | +0.025 | 0.676 | 1.18 | 0.986 | 26.3 | h5, monoclinic | +| 7t5t | I | 1.24 | 1.35 | 101432 | -0.002 | +0.025 | 0.251 | 1.24 | 1.017 | 16.5 | cbf, tetragonal | +| 5nw5 | I | 7.10 | 6.50 | 9711 | -0.039 | +0.026 | - | - | 1.082 | 8.8 | cbf, orthorhombic, low-resolution | +| 6jgj | F | 0.65 | 0.77 | 250748 | -0.050 | +0.026 | 0.390 | 0.65 | 1.131 | 14.7 | cbf, orthorhombic, cdte | +| 6pxc | F | 1.43 | 1.60 | 15638 | +0.013 | +0.028 | 0.330 | 1.43 | 1.028 | 10.1 | cbf, orthorhombic | +| 6moj | I | 2.43 | 2.43 | 13971 | -0.050 | +0.029 | 0.214 | 2.43 | 0.992 | 8.3 | cbf, tetragonal | +| 8sqo | I | 1.33 | 1.55 | 33951 | +0.003 | +0.029 | 0.356 | 1.33 | 1.008 | 14.2 | h5, cubic | +| 8s38 | F | 1.63 | 1.89 | 121216 | -0.003 | +0.030 | 0.356 | 1.63 | 1.008 | 22.3 | cbf, orthorhombic | +| 6pb3 | I | 1.85 | 2.05 | 15267 | -0.001 | +0.033 | 0.226 | 1.85 | 1.010 | 24.6 | cbf, hexagonal | +| 8qj5 | I | 1.35 | 1.63 | 101764 | +0.004 | +0.033 | 0.316 | 1.36 | 1.058 | 8.4 | cbf, monoclinic | +| 9z72 | I | 1.97 | 2.38 | 28228 | +0.009 | +0.033 | 0.285 | 1.97 | 0.993 | 11.8 | cbf, trigonal, long-axis | +| 8pqd | F | 1.30 | 1.50 | 102732 | +0.004 | +0.034 | 0.241 | 1.30 | 1.000 | 14.6 | h5, orthorhombic | +| 5src | I | 0.97 | 1.05 | 138681 | +0.015 | +0.035 | 0.247 | 0.97 | 0.994 | 19.6 | cbf, tetragonal | +| 6iu9 | I | 2.73 | 3.00 | 9191 | +0.014 | +0.035 | 0.156 | 2.73 | 0.984 | 5.3 | cbf, trigonal, twin | +| 9yzk | I | 3.86 | 4.50 | 13466 | -0.000 | +0.036 | 0.025 | 3.86 | 0.998 | 7.8 | cbf, monoclinic | +| 9fcf | F | 1.81 | 2.36 | 9867 | -0.001 | +0.036 | 0.171 | 1.81 | 0.960 | 7.8 | cbf | +| 8xtg | I | 1.54 | 2.00 | 177630 | -0.004 | +0.037 | 0.285 | 1.54 | 1.002 | 7.2 | cbf, trigonal | +| 9ea5 | I | 1.64 | 2.00 | 51543 | +0.007 | +0.040 | 0.315 | 1.64 | 0.985 | 26.6 | cbf, monoclinic | +| 8t7r | I | 3.22 | 3.84 | 13697 | +0.034 | +0.041 | 0.282 | 3.21 | 1.002 | 9.9 | cbf, monoclinic | +| 9khr | I | 1.38 | 2.00 | 11649 | +0.008 | +0.041 | 0.277 | 1.38 | 0.984 | 9.4 | marCCD, orthorhombic | +| 6h2p_native | F | 1.32 | 1.48 | 188243 | -0.006 | +0.042 | 0.298 | 1.32 | 0.975 | 19.9 | cbf, orthorhombic, two-wavelength | +| 8xte | I | 1.64 | 1.99 | 195244 | +0.004 | +0.043 | 0.307 | 1.64 | 0.999 | 12.4 | cbf, trigonal | +| 8owm | F | 1.48 | 1.70 | 293908 | +0.005 | +0.044 | 0.388 | 1.48 | 0.984 | 22.1 | cbf, triclinic | +| 7pq7 | F | 1.38 | 1.55 | 51890 | +0.001 | +0.045 | 0.151 | 1.38 | 0.999 | 13.7 | cbf, monoclinic | +| 8tha | I | 1.34 | 1.68 | 8018 | +0.002 | +0.045 | 0.359 | 1.34 | 0.978 | 25.9 | cbf, hexagonal | +| 7orr | I | 1.65 | 1.79 | 16877 | +0.007 | +0.045 | 0.391 | 1.65 | 1.002 | 25.5 | h5, cubic | +| 9o0h | I | 2.01 | 2.24 | 15448 | +0.036 | +0.046 | 0.263 | 2.01 | 0.981 | 6.1 | cbf, orthorhombic | +| 9rp9 | I | 1.91 | 2.10 | 19429 | +0.015 | +0.047 | 0.317 | 1.92 | 1.012 | 28.8 | h5, monoclinic | +| 8sa8 | I | 1.11 | 1.30 | 409824 | +0.009 | +0.047 | 0.474 | 1.11 | 0.987 | 22.0 | h5, monoclinic | +| 8oic | I | 2.37 | 2.51 | 78747 | +0.014 | +0.047 | 0.231 | 2.37 | 0.970 | 19.6 | h5, triclinic | +| 7ph1 | F | 1.08 | 1.18 | 120936 | -0.007 | +0.050 | 0.379 | 1.08 | 0.996 | 15.9 | cbf, orthorhombic | +| 9w3y | I | 1.20 | 1.50 | 61343 | +0.008 | +0.052 | 0.445 | 1.19 | 1.003 | 19.9 | h5, orthorhombic | +| 6hwj | I | 1.72 | 1.98 | 52134 | +0.010 | +0.053 | 0.370 | 1.72 | 0.972 | 37.8 | cbf, monoclinic | +| 9q41 | I | 1.69 | 1.95 | 42195 | +0.009 | +0.055 | 0.395 | 1.69 | 0.983 | 7.8 | h5, orthorhombic | +| 8ys9 | I | 1.35 | 1.46 | 75646 | +0.006 | +0.056 | 0.350 | 1.35 | 0.992 | 15.4 | h5, orthorhombic | +| 8v4o | I | 2.10 | 2.70 | 59576 | +0.012 | +0.058 | 0.261 | 2.10 | 0.996 | 15.5 | h5, hexagonal | +| 7n0i | I | 1.65 | 1.80 | 117514 | +0.018 | +0.059 | 0.134 | 1.65 | 1.036 | 10.7 | cbf, orthorhombic, tncs | +| 8rud | I | 1.70 | 2.10 | 77721 | +0.009 | +0.059 | 0.367 | 1.70 | 0.917 | 12.3 | cbf, monoclinic | +| 7qij | I | 3.59 | 4.10 | 135378 | +0.027 | +0.062 | 0.082 | 3.60 | 1.010 | 9.4 | cbf, orthorhombic | +| 9fcg | F | 1.39 | 1.54 | 37976 | +0.004 | +0.063 | 0.456 | 1.39 | 0.975 | 9.3 | cbf, tetragonal | +| 7qis | F | 1.75 | 1.83 | 93162 | +0.016 | +0.064 | 0.276 | 1.75 | 1.022 | 17.2 | cbf, hexagonal | +| 6jgi | I | 0.75 | 0.85 | 187106 | +0.004 | +0.070 | 0.407 | 0.75 | 0.983 | 8.4 | marCCD, orthorhombic | +| 9t6s | I | 1.76 | 2.00 | 25144 | +0.021 | +0.075 | 0.211 | 1.76 | 0.953 | 24.9 | h5, orthorhombic, tncs | +| 7ou1 | F | 1.41 | 1.65 | 176468 | +0.009 | +0.083 | 0.385 | 1.40 | 0.963 | 9.5 | marccd | +| 7rji | I | 1.49 | 1.71 | 16698 | +0.004 | +0.087 | 0.332 | 1.48 | 0.976 | 8.3 | cbf, trigonal | +| 5jvn | I | 2.24 | 2.90 | 21045 | +0.030 | +0.092 | 0.235 | 2.24 | 1.030 | 15.6 | cbf, hexagonal | +| 7k1l | F | 1.91 | 2.25 | 54383 | +0.012 | +0.093 | 0.337 | 1.91 | 1.016 | 9.9 | cbf, hexagonal | +| 6toc | I | 1.64 | 1.85 | 6080 | +0.015 | +0.132 | 0.359 | 1.64 | 0.916 | 24.3 | cbf, tetragonal, twin | +| 9i0a | F | 1.81 | 2.22 | 60317 | +0.018 | +0.160 | 0.172 | 1.81 | 0.987 | 13.8 | h5, orthorhombic | +| 8xbp | F | 1.72 | 2.00 | 24156 | +0.013 | +0.207 | 0.113 | 1.71 | 1.059 | 16.2 | h5, monoclinic | +| 5j23 | F | 2.17 | 2.30 | 56769 | +0.192 | +0.307 | 0.407 | 2.17 | 1.012 | 11.5 | marCCD, trigonal, twin | + +No comparison: 9min (the merge does not match the model in any setting); 9rci (the merge does not match the model in any setting); 6z9g (the merge does not match the model in any setting) + +### Like for like with XDS: the reference-range table + +The same merge, binned a second time over XDS's range (--report-resolution; report-only, nothing was processed differently for it), against XDS's CORRECT.LP totals. Where rugnux's own cut is coarser than the reference ('coverage' in the last column), the shells past it are not merged at all: the completeness there is coverage of XDS's range, not a quality loss, and the other rugnux numbers are over the shells it reached. XDS's totals are over its own merged range, which is the reference range except where the reference d_min was derived (see below). + +| set | arm | range A | own d_min | compl % (rugnux / XDS) | mult | I/sigma | R_meas | low-res R_meas (to d A) | CC1/2 | CC1/2 noise ratio | ISa | reading | +|:--|:--|:--|--:|:--|:--|--:|:--|:--|:--|--:|:--|:--| +| cytc_x06da_1 | inhouse | 50.000 1.875 | 1.70 | 100.0 / 99.9 | 10.5 / 7.7 | 14.8 | 8.5% / 8.4% | 3.7% / 3.3% (5.59 / 5.58) | 0.9995 / 0.999 | 0.33 | 17.8 / 24.5 | like for like | +| cytc_x06da_2 | inhouse | 50.000 1.690 | 1.57 | 100.0 / 99.9 | 10.2 / 7.8 | 12.9 | 9.1% / 9.7% | 3.5% / 3.2% (5.05 / 5.03) | 0.9996 / 0.999 | 0.27 | 21.4 / 27.0 | like for like | +| cytc_x10sa | inhouse | 50.000 2.039 | 1.95 | 99.9 / 99.8 | 10.6 / 10.7 | 9.4 | 18.0% / 23.2% | 3.9% / 3.6% (6.08 / 6.05) | 0.9990 / 0.999 | 0.67 | 26.2 / 31.8 | like for like | +| insu_H_x06da_notwin | inhouse | 50.000 1.544 | 1.42 | 98.7 / 97.4 | 4.8 / 3.5 | 15.5 | 6.1% / 6.8% | 4.4% / 4.5% (4.61 / 4.61) | 0.9987 / 0.998 | 0.52 | 20.5 / 17.7 | like for like | +| insu_H_x06da_twin | inhouse | 50.000 1.455 | 1.38 | 96.8 / 95.0 | 4.5 / 3.6 | 8.1 | 13.2% / 11.8% | 11.5% / 10.6% (4.35 / 4.34) | 0.9880 / 0.988 | 0.96 | 6.3 / 6.8 | like for like | +| insu_I_x06da_13keV | inhouse | 50.000 1.635 | 1.48 | 100.0 / 100.0 | 20.2 / 17.2 | 9.4 | 26.8% / 27.9% | 8.0% / 9.3% (4.88 / 4.85) | 0.9983 / 0.998 | 0.68 | 27.2 / 18.8 | like for like | +| insu_I_x06da_5keV | inhouse | 50.000 2.450 | 2.43 | 96.1 / 91.7 | 15.4 / 11.2 | 33.5 | 6.2% / 7.3% | 4.4% / 4.8% (7.28 / 7.21) | 0.9995 / 0.999 | 0.33 | 28.3 / 17.5 | like for like | +| insu_I_x06da_5keV_2 | inhouse | 50.000 2.450 | 2.42 | 96.1 / 91.8 | 15.9 / 12.7 | 26.6 | 7.9% / 7.6% | 5.8% / 4.8% (7.28 / 7.21) | 0.9992 / 0.999 | 0.53 | 24.4 / 20.0 | like for like | +| insu_I_x06da_6keV | inhouse | 50.000 2.040 | 2.03 | 95.0 / 92.4 | 15.5 / 12.4 | 34.4 | 5.8% / 6.5% | 4.8% / 4.7% (6.08 / 6.03) | 0.9995 / 0.999 | 0.33 | 20.6 / 17.9 | like for like | +| insu_I_x06da_low_isa | inhouse | 50.000 1.300 | 1.44 | 74.3 / 90.8 | 16.6 / 13.3 | 7.6 | 22.1% / 22.0% | 15.9% / 16.7% (3.89 / 3.92) | 0.9950 / 0.998 | 2.00 | 5.6 / 4.2 | coverage: own cut 1.44 A is coarser than the reference, 1 shell(s) not merged | +| insu_I_x06da_ref | inhouse | 50.000 1.621 | 1.40 | 100.0 / 100.0 | 15.9 / 13.8 | 17.2 | 24.0% / 34.9% | 5.9% / 14.3% (4.84 / 4.82) | 0.9994 / 0.998 | 0.24 | 32.1 / 25.1 | like for like | +| insu_I_x06da_weak | inhouse | 999.000 1.080 | 1.64 | 28.9 / 92.4 | 40.1 / 29.0 | 15.8 | 16.7% / 40.1% | 6.3% / 5.4% (3.24 / 3.22) | 0.9997 / 0.999 | 0.20 | 14.9 / 18.9 | coverage: own cut 1.64 A is coarser than the reference, 5 shell(s) not merged | +| lysoI_micromax_mono | inhouse | 50.000 1.650 | 1.35 | 100.0 / 100.0 | 6.5 / 6.8 | 22.1 | 5.7% / 6.8% | 3.0% / 2.8% (4.93 / 4.99) | 0.9994 / 0.999 | 0.40 | 39.2 / 31.4 | like for like | +| lysoI_micromax_pink | inhouse | 50.000 1.650 | 1.42 | 100.0 / 100.0 | 6.5 / 6.8 | 19.5 | 6.3% / 7.9% | 3.1% / 2.8% (4.93 / 4.99) | 0.9993 / 0.999 | 0.47 | 35.1 / 29.2 | like for like | +| lyso_micromax_mono | inhouse | 50.000 1.500 | 1.20 | 100.0 / 100.0 | 6.1 / 6.2 | 23.5 | 4.6% / 4.7% | 2.5% / 2.3% (4.48 / 4.54) | 0.9996 / 1.000 | 0.80 | 40.3 / 39.9 | like for like | +| lyso_micromax_pink | inhouse | 50.000 1.450 | 1.26 | 99.9 / 99.9 | 5.8 / 6.0 | 19.8 | 4.9% / 5.1% | 2.5% / 2.2% (4.34 / 4.39) | 0.9995 / 1.000 | 1.00 | 36.6 / 37.4 | like for like | +| lyso_x06da_5keV | inhouse | 50.000 2.450 | 2.43 | 87.8 / 86.4 | 11.1 / 8.8 | 32.3 | 5.9% / 6.4% | 5.4% / 4.7% (7.28 / 7.22) | 0.9991 / 0.998 | 0.36 | 25.6 / 19.4 | like for like | +| lyso_x06da_atten_wedge | inhouse | 50.000 1.264 | 1.16 | 100.0 / 99.9 | 13.0 / 11.0 | 9.1 | 30.9% / 23.9% | 7.1% / 6.9% (3.78 / 3.76) | 0.9984 / 0.998 | 0.64 | 13.2 / 16.6 | like for like | +| lyso_x06da_half_image | inhouse | 50.000 1.650 | 1.56 | 100.0 / 99.6 | 5.5 / 5.3 | 4.0 | 94.7% / 59.5% | 25.6% / 25.8% (4.93 / 4.91) | 0.8590 / 0.958 | 3.70 | 7.4 / 6.6 | like for like | +| lyso_x06da_ice | inhouse | 50.000 1.431 | 1.34 | 100.0 / 100.0 | 10.5 / 11.9 | 9.4 | 16.6% / 21.9% | 4.6% / 5.5% (4.28 / 4.26) | 0.9986 / 0.998 | 0.56 | 23.5 / 23.3 | like for like | +| lyso_x06da_ref | inhouse | 50.000 1.200 | 0.99 | 100.0 / 100.0 | 13.8 / 13.8 | 35.8 | 4.3% / 4.5% | 2.7% / 2.9% (3.59 / 3.58) | 0.9998 / 1.000 | 0.40 | 29.8 / 28.3 | like for like | +| lyso_x10sa_90deg_1 | inhouse | 50.000 1.966 | 1.86 | 100.0 / 99.3 | 3.5 / 3.5 | 6.4 | 16.3% / 18.2% | 4.7% / 4.9% (5.86 / 5.83) | 0.9950 / 0.995 | 0.91 | 18.6 / 20.8 | like for like | +| lyso_x10sa_90deg_2 | inhouse | 50.000 1.973 | 1.88 | 100.0 / 99.3 | 3.5 / 3.5 | 6.4 | 16.2% / 18.0% | 4.8% / 5.1% (5.88 / 5.86) | 0.9949 / 0.994 | 0.78 | 17.9 / 18.1 | like for like | +| lyso_x10sa_strong | inhouse | 999.000 1.180 | 1.37 | 63.8 / 65.8 | 22.7 / 11.5 | 9.0 | 24.0% / 11.2% | 12.9% / 7.7% (3.54 / 3.52) | 0.9963 / 0.998 | 1.48 | 4.9 / 8.6 | coverage: own cut 1.37 A is coarser than the reference, 2 shell(s) not merged | +| myob_x06da | inhouse | 50.000 1.422 | 1.23 | 99.8 / 99.4 | 3.5 / 3.5 | 7.5 | 12.5% / 23.2% | 4.3% / 10.5% (4.25 / 4.23) | 0.9963 / 0.987 | 0.27 | 25.3 / 7.6 | like for like | +| myob_x06da_powder_1 | inhouse | 50.000 1.495 | 1.38 | 99.6 / 74.2 | 5.5 / 2.9 | 1.3 | 83.4% / 49.3% | 17.0% / 12.5% (4.47 / 4.44) | 0.9296 / 0.966 | 2.12 | 2.6 / 5.5 | like for like | +| myob_x06da_powder_2 | inhouse | 999.000 0.990 | 1.41 | 35.0 / 58.0 | 5.4 / 4.7 | 0.9 | 151.9% / 110.6% | 58.8% / 70.6% (2.97 / 2.98) | 0.8878 / 0.655 | 0.24 | 2.8 / 2.2 | coverage: own cut 1.41 A is coarser than the reference, 4 shell(s) not merged | +| myob_x06da_sparse | inhouse | 50.000 2.000 | 1.77 | 100.0 / 73.7 | 5.6 / 5.2 | 3.2 | 41.8% / 32.6% | 19.9% / 14.6% (5.96 / 5.92) | 0.8904 / 0.972 | 4.20 | 3.2 / 5.5 | like for like | +| myob_x06da_split | inhouse | 50.000 1.506 | 1.84 | 54.9 / 77.7 | 5.3 / 4.4 | 2.0 | 69.9% / 57.7% | 20.6% / 9.1% (4.50 / 4.48) | 0.5255 / 0.984 | 53.82 | 3.1 / 12.4 | coverage: own cut 1.84 A is coarser than the reference, 2 shell(s) not merged | +| myob_x10sa | inhouse | 50.000 1.742 | 1.62 | 98.2 / 97.7 | 3.2 / 3.3 | 5.0 | 17.7% / 24.7% | 7.9% / 18.6% (5.20 / 5.16) | 0.9886 / 0.965 | 0.31 | 9.3 / 5.2 | like for like | +| thau_bl1a_3p8keV | inhouse | 100.000 3.100 | 3.02 | 93.5 / 92.2 | 9.1 / 7.9 | 26.4 | 7.4% / 5.7% | 6.9% / 3.9% (9.26 / 9.23) | 0.9956 / 0.997 | 1.26 | 17.4 / 30.8 | like for like | +| thau_bl1a_4p6keV | inhouse | 100.000 2.530 | 2.47 | 93.2 / 92.0 | 9.1 / 7.9 | 26.1 | 6.6% / 6.2% | 5.2% / 3.9% (7.57 / 7.56) | 0.9979 / 0.998 | 0.84 | 33.9 / 35.6 | like for like | +| thau_bl1a_6p5keV | inhouse | 100.000 1.780 | 1.73 | 93.3 / 91.8 | 9.1 / 8.0 | 19.5 | 7.1% / 7.2% | 4.6% / 4.1% (5.33 / 5.33) | 0.9990 / 0.999 | 0.67 | 40.6 / 34.5 | like for like | +| thau_micromax_pink | inhouse | 50.000 1.400 | 1.28 | 99.0 / 98.5 | 9.6 / 9.4 | 14.7 | 8.6% / 8.0% | 4.9% / 4.0% (4.19 / 4.17) | 0.9992 / 0.999 | 0.53 | 15.5 / 21.2 | like for like | +| thau_x10sa_0p1deg | inhouse | 50.000 2.197 | 2.04 | 99.5 / 99.0 | 11.6 / 11.4 | 7.8 | 18.4% / 23.0% | 9.1% / 10.4% (6.54 / 6.51) | 0.9977 / 0.997 | 0.66 | 7.0 / 9.2 | like for like | +| thau_x10sa_16keV | inhouse | 50.000 1.300 | 1.22 | 94.3 / 94.3 | 18.5 / 18.4 | 24.6 | 7.0% / 7.2% | 2.8% / 2.7% (3.89 / 3.89) | 0.9998 / 1.000 | 0.40 | 52.7 / 44.5 | like for like | +| thau_x10sa_injection | inhouse | 50.000 1.280 | 1.26 | 86.3 / 86.2 | 10.1 / 10.0 | 36.1 | 3.9% / 3.7% | 2.6% / 2.2% (3.83 / 3.83) | 0.9998 / 1.000 | 0.40 | 33.2 / 36.2 | like for like | + +Data quality is not scored; this table is for a human to judge. CC1/2 = S / (S + E) (signal and half-set error variances), so 1 / CC1/2 - 1 = E / S is the half-set noise; the noise ratio is rugnux's over XDS's, with XDS's CC1/2 taken at the bottom of its printed rounding (-0.0005); 2 would mean noisier than XDS's merge with half its observations. The low-resolution R_meas is the lowest shell of each table (the shells are XDS's, so both cover the same reflections); reported like everything here. + +XDS merged past its own signal (CC1/2 of its finest shell not significant), so the reference d_min is where XDS's CC1/2 falls through 0.30, and that is also the d_min of the reference-range table: cytc_x10sa (XDS range 2.04 A, reference 2.27 A); insu_I_x06da_weak (XDS range 1.08 A, reference 1.81 A); lyso_x10sa_strong (XDS range 1.18 A, reference 1.24 A). XDS's pooled R_meas, CC1/2 and completeness are over its whole range. + +### Failures + +| set | arm | cause | reason | +|:--|:--|:--|:--| +| 7mzt | open | sym_screw | P 21 21 21 vs reference P 21 21 2 | +| 9min | open | lattice_halved | primitive volume ratio 0.49 | +| 6z9g | open | lattice_halved | primitive volume ratio 0.50 | +| myob_x06da_powder_1 | inhouse | sym_over | P 1 2 1 vs reference P 1 | + +Not scored: 7k1l (P 6 vs reference P 63: the screw along c is undeterminable from these data (offered P 61 | P 65 | P 62 | P 64 | P 63)); cuhf2 (no reference to score against) + +### Accepted alternatives + +5 of the passes above are rows that accept more than one reference: the deposition and our reduction disagree, the disagreement is real, and no test available to us settles it, so either answer passes as long as the program picks one of them. These are open questions, not errors attributed to the deposition; the manifest's `ref` keeps the deposited values verbatim in every one of them. + +- 6pxb (open): we report P 31 1 2, the reference is P 32; accepted as P 32 1 2. The deposition merged in point group 3 (its 55502 unique reflections to 1.747 A are what Laue class -3 holds, twice what -3 1 m would) and refined six chains in P 32. The intensities read point group 312 instead: the three added two-folds correlate at 0.98-0.99, at or above the three-folds nobody disputes (0.98), against 0.37-0.40 for the 321 and 622 operators, POINTLESS on our P1 merge picks P -3 1 m (likelihood 1.000), and the reflections centric in 312 but not in 3 are distributed as centric (+435 nats), which a twin law cannot produce. Against that, the deposited chains pair under the added two-fold at 0.3-0.7 A, more than coordinate error at 1.75 A, and the model tells the two indexings apart (R 0.22 against 0.25), so the two-fold may be a near-exact non-crystallographic one; ZANUDA settles on P 32 2 1, whose operators these data do not support. Neither answer is established, so both are accepted; P 31 1 2, which the data cannot separate from P 32 1 2, is accepted as its hand +- 6toc (open): we report P 42 2 2, the reference is P 42; accepted as P 42 2 2. Our reduction and an independent POINTLESS run on our own P1 merge both read point group 422, and the deposited asymmetric unit's two chains are related by the very two-fold the higher group adds, to 0.16 A CA RMSD over 43 residues - coordinate error at 1.85 A. Merging in P 42 2 2 costs 0.0006 in R_meas for 1.75x the multiplicity and correlates better with the deposited model (0.9802 vs 0.9764). The refinement test is NOT unanimous: ZANUDA 1.097 refines P 42 2 2 to R-free 0.2561 against P 42's 0.2636 at half the parameters and reports the deposited assignment incorrect, while an independent Refmac 5.8.0431 comparison on a symmetry-consistent free set puts P 42 ahead by 0.004-0.020 depending on cycle count - less than the spread between refinement protocols. Neither answer is established: a pseudo-symmetry too exact for any test we have remains a live explanation, and so does the deposited assignment. Either is accepted +- 8xte (open): we report P 31 2 1, the reference is P 32; accepted as P 31 2 1. The evidence favours the higher group here. The twin-immune centric zone of the added two-folds - reflections the higher group makes centric, which are their own twin mates and so cannot be made to read centric by a merohedral twin law - gives <|E^2-1|> = 0.946 +/- 0.012 against a centric expectation of 0.968 and an acentric 0.736, at +707 nats. Re-refinement on a shared free set with the twin law removed from both sides favours P 3_2 2 1 (0.2177/0.2322) over P 3_2 (0.2460/0.2612), and the deposited entry's published R values reproduce only with an undeclared twin law h,-h-k,-l at alpha = 0.50, which is itself a 321-symmetric target. No refinement R can close the question in principle, because a merohedral twin at exactly alpha = 0.5 and true 321 predict identical intensities; the case rests on the centric zone. Both answers are accepted, ours being the better supported +- 8xtg (open): we report P 31 2 1, the reference is P 32; accepted as P 31 2 1. The evidence favours the DEPOSITION here, and this row must not be read as the 8xte one. Every correlation-based instrument we have - our own operator correlations, POINTLESS (0.85 on our P1 merge) - reads point group 321, but our own twin-immune centric-zone test, the only one that separates real symmetry from pseudo-symmetry, reads <|E^2-1|> = 0.869 at -44.9 nats, between the two expectations and on the wrong side, i.e. AGAINST the promotion, with an L-test twin fraction of 0.20-0.26. Whether this crystal is partially twinned or purely pseudo-symmetric is not established. Both answers are accepted, the deposition being the better supported +- 9rci (open): we report P 1, the reference is P 1; accepted as cell 35.869 39.297 199.976. The deposited cell is the (0,1/2,1/2)-centred sublattice of the supercell we report, to 0.17%, and both descriptions of this lattice are defensible. The Patterson has an off-origin peak at 62.5% of the origin, so a real translational NCS relates the two halves of our cell: describing the crystal by the doubled cell with the near-translation left in the content, or by its sublattice with the near-translation absorbed into the lattice, is a choice, not a measurement. The alternative cell is the deposited one doubled along c with the centring removed (c' = b + 2c), computed from the deposited cell alone - not from our output + +### Per set + +Space group: on the open arm the data's own determination. Where --model put the model's enantiomorph on the label, the label is in the note and the group scored is the search's. + +R_free and R_work describe one run against its model and do not carry across runs: the model is scaled to the data by an overall factor and an anisotropic B, so whatever the amplitudes' radial profile does that this shape cannot follow is reported as R, and a change in the profile alone moves R_free further than a real change in the data does. R_model (shell-scaled) is the same R with one free scale per resolution shell removing exactly that, over every reflection rather than the free 5%, and it is the column to read in a delta table. Radial misfit is how big that per-shell rescale had to be (the RMS of its logarithm): when it moves between two runs, R_free between them means little. Both are reported for a human to judge; nothing is scored on them. + +| set | arm | verdict | space group | ref | cell dev % | V ratio | d_min | ref d_min | gain | compl % | mult | R_meas | low-res R_meas | CC1/2 | ISa | ref ISa | idx | R_model (shell-scaled) | radial misfit | CC model | R_free (within-run) | R_work | dep R_free | R_free ratio | REFMAC R_free | REFMAC dep data | REFMAC ratio | dCC dep. outer | model fit | time s | note | +|:--|:--|:--|:--|:--|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|:--| +| 11if | open | pass | P 41 | P 43 | 0.12 | 0.997 | 1.38 | 1.51 | +8.8% | 90.7 | 11.2 | 4.6% | 2.8% | 1.000 | 25.5 | - | - | 0.191 | 0.193 | 0.962 | 0.200 | 0.206 | 0.210 | 0.956 | 0.233 | 0.233 | 0.998 | 0.006 | ACCEPTED | 14 | P 41 vs reference P 43 (hand only (needs anomalous)); labelled P 43 from the model | +| 36gk | open | pass | I 2 2 2 | I 2 2 2 | 0.09 | 0.999 | 2.09 | 2.28 | +8.2% | 99.7 | 13.8 | 15.7% | 7.5% | 0.998 | 11.8 | - | - | 0.203 | 0.112 | 0.933 | 0.203 | 0.212 | 0.223 | 0.913 | 0.255 | 0.252 | 1.014 | 0.024 | NOT_TESTED | 41 | | +| 3inp | open | pass | F 41 3 2 | F 41 3 2 | 0.04 | 1.001 | 1.70 | 2.05 | +16.9% | 99.7 | 18.8 | 10.9% | 5.5% | 0.999 | 13.3 | - | - | 0.239 | 0.068 | 0.946 | 0.239 | 0.240 | 0.177 | 1.351 | 0.240 | 0.244 | 0.984 | 0.003 | NOT_TESTED | 56 | | +| 3ky7 | open | pass | P 43 3 2 | P 43 3 2 | 0.03 | 1.001 | 1.92 | 2.35 | +18.4% | 99.7 | 25.7 | 10.7% | 6.0% | 0.999 | 12.3 | - | - | 0.341 | 0.046 | 0.895 | 0.335 | 0.344 | 0.255 | 1.313 | 0.354 | 0.355 | 0.997 | 0.002 | NOT_TESTED | 57 | | +| 5ebi | open | pass | P 1 21 1 | P 1 21 1 | 0.05 | 1.001 | 0.90 | 1.09 | +17.7% | 99.4 | 3.5 | 10.6% | 6.0% | 0.997 | 14.4 | - | - | 0.549 | 0.072 | 0.501 | 0.535 | 0.551 | 0.169 | 3.168 | 0.178 | 0.181 | 0.981 | 0.008 | ACCEPTED | 52 | | +| 5epe | open | pass | F 2 3 | F 2 3 | 0.00 | 1.000 | 1.78 | 1.90 | +6.3% | 99.8 | 15.3 | 14.3% | 8.2% | 0.998 | 10.0 | - | - | 0.166 | 0.082 | 0.960 | 0.175 | 0.172 | 0.167 | 1.047 | 0.202 | 0.200 | 1.010 | -0.090 | ACCEPTED | 67 | | +| 5f6m | open | pass | P 21 21 21 | P 21 21 21 | 0.08 | 1.002 | 1.09 | 1.10 | +1.0% | 78.8 | 4.0 | 5.2% | 3.8% | 0.998 | 22.2 | - | - | 0.152 | 0.162 | 0.972 | 0.168 | 0.163 | 0.154 | 1.087 | 0.166 | 0.158 | 1.051 | -0.074 | NOT_TESTED | 12 | | +| 5j23 | open | pass | R 3:H | H 3 | 0.15 | 0.998 | 2.17 | 2.30 | +5.6% | 99.7 | 5.2 | 15.2% | 6.3% | 0.996 | 11.5 | - | - | 0.227 | 0.133 | 0.938 | 0.234 | 0.236 | 0.169 | 1.384 | 0.188 | 0.185 | 1.012 | 0.307 | NOT_TESTED | 62 | | +| 5jvn | open | pass | P 6 2 2 | P 6 2 2 | 0.03 | 0.999 | 2.24 | 2.90 | +22.8% | 98.6 | 16.8 | 16.3% | 5.5% | 0.998 | 15.6 | - | - | 0.234 | 0.143 | 0.886 | 0.250 | 0.249 | 0.235 | 1.061 | 0.250 | 0.242 | 1.030 | 0.092 | NOT_TESTED | 40 | | +| 5ky6 | open | pass | P 1 21 1 | P 1 21 1 | 0.64 | 0.994 | 1.55 | 1.94 | +20.2% | 97.1 | 3.2 | 27.3% | 12.5% | 0.987 | 7.2 | - | - | 0.245 | 0.107 | 0.947 | 0.249 | 0.247 | 0.223 | 1.119 | 0.237 | 0.229 | 1.039 | -0.148 | NOT_TESTED | 92 | | +| 5lzl | open | pass | P 31 2 1 | P 31 2 1 | 0.29 | 0.992 | 2.86 | 3.47 | +17.6% | 94.9 | 8.9 | 17.2% | 4.9% | 0.997 | 15.0 | - | - | 0.222 | 0.137 | 0.878 | 0.228 | 0.233 | 0.250 | 0.913 | 0.264 | 0.257 | 1.027 | -0.025 | NOT_TESTED | 25 | | +| 5m17 | open | pass | I 4 | I 4 | 0.08 | 0.998 | 0.98 | 1.03 | +4.5% | 92.6 | 6.0 | 6.0% | 6.0% | 0.994 | 16.0 | - | - | 0.132 | 0.231 | 0.954 | 0.159 | 0.160 | 0.130 | 1.225 | 0.176 | 0.171 | 1.027 | -0.036 | NOT_TESTED | 102 | | +| 5mln | open | pass | P 21 21 2 | P 21 2 21 | 0.10 | 0.999 | 1.26 | 1.60 | +21.4% | 99.7 | 8.1 | 10.4% | 3.6% | 0.999 | 21.4 | - | - | 0.184 | 0.145 | 0.948 | 0.189 | 0.198 | 0.180 | 1.053 | 0.195 | 0.190 | 1.026 | -0.007 | ACCEPTED | 45 | | +| 5nw5 | open | pass | P 21 21 21 | P 21 21 21 | 0.19 | 0.997 | 7.10 | 6.50 | -9.2% | 99.7 | 6.3 | 32.0% | 17.1% | 0.936 | 8.8 | - | - | 0.401 | 0.365 | 0.731 | 0.440 | 0.433 | 0.277 | 1.593 | 0.351 | 0.324 | 1.082 | 0.026 | NOT_TESTED | 47 | | +| 5reo | open | pass | C 1 2 1 | C 1 2 1 | 0.35 | 1.009 | 1.65 | 1.88 | +12.1% | 99.4 | 3.1 | 22.1% | 8.1% | 0.990 | 24.0 | - | - | 0.204 | 0.068 | 0.960 | 0.202 | 0.208 | 0.227 | 0.889 | 0.231 | 0.233 | 0.991 | -0.036 | NOT_TESTED | 10 | | +| 5src | open | pass | P 41 | P 43 | 0.09 | 0.999 | 0.97 | 1.05 | +7.7% | 99.0 | 6.2 | 5.9% | 3.3% | 1.000 | 19.6 | - | - | 0.164 | 0.198 | 0.962 | 0.183 | 0.180 | 0.175 | 1.041 | 0.210 | 0.211 | 0.994 | 0.035 | ACCEPTED | 44 | P 41 vs reference P 43 (hand only (needs anomalous)); labelled P 43 from the model | +| 5t39 | open | pass | P 1 21 1 | P 1 21 1 | 0.13 | 1.003 | 1.01 | 1.10 | +8.3% | 91.1 | 4.3 | 7.0% | 4.6% | 0.998 | 16.0 | - | - | 0.152 | 0.167 | 0.955 | 0.165 | 0.162 | 0.154 | 1.070 | 0.185 | 0.182 | 1.018 | -0.064 | NOT_TESTED | 75 | | +| 6cdl | open | pass | P 21 21 2 | P 21 21 2 | 1.22 | 1.023 | 1.15 | 1.25 | +8.0% | 75.0 | 6.5 | 8.2% | 7.1% | 0.996 | 11.2 | - | - | 0.150 | 0.281 | 0.945 | 0.174 | 0.173 | 0.181 | 0.960 | 0.201 | 0.187 | 1.072 | -0.085 | NOT_TESTED | 57 | | +| 6f3p | open | pass | C 1 2 1 | C 1 2 1 | 0.12 | 0.997 | 1.13 | 1.35 | +16.1% | 99.3 | 3.7 | 10.6% | 6.1% | 0.997 | 9.4 | - | - | 0.144 | 0.178 | 0.967 | 0.167 | 0.167 | 0.127 | 1.318 | 0.176 | 0.174 | 1.013 | 0.001 | NOT_TESTED | 130 | | +| 6fid | open | pass | P 21 21 21 | P 21 21 21 | 0.41 | 1.008 | 1.98 | 2.20 | +10.1% | 72.9 | 12.7 | 9.7% | 10.8% | 0.998 | 13.2 | - | - | 0.194 | 0.085 | 0.943 | 0.199 | 0.201 | 0.220 | 0.908 | 0.238 | 0.242 | 0.982 | -0.024 | NOT_TESTED | 28 | | +| 6fvz | open | pass | C 2 2 2 | C 2 2 2 | 0.43 | 0.989 | 1.49 | 1.80 | +17.2% | 99.5 | 6.7 | 24.2% | 5.1% | 0.996 | 20.6 | - | - | 0.202 | 0.103 | 0.953 | 0.206 | 0.209 | 0.199 | 1.036 | 0.207 | 0.207 | 1.001 | -0.018 | NOT_TESTED | 36 | | +| 6fwc | open | pass | C 2 2 2 | C 2 2 2 | 0.04 | 0.999 | 1.41 | 1.70 | +16.9% | 94.7 | 4.0 | 16.4% | 4.3% | 0.995 | 27.2 | - | - | 0.189 | 0.097 | 0.959 | 0.197 | 0.199 | 0.189 | 1.039 | 0.197 | 0.201 | 0.979 | 0.017 | NOT_TESTED | 48 | | +| 6g1f | open | pass | C 1 2 1 | C 1 2 1 | 0.04 | 1.000 | 1.93 | 2.25 | +14.1% | 99.5 | 3.8 | 11.7% | 4.1% | 0.997 | 21.2 | - | - | 0.210 | 0.141 | 0.954 | 0.224 | 0.222 | 0.211 | 1.062 | 0.230 | 0.226 | 1.017 | 0.016 | NOT_TESTED | 140 | | +| 6h2p_1p89A | open | pass | C 2 2 21 | C 2 2 21 | 0.03 | 1.000 | 1.76 | - | - | 88.1 | 10.2 | 8.3% | 5.2% | 0.999 | 22.5 | - | - | 0.144 | 0.111 | 0.969 | 0.154 | 0.152 | 0.170 | 0.904 | 0.173 | 0.163 | 1.059 | -0.008 | NOT_TESTED | 172 | | +| 6h2p_native | open | pass | C 2 2 21 | C 2 2 21 | 0.04 | 0.999 | 1.32 | 1.48 | +10.8% | 99.5 | 6.6 | 13.7% | 4.0% | 0.999 | 19.9 | - | - | 0.170 | 0.076 | 0.973 | 0.174 | 0.174 | 0.170 | 1.021 | 0.178 | 0.182 | 0.975 | 0.042 | NOT_TESTED | 110 | | +| 6h5t | open | pass | I 4 2 2 | I 4 2 2 | 0.40 | 0.991 | 1.48 | 1.69 | +12.4% | 99.4 | 7.5 | 15.2% | 9.3% | 0.996 | 9.2 | - | - | 0.203 | 0.126 | 0.941 | 0.218 | 0.212 | 0.191 | 1.145 | 0.216 | 0.214 | 1.009 | -0.015 | NOT_TESTED | 35 | | +| 6hv2 | open | pass | P 61 2 2 | P 61 2 2 | 0.04 | 1.001 | 1.42 | 1.71 | +17.2% | 99.7 | 29.5 | 18.1% | 5.7% | 1.000 | 13.6 | - | - | 0.235 | 0.460 | 0.946 | 0.247 | 0.251 | 0.263 | 0.940 | 0.274 | 0.326 | 0.840 | -0.023 | NOT_TESTED | 37 | | +| 6hwj | open | pass | P 1 21 1 | P 1 21 1 | 0.10 | 1.001 | 1.72 | 1.98 | +13.2% | 97.5 | 3.5 | 8.4% | 2.5% | 0.999 | 37.8 | - | - | 0.187 | 0.128 | 0.960 | 0.192 | 0.199 | 0.214 | 0.897 | 0.217 | 0.223 | 0.972 | 0.053 | NOT_TESTED | 19 | | +| 6i3j | open | pass | F 2 2 2 | F 2 2 2 | 0.08 | 0.999 | 2.32 | 2.59 | +10.3% | 98.8 | 7.2 | 23.5% | 16.7% | 0.983 | 7.0 | - | - | 0.202 | 0.161 | 0.915 | 0.233 | 0.230 | 0.226 | 1.031 | 0.217 | 0.190 | 1.141 | -0.007 | NOT_TESTED | 83 | | +| 6iu5 | open | pass | P 31 | P 31 | 0.02 | 1.000 | 2.12 | 2.25 | +6.0% | 99.2 | 4.8 | 18.9% | 12.5% | 0.991 | 7.9 | - | - | 0.212 | 0.081 | 0.930 | 0.215 | 0.222 | 0.250 | 0.861 | 0.261 | 0.258 | 1.008 | -0.018 | ACCEPTED | 89 | | +| 6iu6 | open | pass | P 31 | P 31 | 0.33 | 0.996 | 2.38 | 2.90 | +18.0% | 97.8 | 4.4 | 14.2% | 10.1% | 0.992 | 7.0 | - | - | 0.221 | 0.189 | 0.949 | 0.234 | 0.241 | 0.211 | 1.108 | 0.219 | 0.209 | 1.045 | -0.021 | ACCEPTED | 87 | | +| 6iu8 | open | pass | P 31 | P 31 | 0.02 | 1.000 | 2.33 | 2.70 | +13.9% | 99.7 | 10.2 | 14.3% | 8.0% | 0.997 | 8.1 | - | - | 0.269 | 0.179 | 0.880 | 0.284 | 0.283 | 0.215 | 1.322 | 0.218 | 0.215 | 1.013 | -0.028 | NOT_TESTED | 18 | | +| 6iu9 | open | pass | P 31 | P 31 | 0.22 | 1.005 | 2.73 | 3.00 | +8.9% | 99.6 | 5.0 | 18.0% | 12.4% | 0.989 | 5.3 | - | - | 0.306 | 0.109 | 0.826 | 0.316 | 0.309 | 0.265 | 1.191 | 0.250 | 0.254 | 0.984 | 0.035 | ACCEPTED | 129 | | +| 6jgh | open | pass | P 21 21 21 | P 21 21 21 | 0.39 | 1.008 | 0.87 | 0.94 | +7.0% | 99.3 | 7.2 | 22.7% | 10.7% | 0.989 | 6.7 | - | - | 0.148 | 0.187 | 0.951 | 0.169 | 0.170 | 0.129 | 1.311 | 0.181 | 0.163 | 1.111 | -0.073 | NOT_TESTED | 105 | | +| 6jgi | open | pass | P 21 21 21 | P 21 21 21 | 0.20 | 0.997 | 0.75 | 0.85 | +12.0% | 98.5 | 7.0 | 14.4% | 8.6% | 0.995 | 8.4 | - | - | 0.114 | 0.146 | 0.974 | 0.130 | 0.128 | 0.112 | 1.160 | 0.146 | 0.148 | 0.983 | 0.070 | NOT_TESTED | 110 | | +| 6jgj | open | pass | P 21 21 21 | P 21 21 21 | 0.28 | 0.992 | 0.65 | 0.77 | +15.7% | 85.1 | 7.2 | 7.6% | 4.8% | 0.999 | 14.7 | - | - | 0.168 | 0.066 | 0.962 | 0.171 | 0.170 | 0.125 | 1.366 | 0.189 | 0.167 | 1.131 | 0.026 | NOT_TESTED | 52 | | +| 6moj | open | pass | I 41 2 2 | I 41 2 2 | 0.08 | 0.998 | 2.43 | 2.43 | +0.2% | 99.7 | 26.7 | 52.6% | 8.9% | 0.998 | 8.3 | - | - | 0.275 | 0.367 | 0.934 | 0.285 | 0.287 | 0.250 | 1.140 | 0.280 | 0.282 | 0.992 | 0.029 | NOT_TESTED | 124 | | +| 6nen | open | pass | P 3 1 2 | P 3 1 2 | 0.07 | 0.998 | 1.72 | 2.15 | +20.2% | 99.7 | 22.3 | 26.9% | 10.6% | 0.997 | 6.3 | - | - | 0.210 | 0.082 | 0.954 | 0.212 | 0.213 | 0.208 | 1.023 | 0.211 | 0.210 | 1.004 | 0.002 | ACCEPTED | 24 | | +| 6o2h | open | pass | P 1 | P 1 | 0.74 | 0.982 | 1.10 | 1.21 | +9.5% | 30.3 | 1.1 | 7.2% | 8.7% | 0.978 | 20.8 | - | - | 0.099 | 0.209 | 0.932 | 0.131 | 0.130 | 0.117 | 1.121 | 0.180 | 0.155 | 1.162 | -0.005 | ACCEPTED | 16 | | +| 6oel | open | pass | F 41 3 2 | F 41 3 2 | 0.00 | 1.000 | 2.85 | 3.10 | +8.1% | 99.7 | 41.4 | 36.6% | 8.4% | 0.998 | 9.9 | - | - | 0.248 | 0.137 | 0.813 | 0.255 | 0.259 | 0.256 | 0.998 | 0.278 | 0.288 | 0.965 | -0.024 | NOT_TESTED | 57 | | +| 6p8p | open | pass | P 4 | P 4 | 0.14 | 1.003 | 1.47 | 1.64 | +9.9% | 99.7 | 6.7 | 12.4% | 5.2% | 0.998 | 15.3 | - | - | 0.192 | 0.179 | 0.947 | 0.209 | 0.208 | 0.198 | 1.053 | 0.212 | 0.203 | 1.047 | -0.015 | NOT_TESTED | 25 | | +| 6pb3 | open | pass | P 6 | P 6 | 0.17 | 0.995 | 1.85 | 2.05 | +9.7% | 94.2 | 10.4 | 6.3% | 3.1% | 1.000 | 24.6 | - | - | 0.215 | 0.142 | 0.957 | 0.233 | 0.223 | 0.252 | 0.926 | 0.272 | 0.269 | 1.010 | 0.033 | ACCEPTED | 34 | | +| 6pxb | open | pass | P 31 1 2 | P 32 | 0.23 | 0.994 | 1.39 | 1.75 | +20.4% | 99.7 | 10.0 | 8.8% | 4.7% | 0.999 | 12.0 | - | - | 0.245 | 0.309 | 0.959 | 0.277 | 0.258 | 0.263 | 1.052 | 0.291 | 0.294 | 0.990 | -0.004 | NOT_TESTED | 38 | P 31 1 2 vs reference P 32: accepted alternative P 32 1 2 - a knife-edge this battery does not decide | +| 6pxc | open | pass | I 2 2 2 | I 2 2 2 | 0.23 | 0.996 | 1.43 | 1.60 | +10.5% | 97.2 | 5.9 | 8.5% | 6.3% | 0.996 | 10.1 | - | - | 0.209 | 0.167 | 0.961 | 0.215 | 0.217 | 0.210 | 1.023 | 0.228 | 0.222 | 1.028 | 0.028 | NOT_TESTED | 36 | | +| 6qaj | open | pass | C 2 2 21 | C 2 2 21 | 0.41 | 0.993 | 2.70 | 2.90 | +6.9% | 99.7 | 9.2 | 24.6% | 5.2% | 0.997 | 13.1 | - | - | 0.376 | 0.482 | 0.845 | 0.390 | 0.389 | 0.291 | 1.339 | 0.317 | 0.327 | 0.971 | -0.071 | NOT_TESTED | 67 | | +| 6r72 | open | pass | P 1 21 1 | P 1 21 1 | 1.32 | 0.979 | 4.39 | 3.95 | -11.2% | 99.7 | 7.1 | 14.8% | 4.3% | 0.999 | 15.7 | - | - | 0.374 | 0.060 | 0.580 | 0.383 | 0.378 | 0.321 | 1.194 | 0.398 | 0.380 | 1.048 | -0.244 | NOT_TESTED | 29 | | +| 6rlr | open | pass | P 1 | P 1 | 0.03 | 1.001 | 2.07 | 2.00 | -3.4% | 97.9 | 3.5 | 11.0% | 4.8% | 0.998 | 13.2 | - | - | 0.249 | 0.080 | 0.898 | 0.258 | 0.255 | 0.279 | 0.925 | 0.286 | 0.277 | 1.035 | -0.146 | NOT_TESTED | 18 | | +| 6s1u | open | pass | P 1 21 1 | P 1 21 1 | 0.13 | 1.002 | 1.75 | 1.90 | +7.7% | 95.2 | 3.8 | 22.4% | 5.9% | 0.993 | 13.7 | - | - | 0.204 | 0.092 | 0.955 | 0.208 | 0.211 | 0.235 | 0.884 | 0.245 | 0.251 | 0.978 | -0.029 | NOT_TESTED | 36 | | +| 6toc | open | pass | P 42 2 2 | P 42 | 0.30 | 0.991 | 1.64 | 1.85 | +11.5% | 99.7 | 24.3 | 9.4% | 2.5% | 1.000 | 24.3 | - | - | 0.276 | 0.140 | 0.973 | 0.269 | 0.281 | 0.265 | 1.017 | 0.244 | 0.267 | 0.916 | 0.132 | ACCEPTED | 49 | P 42 2 2 vs reference P 42: accepted alternative P 42 2 2 - a knife-edge this battery does not decide | +| 6ttn | open | pass | P 21 21 21 | P 21 21 21 | 0.41 | 1.012 | 1.08 | 1.12 | +3.1% | 98.5 | 11.9 | 10.7% | 4.5% | 0.999 | 14.5 | - | - | 0.132 | 0.160 | 0.974 | 0.155 | 0.151 | 0.146 | 1.060 | 0.176 | 0.172 | 1.026 | -0.021 | NOT_TESTED | 48 | | +| 6u7g | open | pass | P 1 21 1 | P 1 21 1 | 0.13 | 1.003 | 1.92 | 2.35 | +18.2% | 82.6 | 3.4 | 8.3% | 4.6% | 0.998 | 13.2 | - | - | 0.201 | 0.144 | 0.918 | 0.206 | 0.209 | 0.218 | 0.944 | 0.231 | 0.237 | 0.976 | -0.121 | NOT_TESTED | 70 | | +| 6ukf | open | pass | P 1 21 1 | P 1 21 1 | 0.07 | 1.001 | 0.95 | 1.00 | +4.8% | 90.9 | 7.2 | 10.7% | 6.8% | 0.997 | 9.4 | - | - | 0.166 | 0.086 | 0.959 | 0.162 | 0.168 | 0.166 | 0.975 | 0.183 | 0.191 | 0.958 | 0.011 | NOT_TESTED | 54 | | +| 6vww | open | pass | P 63 | P 63 | 0.20 | 1.005 | 2.00 | 2.20 | +9.1% | 99.6 | 5.6 | 15.9% | 8.8% | 0.993 | 8.6 | - | - | 0.239 | 0.104 | 0.925 | 0.241 | 0.246 | 0.178 | 1.359 | 0.255 | 0.249 | 1.023 | -0.051 | NOT_TESTED | 24 | | +| 6w4h | open | pass | P 31 2 1 | P 31 2 1 | 0.02 | 1.000 | 1.62 | 1.80 | +9.8% | 97.8 | 6.9 | 7.7% | 3.9% | 0.999 | 18.4 | - | - | 0.163 | 0.148 | 0.967 | 0.168 | 0.174 | 0.163 | 1.030 | 0.195 | 0.196 | 0.994 | -0.013 | ACCEPTED | 69 | | +| 6w75 | open | pass | P 31 2 1 | P 32 2 1 | 0.01 | 1.000 | 1.68 | 1.95 | +13.8% | 99.7 | 10.5 | 11.9% | 4.8% | 0.999 | 15.8 | - | - | 0.177 | 0.133 | 0.957 | 0.185 | 0.185 | 0.174 | 1.058 | 0.192 | 0.194 | 0.991 | -0.019 | ACCEPTED | 92 | P 31 2 1 vs reference P 32 2 1 (hand only (needs anomalous)); labelled P 32 2 1 from the model | +| 6wzo | open | pass | P 1 | P 1 | 0.04 | 1.000 | 1.06 | 1.42 | +25.3% | 71.3 | 3.8 | 5.8% | 3.9% | 0.998 | 16.9 | - | - | 0.179 | 0.239 | 0.956 | 0.186 | 0.187 | 0.173 | 1.077 | 0.180 | 0.184 | 0.976 | -0.006 | NOT_TESTED | 57 | | +| 6yqf | open | pass | P 21 21 2 | P 21 21 2 | 0.71 | 1.017 | 3.05 | 3.33 | +8.4% | 99.6 | 5.8 | 50.3% | 19.3% | 0.988 | 3.9 | - | - | 0.432 | 0.334 | 0.693 | 0.470 | 0.469 | 0.367 | 1.280 | 0.440 | 0.420 | 1.047 | -0.233 | NOT_TESTED | 24 | | +| 6z8o | open | pass | P 1 21 1 | P 1 21 1 | 0.63 | 1.016 | 2.23 | 2.20 | -1.2% | 91.6 | 3.6 | 17.1% | 6.2% | 0.995 | 13.8 | - | - | 0.290 | 0.085 | 0.904 | 0.295 | 0.293 | 0.275 | 1.074 | 0.299 | 0.286 | 1.044 | -0.044 | NOT_TESTED | 62 | | +| 6z9g | open | fail | P 1 21 1 | P 1 21 1 | 22.53 | 0.501 | 1.60 | 1.76 | +9.1% | 82.0 | 4.3 | 11.5% | 5.1% | 0.997 | 14.6 | - | - | 0.533 | 0.183 | 0.144 | 0.542 | 0.538 | 0.240 | 2.260 | - | - | - | - | NOT_TESTED | 90 | primitive volume ratio 0.50 | +| 6ze4 | open | pass | P 21 21 21 | P 21 21 21 | 0.59 | 1.009 | 1.30 | 1.60 | +18.4% | 96.6 | 8.1 | 20.4% | 7.0% | 0.992 | 9.2 | - | - | 0.232 | 0.106 | 0.963 | 0.237 | 0.236 | 0.202 | 1.172 | 0.197 | 0.185 | 1.060 | -0.025 | NOT_TESTED | 73 | | +| 6zqr | open | pass | P 4 | P 4 | 0.40 | 1.010 | 1.79 | 1.93 | +7.3% | 99.8 | 8.3 | 19.6% | 8.4% | 0.996 | 8.7 | - | - | 0.190 | 0.125 | 0.951 | 0.198 | 0.206 | 0.191 | 1.036 | 0.212 | 0.201 | 1.052 | -0.143 | NOT_TESTED | 45 | | +| 6zqy | open | pass | P 4 | P 4 | 0.12 | 0.997 | 1.72 | 1.85 | +7.0% | 99.3 | 7.8 | 18.5% | 5.4% | 0.997 | 13.8 | - | - | 0.212 | 0.205 | 0.946 | 0.226 | 0.234 | 0.196 | 1.151 | 0.232 | 0.205 | 1.128 | -0.307 | ACCEPTED | 62 | | +| 6zr0 | open | pass | P 4 | P 4 | 0.08 | 1.001 | 1.73 | 1.94 | +11.1% | 99.5 | 3.9 | 11.1% | 4.1% | 0.997 | 24.3 | - | - | 0.217 | 0.135 | 0.957 | 0.224 | 0.229 | 0.210 | 1.069 | 0.235 | 0.219 | 1.073 | -0.044 | ACCEPTED | 61 | | +| 7arr | open | pass | P 1 | P 1 | 0.28 | 0.993 | 0.92 | 1.10 | +16.7% | 70.5 | 3.6 | 4.7% | 4.4% | 0.993 | 17.3 | - | - | 0.155 | 0.273 | 0.952 | 0.175 | 0.176 | 0.161 | 1.090 | 0.204 | 0.200 | 1.021 | -0.147 | ACCEPTED | 41 | | +| 7atg | open | pass | P 21 21 21 | P 21 21 21 | 0.08 | 1.002 | 0.60 | 0.60 | +0.5% | 86.8 | 3.8 | 5.1% | 7.1% | 0.995 | 22.5 | - | - | 0.135 | 0.231 | 0.443 | 0.199 | 0.196 | 0.095 | 2.092 | - | - | - | -0.046 | NOT_TESTED | 41 | | +| 7bgt | open | pass | P 1 | P 1 | 0.32 | 0.991 | 1.77 | 1.93 | +8.1% | 97.9 | 2.2 | 15.0% | 5.9% | 0.989 | 14.7 | - | - | 0.197 | 0.116 | 0.956 | 0.207 | 0.207 | 0.212 | 0.979 | 0.225 | 0.226 | 0.997 | -0.000 | NOT_TESTED | 32 | | +| 7brr | open | pass | P 1 21 1 | P 1 21 1 | 0.09 | 0.998 | 1.27 | 1.35 | +5.8% | 98.6 | 6.5 | 6.5% | 4.2% | 0.999 | 15.8 | - | - | 0.191 | 0.147 | 0.961 | 0.195 | 0.200 | 0.197 | 0.990 | 0.207 | 0.209 | 0.989 | -0.059 | NOT_TESTED | 25 | | +| 7dkp | open | pass | P 1 21 1 | P 1 21 1 | 0.06 | 1.002 | 1.18 | 1.45 | +18.3% | 66.3 | 7.4 | 11.7% | 4.6% | 0.998 | 26.3 | - | - | 0.158 | 0.083 | 0.969 | 0.161 | 0.163 | 0.160 | 1.001 | 0.170 | 0.172 | 0.986 | 0.025 | NOT_TESTED | 31 | | +| 7k1l | open | unscored | P 6 | P 63 | 0.27 | 0.995 | 1.91 | 2.25 | +15.2% | 98.9 | 6.5 | 16.2% | 8.5% | 0.994 | 9.9 | - | - | 0.264 | 0.094 | 0.910 | 0.271 | 0.270 | 0.192 | 1.414 | 0.271 | 0.266 | 1.016 | 0.093 | NOT_TESTED | 24 | P 6 vs reference P 63: the screw along c is undeterminable from these data (offered P 61 / P 65 / P 62 / P 64 / P 63) | +| 7kcn | open | pass | P 41 2 2 | P 41 2 2 | 0.07 | 1.000 | 1.39 | 1.46 | +4.7% | 92.8 | 20.7 | 7.8% | 7.4% | 0.999 | 11.7 | - | - | 0.172 | 0.153 | 0.961 | 0.189 | 0.189 | 0.182 | 1.039 | 0.209 | 0.198 | 1.054 | -0.020 | NOT_TESTED | 39 | | +| 7l6j | open | pass | I 41 3 2 | I 41 3 2 | 0.00 | 1.000 | 1.51 | 1.78 | +15.4% | 99.7 | 29.0 | 21.1% | 7.5% | 0.999 | 10.5 | - | - | 0.172 | 0.169 | 0.963 | 0.186 | 0.187 | 0.154 | 1.206 | 0.181 | 0.178 | 1.013 | -0.001 | NOT_TESTED | 108 | | +| 7l84 | open | pass | P 41 21 2 | P 43 21 2 | 0.06 | 0.999 | 1.70 | 1.70 | +0.0% | 91.6 | 33.6 | 9.3% | 12.9% | 0.999 | 11.2 | - | - | 0.151 | 0.094 | 0.909 | 0.170 | 0.165 | 0.162 | 1.049 | 0.180 | 0.183 | 0.979 | -0.008 | ACCEPTED | 21 | P 41 21 2 vs reference P 43 21 2 (hand only (needs anomalous)); labelled P 43 21 2 from the model | +| 7mzt | open | fail | P 21 21 21 | P 21 21 2 | 0.56 | 0.998 | 3.25 | 4.07 | +20.1% | 99.5 | 9.2 | 269.6% | 28.3% | 0.968 | 3.4 | - | - | 0.452 | 0.163 | 0.457 | 0.466 | 0.461 | 0.372 | 1.251 | 0.406 | 0.398 | 1.021 | -0.026 | ACCEPTED | 37 | P 21 21 21 vs reference P 21 21 2 | +| 7n0i | open | pass | P 21 21 21 | P 21 21 21 | 0.22 | 0.996 | 1.65 | 2.20 | +25.0% | 99.7 | 6.6 | 14.0% | 5.8% | 0.998 | 10.7 | - | - | 0.290 | 0.358 | 0.948 | 0.310 | 0.311 | 0.271 | 1.144 | 0.294 | 0.284 | 1.036 | 0.059 | NOT_TESTED | 58 | | +| 7n2s | open | pass | P 1 21 1 | P 1 21 1 | 0.07 | 1.000 | 2.57 | 2.37 | -8.5% | 97.6 | 3.4 | 52.8% | 9.1% | 0.903 | 9.6 | - | - | 0.315 | 0.135 | 0.818 | 0.334 | 0.325 | 0.311 | 1.071 | 0.322 | 0.325 | 0.991 | -0.019 | NOT_TESTED | 42 | | +| 7orr | open | pass | I 2 3 | I 21 3 | 0.03 | 0.999 | 1.65 | 1.79 | +8.0% | 99.7 | 16.2 | 6.4% | 4.0% | 1.000 | 25.5 | - | - | 0.181 | 0.139 | 0.964 | 0.190 | 0.190 | 0.184 | 1.028 | 0.181 | 0.181 | 1.002 | 0.045 | NOT_TESTED | 23 | I 2 3 vs reference I 21 3 (UNDECIDABLE from intensities) | +| 7os3 | open | pass | P 21 21 21 | P 21 21 21 | 0.10 | 1.002 | 1.96 | 2.18 | +10.1% | 83.6 | 11.4 | 8.4% | 3.8% | 0.999 | 24.1 | - | - | 0.182 | 0.177 | 0.954 | 0.196 | 0.196 | 0.224 | 0.876 | 0.234 | 0.236 | 0.989 | -0.055 | NOT_TESTED | 40 | | +| 7ou1 | open | pass | P 1 21 1 | P 1 21 1 | 0.04 | 0.999 | 1.41 | 1.65 | +14.8% | 88.1 | 3.4 | 16.1% | 8.3% | 0.991 | 9.5 | - | - | 0.206 | 0.086 | 0.957 | 0.211 | 0.211 | 0.222 | 0.950 | 0.229 | 0.237 | 0.963 | 0.083 | NOT_TESTED | 44 | | +| 7ph1 | open | pass | I 2 2 2 | I 2 2 2 | 0.12 | 0.998 | 1.08 | 1.18 | +8.1% | 99.7 | 7.1 | 11.4% | 5.6% | 0.998 | 15.9 | - | - | 0.160 | 0.165 | 0.967 | 0.178 | 0.175 | 0.166 | 1.071 | 0.208 | 0.209 | 0.996 | 0.050 | NOT_TESTED | 87 | | +| 7pq7 | open | pass | C 1 2 1 | C 1 2 1 | 0.22 | 0.993 | 1.38 | 1.55 | +11.0% | 98.5 | 3.8 | 7.1% | 5.0% | 0.998 | 13.7 | - | - | 0.189 | 0.176 | 0.947 | 0.209 | 0.204 | 0.200 | 1.045 | 0.229 | 0.229 | 0.999 | 0.045 | NOT_TESTED | 16 | | +| 7qij | open | pass | P 21 21 21 | P 21 21 21 | 0.36 | 0.993 | 3.59 | 4.10 | +12.4% | 99.7 | 6.8 | 24.5% | 6.3% | 0.996 | 9.4 | - | - | 0.374 | 0.072 | 0.828 | 0.381 | 0.379 | 0.325 | 1.172 | 0.367 | 0.364 | 1.010 | 0.062 | NOT_TESTED | 94 | | +| 7qis | open | pass | P 61 | P 61 | 0.05 | 0.999 | 1.75 | 1.83 | +4.3% | 99.6 | 9.1 | 20.9% | 6.2% | 0.997 | 17.2 | - | - | 0.169 | 0.198 | 0.965 | 0.192 | 0.191 | 0.189 | 1.019 | 0.214 | 0.209 | 1.022 | 0.064 | NOT_TESTED | 38 | | +| 7raa | open | pass | P 41 21 2 | P 43 21 2 | 0.04 | 0.999 | 2.59 | 2.69 | +3.6% | 98.9 | 35.3 | 17.1% | 5.6% | 1.000 | 11.0 | - | - | 0.299 | 0.180 | 0.906 | 0.336 | 0.303 | 0.294 | 1.143 | 0.306 | 0.308 | 0.993 | -0.035 | ACCEPTED | 186 | P 41 21 2 vs reference P 43 21 2 (hand only (needs anomalous)); labelled P 43 21 2 from the model | +| 7ris | open | pass | P 31 2 1 | P 32 2 1 | 0.08 | 1.000 | 1.51 | 1.72 | +12.1% | 96.3 | 17.9 | 10.0% | 3.3% | 1.000 | 30.2 | - | - | 0.196 | 0.027 | 0.959 | 0.195 | 0.196 | 0.201 | 0.968 | 0.206 | 0.209 | 0.984 | -0.004 | ACCEPTED | 29 | P 31 2 1 vs reference P 32 2 1 (hand only (needs anomalous)); labelled P 32 2 1 from the model | +| 7rji | open | pass | R 3 2:H | H 3 2 | 0.27 | 1.008 | 1.49 | 1.71 | +13.0% | 99.3 | 29.1 | 13.4% | 8.5% | 0.999 | 8.3 | - | - | 0.206 | 0.122 | 0.946 | 0.199 | 0.213 | 0.224 | 0.886 | 0.227 | 0.232 | 0.976 | 0.087 | NOT_TESTED | 30 | | +| 7t5t | open | pass | P 42 21 2 | P 42 21 2 | 0.04 | 0.999 | 1.24 | 1.35 | +8.2% | 98.4 | 12.6 | 6.6% | 4.6% | 0.996 | 16.5 | - | - | 0.164 | 0.254 | 0.943 | 0.193 | 0.189 | 0.168 | 1.147 | 0.193 | 0.189 | 1.017 | 0.025 | NOT_TESTED | 63 | | +| 7tcd | open | pass | C 1 2 1 | C 1 2 1 | 0.20 | 1.004 | 1.72 | 1.70 | -1.4% | 76.2 | 7.2 | 10.0% | 4.6% | 0.999 | 15.0 | - | - | 0.205 | 0.145 | 0.960 | 0.211 | 0.207 | 0.252 | 0.834 | 0.247 | 0.256 | 0.965 | -0.092 | NOT_TESTED | 32 | | +| 7yzx | open | pass | P 63 2 2 | P 63 2 2 | 0.25 | 0.994 | 1.88 | 1.90 | +1.1% | 99.7 | 14.2 | 20.5% | 6.4% | 0.998 | 10.4 | - | - | 0.211 | 0.103 | 0.949 | 0.214 | 0.215 | 0.229 | 0.937 | 0.235 | 0.230 | 1.024 | 0.012 | NOT_TESTED | 53 | | +| 8a1a | open | pass | P 61 | P 65 | 0.42 | 0.987 | 1.95 | 2.05 | +5.1% | 99.7 | 42.8 | 41.5% | 8.0% | 0.998 | 17.9 | - | - | 0.171 | 0.061 | 0.961 | 0.177 | 0.176 | 0.185 | 0.957 | 0.178 | 0.179 | 0.994 | -0.028 | ACCEPTED | 100 | P 61 vs reference P 65 (hand only (needs anomalous)); labelled P 65 from the model | +| 8agq | open | pass | C 1 2 1 | C 1 2 1 | 0.30 | 0.991 | 0.97 | 1.09 | +11.6% | 95.6 | 6.0 | 9.8% | 4.8% | 0.998 | 14.0 | - | - | 0.180 | 0.211 | 0.946 | 0.204 | 0.202 | 0.149 | 1.370 | 0.200 | 0.174 | 1.148 | -0.022 | NOT_TESTED | 29 | | +| 8dqb | open | pass | I 2 3 | I 2 3 | 0.08 | 0.998 | 2.06 | 2.50 | +17.7% | 97.9 | 10.3 | 13.5% | 4.3% | 0.996 | 22.3 | - | - | 0.230 | 0.130 | 0.939 | 0.240 | 0.247 | 0.239 | 1.005 | 0.256 | 0.252 | 1.014 | 0.008 | NOT_TESTED | 26 | | +| 8dyz | open | pass | P 41 21 2 | P 43 21 2 | 0.08 | 0.999 | 1.14 | 1.27 | +10.4% | 64.2 | 3.8 | 3.2% | 2.5% | 0.999 | 37.3 | - | - | 0.120 | 0.060 | 0.973 | 0.121 | 0.122 | 0.133 | 0.909 | 0.171 | 0.178 | 0.959 | -0.045 | ACCEPTED | 17 | P 41 21 2 vs reference P 43 21 2 (hand only (needs anomalous)); labelled P 43 21 2 from the model | +| 8dz7 | open | pass | P 21 21 21 | P 21 21 21 | 0.08 | 0.998 | 1.19 | 1.34 | +11.2% | 37.2 | 3.2 | 3.2% | 2.9% | 0.998 | 33.7 | - | - | 0.119 | 0.067 | 0.965 | 0.129 | 0.123 | 0.136 | 0.945 | 0.173 | 0.170 | 1.015 | -0.008 | NOT_TESTED | 15 | | +| 8egn | open | pass | P 21 21 21 | P 21 21 21 | 0.07 | 1.001 | 1.64 | 1.95 | +15.7% | 92.6 | 5.5 | 6.1% | 3.4% | 0.999 | 22.7 | - | - | 0.200 | 0.158 | 0.944 | 0.214 | 0.208 | 0.219 | 0.976 | 0.231 | 0.231 | 0.999 | 0.002 | NOT_TESTED | 15 | | +| 8iya | open | pass | C 1 2 1 | C 1 2 1 | 0.16 | 0.996 | 1.94 | 2.43 | +20.3% | 98.5 | 5.7 | 20.9% | 12.0% | 0.988 | 5.8 | - | - | 0.247 | 0.043 | 0.948 | 0.236 | 0.249 | 0.247 | 0.956 | 0.249 | 0.254 | 0.979 | 0.001 | NOT_TESTED | 12 | | +| 8k1g | open | pass | I 4 2 2 | I 4 2 2 | 0.82 | 1.019 | 1.63 | 2.09 | +22.2% | 99.7 | 20.9 | 25.7% | 6.8% | 0.999 | 11.6 | - | - | 0.219 | 0.226 | 0.957 | 0.236 | 0.235 | 0.207 | 1.141 | 0.210 | 0.215 | 0.975 | -0.107 | NOT_TESTED | 48 | | +| 8oic | open | pass | P 1 | P 1 | 0.02 | 1.000 | 2.37 | 2.80 | +15.4% | 98.3 | 3.6 | 20.4% | 4.6% | 0.991 | 19.6 | - | - | 0.234 | 0.149 | 0.925 | 0.250 | 0.249 | 0.251 | 0.996 | 0.253 | 0.260 | 0.970 | 0.047 | ACCEPTED | 42 | | +| 8owm | open | pass | P 1 | P 1 | 0.02 | 1.000 | 1.48 | 1.70 | +12.9% | 96.3 | 3.6 | 10.1% | 3.5% | 0.998 | 22.1 | - | - | 0.166 | 0.138 | 0.971 | 0.181 | 0.179 | 0.172 | 1.049 | 0.183 | 0.186 | 0.984 | 0.044 | NOT_TESTED | 69 | | +| 8pqd | open | pass | P 21 21 21 | P 21 21 21 | 0.04 | 0.999 | 1.30 | 1.50 | +13.0% | 94.6 | 13.5 | 9.0% | 5.9% | 0.998 | 14.6 | - | - | 0.185 | 0.235 | 0.942 | 0.205 | 0.201 | 0.194 | 1.057 | 0.210 | 0.210 | 1.000 | 0.034 | NOT_TESTED | 73 | | +| 8qaw | open | pass | R 3:H | H 3 | 0.04 | 0.999 | 1.30 | 1.55 | +16.1% | 99.4 | 10.0 | 12.2% | 7.9% | 0.997 | 10.9 | - | - | 0.144 | 0.187 | 0.957 | 0.156 | 0.159 | 0.161 | 0.964 | 0.176 | 0.183 | 0.962 | 0.021 | NOT_TESTED | 326 | | +| 8qj5 | open | pass | P 1 21 1 | P 1 21 1 | 0.45 | 0.987 | 1.35 | 1.63 | +16.9% | 99.6 | 6.4 | 17.1% | 7.8% | 0.996 | 8.4 | - | - | 0.189 | 0.136 | 0.948 | 0.203 | 0.202 | 0.190 | 1.067 | 0.204 | 0.193 | 1.058 | 0.033 | NOT_TESTED | 59 | | +| 8qq7 | open | pass | P 62 2 2 | P 64 2 2 | 0.65 | 1.017 | 3.16 | 3.62 | +12.8% | 99.4 | 24.9 | 18.8% | 11.0% | 0.993 | 6.3 | - | - | 0.457 | 0.500 | 0.367 | 0.464 | 0.463 | 0.320 | 1.451 | 0.350 | 0.330 | 1.060 | -0.111 | ACCEPTED | 17 | P 62 2 2 vs reference P 64 2 2 (hand only (needs anomalous)); labelled P 64 2 2 from the model | +| 8r5r | open | pass | P 21 21 21 | P 21 21 21 | 0.11 | 0.998 | 2.89 | 3.08 | +6.1% | 99.7 | 10.1 | 27.8% | 4.9% | 0.997 | 15.1 | - | - | 0.274 | 0.141 | 0.878 | 0.290 | 0.281 | 0.252 | 1.150 | 0.265 | 0.252 | 1.052 | -0.134 | NOT_TESTED | 35 | | +| 8rud | open | pass | P 1 21 1 | P 1 21 1 | 0.40 | 0.989 | 1.70 | 2.10 | +19.1% | 99.7 | 6.6 | 29.5% | 6.4% | 0.992 | 12.3 | - | - | 0.244 | 0.111 | 0.929 | 0.255 | 0.252 | 0.262 | 0.975 | 0.240 | 0.262 | 0.917 | 0.059 | NOT_TESTED | 284 | | +| 8s38 | open | pass | I 2 2 2 | I 21 21 21 | 0.16 | 0.996 | 1.63 | 1.89 | +13.8% | 99.5 | 6.7 | 8.6% | 3.5% | 0.999 | 22.3 | - | - | 0.181 | 0.135 | 0.962 | 0.186 | 0.189 | 0.187 | 0.995 | 0.206 | 0.204 | 1.008 | 0.030 | NOT_TESTED | 118 | I 2 2 2 vs reference I 21 21 21 (UNDECIDABLE from intensities) | +| 8sa8 | open | pass | I 1 2 1 | I 1 2 1 | 0.02 | 1.000 | 1.11 | 1.30 | +14.5% | 86.5 | 7.2 | 11.2% | 3.7% | 0.999 | 22.0 | - | - | 0.158 | 0.173 | 0.971 | 0.175 | 0.175 | 0.152 | 1.146 | 0.183 | 0.185 | 0.987 | 0.047 | ACCEPTED | 99 | | +| 8sqo | open | pass | P 4 3 2 | P 4 3 2 | 0.13 | 0.996 | 1.33 | 1.55 | +14.3% | 99.7 | 71.1 | 22.1% | 6.0% | 1.000 | 14.2 | - | - | 0.183 | 0.201 | 0.956 | 0.211 | 0.203 | 0.177 | 1.189 | 0.194 | 0.193 | 1.008 | 0.029 | NOT_TESTED | 72 | | +| 8sqq | open | pass | F 4 3 2 | F 4 3 2 | 0.17 | 0.995 | 1.90 | 2.25 | +15.5% | 99.7 | 39.2 | 25.1% | 5.4% | 0.999 | 17.6 | - | - | 0.211 | 0.105 | 0.963 | 0.206 | 0.218 | 0.245 | 0.839 | 0.242 | 0.258 | 0.940 | 0.018 | NOT_TESTED | 73 | | +| 8sqt | open | pass | F 4 3 2 | F 4 3 2 | 0.11 | 0.997 | 1.89 | 2.20 | +14.3% | 99.7 | 21.6 | 18.9% | 3.5% | 0.999 | 29.6 | - | - | 0.221 | 0.117 | 0.966 | 0.224 | 0.232 | 0.262 | 0.855 | 0.268 | 0.257 | 1.042 | 0.003 | NOT_TESTED | 30 | | +| 8t7r | open | pass | C 1 2 1 | C 1 2 1 | 0.28 | 0.991 | 3.22 | 3.84 | +16.2% | 99.1 | 4.0 | 35.3% | 8.0% | 0.984 | 9.9 | - | - | 0.288 | 0.061 | 0.756 | 0.288 | 0.293 | 0.263 | 1.096 | 0.284 | 0.283 | 1.002 | 0.041 | NOT_TESTED | 66 | | +| 8tha | open | pass | P 62 | P 64 | 0.17 | 0.995 | 1.34 | 1.68 | +20.1% | 99.3 | 19.4 | 13.7% | 3.3% | 1.000 | 25.9 | - | - | 0.208 | 0.162 | 0.960 | 0.225 | 0.220 | 0.215 | 1.047 | 0.215 | 0.220 | 0.978 | 0.045 | ACCEPTED | 23 | P 62 vs reference P 64 (hand only (needs anomalous)); labelled P 64 from the model | +| 8tyy | open | pass | F 4 3 2 | F 4 3 2 | 0.06 | 1.002 | 1.28 | 1.68 | +23.6% | 98.0 | 38.5 | 16.5% | 5.2% | 0.999 | 16.3 | - | - | 0.160 | 0.155 | 0.970 | 0.180 | 0.176 | 0.163 | 1.108 | 0.161 | 0.162 | 0.990 | 0.002 | NOT_TESTED | 179 | | +| 8u0i | open | pass | P 41 21 2 | P 43 21 2 | 0.06 | 1.001 | 1.40 | 1.54 | +9.1% | 98.9 | 11.3 | 7.9% | 3.6% | 0.999 | 16.1 | - | - | 0.176 | 0.065 | 0.964 | 0.180 | 0.180 | 0.210 | 0.856 | 0.211 | 0.210 | 1.004 | 0.016 | ACCEPTED | 24 | P 41 21 2 vs reference P 43 21 2 (hand only (needs anomalous)); labelled P 43 21 2 from the model | +| 8v4o | open | pass | P 61 2 2 | P 61 2 2 | 0.06 | 1.001 | 2.10 | 2.70 | +22.1% | 99.7 | 20.4 | 31.9% | 6.0% | 0.998 | 15.5 | - | - | 0.241 | 0.157 | 0.929 | 0.253 | 0.254 | 0.240 | 1.056 | 0.246 | 0.247 | 0.996 | 0.058 | NOT_TESTED | 92 | | +| 8xbp | open | pass | C 1 2 1 | C 1 2 1 | 0.15 | 0.999 | 1.72 | 1.99 | +13.7% | 86.4 | 6.8 | 10.8% | 4.4% | 0.999 | 16.2 | - | - | 0.315 | 0.155 | 0.890 | 0.325 | 0.319 | 0.288 | 1.127 | 0.328 | 0.310 | 1.059 | 0.207 | NOT_TESTED | 22 | | +| 8xte | open | pass | P 31 2 1 | P 32 | 0.06 | 1.001 | 1.64 | 1.99 | +17.8% | 99.5 | 9.4 | 25.1% | 7.8% | 0.996 | 12.4 | - | - | 0.275 | 0.128 | 0.913 | 0.288 | 0.285 | 0.219 | 1.311 | 0.290 | 0.290 | 0.999 | 0.043 | NOT_TESTED | 37 | P 31 2 1 vs reference P 32: accepted alternative P 31 2 1 - a knife-edge this battery does not decide | +| 8xtf | open | pass | R 3 2:H | H 3 2 | 0.22 | 1.006 | 1.84 | 2.13 | +13.5% | 99.6 | 20.6 | 85.8% | 18.7% | 0.987 | 7.5 | - | - | 0.191 | 0.042 | 0.964 | 0.185 | 0.193 | 0.210 | 0.882 | 0.208 | 0.218 | 0.956 | -0.003 | NOT_TESTED | 38 | | +| 8xtg | open | pass | P 31 2 1 | P 32 | 0.05 | 1.002 | 1.54 | 2.00 | +22.8% | 99.7 | 9.9 | 27.4% | 10.7% | 0.994 | 7.2 | - | - | 0.279 | 0.133 | 0.902 | 0.287 | 0.287 | 0.211 | 1.361 | 0.280 | 0.280 | 1.002 | 0.037 | ACCEPTED | 52 | P 31 2 1 vs reference P 32: accepted alternative P 31 2 1 - a knife-edge this battery does not decide | +| 8y74 | open | pass | C 1 2 1 | C 1 2 1 | 0.34 | 0.992 | 1.71 | 1.90 | +10.3% | 98.3 | 6.3 | 12.3% | 7.9% | 0.997 | 8.8 | - | - | 0.212 | 0.125 | 0.939 | 0.224 | 0.220 | 0.240 | 0.933 | 0.255 | 0.267 | 0.957 | -0.048 | NOT_TESTED | 24 | | +| 8ys9 | open | pass | P 21 21 21 | P 21 21 21 | 0.16 | 0.998 | 1.35 | 1.46 | +7.7% | 99.7 | 13.5 | 11.0% | 4.6% | 0.999 | 15.4 | - | - | 0.173 | 0.138 | 0.960 | 0.191 | 0.184 | 0.197 | 0.970 | 0.211 | 0.213 | 0.992 | 0.056 | NOT_TESTED | 31 | | +| 9b22 | open | pass | P 1 21 1 | P 1 21 1 | 0.07 | 1.002 | 1.18 | 1.30 | +9.1% | 84.3 | 6.2 | 6.0% | 3.5% | 0.999 | 16.3 | - | - | 0.161 | 0.199 | 0.966 | 0.181 | 0.178 | 0.164 | 1.107 | 0.200 | 0.194 | 1.032 | -0.122 | NOT_TESTED | 21 | | +| 9bn8 | open | pass | P 41 | P 41 | 0.09 | 0.997 | 1.20 | 1.35 | +11.3% | 94.4 | 11.5 | 7.5% | 3.9% | 0.999 | 19.8 | - | - | 0.151 | 0.165 | 0.967 | 0.172 | 0.169 | 0.158 | 1.093 | 0.182 | 0.182 | 1.002 | 0.015 | ACCEPTED | 30 | | +| 9c18 | open | pass | P 1 | P 1 | 0.61 | 0.985 | 1.76 | 1.90 | +7.5% | 97.5 | 3.6 | 23.4% | 8.5% | 0.988 | 10.2 | - | - | 0.222 | 0.062 | 0.947 | 0.221 | 0.225 | 0.229 | 0.965 | 0.229 | 0.234 | 0.976 | -0.015 | ACCEPTED | 13 | | +| 9chw | open | pass | P 61 | P 61 | 0.04 | 1.001 | 1.60 | 2.16 | +25.9% | 72.5 | 3.1 | 5.9% | 3.5% | 0.998 | 20.9 | - | - | 0.180 | 0.134 | 0.961 | 0.185 | 0.187 | 0.210 | 0.884 | 0.215 | 0.214 | 1.005 | -0.012 | NOT_TESTED | 71 | | +| 9crw | open | pass | P 1 21 1 | P 1 21 1 | 0.23 | 0.994 | 2.29 | 2.49 | +8.2% | 97.5 | 7.0 | 8.7% | 3.9% | 0.999 | 15.5 | - | - | 0.257 | 0.139 | 0.952 | 0.271 | 0.269 | 0.278 | 0.974 | 0.298 | 0.293 | 1.018 | -0.019 | NOT_TESTED | 19 | | +| 9e2t | open | pass | P 1 | P 1 | 0.06 | 1.001 | 2.33 | 2.28 | -2.4% | 91.7 | 5.7 | 28.1% | 8.1% | 0.990 | 6.9 | - | - | 0.248 | 0.113 | 0.940 | 0.254 | 0.254 | 0.242 | 1.050 | 0.257 | 0.259 | 0.990 | -0.015 | NOT_TESTED | 70 | | +| 9ea5 | open | pass | P 1 21 1 | P 1 21 1 | 0.85 | 0.998 | 1.64 | 2.00 | +17.9% | 90.9 | 6.7 | 17.9% | 4.3% | 0.997 | 26.6 | - | - | 0.198 | 0.108 | 0.954 | 0.203 | 0.205 | 0.207 | 0.982 | 0.212 | 0.215 | 0.985 | 0.040 | ACCEPTED | 96 | | +| 9fcf | open | pass | P 4 | P 4 | 0.04 | 1.001 | 1.81 | 2.36 | +23.3% | 99.5 | 10.4 | 28.6% | 8.4% | 0.995 | 7.8 | - | - | 0.285 | 0.057 | 0.923 | 0.293 | 0.287 | 0.257 | 1.139 | 0.265 | 0.276 | 0.960 | 0.036 | ACCEPTED | 142 | | +| 9fcg | open | pass | P 4 | P 4 | 0.10 | 1.001 | 1.39 | 1.54 | +9.7% | 90.6 | 11.9 | 15.4% | 8.6% | 0.996 | 9.3 | - | - | 0.179 | 0.085 | 0.956 | 0.184 | 0.185 | 0.198 | 0.929 | 0.196 | 0.201 | 0.975 | 0.063 | ACCEPTED | 138 | | +| 9fhc | open | pass | I 2 3 | I 2 3 | 0.25 | 0.993 | 1.97 | 2.20 | +10.5% | 97.0 | 18.7 | 22.4% | 5.4% | 0.995 | 11.6 | - | - | 0.245 | 0.043 | 0.914 | 0.250 | 0.249 | 0.238 | 1.052 | 0.253 | 0.240 | 1.055 | -0.047 | ACCEPTED | 104 | | +| 9gdj | open | pass | P 41 21 2 | P 41 21 2 | 0.16 | 0.996 | 1.40 | 1.47 | +4.8% | 99.7 | 13.1 | 10.9% | 6.2% | 0.999 | 12.8 | - | - | 0.153 | 0.238 | 0.968 | 0.178 | 0.184 | 0.175 | 1.020 | 0.203 | 0.195 | 1.045 | -0.032 | NOT_TESTED | 355 | | +| 9gjx | open | pass | P 1 21 1 | P 1 21 1 | 0.13 | 0.997 | 2.15 | 2.40 | +10.3% | 99.1 | 6.9 | 12.5% | 2.9% | 0.999 | 35.2 | - | - | 0.188 | 0.176 | 0.949 | 0.225 | 0.216 | 0.219 | 1.026 | 0.239 | 0.227 | 1.054 | -0.085 | NOT_TESTED | 33 | | +| 9gqg | open | pass | P 31 2 1 | P 32 2 1 | 0.01 | 1.000 | 1.82 | 2.00 | +9.1% | 99.7 | 7.9 | 11.4% | 6.4% | 0.998 | 13.2 | - | - | 0.233 | 0.112 | 0.909 | 0.241 | 0.239 | 0.261 | 0.922 | 0.273 | 0.267 | 1.026 | 0.007 | ACCEPTED | 59 | P 31 2 1 vs reference P 32 2 1 (hand only (needs anomalous)); labelled P 32 2 1 from the model | +| 9h0q | open | pass | R 3 2:H | H 3 2 | 0.41 | 0.990 | 2.13 | 2.55 | +16.4% | 99.7 | 10.9 | 15.4% | 4.4% | 0.998 | 18.7 | - | - | 0.203 | 0.183 | 0.927 | 0.233 | 0.238 | 0.222 | 1.048 | 0.246 | 0.236 | 1.043 | -0.029 | NOT_TESTED | 72 | | +| 9hnc | open | pass | P 1 2 1 | P 1 2 1 | 0.13 | 0.998 | 1.65 | 1.88 | +12.2% | 92.4 | 6.8 | 52.8% | 44.3% | 0.882 | 1.4 | - | - | 0.461 | 0.159 | 0.545 | 0.465 | 0.468 | 0.211 | 2.204 | 0.469 | 0.246 | 1.903 | -0.625 | ACCEPTED | 80 | | +| 9hs7 | open | pass | P 61 | P 65 | 0.08 | 0.999 | 1.68 | 1.70 | +0.9% | 99.7 | 10.2 | 11.8% | 5.1% | 0.999 | 13.3 | - | - | 0.251 | 0.584 | 0.963 | 0.279 | 0.268 | 0.239 | 1.169 | 0.250 | 0.255 | 0.979 | -0.080 | ACCEPTED | 27 | P 61 vs reference P 65 (hand only (needs anomalous)); labelled P 65 from the model | +| 9i0a | open | pass | P 21 21 2 | P 21 21 2 | 0.43 | 1.008 | 1.81 | 2.22 | +18.4% | 99.0 | 12.8 | 16.8% | 5.1% | 0.999 | 13.8 | - | - | 0.228 | 0.174 | 0.952 | 0.239 | 0.241 | 0.237 | 1.011 | 0.256 | 0.259 | 0.987 | 0.160 | NOT_TESTED | 59 | | +| 9i80 | open | pass | P 41 | P 41 | 0.07 | 1.002 | 1.61 | 1.95 | +17.3% | 99.7 | 13.2 | 23.2% | 11.0% | 0.991 | 6.7 | - | - | 0.216 | 0.151 | 0.901 | 0.221 | 0.226 | 0.203 | 1.090 | 0.212 | 0.207 | 1.025 | -0.026 | NOT_TESTED | 158 | | +| 9ig7 | open | pass | P 21 21 2 | P 21 21 2 | 0.17 | 0.996 | 2.03 | 2.60 | +22.0% | 99.7 | 13.8 | 18.7% | 7.4% | 0.991 | 11.0 | - | - | 0.235 | 0.179 | 0.918 | 0.255 | 0.246 | 0.259 | 0.985 | 0.287 | 0.277 | 1.036 | 0.014 | NOT_TESTED | 85 | | +| 9ih9 | open | pass | C 1 2 1 | C 1 2 1 | 0.21 | 0.997 | 1.41 | 1.70 | +17.3% | 96.1 | 2.4 | 11.7% | 6.3% | 0.995 | 12.6 | - | - | 0.198 | 0.088 | 0.952 | 0.207 | 0.204 | 0.203 | 1.022 | 0.205 | 0.202 | 1.014 | -0.025 | NOT_TESTED | 24 | | +| 9jzo | open | pass | P 1 | P 1 | 0.22 | 0.996 | 1.15 | 1.40 | +18.0% | 61.5 | 3.8 | 9.2% | 8.3% | 0.994 | 8.4 | - | - | 0.175 | 0.084 | 0.961 | 0.175 | 0.177 | 0.195 | 0.899 | 0.197 | 0.194 | 1.015 | -0.020 | NOT_TESTED | 17 | | +| 9khr | open | pass | P 21 21 21 | P 21 21 21 | 0.12 | 0.997 | 1.38 | 2.00 | +30.9% | 96.9 | 6.5 | 18.9% | 11.6% | 0.994 | 9.4 | - | - | 0.248 | 0.114 | 0.944 | 0.261 | 0.257 | 0.235 | 1.113 | 0.245 | 0.249 | 0.984 | 0.041 | NOT_TESTED | 33 | | +| 9mh4 | open | pass | P 21 3 | P 21 3 | 0.31 | 0.991 | 2.78 | 3.05 | +8.9% | 99.8 | 40.5 | 20.2% | 4.9% | 1.000 | 13.8 | - | - | 0.229 | 0.171 | 0.922 | 0.232 | 0.238 | 0.215 | 1.082 | 0.242 | 0.238 | 1.017 | 0.005 | NOT_TESTED | 29 | | +| 9min | open | fail | P 21 21 2 | P 21 21 21 | 36.97 | 0.494 | 1.87 | 2.05 | +8.9% | 99.6 | 23.4 | 26.6% | 6.8% | 0.998 | 10.4 | - | - | 0.573 | 0.197 | 0.242 | 0.578 | 0.581 | 0.267 | 2.161 | - | - | - | - | NOT_TESTED | 77 | primitive volume ratio 0.49 | +| 9o0h | open | pass | P 21 21 21 | P 21 21 21 | 0.25 | 0.994 | 2.01 | 2.24 | +10.4% | 99.7 | 11.8 | 52.6% | 11.5% | 0.985 | 6.1 | - | - | 0.247 | 0.036 | 0.944 | 0.255 | 0.248 | 0.249 | 1.025 | 0.262 | 0.267 | 0.981 | 0.046 | NOT_TESTED | 64 | | +| 9p7q | open | pass | C 1 2 1 | C 1 2 1 | 0.14 | 1.003 | 1.81 | 2.21 | +17.9% | 90.8 | 2.6 | 25.7% | 9.1% | 0.988 | 10.6 | - | - | 0.270 | 0.029 | 0.945 | 0.285 | 0.270 | 0.259 | 1.100 | 0.261 | 0.262 | 0.997 | -0.062 | NOT_TESTED | 15 | | +| 9pbb | open | pass | C 1 2 1 | C 1 2 1 | 0.15 | 0.997 | 1.83 | 2.17 | +15.5% | 78.2 | 2.5 | 20.7% | 7.2% | 0.995 | 14.5 | - | - | 0.236 | 0.038 | 0.973 | 0.236 | 0.238 | 0.223 | 1.058 | 0.234 | 0.238 | 0.984 | -0.129 | NOT_TESTED | 19 | | +| 9q41 | open | pass | C 2 2 21 | C 2 2 21 | 0.20 | 1.004 | 1.69 | 1.95 | +13.2% | 99.7 | 6.9 | 27.3% | 13.5% | 0.984 | 7.8 | - | - | 0.187 | 0.093 | 0.957 | 0.191 | 0.193 | 0.213 | 0.894 | 0.210 | 0.214 | 0.983 | 0.055 | NOT_TESTED | 57 | | +| 9q66 | open | pass | P 1 21 1 | P 1 21 1 | 0.34 | 0.990 | 2.04 | 2.01 | -1.3% | 99.7 | 7.1 | 31.7% | 8.2% | 0.990 | 13.0 | - | - | 0.204 | 0.085 | 0.903 | 0.216 | 0.216 | 0.266 | 0.811 | 0.280 | 0.275 | 1.016 | -0.021 | NOT_TESTED | 68 | | +| 9qw8 | open | pass | P 1 | P 1 | 0.12 | 1.001 | 1.59 | 1.80 | +11.8% | 97.1 | 3.5 | 14.6% | 6.5% | 0.994 | 11.3 | - | - | 0.237 | 0.097 | 0.914 | 0.248 | 0.242 | 0.243 | 1.023 | 0.255 | 0.249 | 1.022 | -0.002 | NOT_TESTED | 73 | | +| 9rci | open | pass | P 1 | P 1 | 97.59 | 1.990 | 1.82 | 1.66 | -9.5% | 85.4 | 2.6 | 29.4% | 13.2% | 0.942 | 6.0 | - | - | 0.576 | 0.086 | 0.277 | 0.571 | 0.579 | 0.283 | 2.017 | - | - | - | - | NOT_TESTED | 25 | P 1 vs reference P 1: accepted alternative cell 35.869 39.297 199.976 - a knife-edge this battery does not decide | +| 9rcs | open | pass | P 1 21 1 | P 1 21 1 | 0.93 | 1.022 | 3.37 | 3.01 | -12.0% | 99.7 | 7.0 | 24.3% | 11.5% | 0.992 | 6.0 | - | - | 0.383 | 0.194 | 0.447 | 0.442 | 0.400 | 0.317 | 1.395 | 0.361 | 0.362 | 0.996 | -0.273 | ACCEPTED | 55 | | +| 9rp9 | open | pass | C 1 2 1 | C 1 2 1 | 0.16 | 0.996 | 1.91 | 2.10 | +8.9% | 99.6 | 5.5 | 16.1% | 4.6% | 0.997 | 28.8 | - | - | 0.206 | 0.082 | 0.942 | 0.224 | 0.215 | 0.222 | 1.012 | 0.230 | 0.228 | 1.012 | 0.047 | NOT_TESTED | 23 | | +| 9sl0 | open | pass | P 21 21 21 | P 21 21 21 | 0.37 | 0.990 | 1.38 | 1.60 | +14.1% | 99.5 | 12.5 | 7.6% | 3.7% | 1.000 | 20.0 | - | - | 0.245 | 0.148 | 0.913 | 0.256 | 0.257 | 0.264 | 0.971 | 0.271 | 0.270 | 1.003 | -0.007 | NOT_TESTED | 50 | | +| 9t6s | open | pass | P 21 21 21 | P 21 21 21 | 0.06 | 1.000 | 1.76 | 2.00 | +12.2% | 96.2 | 5.2 | 10.5% | 3.6% | 0.999 | 24.9 | - | - | 0.220 | 0.029 | 0.970 | 0.236 | 0.221 | 0.241 | 0.976 | 0.240 | 0.252 | 0.953 | 0.075 | NOT_TESTED | 28 | | +| 9upt | open | pass | P 6 | P 6 | 0.20 | 0.994 | 2.03 | 2.37 | +14.3% | 99.7 | 5.8 | 25.9% | 11.9% | 0.988 | 6.4 | - | - | 0.227 | 0.083 | 0.934 | 0.229 | 0.232 | 0.215 | 1.065 | 0.230 | 0.216 | 1.064 | -0.051 | NOT_TESTED | 76 | | +| 9vyb | open | pass | P 21 21 21 | P 21 21 21 | 0.40 | 0.989 | 1.71 | 2.12 | +19.5% | 90.6 | 10.3 | 8.0% | 3.7% | 1.000 | 16.7 | - | - | 0.245 | 0.075 | 0.933 | 0.258 | 0.250 | 0.269 | 0.958 | 0.263 | 0.264 | 0.997 | -0.017 | NOT_TESTED | 41 | | +| 9w3y | open | pass | P 21 21 21 | P 21 21 21 | 0.25 | 1.004 | 1.20 | 1.50 | +20.3% | 99.7 | 6.8 | 23.4% | 5.3% | 0.997 | 19.9 | - | - | 0.186 | 0.108 | 0.972 | 0.200 | 0.199 | 0.198 | 1.012 | 0.216 | 0.215 | 1.003 | 0.052 | NOT_TESTED | 17 | | +| 9yl4 | open | pass | P 21 21 21 | P 21 21 21 | 0.08 | 1.001 | 3.61 | 3.70 | +2.5% | 99.7 | 13.4 | 32.0% | 9.9% | 0.985 | 9.5 | - | - | 0.295 | 0.067 | 0.939 | 0.306 | 0.299 | 0.283 | 1.080 | 0.284 | 0.288 | 0.987 | -0.024 | NOT_TESTED | 85 | | +| 9yzk | open | pass | I 1 2 1 | I 1 2 1 | 0.20 | 0.996 | 3.86 | 5.10 | +24.3% | 98.6 | 3.4 | 29.7% | 7.2% | 0.996 | 7.8 | - | - | 0.375 | 0.289 | 0.916 | 0.394 | 0.394 | 0.304 | 1.296 | 0.313 | 0.314 | 0.998 | 0.036 | ACCEPTED | 15 | | +| 9z44 | open | pass | I 1 2 1 | I 1 2 1 | 1.35 | 0.964 | 6.86 | 7.20 | +4.8% | 98.2 | 3.3 | 26.6% | 17.1% | 0.953 | 5.9 | - | - | 0.336 | 0.166 | 0.788 | 0.325 | 0.345 | 0.345 | 0.941 | 0.364 | 0.350 | 1.039 | -0.009 | ACCEPTED | 32 | | +| 9z72 | open | pass | P 31 2 1 | P 31 2 1 | 0.15 | 1.004 | 1.97 | 2.38 | +17.1% | 99.6 | 10.0 | 76.6% | 13.3% | 0.991 | 11.8 | - | - | 0.241 | 0.062 | 0.901 | 0.256 | 0.245 | 0.269 | 0.950 | 0.275 | 0.277 | 0.993 | 0.033 | NOT_TESTED | 152 | | +| 9zlo | open | pass | P 21 21 21 | P 21 21 21 | 0.39 | 0.995 | 1.58 | 2.00 | +20.9% | 99.7 | 12.9 | 12.0% | 3.9% | 0.999 | 23.0 | - | - | 0.235 | 0.054 | 0.945 | 0.241 | 0.238 | 0.219 | 1.102 | 0.224 | 0.222 | 1.007 | -0.040 | NOT_TESTED | 26 | | +| 9zm0 | open | pass | P 1 21 1 | P 1 21 1 | 0.16 | 0.997 | 1.81 | 2.10 | +14.0% | 99.7 | 6.5 | 23.7% | 10.2% | 0.996 | 8.6 | - | - | 0.262 | 0.121 | 0.952 | 0.283 | 0.266 | 0.298 | 0.952 | 0.301 | 0.302 | 0.996 | -0.023 | NOT_TESTED | 11 | | +| 9zmu | open | pass | P 61 2 2 | P 65 2 2 | 0.25 | 1.007 | 1.71 | 1.98 | +13.6% | 99.6 | 34.3 | 34.0% | 7.7% | 0.999 | 10.8 | - | - | 0.276 | 0.172 | 0.895 | 0.285 | 0.289 | 0.294 | 0.969 | 0.302 | 0.304 | 0.995 | 0.013 | ACCEPTED | 39 | P 61 2 2 vs reference P 65 2 2 (hand only (needs anomalous)); labelled P 65 2 2 from the model | +| cuhf2 | open | unscored | P 2 2 2 | - | - | - | 0.52 | - | - | 77.9 | 8.0 | 17.4% | 16.3% | 0.988 | 3.8 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 56 | no reference to score against | +| cytidine | open | pass | P 21 21 21 | P 21 21 21 | 0.31 | 0.995 | 0.58 | - | - | 90.7 | 3.4 | 20.6% | 18.9% | 0.980 | 4.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 29 | | +| dnba | open | pass | C 1 2/c 1 | C 1 2/c 1 | 0.06 | 0.999 | 0.81 | 0.48 | -68.3% | 75.8 | 2.8 | 16.7% | 14.2% | 0.979 | 4.3 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 10 | | +| lalanine | open | pass | P 21 21 21 | P 21 21 21 | 0.80 | 0.992 | 0.65 | - | - | 78.0 | 2.7 | 12.1% | 10.3% | 0.994 | 7.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 14 | | +| metformin | open | pass | P 1 21/c 1 | P 1 21/c 1 | 0.17 | 1.001 | 0.51 | 0.45 | -12.9% | 73.5 | 4.8 | 6.4% | 6.3% | 0.998 | 13.6 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 8 | | +| nidppe | open | pass | P 1 21/c 1 | P 1 21/c 1 | 0.28 | 0.997 | 0.51 | 0.77 | +34.3% | 66.6 | 5.3 | 8.2% | 4.2% | 0.998 | 25.6 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 9 | | +| cytc_x06da_1 | inhouse | pass | P 31 2 1 | P 31 2 1 | 0.04 | 0.999 | 1.70 | 1.88 | +9.2% | 98.7 | 18.0 | 9.3% | 3.9% | 1.000 | 17.4 | 24.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | 14 | | +| cytc_x06da_2 | inhouse | pass | P 31 2 1 | P 31 2 1 | 0.11 | 1.002 | 1.57 | 1.69 | +7.2% | 99.6 | 17.4 | 9.8% | 3.5% | 1.000 | 21.1 | 27.0 | - | - | - | - | - | - | - | - | - | - | - | - | - | 20 | | +| cytc_x10sa | inhouse | pass | P 31 2 1 | P 31 2 1 | 0.13 | 1.003 | 1.95 | 2.27 | +13.9% | 99.6 | 20.1 | 20.1% | 4.0% | 0.999 | 25.8 | 31.8 | - | - | - | - | - | - | - | - | - | - | - | - | - | 26 | | +| insu_H_x06da_notwin | inhouse | pass | R 3:H | R 3 | 0.06 | 0.999 | 1.42 | 1.54 | +8.4% | 91.4 | 8.7 | 6.3% | 4.3% | 0.999 | 20.3 | 17.7 | - | - | - | - | - | - | - | - | - | - | - | - | - | 11 | | +| insu_H_x06da_twin | inhouse | pass | R 3:H | R 3 | 0.04 | 1.001 | 1.38 | 1.46 | +4.9% | 90.8 | 8.6 | 12.8% | 11.7% | 0.994 | 6.3 | 6.8 | - | - | - | - | - | - | - | - | - | - | - | - | - | 12 | | +| insu_I_x06da_13keV | inhouse | pass | I 2 3 | I 2 3 | 0.27 | 1.008 | 1.48 | 1.64 | +9.5% | 99.9 | 38.9 | 31.8% | 8.3% | 0.999 | - | 18.8 | - | - | - | - | - | - | - | - | - | - | - | - | - | 16 | | +| insu_I_x06da_5keV | inhouse | pass | I 2 3 | I 2 3 | 0.03 | 0.999 | 2.43 | 2.45 | +0.9% | 95.4 | 28.1 | 6.5% | 4.4% | 1.000 | 28.3 | 17.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | 9 | | +| insu_I_x06da_5keV_2 | inhouse | pass | I 2 3 | I 2 3 | 0.05 | 0.999 | 2.42 | 2.45 | +1.1% | 93.9 | 29.4 | 8.3% | 5.9% | 1.000 | 24.4 | 20.0 | - | - | - | - | - | - | - | - | - | - | - | - | - | 8 | | +| insu_I_x06da_6keV | inhouse | pass | I 2 3 | I 2 3 | 0.04 | 0.999 | 2.03 | 2.04 | +0.7% | 95.6 | 28.4 | 6.0% | 4.8% | 1.000 | 20.6 | 17.9 | - | - | - | - | - | - | - | - | - | - | - | - | - | 9 | | +| insu_I_x06da_low_isa | inhouse | pass | I 2 3 | I 2 3 | 0.08 | 1.002 | 1.44 | 1.30 | -10.4% | 99.9 | 31.8 | 22.0% | 15.9% | 0.998 | 5.6 | 4.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | 16 | | +| insu_I_x06da_ref | inhouse | pass | I 2 3 | I 2 3 | 0.13 | 1.004 | 1.40 | 1.62 | +13.5% | 99.9 | 24.5 | 27.8% | 6.0% | 0.999 | 31.1 | 25.1 | - | - | - | - | - | - | - | - | - | - | - | - | - | 15 | | +| insu_I_x06da_weak | inhouse | pass | I 2 3 | I 2 3 | 0.79 | 1.024 | 1.64 | 1.81 | +9.2% | 99.5 | 40.1 | 16.7% | 5.0% | 1.000 | 14.9 | 18.9 | - | - | - | - | - | - | - | - | - | - | - | - | - | 20 | | +| lysoI_micromax_mono | inhouse | pass | P 41 21 2 | P 43 21 2 | 0.06 | 1.001 | 1.35 | 1.65 | +17.9% | 98.8 | 9.5 | 6.9% | 2.9% | 1.000 | 37.3 | 31.4 | - | - | - | - | - | - | - | - | - | - | - | - | - | 25 | P 41 21 2 vs reference P 43 21 2 (screws not judged against XDS) | +| lysoI_micromax_pink | inhouse | pass | P 41 21 2 | P 43 21 2 | 0.09 | 1.002 | 1.42 | 1.65 | +13.9% | 99.4 | 10.3 | 7.4% | 3.1% | 1.000 | 33.5 | 29.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | 17 | P 41 21 2 vs reference P 43 21 2 (screws not judged against XDS) | +| lyso_micromax_mono | inhouse | pass | P 41 21 2 | P 43 21 2 | 0.16 | 1.003 | 1.20 | 1.50 | +19.9% | 83.5 | 8.4 | 4.9% | 2.5% | 1.000 | 39.0 | 39.9 | - | - | - | - | - | - | - | - | - | - | - | - | - | 24 | P 41 21 2 vs reference P 43 21 2 (screws not judged against XDS) | +| lyso_micromax_pink | inhouse | pass | P 41 21 2 | P 43 21 2 | 0.16 | 1.004 | 1.26 | 1.45 | +13.2% | 91.7 | 8.6 | 5.2% | 2.5% | 1.000 | 35.7 | 37.4 | - | - | - | - | - | - | - | - | - | - | - | - | - | 17 | P 41 21 2 vs reference P 43 21 2 (screws not judged against XDS) | +| lyso_x06da_5keV | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.29 | 1.009 | 2.43 | 2.45 | +0.7% | 86.3 | 19.7 | 6.6% | 5.4% | 0.999 | 25.6 | 19.4 | - | - | - | - | - | - | - | - | - | - | - | - | - | 8 | | +| lyso_x06da_atten_wedge | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.25 | 1.006 | 1.16 | 1.26 | +8.5% | 98.5 | 23.0 | 35.8% | 7.6% | 0.999 | 13.0 | 16.6 | - | - | - | - | - | - | - | - | - | - | - | - | - | 19 | | +| lyso_x06da_half_image | inhouse | pass | P 41 21 2 | P 43 21 2 | 0.71 | 1.021 | 1.56 | 1.65 | +5.2% | 99.7 | 9.8 | 101.0% | 26.2% | 0.972 | 7.3 | 6.6 | - | - | - | - | - | - | - | - | - | - | - | - | - | 47 | P 41 21 2 vs reference P 43 21 2 (screws not judged against XDS) | +| lyso_x06da_ice | inhouse | pass | P 41 21 2 | P 4 2 2 | 0.09 | 1.002 | 1.34 | 1.43 | +6.4% | 99.7 | 19.4 | 18.8% | 4.6% | 0.999 | 22.7 | 23.3 | - | - | - | - | - | - | - | - | - | - | - | - | - | 17 | P 41 21 2 vs reference P 4 2 2 (screws not judged against XDS) | +| lyso_x06da_ref | inhouse | pass | P 41 21 2 | P 4 2 2 | 0.04 | 1.000 | 0.99 | 1.20 | +17.3% | 85.7 | 22.2 | 4.8% | 2.7% | 1.000 | 29.4 | 28.3 | - | - | - | - | - | - | - | - | - | - | - | - | - | 18 | P 41 21 2 vs reference P 4 2 2 (screws not judged against XDS) | +| lyso_x10sa_90deg_1 | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.12 | 0.998 | 1.86 | 1.97 | +5.3% | 99.8 | 6.4 | 18.1% | 4.8% | 0.997 | 18.8 | 20.8 | - | - | - | - | - | - | - | - | - | - | - | - | - | 10 | | +| lyso_x10sa_90deg_2 | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.16 | 0.997 | 1.88 | 1.97 | +4.5% | 99.6 | 6.4 | 17.7% | 4.9% | 0.998 | 17.7 | 18.1 | - | - | - | - | - | - | - | - | - | - | - | - | - | 9 | | +| lyso_x10sa_strong | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.63 | 0.984 | 1.37 | 1.24 | -10.8% | 99.6 | 22.7 | 24.0% | 11.4% | 0.996 | 4.9 | 8.6 | - | - | - | - | - | - | - | - | - | - | - | - | - | 40 | | +| myob_x06da | inhouse | pass | P 1 21 1 | P 1 2 1 | 0.12 | 1.003 | 1.23 | 1.42 | +13.3% | 99.7 | 6.7 | 17.0% | 4.2% | 0.998 | 23.9 | 7.6 | - | - | - | - | - | - | - | - | - | - | - | - | - | 11 | P 1 21 1 vs reference P 1 2 1 (screws not judged against XDS) | +| myob_x06da_powder_1 | inhouse | fail | P 1 2 1 | P 1 | 0.97 | 1.019 | 1.38 | 1.50 | +7.7% | 99.4 | 5.6 | 98.7% | 19.8% | 0.929 | 2.4 | 5.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | 16 | P 1 2 1 vs reference P 1 | +| myob_x06da_powder_2 | inhouse | pass | P 1 2 1 | P 1 2 1 | 0.34 | 0.995 | 1.41 | 0.99 | -42.2% | 99.3 | 5.4 | 151.9% | 48.4% | 0.888 | 2.8 | 2.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | 18 | | +| myob_x06da_sparse | inhouse | pass | P 1 2 1 | P 1 2 1 | 0.44 | 0.993 | 1.77 | 2.00 | +11.7% | 99.7 | 5.6 | 49.1% | 20.7% | 0.893 | 3.0 | 5.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | 40 | | +| myob_x06da_split | inhouse | pass | P 1 21 1 | P 1 2 1 | 0.21 | 0.998 | 1.84 | 1.51 | -22.4% | 99.5 | 5.3 | 69.9% | 19.0% | 0.525 | 3.1 | 12.4 | - | - | - | - | - | - | - | - | - | - | - | - | - | 19 | P 1 21 1 vs reference P 1 2 1 (screws not judged against XDS) | +| myob_x10sa | inhouse | pass | P 1 21 1 | P 1 21 1 | 0.31 | 1.004 | 1.62 | 1.74 | +7.2% | 98.0 | 5.9 | 18.4% | 8.1% | 0.994 | 9.1 | 5.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | 18 | | +| nothing_1 | inhouse | pass | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 51 | no lattice reported, as expected | +| nothing_2 | inhouse | pass | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 48 | no lattice reported, as expected | +| thau_bl1a_3p8keV | inhouse | pass | P 41 21 2 | P 4 2 2 | 0.02 | 1.000 | 3.02 | 3.10 | +2.7% | 88.8 | 15.5 | 8.1% | 7.1% | 0.997 | 17.4 | 30.8 | - | - | - | - | - | - | - | - | - | - | - | - | - | 10 | P 41 21 2 vs reference P 4 2 2 (screws not judged against XDS) | +| thau_bl1a_4p6keV | inhouse | pass | P 41 21 2 | P 4 2 2 | 0.08 | 0.998 | 2.47 | 2.53 | +2.5% | 88.5 | 15.9 | 7.0% | 5.3% | 0.999 | 33.8 | 35.6 | - | - | - | - | - | - | - | - | - | - | - | - | - | 11 | P 41 21 2 vs reference P 4 2 2 (screws not judged against XDS) | +| thau_bl1a_6p5keV | inhouse | pass | P 41 21 2 | P 4 2 2 | 0.04 | 0.999 | 1.73 | 1.78 | +2.6% | 88.6 | 16.3 | 7.2% | 4.6% | 0.999 | - | 34.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | 15 | P 41 21 2 vs reference P 4 2 2 (screws not judged against XDS) | +| thau_micromax_pink | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.13 | 0.997 | 1.28 | 1.40 | +8.2% | 98.1 | 14.7 | 8.6% | 5.0% | 0.999 | 15.4 | 21.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | 21 | | +| thau_x10sa_0p1deg | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.23 | 0.993 | 2.04 | 2.20 | +7.1% | 94.7 | 19.0 | 18.7% | 9.6% | 0.999 | 7.0 | 9.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | 23 | | +| thau_x10sa_16keV | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.02 | 1.000 | 1.22 | 1.30 | +6.3% | 82.4 | 17.6 | 7.0% | 2.9% | 1.000 | 52.5 | 44.5 | - | - | - | - | - | - | - | - | - | - | - | - | - | 20 | | +| thau_x10sa_injection | inhouse | pass | P 41 21 2 | P 41 21 2 | 0.02 | 1.000 | 1.26 | 1.28 | +1.6% | 82.6 | 19.0 | 3.9% | 2.6% | 1.000 | 33.2 | 36.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | 28 | | + +## Delta vs baseline 20260919-1854_80e395_poolB + +Baseline rugnux 1.0.0-rc.171. Pass rates on the sets both runs have: + +| arm | common sets | baseline | this run | +|:--|--:|:--|:--| +| open | 152 | 138/151 (91%) | 148/150 (99%) | +| inhouse | 35 | 32/35 (91%) | 34/35 (97%) | + +Rows whose result moved beyond noise (thresholds in report.py NOISE; confirm with `compare --rerun-changed` before believing a single-set change): + +| set | arm | verdict | space group | d_min | ISa | R_meas | CC1/2 | cell dev % | R_model (shell-scaled) | R_free | radial misfit | ref-range R_meas | ref-range low-res R_meas | time s | beyond noise | +|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--| +| cytc_x06da_1 | inhouse | pass | P 31 2 1 | 1.70 | 15.2 -> 17.4 | 8.9% -> 9.3% | 1.000 | 0.04 | - | - | - | 8.3% -> 8.5% | 3.8% -> 3.7% | 16 -> 14 | isa, refres_isa | +| cytc_x06da_2 | inhouse | pass | P 31 2 1 | 1.56 -> 1.57 | 19.0 -> 21.1 | 9.0% -> 9.8% | 1.000 | 0.05 -> 0.11 | - | - | - | 8.5% -> 9.1% | 3.3% -> 3.5% | 23 -> 20 | isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas | +| cytc_x10sa | inhouse | pass | P 31 2 1 | 1.93 -> 1.95 | 18.6 -> 25.8 | 13.6% -> 20.1% | 1.000 -> 0.999 | 0.12 -> 0.13 | - | - | - | 13.3% -> 18.0% | 3.8% -> 3.9% | 34 -> 26 | isa, r_meas, refres_r_meas, refres_isa | +| insu_H_x06da_notwin | inhouse | pass | R 3:H | 1.42 | 18.6 -> 20.3 | 6.3% | 0.999 | 0.10 -> 0.06 | - | - | - | 6.2% -> 6.1% | 4.4% | 14 -> 11 | isa, refres_isa | +| insu_H_x06da_twin | inhouse | pass | R 3:H | 1.38 | 7.5 -> 6.3 | 11.9% -> 12.8% | 0.995 -> 0.994 | 0.04 | - | - | - | 11.8% -> 13.2% | 9.8% -> 11.5% | 16 -> 12 | isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas | +| insu_I_x06da_13keV | inhouse | pass | I 2 3 | 1.47 -> 1.48 | - | 19.3% -> 31.8% | 0.999 | 0.02 -> 0.27 | - | - | - | 18.8% -> 26.8% | 7.9% -> 8.0% | 23 -> 16 | r_meas, cell_dev_pct, refres_r_meas, refres_isa | +| insu_I_x06da_5keV | inhouse | pass | I 2 3 | 2.43 | 26.6 -> 28.3 | 6.5% | 1.000 | 0.02 -> 0.03 | - | - | - | 6.5% -> 6.2% | 4.1% -> 4.4% | 12 -> 9 | isa, refres_r_meas, refres_isa, refres_lowres_r_meas | +| insu_I_x06da_5keV_2 | inhouse | pass | I 2 3 | 2.43 -> 2.42 | - -> 24.4 | 7.5% -> 8.3% | 1.000 | 0.04 -> 0.05 | - | - | - | 7.5% -> 7.9% | 4.5% -> 5.8% | 11 -> 8 | isa, r_meas, refres_isa, refres_lowres_r_meas | +| insu_I_x06da_low_isa | inhouse | pass | I 2 3 | 1.43 -> 1.44 | 6.5 -> 5.6 | 20.3% -> 22.0% | 0.998 | 0.02 -> 0.08 | - | - | - | 20.3% -> 22.1% | 12.3% -> 15.9% | 19 -> 16 | isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas | +| insu_I_x06da_ref | inhouse | pass | I 2 3 | 1.40 | 17.7 -> 31.1 | 22.9% -> 27.8% | 0.999 | 0.04 -> 0.13 | - | - | - | 20.6% -> 24.0% | 6.0% -> 5.9% | 19 -> 15 | isa, r_meas, refres_r_meas, refres_isa | +| insu_I_x06da_weak | inhouse | pass | I 2 3 | 1.66 -> 1.64 | 14.4 -> 14.9 | 11.9% -> 16.7% | 1.000 | 0.64 -> 0.79 | - | - | - | 11.9% -> 16.7% | 6.1% -> 6.3% | 30 -> 20 | r_meas, refres_r_meas, time | +| lyso_x06da_5keV | inhouse | pass | P 41 21 2 | 2.43 | 17.2 -> 25.6 | 7.5% -> 6.6% | 0.999 | 0.29 | - | - | - | 7.5% -> 5.9% | 5.8% -> 5.4% | 14 -> 8 | isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas | +| lyso_x06da_atten_wedge | inhouse | pass | P 41 21 2 | 1.16 | 10.2 -> 13.0 | 26.8% -> 35.8% | 0.999 | 0.25 | - | - | - | 24.4% -> 30.9% | 8.2% -> 7.1% | 21 -> 19 | isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas | +| lyso_x06da_half_image | inhouse | pass | P 41 21 2 | 1.56 | 4.3 -> 7.3 | 73.5% -> 101.0% | 0.978 -> 0.972 | 0.46 -> 0.71 | - | - | - | 71.8% -> 94.7% | 31.0% -> 25.6% | 80 -> 47 | isa, r_meas, cc_half, cell_dev_pct, refres_r_meas, refres_isa, refres_lowres_r_meas, time | +| lyso_x06da_ice | inhouse | pass | P 41 21 2 | 1.34 | 18.0 -> 22.7 | 12.8% -> 18.8% | 0.999 | 0.03 -> 0.09 | - | - | - | 12.7% -> 16.6% | 4.6% | 22 -> 17 | isa, r_meas, refres_r_meas, refres_isa | +| lyso_x06da_ref | inhouse | pass | P 41 21 2 | 0.99 | 29.5 -> 29.4 | 4.7% -> 4.8% | 1.000 | 0.04 | - | - | - | 4.2% -> 4.3% | 2.5% -> 2.7% | 23 -> 18 | refres_lowres_r_meas | +| lyso_x10sa_90deg_1 | inhouse | fail -> pass | P 21 21 21 -> P 41 21 2 | 1.95 -> 1.86 | 14.5 -> 18.8 | 14.8% -> 18.1% | 0.996 -> 0.997 | 0.12 | - | - | - | 14.6% -> 16.3% | 3.9% -> 4.7% | 14 -> 10 | verdict fail->pass, space group, d_min, isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas | +| lyso_x10sa_90deg_2 | inhouse | fail -> pass | P 21 21 21 -> P 41 21 2 | 1.96 -> 1.88 | 15.3 -> 17.7 | 14.6% -> 17.7% | 0.996 -> 0.998 | 0.17 -> 0.16 | - | - | - | 14.4% -> 16.2% | 4.0% -> 4.8% | 14 -> 9 | verdict fail->pass, space group, d_min, isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas | +| lyso_x10sa_strong | inhouse | pass | P 42 21 2 -> P 41 21 2 | 1.41 -> 1.37 | 3.2 -> 4.9 | 26.7% -> 24.0% | 0.987 -> 0.996 | 0.38 -> 0.63 | - | - | - | 26.7% -> 24.0% | 19.0% -> 12.9% | 31 -> 40 | space group, d_min, isa, r_meas, cc_half, cell_dev_pct, refres_r_meas, refres_isa, refres_lowres_r_meas | +| myob_x06da | inhouse | pass | P 1 21 1 | 1.17 -> 1.23 | 20.4 -> 23.9 | 13.0% -> 17.0% | 0.999 -> 0.998 | 0.06 -> 0.12 | - | - | - | 11.0% -> 12.5% | 4.2% -> 4.3% | 13 -> 11 | d_min, isa, r_meas, refres_r_meas, refres_isa | +| myob_x06da_powder_1 | inhouse | fail | P 1 2 1 | 1.65 -> 1.38 | 2.0 -> 2.4 | 76.3% -> 98.7% | 0.852 -> 0.929 | 1.23 -> 0.97 | - | - | - | 76.3% -> 83.4% | 19.0% -> 17.0% | 25 -> 16 | d_min, isa, r_meas, cc_half, cell_dev_pct, refres_r_meas, refres_isa, refres_lowres_r_meas | +| myob_x06da_powder_2 | inhouse | pass | P 1 2 1 | 2.56 -> 1.41 | 6.1 -> 2.8 | 56.7% -> 151.9% | 0.983 -> 0.888 | 0.31 -> 0.34 | - | - | - | 56.7% -> 151.9% | 51.1% -> 58.8% | 25 -> 18 | d_min, isa, r_meas, cc_half, refres_r_meas, refres_isa, refres_lowres_r_meas | +| myob_x06da_sparse | inhouse | pass | P 1 2 1 | 1.78 -> 1.77 | 3.4 -> 3.0 | 45.4% -> 49.1% | 0.956 -> 0.893 | 0.44 | - | - | - | 39.1% -> 41.8% | 14.7% -> 19.9% | 94 -> 40 | isa, r_meas, cc_half, refres_r_meas, refres_isa, refres_lowres_r_meas, time | +| myob_x06da_split | inhouse | pass | P 1 2 1 -> P 1 21 1 | 1.69 -> 1.84 | 2.8 -> 3.1 | 110.6% -> 69.9% | 0.797 -> 0.525 | 0.20 -> 0.21 | - | - | - | 110.6% -> 69.9% | 25.9% -> 20.6% | 29 -> 19 | space group, d_min, isa, r_meas, cc_half, refres_r_meas, refres_isa, refres_lowres_r_meas | +| myob_x10sa | inhouse | pass | P 1 2 1 -> P 1 21 1 | 1.70 -> 1.62 | 8.8 -> 9.1 | 19.8% -> 18.4% | 0.990 -> 0.994 | 0.85 -> 0.31 | - | - | - | 19.3% -> 17.7% | 9.8% -> 7.9% | 31 -> 18 | space group, d_min, r_meas, cell_dev_pct, refres_r_meas, refres_lowres_r_meas, time | +| nothing_1 | inhouse | pass | - | - | - | - | - | - | - | - | - | - | - | 116 -> 51 | time | +| nothing_2 | inhouse | pass | - | - | - | - | - | - | - | - | - | - | - | 108 -> 48 | time | +| thau_bl1a_3p8keV | inhouse | pass | P 41 21 2 | 3.02 | 14.8 -> 17.4 | 8.6% -> 8.1% | 0.997 | 0.02 | - | - | - | 8.6% -> 7.4% | 7.0% -> 6.9% | 15 -> 10 | isa, r_meas, refres_r_meas, refres_isa | +| thau_bl1a_4p6keV | inhouse | pass | P 41 21 2 | 2.47 | 26.2 -> 33.8 | 7.0% | 0.999 | 0.08 | - | - | - | 6.9% -> 6.6% | 4.9% -> 5.2% | 18 -> 11 | isa, refres_isa, refres_lowres_r_meas | +| thau_bl1a_6p5keV | inhouse | pass | P 41 21 2 | 1.73 | 31.1 -> - | 7.4% -> 7.2% | 0.999 | 0.04 | - | - | - | 7.3% -> 7.1% | 4.5% -> 4.6% | 21 -> 15 | isa, refres_isa | +| thau_micromax_pink | inhouse | pass | P 41 21 2 | 1.32 -> 1.28 | 13.6 -> 15.4 | 8.6% | 0.999 | 0.14 -> 0.13 | - | - | - | 8.5% -> 8.6% | 4.9% | 28 -> 21 | d_min, isa, completeness, refres_isa | +| thau_x10sa_0p1deg | inhouse | pass | P 41 21 2 | 2.04 | 4.6 -> 7.0 | 22.0% -> 18.7% | 0.997 -> 0.999 | 0.23 | - | - | - | 21.6% -> 18.4% | 12.6% -> 9.1% | 35 -> 23 | isa, r_meas, refres_r_meas, refres_isa, refres_lowres_r_meas, time | +| thau_x10sa_16keV | inhouse | pass | P 41 21 2 | 1.22 | 43.0 -> 52.5 | 6.9% -> 7.0% | 1.000 | 0.04 -> 0.02 | - | - | - | 6.9% -> 7.0% | 2.7% -> 2.8% | 31 -> 20 | isa, refres_isa, time | +| thau_x10sa_injection | inhouse | pass | P 41 21 2 | 1.26 | 32.0 -> 33.2 | 3.8% -> 3.9% | 1.000 | 0.02 | - | - | - | 3.8% -> 3.9% | 2.5% -> 2.6% | 41 -> 28 | time | +| 11if | open | pass | P 41 | 1.37 -> 1.38 | 22.2 -> 25.5 | 4.9% -> 4.6% | 1.000 | 0.12 | - -> 0.191 | 0.228 -> 0.200 | - -> 0.19 | - | - | 25 -> 14 | isa, r_meas, rfree, time | +| 36gk | open | pass | I 2 2 2 | 2.08 -> 2.09 | 11.8 | 11.7% -> 15.7% | 0.998 | 0.30 -> 0.09 | - -> 0.203 | 0.291 -> 0.203 | - -> 0.11 | - | - | 61 -> 41 | r_meas, cell_dev_pct, rfree, time | +| 3inp | open | pass | F 41 3 2 | 1.70 | 12.0 -> 13.3 | 11.2% -> 10.9% | 0.999 | 0.03 -> 0.04 | - -> 0.239 | 0.245 -> 0.239 | - -> 0.07 | - | - | 76 -> 56 | isa, rfree, time | +| 3ky7 | open | pass | P 43 3 2 | 1.92 | 12.2 -> 12.3 | 10.2% -> 10.7% | 0.999 | 0.06 -> 0.03 | - -> 0.341 | 0.335 | - -> 0.05 | - | - | 75 -> 57 | r_meas | +| 5ebi | open | pass | P 1 1 21 -> P 1 21 1 | 0.90 | 11.8 -> 14.4 | 10.9% -> 10.6% | 0.990 -> 0.997 | 0.06 -> 0.05 | - -> 0.549 | 0.722 -> 0.535 | - -> 0.07 | - | - | 68 -> 52 | isa, cc_half, rfree | +| 5epe | open | pass | F 2 3 | 1.78 | 11.5 -> 10.0 | 13.3% -> 14.3% | 0.998 | 0.00 | - -> 0.166 | 0.217 -> 0.175 | - -> 0.08 | - | - | 64 -> 67 | isa, r_meas, rfree | +| 5f6m | open | pass | P 21 21 21 | 1.09 | 22.0 -> 22.2 | 5.0% -> 5.2% | 0.998 | 0.08 | - -> 0.152 | 0.170 -> 0.168 | - -> 0.16 | - | - | 17 -> 12 | completeness | +| 5j23 | open | fail -> pass | R 3 2:H -> R 3:H | 2.04 -> 2.17 | 2.5 -> 11.5 | 32.1% -> 15.2% | 0.970 -> 0.996 | 0.06 -> 0.15 | - -> 0.227 | 0.319 -> 0.234 | - -> 0.13 | - | - | 75 -> 62 | verdict fail->pass, space group, d_min, isa, r_meas, cc_half, rfree | +| 5jvn | open | pass | P 6 2 2 | 2.21 -> 2.24 | 16.0 -> 15.6 | 13.7% -> 16.3% | 0.999 -> 0.998 | 0.07 -> 0.03 | - -> 0.234 | 0.259 -> 0.250 | - -> 0.14 | - | - | 52 -> 40 | r_meas, rfree | +| 5lzl | open | pass | P 31 2 1 | 2.88 -> 2.86 | 13.9 -> 15.0 | 15.6% -> 17.2% | 0.997 | 0.32 -> 0.29 | - -> 0.222 | 0.249 -> 0.228 | - -> 0.14 | - | - | 52 -> 25 | isa, r_meas, rfree, time | +| 5m17 | open | pass | I 4 | 0.99 -> 0.98 | 15.4 -> 16.0 | 6.3% -> 6.0% | 0.994 | 0.08 | - -> 0.132 | 0.200 -> 0.159 | - -> 0.23 | - | - | 64 -> 102 | completeness, rfree, time | +| 5nw5 | open | pass | P 21 21 21 | 6.91 -> 7.10 | 9.3 -> 8.8 | 24.7% -> 32.0% | 0.988 -> 0.936 | 0.19 | - -> 0.401 | 0.493 -> 0.440 | - -> 0.36 | - | - | 62 -> 47 | d_min, isa, r_meas, cc_half, rfree | +| 5reo | open | pass | C 1 2 1 | 1.63 -> 1.65 | 20.0 -> 24.0 | 23.0% -> 22.1% | 0.992 -> 0.990 | 0.37 -> 0.35 | - -> 0.204 | 0.206 -> 0.202 | - -> 0.07 | - | - | 12 -> 10 | isa | +| 5src | open | pass | P 41 | 0.98 -> 0.97 | 17.9 -> 19.6 | 6.0% -> 5.9% | 0.999 -> 1.000 | 0.19 -> 0.09 | - -> 0.164 | 0.186 -> 0.183 | - -> 0.20 | - | - | 76 -> 44 | isa, time | +| 6fid | open | pass | P 21 21 21 | 1.98 | 10.9 -> 13.2 | 10.3% -> 9.7% | 0.998 | 0.41 | - -> 0.194 | 0.200 -> 0.199 | - -> 0.09 | - | - | 35 -> 28 | isa, r_meas | +| 6fvz | open | pass | C 2 2 2 | 1.49 | 19.8 -> 20.6 | 17.8% -> 24.2% | 0.997 -> 0.996 | 0.14 -> 0.43 | - -> 0.202 | 0.221 -> 0.206 | - -> 0.10 | - | - | 48 -> 36 | r_meas, cell_dev_pct, rfree | +| 6fwc | open | fail -> pass | P 1 1 2 -> C 2 2 2 | 1.44 -> 1.41 | 28.7 -> 27.2 | 15.4% -> 16.4% | 0.993 -> 0.995 | 41.89 -> 0.04 | - -> 0.189 | 0.635 -> 0.197 | - -> 0.10 | - | - | 117 -> 48 | verdict fail->pass, space group, isa, r_meas, completeness, cell_dev_pct, rfree, time | +| 6h2p_1p89A | open | fail -> pass | - -> C 2 2 21 | - -> 1.76 | - -> 22.5 | - -> 8.3% | - -> 0.999 | - -> 0.03 | - -> 0.144 | - -> 0.154 | - -> 0.11 | - | - | 120 -> 172 | verdict fail->pass, space group, d_min, isa, r_meas, cc_half, completeness, cell_dev_pct, time | +| 6h2p_native | open | fail -> pass | P 1 1 21 -> C 2 2 21 | 2.67 -> 1.32 | 1.2 -> 19.9 | 119.6% -> 13.7% | 0.567 -> 0.999 | 29.20 -> 0.04 | - -> 0.170 | 0.542 -> 0.174 | - -> 0.08 | - | - | 242 -> 110 | verdict fail->pass, space group, d_min, isa, r_meas, cc_half, completeness, cell_dev_pct, rfree, time | +| 6h5t | open | pass | I 4 2 2 | 1.46 -> 1.48 | 7.5 -> 9.2 | 14.4% -> 15.2% | 0.996 | 0.41 -> 0.40 | - -> 0.203 | 0.223 -> 0.218 | - -> 0.13 | - | - | 34 -> 35 | isa, r_meas | +| 6hv2 | open | pass | P 61 2 2 | 1.37 -> 1.42 | 10.7 -> 13.6 | 12.2% -> 18.1% | 1.000 | 0.04 | - -> 0.235 | 0.255 -> 0.247 | - -> 0.46 | - | - | 51 -> 37 | d_min, isa, r_meas, rfree, time | +| 6hwj | open | pass | P 1 21 1 | 1.71 -> 1.72 | 31.3 -> 37.8 | 7.9% -> 8.4% | 0.999 | 0.10 | - -> 0.187 | 0.202 -> 0.192 | - -> 0.13 | - | - | 27 -> 19 | isa, r_meas, rfree | +| 6i3j | open | pass | F 2 2 2 | 2.32 | 6.8 -> 7.0 | 21.5% -> 23.5% | 0.981 -> 0.983 | 0.16 -> 0.08 | - -> 0.202 | 0.208 -> 0.233 | - -> 0.16 | - | - | 85 -> 83 | r_meas, rfree | +| 6iu5 | open | pass | P 31 | 2.10 -> 2.12 | 5.6 -> 7.9 | 19.3% -> 18.9% | 0.991 | 0.02 | - -> 0.212 | 0.218 -> 0.215 | - -> 0.08 | - | - | 100 -> 89 | isa | +| 6iu6 | open | pass | P 31 | 2.30 -> 2.38 | 6.1 -> 7.0 | 13.5% -> 14.2% | 0.993 -> 0.992 | 0.07 -> 0.33 | - -> 0.221 | 0.235 -> 0.234 | - -> 0.19 | - | - | 117 -> 87 | d_min, isa, r_meas, completeness, cell_dev_pct, time | +| 6iu8 | open | fail -> pass | P 31 2 1 -> P 31 | 2.23 -> 2.33 | 5.3 -> 8.1 | 17.4% -> 14.3% | 0.997 | 0.09 -> 0.02 | - -> 0.269 | 0.330 -> 0.284 | - -> 0.18 | - | - | 27 -> 18 | verdict fail->pass, space group, d_min, isa, r_meas, rfree | +| 6iu9 | open | fail -> pass | P 31 2 1 -> P 31 | 2.67 -> 2.73 | 3.1 -> 5.3 | 29.1% -> 18.0% | 0.973 -> 0.989 | 0.21 -> 0.22 | - -> 0.306 | 0.313 -> 0.316 | - -> 0.11 | - | - | 95 -> 129 | verdict fail->pass, space group, d_min, isa, r_meas, cc_half, time | +| 6jgi | open | pass | P 21 21 21 | 0.74 -> 0.75 | 8.9 -> 8.4 | 12.2% -> 14.4% | 0.993 -> 0.995 | 0.28 -> 0.20 | - -> 0.114 | 0.137 -> 0.130 | - -> 0.15 | - | - | 116 -> 110 | isa, r_meas, rfree | +| 6jgj | open | pass | P 21 21 21 | 0.69 -> 0.65 | 12.3 -> 14.7 | 7.0% -> 7.6% | 0.998 -> 0.999 | 0.28 | - -> 0.168 | 0.146 -> 0.171 | - -> 0.07 | - | - | 34 -> 52 | d_min, isa, r_meas, completeness, rfree, time | +| 6moj | open | pass | I 41 2 2 | 2.30 -> 2.43 | 8.3 | 31.2% -> 52.6% | 0.999 -> 0.998 | 0.10 -> 0.08 | - -> 0.275 | 0.279 -> 0.285 | - -> 0.37 | - | - | 173 -> 124 | d_min, r_meas, rfree, time | +| 6o2h | open | pass | P 1 | 1.10 | 20.2 -> 20.8 | 7.1% -> 7.2% | 0.978 | 0.10 -> 0.74 | - -> 0.099 | 0.177 -> 0.131 | - -> 0.21 | - | - | 17 -> 16 | cell_dev_pct, rfree | +| 6oel | open | pass | F 41 3 2 | 2.82 -> 2.85 | 7.1 -> 9.9 | 29.3% -> 36.6% | 0.998 | 0.05 -> 0.00 | - -> 0.248 | 0.291 -> 0.255 | - -> 0.14 | - | - | 130 -> 57 | isa, r_meas, rfree, time | +| 6p8p | open | pass | P 4 | 1.43 -> 1.47 | 13.4 -> 15.3 | 10.8% -> 12.4% | 0.998 | 0.28 -> 0.14 | - -> 0.192 | 0.233 -> 0.209 | - -> 0.18 | - | - | 34 -> 25 | d_min, isa, r_meas, rfree | +| 6pb3 | open | pass | P 6 | 1.86 -> 1.85 | 17.3 -> 24.6 | 6.2% -> 6.3% | 1.000 | 0.04 -> 0.17 | - -> 0.215 | 0.267 -> 0.233 | - -> 0.14 | - | - | 37 -> 34 | isa, rfree | +| 6pxb | open | pass | P 31 1 2 | 1.34 -> 1.39 | 11.0 -> 12.0 | 7.4% -> 8.8% | 0.999 | 0.08 -> 0.23 | - -> 0.245 | 0.303 -> 0.277 | - -> 0.31 | - | - | 48 -> 38 | d_min, isa, r_meas, rfree | +| 6r72 | open | pass | P 1 21 1 | 4.07 -> 4.39 | 13.1 -> 15.7 | 9.7% -> 14.8% | 0.999 | 1.03 -> 1.32 | - -> 0.374 | 0.388 -> 0.383 | - -> 0.06 | - | - | 44 -> 29 | d_min, isa, r_meas, cell_dev_pct, time | +| 6rlr | open | pass | P 1 | 2.07 | 21.6 -> 13.2 | 10.6% -> 11.0% | 0.999 -> 0.998 | 0.03 | - -> 0.249 | 0.260 -> 0.258 | - -> 0.08 | - | - | 36 -> 18 | isa, time | +| 6toc | open | pass | P 42 2 2 | 1.65 -> 1.64 | 30.0 -> 24.3 | 6.1% -> 9.4% | 1.000 | 0.09 -> 0.30 | - -> 0.276 | 0.289 -> 0.269 | - -> 0.14 | - | - | 68 -> 49 | isa, r_meas, cell_dev_pct, rfree, time | +| 6ttn | open | pass | P 21 21 21 | 1.05 -> 1.08 | 12.7 -> 14.5 | 9.7% -> 10.7% | 0.999 | 0.10 -> 0.41 | - -> 0.132 | 0.158 -> 0.155 | - -> 0.16 | - | - | 59 -> 48 | d_min, isa, r_meas, completeness, cell_dev_pct | +| 6u7g | open | pass | P 1 21 1 | 1.94 -> 1.92 | 12.1 -> 13.2 | 8.6% -> 8.3% | 0.997 -> 0.998 | 0.09 -> 0.13 | - -> 0.201 | 0.207 -> 0.206 | - -> 0.14 | - | - | 108 -> 70 | isa, completeness, time | +| 6ukf | open | pass | P 1 21 1 | 0.95 | 6.8 -> 9.4 | 12.1% -> 10.7% | 0.995 -> 0.997 | 0.07 | - -> 0.166 | 0.165 -> 0.162 | - -> 0.09 | - | - | 63 -> 54 | isa, r_meas | +| 6vww | open | pass | P 63 | 2.01 -> 2.00 | 7.6 -> 8.6 | 15.4% -> 15.9% | 0.990 -> 0.993 | 0.51 -> 0.20 | - -> 0.239 | 0.257 -> 0.241 | - -> 0.10 | - | - | 43 -> 24 | isa, cell_dev_pct, rfree, time | +| 6w4h | open | pass | P 31 2 1 | 1.62 | 16.3 -> 18.4 | 7.6% -> 7.7% | 0.999 | 0.02 | - -> 0.163 | 0.210 -> 0.168 | - -> 0.15 | - | - | 81 -> 69 | isa, rfree | +| 6wzo | open | pass | P 1 | 1.05 -> 1.06 | 11.4 -> 16.9 | 7.9% -> 5.8% | 0.996 -> 0.998 | 0.04 | - -> 0.179 | 0.201 -> 0.186 | - -> 0.24 | - | - | 64 -> 57 | isa, r_meas, completeness, rfree | +| 6yqf | open | pass | P 21 21 2 | 3.03 -> 3.05 | 3.9 | 77.4% -> 50.3% | 0.981 -> 0.988 | 1.17 -> 0.71 | - -> 0.432 | 0.493 -> 0.470 | - -> 0.33 | - | - | 27 -> 24 | r_meas, cc_half, cell_dev_pct, rfree | +| 6z8o | open | pass | P 1 21 1 | 2.40 -> 2.23 | 13.3 -> 13.8 | 14.0% -> 17.1% | 0.948 -> 0.995 | 0.27 -> 0.63 | - -> 0.290 | 0.283 -> 0.295 | - -> 0.09 | - | - | 63 -> 62 | d_min, r_meas, cc_half, completeness, cell_dev_pct, rfree | +| 6ze4 | open | fail -> pass | P 1 21 1 -> P 21 21 21 | 1.48 -> 1.30 | 11.3 -> 9.2 | 16.4% -> 20.4% | 0.479 -> 0.992 | 0.59 | - -> 0.232 | 0.243 -> 0.237 | - -> 0.11 | - | - | 96 -> 73 | verdict fail->pass, space group, d_min, isa, r_meas, cc_half, completeness, rfree | +| 7arr | open | pass | P 1 | 0.92 | 16.6 -> 17.3 | 5.0% -> 4.7% | 0.994 -> 0.993 | 2.83 -> 0.28 | - -> 0.155 | 0.209 -> 0.175 | - -> 0.27 | - | - | 40 -> 41 | r_meas, cell_dev_pct, rfree | +| 7atg | open | pass | P 21 21 21 | 0.60 | 23.1 -> 22.5 | 4.6% -> 5.1% | 0.995 | 0.08 | - -> 0.135 | 0.191 -> 0.199 | - -> 0.23 | - | - | 59 -> 41 | r_meas, rfree, time | +| 7bgt | open | pass | P 1 | 1.76 -> 1.77 | 14.7 | 14.3% -> 15.0% | 0.983 -> 0.989 | 0.07 -> 0.32 | - -> 0.197 | 0.214 -> 0.207 | - -> 0.12 | - | - | 41 -> 32 | cc_half, cell_dev_pct, rfree | +| 7brr | open | pass | P 1 21 1 | 1.27 | 13.8 -> 15.8 | 7.0% -> 6.5% | 0.999 | 0.03 -> 0.09 | - -> 0.191 | 0.211 -> 0.195 | - -> 0.15 | - | - | 35 -> 25 | isa, r_meas, rfree, time | +| 7dkp | open | pass | P 1 21 1 | 1.24 -> 1.18 | 20.1 -> 26.3 | 11.6% -> 11.7% | 0.995 -> 0.998 | 0.43 -> 0.06 | - -> 0.158 | 0.178 -> 0.161 | - -> 0.08 | - | - | 43 -> 31 | d_min, isa, completeness, cell_dev_pct, rfree, time | +| 7k1l | open | fail -> unscored | P 6 | 1.89 -> 1.91 | 8.7 -> 9.9 | 15.6% -> 16.2% | 0.995 -> 0.994 | 0.23 -> 0.27 | - -> 0.264 | 0.281 -> 0.271 | - -> 0.09 | - | - | 27 -> 24 | verdict fail->unscored, isa, rfree | +| 7kcn | open | pass | P 41 2 2 | 1.39 | 10.4 -> 11.7 | 8.7% -> 7.8% | 0.999 | 0.06 -> 0.07 | - -> 0.172 | 0.201 -> 0.189 | - -> 0.15 | - | - | 46 -> 39 | isa, r_meas, rfree | +| 7l6j | open | pass | I 41 3 2 | 1.51 | 8.0 -> 10.5 | 16.6% -> 21.1% | 0.999 | 0.07 -> 0.00 | - -> 0.172 | 0.217 -> 0.186 | - -> 0.17 | - | - | 100 -> 108 | isa, r_meas, rfree | +| 7mzt | open | fail | P 21 21 21 | 3.10 -> 3.25 | 2.8 -> 3.4 | 169.1% -> 269.6% | 0.961 -> 0.968 | 0.63 -> 0.56 | - -> 0.452 | 0.494 -> 0.466 | - -> 0.16 | - | - | 61 -> 37 | d_min, isa, r_meas, cc_half, rfree, time | +| 7n0i | open | pass | P 21 21 21 | 1.63 -> 1.65 | 10.1 -> 10.7 | 16.2% -> 14.0% | 0.997 -> 0.998 | 0.37 -> 0.22 | - -> 0.290 | 0.314 -> 0.310 | - -> 0.36 | - | - | 76 -> 58 | isa, r_meas | +| 7n2s | open | pass | P 1 21 1 | 2.49 -> 2.57 | 7.3 -> 9.6 | 54.9% -> 52.8% | 0.912 -> 0.903 | 0.07 | - -> 0.315 | 0.334 | - -> 0.14 | - | - | 48 -> 42 | d_min, isa, cc_half | +| 7orr | open | pass | I 2 3 | 1.65 | 17.1 -> 25.5 | 7.1% -> 6.4% | 0.999 -> 1.000 | 0.04 -> 0.03 | - -> 0.181 | 0.192 -> 0.190 | - -> 0.14 | - | - | 35 -> 23 | isa, r_meas, time | +| 7os3 | open | pass | P 21 21 21 | 1.96 | 17.9 -> 24.1 | 9.2% -> 8.4% | 0.999 | 0.09 -> 0.10 | - -> 0.182 | 0.200 -> 0.196 | - -> 0.18 | - | - | 47 -> 40 | isa, r_meas | +| 7ph1 | open | pass | I 2 2 2 | 1.10 -> 1.08 | 13.3 -> 15.9 | 10.4% -> 11.4% | 0.997 -> 0.998 | 0.13 -> 0.12 | - -> 0.160 | 0.217 -> 0.178 | - -> 0.17 | - | - | 51 -> 87 | isa, r_meas, rfree, time | +| 7pq7 | open | pass | C 1 2 1 | 1.35 -> 1.38 | 11.5 -> 13.7 | 8.1% -> 7.1% | 0.992 -> 0.998 | 0.07 -> 0.22 | - -> 0.189 | 0.224 -> 0.209 | - -> 0.18 | - | - | 21 -> 16 | d_min, isa, r_meas, cc_half, completeness, rfree | +| 7qij | open | pass | P 21 21 21 | 3.55 -> 3.59 | 7.8 -> 9.4 | 21.6% -> 24.5% | 0.995 -> 0.996 | 0.30 -> 0.36 | - -> 0.374 | 0.377 -> 0.381 | - -> 0.07 | - | - | 134 -> 94 | isa, r_meas, time | +| 7qis | open | pass | P 61 | 1.67 -> 1.75 | 13.7 -> 17.2 | 15.6% -> 20.9% | 0.997 | 0.05 | - -> 0.169 | 0.244 -> 0.192 | - -> 0.20 | - | - | 51 -> 38 | d_min, isa, r_meas, rfree | +| 7ris | open | pass | P 31 2 1 | 1.51 | 22.9 -> 30.2 | 8.6% -> 10.0% | 1.000 | 0.08 | - -> 0.196 | 0.198 -> 0.195 | - -> 0.03 | - | - | 63 -> 29 | isa, r_meas, time | +| 7rji | open | pass | R 3 2:H | 1.43 -> 1.49 | 5.9 -> 8.3 | 19.9% -> 13.4% | 0.998 -> 0.999 | 0.04 -> 0.27 | - -> 0.206 | 0.226 -> 0.199 | - -> 0.12 | - | - | 36 -> 30 | d_min, isa, r_meas, cell_dev_pct, rfree | +| 7t5t | open | pass | P 42 21 2 | 1.24 | 14.8 -> 16.5 | 6.2% -> 6.6% | 0.999 -> 0.996 | 0.04 | - -> 0.164 | 0.201 -> 0.193 | - -> 0.25 | - | - | 69 -> 63 | isa, r_meas, rfree | +| 7tcd | open | pass | C 1 2 1 | 1.73 -> 1.72 | 12.4 -> 15.0 | 10.4% -> 10.0% | 0.999 | 0.21 -> 0.20 | - -> 0.205 | 0.210 -> 0.211 | - -> 0.14 | - | - | 50 -> 32 | isa, time | +| 7yzx | open | pass | P 63 2 2 | 1.87 -> 1.88 | 9.2 -> 10.4 | 15.8% -> 20.5% | 0.997 -> 0.998 | 0.16 -> 0.25 | - -> 0.211 | 0.224 -> 0.214 | - -> 0.10 | - | - | 70 -> 53 | isa, r_meas, rfree | +| 8a1a | open | pass | P 61 | 1.94 -> 1.95 | 14.2 -> 17.9 | 21.4% -> 41.5% | 0.999 -> 0.998 | 0.34 -> 0.42 | - -> 0.171 | 0.242 -> 0.177 | - -> 0.06 | - | - | 233 -> 100 | isa, r_meas, rfree, time | +| 8agq | open | pass | C 1 2 1 | 0.94 -> 0.97 | 13.2 -> 14.0 | 9.0% -> 9.8% | 0.998 | 0.13 -> 0.30 | - -> 0.180 | 0.223 -> 0.204 | - -> 0.21 | - | - | 46 -> 29 | d_min, isa, r_meas, completeness, rfree, time | +| 8dqb | open | pass | I 2 3 | 2.05 -> 2.06 | 20.5 -> 22.3 | 11.3% -> 13.5% | 0.991 -> 0.996 | 0.09 -> 0.08 | - -> 0.230 | 0.254 -> 0.240 | - -> 0.13 | - | - | 40 -> 26 | isa, r_meas, cc_half, rfree, time | +| 8dyz | open | pass | P 41 21 2 | 1.14 | 34.2 -> 37.3 | 3.4% -> 3.2% | 0.999 | 0.08 | - -> 0.120 | 0.120 -> 0.121 | - -> 0.06 | - | - | 26 -> 17 | isa, r_meas | +| 8iya | open | pass | C 1 2 1 | 1.92 -> 1.94 | 5.4 -> 5.8 | 20.8% -> 20.9% | 0.990 -> 0.988 | 0.10 -> 0.16 | - -> 0.247 | 0.240 -> 0.236 | - -> 0.04 | - | - | 14 -> 12 | isa | +| 8k1g | open | pass | I 4 2 2 | 1.59 -> 1.63 | 10.6 -> 11.6 | 16.0% -> 25.7% | 0.999 | 0.60 -> 0.82 | - -> 0.219 | 0.266 -> 0.236 | - -> 0.23 | - | - | 62 -> 48 | isa, r_meas, cell_dev_pct, rfree | +| 8oic | open | pass | P 1 | 2.31 -> 2.37 | 11.6 -> 19.6 | 19.1% -> 20.4% | 0.991 | 0.15 -> 0.02 | - -> 0.234 | 0.272 -> 0.250 | - -> 0.15 | - | - | 62 -> 42 | d_min, isa, r_meas, rfree, time | +| 8owm | open | pass | P 1 | 1.46 -> 1.48 | 21.6 -> 22.1 | 9.2% -> 10.1% | 0.998 | 0.02 | - -> 0.166 | 0.194 -> 0.181 | - -> 0.14 | - | - | 77 -> 69 | r_meas, rfree | +| 8pqd | open | pass | P 21 21 21 | 1.33 -> 1.30 | 15.0 -> 14.6 | 8.1% -> 9.0% | 0.998 | 0.26 -> 0.04 | - -> 0.185 | 0.224 -> 0.205 | - -> 0.23 | - | - | 251 -> 73 | d_min, r_meas, completeness, cell_dev_pct, rfree, time | +| 8qaw | open | pass | R 3:H | 1.30 | 11.2 -> 10.9 | 10.0% -> 12.2% | 0.998 -> 0.997 | 0.12 -> 0.04 | - -> 0.144 | 0.176 -> 0.156 | - -> 0.19 | - | - | 531 -> 326 | r_meas, rfree, time | +| 8qj5 | open | pass | P 1 21 1 | 1.29 -> 1.35 | 6.7 -> 8.4 | 18.4% -> 17.1% | 0.991 -> 0.996 | 0.12 -> 0.45 | - -> 0.189 | 0.232 -> 0.203 | - -> 0.14 | - | - | 56 -> 59 | d_min, isa, r_meas, completeness, cell_dev_pct, rfree | +| 8qq7 | open | pass | P 62 2 2 | 3.14 -> 3.16 | 5.1 -> 6.3 | 24.5% -> 18.8% | 0.992 -> 0.993 | 0.65 | - -> 0.457 | 0.486 -> 0.464 | - -> 0.50 | - | - | 55 -> 17 | isa, r_meas, rfree, time | +| 8r5r | open | pass | P 21 21 21 | 2.75 -> 2.89 | 14.1 -> 15.1 | 16.2% -> 27.8% | 0.998 -> 0.997 | 0.09 -> 0.11 | - -> 0.274 | 0.328 -> 0.290 | - -> 0.14 | - | - | 32 -> 35 | d_min, isa, r_meas, rfree | +| 8rud | open | pass | P 1 21 1 | 1.70 | 9.5 -> 12.3 | 26.2% -> 29.5% | 0.993 -> 0.992 | 0.33 -> 0.40 | - -> 0.244 | 0.278 -> 0.255 | - -> 0.11 | - | - | 252 -> 284 | isa, r_meas, rfree | +| 8s38 | open | pass | I 2 2 2 | 1.62 -> 1.63 | 21.5 -> 22.3 | 7.8% -> 8.6% | 0.999 | 0.12 -> 0.16 | - -> 0.181 | 0.192 -> 0.186 | - -> 0.14 | - | - | 147 -> 118 | r_meas, rfree | +| 8sa8 | open | pass | C 1 2 1 -> I 1 2 1 | 1.09 -> 1.11 | 18.7 -> 22.0 | 9.4% -> 11.2% | 0.999 | 0.85 -> 0.02 | - -> 0.158 | 0.241 -> 0.175 | - -> 0.17 | - | - | 103 -> 99 | isa, r_meas, completeness, cell_dev_pct, rfree | +| 8sqo | open | pass | P 4 3 2 | 1.33 | 11.3 -> 14.2 | 15.8% -> 22.1% | 1.000 | 0.16 -> 0.13 | - -> 0.183 | 0.262 -> 0.211 | - -> 0.20 | - | - | 91 -> 72 | isa, r_meas, rfree | +| 8sqq | open | pass | F 4 3 2 | 1.91 -> 1.90 | 16.0 -> 17.6 | 15.2% -> 25.1% | 1.000 -> 0.999 | 0.20 -> 0.17 | - -> 0.211 | 0.243 -> 0.206 | - -> 0.10 | - | - | 77 -> 73 | isa, r_meas, rfree | +| 8sqt | open | pass | F 4 3 2 | 1.90 -> 1.89 | 25.6 -> 29.6 | 11.4% -> 18.9% | 1.000 -> 0.999 | 0.11 | - -> 0.221 | 0.252 -> 0.224 | - -> 0.12 | - | - | 50 -> 30 | isa, r_meas, rfree, time | +| 8t7r | open | pass | C 1 2 1 | 3.19 -> 3.22 | 9.0 -> 9.9 | 35.8% -> 35.3% | 0.968 -> 0.984 | 0.28 | - -> 0.288 | 0.279 -> 0.288 | - -> 0.06 | - | - | 102 -> 66 | isa, cc_half, rfree, time | +| 8tha | open | pass | P 62 | 1.33 -> 1.34 | 23.0 -> 25.9 | 10.1% -> 13.7% | 1.000 | 0.07 -> 0.17 | - -> 0.208 | 0.255 -> 0.225 | - -> 0.16 | - | - | 29 -> 23 | isa, r_meas, rfree | +| 8tyy | open | pass | F 4 3 2 | 1.28 | 12.5 -> 16.3 | 12.5% -> 16.5% | 0.999 | 0.16 -> 0.06 | - -> 0.160 | 0.220 -> 0.180 | - -> 0.15 | - | - | 204 -> 179 | isa, r_meas, rfree | +| 8u0i | open | pass | P 41 21 2 | 1.40 | 13.0 -> 16.1 | 8.3% -> 7.9% | 0.999 | 0.08 -> 0.06 | - -> 0.176 | 0.183 -> 0.180 | - -> 0.07 | - | - | 31 -> 24 | isa, r_meas | +| 8v4o | open | pass | P 61 2 2 | 2.07 -> 2.10 | 13.4 -> 15.5 | 19.1% -> 31.9% | 0.999 -> 0.998 | 0.05 -> 0.06 | - -> 0.241 | 0.305 -> 0.253 | - -> 0.16 | - | - | 180 -> 92 | isa, r_meas, rfree, time | +| 8xbp | open | pass | C 1 2 1 | 1.71 -> 1.72 | 16.1 -> 16.2 | 10.4% -> 10.8% | 0.999 | 0.16 -> 0.15 | - -> 0.315 | 0.329 -> 0.325 | - -> 0.15 | - | - | 38 -> 22 | time | +| 8xte | open | pass | P 31 2 1 | 1.62 -> 1.64 | 11.4 -> 12.4 | 22.8% -> 25.1% | 0.996 | 0.04 -> 0.06 | - -> 0.275 | 0.298 -> 0.288 | - -> 0.13 | - | - | 50 -> 37 | isa, r_meas, rfree, time | +| 8xtf | open | pass | R 3 2:H | 1.77 -> 1.84 | 6.1 -> 7.5 | 34.4% -> 85.8% | 0.994 -> 0.987 | 0.20 -> 0.22 | - -> 0.191 | 0.217 -> 0.185 | - -> 0.04 | - | - | 61 -> 38 | d_min, isa, r_meas, cc_half, rfree, time | +| 8xtg | open | pass | P 31 2 1 | 1.52 -> 1.54 | 7.3 -> 7.2 | 22.3% -> 27.4% | 0.995 -> 0.994 | 0.04 -> 0.05 | - -> 0.279 | 0.314 -> 0.287 | - -> 0.13 | - | - | 53 -> 52 | r_meas, rfree | +| 8y74 | open | pass | C 1 2 1 | 1.70 -> 1.71 | 7.9 -> 8.8 | 12.5% -> 12.3% | 0.996 -> 0.997 | 0.20 -> 0.34 | - -> 0.212 | 0.270 -> 0.224 | - -> 0.12 | - | - | 32 -> 24 | isa, rfree | +| 8ys9 | open | pass | P 21 21 21 | 1.35 | 14.1 -> 15.4 | 8.5% -> 11.0% | 0.999 | 0.17 -> 0.16 | - -> 0.173 | 0.220 -> 0.191 | - -> 0.14 | - | - | 42 -> 31 | isa, r_meas, rfree, time | +| 9b22 | open | pass | P 1 1 21 -> P 1 21 1 | 1.19 -> 1.18 | 13.0 -> 16.3 | 7.2% -> 6.0% | 0.999 | 0.03 -> 0.07 | - -> 0.161 | 0.228 -> 0.181 | - -> 0.20 | - | - | 31 -> 21 | isa, r_meas, completeness, rfree | +| 9bn8 | open | pass | P 41 | 1.20 | 17.8 -> 19.8 | 7.4% -> 7.5% | 0.999 | 0.03 -> 0.09 | - -> 0.151 | 0.208 -> 0.172 | - -> 0.17 | - | - | 38 -> 30 | isa, rfree | +| 9c18 | open | pass | P 1 | 1.71 -> 1.76 | 10.2 | 20.0% -> 23.4% | 0.990 -> 0.988 | 0.29 -> 0.61 | - -> 0.222 | 0.229 -> 0.221 | - -> 0.06 | - | - | 21 -> 13 | d_min, r_meas, cell_dev_pct, rfree | +| 9chw | open | pass | P 61 | 1.99 -> 1.60 | 19.8 -> 20.9 | 5.7% -> 5.9% | 0.896 -> 0.998 | 0.05 -> 0.04 | - -> 0.180 | 0.165 -> 0.185 | - -> 0.13 | - | - | 81 -> 71 | d_min, isa, cc_half, completeness, rfree | +| 9crw | open | pass | P 1 21 1 | 2.24 -> 2.29 | 13.9 -> 15.5 | 8.1% -> 8.7% | 0.999 | 0.10 -> 0.23 | - -> 0.257 | 0.273 -> 0.271 | - -> 0.14 | - | - | 26 -> 19 | d_min, isa, r_meas, completeness | +| 9e2t | open | pass | P 1 | 2.27 -> 2.33 | 5.6 -> 6.9 | 28.9% -> 28.1% | 0.986 -> 0.990 | 0.25 -> 0.06 | - -> 0.248 | 0.257 -> 0.254 | - -> 0.11 | - | - | 86 -> 70 | d_min, isa | +| 9ea5 | open | pass | P 1 21 1 | 1.69 -> 1.64 | 27.2 -> 26.6 | 17.1% -> 17.9% | 0.997 | 0.10 -> 0.85 | - -> 0.198 | 0.209 -> 0.203 | - -> 0.11 | - | - | 103 -> 96 | d_min, completeness, cell_dev_pct, rfree | +| 9fcg | open | pass | P 4 | 1.37 -> 1.39 | 8.4 -> 9.3 | 14.6% -> 15.4% | 0.996 | 0.06 -> 0.10 | - -> 0.179 | 0.188 -> 0.184 | - -> 0.09 | - | - | 145 -> 138 | isa, r_meas, completeness | +| 9fhc | open | pass | I 2 3 | 2.01 -> 1.97 | 11.1 -> 11.6 | 31.5% -> 22.4% | 0.988 -> 0.995 | 0.17 -> 0.25 | - -> 0.245 | 0.305 -> 0.250 | - -> 0.04 | - | - | 114 -> 104 | d_min, r_meas, cc_half, completeness, rfree | +| 9gdj | open | pass | P 41 21 2 | 1.83 -> 1.40 | 2.8 -> 12.8 | 33.5% -> 10.9% | 0.958 -> 0.999 | 0.89 -> 0.16 | - -> 0.153 | 0.238 -> 0.178 | - -> 0.24 | - | - | 390 -> 355 | d_min, isa, r_meas, cc_half, cell_dev_pct, rfree | +| 9gjx | open | pass | P 1 21 1 | 2.13 -> 2.15 | 23.5 -> 35.2 | 11.1% -> 12.5% | 0.999 | 0.09 -> 0.13 | - -> 0.188 | 0.245 -> 0.225 | - -> 0.18 | - | - | 58 -> 33 | isa, r_meas, rfree, time | +| 9gqg | open | pass | P 31 2 1 | 1.81 -> 1.82 | 12.6 -> 13.2 | 10.0% -> 11.4% | 0.998 | 0.03 -> 0.01 | - -> 0.233 | 0.246 -> 0.241 | - -> 0.11 | - | - | 102 -> 59 | r_meas, rfree, time | +| 9hnc | open | pass | P 2 1 1 -> P 1 2 1 | 1.66 -> 1.65 | 1.4 | 53.1% -> 52.8% | 0.886 -> 0.882 | 0.40 -> 0.13 | - -> 0.461 | 0.497 -> 0.465 | - -> 0.16 | - | - | 108 -> 80 | cell_dev_pct, rfree, time | +| 9hs7 | open | pass | P 61 | 1.51 -> 1.68 | 11.0 -> 13.3 | 8.9% -> 11.8% | 0.999 | 0.15 -> 0.08 | - -> 0.251 | 0.289 -> 0.279 | - -> 0.58 | - | - | 40 -> 27 | d_min, isa, r_meas, rfree, time | +| 9i0a | open | pass | P 21 21 2 | 1.80 -> 1.81 | 8.7 -> 13.8 | 15.3% -> 16.8% | 0.997 -> 0.999 | 0.32 -> 0.43 | - -> 0.228 | 0.259 -> 0.239 | - -> 0.17 | - | - | 87 -> 59 | isa, r_meas, rfree, time | +| 9i80 | open | fail -> pass | P 41 | 1.70 -> 1.61 | 6.4 -> 6.7 | 17.1% -> 23.2% | 0.988 -> 0.991 | 29.26 -> 0.07 | - -> 0.216 | 0.555 -> 0.221 | - -> 0.15 | - | - | 165 -> 158 | verdict fail->pass, lattice, d_min, r_meas, cell_dev_pct, rfree | +| 9ig7 | open | pass | P 2 21 21 -> P 21 21 2 | 1.99 -> 2.03 | 11.8 -> 11.0 | 14.2% -> 18.7% | 0.993 -> 0.991 | 0.17 | - -> 0.235 | 0.313 -> 0.255 | - -> 0.18 | - | - | 95 -> 85 | isa, r_meas, rfree | +| 9ih9 | open | pass | C 1 2 1 | 1.43 -> 1.41 | 10.8 -> 12.6 | 11.4% -> 11.7% | 0.995 | 0.18 -> 0.21 | - -> 0.198 | 0.201 -> 0.207 | - -> 0.09 | - | - | 36 -> 24 | isa, rfree, time | +| 9jzo | open | pass | P 1 | 1.15 | 7.2 -> 8.4 | 10.1% -> 9.2% | 0.991 -> 0.994 | 0.14 -> 0.22 | - -> 0.175 | 0.175 | - -> 0.08 | - | - | 19 -> 17 | isa, r_meas | +| 9khr | open | pass | P 21 21 21 | 1.37 -> 1.38 | 9.6 -> 9.4 | 17.4% -> 18.9% | 0.994 | 0.12 | - -> 0.248 | 0.264 -> 0.261 | - -> 0.11 | - | - | 40 -> 33 | r_meas | +| 9mh4 | open | pass | P 21 3 | 2.76 -> 2.78 | 10.8 -> 13.8 | 14.2% -> 20.2% | 1.000 | 0.31 | - -> 0.229 | 0.270 -> 0.232 | - -> 0.17 | - | - | 41 -> 29 | isa, r_meas, rfree, time | +| 9min | open | fail | P 2 21 21 -> P 21 21 2 | 1.78 -> 1.87 | 8.4 -> 10.4 | 17.6% -> 26.6% | 0.998 | 36.81 -> 36.97 | - -> 0.573 | 0.638 -> 0.578 | - -> 0.20 | - | - | 159 -> 77 | d_min, isa, r_meas, rfree, time | +| 9o0h | open | pass | P 21 21 21 | 1.98 -> 2.01 | 4.1 -> 6.1 | 39.6% -> 52.6% | 0.979 -> 0.985 | 0.06 -> 0.25 | - -> 0.247 | 0.272 -> 0.255 | - -> 0.04 | - | - | 64 | isa, r_meas, cc_half, rfree | +| 9p7q | open | pass | C 1 2 1 | 1.83 -> 1.81 | 8.1 -> 10.6 | 40.3% -> 25.7% | 0.989 -> 0.988 | 0.14 | - -> 0.270 | 0.279 -> 0.285 | - -> 0.03 | - | - | 19 -> 15 | isa, r_meas, completeness, rfree | +| 9pbb | open | pass | C 1 2 1 | 1.85 -> 1.83 | 9.7 -> 14.5 | 26.0% -> 20.7% | 0.993 -> 0.995 | 0.15 | - -> 0.236 | 0.257 -> 0.236 | - -> 0.04 | - | - | 22 -> 19 | isa, r_meas, completeness, rfree | +| 9q41 | open | pass | C 2 2 21 | 1.70 -> 1.69 | 8.3 -> 7.8 | 19.7% -> 27.3% | 0.989 -> 0.984 | 0.24 -> 0.20 | - -> 0.187 | 0.218 -> 0.191 | - -> 0.09 | - | - | 62 -> 57 | isa, r_meas, rfree | +| 9q66 | open | pass | P 1 21 1 | 2.04 | 11.2 -> 13.0 | 20.9% -> 31.7% | 0.995 -> 0.990 | 0.15 -> 0.34 | - -> 0.204 | 0.250 -> 0.216 | - -> 0.09 | - | - | 80 -> 68 | isa, r_meas, cc_half, rfree | +| 9qw8 | open | pass | P 1 | 1.70 -> 1.59 | 7.2 -> 11.3 | 22.3% -> 14.6% | 0.979 -> 0.994 | 0.42 -> 0.12 | - -> 0.237 | 0.337 -> 0.248 | - -> 0.10 | - | - | 275 -> 73 | d_min, isa, r_meas, cc_half, completeness, cell_dev_pct, rfree, time | +| 9rci | open | fail -> pass | P 1 2 1 -> P 1 | 1.87 -> 1.82 | 3.6 -> 6.0 | 34.7% -> 29.4% | 0.940 -> 0.942 | 98.08 -> 97.59 | - -> 0.576 | 0.591 -> 0.571 | - -> 0.09 | - | - | 40 -> 25 | verdict fail->pass, space group, d_min, isa, r_meas, completeness, cell_dev_pct, rfree, time | +| 9rcs | open | pass | P 1 21 1 | 3.56 -> 3.37 | 5.3 -> 6.0 | 21.4% -> 24.3% | 0.992 | 0.70 -> 0.93 | - -> 0.383 | 0.383 -> 0.442 | - -> 0.19 | - | - | 72 -> 55 | d_min, isa, r_meas, cell_dev_pct, rfree | +| 9rp9 | open | pass | C 1 2 1 | 1.89 -> 1.91 | 23.2 -> 28.8 | 14.9% -> 16.1% | 0.998 -> 0.997 | 0.06 -> 0.16 | - -> 0.206 | 0.237 -> 0.224 | - -> 0.08 | - | - | 40 -> 23 | isa, r_meas, rfree, time | +| 9sl0 | open | pass | P 21 21 21 | 1.38 | 18.3 -> 20.0 | 7.0% -> 7.6% | 1.000 | 0.37 | - -> 0.245 | 0.266 -> 0.256 | - -> 0.15 | - | - | 67 -> 50 | isa, r_meas, rfree, time | +| 9t6s | open | pass | P 21 21 21 | 1.74 -> 1.76 | 15.4 -> 24.9 | 11.0% -> 10.5% | 0.999 | 0.06 | - -> 0.220 | 0.239 -> 0.236 | - -> 0.03 | - | - | 46 -> 28 | isa, time | +| 9upt | open | pass | P 6 | 2.03 | 5.6 -> 6.4 | 28.9% -> 25.9% | 0.984 -> 0.988 | 0.20 | - -> 0.227 | 0.253 -> 0.229 | - -> 0.08 | - | - | 88 -> 76 | isa, r_meas, rfree | +| 9vyb | open | pass | P 21 21 21 | 1.82 -> 1.71 | 10.3 -> 16.7 | 11.6% -> 8.0% | 0.999 -> 1.000 | 0.62 -> 0.40 | - -> 0.245 | 0.312 -> 0.258 | - -> 0.07 | - | - | 29 -> 41 | d_min, isa, r_meas, completeness, cell_dev_pct, rfree, time | +| 9w3y | open | pass | P 21 21 21 | 1.18 -> 1.20 | 16.2 -> 19.9 | 17.4% -> 23.4% | 0.998 -> 0.997 | 0.13 -> 0.25 | - -> 0.186 | 0.228 -> 0.200 | - -> 0.11 | - | - | 24 -> 17 | isa, r_meas, rfree | +| 9yl4 | open | pass | P 21 21 21 | 3.54 -> 3.61 | 7.3 -> 9.5 | 21.1% -> 32.0% | 0.985 | 0.54 -> 0.08 | - -> 0.295 | 0.300 -> 0.306 | - -> 0.07 | - | - | 90 -> 85 | d_min, isa, r_meas, cell_dev_pct, rfree | +| 9yzk | open | pass | C 1 2 1 -> I 1 2 1 | 3.48 -> 3.86 | 9.0 -> 7.8 | 39.6% -> 29.7% | 0.959 -> 0.996 | 1.79 -> 0.20 | - -> 0.375 | 0.482 -> 0.394 | - -> 0.29 | - | - | 21 -> 15 | d_min, isa, r_meas, cc_half, cell_dev_pct, rfree | +| 9z44 | open | pass | C 1 2 1 -> I 1 2 1 | 6.87 -> 6.86 | - -> 5.9 | 14.2% -> 26.6% | 0.997 -> 0.953 | 11.14 -> 1.35 | - -> 0.336 | 0.303 -> 0.325 | - -> 0.17 | - | - | 29 -> 32 | isa, r_meas, cc_half, completeness, cell_dev_pct, rfree | +| 9z72 | open | pass | P 31 2 1 | 1.90 -> 1.97 | 9.5 -> 11.8 | 33.7% -> 76.6% | 0.993 -> 0.991 | 0.05 -> 0.15 | - -> 0.241 | 0.281 -> 0.256 | - -> 0.06 | - | - | 191 -> 152 | d_min, isa, r_meas, rfree | +| 9zlo | open | pass | P 21 21 21 | 1.57 -> 1.58 | 18.7 -> 23.0 | 10.0% -> 12.0% | 0.999 | 0.34 -> 0.39 | - -> 0.235 | 0.242 -> 0.241 | - -> 0.05 | - | - | 40 -> 26 | isa, r_meas, time | +| 9zm0 | open | pass | P 1 21 1 | 1.75 -> 1.81 | 6.2 -> 8.6 | 21.4% -> 23.7% | 0.994 -> 0.996 | 0.41 -> 0.16 | - -> 0.262 | 0.291 -> 0.283 | - -> 0.12 | - | - | 19 -> 11 | d_min, isa, r_meas, cell_dev_pct, rfree | +| 9zmu | open | pass | P 61 2 2 | 1.72 -> 1.71 | 8.2 -> 10.8 | 21.6% -> 34.0% | 0.999 | 0.27 -> 0.25 | - -> 0.276 | 0.309 -> 0.285 | - -> 0.17 | - | - | 87 -> 39 | isa, r_meas, rfree, time | +| cuhf2 | open | unscored | P 4 -> P 2 2 2 | 0.51 -> 0.52 | 2.4 -> 3.8 | 27.1% -> 17.4% | 0.980 -> 0.988 | - | - | - | - | - | - | 140 -> 56 | space group, isa, r_meas, cc_half, completeness, time | +| cytidine | open | pass | P 21 21 21 | 0.58 | 1.5 -> 4.5 | 39.4% -> 20.6% | 0.783 -> 0.980 | 0.31 | - | - | - | - | - | 77 -> 29 | isa, r_meas, cc_half, completeness, time | +| dnba | open | pass | C 1 2/c 1 | 0.81 | 37.3 -> 4.3 | 4.2% -> 16.7% | 0.996 -> 0.979 | 0.06 | - | - | - | - | - | 15 -> 10 | isa, r_meas, cc_half | +| lalanine | open | fail -> pass | P 1 2 1 -> P 21 21 21 | 1.79 -> 0.65 | - -> 7.5 | 57.8% -> 12.1% | 0.944 -> 0.994 | 1.55 -> 0.80 | - | - | - | - | - | 12 -> 14 | verdict fail->pass, space group, d_min, isa, r_meas, cc_half, completeness, cell_dev_pct | +| metformin | open | pass | P 1 21/a 1 -> P 1 21/c 1 | 0.50 -> 0.51 | 8.9 -> 13.6 | 9.1% -> 6.4% | 0.997 -> 0.998 | 0.17 | - | - | - | - | - | 12 -> 8 | isa, r_meas, completeness | +| nidppe | open | pass | P 1 21/c 1 | 0.51 | 34.8 -> 25.6 | 7.5% -> 8.2% | 0.999 -> 0.998 | 0.28 | - | - | - | - | - | 16 -> 9 | isa, r_meas | +| lysoI_micromax_mono | inhouse | - -> pass | - -> P 41 21 2 | - -> 1.35 | - -> 37.3 | - -> 6.9% | - -> 1.000 | - -> 0.06 | - | - | - | - -> 5.7% | - -> 3.0% | - -> 25 | only in B | +| lysoI_micromax_pink | inhouse | - -> pass | - -> P 41 21 2 | - -> 1.42 | - -> 33.5 | - -> 7.4% | - -> 1.000 | - -> 0.09 | - | - | - | - -> 6.3% | - -> 3.1% | - -> 17 | only in B | +| lyso_micromax_mono | inhouse | - -> pass | - -> P 41 21 2 | - -> 1.20 | - -> 39.0 | - -> 4.9% | - -> 1.000 | - -> 0.16 | - | - | - | - -> 4.6% | - -> 2.5% | - -> 24 | only in B | +| lyso_micromax_pink | inhouse | - -> pass | - -> P 41 21 2 | - -> 1.26 | - -> 35.7 | - -> 5.2% | - -> 1.000 | - -> 0.16 | - | - | - | - -> 4.9% | - -> 2.5% | - -> 17 | only in B | +| 5ky6 | open | - -> pass | - -> P 1 21 1 | - -> 1.55 | - -> 7.2 | - -> 27.3% | - -> 0.987 | - -> 0.64 | - -> 0.245 | - -> 0.249 | - -> 0.11 | - | - | - -> 92 | only in B | +| 5mln | open | - -> pass | - -> P 21 21 2 | - -> 1.26 | - -> 21.4 | - -> 10.4% | - -> 0.999 | - -> 0.10 | - -> 0.184 | - -> 0.189 | - -> 0.14 | - | - | - -> 45 | only in B | +| 5t39 | open | - -> pass | - -> P 1 21 1 | - -> 1.01 | - -> 16.0 | - -> 7.0% | - -> 0.998 | - -> 0.13 | - -> 0.152 | - -> 0.165 | - -> 0.17 | - | - | - -> 75 | only in B | +| 6cdl | open | - -> pass | - -> P 21 21 2 | - -> 1.15 | - -> 11.2 | - -> 8.2% | - -> 0.996 | - -> 1.22 | - -> 0.150 | - -> 0.174 | - -> 0.28 | - | - | - -> 57 | only in B | +| 6f3p | open | - -> pass | - -> C 1 2 1 | - -> 1.13 | - -> 9.4 | - -> 10.6% | - -> 0.997 | - -> 0.12 | - -> 0.144 | - -> 0.167 | - -> 0.18 | - | - | - -> 130 | only in B | +| 6g1f | open | - -> pass | - -> C 1 2 1 | - -> 1.93 | - -> 21.2 | - -> 11.7% | - -> 0.997 | - -> 0.04 | - -> 0.210 | - -> 0.224 | - -> 0.14 | - | - | - -> 140 | only in B | +| 6jgh | open | - -> pass | - -> P 21 21 21 | - -> 0.87 | - -> 6.7 | - -> 22.7% | - -> 0.989 | - -> 0.39 | - -> 0.148 | - -> 0.169 | - -> 0.19 | - | - | - -> 105 | only in B | +| 6nen | open | - -> pass | - -> P 3 1 2 | - -> 1.72 | - -> 6.3 | - -> 26.9% | - -> 0.997 | - -> 0.07 | - -> 0.210 | - -> 0.212 | - -> 0.08 | - | - | - -> 24 | only in B | +| 6qaj | open | - -> pass | - -> C 2 2 21 | - -> 2.70 | - -> 13.1 | - -> 24.6% | - -> 0.997 | - -> 0.41 | - -> 0.376 | - -> 0.390 | - -> 0.48 | - | - | - -> 67 | only in B | +| 6s1u | open | - -> pass | - -> P 1 21 1 | - -> 1.75 | - -> 13.7 | - -> 22.4% | - -> 0.993 | - -> 0.13 | - -> 0.204 | - -> 0.208 | - -> 0.09 | - | - | - -> 36 | only in B | +| 6w75 | open | - -> pass | - -> P 31 2 1 | - -> 1.68 | - -> 15.8 | - -> 11.9% | - -> 0.999 | - -> 0.01 | - -> 0.177 | - -> 0.185 | - -> 0.13 | - | - | - -> 92 | only in B | +| 6z9g | open | - -> fail | - -> P 1 21 1 | - -> 1.60 | - -> 14.6 | - -> 11.5% | - -> 0.997 | - -> 22.53 | - -> 0.533 | - -> 0.542 | - -> 0.18 | - | - | - -> 90 | only in B | +| 6zqr | open | - -> pass | - -> P 4 | - -> 1.79 | - -> 8.7 | - -> 19.6% | - -> 0.996 | - -> 0.40 | - -> 0.190 | - -> 0.198 | - -> 0.12 | - | - | - -> 45 | only in B | +| 6zqy | open | - -> pass | - -> P 4 | - -> 1.72 | - -> 13.8 | - -> 18.5% | - -> 0.997 | - -> 0.12 | - -> 0.212 | - -> 0.226 | - -> 0.20 | - | - | - -> 62 | only in B | +| 6zr0 | open | - -> pass | - -> P 4 | - -> 1.73 | - -> 24.3 | - -> 11.1% | - -> 0.997 | - -> 0.08 | - -> 0.217 | - -> 0.224 | - -> 0.14 | - | - | - -> 61 | only in B | +| 7ou1 | open | - -> pass | - -> P 1 21 1 | - -> 1.41 | - -> 9.5 | - -> 16.1% | - -> 0.991 | - -> 0.04 | - -> 0.206 | - -> 0.211 | - -> 0.09 | - | - | - -> 44 | only in B | +| 7raa | open | - -> pass | - -> P 41 21 2 | - -> 2.59 | - -> 11.0 | - -> 17.1% | - -> 1.000 | - -> 0.04 | - -> 0.299 | - -> 0.336 | - -> 0.18 | - | - | - -> 186 | only in B | +| 9fcf | open | - -> pass | - -> P 4 | - -> 1.81 | - -> 7.8 | - -> 28.6% | - -> 0.995 | - -> 0.04 | - -> 0.285 | - -> 0.293 | - -> 0.06 | - | - | - -> 142 | only in B | +| 9h0q | open | - -> pass | - -> R 3 2:H | - -> 2.13 | - -> 18.7 | - -> 15.4% | - -> 0.998 | - -> 0.41 | - -> 0.203 | - -> 0.233 | - -> 0.18 | - | - | - -> 72 | only in B | diff --git a/tools/battery/README.md b/tools/battery/README.md index 5caefd8d1..c851d972d 100644 --- a/tools/battery/README.md +++ b/tools/battery/README.md @@ -493,6 +493,8 @@ whole range; the manifest keeps the rule used as `dmin_rule`. disagrees with an unpinned manifest input. Add the row to `open.json` with the deposited reference (`sg`, `sgno`, `cell`, `dmin`) and tags. Set `"pinned": true` when the choice between sweeps was deliberate. Record the source and DOI in `docs/EXTERNAL_TEST_DATA.md`. +- **Every row** carries `wavelength` (A), the value rugnux reads from the image headers + (`WAVELENGTH` in `p_report.txt`). It is descriptive only; nothing scores it. - **Inhouse / private**: add the row (`id`, `input`, tags, and `"expect": "no_lattice"` for a control), then run `battery.py refs --arm inhouse --write` to fill in `ref` from the `CORRECT.LP` beside the input. Without `--write`, `refs` only shows what would change. diff --git a/tools/battery/inhouse.json b/tools/battery/inhouse.json index 0e6d086b8..121c4c37c 100644 --- a/tools/battery/inhouse.json +++ b/tools/battery/inhouse.json @@ -1,46 +1,46 @@ {"arm": "inhouse", "aliases": {"E08_lyso_2_006": "lyso_x10sa_90deg_1", "E08_lyso_2_007": "lyso_x10sa_90deg_2", "lyso_2": "lyso_x06da_atten_wedge", "lyso_6_5keV": "lyso_x06da_5keV", "lysoC_14_f46f0a": "lyso_x06da_ice", "Thau_9": "thau_x10sa_0p1deg", "thau_BL1A_1p9A": "thau_bl1a_6p5keV", "thau_BL1A_2p7A": "thau_bl1a_4p6keV", "thau_BL1A_3p3A": "thau_bl1a_3p8keV", "Ins_H_2_fa9841": "insu_H_x06da_twin", "Ins_H_3_d05684": "insu_H_x06da_notwin", "Ins_I_16": "insu_I_x06da_weak", "Ins_I_2_1a530d": "insu_I_x06da_low_isa", "Ins_I_3_5e90e3": "insu_I_x06da_ref", "Ins_I_6_13keV": "insu_I_x06da_13keV", "Ins_I_6_5keV": "insu_I_x06da_5keV", "Ins_I_6_6keV": "insu_I_x06da_6keV", "Ins_I_7_5keV": "insu_I_x06da_5keV_2", "cytC_10": "cytc_x10sa", "cytC_2_0770b5": "cytc_x06da_1", "cytC_3_874285": "cytc_x06da_2", "MyoB_13": "myob_x10sa", "MyoB_9_446b70": "myob_x06da", "MyoB2-1": "myob_x06da_powder_1", "MyoB2-2": "myob_x06da_powder_2", "MyoB2-4": "myob_x06da_split", "MyoB2-5": "myob_x06da_sparse", "onlyice": "nothing_ice", "nothing_3400ac": "nothing_2", "nothing_9bf604": "nothing_1", "lyso_90deg_1": "lyso_x10sa_90deg_1", "lyso_90deg_2": "lyso_x10sa_90deg_2", "lyso_atten_wedge": "lyso_x06da_atten_wedge", "lyso_5keV": "lyso_x06da_5keV", "lyso_ice": "lyso_x06da_ice", "lyso_ref": "lyso_x06da_ref", "lyso_strong": "lyso_x10sa_strong", "lyso_half_image": "lyso_x06da_half_image", "thau_16keV": "thau_x10sa_16keV", "thau_0p1deg": "thau_x10sa_0p1deg", "thau_1p9A": "thau_bl1a_6p5keV", "thau_2p7A": "thau_bl1a_4p6keV", "thau_3p3A": "thau_bl1a_3p8keV", "thau_injection": "thau_x10sa_injection", "thau_pink": "thau_micromax_pink", "insu_H_twin": "insu_H_x06da_twin", "insu_H_notwin": "insu_H_x06da_notwin", "insu_I_weak": "insu_I_x06da_weak", "insu_I_low_isa": "insu_I_x06da_low_isa", "insu_I_ref": "insu_I_x06da_ref", "insu_I_13keV": "insu_I_x06da_13keV", "insu_I_5keV": "insu_I_x06da_5keV", "insu_I_6keV": "insu_I_x06da_6keV", "insu_I_5keV_2": "insu_I_x06da_5keV_2", "cytc_eiger": "cytc_x10sa", "cytc_pilatus_1": "cytc_x06da_1", "cytc_pilatus_2": "cytc_x06da_2", "myob_eiger": "myob_x10sa", "myob_pilatus": "myob_x06da", "myob_powder_1": "myob_x06da_powder_1", "myob_powder_2": "myob_x06da_powder_2", "myob_split": "myob_x06da_split", "myob_aggregate": "myob_x06da_sparse", "myob_x06da_aggregate": "myob_x06da_sparse"}, "sets": [ - {"id": "lyso_x10sa_90deg_1", "input": "lyso_x10sa_90deg_1/lyso_2_006_master.h5", "ref": {"sgno": 92, "cell": [76.879, 76.879, 36.961, 90.0, 90.0, 90.0], "anomalous": true, "isa": 20.75, "completeness": 99.3, "r_meas": 0.182, "cc_half": 0.995, "multiplicity": 3.51, "dmin_low": 5.83, "r_meas_low": 0.049, "dmin": 1.966, "dmin_rule": "xds_range", "dmin_xds": 1.966, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, - {"id": "lyso_x10sa_90deg_2", "input": "lyso_x10sa_90deg_2/lyso_2_007_master.h5", "ref": {"sgno": 92, "cell": [76.904, 76.904, 36.976, 90.0, 90.0, 90.0], "anomalous": true, "isa": 18.14, "completeness": 99.3, "r_meas": 0.18, "cc_half": 0.994, "multiplicity": 3.51, "dmin_low": 5.86, "r_meas_low": 0.051, "dmin": 1.973, "dmin_rule": "xds_range", "dmin_xds": 1.973, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, - {"id": "lyso_x06da_atten_wedge", "input": "lyso_x06da_atten_wedge/lyso_2_002_master.h5", "ref": {"sgno": 92, "cell": [78.235, 78.235, 37.798, 90.0, 90.0, 90.0], "anomalous": true, "isa": 16.56, "completeness": 99.9, "r_meas": 0.239, "cc_half": 0.998, "multiplicity": 10.97, "dmin_low": 3.76, "r_meas_low": 0.069, "dmin": 1.264, "dmin_rule": "xds_range", "dmin_xds": 1.264, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, - {"id": "lyso_x06da_5keV", "input": "lyso_x06da_5keV/lyso_6_001_master.h5", "ref": {"sgno": 92, "cell": [78.225, 78.225, 37.897, 90.0, 90.0, 90.0], "anomalous": true, "isa": 19.36, "completeness": 86.4, "r_meas": 0.064, "cc_half": 0.998, "multiplicity": 8.85, "dmin_low": 7.22, "r_meas_low": 0.047, "dmin": 2.45, "dmin_rule": "xds_range", "dmin_xds": 2.45, "dmax": 50.0}, "tags": ["h5", "lysozyme", "long-wavelength"]}, - {"id": "lyso_x06da_ice", "input": "lyso_x06da_ice/lysoC_14_56b5b9_master.h5", "ref": {"sgno": 89, "cell": [78.387, 78.387, 37.427, 90.0, 90.0, 90.0], "anomalous": true, "isa": 23.33, "completeness": 100.0, "r_meas": 0.219, "cc_half": 0.998, "multiplicity": 11.86, "dmin_low": 4.26, "r_meas_low": 0.055, "dmin": 1.431, "dmin_rule": "xds_range", "dmin_xds": 1.431, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, - {"id": "lyso_x06da_ref", "input": "lyso_x06da_ref/hewl_35_001_master.h5", "ref": {"sgno": 89, "cell": [78.091, 78.091, 37.704, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.31, "completeness": 100.0, "r_meas": 0.045, "cc_half": 1.0, "multiplicity": 13.76, "dmin_low": 3.58, "r_meas_low": 0.029, "dmin": 1.2, "dmin_rule": "xds_range", "dmin_xds": 1.2, "dmax": 50.0}, "tags": ["h5", "lysozyme"], "tiers": {"smoke": "lysozyme reference set"}}, - {"id": "lyso_x10sa_strong", "input": "lyso_x10sa_strong/lyso_5_f6ac7e_master.h5", "ref": {"sgno": 92, "cell": [77.769, 77.769, 37.442, 90.0, 90.0, 90.0], "anomalous": false, "isa": 8.63, "completeness": 65.8, "r_meas": 0.112, "cc_half": 0.998, "multiplicity": 11.46, "dmin_low": 3.52, "r_meas_low": 0.077, "dmin": 1.24, "dmin_rule": "cc_half_0.30", "dmin_xds": 1.18, "dmax": 999.0}, "tags": ["h5", "lysozyme"]}, - {"id": "lyso_x06da_half_image", "input": "lyso_x06da_half_image/lyso3_7_b81d83_master.h5", "ref": {"sgno": 96, "cell": [77.74, 77.74, 37.386, 90.0, 90.0, 90.0], "anomalous": true, "isa": 6.61, "completeness": 99.6, "r_meas": 0.595, "cc_half": 0.958, "multiplicity": 5.31, "dmin_low": 4.91, "r_meas_low": 0.258, "dmin": 1.65, "dmin_rule": "xds_range", "dmin_xds": 1.65, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, - {"id": "thau_x10sa_16keV", "input": "thau_x10sa_16keV/lyso_4_d2bb9b_master.h5", "ref": {"sgno": 92, "cell": [57.958, 57.958, 150.547, 90.0, 90.0, 90.0], "anomalous": false, "isa": 44.51, "completeness": 94.3, "r_meas": 0.072, "cc_half": 1.0, "multiplicity": 18.36, "dmin_low": 3.89, "r_meas_low": 0.027, "dmin": 1.3, "dmin_rule": "xds_range", "dmin_xds": 1.3, "dmax": 50.0}, "tags": ["h5", "thaumatin"]}, - {"id": "thau_x10sa_0p1deg", "input": "thau_x10sa_0p1deg/Thau_9_002_master.h5", "ref": {"sgno": 92, "cell": [58.488, 58.488, 150.788, 90.0, 90.0, 90.0], "anomalous": true, "isa": 9.15, "completeness": 99.0, "r_meas": 0.23, "cc_half": 0.997, "multiplicity": 11.37, "dmin_low": 6.51, "r_meas_low": 0.104, "dmin": 2.197, "dmin_rule": "xds_range", "dmin_xds": 2.197, "dmax": 50.0}, "tags": ["h5", "thaumatin"]}, - {"id": "thau_bl1a_6p5keV", "input": "thau_bl1a_6p5keV/JTH14_1p9_50ds_90T_k0_JF4M_01_50001_master.h5", "ref": {"sgno": 89, "cell": [58.031, 58.031, 150.577, 90.0, 90.0, 90.0], "anomalous": true, "isa": 34.46, "completeness": 91.8, "r_meas": 0.072, "cc_half": 0.999, "multiplicity": 7.95, "dmin_low": 5.33, "r_meas_low": 0.041, "dmin": 1.78, "dmin_rule": "xds_range", "dmin_xds": 1.78, "dmax": 100.0}, "tags": ["h5", "thaumatin", "long-wavelength"]}, - {"id": "thau_bl1a_4p6keV", "input": "thau_bl1a_4p6keV/JTH14_2p7_50ds_19T_k0_JF4M_01_50001_master.h5", "ref": {"sgno": 89, "cell": [58.007, 58.007, 150.511, 90.0, 90.0, 90.0], "anomalous": true, "isa": 35.64, "completeness": 92.0, "r_meas": 0.062, "cc_half": 0.998, "multiplicity": 7.94, "dmin_low": 7.56, "r_meas_low": 0.039, "dmin": 2.53, "dmin_rule": "xds_range", "dmin_xds": 2.53, "dmax": 100.0}, "tags": ["h5", "thaumatin", "long-wavelength"]}, - {"id": "thau_bl1a_3p8keV", "input": "thau_bl1a_3p8keV/JTH14_3p3_50ds_12T_k0_JF4M_01_50001_master.h5", "ref": {"sgno": 89, "cell": [57.895, 57.895, 150.213, 90.0, 90.0, 90.0], "anomalous": true, "isa": 30.8, "completeness": 92.2, "r_meas": 0.057, "cc_half": 0.997, "multiplicity": 7.92, "dmin_low": 9.23, "r_meas_low": 0.039, "dmin": 3.1, "dmin_rule": "xds_range", "dmin_xds": 3.1, "dmax": 100.0}, "tags": ["h5", "thaumatin", "long-wavelength"], "tiers": {"smoke": "long wavelength (3.3 A), KEK BL-1A thaumatin"}}, - {"id": "thau_x10sa_injection", "input": "thau_x10sa_injection/lyso_13_0c7e7a_master.h5", "ref": {"sgno": 92, "cell": [57.922, 57.922, 150.678, 90.0, 90.0, 90.0], "anomalous": true, "isa": 36.23, "completeness": 86.2, "r_meas": 0.037, "cc_half": 1.0, "multiplicity": 9.96, "dmin_low": 3.83, "r_meas_low": 0.022, "dmin": 1.28, "dmin_rule": "xds_range", "dmin_xds": 1.28, "dmax": 50.0}, "tags": ["h5", "thaumatin"]}, - {"id": "thau_micromax_pink", "input": "thau_micromax_pink/Thau-Thaumatin_2_master.h5", "ref": {"sgno": 92, "cell": [58.784, 58.784, 152.226, 90.0, 90.0, 90.0], "anomalous": true, "isa": 21.18, "completeness": 98.5, "r_meas": 0.08, "cc_half": 0.999, "multiplicity": 9.38, "dmin_low": 4.17, "r_meas_low": 0.04, "dmin": 1.4, "dmin_rule": "xds_range", "dmin_xds": 1.4, "dmax": 50.0}, "tags": ["h5", "thaumatin", "pink-beam"], "tiers": {"smoke": "pink beam"}}, - {"id": "lyso_micromax_mono", "input": "lyso_micromax_mono/lysozyme-s6_1_master.h5", "tags": ["h5", "lysozyme"], "ref": {"sgno": 96, "cell": [77.668, 77.668, 37.486, 90.0, 90.0, 90.0], "anomalous": true, "isa": 39.93, "completeness": 100.0, "r_meas": 0.047, "cc_half": 1.0, "multiplicity": 6.22, "dmin_low": 4.54, "r_meas_low": 0.023, "dmin": 1.5, "dmin_rule": "xds_range", "dmin_xds": 1.5, "dmax": 50.0}}, - {"id": "lyso_micromax_pink", "input": "lyso_micromax_pink/lysozyme-s6_1_master.h5", "tags": ["h5", "lysozyme", "pink-beam"], "ref": {"sgno": 96, "cell": [77.654, 77.654, 37.483, 90.0, 90.0, 90.0], "anomalous": true, "isa": 37.35, "completeness": 99.9, "r_meas": 0.051, "cc_half": 1.0, "multiplicity": 5.96, "dmin_low": 4.39, "r_meas_low": 0.022, "dmin": 1.45, "dmin_rule": "xds_range", "dmin_xds": 1.45, "dmax": 50.0}}, - {"id": "lysoI_micromax_mono", "input": "lysoI_micromax_mono/lysozyme-Is2_1_master.h5", "tags": ["h5", "lysozyme", "iodine"], "ref": {"sgno": 96, "cell": [78.335, 78.335, 37.46, 90.0, 90.0, 90.0], "anomalous": true, "isa": 31.42, "completeness": 100.0, "r_meas": 0.068, "cc_half": 0.999, "multiplicity": 6.78, "dmin_low": 4.99, "r_meas_low": 0.028, "dmin": 1.65, "dmin_rule": "xds_range", "dmin_xds": 1.65, "dmax": 50.0}}, - {"id": "lysoI_micromax_pink", "input": "lysoI_micromax_pink/lysozyme-Is2_1_master.h5", "tags": ["h5", "lysozyme", "iodine", "pink-beam"], "ref": {"sgno": 96, "cell": [78.302, 78.302, 37.448, 90.0, 90.0, 90.0], "anomalous": true, "isa": 29.19, "completeness": 100.0, "r_meas": 0.079, "cc_half": 0.999, "multiplicity": 6.8, "dmin_low": 4.99, "r_meas_low": 0.028, "dmin": 1.65, "dmin_rule": "xds_range", "dmin_xds": 1.65, "dmax": 50.0}}, - {"id": "insu_H_x06da_twin", "input": "insu_H_x06da_twin/Ins_H_2_fa9841_master.h5", "ref": {"sgno": 146, "cell": [81.393, 81.393, 33.272, 90.0, 90.0, 120.0], "anomalous": true, "isa": 6.82, "completeness": 95.0, "r_meas": 0.118, "cc_half": 0.988, "multiplicity": 3.61, "dmin_low": 4.34, "r_meas_low": 0.106, "dmin": 1.455, "dmin_rule": "xds_range", "dmin_xds": 1.455, "dmax": 50.0}, "tags": ["h5", "insulin"]}, - {"id": "insu_H_x06da_notwin", "input": "insu_H_x06da_notwin/Ins_H_3_d05684_master.h5", "ref": {"sgno": 146, "cell": [81.378, 81.378, 33.269, 90.0, 90.0, 120.0], "anomalous": true, "isa": 17.69, "completeness": 97.4, "r_meas": 0.068, "cc_half": 0.998, "multiplicity": 3.52, "dmin_low": 4.61, "r_meas_low": 0.045, "dmin": 1.544, "dmin_rule": "xds_range", "dmin_xds": 1.544, "dmax": 50.0}, "tags": ["h5", "insulin"], "tiers": {"smoke": "rhombohedral insulin (H3)"}}, - {"id": "insu_I_x06da_weak", "input": "insu_I_x06da_weak/Ins_16_776a92_master.h5", "ref": {"sgno": 197, "cell": [77.085, 77.085, 77.085, 90.0, 90.0, 90.0], "anomalous": false, "isa": 18.91, "completeness": 92.4, "r_meas": 0.401, "cc_half": 0.999, "multiplicity": 29.04, "dmin_low": 3.22, "r_meas_low": 0.054, "dmin": 1.807, "dmin_rule": "cc_half_0.30", "dmin_xds": 1.08, "dmax": 999.0}, "tags": ["h5", "insulin"]}, - {"id": "insu_I_x06da_low_isa", "input": "insu_I_x06da_low_isa/Ins_I_2_1a530d_master.h5", "ref": {"sgno": 197, "cell": [77.582, 77.582, 77.582, 90.0, 90.0, 90.0], "anomalous": true, "isa": 4.15, "completeness": 90.8, "r_meas": 0.22, "cc_half": 0.998, "multiplicity": 13.33, "dmin_low": 3.92, "r_meas_low": 0.167, "dmin": 1.3, "dmin_rule": "xds_range", "dmin_xds": 1.3, "dmax": 50.0}, "tags": ["h5", "insulin"]}, - {"id": "insu_I_x06da_ref", "input": "insu_I_x06da_ref/Ins_I_3_5e90e3_master.h5", "ref": {"sgno": 197, "cell": [77.52, 77.52, 77.52, 90.0, 90.0, 90.0], "anomalous": true, "isa": 25.08, "completeness": 100.0, "r_meas": 0.349, "cc_half": 0.998, "multiplicity": 13.82, "dmin_low": 4.82, "r_meas_low": 0.143, "dmin": 1.621, "dmin_rule": "xds_range", "dmin_xds": 1.621, "dmax": 50.0}, "tags": ["h5", "insulin"]}, - {"id": "insu_I_x06da_13keV", "input": "insu_I_x06da_13keV/Ins_I_6_001_master.h5", "ref": {"sgno": 197, "cell": [77.468, 77.468, 77.468, 90.0, 90.0, 90.0], "anomalous": true, "isa": 18.83, "completeness": 100.0, "r_meas": 0.279, "cc_half": 0.998, "multiplicity": 17.21, "dmin_low": 4.85, "r_meas_low": 0.093, "dmin": 1.635, "dmin_rule": "xds_range", "dmin_xds": 1.635, "dmax": 50.0}, "tags": ["h5", "insulin"]}, - {"id": "insu_I_x06da_5keV", "input": "insu_I_x06da_5keV/Ins_I_6_006_master.h5", "ref": {"sgno": 197, "cell": [77.785, 77.785, 77.785, 90.0, 90.0, 90.0], "anomalous": true, "isa": 17.54, "completeness": 91.7, "r_meas": 0.073, "cc_half": 0.999, "multiplicity": 11.17, "dmin_low": 7.21, "r_meas_low": 0.048, "dmin": 2.45, "dmin_rule": "xds_range", "dmin_xds": 2.45, "dmax": 50.0}, "tags": ["h5", "insulin", "long-wavelength"]}, - {"id": "insu_I_x06da_6keV", "input": "insu_I_x06da_6keV/Ins_I_6_004_master.h5", "ref": {"sgno": 197, "cell": [77.71, 77.71, 77.71, 90.0, 90.0, 90.0], "anomalous": true, "isa": 17.95, "completeness": 92.4, "r_meas": 0.065, "cc_half": 0.999, "multiplicity": 12.42, "dmin_low": 6.03, "r_meas_low": 0.047, "dmin": 2.04, "dmin_rule": "xds_range", "dmin_xds": 2.04, "dmax": 50.0}, "tags": ["h5", "insulin", "long-wavelength"]}, - {"id": "insu_I_x06da_5keV_2", "input": "insu_I_x06da_5keV_2/Ins_I_7_001_master.h5", "ref": {"sgno": 197, "cell": [77.694, 77.694, 77.694, 90.0, 90.0, 90.0], "anomalous": true, "isa": 20.01, "completeness": 91.8, "r_meas": 0.076, "cc_half": 0.999, "multiplicity": 12.73, "dmin_low": 7.21, "r_meas_low": 0.048, "dmin": 2.45, "dmin_rule": "xds_range", "dmin_xds": 2.45, "dmax": 50.0}, "tags": ["h5", "insulin", "long-wavelength"], "tiers": {"smoke": "cubic insulin at 5 keV (long wavelength), fast"}}, - {"id": "cytc_x10sa", "input": "cytc_x10sa/cytC_10_002_master.h5", "ref": {"sgno": 152, "cell": [83.659, 83.659, 86.011, 90.0, 90.0, 120.0], "anomalous": true, "isa": 31.75, "completeness": 99.8, "r_meas": 0.232, "cc_half": 0.999, "multiplicity": 10.65, "dmin_low": 6.05, "r_meas_low": 0.036, "dmin": 2.267, "dmin_rule": "cc_half_0.30", "dmin_xds": 2.039, "dmax": 50.0}, "tags": ["h5", "cytochrome-c"]}, - {"id": "cytc_x06da_1", "input": "cytc_x06da_1/cytC_2_0770b5_master.h5", "ref": {"sgno": 152, "cell": [83.836, 83.836, 86.733, 90.0, 90.0, 120.0], "anomalous": true, "isa": 24.49, "completeness": 99.9, "r_meas": 0.084, "cc_half": 0.999, "multiplicity": 7.67, "dmin_low": 5.58, "r_meas_low": 0.033, "dmin": 1.875, "dmin_rule": "xds_range", "dmin_xds": 1.875, "dmax": 50.0}, "tags": ["h5", "cytochrome-c"]}, - {"id": "cytc_x06da_2", "input": "cytc_x06da_2/cytC_3_874285_master.h5", "ref": {"sgno": 152, "cell": [83.719, 83.719, 88.617, 90.0, 90.0, 120.0], "anomalous": true, "isa": 26.98, "completeness": 99.9, "r_meas": 0.097, "cc_half": 0.999, "multiplicity": 7.82, "dmin_low": 5.03, "r_meas_low": 0.032, "dmin": 1.69, "dmin_rule": "xds_range", "dmin_xds": 1.69, "dmax": 50.0}, "tags": ["h5", "cytochrome-c"]}, - {"id": "myob_x10sa", "input": "myob_x10sa/MyoB_13_110ee0_master.h5", "ref": {"sgno": 4, "cell": [35.229, 28.461, 63.076, 90.0, 106.504, 90.0], "anomalous": true, "isa": 5.21, "completeness": 97.7, "r_meas": 0.247, "cc_half": 0.965, "multiplicity": 3.29, "dmin_low": 5.16, "r_meas_low": 0.186, "dmin": 1.742, "dmin_rule": "xds_range", "dmin_xds": 1.742, "dmax": 50.0}, "tags": ["h5", "myoglobin"]}, - {"id": "myob_x06da", "input": "myob_x06da/MyoB_9_446b70_master.h5", "ref": {"sgno": 3, "cell": [35.182, 28.544, 62.83, 90.0, 106.138, 90.0], "anomalous": true, "isa": 7.59, "completeness": 99.4, "r_meas": 0.232, "cc_half": 0.987, "multiplicity": 3.47, "dmin_low": 4.23, "r_meas_low": 0.105, "dmin": 1.422, "dmin_rule": "xds_range", "dmin_xds": 1.422, "dmax": 50.0}, "tags": ["h5", "myoglobin"], "tiers": {"smoke": "monoclinic myoglobin, screw not arbitrable by XDS"}}, - {"id": "myob_x06da_powder_1", "input": "myob_x06da_powder_1/MyoB2-1_75979a_master.h5", "ref": {"sgno": 1, "cell": [28.705, 34.796, 64.373, 74.314, 89.879, 90.057], "anomalous": false, "isa": 5.54, "completeness": 74.2, "r_meas": 0.493, "cc_half": 0.966, "multiplicity": 2.87, "dmin_low": 4.44, "r_meas_low": 0.125, "dmin": 1.495, "dmin_rule": "xds_range", "dmin_xds": 1.495, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"], "tiers": {"smoke": "known hard: triclinic twin"}}, - {"id": "myob_x06da_powder_2", "input": "myob_x06da_powder_2/MyoB2-2_5ddd7d_master.h5", "ref": {"sgno": 3, "cell": [35.576, 28.811, 63.529, 90.0, 105.593, 90.0], "anomalous": false, "isa": 2.19, "completeness": 58.0, "r_meas": 1.106, "cc_half": 0.655, "multiplicity": 4.71, "dmin_low": 2.98, "r_meas_low": 0.706, "dmin": 0.99, "dmin_rule": "xds_range", "dmin_xds": 0.99, "dmax": 999.0}, "tags": ["h5", "myoglobin", "twin"]}, - {"id": "myob_x06da_split", "input": "myob_x06da_split/MyoB2-4_079740_master.h5", "ref": {"sgno": 3, "cell": [35.39, 28.783, 63.626, 90.0, 105.544, 90.0], "anomalous": false, "isa": 12.41, "completeness": 77.7, "r_meas": 0.577, "cc_half": 0.984, "multiplicity": 4.38, "dmin_low": 4.48, "r_meas_low": 0.091, "dmin": 1.506, "dmin_rule": "xds_range", "dmin_xds": 1.506, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, - {"id": "myob_x06da_sparse", "input": "myob_x06da_sparse/MyoB2-5_7cef9c_master.h5", "ref": {"sgno": 3, "cell": [35.388, 28.752, 64.083, 90.0, 106.352, 90.0], "anomalous": false, "isa": 5.46, "completeness": 73.7, "r_meas": 0.326, "cc_half": 0.972, "multiplicity": 5.21, "dmin_low": 5.92, "r_meas_low": 0.146, "dmin": 2.0, "dmin_rule": "xds_range", "dmin_xds": 2.0, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, - {"id": "nothing_2", "input": "nothing_2/test-28_3400ac_master.h5", "expect": "no_lattice", "tags": ["h5", "control"]}, - {"id": "nothing_1", "input": "nothing_1/test-28_9bf604_master.h5", "expect": "no_lattice", "tags": ["h5", "control"]}, - {"id": "aspirin_x10sa_20keV", "input": "aspirin_x10sa_20keV/aspirin_1_002_master.h5", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.262, 6.543, 11.257, 90.0, 95.916, 90.0], "anomalous": true, "isa": 27.97, "completeness": 84.0, "r_meas": 0.032, "cc_half": 0.999, "multiplicity": 3.14, "dmin_low": 2.01, "r_meas_low": 0.03, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, - {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.55, "completeness": 86.6, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.03, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, - {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.55, "completeness": 81.0, "r_meas": 0.04, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.043, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, - {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.39, "completeness": 91.6, "r_meas": 0.03, "cc_half": 0.999, "multiplicity": 5.64, "dmin_low": 2.0, "r_meas_low": 0.035, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, - {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"} + {"id": "lyso_x10sa_90deg_1", "input": "lyso_x10sa_90deg_1/lyso_2_006_master.h5", "wavelength": 0.99988, "ref": {"sgno": 92, "cell": [76.879, 76.879, 36.961, 90.0, 90.0, 90.0], "anomalous": true, "isa": 20.75, "completeness": 99.3, "r_meas": 0.182, "cc_half": 0.995, "multiplicity": 3.51, "dmin_low": 5.83, "r_meas_low": 0.049, "dmin": 1.966, "dmin_rule": "xds_range", "dmin_xds": 1.966, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, + {"id": "lyso_x10sa_90deg_2", "input": "lyso_x10sa_90deg_2/lyso_2_007_master.h5", "wavelength": 0.99988, "ref": {"sgno": 92, "cell": [76.904, 76.904, 36.976, 90.0, 90.0, 90.0], "anomalous": true, "isa": 18.14, "completeness": 99.3, "r_meas": 0.18, "cc_half": 0.994, "multiplicity": 3.51, "dmin_low": 5.86, "r_meas_low": 0.051, "dmin": 1.973, "dmin_rule": "xds_range", "dmin_xds": 1.973, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, + {"id": "lyso_x06da_atten_wedge", "input": "lyso_x06da_atten_wedge/lyso_2_002_master.h5", "wavelength": 0.95381, "ref": {"sgno": 92, "cell": [78.235, 78.235, 37.798, 90.0, 90.0, 90.0], "anomalous": true, "isa": 16.56, "completeness": 99.9, "r_meas": 0.239, "cc_half": 0.998, "multiplicity": 10.97, "dmin_low": 3.76, "r_meas_low": 0.069, "dmin": 1.264, "dmin_rule": "xds_range", "dmin_xds": 1.264, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, + {"id": "lyso_x06da_5keV", "input": "lyso_x06da_5keV/lyso_6_001_master.h5", "wavelength": 2.47924, "ref": {"sgno": 92, "cell": [78.225, 78.225, 37.897, 90.0, 90.0, 90.0], "anomalous": true, "isa": 19.36, "completeness": 86.4, "r_meas": 0.064, "cc_half": 0.998, "multiplicity": 8.85, "dmin_low": 7.22, "r_meas_low": 0.047, "dmin": 2.45, "dmin_rule": "xds_range", "dmin_xds": 2.45, "dmax": 50.0}, "tags": ["h5", "lysozyme", "long-wavelength"]}, + {"id": "lyso_x06da_ice", "input": "lyso_x06da_ice/lysoC_14_56b5b9_master.h5", "wavelength": 0.95364, "ref": {"sgno": 89, "cell": [78.387, 78.387, 37.427, 90.0, 90.0, 90.0], "anomalous": true, "isa": 23.33, "completeness": 100.0, "r_meas": 0.219, "cc_half": 0.998, "multiplicity": 11.86, "dmin_low": 4.26, "r_meas_low": 0.055, "dmin": 1.431, "dmin_rule": "xds_range", "dmin_xds": 1.431, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, + {"id": "lyso_x06da_ref", "input": "lyso_x06da_ref/hewl_35_001_master.h5", "wavelength": 0.95329, "ref": {"sgno": 89, "cell": [78.091, 78.091, 37.704, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.31, "completeness": 100.0, "r_meas": 0.045, "cc_half": 1.0, "multiplicity": 13.76, "dmin_low": 3.58, "r_meas_low": 0.029, "dmin": 1.2, "dmin_rule": "xds_range", "dmin_xds": 1.2, "dmax": 50.0}, "tags": ["h5", "lysozyme"], "tiers": {"smoke": "lysozyme reference set"}}, + {"id": "lyso_x10sa_strong", "input": "lyso_x10sa_strong/lyso_5_f6ac7e_master.h5", "wavelength": 0.99988, "ref": {"sgno": 92, "cell": [77.769, 77.769, 37.442, 90.0, 90.0, 90.0], "anomalous": false, "isa": 8.63, "completeness": 65.8, "r_meas": 0.112, "cc_half": 0.998, "multiplicity": 11.46, "dmin_low": 3.52, "r_meas_low": 0.077, "dmin": 1.24, "dmin_rule": "cc_half_0.30", "dmin_xds": 1.18, "dmax": 999.0}, "tags": ["h5", "lysozyme"]}, + {"id": "lyso_x06da_half_image", "input": "lyso_x06da_half_image/lyso3_7_b81d83_master.h5", "wavelength": 0.95373, "ref": {"sgno": 96, "cell": [77.74, 77.74, 37.386, 90.0, 90.0, 90.0], "anomalous": true, "isa": 6.61, "completeness": 99.6, "r_meas": 0.595, "cc_half": 0.958, "multiplicity": 5.31, "dmin_low": 4.91, "r_meas_low": 0.258, "dmin": 1.65, "dmin_rule": "xds_range", "dmin_xds": 1.65, "dmax": 50.0}, "tags": ["h5", "lysozyme"]}, + {"id": "thau_x10sa_16keV", "input": "thau_x10sa_16keV/lyso_4_d2bb9b_master.h5", "wavelength": 0.77491, "ref": {"sgno": 92, "cell": [57.958, 57.958, 150.547, 90.0, 90.0, 90.0], "anomalous": false, "isa": 44.51, "completeness": 94.3, "r_meas": 0.072, "cc_half": 1.0, "multiplicity": 18.36, "dmin_low": 3.89, "r_meas_low": 0.027, "dmin": 1.3, "dmin_rule": "xds_range", "dmin_xds": 1.3, "dmax": 50.0}, "tags": ["h5", "thaumatin"]}, + {"id": "thau_x10sa_0p1deg", "input": "thau_x10sa_0p1deg/Thau_9_002_master.h5", "wavelength": 0.99989, "ref": {"sgno": 92, "cell": [58.488, 58.488, 150.788, 90.0, 90.0, 90.0], "anomalous": true, "isa": 9.15, "completeness": 99.0, "r_meas": 0.23, "cc_half": 0.997, "multiplicity": 11.37, "dmin_low": 6.51, "r_meas_low": 0.104, "dmin": 2.197, "dmin_rule": "xds_range", "dmin_xds": 2.197, "dmax": 50.0}, "tags": ["h5", "thaumatin"]}, + {"id": "thau_bl1a_6p5keV", "input": "thau_bl1a_6p5keV/JTH14_1p9_50ds_90T_k0_JF4M_01_50001_master.h5", "wavelength": 1.9, "ref": {"sgno": 89, "cell": [58.031, 58.031, 150.577, 90.0, 90.0, 90.0], "anomalous": true, "isa": 34.46, "completeness": 91.8, "r_meas": 0.072, "cc_half": 0.999, "multiplicity": 7.95, "dmin_low": 5.33, "r_meas_low": 0.041, "dmin": 1.78, "dmin_rule": "xds_range", "dmin_xds": 1.78, "dmax": 100.0}, "tags": ["h5", "thaumatin", "long-wavelength"]}, + {"id": "thau_bl1a_4p6keV", "input": "thau_bl1a_4p6keV/JTH14_2p7_50ds_19T_k0_JF4M_01_50001_master.h5", "wavelength": 2.7, "ref": {"sgno": 89, "cell": [58.007, 58.007, 150.511, 90.0, 90.0, 90.0], "anomalous": true, "isa": 35.64, "completeness": 92.0, "r_meas": 0.062, "cc_half": 0.998, "multiplicity": 7.94, "dmin_low": 7.56, "r_meas_low": 0.039, "dmin": 2.53, "dmin_rule": "xds_range", "dmin_xds": 2.53, "dmax": 100.0}, "tags": ["h5", "thaumatin", "long-wavelength"]}, + {"id": "thau_bl1a_3p8keV", "input": "thau_bl1a_3p8keV/JTH14_3p3_50ds_12T_k0_JF4M_01_50001_master.h5", "wavelength": 3.3, "ref": {"sgno": 89, "cell": [57.895, 57.895, 150.213, 90.0, 90.0, 90.0], "anomalous": true, "isa": 30.8, "completeness": 92.2, "r_meas": 0.057, "cc_half": 0.997, "multiplicity": 7.92, "dmin_low": 9.23, "r_meas_low": 0.039, "dmin": 3.1, "dmin_rule": "xds_range", "dmin_xds": 3.1, "dmax": 100.0}, "tags": ["h5", "thaumatin", "long-wavelength"], "tiers": {"smoke": "long wavelength (3.3 A), KEK BL-1A thaumatin"}}, + {"id": "thau_x10sa_injection", "input": "thau_x10sa_injection/lyso_13_0c7e7a_master.h5", "wavelength": 0.99988, "ref": {"sgno": 92, "cell": [57.922, 57.922, 150.678, 90.0, 90.0, 90.0], "anomalous": true, "isa": 36.23, "completeness": 86.2, "r_meas": 0.037, "cc_half": 1.0, "multiplicity": 9.96, "dmin_low": 3.83, "r_meas_low": 0.022, "dmin": 1.28, "dmin_rule": "xds_range", "dmin_xds": 1.28, "dmax": 50.0}, "tags": ["h5", "thaumatin"]}, + {"id": "thau_micromax_pink", "input": "thau_micromax_pink/Thau-Thaumatin_2_master.h5", "wavelength": 0.98419, "ref": {"sgno": 92, "cell": [58.784, 58.784, 152.226, 90.0, 90.0, 90.0], "anomalous": true, "isa": 21.18, "completeness": 98.5, "r_meas": 0.08, "cc_half": 0.999, "multiplicity": 9.38, "dmin_low": 4.17, "r_meas_low": 0.04, "dmin": 1.4, "dmin_rule": "xds_range", "dmin_xds": 1.4, "dmax": 50.0}, "tags": ["h5", "thaumatin", "pink-beam"], "tiers": {"smoke": "pink beam"}}, + {"id": "lyso_micromax_mono", "input": "lyso_micromax_mono/lysozyme-s6_1_master.h5", "wavelength": 0.95447, "tags": ["h5", "lysozyme"], "ref": {"sgno": 96, "cell": [77.668, 77.668, 37.486, 90.0, 90.0, 90.0], "anomalous": true, "isa": 39.93, "completeness": 100.0, "r_meas": 0.047, "cc_half": 1.0, "multiplicity": 6.22, "dmin_low": 4.54, "r_meas_low": 0.023, "dmin": 1.5, "dmin_rule": "xds_range", "dmin_xds": 1.5, "dmax": 50.0}}, + {"id": "lyso_micromax_pink", "input": "lyso_micromax_pink/lysozyme-s6_1_master.h5", "wavelength": 0.95374, "tags": ["h5", "lysozyme", "pink-beam"], "ref": {"sgno": 96, "cell": [77.654, 77.654, 37.483, 90.0, 90.0, 90.0], "anomalous": true, "isa": 37.35, "completeness": 99.9, "r_meas": 0.051, "cc_half": 1.0, "multiplicity": 5.96, "dmin_low": 4.39, "r_meas_low": 0.022, "dmin": 1.45, "dmin_rule": "xds_range", "dmin_xds": 1.45, "dmax": 50.0}}, + {"id": "lysoI_micromax_mono", "input": "lysoI_micromax_mono/lysozyme-Is2_1_master.h5", "wavelength": 0.95447, "tags": ["h5", "lysozyme", "iodine"], "ref": {"sgno": 96, "cell": [78.335, 78.335, 37.46, 90.0, 90.0, 90.0], "anomalous": true, "isa": 31.42, "completeness": 100.0, "r_meas": 0.068, "cc_half": 0.999, "multiplicity": 6.78, "dmin_low": 4.99, "r_meas_low": 0.028, "dmin": 1.65, "dmin_rule": "xds_range", "dmin_xds": 1.65, "dmax": 50.0}}, + {"id": "lysoI_micromax_pink", "input": "lysoI_micromax_pink/lysozyme-Is2_1_master.h5", "wavelength": 0.95374, "tags": ["h5", "lysozyme", "iodine", "pink-beam"], "ref": {"sgno": 96, "cell": [78.302, 78.302, 37.448, 90.0, 90.0, 90.0], "anomalous": true, "isa": 29.19, "completeness": 100.0, "r_meas": 0.079, "cc_half": 0.999, "multiplicity": 6.8, "dmin_low": 4.99, "r_meas_low": 0.028, "dmin": 1.65, "dmin_rule": "xds_range", "dmin_xds": 1.65, "dmax": 50.0}}, + {"id": "insu_H_x06da_twin", "input": "insu_H_x06da_twin/Ins_H_2_fa9841_master.h5", "wavelength": 0.95375, "ref": {"sgno": 146, "cell": [81.393, 81.393, 33.272, 90.0, 90.0, 120.0], "anomalous": true, "isa": 6.82, "completeness": 95.0, "r_meas": 0.118, "cc_half": 0.988, "multiplicity": 3.61, "dmin_low": 4.34, "r_meas_low": 0.106, "dmin": 1.455, "dmin_rule": "xds_range", "dmin_xds": 1.455, "dmax": 50.0}, "tags": ["h5", "insulin"]}, + {"id": "insu_H_x06da_notwin", "input": "insu_H_x06da_notwin/Ins_H_3_d05684_master.h5", "wavelength": 0.95375, "ref": {"sgno": 146, "cell": [81.378, 81.378, 33.269, 90.0, 90.0, 120.0], "anomalous": true, "isa": 17.69, "completeness": 97.4, "r_meas": 0.068, "cc_half": 0.998, "multiplicity": 3.52, "dmin_low": 4.61, "r_meas_low": 0.045, "dmin": 1.544, "dmin_rule": "xds_range", "dmin_xds": 1.544, "dmax": 50.0}, "tags": ["h5", "insulin"], "tiers": {"smoke": "rhombohedral insulin (H3)"}}, + {"id": "insu_I_x06da_weak", "input": "insu_I_x06da_weak/Ins_16_776a92_master.h5", "wavelength": 0.95373, "ref": {"sgno": 197, "cell": [77.085, 77.085, 77.085, 90.0, 90.0, 90.0], "anomalous": false, "isa": 18.91, "completeness": 92.4, "r_meas": 0.401, "cc_half": 0.999, "multiplicity": 29.04, "dmin_low": 3.22, "r_meas_low": 0.054, "dmin": 1.807, "dmin_rule": "cc_half_0.30", "dmin_xds": 1.08, "dmax": 999.0}, "tags": ["h5", "insulin"]}, + {"id": "insu_I_x06da_low_isa", "input": "insu_I_x06da_low_isa/Ins_I_2_1a530d_master.h5", "wavelength": 0.95375, "ref": {"sgno": 197, "cell": [77.582, 77.582, 77.582, 90.0, 90.0, 90.0], "anomalous": true, "isa": 4.15, "completeness": 90.8, "r_meas": 0.22, "cc_half": 0.998, "multiplicity": 13.33, "dmin_low": 3.92, "r_meas_low": 0.167, "dmin": 1.3, "dmin_rule": "xds_range", "dmin_xds": 1.3, "dmax": 50.0}, "tags": ["h5", "insulin"]}, + {"id": "insu_I_x06da_ref", "input": "insu_I_x06da_ref/Ins_I_3_5e90e3_master.h5", "wavelength": 0.95375, "ref": {"sgno": 197, "cell": [77.52, 77.52, 77.52, 90.0, 90.0, 90.0], "anomalous": true, "isa": 25.08, "completeness": 100.0, "r_meas": 0.349, "cc_half": 0.998, "multiplicity": 13.82, "dmin_low": 4.82, "r_meas_low": 0.143, "dmin": 1.621, "dmin_rule": "xds_range", "dmin_xds": 1.621, "dmax": 50.0}, "tags": ["h5", "insulin"]}, + {"id": "insu_I_x06da_13keV", "input": "insu_I_x06da_13keV/Ins_I_6_001_master.h5", "wavelength": 0.95375, "ref": {"sgno": 197, "cell": [77.468, 77.468, 77.468, 90.0, 90.0, 90.0], "anomalous": true, "isa": 18.83, "completeness": 100.0, "r_meas": 0.279, "cc_half": 0.998, "multiplicity": 17.21, "dmin_low": 4.85, "r_meas_low": 0.093, "dmin": 1.635, "dmin_rule": "xds_range", "dmin_xds": 1.635, "dmax": 50.0}, "tags": ["h5", "insulin"]}, + {"id": "insu_I_x06da_5keV", "input": "insu_I_x06da_5keV/Ins_I_6_006_master.h5", "wavelength": 2.47923, "ref": {"sgno": 197, "cell": [77.785, 77.785, 77.785, 90.0, 90.0, 90.0], "anomalous": true, "isa": 17.54, "completeness": 91.7, "r_meas": 0.073, "cc_half": 0.999, "multiplicity": 11.17, "dmin_low": 7.21, "r_meas_low": 0.048, "dmin": 2.45, "dmin_rule": "xds_range", "dmin_xds": 2.45, "dmax": 50.0}, "tags": ["h5", "insulin", "long-wavelength"]}, + {"id": "insu_I_x06da_6keV", "input": "insu_I_x06da_6keV/Ins_I_6_004_master.h5", "wavelength": 2.06648, "ref": {"sgno": 197, "cell": [77.71, 77.71, 77.71, 90.0, 90.0, 90.0], "anomalous": true, "isa": 17.95, "completeness": 92.4, "r_meas": 0.065, "cc_half": 0.999, "multiplicity": 12.42, "dmin_low": 6.03, "r_meas_low": 0.047, "dmin": 2.04, "dmin_rule": "xds_range", "dmin_xds": 2.04, "dmax": 50.0}, "tags": ["h5", "insulin", "long-wavelength"]}, + {"id": "insu_I_x06da_5keV_2", "input": "insu_I_x06da_5keV_2/Ins_I_7_001_master.h5", "wavelength": 2.47923, "ref": {"sgno": 197, "cell": [77.694, 77.694, 77.694, 90.0, 90.0, 90.0], "anomalous": true, "isa": 20.01, "completeness": 91.8, "r_meas": 0.076, "cc_half": 0.999, "multiplicity": 12.73, "dmin_low": 7.21, "r_meas_low": 0.048, "dmin": 2.45, "dmin_rule": "xds_range", "dmin_xds": 2.45, "dmax": 50.0}, "tags": ["h5", "insulin", "long-wavelength"], "tiers": {"smoke": "cubic insulin at 5 keV (long wavelength), fast"}}, + {"id": "cytc_x10sa", "input": "cytc_x10sa/cytC_10_002_master.h5", "wavelength": 0.99989, "ref": {"sgno": 152, "cell": [83.659, 83.659, 86.011, 90.0, 90.0, 120.0], "anomalous": true, "isa": 31.75, "completeness": 99.8, "r_meas": 0.232, "cc_half": 0.999, "multiplicity": 10.65, "dmin_low": 6.05, "r_meas_low": 0.036, "dmin": 2.267, "dmin_rule": "cc_half_0.30", "dmin_xds": 2.039, "dmax": 50.0}, "tags": ["h5", "cytochrome-c"]}, + {"id": "cytc_x06da_1", "input": "cytc_x06da_1/cytC_2_0770b5_master.h5", "wavelength": 0.95375, "ref": {"sgno": 152, "cell": [83.836, 83.836, 86.733, 90.0, 90.0, 120.0], "anomalous": true, "isa": 24.49, "completeness": 99.9, "r_meas": 0.084, "cc_half": 0.999, "multiplicity": 7.67, "dmin_low": 5.58, "r_meas_low": 0.033, "dmin": 1.875, "dmin_rule": "xds_range", "dmin_xds": 1.875, "dmax": 50.0}, "tags": ["h5", "cytochrome-c"]}, + {"id": "cytc_x06da_2", "input": "cytc_x06da_2/cytC_3_874285_master.h5", "wavelength": 0.95375, "ref": {"sgno": 152, "cell": [83.719, 83.719, 88.617, 90.0, 90.0, 120.0], "anomalous": true, "isa": 26.98, "completeness": 99.9, "r_meas": 0.097, "cc_half": 0.999, "multiplicity": 7.82, "dmin_low": 5.03, "r_meas_low": 0.032, "dmin": 1.69, "dmin_rule": "xds_range", "dmin_xds": 1.69, "dmax": 50.0}, "tags": ["h5", "cytochrome-c"]}, + {"id": "myob_x10sa", "input": "myob_x10sa/MyoB_13_110ee0_master.h5", "wavelength": 0.99988, "ref": {"sgno": 4, "cell": [35.229, 28.461, 63.076, 90.0, 106.504, 90.0], "anomalous": true, "isa": 5.21, "completeness": 97.7, "r_meas": 0.247, "cc_half": 0.965, "multiplicity": 3.29, "dmin_low": 5.16, "r_meas_low": 0.186, "dmin": 1.742, "dmin_rule": "xds_range", "dmin_xds": 1.742, "dmax": 50.0}, "tags": ["h5", "myoglobin"]}, + {"id": "myob_x06da", "input": "myob_x06da/MyoB_9_446b70_master.h5", "wavelength": 0.95365, "ref": {"sgno": 3, "cell": [35.182, 28.544, 62.83, 90.0, 106.138, 90.0], "anomalous": true, "isa": 7.59, "completeness": 99.4, "r_meas": 0.232, "cc_half": 0.987, "multiplicity": 3.47, "dmin_low": 4.23, "r_meas_low": 0.105, "dmin": 1.422, "dmin_rule": "xds_range", "dmin_xds": 1.422, "dmax": 50.0}, "tags": ["h5", "myoglobin"], "tiers": {"smoke": "monoclinic myoglobin, screw not arbitrable by XDS"}}, + {"id": "myob_x06da_powder_1", "input": "myob_x06da_powder_1/MyoB2-1_75979a_master.h5", "wavelength": 0.95373, "ref": {"sgno": 1, "cell": [28.705, 34.796, 64.373, 74.314, 89.879, 90.057], "anomalous": false, "isa": 5.54, "completeness": 74.2, "r_meas": 0.493, "cc_half": 0.966, "multiplicity": 2.87, "dmin_low": 4.44, "r_meas_low": 0.125, "dmin": 1.495, "dmin_rule": "xds_range", "dmin_xds": 1.495, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"], "tiers": {"smoke": "known hard: triclinic twin"}}, + {"id": "myob_x06da_powder_2", "input": "myob_x06da_powder_2/MyoB2-2_5ddd7d_master.h5", "wavelength": 0.95373, "ref": {"sgno": 3, "cell": [35.576, 28.811, 63.529, 90.0, 105.593, 90.0], "anomalous": false, "isa": 2.19, "completeness": 58.0, "r_meas": 1.106, "cc_half": 0.655, "multiplicity": 4.71, "dmin_low": 2.98, "r_meas_low": 0.706, "dmin": 0.99, "dmin_rule": "xds_range", "dmin_xds": 0.99, "dmax": 999.0}, "tags": ["h5", "myoglobin", "twin"]}, + {"id": "myob_x06da_split", "input": "myob_x06da_split/MyoB2-4_079740_master.h5", "wavelength": 0.95373, "ref": {"sgno": 3, "cell": [35.39, 28.783, 63.626, 90.0, 105.544, 90.0], "anomalous": false, "isa": 12.41, "completeness": 77.7, "r_meas": 0.577, "cc_half": 0.984, "multiplicity": 4.38, "dmin_low": 4.48, "r_meas_low": 0.091, "dmin": 1.506, "dmin_rule": "xds_range", "dmin_xds": 1.506, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, + {"id": "myob_x06da_sparse", "input": "myob_x06da_sparse/MyoB2-5_7cef9c_master.h5", "wavelength": 0.95373, "ref": {"sgno": 3, "cell": [35.388, 28.752, 64.083, 90.0, 106.352, 90.0], "anomalous": false, "isa": 5.46, "completeness": 73.7, "r_meas": 0.326, "cc_half": 0.972, "multiplicity": 5.21, "dmin_low": 5.92, "r_meas_low": 0.146, "dmin": 2.0, "dmin_rule": "xds_range", "dmin_xds": 2.0, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, + {"id": "nothing_2", "input": "nothing_2/test-28_3400ac_master.h5", "wavelength": 0.99988, "expect": "no_lattice", "tags": ["h5", "control"]}, + {"id": "nothing_1", "input": "nothing_1/test-28_9bf604_master.h5", "wavelength": 0.99988, "expect": "no_lattice", "tags": ["h5", "control"]}, + {"id": "aspirin_x10sa_20keV", "input": "aspirin_x10sa_20keV/aspirin_1_002_master.h5", "wavelength": 0.61993, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.262, 6.543, 11.257, 90.0, 95.916, 90.0], "anomalous": true, "isa": 27.97, "completeness": 84.0, "r_meas": 0.032, "cc_half": 0.999, "multiplicity": 3.14, "dmin_low": 2.01, "r_meas_low": 0.03, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "wavelength": 0.49594, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.55, "completeness": 86.6, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.03, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "wavelength": 0.61993, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.55, "completeness": 81.0, "r_meas": 0.04, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.043, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, + {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.39, "completeness": 91.6, "r_meas": 0.03, "cc_half": 0.999, "multiplicity": 5.64, "dmin_low": 2.0, "r_meas_low": 0.035, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, + {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"} ]} diff --git a/tools/battery/open.json b/tools/battery/open.json index 374da4e60..1a602d4eb 100644 --- a/tools/battery/open.json +++ b/tools/battery/open.json @@ -1,183 +1,183 @@ {"arm": "open", "sets": [ - {"id": "11if", "input": "11if/BopeA_18500_a_B2-PEF_11if/data/PSL-1503_13949_master.h5", "ref": {"sg": "P 43", "sgno": 78, "cell": [51.14, 51.14, 71.92, 90.0, 90.0, 90.0], "dmin": 1.51}, "tags": ["h5", "tetragonal"], "tiers": {"smoke": "Eiger NXmx from another facility, 4-fold screw"}}, - {"id": "36gk", "input": "36gk/CLS-0074_5-3_36GK/data/CLS-0074_5-3_master.h5", "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [120.58, 189.49, 199.69, 90.0, 90.0, 90.0], "dmin": 2.28}, "tags": ["h5", "orthorhombic"]}, - {"id": "3inp", "input": "3inp/IDP02542_3inp/data/idp02542b.001", "ref": {"sg": "F 41 3 2", "sgno": 210, "cell": [224.08, 224.08, 224.08, 90.0, 90.0, 90.0], "dmin": 2.05}, "tags": ["marCCD", "cubic"]}, - {"id": "3ky7", "input": "3ky7/IDP90258_3ky7/data/idp90258f.001", "ref": {"sg": "P 43 3 2", "sgno": 212, "cell": [125.176, 125.176, 125.176, 90.0, 90.0, 90.0], "dmin": 2.35}, "tags": ["marCCD", "cubic"]}, - {"id": "3mc4", "input": "3mc4/series/206918e4_x0001.img", "ref": {"sg": "H 3", "sgno": 146, "cell": [104.03, 104.03, 105.54, 90.0, 90.0, 120.0], "dmin": 1.95}, "tags": ["smv", "trigonal", "home-source"]}, - {"id": "3meb", "input": "3meb/series/202097g3_x0001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [58.58, 101.15, 81.53, 90.0, 90.62, 90.0], "dmin": 1.9}, "tags": ["smv", "monoclinic", "home-source"]}, - {"id": "3p85", "input": "3p85/series/217175d10_x0001.img", "ref": {"sg": "P 63 2 2", "sgno": 182, "cell": [127.29, 127.29, 72.9, 90.0, 90.0, 120.0], "dmin": 1.9}, "tags": ["smv", "hexagonal", "home-source"]}, - {"id": "3r6o", "input": "3r6o/series/219594b8_x0001.img", "ref": {"sg": "I 41", "sgno": 80, "cell": [90.68, 90.68, 76.13, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["smv", "tetragonal", "home-source"]}, - {"id": "5cc8", "input": "5cc8/data/263060g10_x0001.img", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [87.14, 93.76, 72.49, 90.0, 90.0, 90.0], "dmin": 1.75}, "tags": ["smv", "orthorhombic", "home-source", "tncs"]}, - {"id": "5ebi", "input": "5ebi/dna-rna-chimera_Ba_high_2_001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [35.72, 44.1, 35.72, 90.0, 119.98, 90.0], "dmin": 1.09}, "pinned": true, "tags": ["marCCD", "monoclinic", "twin"]}, - {"id": "5epe", "input": "5epe/030805_5epe/data/E1_7_set.001", "ref": {"sg": "F 2 3", "sgno": 196, "cell": [157.536, 157.536, 157.536, 90.0, 90.0, 90.0], "dmin": 1.9}, "tags": ["marCCD", "cubic"]}, - {"id": "5f6m", "input": "5f6m/crystal3_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [54.81, 58.51, 67.42, 90.0, 90.0, 90.0], "dmin": 1.1}, "tags": ["cbf", "orthorhombic", "diffuse"]}, - {"id": "5j23", "input": "5j23/012132_5j23/data/XZ2_set.001", "ref": {"sg": "H 3", "sgno": 146, "cell": [175.822, 175.822, 136.839, 90.0, 90.0, 120.0], "dmin": 2.3}, "tags": ["marCCD", "trigonal", "twin"]}, - {"id": "5jvn", "input": "5jvn/5jvn/data/PPDK-AV2820_w1_3_0001.cbf", "ref": {"sg": "P 6 2 2", "sgno": 177, "cell": [249.43, 249.43, 84.06, 90.0, 90.0, 120.0], "dmin": 2.9}, "tags": ["cbf", "hexagonal"]}, - {"id": "5lzl", "input": "5lzl/pcalad/alad-levoil6_M3S15_1_0001.cbf", "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [205.561, 205.561, 199.171, 90.0, 90.0, 120.0], "dmin": 3.47}, "tags": ["cbf", "trigonal"]}, - {"id": "5m17", "input": "5m17/BxGH99_WTDD_HA00AG2801_2_0001.cbf", "ref": {"sg": "I 4", "sgno": 79, "cell": [108.576, 108.576, 67.746, 90.0, 90.0, 90.0], "dmin": 1.03}, "tags": ["cbf", "tetragonal"]}, - {"id": "5nw5", "input": "5nw5/5NW5_1_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [92.14, 169.8, 390.16, 90.0, 90.0, 90.0], "dmin": 6.502}, "tags": ["cbf", "orthorhombic", "low-resolution"]}, - {"id": "5reo", "input": "5reo/cbf/Mpro-x0752_1_0001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [112.39, 52.59, 44.38, 90.0, 103.04, 90.0], "dmin": 1.88}, "tags": ["cbf", "monoclinic"], "tiers": {"smoke": "fastest set (10 s): miniCBF, monoclinic"}}, - {"id": "5src", "input": "5src/5src/data/FRS004_15_1_00001.cbf", "ref": {"sg": "P 43", "sgno": 78, "cell": [88.68, 88.68, 39.23, 90.0, 90.0, 90.0], "dmin": 1.05}, "tags": ["cbf", "tetragonal"]}, - {"id": "5uth", "input": "5uth/data/287007e1_0001.img", "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [69.28, 69.28, 153.8, 90.0, 90.0, 120.0], "dmin": 1.95}, "tags": ["smv", "trigonal", "home-source"]}, - {"id": "5vml", "input": "5vml/data/271705c8_x_0001.img", "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [66.31, 66.31, 115.26, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["smv", "tetragonal", "home-source"]}, - {"id": "6cee", "input": "6cee/data/Rachel_AV6_screen_0001.img", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [40.72, 44.11, 55.91, 90.0, 90.0, 90.0], "dmin": 1.55}, "tags": ["smv", "orthorhombic", "home-source"]}, - {"id": "6fid", "input": "6fid/Trypsin_x1_align_2_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [59.947, 64.107, 69.694, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["cbf", "orthorhombic"]}, - {"id": "6fvz", "input": "6fvz/maox215_6fvz/data/maox215_w1_1_0001.cbf", "ref": {"sg": "C 2 2 2", "sgno": 21, "cell": [131.182, 222.753, 86.482, 90.0, 90.0, 90.0], "dmin": 1.8}, "tags": ["cbf", "orthorhombic"]}, - {"id": "6fwc", "input": "6fwc/maox225_6fwc/data/maox225_1_00001.cbf", "ref": {"sg": "C 2 2 2", "sgno": 21, "cell": [131.728, 222.051, 86.293, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["cbf", "orthorhombic"]}, - {"id": "6h2p_native", "input": "6h2p/ProlMut1MnCAC_6h2p/data/Prol_Mut1_Mn_X1_nat_1_0002.cbf", "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [103.453, 107.075, 216.543, 90.0, 90.0, 90.0], "dmin": 1.479}, "tags": ["cbf", "orthorhombic", "two-wavelength"]}, - {"id": "6h2p_1p89A", "input": "6h2p/ProlMut1MnCAC_6h2p/data/Prol_Mut1_Mn_X_pk1_2_0001.cbf", "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [103.453, 107.075, 216.543, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "long-wavelength", "two-wavelength"]}, - {"id": "6h5t", "input": "6h5t/6h5t/data/SH3-1_x2_1_001.img", "ref": {"sg": "I 4 2 2", "sgno": 97, "cell": [86.792, 86.792, 141.768, 90.0, 90.0, 90.0], "dmin": 1.689}, "tags": ["marCCD", "tetragonal"]}, - {"id": "6hv2", "input": "6hv2/6hv2/data/coll_1_master.h5", "ref": {"sg": "P 61 2 2", "sgno": 178, "cell": [68.928, 68.928, 133.563, 90.0, 90.0, 120.0], "dmin": 1.709}, "tags": ["h5", "hexagonal"]}, - {"id": "6hwj", "input": "6hwj/bg3006_2_0001.cbf", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [59.812, 96.07, 80.252, 90.0, 106.69, 90.0], "dmin": 1.979}, "tags": ["cbf", "monoclinic"]}, - {"id": "6i3j", "input": "6i3j/6i3j/data/BF19_go_1_0001.img", "ref": {"sg": "F 2 2 2", "sgno": 22, "cell": [134.382, 203.846, 226.744, 90.0, 90.0, 90.0], "dmin": 2.59}, "tags": ["marCCD", "orthorhombic"]}, - {"id": "6iu5", "input": "6iu5/6iu5_VIT1_MBD_Zn/peak/peak_000001.cbf", "ref": {"sg": "P 31", "sgno": 144, "cell": [84.942, 84.942, 98.19, 90.0, 90.0, 120.0], "dmin": 2.25}, "pinned": true, "tags": ["cbf", "trigonal", "twin"]}, - {"id": "6iu6", "input": "6iu6/6iu6_VIT1_MBD_Ni/peak/peak_000001.cbf", "ref": {"sg": "P 31", "sgno": 144, "cell": [84.744, 84.744, 97.398, 90.0, 90.0, 120.0], "dmin": 2.9}, "pinned": true, "tags": ["cbf", "trigonal", "twin"]}, - {"id": "6iu8", "input": "6iu8/6iu8_VIT1_MBD_Co/low/low_000001.cbf", "ref": {"sg": "P 31", "sgno": 144, "cell": [85.503, 85.503, 98.355, 90.0, 90.0, 120.0], "dmin": 2.7}, "tags": ["cbf", "trigonal"]}, - {"id": "6iu9", "input": "6iu9/6iu9_VIT1_MBD_Fe/peak/data01_000001.cbf", "ref": {"sg": "P 31", "sgno": 144, "cell": [85.321, 85.321, 97.573, 90.0, 90.0, 120.0], "dmin": 3.0}, "pinned": true, "tags": ["cbf", "trigonal", "twin"]}, - {"id": "6jgi", "input": "6jgi/6jgi/data/00001.img", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [50.857, 62.403, 69.172, 90.0, 90.0, 90.0], "dmin": 0.85}, "tags": ["marCCD", "orthorhombic"]}, - {"id": "6jgj", "input": "6jgj/6jgj/data/000001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [50.87, 62.34, 68.85, 90.0, 90.0, 90.0], "dmin": 0.77}, "tags": ["cbf", "orthorhombic", "cdte"]}, - {"id": "6moj", "input": "6moj/A9_1_00001.cbf", "ref": {"sg": "I 41 2 2", "sgno": 98, "cell": [130.418, 130.418, 293.453, 90.0, 90.0, 90.0], "dmin": 2.431}, "tags": ["cbf", "tetragonal"]}, - {"id": "6o2h", "input": "6o2h/lys_nitr_10_1_0001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [27.424, 32.134, 34.513, 88.657, 108.46, 111.877], "dmin": 1.212}, "tags": ["cbf", "triclinic", "diffuse"]}, - {"id": "6oel", "input": "6oel/1449-8_3_001.img", "ref": {"sg": "F 41 3 2", "sgno": 210, "cell": [328.1, 328.1, 328.1, 90.0, 90.0, 90.0], "dmin": 3.1}, "tags": ["SMV", "cubic"], "tiers": {"smoke": "SMV reader (ADSC), cubic F4132"}}, - {"id": "6p8p", "input": "6p8p/14_12_1_0001.cbf", "ref": {"sg": "P 4", "sgno": 75, "cell": [97.525, 97.525, 60.134, 90.0, 90.0, 90.0], "dmin": 1.635}, "tags": ["cbf", "tetragonal"]}, - {"id": "6pb3", "input": "6pb3/14_16_1_000001.cbf", "ref": {"sg": "P 6", "sgno": 168, "cell": [100.436, 100.436, 48.86, 90.0, 90.0, 120.0], "dmin": 2.048}, "tags": ["cbf", "hexagonal"]}, - {"id": "6pxb", "input": "6pxb/TJB1_3_rachel_1_0001.cbf", "pinned": true, "ref": {"sg": "P 32", "sgno": 145, "cell": [64.021, 64.021, 119.447, 90.0, 90.0, 120.0], "dmin": 1.747}, "ref_alternatives": [{"sg": "P 32 1 2", "sgno": 153, "why": "The deposition merged in point group 3 (its 55502 unique reflections to 1.747 A are what Laue class -3 holds, twice what -3 1 m would) and refined six chains in P 32. The intensities read point group 312 instead: the three added two-folds correlate at 0.98-0.99, at or above the three-folds nobody disputes (0.98), against 0.37-0.40 for the 321 and 622 operators, POINTLESS on our P1 merge picks P -3 1 m (likelihood 1.000), and the reflections centric in 312 but not in 3 are distributed as centric (+435 nats), which a twin law cannot produce. Against that, the deposited chains pair under the added two-fold at 0.3-0.7 A, more than coordinate error at 1.75 A, and the model tells the two indexings apart (R 0.22 against 0.25), so the two-fold may be a near-exact non-crystallographic one; ZANUDA settles on P 32 2 1, whose operators these data do not support. Neither answer is established, so both are accepted; P 31 1 2, which the data cannot separate from P 32 1 2, is accepted as its hand"}], "tags": ["cbf", "trigonal"]}, - {"id": "6pxc", "input": "6pxc/TJB6_1_Rachel_1_0001.cbf", "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [44.19, 64.827, 87.239, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["cbf", "orthorhombic"]}, - {"id": "6r72", "input": "6r72/V-CK63-8-ld_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [117.805, 110.754, 155.615, 90.0, 93.23, 90.0], "dmin": 3.95}, "pinned": true, "tags": ["h5", "monoclinic"]}, - {"id": "6rlr", "input": "6rlr/_b4_1_master.h5", "ref": {"sg": "P 1", "sgno": 1, "cell": [39.986, 39.998, 63.643, 80.39, 76.29, 68.15], "dmin": 2.0}, "tags": ["h5", "triclinic", "twin"]}, - {"id": "6toc", "input": "6toc/zenodo/dts_1_00001.cbf", "ref": {"sg": "P 42", "sgno": 77, "cell": [31.523, 31.523, 81.599, 90.0, 90.0, 90.0], "dmin": 1.853}, "ref_alternatives": [{"sg": "P 42 2 2", "sgno": 93, "why": "Our reduction and an independent POINTLESS run on our own P1 merge both read point group 422, and the deposited asymmetric unit's two chains are related by the very two-fold the higher group adds, to 0.16 A CA RMSD over 43 residues - coordinate error at 1.85 A. Merging in P 42 2 2 costs 0.0006 in R_meas for 1.75x the multiplicity and correlates better with the deposited model (0.9802 vs 0.9764). The refinement test is NOT unanimous: ZANUDA 1.097 refines P 42 2 2 to R-free 0.2561 against P 42's 0.2636 at half the parameters and reports the deposited assignment incorrect, while an independent Refmac 5.8.0431 comparison on a symmetry-consistent free set puts P 42 ahead by 0.004-0.020 depending on cycle count - less than the spread between refinement protocols. Neither answer is established: a pseudo-symmetry too exact for any test we have remains a live explanation, and so does the deposited assignment. Either is accepted"}], "tags": ["cbf", "tetragonal", "twin"]}, - {"id": "6ttn", "input": "6ttn/6ttn/data/H6H_33_01_hyo_full_1_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [39.894, 79.841, 104.701, 90.0, 90.0, 90.0], "dmin": 1.12}, "tags": ["cbf", "orthorhombic"]}, - {"id": "6u7g", "input": "6u7g/6u7g/data/SNB02_11_8_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [99.555, 98.682, 147.525, 90.0, 104.6, 90.0], "dmin": 2.35}, "pinned": true, "tags": ["h5", "monoclinic"]}, - {"id": "6ukf", "input": "6ukf/6ukf/data/XDC-7_Pn6_000001.cbf", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [61.018, 37.317, 69.027, 90.0, 109.768, 90.0], "dmin": 1.0}, "tags": ["cbf", "monoclinic"]}, - {"id": "6v2r", "input": "6v2r/data/cbx7.248483.hv6_screen_0001.img", "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [40.193, 40.193, 83.127, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["smv", "tetragonal", "home-source"]}, - {"id": "6vww", "input": "6vww/IDP51000_6vww/data/m4H11g_00001.cbf", "ref": {"sg": "P 63", "sgno": 173, "cell": [150.539, 150.539, 111.31, 90.0, 90.0, 120.0], "dmin": 2.2}, "tags": ["cbf", "hexagonal", "twin"], "tiers": {"smoke": "merohedral twin, P63"}}, - {"id": "6w4h", "input": "6w4h/IDP51000_6W4H/data/idp51000-201-a_1_2_3.001", "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [167.74, 167.74, 51.942, 90.0, 90.0, 120.0], "dmin": 1.8}, "tags": ["marCCD", "trigonal"]}, - {"id": "6wzo", "input": "6wzo/9_1_1_000001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [43.718, 50.061, 69.337, 106.499, 90.094, 97.145], "dmin": 1.42}, "tags": ["cbf", "triclinic"]}, - {"id": "6yqf", "input": "6yqf/6yqf/data/SYCE2TEX12-8_1_0001.cbf", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [42.67, 59.68, 156.49, 90.0, 90.0, 90.0], "dmin": 3.327}, "tags": ["cbf", "orthorhombic"]}, - {"id": "6z8o", "input": "6z8o/HASE01-gaKr1_w1_1_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [63.716, 97.022, 121.318, 90.0, 104.656, 90.0], "dmin": 2.2}, "tags": ["h5", "monoclinic"]}, - {"id": "6ze4", "input": "6ze4/806/o8_1_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [93.56, 109.88, 116.11, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["cbf", "orthorhombic"], "tiers": {"smoke": "known hard: P212121 under-called as P21 since rc168"}}, - {"id": "7arr", "input": "7arr/PF4N_04D_w1_1_00001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [30.937, 32.068, 43.087, 114.16, 91.88, 109.86], "dmin": 1.1}, "tags": ["cbf", "triclinic"]}, - {"id": "7atg", "input": "7atg/7atg/data/put-2-2.0s_5_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [17.97, 31.02, 43.86, 90.0, 90.0, 90.0], "dmin": 0.6}, "tags": ["cbf", "orthorhombic"]}, - {"id": "7bgt", "input": "7bgt/MPMV2243_3_001.img", "ref": {"sg": "P 1", "sgno": 1, "cell": [29.296, 67.618, 69.716, 76.845, 83.875, 83.645], "dmin": 1.93}, "tags": ["marCCD", "triclinic"]}, - {"id": "7brr", "input": "7brr/7brr/data/3C_2_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [55.45, 99.02, 59.583, 90.0, 108.54, 90.0], "dmin": 1.35}, "tags": ["h5", "monoclinic"]}, - {"id": "7dkp", "input": "7dkp/7dkp/data/LOX1-sree-p1a9-1p2_w1_1_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [49.84, 169.451, 49.842, 90.0, 93.51, 90.0], "dmin": 1.45}, "tags": ["h5", "monoclinic"]}, - {"id": "7k1l", "input": "7k1l/IDP52015_7k1l/data/cs1C2eg_00001.cbf", "ref": {"sg": "P 63", "sgno": 173, "cell": [150.83, 150.83, 110.73, 90.0, 90.0, 120.0], "dmin": 2.25}, "tags": ["cbf", "hexagonal"], "tiers": {"smoke": "known hard: screw axis (P6 vs P63)"}}, - {"id": "7kcn", "input": "7kcn/7kcn/data/ttr_hiuase_1_D1_33_00001.cbf", "ref": {"sg": "P 41 2 2", "sgno": 91, "cell": [67.03, 67.03, 116.89, 90.0, 90.0, 90.0], "dmin": 1.46}, "tags": ["cbf", "tetragonal"]}, - {"id": "7l6j", "input": "7l6j/7l6j/data/idp97824-102sm-a_1_1_7.001", "ref": {"sg": "I 41 3 2", "sgno": 214, "cell": [171.693, 171.693, 171.693, 90.0, 90.0, 90.0], "dmin": 1.78}, "tags": ["marCCD", "cubic"]}, - {"id": "7l84", "input": "7l84/301_helical_1_0001.cbf", "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [79.344, 79.344, 37.81, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["cbf", "tetragonal"]}, - {"id": "7mzt", "input": "7mzt/7mzt/data/BVG2_Pn3_000001.cbf", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [113.619, 96.989, 108.33, 90.0, 90.0, 90.0], "dmin": 4.07}, "tags": ["cbf", "orthorhombic"]}, - {"id": "7n0i", "input": "7n0i/8_2_1_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [75.846, 131.557, 140.048, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["cbf", "orthorhombic", "tncs"]}, - {"id": "7n2s", "input": "7n2s/1449-4_1_00001.cbf", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [83.211, 52.789, 106.291, 90.0, 98.278, 90.0], "dmin": 2.37}, "tags": ["cbf", "monoclinic"]}, - {"id": "7orr", "input": "7orr/7orr/data/Nsp10-VT00022_1_master.h5", "ref": {"sg": "I 21 3", "sgno": 199, "cell": [105.88, 105.88, 105.88, 90.0, 90.0, 90.0], "dmin": 1.79}, "tags": ["h5", "cubic"]}, - {"id": "7os3", "input": "7os3/pos2_1/pos2_1_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [78.15, 91.05, 105.84, 90.0, 90.0, 90.0], "dmin": 2.177}, "tags": ["cbf", "orthorhombic"]}, - {"id": "7ph1", "input": "7ph1/7ph1/data/9_1_0001.cbf", "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [74.975, 81.289, 124.248, 90.0, 90.0, 90.0], "dmin": 1.18}, "tags": ["cbf", "orthorhombic"]}, - {"id": "7pq7", "input": "7pq7/7pq7/data/1436_p52_A3_1_1_00001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [120.88, 51.73, 75.54, 90.0, 125.14, 90.0], "dmin": 1.55}, "tags": ["cbf", "monoclinic"]}, - {"id": "7qij", "input": "7qij/907/dg046_2_data_1_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [143.46, 324.92, 369.38, 90.0, 90.0, 90.0], "dmin": 4.1}, "tags": ["cbf", "orthorhombic"]}, - {"id": "7qis", "input": "7qis/7qis/data/Chymo_Dfe_x1_2_0001.cbf", "ref": {"sg": "P 61", "sgno": 169, "cell": [100.269, 100.269, 206.215, 90.0, 90.0, 120.0], "dmin": 1.83}, "tags": ["cbf", "hexagonal"]}, - {"id": "7ris", "input": "7ris/7ris/data/GLVase_Ca_we21108b7b_1_2_2.001_master.h5", "ref": {"sg": "P 32 2 1", "sgno": 154, "cell": [44.54, 44.54, 189.95, 90.0, 90.0, 120.0], "dmin": 1.72}, "pinned": true, "tags": ["h5", "trigonal"]}, - {"id": "7rji", "input": "7rji/7rji/data/DD3_00000.cbf", "ref": {"sg": "H 3 2", "sgno": 155, "cell": [83.036, 83.036, 124.83, 90.0, 90.0, 120.0], "dmin": 1.71}, "tags": ["cbf", "trigonal"]}, - {"id": "7t5t", "input": "7t5t/A1_2_00001.cbf", "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [95.309, 95.309, 104.915, 90.0, 90.0, 90.0], "dmin": 1.35}, "pinned": true, "tags": ["cbf", "tetragonal"]}, - {"id": "7tcd", "input": "7tcd/7tcd/data/col_1_master.h5", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [138.504, 47.902, 78.068, 90.0, 107.56, 90.0], "dmin": 1.7}, "tags": ["h5", "monoclinic"]}, - {"id": "7yzx", "input": "7yzx/7yzx/data/783-1_1_0001.cbf", "ref": {"sg": "P 63 2 2", "sgno": 182, "cell": [169.42, 169.42, 141.83, 90.0, 90.0, 120.0], "dmin": 1.9}, "tags": ["cbf", "hexagonal"]}, - {"id": "8a1a", "input": "8a1a/L1F11v1_8a1a/data/L1-F11-v1_9ly1_D06_01_2_master.h5", "ref": {"sg": "P 65", "sgno": 170, "cell": [191.866, 191.866, 122.409, 90.0, 90.0, 120.0], "dmin": 2.05}, "tags": ["h5", "hexagonal"]}, - {"id": "8agq", "input": "8agq/8agq_zip_8AGQ/data/coll_1_00001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [89.948, 55.447, 54.787, 90.0, 113.55, 90.0], "dmin": 1.093}, "tags": ["cbf", "monoclinic"]}, - {"id": "8dqb", "input": "8dqb/KloxA_17380_a_B1-Apo_I23_Form_8dqb/data/PSL-1405_653_master.h5", "ref": {"sg": "I 2 3", "sgno": 197, "cell": [164.124, 164.124, 164.124, 90.0, 90.0, 90.0], "dmin": 2.5}, "tags": ["h5", "cubic"]}, - {"id": "8dyz", "input": "8dyz/lys_1_2_0001.cbf", "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [79.632, 79.632, 38.296, 90.0, 90.0, 90.0], "dmin": 1.272}, "tags": ["cbf", "tetragonal", "diffuse"]}, - {"id": "8dz7", "input": "8dz7/lys_rt_2_38_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [30.486, 56.401, 73.852, 90.0, 90.0, 90.0], "dmin": 1.34}, "tags": ["cbf", "orthorhombic", "diffuse"]}, - {"id": "8egn", "input": "8egn/PsaeA_00137_b_B5-az13643701_8egn/data/FKC8_4_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [71.69, 75.16, 109.79, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["cbf", "orthorhombic"]}, - {"id": "8iya", "input": "8iya/8iya/data/11_1_master.h5", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [102.747, 50.077, 109.248, 90.0, 91.754, 90.0], "dmin": 2.43}, "tags": ["h5", "monoclinic"]}, - {"id": "8k1g", "input": "8k1g/8k1g/data/KP02_0001.cbf", "ref": {"sg": "I 4 2 2", "sgno": 97, "cell": [182.01, 182.01, 80.71, 90.0, 90.0, 90.0], "dmin": 2.09}, "tags": ["cbf", "tetragonal"]}, - {"id": "8oic", "input": "8oic/8oic/data/_b20_1_master.h5", "ref": {"sg": "P 1", "sgno": 1, "cell": [73.055, 94.665, 120.589, 105.081, 89.959, 93.826], "dmin": 2.8}, "tags": ["h5", "triclinic"]}, - {"id": "8owm", "input": "8owm/p13x4_1_00001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [95.544, 95.629, 95.841, 90.416, 93.59, 117.775], "dmin": 1.7}, "tags": ["cbf", "triclinic"]}, - {"id": "8pqd", "input": "8pqd/8pqd/data/11AT05_w1_1_1_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [59.39, 59.37, 192.89, 90.0, 90.0, 90.0], "dmin": 1.5}, "tags": ["h5", "orthorhombic"]}, - {"id": "8qaw", "input": "8qaw/p18x3/p18x3_1_00001.cbf.gz", "ref": {"sg": "H 3", "sgno": 146, "cell": [137.68, 137.68, 265.907, 90.0, 90.0, 120.0], "dmin": 1.55}, "tags": ["cbf", "trigonal", "long-axis"]}, - {"id": "8qj5", "input": "8qj5/8qj5/data/Lcsbc3_1_2_00001.cbf", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [57.61, 100.63, 77.93, 90.0, 96.06, 90.0], "dmin": 1.63}, "tags": ["cbf", "monoclinic"]}, - {"id": "8qq7", "input": "8qq7/cbf/SpNox-Sp49-B3-B_088_1_0001.cbf", "ref": {"sg": "P 64 2 2", "sgno": 181, "cell": [145.967, 145.967, 153.619, 90.0, 90.0, 120.0], "dmin": 3.62}, "tags": ["cbf", "hexagonal"]}, - {"id": "8r5r", "input": "8r5r/8r5r/data/D5-33805_1_1_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [91.74, 132.91, 137.5, 90.0, 90.0, 90.0], "dmin": 3.08}, "tags": ["h5", "orthorhombic"]}, - {"id": "8rud", "input": "8rud/p08x03_1_00001.cbf", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [78.066, 91.386, 114.547, 90.0, 96.9, 90.0], "dmin": 2.1}, "tags": ["cbf", "monoclinic"]}, - {"id": "8s38", "input": "8s38/MX733_2_00001.cbf", "ref": {"sg": "I 21 21 21", "sgno": 24, "cell": [95.368, 163.1, 219.023, 90.0, 90.0, 90.0], "dmin": 1.89}, "tags": ["cbf", "orthorhombic"]}, - {"id": "8sa8", "input": "8sa8/KlaeA_00906_a_B1-PLP_8sa8/data/PSL-0412_255_master.h5", "ref": {"sg": "I 1 2 1", "sgno": 5, "cell": [87.887, 131.544, 165.363, 90.0, 104.48, 90.0], "dmin": 1.3}, "tags": ["h5", "monoclinic"]}, - {"id": "8sqo", "input": "8sqo/BrabA_00028_a_A1-Mg_8sqo/data/PSL-1801_942_master.h5", "ref": {"sg": "P 4 3 2", "sgno": 207, "cell": [112.91, 112.91, 112.91, 90.0, 90.0, 90.0], "dmin": 1.55}, "tags": ["h5", "cubic"]}, - {"id": "8sqq", "input": "8sqq/BrabA_00028_a_A1-Apo_8sqq/data/PSL-1810_965_master.h5", "ref": {"sg": "F 4 3 2", "sgno": 209, "cell": [171.52, 171.52, 171.52, 90.0, 90.0, 90.0], "dmin": 2.25}, "tags": ["h5", "cubic"]}, - {"id": "8sqt", "input": "8sqt/BrabA_00028_a_A1-Fe_8sqt/data/PSL-1812_971_master.h5", "ref": {"sg": "F 4 3 2", "sgno": 209, "cell": [170.72, 170.72, 170.72, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["h5", "cubic"]}, - {"id": "8t7r", "input": "8t7r/8t7r/data/GREEN-04_Pn5_000001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [357.076, 259.582, 255.361, 90.0, 133.13, 90.0], "dmin": 3.84}, "tags": ["cbf", "monoclinic"]}, - {"id": "8tha", "input": "8tha/1TEL_double_helix_8tha/data/A3_1_00001.cbf", "ref": {"sg": "P 64", "sgno": 172, "cell": [69.16, 69.16, 29.13, 90.0, 90.0, 120.0], "dmin": 1.68}, "tags": ["cbf", "hexagonal"]}, - {"id": "8tyy", "input": "8tyy/CPS_2_1_000001.cbf", "ref": {"sg": "F 4 3 2", "sgno": 209, "cell": [214.891, 214.891, 214.891, 90.0, 90.0, 90.0], "dmin": 1.68}, "tags": ["cbf", "cubic"]}, - {"id": "8u0i", "input": "8u0i/8u0i/data/dI-2-05a_4_00001.cbf", "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [50.317, 50.317, 90.596, 90.0, 90.0, 90.0], "dmin": 1.541}, "tags": ["cbf", "tetragonal"]}, - {"id": "8v4o", "input": "8v4o/CaalA_00629_a_FS11-AMP_8v4o/data/PSL-1602_2180_master.h5", "ref": {"sg": "P 61 2 2", "sgno": 178, "cell": [139.46, 139.46, 544.99, 90.0, 90.0, 120.0], "dmin": 2.7}, "tags": ["h5", "hexagonal"]}, - {"id": "8xbp", "input": "8xbp/8xbp/data/puck2872_3_1_master.h5", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [148.29, 50.78, 60.21, 90.0, 92.33, 90.0], "dmin": 1.99}, "ref_override": {"cell": [148.736, 51.777, 60.181, 90.0, 92.36, 90.0]}, "ref_override_why": "the uploaded sweep is a different crystal from the one the deposited cell describes: master data_collection_date 2023-06-21, deposited _diffrn_detector.pdbx_collection_date 2023-06-23, and the deposited b = 50.78 is 2.0% from the b these images give. The override is the cell DIALS 3.29 indexes de novo on this master, b = 51.777(11); the deposited space group and d_min are kept", "tags": ["h5", "monoclinic"]}, - {"id": "8xte", "input": "8xte/1101/G2_1_00001.cbf", "ref": {"sg": "P 32", "sgno": 145, "cell": [208.77, 208.77, 67.22, 90.0, 90.0, 120.0], "dmin": 1.99}, "ref_alternatives": [{"sg": "P 31 2 1", "sgno": 152, "why": "The evidence favours the higher group here. The twin-immune centric zone of the added two-folds - reflections the higher group makes centric, which are their own twin mates and so cannot be made to read centric by a merohedral twin law - gives <|E^2-1|> = 0.946 +/- 0.012 against a centric expectation of 0.968 and an acentric 0.736, at +707 nats. Re-refinement on a shared free set with the twin law removed from both sides favours P 3_2 2 1 (0.2177/0.2322) over P 3_2 (0.2460/0.2612), and the deposited entry's published R values reproduce only with an undeclared twin law h,-h-k,-l at alpha = 0.50, which is itself a 321-symmetric target. No refinement R can close the question in principle, because a merohedral twin at exactly alpha = 0.5 and true 321 predict identical intensities; the case rests on the centric zone. Both answers are accepted, ours being the better supported"}], "tags": ["cbf", "trigonal"]}, - {"id": "8xtf", "input": "8xtf/1102/puck03_12_1_master.h5", "ref": {"sg": "H 3 2", "sgno": 155, "cell": [211.75, 211.75, 67.42, 90.0, 90.0, 120.0], "dmin": 2.13}, "tags": ["h5", "trigonal"]}, - {"id": "8xtg", "input": "8xtg/1100/mpag15_1_00001.cbf", "ref": {"sg": "P 32", "sgno": 145, "cell": [199.54, 199.54, 67.15, 90.0, 90.0, 120.0], "dmin": 2.0}, "ref_alternatives": [{"sg": "P 31 2 1", "sgno": 152, "why": "The evidence favours the DEPOSITION here, and this row must not be read as the 8xte one. Every correlation-based instrument we have - our own operator correlations, POINTLESS (0.85 on our P1 merge) - reads point group 321, but our own twin-immune centric-zone test, the only one that separates real symmetry from pseudo-symmetry, reads <|E^2-1|> = 0.869 at -44.9 nats, between the two expectations and on the wrong side, i.e. AGAINST the promotion, with an L-test twin fraction of 0.20-0.26. Whether this crystal is partially twinned or purely pseudo-symmetric is not established. Both answers are accepted, the deposition being the better supported"}], "tags": ["cbf", "trigonal"]}, - {"id": "8y74", "input": "8y74/1_1/1_1_master.h5", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [125.782, 76.553, 87.143, 90.0, 92.449, 90.0], "dmin": 1.9}, "tags": ["h5", "monoclinic"]}, - {"id": "8ys9", "input": "8ys9/8YS9_xray-data_8ys9/data/SDS_09_18112_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [71.04, 77.68, 83.16, 90.0, 90.0, 90.0], "dmin": 1.46}, "tags": ["h5", "orthorhombic"]}, - {"id": "9b22", "input": "9b22/KlpnC_20447_a_B1-AR6-AMP_9b22/data/PSL-1011_1580_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [39.826, 92.659, 57.716, 90.0, 91.68, 90.0], "dmin": 1.3}, "tags": ["h5", "monoclinic"]}, - {"id": "9bn8", "input": "9bn8/EscoA_17938_a_AE1-UMA-A1AQS_9bn8/data/PSL-0401_8296_master.h5", "ref": {"sg": "P 41", "sgno": 76, "cell": [65.488, 65.488, 134.764, 90.0, 90.0, 90.0], "dmin": 1.35}, "tags": ["h5", "tetragonal"]}, - {"id": "9c18", "input": "9c18/BLVRB-2_5217_master.h5", "ref": {"sg": "P 1", "sgno": 1, "cell": [41.94, 41.977, 60.212, 84.11, 87.24, 63.66], "dmin": 1.9}, "tags": ["h5", "triclinic"]}, - {"id": "9chw", "input": "9chw/D1.001", "ref": {"sg": "P 61", "sgno": 169, "cell": [98.686, 98.686, 82.062, 90.0, 90.0, 120.0], "dmin": 2.16}, "tags": ["marCCD", "hexagonal"]}, - {"id": "9crw", "input": "9crw/data_9crw/data/Kip3-475-22-2_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [83.96, 104.57, 118.78, 90.0, 93.37, 90.0], "dmin": 2.49}, "tags": ["h5", "monoclinic"]}, - {"id": "9e2t", "input": "9e2t/ga026f4b-1_5_00001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [75.499, 78.082, 101.219, 94.63, 103.39, 114.55], "dmin": 2.28}, "tags": ["cbf", "triclinic"]}, - {"id": "9ea5", "input": "9ea5/K4_1_00001.cbf", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [65.868, 73.064, 98.399, 90.0, 108.725, 90.0], "dmin": 2.0}, "tags": ["cbf", "monoclinic"]}, - {"id": "9fcg", "input": "9fcg/IBCH-15-p02x06_2_00001.cbf.gz", "ref": {"sg": "P 4", "sgno": 75, "cell": [87.79, 87.79, 35.633, 90.0, 90.0, 90.0], "dmin": 1.54}, "tags": ["cbf", "tetragonal"], "tiers": {"smoke": "cbf.gz reader, pure 4-fold"}}, - {"id": "9fhc", "input": "9fhc/te/te312_2_001.img", "ref": {"sg": "I 2 3", "sgno": 197, "cell": [227.46, 227.46, 227.46, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["marCCD", "cubic"]}, - {"id": "9gdj", "input": "9gdj/slh34_w1_1_1_master.h5", "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [123.92, 123.92, 126.44, 90.0, 90.0, 90.0], "dmin": 1.47}, "tags": ["h5", "tetragonal", "cdte"]}, - {"id": "9gjx", "input": "9gjx/BlNTR_9gjx/data/SC6_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [76.776, 115.807, 103.807, 90.0, 110.29, 90.0], "dmin": 2.4}, "tags": ["h5", "monoclinic"]}, - {"id": "9gqg", "input": "9gqg/MX-2555/FKBP51-MSP500-A-5522-11_1_3094759/FKBP51-MSP500-A-5522-11_1_1_master.h5", "ref": {"sg": "P 32 2 1", "sgno": 154, "cell": [48.157, 48.157, 188.036, 90.0, 90.0, 120.0], "dmin": 2.0}, "tags": ["h5", "trigonal"]}, - {"id": "9hnc", "input": "9hnc/asp_7_00001.cbf", "ref": {"sg": "P 1 2 1", "sgno": 3, "cell": [123.764, 123.645, 187.679, 90.0, 90.063, 90.0], "dmin": 1.879}, "tags": ["cbf", "monoclinic"]}, - {"id": "9hs7", "input": "9hs7/9hs7/data/B28X1_1_0001.cbf", "ref": {"sg": "P 65", "sgno": 170, "cell": [65.44, 65.44, 88.77, 90.0, 90.0, 120.0], "dmin": 1.698}, "tags": ["cbf", "hexagonal"]}, - {"id": "9i0a", "input": "9i0a/carm1END468_9i0a_tar_9I0A/data/X151B1_1_master.h5", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [75.237, 98.676, 208.551, 90.0, 90.0, 90.0], "dmin": 2.22}, "tags": ["h5", "orthorhombic"]}, - {"id": "9i80", "input": "9i80/20230422-PX1-LecA_RO5-5315-1/LecA-RO5_1_master.h5", "ref": {"sg": "P 41", "sgno": 76, "cell": [81.214, 81.214, 165.029, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["h5", "tetragonal", "twin"]}, - {"id": "9ig7", "input": "9ig7/9IG7/data/KOD-P596-G2_3_00001.cbf", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [111.472, 153.471, 69.03, 90.0, 90.0, 90.0], "dmin": 2.6}, "tags": ["cbf", "orthorhombic"]}, - {"id": "9ih9", "input": "9ih9/run_01_05_datacollection_9IH9/data/design-Xinjian-P1Thro-4_1_5_1_master.h5", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [78.758, 133.933, 82.32, 90.0, 101.356, 90.0], "dmin": 1.7}, "tags": ["h5", "monoclinic"]}, - {"id": "9jq9", "input": "9jq9/pfplrxdata_9JQ9/data/lp13_001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [48.551, 50.499, 78.571, 90.0, 90.0, 90.0], "dmin": 1.9}, "tags": ["cbf", "orthorhombic", "home-source"]}, - {"id": "9jzo", "input": "9jzo/JS_35P2_C3-002_diffraction_files_9JZO/data/JS_35P2_C3-002_0001.cbf", "ref": {"sg": "P 1", "sgno": 1, "cell": [41.63, 43.1, 54.2, 112.97, 90.11, 118.18], "dmin": 1.4}, "tags": ["cbf", "triclinic"], "tiers": {"smoke": "triclinic P1"}}, - {"id": "9khr", "input": "9khr/PfPlrx-DTT-9KHR/data-PfPlrx-DTT-9KHR/ods1467-lp13D1-01001.mccd", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [48.704, 50.31, 78.018, 90.0, 90.0, 90.0], "dmin": 2.0}, "tags": ["marCCD", "orthorhombic"], "tiers": {"smoke": "marCCD reader (.mccd), fast"}}, - {"id": "9mh4", "input": "9mh4/KlaeA_00150_a_B1_9mh4/data/PSL-0501_2719_master.h5", "ref": {"sg": "P 21 3", "sgno": 198, "cell": [138.65, 138.65, 138.65, 90.0, 90.0, 90.0], "dmin": 3.05}, "tags": ["h5", "cubic"]}, - {"id": "9min", "input": "9min/1151/A8_31_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [95.45, 98.54, 155.74, 90.0, 90.0, 90.0], "dmin": 2.05}, "tags": ["cbf", "orthorhombic"], "tiers": {"smoke": "known hard: halved axis"}}, - {"id": "9o0h", "input": "9o0h/3TEL-GG-TNK1-UBA_9o0h/data/E1_1_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [55.15, 65.54, 112.9, 90.0, 90.0, 90.0], "dmin": 2.24}, "tags": ["cbf", "orthorhombic"]}, - {"id": "9p7q", "input": "9p7q/9p7q_9P7Q/data/SAMDC_MTA_273_40_00001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [96.978, 45.018, 72.061, 90.0, 105.109, 90.0], "dmin": 2.21}, "tags": ["cbf", "monoclinic"]}, - {"id": "9pbb", "input": "9pbb/9pbb_9PBB/data/SAMDC_MTA_293_30_00001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [97.418, 45.876, 72.248, 90.0, 104.971, 90.0], "dmin": 2.17}, "pinned": true, "tags": ["cbf", "monoclinic"]}, - {"id": "9q41", "input": "9q41/apoSPDS_1_master.h5", "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [118.568, 133.704, 82.369, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["h5", "orthorhombic"]}, - {"id": "9q66", "input": "9q66/PREP_JP417_D8a_12694_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [105.853, 67.294, 158.019, 90.0, 99.109, 90.0], "dmin": 2.01}, "tags": ["h5", "monoclinic"]}, - {"id": "9qw8", "input": "9qw8/MX-2555/run_03_datacollection [2023-09-08 22:19:51]/FKBP12BRD4-5519-6-BD1-PPU688_w1_1_1_master.h5", "ref": {"sg": "P 1", "sgno": 1, "cell": [35.648, 35.65, 100.919, 86.458, 84.218, 72.475], "dmin": 1.8}, "tags": ["h5", "triclinic"]}, - {"id": "9rci", "input": "9rci/FEN1-CD029134_A09-3_AD049A-10_1_master.h5", "ref": {"sg": "P 1", "sgno": 1, "cell": [35.869, 39.297, 100.916, 98.3, 90.32, 90.09], "dmin": 1.662}, "ref_alternatives": [{"cell": [35.869, 39.297, 199.976, 87.087, 90.341, 90.09], "why": "The deposited cell is the (0,1/2,1/2)-centred sublattice of the supercell we report, to 0.17%, and both descriptions of this lattice are defensible. The Patterson has an off-origin peak at 62.5% of the origin, so a real translational NCS relates the two halves of our cell: describing the crystal by the doubled cell with the near-translation left in the content, or by its sublattice with the near-translation absorbed into the lattice, is a choice, not a measurement. The alternative cell is the deposited one doubled along c with the centring removed (c' = b + 2c), computed from the deposited cell alone - not from our output"}], "tags": ["h5", "triclinic", "tncs"]}, - {"id": "9rcs", "input": "9rcs/760_B7_x1_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [70.01, 78.8, 82.34, 90.0, 88.64, 90.0], "dmin": 3.012}, "tags": ["h5", "monoclinic", "cdte"]}, - {"id": "9rp9", "input": "9rp9/data_9RP9/data/TAR148-TAR148_X2_1_master.h5", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [73.48, 59.78, 91.67, 90.0, 100.85, 90.0], "dmin": 2.1}, "tags": ["h5", "monoclinic"]}, - {"id": "9sl0", "input": "9sl0/wt-1_9sl0/data/design-Yan-HLA-WT-1_1_5_1_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [60.181, 80.201, 111.589, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["h5", "orthorhombic"]}, - {"id": "9t6s", "input": "9t6s/CadC-AF6586_1_5_1_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [63.004, 64.574, 102.705, 90.0, 90.0, 90.0], "dmin": 2.0}, "tags": ["h5", "orthorhombic", "tncs"]}, - {"id": "9upt", "input": "9upt/Raw data 1A/C12_7_00001.img", "ref": {"sg": "P 6", "sgno": 168, "cell": [158.27, 158.27, 53.962, 90.0, 90.0, 120.0], "dmin": 2.369}, "tags": ["SMV", "hexagonal"]}, - {"id": "9vyb", "input": "9vyb/Antitoxin Phd_9vyb/data/G7-5_4187_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [44.37, 47.76, 48.35, 90.0, 90.0, 90.0], "dmin": 2.12}, "tags": ["h5", "orthorhombic"]}, - {"id": "9w3y", "input": "9w3y/9w3y/data/collect_01_00001_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [60.75, 69.98, 94.24, 90.0, 90.0, 90.0], "dmin": 1.5}, "tags": ["h5", "orthorhombic"]}, - {"id": "9yl4", "input": "9yl4/data_MJ4720/UWO0004_06_0001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [95.837, 111.339, 402.967, 90.0, 90.0, 90.0], "dmin": 3.7}, "pinned": true, "tags": ["cbf", "orthorhombic"]}, - {"id": "9yzk", "input": "9yzk/CC1_21_TGGGAAGATACTGGATGAGGT_empty_9yzk/data/run_29_00001.cbf", "ref": {"sg": "I 1 2 1", "sgno": 5, "cell": [75.836, 163.022, 192.278, 90.0, 98.614, 90.0], "dmin": 5.1}, "tags": ["cbf", "monoclinic"], "tiers": {"smoke": "I2/C2 setting choice at low resolution"}}, - {"id": "9z44", "input": "9z44/CC1_10_CCCGGCCGG_Cclamp_9z44/data/run_41_00001.cbf", "ref": {"sg": "I 1 2 1", "sgno": 5, "cell": [73.491, 127.653, 141.183, 90.0, 91.984, 90.0], "dmin": 7.2}, "tags": ["cbf", "monoclinic"]}, - {"id": "9z72", "input": "9z72/A7_1_00001.cbf", "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [59.173, 59.173, 426.203, 90.0, 90.0, 120.0], "dmin": 2.38}, "tags": ["cbf", "trigonal", "long-axis"], "tiers": {"smoke": "long axis c=426 A, at the FFT ceiling"}}, - {"id": "9zlo", "input": "9zlo/ASP0887_09_0114_master.h5", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [38.39, 89.98, 106.97, 90.0, 90.0, 90.0], "dmin": 2.0}, "tags": ["h5", "orthorhombic"]}, - {"id": "9zm0", "input": "9zm0/Atg23ANNS_9zm0_9ZM0/data/SeAtg23_B_10_2115_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [50.409, 30.101, 91.244, 90.0, 97.146, 90.0], "dmin": 2.1}, "tags": ["h5", "monoclinic"]}, - {"id": "9zmu", "input": "9zmu/BrmeA_18154_a_B2-Apo_9zmu/data/PSL-0613_7518_master.h5", "ref": {"sg": "P 65 2 2", "sgno": 179, "cell": [47.81, 47.81, 492.58, 90.0, 90.0, 120.0], "dmin": 1.98}, "tags": ["h5", "hexagonal"]}, - {"id": "cuhf2", "input": "cuhf2/03_CuHF2pyz2PF6b_P_O/CuHF2pyz2PF6b_P_O_01.nxs", "pinned": true, "tags": ["nxs", "small-molecule"]}, - {"id": "cytidine", "input": "cytidine/20151020-Cytidine-2th-30/fixed-omega--180-phi-scan01_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [13.98, 14.788, 5.119, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "small-molecule"]}, - {"id": "dnba", "input": "dnba/35dnba_30K_2_04_00001.cbf", "ref": {"sg": "C 1 2/c 1", "sgno": 15, "cell": [20.2635, 8.7575, 9.6697, 90.0, 109.941, 90.0], "dmin": 0.48}, "tags": ["cbf", "monoclinic", "small-molecule"]}, - {"id": "lalanine", "input": "lalanine/pgw240050_01_00001.cbf", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [5.7952, 5.933, 12.362, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "small-molecule"], "tiers": {"smoke": "small molecule, miniCBF"}}, - {"id": "metformin", "input": "metformin/013_Mmetformin_01_master.h5", "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [7.9104, 13.8794, 7.931, 90.0, 114.606, 90.0], "dmin": 0.45}, "tags": ["h5", "monoclinic", "small-molecule"], "tiers": {"smoke": "small molecule, Diamond I19 NXmx master"}}, - {"id": "nidppe", "input": "nidppe/001_NiDppeCl2_01_master.h5", "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [11.2779, 13.3386, 15.8739, 90.0, 98.7953, 90.0], "dmin": 0.77}, "tags": ["h5", "monoclinic", "small-molecule"]}, - {"id": "5mln", "input": "5mln/5mln/data/CmADHx6_w1_2_0001.cbf", "ref": {"sg": "P 21 2 21", "sgno": 18, "cell": [74.178, 80.425, 80.52, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["cbf"]}, - {"id": "5t39", "input": "5t39/10mMfuc-12h.001", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [50.222, 41.27, 58.504, 90.0, 98.58, 90.0], "dmin": 1.1004}, "tags": ["marccd"]}, - {"id": "6cdl", "input": "6cdl/nnnn_6cdl/data/wt_32-14A_p6n6.0001", "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [58.26, 85.91, 46.051, 90.0, 90.0, 90.0], "dmin": 1.25}, "tags": ["marccd"]}, - {"id": "6f3p", "input": "6f3p/6f3p/data/saha16_1_1.0002", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [142.9, 85.74, 112.01, 90.0, 122.2, 90.0], "dmin": 1.35}, "tags": ["marccd"]}, - {"id": "6g1f", "input": "6g1f/home/data/dls180217/mx13587-30/mat/DpgA-7-HA00AX9676/DpgA-7-HA00AX9676_1_0001.cbf", "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [329.28, 83.9, 133.42, 90.0, 111.58, 90.0], "dmin": 2.248}, "tags": ["cbf"]}, - {"id": "6jgh", "input": "6jgh/6jgh/data/a_00001.img", "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [50.641, 62.506, 68.179, 90.0, 90.0, 90.0], "dmin": 0.94}, "tags": ["marccd"]}, - {"id": "6nen", "input": "6nen/Pm1tr_2_1_001.img", "ref": {"sg": "P 3 1 2", "sgno": 149, "cell": [105.501, 105.501, 35.132, 90.0, 90.0, 120.0], "dmin": 2.151}, "tags": ["smv"]}, - {"id": "6qaj", "input": "6qaj/95_8_8_8_1_0001.cbf", "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [59.774, 169.332, 374.508, 90.0, 90.0, 90.0], "dmin": 2.901}, "tags": ["cbf"]}, - {"id": "6s1u", "input": "6s1u/new_4_0001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [51.603, 29.413, 85.533, 90.0, 103.75, 90.0], "dmin": 1.9}, "tags": ["marccd"]}, - {"id": "6w75", "input": "6w75/IDP51000_6W75/data/idp51000-410-a_1_1_1.001", "ref": {"sg": "P 32 2 1", "sgno": 154, "cell": [166.245, 166.245, 98.279, 90.0, 90.0, 120.0], "dmin": 1.951}, "tags": ["marccd"]}, - {"id": "6zqr", "input": "6zqr/ib23a11_M2S3_1_001.img", "ref": {"sg": "P 4", "sgno": 75, "cell": [113.6, 113.6, 44.08, 90.0, 90.0, 90.0], "dmin": 1.93}, "tags": ["smv"]}, - {"id": "6zqy", "input": "6zqy/ib22b32_MS_1_001.img", "ref": {"sg": "P 4", "sgno": 75, "cell": [119.29, 119.29, 44.21, 90.0, 90.0, 90.0], "dmin": 1.85}, "tags": ["smv"]}, - {"id": "6zr0", "input": "6zr0/ib23a23_dc_1_0001.cbf", "ref": {"sg": "P 4", "sgno": 75, "cell": [119.219, 119.219, 44.18, 90.0, 90.0, 90.0], "dmin": 1.94}, "tags": ["cbf"]}, - {"id": "7ou1", "input": "7ou1/omega_1_0001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [77.923, 91.308, 114.164, 90.0, 97.111, 90.0], "dmin": 1.65}, "tags": ["marccd"]}, - {"id": "7raa", "input": "7raa/A6_1_00001.cbf", "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [66.372, 66.372, 298.302, 90.0, 90.0, 90.0], "dmin": 2.69}, "tags": ["cbf"]}, - {"id": "9fcf", "input": "9fcf/IBCH-05-p03x03_3_00001.cbf.gz", "ref": {"sg": "P 4", "sgno": 75, "cell": [91.301, 91.301, 35.836, 90.0, 90.0, 90.0], "dmin": 2.36}, "tags": ["cbf"]}, - {"id": "9h0q", "input": "9h0q/Bc2lCnter-pma127_2_master.h5", "ref": {"sg": "H 3 2", "sgno": 155, "cell": [169.506, 169.506, 344.036, 90.0, 90.0, 120.0], "dmin": 2.55}, "tags": ["h5"]}, - {"id": "5ky6", "input": "5ky6/C11_1_001.img", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [84.511, 57.253, 164.016, 90.0, 102.57, 90.0], "dmin": 1.941}, "tags": ["marccd"]}, - {"id": "6z9g", "input": "6z9g/HASE01-ga1o2_w1_2_1_master.h5", "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [120.306, 93.815, 126.951, 90.0, 105.201, 90.0], "dmin": 1.76}, "tags": ["h5", "tncs"]} + {"id": "11if", "input": "11if/BopeA_18500_a_B2-PEF_11if/data/PSL-1503_13949_master.h5", "wavelength": 0.9786, "ref": {"sg": "P 43", "sgno": 78, "cell": [51.14, 51.14, 71.92, 90.0, 90.0, 90.0], "dmin": 1.51}, "tags": ["h5", "tetragonal"], "tiers": {"smoke": "Eiger NXmx from another facility, 4-fold screw"}}, + {"id": "36gk", "input": "36gk/CLS-0074_5-3_36GK/data/CLS-0074_5-3_master.h5", "wavelength": 0.95371, "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [120.58, 189.49, 199.69, 90.0, 90.0, 90.0], "dmin": 2.28}, "tags": ["h5", "orthorhombic"]}, + {"id": "3inp", "input": "3inp/IDP02542_3inp/data/idp02542b.001", "wavelength": 0.97872, "ref": {"sg": "F 41 3 2", "sgno": 210, "cell": [224.08, 224.08, 224.08, 90.0, 90.0, 90.0], "dmin": 2.05}, "tags": ["marCCD", "cubic"]}, + {"id": "3ky7", "input": "3ky7/IDP90258_3ky7/data/idp90258f.001", "wavelength": 0.97856, "ref": {"sg": "P 43 3 2", "sgno": 212, "cell": [125.176, 125.176, 125.176, 90.0, 90.0, 90.0], "dmin": 2.35}, "tags": ["marCCD", "cubic"]}, + {"id": "3mc4", "input": "3mc4/series/206918e4_x0001.img", "wavelength": 1.54178, "ref": {"sg": "H 3", "sgno": 146, "cell": [104.03, 104.03, 105.54, 90.0, 90.0, 120.0], "dmin": 1.95}, "tags": ["smv", "trigonal", "home-source"]}, + {"id": "3meb", "input": "3meb/series/202097g3_x0001.img", "wavelength": 1.54178, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [58.58, 101.15, 81.53, 90.0, 90.62, 90.0], "dmin": 1.9}, "tags": ["smv", "monoclinic", "home-source"]}, + {"id": "3p85", "input": "3p85/series/217175d10_x0001.img", "wavelength": 1.54178, "ref": {"sg": "P 63 2 2", "sgno": 182, "cell": [127.29, 127.29, 72.9, 90.0, 90.0, 120.0], "dmin": 1.9}, "tags": ["smv", "hexagonal", "home-source"]}, + {"id": "3r6o", "input": "3r6o/series/219594b8_x0001.img", "wavelength": 1.54178, "ref": {"sg": "I 41", "sgno": 80, "cell": [90.68, 90.68, 76.13, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["smv", "tetragonal", "home-source"]}, + {"id": "5cc8", "input": "5cc8/data/263060g10_x0001.img", "wavelength": 1.54178, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [87.14, 93.76, 72.49, 90.0, 90.0, 90.0], "dmin": 1.75}, "tags": ["smv", "orthorhombic", "home-source", "tncs"]}, + {"id": "5ebi", "input": "5ebi/dna-rna-chimera_Ba_high_2_001.img", "wavelength": 0.82657, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [35.72, 44.1, 35.72, 90.0, 119.98, 90.0], "dmin": 1.09}, "pinned": true, "tags": ["marCCD", "monoclinic", "twin"]}, + {"id": "5epe", "input": "5epe/030805_5epe/data/E1_7_set.001", "wavelength": 0.97856, "ref": {"sg": "F 2 3", "sgno": 196, "cell": [157.536, 157.536, 157.536, 90.0, 90.0, 90.0], "dmin": 1.9}, "tags": ["marCCD", "cubic"]}, + {"id": "5f6m", "input": "5f6m/crystal3_00001.cbf", "wavelength": 1.03317, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [54.81, 58.51, 67.42, 90.0, 90.0, 90.0], "dmin": 1.1}, "tags": ["cbf", "orthorhombic", "diffuse"]}, + {"id": "5j23", "input": "5j23/012132_5j23/data/XZ2_set.001", "wavelength": 0.97856, "ref": {"sg": "H 3", "sgno": 146, "cell": [175.822, 175.822, 136.839, 90.0, 90.0, 120.0], "dmin": 2.3}, "tags": ["marCCD", "trigonal", "twin"]}, + {"id": "5jvn", "input": "5jvn/5jvn/data/PPDK-AV2820_w1_3_0001.cbf", "wavelength": 0.97625, "ref": {"sg": "P 6 2 2", "sgno": 177, "cell": [249.43, 249.43, 84.06, 90.0, 90.0, 120.0], "dmin": 2.9}, "tags": ["cbf", "hexagonal"]}, + {"id": "5lzl", "input": "5lzl/pcalad/alad-levoil6_M3S15_1_0001.cbf", "wavelength": 1.04346, "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [205.561, 205.561, 199.171, 90.0, 90.0, 120.0], "dmin": 3.47}, "tags": ["cbf", "trigonal"]}, + {"id": "5m17", "input": "5m17/BxGH99_WTDD_HA00AG2801_2_0001.cbf", "wavelength": 0.9795, "ref": {"sg": "I 4", "sgno": 79, "cell": [108.576, 108.576, 67.746, 90.0, 90.0, 90.0], "dmin": 1.03}, "tags": ["cbf", "tetragonal"]}, + {"id": "5nw5", "input": "5nw5/5NW5_1_00001.cbf", "wavelength": 0.91863, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [92.14, 169.8, 390.16, 90.0, 90.0, 90.0], "dmin": 6.502}, "tags": ["cbf", "orthorhombic", "low-resolution"]}, + {"id": "5reo", "input": "5reo/cbf/Mpro-x0752_1_0001.cbf", "wavelength": 0.9126, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [112.39, 52.59, 44.38, 90.0, 103.04, 90.0], "dmin": 1.88}, "tags": ["cbf", "monoclinic"], "tiers": {"smoke": "fastest set (10 s): miniCBF, monoclinic"}}, + {"id": "5src", "input": "5src/5src/data/FRS004_15_1_00001.cbf", "wavelength": 0.88557, "ref": {"sg": "P 43", "sgno": 78, "cell": [88.68, 88.68, 39.23, 90.0, 90.0, 90.0], "dmin": 1.05}, "tags": ["cbf", "tetragonal"]}, + {"id": "5uth", "input": "5uth/data/287007e1_0001.img", "wavelength": 1.54178, "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [69.28, 69.28, 153.8, 90.0, 90.0, 120.0], "dmin": 1.95}, "tags": ["smv", "trigonal", "home-source"]}, + {"id": "5vml", "input": "5vml/data/271705c8_x_0001.img", "wavelength": 1.54178, "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [66.31, 66.31, 115.26, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["smv", "tetragonal", "home-source"]}, + {"id": "6cee", "input": "6cee/data/Rachel_AV6_screen_0001.img", "wavelength": 1.54178, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [40.72, 44.11, 55.91, 90.0, 90.0, 90.0], "dmin": 1.55}, "tags": ["smv", "orthorhombic", "home-source"]}, + {"id": "6fid", "input": "6fid/Trypsin_x1_align_2_0001.cbf", "wavelength": 2.0664, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [59.947, 64.107, 69.694, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["cbf", "orthorhombic"]}, + {"id": "6fvz", "input": "6fvz/maox215_6fvz/data/maox215_w1_1_0001.cbf", "wavelength": 0.8729, "ref": {"sg": "C 2 2 2", "sgno": 21, "cell": [131.182, 222.753, 86.482, 90.0, 90.0, 90.0], "dmin": 1.8}, "tags": ["cbf", "orthorhombic"]}, + {"id": "6fwc", "input": "6fwc/maox225_6fwc/data/maox225_1_00001.cbf", "wavelength": 1.00004, "ref": {"sg": "C 2 2 2", "sgno": 21, "cell": [131.728, 222.051, 86.293, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["cbf", "orthorhombic"]}, + {"id": "6h2p_native", "input": "6h2p/ProlMut1MnCAC_6h2p/data/Prol_Mut1_Mn_X1_nat_1_0002.cbf", "wavelength": 0.9184, "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [103.453, 107.075, 216.543, 90.0, 90.0, 90.0], "dmin": 1.479}, "tags": ["cbf", "orthorhombic", "two-wavelength"]}, + {"id": "6h2p_1p89A", "input": "6h2p/ProlMut1MnCAC_6h2p/data/Prol_Mut1_Mn_X_pk1_2_0001.cbf", "wavelength": 1.89317, "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [103.453, 107.075, 216.543, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "long-wavelength", "two-wavelength"]}, + {"id": "6h5t", "input": "6h5t/6h5t/data/SH3-1_x2_1_001.img", "wavelength": 0.895, "ref": {"sg": "I 4 2 2", "sgno": 97, "cell": [86.792, 86.792, 141.768, 90.0, 90.0, 90.0], "dmin": 1.689}, "tags": ["marCCD", "tetragonal"]}, + {"id": "6hv2", "input": "6hv2/6hv2/data/coll_1_master.h5", "wavelength": 0.99987, "ref": {"sg": "P 61 2 2", "sgno": 178, "cell": [68.928, 68.928, 133.563, 90.0, 90.0, 120.0], "dmin": 1.709}, "tags": ["h5", "hexagonal"]}, + {"id": "6hwj", "input": "6hwj/bg3006_2_0001.cbf", "wavelength": 1.0332, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [59.812, 96.07, 80.252, 90.0, 106.69, 90.0], "dmin": 1.979}, "tags": ["cbf", "monoclinic"]}, + {"id": "6i3j", "input": "6i3j/6i3j/data/BF19_go_1_0001.img", "wavelength": 0.91841, "ref": {"sg": "F 2 2 2", "sgno": 22, "cell": [134.382, 203.846, 226.744, 90.0, 90.0, 90.0], "dmin": 2.59}, "tags": ["marCCD", "orthorhombic"]}, + {"id": "6iu5", "input": "6iu5/6iu5_VIT1_MBD_Zn/peak/peak_000001.cbf", "wavelength": 1.282, "ref": {"sg": "P 31", "sgno": 144, "cell": [84.942, 84.942, 98.19, 90.0, 90.0, 120.0], "dmin": 2.25}, "pinned": true, "tags": ["cbf", "trigonal", "twin"]}, + {"id": "6iu6", "input": "6iu6/6iu6_VIT1_MBD_Ni/peak/peak_000001.cbf", "wavelength": 1.485, "ref": {"sg": "P 31", "sgno": 144, "cell": [84.744, 84.744, 97.398, 90.0, 90.0, 120.0], "dmin": 2.9}, "pinned": true, "tags": ["cbf", "trigonal", "twin"]}, + {"id": "6iu8", "input": "6iu8/6iu8_VIT1_MBD_Co/low/low_000001.cbf", "wavelength": 1.648, "ref": {"sg": "P 31", "sgno": 144, "cell": [85.503, 85.503, 98.355, 90.0, 90.0, 120.0], "dmin": 2.7}, "tags": ["cbf", "trigonal"]}, + {"id": "6iu9", "input": "6iu9/6iu9_VIT1_MBD_Fe/peak/data01_000001.cbf", "wavelength": 1.7401, "ref": {"sg": "P 31", "sgno": 144, "cell": [85.321, 85.321, 97.573, 90.0, 90.0, 120.0], "dmin": 3.0}, "pinned": true, "tags": ["cbf", "trigonal", "twin"]}, + {"id": "6jgi", "input": "6jgi/6jgi/data/00001.img", "wavelength": 0.75, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [50.857, 62.403, 69.172, 90.0, 90.0, 90.0], "dmin": 0.85}, "tags": ["marCCD", "orthorhombic"]}, + {"id": "6jgj", "input": "6jgj/6jgj/data/000001.cbf", "wavelength": 0.35, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [50.87, 62.34, 68.85, 90.0, 90.0, 90.0], "dmin": 0.77}, "tags": ["cbf", "orthorhombic", "cdte"]}, + {"id": "6moj", "input": "6moj/A9_1_00001.cbf", "wavelength": 0.97741, "ref": {"sg": "I 41 2 2", "sgno": 98, "cell": [130.418, 130.418, 293.453, 90.0, 90.0, 90.0], "dmin": 2.431}, "tags": ["cbf", "tetragonal"]}, + {"id": "6o2h", "input": "6o2h/lys_nitr_10_1_0001.cbf", "wavelength": 0.9768, "ref": {"sg": "P 1", "sgno": 1, "cell": [27.424, 32.134, 34.513, 88.657, 108.46, 111.877], "dmin": 1.212}, "tags": ["cbf", "triclinic", "diffuse"]}, + {"id": "6oel", "input": "6oel/1449-8_3_001.img", "wavelength": 0.99994, "ref": {"sg": "F 41 3 2", "sgno": 210, "cell": [328.1, 328.1, 328.1, 90.0, 90.0, 90.0], "dmin": 3.1}, "tags": ["SMV", "cubic"], "tiers": {"smoke": "SMV reader (ADSC), cubic F4132"}}, + {"id": "6p8p", "input": "6p8p/14_12_1_0001.cbf", "wavelength": 0.9792, "ref": {"sg": "P 4", "sgno": 75, "cell": [97.525, 97.525, 60.134, 90.0, 90.0, 90.0], "dmin": 1.635}, "tags": ["cbf", "tetragonal"]}, + {"id": "6pb3", "input": "6pb3/14_16_1_000001.cbf", "wavelength": 0.97918, "ref": {"sg": "P 6", "sgno": 168, "cell": [100.436, 100.436, 48.86, 90.0, 90.0, 120.0], "dmin": 2.048}, "tags": ["cbf", "hexagonal"]}, + {"id": "6pxb", "input": "6pxb/TJB1_3_rachel_1_0001.cbf", "wavelength": 0.9792, "pinned": true, "ref": {"sg": "P 32", "sgno": 145, "cell": [64.021, 64.021, 119.447, 90.0, 90.0, 120.0], "dmin": 1.747}, "ref_alternatives": [{"sg": "P 32 1 2", "sgno": 153, "why": "The deposition merged in point group 3 (its 55502 unique reflections to 1.747 A are what Laue class -3 holds, twice what -3 1 m would) and refined six chains in P 32. The intensities read point group 312 instead: the three added two-folds correlate at 0.98-0.99, at or above the three-folds nobody disputes (0.98), against 0.37-0.40 for the 321 and 622 operators, POINTLESS on our P1 merge picks P -3 1 m (likelihood 1.000), and the reflections centric in 312 but not in 3 are distributed as centric (+435 nats), which a twin law cannot produce. Against that, the deposited chains pair under the added two-fold at 0.3-0.7 A, more than coordinate error at 1.75 A, and the model tells the two indexings apart (R 0.22 against 0.25), so the two-fold may be a near-exact non-crystallographic one; ZANUDA settles on P 32 2 1, whose operators these data do not support. Neither answer is established, so both are accepted; P 31 1 2, which the data cannot separate from P 32 1 2, is accepted as its hand"}], "tags": ["cbf", "trigonal"]}, + {"id": "6pxc", "input": "6pxc/TJB6_1_Rachel_1_0001.cbf", "wavelength": 0.9791, "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [44.19, 64.827, 87.239, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["cbf", "orthorhombic"]}, + {"id": "6r72", "input": "6r72/V-CK63-8-ld_1_master.h5", "wavelength": 0.98011, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [117.805, 110.754, 155.615, 90.0, 93.23, 90.0], "dmin": 3.95}, "pinned": true, "tags": ["h5", "monoclinic"]}, + {"id": "6rlr", "input": "6rlr/_b4_1_master.h5", "wavelength": 0.97949, "ref": {"sg": "P 1", "sgno": 1, "cell": [39.986, 39.998, 63.643, 80.39, 76.29, 68.15], "dmin": 2.0}, "tags": ["h5", "triclinic", "twin"]}, + {"id": "6toc", "input": "6toc/zenodo/dts_1_00001.cbf", "wavelength": 1.00004, "ref": {"sg": "P 42", "sgno": 77, "cell": [31.523, 31.523, 81.599, 90.0, 90.0, 90.0], "dmin": 1.853}, "ref_alternatives": [{"sg": "P 42 2 2", "sgno": 93, "why": "Our reduction and an independent POINTLESS run on our own P1 merge both read point group 422, and the deposited asymmetric unit's two chains are related by the very two-fold the higher group adds, to 0.16 A CA RMSD over 43 residues - coordinate error at 1.85 A. Merging in P 42 2 2 costs 0.0006 in R_meas for 1.75x the multiplicity and correlates better with the deposited model (0.9802 vs 0.9764). The refinement test is NOT unanimous: ZANUDA 1.097 refines P 42 2 2 to R-free 0.2561 against P 42's 0.2636 at half the parameters and reports the deposited assignment incorrect, while an independent Refmac 5.8.0431 comparison on a symmetry-consistent free set puts P 42 ahead by 0.004-0.020 depending on cycle count - less than the spread between refinement protocols. Neither answer is established: a pseudo-symmetry too exact for any test we have remains a live explanation, and so does the deposited assignment. Either is accepted"}], "tags": ["cbf", "tetragonal", "twin"]}, + {"id": "6ttn", "input": "6ttn/6ttn/data/H6H_33_01_hyo_full_1_0001.cbf", "wavelength": 0.9184, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [39.894, 79.841, 104.701, 90.0, 90.0, 90.0], "dmin": 1.12}, "tags": ["cbf", "orthorhombic"]}, + {"id": "6u7g", "input": "6u7g/6u7g/data/SNB02_11_8_1_master.h5", "wavelength": 1.03318, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [99.555, 98.682, 147.525, 90.0, 104.6, 90.0], "dmin": 2.35}, "pinned": true, "tags": ["h5", "monoclinic"]}, + {"id": "6ukf", "input": "6ukf/6ukf/data/XDC-7_Pn6_000001.cbf", "wavelength": 0.8, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [61.018, 37.317, 69.027, 90.0, 109.768, 90.0], "dmin": 1.0}, "tags": ["cbf", "monoclinic"]}, + {"id": "6v2r", "input": "6v2r/data/cbx7.248483.hv6_screen_0001.img", "wavelength": 1.54178, "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [40.193, 40.193, 83.127, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["smv", "tetragonal", "home-source"]}, + {"id": "6vww", "input": "6vww/IDP51000_6vww/data/m4H11g_00001.cbf", "wavelength": 0.97918, "ref": {"sg": "P 63", "sgno": 173, "cell": [150.539, 150.539, 111.31, 90.0, 90.0, 120.0], "dmin": 2.2}, "tags": ["cbf", "hexagonal", "twin"], "tiers": {"smoke": "merohedral twin, P63"}}, + {"id": "6w4h", "input": "6w4h/IDP51000_6W4H/data/idp51000-201-a_1_2_3.001", "wavelength": 0.97872, "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [167.74, 167.74, 51.942, 90.0, 90.0, 120.0], "dmin": 1.8}, "tags": ["marCCD", "trigonal"]}, + {"id": "6wzo", "input": "6wzo/9_1_1_000001.cbf", "wavelength": 0.97918, "ref": {"sg": "P 1", "sgno": 1, "cell": [43.718, 50.061, 69.337, 106.499, 90.094, 97.145], "dmin": 1.42}, "tags": ["cbf", "triclinic"]}, + {"id": "6yqf", "input": "6yqf/6yqf/data/SYCE2TEX12-8_1_0001.cbf", "wavelength": 0.96878, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [42.67, 59.68, 156.49, 90.0, 90.0, 90.0], "dmin": 3.327}, "tags": ["cbf", "orthorhombic"]}, + {"id": "6z8o", "input": "6z8o/HASE01-gaKr1_w1_1_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [63.716, 97.022, 121.318, 90.0, 104.656, 90.0], "dmin": 2.2}, "tags": ["h5", "monoclinic"]}, + {"id": "6ze4", "input": "6ze4/806/o8_1_0001.cbf", "wavelength": 0.9184, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [93.56, 109.88, 116.11, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["cbf", "orthorhombic"], "tiers": {"smoke": "known hard: P212121 under-called as P21 since rc168"}}, + {"id": "7arr", "input": "7arr/PF4N_04D_w1_1_00001.cbf", "wavelength": 0.97626, "ref": {"sg": "P 1", "sgno": 1, "cell": [30.937, 32.068, 43.087, 114.16, 91.88, 109.86], "dmin": 1.1}, "tags": ["cbf", "triclinic"]}, + {"id": "7atg", "input": "7atg/7atg/data/put-2-2.0s_5_00001.cbf", "wavelength": 0.72932, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [17.97, 31.02, 43.86, 90.0, 90.0, 90.0], "dmin": 0.6}, "tags": ["cbf", "orthorhombic"]}, + {"id": "7bgt", "input": "7bgt/MPMV2243_3_001.img", "wavelength": 0.82657, "ref": {"sg": "P 1", "sgno": 1, "cell": [29.296, 67.618, 69.716, 76.845, 83.875, 83.645], "dmin": 1.93}, "tags": ["marCCD", "triclinic"]}, + {"id": "7brr", "input": "7brr/7brr/data/3C_2_1_master.h5", "wavelength": 0.97919, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [55.45, 99.02, 59.583, 90.0, 108.54, 90.0], "dmin": 1.35}, "tags": ["h5", "monoclinic"]}, + {"id": "7dkp", "input": "7dkp/7dkp/data/LOX1-sree-p1a9-1p2_w1_1_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [49.84, 169.451, 49.842, 90.0, 93.51, 90.0], "dmin": 1.45}, "tags": ["h5", "monoclinic"]}, + {"id": "7k1l", "input": "7k1l/IDP52015_7k1l/data/cs1C2eg_00001.cbf", "wavelength": 0.97913, "ref": {"sg": "P 63", "sgno": 173, "cell": [150.83, 150.83, 110.73, 90.0, 90.0, 120.0], "dmin": 2.25}, "tags": ["cbf", "hexagonal"], "tiers": {"smoke": "known hard: screw axis (P6 vs P63)"}}, + {"id": "7kcn", "input": "7kcn/7kcn/data/ttr_hiuase_1_D1_33_00001.cbf", "wavelength": 1.45901, "ref": {"sg": "P 41 2 2", "sgno": 91, "cell": [67.03, 67.03, 116.89, 90.0, 90.0, 90.0], "dmin": 1.46}, "tags": ["cbf", "tetragonal"]}, + {"id": "7l6j", "input": "7l6j/7l6j/data/idp97824-102sm-a_1_1_7.001", "wavelength": 0.97872, "ref": {"sg": "I 41 3 2", "sgno": 214, "cell": [171.693, 171.693, 171.693, 90.0, 90.0, 90.0], "dmin": 1.78}, "tags": ["marCCD", "cubic"]}, + {"id": "7l84", "input": "7l84/301_helical_1_0001.cbf", "wavelength": 1.892, "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [79.344, 79.344, 37.81, 90.0, 90.0, 90.0], "dmin": 1.7}, "tags": ["cbf", "tetragonal"]}, + {"id": "7mzt", "input": "7mzt/7mzt/data/BVG2_Pn3_000001.cbf", "wavelength": 1.0, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [113.619, 96.989, 108.33, 90.0, 90.0, 90.0], "dmin": 4.07}, "tags": ["cbf", "orthorhombic"]}, + {"id": "7n0i", "input": "7n0i/8_2_1_00001.cbf", "wavelength": 1.00004, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [75.846, 131.557, 140.048, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["cbf", "orthorhombic", "tncs"]}, + {"id": "7n2s", "input": "7n2s/1449-4_1_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [83.211, 52.789, 106.291, 90.0, 98.278, 90.0], "dmin": 2.37}, "tags": ["cbf", "monoclinic"]}, + {"id": "7orr", "input": "7orr/7orr/data/Nsp10-VT00022_1_master.h5", "wavelength": 0.97879, "ref": {"sg": "I 21 3", "sgno": 199, "cell": [105.88, 105.88, 105.88, 90.0, 90.0, 90.0], "dmin": 1.79}, "tags": ["h5", "cubic"]}, + {"id": "7os3", "input": "7os3/pos2_1/pos2_1_00001.cbf", "wavelength": 2.0664, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [78.15, 91.05, 105.84, 90.0, 90.0, 90.0], "dmin": 2.177}, "tags": ["cbf", "orthorhombic"]}, + {"id": "7ph1", "input": "7ph1/7ph1/data/9_1_0001.cbf", "wavelength": 0.9184, "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [74.975, 81.289, 124.248, 90.0, 90.0, 90.0], "dmin": 1.18}, "tags": ["cbf", "orthorhombic"]}, + {"id": "7pq7", "input": "7pq7/7pq7/data/1436_p52_A3_1_1_00001.cbf", "wavelength": 0.9789, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [120.88, 51.73, 75.54, 90.0, 125.14, 90.0], "dmin": 1.55}, "tags": ["cbf", "monoclinic"]}, + {"id": "7qij", "input": "7qij/907/dg046_2_data_1_00001.cbf", "wavelength": 0.97625, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [143.46, 324.92, 369.38, 90.0, 90.0, 90.0], "dmin": 4.1}, "tags": ["cbf", "orthorhombic"]}, + {"id": "7qis", "input": "7qis/7qis/data/Chymo_Dfe_x1_2_0001.cbf", "wavelength": 0.9184, "ref": {"sg": "P 61", "sgno": 169, "cell": [100.269, 100.269, 206.215, 90.0, 90.0, 120.0], "dmin": 1.83}, "tags": ["cbf", "hexagonal"]}, + {"id": "7ris", "input": "7ris/7ris/data/GLVase_Ca_we21108b7b_1_2_2.001_master.h5", "wavelength": 1.03329, "ref": {"sg": "P 32 2 1", "sgno": 154, "cell": [44.54, 44.54, 189.95, 90.0, 90.0, 120.0], "dmin": 1.72}, "pinned": true, "tags": ["h5", "trigonal"]}, + {"id": "7rji", "input": "7rji/7rji/data/DD3_00000.cbf", "wavelength": 1.45859, "ref": {"sg": "H 3 2", "sgno": 155, "cell": [83.036, 83.036, 124.83, 90.0, 90.0, 120.0], "dmin": 1.71}, "tags": ["cbf", "trigonal"]}, + {"id": "7t5t", "input": "7t5t/A1_2_00001.cbf", "wavelength": 0.97892, "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [95.309, 95.309, 104.915, 90.0, 90.0, 90.0], "dmin": 1.35}, "pinned": true, "tags": ["cbf", "tetragonal"]}, + {"id": "7tcd", "input": "7tcd/7tcd/data/col_1_master.h5", "wavelength": 0.99987, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [138.504, 47.902, 78.068, 90.0, 107.56, 90.0], "dmin": 1.7}, "tags": ["h5", "monoclinic"]}, + {"id": "7yzx", "input": "7yzx/7yzx/data/783-1_1_0001.cbf", "wavelength": 0.96864, "ref": {"sg": "P 63 2 2", "sgno": 182, "cell": [169.42, 169.42, 141.83, 90.0, 90.0, 120.0], "dmin": 1.9}, "tags": ["cbf", "hexagonal"]}, + {"id": "8a1a", "input": "8a1a/L1F11v1_8a1a/data/L1-F11-v1_9ly1_D06_01_2_master.h5", "wavelength": 1.0, "ref": {"sg": "P 65", "sgno": 170, "cell": [191.866, 191.866, 122.409, 90.0, 90.0, 120.0], "dmin": 2.05}, "tags": ["h5", "hexagonal"]}, + {"id": "8agq", "input": "8agq/8agq_zip_8AGQ/data/coll_1_00001.cbf", "wavelength": 1.0, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [89.948, 55.447, 54.787, 90.0, 113.55, 90.0], "dmin": 1.093}, "tags": ["cbf", "monoclinic"]}, + {"id": "8dqb", "input": "8dqb/KloxA_17380_a_B1-Apo_I23_Form_8dqb/data/PSL-1405_653_master.h5", "wavelength": 0.97856, "ref": {"sg": "I 2 3", "sgno": 197, "cell": [164.124, 164.124, 164.124, 90.0, 90.0, 90.0], "dmin": 2.5}, "tags": ["h5", "cubic"]}, + {"id": "8dyz", "input": "8dyz/lys_1_2_0001.cbf", "wavelength": 0.9768, "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [79.632, 79.632, 38.296, 90.0, 90.0, 90.0], "dmin": 1.272}, "tags": ["cbf", "tetragonal", "diffuse"]}, + {"id": "8dz7", "input": "8dz7/lys_rt_2_38_0001.cbf", "wavelength": 0.976, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [30.486, 56.401, 73.852, 90.0, 90.0, 90.0], "dmin": 1.34}, "tags": ["cbf", "orthorhombic", "diffuse"]}, + {"id": "8egn", "input": "8egn/PsaeA_00137_b_B5-az13643701_8egn/data/FKC8_4_0001.cbf", "wavelength": 1.1807, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [71.69, 75.16, 109.79, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["cbf", "orthorhombic"]}, + {"id": "8iya", "input": "8iya/8iya/data/11_1_master.h5", "wavelength": 0.97918, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [102.747, 50.077, 109.248, 90.0, 91.754, 90.0], "dmin": 2.43}, "tags": ["h5", "monoclinic"]}, + {"id": "8k1g", "input": "8k1g/8k1g/data/KP02_0001.cbf", "wavelength": 0.97942, "ref": {"sg": "I 4 2 2", "sgno": 97, "cell": [182.01, 182.01, 80.71, 90.0, 90.0, 90.0], "dmin": 2.09}, "tags": ["cbf", "tetragonal"]}, + {"id": "8oic", "input": "8oic/8oic/data/_b20_1_master.h5", "wavelength": 0.9795, "ref": {"sg": "P 1", "sgno": 1, "cell": [73.055, 94.665, 120.589, 105.081, 89.959, 93.826], "dmin": 2.8}, "tags": ["h5", "triclinic"]}, + {"id": "8owm", "input": "8owm/p13x4_1_00001.cbf", "wavelength": 0.97625, "ref": {"sg": "P 1", "sgno": 1, "cell": [95.544, 95.629, 95.841, 90.416, 93.59, 117.775], "dmin": 1.7}, "tags": ["cbf", "triclinic"]}, + {"id": "8pqd", "input": "8pqd/8pqd/data/11AT05_w1_1_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [59.39, 59.37, 192.89, 90.0, 90.0, 90.0], "dmin": 1.5}, "tags": ["h5", "orthorhombic"]}, + {"id": "8qaw", "input": "8qaw/p18x3/p18x3_1_00001.cbf.gz", "wavelength": 0.97625, "ref": {"sg": "H 3", "sgno": 146, "cell": [137.68, 137.68, 265.907, 90.0, 90.0, 120.0], "dmin": 1.55}, "tags": ["cbf", "trigonal", "long-axis"]}, + {"id": "8qj5", "input": "8qj5/8qj5/data/Lcsbc3_1_2_00001.cbf", "wavelength": 1.0, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [57.61, 100.63, 77.93, 90.0, 96.06, 90.0], "dmin": 1.63}, "tags": ["cbf", "monoclinic"]}, + {"id": "8qq7", "input": "8qq7/cbf/SpNox-Sp49-B3-B_088_1_0001.cbf", "wavelength": 0.966, "ref": {"sg": "P 64 2 2", "sgno": 181, "cell": [145.967, 145.967, 153.619, 90.0, 90.0, 120.0], "dmin": 3.62}, "tags": ["cbf", "hexagonal"]}, + {"id": "8r5r", "input": "8r5r/8r5r/data/D5-33805_1_1_master.h5", "wavelength": 0.8856, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [91.74, 132.91, 137.5, 90.0, 90.0, 90.0], "dmin": 3.08}, "tags": ["h5", "orthorhombic"]}, + {"id": "8rud", "input": "8rud/p08x03_1_00001.cbf", "wavelength": 0.97626, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [78.066, 91.386, 114.547, 90.0, 96.9, 90.0], "dmin": 2.1}, "tags": ["cbf", "monoclinic"]}, + {"id": "8s38", "input": "8s38/MX733_2_00001.cbf", "wavelength": 0.97626, "ref": {"sg": "I 21 21 21", "sgno": 24, "cell": [95.368, 163.1, 219.023, 90.0, 90.0, 90.0], "dmin": 1.89}, "tags": ["cbf", "orthorhombic"]}, + {"id": "8sa8", "input": "8sa8/KlaeA_00906_a_B1-PLP_8sa8/data/PSL-0412_255_master.h5", "wavelength": 0.97949, "ref": {"sg": "I 1 2 1", "sgno": 5, "cell": [87.887, 131.544, 165.363, 90.0, 104.48, 90.0], "dmin": 1.3}, "tags": ["h5", "monoclinic"]}, + {"id": "8sqo", "input": "8sqo/BrabA_00028_a_A1-Mg_8sqo/data/PSL-1801_942_master.h5", "wavelength": 0.97949, "ref": {"sg": "P 4 3 2", "sgno": 207, "cell": [112.91, 112.91, 112.91, 90.0, 90.0, 90.0], "dmin": 1.55}, "tags": ["h5", "cubic"]}, + {"id": "8sqq", "input": "8sqq/BrabA_00028_a_A1-Apo_8sqq/data/PSL-1810_965_master.h5", "wavelength": 0.97949, "ref": {"sg": "F 4 3 2", "sgno": 209, "cell": [171.52, 171.52, 171.52, 90.0, 90.0, 90.0], "dmin": 2.25}, "tags": ["h5", "cubic"]}, + {"id": "8sqt", "input": "8sqt/BrabA_00028_a_A1-Fe_8sqt/data/PSL-1812_971_master.h5", "wavelength": 0.97949, "ref": {"sg": "F 4 3 2", "sgno": 209, "cell": [170.72, 170.72, 170.72, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["h5", "cubic"]}, + {"id": "8t7r", "input": "8t7r/8t7r/data/GREEN-04_Pn5_000001.cbf", "wavelength": 1.0, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [357.076, 259.582, 255.361, 90.0, 133.13, 90.0], "dmin": 3.84}, "tags": ["cbf", "monoclinic"]}, + {"id": "8tha", "input": "8tha/1TEL_double_helix_8tha/data/A3_1_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "P 64", "sgno": 172, "cell": [69.16, 69.16, 29.13, 90.0, 90.0, 120.0], "dmin": 1.68}, "tags": ["cbf", "hexagonal"]}, + {"id": "8tyy", "input": "8tyy/CPS_2_1_000001.cbf", "wavelength": 0.97918, "ref": {"sg": "F 4 3 2", "sgno": 209, "cell": [214.891, 214.891, 214.891, 90.0, 90.0, 90.0], "dmin": 1.68}, "tags": ["cbf", "cubic"]}, + {"id": "8u0i", "input": "8u0i/8u0i/data/dI-2-05a_4_00001.cbf", "wavelength": 1.00004, "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [50.317, 50.317, 90.596, 90.0, 90.0, 90.0], "dmin": 1.541}, "tags": ["cbf", "tetragonal"]}, + {"id": "8v4o", "input": "8v4o/CaalA_00629_a_FS11-AMP_8v4o/data/PSL-1602_2180_master.h5", "wavelength": 0.97949, "ref": {"sg": "P 61 2 2", "sgno": 178, "cell": [139.46, 139.46, 544.99, 90.0, 90.0, 120.0], "dmin": 2.7}, "tags": ["h5", "hexagonal"]}, + {"id": "8xbp", "input": "8xbp/8xbp/data/puck2872_3_1_master.h5", "wavelength": 0.97856, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [148.29, 50.78, 60.21, 90.0, 92.33, 90.0], "dmin": 1.99}, "ref_override": {"cell": [148.736, 51.777, 60.181, 90.0, 92.36, 90.0]}, "ref_override_why": "the uploaded sweep is a different crystal from the one the deposited cell describes: master data_collection_date 2023-06-21, deposited _diffrn_detector.pdbx_collection_date 2023-06-23, and the deposited b = 50.78 is 2.0% from the b these images give. The override is the cell DIALS 3.29 indexes de novo on this master, b = 51.777(11); the deposited space group and d_min are kept", "tags": ["h5", "monoclinic"]}, + {"id": "8xte", "input": "8xte/1101/G2_1_00001.cbf", "wavelength": 0.97853, "ref": {"sg": "P 32", "sgno": 145, "cell": [208.77, 208.77, 67.22, 90.0, 90.0, 120.0], "dmin": 1.99}, "ref_alternatives": [{"sg": "P 31 2 1", "sgno": 152, "why": "The evidence favours the higher group here. The twin-immune centric zone of the added two-folds - reflections the higher group makes centric, which are their own twin mates and so cannot be made to read centric by a merohedral twin law - gives <|E^2-1|> = 0.946 +/- 0.012 against a centric expectation of 0.968 and an acentric 0.736, at +707 nats. Re-refinement on a shared free set with the twin law removed from both sides favours P 3_2 2 1 (0.2177/0.2322) over P 3_2 (0.2460/0.2612), and the deposited entry's published R values reproduce only with an undeclared twin law h,-h-k,-l at alpha = 0.50, which is itself a 321-symmetric target. No refinement R can close the question in principle, because a merohedral twin at exactly alpha = 0.5 and true 321 predict identical intensities; the case rests on the centric zone. Both answers are accepted, ours being the better supported"}], "tags": ["cbf", "trigonal"]}, + {"id": "8xtf", "input": "8xtf/1102/puck03_12_1_master.h5", "wavelength": 0.97918, "ref": {"sg": "H 3 2", "sgno": 155, "cell": [211.75, 211.75, 67.42, 90.0, 90.0, 120.0], "dmin": 2.13}, "tags": ["h5", "trigonal"]}, + {"id": "8xtg", "input": "8xtg/1100/mpag15_1_00001.cbf", "wavelength": 0.97853, "ref": {"sg": "P 32", "sgno": 145, "cell": [199.54, 199.54, 67.15, 90.0, 90.0, 120.0], "dmin": 2.0}, "ref_alternatives": [{"sg": "P 31 2 1", "sgno": 152, "why": "The evidence favours the DEPOSITION here, and this row must not be read as the 8xte one. Every correlation-based instrument we have - our own operator correlations, POINTLESS (0.85 on our P1 merge) - reads point group 321, but our own twin-immune centric-zone test, the only one that separates real symmetry from pseudo-symmetry, reads <|E^2-1|> = 0.869 at -44.9 nats, between the two expectations and on the wrong side, i.e. AGAINST the promotion, with an L-test twin fraction of 0.20-0.26. Whether this crystal is partially twinned or purely pseudo-symmetric is not established. Both answers are accepted, the deposition being the better supported"}], "tags": ["cbf", "trigonal"]}, + {"id": "8y74", "input": "8y74/1_1/1_1_master.h5", "wavelength": 0.97918, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [125.782, 76.553, 87.143, 90.0, 92.449, 90.0], "dmin": 1.9}, "tags": ["h5", "monoclinic"]}, + {"id": "8ys9", "input": "8ys9/8YS9_xray-data_8ys9/data/SDS_09_18112_master.h5", "wavelength": 0.97957, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [71.04, 77.68, 83.16, 90.0, 90.0, 90.0], "dmin": 1.46}, "tags": ["h5", "orthorhombic"]}, + {"id": "9b22", "input": "9b22/KlpnC_20447_a_B1-AR6-AMP_9b22/data/PSL-1011_1580_master.h5", "wavelength": 0.97856, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [39.826, 92.659, 57.716, 90.0, 91.68, 90.0], "dmin": 1.3}, "tags": ["h5", "monoclinic"]}, + {"id": "9bn8", "input": "9bn8/EscoA_17938_a_AE1-UMA-A1AQS_9bn8/data/PSL-0401_8296_master.h5", "wavelength": 0.97856, "ref": {"sg": "P 41", "sgno": 76, "cell": [65.488, 65.488, 134.764, 90.0, 90.0, 90.0], "dmin": 1.35}, "tags": ["h5", "tetragonal"]}, + {"id": "9c18", "input": "9c18/BLVRB-2_5217_master.h5", "wavelength": 0.9201, "ref": {"sg": "P 1", "sgno": 1, "cell": [41.94, 41.977, 60.212, 84.11, 87.24, 63.66], "dmin": 1.9}, "tags": ["h5", "triclinic"]}, + {"id": "9chw", "input": "9chw/D1.001", "wavelength": 0.97872, "ref": {"sg": "P 61", "sgno": 169, "cell": [98.686, 98.686, 82.062, 90.0, 90.0, 120.0], "dmin": 2.16}, "tags": ["marCCD", "hexagonal"]}, + {"id": "9crw", "input": "9crw/data_9crw/data/Kip3-475-22-2_master.h5", "wavelength": 0.95299, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [83.96, 104.57, 118.78, 90.0, 93.37, 90.0], "dmin": 2.49}, "tags": ["h5", "monoclinic"]}, + {"id": "9e2t", "input": "9e2t/ga026f4b-1_5_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "P 1", "sgno": 1, "cell": [75.499, 78.082, 101.219, 94.63, 103.39, 114.55], "dmin": 2.28}, "tags": ["cbf", "triclinic"]}, + {"id": "9ea5", "input": "9ea5/K4_1_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [65.868, 73.064, 98.399, 90.0, 108.725, 90.0], "dmin": 2.0}, "tags": ["cbf", "monoclinic"]}, + {"id": "9fcg", "input": "9fcg/IBCH-15-p02x06_2_00001.cbf.gz", "wavelength": 0.97626, "ref": {"sg": "P 4", "sgno": 75, "cell": [87.79, 87.79, 35.633, 90.0, 90.0, 90.0], "dmin": 1.54}, "tags": ["cbf", "tetragonal"], "tiers": {"smoke": "cbf.gz reader, pure 4-fold"}}, + {"id": "9fhc", "input": "9fhc/te/te312_2_001.img", "wavelength": 1.0, "ref": {"sg": "I 2 3", "sgno": 197, "cell": [227.46, 227.46, 227.46, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["marCCD", "cubic"]}, + {"id": "9gdj", "input": "9gdj/slh34_w1_1_1_master.h5", "wavelength": 0.8856, "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [123.92, 123.92, 126.44, 90.0, 90.0, 90.0], "dmin": 1.47}, "tags": ["h5", "tetragonal", "cdte"]}, + {"id": "9gjx", "input": "9gjx/BlNTR_9gjx/data/SC6_1_master.h5", "wavelength": 0.9795, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [76.776, 115.807, 103.807, 90.0, 110.29, 90.0], "dmin": 2.4}, "tags": ["h5", "monoclinic"]}, + {"id": "9gqg", "input": "9gqg/MX-2555/FKBP51-MSP500-A-5522-11_1_3094759/FKBP51-MSP500-A-5522-11_1_1_master.h5", "wavelength": 0.87313, "ref": {"sg": "P 32 2 1", "sgno": 154, "cell": [48.157, 48.157, 188.036, 90.0, 90.0, 120.0], "dmin": 2.0}, "tags": ["h5", "trigonal"]}, + {"id": "9hnc", "input": "9hnc/asp_7_00001.cbf", "wavelength": 0.72931, "ref": {"sg": "P 1 2 1", "sgno": 3, "cell": [123.764, 123.645, 187.679, 90.0, 90.063, 90.0], "dmin": 1.879}, "tags": ["cbf", "monoclinic"]}, + {"id": "9hs7", "input": "9hs7/9hs7/data/B28X1_1_0001.cbf", "wavelength": 0.97926, "ref": {"sg": "P 65", "sgno": 170, "cell": [65.44, 65.44, 88.77, 90.0, 90.0, 120.0], "dmin": 1.698}, "tags": ["cbf", "hexagonal"]}, + {"id": "9i0a", "input": "9i0a/carm1END468_9i0a_tar_9I0A/data/X151B1_1_master.h5", "wavelength": 0.97857, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [75.237, 98.676, 208.551, 90.0, 90.0, 90.0], "dmin": 2.22}, "tags": ["h5", "orthorhombic"]}, + {"id": "9i80", "input": "9i80/20230422-PX1-LecA_RO5-5315-1/LecA-RO5_1_master.h5", "wavelength": 0.97856, "ref": {"sg": "P 41", "sgno": 76, "cell": [81.214, 81.214, 165.029, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["h5", "tetragonal", "twin"]}, + {"id": "9ig7", "input": "9ig7/9IG7/data/KOD-P596-G2_3_00001.cbf", "wavelength": 0.97626, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [111.472, 153.471, 69.03, 90.0, 90.0, 90.0], "dmin": 2.6}, "tags": ["cbf", "orthorhombic"]}, + {"id": "9ih9", "input": "9ih9/run_01_05_datacollection_9IH9/data/design-Xinjian-P1Thro-4_1_5_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [78.758, 133.933, 82.32, 90.0, 101.356, 90.0], "dmin": 1.7}, "tags": ["h5", "monoclinic"]}, + {"id": "9jq9", "input": "9jq9/pfplrxdata_9JQ9/data/lp13_001.cbf", "wavelength": 1.3477, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [48.551, 50.499, 78.571, 90.0, 90.0, 90.0], "dmin": 1.9}, "tags": ["cbf", "orthorhombic", "home-source"]}, + {"id": "9jzo", "input": "9jzo/JS_35P2_C3-002_diffraction_files_9JZO/data/JS_35P2_C3-002_0001.cbf", "wavelength": 0.97942, "ref": {"sg": "P 1", "sgno": 1, "cell": [41.63, 43.1, 54.2, 112.97, 90.11, 118.18], "dmin": 1.4}, "tags": ["cbf", "triclinic"], "tiers": {"smoke": "triclinic P1"}}, + {"id": "9khr", "input": "9khr/PfPlrx-DTT-9KHR/data-PfPlrx-DTT-9KHR/ods1467-lp13D1-01001.mccd", "wavelength": 0.97893, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [48.704, 50.31, 78.018, 90.0, 90.0, 90.0], "dmin": 2.0}, "tags": ["marCCD", "orthorhombic"], "tiers": {"smoke": "marCCD reader (.mccd), fast"}}, + {"id": "9mh4", "input": "9mh4/KlaeA_00150_a_B1_9mh4/data/PSL-0501_2719_master.h5", "wavelength": 0.97856, "ref": {"sg": "P 21 3", "sgno": 198, "cell": [138.65, 138.65, 138.65, 90.0, 90.0, 90.0], "dmin": 3.05}, "tags": ["h5", "cubic"]}, + {"id": "9min", "input": "9min/1151/A8_31_00001.cbf", "wavelength": 1.00003, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [95.45, 98.54, 155.74, 90.0, 90.0, 90.0], "dmin": 2.05}, "tags": ["cbf", "orthorhombic"], "tiers": {"smoke": "known hard: halved axis"}}, + {"id": "9o0h", "input": "9o0h/3TEL-GG-TNK1-UBA_9o0h/data/E1_1_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [55.15, 65.54, 112.9, 90.0, 90.0, 90.0], "dmin": 2.24}, "tags": ["cbf", "orthorhombic"]}, + {"id": "9p7q", "input": "9p7q/9p7q_9P7Q/data/SAMDC_MTA_273_40_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [96.978, 45.018, 72.061, 90.0, 105.109, 90.0], "dmin": 2.21}, "tags": ["cbf", "monoclinic"]}, + {"id": "9pbb", "input": "9pbb/9pbb_9PBB/data/SAMDC_MTA_293_30_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [97.418, 45.876, 72.248, 90.0, 104.971, 90.0], "dmin": 2.17}, "pinned": true, "tags": ["cbf", "monoclinic"]}, + {"id": "9q41", "input": "9q41/apoSPDS_1_master.h5", "wavelength": 0.8857, "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [118.568, 133.704, 82.369, 90.0, 90.0, 90.0], "dmin": 1.95}, "tags": ["h5", "orthorhombic"]}, + {"id": "9q66", "input": "9q66/PREP_JP417_D8a_12694_master.h5", "wavelength": 0.91976, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [105.853, 67.294, 158.019, 90.0, 99.109, 90.0], "dmin": 2.01}, "tags": ["h5", "monoclinic"]}, + {"id": "9qw8", "input": "9qw8/MX-2555/run_03_datacollection [2023-09-08 22:19:51]/FKBP12BRD4-5519-6-BD1-PPU688_w1_1_1_master.h5", "wavelength": 0.8856, "ref": {"sg": "P 1", "sgno": 1, "cell": [35.648, 35.65, 100.919, 86.458, 84.218, 72.475], "dmin": 1.8}, "tags": ["h5", "triclinic"]}, + {"id": "9rci", "input": "9rci/FEN1-CD029134_A09-3_AD049A-10_1_master.h5", "wavelength": 0.9801, "ref": {"sg": "P 1", "sgno": 1, "cell": [35.869, 39.297, 100.916, 98.3, 90.32, 90.09], "dmin": 1.662}, "ref_alternatives": [{"cell": [35.869, 39.297, 199.976, 87.087, 90.341, 90.09], "why": "The deposited cell is the (0,1/2,1/2)-centred sublattice of the supercell we report, to 0.17%, and both descriptions of this lattice are defensible. The Patterson has an off-origin peak at 62.5% of the origin, so a real translational NCS relates the two halves of our cell: describing the crystal by the doubled cell with the near-translation left in the content, or by its sublattice with the near-translation absorbed into the lattice, is a choice, not a measurement. The alternative cell is the deposited one doubled along c with the centring removed (c' = b + 2c), computed from the deposited cell alone - not from our output"}], "tags": ["h5", "triclinic", "tncs"]}, + {"id": "9rcs", "input": "9rcs/760_B7_x1_1_master.h5", "wavelength": 0.61992, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [70.01, 78.8, 82.34, 90.0, 88.64, 90.0], "dmin": 3.012}, "tags": ["h5", "monoclinic", "cdte"]}, + {"id": "9rp9", "input": "9rp9/data_9RP9/data/TAR148-TAR148_X2_1_master.h5", "wavelength": 0.97856, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [73.48, 59.78, 91.67, 90.0, 100.85, 90.0], "dmin": 2.1}, "tags": ["h5", "monoclinic"]}, + {"id": "9sl0", "input": "9sl0/wt-1_9sl0/data/design-Yan-HLA-WT-1_1_5_1_master.h5", "wavelength": 0.87313, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [60.181, 80.201, 111.589, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["h5", "orthorhombic"]}, + {"id": "9t6s", "input": "9t6s/CadC-AF6586_1_5_1_master.h5", "wavelength": 0.8856, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [63.004, 64.574, 102.705, 90.0, 90.0, 90.0], "dmin": 2.0}, "tags": ["h5", "orthorhombic", "tncs"]}, + {"id": "9upt", "input": "9upt/Raw data 1A/C12_7_00001.img", "wavelength": 0.99984, "ref": {"sg": "P 6", "sgno": 168, "cell": [158.27, 158.27, 53.962, 90.0, 90.0, 120.0], "dmin": 2.369}, "tags": ["SMV", "hexagonal"]}, + {"id": "9vyb", "input": "9vyb/Antitoxin Phd_9vyb/data/G7-5_4187_master.h5", "wavelength": 1.00003, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [44.37, 47.76, 48.35, 90.0, 90.0, 90.0], "dmin": 2.12}, "tags": ["h5", "orthorhombic"]}, + {"id": "9w3y", "input": "9w3y/9w3y/data/collect_01_00001_master.h5", "wavelength": 1.007, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [60.75, 69.98, 94.24, 90.0, 90.0, 90.0], "dmin": 1.5}, "tags": ["h5", "orthorhombic"]}, + {"id": "9yl4", "input": "9yl4/data_MJ4720/UWO0004_06_0001.cbf", "wavelength": 0.9795, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [95.837, 111.339, 402.967, 90.0, 90.0, 90.0], "dmin": 3.7}, "pinned": true, "tags": ["cbf", "orthorhombic"]}, + {"id": "9yzk", "input": "9yzk/CC1_21_TGGGAAGATACTGGATGAGGT_empty_9yzk/data/run_29_00001.cbf", "wavelength": 1.00002, "ref": {"sg": "I 1 2 1", "sgno": 5, "cell": [75.836, 163.022, 192.278, 90.0, 98.614, 90.0], "dmin": 5.1}, "tags": ["cbf", "monoclinic"], "tiers": {"smoke": "I2/C2 setting choice at low resolution"}}, + {"id": "9z44", "input": "9z44/CC1_10_CCCGGCCGG_Cclamp_9z44/data/run_41_00001.cbf", "wavelength": 1.00005, "ref": {"sg": "I 1 2 1", "sgno": 5, "cell": [73.491, 127.653, 141.183, 90.0, 91.984, 90.0], "dmin": 7.2}, "tags": ["cbf", "monoclinic"]}, + {"id": "9z72", "input": "9z72/A7_1_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [59.173, 59.173, 426.203, 90.0, 90.0, 120.0], "dmin": 2.38}, "tags": ["cbf", "trigonal", "long-axis"], "tiers": {"smoke": "long axis c=426 A, at the FFT ceiling"}}, + {"id": "9zlo", "input": "9zlo/ASP0887_09_0114_master.h5", "wavelength": 0.95365, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [38.39, 89.98, 106.97, 90.0, 90.0, 90.0], "dmin": 2.0}, "tags": ["h5", "orthorhombic"]}, + {"id": "9zm0", "input": "9zm0/Atg23ANNS_9zm0_9ZM0/data/SeAtg23_B_10_2115_master.h5", "wavelength": 0.97988, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [50.409, 30.101, 91.244, 90.0, 97.146, 90.0], "dmin": 2.1}, "tags": ["h5", "monoclinic"]}, + {"id": "9zmu", "input": "9zmu/BrmeA_18154_a_B2-Apo_9zmu/data/PSL-0613_7518_master.h5", "wavelength": 0.97856, "ref": {"sg": "P 65 2 2", "sgno": 179, "cell": [47.81, 47.81, 492.58, 90.0, 90.0, 120.0], "dmin": 1.98}, "tags": ["h5", "hexagonal"]}, + {"id": "cuhf2", "input": "cuhf2/03_CuHF2pyz2PF6b_P_O/CuHF2pyz2PF6b_P_O_01.nxs", "wavelength": 0.4859, "pinned": true, "tags": ["nxs", "small-molecule"]}, + {"id": "cytidine", "input": "cytidine/20151020-Cytidine-2th-30/fixed-omega--180-phi-scan01_00001.cbf", "wavelength": 0.68966, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [13.98, 14.788, 5.119, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "small-molecule"]}, + {"id": "dnba", "input": "dnba/35dnba_30K_2_04_00001.cbf", "wavelength": 0.6889, "ref": {"sg": "C 1 2/c 1", "sgno": 15, "cell": [20.2635, 8.7575, 9.6697, 90.0, 109.941, 90.0], "dmin": 0.48}, "tags": ["cbf", "monoclinic", "small-molecule"]}, + {"id": "lalanine", "input": "lalanine/pgw240050_01_00001.cbf", "wavelength": 0.6889, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [5.7952, 5.933, 12.362, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "small-molecule"], "tiers": {"smoke": "small molecule, miniCBF"}}, + {"id": "metformin", "input": "metformin/013_Mmetformin_01_master.h5", "wavelength": 0.4859, "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [7.9104, 13.8794, 7.931, 90.0, 114.606, 90.0], "dmin": 0.45}, "tags": ["h5", "monoclinic", "small-molecule"], "tiers": {"smoke": "small molecule, Diamond I19 NXmx master"}}, + {"id": "nidppe", "input": "nidppe/001_NiDppeCl2_01_master.h5", "wavelength": 0.4859, "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [11.2779, 13.3386, 15.8739, 90.0, 98.7953, 90.0], "dmin": 0.77}, "tags": ["h5", "monoclinic", "small-molecule"]}, + {"id": "5mln", "input": "5mln/5mln/data/CmADHx6_w1_2_0001.cbf", "wavelength": 0.8729, "ref": {"sg": "P 21 2 21", "sgno": 18, "cell": [74.178, 80.425, 80.52, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["cbf"]}, + {"id": "5t39", "input": "5t39/10mMfuc-12h.001", "wavelength": 0.97872, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [50.222, 41.27, 58.504, 90.0, 98.58, 90.0], "dmin": 1.1004}, "tags": ["marccd"]}, + {"id": "6cdl", "input": "6cdl/nnnn_6cdl/data/wt_32-14A_p6n6.0001", "wavelength": 1.0, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [58.26, 85.91, 46.051, 90.0, 90.0, 90.0], "dmin": 1.25}, "tags": ["marccd"]}, + {"id": "6f3p", "input": "6f3p/6f3p/data/saha16_1_1.0002", "wavelength": 1.0, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [142.9, 85.74, 112.01, 90.0, 122.2, 90.0], "dmin": 1.35}, "tags": ["marccd"]}, + {"id": "6g1f", "input": "6g1f/home/data/dls180217/mx13587-30/mat/DpgA-7-HA00AX9676/DpgA-7-HA00AX9676_1_0001.cbf", "wavelength": 0.97955, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [329.28, 83.9, 133.42, 90.0, 111.58, 90.0], "dmin": 2.248}, "tags": ["cbf"]}, + {"id": "6jgh", "input": "6jgh/6jgh/data/a_00001.img", "wavelength": 0.75, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [50.641, 62.506, 68.179, 90.0, 90.0, 90.0], "dmin": 0.94}, "tags": ["marccd"]}, + {"id": "6nen", "input": "6nen/Pm1tr_2_1_001.img", "wavelength": 0.9537, "ref": {"sg": "P 3 1 2", "sgno": 149, "cell": [105.501, 105.501, 35.132, 90.0, 90.0, 120.0], "dmin": 2.151}, "tags": ["smv"]}, + {"id": "6qaj", "input": "6qaj/95_8_8_8_1_0001.cbf", "wavelength": 1.28189, "ref": {"sg": "C 2 2 21", "sgno": 20, "cell": [59.774, 169.332, 374.508, 90.0, 90.0, 90.0], "dmin": 2.901}, "tags": ["cbf"]}, + {"id": "6s1u", "input": "6s1u/new_4_0001.img", "wavelength": 0.91841, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [51.603, 29.413, 85.533, 90.0, 103.75, 90.0], "dmin": 1.9}, "tags": ["marccd"]}, + {"id": "6w75", "input": "6w75/IDP51000_6W75/data/idp51000-410-a_1_1_1.001", "wavelength": 0.97872, "ref": {"sg": "P 32 2 1", "sgno": 154, "cell": [166.245, 166.245, 98.279, 90.0, 90.0, 120.0], "dmin": 1.951}, "tags": ["marccd"]}, + {"id": "6zqr", "input": "6zqr/ib23a11_M2S3_1_001.img", "wavelength": 0.9795, "ref": {"sg": "P 4", "sgno": 75, "cell": [113.6, 113.6, 44.08, 90.0, 90.0, 90.0], "dmin": 1.93}, "tags": ["smv"]}, + {"id": "6zqy", "input": "6zqy/ib22b32_MS_1_001.img", "wavelength": 0.9702, "ref": {"sg": "P 4", "sgno": 75, "cell": [119.29, 119.29, 44.21, 90.0, 90.0, 90.0], "dmin": 1.85}, "tags": ["smv"]}, + {"id": "6zr0", "input": "6zr0/ib23a23_dc_1_0001.cbf", "wavelength": 0.9795, "ref": {"sg": "P 4", "sgno": 75, "cell": [119.219, 119.219, 44.18, 90.0, 90.0, 90.0], "dmin": 1.94}, "tags": ["cbf"]}, + {"id": "7ou1", "input": "7ou1/omega_1_0001.img", "wavelength": 0.895, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [77.923, 91.308, 114.164, 90.0, 97.111, 90.0], "dmin": 1.65}, "tags": ["marccd"]}, + {"id": "7raa", "input": "7raa/A6_1_00001.cbf", "wavelength": 0.97946, "ref": {"sg": "P 43 21 2", "sgno": 96, "cell": [66.372, 66.372, 298.302, 90.0, 90.0, 90.0], "dmin": 2.69}, "tags": ["cbf"]}, + {"id": "9fcf", "input": "9fcf/IBCH-05-p03x03_3_00001.cbf.gz", "wavelength": 0.97626, "ref": {"sg": "P 4", "sgno": 75, "cell": [91.301, 91.301, 35.836, 90.0, 90.0, 90.0], "dmin": 2.36}, "tags": ["cbf"]}, + {"id": "9h0q", "input": "9h0q/Bc2lCnter-pma127_2_master.h5", "wavelength": 0.97857, "ref": {"sg": "H 3 2", "sgno": 155, "cell": [169.506, 169.506, 344.036, 90.0, 90.0, 120.0], "dmin": 2.55}, "tags": ["h5"]}, + {"id": "5ky6", "input": "5ky6/C11_1_001.img", "wavelength": 0.91841, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [84.511, 57.253, 164.016, 90.0, 102.57, 90.0], "dmin": 1.941}, "tags": ["marccd"]}, + {"id": "6z9g", "input": "6z9g/HASE01-ga1o2_w1_2_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [120.306, 93.815, 126.951, 90.0, 105.201, 90.0], "dmin": 1.76}, "tags": ["h5", "tncs"]} ]} -- 2.54.0 From 2c97e654ed26a5a6d8abb6d362ca1b647a337f3b Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 2 Oct 2026 16:27:52 +0200 Subject: [PATCH 04/97] Battery: add two L-cystine sweeps (in-house arm); say which open sets are pink beam lcystine_x10sa_20keV and lcystine_x10sa_25keV are scored against the literature structure only (hexagonal L-cystine, P 61 2 2, COD 5000005, Oughton & Harrison 1959): XDS indexes under 10% of the spots of either sweep, with the cell supplied and with the spot range cut to 150 frames, so there is no CORRECT.LP to take a reference from. On the current build the 25 keV sweep passes (P 61 2 2, cell within 0.5%) and the 20 keV sweep finds no lattice. EXTERNAL_TEST_DATA.md: none of the open-arm datasets is pink beam; every PDB record is single-wavelength (9Q41 used a multilayer monochromator). The pink-beam sets are the in-house ones. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/EXTERNAL_TEST_DATA.md | 6 ++++++ tools/battery/README.md | 2 +- tools/battery/inhouse.json | 4 +++- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/EXTERNAL_TEST_DATA.md b/docs/EXTERNAL_TEST_DATA.md index dd6dfc16b..6ae4eaed6 100644 --- a/docs/EXTERNAL_TEST_DATA.md +++ b/docs/EXTERNAL_TEST_DATA.md @@ -522,6 +522,12 @@ measurements. rotating anodes and one on a liquid-metal jet. - **Crystal system, from the deposited space group of the 174 PDB-coded rows:** orthorhombic 45, monoclinic 41, tetragonal 25, trigonal 21, hexagonal 17, cubic 13, triclinic 12. +- **Pink beam:** none of these datasets was collected with pink beam. All 174 PDB-coded rows + are deposited as `SINGLE WAVELENGTH` (`_diffrn_radiation.pdbx_diffrn_protocol`), and all + but 5REO, which leaves the field blank, as monochromatic (`pdbx_monochromatic_or_laue_m_l` + `M`); 9Q41 is the one row recorded with a multilayer rather than a crystal monochromator + (CHESS Rh/B4C). The battery's pink-beam data are in-house SLS measurements (tag `pink-beam` + in `tools/battery/inhouse.json`), not on this page. - **Long cell axes:** eleven PDB-coded rows have a deposited cell axis longer than 320 Å - 8V4O, 9ZMU, 9Z72, 9YL4, 5NW5, 6QAJ, 7QIJ, 8T7R, 9H0Q, 6G1F and 6OEL. diff --git a/tools/battery/README.md b/tools/battery/README.md index c851d972d..0ce5abcd1 100644 --- a/tools/battery/README.md +++ b/tools/battery/README.md @@ -15,7 +15,7 @@ run needed): `cd tools/battery && python3 test_score.py`. | arm | datasets | reference | manifest | |---|---|---|---| | **open** | public PDB depositions of raw diffraction data, plus a few published small-molecule sets | the deposited space group, cell and resolution | `open.json` (committed) | -| **inhouse** | standard test crystals measured at the SLS (lysozyme, thaumatin, insulin, cytochrome C, myoglobin), small-molecule standards (aspirin, citric acid, HEPES, YAG), plus no-crystal controls | XDS, from the `CORRECT.LP` beside each dataset; for the small molecules the space group is the literature's (`ref_override`), as XDS reports only Sohncke groups | `inhouse.json` (committed) | +| **inhouse** | standard test crystals measured at the SLS (lysozyme, thaumatin, insulin, cytochrome C, myoglobin), small-molecule standards (aspirin, citric acid, HEPES, YAG, L-cystine), plus no-crystal controls | XDS, from the `CORRECT.LP` beside each dataset; for the small molecules the space group is the literature's (`ref_override`), as XDS reports only Sohncke groups; L-cystine, which XDS does not index, has only the literature reference | `inhouse.json` (committed) | | **private** | user data | XDS, like inhouse | outside the repository; the local site config gives its path | Scoring checks these things in order, and the first one that fails decides the verdict: did it diff --git a/tools/battery/inhouse.json b/tools/battery/inhouse.json index 121c4c37c..a82530e26 100644 --- a/tools/battery/inhouse.json +++ b/tools/battery/inhouse.json @@ -42,5 +42,7 @@ {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "wavelength": 0.49594, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.55, "completeness": 86.6, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.03, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "wavelength": 0.61993, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.55, "completeness": 81.0, "r_meas": 0.04, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.043, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.39, "completeness": 91.6, "r_meas": 0.03, "cc_half": 0.999, "multiplicity": 5.64, "dmin_low": 2.0, "r_meas_low": 0.035, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, - {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"} + {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"}, + {"id": "lcystine_x10sa_20keV", "input": "lcystine_x10sa_20keV/lcystine_2_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"}, + {"id": "lcystine_x10sa_25keV", "input": "lcystine_x10sa_25keV/lcystine_2_003_master.h5", "wavelength": 0.49594, "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"} ]} -- 2.54.0 From a4bb6f74f1bebc6eabb7dfcc75af54f8884cd45e Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 09:46:19 +0200 Subject: [PATCH 05/97] XtalOptimizer: own Levenberg-Marquardt solver in place of Ceres The crystal refinement (XtalOptimizer, both the seven-block and the reduced beam+orientation form, and XtalOptimizerRotationOnly) no longer builds a ceres::Problem. XtalRefine holds the problem as data and solves it with LMSolver, which follows Ceres' trust-region LM step for step - Jacobi scaling, damping and radius updates, stopping rules, box projection, the projected Armijo line search with cubic interpolation on bounded problems, the SphereManifold for the spindle - but takes J^T J and J^T r directly instead of a Jacobian. The residual is the same XtalResidual code, now Ceres-free and evaluated on a forward-mode Dual (Dual.h); everything that depends on parameters alone (detector-angle trig, per-frame back-rotation, reciprocal basis, orientation rotation) is worked out once per evaluation, and the observed and predicted halves carry 6 and 9 derivative lanes rather than 16. The sums are cut into blocks that depend on the residual count alone, so the answer does not depend on the thread count. Because the line-search trial point is the candidate point, a bounded iteration costs one evaluation instead of Ceres' three. Validation (rc174 + this, -march=x86-64-v3): - p.mtz md5 identical to the Ceres build on myob/cytc/thau x10sa, GPU and CPU builds, and on the lyso8 stills reference. - Solve corpus (every 16-parameter solve and every 10th per-image solve of the three sets, 8.3k problems, inputs and Ceres results dumped from a run that reproduced the md5s): usable/failed agree on all, iteration counts identical on all, parameters agree to <2e-11 (in px / rad / 0.01 A units), costs to 1e-13. - Same process, same threads: 7-9x faster per solve than Ceres. - In-run (GPU, loaded box): xtal 16-parameter solves myob 22.2 -> 6.4 core-s, cytc 88 -> 26 core-s; per-image solves 5.4 -> 1.2 core-s (myob); cytc first pass indexing windows 1.9 -> 0.85 s, myob 1.1 -> 0.45 s; solver share of the whole cytc run 17% -> 4% of CPU samples. New tests compare the solver with Ceres on synthetic rotation problems (full/weighted/reduced) and check thread-count independence. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- THIRD_PARTY_NOTICES.md | 7 +- image_analysis/geom_refinement/CMakeLists.txt | 5 + image_analysis/geom_refinement/Dual.h | 142 +++++++ image_analysis/geom_refinement/LMSolver.cpp | 241 ++++++++++++ image_analysis/geom_refinement/LMSolver.h | 365 ++++++++++++++++++ .../geom_refinement/XtalOptimizer.cpp | 351 ++++++----------- .../geom_refinement/XtalOptimizer.h | 2 +- image_analysis/geom_refinement/XtalRefine.cpp | 334 ++++++++++++++++ image_analysis/geom_refinement/XtalRefine.h | 63 +++ image_analysis/geom_refinement/XtalResidual.h | 41 +- tests/CMakeLists.txt | 2 + tests/XtalRefineCeres.h | 92 +++++ tests/XtalRefineTest.cpp | 127 ++++++ 13 files changed, 1529 insertions(+), 243 deletions(-) create mode 100644 image_analysis/geom_refinement/Dual.h create mode 100644 image_analysis/geom_refinement/LMSolver.cpp create mode 100644 image_analysis/geom_refinement/LMSolver.h create mode 100644 image_analysis/geom_refinement/XtalRefine.cpp create mode 100644 image_analysis/geom_refinement/XtalRefine.h create mode 100644 tests/XtalRefineCeres.h create mode 100644 tests/XtalRefineTest.cpp diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 8dcb538ac..3122fff96 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -50,7 +50,7 @@ either way. Eigen is header-only: only its headers reach the binaries, and no Ei ## Vendored directly in the repository -These live in the source tree (see the path) rather than being fetched; traccc is the exception - code adapted into first-party files rather than a vendored directory, see the note at the end of this file. +These live in the source tree (see the path) rather than being fetched; traccc and the Ceres-derived minimiser are the exceptions - code adapted into first-party files rather than a vendored directory, see the notes at the end of this file. | Component | Path | Copyright | License (SPDX) | License text | |---|---|---|---|---| @@ -69,6 +69,7 @@ These live in the source tree (see the path) rather than being fetched; traccc i | [pocketfft](https://github.com/mreineck/pocketfft) | `gemmi_gph/gemmi/third_party/pocketfft_hdronly.h` | Max-Planck-Society; Peter Bell; MIT (FFTW-derived parts) | BSD-3-Clause | [pocketfft.txt](licenses/pocketfft.txt) | | [tinydir](https://github.com/cxong/tinydir) | `gemmi_gph/gemmi/third_party/tinydir.h` | Cong Xu, Lautis Sun, Baudouin Feildel, Andargor | BSD-2-Clause | [tinydir.txt](licenses/tinydir.txt) | | [traccc (ACTS)](https://github.com/acts-project/traccc) | `image_analysis/spot_finding/StrongPixelSet.cpp`, `SpotExtractorGPU.cu` | CERN, for the benefit of the ACTS project | MPL-2.0 | [traccc.txt](licenses/traccc.txt) | +| [Ceres Solver](https://github.com/ceres-solver/ceres-solver) (adapted) | `image_analysis/geom_refinement/LMSolver.h`, `LMSolver.cpp` | Google Inc. | BSD-3-Clause | [ceres-solver.txt](licenses/ceres-solver.txt) | | [xbflash.qspi](https://github.com/Xilinx/XRT) | `tools/xbflash.qspi/` | Xilinx / AMD | Apache-2.0 | [xbflash-qspi.txt](licenses/xbflash-qspi.txt) | | [wingetopt](https://github.com/alex85k/wingetopt) | `tools/wingetopt/` | Todd C. Miller; The NetBSD Foundation | ISC AND BSD-2-Clause | [wingetopt.txt](licenses/wingetopt.txt) | @@ -107,6 +108,10 @@ served frontend, so the shipped web UI carries its own attribution. and `SpotExtractorGPU.cu` follows the design of its GPU counterpart. MPL-2.0 is file-level, so both files name the origin at the top and are covered by `licenses/traccc.txt`. See [ACKNOWLEDGEMENT.md](docs/ACKNOWLEDGEMENT.md) for the citation. +* **Ceres Solver** is also fetched and linked (table above); separately, `LMSolver.h`/`.cpp` re-implement + its trust-region Levenberg-Marquardt minimiser, projected line search, polynomial step choice and + sphere manifold for the crystal refinement, following its source. Both files name the origin at the + top and are covered by `licenses/ceres-solver.txt`. * **FFTW** is GPL-2.0-or-later — compatible with, and absorbed by, this project's GPL-3.0 license. * **Apache-2.0** components: where upstream ships a `NOTICE` file, it is reproduced in the corresponding `licenses/` text. diff --git a/image_analysis/geom_refinement/CMakeLists.txt b/image_analysis/geom_refinement/CMakeLists.txt index b004e8d4e..2330dbb93 100644 --- a/image_analysis/geom_refinement/CMakeLists.txt +++ b/image_analysis/geom_refinement/CMakeLists.txt @@ -24,6 +24,11 @@ ADD_LIBRARY(JFJochGeomRefinement STATIC XtalOptimizer.cpp XtalOptimizer.h XtalResidual.h + XtalRefine.cpp + XtalRefine.h + LMSolver.cpp + LMSolver.h + Dual.h PostRefine.cpp PostRefine.h GeometryRefiner.cpp diff --git a/image_analysis/geom_refinement/Dual.h b/image_analysis/geom_refinement/Dual.h new file mode 100644 index 000000000..932828058 --- /dev/null +++ b/image_analysis/geom_refinement/Dual.h @@ -0,0 +1,142 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +// A forward-mode dual number with N derivative lanes: a value and its gradient with respect to N +// parameters. The residuals of the crystal refinement are written as templates over their scalar type, +// so the same code runs on a plain double and on this. The value part of every operation is the plain +// double arithmetic of the same expression - written the way ceres::Jet writes it, division through the +// reciprocal - so a residual evaluated on a Dual has the same value as on a Jet. + +#include +#include + +#include + +template +struct Dual { + double a = 0.0; + double v[N] = {}; + + Dual() = default; + Dual(double value) : a(value) {} // NOLINT: implicit, a constant is a dual with zero derivatives + + static Dual Variable(double value, int lane) { + Dual d(value); + d.v[lane] = 1.0; + return d; + } + + Dual &operator+=(const Dual &o) { a += o.a; for (int i = 0; i < N; i++) v[i] += o.v[i]; return *this; } + Dual &operator-=(const Dual &o) { a -= o.a; for (int i = 0; i < N; i++) v[i] -= o.v[i]; return *this; } + Dual &operator*=(const Dual &o) { *this = *this * o; return *this; } + Dual &operator/=(const Dual &o) { *this = *this / o; return *this; } + + friend Dual operator+(const Dual &x) { return x; } + friend Dual operator-(const Dual &x) { + Dual r(-x.a); + for (int i = 0; i < N; i++) r.v[i] = -x.v[i]; + return r; + } + + friend Dual operator+(const Dual &x, const Dual &y) { + Dual r(x.a + y.a); + for (int i = 0; i < N; i++) r.v[i] = x.v[i] + y.v[i]; + return r; + } + friend Dual operator+(const Dual &x, double s) { Dual r = x; r.a += s; return r; } + friend Dual operator+(double s, const Dual &x) { Dual r = x; r.a += s; return r; } + + friend Dual operator-(const Dual &x, const Dual &y) { + Dual r(x.a - y.a); + for (int i = 0; i < N; i++) r.v[i] = x.v[i] - y.v[i]; + return r; + } + friend Dual operator-(const Dual &x, double s) { Dual r = x; r.a -= s; return r; } + friend Dual operator-(double s, const Dual &x) { + Dual r(s - x.a); + for (int i = 0; i < N; i++) r.v[i] = -x.v[i]; + return r; + } + + friend Dual operator*(const Dual &x, const Dual &y) { + Dual r(x.a * y.a); + for (int i = 0; i < N; i++) r.v[i] = x.a * y.v[i] + x.v[i] * y.a; + return r; + } + friend Dual operator*(const Dual &x, double s) { + Dual r(x.a * s); + for (int i = 0; i < N; i++) r.v[i] = x.v[i] * s; + return r; + } + friend Dual operator*(double s, const Dual &x) { return x * s; } + + friend Dual operator/(const Dual &x, const Dual &y) { + const double y_inv = 1.0 / y.a; + const double q = x.a * y_inv; + Dual r(q); + for (int i = 0; i < N; i++) r.v[i] = (x.v[i] - q * y.v[i]) * y_inv; + return r; + } + friend Dual operator/(const Dual &x, double s) { + const double s_inv = 1.0 / s; + return x * s_inv; + } + friend Dual operator/(double s, const Dual &y) { + const double y_inv = 1.0 / y.a; + const double d = -s * y_inv * y_inv; + Dual r(s * y_inv); + for (int i = 0; i < N; i++) r.v[i] = d * y.v[i]; + return r; + } + + friend bool operator<(const Dual &x, const Dual &y) { return x.a < y.a; } + friend bool operator>(const Dual &x, const Dual &y) { return x.a > y.a; } + friend bool operator<=(const Dual &x, const Dual &y) { return x.a <= y.a; } + friend bool operator>=(const Dual &x, const Dual &y) { return x.a >= y.a; } + friend bool operator==(const Dual &x, const Dual &y) { return x.a == y.a; } + friend bool operator!=(const Dual &x, const Dual &y) { return x.a != y.a; } + + // The chain rule for a function of one argument: value f, derivative df. + Dual Chain(double f, double df) const { + Dual r(f); + for (int i = 0; i < N; i++) r.v[i] = df * v[i]; + return r; + } + + friend Dual sqrt(const Dual &x) { + const double s = std::sqrt(x.a); + return x.Chain(s, 0.5 / s); + } + friend Dual cos(const Dual &x) { return x.Chain(std::cos(x.a), -std::sin(x.a)); } + friend Dual sin(const Dual &x) { return x.Chain(std::sin(x.a), std::cos(x.a)); } + friend Dual hypot(const Dual &x, const Dual &y, const Dual &z) { + // As ceres::hypot(Jet, Jet, Jet): the value is std::hypot, the derivative x/h dx + y/h dy + z/h dz. + const double h = std::hypot(x.a, y.a, z.a); + Dual r(h); + for (int i = 0; i < N; i++) r.v[i] = x.a / h * x.v[i] + y.a / h * y.v[i] + z.a / h * z.v[i]; + return r; + } + friend int fpclassify(const Dual &x) { return std::fpclassify(x.a); } +}; + +// What Eigen needs to hold a Dual in a fixed-size matrix (the reciprocal basis is built in one). +namespace Eigen { + template + struct NumTraits> : GenericNumTraits { + typedef Dual Real; + typedef Dual NonInteger; + typedef Dual Nested; + typedef Dual Literal; + enum { + IsComplex = 0, IsInteger = 0, IsSigned = 1, RequireInitialization = 1, + ReadCost = 1, AddCost = 1, MulCost = 1 + }; + static inline Real epsilon() { return Real(std::numeric_limits::epsilon()); } + static inline Real dummy_precision() { return Real(1e-12); } + static inline Real highest() { return Real(std::numeric_limits::max()); } + static inline Real lowest() { return Real(-std::numeric_limits::max()); } + static inline int digits10() { return NumTraits::digits10(); } + }; +} diff --git a/image_analysis/geom_refinement/LMSolver.cpp b/image_analysis/geom_refinement/LMSolver.cpp new file mode 100644 index 000000000..967ae965e --- /dev/null +++ b/image_analysis/geom_refinement/LMSolver.cpp @@ -0,0 +1,241 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +// Adapted from https://github.com/ceres-solver/ceres-solver (internal/ceres/polynomial.cc, +// include/ceres/internal/sphere_manifold_functions.h, householder_vector.h) +// Copyright 2023 Google Inc. All rights reserved. +// BSD-3-Clause, see licenses/ceres-solver.txt + +#include "LMSolver.h" + +#include + +namespace { + void HouseholderVector3(const double x[3], double v[3], double &beta) { + const double sigma = x[0] * x[0] + x[1] * x[1]; + v[0] = x[0]; + v[1] = x[1]; + v[2] = 1.0; + beta = 0.0; + const double x_pivot = x[2]; + if (sigma <= std::numeric_limits::epsilon()) { + if (x_pivot < 0.0) + beta = 2.0; + return; + } + const double mu = std::sqrt(x_pivot * x_pivot + sigma); + const double v_pivot = (x_pivot <= 0.0) ? x_pivot - mu : -sigma / (x_pivot + mu); + beta = 2.0 * v_pivot * v_pivot / (sigma + v_pivot * v_pivot); + v[0] /= v_pivot; + v[1] /= v_pivot; + } + + double Norm3(const double x[3]) { + return std::sqrt(x[0] * x[0] + x[1] * x[1] + x[2] * x[2]); + } + + using Vector = Eigen::VectorXd; + using Matrix = Eigen::MatrixXd; + + double EvaluatePolynomial(const Vector &polynomial, double x) { + double v = 0.0; + for (int i = 0; i < polynomial.size(); ++i) + v = v * x + polynomial(i); + return v; + } + + void BalanceCompanionMatrix(Matrix &companion_matrix) { + Matrix offdiagonal = companion_matrix; + offdiagonal.diagonal().setZero(); + const int degree = static_cast(companion_matrix.rows()); + const double gamma = 0.9; + bool scaling_has_changed; + do { + scaling_has_changed = false; + for (int i = 0; i < degree; ++i) { + const double col_norm = offdiagonal.col(i).lpNorm<1>(); + if (std::fpclassify(col_norm) != FP_ZERO) { + const double row_norm = offdiagonal.row(i).lpNorm<1>(); + int exponent = 0; + std::frexp(row_norm / col_norm, &exponent); + exponent /= 2; + if (exponent != 0) { + const double scaled_col_norm = std::ldexp(col_norm, exponent); + const double scaled_row_norm = std::ldexp(row_norm, -exponent); + if (scaled_col_norm + scaled_row_norm < gamma * (col_norm + row_norm)) { + scaling_has_changed = true; + offdiagonal.row(i) *= std::ldexp(1.0, -exponent); + offdiagonal.col(i) *= std::ldexp(1.0, exponent); + } + } + } + } + } while (scaling_has_changed); + offdiagonal.diagonal() = companion_matrix.diagonal(); + companion_matrix = offdiagonal; + } + + // Real parts of the roots, as Ceres' FindPolynomialRoots (the imaginary parts are not used here). + bool FindPolynomialRoots(const Vector &polynomial_in, Vector &real) { + if (polynomial_in.size() == 0) + return false; + int lead = 0; + while (lead < polynomial_in.size() - 1 && polynomial_in(lead) == 0.0) + ++lead; + Vector polynomial = polynomial_in.tail(polynomial_in.size() - lead); + const int degree = static_cast(polynomial.size()) - 1; + if (degree == 0) { + real.resize(0); + return true; + } + if (degree == 1) { + real.resize(1); + real(0) = -polynomial(1) / polynomial(0); + return true; + } + if (degree == 2) { + const double a = polynomial(0); + const double b = polynomial(1); + const double c = polynomial(2); + const double D = b * b - 4 * a * c; + const double sqrt_D = std::sqrt(std::fabs(D)); + real.setZero(2); + if (D >= 0) { + if (b >= 0) { + real(0) = (-b - sqrt_D) / (2.0 * a); + real(1) = (2.0 * c) / (-b - sqrt_D); + } else { + real(0) = (2.0 * c) / (-b + sqrt_D); + real(1) = (-b + sqrt_D) / (2.0 * a); + } + } else { + real(0) = -b / (2.0 * a); + real(1) = -b / (2.0 * a); + } + return true; + } + polynomial /= polynomial(0); + Matrix companion = Matrix::Zero(degree, degree); + companion.diagonal(-1).setOnes(); + companion.col(degree - 1) = -polynomial.reverse().head(degree); + BalanceCompanionMatrix(companion); + Eigen::EigenSolver solver(companion, false); + if (solver.info() != Eigen::Success) + return false; + real = solver.eigenvalues().real(); + return true; + } + + void MinimizePolynomial(const Vector &polynomial, double x_min, double x_max, + double &optimal_x, double &optimal_value) { + optimal_x = (x_min + x_max) / 2.0; + optimal_value = EvaluatePolynomial(polynomial, optimal_x); + const double x_min_value = EvaluatePolynomial(polynomial, x_min); + if (x_min_value < optimal_value) { + optimal_value = x_min_value; + optimal_x = x_min; + } + const double x_max_value = EvaluatePolynomial(polynomial, x_max); + if (x_max_value < optimal_value) { + optimal_value = x_max_value; + optimal_x = x_max; + } + if (polynomial.rows() <= 2) + return; + + const int degree = static_cast(polynomial.rows()) - 1; + Vector derivative(degree); + for (int i = 0; i < degree; ++i) + derivative(i) = (degree - i) * polynomial(i); + Vector roots_real; + if (!FindPolynomialRoots(derivative, roots_real)) + return; + for (int i = 0; i < roots_real.rows(); ++i) { + const double root = roots_real(i); + if (root < x_min || root > x_max) + continue; + const double value = EvaluatePolynomial(polynomial, root); + if (value < optimal_value) { + optimal_value = value; + optimal_x = root; + } + } + } + + Vector FindInterpolatingPolynomial(const std::vector &samples) { + int num_constraints = 0; + for (const auto &s: samples) + num_constraints += (s.value_is_valid ? 1 : 0) + (s.gradient_is_valid ? 1 : 0); + const int degree = num_constraints - 1; + Matrix lhs = Matrix::Zero(num_constraints, num_constraints); + Vector rhs = Vector::Zero(num_constraints); + int row = 0; + for (const auto &s: samples) { + if (s.value_is_valid) { + for (int j = 0; j <= degree; ++j) + lhs(row, j) = std::pow(s.x, degree - j); + rhs(row) = s.value; + ++row; + } + if (s.gradient_is_valid) { + for (int j = 0; j < degree; ++j) + lhs(row, j) = (degree - j) * std::pow(s.x, degree - j - 1); + rhs(row) = s.gradient; + ++row; + } + } + Eigen::FullPivLU lu(lhs); + return lu.setThreshold(0.0).solve(rhs); + } +} + +void SpherePlus3(const double x[3], const double delta[2], double out[3]) { + const double norm_delta = std::sqrt(delta[0] * delta[0] + delta[1] * delta[1]); + if (norm_delta == 0.0) { + out[0] = x[0]; + out[1] = x[1]; + out[2] = x[2]; + return; + } + double v[3], beta; + HouseholderVector3(x, v, beta); + const double sin_delta_by_delta = std::sin(norm_delta) / norm_delta; + const double y[3] = {sin_delta_by_delta * delta[0], sin_delta_by_delta * delta[1], std::cos(norm_delta)}; + const double vy = v[0] * y[0] + v[1] * y[1] + v[2] * y[2]; + const double x_norm = Norm3(x); + for (int i = 0; i < 3; i++) + out[i] = x_norm * (y[i] - v[i] * (beta * vy)); +} + +void SpherePlusJacobian3(const double x[3], double jacobian[3][2]) { + double v[3], beta; + HouseholderVector3(x, v, beta); + const double x_norm = Norm3(x); + for (int i = 0; i < 2; ++i) + for (int r = 0; r < 3; ++r) + jacobian[r][i] = (-beta * v[i] * v[r] + (r == i ? 1.0 : 0.0)) * x_norm; +} + +double LMInterpolatedStepSize(const LMLineSample &lowerbound, const LMLineSample &previous, + const LMLineSample ¤t, double min_step, double max_step) { + if (!current.value_is_valid) + return std::min(std::max(current.x * 0.5, min_step), max_step); + + std::vector samples{lowerbound, current}; + if (previous.value_is_valid) + samples.push_back(previous); + + const Vector polynomial = FindInterpolatingPolynomial(samples); + double step = 0.0, value = 0.0; + MinimizePolynomial(polynomial, min_step, max_step, step, value); + for (const auto &s: samples) { + if (s.x < min_step || s.x > max_step) + continue; + const double v = EvaluatePolynomial(polynomial, s.x); + if (v < value) { + step = s.x; + value = v; + } + } + return step; +} diff --git a/image_analysis/geom_refinement/LMSolver.h b/image_analysis/geom_refinement/LMSolver.h new file mode 100644 index 000000000..ee3b9657d --- /dev/null +++ b/image_analysis/geom_refinement/LMSolver.h @@ -0,0 +1,365 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +// The minimiser below follows Ceres Solver's trust-region Levenberg-Marquardt step for step - its +// options, its Jacobi scaling, its damping and radius updates, its stopping rules, its box projection, +// its projected Armijo line search on bounded problems and its SphereManifold - so that a problem moved +// off Ceres takes the same path to the same answer. Adapted from +// https://github.com/ceres-solver/ceres-solver (internal/ceres/trust_region_minimizer.cc, +// levenberg_marquardt_strategy.cc, trust_region_step_evaluator.cc, line_search.cc, polynomial.cc, +// include/ceres/internal/sphere_manifold_functions.h, householder_vector.h) +// Copyright 2023 Google Inc. All rights reserved. +// BSD-3-Clause, see licenses/ceres-solver.txt +// +// What it does NOT take from Ceres is the Jacobian: the caller hands over J^T J and J^T r directly, +// accumulated however it likes, and the minimiser never sees a row of J. Everything Ceres computes from +// the Jacobian - the column norms, the normal equations, the model cost change - is a function of those +// two alone. Only the options the crystal refinements use are reproduced: monotonic steps, no inner +// iterations, a dense Cholesky of the normal equations. + +#pragma once + +#include +#include +#include +#include + +#include + +struct LMBlock { + int offset = 0; // into the ambient parameter vector + int size = 0; // ambient size, at most 3 + bool constant = false; + bool sphere = false; // Ceres' SphereManifold: the norm is kept, the tangent has size - 1 coordinates + double lower[3] = {-std::numeric_limits::max(), -std::numeric_limits::max(), + -std::numeric_limits::max()}; + double upper[3] = {std::numeric_limits::max(), std::numeric_limits::max(), + std::numeric_limits::max()}; + + int TangentSize() const { return constant ? 0 : (sphere ? size - 1 : size); } +}; + +struct LMOptions { + int max_iterations = 50; + double max_time_s = 1e9; +}; + +enum class LMTermination { Convergence, NoConvergence, Failure }; + +struct LMSummary { + LMTermination termination = LMTermination::Failure; + // Iterations as Ceres counts them in Summary::iterations: iteration 0 included. + int iterations = 0; + int evaluations = 0; + int line_search_steps = 0; + double initial_cost = 0.0; + double final_cost = 0.0; + + bool IsSolutionUsable() const { return termination != LMTermination::Failure; } +}; + +// Ceres' SphereManifold<3> for a three-vector: the Householder reflection that takes x to the pole, the +// Plus that walks a tangent step along the sphere, and the 3x2 Jacobian of that Plus at zero step. +void SpherePlus3(const double x[3], const double delta[2], double out[3]); +void SpherePlusJacobian3(const double x[3], double jacobian[3][2]); + +// The polynomial step-size choice of Ceres' Armijo line search, cubic interpolation. +struct LMLineSample { + double x = 0.0; + double value = 0.0; + double gradient = 0.0; + bool value_is_valid = false; + bool gradient_is_valid = false; +}; +double LMInterpolatedStepSize(const LMLineSample &lowerbound, const LMLineSample &previous, + const LMLineSample ¤t, double min_step, double max_step); + +// Evaluate is called as eval(x, cost, g, H): x the ambient parameters, cost 1/2 sum of squared +// residuals, and - where g and H are not null - the gradient J^T r and J^T J in the TANGENT coordinates +// of the non-constant blocks, in block order. It returns false where anything came out non-finite. +// x is updated in place on success; it is left untouched on failure. +template +LMSummary SolveLM(std::vector &x_io, const std::vector &blocks, const LMOptions &options, + Evaluate &&eval) { + using Vec = Eigen::VectorXd; + using Mat = Eigen::MatrixXd; + constexpr double kMax = std::numeric_limits::max(); + + // Ceres' defaults, which is what the callers always ran with. + constexpr double initial_radius = 1e4; + constexpr double max_radius = 1e16; + constexpr double min_radius = 1e-32; + constexpr double min_relative_decrease = 1e-3; + constexpr double min_lm_diagonal = 1e-6; + constexpr double max_lm_diagonal = 1e32; + constexpr int max_consecutive_invalid_steps = 5; + constexpr double function_tolerance = 1e-6; + constexpr double gradient_tolerance = 1e-10; + constexpr double parameter_tolerance = 1e-8; + constexpr double sufficient_decrease = 1e-4; + constexpr double max_step_contraction = 1e-3; + constexpr double min_step_contraction = 0.6; + constexpr double min_line_search_step = 1e-9; + constexpr int max_line_search_iterations = 20; + + const auto start = std::chrono::steady_clock::now(); + LMSummary summary; + + int n = 0; + bool constrained = false; + for (const auto &b: blocks) { + for (int j = 0; j < b.size; j++) + if (!std::isfinite(x_io[b.offset + j])) + return summary; + n += b.TangentSize(); + for (int j = 0; j < b.size; j++) { + if (b.constant) { + if (x_io[b.offset + j] < b.lower[j] || x_io[b.offset + j] > b.upper[j]) + return summary; + } else { + if (b.lower[j] >= b.upper[j]) + return summary; + if (b.lower[j] > -kMax || b.upper[j] < kMax) + constrained = true; + } + } + } + + // x (+) delta, block by block, projected onto the bounds - Ceres' ParameterBlock::Plus. + const auto plus = [&](const Vec &x, const Vec &delta, Vec &out) { + out = x; + int t = 0; + for (const auto &b: blocks) { + if (b.constant) + continue; + if (b.sphere) { + SpherePlus3(x.data() + b.offset, delta.data() + t, out.data() + b.offset); + } else { + for (int j = 0; j < b.size; j++) + out[b.offset + j] = x[b.offset + j] + delta[t + j]; + } + for (int j = 0; j < b.size; j++) { + out[b.offset + j] = std::max(out[b.offset + j], b.lower[j]); + out[b.offset + j] = std::min(out[b.offset + j], b.upper[j]); + } + t += b.TangentSize(); + } + }; + // Norms over the parameters Ceres keeps in its state: the non-constant blocks only. + const auto free_norm = [&](const Vec &v) { + double s = 0.0; + for (const auto &b: blocks) + if (!b.constant) + for (int j = 0; j < b.size; j++) + s += v[b.offset + j] * v[b.offset + j]; + return std::sqrt(s); + }; + const auto free_max_norm = [&](const Vec &v) { + double m = 0.0; + for (const auto &b: blocks) + if (!b.constant) + for (int j = 0; j < b.size; j++) + m = std::max(m, std::fabs(v[b.offset + j])); + return m; + }; + + Vec x = Eigen::Map(x_io.data(), static_cast(x_io.size())); + if (constrained) { + Vec projected; + plus(x, Vec::Zero(n), projected); + x = projected; + } + + // One evaluation point with everything Ceres computes there. + struct Point { + Vec x; + double cost = kMax; + Vec g; + Mat H; + bool valid = false; + }; + const auto evaluate = [&](const Vec &at, Point &p) { + p.x = at; + p.g.setZero(n); + p.H.setZero(n, n); + summary.evaluations++; + p.valid = eval(at.data(), p.cost, &p.g, &p.H) && std::isfinite(p.cost); + if (!p.valid) + p.cost = kMax; + }; + + Point cur; + evaluate(x, cur); + if (!cur.valid) + return summary; + summary.initial_cost = cur.cost; + + // Jacobi scaling, fixed from the Jacobian at the starting point. + Vec scale(n); + for (int i = 0; i < n; i++) + scale[i] = 1.0 / (1.0 + std::sqrt(cur.H(i, i))); + + Vec gs, neg_g, projected; + Mat Hs; + double gradient_max_norm = 0.0; + const auto take_point = [&]() { + gs = scale.cwiseProduct(cur.g); + Hs = scale.asDiagonal() * cur.H * scale.asDiagonal(); + neg_g = -cur.g; + plus(cur.x, neg_g, projected); + gradient_max_norm = free_max_norm(cur.x - projected); + }; + take_point(); + + double radius = initial_radius; + double decrease_factor = 2.0; + bool reuse_diagonal = false; + Vec diagonal(n); + int consecutive_invalid = 0; + bool any_successful_step = false; + bool step_successful = true; // iteration 0 + int iteration = 0; + + Point trial; // the last point the line search evaluated, reused as the candidate when it is one + + const auto step_rejected = [&]() { + radius = radius / decrease_factor; + decrease_factor *= 2.0; + reuse_diagonal = true; + }; + + const auto finish = [&](LMTermination t) { + summary.termination = t; + summary.final_cost = cur.cost; + if (t != LMTermination::Failure) + for (int i = 0; i < x.size(); i++) + x_io[i] = cur.x[i]; + return summary; + }; + + for (;;) { + // FinalizeIterationAndCheckIfMinimizerCanContinue + summary.iterations++; + if (std::chrono::duration(std::chrono::steady_clock::now() - start).count() + >= options.max_time_s) + return finish(LMTermination::NoConvergence); + if (iteration >= options.max_iterations) + return finish(LMTermination::NoConvergence); + if (step_successful && gradient_max_norm <= gradient_tolerance) + return finish(LMTermination::Convergence); + if (radius <= min_radius) + return finish(LMTermination::Convergence); + + iteration++; + step_successful = false; + + // ComputeTrustRegionStep: the damped normal equations of the scaled Jacobian. + if (!reuse_diagonal) + for (int i = 0; i < n; i++) + diagonal[i] = std::min(std::max(Hs(i, i), min_lm_diagonal), max_lm_diagonal); + Mat lhs = Hs; + for (int i = 0; i < n; i++) { + const double d = std::sqrt(diagonal[i] / radius); + lhs(i, i) += d * d; + } + reuse_diagonal = true; + Eigen::LLT llt(lhs); + bool step_valid = false; + Vec step; + double model_cost_change = 0.0; + if (llt.info() == Eigen::Success) { + step = -llt.solve(gs); + if (step.allFinite()) { + model_cost_change = -(step.dot(gs) + 0.5 * step.dot(Hs * step)); + step_valid = model_cost_change > 0.0; + } + } + if (!step_valid) { + if (++consecutive_invalid >= max_consecutive_invalid_steps) + return finish(LMTermination::Failure); + step_rejected(); + continue; + } + consecutive_invalid = 0; + Vec delta = step.cwiseProduct(scale); + + bool have_trial = false; + if (constrained) { + // Projected Armijo line search along delta, cubic interpolation. + const double initial_gradient = cur.g.dot(delta); + const double direction_max_norm = delta.lpNorm(); + LMLineSample initial{0.0, cur.cost, initial_gradient, true, true}; + LMLineSample previous, current; + const auto line_eval = [&](double alpha, LMLineSample &s) { + s = LMLineSample{}; + s.x = alpha; + Vec moved; + plus(cur.x, Vec(alpha * delta), moved); + evaluate(moved, trial); + have_trial = true; + if (!trial.valid) + return; + s.value = trial.cost; + s.value_is_valid = true; + s.gradient = delta.dot(trial.g); + s.gradient_is_valid = std::isfinite(s.gradient); + }; + line_eval(1.0, current); + bool success = true; + int ls_iterations = 0; + while (!current.value_is_valid + || current.value > initial.value + sufficient_decrease * initial_gradient * current.x) { + ++ls_iterations; + if (ls_iterations >= max_line_search_iterations) { + success = false; + break; + } + const double alpha = LMInterpolatedStepSize(initial, previous, current, + max_step_contraction * current.x, + min_step_contraction * current.x); + if (alpha * direction_max_norm < min_line_search_step) { + success = false; + break; + } + previous = current; + line_eval(alpha, current); + } + summary.line_search_steps += ls_iterations; + if (success) + delta *= current.x; + } + + // ComputeCandidatePointAndEvaluateCost + Vec candidate_x; + plus(cur.x, delta, candidate_x); + Point cand; + if (have_trial && trial.x == candidate_x) + cand = std::move(trial); + else + evaluate(candidate_x, cand); + + if (any_successful_step) { + const double step_norm = free_norm(cur.x - cand.x); + if (step_norm <= parameter_tolerance * (free_norm(cur.x) + parameter_tolerance)) + return finish(LMTermination::Convergence); + } + if (std::fabs(cur.cost - cand.cost) <= function_tolerance * cur.cost) + return finish(LMTermination::Convergence); + + const double relative_decrease = (cand.cost >= kMax) + ? std::numeric_limits::lowest() + : (cur.cost - cand.cost) / model_cost_change; + if (relative_decrease > min_relative_decrease) { + any_successful_step = true; + step_successful = true; + cur = std::move(cand); + take_point(); + radius = radius / std::max(1.0 / 3.0, 1.0 - std::pow(2.0 * relative_decrease - 1.0, 3)); + radius = std::min(max_radius, radius); + decrease_factor = 2.0; + reuse_diagonal = false; + } else { + step_rejected(); + } + } +} diff --git a/image_analysis/geom_refinement/XtalOptimizer.cpp b/image_analysis/geom_refinement/XtalOptimizer.cpp index 58b049e10..23e671a2f 100644 --- a/image_analysis/geom_refinement/XtalOptimizer.cpp +++ b/image_analysis/geom_refinement/XtalOptimizer.cpp @@ -7,84 +7,11 @@ #include "XtalOptimizer.h" #include "XtalResidual.h" -#include "ceres/ceres.h" +#include "XtalRefine.h" #include "ceres/rotation.h" +#include "Dual.h" #include "LatticeReduction.h" -// Soft header prior on ONE beam-centre component (the spindle-parallel, gauge-weak one). Residual = w*(b - b0); -// the caller sets w so the prior behaves like a sigma-pixel restraint that competes with the (unit-weight) -// positional residuals - strong enough to pin the gauge direction, negligible in the well-constrained one. -// Soft restraint on one direction of a two-component block: g.(p - p0), weighted. Used for the beam -// centre and for the detector tilt, which are the same gauge seen twice (see the gauge block below), -// so they take the same direction g and cannot disagree about it. -struct GaugeDirectionPrior { - GaugeDirectionPrior(double gx, double gy, double p0, double weight) - : gx(gx), gy(gy), p0(p0), weight(weight) {} - template - bool operator()(const T *const p, T *residual) const { - residual[0] = T(weight) * (T(gx) * p[0] + T(gy) * p[1] - T(p0)); - return true; - } - double gx, gy, p0, weight; -}; - -struct XtalResidualRotationOnlyPrecomp { - XtalResidualRotationOnlyPrecomp(const Coord &recip_obs, - const CrystalLattice &latt, - double h, double k, double l) - : s_obs(recip_obs), - astar(latt.Astar()), bstar(latt.Bstar()), cstar(latt.Cstar()), - h(h), k(k), l(l) { - } - - template - bool operator()(const T *const rot_aa, T *residual) const { - const T astar_unrot[3] = {T(astar.x), T(astar.y), T(astar.z)}; - const T bstar_unrot[3] = {T(bstar.x), T(bstar.y), T(bstar.z)}; - const T cstar_unrot[3] = {T(cstar.x), T(cstar.y), T(cstar.z)}; - - T astar_rot[3], bstar_rot[3], cstar_rot[3]; - - const AngleAxisRotator rot(rot_aa); - rot.Rotate(astar_unrot, astar_rot); - rot.Rotate(bstar_unrot, bstar_rot); - rot.Rotate(cstar_unrot, cstar_rot); - - const Eigen::Matrix s_pred(T(h) * astar_rot[0] + T(k) * bstar_rot[0] + T(l) * cstar_rot[0], - T(h) * astar_rot[1] + T(k) * bstar_rot[1] + T(l) * cstar_rot[1], - T(h) * astar_rot[2] + T(k) * bstar_rot[2] + T(l) * cstar_rot[2] - ); - - // Residual in reciprocal space - residual[0] = T(s_obs.x) - s_pred[0]; - residual[1] = T(s_obs.y) - s_pred[1]; - residual[2] = T(s_obs.z) - s_pred[2]; - return true; - } - - const Coord s_obs; - const Coord astar, bstar, cstar; - const double h, k, l; -}; - -// Regularizer: penalises ||rot_aa|| to prefer the smallest rotation that -// explains the data. Weight should be chosen in the same units as the -// reciprocal-space residuals (Å⁻¹ per radian). A value of ~0.01–0.1 is -// typically enough to break degeneracy without biasing the solution. -struct RotationNormRegularizer { - explicit RotationNormRegularizer(double weight) : weight(weight) {} - - template - bool operator()(const T *const rot_aa, T *residual) const { - residual[0] = T(weight) * rot_aa[0]; - residual[1] = T(weight) * rot_aa[1]; - residual[2] = T(weight) * rot_aa[2]; - return true; - } - - const double weight; -}; - // Prior confidence weight per spot: how strong the spot is FOR ITS RESOLUTION. The frame's spots are // ordered by resolution and cut into equal-count shells, and each intensity is divided by its shell // median. Refinement needs the high-resolution spots (they carry the cell and distance information) and @@ -153,12 +80,11 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, const int num_threads) { try { // A coplanar basis has no reciprocal cell: 1/V is infinite, every predicted reciprocal vector - // comes out NaN, and Ceres fails on the very first evaluation - after dumping the offending - // block to stderr. There is nothing for the refinement to recover here, so refuse the lattice - // before the problem is built rather than let the solver discover it. The check has to be on - // the vectors: this close to flat, float cell angles no longer carry even the SIGN of the - // metric determinant, and the triclinic branch of XtalResidual then clamps c into the a-b - // plane and divides by the zero volume that makes. + // comes out NaN, and the solver fails on the very first evaluation. There is nothing for the + // refinement to recover here, so refuse the lattice before the problem is built rather than let + // the solver discover it. The check has to be on the vectors: this close to flat, float cell + // angles no longer carry even the SIGN of the metric determinant, and the triclinic branch of + // XtalResidual then clamps c into the a-b plane and divides by the zero volume that makes. if (data.latt.VolumeFraction() < MIN_BASIS_VOLUME_FRACTION) return false; @@ -168,24 +94,21 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, double beta = data.latt.GetUnitCell().beta; // Initial guess for the parameters - double beam[2] = {data.geom.GetBeamX_pxl(), data.geom.GetBeamY_pxl()}; - double distance_mm = data.geom.GetDetectorDistance_mm(); + const double distance_mm = data.geom.GetDetectorDistance_mm(); - double detector_rot[2] = {data.geom.GetPoniRot1_rad(), data.geom.GetPoniRot2_rad()}; - - // The per-frame constants of the reduced residual (see XtalFrameConstants), one entry per frame - // that contributes. Reserved up front and never grown past that, so the residual blocks' pointers - // into it stay valid, and declared before the problem so that it outlives it. - std::vector frame_const; - frame_const.reserve(spots.size()); - - ceres::Problem problem; - - double latt_vec0[3] = {0.0, 0.0, 0.0}; - double latt_vec1[3] = {0.0, 0.0, 0.0}; - double latt_vec2[3] = {0.0, 0.0, 0.0}; - - double rot_vec[3] = {1, 0, 0}; + XtalRefineProblem problem; + problem.crystal_system = data.crystal_system; + problem.distance_mm = distance_mm; + double *beam = problem.beam; + beam[0] = data.geom.GetBeamX_pxl(); + beam[1] = data.geom.GetBeamY_pxl(); + double *detector_rot = problem.detector_rot; + detector_rot[0] = data.geom.GetPoniRot1_rad(); + detector_rot[1] = data.geom.GetPoniRot2_rad(); + double *latt_vec0 = problem.latt_vec0; + double *latt_vec1 = problem.latt_vec1; + double *latt_vec2 = problem.latt_vec2; + double *rot_vec = problem.rot_vec; switch (data.crystal_system) { case gemmi::CrystalSystem::Orthorhombic: @@ -252,14 +175,15 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, const double sin_rot3 = std::sin(data.geom.GetPoniRot3_rad()); // Per-image rotation refinement frees only the beam and the orientation and holds the other five - // blocks constant, so the seven-block residual makes Ceres differentiate 17 parameters to use 5. - // Where that is the configuration, use the reduced residual instead - identical fit, Jet<5> - // autodiff. Any other combination (stills also free the cell, the offline refiner frees distance - // and detector angles) keeps the general form below. + // blocks constant. Where that is the configuration, the solver uses the reduced residual - the + // identical fit, with the crystal half worked out once (see XtalResidualBeamOrientation). Any + // other combination (stills also free the cell, the rotation indexer frees detector angles and + // spindle) keeps the general form. const bool beam_and_orientation_only = data.refine_beam_center && !data.refine_detector_angles && !data.refine_rotation_axis && !data.refine_unit_cell; + problem.beam_and_orientation_only = beam_and_orientation_only; // Sum of w^2 over the spots that entered - the beam prior below is scaled by it so that its // strength relative to the data is the same weighted or not. Equals the residual block count @@ -281,9 +205,8 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, rot_matr = data.axis->GetTransformationAngle(angle_deg); } - if (beam_and_orientation_only) - frame_const.emplace_back(detector_rot, rot_vec, angle_rad, latt_vec1, latt_vec2, - data.crystal_system); + const int frame_index = static_cast(problem.frame_angle_rad.size()); + problem.frame_angle_rad.push_back(angle_rad); // Add residuals for each point for (size_t j = 0; j < spots[i].size(); j++) { @@ -333,7 +256,7 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, const double weight_sq = weight.empty() ? 1.0 : weight[j] * weight[j]; effective_spots += weight_sq; - const XtalResidual residual(pt.x, pt.y, + problem.residuals.emplace_back(pt.x, pt.y, data.geom.GetWavelength_A(), data.geom.GetPixelSize_mm(), cos_rot3, sin_rot3, @@ -341,38 +264,14 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, h, k, l, data.crystal_system, data.geom.GetOrientation()); - - // Ceres has no per-residual weight; ScaledLoss(nullptr, a) multiplies the squared - // residual by the constant a, i.e. it applies a weight of sqrt(a) to the residual. - ceres::LossFunction *loss = weight.empty() - ? nullptr - : new ceres::ScaledLoss(nullptr, weight_sq, - ceres::TAKE_OWNERSHIP); - - if (beam_and_orientation_only) - problem.AddResidualBlock( - new ceres::AutoDiffCostFunction( - new XtalResidualBeamOrientation(residual, distance_mm, frame_const.back())), - loss, - beam, - latt_vec0 - ); - else - problem.AddResidualBlock( - new ceres::AutoDiffCostFunction( - new XtalResidualFixedDistance(residual, distance_mm)), - loss, - beam, - detector_rot, - rot_vec, - latt_vec0, - latt_vec1, - latt_vec2 - ); + problem.frame.push_back(frame_index); + // A per-residual weight w enters the squared residual as w^2. + if (!weight.empty()) + problem.weight_sq.push_back(weight_sq); } } - if (problem.NumResidualBlocks() < data.min_spots) + if (static_cast(problem.residuals.size()) < data.min_spots) return false; // The gauge direction of a single-axis rotation experiment - parallel to the spindle - written @@ -440,9 +339,8 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, const double gauge_w = data.geom.GetPixelSize_mm() / (distance_mm * data.geom.GetWavelength_A()) * std::sqrt(effective_spots) / sigma_px; - if (!data.refine_beam_center) - problem.SetParameterBlockConstant(beam); - else if (data.axis) { + problem.beam_constant = !data.refine_beam_center; + if (data.refine_beam_center && data.axis) { // Gauge handling (single-axis rotation): rotating the whole experiment about the spindle leaves every // spot position unchanged, so the beam-centre component PARALLEL to the spindle is a null/gauge-weak // direction. Refining it freely lets it wander (~+3 px) and absorb centroid systematics into a wrong @@ -450,23 +348,19 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, // does drift - it is only LaB6-monitored to ~a few px), RESTRAIN it toward the header with a soft // prior: the gauge direction has ~zero data sensitivity so the prior pins it near the header, while a // real, well-supported drift can still overcome it. - problem.AddResidualBlock( - new ceres::AutoDiffCostFunction( - new GaugeDirectionPrior(gauge_beam_x, gauge_beam_y, - gauge_beam_x * beam[0] + gauge_beam_y * beam[1], gauge_w)), - nullptr, beam); + problem.priors.push_back({XtalRefinePrior::Block::Beam, gauge_beam_x, gauge_beam_y, + gauge_beam_x * beam[0] + gauge_beam_y * beam[1], gauge_w}); } // Distance, detector angles, rotation axis and cell are parameter blocks only in the general // seven-block residual; the reduced one bakes them in, so there is nothing left to configure. if (!beam_and_orientation_only) { - if (!data.refine_detector_angles) { - problem.SetParameterBlockConstant(detector_rot); - } else { + problem.detector_rot_constant = !data.refine_detector_angles; + if (data.refine_detector_angles) { const double rot_range = 3.0 / 180.0 * PI; for (int i = 0; i < 2; ++i) { - problem.SetParameterLowerBound(detector_rot, i, detector_rot[i] - rot_range); - problem.SetParameterUpperBound(detector_rot, i, detector_rot[i] + rot_range); + problem.detector_rot_lower[i] = detector_rot[i] - rot_range; + problem.detector_rot_upper[i] = detector_rot[i] + rot_range; } // The same gauge as the beam prior above, described a second time: the tilt moves the // direct beam exactly as the beam centre does, at D/pixel px per radian, so leaving @@ -497,20 +391,15 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, for (int i = 0; i < 2; ++i) { if (budget[i] <= 0.0) continue; - problem.AddResidualBlock( - new ceres::AutoDiffCostFunction( - new GaugeDirectionPrior(dirs[i][0], dirs[i][1], - dirs[i][0] * detector_rot[0] - + dirs[i][1] * detector_rot[1], - gauge_w * (sigma_px / budget[i]) * lever)), - nullptr, detector_rot); + problem.priors.push_back({XtalRefinePrior::Block::DetectorRot, dirs[i][0], dirs[i][1], + dirs[i][0] * detector_rot[0] + dirs[i][1] * detector_rot[1], + gauge_w * (sigma_px / budget[i]) * lever}); } } } - if (!data.refine_rotation_axis) { - problem.SetParameterBlockConstant(rot_vec); - } else { + problem.rot_vec_constant = !data.refine_rotation_axis; + if (data.refine_rotation_axis) { // Only the DIRECTION of the goniometer axis is a parameter. The residual applies // angle_rad * |rot_vec|, so a free three-vector also fits a rotation SCALE - which // GoniometerAxis::Axis() then normalises away, leaving the candidate scored by @@ -520,63 +409,46 @@ bool XtalOptimizerInternal(XtalOptimizerData &data, // data it recovers 54 % of a known scale error, repeated first passes on one dataset // disagree with each other in SIGN, and on the one dataset with a real 1.3 % stage // fault it comes out negative. The rotation scale is measured properly, once, with - // four gates and a jackknife, in PostRefine. - problem.SetManifold(rot_vec, new ceres::SphereManifold<3>); + // four gates and a jackknife, in PostRefine. Refined on the sphere (see SolveXtalRefine). } - if (!data.refine_unit_cell) { - problem.SetParameterBlockConstant(latt_vec1); - problem.SetParameterBlockConstant(latt_vec2); - } else { + problem.latt_vec1_constant = !data.refine_unit_cell; + problem.latt_vec2_constant = !data.refine_unit_cell; + if (data.refine_unit_cell) { // Parameter bounds // Lengths for (int i = 0; i < 3; ++i) { - problem.SetParameterLowerBound(latt_vec1, i, data.min_length_A); - problem.SetParameterUpperBound(latt_vec1, i, data.max_length_A); + problem.latt_vec1_lower[i] = data.min_length_A; + problem.latt_vec1_upper[i] = data.max_length_A; } if (data.crystal_system == gemmi::CrystalSystem::Monoclinic) { - const double beta_lo = std::max(1e-6, PI * (data.min_angle_deg / 180.0)); - const double beta_hi = std::min(PI - 1e-6, PI * (data.max_angle_deg / 180.0)); - problem.SetParameterLowerBound(latt_vec2, 0, beta_lo); - problem.SetParameterUpperBound(latt_vec2, 0, beta_hi); + problem.latt_vec2_constant = false; + problem.latt_vec2_lower[0] = std::max(1e-6, PI * (data.min_angle_deg / 180.0)); + problem.latt_vec2_upper[0] = std::min(PI - 1e-6, PI * (data.max_angle_deg / 180.0)); } else if (data.crystal_system == gemmi::CrystalSystem::Triclinic) { // α, β, γ bounds (radians) const double alo = PI * (data.min_angle_deg / 180.0); const double ahi = PI * (data.max_angle_deg / 180.0); for (int i = 0; i < 3; ++i) { - problem.SetParameterLowerBound(latt_vec2, i, alo); - problem.SetParameterUpperBound(latt_vec2, i, ahi); + problem.latt_vec2_lower[i] = alo; + problem.latt_vec2_upper[i] = ahi; } } else { // Orthorhombic / Tetragonal / Cubic / Hexagonal: // latt_vec2 has no meaning for these systems — always freeze it. - problem.SetParameterBlockConstant(latt_vec2); + problem.latt_vec2_constant = true; } } } - // Configure solver - ceres::Solver::Options options; - // Normal equations, not QR. The problem is very tall and thin - thousands of spots against at - // most 17 parameters - and that is the shape DENSE_QR handles worst: it copies the Jacobian out - // of Ceres' row-major storage into a column-major buffer on every solve, and Eigen's blocked - // Householder then degenerates to the unblocked path because its block size is min(48, columns). - // Accumulating J^T J reads the Jacobian once instead. Both solve the same damped system, so the - // step is the same to round-off; the column scaling Ceres applies by default and the LM diagonal - // keep the squared condition number in hand. - options.linear_solver_type = ceres::DENSE_NORMAL_CHOLESKY; - options.minimizer_progress_to_stdout = false; + // Stopping rule: a bound on iterations is reproducible, a bound on wall-clock time is not (see + // XtalOptimizerData::max_iterations). if (data.max_iterations > 0) - options.max_num_iterations = data.max_iterations; + problem.options.max_iterations = data.max_iterations; else - options.max_solver_time_in_seconds = data.max_time; - options.logging_type = ceres::LoggingType::SILENT; - options.num_threads = num_threads; // usually 1 (called from many threads); caller may raise it - ceres::Solver::Summary summary; - - // Run optimization - ceres::Solve(options, &problem, &summary); + problem.options.max_time_s = data.max_time; + const LMSummary summary = SolveXtalRefine(problem, num_threads); // Only a genuine numerical failure is rejected here: a solve that ran out of iterations or // out of time but still descended counts as usable, which is what the real-time caller @@ -653,7 +525,7 @@ bool XtalOptimizerRotationOnly(XtalOptimizerData &data, return false; // Parameter: angle-axis for the extra rotation. Identity == {0,0,0}. - double rot_aa[3] = {0.0, 0.0, 0.0}; + std::vector rot_aa = {0.0, 0.0, 0.0}; // Spot selection by current indexing (same approach as XtalOptimizerInternal) const Coord a0 = data.latt.Vec0(); @@ -662,7 +534,12 @@ bool XtalOptimizerRotationOnly(XtalOptimizerData &data, const float tol_sq = tolerance * tolerance; - ceres::Problem problem; + // Each selected spot: its observed reciprocal vector and the indices it is fitted to. + struct Observation { + Coord s_obs; + double h, k, l; + }; + std::vector observations; for (const auto &pt : spots) { if (!data.index_ice_rings && pt.ice_ring) @@ -697,42 +574,68 @@ bool XtalOptimizerRotationOnly(XtalOptimizerData &data, if (data.axis.has_value()) s_obs = data.axis->GetTransformationAngle(pt.phi) * s_obs; - auto *cost = - new ceres::AutoDiffCostFunction( - new XtalResidualRotationOnlyPrecomp(s_obs, data.latt, h, k, l) - ); - - problem.AddResidualBlock(cost, nullptr, rot_aa); + observations.push_back({s_obs, h, k, l}); } - if (problem.NumResidualBlocks() < data.min_spots) + if (static_cast(observations.size()) < data.min_spots) return false; - // Regularization: prefer the smallest rotation correction that fits the - // data. This is essential when spots are nearly coplanar in reciprocal - // space (e.g. still images), where the rotation component perpendicular - // to the scattering plane is otherwise underdetermined. - // The weight is in Å⁻¹ rad⁻¹; tune relative to your typical residual. - { - const double reg_weight = 0.05; // e.g. 0.05 - problem.AddResidualBlock( - new ceres::AutoDiffCostFunction( - new RotationNormRegularizer(reg_weight)), - nullptr, rot_aa); - } + // Residual: s_obs - R(rot_aa) (h a* + k b* + l c*), the reciprocal basis rotated once per + // evaluation and shared by every spot. + // + // Regularization: prefer the smallest rotation correction that fits the data, w * rot_aa. This is + // essential when spots are nearly coplanar in reciprocal space (e.g. still images), where the + // rotation component perpendicular to the scattering plane is otherwise underdetermined. The + // weight is in A^-1 rad^-1, relative to the typical residual. + const double reg_weight = 0.05; + const Coord astar = data.latt.Astar(), bstar = data.latt.Bstar(), cstar = data.latt.Cstar(); + const auto evaluate = [&](const double *aa, double &cost, Eigen::VectorXd *g, Eigen::MatrixXd *H) { + using D = Dual<3>; + const D aa_d[3] = {D::Variable(aa[0], 0), D::Variable(aa[1], 1), D::Variable(aa[2], 2)}; + const AngleAxisRotator rot(aa_d); + const double astar_unrot[3] = {astar.x, astar.y, astar.z}; + const double bstar_unrot[3] = {bstar.x, bstar.y, bstar.z}; + const double cstar_unrot[3] = {cstar.x, cstar.y, cstar.z}; + D astar_rot[3], bstar_rot[3], cstar_rot[3]; + rot.Rotate(astar_unrot, astar_rot); + rot.Rotate(bstar_unrot, bstar_rot); + rot.Rotate(cstar_unrot, cstar_rot); - ceres::Solver::Options options; - options.linear_solver_type = ceres::DENSE_NORMAL_CHOLESKY; // tall and thin, as above - options.minimizer_progress_to_stdout = false; + cost = 0.0; + const auto add = [&](double r, const double *J) { + cost += 0.5 * r * r; + if (!g) + return; + for (int i = 0; i < 3; i++) { + (*g)[i] += J[i] * r; + for (int j = 0; j < 3; j++) + (*H)(i, j) += J[i] * J[j]; + } + }; + for (const auto &o: observations) { + const double s_obs[3] = {o.s_obs.x, o.s_obs.y, o.s_obs.z}; + for (int c = 0; c < 3; c++) { + const D pred = o.h * astar_rot[c] + o.k * bstar_rot[c] + o.l * cstar_rot[c]; + const double J[3] = {-pred.v[0], -pred.v[1], -pred.v[2]}; + add(s_obs[c] - pred.a, J); + } + } + for (int c = 0; c < 3; c++) { + double J[3] = {0.0, 0.0, 0.0}; + J[c] = reg_weight; + add(reg_weight * aa[c], J); + } + return std::isfinite(cost) && (!g || (g->allFinite() && H->allFinite())); + }; + + std::vector blocks(1); + blocks[0].size = 3; + LMOptions options; if (data.max_iterations > 0) - options.max_num_iterations = data.max_iterations; + options.max_iterations = data.max_iterations; else - options.max_solver_time_in_seconds = data.max_time; - options.logging_type = ceres::LoggingType::SILENT; - options.num_threads = 1; - - ceres::Solver::Summary summary; - ceres::Solve(options, &problem, &summary); + options.max_time_s = data.max_time; + const LMSummary summary = SolveLM(rot_aa, blocks, options, evaluate); if (!summary.IsSolutionUsable()) return false; @@ -747,7 +650,7 @@ bool XtalOptimizerRotationOnly(XtalOptimizerData &data, // rotating the reciprocal vectors (a*, b*, c*) by the same R. No // transpose or inversion of R is needed here. double R_raw[9]; - ceres::AngleAxisToRotationMatrix(rot_aa, R_raw); // row-major 3x3 + ceres::AngleAxisToRotationMatrix(rot_aa.data(), R_raw); // row-major 3x3 Eigen::Matrix3d R; R << R_raw[0], R_raw[3], R_raw[6], diff --git a/image_analysis/geom_refinement/XtalOptimizer.h b/image_analysis/geom_refinement/XtalOptimizer.h index 1bf9084de..0b64937bb 100644 --- a/image_analysis/geom_refinement/XtalOptimizer.h +++ b/image_analysis/geom_refinement/XtalOptimizer.h @@ -62,7 +62,7 @@ struct XtalOptimizerData { std::optional angle_axis; }; -// num_threads sets the Ceres solver thread count for the internal least-squares refine. It defaults +// num_threads sets the thread count of the internal least-squares refine (the answer does not depend on it). It defaults // to 1 because XtalOptimizer is usually called from many threads at once; raise it only when a caller // runs a small number of refinements concurrently and wants each to use several cores. bool XtalOptimizer(XtalOptimizerData &data, std::span> spots, diff --git a/image_analysis/geom_refinement/XtalRefine.cpp b/image_analysis/geom_refinement/XtalRefine.cpp new file mode 100644 index 000000000..3b466b97d --- /dev/null +++ b/image_analysis/geom_refinement/XtalRefine.cpp @@ -0,0 +1,334 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#include "XtalRefine.h" + +#include + +#include "Dual.h" +#include "../../common/ParallelFor.h" + +namespace { + // Ambient layout of the parameter vector and the order of the blocks in it. + constexpr int OFF_BEAM = 0, OFF_ROT = 2, OFF_AXIS = 4, OFF_P0 = 7, OFF_LEN = 10, OFF_ANG = 13, N_AMBIENT = 16; + + // Derivative lanes. The observed half of a residual depends on beam, detector angles and spindle + // (OBS lanes), the predicted half on orientation and cell (PRED lanes); each half is carried on a + // dual number of its own width and the two are put side by side only in the sums. + constexpr int OBS = 6, PRED = 9, LANES = OBS + PRED; + using DO = Dual; + using DP = Dual; + + // The reduced problem: beam (2) and orientation (3) only. + constexpr int R_OBS = 2, R_PRED = 3, R_LANES = R_OBS + R_PRED; + + // Sums over one block of residuals, in lane coordinates. + template + struct Sums { + double cost = 0.0; + double g[L] = {}; + double H[L][L] = {}; // upper triangle + + void Add(double r, const double *J, double w2) { + cost += 0.5 * w2 * r * r; + for (int i = 0; i < L; i++) { + const double wj = w2 * J[i]; + g[i] += wj * r; + for (int j = i; j < L; j++) + H[i][j] += wj * J[j]; + } + } + + void Add(const Sums &o) { + cost += o.cost; + for (int i = 0; i < L; i++) { + g[i] += o.g[i]; + for (int j = i; j < L; j++) + H[i][j] += o.H[i][j]; + } + } + }; + + std::vector MakeBlocks(const XtalRefineProblem &p) { + std::vector blocks(6); + blocks[0].offset = OFF_BEAM; + blocks[0].size = 2; + blocks[0].constant = p.beam_constant; + + blocks[1].offset = OFF_ROT; + blocks[1].size = 2; + blocks[1].constant = p.beam_and_orientation_only || p.detector_rot_constant; + for (int j = 0; j < 2; j++) { + blocks[1].lower[j] = p.detector_rot_lower[j]; + blocks[1].upper[j] = p.detector_rot_upper[j]; + } + + blocks[2].offset = OFF_AXIS; + blocks[2].size = 3; + blocks[2].constant = p.beam_and_orientation_only || p.rot_vec_constant; + blocks[2].sphere = true; + + blocks[3].offset = OFF_P0; + blocks[3].size = 3; + + blocks[4].offset = OFF_LEN; + blocks[4].size = 3; + blocks[4].constant = p.beam_and_orientation_only || p.latt_vec1_constant; + + blocks[5].offset = OFF_ANG; + blocks[5].size = 3; + blocks[5].constant = p.beam_and_orientation_only || p.latt_vec2_constant; + + for (int j = 0; j < 3; j++) { + blocks[4].lower[j] = p.latt_vec1_lower[j]; + blocks[4].upper[j] = p.latt_vec1_upper[j]; + blocks[5].lower[j] = p.latt_vec2_lower[j]; + blocks[5].upper[j] = p.latt_vec2_upper[j]; + } + return blocks; + } + + // Where each lane lands among the tangent coordinates of the free blocks; -1 for a held one. + template + std::array LaneToTangent(const std::vector &blocks, const std::array &lane_block, + const std::array &lane_index) { + std::array map{}; + std::array tangent_offset{}; + int t = 0; + for (size_t b = 0; b < blocks.size(); b++) { + tangent_offset[b] = t; + t += blocks[b].TangentSize(); + } + for (size_t l = 0; l < L; l++) + map[l] = blocks[lane_block[l]].constant ? -1 : tangent_offset[lane_block[l]] + lane_index[l]; + return map; + } + + template + void ToTangent(const Sums &s, const std::array &map, Eigen::VectorXd &g, Eigen::MatrixXd &H) { + for (int i = 0; i < L; i++) { + if (map[i] < 0) + continue; + g[map[i]] += s.g[i]; + for (int j = i; j < L; j++) { + if (map[j] < 0) + continue; + H(map[i], map[j]) += s.H[i][j]; + if (map[i] != map[j]) + H(map[j], map[i]) += s.H[i][j]; + } + } + } + + void AddPriors(const XtalRefineProblem &p, const double *x, double &cost, Eigen::VectorXd *g, + Eigen::MatrixXd *H, int beam_tangent, int rot_tangent) { + for (const auto &prior: p.priors) { + const bool on_beam = prior.block == XtalRefinePrior::Block::Beam; + const double *v = x + (on_beam ? OFF_BEAM : OFF_ROT); + const double r = prior.weight * (prior.gx * v[0] + prior.gy * v[1] - prior.p0); + cost += 0.5 * r * r; + const int t = on_beam ? beam_tangent : rot_tangent; + if (!g || t < 0) + continue; + const double J[2] = {prior.weight * prior.gx, prior.weight * prior.gy}; + for (int i = 0; i < 2; i++) { + (*g)[t + i] += J[i] * r; + for (int j = 0; j < 2; j++) + (*H)(t + i, t + j) += J[i] * J[j]; + } + } + } + + template + D Seed(double value, int lane, bool free) { + return free ? D::Variable(value, lane) : D(value); + } + + // Residual blocks of at least this many residuals; the cut depends on the count alone. + constexpr int MIN_RESIDUALS_PER_BLOCK = 256; + + template + Sums SumResiduals(const XtalRefineProblem &p, int num_threads, Fn &&residual) { + const int n = static_cast(p.residuals.size()); + std::vector> partial(ReductionBlocks(n, MIN_RESIDUALS_PER_BLOCK)); + ParallelBlocks(n, std::max(1, num_threads), [&](int b, int lo, int hi) { + for (int i = lo; i < hi; i++) + residual(i, partial[b]); + }, MIN_RESIDUALS_PER_BLOCK); + Sums total; + for (const auto &s: partial) + total.Add(s); + return total; + } + + double WeightSq(const XtalRefineProblem &p, int i) { + return p.weight_sq.empty() ? 1.0 : p.weight_sq[i]; + } + + bool AllFinite(double cost, const Eigen::VectorXd *g, const Eigen::MatrixXd *H) { + return std::isfinite(cost) && (!g || (g->allFinite() && H->allFinite())); + } + + // Seven-block residual (XtalResidualFixedDistance), with every block that depends on parameters + // alone - detector-angle sines and cosines, the spindle back-rotation of each frame, the reciprocal + // basis of the cell, the orientation's rotation - worked out once per evaluation. + bool EvaluateGeneral(const XtalRefineProblem &p, const std::array &map, int num_threads, + const double *x, double &cost, Eigen::VectorXd *g, Eigen::MatrixXd *H) { + const bool beam_free = map[0] >= 0, rot_free = map[2] >= 0, axis_free = map[4] >= 0; + const bool p0_free = map[OBS] >= 0, len_free = map[OBS + 3] >= 0, ang_free = map[OBS + 6] >= 0; + + const DO beam[2] = {Seed(x[OFF_BEAM], 0, beam_free), + Seed(x[OFF_BEAM + 1], 1, beam_free)}; + const DO rot1 = Seed(x[OFF_ROT], 2, rot_free); + const DO rot2 = Seed(x[OFF_ROT + 1], 3, rot_free); + const DO c1 = cos(rot1), s1 = sin(rot1), c2 = cos(rot2), s2 = sin(rot2); + + DO axis[3] = {x[OFF_AXIS], x[OFF_AXIS + 1], x[OFF_AXIS + 2]}; + if (axis_free) { + double J[3][2]; + SpherePlusJacobian3(x + OFF_AXIS, J); + for (int k = 0; k < 3; k++) { + axis[k].v[4] = J[k][0]; + axis[k].v[5] = J[k][1]; + } + } + std::vector> rot_back; + rot_back.reserve(p.frame_angle_rad.size()); + for (const double angle: p.frame_angle_rad) { + const DO aa_back[3] = {angle * axis[0], angle * axis[1], angle * axis[2]}; + rot_back.emplace_back(aa_back); + } + + DP p0[3], len[3], ang[3]; + for (int k = 0; k < 3; k++) { + p0[k] = Seed(x[OFF_P0 + k], k, p0_free); + len[k] = Seed(x[OFF_LEN + k], 3 + k, len_free); + ang[k] = Seed(x[OFF_ANG + k], 6 + k, ang_free); + } + Eigen::Matrix bxc, cxa, axb; + DP invV; + XtalResidual::ReciprocalBasis(len, ang, p.crystal_system, bxc, cxa, axb, invV); + const AngleAxisRotator rot_p0(p0); + + const Sums s = SumResiduals(p, num_threads, [&](int i, Sums &acc) { + const XtalResidual &res = p.residuals[i]; + DO obs[3]; + res.ObservedRecipCore(beam, p.distance_mm, c1, s1, c2, s2, rot_back[p.frame[i]], obs); + DP unrot[3], pred[3]; + res.CombineRecipUnrot(bxc, cxa, axb, invV, unrot); + rot_p0.Rotate(unrot, pred); + const double w2 = WeightSq(p, i); + for (int k = 0; k < 3; k++) { + double J[LANES]; + for (int l = 0; l < OBS; l++) + J[l] = obs[k].v[l]; + for (int l = 0; l < PRED; l++) + J[OBS + l] = -pred[k].v[l]; + acc.Add(obs[k].a - pred[k].a, J, w2); + } + }); + + cost = s.cost; + if (g) + ToTangent(s, map, *g, *H); + AddPriors(p, x, cost, g, H, map[0], map[2]); + return AllFinite(cost, g, H); + } + + // The reduced residual (XtalResidualBeamOrientation): detector, spindle and cell held, so the + // back-rotation of each frame and the unrotated prediction of each residual are constants. + bool EvaluateBeamOrientation(const XtalRefineProblem &p, const std::array &map, + const std::vector> &rot_back, + const std::vector> &unrot, double c1, double s1, + double c2, double s2, int num_threads, + const double *x, double &cost, Eigen::VectorXd *g, Eigen::MatrixXd *H) { + using DB = Dual; + using DR = Dual; + const bool beam_free = map[0] >= 0; + const DB beam[2] = {beam_free ? DB::Variable(x[OFF_BEAM], 0) : DB(x[OFF_BEAM]), + beam_free ? DB::Variable(x[OFF_BEAM + 1], 1) : DB(x[OFF_BEAM + 1])}; + const DR p0[3] = {DR::Variable(x[OFF_P0], 0), DR::Variable(x[OFF_P0 + 1], 1), + DR::Variable(x[OFF_P0 + 2], 2)}; + const AngleAxisRotator rot_p0(p0); + + const Sums s = SumResiduals(p, num_threads, [&](int i, Sums &acc) { + const XtalResidual &res = p.residuals[i]; + DB obs[3]; + res.ObservedRecipCore(beam, p.distance_mm, c1, s1, c2, s2, rot_back[p.frame[i]], obs); + DR pred[3]; + rot_p0.Rotate(unrot[i].data(), pred); + const double w2 = WeightSq(p, i); + for (int k = 0; k < 3; k++) { + double J[R_LANES]; + for (int l = 0; l < R_OBS; l++) + J[l] = obs[k].v[l]; + for (int l = 0; l < R_PRED; l++) + J[R_OBS + l] = -pred[k].v[l]; + acc.Add(obs[k].a - pred[k].a, J, w2); + } + }); + + cost = s.cost; + if (g) + ToTangent(s, map, *g, *H); + AddPriors(p, x, cost, g, H, map[0], -1); + return AllFinite(cost, g, H); + } +} + +LMSummary SolveXtalRefine(XtalRefineProblem &p, int num_threads) { + const std::vector blocks = MakeBlocks(p); + + std::vector x(N_AMBIENT); + const auto put = [&](int off, const double *v, int n) { for (int i = 0; i < n; i++) x[off + i] = v[i]; }; + put(OFF_BEAM, p.beam, 2); + put(OFF_ROT, p.detector_rot, 2); + put(OFF_AXIS, p.rot_vec, 3); + put(OFF_P0, p.latt_vec0, 3); + put(OFF_LEN, p.latt_vec1, 3); + put(OFF_ANG, p.latt_vec2, 3); + + LMSummary summary; + if (p.beam_and_orientation_only) { + const std::array lane_block = {0, 0, 3, 3, 3}; + const std::array lane_index = {0, 1, 0, 1, 2}; + const auto map = LaneToTangent(blocks, lane_block, lane_index); + + std::vector> rot_back; + rot_back.reserve(p.frame_angle_rad.size()); + for (const double angle: p.frame_angle_rad) + rot_back.push_back(XtalFrameConstants::BackRotator(angle, p.rot_vec)); + Eigen::Matrix bxc, cxa, axb; + double invV; + XtalResidual::ReciprocalBasis(p.latt_vec1, p.latt_vec2, p.crystal_system, bxc, cxa, axb, invV); + std::vector> unrot(p.residuals.size()); + for (size_t i = 0; i < p.residuals.size(); i++) + p.residuals[i].CombineRecipUnrot(bxc, cxa, axb, invV, unrot[i].data()); + const double c1 = std::cos(p.detector_rot[0]), s1 = std::sin(p.detector_rot[0]); + const double c2 = std::cos(p.detector_rot[1]), s2 = std::sin(p.detector_rot[1]); + + summary = SolveLM(x, blocks, p.options, [&](const double *at, double &cost, Eigen::VectorXd *g, + Eigen::MatrixXd *H) { + return EvaluateBeamOrientation(p, map, rot_back, unrot, c1, s1, c2, s2, num_threads, at, cost, g, H); + }); + } else { + const std::array lane_block = {0, 0, 1, 1, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5}; + const std::array lane_index = {0, 1, 0, 1, 0, 1, 0, 1, 2, 0, 1, 2, 0, 1, 2}; + const auto map = LaneToTangent(blocks, lane_block, lane_index); + summary = SolveLM(x, blocks, p.options, [&](const double *at, double &cost, Eigen::VectorXd *g, + Eigen::MatrixXd *H) { + return EvaluateGeneral(p, map, num_threads, at, cost, g, H); + }); + } + + if (summary.IsSolutionUsable()) { + const auto get = [&](int off, double *v, int n) { for (int i = 0; i < n; i++) v[i] = x[off + i]; }; + get(OFF_BEAM, p.beam, 2); + get(OFF_ROT, p.detector_rot, 2); + get(OFF_AXIS, p.rot_vec, 3); + get(OFF_P0, p.latt_vec0, 3); + get(OFF_LEN, p.latt_vec1, 3); + get(OFF_ANG, p.latt_vec2, 3); + } + return summary; +} diff --git a/image_analysis/geom_refinement/XtalRefine.h b/image_analysis/geom_refinement/XtalRefine.h new file mode 100644 index 000000000..4ee88870e --- /dev/null +++ b/image_analysis/geom_refinement/XtalRefine.h @@ -0,0 +1,63 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +#include +#include + +#include "XtalResidual.h" +#include "LMSolver.h" + +// A soft restraint w * (g . p - p0) on one direction g of a two-component block - the beam centre or +// the detector tilt, which are the same gauge seen twice and so take the same direction (see +// XtalOptimizer). +struct XtalRefinePrior { + enum class Block { Beam, DetectorRot } block = Block::Beam; + double gx = 0.0, gy = 0.0, p0 = 0.0, weight = 0.0; +}; + +// The least-squares problem XtalOptimizer solves, as data: the residuals with the frame each belongs +// to, the parameter blocks with what is held and what is bounded, and the priors. The parameter arrays +// are in/out. Parameter blocks: beam(2), detector_rot(2), rot_vec(3, the spindle, refined on the sphere), +// latt_vec0(3, orientation angle-axis), latt_vec1(3, cell lengths), latt_vec2(3, cell angles) - see +// XtalResidual. The distance is a constant of the problem. +struct XtalRefineProblem { + static constexpr double kNoBound = std::numeric_limits::max(); + + gemmi::CrystalSystem crystal_system = gemmi::CrystalSystem::Triclinic; + // Only beam and orientation free, everything else held: the reduced residual (see + // XtalResidualBeamOrientation), whose crystal half is a constant of the problem. + bool beam_and_orientation_only = false; + double distance_mm = 0.0; + + std::vector residuals; + std::vector frame; // per residual, an index into frame_angle_rad + std::vector frame_angle_rad; + std::vector weight_sq; // per residual; empty = unweighted + + double beam[2] = {0, 0}; + double detector_rot[2] = {0, 0}; + double rot_vec[3] = {1, 0, 0}; + double latt_vec0[3] = {0, 0, 0}; + double latt_vec1[3] = {0, 0, 0}; + double latt_vec2[3] = {0, 0, 0}; + + bool beam_constant = false; + bool detector_rot_constant = true; + bool rot_vec_constant = true; + bool latt_vec1_constant = true; + bool latt_vec2_constant = true; + + double detector_rot_lower[2] = {-kNoBound, -kNoBound}, detector_rot_upper[2] = {kNoBound, kNoBound}; + double latt_vec1_lower[3] = {-kNoBound, -kNoBound, -kNoBound}, latt_vec1_upper[3] = {kNoBound, kNoBound, kNoBound}; + double latt_vec2_lower[3] = {-kNoBound, -kNoBound, -kNoBound}, latt_vec2_upper[3] = {kNoBound, kNoBound, kNoBound}; + + std::vector priors; + + LMOptions options; +}; + +// Solves the problem in place. The residual sums are cut into blocks that depend on the problem +// alone, so the answer is the same at any thread count. +LMSummary SolveXtalRefine(XtalRefineProblem &problem, int num_threads); diff --git a/image_analysis/geom_refinement/XtalResidual.h b/image_analysis/geom_refinement/XtalResidual.h index b67e055c8..26c05a26a 100644 --- a/image_analysis/geom_refinement/XtalResidual.h +++ b/image_analysis/geom_refinement/XtalResidual.h @@ -7,8 +7,6 @@ #include -#include "ceres/ceres.h" -#include "ceres/rotation.h" #include "gemmi/symmetry.hpp" #include "../../common/JFJochException.h" @@ -110,6 +108,9 @@ inline void EffectiveCellFromParams(gemmi::CrystalSystem symmetry, const double } } +// The scalar type is a template parameter throughout: a double, a ceres::Jet or a Dual (Dual.h). The +// mathematical functions are called unqualified, so each type's own overload is found by lookup. +// // Detector -> reciprocal geometry residual, shared by the per-image XtalOptimizer (one lattice, one // frame) and the offline GeometryRefiner (shared beam/distance/cell blocks, one orientation block per // frame). Parameter blocks: beam(2), distance_mm(1), detector_rot(2 = rot1,rot2), rotation_axis(3), @@ -164,16 +165,18 @@ struct XtalResidual { // detector_rot[0] = rot1, detector_rot[1] = rot2 are refined; rot3 is fixed // (e.g. from a PONI import) and baked in here as a constant so that a non-zero // rot3 is not silently dropped during refinement. + using std::cos; + using std::sin; const C rot1 = detector_rot[0]; const C rot2 = detector_rot[1]; // Ry(+rot1): rotation around Y-axis - const C c1 = ceres::cos(rot1); - const C s1 = ceres::sin(rot1); + const C c1 = cos(rot1); + const C s1 = sin(rot1); // Rx(-rot2): rotation around X-axis with inverted sign (PyFAI left-handed) - const C c2 = ceres::cos(rot2); - const C s2 = ceres::sin(rot2); + const C c2 = cos(rot2); + const C s2 = sin(rot2); // Apply the goniometer "back-to-start" rotation of this frame's angle. const C aa_back[3] = { @@ -227,7 +230,8 @@ struct XtalResidual { const T z = t2_z; // convert to recip space - const T lab_norm = ceres::sqrt(x * x + y * y + z * z); + using std::sqrt; + const T lab_norm = sqrt(x * x + y * y + z * z); const T inv_norm = T(1) / lab_norm; T recip_raw[3]; @@ -258,6 +262,9 @@ struct XtalResidual { static void ReciprocalBasis(const C *const p1, const C *const p2, gemmi::CrystalSystem symmetry, Eigen::Matrix &bxc, Eigen::Matrix &cxa, Eigen::Matrix &axb, C &invV) { + using std::cos; + using std::sin; + using std::sqrt; // Build unit cell lengths and B (convention: columns are a, b, c prior to global rotation) Eigen::Matrix e_uc_len = Eigen::Matrix::Zero(); Eigen::Matrix B = Eigen::Matrix::Identity(); @@ -275,14 +282,14 @@ struct XtalResidual { } else if (symmetry == gemmi::CrystalSystem::Monoclinic) { // Unique axis b: alpha = gamma = 90°, beta free (angle between a and c) e_uc_len << p1[0], p1[1], p1[2]; - B(0, 2) = ceres::cos(p2[0]); - B(2, 2) = ceres::sin(p2[0]); + B(0, 2) = cos(p2[0]); + B(2, 2) = sin(p2[0]); } else { // Triclinic: p1 = (a,b,c), p2 = (alpha, beta, gamma) in radians - const C ca = ceres::cos(p2[0]); - const C cb = ceres::cos(p2[1]); - const C cg = ceres::cos(p2[2]); - const C sg = ceres::sin(p2[2]); + const C ca = cos(p2[0]); + const C cb = cos(p2[1]); + const C cg = cos(p2[2]); + const C sg = sin(p2[2]); e_uc_len << p1[0], p1[1], p1[2]; @@ -297,7 +304,7 @@ struct XtalResidual { const C cx = cb; const C cy = (ca - cb * cg) / sg; const C v = C(1) - cx * cx - cy * cy; - const C cz = (v >= C(0)) ? ceres::sqrt(v) : C(0); + const C cz = (v >= C(0)) ? sqrt(v) : C(0); B(0, 2) = cx; B(1, 2) = cy; @@ -399,12 +406,12 @@ struct XtalResidual { // reciprocal basis of the fixed cell. They are constants of the whole problem there - one frame per // image, one cell - and deriving them inside each residual costs six trigonometric calls, a hypot and a // division per evaluation for numbers that never change. Built by the caller, which has to keep it alive -// as long as the ceres::Problem that points at it. +// as long as the residuals that point at it. struct XtalFrameConstants { XtalFrameConstants(const double *detector_rot, const double *rotation_axis, double angle_rad, const double *uc_len, const double *uc_angle, gemmi::CrystalSystem symmetry) - : c1(ceres::cos(detector_rot[0])), s1(ceres::sin(detector_rot[0])), - c2(ceres::cos(detector_rot[1])), s2(ceres::sin(detector_rot[1])), + : c1(cos(detector_rot[0])), s1(sin(detector_rot[0])), + c2(cos(detector_rot[1])), s2(sin(detector_rot[1])), rot_back(BackRotator(angle_rad, rotation_axis)) { XtalResidual::ReciprocalBasis(uc_len, uc_angle, symmetry, bxc, cxa, axb, invV); } diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index f01b89506..799c06c0d 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -112,6 +112,8 @@ ADD_EXECUTABLE(jfjoch_test RingsFromProfileTest.cpp CalibrationTest.cpp XtalOptimizerTest.cpp + XtalRefineTest.cpp + XtalRefineCeres.h CrystalLatticeTest.cpp FPGAPTPTest.cpp ResolutionShellsTest.cpp diff --git a/tests/XtalRefineCeres.h b/tests/XtalRefineCeres.h new file mode 100644 index 000000000..77f34b4f9 --- /dev/null +++ b/tests/XtalRefineCeres.h @@ -0,0 +1,92 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +// The reference the LM solver of XtalRefine is checked against: the same XtalRefineProblem handed to +// Ceres exactly as XtalOptimizer used to build it - the same residual functors, losses, priors, bounds, +// manifold and options. + +#include "ceres/ceres.h" +#include "../image_analysis/geom_refinement/XtalRefine.h" + +struct XtalRefineCeresPrior { + XtalRefineCeresPrior(double gx, double gy, double p0, double weight) + : gx(gx), gy(gy), p0(p0), weight(weight) {} + template + bool operator()(const T *const p, T *residual) const { + residual[0] = T(weight) * (T(gx) * p[0] + T(gy) * p[1] - T(p0)); + return true; + } + double gx, gy, p0, weight; +}; + +inline ceres::Solver::Summary SolveXtalRefineCeres(XtalRefineProblem &p, int num_threads) { + std::vector frame_const; + frame_const.reserve(p.frame_angle_rad.size()); + if (p.beam_and_orientation_only) + for (const double angle: p.frame_angle_rad) + frame_const.emplace_back(p.detector_rot, p.rot_vec, angle, p.latt_vec1, p.latt_vec2, p.crystal_system); + + ceres::Problem problem; + for (size_t i = 0; i < p.residuals.size(); i++) { + ceres::LossFunction *loss = p.weight_sq.empty() + ? nullptr + : new ceres::ScaledLoss(nullptr, p.weight_sq[i], ceres::TAKE_OWNERSHIP); + if (p.beam_and_orientation_only) + problem.AddResidualBlock( + new ceres::AutoDiffCostFunction( + new XtalResidualBeamOrientation(p.residuals[i], p.distance_mm, frame_const[p.frame[i]])), + loss, p.beam, p.latt_vec0); + else + problem.AddResidualBlock( + new ceres::AutoDiffCostFunction( + new XtalResidualFixedDistance(p.residuals[i], p.distance_mm)), + loss, p.beam, p.detector_rot, p.rot_vec, p.latt_vec0, p.latt_vec1, p.latt_vec2); + } + for (const auto &prior: p.priors) + problem.AddResidualBlock( + new ceres::AutoDiffCostFunction( + new XtalRefineCeresPrior(prior.gx, prior.gy, prior.p0, prior.weight)), + nullptr, prior.block == XtalRefinePrior::Block::Beam ? p.beam : p.detector_rot); + + const auto bounds = [&](double *block, const double *lo, const double *hi, int n) { + for (int i = 0; i < n; i++) { + if (lo[i] > -XtalRefineProblem::kNoBound) + problem.SetParameterLowerBound(block, i, lo[i]); + if (hi[i] < XtalRefineProblem::kNoBound) + problem.SetParameterUpperBound(block, i, hi[i]); + } + }; + if (p.beam_constant) + problem.SetParameterBlockConstant(p.beam); + if (!p.beam_and_orientation_only) { + if (p.detector_rot_constant) + problem.SetParameterBlockConstant(p.detector_rot); + else + bounds(p.detector_rot, p.detector_rot_lower, p.detector_rot_upper, 2); + if (p.rot_vec_constant) + problem.SetParameterBlockConstant(p.rot_vec); + else + problem.SetManifold(p.rot_vec, new ceres::SphereManifold<3>); + if (p.latt_vec1_constant) + problem.SetParameterBlockConstant(p.latt_vec1); + else + bounds(p.latt_vec1, p.latt_vec1_lower, p.latt_vec1_upper, 3); + if (p.latt_vec2_constant) + problem.SetParameterBlockConstant(p.latt_vec2); + else + bounds(p.latt_vec2, p.latt_vec2_lower, p.latt_vec2_upper, 3); + } + + ceres::Solver::Options options; + options.linear_solver_type = ceres::DENSE_NORMAL_CHOLESKY; + options.minimizer_progress_to_stdout = false; + options.max_num_iterations = p.options.max_iterations; + options.max_solver_time_in_seconds = p.options.max_time_s; + options.logging_type = ceres::LoggingType::SILENT; + options.num_threads = num_threads; + ceres::Solver::Summary summary; + ceres::Solve(options, &problem, &summary); + return summary; +} diff --git a/tests/XtalRefineTest.cpp b/tests/XtalRefineTest.cpp new file mode 100644 index 000000000..a26021f91 --- /dev/null +++ b/tests/XtalRefineTest.cpp @@ -0,0 +1,127 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +// Ceres first: its logging header defines a CHECK macro of its own, which Catch's must replace here. +#include "XtalRefineCeres.h" +#undef CHECK +#include + +#include "../image_analysis/geom_refinement/LatticeReduction.h" +#include "../image_analysis/bragg_prediction/BraggPrediction.h" + +// The LM solver of XtalRefine is meant to take Ceres' path to Ceres' answer: same steps accepted, same +// stopping rule, same point. These cases hand one problem to both and compare. +namespace { + // A monoclinic crystal rotated about X over ten 3-degree frames, its predicted spots as observations; + // the refinement starts from a perturbed beam, tilt and cell. + XtalRefineProblem RotationProblem(bool reduced, bool weighted) { + DiffractionExperiment exp; + exp.IncidentEnergy_keV(WVL_1A_IN_KEV).BeamX_pxl(1000).BeamY_pxl(1000) + .PoniRot1_rad(0.01).PoniRot2_rad(0.02).DetectorDistance_mm(200); + const auto geom = exp.GetDiffractionGeometry(); + const CrystalLattice latt(40, 50, 80, 90, 95, 90); + const GoniometerAxis axis("omega", 0.0f, 3.0f, Coord(1, 0, 0), std::nullopt); + const gemmi::CrystalSystem sys = gemmi::CrystalSystem::Monoclinic; + + XtalRefineProblem p; + p.crystal_system = sys; + p.beam_and_orientation_only = reduced; + p.distance_mm = 200; + + BraggPrediction prediction; + const BraggPredictionSettings settings{.high_res_A = 1.5, .ewald_dist_cutoff = 0.002}; + for (int img = 0; img < 10; img++) { + const float angle_deg = axis.GetAngle_deg(img) + axis.GetWedge_deg() / 2.0f; + const auto n = prediction.Calc(exp, latt.Multiply(axis.GetTransformationAngle(angle_deg).transpose()), + settings); + p.frame_angle_rad.push_back(angle_deg * PI / 180.0); + for (int i = 0; i < n; i++) { + const auto &r = prediction.GetReflections().at(i); + p.residuals.emplace_back(r.predicted_x + 0.3 * std::sin(i), r.predicted_y + 0.3 * std::cos(i), + geom.GetWavelength_A(), geom.GetPixelSize_mm(), 1.0, 0.0, + angle_deg * PI / 180.0, r.h, r.k, r.l, sys); + p.frame.push_back(img); + if (weighted) + p.weight_sq.push_back(0.2 + 0.6 * (i % 5) / 4.0); + } + } + + p.beam[0] = 1000.0; + p.beam[1] = 997.0; + p.detector_rot[0] = 0.012; + p.detector_rot[1] = 0.018; + p.rot_vec[0] = 1.0; + p.rot_vec[1] = 0.0; + p.rot_vec[2] = 0.0; + double beta = 0; + LatticeToRodriguesLengthsBeta_Mono(CrystalLattice(39.7f, 50.6f, 79.6f, 90.0f, 94.5f, 90.0f), + p.latt_vec0, p.latt_vec1, beta); + p.latt_vec2[0] = beta; + + if (!reduced) { + p.detector_rot_constant = false; + p.rot_vec_constant = false; + p.latt_vec1_constant = false; + p.latt_vec2_constant = false; + for (int i = 0; i < 2; i++) { + p.detector_rot_lower[i] = p.detector_rot[i] - 0.05; + p.detector_rot_upper[i] = p.detector_rot[i] + 0.05; + } + for (int i = 0; i < 3; i++) { + p.latt_vec1_lower[i] = 5.0; + p.latt_vec1_upper[i] = 100.0; + } + p.latt_vec2_lower[0] = PI / 3; + p.latt_vec2_upper[0] = 2 * PI / 3; + p.priors.push_back({XtalRefinePrior::Block::Beam, 1.0, 0.0, p.beam[0], 0.5}); + p.priors.push_back({XtalRefinePrior::Block::DetectorRot, 0.0, 1.0, p.detector_rot[1], 50.0}); + } + p.options.max_iterations = 50; + return p; + } + + void CompareWithCeres(XtalRefineProblem p) { + XtalRefineProblem q = p; + const LMSummary lm = SolveXtalRefine(p, 4); + const ceres::Solver::Summary ref = SolveXtalRefineCeres(q, 4); + + REQUIRE(lm.IsSolutionUsable() == ref.IsSolutionUsable()); + CHECK(lm.iterations == static_cast(ref.iterations.size())); + CHECK(lm.final_cost == Catch::Approx(ref.final_cost).epsilon(1e-9)); + const auto same = [](const double *a, const double *b, int n, double tol) { + for (int i = 0; i < n; i++) + CHECK(a[i] == Catch::Approx(b[i]).margin(tol)); + }; + same(p.beam, q.beam, 2, 1e-7); + same(p.detector_rot, q.detector_rot, 2, 1e-10); + same(p.rot_vec, q.rot_vec, 3, 1e-10); + same(p.latt_vec0, q.latt_vec0, 3, 1e-10); + same(p.latt_vec1, q.latt_vec1, 3, 1e-8); + same(p.latt_vec2, q.latt_vec2, 3, 1e-10); + } +} + +TEST_CASE("XtalRefine_matches_Ceres_full", "[XtalOptimizer]") { + CompareWithCeres(RotationProblem(false, false)); +} + +TEST_CASE("XtalRefine_matches_Ceres_full_weighted", "[XtalOptimizer]") { + CompareWithCeres(RotationProblem(false, true)); +} + +TEST_CASE("XtalRefine_matches_Ceres_beam_orientation", "[XtalOptimizer]") { + CompareWithCeres(RotationProblem(true, false)); +} + +TEST_CASE("XtalRefine_same_answer_at_any_thread_count", "[XtalOptimizer]") { + XtalRefineProblem a = RotationProblem(false, false); + XtalRefineProblem b = a; + SolveXtalRefine(a, 1); + SolveXtalRefine(b, 7); + for (int i = 0; i < 3; i++) { + CHECK(a.latt_vec0[i] == b.latt_vec0[i]); + CHECK(a.latt_vec1[i] == b.latt_vec1[i]); + } + CHECK(a.beam[0] == b.beam[0]); + CHECK(a.beam[1] == b.beam[1]); +} -- 2.54.0 From 56f0c5dd0acca9aed6793b4606771772fcfd2f85 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 10:06:05 +0200 Subject: [PATCH 06/97] CPU spot finder and rotation prediction: same results, less work per pixel AdaptiveSpotFinderCPU::AccumulateRingsBlock reads each pixel once for both the ring histogram and the fused azimuthal profile (was two loops), and no longer keeps the per-ring integer sums: they are taken from the histogram, as the two sigma-clip passes already were (ClipRings(INFINITY)). Integer sums, so the same totals; the profile's float sums keep their pixel order. FlagRow is branch-free and works a 32-pixel word at a time, so it vectorises; pixels outside every ring meet a +inf threshold in an extra ring_thr entry. BraggPredictionRot::Calc takes A*h, A*h + B*k, C*l and 4*S0*S0 out of the inner loops; p0 is the same ((A*h) + (B*k)) + (C*l) as before. Measured on cytc (CPU build, both binaries run concurrently on a loaded box): AccumulateRingsBlock -35%, FlagRow -47%, Calc + Coord ops -30% cycles; whole run -5% cycles. p.mtz md5 unchanged on myob/cytc/thau, CPU and GPU builds. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- .../bragg_prediction/BraggPredictionRot.cpp | 16 +- .../spot_finding/AdaptiveSpotFinderCPU.cpp | 137 ++++++++---------- .../spot_finding/AdaptiveSpotFinderCPU.h | 8 +- 3 files changed, 77 insertions(+), 84 deletions(-) diff --git a/image_analysis/bragg_prediction/BraggPredictionRot.cpp b/image_analysis/bragg_prediction/BraggPredictionRot.cpp index 33b022a6c..e9deb1428 100644 --- a/image_analysis/bragg_prediction/BraggPredictionRot.cpp +++ b/image_analysis/bragg_prediction/BraggPredictionRot.cpp @@ -45,6 +45,7 @@ int BraggPredictionRot::Calc(const DiffractionExperiment &experiment, const Crys const Coord m3 = (m1 % m2).Normalize(); const float m2_S0 = m2 * S0; + const float four_S0_sq = 4 * S0 * S0; const float m3_S0 = m3 * S0; int i = 0; @@ -99,17 +100,24 @@ int BraggPredictionRot::Calc(const DiffractionExperiment &experiment, const Crys cos_phi_limit = std::cos(phi_limit); } + // p0 = A* h + B* k + C* l, evaluated as ((A* h) + (B* k)) + (C* l) exactly as before, with the + // terms that do not change in the inner loops taken out of them. + std::vector Cstar_l(2 * settings.max_l + 1); + for (int l = -settings.max_l; l <= settings.max_l; l++) + Cstar_l[l + settings.max_l] = Cstar * l; + for (int h = -settings.max_h; h <= settings.max_h; h++) { - // Precompute A* h contribution + const Coord Astar_h = Astar * h; for (int k = -settings.max_k; k <= settings.max_k; k++) { - // Accumulate B* k contribution + const Coord AB = Astar_h + Bstar * k; for (int l = -settings.max_l; l <= settings.max_l; l++) { if (systematic_absence(h, k, l, settings.centering)) continue; - Coord p0 = Astar * h + Bstar * k + Cstar * l; + const Coord &Cl = Cstar_l[l + settings.max_l]; + const Coord p0(AB.x + Cl.x, AB.y + Cl.y, AB.z + Cl.z); float p0_sq = p0 * p0; if (p0_sq <= 0.0f || p0_sq > one_over_dmax_sq) @@ -129,7 +137,7 @@ int BraggPredictionRot::Calc(const DiffractionExperiment &experiment, const Crys }; // No solution for Laue equations - if ((rho_sq < p_m3 * p_m3) || (p0_sq > 4 * S0 * S0)) + if ((rho_sq < p_m3 * p_m3) || (p0_sq > four_S0_sq)) continue; // Effective rocking width for this reflection: mosaicity broadened by the bandwidth diff --git a/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp b/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp index 62474e7e3..995df923b 100644 --- a/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp +++ b/image_analysis/spot_finding/AdaptiveSpotFinderCPU.cpp @@ -17,7 +17,7 @@ AdaptiveSpotFinderCPU::AdaptiveSpotFinderCPU(const AzimuthalIntegrationMapping & ring_cnt.assign(nbins, 0); ring_mean.assign(nbins, 0.0f); ring_sigma.assign(nbins, 0.0f); - ring_thr.assign(nbins, 0.0f); + ring_thr.assign(nbins + 1, INFINITY); // the last entry is for pixels outside every ring ring_bkg.assign(nbins, NAN); ring_bits.assign(OutputSize(), 0); ring_hist.assign(nbins * HIST_VALUES, 0); @@ -54,84 +54,54 @@ void AdaptiveSpotFinderCPU::BeginRings() { rings_from_blocks = true; } -// The plain pass over pixels [first, first + n). +// The plain pass over pixels [first, first + n): the histogram of each ring's values, from which +// PlainRings() takes the integer sums, and the fused profile when asked for. One loop reads each pixel +// once for both. void AdaptiveSpotFinderCPU::AccumulateRingsBlock(const ImagePreprocessorBuffer &image, size_t first, size_t n) { const auto &pixel_to_bin = mapping.GetPixelToBin(); const size_t nbins = ring_sum.size(); const float *corrections = mapping.Corrections().data(); - - // Consecutive pixels mostly share a ring, so a ring's sums are held in locals while they do and - // written back when the ring changes: the same additions in the same order, without a store and a - // reload of the same address on every pixel. The azimuthal-integration sums get a loop of their own - // over the block, so that neither loop runs out of registers for its sums. - if (fuse_azint) { - size_t cur = nbins; // the ring held in the locals below; nbins = none - float az_sum = 0.0f, az_sum2 = 0.0f; - uint32_t az_cnt = 0; - for (size_t pxl = first; pxl < first + n; ++pxl) { - const int32_t v = image[pxl]; - if (v == INT32_MIN || v == INT32_MAX) continue; // bad / saturated - const uint16_t b = pixel_to_bin[pxl]; - if (b >= nbins) continue; // masked / out of range (UINT16_MAX) - if (b != cur) { - if (cur != nbins) { - azint_sum[cur] = az_sum; - azint_sum2[cur] = az_sum2; - azint_count[cur] = az_cnt; - } - cur = b; - az_sum = azint_sum[b]; - az_sum2 = azint_sum2[b]; - az_cnt = azint_count[b]; - } - const float val = static_cast(v) * corrections[pxl]; - const float val_sq = val * val; - az_sum += val; - az_sum2 += val_sq; - ++az_cnt; - } - if (cur != nbins) { - azint_sum[cur] = az_sum; - azint_sum2[cur] = az_sum2; - azint_count[cur] = az_cnt; - } - } - - // Values outside the histogram are listed by a second loop, run only when the block has any: a - // call in this loop would leave the sums in memory again. uint32_t *hist = ring_hist.data(); + + // Consecutive pixels mostly share a ring, so the ring's profile sums are held in locals while they + // do and written back when the ring changes: the same additions in the same order, without a store + // and a reload of the same address on every pixel. Values outside the histogram are listed by a + // second loop, run only when the block has any. bool overflow = false; - size_t cur = nbins; - int64_t sum = 0, cnt = 0; - uint64_t sum2 = 0; + size_t cur = nbins; // the ring held in the locals below; nbins = none + float az_sum = 0.0f, az_sum2 = 0.0f; + uint32_t az_cnt = 0; for (size_t pxl = first; pxl < first + n; ++pxl) { const int32_t v = image[pxl]; if (v == INT32_MIN || v == INT32_MAX) continue; // bad / saturated const uint16_t b = pixel_to_bin[pxl]; if (b >= nbins) continue; // masked / out of range (UINT16_MAX) - if (b != cur) { - if (cur != nbins) { - ring_sum[cur] = sum; - ring_sum2[cur] = sum2; - ring_cnt[cur] = cnt; - } - cur = b; - sum = ring_sum[b]; - sum2 = ring_sum2[b]; - cnt = ring_cnt[b]; - } - sum += v; - sum2 += static_cast(static_cast(v) * v); - cnt += 1; - if (v >= 0 && v < HIST_VALUES) + if (static_cast(v) < HIST_VALUES) hist[b * HIST_VALUES + v] += 1; else overflow = true; + if (!fuse_azint) continue; + if (b != cur) { + if (cur != nbins) { + azint_sum[cur] = az_sum; + azint_sum2[cur] = az_sum2; + azint_count[cur] = az_cnt; + } + cur = b; + az_sum = azint_sum[b]; + az_sum2 = azint_sum2[b]; + az_cnt = azint_count[b]; + } + const float val = static_cast(v) * corrections[pxl]; + const float val_sq = val * val; + az_sum += val; + az_sum2 += val_sq; + ++az_cnt; } if (cur != nbins) { - ring_sum[cur] = sum; - ring_sum2[cur] = sum2; - ring_cnt[cur] = cnt; + azint_sum[cur] = az_sum; + azint_sum2[cur] = az_sum2; + azint_count[cur] = az_cnt; } if (overflow) @@ -146,7 +116,8 @@ void AdaptiveSpotFinderCPU::AccumulateRingsBlock(const ImagePreprocessorBuffer & } // A sigma-clip pass over the plain pass's values: each distinct value of a ring meets the same test the -// pixels holding it would, and its pixels are added as a count. +// pixels holding it would, and its pixels are added as a count. Integer sums, so with nothing clipped +// (clip_k = INFINITY) they are the plain sums over the pixels themselves. void AdaptiveSpotFinderCPU::ClipRings(float clip_k) { const size_t nbins = ring_sum.size(); @@ -155,6 +126,7 @@ void AdaptiveSpotFinderCPU::ClipRings(float clip_k) { std::fill(ring_cnt.begin(), ring_cnt.end(), 0); const auto keep = [&](uint16_t b, int32_t v) { + if (std::isinf(clip_k)) return true; const float lo = ring_mean[b] - clip_k * ring_sigma[b]; const float hi = ring_mean[b] + clip_k * ring_sigma[b]; return !(v < lo || v > hi); // exclude peaks / outliers @@ -197,6 +169,7 @@ void AdaptiveSpotFinderCPU::Detect(const ImagePreprocessorBuffer &image, AccumulateRingsBlock(image, 0, static_cast(width) * height); } rings_from_blocks = false; + ClipRings(INFINITY); UpdateRingStatistics(); ClipRings(3.0f); UpdateRingStatistics(); @@ -259,19 +232,31 @@ void AdaptiveSpotFinderCPU::Detect(const ImagePreprocessorBuffer &image, void AdaptiveSpotFinderCPU::FlagRow(const ImagePreprocessorBuffer &image, int32_t row) { const auto &pixel_to_bin = mapping.GetPixelToBin(); - const size_t nbins = ring_thr.size(); + const auto nbins = static_cast(ring_thr.size() - 1); // ring_thr[nbins] is +inf + const float *thr = ring_thr.data(); + const int32_t *img = image.data(); + const uint16_t *bin = pixel_to_bin.data(); const size_t first = static_cast(row) * width; + const size_t end = first + width; - for (size_t pxl = first; pxl < first + width; ++pxl) { - const int32_t v = image[pxl]; - const uint16_t b = pixel_to_bin[pxl]; - bool strong = false; - if (v == INT32_MAX) - strong = true; - else if (v != INT32_MIN && b < nbins && v >= ring_thr[b]) - strong = true; - - if (strong) - ring_bits[pxl / 32] |= 1U << (pxl % 32); + // Saturated is strong, bad is not, and a pixel outside every ring (bin >= nbins) meets the +inf + // threshold. Written without branches and a word of 32 pixels at a time, so that the loop vectorises. + const auto strong = [&](size_t pxl) -> uint32_t { + const int32_t v = img[pxl]; + const uint32_t b = std::min(bin[pxl], nbins); + return (v == INT32_MAX) | ((v != INT32_MIN) & (v >= thr[b])); + }; + size_t pxl = first; + while (pxl < end && pxl % 32 != 0) { + ring_bits[pxl / 32] |= strong(pxl) << (pxl % 32); + ++pxl; } + for (; pxl + 32 <= end; pxl += 32) { + uint32_t word = 0; + for (uint32_t j = 0; j < 32; ++j) + word |= strong(pxl + j) << j; + ring_bits[pxl / 32] |= word; + } + for (; pxl < end; ++pxl) + ring_bits[pxl / 32] |= strong(pxl) << (pxl % 32); } diff --git a/image_analysis/spot_finding/AdaptiveSpotFinderCPU.h b/image_analysis/spot_finding/AdaptiveSpotFinderCPU.h index d0a82aa5f..184ef2379 100644 --- a/image_analysis/spot_finding/AdaptiveSpotFinderCPU.h +++ b/image_analysis/spot_finding/AdaptiveSpotFinderCPU.h @@ -57,7 +57,7 @@ class AdaptiveSpotFinderCPU : public ImageSpotFinderCPU { std::vector ring_cnt; std::vector ring_mean; std::vector ring_sigma; - std::vector ring_thr; + std::vector ring_thr; // nbins + 1: the last is +inf, the threshold of a pixel outside every ring // ring_mean of the last Detect(), NaN where the ring holds too few pixels to be its own background. // Kept separately because ring_mean carries the previous frame's value for an empty ring. std::vector ring_bkg; @@ -65,8 +65,8 @@ class AdaptiveSpotFinderCPU : public ImageSpotFinderCPU { // local-box mask that ImageSpotFinderCPU::Detect leaves in output_buffer. std::vector ring_bits; // The plain pass's valid pixels as a per-ring histogram of their values (HIST_VALUES bins per - // ring) plus a list of the values outside it, so the two sigma-clip passes sum over distinct - // values instead of over the image again. Integer sums, so the same totals. + // ring) plus a list of the values outside it. The plain sums and the two sigma-clip passes are all + // taken from it, over distinct values instead of over the image. Integer sums, so the same totals. static constexpr int32_t HIST_VALUES = 1024; std::vector ring_hist; std::vector> ring_overflow; // (ring, value) @@ -84,7 +84,7 @@ class AdaptiveSpotFinderCPU : public ImageSpotFinderCPU { // Zero the sums of the plain ring pass (and of the fused profile). void ResetRings(); - // One sigma-clip pass over the plain pass's values. + // One sigma-clip pass over the plain pass's values; clip_k = INFINITY gives the plain sums. void ClipRings(float clip_k); // ring_mean / ring_sigma from the current sums. void UpdateRingStatistics(); -- 2.54.0 From 871347b7ad1c27bb4fd094a3fc2877df9d2c7e94 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 10:06:22 +0200 Subject: [PATCH 07/97] rugnux: make the P1 merges of the tail beside the critical path The tail of the canonical pass made five merges one after another. Two of them read nothing the space-group search or the in-symmetry merge decides: the all-observation arm of the search and the P1 cross-check. They are now made on a second RotationScaleMerge engine, ingested beside the run's own before any merge writes per-frame values back, and taken where they were made before; where the run re-ingests (a reindex, a cell change) they are made on the run's engine as before. - RotationScaleMerge::SetWriteBackPerFrameScale(false) keeps a merge that is not the run's answer from writing G/CC/mosaicity onto the outcomes. The P1 cross-check no longer overwrites them, so _plot.txt's scale_G, cc_to_merge and cc_n now describe the merge that was written (in the determined group) instead of the P1 cross-check; the cross-check also no longer leaks its scaling iteration count into the report. - RotationScaleMergeGPU runs on its own non-blocking stream instead of the legacy NULL stream, so the two engines (and a probe pass beside them) do not serialise at every launch and synchronisation. - The anisotropy analysis (mostly ScaledObservations) runs beside tNCS, twinning and the other report-only analyses. p.mtz, p_P1.mtz, p.cif, p.hkl and p_unmerged.mtz are byte-identical on myob/cytc/thau (GPU and CPU builds). Tail on cytc GPU 8.1 -> ~6.8-7.7 s under a loaded box. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- .../scale_merge/RotationScaleMerge.cpp | 3 +- .../scale_merge/RotationScaleMerge.h | 7 + .../scale_merge/RotationScaleMergeGPU.cu | 247 ++++++++++-------- rugnux/Rugnux.cpp | 147 +++++++++-- 4 files changed, 271 insertions(+), 133 deletions(-) diff --git a/image_analysis/scale_merge/RotationScaleMerge.cpp b/image_analysis/scale_merge/RotationScaleMerge.cpp index c32a166f0..6eff70b3e 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.cpp +++ b/image_analysis/scale_merge/RotationScaleMerge.cpp @@ -5699,7 +5699,8 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st ReducePartialGroupMeans(n_groups, partial_mean); ComputePerFrameCC(partial_mean, cc, cc_n); } - FinalizePerFrameScale(cc, cc_n, partial_scaled); + if (write_back_per_frame_scale) + FinalizePerFrameScale(cc, cc_n, partial_scaled); // The filters below remove observations by zeroing corr, which is what takes an observation out of // the 3D combine, the merge and the error model alike (excluding them from the ASU grouping is NOT diff --git a/image_analysis/scale_merge/RotationScaleMerge.h b/image_analysis/scale_merge/RotationScaleMerge.h index ea1947c8f..79278560f 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.h +++ b/image_analysis/scale_merge/RotationScaleMerge.h @@ -119,6 +119,12 @@ public: // compares nothing, asks for it to be left out. Result Run(bool for_search, bool full_stats, bool measure_cc_before_corrections); + // Whether Run() writes the per-frame G / CC / mosaicity back onto the outcomes (on by default). Off + // for a merge that is not the run's answer - the P1 cross-check - so the per-image table and the + // unmerged MTZ describe the merge that was written, and so an engine merging beside another one + // does not write the outcomes they share. + void SetWriteBackPerFrameScale(bool on) { write_back_per_frame_scale = on; } + // Override the high-resolution cut for the next Run() - used to gate the de-novo P1 search pass at // >= 1 without cutting the final in-symmetry merge. Reset to the manual limit afterwards. void SetDMinLimit(std::optional d_min_A) { d_min_limit = d_min_A; } @@ -413,6 +419,7 @@ private: std::unique_ptr gpu_; bool gpu_active_ = false; #endif + bool write_back_per_frame_scale = true; // see SetWriteBackPerFrameScale // --- helpers (each a flat pass; see the .cpp) --- // Turn the per-frame mean background under the reflections (accumulated by the ingest fill loop) into diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.cu b/image_analysis/scale_merge/RotationScaleMergeGPU.cu index 8ed922149..6e54fb380 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.cu +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.cu @@ -18,13 +18,16 @@ namespace { constexpr int BLK = 256; constexpr int MIN_REFLECTIONS = 20; - // Every kernel and copy here is queued on the legacy NULL stream, so - as in BeamCenterFFTGPU - - // no buffer comes from the pool. A pooled buffer is freed with cudaFreeAsync on the thread's - // non-blocking allocation stream, which is not ordered after the NULL stream. Each entry point - // below waits for its own work before it returns, so no free here has yet overtaken a read, but - // that holds only by that convention, and not at all for a free while unwinding from a failed - // call; nor can compute-sanitizer --track-stream-ordered-races see it, and it reported every - // reassigned merge buffer as a use-after-free. cudaFree synchronises the device first. + // Every kernel and copy here is queued on the instance's own stream (Impl::stream), not on the + // legacy NULL stream: two merges made side by side - the run's and the one it makes ahead of time - + // and the image analysis of a probe pass beside them would otherwise wait for each other's work at + // every launch and every synchronisation. As in BeamCenterFFTGPU no buffer comes from the pool. A + // pooled buffer is freed with cudaFreeAsync on the thread's allocation stream, which is not + // ordered after this one. Each entry point below waits for its own work before it returns, so no + // free here has yet overtaken a read, but that holds only by that convention, and not at all for a + // free while unwinding from a failed call; nor can compute-sanitizer --track-stream-ordered-races + // see it, and it reported every reassigned merge buffer as a use-after-free. cudaFree + // synchronises the device first. constexpr CudaAlloc ALLOC = CudaAlloc::Synchronous; __device__ __forceinline__ double SafeInvD(double x, double fallback) { @@ -636,6 +639,16 @@ namespace { } } + void CudaCheck(cudaError_t e, const char *what); + + // A copy on the instance's stream, waited for - what cudaMemcpy on the NULL stream was, without also + // waiting for every other stream on the card. + void CopyAndWait(void *dst, const void *src, size_t bytes, cudaMemcpyKind kind, cudaStream_t s, + const char *what) { + CudaCheck(cudaMemcpyAsync(dst, src, bytes, kind, s), what); + CudaCheck(cudaStreamSynchronize(s), what); + } + void CudaCheck(cudaError_t e, const char *what) { if (e != cudaSuccess) throw JFJochException(JFJochExceptionCategory::GPUCUDAError, @@ -703,6 +716,9 @@ namespace { } struct RotationScaleMergeGPU::Impl { + // First, so it goes last: the buffers below are freed before the stream their work ran on. + std::unique_ptr stream; + cudaStream_t s() const { return stream->get(); } int device = 0; // the GPU this instance's buffers live on bool available = false; int n_obs = 0, n_frames = 0, n_groups = 0; @@ -736,7 +752,7 @@ struct RotationScaleMergeGPU::Impl { void Upload(CudaDevicePtr &dst, const T *src, int n) const { dst = Alloc(std::max(1, n)); if (n > 0) - CudaCheck(cudaMemcpy(dst.get(), src, size_t(n) * sizeof(T), cudaMemcpyHostToDevice), "upload"); + CopyAndWait(dst.get(), src, size_t(n) * sizeof(T), cudaMemcpyHostToDevice, s(), "upload"); } // immutable per-obs @@ -833,6 +849,7 @@ RotationScaleMergeGPU::RotationScaleMergeGPU() : impl_(std::make_unique()) // can put the caller's device back instead of leaving the thread moved. impl_->device = 0; DeviceGuard guard(impl_->device, true); + impl_->stream = std::make_unique(); impl_->available = true; } } @@ -878,10 +895,10 @@ void RotationScaleMergeGPU::SetPartialsLayout(int n_obs, int n_frames, namespace { template - void UploadChunk(CudaDevicePtr &dst, int offset, int count, const T *v) { + void UploadChunk(CudaDevicePtr &dst, int offset, int count, const T *v, cudaStream_t s) { if (count > 0) - CudaCheck(cudaMemcpy(dst.get() + offset, v, size_t(count) * sizeof(T), - cudaMemcpyHostToDevice), "upload chunk"); + CopyAndWait(dst.get() + offset, v, size_t(count) * sizeof(T), cudaMemcpyHostToDevice, s, + "upload chunk"); } } @@ -889,34 +906,34 @@ void RotationScaleMergeGPU::SetObsField(ObsField f, int offset, int count, const DeviceGuard guard(impl_->device, impl_->available); auto &d = *impl_; switch (f) { - case ObsField::I: UploadChunk(d.I, offset, count, v); break; - case ObsField::Sigma: UploadChunk(d.sigma, offset, count, v); break; - case ObsField::PrescalingCorr: UploadChunk(d.prescaling_corr, offset, count, v); break; - case ObsField::Partiality: UploadChunk(d.partiality, offset, count, v); break; - case ObsField::Zeta: UploadChunk(d.zeta, offset, count, v); break; - case ObsField::Corr0: UploadChunk(d.corr, offset, count, v); break; - case ObsField::Bkg: UploadChunk(d.bkg, offset, count, v); break; - case ObsField::VarBkg: UploadChunk(d.var_bkg, offset, count, v); break; - case ObsField::ImageNumber: UploadChunk(d.image_number, offset, count, v); break; - case ObsField::D: UploadChunk(d.d_obs, offset, count, v); break; - case ObsField::Px: UploadChunk(d.px_obs, offset, count, v); break; - case ObsField::Py: UploadChunk(d.py_obs, offset, count, v); break; + case ObsField::I: UploadChunk(d.I, offset, count, v, impl_->s()); break; + case ObsField::Sigma: UploadChunk(d.sigma, offset, count, v, impl_->s()); break; + case ObsField::PrescalingCorr: UploadChunk(d.prescaling_corr, offset, count, v, impl_->s()); break; + case ObsField::Partiality: UploadChunk(d.partiality, offset, count, v, impl_->s()); break; + case ObsField::Zeta: UploadChunk(d.zeta, offset, count, v, impl_->s()); break; + case ObsField::Corr0: UploadChunk(d.corr, offset, count, v, impl_->s()); break; + case ObsField::Bkg: UploadChunk(d.bkg, offset, count, v, impl_->s()); break; + case ObsField::VarBkg: UploadChunk(d.var_bkg, offset, count, v, impl_->s()); break; + case ObsField::ImageNumber: UploadChunk(d.image_number, offset, count, v, impl_->s()); break; + case ObsField::D: UploadChunk(d.d_obs, offset, count, v, impl_->s()); break; + case ObsField::Px: UploadChunk(d.px_obs, offset, count, v, impl_->s()); break; + case ObsField::Py: UploadChunk(d.py_obs, offset, count, v, impl_->s()); break; } } void RotationScaleMergeGPU::SetObsFrame(int offset, int count, const int32_t *frame) { DeviceGuard guard(impl_->device, impl_->available); - UploadChunk(impl_->frame, offset, count, frame); + UploadChunk(impl_->frame, offset, count, frame, impl_->s()); } void RotationScaleMergeGPU::SetObsOnIce(int offset, int count, const uint8_t *on_ice) { DeviceGuard guard(impl_->device, impl_->available); - UploadChunk(impl_->on_ice, offset, count, on_ice); + UploadChunk(impl_->on_ice, offset, count, on_ice, impl_->s()); } void RotationScaleMergeGPU::SetObsClipped(int offset, int count, const uint8_t *clipped) { DeviceGuard guard(impl_->device, impl_->available); - UploadChunk(impl_->clipped, offset, count, clipped); + UploadChunk(impl_->clipped, offset, count, clipped, impl_->s()); } void RotationScaleMergeGPU::SetGroups(int n_groups, const int32_t *group, const int32_t *group_perm, @@ -934,8 +951,8 @@ void RotationScaleMergeGPU::SetGroups(int n_groups, const int32_t *group, const void RotationScaleMergeGPU::SetCorr(const float *corr) { DeviceGuard guard(impl_->device, impl_->available); - CudaCheck(cudaMemcpy(impl_->corr.get(), corr, size_t(impl_->n_obs) * sizeof(float), - cudaMemcpyHostToDevice), "upload corr"); + CopyAndWait(impl_->corr.get(), corr, size_t(impl_->n_obs) * sizeof(float), + cudaMemcpyHostToDevice, impl_->s(), "upload corr"); } void RotationScaleMergeGPU::ScalePartials(int iters, double min_partiality, bool /*has_d_min*/) { @@ -943,42 +960,42 @@ void RotationScaleMergeGPU::ScalePartials(int iters, double min_partiality, bool auto &d = *impl_; // Reset per call: the host keeps the G of a frame across calls (RunScalingLoop), so a frame this // call did not fit must read as unfitted, not as fitted with the value of the call before. - CudaCheck(cudaMemset(d.scaled.get(), 0, size_t(d.n_frames) * sizeof(uint8_t)), "memset scaled"); - CudaCheck(cudaMemset(d.g.get(), 0, size_t(d.n_frames) * sizeof(double)), "memset g"); // unscaled g unused + CudaCheck(cudaMemsetAsync(d.scaled.get(), 0, size_t(d.n_frames) * sizeof(uint8_t), impl_->s()), "memset scaled"); + CudaCheck(cudaMemsetAsync(d.g.get(), 0, size_t(d.n_frames) * sizeof(double), impl_->s()), "memset g"); // unscaled g unused const int obs_blocks = (d.n_obs + BLK - 1) / BLK; const int upd_blocks = std::min(65535, obs_blocks); const int grp_blocks = std::min(65535, (d.n_groups + BLK - 1) / BLK); for (int it = 0; it < iters; ++it) { - ReduceGroupMeansKernel<<>>(d.n_groups, min_partiality, + ReduceGroupMeansKernel<<s()>>>(d.n_groups, min_partiality, d.group_perm.get(), d.group_start.get(), d.group_count.get(), d.I.get(), d.sigma.get(), d.partiality.get(), d.corr.get(), d.group_mean.get()); CudaCheck(cudaGetLastError(), "ReduceGroupMeansKernel launch"); - PrepScaleObsKernel<<>>(d.n_obs, min_partiality, d.group.get(), d.partiality.get(), d.prescaling_corr.get(), + PrepScaleObsKernel<<s()>>>(d.n_obs, min_partiality, d.group.get(), d.partiality.get(), d.prescaling_corr.get(), d.zeta.get(), d.on_ice.get(), d.group_mean.get(), d.sigma.get(), d.inv_sigma.get(), d.sco_coeff.get(), d.sco_ok.get()); CudaCheck(cudaGetLastError(), "PrepScaleObsKernel launch"); - FitPerFrameGKernel<<>>(d.n_frames, d.frame_start.get(), d.frame_count.get(), + FitPerFrameGKernel<<s()>>>(d.n_frames, d.frame_start.get(), d.frame_count.get(), d.I.get(), d.inv_sigma.get(), d.sco_coeff.get(), d.sco_ok.get(), nullptr, d.g.get(), d.scaled.get()); CudaCheck(cudaGetLastError(), "FitPerFrameGKernel launch"); - UpdateCorrKernel<<>>(d.n_obs, d.frame.get(), d.prescaling_corr.get(), d.partiality.get(), + UpdateCorrKernel<<s()>>>(d.n_obs, d.frame.get(), d.prescaling_corr.get(), d.partiality.get(), d.g.get(), d.scaled.get(), d.corr.get()); } CudaCheck(cudaGetLastError(), "kernel launch"); - CudaCheck(cudaDeviceSynchronize(), "scale sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "scale sync"); } void RotationScaleMergeGPU::GetCorr(float *corr_out) const { DeviceGuard guard(impl_->device, impl_->available); - CudaCheck(cudaMemcpy(corr_out, impl_->corr.get(), size_t(impl_->n_obs) * sizeof(float), - cudaMemcpyDeviceToHost), "download corr"); + CopyAndWait(corr_out, impl_->corr.get(), size_t(impl_->n_obs) * sizeof(float), + cudaMemcpyDeviceToHost, impl_->s(), "download corr"); } void RotationScaleMergeGPU::GetG(double *g_out, uint8_t *scaled_out) const { DeviceGuard guard(impl_->device, impl_->available); - CudaCheck(cudaMemcpy(g_out, impl_->g.get(), size_t(impl_->n_frames) * sizeof(double), - cudaMemcpyDeviceToHost), "download g"); - CudaCheck(cudaMemcpy(scaled_out, impl_->scaled.get(), size_t(impl_->n_frames) * sizeof(uint8_t), - cudaMemcpyDeviceToHost), "download scaled"); + CopyAndWait(g_out, impl_->g.get(), size_t(impl_->n_frames) * sizeof(double), + cudaMemcpyDeviceToHost, impl_->s(), "download g"); + CopyAndWait(scaled_out, impl_->scaled.get(), size_t(impl_->n_frames) * sizeof(uint8_t), + cudaMemcpyDeviceToHost, impl_->s(), "download scaled"); } void RotationScaleMergeGPU::SetFrameCellOk(const uint8_t *frame_cell_ok) { @@ -1030,19 +1047,19 @@ void RotationScaleMergeGPU::MergeEmSamples(bool for_search, double min_partialit const int grp_blocks = std::min(65535, (ng + BLK - 1) / BLK); const int obs_blocks = std::min(65535, (nf + BLK - 1) / BLK); - MergeEmStatsKernel<<>>(p); - MergeSamplesKernel<<>>(nf, p); + MergeEmStatsKernel<<s()>>>(p); + MergeSamplesKernel<<s()>>>(nf, p); CudaCheck(cudaGetLastError(), "merge em/samples launch"); - CudaCheck(cudaDeviceSynchronize(), "merge em/samples sync"); - CudaCheck(cudaMemcpy(em_mean_out, d.m_em_mean.get(), size_t(ng) * sizeof(double), - cudaMemcpyDeviceToHost), "dl em_mean"); - CudaCheck(cudaMemcpy(cnt_out, d.m_cnt.get(), size_t(ng) * sizeof(int32_t), - cudaMemcpyDeviceToHost), "dl cnt"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "merge em/samples sync"); + CopyAndWait(em_mean_out, d.m_em_mean.get(), size_t(ng) * sizeof(double), + cudaMemcpyDeviceToHost, impl_->s(), "dl em_mean"); + CopyAndWait(cnt_out, d.m_cnt.get(), size_t(ng) * sizeof(int32_t), + cudaMemcpyDeviceToHost, impl_->s(), "dl cnt"); if (nf > 0) { - CudaCheck(cudaMemcpy(s2_out, d.m_s2.get(), size_t(nf) * sizeof(double), cudaMemcpyDeviceToHost), "dl s2"); - CudaCheck(cudaMemcpy(I2_out, d.m_I2.get(), size_t(nf) * sizeof(double), cudaMemcpyDeviceToHost), "dl I2"); - CudaCheck(cudaMemcpy(dev2_out, d.m_dev2.get(), size_t(nf) * sizeof(double), cudaMemcpyDeviceToHost), "dl dev2"); - CudaCheck(cudaMemcpy(valid_out, d.m_valid.get(), size_t(nf) * sizeof(uint8_t), cudaMemcpyDeviceToHost), "dl valid"); + CopyAndWait(s2_out, d.m_s2.get(), size_t(nf) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl s2"); + CopyAndWait(I2_out, d.m_I2.get(), size_t(nf) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl I2"); + CopyAndWait(dev2_out, d.m_dev2.get(), size_t(nf) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl dev2"); + CopyAndWait(valid_out, d.m_valid.get(), size_t(nf) * sizeof(uint8_t), cudaMemcpyDeviceToHost, impl_->s(), "dl valid"); } } @@ -1091,10 +1108,10 @@ void RotationScaleMergeGPU::MergeAccum(double error_model_a, double error_model_ p.rejected_obs = d.m_rejected.get(); const int grp_blocks = std::min(65535, (ng + BLK - 1) / BLK); - MergeAccumKernel<<>>(p); + MergeAccumKernel<<s()>>>(p); CudaCheck(cudaGetLastError(), "merge accum launch"); - CudaCheck(cudaDeviceSynchronize(), "merge accum sync"); - CudaCheck(cudaMemcpy(rejected_obs, d.m_rejected.get(), size_t(d.n_fulls) * sizeof(uint8_t), cudaMemcpyDeviceToHost), + CudaCheck(cudaStreamSynchronize(impl_->s()), "merge accum sync"); + CopyAndWait(rejected_obs, d.m_rejected.get(), size_t(d.n_fulls) * sizeof(uint8_t), cudaMemcpyDeviceToHost, impl_->s(), "dl rejected_obs"); } @@ -1105,7 +1122,7 @@ void RotationScaleMergeGPU::MergeAccumRange(int g0, int n, double *swI, double * DeviceGuard guard(impl_->device, impl_->available); auto &d = *impl_; auto dl = [&](void *h, const auto &s) { - CudaCheck(cudaMemcpy(h, s.get() + g0, size_t(n) * sizeof(*s.get()), cudaMemcpyDeviceToHost), + CopyAndWait(h, s.get() + g0, size_t(n) * sizeof(*s.get()), cudaMemcpyDeviceToHost, impl_->s(), "dl accum"); }; dl(swI, d.a_swI); dl(sw, d.a_sw); dl(swIh0, d.a_swIh0); dl(swIh1, d.a_swIh1); dl(swh0, d.a_swh0); dl(swh1, d.a_swh1); dl(swh_typ0, d.a_swht0); dl(swh_typ1, d.a_swht1); @@ -1139,17 +1156,17 @@ void RotationScaleMergeGPU::MergeRmeas(const double *merged_I, double *absdev, d p.rejected_obs = d.m_rejected.get(); const int grp_blocks = std::min(65535, (ng + BLK - 1) / BLK); - MergeRmeasKernel<<>>(p); + MergeRmeasKernel<<s()>>>(p); CudaCheck(cudaGetLastError(), "merge rmeas launch"); - CudaCheck(cudaDeviceSynchronize(), "merge rmeas sync"); - CudaCheck(cudaMemcpy(absdev, d.r_absdev.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost), "dl absdev"); - CudaCheck(cudaMemcpy(sumI, d.r_sumI.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost), "dl sumI"); - CudaCheck(cudaMemcpy(wabsdev, d.r_wabsdev.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost), "dl wabsdev"); - CudaCheck(cudaMemcpy(wsumI, d.r_wsumI.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost), "dl wsumI"); - CudaCheck(cudaMemcpy(sumv, d.r_sumv.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost), "dl sumv"); - CudaCheck(cudaMemcpy(sumv2, d.r_sumv2.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost), "dl sumv2"); - CudaCheck(cudaMemcpy(n, d.r_n.get(), size_t(ng) * sizeof(int32_t), cudaMemcpyDeviceToHost), "dl rn"); - CudaCheck(cudaMemcpy(nusable, d.r_nusable.get(), size_t(ng) * sizeof(int32_t), cudaMemcpyDeviceToHost), "dl rnusable"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "merge rmeas sync"); + CopyAndWait(absdev, d.r_absdev.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl absdev"); + CopyAndWait(sumI, d.r_sumI.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl sumI"); + CopyAndWait(wabsdev, d.r_wabsdev.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl wabsdev"); + CopyAndWait(wsumI, d.r_wsumI.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl wsumI"); + CopyAndWait(sumv, d.r_sumv.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl sumv"); + CopyAndWait(sumv2, d.r_sumv2.get(), size_t(ng) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), "dl sumv2"); + CopyAndWait(n, d.r_n.get(), size_t(ng) * sizeof(int32_t), cudaMemcpyDeviceToHost, impl_->s(), "dl rn"); + CopyAndWait(nusable, d.r_nusable.get(), size_t(ng) * sizeof(int32_t), cudaMemcpyDeviceToHost, impl_->s(), "dl rnusable"); } void RotationScaleMergeGPU::SmoothCorr(const uint8_t *apply, const double *ratio) { @@ -1158,10 +1175,10 @@ void RotationScaleMergeGPU::SmoothCorr(const uint8_t *apply, const double *ratio d.Upload(d.smooth_apply, apply, d.n_frames); d.Upload(d.smooth_ratio, ratio, d.n_frames); const int blocks = std::min(65535, (d.n_obs + BLK - 1) / BLK); - SmoothCorrKernel<<>>(d.n_obs, d.frame.get(), d.smooth_apply.get(), + SmoothCorrKernel<<s()>>>(d.n_obs, d.frame.get(), d.smooth_apply.get(), d.smooth_ratio.get(), d.corr.get()); CudaCheck(cudaGetLastError(), "smooth corr launch"); - CudaCheck(cudaDeviceSynchronize(), "smooth corr sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "smooth corr sync"); } void RotationScaleMergeGPU::SmoothFullsCorr(const uint8_t *apply, const double *ratio) { @@ -1171,23 +1188,23 @@ void RotationScaleMergeGPU::SmoothFullsCorr(const uint8_t *apply, const double * d.Upload(d.smooth_apply, apply, d.n_frames); d.Upload(d.smooth_ratio, ratio, d.n_frames); const int blocks = std::min(65535, (d.n_fulls + BLK - 1) / BLK); - SmoothCorrKernel<<>>(d.n_fulls, d.f_frame.get(), d.smooth_apply.get(), + SmoothCorrKernel<<s()>>>(d.n_fulls, d.f_frame.get(), d.smooth_apply.get(), d.smooth_ratio.get(), d.f_corr.get()); CudaCheck(cudaGetLastError(), "smooth fulls corr launch"); - CudaCheck(cudaDeviceSynchronize(), "smooth fulls corr sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "smooth fulls corr sync"); } int64_t RotationScaleMergeGPU::FilterCorrByZeta(double min_zeta) { DeviceGuard guard(impl_->device, impl_->available); auto &d = *impl_; CudaDevicePtr dropped = d.Alloc(1); - CudaCheck(cudaMemset(dropped.get(), 0, sizeof(unsigned long long)), "zero zeta drop count"); + CudaCheck(cudaMemsetAsync(dropped.get(), 0, sizeof(unsigned long long), impl_->s()), "zero zeta drop count"); const int blocks = std::min(65535, (d.n_obs + BLK - 1) / BLK); - FilterZetaKernel<<>>(d.n_obs, min_zeta, d.zeta.get(), d.corr.get(), dropped.get()); + FilterZetaKernel<<s()>>>(d.n_obs, min_zeta, d.zeta.get(), d.corr.get(), dropped.get()); CudaCheck(cudaGetLastError(), "zeta filter launch"); - CudaCheck(cudaDeviceSynchronize(), "zeta filter sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "zeta filter sync"); unsigned long long n = 0; - CudaCheck(cudaMemcpy(&n, dropped.get(), sizeof(unsigned long long), cudaMemcpyDeviceToHost), + CopyAndWait(&n, dropped.get(), sizeof(unsigned long long), cudaMemcpyDeviceToHost, impl_->s(), "dl zeta drop count"); return static_cast(n); } @@ -1197,9 +1214,9 @@ void RotationScaleMergeGPU::FilterCorrByFrame(const uint8_t *reject) { auto &d = *impl_; d.Upload(d.filter_reject, reject, d.n_frames); const int blocks = std::min(65535, (d.n_obs + BLK - 1) / BLK); - FilterFrameKernel<<>>(d.n_obs, d.frame.get(), d.filter_reject.get(), d.corr.get()); + FilterFrameKernel<<s()>>>(d.n_obs, d.frame.get(), d.filter_reject.get(), d.corr.get()); CudaCheck(cudaGetLastError(), "frame filter launch"); - CudaCheck(cudaDeviceSynchronize(), "frame filter sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "frame filter sync"); } void RotationScaleMergeGPU::ComputePartialCC(double min_partiality, double *cc_out, int64_t *cc_n_out) { @@ -1208,19 +1225,19 @@ void RotationScaleMergeGPU::ComputePartialCC(double min_partiality, double *cc_o const int grp_blocks = std::min(65535, (d.n_groups + BLK - 1) / BLK); // Post-smooth group means (reuse the scaling reduce; reads the resident, smoothed corr), then the // per-frame CC over the resident partials. Only the tiny per-frame cc/cc_n come back to the host. - ReduceGroupMeansKernel<<>>(d.n_groups, min_partiality, + ReduceGroupMeansKernel<<s()>>>(d.n_groups, min_partiality, d.group_perm.get(), d.group_start.get(), d.group_count.get(), d.I.get(), d.sigma.get(), d.partiality.get(), d.corr.get(), d.group_mean.get()); CudaCheck(cudaGetLastError(), "ReduceGroupMeansKernel launch"); - PerFrameCCKernel<<>>(d.n_frames, min_partiality, + PerFrameCCKernel<<s()>>>(d.n_frames, min_partiality, d.frame_start.get(), d.frame_count.get(), d.I.get(), d.sigma.get(), d.partiality.get(), d.corr.get(), d.on_ice.get(), d.group.get(), d.group_mean.get(), d.cc.get(), d.cc_n.get()); CudaCheck(cudaGetLastError(), "partial CC launch"); - CudaCheck(cudaDeviceSynchronize(), "partial CC sync"); - CudaCheck(cudaMemcpy(cc_out, d.cc.get(), size_t(d.n_frames) * sizeof(double), - cudaMemcpyDeviceToHost), "download cc"); - CudaCheck(cudaMemcpy(cc_n_out, d.cc_n.get(), size_t(d.n_frames) * sizeof(int64_t), - cudaMemcpyDeviceToHost), "download cc_n"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "partial CC sync"); + CopyAndWait(cc_out, d.cc.get(), size_t(d.n_frames) * sizeof(double), + cudaMemcpyDeviceToHost, impl_->s(), "download cc"); + CopyAndWait(cc_n_out, d.cc_n.get(), size_t(d.n_frames) * sizeof(int64_t), + cudaMemcpyDeviceToHost, impl_->s(), "download cc_n"); } void RotationScaleMergeGPU::SetRawRuns(int n_runs, int n_perm, const int32_t *perm, @@ -1247,8 +1264,8 @@ int RotationScaleMergeGPU::Combine(const int32_t *rawrun_group, double min_parti float max_frame_gap) { DeviceGuard guard(impl_->device, impl_->available); auto &d = *impl_; - CudaCheck(cudaMemcpy(d.rr_group.get(), rawrun_group, size_t(d.n_runs) * sizeof(int32_t), - cudaMemcpyHostToDevice), "upload rr_group"); + CopyAndWait(d.rr_group.get(), rawrun_group, size_t(d.n_runs) * sizeof(int32_t), + cudaMemcpyHostToDevice, impl_->s(), "upload rr_group"); CombineParams p{}; p.n_runs = d.n_runs; @@ -1267,13 +1284,13 @@ int RotationScaleMergeGPU::Combine(const int32_t *rawrun_group, double min_parti const int blocks = std::min(65535, (d.n_runs + BLK - 1) / BLK); // Count pass: how many fulls each run emits. - CombineKernel<<>>(p); + CombineKernel<<s()>>>(p); CudaCheck(cudaGetLastError(), "combine count launch"); // Exclusive prefix sum on the host (deterministic) -> per-run output offset + total fulls. std::vector nevents(d.n_runs); - CudaCheck(cudaMemcpy(nevents.data(), d.rr_nevents.get(), size_t(d.n_runs) * sizeof(int32_t), - cudaMemcpyDeviceToHost), "download nevents"); + CopyAndWait(nevents.data(), d.rr_nevents.get(), size_t(d.n_runs) * sizeof(int32_t), + cudaMemcpyDeviceToHost, impl_->s(), "download nevents"); std::vector offset(d.n_runs); int64_t acc = 0; for (int r = 0; r < d.n_runs; ++r) { offset[r] = static_cast(acc); acc += nevents[r]; } @@ -1292,8 +1309,8 @@ int RotationScaleMergeGPU::Combine(const int32_t *rawrun_group, double min_parti d.f_rlp = d.Alloc(nf); d.f_zeta = d.Alloc(nf); d.f_inv_sigma = d.Alloc(nf); d.f_sco_coeff = d.Alloc(nf); d.f_sco_ok = d.Alloc(nf); - CudaCheck(cudaMemcpy(d.rr_offset.get(), offset.data(), size_t(d.n_runs) * sizeof(int32_t), - cudaMemcpyHostToDevice), "upload offset"); + CopyAndWait(d.rr_offset.get(), offset.data(), size_t(d.n_runs) * sizeof(int32_t), + cudaMemcpyHostToDevice, impl_->s(), "upload offset"); p.rr_offset = d.rr_offset.get(); p.f_h = d.f_h.get(); p.f_k = d.f_k.get(); p.f_l = d.f_l.get(); @@ -1303,10 +1320,10 @@ int RotationScaleMergeGPU::Combine(const int32_t *rawrun_group, double min_parti p.f_var_bkg = d.f_var_bkg.get(); p.f_var_per_I = d.f_var_per_I.get(); p.f_on_ice = d.f_on_ice.get(); p.f_clipped = d.f_clipped.get(); if (d.n_fulls > 0) { - CombineKernel<<>>(p); + CombineKernel<<s()>>>(p); CudaCheck(cudaGetLastError(), "combine emit launch"); } - CudaCheck(cudaDeviceSynchronize(), "combine sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "combine sync"); return d.n_fulls; } @@ -1318,7 +1335,7 @@ void RotationScaleMergeGPU::GetFulls(int32_t *h, int32_t *k, int32_t *l, float * const size_t n = static_cast(dd.n_fulls); if (n == 0) return; auto dl = [&](void *dst, const void *src, size_t bytes) { - CudaCheck(cudaMemcpy(dst, src, bytes, cudaMemcpyDeviceToHost), "download fulls"); + CopyAndWait(dst, src, bytes, cudaMemcpyDeviceToHost, impl_->s(), "download fulls"); }; dl(h, dd.f_h.get(), n * sizeof(int32_t)); dl(k, dd.f_k.get(), n * sizeof(int32_t)); dl(l, dd.f_l.get(), n * sizeof(int32_t)); dl(frame, dd.f_frame.get(), n * sizeof(int32_t)); @@ -1334,8 +1351,8 @@ void RotationScaleMergeGPU::GetFullsKeys(int32_t *frame, int32_t *group) const { const auto &d = *impl_; if (d.n_fulls == 0) return; const size_t bytes = size_t(d.n_fulls) * sizeof(int32_t); - CudaCheck(cudaMemcpy(frame, d.f_frame.get(), bytes, cudaMemcpyDeviceToHost), "download f_frame"); - CudaCheck(cudaMemcpy(group, d.f_group.get(), bytes, cudaMemcpyDeviceToHost), "download f_group"); + CopyAndWait(frame, d.f_frame.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "download f_frame"); + CopyAndWait(group, d.f_group.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "download f_group"); } void RotationScaleMergeGPU::SetFullsFrameCSR(const int32_t *frame_perm, int n_perm, @@ -1363,15 +1380,15 @@ void RotationScaleMergeGPU::ResetFullsScale() { if (nf == 0) return; const int obs_blocks = std::min(65535, (nf + BLK - 1) / BLK); // Unity model: partiality/prescaling_corr/zeta = 1 so coeff = mean; corr starts at 1. - FillKernel<<>>(d.f_corr.get(), nf, 1.0f); + FillKernel<<s()>>>(d.f_corr.get(), nf, 1.0f); CudaCheck(cudaGetLastError(), "FillKernel launch"); - FillKernel<<>>(d.f_partiality.get(), nf, 1.0f); + FillKernel<<s()>>>(d.f_partiality.get(), nf, 1.0f); CudaCheck(cudaGetLastError(), "FillKernel launch"); - FillKernel<<>>(d.f_rlp.get(), nf, 1.0f); + FillKernel<<s()>>>(d.f_rlp.get(), nf, 1.0f); CudaCheck(cudaGetLastError(), "FillKernel launch"); - FillKernel<<>>(d.f_zeta.get(), nf, 1.0f); + FillKernel<<s()>>>(d.f_zeta.get(), nf, 1.0f); CudaCheck(cudaGetLastError(), "FillKernel launch"); - CudaCheck(cudaDeviceSynchronize(), "reset fulls scale sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "reset fulls scale sync"); } void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality) { @@ -1383,38 +1400,38 @@ void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality) { const int grp_blocks = std::min(65535, (d.n_groups + BLK - 1) / BLK); // Reset per call, as ScalePartials: the host keeps the G of a frame across calls. - CudaCheck(cudaMemset(d.scaled.get(), 0, size_t(d.n_frames) * sizeof(uint8_t)), "memset f scaled"); - CudaCheck(cudaMemset(d.g.get(), 0, size_t(d.n_frames) * sizeof(double)), "memset f g"); + CudaCheck(cudaMemsetAsync(d.scaled.get(), 0, size_t(d.n_frames) * sizeof(uint8_t), impl_->s()), "memset f scaled"); + CudaCheck(cudaMemsetAsync(d.g.get(), 0, size_t(d.n_frames) * sizeof(double), impl_->s()), "memset f g"); for (int it = 0; it < iters; ++it) { - ReduceGroupMeansKernel<<>>(d.n_groups, min_partiality, + ReduceGroupMeansKernel<<s()>>>(d.n_groups, min_partiality, d.f_gperm.get(), d.f_gstart.get(), d.f_gcount.get(), d.f_I.get(), d.f_sigma.get(), d.f_partiality.get(), d.f_corr.get(), d.group_mean.get()); CudaCheck(cudaGetLastError(), "ReduceGroupMeansKernel launch"); // Not grid-stride, so its grid has to cover every full - unlike the grid-stride kernels // below, which the 65535 cap is there for. Capped, it would silently leave the tail of // sco_coeff/sco_ok stale above 16.8M fulls. - PrepScaleObsKernel<<<(nf + BLK - 1) / BLK, BLK>>>(nf, min_partiality, d.f_group.get(), d.f_partiality.get(), + PrepScaleObsKernel<<<(nf + BLK - 1) / BLK, BLK, 0, impl_->s()>>>(nf, min_partiality, d.f_group.get(), d.f_partiality.get(), d.f_rlp.get(), d.f_zeta.get(), d.f_on_ice.get(), d.group_mean.get(), d.f_sigma.get(), d.f_inv_sigma.get(), d.f_sco_coeff.get(), d.f_sco_ok.get()); CudaCheck(cudaGetLastError(), "PrepScaleObsKernel launch"); - FitPerFrameGKernel<<>>(d.n_frames, + FitPerFrameGKernel<<s()>>>(d.n_frames, d.f_frame_start.get(), d.f_frame_count.get(), d.f_I.get(), d.f_inv_sigma.get(), d.f_sco_coeff.get(), d.f_sco_ok.get(), d.f_frame_perm.get(), d.g.get(), d.scaled.get()); CudaCheck(cudaGetLastError(), "FitPerFrameGKernel launch"); - UpdateCorrKernel<<>>(nf, d.f_frame.get(), d.f_rlp.get(), d.f_partiality.get(), + UpdateCorrKernel<<s()>>>(nf, d.f_frame.get(), d.f_rlp.get(), d.f_partiality.get(), d.g.get(), d.scaled.get(), d.f_corr.get()); } CudaCheck(cudaGetLastError(), "scale fulls launch"); - CudaCheck(cudaDeviceSynchronize(), "scale fulls sync"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "scale fulls sync"); } void RotationScaleMergeGPU::GetFullsCorr(float *corr) const { DeviceGuard guard(impl_->device, impl_->available); const auto &d = *impl_; if (d.n_fulls == 0) return; - CudaCheck(cudaMemcpy(corr, d.f_corr.get(), size_t(d.n_fulls) * sizeof(float), - cudaMemcpyDeviceToHost), "download f_corr"); + CopyAndWait(corr, d.f_corr.get(), size_t(d.n_fulls) * sizeof(float), + cudaMemcpyDeviceToHost, impl_->s(), "download f_corr"); } void RotationScaleMergeGPU::GetFullsPxPy(float *px, float *py) const { @@ -1422,8 +1439,8 @@ void RotationScaleMergeGPU::GetFullsPxPy(float *px, float *py) const { const auto &d = *impl_; if (d.n_fulls == 0) return; const size_t bytes = size_t(d.n_fulls) * sizeof(float); - CudaCheck(cudaMemcpy(px, d.f_px.get(), bytes, cudaMemcpyDeviceToHost), "download f_px"); - CudaCheck(cudaMemcpy(py, d.f_py.get(), bytes, cudaMemcpyDeviceToHost), "download f_py"); + CopyAndWait(px, d.f_px.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "download f_px"); + CopyAndWait(py, d.f_py.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "download f_py"); } void RotationScaleMergeGPU::GetFullsVariance(float *var_bkg, float *var_per_I) const { @@ -1431,8 +1448,8 @@ void RotationScaleMergeGPU::GetFullsVariance(float *var_bkg, float *var_per_I) c const auto &d = *impl_; if (d.n_fulls == 0) return; const size_t bytes = size_t(d.n_fulls) * sizeof(float); - CudaCheck(cudaMemcpy(var_bkg, d.f_var_bkg.get(), bytes, cudaMemcpyDeviceToHost), "download f_var_bkg"); - CudaCheck(cudaMemcpy(var_per_I, d.f_var_per_I.get(), bytes, cudaMemcpyDeviceToHost), + CopyAndWait(var_bkg, d.f_var_bkg.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "download f_var_bkg"); + CopyAndWait(var_per_I, d.f_var_per_I.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "download f_var_per_I"); } @@ -1440,6 +1457,6 @@ void RotationScaleMergeGPU::SetFullsCorr(const float *corr) { DeviceGuard guard(impl_->device, impl_->available); auto &d = *impl_; if (d.n_fulls == 0) return; - CudaCheck(cudaMemcpy(d.f_corr.get(), corr, size_t(d.n_fulls) * sizeof(float), - cudaMemcpyHostToDevice), "upload f_corr"); + CopyAndWait(d.f_corr.get(), corr, size_t(d.n_fulls) * sizeof(float), + cudaMemcpyHostToDevice, impl_->s(), "upload f_corr"); } diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index ca1dab038..3ef633636 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -6637,6 +6637,28 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const auto &rot_ss = experiment_.GetScalingSettings(); const bool is_rotation = experiment_.IsRotationIndexing(); // rotation indexing -> rotation scaling/merge std::optional rsm; + // How many times rsm has been ingested on this pass: a re-ingest follows a reindex or a cell + // change, after which a merge made from the first ingest is a merge of other indices. + int rsm_ingests = 0; + // Two P1 merges of this pass's integration made beside the rest of the tail, on an engine of + // their own: the all-observation arm of the space-group search (see there) and the P1 + // cross-check (see where it is written). Neither reads anything the search, the in-symmetry + // merge or the analyses decide, so neither has to wait for them. The engine is ingested together + // with rsm, before any merge writes per-frame values back onto the outcomes, so the two start + // from the same state, and it writes nothing back itself. Used only while rsm was ingested once + // (rsm_ingest); otherwise both merges are made on rsm, as before. + struct P1MergesAhead { + DiffractionExperiment x; + Logger log = Logger::Buffered(); // what the engine logs + Logger all_observations_log = Logger::Buffered(); // ...up to the all-observation merge + std::optional engine; + std::future ingested; + std::future all_observations; + std::future crosscheck; + bool crosscheck_made = false; + int rsm_ingest = 0; + }; + std::unique_ptr p1_ahead; std::optional prepass_postrefine_obs; // The rotation geometry post-refinement (see its call sites below for what it is for). const auto post_refine_geometry = [&] { @@ -6767,6 +6789,60 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Rotation scaling/merging (RotationScaleMerge) does not support " "wedge refinement"); + // The conditions of p1_crosscheck and of the all-observation arm below that are known here; + // the rest (a pass that turns out superseded, a search that finds no point group) only means + // a merge is made and not used. + const bool all_observations_ahead = !geometry_prepass && !experiment_.GetGemmiSpaceGroup().has_value() + && rot_ss.GetSearchMinZeta() > 0.0; + const bool crosscheck_ahead = !geometry_prepass && write_files && config_.write_merged + && config_.write_p1_crosscheck && result.consensus_cell + && (!experiment_.GetGemmiSpaceGroup().has_value() || indexer->GetPredictionCentring() == 'P'); + if ((all_observations_ahead || crosscheck_ahead) && config_.observation_dump_path.empty()) { + p1_ahead = std::make_unique(); + p1_ahead->x = experiment_; + p1_ahead->rsm_ingest = 1; + p1_ahead->crosscheck_made = crosscheck_ahead; + std::promise ingested; + std::promise all_observations; + p1_ahead->ingested = ingested.get_future(); + if (all_observations_ahead) + p1_ahead->all_observations = all_observations.get_future(); + p1_ahead->crosscheck = std::async(std::launch::async, + [&a = *p1_ahead, &outcomes = indexer->GetIntegrationOutcome(), + cell = result.consensus_cell, iter = static_cast(config_.scaling_iter), + nthreads = config_.nthreads, ingested = std::move(ingested), + all_observations = std::move(all_observations), all_observations_ahead, + crosscheck_ahead]() mutable -> RotationScaleMerge::Result { + try { + a.engine.emplace(a.x, outcomes, cell, iter, nthreads, a.log); + a.engine->SetWriteBackPerFrameScale(false); + a.engine->Ingest(); + } catch (...) { + ingested.set_exception(std::current_exception()); + throw; + } + ingested.set_value(); + // Ingested in the group rsm was, merged in P1 - as both merges on rsm are. + a.x.SpaceGroupNumber(1); + if (all_observations_ahead) { + try { + a.engine->SetSearchMinZeta(0.0); + auto merged = a.engine->Run(/*for_search=*/true, /*full_stats=*/true, + /*measure_cc_before_corrections=*/false); + a.all_observations_log = a.log; + a.log = Logger::Buffered(); + all_observations.set_value(std::move(merged)); + } catch (...) { + all_observations.set_exception(std::current_exception()); + throw; + } + } + if (!crosscheck_ahead) + return {}; + return a.engine->Run(/*for_search=*/false, /*full_stats=*/true, + /*measure_cc_before_corrections=*/false); + }); + } // A reference MTZ is allowed for rotation: it fixes the space group / cell (on the CLI) and // resolves the indexing ambiguity (below), but is NOT used to scale - the rotation merge stays // self-consistent. @@ -6774,6 +6850,9 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); + ++rsm_ingests; + if (p1_ahead) + p1_ahead->ingested.get(); } // The geometry pre-pass reads the per-image reflections exactly twice more: they were just @@ -7108,9 +7187,14 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // all-observation arm on both twin gates and promoted to R32 by the filtered one. So the // losing arm's refusal is logged and carried to the report below; the rule is unchanged. if (rsm && rsm->GetSearchMinZeta() > 0.0 && !sg_search.point_group_hm.empty()) { + std::optional ahead; + if (p1_ahead && p1_ahead->all_observations.valid() && p1_ahead->rsm_ingest == rsm_ingests) { + ahead = p1_ahead->all_observations.get(); + p1_ahead->all_observations_log.ReplayInto(logger); + } const double zeta = rsm->GetSearchMinZeta(); rsm->SetSearchMinZeta(0.0); - auto sm_all = scale_and_merge("P1, all observations", true); + auto sm_all = scale_and_merge("P1, all observations", true, false, std::move(ahead)); rsm->SetSearchMinZeta(zeta); merged_filtered_isa = sg_opts.merge_isa; // the filtered arm's, before it is replaced sg_opts.merge_isa = result.error_model_isa; // this arm's own error model @@ -7789,6 +7873,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); + ++rsm_ingests; } const auto &uc = *result.consensus_cell; logger.Info("{} names a cell of {}x the volume of its reference setting: the " @@ -7984,6 +8069,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b result.consensus_cell, static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); + ++rsm_ingests; } } } @@ -8011,6 +8097,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); + ++rsm_ingests; } } experiment_.SetSpaceGroup(sg); @@ -8344,6 +8431,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); + ++rsm_ingests; phase("Re-merging in the reference's frame"); sm = scale_and_merge(moved->short_name(), false); const auto &uc = *result.consensus_cell; @@ -8389,6 +8477,27 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const auto &twin_sg_opt = experiment_.GetGemmiSpaceGroup(); const gemmi::SpaceGroup *twin_sg = twin_sg_opt ? &*twin_sg_opt : nullptr; + + // Diffraction anisotropy (see where it is reported, below), made beside the analyses that come + // before it: it reads the merge, the integrated observations and the per-frame scales the merge + // wrote back, none of which they change, and most of it is gathering the observations. + std::future anisotropy; + if (!geometry_prepass && !superseded && result.consensus_cell) { + AnisotropyRunInfo aniso_run; + if (sm.statistics.sweep_quality.measured && sm.statistics.sweep_quality.sweep_deg > 0.0f) + aniso_run.observed_rotation_deg = sm.statistics.sweep_quality.sweep_deg; + aniso_run.dose_term_in_scale_model = experiment_.GetScalingSettings().GetCorrectionSurfaces(); + aniso_run.radiation_damage_relative_b = sm.statistics.radiation_damage_delta_b; + const float wedge_deg = experiment_.GetGoniometer() ? experiment_.GetGoniometer()->GetWedge_deg() : 0.0f; + anisotropy = std::async(std::launch::async, + [&, aniso_run, wedge_deg, cell = *result.consensus_cell, + rotation = experiment_.IsRotationIndexing()] { + return AnalyzeAnisotropy(sm.merged, + ScaledObservations(indexer->GetIntegrationOutcome(), rotation, twin_sg, + wedge_deg, 0.5, config_.nthreads), + cell, twin_sg, aniso_run); + }); + } // Not on the geometry pre-pass, nor on a superseded one: the analysis goes into that pass's // statistics text and its written reflections, and neither survives the run. The promotion flag // below is a different thing - it is what the SEARCH did, the second pass reads it, and it is @@ -8619,21 +8728,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // unmerged observations, because a merge has exact Laue symmetry by construction and the // tensor directions the symmetry forbids - the only place a dataset measures its own // systematic error - are identically zero in it. - if (result.consensus_cell) { - AnisotropyRunInfo aniso_run; - if (sm.statistics.sweep_quality.measured && sm.statistics.sweep_quality.sweep_deg > 0.0f) - aniso_run.observed_rotation_deg = sm.statistics.sweep_quality.sweep_deg; - aniso_run.dose_term_in_scale_model = - experiment_.GetScalingSettings().GetCorrectionSurfaces(); - aniso_run.radiation_damage_relative_b = sm.statistics.radiation_damage_delta_b; - sm.statistics.anisotropy = AnalyzeAnisotropy( - sm.merged, - ScaledObservations(indexer->GetIntegrationOutcome(), - experiment_.IsRotationIndexing(), twin_sg, - experiment_.GetGoniometer() - ? experiment_.GetGoniometer()->GetWedge_deg() : 0.0f, - 0.5, config_.nthreads), - *result.consensus_cell, twin_sg, aniso_run); + if (anisotropy.valid()) { + sm.statistics.anisotropy = anisotropy.get(); stats_text << AnisotropyToText(sm.statistics.anisotropy) << "\n"; } } @@ -8831,6 +8927,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b && !geometry_prepass && !superseded && config_.write_p1_crosscheck && p1_integration_complete && is_rotation; std::optional p1_merged_early; + const bool p1_ahead_usable = p1_crosscheck && p1_ahead && p1_ahead->crosscheck_made + && p1_ahead->rsm_ingest == rsm_ingests; // Model validation runs BEFORE the reflection files are written, because it is what settles the // frame they are written in: the enantiomorph, which merged intensities cannot choose, and - @@ -8865,12 +8963,14 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // P1 merge afterwards, through merge_to_written, as it did when it was made below. The // validation's log lines are held and printed as one block once it is done. ModelValidationResult validation; - if (p1_crosscheck && rsm) { + if (p1_crosscheck && rsm && !p1_ahead_usable) { Logger held = Logger::Buffered(); auto pending = std::async(std::launch::async, validate, std::ref(held)); experiment_.SpaceGroupNumber(1); + rsm->SetWriteBackPerFrameScale(false); p1_merged_early = rsm->Run(/*for_search=*/false, /*full_stats=*/true, /*measure_cc_before_corrections=*/false); + rsm->SetWriteBackPerFrameScale(true); experiment_.SetSpaceGroup(data_sg); validation = pending.get(); held.ReplayInto(logger); @@ -9046,6 +9146,9 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const double em_a = result.error_model_a; const double em_b = result.error_model_b; const auto res_fit = result.resolution_fit_A; + const int iter_partials = result.scaling_iterations_partials; + const int iter_fulls = result.scaling_iterations_fulls; + const bool converged = result.scaling_converged; // The unmerged MTZ below does not depend on this merge, so it is built meanwhile, from // the experiment as it stands in the determined group. The merge rewrites each image's // mosaicity, which the file's batch headers carry, so that is filled in only after it. @@ -9062,7 +9165,14 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // Both the merge and the MTZ read the group from the experiment, so it is set for // the whole of it and restored after. experiment_.SpaceGroupNumber(1); + if (p1_ahead_usable) { + p1_merged_early = p1_ahead->crosscheck.get(); + p1_ahead->log.ReplayInto(logger); + } + if (!p1_merged_early) + rsm->SetWriteBackPerFrameScale(false); auto p1 = scale_and_merge("P1 cross-check", false, false, std::move(p1_merged_early)); + rsm->SetWriteBackPerFrameScale(true); // The scaler still holds the observations in the indexing they were merged in, so every // relabelling since (the written setting, the model's indexing) is applied to this merge // too - or it would describe the dataset on other axes than the merged output beside it, @@ -9135,6 +9245,9 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b result.error_model_a = em_a; result.error_model_b = em_b; result.resolution_fit_A = res_fit; + result.scaling_iterations_partials = iter_partials; + result.scaling_iterations_fulls = iter_fulls; + result.scaling_converged = converged; if (determined != nullptr && determined->number > 1) logger.Info("P1 cross-check dataset written to {} ({} unique reflections): the " "same observations merged in P1 instead of {}, so a wrong space group " -- 2.54.0 From 24e36ae740eb7900a9d920350ec7ad0f9984458a Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 10:46:49 +0200 Subject: [PATCH 08/97] Pre-scan: parallel beam-stop mask, leaner background beam-centre fit Exact: p.mtz and the pre-scan products (shadow mask, mean projection, defective-pixel mask, ring and capture centres, compared as hashes and hex floats) are bit-identical to rc174 on three in-house rotation sets, GPU and CPU builds. - ShadowFinder::GetMask: the serial parts run in parallel - connected components by row band joined with union-find (both the shadow and the transmitting-arm searches, and the hole fill), ring binning and the harmonic sector gather by blocks, gap bridging by line; ring pixel counts read off the ring offsets. Mean projection filled in parallel. - ShadowFinder host accumulation: one band-locked projection instead of a 20 B/px shard per pre-scan worker (2.7 GB zeroed and folded on a 16M detector); SetShardCount and the shard argument are gone. - FindBeamCenterFromBackground: the usable-pixel test is made once, the in-band pixels are kept in pixel order so the clipping rounds no longer sweep the whole detector, the 67 MB cell map is gone and the per-iteration block fold runs in parallel - same sums, same order. - HotPixelFinder::GetMask: the chance-rate counts in parallel (integers). Measured on a loaded box (load ~25 from other jobs), pre-scan window: GPU 5.9-6.5 s -> 3.2-3.4 s, CPU 8.4-9.0 s -> 6.1-7.4 s. The GPU-build pre-scan now ends with its background spot measurement (CPU spot finder on ~120 frames, ~13 core-s on 8 workers). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- image_analysis/beam_stop/ShadowFinder.cpp | 533 ++++++++++-------- image_analysis/beam_stop/ShadowFinder.h | 59 +- .../BeamCenterFromBackground.cpp | 82 ++- rugnux/HotPixels.cpp | 22 +- rugnux/Rugnux.cpp | 19 +- tests/ShadowFinderTest.cpp | 38 +- 6 files changed, 427 insertions(+), 326 deletions(-) diff --git a/image_analysis/beam_stop/ShadowFinder.cpp b/image_analysis/beam_stop/ShadowFinder.cpp index 715ad1198..95846c533 100644 --- a/image_analysis/beam_stop/ShadowFinder.cpp +++ b/image_analysis/beam_stop/ShadowFinder.cpp @@ -10,7 +10,6 @@ #include #include #include -#include #include #include @@ -79,7 +78,7 @@ constexpr int MIN_RING_PIXELS = 32; constexpr float BLOCKED_RING_RATIO = 0.35f; // Binary-image helpers on a width*height frame stored row-major as char (0/1). All run once, -// at GetMask() time; the BFS forms keep them O(pixels) rather than O(pixels * radius). +// at GetMask() time, and all are O(pixels) rather than O(pixels * radius). namespace { // A per-pixel array of GetMask(). A std::vector zeroes what it allocates on the thread that makes it, @@ -186,10 +185,12 @@ Plane erode(const Plane &in, int W, int H, int r, size_t nthreads) { // continues on both sides of it is one shadow - but a gap can be wider than BRIDGE_PX reaches (17 px // between the rows of PILATUS modules), and an arm crossing one fell apart into pieces each too small // to be believed. -Plane bridge_gaps(const Plane ®ion, const Plane &valid, int W, int H) { +Plane bridge_gaps(const Plane ®ion, const Plane &valid, int W, int H, size_t nthreads) { Plane out = region; + // The lines of one direction are independent: each reads `region` and only ever sets its own pixels. auto walk = [&](int n_lines, int len, auto index) { - for (int line = 0; line < n_lines; line++) { + ParallelChunks(n_lines, nthreads, [&](int lo, int hi) { + for (int line = lo; line < hi; line++) { int k = 0; while (k < len) { if (valid[index(line, k)]) { k++; continue; } @@ -199,12 +200,99 @@ Plane bridge_gaps(const Plane ®ion, const Plane &valid, int for (int j = start; j < k; j++) out[index(line, j)] = 1; } } + }); }; walk(H, W, [W](int y, int x) { return static_cast(y) * W + x; }); walk(W, H, [W](int x, int y) { return static_cast(y) * W + x; }); return out; } +// The 8-connected components of `member`: each member pixel gets the index of its component, dense +// from 0 and in no particular order, and every other pixel -1. +// +// Labelled in parallel. Each band of rows is flooded on its own, then the pieces that touch across a +// band boundary are joined. Which pixels share a component is all a caller reads, and that does not +// depend on how the rows were split. +struct Components { + Plane id; + int count = 0; +}; + +Components label_components(const Plane &member, int W, int H, size_t nthreads) { + const int bands = std::min(64, H); // never more bands than rows, so none is empty + std::vector band_row(bands + 1); + for (int b = 0; b <= bands; b++) + band_row[b] = static_cast(static_cast(b) * H / bands); + + Components out; + out.id = Plane(member.size()); + std::vector band_pieces(bands, 0); + ParallelFor(bands, nthreads, [&](int b) { + const size_t lo = static_cast(band_row[b]) * W, hi = static_cast(band_row[b + 1]) * W; + std::fill(out.id.begin() + lo, out.id.begin() + hi, -1); + std::vector stack; + int pieces = 0; + for (size_t start = lo; start < hi; start++) { + if (!member[start] || out.id[start] >= 0) + continue; + out.id[start] = pieces; + stack.push_back(start); + while (!stack.empty()) { + const size_t i = stack.back(); stack.pop_back(); + const int y = static_cast(i / W), x = static_cast(i % W); + for (int dy = -1; dy <= 1; dy++) + for (int dx = -1; dx <= 1; dx++) { + const int yy = y + dy, xx = x + dx; + if (yy < band_row[b] || yy >= band_row[b + 1] || xx < 0 || xx >= W) + continue; + const size_t j = static_cast(yy) * W + xx; + if (member[j] && out.id[j] < 0) { out.id[j] = pieces; stack.push_back(j); } + } + } + pieces++; + } + band_pieces[b] = pieces; + }); + + // A piece is named by its band's first index plus its number in the band, and the pieces are + // joined across each boundary row by union-find. + std::vector first(bands + 1, 0); + for (int b = 0; b < bands; b++) + first[b + 1] = first[b] + band_pieces[b]; + std::vector parent(first[bands]); + for (size_t k = 0; k < parent.size(); k++) + parent[k] = static_cast(k); + const auto find = [&](int k) { + while (parent[k] != k) { parent[k] = parent[parent[k]]; k = parent[k]; } + return k; + }; + for (int b = 0; b + 1 < bands; b++) { + const size_t above = static_cast(band_row[b + 1] - 1) * W, below = above + W; + for (int x = 0; x < W; x++) { + if (!member[above + x]) + continue; + for (int xx = std::max(0, x - 1); xx <= std::min(W - 1, x + 1); xx++) + if (member[below + xx]) { + const int ra = find(first[b] + out.id[above + x]); + const int rb = find(first[b + 1] + out.id[below + xx]); + if (ra != rb) parent[std::max(ra, rb)] = std::min(ra, rb); + } + } + } + std::vector dense(parent.size(), -1), component(parent.size()); + for (size_t k = 0; k < parent.size(); k++) { + const int root = find(static_cast(k)); + if (dense[root] < 0) dense[root] = out.count++; + component[k] = dense[root]; + } + ParallelFor(bands, nthreads, [&](int b) { + const size_t lo = static_cast(band_row[b]) * W, hi = static_cast(band_row[b + 1]) * W; + for (size_t i = lo; i < hi; i++) + if (out.id[i] >= 0) out.id[i] = component[first[b] + out.id[i]]; + }); + return out; +} + // Fill holes: background not reachable from the image border becomes region. // // The flood is run over the bounding box of `region` grown by one, not the whole detector. Outside @@ -212,52 +300,53 @@ Plane bridge_gaps(const Plane ®ion, const Plane &valid, int // outside is one border-connected component: a background pixel inside the box is border-connected // exactly when it reaches the ring. The beam stop occupies a small part of a detector, so this is // the same answer over a fraction of the pixels. -Plane fill_holes(const Plane ®ion, int W, int H) { +Plane fill_holes(const Plane ®ion, int W, int H, size_t nthreads) { + std::vector row_x0(H, W), row_x1(H, -1); + ParallelChunks(H, nthreads, [&](int ylo, int yhi) { + for (int y = ylo; y < yhi; y++) + for (int x = 0; x < W; x++) + if (region[static_cast(y) * W + x]) { + row_x0[y] = std::min(row_x0[y], x); + row_x1[y] = x; + } + }); int x0 = W, x1 = -1, y0 = H, y1 = -1; for (int y = 0; y < H; y++) - for (int x = 0; x < W; x++) - if (region[static_cast(y) * W + x]) { - x0 = std::min(x0, x); x1 = std::max(x1, x); - y0 = std::min(y0, y); y1 = std::max(y1, y); - } + if (row_x1[y] >= 0) { + x0 = std::min(x0, row_x0[y]); x1 = std::max(x1, row_x1[y]); + y0 = std::min(y0, y); y1 = y; + } if (x1 < 0) return region; // nothing to enclose x0 = std::max(0, x0 - 1); x1 = std::min(W - 1, x1 + 1); y0 = std::max(0, y0 - 1); y1 = std::min(H - 1, y1 + 1); + // The background of the box, in components; one that reaches the box's edge is outside. const int BW = x1 - x0 + 1, BH = y1 - y0 + 1; - std::vector bg_visited(static_cast(BW) * BH, 0); - std::queue q; // indices into the box - auto push = [&](int bx, int by) { - const int j = by * BW + bx; - if (!region[static_cast(by + y0) * W + bx + x0] && !bg_visited[j]) { - bg_visited[j] = 1; q.push(j); - } + Plane background(static_cast(BW) * BH); + ParallelChunks(BH, nthreads, [&](int lo, int hi) { + for (int by = lo; by < hi; by++) + for (int bx = 0; bx < BW; bx++) + background[static_cast(by) * BW + bx] = !region[static_cast(by + y0) * W + bx + x0]; + }); + const auto pieces = label_components(background, BW, BH, nthreads); + std::vector outside(pieces.count, 0); + const auto edge = [&](int bx, int by) { + const int c = pieces.id[static_cast(by) * BW + bx]; + if (c >= 0) outside[c] = 1; }; - for (int bx = 0; bx < BW; bx++) { push(bx, 0); push(bx, BH - 1); } - for (int by = 0; by < BH; by++) { push(0, by); push(BW - 1, by); } - while (!q.empty()) { - const int i = q.front(); q.pop(); - const int by = i / BW, bx = i % BW; - for (int dy = -1; dy <= 1; dy++) - for (int dx = -1; dx <= 1; dx++) { - const int yy = by + dy, xx = bx + dx; - if (yy < 0 || yy >= BH || xx < 0 || xx >= BW) - continue; - const int j = yy * BW + xx; - if (!region[static_cast(yy + y0) * W + xx + x0] && !bg_visited[j]) { - bg_visited[j] = 1; q.push(j); - } - } - } + for (int bx = 0; bx < BW; bx++) { edge(bx, 0); edge(bx, BH - 1); } + for (int by = 0; by < BH; by++) { edge(0, by); edge(BW - 1, by); } Plane out = region; - for (int by = 0; by < BH; by++) - for (int bx = 0; bx < BW; bx++) { - const size_t i = static_cast(by + y0) * W + bx + x0; - if (!region[i] && !bg_visited[by * BW + bx]) - out[i] = 1; - } + ParallelChunks(BH, nthreads, [&](int lo, int hi) { + for (int by = lo; by < hi; by++) + for (int bx = 0; bx < BW; bx++) { + const int c = pieces.id[static_cast(by) * BW + bx]; + if (c >= 0 && !outside[c]) + out[static_cast(by + y0) * W + bx + x0] = 1; + } + }); return out; } @@ -321,19 +410,40 @@ struct RingValues { RingValues bin_by_ring(const Plane &values, const Plane &valid, const Plane &radius, int max_radius, size_t nthreads) { + // Counted and scattered by blocks of pixels in parallel: each block writes its values of a ring + // after those of the blocks before it, so every ring holds its values in pixel order, as a single + // pass would leave them - and they are sorted below in any case. + constexpr int BLOCKS = 64; + const size_t n = values.size(); + const auto block_begin = [n](int b) { return n * b / BLOCKS; }; + const size_t rings = static_cast(max_radius) + 1; + std::vector cursor(BLOCKS * rings, 0); + ParallelFor(BLOCKS, nthreads, [&](int b) { + int *count = cursor.data() + b * rings; + for (size_t i = block_begin(b); i < block_begin(b + 1); i++) + if (valid[i]) + count[radius[i]]++; + }); + RingValues rv; rv.offset.assign(max_radius + 2, 0); - for (size_t i = 0; i < values.size(); i++) - if (valid[i]) - rv.offset[radius[i] + 1]++; - for (int r = 0; r <= max_radius; r++) - rv.offset[r + 1] += rv.offset[r]; + for (size_t r = 0; r < rings; r++) { + int at = rv.offset[r]; + for (int b = 0; b < BLOCKS; b++) { + const int count = cursor[b * rings + r]; + cursor[b * rings + r] = at; + at += count; + } + rv.offset[r + 1] = at; + } rv.values.resize(rv.offset[max_radius + 1]); - std::vector cursor(rv.offset.begin(), rv.offset.end() - 1); - for (size_t i = 0; i < values.size(); i++) - if (valid[i]) - rv.values[cursor[radius[i]]++] = values[i]; + ParallelFor(BLOCKS, nthreads, [&](int b) { + int *next = cursor.data() + b * rings; + for (size_t i = block_begin(b); i < block_begin(b + 1); i++) + if (valid[i]) + rv.values[next[radius[i]]++] = values[i]; + }); // Sorted once; the three iterations then only pick a rank and count a prefix. ParallelFor(max_radius + 1, nthreads, [&](int r) { @@ -355,7 +465,6 @@ ShadowFinder::ShadowFinder(const DiffractionExperiment &experiment, const PixelM if (pixel_mask.size() != static_cast(width) * height) throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "ShadowFinder: pixel mask does not match the detector"); - SetShardCount(1); #ifdef JFJOCH_USE_CUDA if (get_gpu_count() > 0) { const size_t npixels = static_cast(width) * height; @@ -372,98 +481,37 @@ void ShadowFinder::BeamCenter(float x, float y) { beam_y = y; } -// A shard's accumulators are allocated when a frame is first added to it, not here: with a GPU they -// are never used at all, and on a 16 Mpx detector eight of them are 2.9 GB to allocate and clear - -// which measured 0.8 s of the pre-scan, all of it wasted. -void ShadowFinder::SetShardCount(size_t n) { - shards.clear(); - shards.resize(std::max(1, n)); -} - +#ifdef JFJOCH_USE_CUDA ShadowFinder::Projection ShadowFinder::Reduce() const { -#ifdef JFJOCH_USE_CUDA - // The device holds its own projection. Bring it back and let it take part in the fold below as - // one more shard; when every frame went to the GPU it is the whole answer. - Projection device; - if (Gpu() && gpu->GetFrameCount() > 0) { - gpu->Download(device.max_value, device.sum_value, device.valid_count); - device.frames = gpu->GetFrameCount(); - bool host_empty = true; - for (const auto &p : shards) - host_empty = host_empty && (p.frames == 0); - if (host_empty) - return device; - } -#endif - - // Only when that shard actually holds something: its accumulators are allocated on first use, so - // an unused shard is empty rather than zeroed, and returning it would hand the callers below a - // projection they index by pixel. - if (shards.size() == 1 && shards[0].frames > 0 -#ifdef JFJOCH_USE_CUDA - && !(gpu && gpu->GetFrameCount() > 0) -#endif - ) - return shards[0]; - + // The device holds its own projection. Bring it back; when every frame went to the GPU it is the + // whole answer. Projection out; - const size_t npixels = static_cast(width) * height; - out.max_value.assign(npixels, 0); - out.sum_value.assign(npixels, 0); - out.valid_count.assign(npixels, 0); - for (const auto &p : shards) - out.frames += p.frames; -#ifdef JFJOCH_USE_CUDA - out.frames += device.frames; -#endif - - // Each worker owns a slice of the pixels and folds every shard into it. The sums and counts are - // integers and a pixel is touched by one worker only, so the result is the same as folding them - // one shard at a time on one thread - this is several hundred megabytes per shard and is limited - // by memory rather than by arithmetic. - const size_t nthreads = std::max(1, std::min(std::thread::hardware_concurrency(), - shards.size() * 2)); - const size_t chunk = (npixels + nthreads - 1) / nthreads; - std::vector> futures; - futures.reserve(nthreads); - for (size_t t = 0; t < nthreads; t++) { - const size_t lo = t * chunk, hi = std::min(npixels, lo + chunk); - if (lo >= hi) break; - futures.emplace_back(std::async(std::launch::async, [&, lo, hi] { -#ifdef JFJOCH_USE_CUDA - const Projection *extra[1] = {&device}; - for (const auto *pp : extra) { - const auto &p = *pp; - if (p.frames == 0) continue; - for (size_t i = lo; i < hi; i++) { - if (p.valid_count[i] == 0) - continue; - if (out.valid_count[i] == 0 || p.max_value[i] > out.max_value[i]) - out.max_value[i] = p.max_value[i]; - out.sum_value[i] += p.sum_value[i]; - out.valid_count[i] += p.valid_count[i]; - } - } -#endif - for (const auto &p : shards) { - if (p.frames == 0) continue; // never used, and its accumulators were never allocated - for (size_t i = lo; i < hi; i++) { - if (p.valid_count[i] == 0) - continue; - if (out.valid_count[i] == 0 || p.max_value[i] > out.max_value[i]) - out.max_value[i] = p.max_value[i]; - out.sum_value[i] += p.sum_value[i]; - out.valid_count[i] += p.valid_count[i]; - } - } - })); + if (Gpu() && gpu->GetFrameCount() > 0) { + gpu->Download(out.max_value, out.sum_value, out.valid_count); + out.frames = gpu->GetFrameCount(); } - for (auto &f : futures) f.get(); + if (host.frames == 0) + return out; + + // A pixel is touched by one worker only and the sums and counts are integers, so the result is + // the same as folding on one thread. + out.frames += host.frames; + ParallelChunks(static_cast(out.max_value.size()), std::thread::hardware_concurrency(), [&](int lo, int hi) { + for (int i = lo; i < hi; i++) { + if (host.valid_count[i] == 0) + continue; + if (out.valid_count[i] == 0 || host.max_value[i] > out.max_value[i]) + out.max_value[i] = host.max_value[i]; + out.sum_value[i] += host.sum_value[i]; + out.valid_count[i] += host.valid_count[i]; + } + }); return out; } +#endif template -void ShadowFinder::Add(const T *ptr, Projection &p) { +void ShadowFinder::Add(const T *ptr, size_t begin, size_t end) { // The pixel type's sentinel extreme marks "no data" (module gap / masked): the // preprocessor/writer stores INT*_MIN for signed and UINT*_MAX for unsigned. For signed // types the opposite extreme is a genuine saturated value and is kept, so a saturated @@ -474,27 +522,23 @@ void ShadowFinder::Add(const T *ptr, Projection &p) { else masked = std::numeric_limits::max(); - for (size_t i = 0; i < p.max_value.size(); i++) { + for (size_t i = begin; i < end; i++) { const T v = ptr[i]; if (v == masked) continue; const int64_t vi = static_cast(v); - if (p.valid_count[i] == 0 || vi > p.max_value[i]) - p.max_value[i] = vi; - p.sum_value[i] += vi; - p.valid_count[i]++; + if (host.valid_count[i] == 0 || vi > host.max_value[i]) + host.max_value[i] = vi; + host.sum_value[i] += vi; + host.valid_count[i]++; } - p.frames++; } -void ShadowFinder::AddImage(const DataMessage &data, std::vector &buffer, size_t shard) { +void ShadowFinder::AddImage(const DataMessage &data, std::vector &buffer) { if (static_cast(data.image.GetWidth()) * data.image.GetHeight() != static_cast(width) * height) throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "ShadowFinder: image size does not match the detector"); - if (shard >= shards.size()) - throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, - "ShadowFinder: shard out of range"); #ifdef JFJOCH_USE_CUDA // One device, so the frames queue here - but each is only a chunk upload plus two kernels, and @@ -511,25 +555,37 @@ void ShadowFinder::AddImage(const DataMessage &data, std::vector &buffe } #endif - Projection &p = shards[shard]; - if (p.max_value.empty()) { - const size_t npixels = static_cast(width) * height; - p.max_value.assign(npixels, 0); - p.sum_value.assign(npixels, 0); - p.valid_count.assign(npixels, 0); + const size_t npixels = static_cast(width) * height; + { + std::unique_lock ul(host_mutex); + if (host.max_value.empty()) { + host.max_value.resize(npixels); + host.sum_value.resize(npixels); + host.valid_count.resize(npixels); + } } const auto ptr = data.image.GetUncompressedPtr(buffer); - switch (data.image.GetMode()) { - case CompressedImageMode::Int8: Add(reinterpret_cast(ptr), p); break; - case CompressedImageMode::Uint8: Add(reinterpret_cast(ptr), p); break; - case CompressedImageMode::Int16: Add(reinterpret_cast(ptr), p); break; - case CompressedImageMode::Uint16: Add(reinterpret_cast(ptr), p); break; - case CompressedImageMode::Int32: Add(reinterpret_cast(ptr), p); break; - case CompressedImageMode::Uint32: Add(reinterpret_cast(ptr), p); break; - default: - throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, - "ShadowFinder: unsupported image mode"); + const size_t rows_per_band = (static_cast(height) + BANDS - 1) / BANDS; + const size_t first = next_band.fetch_add(1); + for (size_t b = 0; b < BANDS; b++) { + const size_t band = (first + b) % BANDS; + const size_t begin = std::min(npixels, band * rows_per_band * width); + const size_t end = std::min(npixels, (band + 1) * rows_per_band * width); + std::lock_guard lock(band_mutex[band]); + switch (data.image.GetMode()) { + case CompressedImageMode::Int8: Add(reinterpret_cast(ptr), begin, end); break; + case CompressedImageMode::Uint8: Add(reinterpret_cast(ptr), begin, end); break; + case CompressedImageMode::Int16: Add(reinterpret_cast(ptr), begin, end); break; + case CompressedImageMode::Uint16: Add(reinterpret_cast(ptr), begin, end); break; + case CompressedImageMode::Int32: Add(reinterpret_cast(ptr), begin, end); break; + case CompressedImageMode::Uint32: Add(reinterpret_cast(ptr), begin, end); break; + default: + throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, + "ShadowFinder: unsupported image mode"); + } } + std::unique_lock ul(host_mutex); + host.frames++; } #ifdef JFJOCH_USE_CUDA @@ -552,8 +608,7 @@ ShadowAccumulatorGPU *ShadowFinder::Gpu() const { uint32_t ShadowFinder::GetFrameCount() const { std::unique_lock ul(m); - uint32_t frames = 0; - for (const auto &p : shards) frames += p.frames; + uint32_t frames = host.frames; #ifdef JFJOCH_USE_CUDA if (gpu) frames += gpu->GetFrameCount(); #endif @@ -561,14 +616,21 @@ uint32_t ShadowFinder::GetFrameCount() const { } const ShadowFinder::Projection &ShadowFinder::Reduced() const { - if (!reduced) - reduced = Reduce(); - return *reduced; +#ifdef JFJOCH_USE_CUDA + if (gpu && gpu->GetFrameCount() > 0) { + if (!reduced) + reduced = Reduce(); + return *reduced; + } +#endif + return host; } void ShadowFinder::ReleaseProjection() { +#ifdef JFJOCH_USE_CUDA std::unique_lock ul(m); reduced.reset(); +#endif } std::vector ShadowFinder::GetMeanProjection() const { @@ -577,10 +639,12 @@ std::vector ShadowFinder::GetMeanProjection() const { const auto &sum_value = p.sum_value; const auto &valid_count = p.valid_count; - std::vector mean(static_cast(width) * height, NAN); - for (size_t i = 0; i < mean.size(); i++) - if (valid_count[i] > 0 && pixel_mask[i] == 0) - mean[i] = static_cast(static_cast(sum_value[i]) / valid_count[i]); + std::vector mean(static_cast(width) * height); + ParallelChunks(static_cast(mean.size()), std::thread::hardware_concurrency(), [&](int lo, int hi) { + for (int i = lo; i < hi; i++) + mean[i] = valid_count[i] > 0 && pixel_mask[i] == 0 + ? static_cast(static_cast(sum_value[i]) / valid_count[i]) : NAN; + }); return mean; } @@ -727,9 +791,8 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { // Innermost rings hold only a handful of pixels, too few to judge, so they are stepped over // rather than allowed to end the walk. std::vector ring_pixels(max_radius + 1, 0); - for (int i = 0; i < n_pixels; i++) - if (valid[i]) - ring_pixels[radius[i]]++; + for (int rad = 0; rad <= max_radius; rad++) + ring_pixels[rad] = rings.offset[rad + 1] - rings.offset[rad]; // A ring lies inside the stop when its background is a fraction of what this detector's // background typically is. Counting statistics cannot decide this: on a bright dataset the @@ -789,36 +852,20 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { // and the stop. What keeps the test specific instead is size, since the background wanders by a // pixel or two at a time and hardware does not. const Plane bridged = dilate(low, W, H, BRIDGE_PX, nthreads); - Plane region = filled_plane(n_pixels, 0, nthreads); + Plane region(n_pixels); { - Plane seen = filled_plane(n_pixels, 0, nthreads); - std::vector component; - std::queue q; - for (int start = 0; start < n_pixels; start++) { - if (!bridged[start] || seen[start]) - continue; - component.clear(); - int n_low = 0; - seen[start] = 1; - q.push(start); - while (!q.empty()) { - const int i = q.front(); q.pop(); - component.push_back(i); - n_low += low[i]; - const int y = i / W, x = i % W; - for (int dy = -1; dy <= 1; dy++) - for (int dx = -1; dx <= 1; dx++) { - const int yy = y + dy, xx = x + dx; - if (yy < 0 || yy >= H || xx < 0 || xx >= W) - continue; - const int j = yy * W + xx; - if (bridged[j] && !seen[j]) { seen[j] = 1; q.push(j); } - } - } - if (n_low >= MIN_SHADOW_PIXELS) - for (const int i : component) - region[i] = low[i]; - } + const auto pieces = label_components(bridged, W, H, nthreads); + std::vector> n_low(pieces.count); + ParallelChunks(n_pixels, nthreads, [&](int lo, int hi) { + for (int i = lo; i < hi; i++) + if (pieces.id[i] >= 0 && low[i]) + n_low[pieces.id[i]].fetch_add(1, std::memory_order_relaxed); + }); + ParallelChunks(n_pixels, nthreads, [&](int lo, int hi) { + for (int i = lo; i < hi; i++) + region[i] = pieces.id[i] >= 0 && n_low[pieces.id[i]].load(std::memory_order_relaxed) >= MIN_SHADOW_PIXELS + ? low[i] : 0; + }); } // The rings that lie wholly inside the stop are decided by the ring walk above rather than by @@ -866,7 +913,7 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { region = erode(dilate(region, W, H, 2, nthreads), W, H, 2, nthreads); - region = fill_holes(region, W, H); + region = fill_holes(region, W, H, nthreads); // Expose recorded reflections - done last, with no fill afterwards, so a spot the shadow @@ -903,15 +950,31 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { // allowed to explain a dim sector away - where it would ask for more than the median, the median // stands - so the step can only drop what it found before, never find something new. const int n_bands = max_radius / HARMONIC_BAND_PX + 1; - std::vector> sector_values(static_cast(n_bands) * HARMONIC_SECTORS); - for (int i = 0; i < n_pixels; i++) { - if (!valid[i] || region[i]) - continue; - const float dx = static_cast(i % W) - beam_x, dy = static_cast(i / W) - beam_y; - const double phi = std::atan2(dy, dx) + std::numbers::pi; - const int sector = std::min(HARMONIC_SECTORS - 1, static_cast(phi / (2.0 * std::numbers::pi) * HARMONIC_SECTORS)); - sector_values[static_cast(radius[i] / HARMONIC_BAND_PX) * HARMONIC_SECTORS + sector].push_back(ratio[i]); - } + const size_t n_sectors = static_cast(n_bands) * HARMONIC_SECTORS; + // Gathered by blocks of rows in parallel and joined in block order. Only the median of each + // sector is read, and that is the same whatever order its values were gathered in. + constexpr int SECTOR_BLOCKS = 64; + std::vector>> block_values(SECTOR_BLOCKS); + ParallelFor(SECTOR_BLOCKS, nthreads, [&](int b) { + auto &values = block_values[b]; + values.resize(n_sectors); + const int lo = static_cast(static_cast(n_pixels) * b / SECTOR_BLOCKS); + const int hi = static_cast(static_cast(n_pixels) * (b + 1) / SECTOR_BLOCKS); + for (int i = lo; i < hi; i++) { + if (!valid[i] || region[i]) + continue; + const float dx = static_cast(i % W) - beam_x, dy = static_cast(i / W) - beam_y; + const double phi = std::atan2(dy, dx) + std::numbers::pi; + const int sector = std::min(HARMONIC_SECTORS - 1, static_cast(phi / (2.0 * std::numbers::pi) * HARMONIC_SECTORS)); + values[static_cast(radius[i] / HARMONIC_BAND_PX) * HARMONIC_SECTORS + sector].push_back(ratio[i]); + } + }); + std::vector> sector_values(n_sectors); + ParallelFor(static_cast(n_sectors), nthreads, [&](int k) { + for (const auto &values : block_values) + sector_values[k].insert(sector_values[k].end(), values[k].begin(), values[k].end()); + }); + block_values.clear(); std::vector harm_c(n_bands, 0.0f), harm_s(n_bands, 0.0f); ParallelFor(n_bands, nthreads, [&](int band) { std::vector med(HARMONIC_SECTORS, -1.0), c(HARMONIC_SECTORS), s(HARMONIC_SECTORS); @@ -978,36 +1041,24 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { && poisson_deficit_sigma(pooled[i] * counted, baseline[radius[i]] * model * counted) > MIN_DEFICIT_SIGMA; } }); - const Plane joined = bridge_gaps(dilate(dim, W, H, BRIDGE_PX, nthreads), valid, W, H); - Plane seen = filled_plane(n_pixels, 0, nthreads); - std::vector component; - std::queue q; - for (int start = 0; start < n_pixels; start++) { - if (!joined[start] || seen[start]) - continue; - component.clear(); - int n_dim = 0, n_low = 0; - seen[start] = 1; - q.push(start); - while (!q.empty()) { - const int i = q.front(); q.pop(); - component.push_back(i); - n_dim += dim[i]; - n_low += dim[i] && low[i]; - const int y = i / W, x = i % W; - for (int dy = -1; dy <= 1; dy++) - for (int dx = -1; dx <= 1; dx++) { - const int yy = y + dy, xx = x + dx; - if (yy < 0 || yy >= H || xx < 0 || xx >= W) - continue; - const int j = yy * W + xx; - if (joined[j] && !seen[j]) { seen[j] = 1; q.push(j); } - } + const Plane joined = bridge_gaps(dilate(dim, W, H, BRIDGE_PX, nthreads), valid, W, H, nthreads); + const auto pieces = label_components(joined, W, H, nthreads); + std::vector> n_dim(pieces.count), n_low(pieces.count); + ParallelChunks(n_pixels, nthreads, [&](int lo, int hi) { + for (int i = lo; i < hi; i++) + if (pieces.id[i] >= 0 && dim[i]) { + n_dim[pieces.id[i]].fetch_add(1, std::memory_order_relaxed); + if (low[i]) + n_low[pieces.id[i]].fetch_add(1, std::memory_order_relaxed); + } + }); + ParallelChunks(n_pixels, nthreads, [&](int lo, int hi) { + for (int i = lo; i < hi; i++) { + const int c = pieces.id[i]; + if (c >= 0 && dim[i] && n_dim[c].load(std::memory_order_relaxed) >= MIN_SHADOW_PIXELS + && n_low[c].load(std::memory_order_relaxed) >= MIN_CORE_PIXELS) + mask[i] = TRANSMITTING; } - if (n_dim >= MIN_SHADOW_PIXELS && n_low >= MIN_CORE_PIXELS) - for (const int i : component) - if (dim[i]) - mask[i] = TRANSMITTING; - } + }); return mask; } diff --git a/image_analysis/beam_stop/ShadowFinder.h b/image_analysis/beam_stop/ShadowFinder.h index 1b7e66ff2..866e3df4d 100644 --- a/image_analysis/beam_stop/ShadowFinder.h +++ b/image_analysis/beam_stop/ShadowFinder.h @@ -4,6 +4,7 @@ #pragma once #include +#include #include #include #include @@ -36,9 +37,9 @@ // // Frames are chosen by the caller; the detection needs enough of them that the background // is counted rather than guessed (see MIN_EXPECTED_COUNTS in the .cpp). -// Thread-safe: workers call AddImage concurrently, each naming a shard of its own (see -// SetShardCount) - so no two threads touch the same accumulator and nothing is locked while -// an image is added. The shards are summed when the projection is read. +// Thread-safe: workers call AddImage concurrently. The projection is split into bands of rows, each +// with a lock of its own, and a worker adding a frame starts at a different band from the one before +// it, so workers meet only when they reach the same band. class ShadowFinder { mutable std::mutex m; @@ -62,22 +63,27 @@ class ShadowFinder { std::vector pixel_mask; // pixels already masked carry no background to test - // Per-pixel projection over the frames added so far (converted geometry). One set per shard: - // the sums and counts are integers, so summing the shards is exact and the result does not - // depend on how the frames were spread over them. + // Per-pixel projection over the frames added so far (converted geometry). The sums and counts are + // integers and the maximum is a maximum, so the result does not depend on the order the frames + // arrive in. struct Projection { std::vector max_value; std::vector sum_value; std::vector valid_count; uint32_t frames = 0; }; - std::vector shards; + // The frames added on the host. Allocated by the first of them: with a GPU there are usually none. + Projection host; + std::mutex host_mutex; // guards the allocation and the frame count, not the sums + static constexpr size_t BANDS = 64; + std::mutex band_mutex[BANDS]; + std::atomic next_band{0}; #ifdef JFJOCH_USE_CUDA // Present when a GPU is available. Frames it can decode are accumulated there instead of on the // host - only the compressed chunk crosses PCIe - and its projection is folded in with the - // shards when the mask is read. Frames it cannot take (anything but bitshuffle+LZ4) still go to - // a host shard, so a run mixing compressions is handled without a second code path. + // host projection when the mask is read. Frames it cannot take (anything but bitshuffle+LZ4) still + // go to the host, so a run mixing compressions is handled without a second code path. // Built on a thread of its own: it allocates and clears several hundred megabytes of device // memory, and cudaMalloc synchronises the whole device, so doing it in the constructor would // stall the caller before it has read its first frame. The first AddImage waits for it, by @@ -90,17 +96,22 @@ class ShadowFinder { [[nodiscard]] ShadowAccumulatorGPU *Gpu() const; #endif - template void Add(const T *ptr, Projection &p); + // Add the pixels [begin, end) of one frame to the host projection. + template void Add(const T *ptr, size_t begin, size_t end); - // Sum the shards into one projection. max_value is only taken from a shard that actually - // counted the pixel - a shard that never saw it holds 0, which would beat a genuinely +#ifdef JFJOCH_USE_CUDA + // The device's projection with the host's folded in. max_value is only taken from a projection + // that actually counted the pixel - one that never saw it holds 0, which would beat a genuinely // negative maximum. [[nodiscard]] Projection Reduce() const; - // The projection, reduced on its first read and kept. The ring-centre fit, the mask and the - // beam-centre capture all read the same one, and on a 16 Mpx detector each reduction is 360 MB - // brought back from the device into fresh memory. Called with `m` held. + // That projection, made on its first read and kept. The ring-centre fit, the mask and the + // beam-centre capture all read the same one, and on a 16 Mpx detector each is 360 MB brought back + // from the device into fresh memory. mutable std::optional reduced; +#endif + // The projection the frames added so far make: the host's, or the one above where the device + // took frames. Called with `m` held. [[nodiscard]] const Projection &Reduced() const; public: @@ -116,20 +127,16 @@ public: // hardware. The projection is not centred on anything, so this may be set after the frames. void BeamCenter(float x, float y); - // Give each worker a shard to accumulate into. Must be called before the first AddImage, - // and costs 20 bytes per pixel per shard. - void SetShardCount(size_t n); - - // Accumulate one full converted-geometry image into shard `shard`. Gap / masked pixels - // (the pixel type's sentinel extreme) are skipped. `buffer` is scratch space for - // decompression, reused across the calls of one worker. - void AddImage(const DataMessage &data, std::vector &buffer, size_t shard = 0); + // Accumulate one full converted-geometry image. Gap / masked pixels (the pixel type's sentinel + // extreme) are skipped. `buffer` is scratch space for decompression, reused across the calls of + // one worker. + void AddImage(const DataMessage &data, std::vector &buffer); // Compute the shadow mask (SHADOW, TRANSMITTING or 0 = keep), of the converted pixel count. // TRANSMITTING marks the pieces of hardware that let part of the beam through, added after the // shadow proper; both are masked, and a consumer that must not see those pieces can tell them apart. - // Recomputed on each call from the projection, which is summed over the shards on the first read - // of it (GetMask or GetMeanProjection): frames added after that are not seen. + // Recomputed on each call from the projection, which is put together on the first read of it + // (GetMask or GetMeanProjection): frames added after that are not seen. // nthreads = 0 asks for all hardware threads. The per-pixel passes over a 16M-pixel detector // dominate this, and they are all exactly parallel. [[nodiscard]] std::vector GetMask(size_t nthreads = 0) const; @@ -140,7 +147,7 @@ public: [[nodiscard]] std::vector GetMeanProjection() const; // Let go of the projection the two above read, once the caller has what it wants of it: on a - // 16 Mpx detector it is 360 MB. A later read sums the shards again. + // 16 Mpx detector it is 360 MB. A later read puts it together again. void ReleaseProjection(); [[nodiscard]] uint32_t GetFrameCount() const; diff --git a/image_analysis/geom_refinement/BeamCenterFromBackground.cpp b/image_analysis/geom_refinement/BeamCenterFromBackground.cpp index b2c306b9e..19f468a65 100644 --- a/image_analysis/geom_refinement/BeamCenterFromBackground.cpp +++ b/image_analysis/geom_refinement/BeamCenterFromBackground.cpp @@ -7,6 +7,7 @@ #include #include +#include "../../common/CompressedImage.h" #include "../../common/JFJochMath.h" #include "../../common/ParallelFor.h" @@ -111,7 +112,6 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe float beam_y = start ? start->second : geom.GetBeamY_pxl(); constexpr int n_cells = RADIAL_BINS * SECTORS; - std::vector cell_of(n_pixels); std::vector sum(n_cells), sum_sq(n_cells), sum_jx(n_cells), sum_jy(n_cells); std::vector count(n_cells), count_all(n_cells); std::vector profile(RADIAL_BINS), d_profile(RADIAL_BINS), clip_limit(n_cells); @@ -127,6 +127,38 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe std::vector block_jy(static_cast(BLOCKS) * n_cells); std::vector block_count(static_cast(BLOCKS) * n_cells); + // Whether a pixel can take part at all, which does not depend on the centre. + std::vector> usable(n_pixels); + ParallelFor(BLOCKS, nthreads, [&](int b) { + for (size_t i = static_cast(block_row[b]) * W; i < static_cast(block_row[b + 1]) * W; i++) + usable[i] = pixel_mask[i] == 0 && std::isfinite(mean[i]); + }); + // The pixels of each block that fall in the band at the current centre, with their cell and value, + // in pixel order from the block's first pixel on. The clipping rounds read these instead of the + // whole detector, in the same order. + std::vector> band_cell(n_pixels); + std::vector> band_value(n_pixels); + std::vector band_pixels(BLOCKS); + + // The blocks' cells folded in block order. Each cell is folded on its own, so the cells are split + // over the threads and every cell is still summed in the same order. + const auto fold = [&](bool with_jacobian) { + ParallelChunks(n_cells, nthreads, [&](int c0, int c1) { + for (int c = c0; c < c1; c++) { + double s = 0, ss = 0, jx = 0, jy = 0; + int32_t n = 0; + for (int b = 0; b < BLOCKS; b++) { + const size_t k = static_cast(b) * n_cells + c; + s += block_sum[k]; ss += block_sum_sq[k]; + if (with_jacobian) { jx += block_jx[k]; jy += block_jy[k]; } + n += block_count[k]; + } + sum[c] = s; sum_sq[c] = ss; count[c] = n; + if (with_jacobian) { sum_jx[c] = jx; sum_jy[c] = jy; } + } + }); + }; + // Most pixels lie outside the band. Those clearly outside it in tan(2theta) = rho / lz - by a // margin far above float rounding - skip the square root and both atan2 below; every pixel the exact // test would keep still reaches it. @@ -148,11 +180,13 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe std::fill(b_jx, b_jx + n_cells, 0.0); std::fill(b_jy, b_jy + n_cells, 0.0); std::fill(b_count, b_count + n_cells, 0); + int32_t *cells = band_cell.data() + static_cast(block_row[b]) * W; + float *values = band_value.data() + static_cast(block_row[b]) * W; + size_t n_band = 0; for (int y = block_row[b]; y < block_row[b + 1]; y++) { for (int x = 0; x < W; x++) { const size_t i = static_cast(y) * W + x; - cell_of[i] = -1; - if (pixel_mask[i] != 0 || !std::isfinite(mean[i])) + if (!usable[i]) continue; const float u = (x - beam_x) * pixel_size; const float v = (y - beam_y) * pixel_size; @@ -183,7 +217,9 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe const float g_x = lz * lx / (rho * denominator); const float g_y = lz * ly / (rho * denominator); const float g_z = -rho / denominator; - cell_of[i] = cell; + cells[n_band] = cell; + values[n_band] = mean[i]; + n_band++; b_count[cell]++; b_sum[cell] += mean[i]; b_sum_sq[cell] += static_cast(mean[i]) * mean[i]; @@ -191,18 +227,9 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe b_jy[cell] += -pixel_size * (g_x * rot[1] + g_y * rot[4] + g_z * rot[7]); } } + band_pixels[b] = n_band; }); - for (int c = 0; c < n_cells; c++) { - double s = 0, ss = 0, jx = 0, jy = 0; - int32_t n = 0; - for (int b = 0; b < BLOCKS; b++) { - const size_t k = static_cast(b) * n_cells + c; - s += block_sum[k]; ss += block_sum_sq[k]; - jx += block_jx[k]; jy += block_jy[k]; - n += block_count[k]; - } - sum[c] = s; sum_sq[c] = ss; sum_jx[c] = jx; sum_jy[c] = jy; count[c] = n; - } + fold(true); count_all = count; // the Jacobian sums belong to the unclipped pixel set @@ -220,26 +247,19 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe std::fill(b_sum, b_sum + n_cells, 0.0); std::fill(b_sum_sq, b_sum_sq + n_cells, 0.0); std::fill(b_count, b_count + n_cells, 0); - const size_t lo = static_cast(block_row[b]) * W; - const size_t hi = static_cast(block_row[b + 1]) * W; - for (size_t i = lo; i < hi; i++) { - const int32_t c = cell_of[i]; - if (c < 0 || clip_limit[c] < 0.0f || mean[i] > clip_limit[c]) + const int32_t *cells = band_cell.data() + static_cast(block_row[b]) * W; + const float *values = band_value.data() + static_cast(block_row[b]) * W; + for (size_t j = 0; j < band_pixels[b]; j++) { + const int32_t c = cells[j]; + const float value = values[j]; + if (clip_limit[c] < 0.0f || value > clip_limit[c]) continue; b_count[c]++; - b_sum[c] += mean[i]; - b_sum_sq[c] += static_cast(mean[i]) * mean[i]; + b_sum[c] += value; + b_sum_sq[c] += static_cast(value) * value; } }); - for (int c = 0; c < n_cells; c++) { - double s = 0, ss = 0; - int32_t n = 0; - for (int b = 0; b < BLOCKS; b++) { - const size_t k = static_cast(b) * n_cells + c; - s += block_sum[k]; ss += block_sum_sq[k]; n += block_count[k]; - } - sum[c] = s; sum_sq[c] = ss; count[c] = n; - } + fold(false); } // Radial profile: the median over the sectors that have a mean, on rings that are diff --git a/rugnux/HotPixels.cpp b/rugnux/HotPixels.cpp index 0af35f712..5b898f8ca 100644 --- a/rugnux/HotPixels.cpp +++ b/rugnux/HotPixels.cpp @@ -281,11 +281,25 @@ HotPixelFinder::Result HotPixelFinder::GetMask(double oscillation_deg, double sp // The chance rate per ring, from the pixels lit on no more than half of their frames: whatever // lights those - reflections, zingers, noise above the bound - lights a defect-free pixel too. + // Counted in integers by blocks of rows in parallel, so the totals do not depend on the split. + std::vector> block_lit(BANDS), block_seen(BANDS); + const size_t rows_per_band = (height + BANDS - 1) / BANDS; + ParallelFor(static_cast(BANDS), nthreads, [&](int b) { + block_lit[b].assign(nrings, 0); + block_seen[b].assign(nrings, 0); + const size_t begin = std::min(width * height, b * rows_per_band * width); + const size_t end = std::min(width * height, (b + 1) * rows_per_band * width); + for (size_t i = begin; i < end; i++) + if (key[i] >= 0 && n_valid(i) > 0 && 2 * n_lit[i] <= n_valid(i)) { + block_lit[b][key[i] / SECTORS] += n_lit[i]; + block_seen[b][key[i] / SECTORS] += n_valid(i); + } + }); std::vector lit(nrings, 0.0), seen(nrings, 0.0); - for (size_t i = 0; i < width * height; i++) - if (key[i] >= 0 && n_valid(i) > 0 && 2 * n_lit[i] <= n_valid(i)) { - lit[key[i] / SECTORS] += n_lit[i]; - seen[key[i] / SECTORS] += n_valid(i); + for (int r = 0; r < nrings; r++) + for (size_t b = 0; b < BANDS; b++) { + lit[r] += static_cast(block_lit[b][r]); + seen[r] += static_cast(block_seen[b][r]); } std::vector k_chance(nrings, n + 1); for (int r = 0; r < nrings; r++) diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index ca1dab038..d99d8d8e1 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -244,10 +244,10 @@ namespace { // as signal. The margin is capped at a tenth of the sweep so a short run still has a sample. constexpr int PRESCAN_END_MARGIN_IMAGES = 5; - // Workers reading the pre-scan sample. Each owns a shard of the beam-stop projection so no two - // threads touch the same accumulator, and a shard costs 20 bytes per pixel - 362 MB on a 16M - // detector - so this is capped well below the worker count of the run proper. The accumulation - // is memory-bound rather than compute-bound, so a handful of workers already saturates it. + // Workers reading the pre-scan sample. Each holds detector-sized buffers of its own, and pages of + // fresh memory are slow to fault in when many threads do it at once, so this is capped well below + // the worker count of the run proper. The beam-stop projection is memory-bound rather than + // compute-bound, so a handful of workers already saturates it. constexpr size_t PRESCAN_MAX_WORKERS = 8; // The spot width is measured on a GROWING share of the pre-scan sample: every eighth frame of @@ -1014,9 +1014,9 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru // Read the sample on several workers. The reader serialises on the HDF5 lock, but the // decompression, the projection and the spot finding - which is all of the cost on a large - // detector - run in parallel. Each worker accumulates into a shard of its own, so nothing is - // locked while an image is added, and the per-frame results are stitched together in sample - // order below so the beam centre sees the same input however the workers interleaved. + // detector - run in parallel. The projection is integer sums, so the order the workers add their + // frames in does not reach it, and the per-frame results are stitched together in sample order + // below so the beam centre sees the same input however the workers interleaved. // // Two passes over the sample. The first builds the projection, which is all the shadow, the // defective pixels and the beam-centre capture below read; the second finds the spots, for the @@ -1026,13 +1026,12 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru const std::vector ordinals(sample.begin(), sample.end()); const size_t nworkers = std::min(std::max(config_.nthreads, 1), std::min(PRESCAN_MAX_WORKERS, ordinals.size())); - finder.SetShardCount(nworkers); { std::atomic next{0}; std::vector> futures; futures.reserve(nworkers); for (size_t t = 0; t < nworkers; t++) - futures.emplace_back(std::async(std::launch::async, [&, t] { + futures.emplace_back(std::async(std::launch::async, [&] { std::vector shadow_buffer; JFJochReaderRawImage raw_image; for (size_t i = next.fetch_add(1); i < ordinals.size(); i = next.fetch_add(1)) { @@ -1057,7 +1056,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru msg.image = raw_image.image; msg.number = ordinal; msg.original_number = image_idx; - finder.AddImage(msg, shadow_buffer, t); + finder.AddImage(msg, shadow_buffer); } } })); diff --git a/tests/ShadowFinderTest.cpp b/tests/ShadowFinderTest.cpp index 99473260e..cc89345f4 100644 --- a/tests/ShadowFinderTest.cpp +++ b/tests/ShadowFinderTest.cpp @@ -8,6 +8,7 @@ #include #include #include +#include #include #include "../common/DetectorSetup.h" @@ -168,16 +169,15 @@ TEST_CASE("ShadowFinder_MaskDoesNotDependOnTheThreadCount", "[ShadowFinder]") { CHECK(finder.GetMask(8) == one); } -// Workers accumulate into shards of their own and the shards are summed when the projection is read, -// so which worker saw which frame must not reach the answer - including the maximum, which only one -// shard holds when the reflection is on a single frame. -TEST_CASE("ShadowFinder_ShardingDoesNotChangeTheProjection", "[ShadowFinder]") { +// Workers add their frames concurrently, each starting at a different band of the projection, so +// which worker added which frame, and in what order, must not reach the answer - including the +// maximum, which one frame alone holds when the reflection is on a single frame. +TEST_CASE("ShadowFinder_ConcurrentWorkersDoNotChangeTheProjection", "[ShadowFinder]") { const DiffractionExperiment x = TestExperiment(); const PixelMask pixel_mask(x); ShadowFinder serial(x, pixel_mask); - ShadowFinder sharded(x, pixel_mask); - sharded.SetShardCount(4); + ShadowFinder concurrent(x, pixel_mask); std::vector> frames; std::vector buffer; @@ -185,22 +185,32 @@ TEST_CASE("ShadowFinder_ShardingDoesNotChangeTheProjection", "[ShadowFinder]") { frames.push_back(Scene(/*cross=*/false, /*reflection=*/f == 0)); DataMessage msg{}; msg.image = CompressedImage(frames.back(), W, H); - serial.AddImage(msg, buffer, 0); - sharded.AddImage(msg, buffer, static_cast(f) % 4); + serial.AddImage(msg, buffer); } + std::vector workers; + for (int t = 0; t < 4; t++) + workers.emplace_back([&, t] { + std::vector worker_buffer; + for (int f = NFRAMES - 1 - t; f >= 0; f -= 4) { + DataMessage msg{}; + msg.image = CompressedImage(frames[f], W, H); + concurrent.AddImage(msg, worker_buffer); + } + }); + for (auto &w : workers) w.join(); - CHECK(serial.GetFrameCount() == sharded.GetFrameCount()); + CHECK(serial.GetFrameCount() == concurrent.GetFrameCount()); const auto a = serial.GetMeanProjection(); - const auto b = sharded.GetMeanProjection(); + const auto b = concurrent.GetMeanProjection(); REQUIRE(a.size() == b.size()); // NAN marks a pixel nothing counted, and NAN != NAN, so compare the bits rather than the values. CHECK(memcmp(a.data(), b.data(), a.size() * sizeof(float)) == 0); - // The reflection is on one frame, so its maximum lives in a single shard. If the fold lost it, - // the mask would swallow the reflection instead of giving it back. - CHECK(serial.GetMask(1) == sharded.GetMask(1)); - CHECK(sharded.GetMask(1)[I(C - 14, C - 2)] == 0); + // The reflection is on one frame, so only that frame's maximum sees it. If it were lost, the mask + // would swallow the reflection instead of giving it back. + CHECK(serial.GetMask(1) == concurrent.GetMask(1)); + CHECK(concurrent.GetMask(1)[I(C - 14, C - 2)] == 0); } // Four opaque arms and a centred disk: the scene is invariant under a quarter turn, so the mask must -- 2.54.0 From 03dee2dda7d4e84759188a9708710457afc8218c Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 11:43:12 +0200 Subject: [PATCH 09/97] AzIntEngineGPU: four pixels per thread, one shared add per ring run The standalone GPU azimuthal integration did three shared-memory atomics per pixel on the same few ring addresses, which is what it was limited by. It now reads four pixels per thread as vector loads and keeps a running total per ring, flushed when the ring changes - the scheme the adaptive finder's ring pass (reduce_rings_shared) already uses. The npix % 4 leftovers are done one at a time. Used wherever the fused adaptive engine is not (fixed-threshold spot finding, the broker's non-adaptive path). Measured on a 16 Mpx sweep (1800 frames, --no-adaptive-spots, RTX 5080): 843 -> 295 us per call (min 621 -> 196 us). Not bit-identical, and the old kernel was not either: float atomics arrive in any order, so two runs of the OLD kernel already differ by up to 1.7e-6 relative in the per-frame profile; new vs old differs by up to 1.9e-6, the same order. Per-ring pixel counts are identical. Default rugnux runs do not reach this kernel (p.mtz md5 unchanged on three sets); on the fixed-threshold path p_unmerged.mtz is md5-identical to the old kernel's. New test: GPU vs CPU engine on a pixel count that is not a multiple of four, with masked and saturated pixels. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- image_analysis/azint/AzIntEngineGPU.cu | 57 ++++++++++++++++++++++---- tests/AdaptiveSpotFinderGPUTest.cpp | 41 ++++++++++++++++++ 2 files changed, 89 insertions(+), 9 deletions(-) diff --git a/image_analysis/azint/AzIntEngineGPU.cu b/image_analysis/azint/AzIntEngineGPU.cu index b35adaa95..7de941260 100644 --- a/image_analysis/azint/AzIntEngineGPU.cu +++ b/image_analysis/azint/AzIntEngineGPU.cu @@ -8,6 +8,15 @@ inline void cuda_err(cudaError_t val) { throw JFJochException(JFJochExceptionCategory::GPUCUDAError, cudaGetErrorString(val)); } +// Pushes one ring run's totals to the shared accumulators; nothing for an empty run. +__device__ __forceinline__ void flush_azim_run(float *s_sum, float *s_sum2, uint32_t *s_count, + int b, float r_sum, float r_sum2, uint32_t r_count) { + if (r_count == 0) return; // also covers the initial "no ring yet" + atomicAdd(&s_sum[b], r_sum); + atomicAdd(&s_sum2[b], r_sum2); + atomicAdd(&s_count[b], r_count); +} + __global__ void gpu_azim_shared( const uint16_t *__restrict__ pixel_to_bin, @@ -33,19 +42,49 @@ void gpu_azim_shared( __syncthreads(); - for (size_t idx = blockIdx.x * blockDim.x + threadIdx.x; - idx < num_pixels; - idx += blockDim.x * gridDim.x) { - uint16_t bin = pixel_to_bin[idx]; + // Four pixels per thread, read as vector loads, and a running total per ring pushed to shared + // memory only when the ring changes: consecutive pixels along a row mostly share a ring, and an + // atomic per pixel on the same few addresses is what this kernel was limited by. The same scheme + // as the adaptive spot finder's ring pass (reduce_rings_shared). The buffers come straight from + // cudaMalloc, aligned for int4/float4; the npix % 4 leftovers are done one at a time below. + const size_t stride = static_cast(blockDim.x) * gridDim.x; + const size_t nquad = num_pixels / 4; + for (size_t q = blockIdx.x * blockDim.x + threadIdx.x; q < nquad; q += stride) { + const int4 v4 = reinterpret_cast(input_buffer)[q]; + const ushort4 b4 = reinterpret_cast(pixel_to_bin)[q]; + const float4 c4 = reinterpret_cast(corrections)[q]; + const int32_t vq[4] = {v4.x, v4.y, v4.z, v4.w}; + const uint16_t bq[4] = {b4.x, b4.y, b4.z, b4.w}; + const float cq[4] = {c4.x, c4.y, c4.z, c4.w}; - int32_t v = input_buffer[idx]; - bool valid = (v != INT32_MIN) & (v != INT32_MAX); + int r_b = -1; + float r_sum = 0.0f, r_sum2 = 0.0f; + uint32_t r_count = 0; + #pragma unroll + for (int k = 0; k < 4; k++) { + const int32_t v = vq[k]; + const int b = bq[k]; + if (v == INT32_MIN || v == INT32_MAX || b >= azint_bins) continue; + if (b != r_b) { + flush_azim_run(s_sum, s_sum2, s_count, r_b, r_sum, r_sum2, r_count); + r_b = b; + r_sum = 0.0f; r_sum2 = 0.0f; r_count = 0; + } + const float val = static_cast(v) * cq[k]; + r_sum += val; + r_sum2 += val * val; + r_count += 1; + } + flush_azim_run(s_sum, s_sum2, s_count, r_b, r_sum, r_sum2, r_count); + } - if (bin < azint_bins && valid) { + for (size_t idx = 4 * nquad + blockIdx.x * blockDim.x + threadIdx.x; idx < num_pixels; idx += stride) { + const uint16_t bin = pixel_to_bin[idx]; + const int32_t v = input_buffer[idx]; + if (bin < azint_bins && v != INT32_MIN && v != INT32_MAX) { const float val = static_cast(v) * corrections[idx]; - const float val2 = val * val; atomicAdd(&s_sum[bin], val); - atomicAdd(&s_sum2[bin], val2); + atomicAdd(&s_sum2[bin], val * val); atomicAdd(&s_count[bin], 1); } } diff --git a/tests/AdaptiveSpotFinderGPUTest.cpp b/tests/AdaptiveSpotFinderGPUTest.cpp index 87d9d9e44..efee443fd 100644 --- a/tests/AdaptiveSpotFinderGPUTest.cpp +++ b/tests/AdaptiveSpotFinderGPUTest.cpp @@ -12,6 +12,7 @@ #include "../common/AzimuthalIntegrationMapping.h" #include "../common/AzimuthalIntegrationProfile.h" +#include "../image_analysis/azint/AzIntEngineCPU.h" #include "../image_analysis/azint/AzIntEngineGPU.h" #include "../image_analysis/spot_finding/AdaptiveSpotFinderCPU.h" #include "../image_analysis/spot_finding/AdaptiveSpotFinderGPU.h" @@ -161,6 +162,46 @@ TEST_CASE("AdaptiveSpotFinderGPU_AzimuthalIntegration", "[AdaptiveSpotFinderGPU] } } +// The GPU azimuthal integration against the CPU one, on a pixel count that is not a multiple of four (the +// kernel reads four pixels at a time and does the rest one by one) and with masked and saturated pixels +// in it. The per-ring pixel counts are integers and must agree exactly; the float sums only to rounding. +TEST_CASE("AzIntEngineGPU_MatchesCPU", "[AdaptiveSpotFinderGPU]") { + if (get_gpu_count() == 0) { + WARN("No CUDA GPU present. Skipping AzIntEngineGPU_MatchesCPU"); + return; + } + + DiffractionExperiment x(DetDECTRIS(1031, 1063, "Test", {})); + x.DetectorDistance_mm(80).BeamX_pxl(515).BeamY_pxl(530); + x.QSpacingForAzimInt_recipA(0.05).QRangeForAzimInt_recipA(0.05, 5.0); + REQUIRE(x.GetPixelsNum() % 4 != 0); + PixelMask pixel_mask(x); + AzimuthalIntegrationMapping mapping(x, pixel_mask); + + ImagePreprocessorBufferGPU buffer(x.GetPixelsNum()); + for (size_t i = 0; i < x.GetPixelsNum(); i++) + buffer[i] = (i % 997 == 0) ? INT32_MIN : (i % 1009 == 0) ? INT32_MAX + : 8 + static_cast((i * 7919) % 23); + REQUIRE(cudaMemcpy(buffer.getGPUBuffer(), buffer.getBuffer().data(), + x.GetPixelsNum() * sizeof(int32_t), cudaMemcpyHostToDevice) == cudaSuccess); + REQUIRE(cudaDeviceSynchronize() == cudaSuccess); + + AzimuthalIntegrationProfile cpu_profile(mapping), gpu_profile(mapping); + AzIntEngineCPU(mapping).Run(buffer, cpu_profile); + AzIntEngineGPU(mapping, std::make_shared()).Run(buffer, gpu_profile); + + REQUIRE(gpu_profile.GetPixelCount() == cpu_profile.GetPixelCount()); + const auto ref = cpu_profile.GetResult(); + const auto got = gpu_profile.GetResult(); + REQUIRE(ref.size() == got.size()); + for (size_t b = 0; b < ref.size(); b++) { + if (std::isnan(ref[b])) + CHECK(std::isnan(got[b])); + else + CHECK(got[b] == Catch::Approx(ref[b]).epsilon(1e-5)); + } +} + // The ring sums are built by atomics, which arrive in an arbitrary order, so the same frame has to be // re-run to show the engine agrees with itself: detection is a hard "value >= threshold" on integer // counts, and a threshold that wobbles between runs flips pixels on the boundary and with them the size -- 2.54.0 From f849e2d1bea1faf5a78f5a92cd79bb0563d16506 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 12:19:05 +0200 Subject: [PATCH 10/97] Pre-scan on the GPU: background beam-centre walk and beam-stop mask The two pre-scan steps that were still CPU-bound in a GPU build now run where the projection already is. - FindBeamCenterFromBackground: the per-iteration binning pass and the two clip rounds run on the device (BeamCenterBackgroundGPU); the fit itself stays on the host. Each cell is summed in the host's order (pixel order within the host's row blocks, blocks in order), and the per-pixel cell/derivative formula is shared (BackgroundBand.h). The angles come from BackgroundAtan2 (IEEE ops only) instead of atan2f, and both translation units are compiled without FMA contraction, so host and device give the same bits: 0 of 6.5 M pixels in a different cell, identical walks on the three in-house rotation sets. With glibc/CUDA atan2f and default contraction ~30 pixels per 16 Mpx sweep changed cell and the fitted centre moved by up to 0.05 px. - ShadowFinder::GetMask: the whole mask (pooling, ring medians, components, morphology, hole fill, arm search) runs on the device from ShadowAccumulatorGPU's projection (ShadowMaskGPU), so the 360 MB projection no longer comes back; the mean projection is divided on the device too (same bits). The two small fits over rings and sectors (BlockedOutTo, HarmonicFit) are shared with the host path in ShadowFinderInternal.h. Integers, comparisons, sorts and components are exact; the polarization trig, the Poisson log and the arm-search azimuth are not, so a pixel at a threshold can differ. The one-time change against the previous CPU arithmetic (BackgroundAtan2, no contraction), measured on the myoglobin, cytochrome C and thaumatin rotation sets: ring centre moves 0.002-0.045 px (fit sigma 0.75-1.2 px), beam-centre capture 0.01-0.04 px; beam-stop mask differs on 31 / 144 / 53 pixels of 259k / 144k / 198k (25 of the myoglobin ones are GPU-vs-CPU arithmetic in the mask, the rest follow the centre); hot-pixel mask identical. Spot width, integration radii, bandwidth, beam-centre arbitration, indexing, space group, cell, resolution and the merged statistics table are identical; only the error model moves in its 4th digit. CPU build: the same centres and decisions. Timing (GPU, box at load 30-38): ring walk 0.54 -> 0.23-0.27 s, mask 1.24-1.44 -> 0.18-0.22 s, beam-centre capture walk 1.1-1.3 -> 0.31-0.35 s. Tests: ShadowFinder_DeviceMaskMatchesHost, BeamCenterFromBackground_DeviceMatchesHost (bit-exact), plus [ShadowFinder], [BeamCenter], [HotPixelFinder]. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- image_analysis/CMakeLists.txt | 3 + .../beam_stop/ShadowAccumulatorGPU.cu | 10 + .../beam_stop/ShadowAccumulatorGPU.h | 6 + image_analysis/beam_stop/ShadowFinder.cpp | 242 +++---- .../beam_stop/ShadowFinderInternal.h | 90 +++ image_analysis/beam_stop/ShadowMaskGPU.cu | 636 ++++++++++++++++++ image_analysis/beam_stop/ShadowMaskGPU.h | 39 ++ .../geom_refinement/BackgroundBand.h | 107 +++ .../BeamCenterBackgroundGPU.cu | 210 ++++++ .../geom_refinement/BeamCenterBackgroundGPU.h | 40 ++ .../BeamCenterFromBackground.cpp | 126 ++-- .../BeamCenterFromBackground.h | 6 +- image_analysis/geom_refinement/CMakeLists.txt | 14 +- tests/BeamCenterFromBackgroundTest.cpp | 26 + tests/ShadowFinderTest.cpp | 88 +++ 15 files changed, 1468 insertions(+), 175 deletions(-) create mode 100644 image_analysis/beam_stop/ShadowFinderInternal.h create mode 100644 image_analysis/beam_stop/ShadowMaskGPU.cu create mode 100644 image_analysis/beam_stop/ShadowMaskGPU.h create mode 100644 image_analysis/geom_refinement/BackgroundBand.h create mode 100644 image_analysis/geom_refinement/BeamCenterBackgroundGPU.cu create mode 100644 image_analysis/geom_refinement/BeamCenterBackgroundGPU.h diff --git a/image_analysis/CMakeLists.txt b/image_analysis/CMakeLists.txt index 6abc665d9..d9a50ec73 100644 --- a/image_analysis/CMakeLists.txt +++ b/image_analysis/CMakeLists.txt @@ -99,7 +99,10 @@ ADD_LIBRARY(JFJochImageAnalysis STATIC beam_stop/ShadowFinder.cpp beam_stop/ShadowFinder.h $<$:beam_stop/ShadowAccumulatorGPU.cu> + $<$:beam_stop/ShadowMaskGPU.cu> beam_stop/ShadowAccumulatorGPU.h + beam_stop/ShadowFinderInternal.h + beam_stop/ShadowMaskGPU.h rotation_indexer/RotationIndexer.cpp rotation_indexer/RotationIndexer.h WriteReflections.cpp diff --git a/image_analysis/beam_stop/ShadowAccumulatorGPU.cu b/image_analysis/beam_stop/ShadowAccumulatorGPU.cu index 240c83d86..1499f1fec 100644 --- a/image_analysis/beam_stop/ShadowAccumulatorGPU.cu +++ b/image_analysis/beam_stop/ShadowAccumulatorGPU.cu @@ -190,3 +190,13 @@ void ShadowAccumulatorGPU::Download(std::vector &max_value, std::vector cudaMemcpyDeviceToHost, *stream)); cuda_err(cudaStreamSynchronize(*stream)); } + +std::vector ShadowAccumulatorGPU::Mask(const ShadowMaskSetup &setup, const std::vector &pixel_mask) { + FoldPending(); + return ShadowMaskOnDevice(setup, pixel_mask, gpu_max, gpu_sum, gpu_count, frames, *stream); +} + +std::vector ShadowAccumulatorGPU::MeanProjection(const std::vector &pixel_mask) { + FoldPending(); + return MeanProjectionOnDevice(pixel_mask, gpu_sum, gpu_count, npixels, *stream); +} diff --git a/image_analysis/beam_stop/ShadowAccumulatorGPU.h b/image_analysis/beam_stop/ShadowAccumulatorGPU.h index c45a424c7..71907a0be 100644 --- a/image_analysis/beam_stop/ShadowAccumulatorGPU.h +++ b/image_analysis/beam_stop/ShadowAccumulatorGPU.h @@ -10,6 +10,7 @@ #include "../../common/CompressedImage.h" #include "../image_preprocessing/BSLZ4DecoderGPU.h" #include "../indexing/CUDAMemHelpers.h" +#include "ShadowMaskGPU.h" // The beam-stop projection accumulated on the device: only the compressed chunk crosses PCIe, and // both the decode and the per-pixel maximum / sum / count run on the GPU. The projection comes back @@ -59,6 +60,11 @@ public: [[nodiscard]] uint32_t GetFrameCount() const { return frames; } + // The beam-stop mask and the mean projection, made where the projection is (ShadowMaskGPU.h), so + // that it does not have to come back at all. + std::vector Mask(const ShadowMaskSetup &setup, const std::vector &pixel_mask); + std::vector MeanProjection(const std::vector &pixel_mask); + // Bring the projection back to the host, folding in whatever the last batch still holds. Cheap // to call once; it moves 20 bytes per pixel. void Download(std::vector &max_value, std::vector &sum_value, diff --git a/image_analysis/beam_stop/ShadowFinder.cpp b/image_analysis/beam_stop/ShadowFinder.cpp index 95846c533..3d62d6d6d 100644 --- a/image_analysis/beam_stop/ShadowFinder.cpp +++ b/image_analysis/beam_stop/ShadowFinder.cpp @@ -2,6 +2,7 @@ // SPDX-License-Identifier: GPL-3.0-only #include "ShadowFinder.h" +#include "ShadowFinderInternal.h" #include #include @@ -18,64 +19,9 @@ #include "../../common/ParallelFor.h" #include "../../common/JFJochException.h" -// A pixel is shadow when its background is below this fraction of the background it is -// compared against. -constexpr float SHADOW_RATIO = 0.50f; +using namespace shadow_finder; -// The boundary grows outward into partially shadowed pixels down to this fraction, but no -// further than PENUMBRA_MAX_PX from the core. A pin or a loop casts a wide half-shadow, so the -// reach is a good deal more than the beam stop's own edge needs. -constexpr float PENUMBRA_RATIO = 0.75f; -constexpr int PENUMBRA_MAX_PX = 30; - -// Bridge small breaks along the holder arm. A module gap wider than this is bridged separately for -// the arm search (see bridge_gaps). -constexpr int BRIDGE_PX = 6; - -// A pixel whose maximum reaches this recorded a real reflection and is never masked - a -// beam stop cannot block a reflection that was measured. -constexpr int64_t MIN_REFLECTION = 25; - -// How far below the background it is compared against a pixel must sit before the dip is -// believed, in standard deviations of the counts that back it. The counts are photons, so their -// scatter is Poisson and the deficit is measured against it rather than against a fixed number: -// on a well-exposed sweep a third of the background missing is overwhelming, and on a handful of -// low-background frames the same third is noise. Without this a six-frame pre-scan of a -// low-background sweep masks three quarters of the detector. -constexpr double MIN_DEFICIT_SIGMA = 6.0; - -// Smallest region the per-pixel test may return. A shadow is cast by something physical and is -// correspondingly large; an isolated patch this small is the background wandering, not hardware. -// This is what keeps the test specific now that a shadow no longer has to touch the direct beam. -constexpr int MIN_SHADOW_PIXELS = 2000; - -// ... and of those, how many must be deep (below SHADOW_RATIO) for a region of merely DIM pixels - -// hardware that lets part of the beam through - to count. A thin holder arm is dim along most of -// its length and deep in places; the background drifting over a detector's edge is dim everywhere -// and deep nowhere. -constexpr int MIN_CORE_PIXELS = 200; - -// The arm search compares a pixel with its ring as the ring actually varies around the beam. A ring -// of background is not flat once divided by the polarization factor when that factor is not the -// beam's: the remainder is a second harmonic in azimuth, cos 2phi, which reaches tens of percent at -// high angle. It is measured over radial bands of HARMONIC_BAND_PX, from the median of each of -// HARMONIC_SECTORS sectors that holds at least MIN_SECTOR_PIXELS pixels. -constexpr int HARMONIC_BAND_PX = 64; -constexpr int HARMONIC_SECTORS = 24; -constexpr int MIN_SECTOR_PIXELS = 200; - -// Side of the box the background is pooled over before testing. Its area is how many pixels back -// a ring's countability test, which decides where an azimuthal comparison is possible at all. -constexpr int POOL_PX = 5; -constexpr double MEAN_POOLED_PIXELS = POOL_PX * POOL_PX; - -// A ring with fewer valid pixels than this says nothing about whether it was counted. -constexpr int MIN_RING_PIXELS = 32; - -// A ring lies wholly inside the stop when its background is below this fraction of the background -// further out. This asks about a whole ring rather than about a pixel, so it keeps a threshold of -// its own and does not follow SHADOW_RATIO. -constexpr float BLOCKED_RING_RATIO = 0.35f; +static_assert(ShadowFinder::SHADOW == MASK_SHADOW && ShadowFinder::TRANSMITTING == MASK_TRANSMITTING); // Binary-image helpers on a width*height frame stored row-major as char (0/1). All run once, // at GetMask() time, and all are O(pixels) rather than O(pixels * radius). @@ -635,6 +581,10 @@ void ShadowFinder::ReleaseProjection() { std::vector ShadowFinder::GetMeanProjection() const { std::unique_lock ul(m); +#ifdef JFJOCH_USE_CUDA + if (Gpu() && gpu->GetFrameCount() > 0 && host.frames == 0) + return gpu->MeanProjection(pixel_mask); +#endif const Projection &p = Reduced(); const auto &sum_value = p.sum_value; const auto &valid_count = p.valid_count; @@ -652,6 +602,28 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { std::unique_lock ul(m); if (nthreads == 0) nthreads = std::max(1u, std::thread::hardware_concurrency()); +#ifdef JFJOCH_USE_CUDA + // Where every frame went to the device the mask is made there too, from the projection as it lies. + if (Gpu() && gpu->GetFrameCount() > 0 && host.frames == 0) { + const float diag = std::hypot(static_cast(width), static_cast(height)); + if (!std::isfinite(beam_x) || !std::isfinite(beam_y) + || std::fabs(beam_x - width * 0.5f) > 4.0f * diag || std::fabs(beam_y - height * 0.5f) > 4.0f * diag) + return std::vector(static_cast(width) * height, 0); + ShadowMaskSetup setup; + setup.width = width; + setup.height = height; + setup.beam_x = beam_x; + setup.beam_y = beam_y; + const auto rot = geometry.GetDetectorMatrix().arr(); + for (int k = 0; k < 9; k++) + setup.det_matrix[k] = rot[k]; + setup.pixel_size_mm = geometry.GetPixelSize_mm(); + setup.distance_mm = geometry.GetDetectorDistance_mm(); + setup.has_polarization = polarization.has_value(); + setup.polarization = polarization.value_or(0.0f); + return gpu->Mask(setup, pixel_mask); + } +#endif const Projection &p = Reduced(); const auto &max_value = p.max_value; const auto &sum_value = p.sum_value; @@ -804,25 +776,7 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { // disk it masked. The median over the rings this walk is willing to judge is what "typically" // means, and it is robust from both sides - to a bright ring, and to a corner ring of a handful // of pixels whose median is one pixel's mean. - std::vector judgeable; - for (int rad = 0; rad <= max_radius; rad++) - if (ring_pixels[rad] >= MIN_RING_PIXELS) - judgeable.push_back(baseline[rad]); - float typical_background = 0.0f; - if (!judgeable.empty()) { - const auto middle = judgeable.begin() + judgeable.size() / 2; - std::nth_element(judgeable.begin(), middle, judgeable.end()); - typical_background = *middle; - } - - int blocked_out_to = -1; - for (int rad = 0; rad <= max_radius; rad++) { - if (ring_pixels[rad] < MIN_RING_PIXELS) - continue; - if (baseline[rad] >= BLOCKED_RING_RATIO * typical_background) - break; - blocked_out_to = rad; - } + const int blocked_out_to = BlockedOutTo(baseline, ring_pixels); // The counts a pixel's pooled background is made of, and the counts the ring says it should // have had. The test is on the deficit between them, in units of its own Poisson scatter. @@ -975,51 +929,17 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { sector_values[k].insert(sector_values[k].end(), values[k].begin(), values[k].end()); }); block_values.clear(); - std::vector harm_c(n_bands, 0.0f), harm_s(n_bands, 0.0f); - ParallelFor(n_bands, nthreads, [&](int band) { - std::vector med(HARMONIC_SECTORS, -1.0), c(HARMONIC_SECTORS), s(HARMONIC_SECTORS); - for (int k = 0; k < HARMONIC_SECTORS; k++) { - auto &v = sector_values[static_cast(band) * HARMONIC_SECTORS + k]; - if (v.size() >= MIN_SECTOR_PIXELS) { - std::nth_element(v.begin(), v.begin() + v.size() / 2, v.end()); - med[k] = v[v.size() / 2]; - } - const double phi = (k + 0.5) * 2.0 * std::numbers::pi / HARMONIC_SECTORS - std::numbers::pi; - c[k] = std::cos(2 * phi); - s[k] = std::sin(2 * phi); - } - // Least squares of med = m + p cos 2phi + q sin 2phi over the sectors that are not themselves - // dim against the fit, three times over as the baseline is; the model is relative, 1 + (p cos + q sin)/m. - std::vector use(HARMONIC_SECTORS); - for (int k = 0; k < HARMONIC_SECTORS; k++) - use[k] = med[k] >= 0; - for (int iter = 0; iter < 3; iter++) { - double n = 0, sc = 0, ss = 0, scc = 0, sss = 0, scs = 0, y = 0, yc = 0, ys = 0; - for (int k = 0; k < HARMONIC_SECTORS; k++) { - if (!use[k]) continue; - n++; sc += c[k]; ss += s[k]; scc += c[k] * c[k]; sss += s[k] * s[k]; scs += c[k] * s[k]; - y += med[k]; yc += med[k] * c[k]; ys += med[k] * s[k]; - } - // Sectors crowded into a narrow arc cannot tell a harmonic from a level; the determinant of - // the normal matrix, per sector cubed, is 1/4 on a full ring. - const double det = n * (scc * sss - scs * scs) - sc * (sc * sss - scs * ss) + ss * (sc * scs - scc * ss); - if (n < 6 || det < 0.01 * n * n * n) { - harm_c[band] = harm_s[band] = 0.0f; - return; - } - const double m = (y * (scc * sss - scs * scs) - sc * (yc * sss - scs * ys) + ss * (yc * scs - scc * ys)) / det; - const double p = (n * (yc * sss - ys * scs) - y * (sc * sss - scs * ss) + ss * (sc * ys - yc * ss)) / det; - const double q = (n * (scc * ys - scs * yc) - sc * (sc * ys - yc * ss) + y * (sc * scs - scc * ss)) / det; - if (m <= 0) { - harm_c[band] = harm_s[band] = 0.0f; - return; - } - harm_c[band] = static_cast(p / m); - harm_s[band] = static_cast(q / m); - for (int k = 0; k < HARMONIC_SECTORS; k++) - use[k] = med[k] >= 0 && med[k] >= PENUMBRA_RATIO * (1.0 + harm_c[band] * c[k] + harm_s[band] * s[k]); + // The median of each sector with enough pixels to have one. + std::vector sector_median(n_sectors, -1.0); + ParallelFor(static_cast(n_sectors), nthreads, [&](int k) { + auto &v = sector_values[k]; + if (v.size() >= MIN_SECTOR_PIXELS) { + std::nth_element(v.begin(), v.begin() + v.size() / 2, v.end()); + sector_median[k] = v[v.size() / 2]; } }); + std::vector harm_c, harm_s; + HarmonicFit(sector_median, n_bands, harm_c, harm_s); Plane dim = filled_plane(n_pixels, 0, nthreads); ParallelChunks(n_pixels, nthreads, [&](int lo, int hi) { @@ -1062,3 +982,87 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { }); return mask; } + +namespace shadow_finder { + +int BlockedOutTo(const std::vector &baseline, const std::vector &ring_pixels) { + const int max_radius = static_cast(baseline.size()) - 1; + // A ring lies inside the stop when its background is a fraction of what this detector's + // background typically is. Counting statistics cannot decide this: on a bright dataset the + // shadow is still well counted. The comparison used to be against the LARGEST background of any + // ring further out, and that reads a sample whose background peaks in a strong ring away from + // the beam - a powder standard, a strong solvent ring - as a beam stop the size of that ring: + // the ordinary background inside it is legitimately below a third of the peak. On one corpus + // dataset it declared 16 % of the detector to be stop, with diffraction rings visible inside the + // disk it masked. The median over the rings this walk is willing to judge is what "typically" + // means, and it is robust from both sides - to a bright ring, and to a corner ring of a handful + // of pixels whose median is one pixel's mean. + std::vector judgeable; + for (int rad = 0; rad <= max_radius; rad++) + if (ring_pixels[rad] >= MIN_RING_PIXELS) + judgeable.push_back(baseline[rad]); + float typical_background = 0.0f; + if (!judgeable.empty()) { + const auto middle = judgeable.begin() + judgeable.size() / 2; + std::nth_element(judgeable.begin(), middle, judgeable.end()); + typical_background = *middle; + } + + int blocked_out_to = -1; + for (int rad = 0; rad <= max_radius; rad++) { + if (ring_pixels[rad] < MIN_RING_PIXELS) + continue; + if (baseline[rad] >= BLOCKED_RING_RATIO * typical_background) + break; + blocked_out_to = rad; + } + return blocked_out_to; +} + +void HarmonicFit(const std::vector §or_median, int n_bands, + std::vector &harm_c, std::vector &harm_s) { + harm_c.assign(n_bands, 0.0f); + harm_s.assign(n_bands, 0.0f); + for (int band = 0; band < n_bands; band++) { + std::vector med(HARMONIC_SECTORS), c(HARMONIC_SECTORS), s(HARMONIC_SECTORS); + for (int k = 0; k < HARMONIC_SECTORS; k++) { + med[k] = sector_median[static_cast(band) * HARMONIC_SECTORS + k]; + const double phi = (k + 0.5) * 2.0 * std::numbers::pi / HARMONIC_SECTORS - std::numbers::pi; + c[k] = std::cos(2 * phi); + s[k] = std::sin(2 * phi); + } + // Least squares of med = m + p cos 2phi + q sin 2phi over the sectors that are not themselves + // dim against the fit, three times over as the baseline is; the model is relative, 1 + (p cos + q sin)/m. + std::vector use(HARMONIC_SECTORS); + for (int k = 0; k < HARMONIC_SECTORS; k++) + use[k] = med[k] >= 0; + for (int iter = 0; iter < 3; iter++) { + double n = 0, sc = 0, ss = 0, scc = 0, sss = 0, scs = 0, y = 0, yc = 0, ys = 0; + for (int k = 0; k < HARMONIC_SECTORS; k++) { + if (!use[k]) continue; + n++; sc += c[k]; ss += s[k]; scc += c[k] * c[k]; sss += s[k] * s[k]; scs += c[k] * s[k]; + y += med[k]; yc += med[k] * c[k]; ys += med[k] * s[k]; + } + // Sectors crowded into a narrow arc cannot tell a harmonic from a level; the determinant of + // the normal matrix, per sector cubed, is 1/4 on a full ring. + const double det = n * (scc * sss - scs * scs) - sc * (sc * sss - scs * ss) + ss * (sc * scs - scc * ss); + if (n < 6 || det < 0.01 * n * n * n) { + harm_c[band] = harm_s[band] = 0.0f; + break; + } + const double m = (y * (scc * sss - scs * scs) - sc * (yc * sss - scs * ys) + ss * (yc * scs - scc * ys)) / det; + const double p = (n * (yc * sss - ys * scs) - y * (sc * sss - scs * ss) + ss * (sc * ys - yc * ss)) / det; + const double q = (n * (scc * ys - scs * yc) - sc * (sc * ys - yc * ss) + y * (sc * scs - scc * ss)) / det; + if (m <= 0) { + harm_c[band] = harm_s[band] = 0.0f; + break; + } + harm_c[band] = static_cast(p / m); + harm_s[band] = static_cast(q / m); + for (int k = 0; k < HARMONIC_SECTORS; k++) + use[k] = med[k] >= 0 && med[k] >= PENUMBRA_RATIO * (1.0 + harm_c[band] * c[k] + harm_s[band] * s[k]); + } + } +} + +} // namespace shadow_finder diff --git a/image_analysis/beam_stop/ShadowFinderInternal.h b/image_analysis/beam_stop/ShadowFinderInternal.h new file mode 100644 index 000000000..63e292a70 --- /dev/null +++ b/image_analysis/beam_stop/ShadowFinderInternal.h @@ -0,0 +1,90 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +// What ShadowFinder::GetMask shares with its device twin (ShadowMaskGPU): the constants it tests +// against, and the two small fits made over whole rings and sectors, which stay on the host on both +// paths. + +#include +#include + +namespace shadow_finder { + +// The values of ShadowFinder::SHADOW and ShadowFinder::TRANSMITTING, for the device code, which does +// not include ShadowFinder.h. +inline constexpr uint32_t MASK_SHADOW = 1; +inline constexpr uint32_t MASK_TRANSMITTING = 2; + +// A pixel is shadow when its background is below this fraction of the background it is +// compared against. +inline constexpr float SHADOW_RATIO = 0.50f; + +// The boundary grows outward into partially shadowed pixels down to this fraction, but no +// further than PENUMBRA_MAX_PX from the core. A pin or a loop casts a wide half-shadow, so the +// reach is a good deal more than the beam stop's own edge needs. +inline constexpr float PENUMBRA_RATIO = 0.75f; +inline constexpr int PENUMBRA_MAX_PX = 30; + +// Bridge small breaks along the holder arm. A module gap wider than this is bridged separately for +// the arm search (see bridge_gaps). +inline constexpr int BRIDGE_PX = 6; + +// A pixel whose maximum reaches this recorded a real reflection and is never masked - a +// beam stop cannot block a reflection that was measured. +inline constexpr int64_t MIN_REFLECTION = 25; + +// How far below the background it is compared against a pixel must sit before the dip is +// believed, in standard deviations of the counts that back it. The counts are photons, so their +// scatter is Poisson and the deficit is measured against it rather than against a fixed number: +// on a well-exposed sweep a third of the background missing is overwhelming, and on a handful of +// low-background frames the same third is noise. Without this a six-frame pre-scan of a +// low-background sweep masks three quarters of the detector. +inline constexpr double MIN_DEFICIT_SIGMA = 6.0; + +// Smallest region the per-pixel test may return. A shadow is cast by something physical and is +// correspondingly large; an isolated patch this small is the background wandering, not hardware. +// This is what keeps the test specific now that a shadow no longer has to touch the direct beam. +inline constexpr int MIN_SHADOW_PIXELS = 2000; + +// ... and of those, how many must be deep (below SHADOW_RATIO) for a region of merely DIM pixels - +// hardware that lets part of the beam through - to count. A thin holder arm is dim along most of +// its length and deep in places; the background drifting over a detector's edge is dim everywhere +// and deep nowhere. +inline constexpr int MIN_CORE_PIXELS = 200; + +// The arm search compares a pixel with its ring as the ring actually varies around the beam. A ring +// of background is not flat once divided by the polarization factor when that factor is not the +// beam's: the remainder is a second harmonic in azimuth, cos 2phi, which reaches tens of percent at +// high angle. It is measured over radial bands of HARMONIC_BAND_PX, from the median of each of +// HARMONIC_SECTORS sectors that holds at least MIN_SECTOR_PIXELS pixels. +inline constexpr int HARMONIC_BAND_PX = 64; +inline constexpr int HARMONIC_SECTORS = 24; +inline constexpr int MIN_SECTOR_PIXELS = 200; + +// Side of the box the background is pooled over before testing. Its area is how many pixels back +// a ring's countability test, which decides where an azimuthal comparison is possible at all. +inline constexpr int POOL_PX = 5; +inline constexpr double MEAN_POOLED_PIXELS = POOL_PX * POOL_PX; + +// A ring with fewer valid pixels than this says nothing about whether it was counted. +inline constexpr int MIN_RING_PIXELS = 32; + +// A ring lies wholly inside the stop when its background is below this fraction of the background +// further out. This asks about a whole ring rather than about a pixel, so it keeps a threshold of +// its own and does not follow SHADOW_RATIO. +inline constexpr float BLOCKED_RING_RATIO = 0.35f; + +// The rings that lie wholly inside the stop: walking outward, every judgeable ring (at least +// MIN_RING_PIXELS pixels) before the first whose baseline reaches BLOCKED_RING_RATIO of the typical +// background. -1 when there is none. See GetMask. +int BlockedOutTo(const std::vector &baseline, const std::vector &ring_pixels); + +// The second harmonic in azimuth of each radial band, relative to its level, fitted to the medians of +// its HARMONIC_SECTORS sectors (sector_median[band * HARMONIC_SECTORS + k], negative where the sector +// has too few pixels). See GetMask. +void HarmonicFit(const std::vector §or_median, int n_bands, + std::vector &harm_c, std::vector &harm_s); + +} // namespace shadow_finder diff --git a/image_analysis/beam_stop/ShadowMaskGPU.cu b/image_analysis/beam_stop/ShadowMaskGPU.cu new file mode 100644 index 000000000..5961535c2 --- /dev/null +++ b/image_analysis/beam_stop/ShadowMaskGPU.cu @@ -0,0 +1,636 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#include "ShadowMaskGPU.h" + +#include + +#include "ShadowFinderInternal.h" +#include "../../common/JFJochMath.h" +#include "../indexing/CUDAMemHelpers.h" +#include "../../common/JFJochException.h" + +using namespace shadow_finder; + +namespace { + +constexpr int THREADS = 256; + +void check(cudaError_t err, const char *what) { + if (err != cudaSuccess) + throw JFJochException(JFJochExceptionCategory::GPUCUDAError, + std::string("Beam stop mask: ") + what + ": " + cudaGetErrorString(err)); +} + +unsigned grid(size_t n) { + return static_cast((n + THREADS - 1) / THREADS); +} + +// A float as an unsigned integer that sorts the same way, and back. +__device__ uint32_t float_key(float f) { + const uint32_t u = __float_as_uint(f); + return (u & 0x80000000u) ? ~u : (u | 0x80000000u); +} +__device__ float key_float(uint32_t k) { + return __uint_as_float((k & 0x80000000u) ? (k & 0x7fffffffu) : ~k); +} + +// The first index in a sorted key array whose key is not below `value`. +__device__ size_t lower_bound(const uint64_t *keys, size_t n, uint64_t value) { + size_t lo = 0, hi = n; + while (lo < hi) { + const size_t mid = (lo + hi) / 2; + if (keys[mid] < value) lo = mid + 1; + else hi = mid; + } + return lo; +} + +__device__ double poisson_deficit_sigma(double observed, double expected) { + if (expected <= 0.0 || observed >= expected) + return 0.0; + const double ll = 2.0 * (expected - observed + (observed > 0.0 ? observed * log(observed / expected) : 0.0)); + return ll > 0.0 ? sqrt(ll) : 0.0; +} + +// DiffractionGeometry::CalcAzIntPolarizationCorr about the centre the rings are drawn about. +__device__ float polarization_factor(const ShadowMaskSetup &s, float x, float y) { + const float u = (x - s.beam_x) * s.pixel_size_mm; + const float v = (y - s.beam_y) * s.pixel_size_mm; + const float *m = s.det_matrix; + const float lx = m[0] * u + m[1] * v + m[2] * s.distance_mm; + const float ly = m[3] * u + m[4] * v + m[5] * s.distance_mm; + const float lz = m[6] * u + m[7] * v + m[8] * s.distance_mm; + const float two_theta = atan2f(sqrtf(lx * lx + ly * ly), lz); + float phi = atan2f(ly, lx); + if (phi < 0) + phi += 2.0f * PI; + const float cos_2theta = cosf(two_theta); + const float cos_2theta_2 = cos_2theta * cos_2theta; + const float cos_2phi = cosf(2.0f * phi); + return 0.5f * (1.0f + cos_2theta_2 - s.polarization * cos_2phi * (1.0f - cos_2theta_2)); +} + +__global__ void setup_kernel(ShadowMaskSetup s, const uint32_t *__restrict__ pixel_mask, + const int64_t *__restrict__ sum_value, const uint32_t *__restrict__ valid_count, + float *__restrict__ pol, char *__restrict__ valid, int *__restrict__ radius, + double *__restrict__ num, int32_t *__restrict__ den, int *__restrict__ max_radius) { + const size_t n = static_cast(s.width) * s.height; + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) + return; + const int x = static_cast(i % s.width), y = static_cast(i / s.width); + const float dx = x - s.beam_x, dy = y - s.beam_y; + const float p = s.has_polarization ? polarization_factor(s, static_cast(x), static_cast(y)) : 1.0f; + pol[i] = p; + float mean = 0.0f; + char v = 0; + if (valid_count[i] > 0 && pixel_mask[i] == 0 && p > 0.0f) { + mean = static_cast(static_cast(sum_value[i]) / valid_count[i] / p); + v = 1; + } + valid[i] = v; + num[i] = v ? mean : 0.0; + den[i] = v ? 1 : 0; + const int r = static_cast(lroundf(sqrtf(dx * dx + dy * dy))); + radius[i] = r; + atomicMax(max_radius, r); +} + +// Sum over the k x k box about each pixel, zero outside the frame: one running sum per row, then one +// per column, each with exactly the terms and order of the host's (box_sum in ShadowFinder.cpp). +template +__global__ void box_rows(const T *__restrict__ in, T *__restrict__ out, int W, int H, int half) { + const int y = blockIdx.x * blockDim.x + threadIdx.x; + if (y >= H) return; + const T *src = in + static_cast(y) * W; + T *dst = out + static_cast(y) * W; + T s = 0; + for (int x = 0; x <= min(half, W - 1); x++) + s += src[x]; + for (int x = 0; x < W; x++) { + dst[x] = s; + if (x + half + 1 < W) s += src[x + half + 1]; + if (x - half >= 0) s -= src[x - half]; + } +} + +template +__global__ void box_columns(const T *__restrict__ in, T *__restrict__ out, int W, int H, int half) { + const int x = blockIdx.x * blockDim.x + threadIdx.x; + if (x >= W) return; + T s = 0; + for (int y = 0; y <= min(half, H - 1); y++) + s += in[static_cast(y) * W + x]; + for (int y = 0; y < H; y++) { + out[static_cast(y) * W + x] = s; + if (y + half + 1 < H) s += in[static_cast(y + half + 1) * W + x]; + if (y - half >= 0) s -= in[static_cast(y - half) * W + x]; + } +} + +// Dilation of a 0/1 plane by the (2r+1) square clipped to the frame, as a count over a sliding window +// along rows and then along columns (dilate in ShadowFinder.cpp). +__global__ void dilate_rows(const char *__restrict__ in, char *__restrict__ out, int W, int H, int r) { + const int y = blockIdx.x * blockDim.x + threadIdx.x; + if (y >= H) return; + const char *src = in + static_cast(y) * W; + char *dst = out + static_cast(y) * W; + int count = 0; + for (int x = 0; x <= min(r, W - 1); x++) + count += src[x]; + for (int x = 0; x < W; x++) { + dst[x] = count > 0; + if (x + r + 1 < W) count += src[x + r + 1]; + if (x - r >= 0) count -= src[x - r]; + } +} + +__global__ void dilate_columns(const char *__restrict__ in, char *__restrict__ out, int W, int H, int r) { + const int x = blockIdx.x * blockDim.x + threadIdx.x; + if (x >= W) return; + int count = 0; + for (int y = 0; y <= min(r, H - 1); y++) + count += in[static_cast(y) * W + x]; + for (int y = 0; y < H; y++) { + out[static_cast(y) * W + x] = count > 0; + if (y + r + 1 < H) count += in[static_cast(y + r + 1) * W + x]; + if (y - r >= 0) count -= in[static_cast(y - r) * W + x]; + } +} + +__global__ void pooled_kernel(size_t n, const double *__restrict__ pooled_sum, const int32_t *__restrict__ pooled_count, + const char *__restrict__ valid, const int *__restrict__ radius, + float *__restrict__ pooled, uint64_t *__restrict__ ring_key) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + const float p = pooled_count[i] > 0 ? static_cast(pooled_sum[i] / pooled_count[i]) : 0.0f; + pooled[i] = p; + ring_key[i] = valid[i] ? (static_cast(radius[i]) << 32) | float_key(p) : UINT64_MAX; +} + +// Where each of the keys' leading 32-bit groups (ring or sector) starts in a sorted key array. +__global__ void group_offsets(const uint64_t *__restrict__ keys, size_t n, int groups, int *__restrict__ offset) { + const int g = blockIdx.x * blockDim.x + threadIdx.x; + if (g > groups) return; + offset[g] = static_cast(lower_bound(keys, n, static_cast(g) << 32)); +} + +// The ring's baseline, three iterations of an order statistic over its sorted values (GetMask). +__global__ void baseline_kernel(int rings, const uint64_t *__restrict__ keys, const int *__restrict__ offset, + float *__restrict__ baseline) { + const int r = blockIdx.x * blockDim.x + threadIdx.x; + if (r >= rings) return; + const int lo = offset[r], n = offset[r + 1] - offset[r]; + int excluded = 0; + float b = 0.0f; + for (int iter = 0; iter < 3; iter++) { + const int avail = n - excluded; + b = (avail <= 0) ? 0.0f : key_float(static_cast(keys[lo + excluded + avail / 2])); + const float d = fmaxf(b, 1e-6f); + int excl = 0; + while (excl < n && key_float(static_cast(keys[lo + excl])) / d < SHADOW_RATIO) + excl++; + excluded = excl; + } + baseline[r] = b; +} + +__global__ void low_kernel(size_t n, uint32_t frames, const char *__restrict__ valid, const float *__restrict__ pooled, + const int32_t *__restrict__ pooled_count, const float *__restrict__ pol, + const int *__restrict__ radius, const float *__restrict__ baseline, + float *__restrict__ ratio, float *__restrict__ deficit, char *__restrict__ low) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + const float base = baseline[radius[i]]; + const float rt = valid[i] ? pooled[i] / fmaxf(base, 1e-6f) : 1.0f; + ratio[i] = rt; + float df = 0.0f; + char l = 0; + if (valid[i]) { + const double counted = static_cast(frames) * pooled_count[i] * pol[i]; + df = static_cast(poisson_deficit_sigma(pooled[i] * counted, base * counted)); + l = rt < SHADOW_RATIO && df > MIN_DEFICIT_SIGMA; + } + deficit[i] = df; + low[i] = l; +} + +// 8-connected components by union-find: every component ends up named by its smallest pixel index, +// whatever order the unions ran in. +__device__ int find_root(const int *parent, int x) { + while (parent[x] != x) + x = parent[x]; + return x; +} + +__global__ void cc_init(size_t n, const char *__restrict__ member, int *__restrict__ parent) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + parent[i] = member[i] ? static_cast(i) : -1; +} + +__device__ void cc_unite(int *parent, int a, int b) { + while (true) { + a = find_root(parent, a); + b = find_root(parent, b); + if (a == b) return; + if (a < b) { const int t = a; a = b; b = t; } + if (atomicCAS(&parent[a], a, b) == a) return; + } +} + +__global__ void cc_union(int W, int H, const char *__restrict__ member, int *parent) { + const size_t n = static_cast(W) * H; + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n || !member[i]) return; + const int x = static_cast(i % W), y = static_cast(i / W); + // The four neighbours before this pixel; the other four see it from their side. + if (x > 0 && member[i - 1]) cc_unite(parent, static_cast(i), static_cast(i - 1)); + if (y > 0) { + const size_t up = i - W; + if (member[up]) cc_unite(parent, static_cast(i), static_cast(up)); + if (x > 0 && member[up - 1]) cc_unite(parent, static_cast(i), static_cast(up - 1)); + if (x + 1 < W && member[up + 1]) cc_unite(parent, static_cast(i), static_cast(up + 1)); + } +} + +__global__ void cc_flatten(size_t n, int *parent) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n || parent[i] < 0) return; + parent[i] = find_root(parent, static_cast(i)); +} + +// Per component (by root): how many of its pixels are `a`, and how many are both `a` and `b`. +__global__ void cc_count(size_t n, const int *__restrict__ root, const char *__restrict__ a, const char *__restrict__ b, + int *__restrict__ count_a, int *__restrict__ count_ab) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n || root[i] < 0 || !a[i]) return; + atomicAdd(&count_a[root[i]], 1); + if (count_ab && b[i]) + atomicAdd(&count_ab[root[i]], 1); +} + +__global__ void region_kernel(size_t n, const int *__restrict__ root, const int *__restrict__ n_low, + const char *__restrict__ low, const char *__restrict__ valid, const int *__restrict__ radius, + int blocked_out_to, char *__restrict__ region) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + char r = root[i] >= 0 && n_low[root[i]] >= MIN_SHADOW_PIXELS ? low[i] : 0; + if (valid[i] && radius[i] <= blocked_out_to) + r = 1; + region[i] = r; +} + +__global__ void lit_kernel(size_t n, const uint32_t *__restrict__ valid_count, const int64_t *__restrict__ max_value, + char *__restrict__ lit) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + lit[i] = (valid_count[i] > 0) && (max_value[i] >= MIN_REFLECTION); +} + +__global__ void reflection_kernel(int W, int H, const char *__restrict__ lit, char *__restrict__ reflection) { + const size_t n = static_cast(W) * H; + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + const int x = static_cast(i % W), y = static_cast(i / W); + char r = 0; + if (lit[i]) { + int neighbours = 0; + for (int dy = -1; dy <= 1; dy++) + for (int dx = -1; dx <= 1; dx++) { + const int yy = y + dy, xx = x + dx; + if ((dx || dy) && yy >= 0 && yy < H && xx >= 0 && xx < W && lit[static_cast(yy) * W + xx]) + neighbours++; + } + r = neighbours >= 2; + } + reflection[i] = r; +} + +__global__ void penumbra_kernel(size_t n, const char *__restrict__ penumbra, const char *__restrict__ valid, + const float *__restrict__ ratio, const float *__restrict__ deficit, char *__restrict__ region) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + if (penumbra[i] && valid[i] && ratio[i] < PENUMBRA_RATIO && deficit[i] > MIN_DEFICIT_SIGMA) + region[i] = 1; +} + +__global__ void invert_kernel(size_t n, const char *__restrict__ in, char *__restrict__ out) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + out[i] = !in[i]; +} + +// Background components that touch the frame's edge are outside; the rest are holes. +__global__ void outside_kernel(int W, int H, const int *__restrict__ root, int *__restrict__ outside) { + const int k = blockIdx.x * blockDim.x + threadIdx.x; + const int perimeter = 2 * W + 2 * H; + if (k >= perimeter) return; + int x, y; + if (k < W) { x = k; y = 0; } + else if (k < 2 * W) { x = k - W; y = H - 1; } + else if (k < 2 * W + H) { x = 0; y = k - 2 * W; } + else { x = W - 1; y = k - 2 * W - H; } + const int r = root[static_cast(y) * W + x]; + if (r >= 0) outside[r] = 1; +} + +__global__ void final_kernel(size_t n, const int *__restrict__ background_root, const int *__restrict__ outside, + const char *__restrict__ reflection_grown, char *__restrict__ region, + uint32_t *__restrict__ mask) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + char r = region[i]; + if (background_root[i] >= 0 && !outside[background_root[i]]) + r = 1; // a hole + if (reflection_grown[i]) + r = 0; + region[i] = r; + mask[i] = r ? MASK_SHADOW : 0; +} + +__global__ void sector_key_kernel(ShadowMaskSetup s, const char *__restrict__ valid, const char *__restrict__ region, + const int *__restrict__ radius, const float *__restrict__ ratio, + uint64_t *__restrict__ key) { + const size_t n = static_cast(s.width) * s.height; + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + if (!valid[i] || region[i]) { + key[i] = UINT64_MAX; + return; + } + const float dx = static_cast(i % s.width) - s.beam_x, dy = static_cast(i / s.width) - s.beam_y; + const double phi = atan2f(dy, dx) + PI; + const int sector = min(HARMONIC_SECTORS - 1, static_cast(phi / (2.0 * PI) * HARMONIC_SECTORS)); + const uint64_t k = static_cast(radius[i] / HARMONIC_BAND_PX) * HARMONIC_SECTORS + sector; + key[i] = (k << 32) | float_key(ratio[i]); +} + +__global__ void sector_median_kernel(int sectors, const uint64_t *__restrict__ keys, const int *__restrict__ offset, + double *__restrict__ median) { + const int k = blockIdx.x * blockDim.x + threadIdx.x; + if (k >= sectors) return; + const int n = offset[k + 1] - offset[k]; + median[k] = n >= MIN_SECTOR_PIXELS ? key_float(static_cast(keys[offset[k] + n / 2])) : -1.0; +} + +__global__ void dim_kernel(ShadowMaskSetup s, uint32_t frames, const char *__restrict__ valid, + const char *__restrict__ region, const float *__restrict__ ratio, + const float *__restrict__ deficit, const int *__restrict__ radius, + const float *__restrict__ harm_c, const float *__restrict__ harm_s, + const int32_t *__restrict__ pooled_count, const float *__restrict__ pol, + const float *__restrict__ pooled, const float *__restrict__ baseline, + char *__restrict__ dim) { + const size_t n = static_cast(s.width) * s.height; + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + char d = 0; + if (valid[i] && !region[i] && ratio[i] < PENUMBRA_RATIO) { + // cos 2phi and sin 2phi from the offset to the beam. + const float dx = static_cast(i % s.width) - s.beam_x, dy = static_cast(i / s.width) - s.beam_y; + const float r2 = fmaxf(dx * dx + dy * dy, 1e-6f); + const int band = radius[i] / HARMONIC_BAND_PX; + const float model = fminf(1.0f, 1.0f + harm_c[band] * (dx * dx - dy * dy) / r2 + + harm_s[band] * 2.0f * dx * dy / r2); + if (model >= 1.0f) { + d = deficit[i] > MIN_DEFICIT_SIGMA; + } else { + const double counted = static_cast(frames) * pooled_count[i] * pol[i]; + d = ratio[i] < PENUMBRA_RATIO * model + && poisson_deficit_sigma(pooled[i] * counted, baseline[radius[i]] * model * counted) > MIN_DEFICIT_SIGMA; + } + } + dim[i] = d; +} + +// Join a region across the module gaps it crosses, one line per thread (bridge_gaps in +// ShadowFinder.cpp). Both directions read `region` and only ever set pixels of `out` to 1. +__global__ void bridge_rows(int W, int H, const char *__restrict__ region, const char *__restrict__ valid, + char *out) { + const int y = blockIdx.x * blockDim.x + threadIdx.x; + if (y >= H) return; + const size_t row = static_cast(y) * W; + int k = 0; + while (k < W) { + if (valid[row + k]) { k++; continue; } + const int start = k; + while (k < W && !valid[row + k]) k++; + if (start > 0 && k < W && region[row + start - 1] && region[row + k]) + for (int j = start; j < k; j++) out[row + j] = 1; + } +} + +__global__ void bridge_columns(int W, int H, const char *__restrict__ region, const char *__restrict__ valid, + char *out) { + const int x = blockIdx.x * blockDim.x + threadIdx.x; + if (x >= W) return; + const auto at = [&](int y) { return static_cast(y) * W + x; }; + int k = 0; + while (k < H) { + if (valid[at(k)]) { k++; continue; } + const int start = k; + while (k < H && !valid[at(k)]) k++; + if (start > 0 && k < H && region[at(start - 1)] && region[at(k)]) + for (int j = start; j < k; j++) out[at(j)] = 1; + } +} + +__global__ void transmitting_kernel(size_t n, const int *__restrict__ root, const char *__restrict__ dim, + const int *__restrict__ n_dim, const int *__restrict__ n_low, + uint32_t *__restrict__ mask) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + const int c = root[i]; + if (c >= 0 && dim[i] && n_dim[c] >= MIN_SHADOW_PIXELS && n_low[c] >= MIN_CORE_PIXELS) + mask[i] = MASK_TRANSMITTING; +} + +__global__ void mean_kernel(size_t n, const uint32_t *__restrict__ pixel_mask, const int64_t *__restrict__ sum_value, + const uint32_t *__restrict__ valid_count, float *__restrict__ mean) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= n) return; + mean[i] = valid_count[i] > 0 && pixel_mask[i] == 0 + ? static_cast(static_cast(sum_value[i]) / valid_count[i]) : NAN; +} + +// The device side of one GetMask: planes, sort scratch and the stream to run on. +class MaskEngine { +public: + const ShadowMaskSetup s; + const int W, H; + const size_t n; + cudaStream_t stream; + + MaskEngine(const ShadowMaskSetup &setup, cudaStream_t st) + : s(setup), W(setup.width), H(setup.height), n(static_cast(setup.width) * setup.height), stream(st) {} + + void Check(const char *what) const { + check(cudaGetLastError(), what); + } + + void Dilate(const char *in, char *out, char *scratch, int r) const { + dilate_rows<<>>(in, scratch, W, H, r); + dilate_columns<<>>(scratch, out, W, H, r); + Check("dilate"); + } + + // Components of `member`, each pixel's root in `root` (-1 outside every component). + void Label(const char *member, int *root) const { + cc_init<<>>(n, member, root); + cc_union<<>>(W, H, member, root); + cc_flatten<<>>(n, root); + Check("components"); + } + + void SortKeys(uint64_t *keys, uint64_t *sorted) const { + size_t bytes = 0; + check(cub::DeviceRadixSort::SortKeys(nullptr, bytes, keys, sorted, n, 0, 64, stream), "sort size"); + CudaDevicePtr scratch(bytes); + check(cub::DeviceRadixSort::SortKeys(scratch.get(), bytes, keys, sorted, n, 0, 64, stream), "sort"); + // The scratch is freed on the allocation stream, which knows nothing of this one. + check(cudaStreamSynchronize(stream), "sort"); + } + + template + std::vector Download(const T *device, size_t count) const { + std::vector host(count); + check(cudaMemcpyAsync(host.data(), device, count * sizeof(T), cudaMemcpyDeviceToHost, stream), "download"); + check(cudaStreamSynchronize(stream), "download"); + return host; + } + + template + void Upload(T *device, const std::vector &host) const { + check(cudaMemcpyAsync(device, host.data(), host.size() * sizeof(T), cudaMemcpyHostToDevice, stream), "upload"); + } +}; + +} // namespace + +std::vector ShadowMaskOnDevice(const ShadowMaskSetup &setup, const std::vector &pixel_mask, + const int64_t *max_value, const int64_t *sum_value, + const uint32_t *valid_count, uint32_t frames, cudaStream_t stream) { + const MaskEngine e(setup, stream); + const size_t n = e.n; + const int W = e.W, H = e.H; + + CudaDevicePtr d_pixel_mask(n), mask(n); + e.Upload(d_pixel_mask.get(), pixel_mask); + + // Mean projection over the polarization factor, usable pixels and radius from the beam centre. + CudaDevicePtr pol(n), pooled(n), ratio(n), deficit(n); + CudaDevicePtr valid(n), low(n), region(n), a(n), b(n), c(n); + CudaDevicePtr radius(n), root(n), count_a(n), count_b(n), max_radius(1); + CudaDevicePtr num(n), dsum(n); + CudaDevicePtr den(n), dcount(n); + check(cudaMemsetAsync(max_radius.get(), 0, sizeof(int), stream), "memset"); + setup_kernel<<>>(setup, d_pixel_mask, sum_value, valid_count, pol, valid, radius, + num, den, max_radius); + e.Check("setup"); + const int max_r = e.Download(max_radius.get(), 1)[0]; + const int rings = max_r + 1; + + // The background pooled over a small box. + box_rows<<>>(num, dsum, W, H, POOL_PX / 2); + box_columns<<>>(dsum, num, W, H, POOL_PX / 2); + box_rows<<>>(den, dcount, W, H, POOL_PX / 2); + box_columns<<>>(dcount, den, W, H, POOL_PX / 2); + e.Check("pooling"); + const double *pooled_sum = num; + const int32_t *pooled_count = den; + + // The rings, each sorted once; the baseline is an order statistic of them. + CudaDevicePtr keys(n), sorted(n); + pooled_kernel<<>>(n, pooled_sum, pooled_count, valid, radius, pooled, keys); + e.Check("pooled"); + e.SortKeys(keys, sorted); + CudaDevicePtr ring_offset(rings + 1); + group_offsets<<>>(sorted, n, rings, ring_offset); + CudaDevicePtr baseline(rings); + baseline_kernel<<>>(rings, sorted, ring_offset, baseline); + e.Check("baseline"); + const auto host_baseline = e.Download(baseline.get(), rings); + const auto offsets = e.Download(ring_offset.get(), rings + 1); + std::vector ring_pixels(rings); + for (int r = 0; r < rings; r++) + ring_pixels[r] = offsets[r + 1] - offsets[r]; + const int blocked_out_to = BlockedOutTo(host_baseline, ring_pixels); + + // Low pixels, and the regions of them large enough to be hardware. + low_kernel<<>>(n, frames, valid, pooled, pooled_count, pol, radius, baseline, + ratio, deficit, low); + e.Check("low"); + e.Dilate(low, a, c, BRIDGE_PX); + e.Label(a, root); + check(cudaMemsetAsync(count_a.get(), 0, n * sizeof(int), stream), "memset"); + cc_count<<>>(n, root, low, low, count_a, nullptr); + region_kernel<<>>(n, root, count_a, low, valid, radius, blocked_out_to, region); + e.Check("region"); + + // Recorded reflections; `b` holds them until they are given back at the end. + lit_kernel<<>>(n, valid_count, max_value, a); + reflection_kernel<<>>(W, H, a, b); + e.Check("reflections"); + + // Penumbra, round and fill. + e.Dilate(region, a, c, PENUMBRA_MAX_PX); + penumbra_kernel<<>>(n, a, valid, ratio, deficit, region); + e.Dilate(region, a, c, 2); + invert_kernel<<>>(n, a, region); + e.Dilate(region, a, c, 2); + invert_kernel<<>>(n, a, region); // region = erode(dilate(region)) + invert_kernel<<>>(n, region, a); // the background + e.Label(a, root); + check(cudaMemsetAsync(count_a.get(), 0, n * sizeof(int), stream), "memset"); + outside_kernel<<>>(W, H, root, count_a); + e.Dilate(b, c, a, 1); // the reflections, grown by one + final_kernel<<>>(n, root, count_a, c, region, mask); + e.Check("fill"); + + // The arm search: sector medians of the ratio, the harmonic of each band, the dim pixels. + const int n_bands = max_r / HARMONIC_BAND_PX + 1; + const int n_sectors = n_bands * HARMONIC_SECTORS; + sector_key_kernel<<>>(setup, valid, region, radius, ratio, keys); + e.Check("sectors"); + e.SortKeys(keys, sorted); + CudaDevicePtr sector_offset(n_sectors + 1); + group_offsets<<>>(sorted, n, n_sectors, sector_offset); + CudaDevicePtr sector_median(n_sectors); + sector_median_kernel<<>>(n_sectors, sorted, sector_offset, sector_median); + e.Check("sector medians"); + std::vector harm_c, harm_s; + HarmonicFit(e.Download(sector_median.get(), n_sectors), n_bands, harm_c, harm_s); + CudaDevicePtr d_harm_c(n_bands), d_harm_s(n_bands); + e.Upload(d_harm_c.get(), harm_c); + e.Upload(d_harm_s.get(), harm_s); + dim_kernel<<>>(setup, frames, valid, region, ratio, deficit, radius, d_harm_c, d_harm_s, + pooled_count, pol, pooled, baseline, a); + e.Check("dim"); + e.Dilate(a, b, c, BRIDGE_PX); + check(cudaMemcpyAsync(c.get(), b.get(), n, cudaMemcpyDeviceToDevice, stream), "copy"); + bridge_rows<<>>(W, H, b, valid, c); + bridge_columns<<>>(W, H, b, valid, c); + e.Label(c, root); + check(cudaMemsetAsync(count_a.get(), 0, n * sizeof(int), stream), "memset"); + check(cudaMemsetAsync(count_b.get(), 0, n * sizeof(int), stream), "memset"); + cc_count<<>>(n, root, a, low, count_a, count_b); + transmitting_kernel<<>>(n, root, a, count_a, count_b, mask); + e.Check("transmitting"); + + return e.Download(mask.get(), n); +} + +std::vector MeanProjectionOnDevice(const std::vector &pixel_mask, const int64_t *sum_value, + const uint32_t *valid_count, size_t npixels, cudaStream_t stream) { + CudaDevicePtr d_pixel_mask(npixels); + CudaDevicePtr mean(npixels); + check(cudaMemcpyAsync(d_pixel_mask.get(), pixel_mask.data(), npixels * sizeof(uint32_t), cudaMemcpyHostToDevice, + stream), "upload"); + mean_kernel<<>>(npixels, d_pixel_mask, sum_value, valid_count, mean); + check(cudaGetLastError(), "mean"); + std::vector host(npixels); + check(cudaMemcpyAsync(host.data(), mean.get(), npixels * sizeof(float), cudaMemcpyDeviceToHost, stream), "download"); + check(cudaStreamSynchronize(stream), "mean"); + return host; +} diff --git a/image_analysis/beam_stop/ShadowMaskGPU.h b/image_analysis/beam_stop/ShadowMaskGPU.h new file mode 100644 index 000000000..baa25a00f --- /dev/null +++ b/image_analysis/beam_stop/ShadowMaskGPU.h @@ -0,0 +1,39 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +// Included only under JFJOCH_USE_CUDA. + +#include +#include + +#include + +// What the mask is drawn about: the detector, the centre the rings are drawn about, and the geometry +// the polarization factor is read off (ShadowFinder keeps it as a DiffractionGeometry; the device +// takes it as numbers). +struct ShadowMaskSetup { + int width = 0, height = 0; + float beam_x = 0.0f, beam_y = 0.0f; + float det_matrix[9] = {}; // row major + float pixel_size_mm = 0.0f; + float distance_mm = 0.0f; + bool has_polarization = false; + float polarization = 0.0f; +}; + +// ShadowFinder::GetMask on the device, from the projection ShadowAccumulatorGPU holds there. Step for +// step the host's algorithm - the same pooling, ring medians, components, morphology and arm search - +// and the same answer wherever the arithmetic is exact: every integer, comparison, sort and component +// is. What is not is the floating point the two compilers evaluate differently - the polarization +// factor's trigonometry, the Poisson test's logarithm and the azimuth of the arm search - so a pixel +// within a rounding of one of those thresholds can come out the other way. +std::vector ShadowMaskOnDevice(const ShadowMaskSetup &setup, const std::vector &pixel_mask, + const int64_t *max_value, const int64_t *sum_value, + const uint32_t *valid_count, uint32_t frames, cudaStream_t stream); + +// The mean projection the host's GetMeanProjection makes, computed where the sums are: the same +// division, so the same bits, and a quarter of the bytes to bring back. +std::vector MeanProjectionOnDevice(const std::vector &pixel_mask, const int64_t *sum_value, + const uint32_t *valid_count, size_t npixels, cudaStream_t stream); diff --git a/image_analysis/geom_refinement/BackgroundBand.h b/image_analysis/geom_refinement/BackgroundBand.h new file mode 100644 index 000000000..394ab3e84 --- /dev/null +++ b/image_analysis/geom_refinement/BackgroundBand.h @@ -0,0 +1,107 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +// Where one pixel falls in the background beam-centre fit (FindBeamCenterFromBackground): which +// radial-bin x sector cell of the fitted band it lands in about a trial centre, and the derivative +// of its 2theta with respect to that centre. Written once and compiled both for the host fit and for +// its device twin (BeamCenterBackgroundGPU), so the two read the same formula. +// +// The same formula is the same number only when both sides evaluate it the same way. The library +// atan2f differs between glibc and CUDA in the last bit, and the compilers fuse multiply-adds each +// in their own way; either moves a pixel within a rounding of a cell edge into the neighbouring cell +// (measured on 16 Mpx sweeps: ~30 of 6.5 million pixels, enough to move the fitted centre by up to +// 0.05 px). So the angles come from BackgroundAtan2 below and both translation units are compiled +// without contraction (geom_refinement/CMakeLists.txt), and the host and the device then agree to +// the bit. + +#include +#include + +#include "../../common/JFJochMath.h" + +#ifdef __CUDACC__ +#define BACKGROUND_BAND_HD __host__ __device__ inline +#else +#define BACKGROUND_BAND_HD inline +#endif + +struct BackgroundBand { + static constexpr int SECTORS = 36; + static constexpr int RADIAL_BINS = 120; + static constexpr int CELLS = RADIAL_BINS * SECTORS; + + float rot[9]; // detector matrix, row major + float pixel_size; // mm + float distance; // mm + float tt_lo, tt_hi; // the band in 2theta + float d_tt; // width of one radial bin + double tan_lo, tan_hi; // the band in tan(2theta), widened for the quick rejection +}; + +// atan2(y, x) from IEEE operations alone - add, multiply, divide, square root, all correctly rounded on +// the host and on the device - so that, compiled without contraction (see CMakeLists.txt), the host +// and the device return the same bits. The library atan2f does not: glibc's and CUDA's differ in the +// last place. Two half-angle reductions take the argument below tan(pi/16), where eleven terms of the +// series leave an error under 1e-17. +BACKGROUND_BAND_HD double BackgroundAtan2(double y, double x) { + const double ax = x < 0 ? -x : x, ay = y < 0 ? -y : y; + if (ax == 0.0 && ay == 0.0) + return 0.0; + const bool swap = ay > ax; + double t = swap ? ax / ay : ay / ax; + t = t / (1.0 + sqrt(1.0 + t * t)); + t = t / (1.0 + sqrt(1.0 + t * t)); + const double t2 = t * t; + double series = 1.0 / 23.0; + for (int k = 21; k >= 1; k -= 2) + series = 1.0 / k - t2 * series; + double a = 4.0 * t * series; + if (swap) a = PI / 2 - a; + if (x < 0) a = PI - a; + return y < 0 ? -a : a; +} + +// The cell of pixel (x, y) about (beam_x, beam_y), or -1 when it is outside the band; for a pixel in +// it, also the two components of the derivative of its 2theta with respect to the centre. +BACKGROUND_BAND_HD int BackgroundBandCell(const BackgroundBand &b, int x, int y, float beam_x, float beam_y, + float &jac_x, float &jac_y) { + const float *rot = b.rot; + const float u = (x - beam_x) * b.pixel_size; + const float v = (y - beam_y) * b.pixel_size; + const float lx = rot[0] * u + rot[1] * v + rot[2] * b.distance; + const float ly = rot[3] * u + rot[4] * v + rot[5] * b.distance; + const float lz = rot[6] * u + rot[7] * v + rot[8] * b.distance; + const float rho_sq = lx * lx + ly * ly; + // Most pixels lie outside the band. Those clearly outside it in tan(2theta) = rho / lz - by a + // margin far above float rounding - skip the square root and both atan2 below; every pixel the + // exact test would keep still reaches it. + if (lz > 0.0f) { + const double lz_sq = static_cast(lz) * lz; + if (rho_sq < b.tan_lo * b.tan_lo * lz_sq || rho_sq > b.tan_hi * b.tan_hi * lz_sq) + return -1; + } + const float rho = sqrtf(rho_sq); + const float two_theta = static_cast(BackgroundAtan2(rho, lz)); + if (two_theta < b.tt_lo || two_theta >= b.tt_hi || rho == 0.0f) + return -1; + + const float phi = static_cast(BackgroundAtan2(ly, lx)); + // Both bins are clamped: a pixel one float ulp below the top of the band divides to exactly + // RADIAL_BINS, which is one cell past the end of every accumulator. + int r_bin = static_cast((two_theta - b.tt_lo) / b.d_tt); + r_bin = r_bin < 0 ? 0 : (r_bin > BackgroundBand::RADIAL_BINS - 1 ? BackgroundBand::RADIAL_BINS - 1 : r_bin); + int s_bin = static_cast((phi + PI) / (2 * PI) * BackgroundBand::SECTORS); + s_bin = s_bin < 0 ? 0 : (s_bin > BackgroundBand::SECTORS - 1 ? BackgroundBand::SECTORS - 1 : s_bin); + + // d(2theta)/d(beam), through the lab coordinate: the detector coordinate depends on the centre + // only as (x - beam_x), so moving the centre is moving the pixel. + const float denominator = rho * rho + lz * lz; + const float g_x = lz * lx / (rho * denominator); + const float g_y = lz * ly / (rho * denominator); + const float g_z = -rho / denominator; + jac_x = -b.pixel_size * (g_x * rot[0] + g_y * rot[3] + g_z * rot[6]); + jac_y = -b.pixel_size * (g_x * rot[1] + g_y * rot[4] + g_z * rot[7]); + return r_bin * BackgroundBand::SECTORS + s_bin; +} diff --git a/image_analysis/geom_refinement/BeamCenterBackgroundGPU.cu b/image_analysis/geom_refinement/BeamCenterBackgroundGPU.cu new file mode 100644 index 000000000..550593f1d --- /dev/null +++ b/image_analysis/geom_refinement/BeamCenterBackgroundGPU.cu @@ -0,0 +1,210 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#include "BeamCenterBackgroundGPU.h" + +#include + +#include "../indexing/CUDAMemHelpers.h" + +namespace { + +constexpr int THREADS = 256; + +void check(cudaError_t err, const char *what) { + if (err != cudaSuccess) + throw JFJochException(JFJochExceptionCategory::GPUCUDAError, + std::string("Beam centre from background: ") + what + ": " + cudaGetErrorString(err)); +} + +// Every pixel's cell (BackgroundBand::CELLS where it is outside the band or unusable), its index, +// and its derivatives. +__global__ void bin_kernel(BackgroundBand band, int width, size_t npixels, float beam_x, float beam_y, + const char *__restrict__ usable, int32_t *__restrict__ key, + int32_t *__restrict__ index, float *__restrict__ jac_x, + float *__restrict__ jac_y) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= npixels) + return; + int cell = BackgroundBand::CELLS; + float jx = 0.0f, jy = 0.0f; + if (usable[i]) { + const int c = BackgroundBandCell(band, static_cast(i % width), static_cast(i / width), + beam_x, beam_y, jx, jy); + if (c >= 0) + cell = c; + } + key[i] = cell; + index[i] = static_cast(i); + jac_x[i] = jx; + jac_y[i] = jy; +} + +// Where each cell's pixels start in the sorted keys; offset[CELLS] is the number of binned pixels. +__global__ void offset_kernel(const int32_t *__restrict__ sorted_key, size_t n, int32_t *__restrict__ offset) { + const int c = blockIdx.x * blockDim.x + threadIdx.x; + if (c > BackgroundBand::CELLS) + return; + size_t lo = 0, hi = n; + while (lo < hi) { + const size_t mid = (lo + hi) / 2; + if (sorted_key[mid] < c) lo = mid + 1; + else hi = mid; + } + offset[c] = static_cast(lo); +} + +// One thread per cell, walking its pixels in pixel order: a partial sum per host row block, added to +// the cell's total when the block changes - the host's arithmetic, step for step. A cell whose pixels +// are clipped (clip_limit != nullptr) skips the ones above its limit, as the host's clip rounds do. +__global__ void sum_kernel(int width, const int32_t *__restrict__ offset, const int32_t *__restrict__ index, + const int32_t *__restrict__ row_block, const float *__restrict__ mean, + const float *__restrict__ jac_x, const float *__restrict__ jac_y, + const float *__restrict__ clip_limit, + double *__restrict__ sum, double *__restrict__ sum_sq, + double *__restrict__ sum_jx, double *__restrict__ sum_jy, + int32_t *__restrict__ count) { + const int c = blockIdx.x * blockDim.x + threadIdx.x; + if (c >= BackgroundBand::CELLS) + return; + const bool clipped = clip_limit != nullptr; + const float limit = clipped ? clip_limit[c] : 0.0f; + + double s = 0, ss = 0, jx = 0, jy = 0; + double bs = 0, bss = 0, bjx = 0, bjy = 0; + int32_t n = 0; + int block = -1; + for (int32_t p = offset[c]; p < offset[c + 1]; p++) { + const int32_t i = index[p]; + const float value = mean[i]; + if (clipped && (limit < 0.0f || value > limit)) + continue; + const int b = row_block[i / width]; + if (b != block) { + s += bs; ss += bss; jx += bjx; jy += bjy; + bs = bss = bjx = bjy = 0; + block = b; + } + n++; + bs += value; + bss += static_cast(value) * value; + if (!clipped) { + bjx += jac_x[i]; + bjy += jac_y[i]; + } + } + s += bs; ss += bss; jx += bjx; jy += bjy; + sum[c] = s; + sum_sq[c] = ss; + count[c] = n; + if (!clipped) { + sum_jx[c] = jx; + sum_jy[c] = jy; + } +} + +} // namespace + +struct BeamCenterBackgroundGPU::Impl { + int width, height; + size_t npixels; + CudaStream stream; + CudaDevicePtr usable; + CudaDevicePtr mean; + CudaDevicePtr row_block; + CudaDevicePtr key, sorted_key, index, sorted_index; + CudaDevicePtr jac_x, jac_y; + CudaDevicePtr offset; + CudaDevicePtr clip_limit; + CudaDevicePtr sum, sum_sq, sum_jx, sum_jy; + CudaDevicePtr count; + CudaDevicePtr sort_scratch; + size_t sort_scratch_bytes = 0; + + Impl(int w, int h) + : width(w), height(h), npixels(static_cast(w) * h), + usable(npixels), mean(npixels), row_block(h), + key(npixels), sorted_key(npixels), index(npixels), sorted_index(npixels), + jac_x(npixels), jac_y(npixels), offset(BackgroundBand::CELLS + 1), + clip_limit(BackgroundBand::CELLS), + sum(BackgroundBand::CELLS), sum_sq(BackgroundBand::CELLS), + sum_jx(BackgroundBand::CELLS), sum_jy(BackgroundBand::CELLS), + count(BackgroundBand::CELLS) { + // Keys run to CELLS inclusive, the bin of everything outside the band. + check(cub::DeviceRadixSort::SortPairs(nullptr, sort_scratch_bytes, key.get(), sorted_key.get(), + index.get(), sorted_index.get(), npixels, 0, end_bit(), stream), + "sort size"); + sort_scratch = CudaDevicePtr(sort_scratch_bytes); + } + + static int end_bit() { + int bits = 0; + while ((1 << bits) <= BackgroundBand::CELLS) bits++; + return bits; + } + + void Download(std::vector &s, std::vector &ss, std::vector *jx, + std::vector *jy, std::vector &n) { + const size_t cells = BackgroundBand::CELLS; + check(cudaMemcpyAsync(s.data(), sum.get(), cells * sizeof(double), cudaMemcpyDeviceToHost, stream), "copy"); + check(cudaMemcpyAsync(ss.data(), sum_sq.get(), cells * sizeof(double), cudaMemcpyDeviceToHost, stream), "copy"); + if (jx) check(cudaMemcpyAsync(jx->data(), sum_jx.get(), cells * sizeof(double), cudaMemcpyDeviceToHost, stream), "copy"); + if (jy) check(cudaMemcpyAsync(jy->data(), sum_jy.get(), cells * sizeof(double), cudaMemcpyDeviceToHost, stream), "copy"); + check(cudaMemcpyAsync(n.data(), count.get(), cells * sizeof(int32_t), cudaMemcpyDeviceToHost, stream), "copy"); + check(cudaStreamSynchronize(stream), "sums"); + } +}; + +BeamCenterBackgroundGPU::BeamCenterBackgroundGPU(int width, int height, const std::vector &block_row, + const char *usable, const float *mean) + : impl(std::make_unique(width, height)) { + std::vector row_block(height); + for (size_t b = 0; b + 1 < block_row.size(); b++) + for (int y = block_row[b]; y < block_row[b + 1]; y++) + row_block[y] = static_cast(b); + check(cudaMemcpyAsync(impl->row_block.get(), row_block.data(), height * sizeof(int32_t), + cudaMemcpyHostToDevice, impl->stream), "upload"); + check(cudaMemcpyAsync(impl->usable.get(), usable, impl->npixels, cudaMemcpyHostToDevice, impl->stream), "upload"); + check(cudaMemcpyAsync(impl->mean.get(), mean, impl->npixels * sizeof(float), cudaMemcpyHostToDevice, + impl->stream), "upload"); + check(cudaStreamSynchronize(impl->stream), "upload"); +} + +BeamCenterBackgroundGPU::~BeamCenterBackgroundGPU() = default; + +void BeamCenterBackgroundGPU::Bin(const BackgroundBand &band, float beam_x, float beam_y, + std::vector &sum, std::vector &sum_sq, + std::vector &sum_jx, std::vector &sum_jy, + std::vector &count) { + Impl &d = *impl; + const auto blocks = static_cast((d.npixels + THREADS - 1) / THREADS); + bin_kernel<<>>(band, d.width, d.npixels, beam_x, beam_y, d.usable, + d.key, d.index, d.jac_x, d.jac_y); + check(cudaGetLastError(), "bin"); + // A radix sort is stable, so each cell's pixels come out in pixel order. + check(cub::DeviceRadixSort::SortPairs(d.sort_scratch.get(), d.sort_scratch_bytes, d.key.get(), + d.sorted_key.get(), d.index.get(), d.sorted_index.get(), + d.npixels, 0, Impl::end_bit(), d.stream), "sort"); + constexpr int cell_blocks = (BackgroundBand::CELLS + 1 + THREADS - 1) / THREADS; + offset_kernel<<>>(d.sorted_key, d.npixels, d.offset); + check(cudaGetLastError(), "offsets"); + sum_kernel<<>>(d.width, d.offset, d.sorted_index, d.row_block, d.mean, + d.jac_x, d.jac_y, nullptr, d.sum, d.sum_sq, + d.sum_jx, d.sum_jy, d.count); + check(cudaGetLastError(), "sums"); + d.Download(sum, sum_sq, &sum_jx, &sum_jy, count); +} + +void BeamCenterBackgroundGPU::Clip(const std::vector &clip_limit, + std::vector &sum, std::vector &sum_sq, + std::vector &count) { + Impl &d = *impl; + check(cudaMemcpyAsync(d.clip_limit.get(), clip_limit.data(), BackgroundBand::CELLS * sizeof(float), + cudaMemcpyHostToDevice, d.stream), "upload"); + constexpr int cell_blocks = (BackgroundBand::CELLS + THREADS - 1) / THREADS; + sum_kernel<<>>(d.width, d.offset, d.sorted_index, d.row_block, d.mean, + d.jac_x, d.jac_y, d.clip_limit, d.sum, d.sum_sq, + d.sum_jx, d.sum_jy, d.count); + check(cudaGetLastError(), "clip"); + d.Download(sum, sum_sq, nullptr, nullptr, count); +} diff --git a/image_analysis/geom_refinement/BeamCenterBackgroundGPU.h b/image_analysis/geom_refinement/BeamCenterBackgroundGPU.h new file mode 100644 index 000000000..72d47b266 --- /dev/null +++ b/image_analysis/geom_refinement/BeamCenterBackgroundGPU.h @@ -0,0 +1,40 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +// Included only under JFJOCH_USE_CUDA. Free of CUDA headers, so the host fit can hold one. + +#include +#include +#include + +#include "BackgroundBand.h" + +// The two passes over the pixels of the background beam-centre fit (FindBeamCenterFromBackground), +// on the device: binning every usable pixel into its cell about a trial centre, and summing the +// binned pixels again under a clip. They are all of the fit's cost; the fit itself stays on the host. +// +// Each cell is summed in the order the host sums it - pixel order within each of the host's row +// blocks, the blocks then added in block order - so a pixel that lands in the same cell on both +// sides adds the same rounding on both. Whatever differs comes from BackgroundBandCell (see there). +class BeamCenterBackgroundGPU { + struct Impl; + std::unique_ptr impl; +public: + // block_row: the first row of each of the host's row blocks, and one past the last row at the end. + BeamCenterBackgroundGPU(int width, int height, const std::vector &block_row, + const char *usable, const float *mean); + ~BeamCenterBackgroundGPU(); + + // Bin the band about (beam_x, beam_y) and sum each cell: the values, their squares, the two + // derivatives, and the count. + void Bin(const BackgroundBand &band, float beam_x, float beam_y, + std::vector &sum, std::vector &sum_sq, + std::vector &sum_jx, std::vector &sum_jy, std::vector &count); + + // Sum the pixels the last Bin put in each cell again, leaving out those above the cell's + // clip_limit and every pixel of a cell whose limit is negative. + void Clip(const std::vector &clip_limit, + std::vector &sum, std::vector &sum_sq, std::vector &count); +}; diff --git a/image_analysis/geom_refinement/BeamCenterFromBackground.cpp b/image_analysis/geom_refinement/BeamCenterFromBackground.cpp index 19f468a65..0b99a030c 100644 --- a/image_analysis/geom_refinement/BeamCenterFromBackground.cpp +++ b/image_analysis/geom_refinement/BeamCenterFromBackground.cpp @@ -2,6 +2,7 @@ // SPDX-License-Identifier: GPL-3.0-only #include "BeamCenterFromBackground.h" +#include "BackgroundBand.h" #include #include @@ -10,6 +11,10 @@ #include "../../common/CompressedImage.h" #include "../../common/JFJochMath.h" #include "../../common/ParallelFor.h" +#ifdef JFJOCH_USE_CUDA +#include "../../common/CUDAWrapper.h" +#include "BeamCenterBackgroundGPU.h" +#endif namespace { @@ -18,8 +23,8 @@ namespace { constexpr float BAND_LOW_RES_A = 12.0f; constexpr float BAND_HIGH_RES_A = 2.2f; -constexpr int SECTORS = 36; -constexpr int RADIAL_BINS = 120; +constexpr int SECTORS = BackgroundBand::SECTORS; +constexpr int RADIAL_BINS = BackgroundBand::RADIAL_BINS; // A cell with fewer pixels than this has no usable mean. constexpr int MIN_PIXELS_PER_CELL = 20; @@ -84,7 +89,7 @@ float median_of(std::vector &v) { std::optional FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const PixelMask &mask, const std::vector &mean, size_t nthreads, - std::optional> start) { + std::optional> start, bool allow_device) { if (nthreads == 0) nthreads = std::max(1u, std::thread::hardware_concurrency()); const auto W = static_cast(experiment.GetXPixelsNumConv()); @@ -165,10 +170,31 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe const double tan_lo = std::tan(static_cast(tt_lo)) * (1.0 - 1e-3); const double tan_hi = tt_hi < PI / 2 ? std::tan(static_cast(tt_hi)) * (1.0 + 1e-3) : INFINITY; + BackgroundBand band{}; + for (int k = 0; k < 9; k++) + band.rot[k] = rot[k]; + band.pixel_size = pixel_size; + band.distance = distance; + band.tt_lo = tt_lo; + band.tt_hi = tt_hi; + band.d_tt = d_tt; + band.tan_lo = tan_lo; + band.tan_hi = tan_hi; + +#ifndef JFJOCH_USE_CUDA + (void) allow_device; +#else + // With a GPU the two passes over the pixels run there, and only the cells come back. + std::unique_ptr gpu; + if (allow_device && get_gpu_count() > 0) + gpu = std::make_unique(W, H, block_row, usable.data(), mean.data()); +#endif + float step_x = 0.0f, step_y = 0.0f, sigma_x = 0.0f, sigma_y = 0.0f; float previous_x = 0.0f, previous_y = 0.0f; int reversals = 0; - for (int iteration = 0; iteration < MAX_ITERATIONS; iteration++) { + // The binning pass about the current centre, then the clip rounds over the pixels it binned. + const auto bin_cpu = [&] { ParallelFor(BLOCKS, nthreads, [&](int b) { double *b_sum = block_sum.data() + static_cast(b) * n_cells; double *b_sum_sq = block_sum_sq.data() + static_cast(b) * n_cells; @@ -188,48 +214,54 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe const size_t i = static_cast(y) * W + x; if (!usable[i]) continue; - const float u = (x - beam_x) * pixel_size; - const float v = (y - beam_y) * pixel_size; - const float lx = rot[0] * u + rot[1] * v + rot[2] * distance; - const float ly = rot[3] * u + rot[4] * v + rot[5] * distance; - const float lz = rot[6] * u + rot[7] * v + rot[8] * distance; - const float rho_sq = lx * lx + ly * ly; - if (lz > 0.0f) { - const double lz_sq = static_cast(lz) * lz; - if (rho_sq < tan_lo * tan_lo * lz_sq || rho_sq > tan_hi * tan_hi * lz_sq) - continue; - } - const float rho = std::sqrt(rho_sq); - const float two_theta = std::atan2(rho, lz); - if (two_theta < tt_lo || two_theta >= tt_hi || rho == 0.0f) + float jac_x, jac_y; + const int cell = BackgroundBandCell(band, x, y, beam_x, beam_y, jac_x, jac_y); + if (cell < 0) continue; - - const float phi = std::atan2(ly, lx); - // Both bins are clamped: a pixel one float ulp below the top of the band divides - // to exactly RADIAL_BINS, which is one cell past the end of every accumulator. - const int r_bin = std::clamp(static_cast((two_theta - tt_lo) / d_tt), 0, RADIAL_BINS - 1); - const int s_bin = std::clamp(static_cast((phi + PI) / (2 * PI) * SECTORS), 0, SECTORS - 1); - const int cell = r_bin * SECTORS + s_bin; - - // d(2theta)/d(beam), through the lab coordinate: the detector coordinate depends - // on the centre only as (x - beam_x), so moving the centre is moving the pixel. - const float denominator = rho * rho + lz * lz; - const float g_x = lz * lx / (rho * denominator); - const float g_y = lz * ly / (rho * denominator); - const float g_z = -rho / denominator; cells[n_band] = cell; values[n_band] = mean[i]; n_band++; b_count[cell]++; b_sum[cell] += mean[i]; b_sum_sq[cell] += static_cast(mean[i]) * mean[i]; - b_jx[cell] += -pixel_size * (g_x * rot[0] + g_y * rot[3] + g_z * rot[6]); - b_jy[cell] += -pixel_size * (g_x * rot[1] + g_y * rot[4] + g_z * rot[7]); + b_jx[cell] += jac_x; + b_jy[cell] += jac_y; } } band_pixels[b] = n_band; }); fold(true); + }; + const auto clip_cpu = [&] { + ParallelFor(BLOCKS, nthreads, [&](int b) { + double *b_sum = block_sum.data() + static_cast(b) * n_cells; + double *b_sum_sq = block_sum_sq.data() + static_cast(b) * n_cells; + int32_t *b_count = block_count.data() + static_cast(b) * n_cells; + std::fill(b_sum, b_sum + n_cells, 0.0); + std::fill(b_sum_sq, b_sum_sq + n_cells, 0.0); + std::fill(b_count, b_count + n_cells, 0); + const int32_t *cells = band_cell.data() + static_cast(block_row[b]) * W; + const float *values = band_value.data() + static_cast(block_row[b]) * W; + for (size_t j = 0; j < band_pixels[b]; j++) { + const int32_t c = cells[j]; + const float value = values[j]; + if (clip_limit[c] < 0.0f || value > clip_limit[c]) + continue; + b_count[c]++; + b_sum[c] += value; + b_sum_sq[c] += static_cast(value) * value; + } + }); + fold(false); + }; + + for (int iteration = 0; iteration < MAX_ITERATIONS; iteration++) { +#ifdef JFJOCH_USE_CUDA + if (gpu) + gpu->Bin(band, beam_x, beam_y, sum, sum_sq, sum_jx, sum_jy, count); + else +#endif + bin_cpu(); count_all = count; // the Jacobian sums belong to the unclipped pixel set @@ -240,26 +272,12 @@ FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const Pixe const double variance = std::max(sum_sq[c] / count[c] - m * m, 0.0); clip_limit[c] = static_cast(m + CLIP_SIGMA * std::sqrt(variance)); } - ParallelFor(BLOCKS, nthreads, [&](int b) { - double *b_sum = block_sum.data() + static_cast(b) * n_cells; - double *b_sum_sq = block_sum_sq.data() + static_cast(b) * n_cells; - int32_t *b_count = block_count.data() + static_cast(b) * n_cells; - std::fill(b_sum, b_sum + n_cells, 0.0); - std::fill(b_sum_sq, b_sum_sq + n_cells, 0.0); - std::fill(b_count, b_count + n_cells, 0); - const int32_t *cells = band_cell.data() + static_cast(block_row[b]) * W; - const float *values = band_value.data() + static_cast(block_row[b]) * W; - for (size_t j = 0; j < band_pixels[b]; j++) { - const int32_t c = cells[j]; - const float value = values[j]; - if (clip_limit[c] < 0.0f || value > clip_limit[c]) - continue; - b_count[c]++; - b_sum[c] += value; - b_sum_sq[c] += static_cast(value) * value; - } - }); - fold(false); +#ifdef JFJOCH_USE_CUDA + if (gpu) + gpu->Clip(clip_limit, sum, sum_sq, count); + else +#endif + clip_cpu(); } // Radial profile: the median over the sectors that have a mean, on rings that are diff --git a/image_analysis/geom_refinement/BeamCenterFromBackground.h b/image_analysis/geom_refinement/BeamCenterFromBackground.h index 350cb6ddd..7f1df9efb 100644 --- a/image_analysis/geom_refinement/BeamCenterFromBackground.h +++ b/image_analysis/geom_refinement/BeamCenterFromBackground.h @@ -40,10 +40,14 @@ struct BeamCenterEstimate { // `start` is where the walk begins; the centre in the file when it is not given. The walk advances // by a bounded distance per iteration, so where it starts decides how much of its budget is spent // travelling and - on a surface with more than one basin - which fixed point it can reach at all. +// +// With a GPU the passes over the pixels run on it (BeamCenterBackgroundGPU); allow_device = false +// keeps them on the host, which is what the parity test compares against. std::optional FindBeamCenterFromBackground(const DiffractionExperiment &experiment, const PixelMask &mask, const std::vector &mean, size_t nthreads = 0, - std::optional> start = {}); + std::optional> start = {}, + bool allow_device = true); // The precision of a centre that is the FFT capture alone, with no walk behind it. The capture is // a half-pixel grid position read off a surface, measured over 75 rotation datasets at a median diff --git a/image_analysis/geom_refinement/CMakeLists.txt b/image_analysis/geom_refinement/CMakeLists.txt index b004e8d4e..f431ddbb8 100644 --- a/image_analysis/geom_refinement/CMakeLists.txt +++ b/image_analysis/geom_refinement/CMakeLists.txt @@ -24,6 +24,7 @@ ADD_LIBRARY(JFJochGeomRefinement STATIC XtalOptimizer.cpp XtalOptimizer.h XtalResidual.h + BackgroundBand.h PostRefine.cpp PostRefine.h GeometryRefiner.cpp @@ -37,7 +38,8 @@ ADD_LIBRARY(JFJochGeomRefinement STATIC TARGET_LINK_LIBRARIES(JFJochGeomRefinement Ceres::ceres Eigen3::Eigen JFJochCommon fftw3f) IF (JFJOCH_CUDA_AVAILABLE) - TARGET_SOURCES(JFJochGeomRefinement PRIVATE BeamCenterFFTGPU.cu BeamCenterFFTGPU.h) + TARGET_SOURCES(JFJochGeomRefinement PRIVATE BeamCenterFFTGPU.cu BeamCenterFFTGPU.h + BeamCenterBackgroundGPU.cu BeamCenterBackgroundGPU.h) # Same static/dynamic cuFFT choice as the FFT indexer, and for the same reasons - see the long # note in image_analysis/indexing/CMakeLists.txt. IF (JFJOCH_PORTABLE_ONLY AND TARGET CUDA::cufft_static) @@ -46,3 +48,13 @@ IF (JFJOCH_CUDA_AVAILABLE) TARGET_LINK_LIBRARIES(JFJochGeomRefinement CUDA::cufft) ENDIF() ENDIF() + +# The background beam-centre fit bins every pixel with BackgroundBand.h on the host and on the device +# and must get the same bits on both (see there). That needs no multiply-add contracted on either side: +# GCC and Clang contract by default and nvcc does too, while MSVC does not without /fp:contract. +IF (JFJOCH_CUDA_AVAILABLE) + SET_SOURCE_FILES_PROPERTIES(BeamCenterBackgroundGPU.cu PROPERTIES COMPILE_OPTIONS "--fmad=false") +ENDIF() +IF (CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang") + SET_SOURCE_FILES_PROPERTIES(BeamCenterFromBackground.cpp PROPERTIES COMPILE_OPTIONS "-ffp-contract=off") +ENDIF() diff --git a/tests/BeamCenterFromBackgroundTest.cpp b/tests/BeamCenterFromBackgroundTest.cpp index 73a547e92..d38db14e2 100644 --- a/tests/BeamCenterFromBackgroundTest.cpp +++ b/tests/BeamCenterFromBackgroundTest.cpp @@ -307,3 +307,29 @@ TEST_CASE("FindBeamCenter_BlanksTheBeamStopOutOfTheCapture", "[BeamCenter]") { CHECK(masked_error < 12.0f); CHECK(masked_error <= unmasked_error); } + +#ifdef JFJOCH_USE_CUDA +#include "../common/CUDAWrapper.h" + +// With a GPU the passes over the pixels run on it. The cell a pixel lands in is computed from IEEE +// operations alone and without contraction on both sides, and each cell is summed in the host's order, +// so the walk ends on the same bits - a tilted detector and an offset centre included. +TEST_CASE("BeamCenterFromBackground_DeviceMatchesHost", "[BeamCenter]") { + if (get_gpu_count() == 0) + SKIP("no GPU"); + DiffractionExperiment x = TestExperiment(); + x.PoniRot1_rad(0.005f).PoniRot2_rad(-0.003f); + PixelMask pixel_mask(x); + + const DiffractionGeometry geom_true = OffsetBy(x.GetDiffractionGeometry(), 7.0f, -4.5f); + const auto projection = SynthesiseProjection(x, pixel_mask, geom_true, 60.0f, 0.5f); + const auto host = FindBeamCenterFromBackground(x, pixel_mask, projection, 0, {}, /*allow_device=*/false); + const auto device = FindBeamCenterFromBackground(x, pixel_mask, projection, 0, {}, /*allow_device=*/true); + + REQUIRE(host.has_value()); + REQUIRE(device.has_value()); + CHECK(device->beam_x_pxl == host->beam_x_pxl); + CHECK(device->beam_y_pxl == host->beam_y_pxl); + CHECK(device->sigma_pxl == host->sigma_pxl); +} +#endif diff --git a/tests/ShadowFinderTest.cpp b/tests/ShadowFinderTest.cpp index cc89345f4..f01e2c0b4 100644 --- a/tests/ShadowFinderTest.cpp +++ b/tests/ShadowFinderTest.cpp @@ -471,3 +471,91 @@ TEST_CASE("ShadowFinder_ABrightRingIsNotABeamStop", "[ShadowFinder]") { // Anything much beyond the stop, its arm and their penumbra means the walk ran away. CHECK(std::count(mask.begin(), mask.end(), 1u) < 6000); } + +#ifdef JFJOCH_USE_CUDA +#include "../common/CUDAWrapper.h" +#include "../compression/JFJochCompressor.h" + +namespace { + // The mask and the mean projection of the same frames, once from the host projection (the frames + // handed over uncompressed) and once from the device's (handed over as bitshuffle+LZ4, which is + // what sends them to the GPU). + struct HostAndDevice { + std::vector host_mask, device_mask; + std::vector host_mean, device_mean; + }; + + HostAndDevice MaskBothWays(const DiffractionExperiment &x, const std::vector> &frames) { + const PixelMask pixel_mask(x); + HostAndDevice out; + std::vector buffer; + { + ShadowFinder finder(x, pixel_mask); + for (const auto &frame : frames) { + DataMessage msg{}; + msg.image = CompressedImage(frame, W, H); + finder.AddImage(msg, buffer); + } + out.host_mask = finder.GetMask(); + out.host_mean = finder.GetMeanProjection(); + } + { + ShadowFinder finder(x, pixel_mask); + JFJochBitShuffleCompressor compressor(CompressionAlgorithm::BSHUF_LZ4); + std::vector> compressed; + for (const auto &frame : frames) { + compressed.push_back(compressor.Compress(frame)); + DataMessage msg{}; + msg.image = CompressedImage(compressed.back().data(), compressed.back().size(), W, H, + CompressedImageMode::Int32, CompressionAlgorithm::BSHUF_LZ4); + finder.AddImage(msg, buffer); + } + out.device_mask = finder.GetMask(); + out.device_mean = finder.GetMeanProjection(); + } + return out; + } +} + +// The mask is made on the GPU wherever the projection is there. On scenes where no pixel sits within a +// rounding of a threshold the two must agree to the pixel: every step but the polarization factor, +// the Poisson test's logarithm and the arm search's azimuth is exact on both, and these scenes are +// built so that none of the three decides anything at an edge. +TEST_CASE("ShadowFinder_DeviceMaskMatchesHost", "[ShadowFinder]") { + if (get_gpu_count() == 0) + SKIP("no GPU"); + + SECTION("a beam stop with a reflection behind it") { + std::vector> frames; + for (int f = 0; f < NFRAMES; f++) + frames.push_back(Scene(false, f == 0)); + const auto r = MaskBothWays(TestExperiment(), frames); + CHECK(std::count(r.host_mask.begin(), r.host_mask.end(), 1u) == 2612); + CHECK(r.device_mask == r.host_mask); + CHECK(std::memcmp(r.device_mean.data(), r.host_mean.data(), r.host_mean.size() * sizeof(float)) == 0); + } + + SECTION("an arm that lets part of the beam through, across a module gap") { + constexpr int32_t BRIGHT = 50; + constexpr int ARM_HALF_WIDE = 15, GAP_X0 = 200, GAP_X1 = 216, OPAQUE_FROM_X = 232; + std::vector> frames; + for (int f = 0; f < NFRAMES; f++) { + frames.emplace_back(static_cast(W) * H, BRIGHT); + auto &frame = frames.back(); + for (int y = 0; y < H; y++) + for (int xi = 0; xi < W; xi++) { + const int dx = xi - C, dy = y - C; + if (dx * dx + dy * dy <= STOP_R * STOP_R) + frame[I(xi, y)] = 0; + else if (dx >= 0 && std::abs(dy) <= ARM_HALF_WIDE) + frame[I(xi, y)] = xi >= OPAQUE_FROM_X ? 0 : BRIGHT * 6 / 10; + if (xi >= GAP_X0 && xi <= GAP_X1) + frame[I(xi, y)] = INT32_MIN; + } + } + const auto r = MaskBothWays(TestExperiment(), frames); + CHECK(std::count(r.host_mask.begin(), r.host_mask.end(), ShadowFinder::TRANSMITTING) > 0); + CHECK(r.device_mask == r.host_mask); + } +} +#endif -- 2.54.0 From 3a5340696c6a39ffd57fdbc6ae435e4108a988e0 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 12:33:41 +0200 Subject: [PATCH 11/97] Pre-scan: measure the spots on the card in GPU builds The pre-scan's spot measurement (spot width and bandwidth, powder rings, spot-symmetry beam centre) ran the CPU preprocessor and adaptive spot finder on every sampled frame, ~13-19 core-seconds per 16M-pixel run. In a GPU build it now uses the same engines as the image loops: the frame is decoded and preprocessed on the device (with the host decoder as fallback), the spots are found and extracted there, and the preprocessed image comes back only when the spot width needs it. This is a choice of where, not of what: the device finders reproduce the host's spot list exactly (integer ring sums, the same connected components in the same order). Checked frame by frame with both engines side by side on myob/cytc/thau x10sa and all 21 smoke-tier sets (~540k spots, HDF5, CBF, marCCD, SMV, pink beam, 3.8 keV): identical spot lists in identical order on every frame, and an identical preprocessed image on every width frame. p.mtz md5 identical to the oracle on myob/cytc/thau, GPU and CPU builds. Measured on a loaded box (load 20-30): the spot measurement takes 0.8-0.95 s instead of 2.5-3.6 s and no longer competes for the cores the beam-stop mask and the background beam-centre fit run on; the pre-scan falls from 3.0-4.9 s to 1.7-2.4 s. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- rugnux/Rugnux.cpp | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index d99d8d8e1..736298cb8 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -57,6 +57,7 @@ #ifdef JFJOCH_USE_CUDA #include "../image_analysis/image_preprocessing/ImagePreprocessorGPU.h" #include "../image_analysis/image_preprocessing/ImagePreprocessorBufferGPU.h" +#include "../image_analysis/spot_finding/AdaptiveSpotFinderGPU.h" #endif #include "../image_analysis/scale_merge/Merge.h" #include "../image_analysis/scale_merge/RfreeFlags.h" @@ -929,12 +930,32 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru struct PreScanWorker { std::vector decompression_buffer; JFJochReaderRawImage raw_image; - std::unique_ptr preprocessor; + std::unique_ptr preprocessor; std::unique_ptr preprocessed; std::unique_ptr spot_finder; }; const auto make_worker = [&] { PreScanWorker w; +#ifdef JFJOCH_USE_CUDA + // On the card where there is one, as in the image loops: the frame is decoded and preprocessed + // there and the spots are found and extracted there. The device finders give the host's spot + // list to the bit - integer ring sums, the same connected components in the same order + // (AdaptiveSpotFinderGPU, SpotExtractorGPU) - so this is a choice of where, not of what. The + // preprocessed image comes back only for the spot width, which reads pixels around the spots. + if (want_spots && get_gpu_count() > 0) { + auto stream = std::make_shared(); + w.preprocessor = std::make_unique(prescan_x, prescan_mask, stream, + /*copy_image_to_host=*/want_width); + w.preprocessed = std::make_unique(prescan_x.GetPixelsNum(), + /*host_mirror=*/want_width); + if (config_.spot_finding.adaptive_threshold) + w.spot_finder = std::make_unique(*prescan_mapping, stream); + else + w.spot_finder = std::make_unique(prescan_x.GetXPixelsNumConv(), + prescan_x.GetYPixelsNumConv(), stream); + return w; + } +#endif if (want_spots) { w.preprocessor = std::make_unique(prescan_x, prescan_mask); w.preprocessed = std::make_unique(prescan_x.GetPixelsNum()); @@ -956,8 +977,20 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru bool for_width, std::vector &curves, std::vector &spot_q) { try { - w.preprocessor->Analyze(*w.preprocessed, - image.GetUncompressedPtr(w.decompression_buffer), image.GetMode()); + // As in the image loops: a frame the device cannot decode goes to the host decoder. + ImageStatistics stats; + bool decoded_on_device = false; + try { + decoded_on_device = w.preprocessor->AnalyzeCompressed(*w.preprocessed, image, stats); + } catch (const JFJochException &e) { + cuda_throw_if_context_lost(); + logger.Warning("Pre-scan: device decoding of image {} failed ({}), decompressing it on " + "the host", image_idx, e.what()); + cuda_clear_error(); + } + if (!decoded_on_device) + w.preprocessor->Analyze(*w.preprocessed, + image.GetUncompressedPtr(w.decompression_buffer), image.GetMode()); } catch (const std::exception &e) { if (IsFatalResourceError(e)) throw; logger.Warning("Pre-scan: failed to preprocess image {}: {}", image_idx, e.what()); -- 2.54.0 From d4f4bab158f6f97575359fea4340a77776a11fab Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 13:33:59 +0200 Subject: [PATCH 12/97] Pre-scan defective pixels: keep the GPU finder's work on the device The hot-pixel step of the pre-scan (MaskDefectivePixels) on a GPU build: - The device half is built once, before the workers start (HotPixelFinder::PrepareDevice), instead of by the first worker's frame while the others waited. The unmasked pixels grouped by key are sorted on the device (stable radix sort: the same order the host fill gave) instead of scattered on the host and uploaded. - No per-frame host round trip: the ring-sector levels and lit thresholds are made on the device from the order statistics, and the per-key frame and level sums are kept there too; frames queue on their workers' streams and the per-pixel accumulation is ordered by an event instead of a host synchronisation. - The mask: the chance rate's per-ring counts are summed on the device, and only the pixels the tests can pass (error value on most frames, or lit on at least min(max(2, k_chance), valid frames)) come back with their sums - not the five per-pixel arrays (470 MB pageable on a 16 Mpx detector). The host tests run on them unchanged. The threshold is written as fma(nsigma, noise, level) + offset on the host - what GCC already contracted it to - and the device takes the same two roundings, so the levels are bit-identical (and no longer depend on whether a compiler contracts). Exact: hot-pixel mask and p.mtz byte-identical to f849e2d1b on myoglobin, cytochrome C and thaumatin, GPU and CPU builds. Hot-pixel step (GPU, box at load 18-23, interleaved A/B, two pairs each): 0.94-1.32 s -> 0.65-0.81 s; frames 0.37-0.46 -> 0.18-0.23 s, mask 0.21-0.34 -> 0.08-0.18 s. Device memory of the finder: 543 MB as before, plus a ~0.36 GB transient for the sort while it is built. Tests: [HotPixelFinder] (HotPixelFinder_DeviceMatchesHost bit-exact), [ShadowFinder], [BeamCenter]. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- rugnux/HotPixels.cpp | 120 ++++++++++---------- rugnux/HotPixels.h | 8 +- rugnux/HotPixelsGPU.cu | 245 +++++++++++++++++++++++++++++++++-------- rugnux/HotPixelsGPU.h | 77 +++++++++---- rugnux/Rugnux.cpp | 5 + 5 files changed, 327 insertions(+), 128 deletions(-) diff --git a/rugnux/HotPixels.cpp b/rugnux/HotPixels.cpp index 5b898f8ca..88fdadc9e 100644 --- a/rugnux/HotPixels.cpp +++ b/rugnux/HotPixels.cpp @@ -216,7 +216,9 @@ void HotPixelFinder::AddLevels(const std::vector §or_level, const s const int r = static_cast(k / SECTORS); level[k] = std::max(ring_level[r], sector_level[k]); const float noise = std::max(std::sqrt(static_cast(std::max(level[k], 0))), ring_spread[r]); - threshold[k] = static_cast(level[k]) + LIT_NSIGMA * noise + LIT_OFFSET; + // One rounding for level + nsigma * noise and one for the offset, written out so that every + // compiler takes the same two - the device takes them too (HotPixelsGPU.cu). + threshold[k] = std::fma(LIT_NSIGMA, noise, static_cast(level[k])) + LIT_OFFSET; } // Every sum is an integer, so the result does not depend on the order the frames arrive in. @@ -230,50 +232,24 @@ void HotPixelFinder::AddLevels(const std::vector §or_level, const s } #ifdef JFJOCH_USE_CUDA +void HotPixelFinder::PrepareDevice() { + std::lock_guard lock(m); + if (!gpu) + gpu = std::make_unique(key.get(), width * height, key_begin, nrings, SECTORS, + HotPixelLevelRules{MIN_SECTOR_PIXELS, MIN_RING_PIXELS, + LIT_NSIGMA, LIT_OFFSET}); +} + void HotPixelFinder::AddDeviceImage(const int32_t *device_image, HotPixelFinderGPU::Frame &frame) { - HotPixelFinderGPU *device; - { - std::lock_guard lock(m); - if (!gpu) - gpu = std::make_unique(key.get(), width * height, key_begin, nrings, SECTORS); - device = gpu.get(); - } - std::vector count; - std::vector sector_median, ring_median, ring_mad; - device->Statistics(device_image, frame, count, sector_median, ring_median, ring_mad); - - // The same levels AddImage takes off its scratch buffer, and under the same pixel minima. - const size_t nkeys = static_cast(nrings) * SECTORS; - std::vector sector_level(nkeys, 0); - for (size_t k = 0; k < nkeys; k++) - if (count[k] >= MIN_SECTOR_PIXELS) - sector_level[k] = sector_median[k]; - std::vector ring_level(nrings, 0); - std::vector ring_spread(nrings, 0.0f); - std::vector ring_ok(nrings, 0); - for (int r = 0; r < nrings; r++) { - size_t n = 0; - for (int s = 0; s < SECTORS; s++) - n += count[r * SECTORS + s]; - if (n < MIN_RING_PIXELS) continue; - ring_ok[r] = 1; - ring_level[r] = ring_median[r]; - ring_spread[r] = 1.4826f * static_cast(ring_mad[r]); - } - - std::vector level; - std::vector threshold; - AddLevels(sector_level, ring_level, ring_spread, ring_ok, level, threshold); - device->Accumulate(device_image, frame, level, threshold, ring_ok); + PrepareDevice(); + gpu->Add(device_image, frame); + std::lock_guard lock(m); + frames++; } #endif HotPixelFinder::Result HotPixelFinder::GetMask(double oscillation_deg, double spacing_deg, size_t nthreads) { std::lock_guard lock(m); -#ifdef JFJOCH_USE_CUDA - if (gpu) - gpu->Download(n_lit.get(), n_error.get(), sum_value.get(), n_error_ring_ok.get(), error_level_sum.get()); -#endif Result ret; ret.frames = frames; ret.mask.assign(width * height, 0); @@ -282,25 +258,37 @@ HotPixelFinder::Result HotPixelFinder::GetMask(double oscillation_deg, double sp // The chance rate per ring, from the pixels lit on no more than half of their frames: whatever // lights those - reflections, zingers, noise above the bound - lights a defect-free pixel too. // Counted in integers by blocks of rows in parallel, so the totals do not depend on the split. - std::vector> block_lit(BANDS), block_seen(BANDS); - const size_t rows_per_band = (height + BANDS - 1) / BANDS; - ParallelFor(static_cast(BANDS), nthreads, [&](int b) { - block_lit[b].assign(nrings, 0); - block_seen[b].assign(nrings, 0); - const size_t begin = std::min(width * height, b * rows_per_band * width); - const size_t end = std::min(width * height, (b + 1) * rows_per_band * width); - for (size_t i = begin; i < end; i++) - if (key[i] >= 0 && n_valid(i) > 0 && 2 * n_lit[i] <= n_valid(i)) { - block_lit[b][key[i] / SECTORS] += n_lit[i]; - block_seen[b][key[i] / SECTORS] += n_valid(i); - } - }); std::vector lit(nrings, 0.0), seen(nrings, 0.0); - for (int r = 0; r < nrings; r++) - for (size_t b = 0; b < BANDS; b++) { - lit[r] += static_cast(block_lit[b][r]); - seen[r] += static_cast(block_seen[b][r]); +#ifdef JFJOCH_USE_CUDA + if (gpu) { + std::vector device_lit, device_seen; + gpu->ChanceCounts(device_lit, device_seen); + for (int r = 0; r < nrings; r++) { + lit[r] = static_cast(device_lit[r]); + seen[r] = static_cast(device_seen[r]); } + } else +#endif + { + std::vector> block_lit(BANDS), block_seen(BANDS); + const size_t rows_per_band = (height + BANDS - 1) / BANDS; + ParallelFor(static_cast(BANDS), nthreads, [&](int b) { + block_lit[b].assign(nrings, 0); + block_seen[b].assign(nrings, 0); + const size_t begin = std::min(width * height, b * rows_per_band * width); + const size_t end = std::min(width * height, (b + 1) * rows_per_band * width); + for (size_t i = begin; i < end; i++) + if (key[i] >= 0 && n_valid(i) > 0 && 2 * n_lit[i] <= n_valid(i)) { + block_lit[b][key[i] / SECTORS] += n_lit[i]; + block_seen[b][key[i] / SECTORS] += n_valid(i); + } + }); + for (int r = 0; r < nrings; r++) + for (size_t b = 0; b < BANDS; b++) { + lit[r] += static_cast(block_lit[b][r]); + seen[r] += static_cast(block_seen[b][r]); + } + } std::vector k_chance(nrings, n + 1); for (int r = 0; r < nrings; r++) if (seen[r] > 0.0) @@ -309,6 +297,26 @@ HotPixelFinder::Result HotPixelFinder::GetMask(double oscillation_deg, double sp // Persistent: lit on more frames than one reflection or chance explains. const int min_valid = std::max(10, n / 2); +#ifdef JFJOCH_USE_CUDA + // With a GPU the per-pixel sums stay there. Only the pixels that can be masked come back - those the + // tests below could pass (see GetCandidates) - into the host arrays, which are zero everywhere else, + // so the tests below run on them unchanged. + if (gpu) { + const auto c = gpu->GetCandidates(frames, min_valid, spacing_deg > 0.0, k_chance); + for (size_t j = 0; j < c.index.size(); j++) { + const size_t i = c.index[j]; + n_lit[i] = c.n_lit[j]; + n_error[i] = c.n_error[j]; + n_error_ring_ok[i] = c.n_error_ring_ok[j]; + sum_value[i] = c.sum_value[j]; + error_level_sum[i] = c.error_level_sum[j]; + } + for (size_t k = 0; k < key_frames.size(); k++) { + key_frames[k] = static_cast(c.key_frames[k]); + key_level_sum[k] = c.key_level_sum[k]; + } + } +#endif std::vector persistent(width * height, 0); ParallelChunks(static_cast(height), nthreads, [&](int y0, int y1) { for (size_t y = y0; y < static_cast(y1); y++) diff --git a/rugnux/HotPixels.h b/rugnux/HotPixels.h index 63e6bd7a8..b837ba32e 100644 --- a/rugnux/HotPixels.h +++ b/rugnux/HotPixels.h @@ -94,6 +94,10 @@ public: // `scratch` above. The per-pixel sums are then kept on the device and replace the host's when the // mask is read, so a finder is fed one way or the other, not both. Thread safe. void AddDeviceImage(const int32_t *device_image, HotPixelFinderGPU::Frame &frame); + + // Build the device half now rather than with the first device frame, so the workers do not wait on + // it one behind the other. + void PrepareDevice(); #endif // The mask, from the frames added so far. oscillation_deg is the rotation per image and @@ -132,11 +136,11 @@ private: std::unique_ptr n_error_ring_ok; std::unique_ptr error_level_sum; #ifdef JFJOCH_USE_CUDA - std::unique_ptr gpu; // built by the first device frame + std::unique_ptr gpu; // built by PrepareDevice or the first device frame #endif // Each ring-sector's level and lit threshold, from the frame's order statistics, and the frame's - // share of the per-key sums. The host and the device path both come through here. + // share of the per-key sums. The device does the same in HotPixelsGPU.cu, with the same roundings. void AddLevels(const std::vector §or_level, const std::vector &ring_level, const std::vector &ring_spread, const std::vector &ring_ok, std::vector &level, std::vector &threshold); diff --git a/rugnux/HotPixelsGPU.cu b/rugnux/HotPixelsGPU.cu index 2691c221f..dc6346dc6 100644 --- a/rugnux/HotPixelsGPU.cu +++ b/rugnux/HotPixelsGPU.cu @@ -3,6 +3,8 @@ #include "HotPixelsGPU.h" +#include + #include "../common/JFJochException.h" namespace { @@ -145,6 +147,84 @@ __global__ void accumulate_kernel(const int32_t *__restrict__ image, const int32 } } +// Each key's level and lit threshold from the frame's order statistics, and the frame's share of the +// per-key sums - HotPixelFinder::AddImage and AddLevels, step for step. The threshold is +// fma(nsigma, noise, level) + offset, the two roundings the host takes (see AddLevels). +__global__ void levels_kernel(size_t nkeys, int sectors, HotPixelLevelRules rules, const uint32_t *__restrict__ count, + const int32_t *__restrict__ sector_median, const int32_t *__restrict__ ring_median, + const int32_t *__restrict__ ring_mad, int32_t *__restrict__ level, + float *__restrict__ threshold, char *__restrict__ ring_ok, + uint32_t *__restrict__ key_frames, int64_t *__restrict__ key_level_sum) { + const size_t k = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (k >= nkeys) return; + const size_t r = k / sectors; + uint32_t n = 0; + for (int s = 0; s < sectors; s++) + n += count[r * sectors + s]; + const bool ok = n >= static_cast(rules.min_ring_pixels); + const int32_t ring_level = ok ? ring_median[r] : 0; + const float ring_spread = ok ? 1.4826f * static_cast(ring_mad[r]) : 0.0f; + const int32_t sector_level = count[k] >= static_cast(rules.min_sector_pixels) ? sector_median[k] : 0; + const int32_t lv = max(ring_level, sector_level); + const float root = sqrtf(static_cast(max(lv, 0))); + const float noise = root < ring_spread ? ring_spread : root; + level[k] = lv; + threshold[k] = __fadd_rn(__fmaf_rn(rules.lit_nsigma, noise, static_cast(lv)), rules.lit_offset); + if (k % sectors == 0) + ring_ok[r] = ok; + if (ok) { + atomicAdd(&key_frames[k], 1u); + atomicAdd(reinterpret_cast(&key_level_sum[k]), + static_cast(static_cast(lv))); + } +} + +__device__ int valid_frames(const int32_t *key, const uint32_t *key_frames, const uint16_t *n_error_ring_ok, size_t i) { + return static_cast(key_frames[key[i]]) - static_cast(n_error_ring_ok[i]); +} + +__global__ void chance_kernel(size_t npixels, int sectors, const int32_t *__restrict__ key, + const uint32_t *__restrict__ key_frames, const uint16_t *__restrict__ n_lit, + const uint16_t *__restrict__ n_error_ring_ok, unsigned long long *__restrict__ lit, + unsigned long long *__restrict__ seen) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= npixels || key[i] < 0) return; + const int nv = valid_frames(key, key_frames, n_error_ring_ok, i); + if (nv > 0 && 2 * static_cast(n_lit[i]) <= nv) { + atomicAdd(&lit[key[i] / sectors], static_cast(n_lit[i])); + atomicAdd(&seen[key[i] / sectors], static_cast(nv)); + } +} + +// Writes the candidates' indices from `out` on when `out` is given, and counts them either way. +__global__ void candidate_kernel(size_t npixels, int sectors, uint32_t frames, int min_valid, bool spacing_ok, + const int32_t *__restrict__ key, const uint32_t *__restrict__ key_frames, + const uint16_t *__restrict__ n_lit, const uint16_t *__restrict__ n_error, + const uint16_t *__restrict__ n_error_ring_ok, const int *__restrict__ k_chance, + uint32_t *__restrict__ count, uint32_t *__restrict__ out) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i >= npixels || key[i] < 0) return; + const int nv = valid_frames(key, key_frames, n_error_ring_ok, i); + const bool error = 2 * static_cast(n_error[i]) > frames; + const int lit = n_lit[i]; + const bool persistent = spacing_ok && nv >= min_valid && lit > 0 + && lit >= min(max(2, k_chance[key[i] / sectors]), nv); + if (!error && !persistent) return; + const uint32_t slot = atomicAdd(count, 1u); + if (out) out[slot] = static_cast(i); +} + +__global__ void iota_kernel(size_t n, uint32_t *__restrict__ out) { + const size_t i = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (i < n) out[i] = static_cast(i); +} + +template +__global__ void gather_kernel(size_t n, const uint32_t *__restrict__ index, const T *__restrict__ in, T *__restrict__ out) { + const size_t j = blockIdx.x * static_cast(blockDim.x) + threadIdx.x; + if (j < n) out[j] = in[index[j]]; +} + // The shared tables and sums are filled on a stream of their own, added to on the workers' streams // and downloaded on the NULL stream, so they are allocated synchronously rather than from the pool: // a pooled buffer is freed on the thread's allocation stream, which none of those is ordered before. @@ -156,22 +236,18 @@ constexpr CudaAlloc ALLOC = CudaAlloc::Synchronous; } // namespace HotPixelFinderGPU::HotPixelFinderGPU(const int32_t *host_key, size_t npixels, - const std::vector &host_key_begin, int nrings, int sectors) + const std::vector &host_key_begin, int nrings, int sectors, + const HotPixelLevelRules &rules) : npixels(npixels), nkeys(static_cast(nrings) * sectors), nrings(nrings), - sectors(sectors), - key(npixels, ALLOC), pixels_by_key(host_key_begin.back(), ALLOC), key_begin(host_key_begin.size(), ALLOC), + sectors(sectors), rules(rules), + key(npixels, ALLOC), pixels_by_key(std::max(host_key_begin.back(), 1), ALLOC), + key_begin(host_key_begin.size(), ALLOC), n_lit(npixels, ALLOC), n_error(npixels, ALLOC), n_error_ring_ok(npixels, ALLOC), - sum_value(npixels, ALLOC), error_level_sum(npixels, ALLOC) { - std::vector pixels(host_key_begin.back()); - std::vector filled(host_key_begin.begin(), host_key_begin.end() - 1); - for (size_t i = 0; i < npixels; i++) - if (host_key[i] >= 0) - pixels[filled[host_key[i]]++] = static_cast(i); - + sum_value(npixels, ALLOC), error_level_sum(npixels, ALLOC), + key_frames(std::max(nkeys, 1), ALLOC), key_level_sum(std::max(nkeys, 1), ALLOC) { + cuda_err(cudaEventCreateWithFlags(&last_accumulate, cudaEventDisableTiming)); CudaStream stream; cuda_err(cudaMemcpyAsync(key, host_key, npixels * sizeof(int32_t), cudaMemcpyHostToDevice, stream)); - cuda_err(cudaMemcpyAsync(pixels_by_key, pixels.data(), pixels.size() * sizeof(uint32_t), - cudaMemcpyHostToDevice, stream)); cuda_err(cudaMemcpyAsync(key_begin, host_key_begin.data(), host_key_begin.size() * sizeof(uint32_t), cudaMemcpyHostToDevice, stream)); cuda_err(cudaMemsetAsync(n_lit, 0, npixels * sizeof(uint16_t), stream)); @@ -179,16 +255,34 @@ HotPixelFinderGPU::HotPixelFinderGPU(const int32_t *host_key, size_t npixels, cuda_err(cudaMemsetAsync(n_error_ring_ok, 0, npixels * sizeof(uint16_t), stream)); cuda_err(cudaMemsetAsync(sum_value, 0, npixels * sizeof(int64_t), stream)); cuda_err(cudaMemsetAsync(error_level_sum, 0, npixels * sizeof(int64_t), stream)); - cuda_err(cudaStreamSynchronize(stream)); + cuda_err(cudaMemsetAsync(key_frames, 0, std::max(nkeys, 1) * sizeof(uint32_t), stream)); + cuda_err(cudaMemsetAsync(key_level_sum, 0, std::max(nkeys, 1) * sizeof(int64_t), stream)); + + // The unmasked pixels grouped by key: a stable sort of the pixel indices by key, so each key's + // pixels stay in pixel order. A masked pixel's key, -1, is the largest as unsigned and sorts last. + { + CudaDevicePtr index(npixels, ALLOC), sorted_key(npixels, ALLOC), sorted_index(npixels, ALLOC); + iota_kernel<<((npixels + THREADS - 1) / THREADS), THREADS, 0, stream>>>(npixels, index); + cuda_err(cudaGetLastError()); + const auto *keys_in = reinterpret_cast(key.get()); + size_t bytes = 0; + cuda_err(cub::DeviceRadixSort::SortPairs(nullptr, bytes, keys_in, sorted_key.get(), index.get(), + sorted_index.get(), npixels, 0, 32, stream)); + CudaDevicePtr scratch(bytes, ALLOC); + cuda_err(cub::DeviceRadixSort::SortPairs(scratch.get(), bytes, keys_in, sorted_key.get(), index.get(), + sorted_index.get(), npixels, 0, 32, stream)); + if (host_key_begin.back() > 0) + cuda_err(cudaMemcpyAsync(pixels_by_key, sorted_index, host_key_begin.back() * sizeof(uint32_t), + cudaMemcpyDeviceToDevice, stream)); + cuda_err(cudaStreamSynchronize(stream)); + } } -void HotPixelFinderGPU::Statistics(const int32_t *device_image, Frame &frame, std::vector &count, - std::vector §or_median, std::vector &ring_median, - std::vector &ring_mad) { - count.resize(nkeys); - sector_median.resize(nkeys); - ring_median.resize(nrings); - ring_mad.resize(nrings); +HotPixelFinderGPU::~HotPixelFinderGPU() { + if (last_accumulate) cudaEventDestroy(last_accumulate); +} + +void HotPixelFinderGPU::Add(const int32_t *device_image, Frame &frame) { if (nrings == 0) return; if (!frame.count.get()) { @@ -201,44 +295,101 @@ void HotPixelFinderGPU::Statistics(const int32_t *device_image, Frame &frame, st frame.ring_ok = CudaDevicePtr(nrings); } const cudaStream_t stream = *frame.stream; - sector_kernel<<(nkeys), THREADS, 0, stream>>>(device_image, pixels_by_key, key_begin, frame.count, - frame.sector_median); + sector_kernel<<(nkeys), THREADS, 0, stream>>>(device_image, pixels_by_key, key_begin, + frame.count, frame.sector_median); cuda_err(cudaGetLastError()); ring_kernel<<>>(device_image, pixels_by_key, key_begin, frame.count, sectors, frame.ring_median, frame.ring_mad); cuda_err(cudaGetLastError()); - - cuda_err(cudaMemcpyAsync(count.data(), frame.count, nkeys * sizeof(uint32_t), cudaMemcpyDeviceToHost, stream)); - cuda_err(cudaMemcpyAsync(sector_median.data(), frame.sector_median, nkeys * sizeof(int32_t), - cudaMemcpyDeviceToHost, stream)); - cuda_err(cudaMemcpyAsync(ring_median.data(), frame.ring_median, nrings * sizeof(int32_t), - cudaMemcpyDeviceToHost, stream)); - cuda_err(cudaMemcpyAsync(ring_mad.data(), frame.ring_mad, nrings * sizeof(int32_t), - cudaMemcpyDeviceToHost, stream)); - cuda_err(cudaStreamSynchronize(stream)); -} - -void HotPixelFinderGPU::Accumulate(const int32_t *device_image, Frame &frame, const std::vector &level, - const std::vector &threshold, const std::vector &ring_ok) { - const cudaStream_t stream = *frame.stream; - cuda_err(cudaMemcpyAsync(frame.level, level.data(), nkeys * sizeof(int32_t), cudaMemcpyHostToDevice, stream)); - cuda_err(cudaMemcpyAsync(frame.threshold, threshold.data(), nkeys * sizeof(float), cudaMemcpyHostToDevice, - stream)); - cuda_err(cudaMemcpyAsync(frame.ring_ok, ring_ok.data(), nrings * sizeof(char), cudaMemcpyHostToDevice, stream)); + levels_kernel<<((nkeys + THREADS - 1) / THREADS), THREADS, 0, stream>>>( + nkeys, sectors, rules, frame.count, frame.sector_median, frame.ring_median, frame.ring_mad, + frame.level, frame.threshold, frame.ring_ok, key_frames, key_level_sum); + cuda_err(cudaGetLastError()); std::lock_guard lock(accumulate_mutex); + cuda_err(cudaStreamWaitEvent(stream, last_accumulate, 0)); accumulate_kernel<<((npixels + THREADS - 1) / THREADS), THREADS, 0, stream>>>( device_image, key, npixels, sectors, frame.level, frame.threshold, frame.ring_ok, n_lit, n_error, sum_value, n_error_ring_ok, error_level_sum); cuda_err(cudaGetLastError()); + cuda_err(cudaEventRecord(last_accumulate, stream)); +} + +void HotPixelFinderGPU::ChanceCounts(std::vector &lit, std::vector &seen) { + CudaStream stream; + cuda_err(cudaStreamWaitEvent(stream, last_accumulate, 0)); + const size_t rings = std::max(nrings, 1); + CudaDevicePtr d_lit(rings, ALLOC), d_seen(rings, ALLOC); + cuda_err(cudaMemsetAsync(d_lit, 0, rings * sizeof(unsigned long long), stream)); + cuda_err(cudaMemsetAsync(d_seen, 0, rings * sizeof(unsigned long long), stream)); + chance_kernel<<((npixels + THREADS - 1) / THREADS), THREADS, 0, stream>>>( + npixels, sectors, key, key_frames, n_lit, n_error_ring_ok, d_lit, d_seen); + cuda_err(cudaGetLastError()); + lit.resize(nrings); + seen.resize(nrings); + cuda_err(cudaMemcpyAsync(lit.data(), d_lit, nrings * sizeof(int64_t), cudaMemcpyDeviceToHost, stream)); + cuda_err(cudaMemcpyAsync(seen.data(), d_seen, nrings * sizeof(int64_t), cudaMemcpyDeviceToHost, stream)); cuda_err(cudaStreamSynchronize(stream)); } -void HotPixelFinderGPU::Download(uint16_t *host_n_lit, uint16_t *host_n_error, int64_t *host_sum_value, - uint16_t *host_n_error_ring_ok, int64_t *host_error_level_sum) { - cuda_err(cudaMemcpy(host_n_lit, n_lit, npixels * sizeof(uint16_t), cudaMemcpyDeviceToHost)); - cuda_err(cudaMemcpy(host_n_error, n_error, npixels * sizeof(uint16_t), cudaMemcpyDeviceToHost)); - cuda_err(cudaMemcpy(host_sum_value, sum_value, npixels * sizeof(int64_t), cudaMemcpyDeviceToHost)); - cuda_err(cudaMemcpy(host_n_error_ring_ok, n_error_ring_ok, npixels * sizeof(uint16_t), cudaMemcpyDeviceToHost)); - cuda_err(cudaMemcpy(host_error_level_sum, error_level_sum, npixels * sizeof(int64_t), cudaMemcpyDeviceToHost)); +HotPixelFinderGPU::Candidates HotPixelFinderGPU::GetCandidates(uint32_t frames, int min_valid, bool spacing_ok, + const std::vector &k_chance) { + CudaStream stream; + cuda_err(cudaStreamWaitEvent(stream, last_accumulate, 0)); + const unsigned blocks = static_cast((npixels + THREADS - 1) / THREADS); + CudaDevicePtr d_k_chance(std::max(k_chance.size(), 1), ALLOC); + CudaDevicePtr d_count(1, ALLOC); + if (!k_chance.empty()) + cuda_err(cudaMemcpyAsync(d_k_chance, k_chance.data(), k_chance.size() * sizeof(int), cudaMemcpyHostToDevice, + stream)); + cuda_err(cudaMemsetAsync(d_count, 0, sizeof(uint32_t), stream)); + candidate_kernel<<>>(npixels, sectors, frames, min_valid, spacing_ok, key, key_frames, + n_lit, n_error, n_error_ring_ok, d_k_chance, d_count, nullptr); + cuda_err(cudaGetLastError()); + uint32_t n = 0; + cuda_err(cudaMemcpyAsync(&n, d_count, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream)); + cuda_err(cudaStreamSynchronize(stream)); + + Candidates c; + c.key_frames.resize(nkeys); + c.key_level_sum.resize(nkeys); + if (nkeys > 0) { + cuda_err(cudaMemcpyAsync(c.key_frames.data(), key_frames, nkeys * sizeof(uint32_t), cudaMemcpyDeviceToHost, + stream)); + cuda_err(cudaMemcpyAsync(c.key_level_sum.data(), key_level_sum, nkeys * sizeof(int64_t), + cudaMemcpyDeviceToHost, stream)); + } + if (n > 0) { + CudaDevicePtr index(n, ALLOC); + CudaDevicePtr g_lit(n, ALLOC), g_error(n, ALLOC), g_error_ring_ok(n, ALLOC); + CudaDevicePtr g_sum(n, ALLOC), g_error_level(n, ALLOC); + cuda_err(cudaMemsetAsync(d_count, 0, sizeof(uint32_t), stream)); + candidate_kernel<<>>(npixels, sectors, frames, min_valid, spacing_ok, key, + key_frames, n_lit, n_error, n_error_ring_ok, d_k_chance, + d_count, index); + cuda_err(cudaGetLastError()); + const unsigned gb = (n + THREADS - 1) / THREADS; + gather_kernel<<>>(n, index, n_lit.get(), g_lit.get()); + gather_kernel<<>>(n, index, n_error.get(), g_error.get()); + gather_kernel<<>>(n, index, n_error_ring_ok.get(), g_error_ring_ok.get()); + gather_kernel<<>>(n, index, sum_value.get(), g_sum.get()); + gather_kernel<<>>(n, index, error_level_sum.get(), g_error_level.get()); + cuda_err(cudaGetLastError()); + c.index.resize(n); + c.n_lit.resize(n); + c.n_error.resize(n); + c.n_error_ring_ok.resize(n); + c.sum_value.resize(n); + c.error_level_sum.resize(n); + cuda_err(cudaMemcpyAsync(c.index.data(), index, n * sizeof(uint32_t), cudaMemcpyDeviceToHost, stream)); + cuda_err(cudaMemcpyAsync(c.n_lit.data(), g_lit, n * sizeof(uint16_t), cudaMemcpyDeviceToHost, stream)); + cuda_err(cudaMemcpyAsync(c.n_error.data(), g_error, n * sizeof(uint16_t), cudaMemcpyDeviceToHost, stream)); + cuda_err(cudaMemcpyAsync(c.n_error_ring_ok.data(), g_error_ring_ok, n * sizeof(uint16_t), + cudaMemcpyDeviceToHost, stream)); + cuda_err(cudaMemcpyAsync(c.sum_value.data(), g_sum, n * sizeof(int64_t), cudaMemcpyDeviceToHost, stream)); + cuda_err(cudaMemcpyAsync(c.error_level_sum.data(), g_error_level, n * sizeof(int64_t), + cudaMemcpyDeviceToHost, stream)); + } + cuda_err(cudaStreamSynchronize(stream)); + return c; } diff --git a/rugnux/HotPixelsGPU.h b/rugnux/HotPixelsGPU.h index f062e64cf..e68b6e4f0 100644 --- a/rugnux/HotPixelsGPU.h +++ b/rugnux/HotPixelsGPU.h @@ -10,33 +10,53 @@ #include "../image_analysis/indexing/CUDAMemHelpers.h" -// The device half of HotPixelFinder, for frames already preprocessed on the GPU: the per-frame order -// statistics (each ring-sector's median, each ring's median and median absolute deviation) and the -// per-pixel sums run where the image already is, and only the per-key statistics - tens of thousands -// of numbers - come to the host, which turns them into levels and thresholds with the very code the -// host path uses. Every statistic is an exact order statistic of integers and every sum an integer, -// so the sums, and with them the mask, are identical to what HotPixelFinder::AddImage produces. +// What a frame's levels and lit thresholds are made of (HotPixelFinder's constants), handed to the +// device so the two halves cannot drift apart. +struct HotPixelLevelRules { + int min_sector_pixels; + int min_ring_pixels; + float lit_nsigma; + float lit_offset; +}; + +// The device half of HotPixelFinder, for frames already preprocessed on the GPU. Everything a frame adds +// stays on the device: the per-frame order statistics (each ring-sector's median, each ring's median +// and median absolute deviation), the levels and thresholds made from them, and the per-pixel and +// per-key sums. Every statistic is an exact order statistic of integers, the threshold is computed +// with the same rounding steps as the host's (see HotPixelFinder::AddLevels) and every sum is an +// integer, so the sums are identical to what HotPixelFinder::AddImage produces. When the mask is read +// only what can decide it comes back: per-ring counts for the chance rate, then the few pixels that can +// be persistent or carry the error value. class HotPixelFinderGPU { const size_t npixels; const size_t nkeys; const int nrings; const int sectors; + const HotPixelLevelRules rules; CudaDevicePtr key; // ring * sectors + sector of each pixel, -1 masked - CudaDevicePtr pixels_by_key; // the unmasked pixels, grouped by key + CudaDevicePtr pixels_by_key; // the unmasked pixels, grouped by key, in pixel order CudaDevicePtr key_begin; // where each key's pixels start in pixels_by_key - // The per-pixel sums, exactly those of HotPixelFinder. + // The per-pixel and per-key sums, exactly those of HotPixelFinder. CudaDevicePtr n_lit, n_error, n_error_ring_ok; CudaDevicePtr sum_value, error_level_sum; - // Frames are selected on their workers' streams in parallel, but each pixel's sums are plain - // read-modify-writes, so one frame at a time adds to them. + CudaDevicePtr key_frames; + CudaDevicePtr key_level_sum; + + // Each pixel's sums are plain read-modify-writes, so the frames add to them one after another: + // every accumulation waits on the device for the one before it (an event, not the host). std::mutex accumulate_mutex; + cudaEvent_t last_accumulate = nullptr; public: // One worker's buffers, on the stream its frames are preprocessed on. struct Frame { explicit Frame(std::shared_ptr stream) : stream(std::move(stream)) {} + // Its frames are queued, not waited for (Add), so the buffers below must not be freed before + // the stream has used them. + ~Frame() { if (stream) cudaStreamSynchronize(*stream); } + Frame(Frame &&) = default; std::shared_ptr stream; CudaDevicePtr count; // valid pixels per key CudaDevicePtr sector_median; // per key @@ -46,21 +66,32 @@ public: CudaDevicePtr ring_ok; }; + // The keys of HotPixelFinder: one per pixel, and where each key's pixels start among the unmasked + // pixels sorted by key. HotPixelFinderGPU(const int32_t *key, size_t npixels, const std::vector &key_begin, int nrings, - int sectors); + int sectors, const HotPixelLevelRules &rules); + ~HotPixelFinderGPU(); + HotPixelFinderGPU(const HotPixelFinderGPU &) = delete; + HotPixelFinderGPU &operator=(const HotPixelFinderGPU &) = delete; - // The lower median of the valid values of each key (count[k] of them, 0 where there are none), and - // of each ring the median and the lower median of the absolute deviations from it. - void Statistics(const int32_t *device_image, Frame &frame, std::vector &count, - std::vector §or_median, std::vector &ring_median, - std::vector &ring_mad); + // Add one frame, preprocessed on frame.stream, as HotPixelFinder::AddImage does. Queued on that + // stream; nothing is waited for on the host. + void Add(const int32_t *device_image, Frame &frame); - // Add the frame to the per-pixel sums, with each key's level and lit threshold and each ring's - // verdict on whether it has a level at all - as HotPixelFinder::AddImage does. - void Accumulate(const int32_t *device_image, Frame &frame, const std::vector &level, - const std::vector &threshold, const std::vector &ring_ok); + // Per ring, over the pixels lit on no more than half of their valid frames: the lit frames and the + // valid frames, summed (HotPixelFinder::GetMask's chance rate). Waits for every frame added. + void ChanceCounts(std::vector &lit, std::vector &seen); - // The per-pixel sums, npixels each. - void Download(uint16_t *n_lit, uint16_t *n_error, int64_t *sum_value, uint16_t *n_error_ring_ok, - int64_t *error_level_sum); + // The pixels that can be masked: those holding the error value on more than half of `frames`, and, + // where spacing_ok, those lit on at least min(max(2, k_chance[ring]), valid frames) of at least + // min_valid valid frames - a persistent pixel is lit on at least that many, because one reflection + // explains at least two. For each, its index and per-pixel sums; and every key's sums. + struct Candidates { + std::vector index; + std::vector n_lit, n_error, n_error_ring_ok; + std::vector sum_value, error_level_sum; + std::vector key_frames; + std::vector key_level_sum; + }; + Candidates GetCandidates(uint32_t frames, int min_valid, bool spacing_ok, const std::vector &k_chance); }; diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index d99d8d8e1..10605a01a 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -1669,6 +1669,11 @@ void Rugnux::MaskDefectivePixels(int start_image, const std::vector &sample const size_t nthreads = static_cast(std::max(config_.nthreads, 1)); HotPixelFinder finder(about, pixel_mask_, nthreads); +#ifdef JFJOCH_USE_CUDA + // Built here, once, so the workers' first frames do not queue behind it. + if (get_gpu_count() > 0) + finder.PrepareDevice(); +#endif std::atomic next{0}; std::vector> futures; const size_t nworkers = std::min(nthreads, std::min(PRESCAN_MAX_WORKERS, sample.size())); -- 2.54.0 From eb1e6fa4108859a8fbb3965d1c1d2f700db5c725 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 13:35:43 +0200 Subject: [PATCH 13/97] RotationScaleMerge: correction-surface fit passes on the GPU, same bits ApplyCellSurface (detector modulation, time x detector, crystal-frame SH and goniometer-frame absorption) spends most of its time in two passes per round: the per-group reference sums and the per-(block, cell) fit sums. With a GPU both now run on the device (RotationScaleMergeGPU::Surface*) over the same terms in the same order: - reference: one thread per ASU group, walking a group-order permutation of the terms in fulls order; - fit: each subset cut into the host's reduction blocks with every block's terms ordered by cell (stable counting sort, host); a per-term kernel forms w*Is, w*Iref, Iref and one thread per (block, cell) runs the two fma chains; the block slots are added per cell in block order. Every rounding is spelled out (__dmul_rn/__dadd_rn/fma) to be the one the host build makes: GCC at -march=x86-64-v3 fuses swI's multiply-add only in the parity-filtered copy of the reference loop, and both fit sums. Host side, exact on both paths: the 19 serial nth_element selections of the shell edges become one parallel sort (same order statistics), the per-term shell lookup runs on all threads, and the gate's per-shell CC is one walk over the groups instead of one per shell. Exact: p.mtz md5 identical to the oracle on myob/cytc/thau x10sa, GPU build (all CUDA architectures) and CPU build. CorrectionSurfaceGPU test checks the device sums bit for bit against an explicitly rounded host loop. Measured (cytc/thau, two interleaved A/B pairs, box at load 13-25 from sibling work): ApplyCellSurface host core-seconds -83% on cytc; SG adoption -> writing reflections 4.58->3.75 and 3.85->2.54 s (cytc), 3.15->2.28 and 2.29->2.03 s (thau); RSM final merge -0.7..-0.9 s and P1 cross-check -1.5..-1.8 s on cytc. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- .../scale_merge/RotationScaleMerge.cpp | 147 ++++++++++---- .../scale_merge/RotationScaleMergeGPU.cu | 191 ++++++++++++++++++ .../scale_merge/RotationScaleMergeGPU.h | 28 +++ tests/CMakeLists.txt | 1 + tests/CorrectionSurfaceGPUTest.cpp | 167 +++++++++++++++ 5 files changed, 493 insertions(+), 41 deletions(-) create mode 100644 tests/CorrectionSurfaceGPUTest.cpp diff --git a/image_analysis/scale_merge/RotationScaleMerge.cpp b/image_analysis/scale_merge/RotationScaleMerge.cpp index 6eff70b3e..81dabcd22 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.cpp +++ b/image_analysis/scale_merge/RotationScaleMerge.cpp @@ -3,6 +3,7 @@ #include "../../common/ParallelFor.h" #include "RotationScaleMerge.h" +#include "RotationScaleMergeGPU.h" // SurfaceTerm, the surface fit's term on both paths #include #include @@ -3334,7 +3335,7 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int // traffic of the data it touched, over hundreds of megabytes. Copying once turns them into // sequential walks of a compact array. The copy keeps fulls order, so each sum below is formed // from exactly the same terms in exactly the same order. - struct Term { float I, sigma, corr, d; int32_t cell, group; }; + using Term = RotationScaleMergeGPU::SurfaceTerm; // float I, sigma, corr, d; int32_t cell, group std::vector term; std::vector term_parity; // frame parity, read only by the group-ordered copy below // Which terms each pass walks, as positions in `term`. The cross-validated halves then cost half a @@ -3437,22 +3438,27 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int std::vector shw_shell(nshell, 0.0), shw_cell(nshell > 0 ? ncell : 0, 0.0); std::vector group_shell(n_groups, 0); // the shell each ASU group sits in, for the gate if (nshell > 0) { - std::vector s2; - s2.reserve(term.size()); - for (const Term &t : term) - s2.push_back(t.d > 0.0f ? 1.0f / (t.d * t.d) : 0.0f); + const int n_term = static_cast(term.size()); + std::vector s2(n_term); + ParallelChunks(n_term, nt, [&](int lo, int hi) { + for (int k = lo; k < hi; ++k) + s2[k] = term[k].d > 0.0f ? 1.0f / (term[k].d * term[k].d) : 0.0f; + }); + // The edges are order statistics of s2, so they are the same values however the sort orders + // equal elements among themselves. + std::vector sorted = s2; + ParallelSort(sorted.begin(), sorted.end(), nt, std::less()); std::vector edge(nshell - 1); - size_t prev = 0; - for (int i = 1; i < nshell; ++i) { - const size_t pos = s2.size() * static_cast(i) / static_cast(nshell); - std::nth_element(s2.begin() + prev, s2.begin() + pos, s2.end()); - edge[i - 1] = s2[pos]; - prev = pos; - } + for (int i = 1; i < nshell; ++i) + edge[i - 1] = sorted[sorted.size() * static_cast(i) / static_cast(nshell)]; + std::vector term_shell(n_term); + ParallelChunks(n_term, nt, [&](int lo, int hi) { + for (int k = lo; k < hi; ++k) + term_shell[k] = static_cast(std::upper_bound(edge.begin(), edge.end(), s2[k]) - edge.begin()); + }); for (size_t k = 0; k < term.size(); ++k) { const Term &t = term[k]; - const float v = t.d > 0.0f ? 1.0f / (t.d * t.d) : 0.0f; - const int s = static_cast(std::upper_bound(edge.begin(), edge.end(), v) - edge.begin()); + const int s = term_shell[k]; group_shell[t.group] = s; const double sc = static_cast(t.sigma) * t.corr; if (!(sc > 0.0)) continue; @@ -3475,8 +3481,41 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int std::vector g_start(n_groups + 1, 0); for (const Term &t : term) ++g_start[t.group + 1]; for (int g = 0; g < n_groups; ++g) g_start[g + 1] += g_start[g]; - std::vector gterm(term.size()); - { + std::vector gterm; + // With a GPU the two passes of every round run there (RotationScaleMergeGPU::Surface*), on the same + // terms in the same order: the device gets the terms with the group order as a permutation, and each + // subset cut into this fit's reduction blocks with every block's terms ordered by cell, so that one + // device thread forms what one block of the host loop sums into one cell. + bool on_gpu = false; +#ifdef JFJOCH_USE_CUDA + on_gpu = gpu_active_; + if (on_gpu) { + std::vector gperm(term.size()); + std::vector fill(g_start.begin(), g_start.end() - 1); + for (size_t k = 0; k < term.size(); ++k) + gperm[fill[term[k].group]++] = static_cast(k); + gpu_->SurfaceSetTerms(static_cast(term.size()), term.data(), term_parity.data(), + n_groups, gperm.data(), g_start.data(), ncell); + for (int parity : {0, 1, -1}) { + const std::vector &sel = subset(parity); + const int n = static_cast(sel.size()); + const int nb = ReductionBlocks(n, SURFACE_BLOCK); + std::vector perm(n), seg_start(static_cast(nb) * ncell + 1, n); + ParallelFor(nb, nt, [&](int b) { + const int lo = static_cast(static_cast(n) * b / nb); + const int hi = static_cast(static_cast(n) * (b + 1) / nb); + std::vector pos(ncell + 1, 0); + for (int k = lo; k < hi; ++k) ++pos[term[sel[k]].cell + 1]; + for (int c = 0; c < ncell; ++c) pos[c + 1] += pos[c]; + for (int c = 0; c < ncell; ++c) seg_start[static_cast(b) * ncell + c] = lo + pos[c]; + for (int k = lo; k < hi; ++k) perm[lo + pos[term[sel[k]].cell]++] = sel[k]; + }); + gpu_->SurfaceSetSubset(parity < 0 ? 2 : parity, nb, perm.data(), seg_start.data()); + } + } +#endif + if (!on_gpu) { + gterm.resize(term.size()); std::vector fill(g_start.begin(), g_start.end() - 1); for (size_t k = 0; k < term.size(); ++k) { const Term &t = term[k]; @@ -3488,6 +3527,12 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int // and the score. std::vector sw(n_groups), swI(n_groups); auto reference = [&](int parity, const std::vector &A) { +#ifdef JFJOCH_USE_CUDA + if (on_gpu) { + gpu_->SurfaceReference(parity, A.data()); + return; + } +#endif ParallelChunks(n_groups, nt, [&](int glo, int ghi) { for (int g = glo; g < ghi; ++g) { double s_w = 0.0, s_wI = 0.0; @@ -3520,7 +3565,7 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int const std::vector &sel = subset(parity); std::vector A(ncell, 1.0); // Per-block cell accumulators, allocated once for the whole fit rather than per round. - const int nb = ReductionBlocks(static_cast(sel.size()), SURFACE_BLOCK); + const int nb = on_gpu ? 0 : ReductionBlocks(static_cast(sel.size()), SURFACE_BLOCK); std::vector> tcross(nb, std::vector(ncell)), tref2(nb, std::vector(ncell)); settled = false; n_clamped = 0; @@ -3555,22 +3600,28 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int // there is no ordering that keeps threads off each other's bins, and there are only ncell // of them, so per-block copies are cheap and the fixed blocks keep it the same at any -N. std::vector cross(ncell, 0.0), ref2(ncell, 0.0); - ParallelBlocks(static_cast(sel.size()), nt, [&](int b, int lo, int hi) { - std::vector &xcross = tcross[b], &xref2 = tref2[b]; - std::fill(xcross.begin(), xcross.end(), 0.0); - std::fill(xref2.begin(), xref2.end(), 0.0); - for (int k = lo; k < hi; ++k) { - const Term &o = term[sel[k]]; - if (sw[o.group] <= 0.0) continue; - const double Iref = swI[o.group] / sw[o.group], a = A[o.cell]; - const double Is = static_cast(o.I) * o.corr * a, sc = static_cast(o.sigma) * o.corr * a; - if (!std::isfinite(Iref) || Iref <= 0.0 || !(sc > 0.0)) continue; - const double w = 1.0 / (sc * sc); - xcross[o.cell] += w * Is * Iref; xref2[o.cell] += w * Iref * Iref; - } - }, SURFACE_BLOCK); - for (int b = 0; b < nb; ++b) - for (int c = 0; c < ncell; ++c) { cross[c] += tcross[b][c]; ref2[c] += tref2[b][c]; } +#ifdef JFJOCH_USE_CUDA + if (on_gpu) + gpu_->SurfaceFitSums(parity < 0 ? 2 : parity, cross.data(), ref2.data()); +#endif + if (!on_gpu) { + ParallelBlocks(static_cast(sel.size()), nt, [&](int b, int lo, int hi) { + std::vector &xcross = tcross[b], &xref2 = tref2[b]; + std::fill(xcross.begin(), xcross.end(), 0.0); + std::fill(xref2.begin(), xref2.end(), 0.0); + for (int k = lo; k < hi; ++k) { + const Term &o = term[sel[k]]; + if (sw[o.group] <= 0.0) continue; + const double Iref = swI[o.group] / sw[o.group], a = A[o.cell]; + const double Is = static_cast(o.I) * o.corr * a, sc = static_cast(o.sigma) * o.corr * a; + if (!std::isfinite(Iref) || Iref <= 0.0 || !(sc > 0.0)) continue; + const double w = 1.0 / (sc * sc); + xcross[o.cell] += w * Is * Iref; xref2[o.cell] += w * Iref * Iref; + } + }, SURFACE_BLOCK); + for (int b = 0; b < nb; ++b) + for (int c = 0; c < ncell; ++c) { cross[c] += tcross[b][c]; ref2[c] += tref2[b][c]; } + } std::vector dsorted = cross; std::nth_element(dsorted.begin(), dsorted.begin() + dsorted.size() / 2, dsorted.end()); const double lambda = 0.1 * std::max(1e-30, dsorted[dsorted.size() / 2]); @@ -3672,20 +3723,33 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int const int nsh_cc = nshell > 0 ? nshell : 1; auto half_means = [&](int parity, const std::vector &A) { reference(parity, A); +#ifdef JFJOCH_USE_CUDA + if (on_gpu) + gpu_->SurfaceGetReference(sw.data(), swI.data()); +#endif std::vector m(n_groups, std::numeric_limits::quiet_NaN()); for (int g = 0; g < n_groups; ++g) if (sw[g] > 0.0) m[g] = swI[g] / sw[g]; return m; }; - auto shell_cc = [&](const std::vector &x, const std::vector &y, int s) { - double n = 0, sx = 0, sy = 0, sxx = 0, syy = 0, sxy = 0; + // The correlation within every shell, in one walk over the groups: each shell still sums its own + // groups in group order. + auto shell_cc = [&](const std::vector &x, const std::vector &y) { + struct Sums { double n = 0, sx = 0, sy = 0, sxx = 0, syy = 0, sxy = 0; }; + std::vector sum(nsh_cc); for (int g = 0; g < n_groups; ++g) { - if (group_shell[g] != s || !std::isfinite(x[g]) || !std::isfinite(y[g])) continue; - n += 1; sx += x[g]; sy += y[g]; sxx += x[g] * x[g]; syy += y[g] * y[g]; sxy += x[g] * y[g]; + if (!std::isfinite(x[g]) || !std::isfinite(y[g])) continue; + Sums &u = sum[group_shell[g]]; + u.n += 1; u.sx += x[g]; u.sy += y[g]; u.sxx += x[g] * x[g]; u.syy += y[g] * y[g]; u.sxy += x[g] * y[g]; } - const double vx = sxx - sx * sx / n, vy = syy - sy * sy / n; - return (n >= 50 && vx > 0.0 && vy > 0.0) ? (sxy - sx * sy / n) / std::sqrt(vx * vy) - : std::numeric_limits::quiet_NaN(); + std::vector cc(nsh_cc, std::numeric_limits::quiet_NaN()); + for (int s = 0; s < nsh_cc; ++s) { + const Sums &u = sum[s]; + const double vx = u.sxx - u.sx * u.sx / u.n, vy = u.syy - u.sy * u.sy / u.n; + if (u.n >= 50 && vx > 0.0 && vy > 0.0) + cc[s] = (u.sxy - u.sx * u.sy / u.n) / std::sqrt(vx * vy); + } + return cc; }; const std::vector ident(ncell, 1.0); const std::vector A_even = fit_surface(0), A_odd = fit_surface(1); @@ -3702,8 +3766,9 @@ void RotationScaleMerge::ApplyCellSurface(const std::vector &cell, int // Following Fisher (1915) Biometrika 10, 507-521 double gain = 0.0; int n_cc = 0; + const std::vector cc0 = shell_cc(odd0, even0), cc1 = shell_cc(odd1, even1); for (int sh = 0; sh < nsh_cc; ++sh) { - const double c0 = shell_cc(odd0, even0, sh), c1 = shell_cc(odd1, even1, sh); + const double c0 = cc0[sh], c1 = cc1[sh]; if (std::isfinite(c0) && std::isfinite(c1)) { gain += std::atanh(c1) - std::atanh(c0); ++n_cc; } } if (n_cc > 0) gain /= n_cc; diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.cu b/image_analysis/scale_merge/RotationScaleMergeGPU.cu index 6e54fb380..66aeabacb 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.cu +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.cu @@ -639,6 +639,100 @@ namespace { } } + // --- correction-surface fit (ApplyCellSurface) --- + // The host fit is the reference here: these kernels form the same sums from the same terms in the + // same order, and every rounding is spelled out (__dmul_rn / __dadd_rn round each step on its own, + // fma rounds once) to be the one the host build makes - nvcc would otherwise contract a multiply + // and an add wherever it sees them, and GCC at -march=x86-64-v3 does so only in some of them. The + // products are the host's (I * corr) * a and (sigma * corr) * a. + using SurfaceTerm = RotationScaleMergeGPU::SurfaceTerm; + + __device__ __forceinline__ double SurfaceIs(const SurfaceTerm &t, double a) { + return __dmul_rn(__dmul_rn(double(t.I), double(t.corr)), a); + } + + __device__ __forceinline__ double SurfaceSigma(const SurfaceTerm &t, double a) { + return __dmul_rn(__dmul_rn(double(t.sigma), double(t.corr)), a); + } + + // One thread per ASU group: the group's inverse-variance sums over its terms of frame parity `parity` + // (< 0 = all), in fulls order - the host's `reference`. The host builds that loop twice, and only + // the parity-filtered copy fuses the multiply-add of swI; the unfiltered one rounds the product. + __global__ void SurfaceReferenceKernel(int n_groups, int parity, const int32_t *__restrict__ gperm, + const int32_t *__restrict__ gstart, + const SurfaceTerm *__restrict__ term, + const uint8_t *__restrict__ term_parity, + const double *__restrict__ A, + double *__restrict__ sw, double *__restrict__ swI) { + for (int g = blockIdx.x * blockDim.x + threadIdx.x; g < n_groups; g += gridDim.x * blockDim.x) { + double s_w = 0.0, s_wI = 0.0; + for (int k = gstart[g]; k < gstart[g + 1]; ++k) { + const int i = gperm[k]; + if (parity >= 0 && term_parity[i] != parity) continue; + const SurfaceTerm t = term[i]; + const double a = A[t.cell]; + const double Is = SurfaceIs(t, a), sc = SurfaceSigma(t, a); + const double w = 1.0 / __dmul_rn(sc, sc); + s_w = __dadd_rn(s_w, w); + s_wI = parity >= 0 ? fma(Is, w, s_wI) : __dadd_rn(s_wI, __dmul_rn(Is, w)); + } + sw[g] = s_w; swI[g] = s_wI; + } + } + + // The fit's sums of one round. Each term's contribution is formed on its own thread (one per term, + // in segment order), and only the two multiply-adds that sum them are left to the thread of each + // (reduction block, cell) - the per-block accumulators of the host fit, one slot at a time, walking + // its segment in term order. A term the host skips is marked by a NaN Iref (a kept one is finite). + __global__ void SurfaceFitTermKernel(int n, const int32_t *__restrict__ perm, + const SurfaceTerm *__restrict__ term, + const double *__restrict__ A, + const double *__restrict__ sw, const double *__restrict__ swI, + double *__restrict__ w_Is, double *__restrict__ w_Iref, + double *__restrict__ Iref_out) { + for (int k = blockIdx.x * blockDim.x + threadIdx.x; k < n; k += gridDim.x * blockDim.x) { + const SurfaceTerm t = term[perm[k]]; + Iref_out[k] = NAN; + if (sw[t.group] <= 0.0) continue; + const double Iref = swI[t.group] / sw[t.group], a = A[t.cell]; + const double Is = SurfaceIs(t, a), sc = SurfaceSigma(t, a); + if (!isfinite(Iref) || Iref <= 0.0 || !(sc > 0.0)) continue; + const double w = 1.0 / __dmul_rn(sc, sc); + w_Is[k] = __dmul_rn(w, Is); + w_Iref[k] = __dmul_rn(w, Iref); + Iref_out[k] = Iref; + } + } + + __global__ void SurfaceFitSegmentKernel(int n_seg, const int32_t *__restrict__ seg_start, + const double *__restrict__ w_Is, const double *__restrict__ w_Iref, + const double *__restrict__ Iref, + double *__restrict__ tcross, double *__restrict__ tref2) { + for (int s = blockIdx.x * blockDim.x + threadIdx.x; s < n_seg; s += gridDim.x * blockDim.x) { + double xcross = 0.0, xref2 = 0.0; + for (int k = seg_start[s]; k < seg_start[s + 1]; ++k) { + if (isnan(Iref[k])) continue; + xcross = fma(w_Is[k], Iref[k], xcross); + xref2 = fma(w_Iref[k], Iref[k], xref2); + } + tcross[s] = xcross; tref2[s] = xref2; + } + } + + // One thread per cell: the blocks' sums added up in block order, as the host adds its slots. + __global__ void SurfaceFitCellKernel(int n_blocks, int ncell, const double *__restrict__ tcross, + const double *__restrict__ tref2, + double *__restrict__ cross, double *__restrict__ ref2) { + for (int c = blockIdx.x * blockDim.x + threadIdx.x; c < ncell; c += gridDim.x * blockDim.x) { + double sc = 0.0, sr = 0.0; + for (int b = 0; b < n_blocks; ++b) { + sc = __dadd_rn(sc, tcross[b * ncell + c]); + sr = __dadd_rn(sr, tref2[b * ncell + c]); + } + cross[c] = sc; ref2[c] = sr; + } + } + void CudaCheck(cudaError_t e, const char *what); // A copy on the instance's stream, waited for - what cudaMemcpy on the NULL stream was, without also @@ -817,6 +911,18 @@ struct RotationScaleMergeGPU::Impl { CudaDevicePtr f_sco_ok; CudaDevicePtr f_frame_perm, f_frame_start, f_frame_count; CudaDevicePtr f_gperm, f_gstart, f_gcount; + // correction-surface fit (one ApplyCellSurface call at a time): its terms and their group CSR, the + // three subsets' per-(block, cell) segments, the surface and the sums of the round + int s_ncell = 0, s_n_groups = 0; + int s_n_blocks[3] = {0, 0, 0}; + int s_n_sel[3] = {0, 0, 0}; + size_t s_slots = 0; // capacity of s_tcross / s_tref2 + CudaDevicePtr s_term; + CudaDevicePtr s_parity; + CudaDevicePtr s_gperm, s_gstart; + CudaDevicePtr s_perm[3], s_seg_start[3]; + CudaDevicePtr s_A, s_sw, s_swI, s_tcross, s_tref2, s_cross, s_ref2; + CudaDevicePtr s_w_Is, s_w_Iref, s_Iref; // per term of a subset, in segment order }; // Set the device this instance's memory lives on for the duration of a call, and put the caller's @@ -1460,3 +1566,88 @@ void RotationScaleMergeGPU::SetFullsCorr(const float *corr) { CopyAndWait(d.f_corr.get(), corr, size_t(d.n_fulls) * sizeof(float), cudaMemcpyHostToDevice, impl_->s(), "upload f_corr"); } + +void RotationScaleMergeGPU::SurfaceSetTerms(int n_terms, const SurfaceTerm *term, const uint8_t *parity, + int n_groups, const int32_t *gperm, const int32_t *gstart, + int ncell) { + DeviceGuard guard(impl_->device, impl_->available); + auto &d = *impl_; + d.s_ncell = ncell; + d.s_n_groups = n_groups; + d.Upload(d.s_term, term, n_terms); + d.Upload(d.s_parity, parity, n_terms); + d.Upload(d.s_gperm, gperm, n_terms); + d.Upload(d.s_gstart, gstart, n_groups + 1); + d.s_A = d.Alloc(std::max(1, ncell)); + d.s_cross = d.Alloc(std::max(1, ncell)); + d.s_ref2 = d.Alloc(std::max(1, ncell)); + d.s_sw = d.Alloc(std::max(1, n_groups)); + d.s_swI = d.Alloc(std::max(1, n_groups)); + d.s_w_Is = d.Alloc(std::max(1, n_terms)); + d.s_w_Iref = d.Alloc(std::max(1, n_terms)); + d.s_Iref = d.Alloc(std::max(1, n_terms)); + for (int &nb : d.s_n_blocks) nb = 0; +} + +void RotationScaleMergeGPU::SurfaceSetSubset(int subset, int n_blocks, const int32_t *perm, + const int32_t *seg_start) { + DeviceGuard guard(impl_->device, impl_->available); + auto &d = *impl_; + const int n_seg = n_blocks * d.s_ncell; + d.s_n_sel[subset] = n_blocks > 0 ? seg_start[n_seg] : 0; + d.Upload(d.s_perm[subset], perm, d.s_n_sel[subset]); + d.Upload(d.s_seg_start[subset], seg_start, n_seg + 1); + d.s_n_blocks[subset] = n_blocks; + if (size_t(n_seg) > d.s_slots) { + d.s_tcross = d.Alloc(n_seg); + d.s_tref2 = d.Alloc(n_seg); + d.s_slots = n_seg; + } +} + +void RotationScaleMergeGPU::SurfaceReference(int parity, const double *A) { + DeviceGuard guard(impl_->device, impl_->available); + auto &d = *impl_; + CopyAndWait(d.s_A.get(), A, size_t(d.s_ncell) * sizeof(double), cudaMemcpyHostToDevice, impl_->s(), + "upload surface"); + const int grp_blocks = std::min(65535, (d.s_n_groups + BLK - 1) / BLK); + if (grp_blocks > 0) + SurfaceReferenceKernel<<s()>>>(d.s_n_groups, parity, d.s_gperm.get(), + d.s_gstart.get(), d.s_term.get(), d.s_parity.get(), d.s_A.get(), d.s_sw.get(), d.s_swI.get()); + CudaCheck(cudaGetLastError(), "surface reference launch"); + CudaCheck(cudaStreamSynchronize(impl_->s()), "surface reference sync"); +} + +void RotationScaleMergeGPU::SurfaceGetReference(double *sw, double *swI) const { + DeviceGuard guard(impl_->device, impl_->available); + auto &d = *impl_; + const size_t bytes = size_t(d.s_n_groups) * sizeof(double); + CopyAndWait(sw, d.s_sw.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "dl surface sw"); + CopyAndWait(swI, d.s_swI.get(), bytes, cudaMemcpyDeviceToHost, impl_->s(), "dl surface swI"); +} + +void RotationScaleMergeGPU::SurfaceFitSums(int subset, double *cross, double *ref2) { + DeviceGuard guard(impl_->device, impl_->available); + auto &d = *impl_; + const int ncell = d.s_ncell, nb = d.s_n_blocks[subset], n_seg = nb * ncell; + if (nb == 0) { + std::fill(cross, cross + ncell, 0.0); + std::fill(ref2, ref2 + ncell, 0.0); + return; + } + SurfaceFitTermKernel<<s()>>>( + d.s_n_sel[subset], d.s_perm[subset].get(), d.s_term.get(), d.s_A.get(), d.s_sw.get(), d.s_swI.get(), + d.s_w_Is.get(), d.s_w_Iref.get(), d.s_Iref.get()); + CudaCheck(cudaGetLastError(), "surface fit term launch"); + SurfaceFitSegmentKernel<<s()>>>(n_seg, + d.s_seg_start[subset].get(), d.s_w_Is.get(), d.s_w_Iref.get(), d.s_Iref.get(), + d.s_tcross.get(), d.s_tref2.get()); + CudaCheck(cudaGetLastError(), "surface fit segment launch"); + SurfaceFitCellKernel<<s()>>>(nb, ncell, + d.s_tcross.get(), d.s_tref2.get(), d.s_cross.get(), d.s_ref2.get()); + CudaCheck(cudaGetLastError(), "surface cell sum launch"); + CopyAndWait(cross, d.s_cross.get(), size_t(ncell) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), + "dl surface cross"); + CopyAndWait(ref2, d.s_ref2.get(), size_t(ncell) * sizeof(double), cudaMemcpyDeviceToHost, impl_->s(), + "dl surface ref2"); +} diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.h b/image_analysis/scale_merge/RotationScaleMergeGPU.h index 730a79add..766e2f640 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.h +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.h @@ -192,6 +192,34 @@ public: // Download the fulls' working corr (length = n_fulls), valid after ScaleFulls. void GetFullsCorr(float *corr) const; + // --- correction-surface fit (RotationScaleMerge::ApplyCellSurface) --- + // The two passes every round of the fit makes over its terms, on the device; the host keeps the + // per-cell step, the gauge and the cross-validation. Both sums are formed in exactly the order the + // host forms them, so the fitted surface is the host's to the last bit (see the kernels). + + // One observation as the surface fit sees it: the host's term, uploaded as it stands. + struct SurfaceTerm { float I, sigma, corr, d; int32_t cell, group; }; + + // The terms (in fulls order) with each one's frame parity, the ASU-group CSR over them (gperm lists + // the terms of group g at [gstart[g], gstart[g+1]), in fulls order) and the cell count. + void SurfaceSetTerms(int n_terms, const SurfaceTerm *term, const uint8_t *parity, + int n_groups, const int32_t *gperm, const int32_t *gstart, int ncell); + + // One subset of the terms (0 = even frames, 1 = odd, 2 = all), cut into the host's n_blocks + // reduction blocks and ordered within each block by cell, keeping term order inside a cell: + // block b, cell c is perm[seg_start[b * ncell + c], seg_start[b * ncell + c + 1]). + void SurfaceSetSubset(int subset, int n_blocks, const int32_t *perm, const int32_t *seg_start); + + // The per-group reference sums sw / swI over the terms of frame parity `parity` (< 0 = all) with + // the surface A (length ncell) applied. They stay on the device for SurfaceFitSums; + // SurfaceGetReference downloads them (length n_groups each). + void SurfaceReference(int parity, const double *A); + void SurfaceGetReference(double *sw, double *swI) const; + + // The fit's per-cell sums over one subset against the last SurfaceReference and its A: + // cross = sum w Is Iref and ref2 = sum w Iref^2 (length ncell each). + void SurfaceFitSums(int subset, double *cross, double *ref2); + private: struct Impl; std::unique_ptr impl_; diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 799c06c0d..c6d6df0bb 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -148,6 +148,7 @@ ADD_EXECUTABLE(jfjoch_test AnisotropyAnalysisTest.cpp ModelScalingTest.cpp ModelScaleGPUTest.cpp + CorrectionSurfaceGPUTest.cpp TwinningAnalysisTest.cpp TranslationalNCSTest.cpp RfreeFlagsTest.cpp diff --git a/tests/CorrectionSurfaceGPUTest.cpp b/tests/CorrectionSurfaceGPUTest.cpp new file mode 100644 index 000000000..bfe767db0 --- /dev/null +++ b/tests/CorrectionSurfaceGPUTest.cpp @@ -0,0 +1,167 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#include +#include "../common/CUDAWrapper.h" + +#ifdef JFJOCH_USE_CUDA + +#include +#include +#include +#include + +#include "../common/ParallelFor.h" +#include "../image_analysis/scale_merge/RotationScaleMergeGPU.h" + +namespace { + +using Term = RotationScaleMergeGPU::SurfaceTerm; + +// The roundings RotationScaleMerge::ApplyCellSurface makes on the host (x86-64-v3 build), spelled out: +// a volatile result is rounded on its own and never fused into the next operation, std::fma is fused. +double Mul(double a, double b) { volatile double r = a * b; return r; } +double Add(double a, double b) { volatile double r = a + b; return r; } + +constexpr int SURFACE_BLOCK = 32768; // ApplyCellSurface's reduction block + +struct Surface { + int n_groups = 0, ncell = 0; + std::vector term; + std::vector parity; + std::vector gperm, gstart; + std::vector sel[3]; // even, odd, all - in term order +}; + +Surface MakeSurface(int n_terms, int n_groups, int ncell, uint32_t seed) { + std::mt19937 rng(seed); + std::uniform_int_distribution group(0, n_groups - 1), cell(0, ncell - 1), bit(0, 1); + std::uniform_real_distribution u(0.0f, 1.0f); + Surface s; + s.n_groups = n_groups; + s.ncell = ncell; + for (int i = 0; i < n_terms; ++i) { + // Negative intensities, and now and then a sigma of zero: both reach the host sums as they are. + const float I = 1000.0f * u(rng) - 100.0f; + const float sigma = (i % 997 == 0) ? 0.0f : 1.0f + 30.0f * u(rng); + // Every 50th group gets no terms at all, so its reference is empty. + int g = group(rng); + if (g % 50 == 0) g = (g + 1) % n_groups; + s.term.push_back({I, sigma, 0.5f + u(rng), 1.0f + 3.0f * u(rng), cell(rng), g}); + s.parity.push_back(static_cast(bit(rng))); + s.sel[s.parity.back()].push_back(i); + s.sel[2].push_back(i); + } + s.gstart.assign(n_groups + 1, 0); + for (const Term &t : s.term) ++s.gstart[t.group + 1]; + for (int g = 0; g < n_groups; ++g) s.gstart[g + 1] += s.gstart[g]; + s.gperm.resize(n_terms); + std::vector fill(s.gstart.begin(), s.gstart.end() - 1); + for (int i = 0; i < n_terms; ++i) s.gperm[fill[s.term[i].group]++] = i; + return s; +} + +void HostReference(const Surface &s, int parity, const std::vector &A, + std::vector &sw, std::vector &swI) { + sw.assign(s.n_groups, 0.0); + swI.assign(s.n_groups, 0.0); + for (int g = 0; g < s.n_groups; ++g) { + double s_w = 0.0, s_wI = 0.0; + for (int k = s.gstart[g]; k < s.gstart[g + 1]; ++k) { + const int i = s.gperm[k]; + if (parity >= 0 && s.parity[i] != parity) continue; + const Term &t = s.term[i]; + const double a = A[t.cell]; + const double Is = Mul(Mul(t.I, t.corr), a), sc = Mul(Mul(t.sigma, t.corr), a); + const double w = 1.0 / Mul(sc, sc); + s_w = Add(s_w, w); + s_wI = parity >= 0 ? std::fma(Is, w, s_wI) : Add(s_wI, Mul(Is, w)); + } + sw[g] = s_w; + swI[g] = s_wI; + } +} + +void HostFitSums(const Surface &s, const std::vector &sel, const std::vector &A, + const std::vector &sw, const std::vector &swI, + std::vector &cross, std::vector &ref2) { + cross.assign(s.ncell, 0.0); + ref2.assign(s.ncell, 0.0); + const int n = static_cast(sel.size()), nb = ReductionBlocks(n, SURFACE_BLOCK); + for (int b = 0; b < nb; ++b) { + std::vector xcross(s.ncell, 0.0), xref2(s.ncell, 0.0); + const int lo = static_cast(int64_t(n) * b / nb), hi = static_cast(int64_t(n) * (b + 1) / nb); + for (int k = lo; k < hi; ++k) { + const Term &t = s.term[sel[k]]; + if (sw[t.group] <= 0.0) continue; + const double Iref = swI[t.group] / sw[t.group], a = A[t.cell]; + const double Is = Mul(Mul(t.I, t.corr), a), sc = Mul(Mul(t.sigma, t.corr), a); + if (!std::isfinite(Iref) || Iref <= 0.0 || !(sc > 0.0)) continue; + const double w = 1.0 / Mul(sc, sc); + xcross[t.cell] = std::fma(Mul(w, Is), Iref, xcross[t.cell]); + xref2[t.cell] = std::fma(Mul(w, Iref), Iref, xref2[t.cell]); + } + for (int c = 0; c < s.ncell; ++c) { + cross[c] = Add(cross[c], xcross[c]); + ref2[c] = Add(ref2[c], xref2[c]); + } + } +} + +// The device side of one subset: ApplyCellSurface's per-block counting sort by cell. +void UploadSubset(RotationScaleMergeGPU &gpu, const Surface &s, int id, const std::vector &sel) { + const int n = static_cast(sel.size()), nb = ReductionBlocks(n, SURFACE_BLOCK); + std::vector perm(n), seg_start(static_cast(nb) * s.ncell + 1, n); + for (int b = 0; b < nb; ++b) { + const int lo = static_cast(int64_t(n) * b / nb), hi = static_cast(int64_t(n) * (b + 1) / nb); + std::vector pos(s.ncell + 1, 0); + for (int k = lo; k < hi; ++k) ++pos[s.term[sel[k]].cell + 1]; + for (int c = 0; c < s.ncell; ++c) pos[c + 1] += pos[c]; + for (int c = 0; c < s.ncell; ++c) seg_start[size_t(b) * s.ncell + c] = lo + pos[c]; + for (int k = lo; k < hi; ++k) perm[lo + pos[s.term[sel[k]].cell]++] = sel[k]; + } + gpu.SurfaceSetSubset(id, nb, perm.data(), seg_start.data()); +} + +bool SameBits(const std::vector &a, const std::vector &b) { + return a.size() == b.size() && std::memcmp(a.data(), b.data(), a.size() * sizeof(double)) == 0; +} + +} // namespace + +TEST_CASE("CorrectionSurfaceGPU_SumsMatchHostBitForBit", "[RotationScale][gpu]") { + if (get_gpu_count() == 0) + SKIP("No GPU"); + // Enough terms for several reduction blocks in every subset. + const Surface s = MakeSurface(250000, 4000, 144, 7); + RotationScaleMergeGPU gpu; + REQUIRE(gpu.Available()); + gpu.SurfaceSetTerms(static_cast(s.term.size()), s.term.data(), s.parity.data(), s.n_groups, + s.gperm.data(), s.gstart.data(), s.ncell); + for (int id = 0; id < 3; ++id) + UploadSubset(gpu, s, id, s.sel[id]); + + std::mt19937 rng(11); + std::uniform_real_distribution u(0.7, 1.4); + std::vector A(s.ncell); + for (double &a : A) a = u(rng); + + for (int parity : {0, 1, -1}) { + const int id = parity < 0 ? 2 : parity; + std::vector sw, swI, cross, ref2; + HostReference(s, parity, A, sw, swI); + HostFitSums(s, s.sel[id], A, sw, swI, cross, ref2); + REQUIRE(ReductionBlocks(static_cast(s.sel[id].size()), SURFACE_BLOCK) > 1); + + gpu.SurfaceReference(parity, A.data()); + std::vector dsw(s.n_groups), dswI(s.n_groups), dcross(s.ncell), dref2(s.ncell); + gpu.SurfaceGetReference(dsw.data(), dswI.data()); + gpu.SurfaceFitSums(id, dcross.data(), dref2.data()); + CHECK(SameBits(sw, dsw)); + CHECK(SameBits(swI, dswI)); + CHECK(SameBits(cross, dcross)); + CHECK(SameBits(ref2, dref2)); + } +} + +#endif -- 2.54.0 From 20dbf598be20490c4cc7a3067514b104feaed839 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 20:08:00 +0200 Subject: [PATCH 14/97] Battery: record host hardware in the run manifest, show it in the report Collect CPU model, nominal/base frequency (with its source), physical core/thread/socket counts, RAM and GPU name(s)/memory at run time (Linux via /proc, /sys/.../cpufreq, nvidia-smi; a short best-effort on macOS via sysctl), store it under manifest.json's new "hardware" key, and render it as a line in the report header. Runs from before this change have no such key; the report prints "not recorded" for them. Co-Authored-By: Claude Opus 5.5 (1M context) --- tools/battery/README.md | 5 ++- tools/battery/battery.py | 97 +++++++++++++++++++++++++++++++++++++++- tools/battery/report.py | 14 ++++++ 3 files changed, 113 insertions(+), 3 deletions(-) diff --git a/tools/battery/README.md b/tools/battery/README.md index 0ce5abcd1..eae047c93 100644 --- a/tools/battery/README.md +++ b/tools/battery/README.md @@ -334,7 +334,7 @@ read-only when the run completes. | path | content | |---|---| | `bin/rugnux` | the exact binary that ran (sha256 in the manifest) | -| `manifest.json` | the label, arms, subset, options, binary version, build flags and source HEAD, runner git and dirty flag, host, pid, start and end, `complete` / `aborted`, `gpu_shared`, and every set with its resolved input, reference and tags | +| `manifest.json` | the label, arms, subset, options, binary version, build flags and source HEAD, runner git and dirty flag, host, hardware (CPU model and nominal frequency, physical/thread/socket counts, RAM, GPUs - empty on runs from before this was recorded), pid, start and end, `complete` / `aborted`, `gpu_shared`, and every set with its resolved input, reference and tags | | `results.json` | one row per set (the schema is below) | | `report.md`, `report.html` | the report (`report_PRIVATE.*` for a private run) | | `work///` | `run.log`, `p_report.txt`, `p.mtz`, `p.cif`, ... and `model_check/` | @@ -403,7 +403,8 @@ was run `--unforced`), so its forced XDS-arm rows are left out the same way. Every run writes the same report as Markdown and as a self-contained HTML page (inline CSS and SVG, no external files): -- a header with the binary, build flags, host, subset, each arm's command and whether timing counts; +- a header with the binary, build flags, host, hardware (CPU, cores/threads/sockets, RAM, GPUs - + `not recorded` on an older run), subset, each arm's command and whether timing counts; - a **summary per arm** (pass, of which accepted on an alternative reference / fail / unscored / not run, pass rate, median resolution gain, ISa, R_meas, time) with a verdict bar chart, all from rugnux's own run, and an **Accepted alternatives** section naming those rows and why; diff --git a/tools/battery/battery.py b/tools/battery/battery.py index 6844b076b..ed1098233 100644 --- a/tools/battery/battery.py +++ b/tools/battery/battery.py @@ -171,6 +171,100 @@ def gpu_others(): return len([x for x in out.split() if x.strip().isdigit()]) +def gpu_hardware(): + """[{"name", "memory"}] from nvidia-smi, [] if there are none or it is absent.""" + try: + out = subprocess.run(["nvidia-smi", "--query-gpu=name,memory.total", "--format=csv,noheader"], + capture_output=True, text=True, timeout=30).stdout + except (OSError, subprocess.TimeoutExpired): + return [] + gpus = [] + for line in out.splitlines(): + parts = [p.strip() for p in line.split(",")] + if len(parts) == 2: + gpus.append({"name": parts[0], "memory": parts[1]}) + return gpus + + +def linux_hardware(): + blocks = [] + try: + blocks = [b for b in open("/proc/cpuinfo").read().split("\n\n") if b.strip()] + except OSError: + pass + model = None + sockets, cores = set(), set() + for b in blocks: + m = re.search(r"model name\s*:\s*(.+)", b) + if m and model is None: + model = m.group(1).strip() + p, c = re.search(r"physical id\s*:\s*(\d+)", b), re.search(r"core id\s*:\s*(\d+)", b) + if p and c: + sockets.add(p.group(1)) + cores.add((p.group(1), c.group(1))) + + freq_ghz, freq_source = None, "unknown" + m = re.search(r"@\s*([\d.]+)\s*GHz", model or "") + if m: + freq_ghz, freq_source = float(m.group(1)), "model name" + else: + for name in ("base_frequency", "cpuinfo_max_freq"): + try: + freq_ghz = int(open(f"/sys/devices/system/cpu/cpu0/cpufreq/{name}").read()) / 1e6 + freq_source = name + break + except (OSError, ValueError): + continue + + mem_gib = None + try: + for line in open("/proc/meminfo"): + if line.startswith("MemTotal:"): + mem_gib = int(line.split()[1]) / 1024 / 1024 # kiB -> GiB + break + except OSError: + pass + + return {"cpu_model": model or "unknown", "cpu_base_ghz": freq_ghz, "cpu_base_source": freq_source, + "sockets": len(sockets) or None, "physical_cores": len(cores) or None, + "threads": os.cpu_count(), "memory_gib": mem_gib, "gpus": gpu_hardware()} + + +def macos_hardware(): + def sysctl(key): + r = subprocess.run(["sysctl", "-n", key], capture_output=True, text=True) + return r.stdout.strip() if r.returncode == 0 else None + + model = sysctl("machdep.cpu.brand_string") + freq_ghz, freq_source = None, "unknown" + m = re.search(r"@\s*([\d.]+)\s*GHz", model or "") + if m: + freq_ghz, freq_source = float(m.group(1)), "model name" + cores = sysctl("hw.physicalcpu") + p_cores, e_cores = sysctl("hw.perflevel0.physicalcpu"), sysctl("hw.perflevel1.physicalcpu") + if p_cores and e_cores: + cores = f"{p_cores}P+{e_cores}E" + mem = sysctl("hw.memsize") + return {"cpu_model": model or "unknown", "cpu_base_ghz": freq_ghz, "cpu_base_source": freq_source, + "sockets": int(sysctl("hw.packages") or 1), "physical_cores": cores, + "threads": int(sysctl("hw.logicalcpu") or os.cpu_count()), + "memory_gib": int(mem) / 1024 ** 3 if mem else None, "gpus": []} + + +def hardware_info(): + """The run host's hardware, for the manifest: CPU model, nominal/base frequency (and where it + came from), physical cores, threads, sockets, memory, GPUs. Linux is detailed; macOS is a short + best-effort; anywhere else this is just empty, and the report says so.""" + try: + if sys.platform == "darwin": + return macos_hardware() + if sys.platform.startswith("linux"): + return linux_hardware() + except OSError: + pass + return {} + + # ------------------------------------------------------------------ running def command(binary, e, opts, model=None): @@ -377,7 +471,8 @@ def run_sets(site, sets, arms, binary_src, build_dir, label, opts, subset, basel "binary": info, "runner_git": git(HERE, "rev-parse", "--short", "HEAD") or "unknown", "runner_dirty": bool(git(HERE, "status", "--porcelain", "--", ".")), - "host": socket.gethostname(), "pid": os.getpid(), "start": datetime.datetime.now().isoformat(timespec="seconds"), + "host": socket.gethostname(), "hardware": hardware_info(), "pid": os.getpid(), + "start": datetime.datetime.now().isoformat(timespec="seconds"), "end": None, "complete": False, "results_schema": report.RESULTS_SCHEMA, "options": {k: v for k, v in opts.items() if k != "runs_root"}, "gpu_shared": bool(opts["gpulock"]), "sets": sets} write_json(os.path.join(run_dir, "manifest.json"), man) diff --git a/tools/battery/report.py b/tools/battery/report.py index 4ba1811c7..2cc9e404c 100644 --- a/tools/battery/report.py +++ b/tools/battery/report.py @@ -288,6 +288,19 @@ def status(r): return "accepted_alt" if r.get("cause") == "accepted_alternative" else r["verdict"] +def hardware_line(man): + """One line for the report header; old runs have no "hardware" key.""" + hw = man.get("hardware") + if not hw: + return "hardware: not recorded" + freq = f"{hw['cpu_base_ghz']:.2f} GHz nominal ({hw['cpu_base_source']})" if hw.get("cpu_base_ghz") else \ + f"nominal frequency unknown ({hw.get('cpu_base_source', 'unknown')})" + cores = f"{f(hw.get('physical_cores'), '{}')}C/{hw.get('threads') or '?'}T, {f(hw.get('sockets'), '{}')} socket(s)" + mem = f"{hw['memory_gib']:.0f} GiB RAM" if hw.get("memory_gib") else "RAM unknown" + gpus = ", ".join(f"{g['name']} ({g['memory']})" for g in hw.get("gpus") or []) or "none" + return f"hardware: {hw.get('cpu_model', 'unknown')}, {freq}, {cores}, {mem}, GPU: {gpus}" + + def rate(rs): s = scored(rs) n = sum(r["verdict"] == "pass" for r in s) @@ -316,6 +329,7 @@ def build(run_dir, baseline=None, allow_incomplete=False): doc.ul([ f"rugnux {b.get('version', '?')}, sha256 {b['sha256'][:16]}, build flags: {b.get('flags') or 'unknown'}", f"runner {man['runner_git']}, host {man['host']}, {man['start']} -> {man.get('end') or 'NOT FINISHED'}", + hardware_line(man), f"arms: {', '.join(man['arms'])}; {len(man['sets'])} sets" + (f" (subset: {man['subset']})" if man.get("subset") else " (full arm)"), "timing: NOT a reference - the GPU was shared" if man.get("gpu_shared") -- 2.54.0 From 563254cd6c0878a7f2c99ccc09d032e097ea7be9 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 20:10:29 +0200 Subject: [PATCH 15/97] Battery: fix hardware nominal frequency, separate it from max boost cpuinfo_max_freq is the highest boost state, not the base clock - mislabeling it "nominal" understated the gap between the two on an AMD box using acpi-cpufreq (this machine: 5.08 GHz boost vs 3.40 GHz base). Prefer, in order: the model-name string, cpufreq's own base_frequency (intel_pstate, kHz), ACPI CPPC's nominal_freq (MHz; what acpi-cpufreq exposes instead), else "unknown". cpuinfo_max_freq is now reported alongside, separately, as "max boost", never folded into "nominal". Co-Authored-By: Claude Opus 5.5 (1M context) --- tools/battery/battery.py | 34 ++++++++++++++++++++++++---------- tools/battery/report.py | 7 +++++-- 2 files changed, 29 insertions(+), 12 deletions(-) diff --git a/tools/battery/battery.py b/tools/battery/battery.py index ed1098233..87742a9a7 100644 --- a/tools/battery/battery.py +++ b/tools/battery/battery.py @@ -203,18 +203,30 @@ def linux_hardware(): sockets.add(p.group(1)) cores.add((p.group(1), c.group(1))) + # Nominal (base) clock, never the boost clock: the model-name string, else cpufreq's own + # base_frequency (intel_pstate, kHz), else ACPI CPPC's nominal_freq (MHz; what AMD's + # acpi-cpufreq exposes instead - cpuinfo_max_freq on those boxes is the boost clock, not this). freq_ghz, freq_source = None, "unknown" m = re.search(r"@\s*([\d.]+)\s*GHz", model or "") if m: freq_ghz, freq_source = float(m.group(1)), "model name" else: - for name in ("base_frequency", "cpuinfo_max_freq"): + try: + freq_ghz = int(open("/sys/devices/system/cpu/cpu0/cpufreq/base_frequency").read()) / 1e6 + freq_source = "base_frequency" + except (OSError, ValueError): try: - freq_ghz = int(open(f"/sys/devices/system/cpu/cpu0/cpufreq/{name}").read()) / 1e6 - freq_source = name - break + freq_ghz = int(open("/sys/devices/system/cpu/cpu0/acpi_cppc/nominal_freq").read()) / 1e3 + freq_source = "acpi_cppc" except (OSError, ValueError): - continue + pass + + # Reported separately, and never as "nominal": cpuinfo_max_freq is the highest boost state. + max_boost_ghz = None + try: + max_boost_ghz = int(open("/sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq").read()) / 1e6 + except (OSError, ValueError): + pass mem_gib = None try: @@ -226,8 +238,9 @@ def linux_hardware(): pass return {"cpu_model": model or "unknown", "cpu_base_ghz": freq_ghz, "cpu_base_source": freq_source, - "sockets": len(sockets) or None, "physical_cores": len(cores) or None, - "threads": os.cpu_count(), "memory_gib": mem_gib, "gpus": gpu_hardware()} + "cpu_max_boost_ghz": max_boost_ghz, "sockets": len(sockets) or None, + "physical_cores": len(cores) or None, "threads": os.cpu_count(), "memory_gib": mem_gib, + "gpus": gpu_hardware()} def macos_hardware(): @@ -246,15 +259,16 @@ def macos_hardware(): cores = f"{p_cores}P+{e_cores}E" mem = sysctl("hw.memsize") return {"cpu_model": model or "unknown", "cpu_base_ghz": freq_ghz, "cpu_base_source": freq_source, - "sockets": int(sysctl("hw.packages") or 1), "physical_cores": cores, + "cpu_max_boost_ghz": None, "sockets": int(sysctl("hw.packages") or 1), "physical_cores": cores, "threads": int(sysctl("hw.logicalcpu") or os.cpu_count()), "memory_gib": int(mem) / 1024 ** 3 if mem else None, "gpus": []} def hardware_info(): """The run host's hardware, for the manifest: CPU model, nominal/base frequency (and where it - came from), physical cores, threads, sockets, memory, GPUs. Linux is detailed; macOS is a short - best-effort; anywhere else this is just empty, and the report says so.""" + came from) plus max boost frequency separately, physical cores, threads, sockets, memory, GPUs. + Linux is detailed; macOS is a short best-effort; anywhere else this is just empty, and the + report says so.""" try: if sys.platform == "darwin": return macos_hardware() diff --git a/tools/battery/report.py b/tools/battery/report.py index 2cc9e404c..69c0cdff9 100644 --- a/tools/battery/report.py +++ b/tools/battery/report.py @@ -293,8 +293,11 @@ def hardware_line(man): hw = man.get("hardware") if not hw: return "hardware: not recorded" - freq = f"{hw['cpu_base_ghz']:.2f} GHz nominal ({hw['cpu_base_source']})" if hw.get("cpu_base_ghz") else \ - f"nominal frequency unknown ({hw.get('cpu_base_source', 'unknown')})" + freq_bits = [f"{hw['cpu_base_ghz']:.2f} GHz nominal ({hw['cpu_base_source']})" if hw.get("cpu_base_ghz") + else "nominal frequency unknown"] + if hw.get("cpu_max_boost_ghz"): + freq_bits.append(f"{hw['cpu_max_boost_ghz']:.2f} GHz max boost") + freq = ", ".join(freq_bits) cores = f"{f(hw.get('physical_cores'), '{}')}C/{hw.get('threads') or '?'}T, {f(hw.get('sockets'), '{}')} socket(s)" mem = f"{hw['memory_gib']:.0f} GiB RAM" if hw.get("memory_gib") else "RAM unknown" gpus = ", ".join(f"{g['name']} ({g['memory']})" for g in hw.get("gpus") or []) or "none" -- 2.54.0 From 0f728ecabf08df5d59742bfbcb400ab07fa3ffca Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 20:32:59 +0200 Subject: [PATCH 16/97] rugnux: make the tail's two P1 merges on the run's own engine again The all-observation arm of the space-group search and the P1 cross-check were made on a second RotationScaleMerge engine beside the run's own (871347b7a). That engine holds a second device copy of every observation, and on the largest sets the two no longer fit a 16 GB card: 8a1a, 8qaw and 8tyy (55-125 M partial observations) stopped with an out-of-memory error in scaling. Measured on a quiet box the engine bought 0.7 s (cytc) and 1.0 s (thau) of tail and nothing on myob, which does not justify a memory budget, so it is removed and both merges run on rsm in sequence, as before 871347b7a. Kept from 871347b7a: the GPU scaling's own non-blocking stream, the cross-check not writing per-frame G/CC/mosaicity back (the per-image table still describes the merge that was written), the restored scaling iteration counts, and the anisotropy analysis beside the other analyses. p.mtz byte-identical to before on myob/cytc/thau, GPU and CPU builds; p_plot.txt unchanged apart from the GPU bkg column. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- .../scale_merge/RotationScaleMerge.h | 3 +- .../scale_merge/RotationScaleMergeGPU.cu | 5 +- rugnux/Rugnux.cpp | 98 +------------------ 3 files changed, 5 insertions(+), 101 deletions(-) diff --git a/image_analysis/scale_merge/RotationScaleMerge.h b/image_analysis/scale_merge/RotationScaleMerge.h index 79278560f..6b5a81216 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.h +++ b/image_analysis/scale_merge/RotationScaleMerge.h @@ -121,8 +121,7 @@ public: // Whether Run() writes the per-frame G / CC / mosaicity back onto the outcomes (on by default). Off // for a merge that is not the run's answer - the P1 cross-check - so the per-image table and the - // unmerged MTZ describe the merge that was written, and so an engine merging beside another one - // does not write the outcomes they share. + // unmerged MTZ describe the merge that was written. void SetWriteBackPerFrameScale(bool on) { write_back_per_frame_scale = on; } // Override the high-resolution cut for the next Run() - used to gate the de-novo P1 search pass at diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.cu b/image_analysis/scale_merge/RotationScaleMergeGPU.cu index 66aeabacb..8a0e97f85 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.cu +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.cu @@ -19,9 +19,8 @@ namespace { constexpr int MIN_REFLECTIONS = 20; // Every kernel and copy here is queued on the instance's own stream (Impl::stream), not on the - // legacy NULL stream: two merges made side by side - the run's and the one it makes ahead of time - - // and the image analysis of a probe pass beside them would otherwise wait for each other's work at - // every launch and every synchronisation. As in BeamCenterFFTGPU no buffer comes from the pool. A + // legacy NULL stream: the merge and the image analysis of a probe pass beside it would otherwise + // wait for each other's work at every launch and every synchronisation. As in BeamCenterFFTGPU no buffer comes from the pool. A // pooled buffer is freed with cudaFreeAsync on the thread's allocation stream, which is not // ordered after this one. Each entry point below waits for its own work before it returns, so no // free here has yet overtaken a read, but that holds only by that convention, and not at all for a diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 94a9325af..aed1270d0 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -6674,28 +6674,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const auto &rot_ss = experiment_.GetScalingSettings(); const bool is_rotation = experiment_.IsRotationIndexing(); // rotation indexing -> rotation scaling/merge std::optional rsm; - // How many times rsm has been ingested on this pass: a re-ingest follows a reindex or a cell - // change, after which a merge made from the first ingest is a merge of other indices. - int rsm_ingests = 0; - // Two P1 merges of this pass's integration made beside the rest of the tail, on an engine of - // their own: the all-observation arm of the space-group search (see there) and the P1 - // cross-check (see where it is written). Neither reads anything the search, the in-symmetry - // merge or the analyses decide, so neither has to wait for them. The engine is ingested together - // with rsm, before any merge writes per-frame values back onto the outcomes, so the two start - // from the same state, and it writes nothing back itself. Used only while rsm was ingested once - // (rsm_ingest); otherwise both merges are made on rsm, as before. - struct P1MergesAhead { - DiffractionExperiment x; - Logger log = Logger::Buffered(); // what the engine logs - Logger all_observations_log = Logger::Buffered(); // ...up to the all-observation merge - std::optional engine; - std::future ingested; - std::future all_observations; - std::future crosscheck; - bool crosscheck_made = false; - int rsm_ingest = 0; - }; - std::unique_ptr p1_ahead; std::optional prepass_postrefine_obs; // The rotation geometry post-refinement (see its call sites below for what it is for). const auto post_refine_geometry = [&] { @@ -6826,60 +6804,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Rotation scaling/merging (RotationScaleMerge) does not support " "wedge refinement"); - // The conditions of p1_crosscheck and of the all-observation arm below that are known here; - // the rest (a pass that turns out superseded, a search that finds no point group) only means - // a merge is made and not used. - const bool all_observations_ahead = !geometry_prepass && !experiment_.GetGemmiSpaceGroup().has_value() - && rot_ss.GetSearchMinZeta() > 0.0; - const bool crosscheck_ahead = !geometry_prepass && write_files && config_.write_merged - && config_.write_p1_crosscheck && result.consensus_cell - && (!experiment_.GetGemmiSpaceGroup().has_value() || indexer->GetPredictionCentring() == 'P'); - if ((all_observations_ahead || crosscheck_ahead) && config_.observation_dump_path.empty()) { - p1_ahead = std::make_unique(); - p1_ahead->x = experiment_; - p1_ahead->rsm_ingest = 1; - p1_ahead->crosscheck_made = crosscheck_ahead; - std::promise ingested; - std::promise all_observations; - p1_ahead->ingested = ingested.get_future(); - if (all_observations_ahead) - p1_ahead->all_observations = all_observations.get_future(); - p1_ahead->crosscheck = std::async(std::launch::async, - [&a = *p1_ahead, &outcomes = indexer->GetIntegrationOutcome(), - cell = result.consensus_cell, iter = static_cast(config_.scaling_iter), - nthreads = config_.nthreads, ingested = std::move(ingested), - all_observations = std::move(all_observations), all_observations_ahead, - crosscheck_ahead]() mutable -> RotationScaleMerge::Result { - try { - a.engine.emplace(a.x, outcomes, cell, iter, nthreads, a.log); - a.engine->SetWriteBackPerFrameScale(false); - a.engine->Ingest(); - } catch (...) { - ingested.set_exception(std::current_exception()); - throw; - } - ingested.set_value(); - // Ingested in the group rsm was, merged in P1 - as both merges on rsm are. - a.x.SpaceGroupNumber(1); - if (all_observations_ahead) { - try { - a.engine->SetSearchMinZeta(0.0); - auto merged = a.engine->Run(/*for_search=*/true, /*full_stats=*/true, - /*measure_cc_before_corrections=*/false); - a.all_observations_log = a.log; - a.log = Logger::Buffered(); - all_observations.set_value(std::move(merged)); - } catch (...) { - all_observations.set_exception(std::current_exception()); - throw; - } - } - if (!crosscheck_ahead) - return {}; - return a.engine->Run(/*for_search=*/false, /*full_stats=*/true, - /*measure_cc_before_corrections=*/false); - }); - } // A reference MTZ is allowed for rotation: it fixes the space group / cell (on the CLI) and // resolves the indexing ambiguity (below), but is NOT used to scale - the rotation merge stays // self-consistent. @@ -6887,9 +6811,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); - ++rsm_ingests; - if (p1_ahead) - p1_ahead->ingested.get(); } // The geometry pre-pass reads the per-image reflections exactly twice more: they were just @@ -7224,14 +7145,9 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // all-observation arm on both twin gates and promoted to R32 by the filtered one. So the // losing arm's refusal is logged and carried to the report below; the rule is unchanged. if (rsm && rsm->GetSearchMinZeta() > 0.0 && !sg_search.point_group_hm.empty()) { - std::optional ahead; - if (p1_ahead && p1_ahead->all_observations.valid() && p1_ahead->rsm_ingest == rsm_ingests) { - ahead = p1_ahead->all_observations.get(); - p1_ahead->all_observations_log.ReplayInto(logger); - } const double zeta = rsm->GetSearchMinZeta(); rsm->SetSearchMinZeta(0.0); - auto sm_all = scale_and_merge("P1, all observations", true, false, std::move(ahead)); + auto sm_all = scale_and_merge("P1, all observations", true); rsm->SetSearchMinZeta(zeta); merged_filtered_isa = sg_opts.merge_isa; // the filtered arm's, before it is replaced sg_opts.merge_isa = result.error_model_isa; // this arm's own error model @@ -7910,7 +7826,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); - ++rsm_ingests; } const auto &uc = *result.consensus_cell; logger.Info("{} names a cell of {}x the volume of its reference setting: the " @@ -8106,7 +8021,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b result.consensus_cell, static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); - ++rsm_ingests; } } } @@ -8134,7 +8048,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); - ++rsm_ingests; } } experiment_.SetSpaceGroup(sg); @@ -8468,7 +8381,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b static_cast(config_.scaling_iter), config_.nthreads, logger, config_.observation_dump_path); rsm->Ingest(); - ++rsm_ingests; phase("Re-merging in the reference's frame"); sm = scale_and_merge(moved->short_name(), false); const auto &uc = *result.consensus_cell; @@ -8964,8 +8876,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b && !geometry_prepass && !superseded && config_.write_p1_crosscheck && p1_integration_complete && is_rotation; std::optional p1_merged_early; - const bool p1_ahead_usable = p1_crosscheck && p1_ahead && p1_ahead->crosscheck_made - && p1_ahead->rsm_ingest == rsm_ingests; // Model validation runs BEFORE the reflection files are written, because it is what settles the // frame they are written in: the enantiomorph, which merged intensities cannot choose, and - @@ -9000,7 +8910,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // P1 merge afterwards, through merge_to_written, as it did when it was made below. The // validation's log lines are held and printed as one block once it is done. ModelValidationResult validation; - if (p1_crosscheck && rsm && !p1_ahead_usable) { + if (p1_crosscheck && rsm) { Logger held = Logger::Buffered(); auto pending = std::async(std::launch::async, validate, std::ref(held)); experiment_.SpaceGroupNumber(1); @@ -9202,10 +9112,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // Both the merge and the MTZ read the group from the experiment, so it is set for // the whole of it and restored after. experiment_.SpaceGroupNumber(1); - if (p1_ahead_usable) { - p1_merged_early = p1_ahead->crosscheck.get(); - p1_ahead->log.ReplayInto(logger); - } if (!p1_merged_early) rsm->SetWriteBackPerFrameScale(false); auto p1 = scale_and_merge("P1 cross-check", false, false, std::move(p1_merged_early)); -- 2.54.0 From 1bd39b0563ec3e35dda5219e6fc3fea92aa7f589 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 21:00:25 +0200 Subject: [PATCH 17/97] Space-group search: cut the P1 search merge where a monotone fit of drops under 1 The search merge was cut at the first thin shell whose mean fell under 1. On a merge whose is flat near 1 - its ISa has collapsed, as on a strongly absorbing garnet - the first single shell to dip is decided by noise: the same crystal cut at 1.19 A without -march and 0.85 A with -march=x86-64-v3, and the coarser cut left each glide zone fewer than 20 absences, so Ia-3d became unjudgeable and I4(1)32 was adopted. The cut now comes from a non-increasing (pool-adjacent-violators) fit of the shell means, which moves only as much as its input does; that garnet's profile never falls under 1, so its search sees the full range and adopts Ia-3d under both builds with identical candidate tables. On monotone profiles nothing changes: myob/cytc/thau x10sa keep their cuts and p.mtz byte for byte. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- rugnux/Rugnux.cpp | 34 +++++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 94a9325af..aaaa11d02 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -7133,11 +7133,39 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b [](const auto &a, const auto &b) { return a.first > b.first; }); // low -> high res const int bins = std::clamp(static_cast(rs.size() / 100), 1, 40); const size_t per = (rs.size() + bins - 1) / static_cast(bins); - for (size_t b = 0; b * per < rs.size(); ++b) { - const size_t lo = b * per, hi = std::min(rs.size(), lo + per); + // The shell means, then their best NON-INCREASING fit (pool adjacent violators, weighted + // by shell size). The cut is where the FIT drops under 1, not where the first shell + // does: on a merge whose is flat near 1 - its ISa has collapsed - the first + // single shell to dip under 1 is a coin toss that moved the cut from 0.85 to 1.19 A + // between two compiler flags on one crystal, and the coarser cut left its glide zones + // too few absences to be judged. The fit moves by as much as its input does. + std::vector mean, weight; + for (size_t lo = 0; lo < rs.size(); lo += per) { + const size_t hi = std::min(rs.size(), lo + per); double sum = 0.0; for (size_t j = lo; j < hi; ++j) sum += rs[j].second; - if (sum / static_cast(hi - lo) < 1.0) { + mean.push_back(sum / static_cast(hi - lo)); + weight.push_back(static_cast(hi - lo)); + } + std::vector block_mean, block_weight; + std::vector block_size; + for (size_t b = 0; b < mean.size(); ++b) { + block_mean.push_back(mean[b]); block_weight.push_back(weight[b]); block_size.push_back(1); + while (block_mean.size() > 1 && block_mean[block_mean.size() - 2] < block_mean.back()) { + const size_t k = block_mean.size() - 1; + const double w = block_weight[k - 1] + block_weight[k]; + block_mean[k - 1] = (block_mean[k - 1] * block_weight[k - 1] + block_mean[k] * block_weight[k]) / w; + block_weight[k - 1] = w; + block_size[k - 1] += block_size[k]; + block_mean.pop_back(); block_weight.pop_back(); block_size.pop_back(); + } + } + std::vector fit; + for (size_t k = 0; k < block_mean.size(); ++k) + fit.insert(fit.end(), block_size[k], block_mean[k]); + for (size_t b = 0; b < fit.size(); ++b) { + const size_t lo = b * per; + if (fit[b] < 1.0) { // Cut the noise-dominated high-res shells. When even the lowest-res shell fails, // keep that shell alone rather than abandoning the cut. The bound is absolute // while the merged I/sigma it tests saturates at the merge's own ISa (times the -- 2.54.0 From 445804fa43211064c0cc10932b448638030984ef Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 21:33:58 +0200 Subject: [PATCH 18/97] rugnux: carry pass 1's lattice into the refined pass; revert to pass 1 itself Two knife-edges let a sub-0.05 px change of the measured beam centre turn a P1 crystal (9qw8) from a pass into a merge with R_meas 60000% / ISa 0: - The refined-geometry pass re-indexes de novo and fell back to pass 1's lattice only when the re-index scored under the 1/6 validation floor. The long-axis rescue lifted a related C-centred 303 A cell to 41/60 - over the floor - while pass 1's lattice indexes 55/60 at the same geometry. Pass 1's lattice is now scored as a hypothesis of its own whenever the re-index found a DIFFERENT lattice (class + primitive volume within 2 %), and is integrated when it indexes more validation frames. The same lattice found again is kept as the re-index refined it, so ordinary crystals are untouched. - The two-pass quality guard's "going back to the header geometry" re-ran pass 1 de novo, a hypothesis nobody had judged: at the centre where pass 1 indexed 56/60 it found 5/60, flipped the axis sign and shipped garbage. It now forces pass 1's whole indexing result, which is what the guard preferred. 9qw8: P1 1.71 A on -march=x86-64-v3 and on no-march GPU builds (both failed before, one catastrophically). myob/cytc/thau p.mtz md5 unchanged (GPU). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- rugnux/Rugnux.cpp | 107 ++++++++++++++++++++++++++++------------------ 1 file changed, 66 insertions(+), 41 deletions(-) diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 94a9325af..74e05a89d 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -3097,6 +3097,13 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) { // This pass is the answer whatever it measures - the guard has had its one chance - // so it must not conclude from its own search merge that it is going to be re-run. quality_guard_pass1_.reset(); + // What the guard preferred is pass 1 - its lattice, orientation and axis at this very + // geometry - so that is what is re-integrated, not a fresh de-novo index. A de-novo + // re-index here is a new hypothesis nobody judged: measured, at the same centre where + // pass 1 indexed 56/60 validation frames it found 5/60, took the opposite axis sign and + // shipped a merge with R_meas in the hundreds of percent. + if (prepass_result_) + force_rotation_result_ = *prepass_result_; rerun_if_starved_ = true; auto redo = RunPipeline(observer, /*write_output=*/true, /*geometry_prepass=*/false); rerun_if_starved_ = false; @@ -3105,7 +3112,9 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) { redo.pass_count = pass2.pass_count + 1; redo.pass_decision = fmt::format( "header geometry re-adopted: the post-refined pass was worse ({})", worse); + // Still forced: the re-run at the fixed radius is the same pass. rerun_if_starved(redo, redo.pass_decision); + force_rotation_result_.reset(); return redo; } if (pass2.pass_decision.empty()) @@ -5475,55 +5484,71 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // frame in six, several times below the weakest crystal that still merges. It is checked AFTER // the long-axis rescue, so a metric that rescue recovers is never rejected on its pre-rescue // score. - if (best.score < static_cast(validation.size()) / 6) { - // The second (refined-geometry) pass re-indexes DE NOVO, and the post-refined geometry - - // fitted to the lattice pass 1 already found - can tip the blind FFT onto an axis harmonic - // of a long cell that indexes too few frames to clear this floor (measured: a tripled - // ~100 A c-axis at 7/60). Pass 1 found and integrated the true cell at the header geometry, - // so prefer its whole result rather than abort the run on a re-index only the refined - // geometry made fail. This is the floor-side companion of the supercell-collapse guard - // below: that one only reaches a pass-2 lattice that DID clear the floor, whereas here the - // harmonic fell BELOW it and the throw would run before the guard could ever see it. - if (!geometry_prepass && prepass_result_.has_value()) { - const auto &pc = prepass_result_->search_result.conventional.GetUnitCell(); + // The second (refined-geometry) pass re-indexes DE NOVO, and the post-refined geometry - fitted + // to the lattice pass 1 already found - can tip the blind FFT, or the long-axis rescue above, + // onto another lattice: an axis harmonic of a long cell (measured: a tripled ~100 A c-axis at + // 7/60), or a related centred cell that indexes fewer frames than pass 1's lattice does at + // this very geometry (measured: a C-centred 303 A cell at 41/60 against pass 1's triclinic + // cell at 60/60). Pass 1's lattice is therefore carried here as a hypothesis of its own and + // the two are judged on the same validation frames: where the re-index found a different + // lattice and indexes fewer frames than pass 1's, or too few to be this crystal's lattice at + // all, pass 1's whole result is integrated instead. The same lattice found again is kept as + // the re-index refined it at this geometry. This is the floor-side companion of the + // supercell-collapse guard in RunAllPasses, which only sees a pass-2 lattice that is LARGER. + if (!geometry_prepass && prepass_result_.has_value()) { + RotationIndexerResult pass1_lattice = *prepass_result_; + // Pass 1's result carries pass 1's goniometer; forcing it whole would put the + // uncorrected angles back (the same repair as the supercell re-run in RunAllPasses). + if (prepass_rotation_scale_ && pass1_lattice.axis) + pass1_lattice.axis = ScaleRotation(*pass1_lattice.axis, *prepass_rotation_scale_); + // Pass 1's lattice was FITTED at pass 1's detector distance, and this pass + // integrates at the post-refined one. Re-scoring it here is not the same as + // re-fitting it: a real-space cell is measured against the distance the spots were + // seen at, so carrying it across a distance change scales the whole cell by the + // ratio of the two. Measured on a 545 A axis over 225 -> 226 mm: 0.4 %, exactly + // 225/226, shipped as the run's answer. Scale it to the distance it will be used + // at; the orientation is unaffected. + const float d_fit = prepass_result_->geom.GetDetectorDistance_mm(); + const float d_use = experiment_.GetDetectorDistance_mm(); + const bool rescaled = d_fit > 0 && d_use > 0 && d_fit != d_use; + if (rescaled) { + const float f = d_use / d_fit; + const auto &l = pass1_lattice.lattice; + pass1_lattice.lattice = CrystalLattice(l.Vec0() * f, l.Vec1() * f, l.Vec2() * f); + } + const double pass1_vol = std::abs(pass1_lattice.lattice + .ToPrimitive(pass1_lattice.search_result.centering).CalcVolume()); + // Same lattice as in the beam-centre check: same class and primitive volumes within 2 %. + const bool same_lattice = best.result.has_value() + && best.result->search_result.centering == pass1_lattice.search_result.centering + && best.result->search_result.system == pass1_lattice.search_result.system + && std::abs(best.vol - pass1_vol) <= 0.02 * pass1_vol; + const int floor = static_cast(validation.size()) / 6; + const int pass1_score = (best.score < floor || !same_lattice) + ? count_indexed(*indexer, pass1_lattice) : -1; + if (best.score < floor || pass1_score > best.score) { + const auto &pc = pass1_lattice.search_result.conventional.GetUnitCell(); const auto &bc = best.result->search_result.conventional.GetUnitCell(); - logger.Warning("Two-pass: the refined-geometry re-index indexes only {}/{} validation " - "frames ({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f}) - too " - "few for this crystal's lattice; integrating with pass-1's lattice " - "instead ({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f})", + logger.Warning("Two-pass: the refined-geometry re-index indexes {}/{} validation frames " + "({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f}), pass-1's lattice " + "{}/{} at this geometry ({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} " + "{:.2f}) - integrating with pass-1's lattice", best.score, static_cast(validation.size()), best.result->search_result.centering, gemmi::crystal_system_str(best.result->search_result.system), bc.a, bc.b, bc.c, bc.alpha, bc.beta, bc.gamma, - prepass_result_->search_result.centering, - gemmi::crystal_system_str(prepass_result_->search_result.system), + pass1_score, static_cast(validation.size()), + pass1_lattice.search_result.centering, + gemmi::crystal_system_str(pass1_lattice.search_result.system), pc.a, pc.b, pc.c, pc.alpha, pc.beta, pc.gamma); - best.result = *prepass_result_; - // Pass 1's result carries pass 1's goniometer; forcing it whole would put the - // uncorrected angles back (the same repair as the supercell re-run in RunAllPasses). - if (prepass_rotation_scale_ && best.result->axis) - best.result->axis = ScaleRotation(*best.result->axis, *prepass_rotation_scale_); - // Pass 1's lattice was FITTED at pass 1's detector distance, and this pass - // integrates at the post-refined one. Re-scoring it here is not the same as - // re-fitting it: a real-space cell is measured against the distance the spots were - // seen at, so carrying it across a distance change scales the whole cell by the - // ratio of the two. Measured on a 545 A axis over 225 -> 226 mm: 0.4 %, exactly - // 225/226, shipped as the run's answer. Scale it to the distance it will be used - // at; the orientation is unaffected. - const float d_fit = prepass_result_->geom.GetDetectorDistance_mm(); - const float d_use = experiment_.GetDetectorDistance_mm(); - if (d_fit > 0 && d_use > 0 && d_fit != d_use) { - const float f = d_use / d_fit; - const auto &l = best.result->lattice; - best.result->lattice = CrystalLattice(l.Vec0() * f, l.Vec1() * f, l.Vec2() * f); + if (rescaled) logger.Info("Two-pass: pass-1's cell was fitted at {:.3f} mm and is used at " "{:.3f} mm - scaled by {:.6f} to the distance it is integrated at", - d_fit, d_use, f); - } - best.score = count_indexed(*indexer, *best.result); // re-score at the refined geometry - best.vol = std::abs(best.result->lattice - .ToPrimitive(best.result->search_result.centering).CalcVolume()); - best.name = "pass-1 lattice (refined-geometry re-index too sparse)"; + d_fit, d_use, d_use / d_fit); + best.result = std::move(pass1_lattice); + best.score = pass1_score; + best.vol = pass1_vol; + best.name = "pass-1 lattice (refined-geometry re-index found less)"; } } -- 2.54.0 From 359e749074502e1741c5262cff2217fd5223e7bc Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 21:34:10 +0200 Subject: [PATCH 19/97] viewer: subscripted space group and a pathology checklist in the merge window The space-group hero card (and the "indistinguishable" line) render the short Hermann-Mauguin symbol as rich text: screw axes subscripted, rotoinversions overlined (P2_1/c, P4_12_12, Fd-3m). It is built from gemmi's spaced symbol with short_name()'s monoclinic shortening applied; checked against all 564 gemmi settings to reduce to short_name() once the markup is stripped. The report's SUMMARY now also records its pathology rows as typed checks (ReportDocument::checks): PRESENT exactly when one of the row's codes is in PATHOLOGY_FLAGS, ABSENT when it was measured and did not fire, UNDETERMINED when it could not be measured. A flag no row stands for gets a check named after its code, so the list never shows less than PATHOLOGY_FLAGS. The text report is unchanged. The viewer builds the document once, renders the text from it and shows the checks phenix.xtriage-style: a green/red/grey light per pathology, the summary words and fired warnings in the tooltip. Co-Authored-By: Claude Opus 5.5 (1M context) --- rugnux/ReportDocument.h | 14 +++ rugnux/ResultReport.cpp | 77 +++++++++++++--- tests/ResultReportTest.cpp | 35 +++++++ viewer/JFJochProcessController.cpp | 10 +- viewer/JFJochProcessController.h | 6 +- viewer/windows/JFJochMergeStatsWindow.cpp | 92 ++++++++++++++++++- viewer/windows/JFJochMergeStatsWindow.h | 4 +- viewer/windows/JFJochProcessingJobsWindow.cpp | 5 +- viewer/windows/JFJochProcessingJobsWindow.h | 3 +- 9 files changed, 222 insertions(+), 24 deletions(-) diff --git a/rugnux/ReportDocument.h b/rugnux/ReportDocument.h index 4031ac619..429a16d03 100644 --- a/rugnux/ReportDocument.h +++ b/rugnux/ReportDocument.h @@ -64,6 +64,19 @@ struct ReportWarning { bool developer_only = false; }; +// One line of the SUMMARY's pathology checklist, for a rendering that lights an indicator per +// pathology rather than reading the summary's prose. PRESENT exactly when one of `codes` is in +// PATHOLOGY_FLAGS; ABSENT when it was looked for and did not fire; UNDETERMINED when it could not be +// measured on this run. `text` is the summary row's own words, `detail` the warnings that fired. +struct ReportCheck { + enum class Status { Present, Absent, Undetermined }; + std::string label; + std::vector codes; + Status status = Status::Undetermined; + std::string text; + std::string detail; +}; + // A value with its type kept, so a rendering other than text does not have to parse the string back. // `text` is always the exact spelling the text report prints, because that spelling is the interface. struct ReportValue { @@ -115,4 +128,5 @@ struct ReportDocument { std::string verdict; // closed: OK | WARNINGS | UNUSABLE | FAILED std::string verdict_text; // one to three sentences, free text std::vector pathology_flags; // PathologyCode::*, deduplicated, in first-seen order + std::vector checks; // the SUMMARY's pathologies, in the order it lists them }; diff --git a/rugnux/ResultReport.cpp b/rugnux/ResultReport.cpp index 189295c7b..31a125663 100644 --- a/rugnux/ResultReport.cpp +++ b/rugnux/ResultReport.cpp @@ -3,6 +3,7 @@ #include #include +#include #include #include @@ -2266,6 +2267,23 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, if (!value.empty()) f += fmt::format(" {:<18} {}\n", label, value); }; + // The pathology rows are also recorded as checks, with their status read off the flags. + const auto check = [&](const char *label, std::vector codes, const std::string &value) { + ReportCheck c{label, std::move(codes), ReportCheck::Status::Absent, value, {}}; + for (const auto &w : doc.warnings) + if (!w.developer_only && std::ranges::find(c.codes, w.code) != c.codes.end()) { + c.status = ReportCheck::Status::Present; + c.detail += (c.detail.empty() ? "" : "\n") + w.text; + } + doc.checks.push_back(std::move(c)); + }; + const auto checked_row = [&](const char *label, std::vector codes, const std::string &value) { + row(label, value); + check(label, std::move(codes), value); + }; + const auto untested = [&](const char *label, std::vector codes) { + doc.checks.push_back({label, std::move(codes), ReportCheck::Status::Undetermined, "not measured", {}}); + }; if (result.space_group) row("Space group", group); if (result.consensus_cell) @@ -2281,7 +2299,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, "with -z or --model", result.indexing_ambiguity_operators)); if (!result.powder.rings_q_recipA.empty() || result.consensus_cell) { const auto &pw = result.powder; - row("Powder", pw.rings_q_recipA.empty() ? std::string("no rings detected") + checked_row("Powder", {PathologyCode::POWDER_RINGS}, pw.rings_q_recipA.empty() ? std::string("no rings detected") : fmt::format("{} rings holding {:.0f}% of the spots found - {} ice ({:.0f}%), {} other " "({:.0f}%){}", pw.rings_q_recipA.size(), 100.0f * pw.spot_fraction, pw.ice_ring_count, 100.0f * pw.ice_spot_fraction, @@ -2289,17 +2307,24 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, result.powder_excluded_from_indexing ? "; left out of indexing" : "")); } if (result.ice_ring_score || result.ice_spot_ratio) - row("Ice", result.ice_rings_detected + checked_row("Ice", {PathologyCode::ICE_RINGS}, result.ice_rings_detected ? fmt::format("rings DETECTED: the {:.1f}% of the reflections on them were left out of " "scaling, kept in the merge", 100.0 * result.ice_flagged_fraction) : std::string("no rings detected")); + else + untested("Ice", {PathologyCode::ICE_RINGS}); if (result.harmonic.detected) - row("Beam", fmt::format("higher harmonic INDICATED ({:.0f} % of the unindexed spots " + checked_row("Beam", {PathologyCode::HARMONIC_CONTAMINATION}, + fmt::format("higher harmonic INDICATED ({:.0f} % of the unindexed spots " "are lattice points at lambda/3)", result.harmonic.fraction)); + else if (result.harmonic.measurable) + check("Beam", {PathologyCode::HARMONIC_CONTAMINATION}, "no higher harmonic indicated"); + else + untested("Beam", {PathologyCode::HARMONIC_CONTAMINATION}); if (result.leftover_lattices) { const auto &c = *result.leftover_lattices; const auto strongest = std::ranges::max_element(c.lattices, {}, &LeftoverLattice::intensity_pct); - row("Multiple lattices", strongest == c.lattices.end() ? std::string("none found") + checked_row("Multiple lattices", {PathologyCode::MULTIPLE_LATTICES}, strongest == c.lattices.end() ? std::string("none found") : fmt::format("{}{}: {:.0f}% of spot intensity (main lattice {:.0f}%){}", c.lattices.size() > 1 ? fmt::format("{} further lattices, the strongest: ", c.lattices.size()) : "", @@ -2307,6 +2332,8 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, c.primary_intensity_pct, DomainIntensityPct(c) >= MULTIPLE_LATTICES_INTENSITY_PCT ? " - check the crystal" : "")); + } else { + untested("Multiple lattices", {PathologyCode::MULTIPLE_LATTICES}); } if (merged) { std::string res = fmt::format("{:.2f} A written", o.d_min); @@ -2319,7 +2346,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, if (completeness_narrower) c = fmt::format("{:.1f} % to {:.2f} A; {}", completeness_fit.percent, completeness_fit.d_min, c); - row("Completeness", c); + checked_row("Completeness", {PathologyCode::LOW_COMPLETENESS}, c); } std::string sig; if (std::isfinite(o.mean_i_over_sigma)) @@ -2338,9 +2365,12 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, ? fmt::format("signal present (CC_anom {:.2f})", o.cc_anom) : fmt::format("no usable signal (CC_anom {:.2f})", o.cc_anom)); const auto &an = result.merge_statistics.anisotropy; - if (an.n_reflections > 0) { + if (an.n_reflections == 0) + untested("Anisotropy", {PathologyCode::ANISOTROPY}); + else { if (an.verdict != AnisotropyVerdict::Detected) - row("Anisotropy", "not established above this data set's own noise"); + checked_row("Anisotropy", {PathologyCode::ANISOTROPY}, + "not established above this data set's own noise"); else { const double *lo = nullptr, *hi = nullptr; for (const double *p = an.d_min_axis; p != an.d_min_axis + 3; ++p) { @@ -2351,7 +2381,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, } const char *sev = an.d_min_spread < 0.25 ? "small" : an.d_min_spread < 0.5 ? "noticeable" : "STRONG"; - row("Anisotropy", lo && hi + checked_row("Anisotropy", {PathologyCode::ANISOTROPY}, lo && hi ? fmt::format("{}: deltaB {:.1f} A^2, d_min {:.2f}-{:.2f} A by direction", sev, an.delta_b, *lo, *hi) : fmt::format("{}: deltaB {:.1f} A^2", sev, an.delta_b)); @@ -2376,10 +2406,15 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, t.peak_percent >= PSEUDO_TRANSLATION_WORDING_PCT ? "INDICATED" : "INDICATED, weak - check whether molecular replacement needs it", t.peak_percent, t.vector_length_A); - row("Pseudo-symmetry", ps.empty() ? std::string("no indication") : ps); + checked_row("Pseudo-symmetry", {PathologyCode::PSEUDO_TRANSLATION, PathologyCode::LATTICE_TRANSLATION}, + ps.empty() ? std::string("no indication") : ps); + } else { + untested("Pseudo-symmetry", {PathologyCode::PSEUDO_TRANSLATION, PathologyCode::LATTICE_TRANSLATION}); } if (result.twinning.l_test_pairs > 0) - row("Twinning", TwinVerdict(result)); + checked_row("Twinning", {PathologyCode::TWINNING}, TwinVerdict(result)); + else + untested("Twinning", {PathologyCode::TWINNING}); if (result.supercell) { const auto &sc = *result.supercell; const std::string doubled = fmt::format( @@ -2389,14 +2424,16 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, // Spots of further domains of the crystal can land on the half-integer positions. const bool domains = result.leftover_lattices && DomainIntensityPct(*result.leftover_lattices) >= 5.0; if (!SupercellPossible(sc)) - row("Supercell", "no indication"); + checked_row("Supercell", {PathologyCode::SUPERCELL_POSSIBLE}, "no indication"); else - row("Supercell", fmt::format("POSSIBLE, check - class {} {} {} rocks like Bragg reflections " + checked_row("Supercell", {PathologyCode::SUPERCELL_POSSIBLE}, fmt::format("POSSIBLE, check - class {} {} {} rocks like Bragg reflections " "(rocking part {:.1f}%); correct cells with weak ordered intensity " "read the same{}; {}", sc.h, sc.k, sc.l, sc.rock_pct, domains ? ", and the further lattice domains found may put " "spots there" : "", doubled)); + } else { + untested("Supercell", {PathologyCode::SUPERCELL_POSSIBLE}); } const double db = result.merge_statistics.radiation_damage_delta_b; if (std::isfinite(db)) @@ -2404,7 +2441,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, const auto &sq = result.merge_statistics.sweep_quality; if (sq.measured) { const int n_sweep = sq.frames_merged + sq.frames_downgraded + sq.frames_rejected; - row("Sweep", sq.frames_rejected > 0 + checked_row("Sweep", {PathologyCode::SWEEP_GAPS}, sq.frames_rejected > 0 ? fmt::format("{} of {} frames rejected ({:.1f} deg of {:.1f}), " "{} degraded range(s)", sq.frames_rejected, n_sweep, sq.rejected_deg, sq.sweep_deg, sq.ranges.size()) @@ -2412,8 +2449,22 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, ? fmt::format("no degraded ranges in {:.1f} deg", sq.sweep_deg) : fmt::format("{} degraded range(s) in {:.1f} deg, none rejected", sq.ranges.size(), sq.sweep_deg))); + } else { + untested("Sweep", {PathologyCode::SWEEP_GAPS}); } } + // A flag no row above stands for is still a pathology that fired: its own check, named + // after its code, so the checklist never shows less than PATHOLOGY_FLAGS. + for (const auto &code : doc.pathology_flags) + if (std::ranges::none_of(doc.checks, [&](const ReportCheck &c) { + return std::ranges::find(c.codes, code) != c.codes.end(); })) { + std::string label = code; + std::ranges::replace(label, '_', ' '); + std::transform(label.begin() + 1, label.end(), label.begin() + 1, + [](unsigned char ch) { return std::tolower(ch); }); + check(label.c_str(), {code}, ""); + doc.checks.back().text = doc.checks.back().detail.substr(0, doc.checks.back().detail.find('\n')); + } if (!f.empty()) { f.pop_back(); Add(s, Prose(f)); diff --git a/tests/ResultReportTest.cpp b/tests/ResultReportTest.cpp index b5f26dd21..1f964ed5c 100644 --- a/tests/ResultReportTest.cpp +++ b/tests/ResultReportTest.cpp @@ -1165,3 +1165,38 @@ TEST_CASE("ResultReport_SecondCrystal", "[Diagnostics]") { CHECK(text.find("second crystal, 38.9 deg from the main lattice") != std::string::npos); CHECK(text.find("split crystal, ") == std::string::npos); } + +// The summary's checklist: a status per pathology, PRESENT exactly where its code is flagged, and no +// flag left without a check of its own. +TEST_CASE("ResultReport_Checks", "[Diagnostics]") { + DiffractionExperiment x(DetJF(1)); + ProcessResult result; + result.consensus_cell = UnitCell{.a = 79.0f, .b = 79.0f, .c = 38.0f, + .alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f}; + result.pass_number = 10; + result.pass_count = 10; + result.geometry_not_converged = true; + result.harmonic.measurable = true; + + const auto doc = BuildReportDocument("p", "in.h5", x, result); + const auto find = [&](const std::string &label) { + return std::ranges::find(doc.checks, label, &ReportCheck::label); + }; + REQUIRE(find("Beam") != doc.checks.end()); + CHECK(find("Beam")->status == ReportCheck::Status::Absent); + REQUIRE(find("Multiple lattices") != doc.checks.end()); + CHECK(find("Multiple lattices")->status == ReportCheck::Status::Undetermined); + // GEOMETRY_NOT_CONVERGED has no summary row: it gets a check named after its code. + REQUIRE(find("Geometry not converged") != doc.checks.end()); + CHECK(find("Geometry not converged")->status == ReportCheck::Status::Present); + CHECK(!find("Geometry not converged")->text.empty()); + + for (const auto &code : doc.pathology_flags) + CHECK(std::ranges::any_of(doc.checks, [&](const ReportCheck &c) { + return c.status == ReportCheck::Status::Present + && std::ranges::find(c.codes, code) != c.codes.end(); })); + for (const auto &c : doc.checks) + if (c.status == ReportCheck::Status::Present) + CHECK(std::ranges::any_of(c.codes, [&](const std::string &code) { + return std::ranges::find(doc.pathology_flags, code) != doc.pathology_flags.end(); })); +} diff --git a/viewer/JFJochProcessController.cpp b/viewer/JFJochProcessController.cpp index 84c31c05e..699c82e09 100644 --- a/viewer/JFJochProcessController.cpp +++ b/viewer/JFJochProcessController.cpp @@ -16,6 +16,7 @@ JFJochProcessController::JFJochProcessController(QObject *parent) : QObject(parent) { qRegisterMetaType("ProcessResult"); qRegisterMetaType>("std::shared_ptr"); + qRegisterMetaType>("std::vector"); } JFJochProcessController::~JFJochProcessController() { @@ -98,19 +99,22 @@ void JFJochProcessController::run_(QString file_path, DiffractionExperiment expe // with WriteResultReport, a run that produced results must not be lost to a report // failure, so a throw here leaves the report empty rather than failing the job. QString report; + std::vector checks; if (!result.cancelled) { try { RunProvenance provenance; provenance.wall_time_s = run_timer.elapsed() / 1000.0; - report = QString::fromStdString(RenderResultReport( - output_prefix, file_path.toStdString(), report_experiment, result, provenance)); + ReportDocument doc = BuildReportDocument(output_prefix, file_path.toStdString(), + report_experiment, result, provenance); + report = QString::fromStdString(RenderReportText(doc, provenance.developer)); + checks = std::move(doc.checks); } catch (...) { } } active_ = nullptr; running_ = false; - emit finished(result, report); + emit finished(result, report, checks); } catch (const std::exception &e) { active_ = nullptr; running_ = false; diff --git a/viewer/JFJochProcessController.h b/viewer/JFJochProcessController.h index 8f16f4462..c603dc324 100644 --- a/viewer/JFJochProcessController.h +++ b/viewer/JFJochProcessController.h @@ -18,6 +18,7 @@ #include "../reader/JFJochReaderDataset.h" Q_DECLARE_METATYPE(ProcessResult) +Q_DECLARE_METATYPE(std::vector) // Runs one Rugnux job off the GUI thread and reports back via queued Qt signals. The job // opens its own private reader on the file (HDF5 access is globally serialized, so this @@ -44,8 +45,9 @@ signals: void phaseChanged(QString phase); void progress(quint64 done, quint64 total); // `report` is the rugnux result report (the _report.txt text), rendered from the run's - // own results so the viewer can show it without a file; empty for a cancelled run. - void finished(ProcessResult result, QString report); + // own results so the viewer can show it without a file; empty for a cancelled run. `checks` is + // the same report's pathology checklist. + void finished(ProcessResult result, QString report, std::vector checks); void failed(QString error); // Per-image results accumulated so far, for live dataset-info plots while a job runs (throttled). void liveDataset(std::shared_ptr dataset); diff --git a/viewer/windows/JFJochMergeStatsWindow.cpp b/viewer/windows/JFJochMergeStatsWindow.cpp index 85fb84f49..a49f009ac 100644 --- a/viewer/windows/JFJochMergeStatsWindow.cpp +++ b/viewer/windows/JFJochMergeStatsWindow.cpp @@ -11,7 +11,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -54,6 +56,7 @@ namespace { l->setContentsMargins(12, 8, 12, 8); l->setSpacing(0); auto *v = new QLabel(value, card); + v->setTextFormat(Qt::RichText); // the space group carries QFont f = v->font(); f.setPointSizeF(f.pointSizeF() * 2.2); f.setBold(true); @@ -67,6 +70,53 @@ namespace { l->addWidget(c); return card; } + + // The short Hermann-Mauguin symbol as rich text, screw axes subscripted and rotoinversions + // overlined: "P 1 21 1" -> P21, "P 41 21 2" -> P41212. Built from + // the spaced symbol, where short_name()'s run-together digits are ambiguous (P622 vs P6_2 2). + QString SpaceGroupHtml(const gemmi::SpaceGroup &sg) { + std::string hm = sg.hm; + // the monoclinic shortening of short_name(): "P 1 21 1" -> "P 21" + if (hm.size() > 6 && hm[2] == '1' && hm[hm.size() - 2] == ' ' && hm.back() == '1') + hm = hm[0] + hm.substr(3, hm.size() - 3 - 2); + if (sg.ext == 'H') + hm[0] = 'H'; + QString out; + for (const auto &token : QString::fromStdString(hm).split(' ', Qt::SkipEmptyParts)) { + if (token.size() >= 2 && token[0] == '-') + out += "" + token.mid(1, 1) + "" + token.mid(2); + else if (token.size() >= 2 && token[0].isDigit() && token[1].isDigit()) + out += token.left(1) + "" + token.mid(1, 1) + "" + token.mid(2); + else + out += token; + } + return out; + } + + // Status light of a pathology check: green tick absent, red cross present, grey dash undetermined. + QPixmap CheckIcon(ReportCheck::Status status, int size, qreal dpr) { + QPixmap pm(QSize(size, size) * dpr); + pm.setDevicePixelRatio(dpr); + pm.fill(Qt::transparent); + QPainter p(&pm); + p.setRenderHint(QPainter::Antialiasing); + const QColor fill = status == ReportCheck::Status::Present ? QColor(200, 40, 40) + : status == ReportCheck::Status::Absent ? QColor(40, 150, 70) + : QColor(150, 150, 150); + p.setPen(Qt::NoPen); + p.setBrush(fill); + p.drawEllipse(QRectF(0.5, 0.5, size - 1.0, size - 1.0)); + p.setPen(QPen(Qt::white, size / 8.0, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin)); + const double u = size / 16.0; + if (status == ReportCheck::Status::Absent) + p.drawPolyline(QPolygonF({QPointF(4.5 * u, 8.5 * u), QPointF(7 * u, 11 * u), QPointF(11.5 * u, 5.5 * u)})); + else if (status == ReportCheck::Status::Present) { + p.drawLine(QPointF(5 * u, 5 * u), QPointF(11 * u, 11 * u)); + p.drawLine(QPointF(11 * u, 5 * u), QPointF(5 * u, 11 * u)); + } else + p.drawLine(QPointF(4.5 * u, 8 * u), QPointF(11.5 * u, 8 * u)); + return pm; + } } JFJochMergeStatsWindow::JFJochMergeStatsWindow(const QString &title, const MergeStatistics &stats, @@ -76,6 +126,7 @@ JFJochMergeStatsWindow::JFJochMergeStatsWindow(const QString &title, const Merge const std::optional &space_group_search, const std::vector> &merged_i_sigma, const QString &report, + const std::vector &checks, QWidget *parent) : QWidget(parent, Qt::Window), stats_(stats), has_reference_(has_reference) { setWindowTitle("Merge statistics — " + title); @@ -114,9 +165,45 @@ JFJochMergeStatsWindow::JFJochMergeStatsWindow(const QString &title, const Merge if (has_reference_) hero->addWidget(MakeCard(pct(o.cc_ref * 100.0), "CCref", this)); if (space_group.has_value()) - hero->addWidget(MakeCard(QString::fromStdString(space_group->short_name()), "Space group", this)); + hero->addWidget(MakeCard(SpaceGroupHtml(*space_group), "Space group", this)); layout->addLayout(hero); + // --- Pathology checklist, as phenix.xtriage lists its tests: one status light per pathology the + // report's summary looks for, its words and the warnings it raised in the tooltip. + if (!checks.empty()) { + auto *grid = new QGridLayout(); + grid->setHorizontalSpacing(16); + grid->setVerticalSpacing(2); + constexpr int columns = 4; + for (int i = 0; i < static_cast(checks.size()); ++i) { + const auto &c = checks[i]; + auto *item = new QWidget(this); + auto *l = new QHBoxLayout(item); + l->setContentsMargins(0, 0, 0, 0); + l->setSpacing(6); + auto *icon = new QLabel(item); + icon->setPixmap(CheckIcon(c.status, 14, devicePixelRatioF())); + auto *label = new QLabel(QString::fromStdString(c.label), item); + if (c.status == ReportCheck::Status::Present) { + QFont f = label->font(); + f.setBold(true); + label->setFont(f); + } else if (c.status == ReportCheck::Status::Undetermined) { + label->setStyleSheet("color: gray;"); + } + l->addWidget(icon); + l->addWidget(label); + l->addStretch(); + QString tip = "" + QString::fromStdString(c.label).toHtmlEscaped() + ": " + + QString::fromStdString(c.text).toHtmlEscaped(); + if (!c.detail.empty() && c.detail != c.text) + tip += "

" + QString::fromStdString(c.detail).toHtmlEscaped().replace('\n', "

"); + item->setToolTip("" + tip + ""); + grid->addWidget(item, i / columns, i % columns); + } + layout->addLayout(grid); + } + // De-novo space-group search: the point group + the ranked candidates it scored (only present when // the space group was not fixed by the user). A summary line above a compact table. if (space_group_search.has_value() && !space_group_search->candidates.empty()) { @@ -125,10 +212,11 @@ JFJochMergeStatsWindow::JFJochMergeStatsWindow(const QString &title, const Merge if (!sgs.alternatives.empty()) { QStringList alts; for (const auto &alt : sgs.alternatives) - alts << QString::fromStdString(alt.short_name()); + alts << SpaceGroupHtml(alt); summary += " · indistinguishable from these data: " + alts.join(", "); } auto *sgLabel = new QLabel(summary, this); + sgLabel->setTextFormat(Qt::RichText); sgLabel->setStyleSheet("color: gray;"); layout->addWidget(sgLabel); diff --git a/viewer/windows/JFJochMergeStatsWindow.h b/viewer/windows/JFJochMergeStatsWindow.h index 199fb3169..7c06c2c41 100644 --- a/viewer/windows/JFJochMergeStatsWindow.h +++ b/viewer/windows/JFJochMergeStatsWindow.h @@ -13,6 +13,7 @@ #include "../../image_analysis/scale_merge/Merge.h" // MergeStatistics #include "../../image_analysis/scale_merge/TwinningAnalysis.h" // TwinningAnalysisResult #include "../../image_analysis/scale_merge/SearchSpaceGroup.h" // SearchSpaceGroupResult +#include "../../rugnux/ReportDocument.h" // ReportCheck class QComboBox; class QLabel; @@ -22,7 +23,7 @@ class JFJochSimpleChartView; // Pop-up analysis window for a finished scaling/merging run: a row of eye-catching overall numbers // (ISa, CC1/2, R-meas, completeness, multiplicity, the Wilson B and the radiation-damage Delta-B, and -// CCref when a reference was used) above a per-resolution-shell plot whose metric is chosen from a +// CCref when a reference was used) and the report's pathology checklist above a per-resolution-shell plot whose metric is chosen from a // combo, a per-shell table, and an ISa diagnostic scatter. Opened automatically when a merge job // finishes and on demand from the processing-jobs dock. Modeless; deletes on close. class JFJochMergeStatsWindow : public QWidget { @@ -36,6 +37,7 @@ public: const std::optional &space_group_search, const std::vector> &merged_i_sigma, const QString &report, + const std::vector &checks, QWidget *parent = nullptr); private: diff --git a/viewer/windows/JFJochProcessingJobsWindow.cpp b/viewer/windows/JFJochProcessingJobsWindow.cpp index 0e0aa6f3a..d66b0b7bf 100644 --- a/viewer/windows/JFJochProcessingJobsWindow.cpp +++ b/viewer/windows/JFJochProcessingJobsWindow.cpp @@ -599,7 +599,7 @@ void JFJochProcessingJobsWindow::showStats(const QString &id) { if (j.id == id && j.has_merge_stats) { auto *win = new JFJochMergeStatsWindow(j.label, j.merge_stats, j.isa, j.merge_has_reference, j.twinning, j.space_group, j.space_group_search, - j.merged_i_sigma, j.report, window()); + j.merged_i_sigma, j.report, j.checks, window()); win->show(); return; } @@ -719,7 +719,7 @@ void JFJochProcessingJobsWindow::onProgress(quint64 done, quint64 total) { } } -void JFJochProcessingJobsWindow::onFinished(ProcessResult result, QString report) { +void JFJochProcessingJobsWindow::onFinished(ProcessResult result, QString report, std::vector checks) { emit liveDataset(nullptr); // the finished run takes over from the live overlay const int row = running_row_; running_row_ = -1; @@ -766,6 +766,7 @@ void JFJochProcessingJobsWindow::onFinished(ProcessResult result, QString report if (!result.cancelled && result.has_merge_statistics) { jobs_[row].has_merge_stats = true; jobs_[row].report = report; + jobs_[row].checks = std::move(checks); jobs_[row].merge_stats = result.merge_statistics; jobs_[row].isa = result.error_model_isa; jobs_[row].merge_has_reference = result.has_reference; diff --git a/viewer/windows/JFJochProcessingJobsWindow.h b/viewer/windows/JFJochProcessingJobsWindow.h index 08d8eb692..817c05f8b 100644 --- a/viewer/windows/JFJochProcessingJobsWindow.h +++ b/viewer/windows/JFJochProcessingJobsWindow.h @@ -54,7 +54,7 @@ private slots: void viewResults(); void onPhase(QString phase); void onProgress(quint64 done, quint64 total); - void onFinished(ProcessResult result, QString report); + void onFinished(ProcessResult result, QString report, std::vector checks); void onFailed(QString error); private: @@ -73,6 +73,7 @@ private: std::optional space_group_search; // ranked candidates, when a search ran std::vector> merged_i_sigma; // ISa diagnostic scatter QString report; // the rugnux result report text (a section of the window) + std::vector checks; // the report's pathology checklist // Detector calibration result (Calibration mode): the fit, the header geometry it is compared // against, and where the .poni was written. std::optional calibration; -- 2.54.0 From da5ef0c5032031a329943bab69714c1575146d64 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 21:40:42 +0200 Subject: [PATCH 20/97] SearchSpaceGroup: offer every setting of the chosen rotation set, not only on a near-exact cell The non-reference settings of the adopted point group were offered only when the cell hosted their rotations to within ~0.1 deg, while the reference settings - which hold the very same rotations - were offered without asking. A cell refined free after integration a few tenths of a degree off 90 therefore lost every setting but the reference one. On an orthorhombic set whose measured screws lie on a and c and whose b row was never recorded, that left P2(1)2(1)2(1) as the only candidate covering both screws, and it was reported as determined. With P 21 2 21 offered, the two tie, the b-axis screw is reported as undetermined, and the model check uses the setting the data describe. The point-group stage still asks the cell whether a rotation set it adds is hosted; only the setting enumeration within an already chosen set stops asking. Validation: myob/cytc/thau x10sa p.mtz byte-identical (GPU); 7mzt fail -> unscored (b screw undetermined, P 21 21 21 or P 21 2 21); 5cc8 unchanged; [SearchSpaceGroup] 23 cases pass incl. a new section with a cell 0.2-0.3 deg off 90. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- image_analysis/scale_merge/SearchSpaceGroup.cpp | 14 +++++++++----- tests/SearchSpaceGroupTest.cpp | 11 +++++++++++ 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/image_analysis/scale_merge/SearchSpaceGroup.cpp b/image_analysis/scale_merge/SearchSpaceGroup.cpp index 3b99f6e3e..59eb289eb 100644 --- a/image_analysis/scale_merge/SearchSpaceGroup.cpp +++ b/image_analysis/scale_merge/SearchSpaceGroup.cpp @@ -1900,9 +1900,14 @@ SearchSpaceGroupResult SearchSpaceGroup( // the point group is one only a non-reference setting carries (Stage A's second pass), since // otherwise there is no candidate at all and the group would be lost after being found. These are // alternative namings at the same order, so nothing here can promote the point group; what they - // add is a screw or a centering on the axis the data show it on. Two refusals bound them: a - // setting whose axes the cell does not have is not offered, and one predicting exactly the - // absences a candidate already offered predicts is the same hypothesis under another name. + // add is a screw or a centering on the axis the data show it on. One refusal bounds them: a setting + // predicting exactly the absences a candidate already offered predicts is the same hypothesis + // under another name. The cell is NOT asked again here: a setting of the chosen rotation set holds + // the very rotations the reference candidates hold, so it fits the cell exactly as well as they + // do - and they are offered without asking. Asking only the non-reference ones turned a free- + // refined cell a few tenths of a degree off 90 into a refusal of every setting but the reference + // one, leaving the group with the measured screws on a and c (P 2_1 2 2_1) unoffered and P2_12_12_1 + // the only candidate that covered both. if (opt.cell.has_value() && (opt.enumerate_all_settings || (opt.enumerate_all_rotation_sets && sg_cands.empty()))) { std::vector> signatures; @@ -1910,8 +1915,7 @@ SearchSpaceGroupResult SearchSpaceGroup( signatures.push_back(AbsenceSignature(*c)); for (const auto& sg : gemmi::spacegroup_tables::main) { if (!sg.is_sohncke() || sg.is_reference_setting() || - RotationSetOf(sg) != best_pg->rotation_set || - !CellHostsRotations(*opt.cell, sg.operations())) + RotationSetOf(sg) != best_pg->rotation_set) continue; auto sig = AbsenceSignature(sg); if (std::find(signatures.begin(), signatures.end(), sig) != signatures.end()) diff --git a/tests/SearchSpaceGroupTest.cpp b/tests/SearchSpaceGroupTest.cpp index 88123bdd3..59b912021 100644 --- a/tests/SearchSpaceGroupTest.cpp +++ b/tests/SearchSpaceGroupTest.cpp @@ -444,6 +444,17 @@ TEST_CASE("SearchSpaceGroup names an orthorhombic screw pair on the axes it lies REQUIRE(result.best_space_group.has_value()); CHECK(result.best_space_group->xhm() == "P 2 21 21"); } + + // A cell refined free after integration is a few tenths of a degree off 90. The reference + // candidates hold the same rotations and are offered on it, so the other settings must be too. + SECTION("a free cell a few tenths off 90 still offers it") { + opt.cell = gemmi::UnitCell(40.0, 50.0, 60.0, 90.32, 90.19, 90.19); + opt.enumerate_all_settings = true; + const auto result = SearchSpaceGroup(merged, opt); + INFO(SearchSpaceGroupResultToText(result)); + REQUIRE(result.best_space_group.has_value()); + CHECK(result.best_space_group->xhm() == "P 2 21 21"); + } } // A screw on a row the sweep never recorded is not a group the data refused, it is a question -- 2.54.0 From 42441ac3283ccc359f231dc588ec9cba595a8646 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 21:40:51 +0200 Subject: [PATCH 21/97] Twinning report: warn when a promotion stands on intensities below a perfect twin's; flag compressed twin-immune zone controls A promoted point group whose <|L|> reads below 0.375 (NOT_READABLE) previously raised no TWINNING warning at all; a twinned subgroup whose law the promotion absorbed predicts the same data, so the adopted group is not confirmed. Warn. The twin-immune zone control reading more compressed than a perfect twin's acentric population (0.541) is something no twin fraction produces (overlap or neighbour correlation); genuine symmetry then reads acentric in its zones too (measured: a genuine 622 with its control at 0.528 read its 2-folds at -950 to -2640 nats). Such zones are now marked ambiguous in the text and the zone decision line. Report-only: no decision and no output file but the report changes. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- image_analysis/scale_merge/TwinningAnalysis.cpp | 10 ++++++++++ image_analysis/scale_merge/TwinningAnalysis.h | 9 +++++++++ rugnux/ResultReport.cpp | 11 +++++++++++ rugnux/Rugnux.cpp | 4 +++- 4 files changed, 33 insertions(+), 1 deletion(-) diff --git a/image_analysis/scale_merge/TwinningAnalysis.cpp b/image_analysis/scale_merge/TwinningAnalysis.cpp index 78ffeb15e..3f7ba6f6e 100644 --- a/image_analysis/scale_merge/TwinningAnalysis.cpp +++ b/image_analysis/scale_merge/TwinningAnalysis.cpp @@ -596,6 +596,10 @@ namespace { result.control.evidence_nats - result.control.n * result.control_excess_per_reflection; for (auto& z : result.zones) z.calibrated_evidence_nats = z.evidence_nats - z.n * result.control_excess_per_reflection; + constexpr double PERFECT_TWIN_ACENTRIC_MEAN_ABS_E2_MINUS_1 = 0.541; + result.zones_ambiguous = result.control.n > 0 + && result.control.mean_abs_e2_minus_1 + 3.0 * result.control.standard_error + < PERFECT_TWIN_ACENTRIC_MEAN_ABS_E2_MINUS_1; } } @@ -862,5 +866,11 @@ std::string TwinImmuneZonesToText(const TwinImmuneZoneResult& result) { for (const auto& z : result.zones) row(z); row(result.control); + if (result.zones_ambiguous) + os << " The control reads more compressed than a perfect twin's acentric reflections (0.541), which\n" + " no twin fraction can do: something else - overlapping spots, neighbour correlation - averages\n" + " each reflection with unrelated ones, and it compresses the zones as well. Genuine symmetry then\n" + " reads acentric in its zones too, so here they do not separate a twinned subgroup from the\n" + " adopted group, and no verdict is read from them either way.\n"; return os.str(); } diff --git a/image_analysis/scale_merge/TwinningAnalysis.h b/image_analysis/scale_merge/TwinningAnalysis.h index cd831d2fb..688daf948 100644 --- a/image_analysis/scale_merge/TwinningAnalysis.h +++ b/image_analysis/scale_merge/TwinningAnalysis.h @@ -143,6 +143,15 @@ struct TwinImmuneZoneResult { // nats were the normalisation's, and the verdict rescued a twin law. Calibrated, that zone reads // -115 nats; the zones of genuine promotions keep +140 nats (a 90 deg tetragonal sweep) to +2000. double control_excess_per_reflection = 0.0; + // The control reads more compressed than a PERFECT twin's acentric population (<|E^2-1|> 0.541, by + // more than three standard errors). Twinning cannot do that at any fraction, so something else + // averages each reflection with unrelated ones - overlapping spots of a long cell, neighbour + // correlation - and it compresses the zones as much as the control: genuine symmetry then reads + // acentric in its zones. Measured on the open battery: a 622 crystal with its control at 0.528 + // read its genuine 2-folds at -950 to -2640 nats, and a twinned 4/m crystal promoted to 4/mmm, its + // control at 0.454, read its added operators at -226 to -616. The zones are reported, but no + // verdict is read from them either way. + bool zones_ambiguous = false; double anisotropy_delta_b_A2 = 0.0; // the anisotropy taken out of the intensities before normalising double d_max_A = 0.0; // resolution range of the shells read double d_min_A = 0.0; diff --git a/rugnux/ResultReport.cpp b/rugnux/ResultReport.cpp index 189295c7b..7cd5edeae 100644 --- a/rugnux/ResultReport.cpp +++ b/rugnux/ResultReport.cpp @@ -1663,6 +1663,17 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, Warn(doc, PathologyCode::TWINNING, fmt::format( "Twinning is indicated ({}) - refinement against the merged data needs a twin law", TwinningVerdictLine(tw))); + else if (abnormal && tw.laue_class_was_chosen_by_promotion && tw.mean_abs_l < 0.375) + // Below a perfect twin's 0.375 the L-test says nothing about the twin fraction, but it does + // say the intensities are at least as twin-like as a perfect twin's - and a perfect twin of + // a subgroup, its law absorbed by the promotion, predicts exactly the data the promoted + // group does. The promotion is then unconfirmed, however clean its operators look. + Warn(doc, PathologyCode::TWINNING, fmt::format( + "The point group was promoted although the intensities read at least as twin-like as a " + "perfect twin's (<|L|> = {:.3f}, below 0.375): a twin of a subgroup whose law the promotion " + "absorbed predicts the same data, so the adopted group is not confirmed - refine in the " + "subgroup with that twin law too (p_P1.mtz carries the same observations)", + tw.mean_abs_l)); } // ---- radiation damage diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 94a9325af..917f12968 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -2103,7 +2103,9 @@ namespace { zone.standard_error, z.control.mean_abs_e2_minus_1, z.control_excess_per_reflection, zone.evidence_nats, zone.calibrated_evidence_nats, z.d_max_A, z.d_min_A, z.tncs_normalised ? ", normalised per pseudo-translation class" : "", - verdict < 0 ? "acentric, the added operators are a twin law or a pseudo-symmetry" + verdict < 0 && z.zones_ambiguous + ? "acentric, but the control is compressed beyond a perfect twin, so genuine symmetry would read so too" + : verdict < 0 ? "acentric, the added operators are a twin law or a pseudo-symmetry" : verdict > 0 ? "centric, the added operators are real" : "undecided"); // Recorded only where the zone DECIDED something. An undecided read decides nothing - the // rule stays as it was - and recording it had the report announce "a twin gate refused this -- 2.54.0 From 769a4dd8debc2c5732e700156005a77070a9ed35 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 22:35:00 +0200 Subject: [PATCH 22/97] rugnux: pass-1 lattice hypothesis compares lattices, not classes The first version judged "the same lattice" by class and centring, so an F-centred cubic re-index (57/60) and pass 1's triclinic primitive of the very same lattice (60/60) counted as different, and the unconstrained primitive won the frame count - 6oel lost its cubic setting (R_meas 36.6 -> 42.1 %). The two are now compared on their Niggli-reduced primitive edges (within 2 %, the battery's lattice-identity test), so a symmetric setting never loses to its own primitive, while 9qw8's C-centred re-index (reduced edges 5.7 % off pass 1's) is still judged against pass 1's lattice on the frames. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- rugnux/Rugnux.cpp | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 74e05a89d..bd60493e4 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -5516,13 +5516,25 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const auto &l = pass1_lattice.lattice; pass1_lattice.lattice = CrystalLattice(l.Vec0() * f, l.Vec1() * f, l.Vec2() * f); } - const double pass1_vol = std::abs(pass1_lattice.lattice - .ToPrimitive(pass1_lattice.search_result.centering).CalcVolume()); - // Same lattice as in the beam-centre check: same class and primitive volumes within 2 %. - const bool same_lattice = best.result.has_value() - && best.result->search_result.centering == pass1_lattice.search_result.centering - && best.result->search_result.system == pass1_lattice.search_result.system - && std::abs(best.vol - pass1_vol) <= 0.02 * pass1_vol; + const CrystalLattice pass1_primitive = pass1_lattice.lattice + .ToPrimitive(pass1_lattice.search_result.centering).NiggliReduce(); + const double pass1_vol = std::abs(pass1_primitive.CalcVolume()); + // The same lattice whatever setting or class it is held in - a centred cell and the + // triclinic primitive of the same points are one lattice, and the re-index's symmetric + // setting must not lose to its own unconstrained primitive on a frame count: the Niggli- + // reduced primitive edges agree within 2 % (the battery's lattice-identity test). + const auto reduced_edges = [](const CrystalLattice &l) { + const auto uc = l.GetUnitCell(); + std::array e = {uc.a, uc.b, uc.c}; + std::sort(e.begin(), e.end()); + return e; + }; + const auto e1 = reduced_edges(pass1_primitive); + const auto e2 = reduced_edges(best.result->lattice + .ToPrimitive(best.result->search_result.centering).NiggliReduce()); + bool same_lattice = true; + for (int i = 0; i < 3; i++) + same_lattice = same_lattice && std::abs(e2[i] - e1[i]) <= 0.02 * e1[i]; const int floor = static_cast(validation.size()) / 6; const int pass1_score = (best.score < floor || !same_lattice) ? count_indexed(*indexer, pass1_lattice) : -1; -- 2.54.0 From f69339ce6387234c8e3fdde14f9918238dd50725 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 3 Oct 2026 23:08:12 +0200 Subject: [PATCH 23/97] RotationScaleMerge: fulls-only per-frame scaling (prototype), pooled over sparse frames Prototype switch --no-scale-partials: skip the per-frame scaling of the partials and let the fulls (scale-fulls) carry the per-frame scale, as XDS does. Without partial scaling a frame whose fulls number fewer than MIN_REFLECTIONS is fitted over the nearest frames on either side that together hold enough (PoolHalfWidth), on the host and in the GPU kernel. Default path unchanged (p.mtz md5 identical on the three profiling sets). Why: on fine-sliced small-molecule sweeps the per-frame partial scale and the partiality model are degenerate within a rocking curve, and the fit swings G 0.23..1.2 with a 180 deg period (XDS's own frame scale: 0.79..0.99). That imprints an hkl-dependent bias common to all equivalents, which R_meas/CC1/2/ISa cannot see but a refinement against the known structure does. And scale-fulls never fitted a frame on such data: a full is filed under one frame, about 8 per frame, below MIN_REFLECTIONS, so every frame kept G = 1. Measured with SHELXL refining the COD structures (R1 >4sig), default -> switch: aspirin 20 keV 0.096 -> 0.062, aspirin 25 keV 0.094 -> 0.046, citric acid 0.161 -> 0.109, HEPES 0.090 -> 0.070 (XDS 0.030-0.038). Proteins lose ISa with the switch (myob 9.1 -> 7.6, cytc 25.8 -> 13.7), so it is not a default; the choice is to be made from the data. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- common/ScalingSettings.cpp | 9 ++++ common/ScalingSettings.h | 3 ++ .../scale_merge/RotationScaleMerge.cpp | 41 +++++++++++++--- .../scale_merge/RotationScaleMerge.h | 7 +++ .../scale_merge/RotationScaleMergeGPU.cu | 48 ++++++++++++------- .../scale_merge/RotationScaleMergeGPU.h | 4 +- rugnux/rugnux_cli.cpp | 9 ++++ tests/RotationScaleWalkTest.cpp | 13 +++++ 8 files changed, 110 insertions(+), 24 deletions(-) diff --git a/common/ScalingSettings.cpp b/common/ScalingSettings.cpp index 2016afba5..4fabbd44d 100644 --- a/common/ScalingSettings.cpp +++ b/common/ScalingSettings.cpp @@ -208,6 +208,15 @@ double ScalingSettings::GetSmoothGDegrees() const { return smooth_g_deg; } +ScalingSettings &ScalingSettings::ScalePartials(bool input) { + scale_partials = input; + return *this; +} + +bool ScalingSettings::GetScalePartials() const { + return scale_partials; +} + ScalingSettings &ScalingSettings::RelativeBDegrees(double input) { if (input < 0) throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Relative-B batch width must be non-negative"); diff --git a/common/ScalingSettings.h b/common/ScalingSettings.h index 198c6366c..e9b2cd72e 100644 --- a/common/ScalingSettings.h +++ b/common/ScalingSettings.h @@ -98,6 +98,7 @@ class ScalingSettings { // degrees (like XDS DELPHI), converted to an odd frame window from the oscillation step; this keeps // the smoothing physical (independent of frame slicing). 0 = off. A no-op without rot3d. double smooth_g_deg = 0.0; + bool scale_partials = true; // Per-batch relative-B on the rot3d fulls (beyond the single global decay slope): bin frames into // rotation-range batches of this width in degrees and refine one relative Debye-Waller B per batch, so @@ -143,6 +144,7 @@ public: ScalingSettings& IceMinScore(float input); ScalingSettings& IceMinSpotRatio(float input); ScalingSettings& SmoothGDegrees(double input); + ScalingSettings& ScalePartials(bool input); ScalingSettings& RelativeBDegrees(double input); ScalingSettings& RfreeFraction(double input); @@ -182,6 +184,7 @@ public: [[nodiscard]] float GetIceMinScore() const; [[nodiscard]] float GetIceMinSpotRatio() const; [[nodiscard]] double GetSmoothGDegrees() const; + [[nodiscard]] bool GetScalePartials() const; [[nodiscard]] double GetRelativeBDegrees() const; [[nodiscard]] double GetRfreeFraction() const; diff --git a/image_analysis/scale_merge/RotationScaleMerge.cpp b/image_analysis/scale_merge/RotationScaleMerge.cpp index 81dabcd22..2b71049b4 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.cpp +++ b/image_analysis/scale_merge/RotationScaleMerge.cpp @@ -1407,17 +1407,34 @@ void RotationScaleMerge::ReduceScalingGroupMeans(int n_groups, const std::vector }); } +int RotationScaleMerge::PoolHalfWidth(const std::vector &fcount, int f) { + const int n = static_cast(fcount.size()); + int64_t total = fcount[f]; + int h = 0; + while (total < static_cast(MIN_REFLECTIONS) && h < n) { + ++h; + if (f - h >= 0) total += fcount[f - h]; + if (f + h < n) total += fcount[f + h]; + } + return h; +} + template void RotationScaleMerge::FitPerFrameG(const std::vector &obs, const std::vector &fstart, const std::vector &fcount, const std::vector &group_mean_in, bool unity, std::vector &g) { std::vector scaled(fstart.size(), 0); - ParallelFor(static_cast(fstart.size()), nthreads, [&](int f) { + const int n_fr = static_cast(fstart.size()); + ParallelFor(n_fr, nthreads, [&](int f) { + // A full is a whole rocking event filed under one frame, so on a sparse sweep a frame holds a + // handful of them - too few to fit a scale on, and the frame was left unscaled, which on such a + // sweep was every frame. Those frames are fitted over the nearest frames on either side that + // together hold enough; a frame with enough of its own is fitted on its own, as before. + const int h = unity && pool_sparse_fulls ? PoolHalfWidth(fcount, f) : 0; std::vector so; - so.reserve(fcount[f]); - const int lo = fstart[f], hi = fstart[f] + fcount[f]; - for (int i = lo; i < hi; ++i) { + for (int j = std::max(0, f - h); j <= std::min(n_fr - 1, f + h); ++j) + for (int i = fstart[j]; i < fstart[j] + fcount[j]; ++i) { const auto &o = obs[i]; if (o.group < 0) continue; if (o.on_ice) continue; @@ -5607,9 +5624,15 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st "curves span {} frames)", smooth_window, smooth_window * osc_deg, smooth_g_deg, rocking_event_frames_at_start); } + // Without partial scaling every frame keeps G = 1 here and its scale comes from the fulls alone + // (scale-fulls below), as in XDS: a partial's scale and an error of the partiality model are the + // same thing within a rocking curve, and on a sparse fine-sliced sweep the fit takes the one for + // the other. + const bool scale_partials = s.GetScalePartials(); + pool_sparse_fulls = !scale_partials; ScalingLoopOutcome partial_loop; #ifdef JFJOCH_USE_CUDA - if (gpu_active_) { + if (gpu_active_ && scale_partials) { // The scaling loop runs on the GPU one iteration per call, and corr stays RESIDENT across // scaling -> smooth-G -> CC -> combine (and across passes, exactly as the old host round-trip // did). Only the per-frame G/scaled come back after each iteration, for the gauge pin and the @@ -5630,7 +5653,11 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st scaled_on_gpu = true; } #endif - if (!scaled_on_gpu) { + if (!scale_partials) { + std::fill(g_partial.begin(), g_partial.end(), 1.0); + frame_scaled_scratch.assign(n_frames, 1); + partial_loop.converged = true; + } else if (!scaled_on_gpu) { const PartialLoopKey key{x.GetSpaceGroupOrP1().xhm(), merge_friedel, d_min_limit, d_max_limit, min_partiality, smooth_window, scaling_iter}; const auto memo = std::find_if(partial_loop_memos.begin(), partial_loop_memos.end(), @@ -5920,7 +5947,7 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st std::vector scaled_dev(n_frames); fulls_loop = RunScalingLoop("fulls", g_full, f_count, smooth_window, /*release_restraint=*/true, [&] { - gpu_->ScaleFulls(1, min_partiality); + gpu_->ScaleFulls(1, min_partiality, pool_sparse_fulls); gpu_->GetG(g_dev.data(), scaled_dev.data()); for (int f = 0; f < n_frames; ++f) if (scaled_dev[f]) g_full[f] = g_dev[f]; diff --git a/image_analysis/scale_merge/RotationScaleMerge.h b/image_analysis/scale_merge/RotationScaleMerge.h index 79278560f..30da42438 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.h +++ b/image_analysis/scale_merge/RotationScaleMerge.h @@ -43,6 +43,10 @@ // Stills use the per-image ScaleOnTheFly (fixed partiality) instead. class RotationScaleMerge { public: + // How many frames on either side a frame's fulls scale is fitted over (used without partial + // scaling): none when the frame holds MIN_REFLECTIONS fulls itself, else the fewest that bring the + // pooled count there. + static int PoolHalfWidth(const std::vector &fcount, int f); struct Result { std::vector merged; MergeStatistics statistics; @@ -213,6 +217,9 @@ private: // this is how far they may go before giving up and saying so. int scaling_iter = 100; bool scale_fulls = true; + // Without partial scaling the fulls carry the whole per-frame scale, and a sparse frame is fitted + // over its neighbours (PoolHalfWidth). + bool pool_sparse_fulls = false; bool refine_decay_b = false; // per-time-block Debye-Waller decay correction (radiation damage) int absorption_iter = 0; // >0: fit a goniometer-frame absorption surface over this many iterations int modulation_iter = 0; // >0: fit a detector-plane modulation (flat-field) surface, this many iterations diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.cu b/image_analysis/scale_merge/RotationScaleMergeGPU.cu index 66aeabacb..06b85d5bd 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.cu +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.cu @@ -123,16 +123,31 @@ namespace { const int32_t *__restrict__ frame_count, const float *__restrict__ I, const double *__restrict__ inv_sigma, const float *__restrict__ sco_coeff, const uint8_t *__restrict__ sco_ok, - const int32_t *__restrict__ perm, + const int32_t *__restrict__ perm, bool pool, double *__restrict__ g, uint8_t *__restrict__ scaled) { const int f = blockIdx.x; if (f >= n_frames) return; - const int lo = frame_start[f], hi = frame_start[f] + frame_count[f]; + // pool (the fulls): a frame holding fewer than MIN_REFLECTIONS is fitted over the fewest frames + // on either side that bring the count there - RotationScaleMerge::PoolHalfWidth, same rule. + __shared__ int s_h; + if (threadIdx.x == 0) { + long total = frame_count[f]; + int h = 0; + while (pool && total < long(MIN_REFLECTIONS) && h < n_frames) { + ++h; + if (f - h >= 0) total += frame_count[f - h]; + if (f + h < n_frames) total += frame_count[f + h]; + } + s_h = h; + } + __syncthreads(); + const int j_lo = max(0, f - s_h), j_hi = min(n_frames - 1, f + s_h); __shared__ double sh[BLK]; long cnt_local = 0; - for (int i = lo + threadIdx.x; i < hi; i += blockDim.x) - if (sco_ok[perm ? perm[i] : i]) ++cnt_local; + for (int j = j_lo; j <= j_hi; ++j) + for (int i = frame_start[j] + threadIdx.x; i < frame_start[j] + frame_count[j]; i += blockDim.x) + if (sco_ok[perm ? perm[i] : i]) ++cnt_local; const double cnt = BlockReduceSum(double(cnt_local), sh); __shared__ double s_cnt; if (threadIdx.x == 0) s_cnt = cnt; @@ -140,15 +155,16 @@ namespace { if (s_cnt < MIN_REFLECTIONS) return; // leave g[f]/scaled[f] as-is double num = 0.0, den = 0.0; - for (int i = lo + threadIdx.x; i < hi; i += blockDim.x) { - const int a = perm ? perm[i] : i; - if (!sco_ok[a]) continue; - const double coeff = sco_coeff[a]; - const double w = inv_sigma[a]; - const double w2 = w * w; - num += w2 * coeff * double(I[a]); - den += w2 * coeff * coeff; - } + for (int j = j_lo; j <= j_hi; ++j) + for (int i = frame_start[j] + threadIdx.x; i < frame_start[j] + frame_count[j]; i += blockDim.x) { + const int a = perm ? perm[i] : i; + if (!sco_ok[a]) continue; + const double coeff = sco_coeff[a]; + const double w = inv_sigma[a]; + const double w2 = w * w; + num += w2 * coeff * double(I[a]); + den += w2 * coeff * coeff; + } const double tnum = BlockReduceSum(num, sh); __syncthreads(); const double tden = BlockReduceSum(den, sh); if (threadIdx.x == 0) { @@ -1081,7 +1097,7 @@ void RotationScaleMergeGPU::ScalePartials(int iters, double min_partiality, bool d.sco_coeff.get(), d.sco_ok.get()); CudaCheck(cudaGetLastError(), "PrepScaleObsKernel launch"); FitPerFrameGKernel<<s()>>>(d.n_frames, d.frame_start.get(), d.frame_count.get(), - d.I.get(), d.inv_sigma.get(), d.sco_coeff.get(), d.sco_ok.get(), nullptr, d.g.get(), d.scaled.get()); + d.I.get(), d.inv_sigma.get(), d.sco_coeff.get(), d.sco_ok.get(), nullptr, /*pool=*/false, d.g.get(), d.scaled.get()); CudaCheck(cudaGetLastError(), "FitPerFrameGKernel launch"); UpdateCorrKernel<<s()>>>(d.n_obs, d.frame.get(), d.prescaling_corr.get(), d.partiality.get(), d.g.get(), d.scaled.get(), d.corr.get()); @@ -1497,7 +1513,7 @@ void RotationScaleMergeGPU::ResetFullsScale() { CudaCheck(cudaStreamSynchronize(impl_->s()), "reset fulls scale sync"); } -void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality) { +void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality, bool pool) { DeviceGuard guard(impl_->device, impl_->available); auto &d = *impl_; const int nf = d.n_fulls; @@ -1523,7 +1539,7 @@ void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality) { CudaCheck(cudaGetLastError(), "PrepScaleObsKernel launch"); FitPerFrameGKernel<<s()>>>(d.n_frames, d.f_frame_start.get(), d.f_frame_count.get(), d.f_I.get(), d.f_inv_sigma.get(), - d.f_sco_coeff.get(), d.f_sco_ok.get(), d.f_frame_perm.get(), d.g.get(), d.scaled.get()); + d.f_sco_coeff.get(), d.f_sco_ok.get(), d.f_frame_perm.get(), pool, d.g.get(), d.scaled.get()); CudaCheck(cudaGetLastError(), "FitPerFrameGKernel launch"); UpdateCorrKernel<<s()>>>(nf, d.f_frame.get(), d.f_rlp.get(), d.f_partiality.get(), d.g.get(), d.scaled.get(), d.f_corr.get()); diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.h b/image_analysis/scale_merge/RotationScaleMergeGPU.h index 766e2f640..98aa1b18a 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.h +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.h @@ -184,7 +184,9 @@ public: // Run `iters` of the Unity scaling loop on the resident fulls (reduce group means -> per-frame LS G // -> update corr), in place on the fulls' working corr. Requires SetFullsFrameCSR + SetFullsGroups // and ResetFullsScale. - void ScaleFulls(int iters, double min_partiality); + // pool: a frame with fewer than MIN_REFLECTIONS fulls is fitted over its neighbours + // (RotationScaleMerge::PoolHalfWidth). + void ScaleFulls(int iters, double min_partiality, bool pool); // The fulls' counterpart of SmoothCorr: f_corr[i] *= ratio[f_frame[i]] where apply[f]. void SmoothFullsCorr(const uint8_t *apply, const double *ratio); diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp index 4eb457f6b..9cb7403f5 100644 --- a/rugnux/rugnux_cli.cpp +++ b/rugnux/rugnux_cli.cpp @@ -157,6 +157,7 @@ void print_usage() { std::cout << " --write-process-h5 Also write the (large) _process.h5 when merging (default: only .mtz/.cif when merging)" << std::endl; std::cout << " --finalist-ledger Report the full-resolution evidence for each space group the search considered, not only the one it adopted (report-only; the decision is unchanged)" << std::endl; std::cout << " --developer Write the full _report.txt: the pipeline-internal keys (the anisotropy gate, the space-group candidate and operator tables, the model-fit null, the sweep internals) and the long explanations, which the default report leaves out. Nothing is computed differently - the same report, rendered in full" << std::endl; + std::cout << " --no-scale-partials rot3d: do not fit per-frame scales on the partials; the per-frame scale comes from the fulls (scale-fulls) alone, as in XDS" << std::endl; std::cout << " --smooth-g[=deg] rot3d: smooth per-frame scale G over a deg-degree rotation range (XDS DELPHI-like) before the combine (default: 5 for rot3d; 0 = off)" << std::endl; std::cout << " --relative-b[=deg] rot3d: fit a per-batch relative-B (beyond the single decay slope) over deg-degree batches; cross-validated (default: 10 deg when bare; off otherwise)" << std::endl; std::cout << " --no-scaling-corrections rot3d: disable the (default-on) decay + absorption + modulation correction surfaces fitted on the fulls after scale-fulls" << std::endl; @@ -282,6 +283,7 @@ enum { OPT_MOSAICITY, OPT_PREDICTION_MOSAICITY, OPT_SMOOTH_G, + OPT_NO_SCALE_PARTIALS, OPT_RELATIVE_B, OPT_NO_SCALING_CORRECTIONS, OPT_NO_EXPECTED_VARIANCE_MERGE, @@ -351,6 +353,7 @@ static option long_options[] = { {"finalist-ledger", no_argument, nullptr, OPT_FINALIST_LEDGER}, {"developer", no_argument, nullptr, OPT_DEVELOPER}, {"smooth-g", optional_argument, nullptr, OPT_SMOOTH_G}, + {"no-scale-partials", no_argument, nullptr, OPT_NO_SCALE_PARTIALS}, {"relative-b", optional_argument, nullptr, OPT_RELATIVE_B}, {"no-scaling-corrections", no_argument, nullptr, OPT_NO_SCALING_CORRECTIONS}, {"no-expected-variance-merge", no_argument, nullptr, OPT_NO_EXPECTED_VARIANCE_MERGE}, @@ -773,6 +776,7 @@ static int RunRugnux(int argc, char **argv) { std::optional beam_x, beam_y, detector_distance_mm, wavelength_A, rot1_rad, rot2_rad, rot3_rad, polarization_factor; bool detector_mirror_y = false; int64_t detector_quarter_turns = 0; + bool no_scale_partials = false; // --no-scale-partials (prototype) std::optional smooth_g_deg_arg; // --smooth-g[=deg]; default 5 deg for rot3d, 0 (off) otherwise std::optional relative_b_deg_arg; // --relative-b[=deg]; per-batch relative-B width, 0 (off) unless given bool no_scaling_corrections = false; // --no-scaling-corrections: disable rot3d decay+absorption+modulation surfaces @@ -1207,6 +1211,9 @@ static int RunRugnux(int argc, char **argv) { case OPT_DEVELOPER: provenance.developer = true; break; + case OPT_NO_SCALE_PARTIALS: + no_scale_partials = true; + break; case OPT_SMOOTH_G: smooth_g_deg_arg = optarg ? parse_double_arg(optarg, "--smooth-g", logger) : SMOOTH_G_DEFAULT_DEG; break; @@ -1652,6 +1659,7 @@ static int RunRugnux(int argc, char **argv) { outlier_reject_nsigma.value_or(scaling_settings.GetOutlierRejectNsigma())); scaling_settings.ScaleFulls(scale_fulls_arg.value_or(scaling_settings.GetScaleFulls())); scaling_settings.SmoothGDegrees(smooth_g_deg_arg.value_or(scaling_settings.GetSmoothGDegrees())); + if (no_scale_partials) scaling_settings.ScalePartials(false); scaling_settings.RelativeBDegrees(relative_b_deg_arg.value_or(0.0)); // opt-in only; default off if (no_scaling_corrections) scaling_settings.CorrectionSurfaces(false); @@ -2538,6 +2546,7 @@ static int RunRugnux(int argc, char **argv) { ScalingSettings scaling_settings = RugnuxDefaultScalingSettings(rotation_indexing); scaling_settings.ScaleFulls(scale_fulls); scaling_settings.SmoothGDegrees(smooth_g_deg_arg.value_or(scaling_settings.GetSmoothGDegrees())); + if (no_scale_partials) scaling_settings.ScalePartials(false); scaling_settings.RelativeBDegrees(relative_b_deg_arg.value_or(0.0)); // opt-in only; default off if (no_scaling_corrections) scaling_settings.CorrectionSurfaces(false); diff --git a/tests/RotationScaleWalkTest.cpp b/tests/RotationScaleWalkTest.cpp index 562d46a24..f01334b3f 100644 --- a/tests/RotationScaleWalkTest.cpp +++ b/tests/RotationScaleWalkTest.cpp @@ -5,6 +5,7 @@ #include #include "../rugnux/Rugnux.h" +#include "../image_analysis/scale_merge/RotationScaleMerge.h" namespace { // A synthetic sweep whose stage turned `true_scale` times the stored angles. Scored at a scale k, @@ -92,3 +93,15 @@ TEST_CASE("WalkRotationScale_StoredAnglesStand", "[RotationScale]") { CHECK(sweep.index_calls == 0); } } + +TEST_CASE("PoolHalfWidth_FitsSparseFramesOverTheirNeighbours", "[RotationScale]") { + // 20 fulls of its own: fitted on its own. + CHECK(RotationScaleMerge::PoolHalfWidth({0, 20, 0}, 1) == 0); + // 5 per frame: two frames either side bring 25 >= 20. + const std::vector five(11, 5); + CHECK(RotationScaleMerge::PoolHalfWidth(five, 5) == 2); + // At the end of the sweep the window grows on the one side there is. + CHECK(RotationScaleMerge::PoolHalfWidth(five, 0) == 3); + // A sweep that never holds enough stops at its length. + CHECK(RotationScaleMerge::PoolHalfWidth({1, 1, 1}, 1) == 3); +} -- 2.54.0 From 20ab92fcaba21293209c67daefb5fc97832e235f Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 00:05:36 +0200 Subject: [PATCH 24/97] Build: no FMA contraction; first-pass cell refinement independent of SIMD width The same source built with and without -march=x86-64-v3 gave different results on battery sets (9zmu axis-harmonic supercell arbiter fired in one build only; 8rud resolution cut 1.69 vs 1.70 A; myob_x06da_powder_1 cut 1.07 vs 1.42 A; 7mzt short-axis first pass; lcystine_x10sa_20keV indexed vs no lattice), against the rule that no decision may depend on compiler flags. Two mechanisms, found by building the merged tree four ways (baseline, x86-64-v2, x86-64-v3, x86-64-v3 -mno-fma) with and without -ffp-contract=off and comparing p.mtz: - FMA contraction. GCC contracts a*b+c whenever the target has FMA. With -ffp-contract=off the x86-64-v3 build gives a p.mtz byte-identical to the baseline build on 8 of 9 sets (7mzt, 8rud, 9zmu, insu_I_x06da_5keV_2, myob_x06da_powder_1, myob_x10sa, cytc_x10sa, thau_x10sa_16keV), on both the GPU and the CPU build. Cost: none measurable (user core-s, CPU build, x86-64-v3 vs the same with -ffp-contract=off: 1877/1874, 3285/3253, 2212/2195 on myob/cytc/thau; GPU likewise within noise). Set project-wide for C, C++ and CUDA host code; MSVC does not contract under /fp:precise. - SIMD width. lcystine still differed: baseline and x86-64-v2 (128-bit) agreed, x86-64-v3 with or without FMA (256-bit) disagreed - Eigen's HouseholderQR in the FFT indexer's candidate refinement (PostIndexingRefinement.cpp) reduces column norms over all spots in packets of the target width. Replaced by the 3x3 normal equations summed in spot order in double. All four builds now agree on all nine sets. Changes results of the default x86-64-v3 build (contraction off); lcystine_x10sa_20keV now gives no lattice in every build (its first pass is a knife-edge: 0/60 vs 9/60 validation frames before). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- CMakeLists.txt | 12 +++++++++++ .../indexing/PostIndexingRefinement.cpp | 20 +++++++++++++++---- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 1caea9809..dfc4da29e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -41,6 +41,18 @@ IF (MSVC) ADD_COMPILE_OPTIONS($<$:-Xcompiler=/Zc:preprocessor>) ENDIF() +# No contraction of a*b+c into a fused multiply-add. GCC contracts whenever the target has FMA, so a +# -march=x86-64-v3 build (CI, production) and a baseline build rounded differently, and knife-edge +# decisions - which candidate cell a first pass indexes with, a resolution cut, a symmetry gate - +# came out differently from the same source and data. Off, every x86-64 level computes the same bits +# (measured: identical p.mtz on nine battery sets, no measurable time cost), and aarch64, where FMA is +# always there and GCC contracts by default, follows the same arithmetic. MSVC does not contract under +# its default /fp:precise. Host code of .cu sources gets it through nvcc; device code is not affected. +IF (NOT MSVC) + ADD_COMPILE_OPTIONS($<$:-ffp-contract=off>) + ADD_COMPILE_OPTIONS($<$:-Xcompiler=-ffp-contract=off>) +ENDIF() + SET(JFJOCH_WRITER_ONLY OFF CACHE BOOL "Compile HDF5 writer only") SET(JFJOCH_INSTALL_DRIVER_SOURCE OFF CACHE BOOL "Install kernel driver source (ignored if building writer only; necessary for RPM building)") SET(JFJOCH_USE_CUDA ON CACHE BOOL "Compile Jungfraujoch with CUDA") diff --git a/image_analysis/indexing/PostIndexingRefinement.cpp b/image_analysis/indexing/PostIndexingRefinement.cpp index d8d210eb0..a31bb7de3 100644 --- a/image_analysis/indexing/PostIndexingRefinement.cpp +++ b/image_analysis/indexing/PostIndexingRefinement.cpp @@ -82,7 +82,6 @@ namespace { const unsigned nspots = spots.rows(); const unsigned ncells = scores.rows(); VectorX below{nspots}; - MatrixX3 sel{nspots, 3u}; Mx3 resid{nspots, 3u}; Mx3 miller{nspots, 3u}; M3 cell; @@ -111,9 +110,22 @@ namespace { break; threshold *= cifssr.threshold_contraction; - sel.colwise() = below; - HouseholderQR qr{sel.select(spots, .0f)}; - cell = qr.solve(sel.select(miller, .0f)); + // Least squares spots * cell = miller over the spots below the threshold, by the normal + // equations summed in spot order in double. A QR over all the spots reduces its column + // norms in SIMD packets of the target's width, so its cell - and which candidate a + // borderline first pass indexed with - depended on -march; these sums do not. + Matrix3d ata = Matrix3d::Zero(); + Matrix3d atb = Matrix3d::Zero(); + for (unsigned i = 0; i < nspots; i++) { + if (!below[i]) + continue; + for (int r = 0; r < 3; r++) + for (int c = 0; c < 3; c++) { + ata(r, c) += double(spots(i, r)) * double(spots(i, c)); + atb(r, c) += double(spots(i, r)) * double(miller(i, c)); + } + } + cell = (ata.inverse() * atb).cast(); } resid = CalculateResiduals(spots, cell); -- 2.54.0 From ba92417210e06899810e117e3b9778e912fd3fda Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 00:34:39 +0200 Subject: [PATCH 25/97] RotationScaleMerge: per-frame scale from the fulls alone on sparse sweeps (data-decided) Replaces the prototype --no-scale-partials switch with a rule read off the data: a merge whose sweep holds fewer than 50 rocking events per frame takes its per-frame scale from the fulls alone (scale-fulls, a sparse frame fitted over its neighbours - PoolHalfWidth), and logs that it did. The rocking-event walk already run for the smoothing window now also returns its event count. Within one rocking curve a partial's scale and an error of the partiality model are the same thing; on a sparse fine-sliced sweep the partial fit takes the one for the other and imprints an hkl-dependent bias common to all equivalents. Measured populations: small-molecule sweeps 2.6-23 events per frame, protein sets 84-900; on the proteins the fulls-only scale leaves model R-free unchanged (+-0.002 over the smoke tier's open-arm sets) but lowers ISa, so they keep the partial scale. p.mtz md5 unchanged on the three profiling sets, GPU and CPU builds. SHELXL against the COD structures, R1(>4sig) rc174 -> this: aspirin 20 keV 0.096 -> 0.062, aspirin 25 keV 0.094 -> 0.046, citric acid 0.161 -> 0.109, HEPES 0.090 -> 0.070 (XDS 0.030-0.038); SHELXL's weight a comes off its 0.2 cap on all four. CPU and GPU paths agree. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- common/ScalingSettings.cpp | 9 ---- common/ScalingSettings.h | 3 -- .../scale_merge/RotationScaleMerge.cpp | 42 +++++++++++++------ .../scale_merge/RotationScaleMerge.h | 7 +++- rugnux/rugnux_cli.cpp | 9 ---- 5 files changed, 35 insertions(+), 35 deletions(-) diff --git a/common/ScalingSettings.cpp b/common/ScalingSettings.cpp index 4fabbd44d..2016afba5 100644 --- a/common/ScalingSettings.cpp +++ b/common/ScalingSettings.cpp @@ -208,15 +208,6 @@ double ScalingSettings::GetSmoothGDegrees() const { return smooth_g_deg; } -ScalingSettings &ScalingSettings::ScalePartials(bool input) { - scale_partials = input; - return *this; -} - -bool ScalingSettings::GetScalePartials() const { - return scale_partials; -} - ScalingSettings &ScalingSettings::RelativeBDegrees(double input) { if (input < 0) throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Relative-B batch width must be non-negative"); diff --git a/common/ScalingSettings.h b/common/ScalingSettings.h index e9b2cd72e..198c6366c 100644 --- a/common/ScalingSettings.h +++ b/common/ScalingSettings.h @@ -98,7 +98,6 @@ class ScalingSettings { // degrees (like XDS DELPHI), converted to an odd frame window from the oscillation step; this keeps // the smoothing physical (independent of frame slicing). 0 = off. A no-op without rot3d. double smooth_g_deg = 0.0; - bool scale_partials = true; // Per-batch relative-B on the rot3d fulls (beyond the single global decay slope): bin frames into // rotation-range batches of this width in degrees and refine one relative Debye-Waller B per batch, so @@ -144,7 +143,6 @@ public: ScalingSettings& IceMinScore(float input); ScalingSettings& IceMinSpotRatio(float input); ScalingSettings& SmoothGDegrees(double input); - ScalingSettings& ScalePartials(bool input); ScalingSettings& RelativeBDegrees(double input); ScalingSettings& RfreeFraction(double input); @@ -184,7 +182,6 @@ public: [[nodiscard]] float GetIceMinScore() const; [[nodiscard]] float GetIceMinSpotRatio() const; [[nodiscard]] double GetSmoothGDegrees() const; - [[nodiscard]] bool GetScalePartials() const; [[nodiscard]] double GetRelativeBDegrees() const; [[nodiscard]] double GetRfreeFraction() const; diff --git a/image_analysis/scale_merge/RotationScaleMerge.cpp b/image_analysis/scale_merge/RotationScaleMerge.cpp index 2b71049b4..fef918578 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.cpp +++ b/image_analysis/scale_merge/RotationScaleMerge.cpp @@ -36,6 +36,16 @@ namespace { // These mirror the per-image ScaleOnTheFly / Merge rocking-curve physics verbatim so this flat // implementation is numerically identical - see the comments there for the details. constexpr size_t MIN_REFLECTIONS = 20; // per-frame scale needs at least this many + // Per-frame scaling of the PARTIALS needs a frame to carry many rocking events: within one rocking + // curve a change of scale and an error of the partiality model are the same thing, and only events + // caught at different points of their curves tell them apart. On a fine-sliced sparse sweep the fit + // takes one for the other - G swung 0.23..1.2 with a 180 deg period on a small-molecule sweep whose + // own frame scale varies 0.79..0.99 (XDS), and the merge carried an hkl-dependent bias that the + // equivalents cannot see but a refinement against the known structure does (R1 0.096 -> 0.062 + // without it). Below this the per-frame scale comes from the fulls alone, as XDS takes it. Measured + // on the battery: the small-molecule sweeps hold 2.6 to 23 events per frame, the protein sets 84 to + // 900 - and on the proteins the fulls-only scale leaves model R-free unchanged (+-0.002). + constexpr double MIN_EVENTS_PER_FRAME_SCALE_PARTIALS = 50.0; constexpr int64_t MIN_REFLECTIONS_FOR_IMAGE_CC = 20; // below this a frame's CC means nothing // The per-frame scaling loop has settled when the scales moved less than this between two // iterations, as the rms of |log(G_new/G_old)| over the frames fitted in both. @@ -591,7 +601,7 @@ void RotationScaleMerge::Ingest() { // (see the header). Take its answer now and hand them back. The dump path keeps the full Obs // array: the CPU combine is what writes the dump. if (resident_ingest) { - rocking_event_frames_at_ingest = RockingEventFrames(); + rocking_event_frames_at_ingest = RockingEventFrames(&rocking_events_at_ingest); partials_released = true; } #endif @@ -2473,24 +2483,26 @@ namespace { // image order, bridged by the same max_frame_gap - taken as the median over the run's events. It is // the finest stretch of the sweep this file is allowed to speak about: the partials of one event are // welded into a single full, so two frames closer together than this are not separable observations. -int RotationScaleMerge::RockingEventFrames() const { - if (partials_released) +int RotationScaleMerge::RockingEventFrames(int64_t *n_events) const { + if (partials_released) { + if (n_events) *n_events = rocking_events_at_ingest; return rocking_event_frames_at_ingest; + } if (resident_ingest) // Ingest's own call, before the ingest arrays are handed back return RockingEventFramesOver( [&](int i) { return ingest_rock_ok[i] != 0; }, - [&](int i) { return ingest_image_number[i]; }); + [&](int i) { return ingest_image_number[i]; }, n_events); return RockingEventFramesOver( [&](int i) { const Obs &o = partials[i]; return std::isfinite(o.corr) && o.corr > 0.0f && std::isfinite(o.I) && std::isfinite(o.sigma) && o.sigma > 0.0f; }, - [&](int i) { return partials[i].image_number; }); + [&](int i) { return partials[i].image_number; }, n_events); } template -int RotationScaleMerge::RockingEventFramesOver(UsableFn usable, ImgFn img) const { +int RotationScaleMerge::RockingEventFramesOver(UsableFn usable, ImgFn img, int64_t *n_events) const { // Blocks of runs on all threads, each counting into a histogram of its own. The counts are // integers, so the blocks add up to the serial histogram whatever the split. constexpr int RUNS_PER_BLOCK = 16384; @@ -2525,6 +2537,7 @@ int RotationScaleMerge::RockingEventFramesOver(UsableFn usable, ImgFn img) const hist[w] += h[w]; n_event += h[w]; } + if (n_events) *n_events = n_event; int acc = 0; for (int w = 1; w <= n_frames + 1; ++w) if ((acc += hist[w]) * 2 >= n_event) @@ -5616,7 +5629,7 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st int smooth_window = 1; if (smooth_g_deg > 0.0 && osc_deg > 1e-6) { if (rocking_event_frames_at_start < 0) - rocking_event_frames_at_start = RockingEventFrames(); + rocking_event_frames_at_start = RockingEventFrames(&rocking_events_at_start); smooth_window = std::max(static_cast(std::lround(smooth_g_deg / osc_deg)), SMOOTH_G_MIN_ROCKING_EVENTS * rocking_event_frames_at_start); if (smooth_window % 2 == 0) ++smooth_window; @@ -5624,12 +5637,17 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st "curves span {} frames)", smooth_window, smooth_window * osc_deg, smooth_g_deg, rocking_event_frames_at_start); } - // Without partial scaling every frame keeps G = 1 here and its scale comes from the fulls alone - // (scale-fulls below), as in XDS: a partial's scale and an error of the partiality model are the - // same thing within a rocking curve, and on a sparse fine-sliced sweep the fit takes the one for - // the other. - const bool scale_partials = s.GetScalePartials(); + // Without partial scaling (MIN_EVENTS_PER_FRAME_SCALE_PARTIALS) every frame keeps G = 1 here and + // its scale comes from the fulls alone (scale-fulls below). + if (rocking_event_frames_at_start < 0) + rocking_event_frames_at_start = RockingEventFrames(&rocking_events_at_start); + const double events_per_frame = n_frames > 0 ? static_cast(rocking_events_at_start) / n_frames : 0.0; + const bool scale_partials = events_per_frame >= MIN_EVENTS_PER_FRAME_SCALE_PARTIALS; pool_sparse_fulls = !scale_partials; + if (!scale_partials) + logger.Info("Per-frame scale from the fulls alone: {:.1f} rocking events per frame, under the {:.0f} " + "a partial's scale needs to stay apart from the partiality model", events_per_frame, + MIN_EVENTS_PER_FRAME_SCALE_PARTIALS); ScalingLoopOutcome partial_loop; #ifdef JFJOCH_USE_CUDA if (gpu_active_ && scale_partials) { diff --git a/image_analysis/scale_merge/RotationScaleMerge.h b/image_analysis/scale_merge/RotationScaleMerge.h index 30da42438..c5261da57 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.h +++ b/image_analysis/scale_merge/RotationScaleMerge.h @@ -276,6 +276,7 @@ private: // full-size copy of the partials, gigabytes on a fine-sliced long axis - straight back. bool partials_released = false; int rocking_event_frames_at_ingest = 0; + int64_t rocking_events_at_ingest = 0; std::vector frame_start, frame_count; // CSR ranges of `partials` per frame std::vector frame_cell_ok; // per-frame cell-consistency mask (1 = kept) std::vector finite_ok; // per-obs AcceptReflection finiteness (immutable; 1 = kept) @@ -306,6 +307,7 @@ private: // RockingEventFrames at the start of a Run, where corr has just been restored to corr_ingested - so // the same for every Run on the same ingest; -1 until the first Run takes it. int rocking_event_frames_at_start = -1; + int64_t rocking_events_at_start = 0; // the events that walk counted // Raw-hkl ordering, built ONCE by Ingest and reused: `perm` lists partial indices sorted by // (raw h,k,l, image_number); each distinct raw hkl is a contiguous run [rawrun_start, +count) of it. @@ -626,9 +628,10 @@ private: // The frames one rocking event spans, as the combine cuts them: the median over the run's events. // Walked on the partials, because the combine also runs on the GPU and a full keeps only the frame // of its peak partial. - [[nodiscard]] int RockingEventFrames() const; + // n_events (if given) receives how many rocking events the walk counted. + [[nodiscard]] int RockingEventFrames(int64_t *n_events = nullptr) const; template - [[nodiscard]] int RockingEventFramesOver(UsableFn usable, ImgFn img) const; + [[nodiscard]] int RockingEventFramesOver(UsableFn usable, ImgFn img, int64_t *n_events) const; // Per-batch delta-CC1/2 on the corrected fulls: measure what keeping each batch of the sweep costs // the merged intensities, convict the batches that cost significantly, slide the conviction's edges // onto the frames that carry it, and turn the result into the disposition ledger. Fills diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp index 9cb7403f5..4eb457f6b 100644 --- a/rugnux/rugnux_cli.cpp +++ b/rugnux/rugnux_cli.cpp @@ -157,7 +157,6 @@ void print_usage() { std::cout << " --write-process-h5 Also write the (large) _process.h5 when merging (default: only .mtz/.cif when merging)" << std::endl; std::cout << " --finalist-ledger Report the full-resolution evidence for each space group the search considered, not only the one it adopted (report-only; the decision is unchanged)" << std::endl; std::cout << " --developer Write the full _report.txt: the pipeline-internal keys (the anisotropy gate, the space-group candidate and operator tables, the model-fit null, the sweep internals) and the long explanations, which the default report leaves out. Nothing is computed differently - the same report, rendered in full" << std::endl; - std::cout << " --no-scale-partials rot3d: do not fit per-frame scales on the partials; the per-frame scale comes from the fulls (scale-fulls) alone, as in XDS" << std::endl; std::cout << " --smooth-g[=deg] rot3d: smooth per-frame scale G over a deg-degree rotation range (XDS DELPHI-like) before the combine (default: 5 for rot3d; 0 = off)" << std::endl; std::cout << " --relative-b[=deg] rot3d: fit a per-batch relative-B (beyond the single decay slope) over deg-degree batches; cross-validated (default: 10 deg when bare; off otherwise)" << std::endl; std::cout << " --no-scaling-corrections rot3d: disable the (default-on) decay + absorption + modulation correction surfaces fitted on the fulls after scale-fulls" << std::endl; @@ -283,7 +282,6 @@ enum { OPT_MOSAICITY, OPT_PREDICTION_MOSAICITY, OPT_SMOOTH_G, - OPT_NO_SCALE_PARTIALS, OPT_RELATIVE_B, OPT_NO_SCALING_CORRECTIONS, OPT_NO_EXPECTED_VARIANCE_MERGE, @@ -353,7 +351,6 @@ static option long_options[] = { {"finalist-ledger", no_argument, nullptr, OPT_FINALIST_LEDGER}, {"developer", no_argument, nullptr, OPT_DEVELOPER}, {"smooth-g", optional_argument, nullptr, OPT_SMOOTH_G}, - {"no-scale-partials", no_argument, nullptr, OPT_NO_SCALE_PARTIALS}, {"relative-b", optional_argument, nullptr, OPT_RELATIVE_B}, {"no-scaling-corrections", no_argument, nullptr, OPT_NO_SCALING_CORRECTIONS}, {"no-expected-variance-merge", no_argument, nullptr, OPT_NO_EXPECTED_VARIANCE_MERGE}, @@ -776,7 +773,6 @@ static int RunRugnux(int argc, char **argv) { std::optional beam_x, beam_y, detector_distance_mm, wavelength_A, rot1_rad, rot2_rad, rot3_rad, polarization_factor; bool detector_mirror_y = false; int64_t detector_quarter_turns = 0; - bool no_scale_partials = false; // --no-scale-partials (prototype) std::optional smooth_g_deg_arg; // --smooth-g[=deg]; default 5 deg for rot3d, 0 (off) otherwise std::optional relative_b_deg_arg; // --relative-b[=deg]; per-batch relative-B width, 0 (off) unless given bool no_scaling_corrections = false; // --no-scaling-corrections: disable rot3d decay+absorption+modulation surfaces @@ -1211,9 +1207,6 @@ static int RunRugnux(int argc, char **argv) { case OPT_DEVELOPER: provenance.developer = true; break; - case OPT_NO_SCALE_PARTIALS: - no_scale_partials = true; - break; case OPT_SMOOTH_G: smooth_g_deg_arg = optarg ? parse_double_arg(optarg, "--smooth-g", logger) : SMOOTH_G_DEFAULT_DEG; break; @@ -1659,7 +1652,6 @@ static int RunRugnux(int argc, char **argv) { outlier_reject_nsigma.value_or(scaling_settings.GetOutlierRejectNsigma())); scaling_settings.ScaleFulls(scale_fulls_arg.value_or(scaling_settings.GetScaleFulls())); scaling_settings.SmoothGDegrees(smooth_g_deg_arg.value_or(scaling_settings.GetSmoothGDegrees())); - if (no_scale_partials) scaling_settings.ScalePartials(false); scaling_settings.RelativeBDegrees(relative_b_deg_arg.value_or(0.0)); // opt-in only; default off if (no_scaling_corrections) scaling_settings.CorrectionSurfaces(false); @@ -2546,7 +2538,6 @@ static int RunRugnux(int argc, char **argv) { ScalingSettings scaling_settings = RugnuxDefaultScalingSettings(rotation_indexing); scaling_settings.ScaleFulls(scale_fulls); scaling_settings.SmoothGDegrees(smooth_g_deg_arg.value_or(scaling_settings.GetSmoothGDegrees())); - if (no_scale_partials) scaling_settings.ScalePartials(false); scaling_settings.RelativeBDegrees(relative_b_deg_arg.value_or(0.0)); // opt-in only; default off if (no_scaling_corrections) scaling_settings.CorrectionSurfaces(false); -- 2.54.0 From 1baf926066d4bd9f617a0a84c20a046088b09ddf Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 00:48:35 +0200 Subject: [PATCH 26/97] RotationScaleMerge: refit the error model about the merge's own mean The error model was fitted once, on deviations from a mean that weighs every full by its COUNTING variance, and the outlier test's median took the same weights. Where equivalents disagree beyond counting statistics, the low-count observations dominate that centre: on a strongly absorbing crystal (YAG, Ia-3d, equivalents spread over a factor 100 after scaling) the mean of (0 4 0) sat at 21k among observations from 11k to 1.9M, a ran into its bound (100), b came out at 800% internally, and the six-sigma test about the biased median removed 63% of the observations - the strong ones. Now, after the first fit, the model is refitted about the model-weighted mean (each full weighted by the variance the fitted model gives it at the reflection's mean) until a and b settle, and the rejection median takes the same weights. Where counting statistics are right nothing moves. Following Blessing (1997) J. Appl. Cryst. 30, 421-426. GPU path: the refitted means are uploaded (SetEmMean). Measured (SHELXL R1(>4sigma) against the COD model, sm-a's harness; rc174 scaling): YAG 0.556 -> 0.127 (XDS 0.083 merged), rejected 8055 -> 53, normalised deviations calibrated (median |z| 0.62-0.69 in every intensity decile); aspirin 20 keV 0.0964 -> 0.0958; citric acid 0.161 -> 0.159; HEPES 0.0903 -> 0.0899; L-cystine 25 keV 0.1425 -> 0.1456; aspirin 25 keV 0.094 -> 0.106 (fixed-model R1 0.107 -> 0.173): its strong equivalents split into two frame-dependent populations from the per-frame partial scaling (sm-a's dq-smallmol), which the old under-sized sigmas happened to cut; with that scaling fixed (f69339ce6 + pooling, --no-scale-partials) this change is neutral to better on every small molecule (aspirin 20 .0618 -> .0586, aspirin 25 .0456 -> .0454, citric .1093 -> .1006, HEPES .0703 -> .0697, YAG .649 -> .222; SHELXL GooF ~1.1). => ship together with the scaling fix. Proteins (GPU full runs): CC1/2 and R_meas unchanged to 0.002; ISa myob 9.06 -> 8.20, thau 52.5 -> 47.5, cytc 25.8 -> 25.4, lyso 29.4 -> 29.4 (still above XDS's 5.2 / 44.5 / 31.8 / 28.3 except cytc). CPU build gives the same statistics as the GPU build on aspirin 20 keV and myob. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- .../scale_merge/RotationScaleMerge.cpp | 111 +++++++++++++++++- .../scale_merge/RotationScaleMergeGPU.cu | 8 ++ .../scale_merge/RotationScaleMergeGPU.h | 3 + 3 files changed, 118 insertions(+), 4 deletions(-) diff --git a/image_analysis/scale_merge/RotationScaleMerge.cpp b/image_analysis/scale_merge/RotationScaleMerge.cpp index 81dabcd22..edccd2d8c 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.cpp +++ b/image_analysis/scale_merge/RotationScaleMerge.cpp @@ -4448,6 +4448,104 @@ RotationScaleMerge::Result RotationScaleMerge::MergeAndStats(int n_groups, bool resid * resid / factor, o.d}); } } + fit_error_model(samples); + + // ---- The error model refitted about the mean the merge takes. ---- + // The fit above is made about a mean that weighs each full by its COUNTING variance. Where the + // equivalents of a reflection agree to within counting statistics that is the merge's own centre, + // and the model it gives is the model the merge applies. Where they do not - a systematic error + // the counting variance does not know about - it is the wrong centre, and in the worst way: the + // full with the fewest counts has the smallest variance, so the mean, and every deviation the + // model is fitted on, is pulled toward the observations that read lowest. Measured on a strongly + // absorbing crystal whose equivalents spread over a factor 100: the mean of (0 4 0) sat at 21k + // among observations from 11k to 1.9M, the deviations of the rest read as hundreds of counting + // sigmas, a ran into its bound and the six-sigma test about the median taken with the same + // weights deleted 63% of the observations - the strong ones (SHELXL R1 0.50 on that merge). + // So refit about the mean the MERGE takes - each full weighted by the variance the fitted model + // gives it, evaluated at the reflection's mean - until the model stops moving; the outlier + // test's median below takes the same weights. Where counting statistics were right the first fit + // is the answer and nothing moves; where b dominates, the weights even out, as the model says. + // Following Blessing (1997) J. Appl. Cryst. 30, 421-426: the centre an outlier is judged from + // is weighted as the merge weighs. + if (error_model_active) { + // Fulls in group order, so each pass over a group's members is one contiguous walk and the + // groups split across threads without every thread reading every full. + std::vector gstart(n_groups + 1, 0); + for (int i = 0; i < n_full; ++i) + if (mf.group[i] >= 0 && cnt[mf.group[i]] >= 2) ++gstart[mf.group[i] + 1]; + for (int g = 0; g < n_groups; ++g) gstart[g + 1] += gstart[g]; + std::vector member(gstart[n_groups]); + { + std::vector fill(gstart.begin(), gstart.end() - 1); + for (int i = 0; i < n_full; ++i) + if (mf.group[i] >= 0 && cnt[mf.group[i]] >= 2) member[fill[mf.group[i]]++] = i; + } + std::vector next_mean; + std::vector next; + constexpr int REFIT_ROUNDS = 10; + for (int round = 0; round < REFIT_ROUNDS; ++round) { + const double a = error_model_a, b = error_model_b; + const auto model_var = [&](int i, double mean) { + const double sc = static_cast(mf.sigma[i]) * mf.corr[i]; + return a * counting_variance(fulls[i], mean, sc * sc) + (b * mean) * (b * mean); + }; + std::vector> part(ThreadsForWork(member.size(), nthreads)); + const int nt = static_cast(part.size()); + next_mean.assign(n_groups, NAN); + // Each group's mean and samples come from its own members in index order, so the result + // does not depend on how the groups were split. + ParallelFor(nt, nt, [&](int t) { + const int g0 = static_cast(static_cast(n_groups) * t / nt); + const int g1 = static_cast(static_cast(n_groups) * (t + 1) / nt); + for (int g = g0; g < g1; ++g) { + if (gstart[g + 1] - gstart[g] < 2 || !std::isfinite(em_mean[g])) continue; + double sw = 0.0, swI = 0.0, swh[2] = {0.0, 0.0}, swIh[2] = {0.0, 0.0}; + int nh[2] = {0, 0}; + for (int q = gstart[g]; q < gstart[g + 1]; ++q) { + const int i = member[q]; + const double v = model_var(i, em_mean[g]); + if (!(v > 0.0)) continue; + const double I_corr = static_cast(mf.I[i]) * mf.corr[i]; + sw += 1.0 / v; swI += I_corr / v; + if (merge_friedel && group_has_hands[g]) { + swh[obs_hand[i]] += 1.0 / v; swIh[obs_hand[i]] += I_corr / v; nh[obs_hand[i]]++; + } + } + if (!(sw > 0.0)) continue; + const double mean = swI / sw; + next_mean[g] = mean; + // The samples about the hand's own mean where it has two of its own, as above. + for (int q = gstart[g]; q < gstart[g + 1]; ++q) { + const int i = member[q]; + const double v = model_var(i, em_mean[g]); + if (!(v > 0.0)) continue; + const int hh = obs_hand[i]; + const bool on_hand = merge_friedel && group_has_hands[g] && nh[hh] >= 2 && swh[hh] > 0.0; + const double centre = on_hand ? swIh[hh] / swh[hh] : mean; + const double factor = 1.0 - (1.0 / v) / (on_hand ? swh[hh] : sw); + if (factor < 0.05) continue; + const double sc = static_cast(mf.sigma[i]) * mf.corr[i]; + const double resid = static_cast(mf.I[i]) * mf.corr[i] - centre; + part[t].push_back({counting_variance(fulls[i], mean, sc * sc), centre * centre, + resid * resid / factor, mf.d[i]}); + } + } + }); + for (int g = 0; g < n_groups; ++g) + if (!std::isfinite(next_mean[g])) next_mean[g] = em_mean[g]; + next.clear(); + for (auto &v : part) next.insert(next.end(), v.begin(), v.end()); + em_mean.swap(next_mean); + samples.swap(next); + fit_error_model(samples); + if (std::fabs(error_model_a - a) <= 1e-3 * a + && std::fabs(error_model_b - b) <= 1e-3 * std::max(b, 1e-6)) + break; + } +#ifdef JFJOCH_USE_CUDA + if (use_gpu_merge) gpu_->SetEmMean(em_mean.data()); +#endif + } // Per-group outlier-rejection median of I*corr (host both paths - a per-group median is awkward on // the GPU; cheap here, cnt >= 3 filter from the em pass). Fed to the merge accumulate. @@ -4510,15 +4608,21 @@ RotationScaleMerge::Result RotationScaleMerge::MergeAndStats(int n_groups, bool // Inverse-variance WEIGHTED median: a frame the crystal barely diffracted on is scaled up by // 1/G together with its sigma, so a plain median lets two such observations outvote one // well-measured one - and the test below then rejects the well-measured one against its - // own small sigma. - std::vector> iv(start[n_sets]); // (I*corr, 1/(sigma*corr)^2) + // own small sigma. The variance is the model's, at the reflection's mean - the weight the + // merge gives the observation (see the centre refit above). + std::vector> iv(start[n_sets]); // (I*corr, 1/model variance) { std::vector fill(start.begin(), start.end() - 1); for (int i = 0; i < n_full; ++i) { const int g = mf.group[i]; if (g < 0) continue; const float sc = mf.sigma[i] * mf.corr[i]; - const std::pair v{mf.I[i] * mf.corr[i], sc > 0.0f ? 1.0f / (sc * sc) : 0.0f}; + const double mean = std::isfinite(em_mean[g]) ? em_mean[g] : static_cast(mf.I[i]) * mf.corr[i]; + const double mv = error_model_active + ? error_model_a * counting_variance(fulls[i], mean, static_cast(sc) * sc) + + (error_model_b * mean) * (error_model_b * mean) + : static_cast(sc) * sc; + const std::pair v{mf.I[i] * mf.corr[i], mv > 0.0 ? static_cast(1.0 / mv) : 0.0f}; if (cnt[g] >= 3) iv[fill[g]++] = v; if (!pair_needed.empty() && pair_needed[pair_of_group[g]]) iv[fill[n_groups + pair_of_group[g]]++] = v; @@ -4545,7 +4649,6 @@ RotationScaleMerge::Result RotationScaleMerge::MergeAndStats(int n_groups, bool reject_median[g] = cnt[g] >= 3 ? set_median[g] : !pair_needed.empty() ? set_median[n_groups + pair_of_group[g]] : NAN; } - fit_error_model(samples); } // The full's sigma under the error model, with the variance evaluated at intensity I_for_b. diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.cu b/image_analysis/scale_merge/RotationScaleMergeGPU.cu index 8a0e97f85..60a2e8d76 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.cu +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.cu @@ -1110,6 +1110,14 @@ void RotationScaleMergeGPU::SetFrameCellOk(const uint8_t *frame_cell_ok) { // The per-group inv-var mean (em_mean) + the per-full leverage-corrected error-model samples over the // resident+scaled fulls. Stashes the filter context for the later MergeAccum/MergeRmeas calls. +void RotationScaleMergeGPU::SetEmMean(const double *em_mean) { + DeviceGuard guard(impl_->device, impl_->available); + auto &d = *impl_; + if (d.n_groups > 0) + CopyAndWait(d.m_em_mean.get(), em_mean, size_t(d.n_groups) * sizeof(double), cudaMemcpyHostToDevice, + impl_->s(), "ul em_mean"); +} + void RotationScaleMergeGPU::MergeEmSamples(bool for_search, double min_partiality, const uint8_t *hand, const uint8_t *has_hands, double *em_mean_out, int32_t *cnt_out, double *s2_out, diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.h b/image_analysis/scale_merge/RotationScaleMergeGPU.h index 766e2f640..17b56c520 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.h +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.h @@ -103,6 +103,9 @@ public: // half-set weights multiplied by it. Requires MergeEmSamples first (em_mean resident). // reject_var_add (n_groups) widens the pooled cut by the shell's own measured Bijvoet // variance; null leaves the plain n-sigma test. + // Replace the per-group means MergeEmSamples left on the device (n_groups values): the merge's + // model sigmas are evaluated at them. + void SetEmMean(const double *em_mean); void MergeAccum(double error_model_a, double error_model_b, bool error_model_active, bool reject_outliers, double reject_nsigma, const float *reject_median, const float *reject_var_add, -- 2.54.0 From 7eed4a5e0b6584fe9d7ad593df4394c3ee18a167 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 00:57:06 +0200 Subject: [PATCH 27/97] Battery: SHELXL refinement of each small molecule's COD structure, reported per run shelx_check.py refines the row's published structure (manifest key "cod", cached from the Crystallography Open Database into the site's cod_cache) against rugnux's p.hkl with one fixed recipe: data reindexed into the COD setting (lowest-R1 integer matrix), non-H anisotropic, H fixed, EXTI, MERG 2, three rounds of SHELXL's suggested weights. It records R1/wR2/GooF/EXTI/WGHT/residual density/R(int)/R(sigma)/K of the strongest bin and a fixed-model R1(F) (|Fc| of the COD model as published, gemmi). Reported, never scored. The report gets a small-molecule table; compare lists SHELXL R1/wR2/GooF/EXTI deltas; report/compare fill the check in for older runs. COD entries matched by Niggli-reduced cell and space group: aspirin 7050897, citric acid 5000063, HEPES 2224210, YAG 2003066, L-cystine 1513328 (2005, replaces the 1959 model for refinement), cytidine 2001311, 3,5-dinitrobenzoic acid 4510615, L-alanine 2104782, metformin HCl 2108029, NiCl2(dppe) 2012031. cuhf2 has no reference cell and no match: left without one. SHELXL is called, not shipped (site key "shelxl" or PATH; it comes with CCP4). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- tools/battery/README.md | 34 +++- tools/battery/battery.py | 15 +- tools/battery/inhouse.json | 14 +- tools/battery/open.json | 10 +- tools/battery/report.py | 71 +++++++- tools/battery/shelx_check.py | 290 ++++++++++++++++++++++++++++++++ tools/battery/site.example.json | 2 + 7 files changed, 421 insertions(+), 15 deletions(-) create mode 100644 tools/battery/shelx_check.py diff --git a/tools/battery/README.md b/tools/battery/README.md index eae047c93..ce514aab5 100644 --- a/tools/battery/README.md +++ b/tools/battery/README.md @@ -180,6 +180,8 @@ export `JFJOCH_BATTERY_SITE=/data/battery/site.json`. "baseline": "/data/battery/runs/", "baseline_private": null, "pdb_cache": "/data/battery/pdb_cache", + "cod_cache": "/data/battery/cod_cache", + "shelxl": "/opt/xtal/ccp4-9/bin/shelxl", "arms": { "open": {"manifest": "open.json", "reference": "deposition", "data_root": "/home/data/open"}, "inhouse": {"manifest": "inhouse.json", "reference": "xds", "data_root": "/home/data/inhouse"}, @@ -193,6 +195,9 @@ A relative `manifest` path is resolved against `tools/battery/`. `gpulock` is a `rugnux` is prefixed with under `--gpulock`. At PSI it is a wrapper script that queues for one of the shared GPU slots. Set it to `null` if you have no such wrapper. `pdb_cache` (optional) is where the deposited models and their metadata are cached for `--model` and the REFMAC check. +`cod_cache` (optional, default `/data/battery/cod_cache`) caches the small molecules' COD structures, +and `shelxl` (optional; else `shelxl` on `PATH`, e.g. from CCP4) is the SHELXL the small-molecule +check calls - without one the check is skipped and says so. ## Running @@ -323,6 +328,26 @@ and that shell's limit: the model never saw these, so clearly above zero is sign `dep_kind` (`I` or `F`), `dep_d_min` (the deposited data's limit), `dep_n_common`, `dep_status` and `dep_reason`. +### The SHELXL check (small molecules, always on) + +`tools/battery/shelx_check.py` refines a small-molecule set's published structure against our +`p.hkl` with SHELXL, the ground truth a chemical crystallographer would use. The structure is the +row's `cod` entry of the Crystallography Open Database (downloaded once into `cod_cache`). One fixed +recipe, so two runs compare: the data reindexed into the COD setting (every integer matrix with +entries -1..1 and det +1 that maps our cell onto the COD cell within 3% is tried with 6 cycles, the +lowest R1 wins), non-H anisotropic, H fixed at the COD positions with the COD U, EXTI refined, +`MERG 2` (SHELXL merges equivalents itself, so a merged or an unmerged HKLF 4 file both work), +weights from SHELXL's own suggestion over three rounds of 12 cycles. It also scores the data +against |Fc| of the COD model as published (gemmi, one scale, nothing refined), which no refinement +can absorb. `report` and `compare` compute it for an older run that lacks it (temporary directory; +about 5-40 s a set). Reported, never scored. Row keys: `shx_r1` (R1 for Fo > 4 sigma, `shx_n4` of +them), `shx_r1_all`, `shx_wr2`, `shx_goof`, `shx_exti`, `shx_wght_a`/`shx_wght_b` (the suggested +weights; a near SHELXL's 0.2 ceiling means sigmas or strong reflections are off), `shx_peak`/ +`shx_hole` (residual density, e/A^3), `shx_rint`/`shx_rsigma` (as SHELXL prints them; R(int) only +means something for unmerged data), `shx_k_top` (/ of the strongest analysis-of-variance +bin; below 1: strong reflections read low), `shx_fixed_r1`, `shx_cod`, `shx_status`, `shx_reason`. +`python3 shelx_check.py ` runs it on one finished set. + `model_sweep.py`, which ran `rugnux --model` over the open arm as a separate sweep, is retired. The open arm's command does the same inside the battery. @@ -396,6 +421,7 @@ was run `--unforced`), so its forced XDS-arm rows are left out the same way. | `rmodel`, `rmodel_shell_scaled`, `radial_misfit` | open arm with a model: R_MODEL, R_MODEL_SHELL_SCALED and MODEL_RADIAL_MISFIT - R over all the reflections, with and without one free scale per resolution shell, and how big that rescale was. `rmodel_shell_scaled` is the R that compares between runs (below) | | `refmac_rfree`, `refmac_rwork`, `refmac_rfree_depflags`, `refmac_rfree_depdata`, `refmac_rfree_ratio`, `refmac_twin`, `refmac_untwinned`, `refmac_status`, `refmac_reason`, `refmac_refined_*`, `refmac_shared_free_n` | the REFMAC check (`--model-check`, open arm; see its section for which R-free is which) | | `dep_status`, `dep_reason`, `dep_kind`, `dep_d_min`, `dep_n_common`, `dep_cc_delta_all`, `dep_cc_delta_outer`, `dep_beyond_cc`, `dep_beyond_d` | open arm with a model: our merge against the depositor's data per shell (`depdata_check.py`, above) | +| `shx_*` | small molecule with a `cod` structure: the SHELXL check (`shelx_check.py`, above) | | `wall_s`, `rugnux_wall_s`, `elapsed_s`, `exit_code`, `gpu_others`, `rugnux_verdict` | timing (`wall_s` is rugnux's own WALL_TIME, not the time spent waiting for the GPU queue), the exit code, other GPU processes seen before the set, and rugnux's own verdict line | ## The report @@ -417,6 +443,9 @@ SVG, no external files): the deposited limit, one row per set, sorted by the outer-shell difference; - on the XDS arms the **like-for-like table**: the reference-range numbers beside XDS's, with the rows where rugnux's cut is coarser marked as coverage; +- for the small molecules, the **SHELXL table**: R1, wR2, GooF, EXTI, weights, residual density, + R(int)/R(sigma), the strongest bin's K and the fixed-model R1(F) of the COD structure refined + against our data; - the **failures**, and **one row per set** with all the numbers; - if there is a baseline, the **delta against it**: pass rates per arm on the common sets, and every set that moved beyond noise. @@ -443,7 +472,7 @@ $B compare RUN_A RUN_B --rerun-changed # rerun the changed sets wit `compare` pairs rows by arm and set id, following the manifests' `aliases` across renames. A run from before schema 3 is compared through its `bare` rows (see the schema above). It lists every row whose verdict, space group, lattice, d_min, ISa, R_meas, CC1/2, completeness, cell, R-free, -reference-range R_meas, lowest-shell R_meas or ISa (where both runs have them), or time moved by more than the noise +reference-range R_meas, lowest-shell R_meas or ISa, SHELXL R1, wR2, GooF or EXTI (where both runs have them), or time moved by more than the noise thresholds in `report.py` (`NOISE`). Those thresholds are a first guess. `--rerun-changed` measures the noise directly: it reruns the changed sets with A's saved binary and options. A set that moves again under the same binary is noise. A set that @@ -494,6 +523,9 @@ whole range; the manifest keeps the rule used as `dmin_rule`. disagrees with an unpinned manifest input. Add the row to `open.json` with the deposited reference (`sg`, `sgno`, `cell`, `dmin`) and tags. Set `"pinned": true` when the choice between sweeps was deliberate. Record the source and DOI in `docs/EXTERNAL_TEST_DATA.md`. +- **Small molecules** carry `"cod": ""`, the published structure the SHELXL check refines + (pick an entry whose Niggli-reduced cell and space group match the row's reference; prefer a + modern, ordered, low-R one). - **Every row** carries `wavelength` (A), the value rugnux reads from the image headers (`WAVELENGTH` in `p_report.txt`). It is descriptive only; nothing scores it. - **Inhouse / private**: add the row (`id`, `input`, tags, and `"expect": "no_lattice"` for a diff --git a/tools/battery/battery.py b/tools/battery/battery.py index 87742a9a7..05ecefde0 100644 --- a/tools/battery/battery.py +++ b/tools/battery/battery.py @@ -62,6 +62,7 @@ import report # noqa: E402 import score # noqa: E402 import model_check # noqa: E402 the deposited model (--model); REFMAC check (--model-check) +import shelx_check # noqa: E402 SHELXL refinement of a small molecule's COD structure import depdata_check # noqa: E402 our merge vs the depositor's data, shell by shell PDB_ID = re.compile(r"^[0-9][A-Za-z0-9]{3}$") @@ -338,7 +339,7 @@ def run_one(run_dir, binary, e, opts): row = {"set": e["id"], "arm": e["arm"], "tags": e.get("tags", []), "input": e["input_path"], "gpu_others": gpu_others(), "model": None, "model_note": None, "rfree_deposited": None, "rfree_ratio": None, "cmd": None, "exit_code": None, "elapsed_s": None, "wall_s": None} - row.update({k: None for k in REFMAC_KEYS + depdata_check.KEYS}) + row.update({k: None for k in REFMAC_KEYS + depdata_check.KEYS + shelx_check.KEYS}) if not os.path.exists(e["input_path"]): row.update(score.judge(e, {}, "no input")) return row @@ -372,6 +373,10 @@ def run_one(run_dir, binary, e, opts): row.update(depdata_check.check_set(wd, e["id"], opts["pdb_cache"])) if opts["model_check"] and e["arm"] == "open": row.update(check_model(e, wd)) + if e.get("cod"): + # small molecule: its published structure refined against our p.hkl (report only) + row.update(shelx_check.check(wd, e["cod"], opts.get("cod_cache") or shelx_check.DEFAULT_CACHE, + opts.get("shelxl"))) return row @@ -550,6 +555,8 @@ def cmd_run(a, site): subset = a.only or (f"tier {a.tier}" if a.tier else None) opts = {"threads": a.threads, "timeout": a.timeout, "model_check": a.model_check and "open" in arms, "pdb_cache": site.get("pdb_cache") or model_check.DEFAULT_CACHE, + "cod_cache": site.get("cod_cache") or shelx_check.DEFAULT_CACHE, + "shelxl": shelx_check.find_shelxl(site.get("shelxl")), "gpulock": site["gpulock"] if a.gpulock else None, "extra_args": a.extra.split(), "runs_root": a.runs_root or site["runs_root"]} if a.gpulock and not site.get("gpulock"): @@ -586,6 +593,8 @@ def cmd_abort(a, site): def cmd_compare(a, site): man_a, res_a = report.load_run(a.run_a, a.allow_incomplete) man_b, res_b = report.load_run(a.run_b, a.allow_incomplete) + report.fill_shelx(a.run_a, man_a, res_a) + report.fill_shelx(a.run_b, man_b, res_b) rows = report.compare_rows(res_a, res_b) doc = report.Doc(f"{os.path.basename(os.path.normpath(a.run_a))} vs " f"{os.path.basename(os.path.normpath(a.run_b))}") @@ -608,6 +617,8 @@ def cmd_compare(a, site): opts = dict(man_a["options"], runs_root=a.runs_root or site["runs_root"]) opts["gpulock"] = opts["gpulock"] and site.get("gpulock") opts.setdefault("pdb_cache", site.get("pdb_cache") or model_check.DEFAULT_CACHE) + opts.setdefault("cod_cache", site.get("cod_cache") or shelx_check.DEFAULT_CACHE) + opts.setdefault("shelxl", shelx_check.find_shelxl(site.get("shelxl"))) label = "rerunA-" + man_a["label"] rerun = run_sets(site, sets, man_a["arms"], os.path.join(a.run_a, "bin", "rugnux"), None, label, opts, ",".join(e["id"] for e in sets)) @@ -865,6 +876,8 @@ def main(): site = load_site(a.site) report.ALIASES = load_aliases(site) report.PDB_CACHE = site.get("pdb_cache") or model_check.DEFAULT_CACHE + report.COD_CACHE = site.get("cod_cache") or shelx_check.DEFAULT_CACHE + report.SHELXL = shelx_check.find_shelxl(site.get("shelxl")) report.MANIFESTS = {arm: {e["id"]: e for e in load_sets(site, arm)} for arm, cfg in site["arms"].items() if os.path.exists(cfg["manifest"])} {"run": cmd_run, "list": cmd_list, "compare": cmd_compare, "report": cmd_report, diff --git a/tools/battery/inhouse.json b/tools/battery/inhouse.json index a82530e26..356ad822e 100644 --- a/tools/battery/inhouse.json +++ b/tools/battery/inhouse.json @@ -38,11 +38,11 @@ {"id": "myob_x06da_sparse", "input": "myob_x06da_sparse/MyoB2-5_7cef9c_master.h5", "wavelength": 0.95373, "ref": {"sgno": 3, "cell": [35.388, 28.752, 64.083, 90.0, 106.352, 90.0], "anomalous": false, "isa": 5.46, "completeness": 73.7, "r_meas": 0.326, "cc_half": 0.972, "multiplicity": 5.21, "dmin_low": 5.92, "r_meas_low": 0.146, "dmin": 2.0, "dmin_rule": "xds_range", "dmin_xds": 2.0, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, {"id": "nothing_2", "input": "nothing_2/test-28_3400ac_master.h5", "wavelength": 0.99988, "expect": "no_lattice", "tags": ["h5", "control"]}, {"id": "nothing_1", "input": "nothing_1/test-28_9bf604_master.h5", "wavelength": 0.99988, "expect": "no_lattice", "tags": ["h5", "control"]}, - {"id": "aspirin_x10sa_20keV", "input": "aspirin_x10sa_20keV/aspirin_1_002_master.h5", "wavelength": 0.61993, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.262, 6.543, 11.257, 90.0, 95.916, 90.0], "anomalous": true, "isa": 27.97, "completeness": 84.0, "r_meas": 0.032, "cc_half": 0.999, "multiplicity": 3.14, "dmin_low": 2.01, "r_meas_low": 0.03, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, - {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "wavelength": 0.49594, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.55, "completeness": 86.6, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.03, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, - {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "wavelength": 0.61993, "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.55, "completeness": 81.0, "r_meas": 0.04, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.043, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, - {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.39, "completeness": 91.6, "r_meas": 0.03, "cc_half": 0.999, "multiplicity": 5.64, "dmin_low": 2.0, "r_meas_low": 0.035, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, - {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"}, - {"id": "lcystine_x10sa_20keV", "input": "lcystine_x10sa_20keV/lcystine_2_004_master.h5", "wavelength": 0.61993, "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"}, - {"id": "lcystine_x10sa_25keV", "input": "lcystine_x10sa_25keV/lcystine_2_003_master.h5", "wavelength": 0.49594, "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"} + {"id": "aspirin_x10sa_20keV", "input": "aspirin_x10sa_20keV/aspirin_1_002_master.h5", "wavelength": 0.61993, "cod": "7050897", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.262, 6.543, 11.257, 90.0, 95.916, 90.0], "anomalous": true, "isa": 27.97, "completeness": 84.0, "r_meas": 0.032, "cc_half": 0.999, "multiplicity": 3.14, "dmin_low": 2.01, "r_meas_low": 0.03, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "wavelength": 0.49594, "cod": "7050897", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.55, "completeness": 86.6, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.03, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "wavelength": 0.61993, "cod": "5000063", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.55, "completeness": 81.0, "r_meas": 0.04, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.043, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, + {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "wavelength": 0.61993, "cod": "2224210", "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.39, "completeness": 91.6, "r_meas": 0.03, "cc_half": 0.999, "multiplicity": 5.64, "dmin_low": 2.0, "r_meas_low": 0.035, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, + {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "wavelength": 0.61993, "cod": "2003066", "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"}, + {"id": "lcystine_x10sa_20keV", "input": "lcystine_x10sa_20keV/lcystine_2_004_master.h5", "wavelength": 0.61993, "cod": "1513328", "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"}, + {"id": "lcystine_x10sa_25keV", "input": "lcystine_x10sa_25keV/lcystine_2_003_master.h5", "wavelength": 0.49594, "cod": "1513328", "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"} ]} diff --git a/tools/battery/open.json b/tools/battery/open.json index 1a602d4eb..5327d7a6f 100644 --- a/tools/battery/open.json +++ b/tools/battery/open.json @@ -156,11 +156,11 @@ {"id": "9zm0", "input": "9zm0/Atg23ANNS_9zm0_9ZM0/data/SeAtg23_B_10_2115_master.h5", "wavelength": 0.97988, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [50.409, 30.101, 91.244, 90.0, 97.146, 90.0], "dmin": 2.1}, "tags": ["h5", "monoclinic"]}, {"id": "9zmu", "input": "9zmu/BrmeA_18154_a_B2-Apo_9zmu/data/PSL-0613_7518_master.h5", "wavelength": 0.97856, "ref": {"sg": "P 65 2 2", "sgno": 179, "cell": [47.81, 47.81, 492.58, 90.0, 90.0, 120.0], "dmin": 1.98}, "tags": ["h5", "hexagonal"]}, {"id": "cuhf2", "input": "cuhf2/03_CuHF2pyz2PF6b_P_O/CuHF2pyz2PF6b_P_O_01.nxs", "wavelength": 0.4859, "pinned": true, "tags": ["nxs", "small-molecule"]}, - {"id": "cytidine", "input": "cytidine/20151020-Cytidine-2th-30/fixed-omega--180-phi-scan01_00001.cbf", "wavelength": 0.68966, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [13.98, 14.788, 5.119, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "small-molecule"]}, - {"id": "dnba", "input": "dnba/35dnba_30K_2_04_00001.cbf", "wavelength": 0.6889, "ref": {"sg": "C 1 2/c 1", "sgno": 15, "cell": [20.2635, 8.7575, 9.6697, 90.0, 109.941, 90.0], "dmin": 0.48}, "tags": ["cbf", "monoclinic", "small-molecule"]}, - {"id": "lalanine", "input": "lalanine/pgw240050_01_00001.cbf", "wavelength": 0.6889, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [5.7952, 5.933, 12.362, 90.0, 90.0, 90.0], "dmin": null}, "tags": ["cbf", "orthorhombic", "small-molecule"], "tiers": {"smoke": "small molecule, miniCBF"}}, - {"id": "metformin", "input": "metformin/013_Mmetformin_01_master.h5", "wavelength": 0.4859, "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [7.9104, 13.8794, 7.931, 90.0, 114.606, 90.0], "dmin": 0.45}, "tags": ["h5", "monoclinic", "small-molecule"], "tiers": {"smoke": "small molecule, Diamond I19 NXmx master"}}, - {"id": "nidppe", "input": "nidppe/001_NiDppeCl2_01_master.h5", "wavelength": 0.4859, "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [11.2779, 13.3386, 15.8739, 90.0, 98.7953, 90.0], "dmin": 0.77}, "tags": ["h5", "monoclinic", "small-molecule"]}, + {"id": "cytidine", "input": "cytidine/20151020-Cytidine-2th-30/fixed-omega--180-phi-scan01_00001.cbf", "wavelength": 0.68966, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [13.98, 14.788, 5.119, 90.0, 90.0, 90.0], "dmin": null}, "cod": "2001311", "tags": ["cbf", "orthorhombic", "small-molecule"]}, + {"id": "dnba", "input": "dnba/35dnba_30K_2_04_00001.cbf", "wavelength": 0.6889, "ref": {"sg": "C 1 2/c 1", "sgno": 15, "cell": [20.2635, 8.7575, 9.6697, 90.0, 109.941, 90.0], "dmin": 0.48}, "cod": "4510615", "tags": ["cbf", "monoclinic", "small-molecule"]}, + {"id": "lalanine", "input": "lalanine/pgw240050_01_00001.cbf", "wavelength": 0.6889, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [5.7952, 5.933, 12.362, 90.0, 90.0, 90.0], "dmin": null}, "cod": "2104782", "tags": ["cbf", "orthorhombic", "small-molecule"], "tiers": {"smoke": "small molecule, miniCBF"}}, + {"id": "metformin", "input": "metformin/013_Mmetformin_01_master.h5", "wavelength": 0.4859, "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [7.9104, 13.8794, 7.931, 90.0, 114.606, 90.0], "dmin": 0.45}, "cod": "2108029", "tags": ["h5", "monoclinic", "small-molecule"], "tiers": {"smoke": "small molecule, Diamond I19 NXmx master"}}, + {"id": "nidppe", "input": "nidppe/001_NiDppeCl2_01_master.h5", "wavelength": 0.4859, "ref": {"sg": "P 1 21/c 1", "sgno": 14, "cell": [11.2779, 13.3386, 15.8739, 90.0, 98.7953, 90.0], "dmin": 0.77}, "cod": "2012031", "tags": ["h5", "monoclinic", "small-molecule"]}, {"id": "5mln", "input": "5mln/5mln/data/CmADHx6_w1_2_0001.cbf", "wavelength": 0.8729, "ref": {"sg": "P 21 2 21", "sgno": 18, "cell": [74.178, 80.425, 80.52, 90.0, 90.0, 90.0], "dmin": 1.6}, "tags": ["cbf"]}, {"id": "5t39", "input": "5t39/10mMfuc-12h.001", "wavelength": 0.97872, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [50.222, 41.27, 58.504, 90.0, 98.58, 90.0], "dmin": 1.1004}, "tags": ["marccd"]}, {"id": "6cdl", "input": "6cdl/nnnn_6cdl/data/wt_32-14A_p6n6.0001", "wavelength": 1.0, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [58.26, 85.91, 46.051, 90.0, 90.0, 90.0], "dmin": 1.25}, "tags": ["marccd"]}, diff --git a/tools/battery/report.py b/tools/battery/report.py index 69c0cdff9..b93def55e 100644 --- a/tools/battery/report.py +++ b/tools/battery/report.py @@ -8,6 +8,7 @@ import sys import depdata_check import score +import shelx_check from render import Doc, ratio_dots, verdict_bars # A change smaller than these is treated as run-to-run noise. They are a starting point, to be @@ -26,6 +27,10 @@ NOISE = { "refres_isa": ("rel", 0.05), "refres_lowres_r_meas": ("rel", 0.05), "wall_s": ("rel", 0.25), # and at least 10 s - see beyond_noise() + "shx_r1": ("abs", 0.003), # SHELXL refinement of a small molecule's COD structure + "shx_wr2": ("abs", 0.005), + "shx_goof": ("abs", 0.03), + "shx_exti": ("abs", 0.02), } @@ -100,6 +105,29 @@ def fill_depdata(run_dir, res): r.update(out) +def _shelx(job): + os.nice(10) + return shelx_check.check_tmp(*job) + + +def fill_shelx(run_dir, man, res): + """The SHELXL refinement (shelx_check.py) for small-molecule rows of a run made before it + existed, from the saved p.hkl, four sets at a time. The COD id comes from today's manifest.""" + own = {(e["arm"], e["id"]): e for e in man["sets"]} + todo = [] + for r in res: + e = MANIFESTS.get(r["arm"], {}).get(key(r)[1]) or own.get((r["arm"], r["set"])) or {} + wd = os.path.join(run_dir, "work", r["arm"], r["set"]) + if e.get("cod") and "shx_status" not in r and os.path.exists(os.path.join(wd, "p.hkl")): + todo.append((r, (wd, e["cod"], COD_CACHE or shelx_check.DEFAULT_CACHE, SHELXL))) + if not todo or not SHELXL: + return + print(f"SHELXL refinement for {len(todo)} small-molecule sets of an older run ...", file=sys.stderr) + with multiprocessing.Pool(4) as pool: + for (r, _), out in zip(todo, pool.map(_shelx, [j for _, j in todo], chunksize=1)): + r.update(out) + + def schema1(man, res): """Read a run from before the variants (results schema 1) as one: it ran each set once, plain on the open arm and with XDS's settings on the XDS arms - unless --unforced, which left the @@ -164,6 +192,10 @@ MANIFESTS = {} # the site's cache of deposited models and structure factors (battery.py sets it) PDB_CACHE = None +# the site's cache of COD structures and the SHELXL binary (battery.py sets them; None: no SHELXL) +COD_CACHE = None +SHELXL = None + def key(r): return (r["arm"], ALIASES.get(r["arm"], {}).get(r["set"], r["set"])) @@ -224,6 +256,9 @@ def changes(ra, rb): for name in ("refres_r_meas", "refres_isa", "refres_lowres_r_meas"): if ra.get(name) is not None and rb.get(name) is not None and beyond_noise(name, ra[name], rb[name]): out.append(name) + for name in ("shx_r1", "shx_wr2", "shx_goof", "shx_exti"): + if ra.get(name) is not None and rb.get(name) is not None and beyond_noise(name, ra[name], rb[name]): + out.append("SHELXL " + name[4:]) if beyond_noise("wall_s", ra.get("wall_s"), rb.get("wall_s")): out.append("time") return out @@ -248,7 +283,8 @@ def arrow(x, y, fmt=lambda v: str(v)): def compare_table(doc, rows, only_changed=True): head = ["set", "arm", "verdict", "space group", "d_min", "ISa", "R_meas", "CC1/2", "cell dev %", "R_model (shell-scaled)", "R_free", "radial misfit", - "ref-range R_meas", "ref-range low-res R_meas", "time s", "beyond noise"] + "ref-range R_meas", "ref-range low-res R_meas", "SHELXL R1", "SHELXL wR2", "SHELXL GooF", + "SHELXL EXTI", "time s", "beyond noise"] out = [] for (arm, s), ra, rb, ch in rows: if only_changed and not ch: @@ -267,6 +303,10 @@ def compare_table(doc, rows, only_changed=True): arrow(ra.get("refres_r_meas"), rb.get("refres_r_meas"), lambda v: f"{100 * v:.1f}%"), arrow(ra.get("refres_lowres_r_meas"), rb.get("refres_lowres_r_meas"), lambda v: f"{100 * v:.1f}%"), + arrow(ra.get("shx_r1"), rb.get("shx_r1"), lambda v: f"{v:.4f}"), + arrow(ra.get("shx_wr2"), rb.get("shx_wr2"), lambda v: f"{v:.4f}"), + arrow(ra.get("shx_goof"), rb.get("shx_goof"), lambda v: f"{v:.3f}"), + arrow(ra.get("shx_exti"), rb.get("shx_exti"), lambda v: f"{v:.3f}"), arrow(ra.get("wall_s"), rb.get("wall_s"), lambda v: f"{v:.0f}"), ", ".join(ch)]) doc.table(head, out, row_class=lambda r: "fail" if "verdict" in r[-1] else "") @@ -316,6 +356,7 @@ def build(run_dir, baseline=None, allow_incomplete=False): if baseline: check_baseline(baseline, man.get("private")) fill_depdata(run_dir, res) + fill_shelx(run_dir, man, res) b = man["binary"] doc = Doc(f"Rugnux battery - {os.path.basename(os.path.normpath(run_dir))}") if man.get("private"): @@ -508,6 +549,33 @@ def build(run_dir, baseline=None, allow_incomplete=False): "its observations. The low-resolution R_meas is the lowest shell of each table (the shells " "are XDS's, so both cover the same reflections); reported like everything here.") + shx = [r for r in res if r.get("shx_status")] + if shx: + doc.h(3, "Small molecules: SHELXL refinement of the published structure") + doc.p("The set's structure from the Crystallography Open Database (COD id) refined against our " + "p.hkl by SHELXL with one fixed recipe (shelx_check.py: data reindexed into the COD " + "setting, non-H anisotropic, H fixed at the COD positions, EXTI refined, three rounds with " + "the weights SHELXL suggests, MERG 2), the way a chemical crystallographer judges data. " + "WGHT a near its 0.2 ceiling and a large EXTI mean the strong reflections or the sigmas are " + "off; K top is / of SHELXL's strongest analysis-of-variance bin (below 1: strong " + "reflections read low). Fixed-model R1(F) scores the data against |Fc| of the COD model as " + "published, one scale, nothing refined. R(int) means something only for unmerged data. " + "Reported, never scored.") + rows = [] + for r in sorted(shx, key=lambda r: (order.get(r["arm"], 9), r["set"])): + rows.append([r["set"], r["arm"], r.get("shx_cod") or "-", f(r.get("d_min")), + f(r.get("shx_r1"), "{:.4f}") + (f" ({r['shx_n4']})" if r.get("shx_n4") else ""), + f(r.get("shx_wr2"), "{:.4f}"), f(r.get("shx_goof"), "{:.3f}"), + f(r.get("shx_exti"), "{:.3f}"), + f(r.get("shx_wght_a"), "{:.3f}") + " / " + f(r.get("shx_wght_b"), "{:.2f}"), + f(r.get("shx_peak")) + " / " + f(r.get("shx_hole")), + f(r.get("shx_rint"), "{:.4f}") + " / " + f(r.get("shx_rsigma"), "{:.4f}"), + f(r.get("shx_k_top"), "{:.3f}"), f(r.get("shx_fixed_r1"), "{:.4f}"), + r["shx_status"] if r["shx_status"] == "ok" else f"{r['shx_status']}: {r.get('shx_reason')}"]) + doc.table(["set", "arm", "COD", "own d_min", "R1 >4sig (n)", "wR2", "GooF", "EXTI", "WGHT a / b", + "peak / hole e/A3", "R(int) / R(sigma)", "K top", "fixed-model R1(F)", "status"], + rows, num=range(3, 13)) + derived = {r["set"]: r for r in res if (r.get("d_min_ref_rule") or "xds_range") != "xds_range"} if derived: doc.p("XDS merged past its own signal (CC1/2 of its finest shell not significant), so the " @@ -580,6 +648,7 @@ def build(run_dir, baseline=None, allow_incomplete=False): if baseline: bman, bres = load_run(baseline) + fill_shelx(baseline, bman, bres) doc.h(2, f"Delta vs baseline {os.path.basename(os.path.normpath(baseline))}") doc.p(f"Baseline rugnux {bman['binary'].get('version', '?')}. Pass rates on the sets both runs have:") both = {key(r) for r in res} & {key(r) for r in bres} diff --git a/tools/battery/shelx_check.py b/tools/battery/shelx_check.py new file mode 100644 index 000000000..119919429 --- /dev/null +++ b/tools/battery/shelx_check.py @@ -0,0 +1,290 @@ +#!/usr/bin/env python3 +"""SHELXL refinement of a small-molecule set's reference structure (COD) against rugnux's p.hkl. + +Ground truth for small molecules, the way a chemical crystallographer would judge the data: the +published structure is refined against our intensities with a fixed recipe, and R1, wR2, GooF, +extinction, the weighting scheme and the residual density are read back. Reported, never scored. + +Recipe (the same for every run, so two runs compare): + - the data are reindexed into the COD setting: every integer matrix (entries -1..1, det +1) that + maps the data cell onto the COD cell within 3% is tried with a 6-cycle refinement, and the one + with the lowest R1 is kept (lowest by a wide margin on every set so far); + - non-H atoms anisotropic, H atoms fixed at the COD positions with their COD U, occupancies + fixed at the COD values, EXTI refined; + - weights: WGHT 0.1, then three rounds of 12 cycles each with the weights SHELXL suggests; + - MERG 2 (SHELXL merges equivalents itself, Friedel mates too for a centrosymmetric group), so + the same recipe takes a merged or an unmerged HKLF 4 file; R(int)/R(sigma) are read where + SHELXL prints them (meaningful for unmerged data only). + - "fixed-model R1(F)": the data scored against |Fc| of the COD model as published (gemmi, no + refinement, one scale): a check that does not depend on what SHELXL can absorb into ADPs or EXTI. + +Derived from the small-molecule harness of the 2026-10 data-quality work (codref.py/fixedmodel.py). +SHELXL is called, not shipped: give its path as site key "shelxl" or have it on PATH (it comes with +CCP4). The COD entries are downloaded once into the site's "cod_cache". +""" +import itertools +import os +import re +import shutil +import subprocess +import sys +import tempfile +import urllib.request + +import gemmi +import numpy as np + +DEFAULT_CACHE = "/data/battery/cod_cache" +COD_URL = "https://www.crystallography.net/cod/{}.cif" + +KEYS = ("shx_status", "shx_reason", "shx_cod", "shx_r1", "shx_n4", "shx_r1_all", "shx_wr2", + "shx_goof", "shx_exti", "shx_wght_a", "shx_wght_b", "shx_peak", "shx_hole", "shx_rint", + "shx_rsigma", "shx_k_top", "shx_fixed_r1") + + +def find_shelxl(site_path=None): + if site_path and os.path.exists(site_path): + return site_path + return shutil.which("shelxl") + + +def cod_cif(cod_id, cache): + """The COD entry's CIF, downloaded into the cache on first use.""" + os.makedirs(cache, exist_ok=True) + path = os.path.join(cache, f"{cod_id}.cif") + if not os.path.exists(path): + data = urllib.request.urlopen(COD_URL.format(cod_id), timeout=60).read() + tmp = path + ".part" + with open(tmp, "wb") as fh: + fh.write(data) + os.replace(tmp, path) + return path + + +def read_report(path): + out = {} + for line in open(path, errors="replace"): + if "=" in line: + k, v = line.split("=", 1) + out[k.strip()] = v.strip() + return out + + +def read_hkl(path): + """[(h, k, l, rest of line)] of an HKLF 4 file, up to its 0 0 0 terminator.""" + out = [] + for line in open(path): + if len(line) < 12 or not line.strip(): + continue + h, k, l = int(line[0:4]), int(line[4:8]), int(line[8:12]) + if h == k == l == 0: + break + out.append((h, k, l, line[12:].rstrip("\n"))) + return out + + +def write_hkl(refl, M, path): + with open(path, "w") as fh: + for h, k, l, rest in refl: + n = M @ np.array([h, k, l]) + fh.write("%4d%4d%4d%s\n" % (n[0], n[1], n[2], rest)) + fh.write(" 0 0 0 0.00 0.00\n") + + +def cell_after(dcell, M): + """Cell of the axes a' = M a (rows of M in the data's axes).""" + A = np.array(dcell.orth.mat.tolist()) + N = A @ np.array(M).T + length = [np.linalg.norm(N[:, i]) for i in range(3)] + ang = lambda u, v: np.degrees(np.arccos(np.dot(u, v) / np.linalg.norm(u) / np.linalg.norm(v))) + return length + [ang(N[:, 1], N[:, 2]), ang(N[:, 0], N[:, 2]), ang(N[:, 0], N[:, 1])] + + +def spacegroup(st): + return gemmi.find_spacegroup_by_name(st.spacegroup_hm) + + +def reindex_candidates(st, dcell): + ref = st.cell.parameters + rots = [np.array(o.rot) // 24 for o in spacegroup(st).operations().sym_ops] + out = [] + for e in itertools.product((-1, 0, 1), repeat=9): + M = np.array(e).reshape(3, 3) + if round(np.linalg.det(M)) != 1: + continue + c = cell_after(dcell, M) + if all(abs(c[i] - ref[i]) / ref[i] < 0.03 for i in range(3)) and \ + all(abs(c[i] - ref[i]) < 2.0 for i in range(3, 6)): + # one per class of matrices equivalent under the model's point group + if not any(any(np.array_equal(R @ M, U) for R in rots) for U, _ in out): + out.append((M, c)) + return out + + +def write_ins(st, lam, cellp, wght, cycles, path): + sg = spacegroup(st) + ops = sg.operations() + centro = ops.is_centrosymmetric() + latt = {"P": 1, "I": 2, "R": 3, "F": 4, "A": 5, "B": 6, "C": 7}[sg.hm[0]] + symm = [] + for o in ops.sym_ops: + R = np.array(o.rot) // 24 + if np.array_equal(R, np.eye(3, dtype=int)) and not any(o.tran): + continue + if centro and round(np.linalg.det(R)) < 0: + continue + symm.append(o.triplet().upper()) + counts = {} + for s in st.get_all_unit_cell_sites(): + counts[s.element.name] = counts.get(s.element.name, 0) + s.occ + elems = sorted(counts, key=lambda e: (e != "C", e != "H", e)) + lines = ["TITL cod", "CELL %.5f %.4f %.4f %.4f %.3f %.3f %.3f" % (lam, *cellp), + "ZERR 1 0.001 0.001 0.001 0.01 0.01 0.01", "LATT %d" % (latt if centro else -latt)] + lines += ["SYMM " + s for s in symm] + lines += ["SFAC " + " ".join(elems), "UNIT " + " ".join(str(max(1, round(counts[e]))) for e in elems), + f"L.S. {cycles}", "LIST 4", "FMAP 2", "PLAN 5", "MERG 2", "EXTI 0", "WGHT " + wght, + "FVAR 1.0"] + names, atoms, nonh = set(), [], [] + for i, s in enumerate(st.sites): + name = re.sub(r"[^A-Za-z0-9]", "", s.label)[:4] + if not name or name in names: # SHELXL names: 4 characters, unique + name = f"{s.element.name[:1]}{i:03d}"[:4] + names.add(name) + sf = elems.index(s.element.name) + 1 + x, y, z = s.fract.x, s.fract.y, s.fract.z + occ = 10.0 + (s.occ if s.occ > 0 else 1.0) + if s.element.name == "H": + u = s.u_iso if s.u_iso > 0 else 0.05 + atoms.append("%-4s %d %9.5f %9.5f %9.5f %8.5f %8.5f" % (name, sf, 10 + x, 10 + y, 10 + z, occ, 10 + u)) + else: + u = s.u_iso if s.u_iso > 0 else 0.02 + atoms.append("%-4s %d %9.5f %9.5f %9.5f %8.5f %8.5f" % (name, sf, x, y, z, occ, u)) + nonh.append(name) + lines += ["ANIS " + " ".join(nonh[i:i + 12]) for i in range(0, len(nonh), 12)] + lines += atoms + ["HKLF 4", "END"] + with open(path, "w") as fh: + fh.write("\n".join(lines) + "\n") + + +def run_shelxl(shelxl, name, cwd): + env = dict(os.environ, OMP_NUM_THREADS="4") + subprocess.run([shelxl, name], cwd=cwd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + env=env, timeout=900) + lst_p, res_p = os.path.join(cwd, name + ".lst"), os.path.join(cwd, name + ".res") + lst = open(lst_p, errors="replace").read() if os.path.exists(lst_p) else "" + res = open(res_p, errors="replace").read() if os.path.exists(res_p) else "" + last = lambda pat, text=lst: (re.findall(pat, text, re.M) or [None])[-1] + num = lambda s: float(s) if s is not None else None + wght = (last(r"^WGHT\s+(.*)$", res) or "").split() + k_top = None + i = lst.rfind("Analysis of variance for reflections employed in refinement") + if i >= 0: + m = re.search(r"\n\s+K\s+([\d. ]+)", lst[i:i + 3000]) + k_top = num(m.group(1).split()[-1]) if m else None + n4 = last(r"R1 =\s*[\d.]+ for\s+(\d+) Fo > 4sig") + return {"shx_r1": num(last(r"R1 =\s*([\d.]+) for\s+\d+ Fo > 4sig")), + "shx_n4": int(n4) if n4 else None, + "shx_r1_all": num(last(r"and\s+([\d.]+) for all")), + "shx_wr2": num(last(r"wR2 =\s*([\d.]+)")), + "shx_goof": num(last(r"GooF = S =\s*([\d.]+)")), + "shx_peak": num(last(r"Highest peak\s+([\d.-]+)")), + "shx_hole": num(last(r"Deepest hole\s+([\d.-]+)")), + "shx_rint": num(last(r"R\(int\) =\s*([\d.]+)")), + "shx_rsigma": num(last(r"R\(sigma\) =\s*([\d.]+)")), + "shx_exti": num(last(r"^EXTI\s+([\d.]+)", res)), + "shx_wght_a": num(wght[0]) if wght else None, + "shx_wght_b": num(wght[1]) if len(wght) > 1 else None, + "shx_k_top": k_top}, " ".join(wght) + + +def fixed_model_r1(st, refl, M): + """R1(F) of the (reindexed) data against |Fc| of the COD model as published, one scale.""" + sg = spacegroup(st) + ops = sg.operations() + asu = gemmi.ReciprocalAsu(sg) + calc = gemmi.StructureFactorCalculatorX(st.cell) + st.setup_cell_images() + groups = {} + for h, k, l, rest in refl: + n = M @ np.array([h, k, l]) + if ops.is_systematically_absent(list(map(int, n))): + continue + p = rest.split() + i, s = float(p[0]), float(p[1]) + if s <= 0: + continue + groups.setdefault(tuple(asu.to_asu(list(map(int, n)), ops)[0]), []).append((i, s)) + if not groups: + return None + fo, fc = [], [] + for hkl, obs in groups.items(): + w = [1 / s ** 2 for _, s in obs] + fo.append(sum(i * wi for (i, _), wi in zip(obs, w)) / sum(w)) + fc.append(abs(calc.calculate_sf_from_small_structure(st, list(hkl))) ** 2) + fo, fc = np.array(fo), np.array(fc) + fo = fo / (np.sum(fo * fc) / np.sum(fc * fc)) + Fo, Fc = np.sqrt(np.clip(fo, 0, None)), np.sqrt(fc) + return round(float(np.sum(np.abs(Fo - Fc)) / np.sum(Fo)), 4) + + +def check(wd, cod_id, cache=DEFAULT_CACHE, shelxl=None, work=None): + """shx_* fields for one finished set in work dir `wd` (p.hkl + p_report.txt). `work`: where + SHELXL runs (default /shelx).""" + out = {k: None for k in KEYS} + out["shx_cod"] = cod_id + if not shelxl: + return dict(out, shx_status="skipped", shx_reason="SHELXL not available (site key 'shelxl' or PATH)") + hkl, rep_p = os.path.join(wd, "p.hkl"), os.path.join(wd, "p_report.txt") + if not os.path.exists(hkl) or not os.path.exists(rep_p): + return dict(out, shx_status="skipped", shx_reason="no p.hkl") + try: + rep = read_report(rep_p) + dcell = gemmi.UnitCell(*map(float, rep["UNIT_CELL_CONSTANTS"].split())) + lam = float(rep["WAVELENGTH"]) + st = gemmi.read_small_structure(cod_cif(cod_id, cache)) + refl = read_hkl(hkl) + cands = reindex_candidates(st, dcell) + if not cands: + return dict(out, shx_status="skipped", shx_reason="data cell does not match the COD cell") + work = work or os.path.join(wd, "shelx") + os.makedirs(work, exist_ok=True) + best = None + for j, (M, c) in enumerate(cands): + d = os.path.join(work, f"trial{j}") + os.makedirs(d, exist_ok=True) + write_hkl(refl, M, os.path.join(d, "t.hkl")) + write_ins(st, lam, c, "0.1", 6, os.path.join(d, "t.ins")) + r1 = run_shelxl(shelxl, "t", d)[0]["shx_r1"] + if r1 is not None and (best is None or r1 < best[0]): + best = (r1, M, c) + if best is None: + return dict(out, shx_status="error", shx_reason="SHELXL gave no R1 in any setting") + _, M, c = best + d = os.path.join(work, "final") + os.makedirs(d, exist_ok=True) + write_hkl(refl, M, os.path.join(d, "f.hkl")) + wght = "0.1" + for _ in range(3): + write_ins(st, lam, c, wght, 12, os.path.join(d, "f.ins")) + vals, suggested = run_shelxl(shelxl, "f", d) + wght = suggested or wght + out.update(vals) + out["shx_fixed_r1"] = fixed_model_r1(st, refl, M) + out["shx_status"] = "ok" if vals["shx_r1"] is not None else "error" + if vals["shx_r1"] is None: + out["shx_reason"] = "SHELXL did not finish the refinement (see shelx/final/f.lst)" + except Exception as ex: # a failing check must not lose the set's result + out.update(shx_status="error", shx_reason=f"{type(ex).__name__}: {ex}"[:300]) + return out + + +def check_tmp(wd, cod_id, cache, shelxl): + """check() for a finished (read-only) run directory: SHELXL works in a temporary directory.""" + with tempfile.TemporaryDirectory(prefix="shelx_") as tmp: + return check(wd, cod_id, cache, shelxl, tmp) + + +if __name__ == "__main__": + # shelx_check.py [cod cache] + print(check_tmp(sys.argv[1], sys.argv[2], sys.argv[3] if len(sys.argv) > 3 else DEFAULT_CACHE, + find_shelxl())) diff --git a/tools/battery/site.example.json b/tools/battery/site.example.json index 13e24c8ff..df32abe73 100644 --- a/tools/battery/site.example.json +++ b/tools/battery/site.example.json @@ -6,6 +6,8 @@ "baseline": null, "baseline_private": null, "pdb_cache": "/path/to/battery/pdb_cache", + "cod_cache": "/path/to/battery/cod_cache", + "shelxl": null, "arms": { "open": {"manifest": "open.json", "reference": "deposition", "data_root": "/path/to/battery/open"}, "inhouse": {"manifest": "inhouse.json", "reference": "xds", "data_root": "/path/to/battery/inhouse"} -- 2.54.0 From 9874db5d7fc726dbc82b130b41a7c7094f4885ac Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 01:16:50 +0200 Subject: [PATCH 28/97] rugnux: .hkl holds the unmerged scaled fulls on rotation data The SHELX HKLF 4 file now has one record per full reflection - its partials summed, the per-frame scale and every correction applied, sigma(I) as the merge weighted it - at the index it was measured at, not averaged with its equivalents: the chemical crystallographer's convention, so SHELXL computes Rint and Rsigma itself. Outliers the merge rejected and fulls beyond its resolution cut are left out; no batch column (it would select a BASF scale in SHELXL). The engine hands the fulls back only for a merge that may be written (RotationScaleMerge::SetExportScaledFulls), so the search merges, the pre-pass and the P1 cross-check carry no copy; the fulls follow the same relabelling as the merged reflections (merge_to_written). Stills keep the merged file. --mode scale writes the unmerged form too. Validation: p.mtz md5 unchanged on myob/cytc/thau (GPU). SHELXL on the same runs, merged-old vs unmerged-new (COD models, harness /data/tmp/sm_shared): aspirin 20 keV Rint 0 -> 0.071, Rsigma 0.036 -> 0.043, R1 0.0964 -> 0.0969, wR2 0.312 -> 0.310, GooF 1.53 -> 1.47; HEPES 20 keV Rint 0 -> 0.163, Rsigma 0.057 -> 0.068, R1 0.0903 -> 0.0897, wR2 0.318 -> 0.263, GooF 1.57 -> 1.10; the "input data appear to be merged" warning is gone. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- common/Reflection.h | 13 ++ docs/CHANGELOG.md | 1 + docs/RUGNUX.md | 4 +- docs/RUGNUX_INTEGRATION.md | 27 +-- docs/RUGNUX_TUTORIAL.md | 10 +- image_analysis/WriteReflections.cpp | 162 +++++++++++------- image_analysis/WriteReflections.h | 12 +- .../scale_merge/RotationScaleMerge.cpp | 17 ++ .../scale_merge/RotationScaleMerge.h | 8 + rugnux/Rugnux.cpp | 21 ++- rugnux/rugnux_cli.cpp | 5 +- tests/WriteReflectionsTest.cpp | 29 ++++ 12 files changed, 221 insertions(+), 88 deletions(-) diff --git a/common/Reflection.h b/common/Reflection.h index b5ae95f65..bcb614e40 100644 --- a/common/Reflection.h +++ b/common/Reflection.h @@ -60,6 +60,19 @@ struct Reflection { bool clipped = false; }; +// One full reflection of a rotation sweep as the merge used it - its partials summed into one +// measurement, every correction and the per-frame scale applied, its sigma under the error model the +// merge weighted it with - but NOT averaged with its symmetry mates. h k l are the index it was +// measured at, not its ASU representative. Written as the SHELX HKLF 4 file, so that the program +// reading it sees the equivalents and computes Rint itself. +struct ScaledFull { + int32_t h = 0; + int32_t k = 0; + int32_t l = 0; + float I = NAN; + float sigma = NAN; +}; + struct MergedReflection { int32_t h = 0; int32_t k = 0; diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 9a34391b6..c487269d3 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -3,6 +3,7 @@ ### 1.0.0-rc.174 +* Rugnux: `.hkl` now holds unmerged, scaled full reflections (SHELX HKLF 4) on rotation data, so SHELXL computes Rint itself. * Rugnux reads Rigaku d*TREK SMV images (Saturn CCD), including detector 2theta, image orientation and encoded pixel overflows. ### 1.0.0-rc.173 diff --git a/docs/RUGNUX.md b/docs/RUGNUX.md index f2e6c5d82..a1a23340f 100644 --- a/docs/RUGNUX.md +++ b/docs/RUGNUX.md @@ -57,7 +57,7 @@ in both. A rotation run that merges — the default — leaves seven files next ``` myrun.mtz merged intensities + French-Wilson amplitudes, for CCP4 / phenix myrun.cif the same, as mmCIF - the self-describing format, and what to deposit -myrun.hkl the same, as SHELX HKLF 4 - feed this to SHELXC / SHELXD / ANODE +myrun.hkl the scaled observations unmerged, as SHELX HKLF 4 - for SHELXL, SHELXC / ANODE myrun_unmerged.mtz every observation before scaling, for pointless / aimless / careless - the largest file of the run (--no-export-unmerged skips it) myrun_P1.mtz the same observations merged in P1, so a wrong space-group call can be @@ -85,7 +85,7 @@ A few things worth knowing before reaching for more flags: (see [Rotation data](RUGNUX_TUTORIAL.md#rotation-data)). `-S` takes a Hermann-Mauguin symbol (`P43212`) or a space-group number (`96`), whichever is to hand. - **Anomalous data are there without `-A`.** A rotation merge always keeps the Bijvoet split: a - default run's `.mtz` carries `I(+)`/`I(-)` beside `IMEAN`, and its `.hkl` the ±hkl mates — + default run's `.mtz` carries `I(+)`/`I(-)` beside `IMEAN`, and its `.hkl` every observation at the index it was measured at — `FRIEDELS_LAW= TRUE` in the report says how the *statistics* were counted, not that the signal was averaged away. What `-A` changes is the counting basis and the error model: each hand becomes a merged observation of its own, so multiplicity, completeness and ⟨I/σ⟩ are counted anomalously and diff --git a/docs/RUGNUX_INTEGRATION.md b/docs/RUGNUX_INTEGRATION.md index c98c769ce..06f3596bf 100644 --- a/docs/RUGNUX_INTEGRATION.md +++ b/docs/RUGNUX_INTEGRATION.md @@ -38,12 +38,17 @@ ignores them, and one that does can find them without guessing. > whole-range value. There is no version marker inside the file, so a number taken from an older > `.cif` is not comparable with one taken from a newer one. -**SHELX HKLF 4** (`.hkl`). Fixed-format `3I4,2F8.2` — `h k l I σ(I)`, one record per -reflection, terminated by a `0 0 0` record — which is what **SHELXC**, **SHELXD** and **ANODE** -expect. Two properties worth knowing before using it: +**SHELX HKLF 4** (`.hkl`). Fixed-format `3I4,2F8.2` — `h k l I σ(I)`, terminated by a +`0 0 0` record — which is what **SHELXL**, **SHELXC**, **SHELXD** and **ANODE** expect. Two +properties worth knowing before using it: -- **Bijvoet mates are written separately**, `I(+)` at `+hkl` and `I(-)` at `-hkl`, so the anomalous - differences survive into SHELXC; a reflection with no anomalous split is written once, as its mean. +- **On rotation data it is unmerged**: one record per full reflection (its partials summed), with the + per-frame scale and every correction applied and the σ(I) the merge weighted it with, but not + averaged with its symmetry equivalents, and at the index it was measured at — the chemical + crystallographer's convention, so SHELXL computes Rint and Rsigma itself and Friedel mates keep + their own records. Observations the merge rejected as outliers, or that lie beyond its resolution + cut, are left out. There is no batch number column (in HKLF 4 that selects a BASF scale factor). + A **stills** run writes the merged reflections instead, Bijvoet mates at `+hkl` and `-hkl`. - **Intensities are rescaled** by a single global factor so the largest value fits the `F8.2` field. `I` and `σ(I)` share that factor, so every ratio — and therefore the anomalous signal — is untouched, but the absolute scale is not meaningful. This matters only if you intend to compare @@ -371,12 +376,12 @@ should have produced rather than trust the exit status. **Nothing has to be switched on to get the anomalous signal.** A default rotation merge keeps the Bijvoet split, whether or not `-A` was given: `myrun.mtz` carries `I(+)`/`I(-)` and `F(+)`/`F(-)` -beside the means, and `myrun.hkl` writes each mate as its own record, `I(+)` at `+hkl` and `I(-)` at -`-hkl`. `-A` changes what the merging statistics are counted over, not whether the signal is in the -file. The one case with no anomalous columns at all is a **stills** run, which computes no Bijvoet -split; there `myrun.hkl` holds means only and there is nothing for SHELXC to work with. Unmerged -data are not wanted anywhere in this chain either, so a run with `--no-export-unmerged` is not -missing a file SHELX needs. +beside the means, and `myrun.hkl` holds every observation unmerged at the index it was measured at, so +SHELXC sees both hands. `-A` changes what the merging statistics are counted over, not whether the +signal is in the file. The one case with no anomalous signal at all is a **stills** run, which +computes no Bijvoet split; there `myrun.hkl` holds means only and there is nothing for SHELXC to work +with. `myrun_unmerged.mtz` is not part of this chain (it is unscaled), so a run with +`--no-export-unmerged` is not missing a file SHELX needs. **HKLF 4 carries no metadata**, so the cell and the space group have to be repeated on the SHELXC command — take them from `UNIT_CELL_CONSTANTS` and `SPACE_GROUP_NAME` in section 2 of the diff --git a/docs/RUGNUX_TUTORIAL.md b/docs/RUGNUX_TUTORIAL.md index 90972a914..f7664c38e 100644 --- a/docs/RUGNUX_TUTORIAL.md +++ b/docs/RUGNUX_TUTORIAL.md @@ -205,10 +205,12 @@ goniometer axis but you want per-frame stills processing anyway, add `--force-st - `.cif` — mmCIF, for deposition and as the self-describing native format (also carries the merging statistics, ISa, twinning and radiation-damage indicators). - `.hkl` — SHELX **HKLF 4** text (`h k l I σ(I)`, fixed `3I4,2F8.2`), the direct input for - **SHELXC / ANODE / SHELXD**. Bijvoet mates are written separately (`I(+)` at `+hkl`, `I(-)` at - `-hkl`) so the anomalous signal is preserved; intensities are put on a common scale so the largest - value fits the fixed-width field (the absolute scale is irrelevant to SHELXC/ANODE), and the file - ends with the `0 0 0` terminator record. + **SHELXL** and **SHELXC / ANODE / SHELXD**. On rotation data it holds the scaled full reflections + **unmerged** — every correction applied, symmetry equivalents not averaged, each at the index it was + measured at — so SHELXL reports Rint and Rsigma itself and the anomalous signal is all there; + stills runs write the merged reflections (Bijvoet mates at `+hkl` and `-hkl`). Intensities are put + on a common scale so the largest value fits the fixed-width field, and the file ends with the + `0 0 0` terminator record. All three carry the **refined unit cell** (from rotation indexing) and the **space group determined from systematic absences** (constrained to the indexed lattice symmetry). diff --git a/image_analysis/WriteReflections.cpp b/image_analysis/WriteReflections.cpp index 798467ec4..281cf5629 100644 --- a/image_analysis/WriteReflections.cpp +++ b/image_analysis/WriteReflections.cpp @@ -532,55 +532,35 @@ void WriteMtzReflections(const std::vector &reflections, mtz.write_to_file(filename); } -void WriteShelxHklReflections(const std::vector &reflections, - const DiffractionExperiment &experiment, - const std::string &filename, - size_t nthreads) { - bool has_anom = true; - const std::vector rows = BuildMergedRows(reflections, experiment, has_anom); - - // SHELX HKLF 4 (SHELXC / ANODE input): fixed FORMAT(3I4,2F8.2), one record per reflection as - // h k l I sigma(I). The Bijvoet mates are written separately - I(+) at +hkl, I(-) at -hkl - so the - // anomalous differences survive; a reflection with no anomalous split is written once as its mean. - // Intensities are put on a common scale so the largest value fits the F8.2 field (the absolute scale - // is irrelevant to SHELXC / ANODE, which use only ratios); I and sigma share the scale, so the - // anomalous signal is untouched. The file ends with a 0 0 0 terminator record. - const auto usable = [](float v, float s) { return std::isfinite(v) && std::isfinite(s) && s > 0.0f; }; - - double max_abs = 0.0; - for (const auto& r : rows) { - if (usable(r.Ip, r.sIp)) max_abs = std::max({max_abs, std::fabs(double(r.Ip)), double(r.sIp)}); - if (usable(r.Im, r.sIm)) max_abs = std::max({max_abs, std::fabs(double(r.Im)), double(r.sIm)}); - if (!usable(r.Ip, r.sIp) && !usable(r.Im, r.sIm) && usable(r.Imean, r.sImean)) - max_abs = std::max({max_abs, std::fabs(double(r.Imean)), double(r.sImean)}); - } - const double scale = (std::isfinite(max_abs) && max_abs > 0.0) ? 9999.0 / max_abs : 1.0; - - std::ofstream out(filename); - if (!out) - throw std::runtime_error("WriteShelxHklReflections: cannot open " + filename); - // Up to two records per reflection, five formatted numbers each. Built in parallel into per-worker - // blocks and handed to the file in order, exactly as the mmCIF rows are; "%.2f" right-aligned in - // the fixed field is what `fixed` + `setprecision(2)` + `setw` made the stream write. - const auto column = [](std::string &s, const std::string &v, size_t width) { - if (v.size() < width) s.append(width - v.size(), ' '); - s.append(v); - }; - const auto num2 = [](double v) { - char b[64]; - const int n = std::snprintf(b, sizeof b, "%.2f", v); - return std::string(b, static_cast(std::clamp(n, 0, static_cast(sizeof b) - 1))); - }; - const auto emit = [&column, &num2, scale](std::string &s, int h, int k, int l, float I, float sigma) { - column(s, std::to_string(h), 4); - column(s, std::to_string(k), 4); - column(s, std::to_string(l), 4); - column(s, num2(scale * I), 8); - column(s, num2(scale * sigma), 8); +namespace { + // The SHELX HKLF 4 record, FORMAT(3I4,2F8.2): h k l I sigma(I). "%.2f" right-aligned in the fixed + // field is what `fixed` + `setprecision(2)` + `setw` made the stream write. + void AppendHklf4(std::string &s, int h, int k, int l, double I, double sigma) { + const auto column = [&s](const std::string &v, size_t width) { + if (v.size() < width) s.append(width - v.size(), ' '); + s.append(v); + }; + const auto num2 = [](double v) { + char b[64]; + const int n = std::snprintf(b, sizeof b, "%.2f", v); + return std::string(b, static_cast(std::clamp(n, 0, static_cast(sizeof b) - 1))); + }; + column(std::to_string(h), 4); + column(std::to_string(k), 4); + column(std::to_string(l), 4); + column(num2(I), 8); + column(num2(sigma), 8); s.push_back('\n'); - }; - { - const size_t nrow = rows.size(); + } + + // An HKLF 4 file of nrow source rows, each appending its records with append_row(i, s). Built in + // parallel into per-worker blocks and handed to the file in order, exactly as the mmCIF rows are, + // and closed with the 0 0 0 end-of-data record. + template + void WriteHklf4File(const std::string &filename, size_t nrow, size_t nthreads, AppendRow append_row) { + std::ofstream out(filename); + if (!out) + throw std::runtime_error("WriteShelxHklReflections: cannot open " + filename); const size_t nw = std::max(nthreads, 1); const int nch = static_cast(ThreadsForWork(nrow, nw, 4096)); std::vector block(nch); @@ -589,24 +569,71 @@ void WriteShelxHklReflections(const std::vector &reflections, const size_t lo = nrow * t / nch, hi = nrow * (t + 1) / nch; std::string &s = block[t]; s.reserve((hi - lo) * 2 * 29); - for (size_t i = lo; i < hi; ++i) { - const auto &r = rows[i]; - const bool plus = usable(r.Ip, r.sIp); - const bool minus = usable(r.Im, r.sIm); - if (plus) emit(s, r.h, r.k, r.l, r.Ip, r.sIp); - if (minus) emit(s, -r.h, -r.k, -r.l, r.Im, r.sIm); - if (!plus && !minus && usable(r.Imean, r.sImean)) - emit(s, r.h, r.k, r.l, r.Imean, r.sImean); - } + for (size_t i = lo; i < hi; ++i) + append_row(i, s); } }); for (const std::string &s : block) out.write(s.data(), static_cast(s.size())); + std::string tail; + AppendHklf4(tail, 0, 0, 0, 0.0, 0.0); + out.write(tail.data(), static_cast(tail.size())); } - std::string tail; - emit(tail, 0, 0, 0, 0.0f, 0.0f); // HKLF-4 end-of-data marker - out.write(tail.data(), static_cast(tail.size())); - out.close(); + + // Intensities are put on a common scale so the largest value fits the F8.2 field (the absolute scale + // is irrelevant to every SHELX program, which refines or ignores it); I and sigma share the scale. + double Hklf4Scale(double max_abs) { + return (std::isfinite(max_abs) && max_abs > 0.0) ? 9999.0 / max_abs : 1.0; + } + + bool UsableForHkl(float v, float s) { return std::isfinite(v) && std::isfinite(s) && s > 0.0f; } +} + +void WriteShelxHklReflections(const std::vector &reflections, + const DiffractionExperiment &experiment, + const std::string &filename, + size_t nthreads) { + bool has_anom = true; + const std::vector rows = BuildMergedRows(reflections, experiment, has_anom); + + // One record per merged reflection. The Bijvoet mates are written separately - I(+) at +hkl, I(-) + // at -hkl - so the anomalous differences survive; a reflection with no anomalous split is written + // once as its mean. + double max_abs = 0.0; + for (const auto& r : rows) { + if (UsableForHkl(r.Ip, r.sIp)) max_abs = std::max({max_abs, std::fabs(double(r.Ip)), double(r.sIp)}); + if (UsableForHkl(r.Im, r.sIm)) max_abs = std::max({max_abs, std::fabs(double(r.Im)), double(r.sIm)}); + if (!UsableForHkl(r.Ip, r.sIp) && !UsableForHkl(r.Im, r.sIm) && UsableForHkl(r.Imean, r.sImean)) + max_abs = std::max({max_abs, std::fabs(double(r.Imean)), double(r.sImean)}); + } + const double scale = Hklf4Scale(max_abs); + WriteHklf4File(filename, rows.size(), nthreads, [&](size_t i, std::string &s) { + const auto &r = rows[i]; + const bool plus = UsableForHkl(r.Ip, r.sIp); + const bool minus = UsableForHkl(r.Im, r.sIm); + if (plus) AppendHklf4(s, r.h, r.k, r.l, scale * r.Ip, scale * r.sIp); + if (minus) AppendHklf4(s, -r.h, -r.k, -r.l, scale * r.Im, scale * r.sIm); + if (!plus && !minus && UsableForHkl(r.Imean, r.sImean)) + AppendHklf4(s, r.h, r.k, r.l, scale * r.Imean, scale * r.sImean); + }); +} + +void WriteShelxHklReflections(const std::vector &fulls, + const std::string &filename, + size_t nthreads) { + // One record per full, at the index it was measured at: SHELXL / SHELXC average the equivalents + // themselves and report Rint and Rsigma from them, which a merged file hides. No batch number - in + // HKLF 4 that column selects a BASF scale factor in SHELXL, which these fulls do not want. + double max_abs = 0.0; + for (const auto &f : fulls) + if (UsableForHkl(f.I, f.sigma)) + max_abs = std::max({max_abs, std::fabs(double(f.I)), double(f.sigma)}); + const double scale = Hklf4Scale(max_abs); + WriteHklf4File(filename, fulls.size(), nthreads, [&](size_t i, std::string &s) { + const auto &f = fulls[i]; + if (UsableForHkl(f.I, f.sigma)) + AppendHklf4(s, f.h, f.k, f.l, scale * f.I, scale * f.sigma); + }); } namespace { @@ -1043,12 +1070,17 @@ void WriteReflections(const std::vector &reflections, const ErrorModelReport &error_model, const TwinningAnalysisResult &twinning, const std::string &filename, - size_t nthreads) { + size_t nthreads, + const std::vector *scaled_fulls) { // Write an MTZ, an mmCIF and a SHELX HKLF-4 .hkl - each has its uses downstream (MTZ for the CCP4 / - // phenix reflection tools, mmCIF for deposition and as the self-describing native format, HKLF-4 as - // the SHELXC / ANODE substructure-solution input). + // phenix reflection tools, mmCIF for deposition and as the self-describing native format, HKLF-4 for + // SHELXL refinement and SHELXC / ANODE substructure solution). The .hkl holds the unmerged scaled + // fulls where the merge provides them (rotation), the merged reflections otherwise (stills). WriteMtzReflections(reflections, unitCell, experiment, filename + ".mtz"); WriteMmcifReflections(reflections, unitCell, experiment, statistics, error_model, twinning, filename + ".cif", nthreads); - WriteShelxHklReflections(reflections, experiment, filename + ".hkl", nthreads); + if (scaled_fulls && !scaled_fulls->empty()) + WriteShelxHklReflections(*scaled_fulls, filename + ".hkl", nthreads); + else + WriteShelxHklReflections(reflections, experiment, filename + ".hkl", nthreads); } diff --git a/image_analysis/WriteReflections.h b/image_analysis/WriteReflections.h index 9d1ff5421..697cbe87c 100644 --- a/image_analysis/WriteReflections.h +++ b/image_analysis/WriteReflections.h @@ -42,12 +42,17 @@ void WriteMtzReflections(const std::vector &reflections, const DiffractionExperiment &experiment, const std::string &filename); -// SHELX HKLF-4 text file (h k l I sigma(I), Bijvoet mates separate) for SHELXC / ANODE. -// nthreads: workers for the per-reflection row formatting, as for the mmCIF. +// SHELX HKLF-4 text file, FORMAT(3I4,2F8.2) h k l I sigma(I), scaled so the largest value fits the +// field, ended by a 0 0 0 record. nthreads: workers for the row formatting, as for the mmCIF. +// Merged reflections, Bijvoet mates as separate records (the stills output). void WriteShelxHklReflections(const std::vector &reflections, const DiffractionExperiment &experiment, const std::string &filename, size_t nthreads); +// Unmerged scaled fulls, one record each at its measured index (the rotation output). +void WriteShelxHklReflections(const std::vector &fulls, + const std::string &filename, + size_t nthreads); // Unmerged observations in the column and batch-header layout POINTLESS writes: aimless, pointless, // careless and iotbx.merging_statistics all read that layout. H K L are the ASU indices and M/ISYM @@ -88,4 +93,5 @@ void WriteReflections(const std::vector &reflections, const ErrorModelReport &error_model, const TwinningAnalysisResult &twinning, const std::string &filename, - size_t nthreads); \ No newline at end of file + size_t nthreads, + const std::vector *scaled_fulls = nullptr); diff --git a/image_analysis/scale_merge/RotationScaleMerge.cpp b/image_analysis/scale_merge/RotationScaleMerge.cpp index 81dabcd22..12f4dba1d 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.cpp +++ b/image_analysis/scale_merge/RotationScaleMerge.cpp @@ -4885,6 +4885,23 @@ RotationScaleMerge::Result RotationScaleMerge::MergeAndStats(int n_groups, bool }); } + // The same fulls unmerged, for the SHELX file: those the final merge above accumulated - usable, + // not rejected by either outlier test, inside the cut (judged on the group's d, as the merged + // reflections are) - each with the sigma the merge weighted it by. + if (export_scaled_fulls && full_stats && !for_search) { + result.scaled_fulls.clear(); + for (int i = 0; i < n_full; ++i) { + const int g = mf.group[i]; + if (g < 0 || rejected_obs[i] || !std::isfinite(merged_I[g])) + continue; + if (effective_d_min && acc[g].d < *effective_d_min) + continue; + const Obs &o = fulls[i]; + const float I_corr = o.I * o.corr; + result.scaled_fulls.push_back({o.h, o.k, o.l, I_corr, corrected_sigma(o, I_corr, o.sigma * o.corr)}); + } + } + // Asymptotic I/sigma. ISa is by definition the I -> infinity limit of the signal-to-noise, i.e. the // reproducibility of the strongest reflections (Diederichs, Acta Cryst. D66 (2010), 733-740). The // (a, b) fit above spans the whole intensity range, and a mild excess of scatter at intermediate diff --git a/image_analysis/scale_merge/RotationScaleMerge.h b/image_analysis/scale_merge/RotationScaleMerge.h index 6b5a81216..45ce1f6fa 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.h +++ b/image_analysis/scale_merge/RotationScaleMerge.h @@ -84,6 +84,9 @@ public: int scaling_iterations_partials = 0; int scaling_iterations_fulls = 0; bool scaling_converged = true; + // The fulls the merge kept, unmerged (see ScaledFull): what passed the outlier tests, inside the + // resolution cut. Filled only when SetExportScaledFulls asked for it, and never on a search merge. + std::vector scaled_fulls; }; // experiment: read live (its space group is changed by the caller between Run() calls). @@ -124,6 +127,10 @@ public: // unmerged MTZ describe the merge that was written. void SetWriteBackPerFrameScale(bool on) { write_back_per_frame_scale = on; } + // Whether Run() also hands back the merge's fulls unmerged (Result::scaled_fulls). Off by default: + // it is a copy as long as the fulls, wanted only by a caller that writes them. + void SetExportScaledFulls(bool on) { export_scaled_fulls = on; } + // Override the high-resolution cut for the next Run() - used to gate the de-novo P1 search pass at // >= 1 without cutting the final in-symmetry merge. Reset to the manual limit afterwards. void SetDMinLimit(std::optional d_min_A) { d_min_limit = d_min_A; } @@ -419,6 +426,7 @@ private: bool gpu_active_ = false; #endif bool write_back_per_frame_scale = true; // see SetWriteBackPerFrameScale + bool export_scaled_fulls = false; // see SetExportScaledFulls // --- helpers (each a flat pass; see the .cpp) --- // Turn the per-frame mean background under the reflections (accumulated by the ingest fill loop) into diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index aed1270d0..974404ae1 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -6449,6 +6449,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // The reference-range table and its ISa (--report-resolution); see ProcessResult. std::optional reference_statistics; double reference_isa = 0.0; + // The merge's fulls unmerged, for the .hkl (rotation, written merges only - see below). + std::vector scaled_fulls; }; // The reference path computes each image's G once (per-image scaling against the // reference); the scaling loop below is skipped, so G is stable across the two passes. @@ -6830,6 +6832,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // merge of its own - only the first merge, which the two-pass quality guard reads. // already_run: the engine's result for this merge, when it was made earlier (see the P1 // cross-check); it is then only reported, exactly as if it had been made here. + bool export_scaled_fulls = true; // off around the P1 cross-check, see scale_and_merge auto scale_and_merge = [&](const std::string &label, bool for_search, bool measure_cc = false, std::optional already_run = std::nullopt) @@ -6839,6 +6842,10 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // The geometry pre-pass merges only to choose the space group and to give the quality // guard something to judge the second pass against; its reflections are never written, // so the parts of the merge that only fill in an output file are skipped there. + // Only a merge that may be written hands its fulls back: they are a copy as long as the + // fulls, and the search merges, the pre-pass and the P1 cross-check never write a .hkl. + rsm->SetExportScaledFulls(!for_search && !geometry_prepass && write_files && config_.write_merged + && export_scaled_fulls); auto r = already_run ? std::move(*already_run) : rsm->Run(for_search, /*full_stats=*/!geometry_prepass, /*measure_cc_before_corrections=*/measure_cc); @@ -6855,7 +6862,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b result.search_scaling_converged = r.scaling_converged; return ScaleMergeResult{std::move(r.merged), std::move(r.statistics), r.cc_half_before_corrections, - std::move(r.reference_statistics), r.reference_isa}; + std::move(r.reference_statistics), r.reference_isa, + std::move(r.scaled_fulls)}; } // Stills (rotation goes through RotationScaleMerge above): self-scale each image against the // running merge with ScaleOnTheFly (fixed partiality), then merge directly. This runs even @@ -9022,9 +9030,16 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b ? fmt::format("{:.2f}", result.error_model_isa_asymptotic) : std::string(), result.error_model_a > 0 ? fmt::format("{:.3f}", result.error_model_a) : std::string(), result.error_model_b > 0 ? fmt::format("{:.4e}", result.error_model_b) : std::string()}; + // The fulls carry the index they were measured at in the scaler's frame; every relabelling + // the merged reflections went through since (merge_to_written) takes them to the written one. + if (!(merge_to_written == gemmi::Op::identity())) + for (auto &f : sm.scaled_fulls) { + const gemmi::Op::Miller h = merge_to_written.apply_to_hkl({{f.h, f.k, f.l}}); + f.h = h[0]; f.k = h[1]; f.l = h[2]; + } WriteReflections(sm.merged, *result.consensus_cell, experiment_, sm.statistics, em_report, result.twinning, config_.output_prefix, - static_cast(std::max(1, config_.nthreads))); + static_cast(std::max(1, config_.nthreads)), &sm.scaled_fulls); // P1 cross-check dataset. The group the files above are written in was chosen by the // search, and if that choice is wrong nothing in them says so - every statistic was @@ -9114,7 +9129,9 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b experiment_.SpaceGroupNumber(1); if (!p1_merged_early) rsm->SetWriteBackPerFrameScale(false); + export_scaled_fulls = false; auto p1 = scale_and_merge("P1 cross-check", false, false, std::move(p1_merged_early)); + export_scaled_fulls = true; rsm->SetWriteBackPerFrameScale(true); // The scaler still holds the observations in the indexing they were merged in, so every // relabelling since (the written setting, the model's indexing) is applied to this merge diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp index 4eb457f6b..3a8364894 100644 --- a/rugnux/rugnux_cli.cpp +++ b/rugnux/rugnux_cli.cpp @@ -1781,6 +1781,7 @@ static int RunRugnux(int argc, char **argv) { const auto scale_start = std::chrono::steady_clock::now(); std::vector merged_reflections; + std::vector scaled_fulls; // the .hkl: rotation merges only MergeStatistics merged_statistics; double error_model_isa = 0.0; double error_model_isa_asymptotic = 0.0; @@ -1806,11 +1807,13 @@ static int RunRugnux(int argc, char **argv) { RotationScaleMerge rsm(experiment, reflections, experiment.GetUnitCell(), scaling_iter, nthreads, logger); rsm.Ingest(); + rsm.SetExportScaledFulls(!output_prefix.empty()); // No pass to compare against here - --mode scale merges the stored reflections once - so // decline the extra merge the pre-correction CC1/2 would cost. auto r = rsm.Run(false, /*full_stats=*/true, /*measure_cc_before_corrections=*/false); merged_reflections = std::move(r.merged); merged_statistics = std::move(r.statistics); + scaled_fulls = std::move(r.scaled_fulls); error_model_isa = r.isa; error_model_isa_asymptotic = r.isa_asymptotic; error_model_isa_resolved = r.isa_resolved; @@ -2030,7 +2033,7 @@ static int RunRugnux(int argc, char **argv) { error_model_a > 0 ? fmt::format("{:.3f}", error_model_a) : std::string(), error_model_b > 0 ? fmt::format("{:.4e}", error_model_b) : std::string()}; WriteReflections(merged_reflections, *experiment.GetUnitCell(), experiment, merged_statistics, - em_report, twinning, output_prefix, static_cast(nthreads)); + em_report, twinning, output_prefix, static_cast(nthreads), &scaled_fulls); } // --mode scale re-merges stored reflections, so it determines a merging result and gets the diff --git a/tests/WriteReflectionsTest.cpp b/tests/WriteReflectionsTest.cpp index 24a532447..83ed12ab7 100644 --- a/tests/WriteReflectionsTest.cpp +++ b/tests/WriteReflectionsTest.cpp @@ -4,6 +4,7 @@ #include #include +#include #include #include @@ -185,3 +186,31 @@ TEST_CASE("Unmerged MTZ: built on several workers, the same rows as on one", "[w CHECK(parallel.batches.back().number == serial.batches.back().number); } } + +TEST_CASE("SHELX .hkl of unmerged fulls: fixed 3I4,2F8.2, scaled to fit, 0 0 0 terminator", + "[write_reflections][portable]") { + // Two fulls of the same reflection and a Friedel mate: written as measured, nothing averaged. The + // largest |I| or sigma is put at 9999.00; an unusable full (sigma not positive) is left out. + const std::vector fulls{ + {1, 2, 3, 200000.0f, 2000.0f}, + {-1, -2, -3, 100000.0f, 1200.0f}, + {1, 2, 3, -50.0f, 20.0f}, + {4, 5, 6, 10.0f, 0.0f}, + }; + const auto path = (std::filesystem::temp_directory_path() / "jfjoch_scaled_fulls_test.hkl").string(); + WriteShelxHklReflections(fulls, path, 2); + + std::ifstream in(path); + std::vector lines; + for (std::string line; std::getline(in, line);) + lines.push_back(line); + std::filesystem::remove(path); + + REQUIRE(lines.size() == 4); + for (const auto &line : lines) + CHECK(line.size() == 28); + CHECK(lines[0] == " 1 2 3 9999.00 99.99"); + CHECK(lines[1] == " -1 -2 -3 4999.50 59.99"); + CHECK(lines[2] == " 1 2 3 -2.50 1.00"); + CHECK(lines[3] == " 0 0 0 0.00 0.00"); +} -- 2.54.0 From 8394b5988d8a067b5ad001c3e38401d084b86132 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 02:02:15 +0200 Subject: [PATCH 29/97] Integration follows the measured spot footprint where a spot outgrows the r1 disk The integrator's r1 disk and r2..r3 background ring are fixed in pixels and chosen from spots near the beam. On small-molecule data at 20-25 keV a spot's standard deviation grows from ~1 px near the beam to ~5 px at the edge (radially from parallax/obliquity, tangentially from the crystal's azimuthal spread), so the r1 = 4 disk holds a quarter of the flux there, the background ring a third of it, and the in-disk second moments the Gaussian is built from saturate near r1^2/4. On top of that, the profile/summation runaway guard sent 20-30% of these reflections - the strong, wide ones - back to the truncated r1 box sum. - SpotFootprint: every pre-scan spot (width frames) is measured with a window that follows it (3 sigma, iterated, re-centred), radially and tangentially; the medians per distance-from-beam bin become BraggIntegrationSettings::Footprint. Installed only where some bin outgrows r1, and on the adaptive side like the radius (pre-pass without; the starvation guard falls back to the settings without it). - BraggStencil: where 3 sigma > r1 the background ring starts at 3 sigma along and across the radius, the summation region is the r1 disk plus the 3-sigma footprint ellipse (so the guard's fallback is a complete intensity), and the per-reflection Gaussian takes the footprint widths. Compact spots keep the stencil bit for bit. Both engines build it from the same header. SHELXL against COD (R1 / fixed-XDS-model R1(F)): citric acid .101/.230 -> .077/.055, HEPES .070/.179 -> .048/.050, aspirin 20 keV .059/.070 -> .052/.061, aspirin 25 keV unchanged, L-cystine 25 keV unchanged (.145 -> .144). Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- common/BraggIntegrationSettings.cpp | 13 ++ common/BraggIntegrationSettings.h | 16 +++ .../BraggIntegrationEngine.cpp | 15 ++- .../BraggIntegrationEngineCPU.cpp | 26 +++- .../BraggIntegrationEngineGPU.cu | 24 +++- .../bragg_integration/BraggStencil.h | 103 +++++++++++++- .../bragg_integration/CMakeLists.txt | 2 + .../bragg_integration/SpotFootprint.cpp | 126 ++++++++++++++++++ .../bragg_integration/SpotFootprint.h | 53 ++++++++ rugnux/Rugnux.cpp | 68 +++++++++- tests/BraggStencilTest.cpp | 67 ++++++++++ tests/CMakeLists.txt | 2 + tests/SpotFootprintTest.cpp | 68 ++++++++++ 13 files changed, 560 insertions(+), 23 deletions(-) create mode 100644 image_analysis/bragg_integration/SpotFootprint.cpp create mode 100644 image_analysis/bragg_integration/SpotFootprint.h create mode 100644 tests/SpotFootprintTest.cpp diff --git a/common/BraggIntegrationSettings.cpp b/common/BraggIntegrationSettings.cpp index ae8086b26..03b1a92bf 100644 --- a/common/BraggIntegrationSettings.cpp +++ b/common/BraggIntegrationSettings.cpp @@ -204,3 +204,16 @@ BraggIntegrationSettings &BraggIntegrationSettings::FlightPath(FlightPathMedium FlightPathMedium BraggIntegrationSettings::GetFlightPath() const { return flight_path; } + +BraggIntegrationSettings &BraggIntegrationSettings::Footprint(const SpotFootprint &input) { + if (input.sigma_rad.size() != input.sigma_tan.size() + || input.sigma_rad.size() > static_cast(SpotFootprint::MAX_BINS) + || (!input.empty() && !(input.bin_px > 0.0f))) + throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Invalid spot footprint table"); + footprint = input; + return *this; +} + +const SpotFootprint &BraggIntegrationSettings::GetFootprint() const { + return footprint; +} diff --git a/common/BraggIntegrationSettings.h b/common/BraggIntegrationSettings.h index 4e4b5645e..3555ccbdc 100644 --- a/common/BraggIntegrationSettings.h +++ b/common/BraggIntegrationSettings.h @@ -4,6 +4,7 @@ #pragma once #include +#include // Spot-intensity extraction method used by the Bragg integration engine. ProfileGaussian (default) // profile-fits with a measured-width Gaussian (Kabsch-style) - more accurate intensities than the @@ -34,6 +35,18 @@ enum class FlightPathMedium { Air, Helium, Vacuum }; // the max_hkl default in broker/jfjoch_api.yaml, so an omitting client and an omitting config agree. constexpr int BRAGG_ONLINE_DEFAULT_MAX_HKL = 100; +// The spot footprint measured on the data (image_analysis/bragg_integration/SpotFootprint.h): the +// radial and tangential standard deviations of a spot [px], tabulated against the distance from the +// beam centre in bins of bin_px. Empty until measured; the integrator then uses the fixed r1/r2/r3 +// stencil alone. +struct SpotFootprint { + static constexpr int MAX_BINS = 16; + float bin_px = 0.0f; + std::vector sigma_rad; + std::vector sigma_tan; + [[nodiscard]] bool empty() const { return sigma_rad.empty(); } +}; + class BraggIntegrationSettings { IntegratorMode integrator_mode = IntegratorMode::ProfileGaussian; float r_1 = 4; @@ -148,6 +161,7 @@ class BraggIntegrationSettings { // threshold fitted between two points rather than a physical criterion. The honest design is an // explicit option, a stated default, and the assumption printed in the report. FlightPathMedium flight_path = FlightPathMedium::Air; + SpotFootprint footprint; public: BraggIntegrationSettings& R1(float input); @@ -165,6 +179,7 @@ public: BraggIntegrationSettings& Overlap(OverlapMode input); BraggIntegrationSettings& OverlapMinPeak(float input); BraggIntegrationSettings& FlightPath(FlightPathMedium input); + BraggIntegrationSettings& Footprint(const SpotFootprint &input); [[nodiscard]] IntegratorMode GetIntegrator() const; @@ -186,4 +201,5 @@ public: [[nodiscard]] float GetOverlapMinPeak() const; // See flight_path: air unless the user said otherwise, which no file can say for them. [[nodiscard]] FlightPathMedium GetFlightPath() const; + [[nodiscard]] const SpotFootprint &GetFootprint() const; }; diff --git a/image_analysis/bragg_integration/BraggIntegrationEngine.cpp b/image_analysis/bragg_integration/BraggIntegrationEngine.cpp index a0a8b263d..5a6e85db0 100644 --- a/image_analysis/bragg_integration/BraggIntegrationEngine.cpp +++ b/image_analysis/bragg_integration/BraggIntegrationEngine.cpp @@ -96,6 +96,19 @@ BraggIntegrationEngine::BraggIntegrationEngine(const DiffractionExperiment &expe stencil.bw_sigma = static_cast(bw_sigma); stencil.k_sigma = settings.GetStencilKSigma(); stencil.max_grow = bragg_engine::MAX_STENCIL_GROW_OVER_R3 * r3; + // The measured spot footprint (SpotFootprint.h). It moves the ring only where a spot outgrows the + // r1 disk, so a pattern of compact spots integrates exactly as without it. + static_assert(SpotFootprint::MAX_BINS <= BRAGG_FOOTPRINT_MAX_BINS); + stencil.r1 = settings.GetR1(); + const auto &fp = settings.GetFootprint(); + if (!empirical && !fp.empty()) { + stencil.fp_n = static_cast(fp.sigma_rad.size()); + stencil.fp_bin_px = fp.bin_px; + for (int i = 0; i < stencil.fp_n; ++i) { + stencil.fp_sigma_rad[i] = fp.sigma_rad[i]; + stencil.fp_sigma_tan[i] = fp.sigma_tan[i]; + } + } // Robust background ring, one estimator or the other (see BraggIntegrationSettings): a high-side // sigma-clip (rugnux --background-clip, the default) or, when the clip is switched off, a @@ -137,7 +150,7 @@ BraggIntegrationEngine::BraggIntegrationEngine(const DiffractionExperiment &expe r_max = std::hypot(std::max(beam_x, static_cast(xpixel) - beam_x), std::max(beam_y, static_cast(ypixel) - beam_y)); bkg_radial_built = bkg_radial || bkg_radial_auto; - const float grow_max = bkg_radial_built ? BraggStencilGrow_px(static_cast(r_max), stencil) + const float grow_max = bkg_radial_built ? BraggStencilMaxGrow_px(static_cast(r_max), stencil) : 0.0f; n_kern = static_cast(std::lround(grow_max)) + 1; // Every row must fit: the last one is built at grow = n_kern - 1, which rounding can put just diff --git a/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp b/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp index 9c5d509d9..f7a46ac8d 100644 --- a/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp +++ b/image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp @@ -224,7 +224,7 @@ std::vector BraggIntegrationEngineCPU::RunImpl(const Sampler &img, for (int x = x0; x <= x1; ++x) { const auto d = BraggStencilDistances(st, x - r.predicted_x, y - r.predicted_y); const int32_t px = img[y * W + x]; - if (d.signal < r1_sq) { + if (BraggStencilInSignal(st, x - r.predicted_x, y - r.predicted_y, d.signal, r1_sq)) { // A pixel a nearer neighbour owns carries that neighbour's flux, so in Exclude // mode it leaves the disk entirely - the sum, the pixel count the background is // subtracted with, and the all-or-nothing validity gate alike. The box sum only @@ -519,11 +519,12 @@ std::vector BraggIntegrationEngineCPU::RunImpl(const Sampler &img, int Rf = R; const std::vector *Pvec = &shell_P[sh]; + const BraggStencil st = MakeBraggStencil(predicted[i].predicted_x, predicted[i].predicted_y, stencil); if (!empirical) { const double rx = predicted[i].predicted_x - beam_x, ry = predicted[i].predicted_y - beam_y; const double Rpx = std::hypot(rx, ry); const double tan2t = Rpx / F_px; - const double s2t = shell_sigma2[sh].tan; + double s2t = shell_sigma2[sh].tan; double s2r = s2t, ux = 1.0, uy = 0.0; bool elong = false; if (use_ellipse) { @@ -539,10 +540,18 @@ std::vector BraggIntegrationEngineCPU::RunImpl(const Sampler &img, elong = true; } } + // A spot that outgrows the r1 disk has widths the disk cannot measure - the moments above + // saturate near r1^2/4 - so the measured footprint sets them instead (SpotFootprint.h). + if (st.fp_s2t > 0.0f) { + ux = st.ux; uy = st.uy; + s2t = std::max(s2t, st.fp_s2t); + s2r = std::max(s2r, st.fp_s2r); + elong = true; + } // Build the Gaussian per reflection, centred on the sub-pixel predicted position and (when - // needed) radially elongated, on a grid grown to hold the streak. + // needed) elongated, on a grid grown to hold the wider of its two axes. const double fx = predicted[i].predicted_x - rh.cx, fy = predicted[i].predicted_y - rh.cy; - Rf = elong ? std::min(3 * R, static_cast(std::ceil(r2 + 2.0 * std::sqrt(s2r)))) : R; + Rf = elong ? std::min(3 * R, static_cast(std::ceil(r2 + 2.0 * std::sqrt(std::max(s2r, s2t))))) : R; // The grid's image rows, asked for now (as in pass A): building the profile below takes // long enough for them to arrive before the fit reads them. const int gx0 = std::max(0, rh.cx - Rf), gx1 = std::min(W - 1, rh.cx + Rf); @@ -582,7 +591,14 @@ std::vector BraggIntegrationEngineCPU::RunImpl(const Sampler &img, if (Pp <= 0.0) continue; const int x = rh.cx + dx, y = rh.cy + dy; if (x < 0 || y < 0 || x >= W || y >= H) continue; - const bool in_disk = dx * dx + dy * dy < r1_sq; + // The region the summation seed was taken over (pass A), so the guard below + // compares the two over the same pixels. + const bool in_disk = st.fp_s2t > 0.0f + ? BraggStencilInSignal(st, x - predicted[i].predicted_x, y - predicted[i].predicted_y, + (x - predicted[i].predicted_x) * (x - predicted[i].predicted_x) + + (y - predicted[i].predicted_y) * (y - predicted[i].predicted_y), + r1_sq) + : dx * dx + dy * dy < r1_sq; p_grid += Pp; p_peak = std::max(p_peak, Pp); if (in_disk) m_all += Pp; diff --git a/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu b/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu index 4ff0d148b..3f4e3d4ca 100644 --- a/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu +++ b/image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu @@ -220,7 +220,7 @@ __global__ void boxsum(const float *px_x, const float *px_y, const float *dd, // The window is the bounding box of the outer ellipse, so nearly half of it is neither the // signal disk nor the background ring. Reading the pixel only once it is known to be wanted // keeps those slots from fetching a cache line for nothing. - if (d.signal < p.r1_sq) { + if (BraggStencilInSignal(st, (float) x - cx, (float) y - cy, d.signal, p.r1_sq)) { // A pixel a nearer neighbour owns carries that neighbour's flux; see the CPU engine. ++l_nd; if (p.overlap) { @@ -625,6 +625,7 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x if (!ok_a[i]) { if (threadIdx.x == 0) ok_o[i] = 0; return; } const int cx = cx_a[i], cy = cy_a[i]; + const BraggStencil st = MakeBraggStencil(px_x[i], px_y[i], p.stencil); // As in learn_profile: one thread per block, not one per thread. Two double-precision divisions // on a card whose double throughput is a sixty-fourth of its single is worth doing once. __shared__ int s_sh; @@ -642,7 +643,7 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x __syncthreads(); } else { const int si = use_shell ? sh : N_SHELL; // N_SHELL is the global slot - const float s2t = sigma2_t[si]; + float s2t = sigma2_t[si]; const float rx = px_x[i] - p.beam_x, ry = px_y[i] - p.beam_y; const float Rpx = sqrtf(rx * rx + ry * ry); const float tan2t = Rpx / p.F_px; @@ -655,7 +656,14 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x const float radial_extra = fmaxf(sigma2_r[si] - s2t, sbw * sbw + p.c_radial * tan2t * tan2t); if (Rpx > 1e-6f && radial_extra > 0.25f) { ux = rx / Rpx; uy = ry / Rpx; s2r = s2t + radial_extra; elong = true; } } - const int Rf = elong ? min(3 * p.R, (int) ceilf(p.r2 + 2.0f * sqrtf(s2r))) : p.R; + // A spot that outgrows the r1 disk takes the measured footprint's widths (see the CPU engine). + if (st.fp_s2t > 0.0f) { + ux = st.ux; uy = st.uy; + s2t = fmaxf(s2t, st.fp_s2t); + s2r = fmaxf(s2r, st.fp_s2r); + elong = true; + } + const int Rf = elong ? min(3 * p.R, (int) ceilf(p.r2 + 2.0f * sqrtf(fmaxf(s2r, s2t)))) : p.R; const int Gf = 2 * Rf + 1; if (threadIdx.x == 0) { s_Rf = Rf; s_Gf = Gf; } __syncthreads(); @@ -697,7 +705,11 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x const int dx = k % Gf - Rf, dy = k / Gf - Rf; const int x = cx + dx, y = cy + dy; if (x < 0 || y < 0 || x >= p.W || y >= p.H) continue; - const bool in_disk = (float) (dx * dx + dy * dy) < p.r1_sq; + // The region the summation seed was taken over (boxsum), as on the CPU. + const float ddx = (float) x - px_x[i], ddy = (float) y - px_y[i]; + const bool in_disk = st.fp_s2t > 0.0f + ? BraggStencilInSignal(st, ddx, ddy, ddx * ddx + ddy * ddy, p.r1_sq) + : (float) (dx * dx + dy * dy) < p.r1_sq; l_pgrid += Pp; l_ppeak = fmaxf(l_ppeak, Pp); if (in_disk) l_mall += Pp; @@ -819,11 +831,11 @@ BraggIntegrationEngineGPU::BraggIntegrationEngineGPU(const DiffractionExperiment // Shared radial window of the background curve in boxsum. The stencil spans r0 +- the radial // semi-axis of the outer ellipse, widest at the far corner of the detector, and the window // covers twice that so a reflection anywhere on the detector fits. - rad_w = 2 * (static_cast(std::ceil(r3 + BraggStencilGrow_px(static_cast(r_max), stencil))) + 1) + 1; + rad_w = 2 * (static_cast(std::ceil(r3 + BraggStencilMaxGrow_px(static_cast(r_max), stencil))) + 1) + 1; // The pixels one reflection can mark: the box mark_mask walks, at the widest aperture on the // detector. Run() weighs that against the frame to decide how to clear the mask afterwards. - const int mark_half = static_cast(std::ceil(r2 + BraggStencilGrow_px(static_cast(r_max), stencil))) + 1; + const int mark_half = static_cast(std::ceil(r2 + BraggStencilMaxGrow_px(static_cast(r_max), stencil))) + 1; mask_box_px = static_cast(2 * mark_half + 1) * static_cast(2 * mark_half + 1); boxsum_shared_bytes = static_cast(rad_w) * (sizeof(unsigned long long) + sizeof(int)); diff --git a/image_analysis/bragg_integration/BraggStencil.h b/image_analysis/bragg_integration/BraggStencil.h index d85866f2f..bc49cbd06 100644 --- a/image_analysis/bragg_integration/BraggStencil.h +++ b/image_analysis/bragg_integration/BraggStencil.h @@ -65,13 +65,27 @@ #define BRAGG_STENCIL_HD inline #endif +// The measured spot footprint (SpotFootprint.h): the radial and tangential standard deviations of a +// spot [px], tabulated against distance from the beam centre in bins of fp_bin_px. Held here, by value, +// because both engines build every stencil from these parameters - the GPU gets the table with the +// kernel arguments and needs no buffer of its own. +constexpr int BRAGG_FOOTPRINT_MAX_BINS = 16; +// A spot reaches about this many of its standard deviations: the background ring starts there and the +// profile is fitted out to it. +constexpr float BRAGG_FOOTPRINT_NSIGMA = 3.0f; + // Fixed per-experiment inputs to the stencil law (mirrors BraggIntegrationEngine's members). struct BraggStencilParams { float beam_x = 0.0f, beam_y = 0.0f; + float r1 = 4.0f; float r2 = 6.0f, r3 = 10.0f; float bw_sigma = 0.0f; // radial streak per pixel of radius (bandwidth sigma, dimensionless) float k_sigma = 0.0f; // radial sigmas to push the ring out by; 0 = the old circular stencil float max_grow = 0.0f; // hard cap on the radial growth [px]; <= 0 means uncapped + int fp_n = 0; // bins of the measured footprint; 0 = none measured + float fp_bin_px = 0.0f; + float fp_sigma_rad[BRAGG_FOOTPRINT_MAX_BINS] = {}; + float fp_sigma_tan[BRAGG_FOOTPRINT_MAX_BINS] = {}; }; // One reflection's stencil, in its own radial/tangential frame. @@ -79,7 +93,15 @@ struct BraggStencil { float ux = 1.0f, uy = 0.0f; // unit vector beam -> reflection float r0 = 0.0f; // distance from the beam centre [px] float grow = 0.0f; // radial growth of the ring [px] (0 = circular) + float grow_tan = 0.0f; // tangential growth of the ring [px] (footprint only) float q_in = 0.0f, q_out = 0.0f; // radial shrink coefficients (0 = circular) + float qt_in = 0.0f, qt_out = 0.0f; // tangential shrink coefficients (0 = circular) + // The measured footprint where the spot outgrows the r1 disk (variances [px^2]), else 0: the + // profile fit takes its Gaussian at least this wide. + float fp_s2r = 0.0f, fp_s2t = 0.0f; + // 1 / (BRAGG_FOOTPRINT_NSIGMA * sigma)^2 along and across the radius: the footprint ellipse a wide + // spot is SUMMED over, beside the r1 disk (BraggStencilInSignal). + float core_r = 0.0f, core_t = 0.0f; float ex_in = 0.0f, ey_in = 0.0f; // axis-aligned half-extent of the inner (r2) ellipse float ex_out = 0.0f, ey_out = 0.0f; // axis-aligned half-extent of the outer (r3) ellipse }; @@ -94,6 +116,38 @@ BRAGG_STENCIL_HD float BraggStencilGrow_px(float Rpx, const BraggStencilParams & return (p.max_grow > 0.0f && grow > p.max_grow) ? p.max_grow : grow; } +// The footprint at Rpx: linear between bin centres, flat beyond the first and the last. +BRAGG_STENCIL_HD void BraggFootprintAt(float Rpx, const BraggStencilParams &p, float &s_rad, float &s_tan) { + const float t = Rpx / p.fp_bin_px - 0.5f; + int i = (int) floorf(t); + float f = t - (float) i; + if (i < 0) { i = 0; f = 0.0f; } + if (i >= p.fp_n - 1) { i = p.fp_n - 1; f = 0.0f; } + const int j = i + 1 < p.fp_n ? i + 1 : i; + s_rad = p.fp_sigma_rad[i] + f * (p.fp_sigma_rad[j] - p.fp_sigma_rad[i]); + s_tan = p.fp_sigma_tan[i] + f * (p.fp_sigma_tan[j] - p.fp_sigma_tan[i]); +} + +// Growth of the ring beyond r2 that the footprint asks for along one axis: none while the spot fits +// the r1 disk - there the disk resolves the spot and the stencil stays what it always was - and +// otherwise out to BRAGG_FOOTPRINT_NSIGMA of it, capped as the bandwidth growth is. +BRAGG_STENCIL_HD float BraggFootprintGrow_px(float sigma, const BraggStencilParams &p) { + const float g = BRAGG_FOOTPRINT_NSIGMA * sigma - p.r2; + if (!(g > 0.0f)) return 0.0f; + return (p.max_grow > 0.0f && g > p.max_grow) ? p.max_grow : g; +} + +// The largest growth any reflection on a detector reaching r_max can get, radially - what the +// buffers that hold a ring are sized by. +BRAGG_STENCIL_HD float BraggStencilMaxGrow_px(float r_max, const BraggStencilParams &p) { + float g = BraggStencilGrow_px(r_max, p); + for (int i = 0; i < p.fp_n; ++i) { + const float s = fmaxf(p.fp_sigma_rad[i], p.fp_sigma_tan[i]); + if (BRAGG_FOOTPRINT_NSIGMA * s > p.r1) g = fmaxf(g, BraggFootprintGrow_px(s, p)); + } + return g; +} + BRAGG_STENCIL_HD BraggStencil MakeBraggStencil(float px_x, float px_y, const BraggStencilParams &p) { BraggStencil s; const float rx = px_x - p.beam_x, ry = px_y - p.beam_y; @@ -105,24 +159,44 @@ BRAGG_STENCIL_HD BraggStencil MakeBraggStencil(float px_x, float px_y, const Bra } s.grow = BraggStencilGrow_px(r0, p); - const float grow = s.grow; - if (grow == 0.0f) { // the common case, and the only one the online path can reach + // A spot wider than the r1 disk can resolve keeps its background ring clear of itself: the ring + // starts BRAGG_FOOTPRINT_NSIGMA of the measured footprint out, radially and tangentially apart. + if (p.fp_n > 0) { + float s_rad, s_tan; + BraggFootprintAt(r0, p, s_rad, s_tan); + if (BRAGG_FOOTPRINT_NSIGMA * fmaxf(s_rad, s_tan) > p.r1) { + s.fp_s2r = s_rad * s_rad; + s.fp_s2t = s_tan * s_tan; + s.core_r = 1.0f / (BRAGG_FOOTPRINT_NSIGMA * BRAGG_FOOTPRINT_NSIGMA * s.fp_s2r); + s.core_t = 1.0f / (BRAGG_FOOTPRINT_NSIGMA * BRAGG_FOOTPRINT_NSIGMA * s.fp_s2t); + s.grow = fmaxf(s.grow, BraggFootprintGrow_px(s_rad, p)); + s.grow_tan = BraggFootprintGrow_px(s_tan, p); + } + } + const float grow = s.grow, grow_t = s.grow_tan; + if (grow == 0.0f && grow_t == 0.0f) { // the common case, and the only one the online path can reach s.ex_in = p.r2; s.ey_in = p.r2; s.ex_out = p.r3; s.ey_out = p.r3; return s; } - const float a_in = p.r2 + grow, a_out = p.r3 + grow; + const float a_in = p.r2 + grow, a_out = p.r3 + grow; // semi-axes along the radius + const float b_in = p.r2 + grow_t, b_out = p.r3 + grow_t; // and across it const float si = p.r2 / a_in, so = p.r3 / a_out; s.q_in = 1.0f - si * si; s.q_out = 1.0f - so * so; + if (grow_t > 0.0f) { + const float ti = p.r2 / b_in, to = p.r3 / b_out; + s.qt_in = 1.0f - ti * ti; + s.qt_out = 1.0f - to * to; + } // Axis-aligned half-extents of an ellipse with semi-axes (a along u, b across it). const float ux2 = s.ux * s.ux, uy2 = s.uy * s.uy; - s.ex_in = sqrtf(a_in * a_in * ux2 + p.r2 * p.r2 * uy2); - s.ey_in = sqrtf(a_in * a_in * uy2 + p.r2 * p.r2 * ux2); - s.ex_out = sqrtf(a_out * a_out * ux2 + p.r3 * p.r3 * uy2); - s.ey_out = sqrtf(a_out * a_out * uy2 + p.r3 * p.r3 * ux2); + s.ex_in = sqrtf(a_in * a_in * ux2 + b_in * b_in * uy2); + s.ey_in = sqrtf(a_in * a_in * uy2 + b_in * b_in * ux2); + s.ex_out = sqrtf(a_out * a_out * ux2 + b_out * b_out * uy2); + s.ey_out = sqrtf(a_out * a_out * uy2 + b_out * b_out * ux2); return s; } @@ -183,5 +257,20 @@ BRAGG_STENCIL_HD BraggStencilDist BraggStencilDistances(const BraggStencil &s, f const float rad2 = d.rad * d.rad; d.inner = d.signal - s.q_in * rad2; d.outer = d.signal - s.q_out * rad2; + if (s.qt_in != 0.0f) { // grown tangentially as well (the footprint); tan^2 = d2 - rad^2 + const float tan2 = d.signal - rad2; + d.inner -= s.qt_in * tan2; + d.outer -= s.qt_out * tan2; + } return d; } + +// The signal region: the r1 disk, and for a spot that outgrows it the whole footprint ellipse too, so +// the summation - the seed of the profile fit, and what the fit falls back to - holds the spot rather +// than its core. Without a footprint this is the plain disk test, bit for bit. +BRAGG_STENCIL_HD bool BraggStencilInSignal(const BraggStencil &s, float ddx, float ddy, float d2, float r1_sq) { + if (d2 < r1_sq) return true; + if (!(s.fp_s2t > 0.0f)) return false; + const float rad = ddx * s.ux + ddy * s.uy, tn = -ddx * s.uy + ddy * s.ux; + return rad * rad * s.core_r + tn * tn * s.core_t < 1.0f; +} diff --git a/image_analysis/bragg_integration/CMakeLists.txt b/image_analysis/bragg_integration/CMakeLists.txt index 7978c0439..aa8b3cd80 100644 --- a/image_analysis/bragg_integration/CMakeLists.txt +++ b/image_analysis/bragg_integration/CMakeLists.txt @@ -3,6 +3,8 @@ ADD_LIBRARY(JFJochBraggIntegration STATIC BraggIntegrationEngine.h BraggIntegrationEngineCPU.cpp BraggIntegrationEngineCPU.h + SpotFootprint.cpp + SpotFootprint.h Regression.h CalcISigma.cpp CalcISigma.h diff --git a/image_analysis/bragg_integration/SpotFootprint.cpp b/image_analysis/bragg_integration/SpotFootprint.cpp new file mode 100644 index 000000000..bfa6d0257 --- /dev/null +++ b/image_analysis/bragg_integration/SpotFootprint.cpp @@ -0,0 +1,126 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#include "SpotFootprint.h" + +#include +#include + +namespace { + +// The window around a spot is BRAGG_FOOTPRINT_NSIGMA-like: three of its standard deviations, at least +// a few pixels and at most this many, which is wider than any spot the integrator could hold. +constexpr float WINDOW_NSIGMA = 3.0f; +constexpr float WINDOW_MIN_PX = 3.0f; +constexpr float WINDOW_MAX_PX = 24.0f; +// The background is the median of an elliptical annulus between these multiples of the window. +constexpr float BKG_INNER = 1.5f; +constexpr float BKG_OUTER = 2.2f; +constexpr int ITERATIONS = 8; + +inline bool valid(int32_t v) { return v != INT32_MIN && v != INT32_MAX; } + +float median_of(std::vector &v) { + const size_t m = v.size() / 2; + std::nth_element(v.begin(), v.begin() + static_cast(m), v.end()); + return v[m]; +} + +} // namespace + +void MeasureFootprintSpots(const int32_t *img, int width, int height, float beam_x, float beam_y, + const std::vector &x, const std::vector &y, + std::vector &out) { + const int half = static_cast(std::ceil(BKG_OUTER * WINDOW_MAX_PX)) + 1; + std::vector ring; + for (size_t s = 0; s < x.size(); ++s) { + const float rx = x[s] - beam_x, ry = y[s] - beam_y; + const float r = std::sqrt(rx * rx + ry * ry); + if (!(r > 1.0f)) continue; + const float ux = rx / r, uy = ry / r; + const int ix = static_cast(std::lround(x[s])), iy = static_cast(std::lround(y[s])); + if (ix - half < 0 || iy - half < 0 || ix + half >= width || iy + half >= height) continue; + + // Start from a compact spot at the prediction; each round re-centres on the signal and takes + // the window to three of the widths just measured. + float cx = x[s], cy = y[s]; + float s2r = 1.0f, s2t = 1.0f; + bool ok = true; + for (int it = 0; it < ITERATIONS && ok; ++it) { + const float wr = std::clamp(WINDOW_NSIGMA * std::sqrt(s2r), WINDOW_MIN_PX, WINDOW_MAX_PX); + const float wt = std::clamp(WINDOW_NSIGMA * std::sqrt(s2t), WINDOW_MIN_PX, WINDOW_MAX_PX); + const float reach = BKG_OUTER * std::max(wr, wt); + const int x0 = static_cast(std::floor(cx - reach)), x1 = static_cast(std::ceil(cx + reach)); + const int y0 = static_cast(std::floor(cy - reach)), y1 = static_cast(std::ceil(cy + reach)); + if (x0 < 0 || y0 < 0 || x1 >= width || y1 >= height) { ok = false; break; } + + ring.clear(); + for (int py = y0; py <= y1; ++py) + for (int px = x0; px <= x1; ++px) { + const float dx = px - cx, dy = py - cy; + const float rad = dx * ux + dy * uy, tn = -dx * uy + dy * ux; + const float e = rad * rad / (wr * wr) + tn * tn / (wt * wt); + const int32_t v = img[static_cast(py) * width + px]; + if (e >= BKG_INNER * BKG_INNER && e < BKG_OUTER * BKG_OUTER && valid(v)) + ring.push_back(static_cast(v)); + } + if (ring.size() < 10) { ok = false; break; } + const double bkg = median_of(ring); + + double w = 0.0, mr = 0.0, mt = 0.0, m2r = 0.0, m2t = 0.0; + for (int py = y0; py <= y1 && ok; ++py) + for (int px = x0; px <= x1; ++px) { + const float dx = px - cx, dy = py - cy; + const float rad = dx * ux + dy * uy, tn = -dx * uy + dy * ux; + if (rad * rad / (wr * wr) + tn * tn / (wt * wt) >= 1.0f) continue; + const int32_t v = img[static_cast(py) * width + px]; + if (!valid(v)) { ok = false; break; } + const double net = static_cast(v) - bkg; + w += net; + mr += net * rad; + mt += net * tn; + m2r += net * rad * rad; + m2t += net * tn * tn; + } + if (!ok || !(w > 0.0)) { ok = false; break; } + const double cr = mr / w, ct = mt / w; + s2r = static_cast(std::max(0.25, m2r / w - cr * cr)); + s2t = static_cast(std::max(0.25, m2t / w - ct * ct)); + cx += static_cast(cr * ux - ct * uy); + cy += static_cast(cr * uy + ct * ux); + } + if (ok) + out.push_back({r, std::sqrt(s2r), std::sqrt(s2t)}); + } +} + +SpotFootprint FootprintFromSpots(const std::vector &spots, float r_max) { + SpotFootprint fp; + if (!(r_max > 0.0f)) return fp; + const float bin = r_max / FOOTPRINT_BINS; + std::vector> rad(FOOTPRINT_BINS), tan(FOOTPRINT_BINS); + for (const auto &s : spots) { + const int b = std::clamp(static_cast(s.r_px / bin), 0, FOOTPRINT_BINS - 1); + rad[b].push_back(s.sigma_rad); + tan[b].push_back(s.sigma_tan); + } + std::vector filled; + std::vector mr(FOOTPRINT_BINS, 0.0f), mt(FOOTPRINT_BINS, 0.0f); + for (int b = 0; b < FOOTPRINT_BINS; ++b) + if (static_cast(rad[b].size()) >= FOOTPRINT_MIN_SPOTS_PER_BIN) { + mr[b] = median_of(rad[b]); + mt[b] = median_of(tan[b]); + filled.push_back(b); + } + if (filled.empty()) return fp; + fp.bin_px = bin; + for (int b = 0; b < FOOTPRINT_BINS; ++b) { + // The nearest bin that has enough spots; the inner one on a tie. + int best = filled.front(); + for (int f : filled) + if (std::abs(f - b) < std::abs(best - b)) best = f; + fp.sigma_rad.push_back(mr[best]); + fp.sigma_tan.push_back(mt[best]); + } + return fp; +} diff --git a/image_analysis/bragg_integration/SpotFootprint.h b/image_analysis/bragg_integration/SpotFootprint.h new file mode 100644 index 000000000..d973481a2 --- /dev/null +++ b/image_analysis/bragg_integration/SpotFootprint.h @@ -0,0 +1,53 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#pragma once + +// ============================================================================= +// SpotFootprint - how far a spot really reaches, measured on the data +// ============================================================================= +// +// The integrator's r1 disk and r2..r3 background ring are fixed in pixels, chosen from spots near the +// beam. Away from it a spot can grow several times wider: radially from the sensor's parallax and the +// obliquity of the incidence, tangentially from the crystal's own spread (an azimuthal mosaic spread +// rotates the diffracted beam about the incident one, smearing the spot along the ring). Measured on +// small-molecule data at 20 keV, the standard deviation goes from 1 px near the beam to 5 px at the +// edge, so the fixed disk holds a quarter of the flux there and the background ring a third of it - +// an intensity loss that grows with resolution and that no scaling can see, because symmetry mates +// share their resolution. +// +// The widths the integrator learns itself cannot follow: they are second moments taken inside the r1 +// disk, so they saturate near r1^2/4. Here each strong spot is measured with a window that follows +// the spot instead - three of its own standard deviations, iterated - separately along the radius and +// across it, and the widths are tabulated against the distance from the beam centre. The table goes +// to the integrator through BraggIntegrationSettings; where it says a spot outgrows the r1 disk, the +// background ring is moved clear of the spot and the profile is fitted at the measured width +// (BraggStencil.h). Compact spots leave the integration exactly as it was. +// ============================================================================= + +#include +#include + +#include "../../common/BraggIntegrationSettings.h" + +// One spot's measured widths, and its distance from the beam centre [px]. +struct FootprintSpot { + float r_px = 0.0f; + float sigma_rad = 0.0f; + float sigma_tan = 0.0f; +}; + +// Measure the spots at the given (sub-pixel) positions on one preprocessed frame (INT32_MIN masked, +// INT32_MAX saturated). A spot whose window holds an unreadable pixel, or no signal above its +// background, is left out. Results are appended in the order of the positions. +void MeasureFootprintSpots(const int32_t *img, int width, int height, float beam_x, float beam_y, + const std::vector &x, const std::vector &y, + std::vector &out); + +// Tabulate the spots: the median widths in bins of distance from the beam centre out to r_max. A bin +// holding too few spots takes the nearest bin that has enough; with no such bin the table is empty. +SpotFootprint FootprintFromSpots(const std::vector &spots, float r_max); + +// Spots per distance bin a table needs before it is believed. +constexpr int FOOTPRINT_MIN_SPOTS_PER_BIN = 20; +constexpr int FOOTPRINT_BINS = 12; diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 720f41689..0800758b0 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -7,6 +7,8 @@ #include "WriteModel.h" #include "SpindleCuspLoss.h" #include "SpotWidth.h" +#include "../image_analysis/bragg_integration/SpotFootprint.h" +#include "../image_analysis/bragg_integration/BraggStencil.h" // BRAGG_FOOTPRINT_NSIGMA #include "HotPixels.h" #include "../image_analysis/SensorAbsorption.h" #include "DiagnosticOutput.h" @@ -833,6 +835,7 @@ std::optional Rugnux::RebinAndRefit(const CalibrationResult & return best; } + void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, RugnuxObserver *observer) { Logger logger("Rugnux"); @@ -975,7 +978,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru const auto find_spots = [&](PreScanWorker &w, CompressedImage &image, int image_idx, bool for_beam_center, std::vector &out, bool for_width, std::vector &curves, - std::vector &spot_q) { + std::vector &spot_q, std::vector &footprint) { try { // As in the image loops: a frame the device cannot decode goes to the host decoder. ImageStatistics stats; @@ -1010,10 +1013,23 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru } } // The width goes first: it wants the whole spot list, to know which spots are isolated. - if (for_width) + if (for_width) { MeasureSpotFluxCurves(*w.preprocessed, static_cast(prescan_x.GetXPixelsNumConv()), static_cast(prescan_x.GetYPixelsNumConv()), prescan_x.GetDiffractionGeometry(), spots, curves); + // The footprint is measured on every spot, not only the isolated ones the width takes: + // it is the far, wide spots it is for (SpotFootprint.h). + std::vector fx, fy; + for (const auto &spot : spots) { + const Coord c = spot.RawCoord(); + fx.push_back(c.x); + fy.push_back(c.y); + } + const auto &g = prescan_x.GetDiffractionGeometry(); + MeasureFootprintSpots(w.preprocessed->data(), static_cast(prescan_x.GetXPixelsNumConv()), + static_cast(prescan_x.GetYPixelsNumConv()), + g.GetBeamX_pxl(), g.GetBeamY_pxl(), fx, fy, footprint); + } if (!for_beam_center) return; // The strongest of a crowded frame: the symmetry is over-determined either way, and the @@ -1037,6 +1053,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru // The spots the bandwidth is measured on: the width's, and more where those are too few // (spot_width::BANDWIDTH_POOL_SPOTS). std::vector shape_curves; + // Every spot's measured widths, for the spot footprint (SpotFootprint.h). + std::vector footprint_spots; // Images the width ended up being measured on, for the log: the tiers below stop as soon as the // answer has settled, so this is a property of the crystal and worth reporting. size_t width_images = 0; @@ -1102,6 +1120,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru std::vector> spots_of(ordinals.size()); std::vector> curves_of(ordinals.size()); std::vector> spot_q_of(ordinals.size()); + std::vector> footprint_of(ordinals.size()); // A frame joins the pool only if it could be read, as it did when this was a serial loop. std::vector spot_read(ordinals.size(), 0); // Frames read for the bandwidth alone, past the tier the width settled on. Their spots stay @@ -1185,7 +1204,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru if (for_beam_center) spot_read[i] = 1; if (for_shape && !for_beam_center) shape_only[i] = 1; find_spots(w, w.raw_image.image, image_idx, for_beam_center, spots_of[i], - for_width || for_shape, curves_of[i], spot_q_of[i]); + for_width || for_shape, curves_of[i], spot_q_of[i], footprint_of[i]); } })); for (auto &f : futures) f.get(); @@ -1216,6 +1235,9 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru for (size_t i = 0; i < ordinals.size(); i++) if (!shape_only[i]) prescan_spot_q.insert(prescan_spot_q.end(), spot_q_of[i].begin(), spot_q_of[i].end()); + // In sample order, so the table does not depend on how the workers interleaved. + for (const auto &f : footprint_of) + footprint_spots.insert(footprint_spots.end(), f.begin(), f.end()); // Frame numbering follows the sample order, exactly as the serial read did. for (size_t i = 0; i < ordinals.size(); i++) { @@ -1278,6 +1300,43 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru "integration radii r1={:.1f} r2={:.1f} r3={:.2f}", *r80, spot_width::D_REF_A, width_curves.size(), width_images, r1, r2, r3); } + + // How far the spots reach away from the beam, where the radius chosen above at 5 A no + // longer holds them (SpotFootprint.h). It changes the integration only where a spot + // outgrows the r1 disk. + const auto &g = experiment_.GetDiffractionGeometry(); + const float bx = g.GetBeamX_pxl(), by = g.GetBeamY_pxl(); + const float W = static_cast(experiment_.GetXPixelsNum()); + const float H = static_cast(experiment_.GetYPixelsNum()); + const SpotFootprint fp = FootprintFromSpots(footprint_spots, + std::hypot(std::max(bx, W - bx), std::max(by, H - by))); + // It acts only where a spot outgrows the r1 disk (BraggStencil.h), so a pattern of compact + // spots is left on exactly the settings - and the passes - it had without it. + const float r1_now = experiment_.GetBraggIntegrationSettings().GetR1(); + bool outgrows = false; + for (size_t b = 0; b < fp.sigma_rad.size(); ++b) + outgrows |= BRAGG_FOOTPRINT_NSIGMA * std::max(fp.sigma_rad[b], fp.sigma_tan[b]) > r1_now; + if (fp.empty()) { + logger.Info("Spot footprint: too few spots to tabulate ({})", footprint_spots.size()); + } else if (!outgrows) { + logger.Info("Spot footprint from {} spots: every spot fits the r1={:.1f} disk", footprint_spots.size(), + r1_now); + } else { + std::string table; + for (size_t b = 0; b < fp.sigma_rad.size(); ++b) + table += fmt::format(" {:.0f}:{:.1f}/{:.1f}", (b + 0.5f) * fp.bin_px, + fp.sigma_rad[b], fp.sigma_tan[b]); + logger.Info("Spot footprint from {} spots, sigma radial/tangential [px] by distance from " + "the beam [px]:{}", footprint_spots.size(), table); + // On the adaptive side, like the radius: the geometry pre-pass integrates without it, + // and a canonical pass whose wider rings the neighbours starve falls back to the + // settings before it (bragg_before_adaptive_, see RunAllPasses). + BraggIntegrationSettings bis = experiment_.GetBraggIntegrationSettings(); + if (!bragg_before_adaptive_) + bragg_before_adaptive_ = bis; + bis.Footprint(fp); + experiment_.ImportBraggIntegrationSettings(bis); + } } // The X-ray bandwidth, read off the shapes of the same spots (spot_width::EstimateBandwidth). A @@ -1605,8 +1664,9 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru // The extra frames go to the beam centre alone; the width and the powder // rings have already been measured on the projection's own sample. std::vector unused_q; + std::vector unused_fp; find_spots(w, w.raw_image.image, image_idx, true, extra_spots[i], - false, width_curves, unused_q); + false, width_curves, unused_q, unused_fp); } } })); diff --git a/tests/BraggStencilTest.cpp b/tests/BraggStencilTest.cpp index 4d37aa86a..c6b9f9389 100644 --- a/tests/BraggStencilTest.cpp +++ b/tests/BraggStencilTest.cpp @@ -158,3 +158,70 @@ TEST_CASE("BraggStencil_KernelIndexInRange", "[Integration][portable]") { } } } + +namespace { + +// A footprint growing linearly from sigma 1 px at the beam to `edge` px at 800 px, radial and tangential +// alike unless told otherwise. +BraggStencilParams FootprintParams(float edge_rad, float edge_tan) { + BraggStencilParams p = Params(0.0f, 0.0f); + p.r1 = 4.0f; + p.fp_n = 8; + p.fp_bin_px = 100.0f; + for (int i = 0; i < p.fp_n; ++i) { + const float t = (i + 0.5f) / p.fp_n; + p.fp_sigma_rad[i] = 1.0f + t * (edge_rad - 1.0f); + p.fp_sigma_tan[i] = 1.0f + t * (edge_tan - 1.0f); + } + return p; +} + +} // namespace + +// Where the footprint fits the r1 disk (3 sigma <= r1) the stencil is the circular one, bit for bit: +// compact spots integrate exactly as without a footprint. +TEST_CASE("BraggStencil_FootprintInsideDiskIsInert", "[Integration]") { + const BraggStencilParams p = FootprintParams(1.3f, 1.3f); // 3 sigma < 4 everywhere + const BraggStencilParams none = Params(0.0f, 0.0f); + for (float py = 0.0f; py < 800.0f; py += 37.0f) + for (float px = 0.0f; px < 800.0f; px += 41.0f) { + const BraggStencil s = MakeBraggStencil(px, py, p); + const BraggStencil n = MakeBraggStencil(px, py, none); + REQUIRE(s.fp_s2r == 0.0f); + REQUIRE(s.grow == 0.0f); + REQUIRE(s.grow_tan == 0.0f); + for (int dy = -12; dy <= 12; ++dy) + for (int dx = -12; dx <= 12; ++dx) { + const auto d = BraggStencilDistances(s, static_cast(dx), static_cast(dy)); + const auto e = BraggStencilDistances(n, static_cast(dx), static_cast(dy)); + REQUIRE(d.inner == e.inner); + REQUIRE(d.outer == e.outer); + } + } +} + +// A spot wider than the disk pushes the ring to 3 sigma along each axis separately: a pixel at 3 sigma +// along the radius (or across it) is no longer background, one just beyond the grown ring's inner edge +// is. +TEST_CASE("BraggStencil_FootprintGrowsRingAlongEachAxis", "[Integration]") { + const BraggStencilParams p = FootprintParams(3.0f, 5.0f); + const float px = 400.0f + 700.0f, py = 400.0f; // on +x, 700 px out: radial = x, tangential = y + const BraggStencil s = MakeBraggStencil(px, py, p); + float sr, st; + BraggFootprintAt(700.0f, p, sr, st); + REQUIRE(s.fp_s2r == sr * sr); + REQUIRE(s.fp_s2t == st * st); + REQUIRE_THAT(s.grow, Catch::Matchers::WithinAbs(3.0f * sr - p.r2, 1e-4)); + REQUIRE_THAT(s.grow_tan, Catch::Matchers::WithinAbs(3.0f * st - p.r2, 1e-4)); + const float ar = p.r2 + s.grow, at = p.r2 + s.grow_tan; + // Just inside the inner ellipse along each axis: not background. + REQUIRE(BraggStencilDistances(s, 0.98f * ar, 0.0f).inner < p.r2 * p.r2); + REQUIRE(BraggStencilDistances(s, 0.0f, 0.98f * at).inner < p.r2 * p.r2); + // Just outside: background ring. + REQUIRE(BraggStencilDistances(s, 1.02f * ar, 0.0f).inner >= p.r2 * p.r2); + REQUIRE(BraggStencilDistances(s, 0.0f, 1.02f * at).inner >= p.r2 * p.r2); + // The bounding box holds the outer ellipse. + REQUIRE(s.ex_out >= p.r3 + s.grow - 1e-3f); + REQUIRE(s.ey_out >= p.r3 + s.grow_tan - 1e-3f); + REQUIRE(BraggStencilMaxGrow_px(1000.0f, p) >= std::max(s.grow, s.grow_tan)); +} diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index c6d6df0bb..816808b1b 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -17,6 +17,7 @@ ADD_EXECUTABLE(jfjoch_portable_test EXCLUDE_FROM_ALL AzimuthalIntegrationTest.cpp CalcBraggPredictionTest.cpp BraggStencilTest.cpp + SpotFootprintTest.cpp BraggIntegrationEngineCompressedImageTest.cpp BraggIntegrationEngineCPUTest.cpp WriteReflectionsTest.cpp @@ -79,6 +80,7 @@ ADD_EXECUTABLE(jfjoch_test BraggIntegrationEngineCompressedImageTest.cpp BraggIntegrationEngineCPUTest.cpp BraggStencilTest.cpp + SpotFootprintTest.cpp LossyFilterTest.cpp ImageBufferTest.cpp PixelMaskTest.cpp diff --git a/tests/SpotFootprintTest.cpp b/tests/SpotFootprintTest.cpp new file mode 100644 index 000000000..46699b50e --- /dev/null +++ b/tests/SpotFootprintTest.cpp @@ -0,0 +1,68 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#include +#include + +#include +#include + +#include "../image_analysis/bragg_integration/SpotFootprint.h" + +// Spots drawn as Gaussians elongated along and across the radius are measured back at their widths, +// whatever their azimuth, and tabulated by distance from the beam. +TEST_CASE("SpotFootprint_MeasuresRadialAndTangentialWidths", "[Integration][portable]") { + const int W = 1200, H = 1200; + const float bx = 600.0f, by = 600.0f; + std::vector img(static_cast(W) * H, 10); // flat background + std::vector xs, ys; + // Width grows with distance: sigma_rad = 1 + r/200, sigma_tan = 1 + r/100. + for (int k = 0; k < 400; ++k) { + const float r = 60.0f + 480.0f * static_cast(k % 20) / 20.0f; + const float phi = 0.61f * static_cast(k); + const float x = bx + r * std::cos(phi), y = by + r * std::sin(phi); + const float ux = std::cos(phi), uy = std::sin(phi); + const float sr = 1.0f + r / 200.0f, st = 1.0f + r / 100.0f; + for (int py = static_cast(y) - 30; py <= static_cast(y) + 30; ++py) + for (int px = static_cast(x) - 30; px <= static_cast(x) + 30; ++px) { + if (px < 0 || py < 0 || px >= W || py >= H) continue; + const float dx = px - x, dy = py - y; + const float rad = dx * ux + dy * uy, tn = -dx * uy + dy * ux; + img[static_cast(py) * W + px] += static_cast( + std::lround(2000.0f * std::exp(-rad * rad / (2 * sr * sr) - tn * tn / (2 * st * st)))); + } + xs.push_back(x); + ys.push_back(y); + } + // Overlapping spots are not what this checks: keep those far from every other one. + std::vector kx, ky; + for (size_t i = 0; i < xs.size(); ++i) { + bool alone = true; + for (size_t j = 0; j < xs.size(); ++j) + if (i != j && std::hypot(xs[i] - xs[j], ys[i] - ys[j]) < 45.0f) alone = false; + if (alone) { kx.push_back(xs[i]); ky.push_back(ys[i]); } + } + REQUIRE(kx.size() > 40); + std::vector spots; + MeasureFootprintSpots(img.data(), W, H, bx, by, kx, ky, spots); + REQUIRE(spots.size() > 30); + for (const auto &s : spots) { + CHECK_THAT(s.sigma_rad, Catch::Matchers::WithinRel(1.0f + s.r_px / 200.0f, 0.12f)); + CHECK_THAT(s.sigma_tan, Catch::Matchers::WithinRel(1.0f + s.r_px / 100.0f, 0.12f)); + } +} + +TEST_CASE("SpotFootprint_TableFillsSparseBinsFromNeighbours", "[Integration][portable]") { + std::vector spots; + for (int i = 0; i < FOOTPRINT_MIN_SPOTS_PER_BIN; ++i) { + spots.push_back({50.0f, 1.0f, 1.5f}); // bin 0 + spots.push_back({1150.0f, 3.0f, 4.0f}); // last bin + } + const SpotFootprint fp = FootprintFromSpots(spots, 1200.0f); + REQUIRE(fp.sigma_rad.size() == static_cast(FOOTPRINT_BINS)); + REQUIRE(fp.sigma_rad.front() == 1.0f); + REQUIRE(fp.sigma_tan.back() == 4.0f); + REQUIRE(fp.sigma_rad[2] == 1.0f); // nearer the first bin + REQUIRE(fp.sigma_rad[FOOTPRINT_BINS - 3] == 3.0f); // nearer the last + REQUIRE(FootprintFromSpots({}, 1200.0f).empty()); +} -- 2.54.0 From a6378800b85d2605f5d0a67dd18f32c6eb92e935 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 02:42:09 +0200 Subject: [PATCH 30/97] Footprint integration: GPU/CPU parity sections, docs, changelog BraggIntegrationEngineGPU_MatchesCPU gains three footprint sections (spaced, crowded under overlap exclude, with the radial background correction): both engines classify the summation ellipse, the grown ring and the footprint Gaussian alike. Integration chapter and changelog describe the measured footprint. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- docs/CHANGELOG.md | 1 + docs/CPU_DATA_ANALYSIS_INTEGRATION.md | 2 ++ tests/BraggIntegrationEngineGPUTest.cpp | 32 +++++++++++++++++++++++-- 3 files changed, 33 insertions(+), 2 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 9a34391b6..8cbe704c3 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,6 +4,7 @@ ### 1.0.0-rc.174 * Rugnux reads Rigaku d*TREK SMV images (Saturn CCD), including detector 2theta, image orientation and encoded pixel overflows. +* Rugnux integrates spots that grow wider than the integration disk away from the beam (typical of small molecules at high X-ray energy) over their measured footprint. ### 1.0.0-rc.173 diff --git a/docs/CPU_DATA_ANALYSIS_INTEGRATION.md b/docs/CPU_DATA_ANALYSIS_INTEGRATION.md index cf9aed468..4a55123ab 100644 --- a/docs/CPU_DATA_ANALYSIS_INTEGRATION.md +++ b/docs/CPU_DATA_ANALYSIS_INTEGRATION.md @@ -106,6 +106,8 @@ Only the ring moves. The signal disk $r_1$ stays circular, deliberately: it sets What a circular $r_1$ loses is flux, and that loss is **not** a function of resolution alone: measured per reflection, it carries a directional component worth several Ų with a definite principal axis, on top of the isotropic part. Nor is there anything in the merge to absorb it. There is **no per-shell scale**, and there cannot usefully be one: every scale in §10 is fitted against a reference built from a reflection's own symmetry equivalents, and equivalents share $s^2$ exactly, so any function of $s^2$ lies in the exact null space of the whole scaling model — a per-shell parameter would have zero residual to fit against. (XDS and DIALS have the same null space, for the same reason.) The isotropic part of the loss is instead degenerate with the overall Wilson $B$ and is silently reported as part of it, so **the reported `WILSON_B` / `_reflns.B_iso_Wilson_estimate` carries an $r_1$-dependent contribution**: measured across a constant-ring-area radius sweep it falls monotonically as the disk grows, by 0.5 Ų on sharp strong data and by up to ~10 Ų on weak wide-spot data. What this costs the *data* is much less than what it costs the flux, because most of the loss is matched by a proportional $\sigma$: it moves no CC$_{1/2}$ and no $R_\text{meas}$, and — to within a few hundredths of an ångström — no resolution cut. +**Measured spot footprint (automatic).** The radii above are chosen from spots at 5 Å, which at high X-ray energy sit close to the beam. Away from it a spot can grow several times wider — radially from the sensor's parallax and the obliquity of the incidence, tangentially from the crystal's azimuthal spread, which rotates the diffracted beam about the incident one and smears the spot along its ring. On small-molecule data at 20–25 keV the standard deviation grows from ~1 px near the beam to ~5 px at the detector edge: the $r_1 = 4$ disk holds a quarter of the flux there, the $6\ldots13$ px ring a third of it, and the profile widths learned inside $r_1$ (§9.3) saturate near $r_1^2/4$. So the pre-scan measures every spot it finds with a window that follows the spot — three of its own standard deviations, iterated and re-centred — separately along and across the radius, and tabulates the median widths $\sigma_ ho,\sigma_ au$ against the distance from the beam. Wherever $3\max(\sigma_ ho,\sigma_ au)>r_1$ the integrator then (i) starts the background ring at $3\sigma$ along each axis, (ii) sums the reflection over the $r_1$ disk **and** the $3\sigma$ footprint ellipse, so the summation — the profile fit's seed and its fallback — holds the spot rather than its core, and (iii) builds the per-reflection Gaussian at the measured widths on a grid grown to hold them. Where every spot fits the disk nothing is installed and the integration is unchanged bit for bit, which is the case for compact protein spots; like the measured radius, the footprint applies to the canonical pass and not to the geometry pre-pass, and a canonical pass whose wider rings the neighbours starve falls back to the settings without it. Judged by refining the published structures with SHELXL, it removes the intensity loss that grew with resolution on the small-molecule sets (rugnux/model intensity in the outermost shell 0.81–0.91 → 0.98–1.02). + ### 9.2 Box summation (seed and fallback) Let: diff --git a/tests/BraggIntegrationEngineGPUTest.cpp b/tests/BraggIntegrationEngineGPUTest.cpp index 2f5c45425..55d160c6e 100644 --- a/tests/BraggIntegrationEngineGPUTest.cpp +++ b/tests/BraggIntegrationEngineGPUTest.cpp @@ -146,10 +146,16 @@ double CompareCpuVsGpu(IntegratorMode mode, std::optional bandwidth_fwhm, float stencil_k = 0.0f, float r1 = 0.0f, float r2 = 0.0f, float r3 = 0.0f, OverlapMode overlap = OverlapMode::Off, float companion_dx = 0.0f, - bool clip_spots = false, float odd_partiality = 1.0f) { - const DiffractionExperiment experiment = + bool clip_spots = false, float odd_partiality = 1.0f, + const SpotFootprint *footprint = nullptr) { + DiffractionExperiment experiment = MakeExperiment(mode, bandwidth_fwhm, clip_nsigma, radial, DetJF(2), stencil_k, r1, r2, r3, overlap); + if (footprint) { + BraggIntegrationSettings settings = experiment.GetBraggIntegrationSettings(); + settings.Footprint(*footprint); + experiment.ImportBraggIntegrationSettings(settings); + } const size_t width = experiment.GetXPixelsNum(); const size_t height = experiment.GetYPixelsNum(); const size_t npixel = experiment.GetPixelsNum(); @@ -254,6 +260,28 @@ TEST_CASE("BraggIntegrationEngineGPU_MatchesCPU") { CompareCpuVsGpu(IntegratorMode::ProfileGaussian, 0.04f, 0.0f, false, 120, 4.0f, 6.0f, 8.0f, 12.0f); } SECTION("ProfileEmpirical") { CompareCpuVsGpu(IntegratorMode::ProfileEmpirical, std::nullopt); } + // A measured footprint wider than the r1 disk (SpotFootprint.h): the summation ellipse, the ring + // grown along and across the radius and the footprint-width Gaussian are all reflection-dependent + // geometry the two engines have to classify alike - spaced so the rings stay clear, and crowded + // so the grown neighbour mask and the ring gate come into play. + SpotFootprint fp; + fp.bin_px = 100.0f; + for (int b = 0; b < 8; ++b) { + fp.sigma_rad.push_back(1.5f + 0.2f * b); + fp.sigma_tan.push_back(1.8f + 0.3f * b); + } + SECTION("ProfileGaussian footprint") { + CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 4.0f, false, 120, 0.0f, + 0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, false, 1.0f, &fp); + } + SECTION("ProfileGaussian footprint crowded exclude") { + CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 4.0f, false, 40, 0.0f, + 0.0f, 0.0f, 0.0f, OverlapMode::Exclude, 0.0f, false, 1.0f, &fp); + } + SECTION("ProfileGaussian footprint radial") { + CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 4.0f, true, 120, 0.0f, + 0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, false, 1.0f, &fp); + } // Overlap treatment: companions 4 px apart put each reflection's centre inside its neighbour's // signal disk, so the owner map, the excluded pixels and the profile fraction the two modes act on // all have to come out the same in both engines - the ownership atomic in particular is settled by -- 2.54.0 From c095b19f3afc254cd75c8f6261d64bd842a550c2 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 06:06:23 +0200 Subject: [PATCH 31/97] Battery: in-house small-molecule XDS references at full-saturation OVERLOAD The XDS references of aspirin (20, 25 keV), citric acid, HEPES and YAG were made with OVERLOAD=65534, half of the EIGER2's saturation value (133202). CORRECT ignored 13-313 reflections per set whose peak pixel lay between the two - the strongest, most extinguished reflections - so the references described a data set without them. Re-ran only the CORRECT step on the unchanged INTEGRATE.HKL with every input parameter reconstructed from the old CORRECT.LP and OVERLOAD=133201 (saturation - 1, as the other in-house sets already use); a rerun at 65534 reproduced the old CORRECT.LP. The old outputs are kept in xds_overload_half/ beside each data set. `refs --write` picked up the new values. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- tools/battery/inhouse.json | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tools/battery/inhouse.json b/tools/battery/inhouse.json index 356ad822e..668a7b31a 100644 --- a/tools/battery/inhouse.json +++ b/tools/battery/inhouse.json @@ -38,11 +38,11 @@ {"id": "myob_x06da_sparse", "input": "myob_x06da_sparse/MyoB2-5_7cef9c_master.h5", "wavelength": 0.95373, "ref": {"sgno": 3, "cell": [35.388, 28.752, 64.083, 90.0, 106.352, 90.0], "anomalous": false, "isa": 5.46, "completeness": 73.7, "r_meas": 0.326, "cc_half": 0.972, "multiplicity": 5.21, "dmin_low": 5.92, "r_meas_low": 0.146, "dmin": 2.0, "dmin_rule": "xds_range", "dmin_xds": 2.0, "dmax": 50.0}, "tags": ["h5", "myoglobin", "twin"]}, {"id": "nothing_2", "input": "nothing_2/test-28_3400ac_master.h5", "wavelength": 0.99988, "expect": "no_lattice", "tags": ["h5", "control"]}, {"id": "nothing_1", "input": "nothing_1/test-28_9bf604_master.h5", "wavelength": 0.99988, "expect": "no_lattice", "tags": ["h5", "control"]}, - {"id": "aspirin_x10sa_20keV", "input": "aspirin_x10sa_20keV/aspirin_1_002_master.h5", "wavelength": 0.61993, "cod": "7050897", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.262, 6.543, 11.257, 90.0, 95.916, 90.0], "anomalous": true, "isa": 27.97, "completeness": 84.0, "r_meas": 0.032, "cc_half": 0.999, "multiplicity": 3.14, "dmin_low": 2.01, "r_meas_low": 0.03, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, - {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "wavelength": 0.49594, "cod": "7050897", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.55, "completeness": 86.6, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.03, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, - {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "wavelength": 0.61993, "cod": "5000063", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.55, "completeness": 81.0, "r_meas": 0.04, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.043, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, - {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "wavelength": 0.61993, "cod": "2224210", "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.39, "completeness": 91.6, "r_meas": 0.03, "cc_half": 0.999, "multiplicity": 5.64, "dmin_low": 2.0, "r_meas_low": 0.035, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, - {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "wavelength": 0.61993, "cod": "2003066", "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.2, "completeness": 99.0, "r_meas": 0.425, "cc_half": 0.958, "multiplicity": 27.96, "dmin_low": 1.99, "r_meas_low": 0.447, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"}, + {"id": "aspirin_x10sa_20keV", "input": "aspirin_x10sa_20keV/aspirin_1_002_master.h5", "wavelength": 0.61993, "cod": "7050897", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.262, 6.543, 11.257, 90.0, 95.916, 90.0], "anomalous": true, "isa": 27.4, "completeness": 84.0, "r_meas": 0.033, "cc_half": 0.999, "multiplicity": 3.14, "dmin_low": 2.01, "r_meas_low": 0.033, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "aspirin_x10sa_25keV", "input": "aspirin_x10sa_25keV/aspirin_1_001_master.h5", "wavelength": 0.49594, "cod": "7050897", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.255, 6.541, 11.259, 90.0, 95.916, 90.0], "anomalous": true, "isa": 29.25, "completeness": 86.6, "r_meas": 0.034, "cc_half": 0.999, "multiplicity": 3.15, "dmin_low": 1.63, "r_meas_low": 0.032, "dmin": 0.55, "dmin_rule": "xds_range", "dmin_xds": 0.55, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "aspirin form I, P 21/c; COD 7050897 - C. C. Wilson, New J. Chem. 26 (2002) 1733-1739, doi:10.1039/b203775k. XDS reports only Sohncke groups (P 21)"}, + {"id": "citricacid_x10sa_20keV", "input": "citricacid_x10sa_20keV/citricacid_1_005_master.h5", "wavelength": 0.61993, "cod": "5000063", "tags": ["h5", "monoclinic", "small-molecule"], "ref": {"sgno": 4, "cell": [11.454, 5.579, 12.67, 90.0, 111.587, 90.0], "anomalous": true, "isa": 20.44, "completeness": 82.0, "r_meas": 0.041, "cc_half": 0.997, "multiplicity": 3.1, "dmin_low": 2.02, "r_meas_low": 0.05, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P 1 21/c 1", "sgno": 14}, "ref_override_why": "anhydrous citric acid, P 21/a (P 21/c in the standard setting); COD 5000063 - J. P. Glusker, J. A. Minkin, A. L. Patterson, Acta Cryst. B25 (1969) 1066-1072, doi:10.1107/S0567740869003542. XDS reports only Sohncke groups (P 21)"}, + {"id": "hepes_x10sa_20keV", "input": "hepes_x10sa_20keV/hepes_1_004_master.h5", "wavelength": 0.61993, "cod": "2224210", "tags": ["h5", "orthorhombic", "small-molecule"], "ref": {"sgno": 19, "cell": [8.348, 9.575, 27.068, 90.0, 90.0, 90.0], "anomalous": true, "isa": 28.49, "completeness": 91.8, "r_meas": 0.031, "cc_half": 0.999, "multiplicity": 5.66, "dmin_low": 2.0, "r_meas_low": 0.037, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "P b c a", "sgno": 61}, "ref_override_why": "HEPES, P b c a; COD 2224210 (100 K, 8.341 9.567 27.066) - P. Sledz, T. Minor, M. Chruszcz, Acta Cryst. E65 (2009) o3027-o3028, doi:10.1107/S1600536809042512. XDS reports only Sohncke groups (P 21 21 21)"}, + {"id": "yag_x10sa_20keV", "input": "yag_x10sa_20keV/yag_2_004_master.h5", "wavelength": 0.61993, "cod": "2003066", "tags": ["h5", "cubic", "small-molecule"], "ref": {"sgno": 211, "cell": [11.997, 11.997, 11.997, 90.0, 90.0, 90.0], "anomalous": true, "isa": 3.18, "completeness": 99.0, "r_meas": 0.432, "cc_half": 0.947, "multiplicity": 28.44, "dmin_low": 1.99, "r_meas_low": 0.436, "dmin": 0.68, "dmin_rule": "xds_range", "dmin_xds": 0.68, "dmax": 50.0}, "ref_override": {"sg": "I a -3 d", "sgno": 230}, "ref_override_why": "Y3Al5O12 garnet, I a -3 d; COD 2003066 - A. Nakatsuka, A. Yoshiasa, T. Yamanaka, Acta Cryst. B55 (1999) 266-272, doi:10.1107/S0108768198012567. XDS reports only Sohncke groups (I 4 3 2)"}, {"id": "lcystine_x10sa_20keV", "input": "lcystine_x10sa_20keV/lcystine_2_004_master.h5", "wavelength": 0.61993, "cod": "1513328", "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"}, {"id": "lcystine_x10sa_25keV", "input": "lcystine_x10sa_25keV/lcystine_2_003_master.h5", "wavelength": 0.49594, "cod": "1513328", "tags": ["h5", "hexagonal", "small-molecule"], "ref_override": {"sg": "P 61 2 2", "sgno": 178, "cell": [5.422, 5.422, 56.275, 90.0, 90.0, 120.0]}, "ref_override_why": "hexagonal L-cystine, P 61 2 2; COD 5000005 - B. M. Oughton and P. M. Harrison, Acta Cryst. 12 (1959) 396-404, doi:10.1107/S0365110X59001177. No XDS reference: XDS indexes under 10% of the spots of either sweep, so the row is scored on lattice and symmetry only"} ]} -- 2.54.0 From 4da6159dbb80995777775346fd746b005b2aab20 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 09:36:39 +0200 Subject: [PATCH 32/97] Battery: twelve open sets for point-group and twin decisions Pseudo-merohedral and merohedral twins, pseudo-symmetric cells and their negative controls, chosen to test the space-group search's promotions: - 8c3e (P3121, merohedral twin ~0.23, home-source HyPix CBF) - 4bwl, 2wnq, 2xfw, 2wnn (P21 NAL crystals with the pseudo-merohedral twin law -h,-k,h+l at fractions 0.49/0.46/0.10/0.33; ADSC SMV) and 2wnz, the same crystal form deposited untwinned - 6oww, 6p8j, 9qw2 (P21 with near-90 beta or a~c), 9qvv (I222, b=c), 5ojv (P21212, a~c), 7q6j (P212121, b~c) Data staged in /data/scout_twin, linked from the open data root. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- tools/battery/open.json | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tools/battery/open.json b/tools/battery/open.json index 5327d7a6f..87544a511 100644 --- a/tools/battery/open.json +++ b/tools/battery/open.json @@ -179,5 +179,17 @@ {"id": "9fcf", "input": "9fcf/IBCH-05-p03x03_3_00001.cbf.gz", "wavelength": 0.97626, "ref": {"sg": "P 4", "sgno": 75, "cell": [91.301, 91.301, 35.836, 90.0, 90.0, 90.0], "dmin": 2.36}, "tags": ["cbf"]}, {"id": "9h0q", "input": "9h0q/Bc2lCnter-pma127_2_master.h5", "wavelength": 0.97857, "ref": {"sg": "H 3 2", "sgno": 155, "cell": [169.506, 169.506, 344.036, 90.0, 90.0, 120.0], "dmin": 2.55}, "tags": ["h5"]}, {"id": "5ky6", "input": "5ky6/C11_1_001.img", "wavelength": 0.91841, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [84.511, 57.253, 164.016, 90.0, 102.57, 90.0], "dmin": 1.941}, "tags": ["marccd"]}, - {"id": "6z9g", "input": "6z9g/HASE01-ga1o2_w1_2_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [120.306, 93.815, 126.951, 90.0, 105.201, 90.0], "dmin": 1.76}, "tags": ["h5", "tncs"]} + {"id": "6z9g", "input": "6z9g/HASE01-ga1o2_w1_2_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [120.306, 93.815, 126.951, 90.0, 105.201, 90.0], "dmin": 1.76}, "tags": ["h5", "tncs"]}, + {"id": "8c3e", "input": "8c3e/LCB2_raw_data/lcb2_8_newt_003_0001.cbf", "wavelength": 1.54184, "ref": {"sg": "P 31 2 1", "sgno": 152, "cell": [57.817, 57.817, 41.54, 90.0, 90.0, 120.0], "dmin": 2.1}, "tags": ["cbf", "trigonal", "home-source", "twin"]}, + {"id": "4bwl", "input": "4bwl/y137a_sial5m_MS_1_001.img", "wavelength": 0.9795, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [55.97, 143.22, 83.41, 90, 109.51, 90], "dmin": 2.0}, "tags": ["smv", "monoclinic", "twin", "pseudo-merohedral"]}, + {"id": "2wnq", "input": "2wnq/I2-3_data_MS_1_001.img", "wavelength": 0.9702, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [54.63, 142.77, 84.487, 90, 108.99, 90], "dmin": 1.8}, "tags": ["smv", "monoclinic", "twin", "pseudo-merohedral"]}, + {"id": "2xfw", "input": "2xfw/E192N_alone_MS_1_001.img", "wavelength": 0.9796, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [78.11, 116.53, 83.74, 90, 117.95, 90], "dmin": 1.65}, "tags": ["smv", "monoclinic", "twin", "pseudo-merohedral"]}, + {"id": "2wnn", "input": "2wnn/wt_pyr_pos3_MS_3_001.img", "wavelength": 0.92, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [54.693, 142.456, 83.626, 90, 109.16, 90], "dmin": 1.65}, "tags": ["smv", "monoclinic", "twin", "pseudo-merohedral"]}, + {"id": "2wnz", "input": "2wnz/E192_pyr_MS_1_001.img", "wavelength": 0.9796, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [56.921, 143.041, 83.918, 90, 109.79, 90], "dmin": 1.85}, "tags": ["smv", "monoclinic", "pseudo-symmetry"]}, + {"id": "6oww", "input": "6oww/data/t1_1_00001.cbf", "wavelength": 1.3857, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [85.832, 86.015, 214.34, 90.0, 89.91, 90.0], "dmin": 3.84}, "tags": ["cbf", "monoclinic", "pseudo-symmetry"]}, + {"id": "6p8j", "input": "6p8j/13_12_2_000001.cbf", "wavelength": 0.9792, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [46.484, 152.166, 88.255, 90.0, 90.12, 90.0], "dmin": 1.47}, "tags": ["cbf", "monoclinic", "pseudo-symmetry"]}, + {"id": "9qvv", "input": "9qvv/CHRD_HEP_M/CHRD_404_7_1.nxs", "wavelength": 0.9762, "ref": {"sg": "I 2 2 2", "sgno": 23, "cell": [120.142, 123.8, 123.787, 90.0, 90.0, 90.0], "dmin": 2.72}, "tags": ["nxs", "orthorhombic", "pseudo-symmetry"]}, + {"id": "5ojv", "input": "5ojv/bg3047_w1_1_0001.cbf", "wavelength": 0.97625, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [87.792, 158.242, 87.589, 90.0, 90.0, 90.0], "dmin": 2.062}, "tags": ["cbf", "orthorhombic", "pseudo-symmetry"]}, + {"id": "9qw2", "input": "9qw2/HepPhosHep_Data_for_9QW2/hw28c25_dc1_1_0001.cbf", "wavelength": 0.9762, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [55.439, 108.179, 55.65, 90.0, 90.9, 90.0], "dmin": 1.92}, "tags": ["cbf", "monoclinic", "pseudo-symmetry"]}, + {"id": "7q6j", "input": "7q6j/3_001_00001.cbf", "wavelength": 1.0332, "ref": {"sg": "P 21 21 21", "sgno": 19, "cell": [72.71, 115.88, 116.18, 90.0, 90.0, 90.0], "dmin": 2.2}, "tags": ["cbf", "orthorhombic", "pseudo-symmetry"]} ]} -- 2.54.0 From f0349949d67b0d1870e92a414e2d30a4c5c93bec Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 09:57:45 +0200 Subject: [PATCH 33/97] Battery: ten open-arm sets from under-represented facilities and beamlines SRS Daresbury PX10.1 (6RYM), EMBL Hamburg DORIS X13 (7BGU), NSLS X29A and X25 (8V4J, 8V2T), ESRF ID14-2 (5JK4), Australian Synchrotron MX2 (6CS9), CLSI 08B1-1 (7UDI), SSRF BL17UM (9LXL), MAX IV BioMAX (9S02), ALBA XALOC (6GVK) - marCCD, ADSC (Quantum 4/210r/315), PILATUS and EIGER2 files. Each read and processed with the current rugnux; 7BGU (C2 imposed on a P1 lattice) and 7UDI (P41 under-called to P2) disagree with the deposition. Multi-sweep archives pinned to one sweep. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- docs/EXTERNAL_TEST_DATA.md | 46 ++++++++++++++++++++++++-------------- tools/battery/open.json | 12 +++++++++- 2 files changed, 40 insertions(+), 18 deletions(-) diff --git a/docs/EXTERNAL_TEST_DATA.md b/docs/EXTERNAL_TEST_DATA.md index 6ae4eaed6..7db2cd403 100644 --- a/docs/EXTERNAL_TEST_DATA.md +++ b/docs/EXTERNAL_TEST_DATA.md @@ -44,6 +44,7 @@ the table below; the repositories themselves are cited in | [5EPE](https://www.rcsb.org/structure/5EPE) | IRRMC [10.18430/m3159c](https://doi.org/10.18430/m3159c) | APS 21-ID-G | 1.90 | F 2 3 | 157.5 157.5 157.5 90.0 90.0 90.0 | Rayonix MX-300 | Crystal structure of SAM-dependent methyltransferase from Thiobacillus denitrificans in complex with S-Adenosyl-L-homocysteine | | [5F6M](https://www.rcsb.org/structure/5F6M) | SBGrid [10.15785/sbgrid/201](https://doi.org/10.15785/sbgrid/201) | SSRL BL11-1 | 1.10 | P 21 21 21 | 54.8 58.5 67.4 90.0 90.0 90.0 | PILATUS 6M | Isotropic Trypsin Model for Comparison of Diffuse Scattering | | [5J23](https://www.rcsb.org/structure/5J23) | IRRMC [10.18430/M35J23](https://doi.org/10.18430/M35J23) | APS 21-ID-G | 2.30 | H 3 | 175.8 175.8 136.8 90.0 90.0 120.0 | Rayonix MX-300 | Crystal structure of NADPH-dependent glyoxylate/hydroxypyruvate reductase SMc04462 (SmGhrB) from Sinorhizobium meliloti in complex with 2'-phospho-ADP-ribose | +| [5JK4](https://www.rcsb.org/structure/5JK4) | Zenodo [10.5281/zenodo.49859](https://doi.org/10.5281/zenodo.49859) | ESRF ID14-2 | 1.10 | P 1 21 1 | 37.7 77.9 56.3 90.0 102.1 90.0 | ADSC Quantum 4 | Phosphate-Binding Protein from Stenotrophomonas maltophilia. | | [5JVN](https://www.rcsb.org/structure/5JVN) | IRRMC [10.18430/m35jvn](https://doi.org/10.18430/m35jvn) | ESRF ID29 | 2.90 | P 6 2 2 | 249.4 249.4 84.1 90.0 90.0 120.0 | PILATUS3 6M | C3-type pyruvate phosphate dikinase: intermediate state of the swiveling-domain mechanism | | [5KY6](https://www.rcsb.org/structure/5KY6) | MXRDR [10.18150/repod.1494374](https://doi.org/10.18150/repod.1494374) | BESSY 14.2 | 1.94 | P 1 21 1 | 84.5 57.3 164.0 90.0 102.6 90.0 | marCCD, 225 mm plate | Human muscle fructose-1,6-bisphosphate aldolase | | [5LZL](https://www.rcsb.org/structure/5LZL) | Zenodo [10.5281/zenodo.54757](https://doi.org/10.5281/zenodo.54757) | Diamond I02 | 3.47 | P 31 2 1 | 205.6 205.6 199.2 90.0 90.0 120.0 | PILATUS 6M-F | Pyrobaculum calidifontis 5-aminolaevulinic acid dehydratase | @@ -57,11 +58,13 @@ the table below; the repositories themselves are cited in | [5VML](https://www.rcsb.org/structure/5VML) | IRRMC [10.18430/M35VML](https://doi.org/10.18430/M35VML) | Home source, Rigaku FR-E+ SuperBright | 1.70 | P 42 21 2 | 66.3 66.3 115.3 90.0 90.0 90.0 | Rigaku Saturn 944+ | Crystal Structure of Acetoacetyl-CoA Reductase from Burkholderia Pseudomallei 1710b with bound NADP | | [6CDL](https://www.rcsb.org/structure/6CDL) | IRRMC [10.18430/m36cdl](https://doi.org/10.18430/m36cdl) | APS 22-ID | 1.25 | P 21 21 2 | 58.3 85.9 46.1 90.0 90.0 90.0 | marCCD, 300 mm plate | HIV-1 wild type protease with GRL-03214A, 6-5-5-ring fused umbrella-like tetrahydropyranofuran as the P2-ligand, a cyclopropylaminobenzothiazole as the P2'-ligand and 3,5-difluorophenylmethyl as the P1-ligand | | [6CEE](https://www.rcsb.org/structure/6CEE) | IRRMC [10.18430/M36CEE](https://doi.org/10.18430/M36CEE) | Home source, Rigaku FR-E SuperBright | 1.55 | P 21 21 21 | 40.7 44.1 55.9 90.0 90.0 90.0 | Rigaku Saturn A200 | Crystal structure of fragment 3-(1-Methyl-2-oxo-1,2-dihydroquinoxalin-3-yl)propionic acid bound in the ubiquitin binding pocket of the HDAC6 zinc-finger domain | +| [6CS9](https://www.rcsb.org/structure/6CS9) | SBGrid [10.15785/SBGRID/568](https://doi.org/10.15785/SBGRID/568) | Australian Synchrotron MX2 | 1.85 | P 1 21 1 | 32.9 25.5 40.2 90.0 98.6 90.0 | ADSC Quantum 210r | Crystal structure of human beta-defensin 2 in complex with PIP2 | | [6F3P](https://www.rcsb.org/structure/6F3P) | IRRMC [10.18430/M36F3P](https://doi.org/10.18430/M36F3P) | APS 22-ID | 1.35 | C 1 2 1 | 142.9 85.7 112.0 90.0 122.2 90.0 | marCCD, 300 mm plate | Crystal structure of S-adenosyl-L-homocysteine hydrolase from Pseudomonas aeruginosa in complex with 3'-deoxyadenosine and K+ cation | | [6FID](https://www.rcsb.org/structure/6FID) | SBGrid [10.15785/sbgrid/541](https://doi.org/10.15785/sbgrid/541) | ESRF ID30B | 2.20 | P 21 21 21 | 59.9 64.1 69.7 90.0 90.0 90.0 | PILATUS3 6M | Bovine trypsin solved by S-SAD on ID30B | | [6FVZ](https://www.rcsb.org/structure/6FVZ) | IRRMC [10.18430/m36fvz](https://doi.org/10.18430/m36fvz) | ESRF ID23-2 | 1.80 | C 2 2 2 | 131.2 222.8 86.5 90.0 90.0 90.0 | PILATUS3 X 2M | Crystal structure of human monoamine oxidase B (MAO B) in complex with an inhibitor | | [6FWC](https://www.rcsb.org/structure/6FWC) | IRRMC [10.18430/m36fwc](https://doi.org/10.18430/m36fwc) | ESRF MASSIF-3 | 1.70 | C 2 2 2 | 131.7 222.1 86.3 90.0 90.0 90.0 | PILATUS 2MF | Crystal structure of human monoamine oxidase B (MAO B) in complex with fluorophenyl-chromone-carboxamide | | [6G1F](https://www.rcsb.org/structure/6G1F) | Zenodo [10.5281/zenodo.1059413](https://doi.org/10.5281/zenodo.1059413) | Diamond I03 | 2.25 | C 1 2 1 | 329.3 83.9 133.4 90.0 111.6 90.0 | PILATUS3 6M | Crystal structure of D-phenylglycine aninotransferase (D-PhgAT) from Pseudomonas stutzeri with PLP internal aldimine | +| [6GVK](https://www.rcsb.org/structure/6GVK) | Zenodo [10.5281/zenodo.1286854](https://doi.org/10.5281/zenodo.1286854) | ALBA XALOC | 1.55 | C 1 2 1 | 105.6 59.5 42.4 90.0 113.5 90.0 | PILATUS 6M | Second pair of Fibronectin type III domains of integrin beta4 (T1663R mutant) bound to the bullous pemphigoid antigen BP230 (BPAG1e) | | [6H2P](https://www.rcsb.org/structure/6H2P) | IRRMC [10.18430/m36h2p](https://doi.org/10.18430/m36h2p) | BESSY 14.1 | 1.48 | C 2 2 21 | 103.5 107.1 216.5 90.0 90.0 90.0 | PILATUS 6M | Crystal Structure of Arg184Gln mutant of Human Prolidase with Mn ions and Cacodylate ligand | | [6H5T](https://www.rcsb.org/structure/6H5T) | IRRMC [10.18430/m36h5t](https://doi.org/10.18430/m36h5t) | BESSY 14.3 | 1.69 | I 4 2 2 | 86.8 86.8 141.8 90.0 90.0 90.0 | marCCD, 225 mm plate | Intersectin SH3A short isoform | | [6HV2](https://www.rcsb.org/structure/6HV2) | IRRMC [10.18430/m36hv2](https://doi.org/10.18430/m36hv2) | SLS X06SA | 1.71 | P 61 2 2 | 68.9 68.9 133.6 90.0 90.0 120.0 | Dectris Eiger 16M | MMP-13 in complex with the peptide IMISF | @@ -85,6 +88,7 @@ the table below; the repositories themselves are cited in | [6QAJ](https://www.rcsb.org/structure/6QAJ) | SBGrid [10.15785/sbgrid/637](https://doi.org/10.15785/sbgrid/637) | Diamond I03 | 2.90 | C 2 2 21 | 59.8 169.3 374.5 90.0 90.0 90.0 | PILATUS3 6M | Structure of the tripartite motif of KAP1/TRIM28 | | [6R72](https://www.rcsb.org/structure/6R72) | Zenodo [10.5281/zenodo.14894181](https://doi.org/10.5281/zenodo.14894181) | SOLEIL PROXIMA 2 | 3.95 | P 1 21 1 | 117.8 110.8 155.6 90.0 93.2 90.0 | Dectris Eiger 9M | Crystal structure of BmrA-E504A in an outward-facing conformation | | [6RLR](https://www.rcsb.org/structure/6RLR) | Zenodo [10.5281/zenodo.5886687](https://doi.org/10.5281/zenodo.5886687) | Diamond I04 | 2.00 | P 1 | 40.0 40.0 63.6 80.4 76.3 68.2 | Eiger 16M | Crystal structure of CD9 large extracellular loop | +| [6RYM](https://www.rcsb.org/structure/6RYM) | Keele University [10.21252/xbsq-d621](https://doi.org/10.21252/xbsq-d621) | SRS PX10.1 (Daresbury) | 1.46 | P 43 | 50.2 50.2 51.9 90.0 90.0 90.0 | marCCD 165 mm | Structure of carbohydrate recognition domain with GlcNAc bound | | [6S1U](https://www.rcsb.org/structure/6S1U) | MXRDR [10.18150/repod.0005795](https://doi.org/10.18150/repod.0005795) | BESSY 14.2 | 1.90 | P 1 21 1 | 51.6 29.4 85.5 90.0 103.8 90.0 | marCCD, 225 mm plate | Crystal structure of dimeric M-PMV protease C7A/D26N/C106A mutant in complex with inhibitor | | [6TOC](https://www.rcsb.org/structure/6TOC) | Zenodo [10.5281/zenodo.3571040](https://doi.org/10.5281/zenodo.3571040) | SLS X06DA | 1.85 | P 42 | 31.5 31.5 81.6 90.0 90.0 90.0 | PILATUS 2MF | Crystal structure of the oligomerisation domain of the transcription factor PHOSPHATE STARVATION RESPONSE 1 from Arabidopsis (crystal form 3). | | [6TTN](https://www.rcsb.org/structure/6TTN) | IRRMC [10.18430/m36ttn](https://doi.org/10.18430/m36ttn) | BESSY 14.1 | 1.12 | P 21 21 21 | 39.9 79.8 104.7 90.0 90.0 90.0 | PILATUS 6M | N-terminally truncated hyoscyamine 6-hydroxylase (tH6H) in complex with N-oxalylglycine and hyoscyamine | @@ -105,6 +109,7 @@ the table below; the repositories themselves are cited in | [7ARR](https://www.rcsb.org/structure/7ARR) | MXRDR [10.18150/EM87YL](https://doi.org/10.18150/EM87YL) | PETRA III, EMBL c/o DESY P13 (MX1) | 1.10 | P 1 | 30.9 32.1 43.1 114.2 91.9 109.9 | PILATUS 6M-F | The de novo designed hybrid alpha/beta-miniprotein | | [7ATG](https://www.rcsb.org/structure/7ATG) | IRRMC [10.18430/m37atg](https://doi.org/10.18430/m37atg) | PETRA III, EMBL c/o DESY P13 (MX1) | 0.60 | P 21 21 21 | 18.0 31.0 43.9 90.0 90.0 90.0 | PILATUS 6M-F | Crystal structure of Z-DNA in complex with putrescinium and potassium cations at ultrahigh-resolution | | [7BGT](https://www.rcsb.org/structure/7BGT) | MXRDR [10.18150/1HQGWO](https://doi.org/10.18150/1HQGWO) | BESSY 14.2 | 1.93 | P 1 | 29.3 67.6 69.7 76.8 83.9 83.6 | marCCD, 225 mm plate | Mason-Pfizer Monkey Virus Protease mutant C7A/D26N/C106A in complex with peptidomimetic inhibitor | +| [7BGU](https://www.rcsb.org/structure/7BGU) | MXRDR [10.18150/C9DYSH](https://doi.org/10.18150/C9DYSH) | EMBL/DESY Hamburg (DORIS) X13 | 2.43 | P 1 | 29.1 67.9 69.7 77.1 83.3 83.2 | marCCD 165 mm | Mason-Pfizer Monkey Virus Protease mutant C7A/D26N/C106A in complex with peptidomimetic inhibitor | | [7D1M](https://www.rcsb.org/structure/7D1M) | IRRMC [10.18430/m37brr](https://doi.org/10.18430/m37brr) | SSRF BL17U1 | 1.35 | P 1 21 1 | 55.5 99.0 59.6 90.0 108.5 90.0 | Dectris Eiger 16M | CRYSTAL STRUCTURE OF THE SARS-CoV-2 MAIN PROTEASE COMPLEXED WITH GC376 | | [7DKP](https://www.rcsb.org/structure/7DKP) | IRRMC [10.18430/M37DKP](https://doi.org/10.18430/M37DKP) | ESRF MASSIF-3 | 1.45 | P 1 21 1 | 49.8 169.5 49.8 90.0 93.5 90.0 | Dectris Eiger 4M | Crystal structure of E. coli Grx2 in complex with GSH at 1.45 A resolution | | [7K1L](https://www.rcsb.org/structure/7K1L) | IRRMC [10.18430/m37k1l](https://doi.org/10.18430/m37k1l) | APS 19-ID | 2.25 | P 63 | 150.8 150.8 110.7 90.0 90.0 120.0 | PILATUS3 6M | Crystal Structure of NSP15 Endoribonuclease from SARS CoV-2 in the Complex with Uridine-2',3'-Vanadate | @@ -126,6 +131,7 @@ the table below; the repositories themselves are cited in | [7RJI](https://www.rcsb.org/structure/7RJI) | IRRMC [10.18430/M37RJI](https://doi.org/10.18430/M37RJI) | LNLS W01B-MX2 | 1.71 | H 3 2 | 83.0 83.0 124.8 90.0 90.0 120.0 | PILATUS 2M | BthTX-II variant b, from Bothrops jararacussu venom, complexed with stearic acid | | [7T5T](https://www.rcsb.org/structure/7T5T) | SBGrid [10.15785/sbgrid/864](https://doi.org/10.15785/sbgrid/864) | SSRL BL9-2 | 1.35 | P 42 21 2 | 95.3 95.3 104.9 90.0 90.0 90.0 | PILATUS 6M | Structure of Thauera sp. K11 CapP | | [7TCD](https://www.rcsb.org/structure/7TCD) | IRRMC [10.18430/m37tcd](https://doi.org/10.18430/m37tcd) | SLS X06SA | 1.70 | C 1 2 1 | 138.5 47.9 78.1 90.0 107.6 90.0 | Dectris Eiger 16M | LOV2-DARPIN fusion: D13 | +| [7UDI](https://www.rcsb.org/structure/7UDI) | Zenodo [10.5281/zenodo.10022358](https://doi.org/10.5281/zenodo.10022358) | CLSI 08B1-1 | 2.24 | P 41 | 66.7 66.7 129.6 90.0 90.0 90.0 | PILATUS3 S 6M | Full-length dimer of DNA-Damage Response Protein C from Deinococcus radiodurans | | [7YZX](https://www.rcsb.org/structure/7YZX) | IRRMC [10.18430/M37YZX](https://doi.org/10.18430/M37YZX) | Diamond I24 | 1.90 | P 63 2 2 | 169.4 169.4 141.8 90.0 90.0 120.0 | PILATUS3 6M | ScpA from Streptococcus pyogenes, D783A mutant. | | [8A1A](https://www.rcsb.org/structure/8A1A) | IRRMC [10.18430/M38A1A](https://doi.org/10.18430/M38A1A) | SLS X06SA | 2.05 | P 65 | 191.9 191.9 122.4 90.0 90.0 120.0 | Dectris Eiger 16M | Structure of a leucinostatin derivative determined by host lattice display : L1F11V1 construct | | [8AGQ](https://www.rcsb.org/structure/8AGQ) | IRRMC [10.18430/M38AGQ](https://doi.org/10.18430/M38AGQ) | SLS X06DA | 1.09 | C 1 2 1 | 89.9 55.4 54.8 90.0 113.5 90.0 | PILATUS 2MF | Crystal structure of anthocyanin-related GSTF8 from Populus trichocarpa in complex with (-)-catechin and glutathione | @@ -152,6 +158,8 @@ the table below; the repositories themselves are cited in | [8THA](https://www.rcsb.org/structure/8THA) | IRRMC [10.18430/m38tha](https://doi.org/10.18430/m38tha) | SSRL BL9-2 | 1.68 | P 64 | 69.2 69.2 29.1 90.0 90.0 120.0 | PILATUS 6M | 1TEL, non-compressed, double-helical crystal form | | [8TYY](https://www.rcsb.org/structure/8TYY) | SBGrid [10.15785/sbgrid/1040](https://doi.org/10.15785/sbgrid/1040) | APS 24-ID-E | 1.68 | F 4 3 2 | 214.9 214.9 214.9 90.0 90.0 90.0 | Dectris Eiger 16M | Structure of a bacterial Ubl-deubiquitinase complex (form 2) | | [8U0I](https://www.rcsb.org/structure/8U0I) | IRRMC [10.18430/m38u0i](https://doi.org/10.18430/m38u0i) | ALS 8.2.1 | 1.54 | P 43 21 2 | 50.3 50.3 90.6 90.0 90.0 90.0 | Dectris EIGER2 Si 9M | Crystal structure of PA0012 complexed with cyclic-di-GMP from Pseudomonas aeruginosa | +| [8V2T](https://www.rcsb.org/structure/8V2T) | Zenodo [10.5281/zenodo.10201899](https://doi.org/10.5281/zenodo.10201899) | NSLS X25 | 1.40 | P 42 21 2 | 60.9 60.9 92.7 90.0 90.0 90.0 | PILATUS 6M | Phosphoheptose isomerase GMHA from Burkholderia pseudomallei bound to inhibitor Mut148591 | +| [8V4J](https://www.rcsb.org/structure/8V4J) | Zenodo [10.5281/zenodo.10222807](https://doi.org/10.5281/zenodo.10222807) | NSLS X29A | 1.31 | P 42 21 2 | 61.0 61.0 92.4 90.0 90.0 90.0 | ADSC Quantum 315 | Phosphoheptose isomerase GMHA from Burkholderia pseudomallei bound to inhibitor Mut148233 | | [8V4O](https://www.rcsb.org/structure/8V4O) | IRRMC [10.18430/m38v4o](https://doi.org/10.18430/m38v4o) | NSLS-II 19-ID | 2.70 | P 61 2 2 | 139.5 139.5 545.0 90.0 90.0 120.0 | Dectris EIGER2 Si 9M | Crystal structure of Acetyl-CoA synthetase 2 in complex with AMP from Candida albicans | | [8XBP](https://www.rcsb.org/structure/8XBP) | IRRMC [10.18430/M38XBP](https://doi.org/10.18430/M38XBP) | SOLEIL PROXIMA 1 | 1.99 | C 1 2 1 | 148.3 50.8 60.2 90.0 92.3 90.0 | Dectris Eiger 16M | Crystal structure of AtNATA1 bound to Acetyl CoA | | [8XTE](https://www.rcsb.org/structure/8XTE) | SBGrid [10.15785/sbgrid/1101](https://doi.org/10.15785/sbgrid/1101) | SSRF BL19U1 | 1.99 | P 32 | 208.8 208.8 67.2 90.0 90.0 120.0 | PILATUS3 6M | Crystal structure of methyltransferase MpaG' in complex with SAH and FDHMP | @@ -182,6 +190,7 @@ the table below; the repositories themselves are cited in | [9JQ9](https://www.rcsb.org/structure/9JQ9) | IRRMC [10.18430/M39JQ9](https://doi.org/10.18430/M39JQ9) | Home source, Excillum MetalJet D2+ | 1.90 | P 21 21 21 | 48.6 50.5 78.6 90.0 90.0 90.0 | PILATUS3 1M | Crystal structure of Plasmoredoxin from Plasmodium falciparum a disulfide oxidoreductase protein unique to Plasmodium species | | [9JZO](https://www.rcsb.org/structure/9JZO) | IRRMC [10.18430/m39jzo](https://doi.org/10.18430/m39jzo) | PAL/PLS 11C | 1.40 | P 1 | 41.6 43.1 54.2 113.0 90.1 118.2 | PILATUS3 6M | Crystal structure of PHICD111_20024_EAD. | | [9KHR](https://www.rcsb.org/structure/9KHR) | Zenodo [10.5281/zenodo.14070468](https://doi.org/10.5281/zenodo.14070468) | RRCAT INDUS-2 PX-BL21 | 2.00 | P 21 21 21 | 48.7 50.3 78.0 90.0 90.0 90.0 | marCCD, 225 mm plate | Crystal structure of Plasmoredoxin, a disulfide oxidoreductase from Plasmodium falciparum crystallized in the presence of Dithiothreitol (DTT) | +| [9LXL](https://www.rcsb.org/structure/9LXL) | Zenodo [10.5281/zenodo.15005358](https://doi.org/10.5281/zenodo.15005358) | SSRF BL17UM | 2.19 | P 41 21 2 | 76.8 76.8 225.3 90.0 90.0 90.0 | EIGER2 S 16M | Crystal structure of GH29 family alpha-L-fucosidase from Fusarium proliferatum LE1 | | [9MH4](https://www.rcsb.org/structure/9MH4) | IRRMC [10.18430/M39MH4](https://doi.org/10.18430/M39MH4) | NSLS-II 19-ID | 3.05 | P 21 3 | 138.7 138.7 138.7 90.0 90.0 90.0 | Dectris EIGER2 Si 9M | Crystal Structure of Bifunctional protein GlmU from Klebsiella aerogenes | | [9MIN](https://www.rcsb.org/structure/9MIN) | SBGrid [10.15785/sbgrid/1151](https://doi.org/10.15785/sbgrid/1151) | ALS 8.2.1 | 2.05 | P 21 21 21 | 95.5 98.5 155.7 90.0 90.0 90.0 | Dectris EIGER2 Si 9M | Structure of a designed minibinder to NYESO1-A*02:01 | | [9O0H](https://www.rcsb.org/structure/9O0H) | IRRMC [10.18430/M39O0H](https://doi.org/10.18430/M39O0H) | SSRL BL12-2 | 2.24 | P 21 21 21 | 55.2 65.5 112.9 90.0 90.0 90.0 | Dectris EIGER2 Si 16M | The ubiquitin-associated domain of human thirty-eight negative kinase 1, fused to the 3TEL crystallization chaperone via a 2-glycine linker | @@ -193,6 +202,7 @@ the table below; the repositories themselves are cited in | [9RCI](https://www.rcsb.org/structure/9RCI) | Zenodo [10.5281/zenodo.15615368](https://doi.org/10.5281/zenodo.15615368) | SOLEIL PROXIMA 2 | 1.66 | P 1 | 35.9 39.3 100.9 98.3 90.3 90.1 | Dectris Eiger 9M | Crystal Structure of Flap Endonuclease FEN1 with Compound 28 | | [9RCS](https://www.rcsb.org/structure/9RCS) | XRDa [10.51093/xrd-00383](https://doi.org/10.51093/xrd-00383) | Diamond I24 | 3.01 | P 1 21 1 | 70.0 78.8 82.3 90.0 88.6 90.0 | Eiger 9M | Cardioderma bat coronavirus KY43 receptor binding domain in complex with human CEACAM6 | | [9RP9](https://www.rcsb.org/structure/9RP9) | IRRMC [10.18430/M39RP9](https://doi.org/10.18430/M39RP9) | SOLEIL PROXIMA 1 | 2.10 | C 1 2 1 | 73.5 59.8 91.7 90.0 100.8 90.0 | Dectris Eiger 16M | Crystal structure of mouse pVHL-ElonginB-ElonginC complex | +| [9S02](https://www.rcsb.org/structure/9S02) | MXRDR [10.60884/NRNGS4](https://doi.org/10.60884/NRNGS4) | MAX IV BioMAX | 1.65 | P 21 21 2 | 163.7 88.0 116.7 90.0 90.0 90.0 | EIGER2 X 16M | PYCR1 in complex with 3-(2-thiazolyl)propionic acid | | [9SL0](https://www.rcsb.org/structure/9SL0) | IRRMC [10.18430/M39SL0](https://doi.org/10.18430/M39SL0) | ESRF MASSIF-1 | 1.60 | P 21 21 21 | 60.2 80.2 111.6 90.0 90.0 90.0 | Dectris EIGER2 Si 9M | Crystal structure of HLA-A0201 in complex with peptide LLWNGPMAV | | [9T6S](https://www.rcsb.org/structure/9T6S) | SBGrid [10.15785/sbgrid/1260](https://doi.org/10.15785/sbgrid/1260) | ESRF ID30B | 2.00 | P 21 21 21 | 63.0 64.6 102.7 90.0 90.0 90.0 | Dectris EIGER2 Si 9M | Crystal Structure of the Listeria monocytogenes CadC with Cadmium | | [9UPT](https://www.rcsb.org/structure/9UPT) | XRDa [10.51093/xrd-00191](https://doi.org/10.51093/xrd-00191) | NSRRC TPS 05A | 2.37 | P 6 | 158.3 158.3 54.0 90.0 90.0 120.0 | SMV, S/N 930 | Structure of AtBgl1A, a GH1 beta-Glucosidase from Acetivibrio thermocellus | @@ -206,13 +216,6 @@ the table below; the repositories themselves are cited in | [9ZLO](https://www.rcsb.org/structure/9ZLO) | Zenodo [10.5281/zenodo.18652652](https://doi.org/10.5281/zenodo.18652652) | Australian Synchrotron MX2 | 2.00 | P 21 21 21 | 38.4 90.0 107.0 90.0 90.0 90.0 | Dectris EIGER1 Si 16M | Crystal structure of Proteus mirabilis UreE | | [9ZM0](https://www.rcsb.org/structure/9ZM0) | IRRMC [10.18430/M39ZM0](https://doi.org/10.18430/M39ZM0) | NSLS-II 17-ID-1 | 2.10 | P 1 21 1 | 50.4 30.1 91.2 90.0 97.1 90.0 | Dectris EIGER1 Si 9M | Crystal structure of monomeric Atg23 | | [9ZMU](https://www.rcsb.org/structure/9ZMU) | IRRMC [10.18430/M39ZMU](https://doi.org/10.18430/M39ZMU) | NSLS-II 19-ID | 1.98 | P 65 2 2 | 47.8 47.8 492.6 90.0 90.0 120.0 | Dectris EIGER2 Si 9M | Crystal structure of an Iole protein from Brucella melitensis (hexagonal P form) | -| — | Zenodo [10.5281/zenodo.1036416](https://doi.org/10.5281/zenodo.1036416) | Diamond Light Source I19-1 | | | | PILATUS 2M | 0.48 Angstrom 3,5-dinitrobenzoic acid (3,5-DNBA) C2/c polymorph single crystal X-ray diffraction data set recorded at Diamond Light Source I19-1 | -| — | Zenodo [10.5281/zenodo.14894181](https://doi.org/10.5281/zenodo.14894181) | | | | | Dectris Eiger 9M | Dataset for PDB 6r72 Crystal structure of BmrA-E504A in an outward-facing conformation | -| — | Zenodo [10.5281/zenodo.20041091](https://doi.org/10.5281/zenodo.20041091) | Diamond Light Source I19-2 | | | | Eiger 2X 4M (CdTe) | Single-crystal X-ray diffractometry data for a sample of Ni(dppe)Cl₂ collected on beamline I19-2 at Diamond Light Source with an Eiger 2X 4M with CdTe sensor | -| — | Zenodo [10.5281/zenodo.20135265](https://doi.org/10.5281/zenodo.20135265) | Diamond Light Source I19-2 | | | | Eiger 2X 4M (CdTe) | Single-crystal X-ray diffractometry data for a sample of metformin collected on beamline I19-2 at Diamond Light Source with an Eiger 2X 4M with CdTe sensor | -| — | Zenodo [10.5281/zenodo.6347466](https://doi.org/10.5281/zenodo.6347466) | Diamond Light Source I19-2 | | | | Eiger 2X 4M (CdTe) | Single-crystal X-ray diffractometry data for a sample of [Cu(HF₂)(pyrazine)₂]PF₆ collected on beamline I19-2 at Diamond Light Source | -| — | Zenodo [10.5281/zenodo.33555](https://doi.org/10.5281/zenodo.33555) | Diamond Light Source I19-1 | | | | PILATUS 2M | Example Cytidine data set from I19-1 at Diamond Light Source | -| — | Zenodo [10.5281/zenodo.11946282](https://doi.org/10.5281/zenodo.11946282) | Diamond Light Source I19 | | | | PILATUS 2M | RODIN X-ray Diffraction Data 2360282 (L-alanine) | Seven rows have no PDB code. Six are small-molecule / chemical-crystallography datasets, kept because they exercise short wavelengths, CdTe sensors, fine slicing and non-zero detector @@ -259,6 +262,15 @@ the `phi` value in the image header. | 8DZ7 | 2 | 4 | 200 frames per crystal | | 6O2H | 4 | 1, 3, 2, 5 - 11 in all | 50, 145, 95, 235 frames, one per crystal | +**Four archives added for facility coverage hold more than one sweep.** One sweep is kept, and +the row is pinned to it. 7BGU's MXRDR record is one directory of 900 marCCD frames that are two +sweeps with different oscillation widths: frames 1001-1674 (0.4°) are the row, frames 1675-1900 +were moved to `sweep2/` so the reader sees one sweep. 5JK4's archive holds a high-resolution +sweep of 185 frames (80 mm, 1°) and a low-resolution one of 93 frames (250 mm, 2°); the row is +the high-resolution sweep. 6RYM's zip holds two sweeps (`jmp47a2_1`, 70 frames; `jmp47a2_2`, 60 +frames); the row is the first. 6GVK's Zenodo record has three tarballs (`set1`-`set3`); only +`set1` (1800 miniCBF frames) was downloaded and is the row. + **Seven IRRMC archives hold more than one collection.** In six of them one sweep is kept and the rest were deleted, so a run over the data directory sees a single collection per dataset. 7RIS is the exception: its two sweeps are at different wavelengths and both are kept. @@ -337,7 +349,7 @@ plate`), the comment's name where one is present (`Rayonix MX-300`), or the seri ## Deposited models and structure factors -174 of the 181 datasets have a released PDB entry, and RCSB +184 of the 191 datasets have a released PDB entry, and RCSB reports released structure factors (`status_code_sf = REL`) for every one of them. A merged result from this pipeline can therefore be checked against the deposited model or against the deposited intensities. @@ -511,18 +523,18 @@ symmetries rather than to be easy to process. The counts below describe where it like everything else on this page, they are metadata about the depositions and their files, not measurements. -- **Repository:** IRRMC 94, SBGrid 35, Zenodo 28, MXRDR 14, ESRF 3, Keele University 3, XRDa 3, +- **Repository:** IRRMC 94, SBGrid 36, Zenodo 34, MXRDR 16, Keele University 4, ESRF 3, XRDa 3, UQ eSpace 1. - **Facility** - counted from the facility part of the Facility / beamline column, the beamline - ignored so that entries deposited with and without one count the same, over the 170 rows that - name one: APS 26, Diamond 20, ESRF 16, NSLS-II 14, BESSY 12, PETRA III 12, SSRL 11, ALS 8, - SLS 7, SOLEIL 7, SPring-8 7, SSRF 6, PAL/PLS 5, CHESS 4, CLSI 3, ALBA 2, Australian - Synchrotron 2, ELETTRA 2, LNLS 2, and one each from MAX IV, NSRRC, Photon Factory and RRCAT - Indus-2 - 23 facilities. The other ten rows were collected on laboratory sources: nine on + ignored so that entries deposited with and without one count the same, over the 180 rows that + name one: APS 26, Diamond 20, ESRF 17, NSLS-II 14, BESSY 12, PETRA III 12, SSRL 11, ALS 8, + SLS 7, SOLEIL 7, SPring-8 7, SSRF 7, PAL/PLS 5, CHESS 4, CLSI 4, ALBA 3, Australian + Synchrotron 3, ELETTRA 2, LNLS 2, MAX IV 2, NSLS 2, and one each from NSRRC, Photon Factory, + RRCAT Indus-2, SRS Daresbury and EMBL/DESY Hamburg (DORIS) - 26 facilities. The other ten rows were collected on laboratory sources: nine on rotating anodes and one on a liquid-metal jet. -- **Crystal system, from the deposited space group of the 174 PDB-coded rows:** orthorhombic 45, - monoclinic 41, tetragonal 25, trigonal 21, hexagonal 17, cubic 13, triclinic 12. -- **Pink beam:** none of these datasets was collected with pink beam. All 174 PDB-coded rows +- **Crystal system, from the deposited space group of the 184 PDB-coded rows:** orthorhombic 46, + monoclinic 44, tetragonal 30, trigonal 21, hexagonal 17, cubic 13, triclinic 13. +- **Pink beam:** none of these datasets was collected with pink beam. All 184 PDB-coded rows are deposited as `SINGLE WAVELENGTH` (`_diffrn_radiation.pdbx_diffrn_protocol`), and all but 5REO, which leaves the field blank, as monochromatic (`pdbx_monochromatic_or_laue_m_l` `M`); 9Q41 is the one row recorded with a multilayer rather than a crystal monochromator diff --git a/tools/battery/open.json b/tools/battery/open.json index 5327d7a6f..c4cb6a421 100644 --- a/tools/battery/open.json +++ b/tools/battery/open.json @@ -179,5 +179,15 @@ {"id": "9fcf", "input": "9fcf/IBCH-05-p03x03_3_00001.cbf.gz", "wavelength": 0.97626, "ref": {"sg": "P 4", "sgno": 75, "cell": [91.301, 91.301, 35.836, 90.0, 90.0, 90.0], "dmin": 2.36}, "tags": ["cbf"]}, {"id": "9h0q", "input": "9h0q/Bc2lCnter-pma127_2_master.h5", "wavelength": 0.97857, "ref": {"sg": "H 3 2", "sgno": 155, "cell": [169.506, 169.506, 344.036, 90.0, 90.0, 120.0], "dmin": 2.55}, "tags": ["h5"]}, {"id": "5ky6", "input": "5ky6/C11_1_001.img", "wavelength": 0.91841, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [84.511, 57.253, 164.016, 90.0, 102.57, 90.0], "dmin": 1.941}, "tags": ["marccd"]}, - {"id": "6z9g", "input": "6z9g/HASE01-ga1o2_w1_2_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [120.306, 93.815, 126.951, 90.0, 105.201, 90.0], "dmin": 1.76}, "tags": ["h5", "tncs"]} + {"id": "6z9g", "input": "6z9g/HASE01-ga1o2_w1_2_1_master.h5", "wavelength": 0.9677, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [120.306, 93.815, 126.951, 90.0, 105.201, 90.0], "dmin": 1.76}, "tags": ["h5", "tncs"]}, + {"id": "6rym", "input": "6rym/6RYM/jmp47a2_1_001.img", "wavelength": 1.488, "ref": {"sg": "P 43", "sgno": 78, "cell": [50.225, 50.225, 51.925, 90.0, 90.0, 90.0], "dmin": 1.46}, "tags": ["smv", "tetragonal"], "pinned": true}, + {"id": "7bgu", "input": "7bgu/mpmv_01001.mccd", "wavelength": 0.815, "ref": {"sg": "P 1", "sgno": 1, "cell": [29.068, 67.893, 69.739, 77.07, 83.34, 83.18], "dmin": 2.433}, "tags": ["marCCD", "triclinic"], "pinned": true}, + {"id": "8v4j", "input": "8v4j/data/CB1_3_3_001.img", "wavelength": 1.075, "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [60.972, 60.972, 92.429, 90.0, 90.0, 90.0], "dmin": 1.31}, "tags": ["smv", "tetragonal"]}, + {"id": "8v2t", "input": "8v2t/data/MJ504_2_3_00001.cbf", "wavelength": 1.1, "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [60.906, 60.906, 92.728, 90.0, 90.0, 90.0], "dmin": 1.402}, "tags": ["cbf", "tetragonal"]}, + {"id": "5jk4", "input": "5jk4/dioxyhipeg/dioxyhipeg_1_001.img", "wavelength": 0.933, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [37.729, 77.856, 56.287, 90.0, 102.08, 90.0], "dmin": 1.1}, "tags": ["smv", "monoclinic"], "pinned": true}, + {"id": "6cs9", "input": "6cs9/568/HBD2D7_2_1_001.img", "wavelength": 0.9537, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [32.871, 25.538, 40.17, 90.0, 98.64, 90.0], "dmin": 1.85}, "tags": ["smv", "monoclinic"]}, + {"id": "7udi", "input": "7udi/data/MJ7121_0001.cbf", "wavelength": 0.9795, "ref": {"sg": "P 41", "sgno": 76, "cell": [66.698, 66.698, 129.581, 90.0, 90.0, 90.0], "dmin": 2.24}, "tags": ["cbf", "tetragonal"]}, + {"id": "9lxl", "input": "9lxl/6/1_T10Y-FFp-PNPI_6_master.h5", "wavelength": 0.97919, "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [76.757, 76.757, 225.338, 90.0, 90.0, 90.0], "dmin": 2.191}, "tags": ["h5", "tetragonal"]}, + {"id": "9s02", "input": "9s02/PYCR1-D11_1_master.h5", "wavelength": 0.97625, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [163.674, 88.042, 116.731, 90.0, 90.0, 90.0], "dmin": 1.65}, "tags": ["h5", "orthorhombic"]}, + {"id": "6gvk", "input": "6gvk/212_3_0001.cbf", "wavelength": 0.97915, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [105.58, 59.52, 42.4, 90.0, 113.5, 90.0], "dmin": 1.55}, "tags": ["cbf", "monoclinic"], "pinned": true} ]} -- 2.54.0 From 9c0a8c740cb543a4778547e9bef9441e45503dfe Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 11:09:25 +0200 Subject: [PATCH 34/97] Battery: drop 7UDI from the open arm Its deposited sweep lacks 40 deg (frames 561-720 are not in the archive; the depositors processed it as two sweeps), so it tests gap handling rather than processing. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- docs/EXTERNAL_TEST_DATA.md | 1 - tools/battery/open.json | 1 - 2 files changed, 2 deletions(-) diff --git a/docs/EXTERNAL_TEST_DATA.md b/docs/EXTERNAL_TEST_DATA.md index 7db2cd403..210890797 100644 --- a/docs/EXTERNAL_TEST_DATA.md +++ b/docs/EXTERNAL_TEST_DATA.md @@ -131,7 +131,6 @@ the table below; the repositories themselves are cited in | [7RJI](https://www.rcsb.org/structure/7RJI) | IRRMC [10.18430/M37RJI](https://doi.org/10.18430/M37RJI) | LNLS W01B-MX2 | 1.71 | H 3 2 | 83.0 83.0 124.8 90.0 90.0 120.0 | PILATUS 2M | BthTX-II variant b, from Bothrops jararacussu venom, complexed with stearic acid | | [7T5T](https://www.rcsb.org/structure/7T5T) | SBGrid [10.15785/sbgrid/864](https://doi.org/10.15785/sbgrid/864) | SSRL BL9-2 | 1.35 | P 42 21 2 | 95.3 95.3 104.9 90.0 90.0 90.0 | PILATUS 6M | Structure of Thauera sp. K11 CapP | | [7TCD](https://www.rcsb.org/structure/7TCD) | IRRMC [10.18430/m37tcd](https://doi.org/10.18430/m37tcd) | SLS X06SA | 1.70 | C 1 2 1 | 138.5 47.9 78.1 90.0 107.6 90.0 | Dectris Eiger 16M | LOV2-DARPIN fusion: D13 | -| [7UDI](https://www.rcsb.org/structure/7UDI) | Zenodo [10.5281/zenodo.10022358](https://doi.org/10.5281/zenodo.10022358) | CLSI 08B1-1 | 2.24 | P 41 | 66.7 66.7 129.6 90.0 90.0 90.0 | PILATUS3 S 6M | Full-length dimer of DNA-Damage Response Protein C from Deinococcus radiodurans | | [7YZX](https://www.rcsb.org/structure/7YZX) | IRRMC [10.18430/M37YZX](https://doi.org/10.18430/M37YZX) | Diamond I24 | 1.90 | P 63 2 2 | 169.4 169.4 141.8 90.0 90.0 120.0 | PILATUS3 6M | ScpA from Streptococcus pyogenes, D783A mutant. | | [8A1A](https://www.rcsb.org/structure/8A1A) | IRRMC [10.18430/M38A1A](https://doi.org/10.18430/M38A1A) | SLS X06SA | 2.05 | P 65 | 191.9 191.9 122.4 90.0 90.0 120.0 | Dectris Eiger 16M | Structure of a leucinostatin derivative determined by host lattice display : L1F11V1 construct | | [8AGQ](https://www.rcsb.org/structure/8AGQ) | IRRMC [10.18430/M38AGQ](https://doi.org/10.18430/M38AGQ) | SLS X06DA | 1.09 | C 1 2 1 | 89.9 55.4 54.8 90.0 113.5 90.0 | PILATUS 2MF | Crystal structure of anthocyanin-related GSTF8 from Populus trichocarpa in complex with (-)-catechin and glutathione | diff --git a/tools/battery/open.json b/tools/battery/open.json index 5da289926..1f10a48f8 100644 --- a/tools/battery/open.json +++ b/tools/battery/open.json @@ -198,7 +198,6 @@ {"id": "8v2t", "input": "8v2t/data/MJ504_2_3_00001.cbf", "wavelength": 1.1, "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [60.906, 60.906, 92.728, 90.0, 90.0, 90.0], "dmin": 1.402}, "tags": ["cbf", "tetragonal"]}, {"id": "5jk4", "input": "5jk4/dioxyhipeg/dioxyhipeg_1_001.img", "wavelength": 0.933, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [37.729, 77.856, 56.287, 90.0, 102.08, 90.0], "dmin": 1.1}, "tags": ["smv", "monoclinic"], "pinned": true}, {"id": "6cs9", "input": "6cs9/568/HBD2D7_2_1_001.img", "wavelength": 0.9537, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [32.871, 25.538, 40.17, 90.0, 98.64, 90.0], "dmin": 1.85}, "tags": ["smv", "monoclinic"]}, - {"id": "7udi", "input": "7udi/data/MJ7121_0001.cbf", "wavelength": 0.9795, "ref": {"sg": "P 41", "sgno": 76, "cell": [66.698, 66.698, 129.581, 90.0, 90.0, 90.0], "dmin": 2.24}, "tags": ["cbf", "tetragonal"]}, {"id": "9lxl", "input": "9lxl/6/1_T10Y-FFp-PNPI_6_master.h5", "wavelength": 0.97919, "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [76.757, 76.757, 225.338, 90.0, 90.0, 90.0], "dmin": 2.191}, "tags": ["h5", "tetragonal"]}, {"id": "9s02", "input": "9s02/PYCR1-D11_1_master.h5", "wavelength": 0.97625, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [163.674, 88.042, 116.731, 90.0, 90.0, 90.0], "dmin": 1.65}, "tags": ["h5", "orthorhombic"]}, {"id": "6gvk", "input": "6gvk/212_3_0001.cbf", "wavelength": 0.97915, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [105.58, 59.52, 42.4, 90.0, 113.5, 90.0], "dmin": 1.55}, "tags": ["cbf", "monoclinic"], "pinned": true} -- 2.54.0 From 15fb2543ca99e1efdba2f2a70ba6badffdee82c5 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 11:26:58 +0200 Subject: [PATCH 35/97] Revert "RotationScaleMerge: refit the error model about the merge's own mean" This reverts commit 1baf92606. The refit fixed one small-molecule set but cost a split crystal its screw axis and moved ISa down across the battery; the rc173 error model stays. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- .../scale_merge/RotationScaleMerge.cpp | 111 +----------------- .../scale_merge/RotationScaleMergeGPU.cu | 8 -- .../scale_merge/RotationScaleMergeGPU.h | 3 - 3 files changed, 4 insertions(+), 118 deletions(-) diff --git a/image_analysis/scale_merge/RotationScaleMerge.cpp b/image_analysis/scale_merge/RotationScaleMerge.cpp index 7885d767c..b1371ee4e 100644 --- a/image_analysis/scale_merge/RotationScaleMerge.cpp +++ b/image_analysis/scale_merge/RotationScaleMerge.cpp @@ -4478,104 +4478,6 @@ RotationScaleMerge::Result RotationScaleMerge::MergeAndStats(int n_groups, bool resid * resid / factor, o.d}); } } - fit_error_model(samples); - - // ---- The error model refitted about the mean the merge takes. ---- - // The fit above is made about a mean that weighs each full by its COUNTING variance. Where the - // equivalents of a reflection agree to within counting statistics that is the merge's own centre, - // and the model it gives is the model the merge applies. Where they do not - a systematic error - // the counting variance does not know about - it is the wrong centre, and in the worst way: the - // full with the fewest counts has the smallest variance, so the mean, and every deviation the - // model is fitted on, is pulled toward the observations that read lowest. Measured on a strongly - // absorbing crystal whose equivalents spread over a factor 100: the mean of (0 4 0) sat at 21k - // among observations from 11k to 1.9M, the deviations of the rest read as hundreds of counting - // sigmas, a ran into its bound and the six-sigma test about the median taken with the same - // weights deleted 63% of the observations - the strong ones (SHELXL R1 0.50 on that merge). - // So refit about the mean the MERGE takes - each full weighted by the variance the fitted model - // gives it, evaluated at the reflection's mean - until the model stops moving; the outlier - // test's median below takes the same weights. Where counting statistics were right the first fit - // is the answer and nothing moves; where b dominates, the weights even out, as the model says. - // Following Blessing (1997) J. Appl. Cryst. 30, 421-426: the centre an outlier is judged from - // is weighted as the merge weighs. - if (error_model_active) { - // Fulls in group order, so each pass over a group's members is one contiguous walk and the - // groups split across threads without every thread reading every full. - std::vector gstart(n_groups + 1, 0); - for (int i = 0; i < n_full; ++i) - if (mf.group[i] >= 0 && cnt[mf.group[i]] >= 2) ++gstart[mf.group[i] + 1]; - for (int g = 0; g < n_groups; ++g) gstart[g + 1] += gstart[g]; - std::vector member(gstart[n_groups]); - { - std::vector fill(gstart.begin(), gstart.end() - 1); - for (int i = 0; i < n_full; ++i) - if (mf.group[i] >= 0 && cnt[mf.group[i]] >= 2) member[fill[mf.group[i]]++] = i; - } - std::vector next_mean; - std::vector next; - constexpr int REFIT_ROUNDS = 10; - for (int round = 0; round < REFIT_ROUNDS; ++round) { - const double a = error_model_a, b = error_model_b; - const auto model_var = [&](int i, double mean) { - const double sc = static_cast(mf.sigma[i]) * mf.corr[i]; - return a * counting_variance(fulls[i], mean, sc * sc) + (b * mean) * (b * mean); - }; - std::vector> part(ThreadsForWork(member.size(), nthreads)); - const int nt = static_cast(part.size()); - next_mean.assign(n_groups, NAN); - // Each group's mean and samples come from its own members in index order, so the result - // does not depend on how the groups were split. - ParallelFor(nt, nt, [&](int t) { - const int g0 = static_cast(static_cast(n_groups) * t / nt); - const int g1 = static_cast(static_cast(n_groups) * (t + 1) / nt); - for (int g = g0; g < g1; ++g) { - if (gstart[g + 1] - gstart[g] < 2 || !std::isfinite(em_mean[g])) continue; - double sw = 0.0, swI = 0.0, swh[2] = {0.0, 0.0}, swIh[2] = {0.0, 0.0}; - int nh[2] = {0, 0}; - for (int q = gstart[g]; q < gstart[g + 1]; ++q) { - const int i = member[q]; - const double v = model_var(i, em_mean[g]); - if (!(v > 0.0)) continue; - const double I_corr = static_cast(mf.I[i]) * mf.corr[i]; - sw += 1.0 / v; swI += I_corr / v; - if (merge_friedel && group_has_hands[g]) { - swh[obs_hand[i]] += 1.0 / v; swIh[obs_hand[i]] += I_corr / v; nh[obs_hand[i]]++; - } - } - if (!(sw > 0.0)) continue; - const double mean = swI / sw; - next_mean[g] = mean; - // The samples about the hand's own mean where it has two of its own, as above. - for (int q = gstart[g]; q < gstart[g + 1]; ++q) { - const int i = member[q]; - const double v = model_var(i, em_mean[g]); - if (!(v > 0.0)) continue; - const int hh = obs_hand[i]; - const bool on_hand = merge_friedel && group_has_hands[g] && nh[hh] >= 2 && swh[hh] > 0.0; - const double centre = on_hand ? swIh[hh] / swh[hh] : mean; - const double factor = 1.0 - (1.0 / v) / (on_hand ? swh[hh] : sw); - if (factor < 0.05) continue; - const double sc = static_cast(mf.sigma[i]) * mf.corr[i]; - const double resid = static_cast(mf.I[i]) * mf.corr[i] - centre; - part[t].push_back({counting_variance(fulls[i], mean, sc * sc), centre * centre, - resid * resid / factor, mf.d[i]}); - } - } - }); - for (int g = 0; g < n_groups; ++g) - if (!std::isfinite(next_mean[g])) next_mean[g] = em_mean[g]; - next.clear(); - for (auto &v : part) next.insert(next.end(), v.begin(), v.end()); - em_mean.swap(next_mean); - samples.swap(next); - fit_error_model(samples); - if (std::fabs(error_model_a - a) <= 1e-3 * a - && std::fabs(error_model_b - b) <= 1e-3 * std::max(b, 1e-6)) - break; - } -#ifdef JFJOCH_USE_CUDA - if (use_gpu_merge) gpu_->SetEmMean(em_mean.data()); -#endif - } // Per-group outlier-rejection median of I*corr (host both paths - a per-group median is awkward on // the GPU; cheap here, cnt >= 3 filter from the em pass). Fed to the merge accumulate. @@ -4638,21 +4540,15 @@ RotationScaleMerge::Result RotationScaleMerge::MergeAndStats(int n_groups, bool // Inverse-variance WEIGHTED median: a frame the crystal barely diffracted on is scaled up by // 1/G together with its sigma, so a plain median lets two such observations outvote one // well-measured one - and the test below then rejects the well-measured one against its - // own small sigma. The variance is the model's, at the reflection's mean - the weight the - // merge gives the observation (see the centre refit above). - std::vector> iv(start[n_sets]); // (I*corr, 1/model variance) + // own small sigma. + std::vector> iv(start[n_sets]); // (I*corr, 1/(sigma*corr)^2) { std::vector fill(start.begin(), start.end() - 1); for (int i = 0; i < n_full; ++i) { const int g = mf.group[i]; if (g < 0) continue; const float sc = mf.sigma[i] * mf.corr[i]; - const double mean = std::isfinite(em_mean[g]) ? em_mean[g] : static_cast(mf.I[i]) * mf.corr[i]; - const double mv = error_model_active - ? error_model_a * counting_variance(fulls[i], mean, static_cast(sc) * sc) - + (error_model_b * mean) * (error_model_b * mean) - : static_cast(sc) * sc; - const std::pair v{mf.I[i] * mf.corr[i], mv > 0.0 ? static_cast(1.0 / mv) : 0.0f}; + const std::pair v{mf.I[i] * mf.corr[i], sc > 0.0f ? 1.0f / (sc * sc) : 0.0f}; if (cnt[g] >= 3) iv[fill[g]++] = v; if (!pair_needed.empty() && pair_needed[pair_of_group[g]]) iv[fill[n_groups + pair_of_group[g]]++] = v; @@ -4679,6 +4575,7 @@ RotationScaleMerge::Result RotationScaleMerge::MergeAndStats(int n_groups, bool reject_median[g] = cnt[g] >= 3 ? set_median[g] : !pair_needed.empty() ? set_median[n_groups + pair_of_group[g]] : NAN; } + fit_error_model(samples); } // The full's sigma under the error model, with the variance evaluated at intensity I_for_b. diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.cu b/image_analysis/scale_merge/RotationScaleMergeGPU.cu index 257886bc3..0ecf08707 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.cu +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.cu @@ -1126,14 +1126,6 @@ void RotationScaleMergeGPU::SetFrameCellOk(const uint8_t *frame_cell_ok) { // The per-group inv-var mean (em_mean) + the per-full leverage-corrected error-model samples over the // resident+scaled fulls. Stashes the filter context for the later MergeAccum/MergeRmeas calls. -void RotationScaleMergeGPU::SetEmMean(const double *em_mean) { - DeviceGuard guard(impl_->device, impl_->available); - auto &d = *impl_; - if (d.n_groups > 0) - CopyAndWait(d.m_em_mean.get(), em_mean, size_t(d.n_groups) * sizeof(double), cudaMemcpyHostToDevice, - impl_->s(), "ul em_mean"); -} - void RotationScaleMergeGPU::MergeEmSamples(bool for_search, double min_partiality, const uint8_t *hand, const uint8_t *has_hands, double *em_mean_out, int32_t *cnt_out, double *s2_out, diff --git a/image_analysis/scale_merge/RotationScaleMergeGPU.h b/image_analysis/scale_merge/RotationScaleMergeGPU.h index 349343b1a..98aa1b18a 100644 --- a/image_analysis/scale_merge/RotationScaleMergeGPU.h +++ b/image_analysis/scale_merge/RotationScaleMergeGPU.h @@ -103,9 +103,6 @@ public: // half-set weights multiplied by it. Requires MergeEmSamples first (em_mean resident). // reject_var_add (n_groups) widens the pooled cut by the shell's own measured Bijvoet // variance; null leaves the plain n-sigma test. - // Replace the per-group means MergeEmSamples left on the device (n_groups values): the merge's - // model sigmas are evaluated at them. - void SetEmMean(const double *em_mean); void MergeAccum(double error_model_a, double error_model_b, bool error_model_active, bool reject_outliers, double reject_nsigma, const float *reject_median, const float *reject_var_add, -- 2.54.0 From 52932337fb7819019e7950a99b35c4edf23938ac Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 11:56:32 +0200 Subject: [PATCH 36/97] Space group: a point-group promotion keeps the screws the adopted group decided When a refused point group is re-asked on the adopted group's merge (twin-immune zone / merge comparison), that merge no longer holds the adopted group's screw-absent axial reflections. The higher search then had nothing to judge on those rows, every candidate of the point group tied, and the lowest-numbered - the one claiming no screw - was written. Measured on an open-arm P4_2 2_1 2 crystal (8v2t): adopted P2_1 2_1 2_1, promoted by the twin-immune zone, all three axial rows "no control class", P 4 2 2 written. Among the candidates the higher search cannot separate, take the first that keeps every axial absence the adopted group claims (as the centring already is taken), drop from the alternatives the ones that do not, and stop reporting those rows as undetermined. Rows the adopted group left present are still in the merge and still decided by the higher search. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- rugnux/Rugnux.cpp | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 74aa0c628..8e97094f6 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -8231,13 +8231,57 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b hi_opts.cell = gemmi::UnitCell(*result.consensus_cell); if (end_msg.rotation_lattice_type.has_value()) hi_opts.lattice_system = end_msg.rotation_lattice_type->crystal_system; - const auto hi_search = SearchSpaceGroup(sm.merged, hi_opts); + auto hi_search = SearchSpaceGroup(sm.merged, hi_opts); // ...and its CENTRING is taken rather than read. The adopted group has already // removed its centring-absent class from this merge, so that class holds no data, // every candidate of the point group is equally consistent with it, and the // tie-break takes the lowest-numbered - which is the primitive one. Merging in // that would hand the higher arm the centring-absent reflections as if they were // data and lose the comparison on noise. Same move the metric-lattice ask makes. + // + // Its SCREWS are taken the same way, for the same reason: the adopted group's + // screw-absent axial reflections are not in this merge either, so on those rows + // the higher search has nothing to judge, its candidates tie, and the lowest- + // numbered - the one claiming no screw - is named. Measured on a P4_2 2_1 2 + // crystal adopted as P2_1 2_1 2_1 and promoted by the twin-immune zone: all three + // axial rows read "no control class" and P 4 2 2 was written. So among the + // candidates the higher search cannot separate, the ones that drop an axial + // absence the adopted group claims are dropped; the axial reflections the adopted + // group left present are in this merge and stay the higher search's to decide. + const gemmi::GroupOps lo_ops = sg.operations(); + const auto keeps_screws = [&](const gemmi::SpaceGroup &hi) { + const gemmi::GroupOps hi_ops = hi.operations(); + for (int axis = 0; axis < 3; ++axis) + for (int n = 1; n <= 6; ++n) { + gemmi::Op::Miller hkl{{0, 0, 0}}; + hkl[axis] = n; + if (lo_ops.is_systematically_absent(hkl) && !hi_ops.is_systematically_absent(hkl)) + return false; + } + return true; + }; + if (hi_search.best_space_group.has_value()) { + std::vector tied{*hi_search.best_space_group}; + tied.insert(tied.end(), hi_search.alternatives.begin(), hi_search.alternatives.end()); + const auto kept = std::find_if(tied.begin(), tied.end(), [&](const gemmi::SpaceGroup &c) { + return c.centring_type() == sg.centring_type() && keeps_screws(c); + }); + if (kept != tied.end()) { + hi_search.best_space_group = *kept; + hi_search.alternatives.clear(); + for (const auto &c : tied) + if (c.number != kept->number + && (c.centring_type() != sg.centring_type() || keeps_screws(c))) + hi_search.alternatives.push_back(c); + for (auto &c : hi_search.candidates) + if (c.selected && c.space_group.centring_type() == sg.centring_type()) + c.selected = keeps_screws(c.space_group); + // A row the adopted group decided is not open, whatever this merge holds of it. + std::erase_if(hi_search.undetermined_screws, [&](const auto &u) { + return lo_ops.is_systematically_absent({{u.row[0], u.row[1], u.row[2]}}); + }); + } + } std::optional hi_sg; if (hi_search.best_space_group.has_value() && hi_search.best_space_group->centring_type() == sg.centring_type()) -- 2.54.0 From 3f89759cb5d70fe9feb8c0d45a22de25c6f8a622 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 11:56:32 +0200 Subject: [PATCH 37/97] Space group: screw-absence bound 20 -> 8 nats, recalibrated on the zones Mapped every single-axis screw candidate of every primitive-lattice set in battery 3 (open + in-house) and the private arm against the deposited group. False zones with no violation read at most +4 nats (plus one pseudo- translation row at +30.5 that no bound separates); true zones go down to +10. The true zones under 20 are all short monoclinic rows (b ~ 25-30 A: four to six 0k0-odd reflections inside the search's resolution range, on weak data at a few percent of their row), refused at 20 on five crystals that are P2_1: four myoglobin sweeps (10.0-16.3 nats) and 6cs9 (10.8). The rc174-cand myob_x06da_split call sat at 21.1, one refit away from P2. Re-scoring the stored candidate tables at the new bound changes only those five plus 3r6o (I4_1 2 2 newly eligible, toward the deposited I4_1 screw); no adopted group elsewhere. New test section pins both sides: four 0k0-odd at 2% of their row are claimed (11.3 nats), at 20% they are not. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- image_analysis/scale_merge/SearchSpaceGroup.h | 14 ++++++++++- tests/SearchSpaceGroupTest.cpp | 25 +++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/image_analysis/scale_merge/SearchSpaceGroup.h b/image_analysis/scale_merge/SearchSpaceGroup.h index 1c7bd85f6..9bf15f789 100644 --- a/image_analysis/scale_merge/SearchSpaceGroup.h +++ b/image_analysis/scale_merge/SearchSpaceGroup.h @@ -635,7 +635,19 @@ struct SearchSpaceGroupOptions { // exists to avoid. A thin monoclinic sweep reaching only three 0k0-odd reflections loses its // 2(1) to this. Lowering the floor to 7.7e-4 would let three through; that is a recalibration // and needs its own battery. - double min_screw_absence_evidence = 20.0; + // + // Recalibrated on the zones themselves rather than on the probe crystals: every single-axis + // candidate of every primitive-lattice set of a 230-set battery (open, in-house and private), + // labelled against the deposited group. A false zone with no violation reads at most +4 nats + // (a row that is weak, not dead, adds at most +6 to a group that is eligible without it), and the + // one false zone above that, +30.5, is a row that is genuinely present at a few percent of its + // neighbours under a pseudo-translation - no bound separates it from a true screw, whose rows + // read from +10 up. The true zones under 20 are all short rows: a 2(1) along a monoclinic axis of + // 25-30 A reaches four to six odd reflections inside the search's resolution range, and on weak + // data those sit at a few percent of their row, not at the floor - 10.0 to 16.3 nats, refused at + // 20 on every one of five such crystals. The bound sits in the measured gap. Two absences can now clear it if both are below about 2% of + // their row. + double min_screw_absence_evidence = 8.0; // ---- Glide planes (small-molecule space groups) ------------------------------------------- // Offer the NON-SOHNCKE space groups of the same proper-rotation set as candidates, so a glide diff --git a/tests/SearchSpaceGroupTest.cpp b/tests/SearchSpaceGroupTest.cpp index 59b912021..846e410cc 100644 --- a/tests/SearchSpaceGroupTest.cpp +++ b/tests/SearchSpaceGroupTest.cpp @@ -279,6 +279,31 @@ TEST_CASE("SearchSpaceGroup weighs a screw's absences by evidence, not by how ma CHECK(result.best_space_group->short_name() == "P21"); } + // A short monoclinic axis on weak data: four 0k0-odd reflections in range, each at 2% of its row + // rather than at the floor. That zone reads ~11 nats - refused under the old bound of 20 on five + // crystals that are P2_1 - and must be claimed; the same four at a fifth of their row must not. + SECTION("four absences at a few percent of a short row: the screw is found, at a fifth it is not") { + for (const auto& [fraction, expected] : {std::pair{0.02, "P21"}, std::pair{0.2, "P2"}}) { + auto merged = GenerateMergedReflectionsForSpaceGroup(sg, 18); + int kept = 0; + std::erase_if(merged, [&](MergedReflection& r) { + if (!gops.is_systematically_absent(gemmi::Op::Miller{{r.h, r.k, r.l}})) + return false; + if (kept >= 4) + return true; + ++kept; + r.I = static_cast(fraction * 350.0); + return false; + }); + REQUIRE(kept == 4); + + const auto result = SearchSpaceGroup(merged, opt); + INFO(SearchSpaceGroupResultToText(result)); + REQUIRE(result.best_space_group.has_value()); + CHECK(result.best_space_group->short_name() == expected); + } + } + SECTION("a uniformly weak axial row decides nothing, however many absences it holds") { // The whole 0k0 row badly measured: the predicted-absent reflections are weak, but so is the // rest of their row, so there is no contrast and no screw to claim. A violation count cannot -- 2.54.0 From fc4bbab8e2d19649282bd367d376332870f17bd3 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 4 Oct 2026 12:03:29 +0200 Subject: [PATCH 38/97] v1.0.0-rc.174 Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB --- VERSION | 2 +- broker/gen/model/Azim_int_settings.cpp | 2 +- broker/gen/model/Azim_int_settings.h | 2 +- .../gen/model/Bragg_integration_settings.cpp | 2 +- broker/gen/model/Bragg_integration_settings.h | 2 +- broker/gen/model/Broker_status.cpp | 2 +- broker/gen/model/Broker_status.h | 2 +- .../model/Calibration_statistics_inner.cpp | 2 +- .../gen/model/Calibration_statistics_inner.h | 2 +- broker/gen/model/Dark_mask_settings.cpp | 2 +- broker/gen/model/Dark_mask_settings.h | 2 +- broker/gen/model/Dataset_settings.cpp | 2 +- broker/gen/model/Dataset_settings.h | 2 +- broker/gen/model/Dataset_settings_smargon.cpp | 2 +- broker/gen/model/Dataset_settings_smargon.h | 2 +- ...et_settings_xray_fluorescence_spectrum.cpp | 2 +- ...aset_settings_xray_fluorescence_spectrum.h | 2 +- broker/gen/model/Detector.cpp | 2 +- broker/gen/model/Detector.h | 2 +- broker/gen/model/Detector_list.cpp | 2 +- broker/gen/model/Detector_list.h | 2 +- broker/gen/model/Detector_list_element.cpp | 2 +- broker/gen/model/Detector_list_element.h | 2 +- broker/gen/model/Detector_module.cpp | 2 +- broker/gen/model/Detector_module.h | 2 +- .../gen/model/Detector_module_direction.cpp | 2 +- broker/gen/model/Detector_module_direction.h | 2 +- broker/gen/model/Detector_power_state.cpp | 2 +- broker/gen/model/Detector_power_state.h | 2 +- broker/gen/model/Detector_selection.cpp | 2 +- broker/gen/model/Detector_selection.h | 2 +- broker/gen/model/Detector_settings.cpp | 2 +- broker/gen/model/Detector_settings.h | 2 +- broker/gen/model/Detector_state.cpp | 2 +- broker/gen/model/Detector_state.h | 2 +- broker/gen/model/Detector_status.cpp | 2 +- broker/gen/model/Detector_status.h | 2 +- broker/gen/model/Detector_timing.cpp | 2 +- broker/gen/model/Detector_timing.h | 2 +- broker/gen/model/Detector_type.cpp | 2 +- broker/gen/model/Detector_type.h | 2 +- broker/gen/model/Error_message.cpp | 2 +- broker/gen/model/Error_message.h | 2 +- broker/gen/model/File_writer_format.cpp | 2 +- broker/gen/model/File_writer_format.h | 2 +- broker/gen/model/File_writer_settings.cpp | 2 +- broker/gen/model/File_writer_settings.h | 2 +- broker/gen/model/Fpga_status_inner.cpp | 2 +- broker/gen/model/Fpga_status_inner.h | 2 +- .../gen/model/Geom_refinement_algorithm.cpp | 2 +- broker/gen/model/Geom_refinement_algorithm.h | 2 +- broker/gen/model/Grid_scan.cpp | 2 +- broker/gen/model/Grid_scan.h | 2 +- broker/gen/model/Helpers.cpp | 2 +- broker/gen/model/Helpers.h | 2 +- broker/gen/model/Image_buffer_status.cpp | 2 +- broker/gen/model/Image_buffer_status.h | 2 +- broker/gen/model/Image_format_settings.cpp | 2 +- broker/gen/model/Image_format_settings.h | 2 +- broker/gen/model/Image_pusher_status.cpp | 2 +- broker/gen/model/Image_pusher_status.h | 2 +- broker/gen/model/Image_pusher_type.cpp | 2 +- broker/gen/model/Image_pusher_type.h | 2 +- broker/gen/model/Indexing_algorithm.cpp | 2 +- broker/gen/model/Indexing_algorithm.h | 2 +- broker/gen/model/Indexing_settings.cpp | 2 +- broker/gen/model/Indexing_settings.h | 2 +- broker/gen/model/Instrument_metadata.cpp | 2 +- broker/gen/model/Instrument_metadata.h | 2 +- broker/gen/model/Integration_model.cpp | 2 +- broker/gen/model/Integration_model.h | 2 +- broker/gen/model/Jfjoch_settings.cpp | 2 +- broker/gen/model/Jfjoch_settings.h | 2 +- broker/gen/model/Jfjoch_statistics.cpp | 2 +- broker/gen/model/Jfjoch_statistics.h | 2 +- broker/gen/model/Measurement_statistics.cpp | 2 +- broker/gen/model/Measurement_statistics.h | 2 +- broker/gen/model/Pcie_devices_inner.cpp | 2 +- broker/gen/model/Pcie_devices_inner.h | 2 +- broker/gen/model/Pixel_mask_statistics.cpp | 2 +- broker/gen/model/Pixel_mask_statistics.h | 2 +- broker/gen/model/Plot.cpp | 2 +- broker/gen/model/Plot.h | 2 +- broker/gen/model/Plot_unit_x.cpp | 2 +- broker/gen/model/Plot_unit_x.h | 2 +- broker/gen/model/Plots.cpp | 2 +- broker/gen/model/Plots.h | 2 +- .../model/Powder_calibration_fit_sigma.cpp | 2 +- .../gen/model/Powder_calibration_fit_sigma.h | 2 +- .../gen/model/Powder_calibration_output.cpp | 2 +- broker/gen/model/Powder_calibration_output.h | 2 +- .../gen/model/Powder_calibration_quality.cpp | 2 +- broker/gen/model/Powder_calibration_quality.h | 2 +- .../model/Powder_calibration_spot_check.cpp | 2 +- .../gen/model/Powder_calibration_spot_check.h | 2 +- broker/gen/model/Roi_azim_list.cpp | 2 +- broker/gen/model/Roi_azim_list.h | 2 +- broker/gen/model/Roi_azimuthal.cpp | 2 +- broker/gen/model/Roi_azimuthal.h | 2 +- broker/gen/model/Roi_box.cpp | 2 +- broker/gen/model/Roi_box.h | 2 +- broker/gen/model/Roi_box_list.cpp | 2 +- broker/gen/model/Roi_box_list.h | 2 +- broker/gen/model/Roi_circle.cpp | 2 +- broker/gen/model/Roi_circle.h | 2 +- broker/gen/model/Roi_circle_list.cpp | 2 +- broker/gen/model/Roi_circle_list.h | 2 +- broker/gen/model/Roi_definitions.cpp | 2 +- broker/gen/model/Roi_definitions.h | 2 +- broker/gen/model/Rotation_axis.cpp | 2 +- broker/gen/model/Rotation_axis.h | 2 +- broker/gen/model/Scan_result.cpp | 2 +- broker/gen/model/Scan_result.h | 2 +- broker/gen/model/Scan_result_images_inner.cpp | 2 +- broker/gen/model/Scan_result_images_inner.h | 2 +- broker/gen/model/Spot_finding_settings.cpp | 2 +- broker/gen/model/Spot_finding_settings.h | 2 +- .../gen/model/Standard_detector_geometry.cpp | 2 +- broker/gen/model/Standard_detector_geometry.h | 2 +- broker/gen/model/Tcp_settings.cpp | 2 +- broker/gen/model/Tcp_settings.h | 2 +- broker/gen/model/Unit_cell.cpp | 2 +- broker/gen/model/Unit_cell.h | 2 +- broker/gen/model/Zeromq_metadata_settings.cpp | 2 +- broker/gen/model/Zeromq_metadata_settings.h | 2 +- broker/gen/model/Zeromq_preview_settings.cpp | 2 +- broker/gen/model/Zeromq_preview_settings.h | 2 +- broker/gen/model/Zeromq_settings.cpp | 2 +- broker/gen/model/Zeromq_settings.h | 2 +- broker/jfjoch_api.yaml | 2 +- broker/redoc-static.html | 73 ++++++++-- docs/THIRD_PARTY_NOTICES.md | 9 +- docs/conf.py | 2 +- docs/python_client/README.md | 11 +- docs/python_client/docs/DatasetSettings.md | 1 + docs/python_client/docs/DefaultApi.md | 128 +++++++++++++++--- fpga/hdl/action_config.v | 2 +- fpga/pcie_driver/dkms.conf | 2 +- fpga/pcie_driver/install_dkms.sh | 2 +- fpga/pcie_driver/jfjoch_drv.c | 2 +- fpga/pcie_driver/postinstall.sh | 2 +- fpga/pcie_driver/preuninstall.sh | 2 +- frontend/package-lock.json | 4 +- frontend/package.json | 2 +- frontend/src/version.ts | 2 +- 145 files changed, 326 insertions(+), 178 deletions(-) diff --git a/VERSION b/VERSION index 39d016236..f1d7bb928 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.0.0-rc.173 +1.0.0-rc.174 diff --git a/broker/gen/model/Azim_int_settings.cpp b/broker/gen/model/Azim_int_settings.cpp index e9bcda7f3..1c4720fd8 100644 --- a/broker/gen/model/Azim_int_settings.cpp +++ b/broker/gen/model/Azim_int_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Azim_int_settings.h b/broker/gen/model/Azim_int_settings.h index 8e0ac816f..ba719b315 100644 --- a/broker/gen/model/Azim_int_settings.h +++ b/broker/gen/model/Azim_int_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Bragg_integration_settings.cpp b/broker/gen/model/Bragg_integration_settings.cpp index d9fc9b537..9333e84f2 100644 --- a/broker/gen/model/Bragg_integration_settings.cpp +++ b/broker/gen/model/Bragg_integration_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Bragg_integration_settings.h b/broker/gen/model/Bragg_integration_settings.h index b91a5eb7d..d374c5e16 100644 --- a/broker/gen/model/Bragg_integration_settings.h +++ b/broker/gen/model/Bragg_integration_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Broker_status.cpp b/broker/gen/model/Broker_status.cpp index cd5e893d8..c367ffdb5 100644 --- a/broker/gen/model/Broker_status.cpp +++ b/broker/gen/model/Broker_status.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Broker_status.h b/broker/gen/model/Broker_status.h index 5ce850ed2..e9e206a2b 100644 --- a/broker/gen/model/Broker_status.h +++ b/broker/gen/model/Broker_status.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Calibration_statistics_inner.cpp b/broker/gen/model/Calibration_statistics_inner.cpp index 79c30a5ec..943c5e76d 100644 --- a/broker/gen/model/Calibration_statistics_inner.cpp +++ b/broker/gen/model/Calibration_statistics_inner.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Calibration_statistics_inner.h b/broker/gen/model/Calibration_statistics_inner.h index 02a39f807..b0f835542 100644 --- a/broker/gen/model/Calibration_statistics_inner.h +++ b/broker/gen/model/Calibration_statistics_inner.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dark_mask_settings.cpp b/broker/gen/model/Dark_mask_settings.cpp index 79a473f30..c25f954b5 100644 --- a/broker/gen/model/Dark_mask_settings.cpp +++ b/broker/gen/model/Dark_mask_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dark_mask_settings.h b/broker/gen/model/Dark_mask_settings.h index c33167bc4..b7ea17375 100644 --- a/broker/gen/model/Dark_mask_settings.h +++ b/broker/gen/model/Dark_mask_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dataset_settings.cpp b/broker/gen/model/Dataset_settings.cpp index 5811fc2ad..c9dac3ccc 100644 --- a/broker/gen/model/Dataset_settings.cpp +++ b/broker/gen/model/Dataset_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dataset_settings.h b/broker/gen/model/Dataset_settings.h index 8a57a1025..73d410c06 100644 --- a/broker/gen/model/Dataset_settings.h +++ b/broker/gen/model/Dataset_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dataset_settings_smargon.cpp b/broker/gen/model/Dataset_settings_smargon.cpp index 25ab779de..8fb93f2bb 100644 --- a/broker/gen/model/Dataset_settings_smargon.cpp +++ b/broker/gen/model/Dataset_settings_smargon.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dataset_settings_smargon.h b/broker/gen/model/Dataset_settings_smargon.h index 8b6f9bd93..96de8f42b 100644 --- a/broker/gen/model/Dataset_settings_smargon.h +++ b/broker/gen/model/Dataset_settings_smargon.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.cpp b/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.cpp index deb813a18..17d9ee499 100644 --- a/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.cpp +++ b/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.h b/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.h index d11de5c42..d1e3bb8ea 100644 --- a/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.h +++ b/broker/gen/model/Dataset_settings_xray_fluorescence_spectrum.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector.cpp b/broker/gen/model/Detector.cpp index 35d30290e..fab54fdd2 100644 --- a/broker/gen/model/Detector.cpp +++ b/broker/gen/model/Detector.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector.h b/broker/gen/model/Detector.h index 8d84fd1ee..46a2d25f2 100644 --- a/broker/gen/model/Detector.h +++ b/broker/gen/model/Detector.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_list.cpp b/broker/gen/model/Detector_list.cpp index 74c3ed6d2..9bba81e1f 100644 --- a/broker/gen/model/Detector_list.cpp +++ b/broker/gen/model/Detector_list.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_list.h b/broker/gen/model/Detector_list.h index d4bea19a0..c313c8b01 100644 --- a/broker/gen/model/Detector_list.h +++ b/broker/gen/model/Detector_list.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_list_element.cpp b/broker/gen/model/Detector_list_element.cpp index 9e993750f..2a05876dd 100644 --- a/broker/gen/model/Detector_list_element.cpp +++ b/broker/gen/model/Detector_list_element.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_list_element.h b/broker/gen/model/Detector_list_element.h index 6addd2252..f993082e8 100644 --- a/broker/gen/model/Detector_list_element.h +++ b/broker/gen/model/Detector_list_element.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_module.cpp b/broker/gen/model/Detector_module.cpp index f6da479e0..b45af856c 100644 --- a/broker/gen/model/Detector_module.cpp +++ b/broker/gen/model/Detector_module.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_module.h b/broker/gen/model/Detector_module.h index eecec4757..87ff2efed 100644 --- a/broker/gen/model/Detector_module.h +++ b/broker/gen/model/Detector_module.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_module_direction.cpp b/broker/gen/model/Detector_module_direction.cpp index e497908b3..5524e43d8 100644 --- a/broker/gen/model/Detector_module_direction.cpp +++ b/broker/gen/model/Detector_module_direction.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_module_direction.h b/broker/gen/model/Detector_module_direction.h index 199ae4b04..1d056e741 100644 --- a/broker/gen/model/Detector_module_direction.h +++ b/broker/gen/model/Detector_module_direction.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_power_state.cpp b/broker/gen/model/Detector_power_state.cpp index c7a95602e..982598f20 100644 --- a/broker/gen/model/Detector_power_state.cpp +++ b/broker/gen/model/Detector_power_state.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_power_state.h b/broker/gen/model/Detector_power_state.h index 82cbf5d27..c3207e195 100644 --- a/broker/gen/model/Detector_power_state.h +++ b/broker/gen/model/Detector_power_state.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_selection.cpp b/broker/gen/model/Detector_selection.cpp index 9b7bd00a2..7dab4d19b 100644 --- a/broker/gen/model/Detector_selection.cpp +++ b/broker/gen/model/Detector_selection.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_selection.h b/broker/gen/model/Detector_selection.h index d09d5106e..d9898e9f9 100644 --- a/broker/gen/model/Detector_selection.h +++ b/broker/gen/model/Detector_selection.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_settings.cpp b/broker/gen/model/Detector_settings.cpp index 7624b3391..dc5dec082 100644 --- a/broker/gen/model/Detector_settings.cpp +++ b/broker/gen/model/Detector_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_settings.h b/broker/gen/model/Detector_settings.h index 3c59332ee..2665c70b7 100644 --- a/broker/gen/model/Detector_settings.h +++ b/broker/gen/model/Detector_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_state.cpp b/broker/gen/model/Detector_state.cpp index 4564a205c..d15698a4f 100644 --- a/broker/gen/model/Detector_state.cpp +++ b/broker/gen/model/Detector_state.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_state.h b/broker/gen/model/Detector_state.h index f515fc440..5fe4ca892 100644 --- a/broker/gen/model/Detector_state.h +++ b/broker/gen/model/Detector_state.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_status.cpp b/broker/gen/model/Detector_status.cpp index 1fc0e1fa7..844b13a9c 100644 --- a/broker/gen/model/Detector_status.cpp +++ b/broker/gen/model/Detector_status.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_status.h b/broker/gen/model/Detector_status.h index 12371d4b2..97525cb56 100644 --- a/broker/gen/model/Detector_status.h +++ b/broker/gen/model/Detector_status.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_timing.cpp b/broker/gen/model/Detector_timing.cpp index ce5eb8e11..df07a23c2 100644 --- a/broker/gen/model/Detector_timing.cpp +++ b/broker/gen/model/Detector_timing.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_timing.h b/broker/gen/model/Detector_timing.h index 98ae64ba8..9d0a29f4c 100644 --- a/broker/gen/model/Detector_timing.h +++ b/broker/gen/model/Detector_timing.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_type.cpp b/broker/gen/model/Detector_type.cpp index 6c0171d53..4d890d966 100644 --- a/broker/gen/model/Detector_type.cpp +++ b/broker/gen/model/Detector_type.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Detector_type.h b/broker/gen/model/Detector_type.h index f0a6b753c..322d88fc0 100644 --- a/broker/gen/model/Detector_type.h +++ b/broker/gen/model/Detector_type.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Error_message.cpp b/broker/gen/model/Error_message.cpp index b51defa77..f396a6c12 100644 --- a/broker/gen/model/Error_message.cpp +++ b/broker/gen/model/Error_message.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Error_message.h b/broker/gen/model/Error_message.h index 7059cbf92..e6d74e473 100644 --- a/broker/gen/model/Error_message.h +++ b/broker/gen/model/Error_message.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/File_writer_format.cpp b/broker/gen/model/File_writer_format.cpp index 25979bc2a..57e3e7671 100644 --- a/broker/gen/model/File_writer_format.cpp +++ b/broker/gen/model/File_writer_format.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/File_writer_format.h b/broker/gen/model/File_writer_format.h index dd1d1f4bb..73d87baa2 100644 --- a/broker/gen/model/File_writer_format.h +++ b/broker/gen/model/File_writer_format.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/File_writer_settings.cpp b/broker/gen/model/File_writer_settings.cpp index 658c5214b..8eb0c1775 100644 --- a/broker/gen/model/File_writer_settings.cpp +++ b/broker/gen/model/File_writer_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/File_writer_settings.h b/broker/gen/model/File_writer_settings.h index a41c4465f..5f5531e7f 100644 --- a/broker/gen/model/File_writer_settings.h +++ b/broker/gen/model/File_writer_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Fpga_status_inner.cpp b/broker/gen/model/Fpga_status_inner.cpp index 71fe36b69..ff0abaf42 100644 --- a/broker/gen/model/Fpga_status_inner.cpp +++ b/broker/gen/model/Fpga_status_inner.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Fpga_status_inner.h b/broker/gen/model/Fpga_status_inner.h index 6510d199b..fe916e613 100644 --- a/broker/gen/model/Fpga_status_inner.h +++ b/broker/gen/model/Fpga_status_inner.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Geom_refinement_algorithm.cpp b/broker/gen/model/Geom_refinement_algorithm.cpp index b1ca0532c..7680cda71 100644 --- a/broker/gen/model/Geom_refinement_algorithm.cpp +++ b/broker/gen/model/Geom_refinement_algorithm.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Geom_refinement_algorithm.h b/broker/gen/model/Geom_refinement_algorithm.h index b920ff367..369eb4ad7 100644 --- a/broker/gen/model/Geom_refinement_algorithm.h +++ b/broker/gen/model/Geom_refinement_algorithm.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Grid_scan.cpp b/broker/gen/model/Grid_scan.cpp index 7202ee49d..fe712d7ef 100644 --- a/broker/gen/model/Grid_scan.cpp +++ b/broker/gen/model/Grid_scan.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Grid_scan.h b/broker/gen/model/Grid_scan.h index 1f7fc1b8d..b119dc25f 100644 --- a/broker/gen/model/Grid_scan.h +++ b/broker/gen/model/Grid_scan.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Helpers.cpp b/broker/gen/model/Helpers.cpp index 82def6021..28b0e7ba3 100644 --- a/broker/gen/model/Helpers.cpp +++ b/broker/gen/model/Helpers.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Helpers.h b/broker/gen/model/Helpers.h index 0c0c8edda..2d5918041 100644 --- a/broker/gen/model/Helpers.h +++ b/broker/gen/model/Helpers.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_buffer_status.cpp b/broker/gen/model/Image_buffer_status.cpp index b6be0c851..df3e39d97 100644 --- a/broker/gen/model/Image_buffer_status.cpp +++ b/broker/gen/model/Image_buffer_status.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_buffer_status.h b/broker/gen/model/Image_buffer_status.h index 28510f379..54976e2b6 100644 --- a/broker/gen/model/Image_buffer_status.h +++ b/broker/gen/model/Image_buffer_status.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_format_settings.cpp b/broker/gen/model/Image_format_settings.cpp index fcf774825..345266a49 100644 --- a/broker/gen/model/Image_format_settings.cpp +++ b/broker/gen/model/Image_format_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_format_settings.h b/broker/gen/model/Image_format_settings.h index a17f30b0f..75f3261f3 100644 --- a/broker/gen/model/Image_format_settings.h +++ b/broker/gen/model/Image_format_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_pusher_status.cpp b/broker/gen/model/Image_pusher_status.cpp index 849cbe254..95470ebf6 100644 --- a/broker/gen/model/Image_pusher_status.cpp +++ b/broker/gen/model/Image_pusher_status.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_pusher_status.h b/broker/gen/model/Image_pusher_status.h index da21f0aef..add1c9967 100644 --- a/broker/gen/model/Image_pusher_status.h +++ b/broker/gen/model/Image_pusher_status.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_pusher_type.cpp b/broker/gen/model/Image_pusher_type.cpp index db8ba45b6..ac2310dc6 100644 --- a/broker/gen/model/Image_pusher_type.cpp +++ b/broker/gen/model/Image_pusher_type.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Image_pusher_type.h b/broker/gen/model/Image_pusher_type.h index cf74771b8..847858d93 100644 --- a/broker/gen/model/Image_pusher_type.h +++ b/broker/gen/model/Image_pusher_type.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Indexing_algorithm.cpp b/broker/gen/model/Indexing_algorithm.cpp index 32973fc31..22646fcb4 100644 --- a/broker/gen/model/Indexing_algorithm.cpp +++ b/broker/gen/model/Indexing_algorithm.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Indexing_algorithm.h b/broker/gen/model/Indexing_algorithm.h index 54b2deb07..9a34e2649 100644 --- a/broker/gen/model/Indexing_algorithm.h +++ b/broker/gen/model/Indexing_algorithm.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Indexing_settings.cpp b/broker/gen/model/Indexing_settings.cpp index 7c1e6d0c9..3143c7f20 100644 --- a/broker/gen/model/Indexing_settings.cpp +++ b/broker/gen/model/Indexing_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Indexing_settings.h b/broker/gen/model/Indexing_settings.h index 6682e553b..549867c02 100644 --- a/broker/gen/model/Indexing_settings.h +++ b/broker/gen/model/Indexing_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Instrument_metadata.cpp b/broker/gen/model/Instrument_metadata.cpp index cd630af6d..124495b0c 100644 --- a/broker/gen/model/Instrument_metadata.cpp +++ b/broker/gen/model/Instrument_metadata.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Instrument_metadata.h b/broker/gen/model/Instrument_metadata.h index f2691897b..bde99a7b9 100644 --- a/broker/gen/model/Instrument_metadata.h +++ b/broker/gen/model/Instrument_metadata.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Integration_model.cpp b/broker/gen/model/Integration_model.cpp index 391ce2684..cb914d3f4 100644 --- a/broker/gen/model/Integration_model.cpp +++ b/broker/gen/model/Integration_model.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Integration_model.h b/broker/gen/model/Integration_model.h index bcfe8b589..c022de062 100644 --- a/broker/gen/model/Integration_model.h +++ b/broker/gen/model/Integration_model.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Jfjoch_settings.cpp b/broker/gen/model/Jfjoch_settings.cpp index 93db3b8c7..5c5f48c68 100644 --- a/broker/gen/model/Jfjoch_settings.cpp +++ b/broker/gen/model/Jfjoch_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Jfjoch_settings.h b/broker/gen/model/Jfjoch_settings.h index e867dbdbd..f0190d231 100644 --- a/broker/gen/model/Jfjoch_settings.h +++ b/broker/gen/model/Jfjoch_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Jfjoch_statistics.cpp b/broker/gen/model/Jfjoch_statistics.cpp index 3e73b9059..1a417928d 100644 --- a/broker/gen/model/Jfjoch_statistics.cpp +++ b/broker/gen/model/Jfjoch_statistics.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Jfjoch_statistics.h b/broker/gen/model/Jfjoch_statistics.h index 644c3622e..8053d9d2d 100644 --- a/broker/gen/model/Jfjoch_statistics.h +++ b/broker/gen/model/Jfjoch_statistics.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Measurement_statistics.cpp b/broker/gen/model/Measurement_statistics.cpp index 3eb08018e..4d88c55c3 100644 --- a/broker/gen/model/Measurement_statistics.cpp +++ b/broker/gen/model/Measurement_statistics.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Measurement_statistics.h b/broker/gen/model/Measurement_statistics.h index 937e8e648..71efaac48 100644 --- a/broker/gen/model/Measurement_statistics.h +++ b/broker/gen/model/Measurement_statistics.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Pcie_devices_inner.cpp b/broker/gen/model/Pcie_devices_inner.cpp index 73719814e..21662f46a 100644 --- a/broker/gen/model/Pcie_devices_inner.cpp +++ b/broker/gen/model/Pcie_devices_inner.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Pcie_devices_inner.h b/broker/gen/model/Pcie_devices_inner.h index a95737a9c..15005c341 100644 --- a/broker/gen/model/Pcie_devices_inner.h +++ b/broker/gen/model/Pcie_devices_inner.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Pixel_mask_statistics.cpp b/broker/gen/model/Pixel_mask_statistics.cpp index 5c0437d06..fa25a6d12 100644 --- a/broker/gen/model/Pixel_mask_statistics.cpp +++ b/broker/gen/model/Pixel_mask_statistics.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Pixel_mask_statistics.h b/broker/gen/model/Pixel_mask_statistics.h index a7fa865d5..9c35f9001 100644 --- a/broker/gen/model/Pixel_mask_statistics.h +++ b/broker/gen/model/Pixel_mask_statistics.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Plot.cpp b/broker/gen/model/Plot.cpp index 217979766..67576bd34 100644 --- a/broker/gen/model/Plot.cpp +++ b/broker/gen/model/Plot.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Plot.h b/broker/gen/model/Plot.h index ea9a94a65..513b8ad84 100644 --- a/broker/gen/model/Plot.h +++ b/broker/gen/model/Plot.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Plot_unit_x.cpp b/broker/gen/model/Plot_unit_x.cpp index 49fb0f1ab..7e11c262c 100644 --- a/broker/gen/model/Plot_unit_x.cpp +++ b/broker/gen/model/Plot_unit_x.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Plot_unit_x.h b/broker/gen/model/Plot_unit_x.h index 8157fe036..ce3fc9535 100644 --- a/broker/gen/model/Plot_unit_x.h +++ b/broker/gen/model/Plot_unit_x.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Plots.cpp b/broker/gen/model/Plots.cpp index e45f59900..273ab662b 100644 --- a/broker/gen/model/Plots.cpp +++ b/broker/gen/model/Plots.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Plots.h b/broker/gen/model/Plots.h index 4598e2259..33471b9fe 100644 --- a/broker/gen/model/Plots.h +++ b/broker/gen/model/Plots.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_fit_sigma.cpp b/broker/gen/model/Powder_calibration_fit_sigma.cpp index 002f0d1a0..9f1e85859 100644 --- a/broker/gen/model/Powder_calibration_fit_sigma.cpp +++ b/broker/gen/model/Powder_calibration_fit_sigma.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_fit_sigma.h b/broker/gen/model/Powder_calibration_fit_sigma.h index cb27b81cd..4d8e5f070 100644 --- a/broker/gen/model/Powder_calibration_fit_sigma.h +++ b/broker/gen/model/Powder_calibration_fit_sigma.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_output.cpp b/broker/gen/model/Powder_calibration_output.cpp index 8cfc80175..67da070dc 100644 --- a/broker/gen/model/Powder_calibration_output.cpp +++ b/broker/gen/model/Powder_calibration_output.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_output.h b/broker/gen/model/Powder_calibration_output.h index 9fb51aaad..f48e238d1 100644 --- a/broker/gen/model/Powder_calibration_output.h +++ b/broker/gen/model/Powder_calibration_output.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_quality.cpp b/broker/gen/model/Powder_calibration_quality.cpp index ce5cf0a68..483b340d9 100644 --- a/broker/gen/model/Powder_calibration_quality.cpp +++ b/broker/gen/model/Powder_calibration_quality.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_quality.h b/broker/gen/model/Powder_calibration_quality.h index 1db4f5534..927b4ad91 100644 --- a/broker/gen/model/Powder_calibration_quality.h +++ b/broker/gen/model/Powder_calibration_quality.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_spot_check.cpp b/broker/gen/model/Powder_calibration_spot_check.cpp index c7fe6320f..5d7ca3ecc 100644 --- a/broker/gen/model/Powder_calibration_spot_check.cpp +++ b/broker/gen/model/Powder_calibration_spot_check.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Powder_calibration_spot_check.h b/broker/gen/model/Powder_calibration_spot_check.h index 504bcb406..33a53ebe7 100644 --- a/broker/gen/model/Powder_calibration_spot_check.h +++ b/broker/gen/model/Powder_calibration_spot_check.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_azim_list.cpp b/broker/gen/model/Roi_azim_list.cpp index 70c9b8990..00f849655 100644 --- a/broker/gen/model/Roi_azim_list.cpp +++ b/broker/gen/model/Roi_azim_list.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_azim_list.h b/broker/gen/model/Roi_azim_list.h index 91b00c56b..d0886ac16 100644 --- a/broker/gen/model/Roi_azim_list.h +++ b/broker/gen/model/Roi_azim_list.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_azimuthal.cpp b/broker/gen/model/Roi_azimuthal.cpp index 49a1e82a4..97ba888a2 100644 --- a/broker/gen/model/Roi_azimuthal.cpp +++ b/broker/gen/model/Roi_azimuthal.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_azimuthal.h b/broker/gen/model/Roi_azimuthal.h index 26dd644ad..f2f8c3971 100644 --- a/broker/gen/model/Roi_azimuthal.h +++ b/broker/gen/model/Roi_azimuthal.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_box.cpp b/broker/gen/model/Roi_box.cpp index 5cc2fb2dd..cfbf44bf4 100644 --- a/broker/gen/model/Roi_box.cpp +++ b/broker/gen/model/Roi_box.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_box.h b/broker/gen/model/Roi_box.h index 5eb780d1c..8a2aba778 100644 --- a/broker/gen/model/Roi_box.h +++ b/broker/gen/model/Roi_box.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_box_list.cpp b/broker/gen/model/Roi_box_list.cpp index 57ed5f725..dfa13ecef 100644 --- a/broker/gen/model/Roi_box_list.cpp +++ b/broker/gen/model/Roi_box_list.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_box_list.h b/broker/gen/model/Roi_box_list.h index 6b6bfc83e..0f0e51e2f 100644 --- a/broker/gen/model/Roi_box_list.h +++ b/broker/gen/model/Roi_box_list.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_circle.cpp b/broker/gen/model/Roi_circle.cpp index 6ae9b75ad..dbb2b913e 100644 --- a/broker/gen/model/Roi_circle.cpp +++ b/broker/gen/model/Roi_circle.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_circle.h b/broker/gen/model/Roi_circle.h index 346ff5be6..8c676eb0d 100644 --- a/broker/gen/model/Roi_circle.h +++ b/broker/gen/model/Roi_circle.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_circle_list.cpp b/broker/gen/model/Roi_circle_list.cpp index 97000b590..5976ce1d9 100644 --- a/broker/gen/model/Roi_circle_list.cpp +++ b/broker/gen/model/Roi_circle_list.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_circle_list.h b/broker/gen/model/Roi_circle_list.h index fc2219ed8..9081035ca 100644 --- a/broker/gen/model/Roi_circle_list.h +++ b/broker/gen/model/Roi_circle_list.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_definitions.cpp b/broker/gen/model/Roi_definitions.cpp index 757a7c568..810513d76 100644 --- a/broker/gen/model/Roi_definitions.cpp +++ b/broker/gen/model/Roi_definitions.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Roi_definitions.h b/broker/gen/model/Roi_definitions.h index f4209c295..6771e7cb5 100644 --- a/broker/gen/model/Roi_definitions.h +++ b/broker/gen/model/Roi_definitions.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Rotation_axis.cpp b/broker/gen/model/Rotation_axis.cpp index 7a6819edf..c705b0fb8 100644 --- a/broker/gen/model/Rotation_axis.cpp +++ b/broker/gen/model/Rotation_axis.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Rotation_axis.h b/broker/gen/model/Rotation_axis.h index b4097f132..327eeb41d 100644 --- a/broker/gen/model/Rotation_axis.h +++ b/broker/gen/model/Rotation_axis.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Scan_result.cpp b/broker/gen/model/Scan_result.cpp index 21a04b182..d9ad8cfb1 100644 --- a/broker/gen/model/Scan_result.cpp +++ b/broker/gen/model/Scan_result.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Scan_result.h b/broker/gen/model/Scan_result.h index 120688644..f73f062d5 100644 --- a/broker/gen/model/Scan_result.h +++ b/broker/gen/model/Scan_result.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Scan_result_images_inner.cpp b/broker/gen/model/Scan_result_images_inner.cpp index aa503dee4..baa26213b 100644 --- a/broker/gen/model/Scan_result_images_inner.cpp +++ b/broker/gen/model/Scan_result_images_inner.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Scan_result_images_inner.h b/broker/gen/model/Scan_result_images_inner.h index a70a5bd2a..bcbab890b 100644 --- a/broker/gen/model/Scan_result_images_inner.h +++ b/broker/gen/model/Scan_result_images_inner.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Spot_finding_settings.cpp b/broker/gen/model/Spot_finding_settings.cpp index 783032f5e..db0af26ba 100644 --- a/broker/gen/model/Spot_finding_settings.cpp +++ b/broker/gen/model/Spot_finding_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Spot_finding_settings.h b/broker/gen/model/Spot_finding_settings.h index fb040237d..8efe9fd43 100644 --- a/broker/gen/model/Spot_finding_settings.h +++ b/broker/gen/model/Spot_finding_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Standard_detector_geometry.cpp b/broker/gen/model/Standard_detector_geometry.cpp index 3dae5e8d9..6eff003d6 100644 --- a/broker/gen/model/Standard_detector_geometry.cpp +++ b/broker/gen/model/Standard_detector_geometry.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Standard_detector_geometry.h b/broker/gen/model/Standard_detector_geometry.h index b4b3d7af4..1a0e22f49 100644 --- a/broker/gen/model/Standard_detector_geometry.h +++ b/broker/gen/model/Standard_detector_geometry.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Tcp_settings.cpp b/broker/gen/model/Tcp_settings.cpp index 44f4df22b..3bae509db 100644 --- a/broker/gen/model/Tcp_settings.cpp +++ b/broker/gen/model/Tcp_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Tcp_settings.h b/broker/gen/model/Tcp_settings.h index 56e0e2390..b0efae966 100644 --- a/broker/gen/model/Tcp_settings.h +++ b/broker/gen/model/Tcp_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Unit_cell.cpp b/broker/gen/model/Unit_cell.cpp index af1690cd9..a33a96f38 100644 --- a/broker/gen/model/Unit_cell.cpp +++ b/broker/gen/model/Unit_cell.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Unit_cell.h b/broker/gen/model/Unit_cell.h index 6942d16fb..5754c1055 100644 --- a/broker/gen/model/Unit_cell.h +++ b/broker/gen/model/Unit_cell.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Zeromq_metadata_settings.cpp b/broker/gen/model/Zeromq_metadata_settings.cpp index 826482337..2fbcfed44 100644 --- a/broker/gen/model/Zeromq_metadata_settings.cpp +++ b/broker/gen/model/Zeromq_metadata_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Zeromq_metadata_settings.h b/broker/gen/model/Zeromq_metadata_settings.h index 557a25faa..0866df9f4 100644 --- a/broker/gen/model/Zeromq_metadata_settings.h +++ b/broker/gen/model/Zeromq_metadata_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Zeromq_preview_settings.cpp b/broker/gen/model/Zeromq_preview_settings.cpp index c4186b80c..742126cb4 100644 --- a/broker/gen/model/Zeromq_preview_settings.cpp +++ b/broker/gen/model/Zeromq_preview_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Zeromq_preview_settings.h b/broker/gen/model/Zeromq_preview_settings.h index b61358ab4..9528c7d4d 100644 --- a/broker/gen/model/Zeromq_preview_settings.h +++ b/broker/gen/model/Zeromq_preview_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Zeromq_settings.cpp b/broker/gen/model/Zeromq_settings.cpp index 7b5582491..21aae7f6f 100644 --- a/broker/gen/model/Zeromq_settings.cpp +++ b/broker/gen/model/Zeromq_settings.cpp @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/gen/model/Zeromq_settings.h b/broker/gen/model/Zeromq_settings.h index 33608e355..25ae7fc42 100644 --- a/broker/gen/model/Zeromq_settings.h +++ b/broker/gen/model/Zeromq_settings.h @@ -2,7 +2,7 @@ * Jungfraujoch * API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates. # License Clarification While this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply** when this file is used solely to generate OpenAPI clients or when implementing applications that interact with the API. Generated client code and applications using this API definition are not subject to the GPL license requirements and may be distributed under terms of your choosing. This exception is similar in spirit to the Linux Kernel's approach to userspace API headers and the GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that user programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not derivative works of the kernel and are not subject to the terms of the GPL. This exception is intended to allow wider use of this API specification without imposing GPL requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. * -* The version of the OpenAPI document: 1.0.0-rc.173 +* The version of the OpenAPI document: 1.0.0-rc.174 * Contact: filip.leonarski@psi.ch * * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech). diff --git a/broker/jfjoch_api.yaml b/broker/jfjoch_api.yaml index 7dddf08de..f4b4211ed 100644 --- a/broker/jfjoch_api.yaml +++ b/broker/jfjoch_api.yaml @@ -22,7 +22,7 @@ info: requirements on applications that merely interact with the API, regardless of whether they communicate through network calls or other mechanisms. - version: 1.0.0-rc.173 + version: 1.0.0-rc.174 contact: name: Filip Leonarski (Paul Scherrer Institute) email: filip.leonarski@psi.ch diff --git a/broker/redoc-static.html b/broker/redoc-static.html index f7db1597b..532f91fdc 100644 --- a/broker/redoc-static.html +++ b/broker/redoc-static.html @@ -54,7 +54,8 @@ data-styled.g14[id="sc-jlZhew"]{content:"kbdRLs,"}/*!sc*/ .ivsSRP{height:18px;width:18px;min-width:18px;vertical-align:middle;-webkit-transition:-webkit-transform 0.2s ease-out;-webkit-transition:transform 0.2s ease-out;transition:transform 0.2s ease-out;-webkit-transform:rotateZ(-90deg);-ms-transform:rotateZ(-90deg);transform:rotateZ(-90deg);}/*!sc*/ .hWEnWx{height:1.5em;width:1.5em;min-width:1.5em;vertical-align:middle;float:left;-webkit-transition:-webkit-transform 0.2s ease-out;-webkit-transition:transform 0.2s ease-out;transition:transform 0.2s ease-out;-webkit-transform:rotateZ(-90deg);-ms-transform:rotateZ(-90deg);transform:rotateZ(-90deg);}/*!sc*/ .hWEnWx polygon{fill:#1d8127;}/*!sc*/ -data-styled.g15[id="sc-cwHptR"]{content:"cSkZlF,iVFMLx,ivsSRP,hWEnWx,"}/*!sc*/ +.iZRiKW{height:1.3em;width:1.3em;min-width:1.3em;vertical-align:middle;-webkit-transition:-webkit-transform 0.2s ease-out;-webkit-transition:transform 0.2s ease-out;transition:transform 0.2s ease-out;-webkit-transform:rotateZ(-90deg);-ms-transform:rotateZ(-90deg);transform:rotateZ(-90deg);}/*!sc*/ +data-styled.g15[id="sc-cwHptR"]{content:"cSkZlF,iVFMLx,ivsSRP,hWEnWx,iZRiKW,"}/*!sc*/ .hZbWKe{border-left:1px solid #7c7cbb;box-sizing:border-box;position:relative;padding:10px 10px 10px 0;}/*!sc*/ @media screen and (max-width:50rem){.hZbWKe{display:block;overflow:hidden;}}/*!sc*/ tr:first-of-type > .sc-dLMFU,tr.last > .hZbWKe{border-left-width:0;background-position:top left;background-repeat:no-repeat;background-size:1px 100%;}/*!sc*/ @@ -252,6 +253,31 @@ data-styled.g66[id="sc-ddjGPC"]{content:"jZOCAW,"}/*!sc*/ .eMVOei{border-radius:2px;background-color:rgba(104,104,207,0.05);color:rgba(50,50,159,0.9);margin:0 5px;padding:0 5px;border:1px solid rgba(50,50,159,0.1);}/*!sc*/ .sc-fxwrCY + .sc-fxwrCY{margin-left:0;}/*!sc*/ data-styled.g68[id="sc-fxwrCY"]{content:"eMVOei,"}/*!sc*/ +.gZERHX{margin:0 5px;vertical-align:text-top;}/*!sc*/ +data-styled.g75[id="sc-kAkpmW"]{content:"gZERHX,"}/*!sc*/ +.jlNGfN:after{content:' and ';font-weight:normal;}/*!sc*/ +.jlNGfN:last-child:after{content:none;}/*!sc*/ +.jlNGfN a{-webkit-text-decoration:auto;text-decoration:auto;color:#32329f;}/*!sc*/ +.jlNGfN a:visited{color:#32329f;}/*!sc*/ +.jlNGfN a:hover{color:#6868cf;-webkit-text-decoration:auto;text-decoration:auto;}/*!sc*/ +data-styled.g81[id="sc-ihgnxF"]{content:"jlNGfN,"}/*!sc*/ +.bPNrjo{white-space:nowrap;}/*!sc*/ +.bPNrjo:after{content:' or ';white-space:pre;}/*!sc*/ +.bPNrjo:last-child:after,.bPNrjo:only-child:after{content:none;}/*!sc*/ +.bPNrjo a{-webkit-text-decoration:auto;text-decoration:auto;color:#32329f;}/*!sc*/ +.bPNrjo a:visited{color:#32329f;}/*!sc*/ +.bPNrjo a:hover{color:#6868cf;-webkit-text-decoration:auto;text-decoration:auto;}/*!sc*/ +data-styled.g82[id="sc-jMakVo"]{content:"bPNrjo,"}/*!sc*/ +.fkPTul{-webkit-flex:1 1 auto;-ms-flex:1 1 auto;flex:1 1 auto;cursor:pointer;}/*!sc*/ +data-styled.g83[id="sc-iMTnTL"]{content:"fkPTul,"}/*!sc*/ +.gkwgxc{width:75%;text-overflow:ellipsis;border-radius:4px;overflow:hidden;}/*!sc*/ +@media screen and (max-width:50rem){.gkwgxc{margin-top:10px;}}/*!sc*/ +data-styled.g84[id="sc-krNlru"]{content:"gkwgxc,"}/*!sc*/ +.JjoyL{display:inline-block;margin:0;}/*!sc*/ +data-styled.g85[id="sc-hwdzOV"]{content:"JjoyL,"}/*!sc*/ +.eqYzvz{width:100%;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;margin:1em 0;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;}/*!sc*/ +@media screen and (max-width:50rem){.eqYzvz{-webkit-flex-direction:column;-ms-flex-direction:column;flex-direction:column;}}/*!sc*/ +data-styled.g86[id="sc-jaXxmE"]{content:"eqYzvz,"}/*!sc*/ .UQJzT{margin-top:0;margin-bottom:0.5em;}/*!sc*/ data-styled.g92[id="sc-ktJbId"]{content:"UQJzT,"}/*!sc*/ .csEIgT::before{content:'|';display:inline-block;opacity:0.5;width:15px;text-align:center;}/*!sc*/ @@ -399,7 +425,7 @@ This format doesn't transmit information about X-axis, only values, so it i 55.627 l 55.6165,55.627 -231.245496,231.24803 c -127.185,127.1864 -231.5279,231.248 -231.873,231.248 -0.3451,0 -104.688, -104.0616 -231.873,-231.248 z - " fill="currentColor">

Wait for acquisition running

Block execution of external script till detector and Jungfraujoch are ready to collect data. To not block web server for a indefinite period of time, the procedure is provided with a timeout. Extending timeout is possible, but requires to ensure safety that client will not close the connection and retry the connection.

query Parameters
timeout
integer [ 0 .. 3600 ]
Default: 60

Timeout in seconds (0 == immediate response)

@@ -871,10 +904,13 @@ This can only be done when detector is Idle, Error or

Request samples

Content type
application/json
{
  • "box": {
    },
  • "circle": {
    },
  • "azim": {
    }
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get general statistics

Responses

Response samples

Content type
application/json
{
  • "detector": {
    },
  • "detector_list": {
    },
  • "detector_settings": {
    },
  • "image_format_settings": {
    },
  • "instrument_metadata": {
    },
  • "file_writer_settings": {
    },
  • "data_processing_settings": {
    },
  • "measurement": {
    },
  • "broker": {
    },
  • "fpga": [
    ],
  • "calibration": [
    ],
  • "zeromq_preview": {
    },
  • "zeromq_metadata": {
    },
  • "dark_mask": {
    },
  • "pixel_mask": {
    },
  • "roi": {
    },
  • "az_int": {
    },
  • "buffer": {
    },
  • "indexing": {
    },
  • "bragg_integration": {
    },
  • "image_pusher": {
    }
}

Get data collection statistics

Results of the last data collection

+
http://localhost:5232/config/roi

Request samples

Content type
application/json
{
  • "box": {
    },
  • "circle": {
    },
  • "azim": {
    }
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get general statistics

measurement is omitted unless the request carries a valid bearer token for a protected +dataset (see tokens in /start).

Responses

Response samples

Content type
application/json
{
  • "detector": {
    },
  • "detector_list": {
    },
  • "detector_settings": {
    },
  • "image_format_settings": {
    },
  • "instrument_metadata": {
    },
  • "file_writer_settings": {
    },
  • "data_processing_settings": {
    },
  • "measurement": {
    },
  • "broker": {
    },
  • "fpga": [
    ],
  • "calibration": [
    ],
  • "zeromq_preview": {
    },
  • "zeromq_metadata": {
    },
  • "dark_mask": {
    },
  • "pixel_mask": {
    },
  • "roi": {
    },
  • "az_int": {
    },
  • "buffer": {
    },
  • "indexing": {
    },
  • "bragg_integration": {
    },
  • "image_pusher": {
    }
}

Get data collection statistics

Results of the last data collection (dataset name, unit cell, ...)

+
Authorizations:
bearerAuth

Responses

Response samples

Content type
application/json
{
  • "file_prefix": "string",
  • "run_number": 0,
  • "experiment_group": "string",
  • "images_expected": 0,
  • "images_collected": 0,
  • "images_sent": 0,
  • "images_written": 0,
  • "images_discarded_lossy_compression": 0,
  • "max_image_number_sent": 0,
  • "collection_efficiency": 1,
  • "compression_ratio": 5.3,
  • "cancelled": true,
  • "max_receiver_delay": 0,
  • "indexing_rate": 0.1,
  • "detector_width": 0,
  • "detector_height": 0,
  • "detector_pixel_depth": 2,
  • "bkg_estimate": 0.1,
  • "unit_cell": "string",
  • "error_pixels": 0.1,
  • "saturated_pixels": 0.1,
  • "roi_beam_pixels": 0.1,
  • "roi_beam_sum": 0.1
}

Get calibration statistics

Statistics are provided for each module/storage cell separately

@@ -929,7 +965,7 @@ User mask is not automatically applied - i.e. pixels with user mask will have a

Generate 1D plot from Jungfraujoch

query Parameters
binning
integer
Default: 1

Binning of frames for the plot (0 = default binning)

+
http://localhost:5232/preview/pedestal.tiff

Generate 1D plot from Jungfraujoch

Authorizations:
bearerAuth
query Parameters
binning
integer
Default: 1

Binning of frames for the plot (0 = default binning)

type
required
string
Enum: "bkg_estimate" "spindle_blind_fraction" "azint" "azint_1d" "spot_count" "spot_count_low_res" "spot_count_indexed" "spot_count_ice" "indexing_rate" "indexing_lattice_count" "indexing_unit_cell_length" "indexing_unit_cell_angle" "profile_radius" "mosaicity" "b_factor" "error_pixels" "saturated_pixels" "image_collection_efficiency" "receiver_delay" "receiver_free_send_buf" "strong_pixels" "roi_sum" "roi_mean" "roi_max_count" "roi_pixels" "roi_weighted_x" "roi_weighted_y" "packets_received" "max_pixel_value" "resolution_estimate" "pixel_sum" "processing_time" "beam_center_x" "beam_center_y" "integrated_reflections" "image_scale_factor" "image_scale_cc" "compression_ratio" "ice_ring_score"

Type of requested plot

fill
number <float>

Fill value for elements that were missed during data collection

experimental_coord
boolean
Default: false

If measurement has goniometer axis defined, plot X-axis will represent rotation angle @@ -938,31 +974,36 @@ For still measurement the number is ignored

azint_unit
string
Default: "Q_recipA"
Enum: "Q_recipA" "d_A" "two_theta_deg"

Unit used for azim int.

Responses

Response samples

Content type
application/json
{
  • "title": "string",
  • "unit_x": "image_number",
  • "size_x": 0.1,
  • "size_y": 0.1,
  • "plot": [
    ]
}

Generate 1D plot from Jungfraujoch and send in raw binary format. Data are provided as (32-bit) float binary array. This format doesn't transmit information about X-axis, only values, so it is of limited use for azimuthal integration. -

query Parameters
type
required
string
Enum: "bkg_estimate" "spindle_blind_fraction" "azint" "azint_1d" "spot_count" "spot_count_low_res" "spot_count_indexed" "spot_count_ice" "indexing_rate" "indexing_lattice_count" "indexing_unit_cell_length" "indexing_unit_cell_angle" "profile_radius" "mosaicity" "b_factor" "error_pixels" "saturated_pixels" "image_collection_efficiency" "receiver_delay" "receiver_free_send_buf" "strong_pixels" "roi_sum" "roi_mean" "roi_max_count" "roi_pixels" "roi_weighted_x" "roi_weighted_y" "packets_received" "max_pixel_value" "resolution_estimate" "pixel_sum" "processing_time" "beam_center_x" "beam_center_y" "integrated_reflections" "image_scale_factor" "image_scale_cc" "compression_ratio" "ice_ring_score"

Type of requested plot

+
Authorizations:
bearerAuth
query Parameters
type
required
string
Enum: "bkg_estimate" "spindle_blind_fraction" "azint" "azint_1d" "spot_count" "spot_count_low_res" "spot_count_indexed" "spot_count_ice" "indexing_rate" "indexing_lattice_count" "indexing_unit_cell_length" "indexing_unit_cell_angle" "profile_radius" "mosaicity" "b_factor" "error_pixels" "saturated_pixels" "image_collection_efficiency" "receiver_delay" "receiver_free_send_buf" "strong_pixels" "roi_sum" "roi_mean" "roi_max_count" "roi_pixels" "roi_weighted_x" "roi_weighted_y" "packets_received" "max_pixel_value" "resolution_estimate" "pixel_sum" "processing_time" "beam_center_x" "beam_center_y" "integrated_reflections" "image_scale_factor" "image_scale_cc" "compression_ratio" "ice_ring_score"

Type of requested plot

roi
string non-empty

Name of ROI for which plot is requested

Responses

Get full scan result

Responses

Get full scan result

Authorizations:
bearerAuth

Responses

Response samples

Content type
application/json
{
  • "file_prefix": "string",
  • "rotation_unit_cell": {
    },
  • "rotation_crystal_lattice": [
    ],
  • "rotation_bravais": "string",
  • "images": [
    ]
}

Get Start message in CBOR format

Contains metadata for a dataset (e.g., experimental geometry)

-

Responses

Authorizations:
bearerAuth

Responses

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get image message in CBOR format

Contains full image data and metadata. The image must come from the latest data collection.

-
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

+
Authorizations:
bearerAuth
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

Responses

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in JPEG format using custom settings

query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

+
http://localhost:5232/image_buffer/image.cbor

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in JPEG format using custom settings

Authorizations:
bearerAuth
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

show_user_mask
boolean
Default: false

Show user mask

show_roi
boolean
Default: false

Show ROI areas on the image

show_spots
boolean
Default: true

Show spot finding results on the image

@@ -975,11 +1016,13 @@ This format doesn't transmit information about X-axis, only values, so it i
show_res_est
boolean
Default: false

Show resolution estimation as a ring

Responses

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in TIFF format

query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

+
http://localhost:5232/image_buffer/image.jpeg

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in TIFF format

Authorizations:
bearerAuth
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

Responses

Clear image buffer

Turns off image buffer for the last data collection. Can be only run when Jungfraujoch is not collecting data.

@@ -996,7 +1039,7 @@ then image might be replaced in the buffer between calling /images and /image.cb

Jungfraujoch (1.0.0-rc.173)

Download OpenAPI specification:

Filip Leonarski (Paul Scherrer Institute): filip.leonarski@psi.ch License: GPL-3.0

API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). + " fill="currentColor">

Jungfraujoch (1.0.0-rc.174)

Download OpenAPI specification:

Filip Leonarski (Paul Scherrer Institute): filip.leonarski@psi.ch License: GPL-3.0

API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland). Jungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU. Jungfraujoch uses FPGA boards to acquire data at high data rates.

License Clarification

While this API definition is licensed under GPL-3.0, the GPL copyleft provisions do not apply @@ -436,7 +462,9 @@ If storage cells are used, the execution time might be few minutes. Detector must be in Idle state. Default behavior is for the call to block until detector is ready to accept soft/TTL triggers. However, this behavior can be changed by settings async_start to true in the request body, -in which case the call will return immediately and one needs to use /wait_until_running to ensure detector is ready to run.

+in which case the call will return immediately and one needs to use /wait_until_running to ensure detector is ready to run. +Optional tokens protect the dataset: while set, its statistics, buffered images and plots +require Authorization: Bearer <token>.

Request Body schema: application/json
images_per_trigger
integer <int64> >= 1
Default: 1

For standard synchrotron data collection - this is number of images collected per one TTL trigger For XFEL (pulsed source) - this number is ignored and set to 1 For storage cell mode - this number is ignored and set to number of storage cells

@@ -521,11 +549,16 @@ Assuming that Smargon is used as static positioner and not moving during the sca
max_spot_count
integer [ 10 .. 2000 ]
Default: 250

Maximum number of spots that are saved/used for indexing; spots with highest intensity are selected

detect_ice_rings
boolean

Flag spots as ice rings and reduce their effect on indexing

async_start
boolean
Default: false

When set to true, /start will not wait for detector and Jungfraujoch to be ready for the measurement.

+
tokens
Array of strings[ items non-empty ]

Bearer tokens that protect this dataset: any number, all equivalent (e.g. a beamline +secret and a dataset-specific user secret). While set, the endpoints marked with the +bearerAuth security scheme require Authorization: Bearer <token> and answer 401 otherwise. +Every accepted /start replaces the previous dataset's tokens; omitted or empty means no +protection. Never returned by any endpoint.

object

Responses

Request samples

Content type
application/json
{
  • "images_per_trigger": 1,
  • "ntrigger": 1,
  • "image_time_us": 0,
  • "beam_x_pxl": 0.1,
  • "beam_y_pxl": 0.1,
  • "detector_distance_mm": 0.1,
  • "incident_energy_keV": 0.001,
  • "file_prefix": "",
  • "images_per_file": 1,
  • "space_group_number": 1,
  • "sample_name": "",
  • "compression": "bslz4",
  • "total_flux": 0.1,
  • "transmission": 1,
  • "beam_size_x_um": 0.1,
  • "beam_size_y_um": 0.1,
  • "goniometer": {
    },
  • "grid_scan": {
    },
  • "header_appendix": null,
  • "image_appendix": null,
  • "data_reduction_factor_serialmx": 1,
  • "pixel_value_low_threshold": 0,
  • "run_number": 0,
  • "run_name": "string",
  • "experiment_group": "string",
  • "poisson_compression": 16,
  • "write_nxmx_hdf5_master": true,
  • "save_calibration": true,
  • "polarization_factor": -1,
  • "ring_current_mA": 0.1,
  • "sample_temperature_K": 0.1,
  • "poni_rot1_rad": 0,
  • "poni_rot2_rad": 0,
  • "poni_rot3_rad": 0,
  • "unit_cell": {
    },
  • "spot_finding": true,
  • "smargon": {
    },
  • "max_spot_count": 250,
  • "detect_ice_rings": true,
  • "async_start": false,
  • "xray_fluorescence_spectrum": {
    }
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Wait for acquisition running

Block execution of external script till detector and Jungfraujoch are ready to collect data. +

http://localhost:5232/start

Request samples

Content type
application/json
{
  • "images_per_trigger": 1,
  • "ntrigger": 1,
  • "image_time_us": 0,
  • "beam_x_pxl": 0.1,
  • "beam_y_pxl": 0.1,
  • "detector_distance_mm": 0.1,
  • "incident_energy_keV": 0.001,
  • "file_prefix": "",
  • "images_per_file": 1,
  • "space_group_number": 1,
  • "sample_name": "",
  • "compression": "bslz4",
  • "total_flux": 0.1,
  • "transmission": 1,
  • "beam_size_x_um": 0.1,
  • "beam_size_y_um": 0.1,
  • "goniometer": {
    },
  • "grid_scan": {
    },
  • "header_appendix": null,
  • "image_appendix": null,
  • "data_reduction_factor_serialmx": 1,
  • "pixel_value_low_threshold": 0,
  • "run_number": 0,
  • "run_name": "string",
  • "experiment_group": "string",
  • "poisson_compression": 16,
  • "write_nxmx_hdf5_master": true,
  • "save_calibration": true,
  • "polarization_factor": -1,
  • "ring_current_mA": 0.1,
  • "sample_temperature_K": 0.1,
  • "poni_rot1_rad": 0,
  • "poni_rot2_rad": 0,
  • "poni_rot3_rad": 0,
  • "unit_cell": {
    },
  • "spot_finding": true,
  • "smargon": {
    },
  • "max_spot_count": 250,
  • "detect_ice_rings": true,
  • "async_start": false,
  • "tokens": [
    ],
  • "xray_fluorescence_spectrum": {
    }
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}