From a291c01c09f7b0069743fcbbc35ada04a077a0db Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Thu, 8 Oct 2026 10:47:13 +0200 Subject: [PATCH 1/2] rugnux: the geometry pre-pass integrates at least 120 deg of the sweep The two-pass geometry pre-pass integrates the first --prepass-fraction (0.5) of the sweep, but never less than 120 deg of rotation (header oscillation), so a sweep up to 120 deg is pre-passed whole and one up to 240 deg gets 120 deg. Wider sweeps are unchanged. The range is computed in one place, Rugnux::PrepassEnd. Evidence (fraction sweep with the rc175-p1-baseline binary, 10 sweeps of 360 deg and 10 of 180 deg; pre-pass distance vs the full-sweep fit): median error 0.081 % at 90 deg, 0.042 % at 120, 0.042 % at 150 and 0.033 % at 180 on the 180-deg sweeps; 0.032 / 0.021 / 0.013 % at 120 / 150 / 180 deg on the 360-deg sweeps. 120 deg on a wide sweep matches what the whole-sweep pre-pass gave a 180-deg sweep. Below ~60 deg the pre-pass misses the distance outright (6rym: held at the header, 3.4 % off; 6vww and 6fwc: distance not freed). Merged quality moves by under 1 % at any angle >= 90 deg. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi --- docs/CPU_DATA_ANALYSIS_INDEXING.md | 2 +- docs/RUGNUX_ADVANCED.md | 2 +- docs/RUGNUX_OVERVIEW.md | 2 +- docs/RUGNUX_REPORT.md | 2 +- docs/RUGNUX_TUTORIAL.md | 2 +- rugnux/Rugnux.cpp | 17 ++++++++++++++--- rugnux/Rugnux.h | 4 ++++ rugnux/rugnux_cli.cpp | 2 +- 8 files changed, 24 insertions(+), 9 deletions(-) diff --git a/docs/CPU_DATA_ANALYSIS_INDEXING.md b/docs/CPU_DATA_ANALYSIS_INDEXING.md index 899ce9ba6..a4ccf9d8d 100644 --- a/docs/CPU_DATA_ANALYSIS_INDEXING.md +++ b/docs/CPU_DATA_ANALYSIS_INDEXING.md @@ -315,7 +315,7 @@ So every round is scored on the **widest** gate — the population the first pas The refinement above (§7.2) runs per image against that image's spots. For rotation data an additional **post-refinement** (on by default; `--rotation-no-postrefine` disables it) improves the detector distance, beam centre and crystal cell/axis using **all** frames at once, then re-integrates: -1. **Pass 1** integrates, scales and merges at the header geometry — the first half of the sweep by default (`--prepass-fraction`, `1` for all of it). The first-pass indexing has already sampled the whole sweep; what pass 1 adds is integrated reflections for the post-refinement below, and half a sweep holds plenty of them while costing half the time. Its merge, post-refinement and smoothed mosaicity describe a sweep of those frames alone. +1. **Pass 1** integrates, scales and merges at the header geometry — the first half of the sweep by default, but at least 120° of it, so a sweep up to 120° is pre-passed whole (`--prepass-fraction`, `1` for all of it). The first-pass indexing has already sampled the whole sweep; what pass 1 adds is integrated reflections for the post-refinement below, and half a sweep holds plenty of them while costing half the time. Its merge, post-refinement and smoothed mosaicity describe a sweep of those frames alone. 2. From pass-1's integrated reflections, the crystal and the detector are refined together over all its frames (Ceres, robust loss) in **one joint fit**, against both residuals at once: - the **positional** detector↔reciprocal residual at each partial's observed spot, and - a distance-independent **Ewald excitation** residual at each reflection's observed rocking centroid $\phi_\mathrm{obs}$. diff --git a/docs/RUGNUX_ADVANCED.md b/docs/RUGNUX_ADVANCED.md index 4b3956f5c..c67d5ce0e 100644 --- a/docs/RUGNUX_ADVANCED.md +++ b/docs/RUGNUX_ADVANCED.md @@ -342,7 +342,7 @@ For very long axes the direction grid's angular resolution binds well below 500 | `--single-pass-rotation[=num]` | Online-like single-pass rotation indexing (optional min angular range, deg) | | `--force-rotation-lattice ` | Force rotation lattice (9 floats, Å), skipping the first pass | | `--rotation-no-postrefine` | Rotation: disable the default-on two-pass geometry post-refine (see the rotation section) | -| `--prepass-fraction ` | Rotation: the two-pass geometry pre-pass integrates the first fraction `f` of the sweep (0.05-1, default **0.5**; `1` integrates the whole sweep twice). The second pass always integrates every frame | +| `--prepass-fraction ` | Rotation: the two-pass geometry pre-pass integrates the first fraction `f` of the sweep, but at least 120° of it (0.05-1, default **0.5**; `1` integrates the whole sweep twice). The second pass always integrates every frame | | `--refine-geometry[=N\|off]` | Stills: extra first pass that bundle-adjusts the shared beam/distance/cell from N strongly-indexed frames (default 200) then re-indexes; default ON for stills with a reference cell (`-C` / `-z`), `=off` disables | | `--index-ice-rings[=on\|off]` | Index on the spots flagged as sitting on an ice ring too, instead of setting them aside (default: **off**; no effect without `--detect-ice-rings`, which does the flagging) | diff --git a/docs/RUGNUX_OVERVIEW.md b/docs/RUGNUX_OVERVIEW.md index b2276095f..5739cf812 100644 --- a/docs/RUGNUX_OVERVIEW.md +++ b/docs/RUGNUX_OVERVIEW.md @@ -35,7 +35,7 @@ nothing, gives an axis harmonic of the measured centre's lattice, or loses to it are merged ([§1.4](CPU_DATA_ANALYSIS_IMAGE.md)). A failed pass triggers the discrete rescues — the rotation-axis sign, the first-pass depth ladder, the long-axis rescue — before anything is given up on. -**First integration pass.** At the geometry in the file, every frame of the first half of the sweep +**First integration pass.** At the geometry in the file, every frame of the first half of the sweep, at least 120° of it (`--prepass-fraction`) is predicted ([§8](CPU_DATA_ANALYSIS_INTEGRATION.md)) and profile-fit integrated ([§9](CPU_DATA_ANALYSIS_INTEGRATION.md)); partials are combined into fulls, scaled and merged diff --git a/docs/RUGNUX_REPORT.md b/docs/RUGNUX_REPORT.md index 5fdeb5899..ffaacd498 100644 --- a/docs/RUGNUX_REPORT.md +++ b/docs/RUGNUX_REPORT.md @@ -129,7 +129,7 @@ grep '^JFJOCH_DATASET_SETTINGS=' out_report.txt | cut -d' ' -f2- > geometry.json ``` **Which pass.** A rotation run integrates twice — once at the geometry in the input file (the first -half of the sweep, `--prepass-fraction`), then the whole sweep again at the post-refined geometry — and can integrate a third time if a guard rejects the second pass. +half of the sweep but at least 120°, `--prepass-fraction`), then the whole sweep again at the post-refined geometry — and can integrate a third time if a guard rejects the second pass. There is **one** report, for the pass that became the canonical output, and `PASS=` / `PASS_DECISION=` (`--developer`) in section 1 say which pass that is and on what evidence, so no number in the file is ambiguous about which geometry produced it. diff --git a/docs/RUGNUX_TUTORIAL.md b/docs/RUGNUX_TUTORIAL.md index 1dc615133..213fe8db5 100644 --- a/docs/RUGNUX_TUTORIAL.md +++ b/docs/RUGNUX_TUTORIAL.md @@ -109,7 +109,7 @@ resolution) sharpens both the space-group search and the error model. To tune th treated as independent stills use `--force-still`. By default a rotation run also **post-refines the geometry** in a second pass: the first pass -integrates and merges the first half of the sweep at the header geometry (`--prepass-fraction 1` for all of it), then the detector distance + beam centre and the crystal +integrates and merges the first half of the sweep (at least 120° of it) at the header geometry (`--prepass-fraction 1` for all of it), then the detector distance + beam centre and the crystal cell / rotation-axis are refined against the merged fulls (cross-validated; the distance moves 1 % a step and a larger move is ratified by re-indexing, and the gauge-weak beam centre stays within 15 px of the centre the first pass ran at or of the run's own measured centre, whichever is nearer), and the second pass diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 945ec8fed..135209be2 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -133,6 +133,9 @@ namespace { // Images the pre-scan reads when the beam-stop pre-pass is off and only the spot width is wanted; // with it on, that pre-pass's image count is used and one sample serves both. constexpr int BEAM_CENTER_PROJECTION_IMAGES = 60; + + // The least rotation the two-pass geometry pre-pass integrates, whatever --prepass-fraction asks. + constexpr double PREPASS_MIN_ROTATION_DEG = 120.0; } namespace rugnux_internal { @@ -536,6 +539,16 @@ std::vector Rugnux::FirstPassInputKey() const { return k; } +int Rugnux::PrepassEnd(int start_image, int end_image) const { + int end = start_image + static_cast(std::lround((end_image - start_image) * config_.prepass_fraction)); + // At least PREPASS_MIN_ROTATION_DEG of rotation, so a narrow sweep is pre-passed whole. The angle is + // the file's, so every pass of a run picks the same frames, before and after a rotation-scale fit. + if (const auto g = reader_.GetDataset()->experiment.GetGoniometer(); g && g->GetIncrement_deg() != 0.0f) + end = std::max(end, start_image + static_cast( + std::lround(PREPASS_MIN_ROTATION_DEG / std::abs(g->GetIncrement_deg())))); + return std::min(end, end_image); +} + ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, bool geometry_prepass) { Logger logger("Rugnux"); ProcessResult result; @@ -815,9 +828,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const auto start_time = std::chrono::steady_clock::now(); - // The geometry pre-pass integrates only the first prepass_fraction of the sweep (pass 2 reads it all). - const int prepass_end = start_image - + static_cast(std::lround((end_image - start_image) * config_.prepass_fraction)); + const int prepass_end = PrepassEnd(start_image, end_image); const int prepass_images = (prepass_end - start_image + config_.stride - 1) / config_.stride; const int loop_end = geometry_prepass ? prepass_end : end_image; diff --git a/rugnux/Rugnux.h b/rugnux/Rugnux.h index d9c261f95..7105b868e 100644 --- a/rugnux/Rugnux.h +++ b/rugnux/Rugnux.h @@ -845,6 +845,10 @@ class Rugnux { // geometry_prepass == true stops after post-refining the detector geometry from the first integration // and applies it (+ the refined lattice) to experiment_, so the next pass re-integrates with it. ProcessResult RunPipeline(RugnuxObserver *observer, bool write_output, bool geometry_prepass); + // The end of the first part of a rotation sweep, [start_image, PrepassEnd), that the geometry + // pre-pass integrates: the first config_.prepass_fraction of it, but at least + // PREPASS_MIN_ROTATION_DEG (Rugnux.cpp). + [[nodiscard]] int PrepassEnd(int start_image, int end_image) const; // The phases of RunPipeline that live in files of their own; each reads and writes RunPipeline's // locals through `p` (RugnuxPipeline.h). // The per-image loop over the sweep (azimuthal integration, or the full per-image analysis), what is diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp index 962c17e45..d54535561 100644 --- a/rugnux/rugnux_cli.cpp +++ b/rugnux/rugnux_cli.cpp @@ -148,7 +148,7 @@ void print_usage() { std::cout << " --force-rotation-lattice Force rotation indexer with external lattice (in Angstrom) : \"a0x,a0y,a0z,a1x,a1y,a1z,a2x,a2y,a2z\" (9 floats, skips first pass)" << std::endl; std::cout << " --rotation-scale Goniometer rotation scale: the stage turned k times the angle stored in the file (the commanded one). Applied to both passes; overrides the fitted correction" << std::endl; std::cout << " --rotation-no-postrefine Disable the (default-on) two-pass rotation post-refine (post-refine detector distance/beam + cell/axis, then re-integrate; only the refined pass is written, as the canonical _* output)" << std::endl; - std::cout << " --prepass-fraction Integrate the first fraction f of the sweep in the two-pass rotation geometry pre-pass (0.05-1, default 0.5; 1 = the whole sweep); pass 2 reads every frame" << std::endl; + std::cout << " --prepass-fraction Integrate the first fraction f of the sweep, but at least 120 deg of it, in the two-pass rotation geometry pre-pass (0.05-1, default 0.5; 1 = the whole sweep); pass 2 reads every frame" << std::endl; std::cout << " -X, --indexing-algorithm Indexing algorithm (FFBIDX|FFT|FFTW|Auto|None)" << std::endl; std::cout << " -S, --space-group Space group number (96) or symbol (P43212) - for indexing and scaling" << std::endl; std::cout << " -C, --unit-cell Fix reference unit cell: \"a,b,c,alpha,beta,gamma\"" << std::endl; From 5b9144410218934a22ce30ae816d8f5887089956 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Thu, 8 Oct 2026 09:37:02 +0200 Subject: [PATCH 2/2] rugnux: take the decisions before pass 2 from the first part of a rotation sweep The pre-scan sample (beam-stop shadow, defective pixels, spot width, the background beam centre) and the first pass's frames (the spread and wedge schemes, the validation frames, the prefetch) now come from [start, PrepassEnd) - the part of the sweep the geometry pre-pass integrates - instead of the whole sweep. Rugnux::PrepassEnd is the one place that range is defined, for the pre-pass and for these decisions alike, so any change to the pre-pass range (such as a minimum rotation) moves both. Stills keep the whole range. With rugnux following a collection while it is written, nothing before pass 2 then waits for the second part of the sweep: the pre-scan, both first passes and the pre-pass run while it is collected, and pass 2 starts reading as soon as they are done. An offline run takes the same decisions from the same frames, so the streamed result still equals the offline one by md5 (in-house JUNGFRAU 4M thaumatin replayed at one data file per second, also at -N 8 and with the replay paused and resumed): 2.4 s from the last data file to the result, against 8.0 s for the offline run. Result-changing. p.mtz md5 at af3ea9cca -> here (with the 120 deg minimum of the previous commit), GPU: myob de2ab820 -> 9bdd6916, cytc 3e5d2f91 -> 62e744a1, thau bc5e3de3 -> 3ec88c34, kdp cf83ee5d -> cdc2c068, 8a1a 50ab659e -> 309ea9a8, 8qaw d3287922 -> c2849d52; CPU: myob 3f9d12d8 -> 18c7ac2f, cytc d30c9969 -> 40e17d97, thau 4cdbdd94 -> f14e52da, kdp f1d801f3 -> eb930714. Sweeps of 120 deg or less are unchanged by this commit (their early range is the whole sweep). Needs the battery before it is merged. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi --- docs/CPU_DATA_ANALYSIS_INDEXING.md | 4 ++-- docs/RUGNUX_ADVANCED.md | 2 +- docs/RUGNUX_OVERVIEW.md | 2 +- rugnux/Rugnux.cpp | 10 +++++++--- rugnux/Rugnux.h | 11 +++++++---- rugnux/RugnuxFirstPass.cpp | 15 ++++++++------- rugnux/RugnuxPreScan.cpp | 9 +++++---- rugnux/rugnux_cli.cpp | 2 +- 8 files changed, 32 insertions(+), 23 deletions(-) diff --git a/docs/CPU_DATA_ANALYSIS_INDEXING.md b/docs/CPU_DATA_ANALYSIS_INDEXING.md index a4ccf9d8d..4fa756e0d 100644 --- a/docs/CPU_DATA_ANALYSIS_INDEXING.md +++ b/docs/CPU_DATA_ANALYSIS_INDEXING.md @@ -212,7 +212,7 @@ The lookup can also land *short*: near the Niggli type-I/type-II boundary the ch The rotation first pass refines the **twelve** best candidate lattices rather than four. The pre-refinement indexed fraction is an unreliable ranking, so a correct cell can sit below several degenerate ones and never be refined at all. -For rotation data the first pass therefore refines the constrained cell *and* an unconstrained (triclinic) one from the same spots — which it finds itself, over a sample spread across the sweep, rather than reading what the acquisition wrote — and settles the two on how many of a fixed set of validation frames each actually indexes. The bar is a clear majority, not a margin. An unconstrained refinement holds no cell parameter fixed, so it can only index at least as many frames as the constrained one, and on genuine symmetry it indexes a few more — a percentage margin therefore demotes real lattices (measured: a genuine $I$-centred orthorhombic to $P1$). Only a constrained cell that fails outright while its unconstrained cell works distinguishes a false promotion. The intensities settle the space group later regardless (§13). +For rotation data the first pass therefore refines the constrained cell *and* an unconstrained (triclinic) one from the same spots — which it finds itself, over a sample spread across the first part of the sweep (the part the geometry pre-pass integrates), rather than reading what the acquisition wrote — and settles the two on how many of a fixed set of validation frames each actually indexes. The bar is a clear majority, not a margin. An unconstrained refinement holds no cell parameter fixed, so it can only index at least as many frames as the constrained one, and on genuine symmetry it indexes a few more — a percentage margin therefore demotes real lattices (measured: a genuine $I$-centred orthorhombic to $P1$). Only a constrained cell that fails outright while its unconstrained cell works distinguishes a false promotion. The intensities settle the space group later regardless (§13). Two further hypotheses are weighed at the same point, both by default. Where two first-pass answers have primitive volumes in a small integer ratio (2–4×) and tie on the validation frames, the frame @@ -315,7 +315,7 @@ So every round is scored on the **widest** gate — the population the first pas The refinement above (§7.2) runs per image against that image's spots. For rotation data an additional **post-refinement** (on by default; `--rotation-no-postrefine` disables it) improves the detector distance, beam centre and crystal cell/axis using **all** frames at once, then re-integrates: -1. **Pass 1** integrates, scales and merges at the header geometry — the first half of the sweep by default, but at least 120° of it, so a sweep up to 120° is pre-passed whole (`--prepass-fraction`, `1` for all of it). The first-pass indexing has already sampled the whole sweep; what pass 1 adds is integrated reflections for the post-refinement below, and half a sweep holds plenty of them while costing half the time. Its merge, post-refinement and smoothed mosaicity describe a sweep of those frames alone. +1. **Pass 1** integrates, scales and merges at the header geometry — the first half of the sweep by default, but at least 120° of it, so a sweep up to 120° is pre-passed whole (`--prepass-fraction`, `1` for all of it). The first-pass indexing and the pre-scan have sampled the same part of the sweep, so nothing before pass 2 reads the rest of it - which is what lets a run that follows a collection still being written do all of this while the rest is collected; what pass 1 adds is integrated reflections for the post-refinement below, and half a sweep holds plenty of them while costing half the time. Its merge, post-refinement and smoothed mosaicity describe a sweep of those frames alone. 2. From pass-1's integrated reflections, the crystal and the detector are refined together over all its frames (Ceres, robust loss) in **one joint fit**, against both residuals at once: - the **positional** detector↔reciprocal residual at each partial's observed spot, and - a distance-independent **Ewald excitation** residual at each reflection's observed rocking centroid $\phi_\mathrm{obs}$. diff --git a/docs/RUGNUX_ADVANCED.md b/docs/RUGNUX_ADVANCED.md index c67d5ce0e..2a42120c3 100644 --- a/docs/RUGNUX_ADVANCED.md +++ b/docs/RUGNUX_ADVANCED.md @@ -342,7 +342,7 @@ For very long axes the direction grid's angular resolution binds well below 500 | `--single-pass-rotation[=num]` | Online-like single-pass rotation indexing (optional min angular range, deg) | | `--force-rotation-lattice ` | Force rotation lattice (9 floats, Å), skipping the first pass | | `--rotation-no-postrefine` | Rotation: disable the default-on two-pass geometry post-refine (see the rotation section) | -| `--prepass-fraction ` | Rotation: the two-pass geometry pre-pass integrates the first fraction `f` of the sweep, but at least 120° of it (0.05-1, default **0.5**; `1` integrates the whole sweep twice). The second pass always integrates every frame | +| `--prepass-fraction ` | Rotation: the two-pass geometry pre-pass integrates the first fraction `f` of the sweep, but at least 120° of it (0.05-1, default **0.5**; `1` integrates the whole sweep twice). The second pass always integrates every frame. The pre-scan and the first-pass indexing sample the same part of the sweep | | `--refine-geometry[=N\|off]` | Stills: extra first pass that bundle-adjusts the shared beam/distance/cell from N strongly-indexed frames (default 200) then re-indexes; default ON for stills with a reference cell (`-C` / `-z`), `=off` disables | | `--index-ice-rings[=on\|off]` | Index on the spots flagged as sitting on an ice ring too, instead of setting them aside (default: **off**; no effect without `--detect-ice-rings`, which does the flagging) | diff --git a/docs/RUGNUX_OVERVIEW.md b/docs/RUGNUX_OVERVIEW.md index 5739cf812..3feeed413 100644 --- a/docs/RUGNUX_OVERVIEW.md +++ b/docs/RUGNUX_OVERVIEW.md @@ -8,7 +8,7 @@ rotation run with the defaults; stills differences are at the end. ([What Rugnux reads](RUGNUX_FORMATS.md)). A goniometer axis makes it a rotation run, none makes it serial stills — nothing is asked of the user. -**Pre-scan.** A projection of frames spread over the sweep (60 by default) finds the beam-stop shadow and masks +**Pre-scan.** A projection of frames (60 by default) spread over the first part of the sweep, the part the geometry pre-pass integrates, finds the beam-stop shadow and masks it ([§1.5](CPU_DATA_ANALYSIS_IMAGE.md)), masks pixels its frames show to be defective ([§1.6](CPU_DATA_ANALYSIS_IMAGE.md)), measures the beam centre from the isotropy of the scattered background and compares it with the file's ([§1.4](CPU_DATA_ANALYSIS_IMAGE.md)), and diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 135209be2..cbe0cc636 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -636,10 +636,16 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b if (full && config_.refine_geometry.has_value()) RefineStillsGeometry(start_image, end_image, images_to_process, observer); + // The geometry pre-pass integrates only the first part of the sweep (pass 2 reads it all), and on a + // rotation sweep the decisions before pass 2 read only that part too (PrepassEnd). + const int prepass_end = PrepassEnd(start_image, end_image); + const int prepass_images = (prepass_end - start_image + config_.stride - 1) / config_.stride; + const int early_images = experiment_.IsRotationIndexing() ? prepass_images : images_to_process; + // After any geometry refinement, so the shadow is found about the beam centre actually used, // and before the azimuthal mapping and the output mask below, which both read pixel_mask_. if (config_.detect_beam_stop.has_value() || config_.adaptive_integration_radius) - PreScan(start_image, images_to_process, + PreScan(start_image, early_images, std::min(config_.detect_beam_stop.value_or(BEAM_CENTER_PROJECTION_IMAGES), HotPixelFinder::MAX_SAMPLED_FRAMES), observer); @@ -828,8 +834,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const auto start_time = std::chrono::steady_clock::now(); - const int prepass_end = PrepassEnd(start_image, end_image); - const int prepass_images = (prepass_end - start_image + config_.stride - 1) / config_.stride; const int loop_end = geometry_prepass ? prepass_end : end_image; // Declared here rather than where they are first filled, so that the phases below can share them diff --git a/rugnux/Rugnux.h b/rugnux/Rugnux.h index 7105b868e..ac1abb03f 100644 --- a/rugnux/Rugnux.h +++ b/rugnux/Rugnux.h @@ -815,6 +815,13 @@ class Rugnux { // the defective pixels, and the recorded spot width that sets the integration radius // (config_.adaptive_integration_radius). void PreScan(int start_image, int images_to_process, int frame_count, RugnuxObserver *observer); + // The first part of a rotation sweep, [start_image, PrepassEnd): what the geometry pre-pass + // integrates, and the only frames the decisions taken before the canonical pass read - the + // pre-scan sample (mask, spot width, background beam centre) and the first pass's schemes and + // validation frames. So those decisions are made from frames that exist while the rest of the + // sweep is still being collected, and an offline run makes them from the same frames. It is the + // first config_.prepass_fraction of the sweep, but at least PREPASS_MIN_ROTATION_DEG (Rugnux.cpp). + [[nodiscard]] int PrepassEnd(int start_image, int end_image) const; // The pre-scan's defective-pixel mask (HotPixelFinder), read on `sample` with the rings drawn // about `ring_centre` where the background measured one. void MaskDefectivePixels(int start_image, const std::vector &sample, @@ -845,10 +852,6 @@ class Rugnux { // geometry_prepass == true stops after post-refining the detector geometry from the first integration // and applies it (+ the refined lattice) to experiment_, so the next pass re-integrates with it. ProcessResult RunPipeline(RugnuxObserver *observer, bool write_output, bool geometry_prepass); - // The end of the first part of a rotation sweep, [start_image, PrepassEnd), that the geometry - // pre-pass integrates: the first config_.prepass_fraction of it, but at least - // PREPASS_MIN_ROTATION_DEG (Rugnux.cpp). - [[nodiscard]] int PrepassEnd(int start_image, int end_image) const; // The phases of RunPipeline that live in files of their own; each reads and writes RunPipeline's // locals through `p` (RugnuxPipeline.h). // The per-image loop over the sweep (azimuthal integration, or the full per-image analysis), what is diff --git a/rugnux/RugnuxFirstPass.cpp b/rugnux/RugnuxFirstPass.cpp index 7d1048a31..2547fbbf5 100644 --- a/rugnux/RugnuxFirstPass.cpp +++ b/rugnux/RugnuxFirstPass.cpp @@ -195,7 +195,8 @@ bool Rugnux::FirstPassRotationIndexing(PipelineLocals &p) { auto &observer = p.observer; auto &geometry_prepass = p.geometry_prepass; auto &start_image = p.start_image; - auto &images_to_process = p.images_to_process; + // Every frame the first pass reads is in the first part of the sweep (Rugnux::PrepassEnd). + const int early_images = p.prepass_images; auto &dataset = p.dataset; auto &mapping = p.mapping; auto &indexer_pool = p.indexer_pool; @@ -420,7 +421,7 @@ bool Rugnux::FirstPassRotationIndexing(PipelineLocals &p) { // sub-lattice from an under-sampled scheme) indexes few frames, while the true cell indexes // many - this is the discriminator that lets the best scheme win. const std::vector validation = select_equally_spaced_image_ordinals( - images_to_process, std::min(images_to_process, 60)); + early_images, std::min(early_images, 60)); const float index_tol = experiment_.GetIndexingSettings().GetTolerance(); @@ -600,16 +601,16 @@ bool Rugnux::FirstPassRotationIndexing(PipelineLocals &p) { // Sampling schemes, tried in order of preference. We index each and keep the one that indexes // the most frames; the schemes cover complementary failure modes and no single one is best for // every crystal: - // - "spread": images spread over the whole rotation - full angular range, coarse stride; the - // long-standing default, best for typical crystals. + // - "spread": images spread over the first part of the rotation (PrepassEnd) - wide angular range, + // coarse stride; the long-standing default, best for typical crystals. // - "wedge": consecutive frames from the clean start - native stride keeps each reflection's // rocking curve continuous across frames, so the FFT can trace a long axis whose fine // reciprocal spacing the coarse spread cannot resolve (e.g. an F-cubic cell viewed down // [111]); bounded by the accumulation cap to the least radiation-damaged early wedge. - std::vector all_consecutive(images_to_process); + std::vector all_consecutive(early_images); std::iota(all_consecutive.begin(), all_consecutive.end(), 0); const std::vector spread_ordinals = select_equally_spaced_image_ordinals( - images_to_process, config_.rotation_indexing_image_count); + early_images, config_.rotation_indexing_image_count); const std::vector>> schemes = { {"spread", spread_ordinals}, {"wedge", all_consecutive}, @@ -1129,7 +1130,7 @@ bool Rugnux::FirstPassRotationIndexing(PipelineLocals &p) { { std::vector upfront = spread_ordinals; upfront.insert(upfront.end(), validation.begin(), validation.end()); - for (int i = 0; i < std::min(SPOT_PREFETCH_CHUNK, images_to_process); i++) + for (int i = 0; i < std::min(SPOT_PREFETCH_CHUNK, early_images); i++) upfront.push_back(i); std::sort(upfront.begin(), upfront.end()); upfront.erase(std::unique(upfront.begin(), upfront.end()), upfront.end()); diff --git a/rugnux/RugnuxPreScan.cpp b/rugnux/RugnuxPreScan.cpp index 12f9cccd3..30bc7ae46 100644 --- a/rugnux/RugnuxPreScan.cpp +++ b/rugnux/RugnuxPreScan.cpp @@ -170,10 +170,11 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru if (!want_shadow && !want_width && !want_hot) return; - // The sample is taken from the middle of the sweep, both ends left out: an even spread over - // [0, images_to_process) always picks the very first and the very last image, and those are the - // ones a shutter out of step spoils - the last carries under 90 % of the sample's median counts - // on 12 of 38 regression sweeps and is essentially unexposed on 5. + // The sample is taken from the frames given - on a rotation sweep its first part (PrepassEnd) - + // both ends left out: an even spread over [0, images_to_process) always picks the very first and + // the very last image, and those are the ones a shutter out of step spoils - the last carries + // under 90 % of the sample's median counts on 12 of 38 regression sweeps and is essentially + // unexposed on 5. const int margin = std::min(PRESCAN_END_MARGIN_IMAGES, images_to_process / 10); auto shadow_sample = select_equally_spaced_image_ordinals(images_to_process - 2 * margin, frame_count); diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp index d54535561..90dab4842 100644 --- a/rugnux/rugnux_cli.cpp +++ b/rugnux/rugnux_cli.cpp @@ -148,7 +148,7 @@ void print_usage() { std::cout << " --force-rotation-lattice Force rotation indexer with external lattice (in Angstrom) : \"a0x,a0y,a0z,a1x,a1y,a1z,a2x,a2y,a2z\" (9 floats, skips first pass)" << std::endl; std::cout << " --rotation-scale Goniometer rotation scale: the stage turned k times the angle stored in the file (the commanded one). Applied to both passes; overrides the fitted correction" << std::endl; std::cout << " --rotation-no-postrefine Disable the (default-on) two-pass rotation post-refine (post-refine detector distance/beam + cell/axis, then re-integrate; only the refined pass is written, as the canonical _* output)" << std::endl; - std::cout << " --prepass-fraction Integrate the first fraction f of the sweep, but at least 120 deg of it, in the two-pass rotation geometry pre-pass (0.05-1, default 0.5; 1 = the whole sweep); pass 2 reads every frame" << std::endl; + std::cout << " --prepass-fraction Integrate the first fraction f of the sweep, but at least 120 deg of it, in the two-pass rotation geometry pre-pass (0.05-1, default 0.5; 1 = the whole sweep); pass 2 reads every frame. The pre-scan and the first-pass indexing sample the same part of the sweep" << std::endl; std::cout << " -X, --indexing-algorithm Indexing algorithm (FFBIDX|FFT|FFTW|Auto|None)" << std::endl; std::cout << " -S, --space-group Space group number (96) or symbol (P43212) - for indexing and scaling" << std::endl; std::cout << " -C, --unit-cell Fix reference unit cell: \"a,b,c,alpha,beta,gamma\"" << std::endl;