Files
Jungfraujoch/image_analysis/bragg_prediction/BraggPrediction.cpp
T
leonarski_fandClaude Opus 5 16bf3408f0 Address code-review findings; make detection limits detector-driven
One changeset, developed together in response to a review of this branch, so the
files carry several of the changes at once. Full test suite passes (733 cases).

Spot finding
- Split ImageSpotFinder into Detect() (flag strong pixels - the expensive
  per-pixel pass) and ExtractSpots() (CCL + min/max-pix + resolution mask), with
  Run() = both. The per-image min-pix escalation now detects ONCE and repeats
  only the cheap extraction, instead of re-running the whole finder four times
  per frame as it did on the default path. It also keeps the winning attempt's
  spot list rather than re-extracting it, so the frame that is integrated is
  exactly the frame that was scored - which a GPU re-extract could not guarantee
  (float atomic ordering).
- spot_finding_time_s no longer swallows indexing time, and indexing_time_s now
  sums every escalation call instead of reporting only the last.

Detection limits follow the detector
- The azimuthal-integration upper q and the spot-finding high-resolution limit
  are now std::optional, in the C++ structs AND in the OpenAPI schema, and
  resolve to the detector's own maximum (DiffractionExperiment::GetDetectorMaxQ_
  recipA). Adaptive detection reads a pixel's ring from the azimuthal bins, so a
  pixel outside that q range could never be strong - the integration range
  silently bounded what detection could see, regardless of the requested
  resolution limit. Regenerated the C++ and TypeScript clients; the viewer and
  the web frontend each gained a "to detector edge" switch.

Detection defaults are now per workflow (measured, not assumed)
- Stills: adaptive detection, min-pix chosen per image, no resolution clipping.
- Rotation: fixed-threshold finder, min-pix 2, 1.5 A limit.
  On a 33-crystal rotation battery, adaptive detection helped four hard crystals
  but deterministically broke three (a lost space group, a halved indexing rate,
  a collapsed merge), and the detector-edge limit cost indexing on a strong
  rotation set (100.0 -> 96.8%). Each is still overridable by its flag, and
  --no-adaptive-spots is new.

Indexer seed escalation
- Stop escalating once a seed's lattice explains >= 90% of the seed spots.
  Previously any frame with >= 80 spots always paid three indexer calls, online
  broker included.

Merge-consistency filter
- --min-image-cc gated on a per-image CC computed BEFORE the stills partiality
  post-refinement and never refreshed; the refiner now recomputes it, so the
  reported CC describes the data that are actually merged.
- Replaced the per-call cc_mask argument with one MergeOnTheFly flag, so the
  merge, the error model and MergeStats can no longer disagree about which
  images are in (the --scale path merged unfiltered while its statistics were
  filtered).

Per-image B-factor refinement (-B) removed
- Measured on four serial-stills datasets: it is a no-op where the per-image fit
  is well conditioned and actively harmful where it is not (CC1/2 -8.1, R_meas
  +23.2 on the weakest large-cell set, whose fits hit their [-50, 200] bounds on
  14-25% of images). It had also been silently DISCARDED since the partiality
  post-refinement landed - reported but not applied. Rather than fix and keep a
  knob with no demonstrated benefit, the flag and the whole image_scale_b_factor
  chain are gone: setting, scaling fit, message field, CBOR, HDF5 write and
  read-back, per-image plot, OpenAPI enum, viewer column and checkbox, docs.
  ScaleOnTheFly no longer needs Ceres at all - the fit is a linear IRLS.
  (The Wilson per-image b_factor is a different quantity and stays.)

Stills partiality width now fits both of its components
- sigma^2 = gamma0^2 + (gamma_e*d*)^2 instead of a purely angular gamma_e*d*
  with gamma0 pinned to 0. Fitted per crystal by least squares of dist_ewald^2
  on d*^2. The angular-only width is fitted over a d*^2-dense population, so it
  was pinned by the high-resolution edge and collapsed at low d*: median
  partiality 0.008 beyond 13 A for reflections that were plainly recorded, 55%
  of them under the merge's partiality floor, and the survivors divided by those
  values - which inflated the merged low-resolution intensity scale 3.6x
  (~ +9 A^2 of apparent B). Measured on 5000 stills: the ramp flattens to 0.89x,
  no observation is dropped any more (701750 -> 716811), shell-mean CC1/2 and
  R-free improve slightly. Note CC1/2, R_meas, completeness and a B-refining
  R-free are all blind to that ramp, which is why it survived earlier validation;
  the cost is high-resolution R_meas (98.5 -> 101.9 shell-averaged).

Removed dead code from add-then-remove churn
- Prediction-time "still partiality" (unreachable: no setter), the phantom
  IndexingSettings::min_indexed_spot_fraction knob (getter, no setter - now the
  constant it always was), StillsPartialityRefine's caller-less Settings
  constructor and its reference to a long-gone env var, ProcessImage's unread
  bool return, an unused include, and a dead viewer overlay hook.

Also
- Viewer: the magnifier compared a QImage with itself, so its scene rect was set
  once ever and it could not pan into a larger dataset; the hover tail timer
  could fire after leaveEvent and resurrect the resolution readout outside the
  image.
- update_version.sh regenerated the frontend lock file BEFORE bumping the
  version (every release shipped an off-by-one lock), and did git rm/git add on
  a path that has not existed since the client moved to src/client - with no
  set -e, both failed silently.
- fpga/pcie_driver/postinstall.sh tested "[ ! occurrences > 0 ]", which is a
  redirect, not a test, so dkms add never ran.
- Unit tests for the adaptive-threshold host functions, which had none.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 09:07:00 +02:00

182 lines
7.7 KiB
C++
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <algorithm>
#include "../../common/JFJochMath.h"
#include "BraggPrediction.h"
#include "../bragg_integration/SystematicAbsence.h"
int BraggPrediction::TruncateToOutput(int count) {
if (count <= kPredictionOutput)
return count;
std::partial_sort(reflections.begin(), reflections.begin() + kPredictionOutput,
reflections.begin() + count,
[](const Reflection &a, const Reflection &b) {
if (a.dist_ewald != b.dist_ewald) return a.dist_ewald < b.dist_ewald;
if (a.h != b.h) return a.h < b.h;
if (a.k != b.k) return a.k < b.k;
return a.l < b.l;
});
return kPredictionOutput;
}
namespace {
// Number of bandwidth sigmas included in the (radially thickened) Ewald-shell
// acceptance window. 3σ captures essentially the whole pink-beam smear; matches
// the conservative end of the mosaicity cutoff used by callers.
constexpr float kBandwidthCutoffSigmas = 3.0f;
}
BraggPrediction::BraggPrediction(int max_reflections)
: max_reflections(max_reflections), reflections(max_reflections) {}
const std::vector<Reflection> &BraggPrediction::GetReflections() const {
return reflections;
}
int BraggPrediction::Calc(const DiffractionExperiment &experiment, const CrystalLattice &lattice,
const BraggPredictionSettings &settings) {
const auto geom = experiment.GetDiffractionGeometry();
const auto det_width_pxl = static_cast<float>(experiment.GetXPixelsNum());
const auto det_height_pxl = static_cast<float>(experiment.GetYPixelsNum());
const float one_over_dmax = 1.0f / settings.high_res_A;
const float one_over_dmax_sq = one_over_dmax * one_over_dmax;
float one_over_wavelength = 1.0f / geom.GetWavelength_A();
const Coord Astar = lattice.Astar();
const Coord Bstar = lattice.Bstar();
const Coord Cstar = lattice.Cstar();
const Coord S0 = geom.GetScatteringVector();
std::vector<float> rot = geom.GetPoniRotMatrix().transpose().arr();
// Precompute detector geometry constants
float beam_x = geom.GetBeamX_pxl();
float beam_y = geom.GetBeamY_pxl();
float det_distance = geom.GetDetectorDistance_mm();
float pixel_size = geom.GetPixelSize_mm();
float F = det_distance / pixel_size;
const float epsilon = 1e-5f;
const float s0_sq = S0 * S0;
const float rad_to_deg = 180.0f / static_cast<float>(PI);
int i = 0;
for (int h = -settings.max_hkl; h <= settings.max_hkl; h++) {
// Precompute A* h contribution
const float Ah_x = Astar.x * h;
const float Ah_y = Astar.y * h;
const float Ah_z = Astar.z * h;
for (int k = -settings.max_hkl; k <= settings.max_hkl; k++) {
// Accumulate B* k contribution
const float AhBk_x = Ah_x + Bstar.x * k;
const float AhBk_y = Ah_y + Bstar.y * k;
const float AhBk_z = Ah_z + Bstar.z * k;
for (int l = -settings.max_hkl; l <= settings.max_hkl; l++) {
if (systematic_absence(h, k, l, settings.centering))
continue;
if (i >= max_reflections)
continue;
float recip_x = AhBk_x + Cstar.x * l;
float recip_y = AhBk_y + Cstar.y * l;
float recip_z = AhBk_z + Cstar.z * l;
float recip_sq = recip_x * recip_x + recip_y * recip_y + recip_z * recip_z;
if (recip_sq > one_over_dmax_sq)
continue;
float S_x = recip_x + S0.x;
float S_y = recip_y + S0.y;
float S_z = recip_z + S0.z;
float S_len = sqrtf(S_x * S_x + S_y * S_y + S_z * S_z);
float dist_ewald_sphere = std::fabs(S_len - one_over_wavelength);
// Energy bandwidth thickens the Ewald shell radially: at the
// diffraction condition |S|-1/λ shifts by recip_z·(Δλ/λ), i.e.
// σ_bw = |recip_z|·bandwidth_sigma (= bλ/2d²). Broaden the acceptance
// window in quadrature so high-resolution shells (smeared most, ∝1/d²)
// are not clipped.
float radial_cutoff = settings.ewald_dist_cutoff;
if (settings.bandwidth_sigma > 0.0f) {
const float bw_tol = kBandwidthCutoffSigmas * settings.bandwidth_sigma * std::fabs(recip_z);
radial_cutoff = std::sqrt(radial_cutoff * radial_cutoff + bw_tol * bw_tol);
}
if (dist_ewald_sphere <= radial_cutoff ) {
const float s0_p0 = S0.x * recip_x + S0.y * recip_y + S0.z * recip_z;
const float val = s0_sq * recip_sq - s0_p0 * s0_p0;
float delta_phi_deg = NAN;
if (std::fabs(val) >= epsilon && s0_sq > epsilon) {
const float a_num = (s0_sq - 0.25f * recip_sq) * recip_sq;
if (a_num >= 0.0f) {
const float A = std::sqrt(a_num / val);
const float B = (A * s0_p0 + 0.5f * recip_sq) / s0_sq;
const float p_star_x = A * recip_x - B * S0.x;
const float p_star_y = A * recip_y - B * S0.y;
const float p_star_z = A * recip_z - B * S0.z;
const float p_star_sq = p_star_x * p_star_x + p_star_y * p_star_y + p_star_z * p_star_z;
const float denom = std::sqrt(p_star_sq * recip_sq);
if (denom >= epsilon) {
float c = (p_star_x * recip_x + p_star_y * recip_y + p_star_z * recip_z) / denom;
c = std::fmax(-1.0f, std::fmin(1.0f, c));
delta_phi_deg = std::acos(c) * rad_to_deg;
}
}
}
// Inlined RecipToDector with rot1 and rot2 (rot3 = 0)
// Apply rotation matrix transpose
float S_rot_x = rot[0] * S_x + rot[1] * S_y + rot[2] * S_z;
float S_rot_y = rot[3] * S_x + rot[4] * S_y + rot[5] * S_z;
float S_rot_z = rot[6] * S_x + rot[7] * S_y + rot[8] * S_z;
if (S_rot_z <= 0)
continue;
// Project to detector coordinates
// Assume detector is along x,y,z coordinates after rotation
float x = beam_x + F * S_rot_x / S_rot_z;
float y = beam_y + F * S_rot_y / S_rot_z;
if ((x < 0) || (x >= det_width_pxl) || (y < 0) || (y >= det_height_pxl))
continue;
float d = 1.0f / sqrtf(recip_sq);
reflections[i] = Reflection{
.h = h,
.k = k,
.l = l,
.delta_phi_deg = delta_phi_deg,
.predicted_x = x,
.predicted_y = y,
.observed_x = NAN,
.observed_y = NAN,
.d = d,
.dist_ewald = dist_ewald_sphere,
.rlp = 1.0,
.partiality = 1.0f,
.zeta = 1.0,
.image_scale_corr = 1.0
};
++i;
}
}
}
}
return TruncateToOutput(i);
}