mx/Jungfraujoch
3
2 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 87 Wiki Activity

Radial background: one mechanism for stills and rotation, decided per run on empty apertures (D5)

Browse Source 
Wherever the correction is set for the whole run its curve is the frame's peak-excluded ring
background (the adaptive spot finder's, whole frame) and the ring's clip follows it - as stills have
done since integ-pal - now rotation too. One empty-aperture measurement (BackgroundAperture) decides
it for both workflows:

- The apertures' corrected difference is now taken as the integrator applies the correction: the
  clip follows the curve (each ring pixel's threshold raised by the curve's rise from the centre, in
  whole counts) as well as the kernel correction. Before, the tally clipped flat, so it measured a
  correction the engine never applies.
- Rotation measures the stencils the spot width can give it (r1 = 4, 5, 6 with the ring-area rule,
  RotationStencil) and decides on the one it gets, and separately on the pre-adaptive stencil the
  geometry pre-pass and the starvation guard return to.
- Stills: on wherever the corrected ring predicts the empty disk better (unchanged rule).
  Rotation: the frame's curve on every image only where it takes >= 10% of the apertures' mean
  square error away (ROTATION_RADIAL_MIN_GAIN); otherwise the existing per-image ice gate with the
  reflections' curve is left exactly as it was. Why a margin on rotation only: a still has no other
  correction, a sweep has one. Measured over 18 rotation sets (forced on every image): photon-counting
  sweeps gain 0.1-5% on the apertures (z 3-4) and lose ISa/anomalous signal when corrected
  (insulin, lysozyme, cytochrome c; 9yzk R-free +0.009 at 1.2%), as the earlier stills-rule trial
  found; the home-source CCD and smooth-ice runs gain 13-31%.

Result (CPU A/B vs 87684012f, 18 rotation sets): 12 .hkl byte-identical (every photon-counting
control without ice), 5uth R-free 0.210 -> 0.201 / ISa 10.4 -> 10.8, the ice set neutral; stills:
lcls242_s turns the correction on (the clip-following tally), R-free 0.2399 -> 0.2406.
No integration kernel changed; the tally is host code. Tests: BackgroundAperture_CorrectedClipFollowsTheCurve,
RadialHelps margins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit dd3f3f2da39f5338e7dda1874bbdd535229bce98)
This commit is contained in:
leonarski_f committed 2026-10-24 07:27:12 +02:00
1 parent 134fc56f70
commit f199f23c54
8 files changed
+195 -49
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

rugnux/RugnuxPreScan.cpp
+89 -25
View File
@@ -88,6 +88,13 @@
using namespace rugnux_internal;
namespace {
// The rotation stencil the spot width gives a signal radius r1: r2 follows it, and r3 keeps the
// r2..r3 background ring at the area it has at the shipped 4/6/13.
background_aperture::Stencil RotationStencil(float r1) {
const float r2 = r1 + 2.0f;
return {r1, r2, {std::sqrt(r2 * r2 + 133.0f)}};
}
// A detector-plane displacement resolved ACROSS the spindle - the only component the tolerance
// above is about. Along the spindle the error translates the derotated cloud rigidly and the FFT
// amplitude cannot see it at all; across it the peaks smear and the amplitude falls. (The lattice
@@ -364,21 +371,29 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
return w;
};
// Stills: how far the background ring should reach, measured on empty apertures of the same frames
// (BackgroundAperture.h) at the signal disk and gap the run integrates with. The candidates run up
// to the ring the run starts from, so where the noise wins that ring is what is kept. The ring is
// Empty apertures of the same frames (BackgroundAperture.h): whether the radial background correction
// makes the ring predict the background under the disk better, and on stills also how far the ring
// should reach. Stills measure at the signal disk and gap the run integrates with, the candidate rings
// running up to the one the run starts from, so where the noise wins that ring is what is kept.
// Rotation does not know its stencil yet - the spot width decides it from these same frames - so each
// stencil it can end up with is measured, and the one it does is read (RotationStencil). The ring is
// clipped at the gain the run integrates at.
const bool stills = !experiment_.IsRotationIndexing();
const BraggIntegrationSettings aperture_bis = prescan_x.GetBraggIntegrationSettings();
const std::vector<float> aperture_r3 = stills
? background_aperture::CandidateR3(aperture_bis.GetR2(), aperture_bis.GetR3()) : std::vector<float>();
std::vector<background_aperture::Stencil> aperture_stencils;
if (stills)
aperture_stencils.push_back({aperture_bis.GetR1(), aperture_bis.GetR2(),
background_aperture::CandidateR3(aperture_bis.GetR2(), aperture_bis.GetR3())});
else
for (const float r1 : {4.0f, 5.0f, 6.0f}) // the radii spot_width::R1ForWidth chooses from
aperture_stencils.push_back(RotationStencil(r1));
const float aperture_gain = aperture_bis.GetPixelGain();
// The spots of one image, and what is taken off them.
const auto find_spots = [&](PreScanWorker &w, const CompressedImage &image, int image_idx,
bool for_width, std::vector<spot_width::FluxCurve> &curves,
std::vector<float> &spot_q, std::vector<FootprintSpot> &footprint,
background_aperture::Tally &apertures) {
std::vector<background_aperture::Tally> &apertures) {
try {
// As in the image loops: a frame the device cannot decode goes to the host decoder.
ImageStatistics stats;
@@ -422,7 +437,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
MeasureSpotsOnTiles(*w.preprocessed, static_cast<int>(prescan_x.GetXPixelsNumConv()),
static_cast<int>(prescan_x.GetYPixelsNumConv()), prescan_x.GetDiffractionGeometry(),
prescan_x.GetModules(), spots, curves, footprint, aperture_gain);
if (for_width && !aperture_r3.empty()) {
if (for_width) {
// The frame's ring background, as the integration of this frame will read it - from the
// adaptive finder, which has it whole-frame and peak-excluded.
const std::vector<float> no_rings;
@@ -431,11 +446,14 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
prescan_mapping->GetQBinCount(),
prescan_mapping->Settings().GetLowQ_recipA(),
prescan_mapping->Settings().GetQSpacing_recipA());
background_aperture::MeasureFrame(*w.preprocessed, static_cast<int>(prescan_x.GetXPixelsNumConv()),
static_cast<int>(prescan_x.GetYPixelsNumConv()), spots,
aperture_bis.GetR1(), aperture_bis.GetR2(), aperture_r3,
aperture_bis.GetBackgroundClipNSigma(), aperture_gain, rings,
prescan_x.GetDiffractionGeometry(), prescan_x.GetModules(), apertures);
apertures.resize(aperture_stencils.size());
for (size_t k = 0; k < aperture_stencils.size(); ++k)
background_aperture::MeasureFrame(*w.preprocessed, static_cast<int>(prescan_x.GetXPixelsNumConv()),
static_cast<int>(prescan_x.GetYPixelsNumConv()), spots,
aperture_stencils[k].r1, aperture_stencils[k].r2,
aperture_stencils[k].r3, aperture_bis.GetBackgroundClipNSigma(),
aperture_gain, rings, prescan_x.GetDiffractionGeometry(),
prescan_x.GetModules(), apertures[k]);
}
};
@@ -446,8 +464,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
std::vector<spot_width::FluxCurve> shape_curves;
// Every spot's measured widths, for the spot footprint (SpotFootprint.h).
std::vector<FootprintSpot> footprint_spots;
// The empty apertures of the frames the width is measured on (stills).
background_aperture::Tally aperture_tally;
// The empty apertures of the frames the width is measured on, one tally per aperture_stencils entry.
std::vector<background_aperture::Tally> aperture_tally(aperture_stencils.size());
// Images the width ended up being measured on, for the log: the tiers below stop as soon as the
// answer has settled, so this is a property of the crystal and worth reporting.
size_t width_images = 0;
@@ -565,7 +583,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
std::vector<std::vector<spot_width::FluxCurve>> curves_of(ordinals.size());
std::vector<std::vector<float>> spot_q_of(ordinals.size());
std::vector<std::vector<FootprintSpot>> footprint_of(ordinals.size());
std::vector<background_aperture::Tally> apertures_of(ordinals.size());
std::vector<std::vector<background_aperture::Tally>> apertures_of(ordinals.size());
// Frames read for the bandwidth alone, past the tier the width settled on. Their spots stay
// out of the powder measurement, which is taken on the frames it always was.
std::vector<char> shape_only(ordinals.size(), 0);
@@ -683,7 +701,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
// In sample order, over the frames the width was taken on.
for (size_t i = 0; i < ordinals.size(); i++)
if (position[i] < width_images)
aperture_tally.Add(apertures_of[i]);
for (size_t k = 0; k < apertures_of[i].size(); ++k)
aperture_tally[k].Add(apertures_of[i][k]);
// In sample order, so the table does not depend on how the workers interleaved.
for (const auto &f : footprint_of)
footprint_spots.insert(footprint_spots.end(), f.begin(), f.end());
@@ -733,13 +752,15 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
// The ring is chosen on the plain differences, so its reach does not rest on the radial
// model; the correction is then kept, where it was left to auto, if it makes that ring predict
// the disk better. Read only where every frame gave a ring background.
const std::vector<double> plain = aperture_tally.MeanSquare(false);
const std::vector<double> radial = aperture_tally.apertures_radial == aperture_tally.apertures
? aperture_tally.MeanSquare(true) : std::vector<double>();
const background_aperture::Tally &tally = aperture_tally.front();
const std::vector<float> &aperture_r3 = aperture_stencils.front().r3;
const std::vector<double> plain = tally.MeanSquare(false);
const std::vector<double> radial = tally.apertures_radial == tally.apertures
? tally.MeanSquare(true) : std::vector<double>();
const int best = background_aperture::Best(plain);
const bool use_radial = radial_setting.has_value()
? *radial_setting
: best >= 0 && !radial.empty() && radial[best] < plain[best];
: best >= 0 && background_aperture::RadialHelps(tally, best);
if (best < 0) {
logger.Info("Background ring: no empty aperture measured on {} images, keeping r3={:.1f}",
width_images, bis.GetR3());
@@ -757,10 +778,10 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
logger.Info("Background ring: on {} empty apertures from {} images the r{:.0f}..r3 ring predicts "
"the r{:.0f} disk with a mean square error of (r3:value){}, and with the radial "
"curvature correction{} => integration radii r1={:.1f} r2={:.1f} r3={:.1f}, radial "
"correction {} (z = {:.1f} at that ring)", aperture_tally.apertures, width_images, bis.GetR2(), bis.GetR1(),
"correction {} (z = {:.1f} at that ring)", tally.apertures, width_images, bis.GetR2(), bis.GetR1(),
table(plain), radial.empty() ? std::string(" (no ring background)") : table(radial),
bis.GetR1(), bis.GetR2(), bis.GetR3(), use_radial ? "on" : "off",
aperture_tally.RadialZ(best));
tally.RadialZ(best));
Note(fmt::format("background ring {:.0f}..{:.0f} px, radial correction {}, measured on empty "
"apertures", bis.GetR2(), bis.GetR3(), use_radial ? "on" : "off"));
}
@@ -773,9 +794,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
"integration radius at r1={:.1f}", width_curves.size(), width_images,
experiment_.GetBraggIntegrationSettings().GetR1());
} else {
const float r1 = spot_width::R1ForWidth(*r80);
const float r2 = r1 + 2.0f;
const float r3 = std::sqrt(r2 * r2 + 133.0f);
const background_aperture::Stencil st = RotationStencil(spot_width::R1ForWidth(*r80));
const float r1 = st.r1, r2 = st.r2, r3 = st.r3.front();
BraggIntegrationSettings bis = experiment_.GetBraggIntegrationSettings();
// Remembered where the APERTURE moved, so the starvation guard in RunAllPasses has
// something to give back and cannot announce a fallback that changes nothing.
@@ -799,6 +819,50 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
*r80, r1, r2, r3));
}
// The radial background correction, where it was left to auto, decided on the empty apertures of
// each stencil the run may integrate with: the one the width gave it, and the one it started
// from, which the geometry pre-pass and the starvation guard go back to. Rotation's own default
// is the per-image correction of the frames the ice score marks (MXAnalysisWithoutFPGA), so the
// frame's curve replaces it on every frame only where the apertures show it taking a real part
// of the background error away (background_aperture::ROTATION_RADIAL_MIN_GAIN).
const auto decide_radial = [&](BraggIntegrationSettings &bis, const char *which) {
if (bis.GetBackgroundRadialCorrection().has_value())
return;
size_t k = 0;
while (k < aperture_stencils.size()
&& !(aperture_stencils[k].r1 == bis.GetR1() && aperture_stencils[k].r2 == bis.GetR2()
&& aperture_stencils[k].r3.front() == bis.GetR3()))
++k;
if (k == aperture_stencils.size() || aperture_tally[k].apertures == 0) {
logger.Info("Background ring ({}): r1={:.1f} r2={:.1f} r3={:.2f} not measured on empty "
"apertures; radial correction per image", which, bis.GetR1(), bis.GetR2(),
bis.GetR3());
return;
}
const background_aperture::Tally &tally = aperture_tally[k];
const std::vector<double> plain = tally.MeanSquare(false), radial = tally.MeanSquare(true);
const bool on = background_aperture::RadialHelps(tally, 0,
background_aperture::ROTATION_RADIAL_MIN_GAIN);
if (on)
bis.BackgroundRadialCorrection(true);
logger.Info("Background ring ({}): on {} empty apertures from {} images the r{:.0f}..r{:.0f} ring "
"predicts the r{:.0f} disk with a mean square error of {:.4g}, and {} with the radial "
"curvature correction from the frame's curve (z = {:.1f}) => radial correction {}",
which, tally.apertures, width_images, bis.GetR2(), bis.GetR3(), bis.GetR1(), plain[0],
radial.empty() ? std::string("none (no ring background)")
: fmt::format("{:.4g}", radial[0]),
tally.RadialZ(0), on ? "on every image" : "per image, where the ice score sees a ring");
};
{
BraggIntegrationSettings bis = experiment_.GetBraggIntegrationSettings();
decide_radial(bis, "canonical");
experiment_.ImportBraggIntegrationSettings(bis);
if (bragg_before_adaptive_)
decide_radial(*bragg_before_adaptive_, "pre-adaptive");
if (bis.GetBackgroundRadialCorrection() == std::optional<bool>(true))
Note("radial background correction on, measured on empty apertures");
}
// How far the spots reach away from the beam, where the radius chosen above at 5 A no
// longer holds them (SpotFootprint.h). It changes the integration only where a spot
// outgrows the r1 disk.