leonarski_fandClaude Opus 5.5
96322a8ec9
Four changes to the per-run fixed cost of a stills run (Step 2 geometry refinement, Step 3 pre-scan), none of which changes what is computed: - Module placement: the four held-out fits and the placement on every strong frame are independent, and one Ceres fit keeps only ~4.6-6.5 of 32 cores busy (perf, JF16M, 4000 frames / 150888 spots). They now run side by side on a fifth of the threads each; the placement fit is computed speculatively and used only where the held-out residuals call for it, as before. Baseline (quiet): held-out 12.3 s + placement 5.1 s on JF16M, 16.4 + 6.5 s on JF4M, 19.0 + 7.6 s on MPCCD. - Module scan (+-25 px grid per module ahead of the placement) was one thread: every grid shift is now scored on the workers and the grid then walked in its old order, so the first best still wins (3.3 s on JF16M, ~5 s on JF4M/MPCCD before). New test ScanModuleShifts_FindsShiftedModule checks the shift is found and that 1 and 4 threads give the same answer. - Stills geometry trial: each hypothesis (centre, distance, frame sample) is indexed once. The distance grid ran twice on the same centre whenever the file's centre was kept (40 of ~275 hypotheses, ~15% of the trial: 6.2 s on JF16M, 6.7 s on the flat JF8M), and the final here/there check repeated the first trial. Repeats always gave the same count (checked on three logs). - The saturation histogram (40 frames, 0.1-2.2 s) reads only frames and the mask: it is taken beside the geometry refinement and the limit decided where it was, after it. Plus TIMING marks for the sub-steps (geometry refinement sample, module scan, module placement fits). Outputs identical to the baseline binary (bb7c9c) where checked: JF16M int16 15 runs (geometry, all 32 module shifts, held-out residuals, cell), JF4M int16, MPCCD 236 (indexing rate, CC1/2, R_split, R-free with --model identical), PAL 90 and LCLS 222 (identical reports). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
115 lines
6.7 KiB
C++
115 lines
6.7 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <array>
|
|
#include <cstdint>
|
|
#include <vector>
|
|
|
|
#include "../../common/CrystalLattice.h"
|
|
#include "../../common/DiffractionGeometry.h"
|
|
#include "../../common/ModuleShift.h"
|
|
#include "../../common/UnitCell.h"
|
|
#include "gemmi/symmetry.hpp"
|
|
|
|
// Offline global (multi-frame) geometry refinement for serial stills. Pass 1 indexes many frames
|
|
// independently; this joint bundle adjustment then determines the ONE shared detector geometry (beam
|
|
// centre, distance, cell scale) that per-image refinement cannot reach from a single sparse still, so
|
|
// re-indexing with it converts more frames. Mirrors the validated Python prototype: one Ceres problem
|
|
// with shared beam/distance/cell blocks and a per-frame orientation block, robust loss, and a cell
|
|
// regularizer that anchors the (known) cell to break the distance<->cell-scale degeneracy at low
|
|
// resolution. Detector tilt (rot1/rot2) is NOT refined (gauge-coupled with the beam; zero indexing gain).
|
|
|
|
struct GeomRefineSpot {
|
|
float x = 0, y = 0; // observed (raw) spot position, pixels
|
|
int32_t h = 0, k = 0, l = 0; // integer Miller index assigned in pass 1
|
|
int16_t module = -1; // detector module the spot lies on (GeometryRefinerSettings::modules); -1: none
|
|
};
|
|
|
|
|
|
struct GeomRefineFrame {
|
|
CrystalLattice lattice; // per-frame indexed lattice (real space), for the orientation seed
|
|
std::vector<GeomRefineSpot> spots; // indexed spots on this frame
|
|
std::vector<GeomRefineSpot> unindexed; // the frame's other spots (no index); see ScanModuleShifts
|
|
int32_t ordinal = 0; // image this came from; ties are broken on it, so the
|
|
// selected bundle does not depend on collection order
|
|
};
|
|
|
|
struct GeometryRefinerSettings {
|
|
gemmi::CrystalSystem crystal_system = gemmi::CrystalSystem::Triclinic;
|
|
// Cell-scale regularizer strength (anchors the cell lengths to the input cell so the otherwise
|
|
// degenerate distance is determined). Large => cell effectively fixed (the validated safe regime).
|
|
double cell_reg_coeff = 30.0;
|
|
// With the distance held, the cell is no longer degenerate with it and is refined from the spots.
|
|
bool refine_distance = true;
|
|
int rounds = 3; // hkl-reassignment / tolerance-tightening rounds
|
|
double beam_range_px = 8.0; // beam bound around the nominal value
|
|
double distance_range_mm = 8.0; // distance bound around the nominal value
|
|
int min_spots_total = 100; // refuse to refine below this many indexed spots
|
|
int num_threads = 1;
|
|
// false: beam, distance and cell are held where nominal_geom and input_cell put them and only the
|
|
// crystal orientations follow - the fit quality of a geometry decided elsewhere, on frames it was
|
|
// not fitted to.
|
|
bool refine_beam_distance = true;
|
|
// The detector's modules, with where each is placed to start from; empty: the image is one rigid
|
|
// detector. With refine_modules, every module that has at least min_module_spots spots on its frames'
|
|
// lattices where it starts is moved as a rigid block (shift and turn in the detector plane); the others
|
|
// go back to where they were given (no move), whatever their start.
|
|
std::vector<ModuleShift> modules;
|
|
bool refine_modules = false;
|
|
int min_module_spots = 500;
|
|
// Refine the detector tilt (PONI rot1, rot2) with the beam and distance. A tilt moves the modules far
|
|
// from the beam more than those near it, in a pattern module moves could otherwise take up: with it
|
|
// refined, the modules are also held to no tilt-like pattern of their own (ModuleGauge).
|
|
bool refine_tilt = false;
|
|
};
|
|
|
|
struct GeometryRefinerResult {
|
|
bool ok = false;
|
|
double beam_x_px = 0, beam_y_px = 0; // refined beam centre (== direct beam, tilt not refined)
|
|
double distance_mm = 0;
|
|
UnitCell cell{}; // refined cell
|
|
int frames_used = 0;
|
|
int spots_used = 0;
|
|
double median_residual_px = 0; // final per-spot detector residual (fit quality)
|
|
// The same over EVERY spot of the frames, each spot's residual to its nearest predicted reflection
|
|
// capped at 2 px: unlike the median of the spots that fit, it does not improve by fitting fewer of them,
|
|
// so two geometries can be compared on the same spots.
|
|
double capped_rms_px = 0;
|
|
std::vector<ModuleShift> modules; // the modules' placement at the end (settings.modules if not refined)
|
|
double rot1_rad = 0, rot2_rad = 0; // detector tilt at the end (the nominal one where not refined)
|
|
// Per module (as settings.modules): rms of every spot's residual capped at 2 px, and the mean
|
|
// resolution of its spots - what a module's fit rests on.
|
|
std::vector<double> module_capped_rms_px;
|
|
std::vector<double> module_mean_d_A;
|
|
// Per frame, per spot (as frames[i].spots): where the refined geometry predicts the spot's reflection on
|
|
// the detector, as the spot would be found (before its module's move); NaN where it predicts none.
|
|
std::vector<std::vector<std::array<float, 2>>> predicted_px;
|
|
};
|
|
|
|
// nominal_geom supplies the starting beam/distance/tilt/wavelength/pixel size; input_cell is anchored by
|
|
// the regularizer. Returns ok=false (and leaves the caller's geometry untouched) on any failure.
|
|
GeometryRefinerResult RefineGlobalGeometry(const DiffractionGeometry &nominal_geom,
|
|
const UnitCell &input_cell,
|
|
const std::vector<GeomRefineFrame> &frames,
|
|
const GeometryRefinerSettings &settings);
|
|
|
|
// A coarse search for where each module is, ahead of RefineGlobalGeometry: a module misplaced by more than a
|
|
// fraction of the spot spacing has its spots left unindexed (or indexed wrongly), so the refinement has
|
|
// nothing to pull it with. Each module is shifted over a grid of +-range_px, and at each shift every spot on
|
|
// it (indexed or not) is scored against its frame's lattice as indexed: the shift that puts the most spots on
|
|
// the lattices wins. The frames' lattices come from the other modules, so this works while most of the
|
|
// detector is roughly right. Turns are not searched.
|
|
struct ModuleScan {
|
|
double dx = 0, dy = 0; // the best shift found
|
|
int on_lattice_at_zero = 0;
|
|
int on_lattice_at_best = 0;
|
|
};
|
|
|
|
std::vector<ModuleScan> ScanModuleShifts(const DiffractionGeometry &geom,
|
|
const std::vector<GeomRefineFrame> &frames,
|
|
const std::vector<ModuleShift> &modules,
|
|
double range_px, size_t nthreads = 1);
|
|
|