Files
Jungfraujoch/image_analysis/IndexAndRefine.h
T
jungfrauandClaude Opus 5 ef0be2e52a Bound the offline lattice refinement by iterations, and stop rotating the same axis three times
Two things in the indexing path, one of them a reproducibility hole.

XtalOptimizerData bounds a solve by iterations when it is told to and by WALL-CLOCK SECONDS when it
is not, and its own header says why that matters: the same image refines to a different answer on a
busier machine. The per-image refinement sets the iteration bound for exactly that reason. The
rotation indexer never did, so its candidate-cell refinement ran under a one-second wall clock -
three stages a candidate, up to eight candidates a scheme, twice a run. A run that has just been
made reproducible from its prediction order to its accumulators was still free to pick a different
lattice because the machine was loaded. It now takes the iteration bound offline and keeps the
wall-clock one for a live acquisition, whose budget is real, which is the same split the per-image
path already makes.

The residual itself rotated the same axis three times over. It applies one orientation to three
reciprocal-lattice vectors, and ceres::AngleAxisRotatePoint recomputes the angle, its sine, its
cosine and the normalised axis on each call - and it does not inline at this optimisation level, so
the compiler cannot notice. On a seventeen-parameter Jet each of those is a full dual-number
evaluation. Computing the rotation once and applying it three times removes two hypots, two sines,
two cosines, two divisions and six multiplies from every evaluation, which is about half the libm
calls in it; hoisting a constant member's sine and cosine out of the same function takes two more.
It runs everywhere the residual does - the indexer, the per-image refinement and the geometry
refiner.

Also lifts five SetParameterBlockConstant calls out of a per-observation loop in the detector solve,
where they were executing once per observation to say the same thing.

The rotation hoist was checked against the function it replaces on 200000 random dual numbers,
including the small-angle branch, comparing the value and all seventeen derivative lanes: no
difference in any component. Merged output is byte-identical on a 16 Mpx set and an ordinary one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011n8riB6X59oRjkrSHzNPAU
2026-08-23 14:03:59 -04:00

132 lines
6.6 KiB
C++

// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <vector>
#include <mutex>
#include <functional>
#include "../common/DiffractionSpot.h"
#include "../common/DiffractionExperiment.h"
#include "../common/AzimuthalIntegrationMapping.h"
#include "../common/AzimuthalIntegrationProfile.h"
#include "../common/Reflection.h"
#include "bragg_prediction/BraggPrediction.h"
#include "indexing/IndexerThreadPool.h"
#include "lattice_search/LatticeSearch.h"
#include "rotation_indexer/RotationIndexer.h"
#include "rotation_indexer/RotationIndexerCounter.h"
#include "scale_merge/ReindexAmbiguity.h"
#include "scale_merge/ScaleOnTheFly.h"
#include "scale_merge/ScalingResult.h"
#include "IntegrationOutcome.h"
// Integrates the predicted reflections off whatever image the caller holds: the preprocessed GPU/CPU
// buffer on the WithoutFPGA path (GPU when available), or the assembled detector image read straight,
// on the CPU, on the forced-CPU FPGA path. Keeps IndexAndRefine independent of the image representation.
using BraggIntegrateFn = std::function<std::vector<Reflection>(
const std::vector<Reflection> &predicted, size_t npredicted, int64_t image_number)>;
class IndexAndRefine {
// When false, the current image's result is still returned via the outgoing message, but the
// whole-run integration_outcome vector is not retained (viewer live/interactive use, which never
// scales the accumulated run). rugnux/receiver keep it true so ScaleAllImages/merge have the data.
const bool retain_outcomes_;
const bool real_time; // see the constructor
const DiffractionExperiment& experiment;
const DiffractionGeometry geom_;
std::optional<CrystalLattice> indexed_lattice;
std::optional<GoniometerAxis> axis_;
IndexerThreadPool *indexer_;
std::unique_ptr<RotationIndexer> rotation_indexer;
RotationIndexerCounter rotation_indexer_counter;
struct IndexingOutcome {
std::optional<CrystalLattice> lattice_candidate;
std::vector<CrystalLattice> extra_lattice_candidates;
std::vector<Coord> extra_lattice_rotations;
DiffractionExperiment experiment;
LatticeMessage symmetry{
.centering = 'P',
.niggli_class = 0,
.crystal_system = gemmi::CrystalSystem::Triclinic
};
bool beam_center_updated = false;
explicit IndexingOutcome(const DiffractionExperiment& experiment_ref)
: experiment(experiment_ref) {}
};
mutable std::mutex reflections_mutex;
std::vector<IntegrationOutcome> integration_outcome;
std::vector<float> mosaicity;
// Optional per-frame mosaicity used for Bragg prediction, indexed by image number. When set (the
// second pass of the rotation two-pass), it overrides the per-image spot-shape estimate so prediction
// uses the frame-order-SMOOTHED mosaicity that RotationScaleMerge already fitted in the first pass,
// rather than re-deriving it from scratch.
std::vector<float> prediction_mosaicity_override_;
std::vector<float> scale_cc;
std::vector<std::optional<UnitCell> > unit_cells;
IndexingOutcome DetermineLatticeAndSymmetryRotation(DataMessage &msg);
IndexingOutcome DetermineLatticeAndSymmetry(DataMessage &msg);
// Shared indexing path: determine the lattice/symmetry, refine geometry, and run AnalyzeIndexing.
// Returns the outcome (ready for integration) when the frame indexes, nullopt otherwise. Both the
// real per-image ProcessImage and the first-pass scheme validation go through this, so they cannot
// diverge.
std::optional<IndexingOutcome> DetermineRefineAnalyze(DataMessage &msg,
const SpotFindingSettings &spot_finding_settings);
void RefineGeometryIfNeeded(DataMessage &msg, IndexingOutcome &outcome);
void QuickPredictAndIntegrate(DataMessage &msg,
const SpotFindingSettings &spot_finding_settings,
BraggPrediction &prediction,
const BraggIntegrateFn &integrate,
const IndexingOutcome &outcome);
std::unique_ptr<ReindexAmbiguityResolver> reindex_resolver;
void ScaleImage(DataMessage &msg, IntegrationOutcome& outcome);
std::optional<float> RotationAngle(int64_t image) const; // mid-exposure angle for the indexer
public:
// real_time: bound the geometry refinements - the per-image one here and the candidate-cell ones
// in the rotation indexer - by WALL CLOCK, as online acquisition must, it having a real budget.
// Offline (rugnux, the viewer) passes false and they are bounded by iteration count instead, so the
// same file reprocesses to the same answer regardless of what else the machine was doing.
IndexAndRefine(const DiffractionExperiment &x, IndexerThreadPool *indexer, bool retain_outcomes = true,
bool real_time = false);
void AddImageToRotationIndexer(DataMessage &msg);
void ForceRotationIndexerLattice(const CrystalLattice& lattice);
void ForceRotationIndexerResult(const RotationIndexerResult& result);
// Supply a per-frame (by image number) mosaicity for prediction, overriding the per-image estimate.
void SetPredictionMosaicityOverride(std::vector<float> mosaicity_per_frame) {
prediction_mosaicity_override_ = std::move(mosaicity_per_frame);
}
// Returns whether the frame indexed (a lattice was found and refined). Integration, when it runs,
// is a further step gated on quick_integration.
void ProcessImage(DataMessage &msg, const SpotFindingSettings &settings,
BraggPrediction &prediction, const BraggIntegrateFn &integrate);
// Index a single frame (no integration) with the current forced rotation lattice; used to score
// first-pass sampling schemes on the real per-image path. Returns whether the frame indexed.
bool IndexFrameOnly(DataMessage &msg, const SpotFindingSettings &settings);
IndexAndRefine& ReferenceIntensities(std::vector<MergedReflection> &reference);
ScalingResult ScaleAllImages(const std::vector<MergedReflection> &reference, size_t nthreads = 0);
std::optional<RotationIndexerResult> FinalizeRotationIndexing();
std::optional<UnitCell> GetConsensusUnitCell() const;
// Not thread safe, need to be run after processing is all done
const std::vector<float> &GetImageCC() const;
const std::vector<std::optional<UnitCell> > &GetUnitCells() const;
std::vector<IntegrationOutcome> &GetIntegrationOutcome();
const std::vector<IntegrationOutcome> &GetIntegrationOutcome() const;
};