leonarski_f
3d9c8b878b
The ladder set a trial centre and then scored it on spots found at the starting centre. Raw centroids do not move with the beam centre, but which spots are in the list does: the resolution limit, the ice-ring flag, the beam-stop mask and the strongest-N ranking are all computed against a radius. Every rung was therefore judged on the starting centre's evidence, and the count it produced was not a function of the centre it named. A trial centre now gets its own azimuthal mapping, spot engines and spot cache; the starting centre's cache is parked and restored, so a ladder that adopts nothing leaves the run exactly as it was. Measured against standalone runs pinned at the same centres, the ladder's score goes from a mean absolute error of 21.8 validation frames to 0.43, and at nine pinned centres the patched and unpatched binaries agree exactly - the change is confined to the ladder. Scoring honestly is not by itself an improvement: with the count honest, one crystal adopted a rung at a real 56/60 whose cell is a four-fold axis harmonic of the standing one, and merged the supercell. A longer cell indexes more, which is why the harmonic arbiter exists elsewhere in this file. A rung whose primitive volume is an integer or sqrt(3) multiple of the standing cell is now refused, and that crystal walks past both harmonics to a 60/60 rung. Over a hundred rotation datasets no space group, cell or run status changes. Eight merges move: one gains 126 per cent of its unique reflections, one 3.7, five move by less than half a per cent, and one loses 3.2 - that last was attributed separately and is the difference between two failed merges, both peaking at a half-dataset correlation of 5 per cent, with the honest run the slightly better of the two. Wall time rises 1.7 per cent, the cost of finding spots again at each trial centre. On a corpus processed from a deliberately destroyed beam centre the same change recovers eight datasets outright on their deposited space group and cell, restores a ninth's space-group call by returning its axial absence counts to the values the correct geometry gives, and withdraws one adoption that had shipped a wrong cell in the wrong crystal system at a half-dataset correlation of 0.10. Note that only the spot-cache half of this does the work. Reading the indexer's geometry per call rather than from construction was measured to change nothing on a rotation ladder, because the per-frame validation dispatches to a path that takes its geometry from the forced indexer result; it is kept because the copy was misleading to read, not because it moves a number.
154 lines
8.4 KiB
C++
154 lines
8.4 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <atomic>
|
|
#include <vector>
|
|
#include <mutex>
|
|
#include <functional>
|
|
|
|
#include "../common/DiffractionSpot.h"
|
|
#include "../common/DiffractionExperiment.h"
|
|
#include "../common/AzimuthalIntegrationMapping.h"
|
|
#include "../common/AzimuthalIntegrationProfile.h"
|
|
#include "../common/Reflection.h"
|
|
#include "bragg_prediction/BraggPrediction.h"
|
|
#include "indexing/IndexerThreadPool.h"
|
|
#include "lattice_search/LatticeSearch.h"
|
|
#include "rotation_indexer/RotationIndexer.h"
|
|
#include "rotation_indexer/RotationIndexerCounter.h"
|
|
#include "scale_merge/ReindexAmbiguity.h"
|
|
#include "scale_merge/ScaleOnTheFly.h"
|
|
#include "scale_merge/ScalingResult.h"
|
|
#include "IntegrationOutcome.h"
|
|
|
|
// Integrates the predicted reflections off whatever image the caller holds: the preprocessed GPU/CPU
|
|
// buffer on the WithoutFPGA path (GPU when available), or the assembled detector image read straight,
|
|
// on the CPU, on the forced-CPU FPGA path. Keeps IndexAndRefine independent of the image representation.
|
|
using BraggIntegrateFn = std::function<std::vector<Reflection>(
|
|
const std::vector<Reflection> &predicted, size_t npredicted, int64_t image_number)>;
|
|
|
|
class IndexAndRefine {
|
|
// When false, the current image's result is still returned via the outgoing message, but the
|
|
// whole-run integration_outcome vector is not retained (viewer live/interactive use, which never
|
|
// scales the accumulated run). rugnux/receiver keep it true so ScaleAllImages/merge have the data.
|
|
const bool retain_outcomes_;
|
|
const bool real_time; // see the constructor
|
|
const DiffractionExperiment& experiment;
|
|
|
|
std::optional<CrystalLattice> indexed_lattice;
|
|
|
|
std::optional<GoniometerAxis> axis_;
|
|
|
|
IndexerThreadPool *indexer_;
|
|
std::unique_ptr<RotationIndexer> rotation_indexer;
|
|
RotationIndexerCounter rotation_indexer_counter;
|
|
|
|
struct IndexingOutcome {
|
|
std::optional<CrystalLattice> lattice_candidate;
|
|
std::vector<CrystalLattice> extra_lattice_candidates;
|
|
std::vector<Coord> extra_lattice_rotations;
|
|
DiffractionExperiment experiment;
|
|
LatticeMessage symmetry{
|
|
.centering = 'P',
|
|
.niggli_class = 0,
|
|
.crystal_system = gemmi::CrystalSystem::Triclinic
|
|
};
|
|
bool beam_center_updated = false;
|
|
|
|
explicit IndexingOutcome(const DiffractionExperiment& experiment_ref)
|
|
: experiment(experiment_ref) {}
|
|
};
|
|
|
|
mutable std::mutex reflections_mutex;
|
|
std::vector<IntegrationOutcome> integration_outcome;
|
|
std::vector<float> mosaicity;
|
|
// Optional per-frame mosaicity used for Bragg prediction, indexed by image number. When set (the
|
|
// second pass of the rotation two-pass), it overrides the per-image spot-shape estimate so prediction
|
|
// uses the frame-order-SMOOTHED mosaicity that RotationScaleMerge already fitted in the first pass,
|
|
// rather than re-deriving it from scratch.
|
|
std::vector<float> prediction_mosaicity_override_;
|
|
// Predict every node of the lattice, ignoring the centring absences of a fixed space group. Set
|
|
// for the rotation two-pass GEOMETRY pre-pass, whose job is to measure the detector geometry from
|
|
// spot positions and whose intensities are thrown away: rejecting the absences there costs it half
|
|
// its events and buys nothing. Measured with an I-centred group fixed - the pre-pass fitted a
|
|
// different error model (ISa 7.8 -> 3.6), post-refined the distance 119 um away, and the second
|
|
// pass re-indexed 49 of 60 frames instead of 60.
|
|
bool predict_all_centring_nodes_ = false;
|
|
// The lattice centring the last prediction actually ran in: 'P' unless a user-fixed space group
|
|
// made it reject that group's centring absences. What a caller needs to know whether the
|
|
// integration covers every node of the lattice - the P1 cross-check merge does.
|
|
std::atomic<char> prediction_centring_ = 'P';
|
|
// Whether the outgoing message carries its own copy of the integrated reflections. The per-image
|
|
// file writer and the online stream are the only readers of it - the whole-run scaling/merge reads
|
|
// the retained outcome instead - and it is a copy of every reflection of every image, so a caller
|
|
// that writes no per-image file switches it off.
|
|
bool keep_reflections_in_message_ = true;
|
|
std::vector<float> scale_cc;
|
|
std::vector<std::optional<UnitCell> > unit_cells;
|
|
|
|
IndexingOutcome DetermineLatticeAndSymmetryRotation(DataMessage &msg);
|
|
IndexingOutcome DetermineLatticeAndSymmetry(DataMessage &msg);
|
|
// Shared indexing path: determine the lattice/symmetry, refine geometry, and run AnalyzeIndexing.
|
|
// Returns the outcome (ready for integration) when the frame indexes, nullopt otherwise. Both the
|
|
// real per-image ProcessImage and the first-pass scheme validation go through this, so they cannot
|
|
// diverge.
|
|
std::optional<IndexingOutcome> DetermineRefineAnalyze(DataMessage &msg,
|
|
const SpotFindingSettings &spot_finding_settings);
|
|
void RefineGeometryIfNeeded(DataMessage &msg, IndexingOutcome &outcome);
|
|
void QuickPredictAndIntegrate(DataMessage &msg,
|
|
const SpotFindingSettings &spot_finding_settings,
|
|
BraggPrediction &prediction,
|
|
const BraggIntegrateFn &integrate,
|
|
const IndexingOutcome &outcome);
|
|
|
|
std::unique_ptr<ReindexAmbiguityResolver> reindex_resolver;
|
|
void ScaleImage(DataMessage &msg, IntegrationOutcome& outcome);
|
|
|
|
std::optional<float> RotationAngle(int64_t image) const; // mid-exposure angle for the indexer
|
|
public:
|
|
// real_time: bound the geometry refinements - the per-image one here and the candidate-cell ones
|
|
// in the rotation indexer - by WALL CLOCK, as online acquisition must, it having a real budget.
|
|
// Offline (rugnux, the viewer) passes false and they are bounded by iteration count instead, so the
|
|
// same file reprocesses to the same answer regardless of what else the machine was doing.
|
|
IndexAndRefine(const DiffractionExperiment &x, IndexerThreadPool *indexer, bool retain_outcomes = true,
|
|
bool real_time = false);
|
|
|
|
void AddImageToRotationIndexer(DataMessage &msg);
|
|
void ForceRotationIndexerLattice(const CrystalLattice& lattice);
|
|
void ForceRotationIndexerResult(const RotationIndexerResult& result);
|
|
// Supply a per-frame (by image number) mosaicity for prediction, overriding the per-image estimate.
|
|
void SetPredictionMosaicityOverride(std::vector<float> mosaicity_per_frame) {
|
|
prediction_mosaicity_override_ = std::move(mosaicity_per_frame);
|
|
}
|
|
// Predict the centring-absent reflections too, even with a fixed space group - see the member.
|
|
void PredictAllCentringNodes(bool on) { predict_all_centring_nodes_ = on; }
|
|
// The centring prediction ran in - see the member.
|
|
char GetPredictionCentring() const { return prediction_centring_; }
|
|
// Whether the outgoing message keeps its own copy of the reflections - see the member.
|
|
void KeepReflectionsInMessage(bool on) { keep_reflections_in_message_ = on; }
|
|
|
|
// Returns whether the frame indexed (a lattice was found and refined). Integration, when it runs,
|
|
// is a further step gated on quick_integration.
|
|
void ProcessImage(DataMessage &msg, const SpotFindingSettings &settings,
|
|
BraggPrediction &prediction, const BraggIntegrateFn &integrate);
|
|
// Index a single frame (no integration) with the current forced rotation lattice; used to score
|
|
// first-pass sampling schemes on the real per-image path. Returns whether the frame indexed.
|
|
bool IndexFrameOnly(DataMessage &msg, const SpotFindingSettings &settings);
|
|
IndexAndRefine& ReferenceIntensities(std::vector<MergedReflection> &reference);
|
|
|
|
ScalingResult ScaleAllImages(const std::vector<MergedReflection> &reference, size_t nthreads = 0);
|
|
|
|
std::optional<RotationIndexerResult> FinalizeRotationIndexing();
|
|
|
|
std::optional<UnitCell> GetConsensusUnitCell() const;
|
|
|
|
// Not thread safe, need to be run after processing is all done
|
|
const std::vector<float> &GetImageCC() const;
|
|
const std::vector<std::optional<UnitCell> > &GetUnitCells() const;
|
|
|
|
std::vector<IntegrationOutcome> &GetIntegrationOutcome();
|
|
const std::vector<IntegrationOutcome> &GetIntegrationOutcome() const;
|
|
};
|