Files
Jungfraujoch/image_analysis/IndexAndRefine.h
T
leonarski_fandClaude Opus 4.8 7b8e08cd96 rugnux: index weakly-diffracting low-resolution rotation data (Benas F-cubic)
Two F432 crystals (a~70.2 A) that diffract only to ~4-4.5 A on a detector
reaching ~1.5 A failed to process de-novo. Three independent, physically
motivated fixes, validated on the 24-crystal rotation_test battery with zero
regressions (every other crystal keeps its cell, indexing rate and space group):

1. Per-frame indexing gate (AnalyzeIndexing): the indexed-fraction denominator
   counted every found spot, but for these crystals ~85% of spots are
   unindexable sub-diffraction ice/noise, making the 20% floor unreachable so
   every frame was rejected (0% indexed). Count only spots within the resolution
   range the lattice actually diffracts to (out to the highest-resolution indexed
   spot). This can only shrink the denominator, so it never rejects a frame that
   passes today. Fixes Benas_3 (0 -> 40%, correct F-cubic -> H32).

2. First-pass rotation indexing now runs multiple sampling schemes and selects
   the one that indexes the most frames on the real per-image path (spread over
   the whole rotation vs a consecutive wedge from the clean start). The
   long-standing spread scheme wins for typical crystals (unchanged), while the
   consecutive wedge keeps a reflection's rocking curve continuous across frames
   so the FFT can trace a long axis whose fine reciprocal spacing the coarse
   spread cannot resolve. Fixes Benas_7 (garbage cell 0.06% -> correct hex 69%).
   Scheme selection reuses ProcessImage's indexing path (extracted as
   DetermineRefineAnalyze / IndexFrameOnly) so validation cannot diverge from
   production.

3. Space-group-search resolution gate (Rugnux): the <I/sigma> >= 1 cut collapsed
   to ~the cell dimension when every shell is below the threshold (I/sigma floored
   by the error model though CC1/2 is high), leaving the re-merge with no
   reflections -> "resolution calculation failed" crash. Only cut when a good
   low-res region exists above the cutoff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:30:32 +02:00

115 lines
5.3 KiB
C++

// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <vector>
#include <mutex>
#include <functional>
#include "../common/DiffractionSpot.h"
#include "../common/DiffractionExperiment.h"
#include "../common/AzimuthalIntegrationMapping.h"
#include "../common/AzimuthalIntegrationProfile.h"
#include "../common/Reflection.h"
#include "bragg_prediction/BraggPrediction.h"
#include "indexing/IndexerThreadPool.h"
#include "lattice_search/LatticeSearch.h"
#include "rotation_indexer/RotationIndexer.h"
#include "rotation_indexer/RotationIndexerCounter.h"
#include "scale_merge/ScaleOnTheFly.h"
#include "scale_merge/ScalingResult.h"
#include "IntegrationOutcome.h"
// Integrates the predicted reflections off whatever image the caller holds: the preprocessed GPU/CPU
// buffer on the WithoutFPGA path (GPU when available), or the assembled detector image read straight,
// on the CPU, on the forced-CPU FPGA path. Keeps IndexAndRefine independent of the image representation.
using BraggIntegrateFn = std::function<std::vector<Reflection>(
const std::vector<Reflection> &predicted, size_t npredicted, int64_t image_number)>;
class IndexAndRefine {
const bool index_ice_rings;
// When false, the current image's result is still returned via the outgoing message, but the
// whole-run integration_outcome vector is not retained (viewer live/interactive use, which never
// scales the accumulated run). rugnux/receiver keep it true so ScaleAllImages/merge have the data.
const bool retain_outcomes_;
const DiffractionExperiment& experiment;
const DiffractionGeometry geom_;
std::optional<CrystalLattice> indexed_lattice;
std::optional<GoniometerAxis> axis_;
IndexerThreadPool *indexer_;
std::unique_ptr<RotationIndexer> rotation_indexer;
RotationIndexerCounter rotation_indexer_counter;
struct IndexingOutcome {
std::optional<CrystalLattice> lattice_candidate;
std::vector<CrystalLattice> extra_lattice_candidates;
std::vector<Coord> extra_lattice_rotations;
DiffractionExperiment experiment;
LatticeMessage symmetry{
.centering = 'P',
.niggli_class = 0,
.crystal_system = gemmi::CrystalSystem::Triclinic
};
bool beam_center_updated = false;
explicit IndexingOutcome(const DiffractionExperiment& experiment_ref)
: experiment(experiment_ref) {}
};
mutable std::mutex reflections_mutex;
std::vector<IntegrationOutcome> integration_outcome;
std::vector<float> mosaicity;
std::vector<float> scale_cc;
std::vector<std::optional<UnitCell> > unit_cells;
IndexingOutcome DetermineLatticeAndSymmetryRotation(DataMessage &msg);
IndexingOutcome DetermineLatticeAndSymmetry(DataMessage &msg);
// Shared indexing path: determine the lattice/symmetry, refine geometry, and run AnalyzeIndexing.
// Returns the outcome (ready for integration) when the frame indexes, nullopt otherwise. Both the
// real per-image ProcessImage and the first-pass scheme validation go through this, so they cannot
// diverge.
std::optional<IndexingOutcome> DetermineRefineAnalyze(DataMessage &msg,
const SpotFindingSettings &spot_finding_settings);
void RefineGeometryIfNeeded(DataMessage &msg, IndexingOutcome &outcome);
void QuickPredictAndIntegrate(DataMessage &msg,
const SpotFindingSettings &spot_finding_settings,
BraggPrediction &prediction,
const BraggIntegrateFn &integrate,
const IndexingOutcome &outcome);
std::unique_ptr<ScaleOnTheFly> scaling_engine;
void ScaleImage(DataMessage &msg, IntegrationOutcome& outcome);
std::optional<float> RotationAngle(int64_t image) const; // mid-exposure angle for the indexer
public:
IndexAndRefine(const DiffractionExperiment &x, IndexerThreadPool *indexer, bool retain_outcomes = true);
void AddImageToRotationIndexer(DataMessage &msg);
void ForceRotationIndexerLattice(const CrystalLattice& lattice);
void ForceRotationIndexerResult(const RotationIndexerResult& result);
void ProcessImage(DataMessage &msg, const SpotFindingSettings &settings,
BraggPrediction &prediction, const BraggIntegrateFn &integrate);
// Index a single frame (no integration) with the current forced rotation lattice; used to score
// first-pass sampling schemes on the real per-image path. Returns whether the frame indexed.
bool IndexFrameOnly(DataMessage &msg, const SpotFindingSettings &settings);
IndexAndRefine& ReferenceIntensities(std::vector<MergedReflection> &reference);
ScalingResult ScaleAllImages(const std::vector<MergedReflection> &reference, size_t nthreads = 0);
std::optional<RotationIndexerResult> FinalizeRotationIndexing();
std::optional<UnitCell> GetConsensusUnitCell() const;
// Not thread safe, need to be run after processing is all done
const std::vector<float> &GetImageCC() const;
const std::vector<std::optional<UnitCell> > &GetUnitCells() const;
std::vector<IntegrationOutcome> &GetIntegrationOutcome();
const std::vector<IntegrationOutcome> &GetIntegrationOutcome() const;
};