A CUDA out-of-memory in an indexer worker was turned into an IndexerResult carrying only the message string. The rotation indexer then returned no lattice, the first pass treated that as a genuine negative and the rescues acted on it: in a stress run (two rugnux on one card) a run had both first-pass schemes fail with out-of-memory, the axis-sign rescue compared 0 frames at the flipped sign with the -1 "no lattice" sentinel, adopted the flipped sign, and the beam-centre rescue then moved the centre - a failure of the machine used as evidence about the crystal. The CUDA category was lost on the way, so the fail-fast check (IsFatalResourceError) never saw it. - IndexerResult::error is now the std::exception_ptr the worker caught. RotationIndexer::RunIndexing rethrows it before touching any state, so nothing is memoised for a failed run and the exception keeps its category: rugnux stops with the CUDA message at once (verified with a forced indexer failure: exit 1 after 1.9 s, "CUDA (GPU) error (...)"). FirstPass::indexer_error, RotationIndexer::GetIndexerError and RequireLattice's machine-failure branch are gone. - The beam-centre arm in RunAllPasses rethrows fatal resource errors like its sibling catches; it had caught the out-of-memory and carried on at the file's centre. - IndexAndRefine keeps per-frame indexer failures apart from frames that did not index (count + first exception). The frame stays "not attempted" as before; the receiver logs the count and the first error at the end of a collection (no cancel), rugnux's image pass and stills geometry refinement rethrow it and stop. A rotation-indexer failure inside IndexAndRefine (online rotation indexing, FinalizeRotationIndexing) is recorded the same way, so a live collection is not cancelled by it. - A failed device allocation or cuFFT plan now says how much was asked, how much of the card is free, and - when the card is merely full - that another program is using the GPU or the data set needs more than the card has, with the CPU remedy; previously "Failed to allocate device memory (out of memory)" / "Failed to create CUFFT plan with cufftPlanMany". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
75 lines
3.9 KiB
C++
75 lines
3.9 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <exception>
|
|
#include <optional>
|
|
|
|
#include "../../common/CrystalLattice.h"
|
|
|
|
#include "../../common/DiffractionExperiment.h"
|
|
#include "../../common/JFJochMessages.h"
|
|
#include "../../common/SpotToSave.h"
|
|
#include "SpindleBlindFraction.h"
|
|
|
|
struct IndexerResult {
|
|
std::vector<CrystalLattice> lattice;
|
|
float indexing_time_s;
|
|
bool executed = false; // If indexing was not performed (due to indexer being not-available) mark it, so indexing result is marked accordingly
|
|
// What the indexer threw, when it threw - the exception itself, so its category (a CUDA
|
|
// out-of-memory, say) reaches whoever decides what to do about it. A THIRD outcome, not a flavour
|
|
// of the other two: an empty lattice list with no error is a real negative result about the
|
|
// frame - it was examined and no lattice fits it - whereas an error says nothing about the frame
|
|
// at all, the indexer never having got to look, and will recur on the next one.
|
|
std::exception_ptr error;
|
|
// How much of a single sweep's blind cone this orientation makes unrecoverable, in [0,1].
|
|
// Absent when the frame gives nothing to decide on (no spindle defined, no lattice rows).
|
|
std::optional<float> spindle_blind_fraction;
|
|
std::optional<float> spindle_row_length_A;
|
|
std::optional<float> spindle_miss_angle_deg;
|
|
};
|
|
|
|
class Indexer {
|
|
protected:
|
|
int64_t viable_cell_min_spots = 9;
|
|
float indexing_tolerance = 0.1;
|
|
float dist_tolerance_vs_reference = 0.05;
|
|
bool index_ice_rings = true;
|
|
|
|
DiffractionGeometry geom;
|
|
std::optional<UnitCell> reference_unit_cell;
|
|
// The rotation axis a subsequent sweep would use. Present on a grid scan too, where the axis is
|
|
// defined but stationary - that is exactly the case the spindle severity is for.
|
|
std::optional<Coord> spindle_axis;
|
|
float wavelength_A = 0;
|
|
// Half-angle of the blind double cone a sweep would leave: asin(lambda/2d) at the GEOMETRIC
|
|
// resolution of the setup - the detector corner at the recorded distance and wavelength - not at
|
|
// the still's own spot resolution. A weak, attenuated grid still reaching 3 A would score a row
|
|
// at an 11 deg miss-angle as exactly 0 while the real sweep, at the detector's 1.8 A, loses a
|
|
// fifth of its wider cone. The corner is an upper bound on any sweep collected without moving
|
|
// the detector, it is there on every frame, and it is unbiased where the still's spot list is
|
|
// systematically shallow; a wider cone errs towards reporting loss, which is the cheap error.
|
|
// 0 when the geometry cannot give it.
|
|
float spindle_theta_max_deg = 0;
|
|
|
|
virtual void SetupUnitCell(const std::optional<UnitCell>& cell) = 0;
|
|
virtual std::vector<CrystalLattice> RunInternal(const std::vector<Coord> &coord, size_t nspots) = 0;
|
|
// Set by RunInternal when the implementation computes it; read out by Run.
|
|
virtual std::optional<SpindleSeverity> GetSpindleSeverity() const { return {}; }
|
|
// The row pass alone, over the whole of `coord`, to produce the spindle severity without
|
|
// reducing or refining anything. Implemented by the FFT-family indexers, which own a row
|
|
// search; the others have nothing to answer with and return nothing.
|
|
virtual std::optional<SpindleSeverity> RunSeverityOnly(const std::vector<Coord> &coord) { return {}; }
|
|
public:
|
|
virtual ~Indexer() = default;
|
|
void Setup(const DiffractionExperiment& experiment);
|
|
// severity_only = true runs RunSeverityOnly instead of indexing: no lattice comes back and
|
|
// `executed` stays false, since nothing was an indexing attempt. Used when the seed escalation
|
|
// indexed a frame from fewer spots than the severity's floor - the score must not be absent on
|
|
// precisely the frames that index cleanly.
|
|
IndexerResult Run(const std::vector<Coord> &coord, bool severity_only = false);
|
|
};
|
|
|
|
|