leonarski_fandClaude Opus 5
b5f5879a1d
Ice handling was gated on a measurement the run only made AFTER the images had been processed, so the per-image pass could not use it. The flagging therefore ran unconditionally: ice-band spots were ordered last in the --max-spots budget and held out of the indexer seed and the geometry refinement on every crystal, iced or not. The eleven bands are fixed geometry holding 16-26 % of the unique reflections whether or not there is ice, so on a clean crystal that discards a fifth of the spots - the strongest first - for nothing. Measured on a crystal whose gate never fires, that moved the merged data by a mean of 0.85 sigma against a run-to-run floor of 9.3e-5. Measure it in the first pass instead. That pass already looks at ~100 images spread over the sweep, and it already stops at the spot finder, so it sees the azimuthal profile for the smooth channel and the unfiltered connected components for the spot channel. Both counts SpotAnalyze takes are pre-filter, so pooling them there is the run's own verdict, reached before anything has been discarded and in time for the pass that acts on it. Where the sample sees no ice, the run indexes on the ice-band spots too. It has to be the whole sample: the spot channel is a ratio pooled over images, because one frame carries a handful of control spots. A per-image gate is not an alternative - two of the crystals whose indexing this rescues fire on that channel alone, at profile scores of 1.12 and 1.22, so gating per image on the profile score would drop exactly the cases that matter. This also removes the first-pass spot reuse, and with it --redo-rotation-spots and the reuse path. Finding the ~100 first-pass spots costs little, and reusing was actively wrong here: the stored spots were found online at the acquisition's threshold and have already had their ice-band entries ordered last and dropped by its spot budget, so counting ice from them under-reads it by construction, and the lattice search never saw the spot-finding settings at all. It also removes the need for the machinery that re-found spots whenever a spot-finding option was named, which made those options impossible to A/B. IndexAndRefine cached index_ice_rings at construction, which happens before the first pass; it holds a reference to the experiment, so it now reads the setting where it uses it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
82 lines
3.1 KiB
C++
82 lines
3.1 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
// End-to-end Rugnux runs over real JUNGFRAU datasets that are kept in git-LFS under
|
|
// tests/data. They are tagged [large] and SKIP() when the data is not present (e.g. LFS not
|
|
// pulled), so the default test run stays fast and CI without the data still passes.
|
|
|
|
#include <catch2/catch_all.hpp>
|
|
#include <catch2/reporters/catch_reporter_event_listener.hpp>
|
|
#include <catch2/reporters/catch_reporter_registrars.hpp>
|
|
|
|
#include <iostream>
|
|
#include <thread>
|
|
|
|
#include "TestData.h"
|
|
#include "../common/DiffractionExperiment.h"
|
|
#include "../common/IndexingSettings.h"
|
|
#include "../reader/JFJochHDF5Reader.h"
|
|
#include "../rugnux/Rugnux.h"
|
|
|
|
namespace {
|
|
// Start-up hook: report once whether the large datasets are available, so it is obvious why
|
|
// the [large] tests skip when they do.
|
|
class LargeDataListener : public Catch::EventListenerBase {
|
|
public:
|
|
using Catch::EventListenerBase::EventListenerBase;
|
|
void testRunStarting(Catch::TestRunInfo const &) override {
|
|
const bool rot = jfjoch_test::LargeDataFile("rotation_master.h5").has_value();
|
|
std::cout << "[jfjoch_test] large dataset in " << jfjoch_test::LargeDataDir()
|
|
<< ": rotation=" << (rot ? "yes" : "no")
|
|
<< " ([large] tests skip when absent)" << std::endl;
|
|
}
|
|
};
|
|
|
|
int default_threads() {
|
|
const unsigned hc = std::thread::hardware_concurrency();
|
|
return hc == 0 ? 4 : static_cast<int>(hc);
|
|
}
|
|
}
|
|
|
|
CATCH_REGISTER_LISTENER(LargeDataListener)
|
|
|
|
TEST_CASE("Rugnux_Rotation", "[large]") {
|
|
const auto master = jfjoch_test::LargeDataFile("rotation_master.h5");
|
|
if (!master)
|
|
SKIP("rotation_master.h5 not available (git-lfs data not pulled)");
|
|
|
|
RegisterHDF5Filter();
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile(*master));
|
|
auto dataset = reader.GetDataset();
|
|
REQUIRE(dataset);
|
|
|
|
DiffractionExperiment experiment(dataset->experiment);
|
|
IndexingSettings indexing;
|
|
indexing.Algorithm(IndexingAlgorithmEnum::Auto);
|
|
indexing.RotationIndexing(true);
|
|
indexing.GeomRefinementAlgorithm(GeomRefinementAlgorithmEnum::BeamCenter);
|
|
experiment.ImportIndexingSettings(indexing);
|
|
|
|
ProcessConfig config;
|
|
config.mode = ProcessMode::FullAnalysis;
|
|
config.nthreads = default_threads();
|
|
config.spot_finding = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
config.spot_finding.indexing = true;
|
|
config.rotation_indexing = true;
|
|
config.two_pass_rotation = true;
|
|
|
|
Rugnux process(reader, experiment, *dataset->pixel_mask, config);
|
|
ProcessResult result;
|
|
REQUIRE_NOTHROW(result = process.Run());
|
|
|
|
CHECK_FALSE(result.cancelled);
|
|
CHECK(result.images_processed == reader.GetNumberOfImages());
|
|
REQUIRE(result.indexing_rate.has_value());
|
|
CHECK(result.indexing_rate.value() > 0.1f); // a real rotation series indexes well
|
|
CHECK(result.consensus_cell.has_value());
|
|
|
|
reader.Close();
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|