Until now nothing in the tree said what analysis a run performed. The answer was composed
at each site out of four independent scalars - the detector type, two separate "spot finding
off" switches, an indexing flag and a rotation flag - so what was configured and what
actually ran were different things, and no single place could be read to find out which.
AnalysisMode {None, MXRotation, MXStills, Azint, Grid, PowderCalibration} is that statement,
in common/ because all three programs configure the DiffractionExperiment that carries it.
AnalysisSettings sits on the experiment beside IndexingSettings, outside the DatasetSettings
member, which is the one thing a /start replaces wholesale - so the mode is persistent by
construction rather than by a rule someone has to remember.
The mode does not label a run, it decides it. AnalysisModeStages() is a table - modes as
rows, pipeline stages as columns - and every gate reads that table instead of testing the
mode: spot finding in DiffractionExperiment::IsSpotFindingEnabled, indexing (and with it
prediction and integration, which never run without a lattice) in one gate inside
IndexAndRefine that serves all three front ends, azimuthal integration where the CPU engine
is built. Two rows carry a judgement worth reviewing: powder calibration keeps spot finding,
because --calibration spots fits the pooled spots; grid does not index, because a raster is
thousands of frames and the per-image scoring it ranks on deliberately avoids an indexer that
fires on ice.
There is deliberately no Auto value. GetIndexingAlgorithm() resolves Auto at read time, which
is exactly why an indexing setting cannot be read back off the configuration; removing that
kind of implicitness is the point here, so the mode getter stays a plain accessor. MXStills
is the default because None would silently switch analysis off on every deployment whose
configuration predates the field.
Rotation MX is absent from the OpenAPI schema rather than present and refused: jfjoch_broker
has no rotation analysis path, so the REST and configuration-file routes cannot express it at
all. The shared enum can still carry the value from elsewhere, so CheckAnalysisSettingsOnline
refuses it on both routes with a message naming rugnux. A sweep collected under an MX mode is
not refused - collecting rotation data online is normal and live spot counts are useful - but
it is said out loud in the log, since the mistake worth preventing is the silence about what
was done to it, not the acquisition.
Powder calibration forces azimuthal integration onto the CPU and supplies 32 sectors where
fewer than four were asked for. The FPGA integration core holds 2048 bins in total, so 32
sectors would leave 64 q bins - far too coarse to fit a ring. Frame rate is what this costs
and a calibration exposure does not need it.
The two existing "no analysis" switches, per-dataset dataset_settings.spot_finding and
persistent SpotFindingSettings::enable, are interfaces in too many places to remove now. They
are marked deprecated in the schema and in both headers, and the mode takes precedence over
them: a mode that analyses no spots wins outright, while under a mode that does find spots
they remain the finer control. The precedence is written where it is enforced.
rugnux's ProcessMode is gone, replaced by the shared enum; RugnuxMode stays as the CLI
spelling layer and no existing spelling changes. --mode gains mx_rotation and mx_stills, which
are spellings of -R and --force-still rather than new switches; plain mx still chooses between
them from the goniometer. scale keeps no shared counterpart, since it runs no analysis over
images at all.
The mode reaches the CBOR start message and /entry/MX/analysis_mode in the HDF5 master, so a
written file records which analysis produced it. It is read back as provenance only - what a
stored file was produced by is not what the next run should do.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
244 lines
9.7 KiB
C++
244 lines
9.7 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include <catch2/catch_all.hpp>
|
|
#include <cmath>
|
|
#include <filesystem>
|
|
|
|
#include "../common/DiffractionExperiment.h"
|
|
#include "../common/ScanResultGenerator.h"
|
|
#include "../writer/FileWriter.h"
|
|
#include "../reader/JFJochHDF5Reader.h"
|
|
#include "../rugnux/Rugnux.h"
|
|
#include "../rugnux/RugnuxCommandLine.h"
|
|
#include "../rugnux/SpotWidth.h"
|
|
|
|
namespace {
|
|
// Write a small VDS dataset of `n` flat images and return nothing (prefix_master.h5 +
|
|
// prefix_data_000001.h5 land in the test working directory).
|
|
void WriteTestDataset(const std::string &prefix, int n) {
|
|
RegisterHDF5Filter();
|
|
|
|
DiffractionExperiment x(DetJF(1));
|
|
x.FilePrefix(prefix).ImagesPerTrigger(n).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(n).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
x.BeamX_pxl(512).BeamY_pxl(256).DetectorDistance_mm(150).IncidentEnergy_keV(WVL_1A_IN_KEV)
|
|
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 5);
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
ScanResultGenerator generator(x);
|
|
for (int i = 0; i < n; i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = n;
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
}
|
|
|
|
TEST_CASE("Rugnux_AzInt", "[HDF5][Full]") {
|
|
WriteTestDataset("process_azint_in", 8);
|
|
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("process_azint_in_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
REQUIRE(dataset);
|
|
|
|
ProcessConfig config;
|
|
config.mode = AnalysisMode::Azint;
|
|
config.nthreads = 2;
|
|
config.output_prefix = "process_azint_out";
|
|
|
|
Rugnux process(reader, dataset->experiment, *dataset->pixel_mask, config);
|
|
ProcessResult result;
|
|
REQUIRE_NOTHROW(result = process.Run());
|
|
|
|
CHECK_FALSE(result.cancelled);
|
|
CHECK(result.images_processed == 8);
|
|
REQUIRE(result.written_master_path.has_value());
|
|
|
|
{
|
|
// The _process.h5 links back to the source images and carries an azimuthal profile per image.
|
|
JFJochHDF5Reader out;
|
|
REQUIRE_NOTHROW(out.ReadFile("process_azint_out_process.h5"));
|
|
CHECK(out.GetNumberOfImages() == 8);
|
|
std::shared_ptr<JFJochReaderImage> img;
|
|
REQUIRE_NOTHROW(img = out.LoadImage(0));
|
|
REQUIRE(img);
|
|
CHECK_FALSE(img->ImageData().az_int_profile.empty());
|
|
}
|
|
|
|
reader.Close();
|
|
remove("process_azint_in_master.h5");
|
|
remove("process_azint_in_data_000001.h5");
|
|
remove("process_azint_out_process.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("Rugnux_NoOutput", "[HDF5][Full]") {
|
|
WriteTestDataset("process_noout_in", 6);
|
|
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("process_noout_in_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
|
|
// Empty output prefix => process without writing any file.
|
|
ProcessConfig config;
|
|
config.mode = AnalysisMode::Azint;
|
|
config.nthreads = 3;
|
|
|
|
Rugnux process(reader, dataset->experiment, *dataset->pixel_mask, config);
|
|
auto result = process.Run();
|
|
|
|
CHECK_FALSE(result.cancelled);
|
|
CHECK(result.images_processed == 6);
|
|
CHECK_FALSE(result.written_master_path.has_value());
|
|
|
|
reader.Close();
|
|
remove("process_noout_in_master.h5");
|
|
remove("process_noout_in_data_000001.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("Rugnux_Cancel", "[HDF5][Full]") {
|
|
WriteTestDataset("process_cancel_in", 8);
|
|
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("process_cancel_in_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
|
|
ProcessConfig config;
|
|
config.mode = AnalysisMode::Azint;
|
|
config.nthreads = 2;
|
|
|
|
Rugnux process(reader, dataset->experiment, *dataset->pixel_mask, config);
|
|
process.Cancel(); // cancel before running: the worker loop stops immediately
|
|
auto result = process.Run();
|
|
|
|
CHECK(result.cancelled);
|
|
CHECK(result.images_processed == 0);
|
|
CHECK_FALSE(result.written_master_path.has_value());
|
|
|
|
reader.Close();
|
|
remove("process_cancel_in_master.h5");
|
|
remove("process_cancel_in_data_000001.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("RugnuxCommandLine_Full", "[process]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
IndexingSettings idx;
|
|
idx.Algorithm(IndexingAlgorithmEnum::FFT);
|
|
idx.GeomRefinementAlgorithm(GeomRefinementAlgorithmEnum::BeamCenter);
|
|
x.ImportIndexingSettings(idx);
|
|
x.SpaceGroupNumber(96);
|
|
|
|
ProcessConfig config;
|
|
config.mode = AnalysisMode::MXStills;
|
|
config.nthreads = 8;
|
|
config.output_prefix = "run1";
|
|
config.end_image = 500;
|
|
config.rotation_indexing = true;
|
|
config.two_pass_rotation = true;
|
|
config.rotation_indexing_image_count = 30;
|
|
config.spot_finding = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
|
|
const std::string cmd = RugnuxCommandLine(config, x, "/data/test_master.h5");
|
|
CHECK(cmd.rfind("rugnux", 0) == 0);
|
|
CHECK(cmd.find("-N 8") != std::string::npos);
|
|
CHECK(cmd.find("-e 500") != std::string::npos);
|
|
CHECK(cmd.find("-o run1") != std::string::npos);
|
|
CHECK(cmd.find("-X fft") != std::string::npos);
|
|
CHECK(cmd.find("-S 96") != std::string::npos);
|
|
// -R takes an optional argument, so its value must be attached (-R30); a separate "-R 30" token
|
|
// would not re-parse (getopt would leave 30 as a positional and drop the count).
|
|
CHECK(cmd.find("-R30") != std::string::npos);
|
|
CHECK(cmd.find("-R 30") == std::string::npos);
|
|
CHECK(cmd.find("/data/test_master.h5") != std::string::npos);
|
|
}
|
|
|
|
TEST_CASE("RugnuxCommandLine_AzInt", "[process]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
AzimuthalIntegrationSettings a;
|
|
a.AzimuthalBinCount(4);
|
|
x.ImportAzimuthalIntegrationSettings(a);
|
|
|
|
ProcessConfig config;
|
|
config.mode = AnalysisMode::Azint;
|
|
config.nthreads = 2;
|
|
config.output_prefix = "az";
|
|
|
|
const std::string cmd = RugnuxCommandLine(config, x, "in.h5");
|
|
CHECK(cmd.rfind("rugnux", 0) == 0);
|
|
CHECK(cmd.find("--mode azint") != std::string::npos);
|
|
CHECK(cmd.find("--azim-phi-bins 4") != std::string::npos);
|
|
CHECK(cmd.find("--azim-min-q") != std::string::npos);
|
|
CHECK(cmd.find("in.h5") != std::string::npos);
|
|
}
|
|
|
|
namespace {
|
|
// A field of identical round Gaussian spots on three rings, so that the width estimator sees
|
|
// several resolution bands with the same true width and its 1/d fit has to come back flat.
|
|
void PaintGaussianSpots(ImagePreprocessorBuffer &image, int w, double sigma, double total_counts,
|
|
std::vector<DiffractionSpot> &spots) {
|
|
constexpr int BKG = 3;
|
|
for (size_t i = 0; i < image.size(); i++) image[i] = BKG;
|
|
const double amp = total_counts / (2.0 * M_PI * sigma * sigma);
|
|
for (int radius : {150, 350, 550})
|
|
for (int k = 0; k < 20; k++) {
|
|
const double phi = 2.0 * M_PI * k / 20.0 + 0.1 * radius;
|
|
const int cx = static_cast<int>(std::lround(600 + radius * std::cos(phi)));
|
|
const int cy = static_cast<int>(std::lround(600 + radius * std::sin(phi)));
|
|
for (int dy = -14; dy <= 14; dy++)
|
|
for (int dx = -14; dx <= 14; dx++)
|
|
image[static_cast<size_t>(cy + dy) * w + (cx + dx)] +=
|
|
static_cast<int32_t>(std::lround(
|
|
amp * std::exp(-(dx * dx + dy * dy) / (2.0 * sigma * sigma))));
|
|
spots.emplace_back(static_cast<uint32_t>(cx), static_cast<uint32_t>(cy),
|
|
static_cast<int64_t>(total_counts));
|
|
}
|
|
}
|
|
}
|
|
|
|
// The width the adaptive integration radius is set from. A round Gaussian of width sigma holds 80 %
|
|
// of its flux inside sqrt(2 ln 5) * sigma = 1.794 * sigma, and that is what the estimator has to
|
|
// return - over an aperture that owes nothing to the integrator's r1, which is the whole point of
|
|
// measuring it here rather than reading the integrator's own second moment.
|
|
TEST_CASE("SpotWidth_Gaussian", "[process]") {
|
|
constexpr int W = 1200, H = 1200;
|
|
DiffractionGeometry geometry;
|
|
geometry.BeamX_pxl(600).BeamY_pxl(600).DetectorDistance_mm(200).PixelSize_mm(0.075)
|
|
.Wavelength_A(1.0);
|
|
|
|
for (double sigma : {1.0, 2.2}) {
|
|
ImagePreprocessorBuffer image(static_cast<size_t>(W) * H);
|
|
std::vector<DiffractionSpot> spots;
|
|
PaintGaussianSpots(image, W, sigma, 20000.0, spots);
|
|
|
|
std::vector<spot_width::FluxCurve> curves;
|
|
MeasureSpotFluxCurves(image, W, H, geometry, spots, curves);
|
|
REQUIRE(curves.size() >= 45);
|
|
|
|
const auto r80 = spot_width::R80AtReference(curves);
|
|
REQUIRE(r80.has_value());
|
|
CHECK(*r80 == Catch::Approx(1.794 * sigma).margin(0.3));
|
|
}
|
|
|
|
// The rule the measurement drives: the shipped radius below the line, the capped one above it.
|
|
CHECK(spot_width::R1ForWidth(1.0f) == 4.0f);
|
|
CHECK(spot_width::R1ForWidth(1.794f) == 4.0f);
|
|
CHECK(spot_width::R1ForWidth(2.4f) == 5.0f);
|
|
CHECK(spot_width::R1ForWidth(3.947f) == 6.0f);
|
|
CHECK(spot_width::R1ForWidth(9.0f) == 6.0f);
|
|
}
|