mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 75 Wiki Activity
Files
98132d0f83bb083ca936581b9a2ba9345bdec763
Jungfraujoch/tests/AnalysisSettingsTest.cpp
T
leonarski_fandClaude Opus 5 98132d0f83 analysis: every analysis method carries its own settings, and a raster's indexing is one of them 
AnalysisSettings had begun collecting per-method parameters - the calibrant was already in
it, and the grid thresholds were about to be. That makes the structure every method reads
grow whenever any one method gains a knob, and it puts a field in front of readers for whom
it means nothing. So: AnalysisSettings keeps what all methods share, which for now is the
mode, and each method gets a class of its own bound the same way.

GridScanAnalysisSettings holds the protein-score threshold, the minimum cells per crystal,
the decisive single-cell score, the maximum crystals reported and the indexing switch.
CalibrationSettings holds the calibrant and the ring source. Both sit on
DiffractionExperiment outside the per-run dataset member, both have an Import/Get pair, and
both have their own endpoint - /config/grid_scan_analysis and /config/calibration - which is
how every other settings group in this API is already reached.

Grid indexing is no longer fixed in the stages table. It was turned off there on cost
grounds, and that reasoning does not hold: a raster runs at up to 100 Hz, which the FFT
indexer keeps up with, and a fixed-target serial experiment with a known cell wants ffbidx on
every cell, where a raster that indexes is most of the measurement. So it is a setting, and
DEFAULTS ON. It is additive rather than a change of answer - blobs are still found on the
protein score, so indexing alters nothing about which cells are called crystals and only adds
what was found in them, including the per-cell lattice count, which is the cheapest
multi-lattice or cracked-crystal signal there is.

That makes indexing the one stage a mode does not decide. AnalysisModeStages still carries a
value for it, but only as the setting's default, and DiffractionExperiment::GetAnalysisStages
- which is what every gate reads - substitutes the configured one. The table row is marked so
nobody reads it as the mode's answer.

The calibration knobs stay coupled to the mode but the rule now lives with them:
CalibrationSettings::ApplyToAzimuthalIntegration moves azimuthal integration onto the CPU and
supplies sectors where fewer than four were asked for, carrying the reason with it - the FPGA
integration core holds 2048 bins in total, so 32 sectors leave 64 q bins, which cannot locate
a ring. Stated there because it will otherwise read as an FPGA defect to be fixed back onto
that path, and it is not one: the core is sized for a detector at full rate, and a calibration
exposure is a few images at a few Hz. Both imports apply it, so the order the mode and the
calibration settings are set in does not matter.

CalibrationMethod moves from image_analysis/geom_refinement/PowderCalibration.h into
common/CalibrationSettings.h, which that header now includes. One enum, so the setting and the
code consuming it are not two vocabularies; every existing user sees it unchanged.

The grid thresholds have one home and it is this class. The raster work owns AnalyzeGridScan's
parameter surface and carries PROTEIN_SCORE_THRESHOLD_DEFAULT / MIN_BLOB_CELLS_DEFAULT beside
that header today; the header here states the signature that replaces them, so the two do not
become competing defaults. The beam size deliberately stays a separate argument to
AnalyzeGridScan: it is measured, not configured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-08 07:27:41 +02:00

281 lines
13 KiB
C++
Raw Blame History

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#include "../common/AnalysisSettings.h"
#include "../common/DiffractionExperiment.h"
#include "../frame_serialize/CBORStream2Serializer.h"
#include "../frame_serialize/CBORStream2Deserializer.h"
#include "../writer/FileWriter.h"
#include "../reader/JFJochHDF5Reader.h"
TEST_CASE("AnalysisMode_Default", "[AnalysisMode]") {
// MXStills, not None: a deployment whose configuration predates the mode must keep analysing.
CHECK(AnalysisSettings().GetMode() == AnalysisMode::MXStills);
CHECK(DiffractionExperiment().GetAnalysisMode() == AnalysisMode::MXStills);
}
TEST_CASE("AnalysisMode_Names", "[AnalysisMode]") {
for (auto mode : {AnalysisMode::None, AnalysisMode::MXRotation, AnalysisMode::MXStills,
AnalysisMode::Azint, AnalysisMode::Grid, AnalysisMode::PowderCalibration}) {
auto back = AnalysisModeFromName(AnalysisModeName(mode));
REQUIRE(back.has_value());
CHECK(*back == mode);
}
CHECK_FALSE(AnalysisModeFromName("").has_value());
CHECK_FALSE(AnalysisModeFromName("mx").has_value());
}
TEST_CASE("AnalysisMode_Stages", "[AnalysisMode]") {
// The table itself. Bragg integration must never be on where indexing is off - nothing is
// predicted without a lattice.
for (auto mode : {AnalysisMode::None, AnalysisMode::MXRotation, AnalysisMode::MXStills,
AnalysisMode::Azint, AnalysisMode::Grid, AnalysisMode::PowderCalibration}) {
const auto s = AnalysisModeStages(mode);
CHECK((!s.bragg_integration || s.indexing));
CHECK((!s.scale_merge || s.bragg_integration));
}
CHECK(AnalysisModeStages(AnalysisMode::MXStills).indexing);
CHECK(AnalysisModeStages(AnalysisMode::MXRotation).indexing);
const auto none = AnalysisModeStages(AnalysisMode::None);
CHECK_FALSE(none.spot_finding);
CHECK_FALSE(none.azimuthal_integration);
const auto azint = AnalysisModeStages(AnalysisMode::Azint);
CHECK_FALSE(azint.spot_finding);
CHECK_FALSE(azint.indexing);
CHECK_FALSE(azint.scoring);
CHECK(azint.azimuthal_integration);
// Calibration keeps the spot finder: --calibration spots fits the pooled spots.
const auto calib = AnalysisModeStages(AnalysisMode::PowderCalibration);
CHECK(calib.spot_finding);
CHECK_FALSE(calib.indexing);
// Grid indexing is a setting, so the table only carries its default.
const auto grid = AnalysisModeStages(AnalysisMode::Grid);
CHECK(grid.spot_finding);
CHECK(grid.scoring);
CHECK(grid.indexing == GridScanAnalysisSettings().IsIndexing());
}
TEST_CASE("AnalysisMode_GridIndexingIsASetting", "[AnalysisMode]") {
// The one stage the mode does not fix. The configured value must reach the gates, which read
// GetAnalysisStages and not the table.
DiffractionExperiment experiment;
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Grid));
CHECK(experiment.GetAnalysisStages().indexing); // the default is on
experiment.ImportGridScanAnalysisSettings(GridScanAnalysisSettings().Indexing(false));
CHECK_FALSE(experiment.GetAnalysisStages().indexing);
experiment.ImportGridScanAnalysisSettings(GridScanAnalysisSettings().Indexing(true));
CHECK(experiment.GetAnalysisStages().indexing);
// ...and only under Grid. No other mode's indexing answer moves with it.
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Azint));
experiment.ImportGridScanAnalysisSettings(GridScanAnalysisSettings().Indexing(true));
CHECK_FALSE(experiment.GetAnalysisStages().indexing);
}
TEST_CASE("AnalysisMode_GridScanSettingsDefaults", "[AnalysisMode]") {
const GridScanAnalysisSettings s;
// A single cell is admitted only well above the threshold that admits a cell into a blob;
// otherwise the decisive rule would just be a lower general threshold.
CHECK(s.GetDecisiveSingleCellScore() > s.GetProteinScoreThreshold());
CHECK(s.GetMinBlobCells() >= 1);
CHECK(s.GetMaxCrystals() >= 1);
GridScanAnalysisSettings set;
set.ProteinScoreThreshold(0.7f).MinBlobCells(5).DecisiveSingleCellScore(0.95f)
.MaxCrystals(3).Indexing(false);
CHECK(set.GetProteinScoreThreshold() == Catch::Approx(0.7));
CHECK(set.GetMinBlobCells() == 5);
CHECK(set.GetDecisiveSingleCellScore() == Catch::Approx(0.95));
CHECK(set.GetMaxCrystals() == 3);
CHECK_FALSE(set.IsIndexing());
}
TEST_CASE("AnalysisMode_PrecedenceOverSpotFindingSwitch", "[AnalysisMode]") {
DiffractionExperiment experiment;
DatasetSettings dataset;
dataset.SpotFindingEnable(true).MaxSpotCount(500);
experiment.ImportDatasetSettings(dataset);
CHECK(experiment.IsSpotFindingEnabled());
// A mode that analyses no spots wins over the deprecated per-dataset switch, and takes the spot
// budget with it.
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Azint));
CHECK_FALSE(experiment.IsSpotFindingEnabled());
CHECK(experiment.GetMaxSpotCount() == 0);
// Under a mode that does find spots, the deprecated switch is still able to turn it off.
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::MXStills));
CHECK(experiment.IsSpotFindingEnabled());
dataset.SpotFindingEnable(false);
experiment.ImportDatasetSettings(dataset);
CHECK_FALSE(experiment.IsSpotFindingEnabled());
}
TEST_CASE("AnalysisMode_PowderCalibrationForcesCPUAzInt", "[AnalysisMode]") {
// The FPGA integration core holds 2048 bins in total, so a sectored profile cannot be built there.
DiffractionExperiment experiment;
REQUIRE_FALSE(experiment.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
CHECK(experiment.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
CHECK(experiment.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount()
== CALIBRATION_AZIM_BINS_DEFAULT);
// An explicit sector count stands; the mode only supplies one where none is usable.
DiffractionExperiment explicit_bins;
explicit_bins.ImportAzimuthalIntegrationSettings(
AzimuthalIntegrationSettings().AzimuthalBinCount(64));
explicit_bins.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
CHECK(explicit_bins.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount() == 64);
// The coupling is applied from both imports, so the order the two are set in does not matter.
DiffractionExperiment settings_first;
settings_first.ImportCalibrationSettings(CalibrationSettings().Calibrant("lab6"));
CHECK_FALSE(settings_first.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
settings_first.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
CHECK(settings_first.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
DiffractionExperiment mode_first;
mode_first.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
mode_first.ImportCalibrationSettings(CalibrationSettings().Calibrant("lab6"));
CHECK(mode_first.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
CHECK(mode_first.GetCalibrationSettings().GetCalibrant() == "lab6");
// A mode that is not a calibration is left alone.
DiffractionExperiment stills;
stills.ImportCalibrationSettings(CalibrationSettings().Method(CalibrationMethod::Spots));
CHECK_FALSE(stills.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
CHECK(stills.GetCalibrationSettings().GetMethod() == CalibrationMethod::Spots);
}
TEST_CASE("AnalysisMode_CBORStartRoundTrip", "[AnalysisMode][CBOR]") {
std::vector<uint8_t> buffer(1024 * 1024);
CBORStream2Serializer serializer(buffer.data(), buffer.size());
DiffractionExperiment experiment;
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Grid));
StartMessage message{};
experiment.FillMessage(message);
REQUIRE(message.analysis_mode.has_value());
REQUIRE_NOTHROW(serializer.SerializeSequenceStart(message));
auto deserialized = CBORStream2Deserialize(buffer.data(), serializer.GetBufferSize());
REQUIRE(deserialized);
REQUIRE(deserialized->start_message);
REQUIRE(deserialized->start_message->analysis_mode.has_value());
CHECK(*deserialized->start_message->analysis_mode == AnalysisMode::Grid);
}
TEST_CASE("AnalysisMode_CBORStartAbsentMode", "[AnalysisMode][CBOR]") {
// A stream written before the mode existed says nothing, and is read back as saying nothing -
// naming a mode for it would be an invention rather than provenance.
std::vector<uint8_t> buffer(1024 * 1024);
CBORStream2Serializer serializer(buffer.data(), buffer.size());
StartMessage message{};
REQUIRE_NOTHROW(serializer.SerializeSequenceStart(message));
auto deserialized = CBORStream2Deserialize(buffer.data(), serializer.GetBufferSize());
REQUIRE(deserialized);
REQUIRE(deserialized->start_message);
CHECK_FALSE(deserialized->start_message->analysis_mode.has_value());
}
TEST_CASE("AnalysisMode_CBORGridCrystals", "[AnalysisMode][CBOR]") {
std::vector<uint8_t> buffer(1024 * 1024);
CBORStream2Serializer serializer(buffer.data(), buffer.size());
EndMessage message{};
message.max_image_number = 0;
// Two crystals, not one: a raster can find any number and nothing may assume at most one.
message.grid_crystals.push_back(GridScanCrystal{
.nx = 3.5f, .ny = 7.25f, .x_um = -12.5f, .y_um = 40.0f, .image_number = 143,
.major_um = 22.5f, .minor_um = 8.0f, .angle_deg = 179.5f, .score = 0.92f,
.ice_score = 0.03f, .res_A = 1.85f, .n_images = 17});
message.grid_crystals.push_back(GridScanCrystal{
.nx = 11.0f, .ny = 2.0f, .x_um = 60.0f, .y_um = -30.0f, .image_number = 44,
.major_um = 9.0f, .minor_um = 7.5f, .angle_deg = 0.5f, .score = 0.41f,
.ice_score = 0.30f, .res_A = 3.2f, .n_images = 4});
REQUIRE_NOTHROW(serializer.SerializeSequenceEnd(message));
auto deserialized = CBORStream2Deserialize(buffer.data(), serializer.GetBufferSize());
REQUIRE(deserialized);
REQUIRE(deserialized->end_message);
const auto &out = deserialized->end_message->grid_crystals;
REQUIRE(out.size() == 2);
CHECK(out[0].nx == Catch::Approx(3.5));
CHECK(out[0].image_number == 143);
CHECK(out[0].major_um == Catch::Approx(22.5));
CHECK(out[0].angle_deg == Catch::Approx(179.5));
CHECK(out[0].res_A == Catch::Approx(1.85));
CHECK(out[0].n_images == 17);
CHECK(out[1].score == Catch::Approx(0.41));
CHECK(out[1].ice_score == Catch::Approx(0.30));
CHECK(out[1].y_um == Catch::Approx(-30.0));
}
TEST_CASE("AnalysisMode_HDF5MasterRoundTrip", "[AnalysisMode][HDF5][Full]") {
// The mode and the crystal list are dataset-wide metadata: written to the master, read back from
// it, so a stored raster re-opens knowing what produced it and what it found.
DiffractionExperiment x(DetJF(1));
x.FilePrefix("test_analysis_mode").ImagesPerTrigger(1).OverwriteExistingFiles(true);
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
.IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
x.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Grid));
RegisterHDF5Filter();
std::vector<uint16_t> image(x.GetPixelsNum(), 0);
{
StartMessage start_message;
x.FillMessage(start_message);
FileWriter file_set(start_message);
DataMessage message{};
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
message.number = 0;
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
EndMessage end_message;
end_message.max_image_number = 1;
end_message.grid_crystals.push_back(GridScanCrystal{
.nx = 3.5f, .ny = 7.25f, .x_um = -12.5f, .y_um = 40.0f, .image_number = 143,
.major_um = 22.5f, .minor_um = 8.0f, .angle_deg = 179.5f, .score = 0.92f,
.ice_score = 0.03f, .res_A = 1.85f, .n_images = 17});
end_message.grid_crystals.push_back(GridScanCrystal{
.nx = 11.0f, .ny = 2.0f, .x_um = 60.0f, .y_um = -30.0f, .image_number = 44,
.major_um = 9.0f, .minor_um = 7.5f, .angle_deg = 0.5f, .score = 0.41f,
.ice_score = 0.30f, .res_A = 3.2f, .n_images = 4});
REQUIRE_NOTHROW(file_set.WriteHDF5(end_message));
file_set.Finalize();
}
{
JFJochHDF5Reader reader;
reader.ReadFile("test_analysis_mode_master.h5");
auto dataset = reader.GetDataset();
REQUIRE(dataset->file_analysis_mode.has_value());
CHECK(*dataset->file_analysis_mode == AnalysisMode::Grid);
REQUIRE(dataset->grid_crystals.size() == 2);
CHECK(dataset->grid_crystals[0].nx == Catch::Approx(3.5));
CHECK(dataset->grid_crystals[0].image_number == 143);
CHECK(dataset->grid_crystals[0].angle_deg == Catch::Approx(179.5));
CHECK(dataset->grid_crystals[0].n_images == 17);
CHECK(dataset->grid_crystals[1].score == Catch::Approx(0.41));
CHECK(dataset->grid_crystals[1].res_A == Catch::Approx(3.2));
}
remove("test_analysis_mode_master.h5");
remove("test_analysis_mode_data_000001.h5");
}
Reference in New Issue View Git Blame Copy Permalink