analysis: what runs over the images is one stated mode, shared by broker, rugnux and viewer
Until now nothing in the tree said what analysis a run performed. The answer was composed
at each site out of four independent scalars - the detector type, two separate "spot finding
off" switches, an indexing flag and a rotation flag - so what was configured and what
actually ran were different things, and no single place could be read to find out which.
AnalysisMode {None, MXRotation, MXStills, Azint, Grid, PowderCalibration} is that statement,
in common/ because all three programs configure the DiffractionExperiment that carries it.
AnalysisSettings sits on the experiment beside IndexingSettings, outside the DatasetSettings
member, which is the one thing a /start replaces wholesale - so the mode is persistent by
construction rather than by a rule someone has to remember.
The mode does not label a run, it decides it. AnalysisModeStages() is a table - modes as
rows, pipeline stages as columns - and every gate reads that table instead of testing the
mode: spot finding in DiffractionExperiment::IsSpotFindingEnabled, indexing (and with it
prediction and integration, which never run without a lattice) in one gate inside
IndexAndRefine that serves all three front ends, azimuthal integration where the CPU engine
is built. Two rows carry a judgement worth reviewing: powder calibration keeps spot finding,
because --calibration spots fits the pooled spots; grid does not index, because a raster is
thousands of frames and the per-image scoring it ranks on deliberately avoids an indexer that
fires on ice.
There is deliberately no Auto value. GetIndexingAlgorithm() resolves Auto at read time, which
is exactly why an indexing setting cannot be read back off the configuration; removing that
kind of implicitness is the point here, so the mode getter stays a plain accessor. MXStills
is the default because None would silently switch analysis off on every deployment whose
configuration predates the field.
Rotation MX is absent from the OpenAPI schema rather than present and refused: jfjoch_broker
has no rotation analysis path, so the REST and configuration-file routes cannot express it at
all. The shared enum can still carry the value from elsewhere, so CheckAnalysisSettingsOnline
refuses it on both routes with a message naming rugnux. A sweep collected under an MX mode is
not refused - collecting rotation data online is normal and live spot counts are useful - but
it is said out loud in the log, since the mistake worth preventing is the silence about what
was done to it, not the acquisition.
Powder calibration forces azimuthal integration onto the CPU and supplies 32 sectors where
fewer than four were asked for. The FPGA integration core holds 2048 bins in total, so 32
sectors would leave 64 q bins - far too coarse to fit a ring. Frame rate is what this costs
and a calibration exposure does not need it.
The two existing "no analysis" switches, per-dataset dataset_settings.spot_finding and
persistent SpotFindingSettings::enable, are interfaces in too many places to remove now. They
are marked deprecated in the schema and in both headers, and the mode takes precedence over
them: a mode that analyses no spots wins outright, while under a mode that does find spots
they remain the finer control. The precedence is written where it is enforced.
rugnux's ProcessMode is gone, replaced by the shared enum; RugnuxMode stays as the CLI
spelling layer and no existing spelling changes. --mode gains mx_rotation and mx_stills, which
are spellings of -R and --force-still rather than new switches; plain mx still chooses between
them from the goniometer. scale keeps no shared counterpart, since it runs no analysis over
images at all.
The mode reaches the CBOR start message and /entry/MX/analysis_mode in the HDF5 master, so a
written file records which analysis produced it. It is read back as provenance only - what a
stored file was produced by is not what the next run should do.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
This commit is contained in:
1 parent
b650cda037
commit
a7d3ada3ab
41 files changed
+870
-65
No files matched your search
@@ -204,6 +204,8 @@ void JFJochBrokerHttp::register_routes(httplib::Server &server) {
|
||||
server.Put("/config/indexing", bind_json(&JFJochBrokerHttp::config_indexing_put, Indexing_settings{}));
|
||||
server.Get("/config/bragg_integration", bind_noarg(&JFJochBrokerHttp::config_bragg_integration_get));
|
||||
server.Put("/config/bragg_integration", bind_json(&JFJochBrokerHttp::config_bragg_integration_put, Bragg_integration_settings{}));
|
||||
server.Get("/config/analysis", bind_noarg(&JFJochBrokerHttp::config_analysis_get));
|
||||
server.Put("/config/analysis", bind_json(&JFJochBrokerHttp::config_analysis_put, Analysis_settings{}));
|
||||
server.Get("/config/instrument", bind_noarg(&JFJochBrokerHttp::config_instrument_get));
|
||||
server.Put("/config/instrument", bind_json(&JFJochBrokerHttp::config_instrument_put, Instrument_metadata{}));
|
||||
server.Put("/config/internal_generator_image", [this](const httplib::Request &req, httplib::Response &res) {
|
||||
@@ -601,6 +603,7 @@ void JFJochBrokerHttp::statistics_get(httplib::Response &response) {
|
||||
statistics.setBuffer(Convert(state_machine.GetImageBufferStatus()));
|
||||
statistics.setIndexing(Convert(state_machine.GetIndexingSettings()));
|
||||
statistics.setBraggIntegration(Convert(state_machine.GetBraggIntegrationSettings()));
|
||||
statistics.setAnalysis(Convert(state_machine.GetAnalysisSettings()));
|
||||
statistics.setDarkMask(Convert(state_machine.GetDarkMaskSettings()));
|
||||
statistics.setImagePusher(Convert(state_machine.GetImagePusherStatus()));
|
||||
|
||||
@@ -845,6 +848,16 @@ void JFJochBrokerHttp::config_bragg_integration_put(const Bragg_integration_sett
|
||||
response.status = 200;
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::config_analysis_get(httplib::Response &response) {
|
||||
ProcessOutput(Convert(state_machine.GetAnalysisSettings()), response);
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::config_analysis_put(const Analysis_settings &analysisSettings,
|
||||
httplib::Response &response) {
|
||||
state_machine.SetAnalysisSettings(Convert(analysisSettings));
|
||||
response.status = 200;
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::result_scan_get(httplib::Response &response) {
|
||||
auto ret = state_machine.GetScanResult();
|
||||
if (ret.has_value())
|
||||
|
||||
@@ -181,6 +181,9 @@ class JFJochBrokerHttp {
|
||||
void config_bragg_integration_get(httplib::Response &response);
|
||||
void config_bragg_integration_put(const org::openapitools::server::model::Bragg_integration_settings &braggIntegrationSettings,
|
||||
httplib::Response &response);
|
||||
void config_analysis_get(httplib::Response &response);
|
||||
void config_analysis_put(const org::openapitools::server::model::Analysis_settings &analysisSettings,
|
||||
httplib::Response &response);
|
||||
|
||||
void config_dark_mask_get(httplib::Response &response);
|
||||
void config_dark_mask_put(const org::openapitools::server::model::Dark_mask_settings &darkMaskSettings,
|
||||
|
||||
@@ -188,6 +188,12 @@ void ParseFacilityConfiguration(const org::openapitools::server::model::Jfjoch_s
|
||||
experiment.ImportBraggIntegrationSettings(
|
||||
BraggIntegrationSettings().MaxHKL(BRAGG_ONLINE_DEFAULT_MAX_HKL));
|
||||
|
||||
if (j.analysisIsSet()) {
|
||||
auto analysis = Convert(j.getAnalysis());
|
||||
CheckAnalysisSettingsOnline(analysis);
|
||||
experiment.ImportAnalysisSettings(analysis);
|
||||
}
|
||||
|
||||
if (j.darkMaskIsSet())
|
||||
experiment.ImportDarkMaskSettings(Convert(j.getDarkMask()));
|
||||
}
|
||||
|
||||
@@ -399,6 +399,18 @@ void JFJochStateMachine::Start(const DatasetSettings &settings, bool async) {
|
||||
|
||||
experiment.ImportDatasetSettings(settings);
|
||||
|
||||
// A sweep arriving under an MX mode. The mode says stills - it is the only MX mode this API can
|
||||
// express - but the goniometer says the crystal turns, and analysing a sweep frame by frame is a
|
||||
// different measurement from the one the data support. It is said out loud rather than refused:
|
||||
// collecting rotation data here is normal and per-image spot counts are useful live feedback, so
|
||||
// the mistake worth preventing is not the acquisition but the silence about what was done to it.
|
||||
if (experiment.GetAnalysisSettings().IsMX()
|
||||
&& experiment.GetGoniometer().has_value() && experiment.GetGoniometer()->IsScanning())
|
||||
logger.Warning("Analysis mode is {} but the goniometer scans: this sweep is analysed frame by "
|
||||
"frame as stills. jfjoch_broker has no rotation analysis - process the stored "
|
||||
"file with rugnux for that.",
|
||||
AnalysisModeName(experiment.GetAnalysisMode()));
|
||||
|
||||
cancel_sequence = false;
|
||||
if (experiment.GetStorageCellNumber() == 1)
|
||||
experiment.StorageCellStart(15);
|
||||
@@ -1210,6 +1222,36 @@ void JFJochStateMachine::SetBraggIntegrationSettings(const BraggIntegrationSetti
|
||||
experiment.ImportBraggIntegrationSettings(input);
|
||||
}
|
||||
|
||||
void CheckAnalysisSettingsOnline(const AnalysisSettings &settings) {
|
||||
// The REST schema has no mx_rotation, so this can only arrive from a caller built against a wider
|
||||
// vocabulary than this API offers. Refuse it rather than accept it and quietly run stills: an
|
||||
// analysis silently replaced by a different one is indistinguishable from the one that was asked
|
||||
// for, and jfjoch_broker has no rotation analysis path to run.
|
||||
if (settings.GetMode() == AnalysisMode::MXRotation)
|
||||
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
|
||||
"Rotation MX analysis is not available online: jfjoch_broker has no rotation "
|
||||
"analysis path. Collect with mx_stills, azint or none, and process the "
|
||||
"stored file with rugnux");
|
||||
}
|
||||
|
||||
AnalysisSettings JFJochStateMachine::GetAnalysisSettings() const {
|
||||
std::unique_lock ul(experiment_indexing_settings_mutex);
|
||||
return experiment.GetAnalysisSettings();
|
||||
}
|
||||
|
||||
void JFJochStateMachine::SetAnalysisSettings(const AnalysisSettings &input) {
|
||||
std::unique_lock ul(m);
|
||||
if (IsRunning())
|
||||
throw WrongDAQStateException("Cannot change analysis settings during data collection");
|
||||
|
||||
CheckAnalysisSettingsOnline(input);
|
||||
|
||||
// The analysis engines read the mode off the experiment when they are built at the start of the
|
||||
// next run, so importing it here is all that is needed.
|
||||
std::unique_lock ul2(experiment_indexing_settings_mutex);
|
||||
experiment.ImportAnalysisSettings(input);
|
||||
}
|
||||
|
||||
std::optional<ScanResult> JFJochStateMachine::GetScanResult() const {
|
||||
std::unique_lock ul(m);
|
||||
if (IsRunning())
|
||||
|
||||
@@ -72,6 +72,11 @@ struct MeasurementStatistics {
|
||||
std::optional<float> roi_beam_sum;
|
||||
};
|
||||
|
||||
// What jfjoch_broker will accept as an analysis mode. Both routes to the setting - the /config/analysis
|
||||
// PUT and the facility configuration file - call it, so a config file cannot pin a mode the API would
|
||||
// refuse and leave the broker in a state no client could have set. Throws InputParameterInvalid.
|
||||
void CheckAnalysisSettingsOnline(const AnalysisSettings &settings);
|
||||
|
||||
class JFJochStateMachine {
|
||||
Logger &logger;
|
||||
JFJochServices &services;
|
||||
@@ -244,6 +249,9 @@ public:
|
||||
|
||||
void SetBraggIntegrationSettings(const BraggIntegrationSettings &input);
|
||||
BraggIntegrationSettings GetBraggIntegrationSettings() const;
|
||||
|
||||
void SetAnalysisSettings(const AnalysisSettings &input);
|
||||
AnalysisSettings GetAnalysisSettings() const;
|
||||
PixelMaskStatistics GetPixelMaskStatistics() const;
|
||||
|
||||
void GetStartMessageFromBuffer(std::vector<uint8_t> &v);
|
||||
|
||||
@@ -1120,6 +1120,69 @@ org::openapitools::server::model::Bragg_integration_settings Convert(const Bragg
|
||||
return ret;
|
||||
}
|
||||
|
||||
AnalysisSettings Convert(const org::openapitools::server::model::Analysis_settings &input) {
|
||||
AnalysisSettings ret;
|
||||
// No default: the compiler must object here when a mode is added to the API, rather than let the
|
||||
// new value fall through to whatever the last branch was. MXRotation is not in the API at all -
|
||||
// jfjoch_broker has no rotation analysis path - so there is no case for it.
|
||||
switch (input.getMode().getValue()) {
|
||||
case org::openapitools::server::model::Analysis_mode::eAnalysis_mode::INVALID_VALUE_OPENAPI_GENERATED:
|
||||
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Unknown analysis mode");
|
||||
case org::openapitools::server::model::Analysis_mode::eAnalysis_mode::NONE:
|
||||
ret.Mode(AnalysisMode::None);
|
||||
break;
|
||||
case org::openapitools::server::model::Analysis_mode::eAnalysis_mode::MXSTILLS:
|
||||
ret.Mode(AnalysisMode::MXStills);
|
||||
break;
|
||||
case org::openapitools::server::model::Analysis_mode::eAnalysis_mode::AZINT:
|
||||
ret.Mode(AnalysisMode::Azint);
|
||||
break;
|
||||
case org::openapitools::server::model::Analysis_mode::eAnalysis_mode::GRID:
|
||||
ret.Mode(AnalysisMode::Grid);
|
||||
break;
|
||||
case org::openapitools::server::model::Analysis_mode::eAnalysis_mode::POWDERCALIBRATION:
|
||||
ret.Mode(AnalysisMode::PowderCalibration);
|
||||
break;
|
||||
}
|
||||
|
||||
if (input.calibrantIsSet())
|
||||
ret.Calibrant(input.getCalibrant());
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
org::openapitools::server::model::Analysis_settings Convert(const AnalysisSettings &input) {
|
||||
org::openapitools::server::model::Analysis_settings ret;
|
||||
org::openapitools::server::model::Analysis_mode tmp;
|
||||
switch (input.GetMode()) {
|
||||
case AnalysisMode::None:
|
||||
tmp.setValue(org::openapitools::server::model::Analysis_mode::eAnalysis_mode::NONE);
|
||||
break;
|
||||
case AnalysisMode::MXStills:
|
||||
tmp.setValue(org::openapitools::server::model::Analysis_mode::eAnalysis_mode::MXSTILLS);
|
||||
break;
|
||||
case AnalysisMode::Azint:
|
||||
tmp.setValue(org::openapitools::server::model::Analysis_mode::eAnalysis_mode::AZINT);
|
||||
break;
|
||||
case AnalysisMode::Grid:
|
||||
tmp.setValue(org::openapitools::server::model::Analysis_mode::eAnalysis_mode::GRID);
|
||||
break;
|
||||
case AnalysisMode::PowderCalibration:
|
||||
tmp.setValue(org::openapitools::server::model::Analysis_mode::eAnalysis_mode::POWDERCALIBRATION);
|
||||
break;
|
||||
case AnalysisMode::MXRotation:
|
||||
// Unreachable through the API: the setter refuses it, so it can never be stored. Reported
|
||||
// rather than silently rewritten to stills, which would be the broker claiming to run an
|
||||
// analysis it does not have.
|
||||
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
|
||||
"Rotation MX analysis cannot be reported by this API");
|
||||
}
|
||||
ret.setMode(tmp);
|
||||
if (!input.GetCalibrant().empty())
|
||||
ret.setCalibrant(input.GetCalibrant());
|
||||
return ret;
|
||||
}
|
||||
|
||||
org::openapitools::server::model::Scan_result Convert(const ScanResult& input) {
|
||||
org::openapitools::server::model::Scan_result ret;
|
||||
ret.setFilePrefix(input.file_prefix);
|
||||
@@ -1196,6 +1259,7 @@ org::openapitools::server::model::Scan_result Convert(const ScanResult& input) {
|
||||
}
|
||||
if (input.rotation_crystal_system && input.rotation_centering)
|
||||
ret.setRotationBravais(BravaisSymbol(*input.rotation_crystal_system, *input.rotation_centering));
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@
|
||||
#include "gen/model/Grid_scan.h"
|
||||
#include "gen/model/Indexing_settings.h"
|
||||
#include "gen/model/Bragg_integration_settings.h"
|
||||
#include "gen/model/Analysis_settings.h"
|
||||
#include "gen/model/Scan_result.h"
|
||||
|
||||
#include "../common/JFJochMessages.h"
|
||||
@@ -40,6 +41,7 @@
|
||||
#include "../jungfrau/JFCalibration.h"
|
||||
#include "../common/InstrumentMetadata.h"
|
||||
#include "../common/ScanResult.h"
|
||||
#include "../common/AnalysisSettings.h"
|
||||
#include "../image_pusher/ImagePusher.h"
|
||||
|
||||
SpotFindingSettings Convert(const org::openapitools::server::model::Spot_finding_settings &input);
|
||||
@@ -51,6 +53,9 @@ org::openapitools::server::model::Indexing_settings Convert(const IndexingSettin
|
||||
BraggIntegrationSettings Convert(const org::openapitools::server::model::Bragg_integration_settings &input);
|
||||
org::openapitools::server::model::Bragg_integration_settings Convert(const BraggIntegrationSettings &input);
|
||||
|
||||
AnalysisSettings Convert(const org::openapitools::server::model::Analysis_settings &input);
|
||||
org::openapitools::server::model::Analysis_settings Convert(const AnalysisSettings &input);
|
||||
|
||||
org::openapitools::server::model::Measurement_statistics Convert(const MeasurementStatistics &input);
|
||||
DetectorSettings Convert(const org::openapitools::server::model::Detector_settings &input);
|
||||
|
||||
|
||||
+95
-1
@@ -615,7 +615,14 @@ components:
|
||||
$ref: '#/components/schemas/unit_cell'
|
||||
spot_finding:
|
||||
type: boolean
|
||||
description: Enable spot finding and save spots
|
||||
deprecated: true
|
||||
description: |
|
||||
Enable spot finding and save spots.
|
||||
|
||||
DEPRECATED - use the analysis mode instead (/config/analysis). The mode takes precedence: a
|
||||
mode that analyses no spots (None, Azint, PowderCalibration) switches spot finding off
|
||||
whatever this says, and under a mode that does find spots this remains a finer control that
|
||||
can still turn it off. It is kept while callers move over.
|
||||
default: true
|
||||
smargon:
|
||||
type: object
|
||||
@@ -1071,9 +1078,15 @@ components:
|
||||
enable:
|
||||
type: boolean
|
||||
default: true
|
||||
deprecated: true
|
||||
description: |
|
||||
Enable spot finding. This is temporary setting, i.e. can be changed anytime during data collection.
|
||||
Even if disabled spot finding information will still be send and written, though always with zero spots.
|
||||
|
||||
DEPRECATED - use the analysis mode instead (/config/analysis). The mode takes precedence: a
|
||||
mode that analyses no spots (None, Azint, PowderCalibration) switches spot finding off
|
||||
whatever this says, and under a mode that does find spots this remains a finer control that
|
||||
can still turn it off. It is kept while callers move over.
|
||||
indexing:
|
||||
type: boolean
|
||||
default: true
|
||||
@@ -1972,6 +1985,8 @@ components:
|
||||
$ref: '#/components/schemas/indexing_settings'
|
||||
bragg_integration:
|
||||
$ref: '#/components/schemas/bragg_integration_settings'
|
||||
analysis:
|
||||
$ref: '#/components/schemas/analysis_settings'
|
||||
image_pusher:
|
||||
$ref: '#/components/schemas/image_pusher_status'
|
||||
error_message:
|
||||
@@ -2574,6 +2589,46 @@ components:
|
||||
type: string
|
||||
example: 10.1.1.7
|
||||
description: IPv4 address of the block device
|
||||
analysis_mode:
|
||||
type: string
|
||||
description: |
|
||||
What analysis jfjoch_broker runs over the images. One setting replaces the several independent
|
||||
switches - spot finding on/off in two places, indexing on/off, rotation on/off - that used to be
|
||||
composed at each site into an answer nobody had written down.
|
||||
None - images are received, written and streamed; nothing is analysed.
|
||||
MXStills - spot finding, per-image indexing, refinement and integration. The default, and what
|
||||
an out-of-the-box broker has always done.
|
||||
Azint - azimuthal integration only; no spots are found.
|
||||
Grid - grid scan: per-image scoring and crystal selection.
|
||||
PowderCalibration - detector geometry from a calibrant's powder rings. Forces azimuthal
|
||||
integration onto the CPU, since the FPGA core cannot hold enough bins for the sectored
|
||||
profile a ring fit needs; a calibration exposure is a few images at a few Hz, so the cost
|
||||
does not matter.
|
||||
|
||||
Rotation MX analysis is deliberately absent: jfjoch_broker has no rotation analysis path, so
|
||||
this API cannot express it. Rotation data is collected here and processed offline with rugnux.
|
||||
enum:
|
||||
- "None"
|
||||
- "MXStills"
|
||||
- "Azint"
|
||||
- "Grid"
|
||||
- "PowderCalibration"
|
||||
default: "MXStills"
|
||||
analysis_settings:
|
||||
type: object
|
||||
description: |
|
||||
The analysis mode and the settings that only mean anything under one of them. Persistent: it
|
||||
survives a data collection, unlike the per-dataset settings given to /start.
|
||||
required:
|
||||
- mode
|
||||
properties:
|
||||
mode:
|
||||
$ref: '#/components/schemas/analysis_mode'
|
||||
calibrant:
|
||||
type: string
|
||||
description: |
|
||||
PowderCalibration only - which calibrant's ring d-spacings the geometry is fitted against.
|
||||
Ignored under every other mode.
|
||||
bragg_integration_settings:
|
||||
type: object
|
||||
description: "Settings for Bragg spot integration"
|
||||
@@ -2626,6 +2681,8 @@ components:
|
||||
$ref: '#/components/schemas/indexing_settings'
|
||||
bragg_integration:
|
||||
$ref: '#/components/schemas/bragg_integration_settings'
|
||||
analysis:
|
||||
$ref: '#/components/schemas/analysis_settings'
|
||||
detector_settings:
|
||||
$ref: '#/components/schemas/detector_settings'
|
||||
azim_int:
|
||||
@@ -3006,6 +3063,43 @@ paths:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/bragg_integration_settings'
|
||||
/config/analysis:
|
||||
put:
|
||||
summary: Change analysis settings
|
||||
description: |
|
||||
This can only be done when detector is `Idle`, `Error` or `Inactive` states.
|
||||
The mode is persistent - it is not reset by a data collection.
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/analysis_settings'
|
||||
responses:
|
||||
"200":
|
||||
description: Everything OK
|
||||
"400":
|
||||
description: Input parsing or validation error
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
description: Exception error
|
||||
"500":
|
||||
description: Error within Jungfraujoch code - see output message.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/error_message'
|
||||
get:
|
||||
summary: Get analysis configuration
|
||||
description: Can be done anytime
|
||||
responses:
|
||||
"200":
|
||||
description: Everything OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/analysis_settings'
|
||||
/config/file_writer:
|
||||
put:
|
||||
summary: Change file writer settings
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#include "AnalysisSettings.h"
|
||||
|
||||
// The mode-to-analysis table. One place, one row per mode, and every gate in the pipeline reads it -
|
||||
// so what a mode does can be read off here instead of being reconstructed from `if (mode == ...)`
|
||||
// scattered through the engines.
|
||||
//
|
||||
// spots index bragg merge score azint
|
||||
// None - - - - - -
|
||||
// MXRotation x x x x x x
|
||||
// MXStills x x x x x x
|
||||
// Azint - - - - - x
|
||||
// Grid x - - - x x
|
||||
// PowderCalibration x - - - - x
|
||||
//
|
||||
// Two rows want a word.
|
||||
//
|
||||
// PowderCalibration keeps spot finding. The geometry is fitted either to the ring arcs of the summed
|
||||
// (q x azimuth) profile or to the POOLED SPOTS (rugnux --calibration rings|spots), and the second of
|
||||
// those is a spot finder run - so the mode must not decide this; whoever picks the source does.
|
||||
//
|
||||
// Grid does not index. A raster is thousands of frames and indexing is the expensive stage, and the
|
||||
// per-image scoring it ranks grid points on deliberately does not use it (indexing fires on ice, which
|
||||
// is exactly what the scoring has to see past). The cost of that choice is indexed_lattice_count, which
|
||||
// is the cheapest multi-lattice / cracked-crystal signal there is and which a raster therefore does not
|
||||
// get. It is one field in this table either way.
|
||||
//
|
||||
// The switch has no default: adding a mode must be a compile error here rather than a silent
|
||||
// fall-through to whatever the last row happened to be.
|
||||
AnalysisStages AnalysisModeStages(AnalysisMode mode) {
|
||||
switch (mode) {
|
||||
case AnalysisMode::None:
|
||||
return {false, false, false, false, false, false};
|
||||
case AnalysisMode::MXRotation:
|
||||
case AnalysisMode::MXStills:
|
||||
return {true, true, true, true, true, true};
|
||||
case AnalysisMode::Azint:
|
||||
return {false, false, false, false, false, true};
|
||||
case AnalysisMode::Grid:
|
||||
return {true, false, false, false, true, true};
|
||||
case AnalysisMode::PowderCalibration:
|
||||
return {true, false, false, false, false, true};
|
||||
}
|
||||
return {false, false, false, false, false, false};
|
||||
}
|
||||
|
||||
bool AnalysisModeIsMX(AnalysisMode mode) {
|
||||
switch (mode) {
|
||||
case AnalysisMode::MXRotation:
|
||||
case AnalysisMode::MXStills:
|
||||
return true;
|
||||
case AnalysisMode::None:
|
||||
case AnalysisMode::Azint:
|
||||
case AnalysisMode::Grid:
|
||||
case AnalysisMode::PowderCalibration:
|
||||
return false;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string AnalysisModeName(AnalysisMode mode) {
|
||||
switch (mode) {
|
||||
case AnalysisMode::None: return "none";
|
||||
case AnalysisMode::MXRotation: return "mx_rotation";
|
||||
case AnalysisMode::MXStills: return "mx_stills";
|
||||
case AnalysisMode::Azint: return "azint";
|
||||
case AnalysisMode::Grid: return "grid";
|
||||
case AnalysisMode::PowderCalibration: return "powder_calibration";
|
||||
}
|
||||
return "none";
|
||||
}
|
||||
|
||||
std::optional<AnalysisMode> AnalysisModeFromName(std::string_view name) {
|
||||
if (name == "none") return AnalysisMode::None;
|
||||
if (name == "mx_rotation") return AnalysisMode::MXRotation;
|
||||
if (name == "mx_stills") return AnalysisMode::MXStills;
|
||||
if (name == "azint") return AnalysisMode::Azint;
|
||||
if (name == "grid") return AnalysisMode::Grid;
|
||||
if (name == "powder_calibration") return AnalysisMode::PowderCalibration;
|
||||
return {};
|
||||
}
|
||||
|
||||
AnalysisSettings &AnalysisSettings::Mode(AnalysisMode input) {
|
||||
mode = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
AnalysisSettings &AnalysisSettings::Calibrant(const std::string &input) {
|
||||
calibrant = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
AnalysisMode AnalysisSettings::GetMode() const {
|
||||
return mode;
|
||||
}
|
||||
|
||||
const std::string &AnalysisSettings::GetCalibrant() const {
|
||||
return calibrant;
|
||||
}
|
||||
|
||||
bool AnalysisSettings::IsMX() const {
|
||||
return AnalysisModeIsMX(mode);
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
// What analysis runs over the images - the one place the question is asked, shared by jfjoch_broker,
|
||||
// rugnux and jfjoch_viewer. Before this, the answer was composed at each site out of a detector type,
|
||||
// two independent "spot finding off" switches and a rotation flag, so what was configured and what
|
||||
// actually ran were different things.
|
||||
//
|
||||
// There is deliberately no Auto value. GetIndexingAlgorithm() resolves Auto at read time against the
|
||||
// GPU count and the unit cell, which is exactly why an indexing setting cannot be read back off the
|
||||
// configuration; the mode getter is a plain accessor and stays one.
|
||||
//
|
||||
// MXStills is the default: None would silently switch analysis off on every deployment whose
|
||||
// configuration predates this field.
|
||||
//
|
||||
// MXRotation is not offered by the REST API - jfjoch_broker has no rotation analysis path, so
|
||||
// broker/jfjoch_api.yaml simply cannot express it and JFJochStateMachine refuses it if it arrives by
|
||||
// any other route. Rotation data is processed offline with rugnux.
|
||||
enum class AnalysisMode {
|
||||
None, // images are received, written and streamed; nothing is analysed
|
||||
MXRotation, // spot finding + rotation (two-pass) indexing + refinement + integration
|
||||
MXStills, // spot finding + per-image indexing + refinement + integration
|
||||
Azint, // azimuthal integration only
|
||||
Grid, // grid scan: per-image scoring and crystal selection
|
||||
PowderCalibration // detector geometry from a calibrant's powder rings
|
||||
};
|
||||
|
||||
// Azimuthal sectors a powder calibration integrates into when nothing else is asked for. A plain
|
||||
// radial profile (1 sector) has averaged every ring over every direction, so nothing is left to say
|
||||
// where its centre is (RingsFromAzimuthalProfile refuses below 4). Sweeping the count on a LaB6
|
||||
// exposure and on a crystal's ice rings, 32 and above agree to about 0.1 px with a flat residual,
|
||||
// while 8 sectors is visibly coarser (0.3 px away, twice the residual).
|
||||
//
|
||||
// 32 is also what the FPGA integration core can carry: it holds FPGA_INTEGRATION_BIN_COUNT = 2048
|
||||
// bins in total (the receiver throws above that), so 32 sectors would leave 64 q bins - far too
|
||||
// coarse to fit a ring. PowderCalibration therefore forces azimuthal integration onto the CPU
|
||||
// (AzimuthalIntegrationSettings::ForceCPUinFPGAWorkflow), which lifts the bin limit at the cost of
|
||||
// frame rate. That is not a problem this mode has: a calibration exposure is a few images at a few
|
||||
// Hz, not a data collection.
|
||||
constexpr int CALIBRATION_AZIM_BINS_DEFAULT = 32;
|
||||
|
||||
// Which stages of the pipeline a mode runs. This is the whole point of the mode: it does not label a
|
||||
// run, it decides what happens in it, and every gate in the pipeline reads this rather than testing
|
||||
// the mode itself. The table is in AnalysisSettings.cpp - modes are the rows, stages the columns.
|
||||
struct AnalysisStages {
|
||||
bool spot_finding;
|
||||
bool indexing;
|
||||
// Bragg prediction and integration. It follows indexing in every engine - nothing is predicted
|
||||
// without a lattice - so this never says yes where indexing says no.
|
||||
bool bragg_integration;
|
||||
bool scale_merge;
|
||||
// Per-image protein / ice scoring: what a grid scan ranks its grid points on.
|
||||
bool scoring;
|
||||
bool azimuthal_integration;
|
||||
};
|
||||
|
||||
AnalysisStages AnalysisModeStages(AnalysisMode mode);
|
||||
|
||||
// True for the two MX modes, i.e. the modes that index and integrate Bragg reflections.
|
||||
bool AnalysisModeIsMX(AnalysisMode mode);
|
||||
|
||||
// The wire spelling of a mode: the same token in the CBOR stream, the HDF5 master and the rugnux
|
||||
// --mode option, so a value can be carried between them without a translation table per hop. (The
|
||||
// OpenAPI enum spells its values the way its neighbouring settings enums do, and OpenAPIConvert
|
||||
// translates, as it does for every other enum in the API.)
|
||||
std::string AnalysisModeName(AnalysisMode mode);
|
||||
std::optional<AnalysisMode> AnalysisModeFromName(std::string_view name);
|
||||
|
||||
// The analysis mode and the settings that only mean anything under one of them. Persistent, not
|
||||
// per-dataset: it sits on DiffractionExperiment outside the DatasetSettings member, which is the one
|
||||
// thing a /start replaces wholesale.
|
||||
class AnalysisSettings {
|
||||
AnalysisMode mode = AnalysisMode::MXStills;
|
||||
// PowderCalibration only: which calibrant's ring d-spacings to fit against.
|
||||
std::string calibrant;
|
||||
// Grid decision thresholds land here when Grid is implemented.
|
||||
public:
|
||||
AnalysisSettings& Mode(AnalysisMode input);
|
||||
AnalysisSettings& Calibrant(const std::string &input);
|
||||
|
||||
[[nodiscard]] AnalysisMode GetMode() const;
|
||||
[[nodiscard]] const std::string &GetCalibrant() const;
|
||||
|
||||
[[nodiscard]] bool IsMX() const;
|
||||
};
|
||||
@@ -121,6 +121,8 @@ ADD_LIBRARY(JFJochCommon STATIC
|
||||
ScanResultGenerator.h
|
||||
BraggIntegrationSettings.cpp
|
||||
BraggIntegrationSettings.h
|
||||
AnalysisSettings.cpp
|
||||
AnalysisSettings.h
|
||||
SpotToSave.cpp
|
||||
XrayFluorescenceSpectrum.cpp
|
||||
XrayFluorescenceSpectrum.h
|
||||
|
||||
@@ -67,6 +67,9 @@ class DatasetSettings {
|
||||
float poni_rot_2_rad;
|
||||
float poni_rot_3_rad;
|
||||
|
||||
// DEPRECATED - the analysis mode (AnalysisSettings) is the switch now. It still gates spot finding
|
||||
// beneath a mode that finds spots; see DiffractionExperiment::IsSpotFindingEnabled for the
|
||||
// precedence between the three switches that answer this one question.
|
||||
bool spot_finding_enable;
|
||||
int64_t max_spot_count;
|
||||
|
||||
|
||||
@@ -706,6 +706,7 @@ void DiffractionExperiment::FillMessage(StartMessage &message) const {
|
||||
// signed ones reserve INTx_MIN as the marker (GetUnderflow), so the lowest valid is one above.
|
||||
message.underload_value = IsPixelSigned() ? GetUnderflow() + 1 : 0;
|
||||
message.indexing_algorithm = GetIndexingAlgorithm();
|
||||
message.analysis_mode = analysis.GetMode();
|
||||
message.images_per_trigger = dataset.GetImageNumPerTrigger();
|
||||
|
||||
if (GetDetectorType() == DetectorType::JUNGFRAU) {
|
||||
@@ -1072,7 +1073,13 @@ bool DiffractionExperiment::IsPulsedSource() const {
|
||||
}
|
||||
|
||||
bool DiffractionExperiment::IsSpotFindingEnabled() const {
|
||||
return dataset.IsSpotFindingEnabled() && !IsPedestalRun();
|
||||
// Three switches answer one question, and this is where the precedence between them is fixed. The
|
||||
// analysis mode is the authority: a mode that analyses no spots (none, azint, powder_calibration)
|
||||
// switches spot finding off whatever else is set. Under a mode that does find spots, the two older
|
||||
// switches - the per-dataset dataset_settings.spot_finding here and the persistent
|
||||
// SpotFindingSettings::enable read in MXAnalysisAfterFPGA - remain the finer control beneath it and
|
||||
// can still turn it off. Both are DEPRECATED and go away once their callers have moved to the mode.
|
||||
return GetAnalysisStages().spot_finding && dataset.IsSpotFindingEnabled() && !IsPedestalRun();
|
||||
}
|
||||
|
||||
float DiffractionExperiment::GetPhotonEnergyForConversion_keV() const {
|
||||
@@ -1766,6 +1773,39 @@ BraggIntegrationSettings DiffractionExperiment::GetBraggIntegrationSettings() co
|
||||
return bragg_integration_settings;
|
||||
}
|
||||
|
||||
DiffractionExperiment &DiffractionExperiment::ImportAnalysisSettings(const AnalysisSettings &input) {
|
||||
analysis = input;
|
||||
|
||||
// Powder calibration reads ring positions off a (q x azimuth) profile, so it needs azimuthal
|
||||
// sectors - and enough q bins under each of them to place a ring. The FPGA integration core holds
|
||||
// FPGA_INTEGRATION_BIN_COUNT = 2048 bins in total, which at CALIBRATION_AZIM_BINS_DEFAULT sectors
|
||||
// would leave 64 q bins, so the integration is moved onto the CPU where the limit does not apply.
|
||||
// The cost is frame rate, which this mode does not need: a calibration exposure is a few images at
|
||||
// a few Hz. Setting the mode is therefore all a caller has to do - see AnalysisSettings.h.
|
||||
if (analysis.GetMode() == AnalysisMode::PowderCalibration) {
|
||||
az_integration_settings.ForceCPUinFPGAWorkflow(true);
|
||||
// Only where no usable sector count was asked for. Below four sectors a ring has been averaged
|
||||
// over every direction and nothing is left to say where its centre is (RingsFromAzimuthalProfile
|
||||
// refuses there), and the default of one is exactly that; an explicit count stands.
|
||||
if (az_integration_settings.GetAzimuthalBinCount() < 4)
|
||||
az_integration_settings.AzimuthalBinCount(CALIBRATION_AZIM_BINS_DEFAULT);
|
||||
}
|
||||
|
||||
return *this;
|
||||
}
|
||||
|
||||
AnalysisSettings DiffractionExperiment::GetAnalysisSettings() const {
|
||||
return analysis;
|
||||
}
|
||||
|
||||
AnalysisMode DiffractionExperiment::GetAnalysisMode() const {
|
||||
return analysis.GetMode();
|
||||
}
|
||||
|
||||
AnalysisStages DiffractionExperiment::GetAnalysisStages() const {
|
||||
return AnalysisModeStages(analysis.GetMode());
|
||||
}
|
||||
|
||||
DiffractionExperiment &DiffractionExperiment::ImportScalingSettings(const ScalingSettings &input) {
|
||||
scaling_settings = input;
|
||||
return *this;
|
||||
|
||||
@@ -37,6 +37,7 @@ constexpr int64_t SaturationLimitFromValue(int64_t saturation_value) { return sa
|
||||
#include "CompressedImage.h"
|
||||
#include "IndexingSettings.h"
|
||||
#include "BraggIntegrationSettings.h"
|
||||
#include "AnalysisSettings.h"
|
||||
#include "ScalingSettings.h"
|
||||
|
||||
#include <gemmi/symmetry.hpp>
|
||||
@@ -84,6 +85,9 @@ class DiffractionExperiment {
|
||||
FileWriterSettings file_writer;
|
||||
IndexingSettings indexing;
|
||||
BraggIntegrationSettings bragg_integration_settings;
|
||||
// Persistent, deliberately outside the DatasetSettings member above: a /start replaces `dataset`
|
||||
// wholesale, and the analysis mode is a property of how the instrument is set up, not of one run.
|
||||
AnalysisSettings analysis;
|
||||
ScalingSettings scaling_settings;
|
||||
|
||||
DarkMaskSettings dark_mask_settings;
|
||||
@@ -199,6 +203,14 @@ public:
|
||||
DiffractionExperiment& ImportBraggIntegrationSettings(const BraggIntegrationSettings& input);
|
||||
BraggIntegrationSettings GetBraggIntegrationSettings() const;
|
||||
|
||||
// PowderCalibration also rewrites the azimuthal-integration settings it needs - see the definition.
|
||||
DiffractionExperiment& ImportAnalysisSettings(const AnalysisSettings& input);
|
||||
AnalysisSettings GetAnalysisSettings() const;
|
||||
[[nodiscard]] AnalysisMode GetAnalysisMode() const;
|
||||
// What this experiment's mode runs. Every gate in the pipeline reads this; the table is in
|
||||
// common/AnalysisSettings.cpp.
|
||||
[[nodiscard]] AnalysisStages GetAnalysisStages() const;
|
||||
|
||||
DiffractionExperiment& ImportFileWriterSettings(const FileWriterSettings& input);
|
||||
FileWriterSettings GetFileWriterSettings() const;
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
#include "Reflection.h"
|
||||
#include "CrystalLattice.h"
|
||||
#include "IndexingSettings.h"
|
||||
#include "AnalysisSettings.h"
|
||||
#include "XrayFluorescenceSpectrum.h"
|
||||
#include "../gemmi_gph/gemmi/symmetry.hpp"
|
||||
|
||||
@@ -349,6 +350,11 @@ struct StartMessage {
|
||||
IndexingAlgorithmEnum indexing_algorithm;
|
||||
GeomRefinementAlgorithmEnum geom_refinement_algorithm;
|
||||
|
||||
// Which analysis produced this stream, so a written file records it. Absent in a stream written
|
||||
// before the mode existed, which is why it is optional rather than defaulted: naming a mode for a
|
||||
// file that never carried one would be an invention, not provenance.
|
||||
std::optional<AnalysisMode> analysis_mode;
|
||||
|
||||
std::optional<float> poni_rot1;
|
||||
std::optional<float> poni_rot2;
|
||||
std::optional<float> poni_rot3;
|
||||
|
||||
@@ -119,6 +119,7 @@ There are minor differences at the moment:
|
||||
| - sample_temperature_K | float | Sample temperature \[K\] | |
|
||||
| - detect_ice_rings | bool | Ice ring detection feature is enabled | |
|
||||
| - indexing_algorithm | string | Indexing algorithm used on-the-fly; allowed values: ffbidx, fft, fftw, none | |
|
||||
| - analysis_mode | string | Analysis that produced the stream; allowed values: none, mx_rotation, mx_stills, azint, grid, powder_calibration. Absent in a stream written before this key existed | |
|
||||
| - geom_refinement_algorithm | string | Post-indexing detector geometry refinement algorithm; allowed values: none, beam_center | |
|
||||
| - poni_rot1 | float | Tilt of the detector rot1 according to PyFAI PONI convention \[rad\] | |
|
||||
| - poni_rot2 | float | Tilt of the detector rot2 according to PyFAI PONI convention \[rad\] | |
|
||||
|
||||
@@ -3,6 +3,11 @@
|
||||
|
||||
### 1.0.0-rc.167
|
||||
|
||||
* jfjoch_broker takes an explicit analysis mode (`/config/analysis`, the `analysis` block of the configuration file, and the Analysis panel of the web frontend): `MXStills` (the default), `Azint`, `Grid`, `PowderCalibration` or `None`. The mode decides what runs and takes precedence over `dataset_settings.spot_finding` and `spot_finding_settings.enable`, which are now deprecated.
|
||||
* `PowderCalibration` moves azimuthal integration onto the CPU and sets a sector count of its own, since the FPGA integration core cannot hold enough bins for a sectored profile; a calibration exposure then runs at a few Hz.
|
||||
* Rotation MX analysis is not offered online - it is absent from the REST API and refused by the broker - and a rotation sweep collected under an MX mode is logged as being analysed frame by frame as stills.
|
||||
* `rugnux --mode` accepts `mx_rotation` and `mx_stills` beside `mx`, which continues to choose between them from the goniometer.
|
||||
* The stream and the written file record which analysis produced them, as `analysis_mode` in the CBOR start message and `/entry/MX/analysis_mode` in the HDF5 master.
|
||||
* `rugnux --model` reports CC(model, data) - the correlation of the merged intensities with the placed, scaled model - by resolution shell, on the same shells as CC1/2, with the reflection count and a significance for each.
|
||||
* `rugnux --model` fits the model's scale, anisotropic B and bulk-solvent parameters on the working reflections only, so the R-free it reports is measured against a model no free reflection helped scale.
|
||||
* The bulk-solvent parameters of `rugnux --model` are searched over their physically meaningful range instead of being fitted without bounds, so a model is never scaled with a solvent term that has silently switched itself off.
|
||||
|
||||
@@ -361,6 +361,7 @@ variants.
|
||||
|
||||
| Dataset | Units | Meaning |
|
||||
|---------|-------|---------|
|
||||
| `analysis_mode` | | which analysis produced the file: `none` / `mx_rotation` / `mx_stills` / `azint` / `grid` / `powder_calibration`. Absent in a file written before this dataset existed |
|
||||
| `indexing_algorithm` | | `FFBIDX` / `FFT (CUDA)` / `FFT (FFTW)` |
|
||||
| `geom_refinement_algorithm` | | e.g. `beam_center` |
|
||||
| `rotationLatticeIndexed` | Å | whole-run rotation-indexing lattice (`[9]`) |
|
||||
|
||||
@@ -1198,6 +1198,8 @@ namespace {
|
||||
message.poni_rot3 = j["poni_rot3"];
|
||||
if (j.contains("detect_ice_rings"))
|
||||
message.detect_ice_rings = j["detect_ice_rings"];
|
||||
if (j.contains("analysis_mode"))
|
||||
message.analysis_mode = AnalysisModeFromName(j["analysis_mode"].get<std::string>());
|
||||
if (j.contains("images_per_trigger"))
|
||||
message.images_per_trigger = j["images_per_trigger"];
|
||||
if (j.contains("indexing_algorithm")) {
|
||||
|
||||
@@ -629,6 +629,11 @@ inline void CBOR_ENC_START_USER_DATA(CborEncoder& encoder, const char* key,
|
||||
if (message.detect_ice_rings.has_value())
|
||||
j["detect_ice_rings"] = message.detect_ice_rings.value();
|
||||
|
||||
// Which analysis produced this stream. One shared spelling table (AnalysisModeName), so the token
|
||||
// in the stream is the same one the HDF5 master, the rugnux --mode option and the OpenAPI enum use.
|
||||
if (message.analysis_mode.has_value())
|
||||
j["analysis_mode"] = AnalysisModeName(message.analysis_mode.value());
|
||||
|
||||
switch(message.indexing_algorithm) {
|
||||
case IndexingAlgorithmEnum::FFBIDX:
|
||||
j["indexing_algorithm"] = "ffbidx";
|
||||
|
||||
@@ -43,6 +43,7 @@ import MaskVisualization from "./components/MaskVisualization";
|
||||
import DarkMaskSettings from "./components/DarkMaskSettings";
|
||||
import IndexingSettings from "./components/IndexingSettings";
|
||||
import BraggIntegrationSettings from "./components/BraggIntegrationSettings";
|
||||
import AnalysisSettings from "./components/AnalysisSettings";
|
||||
import AzIntSettings from "./components/AzIntSettings";
|
||||
import ROI from "./components/ROI";
|
||||
import FileWriterSettings from "./components/FileWriterSettings";
|
||||
@@ -211,6 +212,7 @@ function App() {
|
||||
{ id: 'processing', label: 'On-the-fly processing', icon: <BoltIcon/>, render: () => (
|
||||
<Stack spacing={2}>
|
||||
<PanelColumns>
|
||||
<AnalysisSettings s={s.analysis}/>
|
||||
<IndexingSettings s={s.indexing} status={s.broker}/>
|
||||
<BraggIntegrationSettings s={s.bragg_integration}/>
|
||||
<AzIntSettings s={s.az_int}/>
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
import {memo, useEffect, useState} from 'react';
|
||||
|
||||
import FormControl from "@mui/material/FormControl";
|
||||
import InputLabel from "@mui/material/InputLabel";
|
||||
import Select, {SelectChangeEvent} from "@mui/material/Select";
|
||||
import MenuItem from "@mui/material/MenuItem";
|
||||
import TextField from "@mui/material/TextField";
|
||||
import {analysis_mode, analysis_settings} from "../client";
|
||||
import {putConfigAnalysisMutation} from "../client/@tanstack/react-query.gen";
|
||||
import _ from "lodash";
|
||||
import SettingsPanel from "./SettingsPanel";
|
||||
import {useUpload} from "./useUpload";
|
||||
|
||||
type MyProps = {
|
||||
s?: analysis_settings
|
||||
}
|
||||
|
||||
// MXStills, not None: a broker whose configuration predates this setting keeps analysing.
|
||||
const default_analysis_settings: analysis_settings = {
|
||||
mode: analysis_mode.MX_STILLS
|
||||
};
|
||||
|
||||
function AnalysisSettings({s: serverS}: MyProps) {
|
||||
const [s, setS] = useState<analysis_settings>(default_analysis_settings);
|
||||
const [lastDownloadedS, setLastDownloadedS] = useState<analysis_settings>(default_analysis_settings);
|
||||
const {submit, pending, snackbar} = useUpload(putConfigAnalysisMutation());
|
||||
|
||||
useEffect(() => {
|
||||
if ((serverS !== undefined) && !_.isEqual(serverS, lastDownloadedS)) {
|
||||
setS(serverS);
|
||||
setLastDownloadedS(serverS);
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [serverS]);
|
||||
|
||||
const handleModeChange = (event: SelectChangeEvent) => {
|
||||
setS(prev => ({...prev, mode: event.target.value as analysis_mode}));
|
||||
};
|
||||
|
||||
const dirty = !_.isEqual(s, lastDownloadedS);
|
||||
|
||||
return (
|
||||
<SettingsPanel title="Analysis" dirty={dirty}
|
||||
onUpload={() => submit(s)} uploadDisabled={pending} snackbar={snackbar}>
|
||||
<b>Analysis mode</b>
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<InputLabel>Analysis mode</InputLabel>
|
||||
{/* Only what the API can express. Rotation MX is deliberately not here: jfjoch_broker
|
||||
has no rotation analysis path, so the setting must be unaskable rather than asked
|
||||
for and then refused. Rotation data is collected here and processed with rugnux. */}
|
||||
<Select
|
||||
label="Analysis mode"
|
||||
value={s.mode}
|
||||
onChange={handleModeChange}
|
||||
>
|
||||
<MenuItem value={analysis_mode.MX_STILLS}>MX stills - spot finding, indexing, integration (default)</MenuItem>
|
||||
<MenuItem value={analysis_mode.AZINT}>Azimuthal integration only</MenuItem>
|
||||
<MenuItem value={analysis_mode.GRID}>Grid scan - per-image scoring and crystal selection</MenuItem>
|
||||
<MenuItem value={analysis_mode.POWDER_CALIBRATION}>Powder calibration - detector geometry from a calibrant</MenuItem>
|
||||
<MenuItem value={analysis_mode.NONE}>None - acquire and write only</MenuItem>
|
||||
</Select>
|
||||
</FormControl>
|
||||
The mode decides what runs, and takes precedence over the spot-finding switches. Rotation
|
||||
analysis is not available online - collect the sweep here and process the stored file with
|
||||
rugnux.
|
||||
|
||||
{s.mode === analysis_mode.POWDER_CALIBRATION && (
|
||||
<>
|
||||
<b>Calibrant</b>
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<TextField id="calibrant" label="Calibrant" variant="outlined"
|
||||
value={s.calibrant ?? ""}
|
||||
onChange={(e) => setS(prev => ({...prev, calibrant: e.target.value}))}
|
||||
fullWidth/>
|
||||
</FormControl>
|
||||
Which standard's ring d-spacings the geometry is fitted against. Powder calibration
|
||||
also moves azimuthal integration onto the CPU - the FPGA core cannot hold enough bins
|
||||
for a sectored profile - so it runs at a few Hz, which is all a calibration exposure
|
||||
needs.
|
||||
</>
|
||||
)}
|
||||
</SettingsPanel>
|
||||
);
|
||||
}
|
||||
|
||||
export default memo(AnalysisSettings);
|
||||
@@ -604,7 +604,11 @@ void IndexAndRefine::QuickPredictAndIntegrate(DataMessage &msg,
|
||||
|
||||
std::optional<IndexAndRefine::IndexingOutcome>
|
||||
IndexAndRefine::DetermineRefineAnalyze(DataMessage &msg, const SpotFindingSettings &spot_finding_settings) {
|
||||
if (!indexer_ || !spot_finding_settings.indexing)
|
||||
// The analysis mode decides first and the deprecated per-run switch second: a mode that does not
|
||||
// index (none, azint, grid, powder_calibration) does not index whatever the switch says. One gate
|
||||
// for all three frontends - nothing predicts or integrates without a lattice, so this also stops
|
||||
// Bragg prediction and integration.
|
||||
if (!indexer_ || !experiment.GetAnalysisStages().indexing || !spot_finding_settings.indexing)
|
||||
return std::nullopt;
|
||||
|
||||
IndexingOutcome outcome(experiment);
|
||||
|
||||
@@ -40,7 +40,11 @@ MXAnalysisAfterFPGA::MXAnalysisAfterFPGA(const DiffractionExperiment &in_experim
|
||||
if (experiment.IsSpotFindingEnabled())
|
||||
find_spots = true;
|
||||
|
||||
if (experiment.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow())
|
||||
// AnalysisMode::None analyses nothing at all, azimuthal integration included; every other mode
|
||||
// keeps it (see the table in common/AnalysisSettings.cpp). Note this only governs the CPU engine -
|
||||
// on the FPGA path the integration is a stage of the hardware pipeline and is not switched here.
|
||||
if (experiment.GetAnalysisStages().azimuthal_integration
|
||||
&& experiment.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow())
|
||||
cpu_azint = std::make_unique<AzIntEngineCPU>(integration);
|
||||
}
|
||||
|
||||
|
||||
@@ -58,6 +58,9 @@ constexpr bool SpotShapeAccepted(int64_t pixel_count, int64_t bbox_side) {
|
||||
}
|
||||
|
||||
struct SpotFindingSettings {
|
||||
// DEPRECATED - the analysis mode (common/AnalysisSettings.h) is the switch now. It still gates spot
|
||||
// finding beneath a mode that finds spots; see DiffractionExperiment::IsSpotFindingEnabled for the
|
||||
// precedence between the three switches that answer this one question.
|
||||
bool enable = true;
|
||||
float signal_to_noise_threshold = 4.0; // STRONG_PIXEL in XDS
|
||||
int64_t photon_count_threshold = 10; // Threshold in photon counts
|
||||
|
||||
@@ -778,6 +778,12 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
|
||||
}
|
||||
|
||||
if (master_file->Exists("/entry/MX")) {
|
||||
// The analysis that wrote the file, kept as provenance only. It is deliberately not pushed
|
||||
// onto the experiment: what a stored file was produced by is not what the next run should
|
||||
// do, and letting it decide would be exactly the implicitness the mode exists to remove.
|
||||
dataset->file_analysis_mode =
|
||||
AnalysisModeFromName(master_file->GetString("/entry/MX/analysis_mode", ""));
|
||||
|
||||
auto indexing = master_file->GetString("/entry/MX/indexing_algorithm", "none");
|
||||
if (indexing == "fft" || indexing == "FFT (CUDA)" || indexing == "FFT (FFTW)")
|
||||
dataset->experiment.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
||||
|
||||
@@ -31,6 +31,11 @@ struct JFJochReaderDataset {
|
||||
// defaults ice handling by geometry when the file is silent (see rugnux_cli).
|
||||
std::optional<bool> file_detect_ice_rings;
|
||||
|
||||
// The analysis mode the master file records (/entry/MX/analysis_mode) - provenance, not a request:
|
||||
// it says what produced the file, and nothing reads it to decide what to do next. Absent for a file
|
||||
// written before the mode existed.
|
||||
std::optional<AnalysisMode> file_analysis_mode;
|
||||
|
||||
std::string jfjoch_release;
|
||||
|
||||
// Change of basis (3x3 integers, row major) from the setting the per-image reflections and
|
||||
|
||||
+7
-3
@@ -488,6 +488,10 @@ Rugnux::Rugnux(JFJochReader &reader, DiffractionExperiment experiment,
|
||||
: reader_(reader), experiment_(std::move(experiment)),
|
||||
pixel_mask_(std::move(pixel_mask)), config_(std::move(config)),
|
||||
user_fixed_sg_(experiment_.GetGemmiSpaceGroup()) {
|
||||
// The run's analysis mode belongs on the experiment as well as on the config: that is what the
|
||||
// written file records, and what the pipeline's own gates (spot finding, indexing) read.
|
||||
experiment_.ImportAnalysisSettings(experiment_.GetAnalysisSettings().Mode(config_.mode));
|
||||
|
||||
// Bit 9 describes where THIS run found the beam stop, so a mask read back from a file that
|
||||
// already carries one starts clear; the user mask (bit 8) is left as it was loaded.
|
||||
pixel_mask_.ClearBeamStopMask(experiment_);
|
||||
@@ -1540,7 +1544,7 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) {
|
||||
// (refined-geometry) pass writes merged files, under the plain "<prefix>_*" name. The first pass still runs
|
||||
// under a "<prefix>_01" prefix so that the _process.h5 it writes, when one is asked for, does not collide
|
||||
// with the second pass's - the ice-ring flags and geometry stored in each are the ones that pass used.
|
||||
if (config_.mode == ProcessMode::FullAnalysis && config_.rotation_postrefine_geometry
|
||||
if (AnalysisModeIsMX(config_.mode) && config_.rotation_postrefine_geometry
|
||||
&& experiment_.IsRotationIndexing()) {
|
||||
Logger logger("Rugnux");
|
||||
const std::string base_prefix = config_.output_prefix;
|
||||
@@ -1992,13 +1996,13 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
|
||||
return result;
|
||||
}
|
||||
|
||||
const bool full = (config_.mode == ProcessMode::FullAnalysis);
|
||||
const bool full = AnalysisModeIsMX(config_.mode);
|
||||
// Powder calibration runs the same per-image engine as the full analysis (spot finding lives inside
|
||||
// MXAnalysisWithoutFPGA) with indexing switched off in its settings. By rings it needs only the
|
||||
// azimuthal profile and the azint worker would do - but the spots are worth their half second even
|
||||
// then, because the circle through them places the beam centre without reference to the header at
|
||||
// all, which is the one hypothesis that survives a header the profile cannot correct.
|
||||
const bool calibration = (config_.mode == ProcessMode::Calibration);
|
||||
const bool calibration = (config_.mode == AnalysisMode::PowderCalibration);
|
||||
const bool calibration_spots = calibration && config_.calibration_method == CalibrationMethod::Spots;
|
||||
const bool per_image_analysis = full || calibration;
|
||||
const bool write_files = write_output && !config_.output_prefix.empty();
|
||||
|
||||
+5
-7
@@ -36,14 +36,12 @@ class JFJochHDF5Reader;
|
||||
// an optional scaling/merging post-pass, and the _process.h5 output. The detector geometry and all
|
||||
// algorithm settings are configured on the DiffractionExperiment by the caller; ProcessConfig only
|
||||
// carries run control. Cancellable from any thread (e.g. SIGINT or a GUI button) via Cancel().
|
||||
enum class ProcessMode {
|
||||
AzimuthalIntegration, // preprocess + azimuthal integration only (rugnux --mode=azint)
|
||||
FullAnalysis, // spot finding + indexing + refinement + integration (rugnux --mode=mx)
|
||||
Calibration // detector geometry from powder rings (rugnux --mode=calibration)
|
||||
};
|
||||
|
||||
struct ProcessConfig {
|
||||
ProcessMode mode = ProcessMode::FullAnalysis;
|
||||
// The shared analysis vocabulary (common/AnalysisSettings.h), not a rugnux one: the broker, the
|
||||
// viewer and rugnux all say what analysis runs the same way. rugnux's own --mode spellings are a
|
||||
// layer above this (RugnuxMode in rugnux_cli.cpp); AnalysisMode::None never reaches here, since a
|
||||
// rugnux run that analyses nothing has no purpose.
|
||||
AnalysisMode mode = AnalysisMode::MXStills;
|
||||
|
||||
int start_image = 0;
|
||||
int end_image = -1; // -1 => to the end of the dataset
|
||||
|
||||
@@ -54,8 +54,8 @@ std::string RugnuxCommandLine(const ProcessConfig &config,
|
||||
const std::string &input_file,
|
||||
const std::string &calibrant_name) {
|
||||
std::vector<std::string> args;
|
||||
const bool azint = (config.mode == ProcessMode::AzimuthalIntegration);
|
||||
const bool calibration = (config.mode == ProcessMode::Calibration);
|
||||
const bool azint = (config.mode == AnalysisMode::Azint);
|
||||
const bool calibration = (config.mode == AnalysisMode::PowderCalibration);
|
||||
args.emplace_back("rugnux");
|
||||
if (azint) {
|
||||
args.emplace_back("--mode");
|
||||
|
||||
@@ -5,14 +5,14 @@
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "Rugnux.h" // ProcessConfig, ProcessMode
|
||||
#include "Rugnux.h" // ProcessConfig, AnalysisMode
|
||||
|
||||
class DiffractionExperiment;
|
||||
|
||||
// Reconstruct an equivalent rugnux command line (including --mode azint / calibration) for a
|
||||
// configured run, so a job set up in the GUI can be handed off to a cluster. Covers the settings that
|
||||
// matter for the run, not every obscure flag; geometry is taken from the input file, so geometry
|
||||
// overrides are not emitted. calibrant_name is only used by ProcessMode::Calibration - the config
|
||||
// overrides are not emitted. calibrant_name is only used by AnalysisMode::PowderCalibration - the config
|
||||
// carries the calibrant's rings, and --calibrant takes the name they were resolved from.
|
||||
std::string RugnuxCommandLine(const ProcessConfig &config,
|
||||
const DiffractionExperiment &experiment,
|
||||
|
||||
+14
-13
@@ -71,14 +71,6 @@ constexpr double RELATIVE_B_DEFAULT_DEG = 10.0;
|
||||
// mutually exclusive by construction.
|
||||
enum class RugnuxMode { MX, Azint, Scale, Calibration };
|
||||
|
||||
// Azimuthal sectors used by --mode calibration --calibration rings when --azim-phi-bins was not given.
|
||||
// The default of 1 is a plain radial profile, in which a ring has been averaged over every direction so
|
||||
// nothing is left to say where its centre is (RingsFromAzimuthalProfile refuses below 4). Sweeping the
|
||||
// count on a LaB6 exposure and on a crystal's ice rings, 32 and above agree to about 0.1 px and the
|
||||
// residual is flat, while 8 sectors is visibly coarser (0.3 px away, twice the residual). Going higher
|
||||
// buys nothing and runs into the 65534-bin limit on q x azimuth, so 32 is the default.
|
||||
constexpr int CALIBRATION_AZIM_BINS_DEFAULT = 32;
|
||||
|
||||
void print_usage() {
|
||||
std::cout << "Usage rugnux {<options>} <input.h5>" << std::endl;
|
||||
std::cout << "Options:" << std::endl;
|
||||
@@ -94,7 +86,9 @@ void print_usage() {
|
||||
|
||||
std::cout << " Mode" << std::endl;
|
||||
std::cout << " --mode <name> What this run does (default: mx)" << std::endl;
|
||||
std::cout << " mx Full analysis - spot finding, indexing, integration and merging" << std::endl;
|
||||
std::cout << " mx Full analysis - spot finding, indexing, integration and merging. Rotation or stills is chosen from the goniometer" << std::endl;
|
||||
std::cout << " mx_rotation As mx, forced to rotation (two-pass indexing); same as -R" << std::endl;
|
||||
std::cout << " mx_stills As mx, forced to per-image stills; same as --force-still" << std::endl;
|
||||
std::cout << " azint Only azimuthal integration (no spot finding/indexing); writes <prefix>_process.h5" << std::endl;
|
||||
std::cout << " scale Only re-scale/merge the already-integrated reflections in <input> (no re-integration)" << std::endl;
|
||||
std::cout << " calibration Determine the detector geometry from powder rings; writes <prefix>.poni for pyFAI and <prefix>.json, whose dataset_settings member is a jfjoch_broker dataset_settings body as it stands" << std::endl;
|
||||
@@ -1066,11 +1060,16 @@ static int RunRugnux(int argc, char **argv) {
|
||||
case OPT_MODE: {
|
||||
const std::string m = optarg ? optarg : "";
|
||||
if (m == "mx") mode = RugnuxMode::MX;
|
||||
// The two MX modes spelled out. mx picks between them from the goniometer; these say which
|
||||
// one, and are the same tokens the shared AnalysisMode uses. mx_stills is what --force-still
|
||||
// has always done, and mx_rotation what -R does, so they are spellings and not new switches.
|
||||
else if (m == "mx_stills") { mode = RugnuxMode::MX; force_still = true; }
|
||||
else if (m == "mx_rotation") { mode = RugnuxMode::MX; rotation_indexing = true; two_pass_rotation = true; }
|
||||
else if (m == "azint") mode = RugnuxMode::Azint;
|
||||
else if (m == "scale") mode = RugnuxMode::Scale;
|
||||
else if (m == "calibration") mode = RugnuxMode::Calibration;
|
||||
else {
|
||||
logger.Error("Invalid --mode: {} (expected mx|azint|scale|calibration)", m);
|
||||
logger.Error("Invalid --mode: {} (expected mx|mx_rotation|mx_stills|azint|scale|calibration)", m);
|
||||
return 1;
|
||||
}
|
||||
break;
|
||||
@@ -2033,7 +2032,7 @@ static int RunRugnux(int argc, char **argv) {
|
||||
if (polarization_factor) experiment.PolarizationFactor(polarization_factor.value());
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::AzimuthalIntegration;
|
||||
config.mode = AnalysisMode::Azint;
|
||||
config.start_image = start_image;
|
||||
config.end_image = end_image;
|
||||
config.stride = image_stride;
|
||||
@@ -2077,7 +2076,7 @@ static int RunRugnux(int argc, char **argv) {
|
||||
if (polarization_factor) experiment.PolarizationFactor(polarization_factor.value());
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::Calibration;
|
||||
config.mode = AnalysisMode::PowderCalibration;
|
||||
config.calibration_method = calibration_method;
|
||||
config.calibration_refine_tilt = calibration_refine_tilt;
|
||||
// -C wins over --calibrant: a cell given on the command line IS the standard, and the built-in
|
||||
@@ -2653,7 +2652,9 @@ static int RunRugnux(int argc, char **argv) {
|
||||
// Run the shared full-analysis workflow (rotation indexing + scaling/merging live in
|
||||
// Rugnux; the experiment above carries all algorithm settings).
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::FullAnalysis;
|
||||
// Rotation vs stills has been settled above (goniometer autodetect, --force-still, -R), so the
|
||||
// shared mode can state which of the two MX analyses this run is rather than leaving it implicit.
|
||||
config.mode = rotation_indexing ? AnalysisMode::MXRotation : AnalysisMode::MXStills;
|
||||
config.start_image = start_image;
|
||||
config.end_image = end_image;
|
||||
config.stride = image_stride;
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#include <catch2/catch_all.hpp>
|
||||
|
||||
#include "../common/AnalysisSettings.h"
|
||||
#include "../common/DiffractionExperiment.h"
|
||||
#include "../frame_serialize/CBORStream2Serializer.h"
|
||||
#include "../frame_serialize/CBORStream2Deserializer.h"
|
||||
#include "../writer/FileWriter.h"
|
||||
#include "../reader/JFJochHDF5Reader.h"
|
||||
|
||||
TEST_CASE("AnalysisMode_Default", "[AnalysisMode]") {
|
||||
// MXStills, not None: a deployment whose configuration predates the mode must keep analysing.
|
||||
CHECK(AnalysisSettings().GetMode() == AnalysisMode::MXStills);
|
||||
CHECK(DiffractionExperiment().GetAnalysisMode() == AnalysisMode::MXStills);
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_Names", "[AnalysisMode]") {
|
||||
for (auto mode : {AnalysisMode::None, AnalysisMode::MXRotation, AnalysisMode::MXStills,
|
||||
AnalysisMode::Azint, AnalysisMode::Grid, AnalysisMode::PowderCalibration}) {
|
||||
auto back = AnalysisModeFromName(AnalysisModeName(mode));
|
||||
REQUIRE(back.has_value());
|
||||
CHECK(*back == mode);
|
||||
}
|
||||
CHECK_FALSE(AnalysisModeFromName("").has_value());
|
||||
CHECK_FALSE(AnalysisModeFromName("mx").has_value());
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_Stages", "[AnalysisMode]") {
|
||||
// The table itself. Bragg integration must never be on where indexing is off - nothing is
|
||||
// predicted without a lattice.
|
||||
for (auto mode : {AnalysisMode::None, AnalysisMode::MXRotation, AnalysisMode::MXStills,
|
||||
AnalysisMode::Azint, AnalysisMode::Grid, AnalysisMode::PowderCalibration}) {
|
||||
const auto s = AnalysisModeStages(mode);
|
||||
CHECK((!s.bragg_integration || s.indexing));
|
||||
CHECK((!s.scale_merge || s.bragg_integration));
|
||||
}
|
||||
|
||||
CHECK(AnalysisModeStages(AnalysisMode::MXStills).indexing);
|
||||
CHECK(AnalysisModeStages(AnalysisMode::MXRotation).indexing);
|
||||
|
||||
const auto none = AnalysisModeStages(AnalysisMode::None);
|
||||
CHECK_FALSE(none.spot_finding);
|
||||
CHECK_FALSE(none.azimuthal_integration);
|
||||
|
||||
const auto azint = AnalysisModeStages(AnalysisMode::Azint);
|
||||
CHECK_FALSE(azint.spot_finding);
|
||||
CHECK_FALSE(azint.indexing);
|
||||
CHECK_FALSE(azint.scoring);
|
||||
CHECK(azint.azimuthal_integration);
|
||||
|
||||
// Calibration keeps the spot finder: --calibration spots fits the pooled spots.
|
||||
const auto calib = AnalysisModeStages(AnalysisMode::PowderCalibration);
|
||||
CHECK(calib.spot_finding);
|
||||
CHECK_FALSE(calib.indexing);
|
||||
|
||||
const auto grid = AnalysisModeStages(AnalysisMode::Grid);
|
||||
CHECK(grid.spot_finding);
|
||||
CHECK(grid.scoring);
|
||||
CHECK_FALSE(grid.indexing);
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_PrecedenceOverSpotFindingSwitch", "[AnalysisMode]") {
|
||||
DiffractionExperiment experiment;
|
||||
DatasetSettings dataset;
|
||||
dataset.SpotFindingEnable(true).MaxSpotCount(500);
|
||||
experiment.ImportDatasetSettings(dataset);
|
||||
|
||||
CHECK(experiment.IsSpotFindingEnabled());
|
||||
|
||||
// A mode that analyses no spots wins over the deprecated per-dataset switch, and takes the spot
|
||||
// budget with it.
|
||||
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Azint));
|
||||
CHECK_FALSE(experiment.IsSpotFindingEnabled());
|
||||
CHECK(experiment.GetMaxSpotCount() == 0);
|
||||
|
||||
// Under a mode that does find spots, the deprecated switch is still able to turn it off.
|
||||
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::MXStills));
|
||||
CHECK(experiment.IsSpotFindingEnabled());
|
||||
dataset.SpotFindingEnable(false);
|
||||
experiment.ImportDatasetSettings(dataset);
|
||||
CHECK_FALSE(experiment.IsSpotFindingEnabled());
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_PowderCalibrationForcesCPUAzInt", "[AnalysisMode]") {
|
||||
// The FPGA integration core holds 2048 bins in total, so a sectored profile cannot be built there.
|
||||
DiffractionExperiment experiment;
|
||||
REQUIRE_FALSE(experiment.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
|
||||
|
||||
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
|
||||
CHECK(experiment.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
|
||||
CHECK(experiment.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount()
|
||||
== CALIBRATION_AZIM_BINS_DEFAULT);
|
||||
|
||||
// An explicit sector count stands; the mode only supplies one where none is usable.
|
||||
DiffractionExperiment explicit_bins;
|
||||
explicit_bins.ImportAzimuthalIntegrationSettings(
|
||||
AzimuthalIntegrationSettings().AzimuthalBinCount(64));
|
||||
explicit_bins.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
|
||||
CHECK(explicit_bins.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount() == 64);
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_CBORStartRoundTrip", "[AnalysisMode][CBOR]") {
|
||||
std::vector<uint8_t> buffer(1024 * 1024);
|
||||
CBORStream2Serializer serializer(buffer.data(), buffer.size());
|
||||
|
||||
DiffractionExperiment experiment;
|
||||
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Grid));
|
||||
|
||||
StartMessage message{};
|
||||
experiment.FillMessage(message);
|
||||
REQUIRE(message.analysis_mode.has_value());
|
||||
REQUIRE_NOTHROW(serializer.SerializeSequenceStart(message));
|
||||
|
||||
auto deserialized = CBORStream2Deserialize(buffer.data(), serializer.GetBufferSize());
|
||||
REQUIRE(deserialized);
|
||||
REQUIRE(deserialized->start_message);
|
||||
REQUIRE(deserialized->start_message->analysis_mode.has_value());
|
||||
CHECK(*deserialized->start_message->analysis_mode == AnalysisMode::Grid);
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_CBORStartAbsentMode", "[AnalysisMode][CBOR]") {
|
||||
// A stream written before the mode existed says nothing, and is read back as saying nothing -
|
||||
// naming a mode for it would be an invention rather than provenance.
|
||||
std::vector<uint8_t> buffer(1024 * 1024);
|
||||
CBORStream2Serializer serializer(buffer.data(), buffer.size());
|
||||
|
||||
StartMessage message{};
|
||||
REQUIRE_NOTHROW(serializer.SerializeSequenceStart(message));
|
||||
|
||||
auto deserialized = CBORStream2Deserialize(buffer.data(), serializer.GetBufferSize());
|
||||
REQUIRE(deserialized);
|
||||
REQUIRE(deserialized->start_message);
|
||||
CHECK_FALSE(deserialized->start_message->analysis_mode.has_value());
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_HDF5MasterRoundTrip", "[AnalysisMode][HDF5][Full]") {
|
||||
// The mode is dataset-wide metadata: written to the master and read back from it, so a stored
|
||||
// file re-opens knowing what produced it.
|
||||
DiffractionExperiment x(DetJF(1));
|
||||
x.FilePrefix("test_analysis_mode").ImagesPerTrigger(1).OverwriteExistingFiles(true);
|
||||
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
||||
.IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
|
||||
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
||||
x.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Grid));
|
||||
RegisterHDF5Filter();
|
||||
|
||||
std::vector<uint16_t> image(x.GetPixelsNum(), 0);
|
||||
{
|
||||
StartMessage start_message;
|
||||
x.FillMessage(start_message);
|
||||
|
||||
FileWriter file_set(start_message);
|
||||
|
||||
DataMessage message{};
|
||||
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
||||
message.number = 0;
|
||||
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
||||
|
||||
EndMessage end_message;
|
||||
end_message.max_image_number = 1;
|
||||
REQUIRE_NOTHROW(file_set.WriteHDF5(end_message));
|
||||
file_set.Finalize();
|
||||
}
|
||||
{
|
||||
JFJochHDF5Reader reader;
|
||||
reader.ReadFile("test_analysis_mode_master.h5");
|
||||
auto dataset = reader.GetDataset();
|
||||
REQUIRE(dataset->file_analysis_mode.has_value());
|
||||
CHECK(*dataset->file_analysis_mode == AnalysisMode::Grid);
|
||||
}
|
||||
remove("test_analysis_mode_master.h5");
|
||||
remove("test_analysis_mode_data_000001.h5");
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
ADD_EXECUTABLE(jfjoch_test
|
||||
DiffractionExperimentTest.cpp
|
||||
AnalysisSettingsTest.cpp
|
||||
RawToConvertedGeometryTest.cpp
|
||||
../common/RawToConvertedGeometry.h
|
||||
../common/DiffractionExperiment.h
|
||||
|
||||
@@ -59,7 +59,7 @@ TEST_CASE("Rugnux_Rotation", "[large]") {
|
||||
experiment.ImportIndexingSettings(indexing);
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::FullAnalysis;
|
||||
config.mode = AnalysisMode::MXStills;
|
||||
config.nthreads = default_threads();
|
||||
config.spot_finding = DiffractionExperiment::DefaultDataProcessingSettings();
|
||||
config.spot_finding.indexing = true;
|
||||
|
||||
@@ -55,7 +55,7 @@ TEST_CASE("Rugnux_AzInt", "[HDF5][Full]") {
|
||||
REQUIRE(dataset);
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::AzimuthalIntegration;
|
||||
config.mode = AnalysisMode::Azint;
|
||||
config.nthreads = 2;
|
||||
config.output_prefix = "process_azint_out";
|
||||
|
||||
@@ -94,7 +94,7 @@ TEST_CASE("Rugnux_NoOutput", "[HDF5][Full]") {
|
||||
|
||||
// Empty output prefix => process without writing any file.
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::AzimuthalIntegration;
|
||||
config.mode = AnalysisMode::Azint;
|
||||
config.nthreads = 3;
|
||||
|
||||
Rugnux process(reader, dataset->experiment, *dataset->pixel_mask, config);
|
||||
@@ -118,7 +118,7 @@ TEST_CASE("Rugnux_Cancel", "[HDF5][Full]") {
|
||||
auto dataset = reader.GetDataset();
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::AzimuthalIntegration;
|
||||
config.mode = AnalysisMode::Azint;
|
||||
config.nthreads = 2;
|
||||
|
||||
Rugnux process(reader, dataset->experiment, *dataset->pixel_mask, config);
|
||||
@@ -144,7 +144,7 @@ TEST_CASE("RugnuxCommandLine_Full", "[process]") {
|
||||
x.SpaceGroupNumber(96);
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::FullAnalysis;
|
||||
config.mode = AnalysisMode::MXStills;
|
||||
config.nthreads = 8;
|
||||
config.output_prefix = "run1";
|
||||
config.end_image = 500;
|
||||
@@ -174,7 +174,7 @@ TEST_CASE("RugnuxCommandLine_AzInt", "[process]") {
|
||||
x.ImportAzimuthalIntegrationSettings(a);
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = ProcessMode::AzimuthalIntegration;
|
||||
config.mode = AnalysisMode::Azint;
|
||||
config.nthreads = 2;
|
||||
config.output_prefix = "az";
|
||||
|
||||
|
||||
@@ -126,8 +126,8 @@ JFJochViewerSettingsDock::JFJochViewerSettingsDock(const SpotFindingSettings &sp
|
||||
stack->addWidget(new QWidget(stack)); // AzInt: the shared section below is the whole page
|
||||
stack->addWidget(BuildCalibrationPage());
|
||||
connect(group, &QButtonGroup::idClicked, this, [this, stack](int id) {
|
||||
mode_ = id == 1 ? ProcessMode::AzimuthalIntegration
|
||||
: id == 2 ? ProcessMode::Calibration : ProcessMode::FullAnalysis;
|
||||
mode_ = id == 1 ? AnalysisMode::Azint
|
||||
: id == 2 ? AnalysisMode::PowderCalibration : AnalysisMode::MXStills;
|
||||
stack->setCurrentIndex(id);
|
||||
azintSection_->setVisible(id != 0);
|
||||
});
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
#include "../../image_analysis/spot_finding/SpotFindingSettings.h"
|
||||
#include "../../reader/JFJochReaderDataset.h"
|
||||
#include "../../reader/JFJochReaderImage.h"
|
||||
#include "../../rugnux/Rugnux.h" // ProcessMode
|
||||
#include "../../rugnux/Rugnux.h" // AnalysisMode
|
||||
#include "../ReferenceMtzInfo.h"
|
||||
#include "PowderCalibrationWidget.h" // CalibrationSelection
|
||||
|
||||
@@ -58,8 +58,8 @@ signals:
|
||||
void referenceSelected(QString path, QString column); // user picked a reference MTZ / column
|
||||
void reanalyzeImage(bool armed); // "Analyze image" toggle
|
||||
// "Analyze dataset". The mode is the selected page; the calibrant selection is only read for
|
||||
// ProcessMode::Calibration and is carried along so the job needs no second copy of the controls.
|
||||
void analyzeDataset(ProcessMode mode, CalibrationSelection calibration);
|
||||
// AnalysisMode::PowderCalibration and is carried along so the job needs no second copy of the controls.
|
||||
void analyzeDataset(AnalysisMode mode, CalibrationSelection calibration);
|
||||
|
||||
private:
|
||||
SpotFindingSettings spot_;
|
||||
@@ -75,7 +75,7 @@ private:
|
||||
// turns the pair into that field, and per-image only ever applies to stills.
|
||||
bool adaptive_min_pix_ = false;
|
||||
int64_t min_pix_value_ = 2;
|
||||
ProcessMode mode_ = ProcessMode::FullAnalysis; // the selected page, and what "Analyze dataset" runs
|
||||
AnalysisMode mode_ = AnalysisMode::MXStills; // the selected page, and what "Analyze dataset" runs
|
||||
|
||||
// "Analyze dataset" hero button, disabled while a live HTTP source is connected.
|
||||
QPushButton *analyzeDatasetBtn_ = nullptr;
|
||||
|
||||
@@ -55,19 +55,21 @@ namespace {
|
||||
}
|
||||
|
||||
// Short name of a run mode, for the dialog title, the Mode column and the run label.
|
||||
const char *mode_name(ProcessMode mode) {
|
||||
const char *mode_name(AnalysisMode mode) {
|
||||
switch (mode) {
|
||||
case ProcessMode::AzimuthalIntegration: return "AzInt";
|
||||
case ProcessMode::Calibration: return "Calib";
|
||||
case ProcessMode::FullAnalysis:
|
||||
default: return "Full";
|
||||
case AnalysisMode::Azint: return "AzInt";
|
||||
case AnalysisMode::PowderCalibration: return "Calib";
|
||||
case AnalysisMode::Grid: return "Grid";
|
||||
case AnalysisMode::None: return "None";
|
||||
case AnalysisMode::MXRotation:
|
||||
case AnalysisMode::MXStills:
|
||||
default: return "Full";
|
||||
}
|
||||
}
|
||||
|
||||
// Azimuthal sectors a calibration by rings falls back to when the panel has too few of them: with
|
||||
// one sector the profile has averaged the ring over every direction and cannot locate it. Same
|
||||
// value, for the same reason, as rugnux --mode calibration.
|
||||
constexpr int CALIBRATION_AZIM_BINS_DEFAULT = 32;
|
||||
// one sector the profile has averaged the ring over every direction and cannot locate it. The
|
||||
// shared default (common/AnalysisSettings.h) - the same value, for the same reason, everywhere.
|
||||
}
|
||||
|
||||
JFJochProcessingJobsWindow::JFJochProcessingJobsWindow(JFJochImageReadingWorker *worker, QWidget *parent)
|
||||
@@ -141,8 +143,8 @@ int JFJochProcessingJobsWindow::askJob(const ReprocessingInputs &inputs, JobSpec
|
||||
QDialog dlg(window()); // centre on the main window, not inside the processing dock
|
||||
// The kind of job comes from the panel's MX / AzInt / Calib toggle, so the dialog only collects
|
||||
// the run options.
|
||||
const bool azint = spec.mode == ProcessMode::AzimuthalIntegration;
|
||||
const bool calibration = spec.mode == ProcessMode::Calibration;
|
||||
const bool azint = spec.mode == AnalysisMode::Azint;
|
||||
const bool calibration = spec.mode == AnalysisMode::PowderCalibration;
|
||||
dlg.setWindowTitle(calibration ? "New detector-calibration job"
|
||||
: azint ? "New azimuthal-integration job" : "New full-analysis job");
|
||||
|
||||
@@ -208,7 +210,7 @@ int JFJochProcessingJobsWindow::askJob(const ReprocessingInputs &inputs, JobSpec
|
||||
// cell from the strongest indexed frames, then the main pass re-indexes with it. It is stills-only
|
||||
// and anchors on a known cell, so offer it only for a stills-with-cell run; default it on there, to
|
||||
// match the rugnux CLI (a no-op for rotation / de-novo stills). Stills mode = rotation indexing off.
|
||||
const bool stills_with_cell = spec.mode == ProcessMode::FullAnalysis
|
||||
const bool stills_with_cell = AnalysisModeIsMX(spec.mode)
|
||||
&& !inputs.experiment.GetIndexingSettings().GetRotationIndexing()
|
||||
&& inputs.experiment.GetUnitCell().has_value();
|
||||
auto *refine_geometry = new QCheckBox("Refine geometry (stills)", &dlg);
|
||||
@@ -236,7 +238,7 @@ int JFJochProcessingJobsWindow::askJob(const ReprocessingInputs &inputs, JobSpec
|
||||
// cell + rotation axis from the whole sweep, then re-integrates at that geometry (only the refined pass
|
||||
// is written, as the canonical <prefix>_* output). Rotation-only;
|
||||
// default on there, to match the rugnux CLI.
|
||||
const bool rotation_run = spec.mode == ProcessMode::FullAnalysis
|
||||
const bool rotation_run = AnalysisModeIsMX(spec.mode)
|
||||
&& inputs.experiment.GetIndexingSettings().GetRotationIndexing();
|
||||
auto *postrefine = new QCheckBox("Post-refine geometry (rotation, two-pass)", &dlg);
|
||||
postrefine->setChecked(rotation_run);
|
||||
@@ -361,7 +363,7 @@ ProcessConfig JFJochProcessingJobsWindow::buildConfig(const JobSpec &spec, const
|
||||
// Files land at output_prefix; leave it empty (write nothing, stats only) when neither output is
|
||||
// wanted. The two flags then select which files are actually written there. A calibration always
|
||||
// keeps the prefix - the .poni written next to it is the whole point of the run.
|
||||
config.output_prefix = (spec.save_h5 || spec.save_merged || spec.mode == ProcessMode::Calibration)
|
||||
config.output_prefix = (spec.save_h5 || spec.save_merged || spec.mode == AnalysisMode::PowderCalibration)
|
||||
? spec.prefix.toStdString() : std::string();
|
||||
config.write_process_h5 = spec.save_h5;
|
||||
config.write_merged = spec.save_merged;
|
||||
@@ -376,7 +378,7 @@ ProcessConfig JFJochProcessingJobsWindow::buildConfig(const JobSpec &spec, const
|
||||
// (--no-fit-spindle is the deviation), so the dialog's single switch sets both.
|
||||
config.estimate_beam_center = spec.estimate_beam_center;
|
||||
config.fit_spindle = spec.estimate_beam_center;
|
||||
if (spec.mode == ProcessMode::Calibration) {
|
||||
if (spec.mode == AnalysisMode::PowderCalibration) {
|
||||
config.calibration_method = spec.calibration.method;
|
||||
config.calibration_refine_tilt = spec.calibration.refine_tilt;
|
||||
config.calibrant_ring_q = spec.calibration.ring_q;
|
||||
@@ -385,7 +387,7 @@ ProcessConfig JFJochProcessingJobsWindow::buildConfig(const JobSpec &spec, const
|
||||
config.spot_finding.enable = true;
|
||||
config.spot_finding.indexing = false;
|
||||
}
|
||||
if (spec.mode == ProcessMode::FullAnalysis) {
|
||||
if (AnalysisModeIsMX(spec.mode)) {
|
||||
// Rotation indexing follows the panel's rotation axis (= the experiment's indexing
|
||||
// setting); a rotation run uses 60 first-pass images to find the lattice.
|
||||
config.rotation_indexing = inputs.experiment.GetIndexingSettings().GetRotationIndexing();
|
||||
@@ -404,7 +406,7 @@ ProcessConfig JFJochProcessingJobsWindow::buildConfig(const JobSpec &spec, const
|
||||
return config;
|
||||
}
|
||||
|
||||
void JFJochProcessingJobsWindow::newJob(ProcessMode mode, CalibrationSelection calibration) {
|
||||
void JFJochProcessingJobsWindow::newJob(AnalysisMode mode, CalibrationSelection calibration) {
|
||||
const ReprocessingInputs inputs = worker_->GetReprocessingInputs();
|
||||
if (!inputs.valid) {
|
||||
QMessageBox::information(this, "Processing", "Open a file first (processing is not available for live HTTP data).");
|
||||
@@ -439,13 +441,13 @@ void JFJochProcessingJobsWindow::newJob(ProcessMode mode, CalibrationSelection c
|
||||
// Calibrating from the run-summed profile needs the profile to be binned in azimuth; the panel's
|
||||
// bin count is an azimuthal-integration setting and defaults to a plain radial profile, which
|
||||
// carries no information about where the ring centre is. Same fallback as the rugnux CLI.
|
||||
if (spec.mode == ProcessMode::Calibration && spec.calibration.method == CalibrationMethod::Rings
|
||||
if (spec.mode == AnalysisMode::PowderCalibration && spec.calibration.method == CalibrationMethod::Rings
|
||||
&& experiment.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount() < 4) {
|
||||
AzimuthalIntegrationSettings azint = experiment.GetAzimuthalIntegrationSettings();
|
||||
azint.AzimuthalBinCount(CALIBRATION_AZIM_BINS_DEFAULT);
|
||||
experiment.ImportAzimuthalIntegrationSettings(azint);
|
||||
}
|
||||
if (spec.mode == ProcessMode::FullAnalysis && spec.scaling) {
|
||||
if (AnalysisModeIsMX(spec.mode) && spec.scaling) {
|
||||
ScalingSettings scaling = RugnuxDefaultScalingSettings(config.rotation_indexing);
|
||||
// Keep what the settings dock does expose; take the rest from the shared defaults.
|
||||
const auto &dock = inputs.experiment.GetScalingSettings();
|
||||
@@ -487,7 +489,7 @@ void JFJochProcessingJobsWindow::newJob(ProcessMode mode, CalibrationSelection c
|
||||
info.label = label;
|
||||
if (spec.save_h5)
|
||||
info.snapshot_path = QString::fromStdString(config.output_prefix) + "_process.h5";
|
||||
if (spec.mode == ProcessMode::Calibration) {
|
||||
if (spec.mode == AnalysisMode::PowderCalibration) {
|
||||
info.poni_path = spec.prefix + ".poni";
|
||||
info.calibration_header = experiment.GetDiffractionGeometry();
|
||||
calibration_experiment_ = experiment;
|
||||
|
||||
@@ -45,8 +45,8 @@ public slots:
|
||||
void clearJobs(); // reset the table on a new file (re-adds the Original row)
|
||||
void setActiveRun(QString active_id); // bold the row of the run currently shown in the plots
|
||||
// The panel's "Analyze dataset" action; the mode is the panel's selected page. The calibrant
|
||||
// selection is only used by ProcessMode::Calibration.
|
||||
void newJob(ProcessMode mode = ProcessMode::FullAnalysis, CalibrationSelection calibration = {});
|
||||
// selection is only used by AnalysisMode::PowderCalibration.
|
||||
void newJob(AnalysisMode mode = AnalysisMode::MXStills, CalibrationSelection calibration = {});
|
||||
|
||||
private slots:
|
||||
void cancelJob();
|
||||
@@ -80,7 +80,7 @@ private:
|
||||
QToolButton *graph_btn = nullptr; // per-row "show statistics" button (enabled once stats exist)
|
||||
};
|
||||
struct JobSpec {
|
||||
ProcessMode mode = ProcessMode::FullAnalysis;
|
||||
AnalysisMode mode = AnalysisMode::MXStills;
|
||||
int start_image = 0;
|
||||
int end_image = 0; // 0 == to the end
|
||||
int threads = 4;
|
||||
|
||||
@@ -496,6 +496,12 @@ void NXmx::Detector(const StartMessage &start, const EndMessage &end) {
|
||||
|
||||
void NXmx::MX(const StartMessage &start) {
|
||||
HDF5Group(*hdf5_file, "/entry/MX").NXClass("NXcollection");
|
||||
|
||||
// Which analysis produced this file, in the same spelling the CBOR stream and the rugnux --mode
|
||||
// option use. Absent when the stream carried no mode, i.e. was written before the field existed.
|
||||
if (start.analysis_mode.has_value())
|
||||
hdf5_file->SaveScalar("/entry/MX/analysis_mode", AnalysisModeName(start.analysis_mode.value()));
|
||||
|
||||
switch (start.indexing_algorithm) {
|
||||
case IndexingAlgorithmEnum::FFBIDX:
|
||||
hdf5_file->SaveScalar("/entry/MX/indexing_algorithm", "FFBIDX");
|
||||
|
||||
Reference in new issue
Block a user