analysis: every analysis method carries its own settings, and a raster's indexing is one of them
AnalysisSettings had begun collecting per-method parameters - the calibrant was already in it, and the grid thresholds were about to be. That makes the structure every method reads grow whenever any one method gains a knob, and it puts a field in front of readers for whom it means nothing. So: AnalysisSettings keeps what all methods share, which for now is the mode, and each method gets a class of its own bound the same way. GridScanAnalysisSettings holds the protein-score threshold, the minimum cells per crystal, the decisive single-cell score, the maximum crystals reported and the indexing switch. CalibrationSettings holds the calibrant and the ring source. Both sit on DiffractionExperiment outside the per-run dataset member, both have an Import/Get pair, and both have their own endpoint - /config/grid_scan_analysis and /config/calibration - which is how every other settings group in this API is already reached. Grid indexing is no longer fixed in the stages table. It was turned off there on cost grounds, and that reasoning does not hold: a raster runs at up to 100 Hz, which the FFT indexer keeps up with, and a fixed-target serial experiment with a known cell wants ffbidx on every cell, where a raster that indexes is most of the measurement. So it is a setting, and DEFAULTS ON. It is additive rather than a change of answer - blobs are still found on the protein score, so indexing alters nothing about which cells are called crystals and only adds what was found in them, including the per-cell lattice count, which is the cheapest multi-lattice or cracked-crystal signal there is. That makes indexing the one stage a mode does not decide. AnalysisModeStages still carries a value for it, but only as the setting's default, and DiffractionExperiment::GetAnalysisStages - which is what every gate reads - substitutes the configured one. The table row is marked so nobody reads it as the mode's answer. The calibration knobs stay coupled to the mode but the rule now lives with them: CalibrationSettings::ApplyToAzimuthalIntegration moves azimuthal integration onto the CPU and supplies sectors where fewer than four were asked for, carrying the reason with it - the FPGA integration core holds 2048 bins in total, so 32 sectors leave 64 q bins, which cannot locate a ring. Stated there because it will otherwise read as an FPGA defect to be fixed back onto that path, and it is not one: the core is sized for a detector at full rate, and a calibration exposure is a few images at a few Hz. Both imports apply it, so the order the mode and the calibration settings are set in does not matter. CalibrationMethod moves from image_analysis/geom_refinement/PowderCalibration.h into common/CalibrationSettings.h, which that header now includes. One enum, so the setting and the code consuming it are not two vocabularies; every existing user sees it unchanged. The grid thresholds have one home and it is this class. The raster work owns AnalyzeGridScan's parameter surface and carries PROTEIN_SCORE_THRESHOLD_DEFAULT / MIN_BLOB_CELLS_DEFAULT beside that header today; the header here states the signature that replaces them, so the two do not become competing defaults. The beam size deliberately stays a separate argument to AnalyzeGridScan: it is measured, not configured. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
This commit is contained in:
1 parent
eba81013cd
commit
98132d0f83
25 files changed
+855
-85
No files matched your search
@@ -206,6 +206,11 @@ void JFJochBrokerHttp::register_routes(httplib::Server &server) {
|
||||
server.Put("/config/bragg_integration", bind_json(&JFJochBrokerHttp::config_bragg_integration_put, Bragg_integration_settings{}));
|
||||
server.Get("/config/analysis", bind_noarg(&JFJochBrokerHttp::config_analysis_get));
|
||||
server.Put("/config/analysis", bind_json(&JFJochBrokerHttp::config_analysis_put, Analysis_settings{}));
|
||||
server.Get("/config/grid_scan_analysis", bind_noarg(&JFJochBrokerHttp::config_grid_scan_analysis_get));
|
||||
server.Put("/config/grid_scan_analysis",
|
||||
bind_json(&JFJochBrokerHttp::config_grid_scan_analysis_put, Grid_scan_analysis_settings{}));
|
||||
server.Get("/config/calibration", bind_noarg(&JFJochBrokerHttp::config_calibration_get));
|
||||
server.Put("/config/calibration", bind_json(&JFJochBrokerHttp::config_calibration_put, Calibration_settings{}));
|
||||
server.Get("/config/instrument", bind_noarg(&JFJochBrokerHttp::config_instrument_get));
|
||||
server.Put("/config/instrument", bind_json(&JFJochBrokerHttp::config_instrument_put, Instrument_metadata{}));
|
||||
server.Put("/config/internal_generator_image", [this](const httplib::Request &req, httplib::Response &res) {
|
||||
@@ -604,6 +609,8 @@ void JFJochBrokerHttp::statistics_get(httplib::Response &response) {
|
||||
statistics.setIndexing(Convert(state_machine.GetIndexingSettings()));
|
||||
statistics.setBraggIntegration(Convert(state_machine.GetBraggIntegrationSettings()));
|
||||
statistics.setAnalysis(Convert(state_machine.GetAnalysisSettings()));
|
||||
statistics.setGridScanAnalysis(Convert(state_machine.GetGridScanAnalysisSettings()));
|
||||
statistics.setCalibrationSettings(Convert(state_machine.GetCalibrationSettings()));
|
||||
statistics.setDarkMask(Convert(state_machine.GetDarkMaskSettings()));
|
||||
statistics.setImagePusher(Convert(state_machine.GetImagePusherStatus()));
|
||||
|
||||
@@ -858,6 +865,26 @@ void JFJochBrokerHttp::config_analysis_put(const Analysis_settings &analysisSett
|
||||
response.status = 200;
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::config_grid_scan_analysis_get(httplib::Response &response) {
|
||||
ProcessOutput(Convert(state_machine.GetGridScanAnalysisSettings()), response);
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::config_grid_scan_analysis_put(const Grid_scan_analysis_settings &gridScanAnalysisSettings,
|
||||
httplib::Response &response) {
|
||||
state_machine.SetGridScanAnalysisSettings(Convert(gridScanAnalysisSettings));
|
||||
response.status = 200;
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::config_calibration_get(httplib::Response &response) {
|
||||
ProcessOutput(Convert(state_machine.GetCalibrationSettings()), response);
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::config_calibration_put(const Calibration_settings &calibrationSettings,
|
||||
httplib::Response &response) {
|
||||
state_machine.SetCalibrationSettings(Convert(calibrationSettings));
|
||||
response.status = 200;
|
||||
}
|
||||
|
||||
void JFJochBrokerHttp::result_scan_get(httplib::Response &response) {
|
||||
auto ret = state_machine.GetScanResult();
|
||||
if (ret.has_value())
|
||||
|
||||
@@ -184,6 +184,13 @@ class JFJochBrokerHttp {
|
||||
void config_analysis_get(httplib::Response &response);
|
||||
void config_analysis_put(const org::openapitools::server::model::Analysis_settings &analysisSettings,
|
||||
httplib::Response &response);
|
||||
void config_grid_scan_analysis_get(httplib::Response &response);
|
||||
void config_grid_scan_analysis_put(
|
||||
const org::openapitools::server::model::Grid_scan_analysis_settings &gridScanAnalysisSettings,
|
||||
httplib::Response &response);
|
||||
void config_calibration_get(httplib::Response &response);
|
||||
void config_calibration_put(const org::openapitools::server::model::Calibration_settings &calibrationSettings,
|
||||
httplib::Response &response);
|
||||
|
||||
void config_dark_mask_get(httplib::Response &response);
|
||||
void config_dark_mask_put(const org::openapitools::server::model::Dark_mask_settings &darkMaskSettings,
|
||||
|
||||
@@ -194,6 +194,14 @@ void ParseFacilityConfiguration(const org::openapitools::server::model::Jfjoch_s
|
||||
experiment.ImportAnalysisSettings(analysis);
|
||||
}
|
||||
|
||||
if (j.gridScanAnalysisIsSet())
|
||||
experiment.ImportGridScanAnalysisSettings(Convert(j.getGridScanAnalysis()));
|
||||
|
||||
// After the mode, so that selecting PowderCalibration in the same file still lands the azimuthal
|
||||
// integration the fit needs.
|
||||
if (j.calibrationSettingsIsSet())
|
||||
experiment.ImportCalibrationSettings(Convert(j.getCalibrationSettings()));
|
||||
|
||||
if (j.darkMaskIsSet())
|
||||
experiment.ImportDarkMaskSettings(Convert(j.getDarkMask()));
|
||||
}
|
||||
|
||||
@@ -1240,6 +1240,34 @@ void JFJochStateMachine::SetAnalysisSettings(const AnalysisSettings &input) {
|
||||
experiment.ImportAnalysisSettings(input);
|
||||
}
|
||||
|
||||
GridScanAnalysisSettings JFJochStateMachine::GetGridScanAnalysisSettings() const {
|
||||
std::unique_lock ul(experiment_indexing_settings_mutex);
|
||||
return experiment.GetGridScanAnalysisSettings();
|
||||
}
|
||||
|
||||
void JFJochStateMachine::SetGridScanAnalysisSettings(const GridScanAnalysisSettings &input) {
|
||||
std::unique_lock ul(m);
|
||||
if (IsRunning())
|
||||
throw WrongDAQStateException("Cannot change grid scan analysis settings during data collection");
|
||||
|
||||
std::unique_lock ul2(experiment_indexing_settings_mutex);
|
||||
experiment.ImportGridScanAnalysisSettings(input);
|
||||
}
|
||||
|
||||
CalibrationSettings JFJochStateMachine::GetCalibrationSettings() const {
|
||||
std::unique_lock ul(experiment_indexing_settings_mutex);
|
||||
return experiment.GetCalibrationSettings();
|
||||
}
|
||||
|
||||
void JFJochStateMachine::SetCalibrationSettings(const CalibrationSettings &input) {
|
||||
std::unique_lock ul(m);
|
||||
if (IsRunning())
|
||||
throw WrongDAQStateException("Cannot change calibration settings during data collection");
|
||||
|
||||
std::unique_lock ul2(experiment_indexing_settings_mutex);
|
||||
experiment.ImportCalibrationSettings(input);
|
||||
}
|
||||
|
||||
std::optional<ScanResult> JFJochStateMachine::GetScanResult() const {
|
||||
std::unique_lock ul(m);
|
||||
if (IsRunning())
|
||||
|
||||
@@ -252,6 +252,12 @@ public:
|
||||
|
||||
void SetAnalysisSettings(const AnalysisSettings &input);
|
||||
AnalysisSettings GetAnalysisSettings() const;
|
||||
|
||||
void SetGridScanAnalysisSettings(const GridScanAnalysisSettings &input);
|
||||
GridScanAnalysisSettings GetGridScanAnalysisSettings() const;
|
||||
|
||||
void SetCalibrationSettings(const CalibrationSettings &input);
|
||||
CalibrationSettings GetCalibrationSettings() const;
|
||||
PixelMaskStatistics GetPixelMaskStatistics() const;
|
||||
|
||||
void GetStartMessageFromBuffer(std::vector<uint8_t> &v);
|
||||
|
||||
@@ -1146,9 +1146,6 @@ AnalysisSettings Convert(const org::openapitools::server::model::Analysis_settin
|
||||
break;
|
||||
}
|
||||
|
||||
if (input.calibrantIsSet())
|
||||
ret.Calibrant(input.getCalibrant());
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
@@ -1179,6 +1176,61 @@ org::openapitools::server::model::Analysis_settings Convert(const AnalysisSettin
|
||||
"Rotation MX analysis cannot be reported by this API");
|
||||
}
|
||||
ret.setMode(tmp);
|
||||
return ret;
|
||||
}
|
||||
|
||||
GridScanAnalysisSettings Convert(const org::openapitools::server::model::Grid_scan_analysis_settings &input) {
|
||||
GridScanAnalysisSettings ret;
|
||||
ret.ProteinScoreThreshold(input.getProteinScoreThreshold());
|
||||
ret.MinBlobCells(input.getMinBlobCells());
|
||||
ret.DecisiveSingleCellScore(input.getDecisiveSingleCellScore());
|
||||
ret.MaxCrystals(input.getMaxCrystals());
|
||||
ret.Indexing(input.isIndexing());
|
||||
return ret;
|
||||
}
|
||||
|
||||
org::openapitools::server::model::Grid_scan_analysis_settings Convert(const GridScanAnalysisSettings &input) {
|
||||
org::openapitools::server::model::Grid_scan_analysis_settings ret;
|
||||
ret.setProteinScoreThreshold(input.GetProteinScoreThreshold());
|
||||
ret.setMinBlobCells(input.GetMinBlobCells());
|
||||
ret.setDecisiveSingleCellScore(input.GetDecisiveSingleCellScore());
|
||||
ret.setMaxCrystals(input.GetMaxCrystals());
|
||||
ret.setIndexing(input.IsIndexing());
|
||||
return ret;
|
||||
}
|
||||
|
||||
CalibrationSettings Convert(const org::openapitools::server::model::Calibration_settings &input) {
|
||||
CalibrationSettings ret;
|
||||
// No default: the compiler must object here when a source is added to the API.
|
||||
switch (input.getMethod().getValue()) {
|
||||
case org::openapitools::server::model::Calibration_method::eCalibration_method::INVALID_VALUE_OPENAPI_GENERATED:
|
||||
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Unknown calibration method");
|
||||
case org::openapitools::server::model::Calibration_method::eCalibration_method::RINGS:
|
||||
ret.Method(CalibrationMethod::Rings);
|
||||
break;
|
||||
case org::openapitools::server::model::Calibration_method::eCalibration_method::SPOTS:
|
||||
ret.Method(CalibrationMethod::Spots);
|
||||
break;
|
||||
}
|
||||
|
||||
if (input.calibrantIsSet())
|
||||
ret.Calibrant(input.getCalibrant());
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
org::openapitools::server::model::Calibration_settings Convert(const CalibrationSettings &input) {
|
||||
org::openapitools::server::model::Calibration_settings ret;
|
||||
org::openapitools::server::model::Calibration_method tmp;
|
||||
switch (input.GetMethod()) {
|
||||
case CalibrationMethod::Rings:
|
||||
tmp.setValue(org::openapitools::server::model::Calibration_method::eCalibration_method::RINGS);
|
||||
break;
|
||||
case CalibrationMethod::Spots:
|
||||
tmp.setValue(org::openapitools::server::model::Calibration_method::eCalibration_method::SPOTS);
|
||||
break;
|
||||
}
|
||||
ret.setMethod(tmp);
|
||||
if (!input.GetCalibrant().empty())
|
||||
ret.setCalibrant(input.GetCalibrant());
|
||||
return ret;
|
||||
|
||||
@@ -30,6 +30,8 @@
|
||||
#include "gen/model/Indexing_settings.h"
|
||||
#include "gen/model/Bragg_integration_settings.h"
|
||||
#include "gen/model/Analysis_settings.h"
|
||||
#include "gen/model/Grid_scan_analysis_settings.h"
|
||||
#include "gen/model/Calibration_settings.h"
|
||||
#include "gen/model/Grid_scan_result.h"
|
||||
#include "gen/model/Powder_calibration_output.h"
|
||||
#include "gen/model/Scan_result.h"
|
||||
@@ -44,6 +46,8 @@
|
||||
#include "../common/InstrumentMetadata.h"
|
||||
#include "../common/ScanResult.h"
|
||||
#include "../common/AnalysisSettings.h"
|
||||
#include "../common/GridScanAnalysisSettings.h"
|
||||
#include "../common/CalibrationSettings.h"
|
||||
#include "../image_pusher/ImagePusher.h"
|
||||
|
||||
SpotFindingSettings Convert(const org::openapitools::server::model::Spot_finding_settings &input);
|
||||
@@ -58,6 +62,12 @@ org::openapitools::server::model::Bragg_integration_settings Convert(const Bragg
|
||||
AnalysisSettings Convert(const org::openapitools::server::model::Analysis_settings &input);
|
||||
org::openapitools::server::model::Analysis_settings Convert(const AnalysisSettings &input);
|
||||
|
||||
GridScanAnalysisSettings Convert(const org::openapitools::server::model::Grid_scan_analysis_settings &input);
|
||||
org::openapitools::server::model::Grid_scan_analysis_settings Convert(const GridScanAnalysisSettings &input);
|
||||
|
||||
CalibrationSettings Convert(const org::openapitools::server::model::Calibration_settings &input);
|
||||
org::openapitools::server::model::Calibration_settings Convert(const CalibrationSettings &input);
|
||||
|
||||
org::openapitools::server::model::Grid_scan_crystal Convert(const GridScanCrystal &input);
|
||||
org::openapitools::server::model::Powder_calibration_output Convert(const CalibrationResult &input);
|
||||
|
||||
|
||||
+177
-4
@@ -1991,6 +1991,10 @@ components:
|
||||
$ref: '#/components/schemas/bragg_integration_settings'
|
||||
analysis:
|
||||
$ref: '#/components/schemas/analysis_settings'
|
||||
grid_scan_analysis:
|
||||
$ref: '#/components/schemas/grid_scan_analysis_settings'
|
||||
calibration_settings:
|
||||
$ref: '#/components/schemas/calibration_settings'
|
||||
image_pusher:
|
||||
$ref: '#/components/schemas/image_pusher_status'
|
||||
error_message:
|
||||
@@ -2621,18 +2625,109 @@ components:
|
||||
analysis_settings:
|
||||
type: object
|
||||
description: |
|
||||
The analysis mode and the settings that only mean anything under one of them. Persistent: it
|
||||
survives a data collection, unlike the per-dataset settings given to /start.
|
||||
What every analysis method shares, and nothing else. Each method's own parameters have their
|
||||
own settings object and their own endpoint - grid_scan_analysis_settings
|
||||
(/config/grid_scan_analysis), calibration_settings (/config/calibration) - so that adding a
|
||||
parameter to one method does not widen the structure every method reads.
|
||||
|
||||
Persistent: it survives a data collection, unlike the per-dataset settings given to /start.
|
||||
required:
|
||||
- mode
|
||||
properties:
|
||||
mode:
|
||||
$ref: '#/components/schemas/analysis_mode'
|
||||
calibration_method:
|
||||
type: string
|
||||
description: |
|
||||
How the powder rings the detector geometry is fitted to are measured.
|
||||
Rings - the run-summed (q x azimuth) azimuthal profile: the ring measured at every azimuth.
|
||||
Spots - the pooled per-image spot lists: the ring sampled wherever the spot finder bit.
|
||||
enum:
|
||||
- "Rings"
|
||||
- "Spots"
|
||||
default: "Rings"
|
||||
calibration_settings:
|
||||
type: object
|
||||
description: |
|
||||
Settings for analysis mode PowderCalibration and nothing else.
|
||||
|
||||
Selecting that mode also moves azimuthal integration onto the CPU and, where fewer than four
|
||||
azimuthal sectors were asked for, sets a usable count. Ring positions are read off a
|
||||
(q x azimuth) profile, and the FPGA integration core holds 2048 bins in total - at 32 sectors
|
||||
that leaves 64 q bins, far too coarse to locate a ring. The frame rate this costs is not frame
|
||||
rate a calibration exposure needs.
|
||||
required:
|
||||
- method
|
||||
properties:
|
||||
calibrant:
|
||||
type: string
|
||||
description: |
|
||||
PowderCalibration only - which calibrant's ring d-spacings the geometry is fitted against.
|
||||
Ignored under every other mode.
|
||||
Which calibrant's ring d-spacings the geometry is fitted against. Empty selects the
|
||||
processing program's own default rather than meaning "no calibrant".
|
||||
method:
|
||||
$ref: '#/components/schemas/calibration_method'
|
||||
grid_scan_analysis_settings:
|
||||
type: object
|
||||
description: |
|
||||
Settings for analysis mode Grid and nothing else: how the per-image protein score is turned
|
||||
into a list of crystals, and whether the raster indexes as well as scores.
|
||||
required:
|
||||
- protein_score_threshold
|
||||
- min_blob_cells
|
||||
- decisive_single_cell_score
|
||||
- max_crystals
|
||||
- indexing
|
||||
properties:
|
||||
protein_score_threshold:
|
||||
type: number
|
||||
format: float
|
||||
minimum: 0
|
||||
maximum: 1
|
||||
default: 0.5
|
||||
description: |
|
||||
A raster cell counts as protein above this. The per-image protein score saturates, so this
|
||||
only has to separate "something diffracted here" from "nothing did".
|
||||
min_blob_cells:
|
||||
type: integer
|
||||
format: int64
|
||||
minimum: 1
|
||||
default: 3
|
||||
description: |
|
||||
How many cells above that threshold make a shape rather than a coincidence. Two cells can
|
||||
be the two ends of a single hit lying on a cell boundary; three is the smallest patch that
|
||||
is not.
|
||||
decisive_single_cell_score:
|
||||
type: number
|
||||
format: float
|
||||
minimum: 0
|
||||
maximum: 1
|
||||
default: 0.9
|
||||
description: |
|
||||
...unless one cell on its own is decisive. The rule above is about coincidences, and a
|
||||
lone cell scoring near the top of a saturating score is not one - a crystal smaller than
|
||||
the grid step lights exactly one cell, and refusing it would lose precisely the samples a
|
||||
fine raster is run to find. Set well above protein_score_threshold: this admits the
|
||||
obvious case, it does not lower the general threshold by the back door.
|
||||
max_crystals:
|
||||
type: integer
|
||||
format: int64
|
||||
minimum: 1
|
||||
default: 10
|
||||
description: |
|
||||
Most crystals reported. A raster over a loop full of shards can label dozens of blobs, and
|
||||
past the first few the list is no longer a ranking anyone acts on. Crystals are sorted by
|
||||
score, so this keeps the best.
|
||||
indexing:
|
||||
type: boolean
|
||||
default: true
|
||||
description: |
|
||||
Whether each raster cell is indexed as well as scored. On by default: a raster runs at up
|
||||
to 100 Hz, which the FFT indexer keeps up with, and it is additive - blobs are still found
|
||||
on the protein score, so indexing changes nothing about which cells are called crystals
|
||||
and only adds what was found in them. The lattice count per cell is the cheapest
|
||||
multi-lattice or cracked-crystal signal there is, and on a fixed-target serial experiment
|
||||
with a known cell a raster that indexes is most of the measurement. Turn it off for a very
|
||||
large raster where the GPU is the constraint.
|
||||
bragg_integration_settings:
|
||||
type: object
|
||||
description: "Settings for Bragg spot integration"
|
||||
@@ -2687,6 +2782,10 @@ components:
|
||||
$ref: '#/components/schemas/bragg_integration_settings'
|
||||
analysis:
|
||||
$ref: '#/components/schemas/analysis_settings'
|
||||
grid_scan_analysis:
|
||||
$ref: '#/components/schemas/grid_scan_analysis_settings'
|
||||
calibration_settings:
|
||||
$ref: '#/components/schemas/calibration_settings'
|
||||
detector_settings:
|
||||
$ref: '#/components/schemas/detector_settings'
|
||||
azim_int:
|
||||
@@ -3104,6 +3203,80 @@ paths:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/analysis_settings'
|
||||
/config/grid_scan_analysis:
|
||||
put:
|
||||
summary: Change grid scan analysis settings
|
||||
description: |
|
||||
This can only be done when detector is `Idle`, `Error` or `Inactive` states.
|
||||
Persistent - not reset by a data collection.
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/grid_scan_analysis_settings'
|
||||
responses:
|
||||
"200":
|
||||
description: Everything OK
|
||||
"400":
|
||||
description: Input parsing or validation error
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
description: Exception error
|
||||
"500":
|
||||
description: Error within Jungfraujoch code - see output message.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/error_message'
|
||||
get:
|
||||
summary: Get grid scan analysis configuration
|
||||
description: Can be done anytime
|
||||
responses:
|
||||
"200":
|
||||
description: Everything OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/grid_scan_analysis_settings'
|
||||
/config/calibration:
|
||||
put:
|
||||
summary: Change powder calibration settings
|
||||
description: |
|
||||
This can only be done when detector is `Idle`, `Error` or `Inactive` states.
|
||||
Persistent - not reset by a data collection.
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/calibration_settings'
|
||||
responses:
|
||||
"200":
|
||||
description: Everything OK
|
||||
"400":
|
||||
description: Input parsing or validation error
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
description: Exception error
|
||||
"500":
|
||||
description: Error within Jungfraujoch code - see output message.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/error_message'
|
||||
get:
|
||||
summary: Get powder calibration configuration
|
||||
description: Can be done anytime
|
||||
responses:
|
||||
"200":
|
||||
description: Everything OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/calibration_settings'
|
||||
/config/file_writer:
|
||||
put:
|
||||
summary: Change file writer settings
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#include "AnalysisSettings.h"
|
||||
#include "GridScanAnalysisSettings.h"
|
||||
|
||||
// The mode-to-analysis table. One place, one row per mode, and every gate in the pipeline reads it -
|
||||
// so what a mode does can be read off here instead of being reconstructed from `if (mode == ...)`
|
||||
@@ -12,7 +13,7 @@
|
||||
// MXRotation x x x x x x
|
||||
// MXStills x x x x x x
|
||||
// Azint - - - - - x
|
||||
// Grid x - - - x x
|
||||
// Grid x * - - x x * = default; a setting
|
||||
// PowderCalibration x - - - - x
|
||||
//
|
||||
// Two rows want a word.
|
||||
@@ -21,11 +22,12 @@
|
||||
// (q x azimuth) profile or to the POOLED SPOTS (rugnux --calibration rings|spots), and the second of
|
||||
// those is a spot finder run - so the mode must not decide this; whoever picks the source does.
|
||||
//
|
||||
// Grid does not index. A raster is thousands of frames and indexing is the expensive stage, and the
|
||||
// per-image scoring it ranks grid points on deliberately does not use it (indexing fires on ice, which
|
||||
// is exactly what the scoring has to see past). The cost of that choice is indexed_lattice_count, which
|
||||
// is the cheapest multi-lattice / cracked-crystal signal there is and which a raster therefore does not
|
||||
// get. It is one field in this table either way.
|
||||
// Grid indexes by default, but that entry is not the mode's to fix: it is
|
||||
// GridScanAnalysisSettings::indexing, and what stands here is only what that setting defaults to. A
|
||||
// raster runs at up to 100 Hz, which the FFT indexer keeps up with, and a fixed-target serial
|
||||
// experiment with a known cell wants ffbidx on every cell - so whether a raster indexes is a property
|
||||
// of the experiment, not of rasters. DiffractionExperiment::GetAnalysisStages substitutes the
|
||||
// configured value.
|
||||
//
|
||||
// The switch has no default: adding a mode must be a compile error here rather than a silent
|
||||
// fall-through to whatever the last row happened to be.
|
||||
@@ -39,7 +41,7 @@ AnalysisStages AnalysisModeStages(AnalysisMode mode) {
|
||||
case AnalysisMode::Azint:
|
||||
return {false, false, false, false, false, true};
|
||||
case AnalysisMode::Grid:
|
||||
return {true, false, false, false, true, true};
|
||||
return {true, GridScanAnalysisSettings().IsIndexing(), false, false, true, true};
|
||||
case AnalysisMode::PowderCalibration:
|
||||
return {true, false, false, false, false, true};
|
||||
}
|
||||
@@ -87,19 +89,10 @@ AnalysisSettings &AnalysisSettings::Mode(AnalysisMode input) {
|
||||
return *this;
|
||||
}
|
||||
|
||||
AnalysisSettings &AnalysisSettings::Calibrant(const std::string &input) {
|
||||
calibrant = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
AnalysisMode AnalysisSettings::GetMode() const {
|
||||
return mode;
|
||||
}
|
||||
|
||||
const std::string &AnalysisSettings::GetCalibrant() const {
|
||||
return calibrant;
|
||||
}
|
||||
|
||||
bool AnalysisSettings::IsMX() const {
|
||||
return AnalysisModeIsMX(mode);
|
||||
}
|
||||
+10
-22
@@ -31,25 +31,15 @@ enum class AnalysisMode {
|
||||
PowderCalibration // detector geometry from a calibrant's powder rings
|
||||
};
|
||||
|
||||
// Azimuthal sectors a powder calibration integrates into when nothing else is asked for. A plain
|
||||
// radial profile (1 sector) has averaged every ring over every direction, so nothing is left to say
|
||||
// where its centre is (RingsFromAzimuthalProfile refuses below 4). Sweeping the count on a LaB6
|
||||
// exposure and on a crystal's ice rings, 32 and above agree to about 0.1 px with a flat residual,
|
||||
// while 8 sectors is visibly coarser (0.3 px away, twice the residual).
|
||||
//
|
||||
// 32 is also what the FPGA integration core can carry: it holds FPGA_INTEGRATION_BIN_COUNT = 2048
|
||||
// bins in total (the receiver throws above that), so 32 sectors would leave 64 q bins - far too
|
||||
// coarse to fit a ring. PowderCalibration therefore forces azimuthal integration onto the CPU
|
||||
// (AzimuthalIntegrationSettings::ForceCPUinFPGAWorkflow), which lifts the bin limit at the cost of
|
||||
// frame rate. That is not a problem this mode has: a calibration exposure is a few images at a few
|
||||
// Hz, not a data collection.
|
||||
constexpr int CALIBRATION_AZIM_BINS_DEFAULT = 32;
|
||||
|
||||
// Which stages of the pipeline a mode runs. This is the whole point of the mode: it does not label a
|
||||
// run, it decides what happens in it, and every gate in the pipeline reads this rather than testing
|
||||
// the mode itself. The table is in AnalysisSettings.cpp - modes are the rows, stages the columns.
|
||||
struct AnalysisStages {
|
||||
bool spot_finding;
|
||||
// Fixed by the mode everywhere except Grid, where it is a setting
|
||||
// (GridScanAnalysisSettings::indexing). AnalysisModeStages carries the grid DEFAULT;
|
||||
// DiffractionExperiment::GetAnalysisStages is where the configured value takes over, and that is
|
||||
// what every gate reads.
|
||||
bool indexing;
|
||||
// Bragg prediction and integration. It follows indexing in every engine - nothing is predicted
|
||||
// without a lattice - so this never says yes where indexing says no.
|
||||
@@ -72,20 +62,18 @@ bool AnalysisModeIsMX(AnalysisMode mode);
|
||||
std::string AnalysisModeName(AnalysisMode mode);
|
||||
std::optional<AnalysisMode> AnalysisModeFromName(std::string_view name);
|
||||
|
||||
// The analysis mode and the settings that only mean anything under one of them. Persistent, not
|
||||
// per-dataset: it sits on DiffractionExperiment outside the DatasetSettings member, which is the one
|
||||
// thing a /start replaces wholesale.
|
||||
// What every analysis method shares, and nothing else. Each method's own knobs live in its own class
|
||||
// beside this one - GridScanAnalysisSettings, CalibrationSettings - so that adding a parameter to one
|
||||
// method cannot widen the structure every method reads.
|
||||
//
|
||||
// Persistent, not per-dataset: it sits on DiffractionExperiment outside the DatasetSettings member,
|
||||
// which is the one thing a /start replaces wholesale. The per-method classes are bound the same way.
|
||||
class AnalysisSettings {
|
||||
AnalysisMode mode = AnalysisMode::MXStills;
|
||||
// PowderCalibration only: which calibrant's ring d-spacings to fit against.
|
||||
std::string calibrant;
|
||||
// Grid decision thresholds land here when Grid is implemented.
|
||||
public:
|
||||
AnalysisSettings& Mode(AnalysisMode input);
|
||||
AnalysisSettings& Calibrant(const std::string &input);
|
||||
|
||||
[[nodiscard]] AnalysisMode GetMode() const;
|
||||
[[nodiscard]] const std::string &GetCalibrant() const;
|
||||
|
||||
[[nodiscard]] bool IsMX() const;
|
||||
};
|
||||
@@ -123,6 +123,10 @@ ADD_LIBRARY(JFJochCommon STATIC
|
||||
BraggIntegrationSettings.h
|
||||
AnalysisSettings.cpp
|
||||
AnalysisSettings.h
|
||||
GridScanAnalysisSettings.cpp
|
||||
GridScanAnalysisSettings.h
|
||||
CalibrationSettings.cpp
|
||||
CalibrationSettings.h
|
||||
SpotToSave.cpp
|
||||
XrayFluorescenceSpectrum.cpp
|
||||
XrayFluorescenceSpectrum.h
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#include "CalibrationSettings.h"
|
||||
|
||||
CalibrationSettings &CalibrationSettings::Calibrant(const std::string &input) {
|
||||
calibrant = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
CalibrationSettings &CalibrationSettings::Method(CalibrationMethod input) {
|
||||
method = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
const std::string &CalibrationSettings::GetCalibrant() const {
|
||||
return calibrant;
|
||||
}
|
||||
|
||||
CalibrationMethod CalibrationSettings::GetMethod() const {
|
||||
return method;
|
||||
}
|
||||
|
||||
void CalibrationSettings::ApplyToAzimuthalIntegration(AzimuthalIntegrationSettings &azint) const {
|
||||
azint.ForceCPUinFPGAWorkflow(true);
|
||||
if (azint.GetAzimuthalBinCount() < 4)
|
||||
azint.AzimuthalBinCount(CALIBRATION_AZIM_BINS_DEFAULT);
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "AzimuthalIntegrationSettings.h"
|
||||
|
||||
// How the powder rings the detector geometry is fitted to are measured (rugnux --calibration).
|
||||
// Lives here rather than beside the fit itself so that the setting and the code that consumes it are
|
||||
// not two different vocabularies; image_analysis/geom_refinement/PowderCalibration.h includes this.
|
||||
enum class CalibrationMethod {
|
||||
Rings, // the run-summed (q x azimuth) azimuthal profile: the ring measured at every azimuth
|
||||
Spots // the pooled per-image spot lists: the ring sampled wherever the spot finder bit
|
||||
};
|
||||
|
||||
// Azimuthal sectors a powder calibration integrates into when nothing usable was asked for. A plain
|
||||
// radial profile has averaged every ring over every direction, so nothing is left to say where its
|
||||
// centre is (RingsFromAzimuthalProfile refuses below 4), and the azimuthal default of one sector is
|
||||
// exactly that. Sweeping the count on a LaB6 exposure and on a crystal's ice rings, 32 and above
|
||||
// agree to about 0.1 px with a flat residual, while 8 sectors is visibly coarser (0.3 px away, twice
|
||||
// the residual).
|
||||
constexpr int CALIBRATION_AZIM_BINS_DEFAULT = 32;
|
||||
|
||||
// Settings for AnalysisMode::PowderCalibration and nothing else. Per-method by the same rule as
|
||||
// GridScanAnalysisSettings: AnalysisSettings carries what every method shares, each method carries
|
||||
// its own knobs.
|
||||
class CalibrationSettings {
|
||||
// Which calibrant's ring d-spacings the geometry is fitted against. Empty means the front end's
|
||||
// own default (rugnux --calibrant) rather than "no calibrant".
|
||||
std::string calibrant;
|
||||
CalibrationMethod method = CalibrationMethod::Rings;
|
||||
public:
|
||||
CalibrationSettings& Calibrant(const std::string &input);
|
||||
CalibrationSettings& Method(CalibrationMethod input);
|
||||
|
||||
[[nodiscard]] const std::string &GetCalibrant() const;
|
||||
[[nodiscard]] CalibrationMethod GetMethod() const;
|
||||
|
||||
// The azimuthal integration a powder calibration needs, applied to whatever is configured.
|
||||
//
|
||||
// Ring positions are read off a (q x azimuth) profile, so the integration needs azimuthal sectors
|
||||
// AND enough q bins under each of them to place a ring. The FPGA integration core holds
|
||||
// FPGA_INTEGRATION_BIN_COUNT = 2048 bins in TOTAL, which at 32 sectors leaves 64 q bins - far too
|
||||
// coarse to locate a ring - so the integration is moved onto the CPU, where that limit does not
|
||||
// apply. This is not a defect on the FPGA path to be fixed back onto it: the core is sized for a
|
||||
// detector running at full rate, and a calibration exposure is a few images at a few Hz, so the
|
||||
// frame rate this costs is not frame rate this mode needs.
|
||||
//
|
||||
// Only supplies a sector count where fewer than four were asked for; an explicit count stands.
|
||||
void ApplyToAzimuthalIntegration(AzimuthalIntegrationSettings &azint) const;
|
||||
};
|
||||
@@ -1776,20 +1776,11 @@ BraggIntegrationSettings DiffractionExperiment::GetBraggIntegrationSettings() co
|
||||
DiffractionExperiment &DiffractionExperiment::ImportAnalysisSettings(const AnalysisSettings &input) {
|
||||
analysis = input;
|
||||
|
||||
// Powder calibration reads ring positions off a (q x azimuth) profile, so it needs azimuthal
|
||||
// sectors - and enough q bins under each of them to place a ring. The FPGA integration core holds
|
||||
// FPGA_INTEGRATION_BIN_COUNT = 2048 bins in total, which at CALIBRATION_AZIM_BINS_DEFAULT sectors
|
||||
// would leave 64 q bins, so the integration is moved onto the CPU where the limit does not apply.
|
||||
// The cost is frame rate, which this mode does not need: a calibration exposure is a few images at
|
||||
// a few Hz. Setting the mode is therefore all a caller has to do - see AnalysisSettings.h.
|
||||
if (analysis.GetMode() == AnalysisMode::PowderCalibration) {
|
||||
az_integration_settings.ForceCPUinFPGAWorkflow(true);
|
||||
// Only where no usable sector count was asked for. Below four sectors a ring has been averaged
|
||||
// over every direction and nothing is left to say where its centre is (RingsFromAzimuthalProfile
|
||||
// refuses there), and the default of one is exactly that; an explicit count stands.
|
||||
if (az_integration_settings.GetAzimuthalBinCount() < 4)
|
||||
az_integration_settings.AzimuthalBinCount(CALIBRATION_AZIM_BINS_DEFAULT);
|
||||
}
|
||||
// Powder calibration cannot be integrated on the FPGA; the rule and the reason live with the
|
||||
// calibration settings. Applied from both imports, so it does not matter which of the mode and
|
||||
// the calibration settings is set first.
|
||||
if (analysis.GetMode() == AnalysisMode::PowderCalibration)
|
||||
calibration.ApplyToAzimuthalIntegration(az_integration_settings);
|
||||
|
||||
return *this;
|
||||
}
|
||||
@@ -1798,12 +1789,40 @@ AnalysisSettings DiffractionExperiment::GetAnalysisSettings() const {
|
||||
return analysis;
|
||||
}
|
||||
|
||||
DiffractionExperiment &
|
||||
DiffractionExperiment::ImportGridScanAnalysisSettings(const GridScanAnalysisSettings &input) {
|
||||
grid_scan_analysis = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
GridScanAnalysisSettings DiffractionExperiment::GetGridScanAnalysisSettings() const {
|
||||
return grid_scan_analysis;
|
||||
}
|
||||
|
||||
DiffractionExperiment &DiffractionExperiment::ImportCalibrationSettings(const CalibrationSettings &input) {
|
||||
calibration = input;
|
||||
|
||||
if (analysis.GetMode() == AnalysisMode::PowderCalibration)
|
||||
calibration.ApplyToAzimuthalIntegration(az_integration_settings);
|
||||
|
||||
return *this;
|
||||
}
|
||||
|
||||
CalibrationSettings DiffractionExperiment::GetCalibrationSettings() const {
|
||||
return calibration;
|
||||
}
|
||||
|
||||
AnalysisMode DiffractionExperiment::GetAnalysisMode() const {
|
||||
return analysis.GetMode();
|
||||
}
|
||||
|
||||
AnalysisStages DiffractionExperiment::GetAnalysisStages() const {
|
||||
return AnalysisModeStages(analysis.GetMode());
|
||||
auto stages = AnalysisModeStages(analysis.GetMode());
|
||||
// Every stage is the mode's to decide except one: whether a raster indexes is a setting, and this
|
||||
// is where the configured value replaces the default the table carries.
|
||||
if (analysis.GetMode() == AnalysisMode::Grid)
|
||||
stages.indexing = grid_scan_analysis.IsIndexing();
|
||||
return stages;
|
||||
}
|
||||
|
||||
DiffractionExperiment &DiffractionExperiment::ImportScalingSettings(const ScalingSettings &input) {
|
||||
|
||||
@@ -38,6 +38,8 @@ constexpr int64_t SaturationLimitFromValue(int64_t saturation_value) { return sa
|
||||
#include "IndexingSettings.h"
|
||||
#include "BraggIntegrationSettings.h"
|
||||
#include "AnalysisSettings.h"
|
||||
#include "GridScanAnalysisSettings.h"
|
||||
#include "CalibrationSettings.h"
|
||||
#include "ScalingSettings.h"
|
||||
|
||||
#include <gemmi/symmetry.hpp>
|
||||
@@ -87,7 +89,10 @@ class DiffractionExperiment {
|
||||
BraggIntegrationSettings bragg_integration_settings;
|
||||
// Persistent, deliberately outside the DatasetSettings member above: a /start replaces `dataset`
|
||||
// wholesale, and the analysis mode is a property of how the instrument is set up, not of one run.
|
||||
// One shared group plus one per analysis method, bound the same way.
|
||||
AnalysisSettings analysis;
|
||||
GridScanAnalysisSettings grid_scan_analysis;
|
||||
CalibrationSettings calibration;
|
||||
ScalingSettings scaling_settings;
|
||||
|
||||
DarkMaskSettings dark_mask_settings;
|
||||
@@ -203,9 +208,16 @@ public:
|
||||
DiffractionExperiment& ImportBraggIntegrationSettings(const BraggIntegrationSettings& input);
|
||||
BraggIntegrationSettings GetBraggIntegrationSettings() const;
|
||||
|
||||
// PowderCalibration also rewrites the azimuthal-integration settings it needs - see the definition.
|
||||
DiffractionExperiment& ImportAnalysisSettings(const AnalysisSettings& input);
|
||||
AnalysisSettings GetAnalysisSettings() const;
|
||||
|
||||
DiffractionExperiment& ImportGridScanAnalysisSettings(const GridScanAnalysisSettings& input);
|
||||
GridScanAnalysisSettings GetGridScanAnalysisSettings() const;
|
||||
|
||||
// Under AnalysisMode::PowderCalibration this also rewrites the azimuthal integration the fit
|
||||
// needs - see CalibrationSettings::ApplyToAzimuthalIntegration for what and why.
|
||||
DiffractionExperiment& ImportCalibrationSettings(const CalibrationSettings& input);
|
||||
CalibrationSettings GetCalibrationSettings() const;
|
||||
[[nodiscard]] AnalysisMode GetAnalysisMode() const;
|
||||
// What this experiment's mode runs. Every gate in the pipeline reads this; the table is in
|
||||
// common/AnalysisSettings.cpp.
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#include "GridScanAnalysisSettings.h"
|
||||
|
||||
GridScanAnalysisSettings &GridScanAnalysisSettings::ProteinScoreThreshold(float input) {
|
||||
protein_score_threshold = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
GridScanAnalysisSettings &GridScanAnalysisSettings::MinBlobCells(int64_t input) {
|
||||
min_blob_cells = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
GridScanAnalysisSettings &GridScanAnalysisSettings::DecisiveSingleCellScore(float input) {
|
||||
decisive_single_cell_score = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
GridScanAnalysisSettings &GridScanAnalysisSettings::MaxCrystals(int64_t input) {
|
||||
max_crystals = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
GridScanAnalysisSettings &GridScanAnalysisSettings::Indexing(bool input) {
|
||||
indexing = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
float GridScanAnalysisSettings::GetProteinScoreThreshold() const {
|
||||
return protein_score_threshold;
|
||||
}
|
||||
|
||||
int64_t GridScanAnalysisSettings::GetMinBlobCells() const {
|
||||
return min_blob_cells;
|
||||
}
|
||||
|
||||
float GridScanAnalysisSettings::GetDecisiveSingleCellScore() const {
|
||||
return decisive_single_cell_score;
|
||||
}
|
||||
|
||||
int64_t GridScanAnalysisSettings::GetMaxCrystals() const {
|
||||
return max_crystals;
|
||||
}
|
||||
|
||||
bool GridScanAnalysisSettings::IsIndexing() const {
|
||||
return indexing;
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
// Settings for AnalysisMode::Grid and nothing else. Per-method by the same rule as
|
||||
// CalibrationSettings: AnalysisSettings carries what every method shares, each method carries its
|
||||
// own knobs.
|
||||
//
|
||||
// THIS IS THE SINGLE HOME FOR THESE VALUES. AnalyzeGridScan
|
||||
// (image_analysis/grid_scan_analysis/AnalyzeGridScan.h) is to take this object as one argument -
|
||||
//
|
||||
// GridScanResult AnalyzeGridScan(const ScanResult &scan, const GridScanSettings &grid,
|
||||
// float beam_size_x_um, float beam_size_y_um,
|
||||
// const GridScanAnalysisSettings &settings = {});
|
||||
//
|
||||
// - rather than loose parameters carrying defaults of their own, so that a knob has one default and
|
||||
// one place where it is documented. PROTEIN_SCORE_THRESHOLD_DEFAULT and MIN_BLOB_CELLS_DEFAULT are
|
||||
// replaced by the two members below and must not be reintroduced beside that header.
|
||||
//
|
||||
// The beam size stays a separate argument on purpose: it is measured (from the file, or --beam-size),
|
||||
// not configured, and rugnux reports where it came from beside the crystals it sized.
|
||||
class GridScanAnalysisSettings {
|
||||
// A cell counts as protein above this. The per-image protein score saturates, so this only has
|
||||
// to separate "something diffracted here" from "nothing did".
|
||||
float protein_score_threshold = 0.5f;
|
||||
// How many cells above that threshold make a shape rather than a coincidence. Two cells can be
|
||||
// the two ends of a single hit lying on a cell boundary; three is the smallest patch that is not.
|
||||
int64_t min_blob_cells = 3;
|
||||
// ...unless one cell on its own is decisive. The rule above is about coincidences, and a lone
|
||||
// cell scoring near the top of a saturating score is not one - a crystal smaller than the grid
|
||||
// step lights exactly one cell, and refusing it would lose precisely the samples a fine raster is
|
||||
// run to find. Well above protein_score_threshold on purpose: this admits the obvious case, it
|
||||
// does not lower the general threshold by the back door.
|
||||
float decisive_single_cell_score = 0.9f;
|
||||
// Most crystals reported. A raster over a loop full of shards can label dozens of blobs, and past
|
||||
// the first few the list is no longer a ranking anyone acts on - the DAQ collects from the top of
|
||||
// it. Crystals are sorted by score, so this keeps the best.
|
||||
int64_t max_crystals = 10;
|
||||
// Whether each raster cell is indexed as well as scored.
|
||||
//
|
||||
// On by default. It is affordable - a raster runs at up to 100 Hz, which the FFT indexer keeps up
|
||||
// with - and it is additive: the blobs are still found on the protein score, so indexing changes
|
||||
// nothing about which cells are called crystals and only adds what was found in them. The lattice
|
||||
// count per cell is the cheapest multi-lattice or cracked-crystal signal there is, and on a
|
||||
// fixed-target serial experiment with a known cell (-C, ffbidx) a raster that indexes is most of
|
||||
// the measurement. Turn it off for a very large raster where the GPU is the constraint.
|
||||
bool indexing = true;
|
||||
public:
|
||||
GridScanAnalysisSettings& ProteinScoreThreshold(float input);
|
||||
GridScanAnalysisSettings& MinBlobCells(int64_t input);
|
||||
GridScanAnalysisSettings& DecisiveSingleCellScore(float input);
|
||||
GridScanAnalysisSettings& MaxCrystals(int64_t input);
|
||||
GridScanAnalysisSettings& Indexing(bool input);
|
||||
|
||||
[[nodiscard]] float GetProteinScoreThreshold() const;
|
||||
[[nodiscard]] int64_t GetMinBlobCells() const;
|
||||
[[nodiscard]] float GetDecisiveSingleCellScore() const;
|
||||
[[nodiscard]] int64_t GetMaxCrystals() const;
|
||||
[[nodiscard]] bool IsIndexing() const;
|
||||
};
|
||||
@@ -5,6 +5,8 @@
|
||||
|
||||
* jfjoch_broker takes an explicit analysis mode (`/config/analysis`, the `analysis` block of the configuration file, and the Analysis panel of the web frontend): `MXStills` (the default), `Azint`, `Grid`, `PowderCalibration` or `None`. The mode decides what runs and takes precedence over `dataset_settings.spot_finding` and `spot_finding_settings.enable`, which are now deprecated.
|
||||
* `PowderCalibration` moves azimuthal integration onto the CPU and sets a sector count of its own, since the FPGA integration core cannot hold enough bins for a sectored profile; a calibration exposure then runs at a few Hz.
|
||||
* Each analysis method has its own settings and its own endpoint: `/config/grid_scan_analysis` (protein-score threshold, minimum cells per crystal, decisive single-cell score, maximum crystals reported, and whether the raster indexes) and `/config/calibration` (calibrant and ring source). `/config/analysis` now carries the mode alone.
|
||||
* A grid scan indexes each raster cell by default; turn it off with `indexing` in `/config/grid_scan_analysis`.
|
||||
* Rotation MX analysis is not offered online: it is absent from the REST API and refused by the broker. Rotation data is collected with jfjoch_broker and processed with rugnux.
|
||||
* `rugnux --mode` accepts `mx_rotation` and `mx_stills` beside `mx`, which continues to choose between them from the goniometer.
|
||||
* The stream and the written file record which analysis produced them, as `analysis_mode` in the CBOR start message and `/entry/MX/analysis_mode` in the HDF5 master.
|
||||
|
||||
@@ -44,6 +44,8 @@ import DarkMaskSettings from "./components/DarkMaskSettings";
|
||||
import IndexingSettings from "./components/IndexingSettings";
|
||||
import BraggIntegrationSettings from "./components/BraggIntegrationSettings";
|
||||
import AnalysisSettings from "./components/AnalysisSettings";
|
||||
import GridScanAnalysisSettings from "./components/GridScanAnalysisSettings";
|
||||
import CalibrationSettings from "./components/CalibrationSettings";
|
||||
import AzIntSettings from "./components/AzIntSettings";
|
||||
import ROI from "./components/ROI";
|
||||
import FileWriterSettings from "./components/FileWriterSettings";
|
||||
@@ -216,6 +218,8 @@ function App() {
|
||||
<IndexingSettings s={s.indexing} status={s.broker}/>
|
||||
<BraggIntegrationSettings s={s.bragg_integration}/>
|
||||
<AzIntSettings s={s.az_int}/>
|
||||
<GridScanAnalysisSettings s={s.grid_scan_analysis}/>
|
||||
<CalibrationSettings s={s.calibration_settings}/>
|
||||
</PanelColumns>
|
||||
{/* ROI holds three wide side-by-side tables — keep it full width. */}
|
||||
<ROI s={s.roi}/>
|
||||
|
||||
@@ -4,7 +4,6 @@ import FormControl from "@mui/material/FormControl";
|
||||
import InputLabel from "@mui/material/InputLabel";
|
||||
import Select, {SelectChangeEvent} from "@mui/material/Select";
|
||||
import MenuItem from "@mui/material/MenuItem";
|
||||
import TextField from "@mui/material/TextField";
|
||||
import {analysis_mode, analysis_settings} from "../client";
|
||||
import {putConfigAnalysisMutation} from "../client/@tanstack/react-query.gen";
|
||||
import _ from "lodash";
|
||||
@@ -62,23 +61,9 @@ function AnalysisSettings({s: serverS}: MyProps) {
|
||||
</FormControl>
|
||||
The mode decides what runs, and takes precedence over the spot-finding switches. Rotation
|
||||
analysis is not available online - collect the sweep here and process the stored file with
|
||||
rugnux.
|
||||
rugnux. Grid scan and powder calibration have settings of their own, in the panels beside
|
||||
this one.
|
||||
|
||||
{s.mode === analysis_mode.POWDER_CALIBRATION && (
|
||||
<>
|
||||
<b>Calibrant</b>
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<TextField id="calibrant" label="Calibrant" variant="outlined"
|
||||
value={s.calibrant ?? ""}
|
||||
onChange={(e) => setS(prev => ({...prev, calibrant: e.target.value}))}
|
||||
fullWidth/>
|
||||
</FormControl>
|
||||
Which standard's ring d-spacings the geometry is fitted against. Powder calibration
|
||||
also moves azimuthal integration onto the CPU - the FPGA core cannot hold enough bins
|
||||
for a sectored profile - so it runs at a few Hz, which is all a calibration exposure
|
||||
needs.
|
||||
</>
|
||||
)}
|
||||
</SettingsPanel>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
import {memo, useEffect, useState} from 'react';
|
||||
|
||||
import FormControl from "@mui/material/FormControl";
|
||||
import InputLabel from "@mui/material/InputLabel";
|
||||
import Select, {SelectChangeEvent} from "@mui/material/Select";
|
||||
import MenuItem from "@mui/material/MenuItem";
|
||||
import TextField from "@mui/material/TextField";
|
||||
import {calibration_method, calibration_settings} from "../client";
|
||||
import {putConfigCalibrationMutation} from "../client/@tanstack/react-query.gen";
|
||||
import _ from "lodash";
|
||||
import SettingsPanel from "./SettingsPanel";
|
||||
import {useUpload} from "./useUpload";
|
||||
|
||||
type MyProps = {
|
||||
s?: calibration_settings
|
||||
}
|
||||
|
||||
const default_calibration_settings: calibration_settings = {
|
||||
method: calibration_method.RINGS
|
||||
};
|
||||
|
||||
function CalibrationSettings({s: serverS}: MyProps) {
|
||||
const [s, setS] = useState<calibration_settings>(default_calibration_settings);
|
||||
const [lastDownloadedS, setLastDownloadedS] = useState<calibration_settings>(default_calibration_settings);
|
||||
const {submit, pending, snackbar} = useUpload(putConfigCalibrationMutation());
|
||||
|
||||
useEffect(() => {
|
||||
if ((serverS !== undefined) && !_.isEqual(serverS, lastDownloadedS)) {
|
||||
setS(serverS);
|
||||
setLastDownloadedS(serverS);
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [serverS]);
|
||||
|
||||
const handleMethodChange = (event: SelectChangeEvent) => {
|
||||
setS(prev => ({...prev, method: event.target.value as calibration_method}));
|
||||
};
|
||||
|
||||
const dirty = !_.isEqual(s, lastDownloadedS);
|
||||
|
||||
return (
|
||||
<SettingsPanel title="Powder calibration" dirty={dirty}
|
||||
onUpload={() => submit(s)} uploadDisabled={pending} snackbar={snackbar}>
|
||||
<b>Calibrant</b>
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<TextField id="calibrant" label="Calibrant" variant="outlined"
|
||||
value={s.calibrant ?? ""}
|
||||
onChange={(e) => setS(prev => ({...prev, calibrant: e.target.value}))}
|
||||
fullWidth/>
|
||||
</FormControl>
|
||||
Which standard's ring d-spacings the geometry is fitted against. Empty selects the
|
||||
processing program's own default rather than meaning "no calibrant".
|
||||
|
||||
<b>Ring source</b>
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<InputLabel>Ring source</InputLabel>
|
||||
<Select label="Ring source" value={s.method} onChange={handleMethodChange}>
|
||||
<MenuItem value={calibration_method.RINGS}>Rings — the summed azimuthal profile</MenuItem>
|
||||
<MenuItem value={calibration_method.SPOTS}>Spots — the pooled per-image spot lists</MenuItem>
|
||||
</Select>
|
||||
</FormControl>
|
||||
Selecting the powder calibration analysis mode also moves azimuthal integration onto the
|
||||
CPU and gives it enough azimuthal sectors to locate a ring, so it runs at a few Hz — which
|
||||
is all a calibration exposure needs.
|
||||
</SettingsPanel>
|
||||
);
|
||||
}
|
||||
|
||||
export default memo(CalibrationSettings);
|
||||
@@ -0,0 +1,133 @@
|
||||
import {memo, useEffect, useState} from 'react';
|
||||
|
||||
import {Checkbox, FormControl, FormControlLabel} from '@mui/material';
|
||||
import {grid_scan_analysis_settings} from "../client";
|
||||
import {putConfigGridScanAnalysisMutation} from "../client/@tanstack/react-query.gen";
|
||||
import _ from "lodash";
|
||||
import NumberTextField from "./NumberTextField";
|
||||
import SettingsPanel from "./SettingsPanel";
|
||||
import {useUpload} from "./useUpload";
|
||||
|
||||
type MyProps = {
|
||||
s?: grid_scan_analysis_settings
|
||||
}
|
||||
|
||||
const default_grid_scan_analysis_settings: grid_scan_analysis_settings = {
|
||||
protein_score_threshold: 0.5,
|
||||
min_blob_cells: 3,
|
||||
decisive_single_cell_score: 0.9,
|
||||
max_crystals: 10,
|
||||
indexing: true
|
||||
};
|
||||
|
||||
function GridScanAnalysisSettings({s: serverS}: MyProps) {
|
||||
const [s, setS] = useState<grid_scan_analysis_settings>(default_grid_scan_analysis_settings);
|
||||
const [lastDownloadedS, setLastDownloadedS] =
|
||||
useState<grid_scan_analysis_settings>(default_grid_scan_analysis_settings);
|
||||
const [downloadCounter, setDownloadCounter] = useState(0);
|
||||
const [thresholdErr, setThresholdErr] = useState(false);
|
||||
const [minCellsErr, setMinCellsErr] = useState(false);
|
||||
const [decisiveErr, setDecisiveErr] = useState(false);
|
||||
const [maxCrystalsErr, setMaxCrystalsErr] = useState(false);
|
||||
const {submit, pending, snackbar} = useUpload(putConfigGridScanAnalysisMutation());
|
||||
|
||||
useEffect(() => {
|
||||
if ((serverS !== undefined) && !_.isEqual(serverS, lastDownloadedS)) {
|
||||
setS(serverS);
|
||||
setLastDownloadedS(serverS);
|
||||
setDownloadCounter(c => c + 1);
|
||||
setThresholdErr(false);
|
||||
setMinCellsErr(false);
|
||||
setDecisiveErr(false);
|
||||
setMaxCrystalsErr(false);
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [serverS]);
|
||||
|
||||
const hasError = () => thresholdErr || minCellsErr || decisiveErr || maxCrystalsErr;
|
||||
const dirty = !_.isEqual(s, lastDownloadedS);
|
||||
|
||||
return (
|
||||
<SettingsPanel title="Grid scan analysis" dirty={dirty}
|
||||
onUpload={() => submit(s)} uploadDisabled={pending || hasError()} snackbar={snackbar}>
|
||||
<b>Which cells are a crystal</b>
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<NumberTextField
|
||||
default={0.5}
|
||||
start_val={s.protein_score_threshold}
|
||||
label={"Protein score threshold"}
|
||||
min={0}
|
||||
max={1}
|
||||
counter={downloadCounter}
|
||||
callback={(val: number, err: boolean) => {
|
||||
setS(prev => ({...prev, protein_score_threshold: val}));
|
||||
setThresholdErr(err);
|
||||
}}
|
||||
float decimals={2}
|
||||
fullWidth/>
|
||||
</FormControl>
|
||||
The per-image protein score saturates, so this only has to separate "something diffracted
|
||||
here" from "nothing did".
|
||||
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<NumberTextField
|
||||
default={3}
|
||||
start_val={s.min_blob_cells}
|
||||
label={"Minimum cells per crystal"}
|
||||
min={1}
|
||||
counter={downloadCounter}
|
||||
callback={(val: number, err: boolean) => {
|
||||
setS(prev => ({...prev, min_blob_cells: val}));
|
||||
setMinCellsErr(err);
|
||||
}}
|
||||
fullWidth/>
|
||||
</FormControl>
|
||||
Two cells can be the two ends of a single hit on a cell boundary; three is the smallest
|
||||
patch that is a shape rather than a coincidence.
|
||||
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<NumberTextField
|
||||
default={0.9}
|
||||
start_val={s.decisive_single_cell_score}
|
||||
label={"Decisive single-cell score"}
|
||||
min={0}
|
||||
max={1}
|
||||
counter={downloadCounter}
|
||||
callback={(val: number, err: boolean) => {
|
||||
setS(prev => ({...prev, decisive_single_cell_score: val}));
|
||||
setDecisiveErr(err);
|
||||
}}
|
||||
float decimals={2}
|
||||
fullWidth/>
|
||||
</FormControl>
|
||||
One cell scoring this high is a crystal on its own — a crystal smaller than the grid step
|
||||
lights exactly one cell. Keep it well above the threshold above.
|
||||
|
||||
<b>Reporting</b>
|
||||
<FormControl sx={{width: "80%"}}>
|
||||
<NumberTextField
|
||||
default={10}
|
||||
start_val={s.max_crystals}
|
||||
label={"Maximum crystals reported"}
|
||||
min={1}
|
||||
counter={downloadCounter}
|
||||
callback={(val: number, err: boolean) => {
|
||||
setS(prev => ({...prev, max_crystals: val}));
|
||||
setMaxCrystalsErr(err);
|
||||
}}
|
||||
fullWidth/>
|
||||
</FormControl>
|
||||
Crystals are sorted by score, so this keeps the best.
|
||||
|
||||
<b>Indexing</b>
|
||||
<FormControlLabel
|
||||
control={<Checkbox checked={s.indexing ?? true}
|
||||
onChange={(e) => setS(prev => ({...prev, indexing: e.target.checked}))}/>}
|
||||
label="Index each raster cell"/>
|
||||
Additive: crystals are still found on the protein score, so indexing only adds what was
|
||||
found in them. Turn it off for a very large raster where the GPU is the constraint.
|
||||
</SettingsPanel>
|
||||
);
|
||||
}
|
||||
|
||||
export default memo(GridScanAnalysisSettings);
|
||||
@@ -8,18 +8,13 @@
|
||||
#include <vector>
|
||||
|
||||
#include "../../common/AzimuthalIntegrationMapping.h"
|
||||
#include "../../common/CalibrationSettings.h" // CalibrationMethod
|
||||
#include "../../common/DiffractionExperiment.h"
|
||||
#include "../../common/DiffractionGeometry.h"
|
||||
#include "../../common/SpotToSave.h"
|
||||
#include "PowderAutoSeed.h"
|
||||
#include "RingOptimizer.h" // RingFitUncertainty
|
||||
|
||||
// How the powder rings the detector geometry is fitted to are measured (rugnux --calibration).
|
||||
enum class CalibrationMethod {
|
||||
Rings, // the run-summed (q x azimuth) azimuthal profile: the ring measured at every azimuth
|
||||
Spots // the pooled per-image spot lists: the ring sampled wherever the spot finder bit
|
||||
};
|
||||
|
||||
struct CalibrationResult {
|
||||
DiffractionGeometry geometry; // the fitted geometry
|
||||
size_t ring_points = 0; // ring measurements the fit used
|
||||
|
||||
@@ -2075,6 +2075,11 @@ static int RunRugnux(int argc, char **argv) {
|
||||
ApplyRugnuxExperimentDefaults(experiment);
|
||||
if (polarization_factor) experiment.PolarizationFactor(polarization_factor.value());
|
||||
|
||||
// The calibrant and the ring source on the experiment as well as on the config: one home for
|
||||
// them, and what the written file records the calibration as having used.
|
||||
experiment.ImportCalibrationSettings(
|
||||
CalibrationSettings().Calibrant(calibrant).Method(calibration_method));
|
||||
|
||||
ProcessConfig config;
|
||||
config.mode = AnalysisMode::PowderCalibration;
|
||||
config.calibration_method = calibration_method;
|
||||
|
||||
@@ -55,10 +55,48 @@ TEST_CASE("AnalysisMode_Stages", "[AnalysisMode]") {
|
||||
CHECK(calib.spot_finding);
|
||||
CHECK_FALSE(calib.indexing);
|
||||
|
||||
// Grid indexing is a setting, so the table only carries its default.
|
||||
const auto grid = AnalysisModeStages(AnalysisMode::Grid);
|
||||
CHECK(grid.spot_finding);
|
||||
CHECK(grid.scoring);
|
||||
CHECK_FALSE(grid.indexing);
|
||||
CHECK(grid.indexing == GridScanAnalysisSettings().IsIndexing());
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_GridIndexingIsASetting", "[AnalysisMode]") {
|
||||
// The one stage the mode does not fix. The configured value must reach the gates, which read
|
||||
// GetAnalysisStages and not the table.
|
||||
DiffractionExperiment experiment;
|
||||
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Grid));
|
||||
CHECK(experiment.GetAnalysisStages().indexing); // the default is on
|
||||
|
||||
experiment.ImportGridScanAnalysisSettings(GridScanAnalysisSettings().Indexing(false));
|
||||
CHECK_FALSE(experiment.GetAnalysisStages().indexing);
|
||||
|
||||
experiment.ImportGridScanAnalysisSettings(GridScanAnalysisSettings().Indexing(true));
|
||||
CHECK(experiment.GetAnalysisStages().indexing);
|
||||
|
||||
// ...and only under Grid. No other mode's indexing answer moves with it.
|
||||
experiment.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::Azint));
|
||||
experiment.ImportGridScanAnalysisSettings(GridScanAnalysisSettings().Indexing(true));
|
||||
CHECK_FALSE(experiment.GetAnalysisStages().indexing);
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_GridScanSettingsDefaults", "[AnalysisMode]") {
|
||||
const GridScanAnalysisSettings s;
|
||||
// A single cell is admitted only well above the threshold that admits a cell into a blob;
|
||||
// otherwise the decisive rule would just be a lower general threshold.
|
||||
CHECK(s.GetDecisiveSingleCellScore() > s.GetProteinScoreThreshold());
|
||||
CHECK(s.GetMinBlobCells() >= 1);
|
||||
CHECK(s.GetMaxCrystals() >= 1);
|
||||
|
||||
GridScanAnalysisSettings set;
|
||||
set.ProteinScoreThreshold(0.7f).MinBlobCells(5).DecisiveSingleCellScore(0.95f)
|
||||
.MaxCrystals(3).Indexing(false);
|
||||
CHECK(set.GetProteinScoreThreshold() == Catch::Approx(0.7));
|
||||
CHECK(set.GetMinBlobCells() == 5);
|
||||
CHECK(set.GetDecisiveSingleCellScore() == Catch::Approx(0.95));
|
||||
CHECK(set.GetMaxCrystals() == 3);
|
||||
CHECK_FALSE(set.IsIndexing());
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_PrecedenceOverSpotFindingSwitch", "[AnalysisMode]") {
|
||||
@@ -99,6 +137,25 @@ TEST_CASE("AnalysisMode_PowderCalibrationForcesCPUAzInt", "[AnalysisMode]") {
|
||||
AzimuthalIntegrationSettings().AzimuthalBinCount(64));
|
||||
explicit_bins.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
|
||||
CHECK(explicit_bins.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount() == 64);
|
||||
|
||||
// The coupling is applied from both imports, so the order the two are set in does not matter.
|
||||
DiffractionExperiment settings_first;
|
||||
settings_first.ImportCalibrationSettings(CalibrationSettings().Calibrant("lab6"));
|
||||
CHECK_FALSE(settings_first.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
|
||||
settings_first.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
|
||||
CHECK(settings_first.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
|
||||
|
||||
DiffractionExperiment mode_first;
|
||||
mode_first.ImportAnalysisSettings(AnalysisSettings().Mode(AnalysisMode::PowderCalibration));
|
||||
mode_first.ImportCalibrationSettings(CalibrationSettings().Calibrant("lab6"));
|
||||
CHECK(mode_first.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
|
||||
CHECK(mode_first.GetCalibrationSettings().GetCalibrant() == "lab6");
|
||||
|
||||
// A mode that is not a calibration is left alone.
|
||||
DiffractionExperiment stills;
|
||||
stills.ImportCalibrationSettings(CalibrationSettings().Method(CalibrationMethod::Spots));
|
||||
CHECK_FALSE(stills.GetAzimuthalIntegrationSettings().IsForceCPUinFPGAWorkflow());
|
||||
CHECK(stills.GetCalibrationSettings().GetMethod() == CalibrationMethod::Spots);
|
||||
}
|
||||
|
||||
TEST_CASE("AnalysisMode_CBORStartRoundTrip", "[AnalysisMode][CBOR]") {
|
||||
|
||||
Reference in new issue
Block a user