Files
Jungfraujoch/common/DetectorSetup.cpp
T
leonarski_fandjungfrau 4dc2534dbf
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
v1.0.0.rc-162 (#72)
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-25 08:21:39 +02:00

456 lines
16 KiB
C++

// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <filesystem>
#include <utility>
#include "DetectorSetup.h"
#include "JFJochException.h"
#include "NetworkAddressConvert.h"
#define check_max(param, val, max) if ((val) > (max)) throw JFJochException(JFJochExceptionCategory::InputParameterAboveMax, param)
#define check_min(param, val, min) if ((val) < (min)) throw JFJochException(JFJochExceptionCategory::InputParameterBelowMin, param)
DetectorSetup::DetectorSetup(const DetectorGeometryFixed &geom, DetectorType detector_type,
const std::string &description, const std::vector<std::string> &det_modules_hostname)
: DetectorSetup(std::make_shared<DetectorGeometryFixed>(geom),
detector_type, description, det_modules_hostname) {
switch (detector_type) {
case DetectorType::DECTRIS:
break;
default:
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Detector not compatible with fixed geometry");
}
}
DetectorSetup::DetectorSetup(const DetectorGeometryModular &geom, DetectorType detector_type,
const std::string &description, const std::vector<std::string> &det_modules_hostname)
: DetectorSetup(std::make_shared<DetectorGeometryModular>(geom),
detector_type, description, det_modules_hostname) {
switch (detector_type) {
case DetectorType::EIGER:
case DetectorType::JUNGFRAU:
break;
default:
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Detector not compatible with modular geometry");
}
}
DetectorSetup::DetectorSetup(std::shared_ptr<DetectorGeometry> in_geometry,
DetectorType in_detector_type,
const std::string &in_description,
const std::vector<std::string> &
in_det_modules_hostname)
: description(in_description),
geometry(std::move(in_geometry)),
det_modules_hostname(in_det_modules_hostname),
gain_calibration(std::make_shared<JFGainCalibration>()),
detector_type(in_detector_type),
read_out_time(0),
min_count_time(MIN_COUNT_TIME),
min_frame_time(std::chrono::milliseconds(1)) {
if (description.empty())
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Detector description cannot be empty");
switch (detector_type) {
case DetectorType::EIGER:
high_voltage = 150;
read_out_time = PSI_EIGER_READOUT_TIME;
if (!det_modules_hostname.empty() && (2 * geometry->GetModulesNum() != det_modules_hostname.size()))
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Mismatch between number of modules in detector geometry and hostname (For EIGER - one module = 2 hostnames)");
break;
case DetectorType::JUNGFRAU:
high_voltage = 120;
bit_depth_readout = 16;
read_out_time = PSI_JUNGFRAU_READOUT_TIME;
if (!det_modules_hostname.empty() && (geometry->GetModulesNum() != det_modules_hostname.size()))
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Mismatch between number of modules in detector geometry and hostname");
break;
case DetectorType::DECTRIS:
high_voltage = 0;
// bit_depth_readout stays unset: a DECTRIS detector's electronic readout depth is a
// constant we never need, and what is reported downstream is the image depth (see
// DiffractionExperiment::FillMessage).
bit_depth_image = 16; // placeholder, replaced from the stream2 image_dtype when armed
read_out_time = std::chrono::microseconds(0);
if (!det_modules_hostname.empty() && ( det_modules_hostname.size() != 1))
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Just one address need to be provided for DECTRIS detector");
break;
default:
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Detector not supported");
}
}
DetectorSetup &DetectorSetup::Description(const std::string &input) {
if (input.empty())
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Detector description cannot be empty");
description = input;
return *this;
}
const DetectorGeometry &DetectorSetup::GetGeometry() const {
return *geometry;
}
const std::vector<std::string> &DetectorSetup::GetDetectorModuleHostname() const {
return det_modules_hostname;
}
uint64_t DetectorSetup::GetModulesNum() const {
return geometry->GetModulesNum();
}
std::string DetectorSetup::GetDescription() const {
return description;
}
float DetectorSetup::GetPixelSize_mm() const {
return pixel_size_um / 1000.0f;
}
std::string DetectorSetup::GetSensorMaterial() const {
return sensor_material;
}
float DetectorSetup::GetSensorThickness_um() const {
return sensor_thickness_um;
}
void DetectorSetup::LoadGain(const std::vector<std::string> &filenames) {
if (filenames.size() != GetModulesNum())
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Mismatch in number of gain calibration files");
gain_file_names = filenames;
gain_calibration->LoadGain(filenames);
}
DetectorSetup &DetectorSetup::UDPInterfaceCount(int64_t input) {
if ((input != 1) && (input != 2))
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Only 1 and 2 are supported as UDP interface count");
udp_interface_count = input;
return *this;
}
const std::vector<JFModuleGainCalibration> &DetectorSetup::GetGainCalibration() const {
return gain_calibration->GetCalibration();
}
int64_t DetectorSetup::GetUDPInterfaceCount() const {
if (detector_type == DetectorType::EIGER)
return 2;
return udp_interface_count;
}
DetectorSetup &DetectorSetup::SensorMaterial(const std::string &input) {
sensor_material = input;
return *this;
}
DetectorSetup &DetectorSetup::SensorThickness_um(float input) {
sensor_thickness_um = input;
return *this;
}
DetectorSetup &DetectorSetup::PixelSize_um(float input) {
pixel_size_um = input;
return *this;
}
DetectorSetup &DetectorSetup::Geometry(const DetectorGeometryFixed &input) {
if (detector_type != DetectorType::DECTRIS)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"PSI detector geometry cannot be updated during operation");
geometry = std::make_shared<DetectorGeometryFixed>(input);
return *this;
}
DetectorSetup &DetectorSetup::TxDelay(const std::vector<int64_t> &v) {
if (!v.empty() && (v.size() != GetModulesNum()))
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Mismatch between size of TX delay vector and modules number");
for (const auto &i: v) {
if ((i < 0) || (i > 31))
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "TX delay must be in range 0-31");
}
tx_delay = v;
return *this;
}
const std::vector<int64_t> &DetectorSetup::GetTxDelay() const {
return tx_delay;
}
const std::vector<std::string> &DetectorSetup::GetGainFileNames() const {
return gain_file_names;
}
DetectorType DetectorSetup::GetDetectorType() const {
return detector_type;
}
DetectorSetup &DetectorSetup::HighVoltage(int32_t input) {
high_voltage = input;
return *this;
}
int32_t DetectorSetup::GetHighVoltage() const {
return high_voltage;
}
void DetectorSetup::SetTrimFiles(const std::vector<std::string> &filenames) {
if (detector_type != DetectorType::EIGER)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Trim bits make sense only for EIGER");
if ((filenames.size() == 1)
&& std::filesystem::is_directory(filenames[0])) {
trim_file_directory = filenames[0];
trim_file_names.clear();
} else {
if (filenames.size() != 2 * GetModulesNum())
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Mismatch in number of trim bit calibration files");
trim_file_directory = "";
trim_file_names = filenames;
}
}
const std::vector<std::string> &DetectorSetup::GetTrimFileNames() const {
return trim_file_names;
}
std::string DetectorSetup::GetTrimFileDirectory() const {
return trim_file_directory;
}
std::string DetectorSetup::GetSerialNumber() const {
return serial_number;
}
DetectorSetup &DetectorSetup::SerialNumber(const std::string &input) {
serial_number = input;
return *this;
}
DetectorSetup &DetectorSetup::BaseIPv4Addr(const std::string &input) {
ipv4_base_addr = IPv4AddressFromStr(input);
return *this;
}
uint32_t DetectorSetup::GetSrcIPv4Addr(uint32_t half_module) const {
if (half_module >= GetUDPInterfaceCount() * GetModulesNum())
throw JFJochException(JFJochExceptionCategory::ArrayOutOfBounds, "Non existing module");
return ipv4_base_addr + (half_module << 24);
}
std::string DetectorSetup::GetBaseIPv4Addr() const {
return IPv4AddressToStr(ipv4_base_addr);
}
bool DetectorSetup::IsModuleSync() const {
if (GetModulesNum() == 1)
return false;
else
return module_sync;
}
DetectorSetup &DetectorSetup::ModuleSync(bool input) {
module_sync = input;
return *this;
}
DetectorSetup &DetectorSetup::MirrorY(bool input) {
mirror_y = input;
return *this;
}
bool DetectorSetup::IsMirrorY() const {
return mirror_y;
}
DetectorSetup & DetectorSetup::ReadOutTime(std::chrono::nanoseconds input) {
if (input.count() < 0)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Read out time has to be non-negative");
read_out_time = input;
return *this;
}
std::chrono::nanoseconds DetectorSetup::GetReadOutTime() const {
return read_out_time;
}
std::chrono::nanoseconds DetectorSetup::GetMinFrameTime() const {
switch (GetDetectorType()) {
case DetectorType::EIGER:
return MIN_FRAME_TIME_EIGER;
case DetectorType::JUNGFRAU:
if (GetUDPInterfaceCount() == 1)
return MIN_FRAME_TIME_JUNGFRAU_HALF_SPEED;
return MIN_FRAME_TIME_JUNGFRAU_FULL_SPEED;
case DetectorType::DECTRIS:
return min_frame_time;
default:
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Detector not supported");
}
}
std::chrono::nanoseconds DetectorSetup::GetMinCountTime() const {
return min_count_time;
}
DetectorSetup &DetectorSetup::MinCountTime(std::chrono::nanoseconds input) {
min_count_time = input;
return *this;
}
DetectorSetup &DetectorSetup::MinFrameTime(std::chrono::nanoseconds input) {
min_frame_time = input;
return *this;
}
DetectorSetup &DetectorSetup::BitDepthImage(int64_t input) {
if (GetDetectorType() != DetectorType::DECTRIS)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Bit depth image can be only changed for DECTRIS detector");
switch (input) {
case 8:
case 16:
case 32:
bit_depth_image = input;
return *this;
default:
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Bit depth image can be only 8, 16 or 32");
}
}
std::optional<int64_t> DetectorSetup::GetBitDepthReadout() const {
return bit_depth_readout;
}
std::optional<int64_t> DetectorSetup::GetBitDepthImage() const {
return bit_depth_image;
}
std::string DetectorSetup::GetDECTRISStream2Addr() const {
if (GetDetectorType() != DetectorType::DECTRIS)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Stream2 only possible for DECTRIS systems");
if (det_modules_hostname.size() != 1)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Stream2 address not defined");
return "tcp://" + det_modules_hostname[0] + ":" + std::to_string(SimplonStream2Port);
}
float DetectorSetup::GetMinThreshold_keV() const {
return min_energy_threshold_keV;
}
DetectorSetup &DetectorSetup::MinThreshold_keV(float input) {
if (input <= 0.0f)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Min threshold must be positive number");
min_energy_threshold_keV = input;
return *this;
}
DetectorSetup &DetectorSetup::SaturationLimit(std::optional<int64_t> input) {
if (input && input <= 0)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Saturation limit must be positive number");
saturation_limit = input;
return *this;
}
std::optional<int64_t> DetectorSetup::GetSaturationLimit() const {
return saturation_limit;
}
DetectorSetup &DetectorSetup::DECTRISROI(const std::string &input) {
dectris_roi = input;
return *this;
}
std::string DetectorSetup::GetDECTRISROI() const {
if (dectris_roi.empty())
return "disabled";
return dectris_roi;
}
std::optional<DetectorSettings> DetectorSetup::GetDefaultSettings() const {
return settings;
}
DetectorSetup &DetectorSetup::DefaultSettings(const std::optional<DetectorSettings> &input) {
settings = input;
return *this;
}
DetectorSetup DetJF4M(const std::string &description, const std::vector<std::string> &det_modules_hostname) {
return DetJF(8, 2, 8, 36, true, description, det_modules_hostname);
}
DetectorSetup DetJF9M(const std::string &description, const std::vector<std::string> &det_modules_hostname) {
return DetJF(18, 3, 8, 36, true, description, det_modules_hostname);
}
DetectorSetup DetJF(int32_t nmodules, int32_t horizontal_stacking, int32_t gap_x, int32_t gap_y, bool mirror_y,
const std::string &description, const std::vector<std::string> &det_modules_hostname) {
return DetJF(DetectorGeometryModular(nmodules, horizontal_stacking, gap_x, gap_y, mirror_y),
description, det_modules_hostname);
}
DetectorSetup DetJF(const DetectorGeometryModular &geom, const std::string &description,
const std::vector<std::string> &det_modules_hostname) {
return {geom, DetectorType::JUNGFRAU, description, det_modules_hostname};
}
DetectorSetup DetEIGER(int32_t nmodules, int32_t horizontal_stacking, int32_t gap_x, int32_t gap_y, bool mirror_y,
const std::string &description, const std::vector<std::string> &det_modules_hostname) {
return DetEIGER(DetectorGeometryModular(nmodules, horizontal_stacking, gap_x, gap_y, mirror_y),
description, det_modules_hostname);
}
DetectorSetup DetEIGER(const DetectorGeometryModular &geom, const std::string &description,
const std::vector<std::string> &det_modules_hostname) {
return {geom, DetectorType::EIGER, description, det_modules_hostname};
}
DetectorSetup DetDECTRIS(int64_t width, int64_t height, const std::string &description, const std::string &addr) {
if (addr.empty())
return {DetectorGeometryFixed(width, height), DetectorType::DECTRIS, description, {}};
else
return {DetectorGeometryFixed(width, height), DetectorType::DECTRIS, description, {addr}};
}
int32_t DetectorSetup::GetTempThreshold_degC() const {
return temperature_thresold_degC;
}
DetectorSetup &DetectorSetup::TempThreshold_degC(int64_t input) {
check_min("Temperature threshold (degC)", input, 40);
check_max("Temperature threshold (degC)", input, 70);
temperature_thresold_degC = static_cast<int32_t>(input);
return *this;
}
DetectorSetup &DetectorSetup::TrimEnergies_eV(std::vector<int> input) {
trim_energy_eV_values = std::move(input);
return *this;
}
std::vector<int> DetectorSetup::GetTrimEnergies_eV() const {
return trim_energy_eV_values;
}