Files
Jungfraujoch/common/DatasetSettings.h
T
leonarski_fandjungfrau 4dc2534dbf
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
v1.0.0.rc-162 (#72)
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-25 08:21:39 +02:00

173 lines
6.7 KiB
C++

// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <cstdint>
#include <string>
#include <optional>
#include "UnitCell.h"
#include "Coord.h"
#include "../compression/CompressionAlgorithmEnum.h"
#include "JFJochMessages.h"
#include "XrayFluorescenceSpectrum.h"
class DatasetSettings {
int64_t images_per_trigger;
int64_t ntrigger;
float beam_x_pxl;
float beam_y_pxl;
float detector_distance_mm;
float photon_energy_keV;
std::string file_prefix;
std::optional<int64_t> images_per_file;
CompressionAlgorithm compression;
std::string sample_name;
std::optional<UnitCell> unit_cell;
std::optional<int64_t> space_group_number;
std::optional<float> total_flux;
std::optional<float> attenuator_transmission;
std::optional<float> ring_current_mA;
std::optional<GoniometerAxis> goniometer;
std::optional<GridScanSettings> grid_scan;
std::optional<float> sample_temperature_K;
nlohmann::json image_appendix;
nlohmann::json header_appendix;
float data_reduction_factor_serialmx;
std::optional<std::chrono::nanoseconds> image_time;
std::optional<uint64_t> run_number;
std::optional<std::string> run_name;
std::string group;
std::optional<int64_t> compression_poisson_factor;
std::optional<int64_t> pixel_value_low_threshold;
std::optional<int64_t> pixel_value_high_threshold;
std::optional<bool> save_calibration;
bool write_nxmx_hdf5_master;
std::optional<float> polarization_factor;
std::optional<float> bandwidth_fwhm; // relative X-ray bandwidth, FWHM of dlambda/lambda (e.g. 0.01 for 1%)
float poni_rot_1_rad;
float poni_rot_2_rad;
float poni_rot_3_rad;
bool spot_finding_enable;
int64_t max_spot_count;
bool detect_ice_rings; // Enable special handling of ice rings
XrayFluorescenceSpectrum fluorescence_spectrum;
std::optional<SmargonPosition> smargon_position;
public:
DatasetSettings();
DatasetSettings& ImagesPerTrigger(int64_t input);
DatasetSettings& NumTriggers(int64_t triggers);
DatasetSettings& PhotonEnergy_keV(float input);
DatasetSettings& BeamX_pxl(float input);
DatasetSettings& BeamY_pxl(float input);
DatasetSettings& DetectorDistance_mm(float input);
DatasetSettings& FilePrefix(std::string input);
DatasetSettings& FilePrefixTrusted(std::string input); // offline/local: no CheckPath guard (absolute allowed)
DatasetSettings& Compression(CompressionAlgorithm input);
DatasetSettings& SetUnitCell(const std::optional<UnitCell> &cell);
DatasetSettings& SpaceGroupNumber(std::optional<int64_t> input);
DatasetSettings& SampleName(std::string input);
DatasetSettings& AttenuatorTransmission(const std::optional<float> &input);
DatasetSettings& TotalFlux(const std::optional<float> &input);
DatasetSettings& Goniometer(const std::optional<GoniometerAxis>& input);
DatasetSettings& GridScan(const std::optional<GridScanSettings>& input);
DatasetSettings& HeaderAppendix(const nlohmann::json& input);
DatasetSettings& ImageAppendix(const nlohmann::json& input);
DatasetSettings& ImagesPerFile(const std::optional<int64_t> &input);
DatasetSettings& RunNumber(const std::optional<uint64_t> &run_number);
DatasetSettings& RunName(const std::optional<std::string> &input);
DatasetSettings& ExperimentGroup(const std::string &group);
DatasetSettings& ImageTime(const std::optional<std::chrono::nanoseconds> &input);
DatasetSettings& LossyCompressionSerialMX(float input);
DatasetSettings& LossyCompressionPoisson(std::optional<int64_t> input);
DatasetSettings& WriteNXmxHDF5Master(bool input);
DatasetSettings& SaveCalibration(std::optional<bool> input);
DatasetSettings& PixelValueLowThreshold(const std::optional<int64_t> &input);
DatasetSettings& PixelValueHighThreshold(const std::optional<int64_t> &input);
DatasetSettings& PolarizationFactor(const std::optional<float> &input);
DatasetSettings& BandwidthFWHM(const std::optional<float> &input);
DatasetSettings& PoniRot1_rad(float input);
DatasetSettings& PoniRot2_rad(float input);
DatasetSettings& PoniRot3_rad(float input);
DatasetSettings& RingCurrent_mA(const std::optional<float> &input);
DatasetSettings& SampleTemperature_K(const std::optional<float> &input);
DatasetSettings& SpotFindingEnable(bool input);
DatasetSettings& MaxSpotCount(int64_t input);
DatasetSettings& DetectIceRings(bool input);
DatasetSettings& FluorescenceSpectrum(const XrayFluorescenceSpectrum& input);
DatasetSettings& Smargon(const std::optional<SmargonPosition>& input);
std::optional<float> GetAttenuatorTransmission() const;
std::optional<float> GetTotalFlux() const;
std::optional<GoniometerAxis> &Goniometer();
std::optional<GridScanSettings> &GridScan();
const std::optional<GoniometerAxis> &GetGoniometer() const;
const std::optional<GridScanSettings> &GetGridScan() const;
const nlohmann::json& GetHeaderAppendix() const;
const nlohmann::json& GetImageAppendix() const;
std::optional<UnitCell> GetUnitCell() const;
std::optional<int64_t> GetSpaceGroupNumber() const;
std::string GetSampleName() const;
float GetPhotonEnergy_keV() const;
float GetBeamX_pxl() const;
float GetBeamY_pxl() const;
float GetDetectorDistance_mm() const;
Coord GetScatteringVector() const;
std::string GetFilePrefix() const;
CompressionAlgorithm GetCompressionAlgorithm() const;
int64_t GetNumTriggers() const;
int64_t GetImageNumPerTrigger() const;
std::optional<int64_t> GetImagesPerFile() const;
std::optional<uint64_t> GetRunNumber() const;
std::optional<std::string> GetRunName() const;
std::string GetExperimentGroup() const;
std::optional<std::chrono::nanoseconds> GetImageTime() const;
float GetLossyCompressionSerialMX() const;
std::optional<int64_t> GetLossyCompressionPoisson() const;
std::optional<int64_t> GetPixelValueLowThreshold() const;
std::optional<int64_t> GetPixelValueHighThreshold() const;
bool IsWriteNXmxHDF5Master() const;
std::optional<bool> IsSaveCalibration() const;
std::optional<float> GetPolarizationFactor() const;
std::optional<float> GetBandwidthFWHM() const;
float GetPoniRot1_rad() const;
float GetPoniRot2_rad() const;
float GetPoniRot3_rad() const;
std::optional<float> GetRingCurrent_mA() const;
std::optional<float> GetSampleTemperature_K() const;
bool IsSpotFindingEnabled() const;
int64_t GetMaxSpotCount() const;
bool IsDetectIceRings() const;
const XrayFluorescenceSpectrum &GetFluorescenceSpectrum() const;
std::optional<SmargonPosition> GetSmargonPosition() const;
};