Files
Jungfraujoch/docs/python_client/docs/DatasetSettings.md
T
leonarski_fandjungfrau 4dc2534dbf
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
v1.0.0.rc-162 (#72)
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-25 08:21:39 +02:00

7.9 KiB

DatasetSettings

Properties

Name Type Description Notes
images_per_trigger int For standard synchrotron data collection - this is number of images collected per one TTL trigger For XFEL (pulsed source) - this number is ignored and set to 1 For storage cell mode - this number is ignored and set to number of storage cells [optional] [default to 1]
ntrigger int Number of TTL trigger that the detector is expected to receive during data collection [optional] [default to 1]
image_time_us int Image time. If not provided (or zero value) the frame time is assumed as default. For JUNGFRAU image time must be multiple of frame time and max value is 256 * frame_time. In XFEL mode: summation happens for frames collected with multiple triggers. Ignored for storage cells and if raw data are saved. [optional]
beam_x_pxl float /entry/detector/beam_center_x in NXmx Beam center in X direction [pixels]
beam_y_pxl float /entry/detector/beam_center_y in NXmx Beam center in X direction [pixels]
detector_distance_mm float /entry/detector/distance in NXmx Detector distance [mm]
incident_energy_ke_v float Used to calculate /entry/beam/incident_wavelength in NXmx Incident particle (photon, electron) energy in keV
file_prefix str Prefix for filenames. If left empty, no file will be saved. [optional] [default to '']
images_per_file int Number of images in a single HDF5 data file, taken literally when given. If omitted, it is chosen from the acquisition: a rotation sweep of at most 20000 images goes into a single data file, so the dataset stays readable by programs that follow only the first data file of a legacy master; a grid scan is split on a whole number of fast-axis rows; anything else (stills, serial) uses 1000. [optional]
space_group_number int Number of space group for the crystal. Currently used solely as metadata, not relevant for image processing done in Jungfraujoch. [optional]
sample_name str /entry/sample/name in NXmx Sample name [optional] [default to '']
compression str Compression type for the images transferred over ZeroMQ and saved to HDF5 file. [optional] [default to 'bslz4']
total_flux float /entry/beam/total_flux in NXmx Flux incident on beam plane in photons per second. In other words this is the flux integrated over area. [photons/s] [optional]
transmission float /entry/instrument/attenuator/attenuator_transmission Transmission of attenuator (filter) [no units] [optional]
goniometer RotationAxis [optional]
grid_scan GridScan [optional]
header_appendix object Header appendix, added as user_data/user to start ZeroMQ message (can be any valid JSON) In general, it is not saved in HDF5 file. However, if values are placed in &quot;hdf5&quot; object, `jfjoch_writer` will write them in /entry/user of the HDF5 file. This applies solely to string and number (double floating-point). No arrays/sub-objects is allowed. For example {&quot;hdf5&quot;: {&quot;val1&quot;:1, &quot;val2&quot;:&quot;xyz&quot;}}, will write /entry/user/val1 and /entry/user/val2. [optional]
image_appendix object Image appendix, added as user_data to image ZeroMQ message (can be any valid JSON) Not saved in HDF5 file [optional]
data_reduction_factor_serialmx float Rate at which non-indexed images are accepted to be forwarded to writer. Value of 1.0 (default) means that all images are written. Values below zero mean that non-indexed images will be accepted with a given probability. [optional] [default to 1.0]
pixel_value_low_threshold int Set all counts lower than the value to zero. When the value is set, negative numbers other than error pixel value are always set to zero. Setting to zero is equivalent to turning the option off. [optional]
run_number int Number of run within an experimental session. Transferred over CBOR stream as &quot;series ID&quot;, though not saved in HDF5 file. It is highly recommended to keep this number unique for each data collection during experimental series. If not provided, the number will be automatically incremented. [optional]
run_name str Unique ID of run. Transferred over CBOR stream as &quot;unique series ID&quot;, though not saved in HDF5 file. It is highly recommended to keep this name unique for each data collection during experimental series. If not provided, the name will be automatically generated as number + colon + file_prefix. [optional]
experiment_group str Name of group owning the data (e.g. p-group or proposal number). Transferred over CBOR stream, though not saved in HDF5 file. [optional]
poisson_compression int Enable lossy compression of pixel values that preserves Poisson statistics. Requires to provide a numerical factor SQ. Pixel value P will be transformed to round(sqrt(P) * SQ), with rounding to the closest integer. Compression is turned off if the value is missing or it is set to zero. [optional]
write_nxmx_hdf5_master bool Write NXmx formatted HDF5 master file. Recommended to use for macromolecular crystallography experiments and to turn off for other experiments. [optional] [default to True]
save_calibration bool Forward image calibration (at the moment pedestal and pedestal RMS for JUNGFRAU) using the ZeroMQ stream to writer. If parameter is not provided calibration will be saved only if more than 4 images are recorded. [optional]
polarization_factor float Polarization factor for integration; 1.0 is horizontal polarization; -1.0 is vertical polarization [optional]
ring_current_m_a float Ring current at the beginning of the data collection [optional]
sample_temperature_k float Sample temperature in Kelvin [optional]
poni_rot1_rad float PONI angle rot1 (see PyFAI documentation for details) in radians [optional] [default to 0.0]
poni_rot2_rad float PONI angle rot2 (see PyFAI documentation for details) in radians [optional] [default to 0.0]
poni_rot3_rad float PONI angle rot3 (see PyFAI documentation for details) in radians [optional] [default to 0.0]
unit_cell UnitCell [optional]
spot_finding bool Enable spot finding and save spots [optional] [default to True]
smargon DatasetSettingsSmargon [optional]
max_spot_count int Maximum number of spots that are saved/used for indexing; spots with highest intensity are selected [optional] [default to 250]
detect_ice_rings bool Flag spots as ice rings and reduce their effect on indexing [optional]
async_start bool When set to true, `/start` will not wait for detector and Jungfraujoch to be ready for the measurement. [optional] [default to False]
xray_fluorescence_spectrum DatasetSettingsXrayFluorescenceSpectrum [optional]

Example

from jfjoch_client.models.dataset_settings import DatasetSettings

# TODO update the JSON string below
json = "{}"
# create an instance of DatasetSettings from a JSON string
dataset_settings_instance = DatasetSettings.from_json(json)
# print the JSON string representation of the object
print(DatasetSettings.to_json())

# convert the object into a dict
dataset_settings_dict = dataset_settings_instance.to_dict()
# create an instance of DatasetSettings from a dict
dataset_settings_from_dict = DatasetSettings.from_dict(dataset_settings_dict)

[Back to Model list] [Back to API list] [Back to README]