The per-image resolution estimate was the 5th percentile of the spot d spacings - an extreme order statistic, so it measured where detection stops rather than how well the crystal diffracts. A large cell puts more reflections past the same threshold and scored better than a small cell that diffracts further; intensity was not used at all, so a weak crystal padded with spurious high-resolution detections ran away; and nothing clamped the answer to what the detector can deliver. Against the resolution the merged data actually reach it was 42% out in log-RMS, with 1 of 38 rotation datasets inside 0.2 A. Take instead the 1/d^2 beyond which 30% of the sum of sqrt(I) over the image's non-ice spots lies, report 1/(2.25 sqrt of it), clamp at the detector corner, and take the median over images. A quantile from the middle of the distribution measures the shape of the falloff - the crystal's own exp(-B/2d^2) - where an extreme one measures the threshold. sqrt(I) is the Poisson significance of a summed photon count, so a marginal high-resolution detection cannot carry the answer and neither can a handful of very strong low-resolution reflections. The 2.25 is the multiplicity gain: merging keeps measuring intensities a fixed factor in 1/d past the point where a single frame detects them. Spearman 0.881 -> 0.954, log-RMS 42% -> 8.9%, median error 0.79 -> 0.07 A, and 32 of 38 within 0.2 A. Both constants sit on a broad plateau, the scale is stable across dataset halves and across resolution ranges, and no second predictor survives leave-one-out. The residual is around 9%, set by multiplicity, symmetry and radiation damage - none of which a spot list can see. The estimate feeds only reporting: the image stream, HDF5, the plots, the scan result and the preview ring. It sets no cutoff and no search limit, and the scaling and merging output is byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NNnL26LAvruQ9eLUUWvrJ
72 KiB
CBOR messages
To communicate between FPGA-equipped receiver system and writers, Jungfraujoch is using binary CBOR encoding with tinycbor library (Intel). The protocol is based on and compatible with DECTRIS Stream2. There are minor differences at the moment:
- LZ4 alone is not allowed; Bitshuffle+LZ4 and Bitshuffle+Zstandard are allowed
- Few fields are currently absent
- Extra fields are present beyond DECTRIS standard
- There are calibration and metadata messages defined beyond DECTRIS specification
Start message
| Field name | Type | Description | Present in DECTRIS format |
|---|---|---|---|
| type | String | value "start" | X |
| magic_number | uint64 | Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver | |
| detector_distance | float | Detector distance [m] | |
| detector_translation | Array(float) | Detector translation vector [m] | X |
| beam_center_x | float | Beam center in X direction [pixels] | X |
| beam_center_y | float | Beam center in Y direction [pixels] | X |
| countrate_correction_enabled | bool | Countrate correction enabled | X |
| flatfield_enabled | bool | Flatfield enabled | X |
| number_of_images | uint64 | Number of images in the series | X |
| image_size_x | uint64 | Image width [pixels] | X |
| image_size_y | uint64 | Image height [pixels] | X |
| mirror_y | bool | Whether the assembled image is mirrored in Y relative to the detector's raw readout order. True is the MX convention - row 0 at the top of the detector seen from the sample - and is what absence of the key means | |
| incident_energy | float | X-ray energy [eV] | X |
| incident_wavelength | float | X-ray wavelength [Angstrom] | X |
| incident_wavelength_spread | float (optional) | FWHM of the X-ray wavelength distribution [Angstrom] (NXmx incident_wavelength_spread); omitted when the beam is monochromatic | |
| frame_time | float | Frame time, if multiple frames per trigger [s] | X |
| count_time | float | Exposure time [s] | X |
| saturation_value | int64 | Maximum valid sample value | X |
| error_value | int64 (optional) | Value used in images to describe pixels that are in error state or missing | |
| pixel_size_x | float | Pixel width [m] | X |
| pixel_size_y | float | Pixel height [m] | X |
| sensor_thickness | float | Sensor thickness [m] | X |
| sensor_material | string | Sensor material | X |
| arm_date | date | Approximate date of arming | X |
| pixel_mask_enabled | bool | Pixel mask applied on images | X |
| detector_description | string | Name of the detector | X |
| detector_serial_number | string | Detector serial number | X |
| series_unique_id | string | Unique text ID of the series (run_name parameter) | X |
| series_id | uint64 | Unique numeric ID of the series (run_number parameter) | X |
| fluorescence | object (optional) | X-ray fluorescence spectrum collected at start | |
| - energy | Array(float) | Energy of measuring point [eV] | |
| - data | Array(float) | Fluorescence scan result data [arbitrary units]; must be strictly the same length as energy |
|
| goniometer | Map | Definition of rotation axis (optional) | X |
- AXIS |
string | Rotation axis name (e.g. omega) - only one axis is supported in Jungfraujoch | X |
| - - increment | float | Rotation axis increment (per image) in degree [deg] | X |
| - - start | float | Rotation axis start angle [deg] | X |
| - - axis | Array(float) | Vector for the rotation axis | |
| - - helical_step | Array(float) | Translation for helical scan for 1 image [m] | |
| - - screening_wedge | Array(float) | Wedge for screening [deg] (increment would correspond to difference between screening points) | |
| grid_scan | object | Grid scan definition (optional). May be sent together with goniometer: a grid is often collected at a given head position, recorded as a stationary axis (step 0) |
|
| - n_fast | uint64 | Number of elements along fast axis | |
| - n_slow | uint64 | Number of elements along slow axis | |
| - step_x_axis | float | Step along X axis, can be negative [m] | |
| - step_y_axis | float | Step along Y axis, can be negative [m] | |
| - snake_scan | bool | Snake scan (rows alternate direction) | |
| - vertical_scan | bool | Vertical scan (enabled: fast direction = Y, disabled: fast direction = X) | |
| jungfrau_conversion_enabled | bool (optional) | Applying JUNGFRAU pixel conversion (to photons or keV) | |
| jungfrau_conversion_factor | float (optional) | Factor used for JUNGFRAU conversion [eV] | |
| geometry_transformation_enabled | bool (optional) | Transformation from detector module geometry (512x1024) to full detector geometry | |
| pixel_mask | Map(string -> Image) | Pixel mask - multiple in case of storage cells | X |
| channels | Array(string) | List of image channels | X |
| max_spot_count | uint64 | Maximum number of spots identified in spot finding | |
| max_extra_lattices | uint64 | Maximum number of extra lattices | |
| storage_cell_number | uint64 (optional) | Number of storage cells used by JUNGFRAU | |
| storage_cell_delay | Rational | Delay of storage cells in JUNGFRAU | |
| threshold_energy | Map(string -> float) | Per-channel threshold energy [eV] (map of channel name to value) | |
| image_dtype | string | Pixel type of the image data: uint8, uint16, uint32 (DECTRIS), plus int8, int16, int32 as a Jungfraujoch extension. Sole wire encoding of both the bit depth and the sign, and must agree with the per-image typed-array tag |
X |
| unit_cell | object (optional) | Unit cell of the system: a, b, c [angstrom] and alpha, beta, gamma [degree] | |
| az_int_q_bin_count | uint64 | Number of azimuthal integration bins in the radial direction | |
| az_int_phi_bin_count | uint64 | Number of azimuthal integration bins in the phi angle direction | |
| az_int_bin_to_q | Array(float) | Q value for each azimuthal integration bin [angstrom^-1] | |
| az_int_bin_to_two_theta | Array(float) | Two theta angle value for each azimuthal integration bin [deg] | |
| az_int_bin_to_phi | Array(float) | Phi value for each azimuthal integration bin [deg] | |
| az_int_map | Image | Mapping between pixel and bin number | |
| summation | uint64 | Factor of frame summation | |
| user_data | string | JSON serialized to string that can contain the following fields (all fields are optional): | X |
| - file_prefix | string | File prefix | |
| - images_per_file | uint64 | Number of images written per file | |
| - images_per_trigger | uint64 | Number of images collected per trigger | |
| - source_name | string | Facility name | |
| - source_type | string | Type of X-ray source (use NXsource/type values, for example "Synchrotron X-ray Source" or "Free-Electron Laser") | |
| - instrument_name | string | Instrument name | |
| - sample_name | string | Name of the sample | |
| - user | any valid JSON | Value of header_appendix provided at collection start to Jungfraujoch | |
| - attenuator_transmission | float | Attenuator transmission [] | |
| - total_flux | float | Total flux [ph/s] | |
| - space_group_number | uint64 | Space group number | |
| - summation_mode | string | Summation mode (internal|fpga|cpu) | |
| - overwrite | bool | Overwrite existing HDF5 files | |
| - file_format | int | File writer format: 0 = only data files, 1 = NXmx legacy soft links, 2 = NXmx VDS, 3 = NXmx integrated, 4 = CBF, 5 = TIFF, 6 = no file written | |
| - roi | Array(object) | ROI configurations; each element is one of: | |
| type "box": xmin, xmax, ymin, ymax (numbers) | |||
| type "circle": r, x, y (numbers) | |||
| type "azim": qmin, qmax (numbers); optional phi_min, phi_max (numbers, deg) for an angular sector | |||
| - gain_file_names | Array(string) | Names of JUNGFRAU gain files used for the current detector | |
| - write_master_file | bool | With multiple sockets, it selects which socket will provide master file | |
| - write_images | bool | Write images in the HDF5 file (if false, will only write metadata) | |
| - data_reduction_factor_serialmx | uint64 | Data reduction factor for serial MX | |
| - experiment_group | string | ID of instrument user, e.g., p-group (SLS/SwissFEL) or proposal number | |
| - jfjoch_release | string | Jungfraujoch release number | |
| - socket_number | uint64 | Number of ZeroMQ socket (on jfjoch_broker side) used for transmission |
|
| - bit_depth_readout | uint64 | Bit depth of the stored image (see note below), copied to NXmx bit_depth_readout |
|
| - underload_value | int64 | Lowest valid value; copied to NXmx underload_value. 0 for an unsigned image, INTx_MIN + 1 for a signed one |
|
| - writer_notification_zmq_addr | string | ZeroMQ address to inform jfjoch_broker about writers that finished operation |
|
| - xfel_pulse_id | uint64 | Pulse IDs are recorded for images | |
| - ring_current_mA | float | Ring current at the start of the measurement | |
| - sample_temperature_K | float | Sample temperature [K] | |
| - detect_ice_rings | bool | Ice ring detection feature is enabled | |
| - indexing_algorithm | string | Indexing algorithm used on-the-fly; allowed values: ffbidx, fft, fftw, none | |
| - geom_refinement_algorithm | string | Post-indexing detector geometry refinement algorithm; allowed values: none, beam_center | |
| - poni_rot1 | float | Tilt of the detector rot1 according to PyFAI PONI convention [rad] | |
| - poni_rot2 | float | Tilt of the detector rot2 according to PyFAI PONI convention [rad] | |
| - poni_rot3 | float | Tilt of the detector rot3 according to PyFAI PONI convention [rad] |
See DECTRIS documentation for definition of Image as MultiDimArray with optional compression.
Image message
| Field name | Type | Description | Present in DECTRIS format | Optional |
|---|---|---|---|---|
| type | String | value "image" | X | |
| magic_number | uint64 | Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver | ||
| series_unique_id | string | Unique text ID of the series (run_name parameter) | X | |
| series_id | uint64 | Unique numeric ID of the series (run_number parameter) | X | |
| image_id | uint64 | Number of image within the series; for MX lossy compression this is sequential excluding removed frames | X | |
| original_image_id | uint64 | Number of image within the series; for MX lossy compression this includes removed frames in the count | ||
| real_time | Rational | Exposure time | X | |
| start_time | Rational | Exposure start time (highly approximate) | X | |
| end_time | Rational | Exposure end time (highly approximate) | X | |
| spots | Array(object) | Spots: | ||
| - x | float | observed position in x (pixels) | ||
| - y | float | observed position in y (pixels) | ||
| - I | float | intensity (photons) | ||
| - maxc | int64 | max count (photons) | ||
| - ice_ring | bool | spot in resolution range for ice rings | ||
| - indexed | bool | indexed solution | ||
| - latt | int64 | Lattice to which the peak belongs (negative number = not indexed) | ||
| - image | int64 | image number the spot belongs to | ||
| - h | int64 | Miller index (indexed spots only) | ||
| - k | int64 | Miller index (indexed spots only) | ||
| - l | int64 | Miller index (indexed spots only) | ||
| - dist_ewald | float | distance to Ewald sphere [Angstrom^-1] (indexed spots only) | ||
| reflections | Array(object) | Reflections: | ||
| - h | int64 | Miller index | ||
| - k | int64 | Miller index | ||
| - l | int64 | Miller index | ||
| - x | float | prediced position in x (pixels) | ||
| - y | float | predicted position in y (pixels) | ||
| - obs_x | float | observed position in x (pixels) | ||
| - obs_y | float | observed position in y (pixels) | ||
| - d | float | resolution [Angstrom] | ||
| - I | float | integrated intensity (photons) | ||
| - bkg | float | mean background value (photons) | ||
| - var_bkg | float | non-signal (background) part of sigma^2, carried to the merge (photons^2) | ||
| - sigma | float | standard deviation, estimated from counting statistics (photons) | ||
| - image | float | image number (present for each spot) | ||
| - rp | float | Distance to Ewald sphere [Angstrom^-1] | ||
| - rlp | float | Reciprocal Lorentz and polarization corrections | ||
| - partiality | float | Partiality of the reflection | ||
| - phi | float | phi angle from XDS: difference from middle of current frame, not absolute [deg] | ||
| - zeta | float | Lorentz zeta factor (reciprocal-space geometry term) | ||
| - image_scale_corr | float | Per-image scale correction; I_true = image_scale_corr * I | ||
| spot_count | uint64 | Spot count | ||
| spot_count_ice_rings | uint64 | Number of spots within identified rings (experimental) | ||
| spot_count_low_res | uint64 | Number of spots in low resolution (prior to filtering) | ||
| spot_count_indexed | uint64 | Number of spots which fit indexing solution within a given tolerance | ||
| az_int_profile | Array(float) | Azimuthal integration results, use az_int_bin_to_q from start message for legend | ||
| NaN is used for empty bins and has to be taken care by the receiver | ||||
| az_int_profile_std | Array(float) | Standard deviation for azimuthal integration. (NaN for less than 2 samples) | ||
| az_int_profile_count | Array(uint64) | Number of pixels contributing to azimuthal bin | ||
| indexing_result | bool | Indexing successful | ||
| indexing_lattice_count | int64 | Number of indexing lattices found for this image | ||
| indexing_lattice | Array(9 * float) | Indexing result real lattice; present only if indexed | X | |
| indexing_extra_lattices | Array(Array(9*float)) | Additional indexed lattices (orientation variants); present only if found | ||
| indexing_unit_cell | object | Indexing result unit cell: a, b, c [angstrom] and alpha, beta, gamma [degree]; present only if indexed | X | |
| Unit cell is redundant to lattice - yet to simplify downstream programs to analyze results, both are provided | ||||
| profile_radius | float | Profile radius of the image - describes distance of observed reflections from the Ewald sphere [Angstrom^-1] | ||
| integrated_reflections | int64 | Count of integrated reflections | ||
| mosaicity | float | Angular range of spots in image from a rotation scan [degree] | ||
| b_factor | float | Estimated B-factor (Angstrom^2) | ||
| compression_time | float | Time spent on compression/decompressing image [s] | ||
| preprocessing_time | float | Time spent on preparing the image for analysis [s] | ||
| azint_time | float | Time spent on azimuthal integration [s] | ||
| spot_finding_time | float | Time spent on spot finding [s] | ||
| indexing_time | float | Time spent on indexing [s] | ||
| refinement_time | float | Time spent on refinement of indexing solution and experimental geometry [s] | ||
| index_analysis_time | float | Time spent on analyzing idnexing solution, calculating profile radius and mosaicity [s] | ||
| bragg_prediction_time | float | Time spent on predicting Bragg spots [s] | ||
| integration_time | float | Time spent on Bragg integration [s] | ||
| image_scale_time | float | Time spent on on-the-fly scaling [s] | ||
| processing_time | float | Total processing time [s] | ||
| xfel_pulse_id | uint64 | Bunch ID (for pulsed source, e.g., SwissFEL) | X | |
| xfel_event_code | uint64 | Event code (for pulsed source, e.g., SwissFEL) | X | |
| lattice_type | object | Bravais lattice classification of the indexing result (present only if available) | X | |
| - centering | string | One-letter centering code: P, A, B, C, I, F, or R | ||
| - niggli_class | int64 | Integer identifier for the Niggli-reduced Bravais class | ||
| - system | string | Crystal system: triclinic, monoclinic, orthorhombic, tetragonal, trigonal, hexagonal, cubic | ||
| jf_info | uint64 | Detector info field | ||
| receiver_aq_dev_delay | uint64 | Receiver internal delay | ||
| receiver_free_send_buf | uint64 | Receiver internal number of available buffer locations | ||
| receiver_buf_in_sending | uint64 | Receiver internal number of buffer locations currently in sending/writing | ||
| receiver_buf_in_preparation | uint64 | Receiver internal number of buffer locations currently in processing | ||
| storage_cell | uint64 | Storage cell number | ||
| saturated_pixel_count | uint64 | Saturated pixel count | ||
| pixel_sum | uint64 | Sum of all pixels, excl. error and saturation | ||
| error_pixel_count | uint64 | Error pixel count | ||
| strong_pixel_count | uint64 | Strong pixel count (first stage of spot finding) | ||
| min_viable_pixel_value | int64 | Minimal pixel value, excl. error and saturation | ||
| max_viable_pixel_value | int64 | Maximal pixel value, excl. error and saturation | ||
| resolution_estimate | float | Resolution the merged data are predicted to reach, from this image's spots alone [Angstrom] | X | |
| data_collection_efficiency | float | Image collection efficiency [] | ||
| packets_expected | uint64 | Number of packets expected per image (in units of 2 kB) | ||
| packets_received | uint64 | Number of packets received per image (in units of 2 kB) | ||
| bkg_estimate | float | Mean value for pixels in resolution range from 3.0 to 5.0 A [photons] | ||
| ice_ring_score | float | Strongest hexagonal-ice ring intensity over the smooth radial background (1 = no ice) | ||
| spot_count_ice_control | float | Spots in the ice-free flanks beside the hexagonal rings, rescaled to the ring bands' own q width (control for spot_count_ice_rings) | ||
| beam_corr_x | float | Beam center correction X applied during processing [pixel] | X | |
| beam_corr_y | float | Beam center correction Y applied during processing [pixel] | X | |
| image_scale_factor | float | Scaling result: Image scale factor (g) | X | |
| image_scale_mosaicity | float | Scaling result: Image scale mosaicity [deg] | X | |
| image_scale_cc | float | Scaling result: Image scale CC | X | |
| adu_histogram | Array(uint64) | ADU histogram | ||
| roi_integrals | object | Results of ROI calculation | X | |
| - sum | int64 | Sum of pixels in ROI area [photons] | ||
| - sum_square | int64 | Sum of squares of pixels in ROI area [photons] | ||
| - pixels | uint64 | Valid pixels in ROI area | ||
| - max_count | int64 | Highest count in ROI area [photons] | ||
| - x_weighted_sum | int64 | ROI pixel X position multiplied by photon count [photons * pixels] | ||
| - y_weighted_sum | int64 | ROI pixel Y position multiplied by photon count [photons * pixels] | ||
| user_data | string | Optional user defined text information - this is image_appendix serialized to JSON format | X | |
| data | Map(string -> Image) | Image | X |
Metadata message
| Field name | Type | Description | Present in DECTRIS format | Optional |
|---|---|---|---|---|
| type | String | value "metadata" | X | |
| magic_number | uint64 | Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver | ||
| series_unique_id | string | Unique text ID of the series (run_name parameter) | X | |
| series_id | uint64 | Unique numeric ID of the series (run_number parameter) | X | |
| images | Array(object) | Array of images (order and size of the array are not guaranteed) | X | |
| - image_id | uint64 | Number of image within the series; for MX lossy compression this is sequential excluding removed frames | X | |
| - original_image_id | uint64 | Number of image within the series; for MX lossy compression this includes removed frames in the count | ||
| - real_time | Rational | Exposure time | X | |
| - start_time | Rational | Exposure start time (highly approximate) | X | |
| - end_time | Rational | Exposure end time (highly approximate) | X | |
| - spot_count | uint64 | Spot count | ||
| - spot_count_ice_rings | uint64 | Number of spots within identified rings (experimental) | ||
| - az_int_profile | Array(float) | Azimuthal integration results, use az_int_bin_to_q from start message for legend | ||
| - indexing_result | bool | Indexing successful | ||
| - indexing_lattice | Array(9 * float) | Indexing result real lattice; present only if indexed | X | |
| - indexing_unit_cell | object | Indexing result unit cell: a, b, c [angstrom] and alpha, beta, gamma [degree]; present only if indexed | X | |
| Unit cell is redundant to lattice - yet to simplify downstream programs to analyze results, both are provided | ||||
| - xfel_pulse_id | uint64 | Bunch ID (for pulsed source, e.g., SwissFEL) | X | |
| - xfel_event_code | uint64 | Event code (for pulsed source, e.g., SwissFEL) | X | |
| - jf_info | uint64 | Detector info field | ||
| - receiver_aq_dev_delay | uint64 | Receiver internal delay | ||
| - receiver_free_send_buf | uint64 | Receiver internal number of available send buffers | ||
| - storage_cell | uint64 | Storage cell number | ||
| - saturated_pixel_count | uint64 | Saturated pixel count | ||
| - error_pixel_count | uint64 | Error pixel count | ||
| - strong_pixel_count | uint64 | Strong pixel count (first stage of spot finding) | ||
| - data_collection_efficiency | float | Image collection efficiency [] | ||
| - bkg_estimate | float | Mean value for pixels in resolution range from 3.0 to 5.0 A [photons] (with solid angle/polarization corrections, if applied) | X | |
| - resolution_estimate | float | Predicted merged resolution, from spots alone | X | |
| - adu_histogram | Array(uint64) | ADU histogram | X | |
| - roi_integrals | object | Results of ROI calculation | X | |
| - - sum | int64 | Sum of pixels in ROI area [photons] | ||
| - - sum_square | int64 | Sum of squares of pixels in ROI area [photons] | ||
| - - pixels | uint64 | Valid pixels in ROI area | ||
| - - max_count | int64 | Highest count in ROI area [photons] |
End message
| Field name | Type | Description | Present in DECTRIS format |
|---|---|---|---|
| type | String | value "end" | X |
| magic_number | uint64 | Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver | |
| series_unique_id | string | Unique text ID of the series (run_name parameter) | X |
| series_id | uint64 | Unique numeric ID of the series (run_number parameter) | X |
| end_date | string | Approximate end date | |
| max_image_number | uint64 | Number of image with the highest number; counted from 1 to distinguish zero images and one image | |
| transformations | Array(object) (optional) | Sample transformation chain in mounting order, base first. Each element mirrors a NeXus NXtransformations axis: name, transformation_type (rotation/translation), units, vector, offset, depends_on (the axis this one is mounted on, empty for the base), and values - a single number for an axis that does not move, otherwise one per image. An ARRAY because the order matters and a CBOR map has none. Optional: when absent the writer builds the same chain from the start message. It is in the END message because a producer may want to report positions that were measured rather than commanded, which are only known once the run is over |
|
| images_collected | uint64 | Number of images collected | |
| images_sent_to_write | uint64 | Number of images sent to writer; if writer queues were full, it is possible this is less than images collected | |
| data_collection_efficiency | float | Overall network packets collected / network packets expected | |
| az_int_result | Map(text->Array(float)) | Azimuthal integration results, use az_int_bin_to_q from start message for legend | |
| adu_histogram | Map(text->Array(uint64)) | ADU values histogram | |
| adu_histogram_bin_width | uint64 | Width of bins in the above histogram [ADU] | |
| max_receiver_delay | uint64 | Internal performance of Jungfraujoch | |
| bkg_estimate | float | Mean background estimate for the whole run | |
| indexing_rate | float | Mean indexing rate for the whole run | |
| unit_cell | object (optional) | Unit cell of the system, based on the actual experiment: a, b, c [angstrom] and alpha, beta, gamma [degree] | |
| rotation_lattice_type | object | Bravais lattice classification of the total rotation solution over the run, if available; same schema as lattice_type |
|
| - centering | string | One-letter centering code: P, A, B, C, I, F, or R | |
| - niggli_class | int64 | Integer identifier for the Niggli-reduced Bravais class | |
| - system | string | Crystal system: triclinic, monoclinic, orthorhombic, tetragonal, trigonal, hexagonal, cubic | |
| rotation_lattice | Array(9 * float) | Real-space lattice basis, flattened 3x3 in row-major order | |
| rotation_extra_lattices | Array(Array(9*float)) | Additional indexed lattices (orientation variants); present only if found | |
| data_collection_efficiency_image | Array(float) | Per-image data collection efficiency. Missing values are encoded as 0 or 1 depending on producer context | |
| spot_count | Array(int32) | Per-image spot count | |
| spot_count_ice_ring | Array(int32) | Per-image number of spots within identified ice-ring resolution ranges | |
| spot_count_low_res | Array(int32) | Per-image number of low-resolution spots | |
| spot_count_indexed | Array(int32) | Per-image number of spots fitting indexing solution | |
| image_indexed | Array(uint8) | Per-image indexing result; 0 = not indexed, nonzero = indexed | |
| v_bkg_estimate | Array(float) | Per-image background estimate | |
| ice_ring_score | Array(float) | Per-image strongest ice-ring intensity over the smooth radial background (1 = no ice) | |
| spot_count_ice_control | Array(float) | Per-image spot count in the ice-free flanks beside the hexagonal rings, rescaled to the ring bands' q width | |
| ice_ring_score_mean | float | Mean ice-ring score for the whole run (1 = no ice) | |
| profile_radius | Array(float) | Per-image profile radius [Angstrom^-1] | |
| mosaicity | Array(float) | Per-image mosaicity [degree] | |
| bFactor | Array(float) | Per-image estimated B-factor [Angstrom^2] | |
| resolution_estimate | Array(float) | Per-image predicted merged resolution, from spots alone [Angstrom] | |
| min_viable_pixel_value | Array(int64) | Per-image minimum valid pixel value, excluding error/saturated pixels | |
| max_viable_pixel_value | Array(int64) | Per-image maximum valid pixel value, excluding error/saturated pixels | |
| saturated_pixel_count | Array(int32) | Per-image saturated pixel count | |
| error_pixel_count | Array(int32) | Per-image error pixel count | |
| image_scale_factor | Array(float) | Per-image scale factor, if scaling/merging was performed | |
| integrated_reflections | Array(int32) | Per-image count of integrated reflections | |
| indexed_lattice_count | Array(int32) | Per-image count of indexed lattices | |
| niggli_class | Array(uint8) | Per-image Niggli class identifier for indexed images; 0 if unavailable | |
| pixel_sum | Array(int64) | Per-image sum of all valid pixels, excluding error/saturated pixels | |
| image_scale_mosaicity | Array(float) | Scaling result: Image scale mosaicity [deg] | |
| image_scale_cc | Array(float) | Scaling result: Image scale CC |
End-message vector fields are optional. When present, they provide master-file summary data so readers can inspect scan-level and per-image analysis results without opening every linked data file. Missing optional per-image values are encoded by the producer as zero unless otherwise noted.
Calibration message
| Field name | Type | Description | Present in DECTRIS format |
|---|---|---|---|
| type | String | value "calibration" | |
| magic_number | uint64 | Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver | |
| data | Map(string -> Image) | Calibration map (only single pedestal array per message) |
User data
In many cases there is an interest from facilities to forward more metadata, than available explicitly in the Jungfraujoch.
For this reason two fields can be provided: header_appendix (sent with start message) and image_appendix (send with image message).
To increase flexibility, both appendices can contain any valid JSON message.
These appendices are serialized into string and stored in CBOR messages as user_data.
Notably for start message, user_data can contain more information (non-DECTRIS compliant metadata).
Therefore user_data is serialized by Jungfraujoch as CBOR object. There is member user which contains header_appendix defined in OpenAPI of Jungfraujoch.
Notes on images and compression
- Images are encoded as DECTRIS MultiDimArray with typed array tags:
- For RGB: shape [3, height, width], type: u8
- For grayscale: shape [height, width], type according to bit depth and sign (e.g., uint16 LE)
- Compression:
- Uncompressed: raw CBOR byte string
- Bitshuffle+LZ4: tag with ["bslz4", elem_size, bytes]
- Bitshuffle+Zstandard: tag with ["bszstd", elem_size, bytes]
Notes on typed arrays
Jungfraujoch uses RFC 8746-style typed byte-string tags for compact numeric arrays.
Common tags used in this protocol include:
- float32 little-endian arrays for
Array(float) - uint8 arrays for compact boolean/integer flags such as
image_indexed - int32 little-endian arrays for per-image counts
- int64 little-endian arrays for large per-image integer values
- uint64 little-endian arrays for histograms