Jungfraujoch/docs/CBOR.md

CBOR messages

To communicate between the FPGA-equipped receiver system and the writers, Jungfraujoch uses binary CBOR encoding with the tinycbor library (Intel). The protocol is based on and compatible with DECTRIS Stream2. There are minor differences at the moment:

• LZ4 alone is not allowed; Bitshuffle+LZ4 and Bitshuffle+Zstandard are allowed
• A few fields are currently absent
• Extra fields are present beyond DECTRIS standard
• There are calibration and metadata messages defined beyond DECTRIS specification

Start message

Field name	Type	Description	Present in DECTRIS format
type	String	value "start"	X
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
detector_distance	float	Detector distance [m]
detector_translation	Array(float)	Detector translation vector [m]	X
beam_center_x	float	Beam center in X direction [pixels]	X
beam_center_y	float	Beam center in Y direction [pixels]	X
direct_beam_x	float (optional)	Where the undeflected beam lands on the detector, X [pixels]. Not the same point as beam_center_x, which is the PONI - the foot of the perpendicular from the sample - and separates from the beam position as soon as the detector is tilted. This is the number a program that asks for "the beam centre" (XDS ORGX, for one) wants
direct_beam_y	float (optional)	Where the undeflected beam lands on the detector, Y [pixels] (XDS ORGY)
countrate_correction_enabled	bool	Countrate correction enabled	X
countrate_correction_lookup_table	uint32 array (optional)	Maps a measured count c to its corrected value [c], as sent by a DECTRIS detector	X
flatfield_enabled	bool	Flatfield enabled	X
virtual_pixel_interpolation_enabled	bool (optional)	Virtual pixel interpolation enabled, as reported by a DECTRIS detector	X
number_of_images	uint64	Number of images in the series	X
image_size_x	uint64	Image width [pixels]	X
image_size_y	uint64	Image height [pixels]	X
mirror_y	bool	Whether the assembled image is mirrored in Y relative to the detector's raw readout order. True is the MX convention - row 0 at the top of the detector seen from the sample - and is what absence of the key means
detector_orientation_mirror_y	bool	Whether the assembled image is mirrored in Y relative to the frame the PONI angles are stated in. A different setting from mirror_y above, which is about the module layout; this one changes no pixel. Absence means false
detector_orientation_quarter_turns	int	How many multiples of 90 degrees about the beam the assembled image is turned by, relative to the frame the PONI angles are stated in (0-3). Absence means 0
detector_modules	Array(object) (optional)	Panels: where the image's pixels lie on the detector plane the geometry describes. Each has x0, y0, width, height (int, the panel's box in the image) and ox, oy, fx, fy, sx, sy (double, plane pixels): image pixel (x, y) of the panel is at (ox, oy) + (x - x0) (fx, fy) + (y - y0) (sx, sy); fast_axis, slow_axis (string Xp/Xn/Yp/Yn, optional, default Xp/Yp) say which way the panel's readout axes run in the image, naming it in a module placement. Absent for a flat detector, the image being the plane; spot and reflection positions are always plane coordinates
incident_energy	float	X-ray energy [eV]	X
incident_wavelength	float	X-ray wavelength [Angstrom]	X
incident_wavelength_spread	float (optional)	FWHM of the X-ray wavelength distribution [Angstrom] (NXmx incident_wavelength_spread); omitted when the beam is monochromatic
beam_size_x	float (optional)	Horizontal size of the X-ray beam at the sample [m] (first element of NXmx incident_beam_size)
beam_size_y	float (optional)	Vertical size of the X-ray beam at the sample [m] (second element of NXmx incident_beam_size)
frame_time	float	Frame time, if multiple frames per trigger [s]	X
count_time	float	Exposure time [s]	X
saturation_value	int64	Maximum valid sample value	X
error_value	int64 (optional)	Value used in images to describe pixels that are in error state or missing
pixel_size_x	float	Pixel width [m]	X
pixel_size_y	float	Pixel height [m]	X
sensor_thickness	float	Sensor thickness [m]	X
sensor_material	string	Sensor material	X
arm_date	date	Approximate date of arming	X
pixel_mask_enabled	bool	Pixel mask applied on images	X
detector_description	string	Name of the detector	X
detector_serial_number	string	Detector serial number	X
series_unique_id	string	Unique text ID of the series (run_name parameter)	X
series_id	uint64	Unique numeric ID of the series (run_number parameter)	X
fluorescence	object (optional)	X-ray fluorescence spectrum collected at start
- energy	Array(float)	Energy of measuring point [eV]
- data	Array(float)	Fluorescence scan result data [arbitrary units]; must be strictly the same length as energy
goniometer	Map	Definition of rotation axis (optional)	X
- AXIS	string	Rotation axis name (e.g. omega) - only one axis is supported in Jungfraujoch	X
- - increment	float	Rotation axis increment (per image) in degree [deg]	X
- - start	float	Rotation axis start angle [deg]	X
- - axis	Array(float)	Vector for the rotation axis
- - helical_step	Array(float)	Translation for helical scan for 1 image [m]
- - screening_wedge	Array(float)	Wedge for screening [deg] (increment would correspond to difference between screening points)
grid_scan	object	Grid scan definition (optional). Send goniometer with it, increment 0, to state the angle the spindle stood at; without one the spindle is recorded at 0, meaning "nobody said"
- n_fast	uint64	Number of elements along fast axis
- n_slow	uint64	Number of elements along slow axis
- step_x_axis	float	Step along X axis, can be negative [m]
- step_y_axis	float	Step along Y axis, can be negative [m]
- snake_scan	bool	Snake scan (rows alternate direction)
- vertical_scan	bool	Vertical scan (enabled: fast direction = Y, disabled: fast direction = X)
jungfrau_conversion_enabled	bool (optional)	Applying JUNGFRAU pixel conversion (to photons or keV)
jungfrau_conversion_factor	float (optional)	Factor used for JUNGFRAU conversion [eV]
geometry_transformation_enabled	bool (optional)	Transformation from detector module geometry (512x1024) to full detector geometry
pixel_mask	Map(string -> Image)	Pixel mask - multiple in case of storage cells	X
channels	Array(string)	List of image channels	X
max_spot_count	uint64	Maximum number of spots identified in spot finding
max_extra_lattices	uint64	Maximum number of extra lattices
storage_cell_number	uint64 (optional)	Number of storage cells used by JUNGFRAU
storage_cell_delay	Rational	Delay of storage cells in JUNGFRAU
threshold_energy	Map(string -> float)	Per-channel threshold energy [eV] (map of channel name to value)
image_dtype	string	Pixel type of the image data: uint8, uint16, uint32 (DECTRIS), plus int8, int16, int32 as a Jungfraujoch extension. Sole wire encoding of both the bit depth and the sign, and must agree with the per-image typed-array tag	X
unit_cell	object (optional)	Unit cell of the system: a, b, c [angstrom] and alpha, beta, gamma [degree]
az_int_q_bin_count	uint64	Number of azimuthal integration bins in the radial direction
az_int_phi_bin_count	uint64	Number of azimuthal integration bins in the phi angle direction
az_int_bin_to_q	Array(float)	Q value for each azimuthal integration bin [angstrom^-1]
az_int_bin_to_two_theta	Array(float)	Two theta angle value for each azimuthal integration bin [deg]
az_int_bin_to_phi	Array(float)	Phi value for each azimuthal integration bin [deg]
az_int_map	Image	Mapping between pixel and bin number
summation	uint64	Factor of frame summation
user_data	string	JSON serialized to string that can contain the following fields (all fields are optional):	X
- file_prefix	string	File prefix
- images_per_file	uint64	Number of images written per file
- images_per_trigger	uint64	Number of images collected per trigger
- source_name	string	Facility name
- source_type	string	Type of X-ray source (use NXsource/type values, for example "Synchrotron X-ray Source" or "Free-Electron Laser")
- instrument_name	string	Instrument name
- sample_name	string	Name of the sample
- user	any valid JSON	Value of header_appendix provided at collection start to Jungfraujoch
- attenuator_transmission	float	Attenuator transmission []
- total_flux	float	Total flux [ph/s]
- space_group_number	uint64	Space group number
- summation_mode	string	Summation mode (internal|fpga|cpu)
- overwrite	bool	Overwrite existing HDF5 files
- file_format	int	File writer format: 0 = only data files, 1 = NXmx legacy external links, 2 = NXmx VDS, 3 = NXmx integrated, 4 = CBF (retired; rejected), 5 = TIFF (retired; rejected), 6 = no file written
- roi	Array(object)	ROI configurations; each has a name (the key of its roi_integrals entry and of /entry/roi/<name> in HDF5) and is one of:
		type "box": xmin, xmax, ymin, ymax (numbers)
		type "circle": r, x, y (numbers)
		type "azim": qmin, qmax (numbers); optional phi_min, phi_max (numbers, deg) for an angular sector
- gain_file_names	Array(string)	Names of JUNGFRAU gain files used for the current detector
- write_master_file	bool	With multiple sockets, it selects which socket will provide master file
- write_images	bool	Write images in the HDF5 file (if false, will only write metadata)
- data_reduction_factor_serialmx	uint64	Data reduction factor for serial MX
- experiment_group	string	ID of instrument user, e.g., p-group (SLS/SwissFEL) or proposal number
- jfjoch_release	string	Jungfraujoch release number
- socket_number	uint64	Number of ZeroMQ socket (on jfjoch_broker side) used for transmission
- bit_depth_readout	uint64	Bit depth of the stored image (see note below), copied to NXmx bit_depth_readout
- underload_value	int64	Lowest valid value; copied to NXmx underload_value. 0 for an unsigned image, INTx_MIN + 1 for a signed one
- writer_notification_zmq_addr	string	ZeroMQ address to inform jfjoch_broker about writers that finished operation
- xfel_pulse_id	uint64	Pulse IDs are recorded for images
- ring_current_mA	float	Ring current at the start of the measurement
- sample_temperature_K	float	Sample temperature [K]
- detect_ice_rings	bool	Ice ring detection feature is enabled
- indexing_algorithm	string	Indexing algorithm used on-the-fly; allowed values: ffbidx, fft, fftw, none
- geom_refinement_algorithm	string	Post-indexing detector geometry refinement algorithm; allowed values: none, beam_center
- poni_rot1	float	Tilt of the detector rot1 according to PyFAI PONI convention [rad]
- poni_rot2	float	Tilt of the detector rot2 according to PyFAI PONI convention [rad]
- poni_rot3	float	Tilt of the detector rot3 according to PyFAI PONI convention [rad]
- smargon	object	Smargon head position, where one was given: chi_deg, phi_deg and the chi_axis, phi_axis vectors
roi_map	Image	ROI bitmask per pixel (uint16, bit n = pixel belongs to ROI n); present when ROIs are defined

See DECTRIS documentation for definition of Image as MultiDimArray with optional compression.

Image message

Every scalar of the image message is optional: a quantity that was not computed for the image is absent from the map, whatever the column below says. spots[].panel, az_int_profile_std, indexing_extra_lattices and roi_integrals are likewise present only where there is something to report; the reflection fields qe and flight are always written.

Field name	Type	Description	Present in DECTRIS format	Optional
type	String	value "image"	X
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
series_unique_id	string	Unique text ID of the series (run_name parameter)	X
series_id	uint64	Unique numeric ID of the series (run_number parameter)	X
image_id	uint64	Number of image within the series; for MX lossy compression this is sequential excluding removed frames	X
original_image_id	uint64	Number of image within the series; for MX lossy compression this includes removed frames in the count
real_time	Rational	Exposure time	X
start_time	Rational	Exposure start time (highly approximate)	X
end_time	Rational	Exposure end time (highly approximate)	X
spots	Array(object)	Spots:
- x	float	observed position in x (pixels)
- y	float	observed position in y (pixels)
- I	float	intensity (photons)
- maxc	int64	max count (photons)
- ice_ring	bool	spot in resolution range for ice rings
- indexed	bool	indexed solution
- latt	int64	Lattice to which the peak belongs (negative number = not indexed)
- image	int64	image number the spot belongs to
- panel	int64 (optional)	panel (index into detector_modules) the spot was found on; absent for a detector without a panel table
- h	int64	Miller index (indexed spots only)
- k	int64	Miller index (indexed spots only)
- l	int64	Miller index (indexed spots only)
- dist_ewald	float	distance to Ewald sphere [Angstrom^-1] (indexed spots only)
reflections	Array(object)	Reflections:
- h	int64	Miller index
- k	int64	Miller index
- l	int64	Miller index
- x	float	predicted position in x (pixels)
- y	float	predicted position in y (pixels)
- obs_x	float	observed position in x (pixels)
- obs_y	float	observed position in y (pixels)
- d	float	resolution [Angstrom]
- I	float	integrated intensity (photons)
- bkg	float	mean background value (photons)
- var_bkg	float	non-signal (background) part of sigma^2, carried to the merge (photons^2)
- sigma	float	standard deviation, estimated from counting statistics (photons)
- image	int64	image number the reflection belongs to
- rp	float	Distance to Ewald sphere [Angstrom^-1]
- rlp	float	Reciprocal Lorentz-polarization factor: the multiplier taking the raw integrated count toward a quantity proportional to |F|^2. Lorentz x polarization only - a still has no Lorentz term, so there it is the polarization alone
- qe	float	Sensor efficiency at the reflection's angle of incidence, QE(0)/QE(alpha); <= 1, and 1 where the sensor is opaque or unknown. Carried beside rlp, not inside it: the total correction is rlp * qe. Optional
- flight	float	Attenuation of the reflection in the flight path between the sample and its pixel, normalised to normal incidence; >= 1, and exactly 1 for a vacuum path. Carried beside rlp and qe: the total correction is rlp * qe * flight. Optional
- partiality	float	Partiality of the reflection
- phi	float	phi angle from XDS: difference from middle of current frame, not absolute [deg]
- zeta	float	Lorentz zeta factor (reciprocal-space geometry term)
- image_scale_corr	float	Per-image scale correction; I_true = image_scale_corr * I
spot_count	uint64	Spot count
spot_count_ice_rings	uint64	Number of spots within identified rings (experimental)
spot_count_low_res	uint64	Number of spots in low resolution (prior to filtering)
spot_count_indexed	uint64	Number of spots which fit indexing solution within a given tolerance
az_int_profile	Array(float)	Azimuthal integration results, use az_int_bin_to_q from start message for legend
		NaN is used for empty bins and has to be taken care by the receiver
az_int_profile_std	Array(float)	Standard deviation for azimuthal integration. (NaN for less than 2 samples)
az_int_profile_count	Array(uint64)	Number of pixels contributing to azimuthal bin
indexing_result	bool	Indexing successful
indexing_lattice_count	int64	Number of indexing lattices found for this image
indexing_lattice	Array(9 * float)	Indexing result real lattice; present only if indexed		X
indexing_extra_lattices	Array(Array(9*float))	Additional indexed lattices (orientation variants); present only if found
indexing_unit_cell	object	Indexing result unit cell: a, b, c [angstrom] and alpha, beta, gamma [degree]; present only if indexed		X
		Unit cell is redundant to lattice - yet to simplify downstream programs to analyze results, both are provided
profile_radius	float	Profile radius of the image - describes distance of observed reflections from the Ewald sphere [Angstrom^-1]
integrated_reflections	int64	Count of integrated reflections
mosaicity	float	Angular range of spots in image from a rotation scan [degree]
b_factor	float	Estimated B-factor (Angstrom^2)
compression_time	float	Time spent on compression/decompressing image [s]
preprocessing_time	float	Time spent on preparing the image for analysis [s]
azint_time	float	Time spent on azimuthal integration [s]
spot_finding_time	float	Time spent on spot finding [s]
indexing_time	float	Time spent on indexing [s]
refinement_time	float	Time spent on refinement of indexing solution and experimental geometry [s]
index_analysis_time	float	Time spent on analyzing indexing solution, calculating profile radius and mosaicity [s]
bragg_prediction_time	float	Time spent on predicting Bragg spots [s]
integration_time	float	Time spent on Bragg integration [s]
image_scale_time	float	Time spent on on-the-fly scaling [s]
processing_time	float	Total processing time [s]
xfel_pulse_id	uint64	Bunch ID (for pulsed source, e.g., SwissFEL)		X
xfel_event_code	uint64	Event code (for pulsed source, e.g., SwissFEL)		X
lattice_type	object	Bravais lattice classification of the indexing result (present only if available)		X
- centering	string	One-letter centering code: P, A, B, C, I, F, or R
- niggli_class	int64	Integer identifier for the Niggli-reduced Bravais class
- system	string	Crystal system: triclinic, monoclinic, orthorhombic, tetragonal, trigonal, hexagonal, cubic
jf_info	uint64	Detector info field
receiver_aq_dev_delay	uint64	Receiver internal delay
receiver_free_send_buf	uint64	Receiver internal number of available buffer locations
receiver_buf_in_sending	uint64	Receiver internal number of buffer locations currently in sending/writing
receiver_buf_in_preparation	uint64	Receiver internal number of buffer locations currently in processing
storage_cell	uint64	Storage cell number
saturated_pixel_count	uint64	Saturated pixel count
pixel_sum	int64	Sum of pixels, excl. device-masked, error and saturated (negative values included)
error_pixel_count	uint64	Error pixel count
strong_pixel_count	uint64	Strong pixel count (first stage of spot finding)
min_viable_pixel_value	int64	Minimal pixel value, excl. error and saturation
max_viable_pixel_value	int64	Maximal pixel value, excl. error and saturation
resolution_estimate	float	Resolution the merged data are predicted to reach, from this image's spots alone [Angstrom]		X
data_collection_efficiency	float	Image collection efficiency []
packets_expected	uint64	Number of packets expected per image (in units of 2 kB)
packets_received	uint64	Number of packets received per image (in units of 2 kB)
bkg_estimate	float	Mean value for pixels in resolution range from 3.0 to 5.0 A [photons]
ice_ring_score	float	Strongest hexagonal-ice ring intensity over the smooth radial background (1 = no ice)
spindle_blind_fraction	float	Fraction (0-1) of a rotation sweep's blind cone this orientation makes unrecoverable, as a lone-2-fold worst-case bound; >= 0.5 should engage a recovery protocol, and ABSENT means the frame could not be assessed, which automation must treat the same way
spot_count_ice_control	float	Spots in the ice-free flanks beside the hexagonal rings, rescaled to the ring bands' own q width (control for spot_count_ice_rings)
beam_corr_x	float	Beam center correction X applied during processing [pixel]		X
beam_corr_y	float	Beam center correction Y applied during processing [pixel]		X
image_scale_factor	float	Scaling result: Image scale factor (g)		X
image_scale_mosaicity	float	Scaling result: Image scale mosaicity [deg]		X
image_scale_cc	float	Scaling result: Image scale CC		X
adu_histogram	Array(uint64)	ADU histogram
roi_integrals	Map(string -> object)	Results of ROI calculation, one entry per ROI keyed by its name		X
- sum	int64	Sum of pixels in ROI area [photons]
- sum_square	int64	Sum of squares of pixels in ROI area [photons]
- pixels	uint64	Valid pixels in ROI area
- max_count	int64	Highest count in ROI area [photons]
- x_weighted_sum	int64	ROI pixel X position multiplied by photon count [photons * pixels]
- y_weighted_sum	int64	ROI pixel Y position multiplied by photon count [photons * pixels]
user_data	string	Optional user defined text information - this is image_appendix serialized to JSON format	X
data	Map(string -> Image)	Image	X

Metadata message

Field name	Type	Description	Present in DECTRIS format	Optional
type	String	value "metadata"	X
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
series_unique_id	string	Unique text ID of the series (run_name parameter)	X
series_id	uint64	Unique numeric ID of the series (run_number parameter)	X
images	Array(object)	Array of images (order and size of the array are not guaranteed)	X
- ...		Every field of the image message except spots, reflections and data (the image itself), with the same names, types and optionality

End message

Field name	Type	Description	Present in DECTRIS format
type	String	value "end"	X
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
series_unique_id	string	Unique text ID of the series (run_name parameter)	X
series_id	uint64	Unique numeric ID of the series (run_number parameter)	X
end_date	string	Approximate end date
max_image_number	uint64	Number of image with the highest number; counted from 1 to distinguish zero images and one image
transformations	Array(object) (optional)	Sample transformation chain in mounting order, base first. Each element mirrors a NeXus NXtransformations axis: name, transformation_type (rotation/translation), units, vector, offset, depends_on (the axis this one is mounted on, empty for the base), optional equipment and equipment_component (NXtransformations attributes; equipment_component is smargon on a head's chi/phi, which is how a reader tells them from the spindle), and values - a single number for an axis that does not move, otherwise one per image. An ARRAY because the order matters and a CBOR map has none. Optional: when absent the writer builds the same chain from the start message. It is in the END message because a producer may want to report positions that were measured rather than commanded, which are only known once the run is over
images_collected	uint64	Number of images collected
images_sent_to_write	uint64	Number of images sent to writer; if writer queues were full, it is possible this is less than images collected
data_collection_efficiency	float	Overall network packets collected / network packets expected
az_int_result	Map(text->Array(float))	Azimuthal integration results, use az_int_bin_to_q from start message for legend
adu_histogram	Map(text->Array(uint64))	ADU values histogram
adu_histogram_bin_width	uint64	Width of bins in the above histogram [ADU]
max_receiver_delay	uint64	Internal performance of Jungfraujoch
bkg_estimate	float	Mean background estimate for the whole run
refined_beam_center_x	float	Beam centre X the frames were analysed at, where the analysis refined it [pixel]
refined_beam_center_y	float	Beam centre Y the frames were analysed at, where the analysis refined it [pixel]
refined_detector_distance_mm	float	Detector distance the frames were analysed at, where the analysis refined it [mm]
spindle_blind_fraction	float	Run mean of the per-image spindle_blind_fraction, over the frames that had one
spindle_lost_unique_fraction	float	Fraction (0-1) of unique reflections the mounting made unmeasurable, exact under the measured point group; offline (Rugnux) only
indexing_rate	float	Mean indexing rate for the whole run
unit_cell	object (optional)	Unit cell of the system, based on the actual experiment: a, b, c [angstrom] and alpha, beta, gamma [degree]
unit_cell_esd	object (optional)	e.s.d.s of unit_cell, same layout: a, b, c [angstrom] and alpha, beta, gamma [degree]; rotation, offline (Rugnux) and in-broker analysis only
space_group_name	string (optional)	The space group determined by the analysis, gemmi extended Hermann-Mauguin (carries the setting; what a reader should take)
space_group_number	uint64 (optional)	Its International Tables number, beside the name for a reader written before the name existed
rotation_lattice_type	object	Bravais lattice classification of the total rotation solution over the run, if available; same schema as lattice_type
- centering	string	One-letter centering code: P, A, B, C, I, F, or R
- niggli_class	int64	Integer identifier for the Niggli-reduced Bravais class
- system	string	Crystal system: triclinic, monoclinic, orthorhombic, tetragonal, trigonal, hexagonal, cubic
rotation_lattice	Array(9 * float)	Real-space lattice basis, flattened 3x3 in row-major order
rotation_extra_lattices	Array(Array(9*float))	Additional indexed lattices (orientation variants); present only if found
data_collection_efficiency_image	Array(float)	Per-image data collection efficiency. Missing values are encoded as 0 or 1 depending on producer context
spot_count	Array(int32)	Per-image spot count
spot_count_ice_ring	Array(int32)	Per-image number of spots within identified ice-ring resolution ranges	key is singular here; the per-image message uses spot_count_ice_rings
spot_count_low_res	Array(int32)	Per-image number of low-resolution spots
spot_count_indexed	Array(int32)	Per-image number of spots fitting indexing solution
image_indexed	Array(uint8)	Per-image indexing result; 0 = not indexed, nonzero = indexed
v_bkg_estimate	Array(float)	Per-image background estimate
v_spindle_blind_fraction	Array(float)	Per-image spindle_blind_fraction; NaN where the frame had no value (which is "cannot say", not zero)
ice_ring_score	Array(float)	Per-image strongest ice-ring intensity over the smooth radial background (1 = no ice)
spot_count_ice_control	Array(float)	Per-image spot count in the ice-free flanks beside the hexagonal rings, rescaled to the ring bands' q width
ice_ring_score_mean	float	Mean ice-ring score for the whole run (1 = no ice)
profile_radius	Array(float)	Per-image profile radius [Angstrom^-1]
mosaicity	Array(float)	Per-image mosaicity [degree]
bFactor	Array(float)	Per-image estimated B-factor [Angstrom^2]
resolution_estimate	Array(float)	Per-image predicted merged resolution, from spots alone [Angstrom]
min_viable_pixel_value	Array(int64)	Per-image minimum valid pixel value, excluding error/saturated pixels
max_viable_pixel_value	Array(int64)	Per-image maximum valid pixel value, excluding error/saturated pixels
saturated_pixel_count	Array(int32)	Per-image saturated pixel count
error_pixel_count	Array(int32)	Per-image error pixel count
image_scale_factor	Array(float)	Per-image scale factor, if scaling/merging was performed
integrated_reflections	Array(int32)	Per-image count of integrated reflections
indexed_lattice_count	Array(int32)	Per-image count of indexed lattices
niggli_class	Array(uint8)	Per-image Niggli class identifier for indexed images; 0 if unavailable
pixel_sum	Array(int64)	Per-image sum of pixels, excl. device-masked, error and saturated (0 for an image with no analysis)
image_scale_mosaicity	Array(float)	Scaling result: Image scale mosaicity [deg]
image_scale_cc	Array(float)	Scaling result: Image scale CC
merged	object (optional)	Merged reflections of an in-broker analysis run, written by the writer as <file_prefix>.mtz. An MTZ table (layout below)
map_coefficients	object (optional)	Map coefficients of an in-broker analysis run, written as <file_prefix>_maps.mtz. An MTZ table (layout below)
placed_model	object (optional)	Atomic model placed against the data, written as <file_prefix>_model.cif and <file_prefix>_model.pdb. name (string), unit_cell (object, as above), space_group_name (the data's, gemmi extended H-M), model_space_group_name (the coordinates'), then one array per atom: model_number, seq_num, charge (int32), chain, residue_name, atom_name, element (Array(string)), icode (uint8, space where none), het_flag (uint8, A, H or 0 where unknown), altloc (uint8, 0 where none), x, y, z (double, Angstrom), b_iso, occupancy (float), aniso (float, 6 per atom: U11 U22 U33 U12 U13 U23; empty where no atom has one), subchain, entity_id (Array(string), mmCIF label_asym_id / label_entity_id), label_seq (int32, label_seq_id, INT32_MIN where none) - these three optional; metadata_mmcif (string, optional): the input model's metadata (entities, sequences, references, authors, software, secondary structure) as mmCIF categories without coordinates, parsed by the writer with gemmi
refined_model	object (optional)	The placed model after refinement, written as <file_prefix>_refined_model.cif and <file_prefix>_refined_model.pdb; same layout as placed_model
refined_map_coefficients	object (optional)	Map coefficients of the refined model, written as <file_prefix>_refined_maps.mtz. An MTZ table (layout below)
blobs	object (optional)	Unexplained blobs of the refined model's mFo-DFc map, strongest first, written as <file_prefix>_blobs.json. One array per field, one entry per blob: kind (Array(string): WATER, ION, LIGAND, MODEL_ERROR, UNASSIGNED), x, y, z (double, centroid, Angstrom), peak_sigma (double, map rms), volume (double, Angstrom^3), score (double, sigma Angstrom^3), nearest_atom (Array(string)), nearest_atom_distance (double, Angstrom), polar_contacts, metal_contacts (int32), anomalous_sigma (double, 0 where there is no anomalous map)

End-message vector fields are optional. When present, they provide master-file summary data so readers can inspect scan-level and per-image analysis results without opening every linked data file. Missing optional per-image values are encoded by the producer as zero unless otherwise noted.

An MTZ table (merged, map_coefficients, refined_map_coefficients) carries an MTZ file's content as the writer writes it, column by column:

Field name	Type	Description
title	string	MTZ title
unit_cell	object	a, b, c [angstrom] and alpha, beta, gamma [degree]
space_group_name	string	Space group, gemmi extended Hermann-Mauguin (keeps the setting)
history	Array(string)	MTZ HISTORY records
sort_order	Array(int32)	MTZ sort order, 5 entries
datasets	Array(object)	One per MTZ dataset: project_name, crystal_name, dataset_name (string), wavelength (float, Angstrom; 0 = not set)
columns	Array(object)	One per MTZ column: label (string), type (string, one character: the MTZ column type H, J, F, Q, I, ...), dataset_id (int), values (float typed array, one per row)
batches	Array(object)	One per image of unmerged data (empty for merged data): number (int), title (string), ints (int32), floats (float) - the MTZ batch header arrays - and axes (Array(string), goniostat axis names)

Calibration message

Field name	Type	Description	Present in DECTRIS format
type	String	value "calibration"
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
data	Map(string -> Image)	Calibration map (only single pedestal array per message)

User data

Facilities often need to forward more metadata than Jungfraujoch models explicitly. For this reason two fields can be provided: header_appendix (sent with the start message) and image_appendix (sent with the image message). To increase flexibility, both appendices can contain any valid JSON message. These appendices are serialized into string and stored in CBOR messages as user_data.

Notably for start message, user_data carries more information (non-DECTRIS compliant metadata, the fields listed above). It is still a JSON text string; its member user holds the header_appendix defined in OpenAPI of Jungfraujoch.

Notes on images and compression

• Images are encoded as DECTRIS MultiDimArray with typed array tags:
  • For RGB: shape [3, height, width], type: u8
  • For grayscale: shape [height, width], type according to bit depth and sign (e.g., uint16 LE)
• Compression:
  • Uncompressed: raw CBOR byte string
  • Bitshuffle+LZ4: tag with ["bslz4", elem_size, bytes]
  • Bitshuffle+Zstandard: tag with ["bszstd", elem_size, bytes]. The BSHUF_ZSTD, BSHUF_ZSTD_RLE and BSHUF_ZSTD_RLE_HUFF algorithms all travel under this tag, and a receiver decodes the tag as plain BSHUF_ZSTD.
  • The start message's pixel_mask, az_int_map and roi_map are always Bitshuffle+LZ4.

Notes on typed arrays

Jungfraujoch uses RFC 8746-style typed byte-string tags for compact numeric arrays.

Common tags used in this protocol include:

• float32 little-endian arrays for Array(float)
• uint8 arrays for compact boolean/integer flags such as image_indexed
• int32 little-endian arrays for per-image counts
• int64 little-endian arrays for large per-image integer values
• uint64 little-endian arrays for histograms