To communicate between FPGA-equipped receiver system and writers, Jungfraujoch is using binary CBOR encoding with tinycbor library (Intel). The protocol is compatible with DECTRIS Stream2. There are minor differences at the moment:

Pixel mask can be only sent uncompressed
Only single channel is allowed for images (this only matters for deserialization)
LZ4 compression is not allowed; Bitshuffle Zstandard is possible
Few fields are currently absent
Extra fields are present beyond DECTRIS standard
There is calibration message defined to transfer pedestal correction

Start message

Field name	Type	Description	Present in DECTRIS format
type	String	value "start"	X
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
detector_distance	float	Detector distance [m]
detector_translation	float	Detector translation vector [m]	X
beam_center_x	float	Beam center in X direction [pixels]	X
beam_center_y	float	Beam center in Y direction [pixels]	X
number_of_images	uint64	Number of images in the series	X
image_size_x	uint64	Image width [pixels]	X
image_size_y	uint64	Image height [pixels]	X
incident_energy	float	X-ray energy [eV]	X
incident_wavelength	float	X-ray wavelength [Angstrom]	X
frame_time	float	Frame time, if multiple frames per trigger [s]	X
count_time	float	Exposure time [s]	X
saturation_value	int64	Maximum valid sample value	X
error_value	int64	Value used in images to describe pixels that are in error state or missing
pixel_size_x	float	Pixel width [m]	X
pixel_size_y	float	Pixel height [m]	X
sensor_thickness	float	Sensor thickness [m]	X
sensor_material	string	Sensor material	X
arm_data	data	Approximate date of arming	X
pixel_mask_enabled	bool	Pixel mask applied on images	X
detector_description	string	Name of the detector	X
series_unique_id	string	Unique text ID of the series	X
series_id	uint64	Unique numeric ID of the series	X
goniometer	Map	See DECTRIS documentation - only "omega" axis is supported	X
pixel_mask	Map(string -> Image)	Pixel mask - multiple in case of storage cells	X
channels	Array(string)	List of image channels	X
max_spot_count	uint64	Maximum number of spots identified in spot finding
storage_cell_number	uint64	Number of storage cells used by JUNGFRAU
image_dtype	string	Pixel bit type (e.g. uint16)	X
unit_cell	Array(6 * float) (optional)	Unit cell of the system: a, b, c [angstrom] and alpha, beta, gamma [degree]
az_int_bin_number	uint64	Number of azimuthal integration bins
az_int_bin_to_q	Array(float)	Q value for each azimuthal integration bin [angstrom^-1]
summation	uint64	Factor of frame summation
user_data	string	JSON string that can contain the following fields (among others):	X
- source_name	string	Facility name
- source_name_short	string	Facility name (short)
- instrument_name	string	Instrument name
- instrument_name_short	string	Instrument name (short
- attenuator_transmission	float (optional)	Attenuator transmission []
- total_flux	float (optional)	Total flux [ph/s]
- data_file_count	uint64	Number of used data files
- file_prefix	string	File prefix
- sample_name	string	Name of the sample
- omega_rotation_axis	Array(float)	Omega rotation axis (array of size 3)
- space_group_number	uint64	Space group number
- write_master_file	bool	With multiple sockets, it selects which socket will provide master file

See DECTRIS documentation for definition of Image as MultiDimArray with optional compression.

Image message

Field name	Type	Description	Present in DECTRIS format	Optional
type	String	value "image"	X
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
series_unique_id	string	Unique text ID of the series	X
series_id	uint64	Unique numeric ID of the series	X
image_id	uint64	Number of image within the series	X
real_time	Rational	Exposure time	X
start_time	Rational	Exposure start time (highly approximate)	X
end_time	Rational	Exposure end time (highly approximate)	X
spots	Array(object)	Spots:
- x	float	position in x (pixels)
- y	float	position in y (pixels)
- I	float	intensity (photons)
- indexed	bool	indexed solution
az_int_profile	Array(float)	Azimuthal integration results, use az_int_bin_to_q from start message for legend
indexing_result	bool	Indexing successful
indexing_lattice	Array(float * 9) (optional)	Indexing result real lattice; present only if indexed
xfel_pulse_id	uint64	Bunch ID (for SwissFEL)		X
xfel_event_code	uint64	Event code (for SwissFEL)		X
jf_info	uint64	Detector info field
receiver_aq_dev_delay	uint64	Receiver internal delay
receiver_free_send_buf	uint64	Receiver internal number of available send buffers
storage_cell	uint64	Storage cell number
saturated_pixel_count	uint64	Saturated pixel count
error_pixel_count	uint64	Error pixel count
strong_pixel_count	uint64	Strong pixel count (first stage of spot finding)
data_collection_efficiency	float	Image collection efficiency []
bkg_estimate	float	Mean value for pixels in resolution range from 3.0 to 5.0 A [\photons]
adu_histogram	Array(uint64)	ADU histogram
corr_beam_x_pxl	float	Difference between provided and calculated beam center X (pixels) - requires indexing		X
corr_beam_y_pxl	float	Difference between provided and calculated beam center Y (pixels) - requires indexing		X
corr_det_dist_mm	float	Difference between provided and calculated detector distance (mm) - requires indexing		X
user_data	string	Optional user defined text information
roi_sum	int64	Sum of ROI rectangle [photons]		X
image	Map(string -> Image)	Image

End message

Field name	Type	Description	Present in DECTRIS format
type	String	value "end"	X
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
series_unique_id	string	Unique text ID of the series	X
series_id	uint64	Unique numeric ID of the series	X
end_date	date	Approximate end date
max_image_number	uint64	Number of image with the highest number (this is counted from 1 - to distinguish zero images and one image)
images_collected	uint64	Number of image collected
images_sent_to_writer	uint64	Number of image sent to writer; if writer queues were full, it is possible this is less than images collected
data_collection_efficiency	float	Network packets collected / Network packets expected []
az_int_result	Map(text->Array(float))	Azimuthal integration results, use az_int_bin_to_q from start message for legend
adu_histogram	Map(text->Array(uint64))	ADU values histogram
adu_histogram_width	uint64	Width of bins in the above histogram [ADU]
max_receiver_delay	uint64	Internal performance of Jungfraujoch

Calibration message

Field name	Type	Description
type	String	value "end"
magic_number	uint64	Number used to describe version of the Jungfraujoch data interface - to allow to detect inconsistency between sender and receiver
data	Map(string -> Image)	Calibration map (only single pedestal array per message)

Image transfer

Images can be forwarded by Jungraujoch through few interfaces. All of them are ZeroMQ based at the moment, though they differ with serialization layer. All interfaces are subclasses of ImagePusher class. The following options are available:

ZMQStream2Pusher

Pushes images using Stream2 serialization. This is using PUSH ZeroMQ sockets. Behavior is as following:

Start message is sent with timeout of 5s. If within the time the message cannot be put in the outgoing queue or there is no connected puller exception is thrown - this is to stop data collection in absence of writer.
Images are sent in non-blocking way and without timeout. If message cannot be sent false is returned for the send call - need to be reported or handled in calling function.
End message is sent with timeout of 5s. If message cannot be sent false is returned for the send call - need to be reported or handled in calling function.

ZMQStream2PusherGroup

Same as above, but multiple ZeroMQ sockets are used and images are distributed on a round-robin basis between the sockets based on the image number. Behavior for errors is the same as for ZMQStream2Pusher.

TestImagePusher

This interface is used for automated tests. It reads incoming images and compares one image with the reference data provided.