mx/Jungfraujoch
3
2 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 93 Wiki Activity

docs: the receiver of a run and Rugnux live stills on the DECTRIS stream

Browse Source 
From rc186 2feef4b02 (JFJOCH_BROKER.md, IMAGE_STREAM.md), for what this branch has: the table of
the receiver each run gets, the DECTRIS serial live mode (pre-pass, analysis, live merge, model
step, CANCELLED after a cancel, a rotation run the buffer cannot hold getting the per-image
analysis), no calibration forwarded from a DECTRIS stream, nothing on the preview and metadata
streams for the DECTRIS mask sequence.

Adapted: JFJochReceiverFPGA takes every PSI run, pedestal included, and still analyses serial runs
live with rugnux on; a paragraph says how that differs (frames wait in half the image buffer, no
model step). Left out: the broker_pipeline figure (it shows JFJochReceiverPedestal and the FPGA
receiver without live stills), the internal-generator memory check (25542a9af, not ported) and
docs/TESTS.md (the card mock, not ported).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-11-11 13:39:48 +01:00
1 parent 8a7f46a4e2
commit fa0c96a095
2 files changed
+90 -22
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/IMAGE_STREAM.md
+11 -2
View File
@@ -11,6 +11,8 @@ These are:
## Image stream
Images (with metadata) are serialized as CBOR [image message](CBOR.md#image-message).
The stream will also include CBOR [start message](CBOR.md#start-message), [calibration messages](CBOR.md#calibration-message) and [end message](CBOR.md#end-message) with run metadata.
Calibration messages (the JUNGFRAU pedestal and pedestal RMS, with `save_calibration` on) come only from the
FPGA workflow; a DECTRIS stream carries no calibration, so none is forwarded for it.
If `file_prefix` is not provided for a data collection, images won't be sent to image stream (or its HDF5/CBOR replacements).
@@ -239,12 +241,19 @@ For this reason it is also recommended to use the same option on receiver side.
Given PUB socket properties, it is possible to connect multiple viewers to a single socket --- all the viewers should receive all the images sent.
The dark frames of the mask sequence from a DECTRIS detector are no images for a viewer: nothing is sent on
this stream for them, not even a start or end message.
## Metadata stream
Jungfraujoch can also send pure metadata for the purpose of archiving such information.
Metadata are serialized as CBOR [metadata message](CBOR.md#metadata-message).
This is very similar to the image message, but excludes the image array, the spot list and the reflection list.
As metadata are relatively small, to avoid large number of messages, Jungfraujoch bundles metadata of many images in one message.
Order of images within bundle, as well as the size of the bundle, are not guaranteed.
As metadata are relatively small, to avoid large number of messages, Jungfraujoch bundles metadata of many images in one message:
a bundle is cut every `period_ms` of the `zeromq_metadata` configuration and holds the images analysed since the last one.
Order of images within bundle, as well as the size of the bundle, are not guaranteed, and two bundles cut
close together can leave in either order; the end message is always last.
The stream will also include CBOR [start message](CBOR.md#start-message) and [end message](CBOR.md#end-message) with run metadata.
A disabled metadata stream keeps nothing, and the dark frames of the mask sequence from a DECTRIS detector
send nothing on it, as on the preview stream.
This is using PUB socket with watermark, so there is some queuing of messages with ZeroMQ. Multiple receivers can be connected.
docs/JFJOCH_BROKER.md
+79 -20
View File
@@ -214,19 +214,42 @@ rugnux processed the data: by rugnux live mode (below) during the collection, or
`rotation_indexing_angular_stride_deg` and `spot_finding_settings.quick_integration` are deprecated: accepted, ignored,
and returned at their old defaults. `bragg_integration_settings` apply to rugnux live mode.
### The receiver of a run
Each run is taken by one receiver, chosen when it starts from the detector type, the detector mode, the
goniometer and the rugnux live setting:
| Input | Run | Receiver | What it does |
|---|---|---|---|
| FPGA cards (PSI JUNGFRAU, PSI EIGER) | every run | `JFJochReceiverFPGA` | the card's converted pixels, spots, azimuthal and ROI sums into the image; summation on the host above what the card sums; indexing of each image; the calibration to the writer; pedestal runs into the pedestal and its RMS; with rugnux live on, a serial run analysed on the GPU after a rugnux pre-pass (see [Serial collections](#serial-collections-experimental)) |
| DECTRIS stream | dark-mask sequence | `JFJochReceiverDarkMask` | the dark frames into the mask; nothing on any stream |
| DECTRIS stream | rugnux live off, or a rotation run the image buffer cannot hold | `JFJochReceiverLite` | each image on the GPU: masking, azimuthal and ROI integration, spot finding, indexing |
| DECTRIS stream | rugnux live on, rotation that fits the image buffer | `JFJochReceiverRugnuxRotation` | images forwarded as they come; rugnux reads the collection back from the image buffer and runs the whole pipeline |
| DECTRIS stream | rugnux live on, serial | `JFJochReceiverRugnuxStills` | a rugnux pre-pass on the first frames, then every frame analysed and integrated on the GPU and merged live |
`receiver_threads` is the number of image threads. The Lite receiver uses at most 16 of them for a
collection at 100 Hz or slower or of fewer than 256 images (each thread holds a GPU analysis of its
own); the FPGA receiver takes four when it sums images on the host. The GPU FFT indexers and cuFFT
plans the broker's own indexing uses are built once and kept for the life of the process, and the ones
built for earlier indexing settings are released when the settings change. `azim_int.force_cpu`
moves the azimuthal integration of the FPGA workflow from the card to the host, which a profile with
more bins than the card holds needs.
## Rugnux live mode
With rugnux live mode on, rotation collections from a DECTRIS detector are processed by rugnux inside
the broker while they arrive, and serial (stills) collections in the FPGA workflow are analysed on the GPU
with the geometry and mask a rugnux pre-pass decided (see [Serial collections](#serial-collections-experimental)
below). It is off by default; the configuration file switches it on with
With rugnux live mode on, collections from a DECTRIS detector are processed inside the broker while they
arrive: rotation collections by rugnux, and serial (stills) collections on the GPU with the geometry and mask a
rugnux pre-pass decided (see [Serial collections](#serial-collections-experimental) below). In the FPGA workflow
(PSI JUNGFRAU and EIGER) serial collections are analysed on the GPU after a pre-pass as well, with the differences
given there; rotation collections run image by image, as with the mode off. It is off by default; the
configuration file switches it on with
`"rugnux": {"enable": true, "reserved_cpus": 4}`, where `reserved_cpus` is the number of CPU threads
left to the receiver and the image pusher.
For a rotation collection, rugnux reads the images back from the broker's image buffer, so the buffer must hold the whole
collection - one slot per image. Where it holds fewer, the collection is only forwarded to the writer,
as with the mode off; the live status reports it as skipped and the collection's statistics carry a
warning saying how many images the buffer holds against how many the collection has.
collection - one slot per image. Where it holds fewer, the collection gets the per-image analysis of
the mode off (spots, indexing, scan result); the live status reports it as skipped and the collection's
statistics carry a warning saying how many images the buffer holds against how many the collection has.
The results - the merged data, the map coefficients and, with a model, the placed model and its
refinement - travel to the writer in the end message, which writes them beside the master file as
@@ -251,7 +274,8 @@ they prepare the next collection and expose nothing of the current one.
The end message, and with it the end of the collection, waits for the analysis. `deadline_s` (default
60 s; 0 for no cutoff) bounds that wait: this long after the last image the analysis is cut. `POST /cancel` is
the other way to cut it: while images still arrive it cancels the collection as with the mode off,
and the analysis with it; after the last image it ends only the analysis. A cut analysis gives what it
and the analysis with it - the live status then ends with verdict `CANCELLED` and the images that came before
the cancel; after the last image it ends only the analysis. A cut analysis gives what it
has established: a pass that is past its images still merges and makes its outputs, the canonical pass
cut among its images merges those it has read, only the placed model is left out, and
any other pass cut short gives way to the last one that ran to its end. The end message goes out at
@@ -277,9 +301,9 @@ estimate and background of every image in `GET /result/scan` and the master file
### Serial collections (experimental)
Live processing of serial collections is **experimental**. With rugnux live mode on, a serial
(non-rotation) collection in the FPGA workflow, in the standard detector mode, is analysed as frames from a
DECTRIS detector are: the converted frames go to GPU spot finding, indexing and integration, with rugnux's
stills spot finder (adaptive threshold, single-pixel spots kept). Before any frame is analysed, the first
(non-rotation) collection from a DECTRIS detector, in the standard detector mode, is analysed frame by frame: the
frames as the stream delivers them go to GPU spot finding, indexing and integration, with rugnux's stills spot
finder (adaptive threshold, single-pixel spots kept). Before any frame is analysed, the first
500 frames are run through rugnux, in memory, as `rugnux --no-merge` would run on them. That pre-pass
decides the beam centre, refines the detector distance (where the dataset settings give a unit cell) and
masks the beam-stop shadow and defective pixels; every frame of the collection is then analysed with that
@@ -288,14 +312,15 @@ the centre and distance used (`refined_beam_center_x/y`, `refined_detector_dista
[CBOR](CBOR.md)). The broker's own distance is taken as the beamline's: unlike offline rugnux, the pre-pass
does not search the whole distance range. It refines the centre and distance for a detector square to the
beam, as the frames are then analysed. At a serial hit rate the distance needs the 500 frames to be refined at
all: on fewer, too few index for the geometry refinement to run, and the stated distance is kept. The image
buffer has to hold them, and the frames that arrive while the pre-pass runs, in half its slots (for a JUNGFRAU
8M, a slot is the size of an uncompressed frame); a smaller buffer starts the pre-pass on as many as it holds.
all: on fewer, too few index for the geometry refinement to run, and the stated distance is kept. The pre-pass
holds and fits on at most half the slots of the image buffer (a slot is the size of an uncompressed
frame), so a smaller buffer starts it on as many as that.
Where a unit cell is given and those frames are too few for the geometry fit (200 indexed frames), frames go on
being held and the pre-pass runs again on all of them each time they have doubled, until the fit has its frames, half
the image buffer is full, or the collection ends. The analysis starts when the pre-pass ends and catches up from the
frames held, so it starts later on a low hit rate; a frame finding the buffer full meanwhile is dropped as usual.
the image buffer or 2000 frames are held, or the collection ends; frames arriving during a run do not make the next
one larger than that, so the last run stays short. The analysis starts when the pre-pass ends and catches up from the
frames waiting, so it starts later on a low hit rate.
The geometry is fixed when the pre-pass ends and does not change during the collection, so every frame is analysed
and merged at it and the radial profile has one binning throughout. Where the pre-pass could not refine it (too few
@@ -303,9 +328,10 @@ of its frames indexed, as with a small image buffer, a low hit rate or a chip wh
cell given), the distance in the header is used for the whole run: this is a warning in the log and a note of the
live status, and a wrong header distance then costs resolution and indexed frames.
Frames arriving during the pre-pass, or faster than the analysis, wait in the image buffer, in at most half of
its slots; a frame that finds no free slot is dropped and counted as for a full image buffer, so analysis
slower than the frame rate costs frames, not memory. A pre-pass that fails leaves the stated geometry and
Frames arriving during the pre-pass, or faster than the analysis, wait in the image buffer, in all its slots but
one per analysis thread, kept for sending; a frame that finds it full is dropped and counted as for a full image
buffer, so analysis slower than the frame rate costs frames, not memory. Where the writer lags behind, the analysis
waits for slots to come back. A pre-pass that fails leaves the stated geometry and
mask in use. Where the start message cannot be sent after the pre-pass (no writer listening) or the analysis
cannot be set up, the collection ends with that error, reported by `/status` and `/wait_till_done`, and the
next `/start` runs as usual.
@@ -323,11 +349,44 @@ JUNGFRAU 8M, against about half a minute for the same merge made from nothing. T
writer in the end message, as for rotation (`<file_prefix>.mtz`), and the run's step "Merge" carries the
final statistics.
`GET /rugnux/status` reports such a run as three steps (pre-pass, GPU analysis, merge) and a `stills` block: the
With an uploaded model, the model step runs on every one of these merges, as `rugnux --mode scale --model`
places a model and `rugnux --mode refine` refines it: the model is placed by its rigid body where it fits the
data as given - its indexing ambiguity probed by R-free, and the merge put in the model's indexing - and by
molecular replacement where it does not (or is a prediction); the 2mFo-DFc and mFo-DFc maps are made, and the
placed model is refined, with the blobs of its difference map. Without a space group in the dataset settings
the frames are merged in the model's own, where the model's cell is the consensus cell of the run on the same
axes; otherwise the model is left out, with a warning in the live status saying why. The step on a provisional
merge runs on a thread of its own, on 8 CPU threads and the GPU, beside the analysis and the merge, which do not
wait for it: its result - the placed and the refined model, their maps, the scores - goes to the live status
(`model`, `/rugnux/model.cif`, `/rugnux/map.ccp4`) as soon as it is made, so the density can be looked at while
the collection goes on. A step still running when the next merge is made is cut (a refinement cut short keeps
the weight trials it finished), and each step starts from the placement of the one before: the merge is put in
the same indexing, and the model goes in where it was placed, with no null, indexing probe or search made again.
After the last frame the step on the merge of the collection runs on every thread, and the end message waits
for it until `deadline_s` after the last image; then it is cut as for rotation - the refinement first (cut
short, or left out where it has not started), and a molecular replacement that has not started. The end message
carries the merge in the model's indexing (`<file_prefix>.mtz`), the map coefficients, the placed and the
refined model, the refined maps and the blobs, written as for rotation; one that would not fit the pushers'
buffer goes without them. The GPU memory the analysis's indexers hold for the broker's life is counted against
the card when the model step decides what goes on it. `POST /cancel` cuts it
the same way, at any time (`cut` in `/rugnux/status`): the frames that came are still analysed and merged.
The frames wait in the image buffer compressed, and go on to the writer byte for byte, with their analysis. `efficiency` is the
images received over the images expected; a stream has no module packets, so per-image collection
efficiency, ADU histograms and pulse IDs are absent. The dark frames of the mask sequence stay outside this mode.
`GET /rugnux/status` reports such a run as three steps (pre-pass, GPU analysis, merge; with a model, the steps of
the model on the merge of the collection after them) and a `stills` block: the
pre-pass state and duration, the stated and decided beam centre and distance, the pixels masked, and the
images analysed and indexed with the analysis rate. The "Rugnux live" page of the web frontend shows the
same. `tools/jfjoch_stills_gpu_replay` replays a stored serial collection through the same analysis.
In the FPGA workflow (JUNGFRAU), a serial collection with rugnux live mode on is analysed by the receiver of the
card (`JFJochReceiverFPGA`) with the pre-pass, the GPU analysis and the live merge described above, with three
differences: the converted frames wait for the analysis in at most half of the image buffer's slots, a frame that
finds them taken is dropped, and there is no model step - an uploaded model is not placed or refined on these
merges. The end message carries the merged data, as for a DECTRIS stream.
## Setting up a local test for Jungfraujoch
For development, it is possible to set up a local installation of Jungfraujoch.
This will work without FPGA installed in the computer and allows testing the Jungfraujoch software layer, including