docs: receivers, streams and tests as they are at rc.184

JFJOCH_BROKER: the receiver each run gets and what it does (pedestal, FPGA,
dark mask, Lite, rugnux rotation, rugnux stills), the thread rules, the kept
FFT indexers, azim_int.force_cpu and the internal generator's capacity check;
a rotation run the image buffer cannot hold gets the per-image analysis, not
a bare forward; a cancelled live run ends CANCELLED. IMAGE_STREAM: calibration
messages only from the FPGA workflow; pedestal and dark-mask runs send nothing
on the preview and metadata streams; a disabled metadata stream keeps nothing;
how bundles are cut and ordered. TESTS: the mock of the card, [HLSvsMock], and
that the card's arithmetic is no longer tested.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK

docs/IMAGE_STREAM.md

@@ -11,6 +11,8 @@ These are:
 ## Image stream
 Images (with metadata) are serialized as CBOR [image message](CBOR.md#image-message).
 The stream will also include CBOR [start message](CBOR.md#start-message), [calibration messages](CBOR.md#calibration-message) and [end message](CBOR.md#end-message) with run metadata.
+Calibration messages (the JUNGFRAU pedestal and pedestal RMS, with `save_calibration` on) come only from the
+FPGA workflow; a DECTRIS stream carries no calibration, so none is forwarded for it.
 
 If `file_prefix` is not provided for a data collection, images won't be sent to image stream (or its HDF5/CBOR replacements).
 
@@ -236,12 +238,19 @@ For this reason it is also recommended to use the same option on receiver side.
 
 Given PUB socket properties, it is possible to connect multiple viewers to a single socket --- all the viewers should receive all the images sent.
 
+Pedestal runs and the dark frames of the mask sequence are no images for a viewer: nothing is sent on
+this stream for them, not even a start or end message.
+
 ## Metadata stream
 Jungfraujoch can also send pure metadata for the purpose of archiving such information.
 Metadata are serialized as CBOR [metadata message](CBOR.md#metadata-message).
-This is very similar to the image message, but excludes the actual image array and spot positions.
+This is very similar to the image message, but excludes the actual image array, the spot positions and the reflections.
-As metadata are relatively small, to avoid large number of messages, Jungfraujoch bundles metadata of many images in one message.
+As metadata are relatively small, to avoid large number of messages, Jungfraujoch bundles metadata of many images in one message:
-Order of images within bundle, as well as the size of the bundle, are not guaranteed.
+a bundle is cut every `period_ms` of the `zeromq_metadata` configuration and holds the images analysed since the last one.
+Order of images within bundle, as well as the size of the bundle, are not guaranteed, and two bundles cut
+close together can leave in either order; the end message is always last.
 The stream will also include CBOR [start message](CBOR.md#start-message) and [end message](CBOR.md#end-message) with run metadata.
+A disabled metadata stream keeps nothing, and pedestal runs and the dark frames of the mask sequence send
+nothing on it, as on the preview stream.
 
 This is using PUB socket with watermark, so there is some queuing of messages with ZeroMQ. Multiple receivers can be connected.

docs/JFJOCH_BROKER.md

@@ -200,6 +200,31 @@ rugnux processed the data: by rugnux live mode (below) during the collection, or
 `rotation_indexing_angular_stride_deg` and `spot_finding_settings.quick_integration` are deprecated: accepted, ignored,
 and returned at their old defaults. `bragg_integration_settings` apply to rugnux live mode.
 
+### The receiver of a run
+
+Each run is taken by one receiver, chosen when it starts from the detector type, the detector mode, the
+goniometer and the rugnux live setting:
+
+| Input | Run | Receiver | What it does |
+|---|---|---|---|
+| FPGA cards (PSI JUNGFRAU, PSI EIGER) | pedestal (G0, G1, G2) | `JFJochReceiverPedestal` | raw frames into the pedestal and its RMS; no images, nothing on any stream |
+| FPGA cards | standard run, dark-mask sequence | `JFJochReceiverFPGA` | the card's converted pixels, spots, azimuthal and ROI sums into the image; summation on the host above what the card sums; indexing of each image; the calibration to the writer |
+| DECTRIS stream | dark-mask sequence | `JFJochReceiverDarkMask` | the dark frames into the mask; nothing on any stream |
+| DECTRIS stream | rugnux live off, or a rotation run the image buffer cannot hold | `JFJochReceiverLite` | each image on the GPU: masking, azimuthal and ROI integration, spot finding, indexing |
+| DECTRIS stream | rugnux live on, rotation that fits the image buffer | `JFJochReceiverRugnuxRotation` | images forwarded as they come; rugnux reads the collection back from the image buffer and runs the whole pipeline |
+| DECTRIS stream | rugnux live on, serial | `JFJochReceiverRugnuxStills` | a rugnux pre-pass on the first frames, then every frame analysed and integrated on the GPU and merged live |
+
+`receiver_threads` is the number of image threads. The Lite receiver uses at most 16 of them for a
+collection at 100 Hz or slower or of fewer than 256 images (each thread holds a GPU analysis of its
+own); the FPGA receiver takes four when it sums images on the host. The GPU FFT indexers and cuFFT
+plans the broker's own indexing uses are built once and kept for the life of the process, and the ones
+built for earlier indexing settings are released when the settings change. `azim_int.force_cpu`
+moves the azimuthal integration of the FPGA workflow from the card to the host, which a profile with
+more bins than the card holds needs. A run on the card's internal frame generator
+(`detector_settings.internal_frame_generator`) is refused at start when its modules times images do
+not fit the card's memory; the error says how many module frames fit (a 32-bit EIGER frame counts
+twice).
+
 ## Rugnux live mode
 
 With rugnux live mode on, collections from a DECTRIS detector are processed inside the broker while they
@@ -211,9 +236,9 @@ rugnux pre-pass decided (see [Serial collections](#serial-collections-experiment
 left to the receiver and the image pusher.
 
 For a rotation collection, rugnux reads the images back from the broker's image buffer, so the buffer must hold the whole
-collection - one slot per image. Where it holds fewer, the collection is only forwarded to the writer,
+collection - one slot per image. Where it holds fewer, the collection gets the per-image analysis of
-as with the mode off; the live status reports it as skipped and the collection's statistics carry a
+the mode off (spots, indexing, scan result); the live status reports it as skipped and the collection's
-warning saying how many images the buffer holds against how many the collection has.
+statistics carry a warning saying how many images the buffer holds against how many the collection has.
 
 The results - the merged data, the map coefficients and, with a model, the placed model and its
 refinement - travel to the writer in the end message, which writes them beside the master file as
@@ -238,7 +263,8 @@ they prepare the next collection and expose nothing of the current one.
 The end message, and with it the end of the collection, waits for the analysis. `deadline_s` (default
 60 s; 0 for no cutoff) bounds that wait: this long after the last image the analysis is cut. `POST /cancel` is
 the other way to cut it: while images still arrive it cancels the collection as with the mode off,
-and the analysis with it; after the last image it ends only the analysis. A cut analysis gives what it
+and the analysis with it - the live status then ends with verdict `CANCELLED` and the images that came before
+the cancel; after the last image it ends only the analysis. A cut analysis gives what it
 has established: a pass that is past its images still merges and makes its outputs, the canonical pass
 cut among its images merges those it has read, only the placed model is left out, and
 any other pass cut short gives way to the last one that ran to its end. The end message goes out at

docs/TESTS.md

@@ -11,6 +11,16 @@ cd tests
 ./jfjoch_test "[tag]"         # by tag
 ```
 
+The tests of the receivers, the pullers, the state machine and the writer run on a mock of the
+FPGA card (`acquisition_device/MockAcquisitionDevice`): it computes nothing, the test gives it what
+the card would write (`tests/MockCardOutput.h`, calculated on the CPU), and it reproduces the card's
+host contract - packets, module completion, summation groups, cancel and the end of a collection.
+Whether the mock gives the host what the card gives is checked in `MockAcquisitionDeviceTest.cpp`,
+tagged `[HLSvsMock]`: a scenario runs on the high-level-synthesis C model of the firmware
+(`HLSSimulatedDevice`), its output is given to the mock, and everything the host sees has to be the
+same. The card's own arithmetic (conversion, gain modes, summation, spot finder, integration, ROI) is
+no longer tested by `jfjoch_test`: the firmware is frozen.
+
 There are also benchmark and hardware routines, each printing its own usage:
 
 * `jfjoch_hdf5_test` to measure HDF5 dataset writing speed (single threaded). It doubles as the