From ce78662e975978b7929cf3fc3ae5dced6b29b155 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Tue, 10 Nov 2026 12:53:21 +0100 Subject: [PATCH] Docs: broker example config starts with the current schema; TCP stream protocol as implemented The JFJOCH_BROKER.md example now parses and starts: one geometry per detector, EIGER trim energies alongside the trim files, readout_time_ns, the required azim_int and image_format keys, the pedestal RMS limit under image_format where it is read. module_placement entries are described with all eight required fields. IMAGE_STREAM.md gives TCP protocol version 5, the FILE_CLOSED ack, the 64-bit FIFO occupancy, the liveness-based END wait and the two optional tcp timeouts; the metadata stream also drops reflections. SECURITY.md and the broker page list the bearer-protected /rugnux/model.cif and /rugnux/map.ccp4. The etc/ examples drop the deprecated numa_policy and a misplaced pedestal RMS limit. Co-Authored-By: Claude Fable Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK --- docs/IMAGE_STREAM.md | 13 ++++++----- docs/JFJOCH_BROKER.md | 51 +++++++++++++++++++++++++------------------ docs/SECURITY.md | 1 + etc/broker_crmx.json | 2 -- etc/broker_eiger.json | 1 - etc/broker_local.json | 1 - 6 files changed, 39 insertions(+), 30 deletions(-) diff --git a/docs/IMAGE_STREAM.md b/docs/IMAGE_STREAM.md index c9c9e53c8..1cddb6ef5 100644 --- a/docs/IMAGE_STREAM.md +++ b/docs/IMAGE_STREAM.md @@ -70,7 +70,7 @@ If a `START` ACK fails on any connection, Jungfraujoch sends `CANCEL` to all alr For each frame: 1. Read one `TcpFrameHeader` (fixed size, 64-byte aligned). -2. Validate `magic` (`0x4A464A54` / `"JFJT"`) and `version` (`4`). Both ends reject a frame of any other version, so writer and broker must be of the same release. +2. Validate `magic` (`0x4A464A54` / `"JFJT"`) and `version` (`5`). Both ends reject a frame of any other version, so writer and broker must be of the same release. 3. Read `payload_size` bytes (if non-zero). #### Pre-flight @@ -102,13 +102,15 @@ TCP/IP image stream is configured in the broker JSON configuration file under th - `image_socket`: listen address in `tcp://:` format. `0.0.0.0` binds on all interfaces. `*` as port selects a random free port. - `nwriters` (optional): maximum number of simultaneous writer connections accepted. - `send_buffer_size` (optional): OS-level `SO_SNDBUF` size for accepted connections. +- `peer_liveness_timeout_ms` (optional, default 15000): how long a send may block with no sign of life from the writer (no `BUSY` heartbeat, no ACK) before the connection is declared dead. +- `max_backpressure_timeout_ms` (optional, default 60000): hard bound on backpressure - if no bytes can be sent for this long the connection is declared dead even while the writer keeps heartbeating. #### ACK handling ACK handling is mandatory for correct operation: - `PREFLIGHT` **must** be acknowledged (`ack_for=PREFLIGHT`) on each connection within 5 seconds, otherwise the collection is not started. A rejected pre-flight (`OK` clear) carries the reason as error text and does not break the connection. - `START` **must** be acknowledged (`ACK` with `ack_for=START`) on each connection within 5 seconds, otherwise collection start fails and a rollback is triggered. -- `END` **must** be acknowledged (`ack_for=END`) on each connection within 10 seconds for successful completion. +- `END` **must** be acknowledged (`ack_for=END`) on each connection for successful completion. There is no fixed deadline: the writer may still be flushing a backlog, so the broker waits as long as the writer shows signs of life (`BUSY` heartbeats, `DATA` ACKs) and gives up only after `peer_liveness_timeout_ms` (default 15 s) of silence. - `CANCEL` should be acknowledged during rollback paths (500ms timeout). - `DATA` should be acknowledged for every frame. A `DATA` ACK with `FATAL` flag set reports a downstream error (e.g. disk full) which is propagated to `jfjoch_broker` via `Finalize()`. A failed `DATA` ACK does **not** break the TCP connection on its own — data continues to flow. - `CALIBRATION` is not acknowledged at this time. @@ -135,13 +137,14 @@ On Linux, large payload transmission (`DATA` and `CALIBRATION` frames) can use k | 7 | `KEEPALIVE` | Connection liveness probe/pong | | 8 | `BUSY` | Writer alive but stalled; carries its FIFO occupancy | | 9 | `PREFLIGHT` | Dry run before a collection starts: can this run be written? | +| 10 | `FILE_CLOSED` | Not a frame of its own: the value of `ack_for` in a writer `ACK` reporting that a data file was closed and renamed into place (`image_number` = file number from 0, `ack_processed_images` = one past the highest image number it holds). The broker uses it to tell a reader following the collection which files it can read. | #### TCP frame header (`TcpFrameHeader`) | Field | Type | Description | |--------------------------|---|----------------------------------------------------------| | `magic` | `uint32_t` | Protocol magic (`0x4A464A54`, `"JFJT"`) | -| `version` | `uint16_t` | Protocol version (`4`) | +| `version` | `uint16_t` | Protocol version (`5`) | | `type` | `uint16_t` | Frame type (see table above) | | `image_number` | `uint64_t` | Image index for `DATA` frames | | `payload_size` | `uint64_t` | Number of payload bytes after header | @@ -151,7 +154,7 @@ On Linux, large payload transmission (`DATA` and `CALIBRATION` frames) can use k | `ack_processed_images` | `uint32_t` | In `ACK`: number of images processed by receiver | | `ack_code` | `uint16_t` | In `ACK`: error/status code | | `ack_for` | `uint16_t` | In `ACK`: frame type being acknowledged | -| `ack_fifo_occupancy` | `uint16_t` | In `ACK`: occupancy of input FIFO in the `jfjoch_writer` | +| `ack_fifo_occupancy` | `uint64_t` | In `ACK`: occupancy of input FIFO in the `jfjoch_writer` | | `ack_fifo_max_occupancy` | `uint64_t` | In `ACK`: max occupancy of input FIFO | The header is 64-byte aligned (`alignas(64)`). @@ -239,7 +242,7 @@ Given PUB socket properties, it is possible to connect multiple viewers to a sin ## Metadata stream Jungfraujoch can also send pure metadata for the purpose of archiving such information. Metadata are serialized as CBOR [metadata message](CBOR.md#metadata-message). -This is very similar to the image message, but excludes the actual image array and spot positions. +This is very similar to the image message, but excludes the image array, the spot list and the reflection list. As metadata are relatively small, to avoid large number of messages, Jungfraujoch bundles metadata of many images in one message. Order of images within bundle, as well as the size of the bundle, are not guaranteed. The stream will also include CBOR [start message](CBOR.md#start-message) and [end message](CBOR.md#end-message) with run metadata. diff --git a/docs/JFJOCH_BROKER.md b/docs/JFJOCH_BROKER.md index d9e7e7fb1..93441be87 100644 --- a/docs/JFJOCH_BROKER.md +++ b/docs/JFJOCH_BROKER.md @@ -27,10 +27,12 @@ and answer `401` otherwise; `/statistics` omits its `measurement` block instead. `/start` also takes an optional `module_placement`: where the JUNGFRAU modules lie, as `rugnux` measures them on an earlier run. rugnux writes it as `_module_placement.json`, a fragment of the -dataset settings (`{"module_placement": [...]}`) to merge into the `/start` body. Each entry names its -module as `custom_geometry` does - `x0`, `y0` (the image position of its readout pixel 0) and -`fast_axis`, `slow_axis` - but in the image as stored, after `mirror_y` (see below), -so the order does not matter, and a list that does not name exactly the detector's modules (one left out, +dataset settings (`{"module_placement": [...]}`) to merge into the `/start` body. Each entry has eight +required fields: it names its module as `custom_geometry` does - `x0`, `y0` (the image position of its +readout pixel 0) and `fast_axis`, `slow_axis` - but in the image as stored, after `mirror_y` (see below), +and gives where the module lies on the detector plane (`x0_plane`, `y0_plane`, `fast_step`, `slow_step`). +The first four identify the module, so the order does not matter, and a list that does not name exactly +the detector's modules (one left out, named twice, or not the detector's) is refused with `400`. The module count and the assembly are the configuration's (`custom_geometry` below); the placement only moves the modules relative to one another, and goes into the start message, the master file's `module_NN` groups and the broker's analysis. See @@ -47,7 +49,9 @@ on the two differ by the mirror: for an image of height H, `y0` (module_placemen (custom_geometry), `x0` is the same, and `Yp` and `Yn` swap in `fast_axis`/`slow_axis`. With `mirror_y` off they are equal. -Example configuration (not every section is shown): +Example configuration (`etc/broker_local.json`, `broker_eiger.json` and `broker_crmx.json` are +complete, runnable configurations; a detector takes either `standard_geometry` or `custom_geometry`, +never both): ```json { @@ -87,24 +91,22 @@ Example configuration (not every section is shown): "module_sync": true, "sensor_thickness_um": 320, "calibration_file": [ - "gainMaps.bin" + "/opt/jfjoch/eiger/trim/e1m-01.sn001", + "/opt/jfjoch/eiger/trim/e1m-02.sn002" + ], + "trim_energies_eV": [ + 12400 ], "hostname": [ "e1m-01", "e1m-02" ], - "readout_time_us": 3, + "readout_time_ns": 3000, "sensor_material": "Si", "tx_delay": [ 0,1 ], "base_data_ipv4_address": "10.10.10.50", - "standard_geometry": { - "nmodules": 1, - "gap_x": 8, - "gap_y": 36, - "modules_in_row": 1 - }, "custom_geometry": [ { "x0": 0, @@ -127,7 +129,6 @@ Example configuration (not every section is shown): "jungfrau_pedestal_g0_frames": 2000, "jungfrau_pedestal_g1_frames": 300, "jungfrau_pedestal_g2_frames": 300, - "jungfrau_pedestal_g0_rms_limit": 100, "jungfrau_pedestal_min_image_count": 128, "jungfrau_storage_cell_count": 1, "jungfrau_storage_cell_delay_ns": 5000, @@ -135,11 +136,10 @@ Example configuration (not every section is shown): "jungfrau_use_gain_hg0": false }, "azim_int": { - "polarization_factor": -1, + "polarization_corr": true, "solid_angle_corr": true, - "high_q_recipA": 0, - "low_q_recipA": 0, - "q_spacing": 0 + "low_q_recipA": 0.01, + "q_spacing": 0.002 }, "image_format": { "summation": true, @@ -149,7 +149,9 @@ Example configuration (not every section is shown): "bit_depth_image": 16, "signed_output": true, "mask_module_edges": true, - "mask_chip_edges": true + "mask_chip_edges": true, + "apply_mask": true, + "jungfrau_pedestal_g0_rms_limit": 100 }, "image_buffer_MiB": 2048, "receiver_threads": 64, @@ -168,6 +170,13 @@ Example configuration (not every section is shown): } ``` +`standard_geometry` (`nmodules`, `modules_in_row`, `gap_x`, `gap_y`) is the alternative for a regular +module grid. For an EIGER `hostname` lists two entries per module (one per half-module) and +`calibration_file` is either a directory of trim files or one trim file per half-module, with the energies +they were trimmed at in `trim_energies_eV`; for a JUNGFRAU both are one entry per module, the +calibration being the gain maps. The polarization factor of the azimuthal integration is a dataset +setting (`/config/dataset`, `polarization_factor`), not part of `azim_int`. + ## Following a collection A reader can process a collection while it is being written - rugnux does, given the path the master @@ -255,7 +264,7 @@ R-free, the bond and angle rmsd, the time it took and the unexplained blobs of t state, the images received, the step it is on and how far into it, the steps so far with what each found, and once it is over the verdict, the summary, warnings and the time from the last image to every result written. The web frontend shows it on the dashboard and in its "Rugnux live" tab, which -also changes the settings and uploads the model. It needs the dataset's bearer token +also changes the settings and uploads the model. It, `/rugnux/model.cif` and `/rugnux/map.ccp4` need the dataset's bearer token like the dataset routes (`/statistics/data_collection`, `/image_buffer/*`, `/preview/plot*`). `GET /result/scan` carries the rotation lattice, unit cell and Bravais symbol rugnux found. A collection in which rugnux finds no lattice still gets the spot count, resolution estimate and background of every image in `GET /result/scan` and the master file, marked as not indexed. @@ -338,7 +347,7 @@ The `jfjoch` package alone is enough. In this case - it is necessary to update `etc/broker_local.json` file with frontend path in `/usr/share/jfjoch/frontend`. ### Start service -Start broker: +Start broker (the HTTP port is optional; the default is 5232, listening on all interfaces): ``` cd build/broker ./jfjoch_broker ../../etc/broker_local.json 5232 diff --git a/docs/SECURITY.md b/docs/SECURITY.md index ee3ace785..ab38f1501 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -113,6 +113,7 @@ the record of its collection - so the broker is left as open and as empty as a f | `/preview/plot`, `/preview/plot.bin` (per-image plots, unit cell) | 401 without a token | | `/live/events` (the start message and the data files of the collection being written) | 401 without a token | | `/rugnux/status` (live rugnux run of the collection: cell, space group, results) | 401 without a token | +| `/rugnux/model.cif`, `/rugnux/map.ccp4` (the model placed by the live run and its maps) | 401 without a token | | `/statistics` (the aggregate the web UI polls) | 200, but the `measurement` block is omitted | | everything else (`/status`, `/config/*`, `/start`, `/cancel`, masks, pedestal, ...) | open | diff --git a/etc/broker_crmx.json b/etc/broker_crmx.json index adb5cc779..3a4bca782 100644 --- a/etc/broker_crmx.json +++ b/etc/broker_crmx.json @@ -79,7 +79,6 @@ ], "frontend_directory": "/usr/share/jfjoch/frontend/", "image_pusher": "ZeroMQ", - "numa_policy": "n2g2", "receiver_threads": 64, "image_buffer_MiB": 4096, "pcie": [ @@ -94,7 +93,6 @@ "jungfrau_pedestal_g0_frames": 2000, "jungfrau_pedestal_g1_frames": 300, "jungfrau_pedestal_g2_frames": 300, - "jungfrau_pedestal_g0_rms_limit": 100, "jungfrau_pedestal_min_image_count": 128, "jungfrau_storage_cell_count": 1, "jungfrau_storage_cell_delay_ns": 2100, diff --git a/etc/broker_eiger.json b/etc/broker_eiger.json index 1db1a1b7c..88f8886c5 100644 --- a/etc/broker_eiger.json +++ b/etc/broker_eiger.json @@ -68,7 +68,6 @@ ], "frontend_directory": "/usr/share/jfjoch/frontend/", "image_pusher": "ZeroMQ", - "numa_policy": "n2g2", "receiver_threads": 64, "image_buffer_MiB": 4096, "pcie": [ diff --git a/etc/broker_local.json b/etc/broker_local.json index a67311c51..ecd4804cc 100644 --- a/etc/broker_local.json +++ b/etc/broker_local.json @@ -88,7 +88,6 @@ "jungfrau_pedestal_g0_frames": 0, "jungfrau_pedestal_g1_frames": 0, "jungfrau_pedestal_g2_frames": 0, - "jungfrau_pedestal_g0_rms_limit": 100, "jungfrau_pedestal_min_image_count": 128, "jungfrau_storage_cell_count": 1, "jungfrau_storage_cell_delay_ns": 2100,