So that rugnux can process a collection while it is written, the broker reports each data file once
a writer has closed it and renamed it into place:
- Writer: FileWriter takes a callback, called for every closed data file. StreamWriter uses it on TCP
to send an acknowledgement with ack_for = FILE_CLOSED (new TCPFrameType 10): file number and the
images the file holds. TCP protocol version 5. Files closed at END are reported before the END
acknowledgement.
- LiveCollection (image_pusher): the start message the writer of the master file received, the
closed files, the end. Fed by the TCP pusher (FILE_CLOSED acknowledgements; end once every writer
acknowledged END) and by the in-process HDF5 pusher (the FileWriter callback). The ZeroMQ pusher
has no back channel and offers none.
- Two routes, protected by the dataset's bearer tokens like the other dataset routes:
- GET /live/start.cbor: that start message;
- GET /live/events: server-sent events (collection, file, end, superseded) with the history first
and a keepalive every 10 s. One follower at a time; a second gets 409.
Documented in jfjoch_api.yaml; they add no schema, so the generated C++ model is unchanged.
- The HTTP server's thread pool is set explicitly (16, up to 64): an event stream holds a thread for
the whole collection.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
leonarski_fandClaude Opus 5.5
509a54ef92