mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 67 Wiki Activity

TCPStreamPusher: post-zerocopy cleanup + fix queue-path backpressure drop

Browse Source 
Follow-up simplifications after removing the zerocopy machinery, plus a real
backpressure bug the cleanup surfaced:

- SendImage(ZeroCopyReturnValue&) imposed a hard 2s deadline on enqueueing and
  then marked the connection broken. At high frame rate the 128-deep queue
  fills in tens of ms, so any filesystem stall longer than ~2s dropped the run
  even though the writer was alive and heartbeating -- defeating the whole
  BUSY-heartbeat backpressure design. Block instead while the peer is alive
  (!broken && active); the real liveness decision already lives in SendAll's
  peer-liveness timeout, which the writer's BUSY heartbeats keep fresh. This
  makes the queue path consistent with the send path: both wait out arbitrarily
  long stalls and only give up when the peer goes genuinely silent.
- Drop the dead per-connection data_sent counter (written, never read) and the
  redundant ImagePusherQueueElement.image_data set on the TCP path (only the
  HDF5 pusher reads that field).
- Add SetPeerLivenessTimeout() so the liveness window is tunable (and testable).

Add TCPImageCommTest_StalledWriter_SurvivesViaHeartbeat: a controllable raw
writer double connects, ACKs START, then stops draining for 4s while still
sending BUSY heartbeats (peer-liveness window set to 2s). The run must ride out
the stall on the zero-copy queue path and deliver all 1000 images. Verified to
fail (115/1000 delivered, connection dropped) against the old 2s-deadline
behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
leonarski_f
2026-06-25 15:55:49 +02:00
co-authored by Claude Opus 4.8
parent f859f8108f
commit 2a9fd084ab
3 changed files with 241 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
image_pusher/TCPStreamPusher.cpp
+3 -7
View File
@@ -266,9 +266,6 @@ bool TCPStreamPusher::SendFrame(Connection& c, const uint8_t* data, size_t size,
if (size > 0 && !SendAll(c, data, size))
return false;
if (type == TCPFrameType::DATA)
c.data_sent.fetch_add(1, std::memory_order_relaxed);
return true;
}
@@ -661,7 +658,6 @@ void TCPStreamPusher::StartDataCollectionThreads(Connection& c) {
c.data_ack_error_text.clear();
}
c.data_sent.store(0, std::memory_order_relaxed);
c.data_acked_ok.store(0, std::memory_order_relaxed);
c.data_acked_bad.store(0, std::memory_order_relaxed);
c.data_acked_total.store(0, std::memory_order_relaxed);
@@ -887,7 +883,7 @@ bool TCPStreamPusher::SendImage(ZeroCopyReturnValue &z) {
target = use[idx];
}
if (!target || target->broken || !target->active) {
if (!target) {
z.release();
return false;
}
@@ -896,10 +892,10 @@ bool TCPStreamPusher::SendImage(ZeroCopyReturnValue &z) {
// freshly-created file at startup). Keep waiting as long as it proves it is alive —
// BUSY heartbeats / ACKs refresh last_peer_activity_ns from a thread independent of
// its stalled write path — so a slow-but-healthy writer is held back, not dropped.
// Only a peer that has gone completely silent for the liveness window is declared dead.
// Only a peer that has gone completely silent for the liveness window is declared dead;
// a writer that heartbeats but never drains is caught by SendAll's max_backpressure cap.
while (!target->broken && target->active) {
if (target->queue.PutTimeout(ImagePusherQueueElement{
.image_data = static_cast<uint8_t *>(z.GetImage()),
.z = &z,
.end = false
}, std::chrono::milliseconds(50)))