mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 66 Wiki Activity

writer: detect overwrite conflict at start for back-channel transports

Browse Source 
Previously an existing output file was only detected at the final rename,
after the whole collection had run. Add an up-front check in the FileWriter
constructor that fails before acquisition arms.

Only the master file is checked, and only by the single writer that owns it
(write_master_file). In a multi-writer TCP/ZMQ setup the per-image data files
are staggered across writers by file number, and a writer does not even know
the total writer count, so it cannot enumerate its own subset; checking all
data files in every writer would make each writer stat files it never writes
and race the writers already creating them. Data-file conflicts stay caught by
their owning writer at the final rename.

Gate the check on whether the transport can report the failure to the broker:
the direct HDF5 pusher throws in-process and the TCP writer returns a
START-failure ACK, so both abort the collection cleanly (verified: an asymmetric
rejection cancels the already-started sibling). The ZeroMQ path is fire-and-forget
with no back-channel, so StreamWriter opts out via image_puller.SupportsAck() and
keeps writing .tmp files, failing only at the final rename (leaving the images on
disk). Documented in docs/JFJOCH_WRITER.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
leonarski_f
2026-07-09 15:34:37 +02:00
co-authored by Claude Fable 5
parent 36be60774f
commit ad73b4af99
6 changed files with 199 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/HDF5WritingTest.cpp
+52 -15
View File
@@ -4,6 +4,7 @@
#include <catch2/catch_all.hpp>
#include <iostream>
#include <fstream>
#include <filesystem>
#include "../common/DiffractionExperiment.h"
#include "../writer/HDF5Objects.h"
@@ -1445,37 +1446,73 @@ TEST_CASE("HDF5Objects_VDS_reverse_strided", "[HDF5][Unit]") {
remove("scratch_vds_reverse_strided.h5");
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
}
// Reproduces the "pusher stuck after overwrite failure" bug: if EndDataCollection
// throws while finalizing (here: master file already exists, overwrite off), the
// pusher must still tear down its writer so a subsequent collection can start.
TEST_CASE("HDF5FilePusher_overwrite_failure_recovers", "[HDF5FilePusher][Repro]") {
// Overwrite is detected up front for back-channel transports (default) by checking
// the master file only - never the staggered per-writer data files. The ZeroMQ path
// (no back-channel) must opt out and keep writing .tmp files instead.
TEST_CASE("FileWriter_overwrite_detected_at_start", "[HDF5][Overwrite]") {
RegisterHDF5Filter();
DiffractionExperiment x(DetJF4M());
x.FilePrefix("pusher_overwrite_repro").ImagesPerTrigger(1)
x.FilePrefix("fw_overwrite_start").ImagesPerTrigger(3).ImagesPerFile(2)
.Compression(CompressionAlgorithm::NO_COMPRESSION)
.SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(false);
StartMessage start_message;
x.FillMessage(start_message);
REQUIRE(start_message.write_master_file.value_or(false)); // this writer owns the master
// A stray data file (owned by another, staggered writer) must NOT trip the check -
// only the master file is inspected. The temporary writer cleans up its own tmp.
{ std::ofstream(HDF5Metadata::DataFileName(start_message, 0)) << "blocker"; }
REQUIRE_NOTHROW(FileWriter(start_message));
remove(HDF5Metadata::DataFileName(start_message, 0).c_str());
// The master file, on the other hand, does collide.
{ std::ofstream(HDF5Metadata::MasterFileName(start_message)) << "blocker"; }
// Back-channel transport (direct HDF5 / TCP): fail fast in the constructor.
REQUIRE_THROWS_AS(FileWriter(start_message), JFJochException);
// ZeroMQ transport (no back-channel): must not throw - it will write .tmp and
// only fail at the final rename. The un-finalized writer cleans up its own tmp.
REQUIRE_NOTHROW(FileWriter(start_message, /*check_overwrite_at_start=*/false));
remove(HDF5Metadata::MasterFileName(start_message).c_str());
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
}
// If EndDataCollection throws while finalizing (here the master file appears mid-run,
// so the early check can't catch it), the pusher must still tear down its writer so a
// subsequent collection can start instead of dying with "already writing images".
TEST_CASE("HDF5FilePusher_finalize_failure_recovers", "[HDF5FilePusher][Repro]") {
RegisterHDF5Filter();
DiffractionExperiment x(DetJF4M());
x.FilePrefix("pusher_finalize_repro").ImagesPerTrigger(1)
.Compression(CompressionAlgorithm::NO_COMPRESSION)
.SetFileWriterFormat(FileWriterFormat::NXmxVDS)
.OverwriteExistingFiles(false);
StartMessage start_message;
x.FillMessage(start_message);
EndMessage end_message{};
// Make the master-file rename fail at EndDataCollection time.
{ std::ofstream(HDF5Metadata::MasterFileName(start_message)) << "blocker"; }
HDF5FilePusher pusher;
pusher.StartDataCollection(start_message);
// Finalization fails because the target master file already exists.
// Create the conflict after the start-time check has already passed.
{ std::ofstream(HDF5Metadata::MasterFileName(start_message)) << "blocker"; }
REQUIRE_THROWS_AS(pusher.EndDataCollection(end_message), JFJochException);
// The pusher must have released its writer despite the failure - otherwise the
// next collection dies with "Image pusher is already writing images".
// Writer released despite the failure: the next collection starts cleanly
// instead of dying with "already writing images".
remove(HDF5Metadata::MasterFileName(start_message).c_str());
REQUIRE_NOTHROW(pusher.StartDataCollection(start_message));
REQUIRE_THROWS_AS(pusher.EndDataCollection(end_message), JFJochException);
REQUIRE_NOTHROW(pusher.EndDataCollection(end_message));
remove("pusher_overwrite_repro_master.h5");
remove(HDF5Metadata::DataFileName(start_message, 0).c_str());
// The failed finalize intentionally leaves a .tmp behind - sweep the prefix.
for (const auto &e : std::filesystem::directory_iterator("."))
if (e.path().filename().string().rfind("pusher_finalize_repro", 0) == 0)
std::filesystem::remove(e.path());
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
}