Previously an existing output file was only detected at the final rename,
after the whole collection had run. Add an up-front check in the FileWriter
constructor that fails before acquisition arms.
Only the master file is checked, and only by the single writer that owns it
(write_master_file). In a multi-writer TCP/ZMQ setup the per-image data files
are staggered across writers by file number, and a writer does not even know
the total writer count, so it cannot enumerate its own subset; checking all
data files in every writer would make each writer stat files it never writes
and race the writers already creating them. Data-file conflicts stay caught by
their owning writer at the final rename.
Gate the check on whether the transport can report the failure to the broker:
the direct HDF5 pusher throws in-process and the TCP writer returns a
START-failure ACK, so both abort the collection cleanly (verified: an asymmetric
rejection cancels the already-started sibling). The ZeroMQ path is fire-and-forget
with no back-channel, so StreamWriter opts out via image_puller.SupportsAck() and
keeps writing .tmp files, failing only at the final rename (leaving the images on
disk). Documented in docs/JFJOCH_WRITER.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HDF5FilePusher::EndDataCollection released the FileWriter only after
writer->WriteHDF5()/Finalize(), both of which throw at rename time when a
target file already exists and overwrite is off. The throw skipped
writer.reset(), leaving 'writer' non-null so the next StartDataCollection
died with "Image pusher is already writing images".
Tear the writer down on every exit path (try/catch + rethrow) so a failed
collection no longer blocks the next one. The error still propagates and the
orphaned temp files are intentionally left on disk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.
jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.
WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.
Reviewed-on: #56
This is an UNSTABLE release. The release has significant modifications for HDF5 writing logic - in case of troubles go back to 1.0.0-rc.144.
* **Default HDF5 writing mode is with VDS, not soft-links** - this improves DIALS compatibility and makes format more future-proof, NXmx legacy format might be phased-out in the future.
* XDS plugin: Improve performance of VDS reading.
* jfjoch_writer: Significant improvement on how file systems I/O are handled through a dedicated pass-through VFD.
* jfjoch_writer: Clean-up of HDF5 routines to better handle issues.
Reviewed-on: #55
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.
* jfjoch_broker: Azimuthal integration mapping is generated with parallel computations, significantly reducing setup times
* frontend: Fix selection of FFTW in indexing settings
Reviewed-on: #51
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.
* jfjoch_broker: Add better locking for detector object - should help, when detector initialization takes too long
* jfjoch_writer: Enable writing single, integrated HDF5 file with both data and metadata
* XDS plugin: Add generation of Jungfraujoch plugin for XDS
* CI: Add tests with XDS and DIALS (`xia2.ssx`)
Reviewed-on: #43
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.
* jfjoch_broker: Fix bug in saving JUNGFRAU calibration (pedestal/pedestalRMS)
* jfjoch_viewer: Fix calibration (pedestal) images being open flipped
* jfjoch_process: Add space group detection (EXPERIMENTAL)
Reviewed-on: #39
* Writer can send a PUB message with information on closed file
* Writer uses getopt to format input parameters
* DiffractionExperiment: Add frame number limit
leonarski_fandClaude Fable 5