In VDS mode the per-image ROI results (max/sum/sum_sq/npixel/x/y) are
written into the data files but were not exposed in the master, so a VDS
master surfaced no ROI statistics. Add virtual datasets under
/entry/roi/<name> in LinkToData_VDS, one group per ROI, mirroring how the
spot-finding and azimuthal-integration arrays are linked. Integrated and
legacy formats are unaffected (the results are already reachable there).
Extended the reader round-trip test to write real ROI results and check
they read back from the master for both VDS and integrated formats.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relocate the ROI definitions and the roi_map bitmap from /entry/roi to a
separate /entry/roi_defs group in the master file. /entry/roi keeps its
original meaning (per-image ROI results only), so an older reader that
iterates /entry/roi is not disturbed by the bitmap/definition entries
(which would otherwise make it try to read /entry/roi/roi_map/max and
fail to open the file at all). Back-compatibility for already-deployed
viewers is the reason for the split.
JFJochHDF5Reader now reads the logical definitions into experiment.ROI()
and the roi_map bitmask (+ a name->bit index) into the dataset, for all
file formats (the master always carries them). Reading both lets a later
viewer either re-derive ROIs with the current geometry (logical) or show
the exact written footprint (bitmap).
Added a write/read round-trip test over VDS and integrated formats.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist regions of interest so an acquisition's ROI layout can be
recovered from the master (_master.h5 / _process.h5):
- /entry/roi/roi_map: uint16 per-pixel bitmask (bit i == ROI i).
- /entry/roi/<name>: logical definition (type + geometry params, incl.
azimuthal phi sector) plus bit_index tying it to the bitmap plane.
The bitmap rides along in the StartMessage (CBOR, mirroring az_int_map)
and is filled both online (JFJochReceiver::SendStartMessage) and offline
(jfjoch_process). Definitions come from the already-transmitted rois.
Both are dataset-wide metadata, so they are written by the NXmx master
writer (new NXmx::ROI), not the per-image data-file plugin.
Documented the /entry/roi layout in docs/HDF5.md and the master-vs-data
writer convention in CLAUDE.md; added a CBOR roi_map round-trip test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.
* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin
Reviewed-on: #58
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.
jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.
WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.
Reviewed-on: #56
This is an UNSTABLE release. The release has significant modifications for HDF5 writing logic - in case of troubles go back to 1.0.0-rc.144.
* **Default HDF5 writing mode is with VDS, not soft-links** - this improves DIALS compatibility and makes format more future-proof, NXmx legacy format might be phased-out in the future.
* XDS plugin: Improve performance of VDS reading.
* jfjoch_writer: Significant improvement on how file systems I/O are handled through a dedicated pass-through VFD.
* jfjoch_writer: Clean-up of HDF5 routines to better handle issues.
Reviewed-on: #55
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.
* jfjoch_broker: Add better locking for detector object - should help, when detector initialization takes too long
* jfjoch_writer: Enable writing single, integrated HDF5 file with both data and metadata
* XDS plugin: Add generation of Jungfraujoch plugin for XDS
* CI: Add tests with XDS and DIALS (`xia2.ssx`)
Reviewed-on: #43
This is an UNSTABLE release. This version adds scalign and merging. These are experimental at the moment, and should not be used for production analysis.
If things go wrong with analysis, it is better to revert to 1.0.0-rc.124.
* jfjoch_broker: Improve logic on switching on/off spot finding
* jfjoch_broker: Increase maximum spot count for FFBIDX to 65536
* jfjoch_broker: Increase default maximum unit cell for FFT to 500 A (could have performance impact, TBD)
* jfjoch_process: Add scalign and merging functionality - program is experimental at the moment and should not be used for production analysis
* jfjoch_viewer: Display partiality and reciprocal Lorentz-polarization correction for each reflection
* jfjoch_writer: Save more information about each reflection
Reviewed-on: #32
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).
* jfjoch_broker: Improve handling of rotation indexing
* jfjoch_broker: More information saved in CBOR end message (WIP)
* jfjoch_writer: Save rotation indexing lattice parameters and Niggli class
* jfjoch_viewer: Remove (for now) primitive cell information
Reviewed-on: #19
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release.
jfjoch_writer: Fix and improve the way grid scan geometry is saved (non-NXmx extension makes it way easier)
jfjoch_viewer: Display grid scan results in 2D (work in progress)
jfjoch_viewer: Improve auto-scaling on start of images (work in progress)
jfjoch_viewer: Add B-factor and resolution estimate to the dataset info plots
Reviewed-on: #9
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release.
* Fixes in CI pipeline
* jfjoch_broker: Remove PNG preview, no dependency on libpng
* jfjoch_writer: Fix UTC timestamp being generated wrong (mix between milli- and microseconds)
* jfjoch_viewer: Show data collection time in dataset tooltip
* jfjoch_viewer: Allow to choose the calibrant (presets for LaB6 and silver behenate)
* jfjoch_viewer: Auto foreground value
* Use external libjpeg-turbo and libtiff: simpler build stack, these are built and linked statically in automated Docker builds
* Remove OpenBLAS dependency
Reviewed-on: #1
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
leonarski_f