Commit Graph
11 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 4.8 9434878c92 reader: split metadata reading into HDF5MetadataSource + snapshot map
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m46s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m27s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 15m5s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 15m1s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m7s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m6s
Build Packages / build:rpm (rocky8) (push) Successful in 13m25s
Build Packages / XDS test (neggia plugin) (push) Successful in 9m46s
Build Packages / XDS test (durin plugin) (push) Successful in 10m14s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (rocky9) (push) Successful in 13m1s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m33s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 10m45s
Build Packages / Build documentation (push) Successful in 47s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m17s
Build Packages / DIALS test (push) Successful in 13m39s
Build Packages / Unit tests (push) Failing after 58m55s
Complete the image/metadata decoupling. JFJochHDF5Reader is now a thin JFJochReader
facade composing one shared HDF5ImageSource (raw pixels) with a name->HDF5MetadataSource
map and an active selection ("snapshots").

- HDF5MetadataSource owns one master file: dataset-level parse (Open, ex-ReadFile),
  per-image metadata (FillPerImage, ex-LoadImage_i), ReadSpots/ReadReflections/
  ReadCalibration, and the master-first helpers. It locates per-image metadata either via
  the shared image source (original file: metadata co-located with pixels) or in its own
  master at the global index (integrated _process.h5 snapshot) - so switching snapshots
  never reloads pixels.
- Facade adds RegisterSnapshot/SetActiveSnapshot/SnapshotNames/ActiveSnapshot; ReadFile
  registers the original as "Original". All existing JFJochReader callers are unchanged.
- The global hdf5_mutex is taken once in the facade; the sources assume it is held.

Adds JFJochReader_Snapshots test (original _master.h5 + a reprocessing _process.h5 over
the same images: pixels stay from the original, metadata follows the active snapshot).

Verified: tests/jfjoch_test [HDF5] (80 cases / 1810 assertions), plus jfjoch_process/
azint/extract_hkl/scale and jfjoch_viewer all build against the refactored reader.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 10:15:09 +02:00
leonarski_fandClaude Opus 4.8 23d27f30c4 reader: split raw-image reading into HDF5ImageLocator + HDF5ImageSource
Decouple the raw-pixel side of JFJochHDF5Reader from the rest as the first
step toward swappable per-dataset metadata snapshots.

- HDF5ImageLocator: single owner of the legacy/VDS/contiguous layout resolution
  plus a persistent open-file cache, replacing the four duplicated resolvers
  (GetImageLocation, ReadSpots, ReadReflections) and their per-call file caches.
  Also hosts the source-mapping logic (former GetHDF5DataSource body).
- HDF5ImageSource: raw-pixel reading (locator + LoadImageDataset); the part whose
  links to files stay fixed while the metadata master may change.
- JFJochHDF5Reader keeps a thin GetImageLocation/GetRawImage/GetHDF5DataSource that
  delegate to image_source_; the six layout members are gone, parsed into a local
  Layout handed to the source at the end of ReadFile. Cache cleared on Close().

Verified: tests/jfjoch_test [HDF5] (79 cases / 1775 assertions), and
jfjoch_process/azint/extract_hkl/scale relink unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 10:15:09 +02:00
leonarski_f 75de40f52b v1.0.0-rc.147 (#57)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m27s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m20s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 7m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 5m59s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 7m25s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 8m30s
Build Packages / build:rpm (rocky8) (push) Successful in 7m39s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m16s
Build Packages / build:rpm (rocky9) (push) Successful in 9m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m6s
Build Packages / Generate python client (push) Successful in 12s
Build Packages / Build documentation (push) Successful in 31s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 7m6s
Build Packages / DIALS test (push) Successful in 12m3s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m11s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m50s
Build Packages / Unit tests (push) Successful in 57m33s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_viewer: Add reciprocal space viewer
* jfjoch_process: Two pass algorithm that does spot finding/indexing + integration of full dataset
* jfjoch_process: Improve logic for rotation indexer, to make execution more deterministic (still work in progress)

Reviewed-on: #57
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-06-02 11:49:24 +02:00
leonarski_f 4a852b4d6b v1.0.0-rc.134 (#43)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m4s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m18s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m12s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m51s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m59s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m45s
Build Packages / build:rpm (rocky8) (push) Successful in 12m29s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m2s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / XDS test (durin plugin) (push) Successful in 9m50s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9) (push) Successful in 14m15s
Build Packages / Build documentation (push) Successful in 1m6s
Build Packages / DIALS test (push) Successful in 13m10s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m45s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m58s
Build Packages / Unit tests (push) Successful in 1h20m42s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Add better locking for detector object - should help, when detector initialization takes too long
* jfjoch_writer: Enable writing single, integrated HDF5 file with both data and metadata
* XDS plugin: Add generation of Jungfraujoch plugin for XDS
* CI: Add tests with XDS and DIALS (`xia2.ssx`)

Reviewed-on: #43
2026-04-09 13:30:47 +02:00
leonarski_f 6133da1377 v1.0.0-rc.133
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m5s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m6s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m47s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m9s
Build Packages / Generate python client (push) Successful in 49s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m38s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m35s
Build Packages / build:rpm (rocky8) (push) Successful in 9m16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m22s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m20s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m52s
Build Packages / Build documentation (push) Failing after 9s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Use httplib for HTTP server instead of Pistache
* jfjoch_broker: Drop OpenSSL support
* jfjoch_broker: Base work for multi-lattice support in the future
* Update dependencies to more recent versions (spdlog, HDF5, Catch2, httplib)

Reviewed-on: #41
2026-03-26 20:50:33 +01:00
leonarski_f 061152279c v1.0.0-rc.91 2025-10-20 20:43:44 +02:00
leonarski_f 06c5b9cf7f 1.0.0-rc.65 2025-07-16 20:19:48 +02:00
leonarski_f 0d1b388f4d v1.0.0-rc.50 2025-06-20 09:52:26 +02:00
leonarski_f 9b8f7a7f88 v1.0.0-rc.46 2025-06-17 14:56:48 +02:00
leonarski_f b245967df3 v1.0.0-rc.38 2025-05-12 14:17:24 +02:00
leonarski_f ddf4c75645 v1.0.0-rc.31 2025-03-02 13:15:28 +01:00