Commit Graph
9 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 4.8 4cda46c6b0 Wire BraggIntegrationEngine into the pipeline; deterministic prediction; integration_model API
Replace the free functions BraggIntegrate2D/ProfileIntegrate2D with the
BraggIntegrationEngine (CPU/GPU) as the live integrator.

- IndexAndRefine no longer holds the integrator: ProcessImage takes a
  per-worker BraggIntegrateFn callback (ProcessImage is called concurrently by
  the shared IndexAndRefine, so the stateful engine must not be a member).
- WithoutFPGA/jfjoch_process: owns a GPU engine when a GPU is present, else CPU,
  and passes the GPU-resident preprocessed buffer so integration runs on-device.
- AfterFPGA: forces CPU and integrates straight off the assembled CompressedImage
  via a templated per-pixel sampler - only the reflection-disk pixels are read,
  no whole-image copy (the FPGA host runs up to 36 GB/s). Sampler maps type
  min/max to INT32_MIN/INT32_MAX on read; special/saturation only, no +/-1 band.
- Remove BraggIntegrate2D/ProfileIntegrate2D and their test; keep IntegratorMode.

Prediction: buffer up to 20000 candidates but return the 10000 closest to the
Ewald sphere (deterministic partial_sort on |dist_ewald|, hkl tiebreak) instead
of the GPU atomic-fill order. Serialized output stays <=10000, so the frame
transport headroom and its CBOR guard are unchanged.

integration_model exposed via OpenAPI (bragg_integration_settings schema,
/config/bragg_integration PUT/GET, added to jfjoch_settings and jfjoch_statistics)
and the frontend (BraggIntegrationSettings dropdown). Regenerated C++/TS clients
and redoc.

Validated old-vs-new on all 18 /data/rotation_test crystals: indexing rate and
space group bit-identical; ISa/CC identical on 16/18 (one improved, EcwtAL500
ISa 0.0->6.7); new CompressedImage-vs-buffer and GPU-vs-CPU parity tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 14:35:20 +02:00
leonarski_fandClaude Opus 4.8 711216e116 scan_result: short "ice" field + global Bravais lattice type
Two per-object scan_result API tweaks (C++ internal names unchanged):

- Rename the per-image scan_result field ice_ring_score -> ice (it is
  serialized once per image, so keep it short). Only the OpenAPI property and
  its regenerated C++/TS clients change; the DataMessage/EndMessage/CBOR/HDF5
  field stays ice_ring_score, and the ice_ring_score plot_type enum is untouched.

- Add a global rotation_bravais string (crystal-system letter + centering, e.g.
  "tP", "cF", "hR") to scan_result, alongside rotation_unit_cell /
  rotation_crystal_lattice. It comes from the RotationIndexer result
  (LatticeSearchResult system+centering) via the receiver, formatted by a new
  BravaisSymbol() helper in ScanResult.h.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 21:00:12 +02:00
leonarski_fandClaude Fable 5 5737d2e1d0 ice score: measure against a smooth whole-profile background, not shoulders
The score's baseline was two adjacent shoulder bins with a bin-overlap bug - the
ring's edge bins were counted in both the ring and the shoulder, since
GetMeanValueOfBins is inclusive. At the typical (coarse) azint binning (dq ~ 0.05
in q, wider than the 0.03 ring half-width) a shoulder is only ~1 bin, so the ratio
was noisy and poorly separated. Replace it with the ring intensity over a smooth
whole-profile background: a running median of the non-ice bins, interpolated under
each ring.

Clean crystals now sit at ~1.0 and ice separates far more cleanly on
/data/rotation_test: cytC 1.06->1.03, lysoC 1.23->2.77, EP_cs_01-17 1.67->4.51
(max 11.4). A z-score / abnormality probability was tried but is uninformative
here - with many photons any real ice ring is highly significant, so the useful
discriminator is the ice magnitude (this ratio), noted in CPU_DATA_ANALYSIS.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:05:39 +02:00
leonarski_fandClaude Fable 5 f422988cd2 ice score: per-image ice-ring indicator plumbed through all layers
A single per-image ice_ring_score - the strongest hexagonal-ice ring band/shoulder
intensity ratio from the azimuthal profile (1 = no ice) - computed in the CPU and
FPGA analysis paths and the offline azint worker, then plumbed through every layer:
DataMessage/EndMessage, CBOR (frame_serialize), HDF5 (/entry/MX/iceRingScore),
ScanResult, receiver plots (PlotType::IceRingScore), the OpenAPI spec (plot_type +
scan_result schema, with regenerated broker/gen and frontend client) and
OpenAPIConvert, the reader + Qt viewer, and the React frontend plot. Documented in
docs/CBOR.md, docs/HDF5.md and docs/CPU_DATA_ANALYSIS.md, with the general
"add a per-image quantity" recipe added to CLAUDE.md.

Verified in HDF5: lysoC (weak ice) mean 1.23, EP_cs_01-17 (heavy ice) mean 1.67 /
max 2.23. This is a monitoring quantity - it does not gate scaling (which already
excludes all ice rings) or merging (handled by the CC1/2 ring mask).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:37:12 +02:00
leonarski_f ab2176960f OpenAPI: Fix smargon descriptions 2026-07-01 14:12:09 +02:00
leonarski_fandClaude Opus 4.8 b7613c559c smargon: write chi/phi goniometer position into NXmx transformations
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 14m15s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 15m13s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 15m9s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 15m11s
Build Packages / build:rpm (rocky8) (push) Successful in 15m14s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m30s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m52s
Build Packages / build:windows:nocuda (push) Successful in 15m54s
Build Packages / build:windows:cuda (push) Successful in 17m32s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m30s
Build Packages / Generate python client (push) Successful in 30s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m31s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 8m47s
Build Packages / Build documentation (push) Successful in 58s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m37s
Build Packages / build:rpm (rocky9) (push) Successful in 12m37s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m44s
Build Packages / DIALS test (push) Successful in 12m55s
Build Packages / Unit tests (push) Successful in 1h0m54s
Add an optional Smargon static positioner (chi/phi angles + rotation axes)
that is reconstructed into the NXmx sample transformation chain. Chi/phi are
appended at the innermost end of the chain (closest to the sample) for both
the goniometer and grid-scan branches, with axes defaulting to chi {0,0,1}
and phi = omega default {1,0,0}.

- SmargonPosition gains chi_axis/phi_axis (common/JFJochMessages.h)
- OpenAPI: optional phi_axis/chi_axis arrays; clients regenerated
- OpenAPIConvert wires Dataset_settings.smargon -> DatasetSettings
- CBOR serializer/deserializer round-trip the axes
- tests: CBORSerialize_Start_Smargon

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:45:08 +02:00
leonarski_fandClaude Opus 4.8 809f6f92a3 Regenerate OpenAPI clients for bszstd_rlehuf compression value
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m54s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m31s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m51s
Build Packages / build:rpm (rocky8) (push) Successful in 14m49s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m5s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 15m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m48s
Build Packages / XDS test (durin plugin) (push) Successful in 7m55s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m50s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 1m10s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m36s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m7s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m1s
Build Packages / build:rpm (rocky9) (push) Successful in 13m7s
Build Packages / DIALS test (push) Successful in 12m45s
Build Packages / Unit tests (push) Successful in 1h14m59s
Build Packages / build:windows (push) Successful in 18m31s
update_version.sh regenerated the TypeScript client (types.gen.ts, zod.gen.ts)
and Redoc docs to include the new "bszstd_rlehuf" compression enum value added
to jfjoch_api.yaml; the package-lock version field follows VERSION. The C++
server model treats compression as a plain string, so it needed no change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 14:52:12 +02:00
leonarski_f 6136f858af v1.0.0-rc.154 (#64)
Build Packages / Unit tests (push) Successful in 1h26m51s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m56s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m43s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m53s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m44s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m22s
Build Packages / build:rpm (rocky8) (push) Successful in 13m1s
Build Packages / build:rpm (rocky9) (push) Successful in 14m6s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 13m0s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m51s
Build Packages / DIALS test (push) Successful in 13m52s
Build Packages / XDS test (durin plugin) (push) Successful in 9m24s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m35s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m57s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / Build documentation (push) Successful in 47s
Build Packages / Create release (push) Skipped
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_broker: Fix to TCP file pusher (remove kernel zero copy to improve reliability)

Reviewed-on: #64
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-06-25 18:12:00 +02:00
leonarski_f 75e401f0e5 v1.0.0-rc.153 (#63)
Build Packages / Unit tests (push) Successful in 1h31m59s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m43s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m5s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m27s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m24s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m27s
Build Packages / build:rpm (rocky8) (push) Successful in 9m20s
Build Packages / build:rpm (rocky9) (push) Successful in 10m50s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m54s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m38s
Build Packages / DIALS test (push) Successful in 12m13s
Build Packages / XDS test (durin plugin) (push) Successful in 7m8s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m8s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m50s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / Build documentation (push) Successful in 50s
Build Packages / Create release (push) Skipped
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing
* jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files
* jfjoch_broker: Add ROI calculation in non-FPGA workflow
* jfjoch_broker: Fixes to TCP image pusher
* jfjoch_broker: Remove NUMA bindings
* jfjoch_broker: Improvements to indexing
* jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values
* jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors
* jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs
* jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results

Reviewed-on: #63
2026-06-23 20:29:49 +02:00