Commit Graph
9 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 5 639fbb3fbc indexing: select predicted reflections by partiality, build indexers where it pays
Build Packages / build:viewer-tgz:cpu (push) Successful in 7m34s
Build Packages / build:viewer-tgz:cuda (push) Successful in 8m42s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m24s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m31s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m44s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m5s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m16s
Build Packages / build:rpm (rocky8) (push) Successful in 11m28s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m45s
Build Packages / XDS test (durin plugin) (push) Successful in 7m39s
Build Packages / Generate python client (push) Successful in 36s
Build Packages / Build documentation (push) Successful in 1m4s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (rocky9) (push) Successful in 12m20s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m35s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 13m9s
Build Packages / DIALS test (push) Successful in 13m57s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m57s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m39s
Build Packages / Unit tests (push) Successful in 1h1m5s
Build Packages / build:windows:nocuda (push) Failing after 2s
Build Packages / build:windows:cuda (push) Failing after 3s
When more reflections are predicted for a frame than the output can hold, the
surplus was dropped by keeping those closest to the Ewald sphere. On the rotation
path that quantity is identically zero by construction - the rocking coordinate is
chosen so the scattering vector lands exactly on the sphere - so the comparison
fell through to h, k and l and the survivors were whichever came first in
lexicographic order. Measured on a large cell: every value within one float ulp of
zero, and the kept set had a MEAN PARTIALITY BELOW that of the full set, i.e. worse
than choosing at random. Rank by partiality instead, which the predictor already
computes and which is what the header always claimed was being kept. On the one
regression crystal large enough to cross the cap this lifts completeness from 84.8%
to 90.2% on the same observations; multiplicity and R_meas move the way they must
when the same measurements cover more of reciprocal space.

The online path asked for a cap of ten thousand but the truncation was hardcoded to
the offline limit, so the broker predicted and integrated up to six times what it
could transport and discarded the rest after paying for it. Honour the caller's
limit, which also makes the post-integration re-truncation dead code.

Indexer pool construction becomes a policy. The online service needs every indexer
resident before data arrives, because a cuFFT plan built on the first frame is
planning time inside the measurement; spending memory to be ready is the intended
trade there and stays the default. Offline there is no such deadline, and a stills
run with a known cell was holding a fully allocated FFT indexer per worker that the
algorithm resolution can never dispatch - 2.8 GB where 0.4 GB is needed. rugnux and
the viewer opt into building on first use; the broker, the receiver and the tests
are untouched. This also removes a dangling reference that was latent: the worker
held the settings by reference although the pool is routinely constructed from a
temporary, which only survived because eager construction finished inside the
constructor call.

Finally, refuse a first-pass lattice that indexes fewer than a sixth of the
validation frames. It fires on nothing in the regression set - the weakest real
crystal sits at 22 of 60, more than twice the floor - so it is a backstop, but the
failure it prevents is one the set does contain: a dataset with no crystal at all
adopts a lattice from its powder rings, integrates every image against it, and dies
much later inside the merge complaining about resolution. It now stops in the first
pass and says what to try.

Regression set: 36 of 37 crystals byte-identical, the exception being the
completeness gain above; 34 of 37 space groups, no failures. Full unit suite passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 09:12:27 +02:00
leonarski_f 75e401f0e5 v1.0.0-rc.153 (#63)
Build Packages / Unit tests (push) Successful in 1h31m59s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m43s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m5s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m27s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m24s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m27s
Build Packages / build:rpm (rocky8) (push) Successful in 9m20s
Build Packages / build:rpm (rocky9) (push) Successful in 10m50s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m54s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m38s
Build Packages / DIALS test (push) Successful in 12m13s
Build Packages / XDS test (durin plugin) (push) Successful in 7m8s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m8s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m50s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / Build documentation (push) Successful in 50s
Build Packages / Create release (push) Skipped
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing
* jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files
* jfjoch_broker: Add ROI calculation in non-FPGA workflow
* jfjoch_broker: Fixes to TCP image pusher
* jfjoch_broker: Remove NUMA bindings
* jfjoch_broker: Improvements to indexing
* jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values
* jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors
* jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs
* jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results

Reviewed-on: #63
2026-06-23 20:29:49 +02:00
leonarski_f cc3eb8352c v1.0.0-rc.148 (#58)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m28s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m9s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m47s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m39s
Build Packages / build:rpm (rocky8) (push) Successful in 11m43s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m59s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m48s
Build Packages / build:rpm (rocky9) (push) Successful in 12m32s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m24s
Build Packages / XDS test (durin plugin) (push) Successful in 7m35s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m50s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m40s
Build Packages / DIALS test (push) Successful in 11m19s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin

Reviewed-on: #58
2026-06-08 08:30:35 +02:00
leonarski_f c1c170112c v1.0.0-rc.136 (#45)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m17s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m48s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m57s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 15m15s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m35s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m29s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m55s
Build Packages / XDS test (durin plugin) (push) Successful in 8m17s
Build Packages / build:rpm (rocky9) (push) Successful in 12m17s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m2s
Build Packages / Generate python client (push) Successful in 30s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 51s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m13s
Build Packages / DIALS test (push) Successful in 13m19s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m52s
Build Packages / Unit tests (push) Successful in 1h18m25s
Build Packages / build:rpm (rocky8) (push) Successful in 7m2s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Improve logic regarding indexing architecture and thread pools (work in progress).

Reviewed-on: #45
2026-04-20 11:54:33 +02:00
leonarski_f 06949caf1a v1.0.0-rc.112 (#18)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m53s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 9m40s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 8m25s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m17s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m5s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m40s
Build Packages / build:rpm (rocky9) (push) Successful in 9m14s
Build Packages / Unit tests (push) Successful in 1h15m9s
This is an UNSTABLE release and not recommended for production use (please use rc.11 instead).

* jfjoch_broker: Experimental rotation (3D) indexing
* jfjoch_broker: Minor fix to error in optimizer potentially returning NaN values

Reviewed-on: #18
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-30 17:39:22 +01:00
leonarski_f 061152279c v1.0.0-rc.91 2025-10-20 20:43:44 +02:00
leonarski_f 5d9d2de4a4 v1.0.0-rc.81 2025-09-21 19:27:51 +02:00
leonarski_f 20973792e4 v1.0.0-rc.68 2025-08-16 19:59:27 +02:00
leonarski_f 51cbef4c84 1.0.0-rc.66 2025-07-18 11:42:39 +02:00