11 Commits
Author SHA1 Message Date
leonarski_f e0ff51e2a0 lattice: transpose the change of basis to a primitive cell, as everywhere else
gemmi states centred_to_primitive as an operator on COORDINATES and
CrystalLattice::Multiply combines BASIS VECTORS, so the matrix has to be
transposed on the way in - as it already is at both of the other places a gemmi
Op::Rot reaches Multiply, one of them in this same file.

A, B, C, I and F are symmetric matrices, so for them the transpose is a no-op
and the omission never showed. R and H are not. Measured on an R-centred
hexagonal lattice, ToPrimitive('R') returned 59.5 81.7 43.3 / 145.6 124.5 46.7
where the rhombohedral primitive cell is 49.3 49.3 49.3 / 60.9 60.9 60.9. What
hid it is that a determinant is unchanged by transposition, so the VOLUME came
out right - and most callers only take the volume.

It is not only cosmetic: the result feeds the re-seating path that puts a
lattice into a space group the user fixed by hand, so an R-centred lattice was
handed the classifier a "primitive" cell that is not that lattice - broken for
exactly the centring whose setting most needs re-seating.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit 6ca00e927d664e870515c04164defa81d8a18725)
2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5 4e8db41785 indexing: refuse a coplanar candidate, and search the plane normal when the shortlist is flat
Three related changes to the FFT candidate path, batteried together because they touch the same
function.

A COPLANAR CANDIDATE REACHED REFINEMENT. ReduceResults filtered triples on lengths and angles only -
the 30-150 degree bound admits any flat combination - and there was no volume test. On one dataset 41
of 5535 candidates had |V|/abc below 0.05, with a clean decade gap to the next, and three of them
reached the optimizer. UnitCell is float, and for a cell that flat the metric determinant is around
1.5e-7, so float32 gets its sign wrong 19% of the time where float64 never does. The guard against a
negative argument to sqrt then CREATES the singularity it was meant to prevent: it puts c in the a-b
plane, the reciprocal volume is 1/0, and the residual is 0 times infinity. Ceres reported a
not-a-number Jacobian and wrote several hundred lines of solver output per failed solve.

VolumeFraction() is |V|/(|a||b||c|), rejected below 0.02 - about 1.1 degrees off flat, ten times below
the flattest real candidate observed and a thousand times above where float loses the sign. It is
enforced at the producer and at the two optimizer entry points. Note the existing sanity checks use
ABSOLUTE volume, which a 320 cubic-angstrom flat cell passes. The same reciprocal-volume division is
now guarded at the two remaining sites that share the pattern.

A SHORTLIST CONFINED TO ONE PLANE cannot close a cell, and the row it is missing is the plane normal.
That is detected from the scatter-matrix eigenvalue ratio - measured, degenerate clouds score 2e-5 to
3.3e-4 against 0.026 or more for every non-degenerate one, a factor of eighty - and one further
transform is spent with the same direction count inside a three-degree cap about the normal, so the
plan and buffers are untouched. More directions cannot substitute: at the exact true direction the
long axis ranks 1422 of 16384 by prominence while the shortlist cut is four times higher. Ranking, not
sampling, is the obstacle. A four-fold denser grid was measured and rejected - it reaches the same
answer to three decimal places and takes a run from 2.5 to 8 GB of device memory.

fft_min_unit_cell_A is reachable as --fft-min-unit-cell and is lowered automatically by -C, mirroring
how the maximum is already raised. The default of 10 is unchanged: a lower floor admits spurious
sub-cells on protein data, and over 73 protein runs the floor was never lowered while the sibling
maximum did fire twice, so the path is live and correctly inert.

Corpus of 93 datasets, both arms, one build: 72 bit-identical on report content and p.hkl checksum, 13
failing identically, and the count of working datasets rises by one. The volume guard fires on 58 of
93 and 47 of those stay bit-identical - it fires constantly and almost never changes an answer, which
is what it should do. Solver chatter falls from 919 lines across three datasets to none. The cap
fires on 4 of 93, none of them in the in-house or private arms.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
2026-08-29 20:31:20 +02:00
leonarski_f 75e401f0e5 v1.0.0-rc.153 (#63)
Build Packages / Unit tests (push) Successful in 1h31m59s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m43s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m5s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m27s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m24s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m27s
Build Packages / build:rpm (rocky8) (push) Successful in 9m20s
Build Packages / build:rpm (rocky9) (push) Successful in 10m50s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m54s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m38s
Build Packages / DIALS test (push) Successful in 12m13s
Build Packages / XDS test (durin plugin) (push) Successful in 7m8s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m8s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m50s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / Build documentation (push) Successful in 50s
Build Packages / Create release (push) Skipped
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing
* jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files
* jfjoch_broker: Add ROI calculation in non-FPGA workflow
* jfjoch_broker: Fixes to TCP image pusher
* jfjoch_broker: Remove NUMA bindings
* jfjoch_broker: Improvements to indexing
* jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values
* jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors
* jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs
* jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results

Reviewed-on: #63
2026-06-23 20:29:49 +02:00
leonarski_f cc3eb8352c v1.0.0-rc.148 (#58)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m28s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m9s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m47s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m39s
Build Packages / build:rpm (rocky8) (push) Successful in 11m43s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m59s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m48s
Build Packages / build:rpm (rocky9) (push) Successful in 12m32s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m24s
Build Packages / XDS test (durin plugin) (push) Successful in 7m35s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m50s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m40s
Build Packages / DIALS test (push) Successful in 11m19s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin

Reviewed-on: #58
2026-06-08 08:30:35 +02:00
leonarski_f fc68a9baed v1.0.0-rc.146 (#56)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m34s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m16s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m49s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m32s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m15s
Build Packages / XDS test (durin plugin) (push) Successful in 7m16s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m12s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 47s
Build Packages / DIALS test (push) Successful in 10m18s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m46s
Build Packages / build:rpm (rocky8) (push) Successful in 1h41m2s
Build Packages / XDS test (neggia plugin) (push) Successful in 1h59m18s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.

WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.

Reviewed-on: #56
2026-05-28 18:48:35 +02:00
leonarski_f bb9f5c715f v1.0.0-rc.135 (#44)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m55s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m28s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m47s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m7s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m31s
Build Packages / build:rpm (rocky8) (push) Successful in 12m59s
Build Packages / build:rpm (rocky9) (push) Successful in 14m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m30s
Build Packages / Generate python client (push) Successful in 1m18s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m8s
Build Packages / XDS test (durin plugin) (push) Successful in 9m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m59s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m12s
Build Packages / DIALS test (push) Successful in 11m44s
Build Packages / Unit tests (push) Successful in 1h23m8s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* Multiple small bug fixes scattered across the whole code base. (detected with GPT-5.4)
* jfjoch_viewer: Improve image render performance

Reviewed-on: #44
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-16 11:59:59 +02:00
leonarski_f 3036b1d37f v1.0.0-rc.128 (#35)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m39s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m0s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m42s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m3s
Build Packages / Generate python client (push) Successful in 1m25s
Build Packages / Build documentation (push) Successful in 53s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m55s
Build Packages / build:rpm (rocky8) (push) Successful in 12m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m35s
Build Packages / build:rpm (rocky9) (push) Successful in 11m32s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m15s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m53s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Handle properly reuse of image buffer locations
* jfjoch_broker: Fix bug in counting idle slots
* jfjoch_broker: Force obtuse angle for monoclinic cells
* jfjoch_process: Change scaling refinement tolerance

Reviewed-on: #35
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-03-03 22:24:44 +01:00
leonarski_f 06949caf1a v1.0.0-rc.112 (#18)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m53s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 9m40s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 8m25s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m17s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m5s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m40s
Build Packages / build:rpm (rocky9) (push) Successful in 9m14s
Build Packages / Unit tests (push) Successful in 1h15m9s
This is an UNSTABLE release and not recommended for production use (please use rc.11 instead).

* jfjoch_broker: Experimental rotation (3D) indexing
* jfjoch_broker: Minor fix to error in optimizer potentially returning NaN values

Reviewed-on: #18
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-30 17:39:22 +01:00
leonarski_f 061152279c v1.0.0-rc.91 2025-10-20 20:43:44 +02:00
leonarski_f c67337cfe1 v1.0.0-rc.72 2025-09-08 20:28:59 +02:00
leonarski_f 41a3e671f4 v1.0.0-rc.41 2025-06-10 18:14:04 +02:00