Commit Graph
6 Commits
Author SHA1 Message Date
leonarski_f e0ff51e2a0 lattice: transpose the change of basis to a primitive cell, as everywhere else
gemmi states centred_to_primitive as an operator on COORDINATES and
CrystalLattice::Multiply combines BASIS VECTORS, so the matrix has to be
transposed on the way in - as it already is at both of the other places a gemmi
Op::Rot reaches Multiply, one of them in this same file.

A, B, C, I and F are symmetric matrices, so for them the transpose is a no-op
and the omission never showed. R and H are not. Measured on an R-centred
hexagonal lattice, ToPrimitive('R') returned 59.5 81.7 43.3 / 145.6 124.5 46.7
where the rhombohedral primitive cell is 49.3 49.3 49.3 / 60.9 60.9 60.9. What
hid it is that a determinant is unchanged by transposition, so the VOLUME came
out right - and most callers only take the volume.

It is not only cosmetic: the result feeds the re-seating path that puts a
lattice into a space group the user fixed by hand, so an R-centred lattice was
handed the classifier a "primitive" cell that is not that lattice - broken for
exactly the centring whose setting most needs re-seating.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit 6ca00e927d664e870515c04164defa81d8a18725)
2026-08-31 07:17:00 +02:00
leonarski_f 527a5187f4 lattice: restore the character that names a centred monoclinic mI reduced form
ITA character 43 was absent from the table. It is the type-II reduced form of a
centred monoclinic lattice with no length equality - mC, mI, mA and mF are one
Bravais lattice in four settings, and this row names the one whose conventional
cell comes out I-centred. With the row missing such a cell reaches character 44
and is reported as triclinic, losing its centring outright. It accounted for 30
of the 31 demotions left after the two fixes before this one, and for 4.1% of
random centred-monoclinic lattices.

Both of its conditions are equalities on scalar products - International Tables
gives them as 2|D+E+F| = A+B and |2D+F| = B - so the three angle tests are
vacuous for it and the second equality is what selects it. cond_2DF was declared
in the character struct and never tested against anything, because until now no
row used it.

Audited over 14000 exact lattices, the row fires 35 times: 32 are centred
monoclinic lattices it recovers correctly and 3 are triclinic cells it promotes,
all three of which Le Page promotes at the same tolerance. No over-call is
attributable to it. The row is taken from International Tables rather than
derived - no single integer matrix of determinant 2 covers more than 68% of
these cells - and the test checks the conventional cell it produces has two
right angles and twice the primitive volume.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit 4c2baf0cc3b2177196a9448df06e24b5446de96a)
2026-08-31 07:17:00 +02:00
leonarski_f 931d7acc7c lattice: judge a structurally-zero scalar product against the size of the cell
The Niggli type of a reduced cell is the sign of its three scalar products, and
gemmi's reduction was asked to decide those signs against an absolute tolerance
of 1e-9 while the products themselves are 10^3 to 10^5 A^2 and the cell is held
in float. A product that is structurally zero therefore arrives carrying about
1e-4 A^2 of rounding and is read as definitely signed. The reduction lands on
the wrong side of the type-I/type-II boundary, the character written for the
other side matches nothing, and the lattice comes back with less symmetry than
it has.

It is not a corner case. Take an exactly body-centred tetragonal lattice with
c > a*sqrt(2) and merely ROTATE IT IN SPACE: 38 of 60 rotations lose the
4-fold, and it comes back C-centred. Every lattice this code classifies is a
refined, rotated one; the two tetragonal-I cases already in the tests are
axis-aligned, which is the one corner where the rounding vanishes.

The tolerance is now scaled by the cell's own magnitude. Over 14845 exact
lattices that puts it on a plateau three decades wide - demotions 461 -> 31 -
with the over-call count unchanged at every point of it, and with the three
fixes that follow the plateau is four decades wide and the demotions are zero.
The constant is 100x the one Grosse-Kunstleve et al. give because theirs is
calibrated for a double-precision cell and this one is float: measured, their
value recovers 6% of these lattices and this one 93%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit b82f217719450e1e46e94be5b5a43ed8ffbf2ca4)
2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5 10873eee7a lattice search: reject an impossible character, and supplement every angle
Two defects in the Bravais character walk, both of which silently cost symmetry.

An impossible character MATCHED. The walk computes acos(cond_F/sqrt(A*B)); when the character is
geometrically impossible the argument leaves [-1,1], acos returns NaN, and the acceptance test
fabs(NaN - actual) > tol is FALSE - so the character is taken. One corpus crystal matched a
monoclinic-C character whose implied cos(gamma) is 1.086.

The type-boundary retry covered beta only. All three angles carry the Niggli boundary, and negating
two basis vectors supplements the OTHER two, so each angle needs its own flip. A crystal whose
reduced gamma sits at 89.900 degrees needs the alpha/beta flip and never got it.

Measured on 84369 exact lattices spanning all 14 Bravais classes: nothing is lost in any class, and
orthorhombic-I recovery rises from 60.3% to 87.2%. Over 440000 random and perturbed cells the first
change only ever demotes a monoclinic-C match to triclinic and the second only ever promotes out of
triclinic - it is a strict superset of the beta-only retry. On the 94-dataset corpus, two crystals
gain their correct point group and none regresses.

Both regression tests fail without the change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
2026-08-29 19:57:41 +02:00
leonarski_fandjungfrau 4dc2534dbf v1.0.0.rc-162 (#72)
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-25 08:21:39 +02:00
leonarski_f 061152279c v1.0.0-rc.91 2025-10-20 20:43:44 +02:00