11 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 5 004fa5a781 model validation: the model says, shell by shell, where the data still carry signal
A placed and scaled model gives an expected intensity for every reflection, and comparing that
with the merged intensity is a measurement nothing else in the report can make: CC1/2 and R_meas
compare the data with themselves, and can only say whether a shell is reproducible, not whether
it is right. CC(model, data) is reported per shell, on the merge's own shells so the rows line
up, with the reflection count and a significance beside each.

Read it in one direction only. A correlation significantly above zero proves signal is present,
because a model cannot invent agreement with data it never saw. A correlation near zero proves
nothing, since the model may be the thing at fault - one crystal in this corpus reads negative
across its outer shells against a deposited model of a different crystal form. So this can
support keeping a shell and must never be used to discard one, and nothing in the pipeline acts
on it.

Intensities rather than amplitudes, matching CC1/2 and CCref beside it. The raw merged intensity
rather than the French-Wilson amplitude: that amplitude is a posterior mean under a Wilson prior
which pulls weak reflections towards their shell mean, and the shells this number is read in are
exactly the weak ones, so it would correlate the prior with itself.

The free flag is ignored on purpose. Nothing has been refined against these reflections, so the
correlation is unbiased over all of a shell rather than over the test set alone - which is where
the outer-shell reflection counts are, and with them the difference between a statistic that can
resolve something and one that cannot.

REPORT_VERSION is 7.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-07 13:12:32 +02:00
leonarski_fandClaude Opus 5 ac1e8b8603 tests: the report version pin and the screw-gate section follow the code
Two test expectations that the changes under them made stale, and one overstated sentence in
the docs.

REPORT_VERSION was bumped to 12 without touching the line that pins it - the very line whose
comment says a key added to the report is a contract change and this is where it has to be
acknowledged. Acknowledged, and the absence of SOHNCKE_SPACE_GROUP on a fixture that is GIVEN
its group rather than searching for one is now asserted too, since that is what makes the key's
contract honest.

The section documenting that the E^2 gate is what saves a screw from fabricated violations no
longer holds, because there are now two independent defences: with the gate off, the absent
class still sits at 2% of its own row, so the zone is dead per reflection and licenses the
absence evidence to override the count. The section is kept rather than deleted - it now pins
the second defence, and a future P2 there would mean the deferral has stopped licensing a zone
that is genuinely extinct.

The docs said SOHNCKE_SPACE_GROUP is written on every run. It is written on every run whose
group came from the search; a run given its group with -S has no Sohncke candidate to name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-06 21:12:42 +02:00
leonarski_fandClaude Opus 5 8b9598c1c4 twinning: the L-test partners are chosen so a pseudo-translation cannot silence it
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 8m43s
Build Packages / build:windows:nocuda (push) Successful in 17m56s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 19m4s
Build Packages / build:windows:cuda (push) Successful in 19m47s
Build Packages / build:viewer-tgz:cpu (push) Successful in 21m7s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 23m28s
Build Packages / build:viewer-tgz:cuda (push) Successful in 23m41s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 28m21s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 28m26s
Build Packages / build:rugnux:windows (push) Successful in 10m58s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m15s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 21m35s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 26m54s
Build Packages / build:rpm (rocky9) (push) Successful in 23m26s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 22m52s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Build documentation (push) Successful in 1m4s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (rocky8) (push) Successful in 29m35s
Build Packages / XDS test (durin plugin) (push) Successful in 11m1s
Build Packages / DIALS test (push) Successful in 25m57s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 27m28s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 10m24s
Build Packages / XDS test (neggia plugin) (push) Successful in 9m28s
Build Packages / Unit tests (push) Successful in 1h27m23s
The L-test compares a reflection with a partner a fixed step away, and the step it
used preserves the parity class of a half-integer pseudo-translation - so a
pseudo-body-centring was invisible to it by luck rather than by design. A translation
of one third is not so lucky: it puts the two members of a pair in different
modulation classes, inflates the statistic past the bound that is read as evidence
AGAINST twinning, and the twin call is lost silently on a crystal that has one.

Choose the partners so every pair stays inside one modulation class, which is what
the half-integer case was already getting by accident. The obvious alternative -
lengthening the step until it clears a third as well - was measured and rejected: it
puts the statistic past that bound on more than two thirds of the corpus against a
seventh today.

Measured over a hundred and thirty-seven crystals: a hundred and twenty-nine
unchanged, seven repaired, and exactly one verdict moves - a crystal whose twinning
was being denied by its own pseudo-translation. On a synthetic perfect twin carrying
a one-third translation, both indicators are destroyed before the change and both
return after it.

Nothing branches on this verdict, so no merged intensity moves; the whole effect is
what the report says about the crystal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-05 20:09:00 +02:00
leonarski_fandClaude Opus 5 a4be5e6cd0 diagnostics: a translational pseudo-symmetry is detected and named
A pseudo-translation makes one class of reflections systematically strong and the
complementary class weak. It is the classic predictor of a failed molecular
replacement, it raises the second moment where twinning lowers it so each masks the
other's test, and rugnux did not look for it at all.

Look for it: a native Patterson from the merged intensities, the largest off-origin
peak taken as a fraction of the origin, and the vector refined against the data.
Both halves are scored against a null computed for that crystal rather than a bound
taken from elsewhere - the peak against intensities permuted within resolution
shells, the modulation depth against the same search started from random vectors -
because the noise floor of the first runs from 1.35 to 17.7 per cent across the
corpus, so no fixed threshold could mean the same thing twice. Requiring both is
what keeps the false-positive rate down: either alone calls one crystal in ten.

A translation the data are exactly invariant under is a lattice vector, not a
pseudo-symmetry. Recognising that removes eleven false calls from ninety-five merges
folded in P1 - each reading 83 to 102 per cent of origin against a genuine maximum of
62 - and adds one true positive whose pseudo-translation lies underneath its own
centring and is unreachable without it.

Report-only: it gates nothing and changes no reflection, no scale and no group. It
fires on eight of a hundred and thirty-seven crystals, and on those the verdict moves
from OK to WARNINGS, which is the point of it. Median cost 55 ms, worst 1.9 s on the
largest merge here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-05 20:08:40 +02:00
leonarski_fandClaude Opus 5 5b8ce26c83 integration: the flight path between the sample and the detector is corrected for, and named
A reflection arriving at an angle to the detector normal crosses D/cos(alpha) of
whatever lies between the sample and the sensor, not D, so it is attenuated more than
one arriving head-on and reads low. That is the same geometry as the sensor crossing
already corrected here and the opposite sign, and it was missing.

The factor is exp(D/L*(1/cos(alpha)-1)) from the NIST attenuation coefficient of the
medium, the stated distance and the stated wavelength. Nothing in it is fitted, and
it is not justified by any measured amplitude: the flight path and the sensor
crossing are collinear to better than 0.998 over the angular range any single
experiment samples, so no fit of one can be evidence for the other. It is the
tabulated absorption of a known thickness of a known material over a known path.

The medium cannot be detected. No field of the NXmx application definition describes
it, none of the masters this program reads carries one, and it cannot be inferred
from the implied transmission either - in this corpus a station confirmed to use
helium sits at 51% implied air transmission and one confirmed to use air at 63%, so
any rule separating them is a threshold fitted between two points. It is therefore
assumed, stated, and overridable: --flight-path air|helium|vacuum, defaulting to air.
Helium is its own material rather than an alias for vacuum, attenuating about a six
hundredth of air rather than nothing.

On an untilted detector the correction is a function of resolution alone, so its
entire effect on merged data is a shift in the Wilson B - which is what the report
now prints beside the assumption, accurate to better than a tenth of an angstrom
squared against measurement from 0.05 up to 28. Where that shift is large the report
warns, because a wrong medium is then the largest number in the run: applied to data
from the confirmed helium station it returns a B of 14 A^2 at 3.0 A resolution, which
is not a value a crystal can have.

The corpus contains its own control. One crystal, one station, three collections a
quarter of an hour apart at falling energy through the same air: corrected, the
Wilson B rises monotonically with the dose, as it must.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-05 17:55:43 +02:00
leonarski_fandClaude Opus 5 d6ccdbdb61 rugnux: the report says what it found before it says how it found it
The report was written in the order the pipeline runs, so a user reading one had to
reach line 380 before meeting the first evaluative statement, and the warnings were
last. Every run was 314 to 407 lines whether it succeeded or failed, about 200 of
them fixed prose. The anisotropy section printed 21 keys and announced DETECTED
(strong) on 54% of all runs - "strong" is the statistical confidence, which a reader
takes as the severity - and reported a censored fit, which is survival-analysis
vocabulary for good news.

Assemble the report into a document and render from it, rather than streaming it out
as the pipeline goes. That is what allows a verdict to sit above the evidence it was
drawn from: SUMMARY carries VERDICT, one sentence of plain text, the warnings and
ten facts, and it is composed after the sections that produce them. A clean run is
188 lines, a failed one 93, and the verdict is on line 25 in both.

Warnings now carry a closed pathology vocabulary alongside their free text, so a
consumer can switch on the code and a reader still gets the sentence.

Everything removed from the default report is still written under --developer, which
also carries the internals worth having when diagnosing the program rather than the
crystal: the gate keys behind the anisotropy verdict, the operator and candidate
tables, the sweep and spindle internals, and the essays.

Nine statements the report made that were not true are fixed here as well. Among
them: --mode scale printed a detector tilt of exactly zero on tilted data, which is
worse than printing nothing because nothing about it looks wrong; every --mode scale
run carried a "No image indexed" warning, because a key that is absent and a key
measured to be zero were the same value; and a fitted resolution was asserted past
the point where the run's own table shows CC1/2 at zero.

REPORT_VERSION is 8.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-05 13:40:59 +02:00
leonarski_fandClaude Opus 5 7e7103f49f spindle: the blind-cone score compiles on MSVC, and the report version test says 7
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 8m33s
Build Packages / build:windows:nocuda (push) Successful in 17m35s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 18m20s
Build Packages / build:windows:cuda (push) Successful in 20m2s
Build Packages / build:viewer-tgz:cpu (push) Successful in 20m44s
Build Packages / build:viewer-tgz:cuda (push) Successful in 22m50s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 22m54s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 27m16s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 28m0s
Build Packages / build:rugnux:windows (push) Successful in 11m0s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 21m4s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 21m53s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 27m49s
Build Packages / build:rpm (rocky9) (push) Successful in 24m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m32s
Build Packages / Generate python client (push) Successful in 30s
Build Packages / build:rpm (rocky8) (push) Successful in 30m22s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 1m4s
Build Packages / XDS test (durin plugin) (push) Successful in 11m14s
Build Packages / DIALS test (push) Successful in 25m29s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 28m20s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 10m17s
Build Packages / XDS test (neggia plugin) (push) Successful in 9m2s
Build Packages / Unit tests (push) Successful in 1h28m14s
Two CI failures from the spindle series, both self-inflicted.

The severity used M_PI, which is not standard and which MSVC does not define, in a
file under image_analysis - a tree the Windows viewer builds. common/JFJochMath.h
exists for exactly this and is now included. The same defect was found and fixed in
the post-refinement residual earlier in the day; the grep that confirmed it looked at
a worktree that did not yet carry this file, so the second instance survived.

The report-version test pins the version deliberately - adding a key to the report is
a contract change and that line is where it has to be acknowledged - but the key added
with the spindle report keys bumped the constant to 7 without the test following.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-04 20:00:20 +02:00
leonarski_f 680c36c20d v1.0.0-rc.166 (#76)
Build Packages / Unit tests (push) Successful in 1h22m15s
Build Packages / build:windows:nocuda (push) Successful in 18m0s
Build Packages / build:windows:cuda (push) Successful in 20m30s
Build Packages / build:viewer-tgz:cpu (push) Successful in 10m32s
Build Packages / build:viewer-tgz:cuda (push) Successful in 11m39s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 8m55s
Build Packages / build:rugnux:windows (push) Successful in 11m25s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m6s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 16m27s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 20m19s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 15m34s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 20m25s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 19m36s
Build Packages / build:rpm (rocky8) (push) Successful in 17m43s
Build Packages / build:rpm (rocky9) (push) Successful in 13m34s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 21m28s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 18m19s
Build Packages / DIALS test (push) Successful in 12m36s
Build Packages / XDS test (durin plugin) (push) Successful in 6m56s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m48s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m7s
Build Packages / Generate python client (push) Successful in 11s
Build Packages / Build documentation (push) Successful in 36s
Build Packages / Create release (push) Skipped
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 5m11s
* `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands.
* `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion.
* Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants.
* `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing.
* A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed.
* `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing.
* Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences.
* The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to.
* The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after.
* The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution.
* `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have.
* Twinning is no longer reported when the L-test contradicts it.
* The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's.
* `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots.
* The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area.

Reviewed-on: #76
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-09-02 21:17:31 +02:00
leonarski_f 511be0c366 v1.0.0-rc.165 (#75)
Build Packages / build:rpm (rocky8) (push) Successful in 24m0s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m54s
Build Packages / build:windows:cuda (push) Successful in 19m25s
Build Packages / build:viewer-tgz:cpu (push) Successful in 14m44s
Build Packages / build:viewer-tgz:cuda (push) Successful in 16m3s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 13m15s
Build Packages / build:rugnux:windows (push) Successful in 10m45s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 9m34s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 19m7s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 18m9s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 24m48s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 18m13s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 24m51s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 22m58s
Build Packages / build:rpm (rocky9) (push) Successful in 21m23s
Build Packages / Generate python client (push) Successful in 1m2s
Build Packages / Build documentation (push) Successful in 1m23s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 9m45s
Build Packages / XDS test (neggia plugin) (push) Successful in 10m19s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m10s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 22m15s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 17m37s
Build Packages / DIALS test (push) Successful in 17m16s
* `rugnux --model` adopts the model's space group as a label where the data were merged in its enantiomorph, instead of reindexing the reflections - which swapped I(+) with I(-).
* `rugnux --model` warns, naming the atom, when the anomalous density at the model's atoms comes out inverted, which means the data and the model are in opposite hands.
* `rugnux --model` writes an anomalous difference map (`<prefix>_anom.ccp4`) when the merge kept the Bijvoet split, and names the ten model atoms it peaks highest on as `ANOMALOUS_SITE_01`..`_10`.
* `MEAN_ATOM_DENSITY_SIGMA` is read from the map by cubic rather than linear interpolation and comes out around a tenth higher; it is no longer comparable with the figure earlier versions printed.
* `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name.
* A model `rugnux --model` cannot use is reported as a `WARNING:` line in the results report instead of only in the log.
* The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given; `REPORT_VERSION` is 4, `WARNINGS` moves to section 11 and no existing key changed.
* The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=`, `WALL_TIME=` and `GPU_COUNT=` / `GPU=`.
* rugnux says which GPUs it can see before it starts processing.
* `rugnux --export-unmerged` also writes `<prefix>_unmerged.mtz` on a `--no-merge` run, and is ignored on a run with no output prefix instead of writing a file called `_unmerged.mtz`.
* `/start` asks the writer whether the run can be written before the detector is armed, so a run whose master file already exists, or whose output directory cannot be created, is refused up front with the writer's own message. This needs the TCP image stream or the built-in HDF5 writer; the ZeroMQ stream is unchanged.
* A calibration that fails goes to `Error` carrying the reason instead of `Inactive`, so `/wait_till_done` and `/wait_until_running` report it; a cancelled calibration still goes to `Inactive`.
* `/wait_till_done` answers 500 with the message when a collection ended in an error. A cancelled collection and a collection that only triggered a warning still answer 200.
* A pending start failure is discarded by `/cancel` and `/deactivate`, as it already was by `/start` and `/initialize`.
* `/scan_result` no longer reports the previous run's images after a collection that failed to start, or after `/deactivate`.
* The TCP image stream protocol version is 4. `jfjoch_writer` and `jfjoch_broker` have to be of the same release, as before.

Reviewed-on: #75
2026-08-27 22:16:54 +02:00
leonarski_f 749db470ca v1.0.0-rc.164 (#74)
Build Packages / build:rpm (rocky9) (push) Successful in 19m56s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m57s
Build Packages / build:windows:cuda (push) Successful in 19m18s
Build Packages / build:viewer-tgz:cpu (push) Successful in 14m48s
Build Packages / build:viewer-tgz:cuda (push) Successful in 16m18s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 14m19s
Build Packages / build:rugnux:windows (push) Successful in 10m34s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 8m49s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m55s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 17m4s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 20m48s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 19m15s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 24m26s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 20m32s
Build Packages / build:rpm (rocky8) (push) Successful in 23m39s
Build Packages / Generate python client (push) Successful in 46s
Build Packages / Build documentation (push) Successful in 1m45s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 11m3s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m30s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m10s
Build Packages / XDS test (neggia plugin) (push) Successful in 10m17s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 23m12s
Build Packages / DIALS test (push) Successful in 20m12s
* rugnux now tells you whether a crystal diffracts anisotropically and how far it reaches in each direction, without a second program: a new `9. DIFFRACTION ANISOTROPY` section in `<prefix>_report.txt` and matching `_reflns.pdbx_aniso_B_tensor_*` / `_reflns.jfjoch_aniso_*` items in the merged mmCIF report the anisotropic deltaB, the diffraction limit along each principal direction, and a `NOT DETECTED` / `DETECTED` / `CANNOT DETERMINE` verdict measured against the data set's own systematic error. It is a description only - no intensity is corrected, no reflection is removed, and the merged data do not depend on direction.
* rugnux can hand its integrated observations to another scaling program: `--export-unmerged` writes `<prefix>_unmerged.mtz`, an unmerged MTZ readable by aimless, pointless, careless and `iotbx.merging_statistics`, in `--mode mx` and `--mode scale` alike. Each rotation reflection's partials are summed into one full; `--export-unmerged-partials` writes one row per image instead. Intensities carry the Lorentz-polarization factor and nothing else, since those programs scale the data themselves. Lattice-centring absences are not written; screw and glide absences are.
* rugnux integrates crystals with broad spots better - where it changes anything, per-shell mean I/sigma improves by up to 31% and R_meas by up to 24% - because on rotation data the integration signal radius is now taken from the crystal's own measured spot width instead of a fixed 4 px. `--adaptive-integration-radius=off` restores the fixed radius and an explicit `--integration-radius` still overrides both. The widened radius applies to the final integration pass only, and a pattern too dense for it is re-integrated at 4 px with a note in the log.
* rugnux discards fewer stills reflections for want of a background ring, improving per-shell R_meas over most of the signal-bearing range: the stills background ring now runs to 14 px instead of 12. The gain reverses in shells below a mean I/sigma of about 4.
* rugnux determines the space group with thresholds that mean the same thing on a weak crystal as on a strong one: symmetry operators are scored on resolution-normalised intensities (E squared) instead of raw merged intensities, and a reflection counts as genuinely present on its counting significance instead of on the merged I/sigma, which saturates at the merge's own ISa. The search resolution cut is no longer able to move the answer, and the twin-law H bound moves from 1.70 to 1.85, which stops one class of correct high-symmetry assignment being refused as twinning.
* rugnux says what the space-group search tested and what it could not: the twin-law disagreement H is printed for every operator together with the adopted point group's H ratio and its bound; alternatives that are not on the reported lattice are named with how their cell differs; and a lattice centring the data could not test - the crystal having been integrated on the primitive sub-cell, so the reflections it extinguishes were never measured - is marked `UNTESTED` and warned about where it is adopted, as coming from the lattice metric rather than from the intensities.
* rugnux `--mode scale` re-merges a `_process.h5` in the right symmetry without being told it: the file now records the space group on every run - a two-pass rotation run wrote none before, so re-merging defaulted to P1 - together with the change of basis under `/entry/MX/reindexMatrix` where the lattice was re-seated, and `--mode scale` also reports the Wilson B-factor estimate instead of `WILSON_B= nan`. A file written before this stops with a message naming the two cells and the override to use, instead of failing inside the merge. A third-party reader of a `_process.h5` must apply `reindexMatrix` where it is present.
* rugnux installs on its own, as a package called `rugnux` - `dnf install rugnux` or `apt install rugnux` - instead of arriving inside `jfjoch-viewer`. It pulls in none of the acquisition stack, so a machine that only processes data no longer has to carry the broker, the detector libraries or Qt to get it. Installing it over a `jfjoch-viewer` from rc.163 or earlier, which still owns `/usr/bin/rugnux`, upgrades cleanly rather than failing on the duplicate file.
* rugnux is also a standalone download, built for arm64 as well as x86_64: `rugnux-<version>-linux-{x86_64|aarch64}-cuda<major>.tgz` and `rugnux-<version>-win64-cuda<major>.zip` on the release page, for machines that are not managed by a package manager. The aarch64 build targets GH200 and DGX Spark, and is untested on hardware.
* Every portable Linux binary is now a single self-contained file: cuFFT is linked statically instead of being shipped beside the executable and found through an rpath, so `rugnux` and `jfjoch_viewer` need nothing but an NVIDIA driver, and only to use the GPU. The `.rpm`/`.deb` continue to take cuFFT from the distribution. The developer utilities `jfjoch_extract_hkl` and `jfjoch_recompress` are no longer packaged anywhere.
* Jungfraujoch needs six fewer shared libraries on the machine - libopenblas and libmetis, and libgfortran, libquadmath, libgomp and libz behind them - because the Ceres LAPACK, METIS and SuiteSparse back-ends are no longer built. Nothing in the code ever selected them, and results are unchanged.
* The PCIe driver DKMS package builds for the kernel it is being installed for instead of the running one, so a module built while a kernel update is being applied loads after the reboot.
* The PCIe driver builds on RHEL 9.5 and later, and on their CentOS Stream, Rocky and AlmaLinux equivalents, where the `vm_flags` kernel interface was backported into the 5.14 kernel.
* A data collection started with `async_start` that fails to start - a writer refusing to overwrite an existing file, for instance - is reported as an error by `/wait_until_running` and `/wait_till_done` instead of as a timeout and a successful collection respectively. The error message is the one the writer gave.
* A calibration that is cancelled or that fails to collect its pedestals is no longer reported as a successful one. The broker goes to `Inactive` with an error message and has to be initialized again, instead of sitting in `Idle` looking ready to measure while holding partial pedestals - data collected in that state was silently mis-converted.
* A failed `/initialize` is reported to `/wait_until_running` and `/wait_till_done` as soon as it happens, instead of when their timeout expires.
* `space_group_number` accepts space groups up to 230 in the API schema, so cubic space groups can be recorded. The broker always accepted them; the generated clients rejected them before the request was sent.
* The results report's `REPORT_VERSION` is 3, two sections having been added. Existing key names and table columns are unchanged.
* The merged statistics table has **9** resolution shells instead of 10, which is what XDS reports. The bins were already XDS's - equal steps in 1/d^2 between the lowest- and the highest-resolution reflection the merge kept - so at the same resolution limits the two tables now have the same shell boundaries and can be read row for row. `--resolution-shells` sets a different count.
* `rugnux --model` now settles the frame the merged reflections are written in, not only the frame the R-factors and the maps are computed in: the `.mtz`/`.cif`/`.hkl` come out in the model's indexing, and where the data were merged in the model's enantiomorph they take the model's hand and space group - which on anomalous data puts I(+) and I(-) the right way round. The indexing choice is logged with the winning R-free and the runner-up, so a decision made within noise is visible.
* `rugnux --model` can resolve the indexing ambiguity of a **serial stills** run, which a model could not do before: structure factors computed from the model become the per-image reference, the same role a reference MTZ plays. It needs the cell and space group up front (`-C` / `-S`). Without one or the other, a merohedral serial run still merges both hands together and says so.
* The rugnux documentation opens with a quick start - the default run, and runs with a reference MTZ, with a model, or with the space group and cell pinned - and explains the indexing ambiguity: what it costs on rotation and on serial data, and which of `-z` / `--model` resolves it in each case. The long reference pages now carry a table of contents.

Reviewed-on: #74
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-26 22:47:00 +02:00
leonarski_f 538f3504d3 v1.0.0.rc-161 (#71)
Build Packages / build:windows:nocuda (push) Successful in 20m4s
Build Packages / Unit tests (push) Skipped
Build Packages / build:viewer-tgz:cpu (push) Successful in 16m5s
Build Packages / build:viewer-tgz:cuda (push) Successful in 17m26s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 27m46s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m17s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 26m13s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 23m17s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 28m11s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 19m30s
Build Packages / build:rpm (rocky8) (push) Successful in 24m34s
Build Packages / build:rpm (rocky9) (push) Successful in 21m30s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 23m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m18s
Build Packages / DIALS test (push) Successful in 18m23s
Build Packages / XDS test (durin plugin) (push) Successful in 11m30s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 10m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m2s
Build Packages / Generate python client (push) Successful in 49s
Build Packages / Build documentation (push) Successful in 1m21s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:cuda (push) Successful in 29m45s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* **rugnux: significantly better quality of results, and faster.** A large rework of integration, scaling, merging, geometry refinement and space-group determination, together with measurements the program previously made no attempt at - the direct beam before indexing, the beam stop, the goniometer rotation scale, and the stretches of a sweep the crystal did not deliver. A rotation dataset typically gains observations at better <I/sigma> and R_meas, and every `mx` and `scale` run writes a `<prefix>_report.txt` results report modelled on XDS's `CORRECT.LP`. Many defaults moved with it: spot detection is self-calibrating, beam-stop detection and rotation geometry post-refinement are on, resolution limits default to as far as the detector reaches, and ice-ring handling engages only where the crystal is measured to have ice.
* **jfjoch_viewer:** the beam-stop shadow, the detector calibration and the beam-centre measurement are reachable from "Analyze dataset"; the settings panel reports how the sample moved and how polarized the beam was; image rendering and interaction are faster.
* **Performance:** bitshuffle+LZ4 images are decoded on the GPU rather than on the host, with the bitshuffle inverse fused into preprocessing so the decompressed frame is never held in device memory.
* **Broker, writer, packaging and build:** image-slot lifetime and locking fixes, per-image datasets sized by the images actually written, the Debian/Ubuntu broker package renamed to `jfjoch`, and `image_analysis` compiling under MSVC again.

**Breaking change to the rugnux command line:**
* `--azint-only` and `--scale` are **removed**, replaced by `--mode azint` and `--mode scale`; the full pipeline is `--mode mx` and remains the default. A script passing the old flags now fails with the list of valid modes rather than silently running the wrong one.
* `-t`/`--stride` is **refused on rotation data**: skipping frames cuts every reflection's rocking curve, so the combined fulls and their partiality would be measured over frames the sweep never recorded. Select a contiguous range with `-s`/`-e` instead. `--mode azint` and `--force-still` still take a stride.

**Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional:
* `image_scale_b` is removed from the `plot_type` enum, so a client requesting that plot now gets an error rather than a curve.
* `azim_int_settings.high_q_recipA`, `spot_finding_settings.high_resolution_limit` and `spot_finding_settings.low_resolution_limit` are no longer `required`. All three mean "no limit at that end" when unset and are omitted from the response instead of carrying a placeholder value, which raises in a client generated from an rc.160-or-earlier spec. A value of 0 is still accepted and means the same thing.

**Breaking changes to the stored formats** - a consumer reading these fields must treat them as optional:
* The per-image image-scale B factor is no longer computed, so `/entry/MX/imageScaleBFactor` is absent from newly written HDF5 files and the corresponding key is absent from the CBOR DataMessage and END blocks. Files written by rc.160 and earlier still contain it and still open; nothing in the pipeline reads it any more.
* `_reflns.jfjoch_diffrn_ISa` now carries the whole-range `1/sqrt(a*b)` that XDS's ISa denotes, and the error-model `a` and `b` are reported in XDS's convention; the strong-reflection asymptote moves to `_reflns.jfjoch_diffrn_ISa_asymptotic`. **A file written by an earlier version carries the asymptote under the plain `ISa` name.**

Reviewed-on: #71
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-13 17:03:10 +02:00