Commit Graph
3 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 5 28a58e94b9 model validation: write the model as it was placed
`--model` re-fractionalizes the model into the data cell and then places it as
one rigid body, but the placed coordinates never reached disk. On a lysozyme
sweep against a non-isomorphous deposited model the move is 3.058 deg and
1.035 A, so a user overlaying their input model on rugnux's maps was out by
exactly that, and no file on disk corresponded to the maps at all.

`<prefix>_model.cif` is that file: the input's chains, residues, ligands,
waters, B-factors, occupancies and anisotropic Us, at the coordinates the maps
were computed from. `<prefix>.cif` is already the merged reflections, hence the
suffix.

The cell and space group come from the same two values WriteReflections is
given - the unit cell and DiffractionExperiment::GetSpaceGroupOrP1() after
AdoptModelFrame has settled the enantiomorph - so the coordinate file and the
.mtz beside it always agree. Taking them from the input model would not: with
data merged in P4(1)2(1)2 and a P4(3)2(1)2 model, the written reflections take
the model's group, which is neither the data's original label nor, when the
model is rejected, the model's own.

Written whenever the maps are, not only where the rigid-body step was
committed. The model is re-fractionalized and may be relabelled whatever the
placement decided, so an unmoved model is still not the input file; and a model
the null rejected is scored, placed and mapped like any other - the negative
result, and the case where the density is most worth looking at.

Nothing in the tree could write coordinates: gemmi_gph declared to_mmcif.hpp
but src/to_mmcif.cpp had been trimmed from the vendored subset. Both it and
to_pdb.cpp (to_mmcif.cpp calls its use_hetatm) are vendored from the same
gemmi 0.7.5 the rest of gemmi_gph comes from, unmodified, and every header they
include was already there. Same package, same MPL-2.0, same LICENSE.txt already
collected into licenses/gemmi.txt and already listed against `gemmi_gph/` in
THIRD_PARTY_NOTICES.md, so no new row and no new licence text.

Verified end to end: read back with gemmi the file differs from the
re-fractionalized input by exactly the reported 3.058 deg / 1.035 A with 0.0000 A
rms about that rigid move, and REFMAC5 at zero cycles against rugnux's own .mtz
starts at R-free 0.3667 where rugnux reports 0.3826 - against 0.5832 for the
unplaced input model, where rugnux reports 0.5911 before the placement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-02 14:47:38 +02:00
leonarski_fandClaude Opus 5 6b738713e8 model validation: place the model, then weight the maps by sigma_A
--model re-fractionalized the model into the data cell and then left it there.
On a non-isomorphous pair that is a placement error, not a cell error: the box
is squeezed, the body inside it is not moved. Six parameters now recover it -
an angle-axis rotation about the model's centroid and a translation, refined
over a 6 / 4.5 / 3.5 A ladder, the scale (k_overall, anisotropic B, k_sol,
b_sol) re-fitted at every evaluation so the target measures the placement and
not the scale. The refinement sees only the working reflections and the step is
committed only if R-free, on the free set it never saw, drops; otherwise the
model goes back where it was read.

Measured on merged lysozyme data against a deposited lysozyme model whose cell
differs by 3.4% in c: R-work 0.559 -> 0.400, R-free 0.591 -> 0.383. Over the
same 3.5 A range the external arbiter (REFMAC rigid body through dimple) works
in, 0.524 -> 0.330 against REFMAC's 0.522 -> 0.355, and the recovered movement
agrees with REFMAC's to 0.25 deg and 0.03 A (3.05 deg / 1.04 A vs 2.76 / 0.98).
2.4 s of added wall clock, 234 structure-factor evaluations.

The map coefficients become 2mFo-DFc and mFo-DFc. sigma_A is estimated by
maximum likelihood per resolution shell on the free reflections only, with the
number of shells taken from the size of the free set so no shell is thin;
centric and acentric reflections carry their own likelihoods, and a centric
reflection's bias-free coefficient is mFo. Cross-checked against CCP4 SIGMAA
on the same reflections: mean FOM 0.404 against its 0.396, with the same
per-shell structure. The figure of merit is written to _maps.mtz so the
weighting can be undone.

The per-shell scaling refusal in fit_model stands - Fobs is never rescaled and
the R-factors are untouched - but m and D are per-dataset, so maps from one
campaign are no longer scaled identically. That is argued at the code and in
docs/CPU_DATA_ANALYSIS_DECISIONS.md 14.4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-02 13:06:05 +02:00
leonarski_f 511be0c366 v1.0.0-rc.165 (#75)
Build Packages / build:rpm (rocky8) (push) Successful in 24m0s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m54s
Build Packages / build:windows:cuda (push) Successful in 19m25s
Build Packages / build:viewer-tgz:cpu (push) Successful in 14m44s
Build Packages / build:viewer-tgz:cuda (push) Successful in 16m3s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 13m15s
Build Packages / build:rugnux:windows (push) Successful in 10m45s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 9m34s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 19m7s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 18m9s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 24m48s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 18m13s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 24m51s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 22m58s
Build Packages / build:rpm (rocky9) (push) Successful in 21m23s
Build Packages / Generate python client (push) Successful in 1m2s
Build Packages / Build documentation (push) Successful in 1m23s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 9m45s
Build Packages / XDS test (neggia plugin) (push) Successful in 10m19s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m10s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 22m15s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 17m37s
Build Packages / DIALS test (push) Successful in 17m16s
* `rugnux --model` adopts the model's space group as a label where the data were merged in its enantiomorph, instead of reindexing the reflections - which swapped I(+) with I(-).
* `rugnux --model` warns, naming the atom, when the anomalous density at the model's atoms comes out inverted, which means the data and the model are in opposite hands.
* `rugnux --model` writes an anomalous difference map (`<prefix>_anom.ccp4`) when the merge kept the Bijvoet split, and names the ten model atoms it peaks highest on as `ANOMALOUS_SITE_01`..`_10`.
* `MEAN_ATOM_DENSITY_SIGMA` is read from the map by cubic rather than linear interpolation and comes out around a tenth higher; it is no longer comparable with the figure earlier versions printed.
* `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name.
* A model `rugnux --model` cannot use is reported as a `WARNING:` line in the results report instead of only in the log.
* The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given; `REPORT_VERSION` is 4, `WARNINGS` moves to section 11 and no existing key changed.
* The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=`, `WALL_TIME=` and `GPU_COUNT=` / `GPU=`.
* rugnux says which GPUs it can see before it starts processing.
* `rugnux --export-unmerged` also writes `<prefix>_unmerged.mtz` on a `--no-merge` run, and is ignored on a run with no output prefix instead of writing a file called `_unmerged.mtz`.
* `/start` asks the writer whether the run can be written before the detector is armed, so a run whose master file already exists, or whose output directory cannot be created, is refused up front with the writer's own message. This needs the TCP image stream or the built-in HDF5 writer; the ZeroMQ stream is unchanged.
* A calibration that fails goes to `Error` carrying the reason instead of `Inactive`, so `/wait_till_done` and `/wait_until_running` report it; a cancelled calibration still goes to `Inactive`.
* `/wait_till_done` answers 500 with the message when a collection ended in an error. A cancelled collection and a collection that only triggered a warning still answer 200.
* A pending start failure is discarded by `/cancel` and `/deactivate`, as it already was by `/start` and `/initialize`.
* `/scan_result` no longer reports the previous run's images after a collection that failed to start, or after `/deactivate`.
* The TCP image stream protocol version is 4. `jfjoch_writer` and `jfjoch_broker` have to be of the same release, as before.

Reviewed-on: #75
2026-08-27 22:16:54 +02:00