- The option is --mr <file>. What kind of model it is is read from the file: a prediction where it
is ModelCIF (_ma_qa_metric) or has no cell and one B value per residue in 0-100 (AlphaFold PDB/mmCIF);
an experimental structure otherwise. A prediction is handled as before (pLDDT >= 70 trim, B from
pLDDT, PAE domains where the AlphaFold DB's PAE file sits beside it). An experimental structure is
taken as its polymer - no waters, ligands, hydrogens or alternative conformers, occupancy 1, isotropic
B as refined - with a coordinate error of 0.67 A (a prediction's at pLDDT 90) for sigma_A; a B column
above 100 no longer stops the search.
- The report section is MOLECULAR REPLACEMENT (EXPERIMENTAL), its keys MR_* (MR, MR_MODEL_FILE,
MR_MODEL_KIND, MR_DECISION, MR_SPACE_GROUP, MR_COPIES, MR_HAND_CHECK, MR_SECONDS, ...); the <prefix>_mr
outputs are unchanged.
- The twin hypothesis' merge that is made only for the model to test (a metric re-ask promoted the
indexed lattice's group) is written and reported only where the model chooses the subgroup.
Raw deposited coordinates of nine control sets, unconverted: every one placed and confirmed against the
deposition (all chains as one body), the groups the same as with the prediction or the prepared model.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Three defects the battery's wrong-model null found (5 sets x 5 deposited models of unrelated proteins):
CC(model, data) significance. The per-shell sigma was Fisher's transform on independent reflections,
but the data and a scaled model share the falloff across a shell, the anisotropic B fitted to these
very data, the bulk solvent and the symmetry enhancement epsilon - whatever the model is. 21 of 24
wrong models read CC_MODEL_CONFIRMED_TO_D_MIN to the data's d_min (10-15 sigma in most shells).
The correlation is now formed on I/epsilon and |F_model|^2/epsilon, and its null is empirical: the
data at h against the model at h + d for the 26 nearest lattice offsets, which keeps everything smooth
in reciprocal space and loses what the model knows of h. SIGMA = (z - mean) / max(sd, 1/sqrt(N-3)) in
Fisher z over those 26, a new CC_NULL column shows the null, and a shell confirms only beside a coarser
shell that also clears 3 sigma (one shell alone did so once in 216 wrong-model shells). Prototyped on
the p_maps.mtz of a full open-arm run and the null runs: wrong models confirm 0/24 (was 21/24), the
deposited models confirm 197/200 (was 198/200 in the prototype's reading; the one lost has CC_MODEL
0.05-0.13 in every shell). The report no longer says a model cannot agree by accident.
MODEL_FIT gate. Accepted at 3.0 sample sds of a nine-draw null, as if the sd were known; an unrelated
P1 helical model on a P3(1) helical crystal was ACCEPTED at +3.05. Against 48 draws it reads +2.09 and
the best random placement fits as well as it does, and drawing the null about the cell axes instead
changes nothing (+2.15): the cause is the nine-draw sd, not the orientation. The gate is now 3 sigma
allowing for that: t(8) * sqrt(1 + 1/9) = 4.51. The deposited models of the open arm read +7.0 and up.
The indexing lead stays at 3 sigma (INDEXING_MARGIN_SIGMA): it is read only behind the fit gate, and
a deposited model settles one at +4.15.
Rigid-body GPU reserve. The finest zone's observations were counted by the data's d, but the zones
count them by the d of the index in the model's ASU, and a model's group need not keep the data's
metric (an I2(1)3 model on I222 data: 2320 in the zone against 1712 reserved), so the validation died
with "zone over the engine's reserve". Counted the zones' way, under every indexing probed.
Tests: ModelValidation_CCModelUnrelatedModelConfirmsNothing (Fisher alone reads +4.7 on an unrelated
model with an anisotropic falloff; the new sigma stays under 1), ModelValidation_WrongModelInAGroupOffTheDataMetric
(fails with the old count), ResultReport_ModelValidationSection (CC_NULL column, lone-shell rule).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The ENANTIOMORPH hand check ran a whole ValidateAgainstModel on the placed model: model read and frame
probe, a rigid body, a 9-replicate null (~0.5 s of the ~0.6-0.9 s), sigma_A, three maps written. The
decision needs only --model's rule: the model fits its null (the MR choice has already required its
own null of the same construction) and the anomalous map at the model's atoms is not inverted.
ReadModelAnomalousMap (ModelValidation) computes F_calc and F_mask to the data's resolution (GPU engine
where there is one), carries them through the bulk solvent MR scored the placement with (the overall
scale and anisotropic B do not move phases), and reads the anomalous map with the same code as the
validation: its coefficient construction and its reading at the atoms are factored out of Validate
(anomalous_coefficients, read_anomalous_map) and used by both, so plain --model is unchanged.
Hand check 0.6/0.6/0.9 s -> 0.027/0.028/0.040 s on 11if/7ris/9zmu; same ADOPTED/VETOED on all seven
cases tried (the three demos, two anomalous lysozyme sets and their mirrored-detector runs), with the
scatterer means and deepest troughs within 0.1 sigma of the full validation's.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where a twin hypothesis (<prefix>_twin_<SG>) was written, the model is also
validated against that merge: as merged, and as twinned by its law -
|F_twin(h)|^2 = (1-a)|F_model(h)|^2 + a|F_model(Th)|^2 from the placed, scaled
model, one overall scale refitted per a, a scanned 0..0.5 on the working set
and R-free read at it (ValidateAgainstModel's new twin_law argument; GPU
structure factors as before). The maps and the placed model are then written
from whichever merge has the lower R-free; the report gives MAPS_FROM and the
subgroup's plain and twinned R-work/R-free and fraction. The subgroup file
follows its own validation's indexing, as the answer follows its own.
The model chooses the maps only: the space group, <prefix>.mtz and R_FREE stay
the answer's. Documented in RUGNUX_REPORT.md, RUGNUX_ADVANCED.md and
CPU_DATA_ANALYSIS_DECISIONS.md (14.9); Yeates (1997) credited.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Where the validation in the model's setting, started beside the first on
the forecast (b5980c9ba), is going to be kept, it writes the maps, the map
coefficients and the placed model on the written axes, and the first
validation's p_2fofc/p_fofc/p_anom.ccp4 and p_maps.mtz were overwritten
unread. ModelValidationSchedule::maps_superseded is asked once the
verdict, the setting and the indexing are decided, before any map is
made; the first validation then makes none (the map coefficients and the
MTZ are not built, the three map transforms not run, nothing written).
The answer is the same rule that keeps the forecast validation
afterwards (forecast_kept, now one lambda for both). The kept validation
also brings back the reflections it relabelled, free set included, so
relabel_output does not reindex and redraw them a second time. Where the
forecast is discarded or was never started, everything runs as before,
and modelpar's delete-before-rewrite stays for those cases. If the kept
validation then fails, a warning says no maps were written.
Log lines that disappear (first validation only, kept-forecast runs):
"mean 2mFo-DFc density at atom centres = ...", "anomalous difference map
from ... Bijvoet pairs; strongest density ...", "wrote p_2fofc.ccp4, ...";
in their place: "no maps made here - the validation in the model's
setting writes them". (The "anomalous density ... inverted" warning would
also go; it cannot be raised where a forecast is kept.)
Verified on 8sa8, 8xtg, 9ea5 (forecast kept), 8tyy, 7qis, myob_x10sa (no
forecast): p.mtz, p.hkl, p.cif, the three maps, p_maps.mtz and
p_model.cif/pdb md5-identical to the merge commit, repeated and at -N 8;
the discard path (8sa8 with a rotated model) identical files and log.
Time saved is small with the maps already on the GPU: 8sa8 7.6 -> 7.4 s,
8xtg 5.1 -> 5.0 s, 9ea5 within noise.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Where a model that fits is written on other axes than the data, the files
take its setting and the validation is made a second time on the
relabelled data. That second validation depends on the first only through
the setting and the indexing the first settles, and both are known long
before the first has finished: the change of basis right after the frame
scoring, the indexing the probe prefers before the null. So ValidateAgainstModel
now reports them (ModelFrameForecast, through ModelValidationSchedule::
on_forecast), and the second validation is started there, on the
relabelling AdoptModelFrame and relabel_output would make, applied to a
copy of the merge - beside the first one's null, real fit and maps.
It is kept only where the first decides exactly what was forecast (the
model fits, same change of basis, same indexing; a model asserting the
other enantiomorph is not forecast, as the label is decided last, on the
anomalous map). Until then its log is held (Logger::Buffered, replayed
where the serial run logged it) and its files wait on a gate
(ModelValidationSchedule::write_gate) placed before the first map is
written; otherwise it is released with false and returns unwritten, and
the serial validation runs as before.
GPU memory: two validations at once take twice the rigid-body engines.
The parallel start is decided up front from sizes, never from what is
free: allowed where twice what the first pool asked for (bytes per engine
times the engines wanted) fits a quarter of the card's TOTAL memory, the
share one validation may take. Threads: both validations submit to the
one ParallelFor pool from threads outside it (the second runs on a
std::async thread, as the null's replicates do), so no pass runs inline
on a pool worker and the pool's size bounds the workers.
Measured on the loaded 16-core workstation, TIMING model validation:
8sa8 30.6 -> 19.7 s, 8xtg 19.9 -> 13.9 s, 9ea5 22.1 -> 16.4 s (this and
the previous commit together). p.mtz, p.hkl, p.cif, the three maps,
p_maps.mtz and p_model.cif/pdb md5-identical to the base on 8sa8, 8xtg,
9ea5, 7qis and myob_x10sa; the validation's log lines identical as a set
(the null's replicate lines were already in completion order).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Each candidate setting of the model was scored one after another on the
coarse shell (a density, an FFT and an isotropic scale fit, 0.2-0.5 s
each), and a run with a model in another setting scores them in both of
its validations. They share nothing but what they read, so they now run
on ParallelFor, each on its own copy of the model, and are read back in
their own order: the log lines, the ranking and the tie-break (first of
equal R wins) are those of the serial loop.
Measured (16-core workstation, loaded): 8sa8 3 candidates 0.64 s ->
0.22 s; 9ea5 3 candidates 1.47 s -> 0.50 s; 8xtg 1.45 s -> 0.40 s.
p.mtz, the maps, the map MTZ and the placed model md5-identical.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
A pure move. ModelValidation, RigidBodyRefine, RigidBodyGPU, ModelFFT, ModelGrid,
ModelScaling, ModelMaskGPU, ModelScaleGPU and SigmaA - everything that works on an
atomic model - become the JFJochStructureRefinement library, linked by
JFJochImageAnalysis. WriteModel (the placed-model mmCIF/PDB writer) goes to writer/
as its own small JFJochModelWriter target, so JFJochWriter, which a writer-only build
compiles, does not gain a gemmi dependency. Only include paths and CMake lists change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
leonarski_fandClaude Opus 5.5