RUGNUX: a model run also refines the model it placed; the model can be given as an input; only the unmerged MTZ is written by default; the _process.h5 is not a default output; a short "when a run fails" note. TUTORIAL: the .ins, the refined model/maps/blobs, the log and _modules.h5 in the output list; the report is not written with -o ""; stills: -X Auto picks ffbidx with a cell, a run over several files. FORMATS: the XFEL stills formats and model/reference inputs told by content. CALIBRATION: -C and -S as the calibrant. INSTALL: the CPU path runs every indexer and the refinement. MODEL_AND_REFERENCE: the refined model is written as .pdb too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 0062d72b95e996438ddc65dc343b526c84daa7fb)
38 KiB
Models and reference data
What Rugnux does with outside knowledge — an atomic model, a predicted model, a reference dataset, a cell or a space group you already know — and, decision by decision, what that knowledge is and is not allowed to change. The first section is the short version; the tables after it are the precise one; the rest walks through a run step by step.
:local:
:depth: 2
In short
- Rugnux needs nothing but the images. Indexing, the cell, the space group, the resolution cut, scaling, merging and the error model all come from the data. Everything on this page is optional.
- A model (
--model,--mr) does not change how your data are processed. The images are indexed, integrated, scaled and merged the same way with or without it, and the space group is determined before the model is used. The merged intensities and their sigmas in<prefix>.mtzare the same numbers either way. - The model can only relabel, and only if the data accept it. It is first tested against
copies of itself placed in random orientations. A model that fits no better than those changes
nothing at all: the reflection files are byte for byte what a run without a model writes. A model
that does fit may change three labels in
<prefix>.mtz: the hand of the space group (P4₁2₁2 against P4₃2₁2, which the intensities cannot tell apart), the indexing where the lattice allows two equally valid ones, and the axes the files are written on. No intensity is changed, no reflection is added or removed, and no other space group is ever taken. - A different space group chosen by the model goes to a separate file. If the model prefers
another space group, or a twinned subgroup, that merge is written beside the answer as
<prefix>_mr.mtz/.cif, with the model as placed there.<prefix>.mtzstays the data's own answer. - A placed model is then refined, after every decision has been taken, and written beside the
answer as
<prefix>_refined_model.cif/.pdbwith its maps (Refinement);--no-refineskips it. The refinement changes nothing else the run writes. - A reference dataset (
-z) and a space group or cell you give (-S,-C) are different: they are taken as facts about the crystal. They fix the space group and the cell before processing, so with them the space group is yours, not the data's. - One exception to "processed the same", and only one: on serial stills given
--modeltogether with-Cand-S, the model is used during integration to break the indexing ambiguity — to choose, crystal by crystal, between the two equally valid indexings — the same job a reference dataset does there (see Serial stills). Nothing else is taken from it before the merge. This is deliberate: a structure of the same protein (often a cryo structure) usually exists, and it settles the choice reliably. Where the lattice has no ambiguity the model changes nothing. Without a model the crystals are made consistent with each other from the data alone (with-Sgiven), which recovers most of what the model gives; the model still decides the weakest crystals better than the other crystals can.
Every option that brings in a model or reference data
| Option | What you give | What Rugnux uses it for | Can it change <prefix>.mtz? |
|---|---|---|---|
| (none) | — | Everything from the data | — |
--model <file> |
An atomic model, PDB or mmCIF, gzipped or not, recognised by content: an experimental structure or a prediction | Validation after the merge (R-work/R-free, CC(model, data), maps, the placed model); a rigid-body placement checked against a null, with molecular replacement where the rigid body does not place it, and straight away for a prediction; then refinement of the placed model (REFINE_ keys, <prefix>_refined_model.cif/.pdb, _refined_maps.mtz, _blobs.json), which decides nothing |
Labels only (hand, indexing, axes), and only where the model fits. Other space groups → <prefix>_mr.* |
--mr <file> |
The same kinds of model | As --model, but molecular replacement straight away, without the rigid body. --mode mx only, not together with --model |
Hand label only. Other space groups → <prefix>_mr.* |
| PAE file beside a prediction | AF-<id>-F1-predicted_aligned_error_v<n>.json next to AF-<id>-F1-model_v<n>.cif (read automatically, no option) |
Cuts the prediction into rigid domains for molecular replacement | No (a domain alone can only confirm the answer's group) |
-z, --reference <file> (--reference-mtz) |
Merged data of the same crystal form: an MTZ, or a PDB structure-factor mmCIF (-sf.cif) |
Fixes the space group (in its own setting) and the cell (softly), puts the merge on its axes, resolves the indexing ambiguity, hands over its R-free set, reports CCref on stills. Never a scale anchor | Yes: the space group, the axes, the indexing and the free set are the reference's |
--reference-column <label> |
An MTZ column label | Which column of the reference is read | Through -z only |
-S, --space-group |
A space group | Fixes it: no space-group search | Yes: the group is yours (a fitting model may still relabel its hand) |
-C, --unit-cell |
A cell | Indexing looks for this cell (within tolerance); the FFT search range is widened to cover it; on stills it turns on the shared-geometry refinement and, with -S and --model, the per-image use of the model |
Yes: the cell, and the axis order the files are written in |
--force-rotation-lattice |
Nine numbers: a lattice | Skips the first-pass indexing and uses this lattice | Yes: the lattice |
--report-resolution <dmin>[,<dmax>] |
Another program's resolution range | A second statistics table at that range (REFRES_*) |
No — report only |
Without the options. A model and a reference reflection file can also be given among the inputs, in any
order: rugnux data_master.h5 model.cif, rugnux model.pdb.gz data_master.h5 ref.mtz. Each file is told by
its contents, never its name: an MTZ (its MTZ magic) or an mmCIF with a _refln or _diffrn_refln loop
(a PDB -sf.cif) is reference data, taken as -z - in --mode refine it is the input itself; a PDB file (a
PDB record first, with an ATOM or HETATM record) or an mmCIF with an _atom_site loop is a model, taken
as --model; anything else is images, for the readers. An mmCIF holding both coordinates and reflections is
refused as ambiguous and has to be given with --model or -z. At most one model and one reference, counting
--model/--mr and -z as well; gzipped files are read through.
-S, -C, -z and --force-rotation-lattice are assertions: Rugnux takes them as true and
does not test them against a null. Use them when you know; leave them out to let the data decide
(a de-novo run is the recommended default — see Running Rugnux).
--model and --mr are hypotheses: Rugnux tests them, and what it lets them change is limited
and stated below. In --mode calibration a cell given with -C names the calibrant instead
(Detector calibration); nothing on this page applies there.
Four kinds of influence
Every decision and every output of a run falls in one of four classes. The decision table below uses these letters.
- (a) Data only. Never influenced by any model. The same with and without
--model/--mr. - (b) Model-informed label in
<prefix>.mtz. A choice between alternatives the data cannot separate and that are all equally valid descriptions of the same measurement: the hand of an enantiomorphic pair, an indexing ambiguity, the setting (axes). The model may make it only after it has passed its null; the intensities do not change. Reported asSPACE_GROUP_ENANTIOMORPH= ASSUMED_FROM_MODEL,MODEL_DECISIONS_TAKEN=,MODEL_ENANTIOMORPH_ADOPTED=,MODEL_INDEXING_OPERATOR=,SETTING_SOURCE= MODEL, and the withdrawal ofINDEXING_AMBIGUITY_OPERATORS=. - (c) Model-driven, separate files only. A choice the data could in principle make but did not
make the model's way: another space group with the same rotations (other screw axes), or the twin
hypothesis' subgroup. Written only as
<prefix>_mr.mtz/.cifwith<prefix>_mr_model.pdb/.cif(and named byMODEL_CHOICE_FILE_PREFIX=,MODEL_CHOICE_SPACE_GROUP=,MR_DECISION=). Never in<prefix>.mtz. - (d) Report-only model statistics. Numbers about the model against the data, which nothing in
the pipeline acts on:
R_WORK/R_FREE,R_MODEL_SHELL_SCALED,CC_MODEL_*,MODEL_FIT*,MODEL_PLACEMENT, theMR_*keys, the anomalous sites, the maps.
Decision by decision
"Reference" means -z. A dash means the option plays no part.
| Decision | Without anything | With -S / -C |
With -z |
With --model / --mr |
Class |
|---|---|---|---|---|---|
| Spot finding, beam centre, detector geometry | data | — | — | — | (a) |
| Indexing, lattice, cell | data (de novo) | your cell / lattice | the reference's cell, softly | — | (a) |
| Integration | data | — | — | — (stills: see below) | (a) |
| Scaling, error model, ISa | data | — | — (never a scale anchor) | — | (a) |
| Outlier rejection, frame disposition | data | — | — | — | (a) |
| Resolution cut | data (CC1/2 fall-off) | — | — | — (CC_MODEL_CONFIRMED_TO_D_MIN is reported, never used) |
(a) |
| Point group, Laue class, centring | data (space-group search) | -S fixes it |
the reference's | — | (a) |
| Screw axes (space group within the point group) | data | -S |
the reference's | a different choice → <prefix>_mr.* only |
(a) in p.mtz, (c) beside it |
| Hand (enantiomorph) | not decidable: SPACE_GROUP_ENANTIOMORPH= UNDETERMINED |
GIVEN |
GIVEN |
relabelled to the model's where it fits, past the anomalous veto: ASSUMED_FROM_MODEL |
(b) |
| Indexing ambiguity (merohedral reindex) | arbitrary on rotation (INDEXING_AMBIGUITY_OPERATORS=); on stills with -S the crystals made consistent from the data, otherwise mixed (warning) |
— | resolved against the reference | resolved by the rigid-body model where its lead beats its null, rotation; per image on stills with -C/-S. Never by molecular replacement |
(b) |
| Setting (which axis is a, b, c) | standard setting | -C's axis order; -S's symbol |
the reference's | the model's, where it fits and there is no -z (SETTING_SOURCE= MODEL) |
(b) |
| Twinning check (L-test, twin laws) | data | — | — | — | (a) |
Twin hypothesis <prefix>_twin_<SG>.* |
data (when the group may have absorbed a twin law) | — | — | the model may also ask for this merge; written then only where the model chooses it | (a) / (c) |
| Anomalous signal, Friedel handling | data (-A) |
— | — | — (the anomalous map is report-only, and the veto on the hand) | (a) |
| R-free test set | hash of the index | — | the reference's flags, where it matches | — (drawn again on the model's axes if the setting moved; never chosen by the model) | (a) |
Merged intensities and sigmas in p.mtz |
data | — | — | — | (a) |
p.mtz space-group label |
data | yours | the reference's | hand relabel only | (b) |
<prefix>_mr.mtz/.cif, <prefix>_mr_model.* |
not written | — | — | another group or the twin subgroup the model chose | (c) |
Maps, <prefix>_model.*, section 5 of the report |
not written | — | — | always, whatever the verdict | (d) |
Refinement, <prefix>_refined_*, <prefix>_blobs.json, REFINE_ keys |
not written | — | — | where the model was placed (not with --no-refine) |
(d) |
A run with --model, step by step
- The file is read and classified. PDB or mmCIF, by content. A prediction — ModelCIF's
quality metrics, or no cell and one B value per residue between 0 and 100 (an AlphaFold file) —
is told apart from an experimental structure (
MR_MODEL_KIND=). A model that cannot be read does not fail the run; the report saysMODEL_VALIDATION= NOT_PERFORMEDand carries a warning. - The data are processed without the model. Indexing, integration, scaling, the space-group
search, the resolution cut, the error model, outlier rejection: none of them reads the model
(on serial stills with
-C/-S, see the next section). The only thing the model asks for here is one extra merge: the merge in a subgroup (the twin hypothesis) where the indexed lattice's group was promoted by a metric re-ask, or where the model's own group has fewer rotations than the adopted one — so that the model can be scored there. That merge is written only if the model chooses it. - Rigid-body placement (experimental structures, not with
--mr). The model is put into the data's cell and refined as one rigid body on the working reflections; the step is kept only if it lowers R-free. A model written in another description of the lattice (other axes, other centring) is first put into the data's (MODEL_CHANGE_OF_BASIS=). - The null. The same model is turned to nine random orientations about its own centroid and
placed the same way. The real fit has to beat them by 4.51 of their standard deviations on
R-work (3σ allowing for the null being nine draws):
MODEL_FIT= ACCEPTED,MODEL_FIT_SIGMA=. OtherwiseREJECTED, and the model decides nothing. There is no fixed threshold on R (§14.5). - What an accepted model may do (class b), applied to the reflections before they are written:
- hand — where the data's group is the enantiomorph of the model's, the files take the
model's group as their label. No reflection moves and I(+)/I(-) stay as measured. Vetoed where
the anomalous difference map read at the model's atoms is inverted (warning
MODEL_HAND); - indexing — where the lattice admits alternative indexings and no
-zalready settled it, the candidate with the lowest R-free, if its lead over the runner-up beats the lead a random placement of the same model takes (MODEL_INDEXING_MARGIN_SIGMA=≥ 3). The data are reindexed into the model's frame, never the model into the data's; - setting — a model written on other axes takes the files to its setting
(
SETTING_SOURCE= MODEL), so that reflections and coordinates can be used together.
- hand — where the data's group is the enantiomorph of the model's, the files take the
model's group as their label. No reflection moves and I(+)/I(-) stay as measured. Vetoed where
the anomalous difference map read at the model's atoms is inverted (warning
- The twin hypothesis. Where a subgroup merge exists, the model is scored against it plainly
and as twinned by its law. The maps come from whichever merge the model fits better on R-free
(
MAPS_FROM=). Where the model also fits there against its null, it has chosen the subgroup, and that merge is written again as<prefix>_mr.mtz/.cif(class c). - Molecular replacement, for a prediction, with
--mr, or for an experimental structure the rigid body did not place (in the answer's merge or in the subgroup's). As many copies as the cell most probably holds are searched for in every space group the merge cannot tell apart and in the twin hypothesis' subgroup, and the choice has its own null.MR_DECISION=is one of:AGREES(the answer's group — nothing changes);ENANTIOMORPH(the answer's files take the hand label, past the same anomalous veto, elseENANTIOMORPH_VETOED);SPACE_GROUPorTWIN_SUBGROUP(written as<prefix>_mr.*only);UNDECIDEDorNOT_PLACED(nothing is written). Molecular replacement never reindexes and never setsMODEL_FIT. Not placing a model is a result, not a warning (Molecular replacement). - Validation and files. R-factors, CC(model, data), maps and the placed model are computed
for the placement that was kept, against the merge it was chosen on, and written:
<prefix>_maps.mtz(map coefficients),_model.cif/.pdb— or the<prefix>_mr_*equivalents where the choice was<prefix>_mr. A rejected model gets its maps too: that is the case where looking at it in the density is most useful. A placed model is then refined (<prefix>_refined_model.cif/.pdb,_refined_maps.mtz,_blobs.json). - The reflection files —
<prefix>.mtz,.cif,.hkl,_unmerged.mtz— are written last, carrying only the class-(b) labels from step 5 and the hand from step 7.
MODEL_PLACEMENT= summarises steps 3–7: RIGID_BODY, MR or NOT_PLACED.
In --mode scale, --model is steps 3–5 only: the rigid body, its null, and the hand and indexing
it may settle. There is no molecular replacement, no setting change, no twin hypothesis and no refinement, and
--mr is refused.
Serial stills: the one place a model acts before the merge
On serial stills each crystal is indexed on its own, so where the lattice admits two indexings, each
crystal lands in one of them at random and a merge averages reflections that are not equivalent.
That has to be settled per crystal, at integration time — by the time there is a merge, the hands
are already mixed. A reference dataset (-z) is the tool for it. Without one, --model given
together with -C and -S supplies the reference intensities instead: structure factors computed
from the model, against which each crystal's intensities are correlated to choose its indexing.
This is the only point at which a model affects which observations are merged together, and it is
not gated by the null — there is no merge yet to test the model on. It chooses only between
indexings that are equally valid for each crystal; a model that does not belong to the crystal
carries no information to choose with, so the crystals stay mixed, as they would without a model.
Where the lattice is only pseudo-merohedral and the crystals' own cells already say which indexing is
which, the cells win: a crystal its cell anchors keeps that indexing, and the model decides
only the crystals whose cell cannot tell.
After the merge, the model is validated as in the steps above. Without -C and -S, or on rotation
data, the model is not used before the merge at all.
Without a model or reference, a stills run given -S settles the same question from the data: it
cannot say which indexing is the true one, but it can say which crystals share one, and puts each
crystal into the indexing in which it agrees best with all the others, a crystal its own cell decides
again keeping that indexing (INDEXING_AMBIGUITY_RESOLVED= data). Without -S - the space group then
determined by the run, after the merge - nothing resolves it, and the run warns. With a model, that resolution
still runs, as a check only: AMBIGUITY_DATA_DISAGREES= in the report counts the crystals on which
the data alone would have chosen differently from the model.
Where a model can touch the space group
Spelled out, because this is the question that matters most:
- The space group in
<prefix>.mtzis decided without the model — by the space-group search, or by you (-S,-z). The model is read after that decision. - The model can change one thing about that group: its hand, between the two members of an
enantiomorphic pair (P3₁/P3₂, P4₁2₁2/P4₃2₁2, P6₁22/P6₅22, …). The intensities are identical for
both, so this is the one group choice no data can make; a model that fits carries chemical
knowledge (the hand of its amino acids) the intensities do not. It is a label: no reflection
moves. It applies to a group you gave with
-Sor-ztoo. The report saysSPACE_GROUP_ENANTIOMORPH= ASSUMED_FROM_MODEL— assumed, not determined — also where the model simply agrees with the hand the run wrote. - Everything else the model prefers goes beside the answer. Other screw axes with the same
rotations (e.g. P2 against P2₁ where the absences did not decide), or the twin hypothesis'
subgroup:
<prefix>_mr.mtz/.cif. The model never changes the point group, the lattice or the centring of<prefix>.mtz.
The safeguards behind 2 and 3:
- The null. The model must beat its own random placements by 4.51 sd, rigid body and molecular
replacement alike, before it may relabel anything or have a
<prefix>_mrfile written. - The anomalous veto. The hand is taken only where the anomalous difference map, read at the
model's atoms, is not inverted. The veto can act only where the merge kept the Bijvoet split —
every rotation merge does; serial stills only with
-A. R-free cannot see the hand at all. - The indexing margin. An indexing is taken only where the model's preference beats what a randomly placed copy of it would show.
- Molecular replacement's own rules. Its choice of group has its own null (
NULL_SIGMA); a placement must pack (CLASH,MATTHEWS); a choice that translational NCS could explain, or one where the merge's own group was searched to fewer copies, is leftUNDECIDED; a single domain of a prediction cut by its PAE can only confirm the answer's group, never change it; in P1 the first copy's origin is fixed and it counts only on its rotation function. The details are in Molecular replacement. - The wrong-model test. Rugnux's validation battery runs a set of datasets once without a model
and once with each of five models of unrelated proteins (similar size, unrelated sequence and
fold, and for the enantiomorphic sets a model in the other hand), and requires the reflection data
and every report key outside section 5 to be identical,
MODEL_FITnot accepted, molecular replacement not placing the model and no model decision taken ("The wrong-model null" intools/battery/README.md; see also Tests).
Refinement
Giving a model is the opt-in: wherever the model is placed - it fits where it is, or molecular replacement
placed it - rugnux refines it against the merge it was placed on; a model that is not placed is not refined.
A run that should not spend the time is given no model; --no-refine skips the refinement of a placed model
(for debugging). It takes seconds on a GPU and typically 2-15 minutes on the CPU (no GPU, or --no-gpu).
Refinement comes after every processing decision and changes none of them: R_WORK, R_FREE and every
other key outside REFINE_ stay those of the model as placed, and <prefix>.mtz, <prefix>_model.cif and
<prefix>_maps.mtz are the same as they would be without it. The broker's live mode refines an uploaded model
in the same way (see Rugnux live mode).
What the dictionary restrains is fixed at build time: the 20 standard amino acids and MSE, the DNA/RNA
nucleotides and the common modified RNA nucleotides (PSU, 5MC, 1MA, OMG, OMC, OMU, H2U, 2MG, M2G, 7MG, 5MU),
ATP/ADP/AMP/GTP/GDP/GMP, the cofactors NAD, NAI, NAP, NDP, FAD, FMN, HEM, HEC, SAM and SAH, dioxygen (OXY),
DMSO, PEG fragments (PEG, PGE, PG4, 1PE), glycerol, ethylene glycol, sulfate, phosphate, acetate, MPD, water
and common ions (a subset of the CCP4 Monomer Library compiled into
rugnux; nothing is read from disk). Any other residue is held fixed: it stays in the structure factors and
the bulk-solvent mask, and the refined atoms are kept from clashing with it; REFINE_RESIDUES_FIXED_NAMES
lists them. The dictionary's hydrogens are added and ride on their parents. Metal ions, and the iron of a heme, are
restrained to the donor atoms (O, N, S) the model's LINK / _struct_conn records bind them to: by the dictionary's own Zn and Fe
links to Cys and His where they apply, otherwise at the library's typical distance for the metal, the donor
element and the number of such records of the ion (Mg-O 2.07 A, Na-O 2.42 A, K-O 2.82 A at their
usual coordination). A contact without a record, or one to a symmetry mate, is not restrained as a bond. The B factors of bonded atoms and of
atoms in contact (waters included) are restrained to each other in proportion to their size. The weights of
the X-ray term against the geometry and against that B restraint are chosen by R-free among short
refinements whose bond and angle rmsd stay within 0.02 A and 2 degrees: three weights from the placed
model, each with the B factors held and refined; then a trajectory of further short refinements from the
best, each trying the weights as they are and changed, taken while R-free or R-work falls beyond 0.001 and
R-free stays within 0.001 of its best; then a short final ladder, kept only where it lowers R-free beyond
0.001. The free set is the merge's own, so the refined R-free judges the refinement against reflections it
never fitted - but where the model was deposited, those reflections were most likely its depositor's working
reflections, and a refinement against new data then moves R-free up from the unrefined value rather than
down. R_WORK/R_FREE of the model as given are never replaced. On a GPU a refinement takes seconds; the CPU
engine computes the same and is several times slower.
How a model is scored. R-work, R-free and the likelihood target of every stage - the model as given
(REFINE_R_WORK_START) as much as the refined one - come from one model of the structure factors,
Fmodel = k exp(-sTBs/4) (Fcalc + ksol exp(-Bsols2/4) Fmask):
the atoms with the dictionary's riding hydrogens and their B made isotropic; a flat bulk solvent masked with the
REFMAC radii (probe 1.0 A, shrink 0.8 A) on a grid of its own, at most 0.4 A apart, so that the shrink acts at any
resolution; ksol and Bsol searched over the physical range on the working reflections, and the
overall scale and anisotropic B fitted to them by robust least squares, so that a few reflections the model cannot
explain (a beam-stop shadow, an ice ring, a missing part of the model) do not set the scale of the rest. On a set of
fourteen depositions from 1.1 to 3.6 A this gives REFMAC 5.8's 0-cycle R-work and R-free on the same model and data
within 0.002 on twelve, and within 0.006 on the other two (tools/battery/refine_parity.py runs the comparison;
--refine-score-only prints each term under REFINE_SCORE_, see --mode refine).
The algorithms - the target and its gradient, the likelihood's parameters, the restraints, the minimiser and the weight trials - are described in §15, and the keys in The results report.
What it does not do yet. The refinement is a first, automatic pass on a placed model, not a replacement for a refinement program. Not implemented yet:
- custom ligands or a monomer library beyond the embedded compounds above: other residues are held fixed (they stay in Fcalc, the solvent mask and the clash terms), and no restraints are generated for a new ligand;
- TLS;
- anisotropic ADPs: an ANISOU record is turned into its isotropic B, and only isotropic B is refined;
- NCS restraints;
- twin refinement (a twinned target);
- water building or removal (the blobs only point at candidates);
- occupancy refinement and new alternative conformations (existing ones are refined in place);
- rotamer and peptide-flip search, Ramachandran restraints;
- real-space refinement, jelly-body and reference-model restraints;
- a Metal (Apple GPU) path: on macOS the refinement runs on the CPU;
- validation beyond the geometry's rmsd and clash count: no MolProbity-style report.
Refining a model against structure factors (rugnux --mode refine)
--mode refine refines a model against merged structure factors that already exist - processed elsewhere, a
deposition's, or a rugnux .mtz - and processes nothing:
rugnux --mode refine --model model.cif -o refined data.mtz
The input is an MTZ or a structure-factor mmCIF (as the PDB distributes them), gzipped or not, recognised by
content; the model is PDB or mmCIF, taken as placed in the data's cell and refined in the data's space group. The
columns are found, not named: amplitudes with their sigmas (F/SIGF, FP/SIGFP, FOBS, ...; calculated amplitudes and
map coefficients such as FC, FWT, DELFWT are passed over) where the file has them, else intensities (IMEAN/SIGIMEAN,
I/SIGI, ...) made into amplitudes by French-Wilson, else the mean of the anomalous pairs. The free set is read from
a FreeR_flag-like column (flag 0 is the test set; where 0 marks the majority, as phenix and CNS write it, the
minority value is); a file without one gets rugnux's own 5 % set. The refinement is the one run after a merge on a placed model, and
writes the same files - <prefix>_refined_model.cif/.pdb, <prefix>_refined_maps.mtz, <prefix>_blobs.json - with
the REFINE_ keys in <prefix>_report.txt, beside REFINE_FRENCH_WILSON= and REFINE_FREE_FLAGS= FILE|GENERATED
saying what had to be made of the input.
| Option | Description |
|---|---|
--model <file> |
The model (required; it can also be given among the inputs, told by content) |
--refine-score-only |
Score the model as given and refine nothing; every term of the score - R-work and R-free overall and in 20 resolution shells, the overall scale, B and anisotropic B, ksol and Bsol, the likelihood over the working and the free reflections and its figure of merit over the working reflections, and the geometry's rmsd and rmsZ - under REFINE_SCORE_ keys (--developer prints them all) |
--refine-d-min <A> |
Leave out reflections finer than this |
Free set and model bias
How the free set is made. Rugnux draws its own test set (5 %, raised towards about 500 reflections on small data but never above 10 %):
each reflection's flag is a hash of its index, taken over its orbit under the lattice's metric
symmetry, so symmetry mates, Friedel mates and twin-law mates share a flag, every dataset of one
crystal form written on the same axes gets the same set, and the set does not depend on the space
group a file is merged in
(R-free test-set flags). The model
plays no part in it. Where a fitting model moves the files to its setting, the set is drawn again on
the new axes by the same rule; an indexing-ambiguity reindex leaves it unchanged, since twin-law
mates share a flag. With -z, the reference's flags are inherited where the merge matches it
(REFERENCE_FREE_FLAGS_INHERITED=).
What is fitted on which reflections. Everything fitted to place and scale the model — the
overall scale, the anisotropic B, the bulk solvent, the rigid body — is fitted on the working
reflections only. The free set is then used for a few discrete selections: whether the rigid-body
step is kept, which indexing wins, and in molecular replacement which placement and which group.
Each of those leaves R-free very slightly optimistic where it fires. For R_WORK/R_FREE the
coordinates, B-factors and occupancies of the model are never refined; the refinement above refines them on
the side, under REFINE_ keys of its own.
What R_FREE means here — and what it does not. It is the model as placed, scored on Rugnux's
free set: a check that the model fits these data, and a trend number between datasets of one
campaign. It is not a refinement R-free, and it is not cross-validated against the model's own
history:
- A deposited model was refined by its depositor against their working set. Rugnux's free set is a different 5 %, so about 95 % of Rugnux's free reflections were in the working set the model was refined on. R-free here therefore reads close to R-work (the two differ by a median 0.001 over our corpus), and it is optimistic in the usual model-bias sense.
- To score a deposited model on its own test set, give its structure factors as the reference
(
-z 1abc-sf.cif): the depositor's free flags are then inherited. But-zis an assertion — it also fixes the space group and the cell — so that is no longer a de-novo run. - A predicted model has not been refined against any diffraction data, so it carries no such memory; what remains is the selection effect above.
- For comparing two runs against one model, read
R_MODEL_SHELL_SCALED, notR_FREE(Comparing two runs).
What cannot leak, and what can. The model never enters the merged intensities, their sigmas, the
choice of reflections, the resolution cut or the weights, so <prefix>.mtz carries no model bias in
its numbers — only the class-(b) labels. The maps are model-phased, and carry the model bias any
model-phased map carries: read the mFo−DFc map, and treat density that only the 2mFo−DFc map shows
with the usual care. CC(model, data) is read in one direction only: a confirmed shell is evidence for
the data, a weak shell says nothing about them, and nothing acts on it.
Practical guidance
When to give a model.
- You have a structure of this crystal form (a fragment campaign, a ligand soak, a reference
structure): give it with
--model. You get maps, a placed model and Rugnux's own first refinement of it, the indexing of every dataset brought into the model's, and the hand fixed — the things that make a campaign's files directly comparable. A wrong or rejected model costs a few seconds and changes nothing. - You have a structure of the same protein in another crystal form:
--modeltoo. The rigid body will not place it, and molecular replacement is tried automatically. - You have only a prediction (AlphaFold, ModelCIF):
--model(or--mr). Download the PAE JSON with it and keep its AlphaFold DB file name, so it is cut into domains when the whole model does not place. A prediction is good for: confirming the space group, naming the hand where the data cannot, offering a twinned or lower-symmetry alternative where the data are ambiguous, and a first set of maps. It is not a refined model, and its R-factors read accordingly. - You have a previous dataset of the same crystal form and want the same indexing and free set:
-z, not--model. It is the stronger tool for the indexing ambiguity and the only one that carries a free set over, but it pins the space group and cell. - You want to compare with another program's statistics:
--report-resolution, which changes nothing else.
Which file to use for what.
| File | What it is | Use it for |
|---|---|---|
<prefix>.mtz / .cif / .hkl |
The answer: the data's space group, with only the class-(b) labels from the model | Refinement, deposition, everything by default |
<prefix>_model.cif / .pdb, <prefix>_maps.mtz |
The model as placed, in the frame of <prefix>.mtz, and its map coefficients (Coot/PyMOL open them as maps) |
Starting refinement (REFMAC5, phenix.refine), PanDDA, Coot |
<prefix>_refined_model.cif/.pdb, <prefix>_refined_maps.mtz, <prefix>_blobs.json |
The placed model refined by rugnux, its map coefficients, and the unexplained blobs of its difference map | A first look at the refined density and at what the model leaves unexplained; continuing refinement |
<prefix>_twin_<SG>.mtz / .cif |
The same data merged in a subgroup with a twin law — written from the data, model or not | Twin refinement, where you suspect the higher group absorbed a twin law |
<prefix>_mr.mtz / .cif, <prefix>_mr_model.pdb / .cif, <prefix>_mr_* maps |
The merge in the space group the model chose (MODEL_CHOICE_SPACE_GROUP=), and the model placed there |
Refinement if you accept the model's choice; compare R-free against the answer's |
<prefix>_mr and <prefix>_twin_<SG> can describe the same subgroup; the _mr file then also
carries the model placed in it.
Reading section 5 of the report, in order:
MODEL_VALIDATION=— did it run? IfNOT_PERFORMED,MODEL_VALIDATION_REASON=says why.MODEL_PLACEMENT=— how the model was placed:RIGID_BODY,MRorNOT_PLACED.MODEL_FIT=,MODEL_FIT_SIGMA=— the rigid body's verdict against its null. Absent where the rigid body was not tried (a prediction,--mr).MODEL_DECISIONS_TAKEN=—NONEmeans the reflection files are what a run without a model writes. OtherwiseENANTIOMORPH,INDEXING, both, orMR_ENANTIOMORPH, withMODEL_ENANTIOMORPH_ADOPTED=andMODEL_INDEXING_OPERATOR=.MODEL_CHOICE_FILE_PREFIX=,MODEL_CHOICE_SPACE_GROUP=,MR_DECISION=— whether a<prefix>_mrfile was written, and in which group.R_WORK=,R_FREE=,CC_MODEL_CONFIRMED_TO_D_MIN=, the anomalous sites — how well the placed model describes the data (class d).
Outside section 5, SPACE_GROUP_ENANTIOMORPH= (UNDETERMINED, GIVEN or ASSUMED_FROM_MODEL),
SETTING_SOURCE= and the presence or absence of INDEXING_AMBIGUITY_OPERATORS= say what the model
did to the answer's labels. Every key is described in The results report.
Related pages
- Advanced Rugnux — what
-zdoes, the indexing ambiguity, the setting the files are written in, and validating against a model. - The results report — model validation, molecular replacement, the twin hypothesis and the reference-range table.
- Data analysis: space group and validation — the space-group search (§13) and the model-validation algorithms (§14).
- Reference data and R-free test-set flags.