rugnux: a placed model is refined by default - giving a model is the opt-in

Refinement (Refinement.h) now runs whenever --model or --mr places the model (rigid body fits, or
molecular replacement placed it), on the GPU or the CPU: a run that should not spend the time is
given no model. --no-refine skips it (for debugging); --refine is still accepted and ignored. A
model that was not placed (rejected by the rigid body, not placed by MR - e.g. the battery's
wrong-model null) is no longer refined.

Processing-inert as before: it runs after every decision; R_WORK/R_FREE and every key outside
REFINE_ (section 5), and <prefix>.mtz / _model.cif / _maps.mtz, are unchanged. The block moved
after the placement notes so they stay on the step that placed the model. The viewer's processing
jobs refine too. The equivalent command line gives --no-refine back where refinement was off.

Battery: score.py reads keys exactly, so REFINE_R_FREE cannot alias R_FREE; new report-only
fields rfree_refined, rwork_refined, refine_s (part of rugnux_wall_s), shown in the per-set table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit b0bb741831f517b553e59152b3b482a3f75d953a)

docs/ACKNOWLEDGEMENT.md

@@ -518,7 +518,7 @@ molecular replacement; the error itself is AlphaFold's: J. Jumper, R. Evans, A.
 accurate protein structure prediction with AlphaFold" (2021), Nature 596, 583-589
 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2).
 
-**Structure refinement** (`--refine`) — the placed model is refined against the maximum-likelihood
+**Structure refinement** (of a placed model) — the placed model is refined against the maximum-likelihood
 amplitude target, the Rice distribution of an observed amplitude given the model's (its centric
 counterpart for a centric reflection): N. S. Pannu and R. J. Read, "Improved structure refinement
 through maximum likelihood" (1996), Acta Cryst. A52, 659-668
@@ -577,7 +577,7 @@ predicted distances around metal sites in proteins" (2006), Acta Cryst. D62, 678
 nucleic acids, F. Leonarski, L. D'Ascenzo and P. Auffinger, "Mg2+ ions: do they bind to nucleobase
 nitrogens?" (2017), Nucleic Acids Res. 45, 987-1004 [doi:10.1093/nar/gkw1175](https://doi.org/10.1093/nar/gkw1175).
 
-**Unmodelled density** — the blobs `--refine` lists are the connected regions of the difference map above
+**Unmodelled density** — the blobs the refinement lists are the connected regions of the difference map above
 a cutoff, found by flood fill as Coot's "unmodelled blobs" finds them, in GEMMI's implementation
 (`find_blobs_by_flood_fill`): P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, "Features and development
 of Coot" (2010), Acta Cryst. D66, 486-501 [doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493).

docs/CPU_DATA_ANALYSIS.md

@@ -18,7 +18,7 @@ This document describes the crystallographic algorithms implemented in Jungfrauj
 12. auxiliary statistics (Wilson plot, ⟨I/σ(I)⟩, CC1/2, CCref),
 13. amplitude estimation (French–Wilson) and R-free test-set flagging,
 14. optional model-based validation: rigid-body placement of a supplied model (molecular replacement where the rigid body does not place it, or for a predicted model), R-free against it, sigma_A-weighted 2mFo−DFc / mFo−DFc electron-density maps, and an anomalous difference map with the strongest anomalous sites named.
-15. optional restrained refinement of the placed model (`--refine`): rigid bodies and group B, then positions and B against a maximum-likelihood amplitude target with the dictionary's geometry, the X-ray weight chosen by R-free within geometry bounds, and the unexplained blobs of the refined difference map.
+15. restrained refinement of the placed model (wherever a model is placed): rigid bodies and group B, then positions and B against a maximum-likelihood amplitude target with the dictionary's geometry, the X-ray weight chosen by R-free within geometry bounds, and the unexplained blobs of the refined difference map.
 
 
 The reference is split into four parts, in pipeline order; the section numbers run continuously

docs/RUGNUX_ADVANCED.md

@@ -472,6 +472,8 @@ Scaling and merging:
 | `--reference-column <label>` | Reference column to use, by MTZ label — an mmCIF under the labels `gemmi cif2mtz` gives it (default: auto — F-model, else IMEAN/I/…, else FP/FOBS/F) |
 | `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](#validating-against-a-model-rugnux---model)). An experimental model is placed as a rigid body, and by molecular replacement where the rigid body does not place it; a prediction (AlphaFold or ModelCIF) goes to molecular replacement straight away (`MODEL_PLACEMENT=`). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](#the-indexing-ambiguity) where no `-z` did; another space group or the twin hypothesis' subgroup it chooses is written beside the answer as `<prefix>_mr.mtz`/`.cif`. For serial stills given `-C` / `-S`, the model's structure factors become the per-image reference |
 | `--mr <file>` | As `--model`, but placed by molecular replacement straight away, without trying the rigid body first. `--mode mx` only; not with `--model` |
+| `--no-refine` | Do not refine the placed model (for debugging) |
+| `--refine` | Accepted and ignored: a model `--model` or `--mr` places is refined by default against the merge it was placed on, after every decision and changing none (`REFINE_` keys, `<prefix>_refined_model.cif`, `_refined_maps.mtz`, `_blobs.json`; see [Refinement](RUGNUX_MODEL_AND_REFERENCE.md#refinement)). Seconds on a GPU, typically 2-15 minutes on the CPU |
 | `--write-process-h5` | Also write the (large) `_process.h5` when merging (default: only `.mtz`/`.cif`) |
 | `--developer` | Write the full `<prefix>_report.txt`: the pipeline-internal keys and the long explanations the default report leaves out. Nothing is computed differently — the same report, rendered in full (see [The results report](RUGNUX_REPORT.md#the-developer-report)) |
 | `--finalist-ledger` | Report the full-resolution evidence for each space group the search considered, not only the one it adopted, and the twin-immune zone evidence of the adopted group's 2-folds (report-only; the decision is unchanged). Both are read on a merge of the observations in P1 that only this option makes, so it costs one extra merge; rotation only, and not with a fixed centred `-S` |

docs/RUGNUX_MODEL_AND_REFERENCE.md

@@ -48,9 +48,8 @@ the rest walks through a run step by step.
 | Option | What you give | What Rugnux uses it for | Can it change `<prefix>.mtz`? |
 |---|---|---|---|
 | *(none)* | — | Everything from the data | — |
-| `--model <file>` | An atomic model, PDB or mmCIF, gzipped or not, recognised by content: an experimental structure or a prediction | Validation after the merge (R-work/R-free, CC(model, data), maps, the placed model); a rigid-body placement checked against a null, with molecular replacement where the rigid body does not place it, and straight away for a prediction | Labels only (hand, indexing, axes), and only where the model fits. Other space groups → `<prefix>_mr.*` |
+| `--model <file>` | An atomic model, PDB or mmCIF, gzipped or not, recognised by content: an experimental structure or a prediction | Validation after the merge (R-work/R-free, CC(model, data), maps, the placed model); a rigid-body placement checked against a null, with molecular replacement where the rigid body does not place it, and straight away for a prediction; then [refinement](#refinement) of the placed model (`REFINE_` keys, `<prefix>_refined_model.cif`, `_refined_maps.mtz`, `_blobs.json`), which decides nothing | Labels only (hand, indexing, axes), and only where the model fits. Other space groups → `<prefix>_mr.*` |
 | `--mr <file>` | The same kinds of model | As `--model`, but molecular replacement straight away, without the rigid body. `--mode mx` only, not together with `--model` | Hand label only. Other space groups → `<prefix>_mr.*` |
-| `--refine` | Nothing more (with `--model` or `--mr`) | Refines the placed model against the merge it was placed on - each chain as a rigid body and a B per residue, then every atom's position (and, where R-free says so, its B) restrained by the dictionary carried in rugnux, against a maximum-likelihood target - and lists the unexplained blobs of its difference map. Writes `<prefix>_refined_model.cif`, `<prefix>_refined_maps.mtz`, `<prefix>_blobs.json`; reported under `REFINE_` keys | No - it runs after everything is decided and decides nothing |
 | PAE file beside a prediction | `AF-<id>-F1-predicted_aligned_error_v<n>.json` next to `AF-<id>-F1-model_v<n>.cif` (read automatically, no option) | Cuts the prediction into rigid domains for molecular replacement | No (a domain alone can only confirm the answer's group) |
 | `-z, --reference <file>` (`--reference-mtz`) | Merged data of the same crystal form: an MTZ, or a PDB structure-factor mmCIF (`-sf.cif`) | Fixes the space group (in its own setting) and the cell (softly), puts the merge on its axes, resolves the indexing ambiguity, hands over its R-free set, reports CC<sub>ref</sub> on stills. Never a scale anchor | **Yes**: the space group, the axes, the indexing and the free set are the reference's |
 | `--reference-column <label>` | An MTZ column label | Which column of the reference is read | Through `-z` only |
@@ -235,7 +234,16 @@ The safeguards behind 2 and 3:
   replacement not placing the model and no model decision taken
   ("The wrong-model null" in `tools/battery/README.md`; see also [Tests](TESTS.md)).
 
-## Refinement (`--refine`)
+## Refinement
+
+Giving a model is the opt-in: wherever the model is placed - it fits where it is, or molecular replacement
+placed it - rugnux refines it against the merge it was placed on; a model that is not placed is not refined.
+A run that should not spend the time is given no model; `--no-refine` skips the refinement of a placed model
+(for debugging). It takes seconds on a GPU and typically 2-15 minutes on the CPU (no GPU, or `--no-gpu`).
+`--refine` is still accepted and changes nothing. Refinement comes after every processing decision and changes none of them: `R_WORK`, `R_FREE` and every
+other key outside `REFINE_` stay those of the model as placed, and `<prefix>.mtz`, `<prefix>_model.cif` and
+`<prefix>_maps.mtz` are the same as they would be without it. The broker's live mode refines an uploaded model
+in the same way (see [Rugnux live mode](JFJOCH_BROKER.md#rugnux-live-mode)).
 
 What the dictionary restrains is fixed at build time: the 20 standard amino acids and MSE, the DNA/RNA
 nucleotides, ATP/ADP/AMP/GTP/GDP/GMP, DMSO, PEG fragments (PEG, PGE, PG4, 1PE), glycerol, ethylene glycol,
@@ -253,7 +261,7 @@ refinements whose bond and angle rmsd stay within 0.02 A and 2 degrees: three we
 model, each with the B factors held and refined; then a trajectory of further short refinements from the
 best, each trying the weights as they are and changed, taken while R-free or R-work falls beyond 0.001 and
 R-free stays within 0.001 of its best; then a short final ladder, kept only where it lowers R-free beyond
-0.001. The free set is the merge's own, so R-free after `--refine` judges the refinement against reflections it
+0.001. The free set is the merge's own, so the refined R-free judges the refinement against reflections it
 never fitted - but where the model was deposited, those reflections were most likely its depositor's working
 reflections, and a refinement against new data then moves R-free up from the unrefined value rather than
 down. `R_WORK`/`R_FREE` of the model as given are never replaced. On a GPU a refinement takes seconds; the CPU
@@ -314,8 +322,9 @@ mates share a flag. With `-z`, the reference's flags are inherited where the mer
 overall scale, the anisotropic *B*, the bulk solvent, the rigid body — is fitted on the working
 reflections only. The free set is then used for a few discrete *selections*: whether the rigid-body
 step is kept, which indexing wins, and in molecular replacement which placement and which group.
-Each of those leaves R-free very slightly optimistic where it fires. The coordinates, B-factors and
-occupancies of the model are never refined.
+Each of those leaves R-free very slightly optimistic where it fires. For `R_WORK`/`R_FREE` the
+coordinates, B-factors and occupancies of the model are never refined; the refinement above refines them on
+the side, under `REFINE_` keys of its own.
 
 **What `R_FREE` means here — and what it does not.** It is the model as placed, scored on Rugnux's
 free set: a check that the model fits these data, and a trend number between datasets of one
@@ -369,6 +378,7 @@ the data, a weak shell says nothing about them, and nothing acts on it.
 |---|---|---|
 | `<prefix>.mtz` / `.cif` / `.hkl` | The answer: the data's space group, with only the class-(b) labels from the model | Refinement, deposition, everything by default |
 | `<prefix>_model.cif` / `.pdb`, `<prefix>_maps.mtz` | The model as placed, in the frame of `<prefix>.mtz`, and its map coefficients (Coot/PyMOL open them as maps) | Starting refinement (REFMAC5, phenix.refine), PanDDA, Coot |
+| `<prefix>_refined_model.cif`, `<prefix>_refined_maps.mtz`, `<prefix>_blobs.json` | The placed model refined by rugnux, its map coefficients, and the unexplained blobs of its difference map | A first look at the refined density and at what the model leaves unexplained; continuing refinement |
 | `<prefix>_twin_<SG>.mtz` / `.cif` | The same data merged in a subgroup with a twin law — written from the data, model or not | Twin refinement, where you suspect the higher group absorbed a twin law |
 | `<prefix>_mr.mtz` / `.cif`, `<prefix>_mr_model.pdb` / `.cif`, `<prefix>_mr_*` maps | The merge in the space group the model chose (`MODEL_CHOICE_SPACE_GROUP=`), and the model placed there | Refinement *if* you accept the model's choice; compare R-free against the answer's |
 

rugnux/ResultReport.cpp

@@ -455,7 +455,7 @@ namespace {
         return h;
     }
 
-    // The refinement's keys (Refinement.h): REFINE_ of --refine and of --mode refine, with every term of the
+    // The refinement's keys (Refinement.h): REFINE_ of the refinement of a placed model and of --mode refine, with every term of the
     // score of the model as given under REFINE_SCORE_.
     void AddRefinementKeys(ReportSection &s, const RefinementResult &rf, const std::string &output_prefix) {
         Add(s, Blank());
@@ -511,7 +511,7 @@ namespace {
                                    b.centroid.z, b.nearest_atom_distance, b.nearest_atom);
             }
             Add(s, Prose(fmt::format(
-                    " The placed model was refined against these data (--refine): each chain as a rigid body and\n"
+                    " The placed model was refined against these data: each chain as a rigid body and\n"
                     " one B per residue, then every atom restrained by the dictionary carried in rugnux, the\n"
                     " X-ray weight chosen by R-free with the geometry held within bounds. REFINE_ keys describe\n"
                     " the refined model, which is written as {}_refined_model.cif with its maps; R_WORK and\n"
@@ -2871,7 +2871,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
                         mp.t_rotation, mp.t_translation, mp.t_rescore, mp.t_null, mp.t_domains, mp.hand_check.seconds), true));
             }
         }
-        // --refine: the placed model refined (Refinement.h). Keys of their own: R_WORK / R_FREE above stay the
+        // The placed model refined (Refinement.h; always, where a model is placed). Keys of their own: R_WORK / R_FREE above stay the
         // unrefined model's.
         if (result.refinement.has_value())
             AddRefinementKeys(s, *result.refinement, output_prefix);

rugnux/Rugnux.h

@@ -261,10 +261,11 @@ struct ProcessConfig {
     // did not place, is searched for anyway.
     bool force_mr = false;
 
-    // --refine: the placed model refined against the merge it was placed on - each chain as a rigid body and a B
-    // per residue, then every atom restrained (Refinement.h) - and what it leaves in the difference map listed
-    // (DensityBlobs.h). Reported in keys of its own; it decides nothing about the processing or the files the
-    // run writes otherwise.
+    // A placed model refined against the merge it was placed on - each chain as a rigid body and a B per residue,
+    // then every atom restrained (Refinement.h) - and what it leaves in the difference map listed (DensityBlobs.h).
+    // Reported in keys of its own; it decides nothing about the processing or the files the run writes otherwise.
+    // On in the rugnux CLI (--no-refine turns it off), the viewer and the broker's live mode: giving a model is the
+    // opt-in.
     bool refine = false;
 
     // --finalist-ledger: report-only symmetry evidence table (and the twin-immune zones), folded from
@@ -411,7 +412,7 @@ struct RugnuxOutputs {
     std::shared_ptr<ModelMaps> model_prior_maps;              // the same at <prefix>_mr
     std::optional<PlacedModelOutput> placed_model;            // <prefix>_model.cif/.pdb
     std::optional<PlacedModelOutput> model_prior_placed_model;   // <prefix>_mr_model.cif/.pdb
-    std::optional<PlacedModelOutput> refined_model;           // <prefix>_refined_model.cif/.pdb (--refine)
+    std::optional<PlacedModelOutput> refined_model;           // <prefix>_refined_model.cif/.pdb (ProcessConfig::refine)
     std::shared_ptr<ModelMaps> refined_maps;                  // <prefix>_refined_maps.mtz
     std::optional<std::vector<DensityBlob>> blobs;            // <prefix>_blobs.json
     std::optional<MtzTable> unmerged;                         // <prefix>_unmerged.mtz
@@ -734,7 +735,7 @@ struct ProcessResult {
     std::string model_placement;
     // What molecular replacement found, where it ran; failure_reason says why where it could not.
     std::optional<MolecularReplacementResult> model_prior;
-    // What --refine made of the placed model; unset where it was not asked for or there was nothing placed.
+    // What refinement made of the placed model; unset where nothing was placed (or refine was off).
     std::optional<RefinementResult> refinement;
     // The file suffix the model's choice of another group or the twin hypothesis' subgroup is written under
     // (_mr), and that group; empty where nothing was.

rugnux/RugnuxCommandLine.cpp

@@ -86,8 +86,8 @@ std::string RugnuxCommandLine(const ProcessConfig &config,
         }
         if (config.model)
             add(config.force_mr ? "--mr" : "--model", config.model->name);
-        if (config.model && config.refine)
-            args.emplace_back("--refine");
+        if (config.model && !config.refine)
+            args.emplace_back("--no-refine");
     }
     if (config.start_image != 0)
         add("-s", std::to_string(config.start_image));

rugnux/RugnuxScaleMerge.cpp

@@ -4407,7 +4407,7 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
     std::optional<ModelPriorFile> model_prior_file;
     // The coordinates that go with the maps beside the answer's files, kept once the model is done.
     std::optional<PlacedModelOutput> placed_model_out;
-    // The merge that placement was made against, which --refine refines it against.
+    // The merge that placement was made against, which refinement refines it against.
     const std::vector<MergedReflection> *placed_merged = nullptr;
     // The twin hypothesis the model is asked about: its own subgroup's merge where it has one, the data's
     // otherwise - and whether that merge was made only for the model, so is written only where it chooses it.
@@ -4754,29 +4754,6 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
         if (model_prior_file)
             result.model_prior_file_space_group = model_prior_file->space_group.xhm();
         result.outputs->placed_model = placed_model_out;
-        // --refine: the placed model refined against the merge it was placed on, after everything above has been
-        // decided, so that nothing reads what it finds.
-        if (config_.refine && placed_model_out && placed_merged != nullptr && result.model_validation &&
-            result.model_validation->ok) {
-            phase("Refining the model");
-            Step("Refinement");
-            TimingMark mark(logger, "refinement");
-            RefinementSettings settings;
-            settings.nthreads = static_cast<size_t>(config_.nthreads);
-            settings.gpu_resident_bytes = rsm && rsm->OnGPU() ? rsm->DeviceBytes() : 0;
-            const PlacedModelOutput &p = *placed_model_out;
-            RefinementResult refined = RefineModel(*p.model, *placed_merged, gemmi::UnitCell(p.cell), p.space_group,
-                                                   experiment_.GetWavelength_A(), logger, settings);
-            if (refined.ok) {
-                result.outputs->refined_model = PlacedModelOutput{refined.model, p.cell, p.space_group};
-                result.outputs->refined_maps = refined.maps;
-                result.outputs->blobs = refined.blobs;
-                Note(fmt::format("refined: R-work {:.3f}, R-free {:.3f}, rmsd bonds {:.3f} A, {} blobs", refined.r_work,
-                                 refined.r_free, refined.geometry.rmsd_bonds, refined.blobs.size()));
-            } else
-                logger.Warning("Refinement did not run: {}", refined.failure_reason);
-            result.refinement = std::move(refined);
-        }
         // The outcome, as notes of the step that decided it
         if (result.model_prior) {
             const MolecularReplacementResult &mp = *result.model_prior;
@@ -4796,6 +4773,34 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
             if (v.rigid_body_applied)
                 Note(fmt::format("rigid body: {:.2f} deg, {:.2f} A", v.rigid_body_angle_deg, v.rigid_body_shift_A));
         }
+        // The placed model refined against the merge it was placed on, after everything above has been decided,
+        // so that nothing reads what it finds - and after the notes above, which belong to the step that placed
+        // the model. Only a model that was placed: one the rigid body rejected and molecular replacement did not
+        // place has nothing to refine. Left out, as the model is, on a cut.
+        const bool refine_cut = config_.fast_live && cancelled_;
+        if (refine_cut && config_.refine && result.model_placement != "NOT_PLACED")
+            Note("cut: the refinement is left out");
+        if (config_.refine && !refine_cut && result.model_placement != "NOT_PLACED" && placed_model_out
+            && placed_merged != nullptr && result.model_validation && result.model_validation->ok) {
+            phase("Refining the model");
+            Step("Refinement");
+            TimingMark mark(logger, "refinement");
+            RefinementSettings settings;
+            settings.nthreads = static_cast<size_t>(config_.nthreads);
+            settings.gpu_resident_bytes = rsm && rsm->OnGPU() ? rsm->DeviceBytes() : 0;
+            const PlacedModelOutput &p = *placed_model_out;
+            RefinementResult refined = RefineModel(*p.model, *placed_merged, gemmi::UnitCell(p.cell), p.space_group,
+                                                   experiment_.GetWavelength_A(), logger, settings);
+            if (refined.ok) {
+                result.outputs->refined_model = PlacedModelOutput{refined.model, p.cell, p.space_group};
+                result.outputs->refined_maps = refined.maps;
+                result.outputs->blobs = refined.blobs;
+                Note(fmt::format("refined: R-work {:.3f}, R-free {:.3f}, rmsd bonds {:.3f} A, {} blobs", refined.r_work,
+                                 refined.r_free, refined.geometry.rmsd_bonds, refined.blobs.size()));
+            } else
+                logger.Warning("Refinement did not run: {}", refined.failure_reason);
+            result.refinement = std::move(refined);
+        }
     }
 
     // The unmerged MTZ (kept with the other outputs, see there) is built from here on, beside the

rugnux/rugnux_cli.cpp

@@ -249,9 +249,10 @@ void print_usage() {
     std::cout << "   --scaling-iterations <num>      Cap on the per-frame scaling iterations; the loop stops earlier once the scales settle (default: 100)" << std::endl;
     std::cout << "   -z, --reference <file>          Reference data of the same crystal form: an MTZ, or a PDB structure-factor mmCIF (e.g. 1abc-sf.cif, first merged block with a usable column; R-free set from _refln.status f/o, else pdbx_r_free_flag). Gzipped or not, recognised by content. --reference-mtz is the same option" << std::endl;
     std::cout << "   --reference-column <label>      Reference column to use, by MTZ label; an mmCIF is read under the labels gemmi cif2mtz gives it (IMEAN, FP, I(+), FC, F-model, ...) (default: auto - F-model, else IMEAN/I, else FP/FOBS/F)" << std::endl;
-    std::cout << "   --model <file>                  After merging, validate vs this atomic model: R-free, sigma_A-weighted 2mFo-DFc/mFo-DFc maps, the model as placed (written beside them), and - where the merge kept the Bijvoet split - an anomalous difference map with the strongest anomalous scatterers named by the atom they sit on. PDB or mmCIF, gzipped or not, recognised by content rather than by extension. A prediction (AlphaFold or ModelCIF, pLDDT in the B column), or an experimental structure the rigid body does not place against its null, is placed by molecular replacement instead (see --mr)" << std::endl;
+    std::cout << "   --model <file>                  After merging, validate vs this atomic model: R-free, sigma_A-weighted 2mFo-DFc/mFo-DFc maps, the model as placed (written beside them), and - where the merge kept the Bijvoet split - an anomalous difference map with the strongest anomalous scatterers named by the atom they sit on. PDB or mmCIF, gzipped or not, recognised by content rather than by extension. A prediction (AlphaFold or ModelCIF, pLDDT in the B column), or an experimental structure the rigid body does not place against its null, is placed by molecular replacement instead (see --mr). A model that is placed is then refined against the merge it was placed on - each chain as a rigid body and one B per residue, then every atom's position and B restrained by the dictionary carried in rugnux against a maximum-likelihood target, the X-ray weight chosen by R-free within geometry bounds - and the unexplained blobs of its difference map are listed: <prefix>_refined_model.cif, <prefix>_refined_maps.mtz, <prefix>_blobs.json, REFINE_ keys in the report. R_WORK, R_FREE and every other key stay those of the model as placed; the refinement decides nothing about the processing. It costs seconds on a GPU and typically 2-15 minutes on the CPU (no GPU, or --no-gpu); --no-refine skips it" << std::endl;
     std::cout << "   --mr <file>                     As --model, but placed by molecular replacement straight away, without the rigid-body placement first: as many copies as the cell holds, in every space group the merge cannot tell apart and in the twin hypothesis' subgroup. A prediction is trimmed to the residues of pLDDT 70 and above, and where the AlphaFold DB's predicted-aligned-error JSON sits beside it under its own name it is cut into rigid domains; any other PDB or mmCIF is taken as an experimental structure, its polymer only (no waters, ligands, hydrogens or alternative conformers). Which of the two it is is read from the file. It may relabel the enantiomorph of the answer, and a different space group it chooses is written beside the answer as <prefix>_mr.mtz/.cif with the placed model <prefix>_mr_model.pdb. --mode mx only" << std::endl;
-    std::cout << "   --refine                        With --model or --mr: refine the placed model against the merge it was placed on - each chain as a rigid body and one B per residue, then every atom's position and B restrained by the dictionary carried in rugnux (standard amino acids and nucleotides, common ligands and additives, water and ions; anything else is held fixed) against a maximum-likelihood target, the X-ray weight chosen by R-free within geometry bounds - and list the unexplained blobs of its difference map. Writes <prefix>_refined_model.cif, <prefix>_refined_maps.mtz and <prefix>_blobs.json; reported under REFINE_ keys, never in place of the unrefined R-factors. Decides nothing about the processing" << std::endl;
+    std::cout << "   --no-refine                     Do not refine the placed model (for debugging): by default a model that --model or --mr places is refined (see --model)" << std::endl;
+    std::cout << "   --refine                        Accepted and ignored: a placed model is refined by default" << std::endl;
     std::cout << "   --export-unmerged               Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On by default, whenever there is an output prefix. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-captured-fraction (or --min-partiality) is not written, exactly as in the merge, since a full is what the reading program is told these rows are. Intensities carry the deterministic per-reflection corrections and nothing else - Lorentz and polarization in the LP column, the sensor's angle-dependent efficiency in QE and the flight path's attenuation in FLIGHT, so raw counts are I / LP * QE * FLIGHT - while the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
     std::cout << "   --no-export-unmerged            Do not write <prefix>_unmerged.mtz. It is the largest file a run produces, so a run whose observations are not going to another scaling program - a regression battery, say - can skip writing it" << std::endl;
     std::cout << "   --export-unmerged-partials      Write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summed here. Off by default and independent of --export-unmerged; both can be written in one run" << std::endl;
@@ -342,6 +343,7 @@ enum {
     OPT_CRYSTFEL_GEOMETRY,
     OPT_MR,
     OPT_REFINE,
+    OPT_NO_REFINE,
     OPT_EXPORT_UNMERGED,
     OPT_NO_EXPORT_UNMERGED,
     OPT_EXPORT_UNMERGED_PARTIALS,
@@ -419,6 +421,7 @@ static option long_options[] = {
     {"crystfel-geometry", required_argument, nullptr, OPT_CRYSTFEL_GEOMETRY},
     {"mr", required_argument, nullptr, OPT_MR},
     {"refine", no_argument, nullptr, OPT_REFINE},
+    {"no-refine", no_argument, nullptr, OPT_NO_REFINE},
     {"refine-score-only", no_argument, nullptr, OPT_REFINE_SCORE_ONLY},
     {"refine-d-min", required_argument, nullptr, OPT_REFINE_D_MIN},
     {"export-unmerged", no_argument, nullptr, OPT_EXPORT_UNMERGED},
@@ -1053,7 +1056,7 @@ static int RunRugnux(int argc, char **argv, bool no_gpu) {
     std::string crystfel_geometry;   // --crystfel-geometry: read the input as CrystFEL would with this geometry
     std::string model_pdb;           // --model: PDB to validate merged intensities against (R-free + maps)
     std::string mr_model;            // --mr: --model, placed by molecular replacement straight away
-    bool refine_model = false;       // --refine
+    bool refine_model = true;        // --no-refine: a placed model is refined unless it is given
     bool refine_score_only = false;  // --refine-score-only
     double refine_d_min = 0;         // --refine-d-min
     bool export_unmerged = true;            // on by default; --no-export-unmerged turns it off
@@ -1298,8 +1301,10 @@ static int RunRugnux(int argc, char **argv, bool no_gpu) {
         case OPT_MR:
             mr_model = optarg;
             break;
-        case OPT_REFINE:
-            refine_model = true;
+        case OPT_REFINE:   // a placed model is refined by default; kept for old command lines
+            break;
+        case OPT_NO_REFINE:
+            refine_model = false;
             break;
         case OPT_REFINE_SCORE_ONLY:
             refine_score_only = true;
@@ -1779,10 +1784,6 @@ static int RunRugnux(int argc, char **argv, bool no_gpu) {
         logger.Error("--mode refine refines a model: give it with --model");
         exit(EXIT_FAILURE);
     }
-    if (refine_model && mr_model.empty() && model_pdb.empty()) {
-        logger.Error("--refine refines a model: give it with --model or --mr");
-        exit(EXIT_FAILURE);
-    }
     // Read before anything runs: a file that holds no model ends the run here rather than after it.
     std::shared_ptr<const RugnuxModel> model;
     if (const std::string path = mr_model.empty() ? model_pdb : mr_model; !path.empty()) {
@@ -3603,7 +3604,7 @@ static int RunRugnux(int argc, char **argv, bool no_gpu) {
     config.export_unmerged_partials = export_unmerged_partials;
     config.model = model;
     config.force_mr = !mr_model.empty();
-    config.refine = refine_model;
+    config.refine = refine_model;   // a placed model is refined unless --no-refine
     // When merging, the merged reflections (.mtz/.cif) are the wanted output; skip the large
     // _process.h5 unless explicitly requested. Without merging, the _process.h5 is the only output.
     std::optional<ProcessFileSink> sink;

tests/RugnuxTest.cpp

@@ -340,6 +340,10 @@ TEST_CASE("RugnuxCommandLine_Full", "[process]") {
     model->name = "model.cif";
     config.model = model;
     CHECK(RugnuxCommandLine(config, x, "in.h5", "", false).find("--model model.cif") != std::string::npos);
+    // A placed model is refined by default: only --no-refine is given back.
+    CHECK(RugnuxCommandLine(config, x, "in.h5", "", false).find("--no-refine") != std::string::npos);
+    config.refine = true;
+    CHECK(RugnuxCommandLine(config, x, "in.h5", "", false).find("--no-refine") == std::string::npos);
     config.force_mr = true;
     const std::string mr_cmd = RugnuxCommandLine(config, x, "in.h5", "", false);
     CHECK(mr_cmd.find("--mr model.cif") != std::string::npos);

tools/battery/README.md

@@ -730,6 +730,7 @@ was run `--unforced`), so its forced XDS-arm rows are left out the same way.
 | `lowres_d`, `lowres_r_meas`, `refres_lowres_d`, `refres_lowres_r_meas`, `lowres_d_ref`, `lowres_r_meas_ref`, `lowres_r_meas_ratio` | the lowest-resolution shell's high-resolution limit and R_meas: rugnux's own table, the reference-range table, XDS's; and refres_lowres_r_meas / lowres_r_meas_ref |
 | `model`, `model_note` | open arm, and in-house sets with a `model`: the coordinates given to `--model`, or why there were none |
 | `rfree`, `rwork`, `cc_model`, `model_fit`, `rfree_deposited`, `rfree_ratio` | any row with a model: R_FREE, R_WORK, CC_MODEL_OVERALL and MODEL_FIT as rugnux reports them (placement-only, own free set: trend fields), the published R-free, and rfree / rfree_deposited |
+| `rfree_refined`, `rwork_refined`, `refine_s` | any row whose model was placed: REFINE_R_FREE, REFINE_R_WORK and REFINE_SECONDS - the placed model refined, which rugnux always does where it places a model. Reported, never scored; `rfree` and every other field stay the unrefined placement's. `refine_s` is included in `rugnux_wall_s`, so a baseline from before refinement was always on reads seconds slower per placed set on a GPU (minutes on the CPU) |
 | `anom_sigma` | any row with a model: ANOMALOUS_SCATTERER_MEAN_SIGMA, the anomalous difference map (F+ - F- on the model's phases) read at every model atom from phosphorus up (S of Met/Cys, metals, Cl, I, ...) and averaged, in map sigma - how much anomalous signal the merge carries; null where the merge kept no Bijvoet split or the model has no such atom |
 | `rmodel`, `rmodel_shell_scaled`, `radial_misfit` | any row with a model: R_MODEL, R_MODEL_SHELL_SCALED and MODEL_RADIAL_MISFIT - R over all the reflections, with and without one free scale per resolution shell, and how big that rescale was. `rmodel_shell_scaled` is the R that compares between runs (below) |
 | `refmac_rfree`, `refmac_rwork`, `refmac_rfree_depflags`, `refmac_rfree_depdata`, `refmac_rfree_ratio`, `refmac_twin`, `refmac_untwinned`, `refmac_status`, `refmac_reason` | the REFMAC check (`--model-check`, open arm; see its section for which R-free is which) |

tools/battery/report.py

@@ -673,6 +673,8 @@ def build(run_dir, baseline=None, allow_incomplete=False):
              ("dep R_free", "rfree_deposited"),
              ("R_free ratio", "rfree_ratio"),
              ("anom. sigma", "anom_sigma")] if any(r.get("rfree") for r in res) else []
+    model += [("refined R_free", "rfree_refined"), ("refine s", "refine_s")] \
+        if any(r.get("rfree_refined") for r in res) else []
     model += [("REFMAC R_free", "refmac_rfree_depflags"), ("REFMAC dep data", "refmac_rfree_depdata"),
               ("REFMAC ratio", "refmac_rfree_ratio")] if any(r.get("refmac_status") for r in res) else []
     model += [("dCC dep. outer", "dep_cc_delta_outer")] if any(r.get("dep_status") for r in res) else []
@@ -691,7 +693,7 @@ def build(run_dir, baseline=None, allow_incomplete=False):
                      pct(r.get("r_meas")), pct(r.get("lowres_r_meas")), f(r.get("cc_half"), "{:.3f}"),
                      f(r.get("isa"), "{:.1f}"),
                      f(r.get("isa_ref"), "{:.1f}"), pct(r.get("indexing_rate"), 0)]
-                    + [f(r.get(k), "{:.3f}") for _, k in model]
+                    + [f(r.get(k), "{:.1f}" if k == "refine_s" else "{:.3f}") for _, k in model]
                     + ([r.get("model_fit") or "-"] if model else [])
                     + [f(r.get("wall_s"), "{:.0f}"), note])
     doc.table(head, rows, num=range(5, len(head) - 1),

tools/battery/score.py

@@ -256,6 +256,10 @@ def judge(entry, rep, run_note):
         # the R to read in a delta table. radial_misfit says how much there was to take out.
         "rmodel": fnum(rep.get("R_MODEL")), "rmodel_shell_scaled": fnum(rep.get("R_MODEL_SHELL_SCALED")),
         "radial_misfit": fnum(rep.get("MODEL_RADIAL_MISFIT")),
+        # the placed model refined (always, wherever a model is placed; its own REFINE_ keys - rfree
+        # above stays the unrefined placement's): reported, never scored; refine_s is part of rugnux_wall_s
+        "rfree_refined": fnum(rep.get("REFINE_R_FREE")), "rwork_refined": fnum(rep.get("REFINE_R_WORK")),
+        "refine_s": fnum(rep.get("REFINE_SECONDS")),
         "sg_label": None, "accepted_alt": None, "accepted_alt_why": None,
         # the groups rugnux could not tell from the one it wrote (and, with an undetermined screw,
         # the screw alternatives), as it lists them

viewer/windows/JFJochProcessingJobsWindow.cpp

@@ -427,6 +427,7 @@ void JFJochProcessingJobsWindow::newJob(ProcessMode mode, CalibrationSelection c
     if (spec.mode == ProcessMode::FullAnalysis && !inputs.model_path.isEmpty()) {
         try {
             config.model = std::make_shared<const RugnuxModel>(ReadRugnuxModelFile(inputs.model_path.toStdString()));
+            config.refine = true;   // as `rugnux` refines a placed model by default
         } catch (const std::exception &e) {
             QMessageBox::warning(this, "Processing", QString("Cannot read the model: %1").arg(e.what()));
             return;