FOR THE OWNER'S DECISION: without a GPU a placed model is not refined unless --refine asks for it
On the CPU the refinement is still two orders of magnitude slower than on a GPU (cytC with its reference model, --no-gpu -N 16, after the CPU commits before this one: 378 s against 11 s on an RTX 5080; 1610 s before them), and short of the order of magnitude the owner set as the bar for keeping it on. So a run with no GPU (none, or --no-gpu) now places and scores the model as before and leaves the refinement out, with a log line and a Note saying so, and the report's REFINE= NOT_PERFORMED with REFINE_REASON "no GPU (--refine runs it on the CPU)". --refine (new) runs it on the CPU; --no-refine is unchanged, a GPU run refines as before, and --mode refine (an explicit request) refines on the CPU as before. RugnuxConfig::refine_without_gpu carries it (the viewer, which sets refine, follows the same default). Docs: RUGNUX_CLI.md (--refine, --model), RUGNUX_MODEL_AND_REFERENCE.md. Checked: cytC -e 300 --no-gpu -> REFINE= NOT_PERFORMED with the reason; the same on the GPU -> PERFORMED. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UqXYdJDrM65gxsCn2PrbfK
This commit is contained in:
1 parent
a1ea549980
commit
5d61bc3bdb
6 files changed
+29
-3
No files matched your search
+2
-1
@@ -331,9 +331,10 @@ serial stills; `fft` (GPU) / `fftw` (CPU) index **de novo** and suit strong rota
|
|||||||
| `-z, --reference <file>` | Reference data of the same crystal form — an MTZ or a PDB structure-factor mmCIF (`-sf.cif`), gzipped or not, recognised by content; `--reference-mtz` is the same option: fixes the space group, its setting and the cell, resolves the [indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity), hands over the R-free set and reports CC<sub>ref</sub>. Not a scale anchor |
|
| `-z, --reference <file>` | Reference data of the same crystal form — an MTZ or a PDB structure-factor mmCIF (`-sf.cif`), gzipped or not, recognised by content; `--reference-mtz` is the same option: fixes the space group, its setting and the cell, resolves the [indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity), hands over the R-free set and reports CC<sub>ref</sub>. Not a scale anchor |
|
||||||
| `--free-flags <file>` | Take only the R-free test set of this MTZ or structure-factor mmCIF - e.g. the deposited reflections of the `--model` - for the validation, the refinement and the merged files; nothing else of the file is used, so unlike `-z` it fixes no space group, cell or frame. Taken on the model's axes where the merge matches the file's intensities (`R_FREE_FLAGS=`, see [Free set and model bias](RUGNUX_MODEL_AND_REFERENCE.md#free-set-and-model-bias)) |
|
| `--free-flags <file>` | Take only the R-free test set of this MTZ or structure-factor mmCIF - e.g. the deposited reflections of the `--model` - for the validation, the refinement and the merged files; nothing else of the file is used, so unlike `-z` it fixes no space group, cell or frame. Taken on the model's axes where the merge matches the file's intensities (`R_FREE_FLAGS=`, see [Free set and model bias](RUGNUX_MODEL_AND_REFERENCE.md#free-set-and-model-bias)) |
|
||||||
| `--reference-column <label>` | Reference column to use, by MTZ label — an mmCIF under the labels `gemmi cif2mtz` gives it (default: auto — F-model, else IMEAN/I/…, else FP/FOBS/F) |
|
| `--reference-column <label>` | Reference column to use, by MTZ label — an mmCIF under the labels `gemmi cif2mtz` gives it (default: auto — F-model, else IMEAN/I/…, else FP/FOBS/F) |
|
||||||
| `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](RUGNUX_ADVANCED.md#validating-against-a-model-rugnux---model)). An experimental model is placed as a rigid body, and by molecular replacement where the rigid body does not place it; a prediction (AlphaFold or ModelCIF) goes to molecular replacement straight away (`MODEL_PLACEMENT=`). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity) where no `-z` did; another space group or the twin hypothesis' subgroup it chooses is written beside the answer as `<prefix>_mr.mtz`/`.cif`. It does not change the processing: the merge is the same with and without it. A placed model is then [refined](RUGNUX_MODEL_AND_REFERENCE.md#refinement) against the merge it was placed on (`<prefix>_refined_model.cif`, `<prefix>_refined_maps.mtz`, `<prefix>_blobs.json`, `REFINE_` keys) - seconds on a GPU, typically 2-15 minutes on the CPU - which decides nothing about the processing |
|
| `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](RUGNUX_ADVANCED.md#validating-against-a-model-rugnux---model)). An experimental model is placed as a rigid body, and by molecular replacement where the rigid body does not place it; a prediction (AlphaFold or ModelCIF) goes to molecular replacement straight away (`MODEL_PLACEMENT=`). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity) where no `-z` did; another space group or the twin hypothesis' subgroup it chooses is written beside the answer as `<prefix>_mr.mtz`/`.cif`. It does not change the processing: the merge is the same with and without it. A placed model is then [refined](RUGNUX_MODEL_AND_REFERENCE.md#refinement) against the merge it was placed on (`<prefix>_refined_model.cif`, `<prefix>_refined_maps.mtz`, `<prefix>_blobs.json`, `REFINE_` keys) - seconds on a GPU, typically 2-15 minutes on the CPU, so without a GPU it is left out unless `--refine` asks for it - which decides nothing about the processing |
|
||||||
| `--mr <file>` | As `--model`, but placed by molecular replacement straight away, without trying the rigid body first: as many copies as the cell holds, in every space group the merge cannot tell apart and in the twin hypothesis' subgroup. A prediction is trimmed to the residues of pLDDT 70 and above, and where the AlphaFold DB's predicted-aligned-error JSON sits beside it under its own name it is cut into rigid domains; any other PDB or mmCIF is taken as an experimental structure, its polymer only (no waters, ligands, hydrogens or alternative conformers). A model of several chains is searched for as a whole and, beside it, one chain of each molecule in it by itself, and the better placement kept (`MR_SEARCH_MODEL=`). It may relabel the enantiomorph of the answer, and a different space group it chooses is written beside the answer as `<prefix>_mr.mtz`/`.cif` with the placed model `<prefix>_mr_model.pdb`. `--mode mx` only; not with `--model` |
|
| `--mr <file>` | As `--model`, but placed by molecular replacement straight away, without trying the rigid body first: as many copies as the cell holds, in every space group the merge cannot tell apart and in the twin hypothesis' subgroup. A prediction is trimmed to the residues of pLDDT 70 and above, and where the AlphaFold DB's predicted-aligned-error JSON sits beside it under its own name it is cut into rigid domains; any other PDB or mmCIF is taken as an experimental structure, its polymer only (no waters, ligands, hydrogens or alternative conformers). A model of several chains is searched for as a whole and, beside it, one chain of each molecule in it by itself, and the better placement kept (`MR_SEARCH_MODEL=`). It may relabel the enantiomorph of the answer, and a different space group it chooses is written beside the answer as `<prefix>_mr.mtz`/`.cif` with the placed model `<prefix>_mr_model.pdb`. `--mode mx` only; not with `--model` |
|
||||||
| `--no-refine` | Do not refine the placed model (for debugging): by default a model that `--model` or `--mr` places is refined |
|
| `--no-refine` | Do not refine the placed model (for debugging): by default a model that `--model` or `--mr` places is refined |
|
||||||
|
| `--refine` | Refine the placed model also where there is no GPU (none, or `--no-gpu`): on the CPU a refinement takes minutes to tens of minutes, so without a GPU it is left out by default (`REFINE= NOT_PERFORMED`, `REFINE_REASON`); with a GPU it is refined in any case |
|
||||||
| `--refine-solvent` | Also build the solvent of the refined model: waters added at difference-map peaks and removed where the maps no longer hold them, waters and ions relabelled, and each site audited (the `REFINE_SOLVENT_` keys and `<prefix>_solvent.json`). Off by default: the model is refined with its solvent as given, and the report says `REFINE_SOLVENT=NOT_PERFORMED`. Also with `--mode refine` |
|
| `--refine-solvent` | Also build the solvent of the refined model: waters added at difference-map peaks and removed where the maps no longer hold them, waters and ions relabelled, and each site audited (the `REFINE_SOLVENT_` keys and `<prefix>_solvent.json`). Off by default: the model is refined with its solvent as given, and the report says `REFINE_SOLVENT=NOT_PERFORMED`. Also with `--mode refine` |
|
||||||
| `--no-refine-solvent` | The default: no solvent build or audit (turns off an earlier `--refine-solvent`) |
|
| `--no-refine-solvent` | The default: no solvent build or audit (turns off an earlier `--refine-solvent`) |
|
||||||
| `--restraints <file>` | A restraint dictionary for the refinement, over the built-in one: a monomer-library, AceDRG, grade or eLBOW restraint mmCIF (its monomers, links and modifications), or a PDB Chemical Component Dictionary entry, restrained to its ideal coordinates' bonds, angles and planes (no torsions or chiral volumes). An entry replaces the built-in one of the same code; repeatable, the later file winning; gzipped or not. Also given among the inputs, recognised by content. Also with `--mode refine` (see [Refinement](RUGNUX_MODEL_AND_REFERENCE.md#refinement)) |
|
| `--restraints <file>` | A restraint dictionary for the refinement, over the built-in one: a monomer-library, AceDRG, grade or eLBOW restraint mmCIF (its monomers, links and modifications), or a PDB Chemical Component Dictionary entry, restrained to its ideal coordinates' bonds, angles and planes (no torsions or chiral volumes). An entry replaces the built-in one of the same code; repeatable, the later file winning; gzipped or not. Also given among the inputs, recognised by content. Also with `--mode refine` (see [Refinement](RUGNUX_MODEL_AND_REFERENCE.md#refinement)) |
|
||||||
|
|||||||
@@ -247,7 +247,8 @@ The safeguards behind 2 and 3:
|
|||||||
Giving a model is the opt-in: wherever the model is placed - it fits where it is, or molecular replacement
|
Giving a model is the opt-in: wherever the model is placed - it fits where it is, or molecular replacement
|
||||||
placed it - rugnux refines it against the merge it was placed on; a model that is not placed is not refined.
|
placed it - rugnux refines it against the merge it was placed on; a model that is not placed is not refined.
|
||||||
A run that should not spend the time is given no model; `--no-refine` skips the refinement of a placed model
|
A run that should not spend the time is given no model; `--no-refine` skips the refinement of a placed model
|
||||||
(for debugging). It takes seconds on a GPU and typically 2-15 minutes on the CPU (no GPU, or `--no-gpu`).
|
(for debugging). It takes seconds on a GPU and typically 2-15 minutes on the CPU, so a run without a GPU (none, or
|
||||||
|
`--no-gpu`) leaves it out - the log and `REFINE_REASON` say so - unless `--refine` asks for it.
|
||||||
Refinement comes after every processing decision and changes none of them: `R_WORK`, `R_FREE` and every
|
Refinement comes after every processing decision and changes none of them: `R_WORK`, `R_FREE` and every
|
||||||
other key outside `REFINE_` stay those of the model as placed, and `<prefix>.mtz`, `<prefix>_model.cif` and
|
other key outside `REFINE_` stay those of the model as placed, and `<prefix>.mtz`, `<prefix>_model.cif` and
|
||||||
`<prefix>_maps.mtz` are the same as they would be without it. The broker's live mode refines an uploaded model
|
`<prefix>_maps.mtz` are the same as they would be without it. The broker's live mode refines an uploaded model
|
||||||
|
|||||||
@@ -301,6 +301,9 @@ struct ProcessConfig {
|
|||||||
// On in the rugnux CLI (--no-refine turns it off), the viewer and the broker's live mode: giving a model is the
|
// On in the rugnux CLI (--no-refine turns it off), the viewer and the broker's live mode: giving a model is the
|
||||||
// opt-in.
|
// opt-in.
|
||||||
bool refine = false;
|
bool refine = false;
|
||||||
|
// With `refine`: refined also where there is no GPU. On the CPU a refinement takes minutes to tens of minutes,
|
||||||
|
// against seconds on a GPU, so without one it is left out unless asked for (the CLI's --refine).
|
||||||
|
bool refine_without_gpu = false;
|
||||||
// With `refine`: waters built and removed and ions relabelled between refinements, and the solvent audited
|
// With `refine`: waters built and removed and ions relabelled between refinements, and the solvent audited
|
||||||
// (RefinementSettings::solvent); off by default for now (--refine-solvent turns it on): the solvent stays as given.
|
// (RefinementSettings::solvent); off by default for now (--refine-solvent turns it on): the solvent stays as given.
|
||||||
bool refine_solvent = false;
|
bool refine_solvent = false;
|
||||||
|
|||||||
@@ -90,6 +90,8 @@ std::string RugnuxCommandLine(const ProcessConfig &config,
|
|||||||
add(config.force_mr ? "--mr" : "--model", config.model->name);
|
add(config.force_mr ? "--mr" : "--model", config.model->name);
|
||||||
if (config.model && !config.refine)
|
if (config.model && !config.refine)
|
||||||
args.emplace_back("--no-refine");
|
args.emplace_back("--no-refine");
|
||||||
|
if (config.model && config.refine && config.refine_without_gpu)
|
||||||
|
args.emplace_back("--refine");
|
||||||
if (config.model && config.refine && config.refine_solvent)
|
if (config.model && config.refine && config.refine_solvent)
|
||||||
args.emplace_back("--refine-solvent");
|
args.emplace_back("--refine-solvent");
|
||||||
if (config.model && config.refine)
|
if (config.model && config.refine)
|
||||||
|
|||||||
@@ -5664,7 +5664,17 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
|
|||||||
const bool refine_cut = config_.fast_live && cancelled_;
|
const bool refine_cut = config_.fast_live && cancelled_;
|
||||||
if (refine_cut && config_.refine && result.model_placement != "NOT_PLACED")
|
if (refine_cut && config_.refine && result.model_placement != "NOT_PLACED")
|
||||||
Note("cut: the refinement is left out");
|
Note("cut: the refinement is left out");
|
||||||
if (config_.refine && !refine_cut && result.model_placement != "NOT_PLACED" && placed_model_out
|
// Without a GPU the refinement is left out unless asked for (RugnuxConfig::refine_without_gpu)
|
||||||
|
const bool refine_no_gpu = config_.refine && !config_.refine_without_gpu && get_gpu_count() == 0;
|
||||||
|
if (refine_no_gpu && !refine_cut && result.model_placement != "NOT_PLACED") {
|
||||||
|
logger.Info("Refinement: not run - there is no GPU, and on the CPU it takes minutes to tens of minutes "
|
||||||
|
"(--refine runs it on the CPU)");
|
||||||
|
Note("no GPU: the refinement is left out (--refine runs it on the CPU)");
|
||||||
|
RefinementResult not_run;
|
||||||
|
not_run.failure_reason = "no GPU (--refine runs it on the CPU)";
|
||||||
|
result.refinement = std::move(not_run);
|
||||||
|
}
|
||||||
|
if (config_.refine && !refine_no_gpu && !refine_cut && result.model_placement != "NOT_PLACED" && placed_model_out
|
||||||
&& placed_merged != nullptr && result.model_validation && result.model_validation->ok) {
|
&& placed_merged != nullptr && result.model_validation && result.model_validation->ok) {
|
||||||
phase("Refining the model");
|
phase("Refining the model");
|
||||||
Step("Refinement");
|
Step("Refinement");
|
||||||
|
|||||||
@@ -190,6 +190,7 @@ Basic options
|
|||||||
--scaling-high-resolution <A> High-resolution limit (default: from the CC1/2 fall-off)
|
--scaling-high-resolution <A> High-resolution limit (default: from the CC1/2 fall-off)
|
||||||
--model <file> Atomic model (PDB/mmCIF) to place, refine and validate with
|
--model <file> Atomic model (PDB/mmCIF) to place, refine and validate with
|
||||||
--no-refine Place and score the model, but do not refine it
|
--no-refine Place and score the model, but do not refine it
|
||||||
|
--refine Refine the placed model also without a GPU (on the CPU: minutes, not seconds)
|
||||||
--refine-solvent Also build waters and relabel ions in the refined model (default off)
|
--refine-solvent Also build waters and relabel ions in the refined model (default off)
|
||||||
--restraints <file> Restraint dictionary (.cif) for a ligand the built-in one lacks; repeatable
|
--restraints <file> Restraint dictionary (.cif) for a ligand the built-in one lacks; repeatable
|
||||||
-z, --reference <file> Reference reflections of the same crystal form (MTZ/mmCIF)
|
-z, --reference <file> Reference reflections of the same crystal form (MTZ/mmCIF)
|
||||||
@@ -258,6 +259,7 @@ enum {
|
|||||||
OPT_CRYSTFEL_GEOMETRY,
|
OPT_CRYSTFEL_GEOMETRY,
|
||||||
OPT_MR,
|
OPT_MR,
|
||||||
OPT_NO_REFINE,
|
OPT_NO_REFINE,
|
||||||
|
OPT_REFINE,
|
||||||
OPT_REFINE_SOLVENT,
|
OPT_REFINE_SOLVENT,
|
||||||
OPT_NO_REFINE_SOLVENT,
|
OPT_NO_REFINE_SOLVENT,
|
||||||
OPT_RESTRAINTS,
|
OPT_RESTRAINTS,
|
||||||
@@ -347,6 +349,7 @@ static option long_options[] = {
|
|||||||
{"crystfel-geometry", required_argument, nullptr, OPT_CRYSTFEL_GEOMETRY},
|
{"crystfel-geometry", required_argument, nullptr, OPT_CRYSTFEL_GEOMETRY},
|
||||||
{"mr", required_argument, nullptr, OPT_MR},
|
{"mr", required_argument, nullptr, OPT_MR},
|
||||||
{"no-refine", no_argument, nullptr, OPT_NO_REFINE},
|
{"no-refine", no_argument, nullptr, OPT_NO_REFINE},
|
||||||
|
{"refine", no_argument, nullptr, OPT_REFINE},
|
||||||
{"refine-solvent", no_argument, nullptr, OPT_REFINE_SOLVENT},
|
{"refine-solvent", no_argument, nullptr, OPT_REFINE_SOLVENT},
|
||||||
{"no-refine-solvent", no_argument, nullptr, OPT_NO_REFINE_SOLVENT},
|
{"no-refine-solvent", no_argument, nullptr, OPT_NO_REFINE_SOLVENT},
|
||||||
{"restraints", required_argument, nullptr, OPT_RESTRAINTS},
|
{"restraints", required_argument, nullptr, OPT_RESTRAINTS},
|
||||||
@@ -939,6 +942,7 @@ struct RugnuxCli {
|
|||||||
std::string model_pdb; // --model: PDB to validate merged intensities against (R-free + maps)
|
std::string model_pdb; // --model: PDB to validate merged intensities against (R-free + maps)
|
||||||
std::string mr_model; // --mr: --model, placed by molecular replacement straight away
|
std::string mr_model; // --mr: --model, placed by molecular replacement straight away
|
||||||
bool refine_model = true; // --no-refine: a placed model is refined unless it is given
|
bool refine_model = true; // --no-refine: a placed model is refined unless it is given
|
||||||
|
bool refine_without_gpu = false; // --refine: refined also without a GPU (RugnuxConfig::refine_without_gpu)
|
||||||
bool refine_solvent = false; // --refine-solvent: the refinement builds waters and relabels ions
|
bool refine_solvent = false; // --refine-solvent: the refinement builds waters and relabels ions
|
||||||
std::vector<std::string> restraint_files; // --restraints and the restraint files among the inputs
|
std::vector<std::string> restraint_files; // --restraints and the restraint files among the inputs
|
||||||
RestraintDictionary restraints; // read from them, over the embedded dictionary
|
RestraintDictionary restraints; // read from them, over the embedded dictionary
|
||||||
@@ -1272,6 +1276,10 @@ int RugnuxCli::Parse(int argc, char **argv) {
|
|||||||
case OPT_NO_REFINE:
|
case OPT_NO_REFINE:
|
||||||
refine_model = false;
|
refine_model = false;
|
||||||
break;
|
break;
|
||||||
|
case OPT_REFINE:
|
||||||
|
refine_model = true;
|
||||||
|
refine_without_gpu = true;
|
||||||
|
break;
|
||||||
case OPT_REFINE_SOLVENT:
|
case OPT_REFINE_SOLVENT:
|
||||||
refine_solvent = true;
|
refine_solvent = true;
|
||||||
break;
|
break;
|
||||||
@@ -3859,6 +3867,7 @@ int RugnuxInputRun::PrepareRugnuxInput() {
|
|||||||
config.model = model;
|
config.model = model;
|
||||||
config.force_mr = !mr_model.empty();
|
config.force_mr = !mr_model.empty();
|
||||||
config.refine = refine_model; // a placed model is refined unless --no-refine
|
config.refine = refine_model; // a placed model is refined unless --no-refine
|
||||||
|
config.refine_without_gpu = refine_without_gpu;
|
||||||
config.refine_solvent = refine_solvent;
|
config.refine_solvent = refine_solvent;
|
||||||
config.restraints = restraints;
|
config.restraints = restraints;
|
||||||
// When merging, the merged reflections (.mtz/.cif) are the wanted output; skip the large
|
// When merging, the merged reflections (.mtz/.cif) are the wanted output; skip the large
|
||||||
|
|||||||
Reference in new issue
Block a user