mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 72 Wiki Activity

rugnux: the unmerged MTZ is written on request again, not on every run

Browse Source 
0fed51f76 made <prefix>_unmerged.mtz default-on so a run that turned out to need
it would not have to be repeated. It costs more than that is worth: measured on a
dense rotation dataset, it adds 2.6 s to a 7.0 s --mode scale run (+38%) and
writes 272 MB - larger than the merged .mtz, .cif and .hkl put together, and paid
by every run whether or not anything will read it.

--export-unmerged asks for it again. --no-export-unmerged stays accepted and now
does nothing, the mirror of what --export-unmerged did while the default was on.

The two fixes that came with the default-on change are kept: the file is written
in --mode mx and --mode scale and with --no-merge, and both write sites are
guarded on the output prefix, so a prefix-less run no longer writes a file called
"_unmerged.mtz".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016L1qig74oYQzfUJJZbbxFh
This commit is contained in:
leonarski_f
2026-08-27 22:09:51 +02:00
committed by leonarski_f
co-authored by Claude Opus 5
parent edc042aadf
commit 685c562174
3 changed files with 18 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/RUGNUX.md
+11 -10
View File
@@ -245,8 +245,8 @@ rugnux -o quicklook -e 200 dataset_master.h5
# merge as usual, but also keep the per-image file so the data can be re-merged later
rugnux -o myrun --write-process-h5 dataset_master.h5
# skip the unmerged MTZ, which is otherwise written alongside the merged files
rugnux -o myrun --no-export-unmerged dataset_master.h5
# also write the unmerged MTZ, to scale and merge the observations with another program
rugnux -o myrun --export-unmerged dataset_master.h5
# check the merged data against a known structure: R-work / R-free and maps
rugnux -o myrun --model model.pdb dataset_master.h5
@@ -401,9 +401,9 @@ reusing them would hide the spot-finding settings from the lattice search.
from systematic absences** (constrained to the indexed lattice symmetry).
- `<prefix>_unmerged.mtz` — the integrated observations *before* merging, as an unmerged MTZ in
POINTLESS's column layout, so the data can be scaled and merged by **aimless**, **pointless**,
**careless** or `iotbx.merging_statistics` instead of by rugnux. Written by default, alongside the
merged files and with `--no-merge` too; `--no-export-unmerged` turns it off. See
[The unmerged export](#the-unmerged-export) below. `--export-unmerged-partials` additionally writes
**careless** or `iotbx.merging_statistics` instead of by rugnux. Written when `--export-unmerged`
asks for it, alongside the merged files and with `--no-merge` too. See
[The unmerged export](#the-unmerged-export) below. `--export-unmerged-partials` writes
`<prefix>_unmerged_partials.mtz`, one row per image, instead of summing.
- `<prefix>_report.txt` — the **results report**: what the run determined, in a form both a person and
a beamline script can read. Always written, next to the files above. See
@@ -459,9 +459,10 @@ expect. Two properties worth knowing before using it:
`<prefix>_unmerged.mtz` holds every integrated observation, before scaling and merging, in the column
layout POINTLESS writes and **aimless**, **pointless**, **careless** and `iotbx.merging_statistics`
read. It is written by default, in `--mode mx` and `--mode scale` alike and with `--no-merge` as
well, and it replaces nothing — rugnux still writes its own merged files in the same run. It needs an
output prefix (`-o`), and `--no-export-unmerged` turns it off.
read. `--export-unmerged` asks for it, in `--mode mx` and `--mode scale` alike and with `--no-merge`
as well, and it replaces nothing — rugnux still writes its own merged files in the same run. It needs
an output prefix (`-o`). It is off by default because it is the largest file a run produces, larger
on a dense rotation dataset than the merged `.mtz`, `.cif` and `.hkl` put together.
Use it to scale the data with a different program, to have pointless give an independent opinion on
the space group, or to compare rugnux's merge against another one on identical input.
@@ -1011,8 +1012,8 @@ Scaling and merging:
| `--reference-column <label>` | Reference MTZ column to use (default: auto — F-model, else IMEAN/I/…) |
| `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](#validating-against-a-model-rugnux-model)). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](#the-indexing-ambiguity) where no `-z` did. For serial stills given `-C` / `-S`, the model's structure factors become the per-image reference |
| `--write-process-h5` | Also write the (large) `_process.h5` when merging (default: only `.mtz`/`.cif`) |
| `--no-export-unmerged` | Do not write `<prefix>_unmerged.mtz`. It is written by default: an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. Rotation partials are summed into one full per reflection. Intensities carry the Lorentz-polarization factor and nothing else — the partiality is not divided out and the per-image scale is not applied. Written in `--mode mx` and `--mode scale`, and with `--no-merge`. See [The unmerged export](#the-unmerged-export) |
| `--export-unmerged-partials` | Also write `<prefix>_unmerged_partials.mtz`, the same observations with each partial as its own row (one batch per image) for the reading program to sum |
| `--export-unmerged` | Write `<prefix>_unmerged.mtz`, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. Off by default. Rotation partials are summed into one full per reflection. Intensities carry the Lorentz-polarization factor and nothing else — the partiality is not divided out and the per-image scale is not applied. Written in `--mode mx` and `--mode scale`, and with `--no-merge`. See [The unmerged export](#the-unmerged-export) |
| `--export-unmerged-partials` | Write `<prefix>_unmerged_partials.mtz`, the same observations with each partial as its own row (one batch per image) for the reading program to sum. Independent of `--export-unmerged` |
Integration:
rugnux/Rugnux.h
+4 -2
View File
@@ -130,8 +130,10 @@ struct ProcessConfig {
// Write the integrated observations as <prefix>_unmerged.mtz, for aimless / pointless / careless
// and anything else that scales unmerged data itself - each rocking event summed into one full.
// On by default in the rugnux CLI (--no-export-unmerged turns it off); needs an output prefix.
bool export_unmerged = true;
// Off by default (--export-unmerged asks for it): the file is the largest a run produces, and a
// run that is not handing its observations to another scaling program should not pay for it.
// Needs an output prefix.
bool export_unmerged = false;
// The same as <prefix>_unmerged_partials.mtz, with each partial written as its own row for the
// reader to sum (--export-unmerged-partials). Independent of the above; a run can write both.
rugnux/rugnux_cli.cpp
+3 -3
View File
@@ -165,8 +165,8 @@ void print_usage() {
std::cout << " -z, --reference-mtz <file> Reference MTZ file" << std::endl;
std::cout << " --reference-column <label> Reference MTZ column to use (default: auto - F-model, else IMEAN/I, else FP/FOBS/F)" << std::endl;
std::cout << " --model <file> After merging, validate vs this atomic model: R-free, 2Fo-Fc/Fo-Fc maps, and - where the merge kept the Bijvoet split - an anomalous difference map with the strongest anomalous scatterers named by the atom they sit on. PDB or mmCIF, gzipped or not, recognised by content rather than by extension" << std::endl;
std::cout << " --no-export-unmerged Do not write <prefix>_unmerged.mtz. It is written by default: an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-partiality is not written, as in the merge. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
std::cout << " --export-unmerged-partials Also write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summed here" << std::endl;
std::cout << " --export-unmerged Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. Off by default - the file is large, and a run that does not need it should not pay for it. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-partiality is not written, as in the merge. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
std::cout << " --export-unmerged-partials Write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summed here. Independent of --export-unmerged; both can be given" << std::endl;
std::cout << std::endl;
std::cout << " Integration" << std::endl;
@@ -675,7 +675,7 @@ static int RunRugnux(int argc, char **argv) {
std::string ref_mtz;
std::string ref_column;
std::string model_pdb; // --model: PDB to validate merged intensities against (R-free + maps)
bool export_unmerged = true; // <prefix>_unmerged.mtz, off with --no-export-unmerged
bool export_unmerged = false; // --export-unmerged: write <prefix>_unmerged.mtz
bool export_unmerged_partials = false; // --export-unmerged-partials: the same, one row per image
std::string dump_observations; // diagnostic: dump unmerged -P rot3d fulls to this path
double min_partiality = 0.02;