From 685c56217439778ed89e6b1e9f102e8d2cf6933f Mon Sep 17 00:00:00 2001
From: Filip Leonarski <filip.leonarski@psi.ch>
Date: Thu, 27 Aug 2026 16:04:52 +0200
Subject: [PATCH] rugnux: the unmerged MTZ is written on request again, not on
 every run

0fed51f76 made <prefix>_unmerged.mtz default-on so a run that turned out to need
it would not have to be repeated. It costs more than that is worth: measured on a
dense rotation dataset, it adds 2.6 s to a 7.0 s --mode scale run (+38%) and
writes 272 MB - larger than the merged .mtz, .cif and .hkl put together, and paid
by every run whether or not anything will read it.

--export-unmerged asks for it again. --no-export-unmerged stays accepted and now
does nothing, the mirror of what --export-unmerged did while the default was on.

The two fixes that came with the default-on change are kept: the file is written
in --mode mx and --mode scale and with --no-merge, and both write sites are
guarded on the output prefix, so a prefix-less run no longer writes a file called
"_unmerged.mtz".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016L1qig74oYQzfUJJZbbxFh
---
 docs/RUGNUX.md        | 21 +++++++++++----------
 rugnux/Rugnux.h       |  6 ++++--
 rugnux/rugnux_cli.cpp |  6 +++---
 3 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/docs/RUGNUX.md b/docs/RUGNUX.md
index d3d24325..c8ed84f1 100644
--- a/docs/RUGNUX.md
+++ b/docs/RUGNUX.md
@@ -245,8 +245,8 @@ rugnux -o quicklook -e 200 dataset_master.h5
 # merge as usual, but also keep the per-image file so the data can be re-merged later
 rugnux -o myrun --write-process-h5 dataset_master.h5
 
-# skip the unmerged MTZ, which is otherwise written alongside the merged files
-rugnux -o myrun --no-export-unmerged dataset_master.h5
+# also write the unmerged MTZ, to scale and merge the observations with another program
+rugnux -o myrun --export-unmerged dataset_master.h5
 
 # check the merged data against a known structure: R-work / R-free and maps
 rugnux -o myrun --model model.pdb dataset_master.h5
@@ -401,9 +401,9 @@ reusing them would hide the spot-finding settings from the lattice search.
   from systematic absences** (constrained to the indexed lattice symmetry).
 - `<prefix>_unmerged.mtz` — the integrated observations *before* merging, as an unmerged MTZ in
   POINTLESS's column layout, so the data can be scaled and merged by **aimless**, **pointless**,
-  **careless** or `iotbx.merging_statistics` instead of by rugnux. Written by default, alongside the
-  merged files and with `--no-merge` too; `--no-export-unmerged` turns it off. See
-  [The unmerged export](#the-unmerged-export) below. `--export-unmerged-partials` additionally writes
+  **careless** or `iotbx.merging_statistics` instead of by rugnux. Written when `--export-unmerged`
+  asks for it, alongside the merged files and with `--no-merge` too. See
+  [The unmerged export](#the-unmerged-export) below. `--export-unmerged-partials` writes
   `<prefix>_unmerged_partials.mtz`, one row per image, instead of summing.
 - `<prefix>_report.txt` — the **results report**: what the run determined, in a form both a person and
   a beamline script can read. Always written, next to the files above. See
@@ -459,9 +459,10 @@ expect. Two properties worth knowing before using it:
 
 `<prefix>_unmerged.mtz` holds every integrated observation, before scaling and merging, in the column
 layout POINTLESS writes and **aimless**, **pointless**, **careless** and `iotbx.merging_statistics`
-read. It is written by default, in `--mode mx` and `--mode scale` alike and with `--no-merge` as
-well, and it replaces nothing — rugnux still writes its own merged files in the same run. It needs an
-output prefix (`-o`), and `--no-export-unmerged` turns it off.
+read. `--export-unmerged` asks for it, in `--mode mx` and `--mode scale` alike and with `--no-merge`
+as well, and it replaces nothing — rugnux still writes its own merged files in the same run. It needs
+an output prefix (`-o`). It is off by default because it is the largest file a run produces, larger
+on a dense rotation dataset than the merged `.mtz`, `.cif` and `.hkl` put together.
 
 Use it to scale the data with a different program, to have pointless give an independent opinion on
 the space group, or to compare rugnux's merge against another one on identical input.
@@ -1011,8 +1012,8 @@ Scaling and merging:
 | `--reference-column <label>` | Reference MTZ column to use (default: auto — F-model, else IMEAN/I/…) |
 | `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](#validating-against-a-model-rugnux-model)). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](#the-indexing-ambiguity) where no `-z` did. For serial stills given `-C` / `-S`, the model's structure factors become the per-image reference |
 | `--write-process-h5` | Also write the (large) `_process.h5` when merging (default: only `.mtz`/`.cif`) |
-| `--no-export-unmerged` | Do not write `<prefix>_unmerged.mtz`. It is written by default: an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. Rotation partials are summed into one full per reflection. Intensities carry the Lorentz-polarization factor and nothing else — the partiality is not divided out and the per-image scale is not applied. Written in `--mode mx` and `--mode scale`, and with `--no-merge`. See [The unmerged export](#the-unmerged-export) |
-| `--export-unmerged-partials` | Also write `<prefix>_unmerged_partials.mtz`, the same observations with each partial as its own row (one batch per image) for the reading program to sum |
+| `--export-unmerged` | Write `<prefix>_unmerged.mtz`, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. Off by default. Rotation partials are summed into one full per reflection. Intensities carry the Lorentz-polarization factor and nothing else — the partiality is not divided out and the per-image scale is not applied. Written in `--mode mx` and `--mode scale`, and with `--no-merge`. See [The unmerged export](#the-unmerged-export) |
+| `--export-unmerged-partials` | Write `<prefix>_unmerged_partials.mtz`, the same observations with each partial as its own row (one batch per image) for the reading program to sum. Independent of `--export-unmerged` |
 
 Integration:
 
diff --git a/rugnux/Rugnux.h b/rugnux/Rugnux.h
index 04c87a48..4b49e0a1 100644
--- a/rugnux/Rugnux.h
+++ b/rugnux/Rugnux.h
@@ -130,8 +130,10 @@ struct ProcessConfig {
 
     // Write the integrated observations as <prefix>_unmerged.mtz, for aimless / pointless / careless
     // and anything else that scales unmerged data itself - each rocking event summed into one full.
-    // On by default in the rugnux CLI (--no-export-unmerged turns it off); needs an output prefix.
-    bool export_unmerged = true;
+    // Off by default (--export-unmerged asks for it): the file is the largest a run produces, and a
+    // run that is not handing its observations to another scaling program should not pay for it.
+    // Needs an output prefix.
+    bool export_unmerged = false;
 
     // The same as <prefix>_unmerged_partials.mtz, with each partial written as its own row for the
     // reader to sum (--export-unmerged-partials). Independent of the above; a run can write both.
diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp
index 6f1a1600..790e7d03 100644
--- a/rugnux/rugnux_cli.cpp
+++ b/rugnux/rugnux_cli.cpp
@@ -165,8 +165,8 @@ void print_usage() {
     std::cout << "   -z, --reference-mtz <file>      Reference MTZ file" << std::endl;
     std::cout << "   --reference-column <label>      Reference MTZ column to use (default: auto - F-model, else IMEAN/I, else FP/FOBS/F)" << std::endl;
     std::cout << "   --model <file>                  After merging, validate vs this atomic model: R-free, 2Fo-Fc/Fo-Fc maps, and - where the merge kept the Bijvoet split - an anomalous difference map with the strongest anomalous scatterers named by the atom they sit on. PDB or mmCIF, gzipped or not, recognised by content rather than by extension" << std::endl;
-    std::cout << "   --no-export-unmerged            Do not write <prefix>_unmerged.mtz. It is written by default: an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-partiality is not written, as in the merge. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
-    std::cout << "   --export-unmerged-partials      Also write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summed here" << std::endl;
+    std::cout << "   --export-unmerged               Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. Off by default - the file is large, and a run that does not need it should not pay for it. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-partiality is not written, as in the merge. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
+    std::cout << "   --export-unmerged-partials      Write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summed here. Independent of --export-unmerged; both can be given" << std::endl;
     std::cout << std::endl;
 
     std::cout << "  Integration" << std::endl;
@@ -675,7 +675,7 @@ static int RunRugnux(int argc, char **argv) {
     std::string ref_mtz;
     std::string ref_column;
     std::string model_pdb;           // --model: PDB to validate merged intensities against (R-free + maps)
-    bool export_unmerged = true;            // <prefix>_unmerged.mtz, off with --no-export-unmerged
+    bool export_unmerged = false;           // --export-unmerged: write <prefix>_unmerged.mtz
     bool export_unmerged_partials = false;  // --export-unmerged-partials: the same, one row per image
     std::string dump_observations;   // diagnostic: dump unmerged -P rot3d fulls to this path
     double min_partiality = 0.02;