v1.0.0-rc.175 (#85)

* Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`).
* Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection.
* Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS).
* Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead.
* Rugnux: On macOS, Bragg integration runs on the Apple GPU.
* Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand.
* Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`.
* Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit.
* Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed.
* Rugnux: The beam-stop shadow is found correctly on tilted detectors.
* Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement.
* jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps.
* jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection.
* jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images.
* Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges.

---------

Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home>
Reviewed-on: #85
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>

image_analysis/WriteReflections.cpp

@@ -10,6 +10,7 @@
 
 #include <algorithm>
 #include <cmath>
+#include <cctype>
 #include <cstring>
 #include <numeric>
 #include <cstdio>
@@ -68,6 +69,97 @@ std::string CifStr(const std::string& s) {
     return s;
 }
 
+std::string Upper(std::string s) {
+    for (char &c : s)
+        c = static_cast<char>(std::toupper(static_cast<unsigned char>(c)));
+    return s;
+}
+
+// The general class of the detector, _diffrn_detector.detector in the PDBx enumeration, where its make
+// and model name it; empty where they do not.
+std::string DetectorClass(const std::string &description) {
+    const std::string d = Upper(description);
+    if (d.find("CCD") != std::string::npos)
+        return "CCD";
+    if (d.find("IMAGE PLATE") != std::string::npos)
+        return "IMAGE PLATE";
+    for (const char *pixel : {"EIGER", "PILATUS", "JUNGFRAU"})
+        if (d.find(pixel) != std::string::npos)
+            return "PIXEL";
+    return "";
+}
+
+// The general class of the source, _diffrn_source.source in the PDBx enumeration, from the NXmx
+// /entry/source/type the file states; empty where it states none or one PDBx has no class for.
+std::string SourceClass(const std::string &nxmx_type) {
+    const std::string t = Upper(nxmx_type);
+    if (t.find("SYNCHROTRON") != std::string::npos) return "SYNCHROTRON";
+    if (t.find("FREE-ELECTRON") != std::string::npos || t.find("FREE ELECTRON") != std::string::npos)
+        return "FREE ELECTRON LASER";
+    if (t.find("ROTATING ANODE") != std::string::npos) return "ROTATING ANODE";
+    if (t.find("FIXED TUBE") != std::string::npos) return "SEALED TUBE";
+    if (t.find("METAL JET") != std::string::npos) return "LIQUID ANODE";
+    return "";
+}
+
+// The date part of an ISO-8601 time stamp (YYYY-MM-DD...), which is all the PDBx collection date
+// takes; empty for anything else.
+std::string IsoDate(const std::string &timestamp) {
+    if (timestamp.size() < 10 || timestamp[4] != '-' || timestamp[7] != '-')
+        return "";
+    return timestamp.substr(0, 10);
+}
+
+// The commit this build was made from, with its date where the build knows it ("unknown" for a source
+// tree without git history).
+std::string GitDescription() {
+    return jfjoch_git_date().empty() ? jfjoch_git_sha1() : jfjoch_git_sha1() + " " + jfjoch_git_date();
+}
+
+// The MTZ HISTORY records saying where a file came from, after the one naming the program: the
+// commit, the command line and the input file. MTZ holds at most 30 records of 80 characters
+// (mtzformat.html), so a long value runs on over as many records as it needs, up to that limit, each
+// continuation starting with '+'. Readers strip the blanks at both ends of a record, so a record is
+// never cut just after a space: the space starts the next one, behind its '+'.
+void AddProvenanceHistory(gemmi::Mtz &mtz, const OutputProvenance &provenance) {
+    const auto add = [&mtz](const std::string &label, const std::string &value) {
+        if (value.empty())
+            return;
+        std::string rest = label + value;
+        while (!rest.empty() && mtz.history.size() < 30) {
+            size_t n = std::min<size_t>(80, rest.size());
+            while (n > 1 && n < rest.size() && rest[n - 1] == ' ')
+                --n;
+            mtz.history.push_back(rest.substr(0, n));
+            rest = n < rest.size() ? "+" + rest.substr(n) : std::string();
+        }
+    };
+    add("Rugnux git ", GitDescription());
+    add("Command: ", provenance.command_line);
+    add("Input: ", provenance.input_file);
+    add("Input file: ", provenance.input_identity);
+    add("Collected: ", provenance.collection_date);
+}
+
+// _entry, _cell and _symmetry.
+void WriteBlockHeader(std::ostream &out, const std::string &entry_id, const UnitCell &unitCell,
+                      const gemmi::SpaceGroup &sg) {
+    out << "_entry.id   " << entry_id << "\n";
+    out << "#\n";
+    out << "_cell.entry_id     " << entry_id << "\n";
+    out << "_cell.length_a     " << Fmt(unitCell.a, 3) << "\n";
+    out << "_cell.length_b     " << Fmt(unitCell.b, 3) << "\n";
+    out << "_cell.length_c     " << Fmt(unitCell.c, 3) << "\n";
+    out << "_cell.angle_alpha  " << Fmt(unitCell.alpha, 2) << "\n";
+    out << "_cell.angle_beta   " << Fmt(unitCell.beta, 2) << "\n";
+    out << "_cell.angle_gamma  " << Fmt(unitCell.gamma, 2) << "\n";
+    out << "#\n";
+    out << "_symmetry.entry_id               " << entry_id << "\n";
+    out << "_symmetry.space_group_name_H-M  " << CifStr(sg.hm) << "\n";
+    out << "_symmetry.Int_Tables_number      " << sg.number << "\n";
+    out << "#\n";
+}
+
 // One output row per reflection in the standard CCP4 anomalous layout: the merged mean (IMEAN / F)
 // plus the two Bijvoet mates (I(+)/I(-), F(+)/F(-)). The merge keeps the two mates as separate rows
 // (I+ under the ASU representative hkl, I- under -hkl); this collapses them into one row so the MTZ
@@ -162,6 +254,7 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
                            const MergeStatistics &statistics,
                            const ErrorModelReport &error_model,
                            const TwinningAnalysisResult &twinning,
+                           const OutputProvenance &provenance,
                            const std::string &filename,
                            size_t nthreads) {
 
@@ -171,9 +264,16 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
 
     out << std::fixed;
 
+    // The keys PDBx asks for are written with the one value this file has for each: one entry, one
+    // crystal, one diffraction experiment (diffrn_id 1, which _pdbx_reflns_twin below also points at)
+    // and one wavelength.
+    const std::string entry_id = "sample";
+    const gemmi::SpaceGroup &sg = experiment.GetSpaceGroupOrP1();
+
     // ---------- data block ----------
     out << "data_sample" << "\n";
     out << "#\n";
+    WriteBlockHeader(out, entry_id, unitCell, sg);
 
     // ---------- _audit ----------
     out << "_audit.revision_id     1\n";
@@ -182,35 +282,69 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
     out << "#\n";
 
     // ---------- _software ----------
-    out << "_software.name         'Rugnux'\n";
-    out << "_software.version      " << CifStr(jfjoch_version()) << "\n";
-    out << "_software.classification 'data reduction'\n";
+    // Rugnux integrated the images (data reduction) and scaled and merged them (data scaling), the
+    // two rows the wwPDB asks a data-processing program to be named under; the description is the
+    // commit the build was made from.
+    const std::string software_description = CifStr("git " + GitDescription());
+    out << "loop_\n";
+    out << "_software.pdbx_ordinal\n";
+    out << "_software.name\n";
+    out << "_software.version\n";
+    out << "_software.classification\n";
+    out << "_software.description\n";
+    out << "1 Rugnux " << CifStr(jfjoch_version()) << " 'data reduction' " << software_description << "\n";
+    out << "2 Rugnux " << CifStr(jfjoch_version()) << " 'data scaling' " << software_description << "\n";
     out << "#\n";
 
-    // ---------- _cell ----------
-    out << "_cell.length_a     " << Fmt(unitCell.a, 3) << "\n";
-    out << "_cell.length_b     " << Fmt(unitCell.b, 3) << "\n";
-    out << "_cell.length_c     " << Fmt(unitCell.c, 3) << "\n";
-    out << "_cell.angle_alpha  " << Fmt(unitCell.alpha, 2) << "\n";
-    out << "_cell.angle_beta   " << Fmt(unitCell.beta, 2) << "\n";
-    out << "_cell.angle_gamma  " << Fmt(unitCell.gamma, 2) << "\n";
-
-    const gemmi::SpaceGroup &sg = experiment.GetSpaceGroupOrP1();
-
-    // ---------- _symmetry ----------
-    out << "_symmetry.space_group_name_H-M  " << CifStr(sg.hm) << "\n";
-    out << "_symmetry.Int_Tables_number      " << sg.number << "\n";
+    // ---------- provenance ----------
+    // How this file was made and from what: the command line and the input file, which no PDBx item
+    // holds, under the jfjoch local prefix as the other private items here. The two that can carry
+    // any character are text fields.
+    if (!provenance.command_line.empty())
+        out << "_jfjoch_provenance.command_line\n;" << provenance.command_line << "\n;\n";
+    if (!provenance.input_file.empty())
+        out << "_jfjoch_provenance.input_file\n;" << provenance.input_file << "\n;\n";
+    if (!provenance.input_identity.empty())
+        out << "_jfjoch_provenance.input_identity  " << CifStr(provenance.input_identity) << "\n";
     out << "#\n";
 
-    // ---------- _diffrn_source / _diffrn_detector ----------
+    // ---------- _exptl_crystal / _diffrn ----------
+    out << "_exptl_crystal.id     1\n";
+    out << "#\n";
+    out << "_diffrn.id            1\n";
+    out << "_diffrn.crystal_id    1\n";
+    if (const auto t = experiment.GetSampleTemperature_K())
+        out << "_diffrn.ambient_temp  " << Fmt(*t, 1) << "\n";
+    out << "#\n";
+
+    // ---------- _diffrn_radiation / _diffrn_source / _diffrn_detector ----------
+    out << "_diffrn_radiation.diffrn_id             1\n";
+    out << "_diffrn_radiation.wavelength_id         1\n";
+    out << "_diffrn_radiation.pdbx_diffrn_protocol  'SINGLE WAVELENGTH'\n";
+    out << "#\n";
+    out << "_diffrn_radiation_wavelength.id          1\n";
+    out << "_diffrn_radiation_wavelength.wavelength  " << Fmt(experiment.GetWavelength_A(), 5) << "\n";
+    out << "#\n";
+
+    // The source and the detector as the input file names them: the facility and the beamline (NXmx
+    // /entry/source/name and /entry/instrument/name) and the detector's make and model. The general
+    // classes PDBx enumerates are written only where those names settle them.
+    out << "_diffrn_source.diffrn_id  1\n";
+    if (const std::string source = SourceClass(experiment.GetSourceType()); !source.empty())
+        out << "_diffrn_source.source  " << CifStr(source) << "\n";
     if (!experiment.GetSourceName().empty())
         out << "_diffrn_source.pdbx_synchrotron_site  " << CifStr(experiment.GetSourceName()) << "\n";
-
     if (!experiment.GetInstrumentName().empty())
         out << "_diffrn_source.pdbx_synchrotron_beamline  " << CifStr(experiment.GetInstrumentName()) << "\n";
+    out << "_diffrn_source.pdbx_wavelength_list  " << Fmt(experiment.GetWavelength_A(), 5) << "\n";
+    out << "#\n";
 
-    out << "_diffrn_radiation_wavelength.wavelength  " << Fmt(experiment.GetWavelength_A(), 5) << "\n";
-    out << "_diffrn_detector.detector  " << CifStr(experiment.GetDetectorDescription()) << "\n";
+    out << "_diffrn_detector.diffrn_id  1\n";
+    if (const std::string cls = DetectorClass(experiment.GetDetectorDescription()); !cls.empty())
+        out << "_diffrn_detector.detector  " << CifStr(cls) << "\n";
+    out << "_diffrn_detector.type  " << CifStr(experiment.GetDetectorDescription()) << "\n";
+    if (const std::string date = IsoDate(provenance.collection_date); !date.empty())
+        out << "_diffrn_detector.pdbx_collection_date  " << date << "\n";
     // Detector geometry actually used for integration (refined, when geometry refinement ran - the rotation
     // two-pass or the stills global refinement update it on experiment_ before the written pass). jfjoch_
     // local-data-name items: feedback of the distance / beam centre the data was reduced with.
@@ -236,6 +370,9 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
         // Anomalous signal-to-noise (SigAno) is written only when an anomalous split was made, so a
         // non-anomalous merge keeps its previous stats block / shell-loop columns unchanged.
         const bool has_anom = std::isfinite(ov.abs_diff_over_sigma_anomalous);
+        out << "_reflns.entry_id                  " << entry_id << "\n";
+        out << "_reflns.pdbx_ordinal              1\n";
+        out << "_reflns.pdbx_diffrn_id            1\n";
         out << "_reflns.d_resolution_high         " << Fmt(ov.d_min, 2) << "\n";
         out << "_reflns.d_resolution_low          " << Fmt(ov.d_max, 2) << "\n";
         out << "_reflns.number_obs                " << ov.unique_reflections << "\n";
@@ -243,8 +380,11 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
         out << "_reflns.pdbx_redundancy           " << Fmt(mult(ov), 2) << "\n";
         out << "_reflns.percent_possible_obs      " << Fmt(compl_pct(ov), 1) << "\n";
         out << "_reflns.pdbx_netI_over_sigmaI     " << Fmt(ov.mean_i_over_sigma, 2) << "\n";
+        out << "_reflns.pdbx_Rmerge_I_obs         " << Fmt(ov.r_merge, 4) << "\n";
         out << "_reflns.pdbx_Rrim_I_all           " << Fmt(ov.r_meas, 4) << "\n";
+        out << "_reflns.pdbx_Rpim_I_all           " << Fmt(ov.r_pim, 4) << "\n";
         out << "_reflns.pdbx_CC_half              " << Fmt(ov.cc_half, 4) << "\n";
+        out << "_reflns.pdbx_CC_star              " << Fmt(CCStar(ov.cc_half), 4) << "\n";
         if (has_anom)
             out << "_reflns.pdbx_absDiff_over_sigma_anomalous  " << Fmt(ov.abs_diff_over_sigma_anomalous, 3)
                 << "   # SigAno = <|dano|>/<sigma(dano)>\n";
@@ -334,7 +474,7 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
                 out << "_reflns.jfjoch_aniso_significance     " << Fmt(an.significance, 2)
                     << "   # deltaB(linear) / floor\n";
             out << "_reflns.jfjoch_aniso_verdict          " << AnisotropyVerdictCode(an.verdict)
-                << "   # DETECTED / NOT_DETECTED / CANNOT_DETERMINE, at deltaB(linear)/floor > 3.5\n";
+                << "   # DETECTED / NOT_DETECTED / CANNOT_DETERMINE: deltaB(linear)/floor > 3.5, or the directional limits\n";
             for (int i = 0; i < 3; ++i)
                 if (std::isfinite(an.d_min_axis[i]))
                     out << "_reflns.jfjoch_aniso_d_min_" << (i + 1) << "          "
@@ -360,6 +500,8 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
         }
 
         out << "loop_\n";
+        out << "_reflns_shell.pdbx_ordinal\n";
+        out << "_reflns_shell.pdbx_diffrn_id\n";
         out << "_reflns_shell.d_res_high\n";
         out << "_reflns_shell.d_res_low\n";
         out << "_reflns_shell.number_measured_obs\n";
@@ -367,17 +509,23 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
         out << "_reflns_shell.pdbx_redundancy\n";
         out << "_reflns_shell.percent_possible_obs\n";
         out << "_reflns_shell.meanI_over_sigI_obs\n";
+        out << "_reflns_shell.Rmerge_I_obs\n";
         out << "_reflns_shell.pdbx_Rrim_I_all\n";
+        out << "_reflns_shell.pdbx_Rpim_I_all\n";
         out << "_reflns_shell.pdbx_CC_half\n";
+        out << "_reflns_shell.pdbx_CC_star\n";
         if (has_anom)
             out << "_reflns_shell.pdbx_absDiff_over_sigma_anomalous\n";
+        int ordinal = 0;
         for (const auto &s : statistics.shells) {
             if (s.unique_reflections == 0)
                 continue;
-            out << Fmt(s.d_min, 2) << " " << Fmt(s.d_max, 2) << " "
+            out << ++ordinal << " 1 "
+                << Fmt(s.d_min, 2) << " " << Fmt(s.d_max, 2) << " "
                 << s.total_observations << " " << s.unique_reflections << " "
                 << Fmt(mult(s), 2) << " " << Fmt(compl_pct(s), 1) << " "
-                << Fmt(s.mean_i_over_sigma, 2) << " " << Fmt(s.r_meas, 4) << " " << Fmt(s.cc_half, 4);
+                << Fmt(s.mean_i_over_sigma, 2) << " " << Fmt(s.r_merge, 4) << " " << Fmt(s.r_meas, 4) << " "
+                << Fmt(s.r_pim, 4) << " " << Fmt(s.cc_half, 4) << " " << Fmt(CCStar(s.cc_half), 4);
             if (has_anom)
                 out << " " << Fmt(s.abs_diff_over_sigma_anomalous, 3);
             out << "\n";
@@ -454,12 +602,14 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
 
     out << "#\n";
     out << "# End of reflections\n";
+
     out.close();
 }
 
 void WriteMtzReflections(const std::vector<MergedReflection> &reflections,
                          const UnitCell &unitCell,
                          const DiffractionExperiment &experiment,
+                         const OutputProvenance &provenance,
                          const std::string &filename) {
     gemmi::Mtz mtz;
 
@@ -470,6 +620,7 @@ void WriteMtzReflections(const std::vector<MergedReflection> &reflections,
     // Producing-software provenance in the MTZ header (title + HISTORY, the CCP4 convention).
     mtz.title = "Rugnux merged reflections";
     mtz.history.push_back("From Rugnux " + jfjoch_version() + ", data reduction");
+    AddProvenanceHistory(mtz, provenance);
 
     // Add dataset. HKL_base first, so the data dataset is id 1: a dataset written as id 0 is the
     // reserved base dataset as far as CCP4's mtzlib is concerned, and the wavelength on it is
@@ -506,7 +657,7 @@ void WriteMtzReflections(const std::vector<MergedReflection> &reflections,
     mtz.add_column("FreeR_flag", 'I', dataset_id, -1, false);
 
     mtz.nreflections = static_cast<int>(out_rows.size());
-    mtz.data.reserve(out_rows.size() * (has_anom ? 16 : 8));
+    mtz.data.reserve(out_rows.size() * 16);
     for (const auto& row : out_rows) {
         mtz.data.push_back(static_cast<float>(row.h));
         mtz.data.push_back(static_cast<float>(row.k));
@@ -790,12 +941,14 @@ gemmi::Mtz UnmergedMtz(const std::vector<IntegrationOutcome> &outcomes,
                        const UnitCell &unitCell,
                        const DiffractionExperiment &experiment,
                        bool sum_partials,
+                       const OutputProvenance &provenance,
                        size_t nthreads) {
     gemmi::Mtz mtz;
     mtz.spacegroup = &experiment.GetSpaceGroupOrP1();
     mtz.set_cell_for_all(unitCell);
     mtz.title = "Rugnux unmerged observations";
     mtz.history.push_back("From Rugnux " + jfjoch_version() + ", data reduction");
+    AddProvenanceHistory(mtz, provenance);
     mtz.add_base();                 // the HKL_base dataset and the H K L columns
     gemmi::Mtz::Dataset &ds = mtz.add_dataset("unmerged");
     ds.crystal_name = experiment.GetSampleName();
@@ -880,6 +1033,15 @@ gemmi::Mtz UnmergedMtz(const std::vector<IntegrationOutcome> &outcomes,
     // removing them turns its test into an assumption. XDS and DIALS draw the line in the same place.
     const char centering = mtz.spacegroup ? mtz.spacegroup->hm[0] : 'P';
 
+    // On stills only the crystals the merge takes are written: one whose cell is not the run's was indexed
+    // in another lattice (on few spots, typically an axis multiple), so its indices do not name the
+    // reflections they would be read as. A rotation run has the one lattice.
+    const auto &indexing = experiment.GetIndexingSettings();
+    const auto merged_crystal = [&](const IntegrationOutcome &outcome) {
+        return scanning || outcome.latt.GetUnitCell().is_close(unitCell, indexing.GetUnitCellDistTolerance(),
+                                                               indexing.GetUnitCellAngleTolerance_deg());
+    };
+
     std::vector<std::vector<Reflection>> fulls;
     if (scanning && sum_partials)
         fulls = SumRockingEvents(outcomes,
@@ -904,8 +1066,9 @@ gemmi::Mtz UnmergedMtz(const std::vector<IntegrationOutcome> &outcomes,
                 add_row(r);
     } else {
         for (const auto &outcome : outcomes)
-            for (const auto &r : outcome.reflections)
-                add_row(r);
+            if (merged_crystal(outcome))
+                for (const auto &r : outcome.reflections)
+                    add_row(r);
     }
 
     // Every row has its own slot, so they are filled on several workers.
@@ -951,7 +1114,7 @@ gemmi::Mtz UnmergedMtz(const std::vector<IntegrationOutcome> &outcomes,
     // the sweep, which is the one matrix POINTLESS also writes into every batch.
     std::optional<CrystalLattice> lattice_at_zero;
     for (const auto &outcome : outcomes) {
-        if (outcome.reflections.empty() || outcome.latt.CalcVolume() <= 1.0f)
+        if (outcome.reflections.empty() || outcome.latt.CalcVolume() <= 1.0f || !merged_crystal(outcome))
             continue;
         const float mid_deg = phi_start_deg(outcome.reflections.front().image_number) + wedge_deg / 2.0f;
         lattice_at_zero = gon ? outcome.latt.Multiply(gon->GetTransformationAngle(mid_deg)) : outcome.latt;
@@ -1059,9 +1222,10 @@ void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes
                                  const UnitCell &unitCell,
                                  const DiffractionExperiment &experiment,
                                  bool sum_partials,
+                                 const OutputProvenance &provenance,
                                  const std::string &filename,
                                  size_t nthreads) {
-    gemmi::Mtz mtz = UnmergedMtz(outcomes, unitCell, experiment, sum_partials, nthreads);
+    gemmi::Mtz mtz = UnmergedMtz(outcomes, unitCell, experiment, sum_partials, provenance, nthreads);
     SetUnmergedMtzMosaicity(mtz, outcomes);
     mtz.write_to_file(filename);
 }
@@ -1072,6 +1236,7 @@ void WriteReflections(const std::vector<MergedReflection> &reflections,
                       const MergeStatistics &statistics,
                       const ErrorModelReport &error_model,
                       const TwinningAnalysisResult &twinning,
+                      const OutputProvenance &provenance,
                       const std::string &filename,
                       size_t nthreads,
                       const std::vector<ScaledFull> *scaled_fulls) {
@@ -1079,8 +1244,9 @@ void WriteReflections(const std::vector<MergedReflection> &reflections,
     // phenix reflection tools, mmCIF for deposition and as the self-describing native format, HKLF-4 for
     // SHELXL refinement and SHELXC / ANODE substructure solution). The .hkl holds the unmerged scaled
     // fulls where the merge provides them (rotation), the merged reflections otherwise (stills).
-    WriteMtzReflections(reflections, unitCell, experiment, filename + ".mtz");
+    WriteMtzReflections(reflections, unitCell, experiment, provenance, filename + ".mtz");
     WriteMmcifReflections(reflections, unitCell, experiment, statistics, error_model, twinning,
+                          provenance,
                           filename + ".cif", nthreads);
     if (scaled_fulls && !scaled_fulls->empty())
         WriteShelxHklReflections(*scaled_fulls, filename + ".hkl", nthreads);