mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 72 Wiki Activity

rugnux: name the unmerged export from the prefix, and drop centring absences

Browse Source 
--export-unmerged and --export-unmerged-partials no longer take a filename;
they write <prefix>_unmerged.mtz and <prefix>_unmerged_partials.mtz, as
--write-process-h5 already does for <prefix>_process.h5.

Lattice-centring absences are no longer written. They are integrated on
purpose - prediction runs in P so the space-group search can confirm or
disprove the centring - but in an exported file they cost the reader the
answer: pointless reads the interstitial nodes, takes the lattice for
primitive, and demotes I23 to P2_1 3, while aimless roughly halves its
<I/sigma> merging them. They were also only present when the indexer had
settled on the conventional centred cell, so the file's content depended on
which of two equivalent cells it picked.

Screw and glide absences are still written. They are the evidence a space
group is chosen on, so a program reading this file can check that choice
independently; deleting them turns pointless's test into an assumption
("we cannot determine if the reflections are truly systematically absent").
Deleting them changes no merging statistic. XDS and DIALS draw the line in
the same place - XDS ships screw absences and never measures a centring one,
and DIALS's absence machinery has only screw-axis classes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CHMmeM1d489zvNFT7ZMN2P
This commit is contained in:
leonarski_f
2026-08-25 17:21:21 +02:00
co-authored by Claude Opus 5
parent bb5aaca68e
commit ef07770c5f
5 changed files with 46 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CHANGELOG.md
+1 -1
View File
@@ -3,7 +3,7 @@
### 1.0.0-rc.164
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
* rugnux: `--export-unmerged <file.mtz>` writes the integrated observations as an unmerged MTZ - readable by aimless, pointless, careless and `iotbx.merging_statistics` - in `--mode mx` and `--mode scale` alike, with each rotation reflection's partials summed into one full; `--export-unmerged-partials <file.mtz>` writes one row per image instead.
* rugnux: `--export-unmerged` writes the integrated observations as `<prefix>_unmerged.mtz`, an unmerged MTZ readable by aimless, pointless, careless and `iotbx.merging_statistics`, in `--mode mx` and `--mode scale` alike; each rotation reflection's partials are summed into one full, and `--export-unmerged-partials` writes one row per image instead. Lattice-centring absences are not written; screw and glide absences are.
### 1.0.0-rc.163
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
image_analysis/WriteReflections.cpp
+12
View File
@@ -5,6 +5,7 @@
#include "scale_merge/Merge.h"
#include "scale_merge/HKLKey.h"
#include "scale_merge/TwinningAnalysis.h"
#include "bragg_integration/SystematicAbsence.h"
#include "../common/ParallelFor.h"
#include <algorithm>
@@ -692,7 +693,18 @@ void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes
// it, and every program this file is for wants to handle it its own way.
gemmi::UnmergedHklMover hkl_mover(mtz.spacegroup);
std::set<int> batch_numbers;
// Lattice-centring absences are integrated on purpose - prediction runs in P so the space-group
// search can confirm or disprove the centring - but they are not written here. POINTLESS reads
// the interstitial nodes, takes the lattice for primitive and demotes the space group, and they
// roughly halve AIMLESS's <I/sigma>. Screw and glide absences ARE written: they are the evidence
// the space group was chosen on, so a program reading this file can check that choice, and
// removing them turns its test into an assumption. XDS and DIALS draw the line in the same place.
const char centering = mtz.spacegroup ? mtz.spacegroup->hm[0] : 'P';
const auto add_row = [&](const Reflection &r) {
if (systematic_absence(r.h, r.k, r.l, centering))
return;
std::array<int, 3> hkl{r.h, r.k, r.l};
const int isym = hkl_mover.move_to_asu(hkl);
// A summed full's image_number is its rocking-curve centroid, so this is the batch the
rugnux/Rugnux.cpp
+8 -6
View File
@@ -2991,17 +2991,19 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
// rather than the merged ones: the partiality and the per-image scale are left for the reading
// program, which fits a scale model of its own.
if (full && !cancelled_ && write_output && !geometry_prepass && result.consensus_cell) {
if (!config_.unmerged_export_path.empty()) {
if (config_.export_unmerged) {
if (observer) observer->OnPhase("Writing unmerged reflections");
const std::string path = config_.output_prefix + "_unmerged.mtz";
WriteUnmergedMtzReflections(indexer->GetIntegrationOutcome(), *result.consensus_cell,
experiment_, true, config_.unmerged_export_path);
logger.Info("Unmerged observations written to {}", config_.unmerged_export_path);
experiment_, true, path);
logger.Info("Unmerged observations written to {}", path);
}
if (!config_.unmerged_partials_export_path.empty()) {
if (config_.export_unmerged_partials) {
if (observer) observer->OnPhase("Writing unmerged reflections");
const std::string path = config_.output_prefix + "_unmerged_partials.mtz";
WriteUnmergedMtzReflections(indexer->GetIntegrationOutcome(), *result.consensus_cell,
experiment_, false, config_.unmerged_partials_export_path);
logger.Info("Unmerged partials written to {}", config_.unmerged_partials_export_path);
experiment_, false, path);
logger.Info("Unmerged partials written to {}", path);
}
}
rugnux/Rugnux.h
+7 -7
View File
@@ -119,14 +119,14 @@ struct ProcessConfig {
// Diagnostic: if set, the -P rot3d combine writes the unmerged fulls here (for comparison vs XDS).
std::string observation_dump_path;
// If set, the integrated observations are written here as an unmerged MTZ (--export-unmerged),
// for aimless / pointless / careless and anything else that scales unmerged data itself - each
// rocking event summed into one full.
std::string unmerged_export_path;
// Write the integrated observations as <prefix>_unmerged.mtz (--export-unmerged), for aimless /
// pointless / careless and anything else that scales unmerged data itself - each rocking event
// summed into one full.
bool export_unmerged = false;
// The same file with each partial written as its own row instead, for the reader to sum
// (--export-unmerged-partials). Independent of the above; a run can write both.
std::string unmerged_partials_export_path;
// The same as <prefix>_unmerged_partials.mtz, with each partial written as its own row for the
// reader to sum (--export-unmerged-partials). Independent of the above; a run can write both.
bool export_unmerged_partials = false;
// If set, after merging, validate the merged intensities against this atomic model (PDB):
// compute R-free with an optimized bulk solvent and write 2Fo-Fc / Fo-Fc maps (--model).
rugnux/rugnux_cli.cpp
+18 -18
View File
@@ -162,8 +162,8 @@ void print_usage() {
std::cout << " -z, --reference-mtz <file> Reference MTZ file" << std::endl;
std::cout << " --reference-column <label> Reference MTZ column to use (default: auto - F-model, else IMEAN/I, else FP/FOBS/F)" << std::endl;
std::cout << " --model <file.pdb> After merging, validate vs this model: R-free + 2Fo-Fc/Fo-Fc maps" << std::endl;
std::cout << " --export-unmerged <file.mtz> Also write the integrated observations as an unmerged MTZ (POINTLESS column layout) for aimless / pointless / careless. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-partiality is not written, as in the merge. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Works in --mode mx and --mode scale" << std::endl;
std::cout << " --export-unmerged-partials <file.mtz> As --export-unmerged, but writes each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summing them here" << std::endl;
std::cout << " --export-unmerged Also write the integrated observations as <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) for aimless / pointless / careless. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-partiality is not written, as in the merge. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Works in --mode mx and --mode scale" << std::endl;
std::cout << " --export-unmerged-partials As --export-unmerged, but writes <prefix>_unmerged_partials.mtz with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summing them here" << std::endl;
std::cout << std::endl;
std::cout << " Integration" << std::endl;
@@ -289,8 +289,8 @@ static option long_options[] = {
{"reference-mtz", required_argument, nullptr, 'z'},
{"reference-column", required_argument, nullptr, OPT_REFERENCE_COLUMN},
{"model", required_argument, nullptr, OPT_MODEL},
{"export-unmerged", required_argument, nullptr, OPT_EXPORT_UNMERGED},
{"export-unmerged-partials", required_argument, nullptr, OPT_EXPORT_UNMERGED_PARTIALS},
{"export-unmerged", no_argument, nullptr, OPT_EXPORT_UNMERGED},
{"export-unmerged-partials", no_argument, nullptr, OPT_EXPORT_UNMERGED_PARTIALS},
{"dump-observations", required_argument, nullptr, OPT_DUMP_OBSERVATIONS},
{"space-group", required_argument, nullptr, 'S'},
{"anomalous", no_argument, nullptr, 'A'},
@@ -629,8 +629,8 @@ static int RunRugnux(int argc, char **argv) {
std::string ref_mtz;
std::string ref_column;
std::string model_pdb; // --model: PDB to validate merged intensities against (R-free + maps)
std::string export_unmerged; // --export-unmerged: unmerged MTZ of the integrated observations
std::string export_unmerged_partials; // --export-unmerged-partials: the same, one row per image
bool export_unmerged = false; // --export-unmerged: <prefix>_unmerged.mtz
bool export_unmerged_partials = false; // --export-unmerged-partials: the same, one row per image
std::string dump_observations; // diagnostic: dump unmerged -P rot3d fulls to this path
double min_partiality = 0.02;
std::optional<double> min_captured_fraction_arg; // explicit --min-captured-fraction; default depends on rotation
@@ -854,10 +854,10 @@ static int RunRugnux(int argc, char **argv) {
model_pdb = optarg;
break;
case OPT_EXPORT_UNMERGED:
export_unmerged = optarg;
export_unmerged = true;
break;
case OPT_EXPORT_UNMERGED_PARTIALS:
export_unmerged_partials = optarg;
export_unmerged_partials = true;
break;
case OPT_DUMP_OBSERVATIONS:
dump_observations = optarg;
@@ -1542,15 +1542,15 @@ static int RunRugnux(int argc, char **argv) {
// Unmerged observations (--export-unmerged), from the integrated observations rather than the
// merged ones: the partiality and the per-image scale are left for the reading program, which
// fits a scale model of its own.
if (!export_unmerged.empty()) {
WriteUnmergedMtzReflections(reflections, *experiment.GetUnitCell(), experiment, true,
export_unmerged);
logger.Info("Unmerged observations written to {}", export_unmerged);
if (export_unmerged) {
const std::string path = output_prefix + "_unmerged.mtz";
WriteUnmergedMtzReflections(reflections, *experiment.GetUnitCell(), experiment, true, path);
logger.Info("Unmerged observations written to {}", path);
}
if (!export_unmerged_partials.empty()) {
WriteUnmergedMtzReflections(reflections, *experiment.GetUnitCell(), experiment, false,
export_unmerged_partials);
logger.Info("Unmerged partials written to {}", export_unmerged_partials);
if (export_unmerged_partials) {
const std::string path = output_prefix + "_unmerged_partials.mtz";
WriteUnmergedMtzReflections(reflections, *experiment.GetUnitCell(), experiment, false, path);
logger.Info("Unmerged partials written to {}", path);
}
if (!output_prefix.empty()) {
@@ -2147,8 +2147,8 @@ static int RunRugnux(int argc, char **argv) {
config.reference_data = reference_data;
config.reference_has_free_flags = reference_has_free_flags;
config.observation_dump_path = dump_observations;
config.unmerged_export_path = export_unmerged;
config.unmerged_partials_export_path = export_unmerged_partials;
config.export_unmerged = export_unmerged;
config.export_unmerged_partials = export_unmerged_partials;
config.model_path = model_pdb;
// When merging, the merged reflections (.mtz/.cif) are the wanted output; skip the large
// _process.h5 unless explicitly requested. Without merging, the _process.h5 is the only output.