Merge host-mem-2 into rc176: a host-memory budget from the machine's total RAM decides what runs side by side, the CPU ingest reads the reflections in place (24 B per partial), merge scratch is really freed, and CPU MR/validation workers are bounded by their bytes; output unchanged

Peak live heap, CPU path -N10, before -> after: 4nwv scaling 14.8 -> 11.9 GiB, analyses 13.5 -> 7.3;
8qaw scaling 34.0 -> 27.1; 9v16 17.2 -> 13.5; 9zlo/9gjx/9s02 7.7/7.4/8.9 -> 6.3/6.3/7.5. With the
budget of a 16 GB machine, 4nwv validation 18.4 -> 10.8-12.3 and MR ~17 -> 12.5. GPU 4nwv 16.2 ->
14.8. Byte-identical to its base on GPU and --no-gpu (timing standard + inhouse, 4nwv/8qaw/9v16/
lyso_x06da_ref/cytc_x10sa), at the machine's budget and with a 16 GB one. Conflict in
RugnuxScaleMerge.cpp: perf-8gb-cpu's card-byte declaration of the merge and this branch's host-budget
decisions on the post-refinement and the all-observation arm, kept both, in that order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-17 17:59:30 +02:00
commit 9c9b987928
17 files changed
+641 -161

No files matched your search

@@ -35,6 +35,7 @@
#include "../../common/CorrelationCoefficient.h"
#include "../../common/JFJochMath.h" // PI (M_PI is not standard, and MSVC does not define it)
#include "../../common/HostMemory.h"
#include "../../common/Logger.h"
#include "../../common/ParallelFor.h"
#include "../scale_merge/ReindexAmbiguity.h" // ReindexReflections
@@ -1247,7 +1248,17 @@ ModelValidationResult Validate(const std::vector<MergedReflection> &merged,
// the order the concurrent replicates finish in.
std::vector<double> null_r_work(NULL_REPLICATES), null_margin(NULL_REPLICATES);
std::atomic<int> placed_redraws{0};
// A replicate whose structure factors are made on the CPU holds a gigabyte or more of grids on a large
// cell, so it reserves them against the host's worker pool (HostWorkerPool) first: as many replicates
// run at once as it holds, whatever the thread count, and each computes what it computes either way.
size_t replicate_bytes = MODEL_FACTORS_BYTES_PER_GRID_POINT
* ModelGridPoints(null_model.cell, null_model.find_spacegroup(), null_d_min);
#ifdef JFJOCH_USE_CUDA
if ((sf_gpu != nullptr && null_d_min == sf_gpu->DMin()) || (sf_null != nullptr && null_d_min == sf_null->DMin()))
replicate_bytes = 0;
#endif
auto run_replicate = [&](int i) {
const HostWorkerPool::Reservation reserved(HostWorkerPool::Instance(), replicate_bytes);
std::mt19937 rng(NULL_SEED + 1 + i);
gemmi::Mat33 rot = null_rotation[i];
for (int redraw = 0;; redraw++) {