RESULT-CHANGING: Refinement: the bulk-solvent mask grid is never coarser than the density grid

The mask grid is capped at 2^26 points; a cell larger than that got the spacing that fills it. For
4YBB (5.7e7 A^3) that was 0.95 A, coarser than the 0.8 A shrink radius, so the shrink reached no grid
neighbour and was silently skipped: the macromolecule kept the whole 1.0 A probe, the solvent fraction
fell from 0.68 to 0.61 and F_mask correlated only 0.81 / 0.59 with phenix's at 8-5 / 5-3 A. Where the
cap would coarsen the mask past d_min / 3 the mask now uses the density grid itself (0.70 A here),
whose points are on the card anyway, so the engine grows only by the mask's labels (8.22 -> 8.36 GB).

The R-factor against grid spacing has a cliff at the shrink radius (gemmi Refmac masks, rugnux Fc,
python scaling): 0.95 and 0.85 A give R-free 0.286, 0.78 / 0.70 / 0.60 / 0.40 A all 0.271.

4YBB, refine arm (deposited model + SF, depositor's free set):
  as given      R-work 0.2771 -> 0.2573, R-free 0.2876 -> 0.2697   (CPU engine identical)
  placement     R-free 0.2782 -> 0.2627
  restrained    R-free 0.2808 -> 0.2621
  water build   R-work 0.2510 -> 0.2345, R-free 0.2697 -> 0.2547
phenix.model_vs_data on the same files: 0.2349 / 0.2461.
Only a set whose cap exceeds d_min / 3 changes; of the batteries' sets that is 4YBB alone (1hr2,
1k4c score-only bit-identical). Cells capped past 0.8 A at lower resolution (d_min / 3 above the
cap) keep the coarse grid and still lose their shrink.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQqN9qRD57vnShzTC37

docs/CPU_DATA_ANALYSIS_REFINEMENT.md

@@ -86,7 +86,8 @@ gradient gather, whatever the number of atoms. The derivative of a riding hydrog
 
 **Scale and solvent.** The flat bulk-solvent mask is drawn with the REFMAC radii (probe 1.0 Å, shrink 0.8 Å)
 on a grid of its own, with a spacing of $d_\mathrm{min}/3$ or 0.4 Å, whichever is finer (coarser only where
-a cell would need more than $2^{26}$ points). On the atoms' density grid the shrink is lost at low
+a cell would need more than $2^{26}$ points, and then never coarser than the density grid, which a cell that
+large uses for its mask). On the atoms' density grid the shrink is lost at low
 resolution: at 3 Å its 1 Å step leaves no grid point within the shrink radius. $k_\mathrm{sol}$ and
 $B_\mathrm{sol}$ are searched over their physical range on the working reflections, and the overall scale
 and anisotropic B are fitted at each point of that search (§14.2) - at the points of its fine pass by

docs/GPU_DATA_ANALYSIS.md

@@ -269,7 +269,7 @@ and both gradients come back into pinned buffers. The minimiser (L-BFGS) stays o
 Once per macro-cycle ([§15.3](CPU_DATA_ANALYSIS_REFINEMENT.md#153-scale-bulk-solvent-and-the-likelihood-parameters)):
 
 - **F_calc and F_mask** - the forward path above without the target, and the bulk-solvent mask drawn on the card
-  (`ModelMaskGPU.cu`) on **its own fine grid** (d_min/3 or 0.4 A, whichever is finer, at most 2^26 points),
+  (`ModelMaskGPU.cu`) on **its own fine grid** (d_min/3 or 0.4 A, whichever is finer, at most 2^26 points but never coarser than the density's grid),
   with a grid, spectrum and cuFFT plan of its own where that grid is not the density's: one block per
   (atom, symmetry image) sets the points within the atom's REFMAC radius, the islands are removed by a
   union-find labelling (integer `atomicCAS`, rooted at each component's lowest index, so the partition does not

image_analysis/structure_refinement/RefinementXray.cpp

@@ -255,14 +255,24 @@ RefinementReflections PrepareRefinementReflections(const std::vector<MergedRefle
 
 BulkSolventMask::BulkSolventMask(const gemmi::UnitCell &cell, const gemmi::SpaceGroup &sg, double d_min,
                                  size_t nthreads)
-    : grid_(MaskGrid(cell, sg, d_min)), nthreads_(nthreads),
-      spacing_(std::max(std::min(d_min / 3.0, MASK_SPACING_MAX), std::cbrt(cell.volume / MASK_POINTS_MAX))) {}
+    : grid_(MaskGrid(cell, sg, d_min)), nthreads_(nthreads), spacing_(MaskSpacing(cell, d_min)) {}
+
+double BulkSolventMask::MaskSpacing(const gemmi::UnitCell &cell, double d_min) {
+    const double spacing = std::max(std::min(d_min / 3.0, MASK_SPACING_MAX), std::cbrt(cell.volume / MASK_POINTS_MAX));
+    return std::min(spacing, d_min / 3.0);
+}
 
 gemmi::Grid<float> BulkSolventMask::MaskGrid(const gemmi::UnitCell &cell, const gemmi::SpaceGroup &sg, double d_min) {
+    // the cap would coarsen it beyond the atoms' grid: that grid, whose points the density holds anyway
+    if (std::cbrt(cell.volume / MASK_POINTS_MAX) > d_min / 3.0) {
+        gemmi::Grid<float> grid = RigidBodyZoneGrid(cell, sg, d_min);
+        std::vector<float>().swap(grid.data);
+        return grid;
+    }
     gemmi::Grid<float> grid;
     grid.unit_cell = cell;
     grid.spacegroup = &sg;
-    const double spacing = std::max(std::min(d_min / 3.0, MASK_SPACING_MAX), std::cbrt(cell.volume / MASK_POINTS_MAX));
+    const double spacing = MaskSpacing(cell, d_min);
     // set_size_from_spacing()'s size, without allocating the values
     const std::array<int, 3> n = gemmi::good_grid_size({{cell.a / spacing, cell.b / spacing, cell.c / spacing}},
                                                        gemmi::GridSizeRounding::Up, &sg);

image_analysis/structure_refinement/RefinementXray.h

@@ -60,8 +60,11 @@ struct XrayRowModel {
 // probe and shrink radii (1.0 and 0.8 A), which a grid at d_min / 3 cannot resolve at low resolution - at 3 A
 // its 1 A step leaves no grid point within the shrink radius and the shrink is skipped. The step is therefore
 // d_min / 3 or MASK_SPACING_MAX, whichever is finer - where the mask grid stays within MASK_POINTS_MAX points; a
-// cell too large for that gets the spacing that fills it. The R-factors of low-resolution models settle at a
-// step of 0.4 A (within 0.001 of 0.3 A) and move by up to 0.005 between 0.4 and 0.8 A.
+// cell too large for that gets the spacing that fills it, but never one coarser than the atoms' density grid
+// (RigidBodyZoneGrid, d_min / 3), which it then is: the density's points are on the card anyway, and the mask's
+// own grid only ever saved memory. The R-factors of low-resolution models settle at a step of 0.4 A (within 0.001
+// of 0.3 A) and move by up to 0.005 between 0.4 and 0.8 A; a cell coarsened by the cap past the shrink radius has
+// no shrink at all, its macromolecule swollen by the whole probe, which costs far more.
 // The flat solvent model of Fokine & Urzhumtsev (2002) Acta Cryst. D58, 1387-1392
 // The probe-and-shrink mask of Jiang & Brunger (1994) J. Mol. Biol. 243, 100-115
 class BulkSolventMask {
@@ -71,6 +74,8 @@ public:
     BulkSolventMask(const gemmi::UnitCell &cell, const gemmi::SpaceGroup &sg, double d_min, size_t nthreads);
     // The mask's grid (sizes and spacing set, values not allocated) - also the GPU engine's (RefinementGPU.cpp).
     static gemmi::Grid<float> MaskGrid(const gemmi::UnitCell &cell, const gemmi::SpaceGroup &sg, double d_min);
+    // The step that grid is asked for.
+    static double MaskSpacing(const gemmi::UnitCell &cell, double d_min);
     // Fmask (unscaled) of `model` at `hkl`.
     void Fmask(const gemmi::Model &model, const std::vector<gemmi::Miller> &hkl,
                std::vector<std::complex<float>> &f) const;