Commit Graph
3 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 4.8 85fcac09b5 scale_merge: floor the R-free test-set size on small data
Build Packages / build:viewer-tgz:cpu (push) Successful in 7m31s
Build Packages / build:viewer-tgz:cuda (push) Successful in 8m41s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m27s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m46s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m49s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m4s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m10s
Build Packages / build:windows:nocuda (push) Successful in 15m55s
Build Packages / build:windows:cuda (push) Successful in 18m8s
Build Packages / build:rpm (rocky8) (push) Successful in 11m29s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m56s
Build Packages / XDS test (durin plugin) (push) Successful in 7m50s
Build Packages / Generate python client (push) Successful in 33s
Build Packages / Build documentation (push) Successful in 1m6s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (rocky9) (push) Successful in 13m4s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m52s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 13m56s
Build Packages / DIALS test (push) Successful in 14m2s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m4s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m14s
Build Packages / Unit tests (push) Successful in 1h37m26s
AssignRfreeFlags gains a min_free_reflections floor (default 500): on small
data, where the 5% fraction would give too few test reflections for a stable
R-free (Brunger's ~500-2000 rule), the fraction is lifted toward ~500 free
reflections, capped at 10% so a large test set never steals working data. An
explicit rfree_fraction is still honoured; pass 0 to disable.

For ordinary data (5% already clears the floor) this is inactive and the
fraction stays flat, so the cross-dataset-identical property is preserved; the
floor only lifts the fraction on genuinely small datasets, where per-dataset
R-free stability outweighs cross-dataset identity (a shared reference set keeps
exact identity there). Tests updated to isolate the pure-hash guarantees with
the floor off, plus a new small-data floor + cap test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 11:50:28 +02:00
leonarski_fandClaude Opus 4.8 d891551127 rugnux: make model validation batch-consistent for fragment screening
Reworks the --model / French-Wilson / R-free path so a batch of datasets of
one crystal form yields comparable maps and a shared test set, and fixes a
French-Wilson prior bug.

- French-Wilson: fix inverted centric/acentric prior - the call passed
  !centric into the selector, swapping the two Wilson forms for every weak
  reflection. Add a test pinning a centric weak |F| below an acentric one.
- R-free flags: assign by a pure per-hkl hash instead of a per-dataset
  resolution-shell stratification, so every dataset of one crystal form gets
  the same free set (needed for ensemble refinement / PanDDA). Still a pure
  function of the Laue-ASU key, so Bijvoet/symmetry mates never split.
- R-free flags: import a reference MTZ's FreeR_flag column when present
  (CCP4 test=0, auto-complement for a phenix-style test=1), so a whole
  campaign inherits one shared test set. New ApplyReferenceFreeFlags, wired
  post-merge in both the full-analysis and --scale paths.
- Model maps: scale the model with an overall scale, anisotropic B and flat
  bulk solvent only - drop the free-form per-shell K(1/d^2) rescale, which
  reshaped each dataset's radial amplitude profile differently and made a
  batch of maps non-comparable.
- Merohedral indexing: model validation resolves the ambiguity by lowest
  R-free only when there is no reference; with a reference MTZ the indexing
  is already resolved against it (merge stage / stills scaling) and that
  authoritative choice is kept.
- Remove a stale <algorithm> include; update docs and changelog.

Tests: french_wilson, rfree, reference_mtz (new), reindex, hkl_key all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 11:31:33 +02:00
leonarski_fandClaude Opus 4.8 4c95a289b6 scale_merge: harden R-free flag assignment (Bijvoet-safe, deterministic, stratified)
The R-free flags were drawn with a per-reflection std::bernoulli_distribution off a
shared std::mt19937(12345). Two problems: in anomalous mode I(+) and I(-) are separate
merged rows, so a Bijvoet pair could be split across the work and free sets (biasing
R-free); and the draw order depended on merge/call order, so it was not reproducible or
parallel-safe. The resolution-shell grouping was cosmetic — it still drew an independent
Bernoulli per row, so it neither tied mates together nor guaranteed a per-shell quota.

New AssignRfreeFlags() (own file RfreeFlags.{h,cpp}) replaces both call sites (stills
MergeOnTheFly, rotation RotationScaleMerge). It:
  - keys each reflection by its Friedel-merged (Laue) ASU index, so all symmetry- and
    Friedel-equivalent reflections (including a Bijvoet pair) share one flag and are never
    split;
  - selects the free set by a deterministic splitmix64 hash of that key (same idiom as the
    CC1/2 half-set split), so it is reproducible run-to-run and order-independent;
  - stratifies by resolution: exactly round(fraction * n) of the distinct reflections in
    each of 20 shells is flagged free.

Verified on an anomalous lyso merge (29536 refl, 5% free): 13548 Bijvoet pairs, 0 split;
free fraction 4.6-5.4% in every resolution shell.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 21:27:44 +02:00