Commit Graph
4 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 4.8 d891551127 rugnux: make model validation batch-consistent for fragment screening
Reworks the --model / French-Wilson / R-free path so a batch of datasets of
one crystal form yields comparable maps and a shared test set, and fixes a
French-Wilson prior bug.

- French-Wilson: fix inverted centric/acentric prior - the call passed
  !centric into the selector, swapping the two Wilson forms for every weak
  reflection. Add a test pinning a centric weak |F| below an acentric one.
- R-free flags: assign by a pure per-hkl hash instead of a per-dataset
  resolution-shell stratification, so every dataset of one crystal form gets
  the same free set (needed for ensemble refinement / PanDDA). Still a pure
  function of the Laue-ASU key, so Bijvoet/symmetry mates never split.
- R-free flags: import a reference MTZ's FreeR_flag column when present
  (CCP4 test=0, auto-complement for a phenix-style test=1), so a whole
  campaign inherits one shared test set. New ApplyReferenceFreeFlags, wired
  post-merge in both the full-analysis and --scale paths.
- Model maps: scale the model with an overall scale, anisotropic B and flat
  bulk solvent only - drop the free-form per-shell K(1/d^2) rescale, which
  reshaped each dataset's radial amplitude profile differently and made a
  batch of maps non-comparable.
- Merohedral indexing: model validation resolves the ambiguity by lowest
  R-free only when there is no reference; with a reference MTZ the indexing
  is already resolved against it (merge stage / stills scaling) and that
  authoritative choice is kept.
- Remove a stale <algorithm> include; update docs and changelog.

Tests: french_wilson, rfree, reference_mtz (new), reindex, hkl_key all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 11:31:33 +02:00
leonarski_fandClaude Opus 4.8 87af0f419b scale_merge: compute French-Wilson amplitudes at end of merge, write them alongside intensities
Merged reflections carried only intensities; a naive sqrt(max(I,0)) turns every weak or
negative measurement into a biased (or zero) amplitude. Add a French-Wilson step so the
merged reflections themselves carry proper Bayesian amplitudes.

New FrenchWilson.{h,cpp}: ApplyFrenchWilson() fills F and sigmaF on each MergedReflection
with the posterior mean |F| given I and its sigma under the Wilson prior:
  - correct centric vs acentric prior (gemmi is_reflection_centric);
  - epsilon (symmetry-enhancement) multiplicity: the shell mean is <I/eps> and the prior
    mean for a reflection is eps * <I/eps>;
  - numerically stable log-shift integration of the posterior;
  - strong reflections (I > 4 sigma) short-circuit to sqrt(I) where the FW bias is negligible;
  - unusable I/sigma falls back to sqrt(max(I,0)).

It runs as the last step of the merge routine (both MergeOnTheFly and RotationScaleMerge),
so F/sigmaF are part of the merged result and downstream consumers (file writing AND, later,
map calculation) all use the same amplitudes rather than each recomputing FW and risking
divergence. The writers just emit the fields; extended additively:
  - MTZ: F (F) + SIGF (Q) columns;
  - mmCIF: _refln.F_meas_au / _refln.F_meas_sigma_au;
  - HKL text: F sigmaF appended (rfree flag kept in place for back-compat).
Existing intensity columns are unchanged, so current readers keep working while
phenix.refine (etc.) can now refine against the amplitudes.

Verified on lyso (1.3 A): all 173 negative-intensity reflections get F>0, strong reflections
have F/sqrt(I)=1.000, no NaN, and the mmCIF _refln loop is well-formed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 21:27:44 +02:00
leonarski_f fc68a9baed v1.0.0-rc.146 (#56)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m34s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m16s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m49s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m32s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m15s
Build Packages / XDS test (durin plugin) (push) Successful in 7m16s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m12s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 47s
Build Packages / DIALS test (push) Successful in 10m18s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m46s
Build Packages / build:rpm (rocky8) (push) Successful in 1h41m2s
Build Packages / XDS test (neggia plugin) (push) Successful in 1h59m18s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.

WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.

Reviewed-on: #56
2026-05-28 18:48:35 +02:00
leonarski_f 1ab257af6c v1.0.0-rc.125 (#32)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m8s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m55s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m0s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m30s
Build Packages / Generate python client (push) Successful in 20s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 39s
Build Packages / build:rpm (rocky8) (push) Successful in 9m23s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m42s
Build Packages / build:rpm (rocky9) (push) Successful in 9m38s
This is an UNSTABLE release. This version adds scalign and merging. These are experimental at the moment, and should not be used for production analysis.
If things go wrong with analysis, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Improve logic on switching on/off spot finding
* jfjoch_broker: Increase maximum spot count for FFBIDX to 65536
* jfjoch_broker: Increase default maximum unit cell for FFT to 500 A (could have performance impact, TBD)
* jfjoch_process: Add scalign and merging functionality - program is experimental at the moment and should not be used for production analysis
* jfjoch_viewer: Display partiality and reciprocal Lorentz-polarization correction for each reflection
* jfjoch_writer: Save more information about each reflection

Reviewed-on: #32
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-02-18 16:17:21 +01:00