Commit Graph
5 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 4.8 c93d381dc8 Error model: harden the fit against pathological inputs (code review)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 25m44s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 29m14s
Build Packages / build:rpm (rocky8) (push) Successful in 30m58s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 31m34s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 31m40s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 32m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 33m52s
Build Packages / XDS test (durin plugin) (push) Successful in 19m26s
Build Packages / Generate python client (push) Successful in 38s
Build Packages / Build documentation (push) Successful in 1m30s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (neggia plugin) (push) Successful in 20m15s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 22m14s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m41s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m45s
Build Packages / build:rpm (rocky9) (push) Successful in 29m38s
Build Packages / DIALS test (push) Successful in 34m2s
Build Packages / Unit tests (push) Successful in 2h19m4s
Addresses code-review findings on RefineErrorModel:
- Floor the 1/dev^2 bin weight relative to the data scale (1e-3 of the median
  bin dev^2), not an absolute 1e-30: a near-zero-scatter bin could otherwise
  acquire a runaway weight and hijack the global (a,b) fit.
- Reject a near-collinear normal-equation system relatively (det > 1e-10*Ass*AII)
  instead of with an absolute threshold that an ill-conditioned fit can pass.
- Reset the model to identity at entry so any early return leaves it inactive
  rather than keeping a stale a/b alongside a freshly-cleared mean map (which
  would make CorrectedSigma fall back to the per-observation I).
- PixelRefine: correct the orient_prior comment - with the sweep on, the LSQ
  anchor is the swept orientation (intended), not the spot-centroid one.

Verified unchanged on the lyso test set (ISa 1.1, CC1/2 90.3%).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 18:53:02 +02:00
leonarski_fandClaude Opus 4.8 e6a50b45c7 Integration: mean background + global error model (trustworthy sigmas)
Background estimate: use the mean of the local ring, not the median. For a
right-skewed (Poisson) background the median sits below the mean, so subtracting
it under-subtracts and biases every weak intensity positive; over multiplicity
this becomes fake <I/sig> of a few in no-signal high-resolution shells. Fixed in
both PixelRefine and BraggIntegrate2D (the classical route had the same bug).
<I/sig> now tracks CC honestly and the true resolution limit is visible.

Error model: fit a global a, b (XDS form sigma'^2 = a*sigma^2 + (b*I)^2) from the
scatter of symmetry equivalents at the merge level (so both integrators benefit),
and print it with ISa = 1/b in jfjoch_process. The (b*I)^2 term uses the
reflection mean (not the per-observation I_i, which biases the weights and
collapses CC); a,b come from a relative-weighted bin regression. Replaces the
earlier per-resolution-shell variant, which was partly masking the background bias.

METHODS.md: document both (Sections 6-7), integrator-agnostic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 18:39:32 +02:00
leonarski_f 5735302691 Merge: CC1/2 limit adjustment
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m57s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m51s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m47s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m10s
Build Packages / build:rpm (rocky8) (push) Successful in 13m10s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m27s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m16s
Build Packages / XDS test (durin plugin) (push) Successful in 8m36s
Build Packages / Generate python client (push) Successful in 22s
Build Packages / build:rpm (rocky9) (push) Successful in 12m22s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m58s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m18s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m52s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m5s
Build Packages / Build documentation (push) Successful in 38s
Build Packages / DIALS test (push) Successful in 12m2s
Build Packages / Unit tests (push) Successful in 56m39s
2026-06-09 11:06:16 +02:00
leonarski_f 32e91f7287 Minor fixes
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m46s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m52s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m54s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m35s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m49s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m9s
Build Packages / build:rpm (rocky8) (push) Successful in 9m57s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m53s
Build Packages / build:rpm (rocky9) (push) Successful in 11m18s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m3s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 34s
Build Packages / XDS test (durin plugin) (push) Successful in 8m20s
Build Packages / DIALS test (push) Successful in 12m28s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m23s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m23s
Build Packages / Unit tests (push) Successful in 54m9s
2026-06-08 15:56:20 +02:00
leonarski_f fc68a9baed v1.0.0-rc.146 (#56)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m34s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m16s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m49s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m32s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m15s
Build Packages / XDS test (durin plugin) (push) Successful in 7m16s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m12s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 47s
Build Packages / DIALS test (push) Successful in 10m18s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m46s
Build Packages / build:rpm (rocky8) (push) Successful in 1h41m2s
Build Packages / XDS test (neggia plugin) (push) Successful in 1h59m18s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.

WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.

Reviewed-on: #56
2026-05-28 18:48:35 +02:00