Commit Graph
4 Commits
Author SHA1 Message Date
leonarski_fandClaude Fable 5 8c084021e6 ice rings: exclude from scaling, keep in the final merge (recover completeness)
--detect-ice-rings previously dropped ice-ring reflections outright before
scaling, which also removed them from the merge and gutted low/mid-resolution
completeness on crystals that merge them fine (e.g. lysoC 80% -> the ice bands
show as jagged shell completeness). Instead flag them (new Reflection.on_ice_ring,
set in JFJochProcess, carried through the 3D combine) and exclude the flag only
where a model is fit: the per-image scale (ScaleOnTheFly::Accept + the per-image
CC), and - for the de-novo P1 pass - the space-group search and error model
(MergeOnTheFly::ExcludeIceRings, on for for_search). The final in-symmetry merge
and its statistics keep them, so completeness is preserved.

/data/rotation_test battery vs the previous drop-from-merge behaviour: space
group correct on all 18; completeness recovered broadly with CC1/2 and ISa held
(cytC_2 82->99.7%, cytC_3 73->99.7%, InsI3 76->99.5%, lysoC 80->99.7%, MyoB
80->99.7%, InsH3 78->99%). Excluding ice from the P1 search merge is what keeps
the space group correct: without it InsI3 flipped I23->P1 and EP_cs_01-17 P2->P1.

Known limitation: on heavy-ice crystals (EP_cs_01-17) the strong ice is garbage
and keeping it in the final merge collapses CC1/2 in the ice shells (91.7->6.9%).
Distinguishing strong vs weak/absent rings per crystal needs data-driven,
per-ring ice detection (azimuthal radial profile) - the planned next step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:29:37 +02:00
leonarski_fandClaude Opus 4.8 40f650b9e7 scale_merge: pool the per-frame background across each rocking event in Combine3D
Each partial subtracted its own independently-estimated per-frame background,
so a weak full assembled from N frames accumulated one background-estimation
variance per frame. The true background is flat over the few frames of one
rocking event, so replace each partial's background by the event mean and
correct its intensity by n_bkg*(bkg - pooled), where n_bkg = (sigma^2 - I)/bkg
is the effective background-pixel count -- correct for weak AND strong
reflections (sigma^2/bkg would over-count strong ones and over-correct them).
Single-frame events are a no-op.

Measured (rotation lyso @1.0 A): 1.05 A CC1/2 81.3 -> 83.0, R-meas 81.4 ->
77.2, CCref +0.2, CCxds +0.3; overall R-meas 10.0 -> 9.4%; ISa preserved (13.5).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 22:41:37 +02:00
leonarski_fandClaude Opus 4.8 ed98775ba3 rot3d: capture-aware sigma for under-captured fulls + expose per-image scale
The rot3d combine over-extrapolates fulls reconstructed from only a fraction f
of their rocking curve (min_partiality admits f as low as 0.02). Against XDS on
HEWL these low-capture fulls are systematically biased HIGH (+15% at f=0.8 to
+100% at f=0.3), and the bias - not random scatter - is the strong-reflection
floor that hurts anomalous accuracy.

--capture-uncertainty <coeff> (default 0 = off, baseline bit-identical) adds a
systematic uncertainty ~coeff*(1-f)*I to each full's sigma, so the merge
down-weights the over-extrapolated fulls and the error model treats their
scatter as expected. Unlike outlier rejection (which trades accuracy for CC1/2),
this fixes a real bias, so accuracy improves: at coeff=1.0 the anomalous peak
height vs XDS rises CL_CL +16%, SD_MET/SG_CYS +5-6%, ISa 10.7->11.0. Rotation-
only (no-op for stills, which never combine).

Also expose the per-image scale offline: Combine3D now carries the first-pass
per-image scale metadata (G, B, mosaicity, wedge, CC) forward instead of
dropping it, and jfjoch_process -M writes <prefix>_image.dat from it (the
offline self-scaling result was otherwise unobservable - process.h5's per-image
arrays are only filled on the online path). This enabled the XDS DECAY
comparison (jfjoch G tracks XDS, r=0.93).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 19:41:20 +02:00
leonarski_f 54c667190f v1.0.0-rc.155 (#65)
Build Packages / Unit tests (push) Successful in 1h26m8s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m38s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m45s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m39s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m55s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m51s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m35s
Build Packages / build:rpm (rocky8) (push) Successful in 12m28s
Build Packages / build:rpm (rocky9) (push) Successful in 13m20s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m15s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m43s
Build Packages / DIALS test (push) Successful in 14m21s
Build Packages / XDS test (durin plugin) (push) Successful in 7m48s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m52s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m31s
Build Packages / Generate python client (push) Successful in 15s
Build Packages / Build documentation (push) Successful in 53s
Build Packages / Create release (push) Skipped
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_process: Remove pixelrefine option (replaced with ProfileIntegrate2D)
* jfjoch_viewer: Some graphical improvements.
* jfjoch_viewer: Simplify und unify data analysis settings.
* jfjoch_writer: Add TCP keepalive to increase robustness if jfjoch_broker "dies" in the middle of data acquisition.

Reviewed-on: #65
2026-06-25 22:01:48 +02:00