2 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	3d7891c347	rugnux: weight every CC1/2 by the information a reflection carries ... The resolution cutoff, the shell table and the overall CC1/2 counted every unique reflection equally. The merge itself is inverse-variance weighted, so an observation from a frame the crystal barely diffracted on enters it at 1/G^2 of a good one - honestly, with its sigma - but a reflection measured only on such frames is scaled-up noise that then counts as much as a well-measured pair in every Pearson CC1/2 read off the merge. On a sweep where half the frames are weak the curve collapses at every resolution: the cut lands at 3.7 A on a P1 crystal whose good frames reach 1.5 A, and the UNUSABLE verdict fires (CC1/2 0.40 beside I/sigma 10). Each merged reflection now carries cc_weight: the precision its half-sets would have had with every observation at the run's typical frame scale, over the precision they have. G_ref = sum G^3 / sum G^2 over the usable observations is the precision-weighted typical scale, which the dead frames cannot drag down however many there are; the factor per observation is max(1, (G_ref/G)^2), with G the frame's total scale (partial scale, flux and the fulls' own G) taken before the correction surfaces and before collapsed frames are dropped, so nothing intensity- or resolution-dependent enters it. On a sweep without a weak stretch every weight is 1 and the CC1/2 is the plain Pearson it was. The cutoff fit, the shell table and the overall CC1/2 (and so the UNUSABLE verdict and the report's shell checks) all read the same weighted statistic. The two extra per-group sums are accumulated on both merge paths, the host loop and MergeAccumKernel, from one per-frame factor array; a host recompute of the device sums agrees to 1e-15 relative on every merge after the corrected corr is uploaded, and a host-merge run gives the same cut, space group, CC1/2 and ISa on five sets. Weighting by the half-set error variance alone (1/(v0+v1)) is not this: v grows with the intensity, so it weights the weak end of the intensity distribution and biases homogeneous data coarser. Measured (written resolution, together with the weighted outlier median): a P1 sweep with a long weak stretch 3.73 -> 1.40 A against a 1.63 A XDS reference, UNUSABLE withdrawn (overall CC1/2 0.40 -> 0.97); a second 3.61 -> 3.00 A; one with most of the sweep out of beam 7.35 -> 5.20 A. Lysozyme, thaumatin and two insulin sets unchanged to 0.01 A (one lysozyme sweep with a weak wedge 1.13 -> 1.16 A, from the median), same space groups throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-20 18:45:03 +02:00
leonarski_f	fc68a9baed	v1.0.0-rc.146 (#56) ... This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144. jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset. jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling. jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file. jfjoch_scale: Dedicated tool for rescaling/merging existing data. jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel. WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being. Reviewed-on: #56	2026-05-28 18:48:35 +02:00