5 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	107bcae0f0	merge: de-bias the error-model variance fit (chi^2 was ~2.2, now ~1) and report chi^2 ... RefineErrorModel fits the variance model a*sigma^2 + (b*<I>)^2 to the binned *median* of the squared symmetry-mate deviations, chosen for robustness. But a single deviation squared over its variance is chi-square(1)-distributed, whose median is only 0.4549 of its mean, so the fit was calibrating the variances to 0.4549x their true value: the merged sigmas came out ~1.48x too small and the achieved reduced chi^2 was 1/0.4549 = 2.2, not 1. The error model was internally well-behaved (flat chi^2 across resolution) but globally over-confident, which inflated ISa (=1/b) by ~1.48x and made the exported sigmas too optimistic for downstream weighting / French-Wilson. Divide the binned median by the chi-square(1) median (0.4549) to recover an unbiased estimate of the mean E[dev^2]=sigma^2, keeping the robustness of the median while targeting reduced chi^2 = 1. Also compute the achieved median reduced chi^2 (same normalization) and report it on the "Error model" line so mis-calibration can no longer drift silently. Verified: HEWL rotation a 0.588->1.292, b 0.052->0.077, ISa 19.1->12.9, chi^2 2.17->1.06; serial Jet8 ISa 1.0->0.7, chi^2 0.92. Relative ISa comparisons and all CC1/2/CCref/anomalous metrics are unchanged (sigma-independent or a common constant); only the absolute sigma calibration is corrected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 15:57:42 +02:00
leonarski_fandClaude Opus 4.8	ed98775ba3	rot3d: capture-aware sigma for under-captured fulls + expose per-image scale ... The rot3d combine over-extrapolates fulls reconstructed from only a fraction f of their rocking curve (min_partiality admits f as low as 0.02). Against XDS on HEWL these low-capture fulls are systematically biased HIGH (+15% at f=0.8 to +100% at f=0.3), and the bias - not random scatter - is the strong-reflection floor that hurts anomalous accuracy. --capture-uncertainty <coeff> (default 0 = off, baseline bit-identical) adds a systematic uncertainty ~coeff*(1-f)*I to each full's sigma, so the merge down-weights the over-extrapolated fulls and the error model treats their scatter as expected. Unlike outlier rejection (which trades accuracy for CC1/2), this fixes a real bias, so accuracy improves: at coeff=1.0 the anomalous peak height vs XDS rises CL_CL +16%, SD_MET/SG_CYS +5-6%, ISa 10.7->11.0. Rotation- only (no-op for stills, which never combine). Also expose the per-image scale offline: Combine3D now carries the first-pass per-image scale metadata (G, B, mosaicity, wedge, CC) forward instead of dropping it, and jfjoch_process -M writes <prefix>_image.dat from it (the offline self-scaling result was otherwise unobservable - process.h5's per-image arrays are only filled on the online path). This enabled the XDS DECAY comparison (jfjoch G tracks XDS, r=0.93). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-27 19:41:20 +02:00
leonarski_fandClaude Opus 4.8	786af96b3b	SearchSpaceGroup: POINTLESS-style rewrite, pipeline integration, twinning test ... Space-group search (image_analysis/scale_merge/SearchSpaceGroup): - Two-stage POINTLESS-style determination. Stage A scores each distinct rotation operator once (was once per candidate space group, ~34x faster on lysozyme: ~26s -> <1s) and picks the largest point group all of whose operators confirm. Stage B picks the maximal space group whose predicted absences are confirmed weak, fixing the prototype's default to the symmorphic group (it returned P422 instead of P4(3)2(1)2). Enantiomorphic / origin-ambiguous pairs (P4(1) vs P4(3), I222 vs I2(1)2(1)2(1)) are reported as indistinguishable. - Constrain candidates to subgroups of the lattice (metric) holohedry and weigh centering only P-vs-metric, fed from rotation indexing's LatticeSearch result. Integration / pipeline: - With no user-fixed space group, predict in P (IndexAndRefine) so the centering-absent reflections are integrated and the search can confirm/deny centering (catching pseudo-centering / a missed superstructure) instead of trusting the metric; a user-fixed group still rejects absences in integration. - JFJochProcess: scale+merge in P1 -> determine the space group -> set it and re-scale+merge in it (statistics then come out in the right symmetry) -> write it to /entry/sample/space_group_number (new EndMessage.space_group_number, preferred by NXmx::Sample). jfjoch_scale no longer searches; it consumes the file's space group (and no longer clobbers it with an empty -S). Twinning (new image_analysis/scale_merge/TwinningAnalysis): Padilla-Yeates L-test (<|L|>, <L^2>; acentric-only, positive intensities so L is bounded) plus a shell-normalised <I^2>/<I>^2 second moment and a twin-fraction estimate. Reported after the final merge in jfjoch_process and jfjoch_scale, and surfaced in the jfjoch_viewer merge-statistics window with a red outline when twinning is suspected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 16:11:28 +02:00
leonarski_f	54c667190f	v1.0.0-rc.155 (#65) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Remove pixelrefine option (replaced with ProfileIntegrate2D) * jfjoch_viewer: Some graphical improvements. * jfjoch_viewer: Simplify und unify data analysis settings. * jfjoch_writer: Add TCP keepalive to increase robustness if jfjoch_broker "dies" in the middle of data acquisition. Reviewed-on: #65	2026-06-25 22:01:48 +02:00
leonarski_f	75e401f0e5	v1.0.0-rc.153 (#63) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing * jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files * jfjoch_broker: Add ROI calculation in non-FPGA workflow * jfjoch_broker: Fixes to TCP image pusher * jfjoch_broker: Remove NUMA bindings * jfjoch_broker: Improvements to indexing * jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values * jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors * jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs * jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results Reviewed-on: #63	2026-06-23 20:29:49 +02:00