5 Commits

Author	SHA1	Message	Date
leonarski_f	c84d581d3e	Remove the global (joint) scaling path ... Profiling showed the joint Ceres solve took ~120 s versus ~3.5 s for the alternating per-image scaling loop (~35x) for no quality gain (HEWL anomalous 0.54x vs 0.53x), so it is not worth keeping. Drop GlobalScale.{h,cpp}, the jfjoch_process --global-scale flag, and ScalingSettings::GlobalScaling. While here, in the same scaling/process area: fold scale_fulls into ScalingSettings (alongside combine_3d) so the CLI and experiment carry it uniformly, add per-substep [timing] logging to the scaling/merge post-pass (including the serial MergeAll vs parallel ScaleAllImages split), and carry structured MergeStatistics + ISa in ProcessResult. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-25 20:43:04 +02:00
leonarski_f	8d0cd19e48	Add XDS-order scaling of the rot3d fulls: --scale-fulls ... The rot3d combine emits fulls with partiality == 1 and image_scale_corr == 1, so the fulls are only ever scaled as per-frame partials upstream - their per-frame scale G is fit through the rocking-curve/partiality model (G*partiality*B*lp*Itrue - Iobs) and so absorbs any model error. XDS/DIALS instead scale the 3D-integrated fulls directly. --scale-fulls inserts a second scaling pass on the combined fulls with the Unity model (G*Itrue - I_full, no partiality term), between Combine3D and MergeOnTheFly, reusing ScaleOnTheFly on a Unity-configured experiment copy. It is a pure post-correction (updates the fulls' image_scale_corr 1 -> 1/G, no re-combine). HEWL crystal 2, anomalous S-peak height vs XDS: 0.53x -> 0.57x and ISa 9.4 -> 10.5 - improving precision and accuracy together (not the CC1/2-up / anomalous-down trade-off of outlier rejection). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-25 20:43:04 +02:00
leonarski_f	50ebf1694e	Add global (joint) scaling path: jfjoch_process --global-scale ... Refine all per-image scales, per-image mosaicities and the shared per-HKL true intensities (plus the global wedge) together in one Ceres problem (GlobalScale.{h,cpp}, GlobalScaleCeres), as an alternative to the existing alternating merge-then-scale-each-image loop. The result feeds back into each reflection's image_scale_corr exactly as ScaleOnTheFly does, so the rot3d combine and MergeOnTheFly run unchanged and every metric stays comparable. On HEWL crystal 2 the joint fit converges to the same place as the alternating loop (anomalous S-peak vs XDS 0.53x -> 0.54x, ISa 9.4 both), confirming the alternating path already reaches the joint optimum. Kept as a validated alternative and the home for future global corrections. A shared quadratic absorption surface in detector position was prototyped here and dropped: it fit large non-physical coefficients (radially degenerate with the per-HKL resolution structure), lowered the scaling-fit residual but raised the error-model b (ISa 9.4 -> 7.3) and did not improve anomalous accuracy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-25 20:43:04 +02:00
leonarski_f	e4b3064254	Scaling: reference dataset, 3D combine (-P rot3d), R-meas, profile-fit integrator ... Reference dataset (a): LoadReferenceMtz adds column selection + cell/SG/resolution + a data-vs-reference consistency check; jfjoch_process/jfjoch_scale gain --reference-column; the viewer gets a Reference section in the MX settings dock (worker-owned, independent of the loaded dataset) that flows into reprocessing jobs. 3D combine (-P rot3d): Combine3D weight-sums a reflection's per-frame partials into one counting-limited full before merging (orthogonal ScalingSettings::combine_3d flag, not a partiality model), with a de-biased Poisson variance. Crystal 2: ISa 1.7->8.4, R-meas ~67%->18.9%, intensities unchanged (CCref held). Quality metrics (b): R-meas (Diederichs-Karplus) + redundancy columns in MergeStats; ISa logged. jfjoch fulls 18.9% vs XDS 4.5% (same ASU/run). Profile-fit integrator (experimental): ProfileIntegrate2D (--integrator gaussian|empirical) is a reference-free, rot3d-compatible profile-fit extraction (the decomposed PixelRefine intensity step). Gaussian: R-meas 18.9->14.6%, ISa ->9.5. Anisotropy/per-region add nothing (the discriminating info is in the discarded rocking direction). See NEXTGEN_INTEGRATOR.md. --dump-observations exports the unmerged fulls for XDS comparison. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-25 20:43:04 +02:00
leonarski_f	75e401f0e5	v1.0.0-rc.153 (#63) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing * jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files * jfjoch_broker: Add ROI calculation in non-FPGA workflow * jfjoch_broker: Fixes to TCP image pusher * jfjoch_broker: Remove NUMA bindings * jfjoch_broker: Improvements to indexing * jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values * jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors * jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs * jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results Reviewed-on: #63	2026-06-23 20:29:49 +02:00