25 Commits

Author	SHA1	Message	Date
leonarski_f	5d9e1be814	portability: replace M_PI / std::numbers::pi with a host+device-safe PI ... M_PI is a POSIX <math.h> extension that MSVC does not define without _USE_MATH_DEFINES. std::numbers::pi (introduced in the viewer guard commit) is C++20, but CUDA here is compiled as C++17 (CMAKE_CUDA_STANDARD 17) and several common/ headers are pulled into .cu device translation units, so std::numbers is not available there. Add common/JFJochMath.h with a dependency-free `constexpr double PI` that works in host code (including MSVC), in CUDA device code, and under C++17/20, and use it everywhere: - common/ and image_analysis/ (incl. CUDA .cu): 78 M_PI occurrences, 22 files - broker/OpenAPIConvert.cpp - viewer/: the 5 files that used std::numbers::pi now use PI, for one consistent convention across the codebase Verified to build: JFJochImageAnalysis (incl. CUDA), jfjoch_viewer, JFJochBroker. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-18 14:54:01 +02:00
leonarski_f	ecdb7048a0	PixelRefine: drop crystal-system idealisation, use the indexed cell as-is ... PixelRefine no longer refines the cell, so the symmetry-constrained cell parameterisation inherited from XtalOptimizer has no manifold to constrain. The decompose->rebuild round-trip (Gram-Schmidt orientation + symmetry B matrix) merely reconstructed the lattice columns it started from, and for non-triclinic systems it re-idealised the indexed cell (averaging a,b for tetragonal; a,b,c for cubic; forcing alpha=gamma=90). Replace both six-way switches (PredictedNode and BuildParameterBlocks) with a single path: take the three real-space lattice columns (latt.Vec0/1/2()) directly and form the reciprocal node via the general cross-product inverse. This reproduces every crystal system exactly from the actual cell, is more faithful (no re-idealisation), and removes the crystal_system field plus two now-unused includes. PredictedNode de-templated (only ever called with double). Crystal symmetry still lives where it belongs: indexing (upstream) and merging (downstream via the space-group HKL key). A/B (both lyso P4_3 2_1 2 crystals, refined under tetragonal constraint, so the old idealisation was already a no-op): stat tables bit-identical across all shells -- crystal 2 CC1/2 94.6% / CCref 92.5%, jet CC1/2 91.9% / CCref 55.8% -- the only delta is 4th-digit float-ordering noise in the error-model b coefficient (0.8143->0.8141, same ISa). Same merged intensities => R-free unchanged. Net -81 lines. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 14:11:26 +02:00
leonarski_f	102a2a7c81	PixelRefine: strip experimental env knobs (orientation, sweep, Lorentz, ML census) ... Remove the env-gated experiments that A/B'd to dead-ends or are no longer needed, returning PixelRefine to the clean factored Terms 1+2 plus the one validated keeper (r1_multiplier, default 6): - PR_ORIENT (per-image orientation refinement): R-free no-op (0.2618 vs 0.2625) - XtalOptimizer's orientation is already optimal. Removes ShoeboxResidual, OrientationRegularizer, PixelObs::weight, the refinement block and its fields. - PR_SWEEP (orientation + cell-scale sweep): R-free no-op, degraded high-res CC1/2 (per-image overfit). Removes SweepOrientationCell and its fields. - PR_LORENTZ (rotation Lorentz/zeta): hurt both directions (the factored partiality already subsumes it); was already reverted. - PR_MLCENSUS (multi-lattice census in AnalyzeIndexing): served its purpose (~3-5% of jet frames are multi-lattice; shelved). PR_RMULT (the validated Term-2 multiplier knob) is kept. Defaults unchanged: crystal 2 / jet / hybrid -R -r pixelrefine all reproduce. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 13:34:54 +02:00
leonarski_f	45ee8c2b40	PixelRefine: env-gated orientation + cell-scale sweep (PR_SWEEP) for R-free A/B ... R-free validation (full jet, 1.5 A) confirmed the r1_multiplier fix: PR x6 beats traditional (R-free 0.2625 vs 0.2802), the multiplier optimum is ~6 (x6==x9 on R-free; x9 only buys CC1/2 internal consistency), and per-image orientation *refinement* is a no-op (0.2618 vs 0.2625). Re-adds the reference-driven orientation + uniform cell-scale SWEEP behind PR_SWEEP (off by default) to test whether the cell-scale degree of freedom - which the gradient orientation refinement lacks, and which moves the high-res shells radially - helps R-free at 1.5 A on serial data. CCref is a near-no-op on the jet (as for the gradient path), but that does not certify R-free, so it is left for validation. NOTE: PR_RMULT / PR_ORIENT / PR_SWEEP remain temporary diagnostic env knobs; once the sweep R-free is in, bake r1_multiplier=6, drop the no-op paths, strip the knobs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 19:59:50 +02:00
leonarski_f	cfcd4c9e56	PixelRefine: R1 multiplier (fix tight-Term2 regression) + env-gated orientation refine ... Full-jet R-free showed the factored model regressed vs the traditional pipeline (R-free 0.34 vs 0.26, CCref 58 vs 76), traced to Term 2 using the raw measured profile width: it is too tight (~0.002 A^-1), so the template sits off the ~0.4 px centroid-floor scatter and underestimates the intensity (box-sum is immune). Fix, XDS-style (integrate over ~6 sigma): multiply the measured R1 by r1_multiplier (default 6) before use. On the jet at 1.3 A this recovers CCref 55.2 -> 59.3 (~ traditional's 60.5); crystal 2 -0.7. Tunable via env PR_RMULT for R-free. Also reinstates the pre-factored per-image orientation refinement (per-pixel ShoeboxResidual against the shoebox, regularised to the spot-centroid orientation, re-predict), behind env PR_ORIENT (off by default), to A/B its effect on serial-data R-free. On the jet CCref it is a no-op (XtalOptimizer orientation already at the optimum), but CCref is a weak R-free proxy here, so it is left for R-free validation. NOTE: PR_RMULT / PR_ORIENT are temporary diagnostic env knobs; the final multiplier value and the orientation-refine decision are to be fixed against R-free, then baked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 09:18:09 +02:00
leonarski_f	100fe7b7e7	PixelRefine: make factored Terms 1+2 the model, remove old wiring ... PixelRefine is now an intensity-only operation: geometry is fixed (refined upstream by XtalOptimizer) and the only objective is the factored per-reflection likelihood (FACTORED_MODEL.md Terms 1+2) - measured per-resolution profile width R1 plus one Fisher-weighted intensity/scaling residual per reflection, fitting the per-image scale G and B. Validated on crystal 2 (fixed_master.h5 as stills, 1.7 A): CC1/2 84-92%, CCref 77-92%, flat - reproduces the env-flag prototype and matches the rotation path from the stills path. Removed: - the per-pixel ShoeboxResidual loss and PixelResidual cost functor; - all in-PixelRefine geometry refinement (orientation/cell/beam/distance/R), the regularised-orientation LSQ, signal-weighting, and the global sweep; - Term 3 (per-spot recentring) - a confirmed no-op on both crystals; - the diagnostic scaffolding (covariance, centroid, adaptive_R1) and the PR_* env knobs + stderr dumps in IndexAndRefine; - the PredictImage/ChiSquaredImage renderers and the entire viewer PixelRefine window/table/params + worker bindings + shoebox overlay. The sweep box-integrator background median became mean (consistency) by virtue of removing the sweep. METHODS.md rewritten for the current model; findings recorded in FINDINGS-2026-06.md. Net -2200 lines. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-13 22:02:18 +02:00
leonarski_f	6f2033db00	PixelRefine: checkpoint before cleanup (factored model + all diagnostic levers) ... Snapshot of the messy state: factored likelihood Terms 1+2+3 behind PR_* env flags (PR_INTENSITY/PR_SHAPE/PR_RECENTER) alongside the old per-pixel ShoeboxResidual, plus diagnostic scaffolding (PR_R0/R1/COV/FIX_R0/FIX_R/ ADAPT_R1/CENTROID/RECENTER) and the FACTORED_MODEL.md spec. Next commit makes Terms 1+2 the model and strips all of this. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-13 21:34:33 +02:00
leonarski_f	c93d381dc8	Error model: harden the fit against pathological inputs (code review) ... Addresses code-review findings on RefineErrorModel: - Floor the 1/dev^2 bin weight relative to the data scale (1e-3 of the median bin dev^2), not an absolute 1e-30: a near-zero-scatter bin could otherwise acquire a runaway weight and hijack the global (a,b) fit. - Reject a near-collinear normal-equation system relatively (det > 1e-10*Ass*AII) instead of with an absolute threshold that an ill-conditioned fit can pass. - Reset the model to identity at entry so any early return leaves it inactive rather than keeping a stale a/b alongside a freshly-cleared mean map (which would make CorrectedSigma fall back to the per-observation I). - PixelRefine: correct the orient_prior comment - with the sweep on, the LSQ anchor is the swept orientation (intended), not the spot-centroid one. Verified unchanged on the lyso test set (ISa 1.1, CC1/2 90.3%). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-12 18:53:02 +02:00
leonarski_f	e6a50b45c7	Integration: mean background + global error model (trustworthy sigmas) ... Background estimate: use the mean of the local ring, not the median. For a right-skewed (Poisson) background the median sits below the mean, so subtracting it under-subtracts and biases every weak intensity positive; over multiplicity this becomes fake <I/sig> of a few in no-signal high-resolution shells. Fixed in both PixelRefine and BraggIntegrate2D (the classical route had the same bug). <I/sig> now tracks CC honestly and the true resolution limit is visible. Error model: fit a global a, b (XDS form sigma'^2 = a*sigma^2 + (b*I)^2) from the scatter of symmetry equivalents at the merge level (so both integrators benefit), and print it with ISa = 1/b in jfjoch_process. The (b*I)^2 term uses the reflection mean (not the per-observation I_i, which biases the weights and collapses CC); a,b come from a relative-weighted bin regression. Replaces the earlier per-resolution-shell variant, which was partly masking the background bias. METHODS.md: document both (Sections 6-7), integrator-agnostic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-12 18:39:32 +02:00
leonarski_f	db68c8dc38	PixelRefine: Results seem to be much better	2026-06-12 17:28:18 +02:00
leonarski_f	47dc19dd03	PixelRefine: Improvements to accept more reasonable count of reflections	2026-06-12 10:12:23 +02:00
leonarski_f	d31063ca3f	PixelRefine: Some improvements	2026-06-11 21:25:05 +02:00
leonarski_f	bd5fef7f61	PixelRefine: Simplify (remove Lorentz correction, remove background from azimuthal integration)	2026-06-10 18:37:46 +02:00
leonarski_f	efe882f4b6	jfjoch_viewer: Better display (to be tested) of pixel refine	2026-06-09 16:28:17 +02:00
leonarski_f	6c85aaba2b	BraggPrediction: Include X-ray bandwidth	2026-06-09 15:01:44 +02:00
leonarski_f	e4b66f9cd9	PIxelRefine: Another iteration	2026-06-09 08:04:27 +02:00
leonarski_f	e8a9b1840d	PixelRefine: Make it faster by doing one cell calculation per shoe-box	2026-06-08 22:45:22 +02:00
leonarski_f	05711a1077	jfjoch_viewer: Add pixel refinw and magnifier windows (to be tested)	2026-06-08 21:22:19 +02:00
leonarski_f	698a98be35	PixelRefine: Claude fixed my bugs	2026-06-08 20:06:53 +02:00
leonarski_f	155c53acd8	jfjoch_process: First pixel refine integration	2026-06-08 15:37:27 +02:00
leonarski_f	6f31159607	PixelRefine: Add bandwidth contribution	2026-06-08 13:59:01 +02:00
leonarski_f	6f6098d508	PixelRefine: Work in progress (Claude)	2026-06-08 13:44:39 +02:00
leonarski_f	66a48c4266	PixelRefine: Work in progress (Claude)	2026-06-08 12:43:10 +02:00
leonarski_f	9a991b6614	PixelRefine: Work in progress	2026-06-08 11:59:48 +02:00
leonarski_f	50f953bcc5	PixelRefine: Work in progress	2026-06-08 09:41:58 +02:00