Replace the scaling partiality combo + "3D rotation scaling" checkbox with
one "Process as stills" checkbox in the indexing section (enabled only for
rotation datasets, unchecked by default). Unchecked on a rotation dataset
drives the full rotation path (rotation indexing at 60 first-pass images,
Rotation partiality, rot3d combine, scale-fulls); checked treats it as
stills (fixed partiality, per-frame indexing). The Analyze-dataset dialog
drops its rotation options (the panel is the single source) and buildConfig
reads rotation indexing from the experiment.
Fix: the worker's UpdateSpotFindingSettings copied indexing fields one by
one and was dropping RotationIndexing, so the mode never reached jobs.
Also: fold the panel accordions on start except Geometry and Unit cell, and
make the scaling resolution-limit a compact checkbox + field aligned with
the other checkboxes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the inline settings dock the single home for processing settings and
retire the separate Processing-settings window (and the dock<->window sync):
- "Analyze image" / "Analyze dataset" move to the top of the panel; the
MX/AzInt toggle decides the dataset-job kind, so the job dialog drops its
mode combo.
- The panel gains the most-used spot-finding (max spots, high-resolution,
min pixels), Bragg (Gaussian profile-fit, r1/r2/r3) and scaling (partiality,
"3D rotation scaling" = rot3d combine + scale-fulls, merge Friedel, refine
B, resolution limit) handles, a live indexing-algorithm description line,
and now owns the Bragg/Scaling settings. The now-unused window tab classes
are deleted.
- Complete the PixelRefine removal on the viewer side (the "Pixel refinement"
option + profile-multiplier widget), fixing the transient HEAD breakage.
New JFJochMergeStatsWindow: an analysis pop-up for a finished merge (hero
numbers over a per-resolution plot / per-shell table), auto-opened on
completion and reopenable from the processing-jobs dock.
Fixes: disable tear-off dock floating (a floated dock is a dead off-screen
window under WSLg, which has no window manager); version the saved dock
layout so a stale arrangement is discarded instead of restoring a broken one;
keep the Analyze-button icons; right-align and equal-width the stats table;
line-plot / table toggle icons (ToolbarIcons gains linePlot + table).
Add ScalingSettings::HighResolutionLimit_A(optional) so the panel can clear
the resolution limit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Regenerate jfjoch.png (256-wide, transparent), jfjoch.ico (multi-resolution
16/32/48/64/128/256, square) and jfjoch.icns (16-512) from the new
transparent-background jfjoch.svg, and add the SVG as the source.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On the LysozymeJet5 serial stills the default Gaussian profile-fit
integrator (ProfileIntegrate2D) + reference scaling matched or beat
whole-PixelRefine on every per-shell CC1/2 (overall 95.7% vs 91.9%), ISa
(1.6 vs 1.2) and R-meas (98.5% vs 175%), with CCref a tie -- so PixelRefine
has no remaining advantage. Reference-based per-image scaling is
integrator-agnostic (IndexAndRefine::ReferenceIntensities builds a
ScaleOnTheFly(experiment, reference) applied to any integrator's output),
so the reference-dataset feature (CCref + reference scaling) is kept.
Delete image_analysis/pixel_refinement/, GeomRefinementAlgorithmEnum::
PixelRefine and its gates, BraggIntegrationSettings::ProfileMultiplier
(PixelRefine-only; R1 is shared and kept), and the -r pixelrefine /
--profile-multiplier CLI. The inherited lessons (mean background, de-biased
variance, tight-profile-loses / centroid floor, R-refinement futile) are
folded into NEXTGEN_INTEGRATOR.md.
NOTE: this transiently breaks the viewer build -- the committed viewer
still references the removed enum and ProfileMultiplier. It is fixed in the
next commit (the viewer feature work), held separate while the viewer UI is
being tested.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In anomalous mode (Friedel mates kept separate) unique_reflections counts
I+ and I- of each acentric reflection as two unique members, but the
possible-reflection count used the merged ASU, so completeness approached
200%. Count each acentric possible reflection twice (centric once) when not
merging Friedel mates, matching how unique_reflections is counted; verified
overall completeness 99.7% on the HEWL rotation set.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rewrite NEXTGEN_INTEGRATOR.md from the early design draft into a full
record: the diagnosis (our intensities are accurate and unbiased per ANODE,
~2x noisier than XDS), the experiment ledger of what worked and the many
dead ends, the reusable tooling, and an honest assessment of the remaining
gap. Fix the stale ProfileIntegrate2D.h header (it is wired and the default
integrator, not a draft).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Profiling showed the joint Ceres solve took ~120 s versus ~3.5 s for the
alternating per-image scaling loop (~35x) for no quality gain (HEWL
anomalous 0.54x vs 0.53x), so it is not worth keeping. Drop
GlobalScale.{h,cpp}, the jfjoch_process --global-scale flag, and
ScalingSettings::GlobalScaling.
While here, in the same scaling/process area: fold scale_fulls into
ScalingSettings (alongside combine_3d) so the CLI and experiment carry it
uniformly, add per-substep [timing] logging to the scaling/merge post-pass
(including the serial MergeAll vs parallel ScaleAllImages split), and carry
structured MergeStatistics + ISa in ProcessResult.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A larger first-pass image set gives a more stable consensus lattice and so
better per-frame prediction. On HEWL crystal 2 (full 360 deg) this lifts the
anomalous S-peak height vs XDS from 0.57x to 0.61x and ISa from 10.5 to 11.0,
at negligible cost (the first pass is a small fraction of the run). ~100 is the
sweet spot; more (300) does not help further.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The rot3d combine emits fulls with partiality == 1 and image_scale_corr == 1,
so the fulls are only ever scaled as per-frame partials upstream - their
per-frame scale G is fit through the rocking-curve/partiality model
(G*partiality*B*lp*Itrue - Iobs) and so absorbs any model error. XDS/DIALS
instead scale the 3D-integrated fulls directly.
--scale-fulls inserts a second scaling pass on the combined fulls with the
Unity model (G*Itrue - I_full, no partiality term), between Combine3D and
MergeOnTheFly, reusing ScaleOnTheFly on a Unity-configured experiment copy. It
is a pure post-correction (updates the fulls' image_scale_corr 1 -> 1/G, no
re-combine).
HEWL crystal 2, anomalous S-peak height vs XDS: 0.53x -> 0.57x and ISa
9.4 -> 10.5 - improving precision and accuracy together (not the CC1/2-up /
anomalous-down trade-off of outlier rejection).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Refine all per-image scales, per-image mosaicities and the shared per-HKL
true intensities (plus the global wedge) together in one Ceres problem
(GlobalScale.{h,cpp}, GlobalScaleCeres), as an alternative to the existing
alternating merge-then-scale-each-image loop. The result feeds back into each
reflection's image_scale_corr exactly as ScaleOnTheFly does, so the rot3d
combine and MergeOnTheFly run unchanged and every metric stays comparable.
On HEWL crystal 2 the joint fit converges to the same place as the alternating
loop (anomalous S-peak vs XDS 0.53x -> 0.54x, ISa 9.4 both), confirming the
alternating path already reaches the joint optimum. Kept as a validated
alternative and the home for future global corrections.
A shared quadratic absorption surface in detector position was prototyped here
and dropped: it fit large non-physical coefficients (radially degenerate with
the per-HKL resolution structure), lowered the scaling-fit residual but raised
the error-model b (ISa 9.4 -> 7.3) and did not improve anomalous accuracy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Masked/error pixels carry the int type minimum and saturated the maximum,
but the lossy codec can nudge them inward by one (masked observed as
INT32_MIN+1 in the decompressed data). The integrators only checked the
exact extremes, so a shifted sentinel in a reflection's background ring was
treated as a valid pixel -> garbage background and intensity for any
reflection whose box clips a module gap. Reject the +/-1 band too (real
calibrated counts never approach the type extremes). Neutral on the
well-centred lyso test set; a correctness fix for gap-clipping reflections.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
One command to score a jfjoch anomalous merge against XDS on physical
accuracy: feed the -A merge through SHELXC + ANODE (run under qemu-user,
since this WSL2 kernel's vsyscall=none breaks the static SHELX binaries)
and report <d"/sig> per shell and the averaged anomalous peak height at
the model S/Cl sites vs XDS. Unlike R-meas/CC1/2 (precision) it catches
accuracy regressions - e.g. outlier rejection raises CC1/2 but lowers
the anomalous peaks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Gaussian profile-fit (ProfileIntegrate2D) recovers more accurate
intensities than the classical box-sum. On the HEWL rotation set its
ANODE anomalous peak heights at the S/Cl sites rise from 0.46x to 0.53x
of XDS (Met-S 6.0->7.1 sigma, Cys-S 5.6->6.1) and ISa 8.4->9.4, with
R-meas unchanged. Box-sum stays available via --integrator boxsum.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reference dataset (a): LoadReferenceMtz adds column selection + cell/SG/resolution +
a data-vs-reference consistency check; jfjoch_process/jfjoch_scale gain
--reference-column; the viewer gets a Reference section in the MX settings dock
(worker-owned, independent of the loaded dataset) that flows into reprocessing jobs.
3D combine (-P rot3d): Combine3D weight-sums a reflection's per-frame partials into one
counting-limited full before merging (orthogonal ScalingSettings::combine_3d flag, not a
partiality model), with a de-biased Poisson variance. Crystal 2: ISa 1.7->8.4, R-meas
~67%->18.9%, intensities unchanged (CCref held).
Quality metrics (b): R-meas (Diederichs-Karplus) + redundancy columns in MergeStats; ISa
logged. jfjoch fulls 18.9% vs XDS 4.5% (same ASU/run).
Profile-fit integrator (experimental): ProfileIntegrate2D (--integrator gaussian|empirical)
is a reference-free, rot3d-compatible profile-fit extraction (the decomposed PixelRefine
intensity step). Gaussian: R-meas 18.9->14.6%, ISa ->9.5. Anisotropy/per-region add nothing
(the discriminating info is in the discarded rocking direction). See NEXTGEN_INTEGRATOR.md.
--dump-observations exports the unmerged fulls for XDS comparison.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
leonarski_f