5 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	fbebc0d56e	rugnux: run decay + absorption surfaces on the GPU and enable by default ... The decay/absorption correction surfaces previously forced the CPU 3D combine (absorption needs each full's predicted detector position, which the GPU SoA did not carry), so they could not be default-on without losing the fast GPU path. Put them on the GPU: - carry px/py (peak partial) through the GPU combine and download them with the fulls (GetFullsPxPy); - add SetFullsCorr to re-upload the host-corrected corr to the resident fulls before the merge. The surfaces themselves stay as cheap host fits on the downloaded fulls; only px/py download + corr upload are added (~two O(n_fulls) transfers). Measured +2% wall-time on lyso_ref (9.7 -> 9.9 s), and the GPU-combine and CPU-combine (--dump-observations) paths give identical results. Enable by default via a single master toggle ScalingSettings:: CorrectionSurfaces (default true; decoupled from the stills-only -B, which is rejected for rotation again). Both surfaces are cross-validated, so they no-op where their systematic is absent - safe to leave on. Off-switch wired through the rugnux CLI (--no-scaling-corrections; --absorption removed) and the viewer (a "Correction surfaces" checkbox). Documented in docs/RUGNUX.md. Battery (24 rotation crystals), default-on vs --no-scaling-corrections: EcwtCQ066S +2.4 ISa, lyso_ref +0.5 (and R-free 0.194 -> 0.185, better than XDS); every other crystal within run-to-run noise; corrections skip (CV / dB-floor) wherever there is no signal. lyso_ref anomalous S/Cl peaks are already ~45% above XDS's - the ISa gap is metric convention, not data. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 10:56:39 +02:00
leonarski_fandClaude Opus 4.8	bab52a8077	rugnux: optional decay + absorption correction surfaces for rotation scaling ... After scale-fulls, RotationScaleMerge can now fit two optional multiplicative corrections on the combined fulls (both off by default, rotation only): - -B (decay): a global Debye-Waller relative-B rate for radiation damage, ln(Iref/Iobs) = 2 (dB/dn)(n - nbar) s^2, folded into the per-full scale. Engages only above a physical floor (total relative-B > 2 A^2), so it is a no-op on low-dose data. (-B was previously rejected for rotation.) - --absorption[=iter]: a smooth factor over the diffracted-beam direction in the goniometer frame (predicted detector position de-rotated by the spindle, so a crystal-frame direction is sampled at many angles). Wires the existing ScalingSettings::absorption_iter scaffold. Negligible at hard X-rays; matters at low energy. On lyso_ref it is ISa-neutral (+0.5) but drops R-free 0.219 -> 0.212 against the deposited model: a smooth absorption error cancels among symmetry mates (small ISa effect) yet still biases the intensities (real R-free effect). Both are CROSS-VALIDATED: the surface is fit on even frames and applied only if it improves the held-out odd-frame equivalent agreement by a clear margin (and vice versa), so a fit to noise where the systematic is absent does not generalize and is discarded - the correction never adds scatter. Battery: EcwtCQ066S +2.4 ISa, lyso_ref R-free win, others neutral. Also fixes a latent bug this exposed: the GPU->host corr download before the CPU 3D-combine was gated on --dump-observations only, so any other forced-CPU-combine path (the new -B/--absorption) read a stale host corr and mis-scaled every full. The download condition now matches the CPU-combine condition. Predicted detector position (px, py) is carried on the combined fulls for the absorption surface (CPU combine only; unused by the GPU SoA path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 10:14:00 +02:00
leonarski_fandClaude Opus 4.8	dee679380b	rugnux: exclude merge-rejected outliers from the error-model fit ... The rotation error-model (a,b) fit built its sample pool from all usable fulls, while the merge separately dropped symmetry outliers (> Nsigma from the per-reflection median). So the fitted sigmas were calibrated against a dirtier pool than the reflections that actually entered the merge. Refactor the intensity-binned median fit into a `fit_ab` lambda and re-run it once on the misfit-free pool (samples with dev2 <= reject_nsigma^2 * model variance), matching the merge's own rejection. It operates on the shared `samples` array, so the CPU and GPU error-model paths stay bit-identical. Neutral on lyso_ref (the median-binned fit is already robust to the few outliers) - a correctness fix that matters more on dirtier data. Gated on reject_outliers (default 6 sigma for rot3d). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 10:04:43 +02:00
leonarski_f	54c0100e8e	v1.0.0-rc.157 (#67) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Rebrand the offline data-processing subsystem as `rugnux` and consolidate all offline analysis into the single `rugnux` binary - `jfjoch_process` is now `rugnux`, the former `jfjoch_azint` is now `rugnux --azint-only`, and `jfjoch_scale` is now `rugnux --scale` (see the new docs/NAMING.md and docs/RUGNUX.md). Scaling and merging are on by default for rotation and stills (`--no-merge` disables them), replacing the previous opt-in `-M, --scale-merge`. * rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling. * rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics. * rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below <I/sigma> = 1. * rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group. * rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix. * jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation. * jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning. * jfjoch_broker: Drop uncollected placeholder rows (number = -1) from the scan_result REST endpoint. * jfjoch_broker: Fix the inverted per-image compression ratio reported by the Lite receiver (was compressed/uncompressed instead of uncompressed/compressed). * jfjoch_broker: Bragg integration adds a quantization-noise variance floor with a box-sum fallback, and treats the type-maximum marker as an invalid pixel for unsigned image types. * jfjoch_writer: Detect file-overwrite conflicts at start for back-channel transports, and reset the writer when end-of-collection finalisation fails. * jfjoch_viewer: Preview overlays follow the geometry (resolution/ROI arcs, true beam centre, predictions, coral secondary-lattice spots, legend), add save-as-JPEG, and fix an HTTP live-follow memory leak. * Frontend: Improved aesthetics and usability, and added in-browser pixel-mask and JUNGFRAU-pedestal visualisation. * CI: Name the Windows installer jfjoch-viewer-* instead of jfjoch-*.Reviewed-on: #67 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-07-11 07:19:11 +02:00
leonarski_f	d6389e12da	v1.0.0-rc.156 (#66) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output. * jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling. * Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool. * jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection. * jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata. * CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-07-03 19:18:56 +02:00