5 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	87af0f419b	scale_merge: compute French-Wilson amplitudes at end of merge, write them alongside intensities ... Merged reflections carried only intensities; a naive sqrt(max(I,0)) turns every weak or negative measurement into a biased (or zero) amplitude. Add a French-Wilson step so the merged reflections themselves carry proper Bayesian amplitudes. New FrenchWilson.{h,cpp}: ApplyFrenchWilson() fills F and sigmaF on each MergedReflection with the posterior mean |F| given I and its sigma under the Wilson prior: - correct centric vs acentric prior (gemmi is_reflection_centric); - epsilon (symmetry-enhancement) multiplicity: the shell mean is <I/eps> and the prior mean for a reflection is eps * <I/eps>; - numerically stable log-shift integration of the posterior; - strong reflections (I > 4 sigma) short-circuit to sqrt(I) where the FW bias is negligible; - unusable I/sigma falls back to sqrt(max(I,0)). It runs as the last step of the merge routine (both MergeOnTheFly and RotationScaleMerge), so F/sigmaF are part of the merged result and downstream consumers (file writing AND, later, map calculation) all use the same amplitudes rather than each recomputing FW and risking divergence. The writers just emit the fields; extended additively: - MTZ: F (F) + SIGF (Q) columns; - mmCIF: _refln.F_meas_au / _refln.F_meas_sigma_au; - HKL text: F sigmaF appended (rfree flag kept in place for back-compat). Existing intensity columns are unchanged, so current readers keep working while phenix.refine (etc.) can now refine against the amplitudes. Verified on lyso (1.3 A): all 173 negative-intensity reflections get F>0, strong reflections have F/sqrt(I)=1.000, no NaN, and the mmCIF _refln loop is well-formed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 21:27:44 +02:00
leonarski_fandClaude Opus 4.8	4c95a289b6	scale_merge: harden R-free flag assignment (Bijvoet-safe, deterministic, stratified) ... The R-free flags were drawn with a per-reflection std::bernoulli_distribution off a shared std::mt19937(12345). Two problems: in anomalous mode I(+) and I(-) are separate merged rows, so a Bijvoet pair could be split across the work and free sets (biasing R-free); and the draw order depended on merge/call order, so it was not reproducible or parallel-safe. The resolution-shell grouping was cosmetic — it still drew an independent Bernoulli per row, so it neither tied mates together nor guaranteed a per-shell quota. New AssignRfreeFlags() (own file RfreeFlags.{h,cpp}) replaces both call sites (stills MergeOnTheFly, rotation RotationScaleMerge). It: - keys each reflection by its Friedel-merged (Laue) ASU index, so all symmetry- and Friedel-equivalent reflections (including a Bijvoet pair) share one flag and are never split; - selects the free set by a deterministic splitmix64 hash of that key (same idiom as the CC1/2 half-set split), so it is reproducible run-to-run and order-independent; - stratifies by resolution: exactly round(fraction * n) of the distinct reflections in each of 20 shells is flagged free. Verified on an anomalous lyso merge (29536 refl, 5% free): 13548 Bijvoet pairs, 0 split; free fraction 4.6-5.4% in every resolution shell. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 21:27:44 +02:00
leonarski_f	451310f43d	v1.0.0-rc.158 (#68) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * Analysis: The azimuthal-integration solid-angle correction now follows the incidence angle to the detector normal (`cos^3` of that angle) instead of `cos^3(2*theta)`, so it is correct for a tilted detector and matches PyFAI `solidAngleArray` and MAX IV azint (unchanged for an untilted detector). Crystal geometry refinement (`XtalOptimizer`) no longer silently ignores an imported PONI `rot3` (rotation about the beam): it is applied as a fixed rotation in the residual so refinement stays consistent with the rest of the pipeline. Polarization and azimuthal binning already honoured `rot3` through the full PONI rotation. * jfjoch_viewer: Open datasets on the WSL2/UNC filesystem (paths starting `\\`); write processing outputs next to the input file, with a Browse button and independent `_process.h5` / merged `.mtz`/`.cif` toggles; and show the determined space group in the merge-statistics window. * rugnux: Accept an absolute `-o` output prefix in offline processing. * Packaging: The self-contained Linux viewer `.tgz` now bundles cuFFT, so it runs without a system CUDA toolkit (`.deb`/`.rpm` are unchanged, distro-managed). * Docs: Bring the analysis references up to date with the code. `docs/CPU_DATA_ANALYSIS.md` now reflects the unified profile-fit Bragg integration engine, multi-lattice indexing, azimuthal phi binning, the radial parallax/bandwidth profile with sub-pixel centring, the rot3d capture-fraction handling and the automatic CC1/2 resolution cutoff, and drops the descriptions of features that were never implemented (French-Wilson amplitudes, the still excitation-error partiality model); `docs/RUGNUX.md` documents the new `--resolution-cutoff`/`--resolution-cc-target`/`--resolution-shells`, `--min-captured-fraction`, `--mosaicity`, `--reference-column`, the azimuthal correction toggles and the geometry-override options, and corrects the `-N` default. The outdated in-source design notes (ICE_RING_DETECTION, BRAGG_INTEGRATION_ENGINE, NEXTGEN_INTEGRATOR) are removed.Reviewed-on: #68 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-07-12 19:42:29 +02:00
leonarski_f	54c0100e8e	v1.0.0-rc.157 (#67) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Rebrand the offline data-processing subsystem as `rugnux` and consolidate all offline analysis into the single `rugnux` binary - `jfjoch_process` is now `rugnux`, the former `jfjoch_azint` is now `rugnux --azint-only`, and `jfjoch_scale` is now `rugnux --scale` (see the new docs/NAMING.md and docs/RUGNUX.md). Scaling and merging are on by default for rotation and stills (`--no-merge` disables them), replacing the previous opt-in `-M, --scale-merge`. * rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling. * rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics. * rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below <I/sigma> = 1. * rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group. * rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix. * jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation. * jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning. * jfjoch_broker: Drop uncollected placeholder rows (number = -1) from the scan_result REST endpoint. * jfjoch_broker: Fix the inverted per-image compression ratio reported by the Lite receiver (was compressed/uncompressed instead of uncompressed/compressed). * jfjoch_broker: Bragg integration adds a quantization-noise variance floor with a box-sum fallback, and treats the type-maximum marker as an invalid pixel for unsigned image types. * jfjoch_writer: Detect file-overwrite conflicts at start for back-channel transports, and reset the writer when end-of-collection finalisation fails. * jfjoch_viewer: Preview overlays follow the geometry (resolution/ROI arcs, true beam centre, predictions, coral secondary-lattice spots, legend), add save-as-JPEG, and fix an HTTP live-follow memory leak. * Frontend: Improved aesthetics and usability, and added in-browser pixel-mask and JUNGFRAU-pedestal visualisation. * CI: Name the Windows installer jfjoch-viewer-* instead of jfjoch-*.Reviewed-on: #67 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-07-11 07:19:11 +02:00
leonarski_f	d6389e12da	v1.0.0-rc.156 (#66) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output. * jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling. * Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool. * jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection. * jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata. * CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-07-03 19:18:56 +02:00