8 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	ef103d2ea2	rugnux: raise default min-captured-fraction to 0.7 (recover weak F-cubic space groups) ... The de-novo space-group search runs its point-group correlation only on reflections with I/sigma >= 3, but the Diederichs error model floors merged I/sigma at the asymptotic ISa = 1/b. On a weakly-diffracting crystal a few edge-of-sweep truncated fulls (rocking curve captured far below 1) inflate the error-model b term, dropping ISa below 3 so no reflection clears the gate: every operator gets zero pairs and the point group collapses to P1 (or a low subgroup). The chi^2 self-consistency guard is likewise corrupted by those zingers. Raising the rotation default of --min-captured-fraction from 0.5 to 0.7 drops those partiality-extrapolated fulls, lifting ISa back above the gate and cleaning the intensity statistics, so the search recovers the true symmetry. On the F-cubic Benas crystals: Benas_3 P4 -> F432 (CC1/2 78% -> 98%) and Benas_7 P1 -> P6122 (CC1/2 40% -> 99%). Benas_7's data supports hexagonal P6122, not cubic F432 - the cubic 3-fold/4-folds fail the merge self-consistency test - matching XDS. Full /data/rotation_test battery: every other crystal keeps its exact space group; the only cost is a little multiplicity from the extra dropped edge-fulls on a few weak/ narrow-sweep crystals (worst completeness -7% on EP_cs_01-24, whose CC1/2 nonetheless rises), while several crystals' CC1/2 improve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-10 22:11:30 +02:00
leonarski_fandClaude Opus 4.8	f0799f780b	rugnux: drop edge-of-sweep truncated fulls + honest R-meas over accepted obs ... From a fresh decomposition of the low-resolution-shell R-meas gap vs XDS on the reference lysozyme rotation crystal (7.6% vs 2.9%), two rot3d scale/merge changes: - New --min-captured-fraction (default 0.5 for rotation, off otherwise): drop a combined full whose rocking curve was captured below this fraction. These edge-of-sweep, partiality-extrapolated fulls (sweep start/end) are biased and inflate the low-res R-meas. Dropping them lowers low-shell R-meas 7.6->5.5% and *improves* external accuracy (CC vs the F-model and vs an independent dataset, and the ANODE anomalous S-peak 0.93->0.94x XDS) - unlike --min-partiality, which lowers R-meas cosmetically while degrading accuracy. Threaded through ScalingSettings and both the GPU and CPU combine. - R-meas reporting fix: the R_meas re-walk counted observations the merge had rejected as outliers (XDS excludes its misfits before its R-factor). The accumulate pass now flags each rejected full and the R_meas pass skips it, in both the GPU and CPU paths. Merged intensities / ISa / CC1/2 unchanged (verified byte-identical external accuracy); low-shell R-meas 7.6->6.5%. No-op when --reject-outliers 0. Combined default rotation low-shell R-meas 7.6->5.1%, ISa 17.2->17.5, external accuracy equal-or-better than baseline and XDS. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-10 20:13:06 +02:00
leonarski_fandClaude Opus 4.8	a4cc908ff1	rugnux: run the L-test on rhombohedral data + write twinning stats to CIF ... The Padilla-Yeates L-test paired each reflection with a neighbour two steps along an axis ((h+2,k,l) etc.). A step of 2 preserves the reflection condition of P/I/C/F/A/B lattices but violates R-centring (-h+k+l = 0 mod 3, since 2 != 0 mod 3), so every neighbour was systematically absent -> zero pairs, and the L-test silently did not run on ANY rhombohedral crystal (twinning there was left to the weaker second-moment test alone). Add diagonal (1,1,0)/(1,1,3) fallback steps that preserve the mod-3 condition in both obverse and reverse settings, tried only when the axis steps find no present partner (P/I/C/F behaviour unchanged). On the twinned R3 insulin the L-test now runs: 25k pairs, <L^2> = 0.224 -> twinning detected. Also write the twinning indicators to the output mmCIF next to ISa: _reflns.pdbx_L_test_mean_abs_L / _L_test_mean_L_squared / _second_moment_I, as free-text pdbx values, guarded on pair/reflection count. Threaded the TwinningAnalysisResult into WriteReflections (both the full-pipeline and --scale callers). Note: on rhombohedral lattices the L-test reads ~0.03 lower than on P/I/C/F (its allowed step is slightly more local), so it separates the insulin twin pair only weakly (twinned 0.398 vs untwinned 0.427) and the second moment remains the cleaner discriminator there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-09 21:59:37 +02:00
leonarski_fandClaude Opus 4.8	4f3153d6c7	rugnux: automatic CC1/2-logistic high-resolution cutoff ... Merge/report/write to the CC1/2 fall-off instead of the detector edge. After the final merge, fit a logistic to a fine-binned CC1/2 profile and cut one shell past cc_target (default 0.30) - generous, DIALS-style. Only the written MTZ/CIF and the reported shell table are trimmed; scaling, the 3D combine and the error model still run over the full range, and the per-image _process.h5 keeps full data for re-scaling. The de-novo P1 merge that feeds the space-group search is never cut (gated on the final merge). - ScalingSettings: resolution_cutoff {Off,CCHalfLogistic}, resolution_cc_target, report_shell_count (report shell count is now configurable). - New ResolutionCutoff.{h,cpp}: ComputeCCHalfLogisticCutoff (fine-bin CC1/2 + grid/refine logistic fit + one-shell extension + degeneracy/low-res guards). - Wired into RotationScaleMerge (rotation) and MergeOnTheFly/Rugnux (stills), plus the rugnux --scale path. - CLI: --resolution-cutoff cc-logistic|off, --resolution-cc-target, --resolution-shells. --scaling-high-resolution stays the manual override (wins and disables the auto-cut). Validated on the 24-crystal battery: no crashes (incl. Benas/cytC_10/Thau_9); --resolution-cutoff off is bit-identical to baseline; cutoffs reproduce the design table within ~0.05 A (lysoC 1.30, Ins_I 1.43, cytC_2 1.68, EcwtCQ244-E3 1.54); good-to-edge datasets (lyso_ref, EP_cs_02-10) correctly keep the full range. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 17:34:49 +02:00
leonarski_fandClaude Fable 5	dc8a80f3e0	viewer: fix HTTP live-follow OOM via datasetLoaded backpressure + shared pixel mask ... The viewer could grow to ~100 GB RAM when live-following an HTTP broker. The rc.153 images_in_flight backpressure only throttled imageLoaded; the heavy per-frame payload rides datasetLoaded, fanned out over ~10 queued cross-thread connections with no cap. In HTTPSyncDataset follow mode (entered when an operator clicks an image while following live) RefreshDatasetOnly_i emits a fresh full dataset every autoload tick with no imageLoaded, so the gate never engaged and the queued events - each pinning a full JFJochReaderDataset (full-detector PixelMask + per-image plots) - accumulated without bound. Backpressure datasetLoaded the same way as imageLoaded: a datasets_in_flight counter (cap 2), all emits routed through EmitDatasetLoaded_i, and AutoLoadTimerExpired gated on it (covers HTTPSyncDataset). The window routes the worker's datasetLoaded through a single OnDatasetReady sink that fans out synchronously via datasetReady and acks with datasetConsumed. Under load stale datasets are dropped; the next tick sends the latest. Share the pixel mask instead of deep-copying it: JFJochReaderDataset::pixel_mask is now shared_ptr<const PixelMask>, so per-frame dataset copies share the ~72 MB mask. UpdateUserMask does copy-on-write; JFJochHttpReader caches the mask by arm_date so a live refresh reuses one shared mask per acquisition. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 10:58:10 +02:00
leonarski_fandClaude Fable 5	51bfdd2b35	Fix acquisition error-handling and rugnux CLI issues from branch review ... Broker acquisition failure paths: - Report a truncated writer output / missing packets as errors ahead of the queue-full warning, so an incomplete dataset is no longer masked as a "reduce frame rate" warning. - Route PCIe/FPGA hardware faults to the Error state: make PCIeDeviceException a JFJochCriticalException, and stop JFJochServices::Stop() from slicing the caught exception (capture/rethrow via exception_ptr) so the critical type survives to the state machine. - Stop the (possibly partially-armed) detector, not just the receiver, when a detector arm fails, so the next start does not see it BUSY. - Return structured Error_message JSON for bind_json operation failures (e.g. a failed synchronous /start), matching the no-arg endpoints. rugnux CLI: - Default -N to all hardware threads, resolved once after parsing so full analysis, --azint-only and --scale behave the same. - Parse numeric option arguments strictly (reject non-numeric input and trailing garbage) instead of atof/stod, which silently yielded 0 or aborted the process; require --wavelength > 0. - Emit -R attached (-R100) in the reproduced command line so it re-parses. - Compare the reference MTZ against the cell that actually drives the merge in the --scale path. - Correct the --scaling-high-resolution help text and de-duplicate the offline-output experiment setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 15:48:01 +02:00
leonarski_fandClaude Fable 5	01e4f81e44	Consolidate offline tools into rugnux; merge on by default ... Fold jfjoch_azint and rugnux_scale into the single rugnux CLI, and flip scaling/merging from opt-in to on-by-default. - rugnux --azint-only replaces jfjoch_azint (reuses ProcessMode:: AzimuthalIntegration; adds the correction toggles and geometry overrides the old tool carried). - rugnux --scale replaces rugnux_scale (re-scale/merge stored reflections, same simpler scale/merge as before: no space-group search or rot3d defaults). Its workflow is folded into the CLI verbatim. - Merging is now on by default for rotation and stills; --no-merge disables it, replacing -M/--scale-merge. Also fix the batch ReadReflections reader so it works on rugnux's own NXmxIntegrated _process.h5: read per-image reflections/mosaicity/lattice from the master (global index) first, falling back to the source-data locator for legacy/VDS datasets. Without this, both --scale and the former rugnux_scale read zero reflections from an integrated snapshot. RugnuxCommandLine now emits `rugnux --azint-only` and `--no-merge`; docs and tests updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 14:09:15 +02:00
leonarski_fandClaude Opus 4.8	1caa57744a	Rebrand offline processing as rugnux (jfjoch_process -> rugnux) ... Split the naming: rugnux = data-processing subsystem, Jungfraujoch = streaming/acquisition. Executables jfjoch_process -> rugnux (source tools/rugnux_cli.cpp) and jfjoch_scale -> rugnux_scale; the processing library process/ -> rugnux/ with class/target JFJochProcess -> Rugnux (JFJochProcessObserver -> RugnuxObserver, JFJochProcessCommandLine -> RugnuxCommandLine). Doc JFJOCH_PROCESS.md -> RUGNUX.md, reconciled with the live usage message (drop dead -P/--partiality, -w/--wedge; --process-as-stills -> --force-still; add the real rot3d scaling knobs). New docs/NAMING.md explains both names, with pronunciation and a note on Romansh. rugnux now scales and merges rotation data automatically (implicit -M); stills still require an explicit -M. jfjoch_viewer and its classes keep their names (rename deferred); only their references to the renamed library are updated. The _process.h5 output suffix and ProcessConfig/Mode/Result are kept. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-07 22:46:58 +02:00