1147 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	eae7ea90dc	rugnux: always write both MTZ and CIF; drop --scaling-output ... The merged reflections were written in one of mtz/cif/txt selected by --scaling-output. Write both an MTZ and an mmCIF unconditionally instead - each has its uses downstream (MTZ for the CCP4/phenix tools, mmCIF for deposition) - and remove the format selector, the plain-text .hkl writer, and the now-unused IntensityFormat enum / ScalingSettings::FileFormat plumbing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 10:45:08 +02:00
leonarski_fandClaude Opus 4.8	ce12d69751	rugnux: write standard CCP4 anomalous MTZ (IMEAN + I(+)/I(-)) ... In anomalous mode the merge keeps the two Friedel mates as separate rows, which WriteMtzReflections emitted verbatim - two IMEAN rows per Bijvoet pair that downstream tools had to re-collapse. Pair the mates into one row per reflection with the standard CCP4 anomalous layout: IMEAN + I(+)/I(-) and the matching F/F(+)/F(-) amplitudes. A single HKLKeyGenerator(merge_friedel=false) yields both the shared ASU group key and which mate a row is (.plus); IMEAN/F are the inverse-variance Friedel mean, centrics/unpaired keep one hand with the other missing. aimless/mtz2sca/ANODE now read the file directly. The non-anomalous MTZ path is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 10:37:56 +02:00
leonarski_fandClaude Opus 4.8	ce15900471	changelog: add the 1.0.0-rc.160 section ... VERSION was bumped to rc.160 but the new items sat under the rc.159 heading, which released (origin/main) does not contain. Add a proper rc.160 section, move the three misfiled items (supercell fix, -S symbol, FFTW-on-GPU error) into it, and document the rest of the branch (stills geometry refinement, -r flex, stills partiality, r=6 box default, single-pass self-referenced scaling, reference-MTZ-never-a-scale-anchor, E^2 second-lattice cap, --scale on _process.h5 fix, new spot options, CBF/TIFF writer removal, negative total_flux, viewer, packaging). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 22:20:14 +02:00
leonarski_fandClaude Opus 4.8	0e4f91b219	rugnux: validate offline spot-finding settings and bound numeric options ... The offline rugnux path never ran CheckDataProcessingSettings (only the online broker/receiver paths did), so the new --min-pix-per-spot and --spot-low-resolution knobs were unbounded: --min-pix-per-spot 0 silently disabled the per-spot filter, and a low-resolution limit finer than the high-resolution limit made spot finding reject every pixel with no diagnostic. Add a parse_number_arg<T> helper (integral or floating, with optional inclusive bounds) that rejects non-numeric input, trailing garbage, and out-of-range values, replacing the raw atoi/atof in the spot-finding options; parse_double_arg/parse_float_arg become thin wrappers over it. After assembling the settings, validate them with the same CheckDataProcessingSettings the online receivers use, which also enforces the cross-field low>=high-resolution constraint. Also scrubs a residual sample-name comment (the r=6 box-default note) in this file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 22:19:59 +02:00
leonarski_fandClaude Opus 4.8	8fea7f195a	docs: stop advertising the removed CBF/TIFF writers ... The CBF/TIFF writer removal left two docs describing them as available output: JFJOCH_WRITER.md's "Other formats" section and an aside in HDF5.md. A user following them would issue a now-rejected request. State that only NXmx HDF5 is written and that the CBF/TIFF enum values are retained for wire back-compatibility but rejected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 22:19:46 +02:00
leonarski_fandClaude Opus 4.8	d393ec814c	Remove residual sample identities missed by the earlier scrub ... The rc.160 sample-identity cleanup left protein names, internal dataset codes, and a measured cell angle in several source comments and one test. Most sensitive: a measured monoclinic beta tied to an internal code in Rugnux.cpp and IndexAndRefine.cpp. Rewrite each to describe the crystallographic situation only (space group / metric relation), keeping the technical reasoning intact. Comment- and string-only; no behaviour change. (rugnux_cli.cpp's remaining name is scrubbed in the following commit, alongside its other change to that file.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 22:18:32 +02:00
leonarski_f	68272a96e1	VERSION: 1.0.0-rc.160	2026-07-15 21:41:51 +02:00
leonarski_fandClaude Opus 4.8	b450140816	rugnux: warn when the cell/space group has an indexing ambiguity ... At the twinning-analysis print, detect alternative-indexing (twin-law) operators for the final cell + space group via ReindexAmbiguityOperators(). When any exist and no reference was supplied, emit a Warning (also written into the merge stats) that serial-stills crystals are indexed in a random hand and the ambiguity can only be broken against a reference (-z / --model). For an obvious merohedral case (P3/P4/P6...) users expect this, but a PSEUDO-merohedral metric is easy to miss -- e.g. NmHR / 7O8F, where C2 beta=131.8 is pseudo-F-orthorhombic and gemmi reports the reindex operator h-l,-k,-l. Advisory only; no change to processing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 20:38:31 +02:00
leonarski_fandClaude Opus 4.8	3e3545770f	indexing: check candidate angles against the reference cell in Refine() ... When a reference cell is supplied, PostIndexingRefinement::Refine() vetted candidates on sorted edge lengths only; a cell with the right edges but a wrong angle (a pseudo-symmetric near-metric, e.g. a monoclinic beta refined to the wrong value) passed. Add a sorted-angle check against the reference, folding each angle to its acute complement min(x,180-x) so the obtuse/acute setting choice is irrelevant (tolerance 10 deg). Reference-cell path only (no effect de novo, where reference_unit_cell is null): the /data/rotation_test battery is byte-identical (22/25), and NmHR with the correct C2 cell is unchanged (9.1%, beta~131.7). A latent guard against wrong-angle cells slipping through on -C / reference-MTZ runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 19:47:29 +02:00
leonarski_fandClaude Opus 4.8	0f07b7a499	rotation: widen candidate-refine angle bound to [30,150] deg ... RotationIndexer's per-candidate XtalOptimizer relied on the struct-default monoclinic angle bound [60,120], the same latent clamp fixed for the stills path: a rotation crystal with beta>120 (or <60) would have its cell clamped to the boundary. Set [30,150] explicitly, matching the indexers. No change on the /data/rotation_test battery (22/25 SG match, byte-identical) -- no battery crystal has beta outside [60,120]; this is a latent-correctness fix for future high-beta monoclinic rotation data. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 19:47:17 +02:00
leonarski_fandClaude Opus 4.8	643bda3dd3	stills: widen per-frame cell-refine angle bound to [30,150] deg ... The stills per-frame XtalOptimizer in IndexAndRefine used the struct-default monoclinic angle bound [60,120] deg, which clamps a beta outside that window (e.g. NmHR / 7O8F, C2 beta=131.78) to the boundary and corrupts the per-frame cell. Both indexers already use [30,150] (FFBIDXIndexer, FFT settings); this aligns the per-frame refine with them. Rotation path is unaffected (refine_unit_cell is false there). Verified on NmHR serial stills: FFBIDX with the correct C2 cell went from 0.01% to 9.10% indexed, recovering the true mC cell (beta~131.7) instead of a clamped beta=120; 60k-image merge reaches 2.61 A at 99.8% completeness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 19:13:41 +02:00
leonarski_fandClaude Opus 4.8	7b2cfb20f8	changelog: document the supercell fix, -S symbol, and FFTW-on-GPU error ... Also scrub two sample names from existing rc.159 entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 17:19:43 +02:00
leonarski_fandClaude Opus 4.8	d8e909a393	indexing: clear error when the requested indexer is unavailable on this host ... IndexerThreadPool built its per-worker indexers from the raw requested algorithm but dispatched the RESOLVED one. Rotation indexing always resolves to the GPU FFT indexer when a GPU is present, so requesting FFTW (the CPU indexer) on a GPU node created only the CPU indexer and then failed at dispatch with an opaque "no indexer available for the resolved indexing algorithm" deep inside a worker. Validate servability at the pool entry (in the caller's context) and throw an actionable message instead: FFTW is not available on a GPU node (use FFT/Auto, or a CPU-only node); FFT/FFBIDX require a GPU. Stills FFTW on a GPU node, which the algorithm resolution still honours, keeps working. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 17:19:43 +02:00
leonarski_fandClaude Opus 4.8	abbee2d4dc	Remove sample identities from the repository; document the rule ... Datasets may be confidential; sample names and measured unit cells committed to the repo can leak outside the group working on them. Scrub existing occurrences and add a "No sample identities in the repository" section to CLAUDE.md (forbidden: sample/dataset names, internal codes, measured cells tied to a sample; fine: space group / lattice / twinning descriptors). - Comments: replace internal dataset codes and protein names with the crystallographic situation they illustrate (centred vs pseudo-symmetric, holohedral, cubic, F-cubic/hexagonal, ...). - Docs: same, in the analysis/writer/stream references and example configs. - Tests: rename sample-named identifiers, TEST_CASE names, file prefixes and asserted labels to neutral crystallographic names (e.g. tetragonal_uc); behaviour unchanged. Reduce the CrystFEL reference PDB to a bare CRYST1 cell file (cell.pdb) and rename the reference data file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 17:19:43 +02:00
leonarski_fandClaude Opus 4.8	0f3ccda779	rugnux: fix de-novo rotation indexing adopting a spurious axis-multiple supercell ... Two-pass rotation indexing scored each first-pass scheme by how many validation frames it indexes, but a spurious axis multiple (2x/3x...) indexes every frame its true sub-cell does, so the count saturates - both schemes reach the same frame total - and the tie fell to whichever scheme ran first. When that first scheme's full-rotation FFT resolves a true axis only as a 2x/3x harmonic of its length (the fundamental can sit far below the harmonic in the full-360 cloud), it commits a multiplied cell and the dataset collapses to P1, even though the other scheme already found the true cell. Add an integer-supercell cross-scheme tie-break: when two schemes tie on frames but their cell volumes differ by a near-integer factor >=2, the larger is the spurious supercell and the smaller true cell is adopted, independent of scheme order. The near-integer test distinguishes a real axis multiplication from a centering coincidence (a rhombohedral H cell vs its C2 sub-cell is 1.5x and is left alone). Validated de-novo across the rotation regression battery: the affected dataset now indexes its correct space group, the override triggers only where an axis was multiplied, and every other dataset is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 16:51:57 +02:00
leonarski_fandClaude Opus 4.8	bd094e585f	rugnux: accept a space-group symbol for -S, not just a number ... -S now takes either a number ("92") or a Hermann-Mauguin symbol ("P43212"), resolved via gemmi find_spacegroup_by_name. Previously a non-numeric argument was atoi'd to 0 and tripped the early space-group validation, aborting the run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 15:23:31 +02:00
leonarski_fandClaude Opus 4.8	ebe7e7ec5e	jfjoch_viewer: retry deferred initial fit on resize (grid-scan zoom) ... fitToViewShorterSideOnce() skips the initial fit-to-view while the viewport has no real size yet (before the widget is laid out/shown) and relies on a retry that was never wired: resizeEvent only reset the scene rect, and there is no showEvent. When the first Redraw() lands before layout settles the view stays at 1:1, so a small grid-scan plot renders tiny ("zoomed out, not taking the full picture"); the layout-timing race is why it happened only sometimes. resizeEvent now retries fitToViewShorterSideOnce() while the initial fit is still pending (guarded by !initial_fit_done_, so a later user resize never overrides a manual zoom). It lives in the shared base class, so all image views get the same robustness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 11:28:09 +02:00
leonarski_fandClaude Opus 4.8	5d1ddccee8	viewer: expose stills processing settings; rename geom refinement multi -> flex ... Wire the recently-added stills knobs into jfjoch_viewer - they were reachable only from the rugnux CLI, and the viewer had silently diverged from the CLI once --refine-geometry became default-on for stills-with-cell. - Stills geometry-refinement two-pass (--refine-geometry): a checkbox + frame count in the reprocessing-job dialog, offered (and defaulted on, matching the CLI) only for a stills-with-cell run; wired into ProcessConfig and the "Copy command" generator (joined =N, or =off to reproduce opting out of the CLI default-on). - Stills partiality (--still-partiality) checkbox in Bragg integration and the partiality-uncertainty merge term (--partiality-uncertainty) in Scaling, both mirrored in the command-line generator. Rename the "best per-image refinement" mode from "multi" to "flex" everywhere: GeomRefinementAlgorithmEnum::Flex, CLI -r flex (with "multi" kept as a hidden back-compat alias), OpenAPI enum "Flex" + regenerated C++/TS clients, viewer combo, frontend dropdown, and the stored HDF5 string. "multi" collides with CrystFEL's multi-lattice sense; "flex" = let the pipeline decide. The API now carries a dedicated Flex value instead of masquerading as BeamCenter. Also fix a pre-existing bug in Convert(IndexingSettings): OrientationOnly was missing from the outbound switch and serialized as an invalid API value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 11:28:09 +02:00
leonarski_fandClaude Opus 4.8	4bdf9309dd	writer: remove CBF and TIFF writers; only HDF5 is written ... Delete CBFWriter and the TIFF write path from jfjoch_writer so the writer only produces the NXmx HDF5 formats. Drop CBF/TIFF from the internal FileWriterFormat enum and reject them in FileWriterSettings. The CBF/TIFF values are kept (marked deprecated) in the file_writer_format OpenAPI enum for back compatibility: incoming requests using them are now rejected in OpenAPIConvert with a clear "no longer supported" error, and a stale CBF/TIFF value on the CBOR wire decodes to unset rather than a removed enumerator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 10:11:39 +02:00
leonarski_fandClaude Opus 4.8	135765e5b5	rugnux: default --refine-geometry on for stills-with-cell; accept F/FP reference columns ... Two stills usability fixes. 1. --refine-geometry now defaults ON for stills whenever a reference cell is available (-C or a reference MTZ) - exactly the case where the geometry bundle-adjust can act (it anchors on a known cell) and where it lifts weak/ sparse-stills indexing (OCP +42%, KR2 +108% indexed in the target study). It stays a no-op for rotation (own two-pass) and de-novo stills (no cell yet), so auto-enabling it only where it does something avoids spurious "skipping" warnings. --refine-geometry=off opts out; explicit --refine-geometry[=N] forces it. 2. The reference-MTZ auto column selection now falls back to a plain amplitude column (FP / FOBS / F / FC, squared to an intensity) after F-model and the intensity (J) columns. A deposition that carries only structure-factor amplitudes (e.g. KR2's 8cl8, whose only usable column is FP) now seeds CCref without needing an explicit --reference-column FP. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 07:27:25 +02:00
leonarski_fandClaude Opus 4.8	768a926e0c	rugnux: reject 2nd-lattice overlaps in the space-group search (E^2 cap) ... On a two-lattice crystal a second lattice deposits intensity on one reciprocal position but not its symmetry mate, so a contaminated reflection is a one-sided resolution-normalised-E outlier that poisons the operator I(h)/I(Rh) correlation in SearchSpaceGroup Stage A. On EP_cs_02-424 under -A this dropped the monoclinic 2-fold CC to 0.326 (below the 0.5 gate) -> the crystal was under-called P1 instead of P2_1. The existing --reject-outliers cannot see it: it rejects WITHIN a P1 orbit, but the poison is a BETWEEN-orbit effect (inflates orbit h, not its mate Rh). Add SearchSpaceGroupOptions::max_e_squared_for_cc (default 9.0, i.e. E>3): after the resolution-normalised Esq[] is computed, drop the extreme-E tail from the correlation pairs only (the absence stage keeps the full range - that is where the screw signal lives). Clean Wilson-distributed data almost never reaches E^2=9 (P ~ 0.01-0.3%), so it is self-targeting - it trims the overlap tail without touching genuine reflections. EP_cs_02-424 -A: 2-fold CC 0.326 -> 0.64 -> P2_1 (= XDS). Full 24-crystal rotation battery (pre/post): only EP_cs_02-424 changed (P1 -> P2_1, SG match 20 -> 21/24); all 23 others bit-identical, including the marginal cubic Ins_I_3 (still I23, carried by the systematic-b rescue). A Friedel-merge-the-search alternative was tried and rejected (net-negative: did not fix EP424 and regressed Ins_I_3 -> I222). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 23:22:38 +02:00
leonarski_fandClaude Opus 4.8	4c050b576c	rugnux: de-double spurious rotation supercell in candidate selection ... The two-pass rotation indexer could keep a doubled-axis supercell over its own primitive sub-cell: when the supercell won candidate (ci) order and the primitive cleared the indexed-fraction hysteresis by less than the 0.05 "prefer-earlier" margin, the supercell survived. On pding4_003 full data this gave P222 on a 65.6x131.3x173 cell instead of P422 on 65.6x65.6x173 - the orthorhombic metric forecloses the 4-fold before the symmetry search ever runs. The FFT peak-finding is correct (the primitive cell is among the candidates, just out-ranked). Add a sub-cell override to the selection loop: adopt a later candidate that is a genuinely smaller cell (> ROT_SUBCELL_VOLUME_RATIO=1.5x smaller volume; a doubling is 2x) indexing at least as many spots (within ROT_SUBCELL_FRAC_SLACK=0.02). That is the signature of a spurious doubling - the primitive always indexes >= its integer multiple, whereas a real superstructure's larger cell indexes MORE (kept by the existing clearly-more branch) and twins share the cell volume (untouched). Full 24-crystal rotation battery (pre-fix vs post-fix): pding4_003 P222->P422 (cell halves to 65.6x65.6x173, R_meas 6.6% CC1/2 99.9% ISa 11.9), pding4_001 holds P422, all 22 other crystals bit-identical in space group. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 21:53:08 +02:00
leonarski_fandClaude Opus 4.8	4c5da20eda	rugnux: add --partiality-uncertainty stills merge term (partiality-model error) ... A stills reflection recorded at partiality p carries a systematic intensity error ~(dp/p) that is proportional to <I> and grows as p falls; plain counting sigma misses it, so strong low-p partials are over-trusted in the merge. Add sigma^2 += (c*<I>*(1-p))^2 in MergeOnTheFly::CorrectedSigma - the stills-partiality analog of the existing rotation --capture-uncertainty term ((1-captured_fraction)*I in RotationScaleMerge). Inert when partiality==1 (no --still-partiality), and gated on a real systematic (error_model_b > 1, i.e. ISa < 1) so it fires on strong/medium stills but auto-skips weak counting-limited data where it would only over-concentrate the merge and hurt. Opt-in via --partiality-uncertainty <c> (default 0; ~2.5 recommended with --still-partiality). Prototyped (Python replica of the merge): lyso8 CC1/2 +1.7, CCref +4.8, R-free -0.023; LOV CCref +2; harmful on weak OCP (hence the b>1 gate). Full in-binary validation across all serial targets pending. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 19:52:46 +02:00
leonarski_fandClaude Opus 4.8	2f5ed411e2	rugnux: --refine-geometry stills two-pass (global bundle adjustment + re-index) ... Weak/jet serial stills are often geometry-limited: a few-px beam error or a mm-scale detector distance error (unreachable per-image) fails many frames, but is well-determined jointly from the strong frames. Mirror the rotation two-pass: an index-only first pass over a spread sample collects each indexed frame's spots + assigned HKL + orientation; the strongest ~N (default 200) feed one Ceres bundle adjustment; the refined geometry is applied and the main pass re-indexes + integrates + merges every frame from scratch. GeometryRefiner (reusing the extracted XtalResidual - the RecipToDetector geometry residual pulled out of XtalOptimizer, behaviour-preserving): one problem with SHARED beam(2)/distance(1)/cell-length(3) blocks + a PER-FRAME orientation(3) block, robust Cauchy loss, a cell-length regularizer anchoring the known cell to break the low-resolution distance<->cell-scale degeneracy, DENSE_SCHUR eliminating the per-frame orientations, and a 3-round HKL-reassignment / tolerance-tightening loop. Tilt is not refined (gauge-coupled, zero gain). Opt-in via --refine-geometry[=N]; stills only (rotation untouched). Validated: KR2 7.58% -> 21.85% (matches CrystFEL's 21.5%; a real ~1.4mm distance error + ~3px beam), OCP 2.92% -> 4.19% (~3px beam). OFF runs are bit-identical to baseline (XtalResidual extraction non-regressing; rotation lyso_ref de-novo ISa 17.3 unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 18:17:58 +02:00
leonarski_fandClaude Opus 4.8	49963ab855	rugnux: move indexing-ambiguity resolution out of ScaleOnTheFly ... Indexing-ambiguity detection/resolution is no longer part of scaling: it is a dedicated per-image step for stills (or a post-full-merge step for rotation). Pull the stills per-image resolver out of ScaleOnTheFly into a new ReindexAmbiguityResolver in ReindexAmbiguity.{h,cpp}, alongside the existing rotation free functions (ChooseReindex / ReferenceIntensityCC). Both workflows now share the operator generation and the best-op selection (new file-local PickBestReindex helper); ChooseReindex is rewired onto it. ScaleOnTheFly is now purely a scaling engine (no ambiguity_ops, no resolve_ambiguity flag, no ResolveIndexingAmbiguity). Pure refactor: behaviour is unchanged. IndexAndRefine now holds a ReindexAmbiguityResolver and calls Resolve() at the same call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 15:35:40 +02:00
leonarski_fandClaude Opus 4.8	3f6462b30f	rugnux: rotation indexing always uses FFT/FFTW, never FFBIDX ... The rotation two-pass accumulates a dense multi-frame reciprocal-space cloud (all first-pass frames de-rotated into one common frame) and needs a GLOBAL lattice finder - a 3D FFT whose peaks are the basis vectors. ffbidx is a single-still, known-cell ORIENTATION finder: it assumes the spots lie on one Ewald shell (one crystal, one orientation) and maximizes a near-integer-hkl count for a fixed cell. On the volume-filling rotation cloud that objective has no dominant global maximum, so ffbidx's coarse orientation scan locks onto an orientation valid for only a local wedge (~18%) -> garbage merge. GetIndexingAlgorithm resolved Auto->FFBIDX whenever a cell was known, even for rotation - and a reference MTZ (-z) or -C silently supplies the cell. So a routine rotation run with a reference for CCref/R-free flipped the indexer to the one solver that cannot handle its input (lyso_ref: ISa 17.8 with FFT -> 0.5 with FFBIDX, no warning). Force rotation to FFT (GPU) / FFTW (CPU) regardless of the requested algorithm or a known cell. Validated: lyso_ref -R with -z reference MTZ now indexes via FFT -> ISa 17.8 (was 0.5), 15975 refl. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 15:18:58 +02:00
leonarski_fandClaude Opus 4.8	95af137146	rugnux: the external reference MTZ is never a scale anchor (stills + rotation) ... An external reference MTZ is a poor per-image scale reference: it is a different crystal/dataset, so scaling against it injects cross-dataset systematics rather than removing per-image scale error (measured: reference-scaled R-free 0.410 vs self-scaled 0.388 on OCP; 0.378 vs 0.350 on LOV). Its real value is fixing the cell/space group, breaking the merohedral indexing ambiguity, reporting CCref and providing the R-free test set - none of which need it to be a scale anchor. Make that consistent across both workflows: - Stills: the per-image pass (ScaleImage) now uses the reference ONLY to break the indexing ambiguity (once, for good), not to fit G. Scaling self-references at the post-measurement merge for every run, whether or not a reference is given (Rugnux + rugnux_cli --scale). - Rotation: ScaleImage no longer scales against the reference either (it was dead code - RotationScaleMerge recomputes the per-frame scale, so the reference never actually moved the result). Rotation resolves the ambiguity globally (ChooseReindex / ReferenceIntensityCC) and self-scales in RotationScaleMerge. - A reference-as-scale mode, if ever wanted, would be a dedicated later step, not this per-image pass. Validated: OCP CCref 56.5%->64.6%, R-free 0.400->0.393 (honest 2.36 A cutoff instead of a reference- propped 1.5 A); LOV CCref 88.3%->90.1%, R-free 0.336->0.331 with more reflections; rotation lyso_ref with the reference gives ISa 17.8 == de-novo 15.6 (same 15975 reflections, CC1/2 99.8%) - the reference demonstrably does not touch rotation scaling, only cell/SG + ambiguity + CCref. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 13:01:56 +02:00
leonarski_fandClaude Opus 4.8	9527fa6e2c	rugnux: scale self-referenced stills in a single pass (fix weak-data collapse) ... The stills self-scaling loop rebuilt the merge reference FROM the just-scaled reflections every iteration, so on weak data each pass re-fit its own noise and the merged CC1/2 collapsed (PfluDING combine: 0.3% at the default 3 iters). It is provably pointless too: ScaleOnTheFly::Scale re-solves the per-image G from raw I and never reads the prior correction, so against a fixed reference passes 2..N are bit-identical - the single G is the exact one-pass solution and iteration only ever does the harmful reference rebuild (N=1 is best on every dataset, including strong lyso 82% vs 76% at N=3; leave-one-out and external anchors were tested and do not beat it). Do one self-reference pass; rotation (RotationScaleMerge) and the external-reference branch are untouched. Validated: PfluDING-combined CC1/2 0.3% -> 42.0% by default (pinned 2.5 A); strong data unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 12:05:08 +02:00
leonarski_fandClaude Opus 4.8	4a5028d98d	rugnux: add -r multi geometry refinement (per-image best of three) ... Per-image geometry refinement is a tradeoff that flips per dataset: beam+cell refinement extends OCP merge quality but DIVERGES on sparse-spot stills (e.g. KR2's ~20 spots around a 232 A axis), pushing good lattices out of tolerance so they fail the acceptance floor. `-r multi` runs all three (none / orientation / beam_and_lattice) on a copy per image and keeps whichever indexes the most spots (scorer = fractional-Miller-within-tolerance count, mirroring AnalyzeIndexing); ties prefer less refinement, to avoid overfitting the sparse list. Validated: KR2 index 7.08% (-r beam_and_lattice default) -> 10.05% (matches the -r none best), while OCP R-free stays ~equal to beam_and_lattice. New GeomRefinementAlgorithmEnum::Multi handled in the CLI/HDF5/command-line echoes; the API convert maps it to beam_and_lattice (offline only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 12:04:32 +02:00
leonarski_fandClaude Opus 4.8	fb399480e5	rugnux: add opt-in stills partiality (--still-partiality) ... Serial stills currently treat every reflection as a full (partiality hardcoded to 1). Add an opt-in Gaussian excitation-error partiality set at prediction time (CPU + CUDA): p = exp(-dist_ewald^2 / (2*sigma^2)), sigma^2 = profile_radius^2 + (bandwidth_sigma*|recip_z|)^2, with sigma = the per-image profile radius (ewald_dist_cutoff/2), so an edge-of-acceptance reflection keeps p ~ exp(-2). Off by default; the merge weight (~p^2) then down-weights far-from-Ewald partials instead of trusting them as fulls. Validated: helps medium/strong stills (LOV R-free 0.336->0.329, lyso8 0.433->0.410, lowers the systematic error-model b in both) but HURTS weak OCP (dividing by a small, uncertain p amplifies orientation error -> high-res noise, resolution collapse), so it is left opt-in. A static forward p explains only ~6-10% of the partiality scatter; the full win needs per-image post-refinement (future work), for which this is the prediction-side groundwork. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 12:03:21 +02:00
leonarski_fandClaude Opus 4.8	b0202a3aa2	rugnux: default the stills integration box to r=6 (6,8,12) ... Serial-stills spots span a range of crystal orientations within a single shot, so they land wider on the detector than the r1=4 monochromatic-rotation default assumes, and the r1=4 signal box truncates the (bandwidth/mosaicity-broadened) spot wings. The profile-fit integrator can use a larger box for free - its profile weighting drives far-out background pixels to ~zero weight, so the estimator variance saturates rather than growing (a plain box-sum, which sums every pixel at weight 1, degrades). Gated on stills (!rotation_indexing); rotation stays at 4,6,10 (a larger box there only adds background - measured wash-to-negative on the XDS rotation battery, with an SG regression); an explicit --integration-radius wins. Validated: OCP R-free 0.406->0.400, LOV 0.354->0.336; lyso8 1.60->1.57 A; lyso5 (1% DMM) 1.80->1.72 A with +13% reflections at equal CC1/2 - the broadened case benefits most. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 12:00:26 +02:00
leonarski_fandClaude Opus 4.8	db8ae0452e	rugnux: fix --scale on a self-contained _process.h5 (reflections + error model) ... Offline --scale re-scales the reflections stored in a _process.h5 without re-integration, but was broken for a self-contained integrated snapshot: 1. HDF5MetadataSource::ReadReflections fell back per-image to the linked source pixel files for every non-indexed frame (a snapshot's master holds /entry/reflections only for the sparse indexed images). With the raw _data_NNNNNN.h5 absent or not co-located this threw an HDF5 error; when present it needlessly reopened multi-GB files thousands of times. Decide once whether the master is the authoritative reflection store and, if so, never fall back - a missing per-image group just means the frame has none. Legacy/VDS acquisitions (no /entry/reflections in the master) still resolve reflections lazily to the source data files. 2. The scale-only stills branch never fit the (a,b) error model, so it merged with the identity model - far worse intensities than the full pipeline (lyso8 CC1/2 76%->21%). Fit RefineErrorModel and honour --reject-outliers, mirroring Rugnux.cpp. Round-trip validated: full merge == integrate(--no-merge) + --scale (identical error model a=0.793 b=2.287, matching unique reflection counts), and --scale now reads reflections in ~2.5s with no raw-file access. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 10:06:07 +02:00
leonarski_fandClaude Opus 4.8	097b9a29d1	rugnux: add --spot-low-resolution CLI option (exclude beam halo) ... The spot-finding low-resolution limit was hardcoded at 50 A (r ~36 px), which admits the direct-beam halo/overload zone (24-50 A) on weakly-diffracting serial data. On KR2 (I222, 235 A axis, ~3.9 A diffraction) that halo is ~387 strong pixels/frame vs ~25 real Bragg pixels, swamping the signal ~15:1 (CrystFEL cuts it with min-res=75 px = 24 A). Expose the limit so it can be tightened, e.g. --spot-low-resolution 24. Default unchanged (50 A). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 23:14:27 +02:00
leonarski_fandClaude Opus 4.8	2f3041c44e	rugnux: add --min-pix-per-spot CLI option (spot finding) ... The minimum connected strong-pixel count per spot was hardcoded to 2 (SpotFindingSettings::min_pix_per_spot). Expose it on the CLI so serial-stills data can be run with min-pix 1 (plus a higher --spot-threshold), matching the CrystFEL peakfinder8 practice that trades noisier spots for a higher indexing rate. Default unchanged (2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 22:02:49 +02:00
leonarski_fandClaude Opus 4.8	2d7ddb1b67	ScaleOnTheFly: resolve the stills indexing ambiguity per image vs a reference ... Serial stills index each crystal in one of the merohedrally-equivalent hands at random, so the ambiguity must be broken per image, not globally as the rotation path (RotationScaleMerge/ChooseReindex) does. When an external reference MTZ is supplied, IndexAndRefine::ReferenceIntensities now builds the per-image scaler with resolve_ambiguity=true: for each image it picks the reindexing (identity or a twin law from ReindexAmbiguityOperators) whose partiality/Lorentz-corrected intensities correlate best with the reference, then reindexes that image's reflections before scaling. Off for the self-merge scaling pass (no trusted reference to break the tie). Holohedral crystals (no twin law, e.g. lysozyme P43212) are a no-op. Validated on OCP P3221 (twin law -x,-y,z) with a 7ZSJ reference: CCref 24->48% and CC1/2 24->64% at 30k images; CC1/2 96.9% / CCref 76% at 150k. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 19:46:48 +02:00
leonarski_fandClaude Opus 4.8	87778c6500	reader: treat negative total_flux as unknown/absent ... Serial-stills master files can store /entry/instrument/beam/total_flux as a -1.0 "unknown flux" sentinel (seen in the OCP dark dataset). DatasetSettings rejects values below 0, so opening such a file hard-failed with "Input parameter below min (Total flux)". Reset a negative flux to nullopt so it is treated as absent instead of aborting the read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 19:46:48 +02:00
leonarski_fandClaude Opus 4.8	3df9ad631b	docker: fix build_in_rocky9.sh shed-check for RHEL9's glib/selinux sources ... The old check flagged glib/selinux/pcre as failures, inheriting a rocky8-host assumption (glib is Qt-only, selinux is dbus-only) that does not hold on RHEL9: - libglib is pulled by the system harfbuzz/fontconfig/freetype text stack (glib-linked on RHEL9), not by Qt. QT_FEATURE_glib=OFF is honoured (Qt's enabled_features has no glib) but cannot drop the font stack's libglib. - libselinux/libpcre2-8 come from the krb5/GSSAPI stack (libgssapi_krb5 -> libselinux) that static curl links for auth -- kept for the reverse-proxy Kerberos plan, same bucket as libcrypto. So hard-fail only on the dbus/systemd tail we actually remove (libdbus/ libsystemd/zstd/lz4/cap/gcrypt/gpg-error), and just report glib/selinux as expected residuals with the reason. Verified in the rocky9 image: that tail is gone; the survivors trace to the font stack and krb5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 19:27:43 +02:00
leonarski_fandClaude Opus 4.8	d90d498cdb	docker: put the static libdbus on CMAKE_PREFIX_PATH for Qt6::DBus consumers ... The Qt build found dbus via -DDBus1_DIR, but a static Qt records the dependency and every consumer re-runs find_package(DBus1) through Qt's exported config (Qt6Gui -> Qt6DBus -> find_dependency(WrapDBus1)). Without the dbus prefix on the consumer's search path that fails: Could NOT find WrapDBus1 (missing: DBus1_LIBRARY DBus1_INCLUDE_DIR ...) Qt6DBus could not be found because dependency WrapDBus1 could not be found. Add /opt/dbus-<ver>-static to the image's CMAKE_PREFIX_PATH env so the viewer build (and the CI .deb/.rpm and tarball builds) resolve it. Verified against the rocky9 image: find_package(Qt6 ... DBus) fails with the old path and passes with dbus on the env CMAKE_PREFIX_PATH. Also make build_in_rocky9.sh self-sufficient on images built before this fix: it discovers /opt/dbus-*-static/lib/cmake/DBus1 and passes -DDBus1_DIR, so the viewer verification works without rebuilding the image first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 19:13:29 +02:00
leonarski_fandClaude Opus 4.8	b6c2d5848d	docker: build the static libdbus with -fPIC so Qt's PIE tools link it ... The static libdbus linked fine into libQt6DBus.a, but Qt links its D-Bus code generators (qdbusxml2cpp/qdbuscpp2xml) as position-independent executables (-fPIE -pie), and the default autotools dbus build produces non-PIC objects: ld: libdbus-1.a(...dbus-address.o): relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE Build libdbus with --with-pic and CFLAGS=-fPIC so its archive objects are position-independent and link into both the static Qt libs and the PIE tools. Reproduced (non-PIC -> R_X86_64_32 fail) and fixed (PIC -> PIE whole-archive link passes) in a rocky8 container before applying. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 15:52:50 +02:00
leonarski_fandClaude Opus 4.8	254bd82d52	docker: add parallel image-build + in-container viewer-verify scripts ... build_images.sh builds (and optionally pushes) the four build-environment images as gitea.psi.ch/leonarski_f/jfjoch_<variant>:<TAG>. Runs a JOBS-capped parallel job pool (default 2; each build internally runs make -j$(nproc), so a low cap avoids CPU/RAM thrash while still overlapping the network-bound base pull / install / download phases), streams each build to docker/build-logs/<variant>-<TAG>.log, prints an OK/FAIL summary, pushes only the ones that succeeded, and exits non-zero on any failure. Context is the tiny per-variant dir (no COPY in the Dockerfiles), so the repo is never sent to the daemon. build-logs/ is already covered by the root .gitignore build*/ rule. build_in_rocky9.sh builds the viewer (JFJOCH_VIEWER_ONLY) inside a chosen variant's image and ldd-checks that the dbus/systemd/glib/selinux tail is gone -- the quick verification for the static-libdbus + glib-off changes. The repo is mounted but the build tree lives in the container's /tmp, so nothing root-owned lands in the working copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 15:43:18 +02:00
leonarski_fandClaude Opus 4.8	5baf2a2cac	docker: static libdbus + disable Qt glib to shed the viewer's .so tail ... The jfjoch_viewer runtime pulled a large dynamic .so tail that traces to two Qt-side integrations, neither of which the viewer actually needs: - Qt6::DBus -> system libdbus-1.so -> libsystemd (libzstd/liblz4/libcap/ libgcrypt/libgpg-error) + libselinux (libpcre2). The viewer is a pure session-bus client (registers ch.psi.jfjoch_viewer + exports an adaptor for single-instance/remote-control), so the daemon-side systemd/selinux features are irrelevant. Build a static libdbus from source with those integrations disabled and link it into the viewer, the same from-source-static pattern already used for OpenSSL. Swap the dbus dev package for expat (dbus's configure-time dep; the client libdbus-1 links neither expat nor systemd) so nothing pulls the system shared dbus back in. Qt discovers dbus through find_package(DBus1) (its FindWrapDBus1.cmake), NOT pkg-config -- so point it at our build with -DDBus1_DIR=<prefix>/lib/cmake/ DBus1 (the CMake package config dbus installs, which imports libdbus-1.a) and force QT_FEATURE_dbus_linked=ON so QtDBus links the archive instead of dlopen'ing a (now nonexistent) libdbus-1.so.3 at runtime. - Qt's glib event-dispatcher -> libglib-2.0/libgthread/libpcre. Nothing in the viewer (or any lib it links) drives a GLib main context, so switch Qt to its own QEventDispatcherUNIX via QT_FEATURE_glib=OFF and drop the glib dev package. Applied identically across rocky8, rocky9, ubuntu2204, ubuntu2404. Note: system libcrypto still appears at runtime, but it is dragged in by the dynamic system krb5/GSSAPI (built against OpenSSL on RHEL/Rocky), which we keep dynamic for the planned httpd-reverse-proxy Kerberos auth -- not from Qt or curl, which both statically link the from-source OpenSSL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 15:35:43 +02:00
leonarski_f	dd0bffb283	v1.0.0-rc.159 (#69) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell. * rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged. * rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set. * rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme. * rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free. * rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry. * Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md. * Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #69 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.159	2026-07-13 13:54:03 +02:00
leonarski_f	451310f43d	v1.0.0-rc.158 (#68) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * Analysis: The azimuthal-integration solid-angle correction now follows the incidence angle to the detector normal (`cos^3` of that angle) instead of `cos^3(2*theta)`, so it is correct for a tilted detector and matches PyFAI `solidAngleArray` and MAX IV azint (unchanged for an untilted detector). Crystal geometry refinement (`XtalOptimizer`) no longer silently ignores an imported PONI `rot3` (rotation about the beam): it is applied as a fixed rotation in the residual so refinement stays consistent with the rest of the pipeline. Polarization and azimuthal binning already honoured `rot3` through the full PONI rotation. * jfjoch_viewer: Open datasets on the WSL2/UNC filesystem (paths starting `\\`); write processing outputs next to the input file, with a Browse button and independent `_process.h5` / merged `.mtz`/`.cif` toggles; and show the determined space group in the merge-statistics window. * rugnux: Accept an absolute `-o` output prefix in offline processing. * Packaging: The self-contained Linux viewer `.tgz` now bundles cuFFT, so it runs without a system CUDA toolkit (`.deb`/`.rpm` are unchanged, distro-managed). * Docs: Bring the analysis references up to date with the code. `docs/CPU_DATA_ANALYSIS.md` now reflects the unified profile-fit Bragg integration engine, multi-lattice indexing, azimuthal phi binning, the radial parallax/bandwidth profile with sub-pixel centring, the rot3d capture-fraction handling and the automatic CC1/2 resolution cutoff, and drops the descriptions of features that were never implemented (French-Wilson amplitudes, the still excitation-error partiality model); `docs/RUGNUX.md` documents the new `--resolution-cutoff`/`--resolution-cc-target`/`--resolution-shells`, `--min-captured-fraction`, `--mosaicity`, `--reference-column`, the azimuthal correction toggles and the geometry-override options, and corrects the `-N` default. The outdated in-source design notes (ICE_RING_DETECTION, BRAGG_INTEGRATION_ENGINE, NEXTGEN_INTEGRATOR) are removed.Reviewed-on: #68 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.158	2026-07-12 19:42:29 +02:00
leonarski_f	54c0100e8e	v1.0.0-rc.157 (#67) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Rebrand the offline data-processing subsystem as `rugnux` and consolidate all offline analysis into the single `rugnux` binary - `jfjoch_process` is now `rugnux`, the former `jfjoch_azint` is now `rugnux --azint-only`, and `jfjoch_scale` is now `rugnux --scale` (see the new docs/NAMING.md and docs/RUGNUX.md). Scaling and merging are on by default for rotation and stills (`--no-merge` disables them), replacing the previous opt-in `-M, --scale-merge`. * rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling. * rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics. * rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below <I/sigma> = 1. * rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group. * rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix. * jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation. * jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning. * jfjoch_broker: Drop uncollected placeholder rows (number = -1) from the scan_result REST endpoint. * jfjoch_broker: Fix the inverted per-image compression ratio reported by the Lite receiver (was compressed/uncompressed instead of uncompressed/compressed). * jfjoch_broker: Bragg integration adds a quantization-noise variance floor with a box-sum fallback, and treats the type-maximum marker as an invalid pixel for unsigned image types. * jfjoch_writer: Detect file-overwrite conflicts at start for back-channel transports, and reset the writer when end-of-collection finalisation fails. * jfjoch_viewer: Preview overlays follow the geometry (resolution/ROI arcs, true beam centre, predictions, coral secondary-lattice spots, legend), add save-as-JPEG, and fix an HTTP live-follow memory leak. * Frontend: Improved aesthetics and usability, and added in-browser pixel-mask and JUNGFRAU-pedestal visualisation. * CI: Name the Windows installer jfjoch-viewer-* instead of jfjoch-*.Reviewed-on: #67 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.157	2026-07-11 07:19:11 +02:00
leonarski_f	d6389e12da	v1.0.0-rc.156 (#66) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output. * jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling. * Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool. * jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection. * jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata. * CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.156	2026-07-03 19:18:56 +02:00
leonarski_f	54c667190f	v1.0.0-rc.155 (#65) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Remove pixelrefine option (replaced with ProfileIntegrate2D) * jfjoch_viewer: Some graphical improvements. * jfjoch_viewer: Simplify und unify data analysis settings. * jfjoch_writer: Add TCP keepalive to increase robustness if jfjoch_broker "dies" in the middle of data acquisition. Reviewed-on: #65 1.0.0-rc.155	2026-06-25 22:01:48 +02:00
leonarski_f	6136f858af	v1.0.0-rc.154 (#64) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Fix to TCP file pusher (remove kernel zero copy to improve reliability) Reviewed-on: #64 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.154	2026-06-25 18:12:00 +02:00
leonarski_f	75e401f0e5	v1.0.0-rc.153 (#63) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing * jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files * jfjoch_broker: Add ROI calculation in non-FPGA workflow * jfjoch_broker: Fixes to TCP image pusher * jfjoch_broker: Remove NUMA bindings * jfjoch_broker: Improvements to indexing * jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values * jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors * jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs * jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results Reviewed-on: #63 1.0.0-rc.153	2026-06-23 20:29:49 +02:00
leonarski_f	c49bd2ac3b	v1.0.0-rc.152 (#62) ... * jfjoch_broker: Fix bounds for azimuthal integration for Q spacing (allow Q of 1e-5) * jfjoch_viewer: Adjust Q bounds for azimuthal integration * jfjoch_azint: Add tool to do quick azimuthal integration Reviewed-on: #62 1.0.0-rc.152	2026-06-17 20:36:24 +02:00
leonarski_f	ef52dac2ee	v1.0.0-rc.151 (#61) ... * jfjoch_broker: For PSI EIGER detector allow to disable individual half-modules by putting empty hostname Reviewed-on: #61 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.151	2026-06-16 14:13:29 +02:00