Image buffer: the per-image CBOR metadata headroom had been re-derived from the
online reflection cap alone, which cut it from 4 MiB to 2.55 MB while the measured
worst case - reflections plus the capped spot list plus the three azimuthal arrays -
is 2.9 MB, so the receiver dropped the frames with the most to say. Restore it and
give it a name that both the code and its guard test read: written down twice, the
two had drifted and the test kept passing against the value the code had left.
Spot finding: an unset low_resolution_limit means no limit at that end, as an unset
high_resolution_limit already did. An optional rather than a zero sentinel, because
zero is not a natural "no limit" here - every pixel lies above it, so the plain
comparison masked the whole image instead of none of it, and nothing validated the
zero. The API field is no longer required; a zero is folded into the unset case at
the boundary, where older clients still send it, so one spelling reaches the
analysis code. The FPGA takes its fixed-point ceiling instead, since ap_ufixed<16,9>
wraps above 512 A and would have masked everything.
image_preprocessing: check the CUDA calls on the fused decode path - the one new GPU
file with none, and the path fed by bytes we did not produce. An unchecked
synchronise returned the host-written sentinel as if it were a measurement, so the
decode looked successful and the fallback to the host decoder never fired.
rugnux: --stride no longer writes one past the end of the per-image arrays, whose
count floored where the worker loop ceils, and the written process file links the
images actually processed rather than the first N - each frame's picture now sits
next to its own analysis.
Powder calibration: the face-centred calibrants no longer list their systematically
absent rings, so the distance fit starts from a reflection that exists rather than
an extinct one; the triclinic calibrant covers both signs of h and k instead of a
single octant, which is only valid for a diagonal metric. The test asserted the old
behaviour - one ring formula for every cubic standard - and is rewritten.
CBOR: skip an unknown tagged value in the end block, as the other four blocks
already do. One advance lands on the tagged item rather than past it, so an older
reader fed a newer end message threw and never finalized its file.
Viewer: a settings value the setter rejects no longer escapes as an uncaught throw
from a worker slot, and the field offers only what the setter accepts.
Space-group search: judge stage B on the same "present" cut stage A already computes.
Merged sigma is floored so no reflection reads above ISa, so on a low-ISa merge the
fixed cut left both stage B tests unsatisfiable - every screw axis passed unchallenged
and the centering rescue switched itself off on exactly the weak data it exists for.
Where the fixed cut is the smaller of the two they are equal and this is inert: over
the 37-crystal rotation battery every crystal reports the identical space group and
identical merge statistics, so it is a no-op there and the low-ISa case it targets
remains unmeasured.
rugnux: --polarization reaches --mode azint, which parsed the flag and then dropped
it; that mode also applies the same polarization default as every other mode.
Acknowledge the ACTS/traccc project, whose sparse connected-component labelling both
spot extractors take their algorithm from, with its citation and its license.
The rc.161 change list is brought back to one line per entry, and the user-visible
changes that were missing from it added.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
* rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell.
* rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged.
* rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set.
* rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme.
* rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free.
* rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry.
* Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md.
* Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #70
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
* jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output.
* jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling.
* Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool.
* jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection.
* jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata.
* CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
* jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing
* jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files
* jfjoch_broker: Add ROI calculation in non-FPGA workflow
* jfjoch_broker: Fixes to TCP image pusher
* jfjoch_broker: Remove NUMA bindings
* jfjoch_broker: Improvements to indexing
* jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values
* jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors
* jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs
* jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results
Reviewed-on: #63
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.
* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin
Reviewed-on: #58
This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).
* jfjoch_broker: Improve handling of rotation indexing
* jfjoch_broker: More information saved in CBOR end message (WIP)
* jfjoch_writer: Save rotation indexing lattice parameters and Niggli class
* jfjoch_viewer: Remove (for now) primitive cell information
Reviewed-on: #19
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
leonarski_fandClaude Opus 5