9 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	406c406988	Bragg prediction: one limit per index, not one cube ... Each Miller index is bounded by its OWN axis - |h| <= a/d_min, |k| <= b/d_min, |l| <= c/d_min - so a single half-width has to be sized for the longest axis and then walks the short ones far past anything the resolution cut can keep. Give the predictor max_h, max_k and max_l instead, in all four implementations (CPU and GPU, stills and rotation), and derive each from its own axis. On a 149/83/226 A cell that is 23.1M candidates per frame instead of 94.2M, 4.1x fewer. Results are bit-identical, as they must be - the candidates removed are only ones the |q| <= 1/d_min cut rejected anyway: over six rotation crystals every merged observation count, high-shell CC1/2 and space group matches the cube exactly, 6/6 space groups correct. It buys almost no time, and the earlier claim that the cube cost 22% of that crystal's wall clock was wrong. Removing 4.1x of the candidates moves it 1m58s -> 1m57s, so the whole prediction sweep is ~1% of the run. The 22% that crystal costs relative to a fixed max_hkl of 100 is genuine extra work at max_l = 227: real reflections inside the resolution sphere along the long axis, predicted and integrated either way. Per-axis limits do not reduce that and cannot. The user-facing setting stays a single number: it exists to bound the work, not to describe the crystal, and applies to all three indices when set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-31 13:31:34 +02:00
leonarski_fandClaude Opus 5	196c72a7fe	Bragg prediction: the rotation GPU launch was one plane short in each direction ... The kernel guards against 2*max_hkl+1 and maps thread i to h = i - max_hkl, but the host launched a grid sized 2*max_hkl. The h = k = l = +max_hkl planes were therefore never launched while -max_hkl was, so the GPU predicted an asymmetric subset of what the CPU loop (inclusive on both ends) does. The same bug was fixed on the stills twin when the whole hkl range moved to the GPU; the rotation predictor kept the old expression. It only bites where the cell actually reaches |h| = 100 inside d_min - a ~150 A axis at 1.5 A - so most data never noticed. Over the 33-crystal rotation battery 29 crystals are bit-identical and 4 gain observations, all of them large-cell or high-resolution: +8519, +4693, +901 and +758 observations, with the high-shell CC1/2 up 15.0->15.1%, 52.0->52.2%, 76.3->76.6% and 51.6->52.1%. Nothing is lost anywhere, and R-meas and ISa move by at most 0.01. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-31 11:50:10 +02:00
leonarski_fandClaude Opus 5	ade61eea60	Rotation: predict without truncating, and keep the better of the two passes ... Digging into the selection logic showed the caps were not deciding the science - the two-pass geometry post-refinement was, and the caps only fed it randomness. Caps. The prediction buffer now grows to whatever a frame predicts instead of keeping an arbitrary subset of it, and the per-image reflection limit is raised to 65536, with the image-buffer transport headroom derived from the same constant so the two cannot drift. Measured: bit-identical output on five battery crystals, because a normal cell never approached the old limits - only a large cell (~2.8e6 A^3, ~30000-44000 predictions per frame) ever did. Pass-2 guard. The refined pass is normally the better answer, which is why it is the canonical output, but it was adopted whatever it produced. On that same crystal it merged more unique reflections than its own cell can hold - completeness "117%", which is arithmetically impossible - while the header- geometry pass sat at 92.6% and CC1/2 0.98. Compare the two and, when the refined pass is not credible, go back to the header geometry and re-run so the canonical files are the ones that are kept. Both bounds are set where only a failure reaches them. Together on that crystal: 111639 unique against XDS's 118730 (was 88000-99000 and different every run), CC1/2 98.0% (was 96.9-97.7%), ISa 8.54, and two runs now agree bit for bit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 19:20:51 +02:00
leonarski_fandClaude Opus 5	46bb3bdbab	Bragg prediction: say so when a frame overflows the prediction buffer ... Found while chasing a 12% run-to-run spread in the merged reflection count of one crystal. The GPU kernels claim output slots with an atomicAdd and, on overflow, undid the increment with an atomicSub - so the counter saturated at the capacity and the host could not tell a full buffer from an overflowing one. Which reflections survived was then decided by CUDA block scheduling and changed every run. Measured on that dataset: every frame predicts 23000-44000 against a 20000 buffer, and the spread reached the merged output (161591 / 165193 / 166110 / 166479 unique across four runs of the same command). Single-threaded runs diverge too - this is entirely GPU-side. Stop clamping the counter, so the true number predicted reaches the host, and warn once per predictor when it exceeds the buffer. Which reflections are kept is unchanged: making that reproducible means deciding what to keep when a frame predicts more than the pipeline carries, and the obvious answers are worse - the capacity is not the real limit, kPredictionOutput (10000, selected by smallest excitation error) is, and on this crystal both a bigger buffer and a strided selection collapse the merge, because the rotation combine rebuilds fulls from exactly the partials that a smallest-excitation-error cut throws away. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 18:22:39 +02:00
leonarski_f	d6389e12da	v1.0.0-rc.156 (#66) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output. * jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling. * Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool. * jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection. * jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata. * CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-07-03 19:18:56 +02:00
leonarski_f	75e401f0e5	v1.0.0-rc.153 (#63) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing * jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files * jfjoch_broker: Add ROI calculation in non-FPGA workflow * jfjoch_broker: Fixes to TCP image pusher * jfjoch_broker: Remove NUMA bindings * jfjoch_broker: Improvements to indexing * jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values * jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors * jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs * jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results Reviewed-on: #63	2026-06-23 20:29:49 +02:00
leonarski_f	fc68a9baed	v1.0.0-rc.146 (#56) ... This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144. jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset. jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling. jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file. jfjoch_scale: Dedicated tool for rescaling/merging existing data. jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel. WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being. Reviewed-on: #56	2026-05-28 18:48:35 +02:00
leonarski_f	1ab257af6c	v1.0.0-rc.125 (#32) ... This is an UNSTABLE release. This version adds scalign and merging. These are experimental at the moment, and should not be used for production analysis. If things go wrong with analysis, it is better to revert to 1.0.0-rc.124. * jfjoch_broker: Improve logic on switching on/off spot finding * jfjoch_broker: Increase maximum spot count for FFBIDX to 65536 * jfjoch_broker: Increase default maximum unit cell for FFT to 500 A (could have performance impact, TBD) * jfjoch_process: Add scalign and merging functionality - program is experimental at the moment and should not be used for production analysis * jfjoch_viewer: Display partiality and reciprocal Lorentz-polarization correction for each reflection * jfjoch_writer: Save more information about each reflection Reviewed-on: #32 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-02-18 16:17:21 +01:00
leonarski_f	07fe4dd3bb	v1.0.0-rc.124 (#31) ... This is an UNSTABLE release. This version significantly rewrites code to predict reflection position and integrate them, especially in case of rotation crystallography. If things go wrong with analysis, it is better to revert to 1.0.0-rc.123. * jfjoch_broker: Improve refection position prediction and Bragg integration code. * jfjoch_broker: Align with XDS way of calculating Lorentz correction and general notation. * jfjoch_writer: Fix saving mosaicity properly in HDF5 file. * jfjoch_viewer: Introduce high-dynamic range mode for images * jfjoch_viewer: Ctrl+mouse wheel has exponential change in foreground (+/-15%) * jfjoch_viewer: Zoom-in numbers have better readability Reviewed-on: #31 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-02-01 13:29:33 +01:00