4 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5.5	60ca872bc2	GPU adaptive spot finder: same spots, about half the kernel time per frame ... Measured on an RTX 5080, one worker (nsys), 16M EIGER rotation set, per frame: spot-finding kernels 757 -> ~383 us. Output is bit-identical: integer sums and a deterministic candidate/spot order throughout. - flag_strong reads the image only where a strong pixel can be: the plain ring pass notes each 32-pixel word's largest value, a geometry table gives the rings each word touches, and a word below the lowest of their thresholds is skipped. One thread per word writes the whole word, so the ring buffer needs no clearing or atomics. 128 -> 12 us. - analyze_pixel with candidates uses 16-row tiles (only tiles near a candidate are worked, and each warp's serial chain of rows is short) and issues the entering/leaving row loads before the window scans; the window count is summed in 32 bits. 254 -> 72 us. The dense path keeps its tiling. - analyze_candidates reads its column window with unconditional, unrolled loads; through value_at each load waited for its previous-pass bit and the row before. 74 -> 34 us. - scatter_bits emits a block's range with the whole warp (prefix sum over the words' bit counts, same order). 32 -> 6 us. - finish_components split into label (one block), sum (all blocks, integer atomics) and filter/compact (one block); a component's sums no longer walk the list from its first member to its last. 26 -> 12 us. Tried and not kept (slower on sm_120): warp-aggregated ring sums and a low-value histogram fast path in reduce_rings_shared, a precomputed tile-needed map, unrolling analyze_pixel's row loop. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-11 04:29:41 +02:00
leonarski_fandClaude Opus 5.5	672e182d6a	GPU engines wait for their stream before their buffers go; a lost context fails where it is seen ... A pooled CudaDevicePtr frees on the thread's allocation stream, not on the engine's stream, and the pool may hand the memory to another engine - or, past its release threshold, unmap it - as soon as that free is reached, which on an idle allocation stream is at once. An engine destroyed with work still queued (FFTIndexerGPU after SearchCap's last DirectionsChanged upload, a spot finder between DetectAt and Extract, a shadow accumulator after a pending fold, any engine on an exception path) thus had kernels or copies writing memory that was someone else's or no longer mapped. Now: - CudaStream synchronises before cudaStreamDestroy (destructor and move-assignment), which covers engines whose own stream is declared after their buffers (FFTIndexerGPU, the gather buffer); - every engine holding pooled buffers and a stream (shared or own, declared before the buffers) synchronises it in its destructor; BraggIntegrationEngineGPU also before EnsureCapacity reallocates, where a Run that threw leaves work queued. A GPU failure that is handled no longer hides a lost context: ShadowFinder, BeamCenterFFT, the rigid-body pool and model validation call cuda_throw_if_context_lost() before cuda_clear_error(), as the device-decode fallbacks already did; RotationScaleMergeGPU's Alloc does so before waiting up to ten minutes for GPU work beside it and then reporting a lost device as out of memory; and a failed cudaMalloc says why. BeamCenterFFT logs the failure it used to drop silently, the speculative geometry probe logs the exception it swallowed (its GPU fault was otherwise reported by the merge beside it, under the merge's name), and RotationScaleMergeGPU's DeviceGuard no longer throws from its destructor. Only synchronisation and error paths change: p.hkl md5 and the MTZ data (gemmi) are identical to the b530c2d full battery on myob_x10sa, cytc_x10sa, 8a1a, 9gdj, 11if, kdp_x10sa_20keV and 6z9g. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-10 10:59:24 +02:00
leonarski_f	680c36c20d	v1.0.0-rc.166 (#76) ... * `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands. * `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion. * Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants. * `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing. * A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed. * `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing. * Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences. * The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to. * The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after. * The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution. * `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have. * Twinning is no longer reported when the L-test contradicts it. * The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's. * `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots. * The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area. Reviewed-on: #76 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-02 21:17:31 +02:00
leonarski_f	538f3504d3	v1.0.0.rc-161 (#71) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * **rugnux: significantly better quality of results, and faster.** A large rework of integration, scaling, merging, geometry refinement and space-group determination, together with measurements the program previously made no attempt at - the direct beam before indexing, the beam stop, the goniometer rotation scale, and the stretches of a sweep the crystal did not deliver. A rotation dataset typically gains observations at better <I/sigma> and R_meas, and every `mx` and `scale` run writes a `<prefix>_report.txt` results report modelled on XDS's `CORRECT.LP`. Many defaults moved with it: spot detection is self-calibrating, beam-stop detection and rotation geometry post-refinement are on, resolution limits default to as far as the detector reaches, and ice-ring handling engages only where the crystal is measured to have ice. * **jfjoch_viewer:** the beam-stop shadow, the detector calibration and the beam-centre measurement are reachable from "Analyze dataset"; the settings panel reports how the sample moved and how polarized the beam was; image rendering and interaction are faster. * **Performance:** bitshuffle+LZ4 images are decoded on the GPU rather than on the host, with the bitshuffle inverse fused into preprocessing so the decompressed frame is never held in device memory. * **Broker, writer, packaging and build:** image-slot lifetime and locking fixes, per-image datasets sized by the images actually written, the Debian/Ubuntu broker package renamed to `jfjoch`, and `image_analysis` compiling under MSVC again. **Breaking change to the rugnux command line:** * `--azint-only` and `--scale` are **removed**, replaced by `--mode azint` and `--mode scale`; the full pipeline is `--mode mx` and remains the default. A script passing the old flags now fails with the list of valid modes rather than silently running the wrong one. * `-t`/`--stride` is **refused on rotation data**: skipping frames cuts every reflection's rocking curve, so the combined fulls and their partiality would be measured over frames the sweep never recorded. Select a contiguous range with `-s`/`-e` instead. `--mode azint` and `--force-still` still take a stride. **Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional: * `image_scale_b` is removed from the `plot_type` enum, so a client requesting that plot now gets an error rather than a curve. * `azim_int_settings.high_q_recipA`, `spot_finding_settings.high_resolution_limit` and `spot_finding_settings.low_resolution_limit` are no longer `required`. All three mean "no limit at that end" when unset and are omitted from the response instead of carrying a placeholder value, which raises in a client generated from an rc.160-or-earlier spec. A value of 0 is still accepted and means the same thing. **Breaking changes to the stored formats** - a consumer reading these fields must treat them as optional: * The per-image image-scale B factor is no longer computed, so `/entry/MX/imageScaleBFactor` is absent from newly written HDF5 files and the corresponding key is absent from the CBOR DataMessage and END blocks. Files written by rc.160 and earlier still contain it and still open; nothing in the pipeline reads it any more. * `_reflns.jfjoch_diffrn_ISa` now carries the whole-range `1/sqrt(a*b)` that XDS's ISa denotes, and the error-model `a` and `b` are reported in XDS's convention; the strong-reflection asymptote moves to `_reflns.jfjoch_diffrn_ISa_asymptotic`. **A file written by an earlier version carries the asymptote under the plain `ISa` name.** Reviewed-on: #71 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-08-13 17:03:10 +02:00