2 Commits

Include renames

Author	SHA1	Message	Date
leonarski_f	5c8011fe67	compression: verify the bitshuffle seam for Apple Silicon; SIMD test knows NEON ... The x86/ARM split asked for on macOS already exists: BitShuffleBlock.h (fbc507839) sends the per-block transform to bitshuffle_hperf everywhere except aarch64 with NEON, where it uses the classic bitshuffle's NEON path, and both call sites go through it. Nothing new is needed for Apple Silicon - clang --target=arm64-apple-macos defines __aarch64__ and __ARM_NEON exactly as aarch64 Linux does, both vendored files compile to arm64 Mach-O objects, the seam resolves to bshuf_(un)trans_bit_elem and bshuf_using_NEON() is 1. The header now says why the switch is a preprocessor one: a universal build compiles it once per architecture, which a CMake-time answer cannot follow. What had never happened is the NEON code actually running. It has now, under qemu-aarch64 in the project's cross image: 504 blocks (elem 1/2/4/8, 8 elements up to the 128 kB block, odd multiples of 8, random and detector-like data) plus 54 whole-buffer bitshuffle/LZ4 streams with element counts that are no multiple of 8 or of the block. The output is byte-identical (same md5) across aarch64 NEON, aarch64 without NEON (seam falls back to hperf's portable code), x86 hperf AVX2 via ifunc, hperf SSE2, hperf portable fallback, and classic SSE2 / AVX2 / scalar; every implementation decodes every other. Throughput, one thread on a loaded Zen 3, 16/32-bit, GB/s encode/decode: hperf AVX2 ~9-12/~10-11, classic AVX2 ~8/~5.5-8, classic SSE2 ~4-5/~4.5-5.5, hperf portable fallback auto-vectorised ~2.8/~5, not vectorised ~1/~1.8. Classic 128-bit SIMD beating hperf's fallback is the x86 stand-in for the choice the seam makes on ARM; no ARM hardware was available, so the NEON-vs-fallback ranking on a real core is still unmeasured. The one thing that would have failed on aarch64 is the Bshuf_SSE test, which required SSE2 outright. It now accepts NEON as well, which also makes it the check that a Mac or DGX Spark build did not end up on the scalar path. Not built: no CMake configure or jfjoch_test build was run on this machine (busy); the test expression was compiled and run standalone on x86 only. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 5957b15493dfd0cc8a18fb780a18c2573a6eccde)	2026-09-20 18:45:04 +02:00
leonarski_f	538f3504d3	v1.0.0.rc-161 (#71) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * **rugnux: significantly better quality of results, and faster.** A large rework of integration, scaling, merging, geometry refinement and space-group determination, together with measurements the program previously made no attempt at - the direct beam before indexing, the beam stop, the goniometer rotation scale, and the stretches of a sweep the crystal did not deliver. A rotation dataset typically gains observations at better <I/sigma> and R_meas, and every `mx` and `scale` run writes a `<prefix>_report.txt` results report modelled on XDS's `CORRECT.LP`. Many defaults moved with it: spot detection is self-calibrating, beam-stop detection and rotation geometry post-refinement are on, resolution limits default to as far as the detector reaches, and ice-ring handling engages only where the crystal is measured to have ice. * **jfjoch_viewer:** the beam-stop shadow, the detector calibration and the beam-centre measurement are reachable from "Analyze dataset"; the settings panel reports how the sample moved and how polarized the beam was; image rendering and interaction are faster. * **Performance:** bitshuffle+LZ4 images are decoded on the GPU rather than on the host, with the bitshuffle inverse fused into preprocessing so the decompressed frame is never held in device memory. * **Broker, writer, packaging and build:** image-slot lifetime and locking fixes, per-image datasets sized by the images actually written, the Debian/Ubuntu broker package renamed to `jfjoch`, and `image_analysis` compiling under MSVC again. **Breaking change to the rugnux command line:** * `--azint-only` and `--scale` are **removed**, replaced by `--mode azint` and `--mode scale`; the full pipeline is `--mode mx` and remains the default. A script passing the old flags now fails with the list of valid modes rather than silently running the wrong one. * `-t`/`--stride` is **refused on rotation data**: skipping frames cuts every reflection's rocking curve, so the combined fulls and their partiality would be measured over frames the sweep never recorded. Select a contiguous range with `-s`/`-e` instead. `--mode azint` and `--force-still` still take a stride. **Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional: * `image_scale_b` is removed from the `plot_type` enum, so a client requesting that plot now gets an error rather than a curve. * `azim_int_settings.high_q_recipA`, `spot_finding_settings.high_resolution_limit` and `spot_finding_settings.low_resolution_limit` are no longer `required`. All three mean "no limit at that end" when unset and are omitted from the response instead of carrying a placeholder value, which raises in a client generated from an rc.160-or-earlier spec. A value of 0 is still accepted and means the same thing. **Breaking changes to the stored formats** - a consumer reading these fields must treat them as optional: * The per-image image-scale B factor is no longer computed, so `/entry/MX/imageScaleBFactor` is absent from newly written HDF5 files and the corresponding key is absent from the CBOR DataMessage and END blocks. Files written by rc.160 and earlier still contain it and still open; nothing in the pipeline reads it any more. * `_reflns.jfjoch_diffrn_ISa` now carries the whole-range `1/sqrt(a*b)` that XDS's ISa denotes, and the error-model `a` and `b` are reported in XDS's convention; the strong-reflection asymptote moves to `_reflns.jfjoch_diffrn_ISa_asymptotic`. **A file written by an earlier version carries the asymptote under the plain `ISa` name.** Reviewed-on: #71 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-08-13 17:03:10 +02:00