Commit Graph

301 Commits

Author SHA1 Message Date
leonarski_f e6c6536e3a JFJochReceiverPlots: Move to common/, so it can be linked without bringing JFJochReceiver HW dependencies 2026-06-20 16:26:11 +02:00
leonarski_f 4a39f2cfcf Fix FFTW build-tree header wiring and two MSVC warnings
FFTW exposes fftw3.h only via $<INSTALL_INTERFACE:include>, so consuming
fftw3f from the build tree (FetchContent) leaves FFTIndexerCPU.h unable to
find the header. A system-installed fftw3.h masks this on Linux, so it only
bit on Windows; add the source-tree api/ dir to fftw3f's interface includes.

Also two viewer-reachable warnings:
- time_utc.h (C4477): %03ld was fed a chrono milliseconds::rep (64-bit), not
  a long; cast the 0-999 value to int and use %03d.
- HDF5Objects.h (C4700): value-initialize the scalar read buffer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 07:15:47 +02:00
leonarski_f a872b68d6f ADUHistogram: Fix missing exception header
Build Packages / build:rpm (rocky9_nocuda) (push) Failing after 3m50s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Failing after 11m18s
Build Packages / build:rpm (rocky8_nocuda) (push) Failing after 12m7s
Build Packages / build:rpm (rocky9_sls9) (push) Failing after 12m57s
Build Packages / build:rpm (rocky8_sls9) (push) Failing after 13m4s
Build Packages / build:rpm (rocky8) (push) Failing after 13m6s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Failing after 13m24s
Build Packages / build:rpm (rocky9) (push) Failing after 11m13s
Build Packages / Generate python client (push) Successful in 27s
Build Packages / Build documentation (push) Successful in 57s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (neggia plugin) (push) Successful in 8m35s
Build Packages / XDS test (durin plugin) (push) Successful in 8m54s
Build Packages / build:rpm (ubuntu2204) (push) Failing after 10m48s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m56s
Build Packages / build:rpm (ubuntu2404) (push) Failing after 10m12s
Build Packages / DIALS test (push) Successful in 11m57s
Build Packages / Unit tests (push) Successful in 56m50s
2026-06-20 06:14:15 +02:00
leonarski_f ebc28744e2 viewer: fix MSVC compatibility in viewer-reachable code
Address portability issues found building jfjoch_viewer with MSVC:

- common/{ADUHistogram,AzimuthalIntegrationProfile}.h and
  image_analysis/spot_finding/StrongPixelSet.h only need DeviceOutput,
  so include fpga/pcie_driver/jfjoch_fpga.h (plain-C, self-contained)
  directly instead of acquisition_device/AcquisitionDevice.h, which
  dragged <unistd.h> into the viewer tree.
- common/time_utc.h: guard gmtime_r/timegm/localtime_r with the MSVC
  equivalents (gmtime_s/_mkgmtime/localtime_s) under _WIN32; drop the
  duplicated includes and add the headers used directly.
- gemmi_gph/gemmi/utf.hpp: vendor the upstream gemmi header; fileutil.hpp
  includes it on Windows for UTF8_to_wchar but it was never vendored.
- writer/HDF5Objects.cpp: ExtractFilename returns path.filename().string()
  (std::filesystem::path has no implicit conversion to std::string on
  Windows, where it is wchar_t-based).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 23:15:27 +02:00
leonarski_f eadbce51bd common: take ROIAzimuthal phi as plain floats, not optionals
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m48s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m35s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m2s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m58s
Build Packages / build:rpm (rocky8) (push) Successful in 13m0s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m58s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m29s
Build Packages / XDS test (durin plugin) (push) Successful in 7m30s
Build Packages / Generate python client (push) Successful in 39s
Build Packages / Build documentation (push) Successful in 51s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m11s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m55s
Build Packages / build:rpm (rocky9) (push) Successful in 12m15s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m11s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m33s
Build Packages / DIALS test (push) Successful in 12m0s
Build Packages / Unit tests (push) Successful in 57m36s
The phi bounds were stored as plain floats internally but passed through
std::optional in the constructor, which obscured the intent. Take them
as float arguments defaulting to 0 instead. The convention is explicit:
phi_min == phi_max means the full ring (all angles), a sector wraps
across 0 when phi_min > phi_max, and otherwise phi_min <= phi_max. The
"both bounds or none" rule belongs at the API/file boundary
(OpenAPIConvert, reader), not in the core class, so it lives there now.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 11:03:40 +02:00
leonarski_f 03814b1425 writer: save ROI bitmap and definitions in the master file
Persist regions of interest so an acquisition's ROI layout can be
recovered from the master (_master.h5 / _process.h5):

- /entry/roi/roi_map: uint16 per-pixel bitmask (bit i == ROI i).
- /entry/roi/<name>: logical definition (type + geometry params, incl.
  azimuthal phi sector) plus bit_index tying it to the bitmap plane.

The bitmap rides along in the StartMessage (CBOR, mirroring az_int_map)
and is filled both online (JFJochReceiver::SendStartMessage) and offline
(jfjoch_process). Definitions come from the already-transmitted rois.
Both are dataset-wide metadata, so they are written by the NXmx master
writer (new NXmx::ROI), not the per-image data-file plugin.

Documented the /entry/roi layout in docs/HDF5.md and the master-vs-data
writer convention in CLAUDE.md; added a CBOR roi_map round-trip test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 09:41:26 +02:00
leonarski_f 9e57219b31 common: add azimuthal-angle (phi) sectors to ROIAzimuthal
ROIAzimuthal can now be restricted to an azimuthal-angle sector in
addition to its Q/d range, enabling STXM-style directional ROIs. phi
bounds are optional (both-or-neither); absent means a full 360 ring, so
existing Q-only azimuthal ROIs are unchanged.

- ROIElement::CheckROI gains a phi_deg argument; box/circle ignore it.
- MarkROI gains an optional phi_map; ROIMap builds it from Phi_rad
  alongside the resolution map only when azimuthal ROIs are present.
- phi bounds are normalized to [0,360) and wrap-around sectors are
  supported (phi_min > phi_max).
- ROIConfigAzim carries phi_min/phi_max (kept trivially copyable for the
  union; phi_min == phi_max means full ring).

No phi crosses the wire yet (CBOR/API wiring follows separately).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 09:21:39 +02:00
leonarski_f d14b8e9e18 Merge upstream 2026-06-18 18:52:22 +02:00
leonarski_f 0f5271f14c mask: accept any-bit-depth TIFF for user mask upload
PUT /config/user_mask.tiff only accepted 32-bit unsigned TIFF, so masks
exported by tools like PyFAI (8-bit) failed. Route the upload through the
universal ReadTIFF reader and let PixelMask take a CompressedImage directly:
it validates the 2D shape against the detector's converted/raw layouts,
binarizes any 8/16/32-bit integer image (non-zero == masked), and rejects
float/multi-channel images.

Also dedupe the TIFF readers: ReadTIFFFromString16 is now a thin wrapper over
ReadTIFF, and the now-unused ReadTIFFFromString32 is removed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 16:05:04 +02:00
leonarski_f 5d9e1be814 portability: replace M_PI / std::numbers::pi with a host+device-safe PI
M_PI is a POSIX <math.h> extension that MSVC does not define without
_USE_MATH_DEFINES. std::numbers::pi (introduced in the viewer guard commit) is
C++20, but CUDA here is compiled as C++17 (CMAKE_CUDA_STANDARD 17) and several
common/ headers are pulled into .cu device translation units, so std::numbers is
not available there.

Add common/JFJochMath.h with a dependency-free `constexpr double PI` that works
in host code (including MSVC), in CUDA device code, and under C++17/20, and use
it everywhere:
- common/ and image_analysis/ (incl. CUDA .cu): 78 M_PI occurrences, 22 files
- broker/OpenAPIConvert.cpp
- viewer/: the 5 files that used std::numbers::pi now use PI, for one consistent
  convention across the codebase

Verified to build: JFJochImageAnalysis (incl. CUDA), jfjoch_viewer, JFJochBroker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 14:54:01 +02:00
leonarski_f 1558dddbb8 indexing: make the resolved-algorithm invariant explicit in the pool
IndexerThreadPool dispatches on DiffractionExperiment::GetIndexingAlgorithm(),
which already resolves Auto to a concrete algorithm (FFTW/FFT/FFBIDX) or None;
the pool has no policy to resolve Auto itself. The worker handled a stray Auto
with a dead branch and silently produced no result when the resolved algorithm
had no matching indexer built.

- Document on GetIndexingAlgorithm() that it never returns Auto.
- Throw a clear internal error at the pool boundary if Auto ever arrives.
- In the worker, replace the dead Auto branch with a loud failure for any
  resolved algorithm that has no matching indexer (e.g. a GPU algorithm on a
  host without a GPU), instead of returning no result silently.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 14:25:30 +02:00
leonarski_f 01a97d61fe common: spell the azimuthal d-spacing unit d_A consistently
PlotAzintUnit used D_A while MultiLinePlotUnits used d_A for the same
d-spacing-in-angstrom unit. Align PlotAzintUnit to the lowercase crystallographic
convention already used by MultiLinePlotUnits. Generated Plot_unit_x::D_A is
left as-is (controlled by the OpenAPI generator's C++ enum casing).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 14:25:30 +02:00
leonarski_f c49bd2ac3b v1.0.0-rc.152 (#62)
Build Packages / XDS test (neggia plugin) (push) Successful in 6m2s
Build Packages / Unit tests (push) Successful in 1h37m1s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m4s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m30s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m52s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m53s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m38s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m30s
Build Packages / build:rpm (rocky8) (push) Successful in 10m47s
Build Packages / build:rpm (rocky9) (push) Successful in 11m48s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m40s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m44s
Build Packages / DIALS test (push) Successful in 12m59s
Build Packages / XDS test (durin plugin) (push) Successful in 8m33s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m24s
Build Packages / Build documentation (push) Successful in 57s
Build Packages / Create release (push) Skipped
* jfjoch_broker: Fix bounds for azimuthal integration for Q spacing (allow Q of 1e-5)
* jfjoch_viewer: Adjust Q bounds for azimuthal integration
* jfjoch_azint: Add tool to do quick azimuthal integration

Reviewed-on: #62
2026-06-17 20:36:24 +02:00
leonarski_f cc925b2668 Remove NUMAHWPolicy and the libnuma dependency
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m30s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m13s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m45s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m13s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m51s
Build Packages / build:rpm (rocky8) (push) Successful in 8m29s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 9m31s
Build Packages / build:rpm (rocky9) (push) Successful in 9m42s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m47s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m23s
Build Packages / Generate python client (push) Successful in 19s
Build Packages / Build documentation (push) Successful in 38s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 6m18s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m4s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m35s
Build Packages / DIALS test (push) Successful in 10m39s
Build Packages / Unit tests (push) Successful in 1h24m58s
NUMA CPU/memory pinning is no longer worthwhile: the FPGA DMA buffers are
placed device-local by the kernel (dma_alloc_coherent), the big RAM ring
buffer is random-access (first-touch handles placement), and GPU work is
already spread across all visible devices. So drop the pinning entirely
and with it libnuma.

- Delete NUMAHWPolicy; the only concern worth keeping - GPU selection -
  is done directly via pin_gpu() (round-robin over visible GPUs) in the
  indexer pool and the Lite analysis threads. CPU-only threads
  (FPGA acquire/pedestal/summation/frame-transform) no longer bind
  anything.
- Drop get_gpu_numa_node() (sysfs lookup) - only SelectGPUAndItsNUMA
  used it.
- numa_policy broker setting is deprecated and ignored (kept in the API
  for backward compatibility; warns once on startup).
- Remove NUMA_LIBRARY / numa.h / numaif.h detection from CMake.
- Docs: drop the NUMA dependency, remove the numa_policy config example,
  and document running multiple brokers on disjoint GPUs via
  CUDA_VISIBLE_DEVICES.
- Remove NUMA_GPU_REVIEW.md (the planning note; this work is now done).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 20:25:36 +02:00
leonarski_f d373ba0490 ImageBuffer: replace NUMA interleave with parallel first-touch
Allocate the ring buffer with plain malloc and zero it across
hardware_concurrency() threads, so each page is first-touched - and thus
NUMA-placed - by whichever node the scheduler ran the zeroing thread on.
For the random-access buffer this approximates the previous
numa_alloc_interleaved placement, speeds up the one-time fault-in of the
150-200 GB allocation, and drops the libnuma dependency from this file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:03:13 +02:00
leonarski_f 02fa15c2b9 jfjoch_process: spread per-image GPU work across all visible GPUs
The offline worker threads built MXAnalysisWithoutFPGA without selecting a CUDA
device, so all per-image preprocessing/spot-finding/azimuthal integration ran on
GPU 0 (only the indexer pool was distributed). Add pin_gpu() to CUDAWrapper - a
process-wide round-robin counter (counter++ % get_gpu_count(), no thread id, no-op
without a GPU, honours CUDA_VISIBLE_DEVICES) - and call it once per worker before
building the analysis resources so their CUDA streams/engines land on distinct
devices.

Also add NUMA_GPU_REVIEW.md: a working note mapping ImageBuffer/NUMAHWPolicy/GPU
dispatch with goals and a staged plan (multi-broker GPU isolation via
CUDA_VISIBLE_DEVICES, dropping libnuma, reassessing NUMA pinning for the FPGA path).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 15:29:52 +02:00
leonarski_f 1056acc3a6 build: guard the librt link to Linux only
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m56s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m6s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m40s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m31s
Build Packages / build:rpm (rocky8) (push) Successful in 12m46s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m50s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m42s
Build Packages / XDS test (durin plugin) (push) Successful in 8m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m54s
Build Packages / build:rpm (rocky9) (push) Successful in 11m37s
Build Packages / Create release (push) Skipped
Build Packages / Generate python client (push) Successful in 18s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m47s
Build Packages / Build documentation (push) Successful in 38s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m38s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m10s
Build Packages / DIALS test (push) Successful in 11m33s
Build Packages / Unit tests (push) Successful in 57m32s
librt is Linux/glibc-only and does not exist on macOS/Windows. Link it via a
$<$<PLATFORM_ID:Linux>:rt> generator expression (both the main JFJochCommon link and
the CUDA block) so -lrt is still passed on Linux - where some distros need it for
clock_gettime/timer_*/pthreads - and dropped elsewhere. Verified -lrt is still on the
jfjoch_process link line on Linux.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 08:40:09 +02:00
leonarski_f 39c808776f build: portability groundwork toward a Windows/MSVC viewer
- CMakeLists.txt: fetch libzmq at the top level (zeromq/libzmq v4.3.5) before
  slsDetectorPackage, so this project controls the ZeroMQ version instead of sls's
  bundled archive. sls reuses it via its if(NOT libzmq_POPULATED) guard, so a single
  libzmq-static target is built (no duplicate-target/double-symbol clash). Verified the
  full Linux build still links (JFJochZMQ -> JFJochReceiver -> jfjoch_process).
- common/NetworkAddressConvert.cpp: guard the network includes for _WIN32
  (winsock2/ws2tcpip vs arpa/inet).
- common/ImageBuffer.cpp: use std::malloc/std::free for the non-NUMA path instead of an
  anonymous mmap (the mapping had no huge-page/mbind flags, so it was equivalent) -
  portable and removes the POSIX-only dependency. The NUMA-interleave path is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 08:37:21 +02:00
leonarski_f 90e804acd7 v1.0.0-rc.150 (#60)
Build Packages / Unit tests (push) Successful in 42m49s
Build Packages / DIALS test (push) Successful in 29m45s
Build Packages / XDS test (durin plugin) (push) Successful in 19m27s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 18m52s
Build Packages / XDS test (neggia plugin) (push) Successful in 13m0s
Build Packages / Generate python client (push) Successful in 28s
Build Packages / Build documentation (push) Successful in 1m25s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m53s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m49s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m7s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m9s
Build Packages / build:rpm (rocky8) (push) Successful in 13m24s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m11s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m15s
Build Packages / build:rpm (rocky9) (push) Successful in 14m30s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m14s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m5s
* jfjoch_broker: When in FPGA workflow (with PSI detectors) azimuthal integration might be forced to CPU - this will require more computational power, but it enables more integration bins and reports standard deviation of each bin.
* jfjoch_broker: Raise error if one is in FPGA flow and there are too many azimuthal integration bins.

Reviewed-on: #60
2026-06-15 20:24:15 +02:00
leonarski_f 8a9d80eb71 PixelRefine: profile multiplier as --profile-multiplier flag; outlier rejection default off
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 29m33s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 29m48s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 29m54s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 31m23s
Build Packages / build:rpm (rocky8) (push) Successful in 31m27s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 33m18s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 33m33s
Build Packages / XDS test (durin plugin) (push) Successful in 19m8s
Build Packages / Generate python client (push) Successful in 44s
Build Packages / Build documentation (push) Successful in 1m42s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (neggia plugin) (push) Successful in 20m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 24m26s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 22m0s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 26m3s
Build Packages / build:rpm (rocky9) (push) Successful in 29m41s
Build Packages / DIALS test (push) Successful in 33m17s
Build Packages / Unit tests (push) Successful in 2h52m56s
(1) Promote the Term-2 profile-width multiplier from the PR_RMULT env knob to a real
BraggIntegrationSettings::profile_multiplier (default 6) + jfjoch_process
--profile-multiplier flag. Removes the last PR_* env knob; IndexAndRefine reads the
setting. (2) Flip the merge outlier rejection default to OFF (ScalingSettings
outlier_reject_nsigma 6 -> 0); it stays available via --reject-outliers <nsigma>, since
it helps the jet R-free but is dataset-dependent (drops crystal-2 CCref).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 13:48:11 +02:00
leonarski_f 6f733d74c2 Merge: make outlier rejection a ScalingSettings parameter + CLI flag, default 6
Promote the per-observation merge outlier rejection from the temporary PR_REJECT
env knob to a real setting: ScalingSettings::OutlierRejectNsigma (default 6, <=0
disables), driven into MergeOnTheFly via the constructor, with a --reject-outliers
CLI flag in jfjoch_process. Default-on at 6 sigma matches XDS (MISFITS/REJECT) and
DIALS (normalised-deviation test); it validated on the full-jet R-free (0.2625 ->
0.2585). Applies to both the PixelRefine and classical merge paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 12:42:25 +02:00
leonarski_f bf6efc7fe9 Integration radius: default r1=4, CLI flag, PixelRefine shares the knob
Bumped the default signal-box radius to r1=4 (r2=6, r3=8): on the lysozyme jet
(1% DMM) it lifts CCref 50.1->52.2% and CC1/2 90.8->92.5% (its broadened spots
spill past a radius-3 box), is neutral on the mono crystal and on the classical
integrator. Added 'jfjoch_process --integration-radius <r1|r1,r2,r3>' (a single
value derives r2=r1+2, r3=r1+4) and wired PixelRefine's shoebox radius to
BraggIntegrationSettings::GetR1() so it shares the classical integrator's knob.

(Explored but rejected: an elliptical/anisotropic Term-2 profile. The jet's
tangential spots are mildly anisotropic - axis ratio 1.15->1.44 low->high res,
azimuthal mosaic, separate from the radial DMM bandwidth - but using the measured
2x2 covariance as the profile DEGRADED the jet, CC1/2 92.5->83.5: the tight
measured width loses to a generous aperture, same lesson as the radius bump.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 23:27:36 +02:00
leonarski_f d31063ca3f PixelRefine: Some improvements
Build Packages / Generate python client (push) Successful in 23s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (rocky9) (push) Failing after 4m58s
Build Packages / Build documentation (push) Successful in 1m25s
Build Packages / XDS test (neggia plugin) (push) Successful in 13m57s
Build Packages / XDS test (durin plugin) (push) Successful in 15m44s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 16m9s
Build Packages / DIALS test (push) Successful in 22m45s
Build Packages / Unit tests (push) Successful in 1h44m30s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Failing after 3m0s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Failing after 3m4s
Build Packages / build:rpm (rocky8_nocuda) (push) Failing after 3m12s
Build Packages / build:rpm (rocky9_nocuda) (push) Failing after 3m58s
Build Packages / build:rpm (rocky8_sls9) (push) Failing after 4m20s
Build Packages / build:rpm (rocky8) (push) Failing after 4m18s
Build Packages / build:rpm (rocky9_sls9) (push) Failing after 4m55s
Build Packages / build:rpm (ubuntu2204) (push) Failing after 4m20s
Build Packages / build:rpm (ubuntu2404) (push) Failing after 4m17s
2026-06-11 21:25:05 +02:00
leonarski_f 59281f6330 Revert "Bragg integration: option to use azimuthal integration profile"
This reverts commit e4230bc14e.
2026-06-10 16:36:37 +02:00
leonarski_f e4230bc14e Bragg integration: option to use azimuthal integration profile 2026-06-10 14:41:36 +02:00
leonarski_f e051eed033 jfjoch_process: Remove postrefinement
Build Packages / Unit tests (push) Failing after 2s
Build Packages / build:rpm (rocky8_sls9) (push) Failing after 2m55s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Failing after 27m24s
Build Packages / build:rpm (rocky9_nocuda) (push) Failing after 27m47s
Build Packages / build:rpm (rocky8_nocuda) (push) Failing after 28m4s
Build Packages / build:rpm (rocky9) (push) Failing after 29m15s
Build Packages / build:rpm (rocky8) (push) Failing after 29m26s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Failing after 29m57s
Build Packages / Generate python client (push) Successful in 27s
Build Packages / build:rpm (rocky9_sls9) (push) Failing after 31m0s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 1m32s
Build Packages / build:rpm (ubuntu2204) (push) Failing after 29m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 14m37s
Build Packages / XDS test (durin plugin) (push) Successful in 16m33s
Build Packages / build:rpm (ubuntu2404) (push) Failing after 17m51s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 16m31s
Build Packages / DIALS test (push) Successful in 23m52s
2026-06-09 19:46:39 +02:00
leonarski_f 96edee9b2d jfjoch_writer: Add incident_wavelength_spread
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m51s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m36s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m50s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m1s
Build Packages / build:rpm (rocky8) (push) Successful in 13m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m37s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m55s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m20s
Build Packages / build:rpm (rocky9) (push) Successful in 12m18s
Build Packages / XDS test (durin plugin) (push) Successful in 8m43s
Build Packages / Create release (push) Skipped
Build Packages / Generate python client (push) Successful in 13s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m4s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m45s
Build Packages / Build documentation (push) Successful in 34s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m13s
Build Packages / DIALS test (push) Successful in 12m40s
Build Packages / Unit tests (push) Successful in 1h6m47s
2026-06-08 15:47:12 +02:00
leonarski_f 155c53acd8 jfjoch_process: First pixel refine integration 2026-06-08 15:37:27 +02:00
leonarski_f cc3eb8352c v1.0.0-rc.148 (#58)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m28s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m9s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m47s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m39s
Build Packages / build:rpm (rocky8) (push) Successful in 11m43s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m59s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m48s
Build Packages / build:rpm (rocky9) (push) Successful in 12m32s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m24s
Build Packages / XDS test (durin plugin) (push) Successful in 7m35s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m50s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m40s
Build Packages / DIALS test (push) Successful in 11m19s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin

Reviewed-on: #58
2026-06-08 08:30:35 +02:00
leonarski_f 75de40f52b v1.0.0-rc.147 (#57)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m27s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m20s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 7m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 5m59s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 7m25s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 8m30s
Build Packages / build:rpm (rocky8) (push) Successful in 7m39s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m16s
Build Packages / build:rpm (rocky9) (push) Successful in 9m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m6s
Build Packages / Generate python client (push) Successful in 12s
Build Packages / Build documentation (push) Successful in 31s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 7m6s
Build Packages / DIALS test (push) Successful in 12m3s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m11s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m50s
Build Packages / Unit tests (push) Successful in 57m33s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_viewer: Add reciprocal space viewer
* jfjoch_process: Two pass algorithm that does spot finding/indexing + integration of full dataset
* jfjoch_process: Improve logic for rotation indexer, to make execution more deterministic (still work in progress)

Reviewed-on: #57
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-06-02 11:49:24 +02:00
leonarski_f fc68a9baed v1.0.0-rc.146 (#56)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m34s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m16s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m49s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m32s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m15s
Build Packages / XDS test (durin plugin) (push) Successful in 7m16s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m12s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 47s
Build Packages / DIALS test (push) Successful in 10m18s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m46s
Build Packages / build:rpm (rocky8) (push) Successful in 1h41m2s
Build Packages / XDS test (neggia plugin) (push) Successful in 1h59m18s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.

WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.

Reviewed-on: #56
2026-05-28 18:48:35 +02:00
leonarski_f caef26873e v1.0.0-rc.145 (#55)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 16m26s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m26s
Build Packages / build:rpm (rocky8) (push) Successful in 17m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 17m32s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m16s
Build Packages / build:rpm (rocky9) (push) Successful in 12m45s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m58s
Build Packages / XDS test (durin plugin) (push) Successful in 11m22s
Build Packages / DIALS test (push) Successful in 14m28s
Build Packages / Generate python client (push) Successful in 1m1s
Build Packages / Build documentation (push) Successful in 2m40s
Build Packages / Create release (push) Has been skipped
Build Packages / XDS test (neggia plugin) (push) Successful in 10m52s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 15m2s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 17m25s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m49s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m34s
Build Packages / Unit tests (push) Successful in 44m51s
This is an UNSTABLE release. The release has significant modifications for HDF5 writing logic - in case of troubles go back to 1.0.0-rc.144.

* **Default HDF5 writing mode is with VDS, not soft-links** - this improves DIALS compatibility and makes format more future-proof, NXmx legacy format might be phased-out in the future.
* XDS plugin: Improve performance of VDS reading.
* jfjoch_writer: Significant improvement on how file systems I/O are handled through a dedicated pass-through VFD.
* jfjoch_writer: Clean-up of HDF5 routines to better handle issues.

Reviewed-on: #55
2026-05-06 21:50:02 +02:00
leonarski_f 7d34e8a049 v1.0.0-rc.144 (#54)
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m58s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m53s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m39s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m34s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m18s
Build Packages / build:rpm (rocky8) (push) Successful in 10m4s
Build Packages / build:rpm (rocky9) (push) Successful in 11m17s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m47s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m47s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m33s
Build Packages / Generate python client (push) Successful in 27s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 1m8s
Build Packages / XDS test (durin plugin) (push) Successful in 7m40s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m23s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m9s
Build Packages / DIALS test (push) Successful in 11m15s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Improve performance of preview JPEG image generator at receiver startup (saving about 150 ms on measurement start for 16M)

Reviewed-on: #54
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-05-01 17:06:36 +02:00
leonarski_f 6e0bb971ac v1.0.0-rc.143 (#53)
Build Packages / Unit tests (push) Successful in 1h13m11s
Build Packages / Generate python client (push) Successful in 33s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m1s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m12s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m34s
Build Packages / build:rpm (rocky8) (push) Successful in 15m43s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 16m35s
Build Packages / build:rpm (rocky9) (push) Successful in 16m19s
Build Packages / XDS test (durin plugin) (push) Successful in 12m11s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 14m1s
Build Packages / DIALS test (push) Successful in 16m59s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m32s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 15m12s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m33s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m25s
Build Packages / Build documentation (push) Successful in 58s
Build Packages / XDS test (neggia plugin) (push) Successful in 10m33s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

jfjoch_broker: Avoid copying gain calibration together with DiffractionExperiment

Reviewed-on: #53
2026-04-30 22:16:50 +02:00
leonarski_f d760b12a18 v1.0.0-rc.141 (#51)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m32s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m19s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m49s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m23s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m44s
Build Packages / build:rpm (rocky8) (push) Successful in 9m15s
Build Packages / build:rpm (rocky9) (push) Successful in 12m14s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m28s
Build Packages / Generate python client (push) Successful in 21s
Build Packages / XDS test (durin plugin) (push) Successful in 8m58s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m7s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m24s
Build Packages / DIALS test (push) Successful in 14m1s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m26s
Build Packages / Unit tests (push) Failing after 1h20m14s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Azimuthal integration mapping is generated with parallel computations, significantly reducing setup times
* frontend: Fix selection of FFTW in indexing settings

Reviewed-on: #51
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-30 13:04:54 +02:00
leonarski_f 4878318c27 v1.0.0-rc.139 (#49)
Build Packages / Unit tests (push) Successful in 1h17m4s
Build Packages / Generate python client (push) Successful in 1m24s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m34s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m37s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m11s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m4s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m59s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m4s
Build Packages / build:rpm (rocky8) (push) Successful in 13m12s
Build Packages / XDS test (durin plugin) (push) Successful in 9m57s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m52s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m30s
Build Packages / build:rpm (rocky9) (push) Successful in 13m59s
Build Packages / Build documentation (push) Successful in 1m7s
Build Packages / XDS test (neggia plugin) (push) Successful in 9m24s
Build Packages / Create release (push) Has been skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m54s
Build Packages / DIALS test (push) Successful in 13m14s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Further reduce startup time for DECTRIS detectors by selectively modifying SIMPLON parameters on `/start`
* jfjoch_broker: Further reduce startup time for DECTRIS detectors by not setting beam center and detector distance via SIMPLON API on '/start'
* jfjoch_broker: Add an extra message to ZeroMQ puller ready to monitor Lite worklow preparation time
* jfjoch_broker: Image buffer configuration is postponed for Lite receiver flow till start message is received
* jfjoch_broker: Use nanoseconds internally for frame/image/readout time
* jfjoch_broker: Extra messages added for receiver operation (to be removed after debugging finished)
* jfojch_broker: Improve profiling of different data analysis steps
* jfjoch_broker: Record integration reflection count
* jfjoch_broker: Fix bug where ZeroMQ preview frequency was confusing time units (micro vs. milliseconds)
* jfjoch_broker: Fix bug where '/wait_till_done' got deadlocked
* jfjoch_writer: Fix confusion between NaN and zero in floating-point datasets

**Breaking changes**: detector definition is now using nanoseconds to define minimum frame time, minimum count time and readout time.

Reviewed-on: #49
2026-04-29 09:50:50 +02:00
leonarski_f 230480e390 v1.0.0-rc.138 (#48)
Build Packages / XDS test (durin plugin) (push) Successful in 8m5s
Build Packages / DIALS test (push) Successful in 12m55s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m41s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 16m13s
Build Packages / build:rpm (rocky8) (push) Successful in 18m17s
Build Packages / build:rpm (rocky9) (push) Successful in 19m14s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m59s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m17s
Build Packages / Generate python client (push) Successful in 1m15s
Build Packages / XDS test (neggia plugin) (push) Successful in 9m58s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 14m14s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m9s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 18m21s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 19m14s
Build Packages / Build documentation (push) Successful in 1m43s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 10m14s
Build Packages / Unit tests (push) Successful in 57m40s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

    jfjoch_broker: Cleanup DECTRIS start-up code to enable a shorter start time
    jfjoch_broker: Allow for asynchronous start to allow overlapping detector configuration with other beamline preparations
    jfjoch_broker: Goniometer axis name is converted to lowercase
    jfjoch_broker: Fix bug, where wrong HTTP error codes were returned
    jfjoch_broker: Improve sigma estimation during merging (K. Takaba)

---------

Co-authored-by: takaba_k <kiyofumi.takaba@psi.ch>
Reviewed-on: #48
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-27 19:56:14 +02:00
leonarski_f c981e1b91c v1.0.0-rc.137 (#46)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m7s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m35s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m8s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m24s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m29s
Build Packages / build:rpm (rocky8) (push) Successful in 10m27s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m41s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m1s
Build Packages / Generate python client (push) Successful in 45s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9) (push) Successful in 12m48s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m10s
Build Packages / XDS test (durin plugin) (push) Successful in 8m59s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m32s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m39s
Build Packages / DIALS test (push) Successful in 13m13s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Better track time for each operation in the processing stack
* jfjoch_broker: Rewrite preprocessing of diffraction images in the non-FPGA workflow to better use GPUs (work in progress)
* jfjoch_broker: Remove ROI calculation in the non-FPGA workflow (work in progress)
* jfjoch_viewer: Toolbar displays image number starting from 1 (instead of 0)

Reviewed-on: #46
2026-04-25 19:59:21 +02:00
leonarski_f c1c170112c v1.0.0-rc.136 (#45)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m17s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m48s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m57s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 15m15s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m35s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m29s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m55s
Build Packages / XDS test (durin plugin) (push) Successful in 8m17s
Build Packages / build:rpm (rocky9) (push) Successful in 12m17s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m2s
Build Packages / Generate python client (push) Successful in 30s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 51s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m13s
Build Packages / DIALS test (push) Successful in 13m19s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m52s
Build Packages / Unit tests (push) Successful in 1h18m25s
Build Packages / build:rpm (rocky8) (push) Successful in 7m2s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Improve logic regarding indexing architecture and thread pools (work in progress).

Reviewed-on: #45
2026-04-20 11:54:33 +02:00
leonarski_f bb9f5c715f v1.0.0-rc.135 (#44)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m55s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m28s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m47s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m7s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m31s
Build Packages / build:rpm (rocky8) (push) Successful in 12m59s
Build Packages / build:rpm (rocky9) (push) Successful in 14m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m30s
Build Packages / Generate python client (push) Successful in 1m18s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m8s
Build Packages / XDS test (durin plugin) (push) Successful in 9m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m59s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m12s
Build Packages / DIALS test (push) Successful in 11m44s
Build Packages / Unit tests (push) Successful in 1h23m8s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* Multiple small bug fixes scattered across the whole code base. (detected with GPT-5.4)
* jfjoch_viewer: Improve image render performance

Reviewed-on: #44
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-16 11:59:59 +02:00
leonarski_f 4a852b4d6b v1.0.0-rc.134 (#43)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m4s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m18s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m12s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m51s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m59s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m45s
Build Packages / build:rpm (rocky8) (push) Successful in 12m29s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m2s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / XDS test (durin plugin) (push) Successful in 9m50s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9) (push) Successful in 14m15s
Build Packages / Build documentation (push) Successful in 1m6s
Build Packages / DIALS test (push) Successful in 13m10s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m45s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m58s
Build Packages / Unit tests (push) Successful in 1h20m42s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Add better locking for detector object - should help, when detector initialization takes too long
* jfjoch_writer: Enable writing single, integrated HDF5 file with both data and metadata
* XDS plugin: Add generation of Jungfraujoch plugin for XDS
* CI: Add tests with XDS and DIALS (`xia2.ssx`)

Reviewed-on: #43
2026-04-09 13:30:47 +02:00
leonarski_f 6133da1377 v1.0.0-rc.133
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m5s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m6s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m47s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m9s
Build Packages / Generate python client (push) Successful in 49s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m38s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m35s
Build Packages / build:rpm (rocky8) (push) Successful in 9m16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m22s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m20s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m52s
Build Packages / Build documentation (push) Failing after 9s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Use httplib for HTTP server instead of Pistache
* jfjoch_broker: Drop OpenSSL support
* jfjoch_broker: Base work for multi-lattice support in the future
* Update dependencies to more recent versions (spdlog, HDF5, Catch2, httplib)

Reviewed-on: #41
2026-03-26 20:50:33 +01:00
leonarski_f 64002f1e29 v1.0.0-rc.129 (#36)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m14s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m43s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m20s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / Generate python client (push) Successful in 39s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m24s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 1m0s
Build Packages / build:rpm (rocky8) (push) Successful in 10m35s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m35s
Build Packages / build:rpm (rocky9) (push) Successful in 11m17s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m9s
Build Packages / Unit tests (push) Failing after 1h18m57s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Significant improvements in TCP image socket, as a viable alternative for ZeroMQ sockets (only a single port on broker side, dynamically change number of writers, acknowledgments for written files)
* jfjoch_broker: Delta phi is calculated also for still data in Bragg prediction
* jfjoch_broker: Image pusher statistics are accessible via the REST interface
* jfjoch_writer: Supports TCP image socket and for these auto-forking option

Reviewed-on: #36
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-03-05 22:13:12 +01:00
leonarski_f 3036b1d37f v1.0.0-rc.128 (#35)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m39s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m0s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m42s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m3s
Build Packages / Generate python client (push) Successful in 1m25s
Build Packages / Build documentation (push) Successful in 53s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m55s
Build Packages / build:rpm (rocky8) (push) Successful in 12m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m35s
Build Packages / build:rpm (rocky9) (push) Successful in 11m32s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m15s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m53s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Handle properly reuse of image buffer locations
* jfjoch_broker: Fix bug in counting idle slots
* jfjoch_broker: Force obtuse angle for monoclinic cells
* jfjoch_process: Change scaling refinement tolerance

Reviewed-on: #35
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-03-03 22:24:44 +01:00
leonarski_f f3e0a15d26 v1.0.0-rc.127 (#34)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m51s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m0s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m6s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m7s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m47s
Build Packages / Generate python client (push) Successful in 29s
Build Packages / Build documentation (push) Successful in 43s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m46s
Build Packages / build:rpm (rocky8) (push) Successful in 9m33s
Build Packages / Unit tests (push) Has been skipped
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m47s
Build Packages / build:rpm (rocky9) (push) Successful in 9m55s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m4s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Default EIGER readout time is 20 microseconds
* jfjoch_broker: Multiple improvements regarding performance
* jfjoch_broker: Image buffer allows to track frames in preparation and sending
* jfjoch_broker: Dedicated thread for ZeroMQ transmission to better utilize the image buffer
* jfjoch_broker: Experimental implementation of transmission with raw TCP/IP sockets
* jfjoch_writer: Fixes regarding properly closing files in long data collections
* jfjoch_process: Scale & merge has been significantly improved, but it is not yet integrated into mainstream code

Reviewed-on: #34
2026-03-02 15:57:12 +01:00
leonarski_f 1ab257af6c v1.0.0-rc.125 (#32)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m8s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m55s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m0s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m30s
Build Packages / Generate python client (push) Successful in 20s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 39s
Build Packages / build:rpm (rocky8) (push) Successful in 9m23s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m42s
Build Packages / build:rpm (rocky9) (push) Successful in 9m38s
This is an UNSTABLE release. This version adds scalign and merging. These are experimental at the moment, and should not be used for production analysis.
If things go wrong with analysis, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Improve logic on switching on/off spot finding
* jfjoch_broker: Increase maximum spot count for FFBIDX to 65536
* jfjoch_broker: Increase default maximum unit cell for FFT to 500 A (could have performance impact, TBD)
* jfjoch_process: Add scalign and merging functionality - program is experimental at the moment and should not be used for production analysis
* jfjoch_viewer: Display partiality and reciprocal Lorentz-polarization correction for each reflection
* jfjoch_writer: Save more information about each reflection

Reviewed-on: #32
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-02-18 16:17:21 +01:00
leonarski_f 07fe4dd3bb v1.0.0-rc.124 (#31)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m23s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m32s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m15s
Build Packages / Generate python client (push) Successful in 19s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m13s
Build Packages / build:rpm (rocky8) (push) Successful in 9m10s
Build Packages / build:rpm (rocky9) (push) Successful in 9m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m52s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m42s
Build Packages / Unit tests (push) Successful in 1h12m44s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m30s
This is an UNSTABLE release. This version significantly rewrites code to predict reflection position and integrate them,
especially in case of rotation crystallography. If things go wrong with analysis, it is better to revert to 1.0.0-rc.123.

* jfjoch_broker: Improve refection position prediction and Bragg integration code.
* jfjoch_broker: Align with XDS way of calculating Lorentz correction and general notation.
* jfjoch_writer: Fix saving mosaicity properly in HDF5 file.
* jfjoch_viewer: Introduce high-dynamic range mode for images
* jfjoch_viewer: Ctrl+mouse wheel has exponential change in foreground (+/-15%)
* jfjoch_viewer: Zoom-in numbers have better readability

Reviewed-on: #31
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-02-01 13:29:33 +01:00
leonarski_f 1c4dfd03e2 v1.0.0-rc.123 (#30)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m22s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m30s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m41s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m32s
Build Packages / Generate python client (push) Successful in 18s
Build Packages / Build documentation (push) Successful in 54s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m44s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m53s
Build Packages / build:rpm (rocky8) (push) Successful in 9m40s
Build Packages / build:rpm (rocky9) (push) Successful in 10m37s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m54s
Build Packages / Unit tests (push) Successful in 1h6m33s
This is an UNSTABLE release.

* jfjoch_broker: Use newer version of Google Ceres for (potential) CUDA 13 compatibility
* jfjoch_broker: Improve performance of generating preview images, especially for large detectors (9M-16M)
* jfjoch_viewer: Improve performance of displaying images, especially for large detectors (9M-16M)
* jfjoch_viewer: Add more color schemes for better image readability
* HDF5: Common mutex for reading and writing HDF5 if both operations were to happen in the same executable
* HDF5: suppress warning if path (upstream group) doesn't exists when checking if leaf exists

Reviewed-on: #30
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-01-30 13:43:09 +01:00
leonarski_f 27496b8207 v1.0.0-rc.122 (#29)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m9s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m44s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 7m56s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m25s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 7m57s
Build Packages / Generate python client (push) Successful in 18s
Build Packages / Build documentation (push) Successful in 35s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9) (push) Successful in 8m28s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m6s
Build Packages / Unit tests (push) Successful in 1h11m19s
Build Packages / build:rpm (rocky8) (push) Successful in 6m31s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 6m40s
This is an UNSTABLE release.

* jfjoch_broker: Add thresholding to prefer shorter vectors after FFT
* jfjoch_broker: Add experimental mosaicity estimation for rotation experiments (this is work in progress)
* jfjoch_viewer: Display file opening errors
* jfjoch_viewer: When loading files over DBus add retry/back-off till the file is available

Reviewed-on: #29
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-16 15:27:40 +01:00
leonarski_f e2b240356c v1.0.0-rc.121 (#28)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m28s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m25s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m4s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m27s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m36s
Build Packages / Generate python client (push) Successful in 32s
Build Packages / Build documentation (push) Successful in 45s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m45s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m51s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m57s
Build Packages / build:rpm (rocky9) (push) Successful in 9m35s
Build Packages / Unit tests (push) Successful in 1h13m45s
This is an UNSTABLE release.

* jfjoch_broker: Report changes in the image buffer, so viewer doesn't reload constantly
* jfjoch_viewer: Improve performance of loading images
* jfjoch_viewer: Auto-throttle image loading in HTTP-sync / movie modes
* jfjoch_viewer: Auto-foreground calculated with histogram
* jfjoch_viewer: Fix rare segmentation fault

Reviewed-on: #28
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-12 21:24:20 +01:00