1143 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	4aab8078d6	rugnux: keep merohedral twins in true symmetry, robustify twinning stats ... SearchSpaceGroup: drop the log10(chi2_ref) widening of the point-group chi^2-ratio bound and tighten max_merge_chi2_ratio 2.0 -> 1.85. The variance-floor fix removed the error-model miscalibration the widening compensated for, so the widening now only let a partial merohedral twin through (Ins_H_2 R3->R32 twin 2-fold: ratio 2.01). Every genuine high symmetry across the rotation-test battery stays within ~1.7x (worst real case Thau P41212 at 1.71), so 1.85 keeps R3 in its true lower symmetry. TwinningAnalysis: make the <I^2>/<I>^2 second moment robust - skip noise-only shells (<I/sigma> < 1) and reject Wilson outliers (E^2 > 8) with one shell-mean re-iteration, so a single strong reflection in a collapsed-mean shell no longer dominates the moment. Add MerohedralTwinningPossible: in a holohedral Laue class (4/mmm, 6/mmm, m-3m, rhombohedral -3m) no twin law exists, so a low <|L|> there is a statistical artefact and is no longer flagged as twinning. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 16:48:27 +02:00
leonarski_fandClaude Opus 4.8	fbebc0d56e	rugnux: run decay + absorption surfaces on the GPU and enable by default ... The decay/absorption correction surfaces previously forced the CPU 3D combine (absorption needs each full's predicted detector position, which the GPU SoA did not carry), so they could not be default-on without losing the fast GPU path. Put them on the GPU: - carry px/py (peak partial) through the GPU combine and download them with the fulls (GetFullsPxPy); - add SetFullsCorr to re-upload the host-corrected corr to the resident fulls before the merge. The surfaces themselves stay as cheap host fits on the downloaded fulls; only px/py download + corr upload are added (~two O(n_fulls) transfers). Measured +2% wall-time on lyso_ref (9.7 -> 9.9 s), and the GPU-combine and CPU-combine (--dump-observations) paths give identical results. Enable by default via a single master toggle ScalingSettings:: CorrectionSurfaces (default true; decoupled from the stills-only -B, which is rejected for rotation again). Both surfaces are cross-validated, so they no-op where their systematic is absent - safe to leave on. Off-switch wired through the rugnux CLI (--no-scaling-corrections; --absorption removed) and the viewer (a "Correction surfaces" checkbox). Documented in docs/RUGNUX.md. Battery (24 rotation crystals), default-on vs --no-scaling-corrections: EcwtCQ066S +2.4 ISa, lyso_ref +0.5 (and R-free 0.194 -> 0.185, better than XDS); every other crystal within run-to-run noise; corrections skip (CV / dB-floor) wherever there is no signal. lyso_ref anomalous S/Cl peaks are already ~45% above XDS's - the ISa gap is metric convention, not data. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 10:56:39 +02:00
leonarski_fandClaude Opus 4.8	bab52a8077	rugnux: optional decay + absorption correction surfaces for rotation scaling ... After scale-fulls, RotationScaleMerge can now fit two optional multiplicative corrections on the combined fulls (both off by default, rotation only): - -B (decay): a global Debye-Waller relative-B rate for radiation damage, ln(Iref/Iobs) = 2 (dB/dn)(n - nbar) s^2, folded into the per-full scale. Engages only above a physical floor (total relative-B > 2 A^2), so it is a no-op on low-dose data. (-B was previously rejected for rotation.) - --absorption[=iter]: a smooth factor over the diffracted-beam direction in the goniometer frame (predicted detector position de-rotated by the spindle, so a crystal-frame direction is sampled at many angles). Wires the existing ScalingSettings::absorption_iter scaffold. Negligible at hard X-rays; matters at low energy. On lyso_ref it is ISa-neutral (+0.5) but drops R-free 0.219 -> 0.212 against the deposited model: a smooth absorption error cancels among symmetry mates (small ISa effect) yet still biases the intensities (real R-free effect). Both are CROSS-VALIDATED: the surface is fit on even frames and applied only if it improves the held-out odd-frame equivalent agreement by a clear margin (and vice versa), so a fit to noise where the systematic is absent does not generalize and is discarded - the correction never adds scatter. Battery: EcwtCQ066S +2.4 ISa, lyso_ref R-free win, others neutral. Also fixes a latent bug this exposed: the GPU->host corr download before the CPU 3D-combine was gated on --dump-observations only, so any other forced-CPU-combine path (the new -B/--absorption) read a stale host corr and mis-scaled every full. The download condition now matches the CPU-combine condition. Predicted detector position (px, py) is carried on the combined fulls for the absorption surface (CPU combine only; unused by the GPU SoA path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 10:14:00 +02:00
leonarski_fandClaude Opus 4.8	dee679380b	rugnux: exclude merge-rejected outliers from the error-model fit ... The rotation error-model (a,b) fit built its sample pool from all usable fulls, while the merge separately dropped symmetry outliers (> Nsigma from the per-reflection median). So the fitted sigmas were calibrated against a dirtier pool than the reflections that actually entered the merge. Refactor the intensity-binned median fit into a `fit_ab` lambda and re-run it once on the misfit-free pool (samples with dev2 <= reject_nsigma^2 * model variance), matching the merge's own rejection. It operates on the shared `samples` array, so the CPU and GPU error-model paths stay bit-identical. Neutral on lyso_ref (the median-binned fit is already robust to the few outliers) - a correctness fix that matters more on dirtier data. Gated on reject_outliers (default 6 sigma for rot3d). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 10:04:43 +02:00
leonarski_fandClaude Opus 4.8	a854523fee	rugnux: fix resolution-cutoff over-extension on low-res crystals ... The CC1/2-logistic auto resolution cutoff extended the fall-off crossing by "one shell" = s_range/10, where s_range spans to the detector edge. On a low-res crystal read out by a high-res-configured detector (integration runs to the ~1.4 A edge), that range is dominated by high-res noise, so one "shell" is a huge step in s that overshoots the true fall-off by ~1 A: Benas_3's CC1/2=0.30 crossing at 3.77 A was pushed to 2.98 A, Benas_7's 4.04 A to 2.97 A. The logistic fit itself is accurate; only the extension was wrong. Anchor the extension to the range actually kept and reported (low-res plateau -> the crossing), (s_cross - s_lo)/10, instead of the detector-edge range. Benas_3 -> 3.60 A, Benas_7 -> 3.85 A. The change is monotonic in (crossing - edge) and always coarser (never adds noise): zero change for crystals diffracting to the edge, negligible for well-diffracting ones, and only meaningful where the detector over-reaches the diffraction limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 07:08:44 +02:00
leonarski_fandClaude Opus 4.8	27802a30a3	docs: changelog for rc.158 rugnux/viewer/robustness changes ... Faster parallel two-pass rotation indexing, rotation indexing on spot-less DECTRIS datasets, viewer https connection, and hardened TIFF/raw-TCP input handling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 23:26:26 +02:00
leonarski_fandClaude Opus 4.8	0dfaf0a58a	rugnux: don't reuse absent spots on plain DECTRIS datasets ... reuse_rotation_spots defaults on, so the two-pass first phase read stored spots from /entry/MX. A standard DECTRIS dataset has no spot-finding results, so every frame came back with zero spots and indexing failed ("Two-pass rotation indexing failed") instead of falling back to finding spots. Add HasStoredSpots() to the reader (HDF5MetadataSource::HasSpots): spots exist if /entry/MX/nPeaks is present in the master (integrated _process.h5) or the per-image data file (VDS/legacy) - the same master-first-then-source path ReadSpots uses. Rugnux reuses stored spots only when they exist, otherwise it runs spot finding (and logs it); --redo-rotation-spots already forced finding. Verified: a true no-/entry/MX rotation dataset fails on the old code and indexes (P21) on the new; all jungfraujoch files (spots in master or in data files) still reuse unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 22:56:28 +02:00
leonarski_fandClaude Opus 4.8	f737424bdf	rugnux: move rugnux_cli.cpp from tools/ to rugnux/ ... The offline analysis CLI belongs next to its Rugnux library. Move the source and the ADD_EXECUTABLE(rugnux ...) target (plus the Windows wingetopt link) into rugnux/CMakeLists.txt; the ../reader, ../common, ../image_analysis includes are unchanged (both dirs sit one level under the root) and ../rugnux/Rugnux.h becomes a local Rugnux.h. rugnux/ is added before tools/ in every build config, so viewer-only and Windows packages still get the CLI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 22:39:12 +02:00
leonarski_fandClaude Opus 4.8	b1cd49c6b5	rugnux: parallelize two-pass rotation first phase ... The first pass of two-pass rotation indexing was the slow "getting there" phase. Profiling RunIndexing showed the cost is the serial Ceres candidate refinement (up to 4 FFT candidate cells x 2 XtalOptimizer solves each), not spot finding or the cuFFT kernel. - XtalOptimizer: add a num_threads arg (default 1) so a caller running a few refinements concurrently can give each several cores. - RotationIndexer::RunIndexing: seed the candidates serially, run their up-to-8 XtalOptimizer solves concurrently (4 Ceres threads each), select serially in candidate order (identical result). ~10-13x on the candidate loop. - Rugnux: feed both first-pass schemes (spread + wedge) serially, then run their RunIndexing() passes in parallel - overlapping one scheme's cuFFT with the other's Ceres. Size the FFT indexer pool to 2 for the rotation path (it fires only twice) instead of the default 4, avoiding wasted cuFFT-plan init. First-pass wall 2.1-6.5x faster across a 12-crystal battery; whole lysoC run 17.7->10.5s. All 12 crystals bit-identical (scheme, validation count, space group). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 22:35:54 +02:00
leonarski_fandClaude Opus 4.8	c37bc2d005	viewer/curl: keep <curl/curl.h> out of JFJochHttpReader.h (MSVC build fix) ... curl.h transitively includes <windows.h>, whose min/max/ERROR/LoadImage macros clobbered every viewer TU that includes this widely-used header (std::min/max in gemmi, the TaskState::ERROR enum, the LoadImage() calls, the Qt/fftw cascade). It even broke JFJochHttpReader.cpp itself, since inside the header curl.h preceded the reader/common includes that pull in gemmi. Forward-declare the opaque easy handle (CURL is `typedef void` in libcurl) so the header no longer needs curl.h, and include curl.h only in the .cpp, last, after all std/project headers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 21:46:53 +02:00
leonarski_fandClaude Opus 4.8	b3b4f5824e	viewer/curl: link vendored libcurl_static directly, not the CURL::libcurl alias ... The viewer's CURL::libcurl was resolving to a system libcurl on the CI image (curl configured but never compiled; viewer linked the OS curl). The alias is shadowable; the concrete FetchContent target name libcurl_static is not. Link it directly -- this forces the vendored static curl into the build graph and pulls its include dirs + CURL_STATICLIB define via the target's usage requirements. Simplify the top-level guard to just assert libcurl_static exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 19:17:52 +02:00
leonarski_fandClaude Opus 4.8	3b02156fba	viewer/curl: dump CURL::libcurl type + imported path when it shadows ... The CI image has only runtime /usr/lib64/libcurl.so.4 (no libcurl-devel, no .so symlink, no CMake config, no toolchain/dependency provider), so a standard find_package(CURL)/-lcurl shadow is impossible -- whatever provides CURL::libcurl must point at a full path. Expand the guard to print ALIASED_TARGET, TYPE, IMPORTED_LOCATION and INTERFACE_LINK_LIBRARIES so a single configure run reveals exactly what CURL::libcurl resolves to before the fatal error. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 19:11:44 +02:00
leonarski_fandClaude Opus 4.8	180f93341a	viewer/curl: assert CURL::libcurl is the vendored libcurl_static, and log it ... The CI runner uses a fresh per-task workspace volume (not a reused build dir), so the earlier rm -rf build was a no-op and curl still wasn't compiled. The real symptom: on a fresh build the viewer's CURL::libcurl resolves to a system libcurl (libcurl_static exists but is never compiled/linked). The previous guard only checked that libcurl_static exists, which it does. Instead print what CURL::libcurl aliases and hard-fail unless it is the vendored libcurl_static, so a system-curl shadow surfaces in the configure log instead of silently shipping a viewer linked against the OS curl. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 19:04:49 +02:00
leonarski_fandClaude Opus 4.8	7fd2105f96	tools: add rugnux_vs_xds.py — rugnux vs XDS regression table ... Self-contained script (self-bootstrapping venv + gemmi) that runs rugnux de-novo over a directory of test crystals and tabulates its merging statistics against the XDS CORRECT.LP reference: number of reflections, observations, space group (H-M + number), R_meas / CC1/2 overall, R_meas low shell, CC1/2 high shell, ISa, and rugnux wall-clock time — with a per-crystal point-group-level space-group match (OK/DIFF). rugnux stats are read from its mmCIF output (the stable interface, not the console log); resolution limit and Friedel/anomalous setting are matched to XDS for a like-for-like comparison. Default output dir rugnux_cmp/ is ignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 18:52:35 +02:00
leonarski_fandClaude Opus 4.8	53ffb56aae	viewer/curl: clean stale CI build dir + guard against system libcurl shadowing ... Self-hosted runners reuse the workspace Docker volume, so a stale build/ dir could leave jfjoch_viewer linking a system libcurl instead of the vendored FetchContent static one (curl configured but never compiled; viewer linked the OS curl). Add a "Clean previous build" step (rm -rf build) to the two Linux viewer jobs (build-viewer-tgz, build-rpm), and assert libcurl_static exists after FetchContent_MakeAvailable(curl) so any future shadowing fails loudly instead of silently linking the OS curl with the wrong TLS/Kerberos backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 18:45:51 +02:00
leonarski_f	4fcbb09771	Dockerfile: For rocky8 one should NOT run dnf update, because it will like to update cuda to 13.3	2026-07-11 18:00:52 +02:00
leonarski_fandClaude Opus 4.8	cdcd5d28ca	viewer: reuse one libcurl handle so the HTTP reader keeps the connection alive ... JFJochHttpReader::Request previously did curl_easy_init()/curl_easy_cleanup() around every call, which destroys the handle's connection cache -- so each request reconnected from scratch (full TCP + TLS handshake, and a fresh Negotiate/Kerberos round-trip once auth is on). Keep one persistent easy handle instead: create it lazily on the first request (i.e. as soon as an address is selected via ReadURL) and reuse it, calling curl_easy_reset between requests. reset clears the previous request's options but deliberately keeps the live connection and DNS/TLS-session caches, so a same-host request reuses the open socket; if that socket has been closed (idle timeout, server session limit, broker restart) libcurl reconnects and retries on its own -- no manual reconnect logic needed. The handle is torn down on disconnect (Close), when a new address is selected (ReadURL), and in the destructor. Access is serialised by a dedicated leaf mutex, since a single easy handle must not be used from two threads at once. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:07:05 +02:00
leonarski_fandClaude Opus 4.8	007e65cdfe	viewer: accept https:// endpoints and add an http/https selector to the connect dialog ... - Route https:// URLs to the HTTP reader (LoadFile_i scheme check), not only http://. - Replace the fixed "http://" label in the Open HTTP Connection dialog with a small http/https combo box and build the URL from the chosen scheme. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 15:19:59 +02:00
leonarski_fandClaude Opus 4.8	49d078563c	viewer: move JFJochHttpReader to the viewer and switch it to libcurl ... JFJochHttpReader (the viewer's live-broker HTTP client) is the only HTTP client in the reader library and the only piece that needs TLS + Kerberos. Moving it out of the always-built JFJochReader keeps the broker/writer free of a TLS/Kerberos stack; switching it from cpp-httplib to libcurl gives the viewer HTTPS and, later, GSSAPI/Bearer auth. - Move reader/JFJochHttpReader.{cpp,h} -> viewer/, compiled into jfjoch_viewer. - Rewrite the 10 httplib call sites onto one libcurl easy-handle Request() helper (GET + PUT, binary/JSON bodies; status/transport handling preserved). - CMake: FetchContent libcurl, STATIC, HTTP(S)-only, only for viewer builds. OS-native backends keep Windows self-contained (Schannel+SSPI); Linux uses system OpenSSL + GSSAPI (krb5). Drop httplib from JFJochReader. - Block httplib OpenSSL auto-link (HTTPLIB_USE_OPENSSL_IF_AVAILABLE OFF) so the broker stops pulling in libssl/libcrypto for HTTPS it never serves. - docker: add krb5 dev headers (libkrb5-dev / krb5-devel) for CURL_USE_GSSAPI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 15:19:58 +02:00
leonarski_f	efd5f3513a	Remove upload artifact from Gitea for jfjoch_viewer on Windows (upload doesn't work)	2026-07-11 13:12:58 +02:00
leonarski_fandClaude Opus 4.8	6c9a22d7f3	docs: tilt-invariant solid-angle correction; brief changelog docs entry ... Update CPU_DATA_ANALYSIS.md 2.2 to document the azimuthal-integration solid-angle correction as cos^3(alpha) with alpha the incidence angle to the detector normal (cos(alpha) = D / sqrt(u^2+v^2+D^2)), tilt-invariant and matching PyFAI solidAngleArray / MAX IV azint, reducing to cos^3(2*theta) only for an untilted detector - matching the code change in acd20256. Shorten the over-detailed rc.158 "Docs" changelog bullet to a brief line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 13:06:39 +02:00
leonarski_f	05feb74602	VERSION: 1.0.0-rc.158	2026-07-11 11:34:11 +02:00
leonarski_fandClaude Opus 4.8	4b72a89249	harden untrusted-input size handling in TIFF read and raw-TCP frames ... Two memory-safety/robustness fixes for input whose size is attacker- or peer-controlled: - ReadTIFF: a large IMAGELENGTH could overflow scanline_bytes * lines, undersizing the buffer that TIFFReadScanline then writes past. Guard the product against overflow before resize and reuse scanline_bytes in the loop. - Raw-TCP image path: an uncapped header payload_size drove a huge resize() that took down the receive thread (wedging the writer-facing acceptor). Add JFJOCH_TCP_MAX_PAYLOAD_SIZE and reject oversized frames at the single receive-loop choke point on both the pusher and puller sides. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 11:34:11 +02:00
leonarski_fandClaude Opus 4.8	acd2025676	azint: tilt-correct solid-angle correction; honour imported rot3 in refinement ... The azimuthal-integration solid-angle correction used cos^3(2*theta), where 2*theta is the true scattering angle (from LabCoord, including detector tilt). The solid angle of a flat pixel actually depends on the incidence angle to the detector normal, cos(alpha) = det_distance / |detector-frame position|, which is invariant under detector tilt (rot1/rot2/rot3). Only for an untilted detector do the two agree. Switch CalcAzIntSolidAngleCorr(x,y) to the tilt-invariant form, matching PyFAI solidAngleArray and MAX IV azint. Drop the q-only overload (it can only ever be the untilted approximation and was used only in tests) and move its test onto the (x,y) form; add a tilt-invariance test. XtalOptimizer's residual reconstructed each spot's lab position from rot1/rot2 only, hardcoding rot3 = 0, while the rest of the pipeline (and its own spot selection) used the full PONI rotation. An imported non-zero rot3 was therefore silently dropped during refinement. Bake rot3 into the residual as a fixed Rz(-rot3) so refinement stays consistent (no-op when rot3 == 0). Polarization and azimuthal binning already honoured rot3 via the full PONI rotation (Phi_rad), validated against PyFAI chi() by the existing rot3 phi tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 11:34:11 +02:00
leonarski_fandClaude Opus 4.8	e7fbeb527f	docs: update analysis references to match current code ... Reconcile docs/CPU_DATA_ANALYSIS.md and docs/RUGNUX.md with the current image_analysis/ and rugnux CLI (verified section-by-section against the code): unified profile-fit Bragg integration engine, multi-lattice indexing, azimuthal phi binning, radial parallax/bandwidth profile with sub-pixel centring, rot3d capture-fraction handling, automatic CC1/2 resolution cutoff, and the new rugnux options; fix the section numbering and cross-references; remove the never-implemented French-Wilson and still-partiality descriptions. Delete the stale in-source design notes (ICE_RING_DETECTION, BRAGG_INTEGRATION_ENGINE, NEXTGEN_INTEGRATOR) and fix the two code comments that pointed at them; the BraggIntegrationEngine header no longer claims it is 'not yet wired'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:59:15 +02:00
leonarski_fandClaude Opus 4.8	c3468eef6c	ci: use actions/upload-artifact@v4 for the Windows/CUDA artifact ... The internal Gitea action is referenced as actions/upload-artifact@v4 (no gitea- prefix), per its own documentation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:35:00 +02:00
leonarski_fandClaude Opus 4.8	7da60c3385	ci: use gitea-upload-artifact@v4 for the Windows/CUDA artifact ... Switch to the internal Gitea artifact action (actions/gitea-upload-artifact@v4, hosted at gitea.psi.ch) instead of actions/upload-artifact@v3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:33:32 +02:00
leonarski_fandClaude Opus 4.8	54de990100	viewer: show only the high-resolution limit in the stats hero card ... Drop the low-resolution edge from the Resolution card - only the high-res limit (the CC1/2-cutoff value) is of interest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:32:23 +02:00
leonarski_fandClaude Opus 4.8	668855a9e7	viewer: show the merge resolution range in the stats hero row ... The high-resolution limit is now usually set by the automatic CC1/2 logistic cutoff (when no scaling resolution is given), so it varies per run. Add a 'Resolution [Å]' hero card (low-res .. high-res edge from the overall shell) as the first card in the merge-statistics window. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:20:34 +02:00
leonarski_fandClaude Opus 4.8	61eb661eca	ci: upload the Windows/CUDA viewer installer as a build artifact ... The Windows installer was only uploaded on tagged releases, so testing an arbitrary branch build meant cutting a tag. Also upload the CUDA installer as a workflow artifact on every run, with a short 3-day retention (it is a throwaway test build). Only the cuda variant, since that is the one we test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:14:25 +02:00
leonarski_fandClaude Opus 4.8	85cf1b04d1	docs: changelog for rc.158 viewer/rugnux/packaging changes ... Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:14:01 +02:00
leonarski_fandClaude Opus 4.8	4374babe26	viewer: add a Browse button to the processing output prefix ... The output prefix was a plain text field; add a 'Browse…' button that opens a file dialog to pick the output location. The chosen name is treated as a prefix (the _process.h5 / .mtz / .cif suffixes are appended), so a trailing extension is dropped, and the result is stored with native separators. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:06:04 +02:00
leonarski_fandClaude Opus 4.8	936d2efd4a	viewer: bundle cuFFT in the Linux .tar.gz like the Windows build ... The Windows package ships cufft64_*.dll next to the viewer so a host with only an NVIDIA driver (no CUDA toolkit) can run it; the Linux .tar.gz shipped nothing equivalent and had no rpath, so a -linux-cuda archive needed cuFFT on the system. Mirror the Windows behaviour ONLY for the self-contained .tar.gz (JFJOCH_VIEWER_ONLY): install libcufft.so (with its SONAME/version symlink chain) into bin next to the binary and set INSTALL_RPATH=$ORIGIN so the loader picks the bundled copy. The .deb/.rpm builds deliberately do NOT bundle it - CUDA there is centrally managed by the distro's packages, and the .tar.gz is the one with no package manager, where we want the dependency set really minimal. cuFFT is the only dynamically-linked CUDA component (cudart and the fast-feedback indexer are static). macOS has no CUDA so it is excluded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 10:04:56 +02:00
leonarski_fandClaude Opus 4.8	0ed943dd4d	viewer: show the space group in the merge-statistics window ... The de-novo space-group search (SearchSpaceGroup) already ran inside Rugnux but only its chosen number survived on ProcessResult, and the viewer dropped even that. Carry the structured SearchSpaceGroupResult (point group + ranked candidate scores) on ProcessResult and thread it into JFJochMergeStatsWindow, which now shows a 'Space group' hero card (the final group, searched or fixed) plus a compact table of candidate groups and their absence scores. The library no longer renders the search to text (it used to embed it in merge_statistics_text); rugnux_cli formats it for stdout instead, so the CLI keeps its text table while the viewer draws a proper table and does not spew it to stdout. Also surface a user-fixed space group on the result so the card shows it too. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 09:52:25 +02:00
leonarski_fandClaude Opus 4.8	228841870d	rugnux: honour an absolute -o output prefix ... configure_offline_output still routed the -o prefix through FilePrefix (the CheckPath guard), so 'rugnux -o /abs/path' threw 'Path cannot start with slash' before processing even though the writer path already supported it. Use the trusted setter here too. Completes the offline absolute-path support (the writer and Rugnux::Run already skip the guard); the .mtz/.cif and _process.h5 now land at an absolute -o. The --scale path shares this function and writes via WriteReflections (no FileWriter), so it is covered as well. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 09:51:03 +02:00
leonarski_fandClaude Opus 4.8	9b1ab02c41	viewer: control _process.h5 and merged .mtz/.cif output independently ... The 'Save _process.h5' checkbox drove output_prefix, so unchecking it also suppressed the merged .mtz/.cif, and there was no way to run for the ISa/R-meas numbers only. Split it into two checkboxes wired to independent flags: - Save _process.h5 -> ProcessConfig::write_process_h5 (also drives the viewer snapshot, so unchecking it means the viewer is not updated with spots/results) - Write merged .mtz/.cif -> new ProcessConfig::write_merged output_prefix is set when either is wanted; both off = process for stats only. Scaling/merge statistics (hence the ISa/R-meas window) are independent of file writing, so they show in every case. RugnuxCommandLine now emits --write-process-h5 so a copied command matches the GUI's choice. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 09:42:23 +02:00
leonarski_fandClaude Opus 4.8	343af8deed	viewer: write processing output next to the input file ... The New-job dialog seeded the output prefix from QFileInfo::completeBaseName(), which drops the directory, so _process.h5/.cif/.mtz landed in the viewer's working directory (wherever it was installed) instead of beside the dataset. Seed it with the input file's absolute directory instead; the field stays editable. Relies on the trusted output-path support so the absolute prefix is accepted by the writer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 09:40:28 +02:00
leonarski_fandClaude Opus 4.8	8858951419	rugnux: allow absolute output paths for offline processing ... The FilePrefix/CheckPath guard forbids absolute paths and '..' traversal so a remote client cannot make the broker/writer write outside their run directory. Offline processing (rugnux -o, and the viewer writing next to the input file) supplies its own trusted local path, so that guard should not apply to it. Add a trusted opt-in that leaves the broker/writer path untouched: - DatasetSettings/DiffractionExperiment::FilePrefixTrusted() sets the prefix without CheckPath (FilePrefix() = CheckPath + FilePrefixTrusted). - FileWriter gains a trusted_path ctor flag that skips its own CheckPath. - Rugnux uses both. Broker/writer never set them, so their behaviour is identical; this only widens what the offline CLI/viewer may write. Previously 'rugnux -o /abs/path' threw ('Path cannot start with slash') while building the _process.h5; the .mtz/.cif already bypassed the guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 09:39:46 +02:00
leonarski_fandClaude Opus 4.8	4292c990ea	viewer: open WSL/UNC paths by converting to native separators ... Qt's QFileDialog returns '/'-separated paths even on Windows, so a UNC path like \\wsl.localhost\Ubuntu\... arrives as //wsl.localhost/Ubuntu/... and H5Fopen (via the Win32 file layer) does not recognise the forward-slash form as UNC, while C:/... still works - exactly the reported symptom. Normalise the file-dialog result with QDir::toNativeSeparators before opening. No-op on Linux. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 09:36:53 +02:00
leonarski_f	54c0100e8e	v1.0.0-rc.157 (#67) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Rebrand the offline data-processing subsystem as `rugnux` and consolidate all offline analysis into the single `rugnux` binary - `jfjoch_process` is now `rugnux`, the former `jfjoch_azint` is now `rugnux --azint-only`, and `jfjoch_scale` is now `rugnux --scale` (see the new docs/NAMING.md and docs/RUGNUX.md). Scaling and merging are on by default for rotation and stills (`--no-merge` disables them), replacing the previous opt-in `-M, --scale-merge`. * rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling. * rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics. * rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below <I/sigma> = 1. * rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group. * rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix. * jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation. * jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning. * jfjoch_broker: Drop uncollected placeholder rows (number = -1) from the scan_result REST endpoint. * jfjoch_broker: Fix the inverted per-image compression ratio reported by the Lite receiver (was compressed/uncompressed instead of uncompressed/compressed). * jfjoch_broker: Bragg integration adds a quantization-noise variance floor with a box-sum fallback, and treats the type-maximum marker as an invalid pixel for unsigned image types. * jfjoch_writer: Detect file-overwrite conflicts at start for back-channel transports, and reset the writer when end-of-collection finalisation fails. * jfjoch_viewer: Preview overlays follow the geometry (resolution/ROI arcs, true beam centre, predictions, coral secondary-lattice spots, legend), add save-as-JPEG, and fix an HTTP live-follow memory leak. * Frontend: Improved aesthetics and usability, and added in-browser pixel-mask and JUNGFRAU-pedestal visualisation. * CI: Name the Windows installer jfjoch-viewer-* instead of jfjoch-*.Reviewed-on: #67 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.157	2026-07-11 07:19:11 +02:00
leonarski_f	d6389e12da	v1.0.0-rc.156 (#66) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output. * jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling. * Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool. * jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection. * jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata. * CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.156	2026-07-03 19:18:56 +02:00
leonarski_f	54c667190f	v1.0.0-rc.155 (#65) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Remove pixelrefine option (replaced with ProfileIntegrate2D) * jfjoch_viewer: Some graphical improvements. * jfjoch_viewer: Simplify und unify data analysis settings. * jfjoch_writer: Add TCP keepalive to increase robustness if jfjoch_broker "dies" in the middle of data acquisition. Reviewed-on: #65 1.0.0-rc.155	2026-06-25 22:01:48 +02:00
leonarski_f	6136f858af	v1.0.0-rc.154 (#64) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Fix to TCP file pusher (remove kernel zero copy to improve reliability) Reviewed-on: #64 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.154	2026-06-25 18:12:00 +02:00
leonarski_f	75e401f0e5	v1.0.0-rc.153 (#63) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing * jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files * jfjoch_broker: Add ROI calculation in non-FPGA workflow * jfjoch_broker: Fixes to TCP image pusher * jfjoch_broker: Remove NUMA bindings * jfjoch_broker: Improvements to indexing * jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values * jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors * jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs * jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results Reviewed-on: #63 1.0.0-rc.153	2026-06-23 20:29:49 +02:00
leonarski_f	c49bd2ac3b	v1.0.0-rc.152 (#62) ... * jfjoch_broker: Fix bounds for azimuthal integration for Q spacing (allow Q of 1e-5) * jfjoch_viewer: Adjust Q bounds for azimuthal integration * jfjoch_azint: Add tool to do quick azimuthal integration Reviewed-on: #62 1.0.0-rc.152	2026-06-17 20:36:24 +02:00
leonarski_f	ef52dac2ee	v1.0.0-rc.151 (#61) ... * jfjoch_broker: For PSI EIGER detector allow to disable individual half-modules by putting empty hostname Reviewed-on: #61 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.151	2026-06-16 14:13:29 +02:00
leonarski_f	90e804acd7	v1.0.0-rc.150 (#60) ... * jfjoch_broker: When in FPGA workflow (with PSI detectors) azimuthal integration might be forced to CPU - this will require more computational power, but it enables more integration bins and reports standard deviation of each bin. * jfjoch_broker: Raise error if one is in FPGA flow and there are too many azimuthal integration bins. Reviewed-on: #60 1.0.0-rc.150	2026-06-15 20:24:15 +02:00
leonarski_f	ea575f790a	v1.0.0-rc.149 (#59) ... * XDS plugin: Fix HDF5 mutex to run on multiple processors Reviewed-on: #59 1.0.0-rc.149	2026-06-13 21:27:41 +02:00
leonarski_f	cc3eb8352c	v1.0.0-rc.148 (#58) ... This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144. * jfjoch_broker: Improve azimuthal integration (add <I^2> calculation) * jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated) * jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin Reviewed-on: #58 1.0.0-rc.148	2026-06-08 08:30:35 +02:00
leonarski_f	75de40f52b	v1.0.0-rc.147 (#57) ... This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144. * jfjoch_viewer: Add reciprocal space viewer * jfjoch_process: Two pass algorithm that does spot finding/indexing + integration of full dataset * jfjoch_process: Improve logic for rotation indexer, to make execution more deterministic (still work in progress) Reviewed-on: #57 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-06-02 11:49:24 +02:00