Commit Graph
79 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 5.5 a624429090 Mol* viewer: in line with AareDB's StructureView at its current head (8ece2c2)
AareDB's main is still at 8ece2c2, the commit the setup was copied from, so the structure/ files
match it line for line (bar the black stage, mmCIF input and the anomalous layer). What differed was
around them:
- molstar pinned to 5.11.0, AareDB's exact pin, instead of 5.13.0: the renderer's defaults
  (marking, outlines, occlusion, trackball) come with the version, so the same version draws the same.
- Only the first load frames the molecule: a newer result keeps the user's camera and clip slab, as
  AareDB keeps them across result switches.
- Load states as in AareDB: "loading model..." centred on the stage until the model is in, then
  "computing density maps..." in the panel with the map controls disabled until the maps are ready.
- Map rows labelled "2Fo-Fc" and "Fo-Fc (+/-)", AareDB's names.
- createPlugin.ts carries the credit line for the setup; its comment's directory path is ours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
2026-10-22 20:24:42 +02:00
leonarski_f 2df1d8a2ad Mol* viewer: aaredb's structure-viewer setup, on a black stage
The owner's verdict on the screenshots: aaredb (gitea.psi.ch/mx/aaredb, the PSI MX group's own
structure viewer) has it right, so its Mol* setup is taken over, files copied from
frontend/src/components/Structure at 8ece2c2 into frontend/src/components/structure:
- createPlugin.ts: headless DefaultPluginSpec plugin, manual camera reset; background black (owner)
  instead of aaredb's 0x0d1a26.
- MolstarCanvas.tsx: fresh canvas per mount, one-tick StrictMode deferral, resize, WebGL context-loss
  recovery, the Coot control profile wired.
- cootControls.ts / cootTrackball.ts: Coot mouse language - left drag rotates (Coot 0.9.8 trackball),
  right drag zooms, middle drag pans, middle click recentres on a residue, scroll clips the slab,
  shift+scroll zooms (camera sync and residue pick left out: no split view here).
- loadModel.ts: cartoon coloured by chain from a palette clear of the map colours, polymer sticks and
  waters built hidden, ligands as ball-and-stick; display modes cartoon / sticks / both and a waters
  switch; reads mmCIF too, and gives the atom coordinates for carving.
- loadMaps.ts: isosurface wireframe meshes at ABSOLUTE levels from the full cell's mean and RMS,
  2Fo-Fc steel blue, Fo-Fc green/red at +/-sigma, not pickable; plus the anomalous map as a magenta
  +sigma mesh in the same style.
- densityCrop.ts: the density worker's crop to the model's box + 6 A, the 4.5 A carve of the 2Fo-Fc map
  around the atoms (never a difference map) and the CCP4 encoding, applied to the broker's one-cell map.
The panel and defaults follow aaredb's StructureView: map rows with switch and sigma slider (2Fo-Fc 1.0,
0.5-4 by 0.1; Fo-Fc 3.0, 1-8 by 0.5; anomalous 4.0, 1-8 by 0.5), debounced sigma updates, model mode
buttons, a controls legend. This replaces the camera-following 10 A map sphere: aaredb's equivalent is
the carved map with the scroll-driven clip slab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 5763b619fe4ec9999ac790960a5eb033f511c546)
2026-10-22 20:18:51 +02:00
leonarski_f 543d7548c7 Mol* viewer: open with the maps drawn within 10 A of the centre of view
The owner's choice: the viewer opens Coot-style, the maps within a 10 A sphere around the centre of view;
radius 0 on the slider still draws the whole cell.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 63121f35dd008e606f0fbb336b8ee0c330cde370)
2026-10-22 20:18:51 +02:00
leonarski_f 6abc44710d Anomalous difference map: ANOM/PHANOM in _maps.mtz, /rugnux/map.ccp4?type=anom, a magenta mesh in the viewer
Model validation already made the anomalous difference map's coefficients ((F(+) - F(-)) on the model
phases less 90 degrees, where the merge kept the Bijvoet pairs) to read the anomalous sites, then threw
them away. They now go into the map coefficients as ANOM/PHANOM (phenix.maps' labels), missing on the
reflections without a pair, so _maps.mtz opens as an anomalous map in Coot. Nothing in scaling, merging
or the validation's decisions changes; a merge without the Bijvoet split writes the twelve columns as before.

The broker serves the map as /rugnux/map.ccp4?type=anom (404 without Bijvoet pairs), and
rugnux_live_status.anomalous_map says it is there. The Mol* viewer loads it only then, as a magenta mesh
at +4 sigma with its own level slider and switch.

Tests: ANOM/PHANOM appear with Bijvoet pairs (as many as the reported pairs) and not without; the served
anomalous map of a point peaks at it; the auth test covers type=anom. End to end (myoglobin 2M replay):
anomalous_map true, the served map equals gemmi's FFT of ANOM/PHANOM in the written _maps.mtz to 3.4e-6 rms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit c4d6ad74c706181cbb183cfcaa2481653cd7cad6)
2026-10-22 20:18:51 +02:00
leonarski_f ee016d3d13 rugnux live maps: one unit cell centred on the model; the viewer opens on all of it
/rugnux/map.ccp4 now serves one unit cell of map, centred on the model, instead of the model's box plus
5 A; along an axis where the model reaches further than the cell (a predicted model with long termini
in a small cell), the model plus 5 A instead, so that all of it has density. The Mol* viewer opens on
the whole cell and the model (map radius 0); a radius still draws the maps only around the centre of
view, Coot-style.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 1c1e1d96359f112d6820d972005e9f7eca4ebd50)
2026-10-22 20:18:51 +02:00
leonarski_f a105c67f62 frontend: Mol* viewer of the placed model and its density on the Rugnux live page
When the run placed a model, the Rugnux live page shows it in a Mol* viewer (molstar 5.13.0, pinned,
loaded lazily in its own chunk): cartoon, ligands and ions as ball-and-stick, the 2mFo-DFc map as a
blue mesh (1.0 sigma) and the mFo-DFc map in green/red (+/-3 sigma), each with a contour-level slider
and a show/hide switch. As in Coot, the maps are drawn within a sphere (10 A by default, adjustable,
0 = the whole box) around the centre of view, which follows the camera. Sigma levels are relative to
the whole-cell statistics the broker writes into the map headers. Reloaded for every new result;
the plugin is disposed with the component.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 08295be10c1d7a76bc7ecdf5df46fd7a3e1febac)
2026-10-22 20:18:50 +02:00
leonarski_fandClaude Opus 5.5 c2e5e4a4e8 broker: initialize clears the image buffer, turns live rugnux off and removes its model
Initialize is the fresh start between users, so nothing of the previous one's session is left:
the image buffer is cleared, live rugnux processing is disabled and the uploaded MR model is
dropped. It is done at the start of Initialize, under m then rugnux_mutex (the order the other
setters take), so it happens even if configuring the detector then fails. The receiver is always
Idle when the state machine is not running, so the buffer clear has no unmet precondition.

The dashboard's rugnux panel read the settings only once, on mount, so after an initialize from
any client it kept showing the previous run; it now polls them like the Rugnux tab does.

The /initialize description in jfjoch_api.yaml says so; the frontend client is regenerated
(description only). The python client docs and redoc follow at the release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
2026-10-19 22:11:51 +02:00
leonarski_fandClaude Opus 5.5 831b05aebd Live rugnux panel: the timeline spans the collection's set-up length from the first poll
The broker's rugnux live status carries expected_collection_s = images x image time
(DiffractionExperiment::GetImageTime, summation included), set at START. The frontend's
timeline uses it as the expected end until a frame rate is measured, and as the axis floor
while collecting, so the axis no longer starts at 10 s and rescales as images arrive. An
older broker without the field keeps the 10 s floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
2026-10-19 20:48:07 +02:00
leonarski_fandClaude Opus 5.5 7349832309 Live rugnux timing: from the puller's receive time, and from the first image
The live panel showed ~80 Hz on a 100 Hz collection. Its clock started when
the receiver had applied and forwarded the START message, so the START ->
first image gap counted as collection time; the images were stamped when a
forward thread dequeued them, and the last image only after END and the
thread joins; and N images were divided by a span holding N-1 intervals.

- ImagePullerOutput carries `received`, a steady_clock time set by the TCP
  and ZeroMQ pullers as the message comes off the socket, before it is
  queued or deserialised (TestImagePuller stamps it on Put when unset).
- RugnuxLiveStatus takes those times: START's for its origin, each image's
  for the first and last image (min/max, as forward threads report out of
  order). last_image_s is the last data image's puller time.
- Zero of every reported time (elapsed_s, steps' started_s, probe_s,
  last_image_s, cut_s) is now the FIRST IMAGE, as the frontend already
  said; before the first image arrives "now" is 0 and earlier steps are
  negative. The START -> first image time is recorded on its own as
  start_to_first_image_s (snapshot, OpenAPI rugnux_live_status, regenerated
  C++ model and frontend client, OpenAPIConvert).
- The frontend rate is (images_received - 1) / (last - first image); the
  panel headers show "START -> first image X.X s".
- The broker logs "rugnux live: START -> first image ..., N images in T s
  from the first to the last (R Hz)"; "done X s after the last image" and
  last_image_to_written_s run from the puller time of the last image.

The deadline timer is unchanged (it still starts once the forward threads
have joined). No analysis result changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
2026-10-19 19:35:54 +02:00
leonarski_f 8261b7354e v1.0.0-rc.176 (#86)
Build Packages / Create release (push) Successful in 9s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 27m58s
Build Packages / Build documentation (push) Successful in 1m32s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 27m20s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 29m27s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 35m56s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m8s
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m43s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 20m56s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 18m23s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 19m55s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 34m46s
Build Packages / Unit tests (push) Successful in 2h8m44s
Build Packages / Validation Battery (push) Successful in 33m35s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m12s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 22m4s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 26m57s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 28m24s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 29m29s
Build Packages / Generate python client (push) Successful in 32s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 32m41s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 37m25s
* jfjoch_broker: Optional live Rugnux analysis of DECTRIS rotation collections inside the broker, with merged data, maps and the placed model written beside the master file, the rotation lattice in the scan result, and a "Rugnux live" tab and dashboard row in the web frontend.
* Rugnux: Faster processing, with molecular replacement and model validation spread over several GPUs and run on the Apple GPU on macOS.
* Rugnux: The polarization stated in miniCBF and d*TREK headers is used, twinned crystals' space groups are decided more reliably, and a run that finds no lattice still reports every image.

Reviewed-on: #86
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-10-19 07:03:28 +02:00
leonarski_f 84228bf8be v1.0.0-rc.173 (#83)
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 8m27s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 13m39s
Build Packages / build:viewer:windows-x86_64:nocuda (push) Successful in 18m37s
Build Packages / build:viewer:windows-x86_64:cuda (push) Successful in 24m11s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 25m30s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rugnux:windows-x86_64:cuda (push) Successful in 14m33s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 17m55s
Build Packages / Create release (push) Successful in 24s
Build Packages / build:rugnux:macos-arm64:nocuda (push) Successful in 2m43s
Build Packages / build:rugnux:linux-x86_64:cuda (push) Successful in 9m53s
Build Packages / build:viewer:linux-x86_64:nocuda (push) Successful in 9m58s
Build Packages / build:viewer:linux-x86_64:cuda (push) Successful in 11m22s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 16m32s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 19m3s
Build Packages / Build documentation (push) Successful in 1m16s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 21m0s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 18m38s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 20m50s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 18m38s
Build Packages / Unit tests (push) Successful in 1h46m14s
Build Packages / build:viewer:macos-arm64:nocuda (push) Successful in 3m29s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 20m23s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 19m41s
* jfjoch_broker: Optional per-dataset authentication - statistics, images and plots can require a bearer token, which jfjoch_viewer supports.
* jfjoch_viewer: Dark mode and a theme-matched colour scheme, a magnifier panel, and simpler contrast and background controls.
* Rugnux: Multiple performance improvements on GPU and CPU (CPU-only processing up to 40% faster, faster image decoding on ARM), with unchanged results.
* Rugnux: `--model` rigid-body refinement runs on the GPU, and the model-validation check is faster and more reliable.
* Rugnux: Improved scaling and merging - error model, outlier rejection, absorption correction and French-Wilson amplitudes now agree more closely with XDS and ctruncate.
* Rugnux: Improved integration - radial background on powder and ice rings, crowded rotation data keep their reflections, and CPU-only builds integrate large unit cells as GPU builds do.
* Rugnux: More robust detector geometry - measured beam centre, X-ray bandwidth and goniometer rate, and geometry refinement accepted only on significant evidence.
* Rugnux: Merged files are written in the standard setting, or in the setting of a reference MTZ, structure-factor mmCIF or model, with its free-R flags.
* Rugnux: Richer report - ice and powder rings, further lattices, superstructure candidates and mosaicity, with warnings worded as prompts to check.
* Rugnux: Clear error messages when a data set needs more GPU or host memory than is available.

Reviewed-on: #83
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-09-29 15:57:32 +02:00
leonarski_f a39fd29f77 v1.0.0-rc.167 (#77)
Build Packages / build:viewer-tgz:cpu (push) Successful in 15m56s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 14m10s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 7m14s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m13s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 19m17s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 21m21s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 17m26s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 20m48s
Build Packages / build:rpm (rocky9) (push) Successful in 20m38s
Build Packages / XDS test (durin plugin) (push) Successful in 10m43s
Build Packages / Build documentation (push) Successful in 1m5s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 17m46s
Build Packages / build:windows:cuda (push) Successful in 20m20s
Build Packages / build:viewer-tgz:cuda (push) Successful in 17m57s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m56s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m58s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m57s
Build Packages / DIALS test (push) Successful in 18m40s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m4s
Build Packages / Unit tests (push) Skipped
Build Packages / build:rugnux:windows (push) Successful in 11m12s
Build Packages / build:rpm (rocky8) (push) Successful in 23m43s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 24m57s
* `rugnux --model` reports CC(model, data) - the correlation of the merged intensities with the placed, scaled model - by resolution shell, on the same shells as CC1/2, with the reflection count and a significance for each.
* `rugnux --model` fits the model's scale, anisotropic B and bulk-solvent parameters on the working reflections only, so the R-free it reports is measured against a model no free reflection helped scale.
* The bulk-solvent parameters of `rugnux --model` are searched over their physically meaningful range instead of being fitted without bounds, so a model is never scaled with a solvent term that has silently switched itself off.
* The rigid-body placement of `rugnux --model` uses the same bounded bulk solvent as the reported fit, so a model is no longer placed against a target carrying a solvent term with no physical meaning.
* `rugnux --model` puts the model into the data's own description of the lattice before placing it, so a model whose cell is written on other axes - I-centred where the run indexed C-centred, a different unique axis, a permuted orthorhombic cell - is placed rather than scored where it was read; `MODEL_CHANGE_OF_BASIS=` and `MODEL_SETTING_AS_READ=` report it when it happens.
* The rugnux results report opens with a summary - `VERDICT=` (`OK`, `WARNINGS`, `UNUSABLE`, `FAILED`), `VERDICT_TEXT=`, `PATHOLOGY_FLAGS=` with one closed-vocabulary code per condition that warned, and the `WARNING:` lines, which used to close the file - and the sections after it are renumbered 1-5 with no gaps.
* `rugnux --developer` writes the full results report - the pipeline-internal keys and the long explanations the default report now leaves out - and `--finalist-ledger` adds the evidence for every space group the search considered, not only the one it adopted.
* The results report warns when the merged data carry no usable signal and when too little of reciprocal space was measured inside the fitted resolution, and omits `FITTED_RESOLUTION` where the CC1/2 curve it is fitted on never falls off.
* rugnux detects translational pseudo-symmetry and reports it under the `PSEUDO_TRANSLATION` flag as `TNCS_DETECTED=` and the `TNCS_*` keys - a translation the merged data are exactly invariant under is reported as `UNDECLARED_LATTICE_TRANSLATION=` under `LATTICE_TRANSLATION` instead - and a detected pseudo-translation can no longer buy a false screw axis in the space-group search or hide a twin from the L-test (`L_TEST_VS_TNCS=`).
* The space-group search determines glide planes from zonal systematic absences, so a non-Sohncke space group such as P 2_1/c or Pbca is named where the run previously stopped at its Sohncke subgroup; `SOHNCKE_SPACE_GROUP=` carries the best Sohncke group beside it on every run that searched, and a centre of symmetry is never claimed.
* Where the cell metric carries more rotational symmetry than the Bravais class the indexer named, the extra rotations are put to the intensities and the space-group search is asked again on the metric's own cell - adopted only where the intensities confirm the higher symmetry - so a lattice that is nearly but not exactly hexagonal, or whose reduction landed in a sub-cell, still reaches its true point group.
* Systematic-absence calls rest on the evidence rather than on counts: a screw axis whose absent class the data show extinct is no longer refused because a handful of reflections in it read as present, and `SPACE_GROUP_ALTERNATIVES=` no longer drops a candidate that differs only on a zone the sweep never measured.
* A reference correlation measured on too few reflections is refused instead of scored zero, so a run given a reference MTZ is no longer reindexed on an operator that mapped almost everything outside the reference's coverage.
* A frame counts as indexed from 6 spots on its lattice rather than 9, so a weakly diffracting crystal whose frames cannot carry 9 is no longer refused the lattice it fits; `--min-indexed-spots` overrides it.
* `-C` accepts a known cell in any equivalent description - conventional or primitive, centred or not - instead of only the reduced primitive form, so a centred cell given the way it is published no longer makes the run report that it found no lattice.
* Each reflection is corrected for the sensor's quantum efficiency at the angle it meets the detector (attenuation lengths from the NIST tables, which also fixes the spot-width parallax term on CdTe) and for the attenuation of the flight path between the sample and its pixel; `--flight-path air|helium|vacuum` declares the medium - default air, since no file states it - and the report says what was assumed and what it was worth. The unmerged MTZ records the factors in new `QE` and `FLIGHT` columns beside `LP`, so raw counts are `I / LP * QE * FLIGHT`, and `_process.h5` in new optional `qe` and `flight` datasets.
* Rotation geometry post-refinement fits the crystal and the detector at once, against the observed spot positions and the observed rocking angles together, so the refined distance depends far less on how wrong the file's distance was.
* A coarsely sliced sweep integrates correctly: partials are joined into one rocking event by angle rather than by frame count, so two crossings of the Ewald sphere are no longer summed into one full, and at 0.5 degrees per image or coarser the per-frame geometry refinement accepts a spot whose miss the exposure's own rotation accounts for.
* `rugnux --mode scale` reports the detector tilt and direct beam of the geometry it re-scaled at, instead of zeros that read as a flat detector, and no longer warns that no image was indexed on a run whose lattice came from its input file.
* Every rotation run that determined a space group and merged reports what the mounting cost: `SPINDLE_LOST_UNIQUE_FRACTION=` is the fraction (0-1) of unique reflections the mounting made unmeasurable under the measured point group, also written to the master as `/entry/MX/spindleLostUniqueFraction` and what the mounting warning fires on; `SPINDLE_SYMMETRY_AXIS_ANGLE_DEG=` / `SPINDLE_SYMMETRY_AXIS_ORDER=` describe the mounting in the `--developer` report.
* Stills and grid scans carry a per-image `spindle_blind_fraction` - how much of a rotation sweep's blind cone this orientation would make unrecoverable, 0.5 and above calling for a second orientation - through the CBOR stream, HDF5 (`/entry/MX/spindleBlindFraction`), the plot and scan-result APIs, and the viewer and frontend plots; an absent value means the frame could not be assessed and is not a 0.
* The results report's `REPORT_VERSION` is 7.

Reviewed-on: #77
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-09-09 07:25:13 +02:00
leonarski_f 538f3504d3 v1.0.0.rc-161 (#71)
Build Packages / Unit tests (push) Skipped
Build Packages / build:viewer-tgz:cuda (push) Successful in 17m26s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m17s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 23m17s
Build Packages / build:rpm (rocky8) (push) Successful in 24m34s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 23m33s
Build Packages / DIALS test (push) Successful in 18m23s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 10m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m2s
Build Packages / Generate python client (push) Successful in 49s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:cuda (push) Successful in 29m45s
Build Packages / build:windows:nocuda (push) Successful in 20m4s
Build Packages / build:viewer-tgz:cpu (push) Successful in 16m5s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 27m46s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 28m11s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 19m30s
Build Packages / build:rpm (rocky9) (push) Successful in 21m30s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m18s
Build Packages / XDS test (durin plugin) (push) Successful in 11m30s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 26m13s
Build Packages / Build documentation (push) Successful in 1m21s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* **rugnux: significantly better quality of results, and faster.** A large rework of integration, scaling, merging, geometry refinement and space-group determination, together with measurements the program previously made no attempt at - the direct beam before indexing, the beam stop, the goniometer rotation scale, and the stretches of a sweep the crystal did not deliver. A rotation dataset typically gains observations at better <I/sigma> and R_meas, and every `mx` and `scale` run writes a `<prefix>_report.txt` results report modelled on XDS's `CORRECT.LP`. Many defaults moved with it: spot detection is self-calibrating, beam-stop detection and rotation geometry post-refinement are on, resolution limits default to as far as the detector reaches, and ice-ring handling engages only where the crystal is measured to have ice.
* **jfjoch_viewer:** the beam-stop shadow, the detector calibration and the beam-centre measurement are reachable from "Analyze dataset"; the settings panel reports how the sample moved and how polarized the beam was; image rendering and interaction are faster.
* **Performance:** bitshuffle+LZ4 images are decoded on the GPU rather than on the host, with the bitshuffle inverse fused into preprocessing so the decompressed frame is never held in device memory.
* **Broker, writer, packaging and build:** image-slot lifetime and locking fixes, per-image datasets sized by the images actually written, the Debian/Ubuntu broker package renamed to `jfjoch`, and `image_analysis` compiling under MSVC again.

**Breaking change to the rugnux command line:**
* `--azint-only` and `--scale` are **removed**, replaced by `--mode azint` and `--mode scale`; the full pipeline is `--mode mx` and remains the default. A script passing the old flags now fails with the list of valid modes rather than silently running the wrong one.
* `-t`/`--stride` is **refused on rotation data**: skipping frames cuts every reflection's rocking curve, so the combined fulls and their partiality would be measured over frames the sweep never recorded. Select a contiguous range with `-s`/`-e` instead. `--mode azint` and `--force-still` still take a stride.

**Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional:
* `image_scale_b` is removed from the `plot_type` enum, so a client requesting that plot now gets an error rather than a curve.
* `azim_int_settings.high_q_recipA`, `spot_finding_settings.high_resolution_limit` and `spot_finding_settings.low_resolution_limit` are no longer `required`. All three mean "no limit at that end" when unset and are omitted from the response instead of carrying a placeholder value, which raises in a client generated from an rc.160-or-earlier spec. A value of 0 is still accepted and means the same thing.

**Breaking changes to the stored formats** - a consumer reading these fields must treat them as optional:
* The per-image image-scale B factor is no longer computed, so `/entry/MX/imageScaleBFactor` is absent from newly written HDF5 files and the corresponding key is absent from the CBOR DataMessage and END blocks. Files written by rc.160 and earlier still contain it and still open; nothing in the pipeline reads it any more.
* `_reflns.jfjoch_diffrn_ISa` now carries the whole-range `1/sqrt(a*b)` that XDS's ISa denotes, and the error-model `a` and `b` are reported in XDS's convention; the strong-reflection asymptote moves to `_reflns.jfjoch_diffrn_ISa_asymptotic`. **A file written by an earlier version carries the asymptote under the plain `ISa` name.**

Reviewed-on: #71
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-13 17:03:10 +02:00
leonarski_f 67dca388bd v1.0.0-rc.160 (#70)
Build Packages / Unit tests (push) Skipped
Build Packages / build:viewer-tgz:cpu (push) Successful in 6m11s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m40s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m5s
Build Packages / DIALS test (push) Successful in 13m22s
Build Packages / XDS test (durin plugin) (push) Successful in 9m2s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m55s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m57s
Build Packages / Generate python client (push) Successful in 23s
Build Packages / Build documentation (push) Successful in 57s
Build Packages / build:windows:nocuda (push) Successful in 10m24s
Build Packages / build:windows:cuda (push) Successful in 18m44s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m4s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m23s
Build Packages / build:rpm (rocky8) (push) Successful in 11m30s
Build Packages / build:rpm (rocky9) (push) Successful in 12m51s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m21s
Build Packages / Create release (push) Skipped
Build Packages / build:viewer-tgz:cuda (push) Successful in 6m54s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m41s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m10s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m8s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell.
* rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged.
* rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set.
* rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme.
* rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free.
* rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry.
* Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md.
* Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #70

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-19 09:39:28 +02:00
leonarski_f dd0bffb283 v1.0.0-rc.159 (#69)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m54s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m25s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m19s
Build Packages / build:rpm (rocky8) (push) Successful in 12m23s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m30s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m55s
Build Packages / Build documentation (push) Successful in 52s
Build Packages / build:viewer-tgz:cpu (push) Successful in 5m29s
Build Packages / build:windows:nocuda (push) Successful in 11m6s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m27s
Build Packages / XDS test (durin plugin) (push) Successful in 9m26s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m41s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m12s
Build Packages / Create release (push) Skipped
Build Packages / build:viewer-tgz:cuda (push) Successful in 6m12s
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m33s
Build Packages / build:rpm (rocky9) (push) Successful in 13m21s
Build Packages / DIALS test (push) Successful in 13m42s
Build Packages / Generate python client (push) Successful in 19s
Build Packages / build:windows:cuda (push) Successful in 18m36s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell.
* rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged.
* rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set.
* rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme.
* rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free.
* rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry.
* Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md.
* Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #69

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-13 13:54:03 +02:00
leonarski_f 451310f43d v1.0.0-rc.158 (#68)
Build Packages / build:windows:cuda (push) Successful in 18m0s
Build Packages / build:viewer-tgz:cuda (push) Successful in 8m55s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m11s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m57s
Build Packages / build:rpm (rocky9) (push) Successful in 14m0s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m38s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m34s
Build Packages / Generate python client (push) Successful in 28s
Build Packages / build:windows:nocuda (push) Successful in 9m55s
Build Packages / Unit tests (push) Successful in 1h32m35s
Build Packages / build:viewer-tgz:cpu (push) Successful in 7m37s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m13s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m45s
Build Packages / build:rpm (rocky8) (push) Successful in 11m39s
Build Packages / DIALS test (push) Successful in 14m55s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m7s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m23s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 13m42s
Build Packages / XDS test (durin plugin) (push) Successful in 7m11s
Build Packages / Create release (push) Skipped
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* Analysis: The azimuthal-integration solid-angle correction now follows the incidence angle to the detector normal (`cos^3` of that angle) instead of `cos^3(2*theta)`, so it is correct for a tilted detector and matches PyFAI `solidAngleArray` and MAX IV azint (unchanged for an untilted detector). Crystal geometry refinement (`XtalOptimizer`) no longer silently ignores an imported PONI `rot3` (rotation about the beam): it is applied as a fixed rotation in the residual so refinement stays consistent with the rest of the pipeline. Polarization and azimuthal binning already honoured `rot3` through the full PONI rotation.
* jfjoch_viewer: Open datasets on the WSL2/UNC filesystem (paths starting `\\`); write processing outputs next to the input file, with a Browse button and independent `_process.h5` / merged `.mtz`/`.cif` toggles; and show the determined space group in the merge-statistics window.
* rugnux: Accept an absolute `-o` output prefix in offline processing.
* Packaging: The self-contained Linux viewer `.tgz` now bundles cuFFT, so it runs without a system CUDA toolkit (`.deb`/`.rpm` are unchanged, distro-managed).
* Docs: Bring the analysis references up to date with the code. `docs/CPU_DATA_ANALYSIS.md` now reflects the unified profile-fit Bragg integration engine, multi-lattice indexing, azimuthal phi binning, the radial parallax/bandwidth profile with sub-pixel centring, the rot3d capture-fraction handling and the automatic CC1/2 resolution cutoff, and drops the descriptions of features that were never implemented (French-Wilson amplitudes, the still excitation-error partiality model); `docs/RUGNUX.md` documents the new `--resolution-cutoff`/`--resolution-cc-target`/`--resolution-shells`, `--min-captured-fraction`, `--mosaicity`, `--reference-column`, the azimuthal correction toggles and the geometry-override options, and corrects the `-N` default. The outdated in-source design notes (ICE_RING_DETECTION, BRAGG_INTEGRATION_ENGINE, NEXTGEN_INTEGRATOR) are removed.Reviewed-on: #68

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-12 19:42:29 +02:00
leonarski_f 54c0100e8e v1.0.0-rc.157 (#67)
Build Packages / build:windows:nocuda (push) Successful in 14m45s
Build Packages / build:viewer-tgz:cpu (push) Successful in 6m47s
Build Packages / build:viewer-tgz:cuda (push) Successful in 7m22s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m52s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m4s
Build Packages / Build documentation (push) Successful in 1m4s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m16s
Build Packages / build:windows:cuda (push) Successful in 13m13s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m16s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m50s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m40s
Build Packages / build:rpm (rocky9) (push) Successful in 12m4s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m55s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m22s
Build Packages / DIALS test (push) Successful in 13m37s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m45s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Create release (push) Skipped
Build Packages / Unit tests (push) Successful in 1h28m28s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m19s
Build Packages / build:rpm (rocky8) (push) Successful in 11m18s
Build Packages / XDS test (durin plugin) (push) Successful in 8m47s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* rugnux: Rebrand the offline data-processing subsystem as `rugnux` and consolidate all offline analysis into the single `rugnux` binary - `jfjoch_process` is now `rugnux`, the former `jfjoch_azint` is now `rugnux --azint-only`, and `jfjoch_scale` is now `rugnux --scale` (see the new docs/NAMING.md and docs/RUGNUX.md). Scaling and merging are on by default for rotation and stills (`--no-merge` disables them), replacing the previous opt-in `-M, --scale-merge`.
* rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling.
* rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics.
* rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below <I/sigma> = 1.
* rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group.
* rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix.
* jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation.
* jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning.
* jfjoch_broker: Drop uncollected placeholder rows (number = -1) from the scan_result REST endpoint.
* jfjoch_broker: Fix the inverted per-image compression ratio reported by the Lite receiver (was compressed/uncompressed instead of uncompressed/compressed).
* jfjoch_broker: Bragg integration adds a quantization-noise variance floor with a box-sum fallback, and treats the type-maximum marker as an invalid pixel for unsigned image types.
* jfjoch_writer: Detect file-overwrite conflicts at start for back-channel transports, and reset the writer when end-of-collection finalisation fails.
* jfjoch_viewer: Preview overlays follow the geometry (resolution/ROI arcs, true beam centre, predictions, coral secondary-lattice spots, legend), add save-as-JPEG, and fix an HTTP live-follow memory leak.
* Frontend: Improved aesthetics and usability, and added in-browser pixel-mask and JUNGFRAU-pedestal visualisation.
* CI: Name the Windows installer jfjoch-viewer-* instead of jfjoch-*.Reviewed-on: #67

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-11 07:19:11 +02:00
leonarski_f d6389e12da v1.0.0-rc.156 (#66)
Build Packages / build:windows:nocuda (push) Successful in 15m31s
Build Packages / build:viewer-tgz:cpu (push) Successful in 5m46s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m33s
Build Packages / build:rpm (rocky8) (push) Successful in 10m32s
Build Packages / build:rpm (rocky9) (push) Successful in 12m23s
Build Packages / build:windows:cuda (push) Successful in 12m37s
Build Packages / build:viewer-tgz:cuda (push) Successful in 6m9s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m41s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m26s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m12s
Build Packages / Generate python client (push) Successful in 27s
Build Packages / Create release (push) Skipped
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m25s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m21s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m18s
Build Packages / DIALS test (push) Successful in 12m6s
Build Packages / XDS test (durin plugin) (push) Successful in 8m15s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m12s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m35s
Build Packages / Build documentation (push) Successful in 54s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output.
* jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling.
* Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool.
* jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection.
* jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata.
* CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-03 19:18:56 +02:00
leonarski_f 75e401f0e5 v1.0.0-rc.153 (#63)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m43s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m27s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m27s
Build Packages / build:rpm (rocky8) (push) Successful in 9m20s
Build Packages / build:rpm (rocky9) (push) Successful in 10m50s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m54s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m38s
Build Packages / DIALS test (push) Successful in 12m13s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m8s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m50s
Build Packages / Create release (push) Skipped
Build Packages / Unit tests (push) Successful in 1h31m59s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m5s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m24s
Build Packages / XDS test (durin plugin) (push) Successful in 7m8s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / Build documentation (push) Successful in 50s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing
* jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files
* jfjoch_broker: Add ROI calculation in non-FPGA workflow
* jfjoch_broker: Fixes to TCP image pusher
* jfjoch_broker: Remove NUMA bindings
* jfjoch_broker: Improvements to indexing
* jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values
* jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors
* jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs
* jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results

Reviewed-on: #63
2026-06-23 20:29:49 +02:00
leonarski_f c49bd2ac3b v1.0.0-rc.152 (#62)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m30s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m52s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m40s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m24s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m2s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m53s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m38s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m30s
Build Packages / build:rpm (rocky8) (push) Successful in 10m47s
Build Packages / build:rpm (rocky9) (push) Successful in 11m48s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m44s
Build Packages / DIALS test (push) Successful in 12m59s
Build Packages / XDS test (durin plugin) (push) Successful in 8m33s
Build Packages / Unit tests (push) Successful in 1h37m1s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m4s
Build Packages / Build documentation (push) Successful in 57s
Build Packages / Create release (push) Skipped
* jfjoch_broker: Fix bounds for azimuthal integration for Q spacing (allow Q of 1e-5)
* jfjoch_viewer: Adjust Q bounds for azimuthal integration
* jfjoch_azint: Add tool to do quick azimuthal integration

Reviewed-on: #62
2026-06-17 20:36:24 +02:00
leonarski_f 90e804acd7 v1.0.0-rc.150 (#60)
Build Packages / DIALS test (push) Successful in 29m45s
Build Packages / Generate python client (push) Successful in 28s
Build Packages / Build documentation (push) Successful in 1m25s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m9s
Build Packages / build:rpm (rocky8) (push) Successful in 13m24s
Build Packages / Unit tests (push) Successful in 42m49s
Build Packages / XDS test (durin plugin) (push) Successful in 19m27s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 18m52s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m53s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m7s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m11s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m15s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m14s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m5s
Build Packages / XDS test (neggia plugin) (push) Successful in 13m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m49s
Build Packages / build:rpm (rocky9) (push) Successful in 14m30s
* jfjoch_broker: When in FPGA workflow (with PSI detectors) azimuthal integration might be forced to CPU - this will require more computational power, but it enables more integration bins and reports standard deviation of each bin.
* jfjoch_broker: Raise error if one is in FPGA flow and there are too many azimuthal integration bins.

Reviewed-on: #60
2026-06-15 20:24:15 +02:00
leonarski_f cc3eb8352c v1.0.0-rc.148 (#58)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m39s
Build Packages / build:rpm (rocky8) (push) Successful in 11m43s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m59s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m48s
Build Packages / build:rpm (rocky9) (push) Successful in 12m32s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m50s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m40s
Build Packages / DIALS test (push) Successful in 11m19s
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m28s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m9s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 7m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m47s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m24s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin

Reviewed-on: #58
2026-06-08 08:30:35 +02:00
leonarski_f fc68a9baed v1.0.0-rc.146 (#56)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m34s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m16s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m15s
Build Packages / XDS test (durin plugin) (push) Successful in 7m16s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 47s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m46s
Build Packages / XDS test (neggia plugin) (push) Successful in 1h59m18s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m49s
Build Packages / build:rpm (rocky9) (push) Successful in 10m12s
Build Packages / DIALS test (push) Successful in 10m18s
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m32s
Build Packages / build:rpm (rocky8) (push) Successful in 1h41m2s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.

WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.

Reviewed-on: #56
2026-05-28 18:48:35 +02:00
leonarski_f d760b12a18 v1.0.0-rc.141 (#51)
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m23s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / DIALS test (push) Successful in 14m1s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m32s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m19s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 9m15s
Build Packages / build:rpm (rocky9) (push) Successful in 12m14s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m28s
Build Packages / XDS test (durin plugin) (push) Successful in 8m58s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m7s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m24s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m49s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m44s
Build Packages / Generate python client (push) Successful in 21s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m26s
Build Packages / Unit tests (push) Failing after 1h20m14s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Azimuthal integration mapping is generated with parallel computations, significantly reducing setup times
* frontend: Fix selection of FFTW in indexing settings

Reviewed-on: #51
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-30 13:04:54 +02:00
leonarski_f 4878318c27 v1.0.0-rc.139 (#49)
Build Packages / Generate python client (push) Successful in 1m24s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m59s
Build Packages / XDS test (durin plugin) (push) Successful in 9m57s
Build Packages / Create release (push) Has been skipped
Build Packages / Unit tests (push) Successful in 1h17m4s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m34s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m37s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m11s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m4s
Build Packages / build:rpm (rocky8) (push) Successful in 13m12s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m52s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m30s
Build Packages / build:rpm (rocky9) (push) Successful in 13m59s
Build Packages / XDS test (neggia plugin) (push) Successful in 9m24s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m54s
Build Packages / DIALS test (push) Successful in 13m14s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m4s
Build Packages / Build documentation (push) Successful in 1m7s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Further reduce startup time for DECTRIS detectors by selectively modifying SIMPLON parameters on `/start`
* jfjoch_broker: Further reduce startup time for DECTRIS detectors by not setting beam center and detector distance via SIMPLON API on '/start'
* jfjoch_broker: Add an extra message to ZeroMQ puller ready to monitor Lite worklow preparation time
* jfjoch_broker: Image buffer configuration is postponed for Lite receiver flow till start message is received
* jfjoch_broker: Use nanoseconds internally for frame/image/readout time
* jfjoch_broker: Extra messages added for receiver operation (to be removed after debugging finished)
* jfojch_broker: Improve profiling of different data analysis steps
* jfjoch_broker: Record integration reflection count
* jfjoch_broker: Fix bug where ZeroMQ preview frequency was confusing time units (micro vs. milliseconds)
* jfjoch_broker: Fix bug where '/wait_till_done' got deadlocked
* jfjoch_writer: Fix confusion between NaN and zero in floating-point datasets

**Breaking changes**: detector definition is now using nanoseconds to define minimum frame time, minimum count time and readout time.

Reviewed-on: #49
2026-04-29 09:50:50 +02:00
leonarski_f c1c170112c v1.0.0-rc.136 (#45)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m48s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m55s
Build Packages / build:rpm (rocky9) (push) Successful in 12m17s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m2s
Build Packages / Generate python client (push) Successful in 30s
Build Packages / DIALS test (push) Successful in 13m19s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 15m15s
Build Packages / Build documentation (push) Successful in 51s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m13s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m52s
Build Packages / Unit tests (push) Successful in 1h18m25s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m17s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m57s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m35s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m29s
Build Packages / XDS test (durin plugin) (push) Successful in 8m17s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 7m2s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Improve logic regarding indexing architecture and thread pools (work in progress).

Reviewed-on: #45
2026-04-20 11:54:33 +02:00
leonarski_f bb9f5c715f v1.0.0-rc.135 (#44)
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m30s
Build Packages / Create release (push) Has been skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m12s
Build Packages / Unit tests (push) Successful in 1h23m8s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m55s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m7s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m31s
Build Packages / build:rpm (rocky8) (push) Successful in 12m59s
Build Packages / build:rpm (rocky9) (push) Successful in 14m5s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m8s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m59s
Build Packages / DIALS test (push) Successful in 11m44s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m28s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m47s
Build Packages / Generate python client (push) Successful in 1m18s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / XDS test (durin plugin) (push) Successful in 9m16s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* Multiple small bug fixes scattered across the whole code base. (detected with GPT-5.4)
* jfjoch_viewer: Improve image render performance

Reviewed-on: #44
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-16 11:59:59 +02:00
leonarski_f 4a852b4d6b v1.0.0-rc.134 (#43)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m51s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 1m6s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m45s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m58s
Build Packages / Unit tests (push) Successful in 1h20m42s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m59s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m45s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m2s
Build Packages / build:rpm (rocky9) (push) Successful in 14m15s
Build Packages / DIALS test (push) Successful in 13m10s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m4s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m18s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m12s
Build Packages / build:rpm (rocky8) (push) Successful in 12m29s
Build Packages / XDS test (durin plugin) (push) Successful in 9m50s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Add better locking for detector object - should help, when detector initialization takes too long
* jfjoch_writer: Enable writing single, integrated HDF5 file with both data and metadata
* XDS plugin: Add generation of Jungfraujoch plugin for XDS
* CI: Add tests with XDS and DIALS (`xia2.ssx`)

Reviewed-on: #43
2026-04-09 13:30:47 +02:00
leonarski_f 6133da1377 v1.0.0-rc.133
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m9s
Build Packages / build:rpm (rocky8) (push) Successful in 9m16s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m20s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m52s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m6s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m47s
Build Packages / Generate python client (push) Successful in 49s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m38s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m5s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m35s
Build Packages / build:rpm (rocky9) (push) Successful in 10m22s
Build Packages / Build documentation (push) Failing after 9s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Use httplib for HTTP server instead of Pistache
* jfjoch_broker: Drop OpenSSL support
* jfjoch_broker: Base work for multi-lattice support in the future
* Update dependencies to more recent versions (spdlog, HDF5, Catch2, httplib)

Reviewed-on: #41
2026-03-26 20:50:33 +01:00
leonarski_f 64002f1e29 v1.0.0-rc.129 (#36)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m14s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m43s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 10m35s
Build Packages / build:rpm (rocky9) (push) Successful in 11m17s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m35s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / Generate python client (push) Successful in 39s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m24s
Build Packages / Build documentation (push) Successful in 1m0s
Build Packages / Unit tests (push) Failing after 1h18m57s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m20s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m9s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Significant improvements in TCP image socket, as a viable alternative for ZeroMQ sockets (only a single port on broker side, dynamically change number of writers, acknowledgments for written files)
* jfjoch_broker: Delta phi is calculated also for still data in Bragg prediction
* jfjoch_broker: Image pusher statistics are accessible via the REST interface
* jfjoch_writer: Supports TCP image socket and for these auto-forking option

Reviewed-on: #36
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-03-05 22:13:12 +01:00
leonarski_f 1ab257af6c v1.0.0-rc.125 (#32)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m8s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m55s
Build Packages / Unit tests (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 9m23s
Build Packages / build:rpm (rocky9) (push) Successful in 9m38s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m0s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m30s
Build Packages / Generate python client (push) Successful in 20s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m42s
Build Packages / Build documentation (push) Successful in 39s
This is an UNSTABLE release. This version adds scalign and merging. These are experimental at the moment, and should not be used for production analysis.
If things go wrong with analysis, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Improve logic on switching on/off spot finding
* jfjoch_broker: Increase maximum spot count for FFBIDX to 65536
* jfjoch_broker: Increase default maximum unit cell for FFT to 500 A (could have performance impact, TBD)
* jfjoch_process: Add scalign and merging functionality - program is experimental at the moment and should not be used for production analysis
* jfjoch_viewer: Display partiality and reciprocal Lorentz-polarization correction for each reflection
* jfjoch_writer: Save more information about each reflection

Reviewed-on: #32
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-02-18 16:17:21 +01:00
leonarski_f 1c4dfd03e2 v1.0.0-rc.123 (#30)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m30s
Build Packages / Build documentation (push) Successful in 54s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m44s
Build Packages / build:rpm (rocky9) (push) Successful in 10m37s
Build Packages / Unit tests (push) Successful in 1h6m33s
Build Packages / Generate python client (push) Successful in 18s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m53s
Build Packages / build:rpm (rocky8) (push) Successful in 9m40s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m54s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m22s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m41s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m32s
Build Packages / Create release (push) Has been skipped
This is an UNSTABLE release.

* jfjoch_broker: Use newer version of Google Ceres for (potential) CUDA 13 compatibility
* jfjoch_broker: Improve performance of generating preview images, especially for large detectors (9M-16M)
* jfjoch_viewer: Improve performance of displaying images, especially for large detectors (9M-16M)
* jfjoch_viewer: Add more color schemes for better image readability
* HDF5: Common mutex for reading and writing HDF5 if both operations were to happen in the same executable
* HDF5: suppress warning if path (upstream group) doesn't exists when checking if leaf exists

Reviewed-on: #30
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-01-30 13:43:09 +01:00
leonarski_f 36d0507758 v1.0.0-rc.119 (#26)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m22s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m19s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m33s
Build Packages / Unit tests (push) Failing after 1h11m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m27s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 8m44s
Build Packages / build:rpm (rocky8) (push) Successful in 8m44s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m11s
Build Packages / build:rpm (rocky9) (push) Successful in 9m54s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m11s
Build Packages / Generate python client (push) Successful in 17s
This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

* jfjoch_broker: Add binary export of data analysis plots over OpenAPI
* jfjoch_broker: Minor fixes to HTTP error handling
* jfjoch_viewer: Prefer binary plots over JSON plots
* jfjoch_viewer: Change foreground with F button + wheel
* jfjoch_viewer: Change way how angles are displayed
* jfjoch_viewer: Display resolution of the mouse cursor in top left corner

Reviewed-on: #26
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-08 19:53:35 +01:00
leonarski_f 06949caf1a v1.0.0-rc.112 (#18)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m53s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 8m25s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m5s
Build Packages / build:rpm (rocky8) (push) Successful in 8m35s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m40s
Build Packages / build:rpm (rocky9) (push) Successful in 9m14s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m17s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Unit tests (push) Successful in 1h15m9s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 9m40s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
This is an UNSTABLE release and not recommended for production use (please use rc.11 instead).

* jfjoch_broker: Experimental rotation (3D) indexing
* jfjoch_broker: Minor fix to error in optimizer potentially returning NaN values

Reviewed-on: #18
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-30 17:39:22 +01:00
leonarski_f 224cc8b89c v1.0.0-rc.110 (#16)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m46s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 5m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 6m59s
Build Packages / build:rpm (rocky8) (push) Successful in 7m33s
Build Packages / Build documentation (push) Successful in 41s
Build Packages / Unit tests (push) Successful in 1h8m51s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 6m56s
Build Packages / Generate python client (push) Successful in 19s
Build Packages / build:rpm (rocky9) (push) Successful in 8m45s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 7m51s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m12s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m45s
Build Packages / Create release (push) Has been skipped
This is an UNSTABLE release.

* jfjoch_broker: Add auto-contrast option for preview images
* Frontend: Add logo image
* jfjoch_viewer: Add logo image
* jfjoch_viewer: For image chart allow to set min value to zero
* jfjoch_viewer: For resolution estimation plots, visualization uses 1/d^2 as measure
* jfjoch_viewer: Add 3D unit cell visualization (experimental/WIP/not really there)
* Documentation: Add logo image

Reviewed-on: #16
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-28 12:47:35 +01:00
leonarski_f 8e65e8bf25 v1.0.0-rc.99 (#4)
Build Packages / build:rpm (rocky9) (push) Successful in 9m9s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 8m57s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m13s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 7m58s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m45s
Build Packages / Unit tests (push) Successful in 1h15m22s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m30s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 8m43s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 9m51s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m59s
Build Packages / Build documentation (push) Successful in 49s
This is an UNSTABLE release.

jfjoch_broker: Fix output during mask data collection

Reviewed-on: #4
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-11 10:40:25 +01:00
leonarski_f 5d81af1c88 2511-eiger-mask-2 (#3)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m45s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m40s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m41s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 36s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 6m45s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 8m6s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 8m26s
Build Packages / build:rpm (rocky8) (push) Successful in 8m31s
Build Packages / build:rpm (rocky9) (push) Successful in 9m23s
Build Packages / Unit tests (push) Successful in 1h12m33s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m40s
This is an UNSTABLE release and not recommended for production use (please use rc.96 instead).

* jfjoch_broker: For DECTRIS detectors fix dark data collection during initialization

Reviewed-on: #3
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-10 11:42:39 +01:00
leonarski_f 8b356a7001 v1.0.0-rc.96 (#1)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 7m20s
Build Packages / Generate python client (push) Successful in 13s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 36s
Build Packages / Unit tests (push) Successful in 1h10m44s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m47s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m13s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m10s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 7m53s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 7m39s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 6m55s
Build Packages / build:rpm (rocky8) (push) Successful in 7m57s
Build Packages / build:rpm (rocky9) (push) Successful in 9m0s
This is an UNSTABLE release.

* Fixes in CI pipeline
* jfjoch_broker: Remove PNG preview, no dependency on libpng
* jfjoch_writer: Fix UTC timestamp being generated wrong (mix between milli- and microseconds)
* jfjoch_viewer: Show data collection time in dataset tooltip
* jfjoch_viewer: Allow to choose the calibrant (presets for LaB6 and silver behenate)
* jfjoch_viewer: Auto foreground value
* Use external libjpeg-turbo and libtiff: simpler build stack, these are built and linked statically in automated Docker builds
* Remove OpenBLAS dependency

Reviewed-on: #1
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-02 13:45:57 +01:00
leonarski_f 061152279c v1.0.0-rc.91 2025-10-20 20:43:44 +02:00
leonarski_f c44a07aac2 v1.0.0-rc.90 2025-10-02 15:55:39 +02:00
leonarski_f 14f5051692 1.0.0-rc.89 2025-10-01 22:54:40 +02:00
leonarski_f c0bd8e1215 v1.0.0-rc.82 2025-09-22 19:49:39 +02:00
leonarski_f 5d9d2de4a4 v1.0.0-rc.81 2025-09-21 19:27:51 +02:00
leonarski_f c67337cfe1 v1.0.0-rc.72 2025-09-08 20:28:59 +02:00
leonarski_f dba807fadd v1.0.0-rc.71 2025-08-28 07:07:01 +02:00
leonarski_f bb32f27635 v1.0.0-rc.70 2025-08-27 06:21:10 +02:00
leonarski_f 20973792e4 v1.0.0-rc.68 2025-08-16 19:59:27 +02:00
leonarski_f 51cbef4c84 1.0.0-rc.66 2025-07-18 11:42:39 +02:00
leonarski_f 06c5b9cf7f 1.0.0-rc.65 2025-07-16 20:19:48 +02:00
leonarski_f 4da526c51b v1.0.0-rc.63 2025-07-09 14:55:28 +02:00