1156 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_f	86ea443af0	writer: GetCompressionAlgorithm passed a byte count as the filter-parameter count ... H5Pget_filter2 takes cd_nelmts as the number of elements the cd_values buffer can hold and writes up to that many. It was given sizeof(cd_values) = 32 for an 8-element array, so a filter carrying more than 8 parameters would have been written past the end of the buffer. Bitshuffle carries 5, so nothing overflowed in practice. The "Weird value" message printed cd_values[1] while the test above it is on cd_values[4]. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015eAE2K7i5JGDwgwifiCfuA (cherry picked from commit dd1af9576c)	2026-09-19 19:11:42 +02:00
leonarski_fandClaude Opus 5	8f2c6b10cc	tools/battery: verdicts judge lattice and symmetry only ... Data quality - resolution, CC1/2, R_meas, completeness, ISa - is no longer a pass/fail criterion anywhere; it is reported beside the reference in the tables and plots for a human to judge. The CC1/2 noise ratio against XDS stays as a reported guide. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT	2026-09-19 18:41:43 +02:00
leonarski_fandClaude Opus 5	a8a4bed136	tools/battery: merge quality is CC1/2 against XDS only ... The owner's scoring decision: no R_meas criterion, and merge quality judged only by CC1/2 relative to XDS. - The global rule (R_meas > 60% or CC1/2 < 0.5 fails) is gone. - XDS arms: a set fails `merge` when (1/CC1/2 - 1) of the reference-range table is more than twice XDS's, XDS's CC1/2 taken at the bottom of its 0.1% rounding. CC1/2 = S/(S+E), so 1/CC1/2 - 1 = E/S at any CC1/2, and E goes as 1/observations: 2x is XDS's merge with half its observations. Stored as cc_half_noise_ratio. No REFRES table or no XDS CC1/2: no criterion. - Open arm: no merge criterion; CC1/2 and R_meas stay reported numbers. - Low-resolution R_meas is reported, not scored: the lowest shell of the own table, the reference-range table and XDS's (lowres_*), with lowres_r_meas_ratio in the like-for-like table and a ratio plot. - Every schema-3 run is re-scored when it is read (report, compare, baseline delta) with today's scorer and today's manifest rows, so both sides of a comparison are scored alike. results.json keeps the verdicts as scored at run time; rows without a lattice keep them. Re-scoring the rc172 reference run: inhouse 29 -> 27 pass (three new merge fails, one R_meas fail lifted), open 137 -> 138 (one R_meas fail lifted). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT	2026-09-19 18:32:15 +02:00
leonarski_fandClaude Opus 5	5f9a5f789e	tools/battery: the XDS references carry the lowest shell's R_meas ... parse_correct_lp also reads the first row of CORRECT.LP's last resolution table: its high-resolution limit (dmin_low) and R_meas (r_meas_low). inhouse.json regenerated with `refs --arm inhouse --write`; the only change is the two new keys per set (the private manifest was regenerated the same way, outside the repository). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT	2026-09-19 18:32:15 +02:00
leonarski_fandClaude Opus 5	f500c6446d	tools/battery: a set with a known-wrong reference can be left unscored ... Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT	2026-09-19 17:21:46 +02:00
leonarski_fandClaude Opus 5	2c5f26bf7a	tools/battery: the 16 keV thaumatin reference obeys Friedel's law ... At 16 keV a native crystal has no usable anomalous signal, so XDS's CORRECT was rerun with FRIEDEL'S_LAW=TRUE and the set no longer runs with -A. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 11:24:32 +02:00
leonarski_fandClaude Opus 5	a20e27a69f	tools/battery: the like-for-like table covers XDS's own range ... --report-resolution now gets XDS's own d_min, the range its CORRECT.LP totals cover, so the comparison is exact; shells past rugnux's cut print as not merged. The derived reference d_min stays what rugnux's own cut is scored against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 11:22:49 +02:00
leonarski_fandClaude Opus 5	9b3eea72fe	tools/battery: one rugnux run per set ... --model and --report-resolution are both processing-inert (merged reflections bit-identical with and without them), so the bare/model and bare/xds variant pairs collapse into one command per arm: open: rugnux -o p --no-export-unmerged --model <deposited coords> <input> (no --model where there is no deposited model) inhouse/private: rugnux -o p --no-export-unmerged [-A] --report-resolution <reference d_min>,<XDS d_max> <input> Every row is rugnux's own result (own cut vs the reference, as `bare` was), and on the XDS arms also carries the REFRES_* table, scored against CORRECT.LP's completeness, multiplicity, R_meas, CC1/2 and ISa, with ISa and R_meas ratio plots and a like-for-like table; REFRES_SHELLS_PAST_LIMIT > 0 marks the row as coverage (rugnux's cut coarser than the reference). Nothing is forced on the processing any more. On the open arm the space group is scored on the data's own determination (SOHNCKE_SPACE_GROUP where SPACE_GROUP_ENANTIOMORPH is ASSUMED_FROM_MODEL), the model's label kept as sg_label; R_FREE/R_WORK/CC_MODEL /MODEL_FIT are trend fields (placement-only, own free set); REFMAC stays opt-in. results.json schema 3: one row per set, no variant/first_read; older runs are read through their bare rows (open-arm model R-factors folded in), so compare and report keep working against them. Every time is now a first read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 11:18:10 +02:00
leonarski_fandClaude Opus 5	16bb4da2bf	rugnux: the run's own N_obs and R_meas stop at the same cut as N_uniq ... The own statistics table's grid bottoms 0.1% below the finest kept reflection, and the observation-level counts (N_obs, R_meas, the Bijvoet split behind SigAno and CCanom) re-walk the fulls, which still hold every ingested group. Groups in that band just below the auto cut were erased from the merge, so they were not in N_uniq, but their observations still counted in N_obs and R_meas of the finest shell. The own table now floors those counts at the cut by group d, the rule the erase applies - the same floor the reference-range table already used (now passed as a double, so the comparison is exactly the erase's). Report-only: on an auto-cut cubic in-house set the written MTZ, HKL, P1 MTZ, unmerged MTZ and image table are byte-identical; TOTAL_OBSERVATIONS 378754 -> 377933, MULTIPLICITY 40.31 -> 40.22 (now equal to the reference-range table's), finest shell N_obs 60873 -> 60052, R_meas 1136.9% -> 1129.2%, CCanom -0.6% -> -0.3%; overall R_MEAS, CC1/2, SigAno unchanged at printed precision; the mmCIF's pdbx_number_measured_all / pdbx_redundancy / last shell row follow. A detector-limited lysozyme run (no auto cut) is byte-identical throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 11:18:10 +02:00
leonarski_fandClaude Opus 5	f9c5adbfb0	rugnux: the indexing-ambiguity warning is withdrawn only where the model settled it ... The INDEXING_AMBIGUITY warning was suppressed whenever --model was given on rotation data, decided before the model was read. A model that then decided nothing - not tested, rejected, or a probe winner that did not beat its own null - still silenced it, so WARNING_COUNT, PATHOLOGY_FLAGS and possibly VERDICT differed from a run without the model although the written reflections were identical. The warning is now issued exactly as without a model, and withdrawn (from the warnings and from the statistics text) after model validation only where the indexing probe decided the indexing: the model fits and the winner's R-free margin beats the random-placement null (ModelValidationResult::indexing_decided, set where the decision is taken). A reference MTZ, or the model reference on serial stills with -C and -S, suppresses it up front as before. Verified bare vs --model on three open-arm sets: merged MTZ data identical in all three; warnings identical where the model decided nothing (identity probe without null; no twin law); withdrawn where the probe decided (+33 sigma). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 11:18:00 +02:00
leonarski_fandClaude Fable 5.1	8a0da57941	rugnux: --report-resolution, a second statistics table at a reference range ... Comparing a run with another program's table has meant running rugnux AT that program's resolution range (--scaling-high-resolution), which is a different run: the range moves the cut, the space-group decision and everything after them, so the comparison buys itself a different answer. --report-resolution <dmin>[,<dmax>] instead leaves the run alone and adds a second table to section 3 of the report - the REFRES_* keys and a shell table - binned from the same merged reflections over the range given, with the completeness denominator enumerated over that range and the shells in equal steps of 1/d^2 so they read row for row against a CORRECT.LP at the same range. Report-only: the merged files and every decision are byte-identical with and without it. The table holds only what the run kept. Where the reference range is finer than the run's own limit, the shells past it are printed as not merged (with their possible count) rather than as zeros, REFRES_SHELLS_PAST_LIMIT counts them so a consumer can tell "not merged" from a measured zero, REFRES_ COMPLETENESS counts their reflections as missing, and the other overall numbers are over the shells the run reached; nothing is read from the observations the run judged to carry no signal. REFRES_ISA is the error model refitted on the reflections of the table alone, in XDS's convention (rotation only; the stills model is fitted over the whole range already). On the rotation path the statistics block of MergeAndStats becomes a lambda over a shell grid, called once for the run's own grid and once for the reference one; the reference call floors every observation-level count at the cut by group d, the rule the erase applied. The stills MergeStats takes a declared range, whose bounds are the grid's whether or not any reflection reaches them. Both --mode mx and --mode scale report it, the viewer's command line echoes it, and the docs describe the keys. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-19 10:58:42 +02:00
leonarski_fandClaude Fable 5.1	26a80caac4	rugnux: overall SigAno and CCanom over the written range only ... The per-shell SigAno and CCanom of the rotation merge were accumulated on the report grid, but the overall pair of numbers - the SIGANO / CC_ANOM keys and the mmCIF's pdbx_absDiff_over_sigma_anomalous - were summed over every Bijvoet pair the fulls hold, including the ones past the automatic resolution cut that the written reflections do not contain. On a run the cut trims, the overall line therefore described more data than the shells above it add up to. Both overall numbers now count only the pairs that land in a shell, as the rest of the table does; a run with a manual limit, whose ingest already ends at the limit, is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-19 10:58:41 +02:00
leonarski_fandClaude Fable 5.1	8242773938	CUDA: allocation streams are handed back, not leaked; a handled pool failure leaves no error ... A long-running broker began cancelling every data collection with Device decoding failed (CUDA (GPU) error (out of memory)), falling back to host decompression CUDA (GPU) error (out of memory) while nvidia-smi showed the cards less than a fifth full. Two defects, both in CUDAMemHelpers.h and both from the pooled allocator (cudaMallocAsync) that came with rc.162. The leak. cuda_allocation_stream() kept one stream per (thread, device) in a thread_local map of raw cudaStream_t and never destroyed them. That was written against rugnux, where the worker threads live as long as the process. The broker starts fresh std::async threads for every data collection - 16 or 64 of them - so every collection left that many streams behind. Measured: 0.56 MB of device memory per leaked stream, linear to 4928 streams, never returned, with nothing on the host side growing. The RAII wrapper (CudaStream) was there but not used at this site, and using it as-is - a stream destroyed when its thread exits - would not have been safe: ShadowFinder builds its GPU accumulator on a throw-away std::async thread and frees it from another thread long after, and that free is ordered on the allocating thread's stream. So the streams are still never destroyed, but a thread now only borrows one: CudaStream objects live in a process-wide per-device idle list, a thread takes one on first use and hands it back when it exits. Their number is bounded by the threads that were ever alive at once instead of by the threads ever started. The list itself is deliberately leaked, so that nothing calls into CUDA during static destruction. Replaying the broker's pattern against the real header, 60 collections of 64 threads: before 3840 streams, 260 -> 2424 MB of device memory after 64 streams, 260 -> 358 MB The stale error. Every helper here throws a named message, yet the log carried the raw CUDA string, so the failure came through a cuda_err() and not from an allocation. CudaDevicePtr falls back to cudaMalloc when cudaMallocAsync fails, silently - but the failed call stays behind as the thread's last error, and the cudaGetLastError() that follows the next kernel launch reports it. The buffers were all allocated; the frame was lost anyway, once on the device-decode route (caught, hence the warning) and once more on the host fallback (fatal). The pooled attempt failing, and a stream that cannot be created, are both handled by falling back, so both now clear the error they leave. What finite resource the production cards ran out of at under 4 GB used was not established - no cap on the number of streams was found up to 4928 on the card this was measured on. The leak is the only thing on this path that grows with uptime. tests/CUDAMemHelpersTest.cpp: later threads end up on the same stream, concurrent threads on different ones, a buffer is freed cleanly after its allocating thread has exited (and another has borrowed its stream), and a pool that cannot serve a request leaves no error behind - the last by capping a memory pool at 4 MB so that the pooled attempt fails and the fallback succeeds. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-19 10:39:39 +02:00
leonarski_fandClaude Opus 5	aa47b6bbbd	tools/battery: the sparse myoglobin set gets a real XDS reference ... The crystal does have a single P2 lattice, holding ~6% of each frame's spots on top of ice and protein powder; XDS indexes it with the ice rings excluded. The reference is now that P2 run to 2.0 A instead of a failed P1 run, and the set is renamed from aggregate to sparse. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 10:02:08 +02:00
leonarski_fandClaude Opus 5	6e2e2fcaf9	tools/battery: a working XDS reference for the half-shadowed lysozyme set ... XDS was rerun with the pin shadow masked and the frames where the crystal leaves the beam excluded; the reference is now P43212 to 1.65 A instead of a failed P1 run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:47:36 +02:00
leonarski_fandClaude Opus 5	ee85ad1abd	tools/battery: run every variant of a set back to back ... The battery is bound by reading images from disk, so instead of one battery run per setting each set now runs in all its variants in a row while its images are still in the page cache: open arm bare, then model (rugnux --model <deposited coordinates>) XDS arms bare, then xds (XDS's resolution range forced, -A where XDS was anomalous) bare (plain rugnux <input>) runs first on every arm, so its time always carries the set's disk read whichever variants are selected; each row records first_read and the report's timing table says which variant's times are cold. --variants runs a subset; --unforced is gone (--variants bare). Rows, work dirs (work/<arm>/<set>/<variant>/), compare and the report are per variant; compare pairs (arm, set, variant) on the variants both runs have on an arm, and a schema-1 run is read as one variant. The model variant takes the coordinates and published R-free from model_check's RCSB cache (site key pdb_cache) and records rugnux's R-free/R-work and the ratio; the REFMAC check is now opt-in (--model-check) and its keys moved to refmac_*. results_schema 2. model_sweep.py is retired: the model variant replaces it (its --spot/--scaling-low-resolution 50 were rugnux's defaults, so the command is the same). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:42:27 +02:00
leonarski_fandClaude Opus 5	f6f955e19b	tools/battery: -A where XDS kept Friedel mates apart ... On the XDS-referenced arms rugnux merges Friedel mates separately whenever XDS ran with FRIEDEL'S_LAW=FALSE, in forced and unforced mode alike, so the merging statistics compare like with like. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:25:42 +02:00
leonarski_fandClaude Opus 5	17eec06023	tools/battery: the R-free ratio compares like with like ... Our merge and the depositor's structure factors are both scored on the depositor's free set by the same protocol, and the report plots that ratio. R-free on our own free set reads low, because those reflections were mostly work reflections in the deposited refinement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:22:20 +02:00
leonarski_fandClaude Opus 5	81bd42095b	tools/battery: model_check.py, R-factors of the deposited model against the merged data ... Run after rugnux has written its MTZ, so processing never sees the model. Fetches the entry (cached), matches the data to the model's setting by trying every Niggli-cell change of basis and keeping the one that best fits the model's Fcalc on a mid-resolution shell, re-expresses the data in the deposited space group and cell, and has REFMAC compute R-work/R-free of the unmodified model (rigid-body mode read at its first cycle: no dictionaries, no ligand stripped, no refinement). Baseline: the same protocol on the depositor's own structure factors and free set, plus rugnux's data on the depositor's flags. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:20:51 +02:00
leonarski_fandClaude Opus 5	26e256513c	tools/battery: in-house sets named by protein, beamline and feature ... Directories follow <protein>_<beamline>_<feature> (e.g. lyso_x10sa_90deg_1, thau_bl1a_6p5keV, thau_micromax_pink); previous ids stay as aliases so older runs still pair up in compare. The ice-only control moved out of the in-house arm. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:18:19 +02:00
leonarski_fandClaude Opus 5	13d78e5bc3	CLAUDE.md: the battery is tools/battery, with its rules ... Replaces the mention of the retired rugnux_vs_xds.py with a short Battery section: the canonical commands, never a full run without the owner's go-ahead, judge against a persisted baseline, the private arm stays out of the repository. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:15:45 +02:00
leonarski_fandClaude Opus 5	2cfdc939a7	Retire the scattered battery scripts in favour of tools/battery ... - rugnux_vs_xds.py: deleted. The inhouse and private arms of tools/battery run the same comparison against CORRECT.LP, from a fixed manifest instead of a directory scan, with persisted runs and a standard report. - lyso_test/anomalous_scoreboard.sh: deleted. A one-dataset SHELXC/ANODE scoreboard with hard-coded personal paths, superseded by rugnux_anomalous.py. - rugnux_vs_pdb.py -> tools/battery/model_sweep.py. The rugnux --model sweep is not superseded, but it now takes its inputs from the open arm's manifest and the battery site config, so both tools process the same files. - docs/TESTS.md and rugnux_stills_ab.py point at the battery. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:15:45 +02:00
leonarski_fandClaude Opus 5	a7a3c2a00c	tools/battery: one canonical runner, derived XDS d_min, clean inhouse layout ... - Site config: site.example.json is committed; the local site.json (git-ignored) or $JFJOCH_BATTERY_SITE / --site names this machine's paths and the private manifest. Data roots are the arms' own directories, so copying them and editing the site config moves the battery. - Run states: manifest records the runner's pid; a run is complete, running, aborted (Ctrl-C/SIGTERM, or `abort RUN`) or unfinished. compare and report refuse a run that did not finish unless --allow-incomplete, and the report says so. - `list` shows runs and their state; `run` defaults its label to the rugnux version and its baseline to the site's persisted one; a baseline must be complete and of the same privacy as the run; a private report cannot be written into the repo. - XDS reference d_min: when the finest shell of XDS's table has no significant CC1/2, XDS merged past its own signal (no resolution cut, or a cut at the detector edge) and its limit is the detector's reach. The reference is then where XDS's CC1/2 falls through 0.30 (rugnux's own target), and that is also the range forced on rugnux. The manifest keeps XDS's limit (dmin_xds) and the rule (dmin_rule). Forcing the old 1.08 A on the weak insulin set turned rugnux's I23 into I222 (ISa 12.5); forced at the derived 1.81 A it is I23 again (ISa 18.0). Changed in the inhouse arm: insu_I_weak 1.08 -> 1.81, lyso_half_image 0.80 -> 2.60, cytc_eiger 2.04 -> 2.27, lyso_strong 1.18 -> 1.24; every other inhouse limit stands. - inhouse.json follows the directory rename (ids and inputs, data root /home/data/inhouse); `remap` applies such a rename and keeps the old ids as aliases, so runs from before it still compare set by set. - Optional open-arm model check (model_check.py, when present): R-free of the deposited model against each merge, recorded and plotted. - results.json has a fixed, documented row schema (results_schema 1); the HTML report gains inline-SVG plots (verdicts per arm, d_min and R-free ratios per set). - README.md: arms, data layout and provenance, prerequisites, running, run directory, results schema, report, comparing, protocol, adding datasets. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 09:15:37 +02:00
leonarski_fandClaude Opus 5	8afab32d38	tools/battery: 6h2p runs as two sets, one per wavelength ... The crystal was measured at two wavelengths; both sweeps are kept. The native sweep carries the deposited resolution, the 1.89 A sweep is scored on symmetry and cell only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 08:51:07 +02:00
leonarski_fandClaude Opus 5	e24ee4d4b5	CLAUDE.md: the standard test proteins and the open battery arm may be named ... Extends the lysozyme carve-out to thaumatin, insulin, cytochrome C and myoglobin (plus the in-house no-crystal controls), and allows the public PDB depositions of the open battery arm. The private arm stays out of the repository entirely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 08:50:54 +02:00
leonarski_fandClaude Opus 5	e6baa4f153	tools/battery: committed, reproducible validation battery ... One runner for the open (vs PDB deposition) and in-house (vs XDS) arms, replacing the scattered scripts outside the repository. Each run writes a new, read-only directory named <date>_<rugnux git>_<label> holding a copy of the binary with its sha256 and build flags, manifest.json (inputs, references, options, host, GPU sharing) and results.json in one schema for all arms, plus per-set work directories and a Markdown and a self-contained HTML report (per-arm and per-population pass rates, resolution gain, ISa/R_meas distributions, timing, failures, delta vs a baseline run). - Dataset lists and references are data files: open.json (151 public sets, deposited group/cell/resolution, pinned inputs) and inhouse.json (standard test crystals and no-crystal controls, XDS references). The private arm's manifest stays outside the repository; a private run gets its own run directory and report. - Scoring is setting-invariant: lattice by Niggli-reduced primitive cell (beta vs 180-beta, C2 vs I2), symmetry via sgequiv on the open arm and by point group against XDS, then merge usability. Re-scoring the rc169 and rc170 battery outputs reproduces their verdicts. - Input discovery (`discover`) ranks sweeps by frame count, never by file size or sort order; a run never searches, it runs what the manifest names. - `compare A B` flags per-set changes beyond noise; `--rerun-changed` reruns them with A's saved binary to separate real changes from run-to-run noise. - `--tier smoke` selects a 22-set subset spanning the populations; a full-arm run takes a lock; `--gpulock` queues on the shared GPU and marks the timing as not a reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 08:13:46 +02:00
leonarski_fandClaude Opus 5	d6a2b9d19e	broker: error responses carry the content type the OpenAPI spec declares ... The 500 handlers built a correct error_message object but sent it through send_plain(), which labels the body text/plain. The spec declares 500 as application/json + error_message, and the generated clients key their deserialization off the content type: the Python client kept the body as a raw str, handed it to ErrorMessage.from_dict(), and pydantic raised - the exception swallowed by the finally: in response_deserialize, so the caller got a ServiceException with data=None and only the raw text in .body. send_plain becomes send_error, taking the content type alongside the code and the body; the two exception handlers now return it. 400 stays a plain-text exception string, exactly as the spec says; 500 is JSON. Two further 500s did not carry an error_message object at all and now do: the generic std::exception branch of handleParsingException, which returned a bare what(), and ProcessOutput's output-validation failure, which returned the validation dump. A catch-all set_exception_handler covers anything that escapes a route's own handler - httplib would otherwise answer with a bodyless 500, which fails to deserialize the same way. Verified against a running broker: POST /pedestal in the wrong state returns application/json and the generated Python client parses it into ErrorMessage(msg=..., reason='WrongDAQState'); a malformed body still returns 400 text/plain. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 13:59:57 +02:00
leonarski_fandClaude Opus 5	51f51ec5bf	tests: the FFT indexers on a frame of pure noise ... The frame that works the indexer hardest is junk, not a crystal: a real lattice prunes the FFT shortlist to four or five distinct directions, while noise leaves dozens, which is what drives the length sort and the degenerate-plane fallback - where a production broker segfaulted twice. There was no test in that regime. Two cases, both over every FFT back-end the build has (GPU FFT under CUDA, CPU FFTW always), so a non-CUDA job covers them as well: - a cloud of 1500 reciprocal-space vectors of random direction and length, with no periodicity in it. Both back-ends return 70 candidate lattices from such a frame, which is the designed behaviour - Run() offers candidates and the caller scores them - so what the test pins is that none of them looks like a crystal: each takes about 1% of the cloud against the ~100% a real lattice takes, and the assertion allows 5%. - the same noise flattened onto a tilted plane. A coplanar shortlist cannot close a cell, so this is the branch that goes looking for the missing row in a 3 deg cap (SearchCap), verified with a temporary probe to enter it on both back-ends and not to enter it on the isotropic frame. The SAME indexer object then has to index a clean lattice correctly, which is the regression test for the cap search putting the direction grid back. The cloud is drawn from a fixed seed, and from the engine scaled by hand rather than through uniform_real_distribution, whose output is not specified to be the same in every standard library: a crash that needs one particular junk frame is no use as a regression test if the frame is redrawn on each machine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 13:03:06 +02:00
leonarski_fandClaude Opus 5	4b5c6e234d	indexing: the cap search always puts the direction grid back, and every FFT upload is checked ... A production broker on rc.170 segfaulted again in the FilterFFTResults length sort, in a binary that does contain the rc.170 non-finite guard (verified from its disassembly: the two isfinite tests are compiled in and skip the insert). With magnitude and length guaranteed finite, the only remaining way that sort sees a NaN key is a NaN direction vector - and the one place that writes the direction grid is SearchCap. SearchCap swaps in a temporary cap grid but restored it around ExecuteFFT only. The fill can run out of memory, the upload can fail, and the .at() that reads the peak back throws on exactly the corrupted result row this search has to survive - any of those left the cap grid installed on an indexer the pool hands the next image. If the axis was ever not a direction, axis.Normalize() is NaN and the leaked grid is 16384 NaN directions, which is that crash on every image afterwards. The restore now covers the whole body, and an axis that is not finite and non-zero is refused instead of normalized into NaN. The peak is also tested against the minimum length before the .at() rather than after, so an unusable peak with a corrupted index no longer throws at all. DirectionsChanged uploaded the grid with three unchecked cudaMemcpy calls, and the spot upload in ExecuteFFT had three more: a silently failed upload leaves the device holding the previous grid, so the directions the host reads the results against are not the ones the kernel used. Checked like the rest of the file. Tests: [Indexing] (9 cases, 115 assertions) and RotationIndexer pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 12:25:09 +02:00
leonarski_fandClaude Opus 5	20977e89bc	viewer: draw a live grid scan as a grid, and the compact axis names upright ... Two fixes in the dataset-info panel. The live (HTTP) reader took the goniometer axis and the grid scan as alternatives, so a raster collected at a head position - both fields set, the axis with step 0 - lost its grid and was drawn as a line plot, with the Grid button disabled. The broker had the same bug and lost it in rc.162 (OpenAPIConvert.cpp); the HDF5 read-back path already read both. The reader now sets both, like the other two. Qt Charts always draws a vertical axis title rotated, which reads awkwardly beside the two numbers it names in a half-height plot and is what forced that plot's minimum height (Qt elided "Spots" to "Sp..." once the font outgrew it). The title is switched off and the view paints the name itself: upright, one letter under the next, in the series' colour, centred on the plot area, with the left margin reserving its width. It is re-measured on every rebuild, so a font change still fixes itself, and plotAreaChanged re-centres it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 11:34:57 +02:00
leonarski_fandClaude Opus 5	4bff5c2b01	v1.0.0-rc.171 ... VERSION bumped and update_version.sh run: the OpenAPI spec and the three clients it generates, the frontend and docs version strings, the Redoc page, and the FPGA and PCIe-driver version strings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 08:41:29 +02:00
leonarski_fandClaude Opus 5	29a90416b7	docs: condense the rc.171 changelog ... Three lines at the level the rest of the file keeps: what a user gains, not how it works. The detail lives in RUGNUX_FORMATS.md and in the commits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 08:38:42 +02:00
leonarski_fandClaude Opus 5	39a98692e2	marCCD/SMV: keep the default frame time when the header's exposure is not a time ... One deposited marCCD sweep carries -2093438692 in its exposure field. Passed on as written that is a negative count time, DiffractionExperiment refuses it, and the whole dataset fails to open - over a number that feeds no geometry, no indexing and no merge. With the field ignored when it is implausible, that sweep processes and matches its deposition: P2_12_12_1, cell within 0.12%, and 1.375 A against a deposited 2.0. Neither format guarantees the field. The same guard is in the SMV reader, whose TIME key is no better attested, and both say what they did rather than passing over it silently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 08:34:49 +02:00
leonarski_fandClaude Opus 5	8288d4cbe6	viewer: the stacked spots/background plots hold together at any font size ... The two half-height plots were matched once per data load, from JFJochViewerDatasetInfo: it read both y-label widths and pushed the larger back into both. A font change never goes through that path - each chart re-measures its own labels in its FontChange handler - so the padding was still the width measured in the old font, collapsed to zero, and the two curves started at different x at 125 % and 150 %. The pairing now lives in the chart view: each compact plot holds a peer pointer, and every rebuild re-matches the pair, so a font change fixes itself. Matched along with the label width: - the minimum height, taken from the width of the rotated y-axis title (Qt elided "Spots" to "Sp..." once the font outgrew the 70 px that was hardcoded for it), and given to both plots so the layout does not hand the wider-titled one more of the dock; - the plot area height: the upper plot now paints its x labels transparent instead of hiding them, so Qt reserves the same label band on both. The background plot also stops snapping its y axis to zero - it is read for its shape, not its level, and half a dock is not much to draw it in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 08:28:08 +02:00
leonarski_fandClaude Opus 5	7fecd11337	docs: the 51 second-round external datasets, and XRDa ... EXTERNAL_TEST_DATA.md gains a row per dataset in the existing form - deposited beamline, resolution, space group, cell and title from the RCSB API, source repository with its citable DOI, and the detector read out of the image files themselves. Every DOI was resolved before it was written down; one differs from the pattern the rest follow (5EPE is 10.18430/m3159c, not the m3<pdbid> form), which is the reason the page insists on resolving them rather than constructing them. Seven of the 51 name a detector in the PDB entry that the files contradict, now listed with the others. Two are not merely a model or size: one deposited as a "MAR CCD 165 mm" writes a 225 mm plate, and one deposited as a "RAYONIX MX300-HS" writes SMV with 315 mm geometry. marCCD and SMV headers name no model at all, so the Detector column carries what those headers do state - plate size, serial - and the page explains that. A new section gives the composition of the round: repository, file format as the files are on disk, facility and crystal system. It is metadata about the depositions, not measurement: no quantity measured by this software appears on the page, as the page's own preamble promises. The format spread is the point of the round, and is why Rugnux now reads marCCD, SMV and gzipped miniCBF natively. ACKNOWLEDGEMENT.md adds XRDa, the archive three of these came from, in the form the preceding documentation sweep settled on for a repository with no canonical citation paper: name, link and what it asks to be cited, and no substitute reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 08:20:42 +02:00
leonarski_fandClaude Fable 5	70c4d871b6	Documentation sweep: Rugnux naming, repository citations, defect pass ... - Capitalize Rugnux as a proper noun throughout the prose; the command stays lowercase `rugnux` in code font. RUGNUX_OVERVIEW.md is retitled "What Rugnux does". - ACKNOWLEDGEMENT.md cites the raw-data repositories only: dataset counts and DOI prefixes moved out (EXTERNAL_TEST_DATA.md owns them), the ESRF data portal gains its citation (Dimper et al. 2019), and MXRDR remains name + link - it has no canonical citation paper. - RUGNUX_FORMATS.md: the CCD formats (marCCD, SMV) are supported as-is with very limited scope, and per-panel XFEL data is not read. - Fix wrong facts a reader would act on: nonexistent `make jfjoch` targets, invalid udev rules, PUSH sockets documented as PULL, swapped writer width/height, underload semantics, the transposed pixel-mask numpy example (the server checks width and height separately), the Durin/Neggia mask-bit table, FPGA threshold register addresses and the mailbox bit field, the I2C core's document number (PG090), an inverted MODEL_FIT_SIGMA formula, a self-inconsistent worked report example, and 11 cross-page anchors whose slugs carry MyST section numbers. - Unify CC1/2 spelling in prose; math notation and report keys unchanged. - Sweep grammar, typos and editing residue across the FPGA, deployment, streaming and analysis pages, including historical CHANGELOG typos. - rugnux_cli.cpp: the -S usage/error examples pair 96 with P43212; 92 names a different group. - Root THIRD_PARTY_NOTICES.md: scope the GPL-compatibility claim (CUDA EULA) and the vendored-table intro (traccc); the docs copy regenerates via update_version.sh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-09-17 08:07:45 +02:00
leonarski_fandClaude Opus 5	28f222f51d	marCCD: take the distance from the start/end pair when the first field is zero ... Four of the twelve marCCD sets in the new corpus were refused with "states no detector distance". Their headers carry xtal_to_detector (offset 640) as zero and the real distance only in start_xtal_to_detector and end_xtal_to_detector - 70, 170 and 200 mm on three BESSY Rayonix MX-225 sweeps, against a set where all three fields agree at 300 mm. Reading the first field alone is what refused them. The guard itself was right: a distance of zero collapses every resolution and every scattering vector, so refusing beats processing silently. It just fired on data that does state a distance, in the field the format also defines for it. With the fallback, all three process de novo and match their depositions - P2_1 at cell 0.06%, I422 at 0.40%, P1 at 0.07% - and each reaches finer than its deposited resolution (0.889 A against 1.09, 1.440 against 1.69, 1.757 against 1.93). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 06:39:39 +02:00
leonarski_fandClaude Opus 5	2ab8c55dfa	rugnux and the viewer read SMV and gzipped miniCBF ... Two more of the formats deposited data actually arrives in, found by processing a corpus of it: every PETRA III EMBL set is .cbf.gz, and NSRRC and the whole ADSC Quantum era are SMV. Both were previously "no native input". SMV is an ASCII "KEY=value;" block between braces, then the pixels - no container, no compression, nothing to decode by offset - so reader/SMV.{h,cpp} and JFJochSMVReader are a smaller job than the marCCD pair they sit beside, and need no new dependency at all. Two things the format does not give us, both said out loud rather than papered over: * It states no saturation value, so overloads are judged on the 16-bit container alone. That can only fail to call a pixel saturated, never condemn a good one, but a CCD at the top of its range does saturate, so the reader warns once. * Its beam centre is in MILLIMETRES and which of X/Y is the fast direction is a convention rather than a rule. Measured on one ALS ADSC sweep the file's value is TRANSPOSED: as stated it indexes 2/60 frames, and the run's own beam-centre measurement (which adopts the right one automatically) indexes 60/60. Swapping it here would fit that writer and might break another, so the header is read as the format defines it and the measurement stays the arbiter. Revisit with a second vendor's SMV in hand. .cbf.gz needed only Slurp() in MiniCBF.cpp, through which every read already passes: it sniffs the two-byte gzip magic - not the file name - and takes a zlib path when it is there, leaving the plain path free of zlib's buffer copy. zlib-ng is already in the build, so this is a link line, not a dependency. The sweep template grew a suffix, because ".cbf" and ".cbf.gz" are separate sweeps and std::filesystem cannot split the double extension on its own. The viewer's single cbf_reader becomes three, dispatched by CanRead() in the same order as rugnux. Dispatch is by CONTENT in both: ".img" is used by miniCBF, marCCD AND SMV depending on the writer, and a PDB detector label has now been wrong about the format four times, so an extension decides nothing. Measured, de novo, no flags: 9fcg (1800 gzipped frames) gives P4 and a cell 0.06% from the deposited one at 1.37 A against a deposited 1.54; 6oel (ADSC SMV) gives F4132 - 96 operations, the most a protein space group can have - and a cell 0.05% out, 100% indexed. Tests cover both formats and the transposed-beam-centre case with fixtures written byte for byte, so they need no external data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-16 22:35:05 +02:00
leonarski_fandClaude Opus 5	a1889c45e9	rugnux reads marCCD sweeps natively ... A decade of deposited CCD data is archived as marCCD - what Rayonix MX-series and mar Mosaic detectors write - and rugnux could not open any of it. reader/ had two formats, NXmx/HDF5 and PILATUS miniCBF, and rugnux_cli dispatched on the one CanRead(); this adds the third. The format needs no new dependency: a marCCD file is an ordinary uncompressed TIFF whose 3072-byte instrument header sits in the gap between the TIFF header and the pixels, so libtiff - already fetched for JFJochPreview in every build mode - reads the image, and the header is a fixed-offset block of little-endian int32. Two things differ from the miniCBF path and are worth naming: * The pixel size is NOT rounded to whole micrometres. A PILATUS pixel is exactly 172 um so the existing reader can afford lround(); a MAR300 pixel is 73.242 um, and rounding it to 73 is a 0.33% scale error on every cell edge reported. * The sweep template is the last run of digits in the whole file name rather than in the stem, which covers both schemes these detectors use - a numbered stem (xtal_1_00042.mccd) and the frame number as the extension (D1.042). A CCD frame marks no untrusted pixels, so the sweep starts with nothing masked, and the format has nowhere to state the rotation axis' direction, so the run settles its sign from the data exactly as it does for a miniCBF carrying no axis table. Measured on one deposited 300-frame Rayonix MX-300 sweep, de novo with no flags: 100% indexing, the deposited point group, cell within 0.045%, 99.5% complete at multiplicity 3.4, in 26 s. The chosen sweep is confirmed against the instrument header before it is opened, so a directory of ordinary TIFFs is refused rather than read with a pixel size of zero - a unit test covers that, both naming schemes, and the geometry conversion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-16 19:31:58 +02:00
leonarski_f	cb5a2f032a	v1.0.0-rc.170 (#80) ... * Fixed a `jfjoch_broker` crash during indexing: sorting no longer misbehaves on non-finite values, and GPU FFT indexer kernel launches are now error-checked. * rugnux needs about a third less peak memory to scale, merge and post-refine rotation data, with identical results. * `rugnux --model`: the placed coordinate file carries the space group its own coordinates obey, and says so when that is not the group the reflection files beside it carry. * `jfjoch_viewer`: fixes in the dataset plots, inspector and layout; spot markers lose their black outline by default (a checkbox under "Image features" restores it) and the highest-pixel markers are white boxes around the pixel. Reviewed-on: #80 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.170	2026-09-16 18:17:46 +02:00
leonarski_f	9aae0c2ba7	v1.0.0-rc.169 (#79) ... * Building Jungfraujoch no longer needs zlib or Eigen installed on the machine, and the dependencies the build fetches are pinned and updated to current releases. * rugnux: improvements in indexing, lattice selection and geometry post-refinement, which index crystals that previously returned no lattice and keep the better of the two geometries a run measures. * rugnux: improvements in beam-centre measurement, beam-stop detection and space-group determination. * rugnux: the unit cell reported with a determined space group now obeys that group - a cell whose symmetry was confirmed from the intensities is re-refined under it, and a cell the group cannot describe is reported with a warning rather than as it stands. * rugnux drops the stretches of a rotation sweep whose removal measurably improves the merged intensities and reports what became of every frame, and decides the resolution cut on the crystal's own diffraction rather than on its ice rings. * The rugnux results report is machine-readable - every line that is not `KEY= value` data starts with `#` - and states the build it was written by, its authorship and its terms of use (`REPORT_VERSION= 8`). * `jfjoch_viewer`: improvements in the file manager (CBF frames beside HDF5 datasets, a remembered root), the dataset plots, the inspector and the image statistics, plus a settable font size, a view of the rugnux results report, usable performance over a remote display (`ssh -X`) and a reset of all settings to defaults; the reciprocal-space window is removed. * Broker fixes around DECTRIS collections and dark-mask calibration: re-initialising after a run that never started no longer freezes the broker, a cancelled calibration is abandoned instead of reported as done, and a collection whose start message never arrives ends by itself. Reviewed-on: #79 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.169	2026-09-15 17:09:31 +02:00
leonarski_f	77bc0cfe52	v1.0.0-rc.168 (#78) ... * rugnux is substantially faster - a corpus of 145 rotation datasets processes in about two thirds of the time - with identical results. * A crystal whose lattice looks more symmetric than it is because the beam centre is off is no longer processed on the wrong cell. * rugnux prints at startup, and writes at the foot of every results report, a short acknowledgement of the X-ray research community whose methods it implements and of the open-source projects it builds on; `ACKNOWLEDGEMENT.md` now ships in every package beside `LICENSE` and `THIRD_PARTY_NOTICES.md`. Reviewed-on: #78 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.168	2026-09-10 13:51:16 +02:00
leonarski_f	a39fd29f77	v1.0.0-rc.167 (#77) ... * `rugnux --model` reports CC(model, data) - the correlation of the merged intensities with the placed, scaled model - by resolution shell, on the same shells as CC1/2, with the reflection count and a significance for each. * `rugnux --model` fits the model's scale, anisotropic B and bulk-solvent parameters on the working reflections only, so the R-free it reports is measured against a model no free reflection helped scale. * The bulk-solvent parameters of `rugnux --model` are searched over their physically meaningful range instead of being fitted without bounds, so a model is never scaled with a solvent term that has silently switched itself off. * The rigid-body placement of `rugnux --model` uses the same bounded bulk solvent as the reported fit, so a model is no longer placed against a target carrying a solvent term with no physical meaning. * `rugnux --model` puts the model into the data's own description of the lattice before placing it, so a model whose cell is written on other axes - I-centred where the run indexed C-centred, a different unique axis, a permuted orthorhombic cell - is placed rather than scored where it was read; `MODEL_CHANGE_OF_BASIS=` and `MODEL_SETTING_AS_READ=` report it when it happens. * The rugnux results report opens with a summary - `VERDICT=` (`OK`, `WARNINGS`, `UNUSABLE`, `FAILED`), `VERDICT_TEXT=`, `PATHOLOGY_FLAGS=` with one closed-vocabulary code per condition that warned, and the `WARNING:` lines, which used to close the file - and the sections after it are renumbered 1-5 with no gaps. * `rugnux --developer` writes the full results report - the pipeline-internal keys and the long explanations the default report now leaves out - and `--finalist-ledger` adds the evidence for every space group the search considered, not only the one it adopted. * The results report warns when the merged data carry no usable signal and when too little of reciprocal space was measured inside the fitted resolution, and omits `FITTED_RESOLUTION` where the CC1/2 curve it is fitted on never falls off. * rugnux detects translational pseudo-symmetry and reports it under the `PSEUDO_TRANSLATION` flag as `TNCS_DETECTED=` and the `TNCS_*` keys - a translation the merged data are exactly invariant under is reported as `UNDECLARED_LATTICE_TRANSLATION=` under `LATTICE_TRANSLATION` instead - and a detected pseudo-translation can no longer buy a false screw axis in the space-group search or hide a twin from the L-test (`L_TEST_VS_TNCS=`). * The space-group search determines glide planes from zonal systematic absences, so a non-Sohncke space group such as P 2_1/c or Pbca is named where the run previously stopped at its Sohncke subgroup; `SOHNCKE_SPACE_GROUP=` carries the best Sohncke group beside it on every run that searched, and a centre of symmetry is never claimed. * Where the cell metric carries more rotational symmetry than the Bravais class the indexer named, the extra rotations are put to the intensities and the space-group search is asked again on the metric's own cell - adopted only where the intensities confirm the higher symmetry - so a lattice that is nearly but not exactly hexagonal, or whose reduction landed in a sub-cell, still reaches its true point group. * Systematic-absence calls rest on the evidence rather than on counts: a screw axis whose absent class the data show extinct is no longer refused because a handful of reflections in it read as present, and `SPACE_GROUP_ALTERNATIVES=` no longer drops a candidate that differs only on a zone the sweep never measured. * A reference correlation measured on too few reflections is refused instead of scored zero, so a run given a reference MTZ is no longer reindexed on an operator that mapped almost everything outside the reference's coverage. * A frame counts as indexed from 6 spots on its lattice rather than 9, so a weakly diffracting crystal whose frames cannot carry 9 is no longer refused the lattice it fits; `--min-indexed-spots` overrides it. * `-C` accepts a known cell in any equivalent description - conventional or primitive, centred or not - instead of only the reduced primitive form, so a centred cell given the way it is published no longer makes the run report that it found no lattice. * Each reflection is corrected for the sensor's quantum efficiency at the angle it meets the detector (attenuation lengths from the NIST tables, which also fixes the spot-width parallax term on CdTe) and for the attenuation of the flight path between the sample and its pixel; `--flight-path air|helium|vacuum` declares the medium - default air, since no file states it - and the report says what was assumed and what it was worth. The unmerged MTZ records the factors in new `QE` and `FLIGHT` columns beside `LP`, so raw counts are `I / LP * QE * FLIGHT`, and `_process.h5` in new optional `qe` and `flight` datasets. * Rotation geometry post-refinement fits the crystal and the detector at once, against the observed spot positions and the observed rocking angles together, so the refined distance depends far less on how wrong the file's distance was. * A coarsely sliced sweep integrates correctly: partials are joined into one rocking event by angle rather than by frame count, so two crossings of the Ewald sphere are no longer summed into one full, and at 0.5 degrees per image or coarser the per-frame geometry refinement accepts a spot whose miss the exposure's own rotation accounts for. * `rugnux --mode scale` reports the detector tilt and direct beam of the geometry it re-scaled at, instead of zeros that read as a flat detector, and no longer warns that no image was indexed on a run whose lattice came from its input file. * Every rotation run that determined a space group and merged reports what the mounting cost: `SPINDLE_LOST_UNIQUE_FRACTION=` is the fraction (0-1) of unique reflections the mounting made unmeasurable under the measured point group, also written to the master as `/entry/MX/spindleLostUniqueFraction` and what the mounting warning fires on; `SPINDLE_SYMMETRY_AXIS_ANGLE_DEG=` / `SPINDLE_SYMMETRY_AXIS_ORDER=` describe the mounting in the `--developer` report. * Stills and grid scans carry a per-image `spindle_blind_fraction` - how much of a rotation sweep's blind cone this orientation would make unrecoverable, 0.5 and above calling for a second orientation - through the CBOR stream, HDF5 (`/entry/MX/spindleBlindFraction`), the plot and scan-result APIs, and the viewer and frontend plots; an absent value means the frame could not be assessed and is not a 0. * The results report's `REPORT_VERSION` is 7. Reviewed-on: #77 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.167	2026-09-09 07:25:13 +02:00
leonarski_f	680c36c20d	v1.0.0-rc.166 (#76) ... * `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands. * `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion. * Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants. * `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing. * A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed. * `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing. * Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences. * The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to. * The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after. * The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution. * `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have. * Twinning is no longer reported when the L-test contradicts it. * The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's. * `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots. * The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area. Reviewed-on: #76 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.166	2026-09-02 21:17:31 +02:00
leonarski_f	511be0c366	v1.0.0-rc.165 (#75) ... * `rugnux --model` adopts the model's space group as a label where the data were merged in its enantiomorph, instead of reindexing the reflections - which swapped I(+) with I(-). * `rugnux --model` warns, naming the atom, when the anomalous density at the model's atoms comes out inverted, which means the data and the model are in opposite hands. * `rugnux --model` writes an anomalous difference map (`<prefix>_anom.ccp4`) when the merge kept the Bijvoet split, and names the ten model atoms it peaks highest on as `ANOMALOUS_SITE_01`..`_10`. * `MEAN_ATOM_DENSITY_SIGMA` is read from the map by cubic rather than linear interpolation and comes out around a tenth higher; it is no longer comparable with the figure earlier versions printed. * `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name. * A model `rugnux --model` cannot use is reported as a `WARNING:` line in the results report instead of only in the log. * The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given; `REPORT_VERSION` is 4, `WARNINGS` moves to section 11 and no existing key changed. * The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=`, `WALL_TIME=` and `GPU_COUNT=` / `GPU=`. * rugnux says which GPUs it can see before it starts processing. * `rugnux --export-unmerged` also writes `<prefix>_unmerged.mtz` on a `--no-merge` run, and is ignored on a run with no output prefix instead of writing a file called `_unmerged.mtz`. * `/start` asks the writer whether the run can be written before the detector is armed, so a run whose master file already exists, or whose output directory cannot be created, is refused up front with the writer's own message. This needs the TCP image stream or the built-in HDF5 writer; the ZeroMQ stream is unchanged. * A calibration that fails goes to `Error` carrying the reason instead of `Inactive`, so `/wait_till_done` and `/wait_until_running` report it; a cancelled calibration still goes to `Inactive`. * `/wait_till_done` answers 500 with the message when a collection ended in an error. A cancelled collection and a collection that only triggered a warning still answer 200. * A pending start failure is discarded by `/cancel` and `/deactivate`, as it already was by `/start` and `/initialize`. * `/scan_result` no longer reports the previous run's images after a collection that failed to start, or after `/deactivate`. * The TCP image stream protocol version is 4. `jfjoch_writer` and `jfjoch_broker` have to be of the same release, as before. Reviewed-on: #75 1.0.0-rc.165	2026-08-27 22:16:54 +02:00
leonarski_f	749db470ca	v1.0.0-rc.164 (#74) ... * rugnux now tells you whether a crystal diffracts anisotropically and how far it reaches in each direction, without a second program: a new `9. DIFFRACTION ANISOTROPY` section in `<prefix>_report.txt` and matching `_reflns.pdbx_aniso_B_tensor_*` / `_reflns.jfjoch_aniso_*` items in the merged mmCIF report the anisotropic deltaB, the diffraction limit along each principal direction, and a `NOT DETECTED` / `DETECTED` / `CANNOT DETERMINE` verdict measured against the data set's own systematic error. It is a description only - no intensity is corrected, no reflection is removed, and the merged data do not depend on direction. * rugnux can hand its integrated observations to another scaling program: `--export-unmerged` writes `<prefix>_unmerged.mtz`, an unmerged MTZ readable by aimless, pointless, careless and `iotbx.merging_statistics`, in `--mode mx` and `--mode scale` alike. Each rotation reflection's partials are summed into one full; `--export-unmerged-partials` writes one row per image instead. Intensities carry the Lorentz-polarization factor and nothing else, since those programs scale the data themselves. Lattice-centring absences are not written; screw and glide absences are. * rugnux integrates crystals with broad spots better - where it changes anything, per-shell mean I/sigma improves by up to 31% and R_meas by up to 24% - because on rotation data the integration signal radius is now taken from the crystal's own measured spot width instead of a fixed 4 px. `--adaptive-integration-radius=off` restores the fixed radius and an explicit `--integration-radius` still overrides both. The widened radius applies to the final integration pass only, and a pattern too dense for it is re-integrated at 4 px with a note in the log. * rugnux discards fewer stills reflections for want of a background ring, improving per-shell R_meas over most of the signal-bearing range: the stills background ring now runs to 14 px instead of 12. The gain reverses in shells below a mean I/sigma of about 4. * rugnux determines the space group with thresholds that mean the same thing on a weak crystal as on a strong one: symmetry operators are scored on resolution-normalised intensities (E squared) instead of raw merged intensities, and a reflection counts as genuinely present on its counting significance instead of on the merged I/sigma, which saturates at the merge's own ISa. The search resolution cut is no longer able to move the answer, and the twin-law H bound moves from 1.70 to 1.85, which stops one class of correct high-symmetry assignment being refused as twinning. * rugnux says what the space-group search tested and what it could not: the twin-law disagreement H is printed for every operator together with the adopted point group's H ratio and its bound; alternatives that are not on the reported lattice are named with how their cell differs; and a lattice centring the data could not test - the crystal having been integrated on the primitive sub-cell, so the reflections it extinguishes were never measured - is marked `UNTESTED` and warned about where it is adopted, as coming from the lattice metric rather than from the intensities. * rugnux `--mode scale` re-merges a `_process.h5` in the right symmetry without being told it: the file now records the space group on every run - a two-pass rotation run wrote none before, so re-merging defaulted to P1 - together with the change of basis under `/entry/MX/reindexMatrix` where the lattice was re-seated, and `--mode scale` also reports the Wilson B-factor estimate instead of `WILSON_B= nan`. A file written before this stops with a message naming the two cells and the override to use, instead of failing inside the merge. A third-party reader of a `_process.h5` must apply `reindexMatrix` where it is present. * rugnux installs on its own, as a package called `rugnux` - `dnf install rugnux` or `apt install rugnux` - instead of arriving inside `jfjoch-viewer`. It pulls in none of the acquisition stack, so a machine that only processes data no longer has to carry the broker, the detector libraries or Qt to get it. Installing it over a `jfjoch-viewer` from rc.163 or earlier, which still owns `/usr/bin/rugnux`, upgrades cleanly rather than failing on the duplicate file. * rugnux is also a standalone download, built for arm64 as well as x86_64: `rugnux-<version>-linux-{x86_64|aarch64}-cuda<major>.tgz` and `rugnux-<version>-win64-cuda<major>.zip` on the release page, for machines that are not managed by a package manager. The aarch64 build targets GH200 and DGX Spark, and is untested on hardware. * Every portable Linux binary is now a single self-contained file: cuFFT is linked statically instead of being shipped beside the executable and found through an rpath, so `rugnux` and `jfjoch_viewer` need nothing but an NVIDIA driver, and only to use the GPU. The `.rpm`/`.deb` continue to take cuFFT from the distribution. The developer utilities `jfjoch_extract_hkl` and `jfjoch_recompress` are no longer packaged anywhere. * Jungfraujoch needs six fewer shared libraries on the machine - libopenblas and libmetis, and libgfortran, libquadmath, libgomp and libz behind them - because the Ceres LAPACK, METIS and SuiteSparse back-ends are no longer built. Nothing in the code ever selected them, and results are unchanged. * The PCIe driver DKMS package builds for the kernel it is being installed for instead of the running one, so a module built while a kernel update is being applied loads after the reboot. * The PCIe driver builds on RHEL 9.5 and later, and on their CentOS Stream, Rocky and AlmaLinux equivalents, where the `vm_flags` kernel interface was backported into the 5.14 kernel. * A data collection started with `async_start` that fails to start - a writer refusing to overwrite an existing file, for instance - is reported as an error by `/wait_until_running` and `/wait_till_done` instead of as a timeout and a successful collection respectively. The error message is the one the writer gave. * A calibration that is cancelled or that fails to collect its pedestals is no longer reported as a successful one. The broker goes to `Inactive` with an error message and has to be initialized again, instead of sitting in `Idle` looking ready to measure while holding partial pedestals - data collected in that state was silently mis-converted. * A failed `/initialize` is reported to `/wait_until_running` and `/wait_till_done` as soon as it happens, instead of when their timeout expires. * `space_group_number` accepts space groups up to 230 in the API schema, so cubic space groups can be recorded. The broker always accepted them; the generated clients rejected them before the request was sent. * The results report's `REPORT_VERSION` is 3, two sections having been added. Existing key names and table columns are unchanged. * The merged statistics table has **9** resolution shells instead of 10, which is what XDS reports. The bins were already XDS's - equal steps in 1/d^2 between the lowest- and the highest-resolution reflection the merge kept - so at the same resolution limits the two tables now have the same shell boundaries and can be read row for row. `--resolution-shells` sets a different count. * `rugnux --model` now settles the frame the merged reflections are written in, not only the frame the R-factors and the maps are computed in: the `.mtz`/`.cif`/`.hkl` come out in the model's indexing, and where the data were merged in the model's enantiomorph they take the model's hand and space group - which on anomalous data puts I(+) and I(-) the right way round. The indexing choice is logged with the winning R-free and the runner-up, so a decision made within noise is visible. * `rugnux --model` can resolve the indexing ambiguity of a **serial stills** run, which a model could not do before: structure factors computed from the model become the per-image reference, the same role a reference MTZ plays. It needs the cell and space group up front (`-C` / `-S`). Without one or the other, a merohedral serial run still merges both hands together and says so. * The rugnux documentation opens with a quick start - the default run, and runs with a reference MTZ, with a model, or with the space group and cell pinned - and explains the indexing ambiguity: what it costs on rotation and on serial data, and which of `-z` / `--model` resolves it in each case. The long reference pages now carry a table of contents. Reviewed-on: #74 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.164	2026-08-26 22:47:00 +02:00
leonarski_f	61c603b274	v1.0.0-rc.163 (#73) ... * Packaging: each package installs its license notices under a directory of its own - `share/doc/jfjoch_broker`, `share/doc/jfjoch_writer`, `share/doc/jfjoch_viewer`, `share/doc/jfjoch_driver_dkms` - instead of all of them into the shared `share/doc/jfjoch`, so `jfjoch` and `jfjoch-writer` can be upgraded one at a time instead of only together. Reviewed-on: #73 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.163	2026-08-25 13:38:29 +02:00
leonarski_fandjungfrau	4dc2534dbf	v1.0.0.rc-162 (#72) ... **Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one. * HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected. * HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it. * HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset. * HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts. * HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout. * A grid scan and a goniometer axis can both be set; they are no longer alternatives. * `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000. * The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned. * The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer. * rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it. * rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`. * rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way. * rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound. * rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached. * rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use. * rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own. * rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement. * rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged. * rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged. * Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged. * A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses. * rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given. * CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer. * The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2. * Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact. **Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`): * `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file. * `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them. --------- Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch> Reviewed-on: #72 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.162	2026-08-25 08:21:39 +02:00
leonarski_f	538f3504d3	v1.0.0.rc-161 (#71) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * **rugnux: significantly better quality of results, and faster.** A large rework of integration, scaling, merging, geometry refinement and space-group determination, together with measurements the program previously made no attempt at - the direct beam before indexing, the beam stop, the goniometer rotation scale, and the stretches of a sweep the crystal did not deliver. A rotation dataset typically gains observations at better <I/sigma> and R_meas, and every `mx` and `scale` run writes a `<prefix>_report.txt` results report modelled on XDS's `CORRECT.LP`. Many defaults moved with it: spot detection is self-calibrating, beam-stop detection and rotation geometry post-refinement are on, resolution limits default to as far as the detector reaches, and ice-ring handling engages only where the crystal is measured to have ice. * **jfjoch_viewer:** the beam-stop shadow, the detector calibration and the beam-centre measurement are reachable from "Analyze dataset"; the settings panel reports how the sample moved and how polarized the beam was; image rendering and interaction are faster. * **Performance:** bitshuffle+LZ4 images are decoded on the GPU rather than on the host, with the bitshuffle inverse fused into preprocessing so the decompressed frame is never held in device memory. * **Broker, writer, packaging and build:** image-slot lifetime and locking fixes, per-image datasets sized by the images actually written, the Debian/Ubuntu broker package renamed to `jfjoch`, and `image_analysis` compiling under MSVC again. **Breaking change to the rugnux command line:** * `--azint-only` and `--scale` are **removed**, replaced by `--mode azint` and `--mode scale`; the full pipeline is `--mode mx` and remains the default. A script passing the old flags now fails with the list of valid modes rather than silently running the wrong one. * `-t`/`--stride` is **refused on rotation data**: skipping frames cuts every reflection's rocking curve, so the combined fulls and their partiality would be measured over frames the sweep never recorded. Select a contiguous range with `-s`/`-e` instead. `--mode azint` and `--force-still` still take a stride. **Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional: * `image_scale_b` is removed from the `plot_type` enum, so a client requesting that plot now gets an error rather than a curve. * `azim_int_settings.high_q_recipA`, `spot_finding_settings.high_resolution_limit` and `spot_finding_settings.low_resolution_limit` are no longer `required`. All three mean "no limit at that end" when unset and are omitted from the response instead of carrying a placeholder value, which raises in a client generated from an rc.160-or-earlier spec. A value of 0 is still accepted and means the same thing. **Breaking changes to the stored formats** - a consumer reading these fields must treat them as optional: * The per-image image-scale B factor is no longer computed, so `/entry/MX/imageScaleBFactor` is absent from newly written HDF5 files and the corresponding key is absent from the CBOR DataMessage and END blocks. Files written by rc.160 and earlier still contain it and still open; nothing in the pipeline reads it any more. * `_reflns.jfjoch_diffrn_ISa` now carries the whole-range `1/sqrt(a*b)` that XDS's ISa denotes, and the error-model `a` and `b` are reported in XDS's convention; the strong-reflection asymptote moves to `_reflns.jfjoch_diffrn_ISa_asymptotic`. **A file written by an earlier version carries the asymptote under the plain `ISa` name.** Reviewed-on: #71 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.161	2026-08-13 17:03:10 +02:00
leonarski_f	67dca388bd	v1.0.0-rc.160 (#70) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell. * rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged. * rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set. * rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme. * rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free. * rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry. * Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md. * Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #70 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.160	2026-07-19 09:39:28 +02:00