diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 425a4552..8206f8cb 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -10,6 +10,7 @@ This is an UNSTABLE release. It includes many experimental features, as well as * Bragg integration: The local background ring is now made robust with a **high-side sigma clip** (`--background-clip `, default 4) instead of the symmetric trimmed mean, which is biased low on Poisson data and added ~5 counts to every partial. The trim stays reachable with `--background-trim `. Expect `` to fall and edge `R_meas` to rise - that is the removed bias, not a regression; per-shell agreement with independent processing improves. * Bragg integration: The **uncertainty of the background estimate** is now propagated into `sigma`; both engines omitted it, which understated every reflection's `sigma` by `sqrt(1 + n_signal/n_bkg)` = 1.109 with the shipped stencil. Expect `` to fall by that factor on every dataset. * Bragg integration: New **radial background correction** (`--background-radial[=on|off|auto]`, default off) for the bias a curved radial background leaves in a flat ring mean - tens of counts on a single reflection sitting on an ice ring. `auto` applies it per image where that image's ice score shows a *smooth* powder ring, which targets correctly (it fires on ten battery crystals, all ice-positive) but costs 1.35x the wall clock and is sign-mixed on the merge statistics, so it stays opt-in: measured against a fixed atomic model it removes 43 % of the ice bands' excess amplitude on smooth ice, but on ice made of discrete crystallite spots - where a radius-only background model has nothing to fit - it makes the bias worse, and the gate separates the two. +* Azimuthal integration: optional **sigma clipping** of the reported profile (`--azim-sigma-clip `, `azim_int_settings.sigma_clip`; default 0 = off, minimum 2). The profile is a per-bin mean, so strong reflections in a bin lift it exactly as a powder ring does; clipping repeats the integration twice more, rejecting pixels beyond *n* sigma of their bin's mean, and what survives is the smooth background under the peaks. Costs ~3x the azimuthal-integration time (0.02 -> 0.06 ms per image) and nothing else in the frame. Not for use where a ring's integrated intensity is wanted. * rugnux: **Ice-ring handling now runs only when the crystal is measured to have ice** - the eleven fixed bands cost 16-26 % of the unique reflections whether it does or not. Detection uses two channels, the spot finder's peak-excluded radial profile (`--ice-min-score`, default 1.5) and found spots on the rings against ice-free flanks (`--ice-min-spot-ratio`, default 2.0), which catch smooth and textured ice respectively; the previous score was read off the plain azimuthal profile, where strong reflections in a ring's bin score as ice. * rugnux: The merge-time **ice-ring mask has been removed**. Measured against anomalous peak height it returned -0.001 +- 0.018 sigma while deleting reflections whose mean `I/sigma` was *above* the dataset average, leaving `R_meas`, CC1/2 and ISa unchanged and the affected shell 17 completeness points short. Ice reflections are still kept out of the scale fit and the space-group search, as every established scaling program does, and are kept in the final merge. * rugnux: **Multiplicity was over-reported** by the rotation merge: the per-shell observation counter rode on the R_meas re-walk, which deliberately applies a wider filter than the merge, so observations the merge had excluded were counted against a unique count that excluded them - and a group the merge dropped entirely has a NaN resolution, which `ResolutionShells::GetShell` silently binned as the lowest-resolution shell rather than rejecting. Only counts were affected; intensities, sigmas, R_meas, CC1/2, completeness and ISa were always right. @@ -40,7 +41,7 @@ This is an UNSTABLE release. It includes many experimental features, as well as * rugnux / broker: rugnux exited before reading an image on hosts with more than 128 hardware threads, so the refinement thread count is clamped; a config without a `bragg_integration` block now gets the documented fixed `max_hkl` rather than deriving it per crystal; and the device decode time is reported as compression time again, so that plot and `/entry/profiling/compressionTime` are populated. * Packaging: The Debian/Ubuntu broker package is named `jfjoch`, not `jfjoch-jfjoch` (it declares the rename, so `apt upgrade` handles it); the DKMS driver package's `postinst`/`prerm` are executable, so the `.deb` runs them; and the Windows/macOS installers no longer carry the Linux-only Freedesktop menu entry and icon. * jfjoch_writer: `--verbose` no longer swallows the following argument, and the usage message names the root directory as a path. The `-H`/`--http_port` option has done nothing since the HTTP status interface was dropped. -* Build: `PROJECT_VERSION` is derived from the `VERSION` file rather than hardcoded, and the version is read from it once instead of in two places. +* Build: `PROJECT_VERSION` is derived from the `VERSION` file rather than hardcoded, and the version is read from it once instead of in two places; and the Bragg integrator no longer uses `M_PI`, which is not standard C++ and left `image_analysis` - which the Windows viewer builds - failing to compile under MSVC. * Docs: Document device-side bitshuffle+LZ4 image decoding in CPU_DATA_ANALYSIS.md; add a Release contents page (what each artefact ships, its CPU baseline, and how the CUDA builds behave without a GPU); correct the RHEL 9 repository URLs and package names; bring the rugnux (`--model`), viewer (processing jobs) and writer pages up to date. **Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional: diff --git a/docs/CPU_DATA_ANALYSIS.md b/docs/CPU_DATA_ANALYSIS.md index 6069fc09..a038e0c4 100644 --- a/docs/CPU_DATA_ANALYSIS.md +++ b/docs/CPU_DATA_ANALYSIS.md @@ -403,6 +403,10 @@ The output includes: This stage provides centering information used for systematic absences in prediction (§8.4) and for reporting. +**A metric symmetry has to earn itself.** The class is chosen from the *unrefined* candidate against a fixed angular tolerance (3°), so a lattice that is pseudo-symmetric to a few tenths of a degree is promoted a class too far — and the constraint then snaps a real angle to the ideal one, which throws nearly every reflection of every frame outside tolerance. Measured on a monoclinic crystal pseudo-C-orthorhombic to 0.42°, the promoted cell indexed 2 of 60 frames where its own primitive cell indexed 39: the same lattice, $\mathbf{b}_{oC}=-(\mathbf{a}+2\mathbf{c})$, at exactly twice the volume. Note the direction of the trap — **more accurate candidates make it worse**, because a run escapes only when the raw candidate is inaccurate enough to miss the promotion window. + +For rotation data the first pass therefore refines the constrained cell *and* an unconstrained (triclinic) one from the same spots — which it finds itself, over a sample spread across the sweep, rather than reading what the acquisition wrote — and settles the two on how many of a fixed set of validation frames each actually indexes. The bar is a clear majority rather than a margin: an unconstrained refinement holds no cell parameter fixed, so it can only index at least as many frames, and on genuine symmetry it does index a few more. Only a constrained cell that fails outright while its unconstrained cell works is evidence of a false promotion. That asymmetry is what keeps real symmetry — a 10 % margin, tried, demoted a genuine $I$-centred orthorhombic lattice to $P1$. The intensities settle the space group later regardless (§13). + **Note.** In ambiguous or special cases, forcing space group to $P1$ (no symmetry assumptions) is recommended. ---