docs: cover the azimuthal sigma clip, the indexing flag, and the metric-symmetry check
Build Packages / build:windows:nocuda (push) Successful in 17m17s
Build Packages / build:windows:cuda (push) Successful in 21m20s
Build Packages / build:viewer-tgz:cpu (push) Successful in 11m54s
Build Packages / build:viewer-tgz:cuda (push) Successful in 12m55s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 16m59s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m19s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 19m55s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 16m47s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 19m57s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 16m31s
Build Packages / build:rpm (rocky9) (push) Successful in 17m38s
Build Packages / build:rpm (rocky8) (push) Successful in 19m29s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 19m16s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 17m38s
Build Packages / Generate python client (push) Successful in 11s
Build Packages / DIALS test (push) Successful in 15m9s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 9m32s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m24s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m20s
Build Packages / Unit tests (push) Successful in 1h19m27s
Build Packages / build:windows:nocuda (push) Successful in 17m17s
Build Packages / build:windows:cuda (push) Successful in 21m20s
Build Packages / build:viewer-tgz:cpu (push) Successful in 11m54s
Build Packages / build:viewer-tgz:cuda (push) Successful in 12m55s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 16m59s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m19s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 19m55s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 16m47s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 19m57s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 16m31s
Build Packages / build:rpm (rocky9) (push) Successful in 17m38s
Build Packages / build:rpm (rocky8) (push) Successful in 19m29s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 19m16s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 17m38s
Build Packages / Generate python client (push) Successful in 11s
Build Packages / DIALS test (push) Successful in 15m9s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 9m32s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m24s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m20s
Build Packages / Unit tests (push) Successful in 1h19m27s
Three things had reached the code without reaching the documentation. The azimuthal sigma clip had a RUGNUX.md row and a CPU_DATA_ANALYSIS section but no changelog entry - and the only "sigma clip" the changelog mentioned was the background ring's, which is a different thing at a different stage. --index-ice-rings was in the options table but nowhere in the changelog, so the entry describing the ice gate still implied that whether indexing uses the ice-band spots is decided per run, which it no longer is. CPU_DATA_ANALYSIS section 6 still described the Bravais class as simply "the highest-symmetry class that matches within tolerances", which is the behaviour that lost a crystal outright. It now records that the class is chosen from the UNREFINED candidate against a fixed angular tolerance, that a pseudo-symmetric lattice therefore gets promoted a class too far, and that the first pass settles it on validation-frame counts with a clear-majority bar - including why the bar is a majority rather than a margin, since that distinction is the whole reason the check is safe. It also records that the first pass finds its own spots rather than reading the acquisition's, which was not written down anywhere. Also a build note: M_PI is not standard C++ and MSVC does not define it, so the Bragg integrator's use of it broke the Windows viewer build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+2
-1
@@ -10,6 +10,7 @@ This is an UNSTABLE release. It includes many experimental features, as well as
|
||||
* Bragg integration: The local background ring is now made robust with a **high-side sigma clip** (`--background-clip <n>`, default 4) instead of the symmetric trimmed mean, which is biased low on Poisson data and added ~5 counts to every partial. The trim stays reachable with `--background-trim <f>`. Expect `<I/sigma>` to fall and edge `R_meas` to rise - that is the removed bias, not a regression; per-shell agreement with independent processing improves.
|
||||
* Bragg integration: The **uncertainty of the background estimate** is now propagated into `sigma`; both engines omitted it, which understated every reflection's `sigma` by `sqrt(1 + n_signal/n_bkg)` = 1.109 with the shipped stencil. Expect `<I/sigma>` to fall by that factor on every dataset.
|
||||
* Bragg integration: New **radial background correction** (`--background-radial[=on|off|auto]`, default off) for the bias a curved radial background leaves in a flat ring mean - tens of counts on a single reflection sitting on an ice ring. `auto` applies it per image where that image's ice score shows a *smooth* powder ring, which targets correctly (it fires on ten battery crystals, all ice-positive) but costs 1.35x the wall clock and is sign-mixed on the merge statistics, so it stays opt-in: measured against a fixed atomic model it removes 43 % of the ice bands' excess amplitude on smooth ice, but on ice made of discrete crystallite spots - where a radius-only background model has nothing to fit - it makes the bias worse, and the gate separates the two.
|
||||
* Azimuthal integration: optional **sigma clipping** of the reported profile (`--azim-sigma-clip <n>`, `azim_int_settings.sigma_clip`; default 0 = off, minimum 2). The profile is a per-bin mean, so strong reflections in a bin lift it exactly as a powder ring does; clipping repeats the integration twice more, rejecting pixels beyond *n* sigma of their bin's mean, and what survives is the smooth background under the peaks. Costs ~3x the azimuthal-integration time (0.02 -> 0.06 ms per image) and nothing else in the frame. Not for use where a ring's integrated intensity is wanted.
|
||||
* rugnux: **Ice-ring handling now runs only when the crystal is measured to have ice** - the eleven fixed bands cost 16-26 % of the unique reflections whether it does or not. Detection uses two channels, the spot finder's peak-excluded radial profile (`--ice-min-score`, default 1.5) and found spots on the rings against ice-free flanks (`--ice-min-spot-ratio`, default 2.0), which catch smooth and textured ice respectively; the previous score was read off the plain azimuthal profile, where strong reflections in a ring's bin score as ice.
|
||||
* rugnux: The merge-time **ice-ring mask has been removed**. Measured against anomalous peak height it returned -0.001 +- 0.018 sigma while deleting reflections whose mean `I/sigma` was *above* the dataset average, leaving `R_meas`, CC1/2 and ISa unchanged and the affected shell 17 completeness points short. Ice reflections are still kept out of the scale fit and the space-group search, as every established scaling program does, and are kept in the final merge.
|
||||
* rugnux: **Multiplicity was over-reported** by the rotation merge: the per-shell observation counter rode on the R_meas re-walk, which deliberately applies a wider filter than the merge, so observations the merge had excluded were counted against a unique count that excluded them - and a group the merge dropped entirely has a NaN resolution, which `ResolutionShells::GetShell` silently binned as the lowest-resolution shell rather than rejecting. Only counts were affected; intensities, sigmas, R_meas, CC1/2, completeness and ISa were always right.
|
||||
@@ -40,7 +41,7 @@ This is an UNSTABLE release. It includes many experimental features, as well as
|
||||
* rugnux / broker: rugnux exited before reading an image on hosts with more than 128 hardware threads, so the refinement thread count is clamped; a config without a `bragg_integration` block now gets the documented fixed `max_hkl` rather than deriving it per crystal; and the device decode time is reported as compression time again, so that plot and `/entry/profiling/compressionTime` are populated.
|
||||
* Packaging: The Debian/Ubuntu broker package is named `jfjoch`, not `jfjoch-jfjoch` (it declares the rename, so `apt upgrade` handles it); the DKMS driver package's `postinst`/`prerm` are executable, so the `.deb` runs them; and the Windows/macOS installers no longer carry the Linux-only Freedesktop menu entry and icon.
|
||||
* jfjoch_writer: `--verbose` no longer swallows the following argument, and the usage message names the root directory as a path. The `-H`/`--http_port` option has done nothing since the HTTP status interface was dropped.
|
||||
* Build: `PROJECT_VERSION` is derived from the `VERSION` file rather than hardcoded, and the version is read from it once instead of in two places.
|
||||
* Build: `PROJECT_VERSION` is derived from the `VERSION` file rather than hardcoded, and the version is read from it once instead of in two places; and the Bragg integrator no longer uses `M_PI`, which is not standard C++ and left `image_analysis` - which the Windows viewer builds - failing to compile under MSVC.
|
||||
* Docs: Document device-side bitshuffle+LZ4 image decoding in CPU_DATA_ANALYSIS.md; add a Release contents page (what each artefact ships, its CPU baseline, and how the CUDA builds behave without a GPU); correct the RHEL 9 repository URLs and package names; bring the rugnux (`--model`), viewer (processing jobs) and writer pages up to date.
|
||||
|
||||
**Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional:
|
||||
|
||||
@@ -403,6 +403,10 @@ The output includes:
|
||||
|
||||
This stage provides centering information used for systematic absences in prediction (§8.4) and for reporting.
|
||||
|
||||
**A metric symmetry has to earn itself.** The class is chosen from the *unrefined* candidate against a fixed angular tolerance (3°), so a lattice that is pseudo-symmetric to a few tenths of a degree is promoted a class too far — and the constraint then snaps a real angle to the ideal one, which throws nearly every reflection of every frame outside tolerance. Measured on a monoclinic crystal pseudo-C-orthorhombic to 0.42°, the promoted cell indexed 2 of 60 frames where its own primitive cell indexed 39: the same lattice, $\mathbf{b}_{oC}=-(\mathbf{a}+2\mathbf{c})$, at exactly twice the volume. Note the direction of the trap — **more accurate candidates make it worse**, because a run escapes only when the raw candidate is inaccurate enough to miss the promotion window.
|
||||
|
||||
For rotation data the first pass therefore refines the constrained cell *and* an unconstrained (triclinic) one from the same spots — which it finds itself, over a sample spread across the sweep, rather than reading what the acquisition wrote — and settles the two on how many of a fixed set of validation frames each actually indexes. The bar is a clear majority rather than a margin: an unconstrained refinement holds no cell parameter fixed, so it can only index at least as many frames, and on genuine symmetry it does index a few more. Only a constrained cell that fails outright while its unconstrained cell works is evidence of a false promotion. That asymmetry is what keeps real symmetry — a 10 % margin, tried, demoted a genuine $I$-centred orthorhombic lattice to $P1$. The intensities settle the space group later regardless (§13).
|
||||
|
||||
**Note.** In ambiguous or special cases, forcing space group to $P1$ (no symmetry assumptions) is recommended.
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user