5 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	da74197dea	Stills partiality: an unmeasurable CC is not a reason to adopt the refined tilt ... The "keep what the crystal came in with" gate required std::isfinite(cc) before it would reject, so a refined model whose CC could not be measured at all was adopted. ImageReferenceCC returns NaN when fewer than 20 reflections clear the partiality cut - which is exactly what a refinement that collapsed the partialities produces, since the cut is on the partialities it just rewrote. The gate therefore failed open on precisely the crystals it exists to catch, and wrote the NaN into image_scale_cc, on which --min-image-cc then drops the image from the merge, the error model and the statistics. Treat a CC that cannot be measured as worse than one that can, so the crystal is put back exactly as it arrived. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-31 11:51:08 +02:00
leonarski_fandClaude Opus 5	3e56d96921	Stills partiality: adopt the refined tilt only when it fits better ... RefineOne re-measured the image's correlation to the reference after writing the refined partialities - because --min-image-cc drops images by it - and then ignored what it measured. A crystal the tilt model suits worse than the fixed partiality it replaces kept the refined model anyway, and the refinement is on by default. Compare against the CC the crystal arrived with and put it back untouched when the refinement does not improve it, which is the same state a crystal with too few reflections to fit ends in. Also four things noted in review and left until now: AdaptiveThresholdTest.cpp was listed twice in the test target, AdaptiveThreshold.h was the one header in image_analysis/spot_finding not in its library's source list, CLAUDE.md said update_version.sh rewrites VERSION when it only reads it, and the CHANGELOG did not mention that image_scale_b is gone from the plot_type enum - which breaks a client that asks for that plot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 22:03:18 +02:00
leonarski_fandClaude Opus 5	f90969ea21	Stills partiality: do not adopt a tilt from a failed solve ... The Ceres summary was discarded, so a solve that diverged or aborted left its last iterate in psi and that tilt was written onto the partiality of every reflection of the crystal. Restore the tilt the crystal came in with and stop refining it; the scale fit alone is still a usable model, which is what the other three early returns in this function fall back to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-30 11:14:26 +02:00
leonarski_fandClaude Opus 5	16bf3408f0	Address code-review findings; make detection limits detector-driven ... One changeset, developed together in response to a review of this branch, so the files carry several of the changes at once. Full test suite passes (733 cases). Spot finding - Split ImageSpotFinder into Detect() (flag strong pixels - the expensive per-pixel pass) and ExtractSpots() (CCL + min/max-pix + resolution mask), with Run() = both. The per-image min-pix escalation now detects ONCE and repeats only the cheap extraction, instead of re-running the whole finder four times per frame as it did on the default path. It also keeps the winning attempt's spot list rather than re-extracting it, so the frame that is integrated is exactly the frame that was scored - which a GPU re-extract could not guarantee (float atomic ordering). - spot_finding_time_s no longer swallows indexing time, and indexing_time_s now sums every escalation call instead of reporting only the last. Detection limits follow the detector - The azimuthal-integration upper q and the spot-finding high-resolution limit are now std::optional, in the C++ structs AND in the OpenAPI schema, and resolve to the detector's own maximum (DiffractionExperiment::GetDetectorMaxQ_ recipA). Adaptive detection reads a pixel's ring from the azimuthal bins, so a pixel outside that q range could never be strong - the integration range silently bounded what detection could see, regardless of the requested resolution limit. Regenerated the C++ and TypeScript clients; the viewer and the web frontend each gained a "to detector edge" switch. Detection defaults are now per workflow (measured, not assumed) - Stills: adaptive detection, min-pix chosen per image, no resolution clipping. - Rotation: fixed-threshold finder, min-pix 2, 1.5 A limit. On a 33-crystal rotation battery, adaptive detection helped four hard crystals but deterministically broke three (a lost space group, a halved indexing rate, a collapsed merge), and the detector-edge limit cost indexing on a strong rotation set (100.0 -> 96.8%). Each is still overridable by its flag, and --no-adaptive-spots is new. Indexer seed escalation - Stop escalating once a seed's lattice explains >= 90% of the seed spots. Previously any frame with >= 80 spots always paid three indexer calls, online broker included. Merge-consistency filter - --min-image-cc gated on a per-image CC computed BEFORE the stills partiality post-refinement and never refreshed; the refiner now recomputes it, so the reported CC describes the data that are actually merged. - Replaced the per-call cc_mask argument with one MergeOnTheFly flag, so the merge, the error model and MergeStats can no longer disagree about which images are in (the --scale path merged unfiltered while its statistics were filtered). Per-image B-factor refinement (-B) removed - Measured on four serial-stills datasets: it is a no-op where the per-image fit is well conditioned and actively harmful where it is not (CC1/2 -8.1, R_meas +23.2 on the weakest large-cell set, whose fits hit their [-50, 200] bounds on 14-25% of images). It had also been silently DISCARDED since the partiality post-refinement landed - reported but not applied. Rather than fix and keep a knob with no demonstrated benefit, the flag and the whole image_scale_b_factor chain are gone: setting, scaling fit, message field, CBOR, HDF5 write and read-back, per-image plot, OpenAPI enum, viewer column and checkbox, docs. ScaleOnTheFly no longer needs Ceres at all - the fit is a linear IRLS. (The Wilson per-image b_factor is a different quantity and stays.) Stills partiality width now fits both of its components - sigma^2 = gamma0^2 + (gamma_e*d*)^2 instead of a purely angular gamma_e*d* with gamma0 pinned to 0. Fitted per crystal by least squares of dist_ewald^2 on d*^2. The angular-only width is fitted over a d*^2-dense population, so it was pinned by the high-resolution edge and collapsed at low d*: median partiality 0.008 beyond 13 A for reflections that were plainly recorded, 55% of them under the merge's partiality floor, and the survivors divided by those values - which inflated the merged low-resolution intensity scale 3.6x (~ +9 A^2 of apparent B). Measured on 5000 stills: the ramp flattens to 0.89x, no observation is dropped any more (701750 -> 716811), shell-mean CC1/2 and R-free improve slightly. Note CC1/2, R_meas, completeness and a B-refining R-free are all blind to that ramp, which is why it survived earlier validation; the cost is high-resolution R_meas (98.5 -> 101.9 shell-averaged). Removed dead code from add-then-remove churn - Prediction-time "still partiality" (unreachable: no setter), the phantom IndexingSettings::min_indexed_spot_fraction knob (getter, no setter - now the constant it always was), StillsPartialityRefine's caller-less Settings constructor and its reference to a long-gone env var, ProcessImage's unread bool return, an unused include, and a dead viewer overlay hook. Also - Viewer: the magnifier compared a QImage with itself, so its scene rect was set once ever and it could not pan into a larger dataset; the hover tail timer could fire after leaveEvent and resurrect the resolution readout outside the image. - update_version.sh regenerated the frontend lock file BEFORE bumping the version (every release shipped an off-by-one lock), and did git rm/git add on a path that has not existed since the client moved to src/client - with no set -e, both failed silently. - fpga/pcie_driver/postinstall.sh tested "[ ! occurrences > 0 ]", which is a redirect, not a test, so dkms add never ran. - Unit tests for the adaptive-threshold host functions, which had none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-27 09:07:00 +02:00
leonarski_fandClaude Opus 4.8	1a2b0181a5	Add physical partiality post-refinement for stills (default on) ... Replace the frozen scalar-sigma stills partiality with a physical, refined model. Per crystal, refine an orientation tilt (dpsi_x, dpsi_y) against the running merge and recompute each reflection's partiality analytically from the refined geometry (angular Ewald-proximity model, sigma(d*) = gamma_e*d*), with the per-crystal scale G profiled out by the existing robust IRLS - no re-integration. A soft Gaussian prior on dpsi tames weak-data overfit while staying inert on strong data. The merge <-> refine loop iterates a few times. This is now the stills default via ScalingSettings::stills_partiality_refine (on). A single opt-out flag `--simple-stills` reverts to treating every reflection as a full (p=1, single pass). Retires the experimental `--still-partiality` flag. The viewer gains a "Partiality post-refinement (stills)" checkbox in Scaling settings. Validated (integrate-once / --scale): CC1/2 and R_meas both improve on three monochromatic serial-stills datasets (+2.8 / -10, +5.6 / -3.4, +2.1 / -4); neutral on a pink-beam DMM set (already-full reflections); R-free/R-work down vs a fixed model; competitive with CrystFEL partialator on matched frames. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 14:45:43 +02:00