4 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	abfdb89b79	Beam centre: measure the direct beam from the symmetry of the spots, before indexing ... The centre in the file is often a placeholder, and nothing measures it until post-refinement has already indexed the sweep - by which time a wrong centre has chosen the lattice. Two exact facts about a rotation sweep give it from spot positions alone, with no cell, no orientation matrix and nothing indexed. Rotating 180 degrees about the spindle and taking -h negates a reflection's component along the spindle and leaves the rest, so with the spindle perpendicular to the beam the Laue condition is preserved and the spots recorded half a turn apart are mirror images along the spindle. Those are Friedel mates, not the same reflection. The same reflection appears twice for a different reason: it meets the Ewald sphere on two crossings, generally not half a turn apart, differing only in the sign of the component perpendicular to both the spindle and the beam. The first observable gives the coordinate along the spindle, the second the coordinate across it. Each candidate pairing votes and the true value accumulates while wrong pairings scatter. Both observables need guarding, because a vote is a comb and the tallest tooth is not always the right one. Along the spindle a false pairing cannot fake the equality of Friedel amplitudes. Across it, the two crossings of one reflection are separated by a sweep angle its own position fixes, which no accidental pair reproduces. The mirror is exact in the laboratory frame, so it is only as good as the rotation axis. Every file here states an ideal axis and none of them has one; a skew about the beam spreads the vote instead of shifting it, and past a milliradian it moves an otherwise correct answer by pixels while every internal statistic still looks healthy. It is therefore fitted, not assumed. A tilt of the axis towards the beam is measured and reported but not applied, being confounded with the detector rotation until that is fitted too. Nothing inside the fit can see a wrong tooth - when the vote flips, every frame pair flips with it - so the answer is checked from outside, by asking whether it depends on where the search began. That, and a floor on the angular span the pairs cover, are what refuse the cases this cannot measure: a sweep barely past half a turn is the dangerous one, not the short one, because at exactly half a turn there is nothing to fit and just past it there is almost nothing. Where the sweep is too short for any of this the radial background profile gives a coarser centre from a handful of images, and where neither can measure it the file's value is kept. The beam-stop projection now takes its own frames rather than sharing the sample, so turning this on cannot change the mask; and both samples keep away from the ends of the sweep, where shutter synchronisation spoils an image. Reading twice as many frames as before costs a few seconds once, and is what makes the answer independent of which frames were drawn. Off by default. Over the 38-crystal rotation battery it serves every dataset, agrees with XDS's refined direct beam to 0.116 px in the median against 0.135 for the value in the file, and changes no space group. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-12 23:54:09 +02:00
leonarski_fandClaude Opus 5	4d3434e2a5	Beam stop: compare each pixel only against its own ring ... The background belongs to the beam and the shadow to the stop, and the two are not concentric - fitting the stop edge per azimuth gives offsets of 13.4 px on an 85.8 px disk, 22.2 px on 67.3 px and 6.9 px on 23.7 px, 8 to 33 per cent of the stop radius on every crystal measured. The finder bridged that gap with a radial envelope, the largest ring background over an outward window, used as the reference for an individual pixel. That quantity exceeds the local background wherever the background rises outward, so sound pixels near the stop scored below the penumbra threshold and were masked. Measured against the fitted edge on a long-distance disk stop, the mask was displaced rather than mis-sized: short by up to 20 px on one side, over-reaching by up to 45 px on the other, with eight of twenty-four azimuth sectors falling short. The ring median is already the right reference wherever a ring still has unshadowed pixels to measure, which is every ring except those lying wholly inside the disk - and it needs no assumption about where the stop sits. So the envelope is gone from the per-pixel test, and the rings it existed to cover are handled directly: walking outward, a ring whose background is a fraction of the background further out is shadow in its entirety. That comparison is only ever asked whether a whole ring is inside the stop, never to judge a pixel, which is where its failure mode lives. Blockage is deliberately not a counting test - on a bright dataset the shadow interior is still well counted. Detection is now one channel instead of two, and 113 lines shorter. Measured: no azimuth sector falls short by more than 3.4 px, over-reach drops on all three fitted crystals, and mask area moves by at most 0.04 per cent of the detector on six crystals, so this corrects the shape rather than resizing. Battery: space-group agreement with XDS unchanged at 34/37, median change in R_meas and in the lowest shell 0.000 pp. The crystal that suffered worst when masking was introduced recovers to its unmasked quality - R_meas 25.1 -> 17.2 per cent, ISa 4.45 -> 10.04 - which is what removing the over-masking should do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-09 05:31:57 +02:00
leonarski_fandClaude Opus 5	a29c36600f	Beam-stop shadow detection, and a low-resolution limit for scaling ... rugnux finds the beam stop and its holder in a projection of 60 images and marks them in the pixel mask as bit 9 (--detect-beam-stop[=N|off], on by default). Reflections behind the stop are attenuated but not flagged, so they integrate low with a plausible sigma and nothing downstream catches them: the signal-box gate requires 100% valid pixels and shadow pixels are valid, the background clip is high-side only, and the |zeta| cut applies only to the space-group search merge. The detection compares each pixel's background against the typical background at the same radius on two channels. An azimuthal one (the ring median) finds the holder arm, which is a minority of its ring; a radial one (the background just outside) finds the disk, which the ring median cannot see because inside a fully blocked ring the median is the shadow itself. Pixels are pooled over a 5x5 box and tested only where the background has actually been counted, so low-background data no longer masks the whole detector. Recorded reflections are carved back out - a beam stop cannot block a reflection that was measured. Bit 9 belongs to the run that found it, not to the dataset: it is cleared when a run starts, so a mask read back from a file that carries one starts clear. The user mask (bit 8) is left alone. Scaling and merging gain a low-resolution limit, default 50 A (--scaling-low-resolution <num>, 0 removes it), applied per observation before scaling so it also protects the per-frame scale fit and the space-group search. 50 A is the value XDS configurations use; rugnux_vs_xds.py now matches both of XDS's resolution limits instead of only the high one, so the lowest shell is the same shell in the two programs. The viewer draws the detected shadow in coral with a "Show beam stop" switch in the side panel, exposes the low-resolution limit in the settings dock, and offers detection in its processing jobs. Adding an image marker meant giving the reader a MIN_REAL_PXL_VALUE, because several places classify a pixel by range rather than by equality and would otherwise read the new marker as a very negative intensity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-09 01:05:31 +02:00
leonarski_f	67dca388bd	v1.0.0-rc.160 (#70) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell. * rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged. * rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set. * rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme. * rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free. * rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry. * Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md. * Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #70 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-07-19 09:39:28 +02:00