v1.0.0-rc.167 #77

Merged

leonarski_f merged 75 commits from rc167-no-grid into main 2026-09-09 07:25:13 +02:00

Conversation 0 Commits 75 Files Changed 301

+12011 -1957

75 Commits

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	59ba690d10	diagnostics: report a beam that carries its monochromator's third order ... A Si(111) monochromator passes lambda/3 with lambda, and a crystal in that beam writes a second diffraction pattern that no single lattice explains. Read at the fundamental it looks like a reciprocal lattice scaled by 1/3 about the beam centre, which is easy to mistake for a second crystal - one dataset in the corpus was diagnosed as needing a multi-lattice indexer on exactly that evidence. The test is exact and costs nothing extra. A spot's reciprocal vector read at the stated wavelength is q = (s_hat - z_hat)/lambda, so at lambda/n it is n*q, and n*q sits on the Ewald sphere of that wavelength by construction. Each frame's orientation is fitted by least squares on the spots the indexer took, and every spot it did not take is asked whether 3*q is an integer point of that same lattice. Spots already within 0.20 of a lattice point are not asked: those are reflections the indexer only just rejected, they land on an integer at every integer scale, and on a clean run they are most of the unindexed list. The null is the same question at a scale no monochromator passes, and it is a real null - the distance from a random point to the nearest lattice point does not depend on the scale. Measured over five rotation sets, 200 images each: on the one whose depositors report a harmonic, 89% of the tested spots are lattice points at lambda/3 against 0.03% for the null, and the population sits at 6-19 A apparent resolution - a third of that in truth, which is where a harmonic must put it. On four sets with no reported problem the same number is 0.0-0.9%, at or below each set's own null. This does not separate a contaminated beam from a crystal whose true cell is three times the one indexed; the arithmetic is identical. What separates them is where the population stops, so the apparent resolution of the hits is reported beside their number and the prose names the alternative. Report-only: nothing downstream reads it, and no spot is treated differently for it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GxZqDiFP3KqriBhNdcR56	2026-09-09 00:30:41 +02:00
leonarski_fandClaude Opus 5	cee97c55c2	beam centre: the background fit runs to its fixed point, not to ten iterations ... The shift a sector's regression can report is bounded by the width of the features it reads: a sector whose radial profile has moved a long way is not g + d*g' for any d, and the least-squares projection onto g' returns far less than the true d. The walk therefore advances by a bounded distance per iteration - about twenty pixels after damping - however far it still has to go, so MAX_ITERATIONS = 10 was a hard cap of roughly two hundred pixels of travel rather than a safety net around a converging fit. A sweep whose header named the detector's middle while the beam sat 351 px away showed exactly that: the fit walked steadily toward the answer and was cut off still moving 19 px a step, 145 px short, and reported the per-iteration precision of its last step - three pixels - as though it had arrived. Instrumented, the step never decays; it is a truncated walk and not a fixed point. Lifting the cap converges in 58 iterations to 1.4 px of the centre that indexes every frame, and the run then adopts it and merges to 1.70 A. Undamped is not faster (81 iterations against 70), so DAMPING is unchanged. Two changes: MAX_ITERATIONS becomes a travel budget large enough to cross a detector, documented as such; and the returned sigma takes in the step the fit still wanted to make, so one that leaves on the cap widens instead of claiming the precision of a step it was in the middle of. A converged fit stops at 0.02 px, well under any sigma worth reporting, so neither touches one. Measured over eight passing rotation sets on five detector sizes and 0.87-2.07 A: five converged inside ten iterations already and are bit-identical; three had been silently truncated and their centre moves by 0.0-0.25 px, with space group, cell, resolution and indexing rate unchanged throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GxZqDiFP3KqriBhNdcR56	2026-09-09 00:30:41 +02:00
leonarski_fandClaude Opus 5	54a7648eae	symmetry: a pseudo-translation is located by refinement and measured by class means ... The half-integer translation an axial absence is judged against was both assumed and measured on the same idealised vector. Only the component ALONG the row is fixed by the physics at 1/2; a real crystal's translation sits wherever its molecules do, and where the transverse components are not half-integer the parity split by that idealised vector scrambles off the row and reads no modulation on a row that is suppressed eight-fold. On a monoclinic crystal whose translation refines to (1/2, 0.46, 0.03) the parity ratio reads 0.9-1.2 in every shell above the first, and a spurious screw on the unique axis survives on 44 nats of evidence and 3 violations in 31, one violation under the bound. Pin the row's own component at 1/2, refine the other two on the merged intensities from each half-integer start, and read the depth per shell as the mean E^2 of the reflections the translation places near phase 1/2 over the mean of those it places near phase 0. The depth is deliberately NOT read from the fit that locates the vector. For a real modulation seen through a regressor that is set slightly wrong, the least-squares slope is inflated by <cos>/<cos^2>, which exceeds 1 for ANY error in the vector - so a mis-set vector reports a DEEPER modulation than the truth, and a search that maximises the summed slope is pulled towards exactly that error. Measured on an orthorhombic crystal whose translation really is the half-integer one, that search leaves the truth and settles symmetrically 0.012 to either side, where the depth reads zero against a true 0.03-0.13; a depth of zero divides into the absence test and takes the crystal's genuine screw with it. A ratio of class means cannot be inflated that way, and at a half-integer vector it is the parity split exactly. The running maximum with resolution is kept: a crystal can carry both a genuine order-two screw and a translation that dies away by 2.4 A, and the correction has to die with it. Measured on the nine crystals that engage the correction. The monoclinic crystal above adopts its deposited group: the false screw goes from 3 violations in 31 to 15 and from 44 nats to ineligible, refused by the count and by the evidence independently. The orthorhombic crystal that carries both keeps all three of its screws, its modulated zone paying 10 of its 53 nats. The other seven do not move, and every one of the nine matches its deposit. No merged statistic changes anywhere: a screw and its parent share a Laue class, so the merge is byte-identical and only the label moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GxZqDiFP3KqriBhNdcR56	2026-09-08 22:38:59 +02:00
leonarski_fandClaude Opus 5	6ef580aa06	docs: the acknowledgements are grouped by what the credit is for, instead of by when it was added ... The GPU bitshuffle+LZ4 decode and the SparseCCL spot extraction were cited under the paper citation and the funding, ahead of the section that exists for exactly that kind of credit, and the crystallographic section itself was one flat list of twenty entries in no particular order. Every entry is now under a heading that says what it is credit for - the methods section runs in pipeline order from spot finding to the model, spot extraction joins the spot finding it belongs to, and the GPU decode and the Matplotlib colormaps sit under software rather than under the citation. No credit text is changed, only its place, with two exceptions. That section claimed nothing in it was linked or vendored except GEMMI, which was already untrue of fast-feedback-indexer and is now also untrue of traccc, so it names all three. And Rugnux is the software, `rugnux` the command, so the generative-AI declaration says Rugnux. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UdFBP5TjGsJ5Kvy2H8P9L9	2026-09-08 21:22:16 +02:00
leonarski_fandClaude Opus 5	dccc415a53	viewer: the Help menu says what the mouse does and who is credited, and a spot can be hidden by what it is ... The mouse bindings were only in the code and in one bullet of the viewer page: the wheel and its four modifiers, the two ROI drags, and the hover and double-click behaviour of the grid scan, the plots and the auxiliary views. They are now a window under Help, and the same table is a section of docs/JFJOCH_VIEWER.md. docs/ACKNOWLEDGEMENT.md is shipped in the viewer's resources and shown under Help beside the third-party licenses, so the credit travels with the binary rather than only with the docs, and it ends with a declaration of how generative AI was used in developing this code. The image overlay draws every spot it has, which makes a frame with ice or with a large unindexed population hard to read. The Inspector can now leave out the spots that were not assigned to a lattice, and the spots that fall on an ice ring, independently of each other and of the ice-ring highlight. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UdFBP5TjGsJ5Kvy2H8P9L9	2026-09-08 21:17:07 +02:00
leonarski_fandClaude Opus 5	aa13143209	symmetry: an axial row with one control reflection is scored, and a row nothing was recorded on is decided by a stated prior ... Two changes, both about axial rows a sweep only partly reaches. A screw zone whose control class held a single reflection was discarded outright: the scale the zone is judged in is the mean of its control class, and that mean was required to rest on two numbers. One number is also a mean. The requirement measured the mounting rather than the evidence - an axial row lying near the spindle sits inside the blind cone and is recordable only out to 2 sin(alpha)/lambda however long the sweep runs, so on a crystal mounted along one of its own axes that row can never hold two. Measured on an orthorhombic wedge whose h00 held exactly two reflections, the absent one at 5e-5 of the present one: the zone scored zero. What a one-control zone can now earn stays capped by MIN_U_PER_ABSENT_REFLECTION at 6.9 nats per absent reflection, well under min_screw_absence_evidence, so it can add to a screw confirmed elsewhere but cannot license one on its own; the violation threshold is unchanged. Where candidates still tie, they differ only on axial rows this merge holds no control class for, and the representative was the lowest space-group number. That is a prior too, and the wrong one: a crystal that has already shown one screw carries the rest an order of magnitude more often than not, and a sweep short of half a turn routinely records one axial row and not the next - on two consecutive 50 degree wedges of one crystal, the first sees 00l and no 0k0 and the second 0k0 and no 00l. The tie-break therefore prefers the candidate claiming a screw on the unseen rows, and only where at least one zone was actually judged, so it is never spent by a crystal that has shown no screw at all. Centring-driven axial conditions do not count towards the claim, so a centred group and its screw partner stay tied. The alternatives are reported as before; only which one is named first moves. Measured over every run in a 99-dataset corpus that reports indistinguishable groups - the only runs this can reach - 31 in all: 28 unchanged, 3 changed, all three from a wrong group to the deposited one. Two of those three reach the right label over a run whose data are still wrong for unrelated reasons, one having lost a seventh of its frames to a crystal that moved mid-sweep and the other carrying a beam-centre error that leaves an edge 1.6 % out; this decides between tied candidates and repairs neither. jfjoch_test [twin] and SearchSpaceGroup pass, including the calibrated case for an absent class weak only within its own row. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-08 20:52:33 +02:00
leonarski_fandClaude Opus 5	af6d95df8e	lattice: a monoclinic cell is named C, at the least oblique beta that naming offers ... The Le Page namer took the least oblique cell across the P, C and I namings of a monoclinic lattice, while the space-group search it feeds enumerates reference settings only - whose order-2 Sohncke groups are P2, P2_1 and C2, never I-centred. So a lattice the plane-basis search named I could earn its promotion on the intensities and still be refused for its name: the re-ask's chosen->centring_type() == cand.centering has no candidate to match, on every such run. The (a, c) plane-basis search now runs twice: first accepting only the P and C namings, then, only if the plane offers no other, accepting I as well. The obliquity is still minimised - within the reference naming rather than across namings. That is a search for the least oblique C, not a relabel: a -> a + c on such a lattice turns beta 120 into beta 32, outside the 30/150-degree bounds of the constrained refinement, which is what the replaced comment warned about. A new test hands the namer a lattice whose least oblique naming is I and requires the C setting back, with beta inside the bounds; it fails on the previous code, which returns I on all twenty presentations. Blast radius: LePageLattice() has two callers, both in the metric re-ask, and the branch this repairs was a guaranteed refusal, so no run that already adopts a promotion can move. Across all 99 run logs of the external corpus, three runs produce an I-centred monoclinic metric candidate, and all three were re-run with the change. One moves from a primitive triclinic group to the C-centred monoclinic group deposited for it, on the same lattice (overall CC1/2 0.30 -> 0.95, completeness 93.6 -> 98.8 %, multiplicity 2.2 -> 4.0); a second, whose lattice is separately wrong, now at least carries its deposited space group; the third asks in the C naming, is refused on the intensities exactly as before, and its results report is byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-08 20:52:33 +02:00
leonarski_fandClaude Opus 5	c5663b4614	model validation: the model is put into the data's description of the lattice before it is placed ... A deposited model arrives in the cell its depositor chose and rugnux indexes in the cell its own reduction chose. The two are frequently different descriptions of the SAME lattice - I-centred where the other is C-centred, unique axis c where the other took b, a cyclic permutation of an orthorhombic cell - and the space-group NUMBER is identical in every one of those, so the existing enantiomorph check, which compares numbers, cannot see any of them. Re-fractionalizing straight across such a pair scrambles the model, and the rigid body cannot undo it: six parameters about a centroid are not a change of basis. The run then reported a placement R-free near 0.6, CC(model,data) around 0.3 and MODEL_FIT= NOT_TESTED, with no warning, for data that were perfectly good. Before re-fractionalizing, every integer change of basis with entries in {-1,0,1} and determinant +1 that carries the model cell onto the data cell is enumerated, quotiented by the model's own point-group rotations, and scored on an 8-3.5 A shell after an overall scale and an isotropic B. The lowest wins. The identity is always among the candidates, so a probe that finds nothing better than leaving the model alone leaves it alone; and on the ordinary isomorphous run the only operators that map the cell are the identity and its own symmetry equivalents, so they collapse to one candidate and the scoring is never reached. A change of basis is a matrix AND an origin shift, and the shift is not optional. An odd permutation of a screw-axis group lands on the same group on a moved origin, whose operator list GEMMI cannot name because it compares those lists exactly - swapping b and c in P 21 21 21 needs (1/4, 1/4, 1/4) first. Where the bare matrix names nothing, the shift that makes it name something is searched for on the twelfths every crystallographic origin shift lies on. The coarse shell is what makes trying every candidate affordable: a frame that is wrong is wrong at low resolution. The lowest resolution is excluded with it, because there an unmodelled bulk solvent would dominate - equally for every candidate, so it would only add noise. Measured cost 0.04-0.52 s, and only on runs whose cells actually differ. A halved or a doubled axis is deliberately outside the search space: those are indexing errors, not a choice of description, and a probe that silently repaired one would hide a real defect. So is the case where the lattice is right but the symmetry was undercalled to P1, which would need the model expanded to P1 and a centred-to-primitive basis change. Measured over the deposited-data battery: eight runs whose model had never been placed now place it, R-free falling by 0.14 to 0.36 and CC(model,data) rising from 0.14-0.47 to 0.51-0.96; density at atom centres 0.35-2.1 sigma to 1.5-4.8, map mean FOM 0.13-0.28 to 0.36-0.93. Five controls chosen to exercise the probe moved by no more than run-to-run noise and took no operator, one of them after correctly rejecting a real alternative frame at coarse R 0.56 against 0.19 for the identity. The reflections are not touched: they keep the indexing the reduction determined, and it is the model that is expressed in it. What this fixes is that the written model and the written reflections now agree - Fcalc from the placed model against the merged amplitudes, both as written, gives R 0.18-0.31 where it gave 0.54-0.65. The CRYST1 header was already correct before, since WriteModel forces the data's cell and group, so a downstream program would have loaded the pair without complaint and refined against scrambled atoms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XbMaLuuwX4D58QCxZ686Ku	2026-09-08 16:02:25 +02:00
leonarski_fandClaude Opus 5	e5c96cd41b	model validation: the placement target carries a bulk solvent that means something ... The rigid-body target refits the scale at every evaluation, deliberately - held fixed, the target would measure the scale as much as the placement, and the body would translate to repair a scale error instead of moving to where the density is. That refit was gemmi's unbounded fit, the one already replaced for the reported scale, and here it was worse: measured over a corpus of deposited models, 40% of the evaluations that decide where the model goes came out with a b_sol outside 10-80 A^2, on 57 of 89 datasets, ranging from -8072 to +1721. A negative b_sol is a solvent term that GROWS with resolution. One crystal ran its entire committed placement between 290 and 430 A^2, and that placement went into the reported maps. The bulk solvent is fitted once per zone instead, inside the same physical box the reported fit searches, and then held while the overall scale and the anisotropic B keep following the body. That split is the point: k_sol and b_sol describe the crystal's disordered solvent, not the fit of one placement, and measured across a whole zone they drift by a single grid step. Fitting them at every evaluation costs three times the wall clock, makes the scaler 82% of the run, and moves discontinuously under a forward difference - which is poison for a numerical Jacobian. This costs 4%, leaves no evaluation outside the box, and lands the body within 0.37 degrees of the expensive version, against 4.33 degrees for what it replaces. Placements change on fourteen crystals in eighty-nine. R-free is a wash on the mean; the step buys more total R-free from fewer commits, and the gain sits where the runaway actually bit. Two things found while auditing the file and left as they were, because they are right: the refinement sees working reflections only, end to end, and the gauge that removes the origin-free directions of a polar group carries more than half the whole refined translation in ninety-four of a hundred and thirty-nine polar zones. It had no test. It has one now. An observation with no calculated amplitude gets a zero residual, which drops it from the target rather than scoring it as a perfect fit, and is now counted and reported - a large count says the model's reflection conditions do not match the data's, which is a statement about the model. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 15:27:30 +02:00
leonarski_fandClaude Opus 5	004fa5a781	model validation: the model says, shell by shell, where the data still carry signal ... A placed and scaled model gives an expected intensity for every reflection, and comparing that with the merged intensity is a measurement nothing else in the report can make: CC1/2 and R_meas compare the data with themselves, and can only say whether a shell is reproducible, not whether it is right. CC(model, data) is reported per shell, on the merge's own shells so the rows line up, with the reflection count and a significance beside each. Read it in one direction only. A correlation significantly above zero proves signal is present, because a model cannot invent agreement with data it never saw. A correlation near zero proves nothing, since the model may be the thing at fault - one crystal in this corpus reads negative across its outer shells against a deposited model of a different crystal form. So this can support keeping a shell and must never be used to discard one, and nothing in the pipeline acts on it. Intensities rather than amplitudes, matching CC1/2 and CCref beside it. The raw merged intensity rather than the French-Wilson amplitude: that amplitude is a posterior mean under a Wilson prior which pulls weak reflections towards their shell mean, and the shells this number is read in are exactly the weak ones, so it would correlate the prior with itself. The free flag is ignored on purpose. Nothing has been refined against these reflections, so the correlation is unbiased over all of a shell rather than over the test set alone - which is where the outer-shell reflection counts are, and with them the difference between a statistic that can resolve something and one that cannot. REPORT_VERSION is 7. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 13:12:32 +02:00
leonarski_fandClaude Opus 5	cb08f63a52	model validation: the bulk solvent is searched inside its physical range, not fitted without bounds ... gemmi offers two scalers and we were using the one without bounds. Its Levenberg-Marquardt path has nothing stopping the flat-solvent parameters from leaving the range the model means anything in; the alternative path that does declare bounds is behind a compile guard we have never enabled. On this corpus six datasets in fifty-one fitted a b_sol outside it, the worst at 1707 A^2. What that does is subtler than a bad scale, and worth recording because it is why nobody noticed: a b_sol that large does not corrupt the solvent term, it switches it off - 1.4% of it survives at 10 A - so the model is simply scaled without a solvent contribution and the R-factors look unremarkable. k_sol and b_sol now come from a grid search over the physical box, with the scale and the anisotropic B refitted at each candidate pair, following Afonine et al. Refitting at each point is what makes it work: clamping the parameters after an unbounded fit costs up to 0.044 in R-free, because it leaves the scale and B where the rejected fit put them. Non-physical fits go from six in fifty-one to none, and both R-work and R-free come out slightly but significantly better rather than merely no worse. Which reflections are fitted remains the caller's business - the function scales whatever it is handed - so the working-set restriction of the previous commit is not something this can undo. A crystal with no solvent-accessible volume needs no special case: its mask is empty, so the solvent term is identically zero whatever the parameters say. There is a test for that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 10:34:21 +02:00
leonarski_fandClaude Opus 5	2ae8db6623	model validation: the scale is fitted on the working reflections, so R-free is free of it ... The model's overall scale, its anisotropic B and both bulk-solvent parameters were fitted over every reflection, free ones included, and R-free was then computed from the resulting scaled Fmodel. About eleven parameters, minimised against the very sum R is made of, chosen with the help of the reflections that are supposed to be held out of it. The documentation said so and argued it was harmless - the parameters are far too few to absorb individual reflections - and on the numbers that argument is right: the standard optimism bound puts the effect at a few hundredths of a percentage point on an R-free of twenty percent, below the free set's own sampling noise. It is still wrong. R-free means the test reflections took no part in choosing anything the model is scored on, and a reader who greps R_FREE= is owed that without a footnote. The fit now sees the working set; the parameters it returns are applied to every reflection, so the free ones are scored against a scale they had no hand in. Refmac fits scale and B on the working set and says so in its documentation; phenix.model_vs_data, which is the direct analogue of this mode, uses its work-only scaler. Two things still consult the free set, and the documentation now names them instead of leaving them to be discovered: the rigid-body step is committed only if R-free improves, and the alternative indexing is the candidate with the lowest R-free. Both are single discrete decisions rather than fitted parameters, both are the conventional use of a test set, and both leave R-free very slightly optimistic where they fire. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 10:31:24 +02:00
leonarski_fandClaude Opus 5	2574c2e587	docs: two changelog entries that are not a user's business ... One described a fix to a defect this same release introduced - the two-pass guard learning to weigh axial reflections, which exists because the indexed-frame floor moved in this release and exposed it. A reader upgrading from the last release never saw the defect, so the entry above it already says the only thing that changed for them. The other announced a regression-testing script. It is a repository tool, not part of what a user installs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 07:30:51 +02:00
leonarski_fandClaude Opus 5	24dcaf4e67	rugnux_vs_pdb.py: measure a corpus against the models its depositors published ... A companion to rugnux_vs_xds.py, asking a different question. That one compares merging statistics against another program's reduction of the same images; this one asks what an independent party already decided the images were worth - the deposited model, its published resolution and its published R-free - and compares against that. The runs are deliberately bare: --model and nothing else. No space group, no cell, no resolution limit. Anything told to the program is a thumb on the scale, and the question is what a bare invocation reaches on its own. The R-free it reports is a validation number rather than a refinement result: the deposited model is placed rigid-body against our merge and the free residual of that placement is what is compared. The depositor's published R-free came from full refinement against their own reduction, so it is the harder number, and the comparison is deliberately unfair in their favour. Entries are identified by directory name, which must be the PDB entry ID; metadata and coordinates come from the RCSB API. The sweep is resumable, since a hundred datasets is hours. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 07:21:06 +02:00
leonarski_fandClaude Opus 5	e02dd2e924	rugnux: the two-pass guard weighs the reflections a space group is read from ... The guard that chooses between the file's geometry and the refined one judged the two passes on CC1/2 alone. CC1/2 is an average over tens of thousands of reflections and the principal axial rows hold a few dozen, so a pass can win it while measuring half the axial reflections the other one did - and the systematic absences that decide a screw axis are read from nothing else. An axial reflection crosses the Ewald sphere over a narrow range of the sweep, so a pass that loses frames loses whole rows rather than a scattering of reflections. Measured on a crystal that was losing a screw exactly this way: the refined pass held 7 of the 16 low-order axial reflections and won the gate by 0.0114 in CC1/2, which is inside its own noise. The screw evidence fell from 24.2 nats to 10.5, under the bar, and the run reported the symmorphic group. A third arm, beside the two the guard already had: refuse the refined pass when it holds under three quarters of pass 1's axial reflections AND its CC1/2 does not say it is better. It can only decide cases the gate was deciding on a difference inside its own noise, and it needs pass 1 to hold enough of the row for a screw to be claimable at all. Over the corpus it fires on three crystals: the one it was built for, which recovers its screw; a second that gains 0.057 in CC1/2 and was being handed the wrong geometry for reasons nobody was looking for; and a third that loses 0.009. The other 145 are untouched. This closes a regression the six-spot floor exposed. The floor did not cause it - at the old floor the refined pass was broken outright and the CC1/2 arm rejected it for unrelated reasons, which rescued this crystal by accident. What the floor removed was the accident. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 07:18:59 +02:00
leonarski_fandClaude Opus 5	2c4b804f70	docs: the scaling model carries all three deterministic corrections, as the code does ... The observation model said its L term was the prescaling correction, and that predictions carry its reciprocal as prescaling_corr. That was true when that one field held everything. It has since been narrowed to Lorentz x polarization, with the sensor's efficiency and the flight path carried beside it, and every site that corrects an intensity multiplies all three - the scaling fits, the merge ingest, the anisotropy analysis and the unmerged export alike. So the model term is the product, not the first factor of it, and a reader implementing against that page would have applied a third of the correction. The header comment on the efficiency factor had the same fossil: it said the total correction is prescaling_corr * qe_corr, written before the flight path existed. The comment two fields below it already said all three, so the file disagreed with itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:59:40 +02:00
leonarski_fandClaude Opus 5	fc41b1c34c	release: version 1.0.0-rc.167 ... VERSION bumped and update_version.sh run, which rewrites the version string everywhere it is recorded: the OpenAPI specification and the three clients generated from it, the frontend package and its version module, the Sphinx configuration, the Redoc page, and the FPGA HDL and PCIe driver strings. Every hunk is a version string and nothing else, checked rather than assumed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:45:37 +02:00
leonarski_fandClaude Opus 5	2538f5a956	docs: the manual describes the program this branch actually built ... A sweep of docs/ against the code, and a changelog a user can read. The report reference still described the section layout from before the report was rewritten, claimed anomalous keys were withheld from Friedel-merged runs when 144 of 148 stored reports carry them, and listed as default a set of keys that --developer now selects. The tutorial described an enantiomorph message that no longer exists; the option tables were missing --developer and --finalist-ledger and carried a short option the program does not have; the integration page's unmerged-MTZ column list predated the two new columns and asserted the absence of one of them. The analysis pages had the efficiency and flight-path corrections but none of the symmetry work: the pseudo-translation detector, the absence test that divides it out, the glide test, the evidence-keyed alternatives and the metric re-ask are now written up where the methods are described. The usage message omitted FLIGHT from the formula that turns a written intensity back into a raw count, while the writer has been emitting the column. The usage message is the authority on what the program does, so it says so now. The changelog had thirty-one entries filed under the previous release, most of them written from the inside: what a change did to the code rather than what it does for a reader. This release gets its own section with sixteen, each one capability. Every key, flag and column a user could grep for survives; what goes is the seam between one person's work and the next's, which is not something a user can act on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:44:09 +02:00
leonarski_fandClaude Opus 5	c4abcc2591	diagnostics: an image that was never measured reads as unknown, not as zero ... Every per-image vector the scan-result builder fills with value_or(NAN) was sized with the value-initialised zero, so an image that never reached the fill loop at all - dropped, never arrived, numbered outside the run - came out of the END message carrying a measurement it never had. Zero is not missing data. A spindle blind fraction of 0.0 reads as "this frame lost nothing", a mosaicity of 0.0 as a perfect crystal, and a resolution estimate of 0.0 as nothing measurable at all, and each of those is a claim the run is in no position to make. Sized with NAN instead, which is what the loop already writes when the quantity was measured and came back empty, and what the readers and plot builders already test for. The counts keep their zeros: a count of zero is a thing that can be true. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:15:20 +02:00
leonarski_fandClaude Opus 5	e8fbae879a	rugnux: the two offline instruments go, and one release states one report version ... The finalist-ledger and glide instruments were built to measure a question each, over stored merges, and both questions have been answered. They were never asked for and they are not part of the product: neither is installed, both duplicate what rugnux itself does with a flag, and each is one more executable to keep linking. The --finalist-ledger mode stays where it belongs, in rugnux. Their removal also retires the CUDA device-link property they would have needed in the portable build, which is the shape of problem an unused target creates: a configuration nobody builds, breaking for a target nobody runs. The changelog claimed four different REPORT_VERSION values inside one unreleased section, which is development history rather than something a user can act on. A release states its report version once; the entries keep what they changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 21:53:17 +02:00
leonarski_fandClaude Opus 5	4b13098143	build: the offline instruments get the CUDA device-link step too, and the spot floor says what it drives ... Two corrections from an independent review of this branch. CUDA_RESOLVE_DEVICE_SYMBOLS was set on rugnux alone. The two offline instruments beside it link the same analysis library and therefore pull the same relocatable-device-code object out of libcufft_static, so in the portable-plus-CUDA configuration - which is the intended Windows build - their host link fails on an undefined __cudaRegisterLinkedBinary_* symbol. The property belongs on every executable that links the library, not on the one that happened to be tested. And the comment on the indexed-frame floor claimed less than the constant does. The same number is also the order statistic the candidate scorer ranks cells on, the stopping condition of the iterative residual contraction, and the adoption gate of per-frame refinement. Those ask whether a fit can be trusted, which is a different question from whether a frame had spots to spare; they share one constant only by history. Recorded rather than split, because splitting them changes behaviour and would need its own measurement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 21:15:24 +02:00
leonarski_fandClaude Opus 5	ac1e8b8603	tests: the report version pin and the screw-gate section follow the code ... Two test expectations that the changes under them made stale, and one overstated sentence in the docs. REPORT_VERSION was bumped to 12 without touching the line that pins it - the very line whose comment says a key added to the report is a contract change and this is where it has to be acknowledged. Acknowledged, and the absence of SOHNCKE_SPACE_GROUP on a fixture that is GIVEN its group rather than searching for one is now asserted too, since that is what makes the key's contract honest. The section documenting that the E^2 gate is what saves a screw from fabricated violations no longer holds, because there are now two independent defences: with the gate off, the absent class still sits at 2% of its own row, so the zone is dead per reflection and licenses the absence evidence to override the count. The section is kept rather than deleted - it now pins the second defence, and a future P2 there would mean the deferral has stopped licensing a zone that is genuinely extinct. The docs said SOHNCKE_SPACE_GROUP is written on every run. It is written on every run whose group came from the search; a run given its group with -S has no Sohncke candidate to name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 21:12:42 +02:00
leonarski_fandClaude Opus 5	485bbeaf71	rugnux: a cell given with -C is matched in any equivalent description ... Every candidate the indexer forms is Niggli-reduced and primitive; the reference cell was compared against them exactly as typed. For a centred lattice those are different numbers entirely - a conventional F-cubic cell reduces to a rhombohedron with 60-degree angles, a centred monoclinic cell to axes it shares none of - so the true candidate failed the 5% test and the run reported that it had found no lattice. A user who supplied the cell as it is published was worse off than one who supplied nothing. The reference is now expanded into the primitive lattices its six numbers could stand for, one per centring, each reduced the way a candidate is; a candidate matching any of them is kept. The cell as typed stays in the set, so the indexer that returns the basis it was handed is unaffected. The cost is real and stated: a wrong cell now gets several chances to match rather than one. The controls that hold are on data - deliberately wrong cells are still refused, and every cell that worked before gives the same answer to the digit - but a synthetic lattice can be made to match through a centring hypothesis it has no right to, and no volume test tightens that without excluding a true candidate this corpus already contains. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:26:38 +02:00
leonarski_fandClaude Opus 5	78a941156b	symmetry: absence evidence overrides the violation count, one zone at a time ... A screw candidate was refused whenever more than a tenth of its predicted-absent reflections read as present, whatever the absences themselves said. On a strong axial row near the spindle those readings need not be structure factors at all: measured on one crystal, the same reflection reads +268 at one Ewald crossing and -44 at the other, a directional smear tail of the monsters beside it. Three such reflections refused a group whose absence evidence was the highest of every candidate scored, on exactly the axes the deposition names. So a class whose evidence clears the claim bar may now override the count. The likelihood has already priced those reflections in - they are inside sum_u - and still reads the class as extinct, so the count adds nothing it has not weighed, and adds it unweighted. The deferral is licensed ZONE BY ZONE, which is the part that makes it safe. The evidence is a group-level number while the count indicts particular zones, so an overwhelming genuine zone would otherwise pay another zone's debts: a four-fold screw row at 194 nats lifted the veto for two claimed two-fold rows reading 9 nats each with half their class individually present. Each zone carrying violations must now hold the deferral itself - dead at a nat per predicted absence rather than merely accumulating one over a long row, and not judged against a measured pseudo-translation, where the corrected evidence is a residue of the cone's own under-measure and the corrected count is the only instrument the modulation does not reach. Measured over the corpus: two crystals gain the screw their absences support, no crystal loses one, and every merged statistic is unchanged - a screw and its parent share a Laue class, so nothing about the merge moves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:26:23 +02:00
leonarski_fandClaude Opus 5	118746d8da	symmetry: indistinguishable groups are keyed on the evidence, not on a count of absences ... The alternatives a run reports were the candidates whose absent AND violation counts matched the winner's exactly. Two settings of one point group routinely predict a different NUMBER of absences on a zone the sweep never measured - one extra reflection on a row with no control class of its own, contributing nothing either way - and that one reflection was enough to drop a candidate the data support exactly as well as the one adopted. Measured on an orthorhombic wedge: four candidates at the same 100.7 nats with no violations, of which the two predicting one more unmeasured absence were excluded, and one of those two was the right answer. The report then said the space group followed from the systematic absences, on an axis whose row was never collected. Keyed on the evidence, a super-centering that nets the same count but over-claims still scores differently and is still not offered. The adopted group does not change - it is the same candidate either way - so this only widens what the run admits it cannot separate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:25:51 +02:00
leonarski_fandClaude Opus 5	f98cbf5373	symmetry: a reference correlation with too little to correlate is refused, not scored zero ... PickBestReindex discards only non-finite scores, so returning 0.0 when fewer than the minimum number of reflections matched made that a SCORE rather than an abstention. An operator that mapped almost everything outside the reference's coverage - and therefore had nothing to be judged on - beat a measured identity that merely correlated a little negatively, and the whole run was reindexed on no data. NaN is what the per-image resolver's ReferenceCC already returns in the same situation, against the same minimum. This makes the two agree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:25:39 +02:00
leonarski_fandClaude Opus 5	de1b9299cb	rugnux: the report names the best Sohncke group on every run ... A crystal built from chiral molecules - any protein - has no glide plane and no inversion centre, so where the search names a group carrying a glide, the group that crystal could actually have is a different one. The run knows both and printed only the adopted one as a key. SOHNCKE_SPACE_GROUP= is written whenever a space group was determined, and repeats SPACE_GROUP_NAME= where no glide was found. That repetition is the point: a key that appears only sometimes has to be tested for before it can be read, and the reader this is for - someone who knows their sample is a protein and wants the answer their sample can have - should be able to grep one name without first deciding what the run concluded. REPORT_VERSION is 12. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:33:57 +02:00
leonarski_fandClaude Opus 5	6773e8516d	symmetry: a metric two-fold the lattice search refused is asked of the intensities ... The Niggli character walk takes the first character its tolerance matches, and on a lattice that is nearly but not exactly hexagonal it matches hexagonal. Under a hexagonal holohedry no candidate point group can carry the two strongest two-folds the data actually have, so the search lands on the weakest one and the crystal is processed in a group of order two where it should have been eight. The metric group knows better than the character walk: it holds every rotation the cell can host, including the ones the named lattice system has no room for. Each of those is now put to the intensities directly - one operator, scored the way Stage A scores its own, on the same reflection population, with the same strong-reflection gate and the same E^2 cap. That last part is what makes the answer usable: normalising over the full resolution range of a merge whose outer shells are correlated junk reads a genuine two-fold at CC 0.05, and the same operator over the population the pipeline itself pairs reads 0.88. The flattening and normalising the search does at its start is now one function, so an operator can be asked about without enumerating a point group around it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:21:41 +02:00
leonarski_fandClaude Opus 5	324a2b0cdf	integration: the aperture guard watches the aperture, and the width law can be read at any resolution ... Two changes to the same file, both no-ops on every run today, measured bit-identical across the corpus on every merged statistic. The starvation guard remembers the pre-adaptive settings so it has something to fall back to, and it remembered them only when r1 moved. r1 is a proxy for the aperture, not the aperture: a prototype that widened the background radii at high angle while leaving r1 alone took the neighbour-starved fraction of a dense low-symmetry crystal to eight times the bound this guard exists to enforce, and the guard could not fire, because nothing had been remembered. It now tests all three radii. Today they move together, so nothing changes; the hole was latent, and latent is where it should be closed. The width fit computes how much wider a crystal's spots get at high angle - the 1/d coefficient of r80(d) - and threw it away, returning only the value at one resolution. The law is now returned whole and held at the ends of the measured range rather than extrapolated, since two coefficients may rest on as few as two bands. Read at that one resolution it reproduces the old number bit for bit, which is why the coefficients are carried in double. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:10:57 +02:00
leonarski_fandClaude Opus 5	a452797ce5	rugnux: the short-axis pass checks the premise it was built on ... The pass exists because a lattice row shorter than the standard search floor is reported at its first harmonic above it, so a cell it may rightfully adopt has a reduced axis below that floor. It never checked that. Where the alternative has no axis under the floor, the floor cannot be what the standard pass tripped over, and the integer volume ratio is an ordinary axis multiple - the question the axis-harmonic arbiter settles a few lines above, on the occupancy of the class the multiple adds. That is evidence about the reflections in dispute, where the tie this pass reads is a frame count a sublattice cannot lose; where the two disagree the arbiter is right, and this block was overruling it. Two crystals were being halved on that account. Both keep their cells now, and the small molecules the pass was written for are untouched: their reduced axes are all under the floor, which is the premise holding. The environment variable that held the pass off is gone with it. It was there to diagnose exactly this, and a check in the code is the better answer than a switch nobody sets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:10:43 +02:00
leonarski_fandClaude Opus 5	77390a847b	rugnux: a frame counts as indexed from six spots, the floor the setter already allowed ... The bar a frame had to clear to validate a lattice was 9, while the setter's own minimum was 6. On a crystal whose frames cannot carry 9 the difference is not a standard of evidence, it is arithmetic: the true lattice is found, indexes every frame that has the spots for it, and is then refused because too few frames could clear a count they never had spots to reach. A 7 A crystal with a median of eleven in-range spots per frame, a third of whose frames cannot reach 9 at all, validated on six frames of sixty and was thrown away - the lattice it named merges to completeness at its deposited resolution. The gate that carries the meaning is the fraction of frames that must validate, which is untouched: a wrong lattice does not index 20% of frames on six spots either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:10:07 +02:00
leonarski_fandClaude Opus 5	0d56a6219b	symmetry: a glide plane named from its own zone, and the Sohncke answer kept beside it ... A screw axis extinguishes a row; a glide plane extinguishes a zone. The search scored the first and had no candidate that could carry the second, so a small molecule whose cell and whose rotations were both found correctly stopped one glide short of its group. The zone is keyed by the rotation part of the improper operator, so a glide and its centring partner - the c and the n of the same plane - are one zone rather than two scored twice. A Sohncke group has no improper operator at all, so on chiral data the zone loop never runs and every Sohncke candidate scores what it scored before, bit for bit. That is why this is offered inline rather than behind a flag; it was still measured, on 140 protein datasets, and no protein answer moves and no non-Sohncke candidate reaches the top of the ranking. A glide is judged per reflection rather than on the zone's sum. The statistic is linear in the number of absences at fixed deadness, and a plane holds hundreds to thousands of reflections where a row holds tens, so a sum reaches hundreds of nats on a class that is merely a few times weak; per reflection the false and the genuine zones separate by a factor of nine. An unmeasurable zone refuses its candidate rather than abstaining, because a glide is an extra claim on top of a group that already fits without it. Both readings are reported on every run. A crystal of chiral molecules cannot have a glide, so a reader who knows the sample is a protein must be able to see the Sohncke group without processing the images again. A centre of symmetry is not offered at all: a candidate is enumerated only when its absences differ from a Sohncke candidate's, and an inversion centre predicts exactly what its Sohncke subgroup predicts. It cannot be claimed from intensities and this does not claim it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:09:26 +02:00
leonarski_fandClaude Opus 5	49938d2a24	symmetry: an axial absence is judged after the pseudo-translation modulating it is divided out ... A translational pseudo-symmetry splits every reflection into two classes by the parity of its index along the translation, one systematically strong and the other systematically weak. Where the translation is a half-integer along an axis, those two classes are exactly the absent class and the control class of a screw along that same axis - so the absence test divides one by the other and pays the suppression twice, reading a class that is present but suppressed as extinct, and buying a screw the crystal does not have. Measure the modulation along each axis from the intensities and divide it out before scoring. Per axis, not pooled: on the crystal this was found on it is 0.034 along the row carrying the false screw against 0.287 and 0.159 along the two rows whose screws are real, and a pooled estimate under-corrects the row that needs it while over-correcting the rows that do not. The violation counter is corrected with the evidence, because candidates rank on the sum of their zones and a correction that only ever lowers a score can never unseat a screw that has already been added. The false zone goes from 75.7 nats for its screw to 56.3 against it and from five violations in ninety to fourteen; the run adopts the deposited group. The two genuine screws on the same crystal pay 2.4 and 1.2 nats and gain no violations. On a crystal whose deposit puts a real screw on a row that also carries a half-integer translation, the correction fires, that screw pays 18.4 nats and still carries 59.5, and the verdict does not move. Across the corpus nine crystals engage it and one zone changes state. Only order-two screws are treated: a three-fold with a one-third translation is left unanswered rather than answered no. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 23:56:11 +02:00
leonarski_fandClaude Opus 5	64169bb04d	rugnux: a measured spot budget below the input floor degrades the run instead of killing it ... The rotation first pass measures how deep each image's intensity-ordered spot list still lies on the lattice and adopts that depth as the spot budget. The measurement can honestly land below ten - seen where a wrong header detector distance left indexing a compensating, uniformly scaled cell that matches only the few brightest, most central spots - but the setter rejects anything under ten, so the run died on its own measurement, reporting a parameter the user never set. The floor predates the estimator by a year and was written to guard user input; by provenance it is the stills fitting bound, one more than a viable cell needs, which a rotation pass that only tests a lattice already in hand does not owe anything to. Exempting the measurement entirely would be worse than the crash: the validation gate still needs that many indexed spots per frame, so a shorter list can never index anything and the pass would abort one confusing error later. So the floor gets a name beside its companion, keeps its job for input, and the measured budget is clamped to it before adoption - with a warning saying what a sub-floor measurement means: the geometry in the file or the lattice is wrong, not that the crystal is weak. The dataset that died now completes like its siblings with the cause named; the same data at the reference-refined distance indexes 99% of images and measures no budget cut at all; and runs whose budgets sit at or above the floor are report-identical, checked at the two lowest in the corpus. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 23:33:43 +02:00
leonarski_fandClaude Opus 5	8b9598c1c4	twinning: the L-test partners are chosen so a pseudo-translation cannot silence it ... The L-test compares a reflection with a partner a fixed step away, and the step it used preserves the parity class of a half-integer pseudo-translation - so a pseudo-body-centring was invisible to it by luck rather than by design. A translation of one third is not so lucky: it puts the two members of a pair in different modulation classes, inflates the statistic past the bound that is read as evidence AGAINST twinning, and the twin call is lost silently on a crystal that has one. Choose the partners so every pair stays inside one modulation class, which is what the half-integer case was already getting by accident. The obvious alternative - lengthening the step until it clears a third as well - was measured and rejected: it puts the statistic past that bound on more than two thirds of the corpus against a seventh today. Measured over a hundred and thirty-seven crystals: a hundred and twenty-nine unchanged, seven repaired, and exactly one verdict moves - a crystal whose twinning was being denied by its own pseudo-translation. On a synthetic perfect twin carrying a one-third translation, both indicators are destroyed before the change and both return after it. Nothing branches on this verdict, so no merged intensity moves; the whole effect is what the report says about the crystal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 20:09:00 +02:00
leonarski_fandClaude Opus 5	a4be5e6cd0	diagnostics: a translational pseudo-symmetry is detected and named ... A pseudo-translation makes one class of reflections systematically strong and the complementary class weak. It is the classic predictor of a failed molecular replacement, it raises the second moment where twinning lowers it so each masks the other's test, and rugnux did not look for it at all. Look for it: a native Patterson from the merged intensities, the largest off-origin peak taken as a fraction of the origin, and the vector refined against the data. Both halves are scored against a null computed for that crystal rather than a bound taken from elsewhere - the peak against intensities permuted within resolution shells, the modulation depth against the same search started from random vectors - because the noise floor of the first runs from 1.35 to 17.7 per cent across the corpus, so no fixed threshold could mean the same thing twice. Requiring both is what keeps the false-positive rate down: either alone calls one crystal in ten. A translation the data are exactly invariant under is a lattice vector, not a pseudo-symmetry. Recognising that removes eleven false calls from ninety-five merges folded in P1 - each reading 83 to 102 per cent of origin against a genuine maximum of 62 - and adds one true positive whose pseudo-translation lies underneath its own centring and is unreachable without it. Report-only: it gates nothing and changes no reflection, no scale and no group. It fires on eight of a hundred and thirty-seven crystals, and on those the verdict moves from OK to WARNINGS, which is the point of it. Median cost 55 ms, worst 1.9 s on the largest merge here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 20:08:40 +02:00
leonarski_fandClaude Opus 5	5b8ce26c83	integration: the flight path between the sample and the detector is corrected for, and named ... A reflection arriving at an angle to the detector normal crosses D/cos(alpha) of whatever lies between the sample and the sensor, not D, so it is attenuated more than one arriving head-on and reads low. That is the same geometry as the sensor crossing already corrected here and the opposite sign, and it was missing. The factor is exp(D/L*(1/cos(alpha)-1)) from the NIST attenuation coefficient of the medium, the stated distance and the stated wavelength. Nothing in it is fitted, and it is not justified by any measured amplitude: the flight path and the sensor crossing are collinear to better than 0.998 over the angular range any single experiment samples, so no fit of one can be evidence for the other. It is the tabulated absorption of a known thickness of a known material over a known path. The medium cannot be detected. No field of the NXmx application definition describes it, none of the masters this program reads carries one, and it cannot be inferred from the implied transmission either - in this corpus a station confirmed to use helium sits at 51% implied air transmission and one confirmed to use air at 63%, so any rule separating them is a threshold fitted between two points. It is therefore assumed, stated, and overridable: --flight-path air|helium|vacuum, defaulting to air. Helium is its own material rather than an alias for vacuum, attenuating about a six hundredth of air rather than nothing. On an untilted detector the correction is a function of resolution alone, so its entire effect on merged data is a shift in the Wilson B - which is what the report now prints beside the assumption, accurate to better than a tenth of an angstrom squared against measurement from 0.05 up to 28. Where that shift is large the report warns, because a wrong medium is then the largest number in the run: applied to data from the confirmed helium station it returns a B of 14 A^2 at 3.0 A resolution, which is not a value a crystal can have. The corpus contains its own control. One crystal, one station, three collections a quarter of an hour apart at falling energy through the same air: corrected, the Wilson B rises monotonically with the dose, as it must. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 17:55:43 +02:00
leonarski_fandClaude Opus 5	7c10d62dab	integration: the sensor efficiency is carried as its own quantity, not folded into the Lorentz-polarization factor ... It was multiplied into the per-reflection factor at prediction, so that factor held Lorentz, polarization and efficiency at once and the two spellings that reach a file - the wire key and the reflection dataset - meant something different from what they had meant the day before. The unmerged MTZ had to divide the two apart again at write time to fill its own columns, which is a good sign the wrong thing was being carried. Carry them separately. The prescaling factor is Lorentz and polarization again, what its name and both reference implementations mean by it, and the efficiency is its own field through prediction, integration, serialization and storage. Fifteen sites that want the total now multiply the two - once per reflection, not once per pixel. The efficiency is stored rather than recomputed on read, because the writer has no geometry to recompute it from, and because a file written before the correction existed would have had a radial trend invented for it. Sixty stored files were checked for the one combination that would be ambiguous - the old meaning of the factor beside a stored efficiency - and none carries it. Output does not move. Re-scaling a file written before the efficiency existed is byte-identical, which is a proof rather than a sample, since the stored factor is exactly one there. Where the efficiency is live, one product is reassociated - (L*Q)/P becomes (L/P)*Q - and about a third of the values differ in the last bit or two: every structural column is identical, so no reflection is gained, lost or reindexed, and no intensity in 1.4 million observations moves by as much as 1e-4 of its own sigma. The parity tests now compare the efficiency as well, and their non-vacuity guard watches it rather than the factor it left - which is the same guard that went blind when the efficiency was added to a field it was not watching. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 16:28:15 +02:00
leonarski_fandClaude Opus 5	e44ce069a2	docs: the polarization factor is credited, and the sign that sets its plane is documented ... The correction is Kahn's, and the paper was not cited anywhere. It is now, at the function and in the acknowledgements. The plane was already settable and only the API said so: a negative factor is a vertically polarized beam. Measured on the corpus, dividing by the factor as given collapses the azimuthal modulation of a real background from 27.6% to 0.77%, and asking for the wrong plane doubles it to 55.7% - so the convention is right and the sign is load-bearing. The usage text now says which sign means which plane. No dataset in the corpus declares a plane and every one that can be measured is horizontally polarized, so nothing here changes what any current run computes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 14:30:04 +02:00
leonarski_fandClaude Opus 5	6140de7249	rugnux: the unmerged MTZ carries the sensor efficiency in its own column, not inside LP ... LP is a column other programs read and un-apply. It is documented as the Lorentz-polarization factor, and until the sensor efficiency was folded into the same product that is what it held. Measured on our own unmerged file, LP spanned a factor of 1.1343 across the detector where pure L/P spans nothing of the sort - the excess is the efficiency, 13% end to end at 13 keV and 34% at 18 keV. Both reference implementations keep it out. Recomputing pure L/P from a stored XDS file's own geometry over its 124k reflections leaves RLP flat to 0.1% from 8.6 to 33.7 degrees, where a folded-in efficiency would have shown a 7% trend - and XDS has the sensor numbers in hand. DIALS fills LP from lorentz and polarization alone and writes QE as a separate column, even a column of ones. Split them: LP is L/P again, QE is the efficiency, and the intensity is unchanged, so a reader that un-applies LP recovers what it expects and one that wants raw counts divides by LP and multiplies by QE. Only the unmerged file moves - every other column is bit-identical and the file grows by exactly one float per reflection plus one header record. The merged files are byte for byte what they were. The process file gains an optional qe dataset beside the existing one rather than changing what that one means, so a file written before the efficiency existed still loads, and reads back as a correction of exactly one - which is what it was. It is stored rather than recomputed on read because the writer has no geometry to recompute it from, and because recomputing would have written a radial trend into every stored file that never had one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 14:29:45 +02:00
leonarski_fandClaude Opus 5	db4af06a2b	integration: the sensor efficiency reaches the stills GPU predictor too ... The efficiency correction was added to the rotation predictor on both the CPU and the GPU, and to the stills predictor on the CPU, but not to the stills predictor on the GPU - which the factory selects for stills whenever a card is present. So the correction was applied on a machine without a GPU and silently skipped on every machine with one: 6.6% apart on the geometry the tests use, 23.9% at 18 keV through a 450 micron sensor. The parity test that should have caught this is the reason it was not caught: it compares the predicted position and resolution of every reflection and nothing else, so a factor carried alongside them is invisible to it. It now compares the prescaling factor as well, with an assertion that the factor is not uniformly one so the comparison cannot pass vacuously, and the rotation path - which had no parity test at all - gets the same one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 14:29:22 +02:00
leonarski_fandClaude Opus 5	db9cc9106f	rugnux: the per-reflection correction factor is named for what it is, not for what it once held ... The factor multiplied into each integrated intensity was called rlp, for reciprocal Lorentz-polarization, and until this week that is all it held. It now also carries the sensor efficiency at the angle the beam arrives, and on the stills path it holds that efficiency and the polarization with no Lorentz term at all - correctly, since the Lorentz factor of a still is one. Three different products under one name that promises exactly one of them, in code where the neighbouring member is the total correction. Rename it prescaling_corr: multiplicative, applied before scaling, therefore not a scale, and silent about its contents - which is the point, since the contents have now grown twice. It is also what DIALS calls the same product. The stills refinement member spelled "1 / rlp" becomes inv_corr, and the comments and usage text that promised "the Lorentz-polarization factor and nothing else" now say what is actually there. The Lorentz term keeps its own name where it is computed, because that name is correct. The two external spellings are untouched: the CBOR key and the reflection dataset are a published format, and a reader that meets an unknown key would take the factor as zero, which both the merge key and the ingest treat as a reflection to drop - so every reflection would vanish and the run would still exit zero. No output changes: the merged and unmerged files of two full runs are byte for byte what the previous binary wrote, four stored files from before the efficiency correction still re-scale identically, and the reflection datasets of the process file are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 13:41:16 +02:00
leonarski_fandClaude Opus 5	d6ccdbdb61	rugnux: the report says what it found before it says how it found it ... The report was written in the order the pipeline runs, so a user reading one had to reach line 380 before meeting the first evaluative statement, and the warnings were last. Every run was 314 to 407 lines whether it succeeded or failed, about 200 of them fixed prose. The anisotropy section printed 21 keys and announced DETECTED (strong) on 54% of all runs - "strong" is the statistical confidence, which a reader takes as the severity - and reported a censored fit, which is survival-analysis vocabulary for good news. Assemble the report into a document and render from it, rather than streaming it out as the pipeline goes. That is what allows a verdict to sit above the evidence it was drawn from: SUMMARY carries VERDICT, one sentence of plain text, the warnings and ten facts, and it is composed after the sections that produce them. A clean run is 188 lines, a failed one 93, and the verdict is on line 25 in both. Warnings now carry a closed pathology vocabulary alongside their free text, so a consumer can switch on the code and a reader still gets the sentence. Everything removed from the default report is still written under --developer, which also carries the internals worth having when diagnosing the program rather than the crystal: the gate keys behind the anisotropy verdict, the operator and candidate tables, the sweep and spindle internals, and the essays. Nine statements the report made that were not true are fixed here as well. Among them: --mode scale printed a detector tilt of exactly zero on tilted data, which is worse than printing nothing because nothing about it looks wrong; every --mode scale run carried a "No image indexed" warning, because a key that is absent and a key measured to be zero were the same value; and a fitted resolution was asserted past the point where the run's own table shows CC1/2 at zero. REPORT_VERSION is 8. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 13:40:59 +02:00
leonarski_fandClaude Opus 5	eeb66ae5b4	symmetry: the systematic-b bounds follow the divisor that was changed under them ... Dividing the b fit by its degrees of freedom rather than by the observation count rescaled the statistic those bounds are compared against, and the bounds were left where they were. The header said what would happen: it records the rescue bound's calibration as a genuine step at 1.71 against a twin at 1.85 - a 4% margin - names a trigonal 3 -> 32 twin as the case it was fitted on, and instructs that it be re-measured whenever the scaling changes. It was not, and the margin closed on the wrong side. On a merohedrally twinnable crystal the b ratio moved 1.74 -> 1.40 under the new divisor, because the low-multiplicity parent gained 43% where the candidate gained 16%; with chi^2 failing at 1.97 but inside the rescue band and H confirming, the b rescue alone then promoted the twin. Merged statistics got worse doing it: R_meas 0.1310 -> 0.1732, CC_half 0.9978 -> 0.9966. Rescale the three bounds by the same factor as the statistic. This is a change of units, not a re-tuning: b_new > 0.808 * 1.78 is the same condition as b_old > 1.78, and three independent derivations agree on 1.44. Verified by running one binary with only these constants changed - the promotion reverts to the deposited group at the original numbers, while an unrelated rescue at order 1 -> 2 is untouched, being structurally immune where the parent b is zero. The census that missed this counted the veto's firings. The same constant drives the rescue, and a rescue leaves no refusal to count - it is recorded as an adoption. When a change moves the scale of a statistic, census the statistic's distribution and every bound compared against it, not the code path edited. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 13:30:27 +02:00
leonarski_fandClaude Opus 5	0f9d6a1bde	symmetry: --finalist-ledger reports the evidence for every group the search considered ... The search folds orbits and refits the error model for every point group it confirms, ranks them, adopts one and discards the rest. The discarded numbers are the only per-run measurement of how much worse the rivals actually are, and until now the only way to see them was to re-run with -S and compare by hand. Behind --finalist-ledger the run prints them: for the adopted group, the refused higher point group's best, the parent's best and any re-ask candidate that scored and lost, an evidence row folded from the P1 cross-check merge. That merge is the substrate the audit kept asking for and nothing else in the run has - the same observations at full resolution, production-scaled, folded under no symmetry, so every hypothesis is folded from it on equal terms. It also carries half-set intensities, which is what lets the noise floor be formed for the first step out of P1, and the reason this cannot be done from the stored reflection file afterwards. Report-only, and deliberately so: the accept side has 0.10 of headroom between the largest genuine ratio measured on the corpus and the bound the gate already uses, so reading these numbers as a decision before they are calibrated is the one way this can do harm. The merged output is byte for byte what a run without the flag writes. Two of the six channels are printed as evidence but marked as measured-contaminated: a ratio taken against the parent is meaningless when the parent is itself a false promotion, and on this corpus both false cases sit strictly inside the genuine range for them. A screw over-call is invisible here by construction - the folds are identical - and the table says so rather than reading clean. Every path that cannot produce a ledger now declines out loud instead of printing nothing: stills, --no-p1-crosscheck, a fixed centred group whose absences were never predicted, --no-merge, and --mode scale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 11:25:33 +02:00
leonarski_fandClaude Opus 5	9b96906150	integration: the parallax variance takes its attenuation length from the tables, not from lambda^3 ... The radial parallax broadening is the variance of the depth at which a photon converts, so it scales with the attenuation length. That length was approximated as photoelectric-dominated and scaled by lambda^3 from a single 13 keV reference per material. For silicon above 10 keV that is within 0.2%, but for CdTe it overstates the attenuation length by up to a factor of two, and by six above the cadmium K edge - which made this variance term 1.9x too large on 750 um CdTe data. Take the length from the same tabulated coefficients the efficiency correction uses. Silicon data is unaffected to within the approximation's own error; CdTe data gets a spot-width variance that matches the sensor it was recorded on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 11:07:17 +02:00
leonarski_fandClaude Opus 5	efa8ab1221	integration: each reflection is corrected for the sensor's efficiency at the angle it arrives ... A photon entering a flat sensor at an angle alpha to its normal crosses t/cos(alpha) of material instead of t, so the absorbed fraction rises toward the detector edge. The correction is QE(0)/QE(alpha) taken on the diffracted-beam direction against the detector normal, not on the scattering angle, so it follows a tilted or swung-out detector rather than assuming the two coincide. On an untilted detector this is a function of |s| alone: it is 99.7% a Wilson B offset and cancels exactly within a resolution shell, so merged protein data barely moves and no gain is claimed. It stops cancelling the moment the detector is tilted, because the incidence angle then acquires an azimuthal dependence: on a 30 degree swung-out geometry at 18 keV the within-shell spread reaches 21% median and 31% peak, and the anisotropy tensor moves with it. Attenuation lengths are the tabulated NIST coefficients rather than a wavelength-cubed approximation, which is within 0.2% for silicon above 10 keV but wrong for CdTe by a factor of two, and by six above the cadmium K edge. Photoelectric branching cancels in the ratio; K-fluorescence escape is not modelled, and the header says so. The correction self-disables where the physics makes it meaningless - an opaque sensor - so it needs no flag and is exactly neutral on all long-wavelength data and on thick CdTe. That also makes it a no-op on a file that stores its sensor thickness in the wrong unit, of which the corpus holds one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 11:07:01 +02:00
leonarski_fandClaude Opus 5	5761bcc94d	lattice: the metric re-ask reads the centring from the alternatives, and its own arm's ISa ... Two defects in the Le Page re-ask, both of which refused promotions the intensities had confirmed. The centring cannot be read from the absences in this setting: reindexing into the metric candidate's conventional cell leaves its centring-absent class holding no data at all, so among the point-group-equivalent candidates the intensities cannot separate, the tie-break takes the lowest number, and that is the primitive one. The re-ask then compared that primitive representative against the metric's centred candidate and refused. Pick the candidate matching the metric's own centring out of the alternatives first, exactly as the centred-lattice test above already does, and leave the displaced candidate among the alternatives so the reported "or ..." list stays the set of groups the data cannot separate. The refusal message was wrong as well, and said so in a way that read as an intensity verdict: a promotion refused only for the centring technicality above now says the point group IS higher and names what this setting cannot test. Separately, the filtered arm's search was handed the all-observation arm's ISa, which is re-set for that merge a few hundred lines earlier and differs by up to a factor of six - so the filtered merge's "genuinely present" cut was judged on the other arm's error model. Capture it before it is replaced. A census of every run log in the external corpus bounds the effect: the re-ask fires six times, two already adopt, three are honestly refused at equal order, and exactly one dataset can move - which it does, from a primitive triclinic group to the centred monoclinic one deposited for it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 10:11:35 +02:00
leonarski_fandClaude Opus 5	431ac4e3d9	symmetry: the systematic-b fit divides by its degrees of freedom, as its own chi2 does ... Each deviation is taken from a mean fitted on its own orbit, so one degree of freedom per orbit is spent: dof = sum(n-1) = N - G, the divisor chi2_under a few lines above already uses. Dividing by N understated the reduced chi^2 and so overstated the b that brings it to 1 - unevenly, because a low-multiplicity parent loses a larger fraction of its dof than the higher-multiplicity candidate it is compared against, and the difference landed on the b RATIO the gates read. Measured three times independently on disjoint corpora: median ratio inflation +23.7%, +23.7% and +23.8%, with the corrected ratio lower in 42 of 42 promotions, so the error pushed toward over-calling. Expect no verdict to move: the b veto has not been observed to fire on any corpus it has been measured over, and refused_by=b fired zero times in 25 refusals. The bounds are left at 1.78 / 2.00 / 0.05 rather than rescaled. They were calibrated in the old convention, but re-measured on the corrected statistic against independent space groups they still separate: the genuine maximum is 1.640 and the nearest false step 3.208, so both bounds sit inside the only clean gap the corrected distribution offers. A second derivation from population medians argues for 1.38 / 1.56 / 0.07 instead; that is a calibration question for a battery, not a condition of fixing the arithmetic. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 10:11:19 +02:00
leonarski_fandClaude Opus 5	d4fc09277e	twinning: the promotion-circularity flag is set on the struct the text is rendered from ... AnalyzeTwinning returns a fresh result with the flag clear, so setting it only afterwards left the statistics text asserting "No twinning: the Laue class is holohedral" as a fact while the report, rendering the same struct once the flag had been set, said the opposite. One run, two twin verdicts, and the flat one landed on exactly the case the flag exists to mark. Assigning before the call would not do either: the call replaces the whole struct. Reproduced on 18 of 95 stored runs from the current corpus. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 10:11:07 +02:00
leonarski_fandClaude Opus 5	e1aa84cc9e	symmetry: an operator is judged against a reference a false operator cannot contaminate ... The promotion decision compared each candidate operator against its parent's mean H, and took the maximal-order confirmed subgroup as that parent. Where a false operator was already confirmed, the mean pooled it with the real one, the true lower group was barred from being the reference at all, and the systematic-absence veto - gated on the H test failing - was disarmed by the same false confirmation. The first step out of P1 had no H parent to compare against and stood on the operator correlation floor alone. Each candidate now also carries an intensity-weighted R over the pairs the H test already collects, summed rather than averaged so strong reflections decide it, and judged against two references a false operator cannot move: the best-agreeing operator where two or more are confirmed, and the half-dataset noise floor on the first step out of P1. The gate only ever refuses, so a wrong refusal leaves P1 and the run recoverable. Measured on the 152-dataset corpus: three point-group over-calls corrected, one of them to an exact match with the deposited group; four datasets improve R_meas by 0.15 to 0.27 and none worsens; the remaining 139 are bit-identical. One crystal whose 2-fold was already being over-called is now under-called instead - the bounds were fitted on few exemplars and the report prints them per run so the population can be measured. Screw axes are untouched: they cannot be called from I/sigma and wait on their own rewrite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 07:31:43 +02:00
leonarski_fandClaude Opus 5	814f864daf	rotation: a pass-2 floor miss falls back on pass 1 rather than throwing ... Two-pass rotation indexing re-indexes de novo at the post-refined geometry, and where the second pass lands on an integer harmonic of a long axis it can fall below the validation-frame floor and abort - discarding a pass-1 result that was already correct. The frame count cannot arbitrate: a spurious axis multiple indexes every frame its true cell does, so the harmonic is not recognised as one. Where the second pass misses the floor and the first pass produced a lattice, its whole result is substituted and integration continues. This is the floor-side companion of the supercell-collapse guard just below, which the throw pre-empted. Measured on the 152-dataset corpus: every run that completed before completes with bit-identical results - 143 of 143 unchanged, no space group and no R_meas moved - and three runs that aborted now finish, one of them recovering the deposited cell and passing. The other two carry a pass-1 lattice that was itself on a harmonic; the substitution is faithful to pass 1 and does not claim to repair it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-05 07:30:14 +02:00
leonarski_fandClaude Opus 5	7e7103f49f	spindle: the blind-cone score compiles on MSVC, and the report version test says 7 ... Two CI failures from the spindle series, both self-inflicted. The severity used M_PI, which is not standard and which MSVC does not define, in a file under image_analysis - a tree the Windows viewer builds. common/JFJochMath.h exists for exactly this and is now included. The same defect was found and fixed in the post-refinement residual earlier in the day; the grep that confirmed it looked at a worktree that did not yet carry this file, so the second instance survived. The report-version test pins the version deliberately - adding a key to the report is a contract change and that line is where it has to be acknowledged - but the key added with the spindle report keys bumped the constant to 7 without the test following. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 20:00:20 +02:00
leonarski_fandClaude Opus 4.8	77a8a07c51	docs: 20 scouted external datasets recorded, credited, and reconciled ... Add the datasets pulled from the archive-wide PDB raw-data scout to the external-test-data page: new table rows (each with a DataCite-verified DOI, RCSB-deposited SG/cell/resolution, and the detector read from the image file), their multi-collection layouts, two more image-vs-PDB detector conflicts, and the two small-molecule reference cells. Multi-crystal wedge and non-native sets are excluded. Credit the two new repositories the data came from - MXRDR (ICM Warsaw) and the ESRF data portal - and bring the SBGrid and Zenodo counts in line with the table. All per-repository and per-section totals reconcile against the row count (95 PDB-coded + 7 without = 102). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DSMszqvyZUb6FHnSSS5nXY	2026-09-04 19:38:14 +02:00
leonarski_fandClaude Opus 5	2adf5d2e28	docs: the spindle severity documented as the worst-case trigger it is ... Section 5.5 of the indexing documentation describes the per-image score as shipped: the folded formula, the geometric cone width, the pair-normal recovery of an axis too long to see, and the three trigger states with the rule that CANNOT SAY must engage. It states plainly that the score is a worst-case bound under an assumption of no symmetry rather than an estimate - an axis of order three or higher perpendicular to the spindle in fact repairs the cone, which a still cannot know - and gives the engagement rates actually measured instead of the decoy-null figure, which showed only that the estimator does not hallucinate rows and was never a false-alarm rate over harmless mountings. The run-level orbit-based fraction is pointed to as the exact measure available once the group is known. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	307987c865	rugnux: the mounting's cost is computed exactly from the measured group, not guessed from an angle ... Offline, a merged rotation run has what a still lacks - a determined point group and an exact indexed orientation - so the run-level number no longer needs the pessimistic presumed-diad bound, and it no longer uses the nearest-axis angle either. That 15-deg warning heuristic was wrong in both directions: an aligned in-plane 2-fold of a dihedral group is repaired by the principal axis, a cubic group is never severe in any orientation, and a lone diad perpendicular to the spindle is severe with no axis anywhere near the spindle at all. The group's proper rotations are applied to the sweep's blind double cone in the crystal's actual orientation, and what no operator maps onto measured territory is counted, weighted by each shell's own cone width so the result is a fraction of unique reflections to this run's resolution limit. Friedel and the improper operators need no separate handling - the cone and the measured region are both inversion-symmetric. The number is machine-readable on purpose: SPINDLE_LOST_UNIQUE_FRACTION in the report (0-1, a bare number a pipeline can act on) and /entry/MX/spindleLostUniqueFraction in the master, with the warning prose only on top of it, fired when the group recovers less than half the cone's content. REPORT_VERSION stays 7: the format's own rule is that adding a key does not move it. This also settles what the nearest-axis keys hedged: with the measured group the mounting is cleared or convicted exactly, so their documentation now calls them descriptive and points at the new key for the verdict. Verified against Monte Carlo: P1 loses 2.0% of unique reflections at theta_max = 15 deg with nothing repaired; a lone diad on or perpendicular to the spindle repairs nothing; an axis of order >= 3 perpendicular to the spindle repairs everything; 622 with an in-plane diad on the spindle loses nothing; cubic loses nothing in any orientation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	26a82ddc72	spindle: the per-image severity travels the whole data path, and absence travels with it ... The score was computed and then thrown away - carried on the per-image message but transported nowhere - so the automation it exists for could not read it. It now flows like bkg_estimate at every layer: CBOR (per-image key, END-block run mean and per-image array), HDF5 (per-image /entry/MX/spindleBlindFraction in the data files, the array and spindleBlindFractionMean in the master), read-back into a re-opened dataset, the scan result, the receiver plots, the REST plot and scan_result schemas, and the viewer and frontend plot menus. Absence is load-bearing and every transport keeps it distinguishable from a measured zero: the CBOR key is simply missing, the HDF5 array holds NaN, and read-back turns NaN back into an absent optional rather than a value. A pipeline that read 0 where the truth is "no value" would take exactly the wrong action - a measured 0 says one sweep loses nothing, absence says nobody could look, and the second must engage the recovery protocol while the first must not. The round-trip tests pin all three states through CBOR and through a written-and-reopened file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	bc3d693721	spindle: three trigger states, and the threshold is geometry rather than tuning ... The score exists so beamline automation can engage a recovery protocol - a two-sweep collection, a goniometer reorientation - with no human in the loop, so its canonical reading is fixed in one place with nothing for a beamline to tune: engage at 0.5, do not below, and NO VALUE is a third state that automation must treat as engage. The error costs are asymmetric - a false negative leaves the data permanently short, a false positive costs minutes of beamtime - and a frame nobody could measure must not be read as a frame measured safe. The 0.5 is derived, not tuned: the score is monotone in the folded miss-angle, so any threshold is a fold-angle gate, and 0.5 gates at fold <= 0.4040 * theta_max (verified root). Engaging on any overlap at all would gate at fold < theta_max, whose perpendicular band alone spans sin(theta_max) per row - 26% of orientation space at 15 deg - and unions over a frame's rows to well over half of all mountings, degenerating the trigger into "always"; at 0.5 the residual missed loss stays below half the cone. Also correct the message-field comment that said the whole cone means "1 - cos theta_max of every shell": an inner shell loses 1 - cos theta(d) at its own, smaller theta(d). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	89962574ef	spindle: the severity no longer rides on the indexing seed or on which indexer is configured ... The score gated on 60 spots, but the seed escalation stops at the leanest seed that indexes - 30 spots on precisely the clean frames a grid scan produces - so the value was absent exactly where beamline automation most needs it, and absence maps to "engage": the protocol would have fired on every good frame, which degenerates the trigger into "always". The floor itself stays where it was calibrated; what changes is what it gates. When no escalation pass could answer, one severity-only pass runs over the full spot list - the row search alone, no reduction, no refinement - purely to produce the number. The same was true of the indexer choice: only the FFT family computes a row shortlist, so a deployment configured with the known-cell indexer - the ordinary online stills path - never produced the score at all. Where the severity-only pass has no row search to run, the severity is read off the rows of the winning lattice instead, which any indexer produces: the lattice's shortest few distinct directions, as many as the FFT shortlist resolves in practice, fed through the same window and scoring with equal magnitudes. The count parity is load-bearing - a worst case over every enumerable lattice direction fires on 100% of harmless mounts of a generic triclinic cell against 74% for this selection at theta_max = 15 deg, and an always-firing trigger decides nothing - while the diad-detection rate stays 1.00 on the monoclinic classes either way, a dropped axis row being recovered by the pair normals exactly as an invisible one is. A frame that neither indexed nor reached the spot floor still reports nothing, which is the honest answer and maps to the recoverable error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	0ee1e5e070	spindle: an axis too long to see still shows its direction in the normals of the rows that are not ... The length window is deliberate - a row 2.5x the crystal's shortest is not a plausible symmetry axis in a crystal that small - but it made the score blind to a lone 2-fold on an axis LONGER than the window: a monoclinic crystal with a long unique axis, mounted near-perpendicular at an unlucky azimuth, returned a confident 0 rather than a refusal. The axis is recoverable without ever seeing its row: the normal to two direct-lattice rows is itself a reciprocal-lattice row, and a symmetry axis is parallel in the direct and reciprocal bases, so cross(a, c) is the unique-axis direction whatever the length of b. The normals of the strong in-window row pairs are now scored alongside the rows themselves, with row_length_A = 0 marking a direction the frame inferred rather than measured. Measured on a synthetic lone-diad crystal with a 300 A unique axis over random mounts, the fraction of severe mounts reported severe at the 0.5 trigger rises from 0.60 to 1.00, the engagement rate on harmless mounts of that class does not move, and the recovered direction reproduces the true axis exactly (every visible row is perpendicular to it). Also state the shortlist-consistency calibration on its per-crystal basis - 22 independent mounts, not the several hundred frames they contributed - and carry the conditioning the perpendicular case needs: an axis of order >= 3 there fully repairs the cone (measured 0.000 unrepaired for orders 3, 4, 6 against 1.000 for a diad), which a still cannot know, so the lone diad stays the operative worst case and the bound stays deliberately pessimistic on higher-symmetry crystals. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	f5c2720b02	spindle: the cone is the sweep's, so its width comes from the detector, not the still ... The severity took theta_max from the still's own spot resolution - the 95th percentile of |q| - so a weak, attenuated grid still reaching 3 A scored a row at an 11 deg miss-angle as exactly 0, while the real sweep collected afterwards at the detector's 1.8 A loses a fifth of a cone that wide. The number exists to decide whether that sweep needs a second orientation, so the cone it must describe is the sweep's, and the still's spot list systematically understates it in the one direction that produces silent misses. theta_max is now asin(lambda/2d) at the geometric resolution of the setup - the detector corner at the recorded distance and wavelength. That is an upper bound on any sweep collected without moving the detector, it is available on every frame however weak, and where it overstates what the sweep will reach it errs towards reporting loss, which is the cheap error. Computed once per Setup in the Indexer base rather than per frame in the FFT implementation, because nothing about it is specific to either the frame or the indexer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	c477cf0be3	spindle: a row perpendicular to the axis is as blind as a row on it ... The blind region a sweep leaves is a double cone, so a 2-fold sends it to two places: 2*beta away, and 180-2*beta away. The severity took only the first, and scored a row perpendicular to the spindle as 0 - "symmetry repairs everything" - when such a 2-fold in fact carries the cone onto its opposite lobe, which the sweep leaves equally unmeasured. Folding the miss-angle to min(beta, 90-beta) covers both images and reproduces a Monte-Carlo of the true overlap to 0.002. The failure was silent and in the dangerous direction, and it fired on the more common geometry: for a random axis the perpendicular band is several times wider than the aligned one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	441b802e0c	docs: a large spindle-to-symmetry-axis angle does not clear the mounting ... The paragraph introduced with the two new report keys said an incomplete cusp could be "attributed to the mounting or cleared of it". The first half holds; the second does not. A symmetry axis within theta_max of PERPENDICULAR to the spindle carries the blind cone onto its opposite lobe, which a sweep leaves equally unmeasured, so a large angle is not on its own evidence that the mounting was harmless - and the key reports only the nearest axis. Also record the two keys in the changelog, and correct REPORT_VERSION there: the entry still said 6 after the bump to 7. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:59:05 +02:00
leonarski_fandClaude Opus 5	555c686205	merge: bridge a rocking event by an angle rather than by a frame count ... A rotation reflection arrives as a run of partials and is cut into rocking events wherever two successive frames are more than MAX_FRAME_GAP apart. That constant was 2.0 frames, in five places, and the quantity it stands for is not a frame count at all: it is an angle, a reflecting range of a tenth to half a degree. At a tenth of a degree per image two frames is a fifth of a degree of dead rotation and the bridge does what it was meant to; at a degree per image it is two degrees, as wide as a whole event, and two genuine crossings of the Ewald sphere are joined into one "full". Reconstructing the events on a paired control - the same photons sliced two ways - the fraction of events whose partialities sum past 1.5 is 14.3% at 1.0 degree against 6.0% at 0.1, the fused ones spanning a median of five frames. So the gap becomes half a degree of rotation, floored at one frame so an event is never cut at its own neighbours and capped at the two frames that were always allowed. For any oscillation of 0.25 degrees or less the quotient is at least two and the cap returns the literal 2.0f, so every finely sliced sweep is bridged exactly as before, on both the CPU and the GPU path. The value is taken once per run and carried to the device in CombineParams rather than recomputed in the kernel, so the two paths compare the same float by construction and the bit-parity contract the combine documents is preserved. The defect above is measured; the repair is not. No arm has been run with this change, which is why it is a commit of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:09:08 +02:00
leonarski_fandClaude Opus 5	91775cc4fb	geometry refinement: at coarse slicing, forgive the miss the exposure could have supplied ... The acceptance gate that decides which spots enter the geometry refinement compares a fractional-index miss against a fixed tolerance, and that comparison assumes the spot diffracted at the frame's midpoint. It did not. The rotation coordinate is bracketed, not observed - the spot diffracted somewhere inside the exposure - and profiling that unknown angle out of the least squares has a closed form. Writing u = m x q for the direction a rotation delta moves the observation along, the component of the miss along u is free up to the exposure's rms half-width and only the excess is charged; no other direction is touched, so |q|, and with it every d-spacing, is unaffected. Without it the gate is a resolution cut that tightens with the frame width, the miss growing as a/d. Where the benefit lives is the per-frame refinement, which holds one frame and therefore deliberately does not back-rotate - the frame's angle is a gauge its orientation block absorbs - and passes no goniometer at all. The spots on that frame still span the exposure, and that spread does not go away with the gauge, so the spindle and the wedge are handed over separately, for the gate alone. The spindle is also written into the rotation vector the frame constants carry, which was left at the lab x axis whenever no goniometer was passed; with a zero angle that vector is multiplied out and the block is held constant, so nothing changes by it today, but it was a trap for the next reader. The width is k = 1/sqrt(12), the rms of a rotation coordinate uniform over the frame, which is what a least-squares is calibrated on. Half the exposure - the worst case a spot could sit at - degrades accuracy against a ground truth, and so does forgiving the direction outright with no bound: the bound is load-bearing, and what it protects is fine slicing. Measured on 112 paired datasets, scored against deposited structure factors because neither ISa nor R_meas can arbitrate this - both move opposite to accuracy along the neighbouring partiality knob. At 0.5 degrees per image or coarser, n = 16: CC to the deposited data better on 9 of 12, median dISa +0.87%, and at the top of the a x oscillation lever better on 9 of 10 with the per-shell CC improving in all ten shells, most in the outer half. The win rises with that lever and the population below it is inert. It is applied only at 0.5 degrees per image and above. The band is principled - the dead zone matters once the exposure's rms rotation ambiguity, 0.29 of the wedge, is comparable to the crystal's own along-u rocking spread, which measures around 0.26 degrees, putting the boundary between 0.25 and 1.0 degrees - while the point inside that band is empirical and taken at the conservative end. It has to be there: the two datasets the corpus lost outright both sit at 0.2 degrees or finer, one of them finely sliced, and the response is not monotone in the width, so no bound on the width alone makes them safe. Below the trigger the block is skipped and the gate computes what it always computed; a finely sliced sweep reprocessed across this change is byte for byte identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:08:31 +02:00
leonarski_fandClaude Opus 5	8b88074001	rugnux: the joint post-refinement's gate is asked of the geometry it commits ... Five corrections to the joint fit, none of them meant to move a well-conditioned result. EffectiveCellFromParams used M_PI, which is not standard C++ and which MSVC does not define without _USE_MATH_DEFINES. image_analysis/ is on the viewer's dependency path, so it takes PI from JFJochMath.h like the rest of the tree. The bounds test ran on the fit-half solution, and what the run actually commits is the re-fit on all the reflections that follows it. That second solve moves the answer, and nothing re-asked the bounds of it, so the committed distance was held only by the solver's +-5 % box rather than by the 1 % the gate advertises. The test is now a function of a candidate geometry and is asked twice - once of the half that earns the right to fit, once of what will be written - and a re-fit that lands outside leaves the run at its header geometry, named in the log like the other refusals. It is inert on the corpus: over the 98 committed fits the largest committed move is 0.709 % of the distance, 0.885 % of a cell length and 1.07 px of the beam, so nothing that commits today stops committing. The bounds covered the distance, the beam and the cell lengths and left the cell ANGLES - beta for monoclinic, all three for triclinic - free to the solver's +-0.05 rad, which is +-2.86 deg. The uniform scale this replaced could not move an angle at all, so that was a new unguarded degree of freedom on the two systems whose conditioning is weakest. The bound is one degree, chosen from what the fit actually does: the largest angle it moves anywhere on the corpus is 0.35 deg, on a monoclinic beta, and the two triclinic cases with the largest moves are 0.09 and 0.07 deg. A degree is about three times the worst of those and still well inside the solver's box, so the box goes on reaching everywhere the gate accepts and no fit that commits today is refused. The gate pooled the two residual families into one held-out mean. With both caps saturated that is some 30000 positional values against 10000 excitation ones, so the excitation residual - the only thing in the problem that identifies the cell SCALE - is outvoted three to one in the decision that commits the cell. Neither family may now degrade. This one is not free. It refuses 1 of the 98 fits the corpus commits, a cubic case whose held-out excitation rose 15 % while its cell scale moved 0.75 %; refused, that dataset keeps its header geometry and lands where the two-step arm did, R_meas 0.2699 -> 0.2724 and ISa 3.15 -> 3.10 with the space group unchanged, and its cell deviation from the reference goes back from 0.05 % to 0.40 %. That is the shape of failure the split is meant to catch, and on the one dataset where the two disagree it costs a real if small improvement. The changelog claimed the refined distance "no longer depends on how wrong the file's distance was". It does. Swept over a millimetre of header distance the committed distance still moves monotonically and the merge degrades with it; the dependence is reduced about thirteenfold, not removed, and the line now says reduced. Finally the log printed the half-data fit's distance, beam and cell beside the word COMMIT while different numbers were committed - 107.908 mm printed against 107.913 mm applied two lines further down, on one sweep. It now prints whichever geometry the verdict is about, and prints the cell angles beside the lengths, since the angles are refined now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:00:24 +02:00
leonarski_fandClaude Opus 5	bb2a42774a	rugnux: the rotation post-refinement fits the crystal and the detector at once ... Post-refinement scaled the pass-1 lattice by one scalar against the observed rocking angles (step A), then read the detector distance off that scaled cell (step B). The split was there because the positional residual is degenerate with the cell scale - which is true of the positions ALONE, and is exactly what the excitation residual already computed in the same struct breaks. So the degeneracy that motivated the split was resolvable inside one problem all along, and splitting it manufactured error twice over: * Pass 1 frees the whole lattice against a frozen distance, so what it absorbs of a header distance error is ANISOTROPIC (0.16 % on the crystal measured), and no single scale can undo it. * Whatever bias is left in that scale goes straight into the distance, since the distance is only ever determined relative to the cell. Over six runs spanning three wavelengths of one crystal, distance error (%) = 1.33 x step-A bias (%/axis), r = 0.9947 - and the bias changed SIGN with the wavelength (-0.067 % at 1.9 A, +0.049 % at 2.7 A, +0.19 % at 3.3 A), reproducing the whole wavelength ordering of the distance error. The two steps' own printed numbers say it, on 112 of the corpus's datasets. Comparing each run's held-out POSITIONAL residual at the unrefined cell with the same residual at step A's scaled cell and at step B's committed geometry: step A leaves it WORSE than the unrefined cell on 110 of 112 (median 3.41x), and step B - which does fit that residual - cannot get back: its committed value is still worse than the unrefined cell on 110 of 112 (median 2.97x). Step B was improving on a baseline step A had corrupted. This part of the measurement is of the old code alone and owes nothing to the replacement. Now one Ceres problem: free crystal orientation, the cell (every parameter the crystal system leaves free), goniometer axis, detector distance and beam centre; residuals are the positional detector<->reciprocal one at each partial's observed spot and the distance-independent Ewald excitation one at each rocking centroid. Same deterministic hkl-hash split-half gate as before, with the move bounded: distance and every cell length within 1 %, beam within the existing bound. Detector tilt stays fixed (gauge-coupled to the orientation on one crystal). Committed, it leaves the held-out positional residual BELOW the unrefined cell's on the same 110 of 112 (median 0.90x) - though that residual is half its own objective, so read that line as a sanity check, not as an independent arbiter. The rotation-scale fit that followed step A now runs on the jointly committed lattice and axis; it is unchanged otherwise and reproduces its previous k to five digits. MEASURED. Three wavelengths of one rotation crystal, scored against an independent determination of the cell (unconstrained joint fit to XDS spot centroids) and against XDS's own refined cell: set cell error vs XDS committed distance R_meas ISa 1.9 A -0.048 % -> -0.041 % 60.274 -> 60.279 mm .0725/.0725 31.2/31.1 2.7 A +0.057 % -> -0.058 % 60.360 -> 60.266 mm .0681/.0676 25.8/26.3 3.3 A +0.230 % -> -0.033 % 60.491 -> 60.267 mm .1193/.0835 9.2/14.9 The three committed distances now agree with each other to 0.013 mm and with the external value (60.26 +- 0.04) instead of drifting 0.22 mm with the wavelength. Anomalous peak heights at the known sulphur sites rise from 7.82/8.18 to 10.09/9.04 - the best previously reachable state was 10.08/9.10 and required knowing the answer. The bistability documented on the long-wavelength set is gone. Sweeping the header distance over 1.0 mm, the two-step commits 60.49-60.54 however it starts (or refuses outright, losing the dataset: R_meas 0.276, ISa 2.9); the joint fit commits 60.256-60.313, tracking the truth, and lands in the correct cell basin every time. Same from two starting beam centres. CORPUS. 115 datasets, paired against the tip, 113 processed in both arms: * cell deviation from the reference: 58 better, 38 worse, 16 unchanged (sign test p = 0.052); median 0.171 % -> 0.128 %. The asymmetry is in the magnitudes: the largest improvement is 2.4 percentage points, the largest regression 0.125. The 24.4-point figure at the head of that list is not a cell change at all - it is a hexagonal lattice the base arm described in its C-centred orthorhombic setting, which a sorted-axis comparison reports as a large difference; the scorer's own cause for that dataset is an under-called point group, and the primitive volume ratio passed in both arms. * POINT GROUP == reference 105 -> 106. Exact space group 82 -> 81: one screw-axis call (P2 -> P2_1) on a crystal whose R_meas is above 139 %, where the cell itself improved. * V ratio in [0.97, 1.03]: 106 -> 106. Datasets processed: 113 -> 113. * R_meas and ISa are confounded here: of the nine material regressions, seven reached a FINER resolution limit, which raises R_meas and lowers ISa mechanically. The two that did not are small (+0.55 pp R_meas, -0.9 ISa). One unconfounded rescue is large - R_meas 105 % -> 27 % at 0.72 -> 0.51 A on a small-molecule set whose uniform scale step A had refused. Cost: 90-150 ms on runs of minutes. Bit-identical across repeat runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 10:00:24 +02:00
leonarski_fandClaude Opus 5	0dfb123219	merge: a median of two is not a median, and the time axis wants a width ... Three changes to the rotation merge, measured together over 106 paired datasets: 31 better, 10 worse, 65 unchanged in R_meas, and two space groups rescued to their references. OUTLIER REJECTION. The per-reflection median was taken from two observations up. At multiplicity two the median IS one of the two, so it is immune to its own test and only the other can be rejected - against its OWN sigma. On a decaying crystal that pairs one live observation with one dead one, and wherever a dead-frame noise excursion is the larger of the pair the median becomes the noise and the LIVE observation is deleted. Measured on a room-temperature sweep at multiplicity 1.96: 92% of rejected observations came from the live half against a 60% baseline, and correlation with an external reference merge fell from 0.966 to 0.845 overall and from 0.876 to 0.265 in the outer shell. The threshold is not the problem - 12 sigma rejects 1.35% of observations and gives the same answer as 6 - the rule is, and a median is only defined from three. Take it from three. THE REJECTION COUNT IS NOW REPORTED. Rejected observations are excluded from the merge and from R_meas and the CC(1/2) half-sets, which is right - those describe the data as merged - but it means an over-rejecting run looks BETTER by every number it prints. Without the count the failure above is silent, and it stayed silent for as long as the rule was wrong. Over the corpus the new key shows 325652 observations rejected, one dataset dropping 51971 of them. THE TIME-DEPENDENT ABSORPTION GRID. Its time axis was a fixed bin COUNT, so twelve bins is 7.5-30 deg on an ordinary sweep and over 100 deg on a multi-turn one, where a surface meant to follow a crystal drifting through the beam can no longer see the drift. Make it a fixed angular WIDTH, floored at the old count so it is bit-identical below 120 deg of sweep. On a four-turn sweep: R_meas 0.098 -> 0.086, ISa 10.9 -> 12.8. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-04 08:50:28 +02:00
leonarski_fandClaude Opus 5	025ecf066e	report: the spindle-to-symmetry-axis angle reaches p_report.txt, not only stdout ... Every rotation run already computes the angle between the spindle and the nearest symmetry axis, and writes it into the merge statistics text - which is stdout only. Unless the angle crossed the 15 deg warning threshold it therefore did not survive the run, so the ordinary "the mounting was fine" case could not be greppped, and an incomplete cusp could not be attributed to the mounting or cleared of it after the fact. Section 4 now carries the angle and the axis order as keys on every run that determined a space group. REPORT_VERSION moves to 7. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-03 23:57:21 +02:00
leonarski_fandClaude Opus 5	28a98e1a14	stills: a per-image number for how much of a sweep's blind cone this orientation loses ... A rotation sweep never records a double cone of half-angle asin(lambda/2d) about the spindle. That loss is normally repaired by the point group; it is not repaired when an operator's axis lies inside the cone, because the cone then maps onto itself. A still cannot know the point group, but it can see where the crystal's short lattice rows are, and a symmetry axis is always one of them - measured over 107 solved cells its length is 1.4x the shortest row at the median. The number is the fraction of the blind cone that a 2-fold about the nearest short row carries back into the blind cone: two equal caps of angular radius theta_max whose centres are 2*beta apart, which is the circle-lens area in beta/theta_max and within 0.035 of the spherical value even for a 55 deg cone. 0 means one sweep reaches everything symmetry could give; 1 means the row is on the spindle and the whole cone is lost coherently. No goniometer geometry enters, so the number describes the problem and leaves the remedy to the beamline. Free: it rides on the FilterFFTResults shortlist the indexer already builds, needing only the spindle, the wavelength and the frame's own resolution. FilterFFTResults gains an optional out-parameter for each row's peak prominence, which the length window is taken over. Measured on 384 stills of 22 solved crystals against the frames' own symmetry axes, with the same procedure re-run along five decoy directions per frame as the null: 0.93 of severe orientations reported severe, 0.010 of harmless ones reported severe, AUC 0.948, and the value tracks the true severity to 0.10 at the 90th percentile. Below 60 spots the misses triple, so that is where it stops answering. It is blind to a symmetry axis much longer than the crystal's shortest row: measured on synthetic stills the search grid finds a 150 A row every time, a 200 A row half the time and a 300 A row once in sixteen. Two cheap consistency tests refuse to answer for part of that regime and cost nothing on real frames, but they do not cover it - the reach is a documented property of the number, not something every frame can detect. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-03 23:57:12 +02:00
leonarski_fandClaude Opus 5	e5a471a5de	rotation indexing: drive the winning candidate to the fit's fixed point ... The candidate loop calls the geometry optimiser once per candidate. One round is enough to RANK candidates against each other; it does not MEASURE the geometry, and the first pass's fit is what the second pass integrates at. So the tilt the run integrates at is whatever one round of a shared starting point produced. Re-solve the winning candidate alone against a re-accumulated reciprocal cloud until the tilt stops moving - 20 rounds, 0.6 mdeg, offline only. Measured on a long-wavelength rotation sweep: R_meas 0.1369 -> 0.1193, ISa 7.81 -> 9.19, and anomalous density at the known sulphur positions of a standard test protein 7.01 -> 7.82 sigma at Met and 7.70 -> 8.18 at Cys. Converged by round 10; the extra rounds cost 50% of wall time on a 3605-image sweep. The point is not the size of the gain but that the answer stops depending on where it started. The tilt component parallel to the spindle is a gauge - it re-expresses the beam centre it was handed - and one round launders the starting beam position into it at 11% of the geometric one-for-one rate. Iterating drops that to 0.6%, which is what makes it a measurement rather than an echo. The existing soft restraint stays and is complementary: it bounds each step while iteration walks the pair to their joint optimum. Paired over 113 datasets. R_meas better/worse 36/16, 12/10 and 3/9 on the three arms; one crystal's space group is rescued to its deposited value, and one moves to the ortho-hexagonal setting of the same lattice - a known under-call whose merged data improved. Both movers sit where the tilt is ill-conditioned. Also: the report claimed the refined tilt "is never written back onto the geometry". It is, first pass to second. Two statements corrected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-03 18:54:53 +02:00
leonarski_fandClaude Opus 5	491167c90d	rugnux: a Bravais class the reduction decided by rounding is re-asked on the metric's own cell ... The class is named by Niggli-reducing the indexed cell and looking the reduced cell up in the 44 lattice characters, and that lookup is a coin flip for any lattice whose Buerger cells straddle the Niggli type-I/type-II boundary. An F-centred cubic lattice does so by construction: it has reduced forms on both sides, the two sides carry different characters, and which side the reduction lands on is set by the last digits of whatever refinement produced the cell. Measured over 600 perturbations of one such lattice: 43% cubic F, 36% tetragonal I, 21% orthorhombic I, and the split is flat over a factor of ten in the noise. The class then caps the point-group search, so from the body-centred sub-cell the cubic three-fold is never enumerated and the run reports that nothing was refused - which is accurate, because nothing was asked. Le Page's two-fold search has no such key: it measures each rotation's obliquity on the lattice itself, in a primitive basis. LePageLattice turns the rotation group it finds into a conventional cell, a centring letter and an integral change of basis, and where that group is larger than the adopted class's holohedry the merge is reindexed into that cell and the space-group search is run again there, on both merges, with every gate live. Nothing here decides: the reindex is committed only where the search in the new setting confirms a strictly higher point group AND the centring the new cell describes, so a pseudo-symmetric metric leaves the answer already in hand standing. Measured as a paired battery over 113 rotation datasets: the re-ask fires on 7 and adopts on 1, and that one crystal - an F-centred cubic lattice the reduction had named body-centred tetragonal - moves to its deposited group, gaining 0.10 A of resolution and 2.8x the multiplicity at R_meas 0.117 -> 0.120. Nothing else moves, in space group, resolution, CC1/2, R_meas, multiplicity, I/sigma or completeness. A second such crystal, named body-centred orthorhombic, is offered the same cubic cell and confirms all 23 added operators at CC 0.96 with an H ratio of 1.00, and is still refused, on the merge chi^2 ratio at 2.50x a bound of 1.85. Letting H rescue that refusal is the one-line change an earlier round measured and rejected - it promotes the synthetic P 4_3 2_1 2 in the test suite to point group 432 - so it is not here, and that crystal is left where it was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-03 13:53:14 +02:00
leonarski_fandClaude Opus 5	203dbf417b	rugnux: two messages that asserted more than was measured ... The metric-symmetry warning told the user the intensities "did not support" the extra rotations. They may never have been asked. SearchSpaceGroup takes a lattice class from rotation indexing (SearchSpaceGroupOptions::lattice_system), and EnumeratePointGroups keeps only point groups whose rotation set is a subset of that class's holohedry - so where the class is lower than the metric, the extra operators are skipped before a single intensity is read, and their absence from the answer is not evidence against them. The warning now says the answer depends on the class the search was given. Placeholders and arguments are unchanged. The comment above LogXDSGeometry claimed the printed ROTATION_AXIS sign is simply wrong and that a user must negate it. Two things refute that. The line is printed once per pass, and the sign rescue rewrites the axis between passes, so a run's first and last line carry opposite signs on 12 of 115 corpus runs; "printed as stored" describes only the first, and the last is the axis the run integrated at. And the sign is not a constant: it is paired with the detector axis directions. Two datasets whose detector axes are both +x/+y do want the negation (8.9% of COLSPOT spots indexed as printed against 92.6%, and 1060/10080 against 9624/10080), but a third whose axes are negative wants the sign as printed (43.9% against 13.5%), so the universal instruction was wrong. The comment now says which line to read, quotes that statistic with its chance floor (a permuted-frame null reaches 14.9% against a real 15.4%, so the low members mean IDXREF gave up rather than a measured zero), records that one of the two runs read a file our own converter had written and so testifies about XDS rather than about foreign files, and tells the user to try both signs and keep the one that indexes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-03 13:49:32 +02:00
leonarski_fandClaude Opus 5	605312c2f4	twinning: the high-<|L|> comment says only what the corpus shows ... The L-test comment explained a high <|L|> as a structure whose intensities "behave centric", and a draft replacement explained it by few atoms, several of them on special positions, contributing a large term common to every structure factor. Both readings are wrong, and the second is wrong twice over. Over the 113 merged datasets in the corpus the maximum is <|L|> 0.712 with a second moment of 3.089. Those sit ABOVE the centric expectations, which are 2/pi = 0.6366 and 3.0 (simulated: 0.6367 and 3.003 for I ~ chi^2_1), so no Wilson distribution, centric or acentric, reaches them and they cannot be read as a statement about the structure's symmetry at all. The crystal carrying them is in an orthorhombic Sohncke group whose three 2_1 axes each show five absences against five controls with no violations - and that group has no special positions whatever, so the proposed mechanism cannot apply to it. Simulated, both halves of that mechanism move the statistics the wrong way: N randomly placed atoms give a second moment of 2 - 1/N, always below 2, and a large real term common to every F drives the moment from 2.00 down to 1.03 and <|L|> from 0.50 to 0.10 as it grows. The comment now gives the measured numbers, records that the cause is not established here, and names those two guesses as already excluded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-03 13:48:26 +02:00
leonarski_fandClaude Opus 5	c77ebe56c3	reader: a one-element string array is a scalar ... Masters exist that store sensor_material, description and the compression name as shape (1,) rather than as true scalars - JUNGFRAU files from an early beamline deployment do, and the application definition permits it. ReadString() required rank 0 and threw on anything else, so the whole file was lost: the metadata parse never finished and no image was ever examined. Accept any shape holding exactly one element, and keep throwing for a string dataset that genuinely holds several, which is a different quantity and cannot be read into one std::string. The numeric path already worked this way (HDF5DataSet_scalar_stored_rank1); this is the string half. Measured: three long-wavelength rotation datasets that could not be opened at all now process end to end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-03 10:58:38 +02:00