1199 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	c84b91be8a	calibration: take the beam centre from the rings too ... The header's beam centre was the last input the ring fit had to be roughly right about. Each ring is looked for in a window a few pixels of radius wide, and a centre wrong by (dx, dy) puts a ring at a different q in every sector, so past about ten pixels the ring leaves that window over much of the turn - and the fit then reads its cos(phi) signal off whichever sectors are left, which are the ones where the signal is weakest. A 20 px error ended 31 px wrong. The rings answer this without a calibrant and without a distance. A powder ring is a conic centred on the beam, so a wrong centre makes EVERY ring's radius oscillate once per turn by the same amount: r(phi) = R + dx cos(phi) + dy sin(phi), solved directly and pooled over every ring the profile shows, with each ring searched about its own measured radius rather than about where a standard says it should be. Using it needs the extraction to follow the rings sector by sector, which is what ProfileRingTrack now does - exactly, and in all five parameters at once, by walking the ring in the geometry believed true and asking the binned geometry what q and azimuth it would have given each point. That replaces the flat-detector distance correction it grew out of. Following the rings is not free, and the reason is worth stating: a window that moves with phi makes every systematic of the peak finder - where the background line is taken, how the centroid sits in the window - vary with phi as well, and phi is exactly the axis the beam centre is read off. Measured, it costs rms 0.415 -> 0.525 px on a good 110 mm fit, and 0.831 when the window follows the fitted tilt too. So a second measurement is taken with a window that is the same in every sector - the binned geometry with only its DISTANCE replaced, which is phi-independent by construction - and both are offered to the same rule that ranks everything else here. Acquire by following, measure by holding still. The seeded centre is likewise a hypothesis and not a belief. It reads a once-per-turn wobble, and a tilt puts a term of that shape there too - one that grows as the radius squared, where a centre error does not - so pooling the rings absorbs part of the tilt into the centre. Believed outright it made a good 110 mm fit worse; offered as an alternative start it costs one more fit and needs no rule about when it applies. It is skipped entirely below a pixel, where it is not a different hypothesis at all, which keeps a well-headed run at 0.71 s. Measured on the 110 mm LaB6 exposure, whose true PONI is 765.90: a header centre 20 px out now lands within 0.5 px, where before it landed 31 px away. All five datasets are unchanged from their correct headers, and the distance still recovers from any header between 25 and 1200 mm. The limit is now understood rather than merely reached. Past a few pixels the azimuthally averaged profile stops showing rings: a ring tracing r(phi) piles up density where that turns round, so it averages into the two HORNS of the sinusoid, at R-|d| and R+|d|. The radius finder reports two rings where there is one, and the gap between them is 2|d| - the search window shrinks to exactly the offset it was meant to span. That caps recovery at roughly half the ring spacing, about 20 px here and failing by 40. Beyond it nothing is left in an azimuthally binned profile, and --calibration spots, which works from the spot positions themselves, is the method that still can. One pre-existing limit measured and NOT introduced here: a wrong distance together with a centre more than about 5 px out fails, because the centre error splits the radius list the distance search reads. The committed code before this change fails identically on those cases. Also fixed: fit_from now takes a whole geometry rather than a distance, and the declined-tilt refit was inheriting rot1/rot2 from it - pinning the tilt at exactly the unvalidated value the gate had just rejected. Same fault the gate exists to catch, one level up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:38:36 +02:00
leonarski_fandClaude Opus 5	9267bd67de	report: the detector tilt, and the two points the beam centre is not ... BEAM_CENTRE is the PONI - the foot of the perpendicular from the sample - so on a tilted detector it is not where the direct beam lands. The report printed only that one number and never mentioned the tilt at all, so the two points were indistinguishable to a reader. On a detector at 85 mm with a 0.22 degree tilt they are 8.3 px apart. Print DETECTOR_TILT (rot1/rot2/rot3, degrees, as the run used them - nothing is refined here) and DIRECT_BEAM beside BEAM_CENTRE, and say which is which. JFJOCH_DATASET_SETTINGS was the more serious half: it exists to carry a geometry back into the instrument, and dataset_settings carries poni_rot1/2/3_rad, but the block omitted them - so it described a FLAT detector, silently, on every tilted setup. The rotations now ride with it when they are non-zero; they stay out when they are zero, because the API's own default is 0.0 and the shorter block means the same thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 17:35:03 +02:00
leonarski_fandClaude Opus 5	01ad1e1743	calibration: only report a tilt the fit actually measured ... The detector tilt is refined by default, and on a pattern that cannot separate it from the beam centre the fit returns one anyway - there was nothing to stop it. Both displace a ring's radius as cos(phi), and only how that amplitude grows with the ring's radius tells them apart, which takes two well-sampled rings. At 500 mm on the LaB6 series only two rings reach the detector and the outer one is barely there: the tilt came out at the opposite sign to every shorter distance, dragged the PONI 28 px, and bought a residual of 0.960 px against 0.962 pinned. The covariance says so plainly - 0.1 sigma, and a beam centre quoted to +-180 px. So ask it. A tilt is kept only where the fit had it free AND it stands at least three times its own uncertainty; otherwise rot1/rot2 go back to the header's values and the beam centre and distance are refitted around them. Over the series the tilt stands at 50, 33, 15 and 8 sigma at 110 to 300 mm and 0.1 at 500 mm, so any threshold between 2 and 5 gives the same verdict on all five - this says which regime a fit is in, not where a line was drawn. The declined 500 mm fit lands on a direct beam of 773.53 px, against 773.56 for the pinned fit measured independently. It is a rejection criterion and nothing more. Clearing it does not certify a tilt: that estimator is limited by systematics rather than by this sigma, and a coherent half-pixel error in the ring positions fakes a tilt of the usual size while leaving sigma small. The report says "refined", never "verified". Writing the gate turned up a related fault in the pass loop. RingOptimizer pins the tilt by itself when every point it is given lies on one ring, and on a barely-sampled pattern a later pass lands in exactly that state - which froze the tilt at whatever the FIRST pass had produced and returned it with sigma zero, an unmeasured tilt wearing the appearance of a fixed one. The gate reads that as "not measured" and refits pinned, which is why it is stated over the geometry that gets reported rather than over what the last fit happened to do. Both paths are covered, profile and spots; the spots path was reporting a refined tilt as declined for the same reason. Two things measured and NOT taken: A robust loss. A Cauchy loss scaled to the previous pass's median residual changed nothing on the series - rms 0.415 to 0.421 at 110 mm, no case improved, every direct beam within 0.06 px. Ring points are per-sector peaks that already had to stand 3 sigma clear of their own background, so there are no gross outliers left to reject. Recorded at the call site rather than left as an unused option. A quality gate that refuses a bad calibration. Three candidate signals, all measured against naming the wrong standard on LaB6 data: sigma(PONI) does not see it at all (0.52-0.65 px, indistinguishable from healthy); the residual only half sees it (3.2-3.6 px wrong against 0.4-1.0 right, but a correct run from a wrong header sits at 1.0-2.4 and would be caught too); and the seed's match score is dominated by how many rings the calibrant lists, scoring 0.29 for a perfect LaB6 fit against 0.21 for a wrongly named silicon. None of the three separates, so no gate is shipped. What the run does say is the recovered distance against the header, and a wrong standard moves that to 446 mm on a 110 mm exposure - unmissable, and the operator's call rather than a threshold's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:07:37 +02:00
leonarski_fandClaude Opus 5	a5f416fcdc	calibration: take the detector distance from the rings, not from the header ... A powder calibration is run because nobody is sure the header is right, and the header's distance was the one number the fit could not survive being wrong about. The ring search is local - each ring is looked for inside a window a few pixels of radius wide - so a distance more than a percent or two out puts every ring outside its own window, and the fit then converges on whatever background fluctuation each window contains. It does not fail: a 110 mm exposure told the detector was at 150 mm reported 149.8 mm, with 146 ring points and exit 0. Only its residual said anything, 5.3 px against 0.4 px, and nothing read it. Measure the distance from the rings instead. The peaks of the azimuthally averaged profile give ring RADII, and a radius does not depend on the assumed distance at all - bin i holds the pixels at one particular radius whatever q that radius was called - so the radii are a property of the image. Against the calibrant's d-spacings, r = D tan(2 asin(lambda/2d)) then has one unknown. It is scanned rather than solved because the pairing of observed rings to d-spacings is unknown too, and the winning basin is solved in closed form. Nothing here reads the header distance except to bin the profile; it needs only the wavelength, the pixel size and the detector's extent. A powder pattern has genuine distance aliases, so one answer is not enough. A cubic primitive standard puts its rings at radii proportional to sqrt(N), and scaling the distance by sqrt(2) maps ring N onto ring 2N - most of the comb still lands on peaks. Measured: the 110 mm exposure with a 115 mm header scored its best at 156.5 mm, which is 110*sqrt(2). No adjustment of the score removes an alias the lattice really has, so the scan hands back the few best distances and each is fitted, the header among them as one hypothesis of several. The residual then separates them - 0.4 px against 5.2 px on that case - subject to an attempt explaining a comparable share of the pattern first, because a start so wrong that one ring point survives leaves a residual of exactly zero. Each attempt re-extracts at the geometry it converged to and fits again. The seed is measured from blended peaks and is good to about a per cent, close enough to converge from but far enough to sit every search window a few pixels off its ring, and an off-centre window takes its background off the ring's own flank. Nothing is re-read from disk, so the loop is free. Measured on the LaB6 distance series. A 110 mm dataset now recovers 110.03-110.17 mm from any header between 25 and 1200 mm, against +-2 mm before. All five datasets recover their own distance from a fixed wrong 250 mm header. With correct headers, four of the five are bit-identical to before and the 500 mm one moves by a single ring point - the two-ring fit whose tilt is 0.1 sigma anyway. Run time is unchanged at 0.62 s. The residual is larger on a run whose header was wrong (1.1 px against 0.4 px on the 110 mm case), because the profile was still binned at the wrong distance and its radial sampling is correspondingly coarse. The geometry is right; only the scatter about it is inflated. Re-running with the recovered distance recovers the residual too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:48:32 +02:00
leonarski_fandClaude Opus 5	d275bbcae7	calibration: -C overrides the calibrant, with absences from -S ... The five named standards are a convenience, not the limit of what a powder calibration can be run against. A unit cell given with -C now IS the standard in --mode calibration, and its rings are enumerated from that cell. A cell alone does not give a ring list, though: the centring and any glide decide which hkl the lattice actually diffracts into, and the fit pairs the innermost OBSERVED ring with the innermost LISTED one - so a list opening with a reflection that is not there scales the whole calibration by the ratio between them. Where -S is given, the absences come from the space group itself via gemmi, which covers centring, glides and screws in one mechanism rather than the three hand-written conditions the built-in table uses. The two agree exactly on LaB6 and CeO2, which is the cross-check that says the gemmi route is safe to hand a user's cell. They do NOT agree on silicon, and the test now pins that: Fd-3m's symmetry absences are only the F centring, while silicon's 222 and its relatives are extinguished by its two-atom basis - a structure-factor absence, not a symmetry one, so no symmetry handler can know it. gemmi offers 24 rings where the diamond condition gives 18. The extra rings do not move the first one, so the distance is not scaled, but they are rings with no intensity offered to the matcher - which is why --calibrant si stays, and why the usage text says -S supplies symmetry absences only. Without -S the cell is taken as primitive and the log says "assumed primitive". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:23:37 +02:00
leonarski_fandClaude Opus 5	5a80d2df53	calibration: fix four ways the powder fit quietly loses its input ... None of these changes the answer on a well-separated cubic standard - the LaB6 distance series is bit-identical by both methods - but each one is a case where input is dropped or mis-assigned without saying so. The circumcentre vote grid was a fixed 4000x4000 box, and the caller never passed anything else. That allocated 128 MB whatever the detector, and on a detector larger than 4000 px in either direction it put the beam centre outside the grid, so every vote was discarded and the guess failed with "Beam center not found". Span the spots' own bounding box instead: a powder ring encloses its centre, so that is where the answer has to be. uint32 votes while there - the most any bin can take is C(500,3). Spots were assigned to the FIRST calibrant ring within a fixed 0.1 1/A, not the nearest. Silver behenate's orders sit 0.108 1/A apart and hexagonal ice has three rings inside 0.06, so for those two standards the window reaches the neighbour and every point lands on the lower-q ring of the pair, biasing the distance. Take the nearest ring, and clamp the window to half the gap to the neighbour - which is what the profile path already did inline, now shared as RingMatchWindow and covered by a test that checks it actually narrows on the crowded standards and not on LaB6. A profile bin no pixel fell in is NaN. SectorPeakQ dropped such a sector by accident, through NaN comparisons falling false; check the four background bins and return explicitly. Ice-ring handling is switched off in calibration mode. Flagged spots are sorted last by the spot budget and so discarded first, which for --calibrant ice throws away exactly what is being calibrated on. The two per-ring std::cout lines in GuessGeometry are gone: a library has no business writing to a terminal, and constructing a Logger to keep them would emit a version banner from inside a fit. What matched belongs in the result struct, which the quality gating still to come needs anyway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:18:27 +02:00
leonarski_fandClaude Opus 5	fba5435c38	calibration: report what the ring fit knows about its own answer ... The ring fit reported the SCATTER of its measurements (rms, and the beam-centre standard error that follows from it) but nothing about how well the fit pinned each parameter. Those two part company exactly where a calibration is worth doubting: as the rings run out, the tilt and the beam centre stop being separable - both displace a ring's radius as cos(phi) and only the way that amplitude scales with radius tells them apart - so the fit can sit tightly on the few points it has while being free to spend tens of pixels of beam centre on a tilt the data do not support. Take the covariance of the converged problem from Ceres and report it. Measured on a LaB6 distance series, the fitted tilt is 50 sigma at 110 mm and 0.1 sigma at 500 mm, where only two rings reach the detector; at 500 mm the fit quotes its own beam centre to +-180 px and its tilt to +-2.9 deg on a 0.35 deg value, and the correlation between them is 1.000. Nothing acts on this yet - it is printed so the next change can gate on it. Ceres returns the bare (J'J)^-1 of an unweighted problem, so it is scaled by chi2 per degree of freedom; that leaves the sigmas in pixels, mm and radians whatever unit the residual is stated in. Cost is one 5x5 SVD per run, below the noise of the surrounding I/O. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:11:18 +02:00
leonarski_fandClaude Opus 5	f34d9fe62b	calibration: move the powder fit into image_analysis/geom_refinement ... CalibrateFromProfile/CalibrateFromSpots/WritePoniFile sat in rugnux/, so the only way to reach them was to link the Rugnux library - which drags in JFJochWriter and gemmi. Nothing in them needs either: the includes are all common/ and image_analysis/geom_refinement/, next to the RingOptimizer and RingsFromProfile they call. Moving them to image_analysis/geom_refinement/PowderCalibration.{h,cpp} puts the powder fit beside the rest of the geometry refinement and makes it reachable from anything that already links JFJochImageAnalysis - the receiver and so the broker included, which is what an online geometry calibration would need. Pure move: the file contents differ from their previous form only in the include paths. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:09:42 +02:00
leonarski_fandClaude Opus 5	ba24946c0f	twinning: do not report a twin the L-test rules out ... The two indicators can only move one way under twinning - <|L|> down from 0.500 toward 0.375, the second moment down from 2.0 toward 1.5 - but they were combined with an OR, so a narrow intensity distribution could report a twin on its own while the L-test said the opposite. That is not a marginal disagreement. It fires on small-cell rotation data, where a twin fraction of 0.50 was reported for a crystal whose <|L|> was 0.63: a value no twin can produce, and one that points at the opposite situation - a structure whose intensities behave centric. Suppress the flag when <|L|> sits at or above its untwinned value; the two indicators are otherwise combined exactly as before. Measured over 113 stored reports: two flip, both of them wrong today, and nothing else moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 15:34:16 +02:00
leonarski_fandClaude Opus 5	71cca91ec1	rugnux: rank screw candidates without the control count ... A screw's control class is the complement of its absent class on one axial row, so the two move together: a candidate that predicts more of the row absent leaves fewer reflections to be judged against. The Beta tail AbsenceEvidence computes grows with that control count, so the stricter candidate was charged for the very reflections it correctly called extinct, and a group whose predicted-absent class is a strict superset of another's - with the extra reflections equally dead - could score LOWER than the group that explains only part of the row. Two mechanisms, one dataset each. * The control-count minimum gated the row MEAN, which is the scale the zone's evidence is stated in, on the same count as the row MEDIAN, which is a violation threshold. A candidate could therefore forfeit a whole zone by being right: on a tetragonal 4_1/4_3 wedge, thirteen 00l reflections measured at 0.1% of the two l = 4n beside them scored ZERO because only two control reflections were left, while the nine of them a 4_2 also predicts absent scored 38.7. The mean now stands on two, the median still on three. * The Beta tail is replaced, for a SCREW zone only, by its b -> infinity limit - the same statistic with the control count dropped. A p-value computed against each candidate's own null is not one scale across candidates; what is left is the likelihood ratio of the absent class against Wilson at the row's own mean, which is a sum over reflections and therefore comparable. Asymptotically it is n_absent * (log(1/ubar) - 1), so an equally dead superset can no longer score lower. Measured on a tetragonal 4_1/4_3 crystal: 29 dead 00l against 8 control read 47.7 nats where a subset of 19 of them against 18 control read 55.5. The centring statistic is untouched: its control is the whole present population, not the complement of a claim on one row, so neither mechanism applies to it. Measured. Analytic superset monotonicity, on a 176368-point grid with no calibration and no corpus: 1.29% violations -> 0.00%, exactly monotone wherever both candidates keep a control class. Over 112 stored merges (open, in-house and private arms) exactly ONE decision changes and it is a gain, a 4_2 -> 4_1/4_3 swap on a deposited 4_3; the screw column over the deposited arm goes 33/41 -> 34/41 with zero screw over-calls before and after, and the four centring outcomes are bit-identical. An eight-wedge reproducer whose 00l row a strict candidate eats goes from two mis-calls to none, end to end. The refused side moves the safe way: 103 zones the old statistic scored negative go lower, none turns positive, and the minimum evidence of an adopted screw rises 20.4 -> 23.8 against an unchanged bound of 20. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 15:33:29 +02:00
leonarski_fandClaude Opus 5	c25418319a	report: the twinning measured before the search, and the strong-direction limit ... Three additions to the report, none of which changes a number the run computes. The twinning statistics printed today are measured in the Laue class the run ADOPTED, so where the search promoted the point group they can say no more than "no twin law exists inside the class it chose" - and a twin is precisely what would have caused that promotion. The uncontaminated pair was already computed on the subgroup merge the search was given and reached only a log line; it is now printed as L_TEST_MEAN_ABS_L_BEFORE_SEARCH and SECOND_MOMENT_I_BEFORE_SEARCH, beside the post-adoption pair rather than instead of it, since the two answer different questions. ANISOTROPY_D_MIN_BEST names the finest of the three principal limits so it can be grepped, and a paragraph distinguishes the three resolutions the report now carries: what was written, where the isotropic fit crossed, and how far the crystal reaches where it reaches furthest. It is read off the fitted tensor rather than off the cut, so it can land either side of the other two - measured on one dataset at 2.01 A against a 1.81 A cut - and the text points at ANISOTROPY_D_MIN_CENSORED for when it is the edge of the data instead. A warning when CC1/2 falls and then climbs again, which means the fall-off the cut is read off does not describe the data. A climb counts only above CC1/2 0.30 and only if it exceeds 0.05: past that the number is oscillating in noise beyond any cut this run would take. It advises no re-cut - the generous cut is deliberate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 14:06:42 +02:00
leonarski_fandClaude Opus 5	6491345fea	comment: say what the refined geometry in the end message actually is ... Two things the field names hide, both verified in code and both able to mislead a reader into publishing a wrong geometry. The beam centre written here is the PONI - the foot of the perpendicular from the sample to the detector plane - so on a tilted detector it is not where the direct beam lands. That point is D*tan(rot)/pixel away and is GetDirectBeam_pxl(). And the tilt is not written only for the record, which the old comment implied by mentioning nothing but the _process.h5: IndexAndRefine puts it on outcome.experiment, which is the geometry Bragg prediction and integration then run at. It is fitted with the distance frozen and nothing checks the covariance, so the split between the PONI and the tilt is not reproducible between passes even though their sum - the direct beam - is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 14:06:03 +02:00
leonarski_fandClaude Opus 5	d189faeec7	viewer: draw grid scan cells in the proportion of the scan steps ... A grid scan with, say, a 20 um step in x and a 5 um step in y was drawn as a square grid, so the picture had nothing to do with the shape of the area that was scanned. The cells now carry that proportion. The anisotropy lives only in the view transform: JFJochImage gains a pixel_aspect_ (the drawn height of one pixel in units of its width, 1.0 for a detector image), the initial fit fits the image as if it were that much taller and then puts the factor back into the transform, and the grid view sets it from |step_y / step_x|. Scene coordinates stay one unit per cell, so the mouse mapping, the selected-image box, the pixel labels and the ROI code need no change and the uniform wheel zoom keeps the proportion. Measured on an 8x4 grid driven offscreen: m11/m22 comes out 4.000 for steps 20/5 um, 0.250 for 5/20, 12.333 for 37/3 and exactly 1.000 - the old behaviour - for a square step. Every cell centre still maps back through mapToScene to its own cell in all four cases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hcoh6VNrmSswjfeMeDQqWP	2026-08-31 13:38:51 +02:00
leonarski_fandClaude Opus 5	756c3a22bb	docs: confirmed cells for three small-molecule sets; drop the unprocessable CCD one ... The four Diamond I19 small-molecule datasets had no reference of any kind, so a run on them could not be scored at all. Three now have one, each from a published structure with its DOI verified against Crossref; the fourth has a published space group but no numeric cell anywhere, which is recorded as such. Also removes the "obtained but not in the table" section. That dataset is stored as CCD TIFFs the reader does not support, so it is not processed here and adds nothing to a page about what the test corpus contains. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 11:02:38 +02:00
leonarski_fandClaude Opus 5	6d37b31948	report: give the resolution the CC1/2 fit reached, not only the one it was cut at ... The automatic cutoff fits the CC1/2 fall-off, takes the crossing of the target (0.30 by default) and then deliberately keeps one more shell. Only the extended limit was reported, as INCLUDE_RESOLUTION_RANGE, and that is the number a reader takes for "the resolution of this dataset" - so the report quoted a limit that is generous on purpose as if it were the measurement. The generosity itself is right and stays: a shell that is included can still be downweighted or dropped by refinement, while one that was truncated cannot be put back. What was missing is the other number. ComputeCCHalfLogisticCutoff now also returns the crossing, ApplyResolutionCutoff passes it out, and the report prints it as FITTED_RESOLUTION beside the range the reflections were written to, with a sentence saying which is which. The log line names both as well. Nothing about the cut, the merge or the written reflections changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 11:02:37 +02:00
leonarski_fandClaude Opus 5	55d9e2536f	release: 1.0.0-rc.166 ... The changelog entry for this release is six lines, not the fifty-four the development log had accumulated: what a user of rc.166 gets is native miniCBF input, masters from other facilities opening, the beam centre measured on every run, a 2theta-swung detector placed where the file says, the unmerged MTZ and a P1 merge written by default, and a substantially reworked symmetry determination. The per-change detail is in the commits. update_version.sh regenerates the three clients from broker/jfjoch_api.yaml, so this also carries the one API description that had drifted from the generated code - fft_high_resolution_A, which sizes the FFT's projection histogram and does not filter spots by resolution. Everything else in the regenerated tree is the version string. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 09:11:22 +02:00
leonarski_fandClaude Opus 5	3a30dc8ab5	rugnux: a space group that names twice the cell is a signal to halve it ... C 1 1 2 and C 1 1 2_1 are the only two monoclinic settings whose change of basis from the reference has determinant 1/2, and the reason is physical: with unique axis c the C-centring vector lies wholly in the oblique a-b plane and reduces away. Such a group is never a distinct Bravais centring - it is a primitive lattice written on twice the cell - and it wins the candidate table only because the table is scored on that doubled cell, where half the reflection positions are absent by construction. Refusing to enumerate those settings was measured and is worse: the volume-preserving alternative keeps the by-construction absences as real observations and merges no better than P1. So keep the enumeration as it is and act on the win instead: halve the cell, re-classify in the group's own system, carry the reflections across (the invented positions map to non-integral indices and are dropped, which is what the centred candidate was already doing) and search again there. Measured on one rotation crystal: C 1 1 2_1 on 68.6 73.0 170.3 90 90 90 becomes P 1 2_1 1 on 50.1 170.3 50.1 90 93.5 90, exactly the deposited setting, with every merge statistic unchanged (R_meas 0.1209, CC1/2 0.9979, d_min 1.238) and ISa 18.27 -> 18.31. Stable across +/-0.5 mm of detector distance. Eight other crystals are bit-identical, and across 112 stored battery reports exactly one adopts a volume-doubling setting. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:19:38 +02:00
leonarski_fandClaude Opus 5	5e3a7d113c	rugnux: keep a rotation-axis sign the first pass corrected ... The two-pass post-refinement snapshots the goniometer before pass 1 and restores it after, to undo the start/stride shift a sub-range run applies inside the pass. But the rotation-axis sign rescue also runs inside the pass, and adopts the opposite axis when the file's own indexes almost nothing - so the restore threw that away, handed pass 2 the sign that had already failed, and applied pass 1's fitted rotation scale to the axis it was not fitted on. It self-heals: pass 2's rescue fires again, and a genuinely wrong sign indexes almost no frames, so the run reaches the same answer having paid for one redundant first pass. But the correctness of that rests entirely on "a wrong sign always scores below half the validation frames", which nothing states and nothing tests. The restore now keeps the axis direction and reverts only the angles, which is what it was for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:41 +02:00
leonarski_fandClaude Opus 5	7c3ab72fed	reader: refuse a CBF that is not a detector image instead of misreading it ... Three ways a miniCBF opened silently wrong. A byte-offset CBF with no PILATUS header at all was accepted: Count_cutoff then defaulted to 0, SaturationLimitFromValue(0) is 1, and every pixel at or above one count was flagged saturated - the integration accept gate drops the whole reflection, so the run comes out empty for a reason nothing reports. The pixel size defaulted to 0 with no validation anywhere downstream, which collapses every resolution, every scattering vector and the beam centre in millimetres. XDS writes its correction files in exactly this shape, so this is not hypothetical. A pixel size is now required to claim the file at all, and a missing Count_cutoff leaves the saturation limit unset - falling back to the container's own overflow, which can only fail to call a pixel saturated - with a warning saying so. And the header captures are character classes, not number grammars: "[\d.eE+-]+" matches a bare "." and "(\d+)" matches a digit string too long for int64. std::stod and std::stoll answer both with a raw std:: exception, which escaped the format probe - CanRead catches JFJochException only - so merely LOOKING at a corrupt file threw out of the viewer's open path. A header field that does not parse is now reported as a malformed header. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:40 +02:00
leonarski_fandClaude Opus 5	8d719c1383	symmetry: a zone whose absences were never measurable must not outrank a dead one ... sum_u is a sum of max(0, E^2) over a zone's predicted-absent reflections, in units of their control class's mean, so it is EXACTLY zero when every one of them merged non-positive. The Beta tail then diverges and the clamp that caught it made each such reflection worth about 690 nats. That was harmless while the value only had to clear a bound of 20. Since the absence evidence became the primary sort key, summed across zones with no minimum count per zone, it has ranked the candidates - and it ranked them backwards. A zone of two absences that were never measurable scores 1378 nats where a genuine zone of six absences at 1% of its own row scores 22, so the candidate claiming a screw on an UNMEASURED row beat the one whose rows are actually dead, by sixty times. The comment on the ranking claimed an extra condition "LOSES the zone's evidence when the row is not dead"; an unmeasurable row could not lose, it won outright. No measurement places a merged intensity at exactly zero, so sum_u is floored at a thousandth of the control mean per absent reflection. The constant is an order of magnitude below the precision any real merge reaches - a thousandth of the row mean needs I/sigma ~ 1000 against that row, where ISa tops out near 40 - so it can only remove the singularity, never suppress evidence a measurement could have produced. Measured across the realistic range it changes no genuine zone at all (22.0, 34.7 and 65.4 nats, unchanged to four figures) and takes the unmeasurable ones to 13 and 36, below and around the bound respectively. AbsenceEvidence moves to the header, where the comments already named it, so the ordering it has to satisfy can be tested directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:26 +02:00
leonarski_f	e0ff51e2a0	lattice: transpose the change of basis to a primitive cell, as everywhere else ... gemmi states centred_to_primitive as an operator on COORDINATES and CrystalLattice::Multiply combines BASIS VECTORS, so the matrix has to be transposed on the way in - as it already is at both of the other places a gemmi Op::Rot reaches Multiply, one of them in this same file. A, B, C, I and F are symmetric matrices, so for them the transpose is a no-op and the omission never showed. R and H are not. Measured on an R-centred hexagonal lattice, ToPrimitive('R') returned 59.5 81.7 43.3 / 145.6 124.5 46.7 where the rhombohedral primitive cell is 49.3 49.3 49.3 / 60.9 60.9 60.9. What hid it is that a determinant is unchanged by transposition, so the VOLUME came out right - and most callers only take the volume. It is not only cosmetic: the result feeds the re-seating path that puts a lattice into a space group the user fixed by hand, so an R-centred lattice was handed the classifier a "primitive" cell that is not that lattice - broken for exactly the centring whose setting most needs re-seating. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 6ca00e927d664e870515c04164defa81d8a18725)	2026-08-31 07:17:00 +02:00
leonarski_f	527a5187f4	lattice: restore the character that names a centred monoclinic mI reduced form ... ITA character 43 was absent from the table. It is the type-II reduced form of a centred monoclinic lattice with no length equality - mC, mI, mA and mF are one Bravais lattice in four settings, and this row names the one whose conventional cell comes out I-centred. With the row missing such a cell reaches character 44 and is reported as triclinic, losing its centring outright. It accounted for 30 of the 31 demotions left after the two fixes before this one, and for 4.1% of random centred-monoclinic lattices. Both of its conditions are equalities on scalar products - International Tables gives them as 2|D+E+F| = A+B and |2D+F| = B - so the three angle tests are vacuous for it and the second equality is what selects it. cond_2DF was declared in the character struct and never tested against anything, because until now no row used it. Audited over 14000 exact lattices, the row fires 35 times: 32 are centred monoclinic lattices it recovers correctly and 3 are triclinic cells it promotes, all three of which Le Page promotes at the same tolerance. No over-call is attributable to it. The row is taken from International Tables rather than derived - no single integer matrix of determinant 2 covers more than 68% of these cells - and the test checks the conventional cell it produces has two right angles and twice the primitive volume. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 4c2baf0cc3b2177196a9448df06e24b5446de96a)	2026-08-31 07:17:00 +02:00
leonarski_f	931d7acc7c	lattice: judge a structurally-zero scalar product against the size of the cell ... The Niggli type of a reduced cell is the sign of its three scalar products, and gemmi's reduction was asked to decide those signs against an absolute tolerance of 1e-9 while the products themselves are 10^3 to 10^5 A^2 and the cell is held in float. A product that is structurally zero therefore arrives carrying about 1e-4 A^2 of rounding and is read as definitely signed. The reduction lands on the wrong side of the type-I/type-II boundary, the character written for the other side matches nothing, and the lattice comes back with less symmetry than it has. It is not a corner case. Take an exactly body-centred tetragonal lattice with c > a*sqrt(2) and merely ROTATE IT IN SPACE: 38 of 60 rotations lose the 4-fold, and it comes back C-centred. Every lattice this code classifies is a refined, rotated one; the two tetragonal-I cases already in the tests are axis-aligned, which is the one corner where the rounding vanishes. The tolerance is now scaled by the cell's own magnitude. Over 14845 exact lattices that puts it on a plateau three decades wide - demotions 461 -> 31 - with the over-call count unchanged at every point of it, and with the three fixes that follow the plateau is four decades wide and the demotions are zero. The constant is 100x the one Grosse-Kunstleve et al. give because theirs is calibrated for a double-precision cell and this one is float: measured, their value recovers 6% of these lattices and this one 93%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit b82f217719450e1e46e94be5b5a43ed8ffbf2ca4)	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	b31fb0a55c	symmetry: take the point group from whichever merge found it, the absences from all observations ... The search runs on two merges - one holding every observation, one holding only the well-measured ones - and the second was allowed to report but never to decide. That cost real symmetry: over the corpus the two arms disagree on eight crystals and the arm that found the LARGER point group is right on all eight. A promotion is refused by the most damning statistic it can be shown, so only one of the two arms has to be starved for a genuine operator to be thrown away, and which arm that is depends on the crystal. But the two halves of the answer do not come from the same place. Absences live in the weak reflections the filter discards, so adopting the filtered arm whole recovers the point group and loses the screw - a P6_5 crystal comes back as P6. So the arms are split rather than ranked: the point group is taken from whichever supports the larger one, and its absences are then judged here, on every observation. SearchSpaceGroupOptions::fixed_point_group was written for exactly this and had no caller. Measured on the deposited arm: a trigonal-3 answer becomes P 6_1 against a deposited P 6_5 (an enantiomorph pair, so correct), and a monoclinic answer becomes P 2 2 2 against a deposited P 2_1 2_1 2. The screw survives the split, which it did not when the filtered arm was adopted whole. No crystal that was already right moved, and the over-call column is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	d35e8f680f	symmetry: say when the cell metric hosts more symmetry than the group adopted ... Every symmetry under-call in the corpus has the same signature: a lattice whose metric carries rotations the adopted group does not, with nothing in the run saying so. The user is left to notice that a P1 answer sits on a cell whose axes are equal and whose angles are 60 degrees. So a run that determines its own group now compares the two and says what it sees. It decides nothing - no threshold, no promotion, no demotion, no reprocessing - and the message says as much, because a pseudo-symmetric metric is ordinary and only the intensities can settle whether the extra rotations are real. Measured over 83 crystals it fires on 19 and covers 6 of the 7 point-group under-calls; on the crystal whose deposited group is F 4 3 2 and which rugnux reports as P 1, the line reads "the cell metric is cubic - it admits 24 rotations where P 1 has 1". The metric symmetry is Le Page's, taken through GEMMI's implementation, at an obliquity of 1 degree - the middle of the band over which the answer is stable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	493be2cf19	symmetry: enumerate the settings the cell can host, not only the reference ones ... The space-group search offered a candidate only if gemmi calls it the reference setting. A setting is a statement about direction, so that restricted the search to the axes the convention chose: a crystal whose 2-fold lies on c had no rung between P1 and 222 and fell to P1, and one whose screws lie on b and c was reported as the group with a single screw on c - the wrong group, not a lower one, because the candidate that predicts a subset of the real absences and nothing else wins on no evidence at all. Both stages now enumerate more, under refusals rather than thresholds. Stage B offers the non-reference settings of the chosen point group. Their rotation set is equal to the chosen one, not merely contained in it, so this cannot raise the symmetry; what it adds is a screw or a centring on the axis the data show it on. A candidate is offered only if the cell's own metric admits the rotations its setting names, and one predicting exactly the absences another candidate already predicts is dropped as the same hypothesis under a second name. A non-reference candidate whose centring class this merge does not contain is refused outright: the reference path may adopt an untested centring because the caller's centred-lattice re-test backs it, and a non-reference setting has no such backing. Stage A offers the rotation sets no reference setting carries - the a-unique and c-unique monoclinic 2-folds, and the two rhombohedral-axes trigonal groups - and only those, so every point group reachable before is still reached by the same group in the same setting. A rung reached only that way may be ADOPTED but does not judge anything else: it is skipped when the reference chi^2 is formed and when a higher promotion's parents are collected. Without that it made higher promotions strictly harder - a promotion answers to the most damning of its parents, and offering two more order-2 subgroups of 222 refused 222 and 432 on crystals that had them. The run report gains SPACE_GROUP_NAME beside SPACE_GROUP_NUMBER, since the number alone does not say which axes a group's symmetry lies on, and a group that is not a reference setting is printed as its extended Hermann-Mauguin name. The two-arm reconciliation compares point groups by that name rather than by number for the same reason. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	26fc4b02b3	symmetry: carry the space group as the group, not as its number ... The adopted space group travelled the pipeline as a bare int and was rebuilt downstream with find_spacegroup_by_number, which returns the reference setting. So every setting a number cannot name was destroyed one line after it was determined: P 1 1 2 came back as P 1 2 1, I 1 1 2 as C 1 2 1, R 3:R as R 3:H. DatasetSettings now holds the gemmi::SpaceGroup itself, DiffractionExperiment exposes it as GetGemmiSpaceGroup() / GetSpaceGroupOrP1(), and everything that used to take an int - HKLKeyGenerator (its int constructor is gone, so the compiler finds the callers), the merge, the R-free flags, French-Wilson, the reindexing ambiguity, the completeness enumeration, the MTZ and mmCIF exports, the model validation - takes the group. -S keeps the setting the symbol names rather than reducing it to a number. The end message carries both spellings and a reader prefers the name, since only the name keeps the setting while the number is what a reader written before the name understands. It carries them over CBOR too: the determined group was never serialised at all, so a group rugnux chose reached the master file only when the same process wrote it, and an online writer fell back to whatever the user had supplied at the start. Both keys are optional additions, so an older reader skips them and a newer one reads an older sender. On disk the master's /entry/sample/space_group carries the extended Hermann-Mauguin name and is what the reader takes the group from, so a setting survives a _process.h5 and the --mode scale that re-reads it; the number stays beside it and is the fallback for files written before. Every one of the 230 reference settings the old writer could produce reads back as itself, so older files are unaffected. Stage A and Stage B of the search still enumerate reference settings only, so this determines no group differently today - it is what the enumeration needs before it can be widened. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:16:43 +02:00
leonarski_f	6ed4ea541e	reader: a link to a file that is not there is not a dataset that exists ... Every DECTRIS Eiger master links saturation_value, pixel_mask, bit_depth_readout and serial_number into a companion <prefix>_meta.h5, and that file is routinely not kept when a dataset is archived or deposited. The existence test asked only whether the LINK was written, which it is, so every optional-field guard in the reader answered yes and the read that followed threw. A deposited Eiger 16M set could not be opened at all, over values the reader was perfectly prepared to do without. Exists() now asks the second question too - whether the object the link names can be reached - so an orphaned link reads as absent and the fallbacks behind it do their job. The saturation value is then allowed to be missing outright, because on such a file it is: neither the NXmx name nor the DECTRIS one is readable, and there is no third place to look. Left unset, GetSaturationLimit() falls back to the container's own overflow. That is the safe direction - it can only fail to call a pixel saturated, where too LOW a value drops the whole reflection and silently removes the strongest data - and the run says out loud that nothing will be called saturated. The set that could not be opened now processes to 2.17 A against a deposited 2.40 A, in the deposited space group, with a cell agreeing to 0.08%. Output is byte-identical on datasets that already opened. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 0637979f6d95b446406ab70d1f3982841d195b36)	2026-08-30 21:43:54 +02:00
leonarski_fandClaude Opus 5	fb18457e6f	docs: bring the changelog and the method notes up to rc.166 ... The rc.166 changelog was missing fourteen user-visible changes and carried rationale and measurements that belong here instead. Added: the native miniCBF sweep reader, the third-party and firmware-1.x NXmx masters, the plain LZ4 filter, the image orientation taken from the file's module direction vectors, the two beam-centre flags and their rescues, and the FFT reach past 500 A. Trimmed the rest to one line each, moving the numbers out of the user-facing file. RUGNUX.md described the input as a single Jungfraujoch master file, which it has not been since this branch; it now covers the foreign and legacy masters, the accepted compression filters and the miniCBF sweep, including how a sweep is collected from one named frame. Six options existed with no entry in the table - --beam-center-check, --beam-center-search, --fft-min-unit-cell, --min-indexed-spots, --rot3 and --no-p1-crosscheck - and -C now moves both FFT cell bounds, which was not written down anywhere. CPU_DATA_ANALYSIS.md carried two statements this branch made false: 7.5 still said pass 2 reuses pass 1's space group, and 13.1 still said centrings are ranked by net absence count. Both now describe what the code does - the group is determined after pass 2, and centrings are ranked by the same Beta-tail likelihood the screw test uses. Also documents the per-zone screw scoring, the coplanarity volume-fraction guard, the plane-normal transform, the FFT cell bounds and the twelve refined candidates. The miniCBF reader implements the x-CBF_BYTE_OFFSET scheme and reads the imgCIF axis table from the specification alone. No CBF code is vendored or linked, so there is no licence obligation, but reimplementing a published specification carries one of credit: ACKNOWLEDGEMENT.md gains a section and the two algorithms carry a one-line reference each. Both DOIs were resolved before being written. Rottger's initial was wrong where this branch first cited it - K, not A. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 21:15:31 +02:00
leonarski_fandClaude Opus 5	9924dd9fc3	docs: the open test battery, updated - 82 datasets, and not only non-SLS ... The public data the pipeline is exercised on has grown from 59 datasets to 82, 77 of them with a released PDB entry and released structure factors, so the page that credits the depositors and carries the DOI to cite for each is brought up to date with what is actually run. Renamed from NON_SLS_TEST_DATA to EXTERNAL_TEST_DATA, because the old title stopped being true: a few of the sets were collected at SLS beamlines, where the data are still written by someone else's detector and someone else's acquisition system. What the battery tests is foreign files, not a foreign facility. Also rewritten from the current archives rather than the earlier sample: - Multi-collection archives: eleven are not a single continuous rotation, not four. Seven IRRMC archives hold more than one collection; one sweep is kept in six of them, and both are kept in the one whose two sweeps are at different wavelengths. A repository project page is not a reliable guide here - one describes a 900-frame sweep its own tarball does not contain. - Detector labels: 76 rows can be compared against the PDB entry. Seven genuinely conflict, and one of those the file settles outright - pixel count, pixel size, sensor thickness and firmware string agree with an EIGER2 9M against the entry's PILATUS4 4M. A further 29 differ only in how much they state, which is not a conflict. - One archive ships 30 placeholder files named like images that are 64-byte text; named on the page so a reader that globs the directory is not surprised by them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 20:50:10 +02:00
leonarski_fandClaude Opus 5	03456dcddd	calibration: an option to hold the detector tilt fixed ... --mode calibration fits five parameters - beam centre, distance and the two PONI tilts - and a program that cannot express a tilted detector has nowhere to put the last two. Dropping them after the fact is worse than never fitting them: the centre and the distance of a tilted fit have already absorbed the tilt, so the flattened geometry is right nowhere. rugnux --no-refine-tilt, the "Refine detector tilt" tick box on the viewer's Calib page and RingOptimizer's refine_tilt argument hold rot1/rot2 at the value the geometry came in with and fit the remaining three. That is the best flat-detector answer, and the one such a program would refine to itself. Measured on a five-distance calibrant series. At short distance the tilt is real and reproducible - three independent fits agreeing to 0.01 deg, radial rms 1.4 -> 0.4 px - and its direct beam agrees with the background beam-centre estimator to 0.05 px, so the tilted model is the physically right one. The pinned fit then displaces the centre 2.6 px to absorb the tilt and lands within 0.03 px of the same place at every distance. Past ~300 mm, where only two rings reach the detector, the tilt is instead under-determined: it comes out with the opposite sign to every short-distance fit and drags the PONI 28 px while the rms does not move (0.960 against 0.962). The existing degeneracy guard only fires on a strictly single ring, so it does not catch that; declining a tilt that does not pay for itself in rms is left for a separate change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 20:44:12 +02:00
leonarski_f	d5818713cb	rugnux: also merge in P1, so a wrong space group is recoverable ... A rotation run that determines its own space group now also writes <prefix>_P1.mtz: the same observations, the same scaling, merged in P1. If the group was wrong there is no route back today except processing the images again, and the P1 data settle it - re-merge, re-solve or re-refine in any subgroup. Written whether or not the search adopted P1, because a file whose presence depends on what the pipeline decided cannot be harvested by a script. A run given -S writes nothing: its group's centring absences were never integrated, so the P1 merge would be missing whole classes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f # Conflicts: # docs/CHANGELOG.md # rugnux/rugnux_cli.cpp	2026-08-30 14:15:20 +02:00
leonarski_f	123dc22be2	rugnux: write the unmerged MTZ by default ... A run that does not ask for it still has to hand its data to aimless, pointless or careless eventually, and the file it needs is one a user had to know a flag to get. It is the largest a run produces, but it is 3.4 to 5.8 per cent of the images it replaces - and the better it is, the less anyone needs to keep those. The batteries pass --no-export-unmerged: they process the whole corpus and would write a few gigabytes nobody reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f # Conflicts: # docs/CHANGELOG.md	2026-08-30 14:14:28 +02:00
leonarski_fandClaude Opus 5	f75cf72d1a	beam centre: measure it on every run, and try it when the header fails ... The header beam centre is wrong by more than the geometry absorbs on two thirds of foreign depositions, and nothing measured it. The solvent ring already gives it away: the background projection the beam-stop pre-scan builds is enough to fit the centre, so the measurement is a fit over an array the run has already paid for. It is reported on every run and committed on none. The first pass runs again at the measured centre and the two lattices are compared; the measured centre is taken only where the header indexes nothing, and a disagreement is reported rather than resolved, because the only arbiter available at that stage is the frame count and it is inverted. Recovers two depositions whose header is the geometric detector centre. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:07:13 +02:00
leonarski_fandClaude Opus 5	65dfdb1324	reader: read the detector arm and the mounting each file states ... A detector swung out on a 2theta arm was read as if it stood square on, and a miniCBF written with a vertical spindle or a quarter-turned image was read with the standard mounting assumed. Both are stated in the file and both were ignored: the NXmx depends_on chain was never followed, and Detector_2theta was parsed into a field nothing read. Recovers three datasets that produced no usable lattice, and the small-molecule sweeps at 30 and 55 degrees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:07:04 +02:00
leonarski_fandClaude Opus 5	7528bd8761	docs: write down the three directions this work is heading in ... Three preferences that have been decided in conversation and applied to several changes already, but were nowhere a new reader would find them. They are directional, not absolute. Spending compute to buy quality is now affordable where it once was not, so the question to ask of a slower design is what it buys. Deciding late beats deciding at a threshold, because a gate calibrated on one population refuses another - measured repeatedly in the lattice and symmetry code. And the target is a bare invocation with no flags, because a capability nobody knows to switch on has not solved the problem. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:06:54 +02:00
leonarski_fandClaude Opus 5	a27c4cf26f	reader: take a miniCBF's mounting from the imgCIF axis table its header states ... A miniCBF header states three things about how the instrument is put together that the reader was assuming instead: which laboratory direction the image's columns run along, which its rows run along, and which the spindle turns about. Some beamlines append a CBF template block holding the full imgCIF axis table, which says all three outright. Two instruments in the corpus are not what was assumed, in two different ways. One mounts its detector a quarter turn round, so the image's columns run vertically. Another turns its spindle about the VERTICAL, with the image mounted the usual way; its table says so, and its "# Oscillation_axis" line says so a second way, by naming the image direction the spindle runs along rather than a vector. Either error leaves the spindle 90 degrees from the image. That is not a sign, so the run's axis-sign rescue cannot reach it, and no refinement recovers it: all three affected sweeps indexed nothing usable. So the table is read. The element axes give the image orientation, matched against the eight discrete mountings exactly as the NXmx module directions already are - the match itself moves to DetectorOrientation, so both readers share one definition rather than two copies. The goniometer axis with no parent gives the spindle DIRECTION; its sign stays the rescue's business, which is the part a convention can legitimately differ on. The detector axis with no parent gives the 2theta arm, replacing the assumption that the arm shares the spindle's axis - the one header stating both states them with the same vector, so this changes no answer, only what it rests on. imgCIF's frame differs from the internal one by a half turn about x, a rotation and not a mirror, as writer/HDF5NXmx.cpp already records from the other side. Where a header carries no table, a "+SLOW" on the Oscillation_axis line still says the spindle runs along the image's slow direction. That is the only thing one of the three affected sets says about it. The axis NAME on that line stays unusable - the header that carries both says "X.CW" where its own table says Y - but the direction token is not: where both are present they agree, which is what makes reading it evidence rather than a guess. Also: naming a frame with no directory at all now finds its sweep. parent_path() of a bare filename is empty and iterating an empty path finds nothing, so running from inside the data directory reported that no images were found. Measured, with nothing on the command line. The vertical-spindle protein set goes from no usable lattice to 100% indexed, P 6(3) 2 2 with a cell 0.43% from deposited, 87846 reflections at 86.3% completeness and CC(1/2) 0.995. Its companion from the same detector, which has no table and only the +SLOW token, goes from a spurious monoclinic cell at 2.3% completeness and I/sigma 0.21 to the right orthorhombic lattice, 97.7% indexed, 59.7% complete, CC(1/2) 0.996. The quarter-turned set's three sweeps, at three arm positions, now all index without the hand-passed quarter turn they needed and agree on one cell to 0.03 A. Six miniCBF sets that state no table and no +SLOW - including one whose Oscillation_axis line names an axis in a third dialect - are byte-identical in .hkl, .mtz, .cif and the image statistics, as are two NXmx sets, which is the shared orientation matcher moving nothing on that path either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:03:32 +02:00
leonarski_fandClaude Opus 5	dc71cb7299	rugnux: keep the P1 merge, so a wrong space group is recoverable ... A de-novo rotation run merges in P1 to search for the symmetry, adopts a group, and then overwrites that merge with the in-symmetry one. If the adopted group is wrong the user has no route back: every file the run wrote, and every statistic in them, is computed in the group that was assumed, so nothing in the output says the choice was wrong and the only way to a different answer is to process the images again. Merge the same integration once more in P1 after the run's own files are written, and put it beside them as <prefix>_P1.mtz. From it the space group can be re-determined and the data re-merged, re-solved or re-refined in any subgroup - measured end to end on three crystals: POINTLESS reads the file, recovers the group, and AIMLESS re-merges it, reproducing rugnux's own merged intensities at CC 0.9965 where the two agree. On one of the three it recovered the deposited/XDS group where this run had under-called the screw axis. The merge is the full one - correction surfaces fitted, ice rings and near-tangential observations kept, whole resolution range - not the deliberately degraded merge the space-group search itself runs on, and it is what `rugnux --mode scale -S P1` produces from a _process.h5. That route already existed but needs a _process.h5, which a merging run does not write, so it only helped a user who had foreseen the problem. Every de-novo rotation run writes the file, including one whose search concluded P1 and where it therefore repeats the merged output byte for byte. Whether a file exists must not depend on what the pipeline decided: a script harvesting results would otherwise have to reproduce the search's decision to know whether to expect it, and a missing file would not separate "the run chose P1" from "the run failed". A user-fixed -S writes nothing, and that condition is not a pipeline decision. With a group fixed, prediction rejects that group's centring absences (IndexAndRefine.cpp:499-506), so those reflections are never integrated; a P1 merge built from such a run would be missing whole centring classes and would mislead rather than merely be smaller. Cost, median of five paired runs read off the log timestamps (the box is shared, so end-to-end wall time is noise): +0.32 s of 4.2 s, +2.18 s of 31.4 s, +0.19 s of 17.9 s, +0.66 s of 15.9 s - 1 to 8% of a run. The file is 1.6x the merged MTZ on a monoclinic crystal and 30x on a cubic one, where the merged MTZ is tiny; it is well under the unmerged export in every case measured, and 0.02 to 0.6% of the raw dataset. Rotation only for now: the stills merge re-fits per-image scales and per-reflection partialities onto the integration outcomes, and _unmerged.mtz is written from those afterwards, so on stills this extra merge alters a file that is the run's own output. Fixing that means running the cross-check below the unmerged export, which needs the merge lambda hoisted out of its block; deferred, since rotation is what the online pipeline processes. On rotation the merged .mtz, .cif, .hkl, _image.dat, _unmerged.mtz and _unmerged_partials.mtz are all byte-identical with and without this change, including on a run whose search returns P1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 12:36:49 +02:00
leonarski_fandClaude Opus 5	5c44e544dc	reader: place a detector swung out on a 2theta arm where the file says it stands ... Chemical crystallography reaches high angle by swinging the detector out on a 2theta arm. Both readers had the number and neither used it: the miniCBF header's Detector_2theta was parsed into a struct member nothing ever read, and on the NXmx side the rotation was in the depends_on chain, which was not followed at all. A sweep taken at 30 degrees was therefore processed with its detector plane 30 degrees from where it stood, and nothing indexed. The geometry could already express it, and needed no change: the arm turns the detector about the sample, so the distance is still measured along the detector normal and the beam centre is still the point of normal incidence - which is exactly the PONI convention, and a swung detector is one PONI rotation. What moves is the direct beam, by distance*tan(2theta), off the beam centre and often off the detector. NXmx is the harder half, because the swing has no field of its own: it is one rotation in the chain the detector's position depends on, and "two_theta" is only one beamline's name for that dataset. So the chain is followed and its rotations composed, rather than a field of one name being looked for - each transformation states its vector in the frame of the one it depends on, which is why the product is the whole placement. Translations are skipped; they are the distance and the beam centre, which the file states separately in the square-on frame. Vectors come from McStas through the same 180-degree turn about z the module directions already use, a proper rotation, so an axis carried through it turns the same way. The three rotations a file this system writes ARE that chain, and are also read as the PONI angles - so those three paths are skipped, or every tilted file we have ever written would come back tilted twice. That is the one way this change could have broken existing data, and the test for it writes a tilted file and reads it back. For miniCBF the arm turns about the base spindle axis: on the four-circle geometry those headers describe the two are one axis, and the imgCIF axis table such a header carries states them with the same vector. Both now come from one constant, so a later correction to the frame moves them together. Measured. On a swung NXmx sweep the chain gives rot2 = -0.34907 rad for the 20 degrees it states, and the sweep goes from "nothing was integrated" to 25000 reflections at 82.2% completeness and CC(1/2) 0.9993, in the same space group and the same cell to 0.03 A as the square-on sweep of that crystal; the opposite sign indexes nothing. A miniCBF sweep at 30 degrees goes the same way, to 0.585 A, and a second sweep of that crystal at 55 degrees reaches 0.476 A and reproduces the cell again - with a low-resolution limit of 2.36 A rather than 13 A, which is what a detector swung that far records. On all of them post-refinement recovers the header's own beam centre and distance, and the beam stop shadow sits within four pixels of where the swung geometry puts the direct beam, 417 and 537 pixels from where the unswung one does. Seven sets whose detector is square to the beam, three of them carrying a chain whose 2theta is zero, are byte-identical in .hkl, .mtz, .cif and the image statistics. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 12:36:46 +02:00
leonarski_fandClaude Opus 5	5f47d73cc1	rugnux: write the unmerged MTZ by default ... The unmerged export is how a run's observations reach the rest of the field. It is what aimless, pointless and careless read, it is what a head-to-head against another program's answer runs through, and handing several of these files to pointless is the only way to merge sweeps rugnux does not combine itself. Behind a flag it reached only the people who already knew the flag existed, which is the shape of defect the "rugnux <file> with nothing else" direction asks to design out. --no-export-unmerged turns it off, and the regression batteries now pass it: the objection to defaulting it on was their disk and time cost, not the product's. Measured on three rotation crystals of 900, 1800 and 3600 frames: the file is 21, 86 and 26 MB - two to five times the merged .mtz, .cif and .hkl put together - and the write costs 0.24, 2.29 and 1.11 s, 5.5 %, 12.2 % and 3.1 % of each run's own wall time. The cost is linear in the number of observations and independent of how long processing took, so it is the largest fraction of the runs rugnux finishes fastest, not of the longest sweeps. A run's .hkl, .mtz and .cif are byte-identical with the export on and off. --export-unmerged-partials stays off: it is a second, larger artefact and a separate question. The viewer's reprocessing dialog ties the export to its merged-output switch, so it appears beside the merged files rather than beside a job that only asked for the per-image _process.h5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 10:51:25 +02:00
leonarski_fandClaude Opus 5	bf41b1fab1	rugnux: index both beam centres always, without a significance gate first ... The second first pass was skipped where the measured centre sat within three times the fit's own sigma of the file's - a quarter of the non-SLS corpus, eight of thirty-two runs. That is a decision taken at a threshold before the evidence is in, and the evidence here costs a median 0.55 s: the spots are already found and cached, and their positions do not depend on the centre, so a second first pass is a median 20 % of what the first one costs. Worse, the gate removed exactly the case worth asking about. A centre error ALONG the spindle does not fail - it holds 96-100 % of frames indexed and quietly returns an axis harmonic - and the scale it turns on is a fraction of a pixel: measured on real data, 0.12 px of centre is the whole difference between the deposited cell and a halved axis. No sigma small enough to gate on would notice that, so the gate could only ever hide the question, never answer it. The three-sigma test stays where it belongs, in the report: it says whether the difference is a measurement or noise, which is what a reader wants to know. It no longer decides whether to look. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 10:28:35 +02:00
leonarski_fandClaude Opus 5	d8816169e1	rugnux: ask the rotation-axis sign at the measured centre, and say when only the symmetry differs ... Two corrections to the check, both from measuring it on the non-SLS corpus. Where the file's centre indexes nothing and the measured one does not either, the rotation-axis sign is now asked again AT the measured centre. The two unknowns are discrete and coupled: on the two public depositions in the corpus whose header beam centre is the geometric centre of the detector, the file is 73 px out AND its axis sign is the opposite of the one that indexes. The sign rescue therefore asks its question at a centre 73 px wrong (0/60) and puts the sign back, and the centre is then asked at the wrong sign (0/60). Each error hides the other and the run produces nothing. Asking the pair takes both runs to 100 % indexed on the deposited lattice - 199.40 67.11 against a deposited 199.54 67.15, and 208.77 67.20 against 208.77 67.22 - at the cost of one more first pass on a run that has already failed twice. Asked here rather than by moving the check ahead of the sign rescue, which was tried: that also works, but it moves the beam centre of three datasets whose only fault is the axis sign, for no benefit. Where nothing works the file's centre is put back, so a run that fails for another reason fails at the geometry it was given - 6yqf, whose spindle is along the detector's slow axis, and 7atg both come out exactly as they did before. And the beam centre decides the primitive VOLUME, not the Bravais class: a centre error along the spindle makes the FFT take an axis harmonic, which changes the volume by an integer factor or by sqrt(3), while a class differs for a reason that has nothing to do with the centre. The two are now reported separately. Both "disagreements" in the corpus are of the second kind - primitive volumes agreeing to 0.16 % and 0.57 % while one pass reads R-centred trigonal or P orthorhombic and the other stays triclinic - and in both the file's centre is the one that finds the symmetry and the run's cell matches the deposited one to 0.23 % or better. The cell was never in question, and the strong warning now fires zero times in 32 datasets. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 09:59:41 +02:00
leonarski_fandClaude Opus 5	60f6a46b25	rugnux: index the measured beam centre too, and say when the two answers differ ... The estimate the previous commit makes is free, and a first pass re-uses spots it has already found, so running the first pass a second time at the measured centre costs about what one rung of --beam-center-search costs. That buys the one comparison an on-failure trigger structurally cannot make. A centre wrong ACROSS the spindle announces itself - the indexed fraction collapses - but a centre wrong ALONG it does not: the run holds 96-100 % of frames indexed and quietly returns a 2x, 3x or sqrt(3) axis harmonic. Nothing fails, so nothing fires. Indexing both centres and comparing the two lattices is what can see that at all. What must not arbitrate the two is the indexed frame count. Acceptance is a fractional-Miller test, so a cell twice as long has to place every spot twice as accurately to score the same; measured on real data, two centres 0.12 px apart gave the deposited cell at 99.23 % and a halved axis at 100.00 %, and the wrong answer indexed better. A rule of the form "take the centre that indexes more" picks wrong in exactly the case the comparison exists for. So only what needs no arbiter is decided. The file's centre indexing nothing where the measured one indexes a majority is not a comparison, it is a run that produced nothing and now does: take the measured centre. The two agreeing on the lattice is reported and nothing else - it is a free statement that the header is good enough for this crystal, which is most runs and is worth saying. A disagreement between two passes that both worked is reported with both cells, their primitive volumes and the ratio, flagged when that ratio is an axis harmonic, and left undecided. Runs before the blind ladder, so a measured hypothesis is tried before any grid, and only where the move exceeds three times the fit's own sigma - below that the two centres are one measurement twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 09:00:55 +02:00
leonarski_fandClaude Opus 5	0644404286	rugnux: say where the scattered background puts the beam, on every run ... A beam centre in a file is the metadata field everyone jokes about and nobody measures. rugnux already has an estimator that can measure it - the isotropy of the scattered background, whose leverage is the curvature of the water ring - and it already builds, on every run, the projection that estimator needs: --detect-beam-stop makes one to find the shadow. So the measurement is a fit over a mean image that already exists, no frames of its own, and it can simply be done always. --beam-center-check, on by default, does that and reports it: what the file claims, what the background says, how far apart the two are, how well the fit knows its own answer, and how right this particular geometry needs the centre to be. That last term is what makes the line a verdict rather than a pixel count - 3 px is nothing at 100 mm and a lost lattice at 500 mm. On the two public depositions in the test corpus whose header is the geometric centre of the detector, the line reads 73.49 px and 70.96 px against a 1.57 px and 1.82 px tolerance, on runs that until now said nothing at all about it. Nothing is committed. The run keeps the centre it was given, and where the fit cannot place the centre - a flat background has nothing to separate a radial shift from an amplitude - it says so rather than guessing. Measured on a 0.3 Mpx detector the fit takes 8 ms. The fall-through in --estimate-beam-center now reads this same result instead of repeating the fit; it is the same call on the same projection with the same mask, so that flag's answer is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 08:53:05 +02:00
leonarski_fandClaude Opus 5	a2f451cf12	rugnux: try the beam centre as an indexing hypothesis after a failed first pass ... A beam-centre error is not repairable downstream. It is fixed in the lab frame, so accumulating a sweep smears every reciprocal-lattice point around a circle and the FFT amplitude at an axis of length a is multiplied by J0(2 pi delta p a/(D lambda)); past the first zero the true axis is gone and its harmonic wins. But it is decidable from the data, on exactly the count the scheme choice and the axis-sign rescue already use: the right centre indexes and the wrong one does not. So after a first pass that indexes fewer than half the validation frames, step the centre a pixel at a time and keep the first rung that clears the same majority the other guards test. It runs only after a pass that has already failed, so a run that works never pays for it, and spot finding is not repeated - the cache is keyed by image and the spot positions do not depend on the centre. On the prototype this rescued 2 of 12 failing non-SLS datasets and declined cleanly on the other 10. BOTH detector directions are searched, and that is the part to keep. Measured by injection on two crystals: across the spindle the cell stays right and the indexed fraction collapses, 99 % to 25 %, so that direction announces itself; along the spindle - the one the J0 derivation calls free, because the FFT amplitude is translation-invariant - the run holds 96-100 % indexed and quietly adopts a 2x, 3x or sqrt(3) supercell. Searching only the perpendicular direction would search the failure that already shows. The step is one pixel, flat. Deriving it from the J0 law was tried and is wrong: the only cell available at that point is the one the FAILED pass returned, and on a dataset whose failed cell was a small spurious sub-cell the formula asked for a 6 px step, which steps over the lobe it is looking for. Taking any improvement rather than requiring a majority was also tried and is wrong: where no centre works, the ladder wandered to the far end of its range on 10/60 against 5/60. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 08:37:55 +02:00
leonarski_fandClaude Opus 5	4d2bb98a13	rugnux: name the beam centre when two candidate cells are axis harmonics ... The first pass already has the evidence and says nothing about what it means. When two schemes come back with primitive volumes related by a small integer - or by sqrt(3), which is the hexagonal harmonic and is not a whole number - one of them is the other's axis harmonic, and what decides between them is the beam centre to a fraction of a pixel. The mechanism is that the J0 law is about the FFT AMPLITUDE, which is translation-invariant. A centre error along the spindle translates the derotated cloud rigidly, so the transform cannot see it, the peaks stay sharp, and the lattice fit that follows - whose origin is the beam, and which is not translation-invariant - commits with confidence to a sub-multiple. Measured on a deposited dataset: 0.12 px of centre, 0.09 px of it across the spindle against a 2.78 px need, is the whole difference between the deposited cell and a halved axis, and the halved one is the answer that indexed MORE frames - 100.00 % against 99.23 %. So the message is deliberately not conditioned on any indexing-quality signal, and it is taken over the whole pair rather than only over the tie: on this failure the frame count points the wrong way, which is exactly why it cannot arbitrate. Nothing is decided differently; the integer-subcell tie-break below is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 08:37:55 +02:00
leonarski_fandClaude Opus 5	52e9e9da2b	beam centre: gate on what the geometry needs, and on the move being significant ... The accept bound was one flat 1.0 px constant applied at three points. It was calibrated on a 244 mm / 0.95 A / 0.15 mm geometry, and the displacement the J0 smearing law actually allows runs from 0.6 px to 9 px over the in-house and non-SLS corpora - a fifteen-fold spread - so on a loose geometry the constant is nine times too tight and throws away answers that are perfectly usable. The bound is now the LARGER of the constant and what the geometry asks, which is one-sided by construction: nothing today's gate accepts can be lost. What the geometry asks is printed and never tested against. It says how wrong the header may be; the estimator's sigma says how well the estimator knows its own answer, and gating one on the other rejects a centre correct to 0.03 px because its sigma was 0.98. It is not a relevance floor either - measured on real data, a 0.12 px change of centre, 0.03x of what the law asks across the spindle, is the difference between the deposited cell and a halved axis - so the printed line says so. Added alongside: the move has to be worth making. Under three times the estimator's own sigma it is not a measurement of anything, and a centre that is not moved cannot push the two-pass loop off its fixed point. On the 39 rotation regression crystals this adopts 38 and keeps the header on one. Two things that were happening silently now say so: the move resolved across the spindle, which is the only component the law is about, and the fall-through itself. Below about 220 deg of sweep the spot symmetry never clears the bound and the background answers instead - correctly, to within 0.23 px of the full-sweep centre at every span from 120 to 220 deg - and until now the log did not say the spot arm had even been tried. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 08:37:55 +02:00
leonarski_fandClaude Opus 5	d1818b195e	rugnux: say what the file's beam centre looks like, and when no beam stop is found ... Four things a run can say about the beam centre before it reads a single frame. All of them are free, so none of them is behind a flag. Two are worth acting on. A centre that lands on a masked pixel is wrong about something - a real beam does not sit on a dead pixel or in a module gap - and over the 29 non-SLS datasets whose outcome is on record it fires on 4, of which 3 went wrong, against 2 false alarms in 42 in-house masters. And a beam-stop pre-scan that finds exactly zero pixels is not a beamline without a beam stop: the flood starts from seeds within 4 px of the ASSUMED centre, so a centre far enough out puts every seed in a module gap. That one fires on 3 of 29 foreign datasets, all 3 of which had a bad outcome, with no false alarm in 39 in-house sweeps. The other two are provenance and are said rather than warned about: a centre equal to the geometric detector centre, or a whole number of pixels in both coordinates, was typed rather than measured. They are true of 29 of 42 in-house masters and those are out by a median 3.5 px, but as predictors of harm they sit at or below the base rate, so they gate nothing. They earn their line because the spot-symmetry search reaches only about +-55 px: a placeholder tens of pixels out is unreachable by construction, all five of its searches agree, and the sigma comes back small. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 08:37:55 +02:00
leonarski_fandClaude Opus 5	97f57a430d	beam centre: run the four consistency restarts at once ... FindBeamCenterFromSpotSymmetry calls Estimate() five times - once for the answer and once from each of four starts 25 px away - and three quarters of each of those is an 861-point brute-force grid over the spindle. So making the uncertainty gate live was paid for by multiplying the estimator by five, which is the whole of the pre-scan's cost. The four restarts share nothing: each takes its own copy of the geometry and only reads the spots. What is wanted from them is a max, which is order-independent, so running them concurrently gives the same number. Measured on three rotation crystals, the committed centre, the reported sigma and the fitted spindle angles are unchanged to every printed digit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 08:37:55 +02:00
leonarski_fandClaude Opus 5	3ddfbb2da9	tests: give both beam-centre estimators a case with the detector tilted ... No test anywhere set a detector tilt, so PoniRot1/2 were zero in every one of them and the PONI and the direct beam sat on top of each other. That matters because the conversion between the two is used three times in the spot estimator - to centre the vote, to start each tooth's refinement, and to turn the answer back out of the spindle frame - and with the two centres coincident it is the identity, so its sign was unobservable. Verified by flipping it: with DirectBeamOffset negated, the ten pre-existing beam-centre cases all still pass and only the new one fails. The tilt used here puts the direct beam about 12 px from the PONI, twenty-four times the tolerance asserted, and the case also pins that the tilt is not read as a spindle azimuth - what the fit sees of the detector belongs to the detector. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 08:37:55 +02:00