1245 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	6081b6bc43	docs: credit the L test, FFT indexing, TORO, Niggli, peakfinder8 and SparseCCL ... Six methods the pages name or describe carried no citation: Padilla & Yeates (the L test), Steller, Bolotovsky & Rossmann (the projection/FFT autoindexing MOSFLM implements), TORO (what ffbidx implements), Krivy & Gruber and the ITA lattice-character table (the reduction and Bravais assignment), Cheetah's peakfinder8 (the per-ring background statistics of the adaptive finder) and Hennequin et al.'s SparseCCL (already credited to traccc, now also to its authors). Each gets its ACKNOWLEDGEMENT.md paragraph, a References entry in CPU_DATA_ANALYSIS.md, and a one-line credit at the algorithm. The Sheriff & Hendrickson / Popov & Bourenkov entry is re-scoped so each claim sits on the paper that supports it - P&B 2003 is titled, and credited for the sigma-aware anisotropy estimation its statistic modelling contains, not for the tensor and its constraints. All DOIs verified against the publishers; the SparseCCL DOI resolves to IEEE document 9049184 (IEEE blocks content scraping, so verified by the resolved document id plus two independent sources). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	86f8edfabc	docs: the FFT axis ceiling, one-sweep inputs, exit status and _anom.ccp4 ... The longest FFT axis (500 A, no flag; -C moves it) was implied twice and never stated, and its failure mode is a plausible sub-cell rather than a refusal. One input is one sweep - said affirmatively where inputs are described instead of in an aside about pointless. A default 50 A low-resolution cut discards real reflections on a very large cell; the option row says so. Exit status is documented for scripts (0 = completed, non-zero = stopped), and the --model output list gains _anom.ccp4, which was written but undocumented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	87b1402c50	docs: hand the output onward - downstream commands and the three blocking facts ... The page named phenix, REFMAC, POINTLESS, AIMLESS, careless and SHELXC a dozen times without one command line, and left unstated the three facts those programs stop on: the free-flag convention (0=work 1=free; REFMAC needs FREE 1), the phenix label choice the double intensity array forces, and the unmerged file's header symmetry and sort order (determined group, sorted H K L M/ISYM BATCH - WriteReflections.cpp sorts it). A new 'Taking the data onward' section carries the worked lines, the careless column renames (BGVAR is a variance), and the Phaser SGALTERNATIVE keywords for the enantiomorph the report leaves open. The POINTLESS series trap now names ALLOW OUTOFSEQUENCEFILES instead of telling users to touch their data, the report-grep block warns that SPACE_GROUP_NAME carries one member of an enantiomorphic pair by convention, and the P1 cross-check's free set is declared to be its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	b1ae1ece67	docs: a default rotation run already writes real anomalous data ... The -A row and FRIEDELS_LAW=TRUE together read as 'without -A your Friedel pairs were averaged', which is false: the rotation merge always keeps the Bijvoet split and the default .mtz/.hkl carry it (WriteReflections.cpp, BuildMergedRows). Say what -A actually changes - the counting basis and the error model - and give the merged MTZ's exact column labels, which scripting against phenix needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	40cb5cd6db	docs: the quick start tells the truth about inputs, outputs and the cutoff ... A default rotation run writes seven files, not five - the two the list omitted are most of the bytes. The input is any NXmx/EIGER master or miniCBF sweep, which the page said only 350 lines later after twice implying Jungfraujoch data only. The CC1/2-0.30 resolution trim moves up to the quick start, and _image.dat's columns are finally named (ScalingResult.cpp writes a # header). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	40403d7beb	docs: the direction grid bounds the longest findable axis; say so in 5.3 ... The ranking-not-sampling observation was measured on the coplanar-shortlist rescue and is scoped to it now. The angular-resolution bound theta < d_min/(2a) means the shipped 16384-direction grid resolves axes only to roughly 120-150 A, far below the 1200 A the accepted maximum admits, and the failure mode is a plausible sub-cell, not a refusal - the reader raising fft_max_unit_cell alone deserved to know it cannot work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	32adb882eb	docs: disambiguate the reused symbols and the bandwidth definition ... sigma_bw is one physical smear written in reciprocal units in 8.2/11.1 and in pixels in 9; the two Delta-phi of the partiality formula are named; 13.5's |s| = 1/d is reconciled with the s = sin(theta)/lambda of 10.6/14.2; bandwidth is the rms spread, with the FWHM-input conversion (/2.355, BraggIntegrationEngine.cpp) stated. The ice-extinction clause now covers (104), not only (00l), and the AIMLESS <I/sigma> = 2 constant is named as AIMLESS's default rather than pointed at a criterion this project does not use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	e94548c614	docs: the twinning exemption list, and why -1, 2/m and mmm are not on it ... The stated criterion (a merohedral twin law exists) formally exempts every holohedral Laue class, but the code (TwinningAnalysis.cpp) deliberately keeps the low-symmetry ones eligible because pseudo-merohedral twinning through a special metric cannot be excluded there. Say both halves, and give the reference values of the statistics so the mmCIF numbers can be read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	9814846ebd	docs: state the q convention at every numeric q and on the q flags ... Every q in the pipeline is 2*pi/d (Definitions.h, the azint bin mapping), but the numbers in 3.3, 7.6 and 10.10 and the --azim-* flags never said so, and a reader taking q = 1/d would set --azim-q-spacing or --azim-max-q wrong by 2*pi. The 7.6 ice triplet is spelled out so 'within 0.06 of one another' reads as the adjacent-ring spacing it is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	14b79b0314	docs: fix the sigma-ratio, the profile-fit term and the outlier cut as written ... The 1.109 in 9.2 is the variance ratio of the shipped 4/6/13 stencil (45/408 pixels), not the sigma ratio the sentence attached it to, and the effect is a bound attained on weak reflections, not uniform. The profile-fit background term is (sum P/v / sum P^2/v)^2 var(b), matching the code; the undefined w is gone. The 13.3 refit cut is N^2 times the model variance (RotationScaleMerge.cpp), not N*sigma^2. The sigma floor is 1 count. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	f68fd3d7af	docs: pin the sign convention of the back-rotation in 7.3 ... R(phi) rotates the observations by +phi about the stored axis, which makes it the inverse of the crystal's own rotation - the reading under which the formula as written is correct, and the same convention that has the exported MTZ batch axis and the XDS echo negated. Said explicitly, so a reimplementation cannot take R(phi) for the crystal rotation and land at 2*phi. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	694e687fce	docs: name the beam parameter as the PONI and state the laboratory frame ... Section 1.1 called (x_beam, y_beam) the direct-beam position while constructing it as the point of normal incidence; on a tilted detector the two differ by D*tan(tilt), and a reader loading a header direct beam into it would be wrong by several pixels. Name it as the PONI (the system-wide convention, DETECTOR_GEOMETRY.md), say which point the construction pins, and state the laboratory frame's axes and handedness, which the rotation-sign, R-centring and Bijvoet-hand discussions all silently depended on. Two-theta is computed as a two-argument arctangent; say so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	d3879d0112	docs: the change of hand is a label, only the ambiguity choice reindexes ... Both pages said in one place that adopting a model's enantiomorph moves no reflection, and in another that it is applied to the merged reflections and exchanges the Bijvoet mates. The code (ModelValidation.cpp, AdoptModelFrame) does the former: the model's group is a label on the written files, I(+)/I(-) stay as measured, and only the merohedral-ambiguity reindex touches reflection indices. Say that once, consistently, in CPU_DATA_ANALYSIS.md §14.5 and in the two RUGNUX.md passages that carried the wrong reading. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:36 +02:00
leonarski_fandClaude Opus 5	200a2d53b7	report: name the groups the data could not separate, as keys ... SPACE_GROUP_NAME is a scalar and reads like a determination, and the docs nominate a grep of it as the interface. Where several groups predict the same absences it is one of them, picked by convention: the report said so in prose and nothing machine-readable carried it, so a script recorded a coin flip as an answer. Refining against a deposited model in the wrong enantiomorph gives R = 0.549. Four keys, none of which change which group is adopted: the alternatives the data cannot separate; whether the enantiomorph was determined, given, or is undetermined - decidable from the group number alone, so it is answered even where the search did not run; and the higher point group whose promotion was refused, with its reason, which until now existed only as prose. REPORT_VERSION is 5. No existing key changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:22 +02:00
leonarski_fandClaude Opus 5	247130a841	-S: check the metric the fixed group needs, not only its centring ... Both guards on a user-fixed space group keyed on the centring differing, so an axis permutation within the same centring passed: a run that indexed a=37.909 b=78.031 c=77.594 and was told -S P41212 - which needs a=b with the 4-fold along c - merged through operators that do not act on its own indices, and --mode scale on it reported COMPLETENESS= 195.3, an arithmetically impossible number, without complaint. MetricViolation asks the setting-independent question instead: a group's rotations must leave the cell's metric tensor invariant. The re-seating arm now runs on either failure, so a permuted cell can be reindexed into the setting the group needs rather than merely refused; the refusal arm catches what re-seating could not fix. --mode scale never reaches either arm, so it gets the same test where it fixes its cell and its group, which is the only place an impossible completeness could still be produced. The tolerance is a refusal bound, so it sits above what a correct answer reaches: over 113 corpus runs every group determined from its own cell scores under 0.032 and the permuted case scores 0.764. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:22 +02:00
leonarski_fandClaude Opus 5	47ec939c39	merged MTZ: FreeR_flag in the convention its own column name carries ... The column was written with 1 for the test set and 0 for the working set - the phenix/CNS numbering - under FreeR_flag, which is CCP4's column name. CCP4's own freerflag writes 0..19 with 0 as the test bin, and REFMAC5's default FREE 0 reads it that way, so REFMAC5 stopped on every file we wrote: "more than half of reflections are in free R set", then "Cannot switch free R flag", exit 1. phenix auto-detects either numbering with equal confidence (measured on both, score 3 each), and rugnux's own reference-MTZ reader already takes flag 0 as the test set, so 0 = test is the numbering that works everywhere and the one the rest of the code assumes. This changes every merged .mtz we write: a script that reads FreeR_flag == 1 as the test set has to be inverted. The mmCIF's _refln.status_free is a separate item with its own convention and is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:22 +02:00
leonarski_fandClaude Opus 5	762c093b10	unmerged MTZ: export the events the merge would keep, not every event ... The export declared its rows FULL - LDTYPE=2 in the batch header, M=0 in M/ISYM - while filtering them on --min-partiality alone (0.02), where the 3D combine also applies --min-captured-fraction (0.7 on rotation) to the same summed event. AIMLESS reads the FULL declaration, reports "all runs have only fulls" and ignores FRACTIONCALC, so an event that caught a twentieth of its rocking curve entered the merge whole, with a small sigma, and was weighted heavily. There was no cut for the reader to make: the column that would let it make one is the one the reading program has been told to ignore. Dropping those events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from 0.985 to 0.993. FRACTIONCALC itself is unchanged, values above 1 included. Each part's partiality is the erf pair the predictor computed on that frame, from that frame's own refined lattice and mosaicity, so the parts of one event do not tile the rocking curve exactly - the offset steps by the wedge to within 12% of it, and the sums land in a peak at 1.000 whose 95th percentile is 1.09. It is an honest estimate of a captured fraction, and it is not the number rugnux scales on, so clamping it would only hide the spread. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:22 +02:00
leonarski_fandClaude Opus 5	b33cae9068	unmerged MTZ: a batch header for every image the sweep spans ... The header set was built from the batches that produced an observation, so an image that indexed nothing left a hole in the phi series. AIMLESS starts a new run at such a discontinuity: measured on a 360 degree sweep with 67 unindexed images it made 23 runs of one sweep, its scale model diverged, Rmeas overflowed to -1266 and the result no longer correlated with rugnux's own merge (Pearson 0.0018). Renumbering the batches contiguously does not help - the split is on phi, not on numbering - so the fix is a header per image over the span the observations cover. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:22 +02:00
leonarski_fandClaude Opus 5	fb07263025	predict: integrate every centring node, so a fixed space group costs no data ... With -S the prediction rejected the fixed group's centring absences, so those reflections were never integrated. Two things followed, and only the second was known. The P1 cross-check was withheld on such a run, because a P1 merge missing whole centring classes is misleading rather than merely small - 50% of the nodes on an I lattice, 75% on F, 67% on R. That was the documented reason and it was right. The unknown one is that it cost intensity accuracy. Every predicted reflection marks its signal region so a neighbour's background ring can exclude it (BraggIntegrationEngineCPU, the reflection mask); an unpredicted node is an unclaimed patch of detector, and the neighbouring reflections sweep those pixels into their background and over-subtract - worst at high angle, where the background dominates. On a fixed F-centred group that is three quarters of the nodes: measured against the de-novo run of the same data, <I/sigma> 16.07 against 17.21, CC1/2 0.9862 against 0.9895, ISa 10.93 against 11.92. Predicting them costs nothing downstream, because both merges already decide absence against the group they are merging in: the run's own merge drops them again, and the P1 cross-check keeps them because P1 has none. One integration, two correct merges. The -S output becomes byte-identical to the de-novo run on the five crystals measured, which is the point - pinning a group should not change the answer - and such a run can never be slower than de novo, since it predicts the same reflections and additionally skips the space-group search. The de-novo path is untouched by construction: it already predicted in P. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-02 09:18:13 +02:00
leonarski_fandClaude Opus 5	ffce27ac33	rugnux: the two-pass supercell guard compares primitive cells, not settings ... The guard that catches the bistable supercell collapse between the two rotation passes took the volume of each pass's CONVENTIONAL cell. A centred conventional cell is an exact integer multiple of its primitive one - C and I twice, R three times, F four times - so two settings of the same lattice differ by exactly that factor, and the guard read a change of setting as a supercell. It then forced pass 1's result, and with it pass 1's lower symmetry, on a pass that had found the same lattice in a better one. Ten of 116 rotation datasets tripped it, every one at an exact centring multiplicity: seven at 2.00x, one at 3.06x, one at 3.99x. The last is an F-centred lattice; it was held in P1 where the second pass had found it centred orthorhombic. The two structurally identical guards inside RunPipeline already convert with ToPrimitive first, and their comments say why. This one could not: ProcessResult carried the consensus cell with no centring beside it, so at the comparison there was nothing to convert with. The centring is now carried alongside the cell, set at each of the four places the cell is - the finalized rotation lattice, the reindex into a user-fixed group's setting, the reduction of a doubled cell, and the committed higher-symmetry reindex - and the guard converts both sides before comparing. Nothing else reads it; every other consumer of the cell is unchanged. On the F-centred dataset the run goes from P1 at 2.232 A to I 2 2 2 at 2.077 A, multiplicity 1.86 to 7.07. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-01 23:20:32 +02:00
leonarski_fandClaude Opus 5	ecb0571d8d	integration: the observed centroid is the signal's, not the disk's ... The centroid was a first moment of the RAW counts over the signal disk, so it weighted signal plus background. The background is flat over a disk centred on the PREDICTION, which makes its own centroid the prediction exactly: it adds nothing to the displacement and everything to the denominator, and the measured offset comes out shrunk by I/(I + n*bkg). That factor is worst where the background dominates the signal, which is at high resolution - so the one consumer of this quantity, the geometry post-refinement, fits the beam centre and the detector distance on displacements that are systematically too small, by a factor that varies with resolution. An estimator whose bias depends on the very coordinate it is correcting. Subtracting a flat pedestal from a first moment is exact, and the background is not known until the ring has been read, so the positions of the pixels behind the intensity sum are accumulated alongside it and the correction is applied afterwards: sum(x*(px-bkg)) = sum(x*px) - bkg*sum(x). Both engines, identically. Where nothing rises above background there is no signal centroid to compute and the raw one is kept. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 19:55:47 +02:00
leonarski_fandClaude Opus 5	4026ebc6ab	writer: the direct beam is absent where there is no geometry, not fatal ... Computing it needs a DiffractionGeometry, and that refuses a detector distance under 1 mm or a pixel size of zero. A start message can legitimately carry neither: the writer's own pre-flight check asks it to prove it can create the files for a dataset that never describes a detector, and the master write then threw where it used to succeed - Preflight_TCP fails at rc.166 and passes at rc.165. The dataset is a convenience for whoever reads the file later, so where the geometry is not there to compute it from it is simply not written. Everything else in the master is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 18:58:38 +02:00
leonarski_fandClaude Opus 5	8650ac1c59	rugnux: carry the tilt across the two-pass handoff, and post-refine about it ... Post-refinement was handed the HEADER geometry while the outcomes it refines had been integrated at the first pass's own, and only three numbers came back - beam_x, beam_y, distance. The tilt was dropped at both ends, so the pass that ships started at the header tilt with a beam that step B had moved to absorb a tilt the images do not have. On one in-house mounting the first pass ends at rot1 -0.2995 deg and the shipped value is -0.0992, against a powder calibration of -0.282: the two geometries the two passes use differ by the whole of the tilt error, inside a single run. Both ends move together, and they have to. Step B holds the tilt fixed while it fits the beam and the distance, so the beam it returns is only meaningful about the tilt it was given: handing it the refined tilt without carrying that tilt forward, or carrying the tilt forward under a beam fitted about the header's, each describe a geometry that never existed. The carrier goes from three floats to five and the header snapshot the quality guard reverts to grows with it, so a rejected second pass still returns to the geometry the file states. This is not a tilt measurement and does not make one: rot1 remains the gauge direction of a single-axis rotation experiment. It only stops the two passes of one run from working in two different geometries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 18:49:33 +02:00
leonarski_fandClaude Opus 5	9c19646f6a	refine: fit the direction of the goniometer axis, not its length ... The residual applies angle_rad * |rot_vec| and rot_vec is a free three-vector, so the first pass has been fitting a goniometer rotation SCALE nobody asked for. GoniometerAxis::Axis() then normalises it away on write-back, and RotationIndexer scores the candidate with the normalised axis - so the cell that won the fit is judged under a rotation model the fit did not use. Measured over 43 rotation datasets: the length reaches 1.2%, and the fit-vs-score disagreement a median 0.124 deg and up to 6.19 deg of goniometer angle, against rocking widths of 0.05-0.36 deg. That score picks the lattice class, which nothing later revisits. The fitted length is not a usable measurement of anything either: on synthetic data it recovers 54% of a known scale error, repeated first passes on one dataset disagree with each other in sign, 26 of 43 datasets disagree with themselves, and on the one dataset with a proven 1.3% stage fault it comes out negative. It is absorbing other systematics. The rotation scale is measured properly, once, with cross-validation and gates, in PostRefine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 18:36:32 +02:00
leonarski_f	d32c8d526b	geometry: make which tilt component is restrained two numbers rather than a shape ... Both directions of the tilt now go through the same restraint, with a budget each in shared direct-beam pixels; zero means free. Swapping which component is held, or holding both, is a change to two constants. The perpendicular budget is zero today, which is the arrangement the measurements support: it is the component whose conditioning tracks the 2theta the fit reaches, while the parallel one's does not move with 2theta at all - a conditioning number that ignores the data is the prior talking. This is a clarity change, not a precision one, and the record should not read otherwise: the standing 'freeing the tilt inflates both beam components' cost is the perpendicular component's doing, not the parallel one's, so restraining the parallel one recovers essentially none of it. What it buys is that a beam-centre error is no longer reported as an angle.	2026-09-01 15:34:06 +02:00
leonarski_f	657b6c5d1a	geometry: say that the gauge argument is local to the fit that co-refines the orientation ... The restraint is correct here because the crystal orientation is free alongside the tilt and absorbs the spindle-parallel difference. A stage that freezes the orientation has no such compensator, so whether that component is measurable THERE is a separate question; the comment no longer reads as a claim about the tilt everywhere.	2026-09-01 15:33:51 +02:00
leonarski_f	c0bb159c98	geometry: take the gauge direction in the frame each parameter lives in, and tighten the tilt's budget ... Two corrections to the restraint added in the previous commit, both measured. The beam prior compared the goniometer vector's LABORATORY components against an index into the PIXEL-frame beam centre. det_matrix is PoniRotMatrix * DetectorOrientation::Matrix(), so on a quarter turn of 1 or 3 the pixel X axis is the laboratory Y axis and the comparison picks the wrong component - pinning the determined one and freeing the gauge one, which is worse than having no prior. Two datasets in the corpus are in that state. The direction now comes from projecting the spindle onto the pixel axes' own laboratory images, which is exact for any orientation, any tilt and a spindle at any angle, and equal to the old comparison when the orientation is the identity. The tilt keeps the spindle's laboratory components, because rot1 and rot2 are laboratory rotations applied outside that orientation matrix - same physical direction, each in the frame its own parameters live in. And the tilt's restraint is three times tighter than the beam's, in shared direct-beam pixels, because the data determine the SUM of the two: at equal budgets the shift splits evenly and the reported tilt still followed the starting beam centre at 41% of one-for-one. At a one-pixel budget it follows at 8%, from 79% before this work, while the free component moves by 0.0035 deg over the same eight-pixel swing and the indexing rate does not change. The comment says plainly what the change does not do: it does not make the tilt accurate.	2026-09-01 15:33:51 +02:00
leonarski_f	e441434644	geometry: restrain the spindle-parallel tilt, the same gauge the beam prior already restrains ... The beam prior at XtalOptimizer already treats the beam component parallel to the spindle as the gauge direction of a single-axis rotation experiment and restrains it toward the value it was handed. The detector tilt is that same gauge described a second time, and it was left free with a flat +-3 deg box. The correspondence is not an observation about one beamline, it follows from the convention: PoniRotMatrix builds the detector matrix as R(-rot3,z)R(-rot2,x)R(rot1,y), which puts the direct beam at (beam_x - rot1*D/pixel, beam_y - rot2*D/pixel). rot1 IS beam_x written as an angle and rot2 IS beam_y, component for component. So on a horizontal spindle the gauge tilt is rot1 and rot2 is refined; on a vertical spindle it is the other way round. With only one end of the alias restrained, a beam-centre error the prior refuses to absorb reappears as an angle: measured elsewhere in this campaign, refined rot1 tracks the starting beam centre at 0.072 deg per pixel against a geometric 0.080, while the refined beam never leaves its anchor by more than 0.24 px. Restraining both ends, in the same direction and to the same three-pixel budget, leaves the determined component - the direct beam, and the tilt perpendicular to the spindle - alone. Both restraints now take their direction from one projection of the spindle onto the detector plane rather than from two independent snaps to whichever of X/Y dominates, so they cannot disagree, and a spindle at any angle is handled. On a spindle along a detector axis the projection is the snap.	2026-09-01 15:33:51 +02:00
leonarski_fandClaude Opus 5	5869a27fe2	docs: one line per behaviour change in the rc.166 changelog, named by program ... The section had grown to eighteen entries, several of them describing the same report or carrying detail that belongs in a commit message. Consolidated to thirteen one-liners, each naming the program it concerns - rugnux, jfjoch_writer, jfjoch_broker, jfjoch_viewer - so a reader can find what changed in the part they use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:20:40 +02:00
leonarski_fandClaude Opus 5	51e90f5549	report: say what the measured detector tilt is worth, and what it is not ... The report invited the reader to check REFINED_DETECTOR_TILT against a powder calibration. Measured over two detectors' worth of crystals, that check misleads: the rotation fit does one outer round, so it leaves its starting value by only a small and crystal-dependent fraction of the distance to the calibrated value, and on the worse of the two detectors the per-crystal median sits an order of magnitude further from the powder answer than either method's uncertainty. A user comparing one run against their own calibration would conclude the calibration was wrong. What survives the aliasing is the direct beam, which is already printed beside it, and the MEDIAN of the tilt over several crystals on one detector - enough to show up a placeholder or a stale value in the file, not enough to replace a calibration. Say that, and say not to feed one run's value back into the instrument. The conditioning law (VIF = 4.40 tan(2theta_95)^-1.66) was tested as a per-run gate and does NOT order the errors - the crystals whose fit stays put are found in the best-conditioned band as often as the worst - so no threshold is added here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:15:01 +02:00
leonarski_fandClaude Opus 5	1a25d396a8	docs: the beam centre is the PONI, the direct beam is the beam ... Document direct_beam_x/direct_beam_y in the CBOR start-message table and in the HDF5 detectorSpecific catalogue, and add the section that says what the distinction is: the PONI is the foot of the perpendicular, the direct beam is where the beam lands, they are distance*tan(tilt)/pixel apart, and XDS ORGX/ORGY wants the second. Also record that the same field is a statement about the user's geometry in a broker master and a measurement in a rugnux _process.h5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:15:00 +02:00
leonarski_fandClaude Opus 5	26ba019752	writer: the direct beam beside the beam centre in the NXmx master ... Write direct_beam_x/direct_beam_y, in pixels, into /entry/instrument/detector/detectorSpecific - snake_case and unsuffixed like the rest of that Dectris-style group, with a units attribute like the beam_center_x it must be read against. Computed inside Metrology from the very locals that produce the beam_center_x/y and rot1/2/3 written a few lines away - refined when the offline analysis refined them, the StartMessage values otherwise - and from the same start.detector_distance the translation vector uses. The file therefore cannot disagree with itself: one composed geometry feeds all of them. Provenance differs by producer (the broker's master states the user's geometry, a rugnux _process.h5 a measured one), the field does not. The test writes a real master in all three NXmx layouts at a non-zero tilt and reads the datasets back, and requires the tilt to have moved the point, so a writer that stored the PONI would fail it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:12:29 +02:00
leonarski_fandClaude Opus 5	336fafffd0	cbor: carry the direct beam on the start message ... beam_center_x/y is the PONI - the foot of the perpendicular from the sample - so on a tilted detector it is not where the beam lands, and the two are distance*tan(tilt)/pixel apart (~8 px on a real in-house setup). A consumer that wants the beam position, which is what most of them mean, had to redo the tilt arithmetic or get it wrong. Add optional direct_beam_x/direct_beam_y to StartMessage, filled from the geometry's own GetDirectBeam_pxl() so there is no second formula, and put them at the top level of the CBOR start map beside beam_center_x/y. Optional keys, so a consumer that does not know them skips them. No _pxl suffix: the stream2 neighbours (beam_center_x, pixel_size_x, detector_distance) carry none and the units are in docs/CBOR.md. The API's calibration schema spells it direct_beam_x_pxl because its neighbours there are beam_x_pxl - same quantity, each matching its own neighbourhood. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:12:28 +02:00
leonarski_fandClaude Opus 5	0e245d28cf	docs: name the in-house SLS 2.0 data beside the public sets ... The acknowledgement read as though the whole test corpus came from other people's beamlines. It does not - in-house data collected at SLS 2.0 is tested alongside it. One clause, at the top of the section; everything that follows still concerns the public data, which is the part that carries an obligation to cite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:58:40 +02:00
leonarski_fandClaude Opus 5	04d823135e	rugnux: record what the two-pass completeness arm can decide, and what three stand-ins could not ... Only under -S does the arm read a completeness with headroom. De novo the search merge is in P1 and does not count its possible reflections, so there is no number and CC1/2 is the only live term. Three ways to give the arm one back were measured on 28 rotation crystals and none lands, so say so at the guard rather than leave the next reader to rediscover them: * Count the possible reflections on the search merge too. It is affordable - 5.2 ms against a 38 s run - but a healthy crystal reads 27-43% of the P1 hemisphere, so the 100.5% bound is never approached and no decision changes. It would only add a figure to the user's report that reads as the dataset's completeness while being a fraction of a different group's asymmetric unit. * Observation count. The two search merges are built in the same terms, so pass 2 retaining under 90% of pass 1's observations looks like the signal the completeness ratio stood in for. It never fires - bit-identical on all 28, including the crystal whose pass 2 predicted 44% fewer partials. Predicted partials and observations that survive into the merge are not the same population. * Completeness ratio. Same idea one level up, and it does fire - on the wrong crystals. This merge is deliberately never resolution-cut, so the possible list grows with whatever range the pass reached and a pass that predicts FINER scores as one that lost the sweep. At a 0.90 bound it reverted two crystals indexing at 100%, taking one of them from 0.07% to 0.29% cell deviation and ISa 19.5 to 17.1, and rescued nothing. Bringing the wrong-cell detector back for de-novo data means putting it on the final merge or retiring the arm and saying so; neither is done here. No test covers the decision line: Rugnux_Rotation does not reach this guard - the two passes in its log are the indexer's, not RunAllPasses' - and there is no fixture for building a RotationScaleMerge, so an assertion placed there would never execute. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:48:39 +02:00
leonarski_fandClaude Opus 5	8935a59721	rugnux: the two-pass guard says when it has no completeness, instead of printing 0.0% ... The guard that judges the post-refined pass against the header-geometry pass reads completeness off each pass's space-group search merge. That merge does not count possible reflections, so the quotient is 0.0 on both sides and every de-novo rotation report has been printing completeness 0.0% vs 0.0%, CC1/2 before corrections 0.994 vs 0.993 as though a comparison had happened. On the 28-dataset in-house battery that is all 28 runs. The number is not just uninformative, it is a test that did not run: "completeness above 100% means the cell is wrong" is the guard's wrong-cell arm, and it cannot fire against a constant zero, so only the CC1/2 arm decides - which is computed on whatever data the pass kept, so a pass that discards a large part of the sweep can post an equal CC1/2 and win. Carry a measured flag beside the number. The arm is skipped and the log and PASS_DECISION say "completeness not measured" where there is nothing to read. No behaviour changes: the arm could not fire before either. The next commit gives the merge the count so it can. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:48:25 +02:00
leonarski_fandClaude Opus 5	5ce7eb3014	docs: one changelog line for the geometry reporting work ... Three lines were appended over the course of the work - the tilt and direct beam in the report, the measured tilt beside them, and the post-refine beam bound. They are one change to a user: the report now describes one geometry and says which point is which, and the bound that governs it is measured against the run's own measurement rather than against the file it is meant to correct. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 07:39:23 +02:00
leonarski_fandClaude Opus 5	f98442077e	post-refine: bound the beam move on the measurement, not on the header ... PostRefine step B refuses a fitted beam centre more than 15 px from the geometry the run started from. Two things were wrong with that and they compounded. The 15 px was BOTH the Ceres search box and the acceptance threshold, so a fit that wanted to move further was pinned at the box face, landed at exactly hypot >= 15.0, and was then refused for being there. The gate never saw the fit it was judging. Measured on one crystal at three detector distances, the fit came back at 15.128, 15.133 and 15.130 px - the box corner, three times. And it was measured from the header, which is the value most worth correcting exactly when it is most wrong. The run already knows better: the default beam-centre check fits the centre from the isotropy of the scattered background on every rotation run, reported it, and then discarded it. The bound is now measured from whichever of the nominal centre and that measurement is nearer - accept a move that is small relative to something independent, rather than small relative to the header alone - and the search box is widened to match, so the gate decides rather than the box. The measurement is used only where it was precise enough to be adopted as a centre in its own right; a run without one is bounded exactly as before. Step B also now says WHICH of its four tests refused a fit, since all four leave the same geometry behind. Measured over 113 rotation datasets against the same corpus, one binary per arm, with a control arm that reproduced the stored battery on 112 of 112 reports: the bound fired and refused a real correction 1 6ukf the bound fired and caught a runaway 0 the bound never fired 112 On 6ukf the header centre is 22.1 px from what the background fit measures. The old fit, pinned at the box, improved the held-out positional residual 9-fold and was refused; the new fit lands 0.46 px from the measurement and improves it 418-fold. ISa 4.42 -> 6.71, R_meas 18.5 -> 12.8 %, CC1/2 0.987 -> 0.995, d_min 1.119 -> 0.973 A; at matched resolution, 1.68 A goes I/sigma 8.9 -> 15.1 and R_meas 28.8 -> 13.4 %. The refined cell moves from 0.4 % off the deposited one to under 0.1 %. It survives perturbation, and inverts it: over +-0.5 mm of detector distance the corrected run is flat (ISa 6.62 / 6.71 / 6.94) while the uncorrected one is bistable (4.02 / 4.42 / 6.90). The change removes an instability rather than exploiting one. No space group anywhere in the corpus moves, and no other dataset's report changes at all - 112 of 113 are identical. The distance half of the gate is untouched: it is separately argued, and it is what legitimately refuses the one dataset whose distance wants to move 1.05 %. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 20:19:36 +02:00
leonarski_fandClaude Opus 5	efca844bce	calibration: --no-refine-tilt holds the header tilt, it does not zero it ... GuessInitialGeometry resets rot1/rot2/rot3 to zero before seeding, and the branch that puts the header's tilt back was guarded by `refine_tilt && !tilt_refined`. With --no-refine-tilt the tilt is never free, so tilt_refined is false, so the guard is false, so the restore never ran - and the fit reported a zero tilt while the CLI printed "held at the header value". The guard only needs !tilt_refined: a tilt that was declined and a tilt nobody asked to refine both want the header put back and the beam centre and distance refitted around it. That is what the branch already does. Measured on a LaB6 sweep with a tilt imposed on the command line and asked to be held, --calibration spots: before Rot1= +0.0000 deg (+0.001400 rad from the header) after Rot1= -0.0802 deg (+0.000000 rad from the header) The rings path reaches this only when the spot-derived start wins, which is why it does not show on a file whose profile start is chosen; the spots path always did. A calibration whose header tilt is already zero is unaffected - checked, byte-equal distance either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 19:47:02 +02:00
leonarski_fandClaude Opus 5	dcfcff5bc4	report: the measured tilt, and one geometry instead of two ... Adds REFINED_DETECTOR_TILT - rot1/rot2 in degrees as rotation indexing measured them - so the value can be checked against a powder calibration, which measures the tilt from ring shape and is the better instrument for it. rot3 is not carried: a rotation about the beam is an exact null of a single-axis rotation experiment, so the fit cannot move it and printing it would suggest otherwise. Also repairs two defects in the commit that added DETECTOR_TILT, found in review. The geometry was taken from the caller's copy of the experiment, which is the state BEFORE the run, while BEAM_CENTRE and DETECTOR_DISTANCE come from result.used_*, which is the state after - so on a rotation two-pass the report mixed a post-refined centre with the file's rotations and DIRECT_BEAM was off by the whole post-refinement shift. JFJOCH_DATASET_SETTINGS carried the same mixture, which is a geometry no pass ever ran at. The tilt and the direct beam are now taken from experiment_ alongside the other three, so the block describes one geometry. And the prose claimed the tilt "is not refined here". It is: rotation indexing refines rot1/rot2 on every rotation run. What is true is that the result is never written back onto the geometry, which is why it needs a key of its own. M_PI -> PI (common/JFJochMath.h). rugnux is in the forced-Windows viewer-only set and M_PI is not portable to MSVC; the file did not include <cmath> either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 19:35:27 +02:00
leonarski_fandClaude Opus 5	7c00731992	docs: publish the powder calibration schemas in the python client docs ... docs/python_client is tracked and is what readthedocs publishes, but it is generated - update_version.sh copies it out of python-client after regenerating the client - so adding schemas to the spec left it four files behind. The README's model list did not name them and their pages did not exist. Copied the way update_version.sh copies them. The four new pages and four lines in the README, nothing else: the generator's README is built from the spec, so the stale Calibration* pages it had left on disk under the old names are absent from it and were removed before copying rather than published alongside their replacements. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 19:02:12 +02:00
leonarski_fandClaude Opus 5	bd6e933f04	api: name the calibration schemas powder_, not calibration_ ... "Calibration" already means something else in this broker: the JUNGFRAU pedestal and gain measurement, which owns /statistics/calibration, calibration_statistics and JFJochState::Calibration. Four schemas called calibration_* would have sat beside it meaning a completely different procedure, and the confusion is cheapest to remove now, before an endpoint returns them. calibration_output, calibration_quality, calibration_fit_sigma and calibration_spot_check become powder_calibration_output, powder_calibration_quality, powder_calibration_fit_sigma and powder_calibration_spot_check. All four, not only the outer one - the collision is in the word, and half a rename would read as though the inner three belonged to the other kind of calibration. Rename only. Every client regenerated from the spec: the C++ server model, the TypeScript frontend client, redoc-static.html, and the python client, which is gitignored but was checked to still decode the file rugnux writes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:58:24 +02:00
leonarski_fandClaude Opus 5	6fe3f30ab4	api: define the powder calibration result as calibration_output ... The JSON a calibration writes was a shape invented at its writer, described only by the comments around it. That is enough for a file somebody reads with jq and not enough for anything else: a client cannot type it, and an endpoint returning it later would have to declare the shape a second time and keep the two in step by hand. So declare it where every other shape in this system is declared. calibration_output holds dataset_settings and a calibration member; the latter is calibration_quality, which nests calibration_fit_sigma and calibration_spot_check. The descriptions carry what a reader has to know to use the numbers rather than only what they are named - that beam_x_pxl is the PONI and the direct beam is elsewhere, that the rotations travel together because a body omitting them states a flat detector, that a tilt below about three sigma was declined and pinned, and that the two correlations approach 1 as the tilt stops being separable from the beam centre. Nothing references it yet. It is declared now because /powder_calibration will return exactly this, and because the file rugnux already writes is decodable today: jfjoch_client's CalibrationOutput.from_dict reads it as it stands, with o.calibration.fit_sigma.correlation_beam_x_rot1 and the rest typed. Generated clients regenerated from the spec, as the spec requires: the C++ server model (four new pairs under broker/gen/model), the TypeScript frontend client, and broker/redoc-static.html. Both regenerations are purely additive - no existing generated file changed except to export the new names. The python client regenerates from the same spec and is gitignored. The test now validates the WHOLE file against the generated Calibration_output rather than only its geometry member against Dataset_settings, so the quality block is under the same contract: a field renamed or newly required in jfjoch_api.yaml fails here rather than at a client. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:54:09 +02:00
leonarski_fandClaude Opus 5	8691bf4876	calibration: write the result as JSON, not only as a PONI file ... A calibration run produced one file, and it was a pyFAI PONI - which pyFAI and its neighbours read and nothing in this system does. Carrying the answer back into the instrument meant a person reading numbers off a printed report and retyping them into a dataset_settings body, and the report is where the two points a "beam centre" can mean are easiest to confuse. So write <prefix>.json beside it. Its "dataset_settings" member holds the geometry under the property names broker/jfjoch_api.yaml gives them and holds nothing else, so it is a valid dataset_settings body as it stands: curl -X POST -H 'Content-Type: application/json' \ -d "$(jq -c .dataset_settings det.json)" http://broker:5232/start beam_x_pxl is the PONI, as everywhere here. The three poni_rot*_rad ride along whenever any is non-zero and are left out when all are zero: a body without them does not leave the tilt unstated, it states a FLAT detector, so they travel together or not at all - the same rule the report's JFJOCH_DATASET_SETTINGS block already follows. The "calibration" member holds what the run knows about that geometry: the residual, the fit's own sigmas and the correlation between the tilt and the beam centre, whether the tilt cleared its significance test or was declined and pinned, where the direct beam lands, and where the spots independently put the beam. A calibration that has gone wrong looks exactly like one that has not until those are read, and a machine-readable file that carried only the geometry would be the easiest possible way to feed a bad one into an instrument. Tested against the model generated from the spec rather than against a list of field names written out by hand: the file's dataset_settings member is parsed into org::openapitools::server::model::Dataset_settings and validated, so a field renamed or newly required in jfjoch_api.yaml fails here rather than at someone's POST. The tilted and untilted branches are both covered, and the PONI/direct-beam distinction is asserted rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:47:51 +02:00
leonarski_fandClaude Opus 5	16b368739e	calibration: re-integrate the images about the geometry that was fitted ... Everything the ring fit reads was binned with the geometry the run started from, and a calibration is run precisely because that geometry is in doubt. Binning is not something a later fit can undo: which pixel landed in which (q, phi) bin was decided when the images were read. Get the distance wrong and the radial sampling is the wrong scale - a run recovering 110 mm from a 250 mm header ended at rms 1.07 px where the same data from a right header gives 0.42. Get the centre wrong and every ring is smeared across its own sectors, and past about a hundred pixels the fit leaves rms 4.8 px even when it is started from the exact answer, because there is nothing left in the profile to fit. So integrate the images a second time, binned about what was fitted, and fit that. A calibration run is a handful of images and the answer is worth far more than the extra read. Re-binning about the FIRST fit is not enough on its own. Where the beam centre was badly out that fit is itself wrong, and binning about it digs deeper - rms 3.75 -> 5.66 on an exposure 200 px off. The spots' geometry is the one that does not degrade there, having never read the header, so both are tried where they differ and whichever comes back better is kept. On that exposure the first candidate gives 288 points at rms 5.66 and the second 369 at 0.81. The second pass is taken only if it is actually better, by the same rule that ranks everything else here - at least half as many ring points and a smaller residual - and it is skipped altogether when the fitted geometry moves a ring by less than the radial width of one profile bin, since re-binning would then put every pixel back where it already is. On the 110 mm exposure with its own header that is 1.77 px against a 1.84 px bin, so the run is untouched and its answer bit-identical. Measured on the 110 mm LaB6 exposure, true PONI x 765.90 at 110.03 mm. The calibration is now independent of the header it was given: beam-x out by +20 +40 +100 +200 +400 px -> 765.9-766.6, 110.02-110.05 mm 250 mm header, beam-x 780 -> 765.998 at 110.037, rms 0.394 250 mm header, beam-x 867 -> 765.723 at 110.038, rms 0.413 900 mm header, beam-x 1167 -> 766.496 at 110.038, rms 0.804 30 mm header, beam-x 967 -> 766.486 at 110.039, rms 0.807 All of those failed before this, most of them catastrophically. The residual inflation is gone with them: the 250 mm case now leaves 0.394 px, better than the same data from its own correct header. The four other datasets are unchanged where the re-bin fires, and slightly better where it does: rms 0.433 -> 0.423 at 150 mm and 0.555 -> 0.550 at 200 mm, with the geometry moving under a tenth of a pixel. A run that needs the second pass costs about twice one that does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:17:17 +02:00
leonarski_fandClaude Opus 5	49dd01331b	calibration: let the spots vote on the geometry in ring mode too ... The rings and the spots fail in different regimes, which is the whole reason to carry both. The header is right on a well-configured instrument and is the thing a calibration is run to check. The profile's own estimates are exact while the error stays small and stop meaning anything beyond that - past about half a ring spacing each ring reaches the azimuthally averaged profile as two horns rather than one peak, and the distance search reads a list of horns as a list of rings. The circle through the spots reads nothing from the header at all: measured on a 110 mm LaB6 exposure, --calibration spots returns the same geometry from a header 400 px out in the centre AND eight times out in distance. So ring mode now finds spots as well - half a second - and offers what they make of the geometry as one more starting hypothesis, fitted like the others with the residual left to choose. It is added whole rather than as a centre alone: GuessGeometry votes for the circle centre, clusters the radii into rings and takes the distance from the innermost, and those two belong together. Taking only its centre would not have helped, because the distance candidates come from a profile averaged about the header's centre, and a ring smeared over hundreds of pixels cannot be un-smeared by reading its bins differently. Measured on the 110 mm exposure, whose true PONI is 765.90 at 110.03 mm. A header 40 px out in the centre now lands within 0.6 px (it landed 41 px away before). The cases with BOTH wrong, which failed before this and equally before the beam centre work, now come out: 250 mm / 780 px gives 110.041 mm and 766.10 px against 42.3 mm and 782.7; 250 mm / 867 px gives 110.065 and 765.48. All five datasets are unchanged from their correct headers and the distance still recovers from any header between 25 and 1200 mm. Past about a hundred pixels nothing rescues ring mode, and the reason is the profile rather than the seeding: binned about a centre that far out it shows each ring smeared across its own sectors, so even started from the exact answer the fit leaves rms 4.8 px and drifts. Re-binning would fix it and would need the images read a second time; --calibration spots, which never touches the profile, already covers it. That regime is now visible rather than silent. The spots' beam centre is printed beside the fitted one as a cross-check - two methods sharing no assumption, so a reader can see at a glance whether they agree. It costs nothing, the spots having been found already, and it separates cleanly: 0.3-0.4 px on the good runs against 201.4 px on the exposure whose profile could not be fitted at all. Reported as a fact and not gated on, since at long distance both methods weaken together and the honest thing is to show the number (5.2 px at 300 mm, 12.6 at 500). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:58:26 +02:00
leonarski_fandClaude Opus 5	c84b91be8a	calibration: take the beam centre from the rings too ... The header's beam centre was the last input the ring fit had to be roughly right about. Each ring is looked for in a window a few pixels of radius wide, and a centre wrong by (dx, dy) puts a ring at a different q in every sector, so past about ten pixels the ring leaves that window over much of the turn - and the fit then reads its cos(phi) signal off whichever sectors are left, which are the ones where the signal is weakest. A 20 px error ended 31 px wrong. The rings answer this without a calibrant and without a distance. A powder ring is a conic centred on the beam, so a wrong centre makes EVERY ring's radius oscillate once per turn by the same amount: r(phi) = R + dx cos(phi) + dy sin(phi), solved directly and pooled over every ring the profile shows, with each ring searched about its own measured radius rather than about where a standard says it should be. Using it needs the extraction to follow the rings sector by sector, which is what ProfileRingTrack now does - exactly, and in all five parameters at once, by walking the ring in the geometry believed true and asking the binned geometry what q and azimuth it would have given each point. That replaces the flat-detector distance correction it grew out of. Following the rings is not free, and the reason is worth stating: a window that moves with phi makes every systematic of the peak finder - where the background line is taken, how the centroid sits in the window - vary with phi as well, and phi is exactly the axis the beam centre is read off. Measured, it costs rms 0.415 -> 0.525 px on a good 110 mm fit, and 0.831 when the window follows the fitted tilt too. So a second measurement is taken with a window that is the same in every sector - the binned geometry with only its DISTANCE replaced, which is phi-independent by construction - and both are offered to the same rule that ranks everything else here. Acquire by following, measure by holding still. The seeded centre is likewise a hypothesis and not a belief. It reads a once-per-turn wobble, and a tilt puts a term of that shape there too - one that grows as the radius squared, where a centre error does not - so pooling the rings absorbs part of the tilt into the centre. Believed outright it made a good 110 mm fit worse; offered as an alternative start it costs one more fit and needs no rule about when it applies. It is skipped entirely below a pixel, where it is not a different hypothesis at all, which keeps a well-headed run at 0.71 s. Measured on the 110 mm LaB6 exposure, whose true PONI is 765.90: a header centre 20 px out now lands within 0.5 px, where before it landed 31 px away. All five datasets are unchanged from their correct headers, and the distance still recovers from any header between 25 and 1200 mm. The limit is now understood rather than merely reached. Past a few pixels the azimuthally averaged profile stops showing rings: a ring tracing r(phi) piles up density where that turns round, so it averages into the two HORNS of the sinusoid, at R-|d| and R+|d|. The radius finder reports two rings where there is one, and the gap between them is 2|d| - the search window shrinks to exactly the offset it was meant to span. That caps recovery at roughly half the ring spacing, about 20 px here and failing by 40. Beyond it nothing is left in an azimuthally binned profile, and --calibration spots, which works from the spot positions themselves, is the method that still can. One pre-existing limit measured and NOT introduced here: a wrong distance together with a centre more than about 5 px out fails, because the centre error splits the radius list the distance search reads. The committed code before this change fails identically on those cases. Also fixed: fit_from now takes a whole geometry rather than a distance, and the declined-tilt refit was inheriting rot1/rot2 from it - pinning the tilt at exactly the unvalidated value the gate had just rejected. Same fault the gate exists to catch, one level up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:38:36 +02:00
leonarski_fandClaude Opus 5	9267bd67de	report: the detector tilt, and the two points the beam centre is not ... BEAM_CENTRE is the PONI - the foot of the perpendicular from the sample - so on a tilted detector it is not where the direct beam lands. The report printed only that one number and never mentioned the tilt at all, so the two points were indistinguishable to a reader. On a detector at 85 mm with a 0.22 degree tilt they are 8.3 px apart. Print DETECTOR_TILT (rot1/rot2/rot3, degrees, as the run used them - nothing is refined here) and DIRECT_BEAM beside BEAM_CENTRE, and say which is which. JFJOCH_DATASET_SETTINGS was the more serious half: it exists to carry a geometry back into the instrument, and dataset_settings carries poni_rot1/2/3_rad, but the block omitted them - so it described a FLAT detector, silently, on every tilted setup. The rotations now ride with it when they are non-zero; they stay out when they are zero, because the API's own default is 0.0 and the shorter block means the same thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 17:35:03 +02:00
leonarski_fandClaude Opus 5	01ad1e1743	calibration: only report a tilt the fit actually measured ... The detector tilt is refined by default, and on a pattern that cannot separate it from the beam centre the fit returns one anyway - there was nothing to stop it. Both displace a ring's radius as cos(phi), and only how that amplitude grows with the ring's radius tells them apart, which takes two well-sampled rings. At 500 mm on the LaB6 series only two rings reach the detector and the outer one is barely there: the tilt came out at the opposite sign to every shorter distance, dragged the PONI 28 px, and bought a residual of 0.960 px against 0.962 pinned. The covariance says so plainly - 0.1 sigma, and a beam centre quoted to +-180 px. So ask it. A tilt is kept only where the fit had it free AND it stands at least three times its own uncertainty; otherwise rot1/rot2 go back to the header's values and the beam centre and distance are refitted around them. Over the series the tilt stands at 50, 33, 15 and 8 sigma at 110 to 300 mm and 0.1 at 500 mm, so any threshold between 2 and 5 gives the same verdict on all five - this says which regime a fit is in, not where a line was drawn. The declined 500 mm fit lands on a direct beam of 773.53 px, against 773.56 for the pinned fit measured independently. It is a rejection criterion and nothing more. Clearing it does not certify a tilt: that estimator is limited by systematics rather than by this sigma, and a coherent half-pixel error in the ring positions fakes a tilt of the usual size while leaving sigma small. The report says "refined", never "verified". Writing the gate turned up a related fault in the pass loop. RingOptimizer pins the tilt by itself when every point it is given lies on one ring, and on a barely-sampled pattern a later pass lands in exactly that state - which froze the tilt at whatever the FIRST pass had produced and returned it with sigma zero, an unmeasured tilt wearing the appearance of a fixed one. The gate reads that as "not measured" and refits pinned, which is why it is stated over the geometry that gets reported rather than over what the last fit happened to do. Both paths are covered, profile and spots; the spots path was reporting a refined tilt as declined for the same reason. Two things measured and NOT taken: A robust loss. A Cauchy loss scaled to the previous pass's median residual changed nothing on the series - rms 0.415 to 0.421 at 110 mm, no case improved, every direct beam within 0.06 px. Ring points are per-sector peaks that already had to stand 3 sigma clear of their own background, so there are no gross outliers left to reject. Recorded at the call site rather than left as an unused option. A quality gate that refuses a bad calibration. Three candidate signals, all measured against naming the wrong standard on LaB6 data: sigma(PONI) does not see it at all (0.52-0.65 px, indistinguishable from healthy); the residual only half sees it (3.2-3.6 px wrong against 0.4-1.0 right, but a correct run from a wrong header sits at 1.0-2.4 and would be caught too); and the seed's match score is dominated by how many rings the calibrant lists, scoring 0.29 for a perfect LaB6 fit against 0.21 for a wrongly named silicon. None of the three separates, so no gate is shipped. What the run does say is the recovered distance against the header, and a wrong standard moves that to 446 mm on a 110 mm exposure - unmissable, and the operator's call rather than a threshold's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:07:37 +02:00
leonarski_fandClaude Opus 5	a5f416fcdc	calibration: take the detector distance from the rings, not from the header ... A powder calibration is run because nobody is sure the header is right, and the header's distance was the one number the fit could not survive being wrong about. The ring search is local - each ring is looked for inside a window a few pixels of radius wide - so a distance more than a percent or two out puts every ring outside its own window, and the fit then converges on whatever background fluctuation each window contains. It does not fail: a 110 mm exposure told the detector was at 150 mm reported 149.8 mm, with 146 ring points and exit 0. Only its residual said anything, 5.3 px against 0.4 px, and nothing read it. Measure the distance from the rings instead. The peaks of the azimuthally averaged profile give ring RADII, and a radius does not depend on the assumed distance at all - bin i holds the pixels at one particular radius whatever q that radius was called - so the radii are a property of the image. Against the calibrant's d-spacings, r = D tan(2 asin(lambda/2d)) then has one unknown. It is scanned rather than solved because the pairing of observed rings to d-spacings is unknown too, and the winning basin is solved in closed form. Nothing here reads the header distance except to bin the profile; it needs only the wavelength, the pixel size and the detector's extent. A powder pattern has genuine distance aliases, so one answer is not enough. A cubic primitive standard puts its rings at radii proportional to sqrt(N), and scaling the distance by sqrt(2) maps ring N onto ring 2N - most of the comb still lands on peaks. Measured: the 110 mm exposure with a 115 mm header scored its best at 156.5 mm, which is 110*sqrt(2). No adjustment of the score removes an alias the lattice really has, so the scan hands back the few best distances and each is fitted, the header among them as one hypothesis of several. The residual then separates them - 0.4 px against 5.2 px on that case - subject to an attempt explaining a comparable share of the pattern first, because a start so wrong that one ring point survives leaves a residual of exactly zero. Each attempt re-extracts at the geometry it converged to and fits again. The seed is measured from blended peaks and is good to about a per cent, close enough to converge from but far enough to sit every search window a few pixels off its ring, and an off-centre window takes its background off the ring's own flank. Nothing is re-read from disk, so the loop is free. Measured on the LaB6 distance series. A 110 mm dataset now recovers 110.03-110.17 mm from any header between 25 and 1200 mm, against +-2 mm before. All five datasets recover their own distance from a fixed wrong 250 mm header. With correct headers, four of the five are bit-identical to before and the 500 mm one moves by a single ring point - the two-ring fit whose tilt is 0.1 sigma anyway. Run time is unchanged at 0.62 s. The residual is larger on a run whose header was wrong (1.1 px against 0.4 px on the 110 mm case), because the profile was still binned at the wrong distance and its radial sampling is correspondingly coarse. The geometry is right; only the scatter about it is inflated. Re-running with the recovered distance recovers the residual too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:48:32 +02:00