209 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	9c19646f6a	refine: fit the direction of the goniometer axis, not its length ... The residual applies angle_rad * |rot_vec| and rot_vec is a free three-vector, so the first pass has been fitting a goniometer rotation SCALE nobody asked for. GoniometerAxis::Axis() then normalises it away on write-back, and RotationIndexer scores the candidate with the normalised axis - so the cell that won the fit is judged under a rotation model the fit did not use. Measured over 43 rotation datasets: the length reaches 1.2%, and the fit-vs-score disagreement a median 0.124 deg and up to 6.19 deg of goniometer angle, against rocking widths of 0.05-0.36 deg. That score picks the lattice class, which nothing later revisits. The fitted length is not a usable measurement of anything either: on synthetic data it recovers 54% of a known scale error, repeated first passes on one dataset disagree with each other in sign, 26 of 43 datasets disagree with themselves, and on the one dataset with a proven 1.3% stage fault it comes out negative. It is absorbing other systematics. The rotation scale is measured properly, once, with cross-validation and gates, in PostRefine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 18:36:32 +02:00
leonarski_f	d32c8d526b	geometry: make which tilt component is restrained two numbers rather than a shape ... Both directions of the tilt now go through the same restraint, with a budget each in shared direct-beam pixels; zero means free. Swapping which component is held, or holding both, is a change to two constants. The perpendicular budget is zero today, which is the arrangement the measurements support: it is the component whose conditioning tracks the 2theta the fit reaches, while the parallel one's does not move with 2theta at all - a conditioning number that ignores the data is the prior talking. This is a clarity change, not a precision one, and the record should not read otherwise: the standing 'freeing the tilt inflates both beam components' cost is the perpendicular component's doing, not the parallel one's, so restraining the parallel one recovers essentially none of it. What it buys is that a beam-centre error is no longer reported as an angle.	2026-09-01 15:34:06 +02:00
leonarski_fandClaude Opus 5	5869a27fe2	docs: one line per behaviour change in the rc.166 changelog, named by program ... The section had grown to eighteen entries, several of them describing the same report or carrying detail that belongs in a commit message. Consolidated to thirteen one-liners, each naming the program it concerns - rugnux, jfjoch_writer, jfjoch_broker, jfjoch_viewer - so a reader can find what changed in the part they use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:20:40 +02:00
leonarski_fandClaude Opus 5	51e90f5549	report: say what the measured detector tilt is worth, and what it is not ... The report invited the reader to check REFINED_DETECTOR_TILT against a powder calibration. Measured over two detectors' worth of crystals, that check misleads: the rotation fit does one outer round, so it leaves its starting value by only a small and crystal-dependent fraction of the distance to the calibrated value, and on the worse of the two detectors the per-crystal median sits an order of magnitude further from the powder answer than either method's uncertainty. A user comparing one run against their own calibration would conclude the calibration was wrong. What survives the aliasing is the direct beam, which is already printed beside it, and the MEDIAN of the tilt over several crystals on one detector - enough to show up a placeholder or a stale value in the file, not enough to replace a calibration. Say that, and say not to feed one run's value back into the instrument. The conditioning law (VIF = 4.40 tan(2theta_95)^-1.66) was tested as a per-run gate and does NOT order the errors - the crystals whose fit stays put are found in the best-conditioned band as often as the worst - so no threshold is added here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:15:01 +02:00
leonarski_fandClaude Opus 5	1a25d396a8	docs: the beam centre is the PONI, the direct beam is the beam ... Document direct_beam_x/direct_beam_y in the CBOR start-message table and in the HDF5 detectorSpecific catalogue, and add the section that says what the distinction is: the PONI is the foot of the perpendicular, the direct beam is where the beam lands, they are distance*tan(tilt)/pixel apart, and XDS ORGX/ORGY wants the second. Also record that the same field is a statement about the user's geometry in a broker master and a measurement in a rugnux _process.h5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 15:15:00 +02:00
leonarski_fandClaude Opus 5	0e245d28cf	docs: name the in-house SLS 2.0 data beside the public sets ... The acknowledgement read as though the whole test corpus came from other people's beamlines. It does not - in-house data collected at SLS 2.0 is tested alongside it. One clause, at the top of the section; everything that follows still concerns the public data, which is the part that carries an obligation to cite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:58:40 +02:00
leonarski_fandClaude Opus 5	8935a59721	rugnux: the two-pass guard says when it has no completeness, instead of printing 0.0% ... The guard that judges the post-refined pass against the header-geometry pass reads completeness off each pass's space-group search merge. That merge does not count possible reflections, so the quotient is 0.0 on both sides and every de-novo rotation report has been printing completeness 0.0% vs 0.0%, CC1/2 before corrections 0.994 vs 0.993 as though a comparison had happened. On the 28-dataset in-house battery that is all 28 runs. The number is not just uninformative, it is a test that did not run: "completeness above 100% means the cell is wrong" is the guard's wrong-cell arm, and it cannot fire against a constant zero, so only the CC1/2 arm decides - which is computed on whatever data the pass kept, so a pass that discards a large part of the sweep can post an equal CC1/2 and win. Carry a measured flag beside the number. The arm is skipped and the log and PASS_DECISION say "completeness not measured" where there is nothing to read. No behaviour changes: the arm could not fire before either. The next commit gives the merge the count so it can. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:48:25 +02:00
leonarski_fandClaude Opus 5	5ce7eb3014	docs: one changelog line for the geometry reporting work ... Three lines were appended over the course of the work - the tilt and direct beam in the report, the measured tilt beside them, and the post-refine beam bound. They are one change to a user: the report now describes one geometry and says which point is which, and the bound that governs it is measured against the run's own measurement rather than against the file it is meant to correct. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 07:39:23 +02:00
leonarski_fandClaude Opus 5	f98442077e	post-refine: bound the beam move on the measurement, not on the header ... PostRefine step B refuses a fitted beam centre more than 15 px from the geometry the run started from. Two things were wrong with that and they compounded. The 15 px was BOTH the Ceres search box and the acceptance threshold, so a fit that wanted to move further was pinned at the box face, landed at exactly hypot >= 15.0, and was then refused for being there. The gate never saw the fit it was judging. Measured on one crystal at three detector distances, the fit came back at 15.128, 15.133 and 15.130 px - the box corner, three times. And it was measured from the header, which is the value most worth correcting exactly when it is most wrong. The run already knows better: the default beam-centre check fits the centre from the isotropy of the scattered background on every rotation run, reported it, and then discarded it. The bound is now measured from whichever of the nominal centre and that measurement is nearer - accept a move that is small relative to something independent, rather than small relative to the header alone - and the search box is widened to match, so the gate decides rather than the box. The measurement is used only where it was precise enough to be adopted as a centre in its own right; a run without one is bounded exactly as before. Step B also now says WHICH of its four tests refused a fit, since all four leave the same geometry behind. Measured over 113 rotation datasets against the same corpus, one binary per arm, with a control arm that reproduced the stored battery on 112 of 112 reports: the bound fired and refused a real correction 1 6ukf the bound fired and caught a runaway 0 the bound never fired 112 On 6ukf the header centre is 22.1 px from what the background fit measures. The old fit, pinned at the box, improved the held-out positional residual 9-fold and was refused; the new fit lands 0.46 px from the measurement and improves it 418-fold. ISa 4.42 -> 6.71, R_meas 18.5 -> 12.8 %, CC1/2 0.987 -> 0.995, d_min 1.119 -> 0.973 A; at matched resolution, 1.68 A goes I/sigma 8.9 -> 15.1 and R_meas 28.8 -> 13.4 %. The refined cell moves from 0.4 % off the deposited one to under 0.1 %. It survives perturbation, and inverts it: over +-0.5 mm of detector distance the corrected run is flat (ISa 6.62 / 6.71 / 6.94) while the uncorrected one is bistable (4.02 / 4.42 / 6.90). The change removes an instability rather than exploiting one. No space group anywhere in the corpus moves, and no other dataset's report changes at all - 112 of 113 are identical. The distance half of the gate is untouched: it is separately argued, and it is what legitimately refuses the one dataset whose distance wants to move 1.05 %. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 20:19:36 +02:00
leonarski_fandClaude Opus 5	efca844bce	calibration: --no-refine-tilt holds the header tilt, it does not zero it ... GuessInitialGeometry resets rot1/rot2/rot3 to zero before seeding, and the branch that puts the header's tilt back was guarded by `refine_tilt && !tilt_refined`. With --no-refine-tilt the tilt is never free, so tilt_refined is false, so the guard is false, so the restore never ran - and the fit reported a zero tilt while the CLI printed "held at the header value". The guard only needs !tilt_refined: a tilt that was declined and a tilt nobody asked to refine both want the header put back and the beam centre and distance refitted around it. That is what the branch already does. Measured on a LaB6 sweep with a tilt imposed on the command line and asked to be held, --calibration spots: before Rot1= +0.0000 deg (+0.001400 rad from the header) after Rot1= -0.0802 deg (+0.000000 rad from the header) The rings path reaches this only when the spot-derived start wins, which is why it does not show on a file whose profile start is chosen; the spots path always did. A calibration whose header tilt is already zero is unaffected - checked, byte-equal distance either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 19:47:02 +02:00
leonarski_fandClaude Opus 5	dcfcff5bc4	report: the measured tilt, and one geometry instead of two ... Adds REFINED_DETECTOR_TILT - rot1/rot2 in degrees as rotation indexing measured them - so the value can be checked against a powder calibration, which measures the tilt from ring shape and is the better instrument for it. rot3 is not carried: a rotation about the beam is an exact null of a single-axis rotation experiment, so the fit cannot move it and printing it would suggest otherwise. Also repairs two defects in the commit that added DETECTOR_TILT, found in review. The geometry was taken from the caller's copy of the experiment, which is the state BEFORE the run, while BEAM_CENTRE and DETECTOR_DISTANCE come from result.used_*, which is the state after - so on a rotation two-pass the report mixed a post-refined centre with the file's rotations and DIRECT_BEAM was off by the whole post-refinement shift. JFJOCH_DATASET_SETTINGS carried the same mixture, which is a geometry no pass ever ran at. The tilt and the direct beam are now taken from experiment_ alongside the other three, so the block describes one geometry. And the prose claimed the tilt "is not refined here". It is: rotation indexing refines rot1/rot2 on every rotation run. What is true is that the result is never written back onto the geometry, which is why it needs a key of its own. M_PI -> PI (common/JFJochMath.h). rugnux is in the forced-Windows viewer-only set and M_PI is not portable to MSVC; the file did not include <cmath> either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 19:35:27 +02:00
leonarski_fandClaude Opus 5	7c00731992	docs: publish the powder calibration schemas in the python client docs ... docs/python_client is tracked and is what readthedocs publishes, but it is generated - update_version.sh copies it out of python-client after regenerating the client - so adding schemas to the spec left it four files behind. The README's model list did not name them and their pages did not exist. Copied the way update_version.sh copies them. The four new pages and four lines in the README, nothing else: the generator's README is built from the spec, so the stale Calibration* pages it had left on disk under the old names are absent from it and were removed before copying rather than published alongside their replacements. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 19:02:12 +02:00
leonarski_fandClaude Opus 5	8691bf4876	calibration: write the result as JSON, not only as a PONI file ... A calibration run produced one file, and it was a pyFAI PONI - which pyFAI and its neighbours read and nothing in this system does. Carrying the answer back into the instrument meant a person reading numbers off a printed report and retyping them into a dataset_settings body, and the report is where the two points a "beam centre" can mean are easiest to confuse. So write <prefix>.json beside it. Its "dataset_settings" member holds the geometry under the property names broker/jfjoch_api.yaml gives them and holds nothing else, so it is a valid dataset_settings body as it stands: curl -X POST -H 'Content-Type: application/json' \ -d "$(jq -c .dataset_settings det.json)" http://broker:5232/start beam_x_pxl is the PONI, as everywhere here. The three poni_rot*_rad ride along whenever any is non-zero and are left out when all are zero: a body without them does not leave the tilt unstated, it states a FLAT detector, so they travel together or not at all - the same rule the report's JFJOCH_DATASET_SETTINGS block already follows. The "calibration" member holds what the run knows about that geometry: the residual, the fit's own sigmas and the correlation between the tilt and the beam centre, whether the tilt cleared its significance test or was declined and pinned, where the direct beam lands, and where the spots independently put the beam. A calibration that has gone wrong looks exactly like one that has not until those are read, and a machine-readable file that carried only the geometry would be the easiest possible way to feed a bad one into an instrument. Tested against the model generated from the spec rather than against a list of field names written out by hand: the file's dataset_settings member is parsed into org::openapitools::server::model::Dataset_settings and validated, so a field renamed or newly required in jfjoch_api.yaml fails here rather than at someone's POST. The tilted and untilted branches are both covered, and the PONI/direct-beam distinction is asserted rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:47:51 +02:00
leonarski_fandClaude Opus 5	9267bd67de	report: the detector tilt, and the two points the beam centre is not ... BEAM_CENTRE is the PONI - the foot of the perpendicular from the sample - so on a tilted detector it is not where the direct beam lands. The report printed only that one number and never mentioned the tilt at all, so the two points were indistinguishable to a reader. On a detector at 85 mm with a 0.22 degree tilt they are 8.3 px apart. Print DETECTOR_TILT (rot1/rot2/rot3, degrees, as the run used them - nothing is refined here) and DIRECT_BEAM beside BEAM_CENTRE, and say which is which. JFJOCH_DATASET_SETTINGS was the more serious half: it exists to carry a geometry back into the instrument, and dataset_settings carries poni_rot1/2/3_rad, but the block omitted them - so it described a FLAT detector, silently, on every tilted setup. The rotations now ride with it when they are non-zero; they stay out when they are zero, because the API's own default is 0.0 and the shorter block means the same thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 17:35:03 +02:00
leonarski_fandClaude Opus 5	ba24946c0f	twinning: do not report a twin the L-test rules out ... The two indicators can only move one way under twinning - <|L|> down from 0.500 toward 0.375, the second moment down from 2.0 toward 1.5 - but they were combined with an OR, so a narrow intensity distribution could report a twin on its own while the L-test said the opposite. That is not a marginal disagreement. It fires on small-cell rotation data, where a twin fraction of 0.50 was reported for a crystal whose <|L|> was 0.63: a value no twin can produce, and one that points at the opposite situation - a structure whose intensities behave centric. Suppress the flag when <|L|> sits at or above its untwinned value; the two indicators are otherwise combined exactly as before. Measured over 113 stored reports: two flip, both of them wrong today, and nothing else moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 15:34:16 +02:00
leonarski_fandClaude Opus 5	71cca91ec1	rugnux: rank screw candidates without the control count ... A screw's control class is the complement of its absent class on one axial row, so the two move together: a candidate that predicts more of the row absent leaves fewer reflections to be judged against. The Beta tail AbsenceEvidence computes grows with that control count, so the stricter candidate was charged for the very reflections it correctly called extinct, and a group whose predicted-absent class is a strict superset of another's - with the extra reflections equally dead - could score LOWER than the group that explains only part of the row. Two mechanisms, one dataset each. * The control-count minimum gated the row MEAN, which is the scale the zone's evidence is stated in, on the same count as the row MEDIAN, which is a violation threshold. A candidate could therefore forfeit a whole zone by being right: on a tetragonal 4_1/4_3 wedge, thirteen 00l reflections measured at 0.1% of the two l = 4n beside them scored ZERO because only two control reflections were left, while the nine of them a 4_2 also predicts absent scored 38.7. The mean now stands on two, the median still on three. * The Beta tail is replaced, for a SCREW zone only, by its b -> infinity limit - the same statistic with the control count dropped. A p-value computed against each candidate's own null is not one scale across candidates; what is left is the likelihood ratio of the absent class against Wilson at the row's own mean, which is a sum over reflections and therefore comparable. Asymptotically it is n_absent * (log(1/ubar) - 1), so an equally dead superset can no longer score lower. Measured on a tetragonal 4_1/4_3 crystal: 29 dead 00l against 8 control read 47.7 nats where a subset of 19 of them against 18 control read 55.5. The centring statistic is untouched: its control is the whole present population, not the complement of a claim on one row, so neither mechanism applies to it. Measured. Analytic superset monotonicity, on a 176368-point grid with no calibration and no corpus: 1.29% violations -> 0.00%, exactly monotone wherever both candidates keep a control class. Over 112 stored merges (open, in-house and private arms) exactly ONE decision changes and it is a gain, a 4_2 -> 4_1/4_3 swap on a deposited 4_3; the screw column over the deposited arm goes 33/41 -> 34/41 with zero screw over-calls before and after, and the four centring outcomes are bit-identical. An eight-wedge reproducer whose 00l row a strict candidate eats goes from two mis-calls to none, end to end. The refused side moves the safe way: 103 zones the old statistic scored negative go lower, none turns positive, and the minimum evidence of an adopted screw rises 20.4 -> 23.8 against an unchanged bound of 20. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 15:33:29 +02:00
leonarski_fandClaude Opus 5	c25418319a	report: the twinning measured before the search, and the strong-direction limit ... Three additions to the report, none of which changes a number the run computes. The twinning statistics printed today are measured in the Laue class the run ADOPTED, so where the search promoted the point group they can say no more than "no twin law exists inside the class it chose" - and a twin is precisely what would have caused that promotion. The uncontaminated pair was already computed on the subgroup merge the search was given and reached only a log line; it is now printed as L_TEST_MEAN_ABS_L_BEFORE_SEARCH and SECOND_MOMENT_I_BEFORE_SEARCH, beside the post-adoption pair rather than instead of it, since the two answer different questions. ANISOTROPY_D_MIN_BEST names the finest of the three principal limits so it can be grepped, and a paragraph distinguishes the three resolutions the report now carries: what was written, where the isotropic fit crossed, and how far the crystal reaches where it reaches furthest. It is read off the fitted tensor rather than off the cut, so it can land either side of the other two - measured on one dataset at 2.01 A against a 1.81 A cut - and the text points at ANISOTROPY_D_MIN_CENSORED for when it is the edge of the data instead. A warning when CC1/2 falls and then climbs again, which means the fall-off the cut is read off does not describe the data. A climb counts only above CC1/2 0.30 and only if it exceeds 0.05: past that the number is oscillating in noise beyond any cut this run would take. It advises no re-cut - the generous cut is deliberate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 14:06:42 +02:00
leonarski_fandClaude Opus 5	d189faeec7	viewer: draw grid scan cells in the proportion of the scan steps ... A grid scan with, say, a 20 um step in x and a 5 um step in y was drawn as a square grid, so the picture had nothing to do with the shape of the area that was scanned. The cells now carry that proportion. The anisotropy lives only in the view transform: JFJochImage gains a pixel_aspect_ (the drawn height of one pixel in units of its width, 1.0 for a detector image), the initial fit fits the image as if it were that much taller and then puts the factor back into the transform, and the grid view sets it from |step_y / step_x|. Scene coordinates stay one unit per cell, so the mouse mapping, the selected-image box, the pixel labels and the ROI code need no change and the uniform wheel zoom keeps the proportion. Measured on an 8x4 grid driven offscreen: m11/m22 comes out 4.000 for steps 20/5 um, 0.250 for 5/20, 12.333 for 37/3 and exactly 1.000 - the old behaviour - for a square step. Every cell centre still maps back through mapToScene to its own cell in all four cases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hcoh6VNrmSswjfeMeDQqWP	2026-08-31 13:38:51 +02:00
leonarski_fandClaude Opus 5	756c3a22bb	docs: confirmed cells for three small-molecule sets; drop the unprocessable CCD one ... The four Diamond I19 small-molecule datasets had no reference of any kind, so a run on them could not be scored at all. Three now have one, each from a published structure with its DOI verified against Crossref; the fourth has a published space group but no numeric cell anywhere, which is recorded as such. Also removes the "obtained but not in the table" section. That dataset is stored as CCD TIFFs the reader does not support, so it is not processed here and adds nothing to a page about what the test corpus contains. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 11:02:38 +02:00
leonarski_fandClaude Opus 5	6d37b31948	report: give the resolution the CC1/2 fit reached, not only the one it was cut at ... The automatic cutoff fits the CC1/2 fall-off, takes the crossing of the target (0.30 by default) and then deliberately keeps one more shell. Only the extended limit was reported, as INCLUDE_RESOLUTION_RANGE, and that is the number a reader takes for "the resolution of this dataset" - so the report quoted a limit that is generous on purpose as if it were the measurement. The generosity itself is right and stays: a shell that is included can still be downweighted or dropped by refinement, while one that was truncated cannot be put back. What was missing is the other number. ComputeCCHalfLogisticCutoff now also returns the crossing, ApplyResolutionCutoff passes it out, and the report prints it as FITTED_RESOLUTION beside the range the reflections were written to, with a sentence saying which is which. The log line names both as well. Nothing about the cut, the merge or the written reflections changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 11:02:37 +02:00
leonarski_fandClaude Opus 5	55d9e2536f	release: 1.0.0-rc.166 ... The changelog entry for this release is six lines, not the fifty-four the development log had accumulated: what a user of rc.166 gets is native miniCBF input, masters from other facilities opening, the beam centre measured on every run, a 2theta-swung detector placed where the file says, the unmerged MTZ and a P1 merge written by default, and a substantially reworked symmetry determination. The per-change detail is in the commits. update_version.sh regenerates the three clients from broker/jfjoch_api.yaml, so this also carries the one API description that had drifted from the generated code - fft_high_resolution_A, which sizes the FFT's projection histogram and does not filter spots by resolution. Everything else in the regenerated tree is the version string. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 09:11:22 +02:00
leonarski_fandClaude Opus 5	3a30dc8ab5	rugnux: a space group that names twice the cell is a signal to halve it ... C 1 1 2 and C 1 1 2_1 are the only two monoclinic settings whose change of basis from the reference has determinant 1/2, and the reason is physical: with unique axis c the C-centring vector lies wholly in the oblique a-b plane and reduces away. Such a group is never a distinct Bravais centring - it is a primitive lattice written on twice the cell - and it wins the candidate table only because the table is scored on that doubled cell, where half the reflection positions are absent by construction. Refusing to enumerate those settings was measured and is worse: the volume-preserving alternative keeps the by-construction absences as real observations and merges no better than P1. So keep the enumeration as it is and act on the win instead: halve the cell, re-classify in the group's own system, carry the reflections across (the invented positions map to non-integral indices and are dropped, which is what the centred candidate was already doing) and search again there. Measured on one rotation crystal: C 1 1 2_1 on 68.6 73.0 170.3 90 90 90 becomes P 1 2_1 1 on 50.1 170.3 50.1 90 93.5 90, exactly the deposited setting, with every merge statistic unchanged (R_meas 0.1209, CC1/2 0.9979, d_min 1.238) and ISa 18.27 -> 18.31. Stable across +/-0.5 mm of detector distance. Eight other crystals are bit-identical, and across 112 stored battery reports exactly one adopts a volume-doubling setting. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:19:38 +02:00
leonarski_fandClaude Opus 5	7c3ab72fed	reader: refuse a CBF that is not a detector image instead of misreading it ... Three ways a miniCBF opened silently wrong. A byte-offset CBF with no PILATUS header at all was accepted: Count_cutoff then defaulted to 0, SaturationLimitFromValue(0) is 1, and every pixel at or above one count was flagged saturated - the integration accept gate drops the whole reflection, so the run comes out empty for a reason nothing reports. The pixel size defaulted to 0 with no validation anywhere downstream, which collapses every resolution, every scattering vector and the beam centre in millimetres. XDS writes its correction files in exactly this shape, so this is not hypothetical. A pixel size is now required to claim the file at all, and a missing Count_cutoff leaves the saturation limit unset - falling back to the container's own overflow, which can only fail to call a pixel saturated - with a warning saying so. And the header captures are character classes, not number grammars: "[\d.eE+-]+" matches a bare "." and "(\d+)" matches a digit string too long for int64. std::stod and std::stoll answer both with a raw std:: exception, which escaped the format probe - CanRead catches JFJochException only - so merely LOOKING at a corrupt file threw out of the viewer's open path. A header field that does not parse is now reported as a malformed header. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:40 +02:00
leonarski_f	e0ff51e2a0	lattice: transpose the change of basis to a primitive cell, as everywhere else ... gemmi states centred_to_primitive as an operator on COORDINATES and CrystalLattice::Multiply combines BASIS VECTORS, so the matrix has to be transposed on the way in - as it already is at both of the other places a gemmi Op::Rot reaches Multiply, one of them in this same file. A, B, C, I and F are symmetric matrices, so for them the transpose is a no-op and the omission never showed. R and H are not. Measured on an R-centred hexagonal lattice, ToPrimitive('R') returned 59.5 81.7 43.3 / 145.6 124.5 46.7 where the rhombohedral primitive cell is 49.3 49.3 49.3 / 60.9 60.9 60.9. What hid it is that a determinant is unchanged by transposition, so the VOLUME came out right - and most callers only take the volume. It is not only cosmetic: the result feeds the re-seating path that puts a lattice into a space group the user fixed by hand, so an R-centred lattice was handed the classifier a "primitive" cell that is not that lattice - broken for exactly the centring whose setting most needs re-seating. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 6ca00e927d664e870515c04164defa81d8a18725)	2026-08-31 07:17:00 +02:00
leonarski_f	527a5187f4	lattice: restore the character that names a centred monoclinic mI reduced form ... ITA character 43 was absent from the table. It is the type-II reduced form of a centred monoclinic lattice with no length equality - mC, mI, mA and mF are one Bravais lattice in four settings, and this row names the one whose conventional cell comes out I-centred. With the row missing such a cell reaches character 44 and is reported as triclinic, losing its centring outright. It accounted for 30 of the 31 demotions left after the two fixes before this one, and for 4.1% of random centred-monoclinic lattices. Both of its conditions are equalities on scalar products - International Tables gives them as 2|D+E+F| = A+B and |2D+F| = B - so the three angle tests are vacuous for it and the second equality is what selects it. cond_2DF was declared in the character struct and never tested against anything, because until now no row used it. Audited over 14000 exact lattices, the row fires 35 times: 32 are centred monoclinic lattices it recovers correctly and 3 are triclinic cells it promotes, all three of which Le Page promotes at the same tolerance. No over-call is attributable to it. The row is taken from International Tables rather than derived - no single integer matrix of determinant 2 covers more than 68% of these cells - and the test checks the conventional cell it produces has two right angles and twice the primitive volume. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 4c2baf0cc3b2177196a9448df06e24b5446de96a)	2026-08-31 07:17:00 +02:00
leonarski_f	931d7acc7c	lattice: judge a structurally-zero scalar product against the size of the cell ... The Niggli type of a reduced cell is the sign of its three scalar products, and gemmi's reduction was asked to decide those signs against an absolute tolerance of 1e-9 while the products themselves are 10^3 to 10^5 A^2 and the cell is held in float. A product that is structurally zero therefore arrives carrying about 1e-4 A^2 of rounding and is read as definitely signed. The reduction lands on the wrong side of the type-I/type-II boundary, the character written for the other side matches nothing, and the lattice comes back with less symmetry than it has. It is not a corner case. Take an exactly body-centred tetragonal lattice with c > a*sqrt(2) and merely ROTATE IT IN SPACE: 38 of 60 rotations lose the 4-fold, and it comes back C-centred. Every lattice this code classifies is a refined, rotated one; the two tetragonal-I cases already in the tests are axis-aligned, which is the one corner where the rounding vanishes. The tolerance is now scaled by the cell's own magnitude. Over 14845 exact lattices that puts it on a plateau three decades wide - demotions 461 -> 31 - with the over-call count unchanged at every point of it, and with the three fixes that follow the plateau is four decades wide and the demotions are zero. The constant is 100x the one Grosse-Kunstleve et al. give because theirs is calibrated for a double-precision cell and this one is float: measured, their value recovers 6% of these lattices and this one 93%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit b82f217719450e1e46e94be5b5a43ed8ffbf2ca4)	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	b31fb0a55c	symmetry: take the point group from whichever merge found it, the absences from all observations ... The search runs on two merges - one holding every observation, one holding only the well-measured ones - and the second was allowed to report but never to decide. That cost real symmetry: over the corpus the two arms disagree on eight crystals and the arm that found the LARGER point group is right on all eight. A promotion is refused by the most damning statistic it can be shown, so only one of the two arms has to be starved for a genuine operator to be thrown away, and which arm that is depends on the crystal. But the two halves of the answer do not come from the same place. Absences live in the weak reflections the filter discards, so adopting the filtered arm whole recovers the point group and loses the screw - a P6_5 crystal comes back as P6. So the arms are split rather than ranked: the point group is taken from whichever supports the larger one, and its absences are then judged here, on every observation. SearchSpaceGroupOptions::fixed_point_group was written for exactly this and had no caller. Measured on the deposited arm: a trigonal-3 answer becomes P 6_1 against a deposited P 6_5 (an enantiomorph pair, so correct), and a monoclinic answer becomes P 2 2 2 against a deposited P 2_1 2_1 2. The screw survives the split, which it did not when the filtered arm was adopted whole. No crystal that was already right moved, and the over-call column is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	d35e8f680f	symmetry: say when the cell metric hosts more symmetry than the group adopted ... Every symmetry under-call in the corpus has the same signature: a lattice whose metric carries rotations the adopted group does not, with nothing in the run saying so. The user is left to notice that a P1 answer sits on a cell whose axes are equal and whose angles are 60 degrees. So a run that determines its own group now compares the two and says what it sees. It decides nothing - no threshold, no promotion, no demotion, no reprocessing - and the message says as much, because a pseudo-symmetric metric is ordinary and only the intensities can settle whether the extra rotations are real. Measured over 83 crystals it fires on 19 and covers 6 of the 7 point-group under-calls; on the crystal whose deposited group is F 4 3 2 and which rugnux reports as P 1, the line reads "the cell metric is cubic - it admits 24 rotations where P 1 has 1". The metric symmetry is Le Page's, taken through GEMMI's implementation, at an obliquity of 1 degree - the middle of the band over which the answer is stable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	493be2cf19	symmetry: enumerate the settings the cell can host, not only the reference ones ... The space-group search offered a candidate only if gemmi calls it the reference setting. A setting is a statement about direction, so that restricted the search to the axes the convention chose: a crystal whose 2-fold lies on c had no rung between P1 and 222 and fell to P1, and one whose screws lie on b and c was reported as the group with a single screw on c - the wrong group, not a lower one, because the candidate that predicts a subset of the real absences and nothing else wins on no evidence at all. Both stages now enumerate more, under refusals rather than thresholds. Stage B offers the non-reference settings of the chosen point group. Their rotation set is equal to the chosen one, not merely contained in it, so this cannot raise the symmetry; what it adds is a screw or a centring on the axis the data show it on. A candidate is offered only if the cell's own metric admits the rotations its setting names, and one predicting exactly the absences another candidate already predicts is dropped as the same hypothesis under a second name. A non-reference candidate whose centring class this merge does not contain is refused outright: the reference path may adopt an untested centring because the caller's centred-lattice re-test backs it, and a non-reference setting has no such backing. Stage A offers the rotation sets no reference setting carries - the a-unique and c-unique monoclinic 2-folds, and the two rhombohedral-axes trigonal groups - and only those, so every point group reachable before is still reached by the same group in the same setting. A rung reached only that way may be ADOPTED but does not judge anything else: it is skipped when the reference chi^2 is formed and when a higher promotion's parents are collected. Without that it made higher promotions strictly harder - a promotion answers to the most damning of its parents, and offering two more order-2 subgroups of 222 refused 222 and 432 on crystals that had them. The run report gains SPACE_GROUP_NAME beside SPACE_GROUP_NUMBER, since the number alone does not say which axes a group's symmetry lies on, and a group that is not a reference setting is printed as its extended Hermann-Mauguin name. The two-arm reconciliation compares point groups by that name rather than by number for the same reason. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	26fc4b02b3	symmetry: carry the space group as the group, not as its number ... The adopted space group travelled the pipeline as a bare int and was rebuilt downstream with find_spacegroup_by_number, which returns the reference setting. So every setting a number cannot name was destroyed one line after it was determined: P 1 1 2 came back as P 1 2 1, I 1 1 2 as C 1 2 1, R 3:R as R 3:H. DatasetSettings now holds the gemmi::SpaceGroup itself, DiffractionExperiment exposes it as GetGemmiSpaceGroup() / GetSpaceGroupOrP1(), and everything that used to take an int - HKLKeyGenerator (its int constructor is gone, so the compiler finds the callers), the merge, the R-free flags, French-Wilson, the reindexing ambiguity, the completeness enumeration, the MTZ and mmCIF exports, the model validation - takes the group. -S keeps the setting the symbol names rather than reducing it to a number. The end message carries both spellings and a reader prefers the name, since only the name keeps the setting while the number is what a reader written before the name understands. It carries them over CBOR too: the determined group was never serialised at all, so a group rugnux chose reached the master file only when the same process wrote it, and an online writer fell back to whatever the user had supplied at the start. Both keys are optional additions, so an older reader skips them and a newer one reads an older sender. On disk the master's /entry/sample/space_group carries the extended Hermann-Mauguin name and is what the reader takes the group from, so a setting survives a _process.h5 and the --mode scale that re-reads it; the number stays beside it and is the fallback for files written before. Every one of the 230 reference settings the old writer could produce reads back as itself, so older files are unaffected. Stage A and Stage B of the search still enumerate reference settings only, so this determines no group differently today - it is what the enumeration needs before it can be widened. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:16:43 +02:00
leonarski_f	6ed4ea541e	reader: a link to a file that is not there is not a dataset that exists ... Every DECTRIS Eiger master links saturation_value, pixel_mask, bit_depth_readout and serial_number into a companion <prefix>_meta.h5, and that file is routinely not kept when a dataset is archived or deposited. The existence test asked only whether the LINK was written, which it is, so every optional-field guard in the reader answered yes and the read that followed threw. A deposited Eiger 16M set could not be opened at all, over values the reader was perfectly prepared to do without. Exists() now asks the second question too - whether the object the link names can be reached - so an orphaned link reads as absent and the fallbacks behind it do their job. The saturation value is then allowed to be missing outright, because on such a file it is: neither the NXmx name nor the DECTRIS one is readable, and there is no third place to look. Left unset, GetSaturationLimit() falls back to the container's own overflow. That is the safe direction - it can only fail to call a pixel saturated, where too LOW a value drops the whole reflection and silently removes the strongest data - and the run says out loud that nothing will be called saturated. The set that could not be opened now processes to 2.17 A against a deposited 2.40 A, in the deposited space group, with a cell agreeing to 0.08%. Output is byte-identical on datasets that already opened. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 0637979f6d95b446406ab70d1f3982841d195b36)	2026-08-30 21:43:54 +02:00
leonarski_fandClaude Opus 5	fb18457e6f	docs: bring the changelog and the method notes up to rc.166 ... The rc.166 changelog was missing fourteen user-visible changes and carried rationale and measurements that belong here instead. Added: the native miniCBF sweep reader, the third-party and firmware-1.x NXmx masters, the plain LZ4 filter, the image orientation taken from the file's module direction vectors, the two beam-centre flags and their rescues, and the FFT reach past 500 A. Trimmed the rest to one line each, moving the numbers out of the user-facing file. RUGNUX.md described the input as a single Jungfraujoch master file, which it has not been since this branch; it now covers the foreign and legacy masters, the accepted compression filters and the miniCBF sweep, including how a sweep is collected from one named frame. Six options existed with no entry in the table - --beam-center-check, --beam-center-search, --fft-min-unit-cell, --min-indexed-spots, --rot3 and --no-p1-crosscheck - and -C now moves both FFT cell bounds, which was not written down anywhere. CPU_DATA_ANALYSIS.md carried two statements this branch made false: 7.5 still said pass 2 reuses pass 1's space group, and 13.1 still said centrings are ranked by net absence count. Both now describe what the code does - the group is determined after pass 2, and centrings are ranked by the same Beta-tail likelihood the screw test uses. Also documents the per-zone screw scoring, the coplanarity volume-fraction guard, the plane-normal transform, the FFT cell bounds and the twelve refined candidates. The miniCBF reader implements the x-CBF_BYTE_OFFSET scheme and reads the imgCIF axis table from the specification alone. No CBF code is vendored or linked, so there is no licence obligation, but reimplementing a published specification carries one of credit: ACKNOWLEDGEMENT.md gains a section and the two algorithms carry a one-line reference each. Both DOIs were resolved before being written. Rottger's initial was wrong where this branch first cited it - K, not A. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 21:15:31 +02:00
leonarski_fandClaude Opus 5	9924dd9fc3	docs: the open test battery, updated - 82 datasets, and not only non-SLS ... The public data the pipeline is exercised on has grown from 59 datasets to 82, 77 of them with a released PDB entry and released structure factors, so the page that credits the depositors and carries the DOI to cite for each is brought up to date with what is actually run. Renamed from NON_SLS_TEST_DATA to EXTERNAL_TEST_DATA, because the old title stopped being true: a few of the sets were collected at SLS beamlines, where the data are still written by someone else's detector and someone else's acquisition system. What the battery tests is foreign files, not a foreign facility. Also rewritten from the current archives rather than the earlier sample: - Multi-collection archives: eleven are not a single continuous rotation, not four. Seven IRRMC archives hold more than one collection; one sweep is kept in six of them, and both are kept in the one whose two sweeps are at different wavelengths. A repository project page is not a reliable guide here - one describes a 900-frame sweep its own tarball does not contain. - Detector labels: 76 rows can be compared against the PDB entry. Seven genuinely conflict, and one of those the file settles outright - pixel count, pixel size, sensor thickness and firmware string agree with an EIGER2 9M against the entry's PILATUS4 4M. A further 29 differ only in how much they state, which is not a conflict. - One archive ships 30 placeholder files named like images that are 64-byte text; named on the page so a reader that globs the directory is not surprised by them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 20:50:10 +02:00
leonarski_fandClaude Opus 5	03456dcddd	calibration: an option to hold the detector tilt fixed ... --mode calibration fits five parameters - beam centre, distance and the two PONI tilts - and a program that cannot express a tilted detector has nowhere to put the last two. Dropping them after the fact is worse than never fitting them: the centre and the distance of a tilted fit have already absorbed the tilt, so the flattened geometry is right nowhere. rugnux --no-refine-tilt, the "Refine detector tilt" tick box on the viewer's Calib page and RingOptimizer's refine_tilt argument hold rot1/rot2 at the value the geometry came in with and fit the remaining three. That is the best flat-detector answer, and the one such a program would refine to itself. Measured on a five-distance calibrant series. At short distance the tilt is real and reproducible - three independent fits agreeing to 0.01 deg, radial rms 1.4 -> 0.4 px - and its direct beam agrees with the background beam-centre estimator to 0.05 px, so the tilted model is the physically right one. The pinned fit then displaces the centre 2.6 px to absorb the tilt and lands within 0.03 px of the same place at every distance. Past ~300 mm, where only two rings reach the detector, the tilt is instead under-determined: it comes out with the opposite sign to every short-distance fit and drags the PONI 28 px while the rms does not move (0.960 against 0.962). The existing degeneracy guard only fires on a strictly single ring, so it does not catch that; declining a tilt that does not pay for itself in rms is left for a separate change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 20:44:12 +02:00
leonarski_f	d5818713cb	rugnux: also merge in P1, so a wrong space group is recoverable ... A rotation run that determines its own space group now also writes <prefix>_P1.mtz: the same observations, the same scaling, merged in P1. If the group was wrong there is no route back today except processing the images again, and the P1 data settle it - re-merge, re-solve or re-refine in any subgroup. Written whether or not the search adopted P1, because a file whose presence depends on what the pipeline decided cannot be harvested by a script. A run given -S writes nothing: its group's centring absences were never integrated, so the P1 merge would be missing whole classes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f # Conflicts: # docs/CHANGELOG.md # rugnux/rugnux_cli.cpp	2026-08-30 14:15:20 +02:00
leonarski_f	123dc22be2	rugnux: write the unmerged MTZ by default ... A run that does not ask for it still has to hand its data to aimless, pointless or careless eventually, and the file it needs is one a user had to know a flag to get. It is the largest a run produces, but it is 3.4 to 5.8 per cent of the images it replaces - and the better it is, the less anyone needs to keep those. The batteries pass --no-export-unmerged: they process the whole corpus and would write a few gigabytes nobody reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f # Conflicts: # docs/CHANGELOG.md	2026-08-30 14:14:28 +02:00
leonarski_fandClaude Opus 5	a27c4cf26f	reader: take a miniCBF's mounting from the imgCIF axis table its header states ... A miniCBF header states three things about how the instrument is put together that the reader was assuming instead: which laboratory direction the image's columns run along, which its rows run along, and which the spindle turns about. Some beamlines append a CBF template block holding the full imgCIF axis table, which says all three outright. Two instruments in the corpus are not what was assumed, in two different ways. One mounts its detector a quarter turn round, so the image's columns run vertically. Another turns its spindle about the VERTICAL, with the image mounted the usual way; its table says so, and its "# Oscillation_axis" line says so a second way, by naming the image direction the spindle runs along rather than a vector. Either error leaves the spindle 90 degrees from the image. That is not a sign, so the run's axis-sign rescue cannot reach it, and no refinement recovers it: all three affected sweeps indexed nothing usable. So the table is read. The element axes give the image orientation, matched against the eight discrete mountings exactly as the NXmx module directions already are - the match itself moves to DetectorOrientation, so both readers share one definition rather than two copies. The goniometer axis with no parent gives the spindle DIRECTION; its sign stays the rescue's business, which is the part a convention can legitimately differ on. The detector axis with no parent gives the 2theta arm, replacing the assumption that the arm shares the spindle's axis - the one header stating both states them with the same vector, so this changes no answer, only what it rests on. imgCIF's frame differs from the internal one by a half turn about x, a rotation and not a mirror, as writer/HDF5NXmx.cpp already records from the other side. Where a header carries no table, a "+SLOW" on the Oscillation_axis line still says the spindle runs along the image's slow direction. That is the only thing one of the three affected sets says about it. The axis NAME on that line stays unusable - the header that carries both says "X.CW" where its own table says Y - but the direction token is not: where both are present they agree, which is what makes reading it evidence rather than a guess. Also: naming a frame with no directory at all now finds its sweep. parent_path() of a bare filename is empty and iterating an empty path finds nothing, so running from inside the data directory reported that no images were found. Measured, with nothing on the command line. The vertical-spindle protein set goes from no usable lattice to 100% indexed, P 6(3) 2 2 with a cell 0.43% from deposited, 87846 reflections at 86.3% completeness and CC(1/2) 0.995. Its companion from the same detector, which has no table and only the +SLOW token, goes from a spurious monoclinic cell at 2.3% completeness and I/sigma 0.21 to the right orthorhombic lattice, 97.7% indexed, 59.7% complete, CC(1/2) 0.996. The quarter-turned set's three sweeps, at three arm positions, now all index without the hand-passed quarter turn they needed and agree on one cell to 0.03 A. Six miniCBF sets that state no table and no +SLOW - including one whose Oscillation_axis line names an axis in a third dialect - are byte-identical in .hkl, .mtz, .cif and the image statistics, as are two NXmx sets, which is the shared orientation matcher moving nothing on that path either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:03:32 +02:00
leonarski_fandClaude Opus 5	dc71cb7299	rugnux: keep the P1 merge, so a wrong space group is recoverable ... A de-novo rotation run merges in P1 to search for the symmetry, adopts a group, and then overwrites that merge with the in-symmetry one. If the adopted group is wrong the user has no route back: every file the run wrote, and every statistic in them, is computed in the group that was assumed, so nothing in the output says the choice was wrong and the only way to a different answer is to process the images again. Merge the same integration once more in P1 after the run's own files are written, and put it beside them as <prefix>_P1.mtz. From it the space group can be re-determined and the data re-merged, re-solved or re-refined in any subgroup - measured end to end on three crystals: POINTLESS reads the file, recovers the group, and AIMLESS re-merges it, reproducing rugnux's own merged intensities at CC 0.9965 where the two agree. On one of the three it recovered the deposited/XDS group where this run had under-called the screw axis. The merge is the full one - correction surfaces fitted, ice rings and near-tangential observations kept, whole resolution range - not the deliberately degraded merge the space-group search itself runs on, and it is what `rugnux --mode scale -S P1` produces from a _process.h5. That route already existed but needs a _process.h5, which a merging run does not write, so it only helped a user who had foreseen the problem. Every de-novo rotation run writes the file, including one whose search concluded P1 and where it therefore repeats the merged output byte for byte. Whether a file exists must not depend on what the pipeline decided: a script harvesting results would otherwise have to reproduce the search's decision to know whether to expect it, and a missing file would not separate "the run chose P1" from "the run failed". A user-fixed -S writes nothing, and that condition is not a pipeline decision. With a group fixed, prediction rejects that group's centring absences (IndexAndRefine.cpp:499-506), so those reflections are never integrated; a P1 merge built from such a run would be missing whole centring classes and would mislead rather than merely be smaller. Cost, median of five paired runs read off the log timestamps (the box is shared, so end-to-end wall time is noise): +0.32 s of 4.2 s, +2.18 s of 31.4 s, +0.19 s of 17.9 s, +0.66 s of 15.9 s - 1 to 8% of a run. The file is 1.6x the merged MTZ on a monoclinic crystal and 30x on a cubic one, where the merged MTZ is tiny; it is well under the unmerged export in every case measured, and 0.02 to 0.6% of the raw dataset. Rotation only for now: the stills merge re-fits per-image scales and per-reflection partialities onto the integration outcomes, and _unmerged.mtz is written from those afterwards, so on stills this extra merge alters a file that is the run's own output. Fixing that means running the cross-check below the unmerged export, which needs the merge lambda hoisted out of its block; deferred, since rotation is what the online pipeline processes. On rotation the merged .mtz, .cif, .hkl, _image.dat, _unmerged.mtz and _unmerged_partials.mtz are all byte-identical with and without this change, including on a run whose search returns P1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 12:36:49 +02:00
leonarski_fandClaude Opus 5	5c44e544dc	reader: place a detector swung out on a 2theta arm where the file says it stands ... Chemical crystallography reaches high angle by swinging the detector out on a 2theta arm. Both readers had the number and neither used it: the miniCBF header's Detector_2theta was parsed into a struct member nothing ever read, and on the NXmx side the rotation was in the depends_on chain, which was not followed at all. A sweep taken at 30 degrees was therefore processed with its detector plane 30 degrees from where it stood, and nothing indexed. The geometry could already express it, and needed no change: the arm turns the detector about the sample, so the distance is still measured along the detector normal and the beam centre is still the point of normal incidence - which is exactly the PONI convention, and a swung detector is one PONI rotation. What moves is the direct beam, by distance*tan(2theta), off the beam centre and often off the detector. NXmx is the harder half, because the swing has no field of its own: it is one rotation in the chain the detector's position depends on, and "two_theta" is only one beamline's name for that dataset. So the chain is followed and its rotations composed, rather than a field of one name being looked for - each transformation states its vector in the frame of the one it depends on, which is why the product is the whole placement. Translations are skipped; they are the distance and the beam centre, which the file states separately in the square-on frame. Vectors come from McStas through the same 180-degree turn about z the module directions already use, a proper rotation, so an axis carried through it turns the same way. The three rotations a file this system writes ARE that chain, and are also read as the PONI angles - so those three paths are skipped, or every tilted file we have ever written would come back tilted twice. That is the one way this change could have broken existing data, and the test for it writes a tilted file and reads it back. For miniCBF the arm turns about the base spindle axis: on the four-circle geometry those headers describe the two are one axis, and the imgCIF axis table such a header carries states them with the same vector. Both now come from one constant, so a later correction to the frame moves them together. Measured. On a swung NXmx sweep the chain gives rot2 = -0.34907 rad for the 20 degrees it states, and the sweep goes from "nothing was integrated" to 25000 reflections at 82.2% completeness and CC(1/2) 0.9993, in the same space group and the same cell to 0.03 A as the square-on sweep of that crystal; the opposite sign indexes nothing. A miniCBF sweep at 30 degrees goes the same way, to 0.585 A, and a second sweep of that crystal at 55 degrees reaches 0.476 A and reproduces the cell again - with a low-resolution limit of 2.36 A rather than 13 A, which is what a detector swung that far records. On all of them post-refinement recovers the header's own beam centre and distance, and the beam stop shadow sits within four pixels of where the swung geometry puts the direct beam, 417 and 537 pixels from where the unswung one does. Seven sets whose detector is square to the beam, three of them carrying a chain whose 2theta is zero, are byte-identical in .hkl, .mtz, .cif and the image statistics. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 12:36:46 +02:00
leonarski_fandClaude Opus 5	5f47d73cc1	rugnux: write the unmerged MTZ by default ... The unmerged export is how a run's observations reach the rest of the field. It is what aimless, pointless and careless read, it is what a head-to-head against another program's answer runs through, and handing several of these files to pointless is the only way to merge sweeps rugnux does not combine itself. Behind a flag it reached only the people who already knew the flag existed, which is the shape of defect the "rugnux <file> with nothing else" direction asks to design out. --no-export-unmerged turns it off, and the regression batteries now pass it: the objection to defaulting it on was their disk and time cost, not the product's. Measured on three rotation crystals of 900, 1800 and 3600 frames: the file is 21, 86 and 26 MB - two to five times the merged .mtz, .cif and .hkl put together - and the write costs 0.24, 2.29 and 1.11 s, 5.5 %, 12.2 % and 3.1 % of each run's own wall time. The cost is linear in the number of observations and independent of how long processing took, so it is the largest fraction of the runs rugnux finishes fastest, not of the longest sweeps. A run's .hkl, .mtz and .cif are byte-identical with the export on and off. --export-unmerged-partials stays off: it is a second, larger artefact and a separate question. The viewer's reprocessing dialog ties the export to its merged-output switch, so it appears beside the merged files rather than beside a job that only asked for the per-image _process.h5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 10:51:25 +02:00
leonarski_fandClaude Opus 5	bec2a10a3a	docs: add the eight new public datasets to the non-SLS test list ... Three IUCrData Raw Data Letters (Zenodo, CC-BY-4.0) and five SBGrid Data Bank depositions (CC0) have been added to the data the pipeline is exercised on. Same rules as the rest of the page: the source is the repository and its own citable DOI, every one of which was resolved before it was written down; beamline, resolution, space group and cell are the values deposited with the PDB entry; the detector is read out of the image files. None of the new detectors disagree with their PDB entry. Two of them do not fit the page's one-row-per-sweep shape, so the shape is described rather than flattened. The 6R72 Zenodo record holds two complete 360-degree collections on one crystal - a helical one that produced the deposited structure and a low-dose one that has no PDB entry - and both are listed, sharing a DOI, with the second in the no-PDB-entry table. The three CHESS depositions are 4-11 wedges of 50 degrees per crystal plus a rotation taken with the crystal translated out of the beam, tabulated in a new section. One SBGrid deposition is named but not in the table: its images are 1995 CCD TIFFs, a format the reader does not support, so it is not processed here and saying so is more useful than leaving it out. ACKNOWLEDGEMENT gains the new per-repository counts and a pointer to the Raw Data Letter citations; TESTS points at the page. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-29 23:40:10 +02:00
leonarski_fandClaude Opus 5	fed077e683	geometry: hold the detector plane as axis vectors, and give the discrete part its own home ... The detector plane was three PONI angles and nothing else, so the two things it cannot express - an image mirrored in Y, and one mounted at a multiple of 90 degrees - had no home at all. They are now the DetectorOrientation carried by the detector setup, composed with the PONI rotation into one orthogonal matrix whose columns ARE the fast axis, the slow axis and the sample->PONI normal: lab = R(rot1, rot2, rot3) * Delta * ( (x-bx)*p , (y-by)*p , distance ) GetFastAxis/GetSlowAxis/GetNormalAxis read those columns and DetectorAxes() sets the plane from them, decomposing back to the angles; PoniRotMatrix and PoniAnglesFromMatrix are the conversion in both directions, exact on the canonical branch (rot2 in [-pi/2, pi/2]) and with a stated convention at gimbal lock. The angles stay stored rather than re-derived, so a geometry given as angles is written back as the same angles, to the bit. Delta is never inferred. In particular an arbitrary rot3 is NOT decomposed into a quarter turn plus a residual: rot3 is a fitted quantity, and a least-squares step must not be able to turn the stored image. It is set only where something states it - the detector setup, --detector-mirror-y / --detector-quarter-turns, or the value a file this system wrote records - and defaults to the identity, which makes the whole change a no-op for every existing detector and every existing file. It is a different setting from DetectorSetup::mirror_y, which flips the MODULE LAYOUT while an image is assembled and so decides what the stored pixels are. Merging the two would apply the mirror twice for every modular detector, or change the pixel content of every file written; both are ruled out. The new one earns its keep exactly where the old one is a no-op: a detector whose image arrives already assembled has no layout to flip. Both generators are signed permutations of the in-plane offset, so they preserve the distance from the PONI. That is why almost nothing downstream changes: everything needing an azimuth already goes through LabCoord, and everything that does not needs only a radius. The two hand-written copies of the rotation - XtalResidual and RingOptimizer - take the discrete part as four constants next to cos_rot3/sin_rot3, since it acts in the detector frame where rot3 acts in the laboratory and cannot be folded into it. RingOptimizer needs it despite being a radial fit: it fits the tilt, and the discrete part changes which way the tilt tips a ring. Carried as two optional CBOR keys and two detectorSpecific datasets, both back-compatible; the NXmx module axis vectors and the translation direction stop being hardcoded and are computed from it, reproducing today's values exactly at the identity. GetPoniRotMatrix is renamed GetDetectorMatrix, because it is no longer only the PONI rotation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 23:00:35 +02:00
leonarski_fandClaude Opus 5	9d3c2787f8	docs: credit the public diffraction data the pipeline is tested against ... Jungfraujoch is developed at one facility, so the reader and the reduction pipeline are exercised on data collected elsewhere - other detectors, other file formats, other conventions. That data was collected and published by other people, and until now nothing in the repository said so. NON_SLS_TEST_DATA lists all 51 datasets: the DOI to cite for each, the repository it came from, and the experiment as deposited. Beamline, resolution, space group and cell are the values deposited with the corresponding PDB entry, read from the RCSB data API - not results measured here; no quantity produced by this software appears on the page. The detector is read out of the image file instead, because the detector named in a PDB entry is often only approximate, and the twelve cases where the two disagree are listed rather than silently reconciled. ACKNOWLEDGEMENT gains a section for the repositories themselves, with the IRRMC, SBGrid and Zenodo citations and the PDB citation for the metadata. Every DOI on both pages was resolved before it was written down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 21:59:49 +02:00
leonarski_fandClaude Opus 5	84efb0666c	viewer: open a miniCBF sweep, the same way rugnux already does ... The native miniCBF reader added for rugnux is a plain JFJochReader, so the viewer only needed to be told which one to open a file with. JFJochImageReadingWorker held a concrete JFJochHDF5Reader; it now holds both readers and a JFJochReader* pointing at whichever the open file needs, chosen by JFJochCBFReader::CanRead. Naming any frame opens the whole sweep - the reader's own template matcher decides which frames belong to it, so a directory holding two sweeps or XDS auxiliary files is not spliced together. The previous file is closed before the new one is opened, so switching format between HDF5 and CBF in one session leaves nothing behind. Everything the viewer draws already goes through JFJochReader and JFJochReaderDataset, so nothing else had to move. The two HDF5-only features stay on the HDF5 reader: calibration images, and the reprocessing snapshots, which are metadata read back over the same images the reader is serving and have nothing to attach to for a directory of CBFs. A raw sweep therefore shows geometry and pictures with an empty run list, which is the state a live HTTP stream is already in. Reprocessing jobs run on a CBF sweep too - JFJochProcessController opens its own reader the same way, and Rugnux has handled a CBF source since the reader landed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 21:49:09 +02:00
leonarski_fandClaude Opus 5	172a845cbb	rotation indexing: refine twelve candidate lattices instead of four ... The first pass refined only min(candidates, 4). The comment at the selection already conceded that the pre-refinement indexed fraction is an unreliable discriminator, and on one dataset degenerate cells occupied three of those four slots - so the right cell, ranked fifth, was never refined. The preceding commit's volume guard frees those slots but does not widen them. This is a one-for-one trade, measured by rebuilding with the old value to confirm the attribution. It gains one dataset: R_meas 27.2% to 23.8%, with mean I over sigma better in EVERY shell at identical shell edges, so it is not a resolution ramp. It costs another: a clean abort becomes a wrong four-fold supercell in P1 at 36% completeness. That is the same shape of trade already accepted for the plane-normal cap search in the preceding commit - an honest failure becoming a wrong answer - on a different dataset. It is committed separately so it can be reverted on its own if the maintainer weighs this instance differently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 20:31:35 +02:00
leonarski_fandClaude Opus 5	4e8db41785	indexing: refuse a coplanar candidate, and search the plane normal when the shortlist is flat ... Three related changes to the FFT candidate path, batteried together because they touch the same function. A COPLANAR CANDIDATE REACHED REFINEMENT. ReduceResults filtered triples on lengths and angles only - the 30-150 degree bound admits any flat combination - and there was no volume test. On one dataset 41 of 5535 candidates had |V|/abc below 0.05, with a clean decade gap to the next, and three of them reached the optimizer. UnitCell is float, and for a cell that flat the metric determinant is around 1.5e-7, so float32 gets its sign wrong 19% of the time where float64 never does. The guard against a negative argument to sqrt then CREATES the singularity it was meant to prevent: it puts c in the a-b plane, the reciprocal volume is 1/0, and the residual is 0 times infinity. Ceres reported a not-a-number Jacobian and wrote several hundred lines of solver output per failed solve. VolumeFraction() is |V|/(|a||b||c|), rejected below 0.02 - about 1.1 degrees off flat, ten times below the flattest real candidate observed and a thousand times above where float loses the sign. It is enforced at the producer and at the two optimizer entry points. Note the existing sanity checks use ABSOLUTE volume, which a 320 cubic-angstrom flat cell passes. The same reciprocal-volume division is now guarded at the two remaining sites that share the pattern. A SHORTLIST CONFINED TO ONE PLANE cannot close a cell, and the row it is missing is the plane normal. That is detected from the scatter-matrix eigenvalue ratio - measured, degenerate clouds score 2e-5 to 3.3e-4 against 0.026 or more for every non-degenerate one, a factor of eighty - and one further transform is spent with the same direction count inside a three-degree cap about the normal, so the plan and buffers are untouched. More directions cannot substitute: at the exact true direction the long axis ranks 1422 of 16384 by prominence while the shortlist cut is four times higher. Ranking, not sampling, is the obstacle. A four-fold denser grid was measured and rejected - it reaches the same answer to three decimal places and takes a run from 2.5 to 8 GB of device memory. fft_min_unit_cell_A is reachable as --fft-min-unit-cell and is lowered automatically by -C, mirroring how the maximum is already raised. The default of 10 is unchanged: a lower floor admits spurious sub-cells on protein data, and over 73 protein runs the floor was never lowered while the sibling maximum did fire twice, so the path is live and correctly inert. Corpus of 93 datasets, both arms, one build: 72 bit-identical on report content and p.hkl checksum, 13 failing identically, and the count of working datasets rises by one. The volume guard fires on 58 of 93 and 47 of those stay bit-identical - it fires constantly and almost never changes an answer, which is what it should do. Solver chatter falls from 919 lines across three datasets to none. The cap fires on 4 of 93, none of them in the in-house or private arms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 20:31:20 +02:00
leonarski_fandClaude Opus 5	10873eee7a	lattice search: reject an impossible character, and supplement every angle ... Two defects in the Bravais character walk, both of which silently cost symmetry. An impossible character MATCHED. The walk computes acos(cond_F/sqrt(A*B)); when the character is geometrically impossible the argument leaves [-1,1], acos returns NaN, and the acceptance test fabs(NaN - actual) > tol is FALSE - so the character is taken. One corpus crystal matched a monoclinic-C character whose implied cos(gamma) is 1.086. The type-boundary retry covered beta only. All three angles carry the Niggli boundary, and negating two basis vectors supplements the OTHER two, so each angle needs its own flip. A crystal whose reduced gamma sits at 89.900 degrees needs the alpha/beta flip and never got it. Measured on 84369 exact lattices spanning all 14 Bravais classes: nothing is lost in any class, and orthorhombic-I recovery rises from 60.3% to 87.2%. Over 440000 random and perturbed cells the first change only ever demotes a monoclinic-C match to triclinic and the second only ever promotes out of triclinic - it is a strict superset of the beta-only retry. On the 94-dataset corpus, two crystals gain their correct point group and none regresses. Both regression tests fail without the change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 19:57:41 +02:00
leonarski_fandClaude Opus 5	1ea6ba25be	reader: open a master that states its goniometer axes in one place and their angles in another ... A hybrid master carries an NXmx /entry/sample/transformations group holding one EMPTY SUBGROUP per axis - the direction as a vector attribute, no NX_class, no units, no angles - beside a legacy /entry/sample/goniometer group holding all the angles. Opening it failed outright with "Cannot open HDF5 dataset /entry/sample/transformations/omega": the existing legacy fallback keys off Exists("/entry/sample/transformations"), which is true here, so it never fired, and the axis stub was then opened as if it were the angle dataset. Present is not the same as usable. GoniometerGroup now takes transformations only if it holds at least one DATASET, and ReadAxis asks IsDataSet rather than Exists, so a member that is not a dataset can no longer be read as one. HDF5Object gains that predicate, in the style of the neighbouring Exists. The stub is the load-bearing half, not merely the thing that crashed. In the legacy branch, before falling back to the assumed (-1,0,0), the reader now looks for the NXmx stub and takes its stated vector. With it the axis is (0,-1,0) and the run indexes 60/60 validation frames; with the stub deleted the assumption applies and the same file indexes 0/60 on both schemes and both signs, and the run stops with no lattice. So without this half the fix would have turned "cannot open" into "found no lattice" - a differently shaped failure, not a success. The direction stated here is 90 degrees from the assumption, not merely its negation, which the rotation first pass could have recovered on its own. The vector size check moved out of the first branch so it now covers every path that produces one. Verified: the file processes end to end, 100% of frames indexed, cubic cell 105.87 against a deposited 105.88 (0.009%), space group reported as I23 or I213 - correctly refusing to choose, since the reflections that separate them are extinguished by the I-centring and were never measured. A both-layout master reprocesses unchanged (100% indexed, P212121). [HDF5] passes: 2194 assertions in 91 cases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 15:47:41 +02:00
leonarski_fandClaude Opus 5	6319d5c600	indexing: tell an indexer that failed apart from one that found nothing ... IndexerThreadPool collapsed two different outcomes into the same empty reply. A worker that threw set result = nullptr, and so did a dispatch that never found a free worker; both then became a default-constructed IndexerResult, indistinguishable downstream from an indexer that ran and found no lattice. The first says nothing about the frame at all - the indexer never looked, and it will fail again on the next one - while the second is a real negative result about the crystal. The visible cost was the advice a failed run gave. With the card full, rugnux printed "Indexer thread 0 failed: CUDA (GPU) error" and then ended with "Two-pass rotation indexing found no lattice. Check the beam centre (--beam-x / --beam-y), raise --max-spots ..." - sending the operator to look at geometry that was never wrong, for a machine that was simply out of memory. None of those remedies can help when the frames were not examined. IndexerResult gains an optional error, set by the worker and by the pool's own catch; the remaining nullptr path keeps its meaning of "not attempted" and deliberately carries no error. RotationIndexer records it and exposes GetIndexerError(), and rugnux's first pass branches on it at the throw site, naming the resource failure instead. The error travels as DATA through image_analysis/ rather than as an exception, because RotationIndexer::RunIndexing() is on the online path - IndexAndRefine calls it on a schedule from the broker and the receiver, where dropping a frame is the right failure and killing a live acquisition is not. Only rugnux, which owns the "this run is over" decision, turns it into one. The failure result is byte-identical to the default-constructed one it replaces, so any_executed and the online frame-drop behaviour are unchanged. The new IndexerError category exists because the category is only the display prefix on what() - Category() is read nowhere - and the old line read "Processing failed: Input parameter invalid" for a GPU fault, which is the same defect one layer up. SpotFinderError is the precedent. msg.indexing_result is deliberately left alone: of its three states, "not attempted" is the honest one for a frame the indexer never examined, and asserting false would be the same collapse again. Verified on a squeezed card (246 MB free, -N 4): exit 1, zero dropped frames, the new message, and no mention of --beam-x. On a free card, against the previous binary at -N 6 on the same input, the logs differ only in paths and timing - same space group, cell and merge statistics. Targeted Catch2 cases pass, including three that drive the pool through the online path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 15:30:28 +02:00
leonarski_fandClaude Opus 5	92ad304f15	rugnux: fail on an exhausted GPU instead of dropping the images it could not process ... A per-image worker caught every std::exception and continued to the next image. That is right for one undecodable image and wrong for a resource fault: a CUDA out-of-memory says nothing about the image that happened to be in flight and everything about the machine, and it recurs on the next one. The run therefore logged an error per image, skipped each of them, and exited 0 with a dataset that was silently short - the merged numbers all moved, and nothing in the exit code or the summary said so. Observed with several processes sharing one card: two arms of the same comparison lost 21 and 15 images, by different amounts, so the arms no longer saw the same data. IsFatalResourceError rethrows for std::bad_alloc and for a JFJochException of category GPUCUDAError or MemAllocFailed, and every per-image catch consults it first. There are ten of them, not the three the obvious grep finds: besides the analyze/integrate/load sites in the two image loops, the first-pass spot read swallows the same way at Warning severity, the pre-scan swallows it in three places, and one geometry-refinement site caught it with no log line at all. The first-pass one matters most - those frames build the reciprocal-space cloud the indexer runs on, so losing them thins the input that decides whether the crystal indexes, and it does not change IMAGES_PROCESSED, so a frame-count audit cannot see it. Every one of these loops joins with future::get() or ParallelFor's RunTasks, both of which hold the first exception and rethrow it at the join, so the throw propagates and nothing terminates. JFJochException gained a Category() accessor for the test. Verified by squeezing the card to 804 MB free and running at -N 4: the run now exits 1 with zero dropped-image lines, where the same test on the previous binary produced the silent skips above. The broker and receiver paths are deliberately untouched - dropping a frame is the better failure there than killing a live acquisition. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW	2026-08-29 14:58:04 +02:00