1212 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	0e245d28cf	docs: name the in-house SLS 2.0 data beside the public sets ... The acknowledgement read as though the whole test corpus came from other people's beamlines. It does not - in-house data collected at SLS 2.0 is tested alongside it. One clause, at the top of the section; everything that follows still concerns the public data, which is the part that carries an obligation to cite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:58:40 +02:00
leonarski_fandClaude Opus 5	04d823135e	rugnux: record what the two-pass completeness arm can decide, and what three stand-ins could not ... Only under -S does the arm read a completeness with headroom. De novo the search merge is in P1 and does not count its possible reflections, so there is no number and CC1/2 is the only live term. Three ways to give the arm one back were measured on 28 rotation crystals and none lands, so say so at the guard rather than leave the next reader to rediscover them: * Count the possible reflections on the search merge too. It is affordable - 5.2 ms against a 38 s run - but a healthy crystal reads 27-43% of the P1 hemisphere, so the 100.5% bound is never approached and no decision changes. It would only add a figure to the user's report that reads as the dataset's completeness while being a fraction of a different group's asymmetric unit. * Observation count. The two search merges are built in the same terms, so pass 2 retaining under 90% of pass 1's observations looks like the signal the completeness ratio stood in for. It never fires - bit-identical on all 28, including the crystal whose pass 2 predicted 44% fewer partials. Predicted partials and observations that survive into the merge are not the same population. * Completeness ratio. Same idea one level up, and it does fire - on the wrong crystals. This merge is deliberately never resolution-cut, so the possible list grows with whatever range the pass reached and a pass that predicts FINER scores as one that lost the sweep. At a 0.90 bound it reverted two crystals indexing at 100%, taking one of them from 0.07% to 0.29% cell deviation and ISa 19.5 to 17.1, and rescued nothing. Bringing the wrong-cell detector back for de-novo data means putting it on the final merge or retiring the arm and saying so; neither is done here. No test covers the decision line: Rugnux_Rotation does not reach this guard - the two passes in its log are the indexer's, not RunAllPasses' - and there is no fixture for building a RotationScaleMerge, so an assertion placed there would never execute. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:48:39 +02:00
leonarski_fandClaude Opus 5	8935a59721	rugnux: the two-pass guard says when it has no completeness, instead of printing 0.0% ... The guard that judges the post-refined pass against the header-geometry pass reads completeness off each pass's space-group search merge. That merge does not count possible reflections, so the quotient is 0.0 on both sides and every de-novo rotation report has been printing completeness 0.0% vs 0.0%, CC1/2 before corrections 0.994 vs 0.993 as though a comparison had happened. On the 28-dataset in-house battery that is all 28 runs. The number is not just uninformative, it is a test that did not run: "completeness above 100% means the cell is wrong" is the guard's wrong-cell arm, and it cannot fire against a constant zero, so only the CC1/2 arm decides - which is computed on whatever data the pass kept, so a pass that discards a large part of the sweep can post an equal CC1/2 and win. Carry a measured flag beside the number. The arm is skipped and the log and PASS_DECISION say "completeness not measured" where there is nothing to read. No behaviour changes: the arm could not fire before either. The next commit gives the merge the count so it can. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 09:48:25 +02:00
leonarski_fandClaude Opus 5	5ce7eb3014	docs: one changelog line for the geometry reporting work ... Three lines were appended over the course of the work - the tilt and direct beam in the report, the measured tilt beside them, and the post-refine beam bound. They are one change to a user: the report now describes one geometry and says which point is which, and the bound that governs it is measured against the run's own measurement rather than against the file it is meant to correct. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-01 07:39:23 +02:00
leonarski_fandClaude Opus 5	f98442077e	post-refine: bound the beam move on the measurement, not on the header ... PostRefine step B refuses a fitted beam centre more than 15 px from the geometry the run started from. Two things were wrong with that and they compounded. The 15 px was BOTH the Ceres search box and the acceptance threshold, so a fit that wanted to move further was pinned at the box face, landed at exactly hypot >= 15.0, and was then refused for being there. The gate never saw the fit it was judging. Measured on one crystal at three detector distances, the fit came back at 15.128, 15.133 and 15.130 px - the box corner, three times. And it was measured from the header, which is the value most worth correcting exactly when it is most wrong. The run already knows better: the default beam-centre check fits the centre from the isotropy of the scattered background on every rotation run, reported it, and then discarded it. The bound is now measured from whichever of the nominal centre and that measurement is nearer - accept a move that is small relative to something independent, rather than small relative to the header alone - and the search box is widened to match, so the gate decides rather than the box. The measurement is used only where it was precise enough to be adopted as a centre in its own right; a run without one is bounded exactly as before. Step B also now says WHICH of its four tests refused a fit, since all four leave the same geometry behind. Measured over 113 rotation datasets against the same corpus, one binary per arm, with a control arm that reproduced the stored battery on 112 of 112 reports: the bound fired and refused a real correction 1 6ukf the bound fired and caught a runaway 0 the bound never fired 112 On 6ukf the header centre is 22.1 px from what the background fit measures. The old fit, pinned at the box, improved the held-out positional residual 9-fold and was refused; the new fit lands 0.46 px from the measurement and improves it 418-fold. ISa 4.42 -> 6.71, R_meas 18.5 -> 12.8 %, CC1/2 0.987 -> 0.995, d_min 1.119 -> 0.973 A; at matched resolution, 1.68 A goes I/sigma 8.9 -> 15.1 and R_meas 28.8 -> 13.4 %. The refined cell moves from 0.4 % off the deposited one to under 0.1 %. It survives perturbation, and inverts it: over +-0.5 mm of detector distance the corrected run is flat (ISa 6.62 / 6.71 / 6.94) while the uncorrected one is bistable (4.02 / 4.42 / 6.90). The change removes an instability rather than exploiting one. No space group anywhere in the corpus moves, and no other dataset's report changes at all - 112 of 113 are identical. The distance half of the gate is untouched: it is separately argued, and it is what legitimately refuses the one dataset whose distance wants to move 1.05 %. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 20:19:36 +02:00
leonarski_fandClaude Opus 5	efca844bce	calibration: --no-refine-tilt holds the header tilt, it does not zero it ... GuessInitialGeometry resets rot1/rot2/rot3 to zero before seeding, and the branch that puts the header's tilt back was guarded by `refine_tilt && !tilt_refined`. With --no-refine-tilt the tilt is never free, so tilt_refined is false, so the guard is false, so the restore never ran - and the fit reported a zero tilt while the CLI printed "held at the header value". The guard only needs !tilt_refined: a tilt that was declined and a tilt nobody asked to refine both want the header put back and the beam centre and distance refitted around it. That is what the branch already does. Measured on a LaB6 sweep with a tilt imposed on the command line and asked to be held, --calibration spots: before Rot1= +0.0000 deg (+0.001400 rad from the header) after Rot1= -0.0802 deg (+0.000000 rad from the header) The rings path reaches this only when the spot-derived start wins, which is why it does not show on a file whose profile start is chosen; the spots path always did. A calibration whose header tilt is already zero is unaffected - checked, byte-equal distance either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 19:47:02 +02:00
leonarski_fandClaude Opus 5	dcfcff5bc4	report: the measured tilt, and one geometry instead of two ... Adds REFINED_DETECTOR_TILT - rot1/rot2 in degrees as rotation indexing measured them - so the value can be checked against a powder calibration, which measures the tilt from ring shape and is the better instrument for it. rot3 is not carried: a rotation about the beam is an exact null of a single-axis rotation experiment, so the fit cannot move it and printing it would suggest otherwise. Also repairs two defects in the commit that added DETECTOR_TILT, found in review. The geometry was taken from the caller's copy of the experiment, which is the state BEFORE the run, while BEAM_CENTRE and DETECTOR_DISTANCE come from result.used_*, which is the state after - so on a rotation two-pass the report mixed a post-refined centre with the file's rotations and DIRECT_BEAM was off by the whole post-refinement shift. JFJOCH_DATASET_SETTINGS carried the same mixture, which is a geometry no pass ever ran at. The tilt and the direct beam are now taken from experiment_ alongside the other three, so the block describes one geometry. And the prose claimed the tilt "is not refined here". It is: rotation indexing refines rot1/rot2 on every rotation run. What is true is that the result is never written back onto the geometry, which is why it needs a key of its own. M_PI -> PI (common/JFJochMath.h). rugnux is in the forced-Windows viewer-only set and M_PI is not portable to MSVC; the file did not include <cmath> either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 19:35:27 +02:00
leonarski_fandClaude Opus 5	7c00731992	docs: publish the powder calibration schemas in the python client docs ... docs/python_client is tracked and is what readthedocs publishes, but it is generated - update_version.sh copies it out of python-client after regenerating the client - so adding schemas to the spec left it four files behind. The README's model list did not name them and their pages did not exist. Copied the way update_version.sh copies them. The four new pages and four lines in the README, nothing else: the generator's README is built from the spec, so the stale Calibration* pages it had left on disk under the old names are absent from it and were removed before copying rather than published alongside their replacements. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 19:02:12 +02:00
leonarski_fandClaude Opus 5	bd6e933f04	api: name the calibration schemas powder_, not calibration_ ... "Calibration" already means something else in this broker: the JUNGFRAU pedestal and gain measurement, which owns /statistics/calibration, calibration_statistics and JFJochState::Calibration. Four schemas called calibration_* would have sat beside it meaning a completely different procedure, and the confusion is cheapest to remove now, before an endpoint returns them. calibration_output, calibration_quality, calibration_fit_sigma and calibration_spot_check become powder_calibration_output, powder_calibration_quality, powder_calibration_fit_sigma and powder_calibration_spot_check. All four, not only the outer one - the collision is in the word, and half a rename would read as though the inner three belonged to the other kind of calibration. Rename only. Every client regenerated from the spec: the C++ server model, the TypeScript frontend client, redoc-static.html, and the python client, which is gitignored but was checked to still decode the file rugnux writes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:58:24 +02:00
leonarski_fandClaude Opus 5	6fe3f30ab4	api: define the powder calibration result as calibration_output ... The JSON a calibration writes was a shape invented at its writer, described only by the comments around it. That is enough for a file somebody reads with jq and not enough for anything else: a client cannot type it, and an endpoint returning it later would have to declare the shape a second time and keep the two in step by hand. So declare it where every other shape in this system is declared. calibration_output holds dataset_settings and a calibration member; the latter is calibration_quality, which nests calibration_fit_sigma and calibration_spot_check. The descriptions carry what a reader has to know to use the numbers rather than only what they are named - that beam_x_pxl is the PONI and the direct beam is elsewhere, that the rotations travel together because a body omitting them states a flat detector, that a tilt below about three sigma was declined and pinned, and that the two correlations approach 1 as the tilt stops being separable from the beam centre. Nothing references it yet. It is declared now because /powder_calibration will return exactly this, and because the file rugnux already writes is decodable today: jfjoch_client's CalibrationOutput.from_dict reads it as it stands, with o.calibration.fit_sigma.correlation_beam_x_rot1 and the rest typed. Generated clients regenerated from the spec, as the spec requires: the C++ server model (four new pairs under broker/gen/model), the TypeScript frontend client, and broker/redoc-static.html. Both regenerations are purely additive - no existing generated file changed except to export the new names. The python client regenerates from the same spec and is gitignored. The test now validates the WHOLE file against the generated Calibration_output rather than only its geometry member against Dataset_settings, so the quality block is under the same contract: a field renamed or newly required in jfjoch_api.yaml fails here rather than at a client. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:54:09 +02:00
leonarski_fandClaude Opus 5	8691bf4876	calibration: write the result as JSON, not only as a PONI file ... A calibration run produced one file, and it was a pyFAI PONI - which pyFAI and its neighbours read and nothing in this system does. Carrying the answer back into the instrument meant a person reading numbers off a printed report and retyping them into a dataset_settings body, and the report is where the two points a "beam centre" can mean are easiest to confuse. So write <prefix>.json beside it. Its "dataset_settings" member holds the geometry under the property names broker/jfjoch_api.yaml gives them and holds nothing else, so it is a valid dataset_settings body as it stands: curl -X POST -H 'Content-Type: application/json' \ -d "$(jq -c .dataset_settings det.json)" http://broker:5232/start beam_x_pxl is the PONI, as everywhere here. The three poni_rot*_rad ride along whenever any is non-zero and are left out when all are zero: a body without them does not leave the tilt unstated, it states a FLAT detector, so they travel together or not at all - the same rule the report's JFJOCH_DATASET_SETTINGS block already follows. The "calibration" member holds what the run knows about that geometry: the residual, the fit's own sigmas and the correlation between the tilt and the beam centre, whether the tilt cleared its significance test or was declined and pinned, where the direct beam lands, and where the spots independently put the beam. A calibration that has gone wrong looks exactly like one that has not until those are read, and a machine-readable file that carried only the geometry would be the easiest possible way to feed a bad one into an instrument. Tested against the model generated from the spec rather than against a list of field names written out by hand: the file's dataset_settings member is parsed into org::openapitools::server::model::Dataset_settings and validated, so a field renamed or newly required in jfjoch_api.yaml fails here rather than at someone's POST. The tilted and untilted branches are both covered, and the PONI/direct-beam distinction is asserted rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:47:51 +02:00
leonarski_fandClaude Opus 5	16b368739e	calibration: re-integrate the images about the geometry that was fitted ... Everything the ring fit reads was binned with the geometry the run started from, and a calibration is run precisely because that geometry is in doubt. Binning is not something a later fit can undo: which pixel landed in which (q, phi) bin was decided when the images were read. Get the distance wrong and the radial sampling is the wrong scale - a run recovering 110 mm from a 250 mm header ended at rms 1.07 px where the same data from a right header gives 0.42. Get the centre wrong and every ring is smeared across its own sectors, and past about a hundred pixels the fit leaves rms 4.8 px even when it is started from the exact answer, because there is nothing left in the profile to fit. So integrate the images a second time, binned about what was fitted, and fit that. A calibration run is a handful of images and the answer is worth far more than the extra read. Re-binning about the FIRST fit is not enough on its own. Where the beam centre was badly out that fit is itself wrong, and binning about it digs deeper - rms 3.75 -> 5.66 on an exposure 200 px off. The spots' geometry is the one that does not degrade there, having never read the header, so both are tried where they differ and whichever comes back better is kept. On that exposure the first candidate gives 288 points at rms 5.66 and the second 369 at 0.81. The second pass is taken only if it is actually better, by the same rule that ranks everything else here - at least half as many ring points and a smaller residual - and it is skipped altogether when the fitted geometry moves a ring by less than the radial width of one profile bin, since re-binning would then put every pixel back where it already is. On the 110 mm exposure with its own header that is 1.77 px against a 1.84 px bin, so the run is untouched and its answer bit-identical. Measured on the 110 mm LaB6 exposure, true PONI x 765.90 at 110.03 mm. The calibration is now independent of the header it was given: beam-x out by +20 +40 +100 +200 +400 px -> 765.9-766.6, 110.02-110.05 mm 250 mm header, beam-x 780 -> 765.998 at 110.037, rms 0.394 250 mm header, beam-x 867 -> 765.723 at 110.038, rms 0.413 900 mm header, beam-x 1167 -> 766.496 at 110.038, rms 0.804 30 mm header, beam-x 967 -> 766.486 at 110.039, rms 0.807 All of those failed before this, most of them catastrophically. The residual inflation is gone with them: the 250 mm case now leaves 0.394 px, better than the same data from its own correct header. The four other datasets are unchanged where the re-bin fires, and slightly better where it does: rms 0.433 -> 0.423 at 150 mm and 0.555 -> 0.550 at 200 mm, with the geometry moving under a tenth of a pixel. A run that needs the second pass costs about twice one that does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 18:17:17 +02:00
leonarski_fandClaude Opus 5	49dd01331b	calibration: let the spots vote on the geometry in ring mode too ... The rings and the spots fail in different regimes, which is the whole reason to carry both. The header is right on a well-configured instrument and is the thing a calibration is run to check. The profile's own estimates are exact while the error stays small and stop meaning anything beyond that - past about half a ring spacing each ring reaches the azimuthally averaged profile as two horns rather than one peak, and the distance search reads a list of horns as a list of rings. The circle through the spots reads nothing from the header at all: measured on a 110 mm LaB6 exposure, --calibration spots returns the same geometry from a header 400 px out in the centre AND eight times out in distance. So ring mode now finds spots as well - half a second - and offers what they make of the geometry as one more starting hypothesis, fitted like the others with the residual left to choose. It is added whole rather than as a centre alone: GuessGeometry votes for the circle centre, clusters the radii into rings and takes the distance from the innermost, and those two belong together. Taking only its centre would not have helped, because the distance candidates come from a profile averaged about the header's centre, and a ring smeared over hundreds of pixels cannot be un-smeared by reading its bins differently. Measured on the 110 mm exposure, whose true PONI is 765.90 at 110.03 mm. A header 40 px out in the centre now lands within 0.6 px (it landed 41 px away before). The cases with BOTH wrong, which failed before this and equally before the beam centre work, now come out: 250 mm / 780 px gives 110.041 mm and 766.10 px against 42.3 mm and 782.7; 250 mm / 867 px gives 110.065 and 765.48. All five datasets are unchanged from their correct headers and the distance still recovers from any header between 25 and 1200 mm. Past about a hundred pixels nothing rescues ring mode, and the reason is the profile rather than the seeding: binned about a centre that far out it shows each ring smeared across its own sectors, so even started from the exact answer the fit leaves rms 4.8 px and drifts. Re-binning would fix it and would need the images read a second time; --calibration spots, which never touches the profile, already covers it. That regime is now visible rather than silent. The spots' beam centre is printed beside the fitted one as a cross-check - two methods sharing no assumption, so a reader can see at a glance whether they agree. It costs nothing, the spots having been found already, and it separates cleanly: 0.3-0.4 px on the good runs against 201.4 px on the exposure whose profile could not be fitted at all. Reported as a fact and not gated on, since at long distance both methods weaken together and the honest thing is to show the number (5.2 px at 300 mm, 12.6 at 500). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:58:26 +02:00
leonarski_fandClaude Opus 5	c84b91be8a	calibration: take the beam centre from the rings too ... The header's beam centre was the last input the ring fit had to be roughly right about. Each ring is looked for in a window a few pixels of radius wide, and a centre wrong by (dx, dy) puts a ring at a different q in every sector, so past about ten pixels the ring leaves that window over much of the turn - and the fit then reads its cos(phi) signal off whichever sectors are left, which are the ones where the signal is weakest. A 20 px error ended 31 px wrong. The rings answer this without a calibrant and without a distance. A powder ring is a conic centred on the beam, so a wrong centre makes EVERY ring's radius oscillate once per turn by the same amount: r(phi) = R + dx cos(phi) + dy sin(phi), solved directly and pooled over every ring the profile shows, with each ring searched about its own measured radius rather than about where a standard says it should be. Using it needs the extraction to follow the rings sector by sector, which is what ProfileRingTrack now does - exactly, and in all five parameters at once, by walking the ring in the geometry believed true and asking the binned geometry what q and azimuth it would have given each point. That replaces the flat-detector distance correction it grew out of. Following the rings is not free, and the reason is worth stating: a window that moves with phi makes every systematic of the peak finder - where the background line is taken, how the centroid sits in the window - vary with phi as well, and phi is exactly the axis the beam centre is read off. Measured, it costs rms 0.415 -> 0.525 px on a good 110 mm fit, and 0.831 when the window follows the fitted tilt too. So a second measurement is taken with a window that is the same in every sector - the binned geometry with only its DISTANCE replaced, which is phi-independent by construction - and both are offered to the same rule that ranks everything else here. Acquire by following, measure by holding still. The seeded centre is likewise a hypothesis and not a belief. It reads a once-per-turn wobble, and a tilt puts a term of that shape there too - one that grows as the radius squared, where a centre error does not - so pooling the rings absorbs part of the tilt into the centre. Believed outright it made a good 110 mm fit worse; offered as an alternative start it costs one more fit and needs no rule about when it applies. It is skipped entirely below a pixel, where it is not a different hypothesis at all, which keeps a well-headed run at 0.71 s. Measured on the 110 mm LaB6 exposure, whose true PONI is 765.90: a header centre 20 px out now lands within 0.5 px, where before it landed 31 px away. All five datasets are unchanged from their correct headers, and the distance still recovers from any header between 25 and 1200 mm. The limit is now understood rather than merely reached. Past a few pixels the azimuthally averaged profile stops showing rings: a ring tracing r(phi) piles up density where that turns round, so it averages into the two HORNS of the sinusoid, at R-|d| and R+|d|. The radius finder reports two rings where there is one, and the gap between them is 2|d| - the search window shrinks to exactly the offset it was meant to span. That caps recovery at roughly half the ring spacing, about 20 px here and failing by 40. Beyond it nothing is left in an azimuthally binned profile, and --calibration spots, which works from the spot positions themselves, is the method that still can. One pre-existing limit measured and NOT introduced here: a wrong distance together with a centre more than about 5 px out fails, because the centre error splits the radius list the distance search reads. The committed code before this change fails identically on those cases. Also fixed: fit_from now takes a whole geometry rather than a distance, and the declined-tilt refit was inheriting rot1/rot2 from it - pinning the tilt at exactly the unvalidated value the gate had just rejected. Same fault the gate exists to catch, one level up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:38:36 +02:00
leonarski_fandClaude Opus 5	9267bd67de	report: the detector tilt, and the two points the beam centre is not ... BEAM_CENTRE is the PONI - the foot of the perpendicular from the sample - so on a tilted detector it is not where the direct beam lands. The report printed only that one number and never mentioned the tilt at all, so the two points were indistinguishable to a reader. On a detector at 85 mm with a 0.22 degree tilt they are 8.3 px apart. Print DETECTOR_TILT (rot1/rot2/rot3, degrees, as the run used them - nothing is refined here) and DIRECT_BEAM beside BEAM_CENTRE, and say which is which. JFJOCH_DATASET_SETTINGS was the more serious half: it exists to carry a geometry back into the instrument, and dataset_settings carries poni_rot1/2/3_rad, but the block omitted them - so it described a FLAT detector, silently, on every tilted setup. The rotations now ride with it when they are non-zero; they stay out when they are zero, because the API's own default is 0.0 and the shorter block means the same thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 17:35:03 +02:00
leonarski_fandClaude Opus 5	01ad1e1743	calibration: only report a tilt the fit actually measured ... The detector tilt is refined by default, and on a pattern that cannot separate it from the beam centre the fit returns one anyway - there was nothing to stop it. Both displace a ring's radius as cos(phi), and only how that amplitude grows with the ring's radius tells them apart, which takes two well-sampled rings. At 500 mm on the LaB6 series only two rings reach the detector and the outer one is barely there: the tilt came out at the opposite sign to every shorter distance, dragged the PONI 28 px, and bought a residual of 0.960 px against 0.962 pinned. The covariance says so plainly - 0.1 sigma, and a beam centre quoted to +-180 px. So ask it. A tilt is kept only where the fit had it free AND it stands at least three times its own uncertainty; otherwise rot1/rot2 go back to the header's values and the beam centre and distance are refitted around them. Over the series the tilt stands at 50, 33, 15 and 8 sigma at 110 to 300 mm and 0.1 at 500 mm, so any threshold between 2 and 5 gives the same verdict on all five - this says which regime a fit is in, not where a line was drawn. The declined 500 mm fit lands on a direct beam of 773.53 px, against 773.56 for the pinned fit measured independently. It is a rejection criterion and nothing more. Clearing it does not certify a tilt: that estimator is limited by systematics rather than by this sigma, and a coherent half-pixel error in the ring positions fakes a tilt of the usual size while leaving sigma small. The report says "refined", never "verified". Writing the gate turned up a related fault in the pass loop. RingOptimizer pins the tilt by itself when every point it is given lies on one ring, and on a barely-sampled pattern a later pass lands in exactly that state - which froze the tilt at whatever the FIRST pass had produced and returned it with sigma zero, an unmeasured tilt wearing the appearance of a fixed one. The gate reads that as "not measured" and refits pinned, which is why it is stated over the geometry that gets reported rather than over what the last fit happened to do. Both paths are covered, profile and spots; the spots path was reporting a refined tilt as declined for the same reason. Two things measured and NOT taken: A robust loss. A Cauchy loss scaled to the previous pass's median residual changed nothing on the series - rms 0.415 to 0.421 at 110 mm, no case improved, every direct beam within 0.06 px. Ring points are per-sector peaks that already had to stand 3 sigma clear of their own background, so there are no gross outliers left to reject. Recorded at the call site rather than left as an unused option. A quality gate that refuses a bad calibration. Three candidate signals, all measured against naming the wrong standard on LaB6 data: sigma(PONI) does not see it at all (0.52-0.65 px, indistinguishable from healthy); the residual only half sees it (3.2-3.6 px wrong against 0.4-1.0 right, but a correct run from a wrong header sits at 1.0-2.4 and would be caught too); and the seed's match score is dominated by how many rings the calibrant lists, scoring 0.29 for a perfect LaB6 fit against 0.21 for a wrongly named silicon. None of the three separates, so no gate is shipped. What the run does say is the recovered distance against the header, and a wrong standard moves that to 446 mm on a 110 mm exposure - unmissable, and the operator's call rather than a threshold's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 17:07:37 +02:00
leonarski_fandClaude Opus 5	a5f416fcdc	calibration: take the detector distance from the rings, not from the header ... A powder calibration is run because nobody is sure the header is right, and the header's distance was the one number the fit could not survive being wrong about. The ring search is local - each ring is looked for inside a window a few pixels of radius wide - so a distance more than a percent or two out puts every ring outside its own window, and the fit then converges on whatever background fluctuation each window contains. It does not fail: a 110 mm exposure told the detector was at 150 mm reported 149.8 mm, with 146 ring points and exit 0. Only its residual said anything, 5.3 px against 0.4 px, and nothing read it. Measure the distance from the rings instead. The peaks of the azimuthally averaged profile give ring RADII, and a radius does not depend on the assumed distance at all - bin i holds the pixels at one particular radius whatever q that radius was called - so the radii are a property of the image. Against the calibrant's d-spacings, r = D tan(2 asin(lambda/2d)) then has one unknown. It is scanned rather than solved because the pairing of observed rings to d-spacings is unknown too, and the winning basin is solved in closed form. Nothing here reads the header distance except to bin the profile; it needs only the wavelength, the pixel size and the detector's extent. A powder pattern has genuine distance aliases, so one answer is not enough. A cubic primitive standard puts its rings at radii proportional to sqrt(N), and scaling the distance by sqrt(2) maps ring N onto ring 2N - most of the comb still lands on peaks. Measured: the 110 mm exposure with a 115 mm header scored its best at 156.5 mm, which is 110*sqrt(2). No adjustment of the score removes an alias the lattice really has, so the scan hands back the few best distances and each is fitted, the header among them as one hypothesis of several. The residual then separates them - 0.4 px against 5.2 px on that case - subject to an attempt explaining a comparable share of the pattern first, because a start so wrong that one ring point survives leaves a residual of exactly zero. Each attempt re-extracts at the geometry it converged to and fits again. The seed is measured from blended peaks and is good to about a per cent, close enough to converge from but far enough to sit every search window a few pixels off its ring, and an off-centre window takes its background off the ring's own flank. Nothing is re-read from disk, so the loop is free. Measured on the LaB6 distance series. A 110 mm dataset now recovers 110.03-110.17 mm from any header between 25 and 1200 mm, against +-2 mm before. All five datasets recover their own distance from a fixed wrong 250 mm header. With correct headers, four of the five are bit-identical to before and the 500 mm one moves by a single ring point - the two-ring fit whose tilt is 0.1 sigma anyway. Run time is unchanged at 0.62 s. The residual is larger on a run whose header was wrong (1.1 px against 0.4 px on the 110 mm case), because the profile was still binned at the wrong distance and its radial sampling is correspondingly coarse. The geometry is right; only the scatter about it is inflated. Re-running with the recovered distance recovers the residual too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:48:32 +02:00
leonarski_fandClaude Opus 5	d275bbcae7	calibration: -C overrides the calibrant, with absences from -S ... The five named standards are a convenience, not the limit of what a powder calibration can be run against. A unit cell given with -C now IS the standard in --mode calibration, and its rings are enumerated from that cell. A cell alone does not give a ring list, though: the centring and any glide decide which hkl the lattice actually diffracts into, and the fit pairs the innermost OBSERVED ring with the innermost LISTED one - so a list opening with a reflection that is not there scales the whole calibration by the ratio between them. Where -S is given, the absences come from the space group itself via gemmi, which covers centring, glides and screws in one mechanism rather than the three hand-written conditions the built-in table uses. The two agree exactly on LaB6 and CeO2, which is the cross-check that says the gemmi route is safe to hand a user's cell. They do NOT agree on silicon, and the test now pins that: Fd-3m's symmetry absences are only the F centring, while silicon's 222 and its relatives are extinguished by its two-atom basis - a structure-factor absence, not a symmetry one, so no symmetry handler can know it. gemmi offers 24 rings where the diamond condition gives 18. The extra rings do not move the first one, so the distance is not scaled, but they are rings with no intensity offered to the matcher - which is why --calibrant si stays, and why the usage text says -S supplies symmetry absences only. Without -S the cell is taken as primitive and the log says "assumed primitive". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:23:37 +02:00
leonarski_fandClaude Opus 5	5a80d2df53	calibration: fix four ways the powder fit quietly loses its input ... None of these changes the answer on a well-separated cubic standard - the LaB6 distance series is bit-identical by both methods - but each one is a case where input is dropped or mis-assigned without saying so. The circumcentre vote grid was a fixed 4000x4000 box, and the caller never passed anything else. That allocated 128 MB whatever the detector, and on a detector larger than 4000 px in either direction it put the beam centre outside the grid, so every vote was discarded and the guess failed with "Beam center not found". Span the spots' own bounding box instead: a powder ring encloses its centre, so that is where the answer has to be. uint32 votes while there - the most any bin can take is C(500,3). Spots were assigned to the FIRST calibrant ring within a fixed 0.1 1/A, not the nearest. Silver behenate's orders sit 0.108 1/A apart and hexagonal ice has three rings inside 0.06, so for those two standards the window reaches the neighbour and every point lands on the lower-q ring of the pair, biasing the distance. Take the nearest ring, and clamp the window to half the gap to the neighbour - which is what the profile path already did inline, now shared as RingMatchWindow and covered by a test that checks it actually narrows on the crowded standards and not on LaB6. A profile bin no pixel fell in is NaN. SectorPeakQ dropped such a sector by accident, through NaN comparisons falling false; check the four background bins and return explicitly. Ice-ring handling is switched off in calibration mode. Flagged spots are sorted last by the spot budget and so discarded first, which for --calibrant ice throws away exactly what is being calibrated on. The two per-ring std::cout lines in GuessGeometry are gone: a library has no business writing to a terminal, and constructing a Logger to keep them would emit a version banner from inside a fit. What matched belongs in the result struct, which the quality gating still to come needs anyway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:18:27 +02:00
leonarski_fandClaude Opus 5	fba5435c38	calibration: report what the ring fit knows about its own answer ... The ring fit reported the SCATTER of its measurements (rms, and the beam-centre standard error that follows from it) but nothing about how well the fit pinned each parameter. Those two part company exactly where a calibration is worth doubting: as the rings run out, the tilt and the beam centre stop being separable - both displace a ring's radius as cos(phi) and only the way that amplitude scales with radius tells them apart - so the fit can sit tightly on the few points it has while being free to spend tens of pixels of beam centre on a tilt the data do not support. Take the covariance of the converged problem from Ceres and report it. Measured on a LaB6 distance series, the fitted tilt is 50 sigma at 110 mm and 0.1 sigma at 500 mm, where only two rings reach the detector; at 500 mm the fit quotes its own beam centre to +-180 px and its tilt to +-2.9 deg on a 0.35 deg value, and the correlation between them is 1.000. Nothing acts on this yet - it is printed so the next change can gate on it. Ceres returns the bare (J'J)^-1 of an unweighted problem, so it is scaled by chi2 per degree of freedom; that leaves the sigmas in pixels, mm and radians whatever unit the residual is stated in. Cost is one 5x5 SVD per run, below the noise of the surrounding I/O. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:11:18 +02:00
leonarski_fandClaude Opus 5	f34d9fe62b	calibration: move the powder fit into image_analysis/geom_refinement ... CalibrateFromProfile/CalibrateFromSpots/WritePoniFile sat in rugnux/, so the only way to reach them was to link the Rugnux library - which drags in JFJochWriter and gemmi. Nothing in them needs either: the includes are all common/ and image_analysis/geom_refinement/, next to the RingOptimizer and RingsFromProfile they call. Moving them to image_analysis/geom_refinement/PowderCalibration.{h,cpp} puts the powder fit beside the rest of the geometry refinement and makes it reachable from anything that already links JFJochImageAnalysis - the receiver and so the broker included, which is what an online geometry calibration would need. Pure move: the file contents differ from their previous form only in the include paths. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27	2026-08-31 16:09:42 +02:00
leonarski_fandClaude Opus 5	ba24946c0f	twinning: do not report a twin the L-test rules out ... The two indicators can only move one way under twinning - <|L|> down from 0.500 toward 0.375, the second moment down from 2.0 toward 1.5 - but they were combined with an OR, so a narrow intensity distribution could report a twin on its own while the L-test said the opposite. That is not a marginal disagreement. It fires on small-cell rotation data, where a twin fraction of 0.50 was reported for a crystal whose <|L|> was 0.63: a value no twin can produce, and one that points at the opposite situation - a structure whose intensities behave centric. Suppress the flag when <|L|> sits at or above its untwinned value; the two indicators are otherwise combined exactly as before. Measured over 113 stored reports: two flip, both of them wrong today, and nothing else moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 15:34:16 +02:00
leonarski_fandClaude Opus 5	71cca91ec1	rugnux: rank screw candidates without the control count ... A screw's control class is the complement of its absent class on one axial row, so the two move together: a candidate that predicts more of the row absent leaves fewer reflections to be judged against. The Beta tail AbsenceEvidence computes grows with that control count, so the stricter candidate was charged for the very reflections it correctly called extinct, and a group whose predicted-absent class is a strict superset of another's - with the extra reflections equally dead - could score LOWER than the group that explains only part of the row. Two mechanisms, one dataset each. * The control-count minimum gated the row MEAN, which is the scale the zone's evidence is stated in, on the same count as the row MEDIAN, which is a violation threshold. A candidate could therefore forfeit a whole zone by being right: on a tetragonal 4_1/4_3 wedge, thirteen 00l reflections measured at 0.1% of the two l = 4n beside them scored ZERO because only two control reflections were left, while the nine of them a 4_2 also predicts absent scored 38.7. The mean now stands on two, the median still on three. * The Beta tail is replaced, for a SCREW zone only, by its b -> infinity limit - the same statistic with the control count dropped. A p-value computed against each candidate's own null is not one scale across candidates; what is left is the likelihood ratio of the absent class against Wilson at the row's own mean, which is a sum over reflections and therefore comparable. Asymptotically it is n_absent * (log(1/ubar) - 1), so an equally dead superset can no longer score lower. Measured on a tetragonal 4_1/4_3 crystal: 29 dead 00l against 8 control read 47.7 nats where a subset of 19 of them against 18 control read 55.5. The centring statistic is untouched: its control is the whole present population, not the complement of a claim on one row, so neither mechanism applies to it. Measured. Analytic superset monotonicity, on a 176368-point grid with no calibration and no corpus: 1.29% violations -> 0.00%, exactly monotone wherever both candidates keep a control class. Over 112 stored merges (open, in-house and private arms) exactly ONE decision changes and it is a gain, a 4_2 -> 4_1/4_3 swap on a deposited 4_3; the screw column over the deposited arm goes 33/41 -> 34/41 with zero screw over-calls before and after, and the four centring outcomes are bit-identical. An eight-wedge reproducer whose 00l row a strict candidate eats goes from two mis-calls to none, end to end. The refused side moves the safe way: 103 zones the old statistic scored negative go lower, none turns positive, and the minimum evidence of an adopted screw rises 20.4 -> 23.8 against an unchanged bound of 20. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 15:33:29 +02:00
leonarski_fandClaude Opus 5	c25418319a	report: the twinning measured before the search, and the strong-direction limit ... Three additions to the report, none of which changes a number the run computes. The twinning statistics printed today are measured in the Laue class the run ADOPTED, so where the search promoted the point group they can say no more than "no twin law exists inside the class it chose" - and a twin is precisely what would have caused that promotion. The uncontaminated pair was already computed on the subgroup merge the search was given and reached only a log line; it is now printed as L_TEST_MEAN_ABS_L_BEFORE_SEARCH and SECOND_MOMENT_I_BEFORE_SEARCH, beside the post-adoption pair rather than instead of it, since the two answer different questions. ANISOTROPY_D_MIN_BEST names the finest of the three principal limits so it can be grepped, and a paragraph distinguishes the three resolutions the report now carries: what was written, where the isotropic fit crossed, and how far the crystal reaches where it reaches furthest. It is read off the fitted tensor rather than off the cut, so it can land either side of the other two - measured on one dataset at 2.01 A against a 1.81 A cut - and the text points at ANISOTROPY_D_MIN_CENSORED for when it is the edge of the data instead. A warning when CC1/2 falls and then climbs again, which means the fall-off the cut is read off does not describe the data. A climb counts only above CC1/2 0.30 and only if it exceeds 0.05: past that the number is oscillating in noise beyond any cut this run would take. It advises no re-cut - the generous cut is deliberate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 14:06:42 +02:00
leonarski_fandClaude Opus 5	6491345fea	comment: say what the refined geometry in the end message actually is ... Two things the field names hide, both verified in code and both able to mislead a reader into publishing a wrong geometry. The beam centre written here is the PONI - the foot of the perpendicular from the sample to the detector plane - so on a tilted detector it is not where the direct beam lands. That point is D*tan(rot)/pixel away and is GetDirectBeam_pxl(). And the tilt is not written only for the record, which the old comment implied by mentioning nothing but the _process.h5: IndexAndRefine puts it on outcome.experiment, which is the geometry Bragg prediction and integration then run at. It is fitted with the distance frozen and nothing checks the covariance, so the split between the PONI and the tilt is not reproducible between passes even though their sum - the direct beam - is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 14:06:03 +02:00
leonarski_fandClaude Opus 5	d189faeec7	viewer: draw grid scan cells in the proportion of the scan steps ... A grid scan with, say, a 20 um step in x and a 5 um step in y was drawn as a square grid, so the picture had nothing to do with the shape of the area that was scanned. The cells now carry that proportion. The anisotropy lives only in the view transform: JFJochImage gains a pixel_aspect_ (the drawn height of one pixel in units of its width, 1.0 for a detector image), the initial fit fits the image as if it were that much taller and then puts the factor back into the transform, and the grid view sets it from |step_y / step_x|. Scene coordinates stay one unit per cell, so the mouse mapping, the selected-image box, the pixel labels and the ROI code need no change and the uniform wheel zoom keeps the proportion. Measured on an 8x4 grid driven offscreen: m11/m22 comes out 4.000 for steps 20/5 um, 0.250 for 5/20, 12.333 for 37/3 and exactly 1.000 - the old behaviour - for a square step. Every cell centre still maps back through mapToScene to its own cell in all four cases. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hcoh6VNrmSswjfeMeDQqWP	2026-08-31 13:38:51 +02:00
leonarski_fandClaude Opus 5	756c3a22bb	docs: confirmed cells for three small-molecule sets; drop the unprocessable CCD one ... The four Diamond I19 small-molecule datasets had no reference of any kind, so a run on them could not be scored at all. Three now have one, each from a published structure with its DOI verified against Crossref; the fourth has a published space group but no numeric cell anywhere, which is recorded as such. Also removes the "obtained but not in the table" section. That dataset is stored as CCD TIFFs the reader does not support, so it is not processed here and adds nothing to a page about what the test corpus contains. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 11:02:38 +02:00
leonarski_fandClaude Opus 5	6d37b31948	report: give the resolution the CC1/2 fit reached, not only the one it was cut at ... The automatic cutoff fits the CC1/2 fall-off, takes the crossing of the target (0.30 by default) and then deliberately keeps one more shell. Only the extended limit was reported, as INCLUDE_RESOLUTION_RANGE, and that is the number a reader takes for "the resolution of this dataset" - so the report quoted a limit that is generous on purpose as if it were the measurement. The generosity itself is right and stays: a shell that is included can still be downweighted or dropped by refinement, while one that was truncated cannot be put back. What was missing is the other number. ComputeCCHalfLogisticCutoff now also returns the crossing, ApplyResolutionCutoff passes it out, and the report prints it as FITTED_RESOLUTION beside the range the reflections were written to, with a sentence saying which is which. The log line names both as well. Nothing about the cut, the merge or the written reflections changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 11:02:37 +02:00
leonarski_fandClaude Opus 5	55d9e2536f	release: 1.0.0-rc.166 ... The changelog entry for this release is six lines, not the fifty-four the development log had accumulated: what a user of rc.166 gets is native miniCBF input, masters from other facilities opening, the beam centre measured on every run, a 2theta-swung detector placed where the file says, the unmerged MTZ and a P1 merge written by default, and a substantially reworked symmetry determination. The per-change detail is in the commits. update_version.sh regenerates the three clients from broker/jfjoch_api.yaml, so this also carries the one API description that had drifted from the generated code - fft_high_resolution_A, which sizes the FFT's projection histogram and does not filter spots by resolution. Everything else in the regenerated tree is the version string. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 09:11:22 +02:00
leonarski_fandClaude Opus 5	3a30dc8ab5	rugnux: a space group that names twice the cell is a signal to halve it ... C 1 1 2 and C 1 1 2_1 are the only two monoclinic settings whose change of basis from the reference has determinant 1/2, and the reason is physical: with unique axis c the C-centring vector lies wholly in the oblique a-b plane and reduces away. Such a group is never a distinct Bravais centring - it is a primitive lattice written on twice the cell - and it wins the candidate table only because the table is scored on that doubled cell, where half the reflection positions are absent by construction. Refusing to enumerate those settings was measured and is worse: the volume-preserving alternative keeps the by-construction absences as real observations and merges no better than P1. So keep the enumeration as it is and act on the win instead: halve the cell, re-classify in the group's own system, carry the reflections across (the invented positions map to non-integral indices and are dropped, which is what the centred candidate was already doing) and search again there. Measured on one rotation crystal: C 1 1 2_1 on 68.6 73.0 170.3 90 90 90 becomes P 1 2_1 1 on 50.1 170.3 50.1 90 93.5 90, exactly the deposited setting, with every merge statistic unchanged (R_meas 0.1209, CC1/2 0.9979, d_min 1.238) and ISa 18.27 -> 18.31. Stable across +/-0.5 mm of detector distance. Eight other crystals are bit-identical, and across 112 stored battery reports exactly one adopts a volume-doubling setting. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:19:38 +02:00
leonarski_fandClaude Opus 5	5e3a7d113c	rugnux: keep a rotation-axis sign the first pass corrected ... The two-pass post-refinement snapshots the goniometer before pass 1 and restores it after, to undo the start/stride shift a sub-range run applies inside the pass. But the rotation-axis sign rescue also runs inside the pass, and adopts the opposite axis when the file's own indexes almost nothing - so the restore threw that away, handed pass 2 the sign that had already failed, and applied pass 1's fitted rotation scale to the axis it was not fitted on. It self-heals: pass 2's rescue fires again, and a genuinely wrong sign indexes almost no frames, so the run reaches the same answer having paid for one redundant first pass. But the correctness of that rests entirely on "a wrong sign always scores below half the validation frames", which nothing states and nothing tests. The restore now keeps the axis direction and reverts only the angles, which is what it was for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:41 +02:00
leonarski_fandClaude Opus 5	7c3ab72fed	reader: refuse a CBF that is not a detector image instead of misreading it ... Three ways a miniCBF opened silently wrong. A byte-offset CBF with no PILATUS header at all was accepted: Count_cutoff then defaulted to 0, SaturationLimitFromValue(0) is 1, and every pixel at or above one count was flagged saturated - the integration accept gate drops the whole reflection, so the run comes out empty for a reason nothing reports. The pixel size defaulted to 0 with no validation anywhere downstream, which collapses every resolution, every scattering vector and the beam centre in millimetres. XDS writes its correction files in exactly this shape, so this is not hypothetical. A pixel size is now required to claim the file at all, and a missing Count_cutoff leaves the saturation limit unset - falling back to the container's own overflow, which can only fail to call a pixel saturated - with a warning saying so. And the header captures are character classes, not number grammars: "[\d.eE+-]+" matches a bare "." and "(\d+)" matches a digit string too long for int64. std::stod and std::stoll answer both with a raw std:: exception, which escaped the format probe - CanRead catches JFJochException only - so merely LOOKING at a corrupt file threw out of the viewer's open path. A header field that does not parse is now reported as a malformed header. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:40 +02:00
leonarski_fandClaude Opus 5	8d719c1383	symmetry: a zone whose absences were never measurable must not outrank a dead one ... sum_u is a sum of max(0, E^2) over a zone's predicted-absent reflections, in units of their control class's mean, so it is EXACTLY zero when every one of them merged non-positive. The Beta tail then diverges and the clamp that caught it made each such reflection worth about 690 nats. That was harmless while the value only had to clear a bound of 20. Since the absence evidence became the primary sort key, summed across zones with no minimum count per zone, it has ranked the candidates - and it ranked them backwards. A zone of two absences that were never measurable scores 1378 nats where a genuine zone of six absences at 1% of its own row scores 22, so the candidate claiming a screw on an UNMEASURED row beat the one whose rows are actually dead, by sixty times. The comment on the ranking claimed an extra condition "LOSES the zone's evidence when the row is not dead"; an unmeasurable row could not lose, it won outright. No measurement places a merged intensity at exactly zero, so sum_u is floored at a thousandth of the control mean per absent reflection. The constant is an order of magnitude below the precision any real merge reaches - a thousandth of the row mean needs I/sigma ~ 1000 against that row, where ISa tops out near 40 - so it can only remove the singularity, never suppress evidence a measurement could have produced. Measured across the realistic range it changes no genuine zone at all (22.0, 34.7 and 65.4 nats, unchanged to four figures) and takes the unmeasurable ones to 13 and 36, below and around the bound respectively. AbsenceEvidence moves to the header, where the comments already named it, so the ordering it has to satisfy can be tested directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:26 +02:00
leonarski_f	e0ff51e2a0	lattice: transpose the change of basis to a primitive cell, as everywhere else ... gemmi states centred_to_primitive as an operator on COORDINATES and CrystalLattice::Multiply combines BASIS VECTORS, so the matrix has to be transposed on the way in - as it already is at both of the other places a gemmi Op::Rot reaches Multiply, one of them in this same file. A, B, C, I and F are symmetric matrices, so for them the transpose is a no-op and the omission never showed. R and H are not. Measured on an R-centred hexagonal lattice, ToPrimitive('R') returned 59.5 81.7 43.3 / 145.6 124.5 46.7 where the rhombohedral primitive cell is 49.3 49.3 49.3 / 60.9 60.9 60.9. What hid it is that a determinant is unchanged by transposition, so the VOLUME came out right - and most callers only take the volume. It is not only cosmetic: the result feeds the re-seating path that puts a lattice into a space group the user fixed by hand, so an R-centred lattice was handed the classifier a "primitive" cell that is not that lattice - broken for exactly the centring whose setting most needs re-seating. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 6ca00e927d664e870515c04164defa81d8a18725)	2026-08-31 07:17:00 +02:00
leonarski_f	527a5187f4	lattice: restore the character that names a centred monoclinic mI reduced form ... ITA character 43 was absent from the table. It is the type-II reduced form of a centred monoclinic lattice with no length equality - mC, mI, mA and mF are one Bravais lattice in four settings, and this row names the one whose conventional cell comes out I-centred. With the row missing such a cell reaches character 44 and is reported as triclinic, losing its centring outright. It accounted for 30 of the 31 demotions left after the two fixes before this one, and for 4.1% of random centred-monoclinic lattices. Both of its conditions are equalities on scalar products - International Tables gives them as 2|D+E+F| = A+B and |2D+F| = B - so the three angle tests are vacuous for it and the second equality is what selects it. cond_2DF was declared in the character struct and never tested against anything, because until now no row used it. Audited over 14000 exact lattices, the row fires 35 times: 32 are centred monoclinic lattices it recovers correctly and 3 are triclinic cells it promotes, all three of which Le Page promotes at the same tolerance. No over-call is attributable to it. The row is taken from International Tables rather than derived - no single integer matrix of determinant 2 covers more than 68% of these cells - and the test checks the conventional cell it produces has two right angles and twice the primitive volume. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 4c2baf0cc3b2177196a9448df06e24b5446de96a)	2026-08-31 07:17:00 +02:00
leonarski_f	931d7acc7c	lattice: judge a structurally-zero scalar product against the size of the cell ... The Niggli type of a reduced cell is the sign of its three scalar products, and gemmi's reduction was asked to decide those signs against an absolute tolerance of 1e-9 while the products themselves are 10^3 to 10^5 A^2 and the cell is held in float. A product that is structurally zero therefore arrives carrying about 1e-4 A^2 of rounding and is read as definitely signed. The reduction lands on the wrong side of the type-I/type-II boundary, the character written for the other side matches nothing, and the lattice comes back with less symmetry than it has. It is not a corner case. Take an exactly body-centred tetragonal lattice with c > a*sqrt(2) and merely ROTATE IT IN SPACE: 38 of 60 rotations lose the 4-fold, and it comes back C-centred. Every lattice this code classifies is a refined, rotated one; the two tetragonal-I cases already in the tests are axis-aligned, which is the one corner where the rounding vanishes. The tolerance is now scaled by the cell's own magnitude. Over 14845 exact lattices that puts it on a plateau three decades wide - demotions 461 -> 31 - with the over-call count unchanged at every point of it, and with the three fixes that follow the plateau is four decades wide and the demotions are zero. The constant is 100x the one Grosse-Kunstleve et al. give because theirs is calibrated for a double-precision cell and this one is float: measured, their value recovers 6% of these lattices and this one 93%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit b82f217719450e1e46e94be5b5a43ed8ffbf2ca4)	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	b31fb0a55c	symmetry: take the point group from whichever merge found it, the absences from all observations ... The search runs on two merges - one holding every observation, one holding only the well-measured ones - and the second was allowed to report but never to decide. That cost real symmetry: over the corpus the two arms disagree on eight crystals and the arm that found the LARGER point group is right on all eight. A promotion is refused by the most damning statistic it can be shown, so only one of the two arms has to be starved for a genuine operator to be thrown away, and which arm that is depends on the crystal. But the two halves of the answer do not come from the same place. Absences live in the weak reflections the filter discards, so adopting the filtered arm whole recovers the point group and loses the screw - a P6_5 crystal comes back as P6. So the arms are split rather than ranked: the point group is taken from whichever supports the larger one, and its absences are then judged here, on every observation. SearchSpaceGroupOptions::fixed_point_group was written for exactly this and had no caller. Measured on the deposited arm: a trigonal-3 answer becomes P 6_1 against a deposited P 6_5 (an enantiomorph pair, so correct), and a monoclinic answer becomes P 2 2 2 against a deposited P 2_1 2_1 2. The screw survives the split, which it did not when the filtered arm was adopted whole. No crystal that was already right moved, and the over-call column is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	d35e8f680f	symmetry: say when the cell metric hosts more symmetry than the group adopted ... Every symmetry under-call in the corpus has the same signature: a lattice whose metric carries rotations the adopted group does not, with nothing in the run saying so. The user is left to notice that a P1 answer sits on a cell whose axes are equal and whose angles are 60 degrees. So a run that determines its own group now compares the two and says what it sees. It decides nothing - no threshold, no promotion, no demotion, no reprocessing - and the message says as much, because a pseudo-symmetric metric is ordinary and only the intensities can settle whether the extra rotations are real. Measured over 83 crystals it fires on 19 and covers 6 of the 7 point-group under-calls; on the crystal whose deposited group is F 4 3 2 and which rugnux reports as P 1, the line reads "the cell metric is cubic - it admits 24 rotations where P 1 has 1". The metric symmetry is Le Page's, taken through GEMMI's implementation, at an obliquity of 1 degree - the middle of the band over which the answer is stable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	493be2cf19	symmetry: enumerate the settings the cell can host, not only the reference ones ... The space-group search offered a candidate only if gemmi calls it the reference setting. A setting is a statement about direction, so that restricted the search to the axes the convention chose: a crystal whose 2-fold lies on c had no rung between P1 and 222 and fell to P1, and one whose screws lie on b and c was reported as the group with a single screw on c - the wrong group, not a lower one, because the candidate that predicts a subset of the real absences and nothing else wins on no evidence at all. Both stages now enumerate more, under refusals rather than thresholds. Stage B offers the non-reference settings of the chosen point group. Their rotation set is equal to the chosen one, not merely contained in it, so this cannot raise the symmetry; what it adds is a screw or a centring on the axis the data show it on. A candidate is offered only if the cell's own metric admits the rotations its setting names, and one predicting exactly the absences another candidate already predicts is dropped as the same hypothesis under a second name. A non-reference candidate whose centring class this merge does not contain is refused outright: the reference path may adopt an untested centring because the caller's centred-lattice re-test backs it, and a non-reference setting has no such backing. Stage A offers the rotation sets no reference setting carries - the a-unique and c-unique monoclinic 2-folds, and the two rhombohedral-axes trigonal groups - and only those, so every point group reachable before is still reached by the same group in the same setting. A rung reached only that way may be ADOPTED but does not judge anything else: it is skipped when the reference chi^2 is formed and when a higher promotion's parents are collected. Without that it made higher promotions strictly harder - a promotion answers to the most damning of its parents, and offering two more order-2 subgroups of 222 refused 222 and 432 on crystals that had them. The run report gains SPACE_GROUP_NAME beside SPACE_GROUP_NUMBER, since the number alone does not say which axes a group's symmetry lies on, and a group that is not a reference setting is printed as its extended Hermann-Mauguin name. The two-arm reconciliation compares point groups by that name rather than by number for the same reason. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-31 07:17:00 +02:00
leonarski_fandClaude Opus 5	26fc4b02b3	symmetry: carry the space group as the group, not as its number ... The adopted space group travelled the pipeline as a bare int and was rebuilt downstream with find_spacegroup_by_number, which returns the reference setting. So every setting a number cannot name was destroyed one line after it was determined: P 1 1 2 came back as P 1 2 1, I 1 1 2 as C 1 2 1, R 3:R as R 3:H. DatasetSettings now holds the gemmi::SpaceGroup itself, DiffractionExperiment exposes it as GetGemmiSpaceGroup() / GetSpaceGroupOrP1(), and everything that used to take an int - HKLKeyGenerator (its int constructor is gone, so the compiler finds the callers), the merge, the R-free flags, French-Wilson, the reindexing ambiguity, the completeness enumeration, the MTZ and mmCIF exports, the model validation - takes the group. -S keeps the setting the symbol names rather than reducing it to a number. The end message carries both spellings and a reader prefers the name, since only the name keeps the setting while the number is what a reader written before the name understands. It carries them over CBOR too: the determined group was never serialised at all, so a group rugnux chose reached the master file only when the same process wrote it, and an online writer fell back to whatever the user had supplied at the start. Both keys are optional additions, so an older reader skips them and a newer one reads an older sender. On disk the master's /entry/sample/space_group carries the extended Hermann-Mauguin name and is what the reader takes the group from, so a setting survives a _process.h5 and the --mode scale that re-reads it; the number stays beside it and is the fallback for files written before. Every one of the 230 reference settings the old writer could produce reads back as itself, so older files are unaffected. Stage A and Stage B of the search still enumerate reference settings only, so this determines no group differently today - it is what the enumeration needs before it can be widened. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-08-31 07:16:43 +02:00
leonarski_f	6ed4ea541e	reader: a link to a file that is not there is not a dataset that exists ... Every DECTRIS Eiger master links saturation_value, pixel_mask, bit_depth_readout and serial_number into a companion <prefix>_meta.h5, and that file is routinely not kept when a dataset is archived or deposited. The existence test asked only whether the LINK was written, which it is, so every optional-field guard in the reader answered yes and the read that followed threw. A deposited Eiger 16M set could not be opened at all, over values the reader was perfectly prepared to do without. Exists() now asks the second question too - whether the object the link names can be reached - so an orphaned link reads as absent and the fallbacks behind it do their job. The saturation value is then allowed to be missing outright, because on such a file it is: neither the NXmx name nor the DECTRIS one is readable, and there is no third place to look. Left unset, GetSaturationLimit() falls back to the container's own overflow. That is the safe direction - it can only fail to call a pixel saturated, where too LOW a value drops the whole reflection and silently removes the strongest data - and the run says out loud that nothing will be called saturated. The set that could not be opened now processes to 2.17 A against a deposited 2.40 A, in the deposited space group, with a cell agreeing to 0.08%. Output is byte-identical on datasets that already opened. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N (cherry picked from commit 0637979f6d95b446406ab70d1f3982841d195b36)	2026-08-30 21:43:54 +02:00
leonarski_fandClaude Opus 5	fb18457e6f	docs: bring the changelog and the method notes up to rc.166 ... The rc.166 changelog was missing fourteen user-visible changes and carried rationale and measurements that belong here instead. Added: the native miniCBF sweep reader, the third-party and firmware-1.x NXmx masters, the plain LZ4 filter, the image orientation taken from the file's module direction vectors, the two beam-centre flags and their rescues, and the FFT reach past 500 A. Trimmed the rest to one line each, moving the numbers out of the user-facing file. RUGNUX.md described the input as a single Jungfraujoch master file, which it has not been since this branch; it now covers the foreign and legacy masters, the accepted compression filters and the miniCBF sweep, including how a sweep is collected from one named frame. Six options existed with no entry in the table - --beam-center-check, --beam-center-search, --fft-min-unit-cell, --min-indexed-spots, --rot3 and --no-p1-crosscheck - and -C now moves both FFT cell bounds, which was not written down anywhere. CPU_DATA_ANALYSIS.md carried two statements this branch made false: 7.5 still said pass 2 reuses pass 1's space group, and 13.1 still said centrings are ranked by net absence count. Both now describe what the code does - the group is determined after pass 2, and centrings are ranked by the same Beta-tail likelihood the screw test uses. Also documents the per-zone screw scoring, the coplanarity volume-fraction guard, the plane-normal transform, the FFT cell bounds and the twelve refined candidates. The miniCBF reader implements the x-CBF_BYTE_OFFSET scheme and reads the imgCIF axis table from the specification alone. No CBF code is vendored or linked, so there is no licence obligation, but reimplementing a published specification carries one of credit: ACKNOWLEDGEMENT.md gains a section and the two algorithms carry a one-line reference each. Both DOIs were resolved before being written. Rottger's initial was wrong where this branch first cited it - K, not A. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 21:15:31 +02:00
leonarski_fandClaude Opus 5	9924dd9fc3	docs: the open test battery, updated - 82 datasets, and not only non-SLS ... The public data the pipeline is exercised on has grown from 59 datasets to 82, 77 of them with a released PDB entry and released structure factors, so the page that credits the depositors and carries the DOI to cite for each is brought up to date with what is actually run. Renamed from NON_SLS_TEST_DATA to EXTERNAL_TEST_DATA, because the old title stopped being true: a few of the sets were collected at SLS beamlines, where the data are still written by someone else's detector and someone else's acquisition system. What the battery tests is foreign files, not a foreign facility. Also rewritten from the current archives rather than the earlier sample: - Multi-collection archives: eleven are not a single continuous rotation, not four. Seven IRRMC archives hold more than one collection; one sweep is kept in six of them, and both are kept in the one whose two sweeps are at different wavelengths. A repository project page is not a reliable guide here - one describes a 900-frame sweep its own tarball does not contain. - Detector labels: 76 rows can be compared against the PDB entry. Seven genuinely conflict, and one of those the file settles outright - pixel count, pixel size, sensor thickness and firmware string agree with an EIGER2 9M against the entry's PILATUS4 4M. A further 29 differ only in how much they state, which is not a conflict. - One archive ships 30 placeholder files named like images that are 64-byte text; named on the page so a reader that globs the directory is not surprised by them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 20:50:10 +02:00
leonarski_fandClaude Opus 5	03456dcddd	calibration: an option to hold the detector tilt fixed ... --mode calibration fits five parameters - beam centre, distance and the two PONI tilts - and a program that cannot express a tilted detector has nowhere to put the last two. Dropping them after the fact is worse than never fitting them: the centre and the distance of a tilted fit have already absorbed the tilt, so the flattened geometry is right nowhere. rugnux --no-refine-tilt, the "Refine detector tilt" tick box on the viewer's Calib page and RingOptimizer's refine_tilt argument hold rot1/rot2 at the value the geometry came in with and fit the remaining three. That is the best flat-detector answer, and the one such a program would refine to itself. Measured on a five-distance calibrant series. At short distance the tilt is real and reproducible - three independent fits agreeing to 0.01 deg, radial rms 1.4 -> 0.4 px - and its direct beam agrees with the background beam-centre estimator to 0.05 px, so the tilted model is the physically right one. The pinned fit then displaces the centre 2.6 px to absorb the tilt and lands within 0.03 px of the same place at every distance. Past ~300 mm, where only two rings reach the detector, the tilt is instead under-determined: it comes out with the opposite sign to every short-distance fit and drags the PONI 28 px while the rms does not move (0.960 against 0.962). The existing degeneracy guard only fires on a strictly single ring, so it does not catch that; declining a tilt that does not pay for itself in rms is left for a separate change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5	2026-08-30 20:44:12 +02:00
leonarski_f	d5818713cb	rugnux: also merge in P1, so a wrong space group is recoverable ... A rotation run that determines its own space group now also writes <prefix>_P1.mtz: the same observations, the same scaling, merged in P1. If the group was wrong there is no route back today except processing the images again, and the P1 data settle it - re-merge, re-solve or re-refine in any subgroup. Written whether or not the search adopted P1, because a file whose presence depends on what the pipeline decided cannot be harvested by a script. A run given -S writes nothing: its group's centring absences were never integrated, so the P1 merge would be missing whole classes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f # Conflicts: # docs/CHANGELOG.md # rugnux/rugnux_cli.cpp	2026-08-30 14:15:20 +02:00
leonarski_f	123dc22be2	rugnux: write the unmerged MTZ by default ... A run that does not ask for it still has to hand its data to aimless, pointless or careless eventually, and the file it needs is one a user had to know a flag to get. It is the largest a run produces, but it is 3.4 to 5.8 per cent of the images it replaces - and the better it is, the less anyone needs to keep those. The batteries pass --no-export-unmerged: they process the whole corpus and would write a few gigabytes nobody reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f # Conflicts: # docs/CHANGELOG.md	2026-08-30 14:14:28 +02:00
leonarski_fandClaude Opus 5	f75cf72d1a	beam centre: measure it on every run, and try it when the header fails ... The header beam centre is wrong by more than the geometry absorbs on two thirds of foreign depositions, and nothing measured it. The solvent ring already gives it away: the background projection the beam-stop pre-scan builds is enough to fit the centre, so the measurement is a fit over an array the run has already paid for. It is reported on every run and committed on none. The first pass runs again at the measured centre and the two lattices are compared; the measured centre is taken only where the header indexes nothing, and a disagreement is reported rather than resolved, because the only arbiter available at that stage is the frame count and it is inverted. Recovers two depositions whose header is the geometric detector centre. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:07:13 +02:00
leonarski_fandClaude Opus 5	65dfdb1324	reader: read the detector arm and the mounting each file states ... A detector swung out on a 2theta arm was read as if it stood square on, and a miniCBF written with a vertical spindle or a quarter-turned image was read with the standard mounting assumed. Both are stated in the file and both were ignored: the NXmx depends_on chain was never followed, and Detector_2theta was parsed into a field nothing read. Recovers three datasets that produced no usable lattice, and the small-molecule sweeps at 30 and 55 degrees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:07:04 +02:00
leonarski_fandClaude Opus 5	7528bd8761	docs: write down the three directions this work is heading in ... Three preferences that have been decided in conversation and applied to several changes already, but were nowhere a new reader would find them. They are directional, not absolute. Spending compute to buy quality is now affordable where it once was not, so the question to ask of a slower design is what it buys. Deciding late beats deciding at a threshold, because a gate calibrated on one population refuses another - measured repeatedly in the lattice and symmetry code. And the target is a bare invocation with no flags, because a capability nobody knows to switch on has not solved the problem. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:06:54 +02:00
leonarski_fandClaude Opus 5	a27c4cf26f	reader: take a miniCBF's mounting from the imgCIF axis table its header states ... A miniCBF header states three things about how the instrument is put together that the reader was assuming instead: which laboratory direction the image's columns run along, which its rows run along, and which the spindle turns about. Some beamlines append a CBF template block holding the full imgCIF axis table, which says all three outright. Two instruments in the corpus are not what was assumed, in two different ways. One mounts its detector a quarter turn round, so the image's columns run vertically. Another turns its spindle about the VERTICAL, with the image mounted the usual way; its table says so, and its "# Oscillation_axis" line says so a second way, by naming the image direction the spindle runs along rather than a vector. Either error leaves the spindle 90 degrees from the image. That is not a sign, so the run's axis-sign rescue cannot reach it, and no refinement recovers it: all three affected sweeps indexed nothing usable. So the table is read. The element axes give the image orientation, matched against the eight discrete mountings exactly as the NXmx module directions already are - the match itself moves to DetectorOrientation, so both readers share one definition rather than two copies. The goniometer axis with no parent gives the spindle DIRECTION; its sign stays the rescue's business, which is the part a convention can legitimately differ on. The detector axis with no parent gives the 2theta arm, replacing the assumption that the arm shares the spindle's axis - the one header stating both states them with the same vector, so this changes no answer, only what it rests on. imgCIF's frame differs from the internal one by a half turn about x, a rotation and not a mirror, as writer/HDF5NXmx.cpp already records from the other side. Where a header carries no table, a "+SLOW" on the Oscillation_axis line still says the spindle runs along the image's slow direction. That is the only thing one of the three affected sets says about it. The axis NAME on that line stays unusable - the header that carries both says "X.CW" where its own table says Y - but the direction token is not: where both are present they agree, which is what makes reading it evidence rather than a guess. Also: naming a frame with no directory at all now finds its sweep. parent_path() of a bare filename is empty and iterating an empty path finds nothing, so running from inside the data directory reported that no images were found. Measured, with nothing on the command line. The vertical-spindle protein set goes from no usable lattice to 100% indexed, P 6(3) 2 2 with a cell 0.43% from deposited, 87846 reflections at 86.3% completeness and CC(1/2) 0.995. Its companion from the same detector, which has no table and only the +SLOW token, goes from a spurious monoclinic cell at 2.3% completeness and I/sigma 0.21 to the right orthorhombic lattice, 97.7% indexed, 59.7% complete, CC(1/2) 0.996. The quarter-turned set's three sweeps, at three arm positions, now all index without the hand-passed quarter turn they needed and agree on one cell to 0.03 A. Six miniCBF sets that state no table and no +SLOW - including one whose Oscillation_axis line names an axis in a third dialect - are byte-identical in .hkl, .mtz, .cif and the image statistics, as are two NXmx sets, which is the shared orientation matcher moving nothing on that path either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f	2026-08-30 14:03:32 +02:00