1208 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5	d4f280047f	grid scan: a raster reaches the spot engine, and the crystal cap says none rather than zero ... Two defects the merge created and one the API carried. Rugnux gated the per-image spot engine on AnalysisModeIsMX, so AnalysisMode::Grid fell through to the azimuthal-integration-only path: a raster ran, scored nothing, and reported no crystals. The gate now asks the stages table whether the mode does spot finding, which is the actual question - three modes need that engine for three different reasons, and a fourth would otherwise have to be remembered here too. max_crystals was a required integer defaulting to 10, with 0 meaning "all". Zero reads as "report no crystals", the opposite of what it did. It is now optional, and absent means no cap; a crystal found and then dropped is information the caller cannot get back. grow_score_threshold was missing from the schema entirely. Measured over the labelled corpus after these fixes: 34 of 34 confirmed-protein rasters yield a crystal, 0 of 8 water, 0 of 6 ice, 19 of 19 heldout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 08:22:14 +02:00
leonarski_fandClaude Opus 5	37efa573c4	generated: regenerate the API clients for the analysis mode, the grid settings and the crystal list ... Output of update_version.sh. The five lanes each reverted their generated trees before handing back, so that two independently regenerated copies never had to be merged; this is the single pass that replaces them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:59:42 +02:00
leonarski_fandClaude Opus 5	5fe8a967cd	grid scan: the five lanes become one, with a single home for the crystal type and its settings ... Integration of the per-image detection scores, the analysis mode, the grid-scan crystal search, its rugnux entry point and the viewer display. GridScanCrystal/GridScanResult had two definitions - a placeholder in common/ and the real one in image_analysis/ - which is a redefinition in any translation unit reaching both, and tests/RasterReportTest.cpp reaches both. Unified into common/GridScanResult.h, beside ScanResult where the data type belongs, leaving the algorithm in image_analysis/. Same reason UnitCell lives in common while the indexers do not. GridScanAnalysisSettings is now the only home for the search parameters, replacing the loose GridScanAnalysisParameters struct the raster lane carried while the class did not yet exist. Three values changed with the move: - decisive_single_cell_score 0.9 -> 0.6. 0.9 drops a real two-cell crystal peaking at 0.751 and costs a loop on the labelled corpus. 0.6 is the middle of a measured gap: over 67 rasters no water raster peaks above 0.15 and no ice raster above 0.50, while the weakest confirmed-protein raster peaks at 0.67. - max_crystals is std::optional, unset meaning no cap. 0 as a sentinel for "unlimited" reads as "find nothing", which is the opposite of what it did. - grow_score_threshold was missing from the class entirely. The viewer reads protein_score, ice_score and the crystal list from the reader rather than a local stub, and asks the broker for ice_ring_ratio rather than the retired ice_ring_score spelling. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:59:04 +02:00
leonarski_f	de3fdf9534	Merge branch 'worktree-agent-ae76aa0c66b33a4db' into grid-scan-integration	2026-09-08 07:55:31 +02:00
leonarski_f	cb1d88dd4d	Merge branch 'worktree-agent-a4c58d76a0be29dd6' into grid-scan-integration ... # Conflicts: # rugnux/rugnux_cli.cpp	2026-09-08 07:55:31 +02:00
leonarski_f	36a082c2f8	Merge branch 'worktree-agent-aa55c81220288558b' into grid-scan-integration ... # Conflicts: # common/ScanResultGenerator.cpp # docs/CBOR.md # docs/HDF5.md # frame_serialize/CBORStream2Deserializer.cpp # frame_serialize/CBORStream2Serializer.cpp # writer/HDF5DataFilePluginMX.cpp # writer/HDF5DataFilePluginMX.h # writer/HDF5NXmx.cpp	2026-09-08 07:54:46 +02:00
leonarski_fandClaude Opus 5	748d14d2cd	file format: the ice ring ratio is stored and sent under its own name, and the retired one is still read ... Completes the rename the C++ identifiers already carried. Producers now emit ice_ring_ratio / ice_ring_ratio_mean over CBOR and write /entry/MX/iceRingRatio and iceRingRatioMean; both readers accept the retired ice_ring_score / iceRingScore spellings as well, so nothing that exists on disk or on the wire stops being readable. The REST plot_type gains ice_ring_ratio and KEEPS ice_ring_score, both mapping to the same plot, so no client breaks. --ice-min-score is deliberately unchanged: it is a threshold on the ratio, it is the one surface a user types, and this program has 91 long options and no aliases. The fallback is not a formality. rugnux --mode scale reads the stored per-image value to reproduce the ice gate the writing run applied, and an unread dataset does not fail loudly - ice_n == 0 sends the gate to its fail-safe branch, "ice present". Demonstrated on a clean crystal whose stored file uses the old names: with the fallback the gate reads 1.06 against a 1.50 threshold and skips ice handling; with the fallback removed the same file has 272056 of 1032493 reflections (26%) excluded from the scale fit, and ISa, R_meas, I/sigma, SIGANO and both error-model terms all move. That is a silent change to merged intensities on files already written, which is why the two new CBOR test cases were each checked to FAIL when their fallback is taken out rather than merely to pass. Verified: same binary on an old-name and a new-name copy of one file gives byte-identical .hkl, .mtz and unmerged .mtz and an identical report; a file written after the rename scales to the same bytes again; the master and the data-file read paths were each exercised with both spellings; and the live API serves the same plot under either name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:42:23 +02:00
leonarski_fandClaude Opus 5	273591af11	grid scan: a crystal is grown out of its seeds, so one weak grid point no longer splits it ... A crystal in the corpus was reported as two because the grid point joining its halves scored 0.498 - two thousandths under the threshold. That is an artefact of putting a hard edge through a continuous quantity, not a gap in a crystal. The patch search is now hysteresis. A patch is labelled at a lower grow threshold (0.35) and kept only if it holds a cell above the seed threshold (0.50), which is the same thing as growing out of the seeds in one pass of the labeller rather than two. A cell between the two levels joins a crystal that already exists but can never start one, so no lowering of the grow threshold can turn a background into a crystal - and the negatives are safe by construction, since water peaks at 0.146 and ice at 0.490, both under the seed level. Both halves of the small-patch rule read the GROWN patch - the count includes the cells growth added and the peak is the patch's best cell wherever it lies - which is stated at the test, because a reader will otherwise wonder whether a grown cell can rescue a patch no seed would have admitted. It cannot: the patch is discarded before that test unless it holds a seed. The reported extents are those of the grown patch, so they reach the 0.35 contour. Over the corpus that is major_um x1.04 and n_images x1.07 at the median (x1.11 and x1.14 at the mean, +7% cells overall), and the cells it adds are spread fairly evenly over 0.35-0.50 rather than piled at the bottom - a crystal edge sampled at one grid step, not bleed into a neighbour that never diffracted. The patch mean falls with them, and can now sit below the seed threshold; the peak beside it is what the admission was decided on. RASTER_REPORT_VERSION is 3, because N_CELLS and the extents are a different measurement under the same names. The corpus table is unchanged: 17/17 protein, 0/4 water, 0/3 ice, 10/10 heldout. The split heals into one crystal, and one raster's five patches become three. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:34:36 +02:00
leonarski_fandClaude Opus 5	c08e37443a	grid scan: PLACEHOLDER GridScanCrystal/GridScanResult, to be replaced at integration ... Revert this commit and nothing else when the real types land. The two structs and the two OpenAPI schemas here are stand-ins. The real GridScanCrystal/GridScanResult belong to the grid-scan analysis, in image_analysis/grid_scan_analysis/GridScanResult.h, alongside the ranking, clustering and crystal selection that produce them. They are duplicated here only so that the serialisation in the previous commit - the /result/scan payload, the CBOR END block, the HDF5 master - could land without waiting for that work. The field list was checked against the real header and agrees with it: all twelve GridScanCrystal fields match by name, type and meaning, res_A included (NaN where nothing was measured), and GridScanResult carries the two beam sizes beside the crystal list. To integrate: delete common/GridScanResult.h and its line in common/CMakeLists.txt, point the two includes - common/ScanResult.h and common/JFJochMessages.h - at the real header, and replace the grid_scan_crystal / grid_scan_result schemas in broker/jfjoch_api.yaml. Nothing else refers to these two structs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:28:01 +02:00
leonarski_fandClaude Opus 5	98132d0f83	analysis: every analysis method carries its own settings, and a raster's indexing is one of them ... AnalysisSettings had begun collecting per-method parameters - the calibrant was already in it, and the grid thresholds were about to be. That makes the structure every method reads grow whenever any one method gains a knob, and it puts a field in front of readers for whom it means nothing. So: AnalysisSettings keeps what all methods share, which for now is the mode, and each method gets a class of its own bound the same way. GridScanAnalysisSettings holds the protein-score threshold, the minimum cells per crystal, the decisive single-cell score, the maximum crystals reported and the indexing switch. CalibrationSettings holds the calibrant and the ring source. Both sit on DiffractionExperiment outside the per-run dataset member, both have an Import/Get pair, and both have their own endpoint - /config/grid_scan_analysis and /config/calibration - which is how every other settings group in this API is already reached. Grid indexing is no longer fixed in the stages table. It was turned off there on cost grounds, and that reasoning does not hold: a raster runs at up to 100 Hz, which the FFT indexer keeps up with, and a fixed-target serial experiment with a known cell wants ffbidx on every cell, where a raster that indexes is most of the measurement. So it is a setting, and DEFAULTS ON. It is additive rather than a change of answer - blobs are still found on the protein score, so indexing alters nothing about which cells are called crystals and only adds what was found in them, including the per-cell lattice count, which is the cheapest multi-lattice or cracked-crystal signal there is. That makes indexing the one stage a mode does not decide. AnalysisModeStages still carries a value for it, but only as the setting's default, and DiffractionExperiment::GetAnalysisStages - which is what every gate reads - substitutes the configured one. The table row is marked so nobody reads it as the mode's answer. The calibration knobs stay coupled to the mode but the rule now lives with them: CalibrationSettings::ApplyToAzimuthalIntegration moves azimuthal integration onto the CPU and supplies sectors where fewer than four were asked for, carrying the reason with it - the FPGA integration core holds 2048 bins in total, so 32 sectors leave 64 q bins, which cannot locate a ring. Stated there because it will otherwise read as an FPGA defect to be fixed back onto that path, and it is not one: the core is sized for a detector at full rate, and a calibration exposure is a few images at a few Hz. Both imports apply it, so the order the mode and the calibration settings are set in does not matter. CalibrationMethod moves from image_analysis/geom_refinement/PowderCalibration.h into common/CalibrationSettings.h, which that header now includes. One enum, so the setting and the code consuming it are not two vocabularies; every existing user sees it unchanged. The grid thresholds have one home and it is this class. The raster work owns AnalyzeGridScan's parameter surface and carries PROTEIN_SCORE_THRESHOLD_DEFAULT / MIN_BLOB_CELLS_DEFAULT beside that header today; the header here states the signature that replaces them, so the two do not become competing defaults. The beam size deliberately stays a separate argument to AnalyzeGridScan: it is measured, not configured. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:27:41 +02:00
leonarski_fandClaude Opus 5	eba81013cd	analysis: a scanning goniometer under an MX mode is normal, and no longer warned about ... The warning claimed a sweep collected under mx_stills was being analysed as something the data did not support. It is not a problem to report: a scanning goniometer is REQUIRED for the file to carry correct metadata, mx_stills included, so its presence under an MX mode is the ordinary case and firing on it would train an operator to ignore the log. Removed rather than downgraded. Nothing is left behind saying the condition is tolerated, because there is no condition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:26:58 +02:00
leonarski_fandClaude Opus 5	0f00b76a9a	image analysis: the ice score takes the pipeline's own band width, and the ice quantities are named for what kind of number they are ... Two things, both about telling one ice quantity from another. The ice score's spot channel had its own band half-width of 0.02 A^-1 while the spot finder marks ice rings at 0.03 (ice_ring_width_Q_recipA). The 0.02 was justified by a 5 pp specificity gain measured on the PYTHON PROTOTYPE, which used a fitted beam centre and a mask-derived coverage table; the shipped port, which takes the geometry's centre and the azimuthal profile's own live pixel count, does not reproduce it. Measured over the corpus by truth class rather than by directory label, at 0.02 vs 0.03 on the combined score: ice loops 62.13/62.19%, _icy protein 89.03/89.79%, _clean protein 16.51/16.31%, water 17.19/20.03%. The widths are indistinguishable except on water, where one of the four loops is independently known to carry a full hexagonal pattern. So the width is now a parameter and the pipeline's own value is passed in - one band width, not two. The 0.012 tolerance in the radial channel is NOT a second band width, and is renamed CENTRE_SMEAR_Q to say so: it is how far either side the channel looks for the bin a mis-set beam centre moved the ring to. The rest is naming. Three kinds of number were all called score, or built from things called count, and a reader could not tell from the name whether 1 meant "none" or "certain" - which are opposite. The convention, now stated in docs/CPU_DATA_ANALYSIS.md: *_score is bounded [0,1] and 1 is certainty, *_ratio is unbounded and 1 is nothing, *_count is a count. The C++ identifiers for the ice ring ratio follow it (ice_ring_score -> ice_ring_ratio, GetIceRingScore -> GetIceRingRatio, PlotType::IceRingScore -> IceRingRatio), and the local in the scaling gate that shadowed the new ice_score while meaning the ring ratio is renamed with them. Nothing outside the source moved: the CBOR keys ice_ring_score and ice_ring_score_mean, the datasets /entry/MX/iceRingScore and iceRingScoreMean, the ice_ring_score plot type and the --ice-min-score flag are all unchanged, and were checked to be after the rename. Renaming those changes stored files, the stream format, the REST API and a CLI flag, and is a separate decision. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:19:03 +02:00
leonarski_fandClaude Opus 5	cdb60c2916	grid scan: one cell is a crystal when that cell is decisive, and the list can be capped ... The minimum patch size was a flat floor, so a crystal that covers one or two grid points was thrown away however strong its diffraction. It is now a floor OR a piece of evidence: a patch is reported when it has min_blob_cells cells, or when its best cell reaches decisive_protein_score. One condition, and both halves are parameters. The bar is the patch PEAK, not the patch mean. A two-cell patch with one strong cell and one marginal one is the case this exists for, and the mean averages exactly that evidence away. Over the 67 labelled rasters the two populations do not overlap: no water raster reaches a peak of 0.15 and no ice raster reaches 0.50, while the weakest protein raster peaks at 0.67 - so 0.6, the middle of that gap, is the default. The peak is reported beside the mean, in the table and in the JSON, so an admission can be checked against the number that decided it. max_crystals caps the returned list after the sort, best first; 0, the default, is all of them, because a crystal that was found and then dropped is information the caller cannot get back. The four parameters now travel as one GridScanAnalysisParameters, which is the shape the GridScanAnalysisSettings class in common/ is to take: when it lands it replaces this struct in the signature and nothing else changes. On the corpus this reaches 17/17 protein loops (it was 16/17 - the miss was a crystal covering two grid points of a 4x4 raster) with water still 0/4 and ice still 0/3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 07:15:30 +02:00
leonarski_fandClaude Opus 5	1f54b3a971	rugnux: --mode raster says what it really does with the _process.h5 ... Nothing indexes on the raster path, so RunPipeline's nothing-indexed fallback writes a small _process.h5 whatever --write-process-h5 says. That is the right outcome - for a raster the file is the per-image scores and it is worth having - but the comment beside config.write_process_h5 claimed the opposite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 00:24:54 +02:00
leonarski_f	cd9cc4800e	Merge branch 'rc167' into worktree-agent-ab3a3e173261a95d8 ... # Conflicts: # common/JFJochMessages.h # common/ScanResultGenerator.cpp # docs/CBOR.md # docs/HDF5.md # frame_serialize/CBORStream2Deserializer.cpp # frame_serialize/CBORStream2Serializer.cpp # frontend/src/components/DataProcessingPlot.tsx # tests/CBORTest.cpp # writer/HDF5DataFilePluginMX.cpp # writer/HDF5DataFilePluginMX.h # writer/HDF5NXmx.cpp	2026-09-08 00:24:42 +02:00
leonarski_fandClaude Opus 5	550f7c7d7b	scan result: a grid scan's crystals and a calibration's geometry reach the stream, the file and the API ... A grid scan and a powder calibration each produce an answer that is not a per-image number, and neither had anywhere to go. ScanResult gains two top-level optionals, grid and calibration, in the same idiom the three rotation_* members already use: a mode's payload is present when that mode ran and absent otherwise. images stays required, so every existing /result/scan client - the python client and the beamline scripts on it - is untouched. No oneOf or discriminator. The spec contains none today, so polymorphism is unexercised across all three generators at once, and the only thing it would buy is making "exactly one payload" structurally true rather than true by convention. powder_calibration_output has been fully specified and referenced by nothing since it was written; the calibration member reuses it. The crystal list travels the same three hops a per-image quantity does. In the CBOR end message grid_crystals is an array of maps keyed by field name - the shape a spot already uses - so a reader that does not know a field skips it and the record survives a field being added; the deserializer steps over an unknown key rather than reading its payload as the next one. In the HDF5 master it is parallel 1-D datasets under /entry/MX/crystals, one per field, because crystals are numbered rather than named and a group per crystal would put the index in the path. The reader reads them back so a stored raster re-opens with what it found. Nothing caps the list at one. A raster can find several crystals and the ordering carries the answer: sorted by score, best first. angle_deg is an axis, not a direction. It lies in [0, 180) and wraps, so 179 and 0 are adjacent and two of these do not average arithmetically - a pair of nearly parallel needles averaged across the wrap comes out as a right angle. That is stated at every layer it passes through, since it is the kind of field a consumer will reach for without reading the producer. The producer side is not wired yet: nothing fills ScanResult::grid or EndMessage::grid_crystals here. The seam is receiver/JFJochReceiver.cpp, where rotation_lattice is filled in EndDataCollection and GetFinalStatistics. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 00:17:10 +02:00
leonarski_fandClaude Opus 5	a7d3ada3ab	analysis: what runs over the images is one stated mode, shared by broker, rugnux and viewer ... Until now nothing in the tree said what analysis a run performed. The answer was composed at each site out of four independent scalars - the detector type, two separate "spot finding off" switches, an indexing flag and a rotation flag - so what was configured and what actually ran were different things, and no single place could be read to find out which. AnalysisMode {None, MXRotation, MXStills, Azint, Grid, PowderCalibration} is that statement, in common/ because all three programs configure the DiffractionExperiment that carries it. AnalysisSettings sits on the experiment beside IndexingSettings, outside the DatasetSettings member, which is the one thing a /start replaces wholesale - so the mode is persistent by construction rather than by a rule someone has to remember. The mode does not label a run, it decides it. AnalysisModeStages() is a table - modes as rows, pipeline stages as columns - and every gate reads that table instead of testing the mode: spot finding in DiffractionExperiment::IsSpotFindingEnabled, indexing (and with it prediction and integration, which never run without a lattice) in one gate inside IndexAndRefine that serves all three front ends, azimuthal integration where the CPU engine is built. Two rows carry a judgement worth reviewing: powder calibration keeps spot finding, because --calibration spots fits the pooled spots; grid does not index, because a raster is thousands of frames and the per-image scoring it ranks on deliberately avoids an indexer that fires on ice. There is deliberately no Auto value. GetIndexingAlgorithm() resolves Auto at read time, which is exactly why an indexing setting cannot be read back off the configuration; removing that kind of implicitness is the point here, so the mode getter stays a plain accessor. MXStills is the default because None would silently switch analysis off on every deployment whose configuration predates the field. Rotation MX is absent from the OpenAPI schema rather than present and refused: jfjoch_broker has no rotation analysis path, so the REST and configuration-file routes cannot express it at all. The shared enum can still carry the value from elsewhere, so CheckAnalysisSettingsOnline refuses it on both routes with a message naming rugnux. A sweep collected under an MX mode is not refused - collecting rotation data online is normal and live spot counts are useful - but it is said out loud in the log, since the mistake worth preventing is the silence about what was done to it, not the acquisition. Powder calibration forces azimuthal integration onto the CPU and supplies 32 sectors where fewer than four were asked for. The FPGA integration core holds 2048 bins in total, so 32 sectors would leave 64 q bins - far too coarse to fit a ring. Frame rate is what this costs and a calibration exposure does not need it. The two existing "no analysis" switches, per-dataset dataset_settings.spot_finding and persistent SpotFindingSettings::enable, are interfaces in too many places to remove now. They are marked deprecated in the schema and in both headers, and the mode takes precedence over them: a mode that analyses no spots wins outright, while under a mode that does find spots they remain the finer control. The precedence is written where it is enforced. rugnux's ProcessMode is gone, replaced by the shared enum; RugnuxMode stays as the CLI spelling layer and no existing spelling changes. --mode gains mx_rotation and mx_stills, which are spellings of -R and --force-still rather than new switches; plain mx still chooses between them from the goniometer. scale keeps no shared counterpart, since it runs no analysis over images at all. The mode reaches the CBOR start message and /entry/MX/analysis_mode in the HDF5 master, so a written file records which analysis produced it. It is read back as provenance only - what a stored file was produced by is not what the next run should do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 00:16:26 +02:00
leonarski_fandClaude Opus 5	14e2bf3c0f	Merge rc167 into the detection-score branch ... Both lanes added a per-image scalar to the same eleven files, so every conflict was two additions competing for one line. All were resolved by keeping both, with three that needed more than that: - ScanResultGenerator: rc167 changed the per-image float vectors to resize(n, NAN) so a frame that never arrived does not read back as a real 0. v_protein_score and v_ice_score are exactly that case - 0 is a real answer ("nothing detected here") - so they take the NAN default too. - HDF5MetadataSource: rc167 established that NaN in a stored per-image array means "no value" and the optional must come back absent. The two detection scores now follow it, which they did not before the merge; without the guard a missing score would come back as a NaN that a threshold would silently compare against. - CBORTest: designated initialisers must follow member declaration order, so spindle_blind_fraction precedes the two scores in the DataMessage aggregate. Verified after the merge that every CBOR key that is encoded is also decoded (198 encoded keys, one intentional exception: the "type" discriminator), that both lanes' fields survive in the writer, the reader, the plots and the API, and that a stored file still round-trips. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 00:13:51 +02:00
leonarski_fandClaude Opus 5	21eca107fa	rugnux: --mode raster reads a stored grid scan and reports the crystals in it ... The offline entry point for a raster. It scores every image of a stored grid scan - spot finding for the per-cell resolution, no indexing, because a raster answers where the crystal is and not what its lattice is - hands the per-image protein and ice scores to AnalyzeGridScan, and writes what came back as two files beside the usual output: <prefix>_raster_report.txt in the idiom of the results report, and <prefix>_raster.json with the same content typed, so a battery can aggregate a sweep without scraping prose. --beam-size states the beam at the sample, defaulting to the file's incident_beam_size. It matters more than it looks: the reported crystal sizes are measured and still contain the beam, and taking an anisotropic beam back out is a subtraction of two covariance matrices, so a beam given as square when it is not rotates the reported crystal axis. --raster-protein-threshold and --raster-min-cells expose the two constants AnalyzeGridScan held at file scope, so a sweep can vary them without a rebuild; they are now defaulted parameters with the old values, and every existing call is unchanged. The observer feeding the analysis reads the file's own image number (DataMessage::original_number), not the loop's ordinal, so -s/-e/--stride cannot silently shift the grid mapping. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 00:01:39 +02:00
leonarski_f	3be28a99ad	Merge branch 'worktree-agent-aa55c81220288558b' into worktree-agent-ab3a3e173261a95d8	2026-09-08 00:01:07 +02:00
leonarski_f	294eabdb4b	Merge branch 'rc167' into worktree-agent-ae76aa0c66b33a4db ... # Conflicts: # viewer/JFJochViewerDatasetInfo.cpp	2026-09-08 00:00:55 +02:00
leonarski_fandClaude Opus 5	cd31747c99	viewer: TEMPORARY STUB - the grid-scan composite reads protein, ice and the crystal list from a local placeholder ... REVERT THIS COMMIT AT INTEGRATION. protein_score, ice_score and the grid-scan crystal list are not on JFJochReaderDataset yet; they arrive through reader/ with a separate change. Until then GridScanDataStub.h defines GridScanCrystal and three accessors that return nothing, so the widget and the dataset-info panel are already written against the real shape of the data and compile today. Reverting this commit leaves exactly the two includes and the CMake entry to replace: take GridScanCrystal from the reader, and the three accessors become ds.protein_score, ds.ice_score and ds.grid_scan_result at their call sites in JFJochViewerDatasetInfo.cpp. Nothing else changes - both data paths fill the same JFJochReaderDataset, so HTTP and HDF5 light up together. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-08 00:00:13 +02:00
leonarski_fandClaude Opus 5	0c6167b7c5	viewer: a grid scan shows protein and ice over the shape of the loop, with a frame around each crystal ... Two scalar maps side by side never answered the question a raster is taken to answer. The grid-scan view gains a composite: the normalised background gives the loop its shape in grey, ice paints it cyan, protein orange, and where both are present the two mix in proportion. Protein and ice go in as they come - they are saturating scores already in [0,1], and stretching one would turn "no protein anywhere" into "protein everywhere". Only the background is normalised, with its range floored, so a scan of pure air stays flat instead of having its noise stretched into a convincing loop. Each crystal found in the raster gets a frame, drawn oriented: centred on the crystal, sized along its own principal axes and rotated by its angle. An axis-aligned box would hide the case the axes exist to expose - a 100 x 10 um needle at 45 degrees reads as a compact blob. The extent is in micrometres and goes through both step sizes separately, which a real 20 x 16 um raster needs. A blob within 1.2 of round is drawn as a circle, since its angle means nothing. The single-scalar path is untouched; the composite is one more entry in the metric combo, offered first for a grid scan. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 23:59:56 +02:00
leonarski_fandClaude Opus 5	62ca2b24f6	image analysis: two per-image detection scores say whether there is protein and whether there is ice ... proteinScore and iceScore are saturating scores in [0,1] that answer presence, not quality: a superb crystal and a barely-diffracting one both read near 1, and neither the spot count nor the resolution enters either of them as a term. iceRingScore already reports a magnitude - a ratio, unbounded - which is not a number that can be thresholded; these are. The protein score counts distinct d SHELLS above 5 A rather than spots, so a parasitic ring in the low-resolution band cannot accumulate evidence, and weights each spot against the frame's own median so a scattering of the weakest detections cannot fill a shell either. The ice score carries two channels and takes the stronger: a radial one over the azimuthal profile, which runs the hexagonal and the CUBIC phase as separate hypotheses and decides between them at the end (flash-cooled loops show cubic or stacking-disordered ice at least as often as hexagonal, the two share only three lines, and dropping the cubic hypothesis costs about 5 pp on iced loops), and a spot one that reads an excess on the ice radii against the same band slid to every ice-free offset, which is what catches ice arriving as discrete crystallites and leaving the radial profile flat. Both read d out of the geometry, so both move with a beam-centre error; the centre is not fitted here, and the one they were computed with is written beside them as scoreBeamCenterX/Y so a later rescoring can tell an algorithm disagreement from a geometry one. Ported from validated prototypes and checked against them frame by frame on stored data: mean absolute difference 2.7e-5 (protein), 1.3e-8 (ice radial) and 3.4e-4 (ice spots). On a 41-loop battery the protein score reaches 98.4% of confirmed-protein frames and 0.00% of water frames, and finds no cluster on any water or ice raster. Cost is 0.01 ms/frame for the protein score and 0.08-0.32 ms/frame for the ice score. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 23:59:46 +02:00
leonarski_fandClaude Opus 5	92bdb5d0c9	TEMPORARY STUB - DROP AT INTEGRATION: per-image protein and ice score on ScanResultElem ... Revert this commit, and only this commit, when the lane adding protein_score and ice_score properly lands. It exists solely so the preceding grid-scan commit compiles in isolation: AnalyzeGridScan reads both members, and they are being added to ScanResultElem by concurrent work that computes and serialises them through the whole analysis -> message -> CBOR -> HDF5 -> API chain. Nothing here computes, fills, serialises or reads back either field. They are declared and left empty, so every consumer sees nullopt and the grid scan finds nothing until the real producer arrives. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 23:39:29 +02:00
leonarski_fandClaude Opus 5	ee0ad8d149	grid scan: a completed raster is reduced to the crystals in it, oriented and ranked ... AnalyzeGridScan takes a finished ScanResult plus its GridScanSettings and returns the list of crystals the raster hit, sorted by score so element 0 is the one to collect. Pure function - no I/O, no FPGA, no JSON. Today the list holds nought or one entry; N is the point of the shape. The per-image protein score is scattered back onto the display grid through Rearrange, which already knows the snake order, the vertical flag and the step signs, thresholded, and labelled into blobs. Each blob is then measured: - Centre is a weighted centroid, pulled towards the cells that diffract best. The pull uses the RANK of the resolution inside the blob, never its value, so a salt grain reporting an absurd 0.8 A weighs exactly what a genuine best cell weighs and cannot drag the centre however extreme its number. A cell with no resolution gets the lowest weight rather than being dropped. The centroid of a banana- or L-shaped blob can land outside the blob, where no image exists, so image_number is snapped to the nearest cell that was actually collected. - Second moments are taken in micrometres, not in cells. A 20 x 16 um raster is ordinary and moments in cell units give a wrong angle - eight degrees wrong on the staircase in the tests. The angle is an axis, so it lives in [0,180) and wraps there. - The axis DIRECTION comes from the eigenvector but the LENGTH from the projected extent, because "how far do I scan" is an extent question and the constant taking a second moment to a length assumes a shape a blob of five cells does not have. Where the two disagree about which axis is longer - a moment dominated by clumps at the ends - the extents are swapped and the angle turned a quarter turn, so major_um >= minor_um with angle_deg along it is an invariant a consumer can draw a frame from. - score is the MEAN protein score over the blob, not the peak: the score saturates, so the peak is 1.0 for every real crystal and ranks nothing. res_A is the 25th percentile, not the minimum, the minimum being precisely where a salt spot or a hot pixel shows up; it is NaN when nothing in the blob measured a resolution. Sizes are measured and the beam is left in them. The beam is already in the file as incident_beam_size, so a consumer can deconvolve reproducibly and reversibly instead of inheriting ours; the result carries the beam size so it says what the extents contain. The header records that removing an anisotropic beam is a covariance-matrix subtraction followed by re-diagonalisation, not a per-axis quadrature removal, which is silently wrong whenever the crystal is not aligned with the grid - the needle case this design exists for. Labelling is a small dense flood fill in common/, 8-connected. StrongPixelSet::sparseccl is the wrong abstraction for a dense grid map: sparse union-find over raster-ordered strong pixels, hardcoded module dimensions, a 4000-pixel cap, spot-shape acceptance, and an FPGA header. 8-connected rather than 4 because where the step is coarser than the beam a needle at 45 degrees lands as corner-touching cells; under 4-connectivity that breaks into single cells and the minimum-size rule then discards the crystal entirely, which is the case oriented axes exist to catch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 23:39:16 +02:00
leonarski_fandClaude Opus 5	b650cda037	Revert "integration: the background is fitted over the ring that survives, not averaged over it" ... This reverts commit 6a3ae9c65. The rotation battery refuses it. The median is clean - zero movement in R_meas and CC1/2, most crystals untouched or moving in the third decimal, which is what was predicted - but the tail is not a cost, it is a failure: one crystal's CC1/2 falls from 91.3% to 6.8% with its reflection and observation counts essentially unchanged, so the intensities are being corrupted rather than the weak ones discarded. Another loses seven points of R_meas and half its ISa; a third has its high-shell CC1/2 go negative; a fourth's ISa doubles, which is an error model measuring something different rather than measuring better. The change was predicted to cost one to two points of high-shell CC1/2 on crowded patterns, and that prediction was made on six geometries. Fifty-one say otherwise. The defect it fixes is real and stays on the record: within the outer background radius of the array edge the ring is truncated inward while the disk is not, so the surviving ring sits where the radial background is higher and the reflection reads low - measured at 1.79 counts per pixel four pixels from a border, and confirmed by a masked gap producing the same bias signed by the direction of its displacement. Fitting the background over the surviving ring is the right shape of answer. Doing it per reflection, from a ring that a neighbour mask has often already truncated, is evidently not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 16:30:22 +02:00
leonarski_fandClaude Opus 5	6a3ae9c658	integration: the background is fitted over the ring that survives, not averaged over it ... The signal disk and the background ring are concentric, which is the whole reason a linear background cancels between them. Within the outer ring radius of the edge of the sensor array that concentricity is gone: the ring loses its outer part while the disk barely loses anything, so what is left of the ring sits further into the detector, where the radial background is higher, and the reflection reads low. Measured at signal-free positions four pixels from a border: the ring reads 162.35 counts per pixel against a true background over the disk of 160.57, which over a hundred disk pixels is 182 counts of deficit, against 216 to 239 observed. <I/sigma> runs -1.83, -2.34 and -1.18 at nought to three, three to six and six to nine pixels from the border, and recovers exactly at the outer ring radius. The same reflection measured at a border reads 179 counts lower than in the interior over seven thousand matched pairs. A masked module gap does the same thing but signed by the direction of the displacement, which is why nothing has caught this: at a gap the two populations cancel in the mean, while at the sensor border the truncation is always inward, so the bias is always negative. The background is now the intercept of a straight line in radial offset over whatever ring pixels survive, read at the reflection's centre. Three extra sums per ring pixel and no extra reads; the radial distance was already computed there. It is exact under any truncation and reduces to the mean when the ring is whole, so it is unconditional rather than a mode: a badly truncated ring pays in sigma, through the variance the fit honestly reports, rather than in a rejection. On the crystal where this surfaced the outermost shell's correlation with a deposited model goes from -0.234 to +0.004, and the shell above it from -0.091 to +0.179. Correcting beats discarding: dropping every observation within fifteen pixels of a border reached only -0.019 and +0.127, because the corrected observations still carry signal. Interior reflections do not move. The cost is one geometry: where a neighbour mask has already truncated the ring almost everywhere, the fit roughly doubles the variance of the background estimate while finding no gradient worth removing, and a crowded small detector loses one to two points of CC1/2 in its finest shells. Also: the MINPK denominator counted only profile mass that lands on the detector, so a reflection whose peak is off the sensor scored a perfect one and no guard could fire. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 15:46:32 +02:00
leonarski_fandClaude Opus 5	e5c96cd41b	model validation: the placement target carries a bulk solvent that means something ... The rigid-body target refits the scale at every evaluation, deliberately - held fixed, the target would measure the scale as much as the placement, and the body would translate to repair a scale error instead of moving to where the density is. That refit was gemmi's unbounded fit, the one already replaced for the reported scale, and here it was worse: measured over a corpus of deposited models, 40% of the evaluations that decide where the model goes came out with a b_sol outside 10-80 A^2, on 57 of 89 datasets, ranging from -8072 to +1721. A negative b_sol is a solvent term that GROWS with resolution. One crystal ran its entire committed placement between 290 and 430 A^2, and that placement went into the reported maps. The bulk solvent is fitted once per zone instead, inside the same physical box the reported fit searches, and then held while the overall scale and the anisotropic B keep following the body. That split is the point: k_sol and b_sol describe the crystal's disordered solvent, not the fit of one placement, and measured across a whole zone they drift by a single grid step. Fitting them at every evaluation costs three times the wall clock, makes the scaler 82% of the run, and moves discontinuously under a forward difference - which is poison for a numerical Jacobian. This costs 4%, leaves no evaluation outside the box, and lands the body within 0.37 degrees of the expensive version, against 4.33 degrees for what it replaces. Placements change on fourteen crystals in eighty-nine. R-free is a wash on the mean; the step buys more total R-free from fewer commits, and the gain sits where the runaway actually bit. Two things found while auditing the file and left as they were, because they are right: the refinement sees working reflections only, end to end, and the gauge that removes the origin-free directions of a polar group carries more than half the whole refined translation in ninety-four of a hundred and thirty-nine polar zones. It had no test. It has one now. An observation with no calculated amplitude gets a zero residual, which drops it from the target rather than scoring it as a perfect fit, and is now counted and reported - a large count says the model's reflection conditions do not match the data's, which is a statement about the model. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 15:27:30 +02:00
leonarski_fandClaude Opus 5	004fa5a781	model validation: the model says, shell by shell, where the data still carry signal ... A placed and scaled model gives an expected intensity for every reflection, and comparing that with the merged intensity is a measurement nothing else in the report can make: CC1/2 and R_meas compare the data with themselves, and can only say whether a shell is reproducible, not whether it is right. CC(model, data) is reported per shell, on the merge's own shells so the rows line up, with the reflection count and a significance beside each. Read it in one direction only. A correlation significantly above zero proves signal is present, because a model cannot invent agreement with data it never saw. A correlation near zero proves nothing, since the model may be the thing at fault - one crystal in this corpus reads negative across its outer shells against a deposited model of a different crystal form. So this can support keeping a shell and must never be used to discard one, and nothing in the pipeline acts on it. Intensities rather than amplitudes, matching CC1/2 and CCref beside it. The raw merged intensity rather than the French-Wilson amplitude: that amplitude is a posterior mean under a Wilson prior which pulls weak reflections towards their shell mean, and the shells this number is read in are exactly the weak ones, so it would correlate the prior with itself. The free flag is ignored on purpose. Nothing has been refined against these reflections, so the correlation is unbiased over all of a shell rather than over the test set alone - which is where the outer-shell reflection counts are, and with them the difference between a statistic that can resolve something and one that cannot. REPORT_VERSION is 7. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 13:12:32 +02:00
leonarski_fandClaude Opus 5	cb08f63a52	model validation: the bulk solvent is searched inside its physical range, not fitted without bounds ... gemmi offers two scalers and we were using the one without bounds. Its Levenberg-Marquardt path has nothing stopping the flat-solvent parameters from leaving the range the model means anything in; the alternative path that does declare bounds is behind a compile guard we have never enabled. On this corpus six datasets in fifty-one fitted a b_sol outside it, the worst at 1707 A^2. What that does is subtler than a bad scale, and worth recording because it is why nobody noticed: a b_sol that large does not corrupt the solvent term, it switches it off - 1.4% of it survives at 10 A - so the model is simply scaled without a solvent contribution and the R-factors look unremarkable. k_sol and b_sol now come from a grid search over the physical box, with the scale and the anisotropic B refitted at each candidate pair, following Afonine et al. Refitting at each point is what makes it work: clamping the parameters after an unbounded fit costs up to 0.044 in R-free, because it leaves the scale and B where the rejected fit put them. Non-physical fits go from six in fifty-one to none, and both R-work and R-free come out slightly but significantly better rather than merely no worse. Which reflections are fitted remains the caller's business - the function scales whatever it is handed - so the working-set restriction of the previous commit is not something this can undo. A crystal with no solvent-accessible volume needs no special case: its mask is empty, so the solvent term is identically zero whatever the parameters say. There is a test for that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 10:34:21 +02:00
leonarski_fandClaude Opus 5	2ae8db6623	model validation: the scale is fitted on the working reflections, so R-free is free of it ... The model's overall scale, its anisotropic B and both bulk-solvent parameters were fitted over every reflection, free ones included, and R-free was then computed from the resulting scaled Fmodel. About eleven parameters, minimised against the very sum R is made of, chosen with the help of the reflections that are supposed to be held out of it. The documentation said so and argued it was harmless - the parameters are far too few to absorb individual reflections - and on the numbers that argument is right: the standard optimism bound puts the effect at a few hundredths of a percentage point on an R-free of twenty percent, below the free set's own sampling noise. It is still wrong. R-free means the test reflections took no part in choosing anything the model is scored on, and a reader who greps R_FREE= is owed that without a footnote. The fit now sees the working set; the parameters it returns are applied to every reflection, so the free ones are scored against a scale they had no hand in. Refmac fits scale and B on the working set and says so in its documentation; phenix.model_vs_data, which is the direct analogue of this mode, uses its work-only scaler. Two things still consult the free set, and the documentation now names them instead of leaving them to be discovered: the rigid-body step is committed only if R-free improves, and the alternative indexing is the candidate with the lowest R-free. Both are single discrete decisions rather than fitted parameters, both are the conventional use of a test set, and both leave R-free very slightly optimistic where they fire. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 10:31:24 +02:00
leonarski_fandClaude Opus 5	2574c2e587	docs: two changelog entries that are not a user's business ... One described a fix to a defect this same release introduced - the two-pass guard learning to weigh axial reflections, which exists because the indexed-frame floor moved in this release and exposed it. A reader upgrading from the last release never saw the defect, so the entry above it already says the only thing that changed for them. The other announced a regression-testing script. It is a repository tool, not part of what a user installs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 07:30:51 +02:00
leonarski_fandClaude Opus 5	24dcaf4e67	rugnux_vs_pdb.py: measure a corpus against the models its depositors published ... A companion to rugnux_vs_xds.py, asking a different question. That one compares merging statistics against another program's reduction of the same images; this one asks what an independent party already decided the images were worth - the deposited model, its published resolution and its published R-free - and compares against that. The runs are deliberately bare: --model and nothing else. No space group, no cell, no resolution limit. Anything told to the program is a thumb on the scale, and the question is what a bare invocation reaches on its own. The R-free it reports is a validation number rather than a refinement result: the deposited model is placed rigid-body against our merge and the free residual of that placement is what is compared. The depositor's published R-free came from full refinement against their own reduction, so it is the harder number, and the comparison is deliberately unfair in their favour. Entries are identified by directory name, which must be the PDB entry ID; metadata and coordinates come from the RCSB API. The sweep is resumable, since a hundred datasets is hours. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 07:21:06 +02:00
leonarski_fandClaude Opus 5	e02dd2e924	rugnux: the two-pass guard weighs the reflections a space group is read from ... The guard that chooses between the file's geometry and the refined one judged the two passes on CC1/2 alone. CC1/2 is an average over tens of thousands of reflections and the principal axial rows hold a few dozen, so a pass can win it while measuring half the axial reflections the other one did - and the systematic absences that decide a screw axis are read from nothing else. An axial reflection crosses the Ewald sphere over a narrow range of the sweep, so a pass that loses frames loses whole rows rather than a scattering of reflections. Measured on a crystal that was losing a screw exactly this way: the refined pass held 7 of the 16 low-order axial reflections and won the gate by 0.0114 in CC1/2, which is inside its own noise. The screw evidence fell from 24.2 nats to 10.5, under the bar, and the run reported the symmorphic group. A third arm, beside the two the guard already had: refuse the refined pass when it holds under three quarters of pass 1's axial reflections AND its CC1/2 does not say it is better. It can only decide cases the gate was deciding on a difference inside its own noise, and it needs pass 1 to hold enough of the row for a screw to be claimable at all. Over the corpus it fires on three crystals: the one it was built for, which recovers its screw; a second that gains 0.057 in CC1/2 and was being handed the wrong geometry for reasons nobody was looking for; and a third that loses 0.009. The other 145 are untouched. This closes a regression the six-spot floor exposed. The floor did not cause it - at the old floor the refined pass was broken outright and the CC1/2 arm rejected it for unrelated reasons, which rescued this crystal by accident. What the floor removed was the accident. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-07 07:18:59 +02:00
leonarski_fandClaude Opus 5	2c4b804f70	docs: the scaling model carries all three deterministic corrections, as the code does ... The observation model said its L term was the prescaling correction, and that predictions carry its reciprocal as prescaling_corr. That was true when that one field held everything. It has since been narrowed to Lorentz x polarization, with the sensor's efficiency and the flight path carried beside it, and every site that corrects an intensity multiplies all three - the scaling fits, the merge ingest, the anisotropy analysis and the unmerged export alike. So the model term is the product, not the first factor of it, and a reader implementing against that page would have applied a third of the correction. The header comment on the efficiency factor had the same fossil: it said the total correction is prescaling_corr * qe_corr, written before the flight path existed. The comment two fields below it already said all three, so the file disagreed with itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:59:40 +02:00
leonarski_fandClaude Opus 5	fc41b1c34c	release: version 1.0.0-rc.167 ... VERSION bumped and update_version.sh run, which rewrites the version string everywhere it is recorded: the OpenAPI specification and the three clients generated from it, the frontend package and its version module, the Sphinx configuration, the Redoc page, and the FPGA HDL and PCIe driver strings. Every hunk is a version string and nothing else, checked rather than assumed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:45:37 +02:00
leonarski_fandClaude Opus 5	2538f5a956	docs: the manual describes the program this branch actually built ... A sweep of docs/ against the code, and a changelog a user can read. The report reference still described the section layout from before the report was rewritten, claimed anomalous keys were withheld from Friedel-merged runs when 144 of 148 stored reports carry them, and listed as default a set of keys that --developer now selects. The tutorial described an enantiomorph message that no longer exists; the option tables were missing --developer and --finalist-ledger and carried a short option the program does not have; the integration page's unmerged-MTZ column list predated the two new columns and asserted the absence of one of them. The analysis pages had the efficiency and flight-path corrections but none of the symmetry work: the pseudo-translation detector, the absence test that divides it out, the glide test, the evidence-keyed alternatives and the metric re-ask are now written up where the methods are described. The usage message omitted FLIGHT from the formula that turns a written intensity back into a raw count, while the writer has been emitting the column. The usage message is the authority on what the program does, so it says so now. The changelog had thirty-one entries filed under the previous release, most of them written from the inside: what a change did to the code rather than what it does for a reader. This release gets its own section with sixteen, each one capability. Every key, flag and column a user could grep for survives; what goes is the seam between one person's work and the next's, which is not something a user can act on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:44:09 +02:00
leonarski_fandClaude Opus 5	c4abcc2591	diagnostics: an image that was never measured reads as unknown, not as zero ... Every per-image vector the scan-result builder fills with value_or(NAN) was sized with the value-initialised zero, so an image that never reached the fill loop at all - dropped, never arrived, numbered outside the run - came out of the END message carrying a measurement it never had. Zero is not missing data. A spindle blind fraction of 0.0 reads as "this frame lost nothing", a mosaicity of 0.0 as a perfect crystal, and a resolution estimate of 0.0 as nothing measurable at all, and each of those is a claim the run is in no position to make. Sized with NAN instead, which is what the loop already writes when the quantity was measured and came back empty, and what the readers and plot builders already test for. The counts keep their zeros: a count of zero is a thing that can be true. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 22:15:20 +02:00
leonarski_fandClaude Opus 5	e8fbae879a	rugnux: the two offline instruments go, and one release states one report version ... The finalist-ledger and glide instruments were built to measure a question each, over stored merges, and both questions have been answered. They were never asked for and they are not part of the product: neither is installed, both duplicate what rugnux itself does with a flag, and each is one more executable to keep linking. The --finalist-ledger mode stays where it belongs, in rugnux. Their removal also retires the CUDA device-link property they would have needed in the portable build, which is the shape of problem an unused target creates: a configuration nobody builds, breaking for a target nobody runs. The changelog claimed four different REPORT_VERSION values inside one unreleased section, which is development history rather than something a user can act on. A release states its report version once; the entries keep what they changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 21:53:17 +02:00
leonarski_fandClaude Opus 5	4b13098143	build: the offline instruments get the CUDA device-link step too, and the spot floor says what it drives ... Two corrections from an independent review of this branch. CUDA_RESOLVE_DEVICE_SYMBOLS was set on rugnux alone. The two offline instruments beside it link the same analysis library and therefore pull the same relocatable-device-code object out of libcufft_static, so in the portable-plus-CUDA configuration - which is the intended Windows build - their host link fails on an undefined __cudaRegisterLinkedBinary_* symbol. The property belongs on every executable that links the library, not on the one that happened to be tested. And the comment on the indexed-frame floor claimed less than the constant does. The same number is also the order statistic the candidate scorer ranks cells on, the stopping condition of the iterative residual contraction, and the adoption gate of per-frame refinement. Those ask whether a fit can be trusted, which is a different question from whether a frame had spots to spare; they share one constant only by history. Recorded rather than split, because splitting them changes behaviour and would need its own measurement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 21:15:24 +02:00
leonarski_fandClaude Opus 5	ac1e8b8603	tests: the report version pin and the screw-gate section follow the code ... Two test expectations that the changes under them made stale, and one overstated sentence in the docs. REPORT_VERSION was bumped to 12 without touching the line that pins it - the very line whose comment says a key added to the report is a contract change and this is where it has to be acknowledged. Acknowledged, and the absence of SOHNCKE_SPACE_GROUP on a fixture that is GIVEN its group rather than searching for one is now asserted too, since that is what makes the key's contract honest. The section documenting that the E^2 gate is what saves a screw from fabricated violations no longer holds, because there are now two independent defences: with the gate off, the absent class still sits at 2% of its own row, so the zone is dead per reflection and licenses the absence evidence to override the count. The section is kept rather than deleted - it now pins the second defence, and a future P2 there would mean the deferral has stopped licensing a zone that is genuinely extinct. The docs said SOHNCKE_SPACE_GROUP is written on every run. It is written on every run whose group came from the search; a run given its group with -S has no Sohncke candidate to name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 21:12:42 +02:00
leonarski_fandClaude Opus 5	485bbeaf71	rugnux: a cell given with -C is matched in any equivalent description ... Every candidate the indexer forms is Niggli-reduced and primitive; the reference cell was compared against them exactly as typed. For a centred lattice those are different numbers entirely - a conventional F-cubic cell reduces to a rhombohedron with 60-degree angles, a centred monoclinic cell to axes it shares none of - so the true candidate failed the 5% test and the run reported that it had found no lattice. A user who supplied the cell as it is published was worse off than one who supplied nothing. The reference is now expanded into the primitive lattices its six numbers could stand for, one per centring, each reduced the way a candidate is; a candidate matching any of them is kept. The cell as typed stays in the set, so the indexer that returns the basis it was handed is unaffected. The cost is real and stated: a wrong cell now gets several chances to match rather than one. The controls that hold are on data - deliberately wrong cells are still refused, and every cell that worked before gives the same answer to the digit - but a synthetic lattice can be made to match through a centring hypothesis it has no right to, and no volume test tightens that without excluding a true candidate this corpus already contains. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:26:38 +02:00
leonarski_fandClaude Opus 5	78a941156b	symmetry: absence evidence overrides the violation count, one zone at a time ... A screw candidate was refused whenever more than a tenth of its predicted-absent reflections read as present, whatever the absences themselves said. On a strong axial row near the spindle those readings need not be structure factors at all: measured on one crystal, the same reflection reads +268 at one Ewald crossing and -44 at the other, a directional smear tail of the monsters beside it. Three such reflections refused a group whose absence evidence was the highest of every candidate scored, on exactly the axes the deposition names. So a class whose evidence clears the claim bar may now override the count. The likelihood has already priced those reflections in - they are inside sum_u - and still reads the class as extinct, so the count adds nothing it has not weighed, and adds it unweighted. The deferral is licensed ZONE BY ZONE, which is the part that makes it safe. The evidence is a group-level number while the count indicts particular zones, so an overwhelming genuine zone would otherwise pay another zone's debts: a four-fold screw row at 194 nats lifted the veto for two claimed two-fold rows reading 9 nats each with half their class individually present. Each zone carrying violations must now hold the deferral itself - dead at a nat per predicted absence rather than merely accumulating one over a long row, and not judged against a measured pseudo-translation, where the corrected evidence is a residue of the cone's own under-measure and the corrected count is the only instrument the modulation does not reach. Measured over the corpus: two crystals gain the screw their absences support, no crystal loses one, and every merged statistic is unchanged - a screw and its parent share a Laue class, so nothing about the merge moves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:26:23 +02:00
leonarski_fandClaude Opus 5	118746d8da	symmetry: indistinguishable groups are keyed on the evidence, not on a count of absences ... The alternatives a run reports were the candidates whose absent AND violation counts matched the winner's exactly. Two settings of one point group routinely predict a different NUMBER of absences on a zone the sweep never measured - one extra reflection on a row with no control class of its own, contributing nothing either way - and that one reflection was enough to drop a candidate the data support exactly as well as the one adopted. Measured on an orthorhombic wedge: four candidates at the same 100.7 nats with no violations, of which the two predicting one more unmeasured absence were excluded, and one of those two was the right answer. The report then said the space group followed from the systematic absences, on an axis whose row was never collected. Keyed on the evidence, a super-centering that nets the same count but over-claims still scores differently and is still not offered. The adopted group does not change - it is the same candidate either way - so this only widens what the run admits it cannot separate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:25:51 +02:00
leonarski_fandClaude Opus 5	f98cbf5373	symmetry: a reference correlation with too little to correlate is refused, not scored zero ... PickBestReindex discards only non-finite scores, so returning 0.0 when fewer than the minimum number of reflections matched made that a SCORE rather than an abstention. An operator that mapped almost everything outside the reference's coverage - and therefore had nothing to be judged on - beat a measured identity that merely correlated a little negatively, and the whole run was reindexed on no data. NaN is what the per-image resolver's ReferenceCC already returns in the same situation, against the same minimum. This makes the two agree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 20:25:39 +02:00
leonarski_fandClaude Opus 5	de1b9299cb	rugnux: the report names the best Sohncke group on every run ... A crystal built from chiral molecules - any protein - has no glide plane and no inversion centre, so where the search names a group carrying a glide, the group that crystal could actually have is a different one. The run knows both and printed only the adopted one as a key. SOHNCKE_SPACE_GROUP= is written whenever a space group was determined, and repeats SPACE_GROUP_NAME= where no glide was found. That repetition is the point: a key that appears only sometimes has to be tested for before it can be read, and the reader this is for - someone who knows their sample is a protein and wants the answer their sample can have - should be able to grep one name without first deciding what the run concluded. REPORT_VERSION is 12. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:33:57 +02:00
leonarski_fandClaude Opus 5	6773e8516d	symmetry: a metric two-fold the lattice search refused is asked of the intensities ... The Niggli character walk takes the first character its tolerance matches, and on a lattice that is nearly but not exactly hexagonal it matches hexagonal. Under a hexagonal holohedry no candidate point group can carry the two strongest two-folds the data actually have, so the search lands on the weakest one and the crystal is processed in a group of order two where it should have been eight. The metric group knows better than the character walk: it holds every rotation the cell can host, including the ones the named lattice system has no room for. Each of those is now put to the intensities directly - one operator, scored the way Stage A scores its own, on the same reflection population, with the same strong-reflection gate and the same E^2 cap. That last part is what makes the answer usable: normalising over the full resolution range of a merge whose outer shells are correlated junk reads a genuine two-fold at CC 0.05, and the same operator over the population the pipeline itself pairs reads 0.88. The flattening and normalising the search does at its start is now one function, so an operator can be asked about without enumerating a point group around it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:21:41 +02:00
leonarski_fandClaude Opus 5	324a2b0cdf	integration: the aperture guard watches the aperture, and the width law can be read at any resolution ... Two changes to the same file, both no-ops on every run today, measured bit-identical across the corpus on every merged statistic. The starvation guard remembers the pre-adaptive settings so it has something to fall back to, and it remembered them only when r1 moved. r1 is a proxy for the aperture, not the aperture: a prototype that widened the background radii at high angle while leaving r1 alone took the neighbour-starved fraction of a dense low-symmetry crystal to eight times the bound this guard exists to enforce, and the guard could not fire, because nothing had been remembered. It now tests all three radii. Today they move together, so nothing changes; the hole was latent, and latent is where it should be closed. The width fit computes how much wider a crystal's spots get at high angle - the 1/d coefficient of r80(d) - and threw it away, returning only the value at one resolution. The law is now returned whole and held at the ends of the measured range rather than extrapolated, since two coefficients may rest on as few as two bands. Read at that one resolution it reproduces the old number bit for bit, which is why the coefficients are carried in double. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:10:57 +02:00
leonarski_fandClaude Opus 5	a452797ce5	rugnux: the short-axis pass checks the premise it was built on ... The pass exists because a lattice row shorter than the standard search floor is reported at its first harmonic above it, so a cell it may rightfully adopt has a reduced axis below that floor. It never checked that. Where the alternative has no axis under the floor, the floor cannot be what the standard pass tripped over, and the integer volume ratio is an ordinary axis multiple - the question the axis-harmonic arbiter settles a few lines above, on the occupancy of the class the multiple adds. That is evidence about the reflections in dispute, where the tie this pass reads is a frame count a sublattice cannot lose; where the two disagree the arbiter is right, and this block was overruling it. Two crystals were being halved on that account. Both keep their cells now, and the small molecules the pass was written for are untouched: their reduced axes are all under the floor, which is the premise holding. The environment variable that held the pass off is gone with it. It was there to diagnose exactly this, and a check in the code is the better answer than a switch nobody sets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N	2026-09-06 07:10:43 +02:00