196 Commits

Include renames

Author	SHA1	Message	Date
leonarski_f	07c8b2ddd4	docs: the Rugnux card and the naming table mention refinement and live collections ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit f8d07397288137dd5e3a0e5c101f2036d51228bd)	2026-10-26 14:50:42 +01:00
leonarski_f	c3c5a32a0d	docs: rugnux getting-started pages - refinement, the full output list, a troubleshooting note ... RUGNUX: a model run also refines the model it placed; the model can be given as an input; only the unmerged MTZ is written by default; the _process.h5 is not a default output; a short "when a run fails" note. TUTORIAL: the .ins, the refined model/maps/blobs, the log and _modules.h5 in the output list; the report is not written with -o ""; stills: -X Auto picks ffbidx with a cell, a run over several files. FORMATS: the XFEL stills formats and model/reference inputs told by content. CALIBRATION: -C and -S as the calibrant. INSTALL: the CPU path runs every indexer and the refinement. MODEL_AND_REFERENCE: the refined model is written as .pdb too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 0062d72b95e996438ddc65dc343b526c84daa7fb)	2026-10-26 14:50:42 +01:00
leonarski_f	8edd0e5f3e	docs: rugnux options, model and export pages brought in line with the usage text ... RUGNUX_ADVANCED: --simulate-live/--fast-live rows, -e exclusive, stills behaviour of --beam-center-check, -S, --min-partiality (0.5), --capture-uncertainty, --adaptive-integration-radius (on for stills too), --detector-distance/--rot1/--rot2/--no-refine-tilt; --background-radial auto as it decides now; --refine-modules syntax; _cell.*_esd. MODEL_AND_REFERENCE: the default refinement of a placed model in the summary, the decision table and the step list; --mode scale does not refine. INTEGRATION: the .ins written beside every .hkl (ZERR on rotation). VIEWER: rugnux writes _process.h5 only with --no-merge/--write-process-h5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 51d2564356ba46bda8730c855ff8d3ee7c23d605)	2026-10-26 14:50:42 +01:00
leonarski_f	d8c14046b2	rugnux: unit-cell e.s.d.s - ZERR in a new SHELX .ins beside the .hkl, _cell.*_esd in the mmCIF, CELL_ESD_* in the report ... Rotation runs now estimate the standard uncertainties of the cell. Every strong integrated observation (I > 3 sigma, unclipped, unsaturated, off ice rings) gives |q| = 1/d at its observed centroid, which depends on the detector geometry only; q^2 = h^T G* h is linear in the reciprocal metric, so the components of G* the point group leaves free (projected from the group's rotation parts, any setting) are fitted by weighted linear least squares with 4-sigma rejection, and the GoF-scaled covariance is carried analytically onto a, b, c, alpha, beta, gamma (ties give equal e.s.d.s, fixed angles zero). Least-squares cell e.s.d.s are known to be too small (Herbstein 2000), so the same fit is made on 8 blocks of consecutive frames and their scatter / sqrt(8) is the second estimate; the written e.s.d. is the larger of the two per parameter. Stills get none. Outputs: <prefix>.ins next to every .hkl (TITL, CELL at the wavelength, ZERR with Z = order of the group, LATT/SYMM from the space group - positive LATT only with -1 at the origin - a placeholder organic SFAC/UNIT at 18 A^3 per non-H atom, HKLF 4, END); the mmCIF gains _cell.*_esd ("." where fixed) and _cell_measurement.reflns_used/theta_min/theta_max, and writes the cell to 4/3 decimals; report keys CELL_ESD, CELL_ESD_LSQ, CELL_ESD_BLOCKS, CELL_ESD_BLOCK_COUNT, CELL_ESD_REFLECTIONS, CELL_ESD_THETA_RANGE_DEG (+ dev CELL_ESD_OBSERVATIONS, CELL_DSPACING_FIT). On 13 small-molecule sets the block scatter is 10-40x the least-squares e.s.d. and is what gets written; it lands within ~0.3-5x of XDS's CORRECT.LP e.s.d.s. SHELXT solves from the generated .ins and SHELXL carries ZERR into its CIF su. Costs 12 ms on a small molecule, 0.16 s on 690k lysozyme observations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit e74a3be76261fe1b6c0ff7b80c21afc4768c46cd)	2026-10-26 14:33:38 +01:00
leonarski_f	22debb1be6	docs: the stills rocking-curve datasets in the _process.h5 master ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 1195b9be6e0ca66774774f6de4867180b7bd0a47)	2026-10-26 14:09:32 +01:00
leonarski_f	00662c4110	Refinement: common cofactors and modified RNA nucleotides in the embedded dictionary; heme iron restrained from LINK records ... The embedded CCP4 Monomer Library subset grows from 62 to 84 compounds: - the modified tRNA/rRNA nucleotides PSU, 5MC, 1MA, OMG, OMC, OMU, H2U, M2G, 7MG, 2MG, 5MU (RNA group, joined by the library's backbone link "p"); - the cofactors NAD, NAI, NAP, NDP, FAD, FMN, HEM, HEC, SAM, SAH, and dioxygen (OXY). CMO is left out: its dictionary has C-O at 1.25 A, where bound CO is near 1.15 A. links_and_mods.cif gains the links these compounds use: HEC-CYS1/HEC-CYS2 (heme C thioethers), CYS-HEMB/CYS-HEMC, CYS-FAD and HIS-FAD1/2, with their modifications. AddMetalLinks now takes a metal atom of any residue, not only a one-atom ion, so a heme iron's LINK records to His NE2 / Met SD / water become bonds at the library's metal-site distances; the site's coordination counts the metal's bonds in its own dictionary (a heme's four pyrrole N) as well as the records. A metal within a ligand is also a metal for the repulsions (1.9 A to N, O, S) and its dictionary bonds are kept with the metal bonds. Note HEC.cif's own Fe-N(pyrrole) ideal is 1.90 A (HEM: 2.04 A), so a cytochrome c reports a metal-bond rmsd near 0.1 A. Test RefinementRestraints_deposited_ligands builds the topology of eleven deposited models cut down to the ligand and its surroundings (tests/test_data/ligands, 66 KB, from PDB 1EHZ, 8BKH, 6K9I, 2W3V, 5VML, 6F3P, 8CON, 3TK0, 7TCD, 6W4H, 5EPE): every ligand restrained, bond rmsd < 0.03 A, the RNA backbone bonded to each modified nucleotide, the heme Fe-His bonds, and the HEC thioethers. rugnux --mode refine on the full entries against their deposited structure factors: R-free within +-0.003 of the build where these were held fixed, ligand bond rmsd vs the dictionary from 0.004-0.058 A down to 0.002-0.042 A. rugnux grows by 425 KiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit f4d407c8de6cb462320edb21a70665350e436d28)	2026-10-26 13:56:29 +01:00
leonarski_f	6491cc5a7e	docs: serial-stills indexing ambiguity described the same way everywhere ... Two mechanisms, as the code has them: with -S and no reference or model, the crystals made consistent from the data (Brehm & Diederichs objective, cell-metric anchors); with -z or --model (-C/-S), per image against the reference, crystals their own cell decides keeping that indexing, the data-only result as a check. Removes the statements that the ambiguity needs a reference or model (tutorial, quick start, overview, §10.9), adds the anchoring to the reference/model rows, the "no -S" case (mixed merge, can pass for higher symmetry, warning), and the report keys in the reference case (AMBIGUITY_DATA_DISAGREES in place of AMBIGUITY_REINDEXED, AMBIGUITY_TIME_S, none in --mode scale). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 8db1d3ecb97dba90471c4db238aff632d11bc3e8)	2026-10-26 11:15:54 +01:00
leonarski_f	0cc3129650	docs: refinement's GPU engine in GPU data analysis, refinement in the pipeline figures ... GPU_DATA_ANALYSIS.md: the refinement section at the level of the other engines - density gather, cuFFT r2c/c2r, compose + likelihood target, CSR adjoint, warp-per-atom gradient, restraints and slot gather, fine mask and robust scale fit (on-device median), reductions and determinism, stream priority and blocking-sync waits, trial slots over cards, DeviceBytes budget, CPU fallback, no Metal; indicative RTX 5080 timings. Pipeline figures: a dashed Refinement box after Structure & map on both, the model branch labelled as deciding nothing; the stills row gains the indexing-ambiguity step before the first merge. RUGNUX_OVERVIEW.md: captions, a refinement paragraph, refined outputs under Write. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 8645fb2aca1abbbddec29cdfa99056471f8f38a5)	2026-10-26 11:15:54 +01:00
leonarski_f	e406eb17ea	docs: refinement chapter (§15), REFINE_ report keys, acknowledgements and in-source credits ... - New docs/CPU_DATA_ANALYSIS_REFINEMENT.md (§15): the ML amplitude target and its FFT gradient, scale/robust fit/fine solvent mask, D and the variance (work/free), the embedded Monomer Library subset and topology, geometry/metal/KL-B restraints, the minimiser, the weight trials, refined maps, blobs, GPU/CPU, and a "Not yet" list; with its own References. Linked from TECHNICAL.md and CPU_DATA_ANALYSIS.md; §14 no longer says the structure is never refined. - RUGNUX_REPORT.md: a Refinement (REFINE_*) section - every REFINE_ and REFINE_SCORE_ key, the --mode refine input keys and the _blobs.json fields. - RUGNUX_MODEL_AND_REFERENCE.md: "What it does not do yet"; RUGNUX_ADVANCED.md: --model mentions the refinement, --no-refine as in the usage, a --mode refine options table; GPU_DATA_ANALYSIS.md: the refinement's CUDA engines and the missing Metal path. - ACKNOWLEDGEMENT.md: Brunger, Karplus & Petsko 1989 (gradient-balanced starting weight), Nocedal 1980 (two-loop recursion), Charbonnier et al. 1997 (pseudo-Huber), Jiang & Brunger 1994 (flat solvent mask), Kullback & Leibler 1951; all DOIs checked against Crossref. - In-source credit lines: the weight balancing (replacing a "Brunger (1992)-style" line that named the R-free paper), Nocedal 1980, Charbonnier 1997, Jiang & Brunger 1994, Emsley et al. 2010 at FindDensityBlobs, Read 1986 at the refined maps. Comments only. - docs/THIRD_PARTY_NOTICES.md regenerated from the root file with update_version.sh's own step (it lacked the CCP4 Monomer Library row). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 09375dba351bae0cc73e9a6f0f79455dc92b3a10)	2026-10-26 11:07:43 +01:00
leonarski_f	f960bd7a80	Docs landing page: a real diffraction frame in the hero, and the command with a model ... The generated spot-and-ring drawing is replaced by one 0.2 deg rotation frame of the in-house lysozyme standard (EIGER2 16M, SLS X10SA): Bragg spots on their lunes, the beam stop and its arm, the module gaps. docs/landing/hero_frame.py makes the 920 px WebP (67 KB) from the master file - crop about the beam centre, log tone map from the frame's own background, block mean+max downsampling - and is run by hand; the WebP is committed and copied by the landing-page step. The dark theme shows it inverted through a CSS filter. The hero's command line drops its explanatory sentence and gains the second form, rugnux <diffraction image file> <CIF/PDB model file>. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit ff26000bfe7446cb4fda739c3b72596f8a8b7047)	2026-10-26 10:53:25 +01:00
leonarski_fandClaude Opus 5.5	6da51dc5d7	rugnux: no --refine flag - a placed model is refined by default, --no-refine skips it ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-26 06:54:15 +01:00
leonarski_f	3882b0f062	Serial stills with a reference or model: crystals their own cells anchor keep that indexing ... The per-image reference/model choice of indexing ignored the crystals' refined cells. On a pseudo-merohedral cell, where at least 90% of the crystals' cells fit the run's cell better as indexed than after a twin law, the cells decide that law - the rule the data-only resolution already uses (AnchorOnCells, factored out of ResolveReindexReferenceFree and shared). The resolver now keeps each image's CC per indexing (IntegrationOutcome::reference_cc); before the stills merge, AnchorReferenceReindex puts every image back as indexed, applies the cell rule, and lets the reference choose only among the indexings left open. Exactly merohedral lattices (no metric signal) are unaffected. Still battery (REFMAC 0-cycle R-free, model before -> anchored; data-only in brackets): pal90 0.1883 -> 0.1834 (0.1832), alvra213 0.2073 -> 0.2011 (0.2013), alvra213_int16 0.2081 -> 0.2021 (0.2024); euxfel83 0.2037, euxfel152 0.2452, sacla237, lcls222 unchanged; lcls242_t 0.2847 -> 0.2952 (0.2942): there the model had moved about half the crystals against cells that decide all three laws (96/96/100%). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 85c804676ffd15ffd66e2ba05928000d6f4e765f)	2026-10-26 06:49:20 +01:00
leonarski_fandClaude Opus 5.5	ba52aadbce	rugnux: the geometry refinement algorithm is -r, --geom-refine; --refine was declared twice and its no-argument form shadowed it ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-26 06:49:20 +01:00
leonarski_f	a78bfe951d	rugnux: a model and a reference among the inputs, in any order, told by their contents ... `rugnux <images> <model> <reflections>` in any order, without --model or -z. Each positional file is classified by its contents (RugnuxInputKind.h, in JFJochRugnuxOutput - the library still reads no files): an MTZ (magic) or an mmCIF with a _refln / _diffrn_refln loop is reference data (-z; the input itself in --mode refine), a PDB file (a PDB record first, then ATOM/HETATM) or an mmCIF with an _atom_site loop is a model (--model), anything else is images for the readers. CBF (###CBF or _array_data) counts as images; a file whose first line is neither data_ nor a PDB record is decided there, so image files are not read through. An mmCIF holding both atoms and reflections is refused as ambiguous. At most one model and one reference, --model/--mr and -z included. Gzip read through. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 8f844b0bf86f338d20e8aed426a387722dd8d7cc)	2026-10-26 01:38:25 +01:00
leonarski_f	794db0ca34	Refinement cut by the run's cancel; the hero shows the refined R-work / R-free ... The refinement of a placed model gets the run's cancel flag (RefinementSettings::cancel, through CancelFlag::RunFlag), so the broker's deadline or a cancel - and Ctrl+C in rugnux - cuts a running refinement short with what its finished trials found. RefinementResult::cancelled goes to the live status as rugnux_live_refinement.cancelled (API, regenerated clients) and to the step's note. Hero: once refined, the model card's headline is "R-work / R-free, refined" (e.g. 0.32 / 0.35), its green/red still the placement verdict, the placement and fit in the context line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 395016a30c2bc2aa26fdf2cec2ced9cebe6fad80)	2026-10-26 01:38:25 +01:00
leonarski_f	baec34685d	Rugnux live: the uploaded model is refined after placement; refined scores, model, maps and blobs in the status, the viewer and the hero ... The live run refines a placed model as `rugnux` does (ProcessConfig::refine), after every decision and before the end message; a cut before it starts leaves it out, as the model. The writer still gets the placed model; the refined one stays with the broker. API (broker/jfjoch_api.yaml): - rugnux_live_model.refinement (rugnux_live_refinement): performed, reason, on_gpu, seconds, r_work, r_free, rmsd_bonds_A, rmsd_angles_deg, blob_count, blobs (at most 50, rugnux_live_blob: kind, x, y, z, peak_sigma, volume_A3, nearest_atom, nearest_atom_distance_A) - rugnux_live_status.refined_model - /rugnux/model.cif and /rugnux/map.ccp4 take stage=placed|refined, by default the refined model where there is one; the anomalous map stays on the placed model's phases (the refined coefficients carry none). Regenerated broker/gen, frontend/src/client, redoc, and the python client docs (which also brings the hero fields and rugnux_live_model it was missing). Frontend: the model card keeps its green/red and gets "R-free 0.xx (refined)" as its context line, "refining..." while the step runs; the model section adds refined R-free and blob tiles; Mol* shows the refined model and maps, with the blobs as small spheres coloured by reading (switch, off at first). The Refinement step is in the timeline like any other. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 3b47fb9a53b7dfad1c1ed289e169094c8d3d9d3e)	2026-10-26 01:38:25 +01:00
leonarski_f	7bf906e4a2	rugnux: a placed model is refined by default - giving a model is the opt-in ... Refinement (Refinement.h) now runs whenever --model or --mr places the model (rigid body fits, or molecular replacement placed it), on the GPU or the CPU: a run that should not spend the time is given no model. --no-refine skips it (for debugging); --refine is still accepted and ignored. A model that was not placed (rejected by the rigid body, not placed by MR - e.g. the battery's wrong-model null) is no longer refined. Processing-inert as before: it runs after every decision; R_WORK/R_FREE and every key outside REFINE_ (section 5), and <prefix>.mtz / _model.cif / _maps.mtz, are unchanged. The block moved after the placement notes so they stay on the step that placed the model. The viewer's processing jobs refine too. The equivalent command line gives --no-refine back where refinement was off. Battery: score.py reads keys exactly, so REFINE_R_FREE cannot alias R_FREE; new report-only fields rfree_refined, rwork_refined, refine_s (part of rugnux_wall_s), shown in the per-set table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit b0bb741831f517b553e59152b3b482a3f75d953a)	2026-10-26 01:38:24 +01:00
leonarski_f	3b06bcee78	Refinement: the trajectory ends at its best step, steps try the positional weight halved too, the final ladder must win ... Three changes to how the trajectory of commit 2bfb84bc4 moves and ends: - A step is taken while its trial lowers R-free or R-work by more than the tie and its R-free stays within the tie of the best so far; the trajectory then ends at its best step. On a deposited model, whose free set it was fitted to, R-work falls while R-free rises, and the trajectory now ends where it began instead of walking on while R-work falls. - Positional steps try the weight halved as well as doubled: at a fixed weight the geometry loosens as the model improves (the X-ray gradient grows with D) until the trials leave the bounds, which stopped 8dqb's trajectories at 2 A (angles > 2.0 deg at w0/128 by step 6). - The final ladder's choice replaces the trajectory's end only where it displaces it (geometry within bounds, R-free lower by more than the tie); before, its first rung was taken by default, two more cycles of movement that on deposited models cost R-free. Deposited models (14 parity sets), R-free after refinement vs refine-combined: mean +0.0058 -> +0.0016 (e.g. 7n2s +0.012 -> +0.003, 9rcs +0.013 -> -0.008, 7d98 +0.015 -> +0.003, 9yl4 +0.010 -> +0.003), the time halved against refine-combined. MD starts, median R-free (REFMAC 0-cycle scorer), previous commit -> this: 5f6m 0.167 -> 0.168, 8dqb 0.294 -> 0.297, 7arr 0.278 -> 0.290 (REFMAC 0.311), 11if 0.244 -> 0.245, 5reo 0.258 -> 0.259, 7d98 0.335 -> 0.335. Determinism: run A == run B == --trials-at-once 1 on 5reo, 9rcs and a 5f6m MD start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit c2b55aab1aaff637d9484dec0d25003b5c4a3857)	2026-10-26 01:38:24 +01:00
leonarski_f	bda90dbdd5	Refinement: B restraints in proportion to B (Kullback-Leibler form) with contacts, an own ADP weight, overall B into the atoms ... - B similarity: E = (B_i - B_j)^2 / (kappa^2 B_i B_j), the symmetric Kullback-Leibler divergence of two isotropic Gaussians (Murshudov et al. 2011), so sigma = kappa sqrt(B_i B_j) instead of a fixed 1.5-4.5 A^2. kappa 0.1 / 0.133 / 0.2 / 0.3 for main-chain bonded / angle-related and side-chain pairs (REFMAC's logs list sigma ~0.1 B for main-chain bonded pairs at B 11-140; the ratios are those of the old sigmas, which these equal at B = 15). New class: atoms within 4 A that no bond or angle relates, waters and symmetry mates included, kappa 0.5 (BuildBContacts, sharing the neighbour search with BuildRepulsions). CPU and GPU. - The B factors' X-ray weight is balanced on its own (BalancedADPWeight: B-restraint over X-ray B-gradient on the model with B shaken by 20%) instead of being a multiple of the positional weight; the ladder tries B held and refined at that weight, the trajectory's B steps divide and multiply it by four, positional steps and the final ladder leave it. - Before each B-refining macro-cycle the overall isotropic B of the scale moves into the atoms and the solvent's B (F_model unchanged), so the two do not drift against each other. Tried and not kept: xyz and B as one L-BFGS vector (the B block scaled by its curvature, the positions by the median X-ray curvature): no better than alternation with a memory per block on 12 starts (mean R-free 0.265 vs 0.262), worse on 7arr/7d98; a refit before each B step: no gain for +20-40% time. Refinement battery MD starts, median R-free by the REFMAC 0-cycle scorer, previous commit -> this (REFMAC to convergence): 5f6m x8 0.167 -> 0.167 (0.168), 8dqb x8 0.291 -> 0.294 (0.284), 7arr x4 0.270 -> 0.278 (0.311), 11if x4 0.245 -> 0.244 (0.251), 5reo x4 0.312 -> 0.258 (0.261: the ladder no longer keeps B held where B is the model's largest error), 7d98 x4 0.342 -> 0.335 (0.361). Deposited models: unchanged (B stays held there). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit ed292f35b482b696d81f71f1d97e0016ca05b144)	2026-10-26 01:38:24 +01:00
leonarski_f	4424d8884d	Refinement: an own L-BFGS whose memory outlives the macro-cycle, B steps scaled by their curvature ... Each macro-cycle of a weight trial used to start a fresh Ceres solve: the L-BFGS memory was dropped every 10 (positions) or 5 (B) iterations and every cycle began with a steepest-descent step. RefinementMinimiser keeps one memory for the positions and one for the B factors across a trial's macro-cycles (the target's scale and likelihood parameters are refitted in between, which the memory tolerates), with a backtracking Armijo line search and Oren-Shanno scaling of a diagonal initial inverse Hessian. The diagonal: for B, the X-ray curvature per atom - the likelihood's curvature in |F_model| per resolution shell against the atom's (s^2/4)^2 f^2 exp(-B s^2/2), calibrated once per trial by a central-difference probe along a random direction - plus the B restraints'. For the positions none: tried with the restraints' Gauss-Newton diagonal (bonds and angles, ~1e4 per axis against the X-ray's ~1e2) it slowed the minimiser, since that stiffness is in directions that move bonded atoms together. Same budget (10 cycles x 10 + 5 iterations, same ladder), median R-free (REFMAC 0-cycle scorer) from the refinement battery's MD starts, base -> this: 5f6m (8 starts) 0.250 -> 0.210, 8dqb (8) 0.387 -> 0.345, 7arr 0.383 -> 0.356, 11if 0.325 -> 0.295, 5reo 0.316 -> 0.326 (CA RMSD 0.48 -> 0.43), 7d98 0.416 -> 0.393; REFMAC converged: 0.168 / 0.284 / 0.311 / 0.251 / 0.261 / 0.361. Deposited models (14 parity sets): R-work lower, R-free +0.0035 on average (their free sets were fitted to; the minimiser now moves the bottom rung further). Time unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit dfba4c1bd3c9345f31b64fd1cadc805810960825)	2026-10-26 01:38:24 +01:00
leonarski_f	6075239be0	Refinement likelihood: D on the working set, the measurement error per component, as REFMAC 5.8 estimates them ... The likelihood's variance per bin (beta) is fitted by maximum likelihood on the free reflections as the whole spread of Fo about D Fc, and D is then fitted on the working reflections with the variance eps beta + 2 sigma^2 (acentric; + sigma^2 centric), which is also the target's variance. Murshudov et al. (2011) Acta D67 355: D from the working set, the variance from the free set. The measurement error is thereby counted twice, so D and the figure of merit fall where sigma is a large part of the spread - at high resolution. Measured on REFMAC 5.8.0431 0-cycle output (black box) of the 14 parity sets: its FOM inverted per reflection gives var = eps a + 2 sigma^2 (acentric, fitted coefficient 2.00) and eps a + sigma^2 (centric); its D (FC_ALL/FC_ALL_LS = the log's SigmaA_Fc1) and a are reproduced by this estimate (on REFMAC's own Fc, one set to rms 0.009 in D over all 20 bins). The previous per-bin free-set ML put D at 0.39-0.48 where REFMAC has 0.06-0.13 in the outermost bins. rugnux score vs REFMAC 0-cycle, 14 sets: mean |FOM difference| 0.056 -> 0.012 (max 0.112 -> 0.042), per-bin D rms 0.107 -> 0.068; R-work/R-free unchanged (D does not enter them). The score's FOM is now the mean over the working reflections, as REFMAC reports it, and the refined model's maps are weighted by the refinement's own m and D rather than a separate sigma_A estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit d00d635e2874bff6cd216b7f24367362d0cc2e56) (cherry picked from commit b93e0ea2fc1ffcc6bb40dae9b77816a321f4d6ec)	2026-10-26 01:38:24 +01:00
leonarski_f	bcb71136ab	Refinement restraints: metal ions bound to their donors by the model's LINK records ... Metal sites were held only by the 1.9 A metal contact of the repulsions; REFMAC restrains the metal LINKs of the coordinate file (its documentation), which was the one geometry term the 0-cycle comparison did not share (one 2.8 A set: bond rmsd 0.018 against REFMAC's 0.042). - monomers/metals.cif: the Monomer Library's own statistics of metal sites (metals.json, LGPL data as the rest of monomers/), cut by update_monomers.py to the metal ions of its compound list and O, N, S donors: mean distance, spread and number of sites per metal, donor element and coordination number. - AddMetalLinks (before prepare_topology): every LINK / struct_conn record between a metal ion and an O, N or S that no link of the dictionary describes (its ZN-CYS, ZN-HIS, FE-CYS, FE-HIS are taken as they are) gets a link of one bond at EmbeddedMetalDistance for the two elements and the ion's coordination - the number of its donor records, symmetry mates included - from the rows resting on at least 20 sites (the coordination's own row, else the neighbours interpolated), sigma their spread (>= 0.02 A); a His or Cys donor loses the proton the bond replaces (DEL-HD1 / DEL-HE2 / CYSmod1, as the library's links do). Records only, no distance search; records to a symmetry mate count for the coordination but are not restrained (bonds to symmetry mates are not, anywhere). - The metal bonds are a restraint set of their own (RefinementTopology::metal_bonds), evaluated with the geometry and reported apart (GeometryStatistics::rmsd_metal, REFINE_RMSD_METAL_BONDS, REFINE_SCORE_RMSD_METAL_BONDS / _BONDS_ALL): their sigma is the spread of real sites, so they sit further from ideal than covalent bonds, and the weight ladder's 0.02 A bound stays a bound on the covalent ones. refine_parity.py compares REFMAC's bond rmsd (links included) with the combined one, and takes --gpu. Score-only geometry against REFMAC 5.8 0-cycle, all bonds (count): 2.8 A Zn/Fe set 0.0402 (4934) vs 0.042 (4935), was 0.0178; a Zn-Cys set 0.0186 (7102) vs 0.019 (7102); a Ca set 0.0297 vs 0.026 (REFMAC also refines the sugars and uses one Ca-O distance); a Mg set 0.0114 vs 0.017 (REFMAC restrains 8 symmetry links and puts Mg-O at 1.99 A, the library's statistics at 2.07). R of the scored model unchanged (+0.0001 on the first from the removed His/Cys protons). Refined R-free with the links vs without: +0.0016, -0.0002, +0.0005, +0.0004. Test: RefinementRestraints_metal_links (Zn-Cys4 by the dictionary's link, Mg(H2O)6 by the new ones, both back at their distances). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 0fcffc5c71e03e184cff18f07524473462a57b68)	2026-10-26 01:38:24 +01:00
leonarski_f	a218c0bec8	rugnux --mode refine: refine a model against merged structure factors, nothing processed ... rugnux --mode refine --model model.cif -o out data.mtz The input is an MTZ or a structure-factor mmCIF, gzipped or not, by content (writer/RugnuxReflectionFile: parsing in the file layer, the library reads no file). Columns are found, not named: F/SIGF-like amplitudes (calculated amplitudes and map coefficients passed over), else intensities made into amplitudes by French-Wilson, else the mean of the anomalous pairs; the free set from a FreeR_flag-like column (0 the test set, or the minority value), else rugnux's own 5 % set. RunRefineOnly (rugnux/RugnuxRefine) runs RefineModel; the outputs are --refine's (_refined_model.cif, _refined_maps.mtz, _blobs.json) and a report with the REFINE_ keys, the input's REFINE_FRENCH_WILSON / REFINE_FREE_FLAGS, and every term of the start score under REFINE_SCORE_ (also in --refine's section). --refine-score-only scores the model as given; --refine-d-min cuts the resolution. tools/battery/refine_parity.py runs REFMAC 5.8 at 0 cycles and rugnux --refine-score-only on the same files and prints the terms side by side, with R by resolution from both programs' model amplitudes on the same reflections. Tests: [refine_only] - column choice (observed over map coefficients, intensities, 1-marks-free flags, no flags, no usable column), an MTZ read from its bytes, and the mode end to end on a synthetic cell (French-Wilson, generated flags, score-only, the report keys). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 63ee830c39c988588cffb8c9d4638a678815ec2c) (cherry picked from commit d7397311e6ce0d3bb37ac91e8973b6052e38589f)	2026-10-26 01:38:24 +01:00
leonarski_f	c4d2d419ce	Refinement score: the model's structure factors as REFMAC 5.8 scores them at 0 cycles ... Three terms of the score of a model - the same F_model at every stage, the as-given start and the refined end alike - and a breakdown of it: - Riding hydrogens from the start: the dictionary's hydrogens are added before the model is first scored (REFMAC's 0-cycle restores them; they move R by 0.010-0.013 at 1.1 A). The engine is made once, with them. - The bulk-solvent mask on a grid of its own, min(d_min/3, 0.4 A), at most 2^26 points. On the density grid (1.1 A at 3.3 A) gemmi's 0.8 A shrink found no grid offset and was skipped, which put R 0.015-0.03 high at >= 2.5 A. 0.3 and 0.4 A agree to 0.001. - The overall scale and anisotropic B by robust least squares (pseudo-Huber, iteratively reweighted, c = median |residual|) in the fine pass of the k_sol/b_sol grid (ModelScaleBox::robust, refinement only; MR and validation keep the plain fit). Plain least squares on |F| is the exact LS optimum but lets a few badly-fitted strong reflections set the scale (1.1 A set: R 0.1706 against REFMAC's 0.1600). The scale fit of refinement now runs on the CPU (no ModelScaleGPUPool). RefinementSettings::score_only scores the model as given and stops; RefinementResult::score_start carries every term (k, B, anisotropic B, k_sol, b_sol, mask spacing, R in 20 shells, -lnL work/free, the likelihood FOM, hydrogens; per-reflection terms with score_only). GeometryStatistics gains rmsZ. rugnux_refine_test: --score-only, --dump. Measured against REFMAC 5.8.0431 ncyc 0 (refi bref iso) on the same model, data and flags, 14 open-arm sets 1.1-3.6 A: R-work/R-free within 0.002 on 12 (was +0.003..+0.03); 7n2s and 9yl4 score 0.003/0.005 below REFMAC, which a 0.6-0.8 A mask grid reproduces. REFMAC with ANISOU stripped scores identically, so isotropic B is not a parity term. Open: the likelihood's D/FOM (ours 0.01-0.11 higher at high resolution); geometry differs only by REFMAC's metal LINK restraints and residues outside the embedded dictionary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 4f94b21611874ad9dcf89e6771fe311a0e10bb06) (cherry picked from commit 00db6a6f539ac2da515c0a649474335f4c4b07b8)	2026-10-26 01:38:23 +01:00
leonarski_f	07ae590e54	Serial stills: the indexing ambiguity resolved from the data when no reference or model is given ... Where the space group is given (-S) and its lattice admits alternative indexings, a stills run without -z or --model used to merge the crystals' random indexings together (and warned). The crystals are now made consistent with each other before anything is scaled (ReferenceFreeReindex): each takes the indexing in which it agrees best with all the others (Brehm & Diederichs 2014), agreement being the sum over shared reflections of the products of shell-normalised, per-crystal standardised intensities, weighted by 1/(1+(sigma/sd)^2). The sum over all pairs is (|M|^2 - const)/2 for M the running sum, so a pass over the crystals is one pass over the reflections rather than over all pairs: coordinate ascent, four seeded starts, stop when <= 1 crystal in 1000 moves, the overall relabel that moves the fewest crystals. Deterministic (fixed summation order, sequential passes). A twin law of a pseudo-merohedral cell whose axes differ by a few per cent is decided by the crystals' own refined cells, which the indexer has already used: where >= 90% of the crystals' cells fit better as indexed, those crystals are anchors and only the others are decided by the intensities. Without this, two public sets got worse (pal90 R-free 0.183 -> 0.207, lcls242_t 0.296 -> 0.350). With a reference or model the per-image choice stands; the same resolution runs as a check, reported as AMBIGUITY_DATA_DISAGREES. New report keys (section 2): INDEXING_AMBIGUITY_RESOLVED, AMBIGUITY_CRYSTALS, AMBIGUITY_REINDEXED, AMBIGUITY_SEPARATION_SIGMA, AMBIGUITY_DECIDED_FRACTION (+ AMBIGUITY_TIME_S, dev). --mode scale is not changed. Still battery, no --model (R-free, REFMAC 0-cycle against the deposited model; before -> after; with --model for comparison): euxfel83 0.321 -> 0.207 (0.204), euxfel152 0.346 -> 0.257 (0.245), alvra213 0.209 -> 0.201 (0.207), alvra213_int16 0.210 -> 0.202 (0.208), pal90 0.183 -> 0.183 (0.188), lcls242_t 0.296 -> 0.294 (0.285); controls sacla237 / lcls222 unchanged (no ambiguity, no-op). The step takes 0.2-1.6 s on 2.7k-54k crystals; synthetic 64k crystals x 200 reflections 0.5 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 80d378a1fe92364fd5b99a17f5c9ee8498412d36)	2026-10-25 23:26:32 +01:00
leonarski_fandClaude Opus 5.5	2daa173859	docs: --model on serial stills - breaking the indexing ambiguity is the one thing taken from it before the merge, and why ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-25 20:10:38 +01:00
leonarski_f	cf7676f5d5	Stills merge: outlier rejection on by default (10 sigma), the band linear in the error model's sigma ... The stills merge had no working outlier rejection. --reject-outliers was 0 for stills, and when set its band took the systematic part as a factor exp(+-n s), s the spread of ln(I / median) - on stills a reflection's observations are partials and per-crystal scales, many near zero, so s is wide and the band let through observations hundreds of sigma out. On a high-multiplicity broad-bandpass run with a spotty powder contaminant, single grains in a weak reflection's box were that reflection's whole merged intensity (half-set means -2 vs 2284 on 141 observations); the shells holding the powder rings and the lowest shell merged at CC1/2 0.6-0.75 beside 0.99 elsewhere, at multiplicity ~400-1000. Stills now test the plain linear band, n sqrt(a sigma^2 + (b <I>)^2) about the median - the error model's b is the spread as it is - at n = 10 by default (rotation keeps 6 and its multiplicative band). The stills merge statistics now report the rejected count (OBSERVATIONS_REJECTED was always 0 there). Measured (REFMAC 0-cycle R-free of an isomorphous room-temperature model, depositor's free set, scale mode on stored integrations, vs the same code with --reject-outliers 0): - broad-bandpass in-house stills, 50k frames: 0.318 -> 0.265; 20k: 0.274 -> 0.244; R_split 0.29 -> 0.06, CC1/2 0.79 -> 0.98. 0.1% of the observations are rejected. - monochromatic in-house stills, 30k frames: 0.216 -> 0.212; R_split 0.126 -> 0.087. - n = 6 gains ~0.003 more there but costs the public XFEL sets 0.003-0.006 on three of four; 20 keeps most of the gain. Still battery, 12 non-float sets, against rc180: R_split better on all 12; R-free change mean -0.0005 (range -0.006..+0.003, plus pal86 +0.009 that is within that set's run-to-run spread of ~0.007 - a rerun of rc180's behaviour gave 0.192 against the baseline's 0.185); one draw becomes a win (lcls242_s), nothing else changes class. Rotation is untouched (the stills band and default are gated on the experiment not being a rotation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 3a7142801056f21c720956255f1207e0bf1a8f43)	2026-10-25 19:58:38 +01:00
leonarski_f	7759643d4c	rugnux --refine: refine the placed model, list its difference-map blobs, report under REFINE_ keys ... Opt-in. After the model's placement and everything it decides (setting, indexing, enantiomorph, MR, twin choice), the placed model is refined against the merge it was placed on (RefineModel) and the blobs of the refined mFo-DFc map are listed (FindDensityBlobs). Nothing reads the outcome: the reflection files, the space group and R_WORK/R_FREE of the model as given are those of a run without --refine. Writes <prefix>_refined_model.cif/.pdb, <prefix>_refined_maps.mtz (FP FC PHIC FWT PHWT DELFWT PHDELWT FOM FREE) and <prefix>_blobs.json; the report's model section gains REFINE_* keys (start, placement and restrained R-factors, rmsd bonds/angles, clashes, fixed residues, weights, blob counts by kind) and the five strongest blobs. Documented in RUGNUX_MODEL_AND_REFERENCE.md and CPU_DATA_ANALYSIS.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit aaee92711a205bf6a575a47c4db4555d34e7e8fe)	2026-10-25 07:20:32 +01:00
leonarski_f	b4b68d368f	Restrained refinement on the GPU: ML amplitude target, FFT gradient, dictionary restraints, R1 + R2 driver, blobs ... RefineModel (Refinement.h) refines a placed model against the merge, reading and writing nothing: - placement stage (R1): every chain of 50+ atoms as a rigid body at once (L-BFGS on six parameters per chain, their gradient from the per-atom one through Ceres jets), then one B shift per residue; - restrained stage (R2): positions and isotropic B of every atom of a residue the embedded dictionary describes, the rest held fixed but kept in Fcalc and the mask; riding hydrogens added by gemmi's topology, moved with their parents, re-placed every macro-cycle. Macro-cycles refit the mask, the scale and bulk solvent (FitModelScale / ModelScaleGPUPool) and the likelihood's D and beta (ML on the free set in resolution bins), rebuild the repulsion pairs (symmetry mates included), then L-BFGS on positions and, where chosen, on B. - weights: a ladder of full refinements at multiples of the gradient-balanced X-ray weight, chosen by R-free within bond/angle bounds (ties to the tighter geometry), then the B refinement tried at its own weights and kept only where it beats B held by more than the tie. The target and its gradient (RefinementXray, RefinementGPU): the Rice/centric amplitude likelihood (RefinementMath.h, shared host/device code), the gradient by Agarwal's FFT method - the adjoint of the reciprocal-space symmetry composition into a half-l (CPU) or half-u (cuFFT) spectrum, one inverse real transform, then each atom's density derivatives (xyz and B) summed over its box. On the GPU everything per evaluation stays on the device: ModelDensityGPU's gather, r2c, a fused compose + likelihood kernel, a CSR gather for the adjoint (no atomics), c2r, one warp per atom for the gradient; the restraints are one kernel per restraint plus a per-atom gather of gradient slots; cub reductions. Deterministic. Restraints (RefinementRestraints): bonds, angles, torsions (with esd, no phi/psi, no alternative sugar puckers), chiral volumes, least-squares planes from gemmi's Topo; repulsions beyond 1-4 (vdW radii from the dictionary's energy library, 2.6 A for H-bond pairs, 1.9 A metal-N/O/S); B similarity of bonded and angle-related atoms. DensityBlobs (D1): gemmi's flood-fill blobs of mFo-DFc above 3 sigma peaking above 4 sigma, placed beside the model and labelled WATER / ION / LIGAND / MODEL_ERROR by distance, contacts, volume and the anomalous map. tools/rugnux_refine_test refines a model against a finished run's p.mtz (developer utility, not installed). Tests: likelihood derivative, the embedded library, restraint values on ideal residues and every restraint kind's gradient against finite differences (CPU, and the GPU against the CPU), the X-ray gradient against finite differences in P1, P212121, C2 and P6122 (CPU and GPU), and a synthetic refinement that recovers a shaken model. Credits in docs/ACKNOWLEDGEMENT.md and at the algorithms. Measured on the open arm's battery outputs against REFMAC 5.8, 10 cycles, same p.mtz, free set and start model (R-free): 5reo 0.2228 vs 0.2236, 8dyz 0.1733 vs 0.1651, 9chw 0.1995 vs 0.1969 (CPU engine). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 8efe6f2322b3bfdae580bb34f256e67730dfc3c2)	2026-10-25 07:20:32 +01:00
leonarski_f	8ff2abf3aa	docs: a standalone front page with the three programs and the downloads, the Sphinx tree one page below it ... docs/landing/ is a self-contained page (HTML, CSS, a little JS, no framework): a hero, a card per program, the download section with the archives of the newest release and the reader's own platform marked, the licence in one paragraph, light and dark. The jfjoch_downloads extension renders it into the HTML output root as index.html at the end of every Sphinx build, so it is served wherever the documentation is - Read the Docs, the gitea-pages branch and the broker's frontend bundle - and its download buttons are the same ones as on the Sphinx download pages, built from the configured release and repointed at the newest published one by the shared script. The Sphinx root page moves to documentation.html (root_doc), and the frontend's documentation frame opens that page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 64cde61f0b941b3e8c99eaa9b74afeaa75aefc78)	2026-10-25 07:20:29 +01:00
leonarski_f	9c543f60b6	docs: home page with the three programs, download pages for rugnux and the viewer, technical reference grouped below ... The home page introduces Rugnux, the viewer and the acquisition system in three cards, with the licence stated once, and the sidebar is regrouped by program: what a user of Rugnux or the viewer needs comes first, and the algorithms, formats, interfaces, hardware and FPGA pages sit under one collapsed "Technical reference" hub (TECHNICAL.md). No page moves, so every existing URL stays. Download pages (RUGNUX_DOWNLOAD.md, VIEWER_DOWNLOAD.md) are rendered by a small Sphinx extension (docs/_ext/jfjoch_downloads.py) from the `release` in conf.py, which update_version.sh rewrites, so no version string on them is maintained by hand; a script then asks the Gitea API for the newest published release and repoints the buttons at its assets, naming any archive that release does not carry. Without network access the build-time links stand. JUNGFRAUJOCH.md is the landing page of the acquisition side with its install pointers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 050442592504ce879dc51d78b6d3d1c58268a800)	2026-10-25 07:20:28 +01:00
leonarski_f	c62a67975e	Polarization plane: measured on the pooled background, refused beyond what a source can give, logged, and kept across passes ... The pre-scan's plane test read per-pixel sector medians of the mean projection. On a large photon-counting detector with a background of a fraction of a count per pixel over the 60 frames read, those medians were zero and the test returned no harmonic at all: on 21 of 255 battery sweeps (EIGER2 16M and JUNGFRAU 4M sweeps, the laboratory EIGER sweeps, the small-molecule sweeps at 20-25 keV) the correlation was NaN or within 0.01 of zero, so the plane could not be decided there. The background is now pooled over 9 x 9 pixels first, as the mask itself already is; 216 of 244 synchrotron sweeps then show their (horizontal) plane past both gates, against 159 of 229 before. Pooling also lifted one long-distance sweep (500 mm, 2theta under 21 deg, polarization can move its background by at most 4 %) over the correlation gate: its background carries a 10 % cos 2phi harmonic along image y, 2.9 times what a fully polarized source would give, on a horizontally polarized beamline. No polarized source exceeds a scale of 1, so a harmonic of twice that or more now decides nothing (PLANE_MODULATION_MAX). The pre-scan logs the correlation and the scale on every run that detects the beam stop (GetPlaneEvidence). A turned plane is kept on the run and re-applied where a pass starts again from an experiment taken before the decision while the shadow is already in the mask (the rotation measured-centre arm, the no-lattice pass), which previously went on with the stated plane. No battery sweep turns; where the plane is not turned the applied coefficient is unchanged. The degree is still never fitted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 50a145a4f1ddf6d58332a8ab745627dc36a2dc3d)	2026-10-25 07:20:25 +01:00
leonarski_f	fca2a77b0c	Scaling: the low-resolution limit bounds the scale fit, not the merge ... Reflections coarser than --scaling-low-resolution (default 50 A) are now merged and written - what maps and refinement want of the lowest resolutions - but stay out of every fit of the scale model, by the same path an ice-ring observation takes (excluded from scaling, kept for merging), under a name of its own: RotationScaleMerge's per-observation flag is now `unscaled` (UNSCALED_ICE | UNSCALED_LOW_RES; on the device OBS_UNSCALED = OBS_ON_ICE_FLAG | OBS_LOW_RES_FLAG), set from the reflection on the host and in the device ingest, and only its ice bit marks a merged reflection as on an ice ring. The stills merge drops its low-resolution cut; ScaleOnTheFly, StillsPartialityRefine and the reindexing correlation keep it. The merge statistics run over the whole sphere unless --report-resolution declares a range, so the reflections the beam stop hides count as missing. Which coarse reflections exist at all is the beam-stop shadow test's decision (a7610d847). Why not simply lift the limit: on 5nw5, 35 reflections beyond 50 A bent the scale model enough to move the merged 20-50 A intensities by +61 % and 10-20 A by +19 % (Wilson B 319 -> 382, REFMAC R-free +0.006). Kept out of the fit, the merged intensities at 10-50 A are unchanged to 0.2 % and Wilson B is 307. CPU subset battery against the previous commit (lp-cpu-v2 / lp-cpu-v1): REFMAC 0-cycle R-free 5nw5 +0.0004, 6oel +0.0005, others within 0.0001; rugnux's own R-free/R_model rise where the new >50 A terms enter it (lyso R_model 0.2714 -> 0.2739; the model is poorest there). Kept beyond 50 A: 7qij 61, 5nw5 35, 6oel 14 unique. In-house photon-counting sets unchanged, except insu_I_x06da_ref, where the delta-CC1/2 frame exclusion now removes 449 instead of 78 frames: refres R_meas 0.246 -> 0.167, ISa 33.5 -> 35.4, R_free 0.2443 -> 0.2427, anomalous signal 3.31 -> 3.38 (XDS: R_meas 0.349, ISa 25.1). Stills (lcls242_s/t, lcls222, euxfel83, sacla236) within 0.0002 of the previous commit. GPU (device ingest) and CPU give the same 5nw5 merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 32a9fbcb3ca67879ceb051c13f58d61fe85530f4)	2026-10-25 07:20:21 +01:00
leonarski_f	e97b902603	Beam stop: a reflection whose signal disk reaches the shadow is not integrated; no-data pixels are dead, not shadow ... The profile fit keeps a signal disk that lost pixels to the mask and fits the pixels that remain. That holds for a gap or a dead pixel, but not for the beam-stop shadow: the mask is drawn where the background is dimmed, and the Bragg half-shadow does not end there, so the rest of such a disk is attenuated too. Measured on summed rocking events against their symmetry mates (stratified by resolution and zeta, the detector edge excluded), observations whose disk touches the shadow read low by 12-60 % on 5uth, 7qij, 5nw5, 3r6o, 3meb (CCD and PILATUS alike) and level on the in-house photon-counting sweeps; observations with a clean disk are unbiased except on 5nw5, whose half-shadow reaches ~20 px past the mask. The test sits in MXAnalysisWithoutFPGA::IntegrateOnImage, so rotation, stills, the _process.h5 and --mode scale see the same reflections; with no shadow in the mask (the broker) it does nothing. The shadow finder also reports pixels holding a detector's no-data mark (PILATUS CBF module gaps at -1): they are the darkest pixels there are. The pre-scan now masks those as dead (bit 10) instead of beam-stop shadow (bit 9), so that the shadow test does not drop every reflection clipped by a module gap and the reported shadow share is the shadow's. A mean of zero is not taken as dead: a deep shadow on a still counts nothing in the projected frames. CPU subset battery vs rc180 (REFMAC 0-cycle R-free): 3meb -0.0004, 5uth -0.0004, 6h2p -0.0003, 6oel -0.0002, 3r6o +0.0001, 7qij/7rji 0, 5nw5 +0.002 (its lowest-shell CC1/2 0.78 -> 0.89, ISa 8.4 -> 9.3); in-house refres ISa up 0.1-0.3 on lyso/thau/insu/cytc/myob, half_image refres R_meas 75 -> 64 %, no set lost quality. Stills (lcls242_s/t, lcls222, euxfel83, sacla236): R-free within +-0.002, the noise of a perturbed run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit a7610d847694b5540dacc9ef609344ea7c7ae8fd)	2026-10-25 07:20:21 +01:00
leonarski_fandClaude Opus 5.5	672a375b31	v1.0.0-rc.180: VERSION and update_version.sh ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-24 16:57:12 +02:00
leonarski_f	b00c668e3a	v1.0.0-rc.179 (#90) ... * Rugnux: Stills processing needs only the unit cell and space group beyond what the file records - detector distance, beam centre, tilt, panel placement, saturation level, gain and polarisation plane are found from the images - and is up to twice as fast. * Rugnux: Stills can be indexed without a GPU, with a CPU implementation of the fast-feedback indexer (macOS, computers without a GPU). * Rugnux: Rotation and stills data are integrated in the detector's own units with one gain model, improving data from CCD detectors. * Web frontend: The Rugnux live page shows the placed model with its electron density and anomalous difference map in a Mol* viewer. Reviewed-on: #90 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-24 14:52:29 +02:00
leonarski_f	afbe844b20	v1.0.0-rc.178 (#88) ... * Rugnux: Processes serial crystallography data from XFELs - SACLA (MPCCD), PAL-XFEL and European XFEL (CXI), LCLS (XTC) and SwissFEL Alvra (JUNGFRAU with a CrystFEL module geometry) - with several files or Jungfraujoch master files merged in one run (`rugnux *_master.h5`). * Rugnux: Stills processing takes a known unit cell and space group as a starting point, finds the beam centre (and, with a cell, the detector distance) from the images, detects the saturation level in other facilities' files and refines the placement of individual detector panels. * Rugnux: The stills report gives R_split and CC1/2 from frame half-sets, comparable with CrystFEL. * jfjoch_viewer: Opens the same XFEL file formats. * Validation: A stills battery checks Rugnux against deposited structures from SACLA, PAL-XFEL, LCLS, European XFEL and SwissFEL. * Rugnux: More reliable convergence of rotation scaling, and live molecular replacement starts as soon as the model is uploaded. * jfjoch_broker: Initialize clears the previous dataset's bearer tokens, image buffer and live Rugnux status, and a slow CUDA JIT cache is reported at start. Reviewed-on: #88 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-21 11:20:01 +02:00
leonarski_f	7ed32b899f	v1.0.0-rc.177 (#87) ... * jfjoch_broker: Faster live Rugnux analysis, with the GPU indexers prepared at the start of a collection and kept between collections; Rugnux no longer writes `.ccp4` maps (the map coefficients are in `<prefix>_maps.mtz`). Reviewed-on: #87 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-19 16:07:31 +02:00
leonarski_f	8261b7354e	v1.0.0-rc.176 (#86) ... * jfjoch_broker: Optional live Rugnux analysis of DECTRIS rotation collections inside the broker, with merged data, maps and the placed model written beside the master file, the rotation lattice in the scan result, and a "Rugnux live" tab and dashboard row in the web frontend. * Rugnux: Faster processing, with molecular replacement and model validation spread over several GPUs and run on the Apple GPU on macOS. * Rugnux: The polarization stated in miniCBF and d*TREK headers is used, twinned crystals' space groups are decided more reliably, and a run that finds no lattice still reports every image. Reviewed-on: #86 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-19 07:03:28 +02:00
leonarski_fandFilip Leonarski	19b135186d	v1.0.0-rc.175 (#85) ... * Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-16 15:34:43 +02:00
leonarski_f	a395f358ef	1.0.0-rc.174 (#84) ... * Rugnux: Performance improvements on GPU and CPU (more of the pre-scan and of scaling on the GPU, faster CPU spot finding and crystal refinement), with unchanged results. * Rugnux: More robust processing - patches of persistently hot pixels are masked, an inconsistent merge triggers a retry at the measured beam centre, and builds targeting different CPU levels give the same results. * Rugnux: Improved scaling and merging - reflections with an overloaded pixel are dropped, as in XDS, sparse rotation sweeps are scaled more reliably, and French-Wilson amplitudes use an anisotropic Wilson prior. * Rugnux: Improved space-group determination - glide planes in groups without a centre of symmetry, screw axes from short or weak axial rows kept when a higher group is adopted, and more reliable decisions on twinned and pseudo-symmetric crystals. * Rugnux: Improved small-molecule processing - spots that grow wider than the integration disk and split spots are integrated over their measured footprint, sparse lattices are integrated on every frame, and the `.hkl` file holds unmerged scaled reflections (SHELX HKLF 4). * Rugnux: Reads Rigaku d*TREK SMV images (Saturn CCD), including detector 2theta and encoded pixel overflows; home-source (rotating-anode) datasets were added to the validation battery. * jfjoch_viewer: Fixed processing failing at the end with "Wrong JPEG library version" on Linux; the merge window shows the space group with proper subscripts and a checklist of crystal pathologies. Reviewed-on: #84 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-06 14:03:18 +02:00
leonarski_f	84228bf8be	v1.0.0-rc.173 (#83) ... * jfjoch_broker: Optional per-dataset authentication - statistics, images and plots can require a bearer token, which jfjoch_viewer supports. * jfjoch_viewer: Dark mode and a theme-matched colour scheme, a magnifier panel, and simpler contrast and background controls. * Rugnux: Multiple performance improvements on GPU and CPU (CPU-only processing up to 40% faster, faster image decoding on ARM), with unchanged results. * Rugnux: `--model` rigid-body refinement runs on the GPU, and the model-validation check is faster and more reliable. * Rugnux: Improved scaling and merging - error model, outlier rejection, absorption correction and French-Wilson amplitudes now agree more closely with XDS and ctruncate. * Rugnux: Improved integration - radial background on powder and ice rings, crowded rotation data keep their reflections, and CPU-only builds integrate large unit cells as GPU builds do. * Rugnux: More robust detector geometry - measured beam centre, X-ray bandwidth and goniometer rate, and geometry refinement accepted only on significant evidence. * Rugnux: Merged files are written in the standard setting, or in the setting of a reference MTZ, structure-factor mmCIF or model, with its free-R flags. * Rugnux: Richer report - ice and powder rings, further lattices, superstructure candidates and mosaicity, with warnings worded as prompts to check. * Rugnux: Clear error messages when a data set needs more GPU or host memory than is available. Reviewed-on: #83 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-29 15:57:32 +02:00
leonarski_f	6dfe065365	v1.0.0-rc.172 (#82) ... * Fixed `jfjoch_broker` cancelling every data collection with a CUDA "out of memory" error after long operation: GPU memory no longer leaks with each collection. * Rugnux scales a rotation sweep until the per-frame scales settle instead of for a fixed three rounds, and says so when they did not - merged intensities, and the space group, resolution cut and frame rejection read off them, change accordingly; `--scaling-iterations` is now the cap on that loop (default 100). * Rugnux places every frame of a marCCD, SMV or miniCBF series at the spindle angle its own header states, so a series with missing frames, or with angles written modulo 360, is no longer read at the wrong geometry or refused. * Every rotation run writes two diagnostic files beside its reflections: `<prefix>_detector.jpg`, the detector projection with the pixel mask and the detected beam-stop shadow drawn on it, and `<prefix>_plot.txt`, one row per image. Reviewed-on: #82 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-22 06:48:37 +02:00
leonarski_f	30b6800289	v1.0.0-rc.171 (#81) ... * Rugnux: basic support for CCD images (marCCD, SMV) and for gzipped miniCBF. * `jfjoch_viewer`: opens the CCD formats, and fixes to the dataset plots. * Documentation updates. Reviewed-on: #81 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-17 14:42:52 +02:00
leonarski_f	cb5a2f032a	v1.0.0-rc.170 (#80) ... * Fixed a `jfjoch_broker` crash during indexing: sorting no longer misbehaves on non-finite values, and GPU FFT indexer kernel launches are now error-checked. * rugnux needs about a third less peak memory to scale, merge and post-refine rotation data, with identical results. * `rugnux --model`: the placed coordinate file carries the space group its own coordinates obey, and says so when that is not the group the reflection files beside it carry. * `jfjoch_viewer`: fixes in the dataset plots, inspector and layout; spot markers lose their black outline by default (a checkbox under "Image features" restores it) and the highest-pixel markers are white boxes around the pixel. Reviewed-on: #80 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-16 18:17:46 +02:00
leonarski_f	9aae0c2ba7	v1.0.0-rc.169 (#79) ... * Building Jungfraujoch no longer needs zlib or Eigen installed on the machine, and the dependencies the build fetches are pinned and updated to current releases. * rugnux: improvements in indexing, lattice selection and geometry post-refinement, which index crystals that previously returned no lattice and keep the better of the two geometries a run measures. * rugnux: improvements in beam-centre measurement, beam-stop detection and space-group determination. * rugnux: the unit cell reported with a determined space group now obeys that group - a cell whose symmetry was confirmed from the intensities is re-refined under it, and a cell the group cannot describe is reported with a warning rather than as it stands. * rugnux drops the stretches of a rotation sweep whose removal measurably improves the merged intensities and reports what became of every frame, and decides the resolution cut on the crystal's own diffraction rather than on its ice rings. * The rugnux results report is machine-readable - every line that is not `KEY= value` data starts with `#` - and states the build it was written by, its authorship and its terms of use (`REPORT_VERSION= 8`). * `jfjoch_viewer`: improvements in the file manager (CBF frames beside HDF5 datasets, a remembered root), the dataset plots, the inspector and the image statistics, plus a settable font size, a view of the rugnux results report, usable performance over a remote display (`ssh -X`) and a reset of all settings to defaults; the reciprocal-space window is removed. * Broker fixes around DECTRIS collections and dark-mask calibration: re-initialising after a run that never started no longer freezes the broker, a cancelled calibration is abandoned instead of reported as done, and a collection whose start message never arrives ends by itself. Reviewed-on: #79 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-15 17:09:31 +02:00
leonarski_f	77bc0cfe52	v1.0.0-rc.168 (#78) ... * rugnux is substantially faster - a corpus of 145 rotation datasets processes in about two thirds of the time - with identical results. * A crystal whose lattice looks more symmetric than it is because the beam centre is off is no longer processed on the wrong cell. * rugnux prints at startup, and writes at the foot of every results report, a short acknowledgement of the X-ray research community whose methods it implements and of the open-source projects it builds on; `ACKNOWLEDGEMENT.md` now ships in every package beside `LICENSE` and `THIRD_PARTY_NOTICES.md`. Reviewed-on: #78 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-10 13:51:16 +02:00
leonarski_f	a39fd29f77	v1.0.0-rc.167 (#77) ... * `rugnux --model` reports CC(model, data) - the correlation of the merged intensities with the placed, scaled model - by resolution shell, on the same shells as CC1/2, with the reflection count and a significance for each. * `rugnux --model` fits the model's scale, anisotropic B and bulk-solvent parameters on the working reflections only, so the R-free it reports is measured against a model no free reflection helped scale. * The bulk-solvent parameters of `rugnux --model` are searched over their physically meaningful range instead of being fitted without bounds, so a model is never scaled with a solvent term that has silently switched itself off. * The rigid-body placement of `rugnux --model` uses the same bounded bulk solvent as the reported fit, so a model is no longer placed against a target carrying a solvent term with no physical meaning. * `rugnux --model` puts the model into the data's own description of the lattice before placing it, so a model whose cell is written on other axes - I-centred where the run indexed C-centred, a different unique axis, a permuted orthorhombic cell - is placed rather than scored where it was read; `MODEL_CHANGE_OF_BASIS=` and `MODEL_SETTING_AS_READ=` report it when it happens. * The rugnux results report opens with a summary - `VERDICT=` (`OK`, `WARNINGS`, `UNUSABLE`, `FAILED`), `VERDICT_TEXT=`, `PATHOLOGY_FLAGS=` with one closed-vocabulary code per condition that warned, and the `WARNING:` lines, which used to close the file - and the sections after it are renumbered 1-5 with no gaps. * `rugnux --developer` writes the full results report - the pipeline-internal keys and the long explanations the default report now leaves out - and `--finalist-ledger` adds the evidence for every space group the search considered, not only the one it adopted. * The results report warns when the merged data carry no usable signal and when too little of reciprocal space was measured inside the fitted resolution, and omits `FITTED_RESOLUTION` where the CC1/2 curve it is fitted on never falls off. * rugnux detects translational pseudo-symmetry and reports it under the `PSEUDO_TRANSLATION` flag as `TNCS_DETECTED=` and the `TNCS_*` keys - a translation the merged data are exactly invariant under is reported as `UNDECLARED_LATTICE_TRANSLATION=` under `LATTICE_TRANSLATION` instead - and a detected pseudo-translation can no longer buy a false screw axis in the space-group search or hide a twin from the L-test (`L_TEST_VS_TNCS=`). * The space-group search determines glide planes from zonal systematic absences, so a non-Sohncke space group such as P 2_1/c or Pbca is named where the run previously stopped at its Sohncke subgroup; `SOHNCKE_SPACE_GROUP=` carries the best Sohncke group beside it on every run that searched, and a centre of symmetry is never claimed. * Where the cell metric carries more rotational symmetry than the Bravais class the indexer named, the extra rotations are put to the intensities and the space-group search is asked again on the metric's own cell - adopted only where the intensities confirm the higher symmetry - so a lattice that is nearly but not exactly hexagonal, or whose reduction landed in a sub-cell, still reaches its true point group. * Systematic-absence calls rest on the evidence rather than on counts: a screw axis whose absent class the data show extinct is no longer refused because a handful of reflections in it read as present, and `SPACE_GROUP_ALTERNATIVES=` no longer drops a candidate that differs only on a zone the sweep never measured. * A reference correlation measured on too few reflections is refused instead of scored zero, so a run given a reference MTZ is no longer reindexed on an operator that mapped almost everything outside the reference's coverage. * A frame counts as indexed from 6 spots on its lattice rather than 9, so a weakly diffracting crystal whose frames cannot carry 9 is no longer refused the lattice it fits; `--min-indexed-spots` overrides it. * `-C` accepts a known cell in any equivalent description - conventional or primitive, centred or not - instead of only the reduced primitive form, so a centred cell given the way it is published no longer makes the run report that it found no lattice. * Each reflection is corrected for the sensor's quantum efficiency at the angle it meets the detector (attenuation lengths from the NIST tables, which also fixes the spot-width parallax term on CdTe) and for the attenuation of the flight path between the sample and its pixel; `--flight-path air|helium|vacuum` declares the medium - default air, since no file states it - and the report says what was assumed and what it was worth. The unmerged MTZ records the factors in new `QE` and `FLIGHT` columns beside `LP`, so raw counts are `I / LP * QE * FLIGHT`, and `_process.h5` in new optional `qe` and `flight` datasets. * Rotation geometry post-refinement fits the crystal and the detector at once, against the observed spot positions and the observed rocking angles together, so the refined distance depends far less on how wrong the file's distance was. * A coarsely sliced sweep integrates correctly: partials are joined into one rocking event by angle rather than by frame count, so two crossings of the Ewald sphere are no longer summed into one full, and at 0.5 degrees per image or coarser the per-frame geometry refinement accepts a spot whose miss the exposure's own rotation accounts for. * `rugnux --mode scale` reports the detector tilt and direct beam of the geometry it re-scaled at, instead of zeros that read as a flat detector, and no longer warns that no image was indexed on a run whose lattice came from its input file. * Every rotation run that determined a space group and merged reports what the mounting cost: `SPINDLE_LOST_UNIQUE_FRACTION=` is the fraction (0-1) of unique reflections the mounting made unmeasurable under the measured point group, also written to the master as `/entry/MX/spindleLostUniqueFraction` and what the mounting warning fires on; `SPINDLE_SYMMETRY_AXIS_ANGLE_DEG=` / `SPINDLE_SYMMETRY_AXIS_ORDER=` describe the mounting in the `--developer` report. * Stills and grid scans carry a per-image `spindle_blind_fraction` - how much of a rotation sweep's blind cone this orientation would make unrecoverable, 0.5 and above calling for a second orientation - through the CBOR stream, HDF5 (`/entry/MX/spindleBlindFraction`), the plot and scan-result APIs, and the viewer and frontend plots; an absent value means the frame could not be assessed and is not a 0. * The results report's `REPORT_VERSION` is 7. Reviewed-on: #77 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-09 07:25:13 +02:00
leonarski_f	680c36c20d	v1.0.0-rc.166 (#76) ... * `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands. * `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion. * Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants. * `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing. * A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed. * `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing. * Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences. * The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to. * The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after. * The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution. * `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have. * Twinning is no longer reported when the L-test contradicts it. * The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's. * `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots. * The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area. Reviewed-on: #76 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-02 21:17:31 +02:00
leonarski_f	511be0c366	v1.0.0-rc.165 (#75) ... * `rugnux --model` adopts the model's space group as a label where the data were merged in its enantiomorph, instead of reindexing the reflections - which swapped I(+) with I(-). * `rugnux --model` warns, naming the atom, when the anomalous density at the model's atoms comes out inverted, which means the data and the model are in opposite hands. * `rugnux --model` writes an anomalous difference map (`<prefix>_anom.ccp4`) when the merge kept the Bijvoet split, and names the ten model atoms it peaks highest on as `ANOMALOUS_SITE_01`..`_10`. * `MEAN_ATOM_DENSITY_SIGMA` is read from the map by cubic rather than linear interpolation and comes out around a tenth higher; it is no longer comparable with the figure earlier versions printed. * `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name. * A model `rugnux --model` cannot use is reported as a `WARNING:` line in the results report instead of only in the log. * The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given; `REPORT_VERSION` is 4, `WARNINGS` moves to section 11 and no existing key changed. * The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=`, `WALL_TIME=` and `GPU_COUNT=` / `GPU=`. * rugnux says which GPUs it can see before it starts processing. * `rugnux --export-unmerged` also writes `<prefix>_unmerged.mtz` on a `--no-merge` run, and is ignored on a run with no output prefix instead of writing a file called `_unmerged.mtz`. * `/start` asks the writer whether the run can be written before the detector is armed, so a run whose master file already exists, or whose output directory cannot be created, is refused up front with the writer's own message. This needs the TCP image stream or the built-in HDF5 writer; the ZeroMQ stream is unchanged. * A calibration that fails goes to `Error` carrying the reason instead of `Inactive`, so `/wait_till_done` and `/wait_until_running` report it; a cancelled calibration still goes to `Inactive`. * `/wait_till_done` answers 500 with the message when a collection ended in an error. A cancelled collection and a collection that only triggered a warning still answer 200. * A pending start failure is discarded by `/cancel` and `/deactivate`, as it already was by `/start` and `/initialize`. * `/scan_result` no longer reports the previous run's images after a collection that failed to start, or after `/deactivate`. * The TCP image stream protocol version is 4. `jfjoch_writer` and `jfjoch_broker` have to be of the same release, as before. Reviewed-on: #75	2026-08-27 22:16:54 +02:00