238 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5.5	76adf3f0fa	Merge origin/rc175 (macOS signing and notarization) into rc175 ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 18:47:07 +02:00
Filip LeonarskiandClaude Fable 5.1	82e5de1261	macOS: sign jfjoch_viewer and rugnux with a Developer ID and notarize them ... JFJOCH_MACOS_SIGNING_IDENTITY (the certificate's SHA-1) makes the install step sign the shipped program with hardened runtime and secure timestamp: macdeployqt's -sign-for-notarization for the viewer bundle, a codesign INSTALL(CODE) for rugnux. tools/notarize_macos.sh then submits the cpack artefact to Apple, waits for the verdict and staples the ticket to the .dmg; the rugnux binary inside the .tgz is submitted as a zip, as a bare executable cannot carry a ticket. Both macOS CI jobs resolve the identity from the runner's keychain, check for the hardened runtime and notarize on every push, so a rejection shows up with the change that caused it. The identity is passed as the hash, not the name: Qt's generated deploy script writes DEPLOY_TOOL_OPTIONS unquoted, so a name with spaces splits into several macdeployqt arguments. The notarytool profile has to live in the login keychain file (store-credentials --keychain): the default data-protection keychain becomes unreadable once the screen locks, which an unattended runner's does. Verified end to end on the runner: both artefacts accepted by Apple, the stapled image and a quarantined copy of the app and of rugnux pass Gatekeeper as Notarized Developer ID, signing and the credential lookup work from a launchd-started process with the console locked. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-10-14 15:01:30 +02:00
leonarski_fandClaude Opus 5.5	b574a0e600	docs: the min-pix 1 search gate is de novo only ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 14:59:04 +02:00
leonarski_f	cb044459ba	Merge commit '86f17740f' into stillsidx	2026-10-14 13:41:39 +02:00
leonarski_fandClaude Opus 5.5	86f17740f6	ACKNOWLEDGEMENT: Markstein reference without the working note ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 13:15:22 +02:00
leonarski_fandClaude Opus 5.5	51397c15f4	Merge FFT indexer GPU peak search rewrite into rc175: ~45% faster per call, output unchanged ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 13:15:17 +02:00
leonarski_fandClaude Opus 5.5	804206155b	PROTOTYPE (owner decision): stills min-pix choice searches the lattice once per frame, ~2x faster ... Changes results - not for merging without the owner's approval. The per-image min-pix scheme indexed every frame at min-pix 3, 2 and 1, each with the 30/80/all seed escalation, then indexed the winner again: 3.9 indexer calls per frame. Measured per frame on a serial lysozyme run (2000 frames, de novo and against the known cell): - where several lists index, the searches return the same lattice (97-98% of pairs); - the gain over a fixed min-pix 2 has two sources: (A) frames min-pix 2 indexes but whose min-pix 1/3 list indexes ~26 more spots (a better per-frame refinement), and (B) frames with almost no spot of two pixels (min-pix 3/2 lists of 1-3 spots, ~50 single pixels at min-pix 1) on which the min-pix 1 search fits 8-9 spots. De novo, 189 of 194 such B lattices had cells unrelated to the crystal's; against the known cell they index, but leaving them out raises I/sigma and lowers R_pim. Now: the lattice is searched for on the min-pix 2 list (min-pix 3 where that does not index); the first list that indexes hands its indexer answer to the others, which are only refined and scored (FrameIndexerCalls::lattice_found). The min-pix 1 list is searched on its own only where the min-pix 2 list holds at least ViableCellMinSpots spots. Ties keep the larger min-pix, as before. Measured on two full 24960-frame serial runs (RTX 5080, box loaded by other agents, load 9-20; loop rate is frames/s of the image loop; I/sig, R_pim, observations of the P41212 merge): known cell (ffbidx, -C): base 785 / 750 -> 1130 / 1017 frames/s (warm 6000-frame ABAB 860 -> 1453) run A: obs 5.37M -> 5.30M, I/sig 13.09 -> 13.02, R_pim 6.48 -> 6.46%, CC1/2 .970 -> .973 run B: obs 4.91M -> 4.89M, I/sig 12.00 -> 11.95, R_pim 6.50 -> 6.47%, CC1/2 .971 -> .970 de novo (FFT): base 89 / 91 -> 174 / 195 frames/s run A: obs 3.96M -> 3.91M, I/sig 10.18 -> 10.12, R_pim 6.94 -> 6.98%, CC1/2 .960 -> .967 run B: obs 3.91M -> 3.89M, I/sig 10.53 -> 10.52, R_pim 7.01 -> 7.03%, CC1/2 .965 -> .961 fixed min-pix 2, for scale: ffbidx 1496 / 1316 frames/s, obs 4.94M / 4.54M, I/sig 12.62 / 11.77. The indexing rate drops (ffbidx 59.5 -> 56.9%, 55.2 -> 52.4%; FFT 60.5 -> 52.1%, 60.2 -> 47.9%): the frames that go are the spurious B lattices; observations stay within 1.3%. Indexer calls per frame 3.89 -> 1.67 (de novo, 2000 frames); the min-pix 1 search is skipped on 58% of frames. De novo, 23 of 691 frames whose lattice was the crystal's now take a superlattice (13, same spots indexed) or a cell fitted to 6-12 spots of a weak frame (10) from the min-pix 2 search. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 10:34:04 +02:00
leonarski_fandClaude Opus 5.5	ab645847ed	FFT indexer GPU peak search ~45% faster per call, output unchanged ... calculate_fft_result (the per-direction prominence peak search) was 61-65% of the GPU kernel time of a de-novo stills run, and the card is what limits that loop. It is one dependent chain per direction, latency-bound. Two exact changes, taken where the spectrum holds a whole background window (any FFT max cell above ~30 A; the general loop stays for the rest): - the sliding window written out as its three phases (clamped at the start, centred, clamped at the end): the same additions and subtractions in the same order, so the running double sum is bit for bit the same, but the loads no longer sit behind data-dependent while loops and are issued ahead; - the window mean divides by a constant width, so the double division (a long FP64 iteration on a consumer card) is replaced by the reciprocal multiply plus one correction by the exact remainder, which returns the correctly rounded quotient (Markstein). Checked on the CPU against the division for 4e8 random dividends over all odd widths 3-401: 0 differences. Measured (RTX 5080, box loaded by other builds, load 12-25): kernel minimum 1.50 -> 0.83 ms, mean under the 8-stream pool 4.7 -> 2.8 ms; de-novo stills 2000 frames ABAB 113/120 -> 127/138 frames/s. p.hkl byte-identical (2000 and 3000 frames, base and new min-pix schemes). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 10:32:03 +02:00
leonarski_fandClaude Opus 5.5	593590b357	rugnux -X FFT is the algorithm: cuFFT on a GPU, FFTW without one; -X cuFFT names the GPU implementation ... In rugnux people choose the algorithm, not where it runs. -X fft (and the old -F) now resolves at parse time to the GPU FFT indexer where a GPU is visible and --no-gpu is not given, and to FFTW otherwise. -X cufft is the GPU implementation alone, as -X fft was. -X cufft and -X ffbidx without a GPU now stop at once with a clear message instead of failing later in the indexer pool. Auto is unchanged and not redundant: on stills with a known cell and a GPU it takes FFBIDX. Only rugnux's CLI spelling changes. The internal IndexingAlgorithmEnum, the broker and its OpenAPI enum keep FFT = the GPU implementation; the CLI maps onto it, and the command line the viewer writes for a run (RugnuxCommandLine) gives the enum's FFT back as -X cufft, which re-parses to the same thing. A run without -X, and -X fft on a machine with a GPU, process exactly as before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 10:13:23 +02:00
leonarski_fandClaude Opus 5.5	bfd715b64d	rugnux --no-gpu: run on the CPU only, without touching CUDA, on a machine with a GPU ... hide_gpus() in CUDAWrapper makes get_gpu_count() answer 0 without calling cudaGetDeviceCount, so every GPU decision in the process (indexer Auto -> FFTW, preprocessing/spot finding/decode, pre-scan, scaling engine, model validation, MR) takes the path it takes on a machine without a GPU, and no CUDA context is created. rugnux reads --no-gpu in main, ahead of the parser, because set_gpu_blocking_sync() already asks CUDA for the devices. The banner says 'CPU only (--no-gpu)'. The 'run it on the CPU' advice in the out-of-GPU-memory messages and in the install docs now names --no-gpu instead of CUDA_VISIBLE_DEVICES=. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-14 10:13:23 +02:00
leonarski_fandClaude Opus 5.5	6e9b891eb4	MR score-engine cache and merge buffers: no hang, no doubled wait; near-tie doc follows offline decide-late ... - MolecularReplacement: an engine whose making throws anything (std::bad_alloc, a gemmi error), not only a JFJochException, is refused and its reservation given back, so a thread waiting for the same engine is woken instead of waiting forever. - RotationScaleMergeGPU::Alloc waits for the GPU work beside itself (GPU_BUSY_TIMEOUT), so its buffers are allocated plainly Synchronous; SynchronousAfterWorkBeside added a second 10-minute wait in front of it. - CPU_DATA_ANALYSIS_INDEXING: an offline run decides its near ties on the whole sweep from the start; only a live run (broker URL or --simulate-live) decides early and re-checks. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-13 21:25:15 +02:00
leonarski_fandClaude Opus 5.5	7d64a62872	Merge branch 'plots-disposition' into rc175: <prefix>_plot.txt gains a per-frame disposition column (1 merged, 0.5 downgraded, 0 rejected) ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-13 15:44:34 +02:00
leonarski_fandClaude Opus 5.5	5901c75547	rugnux: _plot.txt gets a graded per-frame disposition column ... A new last column, "disposition", gives each frame's merge disposition as a number to plot directly: 1 merged, 0.5 downgraded, 0 rejected; nan for a frame that never got a scale (never reached the merge) and on a run that merged nothing, as the file writes every value nothing measured. Existing columns keep their indices. 0.5 is a marker, not a weight. "Downgraded" is a label applied after the merge (RotationScaleMerge::CountSweepDisposition marks the merged frames inside a flagged sweep-quality range); the merge applies no factor of its own to them - they are carried at whatever weight their own scale and sigmas give - so there is no fixed or per-frame reduction factor that could honestly be written instead. Writer-only change: on three open-arm sets with downgraded and rejected frames p.hkl is identical to the same commit without it, the first ten columns are unchanged, and the column's 1/0.5/0 counts equal the report's merged/downgraded/rejected counts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 15:44:04 +02:00
leonarski_fandClaude Opus 5.5	53c81dc5df	CHANGELOG: rc.175 gains a jfjoch_writer line for the writer-end-catch work ... Bounded memory and efficient closing of old data files on long runs, a failure ending the run in an error state the broker hears about, and the pusher's log line when a stalled writer makes it drop images; the master-file ordering moves here from the Fixes line. Owner-requested edit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 13:30:21 +02:00
leonarski_f	7b4dcce480	Merge rc175 into review-fixes-rc175	2026-10-13 13:30:00 +02:00
leonarski_fandClaude Fable 5.1	1be6093429	Writer: received images live in a byte-bounded ring buffer until written and republished ... Every stage between the socket and the HDF5 write was bounded by a message count and never by bytes: ZeroMQ receive watermark 100, cbor_fifo 200, outside_fifo 1000, repub_fifo 200 - up to some 1500 images in flight per writer process, each a whole compressed image, so 6-12 GB for a JUNGFRAU 4M and 30-60 GB for a 9M, and the writer's memory was a quantity nobody had chosen. ReceiveRingBuffer is a fixed ring of bytes owned by the ImagePuller base class, so the ZeroMQ and TCP pullers share it. A message is received straight into it (TCP reads the payload into the ring; ZeroMQ copies the message in) and travels as a shared_ptr region in ImagePullerOutput; the deserialised image points into that region, republish sends it as is, and the bytes come back when the last holder lets go. Space is reclaimed in arrival order: a region released out of turn - republish finishes with messages in its own order - is marked and waits for the ones in front of it. Because of that, what republish may hold is capped at a quarter of the ring, and an image over the cap is not republished, exactly as one that finds the republish queue full; otherwise a stalled republish consumer would pin the ring and slow the writer to its rate. A message larger than the whole ring (a START or END of a very long run) goes on the heap, so it cannot wedge the receiver. A full ring stalls the receive thread, and what that does depends on the stream. On TCP the pusher waits behind the writer's BUSY heartbeats and gives up only after a minute in which no image at all was written. On ZeroMQ the pusher's send blocks for one second, then drops the image and sends without waiting for the rest of the run - so on ZeroMQ a ring that is smaller than the writer's stalls costs images, where the old 1500-message queues absorbed them. That is the trade: the budget is now a number, and it has to be chosen. jfjoch_writer sets it with -m/--rcv_ring_mb, default 2048 MB per writer process; the ZeroMQ receive watermark default goes from 100 to 10 messages, since it now only counts what sits beyond the ring. The ring's peak use is in the end-of-run log line, and docs/JFJOCH_WRITER.md has a Memory section with the sizing rule. The puller's own default is no ring at all, which keeps every other user as it was: the broker's DECTRIS stream2 puller (JFJochServices, JFJochReceiverService) stays zero-copy with the ZeroMQ message as the buffer, and the broker's direct HDF5 writer (HDF5FilePusher -> FileWriter) never goes through a puller. The puller threads now also catch std::exception, as the writer does since the previous commit: deserialising the END message of a million-image run allocates a vector per per-image array, and a bad_alloc leaving the thread ended the process. Tests: ReceiveRingBufferTest (reuse, wrap-around, full ring, out-of-order release, heap fallback, zero size, stop, a region outliving its ring); ZMQImageCommTest_SmallRingBuffer (64 one-megabyte images through a three-image ring, nothing lost, peak within the bound); one of the two writers in TCPImageCommTest_2Writers_WithAck runs on a three-image ring. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-10-13 12:51:34 +02:00
leonarski_fandClaude Opus 5.5	8b43cce9eb	docs: Models and reference data - what a model or reference can and cannot change ... A new page, RUGNUX_MODEL_AND_REFERENCE.md, beside the other rugnux pages in the toctree and linked from RUGNUX.md and the --model section of RUGNUX_ADVANCED.md. It lists every option that brings in a model or reference data (--model, --mr, the PAE file beside a prediction, -z and --reference-column, -S, -C, --force-rotation-lattice, --report-resolution), sorts every decision into data-only, model-informed label in <prefix>.mtz (hand past the anomalous veto, indexing ambiguity, setting), model-driven separate files (<prefix>_mr) and report-only statistics, walks a --model run step by step, says exactly where a model can touch the space group and what guards it (the null, the anomalous veto, the indexing margin, the MR rules, the wrong-model battery step), and describes the free set and model bias (a deposited model's R-free on rugnux's own test set is not cross-validated; -z with the PDB's structure factors inherits the depositor's set). Includes the one pre-merge use of a model: serial stills with -C/-S, where it breaks the per-image indexing ambiguity. Documentation only; behaviour as merged in rc175 (7ed970503). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 09:05:18 +02:00
leonarski_fandClaude Opus 5.5	402b3d1293	ACKNOWLEDGEMENT: autoPROC and Clemens Vonrhein; DOIs verified; DISTL; tNCS credit in MR ... - All 84 DOIs checked (82 on Crossref, 2 DataCite): title, journal, volume, pages and first author match; Matthews 1968 and Yeates 1997 links percent-encode their parentheses; the MOSFLM link points at a page that answers. - autoPROC: an explicit acknowledgement of Clemens Vonrhein (Global Phasing) for the principle that a pipeline should verify the metadata it is given - beam centre, geometry, goniometer - and correct it where it is wrong; Vonrhein et al. (2011) Acta Cryst. D67, 293-302 (DOI checked). - Zhang, Sauter et al. (2006) DISTL, already credited in SpotFindingSettings.h, gets its entry. - Molecular replacement: tNCS looked for in the native Patterson as Read, Adams & McCoy do; the same credit at DetectTranslationalNcs (comment only). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 09:03:32 +02:00
leonarski_fandClaude Opus 5.5	1bd251cdd1	docs: GPU_DATA_ANALYSIS, every GPU engine of the crystallographic pipeline ... One page listing the CUDA engines by pipeline stage - decode and preprocessing, per-image analysis (azimuthal and ROI integration, spot finders, extractor), pre-scan accumulators, indexing, prediction and integration, the scaling engine, model validation and molecular replacement - each with what it computes, how closely it agrees with the CPU, and a link to the section describing the method. Added to the toctree after CPU_DATA_ANALYSIS_DECISIONS. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 09:03:31 +02:00
leonarski_fandClaude Opus 5.5	30cf5856d4	docs: bring the rugnux and analysis pages up to the code (MR fallback, GPU decode, live runs) ... - CPU_DATA_ANALYSIS_IMAGE §0: CBF byte-offset and marCCD widening are decoded on the card; the host decoders are the CPU build's path. - CPU_DATA_ANALYSIS_DECISIONS §14: the null runs for every model the rigid body is tried on; the device choice is made once on the card's total (structure factors and rigid body a quarter, scale fits an eighth) and a CUDA failure ends the validation, deciding nothing; a subgroup the model fits is written as <prefix>_mr; new §14.10 on molecular replacement (model kind, rotation and translation functions, likelihood, Matthews, tNCS and P1 rules, null, what it may change). - CPU_DATA_ANALYSIS: scope and references gain molecular replacement. - CPU_DATA_ANALYSIS_INDEXING: the near tie decided early and re-checked once every frame is in. - RUGNUX: merging is the default and _process.h5 optional; live following of a collection with a broker URL; --model's MR fallback. - RUGNUX_INSTALL, RUGNUX_OVERVIEW, RUGNUX_TUTORIAL, RUGNUX_REPORT: GPU work list, <prefix>_mr outputs, MR in --mode mx for rotation and stills, rigid body only in --mode scale. - RUGNUX_ADVANCED: -N default as the usage states it. - battery README: the ci tier has 28 sets. - ModelValidation.cpp: a comment still said a GPU failure reran the validation on the CPU (comment only). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 09:03:31 +02:00
leonarski_fandClaude Opus 5.5	5e916fe120	docs: --model places by rigid body, then molecular replacement; --mr; <prefix>_mr files ... RUGNUX_ADVANCED's --model page described the rigid body alone. It now says how the unified flow places a model: an experimental model by the rigid body, checked against its null in the answer's or the twin hypothesis' merge, and by molecular replacement where it is not placed; a prediction straight to molecular replacement; --mr skipping the rigid body; MODEL_PLACEMENT. It says what a placed model may change in <prefix>.mtz (the hand, and from the rigid body an indexing-ambiguity reindex) and that another group or the twin subgroup the model chooses goes to <prefix>_mr.mtz/.cif with <prefix>_mr_model.pdb/.cif. --mode scale places by the rigid body only (no molecular replacement, --mr refused); a full run does molecular replacement on serial stills too (checked: a stills run with a predicted model reports MODEL_PLACEMENT= MR). The options table gains --mr and the --model row the fallback. RUGNUX_INTEGRATION no longer says rugnux does no molecular replacement, and ACKNOWLEDGEMENT names --model beside --mr for predicted models. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 08:07:44 +02:00
leonarski_fandClaude Opus 5.5	645d1cc1bb	CHANGELOG: rc.175 says --model falls back to molecular replacement and takes predicted models; viewer refresh ... The --mr line now says what the unified --model flow does for a user: a predicted model is accepted, and a model the rigid body does not place goes to molecular replacement, with --mr forcing it. One viewer line for the Files panel refresh (F5 and the poll) and the combo boxes that no longer reset during live collection. Owner-requested edit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-13 07:41:37 +02:00
leonarski_fandClaude Opus 5.5	d0d225cb99	rugnux --model: molecular replacement where the rigid body does not place the model ... One --model now does the right thing by itself: - A prediction (AlphaFold/ModelCIF, read from the file) goes straight to molecular replacement. - An experimental structure is placed by the rigid body as before. The null now runs on every --model run (no NOT_TESTED): "placed" = the fit beats its null (4.51 sd) in the answer's merge or in the twin hypothesis' merge. Placed: behaviour unchanged (hand and reindex may go into p.mtz); a subgroup the model fits and prefers is written as <prefix>_mr. - Not placed (or no crystal cell): molecular replacement with the experimental-model path. It may relabel the hand into p.mtz past the anomalous veto, never reindexes; another group or the twin subgroup goes only to <prefix>_mr. Maps, R-free and CC come from validating the MR placement with every decision switched off; MODEL_FIT stays the rigid body's verdict. - The twin hypothesis' merge is made in memory with a model where a metric re-ask promoted the group or the model's group has fewer rotations than the adopted one; written and reported only where the model chooses it. - --mr is kept as "--model, searched for by MR straight away". - Report: MODEL_PLACEMENT= RIGID_BODY|MR|NOT_PLACED, MODEL_DECISIONS_TAKEN gains MR_ENANTIOMORPH, MODEL_CHOICE_FILE_PREFIX/SPACE_GROUP; the MR keys move into 5. MODEL VALIDATION; MODEL_FIT absent where no rigid body ran. MR NOT_PLACED is never a warning. - Battery wrong-model step: check (c) also fails a wrong model placed by MR. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-12 22:20:36 +02:00
leonarski_fandClaude Opus 5.5	f567771d56	CHANGELOG: rc.175 names --mr, which takes predicted and experimental models ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-12 20:08:50 +02:00
leonarski_fandClaude Opus 5.5	30c7bee15f	rugnux: --af-model becomes --mr, for predictions and experimental structures alike ... - The option is --mr <file>. What kind of model it is is read from the file: a prediction where it is ModelCIF (_ma_qa_metric) or has no cell and one B value per residue in 0-100 (AlphaFold PDB/mmCIF); an experimental structure otherwise. A prediction is handled as before (pLDDT >= 70 trim, B from pLDDT, PAE domains where the AlphaFold DB's PAE file sits beside it). An experimental structure is taken as its polymer - no waters, ligands, hydrogens or alternative conformers, occupancy 1, isotropic B as refined - with a coordinate error of 0.67 A (a prediction's at pLDDT 90) for sigma_A; a B column above 100 no longer stops the search. - The report section is MOLECULAR REPLACEMENT (EXPERIMENTAL), its keys MR_* (MR, MR_MODEL_FILE, MR_MODEL_KIND, MR_DECISION, MR_SPACE_GROUP, MR_COPIES, MR_HAND_CHECK, MR_SECONDS, ...); the <prefix>_mr outputs are unchanged. - The twin hypothesis' merge that is made only for the model to test (a metric re-ask promoted the indexed lattice's group) is written and reported only where the model chooses the subgroup. Raw deposited coordinates of nine control sets, unconverted: every one placed and confirmed against the deposition (all chains as one body), the groups the same as with the prediction or the prepared model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-12 19:30:22 +02:00
leonarski_fandClaude Opus 5.5	6ee167df9a	Merge rc175 (the null turns each copy about its own centroid) into mr-7qij ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-12 16:49:20 +02:00
leonarski_fandClaude Opus 5.5	c6912e2a06	rugnux --af-model: tNCS half-vector moves checked, the merge's own group not ruled out beside tNCS, P1's first copy judged by its orientation (WIP, validation pending) ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-12 16:49:02 +02:00
leonarski_fandClaude Opus 5.5	aabee7423f	CHANGELOG: rc.175 release notes ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-12 14:10:51 +02:00
leonarski_fandClaude Opus 5.5	0d33d7a40a	rugnux --af-model: likelihood judging, PAE domains, twin comparison, tNCS carry test, faster multi-copy search ... - Likelihood (Read 2001; Z-score over random positions as McCoy et al. 2005): log-likelihood gain of Rice/Woolfson E distributions, sigma_A from the pLDDT coordinate error with its scale refined, to 2.5 A (or the 20000 lowest-resolution reflections). Further-copy TF peaks ranked by the gain on the search's reflections (peaks that lower it dropped); a copy whose TF Z is below 8 also stands out where its refined likelihood Z-score reaches 8 - only where the copies placed are as many as Matthews makes most likely. The likelihood is computed only where a translation function does not stand out. Copy structure factors exact from one FFT of the cell's grid, or read off the molecular transform where it reaches the likelihood's resolution. - PAE domains: the AlphaFold DB's predicted_aligned_error JSON beside the model (same name pattern) is read automatically; average linkage on the symmetrised PAE (10 A, domains >= 40 residues). Where the whole model is placed nowhere, each domain is searched by itself side by side; placed domains must agree, and a domain alone may only confirm the answer's group. - Twin comparison: where the answer's merge chose and the twin hypothesis' merge exists, the chosen placement read as a crystal in the subgroup (its symmetry images as copies, the subgroup's setting found from the two cells) against the model searched for afresh in the subgroup, both against the data twinned by the law (fraction refined on the working set, Yeates 1997); the subgroup is taken by a 0.03 R-free lead plus its own null/margin -> TWIN_SUBGROUP, written to p_mr with law and fraction. With --af-model rugnux makes the subgroup merge wherever a metric re-ask promoted the indexed group. - tNCS rule: a group with fewer copies beside tNCS (screws bridged by the vector, or the enantiomorph) no longer leaves the choice undecided where the chosen copies, carried into it and refined, fit worse by 0.03 in R-free. - Speed: the molecular transform's FFT on the GPU (cuFFT, serialised), no rigid body for trials that lower the likelihood, re-search only with tNCS, PAE domains searched in parallel. Survey (193 sets, MR replay on stored merges): single GREEN 58 -> 68, multi 48 -> 58, partial 8 -> 9; 4bwl (and, in full runs, 2wnq and a withdrawn entry of the same protein) TWIN_SUBGROUP P21 instead of a wrong AGREES; a P21212 set with tNCS now SPACE_GROUP P21212 instead of UNDECIDED; no wrong call. CPU and GPU builds make the same decisions on 18 demo/multi-copy/twin sets. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-12 13:21:51 +02:00
leonarski_fandClaude Opus 5.5	fcbe92042b	Merge branch 'af-mr3' into rc175: --af-model searches several copies per asymmetric unit (Matthews probability, sequential translation function, fixed-copy rigid body, tNCS) ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-12 01:36:26 +02:00
leonarski_fandClaude Opus 5.5	63e0a85467	rugnux --af-model: several copies in the asymmetric unit, tNCS, a search-matched null, faster ... Copy number. The hard "Vm >= 4: MULTI_COPY, no decision" rule is gone. Copies are searched one after another up to the largest copy number with a Matthews probability of 1% (a Gumbel distribution with Kantardjieff & Rupp's 2003 mode and mean, no resolution dependence); COPIES_LIKELY reports the most probable. Sequential search. Copy n's translation function carries copies 1..n-1 as a known partial structure: the cross term 2 Re(conj(F_known) F(t)) goes on the same FFT grid (CPU FFTW and GPU cuFFT; KnownStructureFactors, AssemblyCorrelation). Its peaks, past those that clash, are ranked by the correlation of the whole assembly with E^2 (the TF itself, linear in |F|^2, rewards copying a placed copy onto an origin shift), the first that stands out (TF Z >= 8), packs and lowers R-free by 0.01 is kept; each copy is refined beside the others held fixed (RefineRigidBody's fixed copies), and the first copy again beside the second, the earlier ones once more at the end. Packing: ClashFraction counts C-alpha clashes with other copies and all their mates, binned (7mzt: 51 s of clash checks -> 1.9 s). Search quality: rotation-function peaks hill-climbed off the 6 deg sampling (6ze4: RF Z 10.4 -> 16.0, 3.3 deg off the deposited orientation -> 0.2); the first copy goes down the TF list until three placements that pack are refined, so a group is CLASH only where no offered orientation packs; groups are searched side by side, and every group the best one leads is searched again with the best one's copy orientations tried first, the better search kept. tNCS: the native Patterson of E^2 (5-10 A) off the lattice; placed copies' orientations join every further copy's search; and where the tNCS vector turns one group's screw axes into another's (TncsRelatesGroups), a group whose search stopped at fewer copies does not lose to the other - UNDECIDED. Measured on an in-house tNCS crystal (vector c/3 along the 3-fold): both hands fit with three copies to within 0.01 in R-free, and where one hand's search failed the other was taken as ENANTIOMORPH; now UNDECIDED. Null: the replicates are placed by the translation function in their own orientation, as the search placed the real one (a null at the real centroid reads high), and the gate is 4.51 = t(8) sqrt(1 + 1/9), 3 sd of a nine-draw null, the gate --model's MODEL_FIT is now read at. Speed: placements scored on the GPU structure-factor engines; further copies' trials score with the first copy's bulk solvent held, the assembly refitted at the end. Survey of 193 sets (jfjoch_mr_replay on stored merges, RTX 5080 shared with other jobs): MR median 1.15 s, max 24 s; demos 0.8-1.1 s. Report: COPIES and COPIES_LIKELY columns, MODEL_PRIOR_COPIES. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-12 00:27:17 +02:00
leonarski_fandClaude Opus 5.5	ee350f1f74	rugnux --model: a wrong model confirms no shell, fits no hand, and fits the GPU reserve ... Three defects the battery's wrong-model null found (5 sets x 5 deposited models of unrelated proteins): CC(model, data) significance. The per-shell sigma was Fisher's transform on independent reflections, but the data and a scaled model share the falloff across a shell, the anisotropic B fitted to these very data, the bulk solvent and the symmetry enhancement epsilon - whatever the model is. 21 of 24 wrong models read CC_MODEL_CONFIRMED_TO_D_MIN to the data's d_min (10-15 sigma in most shells). The correlation is now formed on I/epsilon and |F_model|^2/epsilon, and its null is empirical: the data at h against the model at h + d for the 26 nearest lattice offsets, which keeps everything smooth in reciprocal space and loses what the model knows of h. SIGMA = (z - mean) / max(sd, 1/sqrt(N-3)) in Fisher z over those 26, a new CC_NULL column shows the null, and a shell confirms only beside a coarser shell that also clears 3 sigma (one shell alone did so once in 216 wrong-model shells). Prototyped on the p_maps.mtz of a full open-arm run and the null runs: wrong models confirm 0/24 (was 21/24), the deposited models confirm 197/200 (was 198/200 in the prototype's reading; the one lost has CC_MODEL 0.05-0.13 in every shell). The report no longer says a model cannot agree by accident. MODEL_FIT gate. Accepted at 3.0 sample sds of a nine-draw null, as if the sd were known; an unrelated P1 helical model on a P3(1) helical crystal was ACCEPTED at +3.05. Against 48 draws it reads +2.09 and the best random placement fits as well as it does, and drawing the null about the cell axes instead changes nothing (+2.15): the cause is the nine-draw sd, not the orientation. The gate is now 3 sigma allowing for that: t(8) * sqrt(1 + 1/9) = 4.51. The deposited models of the open arm read +7.0 and up. The indexing lead stays at 3 sigma (INDEXING_MARGIN_SIGMA): it is read only behind the fit gate, and a deposited model settles one at +4.15. Rigid-body GPU reserve. The finest zone's observations were counted by the data's d, but the zones count them by the d of the index in the model's ASU, and a model's group need not keep the data's metric (an I2(1)3 model on I222 data: 2320 in the zone against 1712 reserved), so the validation died with "zone over the engine's reserve". Counted the zones' way, under every indexing probed. Tests: ModelValidation_CCModelUnrelatedModelConfirmsNothing (Fisher alone reads +4.7 on an unrelated model with an anisotropic falloff; the new sigma stays under 1), ModelValidation_WrongModelInAGroupOffTheDataMetric (fails with the old count), ResultReport_ModelValidationSection (CC_NULL column, lone-shell rule). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-11 21:30:34 +02:00
leonarski_fandClaude Opus 5.5	50e3b59fb7	rugnux --af-model: hand check reads only the anomalous map, on MR's placement and solvent ... The ENANTIOMORPH hand check ran a whole ValidateAgainstModel on the placed model: model read and frame probe, a rigid body, a 9-replicate null (~0.5 s of the ~0.6-0.9 s), sigma_A, three maps written. The decision needs only --model's rule: the model fits its null (the MR choice has already required its own null of the same construction) and the anomalous map at the model's atoms is not inverted. ReadModelAnomalousMap (ModelValidation) computes F_calc and F_mask to the data's resolution (GPU engine where there is one), carries them through the bulk solvent MR scored the placement with (the overall scale and anisotropic B do not move phases), and reads the anomalous map with the same code as the validation: its coefficient construction and its reading at the atoms are factored out of Validate (anomalous_coefficients, read_anomalous_map) and used by both, so plain --model is unchanged. Hand check 0.6/0.6/0.9 s -> 0.027/0.028/0.040 s on 11if/7ris/9zmu; same ADOPTED/VETOED on all seven cases tried (the three demos, two anomalous lysozyme sets and their mirrored-detector runs), with the scatterer means and deepest troughs within 0.1 sigma of the full validation's. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-11 18:11:01 +02:00
leonarski_fandClaude Opus 5.5	2a3d60fb82	rugnux --af-model: anomalous veto on the enantiomorph, search on the GPU, cheaper rescoring ... - An MR ENANTIOMORPH decision is taken into the answer's label only as --model takes a model's enantiomorph: ValidateAgainstModel runs on the placed model (in a scratch directory, removed after) and its adopted_model_enantiomorph - fit over its null, anomalous map at the atoms not inverted - decides; otherwise ENANTIOMORPH_VETOED. The hand-check numbers go to the MODEL PRIOR section. - Rotation and translation functions on the GPU (MolecularReplacementGPU.cu): one block per orientation for the rotation function; per orientation the copies, the bin weights, a gather of the Crowther-Blow coefficients (CSR lists, fixed order, no atomics) and a cuFFT C2R, block-reduced statistics finished on the host in a fixed order. The CPU path (FFTW) is unchanged in what it computes; the orientation-independent part of the translation function (TranslationSetup) is now built once per group on either path. - Rescoring: a placement clashing with its mates over twice the limit is not refined, one rigid-body engine per group, and the null is built only for the candidate the choice falls on. Measured MR step on the RTX 5080 (before -> after, seconds; hand check extra where it runs): 3r6o 1.77 -> 0.67, neg 2.14 -> 0.37, 11if 1.20 -> 0.51 (+0.6 hand check), 7ris 1.46 -> 0.56 (+0.6), 9zmu 4.71 -> 1.07 (+0.9). Rotation 0.50 -> 0.02 s and translation 1.17 -> 0.13 s on 9zmu. Mirrored-detector runs of two anomalous lysozyme sets: the hand check vetoes (map inverted at the model's atoms); the unmirrored runs adopt. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-11 17:10:48 +02:00
leonarski_fandClaude Opus 5.5	60a8bf3981	rugnux --af-model (experimental): single-copy molecular replacement of a predicted model ... A predicted model (AlphaFold mmCIF, pLDDT in the B column) is trimmed at pLDDT 70, given B-factors from the pLDDT (Oeffner et al. 2022) and placed by molecular replacement, one copy in the ASU: - rotation function on a uniform 6 deg sampling of the point group's rotation ASU, each orientation scored by the correlation of E^2 with the symmetry-summed |F|^2 read off the model's molecular transform, computed once by FFT and interpolated (Navaza 1994); - full-symmetry Crowther & Blow translation function (one FFT per orientation and group, through MapFromFPhi/FFTW) for the 50 best orientations in every group with the merge's rotations; - the best three placements per group through RefineRigidBody (GPU where there is one), bulk solvent and R-free to 3 A, a C-alpha packing check against symmetry mates, Matthews, and the random-orientation null for a placed candidate. Decision: PLACED = TF Z >= 8, clash <= 5%, Vm in [1.5, 4), null >= 3 sd. Within the answer's merge the placed group with the lowest R-free is chosen where it leads every other packing group by 0.03; the twin hypothesis' merge is searched, and chosen, only where no group of the answer's merge can hold the model at all (Matthews, packing). An enantiomorph relabels the answer as --model does; another group goes to <prefix>_mr.mtz/.cif plus <prefix>_mr_model.pdb; the MODEL PRIOR report section carries the table. The search runs on the CPU in both builds; without --af-model nothing changes. Measured (RTX 5080, MR step): 3r6o twin I41 chosen over I4122 (Matthews + clash) in 1.8 s, solution 0.09 A CA rmsd from Phaser's; 11if P41 -> P43 1.2 s; 7ris P3121 -> P3221 1.7 s; 9zmu P6122 -> P6522 4.7 s; 11if's model against 3r6o's data NOT_PLACED. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-11 13:35:05 +02:00
leonarski_fandClaude Opus 5.5	d7521bf1ea	TCP pusher: never block the receiver thread; queue bounded by image-buffer slots ... TCPStreamPusher::SendImage is called from the receiver's frame-transformation threads. It used a 128-deep per-connection queue and waited when it was full. With images routed per file ((n / images_per_file) % writers), every writer receives a whole file as a burst at the full frame rate (1000 images/file at 2 kHz fills 128 in ~64 ms), so the queue filled on every burst and all transformation threads ended up waiting on one writer. They then stopped returning FPGA frame buffers, and the FPGA dropped packets: partial images, images without a background estimate, data loss on every writer. The ZeroMQ pusher has an unbounded queue and never blocks, which is why it coped. - The per-connection queue is unbounded: every element owns a distinct image-buffer slot, so the bound is the image buffer's slot count. When slots run out the receiver drops only the write (GetImageSlot() == nullptr), after analysis. Absorbing one file burst therefore needs an image buffer of about images_per_file slots (~20 GB for a 9M at 1000 images/file); below that, images are still dropped, now as writer-queue-full instead of packet loss. - SendImage never blocks. A dead or wedged writer is still caught in SendAll (15 s silence / 60 s without progress), after which WriterThread returns the queued slots unsent. - The 200 ms fallback in StopDataCollectionThreads is gone: at END it closed the connection of any writer whose queue stayed full for 200 ms - a normal state for a slow but healthy writer - discarding up to 128 queued images and never sending END. END now follows the whole backlog. - A per-connection enqueue mutex makes "check active, enqueue" atomic against "clear active, enqueue the end sentinel": an image could land behind the sentinel when a connection was torn down mid-run, leaking its slot (CheckIfBufferReturned then threw) or being sent as stale data in the next run. New stress test SendImageRacingEnd_NoSlotLeak (verified to fail with the race widened and the lock removed). - TearDownConnection marks the connection broken first, so the destructor returns a backlog instead of sending it. Writer side: - jfjoch_writer -b/--rcv_buffer sets the TCP receive socket buffer (was hardcoded to the kernel default). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-10 21:11:09 +02:00
leonarski_fandClaude Opus 5.5	a897b08a7d	Merge branch 'doubled-cell' into rc175: a doubled-cell hypothesis for a half-translation pseudo-symmetry that breaks down with resolution ... Narrow, guarded: the doubled pass runs only where the added class grows with resolution and is resolved; the doubled cell is adopted only where one pseudo-translation explains the added class in every shell and the point group does not rise. Fixes the one remaining lattice failure (half cell -> deposited cell, R-free 0.574 -> 0.275); p.hkl identical on the ci tier, the supercell gate set, 78 further supercell-candidate sets and all private sets. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-10 15:18:14 +02:00
leonarski_fandClaude Opus 5.5	b99cd3b5c6	rugnux doubled cell: model validation back in the doubled pass; docs for the final rules ... The doubled pass now runs only on resolved, rising classes (two sets of ~134 with probe logs), so the twenty-minute model placement in a doubled cubic cell can no longer arise, and the adopted answer is validated again (9min: R-free 0.275 against the deposited model, 0.574 in the half cell). Final rules, measured on gate + ci tier + all sets with a de-modulated axial row + 49 sets with a baseline supercell occupancy >= 8% + the private arm: 9min fail -> pass (P212121, doubled cell); every other set run keeps its verdict and a bit-identical p.hkl. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-10 15:16:58 +02:00
leonarski_fandClaude Opus 5.5	7580256ac7	WIP: doubled-cell rise must end at a quarter of the lattice's intensity ... On strong data a class at one or two percent of the lattice rises significantly (0.009 to 0.059 at 4 standard errors on a ci-tier triclinic crystal, which was then adopted doubled and failed), and a half cell missing a few percent of its intensity still describes the crystal. The doubled deposition ends at 0.35-0.40, those classes at 0.06. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-10 14:14:10 +02:00
leonarski_fandClaude Opus 5.5	57543845c0	WIP: doubled-cell pass only where the class rises; no model validation in it ... A class that does not rise keeps the half cell whatever the doubled merge shows, so the doubled pass is no longer run for it - on a ci-tier crystal (R-centred, class at 1-2% of the lattice) that pass ran the card out of memory and failed the run. The doubled pass also runs without --model: placing the model in a doubled cubic crystal's 776 A R-centred cell took twenty minutes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-10 14:00:39 +02:00
leonarski_fandClaude Opus 5.5	55dd0d4f6b	rugnux: doubled-cell hypothesis - the supercell probe reads its class to the limit, and a class that stands clear is merged in the doubled cell as well ... Supercell probe, log-only part: the class it reads over 20-3 A is followed out to the run's own limit (a second prediction of the same supercell on the same 60 frames, so the 20-3 A band and the SUPERCELL_* keys are unchanged). Half-integer reflections are counted only where no lattice reflection predicted on the same frame lies within r3, and each resolution shell is read against the median of the six other parity classes - which a single doubling leaves empty - as an excess in units of the lattice's intensity and as a z in the class's own standard errors, the strongest sibling's z beside it ("Supercell probe to the limit" in the log). Doubled-cell hypothesis: where that class stands clear of the six empty ones (measured above its noise and at more than twice the strongest of them, in at least three shells), the run integrates and merges the doubled cell too, forcing the last pass's indexing result with its lattice doubled along the class (orientation, geometry and axis kept). One of the two is written as the answer, the other's merged .mtz/.cif/.hkl beside it as <prefix>_doubled or <prefix>_halfcell; the report names both (DOUBLED_CELL_*). Which one is the answer is a policy behind one constant: ADDED_CLASS_SIGNIFICANT (the doubled cell wherever the class stands clear) or HALF_CELL_BREAKS_DOWN (default: only where the class reaches a quarter of the lattice's intensity in the finest shell both are measured in). The doubled lattice is classed with a near-zero tolerance: the default lattice search promoted a triclinic doubled cell with an 88.2 deg angle to orthorhombic C, and no frame indexed. Measured on forced doubled-cell merges (rc175 b530c2d base): the open-arm crystal deposited in the doubled cell merges at R-free 0.275 against its deposited model, against 0.574 in the half cell (odd/even 0.02 at >8 A rising to 0.65 at 1.95 A, odd half-set CC1/2 0.93 at 8-5 A). Five crystals deposited in the half cell read classes the doubled merge holds as well (odd CC1/2 0.73-0.98 at 8-4 A): two rise with resolution like the first (to 0.4-0.5 at 2.5 A), one is strong from low resolution (0.3-0.5), two stay flat at 0.1-0.2. Nothing measured on intensities - the trend, the neighbour correlation, the separation from the nearest lattice reflection after the r3 filter - separated the doubled deposition from the rising half-cell ones, so the policy is the owner's call. Also: the SUPERCELL_POSSIBLE prose no longer says correct cells "read the same", and the warning no longer advises -C with the doubled cell, which the indexer reduces back to the half cell. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-10 10:47:55 +02:00
leonarski_fandClaude Opus 5.5	2b105bd259	docs: acknowledge autoPROC and STARANISO as inspiration for header distrust, beam centre and anisotropy ... Reframe the Global Phasing / autoPROC acknowledgement as inspiration and name the ideas: header metadata (beam centre, geometry, axis) must not be trusted blindly (autoPROC, Clemens Vonrhein in particular); the beam centre is determined from the images (rugnux does it its own way); and diffraction anisotropy must be analysed and reported (STARANISO, cited with the reference the STARANISO server recommends; no DOI exists). One-line pointers in data-analysis §1.4 and §13.5, and both works added to the CPU_DATA_ANALYSIS references. The Sheriff & Hendrickson, Popov & Bourenkov and Evans & Murshudov credits for the estimators are kept. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-10 09:31:17 +02:00
leonarski_fandClaude Opus 5.5	1f6d6b35e2	docs: credit Bricogne's Fourier-transform indexing, autoPROC and Global Phasing ... The de-novo FFT indexers implement the 1D-projection method of Steller, Bolotovsky & Rossmann (1997), which was already credited; the 3D Fourier transform indexing idea it descends from (Bricogne 1986, EEC PSD software workshop, Phase III, LURE, p. 28 - no DOI) was not. Add it to ACKNOWLEDGEMENT.md, the CPU_DATA_ANALYSIS references, the §5 text and the in-source credit, and add the DOI of Powell (1999) to the MOSFLM paragraph. 3D-FFT papers (Campbell 1998, Gildea et al. 2014) are not cited: the code does not compute a 3D transform. Add a section acknowledging Global Phasing (Bricogne and colleagues) and autoPROC (Vonrhein et al. 2011) as the expert-system model rugnux aims at. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-10 09:29:33 +02:00
leonarski_fandClaude Opus 5.5	e25b74dd1c	rugnux first pass: recognise the coincidence lattice of two twin domains ... A crystal twinned by reticular merohedry of index n - obverse/reverse rhombohedral domains, or a cubic crystal twinned by a 60 deg rotation about [111] (the spinel law, index 3) - has its two domains' reflections on a common lattice of n times the primitive volume of either domain's own cell. That coincidence cell indexes more frames and more spots than the domain's cell, so the first pass adopted it (a hexagonal/orthohexagonal or R:R cell for an F- or I-cubic crystal), and nothing downstream could recover the true lattice or Laue class. A new step after the sub-lattice hypothesis (TwinCompositeHypothesis), for n = 2 and 3, counts the validation spots on the standing cell in the four classes of the pair of index-n sub-lattices whose union holds the most spots: on both, on each alone, and on neither. Two domains leave the (1 - 1/n)^2 of the nodes on neither empty but for chance coincidences; one crystal, whatever sub-lattice a pseudo-translation favours, populates the nodes off the stronger sub-lattice alike whether they lie on the second sub-lattice or on neither. The step adopts the stronger domain's cell when the two sub-lattice cells are congruent (2 % / 2 deg), the weaker domain's own class holds at least 20 spots at more than four times the per-node density of the class on neither, and the domain's cell indexes a majority of the validation frames. The other domain is left to the leftover-lattice census. Evaluated on the private, inhouse and a twin/tNCS/rhombohedral/cubic/ long-axis selection of the open arm: it fires on the two known index-3 twins (a spinel-twinned F-cubic crystal, now indexed F-cubic; the in-house I-cubic insulin set with a 60 deg [111] second domain, R3:R -> I23) with a weak/neither per-node density ratio of 60 and more (the class on neither holds 0-8 spots against 250-2850 on the weaker domain), and nowhere else. On every other set the ratio stays between 1.0 and 1.6, the top of that range only on runs with a hundred or so validation spots, where the pair picked as the emptiest of 7 or 13 forms carries the scatter; hence the factor 4. Both twins now merge in their cubic group, and the sets re-run against the release baseline otherwise did not change. Cost when it does not fire: tens of milliseconds per first pass. Merges two prototypes: the class census and n = 2 coverage of one, the sub-cell congruence requirement of the other; no environment switch. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-10 00:19:57 +02:00
leonarski_fandClaude Opus 5.5	0457cecafa	Merge branch 'battery-large-cells' into rc175: five large-cell open-arm sets (4mtk, 4nwv, 7d98, 9v16, 4p8r) ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-09 20:36:30 +02:00
leonarski_fandClaude Opus 5.5	d8a3c6ea15	battery: add five large-cell sets to the open arm ... 4MTK (P61, c 653 A, PILATUS 6M miniCBF, SBGrid 235), 4NWV (I222 virus-like particle, 370-411 A axes, Rayonix MX-300 marCCD, SBGrid 9), 7D98 (P6122, c 602 A, EIGER 4M miniCBF at 1.9 A, XRDa 13), 9V16 (P41212, c 451 A, EIGER1 16M HDF5, XRDa 338) and 4P8R (P6522, c 701 A, Rayonix MX-225 marCCD, IRRMC), tagged large-cell (plus long-axis / virus). References are the deposited space group, cell and resolution. 7D98 is pinned to its 250-frame miniCBF sweep: the HDF5 masters beside it repeat frames 61-260 in 60-frame pieces. 4P8R's archive ships per-frame .bz2 files, decompressed after extraction. EXTERNAL_TEST_DATA.md gains the five rows (DOIs resolved), the archive notes and the updated collection counts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-09 20:31:28 +02:00
leonarski_fandClaude Opus 5.5	8442d55f6f	rugnux --model: score the twin hypothesis too, and take the maps from the merge the model prefers ... Where a twin hypothesis (<prefix>_twin_<SG>) was written, the model is also validated against that merge: as merged, and as twinned by its law - |F_twin(h)|^2 = (1-a)|F_model(h)|^2 + a|F_model(Th)|^2 from the placed, scaled model, one overall scale refitted per a, a scanned 0..0.5 on the working set and R-free read at it (ValidateAgainstModel's new twin_law argument; GPU structure factors as before). The maps and the placed model are then written from whichever merge has the lower R-free; the report gives MAPS_FROM and the subgroup's plain and twinned R-work/R-free and fraction. The subgroup file follows its own validation's indexing, as the answer follows its own. The model chooses the maps only: the space group, <prefix>.mtz and R_FREE stay the answer's. Documented in RUGNUX_REPORT.md, RUGNUX_ADVANCED.md and CPU_DATA_ANALYSIS_DECISIONS.md (14.9); Yeates (1997) credited. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-09 19:40:22 +02:00
leonarski_fandClaude Opus 5.5	475df9e5cb	rugnux: write the twin hypothesis beside the answer as <prefix>_twin_<SG>.mtz/.cif ... Near-perfect twins of a subgroup are undecidable from intensities, and a merge in the higher group cannot be twin-refined. Where the adopted group may have absorbed a twin law - a twin gate refused it and was overruled, or a group above P1 was adopted on a merge that read twin-like before the search - the same observations are also merged in a subgroup and written beside the answer, with the twin law and its fraction in _pdbx_reflns_twin. The subgroup is the one a decision named (the group a refused promotion was adopted over, the indexed lattice's group a metric re-ask promoted from), else the index-2 subgroup the search confirmed as a finalist whose lost two-fold agrees worst with the intensities. It is merged from the same engine, to the answer's resolution, without writing per-frame scales back; it carries the answer's free set (flags read off the answer's merge by the adopted group's asymmetric unit) and is written in the subgroup's reference and standard setting (a C 1 1 2_1 description is reduced to P 1 2_1 1). The fraction is Yeates' H on the pairs the law relates (1/2 - median H, a lower bound). At most one file; other qualifying subgroups are named in the report (TWIN_SUBGROUP_* keys). The answer's files are byte-identical. IndexTwoTwinSubgroups and EstimateTwinLawFraction are new and tested; the point-group ledger now carries each hypothesis's representative group. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-09 19:40:22 +02:00
leonarski_fandClaude Opus 5.5	5eba68432e	Merge branch 'mmcif-provenance' into rc175: deposition-grade mmCIF, provenance in the reflection files, git hash for builds without git ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-09 19:32:50 +02:00
leonarski_fandClaude Opus 5.5	c0bcf1a7a9	rugnux report: XDS geometry keywords as one report section, not p_XDS.INP ... The refined geometry is no longer written as a whole p_XDS.INP (and the report's XDS_INPUT= key is gone). Instead the report carries one section, XDS GEOMETRY KEYWORDS, with the XDS.INP keywords that carry the geometry the result was integrated at: ORGX/ORGY (PONI, 1-based), DETECTOR_DISTANCE, the detector axes where the detector is tilted, INCIDENT_BEAM_DIRECTION with X-RAY_WAVELENGTH, and ROTATION_AXIS in XDS's sense (rugnux's axis negated, as verified with XDS IDXREF). No cell or space group: XDS determines those. The lines carry a "# XDS.INP> " prefix, so sed lifts them ready to paste. The same keywords were logged by LogXDSGeometry at the start of every pass, with the axis in rugnux's sense; that printing is removed, so the report section is the one place they are stated. Docs updated (RUGNUX_REPORT, RUGNUX_INTEGRATION, RUGNUX, RUGNUX_TUTORIAL). Output only; p.mtz md5-identical on 3 battery sets. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi	2026-10-09 19:09:27 +02:00