The rc178 branch continued after #88 was squash-merged; this is its content from b9ada7914 (the tree
#88 squashed) to 094eee31a, as one change on main. The individual commits stay on the rc178 branch:
- still battery: --cold evicts each set's input from the page cache; Alvra pixel masks; manifest fixes
- AGIPD: converter to one rigid panel per module, int16 photons, mask folded in; still battery reads the converted sets
- DetectorModules: panel look-up by id, plane -> panel, plane bounds and a per-pixel walk; mask size checked
- tests: [PanelLayout] equivalence harness, stacked vs assembled panel layout
- Dense per-pixel maps through the panel table (panel layout stage 1)
- Panel id on spots and the panel table in the start message
- Viewer and preview show a panel detector as it lies on the detector plane
- Panel layout stage 3: per-panel spot finding (CPU+GPU, CCL) and raster algorithms on the plane raster
- Panel layout stage 2: prediction and Bragg integration through the panel table
- XTC reader: frames handed up as stored, streams indexed in parallel
- XFEL reader: integer frames stay integer, one gain, placement by panel
- EXTERNAL_TEST_DATA_STILLS.md: images used vs the entry for every set; drop "Not in the battery"
- still battery: EuXFEL sets read the depositors' Cheetah CXI files directly; AGIPD converter removed
- float32_to_int16.py: a non-finite pixel becomes -32768, the int16 error marker, not a zero count
- CrystFEL geometry: read the pixel masks and bad regions instead of ignoring them
- Panel layout stage 5: the XFEL reader hands up the stored frame and its panel table
- Panel table persisted in NXmx; modulation grid and pre-scan memo key on the plane
- still battery: Alvra sets take their pixel mask from the CrystFEL geometry, not --mask-good
- still battery: take the deposited model as mmCIF where the RCSB has no PDB-format file
- still battery: dimple gets PDB-format coordinates converted from an mmCIF-only deposition
- stills fixed cost: geometry refinement and pre-scan do the same work in less wall time
- stills geometry sample and trial spot finding run as many workers as the image pass
- Stills integration: background ring reach measured on empty apertures, radial curvature from the whole frame
- StillsPartialityRefine: the partiality width is the run's, measured on the crystals, not the prediction window's
- MergeOnTheFly: capture term and multiplicative outlier band for the stills merge (both opt-in)
- Stills given a cell: the written cell is the mean over the frames that are merged
- Stills: the radial curvature correction decided per run on the same empty apertures
- StillsPartialityRefine: measure the run width only on shells with signal
- Stills pre-scan: report the paired z of the radial-correction comparison
- module placement: the five concurrent fits each keep every thread
- still battery: score R-free against the deposited data through the same dimple protocol
- still battery: the R-free target is the lower of the deposition's dimple and published values
- Stills: background clip follows the radial curve; signal disk from the lattice spacing
- Stills keep an overloaded observation as a lower bound instead of dropping it
- Stills: drop the lattice-spacing signal disk; the correction decision back on the plain clip
- battery: R-free against the deposition without refinement only, one protocol for rotation and stills
- still battery: EuXFEL sets run on a 2,000-image subset for now
- stills geometry refinement: fit the detector distance and the cell together
- stills geometry refinement: a distance given with --detector-distance is held in the joint fit
- Metal integration engine: back to its panel-s5 kernels; Mac stills keep the old clip, no saturation count
- Metal integration test: the CPU comparison runs on a rotation experiment
- Stills: kept overloads go back to the drop where they are not concentrated at low resolution
- stills distance search: an absolute 30-3000 mm scan, a sample found without d-limits, the fit iterated
- battery results: XFEL still battery 20261022-1358 on rc178 0cf62452c - 12 sets, cold, R_free unrefined
- battery results: still summary gains the resolution of our merge and of the deposition
- Pre-scan masks a pixel at the saturation limit on most frames, as one holding the error value
- Stills keep an overload as a lower bound only where it has pixels at the limit; run-level gate removed
- XFEL reader: a per-frame mask is read the way the frames are, its chunk inflated outside the HDF5 lock
- stills distance search: every centre scanned with a wider acceptance, a larger sample where nothing indexes
- rugnux usage: --beam-center-check describes the absolute 30-3000 mm stills distance scan, not the old 60-160 % grid
- stills: detector tilt refined in the geometry pre-pass, kept where held-out frames agree
- CLAUDE.md: the changelog is written once, just before the merge into main - condensed one-liners of what users get
- broker: serve the placed model and its maps of the last rugnux live run (/rugnux/model.cif, /rugnux/map.ccp4)
- frontend: Mol* viewer of the placed model and its density on the Rugnux live page
- writer: _model.cif from rugnux live keeps the input model's metadata, as the served model.cif does
- rugnux live maps: one unit cell centred on the model; the viewer opens on all of it
- Anomalous difference map: ANOM/PHANOM in _maps.mtz, /rugnux/map.ccp4?type=anom, a magenta mesh in the viewer
- Mol* viewer: open with the maps drawn within 10 A of the centre of view
- Mol* viewer: aaredb's structure-viewer setup, on a black stage
- rugnux: the gain of a stills file that states none is read off its single-photon peak where the background resolves photons
- Mol* viewer: in line with AareDB's StructureView at its current head (8ece2c2)
- ACKNOWLEDGEMENT: the web frontend's Mol* viewer follows AareDB's setup (G. Gotthard, PSI)
- stills tilt decision: the modules stay where the run placed them
- still battery: no distance pins and no per-set tuning flags; gain from the data except where a file's baseline hides it
- battery results: rc178 6f6f69 - full still battery and full rotation battery with REFMAC
- Bragg integration: a CCD rotation sweep's sigmas at the measured gain, intensities in photons
- BackgroundGain: the gain of a frame that resolves photons is the spacing of its photon peaks, so a baseline left in the frame no longer hides it
- ShadowFinder: measure the background's azimuthal modulation instead of predicting it from the polarization
- BackgroundGain: one gain estimate for rotation and stills; the still battery no longer gives EuXFEL 152 its gain
- ShadowFinder: measure the modulation robustly under a wide one-sided shadow
- ShadowFinderTest: a one-sided shadow is not fitted as part of the background's modulation
- ShadowFinder: back to the predicted polarization factor, in the plane the background shows
- MXAnalysisAfterFPGA: the Bragg engine's edge mask in the image's layout, so raw / untransformed FPGA runs start again
- battery results: still battery on rc178 e54f55 replaces the earlier rc178 still tables
- XFELReader: per-frame mask and photon conversion without a library call or a branch per pixel
- stills geometry trial: the hypotheses a search asks are indexed side by side
- stills tilt decision: the fits start beside the pre-scan and the geometry trial
- XFELReader: a deflated frame is inflated and converted a piece at a time, while it is in the cache
- XFELReader: a byte-shuffled frame is unshuffled a piece at a time, a whole pixel per step
- GeometryRefiner: residuals carry only the blocks a fit moves; beam and distance one block
- GeometryRefiner: a spot's residual differentiates its observed and predicted halves apart
- docs: EXTERNAL_TEST_DATA lists every open-arm battery set, with corrected values and counts
- battery: doc_check.py cross-checks the manifests against the dataset pages
- stills module placement: the placement fit gets half the threads, each held-out fit an eighth
- still battery: the EuXFEL sets run on all their frames, as the published still runs did
- battery results: still battery on rc178 a0ce6c, cold on a quiet machine, replaces the earlier rc178 table
- macOS build: explicit size_t in the XFEL reader's row bounds; parentheses in the CPU preprocessor's select
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
* jfjoch_broker: Optional live Rugnux analysis of DECTRIS rotation collections inside the broker, with merged data, maps and the placed model written beside the master file, the rotation lattice in the scan result, and a "Rugnux live" tab and dashboard row in the web frontend.
* Rugnux: Faster processing, with molecular replacement and model validation spread over several GPUs and run on the Apple GPU on macOS.
* Rugnux: The polarization stated in miniCBF and d*TREK headers is used, twinned crystals' space groups are decided more reliably, and a run that finds no lattice still reports every image.
Reviewed-on: #86
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
leonarski_fandClaude Opus 5.5