4 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5.5	dda86080e2	BackgroundGain: one gain estimate for rotation and stills; the still battery no longer gives EuXFEL 152 its gain ... background_gain::Estimate is the gain of any input that reports detector units: the photon peaks where the histogram of a few frames shows them (PhotonPeak), else the background's variance per unit mean (Gain). The stills reader path (MeasureGain) and the pre-scan's CCD measurement now both call it, so a rotation sweep and a still read the same number from the same frames. On a CCD the photon-peak test now also sees the pixels outside the active area, all at zero: on one Saturn sweep that spike, the background hump far above it and the hump's echo at twice the value passed as photon peaks (gain 2725). A zero-photon peak is the detector's noise and has a width, so a single bin with nothing beside it is refused. Checked on the 39 SMV/marCCD sets of the battery: every pre-scan gain is the same as rc178's (no photon peaks; the variance per unit mean as before). Where the gain is applied stays as it is: a stills file is converted to photons as it is read (the spot finder and the stills measurements of spot width, bandwidth and partiality work in photons), a rotation CCD sweep keeps its raw counts and the gain enters the integrator (pixel_gain, 46da7fb35). Moving the stills to the integrator route too was tried (stills PixelGain, no reader conversion) and lost badly on MPCCD: R-free +0.032 and +0.047 on two SACLA sets, R_split 0.055 -> 0.091 and 0.067 -> 0.144, with the indexing unchanged; the stills rocking width/bandwidth fit, measured on frames in ADU, collapsed (bandwidth 0.0015 -> 0.0000). still_manifest.json: euxfel152 drops --gain 62; the gain is now read off its photon peaks (62.65, on a baseline of 124.2 left in the frames): R-free 0.252, the same as with --gain 62.649 or 61.5 given by hand (0.252, 0.252); --gain 62 gives 0.2495 - the R-free moves by ~0.003 with the gain. 66.4% indexed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-23 04:01:11 +02:00
leonarski_fandClaude Opus 5.5	eeda3f9088	BackgroundGain: the gain of a frame that resolves photons is the spacing of its photon peaks, so a baseline left in the frame no longer hides it ... PhotonPeak read the gain as the single-photon peak's value, so it assumed a subtracted baseline: on frames whose baseline was never subtracted (the zero-photon peak ~2 photons above zero) it found no peak and fell back to the variance per unit mean, which the offset biases. It now takes the zero-photon peak as the histogram's maximum, the gain as the distance from it to the single-photon peak, and the narrowness and two-photon tests relative to that spacing (narrow: below half height at the midpoint to the next peak; the AGIPD single photon is too wide for the old quarter-spacing test). The zero peak's position is returned and logged as the baseline. A zero peak in the first bin - a subtracted baseline - is taken at zero, so files that already worked read the same gain. A float file that holds whole numbers, measured at a sixteenth of its unit, fills only every sixteenth histogram bin; the histogram is read at that step. The histogram spans 16384 bins (a baseline of ~120 units at 1/16 resolution). The baseline is not subtracted. Taking it off before the conversion (tried: still battery, the AGIPD set with the unsubtracted baseline, all 33,528 frames) left R-free where it was (0.251 vs 0.252) but indexed 53.8% of the frames instead of 66.4%: on a background of zero the spot finder's threshold falls from ~9 to ~4 photons and picks up noise and diffuse rings. Measured gain (16 frames, as in a run): the AGIPD set with the unsubtracted baseline 62.65 on a baseline of 124.2 (was: no peak; the battery passed --gain 62); the other AGIPD set 53.77 (was 53.30, the variance fallback); a JUNGFRAU float file at 12 keV 11.880 (unchanged); a JUNGFRAU float file at 4.57 keV 4.455 on a baseline of 0.1 keV (was 4.52 read as the peak's value; the 1->2 photon spacing of the same histogram is 4.44); MPCCD, Rayonix and the int16 JUNGFRAU copies unchanged (no peaks). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-23 02:29:45 +02:00
leonarski_fandClaude Opus 5.5	87560311bc	rugnux: the gain of a stills file that states none is read off its single-photon peak where the background resolves photons ... A pedestal-subtracted JUNGFRAU background is mostly empty pixels; the variance per unit mean leaves out the zeros and read 6.7 keV per photon on a 12 keV run in keV, 2.4 on a 4.6 keV one, so such files needed --gain. MeasureGain now also histograms the sampled pixel values and takes the gain from the single-photon peak (background_gain::PhotonPeak): the highest bin past the zero peak's descent, standing 5 sigma above the valley before it, narrow (below half height a quarter of its value away), and repeated by a two-photon peak near twice its value. Where there is no such pair the variance per unit mean decides as before. Float frames are read for the measurement at a sixteenth of their unit, so a photon of a few keV is resolved rather than rounded to whole keV. Measured: float32 keV JUNGFRAU files 11.96 (photon energy 12.04 keV) and 4.53 (4.57 keV); the int16 photon copies show no peak past the first bin and keep the 0.55 variance answer, i.e. stay in photons; the SACLA MPCCD and PAL Rayonix files show humps, not photon peaks, and measure as before. A gain the file states or --gain gives still wins. Rotation is untouched (the pre-scan's CCD gain uses only the variance estimator). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-22 20:22:15 +02:00
leonarski_fandFilip Leonarski	19b135186d	v1.0.0-rc.175 (#85) ... * Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-16 15:34:43 +02:00