v1.0.0-rc.178 #89

Merged

leonarski_f merged 172 commits from rc178 into main 2026-10-23 16:55:14 +02:00

Conversation 0 Commits 172 Files Changed 219

+55736 -1769

leonarski_f commented 2026-10-23 16:51:59 +02:00

Owner

Copy link

Copy source

• Rugnux: Processes serial crystallography data from XFELs - SACLA, PAL-XFEL, European XFEL, LCLS and SwissFEL files, several files in one run - and jfjoch_viewer opens the same formats.
• Rugnux: Stills need only the unit cell and space group: beam centre, detector distance and tilt, panel placement, saturation level and gain are found from the images, and the report gives R_split and CC1/2.
• Web frontend: The Rugnux live page shows the placed model with its electron density and anomalous difference maps in a Mol* viewer.
• Rugnux: More reliable convergence of rotation scaling, and live molecular replacement starts as soon as the model is uploaded.
• jfjoch_broker: Initialize clears the previous dataset's bearer tokens, image buffer and live Rugnux status, and a slow CUDA JIT cache is reported at start.

* Rugnux: Processes serial crystallography data from XFELs - SACLA, PAL-XFEL, European XFEL, LCLS and SwissFEL files, several files in one run - and jfjoch_viewer opens the same formats. * Rugnux: Stills need only the unit cell and space group: beam centre, detector distance and tilt, panel placement, saturation level and gain are found from the images, and the report gives R_split and CC1/2. * Web frontend: The Rugnux live page shows the placed model with its electron density and anomalous difference maps in a Mol* viewer. * Rugnux: More reliable convergence of rotation scaling, and live molecular replacement starts as soon as the model is uploaded. * jfjoch_broker: Initialize clears the previous dataset's bearer tokens, image buffer and live Rugnux status, and a slow CUDA JIT cache is reported at start.

leonarski_f added 171 commits 2026-10-23 16:51:59 +02:00

RotationScaleMerge: Anderson-accelerate the per-frame scaling loops (RESULT-CHANGING) ... 138abafa14

The partials and fulls scaling loops are alternating least squares that converge linearly:
over the rc176 battery's 2527 loops a round shrinks the step by a median 0.86 (partials) and
0.93 (fulls), up to 0.98, so the 1e-3 step tolerance stopped them 0.6% / 1.4% (median) and up to
17% (fulls p90) short of their fixed point, and 313 of them were stopped as "stalled" when they
were only slow - a stalled fulls loop also skipped ShrinkToRestrained.

Each round is now accelerated in log G by type-II Anderson mixing (Walker & Ni 2011), depth 5,
each frame weighted by its observation count as in the step test; frames below the credible floor
take the plain round's scale. The history is dropped when the weighted residual grows or the set
of fitted frames changes (new AndersonMixing class, unit-tested on a synthetic slow ALS loop). The
mixed scales are gauge-pinned again before the rescale, and the stop test is unchanged: it reads
the round's residual F(x) - x, so "stalled" now means a loop that does not settle under
acceleration either.

Measured (CPU, -N 8): 4nwv 311 -> 82 iterations over 9 loops, 9v16 76 iterations over 11 loops,
the same counts as the env-gated prototype on tail-solver; no history restart fired on either.
Depth 3/5/8 give 92/91/90 iterations on 4nwv. A restart on a change of the fulls loop's
cross-validated lambda was tried and gave nothing (83 vs 82). Same space group and merging
statistics on 4nwv and 9v16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

Anderson: accelerate a scaling loop only near its fixed point, and never again once its step grows (RESULT-CHANGING) ... 0894230671

The Anderson commit before this one changed three rc176 battery results, which is why it was
reverted there. One of them is the acceleration's own doing: on a split myoglobin sweep (P2_1,
ISa 2.5) the partials loops never settle - one walks at a step of ~2e-2 for as long as it runs -
and the fulls loops walk from their first round (their cross-validated lambda moves every round,
so the map itself is not fixed). Restarting the history whenever the step grew only rebuilt a
one- or two-round history and extrapolated along the walk: the stalled loops ended somewhere
else, the pre-pass fulls smoothing window went from ~7 frames to ~1, and CC1/2 fell 0.701 -> 0.595.

Two rules now confine the acceleration to the regime it is for - a map contracting to its fixed
point:
- a loop is accelerated only once its plain step is below ten times the tolerance (1e-2);
- the first round whose weighted residual grew turns the acceleration off for the rest of the
  loop (AndersonMixing::Stopped), instead of restarting it.
A loop that walks therefore stays the plain loop it was in rc176. The growth rule alone gave
CC1/2 0.625; with the onset, 0.701, as rc176.

Measured against rc176 bc191bb80 (GPU, workstation):
- myob_x06da_split: P2_1, CC1/2 0.701, d 1.81 A - as rc176; 481 -> 257 scaling iterations.
- 7qij 434 -> 126 iterations, 9v16 273 -> 101, same space group and statistics.
- ci tier (29 sets, no null model): 28 unchanged beyond noise; sum of times 580 -> 585 s
  (9gdj +9 s, below).
The two other changes are NOT Anderson's and stay:
- insu_I_x06da_weak I23 -> I222: the all-observations search refuses I23 on a merge chi2 ratio
  2.08 against a 1.85 bound; plain scaling converged to 1e-5 gives 2.07-2.09 and refuses it too.
  Only the plain loop stopped at 1e-3 passed - the bound was set on unconverged scaling.
- 9gdj +9 s, same answer: the early beam-centre comparison's "loses beyond noise" margin
  (-0.002 against 2 sigma 0.0016) moves by ~0.07 between scaling paths, so the converged loop no
  longer takes the shortcut and both centres are judged on all frames; the file's centre is kept
  either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

Beam-centre gates: decide on the strong-reflection count, not on how a scaling loop stopped ... e724f3d26a

MeasuredCentreClearlyLoses, different-lattice branch: the measured centre loses on the pre-pass
subset when it measures clearly fewer strong reflections (Poisson, 2 sigma) than the file's centre.
The subset CC1/2 is no longer read there (still logged). A third of the frames in five blocks merges
to a multiplicity of ~1.09 in P1, so its CC1/2 reads the relative scale of the blocks: on 9gdj the
file centre's subset CC1/2 moved 0.899 / 0.880 / 0.832 / 0.846 with the scaling path (plain 1e-3 /
1e-5, Anderson 1e-3 / 1e-5), ~50x the Fisher noise the gate allowed, and flipped the early verdict.
The count is comparable across lattice classes (taken on the primitive lattice) and stable across
paths: 36.6-38.6k against 103.4-103.6k on 9gdj.

MeasuredCentreWins, same-lattice branch: the measured centre has to be ahead on the strong count by
more than 2 Poisson sigmas; within that the file's centre stays (it adopted at 1874 vs 1873 on one
in-house small-molecule set, and at 0.4 sigma on 9zmu).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

SearchSpaceGroup: the chi^2-ratio and systematic-b gates are not asked of a merge whose L-test reads untwinned (RESULT-CHANGING) ... 37c4f152e1

Both statistics divide the disagreement a promotion's added operators bring by the P1 search
merge's sigmas, which carry the error model the P1 scaling fitted. P1 scaling never ties symmetry
mates together, so the better the per-frame scaling converges, the smaller those sigmas and the
larger the same real systematic floor between symmetry mates reads: the ratios tend to functions
of the sigma-free disagreement H measures directly (chi^2 ratio ~ H ratio^2, b ratio ~ H ratio).

Measured on the weak in-house cubic insulin set (true I23), all-observation arm, 23 over 222:
  plain, tol 1e-3     chi^2 2.024  b ratio 1.438 (bound 1.44, rescued) -> I23
  plain, tol 3e-4     chi^2 2.072  b ratio 1.453                       -> I222
  plain, tol 1e-5     chi^2 2.094  b ratio 1.460                       -> I222
  Anderson 5, 1e-3    chi^2 2.077  b ratio 1.453                       -> I222
chi2_ref 4.16-4.18 (> chi2_ref_reliable, so the b ratio alone decided); H ratio 1.46-1.50, every
sigma-free gate passing. The late re-merge comparison (R_meas 0.1388 -> 0.1419, ISa 21.5 -> 19.5)
then kept I222: a genuine cubic step with a 3-fold systematic floor always costs a little.

These two tests exist for the twin law H cannot tell from real symmetry. A twin compresses the
intensity distribution; this crystal's <|L|> is 0.48, the twins they refuse read 0.39-0.43
(including a trigonal twin, the only corpus case so far refused by them alone). So they are now
skipped where the merge reads untwinned (<|L|> >= 0.44, AnalyzeTwinning's bar), and the
sigma-free gates decide there alone. With it the set gives I23 under plain scaling at 1e-3, 3e-4
and 1e-5 and under Anderson at 1e-3 and 1e-5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

Live rugnux timing: from the puller's receive time, and from the first image ... 7349832309

The live panel showed ~80 Hz on a 100 Hz collection. Its clock started when
the receiver had applied and forwarded the START message, so the START ->
first image gap counted as collection time; the images were stamped when a
forward thread dequeued them, and the last image only after END and the
thread joins; and N images were divided by a span holding N-1 intervals.

- ImagePullerOutput carries `received`, a steady_clock time set by the TCP
  and ZeroMQ pullers as the message comes off the socket, before it is
  queued or deserialised (TestImagePuller stamps it on Put when unset).
- RugnuxLiveStatus takes those times: START's for its origin, each image's
  for the first and last image (min/max, as forward threads report out of
  order). last_image_s is the last data image's puller time.
- Zero of every reported time (elapsed_s, steps' started_s, probe_s,
  last_image_s, cut_s) is now the FIRST IMAGE, as the frontend already
  said; before the first image arrives "now" is 0 and earlier steps are
  negative. The START -> first image time is recorded on its own as
  start_to_first_image_s (snapshot, OpenAPI rugnux_live_status, regenerated
  C++ model and frontend client, OpenAPIConvert).
- The frontend rate is (images_received - 1) / (last - first image); the
  panel headers show "START -> first image X.X s".
- The broker logs "rugnux live: START -> first image ..., N images in T s
  from the first to the last (R Hz)"; "done X s after the last image" and
  last_image_to_written_s run from the puller time of the last image.

The deadline timer is unchanged (it still starts once the forward threads
have joined). No analysis result changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

RotationScaleMerge: the fulls loop keeps its cross-validated smoothness once within 10x of its tolerance (RESULT-CHANGING) ... 7bb8bc4812

The fulls loop re-chose its smoothing lambda by cross-validation every round, so every round was a
different map and the loop had no fixed point: on 149 of the 245 fulls loops that ended unsettled in
the rc176 full battery lambda jumped >= 2x over the last rounds, and one sweep cycled between 1.2e5 and
3.8e7. The choice is now kept from the round whose step falls below SCALING_ANDERSON_ONSET (10x the
tolerance) - the round acceleration starts on, which models one map too. In the audit (env knob
RUGNUX_FREEZE_LAMBDA, non-sticky) this took settled fulls loops 12/30 -> 21/30 on five sets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

RotationScaleMerge: the fulls' own deviation is handed back however the loop stopped (RESULT-CHANGING) ... dd75c6eb2b

ShrinkToRestrained ran only off a fulls loop that reported itself converged, so whether each frame
kept what its own fulls say beyond the smooth curve depended on whether the last step happened to
cross the tolerance: one sweep's ISa read 35.09 or 34.46 by how its loop stopped. The hand-back is a
continuous decision already - each frame keeps the fraction its observations support, read off the
lag-1 covariance of the sweep's deviations - so it is now applied after every fulls loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

RotationScaleMerge: a scaling loop is stalled only when its steps stop falling, and ends on its best round (RESULT-CHANGING) ... b5d9488ceb

The stall rule ('step not below 0.9 x its best for five rounds') called a steady slow fall a stall -
82 of the 245 loops that ended unsettled in the rc176 full battery were still falling at < 2% a round -
and every unsettled loop returned its LAST state: a fulls loop that diverged from round 1 handed on
the state after five rounds of walking. Now a loop is stalled when the median of the pairwise slopes
of log(step) over its last six rounds (Theil-Sen) is not negative - a median, because a least-squares
slope let one round's jump (the smoothness re-chosen five decades away) end a loop falling by a
quarter a round - and however a loop stops (settled, stalled or at the cap) it ends on the plain round
with the smallest step. On a settled loop that is the last round, so nothing changes there; on any
other it is the state closest to a fixed point the loop reached. The partials' loop restores it
through the round's own rescale (ratio fitted G / best G), so the corr bookkeeping of the CPU and GPU
paths is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

rugnux: the banner's Compute line names the card count once ("4x NVIDIA L4", not "4 x 4x NVIDIA L4") ... 638794158c

get_gpu_description already collapses identical cards into "<n>x <name>"; the banner prefixed the
count a second time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

Live rugnux panel: the timeline spans the collection's set-up length from the first poll ... 831b05aebd

The broker's rugnux live status carries expected_collection_s = images x image time
(DiffractionExperiment::GetImageTime, summation included), set at START. The frontend's
timeline uses it as the expected end until a frame rate is measured, and as the axis floor
while collecting, so the axis no longer starts at 10 s and rescales as images arrive. An
older broker without the field keeps the 10 s floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

Battery: in-house set cytc_x10sa_mr - cytochrome C from the production server's live test ... aa05882c46

EIGER2 16M, 1800 images of 0.2 deg, the same P 32 2 1 three-copy form as
the other cytochrome C sets (c 86.5). XDS reference made with XDS + the
Durin plugin (FRIEDEL'S_LAW=FALSE, CORRECT in 152 like the siblings,
INCLUDE_RESOLUTION_RANGE 50-1.80 A where CC1/2 has fallen to 5%): ISa 22.1,
CC1/2 99.9, refs rule cc_half_0.30 -> d_min 1.98 A. Same tags, hand
override and rationale as cytc_x10sa; not in the ci tier.

Reference model (beside the data, outside the repository): cytc_x10sa's
model re-refined with phenix.refine (rigid body, sites, ADP, TLS, waters)
against this set's XDS data with rugnux's free flags; XDS indexed it in
the other setting of the 321 ambiguity, so the data were reindexed
-h,-k,l. R-work 0.269, R-free 0.319.

battery --only cytc_x10sa_mr (rc178 build): pass, merged in P 32 2 1 via
the model, d_min 1.72 A, placed-model R-free 0.337.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

broker: initialize clears the image buffer, turns live rugnux off and removes its model ... c2e5e4a4e8

Initialize is the fresh start between users, so nothing of the previous one's session is left:
the image buffer is cleared, live rugnux processing is disabled and the uploaded MR model is
dropped. It is done at the start of Initialize, under m then rugnux_mutex (the order the other
setters take), so it happens even if configuring the detector then fails. The receiver is always
Idle when the state machine is not running, so the buffer clear has no unmet precondition.

The dashboard's rugnux panel read the settings only once, on mount, so after an initialize from
any client it kept showing the previous run; it now polls them like the Rugnux tab does.

The /initialize description in jfjoch_api.yaml says so; the frontend client is regenerated
(description only). The python client docs and redoc follow at the release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

broker: the journal keeps operational events; rugnux's processing log goes to it only when verbose ... 48c14b6d1b

The live rugnux run wrote its whole processing log into the broker's journal - every
[Rugnux] line of the library (tables included) plus a "rugnux:" line per step and note -
about 300 lines per collection, all of them results the user already gets through
/rugnux/status, the end message and the written files.

- Logger::ServiceLevel(service, level): every Logger of that name made afterwards keeps only
  that level and above. The library makes its own Logger("Rugnux") (and
  Logger("ScaleOnTheFly")) wherever it logs, so this is the one switch: the broker sets them
  to error, or to debug with the existing `verbose` setting. The per-Logger "Version" line goes
  with it. The rugnux CLI routes every Logger to its own sinks (RouteAll) and never calls
  this, so its console and _log.txt are unchanged.
- The live observer feeds steps and notes to the live status only.
- The run's outcome is one line at the end: state, START -> first image and frame rate,
  done after the last image, last image -> all written. The probe result, the warm-up time
  and the cut reason go to debug (the cut is in the outcome line).
- Per-collection step chatter goes to debug: receiver init/teardown, thread start/stop,
  "Wait for receiver done" and friends, the per-image mean processing time. The JFJochBroker
  logger now follows `verbose` too, so these are available again with it.
- Kept: Start request, measurement start, receiving started/finished, receiver efficiency
  (and the per-module packet counts on loss), calibration sequence, all warnings and errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

broker: initialize and start clear the last collection's live rugnux status ... 02e5bc387f

/rugnux/status kept answering with the previous collection's rugnux run after POST /initialize,
and also after a /start whose collection does not run rugnux - the latter under the new run's
tokens. RugnuxLiveStatus gets a Clear(), called from JFJochReceiverService::ClearResults(), which
/start already uses to drop the previous run's status, plots and buffered images before the tokens
change. Initialize now calls ClearResults() instead of ClearImageBuffer(), so a fresh start also
drops the previous run's receiver status and plots.

Authentication: every rugnux endpoint (GET/PUT /config/rugnux, PUT/DELETE /config/rugnux/model,
GET /rugnux/status) was already in the broker's bearer-protected set and carries bearerAuth in
jfjoch_api.yaml; the HTTP auth test now checks all five.

The yaml descriptions of /initialize and rugnux_live_status say so; the C++ model and frontend
client are regenerated (comments only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

jfjoch_broker and rugnux check the CUDA JIT cache at start and warn when cuFFT plans are slow ... 6936b04d5f

cuFFT ships its transforms as PTX for every card (cuFFT 11 and 12 alike), so each plan has the driver
compile kernels, and a second plan of the same shape compiles them again. From the driver's JIT cache
that costs ~0.5 ms a plan; without it 45-130 ms, under cuFFT's process-wide lock - which, over the
~100 plans of molecular replacement, matches the stage pattern of the slow MR in the production broker.

The cache is unusable in more ways than CUDA_CACHE_DISABLE: with no HOME at all (a systemd system
service without User= gets none, and the driver does not fall back to the passwd entry), on a read-only
or unwritable directory, on a filesystem whose fcntl locks fail (NFS without locking) - each measured
as slow as no cache - or on a slow network filesystem (each plan opens and locks the cache ~10 times).

- common/CudaJitCache: the path the driver uses (CUDA_CACHE_PATH, else $HOME/.nv/ComputeCache), its
  filesystem type, whether it is writable, whether its index opens read-write and takes the driver's
  fcntl read lock, CUDA_CACHE_DISABLE and a CUDA_CACHE_MAXSIZE below 64 MiB. One line: info when fine,
  a warning with the cause and the remedy (CUDA_CACHE_PATH on a local disk) when not, or when a plan
  takes over 10 ms.
- WarmUpCuFFT returns the fastest of three 64^3 R2C plan makes after a first one that pays the
  initialisation (and a cold-cache compile): ~2 ms in all with a cache, ~0.3 s without.
- jfjoch_broker runs it after its version line (0.15 s at start, cuFFT init and card 0's context
  included); rugnux on its existing background warm-up thread.
- CUDAMemHelpers.h: the modules are PTX on every platform, not only on Windows.

No result changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

CudaFFTPlan: one cuFFT plan per card and shape, shared by its users; rigid body transforms one grid at a time ... 97ef0936b2

Every cuFFT plan has the driver compile cuFFT's PTX kernels: cufftMakePlanMany ->
cuModuleLoadData -> the PTX JIT, under cuFFT's process-wide lock. A second plan of the same
shape pays again, even while the first is alive. The CUDA JIT cache makes that ~1 ms. Without
one (CUDA_CACHE_DISABLE, no writable $HOME) a plan costs 20-250 ms. That holds for every size
(64^3 and 1D 256 included), every transform type and batch, on sm_75 and sm_120, with cuFFT 11.2,
12.0 and 12.4. No CUFFT_* setting changes it, so a choice of grid sizes cannot avoid it; only
the number of plans made can be cut.

Before this change, a plan was lent to one user at a time, so each concurrent user of a shape
on a card made its own: 9-24 rigid-body engines, the translation slices and the score engines.
Now a plan is made once per card and shape and shared. Each user keeps its own stream and work
area, and CudaFFTPlan::R2C/C2R set both and launch under the plan's mutex. Shared plans give the
same bits as private ones: 8 threads on one plan with their own work areas, 9 shapes x 160
transforms, 0 mismatches on RTX 5080 and T4. A shared work area, the negative control, gives
2-13 mismatches per shape where the plan has a work area.

The rigid-body engine also dropped its batch-3 plan: the Jacobian transforms its three grids one
at a time on the batch-1 plan. That gives the same bits (10 zone-grid shapes, sm_75 and sm_120)
and one plan less per zone and card.

The cytochrome C and thaumatin MR test sets, AF models (MR step, s):
  4x T4   cytc  cache on 2.22 -> 2.22   cache off 8.41 -> 4.00   plans 86 -> 24
  4x T4   thau  cache on 1.58 -> 1.61   cache off 11.13 -> 3.08  plans 131 -> 23
  1x T4   cytc  cache on 2.87 -> 2.81   cache off 8.82 -> 3.84   plans 80 -> 12
  1x T4   thau  cache on 2.41 -> 2.39   cache off 10.42 -> 3.17  plans 104 -> 11
  RTX5080 cytc  cache on 1.41 -> 1.12   cache off 6.5-10.7 -> 2.02
  RTX5080 thau  cache on 0.90 -> 0.85   cache off 8.03 -> 1.60
Outputs are bit-identical before and after, cache on or off: model, merged hkl, maps, and every
MR and validation log line. The mtz/cif differ only in the command line they record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

RotationScaleMerge: scaling loops run to an rms step of 1e-5, accelerated from 1e-2 (RESULT-CHANGING) ... 4620130f71

The per-frame scaling tolerance goes from 1e-3 to 1e-5, and the Anderson onset (also the round the
fulls loop keeps its cross-validated lambda) becomes an absolute 1e-2 instead of 10x the tolerance.

Evaluated on 35 merged sets (ci tier + scaling-audit sets), on top of the loop fixes:
- plain 1e-3: 8913 iterations, 38 s of loop time on GPU, 7 loops at the 100-round cap;
- plain 1e-5: 35212 iterations, 120 s, 17 loops at the cap;
- Anderson 1e-3 (previous default): 3715 iterations, 16 s, merged intensities as far from
  the converged ones as plain 1e-3 (median 0.13%, max 4.3%);
- Anderson 1e-5 (this commit): 6507 iterations, 27 s, longest loop 89 rounds, merged
  intensities within 0.2% of plain 1e-5 and identical SG, d_min, CC1/2, ISa, R_meas, R_free.
No space group or verdict changes in any mode. The converged merge is not measurably better
against the references, but it no longer depends on where a loop happened to stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

Rotation: a blank sweep end is trimmed before scaling, and a delta-CC1/2 stretch takes the sliver it leaves at an end (RESULT-CHANGING) ... 1949155c12

Shutters that open late or close early, and beams that go off, leave frames at the ends of many
sweeps with no exposure at all. Until now those frames went into the scaling like any other: on a
CCD their noise, divided by an incident flux of ~0.005, made the partials loop walk their scales
toward zero until the 50x collapse guard dropped them - or, if the loop stopped first, merged one as
noise, which delta-CC1/2 then removed with its whole 10 deg batch and reported as SWEEP_GAPS (3p85,
frames 338-359). On photon-counting detectors the blank frames fitted G = 0, escaped the guard and
were reported as merged although nothing of them was (cytc_x10sa, myob_x10sa, thau_micromax_pink,
lysoI_micromax_pink, 3p85).

TrimBlankSweepEnds (RotationScaleMerge.h) walks in from each end and clears the reflections of every
frame that
 - recorded no diffraction: as many of its reflections stand more than 3 sigma below zero as above
   it (a sign count, so no sigma calibration is needed and a slight background bias cannot hide the
   strong reflections), and that excess is 3 sigma clear of the run's typical frame - so a crystal
   too weak for one frame to show it is never trimmed; and
 - had no beam: its mean background is nearer to none than to the run's typical frame. That is what
   keeps a crystal leaving the beam, which also records nothing but still scatters, in the sweep
   ledger as a finding (insu_I_x06da_ref, 9zm0, 9fhc, 5reo are untouched).
No stretch length: one blank frame is as blank as twenty. Partly exposed first/last frames recorded
diffraction and stay (7tcd, 7q6j, 9z72, lyso_micromax_pink). It runs on the integration outcome
before the overload count, so the merge, post-refinement and unmerged output all see the trimmed
sweep, and in --mode scale on the stored reflections. The report gives FRAMES_BLANK_START/END and a
prose line - information, no warning; the frames count as rejected. The sweep ledger looks for
degraded stretches, and reads its running medians, only between the first and last frame with data.

Delta-CC1/2: a convicted stretch that stops short of a sweep end by less than one rocking event takes
the rest, where those frames read as the stretch does (5reo: 331-396 rejected, 397-399 were left in as
a three-frame weak_diffraction range with its own warning).

Battery --only, 31 sets, tol5 f217f180a -> this commit: no space group, d_min or verdict changes;
merged statistics identical within 0.001 except 5reo (R_meas 0.236 -> 0.231, ISa 23.0 -> 23.3,
R-free 0.2013 -> 0.2004, warnings 3 -> 2). Trimmed: 3p85 0/19 (+3 empty frames; collapse-guard drops
13 -> 0), 6w4h 0/5 (guard 5 -> 0), 7rji 0/4 (delta-CC1/2 no longer removes 3406-3412, two of which
diffracted: 7 -> 0), cytc_x10sa 0/2, thau_x10sa_* 0/1-2, myob_x10sa 0/1, lyso_x10sa_strong 0/1, 8a1a
0/1, lyso_micromax_mono 1/0, lysoI_micromax_pink 2/0, thau_micromax_pink 1/0. Controls (11if, 5f6m,
6moj, 8u0i, 9fhc, 9z44, 9z72, 9zm0, 6iu6, insu_I_x06da_ref, myob_x06da_split, lyso_x06da_half_image,
thau_bl1a_4p6keV) trim nothing and are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

RotationScaleMerge: the partials' credible-scale floor is a fraction of the typical frame, not of the median (RESULT-CHANGING) ... 2a5d303886

The floor below which a frame's partial scale is not credible (MIN_CREDIBLE_SCALE_RATIO = 0.02) was
read off the MEDIAN fitted scale - in the loop's smoothing window, in its step, and in the guard
after it (DropCollapsedScales). On a sweep that spent most of its turn out of the beam the median is
itself a dead frame, and it walks: an empty frame's scale has no fixed point above zero, so the dead
frames slide toward zero round after round, the median and the floor slide with them, and frames on
the floor cross it, re-enter the window at their neighbours' scale (3x up in one round) and cross it
again. The loop then has no fixed point, and which frames reach the merge depends on the round it
stops on.

The floor is now read off the run's typical frame (TypicalFrameScale), which is what the loop pins
its gauge to - inside the loop it is the constant 0.02, so it stands still. The fulls' guard keeps
the median (their scales are not gauge-pinned). Dropped frames still carry the median scale, so the
sweep ledger reads them as before.

lyso_x06da_half_image (1800 frames, two thirds weak or out of the beam; median frame at 1/5.7 of the
typical one): traced, the final loop's floor fell 2.4x and every spike in its step was one frame
re-entering the window; the R_meas 287% -> 685% of the rc178 loop fixes was the same non-settling
loop ended by two stopping rules (delta-CC1/2 split the weak stretch differently, 206 frames at
1/20-1/50 of the median merged or not). The pre-pass P1 loop that crept to the cap is the geometry
pre-pass, which commits nothing here.

battery --only, before (rc178-aa-tol5 f217f180a) -> after, GPU:
- lyso_x06da_half_image: R_meas 689% -> 81%, CC1/2 0.932 -> 0.971, REFRES CC1/2 0.484 -> 0.911,
  ISa 4.13 -> 4.22, model R-free 0.3302 -> 0.3271, CC_model 0.826 -> 0.829; final loop's smallest
  step 5.6e-3 -> 1.8e-4. --no-gpu gives the same numbers.
- lcystine_x10sa_25keV: R_meas 0.396 -> 0.202, ISa 4.45 -> 8.49, CC1/2 0.9987 -> 0.9998.
- lyso_x10sa_strong: R_meas 0.238 -> 0.151, ISa 5.51 -> 5.74, R-free 0.3653 -> 0.3658,
  CC_model 0.859 -> 0.857, d_min 1.354 -> 1.344.
- insu_I_x06da_ref: R_meas 0.347 -> 0.290, R-free and ISa unchanged.
- 7mzt: CC1/2 0.967 -> 0.983, CC_model 0.532 -> 0.548, R-free 0.4183 -> 0.4197, d_min 3.18 -> 3.21.
- unchanged: lyso_x06da_ref, myob_x06da_split, myob_x06da_powder_2, thau_bl1a_4p6keV, 6o2h,
  3meb, 9z72, 9o0h. Space groups identical on all 13.
Tests: [RotationScale], AndersonMixing* pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

RotationScaleMerge: the correction surfaces stop at an rms step of 1e-3, not at the per-frame scaling tolerance (RESULT-CHANGING) ... 456fd5d5cf

f217f180a tightened SCALING_TOLERANCE to 1e-5 for the per-frame scaling loops, but ApplyCellSurface
used the same constant for its own stop. The surfaces then ran to their 30-round cap ("NOT settled"),
and that - not the per-frame loops - is what lost ISa on the tol-1e-5 full battery
(20261019-2106 vs 20261019-1726): on hepes, myob_x06da, 9rcs and 7bgt the per-frame loops settle on
the same trajectory either way, while the modulation/absorption fits and their half-set gates changed.

The surfaces' own fixed point is the worse one: run to 1e-5 with 300 rounds allowed, the detector-frame
surfaces walk along the resolution direction the gauge projects out (7bgt modulation: 15 -> 283 rounds,
1.13x -> 8.47x ramp removed), their held-out half-set gain drops (hepes +0.026 -> -0.031, 6o2h
-0.28 -> -0.78), and ISa falls (hepes 37.0 -> 18.3, myob_x06da 22.8 -> 19.6, 9rcs 7.4 -> 5.3,
7bgt 17.0 -> 15.3, 6o2h 31.0 -> 18.1). The surfaces' prior and gates were calibrated at 1e-3, so they
get their own SURFACE_TOLERANCE = 1e-3 again; the per-frame loops stay at 1e-5.

Tried and rejected (23 validation sets, runs 20261020-0040/0104/0129 cvfold-*):
- leave-one-out reference in the surface fit: 7bgt R_meas 0.14 -> 0.31, R-free 0.207 -> 0.277,
  9rcs CC1/2 0.994 -> 0.956. Regressing the observation on a mean that excludes it puts the rest of
  the group's noise in the regressor alone and attenuates the slope; the mean that includes it is
  matched by the same noise on both sides.
- leave-one-out reference in the per-frame fit (CPU + GPU): same attenuation; 5reo R_meas
  0.23 -> 0.49, 3meb 0.26 -> 0.37, myob_x06da_powder_2 d_min 1.54 -> 1.94 (aspirin and yag gained).
- cross-validating the fulls smoothness on the held-out per-frame fit while scaling on the usual one:
  thau_bl1a_4p6keV ISa 34.3 -> 36.4, but kdp 23.9 -> 21.3, aspirin 35.5 -> 33.6, L-cystine R1
  0.124 -> 0.137, 6o2h ISa 31.0 -> 25.7; no fewer stalled loops.

Validation, 23 sets (small molecules, the four losers, stall sets, controls; run 20261020-0152
cvfold-final-sets): hepes, myob_x06da, 9rcs, 7bgt back to their tol-1e-3 values; the other 19 equal
to the base within noise (aspirin 20 keV ISa 35.8 -> 35.5, L-cystine 25 keV ISa 8.5 -> 8.2 with SHELXL
R1 0.1152 -> 0.1154). No SG, d_min or verdict change. ci tier (20261020-0152 cvfold-final-ci): 29/29
pass, every row within noise of both earlier full runs. [RotationScale] and AndersonMixing* pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

broker: rugnux live settings and model need no bearer token ... 147f750c90

GET/PUT /config/rugnux and PUT/DELETE /config/rugnux/model prepare the
next collection and expose nothing of the current one, so they are open
like the rest of the configuration. Protecting them meant the next user
needed the previous dataset's token to set up their own run.
/rugnux/status, which reports the current collection, stays protected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

CUDA JIT cache: a private one under /var/tmp when the default is unusable ... ddf41e3a18

A service started by systemd without User= runs as root with no HOME, so
the driver keeps no JIT cache and every cuFFT plan is compiled again
(50-130 ms each, under cuFFT's process-wide lock) - seconds of MR in the
broker. Before anything touches CUDA, the broker and rugnux now point
CUDA_CACHE_PATH at /var/tmp/jfjoch-cuda-cache-<uid> when the default
cache is missing or unusable and the site has set no CUDA_CACHE_PATH.
The directory is used only if it is a real directory of this user's,
mode 0700; otherwise the start-up warning stays as before.

Measured without HOME: a cuFFT plan 51 ms -> 0.5 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

MolecularReplacement: the choice's null and comparisons start beside the work that decides where it falls ... 13dd2a8710

The placed candidate of lowest R-free is known once the groups' searches are done. Its null and its
comparisons with the other candidates (the copies carried into another group beside tNCS, the paired R)
used to start only after the tNCS searches again and the null, one after the other. They now start as
soon as the groups are done, beside the searches again, and the choice takes each where it comes to the
same placements (matched by the placement itself); anything the choice does not reach is waited for and
dropped. A null and a comparison are the same number whenever they are computed, so nothing changes but
when they run. All of them are waited for before a merge's rigid-body engines are given back.

The rigid-body pool gets engines for those fits at once (a null's replicates, a carry and a group
searched again: at least 13); the transform's layout is still decided on the search's own fits.

Measured on 4x Tesla T4 (24 threads), MR step, two runs each, byte-identical p.hkl, p_model.pdb,
p_maps.mtz and MR log lines against the base on 1 and 4 cards:
- cytochrome C (AF model, 3 copies, tNCS, enantiomorph decision): 4 cards 2.18 -> 1.74 s, 1 card
  2.91 -> 2.71 s;
- thaumatin (AF model, one copy, no tNCS): unchanged (1.62 / 2.49 s), it has no comparisons to overlap;
- 9s02 (AF model, 4 cards): 8.53 -> 8.39 s, identical.
[MolecularReplacement] and RigidBody* pass on 4 cards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

Live molecular replacement: warmed up at model upload, searched beside the second pass and carried to the answer's merge ... 6f5c235fd7

- WarmUpModelValidation did nothing for a prediction (no crystal cell: "no atoms or no unit cell"), so a broker's
  first molecular replacement paid every first use on the GPU. A prediction is now warmed up by a search of its
  own (WarmUpMolecularReplacement): its search model in a P 1 21 1 cell four radii on each edge, that crystal's
  intensities to 2.5 A, the search, and the validation of what it placed. RunMolecularReplacement waits for a
  warm-up still running, as ValidateAgainstModel does. --simulate-live warms the model up as the broker does at
  upload. Result-identical.
- fast_live with a prediction (or --mr): the search starts on the geometry pre-pass's merge, averaged into a group
  from that merge's own space-group search (to 3 A, standard setting), beside the second pass, on card 1 % cards,
  a quarter of the threads, under GPUWorkBeside. At the answer's merge CarryMolecularReplacement takes it over
  where it holds - same group and a cell within 1%, every copy refined by the rigid body beside the others
  against the answer's merge and its R-free within R_FREE_MARGIN of the one it was found at - and the merge is
  searched itself otherwise (and always where the twin hypothesis' merge is searched too). RESULT-CHANGING only
  in the refinement end point of the placement.

Sim-live (10 ms/frame, RTX 5080): molecular-replacement step thaumatin 0.96 -> 0.31 s, cytochrome C 1.19 ->
0.25 s; merged reflections identical; thaumatin placed model identical (0.000 A), R-free 0.3765 both;
cytochrome C a symmetry-equivalent placement, CA rmsd 0.03-0.05 A per copy, R-free 0.4443 -> 0.4431.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

broker: initialize drops the dataset's bearer tokens and everything they protected ... 9e09cd4b3d

Initialize is the fresh start between users. It already cleared the
images, plots, scan result and live rugnux status; it now also drops the
tokens, resets the dataset settings (name, file prefix, unit cell, ...)
to their defaults and forgets the last collection followed through
/live/events, so after it nothing of the previous dataset is readable,
with or without its token - the broker is as open and as empty as a
freshly started one.

The auth test checks that after /initialize the data-collection
statistics and the /statistics aggregate are open and no longer carry
the previous dataset's name. SECURITY.md also lists /rugnux/status
among the protected endpoints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

WIP: SACLA MPCCD (Cheetah-SACLA HDF5) stills reader for rugnux - not built, not tested ... 923a210142

Reads the per-hit /tag-*/data int16 8192x512 stacks (eight 1024x512 sensors), places the sensors on
one flat image from /metadata posx/posy/angle (or a refined CrystFEL .geom via --crystfel-geometry),
masks gaps, 5 px sensor edges and the amplifier-shield rows, converts ADU to photons at
0.00125 ADU/eV, and decodes chunks (shuffle+deflate) outside the HDF5 lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux: read stills through a CrystFEL geometry file (--crystfel-geometry) ... ce528f3a14

A CrystFEL .geom (current and old syntax) is parsed for panels (pixel ranges, fs/ss, corners),
clen/coffset (a number, or an HDF5 path read as mm), res, adu_per_eV/adu_per_photon, max_adu,
no_index, bad regions (panel, whole-frame or lab-plane) and the HDF5 mask (mask/mask_file,
mask_good/mask_bad). JFJochCrystFELReader reads the frames the geometry's data path names - a 3D
stack (Cheetah CXI), one 2D frame per file, or one group per shot (/%/data) - from a file or a
directory, puts the panels on one flat image at their rounded positions with gaps and bad regions
masked, and converts to photons where a gain is stated (otherwise the pre-scan measures it). One
wavelength and one camera length per run: values taken from the files are averaged over the first
file. Panel positions are used as given. Frames stored as one deflate(+shuffle) chunk are read raw
under the HDF5 lock and decoded outside it.

Replaces the MPCCD-specific reader of the previous commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

tests: CrystFEL geometry parser - old-syntax fixture and a multi-panel file with path-valued clen, directions, masks and bad regions ... acf6b3604f

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

WIP: LCLS-I XTC reader for single-panel camera stills (Rayonix), geometry from a DIALS .expt ... ae4786d14f

Not yet compiled or tested: work stopped after the feasibility study.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

JFJochXTCReader: unit test with a synthetic XTC stream and a one-panel DIALS .expt ... 2a45eeb831

The test writes a Configure and two L1Accepts, the first with a second, smaller camera beside the
diffraction detector, and checks that the reader takes the larger image, reads the geometry,
wavelength and polarization from the .expt, and hands out the stored pixels. The reader itself
(previous commit) now builds and has processed an LCLS MFX Rayonix stream end to end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Stills merge: R_split and a frame-split CC1/2, per shell and overall ... 6a1d859d49

Serial crystallography quotes R_split and a CC1/2 whose half-sets keep each
crystal whole (CrystFEL's partialator), and rugnux's stills report had
neither, so its numbers could not be set beside CrystFEL's. MergeOnTheFly now
also sums every frame's observations into the frame's own half (the
existing per-image hash preference), and MergeStats reports from those two
means R_split = (1/sqrt 2) sum|I1-I2| / (1/2 sum(I1+I2)) and their CC1/2.
They are printed as a "Serial statistics" table under the shell table (also
in the --report-resolution table) and as R_SPLIT / CC_HALF_SPLIT and their
REFRES_ keys.

The balanced per-reflection split behind CC_HALF is unchanged, and so is
everything decided on it (resolution cut, space-group search, pass gates).
Rotation output is unchanged: RotationScaleMerge does not fill the new
fields and the table is not printed.

On a 4000-frame jet subset over 30-1.5 A: CC1/2 0.830, R_split 31.8%, against
0.834 / 31.3% computed by hand from two separate even/odd runs and 0.842 /
30.5% from CrystFEL on the same frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux stills: the beam centre (and, with -C, the distance) decided on the frames; Cheetah CXI read without a geometry file ... 3f63299557

Stills never used the beam centre the pre-scan measures - the "second first pass indexes both
centres" exists only in the rotation path - so a header centre a dozen pixels off indexed a few
percent of the frames, or a wrong cell.

DecideStillsGeometry runs after the pre-scan: spots are found once on 96 spread frames, and each
hypothesis re-indexes those spots (IndexFrameOnly) at its geometry; the score is the number of
found spots on the lattice of their frame. Centres: the file's, the background-isotropy
measurement and the point-symmetry capture (kept beside it now; the two estimators fail
differently - on a single-panel Rayonix with a tape shadow the isotropy fit reads 4 px off, the
capture 1.4 px). With a reference cell the distance is a hypothesis too: a 2.5% grid over
60-160% of the file's on the 32 strongest frames, a 1% one about its best on all 96, then the
centre decided again at the new distance. The file's value is kept unless another puts at least
1/0.9 times as many spots on the lattice. Where the geometry moved, the stills bundle adjustment
(-C) runs again from there, since its bounds are +-8 px / +-8 mm about its start. Without a cell
the distance is not searched: a wrong distance indexes as a cell of the wrong size.

The CrystFEL reader also opens a Cheetah CXI file (or a directory of them) with no .geom: one
flat panel, pixel size, photon energy and camera length from the file, centred on the detector.

Measured (single-panel Rayonix XFEL stills, a P6122 crystal, 1316 frames, -C/-S, ice rings on):
  hand-built .geom (measured centre, 105 mm)    54.56% indexed, rugnux R-free 0.376
  plain CXI + --detector-distance 105           54.71%, 0.382 (centre found: 733.2,717.8)
  plain CXI, file's wrong 89.4 mm kept          54.41%, 0.367 (distance found 105.5 -> 104.97 refined)
A second single-panel Rayonix set, P6522, 1000 frames, -C/-S, centre moved off the .expt value:
  0 px 59.26% R-free 0.286; 14 px 59.16% 0.298; 28 px 59.16% 0.285 (before: 5-10% indexed)
Unchanged (bit-identical merge statistics and R-free): JUNGFRAU lysozyme stills, 4,000 frames (file centre
kept, +1.6 s) and mpCCD 1339 frames (file centre kept, +2.5 s). The trial costs 1.6-2.5 s without
a cell and 2-11 s with one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux stills: the saturation level read off the frames; Cheetah CXI only for a square frame ... e4cb3e1b09

Foreign stills files state their saturation level wrongly or not at all: an MPCCD Cheetah file is
dark-subtracted per pixel, so each pixel clips at the ADC limit less its own dark, and its
CrystFEL geometry leaves max_adu commented out - the run reported "Overloads: 0" while the inner
panels saturated in most frames.

saturation_ceiling (image_preprocessing/SaturationCeiling) histograms pixel values on a log
scale (64 bins per octave from 256) and returns the lower edge of the lowest bin holding over
twice what any of the 8 bins below it held (and at least 50 values): a bright tail only falls, a
clipped one piles up below where it stops, as a band where the level differs per pixel.
DetectSaturation runs it on 40 frames spread over every stills run and logs it beside the file's
limit; for any input but a Jungfraujoch-written file (ProcessConfig::detect_saturation) a level
below the file's replaces it. The CrystFEL/CXI reader, where the geometry states no max_adu, now
takes the stored integer type's maximum (an int16 Cheetah file clips at 32767).

The geometry-file-free CXI path (previous commit) is limited to square frames: a tiled detector's
Cheetah slab (CSPAD, AGIPD, MPCCD, ...) read as one flat panel would place every pixel wrongly
without a word, and those slabs are not square.

Measured:
  MPCCD stills, 1339 frames: pile-up from 1218 photons (by hand 12000 ADU ~ 1200 ph); 5.5% of
    reflections coarser than 4 A flagged; lowest-shell CC1/2 0.545 -> 0.674, overall 0.741 ->
    0.793; same dimple refinement against the deposited model R-free 0.2586 -> 0.2556
  single-panel Rayonix CXI, 1316 frames: no pile-up (20 pixels at the int16 limit in 40 frames);
    the int16 limit flags 123 reflections; CC1/2 0.737 -> 0.787, rugnux R-free 0.382 -> 0.358
  JUNGFRAU stills (Jungfraujoch files: lysozyme and two private sets): no pile-up below the stated
    limit, results bit-identical
Cost 0.5 s per run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux: --pixel-size, --gain and --saturation for what a foreign file states wrongly or not at all ... 3643b890b0

Beside --beam-x/--beam-y/--detector-distance/--wavelength, the remaining metadata a foreign XFEL
file can get wrong are given on the command line rather than through a hand-built geometry file:
--pixel-size (um); --saturation (the pixel value from which a pixel is overloaded, in the units
the frames are read in), which also replaces the level stills read off the frames; and --gain
(detector units per photon), for CXI and --crystfel-geometry input, whose reader converts the
frames to photons with it - refused for other inputs, which are read in the units they state.

Single-panel Rayonix CXI, 1316 frames, -C/-S, --detector-distance 105: --gain 0.37 (the pre-scan's
background measurement) gives rugnux R-free 0.344 against 0.358 in detector units.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux: an LCLS XTC stream is read without a .expt when --pixel-size, --detector-distance and --wavelength are given ... 63595c0513

Brings the stills-geom-auto-lcls merge resolution (b10a8ce00) onto the rc178-based stills branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

_process.h5 carries the offline per-image scale; -e documented as exclusive ... 750579bfe4

The writer's MX plugin decided the image-scale arrays were present whenever
its own vector was non-empty, which is always, so every data file wrote them
as NaN - and the master's SaveVectorIfMissing then left the end message's
values out. It now marks them present only when an image message carries a
scale. rugnux, whose scale exists only after the image messages went to the
file, puts the per-image scale, its CC and mosaicity into the end message, so
imageScaleFactor/imageScaleCC/imageScaleMosaicity hold the run's values
(NaN only for frames without a scale).

The usage line of -e now says the end image is exclusive, which is what the
code does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux stills: a given cell is where the geometry refinement starts, not what it fits the distance to ... ae7b68dd68

On stills a cell and a detector distance are degenerate: a spot's radius fixes only their ratio. The
geometry refinement that -C switches on anchored the cell and refined the distance, so a deposited cell
a percent off this crystal moved a correct distance by as much (MPCCD: 50.47 -> 50.14 mm, R-free
0.259 -> 0.318). Now the distance is held and the cell refined from the spots, starting from the given
one. Only where the frames have rejected the file's distance (DecideStillsGeometry's grid, which is what
-C recovers a bad header distance with) is the cell anchored and the distance refined from it. The grid
itself asks the given cell, not one refined at the file's distance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux stills with a given cell: one crystal form merged, distance and centre searched together, a stated distance kept ... 380cdf3dc7

- A given cell keeps only the frames whose refined cell belongs to its population (GivenCellPopulation):
  the peak of the refined-volume histogram nearest the given cell, out to the valley that separates it
  from another population, with the bin set by the data's own spread. A sample of two crystal forms
  indexed against one form's cell otherwise merges both; a run with one population keeps every frame.
- The distance grid is run from every centre hypothesis, not only the one decided at the file's
  distance, and a local grid of centres follows at the distance decided. A wrong header distance and a
  centre several pixels off (both background estimators misread it on a Rayonix set) otherwise leave
  each other flat; now the cell alone finds both.
- A distance given with --detector-distance is not searched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

GivenCellPopulation: the population nearest the given cell among those holding a real share of the frames ... 990bbaf9a7

The peak nearest the given cell could be a single stray frame when the given cell is a few percent off
the crystal (a cryo cell given for room-temperature lysozyme stills): the run then merged one frame.
Every population is now found from its own peak and valleys, and of those holding at least a tenth of
the frames the one centred nearest the given cell is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux stills: the given-cell filter says how many frames it left out on either side of the population ... ace890e300

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux stills: the given-cell filter logs the refined volumes it decided on (debug) ... f483033383

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rugnux stills: a given cell is matched in any setting, and per-image reindexing moves the lattice with the indices ... 093069be29

Three defects that stopped known-cell stills runs:

- The consensus cell compared every frame's cell to -C axis by axis. A run whose frames index in another
  setting of the same lattice (axes in another order, another monoclinic choice) had no frame accepted, no
  consensus cell and no output ("No crystal lattice was determined"), although it merged fine. A frame
  whose lattice contains the given cell now counts on the given cell's axes (CellOnReference, the same
  FindSublatticeBases the merge-stage relabel uses).
- The per-image ambiguity resolver relabelled a frame's reflections but left its lattice on the old axes.
  Everything after it predicts from index and lattice: the sublattice relabel put such a frame through a
  second transformation, and on a monoclinic cell with a near-symmetry (a+c nearly as long as a) the
  merge came out at indices finer than anything measured, shells over 100% complete. The lattice now
  follows the indices (ReindexLattice, B' = R^-1 B).
- With -C/-S the stills path replaced the known-cell indexer's lattice, already on the given axes, by
  LatticeSearch's conventional choice, which on that near-symmetric cell is the other description on half
  the frames. The indexer's lattice is now kept when it is close to the given cell.

Measured (cell + space group given, dimple against the deposited model and free set):
- monoclinic MbCO-type still set: was over-complete/broken with --model; now 2.06 A, R/Rfree 0.233/0.314
  at 2.11 A (P1 merge before: 0.275/0.306).
- C-centred orthorhombic set: no MTZ before; now 3.27 A, R/Rfree 0.298/0.403 at 3.68 A (de novo 0.336/0.448).
- monoclinic two-form set, one form: cell P 1 21 1 with beta (was alpha); Rfree 0.310 -> 0.260, 3.05 -> 2.72 A.
- No change on seven other stills sets, nor on two rotation battery sets (5reo, lyso_x10sa_90deg_1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

v1.0.0-rc.178: version bump ... 02dbdb0b9d

VERSION and the strings update_version.sh rewrites (API spec, python and TypeScript clients, C++ model,
Redoc page, docs/conf.py, FPGA HDL and PCIe driver). The regenerated clients and Redoc page also pick up
rc178's API changes made since the last regeneration (rugnux live settings and model without a bearer
token, the live status fields).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

CHANGELOG: 1.0.0-rc.178 ... d49210354b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

tools/battery: the still battery - serial (XFEL stills) datasets against their depositions ... 59332d4694

still_battery.py runs the public serial datasets of still_manifest.json with their deposition's
cell and space group, refines the deposited model against each merge with dimple and the
deposition's own free set, and reports per set: facility, beamline, detector, PDB, images,
indexing rate, R_split, our R-free beside the deposition's published R-free, the wall time of the
rugnux command, ms per image, the input's I/O floor (bytes over 5 GB/s) and how much of the input
was already in the page cache. It shares battery.py's site config (a "still" section), verbs
(run, compare, report, list, keep) and run-directory layout; run directories carry "still-" in
their label.

The datasets (CXIDB 243, 237, 236, 86, 90, 222, 242, 83, 152, 103, 213; dark and ground-state
entries only) are listed with their verified DataCite DOIs in docs/EXTERNAL_TEST_DATA_STILLS.md,
linked from EXTERNAL_TEST_DATA.md; CXIDB is credited in ACKNOWLEDGEMENT.md. The SwissFEL sets also
run as int16 bitshuffle/LZ4 copies, made with still/float32_to_int16.py.

The stray scripts in the repository root move under tools/battery: rugnux_stills_ab.py and its
dataset template to still/, rugnux_anomalous.py beside battery.py; docs/TESTS.md follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

rugnux: several stills inputs in one run; inputs are files, never a directory ... 6557ba87f4

Every input is a file named on the command line, read in the order given (a shell glob gives them
sorted by name); a directory is refused with the glob form to use instead.

- Several Jungfraujoch/NXmx masters of stills are read as one run (JFJochMultiReader): their images
  one after another, the pixel masks combined, the per-image efficiency placed at each input's
  images. They must share the detector, wavelength (0.1%), distance (0.1 mm) and beam centre (1 px),
  since stills geometry refinement refines one geometry for all frames; rotation sweeps are refused.
  The log gives the indexing rate of each input beside the run's (ProcessResult::image_indexed).
- SACLA HDF5 / CXI (JFJochCrystFELReader::ReadFiles, ReadCXI, Open) and XTC
  (JFJochXTCReader::ReadFiles) take the list of files instead of scanning a directory;
  CanReadCXI and xtc::CanRead accept only a file; ListFiles and xtc::CollectStreams are gone.
- Several inputs must be one format, --mode mx, and stored files (no broker, no --simulate-live);
  one frame of a CBF/marCCD/SMV sweep still names the whole sweep.

Validated: two Jungfraujoch lysozyme stills runs together (49,920 frames, known cell) 57.1% indexed (59.3% /
54.9% per run), 1.22 A, CC1/2 0.986, R_split 8.8%, REFMAC R/R-free 0.253/0.269 against 0.257/0.273
for the earlier hand-built combined master. Rotation: 5reo and lyso_x10sa_90deg_1 unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Serial XFEL stills inputs: readers from the files themselves, module geometry and its refinement ... ebce906025

Combines the stills reader and module work (stills-readers, stills-modgeom, stills-euxfel) onto
rc178 with the owner's decisions for this release:

- Inputs are files, never a directory; several files of one format are one dataset (the multi-input
  CLI of the previous commit), so every reader takes an explicit list: JFJochXFELReader::ReadFiles,
  ReadCXI, ReadPanelTable and JFJochXTCReader::ReadFiles take std::vector<std::string>; ListFiles and
  xtc::CollectStreams are gone, CanRead* accept a regular file only.
- One HDF5 frame reader (JFJochCrystFELReader -> JFJochXFELReader) parameterised by what the file holds:
  per-shot groups or a 3D stack, a /metadata panel table (posx/posy/angle, pixel size, nominal camera
  length, pixel mask: as SACLA writes), a Cheetah CXI single panel (square frame; a tiled slab is
  refused with a message). At most 8 files open; every file checked against the first. Signed 32-bit
  decode, per-frame 3D masks, panels at their own coffset (from the EuXFEL work).
- Frames stored one chunk each are read the way the NXmx reader reads its images: only the chunk's
  place is asked of HDF5 under its lock (H5Dget_chunk_info_by_coord), the bytes are read through
  HDF5ImageSource::RawFile (positional, no lock) and decoded outside it - deflate(+shuffle) by zlib,
  bitshuffle/LZ4 (and zstd, plain LZ4) by CompressedImage's decoders. Bitshuffle/LZ4 used to go through
  HDF5's filter under the global lock, one frame at a time.
- LCLS-I XTC (Rayonix): wavelength from the beam record, pixel size from the Rayonix model and
  binning, only --detector-distance needed; damaged/compressed frames skipped, a truncated last event
  dropped; std::ifstream, no POSIX calls: builds with MSVC. No DIALS .expt support at all.
- CrystFEL .geom: module geometry only (panel ranges, fs/ss, corners, pixel size, coffset, data path)
  and optionally the camera length; everything else is ignored with a note in the log.
- --mask-good / --mask-bad <file>:<dataset>: a mask array in another HDF5 file, in the frames' stored
  layout (what a .geom's mask_file used to give).
- Gain measured on the background for panel-table and CXI input that states none (--gain overrides).
- Detector modules from explicit geometry only (DetectorModules: the reader's panels, an NXmx file with
  one NXdetector_module per module, or --module-geometry); never from the image. Spots and predictions
  are mapped through the modules, the image is not re-assembled.
- Module refinement in the stills bundle adjustment (Ceres; per-module shift and in-plane turn, gauge
  on mean shift/turn/scale and on the two tilt-like patterns when the tilt is refined; adopted only
  where it predicts held-out frames better). Applied for a foreign multi-panel detector (panel table,
  .geom panels); for a JUNGFRAU measured and reported only. Written to <prefix>_modules.h5 (NXmx
  NXdetector_module groups), read back with --module-geometry.
- A panel-table file's camera length is nominal: with a reference cell it is refined from the cell.
- Broker and writer output unchanged: masters keep the single NXdetector_module group, the start
  message carries no module list.

Tests: [CrystFELGeometry], JFJochXFELReader_*, JFJochXTCReader, [JFJochMultiReader], [CellPopulation],
SaturationCeiling_*, [DetectorModules], RefineGlobalGeometry_ModulePlacement, [reindex], Sublattice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

jfjoch_viewer: opens the serial XFEL inputs, and the files of one dataset together ... f2e93d847c

The viewer reads what rugnux reads: HDF5 files with a /metadata panel table, Cheetah CXI single-panel
stacks and LCLS-I XTC streams, by their content as the raw sweeps are. The open dialog takes several
files; they are one dataset (as on rugnux's command line), and a CrystFEL geometry file among them is
their module geometry. The images are shown in detector units, at the geometry the files state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

rugnux stills: masks in the data files themselves and by bits, a sub-unity gain left alone, HDF5 probed quietly ... 26efa8c395

- --mask-good / --mask-bad take [<file>:]<dataset>[@<bits>]: without a file the dataset is read from
  each data file (a per-frame mask stored beside the frames, as a Cheetah CXI holds it), and <bits>
  says which bits mark a pixel bad (or good) - what a CrystFEL geometry's mask/mask_good/mask_bad
  said, now that the geometry file gives module geometry only.
- A gain the background shows at or below one detector unit per photon is not applied; such frames
  stay in detector units, as a CCD's do. Scaling a low-gain Rayonix's frames up by it cost one PAL
  set most of its indexing (69% -> 33%).
- The HDF5 stills readers ask H5Fis_accessible with the error stack silenced before opening a file,
  so an XTC stream offered to them no longer prints HDF5's diagnostics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge rc178-version into rc178: v1.0.0-rc.178 and its CHANGELOG ... 063a63e55d

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge still-battery into rc178: the still battery and the external stills test data ... 2bd3616388

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

docs: the stills workflow figure, overview and methods for rc178's stills ... 0f560ecddc

The stills pipeline figure keeps its layout and adds what rc178 does: the input box lists CXI and
XTC, the geometry box is the beam centre decided on the frames and, with a cell, the distance and the
module placement; the pre-scan reads the saturation level; the P1 merge keeps one crystal form.
RUGNUX_OVERVIEW's stills paragraph says the same in prose. Methods: §1.4 describes the stills centre
and distance hypotheses, a new §1.7 the saturation level read off the frames, and a new §7.7 a given
cell as a starting point, the crystal-form filter and the module placement with its gauge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

JFJochXFELReader: the positional-read handles are held for the most recent files only ... 3d8728e3a1

The direct chunk read opened one HDF5ImageSource::RawFile per file and never released it, so the
MAX_OPEN_FILES bound on the HDF5 handles did not hold for that path: a dataset stored one frame per
file would keep a descriptor per file open. They now follow the same least-recently-used bound; a
reading thread still holding a released handle keeps it through its shared_ptr.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

rugnux stills: the saturation log says why a stated limit is kept ... 50397ed9ff

It named only a Jungfraujoch-written file, also when the limit was given with --saturation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge rc178-review into rc178: raw chunk-read handles bounded, saturation log names its source ... f5d0d4ea5e

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

DetectorModules.h: include <cstddef> for size_t ... b9ada7914b

GCC 12 (the Ubuntu 22.04 CI job) does not pull size_t in through <cstdint>/<vector>, so the header failed
to compile there. Checked: all 601 project sources pass -fsyntax-only with GCC 12.2 (gcc-toolset-12).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: --cold evicts each set's input from the page cache; Alvra pixel masks; manifest fixes ... 43e4207aaa

- still_battery.py run --cold: posix_fadvise(DONTNEED) on a set's input files just before its run, so
  every timed row is a cold read without root (the page-cache column confirms it).
- Alvra sets use the depositors' pixel masks (--mask-good); the MbCO int16 glob now finds the copies,
  which keep the tarball's directory tree; LCLS 242 states its wavelength (the XTC does not).
- EXTERNAL_TEST_DATA_STILLS.md names the mask files and the one MbCO dark run that holds no frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

AGIPD: converter to one rigid panel per module, int16 photons, mask folded in; still battery reads the converted sets ... e4eaeba874

tools/battery/still/agipd_to_int16.py converts an AGIPD 1M Cheetah CXI file into the only form rugnux
supports for AGIPD: each module's eight 64-row ASICs placed 66 rows apart (the 2-pixel ASIC gaps
inserted), one 526 x 128 panel per module; the per-frame mask and mask_shared (bad bits = the
geometry's mask_bad) and the geometry's bad regions folded into the data as -32768; values
round(value / ADU per photon) as int16 (clipped to +-32767; no pixel clipped in either set);
bitshuffle/LZ4, one chunk per frame at /entry_1/data_1/data. It writes the matching CrystFEL geometry
(16 panels, each the rigid fit of its ASICs, module coffset kept) and refuses a module whose ASICs are
not on one straight line at a whole-pixel pitch within 0.5 px (worst here: 0.16 px for CXIDB 83,
0.002 px for 152).

Reader: a stored value equal to the smallest value of its signed integer type (INT16_MIN, INT32_MIN)
is no measurement. The reader decodes to float and divides by the gain, so -32768 used to come out as a
count; the CrystFEL parser takes only module geometry, so a flag_lessthan in the .geom could not say it.

Still battery: euxfel83 / euxfel152 read conv/*_int16.h5 with conv/agipd_int16.geom and --gain 1.
Full sets, -N 16, --refine-geometry=off, --model (other jobs on the machine, load 7-23):

                      indexed  d_min  CC1/2  CC_HALF_SPLIT  R_split  R_FREE  main pass       wall
  CXIDB 83  original  90.9%   1.84   0.641  0.617          0.482    0.280   89.3 s (31/s)   105 s
            converted 94.1%   1.84   0.686  0.644          0.470    0.277    6.5 s (429/s)   20 s
  CXIDB 152 original  17.5%   5.37   0.521  0.479          0.590    0.308  131.7 s (29/s)   153 s
            converted 62.8%   2.92   0.876  0.891          0.354    0.284   13.2 s (287/s)   27 s

Files 4.7 -> 1.3 GB and 5.7 -> 1.1 GB. The 152 gain is the mask: the reader took only Cheetah's
generic bad bits, while the geometry's mask_bad (0xffff) also marks three broken modules (bit 32) and
its bad region a fourth; the converter run with the generic bits instead gives 17.5% / 3.41 A again.
With geometry refinement on (16 rigid modules): 83 94.2%, 1.75 A, CC1/2 0.665, R_FREE 0.284, 49 s;
152 65.1%, 2.94 A, CC1/2 0.843, R_FREE 0.278, 27 s - mixed, the manifest keeps it off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

DetectorModules: panel look-up by id, plane -> panel, plane bounds and a per-pixel walk; mask size checked ... a8202789f5

Stage 0 of analysing panel detectors in their stored layout. The table of modules already is the panel
table; what the consumers converted in the next stages need from it:

- ImageToPlane / PlaneToImage on a known panel, no search; panel -1 (empty table) is the identity.
- PlaneToPanel: the panel holding a plane point and its image position, nullopt in a gap (never passed
  through, unlike the existing PlaneToImage).
- PlaneBounds: the box of all panel footprints on the plane; with an empty table exactly the image
  rectangle today's prediction bounds test uses.
- ForEachPixel: f(panel, index, px, py) panel by panel, row by row, gaps not visited; with an empty
  table the plain row-major loop with plane == image.

The image preprocessors (CPU, GPU) and the Bragg engines (CPU, GPU; Metal derives from CPU) now refuse
a pixel mask whose size is not the image's, instead of reading a mask of another size out of range.
Every reader sizes the experiment's detector, and so the mask, by the image it reads, so nothing that
runs today is refused. Nothing calls the new functions yet: no output changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

tests: [PanelLayout] equivalence harness, stacked vs assembled panel layout ... c8fb472f4d

A synthetic 2 x 2 panel detector, its panels turned 0/180/90/-90 degrees, recorded twice: assembled
with the gaps masked, and stacked one panel under the other with the DetectorModules that place them -
the same value in every physical pixel. What is already layout-correct runs as [PanelLayout]: the
panel table itself, spot centroids on the plane through SpotAnalyze, and predictions going plane ->
panel -> Bragg engine -> plane.

The equivalences the next stages have to deliver are hidden cases tagged [PanelLayoutTodo], failing
today: the azimuthal-integration map and the resolution mask per stored pixel (stage 1), Bragg
integration measuring the same intensity (stage 2), and spot finding that never joins two stacked
panels (stage 3). A stage that makes its case pass retags it [PanelLayout].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Dense per-pixel maps through the panel table (panel layout stage 1) ... dcad4104ce

Every dense map that needs geometry now evaluates each image pixel at its
place on the detector plane (DetectorModules), so it is right for a stacked
panel layout and picks up refined module placements:

- AzimuthalIntegrationMapping: bins, resolution, solid-angle/polarisation
  corrections built per pixel through the table; a pixel on no panel is left
  as a masked one. The geometry cache key now includes the panel table, so a
  mapping built after the stills module placement is rebuilt rather than
  served from the cache. ResolutionMaskBits follows (it is a lookup).
- ROIMap / ExportROIMap: azimuthal ROIs on the plane (box and circle ROIs
  stay in image pixels).
- ROIIntegration CPU+GPU: the weighted sums are taken per ROI and panel in
  image pixels and placed on the plane at the end (exact; the affine commutes
  with a weighted mean).
- HotPixelFinder: ring/sector keys and the zeta of the persistence rule on
  the plane.
- Background beam-centre walk CPU+GPU: pixels binned at their plane position;
  host and device share BackgroundPlanePoint so they stay bit-identical.

New common/PanelRows.h: ForEachPixelInRows, ForEachPixel cut to a band of
rows in index order, for the row-parallel builders.

An empty table takes the old arithmetic in the old order: rotation sets and
every single-panel input are unchanged. [PanelLayout] AzintMapping and
ResolutionMask enabled (+ azimuthal ROI map and ROI centroid on the plane);
new BeamCenterFromBackground_BinsOnThePlane and
ROIIntegrationGPU_MatchesCPU_Panels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Panel id on spots and the panel table in the start message ... dc784c073d

SpotToSave gains int16_t panel (-1 = no panel table). It sits in the padding
after the two bools, so sizeof(SpotToSave) stays 80. It is serialised only
where it is set: CBOR spot key "panel", HDF5 /entry/MX/peakPanel (data file,
plus a virtual dataset in a VDS master when the start message has panels),
read back by HDF5MetadataSource. For a flat detector none of these is written,
so files and streams for existing data are byte-for-byte unchanged.

The start message carries the panel table (detector_modules, CBOR key absent
when empty), filled from the experiment; JFJochHttpReader takes it, so a
stream viewer knows the panels too.

AssignModules takes the spot's panel id where it has one - the stored pixel box
it was found in - and falls back to the plane position otherwise (every spot
until per-panel spot finding sets the id).

Reflection deliberately gets no panel id (owner decision): it is needed only
between prediction and integration, and the struct feeds GPU layout and
memory budgets.

Tests: CBORSerialize_SpotPanelId, CBORSerialize_StartDetectorModules,
JFJochReader_PanelId (VDS/legacy/integrated, with and without panels).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Viewer and preview show a panel detector as it lies on the detector plane ... 4d90053899

With a panel table, the viewer and the broker preview gather the stored image
onto the detector plane (AssembleToPlane) and draw on that: spots,
predictions, rings, beam centre and ROIs are plane coordinates, so they now
land on the right pixels for a stored (stacked or moved-module) layout, where
they were drawn as image pixels before. The viewer scene is the plane; its
image item sits at the raster's origin (JFJochImage::origin_, zero for a flat
detector). Mouse readout, pixel labels and the magnifier read the shown
raster; top pixels and saturation crosses are mapped from the stored pixel to
the plane; an ROI turned into a user mask is tested per stored pixel at its
plane position.

A flat detector (empty table) takes the old path verbatim: the shown pixels
alias the image, origin (0, 0) - a JF/EIGER file renders identically.

common/AssembleToPlane.h is a TEMPORARY header-only version so this stage
builds on its own; stage 3 owns the file and its version replaces this one at
merge (callers use AssembleToPlane(modules, image, width, height,
PlaneBounds, gap) -> PlaneRaster{x0, y0, width, height, pixels}).

Test: PreviewImage_PanelLayout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Panel layout stage 3: per-panel spot finding (CPU+GPU, CCL) and raster algorithms on the plane raster ... fd798cd665

Spot finding works panel by panel, as the FPGA path does per JUNGFRAU module. PanelBoxes (common/) gives
one box per DetectorModule; an empty table gives one box, the whole image, so every clamp reduces to the
old image bounds in the same evaluation order (rotation sets, EIGER, CCDs, our JUNGFRAU unchanged).
- ImageSpotFinder::SetPanels: the CPU local-box pass (dense, count-threshold and the DetectAt/candidate
  path of the adaptive finder) slides its windows inside each panel box; host CCL labels panel by panel;
  pixels on no panel are folded into the resolution mask.
- GPU: analyze_pixel tiles each panel (warps numbered panel after panel), analyze_candidates and
  union_neighbours clip to the pixel's panel, the device spot carries its panel; the GPU list is put in
  panel order so it equals the host's.
- DiffractionSpot carries its panel; SpotAnalyze maps by ImageToPlane(panel, ...) (d_A on the plane).
- ImagePreprocessorBuffer::GatherTiles (CPU+GPU) clips each square to the panel of its centre.
- MXAnalysisWithoutFPGA and the pre-scan workers hand the module table to finders and buffers.

Raster algorithms: new common/AssembleToPlane.h (PlaneRasterOf, AssembleToPlane nearest-pixel gather with
gaps, GatherFromPlane back to stored pixels; empty table = copy). With panels, ShadowFinder::GetMask runs on
the projection assembled to the plane raster and gathers the mask back per stored pixel (CPU route);
FindBeamCenterFFT scores the assembled projection and reports plane coordinates; the detector diagnostic
JPEG shows the assembled plane. Without panels all three take their old path.

Harness: PanelLayout_SpotsNeverCrossPanels enabled (count threshold and SNR on the edge scene), new
PanelLayout_SpotFinderParity (CPU vs GPU, local box and adaptive, on the stacked layout),
PanelLayout_AssembleToPlane, PanelLayout_ShadowFinder, PanelLayout_BeamCenterFFT,
PanelLayout_DetectorDiagnostic (stacked vs assembled image cut to the raster).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Panel layout stage 2: prediction and Bragg integration through the panel table ... d6166f6fa5

Everything sparse that took a radial quantity from an image coordinate now takes it from the
reflection's (or spot's) position on the detector plane, through its panel (DetectorModules).
With an empty module table every changed expression reduces to the old one in the same
floating-point order, so flat detectors - every rotation set, EIGER, CCDs, assembled JUNGFRAU -
integrate bit for bit as before.

- Prediction (CPU, GPU, rotation CPU and GPU): bounds test against PlaneBounds instead of the
  image rectangle.
- IntegrateOnImage: a prediction in a gap or off every panel is dropped (PlaneToPanel), no longer
  passed through onto an unrelated stored pixel. The panel id stays local: the engine and the way
  back find it from the image position (the panel boxes tile the image), and a prediction that
  rounding to float put on another panel's edge is dropped so the two agree. Nothing is added to
  Reflection.
- Bragg engines (CPU, CUDA, Metal) take a BraggPanel per reflection (BraggStencil.h): its plane
  position, the panel's pixel axes on the plane and its box. The stencil's radial direction is
  taken on the plane and expressed in the panel's pixel axes; r0, the radial background curve,
  the Gaussian's elongation, the learned moments, tan(2theta) and Finalize's polarisation use the
  plane position; r_max / n_rad come from PlaneBounds. The neighbour mask and the owner map are
  clamped to the reflection's own panel; inside the walked box a pixel off the reflection's panel
  is read as a gap (unreadable, statically masked - not an overload), which is what the assembled
  image has there, so a disk cut by a panel edge is partial rather than silently "full".
- Risk R4: the ProfileEmpirical grid is learned and applied in the plane's axes (the panel axes
  taken to the nearest quarter turn, BraggProfileCell), so turned panels pool one shape. Without
  it the stacked layout read up to 0.3 % off the assembled one in the harness.
- SpotWidth / SpotFootprint: aperture bounds against the spot's panel box, beam distance,
  resolution and the shape's 2theta Jacobian on the plane, moments read in the panel's axes.
- GetDetectorMaxQ_recipA, the far-corner radii (first-pass tilt report, metric-symmetry arm,
  footprint table range) from PlaneBounds; pre-scan spot resolution on the plane;
  LogHeaderBeamCenter: geometric centre of the panels' plane box, and the masked-pixel check on
  the panel the beam lands on (none when it is in a gap).

Harness: PanelLayout_BraggIntegration is now a real [PanelLayout] check, over six settings
(default, bandwidth-elongated ring, radial background, overlap exclude, box sum, empirical
profile, all with polarisation), stacked vs assembled to 1e-5; PanelLayout_BraggIntegrationGPU
runs the same on the CUDA engine. Its spots avoid exact half pixels: the grid centres on the
rounded position, and a half rounds to the other neighbour on a turned panel.

The Metal engine (.metal/.mm) mirrors the CPU change exactly but is UNCOMPILED here (no macOS):
it needs a build and BraggIntegrationEngineMetal_BitIdenticalToCPU on a Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

XTC reader: frames handed up as stored, streams indexed in parallel ... eb47013d4f

The main loop's read (ReadRawImage) now hands a camera frame up as the
file stores it - 16-bit pixels, UINT16_WIDENED, as the marCCD reader
does - wherever there is no pixel offset to take off and no gain to
divide by; the GPU preprocessor widens it on the device. The host no
longer widens 3.7 Mpx to int32 per frame (the loop that was ~79% of
the main pass's user cycles). xtc::ReadFrame becomes xtc::ReadPixels,
the stored pixels only; the int32 decode (LoadImage, or an offset or
gain) widens them in one plain loop.

The streams are indexed in parallel (one worker per stream, up to the
hardware threads), merged in the order given: each index is a walk of
small reads, waiting on the disk one at a time.

Measured on a Rayonix MX340 run, 1920x1920 (RTX 5080, 5950X, page
cache warm, other agents' builds running; output identical but for the
known run-to-run flip of one observation on one set, which the
baseline shows between its own runs):
  main pass, 2000 frames:  479 -> 789 frames/s (lcls222),
                           413 -> 477, 388 -> 452 (the two forms of
                           lcls242, there indexing-bound)
  Step 1, 48 streams/82k frames: 34-51 s -> 2.4 s with the headers
                           warm, 58.2 s -> 5.7 s evicted from cache

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

XFEL reader: integer frames stay integer, one gain, placement by panel ... 95e51f868d

DecodeInto no longer turns every stored pixel into a float and back:
- the gain is one value (adu_per_photon_), not a float per stored pixel
  - it always was one value; only --gain or a measured gain set it;
- a 16-bit signed frame (Cheetah CXI, SACLA) without a gain is handed
  up as Int16, in its own type; other integer frames are widened to
  int32 without a float; only a float frame or a gain other than one
  goes through lround(v / gain), exactly as before;
- placement is per panel (start, size, destination steps) instead of a
  64-bit destination per stored pixel; a panel square with the image
  is copied row by row (PAL: 1440x1440 into 1444x1444 - the two-pixel
  margin is Place()'s, kept so the geometry does not move);
- the decode buffers (chunk, inflated, unshuffled) are reused per
  thread, the bitshuffle decode is not copied again;
- a chunk whose HDF5 filter_mask is not zero (a filter skipped) is
  read by HDF5 instead of being decoded as if it were not;
- the never-set value flags (flag_lessthan/morethan/equal) are gone
  from the layout and from the per-pixel loop.

MeasureGain in the CLI now reads Int16 frames as well, so a SACLA run
still has its gain measured.

Tests: JFJochXFELReader_SACLA/_CXI updated for the mode, new
JFJochXFELReader_CXI_Int16 (deflate without shuffle, a chunk with its
filter skipped) and JFJochXTCReader_StoredPixels (synthetic XTC,
stored hand-up and the offset path).

Measured on two PAL-XFEL Rayonix sets (int16, deflate; RTX 5080,
5950X, page cache warm; reports and .hkl identical to the baseline):
  main pass, 2000 frames:  296 -> 799 frames/s (pal86),
                           302 -> 755 frames/s (pal90)
  wall, 2000 frames:       26.4 -> 13.6 s, 32.1 -> 19.4 s
  (the geometry-refinement steps, which read frames too, halve)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge branch 'panel-s2' into panel-s5 e065011826

Merge branch 'panel-s3' into panel-s5 ... 648ae8f0d5

# Conflicts:
#	common/CMakeLists.txt
#	tests/PanelLayoutTest.cpp

Merge branch 'panel-s4' into panel-s5 ... 481ddeae24

# Conflicts:
#	common/AssembleToPlane.h

Merge branch 'ccd-reader-speed' into panel-s5 6a7706b955

Merge branch 'rc178-still-results' into panel-s5 ... 64f6b7a5c4

# Conflicts:
#	reader/JFJochXFELReader.cpp
#	tests/XFELReaderTest.cpp

EXTERNAL_TEST_DATA_STILLS.md: images used vs the entry for every set; drop "Not in the battery" ... dafb112d93

Each set's row now says how many images we use and, where that is a subset, what the entry or its
deposition holds (SACLA 237 ~1.8% of dark_ref, LCLS 222 one of four runs, LCLS 242 runs 256-267, both
AGIPD sets and A2A against their deposited frame counts). The section listing what is not used is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: EuXFEL sets read the depositors' Cheetah CXI files directly; AGIPD converter removed ... a2b770fe43

The AGIPD sets are run on the files as deposited (Cheetah CXI with the per-ASIC CrystFEL geometry),
and their processing speed is reported as it is on those files; only the SwissFEL float32 sets keep
an int16 photon copy. The reader rule that a signed integer frame's smallest value is no measurement
stays: it is general.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge branch 'rc178-still-results' into geom-mask 30ea607686

float32_to_int16.py: a non-finite pixel becomes -32768, the int16 error marker, not a zero count ... 074f5e4c34

Measured values are clipped to [-32767, 32767] so that only a pixel without a measurement holds -32768.
The existing copies are unaffected: none of the converted SwissFEL runs had a non-finite pixel or a
clipped value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

CrystFEL geometry: read the pixel masks and bad regions instead of ignoring them ... 7c23b4befe

The .geom parser now takes the masking information the format defines:
- mask arrays: mask/mask_file/mask_good/mask_bad (mask 0) and maskN_data/maskN_file/maskN_goodbits/
  maskN_badbits for N = 0..7; global keys hold for the panels after them, a panel may override. A mask
  that is the same on every panel is one whole-frame mask, otherwise one per panel, applied on that
  panel's pixels only.
- a relative mask_file is looked for in the working directory (as CrystFEL does), then beside the .geom.
- bad regions (bad*): a lab-plane box (min_x..max_y) or a fs/ss range on a panel (or, old syntax, on
  the whole frame); mask_edge_pixels and no_index.

JFJochXFELReader already had the static pixel mask and the per-frame mask path for these; the masks
go through them unchanged. A mask whose path holds % (one per shot group) is applied frame by frame
like a 3D one. Where the geometry names mask arrays, the data file's own Cheetah masks (CXI
mask_shared and per-shot mask with Cheetah's generic bad bits, /metadata/pixelmask) are not added:
the geometry says which bits matter (e.g. an AGIPD mask_bad that includes bit 32). --mask-good /
--mask-bad still add their masks to these.

Tests: [CrystFELGeometry] (new keys, per-panel override, relative mask file), JFJochXFELReader_*
(new GeometryMasks case on a synthetic CXI + .geom), [JFJochMultiReader], JFJochXTCReader*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Panel layout stage 5: the XFEL reader hands up the stored frame and its panel table ... 451fd25857

A detector of two or more panels is no longer assembled. The image is the stored frame, and each panel of
the geometry (CrystFEL .geom or the file's own panel table) is a DetectorModule over its box of the frame,
at its true placement on the detector plane: fractional origin, the panel's own axes, no snapping to an
image axis or a whole pixel. Pixels on no panel, edge, bad-region and file-masked pixels are masked in the
stored layout. The block scatter (a transpose for the 90-degree-turned JUNGFRAU and MPCCD panels) is gone
for these inputs. A single panel keeps the old block path, so single-panel CXI files are unchanged.

Signed-integer frames in their own units (gain 1, no per-frame mask) go up with no copy: a bitshuffle/LZ4
chunk still compressed, for the GPU preprocessor's BSLZ4 decoder (int16's smallest value is its error
value there, so the type-min rule holds), a deflated one inflated outside the HDF5 lock straight into the
caller's buffer. Everything else (float, unsigned, a gain, a per-frame mask) is converted once in stored
order. A float NaN/inf now reads as no measurement (INT32_MIN) instead of 0, and values past int32 are
clamped before the cast.

Also: SpotToSave.panel is filled from DiffractionSpot::Panel(), so AssignModules takes the panel the spot
was found on. PreviewImage_PanelLayout expects stage 3's PlaneRasterOf raster (merge fallout).

Main pass, 2000 frames, vs the CCD-reader build: JF4M int16 bslz4 465 -> 1860 frames/s, JF16M 105 -> 1111,
MPCCD 212 -> 252; flat Jungfraujoch data unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Panel table persisted in NXmx; modulation grid and pre-scan memo key on the plane ... c942dcb584

- The NXmx master (and _process.h5) writes module_0..N with the panel table when the start message has one,
  besides the whole-image module, so a stored-layout file reads back with its panels
  (JFJochReader_PanelId checks the round trip).
- RotationScaleMerge's modulation grid spans PlaneBounds instead of the image rectangle; with no panel
  table x0 = 0 and the extent is the image, so the cells are unchanged.
- The beam-stop / pre-scan memo key includes each panel's box and placement; nothing is added with no
  table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: Alvra sets take their pixel mask from the CrystFEL geometry, not --mask-good ... 82f90259a0

The SwissFEL .geom files name their pixel mask with an absolute facility path, so the sets read a
copy beside the original (jf-16M_mask.geom, mbco_march_mask.geom) whose mask_file is relative to the
copy's directory; only that line differs. EXTERNAL_TEST_DATA_STILLS.md says how to make them.

On a 2000-frame subset of the int16 MbCO set, before (geometry without masks + --mask-good) and after
(the geometry's mask and bad regions): indexed 98.2% both, R_FREE (--model) 0.244 -> 0.234.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge branch 'geom-mask' into panel-s5 91861e9b08

Merge branch 'rc178-still-results' into panel-s5 bb7c9c8fa3

still battery: take the deposited model as mmCIF where the RCSB has no PDB-format file ... 69b3390458

Entries too large or too new for the PDB format exist only as mmCIF, and rugnux and dimple read both;
such a set ran without --model and without the dimple refinement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: dimple gets PDB-format coordinates converted from an mmCIF-only deposition ... 987e812e23

dimple refuses an mmCIF model ("neither mtz nor pdb"), so a set whose deposition exists only as mmCIF
lost its REFMAC R-free; the coordinates are converted with gemmi first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills fixed cost: geometry refinement and pre-scan do the same work in less wall time ... 96322a8ec9

Four changes to the per-run fixed cost of a stills run (Step 2 geometry refinement, Step 3 pre-scan),
none of which changes what is computed:

- Module placement: the four held-out fits and the placement on every strong frame are independent, and
  one Ceres fit keeps only ~4.6-6.5 of 32 cores busy (perf, JF16M, 4000 frames / 150888 spots). They
  now run side by side on a fifth of the threads each; the placement fit is computed speculatively and
  used only where the held-out residuals call for it, as before. Baseline (quiet): held-out 12.3 s +
  placement 5.1 s on JF16M, 16.4 + 6.5 s on JF4M, 19.0 + 7.6 s on MPCCD.
- Module scan (+-25 px grid per module ahead of the placement) was one thread: every grid shift is now
  scored on the workers and the grid then walked in its old order, so the first best still wins
  (3.3 s on JF16M, ~5 s on JF4M/MPCCD before). New test ScanModuleShifts_FindsShiftedModule checks the
  shift is found and that 1 and 4 threads give the same answer.
- Stills geometry trial: each hypothesis (centre, distance, frame sample) is indexed once. The distance
  grid ran twice on the same centre whenever the file's centre was kept (40 of ~275 hypotheses,
  ~15% of the trial: 6.2 s on JF16M, 6.7 s on the flat JF8M), and the final here/there check repeated
  the first trial. Repeats always gave the same count (checked on three logs).
- The saturation histogram (40 frames, 0.1-2.2 s) reads only frames and the mask: it is taken beside
  the geometry refinement and the limit decided where it was, after it.

Plus TIMING marks for the sub-steps (geometry refinement sample, module scan, module placement fits).

Outputs identical to the baseline binary (bb7c9c) where checked: JF16M int16 15 runs (geometry, all
32 module shifts, held-out residuals, cell), JF4M int16, MPCCD 236 (indexing rate, CC1/2, R_split,
R-free with --model identical), PAL 90 and LCLS 222 (identical reports).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills geometry sample and trial spot finding run as many workers as the image pass ... b87b015253

The geometry refinement's sample (up to 10000 frames: spot finding + indexing on the card, the same
per-frame work as the image pass) and the stills geometry trial's 96-frame spot finding ran -N workers
(32 here), each with its own GPU engines. The image pass caps the workers that drive a card at 16 per
card, fewer where their engines would not fit half the card (GPUWorkersPerCard), because past that a
card stops taking queue work and more workers make the loop slower. The cap is now one function,
ImageWorkers(), used by all three loops. On a 16.9 Mpx JUNGFRAU 16M the 32 sample workers held ~13 GB
of a 16 GB card's engines (24 B/px each) where the cap allows 19.

Which frames the sample examines does not depend on the worker count (fixed stripes), and each trial
frame is found on its own: outputs identical (JF16M int16 15 runs: same 5174 frames sampled, same
module placement, geometry and merge).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills integration: background ring reach measured on empty apertures, radial curvature from the whole frame ... e8145d8d2f

Two changes to the stills integration background, both stills-only (rotation keeps its path bit for bit):

1. The r2..r3 background ring's outer radius is measured per run. The pre-scan lays apertures on a
   grid over its frames, away from every found spot, and for each candidate r3 (r2+2 .. 14) takes the
   mean square difference between the r1 disk mean and the clipped ring mean. With nothing in the
   aperture that is exactly the background error the ring makes (noise + bias); the run keeps the
   minimum. On a photon-counting pixel detector the noise wins and the shipped r3=14 is kept; on a
   binned CCD the background structure wins and the narrowest ring is taken. r1 stays 6: a smaller
   disk (4,6,14) costs 0.0013 dimple R_free on a JUNGFRAU set, so it does not pass the
   photon-counting gate. --adaptive-integration-radius now defaults on for stills too (=off restores
   the fixed 6/8/14). The pre-scan bandwidth estimate is not applied to stills.

2. The radial background-curvature correction on stills takes its curve from the adaptive spot
   finder's peak-excluded per-ring background (whole frame) instead of the predicted reflections'
   own ring pixels, and auto applies it to every stills image (no ice-score gate: a solvent/carrier
   hump curves the background as much as ice). New engine call RadialBackgroundFromRings; CPU, GPU
   and Metal read the handed-over curve. The GPU box sum now sizes the radial bin index separately
   from accumulating, so a handed-over curve is read at the right radius.

Measured (dimple R_free, deposited free set, battery arguments, all frames unless noted):
  pal86  (Rayonix 4x4, 156 um): 0.354 -> 0.218   (ring 10; at 4,6,10 0.213)
  lcls222 (Rayonix 4x4, 177 um): 0.205 -> 0.196  (ring 10)
  sacla236 (MPCCD): 0.2588 -> 0.2558             (ring 11)
  alvra213_int16 (JUNGFRAU): 0.1986 -> 0.1986    (ring 14, unchanged)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

StillsPartialityRefine: the partiality width is the run's, measured on the crystals, not the prediction window's ... 3e0285c425

The stills partiality width came from a per-crystal least-squares fit of de^2 against d*^2 over the
integrated reflections. Those fill the prediction window evenly, so the fit returned the window's own
width (W^2/3, constant in the linear Ewald distance, gamma_e ~ 0): at low resolution several times the
crystal's rocking width, which mis-corrected every partial there.

The width is now measured once per run from what the reflections RECORD: the intensity-weighted second
moment of the excitation error, sum I de^2 / sum I, which is the width of a Gaussian partiality whatever
the window, in 8 shells of d*^2 over all crystals. Each shell is corrected for the window cutting the
curve's tails (a curve wider than the window is taken as wide as the window), and the shells are fitted
as s^2 = gamma0^2 + (gamma_e d*)^2 + (bw lambda d*^2 / 2)^2 with every term non-negative. The per-crystal
fit stays as the fallback when the run is too small to measure (< 800 reflections). The prediction window
is unchanged (2x the per-frame profile radius).

Measured: on the SwissFEL lysozyme stills the width comes out ~0.035 deg angular at every resolution,
the ~0.03-0.045 deg that the empirical rocking profile of the observations gives; the window-moment
model was 3-4x wider at low resolution.

dimple R-free (deposited free set, matrix subsets/cuts, PARTIALITY_MATRIX.md), shipped -> this:
  SwissFEL run0237 4k     0.2692 -> 0.2454
  same, -C/-S given       0.2679 -> 0.2513
  MicroMAX mono 4k        0.3341 -> 0.3189
  MicroMAX pink 4k        0.2415 -> 0.2442
  SACLA 243 subset        0.2576 -> 0.2454
  SACLA 237 subset        0.2342 -> 0.2331
  LCLS 222 subset         0.2344 -> 0.2330
  PAL 86 subset           0.3499 -> 0.3629  (binned CCD, stencil-limited)
For comparison --bandwidth 0.003 by hand: 0.2616 / - / 0.3108 / 0.2377 / 0.2460 / 0.2270 / 0.2404 / 0.3469.

Stills only: rotation never runs StillsPartialityRefine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

MergeOnTheFly: capture term and multiplicative outlier band for the stills merge (both opt-in) ... 419db6ba0e

--capture-uncertainty now reaches the stills merge: each partial carries (c (1 - p) <I>)^2 in its
variance, the rotation combine's capture term per observation, in the merge weights and in the
error-model samples alike. --reject-outliers on stills now uses the rotation merge's band
(OutlierBand.h): exp(+-n s) about the reflection mean for the systematic part, s the weighted median
|ln(I / median)| of the merge's own observations, instead of a symmetric +-n sigma band that reaches
below zero when b is large (stills b 1.4-1.7). A median needs three observations.

Neither is switched on for stills: measured with the run-measured partiality width before its
window-truncation correction (dimple R-free, SwissFEL 4k / SACLA 243 / MicroMAX mono / MicroMAX pink):
no options 0.2567 / 0.2497 / 0.3279 / 0.2477, --capture-uncertainty 1 0.2510 / 0.2470 / 0.3261 /
0.2527, --reject-outliers 6 0.2619 / 0.2487 / 0.3317 / 0.2630 - mixed, and the photon-counting pink
set loses with both. Raising --min-partiality to 0.1 or 0.2 lost 0.008-0.019 on SwissFEL and mono.
Defaults unchanged, so stills output is unchanged; rotation does not use MergeOnTheFly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills given a cell: the written cell is the mean over the frames that are merged ... 90b1b7a659

GetConsensusUnitCell averaged every indexed frame within 5% of the -C cell. The given-cell
population filter runs after it, so the MTZ, report and model-validation cell was a mixture of
both forms. Frames whose retained outcome is empty now do not count, and the consensus cell is
read again once the population filter has cleared the other form's frames.

Two-form P21 serial data (LCLS):
- T run: 93.85/93.66/49.98/92.00 -> 95.21/94.50/50.42/91.09 (kept population ~95.5/94.1/50.3/91.2)
- S run: 92.38/93.29/49.59/93.26 -> 91.90/93.06/49.45/93.65 (deposited 92.04/93.02/49.01)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills: the radial curvature correction decided per run on the same empty apertures ... 07a43a13eb

The first stills battery of e8145d8 lost on one Rayonix set (lcls242_s, dimple R_free 0.2302 ->
0.2323): there the whole-frame radial correction hurts. On 40,000 of its frames: battery default
0.2462, ring 12 with the correction 0.2444, ring 12 without it 0.2336, ring 14 with it 0.2467. Its
beam is strongly polarised and the azimuthally averaged ring background does not describe the local
background.

So the empty apertures now also carry the corrected difference: the frame's ring background
(RingBackground, factored out of the engine) averaged over the ring pixels minus over the disk pixels
- what the integrator's kernel takes off a reflection - is subtracted from each disk-minus-ring
difference. The ring is still chosen on the plain differences, so its reach does not rest on the
radial model; where the correction was left to auto it is then kept only if it makes that ring predict
the disk better. Decisions on 15-60 pre-scan frames:
  pal86 ring 10, correction on (240.6 -> 221.1)   lcls222 ring 10, on (6.34e4 -> 3.41e4)
  lcls242_s ring 12, off (277.2 -> 285.8)         alvra213_int16 ring 14, on (0.01424 -> 0.01422)
Choosing ring and correction jointly instead picked ring 14 with the correction on lcls222, where dimple
reads 0.204 against 0.196 at ring 10; a median instead of the mean square picked ring 14 with the
correction on lcls242_s, the worst arm there. The sequential rule is the one that agrees with dimple on
every set measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

StillsPartialityRefine: measure the run width only on shells with signal ... b440e3792b

Without a resolution limit the outer d*^2 shells of the width measurement lie beyond the diffraction
limit, where the summed intensity is noise or even negative; one such shell made the whole measurement
fail, and 5 of 11 still-battery sets (sacla237, pal86, pal90, lcls242_s, lcls242_t) fell back to the
per-crystal window fit. A shell now enters the fit only where its summed intensity is measured to 10 %
(sum I / sigma >= 10), and at least three shells are needed.

Weighting the fit by the shells' signal instead was tried and rejected: it lets the strong low-resolution
shells set the width alone, and on the MicroMAX pink set the dimple R-free went 0.2442 -> 0.2748.

The matrix subsets, which are cut at their resolution, are unchanged within 0.0010 dimple R-free
(SwissFEL 0.2453, -C/-S 0.2513, mono 0.3188, pink 0.2442, SACLA 243 0.2455, SACLA 237 0.2327,
LCLS 222 0.2335, PAL 86 0.3628).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills pre-scan: report the paired z of the radial-correction comparison ... 29000dd3d6

The background-ring log line now gives how many standard errors the correction lowers the chosen
ring's mean square by, over the apertures measured both ways. Diagnostic only; the decision is
unchanged. Full pal90: z = -2.7 (correction off), where dimple prefers it on (0.2124 vs 0.2183).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

module placement: the five concurrent fits each keep every thread ... 677ef2dfd4

One Ceres fit keeps only ~5-6 of 32 cores busy whatever it is given, and on a loaded machine (load
28-36, JF16M int16, 4000 strong frames) the five side-by-side fits measured 10.9 s with every thread
each, 13.8 s with half, 14.6 s with a fifth, against 18.9 s one after the other (bb7c9c). The
thread split is dropped: each fit gets gr_settings' threads, as before the fits ran side by side.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: score R-free against the deposited data through the same dimple protocol ... 0bf4bdb9d0

The published R-free comes from the depositors' own refinement (mostly phenix.refine), while the
battery refines with dimple's 8 REFMAC cycles, so the two are not like-for-like: on LCLS 222 (8TDQ)
the deposited intensities reach 0.1918 under dimple against a published 0.2012.

Every set now carries "R-free deposition (dimple)": the deposition's -sf.cif (intensity_meas, else
the mean of the anomalous pair, else F_meas_au squared) refined with the same refine() as our merge -
same deposited model, the deposition's free set, cut at its resolution. It is computed once per entry
and resolution and kept in the PDB cache (<pdb>-dimple-<d_cut>/reference.json); `run` and `report`
fill in a missing one, `reference` computes or lists them up front. The report gains the difference
(ours minus the deposition's) and a quality verdict (win below -0.01, draw within 0.01, loss
otherwise); the published R-free stays as an informational column.

Reproduces the hand-computed references: 8TDQ 0.1918, 9V3D 0.2220, 31JT 0.1945.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: the R-free target is the lower of the deposition's dimple and published values ... ccf81183cc

Owner's rule: a set is scored against min(R-free of the deposited data through our dimple protocol,
published R-free). The report shows both values, a "target" column naming which one was lower, and
the difference and win/draw/loss verdict against that target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills: background clip follows the radial curve; signal disk from the lattice spacing ... 61def00a1a

Two stills-only changes to the integration stencil.

1. Where the whole-frame radial curve is in use (the correction on), the ring's high-side clip
   follows it: a ring pixel is clipped against the threshold plus the curve's rise from the
   reflection's radius to its own (BraggIntegrationEngine::RadialClipOffset, rounded to whole counts
   so CPU, CUDA and Metal keep the same pixels). On a background falling steeply away from the beam
   the flat mean + 4 sqrt(mean) threshold cut the beam-side half of the ring as outliers, so the
   background read low and reflections near the beam stop read 3-150x too strong. The empty-aperture
   test's corrected column clips the same way, so the per-run correction decision sees it.
   Measured with r1 6 (dimple R_free, battery arguments, all frames):
     lcls222 0.196 -> 0.1856, pal86 0.216 -> 0.1997, sacla236 0.2559 -> 0.2559, alvra213_int16 0.1986 -> 0.1986.

2. The signal disk is taken from the lattice: it must not reach into the next node's spot. The
   closest two nodes can come on the detector is the shortest reciprocal-lattice vector the centring
   allows, s = F_px tan(2 asin(lambda / 2 d_max)), and a spot reaches 2 r80 (measured in the
   pre-scan); r1 = the largest of 4, 5, 6 px not above s - 2 r80, never below 4, never above the
   disk the run starts with (6). The empty apertures are measured for each candidate disk and the
   ring is chosen on the chosen disk's tally. Pixel detectors keep 6 (sacla236 s 22 px, alvra213
   20 px); the binned CCDs with a long axis shrink to 4 (lcls222 5.4 px, pal86 6.7 px), which the
   radius scans there measured as the optimum; pal90 (s 21 px) and lcls242_s (15.5 px) keep 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills keep an overloaded observation as a lower bound instead of dropping it ... 72cc542d78

The stills merge dropped every observation with a saturated pixel. On a still the saturated
observations are the fully recorded ones of the strongest low-resolution reflections, so what
survived were their partial observations, whose partiality correction then set the merged
intensity: on clipped CCD stills the merged low-resolution intensities came out at 0.4-0.6x of
the deposited ones, a ramp no Wilson B absorbs, invisible to CC1/2 and R_split.

The pixels are really clipped (flat-topped plateaus a few per mille wide, a pile-up in the
pixel-value histogram from the per-pixel ADC limit), so the auto ceiling is right; the drop was
the defect. A still now keeps the observation: the box sum with every saturated pixel counted at
the saturation limit, unflagged and marked clipped, counted in BraggIntegrationCounts and
reported as OVERLOADED_REFLECTIONS_KEPT. Rotation data (IsRotationIndexing) are unchanged and
keep the flag for the rocking-event drop.

Measured on a clipped-CCD stills subset (dimple R-free, deposited free set): dropped 0.247,
lower bound 0.230 (same as switching saturation off). A Kabsch fit to the unsaturated wings read
lower than the lower bound (0.237), and inflating the lower bound's sigma lost (0.255).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Stills: drop the lattice-spacing signal disk; the correction decision back on the plain clip ... aee757c471

The validation battery of 61def00a1 (20261022-0925_29000d_still-integ-pal-r1clip) shows the r1 rule
buys nothing once the ring clip follows the radial curve: lcls222 at r1 4 (4,6,8) 0.1893 against
0.1856 at r1 6; pal86 0.1998 against 0.1997. The radius scans that motivated it were taken with the
flat clip, whose beam-side cut the smaller ring partly avoided. r1 stays 6 on stills.

The empty apertures' corrected column again clips flat. With the clip following the curve there,
lcls242_s read the correction as a clear gain (z = 11) and the run took it, but dimple on 40,000
frames says it costs 0.013 (0.2461 on against 0.2335 off); judged on the flat clip, as in 07a43a13e,
the decision is off there and unchanged on every other set. The integrator still follows the curve
in its clip wherever the correction is on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

battery: R-free against the deposition without refinement only, one protocol for rotation and stills ... 4d8c2cc031

The batteries now score the deposited model against a merge one way: model_check.py, REFMAC's first
cycle before any shift (bulk solvent and anisotropic scale by default), the depositor's free set,
d_min_used = max(ours, deposited), compared against the deposited data through the same protocol.

- model_check.py: the restrained-refinement protocol is removed (refine, refined_shared,
  rename_misfit_ligands, REFINE_CYCLES, the refined parameter, --no-refine). Both data sets are now
  scored on their common reflections only, so a low-resolution or anisotropic cut of either applies
  to both (measured on three still sets: <= 0.001 change).
- battery.py / paper_plots.py: refmac_refined_* and refmac_refined_ratio are gone; refmac_rfree_ratio
  is the headline; paper_plots draws the first-cycle pair only (--rfree removed).
- still_battery.py: dimple is replaced by model_check.check on the same p.mtz. R_free rugnux is
  rfree_depflags, the target is rfree_depdata (no longer min(dimple, published)); the published
  R-free stays as an information column. The dimple cache, free-flag writer, deposited-MTZ writer
  and the reference verb are removed. This also retires dimple's freerflag COMPLETE renumbering,
  which scored R-free on depositor work reflections whenever our merge had hkl the deposition
  lacks (model_check carries the depositor's flags per hkl and marks ours-only hkl as work, and
  with the common-hkl restriction there are none). `report RUN --out DIR` rescores an older run's
  merges without rerunning rugnux.

Refinement cost about 6.4 h per full rotation battery; the 0-cycle check takes seconds per set.
Verified: 9V3D still merge reproduces the 0-cycle common-hkl numbers (0.2461 ours, 0.2367 deposited).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: EuXFEL sets run on a 2,000-image subset for now ... cf125bddbe

A manifest entry may carry max_images, applied when no tier is given; both
EuXFEL entries get 2,000 (and a smoke tier of the same size), which keeps
their full-set run time out of the development batteries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge fixed-cost into stills-easy: stills geometry and pre-scan run in parallel, no output change ... 4dcd5f188d

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills geometry refinement: fit the detector distance and the cell together ... a54dbcd977

The stills bundle held whichever of the two it trusted - a given or stated distance, with the cell
refined to absorb any error, or, after DecideStillsGeometry had rejected the file's distance, the
user's cell anchored and the distance refined from it. The two are degenerate only near the beam: the
radius of a spot goes as D tan(2 theta) and the cell scales sin(theta), so the strong frames'
high-angle spots fix both. A distance given 1.4% short (the user's cell was the depositors' starting
cell, about a percent off the crystal) was absorbed into the cell, mispredicted the spots past 2.5 A
and made the merged intensities there negative.

Both are now free (refine_distance, no cell restraint) whatever their source; a --detector-distance
on stills with a reference cell is the fit's starting value, no longer a pin. The flags that chose
between the two regimes (stills_distance_nominal, stills_distance_from_cell_) are gone.

On a Rayonix stills set given 105 mm the fit converges to 106.1 mm from 105 and from 103 mm, and the
deposited model's R_free against the merge drops 0.354 -> 0.283 (all frames); a stills set whose
distance is right holds it (193.43 -> 193.42 mm, R_free unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills geometry refinement: a distance given with --detector-distance is held in the joint fit ... 45abde17a1

The joint distance+cell fit freed the distance even where the user had given it. A user-given distance
is a hard pin: the cell alone absorbs any error. The file's distance and DecideStillsGeometry's stay
free as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge integ-part into stills-easy ... 63d54ab883

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge integ-pal into stills-easy ... c89a4063ce

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge integ-a2a into stills-easy ... 948222606b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge stills-sat into stills-easy ... 6b5647e8ab

The one conflict, tests/BraggIntegrationEngineCPUTest.cpp, where both branches add cases at the same
place: integ-pal's side is taken and stills-sat's test diff applied on top, which keeps all four cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Metal integration engine: back to its panel-s5 kernels; Mac stills keep the old clip, no saturation count ... ecd67ec44f

The Metal kernel and host code are restored to their panel-s5 content, so neither the background clip
that follows a handed-over radial curve nor the count of saturated pixels in pass A runs on the Metal
path. This is an accepted, temporary platform difference, to be closed once the kernels can be checked
on a Mac: on macOS a still's background clip stays the plain one (the radial correction itself still
uses the handed-over curve, as on the CPU), and an overloaded observation kept as a lower bound counts
its saturated pixels as 0 instead of at the saturation limit.

One host-side change is kept, with no effect on any result: the radial sums are accumulated on the GPU
only where no curve was handed over. StartPassA now sizes those sums for that case alone, so the old
host code would write past them on every still with a curve.

The shared structs need nothing on the Metal side: BraggPassA.h is compiled into the kernel source, the
box-sum kernel zero-initialises BraggBoxSums (n_saturated = 0) and the host zero-initialises
BraggPassAFrame (n_curve = 0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Metal integration test: the CPU comparison runs on a rotation experiment ... 5f8c739358

Stills keep overloads as lower bounds on CPU/CUDA but not yet on Metal, an accepted platform difference
to be fixed. On a rotation experiment both engines flag the overloads as before, so the bit-identity
comparison stays meaningful. The engine reads the rotation flag only for that decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills: kept overloads go back to the drop where they are not concentrated at low resolution ... 0cf62452cd

Saturation clips the strongest spots, and those are at low resolution. On a Rayonix stills set the
saturated pixels sat almost entirely past 4 A (0.002% of the reflections coarser than 4 A against
0.59% overall) - flagged or hot pixels, not clipped spots - and keeping those observations as lower
bounds put the saturation limit into high-resolution intensities: R_split 0.12 -> 0.93, CC1/2 0.99
-> 0.33. The run now keeps overloads only where their share coarser than 4 A is at least 1.5x their
overall share; otherwise they are flagged and dropped as before, with a log line saying so.

On clipped CCD stills the share coarser than 4 A is 2.6-5.9x the overall one and nothing changes
(R_split and R_free identical); on the Rayonix set R_split returns to 0.12. A pragmatic gate until
saturation is told from bad pixels per spot. Rotation never keeps overloads and is unaffected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills distance search: an absolute 30-3000 mm scan, a sample found without d-limits, the fit iterated ... 1629b47eae

The stills distance grid spanned 60-160 % of the file's value, so where it looked depended on the value it
was checking: a Cheetah file stating 839 mm for a detector at about 106 mm left every grid point empty and
the run had to be given the distance. The search is now global first and local after, like the beam
centre's capture and walk:

- a scan over 30-3000 mm in 2.5 % steps (the peak's width) at one centre that does not depend on the file
  (the background's, else the capture's), on the 8 frames with most spots;
- its three strongest local maxima, a step either side, at every centre hypothesis on 32 frames; then the
  fine +-2 % step on the whole sample, which decides against the file's value with the rule as before.
  This replaces the relative grid run at every centre, and costs fewer trials than it did.

The geometry sample is found without the spot finder's resolution limits, which are d-spacings evaluated
at the distance being checked, and spots piled up at one radius (ice and other powder rings) are left out
by their radius in mm. Where the decision moves the distance by more than 5 % the sample is found again
there and the fine step asked again.

A --detector-distance on stills is a hard pin: not searched, and held by the joint fit (the cell alone is
refined). Without one, the file's value is a hypothesis against the search's best, kept unless that puts
materially more spots on the lattice. An XTC that states no distance now runs with -C: the search decides
it, and the first refinement, which would start from a placeholder, is skipped.

The joint distance+cell fit indexed its bundle at the geometry it started from, so its end point carried
the start (0.2 % apart from two starts a percent apart). Where it moves the distance by more than 0.3 %
(and no modules are being placed) the bundle's frames are indexed again at the fitted geometry and fitted
again, up to three times, until the distance moves by less than 0.05 %.

From 839, 50 and 300 mm a Rayonix set ends at 106.24, 106.23 and 106.22 mm; an XTC set run with no distance
ends at 193.41 mm against the 193.43 it is otherwise given.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

battery results: XFEL still battery 20261022-1358 on rc178 0cf62452c - 12 sets, cold, R_free unrefined ... 2d7125260e

Twelve public XFEL serial sets (SACLA, PAL-XFEL, LCLS, European XFEL, SwissFEL), each read cold from
disk on the workstation, with time to map, processing rate and R_free of the deposited model against
our merge and against the deposited structure factors under one unrefined protocol: 6 wins, 4 draws,
2 losses (more than 0.01 apart counts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

battery results: still summary gains the resolution of our merge and of the deposition ... 4858816e2a

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Pre-scan masks a pixel at the saturation limit on most frames, as one holding the error value ... b7e4ea23a3

A pixel stuck at the saturation limit was never masked: the defective-pixel scan counted only the
error value as "invalid on most frames", and a saturated pixel merely as lit, which the hot rule
does not judge on stills. A Rayonix stills set has 3,627 pixels at 65535 on every frame, at high
resolution; each flagged every reflection whose disk it fell in as overloaded, and the spot finder
found them on every frame. The saturation limit now counts with the error value (CPU and CUDA), so
such a pixel is masked like one holding the error value.

On that set the indexed fraction rises from 10% to 33% (and 7% to 30% on the second lattice of the
same run), R_split 0.120 -> 0.106 and the unrefined R-free against the deposition -0.001 / -0.024.
Rotation merges checked bit-identical (three in-house sets with overloads).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Stills keep an overload as a lower bound only where it has pixels at the limit; run-level gate removed ... 5905d34cba

The lower bound counts each saturated pixel at the saturation limit, so it needs one. An observation
overloaded only by pixels unreadable on this frame (a per-frame bad-pixel flag, data value 0) was kept
as a box sum with those pixels counted as zero, which bounds nothing; it now stays overloaded and is
dropped, per observation.

With the stuck pixels masked by the pre-scan, the run-level gate (kept overloads must be enriched
coarser than 4 A) has nothing left to catch: without it every still set gives the gated result
(clipped CCD and Rayonix sets identical to the fourth decimal, photon-counting sets unchanged), so it
is removed with Reflection::overload_kept. A neighbour-brightness test per saturated pixel was tried
and not kept: after the mask it only dropped 9-20% of the Rayonix overloads and worsened R_split on
one set (0.245 -> 0.278).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

XFEL reader: a per-frame mask is read the way the frames are, its chunk inflated outside the HDF5 lock ... 9507b064b4

The Cheetah CXI per-frame mask (N x H x W uint16, one gzip chunk per frame) was read with
ReadVector<uint32_t> under hdf5_mutex: the inflate and HDF5's element-wise type conversion serialised
every reading thread and held an AGIPD run to ~40 frames/s. It now goes through ReadStored, as the
data does: only the chunk lookup is under the lock; the read, inflate and unshuffle are not, and the
stored uint16/uint32 values are widened to uint32 exactly as HDF5 converted them.

The direct path is taken where the mask is plain unsigned integers of the frame's size, stored alike
in every file (checked once, in Place); a chunk stored with a filter skipped is read by HDF5 as for
the data, and any other mask keeps the locked HDF5 read. Images and pixel mask are bit-identical to
before (hash of every pixel of 2000 images on two AGIPD runs and one MPCCD run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge stills-satspot into rc178: per-spot saturation on stills (stuck pixels masked, lower bound only with pixels at the limit), run-level gate removed ... fe9e335660

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge agipd-mask-speed into rc178: XFEL per-frame mask read outside the HDF5 lock ... a6835fe9d3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills distance search: every centre scanned with a wider acceptance, a larger sample where nothing indexes ... be45f99298

The absolute distance scan ran at one centre on the 8 strongest of 96 sampled frames, at the run's indexing
tolerance. Three ways that found nothing, and the run then stayed at the file's value or, on an XTC, at the
100 mm placeholder:

- a sparse I-centred lattice whose file states 180 mm for a detector at ~135 mm indexed at no distance from
  the background centre (7 px off) - only from the file's centre. Every centre hypothesis is now scanned;
- a 92 A cell at ~199 mm indexed on no centre 3 px off, nor at a distance half a scan step off. The scan's
  frames are now indexed at tolerance 0.2 instead of 0.1; the scan only has to find the peak, and every
  later step (the peaks on 32 frames, the fine step, the centre grids) counts at the run's own tolerance;
- on a run with a hit rate of a few percent the 8 strongest of 96 frames held no crystal. Where nothing
  indexes at any distance at the run's tolerance, the sample is made ten times larger and the search run
  again, the scan on 16 frames.

With these an LCLS XTC (no distance stated) with two crystal forms finds 198.93 mm against the 199.0 it was
given (R-free unpinned 0.236, pinned 0.238); a Rayonix CXI stating 839 mm finds 106.23 mm (0.206 vs 0.204
pinned at 105 mm); another XTC finds 193.41 against 193.43 (0.202 vs 0.200); the 180 mm file still finds
136.15 mm. Sets whose stated distance is right keep it, with unchanged results. The fallback costs ~30 s,
only where it fires.

RefineStillsGeometry also reports the slope of the indexed spots' signed excitation error against d*^2 at the
geometry it starts from: positive where the distance (and the cell) is short, negative where long. Reported
only.

The usage message and docs no longer describe --detector-distance as the joint fit's starting value (it is
held since 45abde17a), and say that an XTC with -C has its distance searched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge stills-dist: stills distance found without a given distance (absolute scan at every centre, enlarged sample fallback) ... b21057ed10

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

rugnux usage: --beam-center-check describes the absolute 30-3000 mm stills distance scan, not the old 60-160 % grid ... a19af56067

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge branch 'stills-dist' into stills-tilt f72950612a

stills: detector tilt refined in the geometry pre-pass, kept where held-out frames agree ... a39e18b80f

The stills geometry bundle (RefineGlobalGeometry) can now refine the detector tilt
together with the beam, the distance, the cell and the modules. With the tilt free the
beam block is the direct beam, from which the PONI is worked out inside the residual: in
PONI terms a tilt moves the whole pattern as a moved beam does (D tan(tilt)/pixel), more
than the beam's 8 px box holds, while the direct beam does not move with it and the tilt
is left with the keystone. The two tilt-like rows of ModuleGauge are gone: within a
module a tilt is a gradient no rigid move makes, and the rows cost held-out accuracy.
The shift, turn and outward gauges stay.

The tilt is a hypothesis (DecideStillsTilt): the strong frames of the last geometry
refinement are split in two halves, each fitted with the tilt held and refined (modules
placed in both where the run placed them), each fit scored on the other half, and the
tilt is kept where the refined fits predict the held-out spots better. No threshold.
It is decided after the stills beam-centre trial and its re-refinement, on frames
indexed at the settled centre: a centre error on frames indexed at it reads as a tilt.
--rot1/--rot2 or --no-refine-tilt hold the tilt on stills.

The stills beam-centre trial now compares direct beams. Its hypotheses (the file's
centre, the background walk, the point-symmetry capture) are taken as the point the
beam lands on and each trial puts the PONI where its distance and the run's tilt land
the direct beam there; the walk, fitted through the geometry's tilt, returns a PONI and
is converted. The pre-scan beam-centre check log compares the walk's direct beam, not
its PONI, with the file's direct beam. Without a tilt nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

CLAUDE.md: the changelog is written once, just before the merge into main - condensed one-liners of what users get ... ae1169072a

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

broker: serve the placed model and its maps of the last rugnux live run (/rugnux/model.cif, /rugnux/map.ccp4) ... 15ca4b329f

A rugnux live run that placed a model keeps it, with a copy of its map coefficients, beside the live
status (RugnuxStructure), for as long as /rugnux/status keeps the run. Made in memory on the first
request and cached, never written to disk:
- GET /rugnux/model.cif: the placed model as mmCIF, labelled as WritePlacedModel labels _model.cif
  (PlacedModelMmcif, the labelling factored out of WritePlacedModel; the file output is unchanged).
- GET /rugnux/map.ccp4?type=2fofc|fofc: the 2mFo-DFc / mFo-DFc map as a CCP4 map, FFT of the
  FWT/PHWT or DELFWT/PHDELWT coefficients (get_f_phi_on_grid + MapFromFPhi, sampled as ModelValidation
  samples), cut to the box around the model plus 5 A, with the whole cell's mean and RMS in the header.
- 404 without a placed model or maps, 400 for an unknown map type; both routes protected by the dataset
  bearer token like /rugnux/status. rugnux_live_status gets placed_model.

C++ model, python client docs, TypeScript client and redoc regenerated.

Tests: a point's coefficients give a map peaking at the atom in a cropped box; the mmCIF; the structure
is cleared at the next start; the auth test covers the new routes. End to end (myoglobin 2M replay with
an AlphaFold model): the maps equal gemmi's FFT of the written _maps.mtz to 2.5e-6 rms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 33124c3288ba112def3f51c0d91f5206886f3495)

frontend: Mol* viewer of the placed model and its density on the Rugnux live page ... a105c67f62

When the run placed a model, the Rugnux live page shows it in a Mol* viewer (molstar 5.13.0, pinned,
loaded lazily in its own chunk): cartoon, ligands and ions as ball-and-stick, the 2mFo-DFc map as a
blue mesh (1.0 sigma) and the mFo-DFc map in green/red (+/-3 sigma), each with a contour-level slider
and a show/hide switch. As in Coot, the maps are drawn within a sphere (10 A by default, adjustable,
0 = the whole box) around the centre of view, which follows the camera. Sigma levels are relative to
the whole-cell statistics the broker writes into the map headers. Reloaded for every new result;
the plugin is disposed with the component.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 08295be10c1d7a76bc7ecdf5df46fd7a3e1febac)

writer: _model.cif from rugnux live keeps the input model's metadata, as the served model.cif does ... 90bbba33bc

The placed-model END block lost everything but the atoms, so the writer's _model.cif had no sequence
references, authors, software or secondary structure, and gemmi relabelled the subchains. The block now
also carries label_asym_id, label_entity_id and label_seq_id per atom and the input model's metadata as the
mmCIF categories gemmi writes without coordinates (metadata_mmcif), which the writer parses with gemmi and
puts back. The writer's _model.cif is now the same text as the one rugnux writes itself and as
/rugnux/model.cif serves. Older blocks without the new fields are read as before.

Test: a predicted myoglobin model through END -> CBOR -> writer gives byte-identical _model.cif/_model.pdb
to WritePlacedModel and to PlacedModelMmcif, with its _struct_ref, _entity_poly_seq, _audit_author,
_software and _struct_conf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 29d752e6a107e3b8bc71ad6e9af0d81064966537)

rugnux live maps: one unit cell centred on the model; the viewer opens on all of it ... ee016d3d13

/rugnux/map.ccp4 now serves one unit cell of map, centred on the model, instead of the model's box plus
5 A; along an axis where the model reaches further than the cell (a predicted model with long termini
in a small cell), the model plus 5 A instead, so that all of it has density. The Mol* viewer opens on
the whole cell and the model (map radius 0); a radius still draws the maps only around the centre of
view, Coot-style.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 1c1e1d96359f112d6820d972005e9f7eca4ebd50)

Anomalous difference map: ANOM/PHANOM in _maps.mtz, /rugnux/map.ccp4?type=anom, a magenta mesh in the viewer ... 6abc44710d

Model validation already made the anomalous difference map's coefficients ((F(+) - F(-)) on the model
phases less 90 degrees, where the merge kept the Bijvoet pairs) to read the anomalous sites, then threw
them away. They now go into the map coefficients as ANOM/PHANOM (phenix.maps' labels), missing on the
reflections without a pair, so _maps.mtz opens as an anomalous map in Coot. Nothing in scaling, merging
or the validation's decisions changes; a merge without the Bijvoet split writes the twelve columns as before.

The broker serves the map as /rugnux/map.ccp4?type=anom (404 without Bijvoet pairs), and
rugnux_live_status.anomalous_map says it is there. The Mol* viewer loads it only then, as a magenta mesh
at +4 sigma with its own level slider and switch.

Tests: ANOM/PHANOM appear with Bijvoet pairs (as many as the reported pairs) and not without; the served
anomalous map of a point peaks at it; the auth test covers type=anom. End to end (myoglobin 2M replay):
anomalous_map true, the served map equals gemmi's FFT of ANOM/PHANOM in the written _maps.mtz to 3.4e-6 rms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit c4d6ad74c706181cbb183cfcaa2481653cd7cad6)

Mol* viewer: open with the maps drawn within 10 A of the centre of view ... 543d7548c7

The owner's choice: the viewer opens Coot-style, the maps within a 10 A sphere around the centre of view;
radius 0 on the slider still draws the whole cell.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 63121f35dd008e606f0fbb336b8ee0c330cde370)

Mol* viewer: aaredb's structure-viewer setup, on a black stage ... 2df1d8a2ad

The owner's verdict on the screenshots: aaredb (gitea.psi.ch/mx/aaredb, the PSI MX group's own
structure viewer) has it right, so its Mol* setup is taken over, files copied from
frontend/src/components/Structure at 8ece2c2 into frontend/src/components/structure:
- createPlugin.ts: headless DefaultPluginSpec plugin, manual camera reset; background black (owner)
  instead of aaredb's 0x0d1a26.
- MolstarCanvas.tsx: fresh canvas per mount, one-tick StrictMode deferral, resize, WebGL context-loss
  recovery, the Coot control profile wired.
- cootControls.ts / cootTrackball.ts: Coot mouse language - left drag rotates (Coot 0.9.8 trackball),
  right drag zooms, middle drag pans, middle click recentres on a residue, scroll clips the slab,
  shift+scroll zooms (camera sync and residue pick left out: no split view here).
- loadModel.ts: cartoon coloured by chain from a palette clear of the map colours, polymer sticks and
  waters built hidden, ligands as ball-and-stick; display modes cartoon / sticks / both and a waters
  switch; reads mmCIF too, and gives the atom coordinates for carving.
- loadMaps.ts: isosurface wireframe meshes at ABSOLUTE levels from the full cell's mean and RMS,
  2Fo-Fc steel blue, Fo-Fc green/red at +/-sigma, not pickable; plus the anomalous map as a magenta
  +sigma mesh in the same style.
- densityCrop.ts: the density worker's crop to the model's box + 6 A, the 4.5 A carve of the 2Fo-Fc map
  around the atoms (never a difference map) and the CCP4 encoding, applied to the broker's one-cell map.
The panel and defaults follow aaredb's StructureView: map rows with switch and sigma slider (2Fo-Fc 1.0,
0.5-4 by 0.1; Fo-Fc 3.0, 1-8 by 0.5; anomalous 4.0, 1-8 by 0.5), debounced sigma updates, model mode
buttons, a controls legend. This replaces the camera-following 10 A map sphere: aaredb's equivalent is
the carved map with the scroll-driven clip slab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 5763b619fe4ec9999ac790960a5eb033f511c546)

rugnux: the gain of a stills file that states none is read off its single-photon peak where the background resolves photons ... 87560311bc

A pedestal-subtracted JUNGFRAU background is mostly empty pixels; the variance per unit mean leaves
out the zeros and read 6.7 keV per photon on a 12 keV run in keV, 2.4 on a 4.6 keV one, so such
files needed --gain. MeasureGain now also histograms the sampled pixel values and takes the gain from
the single-photon peak (background_gain::PhotonPeak): the highest bin past the zero peak's descent,
standing 5 sigma above the valley before it, narrow (below half height a quarter of its value away),
and repeated by a two-photon peak near twice its value. Where there is no such pair the variance per
unit mean decides as before. Float frames are read for the measurement at a sixteenth of their unit,
so a photon of a few keV is resolved rather than rounded to whole keV.

Measured: float32 keV JUNGFRAU files 11.96 (photon energy 12.04 keV) and 4.53 (4.57 keV); the int16
photon copies show no peak past the first bin and keep the 0.55 variance answer, i.e. stay in
photons; the SACLA MPCCD and PAL Rayonix files show humps, not photon peaks, and measure as before.
A gain the file states or --gain gives still wins. Rotation is untouched (the pre-scan's CCD gain
uses only the variance estimator).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Mol* viewer: in line with AareDB's StructureView at its current head (8ece2c2) ... a624429090

AareDB's main is still at 8ece2c2, the commit the setup was copied from, so the structure/ files
match it line for line (bar the black stage, mmCIF input and the anomalous layer). What differed was
around them:
- molstar pinned to 5.11.0, AareDB's exact pin, instead of 5.13.0: the renderer's defaults
  (marking, outlines, occlusion, trackball) come with the version, so the same version draws the same.
- Only the first load frames the molecule: a newer result keeps the user's camera and clip slab, as
  AareDB keeps them across result switches.
- Load states as in AareDB: "loading model..." centred on the stage until the model is in, then
  "computing density maps..." in the panel with the map controls disabled until the maps are ready.
- Map rows labelled "2Fo-Fc" and "Fo-Fc (+/-)", AareDB's names.
- createPlugin.ts carries the credit line for the setup; its comment's directory path is ours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

ACKNOWLEDGEMENT: the web frontend's Mol* viewer follows AareDB's setup (G. Gotthard, PSI) ... 4abf498f54

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills tilt decision: the modules stay where the run placed them ... c7373be200

Refitting the module placement inside the tilt decision did not work on half the
frames: a module that half barely reaches is not placed at all and goes back to where
it was given, which made both arms worse and, on a modular detector, hid a real tilt
behind the placement's loss. The decision's arms and the tilted fit on every frame
now hold the modules where the module placement put them, so what is applied is what
the held-out test judged. Re-placing the modules with the tilt on every frame was
also tried end to end and gave a worse merge than holding them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge rc178 (Mol* viewer, changelog rules) into stills-tilt ... e6b0dfafca

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge stills-flags-alvra: a stills file's gain read off its single-photon peak ... 5bb4c30aa6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: no distance pins and no per-set tuning flags; gain from the data except where a file's baseline hides it ... 6f6f69a979

The stills distance is found from the data, so the --detector-distance pins go. Ice-ring detection,
the polarisation degree on one Rayonix set, the geometry pre-pass switch-off and the Alvra/one AGIPD
gains are no longer given: each was neutral or better without it. Kept: --wavelength where the file
states none or a different one, the polarisation on the mixed-form set (its beam-stop mask depends on
it) and the gain of one AGIPD set whose frames carry an uncorrected baseline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

battery results: rc178 6f6f69 - full still battery and full rotation battery with REFMAC ... 2b757b7574

Stills: 12 sets with no detector distance given to any of them. Against the deposition (0-cycle
unrefined R-free, same protocol): 6 better, 5 draws, 1 worse. The worse one is the minor crystal form of
a run that holds two forms; summary.md has the table and notes.

Rotation: 255 sets, compared with the rc176 full run. Pass rates are unchanged (open 197/202, in-house
47/47, plus one set new in the manifest), no verdict changed, total time +2%. The rows that moved beyond
noise are mostly lower R_meas.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

CHANGELOG: rc.178 condensed - XFEL reading and stills processing, Mol* view on the live page ... b6f4973908

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Bragg integration: a CCD rotation sweep's sigmas at the measured gain, intensities in photons ... 46da7fb359

The pre-scan measures the gain of an SMV/marCCD detector (background variance per unit mean) but the
integrator used it only for the background clip: sigma, var_bkg, the strong-spot flag (I/sigma >= 5)
and the profile-vs-box-sum guard (10 box sigmas) all treated one ADU as one photon, so on a Saturn
(g 4.7-5.8) the guard sat at ~4.5 true sigma. The new BraggIntegrationSettings::PixelGain (set by
rugnux from the same measurement, rotation only, at max(1, g)) takes every variance the engine forms
at the gain, and Finalize converts I, sigma, bkg and var_bkg to photons, which is what the merge's
variance model (var_bkg + I) assumes. CPU and CUDA engines; Metal runs the CPU code for this. Gain 1
(every photon-counting input, ADSC/marCCD whose measured ratio is below 1, and stills) is bit-identical.

Battery --only, 12 open-arm CCD/background-dominated sets vs the base build (987e812e2), GPU:
the six Saturn sets (g 4.7-5.8) all improve - guard fallbacks 4.1-17.3% -> 1.2-6.1%,
anomalous peak height 3mc4 x1.10, 3p85 x1.02, 3r6o x1.06, 5cc8 x1.03, 5uth x1.05, 5vml x1.04;
R_meas 0.880-0.993x on all six; R_model shell-scaled better on 5/6 (5cc8 +0.0005); ISa 5uth +1.5%,
5vml +15%, 3p85 +3%, the twinned 3mc4/3r6o -5%/-1%. 6cee, 6i3j, 6v2r, 7kcn, 8v4j, 9jzo unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

BackgroundGain: the gain of a frame that resolves photons is the spacing of its photon peaks, so a baseline left in the frame no longer hides it ... eeda3f9088

PhotonPeak read the gain as the single-photon peak's value, so it assumed a subtracted baseline: on
frames whose baseline was never subtracted (the zero-photon peak ~2 photons above zero) it found no
peak and fell back to the variance per unit mean, which the offset biases. It now takes the zero-photon
peak as the histogram's maximum, the gain as the distance from it to the single-photon peak, and the
narrowness and two-photon tests relative to that spacing (narrow: below half height at the midpoint to
the next peak; the AGIPD single photon is too wide for the old quarter-spacing test). The zero peak's
position is returned and logged as the baseline. A zero peak in the first bin - a subtracted baseline -
is taken at zero, so files that already worked read the same gain. A float file that holds whole
numbers, measured at a sixteenth of its unit, fills only every sixteenth histogram bin; the histogram
is read at that step. The histogram spans 16384 bins (a baseline of ~120 units at 1/16 resolution).

The baseline is not subtracted. Taking it off before the conversion (tried: still battery, the AGIPD
set with the unsubtracted baseline, all 33,528 frames) left R-free where it was (0.251 vs 0.252) but
indexed 53.8% of the frames instead of 66.4%: on a background of zero the spot finder's threshold falls
from ~9 to ~4 photons and picks up noise and diffuse rings.

Measured gain (16 frames, as in a run): the AGIPD set with the unsubtracted baseline 62.65 on a
baseline of 124.2 (was: no peak; the battery passed --gain 62); the other AGIPD set 53.77 (was 53.30,
the variance fallback); a JUNGFRAU float file at 12 keV 11.880 (unchanged); a JUNGFRAU float file at
4.57 keV 4.455 on a baseline of 0.1 keV (was 4.52 read as the peak's value; the 1->2 photon spacing
of the same histogram is 4.44); MPCCD, Rayonix and the int16 JUNGFRAU copies unchanged (no peaks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

ShadowFinder: measure the background's azimuthal modulation instead of predicting it from the polarization ... 59ae6fc5c8

The shadow finder divided the background by the Kahn factor computed from the experiment's
polarization degree, plane and detector distance before comparing each pixel with its ring. On a
stills run none of the three is trustworthy at the pre-scan: the degree is not identifiable from
the data, the plane can lie along the image's y axis rather than lab x (a detector mounted turned),
and the distance is still the file's placeholder, so 2 theta is overestimated. On such a run the
division made the ring anisotropy worse instead of removing it: a residual cos 2phi of 0.8 at high
angle, dim lobes over the detector corners sitting on the PENUMBRA_RATIO threshold, and the
transmitting-hardware search turned them into false pieces of 17-21k pixels whose extent followed
the third decimal of p (22973 / 25102 / 26190 pixels masked at p = 0.99 / 0.995 / 0.998, of which
only 5248 were the actual stop).

A polarized background is exactly a second harmonic in azimuth whatever the degree, plane and
2 theta, so it is now measured: the pooled background relative to its ring median, the medians of
24 sectors per 64 px band fitted with 1 + c cos 2phi + s sin 2phi (HarmonicFit, dim sectors left
out), band values interpolated in radius, a band that cannot be fitted taking the nearest fitted
one. The background is divided by that before the ring baseline is taken; the Poisson test still
puts the factor back. The finder no longer reads the polarization at all, on the host and on the
device. The arm search's own capped harmonic, which patched the old residual, is gone: the ratio it
works on is flat now.

On the stills set that showed it, the mask is the 5248-pixel stop at every p and at either plane,
and the merge statistics are the same at p = 0.99, 0.995 and 0.998 except for continuous changes
from the integration's own polarization correction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

BackgroundGain: one gain estimate for rotation and stills; the still battery no longer gives EuXFEL 152 its gain ... dda86080e2

background_gain::Estimate is the gain of any input that reports detector units: the photon peaks where
the histogram of a few frames shows them (PhotonPeak), else the background's variance per unit mean
(Gain). The stills reader path (MeasureGain) and the pre-scan's CCD measurement now both call it, so a
rotation sweep and a still read the same number from the same frames.

On a CCD the photon-peak test now also sees the pixels outside the active area, all at zero: on one
Saturn sweep that spike, the background hump far above it and the hump's echo at twice the value passed
as photon peaks (gain 2725). A zero-photon peak is the detector's noise and has a width, so a single bin
with nothing beside it is refused. Checked on the 39 SMV/marCCD sets of the battery: every pre-scan
gain is the same as rc178's (no photon peaks; the variance per unit mean as before).

Where the gain is applied stays as it is: a stills file is converted to photons as it is read (the
spot finder and the stills measurements of spot width, bandwidth and partiality work in photons), a
rotation CCD sweep keeps its raw counts and the gain enters the integrator (pixel_gain, 46da7fb35).
Moving the stills to the integrator route too was tried (stills PixelGain, no reader conversion) and
lost badly on MPCCD: R-free +0.032 and +0.047 on two SACLA sets, R_split 0.055 -> 0.091 and 0.067 ->
0.144, with the indexing unchanged; the stills rocking width/bandwidth fit, measured on frames in ADU,
collapsed (bandwidth 0.0015 -> 0.0000).

still_manifest.json: euxfel152 drops --gain 62; the gain is now read off its photon peaks (62.65, on a
baseline of 124.2 left in the frames): R-free 0.252, the same as with --gain 62.649 or 61.5 given by
hand (0.252, 0.252); --gain 62 gives 0.2495 - the R-free moves by ~0.003 with the gain. 66.4% indexed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

ShadowFinder: measure the modulation robustly under a wide one-sided shadow ... 2fce66598b

The first version took each ring relative to its median and fitted the harmonic of every band
on its own. On a sweep whose detector is a third covered by a smooth shadow on one side, the
median of a half-covered ring sits on the shadow's edge and jumps from ring to ring, and the
outer bands, seen only as arcs beside the shadow, fitted a harmonic of the wrong sign; the arm
search, no longer gated by its own harmonic, then joined the shadow's soft edge into large pieces.

- Each ring is taken relative to its upper decile, which stays on the lit side of such a shadow.
- Each sector is replaced by the brighter of itself and the opposite sector before the fit: the
  modulation is the same at phi and phi + 180 deg, a shadow almost never is, and shadows only
  darken.
- A band is fitted only when three quarters of the azimuth are seen that way; otherwise it takes
  the nearest fitted band's harmonic.
- The arm search keeps its capped harmonic of the ratio, as before the first version, so a wide
  smooth shadow is explained rather than added as a piece.

On that sweep the measured harmonic now rises smoothly with radius, and the mask covers the
shadow's soft edge (the shadow was under-masked before): R_free 0.327 -> 0.302, ISa 4.2 -> 4.6.
The stills mask stays the stop alone at every polarization, on the host and on the device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

ShadowFinderTest: a one-sided shadow is not fitted as part of the background's modulation ... f6ef620afd

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge x-gain: one gain estimator for rotation and stills, read through an unsubtracted baseline; CCD rotation gain in every variance ... 6de7d146c7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge x-polar: the shadow finder measures the background's azimuthal shape instead of predicting it from the polarization ... fcf3867ba3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

ShadowFinder: back to the predicted polarization factor, in the plane the background shows ... fc86fd4a10

The measured-modulation finder of the previous three commits is withdrawn. On a CCD sweep with a
holder arm along x it lost the arm: a small rise in the deep pixels' count promoted part of the
arm to shadow, which left the rest of the arm too few deep pixels to pass as transmitting
hardware, and the mask fell from 2.5 % to 0.4 % of the detector. On a 16-Mpixel stills detector
the per-band fit went wrong in the outer bands and masked 3.6 % of the detector instead of 0.03 %.
The shadow finder is rc178's again.

The stills defect it was meant to fix came from dividing by the factor of the wrong plane: the
background of that run is suppressed along the image's y axis, not lab x, and the division left
dim lobes over the corners on the arm search's thresholds, so that the third decimal of p moved
thousands of pixels. The degree cannot be told from the data, but the plane can. Before the
correction the mean projection is compared with the profile a polarized source gives each radial
band (rings relative to their upper decile, each sector the brighter of itself and the one opposite,
bands seen over three quarters of the azimuth); the stated degree is applied in the plane the
background follows when it follows one clearly (correlation > 0.5, at least 0.3 of a fully
polarized source's modulation), and as stated otherwise. Over 46 rotation sweeps and 11 stills runs
only that one run is turned, so every other mask is unchanged; there the mask is the 5248-pixel stop
at p = 0.99, 0.995 and 0.998.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge x-polar: rc178's shadow finder, with the polarization plane read from the background where it clearly follows one ... e54f5547d6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

MXAnalysisAfterFPGA: the Bragg engine's edge mask in the image's layout, so raw / untransformed FPGA runs start again ... 0bd60896c1

The FPGA path built its CPU Bragg engine with PixelMask(experiment), which is always in the converted
(transformed) layout. With Raw() or GeometryTransformation(false) the image is in the raw layout, so the
mask was larger than the image - read out of place, but silently. a8202789f made the Bragg engine refuse
a mask whose size is not the image's, so every FrameTransformationThread threw "Pixel mask size doesn't
match the image" at setup, the collection was cancelled and no image reached the pusher (9 unit tests:
JFJochReceiverTest_Raw/_PacketLost_Raw/_EIGER/_Conversion_NoGeomTransform, HDF5FilePusher_Raw and the
StreamWriterTest_ZMQ family).

The mask is now taken with GetMask(experiment), i.e. the raw-layout copy when the geometry is not
transformed; unchanged for transformed runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge ci-fix-rc178: raw/untransformed FPGA runs start again (Bragg engine edge mask in the image's layout) ... df47a2f74b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

battery results: still battery on rc178 e54f55 replaces the earlier rc178 still tables ... 077fad9379

Same 12 sets, cold, now with no gain given to any set (EuXFEL 152's is read from its photon peaks)
and the shadow finder's polarisation plane read from the background. 6 wins, 5 draws, 1 loss as before;
LCLS 242 T 0.349 -> 0.294, EuXFEL 83 0.222 -> 0.217, EuXFEL 152 0.249 -> 0.252, the rest equal.
The two earlier rc178 still result directories (20261022-1358, 20261022-2228) are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

XFELReader: per-frame mask and photon conversion without a library call or a branch per pixel ... d345deaecd

On a deflated Cheetah CXI with a per-frame mask (AGIPD 1M), half of the reader's time after the inflate
went to turning the stored frame into the image: the mask plane widened into a zeroed uint32 copy before
being tested, MarkBad testing it with a branch per pixel through a reference the store could alias, and
the photon conversion calling lroundf and branching on the mask, the finiteness and the clamp. Now the
mask is tested as stored (uint16 or uint32), and both loops are written so that the compiler vectorises
them: std::lround spelled out exactly (truncate, the exact fraction, a half or more away from zero) and
the selects done as arithmetic (Pick). Plain C++, no intrinsics.

Bit-identical: FNV hash of every image handed out (2000 frames) plus the pixel mask, before and after,
on a CXI with a CrystFEL geometry (with and without a gain), a panel-table HDF5 (with and without a
gain) and a bitshuffle/LZ4 int16 set: all identical. Reader alone, one thread: 14.1 -> 7.0 ms per
frame. rugnux on the 14,445-frame CXI run (16 workers, A/B back to back): geometry refinement sample
11.4 -> 8.9 s, image pass 27.6 -> 21.1 s, whole run 67.4 -> 58.2 s; geometry and refinement log lines
identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills geometry trial: the hypotheses a search asks are indexed side by side ... 19a38e538f

Each hypothesis used to be indexed on its own: 8 frames over a four-thread indexer pool, one after another,
about 700 of them on a run that searches the distance. The trial ran at 1.5 cores with the card two
thirds busy. Now the hypotheses a search compares (the distance scan of every centre, the grid about the
scan's peaks, the fine step, the centre grid's coarse and fine rings, the centre candidates) are indexed
together, 64 at a time, every frame of every hypothesis one work item, on a pool of as many indexer
threads as the image pass has (8). The counts are then read back one by one in the order the search
compares them, so every decision is taken exactly as before; a hypothesis' count does not depend on what
is indexed beside it.

Every trial count, the centre and distance chosen are identical on the five stills sets checked
(the 511-703 per-hypothesis debug lines match line for line). Trial alone on the 1 Mpx AGIPD set:
8.3-8.5 s -> 6.9 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills tilt decision: the fits start beside the pre-scan and the geometry trial ... edceecdae8

The tilt decision's five fits (the two halves with the tilt held and refined, each scored on the other
half, and the tilted fit on every frame) read only the frames the geometry refinement bundled, the
geometry it left and the cell. They now start as soon as the first refinement has made the bundle, on a
copy of all three, and run beside the pre-scan and the geometry trial: the trial is GPU-bound at 1.5
cores, the fits are CPU-bound. DecideStillsTilt still decides where it did, after the trial, and takes the
fits started ahead only where the trial did not refine the geometry again and the beam, distance, tilt,
wavelength, pixel size, cell and crystal system are bit-identical to what they started from; otherwise
it fits again as before. Nothing is decided earlier.

On the multi-module sets, where the bundle is the 4000 placement frames, the tilt decision's 8-14 s are
hidden: the pre-scan step went from 17-25 s to 12-17 s (the trial itself slows from ~7 to ~11 s beside
the fits on a 16-core box). The tilt result, every trial count and the geometry chosen are identical on
the five stills sets checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

XFELReader: a deflated frame is inflated and converted a piece at a time, while it is in the cache ... f5a990dc8d

The reader inflated a whole deflated chunk (4 MB for a 1 Mpx int32 frame, 2 MB for its uint16 per-frame
mask) into a buffer and then converted it in a second pass; with 16 reading threads those buffers never
stay in the cache, so the conversion ran at memory speed. A deflated, unshuffled chunk is now inflated in
128 kB pieces (InflateInPieces, the zlib stream API - the stream's checksum is checked as before), and
each piece is converted into the image, or marked in the mask, before the next is inflated. Shuffled,
bitshuffle/LZ4, HDF5-read and single-panel (placed) frames are converted as a whole, as before.

Bit-identical: the FNV pixel hashes of 2000 frames on the five reader configurations of the previous
commit are unchanged, and a new test reads 300 x 300 frames (several pieces, a final partial one) with a
per-frame mask and a gain pixel by pixel. Reader alone, 16 threads, 3000 frames: 2.43 -> 1.73 s (the
previous commit's state -> this). rugnux on the 14,445-frame CXI run, against rc178 (A/B, two runs each):
geometry refinement sample 11.6 -> 5.1 s, image pass 27.7 -> 12.9 s, whole run 67.7 -> 46.2 s; the
geometry refinement, module placement and tilt log lines are identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

XFELReader: a byte-shuffled frame is unshuffled a piece at a time, a whole pixel per step ... 33eacea54c

The unshuffle of a deflated + shuffled chunk (a panel-table HDF5 run: 8.4 MB per frame) moved one byte at
a time with the pixel size read at run time, and wrote the whole frame before it was converted. It now
assembles each 2- or 4-byte pixel from its byte planes in a loop the compiler vectorises, a 128 kB piece
at a time, and each piece is converted into the image while it is in the cache, as for an unshuffled
chunk.

Bit-identical: the FNV pixel hashes of 2000 frames on the five reader configurations are unchanged.
Reader alone on the panel-table run, 16 threads, 2000 frames: the unshuffle went from 58% of the
reader's cycles to 13%; wall 7.7-8.6 -> 5.1-7.0 s on a loaded machine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

GeometryRefiner: residuals carry only the blocks a fit moves; beam and distance one block ... f6c948732b

The stills geometry refinement (module placement fits, the held-out fits that decide them, the tilt
decision) spent most of its time differentiating parameters it never moves and in Ceres' general
Schur elimination:

- Each spot's residual declared the rotation axis, the cell angles and the (usually held) tilt as
  blocks. Ceres sizes its dual numbers from the declared blocks, so a placement residual carried 20
  derivatives to use 12. They now enter as plain numbers (SpotResidual / TiltedSpotResidual).
- A fit that moves only the orientations (the held-out scoring) gets a residual over the
  orientation alone, with the observed and predicted vectors taken once per spot
  (OrientationResidual): 3 derivatives instead of 17.
- The beam and the distance are one block of three, so every refined block has three parameters
  and Ceres uses its fixed-size Schur eliminator (1.7x faster linear solve on one thread). A held
  distance is a SubsetManifold on that block.
- One robust loss shared by every spot instead of one allocated per spot.

Same arithmetic on the same values: replaying the dumped placement inputs of four multi-panel
stills runs (AGIPD 128 ASICs, MPCCD 8, JF4M 8, JF16M 32 modules) on one thread gives identical
held-out residuals, beam, distance, cell and every module shift at the printed precision. Single-
thread CPU of the nine placement fits: 397 -> 112 s (MPCCD), 179 -> 80 s (JF4M), 131 -> 68 s
(JF16M). On AGIPD the concurrent step goes 8.5 -> 6.4 s wall (quiet replay, 32 threads).

Test: RefineGlobalGeometry_ModulePlacement now also covers a held distance and an orientation-only
fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

GeometryRefiner: a spot's residual differentiates its observed and predicted halves apart ... c83b949b69

The residual of a spot is the observed reciprocal vector (beam and distance, the tilt where refined,
the module's move where placed) minus the predicted one (orientation, cell lengths), and the two
halves share no parameter block. AutoDiffCostFunction carried all 12 (up to 14) derivatives through
both halves; SpotCost differentiates each half on its own blocks (6-8 and 6 derivatives) and writes
the Jacobian blocks from them. Values and derivatives are the same numbers.

Replaying the dumped placement inputs of four multi-panel stills runs on one thread: identical
held-out residuals, beam, distance, cell and module shifts; single-thread time of the nine fits
(after the previous commit -> now): AGIPD 66.8 -> 37.4 s, MPCCD 111.8 -> 92.1 s, JF4M 79.8 -> 52.3 s,
JF16M 68.4 -> 43.1 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

docs: EXTERNAL_TEST_DATA lists every open-arm battery set, with corrected values and counts ... cf7d35d554

Twelve open-arm sets were missing from the page: 2WNN, 2WNQ, 2WNZ, 2XFW, 4BWL (the Zenodo
pathological-data series, 10.5281/zenodo.54568, with its Sci. Rep. 2018 paper), 5OJV, 6P8J
(SBGrid), 6OWW, 7Q6J, 8C3E, 9QVV (Zenodo) and 9QW2 (Keele). Their rows carry the RCSB
deposited values, the detector read from the files and a DOI verified against DataCite.
A new section describes the twinned and pseudo-symmetric rows (the C 2 2 21 vs P 21 question
of the series, 8C3E's merohedral twin and open screw, the 3MC4/3R6O reference_mismatch notes),
and 6OWW's accepted alternative joins the section on rows where reduction and deposition disagree.

Corrections to what was already there:
- d_min of 6JGJ (0.78), 7L84 (1.70) and 9YZK (5.10) are now the entries' refinement resolution
  like every other row; open.json's 6jgj dmin follows (report-only on the open arm).
- nine titles that had been shortened or paraphrased are now the RCSB titles verbatim.
- 9VX7 was on the page but never in the battery: it is now listed as not in the battery, why.
- the 6r72 battery row runs the low-dose collection, not the helical one; the page said both
  were rows and listed the low-dose one as a dataset without a PDB entry.
- 6H2P is two battery rows (6h2p_native, 6h2p_1p89A); the page now says so.
- the five XRDa DOIs are registered with JaLC, not DataCite.
- "five" primitive no-screw negative controls are twelve; 6PXB is an accepted alternative, not
  unscored; the stale "seven rows have no PDB code" paragraph is replaced.
- the collection counts (repository, facility, crystal system, structure factors) are recomputed;
  several were already wrong before this change (Diamond 20 for 14, CLSI 4 for 3, ...).

The stills page notes that pal90's beamline is CXIDB's (NCI) while 6IRK records CSI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

battery: doc_check.py cross-checks the manifests against the dataset pages ... 2ad4dd2e71

open.json against docs/EXTERNAL_TEST_DATA.md (membership both ways, space group, cell, d_min,
the stated row and repository counts, the small-molecule references), still_manifest.json
against docs/EXTERNAL_TEST_DATA_STILLS.md (membership both ways, CXIDB entry, PDB entry, space
group, d_min) and inhouse.json against the README's Arms table; exits 1 on a mismatch.
Stdlib only, no data needed. The README names it beside test_score.py and in "Adding datasets".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge fc-sample: the XFEL reader decodes a frame in cache-sized pieces with vectorised mask and photon conversion (bit-identical) ... 764d626867

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge docs-extdata: every battery dataset documented, manifests and data pages checked against each other (doc_check.py) ... 60f2ee5462

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

stills module placement: the placement fit gets half the threads, each held-out fit an eighth ... b84d648c05

The five placement fits run side by side and the placement on every strong frame is the one the
step waits for. With every fit on every thread, most of the extra threads spin on the Schur
eliminator's locks (system time ~50 s per step). Replaying the dumped inputs on 32 threads, quiet
machine (every fit on every thread -> placement 1/2, held-out 1/8), wall of the step:
AGIPD 5.5 -> 4.8 s, MPCCD 8.3 -> 7.3 s, JF4M 7.7 -> 6.7 s, JF16M 5.5 -> 4.9 s. Results identical
(the fits' outcome does not depend on their thread count, checked on the AGIPD and MPCCD dumps).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge fc-trial: the stills geometry trial indexes its hypotheses side by side; the tilt fits run beside the pre-scan (decision-identical) ... 3a7bb28ed2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

Merge fc-modules: module placement fits differentiate only what they move, the observed and predicted halves apart; threads split between the fits (decision-identical) ... 27a03fbb06

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

still battery: the EuXFEL sets run on all their frames, as the published still runs did ... a0ce6c1a57

The 2,000-image cap was left over from before the per-frame mask was read outside the HDF5 lock; the
smoke tier keeps its 2,000.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

battery results: still battery on rc178 a0ce6c, cold on a quiet machine, replaces the earlier rc178 table ... f7709f82c0

Same 12 sets plus the float32 Alvra originals; every R_free as before (6 wins, 5 draws, 1 loss), with
the reader, geometry-trial, tilt and module-placement speed-ups: the twelve runs take 1,237 s instead of
1,624 s, EuXFEL 83 35 s (408 Hz) instead of 69 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

macOS build: explicit size_t in the XFEL reader's row bounds; parentheses in the CPU preprocessor's select ... 094eee31a2

On macOS size_t (unsigned long) plus int64_t (long long) is unsigned long long, so std::max/std::min had
two argument types; the template argument is now given. The preprocessor's (masked | error) ? ... is what
the code already did - the parentheses only silence clang's -Wbitwise-conditional-parentheses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

leonarski_f added 1 commit 2026-10-23 16:54:05 +02:00

Merge main (v1.0.0-rc.178 squash, #88) into rc178: its content is already here (identical tree to b9ada7914), recorded so rc178 merges into main cleanly ... 7395cd2d9a

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu

leonarski_f merged commit dae3644fe9 into main 2026-10-23 16:55:14 +02:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

buntschu_d (buntschu_d) duan_j (duan_j) glettig_w (glettig_w) gotthard_g (gotthard_g) leonarski_f (leonarski_f) mx-gitea perl_d (perl_d) takaba_k (takaba_k) wang_m (wang_m)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mx/Jungfraujoch#89