docs: big-picture rotation 3D integration + intrinsic widths (CPU_DATA_ANALYSIS)

Document the rot3d path that was missing: a new section on combining a
reflection's per-frame partials into one full (de-biased weighted combine,
captured fraction, capture-aware systematic uncertainty, XDS-order full
re-scaling) so the merge sees counting statistics instead of rocking-curve
slicing. Recast the profile-radius and mosaicity sections as what the system
does - profile radius as the intrinsic (bandwidth-deconvolved) width, mosaicity
by ML with a search window wide enough to capture the rocking tail and held
fixed during scaling - rather than the optimisation narrative.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs/CPU_DATA_ANALYSIS.md

@@ -537,47 +537,37 @@ Per-shell and overall merging statistics are computed on corrected intensities,
 
 Completeness requires enumeration of possible reflections given a unit cell and symmetry; where this is not fully available, completeness may be reported as 0 or omitted.
 
+### 10.6 Rotation datasets: combining partials into fulls (3D integration)
+
+In a rotation scan a reflection is recorded as a series of *partials* spread across the frames its rocking curve crosses. Merging those partials directly would force the merge error model to absorb the rocking-curve slicing as if it were measurement noise, capping the achievable $I/\sigma$. For rotation data Jungfraujoch instead **combines** each reflection's partials into a single *full* intensity first, then scales and merges the fulls — a 3D integration over the rocking curve.
+
+The combine groups each reflection's partials into rocking events (contiguous runs of frames) and reduces each event to one full:
+
+- **De-biased weighted sum.** Partials are combined by inverse-variance weighting, where each partial's variance is its background-noise component plus the *model* signal shared across the event (Kabsch profile-fit form). Using the shared model signal rather than the individual down-fluctuating intensity stops weak partials from being over-weighted, which would otherwise inflate the merged error model. The weights depend on the full, so the estimate is iterated.
+- **Captured fraction.** The partiality summed over the event, $f=\min(1,\sum_j p_j)$, measures how completely the rocking curve was sampled; it replaces a per-partial minimum-partiality cut, because an event seen over only a few percent of its curve is unreliable however many frames it spans.
+- **Capture-aware uncertainty.** A full captured incompletely ($f<1$) is extrapolated and biased high. The unobserved fraction is charged as an extra systematic uncertainty, $\sigma^2 \leftarrow \sigma^2 + \big(c\,(1-f)\,I\big)^2$, so the merge down-weights these extrapolated fulls and the error model treats their scatter as expected. It is enabled by default for the rotation path.
+
+The fulls are then re-scaled in the XDS sense — a per-image scale refit directly on the complete reflections under the unity partiality model — and merged (§10.4). Because every merged observation is now a counting-statistics-limited full rather than a partiality-divided slice, the error model reaches a far higher asymptotic $I/\sigma$.
+
 ---
 
 ## 11. Mosaicity and “profile radius” monitoring
 
-### 11.1 Profile radius (still excitation error width)
+### 11.1 Profile radius (intrinsic excitation-error width)
 
-A simple scalar “profile radius” is estimated from indexed spots using the distribution of $\Delta_\mathrm{Ewald}$. Two estimators are available:
-
-- standard deviation:
-  $
-  R \approx \sqrt{\frac{1}{N}\sum_i \Delta_{\mathrm{Ewald},i}^2},
-  $
-- robust MAD-based alternative (median absolute deviation), scaled by 1.4826.
-
-Operationally, predictions for still data may use a cutoff proportional to this width (e.g. $\Delta_\mathrm{cut}\approx 2R$).
-
-### 11.2 Mosaicity from rotation data (maximum likelihood)
-
-For rotation data, Jungfraujoch estimates mosaicity by maximizing a likelihood based on the XDS reflection fraction $R(\tau;\sigma_M/\zeta)$ as described by Kabsch (2010). In brief:
-
-- for each indexed spot, find its exact Bragg angle and the angular deviation $\tau$ from the frame,
-- compute $\zeta$ (the Lorentz/rotation-axis factor) for each reflection,
-- maximize $\sum_i \log R(\tau_i;\sigma_M/\zeta_i)$ over $\sigma_M$ (golden-section search).
-
-This yields a physically meaningful mosaicity estimate that feeds the rotation prediction (§8.3) and the rotation partiality model (§10.2).
-
-**Search-window caveat (important).** The per-spot Bragg angle is located by searching $\phi$ in a window around the frame. That window must be **wider than the oscillation** — wide enough to admit reflections recorded at large rocking offset (those with $|\tau|$ up to $\sim\sigma_M + \Delta\phi/2$). These tail reflections are exactly the ones that *carry the information about the mosaic width*: if the window is only $\pm\Delta\phi$ (the oscillation), the $\tau$ distribution is truncated at the oscillation width and the MLE **underestimates $\sigma_M$ by roughly $2\times$** (e.g. $0.066°$ instead of the true $0.13°$ on the HEWL test crystal). Jungfraujoch therefore searches $\pm(\Delta\phi + 0.8°)$; the MLE is then insensitive to widening further, because it weights each spot by its recorded fraction $R(\tau)$ which decays for large $|\tau|$.
-
-**Do not re-refine mosaicity during scaling.** The mosaicity also appears in the per-image scaling fit (§10.1), where it is *degenerate with the per-image scale* $G$ (both rescale the predicted intensity). A free refinement there collapses $\sigma_M$ toward its lower bound, undoing the estimate above. The scaling step therefore holds the indexing mosaicity **fixed**.
-
-**Why it matters.** A too-small mosaicity has two compounding effects on rotation integration: (i) the prediction (§8.3) admits each reflection on too few frames, truncating its rocking curve; and (ii) the rotation partiality $p$ (§10.2) is over-peaked, so dividing each frame’s partial by $p$ to recover the full amplifies error. The net result is rot3d-combined fulls (§10.4) that are substantially noisier per observation. Correcting the mosaicity is the single largest lever for rotation per-observation precision (asymptotic $I/\sigma$, “ISa”).
-
-### 11.3 Per-observation precision of rotation fulls (capture-aware weighting)
-
-In the rot3d combine (§10.4), one reflection’s per-frame partials are weight-summed into a single “full”. A full that was only captured over a fraction $f<1$ of its rocking curve is *extrapolated* — and these under-captured fulls are systematically biased and noisy (the partiality division magnifies error in the tail). Jungfraujoch charges the unobserved fraction as an extra systematic uncertainty,
+The “profile radius” is the intrinsic angular width of a reflection — crystal mosaicity plus beam divergence — estimated from the spread of $\Delta_\mathrm{Ewald}$ over indexed spots,
 $
-\sigma^2 \leftarrow \sigma^2 + \big(c\,(1-f)\,I\big)^2,
+R \approx \sqrt{\tfrac{1}{N}\sum_i \Delta_{\mathrm{Ewald},i}^2}.
 $
-with captured fraction $f=\min(1,\sum_j p_j)$ and coefficient $c$ (`--capture-uncertainty`, default $1.0$ for rot3d). The merge then down-weights the over-extrapolated fulls and the error model treats their scatter as expected, improving both precision (ISa) and accuracy (anomalous peak height). Unlike post-hoc outlier rejection, this corrects a real systematic rather than trading accuracy for $\mathrm{CC}_{1/2}$.
+When the beam has a finite energy bandwidth, that bandwidth smears each reflection radially by $\sigma_\mathrm{bw}\approx \mathrm{bandwidth}\cdot\lambda/2d^2$ (largest at high resolution), which also broadens the measured $\Delta_\mathrm{Ewald}$ spread. Since prediction re-applies the bandwidth term per reflection (§8.2), this contribution is deconvolved from the estimate — $R^2 = \langle\Delta_\mathrm{Ewald}^2\rangle - \langle\sigma_\mathrm{bw}^2\rangle$ — so that $R$ is the intrinsic width and bandwidth is not double-counted. Still predictions use an excitation-error cutoff proportional to $R$.
 
-Together with the mosaicity fix (§11.2), this brings the HEWL rotation crystal to roughly the ceiling of XDS’s per-image/absorption correction surfaces; the residual difference to XDS is the finer (azimuthal absorption, per-frame $B$) corrections, which are small here.
+### 11.2 Mosaicity from rotation data
+
+For rotation data the mosaicity $\sigma_M$ is estimated by maximum likelihood from the rocking offsets $\tau$ of indexed spots, using the XDS reflection-fraction model $R(\tau;\sigma_M/\zeta)$ (Kabsch 2010): each spot's exact Bragg angle is located near its frame, $\zeta$ (the rotation-axis Lorentz component) is computed, and $\sigma_M$ is chosen to maximize $\sum_i \log R(\tau_i;\sigma_M/\zeta_i)$.
+
+The $\phi$ search window for the Bragg angle is set **wider than the oscillation**, so that reflections recorded at large rocking offset are included. These tail reflections carry most of the information about the mosaic width; a window limited to the oscillation range would truncate the $\tau$ distribution and bias $\sigma_M$ low.
+
+The estimated mosaicity feeds the rotation prediction (how many frames each reflection spans, §8.3) and the rotation partiality (§10.2). It is **held fixed during scaling**: in the per-image scale fit the mosaicity is degenerate with the scale $G$ (both rescale the predicted intensity), so refining it there is unstable. A correct mosaicity matters because it controls both how much of each rocking curve is captured and the partiality used to form fulls (§10.6); too small a value truncates the captured curve and over-peaks the partiality, degrading the combined fulls.
 
 ---