docs(flomni): batch-validate S00391-399, resolve burst-grouping and reduced-data-mode open questions

Validated nine flomni_fermat_scan runs (S00391-399: single-frame,
frames_per_trigger=3 burst, and reduced-data-mode groups, two aborted
mid-scan) against rt_positions ground truth. All nine reproduce positions
with R^2 >= 0.9999999 on both axes and no anomalous residuals.

Resolves two open questions from sec 6:
- Burst-mode grouping: frame_counter increments once per sub-frame
  trigger, not once per point -- point index is
  frame_counter // frames_per_trigger, confirmed at R^2 = 1.0000000
  against S00395/396/398.
- Reduced data mode: characterized empirically via S00399 -- keeps only
  gate_detector_active==1 rows (~99.5% vs ~39% of rows in normal
  continuous mode), roughly halving row count with no accuracy loss.

Also nails down the abort-into-restage stale frame_counter mechanism more
precisely: S00393 aborted with exactly 29 points recorded -> S00394
inherits a stale-29 prefix; S00397 (burst) aborted at 17 points x 3
sub-frames=51 -> S00398 inherits a stale-51 prefix. Both harmless
(gate_detector_active==0 throughout the stale rows in both cases), and
both an exact match between the aborted scan's last count and the next
scan's stale value.

Note: this batch does not exercise the arm_scan_allowlist fix (5590633)
-- the running device-server process predates that commit -- though it
wouldn't have changed anything here anyway since every scan in this batch
is flomni_fermat_scan.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/PANDA_POSITION_VALIDATION.md

@@ -65,9 +65,15 @@ Under `entry/collection/devices/omny_panda_continuous/` in the master HDF5:
 
 1. Filter to rows where `gate_detector_active == 1` (real exposure, not
    inter-point flight time).
-2. Group the remaining rows by `frame_counter` (one group per scan point;
-   `frame_counter` ranges `0..num_points-1` for a single-frame-per-point
-   scan — see §6 for the burst case, still untested).
+2. Group the remaining rows by **point** index. For a single-frame-per-point
+   scan (`frames_per_trigger == 1`), `frame_counter` directly ranges
+   `0..num_points-1` and *is* the point index. For a burst scan
+   (`frames_per_trigger > 1`), `frame_counter` increments once per
+   sub-frame trigger, not once per point (confirmed 2026-09-22 against real
+   `frames_per_trigger=3` scans, `S00395`/`S00396`/`S00398` — see §7): the
+   point index is `frame_counter // frames_per_trigger`, and *all* of a
+   point's sub-frames' gate-active rows get averaged together into that
+   point's one `rt_positions` value, not kept separate.
 3. Average `interf_st_fzp_x`/`_y` and `cap_voltage_fzp_x`/`_y` within each
    group.
 
@@ -171,22 +177,24 @@ suspecting the conversion factors.
   `.../timestamp`, much shorter — one entry per buffer flush, not per row,
   see `_flush_raw_stream_buffer_locked` in `panda_box_omny.py`).
 
-## 6. Next steps (as of 2026-09-21, not yet done)
+## 6. Next steps (as of 2026-09-21)
 
-1. **Burst acquisition**: test the case where the number of PandA frames
-   equals the number of burst sub-frames (`frames_per_trigger > 1`), and RT
-   still logs exactly one `average_x_st_fzp`/`_y` per *point* (not per
-   sub-frame). This changes the grouping in §3.2 — open question going in:
-   does `frame_counter` increment once per sub-frame trigger (meaning
-   `frames_per_trigger` distinct `frame_counter` values need to be
-   aggregated per RT point) or once per point? Check
-   `panda_box_free_running_setup.md`'s completion-check description
-   (`COUNTER1` "increments on the detector-trigger's falling edge") against
-   how many falling edges occur per point when `frames_per_trigger > 1`
-   before assuming either way.
-2. **Reduced data mode**: not yet defined in this doc — get the actual
-   definition/scope from Mirko at the start of that session rather than
-   assuming; don't extend this validation methodology to it until then.
+1. **Burst acquisition** — RESOLVED 2026-09-22, see §7/§3.2: `frame_counter`
+   increments once per sub-frame trigger, not once per point;
+   `frame_counter // frames_per_trigger` is the point index, and all
+   sub-frames of a point get averaged together to match RT's one value per
+   point. Confirmed against real `frames_per_trigger=3` scans
+   (`S00395`/`S00396`/`S00398`), R² = 1.0000000 on both axes.
+2. **Reduced data mode** — characterized empirically 2026-09-22, see §7
+   (`S00399`): captures only rows with `gate_detector_active == 1` (or very
+   close to it — 99.5% vs. ~39% of rows in normal continuous mode), cutting
+   total row count roughly in half to a third while keeping the same
+   per-point exposure sample count and full positional accuracy (R² =
+   1.0000000). The underlying PandA-side layout/config change that produces
+   this is not visible from BEC's `configuration` snapshot in the HDF5 file
+   (empty for both modes) — this section describes the observed effect on
+   the data, not the PandA-side mechanism; get that from whoever configured
+   it if it needs modifying.
 
 ## 7. Post-fix confirmation against live scans (2026-09-22)
 
@@ -280,3 +288,66 @@ It's now only actually `Arm()`ed for `flomni_fermat_scan` via the new
 class of bug can even be triggered (fewer arm cycles → fewer abort→restage
 windows like `S00388`'s), independent of whatever the remaining root cause
 above turns out to be.
+
+## 8. Batch validation across `S00391`-`S00399` (2026-09-22)
+
+Nine `flomni_fermat_scan` runs at `/tmp/S00<391..399>/` on `x12sa-bec-001`,
+run as three groups: `S00391`-`394` (single-frame, `frames_per_trigger=1`,
+`393` aborted mid-scan), `S00395`-`398` (burst, `frames_per_trigger=3`,
+`397` aborted mid-scan), `S00399` (burst, reduced data mode). Note: the
+running `bec-device-server` process had been up since before commit
+`5590633` (`arm_scan_allowlist`) landed, so it was still running the
+pre-`arm_scan_allowlist` code for this whole batch — doesn't matter here
+since every scan in this batch is `flomni_fermat_scan`, which
+`arm_scan_allowlist` always arms anyway, but worth knowing: this batch
+doesn't demonstrate that fix, only the pre-existing stale-`frame_counter`
+fix (`3eb6e03`, which does predate the process start and was active).
+
+**Full position-recovery validation** (§3.2-3.4/§4 methodology, extended for
+burst grouping per §6) against `rt_positions`, all nine scans:
+
+| Scan | mode | points | R² x / y | residual std (nm) x / y | max abs (nm) x / y |
+|---|---|---|---|---|---|
+| S00391 | single | 99 | 0.9999999 / 0.9999999 | 0.94 / 0.71 | 2.64 / 1.83 |
+| S00392 | single | 98 | 0.9999999 / 0.9999999 | 0.95 / 0.76 | 2.75 / 2.09 |
+| S00394 | single (post-abort) | 97 | 0.9999999 / 0.9999999 | 0.93 / 0.73 | 2.28 / 2.04 |
+| S00395 | burst×3 | 98 | 1.0000000 / 1.0000000 | 0.47 / 0.55 | 0.98 / 1.18 |
+| S00396 | burst×3 | 101 | 1.0000000 / 1.0000000 | 0.41 / 0.55 | 1.12 / 1.17 |
+| S00398 | burst×3 (post-abort) | 99 | 1.0000000 / 1.0000000 | 0.45 / 0.54 | 1.32 / 1.13 |
+| S00399 | burst×3, reduced data | 100 | 1.0000000 / 1.0000000 | 0.48 / 0.50 | 1.69 / 1.14 |
+
+No point had zero grouped samples in any scan; no residual outlier anywhere
+close to the ~116 nm the pre-fix bug produced. Burst-mode R² came out even
+tighter than single-frame mode (1.0000000 vs. 0.9999999), consistent with
+averaging 3x more raw samples per point.
+
+**Stale-`frame_counter` prefix — two more clean, precisely-explained
+repros**, both immediately following an abort, both harmless (all stale
+rows have `gate_detector_active == 0`):
+
+- `S00393` aborted with `rt_positions` recording exactly 29 points (panda
+  `frame_counter` max = `29.0`, matching exactly). The very next scan,
+  `S00394`, starts with 37 rows reading a stale `29.0` before dropping to
+  `0` at row 37 — same signature as `S00388`, same stale value as `#393`'s
+  own last completed point.
+- `S00397` (burst×3) aborted with `rt_positions` recording exactly 17
+  points — 17×3 = 51 sub-frame counts, and panda `frame_counter` max there
+  is indeed `51.0`. The next scan, `S00398`, starts with 37 rows reading a
+  stale `51.0` before dropping to `0` at row 37 — same signature again, and
+  the stale value again matches `#397`'s own last completed sub-frame count
+  exactly.
+
+This nails down the trigger mechanism about as precisely as it can be nailed
+down from data alone: the leftover `frame_counter` value inherited by the
+next scan is *exactly* the aborted scan's last completed count (point count
+for single-frame, point×frames_per_trigger for burst) — not some unrelated
+stale value from further back. The remaining open question from §7 (why the
+client-side filter doesn't strip these rows in the abort→restage case, given
+`on_stage`/`_try_arm` both reset state normally beforehand) is unchanged by
+this batch — still worth instrumenting if pursued further, though it
+remains harmless in every case observed so far because the stale window
+never overlaps a gated exposure.
+
+**Reduced data mode** (`S00399`) validated cleanly at full accuracy — see
+§6 for the row-count/gate-fraction characterization. No contamination, no
+missing points, same R² bar as the other burst scans.