# PandABox free-running (`raw_stream_mode`) raw data schema

> **Status: schema for the acquisition side only.** This document defines
> what raw data `omny_panda` writes to its HDF5 async dataset when
> `raw_stream_mode: true` (see `docs/developer/panda_box_free_running_setup.md`
> for the hardware/mode setup, confirmed against real PandA hardware
> 2026-09-15). Reconstructing the previous averaged/stdev-per-position view
> from this raw data (binning by the `gate_detector_active` transitions or
> timestamps) is a separate, not-yet-planned analysis task and is
> intentionally out of scope here.

## Where this data lands

`omny_panda` is configured with `readoutPriority: async`
(`csaxs_bec/device_configs/ptycho_flomni.yaml`). Its `data` signal is a
`DynamicSignal` (`ophyd_devices/devices/panda_box/panda_box.py`) that BEC's
async file writer appends to an HDF5 dataset per scan, under this device's
async data group. In `raw_stream_mode`, each write corresponds to one
coalesced flush (see the setup doc's mode-comparison table), not one raw
PandA network frame — the row-level data below is still preserved exactly,
just batched differently for publishing efficiency.

## Per-row fields

Each row corresponds to one PCAP capture tick (one raw sample at the
free-running capture rate — `CLOCK1`'s configured rate, started at 100 Hz for
initial testing on 2026-09-15, see the setup doc for the current rate):

| Field (BEC signal name) | Source PandA block | Meaning |
|---|---|---|
| `cap_voltage_fzp_y`/`cap_voltage_fzp_x` etc. (raw analog/encoder value) | `FMC_IN.VAL1.Value` / `FMC_IN.VAL2.Value` / `INENC1`-`4.VAL.Value` | The instantaneous raw sample at this capture tick, replacing the Mode A aggregate (Mean/Min/Max). All six channels (both FMC analog inputs and all four encoders) are switched to raw capture together on this layout — not a subset. |
| `gate_detector_active` | `PCAP.BITS0.Value`, bit 0 (`TTLIN1.VAL`), masked in software — see setup doc | The detector-trigger signal, sampled at this same tick: `1` while a detector trigger is active, `0` otherwise. `PCAP.BITS0` is a shared 32-bit word; `PandaBoxOMNY` masks it down to bit 0 before this reaches the HDF5 file (see `_GATE_BIT_OFFSET` in `panda_box_omny.py`) — the raw word itself is not a clean 0/1. This is what a future reconstruction step would use to delimit which raw samples belong to which detector-gate window. |
| `frame_counter` | `COUNTER1.OUT.Value`, triggered off `TTLIN1.VAL`'s falling edge (i.e. increments once per *completed* detector exposure, not once per PCAP capture tick as originally guessed here) | Monotonically incrementing hardware counter. Doubles as the completion signal for `on_complete` (`PandaBoxOMNY._on_complete_free_running`, `panda_box_omny.py`) — waits for it to reach `num_points * frames_per_trigger`, since `*PCAP.CAPTURED?` free-runs continuously and never naturally stops while armed. Also the post-hoc data-completeness record: a gap-free, strictly increasing sequence means no detector exposures were missed between the PandA and the HDF5 file; a gap indicates lost data (e.g. from the Redis stream's `MAXLEN` trimming if the file writer fell behind — see the acquisition plan's data-path notes). Because it counts completed exposures rather than capture ticks, it verifies detector-frame completeness directly rather than raw-sample completeness. |
| `PCAP.TS_TRIG.Value` (existing) | `PCAP.TS_TRIG` | Absolute/relative timestamp already captured today; usable for time-alignment against other detectors' data during future reconstruction. |
| `PCAP.GATE_DURATION.Value` (existing, aliased as `pcap_gate_duration_value`) | `PCAP.GATE_DURATION` | Existing field, retained; less meaningful once the gate is held open continuously for the whole burst window rather than pulsed per point (see setup doc, Mode B gate/arm row) — re-evaluate whether this still needs to be captured now that real hardware testing has started. |

All rows across a burst window share the same set of fields; there is no
per-point boundary marker in this raw stream other than what can be derived
from `gate_detector_active` transitions and/or absolute timestamps — no
explicit "scan point index" field is captured by the PandA itself.

## What this does *not* include

- No on-the-fly or post-scan computation of Mean/Min/Max/stdev from the raw
  samples — the raw stream is the complete record; deriving the previous
  per-point statistics from it is a separate future task.
- No changes to how Mode A (legacy) data is written — this schema only
  applies when `raw_stream_mode: true`.