Three follow-ups to the status_server namespace component and
recording feature:
1. StatusServer's recording settings are now AdjustableFS, not
AdjustableMemory - each a small JSON file under the same shared
eco_cnf_bernina/configuration tree channels_JF/config_JFs already
use. The point: a setting changed in one process is visible to any
other process reading the same filesystem (another session, or the
status server itself) with no REST call or other new communication
- confirmed with a test constructing two independent StatusServer
instances against the same config_dir.
2. RecordingSession.size_report(): per-channel projected bandwidth -
rate (this channel's own point count over elapsed time) times a
sample value's size (element count * dtype itemsize, "frequency *
data shape * bitdepth") - to judge storage/bandwidth cost from a
short trial recording before committing to a long one. Works on a
still-running recording. Exposed via GET /recording/<id>?size=1
(optional &size_top_n=N) and StatusServerClient.recording(...,
size=True).
3. failed_required for recordings: a channel that fails to attach and
belongs to a required namespace component (namespace.required_names())
is now reported separately in a recording's report() - the same
"everyone else failing is expected, this failing is not" distinction
/health's failed_required already makes for init failures. New
client-side warn_recording_failed_required (red, but does not
suggest reinit() the way the health one does - a demoted/
disconnected channel isn't fixed by rebuilding the namespace),
wired into Daq.start_scan_monitoring.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New eco.status_server.namespace_component.StatusServer (Assembly),
wired into bernina_daq.py as a top-level namespace object alongside
daq/scans - always built, independent of whether this session's own
Daq is using the server (ECO_STATUS_SERVER), since the server itself
runs regardless.
- Control: .restart()/.reinit(), delegating to StatusServerClient.
- Inspection: .status() (prints a one-line summary, warns in red about
failed_required, returns the raw /health body), .stats(),
.failures(), .monitor_policy() (new GET /monitor_policy route +
client method, surfacing eco.epics_utils.ca_tuning's adaptive
monitor policy from inside the server's own process - max_rate_hz,
how many channels are monitored/known-fast, sensitive-period state;
read-only, the threshold itself is a module constant today).
- .gui(): launches the Qt monitor as a detached subprocess, same
window `eco-status-server gui` opens (not an embedded widget -
noted as a separate, bigger follow-up if wanted).
- Recording settings (mode, min_interval, sample_interval,
max_value_elements, max_points_per_channel) as real AdjustableMemory
children with is_setting=True, so they show up the way any other
device's configuration does. Not yet consumed by Daq.start_scan_
monitoring (still its own mode="throttle"/min_interval=0.1
defaults) - wiring that through is the natural next step, flagged
explicitly in the module docstring rather than left implicit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Hit live on a real test scan: _push_acquiring_scan_status handed
datetime.now() straight to push_status(), which POSTs via
requests.post(json=body) - that serializes with plain stdlib
json.dumps and no encoder hook at all, so it raised TypeError before
the client ever got a response to fall back from.
Two fixes: the actual datetime is now a unix timestamp
(datetime.now().timestamp()) at the source, matching every other
timestamp already flowing through this file; and
StatusServerClient._post() now serializes the body itself
(json.dumps(..., default=_json_default)) instead of using requests'
json= convenience, so any future datetime/numpy value through any
POST this client makes degrades gracefully (numpy scalars/arrays
converted, anything else str()'d) instead of crashing outright.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
DetectorVirtual had no set_current_value_callback() at all, so a
computed Detector could never be live-monitored no matter how
monitorable its own parents were - it landed in a recording's
n_attach_failed the same as a genuinely dead PV, for a structurally
different reason. AdjustableVirtual already solved this generically
(CallbackComposedValue: subscribe to every parent's own callback,
recompute get_current_value() on any update, independent of whether a
parent is CA-backed or itself composed) - give DetectorVirtual the
same treatment via the same class, generalized to take its children
explicitly instead of reaching into AdjustableVirtual's own
._adjustables attribute.
Discovery is automatic: NamespaceMonitorStore already finds every
monitorable status detector via isinstance(ts, MonitorableValueUpdate)
(eco.elements.protocols), a pure duck-typed check - no wiring needed
beyond DetectorVirtual having the method now.
RecordingSession.start()'s recent seed-if-cheap logic (seed a channel
with its current value for free when that costs no CA traffic) now
also recognizes a CallbackComposedValue-backed monitor as free to
seed - it recomputes from already-monitored parents rather than
issuing a get of its own.
Found and fixed along the way: CallbackComposedValue.start() only
accepted add_current_value, not with_ctrlvars/auto_monitor -
RecordingSession.start() (and anything else treating a monitor
uniformly regardless of whether it is CA-backed or composed) calls
.start() with all three, so this raised TypeError for any composed
value that reached this path - previously unreachable for a
DetectorVirtual (no method at all) and apparently never exercised for
an AdjustableVirtual monitored this way either. Both kwargs are now
forwarded to every child, recursively, matching CallbackEpics.start()'s
interface exactly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two-tier fix for a namespace recording's "channel with zero updates
is simply absent from the file" gap:
1. RecordingSession.start() now seeds a channel with its current value
(add_current_value=True) when that's actually free: pyepics'
PV.get_with_metadata() returns the cached value with no CA traffic
when auto_monitor is already True and the channel is connected -
true for the large majority of channels under ca_tuning.make_pv()'s
default policy. A demoted "fast" channel or a disconnected one
would make this a real blocking get, so those are left unseeded, as
before. New n_seeded counter reports how many got the free seed.
2. For the rest, RecordingSession.backfill_from_status() lets a status
snapshot that was already being taken for another reason (a scan's
own status_run_start/status_run_end capture) opportunistically fill
in one value for any channel still at zero points - never a trigger
of new CA traffic on its own. NamespaceMonitorStore gains
pgroup/run_number on start_recording() (purely for this lookup) and
backfill_running_recordings(), wired into both /status/capture and
/status/snapshot right after their own snapshot() call. New
n_backfilled counter.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
POST /recording/capture: stops a recording, writes it, and uploads it
to the run - all in the background, mirroring /status/capture and
/aliases/capture. Stopping (detaching callbacks) happens synchronously
before responding, so the recording is unambiguously over by the time
the caller gets a job id; only the write (scales with points recorded)
and the broker upload are backgrounded. Load-tested against the live
server first (start/get_status/stop concurrently): snapshot latency is
unaffected by a concurrent recording (~11s either way), ~38-44% of one
core combined, no thread/memory leak across repeated start/stop.
Daq gains start_scan_monitoring()/end_scan_monitoring() - start a
namespace-wide recording keyed to the run, and capture+upload it -
built to be called at a scan's start/end but deliberately NOT added to
callbacks_start_scan/callbacks_end_scan yet. No local fallback: unlike
status/aliases, a local recording would need this session's own
namespace holding a live monitor per channel for the whole scan, which
is exactly the per-session cost the status server exists to avoid.
Also fixes a latent cross-feature bug found while adding this: GET
/status/job/<id>?include_status=1 merged in any pending
scans.acquiring_scan.* push (from the just-merged push_status
mechanism) by defaulting the job's "key" to "status_run_start" when
absent - but aliases-capture jobs (and now recording-capture jobs)
carry pgroup/run_number for their own reporting without ever setting
"key". Reading such a job with include_status=1 could have silently
stolen a push meant for the real status_run_start job of the same run.
Fixed by requiring an explicit "key" on the job before merging - only
genuine status/capture jobs set one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
scans.acquiring_scan.* (description, scan_command, initial_values, ...) can
never reach the status server's own snapshot: it's built from DetectorMemory
objects with no CA channel, and the server's channel registry is driven
entirely by Alias.get_all(), which only returns aliases that have one. The
run table's "Custom table" sheet already has header cells expecting these
exact column names, so they've been silently blank.
Add a small push mechanism instead of another metadata.*-style workaround:
Daq now hands the values it already resolves (the same ones metadata.* is
built from) to the status server via a new POST /status/push, which merges
them into the matching /status/capture job's status the next time it is
read with include_status=True - the natural point, since that's exactly
when daq_client asks for the run's status to fill the run table.
NamespaceMonitorStore.push_status/pop_pushed_status hold the pushed values
scoped by (pgroup, run_number, key), consumed once and dropped after 5
minutes if nothing ever reads them, so a push whose scan aborts can't
attach itself to a later, unrelated run.
The server already holds a fully-initialized namespace, so its
namespace.alias.get_all() is complete and free (pure in-memory tree
walk, no CA traffic). The client side's own copy_aliases_to_scan
called self.namespace.alias.get_all() unconditionally, including in
status-server mode - against a namespace that mode deliberately never
initializes, so aux/aliases.json was silently missing everything the
local session had not happened to touch.
- namespace_server.py: GET /aliases (synchronous, for direct
inspection) and POST /aliases/capture (async: compute, write
aliases.json, upload to the run - mirrors /status/capture, shares
its job-id space via GET /status/job/<id>).
- storage.py: write_aliases_file(), same group-writable-tree handling
as write_status_snapshot().
- client.py: StatusServerClient.aliases() / .capture_aliases().
- daq_client.py: copy_aliases_to_scan now asks the same per-scan
status-server decision append_start_status_to_scan already makes
and caches, delegating to the server when it's in use and falling
back to the local path (unchanged) otherwise.
Also fixes a latent bug found along the way: Daq.__init__ accepted an
`instrument` kwarg but never stored it as self.instrument, so every
f"/sf/{self.instrument or 'bernina'}/..." path in
_write_status_locally()/write_status() raised AttributeError on any
real Daq - masked because the only test reaching that path tolerates
"any exception" as "fell back correctly".
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Henrik Lemke
lemke_handClaude Sonnet 5