Fixes/flomni beamtime 2 #323

Merged
holler merged 78 commits from fixes/flomni_beamtime_2 into main 2026-09-22 16:27:21 +02:00
71 changed files with 5155 additions and 372 deletions
+62
View File
@@ -0,0 +1,62 @@
# Personal notes — mirko
Personal working notes for Claude Code sessions in this repo, for my own use across machines.
Not project-wide guidance — nothing here applies to other contributors, and nothing else in
this repo references or enforces it.
## Repo ownership boundaries
Root causes traced into BEC core (`bec_lib`/`bec_server`/`bec_ipython_client`, the `bec` repo)
or into core `bec_widgets` are not mine to fix here — those are owned by their respective core
teams. When an investigation in `csaxs_bec` turns up a bug whose fix belongs in one of those
repos, write it up (a short report or plan) instead of implementing/committing it there.
Fixes that land in `csaxs_bec` itself (beamline plugin code, widgets, device configs, scans)
are mine to implement, commit, and push directly.
## Where notes go
Never write plans, TODOs or notes to Claude's local memory / local plan files (they are
machine-local and invisible on other machines). Put them in this file, or in a markdown file in
the repo when I ask for one.
## TODO
- 2026-09-21: port "always define `tomo_alignment_fit`" from Flomni to OMNY. Flomni is done
(commit `5b688df`): `reset_tomo_alignment_fit()` writes zeros instead of deleting the global
var, and `ensure_tomo_alignment_fit()` (called from `Flomni.__init__`) creates the zero fit
only if missing, so manual tweaks like `fit[0][2] += 5.0` work. OMNY
(`omny/omny_alignment_mixin.py`) still deletes the var and never defines it at startup; nothing
in `omny/` calls the reset yet, so decide where OMNY should reset it. LamNI does not use it.
Details and steps: `csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/TODO_tomo_alignment_fit_always_defined.md`.
- 2026-09-20: `flomni_fermat_scan` hung on `pre_scan_all_devices()` after positions were
already sent to the rt controller (scans 11998, 12000 in `/var/log/bec/ScanServer.log` /
`DeviceServer.log`). Positions/PID-correction step (`prepare_setup_part_2()`) completed
fine; `kickoff()`/`start_scan()` (the actual rt-side trigger, logged as `Read N out of 106`)
was never reached because the joined `pre_scan_all_devices()` wait never returned. No error,
no timeout — DeviceServer just went silent until manually aborted.
Most likely culprit: `omny_panda_continuous` (PandaBox) — `on_pre_scan()` in
`ophyd_devices/devices/panda_box/panda_box.py:696` arms the PCAP block and returns a status
that only resolves on a `PandaState.READY`/`FRAME`/`END` event, with **no timeout**. Logs
show `Cannot emit async signal omny_panda_continuous_data ... with status aborted` spamming
throughout the hang window, consistent with the box being left in a stuck/unarmed state after
the preceding rapid interlock-triggered abort/retry cycle (1199112001, ~11 attempts in 5
minutes).
This traces into `ophyd_devices`, not `csaxs_bec` — per the ownership note above, not mine to
fix directly here. Follow-ups to consider: (a) file/flag this with whoever owns
`ophyd_devices` (add a timeout + explicit failure path to `PandaBox.on_pre_scan`'s status
callback so a stuck arm surfaces as an error instead of hanging forever), (b) check whether
`on_unstage`/`_reset_panda()` is reliably called on scan abort so the box doesn't carry a
stale state into the next attempt.
**Status 2026-09-20 (csaxs_bec side done):** `PandaBoxOMNY.on_pre_scan`
(`csaxs_bec/devices/panda_box/panda_box_omny.py`) no longer waits for READY forever. Each
arming attempt waits `pre_scan_timeout` (default 5 s); if READY does not arrive, the box is
force-reset (`_reset_panda()`), the readout thread restarted and arming retried once (retry is
printed and logged as a warning with diagnostics). Only if the second attempt fails as well
does the status fail with a `RuntimeError`, so the scan aborts instead of hanging.
Still open for the ophyd_devices owner: the base-class race where the readout loop's
`finally: _reset_panda()` can clear `data_thread_run_event` after a fresh `on_stage()` set it
(likely root cause of the missing READY after rapid abort/retry); the new diagnostics line in
the DeviceServer log (`data_thread_run_event=False`) would confirm it.
@@ -82,12 +82,13 @@ implementation plan — nothing here has been started.
## 5. No hard-stop button wired into lamni's GUI
- flomni: `flomnigui_show_cameras()` (`flomni/gui_tools.py:143-150`) wires up
a `ConsoleButtonsWidget` (`hard_stop_device_name="ftransy"`,
`extra_hard_stop_device_name="foptx"`) calling the shared, generic
`GalilController.hard_abort_and_restore_positioning_mode()` — added
specifically to replace an older blind stop-all-devices broadcast that
could crash the scan worker thread.
- flomni: `flomnigui_show_cameras()` (`flomni/gui_tools.py`) wires up
a `ConsoleButtonsWidget` (`hard_stop_device_names=["ftransy", "foptx",
"fsamroy"]`, one device per Galil controller, transfer controller first)
that sends `stop_all_axes()` to each controller and then calls the shared,
generic `GalilController.hard_abort_and_restore_positioning_mode()` on the
first — added specifically to replace an older blind stop-all-devices
broadcast that could crash the scan worker thread.
- lamni: `LamNI/gui_tools.py` has no `ConsoleButtonsWidget`/hard-stop wiring
at all.
- The original commit message explicitly notes this was "scoped to flomni
@@ -273,15 +273,20 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
self._webpage_gen.start(force=True)
def _maybe_reset_params_on_account_change(self) -> None:
"""Offer a tomo-parameter reset when the active account has changed.
"""Offer a tomo-parameter/queue/progress reset when the active
account has changed.
Called at BEC session start. The account for which defaults were last
applied is stored in the global var ``defaults_applied_for_account`` so
it survives client restarts. Only a genuine change of account triggers
the (interactive) reset; the same account is a silent no-op, so
restarting a client within the same experiment never disturbs tuned
parameters. Mirrors Flomni._maybe_reset_params_on_account_change()
exactly.
parameters, the tomo queue, or in-progress acquisition state. Mirrors
Flomni._maybe_reset_params_on_account_change() exactly.
On a confirmed reset, this also clears the tomo queue and resets tomo
acquisition progress (via tomo_queue_clear()/tomo_progress_reset()),
since both are leftover state from the previous account/experiment.
"""
bec = builtins.__dict__.get("bec")
try:
@@ -298,11 +303,13 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
if self.OMNYTools.yesno(
f"New account '{account}' detected (previous: '{last_account}').\n"
"Reset tomo parameters to defaults for the new experiment?",
"Reset tomo parameters to defaults, clear the tomo queue, and "
"reset tomo acquisition progress for the new experiment?",
"y",
):
self._set_default_tomo_params()
print(f"Tomo parameters reset to defaults for account '{account}'.")
self.tomo_queue_clear()
self.tomo_progress_reset()
self.client.set_global_var("defaults_applied_for_account", account)
def _set_default_tomo_params(self) -> None:
@@ -408,6 +415,66 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
self._progress_proxy.reset()
print("Tomo progress reset.")
def _build_timing_lines(self, header: str = "Tomoscan finished.") -> list:
"""Build the timing-report lines (elapsed / idle / excl.-idle
breakdown) from the current ``self.progress`` state.
Shared by the end-of-scan report in ``tomo_scan()`` and the
on-demand ``tomo_print_timing_report()`` / ``_and_scilog()``
commands, so a scan interrupted mid-flight can still get this
breakdown -- ``self.progress`` is a BEC global var, not a local,
so it survives a KeyboardInterrupt raised inside ``tomo_scan()``.
"""
idle_s = self.progress.get("accumulated_idle_time", 0.0)
start_str = self.progress.get("tomo_start_time")
elapsed_s = None
if start_str is not None:
try:
elapsed_s = (
datetime.datetime.now() - datetime.datetime.fromisoformat(start_str)
).total_seconds()
except (ValueError, TypeError):
elapsed_s = None
timing_lines = [
header,
f"Measurement ID: {self.tomo_id}",
f"Sample: {self.sample_name}",
]
if self.at_each_angle_hook:
timing_lines.append(f"At-each-angle hook: {self._describe_active_hook()}")
if elapsed_s is not None:
timing_lines.append(f"Total measurement time: {self._format_duration(elapsed_s)}")
timing_lines.append(
f"Total measurement time excluding detected gaps: {self._format_duration(elapsed_s - idle_s)}"
)
timing_lines.append(
f"Total measurement time lost to detected gaps: {self._format_duration(idle_s)}"
)
return timing_lines
def tomo_print_timing_report(self) -> None:
"""Print the current tomo timing breakdown (elapsed / idle / excl.
idle) for whatever scan is or was last in progress.
Callable at any time, including right after interrupting a scan
with Ctrl-C, since the underlying data (self.progress) survives
the interrupt. Console only -- does not write to scilog.
"""
timing_lines = self._build_timing_lines(header="Tomo timing report (current progress).")
for line in timing_lines:
print(line)
def tomo_print_timing_report_and_scilog(self) -> None:
"""Same as tomo_print_timing_report(), but also writes the report
to scilog -- useful for documenting the timing of a scan that was
interrupted rather than left undocumented.
"""
timing_lines = self._build_timing_lines(header="Tomo timing report (current progress).")
for line in timing_lines:
print(line)
timing_content = "\n".join(timing_lines)
self.write_to_scilog(timing_content, ["tomoscan"])
@property
def alignment_scan_progress(self) -> _AlignmentScanProgressProxy:
"""Proxy dict backed by the BEC global variable ``alignment_scan_progress``.
@@ -1134,12 +1201,14 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
fovx = rec.get("lamni_piezo_range_x")
fovy = rec.get("lamni_piezo_range_y")
fov_str = f"{fovx} x {fovy} um" if fovx is not None and fovy is not None else "?"
step = rec.get("tomo_shellstep")
step_str = f"{step} um" if step is not None else "?"
exposure = rec.get("tomo_countingtime")
exp_str = f"{exposure} s" if exposure is not None else "?"
duration = rec.get("duration_s")
dur_str = self._format_duration(duration) if duration is not None else "?"
lines.append(
f"scan {scan_str}: FOV {fov_str}, exposure {exp_str}, duration {dur_str}"
f"scan {scan_str}: FOV {fov_str}, step {step_str}, exposure {exp_str}, duration {dur_str}"
)
content = "\n".join(lines)
@@ -1767,6 +1836,18 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
self.progress["estimated_finish_time"] = None
self.progress["accumulated_idle_time"] = 0.0
self.progress["heartbeat"] = None
# Unconditional single 0-degree reference projection at the very
# start of a new tomogram, independent of
# zero_deg_reference_at_each_subtomo /
# golden_projections_at_0_deg_for_damage_estimation -- gives an
# immediate radiation-damage baseline even when those flags are
# off. Real data (uses _tomo_scan_at_angle like any grid
# projection), included in reconstruction like any other
# projection. Logged under subtomo_number=1. Mirrors Flomni's
# equivalent addition -- no flat-field/empty-frame step here
# (unlike Flomni's collect_empty_frames()), since LamNI has no
# way to retract the sample out of the beam.
self._tomo_scan_at_angle(0, 1)
try:
with scans.dataset_id_on_hold:
@@ -1904,31 +1985,7 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
self._log_tomogram_timing()
self.OMNYTools.printgreenbold("Tomoscan finished")
idle_s = self.progress.get("accumulated_idle_time", 0.0)
start_str = self.progress.get("tomo_start_time")
elapsed_s = None
if start_str is not None:
try:
elapsed_s = (
datetime.datetime.now() - datetime.datetime.fromisoformat(start_str)
).total_seconds()
except (ValueError, TypeError):
elapsed_s = None
timing_lines = [
"Tomoscan finished.",
f"Measurement ID: {self.tomo_id}",
f"Sample: {self.sample_name}",
]
if self.at_each_angle_hook:
timing_lines.append(f"At-each-angle hook: {self._describe_active_hook()}")
if elapsed_s is not None:
timing_lines.append(f"Total measurement time: {self._format_duration(elapsed_s)}")
timing_lines.append(
f"Total measurement time excluding detected gaps: {self._format_duration(elapsed_s - idle_s)}"
)
timing_lines.append(
f"Total measurement time lost to detected gaps: {self._format_duration(idle_s)}"
)
timing_lines = self._build_timing_lines()
for line in timing_lines[3:]:
print(line)
timing_content = "\n".join(timing_lines)
@@ -1992,6 +2049,12 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
the real per-projection count can vary a bit with angle/stitch
tile, since the circular FOV crop (tomo_circfov) is checked against
the rotated stage position.
The real scan also draws a fresh random sub-step offset on every
call (see get_lamni_fermat_spiral_pos()'s `seed` argument), adding a
further small amount of per-projection count variation on top of
the angle/stitch dependence above. A fixed seed is used here only so
that this estimate itself is reproducible from one call to the next.
"""
from csaxs_bec.scans.lamni_fermat_scan import LamniFermatScan
@@ -2010,6 +2073,7 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
fovx=self.lamni_piezo_range_x,
fovy=self.lamni_piezo_range_y,
fov_circular=self.tomo_circfov,
seed=0,
)
return len(positions)
@@ -2024,7 +2088,7 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
def tomo_parameters(self):
"""Print and interactively update the tomo parameters."""
print("Current settings:")
print(f"Counting time <ctime> = {self.tomo_countingtime} s")
print(f"Counting time per frame <ctime> = {self.tomo_countingtime} s")
print(f"Stepsize microns <step> = {self.tomo_shellstep}")
print(
f"Piezo range (max 80) <microns> = {self.lamni_piezo_range_x},"
@@ -2089,17 +2153,26 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
print("OK. continue.")
return
self.tomo_countingtime = self._get_val("<ctime> s", self.tomo_countingtime, float)
self.tomo_shellstep = self._get_val("<step size> um", self.tomo_shellstep, float)
self.tomo_countingtime = self._get_val(
"<ctime per frame> s", self.tomo_countingtime, float, min_=0.001, max_=100.0
)
self.tomo_shellstep = self._get_val(
"<step size> um", self.tomo_shellstep, float, min_=0.025, max_=20.0
)
self.lamni_piezo_range_x = self._get_val(
"<piezo range X (max 80)> um", self.lamni_piezo_range_x, float
"<piezo range X (max 80)> um", self.lamni_piezo_range_x, float, min_=0.1, max_=79.9
)
self.lamni_piezo_range_y = self._get_val(
"<piezo range Y (max 80)> um", self.lamni_piezo_range_y, float
"<piezo range Y (max 80)> um", self.lamni_piezo_range_y, float, min_=0.1, max_=79.9
)
self.lamni_stitch_x = self._get_val("<stitch X>", self.lamni_stitch_x, int)
self.lamni_stitch_y = self._get_val("<stitch Y>", self.lamni_stitch_y, int)
self.tomo_circfov = self._get_val("<circular FOV> um", self.tomo_circfov, float)
self.tomo_stitch_overlap = self._get_val(
"<stitch overlap> um", self.tomo_stitch_overlap, float, min_=0.0, max_=10.0
)
self.tomo_circfov = self._get_val(
"<circular FOV> um", self.tomo_circfov, float, min_=0.0, max_=500.0
)
self.ptycho_reconstruct_foldername = self._get_val(
"Reconstruction queue", self.ptycho_reconstruct_foldername, str
)
@@ -2179,8 +2252,22 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools
)
@staticmethod
def _get_val(msg: str, default_value, data_type):
return data_type(input(f"{msg} ({default_value}): ") or default_value)
def _get_val(msg: str, default_value, data_type, min_=None, max_=None):
"""Prompt for a value, casting to data_type. When min_/max_ are
given, re-prompts (rather than silently clamping) until the typed
value is in range, mirroring the Tomo Parameters GUI's spinbox
bounds (tomo_params.py) so tomo_parameters() enforces the same
limits instead of accepting anything of the right type. Mirrors
Flomni._get_val() exactly."""
while True:
val = data_type(input(f"{msg} ({default_value}): ") or default_value)
if min_ is not None and val < min_:
print(f"Value must be >= {min_}. Please try again.")
continue
if max_ is not None and val > max_:
print(f"Value must be <= {max_}. Please try again.")
continue
return val
def _confirm_sequence_override(self, warning: str, force: bool) -> bool:
"""Print *warning* and ask whether to proceed anyway.
@@ -0,0 +1,114 @@
# TODO: interferometer signal-strength trend on the webpage
Status: **proposed, not implemented.** Discussed 2026-09-19. This is a design
note to capture the idea before it's lost -- no code changes yet.
## Goal
A new section on the flomni/lamni/omny status webpage showing the trend of
interferometer signal strength over time (a sparkline/chart, not a live
readout), so drift/degradation is visible at a glance across a session or
beamtime, not just from an interactive `show_signal_strength_interferometer()`
call in the moment.
## Sampling design
- **Very low rate on purpose**: roughly one data point per hour, or less --
explicitly *not* per-projection. A tomo scan already calls a signal-strength
check regularly (flomni: every projection, see below), but sampling that
often would produce far too many points for a trend view (tens of points
per hour per axis) for no benefit -- this is about long-term drift, not
per-projection diagnostics.
- **Timestamped, so per-axis cadence can differ freely.** Each data point is
`(timestamp, value)`; there's no requirement that every axis be sampled at
the same time or the same rate. An axis that's cheap to read opportunistically
(e.g. already fetched as a side effect of something else) can be sampled
more often than one that costs a dedicated round-trip, and the trend for
each axis is still valid independently. Staleness/eventual-consistency
between axes, or between this and the live scan, is fine -- the goal is a
general trend, not point-in-time precision.
- **Retention**: apply the lesson from `eps_alarm_history.json`'s HTTP 413
bug (see `docs/developer/...` -- fixed 2026-09-18 in `eps_status_generator.py`,
`_MAX_UPLOAD_BYTES`/`_shrink_to_fit`): cap this ring buffer by *both* count
and a hard byte budget from the start, reusing that same pattern, not just
a count cap discovered insufficient after the fact.
## Per-setup reality check (2026-09-19 survey -- see below for what's actually there)
The three setups' interferometer-read primitives are **not interchangeable**
-- this can't be one generic helper at the RT-controller layer:
| Setup | Read primitive | Axes/channels | Threshold/check helper |
|---|---|---|---|
| flomni | `RtFlomniController.read_ssi_interferometer(axis_number)` (`rt_flomni_ophyd.py:462`), generic `j{n}` socket read | 0-3 | `laser_tracker_check_signalstrength()` (`rt_flomni_ophyd.py:466`) -- reads axis 1, warns only, does not act |
| LamNI | **no equivalent single-axis reader.** `RtLamniController.show_signal_strength_interferometer()` (`rt_lamni_ophyd.py:268`) reads 3 channels (ST FZP horizontal/vertical, angle) via distinct `J2`/`J3`/`J7` commands | 3 (different meaning than flomni's 0-3) | none -- no `min_signal`/`low_signal` threshold logic exists in this file at all |
| omny | `RtOMNYController.laser_tracker_check_and_wait_for_signalstrength()` (`rt_omny_ophyd.py:270`) -- different behavior class: polls and *auto-triggers* `omny_interferometer_align_tracking()` if below `low_signal`, not just a warning. `show_signal_strength_interferometer()` (`rt_omny_ophyd.py:736`) reads 5 channels via a `self.ssi` dict from `feedback_get_status_and_ssi()` | 5 (OSA FZP Y/X, ST OSA Y/X, Angle) | yes, but actively corrective, not passive |
There is no shared `csaxs_bec`-level RT-controller base across the three
(`RtFlomniController`/`RtLamniController`/`RtOMNYController` each subclass the
generic external `ophyd_devices.utils.controller.Controller` directly). LamNI
also doesn't reuse the one shared local module that does exist
(`csaxs_bec/devices/omny/rt/rt_ophyd.py`'s `RtSignalRO`/etc., used by flomni
and omny) -- it redefines its own parallel Signal classes.
**Conclusion**: the shared piece of this feature should be the trend
storage/rendering (a common global-var shape + webpage section), not the
signal read itself. Each setup needs its own small adapter that knows its own
axis semantics and feeds the shared throttled-stash helper.
## Where the throttled read would hook in, per setup
- **flomni**: `laser_tracker_check_signalstrength()` is already called once
per projection during `tomo_scan()` (`flomni.py:3624`) -- cheapest option is
to throttle *at that call site* (check "have I stashed a data point in the
last hour?" before writing one), reusing an already-happening RT round-trip
rather than adding a new one.
- **LamNI**: confirmed (2026-09-19 survey) there is **no** scan-time
signal-strength call anywhere in the LamNI plugin -- `show_signal_strength`
(`lamni.py:724`) is a manual/interactive-only helper, never called from
`tomo_scan_projection` or `tomo_scan`. There's no existing per-projection
hook to throttle against, so this would need a separate low-rate timer
instead -- the natural fit is `WebpageGeneratorBase`'s own `_cycle()` loop
(`OMNY_shared/webpage_generator_base.py`, default 15s cadence, already runs
continuously including when idle), which already has a documented
subclass-override hook for instrument-specific data
(`_collect_setup_data()`, used today by `FlomniWebpageGenerator`/
`LamniWebpageGenerator` for temperatures/sample name/etc.) -- add the
throttle check there instead of in the scan loop.
- **omny**: blocked on a prerequisite -- **omny has no working webpage
generator at all right now.** `omny_webpage_generator.py` looks like one
(defines `WebpageGeneratorBase`, `FlomniWebpageGenerator`, a
`make_webpage_generator()`, etc.) but is confirmed dead/orphaned: nothing
in `omny.py`/`omny/__init__.py` imports it, and the real factory
(`OMNY_shared/webpage_generator_base.py` ~line 1409) explicitly falls
through to the plain base class for the omny session with a comment noting
this file is "an unwired, stale duplicate." Building a real
`OmnyWebpageGenerator` subclass is a prerequisite for this feature on omny,
not just an extra per-setup adapter -- do not revive the stale duplicate
as-is.
## Suggested shape (for whenever this is implemented)
- A global var, e.g. `interferometer_signal_trend`, holding a small
ring-buffer list of `{timestamp, axis, value}` (or per-axis sub-lists),
written via a proxy class mirroring `_ProgressProxy`/`_TomoQueueProxy`
(`OMNY_shared/tomo_queue_mixin.py`) -- the established pattern in this
codebase for cross-process/cross-thread state via
`client.get_global_var`/`set_global_var`.
- A shared rendering piece (webpage HTML/JS section, sparkline or simple line
chart) that reads this same global var shape regardless of which setup
wrote it, so the display code doesn't need to know about per-setup axis
differences -- only each setup's adapter does.
- Byte-budget-capped storage per the eps history lesson above.
## Open questions (not yet decided)
- Exact axis(es) worth trending per setup -- flomni's axis 1 is free (already
read every projection); trending flomni's other 3 axes, or any of LamNI's/
omny's channels, means new dedicated round-trips on whatever cadence is
chosen for those.
- Whether sampling should continue during idle time between scans (needs a
timer-based approach for all three setups, not just LamNI/omny) or only
while a scan is actively running (cheaper, but leaves gaps covering the
most interesting idle-drift periods).
- Exact webpage layout/placement of the new section -- not designed yet.
@@ -96,6 +96,18 @@ _ALARM_TEXTS_MAX = 100 # distinct alarm texts kept on a single open eps_e
# only one event is ever open at a time, so this bounds
# total history growth during a long-running flapping alarm
_MAX_UPLOAD_BYTES = 250_000 # hard ceiling on eps_alarm_history.json's serialized size,
# enforced in HistoryTracker.save(). _HISTORY_MAX/
# _ALARM_TEXTS_MAX/_TRACE_MAX bound the *shape* of the data
# but not each alarm text's length, and _load() never re-caps
# a stored event's texts list -- only new appends are capped
# -- so a sufficiently verbose/flapping EPS alarm can still
# push the file past the upload server's request-size limit
# (a bare HTTP 413, no size check in upload.php itself; almost
# certainly nginx's client_max_body_size, default 1m). This
# keeps comfortable headroom under that regardless of how the
# data grows.
_SEVERITY_NAMES = {0: "NO_ALARM", 1: "MINOR", 2: "MAJOR", 3: "INVALID"}
_BEAMLINE_SUBNET_PREFIX = "129.129.122." # X12SA subnet, where EPS/machine PVs are reachable
@@ -165,6 +177,56 @@ def _clean(value):
return value
def _payload_size(payload: dict) -> int:
return len(json.dumps(payload, default=str))
def _shrink_to_fit(payload: dict, max_bytes: int = _MAX_UPLOAD_BYTES) -> dict:
"""Return a shrunk copy of a HistoryTracker.save() payload that
serializes to at most max_bytes.
Trims progressively less-essential data -- alarm texts per event, then
the ring-current trace, then the oldest events, then (last resort, for a
single pathologically long string) individual field lengths -- stopping
as soon as the payload fits.
"""
payload = {
"eps_events": [dict(ev) for ev in payload["eps_events"]],
"machine_events": list(payload["machine_events"]),
"ring_current_trace": list(payload["ring_current_trace"]),
}
if _payload_size(payload) <= max_bytes:
return payload
for texts_cap in (20, 5, 1):
for ev in payload["eps_events"]:
texts = ev.get("texts")
if isinstance(texts, list) and len(texts) > texts_cap:
ev["texts"] = texts[-texts_cap:]
if _payload_size(payload) <= max_bytes:
return payload
while len(payload["ring_current_trace"]) > 12 and _payload_size(payload) > max_bytes:
payload["ring_current_trace"] = payload["ring_current_trace"][::2]
while len(payload["eps_events"]) > 1 and _payload_size(payload) > max_bytes:
payload["eps_events"].pop(0)
while len(payload["machine_events"]) > 1 and _payload_size(payload) > max_bytes:
payload["machine_events"].pop(0)
if _payload_size(payload) > max_bytes:
for ev in payload["eps_events"]:
if isinstance(ev.get("texts"), list):
ev["texts"] = [t[:200] for t in ev["texts"]]
for mev in payload["machine_events"]:
for key in ("from", "to"):
if isinstance(mev.get(key), str):
mev[key] = mev[key][:200]
return payload
# ---------------------------------------------------------------------------
# History tracking
# ---------------------------------------------------------------------------
@@ -210,9 +272,12 @@ class HistoryTracker:
"machine_events": self.machine_events[-_HISTORY_MAX:],
"ring_current_trace": self.ring_trace[-_TRACE_MAX:],
}
# Shrink a copy for writing/uploading; the in-memory lists above keep
# their full (count-capped) history regardless of upload-size limits.
payload = _shrink_to_fit(payload)
try:
tmp = self._path.with_suffix(".tmp")
tmp.write_text(json.dumps(payload, indent=1, default=str))
tmp.write_text(json.dumps(payload, default=str))
tmp.replace(self._path)
except Exception as exc:
logger.warning(f"HistoryTracker: cannot write {self._path}: {exc}")
@@ -9,12 +9,24 @@ def filters_out_of_beam(dev, tol: float = 0.1) -> tuple[bool, list[str]]:
Reuses cSAXSFilterTransmission's own axis list/position table (index 0 of
each row is the "out" position) so there is one source of truth for what
"out" means, matching csaxs.fil_trans()/_fil_trans_report()'s own check.
Tolerates a deployment missing one or more filter axes (e.g. a simulated
session without the real beamline's filter_array_* devices), and `dev`
itself being None (the usual state of the module-level `dev =
builtins.__dict__.get("dev")` idiom used throughout this codebase before
a real BEC session has set it). BEC's device container raises
DeviceConfigError -- not AttributeError -- for a missing device, so a
plain `getattr(dev, axis_name, None)` does NOT safely skip it; wrapping
the lookup itself in try/except does, for both cases.
"""
offending = []
for axis_name, positions in zip(
cSAXSFilterTransmission._AXES, cSAXSFilterTransmission._POSITIONS_USER
):
axis_obj = getattr(dev, axis_name, None)
try:
axis_obj = getattr(dev, axis_name, None)
except Exception:
axis_obj = None
out_position = positions[0]
if axis_obj is None or out_position is None:
continue
@@ -327,6 +327,8 @@ class TomoIDManager:
"""Registers a tomography measurement in the OMNY sample database
and returns its assigned tomo ID.
Real e-accounts (personal, e.g. "e12345") and p-accounts (proposal
accounts, e.g. "p23092") are both registered against OMNY_URL.
Non-production accounts (e.g. test accounts like "gac-x01dc") are not
registered against OMNY_URL at all -- there is no longer a separate
test server to redirect them to, and registering test runs against
@@ -351,8 +353,8 @@ class TomoIDManager:
@staticmethod
def _is_valid_eaccount(eaccount: str) -> bool:
"""True for real e-accounts (e.g. "e12345"), False for test accounts."""
return bool(re.fullmatch(r"e\d{5}", eaccount.strip()))
"""True for real e- or p-accounts (e.g. "e12345", "p23092"), False for test accounts."""
return bool(re.fullmatch(r"[ep]\d{5}", eaccount.strip()))
def register(
self,
@@ -366,7 +368,7 @@ class TomoIDManager:
) -> int:
"""Register a new measurement and return the assigned tomo ID.
Registers against OMNY_URL for a real e-account. For any other
Registers against OMNY_URL for a real e- or p-account. For any other
account (e.g. a test/"gac-*" account), skips registration entirely
and returns FALLBACK_TOMO_ID (0) -- there is no separate test
server to register against, and registering test runs against
@@ -376,7 +378,7 @@ class TomoIDManager:
"""
if not self._is_valid_eaccount(eaccount):
logger.warning(
f"Account '{eaccount}' is not a valid e-account; skipping OMNY "
f"Account '{eaccount}' is not a valid e- or p-account; skipping OMNY "
f"registration (no separate test server to register against) "
f"and falling back to tomo ID {self.FALLBACK_TOMO_ID}."
)
@@ -757,7 +757,7 @@ class cSAXSFilterTransmission:
def _fil_trans_report(self, tol: float = 0.1, energy_kev: Optional[float] = None) -> None:
"""
Report the currently active exposurebox filter combination.
Determines stage positions via dev.<axis>.readback.get()
Determines stage positions via dev.<axis>.readback.get(cached=True)
with a tolerance window of ±tol relative to nominal positions.
Parameters
@@ -799,7 +799,9 @@ class cSAXSFilterTransmission:
return
try:
rb = float(axis_obj.readback.get())
# Status report only: the last published readback (cached=True, from redis)
# avoids one RPC round trip per axis. A missing value raises in float() below.
rb = float(axis_obj.readback.get(cached=True))
except Exception:
print(f"ERROR: readback unavailable for axis {axis_name}")
return
@@ -47,7 +47,10 @@ class cSAXSSlits:
return None
try:
reading = device.read()
# cached=True reads the last published readback from redis. The default
# (cached=False) is one RPC round trip to the device server per device,
# i.e. 24 of them for slits_show_all().
reading = device.read(cached=True)
value = reading[device_name]["value"]
return float(value)
except Exception:
@@ -0,0 +1,353 @@
# Validating PandaBox (`omny_panda_continuous`) recovered positions against RT
**Purpose of this document:** how to check whether positions recovered from
the PandaBox's Mode B (free-running raw stream, `raw_stream_mode: true`)
capture actually match the true stage positions, using the RT controller's
own trusted per-point averages as ground truth. Read this before continuing
that testing in a new session — it captures the full methodology, the exact
conversion formula, and a bug that was found and fixed this way, so none of
it needs re-deriving from scratch.
Companion docs:
- `docs/developer/panda_box_free_running_setup.md` — the PandA layout/mode
design doc. Has the mode comparison, signal alias mapping, and (as of
2026-09-21) the "Known data-quality caveat: stale `frame_counter`
carryover" section this investigation produced.
- `docs/developer/panda_box_free_running_data_schema.md` — the HDF5 field
list for the Mode B raw stream.
- `csaxs_bec/devices/panda_box/panda_box_omny.py` — the device. See
`_drop_stale_frame_counter_rows`/`_reset_raw_stream_state` for the fix
described below.
---
## 1. The problem this validates
Mode B captures raw samples continuously at `CLOCK1`'s rate for the whole
burst window — **not** one sample per scan point. Only a subset of rows
coincide with an actual detector exposure (`gate_detector_active == 1`), and
even those need to be grouped by which point they belong to
(`frame_counter`). Treating every raw row as its own recovered position
(without grouping/filtering) produces a scatter that roughly traces the
scan's envelope but does not correspond point-for-point to the true
positions — and critically, **no scale/offset correction can fix that**,
because it's a many-to-one correspondence problem, not a coordinate error.
This was confirmed visually (see the git history around 2026-09-20/21 for the
red/blue scatter plot that started this investigation) before the grouping
method below was worked out.
## 2. Ground truth: `rt_positions`
The RT controller logs its own per-point values, already correctly averaged
and known-good (used successfully in reconstruction):
- `rt_positions_data_target_x`/`_y` — commanded (programmed) position.
- `rt_positions_data_average_x_st_fzp`/`_y` — **the trusted measured
position**, in microns. Compare recovered PandA positions against these,
not against `target_x`/`_y`.
- `rt_positions_data_stdev_x_st_fzp`/`_y` — RT's own per-point stdev, useful
as a sanity scale for how much residual noise to expect (order 0.005 µm in
the validated scan).
One row per scan point (472 points in the validated scan, `S06313`).
## 3. Recovering positions from the Mode B raw stream
### 3.1 Raw fields (all same length, one entry per raw sample)
Under `entry/collection/devices/omny_panda_continuous/` in the master HDF5:
`omny_panda_continuous_data_<field>/value`, where `<field>` is one of
`interf_st_fzp_x`, `interf_st_fzp_y`, `cap_voltage_fzp_x`, `cap_voltage_fzp_y`,
`frame_counter`, `gate_detector_active` (plus `interf_st_rotx`/`_rotz`,
`pcap_gate_duration_value`, not needed for x/y position recovery).
### 3.2 Required grouping/filtering — do this before anything else
1. Filter to rows where `gate_detector_active == 1` (real exposure, not
inter-point flight time).
2. Group the remaining rows by **point** index. For a single-frame-per-point
scan (`frames_per_trigger == 1`), `frame_counter` directly ranges
`0..num_points-1` and *is* the point index. For a burst scan
(`frames_per_trigger > 1`), `frame_counter` increments once per
sub-frame trigger, not once per point (confirmed 2026-09-22 against real
`frames_per_trigger=3` scans, `S00395`/`S00396`/`S00398` — see §7): the
point index is `frame_counter // frames_per_trigger`, and *all* of a
point's sub-frames' gate-active rows get averaged together into that
point's one `rt_positions` value, not kept separate.
3. Average `interf_st_fzp_x`/`_y` and `cap_voltage_fzp_x`/`_y` within each
group.
Skipping this step is the #1 way to get a result that looks roughly
right-shaped but doesn't actually match — see §1.
### 3.3 Stale `frame_counter` carryover — check this if a session predates the fix
PandA's `COUNTER1.OUT` is not cleared by `Arm()`, so the first raw samples of
an acquisition can briefly carry a stale `frame_counter` value left over from
the *previous* acquisition's tail. If that stale value collides with a real
point index later in the same scan, it contaminates that point's group.
`panda_box_omny.py` fixes this client-side as of commit `3eb6e03`
(2026-09-21) — it drops rows before the first observed `frame_counter == 0`
on every arm. **Any HDF5 file captured before that fix was live on the real
device** (i.e. anything from before 2026-09-21, `S06313` included) still has
the stale rows in it and needs the same drop applied manually before
grouping:
```python
reset_idx = np.argmax(frame_counter_raw == 0.0) # first occurrence
frame_counter_raw = frame_counter_raw[reset_idx:]
# ...apply the same [reset_idx:] slice to every other raw field, in lockstep
```
For anything captured after the fix, this is unnecessary (already handled
device-side) — but it's cheap and harmless to keep doing as a defense-in-depth
check; if it ever finds `reset_idx > 0` again on a fixed device, that's a
regression worth investigating.
### 3.4 Conversion formula (from Mirko, confirmed 2026-09-21)
```python
factor_x = -0.000309078
factor_y = 0.000154539
interfx = grouped_interf_st_fzp_x * factor_x # microns
interfy = grouped_interf_st_fzp_y * factor_y # microns
capmuy = -(grouped_cap_voltage_fzp_y) * 5.0 / 20.0
capmux = (grouped_cap_voltage_fzp_x) * 5.0 / 20.0
CapX_FZP_microns = 0.9539 * capmux + 0.0009008 * capmuy
CapY_FZP_microns = -0.023140 * capmux + 1.0120 * capmuy
pos_x_um = interfx - CapX_FZP_microns # microns -- compare directly against average_x_st_fzp
pos_y_um = interfy - CapY_FZP_microns # microns -- compare directly against average_y_st_fzp
```
**Units trap:** the original formula this was transcribed from ends with
`pos_x = (interfx - CapX_FZP_microns) * 1e-6` to get meters, for whatever
consumed it downstream. `rt_positions_data_average_x_st_fzp`/`_y` are in
**microns**, so compare against `pos_x_um`/`pos_y_um` *before* that final
`* 1e-6` — don't convert one side to meters and leave the other in microns.
`factor_x`/`factor_y` were independently re-derived by least-squares fit
against `average_x_st_fzp`/`_y` from `S06313` and matched Mirko's values to 4
significant figures (fit: `-3.089108e-04`/`1.545243e-04` vs. given
`-0.000309078`/`0.000154539`) — good cross-check if a future factor is ever
in doubt.
## 4. Validating a match
```python
def r2(pred, true):
ss_res = np.sum((true - pred) ** 2)
ss_tot = np.sum((true - true.mean()) ** 2)
return 1 - ss_res / ss_tot
```
Expected result on a correctly grouped, correctly converted scan (from
`S06313`, post-fix):
- A single near-constant offset per axis between `pos_x_um`/`pos_y_um` and
`average_x_st_fzp`/`_y` (validated: X ≈ 1.26 µm, Y ≈ +0.99 µm) — this is
expected (RT's internal reference zero vs. the PandA/`INENC` counter's own
free-running zero don't coincide) and irrelevant to reconstruction, since
it's the same constant for every point in one scan. Subtract the mean
residual per axis before judging fit quality.
- After removing that offset: R² > 0.9999 on both axes, residual std ~0.25
0.3 nm, max residual under ~1 nm across all points (post-fix; pre-fix, one
point was off by ~116 nm — see §3.3).
If R² is poor and doesn't clean up after removing a constant offset (i.e. a
best-fit affine still doesn't work — this is the actual symptom that flagged
the original bug, see §1), suspect the grouping/filtering step (§3.2) before
suspecting the conversion factors.
## 5. Practical notes
- Use a BEC production venv `python3` for any h5py-based analysis — the
system `python3` doesn't have `h5py` installed. Path is host-dependent:
`/opt/bec_deployments/production/bec_venv/bin/python3` on some hosts,
`/sls/x12sa/config/bec/production/bec_venv/bin/python3` on others (e.g.
`x12sa-bec-001`) — check both if one is missing.
- `S06313`'s files were at `/tmp/S06313/` (`S06313_master.h5` has both
`omny_panda_continuous` and `rt_positions`; `S06313_eiger_*` are detector
data, not needed for this). `/tmp` is not persistent — get a fresh scan's
files from whoever's driving the beamtime if this path is gone.
- Relevant H5 group prefix:
`entry/collection/devices/<device>/<device>_data_<field>/value` (and
`.../timestamp`, much shorter — one entry per buffer flush, not per row,
see `_flush_raw_stream_buffer_locked` in `panda_box_omny.py`).
## 6. Next steps (as of 2026-09-21)
1. **Burst acquisition** — RESOLVED 2026-09-22, see §7/§3.2: `frame_counter`
increments once per sub-frame trigger, not once per point;
`frame_counter // frames_per_trigger` is the point index, and all
sub-frames of a point get averaged together to match RT's one value per
point. Confirmed against real `frames_per_trigger=3` scans
(`S00395`/`S00396`/`S00398`), R² = 1.0000000 on both axes.
2. **Reduced data mode** — characterized empirically 2026-09-22, see §7
(`S00399`): captures only rows with `gate_detector_active == 1` (or very
close to it — 99.5% vs. ~39% of rows in normal continuous mode), cutting
total row count roughly in half to a third while keeping the same
per-point exposure sample count and full positional accuracy (R² =
1.0000000). The underlying PandA-side layout/config change that produces
this is not visible from BEC's `configuration` snapshot in the HDF5 file
(empty for both modes) — this section describes the observed effect on
the data, not the PandA-side mechanism; get that from whoever configured
it if it needs modifying.
## 7. Post-fix confirmation against live scans (2026-09-22)
Checked two scans taken after `3eb6e03` landed (both at `/tmp/S<id>/` on
`x12sa-bec-001`, files from 2026-09-22 ~12:4512:58):
- **`S00386`** (174 points, ~348k raw rows): `frame_counter` starts at `0` on
the very first raw row and has zero backward steps across the whole raw
stream — no leading stale-value prefix at all, unlike pre-fix `S06313`.
Full grouping+conversion validation against `rt_positions`: R² =
0.99999997 (X) / 0.99999998 (Y) after removing the expected constant
per-axis offset (1.23 µm X, +1.08 µm Y, consistent with §4's `S06313`
values); residual std 0.30 nm (X) / 0.25 nm (Y); worst single-point
residual 1.02 nm (X) / 0.51 nm (Y) — no outlier anywhere near the ~116 nm
the pre-fix bug produced. Clean confirmation of the fix.
- **`S00388`** (785 points, ~1.4M raw rows): one backward step, at raw row
36 (`frame_counter` reads `28` for rows 036, then drops to `0` from row
37 on) — structurally the same leading-stale-prefix signature the fix is
meant to strip, still present in data captured after the fix. However,
every one of those 37 stale rows has `gate_detector_active == 0`, so
they'd never enter a point's average regardless (§3.2 already filters to
`gate_detector_active == 1` before grouping) — this scan happened not to
be exposed to the bug's actual failure mode. Full validation still comes
out clean: R² = 0.99999999 (X) / 0.99999998 (Y), residual std 0.27 nm (X)
/ 0.26 nm (Y), max residual 0.85 nm (X) / 0.56 nm (Y).
**Root-cause investigation (updated 2026-09-22, still not fully resolved):**
Ruled out first: the leading `frame_counter` column can't be missing from
`S00388`'s first post-arm chunk the way an earlier version of this note
speculated. `pandablocks.connections.DataConnection`'s captured field set is
fixed once per TCP data connection (established once per `on_stage`, before
any `FrameData` is produced), so every `FrameData` chunk within one
acquisition carries the same columns — the "layout doesn't capture
`frame_counter`" fallback in `_drop_stale_frame_counter_rows`
(`counter_entry is None`) isn't reachable here.
What actually happened, confirmed from `ScanServer.log`/`DeviceServer.log`
for 2026-09-22: the scan immediately preceding `S00388` — scan **#387**
(786 points, `eb7cfd11-3610-4511-94a0-75f49b403e62`) — was opened at
`12:50:10` and force-**aborted** by the user only ~7 seconds later
(`12:50:16``17`, `action: 'abort'` in the scan queue). At `exp_time=0.1`s/point
that's on the right order to land around point ~2030, consistent with the
leftover `frame_counter == 28` that then showed up at the start of `#388`'s
raw stream (`#388` itself opened cleanly at `12:50:39`, no arm retries
logged — matches the "one `on_stage` per scan number" mapping confirmed
against the log). This is exactly the hardware mechanism `3eb6e03`'s commit
message describes (`COUNTER1.OUT` not cleared by `Arm()`) — just triggered
by an **abort mid-scan** rather than a normal scan-to-scan transition, which
is what `S06313` (the original bug report) and `S00386` (clean, §7 above)
both were.
Why the client-side filter didn't strip it is not fully pinned down.
`PandaBoxOMNY.stop()` (inherited, not overridden) does not call
`_reset_raw_stream_state()` — only `on_stage()` (once per scan) and
`_try_arm()` (right before every `Arm()`) do, and both ran normally for
`#388` per the logs. So by the code's own logic the state should have been
freshly reset well before `#388`'s `Arm()` went out. One structural
suspect, not yet confirmed as the actual cause: `_raw_stream_buffer`/
`_raw_stream_buffer_row_count`/`_raw_stream_last_flush` are all mutated only
under `_raw_stream_buffer_lock`, but `_raw_stream_seen_reset` is set outside
that lock in `_reset_raw_stream_state()` (after the `with` block) and read
outside the lock at the top of `_drop_stale_frame_counter_rows()` — the one
piece of this acquisition's reset state that isn't synchronized the same
way as everything else it's reset alongside. Whether that gap is actually
reachable in a plain abort → restage sequence (as opposed to needing an
`on_pre_scan` retry, which `#388` didn't have) isn't shown by the logs
available here; would need instrumentation (e.g. temporarily logging
`_raw_stream_seen_reset` transitions) against a reproduced abort-then-rescan
to confirm.
Practically: harmless in both scans checked so far, since the stale rows'
`gate_detector_active` was `0` throughout — they never entered a point's
average regardless of the row-level filter. But an abort mid-scan is a
routine, user-triggered event (unlike an `on_pre_scan` retry), so this path
is worth closing rather than relying on the gate happening to be low during
the stale window. If revisited: consider (a) locking `_raw_stream_seen_reset`
reads/writes the same as the buffer fields, and (b) also calling
`_reset_raw_stream_state()` from `on_stop`/`stop()`, not only `on_stage`/
`_try_arm`, so an aborted acquisition can't leave stale state for the next
one to inherit. Re-check against a scan where the stale run's rows show
`gate_detector_active == 1` before considering this fully closed.
**Mitigation added 2026-09-22:** `omny_panda_continuous` was being armed and
disarmed for *every* scan (any `grid_scan`/`line_scan`/alignment move, not
just `flomni_fermat_scan`), since generic BEC staging has no per-scan device
opt-in and `deviceTags` aren't consulted by it — see "Conditional arming
(`arm_scan_allowlist`)" in `docs/developer/panda_box_free_running_setup.md`.
It's now only actually `Arm()`ed for `flomni_fermat_scan` via the new
`arm_scan_allowlist` device config, which directly cuts down how often this
class of bug can even be triggered (fewer arm cycles → fewer abort→restage
windows like `S00388`'s), independent of whatever the remaining root cause
above turns out to be.
## 8. Batch validation across `S00391`-`S00399` (2026-09-22)
Nine `flomni_fermat_scan` runs at `/tmp/S00<391..399>/` on `x12sa-bec-001`,
run as three groups: `S00391`-`394` (single-frame, `frames_per_trigger=1`,
`393` aborted mid-scan), `S00395`-`398` (burst, `frames_per_trigger=3`,
`397` aborted mid-scan), `S00399` (burst, reduced data mode). Note: the
running `bec-device-server` process had been up since before commit
`5590633` (`arm_scan_allowlist`) landed, so it was still running the
pre-`arm_scan_allowlist` code for this whole batch — doesn't matter here
since every scan in this batch is `flomni_fermat_scan`, which
`arm_scan_allowlist` always arms anyway, but worth knowing: this batch
doesn't demonstrate that fix, only the pre-existing stale-`frame_counter`
fix (`3eb6e03`, which does predate the process start and was active).
**Full position-recovery validation** (§3.2-3.4/§4 methodology, extended for
burst grouping per §6) against `rt_positions`, all nine scans:
| Scan | mode | points | R² x / y | residual std (nm) x / y | max abs (nm) x / y |
|---|---|---|---|---|---|
| S00391 | single | 99 | 0.9999999 / 0.9999999 | 0.94 / 0.71 | 2.64 / 1.83 |
| S00392 | single | 98 | 0.9999999 / 0.9999999 | 0.95 / 0.76 | 2.75 / 2.09 |
| S00394 | single (post-abort) | 97 | 0.9999999 / 0.9999999 | 0.93 / 0.73 | 2.28 / 2.04 |
| S00395 | burst×3 | 98 | 1.0000000 / 1.0000000 | 0.47 / 0.55 | 0.98 / 1.18 |
| S00396 | burst×3 | 101 | 1.0000000 / 1.0000000 | 0.41 / 0.55 | 1.12 / 1.17 |
| S00398 | burst×3 (post-abort) | 99 | 1.0000000 / 1.0000000 | 0.45 / 0.54 | 1.32 / 1.13 |
| S00399 | burst×3, reduced data | 100 | 1.0000000 / 1.0000000 | 0.48 / 0.50 | 1.69 / 1.14 |
No point had zero grouped samples in any scan; no residual outlier anywhere
close to the ~116 nm the pre-fix bug produced. Burst-mode R² came out even
tighter than single-frame mode (1.0000000 vs. 0.9999999), consistent with
averaging 3x more raw samples per point.
**Stale-`frame_counter` prefix — two more clean, precisely-explained
repros**, both immediately following an abort, both harmless (all stale
rows have `gate_detector_active == 0`):
- `S00393` aborted with `rt_positions` recording exactly 29 points (panda
`frame_counter` max = `29.0`, matching exactly). The very next scan,
`S00394`, starts with 37 rows reading a stale `29.0` before dropping to
`0` at row 37 — same signature as `S00388`, same stale value as `#393`'s
own last completed point.
- `S00397` (burst×3) aborted with `rt_positions` recording exactly 17
points — 17×3 = 51 sub-frame counts, and panda `frame_counter` max there
is indeed `51.0`. The next scan, `S00398`, starts with 37 rows reading a
stale `51.0` before dropping to `0` at row 37 — same signature again, and
the stale value again matches `#397`'s own last completed sub-frame count
exactly.
This nails down the trigger mechanism about as precisely as it can be nailed
down from data alone: the leftover `frame_counter` value inherited by the
next scan is *exactly* the aborted scan's last completed count (point count
for single-frame, point×frames_per_trigger for burst) — not some unrelated
stale value from further back. The remaining open question from §7 (why the
client-side filter doesn't strip these rows in the abort→restage case, given
`on_stage`/`_try_arm` both reset state normally beforehand) is unchanged by
this batch — still worth instrumenting if pursued further, though it
remains harmless in every case observed so far because the stale window
never overlaps a gated exposure.
**Reduced data mode** (`S00399`) validated cleanly at full accuracy — see
§6 for the row-count/gate-fraction characterization. No contamination, no
missing points, same R² bar as the other burst scans.
@@ -198,6 +198,13 @@ cannot drift.
- **Consider**: the PSI **EPICS archiver REST interface** to sidestep CA firewall
concerns.
### Interferometer signal-strength trend (proposed, not implemented)
- See `OMNY_shared/AI_docs/TODO_interferometer_signal_trend.md` for the full
design note (2026-09-19) — a new webpage section trending interferometer
signal strength over time, sampled at ~1 point/hour or less (not
per-projection). Blocked on omny specifically needing a real
`OmnyWebpageGenerator` built first (its current file is dead/unwired code).
### Deferred
- **LamNI**: fill in `LamniWebpageGenerator` — real `tomo_type` key values in
`TOMO_TYPES`, device paths / `_collect_setup_data()`, `LamNI.png`. Add a
@@ -386,10 +386,10 @@ class FlomniSampleTransferMixin:
def ensure_fheater_up(self):
axis_id = dev.fheater._config["deviceConfig"].get("axis_Id")
axis_id_numeric = self.axis_id_to_numeric(axis_id)
low, high = dev.fheater.controller.get_motor_limit_switch(axis_id)
_low, high = dev.fheater.controller.get_motor_limit_switch(axis_id)
if high:
raise FlomniError("fheater in high limit. How did we get here?? Aborting.")
if not low:
if not self._fheater_is_up():
self.ensure_osa_back()
if dev.fheater.readback.get() < -0.2:
umv(dev.fheater, -0.2)
@@ -465,6 +465,8 @@ class FlomniSampleTransferMixin:
umv(dev.fsamx, fsamx_in)
dev.fsamx.limits = [fsamx_in - 0.4, fsamx_in + 0.4]
umv(dev.fsamroy, 0)
print("Moving X-ray eye in.")
# if self.OMNYTools.yesno(
@@ -479,6 +481,8 @@ class FlomniSampleTransferMixin:
self.feye_in()
print("Moving X-ray optics out.")
self.foptics_out()
if dev.fheater.user_parameter.get("enabled"):
self.move_fheater_down()
self.xrayeye_update_frame()
def laser_tracker_show_all(self):
@@ -716,6 +720,9 @@ class FlomniSampleTransferMixin:
umv(dev.fsamroy, 0)
self.feye_in()
self.foptics_out()
self.feedback_disable()
self.ensure_fheater_up()
@@ -735,6 +742,9 @@ class FlomniSampleTransferMixin:
self.laser_tracker_off()
time.sleep(0.05)
umv(dev.fsamroy, 90)
self.drive_axis_to_limit(dev.fsamx, "forward")
dev.fsamx.limits = [-162, 0]
dev.fsamx.controller.socket_put_confirmed("axspeed[4]=25*stppermm[4]")
@@ -1177,7 +1187,7 @@ class FlomniSampleTransferMixin:
self.check_tray_in()
if position == 0:
umv(dev.ftransx, 11.02, dev.ftransz, 3.5950)
umv(dev.ftransx, 10.95, dev.ftransz, 3.5950)
if position == 1:
umv(
dev.ftransx,
@@ -1402,7 +1412,21 @@ class FlomniAlignmentMixin:
logger.warning("Not applying any correction.")
def reset_tomo_alignment_fit(self):
self.client.delete_global_var("tomo_alignment_fit")
"""Reset the alignment fit to all zeros (no offset applied).
Writes zeros rather than deleting the global var, so that
"tomo_alignment_fit" is always defined and can be tweaked by hand
(see the layout comment above get_alignment_offset).
"""
self.client.set_global_var("tomo_alignment_fit", np.zeros((2, 5)).tolist())
def ensure_tomo_alignment_fit(self):
"""Define "tomo_alignment_fit" (all zeros) if it does not exist yet.
An existing fit is left untouched so it survives a client restart.
"""
if self.client.get_global_var("tomo_alignment_fit") is None:
self.reset_tomo_alignment_fit()
def read_alignment_offset(
self,
@@ -1760,9 +1784,15 @@ class Flomni(
cSAXSBeamlineChecks,
flomniGuiTools,
):
def __init__(self, client):
# Backed by a BEC global var (Redis) so the pre-open slit position
# survives a kernel restart while the eye is still in.
_sl1xs_before_eye = _GlobalVarParam(None)
_sl1ys_before_eye = _GlobalVarParam(None)
def __init__(self, client, csaxs=None):
super().__init__()
self.client = client
self.csaxs = csaxs
self.device_manager = client.device_manager
self.special_angles = []
self.special_angle_repeats = 20
@@ -1800,6 +1830,7 @@ class Flomni(
self.label_printer = PTouchLabelPrinter(host="BRN94DDF8AAB8EC.psi.ch", template_number=1)
self.set_client(client)
self.ensure_tomo_alignment_fit()
self._maybe_reset_params_on_account_change()
def set_web_password(self, password: str) -> None:
@@ -1814,14 +1845,19 @@ class Flomni(
self._webpage_gen.start(force=True)
def _maybe_reset_params_on_account_change(self) -> None:
"""Offer a tomo-parameter reset when the active account has changed.
"""Offer a tomo-parameter/queue/progress reset when the active
account has changed.
Called at BEC session start. The account for which defaults were last
applied is stored in the global var ``defaults_applied_for_account`` so
it survives client restarts. Only a genuine change of account triggers
the (interactive) reset; the same account is a silent no-op, so
restarting a client within the same experiment never disturbs tuned
parameters.
parameters, the tomo queue, or in-progress acquisition state.
On a confirmed reset, this also clears the tomo queue and resets tomo
acquisition progress (via tomo_queue_clear()/tomo_progress_reset()),
since both are leftover state from the previous account/experiment.
"""
bec = builtins.__dict__.get("bec")
try:
@@ -1838,11 +1874,13 @@ class Flomni(
if self.OMNYTools.yesno(
f"New account '{account}' detected (previous: '{last_account}').\n"
"Reset tomo parameters to defaults for the new experiment?",
"Reset tomo parameters to defaults, clear the tomo queue, and "
"reset tomo acquisition progress for the new experiment?",
"y",
):
self._set_default_tomo_params()
print(f"Tomo parameters reset to defaults for account '{account}'.")
self.tomo_queue_clear()
self.tomo_progress_reset()
self.client.set_global_var("defaults_applied_for_account", account)
def _set_default_tomo_params(self) -> None:
@@ -1853,6 +1891,7 @@ class Flomni(
not reset here, as it is overwritten by the next alignment anyway.
"""
self.tomo_shellstep = 1
self.fermat_asymmetry = 1.0
self.tomo_countingtime = 0.1
self.manual_shift_y = 0.0
self.single_point_random_shift_max = 0.0
@@ -1998,6 +2037,8 @@ class Flomni(
tomo_shellstep = _GlobalVarParam(1)
fermat_asymmetry = _GlobalVarParam(1.0)
tomo_countingtime = _GlobalVarParam(0.1)
manual_shift_y = _GlobalVarParam(0.0)
@@ -2036,8 +2077,8 @@ class Flomni(
@fovx.setter
def fovx(self, val: float):
if val > 200:
raise ValueError("FOV cannot be larger than 200 um.")
if val > 220:
raise ValueError("FOV cannot be larger than 220 um.")
self.client.set_global_var("fovx", val)
@property
@@ -2049,8 +2090,14 @@ class Flomni(
@fovy.setter
def fovy(self, val: float):
if val > 100:
raise ValueError("FOV cannot be larger than 100 um.")
# Strict "<" to match FlomniFermatScan's own ScanArgument bound
# (gt=0, lt=100) -- fovy=100.0 was previously accepted here (and by
# the GUI/CLI, both capped at an inclusive 100.0) but rejected at
# scan time with ScanInputValidationError, only surfacing on real
# hardware. fovx has no such gap (property and FlomniFermatScan both
# use an inclusive <=220).
if val >= 100:
raise ValueError("FOV must be less than 100 um.")
self.client.set_global_var("fovy", val)
@property
@@ -2715,6 +2762,16 @@ class Flomni(
self.progress["accumulated_idle_time"] = 0.0
self.progress["heartbeat"] = None
self.collect_empty_frames()
# Unconditional single 0-degree reference projection at the very
# start of a new tomogram, independent of
# zero_deg_reference_at_each_subtomo /
# golden_projections_at_0_deg_for_damage_estimation -- gives an
# immediate radiation-damage baseline even when those flags are
# off. Real data (uses _tomo_scan_at_angle like any grid
# projection), included in reconstruction like any other
# projection -- not a flat-field/reference-only frame like
# collect_empty_frames() above. Logged under subtomo_number=1.
self._tomo_scan_at_angle(0, 1)
try:
with scans.dataset_id_on_hold:
@@ -2887,32 +2944,7 @@ class Flomni(
datetime.date.today().strftime("%Y-%m"),
status="completed",
)
idle_s = self.progress.get("accumulated_idle_time", 0.0)
start_str = self.progress.get("tomo_start_time")
elapsed_s = None
if start_str is not None:
try:
elapsed_s = (
datetime.datetime.now() - datetime.datetime.fromisoformat(start_str)
).total_seconds()
except (ValueError, TypeError):
elapsed_s = None
timing_lines = [
"Tomoscan finished.",
f"Measurement ID: {self.tomo_id}",
f"Sample: {self.sample_name}",
]
hook_description = self._describe_active_hook()
if hook_description:
timing_lines.append(f"At-each-angle hook: {hook_description}")
if elapsed_s is not None:
timing_lines.append(f"Total measurement time: {self._format_duration(elapsed_s)}")
timing_lines.append(
f"Total measurement time excluding detected gaps: {self._format_duration(elapsed_s - idle_s)}"
)
timing_lines.append(
f"Total measurement time lost to detected gaps: {self._format_duration(idle_s)}"
)
timing_lines = self._build_timing_lines()
for line in timing_lines[3:]:
print(line)
timing_content = "\n".join(timing_lines)
@@ -2972,6 +3004,72 @@ class Flomni(
self._progress_proxy.reset()
print("Tomo progress reset.")
def _build_timing_lines(self, header: str = "Tomoscan finished.") -> list:
"""Build the timing-report lines (elapsed / idle / excl.-idle
breakdown) from the current ``self.progress`` state.
Shared by the end-of-scan report in ``tomo_scan()`` and the
on-demand ``tomo_print_timing_report()`` / ``_and_scilog()``
commands, so a scan interrupted mid-flight can still get this
breakdown -- ``self.progress`` is a BEC global var, not a local,
so it survives a KeyboardInterrupt raised inside ``tomo_scan()``.
"""
idle_s = self.progress.get("accumulated_idle_time", 0.0)
start_str = self.progress.get("tomo_start_time")
elapsed_s = None
if start_str is not None:
try:
elapsed_s = (
datetime.datetime.now() - datetime.datetime.fromisoformat(start_str)
).total_seconds()
except (ValueError, TypeError):
elapsed_s = None
timing_lines = [
header,
f"Measurement ID: {self.tomo_id}",
f"Sample: {self.sample_name}",
]
hook_description = self._describe_active_hook()
if hook_description:
timing_lines.append(f"At-each-angle hook: {hook_description}")
if elapsed_s is not None:
timing_lines.append(f"Total measurement time: {self._format_duration(elapsed_s)}")
timing_lines.append(
f"Total measurement time excluding detected gaps: {self._format_duration(elapsed_s - idle_s)}"
)
timing_lines.append(
f"Total measurement time lost to detected gaps: {self._format_duration(idle_s)}"
)
return timing_lines
def tomo_print_timing_report(self) -> None:
"""Print the current tomo timing breakdown (elapsed / idle / excl.
idle) for whatever scan is or was last in progress.
Callable at any time, including right after interrupting a scan
with Ctrl-C, since the underlying data (self.progress) survives
the interrupt. Console only -- does not write to scilog.
"""
timing_lines = self._build_timing_lines(header="Tomo timing report (current progress).")
for line in timing_lines:
print(line)
def tomo_print_timing_report_and_scilog(self) -> None:
"""Same as tomo_print_timing_report(), but also writes the report
to scilog -- useful for documenting the timing of a scan that was
interrupted rather than left undocumented.
"""
timing_lines = self._build_timing_lines(header="Tomo timing report (current progress).")
for line in timing_lines:
print(line)
timing_content = "\n".join(timing_lines)
hook_source = self._active_hook_source()
if hook_source:
timing_content += (
f"\n\nAt-each-angle hook source ('{self.at_each_angle_hook}'):\n{hook_source}"
)
self._scilog_write(timing_content, "tomoscan")
@staticmethod
def _format_duration(seconds: float) -> str:
"""Format a duration in seconds as a human-readable string, e.g. '2h 03m 15s'."""
@@ -3342,12 +3440,14 @@ class Flomni(
fovx = rec.get("fovx")
fovy = rec.get("fovy")
fov_str = f"{fovx} x {fovy} um" if fovx is not None and fovy is not None else "?"
step = rec.get("tomo_shellstep")
step_str = f"{step} um" if step is not None else "?"
exposure = rec.get("tomo_countingtime")
exp_str = f"{exposure} s" if exposure is not None else "?"
duration = rec.get("duration_s")
dur_str = self._format_duration(duration) if duration is not None else "?"
lines.append(
f"scan {scan_str}: FOV {fov_str}, exposure {exp_str}, duration {dur_str}"
f"scan {scan_str}: FOV {fov_str}, step {step_str}, exposure {exp_str}, duration {dur_str}"
)
content = "\n".join(lines)
@@ -3458,7 +3558,8 @@ class Flomni(
ceny = sum_offset_y + stitch_y * (self.fovy - self.tomo_stitch_overlap)
logger.info(
f"scans.flomni_fermat_scan(fovx={self.fovx}, fovy={self.fovy},"
f" step={self.tomo_shellstep}, cenx={cenx}, ceny={ceny},"
f" step={self.tomo_shellstep}, fermat_asymmetry={self.fermat_asymmetry},"
f" cenx={cenx}, ceny={ceny},"
f" zshift={sum_offset_z}, angle={angle},"
f" exp_time={self.tomo_countingtime}, corridor_size={corridor_size})"
)
@@ -3471,6 +3572,7 @@ class Flomni(
fovx=self.fovx,
fovy=self.fovy,
step=self.tomo_shellstep,
fermat_asymmetry=self.fermat_asymmetry,
cenx=cenx,
ceny=ceny,
zshift=sum_offset_z,
@@ -3624,6 +3726,13 @@ class Flomni(
cenx/ceny/zshift are deliberately left at their defaults: they only
shift the final position, applied after the FOV keep-or-discard
check, so they can never affect the resulting count.
The real scan draws a fresh random sub-step offset on every call
(see get_flomni_fermat_spiral_pos()'s `seed` argument), which can
shift a point or two across the FOV edge -- so the actual scan-time
count can differ from this estimate by a small amount near the
threshold. A fixed seed is used here only so that this estimate
itself is reproducible from one call to the next.
"""
from csaxs_bec.scans.flomni_fermat_scan import FlomniFermatScan
@@ -3635,6 +3744,8 @@ class Flomni(
step=self.tomo_shellstep,
spiral_type=0,
center=False,
seed=0,
fermat_asymmetry=self.fermat_asymmetry,
)
return len(positions)
@@ -3649,9 +3760,10 @@ class Flomni(
def tomo_parameters(self):
"""print and update the tomo parameters"""
print("Current settings:")
print(f"Counting time <ctime> = {self.tomo_countingtime} s")
print(f"Counting time per frame <ctime> = {self.tomo_countingtime} s")
print(f"Stepsize microns <step> = {self.tomo_shellstep}")
print(f"FOV (200/100) <microns> = {self.fovx}, {self.fovy}")
print(f"Fermat asymmetry (x/y) = {self.fermat_asymmetry}")
print(f"FOV (220/100) <microns> = {self.fovx}, {self.fovy}")
print(f"Stitching number x,y = {self.stitch_x}, {self.stitch_y}")
print(f"Stitching overlap = {self.tomo_stitch_overlap}")
if not self.single_point_instead_of_fermat_scan:
@@ -3737,10 +3849,23 @@ class Flomni(
if self.OMNYTools.yesno("Are these parameters correctly set for your scan?", "y"):
print("... excellent!")
else:
self.tomo_countingtime = self._get_val("<ctime> s", self.tomo_countingtime, float)
self.tomo_shellstep = self._get_val("<step size> um", self.tomo_shellstep, float)
self.fovx = self._get_val("<FOV X (max 200)> um", self.fovx, float)
self.fovy = self._get_val("<FOV Y (max 100)> um", self.fovy, float)
from csaxs_bec.scans.flomni_fermat_scan import FlomniFermatScan
self.tomo_countingtime = self._get_val(
"<ctime per frame> s", self.tomo_countingtime, float, min_=0.001, max_=100.0
)
self.tomo_shellstep = self._get_val(
"<step size> um", self.tomo_shellstep, float, min_=0.025, max_=20.0
)
self.fermat_asymmetry = self._get_val(
"<fermat asymmetry x/y, 1=symmetric>",
self.fermat_asymmetry,
float,
min_=FlomniFermatScan.MIN_FERMAT_ASYMMETRY,
max_=FlomniFermatScan.MAX_FERMAT_ASYMMETRY,
)
self.fovx = self._get_val("<FOV X (max 220)> um", self.fovx, float, min_=0.1, max_=220.0)
self.fovy = self._get_val("<FOV Y (max 100)> um", self.fovy, float, min_=0.1, max_=99.9)
if self.single_point_instead_of_fermat_scan:
print(
"Stitching is disabled while single point instead of fermat scan is"
@@ -3749,14 +3874,16 @@ class Flomni(
self.stitch_x = 0
self.stitch_y = 0
else:
self.stitch_x = self._get_val("<stitch X>", self.stitch_x, int)
self.stitch_y = self._get_val("<stitch Y>", self.stitch_y, int)
self.stitch_x = self._get_val("<stitch X>", self.stitch_x, int, min_=0, max_=10)
self.stitch_y = self._get_val("<stitch Y>", self.stitch_y, int, min_=0, max_=10)
self.ptycho_reconstruct_foldername = self._get_val(
"Reconstruction queue ", self.ptycho_reconstruct_foldername, str
)
self.manual_shift_y = self._get_val("<manual_shift_y> um", self.manual_shift_y, float)
self.manual_shift_y = self._get_val(
"<manual_shift_y> um", self.manual_shift_y, float, min_=-50.0, max_=50.0
)
self.frames_per_trigger = self._get_val(
"Frames per trigger (burst)", self.frames_per_trigger, int
"Frames per trigger (burst)", self.frames_per_trigger, int, min_=1, max_=100
)
self.single_point_instead_of_fermat_scan = bool(
self._get_val(
@@ -3779,6 +3906,8 @@ class Flomni(
"<single point random shift max> um (0 = off)",
self.single_point_random_shift_max,
float,
min_=0.0,
max_=10.0,
)
print("Tomography type:")
@@ -3800,7 +3929,7 @@ class Flomni(
# never reflect a prior adjustment.
_, _, current_total = self._tomo_type1_actual_grid()
tomo_numberofprojections = self._get_val(
"Total number of projections", current_total, int
"Total number of projections", current_total, int, min_=8, max_=10000
)
# N/step (and therefore total projections) are always
# computed against a fixed 180 degrees, independent of
@@ -3849,11 +3978,15 @@ class Flomni(
"Number of projections sorted per bunch (default 20)",
self.golden_ratio_bunch_size,
int,
min_=1,
max_=100,
)
self.golden_max_number_of_projections = self._get_val(
"Stop after number of projections (zero for endless)",
self.golden_max_number_of_projections,
int,
min_=0,
max_=1_000_000,
)
self.golden_projections_at_0_deg_for_damage_estimation = self._get_val(
"Repeat projections at 0 deg every second subtomo 1/0 ?",
@@ -3866,12 +3999,16 @@ class Flomni(
"Number of projections per sub-tomogram",
int(180 / self.tomo_angle_stepsize),
int,
min_=1,
max_=10000,
)
self.tomo_angle_stepsize = 180 / numprj
self.golden_max_number_of_projections = self._get_val(
"Stop after number of projections (zero for endless)",
self.golden_max_number_of_projections,
int,
min_=0,
max_=1_000_000,
)
self.golden_projections_at_0_deg_for_damage_estimation = self._get_val(
"Repeat projections at 0 deg every second subtomo",
@@ -3880,18 +4017,33 @@ class Flomni(
)
@staticmethod
def _get_val(msg: str, default_value, data_type):
return data_type(input(f"{msg} ({default_value}): ") or default_value)
def _get_val(msg: str, default_value, data_type, min_=None, max_=None):
"""Prompt for a value, casting to data_type. When min_/max_ are
given, re-prompts (rather than silently clamping) until the typed
value is in range, mirroring the Tomo Parameters GUI's spinbox
bounds (tomo_params.py) so tomo_parameters() enforces the same
limits instead of accepting anything of the right type."""
while True:
val = data_type(input(f"{msg} ({default_value}): ") or default_value)
if min_ is not None and val < min_:
print(f"Value must be >= {min_}. Please try again.")
continue
if max_ is not None and val > max_:
print(f"Value must be <= {max_}. Please try again.")
continue
return val
# Ordered set of all global-var-backed tomo scan parameters: exactly the
# settings shown by tomo_parameters(), plus manual_shift_y/
# tomo_stitch_overlap/corridor_size/at_each_angle_hook which also affect
# the scan but are only set directly as properties. This is the full
# "parameter set" that tomo_queue_add()/tomo_queue_execute() snapshot
# and restore.
# settings shown by tomo_parameters() (including manual_shift_y, which
# is prompted there too), plus tomo_stitch_overlap/corridor_size/
# at_each_angle_hook, which also affect the scan but are only ever set
# directly as properties -- tomo_parameters() never prompts for them.
# This is the full "parameter set" that tomo_queue_add()/
# tomo_queue_execute() snapshot and restore.
_TOMO_SCAN_PARAM_NAMES = (
"tomo_countingtime",
"tomo_shellstep",
"fermat_asymmetry",
"fovx",
"fovy",
"stitch_x",
@@ -4117,6 +4269,7 @@ class Flomni(
f"{'Current photon energy:':<{padding}}{energy_str}\n",
f"{'Exposure time:':<{padding}}{self.tomo_countingtime:.2f}\n",
f"{'Fermat spiral step size:':<{padding}}{self.tomo_shellstep:.2f}\n",
f"{'Fermat asymmetry (x/y):':<{padding}}{self.fermat_asymmetry:.2f}\n",
f"{'FOV:':<{padding}}{fovxy}\n",
f"{'Stitching:':<{padding}}{stitching}\n",
f"{subtomo_count_label:<{padding}}{subtomo_count_value}\n",
@@ -23,8 +23,49 @@ class FlomniOpticsMixin:
foptz_val = dev.foptz.readback.get()
return -fosaz_target + (33 - foptz_val)
def _fheater_is_up(self) -> bool:
axis_id = dev.fheater._config["deviceConfig"].get("axis_Id")
low, _high = dev.fheater.controller.get_motor_limit_switch(axis_id)
return bool(low)
def _fheater_is_down(self) -> bool:
fheater_in = dev.fheater.user_parameter.get("in")
if fheater_in is None:
return False
return np.isclose(dev.fheater.readback.get(), fheater_in, atol=0.1)
def feye_out(self):
dev.fsh.fshclose()
if self.csaxs is not None:
self.csaxs.fil_trans(1, print_only=False)
# sl1xs_before = self._sl1xs_before_eye
# sl1ys_before = self._sl1ys_before_eye
# if sl1xs_before is not None and sl1ys_before is not None:
# # if the slit was manually re-adjusted while the eye was in,
# # offer to keep the current size instead of restoring the
# # pre-eye value
# current_sl1xs = dev.sl1xs.user_readback.get()
# current_sl1ys = dev.sl1ys.user_readback.get()
# expected_sl1xs = sl1xs_before + 0.3
# expected_sl1ys = sl1ys_before + 0.3
# tol = 0.01 # mm (10 microns)
# manually_adjusted = not np.isclose(
# current_sl1xs, expected_sl1xs, atol=tol
# ) or not np.isclose(current_sl1ys, expected_sl1ys, atol=tol)
# if manually_adjusted and self.OMNYTools.yesno(
# "sl1xs/sl1ys appear to have been adjusted manually while the "
# "eye was in. Keep the current slit size instead of "
# "restoring the pre-eye value?"
# ):
# print("Keeping current slit size, not restoring pre-eye value.")
# else:
# umv(dev.sl1xs, sl1xs_before, dev.sl1ys, sl1ys_before)
# self._sl1xs_before_eye = None
# self._sl1ys_before_eye = None
self.foptics_in()
self.flomnigui_show_xeyealign()
self.xrayeye_update_frame()
@@ -34,6 +75,7 @@ class FlomniOpticsMixin:
print("Aborting. With visible parts of the direct beam on the xray eye, it cannot be removed.")
return
feyex_out = self._get_user_param_safe("feyex", "out")
umv(dev.feyex, feyex_out)
# move rotation stage to zero to avoid problems with wires
@@ -73,13 +115,22 @@ class FlomniOpticsMixin:
current_feyex = dev.feyex.readback.get()
current_feyey = dev.feyey.readback.get()
# check if both are close enough (within 0.01)
if np.isclose(current_feyex, feyex_in, atol=0.01) and np.isclose(current_feyey, feyey_in, atol=0.01):
# both already in position → do nothing
pass
else:
# move both axes to the desired "in" positions
umv(dev.feyex, feyex_in, dev.feyey, feyey_in)
# # check if both are close enough (within 0.01)
# if np.isclose(current_feyex, feyex_in, atol=0.01) and np.isclose(current_feyey, feyey_in, atol=0.01):
# # both already in position → do nothing
# pass
# else:
# if "sl1xs" in dev and "sl1ys" in dev:
# self._sl1xs_before_eye = dev.sl1xs.user_readback.get()
# self._sl1ys_before_eye = dev.sl1ys.user_readback.get()
# scans.umv(dev.sl1xs, 0.3, dev.sl1ys, 0.3, relative=True)
# if self.csaxs is not None:
# self.csaxs.fil_trans(0.05, print_only=False)
# # move both axes to the desired "in" positions
# umv(dev.feyex, feyex_in, dev.feyey, feyey_in)
umv(dev.feyex, feyex_in, dev.feyey, feyey_in)
def _ffzp_in(self):
foptx_in = self._get_user_param_safe("foptx", "in")
@@ -167,13 +218,6 @@ class FlomniOpticsMixin:
dev.rtx.controller.feedback_enable_with_reset()
def fosa_in(self):
# TODO(commissioning): the OSA travels inside the heater's (fheater)
# envelope, but this function never checks fheater position before
# driving fosaz. fosaz must only move while fheater is fully "up" or
# fully "down" -- an intermediate heater position here risks a
# collision. Add a heater-state check (see ensure_osa_back()/
# ensure_fheater_up() in flomni.py for the pattern) once fheater's
# up/down limits are commissioned.
# 6.2 keV, 170 um FZP
# umv(dev.losax, -1.4450000, dev.losay, -0.1800)
# umv(dev.losaz, -1)
@@ -205,6 +249,13 @@ class FlomniOpticsMixin:
)
if need_move_osa:
if not (self._fheater_is_up() or self._fheater_is_down()):
raise FlomniOpticsError(
"Refusing to move OSA to its IN position: fheater is neither "
"at its 'up' (retracted) limit nor at its 'down' (heating) "
"position. An intermediate heater position risks a collision "
"with the OSA. Move the heater fully up or fully down first."
)
remaining = self._osa_remaining_space(fosaz_in)
if remaining <= 0:
foptz_val = dev.foptz.readback.get()
@@ -268,7 +319,13 @@ class FlomniOpticsMixin:
def fosa_out(self):
self.ensure_fheater_up()
if not (self._fheater_is_up() or self._fheater_is_down()):
raise FlomniOpticsError(
"Refusing to move OSA to its OUT position: fheater is neither "
"at its 'up' (retracted) limit nor at its 'down' (heating) "
"position. An intermediate heater position risks a collision "
"with the OSA. Move the heater fully up or fully down first."
)
curtain_is_triggered = dev.foptz.controller.fosaz_light_curtain_is_triggered()
if not curtain_is_triggered:
fosaz_out = self._get_user_param_safe("fosaz", "out")
@@ -138,20 +138,18 @@ class flomniGuiTools:
print("Cannot open camera_overview. Device does not exist.")
# Confirm/abort console, docked below the cameras. The hard-stop
# button directly calls dev.ftransy.controller
# .hard_abort_and_restore_positioning_mode() (the same Galil
# hard-stop ftransfer_abort() uses) -- see ConsoleButtonsWidget's
# docstring for why this replaced a blind stop-all-devices
# broadcast. foptx sits on a physically separate Galil
# controller (the optics stage board, not the transfer stage
# board) so it needs its own stop_all_axes() call -- also
# explained in ConsoleButtonsWidget's docstring.
# button sends a stop to each of flomni's three Galil controllers
# (one representative device per controller, in this order, the
# sample-transfer controller first): ftransy (fsamx, fsamy,
# ftransx/y/z, ftray), foptx (optics, eye, heater, tracking) and
# fsamroy (rotation). Afterwards it restores positioning mode on
# the first one (the same Galil hard-stop ftransfer_abort() uses)
# -- see ConsoleButtonsWidget's docstring.
self.console = self.gui.flomni.new(
"z_ConsoleButtonsWidget",
object_name="console",
where="bottom",
hard_stop_device_name="ftransy",
extra_hard_stop_device_name="foptx",
hard_stop_device_names=["ftransy", "foptx", "fsamroy"],
hard_stop_label="Flomni Motion Stop",
)
# set_layout_ratios uses relative weights, not pixels -- there is
@@ -41,6 +41,14 @@ class XrayEyeAlign:
HEIGHT_CENTERING_SIGN = -1
# Fit steps k = 1..N_FIT_STEPS are the angles 0, 45, ..., 180 deg (angle = (k - 1) *
# ANGLE_STEP_DEG). The sine fit has 3 free parameters, so at most
# N_FIT_STEPS - MIN_FIT_POINTS of them may be skipped from the GUI (submit == -1), e.g.
# the 90 deg step for a flat sample that is edge-on there.
N_FIT_STEPS = 5
ANGLE_STEP_DEG = 45
MIN_FIT_POINTS = 4
def __init__(self, client, flomni: Flomni) -> None:
self.client = client
self.flomni = flomni
@@ -95,14 +103,50 @@ class XrayEyeAlign:
# image_idx refers to alignment_images[image_idx], i.e. the last
# frame captured before that submit (shutter is closed at submit time).
self.roi_pixel_data = []
# Fit steps (k) the user skipped from the GUI; they have no alignment_values entry.
self.skipped_steps = []
def _save_alignment_data(self, file_path: str, fit_data: np.ndarray | None = None):
def _step_angle(self, k: int) -> int:
"""Rotation angle in deg of fit step k (1..N_FIT_STEPS)."""
return (k - 1) * self.ANGLE_STEP_DEG
def _step_prompt(self, k: int) -> str:
"""The instruction shown to the user while waiting for the submission of step k."""
if k == 0:
return "Submit center value of FZP."
if k == 1 and not self._height_centered:
return "Mark the sample height with the selection tool, then submit."
return f"Step {k}/{self.N_FIT_STEPS}: Submit sample center"
def _skip_refusal_reason(self, k: int) -> str | None:
"""Why step k cannot be skipped, or None if a skip is allowed.
Only real fit points can be skipped: not the FZP center (k == 0) nor the height-marking
submission (the first submission at k == 1), and only while at least MIN_FIT_POINTS
remain for the fit.
"""
if k == 0:
return "the FZP center cannot be skipped"
if k == 1 and not self._height_centered:
return "the sample height marking cannot be skipped"
if len(self.skipped_steps) + 1 > self.N_FIT_STEPS - self.MIN_FIT_POINTS:
return f"at least {self.MIN_FIT_POINTS} positions are needed for the fit"
return None
def _save_alignment_data(
self,
file_path: str,
fit_data: np.ndarray | None = None,
skipped_angles: list[int] | None = None,
):
expanded = os.path.expanduser(file_path)
os.makedirs(os.path.dirname(expanded), exist_ok=True)
with h5py.File(expanded, "w") as f:
f.create_dataset(
ds = f.create_dataset(
"alignment_values", data=np.array(list(self.alignment_values.values()))
)
# skipped steps have no entry, so say which step each value belongs to
ds.attrs["step_k"] = list(self.alignment_values.keys())
f.create_dataset("alignment_images", data=np.array(self.alignment_images))
if self.roi_pixel_data:
ds = f.create_dataset("roi_pixel_data", data=np.array(self.roi_pixel_data))
@@ -110,6 +154,7 @@ class XrayEyeAlign:
if fit_data is not None:
ds = f.create_dataset("alignment_fit", data=fit_data)
ds.attrs["rows"] = ["angles_deg", "offsets_um", "zeros"]
ds.attrs["skipped_angles_deg"] = list(skipped_angles or [])
def update_frame(self, keep_shutter_open=False):
if self.flomni._flomnigui_check_attribute_not_exists("xeyegui"):
@@ -164,6 +209,10 @@ class XrayEyeAlign:
try:
self._align_impl(keep_shutter_open)
finally:
try:
self.gui.enable_skip_button(False)
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Failed to disable XRayEye skip button: {exc}")
try:
self.gui.set_dap_params_forwarding(False)
except Exception as exc: # pylint: disable=broad-except
@@ -172,6 +221,10 @@ class XrayEyeAlign:
self.gui.hide_crosshair()
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Failed to hide XRayEye alignment crosshair: {exc}")
try:
self.gui.unlock_vertical_center()
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Failed to unlock XRayEye ROI vertical center: {exc}")
def _align_impl(self, keep_shutter_open=False):
if not keep_shutter_open:
@@ -182,6 +235,7 @@ class XrayEyeAlign:
self.alignment_images = []
self.gui.enable_submit_button(False)
self.gui.enable_skip_button(False)
# Initialize xray align device
# clear potential pending movement requests
@@ -194,6 +248,9 @@ class XrayEyeAlign:
# reset shift xy and fov params
self._reset_init_values()
# a fresh run must not inherit a stale lock from a previous (possibly
# interrupted) run -- see lock_vertical_center()/unlock_vertical_center()
self.gui.unlock_vertical_center()
# Moved here from __init__: this is the actual start of a fresh
# alignment run (matching what start_x_ray_eye_alignment() already
@@ -240,32 +297,59 @@ class XrayEyeAlign:
k = 0
while True:
if dev.omny_xray_gui.submit.get() == 1:
# submit: 0 = idle, 1 = submitted, -1 = skip this step (see the Skip button)
submit_val = dev.omny_xray_gui.submit.get()
if submit_val == -1:
refusal = self._skip_refusal_reason(k)
if refusal is not None:
# refused: the run is untouched, keep waiting at the same step
dev.omny_xray_gui.submit.set(0)
self.gui.enable_skip_button(False)
self.send_message(f"Cannot skip: {refusal}. {self._step_prompt(k)}")
print(f"Skip refused at step {k}: {refusal}.")
submit_val = 0
if submit_val in (1, -1):
skipped = submit_val == -1
self.alignment_values[k] = (
getattr(dev.omny_xray_gui, f"xval_x_{k}").get() * self.pixel_calibration
) # in mm
print(f"Clicked position {k}: x {self.alignment_values[k]}")
rtx_position = dev.rtx.readback.get() / 1000
print(f"Current rtx position {rtx_position}")
self.alignment_values[k] -= rtx_position
print(f"Corrected position {k}: x {self.alignment_values[k]}")
# reset submit channel
dev.omny_xray_gui.submit.set(0)
if skipped:
# no xval_x_k / width_*_k were written for this step, so never read them
dev.omny_xray_gui.submit.set(0)
self.skipped_steps.append(k)
_img_idx = len(self.alignment_images) - 1
self.roi_pixel_data.append([k, np.nan, np.nan, np.nan, np.nan, _img_idx])
print(f"Skipped position {k} ({self._step_angle(k)} deg)")
if self._step_angle(k) in (0, 180):
print(
"Warning: skipping an end point (0/180 deg) makes the fit noticeably"
" less stable than skipping 45/90/135 deg."
)
# only allow another skip if the fit can still spare a point
self.gui.enable_skip_button(self._skip_refusal_reason(k) is None)
else:
self.alignment_values[k] = (
getattr(dev.omny_xray_gui, f"xval_x_{k}").get() * self.pixel_calibration
) # in mm
print(f"Clicked position {k}: x {self.alignment_values[k]}")
rtx_position = dev.rtx.readback.get() / 1000
print(f"Current rtx position {rtx_position}")
self.alignment_values[k] -= rtx_position
print(f"Corrected position {k}: x {self.alignment_values[k]}")
# reset submit channel
dev.omny_xray_gui.submit.set(0)
# Raw pixel position and ROI size at submit time.
# The relevant image is the last captured frame (shutter is
# closed by the time the user clicks submit).
_raw_x = getattr(dev.omny_xray_gui, f"xval_x_{k}").get()
_raw_y = getattr(dev.omny_xray_gui, f"yval_y_{k}").get()
_raw_w = getattr(dev.omny_xray_gui, f"width_x_{k}").get()
_raw_h = getattr(dev.omny_xray_gui, f"width_y_{k}").get()
_img_idx = len(self.alignment_images) - 1
print(
f" Submit k={k}: px x={_raw_x:.1f} y={_raw_y:.1f} "
f"w={_raw_w:.1f} h={_raw_h:.1f} img={_img_idx}"
)
self.roi_pixel_data.append([k, _raw_x, _raw_y, _raw_w, _raw_h, _img_idx])
# Raw pixel position and ROI size at submit time.
# The relevant image is the last captured frame (shutter is
# closed by the time the user clicks submit).
_raw_x = getattr(dev.omny_xray_gui, f"xval_x_{k}").get()
_raw_y = getattr(dev.omny_xray_gui, f"yval_y_{k}").get()
_raw_w = getattr(dev.omny_xray_gui, f"width_x_{k}").get()
_raw_h = getattr(dev.omny_xray_gui, f"width_y_{k}").get()
_img_idx = len(self.alignment_images) - 1
print(
f" Submit k={k}: px x={_raw_x:.1f} y={_raw_y:.1f} "
f"w={_raw_w:.1f} h={_raw_h:.1f} img={_img_idx}"
)
self.roi_pixel_data.append([k, _raw_x, _raw_y, _raw_w, _raw_h, _img_idx])
# Controls whether `k` advances to the next step below. Left
# True except for the height-centering submission, which
@@ -299,7 +383,7 @@ class XrayEyeAlign:
self.gui.show_crosshair()
self.send_message(
"Submit height. Use arrows if far off."
"Mark the sample height with the selection tool, then submit. Use arrows if far off."
)
self.gui.enable_submit_button(True)
self.movement_buttons_enabled(True, True)
@@ -332,10 +416,16 @@ class XrayEyeAlign:
self.flomni.feedback_enable_with_reset()
self._height_centered = True
# From here through step 5, the sample's vertical position is
# physically fixed at the FZP height -- auto-snap every new/
# dragged/resized selection box's vertical center to the
# crosshair instead of requiring manual placement each time.
self.gui.lock_vertical_center()
self.update_frame(keep_shutter_open)
self.send_message("Step 1/5: Submit sample center")
self.gui.enable_submit_button(True)
self.gui.enable_skip_button(True)
self.movement_buttons_enabled(True, False)
advance_step = False
@@ -351,13 +441,16 @@ class XrayEyeAlign:
self.send_message(f"Step {k+1}/5: Submit sample center")
self.gui.enable_submit_button(True)
self.movement_buttons_enabled(True, False)
self.update_fov(k)
if not skipped:
self.update_fov(k)
elif k == 5: # received sample center value at samroy 270 and done
self.send_message("done...")
self.gui.enable_submit_button(False)
self.movement_buttons_enabled(False, False)
self.update_fov(k)
self.gui.enable_skip_button(False)
if not skipped:
self.update_fov(k)
self.gui.hide_crosshair()
break
@@ -422,19 +515,34 @@ class XrayEyeAlign:
timestamp = time.strftime("%Y%m%d_%H%M%S")
file_h5 = f"~/data/raw/logs/xrayeye_alignmentvalues/xrayeye_alignmentvalues_{timestamp}.h5"
fovx_offsets = np.zeros(5)
for k in range(1, 6):
steps = [k for k in range(1, self.N_FIT_STEPS + 1) if k not in self.skipped_steps]
if len(steps) < self.MIN_FIT_POINTS:
raise RuntimeError(
f"Only {len(steps)} alignment positions available, at least"
f" {self.MIN_FIT_POINTS} are needed for the fit."
)
fovx_offsets = np.zeros(len(steps))
for i, k in enumerate(steps):
fovx_offset = self.alignment_values[0] - self.alignment_values[k]
fovx_offsets[k - 1] = fovx_offset
fovx_offsets[i] = fovx_offset
print(f"Alignment number {k}, value x {fovx_offset}")
skipped_angles = [self._step_angle(k) for k in self.skipped_steps]
if skipped_angles:
print(
f"Skipped angles {skipped_angles} deg, fitting {len(steps)} positions. With 4"
" points the fit has 1 degree of freedom and cannot flag a bad click - check the"
" plot."
)
data = np.array(
[
[0, 45, 90, 135, 180], # angles
[self._step_angle(k) for k in steps], # angles
fovx_offsets * 1000, # fovx_offset values
[0, 0, 0, 0, 0],
np.zeros(len(steps)),
]
)
self._save_alignment_data(file_h5, fit_data=data)
self._save_alignment_data(file_h5, fit_data=data, skipped_angles=skipped_angles)
self.gui.submit_fit_array(data)
print(f"fit submited with {data}")
@@ -0,0 +1,31 @@
# TODO: document stitch_x/stitch_y in the omny user manual
Status: **not implemented.** `docs/user/ptychography/omny.md` has zero
mentions of stitching, same gap just fixed 2026-09-19 in `flomni.md` and
`lamni.md`.
## What to add
Omny's mechanism is essentially identical to Flomni's (same parameter
names, even) — `omny.py:1263-1269`:
```python
for stitch_x in range(-self.stitch_x, self.stitch_x + 1):
for stitch_y in range(-self.stitch_y, self.stitch_y + 1):
...
cenx = sum_offset_x + stitch_x * (self.fovx - self.tomo_stitch_overlap)
ceny = sum_offset_y + stitch_y * (self.fovy - self.tomo_stitch_overlap)
```
Same paragraph shape as the ones just added to `flomni.md`/`lamni.md` should
carry over directly, substituting nothing (param names are identical):
`stitch_x`/`stitch_y` (default 0), tiled `(2*stitch_x+1) × (2*stitch_y+1)`
grid of scans per projection angle, spaced by `fovx`/`fovy` minus
`tomo_stitch_overlap` (µm, default 0.2), effective FOV roughly
`(2*stitch_x+1) × fovx` by `(2*stitch_y+1) × fovy`.
Not yet checked: omny.md's actual section structure/where the equivalent of
flomni's "### Tomography" / lamni's "### Test projections and Laminography
scan" section lives, and whether omny has anything resembling
`single_point_instead_of_fermat_scan`'s stitch-forced-to-0 interaction
(flomni does; not confirmed for omny).
@@ -0,0 +1,103 @@
# TODO: port `fermat_asymmetry` (x/y spacing ratio) to OMNY now that it's landed on Flomni
**Status: `fermat_asymmetry` is now implemented on Flomni** (branch
`fixes/flomni_beamtime_2`), not yet ported to OMNY. See
`docs/plans/flomni-fermat-asymmetry.md` for the original design (the
implementation follows it as written) and `csaxs_bec/scans/flomni_fermat_scan.py`
for the actual landed code (`get_flomni_fermat_spiral_pos`, line ~420 at
time of writing; `MIN_FERMAT_ASYMMETRY`/`MAX_FERMAT_ASYMMETRY` class
constants, line ~70).
`fermat_asymmetry` is a ratio of x spacing to y spacing; default `1.0` =
symmetric, keeps x spacing fixed at `step` and scales y spacing by the ratio
while preserving the requested FOV. Soft-clamped to `[0.1, 10]` (with a
`logger.warning` when clamped), same style as `zshift`/`corridor_size`.
**Effective spacing (verified against generated data, see
`docs/plans/flomni-fermat-asymmetry.md`'s conversation for the measurement
method):** x spacing always stays at `step`; y spacing becomes `step /
fermat_asymmetry`. E.g. with `step=1 um`, `fovx=fovy=10 um`: `fermat_asymmetry=0.7`
-> x spacing 1.0 um, y spacing ~1.43 um (sparser), ~70 points;
`fermat_asymmetry=1.3` -> x spacing 1.0 um, y spacing ~0.77 um (denser),
~130 points. Point count scales with the ratio since the FOV is fixed but
the density isn't. Same relationship should be documented for OMNY once
ported (Flomni's user doc has this table -- see
`docs/user/ptychography/flomni.md`, "Effective spacing with
`fermat_asymmetry`" -- OMNY's equivalent doc, if any, should get the same).
## Important: compose with the already-ported sub-step randomization
Before this, `fixes/flomni_beamtime_2` already ported a *different* fix to
all three setups including OMNY: a per-call random sub-step offset (see
`docs/plans/fermat-scan-randomization.md`). `OmnyFermatScan.get_omny_fermat_spiral_pos`
(`csaxs_bec/scans/omny_fermat_scan.py`) already has a `seed: int | None = None`
parameter and draws `offset = np.random.default_rng(seed).uniform(0, step)`,
subtracted from both `x` and `y` before the FOV crop check -- this landed
*before* fermat_asymmetry and is unrelated to it, but the two must compose
correctly when porting fermat_asymmetry to OMNY.
Flomni's landed implementation composes them as: `x = radius*sin(...) -
offset` (unchanged); `y_raw = radius*cos(...) - offset` (offset subtracted
at the *raw*, pre-compression scale, same as x); check `abs(y_raw) >
(length_axis2 * fermat_asymmetry) / 2` (inflated window); on accept, `y =
y_raw / fermat_asymmetry` (compression happens *after* the offset and the
check, not before). This keeps the offset's phase-shift proportional to the
local spacing on both axes (x spacing = step, y spacing = step/fermat_asymmetry,
offset/fermat_asymmetry is the resulting y-phase-shift once compressed) --
port this exact ordering (offset subtraction -> inflated check -> compression),
not just the offset or the asymmetry in isolation.
## What needs to happen
OMNY's `OmnyFermatScan.get_omny_fermat_spiral_pos`
(`csaxs_bec/scans/omny_fermat_scan.py`) uses the byte-for-byte same spiral
algorithm as Flomni's `get_flomni_fermat_spiral_pos` (`radius = step * 0.57
* sqrt(ii)`, same FOV-window filter, same offset-subtraction-before-check
shape already ported), so the same change ports directly:
1. Add `MIN_FERMAT_ASYMMETRY = 0.1` / `MAX_FERMAT_ASYMMETRY = 10.0` class
constants to `OmnyFermatScan`, mirroring Flomni's.
2. Add `fermat_asymmetry` to `OmnyFermatScan.__init__`
(`omny_fermat_scan.py:63-79`) as a new `ScanArgument`, store it, pass it
through in `prepare_scan()`'s call to `get_omny_fermat_spiral_pos`.
3. Add the clamp + inflated-window-check + y-compression logic to
`get_omny_fermat_spiral_pos` itself (same reasoning as Flomni: it's also
called directly by preview/estimate code, so clamping there keeps every
caller in sync automatically) -- apply it on top of the existing offset
logic exactly as described above, not before it.
4. Mirror the CLI-side plumbing on `OMNY`
(`csaxs_bec/bec_ipython_client/plugins/omny/omny.py`): a
`tomo_shellstep`-style property pair (OMNY hand-writes
`get_global_var`/`set_global_var` property pairs, e.g. `omny.py:671-679`,
rather than using the `_GlobalVarParam` descriptor Flomni uses), the
`scan_kwargs` dict in the per-angle scan call (~`omny.py:1282-1284`), and
`tomo_parameters()` (~`omny.py:1293-1341`).
## Difference from Flomni to account for when porting
OMNY has no tomo-queue system -- `OMNY` does not mix in `TomoQueueMixin` and
has no `_TOMO_SCAN_PARAM_NAMES` (unlike `Flomni`/`LamNI`). So there is no
queue-snapshot list to add the new parameter to on OMNY's side, and no
`tomo_params` Qt widget profile for OMNY today either (`SETUP_PROFILES` in
`tomo_params.py` only has `"flomni"` and `"lamni"` entries) -- the widget
part of the Flomni implementation (the "Fermat asymmetry (x/y)" field with
its "?" help button in `_build_fermat_asymmetry_row`/
`_show_fermat_asymmetry_help`) doesn't apply until/unless OMNY gets its own
queue + widget support. Scope the OMNY port to: the scan class parameter,
the global var, and `tomo_parameters()` display/prompt only.
## Tests
Mirror the three new cases added to `tests/tests_scans/test_flomni_fermat_scan.py`
(`test_fermat_asymmetry_default_matches_symmetric_spiral`,
`test_fermat_asymmetry_biases_spacing_while_preserving_fov`,
`test_fermat_asymmetry_out_of_range_is_clamped_not_raised`) in
`tests/tests_scans/test_omny_fermat_scan.py` -- same technique (compare
against an equivalent call with an inflated FOV and `fermat_asymmetry=1.0`,
same `seed`, to verify the inflate/compress math directly rather than via
an approximate spacing-ratio heuristic).
## Explicitly not for LamNI
This feature was scoped to Flomni (and, via this TODO, OMNY) only; do not
add it to `LamniFermatScan` / `LamNI` as part of this port.
@@ -0,0 +1,47 @@
# TODO: port `randomize_offset` (switch off the sub-step randomization) to OMNY
**Status: `randomize_offset` is implemented on Flomni only (branch `fixes/flomni_beamtime_2`,
2026-09-20), not yet ported to OMNY.**
## What landed on Flomni
The per-call random sub-step offset of the Fermat spiral (see `docs/plans/fermat-scan-randomization.md`)
had no way to be turned off from the scan: `seed` exists only as an argument of the static position
function and is not a scan argument. A fixed seed would only make the offset reproducible anyway --
the offset is always `uniform(0, step)`, so no seed gives the unshifted lattice.
Flomni now has an optional scan argument `randomize_offset: bool = True`
(`csaxs_bec/scans/flomni_fermat_scan.py`):
- `FlomniFermatScan.__init__` takes it as a `ScanArgument` (default `True`, so behaviour is
unchanged), stores `self.randomize_offset` and passes it in `prepare_scan()`.
- `get_flomni_fermat_spiral_pos(..., randomize_offset=True)` computes
`offset = np.random.default_rng(seed).uniform(0, step) if randomize_offset else 0.0`.
With `False` the offset is 0 and `seed` is irrelevant.
- Tests: `test_randomize_offset_default_is_active` and
`test_randomize_offset_false_gives_unshifted_deterministic_lattice` in
`tests/tests_scans/test_flomni_fermat_scan.py`.
- User doc: `docs/user/ptychography/flomni.md` (parameter table + paragraph on the randomization).
## What needs to happen on OMNY
`OmnyFermatScan.get_omny_fermat_spiral_pos` (`csaxs_bec/scans/omny_fermat_scan.py`, offset drawn at
~line 319) has the same `seed` argument and the same `offset = ...uniform(0, step)` line, so the
change ports directly:
1. Add `randomize_offset: Annotated[bool, ScanArgument(...)] = True` to `OmnyFermatScan.__init__`,
with docstring entry, store it and pass it through in `prepare_scan()`.
2. Add `randomize_offset: bool = True` to `get_omny_fermat_spiral_pos` and use
`0.0` instead of the random draw when it is `False`.
3. Mirror the two Flomni tests in `tests/tests_scans/test_omny_fermat_scan.py`.
4. Add the parameter to the parameter table in `docs/user/ptychography/omny.md` (~line 322-339;
the omny table has no `fermat_asymmetry` row yet either, see
`TODO_fermat_asymmetry_port.md`).
Scope notes:
- The OMNY tomo loop (`omny.py`, `scan_kwargs` in the per-angle scan call) does not need to pass it;
as on Flomni, tomo scans keep the default (randomized).
- Compose with `fermat_asymmetry` as described in `TODO_fermat_asymmetry_port.md`: `randomize_offset=False`
only sets `offset = 0.0`; the offset-subtraction -> inflated check -> compression ordering stays as is.
- LamNI has the identical offset code (`lamni_fermat_scan.py:323`) but was not asked for this switch.
@@ -0,0 +1,44 @@
# TODO: OMNY is missing the scilog_last_ptycho_scans command (and its timing-log infrastructure) that flomni/LamNI have
Found while adding a step-size field to flomni's and LamNI's
`scilog_last_ptycho_scans` per-scan summary line (branch
`fixes/flomni_beamtime_2`). OMNY has no equivalent at all, not even the
older/pre-step-size version — porting it is a bigger job than the two-line
fixes just made to flomni/lamni, so it's flagged here instead of attempted
opportunistically.
## 1. scilog_last_ptycho_scans + timing-log infrastructure (new feature, not yet on OMNY at all)
flomni (`flomni.py`) and LamNI (`lamni.py`) both have a full
timing-instrumented pipeline behind `scilog_last_ptycho_scans(number_of_scans=1)`:
- `_TIMING_LOG_DIR`/`_PROJECTION_TIMING_LOG` — an append-only JSONL log of
per-projection timing records, one written per completed projection via
`_log_projection_timing(...)` (flomni.py:3180, lamni.py:966), called from
each setup's per-angle tomo-scan loop (flomni.py:2578/3502, lamni.py:1611).
- `_read_last_timing_records(number_of_scans)` (flomni.py:3265, lamni.py:1051)
— reads back the last N records from that log.
- `scilog_last_ptycho_scans(number_of_scans=1)` (flomni.py:3291, lamni.py:1077)
— prompts for a free-text comment, builds one summary line per record (scan
number(s), FOV, step size, exposure, duration), and sends it via each
setup's tolerant scilog-write helper (`Flomni._scilog_write`, flomni.py:2993;
`LamNI.write_to_scilog`, lamni.py:928).
OMNY (`omny.py`) only has the much older `write_to_scilog(content, tags=None)`
(omny.py:855) and `_write_subtomo_to_scilog(subtomo_number)` (omny.py:916) — a
per-subtomo one-liner, not a "last N scans" summary command — and no timing
log, no duration measurement, and no `scilog_last_ptycho_scans` at all.
`OMNY.tomo_scan()`/`_tomo_scan_at_angle()` (omny.py:1005/966) would be the
natural hook point for a `_log_projection_timing`-style call, mirroring where
flomni/lamni call theirs.
Porting this means: adding the timing-log write call to OMNY's per-angle
loop, then copying `_read_last_timing_records`/`scilog_last_ptycho_scans`
across (with OMNY-specific field names in place of `fovx`/`fovy` or
`lamni_piezo_range_x`/`y` — whatever OMNY's equivalent scan-size parameter is
called). Remember to include the `step {value} um` field in the per-scan
line from the start (already added to flomni's and lamni's versions, keyed
off `tomo_shellstep`) rather than needing a second follow-up fix like this
one.
Not scoped/designed further here — just flagging it so it isn't lost.
@@ -0,0 +1,62 @@
# TODO: tighten tomo-parameter ranges + enforce them in the CLI wizard
Status: **not implemented for omny.** Discussed and implemented for Flomni
and LamNI on 2026-09-19 — this note exists so the same pass isn't forgotten
for omny.
## What was done for Flomni/LamNI
Two separate things, both worth doing here too:
1. **Tightened several parameter ranges** that had grown far looser than
practically useful (GUI spinbox ranges in `tomo_params.py`, plus the
matching property/CLI bounds where they existed):
- `tomo_shellstep`: 0.0011000.0 → **0.02520.0 µm**
- `stitch_x`/`stitch_y`: 050 → **010** (flomni only — lamni's own
`lamni_stitch_x`/`y` were left at 050)
- `golden_ratio_bunch_size`: 110000 → **1100**
- `tomo_stitch_overlap`: 0.050.0 → **0.010.0 µm**
- `manual_shift_y` (flomni): -10001000 → **-5050 µm**
- lamni-specific: `tomo_circfov` 0.0200.0 → **0.0500.0 µm** (floor kept
at 0.0 — `fov_circular == 0` means "disabled", checked explicitly in
`lamni_fermat_scan.py:484`, so don't raise the floor above 0);
`lamni_piezo_range_x`/`y` 0.0200.0 → **0.080.0 µm**, matching the
"(max 80)" already printed in lamni's CLI prompt text.
2. **`_get_val()` (the `tomo_parameters()` CLI wizard's input helper) now
supports `min_`/`max_`** and re-prompts until the value is in range,
instead of accepting anything of the right type with zero validation.
Both Flomni's and LamNI's `_get_val()` got this treatment (they're
separate static methods, not shared code) and every prompted field that
has a GUI-defined range got the matching `min_`/`max_` wired in.
## What's needed for omny
Omny has its own `tomo_parameters()` and `_get_val()`
(`csaxs_bec/bec_ipython_client/plugins/omny/omny.py:1293` /
`omny.py:1422`) — same bare/unvalidated pattern Flomni and LamNI both had
before this pass. Confirmed (2026-09-19) that omny's parameter names mirror
**flomni's** naming exactly, not lamni's: `fovx`/`fovy`, `manual_shift_y`,
`tomo_type`, `stitch_x`/`stitch_y`, `tomo_stitch_overlap`,
`golden_ratio_bunch_size`, `tomo_shellstep`, `tomo_countingtime` all exist
under those same names in `omny.py`. So the flomni numbers above likely
carry over directly, but this hasn't been checked against omny's own
physical setup/hardware constraints (e.g. its FOV/shellstep/piezo range may
have different practical limits than flomni's cryo ptycho setup) — don't
just copy the numbers blind.
Steps, mirroring what was done for flomni/lamni:
1. Confirmed (2026-09-19): `tomo_params.py`'s `SETUP_PROFILES` dict has only
`"flomni"` and `"lamni"` entries — **no `"omny"` profile exists**, so
omny doesn't use this shared Tomo Parameters GUI widget at all right now.
Whatever GUI (if any) omny actually uses for entering these parameters
needs to be located first; there's nothing to tighten in `tomo_params.py`
for omny today. (Separately, `omny_webpage_generator.py` was confirmed in
an earlier session to be dead/unwired code — not related to this GUI
question, just don't confuse the two when investigating.)
2. Add `min_`/`max_` support to omny's own `_get_val()` (same mechanism,
copy-pasted independently — these are three separate static methods, not
shared code, by design in this codebase).
3. Confirm with whoever runs omny scans what the actual practical ranges
should be for omny's hardware, rather than assuming flomni's numbers
apply unchanged.
@@ -0,0 +1,58 @@
# TODO: port "always define `tomo_alignment_fit`" to OMNY now that it's landed on Flomni
**Status: implemented on Flomni** (commit `5b688df`, branch
`fixes/flomni_beamtime_2`), not yet ported to OMNY.
## Problem (as seen on Flomni)
The `tomo_alignment_fit` global var was deleted by `reset_tomo_alignment_fit()`
(on stage-in and at the start of a new alignment) and never created at
startup. Manual offset tweaks from the client failed with an undefined
variable:
```python
fit = flomni.client.get_global_var("tomo_alignment_fit") # -> None
fit[0][2] += 5.0 # x offset, microns -> TypeError
```
## What Flomni does now (`flomni/flomni.py`)
- `reset_tomo_alignment_fit()` writes `np.zeros((2, 5)).tolist()` instead of
calling `delete_global_var`, so the var is always defined.
- New `ensure_tomo_alignment_fit()` sets the zero fit **only if**
`get_global_var("tomo_alignment_fit") is None`. It is called from
`Flomni.__init__`, so an existing fit survives a client restart.
- `get_alignment_offset()` is unchanged; with zeros it prints zero offsets
instead of the "Not applying any alignment offsets" message. Same result.
Fit layout (row 0 = x, row 1 = y):
```
# x amp, phase, offset, y amp, phase, offset, 3rd order amp, 3rd order phase
# 0 0 0 1 0 2 1 0 1 1 1 2 1 3 1 4
```
## What to do on OMNY
`omny/omny_alignment_mixin.py` still has the old behaviour:
- `reset_tomo_alignment_fit()` (~line 57) does
`self.client.delete_global_var("tomo_alignment_fit")`.
- Nothing defines the var at startup. `OMNY.__init__` is in `omny/omny.py`
(~line 557).
Steps:
1. Change `reset_tomo_alignment_fit()` to write `np.zeros((2, 5)).tolist()`.
2. Add `ensure_tomo_alignment_fit()` exactly as on Flomni.
3. Call it from `OMNY.__init__` (after `self.client` is set).
4. Check the callers: at the time of writing, nothing in `omny/` calls
`reset_tomo_alignment_fit()` apart from its definition, so decide where
OMNY should reset the fit (start of an alignment / sample transfer),
mirroring Flomni's `ftransfer_flomni_stage_in()` and
`XrayEyeAlign._align_impl()`.
5. `read_alignment_offset()` in the OMNY mixin builds the fit as `np.zeros((2, 5))`
and stores it with `set_global_var`, so it needs no change.
LamNI does not reference `tomo_alignment_fit` at all (checked by grep), so no
action there.
@@ -0,0 +1,38 @@
# TODO: OMNY has no end-of-scan timing summary at all, so it's also missing the new tomo_print_timing_report() commands
Found while adding `tomo_print_timing_report()` / `tomo_print_timing_report_and_scilog()`
to flomni and LamNI (branch `fixes/flomni_beamtime_2`) so the elapsed/idle
timing breakdown that normally only prints at the end of a *completed*
`tomo_scan()` can also be pulled on demand -- in particular right after
interrupting a scan with Ctrl-C, since the underlying `self.progress` data
(a BEC global var) survives the interrupt even though the old inlined
end-of-scan print block did not run.
## 1. No "Tomoscan finished" timing block on OMNY at all (pre-existing gap)
flomni (`flomni.py`) and LamNI (`lamni.py`) both print an elapsed/idle/
excl.-idle timing breakdown at the tail of `tomo_scan()`, built from
`self.progress` (`tomo_start_time`, `accumulated_idle_time`) via a shared
`_build_timing_lines()` helper (flomni.py, near line 2984; lamni.py, near
line 411). `OMNY.tomo_scan()` (omny.py:1005) has no equivalent block at
all -- it was added to flomni/lamni after the flomni-to-lamni migration and
was never back-ported to OMNY.
## 2. New on-demand commands also missing
Both setups now also expose:
- `tomo_print_timing_report()` -- prints the current timing breakdown to
console at any time (mid-scan, after an interrupt, or after completion),
no scilog write.
- `tomo_print_timing_report_and_scilog()` -- same, plus writes the report to
scilog.
If OMNY is still in active use, porting this means: adding an
`_ProgressProxy`-equivalent `tomo_start_time`/`accumulated_idle_time`
tracking (OMNY may already track some of this differently -- check before
assuming it needs the same proxy), then copying
`_build_timing_lines()`/`tomo_print_timing_report()`/
`tomo_print_timing_report_and_scilog()` across with OMNY's own
`write_to_scilog`/`_scilog_write` equivalent (omny.py:855).
Not scoped/designed further here -- just flagging it so it isn't lost.
@@ -0,0 +1,37 @@
# Note: X-ray-eye DAP sine fit (start values) -- what omny inherits, and what is still open
Status: **nothing to fix in omny code for the fit itself; recorded 2026-09-20 so it isn't rediscovered.**
## What was fixed (flomni, 2026-09-20)
The GUI fit tab (`OMNY_XRayEye` in `csaxs_bec/bec_widgets/widgets/xray_eye/x_ray_eye.py`) fits
`A*sin(f*x + shift) + intercept` through the DAP server with `f` fixed to 1 deg. The DAP server
used lmfit's `SineModel.guess()` for the start values. For flomni's 5 points over only 0..180 deg
that FFT-based guess is arbitrary and regularly left `shift` stuck at its bound (2*pi), far from
the data (plotted fit missed the submitted points; wrong `X Phase` loaded into flomni).
Fix: `SINE_FIT_DAP_PARAMETERS` (same file) overrides all five parameters (signed amplitude, start
A=30 / shift=0 within +-pi / intercept=0), so DAP skips `guess()`. 5-pt/0..180 deg: 0 failures in 300
random datasets through the real `LmfitService1D` (old parameters: 16 of 300).
## Consequences for omny
- The widget is shared by flomni, LamNI and omny, so omny's GUI fit tab already uses the new
parameters. Checked with 8 points over 0..315 deg (omny/LamNI layout): 0 of 300 failures both
with the old and the new parameters, i.e. the 8-point full-period case was never affected and
is not made worse. No omny change required.
- The fitted parameters use the same form `A*sin(radians(angle) + phase) + offset` in flomni,
LamNI and omny consumers, so the (now possibly negative) amplitude with phase in [-pi, pi] is
the same curve as before. Nothing downstream needs a sign fix.
- Cosmetic: a row of all zeros (flomni's y row) now fits to ~1e-6 noise instead of exactly 0;
the y plot autoscales that to a curved line at the 1e-6 level. Harmless (flomni overrides y
with zeros in `read_alignment_offset`).
## Open for omny
- `omny/x_ray_eye_align.py` is still a verbatim copy of LamNI's (class docstring "for LamNI",
uses `self.lamni.*` and `self.lamni.align.read_xray_eye_correction_from_gui()`), while
`omny/omny_alignment_mixin.py` only reads the Matlab fit from `ptychotomoalign_*.txt`
(`read_alignment_offset`) and has no `read_xray_eye_correction_from_gui`. So omny currently
does not consume the GUI fit at all. When omny's alignment is properly ported, decide whether
it should use the GUI fit (then this DAP fix is what it relies on) or keep the Matlab files.
@@ -0,0 +1,40 @@
# TODO: unconditional 0-degree reference projection at tomogram start
Status: **not implemented for omny.** Discussed 2026-09-19; implemented for
Flomni and LamNI the same day (see their `tomo_scan()` in `flomni.py`/
`lamni.py` — search for "Unconditional single 0-degree reference
projection").
## What was implemented elsewhere
At the very start of a new tomogram (not on resume), both Flomni and LamNI
now unconditionally take one real 0-degree projection — `self._tomo_scan_at_angle(0, 1)`
— independent of the `zero_deg_reference_at_each_subtomo` /
`golden_projections_at_0_deg_for_damage_estimation` flags. It's real data
(included in reconstruction, logged under `subtomo_number=1`), not a
reference-only/flat-field frame — this was an explicit choice: the point is
an immediate radiation-damage baseline that exists even when the periodic
damage-estimation flags are off, not a normalization frame.
Flomni additionally takes 10 flat-field/empty frames at 0° before this
(`collect_empty_frames()` — sample retracted out of the beam via `fsamx`
shifted by `fovx/2`, excluded from reconstruction). **LamNI does not get an
equivalent** — confirmed 2026-09-19 that LamNI has no way to retract the
sample out of the beam at all, so only the 0-degree projection was added
there, no flat-field step.
## What's needed for omny
1. **Confirm whether omny can retract the sample out of the beam at all**
(same open question as LamNI, not yet checked for omny specifically) —
determines whether an empty-frames/flat-field step is even possible here,
or whether omny should only get the 0-degree-projection part like LamNI.
2. Locate omny's `tomo_scan()` equivalent (or whatever its tomogram-start
entry point is) and its own `_tomo_scan_at_angle`-equivalent method, and
add the same unconditional call in its "new scan" branch, following the
same placement pattern used in `flomni.py`/`lamni.py`.
3. Decide the logged `subtomo_number` / whether omny's tomo-type numbering
matches flomni/lamni's convention closely enough to reuse `1` directly.
Not investigated yet — this note exists so the intent isn't lost, not
because the omny-side work has been scoped in detail.
@@ -85,7 +85,10 @@ elif _args.session.lower() == "flomni":
from csaxs_bec.bec_ipython_client.plugins.flomni import Flomni
_session_name = "flomni"
flomni = Flomni(bec)
if "fttrx1" in dev:
flomni = Flomni(bec, csaxs=csaxs)
else:
flomni = Flomni(bec)
logger.success("flomni session loaded.")
print(r"""
██████╗ ███████╗ ██████╗ ███████╗██╗ ██████╗ ███╗ ███╗███╗ ██╗██╗
+38
View File
@@ -194,6 +194,19 @@ class OMNY_XRayEye(RPCBase):
enable(int): -1 disable else enable
"""
@rpc_timeout(20)
@rpc_call
def enable_skip_button(self, enable: "bool"):
"""
Allow/forbid skipping the current alignment angle.
The first call also shows the button (it is hidden for clients that never call this).
The button is only clickable while this is True *and* the submit button is enabled,
so it follows the client's "please wait ..." phases automatically.
Args:
enable(bool): True if the client accepts a skip for the current step
"""
@property
@rpc_call
def sample_name(self):
@@ -257,6 +270,31 @@ class OMNY_XRayEye(RPCBase):
Hide the alignment target crosshair on the image view.
"""
@rpc_timeout(20)
@rpc_call
def lock_vertical_center(self):
"""
Snap every ROI's vertical center to the crosshair's fixed y position
(see ``crosshair_position()``), immediately and again on every future
draw/drag/resize, until ``unlock_vertical_center()`` is called. Used by
``x_ray_eye_align.py`` once the sample's height has been physically
centered on the FZP, so subsequent alignment boxes no longer need manual
vertical placement. See ``_snap_roi_vertical()``.
Also settable by the operator directly via the "ROI vertical lock"
toggle in the control panel (``_on_vertical_lock_toggle_changed``) --
e.g. to temporarily release a script-engaged lock mid-run in order to
freely draw/measure an unrelated box, then re-engage it. Both paths
keep the toggle's visual state in sync with ``_vertical_lock_enabled``.
"""
@rpc_timeout(20)
@rpc_call
def unlock_vertical_center(self):
"""
Stop auto-snapping ROIs' vertical center -- see ``lock_vertical_center()``.
"""
@property
@rpc_call
def crosshair_visible(self) -> "bool":
@@ -2,6 +2,8 @@ from __future__ import annotations
import os
import signal
import threading
import time
from bec_lib import bec_logger
from bec_widgets import BECWidget, SafeProperty, SafeSlot
@@ -26,11 +28,13 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
let an unrelated, un-suppressed device error kill the scan worker
thread and require a full BEC restart), then sends a real SIGINT to
the BEC IPython client process (the parent of the GUI server
process), then directly calls ``hard_stop_device_name``'s
``.controller.hard_abort_and_restore_positioning_mode()`` -- the same
Galil hard-stop (XQ#STOP,1) the CLI's ``ftransfer_abort()`` uses --
and, if configured, ``extra_hard_stop_device_name``'s
``.controller.stop_all_axes()``.
process), then sends a stop (``controller.stop_all_axes()``, i.e.
``XQ#STOP,1``) to the Galil controller of EVERY device listed in
``hard_stop_device_names`` -- one device per physical controller, in
that order (put the sample-transfer controller first) -- and finally
restores positioning mode on the FIRST listed controller via
``.controller.hard_abort_and_restore_positioning_mode()`` (the same
Galil hard-stop the CLI's ``ftransfer_abort()`` uses).
SIGINT is sent BEFORE the hard stop, not after: if a client-side loop
is actively polling this same transfer (e.g. ftransfer_get_sample's
@@ -41,41 +45,38 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
``ensure_gripper_up()``, which must not run mid-transfer. Sending
SIGINT first (an os.kill() call, effectively instant) gives the
target process's own ``except KeyboardInterrupt: ftransfer_abort();
raise FlomniError(...)`` handler a head start over the hard stop's
own multi-step sequence (which itself takes hundreds of ms due to
its internal polling/settling waits) -- without adding any
artificial delay of its own, so the actual motion stop is still
effectively immediate.
raise FlomniError(...)`` handler a head start over the hard stop.
``hard_stop_device_name`` names the device whose ``.controller`` the
hard stop is called on (e.g. ``"ftransy"`` for flomni). Without a device
*name* configured, the button is disabled rather than silently doing
nothing. A configured name that doesn't actually resolve to a device
(e.g. missing from the current session's config) is not checked ahead
of time -- it's re-resolved at click time in ``_on_abort()``, which
already logs and no-ops on failure -- so the button being enabled is
not a guarantee the device exists, only that a target was configured.
``hard_stop_device_names`` is a list of device names, one per Galil
controller (flomni: ``["ftransy", "foptx", "fsamroy"]`` -- the transfer
stages' controller, the optics controller and the rotation controller).
The legacy kwargs ``hard_stop_device_name`` / ``extra_hard_stop_device_name``
are still accepted and simply form the list ``[first, extra]``. With no
names configured, the button is disabled rather than silently doing
nothing. Names are NOT checked when the widget is built (that live check
proved unreliable, see git history), but they are resolved freshly from
the GUI client's device manager at click time, with a few retries, and
every outcome -- including "device could not be resolved" -- is logged
and shown in the widget's message label, so a controller that was not
stopped is never silent.
``extra_hard_stop_device_name`` optionally names a second device on a
physically separate Galil controller (e.g. ``"foptx"`` for flomni's
optics stage, a different board/socket port from the transfer stage
``ftransy`` sits on -- one abort press only halts axes on ``ftransy``'s
controller otherwise). Its ``.controller.stop_all_axes()`` is called
instead of ``hard_abort_and_restore_positioning_mode()``: the latter's
``#POSMODE``/``mntmod`` handling is specific to the sample-transfer/
mount program that only runs on the transfer controller, so
``stop_all_axes()`` (a plain ``XQ#STOP,1``) is the correct generic
"halt everything on this controller" primitive for any other board.
Entirely optional -- unset means only ``hard_stop_device_name`` is
stopped, as before.
The stops are emitted from one thread per controller, started in the
configured order, so a hung RPC to one controller cannot delay the
others. If the client-side proxy does not (yet) expose
``stop_all_axes`` (the device server publishes exposed methods when it
starts), the raw command ``XQ#STOP,1`` is sent through the always
exposed ``socket_put_and_receive`` instead.
"""
USER_ACCESS = ["message", "message.setter", "response", "clear_response"]
PLUGIN = True
def __init__(self, parent=None, **kwargs):
self._hard_stop_device_name = kwargs.pop("hard_stop_device_name", None)
self._extra_hard_stop_device_name = kwargs.pop("extra_hard_stop_device_name", None)
self._hard_stop_device_names = self._collect_hard_stop_names(
kwargs.pop("hard_stop_device_names", None),
kwargs.pop("hard_stop_device_name", None),
kwargs.pop("extra_hard_stop_device_name", None),
)
self._hard_stop_label = kwargs.pop("hard_stop_label", "Motion Stop")
super().__init__(parent=parent, **kwargs)
self._response = ""
@@ -87,22 +88,39 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
self._client_pid = os.getppid()
self._init_ui()
def _hard_stop_available(self) -> bool:
"""True if a hard_stop_device_name was configured for this widget.
# A fresh lookup at click time is retried a few times: a lookup can fail
# for a moment while the GUI client's device manager is being (re)filled.
_RESOLVE_ATTEMPTS = 3
_RESOLVE_DELAY_S = 0.05
# Upper bound for waiting on the per-controller stop threads.
_STOP_JOIN_TIMEOUT_S = 10.0
Deliberately does not check whether the name actually resolves to a
live, enabled device: that live check (a getattr(self.dev, name)
device-manager lookup) proved unreliable in practice -- a
confirmed-present, confirmed-enabled device could still evaluate as
unavailable here, permanently disabling the button for the widget's
lifetime with no way to recover short of recreating the dock. Since
whatever device is named is re-resolved with its own try/except in
_on_abort() anyway, a stale or wrong name still fails safely
(logged, no-op) instead of silently doing nothing -- so skipping the
redundant, unreliable pre-check here only removes a false-negative
failure mode, not real safety.
@staticmethod
def _collect_hard_stop_names(names, first=None, extra=None) -> list[str]:
"""Ordered, de-duplicated list of the devices whose controllers get a
stop. ``names`` (the ``hard_stop_device_names`` kwarg) wins; without
it the legacy pair ``[first, extra]`` is used."""
if isinstance(names, str):
names = [names]
candidates = list(names) if names else [first, extra]
result: list[str] = []
for name in candidates:
if name and name not in result:
result.append(name)
return result
def _hard_stop_available(self) -> bool:
"""True if at least one hard-stop device name was configured.
Deliberately does not check whether the names actually resolve to
live devices: that live check (a getattr(self.dev, name)
device-manager lookup at construction time) proved unreliable in
practice -- a confirmed-present, confirmed-enabled device could still
evaluate as unavailable, permanently disabling the button for the
widget's lifetime. Names are resolved at click time instead, and an
unresolvable name is reported loudly (log + message label).
"""
return bool(self._hard_stop_device_name)
return bool(self._hard_stop_device_names)
def _init_ui(self):
layout = QVBoxLayout(self)
@@ -170,6 +188,9 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
@SafeSlot()
def _on_abort(self):
if not self._hard_stop_device_names:
return # button is disabled in this case; defensive only
# 1) Coordinated abort first: a safe no-op if nothing is queued: when
# something IS queue-tracked, this registers a real stop_id so the
# device server suppresses the resulting error instead of killing
@@ -190,34 +211,113 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
logger.warning(f"z_ConsoleButtonsWidget: sending SIGINT to client pid {self._client_pid}")
os.kill(self._client_pid, signal.SIGINT)
# 3) Hard motion stop(s) -- not delayed by anything above: sending
# SIGINT is a near-instant os.kill() call, so these still run
# effectively immediately. The two devices are independent
# controllers (see class docstring): each is attempted and
# logged on its own, so a failure on one doesn't skip the other.
if self._hard_stop_device_name:
self._stop_device_controller(
self._hard_stop_device_name, "hard_abort_and_restore_positioning_mode"
)
if self._extra_hard_stop_device_name:
self._stop_device_controller(self._extra_hard_stop_device_name, "stop_all_axes")
# 3) Stop every Galil controller, in the configured order (transfer
# controller first), then 4) restore positioning mode on the first
# one -- last, because it takes >= 0.5 s and must not delay the
# other stops. Nothing here raises: every failure is logged and
# shown in the message label.
stop_results = self._stop_all_controllers()
parts = [
f"{name}: OK" if ok else f"{name}: NOT STOPPED ({detail})"
for name, ok, detail in stop_results
]
text = "Motion stop sent -> " + " | ".join(parts)
if not all(ok for _, ok, _ in stop_results):
text += "\nCHECK THE STAGES MANUALLY!"
self.message_label.setText(text)
def _stop_device_controller(self, device_name: str, method_name: str) -> None:
"""Resolve device_name and call method_name() on its .controller,
logging and swallowing any failure -- so a missing device or a
failed stop on one controller can't prevent an already-issued stop
on another, or crash the rest of _on_abort()."""
restore_ok, restore_detail = self._restore_positioning_mode(self._hard_stop_device_names[0])
self.message_label.setText(
text + f"\nPositioning mode: {'restored' if restore_ok else 'NOT restored'}"
+ ("" if restore_ok else f" ({restore_detail})")
)
def _resolve_controller(self, device_name: str):
"""Fresh lookup of ``device_name`` in the GUI client's device manager
(not the ``self.dev`` reference cached at widget construction), with a
few retries. Returns the device's ``.controller`` RPC proxy or raises
RuntimeError with the reason."""
reason = "unknown"
for attempt in range(self._RESOLVE_ATTEMPTS):
try:
device = self.client.device_manager.devices.get(device_name)
if device is not None:
return device.controller
reason = "device not found in the GUI client's device manager"
except Exception as exc: # DeviceConfigError etc. are not AttributeErrors
reason = f"{type(exc).__name__}: {exc}"
if attempt < self._RESOLVE_ATTEMPTS - 1:
time.sleep(self._RESOLVE_DELAY_S)
raise RuntimeError(reason)
def _stop_controller(self, device_name: str) -> tuple[bool, str]:
"""Send a stop to the Galil controller behind ``device_name``.
Returns (sent, detail); never raises."""
try:
device = getattr(self.dev, device_name, None)
except Exception:
device = None
if device is None:
return
logger.warning(f"z_ConsoleButtonsWidget: hard-stopping {device_name} ({method_name})")
controller = self._resolve_controller(device_name)
except Exception as exc:
logger.error(f"z_ConsoleButtonsWidget: cannot resolve {device_name}: {exc}")
return False, f"not resolved: {exc}"
logger.warning(f"z_ConsoleButtonsWidget: stopping all axes on {device_name}'s controller")
try:
getattr(device.controller, method_name)()
except Exception:
logger.exception(f"z_ConsoleButtonsWidget: hard motion stop of {device_name} failed")
stop_all_axes = getattr(controller, "stop_all_axes", None)
if not callable(stop_all_axes):
raise AttributeError("stop_all_axes is not exposed by the device server yet")
reply = stop_all_axes()
logger.warning(f"z_ConsoleButtonsWidget: {device_name} stop_all_axes -> {reply!r}")
return True, str(reply)
except Exception as exc:
logger.warning(
f"z_ConsoleButtonsWidget: stop_all_axes on {device_name} failed ({exc!r}); "
"falling back to raw XQ#STOP,1"
)
try:
reply = controller.socket_put_and_receive("XQ#STOP,1")
logger.warning(f"z_ConsoleButtonsWidget: {device_name} raw XQ#STOP,1 -> {reply!r}")
return True, str(reply)
except Exception as exc:
logger.exception(f"z_ConsoleButtonsWidget: stopping {device_name} failed")
return False, f"stop failed: {exc}"
def _stop_all_controllers(self) -> list[tuple[str, bool, str]]:
"""Stop the controllers of all configured devices. One thread per
controller, started in the configured order, so a hung RPC to one
controller cannot delay the others. Returns (name, sent, detail) in
the configured order."""
results: dict[str, tuple[bool, str]] = {}
def worker(name: str) -> None:
results[name] = self._stop_controller(name)
threads = []
for name in self._hard_stop_device_names:
thread = threading.Thread(target=worker, args=(name,), daemon=True)
thread.start()
threads.append(thread)
deadline = time.monotonic() + self._STOP_JOIN_TIMEOUT_S
for thread in threads:
thread.join(max(0.0, deadline - time.monotonic()))
out = []
for name in self._hard_stop_device_names:
ok, detail = results.get(name, (False, f"no answer within {self._STOP_JOIN_TIMEOUT_S} s"))
out.append((name, ok, detail))
return out
def _restore_positioning_mode(self, device_name: str) -> tuple[bool, str]:
"""Wait for the transfer routine to halt and switch the controller
behind ``device_name`` back to positioning mode. Never raises."""
try:
controller = self._resolve_controller(device_name)
logger.warning(
f"z_ConsoleButtonsWidget: restoring positioning mode on {device_name}'s controller"
)
controller.hard_abort_and_restore_positioning_mode()
return True, ""
except Exception as exc:
logger.exception(
f"z_ConsoleButtonsWidget: restoring positioning mode on {device_name} failed"
)
return False, str(exc)
@SafeProperty(str)
def message(self):
@@ -797,9 +797,9 @@ class OMNY_SampleStorage(BECWidget, QWidget):
marker = ""
if measured_status == "completed":
marker = "✓ measured"
marker = "measured"
elif measured_status == "started":
marker = "◐ started"
marker = "started"
pdf.set_xy(x + 1, y + h - 6)
pdf.set_font("Helvetica", "", 7)
pdf.cell(w - 2, 4, marker, align="C")
@@ -127,6 +127,7 @@ _QUEUE_IDLE_STATUSES = ("STOPPED", "COMPLETED", "IDLE")
QUEUE_PARAM_NAMES = (
"tomo_countingtime",
"tomo_shellstep",
"fermat_asymmetry",
"fovx",
"fovy",
"stitch_x",
@@ -151,6 +152,7 @@ QUEUE_PARAM_NAMES = (
DEFAULTS: dict[str, Any] = {
"tomo_countingtime": 0.1,
"tomo_shellstep": 1.0,
"fermat_asymmetry": 1.0,
"fovx": 20.0,
"fovy": 20.0,
"stitch_x": 0,
@@ -455,7 +457,7 @@ class OMNY_TomoParams(BECWidget, QWidget):
"tomo_countingtime": lambda: self._add_double(
common_form,
"tomo_countingtime",
"Counting time (s)",
"Counting time per frame (s)",
min_=0.001,
max_=100.0,
decimals=3,
@@ -464,10 +466,11 @@ class OMNY_TomoParams(BECWidget, QWidget):
common_form,
"tomo_shellstep",
"Shell step (µm)",
min_=0.001,
max_=1000.0,
min_=0.025,
max_=20.0,
decimals=3,
),
"fermat_asymmetry": lambda: self._build_fermat_asymmetry_row(common_form),
"fov": lambda: self._build_double_group(common_form, self._profile["fov_fields"]),
"stitch": lambda: self._build_int_group(common_form, self._profile["stitch_fields"]),
"tomo_stitch_overlap": lambda: self._add_double(
@@ -475,7 +478,7 @@ class OMNY_TomoParams(BECWidget, QWidget):
"tomo_stitch_overlap",
"Stitch overlap (µm)",
min_=0.0,
max_=50.0,
max_=10.0,
decimals=3,
),
"ptycho_reconstruct_foldername": lambda: self._add_text(
@@ -586,6 +589,39 @@ class OMNY_TomoParams(BECWidget, QWidget):
"constant shift that will not rotate.",
)
def _build_fermat_asymmetry_row(self, form: QFormLayout) -> None:
"""Fermat asymmetry (x/y spacing ratio) field, with a "?" help button
next to it -- same pattern as the offsets section header
(_build_offsets_section/_show_offsets_help above)."""
from csaxs_bec.scans.flomni_fermat_scan import FlomniFermatScan
row = QHBoxLayout()
w = QDoubleSpinBox()
w.setDecimals(2)
w.setRange(FlomniFermatScan.MIN_FERMAT_ASYMMETRY, FlomniFermatScan.MAX_FERMAT_ASYMMETRY)
w.setEnabled(False)
self._pw["fermat_asymmetry"] = w
row.addWidget(w, stretch=1)
btn_help = QPushButton("?")
btn_help.setFixedWidth(24)
btn_help.setToolTip("What does Fermat asymmetry do?")
btn_help.clicked.connect(self._show_fermat_asymmetry_help)
row.addWidget(btn_help)
form.addRow("Fermat asymmetry (x/y):", row)
def _show_fermat_asymmetry_help(self) -> None:
QMessageBox.information(
self,
"Fermat asymmetry",
"Ratio of the Fermat spiral's x spacing to its y spacing (x/y). 1 = "
"symmetric (default). x spacing always stays fixed at the shell step; "
"y spacing scales by this ratio -- values above 1 widen x spacing "
"relative to y (denser sampling in y), values below 1 do the "
"opposite. The requested field of view is preserved either way, so "
"the total number of scan points changes with the ratio (see the "
"'Estimated Fermat scan points' preview below).",
)
def _build_at_each_angle_hook_row(self, form: QFormLayout) -> None:
self._pw["at_each_angle_hook"] = QComboBox()
self._pw["at_each_angle_hook"].setEnabled(False)
@@ -641,7 +677,7 @@ class OMNY_TomoParams(BECWidget, QWidget):
form, "_numprj_type3", "Projections per sub-tomo (type 3)", min_=1, max_=10000
)
self._add_int(
form, "golden_ratio_bunch_size", "Bunch size (type 2 only)", min_=1, max_=10000
form, "golden_ratio_bunch_size", "Bunch size (type 2 only)", min_=1, max_=100
)
self._add_double(
form,
@@ -2173,6 +2209,11 @@ def _compute_fermat_positions_flomni(params: dict[str, Any]) -> tuple[int, int]:
doesn't affect flomni's count either -- every stitched tile is an
identically-shaped Fermat scan, just centered differently.
The real scan also draws a fresh random sub-step offset on every call
(see get_flomni_fermat_spiral_pos()'s `seed` argument), which can shift
the actual scan-time count by a point or two near the threshold. A
fixed seed is used here only so this estimate is itself reproducible.
Returns:
(estimated_count, minimum_required)
"""
@@ -2181,10 +2222,18 @@ def _compute_fermat_positions_flomni(params: dict[str, Any]) -> tuple[int, int]:
fovx = params.get("fovx", 0.0)
fovy = params.get("fovy", 0.0)
step = params.get("tomo_shellstep", 0.0)
fermat_asymmetry = params.get("fermat_asymmetry", 1.0)
if step <= 0:
return 0, FlomniFermatScan.MIN_POSITIONS
positions = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-abs(fovx / 2), abs(fovx / 2), -abs(fovy / 2), abs(fovy / 2), step=step, spiral_type=0
-abs(fovx / 2),
abs(fovx / 2),
-abs(fovy / 2),
abs(fovy / 2),
step=step,
spiral_type=0,
seed=0,
fermat_asymmetry=fermat_asymmetry,
)
return len(positions), FlomniFermatScan.MIN_POSITIONS
@@ -2198,6 +2247,12 @@ def _compute_fermat_positions_lamni(params: dict[str, Any]) -> tuple[int, int]:
no angle field in this widget -- the real per-projection count can vary
a bit with angle, since the circular crop is angle-dependent).
The real scan also draws a fresh random sub-step offset on every call
(see get_lamni_fermat_spiral_pos()'s `seed` argument), adding a further
small amount of per-projection count variation on top of the
angle/stitch dependence above. A fixed seed is used here only so this
estimate is itself reproducible.
Returns:
(estimated_count, minimum_required)
"""
@@ -2222,6 +2277,7 @@ def _compute_fermat_positions_lamni(params: dict[str, Any]) -> tuple[int, int]:
fovx=piezo_x,
fovy=piezo_y,
fov_circular=params.get("tomo_circfov", 0.0),
seed=0,
)
return len(positions), LamniFermatScan.MIN_POSITIONS
@@ -2267,11 +2323,11 @@ SETUP_PROFILES: dict[str, dict[str, Any]] = {
"has_single_point": True,
"has_zero_deg_reference": True,
"fov_fields": [
("fovx", "FOV x (µm)", 0.1, 200.0, 2),
("fovy", "FOV y (µm)", 0.1, 100.0, 2),
("fovx", "FOV x (µm)", 0.1, 220.0, 2),
("fovy", "FOV y (µm)", 0.1, 99.9, 2),
],
"stitch_fields": [("stitch_x", "Stitch x", 0, 50), ("stitch_y", "Stitch y", 0, 50)],
"manual_shift_fields": [("manual_shift_y", "Manual shift y (µm)", -1000.0, 1000.0, 3)],
"stitch_fields": [("stitch_x", "Stitch x", 0, 10), ("stitch_y", "Stitch y", 0, 10)],
"manual_shift_fields": [("manual_shift_y", "Manual shift y (µm)", -50.0, 50.0, 3)],
"piezo_range_fields": [],
"offset_fields": [],
"offset_getter": None,
@@ -2280,6 +2336,7 @@ SETUP_PROFILES: dict[str, dict[str, Any]] = {
"field_order": [
"tomo_countingtime",
"tomo_shellstep",
"fermat_asymmetry",
"fov",
"stitch",
"tomo_stitch_overlap",
@@ -2293,8 +2350,8 @@ SETUP_PROFILES: dict[str, dict[str, Any]] = {
"compute_type1": _compute_type1,
"requested_to_stepsize": _requested_to_stepsize,
"compute_fermat_positions": _compute_fermat_positions_flomni,
"fermat_position_fields": ["tomo_shellstep", "fovx", "fovy"],
"sample_name_getter": lambda w: w.dev.flomni_samples.sample_names.sample0.get(),
"fermat_position_fields": ["tomo_shellstep", "fermat_asymmetry", "fovx", "fovy"],
"sample_name_getter": lambda w: w.dev.flomni_samples.sample_names.sample0.get(cached=True),
"cli_hint_name": "flomni",
},
"lamni": {
@@ -2305,7 +2362,7 @@ SETUP_PROFILES: dict[str, dict[str, Any]] = {
"has_180_mode": False,
"has_single_point": False,
"has_zero_deg_reference": True,
"fov_fields": [("tomo_circfov", "Circular FOV (µm, 0 = disabled)", 0.0, 200.0, 2)],
"fov_fields": [("tomo_circfov", "Circular FOV (µm, 0 = disabled)", 0.0, 500.0, 2)],
"stitch_fields": [
("lamni_stitch_x", "Stitch x", 0, 50),
("lamni_stitch_y", "Stitch y", 0, 50),
@@ -2315,8 +2372,8 @@ SETUP_PROFILES: dict[str, dict[str, Any]] = {
("manual_shift_y", "Manual shift y (µm)", -1000.0, 1000.0, 3),
],
"piezo_range_fields": [
("lamni_piezo_range_x", "Piezo range x (µm)", 0.0, 200.0, 2),
("lamni_piezo_range_y", "Piezo range y (µm)", 0.0, 200.0, 2),
("lamni_piezo_range_x", "Piezo range x (µm)", 0.1, 79.9, 2),
("lamni_piezo_range_y", "Piezo range y (µm)", 0.1, 79.9, 2),
],
# tomo_fovx_offset/tomo_fovy_offset: alignment values (LamNIAlignmentMixin),
# not part of _TOMO_SCAN_PARAM_NAMES -- shown/editable here (mirroring
@@ -34,6 +34,26 @@ from qtpy.QtWidgets import (
logger = bec_logger.logger
CAMERA = ("cam_xeye", "image")
# Start values / bounds for the DAP fit A*sin(f*x + shift) + intercept of the
# alignment offsets vs. angle (5 points, 0..180 deg, f fixed to 1 deg in rad).
# All five parameters are given so the DAP server does not use lmfit's
# SineModel.guess(): its FFT-based shift guess is arbitrary for 5 points and
# regularly left the fit stuck at the shift bound (2*pi) far from the data.
# A signed amplitude (SineModel defaults to A >= 0) plus a fixed start
# (A=30, shift=0, intercept=0) converged to the global least-squares optimum
# in 1500/1500 random test datasets; A >= 0 failed in a few percent.
SINE_FIT_DAP_PARAMETERS = [
{
"amplitude": {"value": 30, "vary": True, "min": -1e6, "max": 1e6},
"frequency": {"value": 0.0174533, "vary": False, "min": 0.01, "max": 0.02},
"shift": {"value": 0, "vary": True, "min": -math.pi, "max": math.pi},
},
{
"slope": {"value": 0, "vary": False, "min": 0.0, "max": 0.02},
"intercept": {"value": 0, "vary": True, "min": -1e6, "max": 1e6},
},
]
class TargetCrosshair:
"""
@@ -242,6 +262,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
"on_live_view_enabled",
"on_motors_enable",
"enable_submit_button",
"enable_skip_button",
"sample_name",
"sample_name.setter",
"enable_move_buttons",
@@ -251,6 +272,8 @@ class OMNY_XRayEye(BECWidget, QWidget):
"submit_fit_array",
"show_crosshair",
"hide_crosshair",
"lock_vertical_center",
"unlock_vertical_center",
"crosshair_visible",
"crosshair_visible.setter",
"set_crosshair_position",
@@ -297,6 +320,8 @@ class OMNY_XRayEye(BECWidget, QWidget):
self._init_ui()
self.target_crosshair = TargetCrosshair(self.image.plot_item)
# See lock_vertical_center()/unlock_vertical_center()/_snap_roi_vertical()
self._vertical_lock_enabled = False
self._make_connections()
# Connection to redis endpoints
@@ -325,7 +350,20 @@ class OMNY_XRayEye(BECWidget, QWidget):
)
self.connect_motors()
self.resize(800, 600)
# No self.resize() here: this widget is normally embedded as a dock
# (see gui_tools.py's flomnigui_show_xeyealign()), where the dock
# manager (Qt Advanced Docking System) owns its geometry -- an
# explicit self-resize call fights that, and on a screen too small
# for the assumed geometry (gui_tools.py's hardcoded 2560px-wide
# assumption) corrupts the dock's layout until the operator manually
# moves/resizes the window (forcing Qt to reflow within the actual
# available space) -- confirmed live: reproduces reliably on a
# smaller screen, not on a large one, and only for this widget (the
# only one in this GUI with a self-resize call), never for the
# camera/console docks shown via flomnigui_show_cameras(). The
# standalone `python x_ray_eye.py` harness below already does its
# own win.resize(1000, 800) after construction regardless, so this
# call was never actually needed even for that use case.
QTimer.singleShot(0, self._init_queue_status)
QTimer.singleShot(0, self._init_gui_trigger)
QTimer.singleShot(0, self._init_pixel_clock_options)
@@ -373,12 +411,13 @@ class OMNY_XRayEye(BECWidget, QWidget):
header_row.addWidget(self.live_preview_toggle, 0, Qt.AlignmentFlag.AlignVCenter)
self.control_panel_layout.addLayout(header_row)
# Shutter/camera-running (row 0) and the smear aid's own status
# switches (row 1) share one grid so the toggle columns actually
# line up regardless of each row's label text width -- two
# independent QHBoxLayouts (the previous approach) each size
# themselves from their own labels' widths, so corresponding
# switches in different rows land at different x-positions.
# Shutter/camera-running (row 0), a separator (row 1), the smear
# aid's own status switches (row 2), and the ROI-vertical-lock
# toggle (row 3) share one grid so the toggle columns actually line
# up regardless of each row's label text width -- independent
# QHBoxLayouts per row (the previous approach) each size themselves
# from their own labels' widths, so corresponding switches in
# different rows would land at different x-positions.
self.switch_grid_widget = QWidget(parent=self)
switch_grid = QGridLayout(self.switch_grid_widget)
switch_grid.setContentsMargins(0, 0, 0, 0)
@@ -406,15 +445,30 @@ class OMNY_XRayEye(BECWidget, QWidget):
self.smear_active_toggle.checked = False
self.smear_active_toggle.setEnabled(False) # read-only status, not operator-togglable
# Manual override for the ROI-vertical-lock feature (see
# lock_vertical_center()/unlock_vertical_center()): a script (e.g.
# x_ray_eye_align.py) engages the lock automatically during the
# angle-alignment steps, but an operator may still want to freely
# draw/measure an unrelated box mid-run -- this toggle lets them
# release it without aborting the run, and re-engage it after.
self.vertical_lock_label = QLabel("ROI vertical lock", parent=self)
self.vertical_lock_toggle = ToggleSwitch(parent=self)
self.vertical_lock_toggle.checked = False
self.vertical_lock_toggle.enabled.connect(self._on_vertical_lock_toggle_changed)
_right_vcenter = Qt.AlignmentFlag.AlignRight | Qt.AlignmentFlag.AlignVCenter
switch_grid.addWidget(self.shutter_label, 0, 1, _right_vcenter)
switch_grid.addWidget(self.shutter_toggle, 0, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.camera_running_label, 0, 3, _right_vcenter)
switch_grid.addWidget(self.camera_running_toggle, 0, 4, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.smear_active_label, 1, 1, _right_vcenter)
switch_grid.addWidget(self.smear_active_toggle, 1, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.smear_preview_label, 1, 3, _right_vcenter)
switch_grid.addWidget(self.smear_preview_toggle, 1, 4, Qt.AlignmentFlag.AlignVCenter)
# separator: below shutter/camera-running, above the smear-integration switches
switch_grid.addWidget(self._create_separator(), 1, 0, 1, 5)
switch_grid.addWidget(self.smear_active_label, 2, 1, _right_vcenter)
switch_grid.addWidget(self.smear_active_toggle, 2, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.smear_preview_label, 2, 3, _right_vcenter)
switch_grid.addWidget(self.smear_preview_toggle, 2, 4, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.vertical_lock_label, 3, 1, _right_vcenter)
switch_grid.addWidget(self.vertical_lock_toggle, 3, 2, Qt.AlignmentFlag.AlignVCenter)
self.control_panel_layout.addWidget(self.switch_grid_widget)
@@ -525,10 +579,19 @@ class OMNY_XRayEye(BECWidget, QWidget):
self.step_size.setValue(100)
# Submit button
self.submit_button = QPushButton("Submit", parent=self)
# Skip button: leaves out the current angle of the alignment (flat samples are
# edge-on at 90 deg). Hidden and disabled until a client script opts in via
# enable_skip_button(), so LamNI/omny never see it.
self._skip_enabled = False
self.skip_button = QPushButton("Skip", parent=self)
self.skip_button.setVisible(False)
self.skip_button.setEnabled(False)
self.skip_button.setToolTip("Leave out this angle (at least 4 positions are needed)")
# Add to layout form
step_size_form.addWidget(QLabel("Step Size", parent=self), 0, 0)
step_size_form.addWidget(self.step_size, 0, 1)
step_size_form.addWidget(self.submit_button, 2, 0, 1, 2)
step_size_form.addWidget(self.skip_button, 3, 0, 1, 2)
# Add form to control panel
self.control_panel_layout.addLayout(step_size_form)
@@ -582,10 +645,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
y=[1],
label="fit-x",
dap=["SineModel", "LinearModel"],
dap_parameters=[
{"frequency": {"value": 0.0174533, "vary": False, "min": 0.01, "max": 0.02}},
{"slope": {"value": 0, "vary": False, "min": 0.0, "max": 0.02}},
],
dap_parameters=SINE_FIT_DAP_PARAMETERS,
dap_oversample=5,
)
self.waveform_y.plot(
@@ -593,10 +653,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
y=[2],
label="fit-y",
dap=["SineModel", "LinearModel"],
dap_parameters=[
{"frequency": {"value": 0.0174533, "vary": False, "min": 0.01, "max": 0.02}},
{"slope": {"value": 0, "vary": False, "min": 0.0, "max": 0.02}},
],
dap_parameters=SINE_FIT_DAP_PARAMETERS,
dap_oversample=5,
)
self.fit_x = self.waveform_x.curves[0]
@@ -623,6 +680,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
lambda x: self.motor_control_2d.setProperty("step_size", x)
)
self.submit_button.clicked.connect(self.submit)
self.skip_button.clicked.connect(self.skip)
# ROIPropertyTree's compact_color only styles the line color; line width
# still needs to be forced per-ROI here.
self.roi_manager.controller.roiAdded.connect(self._style_new_roi)
@@ -634,6 +692,33 @@ class OMNY_XRayEye(BECWidget, QWidget):
# Live-update the microns readout while this ROI is drawn/resized.
roi.sigRegionChanged.connect(lambda r=roi: self._update_roi_size_readout(r))
self._update_roi_size_readout(roi)
# If the vertical lock is on (see lock_vertical_center()), snap this box's
# vertical center to the crosshair immediately, and again on every future
# drag/resize.
self._snap_roi_vertical(roi)
roi.sigRegionChangeFinished.connect(lambda r=roi: self._snap_roi_vertical(r))
def _snap_roi_vertical(self, roi):
"""Snap `roi`'s vertical center to the crosshair's fixed y position, keeping
its x position and height/diameter unchanged. No-op unless the vertical lock
is enabled (``lock_vertical_center()``) and the crosshair is visible. Safe to
call from ``roi.sigRegionChangeFinished`` -- if the ROI is already at the
target position this is a no-op, so it cannot loop back into itself.
"""
if not self._vertical_lock_enabled or not self.target_crosshair.is_visible():
return
_, fzp_y = self.crosshair_position()
if isinstance(roi, RectangularROI):
height = roi.get_coordinates(typed=True)["height"]
elif isinstance(roi, CircularROI):
height = roi.get_coordinates(typed=True)["diameter"]
else:
return
current_x = roi.pos().x()
new_origin_y = fzp_y - height / 2
if math.isclose(roi.pos().y(), new_origin_y, abs_tol=1e-9):
return
roi.set_position(current_x, new_origin_y)
def _microns_per_pixel(self):
"""Resolve microns/pixel and unit label from the camera user parameter.
@@ -840,6 +925,48 @@ class OMNY_XRayEye(BECWidget, QWidget):
"""Hide the alignment target crosshair on the image view."""
self.target_crosshair.set_visible(False)
@SafeSlot()
@rpc_timeout(20)
def lock_vertical_center(self):
"""Snap every ROI's vertical center to the crosshair's fixed y position
(see ``crosshair_position()``), immediately and again on every future
draw/drag/resize, until ``unlock_vertical_center()`` is called. Used by
``x_ray_eye_align.py`` once the sample's height has been physically
centered on the FZP, so subsequent alignment boxes no longer need manual
vertical placement. See ``_snap_roi_vertical()``.
Also settable by the operator directly via the "ROI vertical lock"
toggle in the control panel (``_on_vertical_lock_toggle_changed``) --
e.g. to temporarily release a script-engaged lock mid-run in order to
freely draw/measure an unrelated box, then re-engage it. Both paths
keep the toggle's visual state in sync with ``_vertical_lock_enabled``.
"""
self._vertical_lock_enabled = True
self.vertical_lock_toggle.blockSignals(True)
self.vertical_lock_toggle.checked = True
self.vertical_lock_toggle.blockSignals(False)
for roi in self.roi_manager.controller.rois:
self._snap_roi_vertical(roi)
@SafeSlot()
@rpc_timeout(20)
def unlock_vertical_center(self):
"""Stop auto-snapping ROIs' vertical center -- see ``lock_vertical_center()``."""
self._vertical_lock_enabled = False
self.vertical_lock_toggle.blockSignals(True)
self.vertical_lock_toggle.checked = False
self.vertical_lock_toggle.blockSignals(False)
@SafeSlot(bool)
def _on_vertical_lock_toggle_changed(self, enabled: bool):
"""Operator flipped the "ROI vertical lock" toggle directly -- delegate to
lock_vertical_center()/unlock_vertical_center() so both entry points
(this toggle, and script/RPC calls) stay in sync."""
if enabled:
self.lock_vertical_center()
else:
self.unlock_vertical_center()
@SafeProperty(bool)
def crosshair_visible(self) -> bool:
"""Whether the alignment target crosshair is currently shown."""
@@ -1173,6 +1300,26 @@ class OMNY_XRayEye(BECWidget, QWidget):
self.submit_button.setEnabled(True)
else:
self.submit_button.setEnabled(False)
self._refresh_skip_button()
@SafeSlot(bool)
@rpc_timeout(20)
def enable_skip_button(self, enable: bool):
"""
Allow/forbid skipping the current alignment angle.
The first call also shows the button (it is hidden for clients that never call this).
The button is only clickable while this is True *and* the submit button is enabled,
so it follows the client's "please wait ..." phases automatically.
Args:
enable(bool): True if the client accepts a skip for the current step
"""
self.skip_button.setVisible(True)
self._skip_enabled = bool(enable)
self._refresh_skip_button()
def _refresh_skip_button(self):
self.skip_button.setEnabled(self._skip_enabled and self.submit_button.isEnabled())
@SafeSlot(bool)
@rpc_timeout(20)
@@ -1268,6 +1415,14 @@ class OMNY_XRayEye(BECWidget, QWidget):
finally:
self.submit_button.blockSignals(False)
@SafeSlot()
def skip(self):
"""Skip the current alignment angle: signal the client with submit = -1 (no ROI needed)."""
# Disable right away so a double click cannot skip the following step as well; the
# client re-enables the buttons once it is ready for the next step.
self.skip_button.setEnabled(False)
self.dev.omny_xray_gui.submit.set(-1)
def cleanup(self):
"""Cleanup connections on widget close -> disconnect slots and stop live mode of camera."""
self._queue_idle_timer.stop()
+29
View File
@@ -51,3 +51,32 @@ operator:
softwareTrigger: false
deviceTags:
- machine
injection_status:
# Top-up gating: '0' / 'Filling Storage Ring' means the storage ring is
# being refilled. readoutPriority is 'monitored' (not 'baseline') so this
# is captured continuously throughout the scan, not just read once at
# scan start - a top-up can start or stop mid-scan.
#
# Enum values of AGETI-CVME-MASTER-TMA:SR-Inj-Status-I (8 states):
# 0: Filling Storage Ring <- top-up in progress
# 1: Waiting manual trig...
# 2: Electron gun event OFF!
# 3: SR Injection ready
# 4: Inj. events OFF
# 5: Beam Dump
# 6: !Event LI-RF set on 10Hz!
# 7: !Injection guard inhibit!
description: SLS injection/top-up status
deviceClass: ophyd_devices.EpicsSignalRO
deviceConfig:
read_pv: AGETI-CVME-MASTER-TMA:SR-Inj-Status-I
auto_monitor: true
string: true
onFailure: buffer
enabled: true
readoutPriority: monitored
readOnly: true
softwareTrigger: false
deviceTags:
- machine
+43 -30
View File
@@ -46,13 +46,7 @@ feyey:
in: -10.09
deviceTags:
- ptycho_flomni
# TODO(commissioning): needs userParameter (e.g. in/up, down) -- move_fheater_down()
# in flomni.py currently reads user_parameter.get("in") and gets None.
# KNOWN GAP: fosa_in()/foptics_in() (flomni_optics_mixin.py) never check fheater
# position before driving fosaz. The OSA travels inside the heater's envelope, so
# fosaz must only move while fheater is fully at its "up" or fully at its "down"
# limit -- an intermediate heater position during fosa_in() risks a collision.
# Verify/add that check once real up/down limits are known from hardware.
fheater:
description: Heater Y
deviceClass: csaxs_bec.devices.omny.galil.fgalil_ophyd.FlomniGalilMotor
@@ -69,6 +63,10 @@ fheater:
readOnly: false
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#in: -0.5
in: -11.5
enabled: 1
deviceTags:
- ptycho_flomni
@@ -89,21 +87,24 @@ foptx:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#120 micros, 60
# in: -13.890
# out: -14
#150 micros, 60
in: -14.191
out: -13.831
# in: -14.191
# out: -13.831
#170 micron, 60 nm
#in: -13.8909
#out: -13.831
in: -13.8909
out: -13.831
#250 micron, 30 nm, Abe structures
# in: -13.8809375
# out: -14.1809
#250 micron, 30 nm, Tomas structures
# in: -14.5490625
# out: -14.1809
fzp_diameter: 150 # microns
fzp_diameter: 170 # microns
fzp_outermost_zone_width: 60 # nm
detector_distance: -1 # mm, sample-to-detector; unknown for now
detector_distance: 7196 # mm, sample-to-detector; unknown for now
fzp_details: "manufacturing notes here" # free-text FZP manufacturing notes
deviceTags:
- ptycho_flomni
@@ -126,11 +127,11 @@ fopty:
connectionTimeout: 20
userParameter:
#150 micron, 60
in: 1.02
out: 0.57
# in: 1.02
# out: 0.869
#170 micron, 60 nm
#in: 0.42
#out: 0.57
in: 0.42
out: 0.57
#250 micron, 30 nm, Abe structures
# in: 2.8299
# out: 2.8299
@@ -367,15 +368,18 @@ fosax:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#120micron, 60 nm, 7.9 kev
# in: 8.74251
# out: 5.1
#150micron, 60 nm, 7.9 kev
in: 9.0393
out: 5.1
# in: 9.0393
# out: 5.1
#170 micron, 60 nm, 7.6 kev
#in: 8.7568
#out: 5.1
# in: 8.7568
# out: 5.1
#170 micron, 60 nm, 7.9 kev
#n: 8.722151
#ut: 5.1
in: 8.718233
out: 5.1
#250 micron, 30 nm, Abe structures
# in: 8.7392
# out: 5.1
@@ -402,12 +406,14 @@ fosay:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#120 micron, 60 nm, 7.9 kev
# in: -0.645276
#150 micron, 60 nm, 7.9 kev
in: -0.6422
#in: -0.6422
#170 micron, 60 nm, 7.6 kev
#in: -0.0235
#170 micron, 60 nm, 7.9 kev
#in: -0.0563
in: -0.062253
#250 micron, 30 nm, Abe structures
# in: -2.3684
#250 micron, 30 nm, Tomas structures
@@ -431,15 +437,15 @@ fosaz:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#150 micron, 60 nm, 7.9 kev, foptz 16.9, probe size 7.5 mu
in: 11.0998
out: 6
#150 micron, 60 nm
# in: 11.0998
# out: 6
#170 micron, 60 nm, 7.6 kev
#in: 8.5
#out: 6
#170 micron, 60 nm, 7.9 kev, foptz 16.9, probe size 7.5 mu
# in: 13.1
# out: 6
in: 14.1
out: 4
# micron, 30 nm, 7.9 kev, very close to the sample. make sure foptz is 32.02 or smaller //abe's fzp's
# in: 0.5
# out: -5
@@ -701,6 +707,13 @@ omny_panda_continuous:
deviceConfig:
host: omny-panda.psi.ch
raw_stream_mode: true
# Only actually Arm() PandA for scans that consume its data -- generic BEC staging
# (stage_all_devices/pre_scan_all_devices) otherwise arms this device for every scan type
# (grid_scan, line_scan, alignment moves, ...), not just the ptycho scan that reads it. See
# docs/developer/panda_box_free_running_setup.md and
# AI_docs/PANDA_POSITION_VALIDATION.md sec 7 (stale frame_counter exposure).
arm_scan_allowlist:
- flomni_fermat_scan
signal_alias:
FMC_IN.VAL1.Value: cap_voltage_fzp_y
FMC_IN.VAL2.Value: cap_voltage_fzp_x
@@ -104,6 +104,9 @@ fheater:
readOnly: false
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: -0.5
enabled: 1
deviceTags:
- simulated_flomni
@@ -42,6 +42,7 @@ class FlomniGalilController(GalilController):
"lights_on",
"print_command_history",
"hard_abort_and_restore_positioning_mode",
"stop_all_axes",
]
def is_axis_moving(self, axis_Id, axis_Id_numeric) -> bool:
@@ -47,6 +47,7 @@ class GalilController(Controller):
"all_axes_referenced",
"print_command_history",
"hard_abort_and_restore_positioning_mode",
"stop_all_axes",
]
OKBLUE = "\033[94m"
+220 -5
View File
@@ -8,6 +8,7 @@ from bec_lib.logger import bec_logger
from bec_server.scan_server.scans.scan_base import ScanInfo as ScanServerScanInfo
from ophyd_devices import StatusBase
from ophyd_devices.devices.panda_box.panda_box import PandaBox, PandaState
from ophyd_devices.interfaces.base_classes.psi_device_base import DeviceStoppedError
from pandablocks.responses import FrameData
from csaxs_bec.devices.utils.utils import fetch_scan_info
@@ -50,6 +51,25 @@ class PandaBoxOMNY(PandaBox):
_GATE_SIGNAL_NAME = "gate_detector_active"
_GATE_BIT_OFFSET = 0
# Arming in on_pre_scan is tried this many times (one forced abort/reset in between) before
# the scan is failed. READY normally arrives within milliseconds of the readout thread
# connecting, so a missing READY means the readout thread/box is in a stale state.
_PRE_SCAN_ARM_ATTEMPTS = 2
# PandA's COUNTER1.OUT (the block behind frame_counter) is not cleared by Arm() --
# COUNTER1.ENABLE=PCAP.ACTIVE only gates counting, not the persisted output value, and
# COUNTER1.SET is wired to a constant (never pulsed) in the saved layout (see
# layouts/omny_freerun.ini). So the first raw samples of a new acquisition can briefly
# carry a stale frame_counter value left over from the previous acquisition's tail --
# confirmed 2026-09-21 against a real scan (S06313): 36 samples reading a stale value
# before dropping to 0 and counting correctly from there. If that stale value collides
# with a real point index later in the same scan, grouping raw samples by frame_counter
# silently mixes those stale rows into that point's average (see
# docs/developer/panda_box_free_running_setup.md, "Known data-quality caveat"). Rows are
# dropped client-side until the first observed frame_counter == 0 per acquisition to
# guard against this; see _reset_raw_stream_state/_drop_stale_frame_counter_rows below.
_FRAME_COUNTER_SIGNAL_NAME = "frame_counter"
def __init__(
self,
*,
@@ -61,6 +81,8 @@ class PandaBoxOMNY(PandaBox):
raw_stream_mode: bool = False,
raw_stream_flush_interval: float = 0.05,
raw_stream_flush_row_count: int = 500,
pre_scan_timeout: float = 5.0,
arm_scan_allowlist: list[str] | None = None,
**kwargs,
) -> None:
# Free-running raw-stream configuration. Defaults preserve today's behavior:
@@ -73,6 +95,26 @@ class PandaBoxOMNY(PandaBox):
self._raw_stream_buffer_row_count = 0
self._raw_stream_buffer_lock = threading.Lock()
self._raw_stream_last_flush = time.monotonic()
# Whether frame_counter == 0 has been observed yet for the current acquisition; see
# _FRAME_COUNTER_SIGNAL_NAME above. Reset for every fresh arm, not just once per stage,
# since on_pre_scan retries re-arm the box (and thus reset COUNTER1's counting) too.
self._raw_stream_seen_reset = False
# Scan names (ScanServerScanInfo.scan_name, e.g. "flomni_fermat_scan") this device
# should actually be armed for. None (default) means "arm for every scan" -- today's
# behavior, fully backward compatible. Generic BEC staging (stage_all_devices/
# pre_scan_all_devices in bec_server) arms every enabled device for every scan type,
# not just the ones that consume this device's data; this lets a beamline config opt
# this device out of Arm()/COUNTER1 cycling (and the associated stale-frame_counter
# exposure, see AI_docs/PANDA_POSITION_VALIDATION.md sec 7) for scans that don't need it.
self.arm_scan_allowlist = arm_scan_allowlist
# Seconds to wait for the PandA READY event per arming attempt in on_pre_scan.
self.pre_scan_timeout = pre_scan_timeout
# Set while the data readout thread is parked (not inside _run_data_readout), so a
# retry in on_pre_scan can wait for the old readout loop's cleanup to be finished.
self._readout_idle = threading.Event()
self._readout_idle.set()
super().__init__(
name=name,
@@ -88,16 +130,30 @@ class PandaBoxOMNY(PandaBox):
self._acquisition_group = "burst"
self._timeout_on_completed = 10
self.scan_parameters: ScanServerScanInfo | None = None
# Safe default (arm): matches arm_scan_allowlist=None semantics, and covers the case
# where on_pre_scan/on_complete are called before any on_stage (some tests do this).
self._should_arm_panda: bool = True
def _compute_should_arm_panda(self) -> bool:
"""Whether this scan should actually Arm() the PandA and wait on completion. See the
class-level note on arm_scan_allowlist. Fails open (arms) whenever the allowlist isn't
set or scan_parameters isn't available yet, preserving today's unconditional-arm
behavior in both cases."""
if self.arm_scan_allowlist is None:
return True
if self.scan_parameters is None:
return True
return self.scan_parameters.scan_name in self.arm_scan_allowlist
def on_stage(self):
start_time = time.time()
super().on_stage()
self.scan_parameters = fetch_scan_info(self.scan_info)
# Recomputed fresh every stage cycle: the same device instance is staged for many
# different scan types across a session, not just the ones in arm_scan_allowlist.
self._should_arm_panda = self._compute_should_arm_panda()
with self._raw_stream_buffer_lock:
self._raw_stream_buffer = {}
self._raw_stream_buffer_row_count = 0
self._raw_stream_last_flush = time.monotonic()
self._reset_raw_stream_state()
# TODO, adjust as seen fit.
# Adjust the acquisition group based on scan parameters if needed
@@ -113,13 +169,120 @@ class PandaBoxOMNY(PandaBox):
logger.info(f"PandaBox {self.name} on_stage completed in {time.time() - start_time:.3f}s.")
def _run_data_readout(self) -> None:
"""Same as the base class, but tracks whether the readout loop is running. The base
loop ends with `_reset_panda()`, which clears `data_thread_run_event`; a retry in
`_arm_with_retry` must not set that event again before this cleanup has run."""
self._readout_idle.clear()
try:
super()._run_data_readout()
finally:
self._readout_idle.set()
def on_pre_scan(self):
"""Arm the PCAP module. Unlike the base class this does not wait forever for the READY
event: if it does not arrive, the box is forcibly reset and arming is retried once,
and only if that fails as well the returned status fails, so the scan is aborted with
an error instead of hanging silently in pre_scan.
If arm_scan_allowlist excludes the current scan (see _should_arm_panda/on_stage), PandA
is left untouched entirely -- no Arm() is sent -- and this resolves immediately.
"""
if not self._should_arm_panda:
status = StatusBase(obj=self)
status.set_finished()
return status
# Not registered with cancel_on_stop: the status of each single attempt in _try_arm is,
# so a stop makes the task raise DeviceStoppedError, which fails this status. Cancelling
# this one as well would make the task handler try to resolve it a second time.
return self.task_handler.submit_task(self._arm_with_retry, run=True)
def _arm_with_retry(self) -> None:
"""Task run by on_pre_scan: try to arm, force an abort/reset and try again if the
first attempt fails, raise if all attempts fail. Every retry is printed and logged."""
attempts = self._PRE_SCAN_ARM_ATTEMPTS
for attempt in range(1, attempts + 1):
if self._try_arm():
if attempt > 1:
logger.info(
f"PandaBox {self.name} armed successfully on attempt {attempt}/{attempts}."
)
return
if attempt < attempts:
msg = (
f"PandaBox {self.name} did not report READY within {self.pre_scan_timeout}s "
f"(arming attempt {attempt}/{attempts}). Forcing abort/reset and retrying."
)
print(msg)
logger.warning(f"{msg} {self._arm_diagnostics()}")
self._reset_panda()
# Wait for the old readout loop to finish its own _reset_panda() cleanup, it
# would otherwise clear the run event we set next.
self._readout_idle.wait(timeout=2)
self.data_thread_run_event.set()
msg = (
f"PandaBox {self.name} could not be armed after {attempts} attempts (no READY within "
f"{self.pre_scan_timeout}s each). The box was reset; check the connection and the "
"PandA state."
)
logger.error(f"{msg} {self._arm_diagnostics()}")
self._reset_panda()
raise RuntimeError(msg)
def _try_arm(self) -> bool:
"""One arming attempt: wait for the READY event, then send Arm(). Returns False if
READY did not arrive in time or the box reported a conflicting state (FRAME/END,
i.e. an acquisition is still running). A stop/unstage is not retried."""
if self.stopped:
raise DeviceStoppedError(f"Device {self.name} has been stopped")
status = StatusBase(obj=self)
cb_id = self.add_status_callback(
status=status, success=[PandaState.READY], failure=[PandaState.FRAME, PandaState.END]
)
self.cancel_on_stop(status)
try:
status.wait(timeout=self.pre_scan_timeout)
except DeviceStoppedError:
self.remove_status_callback(cb_id)
raise
except Exception as exc: # pylint: disable=broad-except
# Timeout, or the box reported FRAME/END while we waited for READY
self.remove_status_callback(cb_id)
logger.warning(f"PandaBox {self.name} arming attempt failed: {exc!r}")
return False
# Reset raw-stream state right before arming, not just once in on_stage: a retry here
# re-arms the box (and thus resets COUNTER1's counting) too, so the stale-value window
# applies again on every attempt, not just the first.
self._reset_raw_stream_state()
# Sent from here rather than from a status callback (as the base class does), so the
# scan is only released after Arm() actually went out.
self._arm()
return True
def _arm_diagnostics(self) -> str:
return (
f"[host={self.host}, panda_state={self.panda_state}, "
f"data_thread_alive={self.data_thread.is_alive()}, "
f"data_thread_run_event={self.data_thread_run_event.is_set()}]"
)
def on_unstage(self):
"""Make sure no buffered raw-stream data is left behind before resetting the device."""
self._flush_raw_stream_buffer()
return super().on_unstage()
def on_complete(self):
"""On complete is called after the scan is complete. We need to wait for the capture to complete before we can disarm the PandaBox."""
"""On complete is called after the scan is complete. We need to wait for the capture to
complete before we can disarm the PandaBox -- unless this device was never armed for
this scan (see _should_arm_panda/on_stage), in which case there is nothing to wait for
and no wait-loop is started. _disarm() is deliberately not called here in that case:
the box is already guaranteed disarmed (on_stage waits for DISARMED before staging),
nothing armed it since Arm() was skipped, and on_unstage()/stop() unconditionally
disarm anyway at the end of the lifecycle."""
if not self._should_arm_panda:
status = StatusBase(obj=self)
status.set_finished()
return status
if self.raw_stream_mode:
return self._on_complete_free_running()
return self._on_complete_burst()
@@ -221,6 +384,11 @@ class PandaBoxOMNY(PandaBox):
return
converted = self.convert_frame_data(frame_data=data)
converted = self._drop_stale_frame_counter_rows(converted)
if converted is None:
# Every row in this chunk still carried a stale frame_counter from the previous
# acquisition; nothing here belongs to the current one.
return
with self._raw_stream_buffer_lock:
self._merge_into_raw_stream_buffer(converted)
elapsed = time.monotonic() - self._raw_stream_last_flush
@@ -230,6 +398,53 @@ class PandaBoxOMNY(PandaBox):
):
self._flush_raw_stream_buffer_locked()
def _reset_raw_stream_state(self) -> None:
"""Drop any leftover buffered rows and mark that frame_counter == 0 has not yet been
observed, for a fresh acquisition. Called from on_stage (once per scan) and from
_try_arm right before every Arm() (including on_pre_scan retries), since each Arm()
resets COUNTER1's counting and thus reopens the stale-value window described at
_FRAME_COUNTER_SIGNAL_NAME above. Also drops whatever was buffered from a previous,
now-abandoned attempt, since that data belongs to an acquisition that never completed.
"""
with self._raw_stream_buffer_lock:
self._raw_stream_buffer = {}
self._raw_stream_buffer_row_count = 0
self._raw_stream_last_flush = time.monotonic()
self._raw_stream_seen_reset = False
def _drop_stale_frame_counter_rows(
self, converted: dict[str, dict[str, Any]]
) -> dict[str, dict[str, Any]] | None:
"""Drop rows captured before frame_counter first reads 0 for this acquisition.
See the class-level note on `_FRAME_COUNTER_SIGNAL_NAME`: PandA's COUNTER1.OUT is not
cleared by Arm(), so the first raw samples of a new acquisition can briefly carry a
stale value left over from the previous acquisition's tail. Truncating every signal's
row list at the same index keeps all channels aligned.
Returns the (possibly truncated) dict, or None if every row in this chunk is still
stale and should be dropped entirely.
"""
if self._raw_stream_seen_reset:
return converted
counter_entry = converted.get(self._FRAME_COUNTER_SIGNAL_NAME)
if counter_entry is None:
# This layout/config doesn't capture frame_counter -- nothing to filter against.
self._raw_stream_seen_reset = True
return converted
reset_index = next(
(i for i, v in enumerate(counter_entry["value"]) if v == 0), None
)
if reset_index is None:
return None
self._raw_stream_seen_reset = True
if reset_index == 0:
return converted
return {
key: {**entry, "value": entry["value"][reset_index:]}
for key, entry in converted.items()
}
def _merge_into_raw_stream_buffer(self, converted: dict[str, dict[str, Any]]) -> None:
"""Append newly converted rows into the pending raw-stream buffer. Must be called
while holding `_raw_stream_buffer_lock`."""
+13 -2
View File
@@ -238,7 +238,13 @@ class _SimIDSBackend:
self._connected = False
self._rgb = rgb
self._noise_std = float(noise_std)
self._exposure_time = 10000.0 # ms
# Realistic default well under a 5 Hz (200 ms) live-mode frame period --
# was 10000.0 (10 s), inconsistent with get_exposure_range()'s own
# microsecond-scaled formula below and with the real IDSCamera Cpt's
# documented units/placeholder (exposure_time is in ms, default max
# placeholder 1000.0 ms). A 10 s default meant every fresh sim connect
# seeded exposure_time.put(10000) -- ~50x a 5 Hz period.
self._exposure_time = 10.0 # ms
self._auto_exposure = True
self._auto_gain = True
self._pixel_clock = 20 # MHz
@@ -307,7 +313,12 @@ class _SimIDSBackend:
def get_exposure_range(self) -> tuple[float, float, float]:
# Loosely mimics the real driver's pixel-clock-dependent ceiling: a
# higher pixel clock -> shorter max frame time -> lower max exposure.
max_exposure = 1_000_000.0 / max(self._pixel_clock, 1)
# Scaled in ms (the real IDSCamera's documented exposure_time unit):
# 20_000.0 / pixel_clock_MHz gives ~1000 ms at the default 20 MHz
# pixel clock, matching exposure_time_max's own real-device placeholder
# default. The previous 1_000_000.0 scale was microsecond-sized,
# producing a ~50 s ceiling here instead.
max_exposure = 20_000.0 / max(self._pixel_clock, 1)
return 0.1, max_exposure, 0.1
def get_pixel_clock(self) -> int:
+35
View File
@@ -0,0 +1,35 @@
"""Shared helpers for Fermat-spiral scan position generation (flomni/omny/lamni)."""
from __future__ import annotations
import numpy as np
def compute_fermat_spiral_n_max(
length_axis1: float, length_axis2: float, step: float, safety_factor: float = 1.1
) -> int:
"""
Size the Fermat-spiral loop's iteration cap.
The spiral radius at iteration ``ii`` grows as ``step * 0.57 * sqrt(ii)``, so sizing
the cap from the box's area alone (``length_axis1 * length_axis2 * 3.2 / step**2``)
under-estimates the iterations needed for elongated (non-square) boxes: the maximum
radius that formula reaches is a function of area, not of the box's actual corner
distance, so the spiral can exhaust its iterations before reaching the far corners of
a long, narrow box. Taking the max with a cap derived from the corner distance itself
fixes that while leaving square/near-square boxes unaffected (the area-based term
already dominates there).
Args:
length_axis1 (float): Full extent of the scan box along the first axis.
length_axis2 (float): Full extent of the scan box along the second axis.
step (float): Spiral step size.
safety_factor (float): Margin applied to the corner-distance-based term.
Returns:
int: Iteration cap for the Fermat-spiral generation loop.
"""
n_max_area = int(length_axis1 * length_axis2 * 3.2 / step / step)
corner_radius = np.sqrt((length_axis1 / 2) ** 2 + (length_axis2 / 2) ** 2)
n_max_corner = int((corner_radius / (0.57 * step)) ** 2 * safety_factor)
return max(n_max_area, n_max_corner)
+68 -17
View File
@@ -32,6 +32,7 @@ from bec_server.scan_server.scans.scan_base import ScanBase, ScanType
from bec_server.scan_server.scans.scan_modifier import scan_hook
from csaxs_bec.devices.epics.delay_generator_csaxs.delay_generator_csaxs import TRIGGERSOURCE
from csaxs_bec.scans.fermat_spiral_utils import compute_fermat_spiral_n_max
logger = bec_logger.logger
@@ -56,6 +57,7 @@ class FlomniFermatScan(ScanBase):
"cenx",
"ceny",
"step",
"fermat_asymmetry",
"zshift",
"angle",
"corridor_size",
@@ -65,19 +67,23 @@ class FlomniFermatScan(ScanBase):
MIN_POSITIONS = 20 # Minimum number of positions required for a valid scan
MAX_CORRIDOR_SIZE = 3 # Corridor size is capped at this value (in um) for stability
MIN_FERMAT_ASYMMETRY = 0.1 # Practical working range for fermat_asymmetry (soft-clamped)
MAX_FERMAT_ASYMMETRY = 10.0
def __init__(
# fmt: off
self,
fovx: Annotated[float, ScanArgument(display_name="Fovx", description="FOV in the piezo plane (i.e. piezo range). Max 200 um.", units=Units.µm, gt=0, lt=200)],
fovy: Annotated[float, ScanArgument(display_name="Fovy", description="FOV in the piezo plane (i.e. piezo range). Max 100 um.", units=Units.µm, gt=0, lt=100)],
cenx: Annotated[float, ScanArgument(display_name="Cenx", description="Center position in x.", units=Units.µm)],
ceny: Annotated[float, ScanArgument(display_name="Ceny", description="Center position in y.", units=Units.µm)],
step: Annotated[float, ScanArgument(display_name="Step", description="Step size.", units=Units.µm)],
zshift: Annotated[float, ScanArgument(display_name="Zshift", description="Shift in z. ", units=Units.µm)],
angle: Annotated[float, ScanArgument(display_name="Angle", description="Rotation angle (will rotate first)", units=Units.deg)],
fovx: Annotated[float, ScanArgument(display_name="Fovx", description="FOV in the piezo plane (i.e. piezo range). Max 220 um.", units=Units.µm, gt=0, le=220, example=10)],
fovy: Annotated[float, ScanArgument(display_name="Fovy", description="FOV in the piezo plane (i.e. piezo range). Max 100 um.", units=Units.µm, gt=0, lt=100, example=10)],
cenx: Annotated[float, ScanArgument(display_name="Cenx", description="Center position in x.", units=Units.µm, example=0)],
ceny: Annotated[float, ScanArgument(display_name="Ceny", description="Center position in y.", units=Units.µm, example=0)],
step: Annotated[float, ScanArgument(display_name="Step", description="Step size.", units=Units.µm, example=1)],
zshift: Annotated[float, ScanArgument(display_name="Zshift", description="Shift in z. ", units=Units.µm, example=0)],
angle: Annotated[float, ScanArgument(display_name="Angle", description="Rotation angle (will rotate first)", units=Units.deg, example=0)],
corridor_size: Annotated[float | None, ScanArgument(display_name="Corridor Size", description="Corridor size for the corridor optimization.", units=Units.µm)] = None,
exp_time: DefaultArgType.ExposureTime = 0,
fermat_asymmetry: Annotated[float, ScanArgument(display_name="Fermat Asymmetry", description="Ratio of x spacing to y spacing (x/y). 1 = symmetric. >1 widens x spacing / tightens y spacing; <1 the opposite. FOV is preserved; point count changes with the ratio.", gt=0, example=1)] = 1.0,
randomize_offset: Annotated[bool, ScanArgument(display_name="Randomize Offset", description="Randomize the sub-step offset of the Fermat lattice on every scan (decorrelates position-tied reconstruction errors across a tomographic series). Set to False for the unshifted lattice.", example=True)] = True,
exp_time: DefaultArgType.ExposureTime = 0.1,
frames_per_trigger: DefaultArgType.FramesPerTrigger = 1,
burst_at_each_point: DefaultArgType.BurstAtEachPoint = 1,
**kwargs,
@@ -87,7 +93,7 @@ class FlomniFermatScan(ScanBase):
flOMNI Fermat Scan
Args:
fovx (float): FOV in the piezo plane (i.e. piezo range). Max 200 um.
fovx (float): FOV in the piezo plane (i.e. piezo range). Max 220 um.
fovy (float): FOV in the piezo plane (i.e. piezo range). Max 100 um.
cenx (float): Center position in x.
ceny (float): Center position in y.
@@ -95,15 +101,17 @@ class FlomniFermatScan(ScanBase):
zshift (float): Shift in z.
angle (float): Rotation angle (will rotate first)
corridor_size (float | None): Corridor size for the corridor optimization.
fermat_asymmetry (float): Ratio of x spacing to y spacing (x/y). 1 = symmetric.
x spacing stays fixed at step, y spacing scales by this ratio. The FOV is
preserved, so the point count changes with the ratio.
randomize_offset (bool): Randomize the sub-step offset of the Fermat lattice on every
scan. Default True. If False, the lattice is not shifted.
exp_time (float): Exposure time in seconds
frames_per_trigger (int): Number of frames per trigger for devices that support configurable frame counts per trigger.
burst_at_each_point (int): Number of triggers and readouts at each point.
Returns:
ScanReport
Examples:
>>> scans.flomni_fermat_scan(10, 10, 0, 0, 1, 0, 0, exp_time=0.1, frames_per_trigger=1, burst_at_each_point=1)
"""
super().__init__(**kwargs)
self._baseline_readout_status = None
@@ -115,6 +123,8 @@ class FlomniFermatScan(ScanBase):
self.zshift = zshift
self.angle = angle
self.corridor_size = corridor_size
self.fermat_asymmetry = fermat_asymmetry
self.randomize_offset = randomize_offset
self.exp_time = exp_time
self.frames_per_trigger = frames_per_trigger
self.burst_at_each_point = burst_at_each_point
@@ -160,6 +170,8 @@ class FlomniFermatScan(ScanBase):
cenx=self.cenx,
ceny=self.ceny,
zshift=self.zshift,
fermat_asymmetry=self.fermat_asymmetry,
randomize_offset=self.randomize_offset,
)
if len(positions) < self.MIN_POSITIONS:
@@ -421,6 +433,9 @@ class FlomniFermatScan(ScanBase):
cenx: float = 0.0,
ceny: float = 0.0,
zshift: float = 0.0,
seed: int | None = None,
fermat_asymmetry: float = 1.0,
randomize_offset: bool = True,
):
"""
Calculate positions for a Fermat spiral scan.
@@ -433,10 +448,32 @@ class FlomniFermatScan(ScanBase):
step(float): stepsize
spiral_type(int): 0 for traditional Fermat spiral
center(bool): whether to include the center position
seed(int | None): seed for the per-call random sub-step offset (see below).
Pass a fixed value for reproducible output (e.g. in tests); leave as None
to draw a fresh offset every call.
fermat_asymmetry(float): ratio of x spacing to y spacing (x/y). 1 = symmetric
(default). x spacing stays fixed at step; y spacing scales by this ratio.
The requested FOV (m1/m2 range) is preserved, so the point count changes
with the ratio. Soft-clamped to [MIN_FERMAT_ASYMMETRY, MAX_FERMAT_ASYMMETRY].
randomize_offset(bool): draw the random sub-step offset (default). If False the
offset is 0 and the lattice is unshifted; seed is then irrelevant.
Returns:
positions(array): positions
"""
if fermat_asymmetry < FlomniFermatScan.MIN_FERMAT_ASYMMETRY:
logger.warning(
f"fermat_asymmetry is smaller than {FlomniFermatScan.MIN_FERMAT_ASYMMETRY}. It"
f" will be limited to {FlomniFermatScan.MIN_FERMAT_ASYMMETRY}."
)
fermat_asymmetry = FlomniFermatScan.MIN_FERMAT_ASYMMETRY
elif fermat_asymmetry > FlomniFermatScan.MAX_FERMAT_ASYMMETRY:
logger.warning(
f"fermat_asymmetry is larger than {FlomniFermatScan.MAX_FERMAT_ASYMMETRY}. It"
f" will be limited to {FlomniFermatScan.MAX_FERMAT_ASYMMETRY}."
)
fermat_asymmetry = FlomniFermatScan.MAX_FERMAT_ASYMMETRY
positions = []
phi = 2 * np.pi * ((1 + np.sqrt(5)) / 2.0) + spiral_type * np.pi
@@ -444,19 +481,33 @@ class FlomniFermatScan(ScanBase):
length_axis1 = np.abs(m1_stop - m1_start)
length_axis2 = np.abs(m2_stop - m2_start)
n_max = int(length_axis1 * length_axis2 * 3.2 / step / step)
# y is checked against an inflated half-width (see below), so the spiral must be
# grown far enough to still cover the real fovy after the y-compression.
n_max = compute_fermat_spiral_n_max(length_axis1, length_axis2 * fermat_asymmetry, step)
z_pos = zshift
# Randomize the sampling lattice's sub-step phase on every call (i.e. every
# projection), matching the legacy spec behaviour: this decorrelates position-tied
# reconstruction errors that would otherwise accumulate into ring artifacts across
# a tomographic series.
offset = np.random.default_rng(seed).uniform(0, step) if randomize_offset else 0.0
logger.info(f"Fermat spiral sub-step offset: {offset:.4f} um")
for ii in range(start, n_max):
radius = step * 0.57 * np.sqrt(ii)
x = radius * np.sin(ii * phi) - offset
# y_raw is the pre-compression lattice coordinate: kept at the isotropic
# spacing/phase so the crop check below (against the inflated fovy) sees the
# same sub-step randomization as x, then compressed by fermat_asymmetry
# afterwards -- x spacing stays fixed at step, y spacing becomes step/fermat_asymmetry.
y_raw = radius * np.cos(ii * phi) - offset
# FOV is restructed below at check pos in range
if abs(radius * np.sin(ii * phi)) > length_axis1 / 2:
if abs(x) > length_axis1 / 2:
continue
if abs(radius * np.cos(ii * phi)) > length_axis2 / 2:
if abs(y_raw) > (length_axis2 * fermat_asymmetry) / 2:
continue
x = radius * np.sin(ii * phi)
y = radius * np.cos(ii * phi)
y = y_raw / fermat_asymmetry
positions.append([x + cenx, y + ceny, z_pos])
left_lower_corner = [min(m1_start, m1_stop) + cenx, min(m2_start, m2_stop) + ceny, z_pos]
right_upper_corner = [max(m1_start, m1_stop) + cenx, max(m2_start, m2_stop) + ceny, z_pos]
+18 -6
View File
@@ -26,6 +26,7 @@ from bec_server.scan_server.errors import ScanAbortion
from bec_server.scan_server.scans.scan_base import ScanBase, ScanType
from bec_server.scan_server.scans.scan_modifier import scan_hook
from csaxs_bec.scans.fermat_spiral_utils import compute_fermat_spiral_n_max
from csaxs_bec.scans.lamni_components import LamNIComponents
logger = bec_logger.logger
@@ -67,9 +68,9 @@ class LamniFermatScan(ScanBase):
def __init__(
# fmt: off
self,
fovx: Annotated[float, ScanArgument(display_name="FOV x", description="FOV in the piezo plane (i.e. piezo range). Max 80 um", units=Units.µm, gt=0, lt=80)],
fovy: Annotated[float, ScanArgument(display_name="FOV y", description="FOV in the piezo plane (i.e. piezo range). Max 80 um", units=Units.µm, gt=0, lt=80)],
step: Annotated[float, ScanArgument(display_name="Step", description="Step size", units=Units.µm)],
fovx: Annotated[float, ScanArgument(display_name="FOV x", description="FOV in the piezo plane (i.e. piezo range). Max 80 um", units=Units.µm, gt=0, lt=80, example=10)],
fovy: Annotated[float, ScanArgument(display_name="FOV y", description="FOV in the piezo plane (i.e. piezo range). Max 80 um", units=Units.µm, gt=0, lt=80, example=10)],
step: Annotated[float, ScanArgument(display_name="Step", description="Step size", units=Units.µm, example=1)],
shift_x: Annotated[float, ScanArgument(display_name="Shift X", description="Extra shift in x. The shift is directly applied to the scan. It will not be auto-rotated.", units=Units.mm)] = 0,
shift_y: Annotated[float, ScanArgument(display_name="Shift Y", description="Extra shift in y. The shift is directly applied to the scan. It will not be auto-rotated.", units=Units.mm)] = 0,
center_x: Annotated[float, ScanArgument(display_name="Center X", description="Center position in x at 0 deg. This shift is rotated using the geometry of LamNI. It is determined by the first 'click' in the x-ray eye alignment procedure.", units=Units.mm)] = 0,
@@ -286,6 +287,7 @@ class LamniFermatScan(ScanBase):
fovx: float | None = None,
fovy: float | None = None,
fov_circular: float = 0.0,
seed: int | None = None,
) -> np.ndarray:
"""Generate positions for a LamNI Fermat spiral scan.
@@ -298,6 +300,9 @@ class LamniFermatScan(ScanBase):
spiral_type (float, optional): Angular offset in radians that determines the shape of the spiral.
A spiral with spiral_type=2 is the same as spiral_type=0. Defaults to 0.
center (bool, optional): Add a center point. Defaults to False.
seed (int | None, optional): seed for the per-call random sub-step offset (see
below). Pass a fixed value for reproducible output (e.g. in tests); leave as
None to draw a fresh offset every call.
Returns:
np.ndarray: Array of positions for the Fermat spiral scan.
@@ -309,7 +314,14 @@ class LamniFermatScan(ScanBase):
length_axis1 = np.abs(m1_stop - m1_start)
length_axis2 = np.abs(m2_stop - m2_start)
n_max = int(length_axis1 * length_axis2 * 3.2 / step / step)
n_max = compute_fermat_spiral_n_max(length_axis1, length_axis2, step)
# Randomize the sampling lattice's sub-step phase on every call (i.e. every
# projection), matching the legacy spec behaviour: this decorrelates position-tied
# reconstruction errors that would otherwise accumulate into ring artifacts across
# a tomographic series.
offset = np.random.default_rng(seed).uniform(0, step)
logger.info(f"Fermat spiral sub-step offset: {offset:.4f} um")
total_shift_x, total_shift_y = LamniFermatScan._compute_total_shift(
center_x=center_x,
@@ -326,8 +338,8 @@ class LamniFermatScan(ScanBase):
for ii in range(start, n_max):
radius = step * 0.57 * np.sqrt(ii)
x = radius * np.sin(ii * phi)
y = radius * np.cos(ii * phi)
x = radius * np.sin(ii * phi) - offset
y = radius * np.cos(ii * phi) - offset
if LamniFermatScan._lamni_check_pos_in_fov_range_and_circ_fov(
x=x,
y=y,
+24 -14
View File
@@ -28,6 +28,8 @@ from bec_server.scan_server.scans import MessageEndpoints
from bec_server.scan_server.scans.scan_base import ScanBase, ScanType
from bec_server.scan_server.scans.scan_modifier import scan_hook
from csaxs_bec.scans.fermat_spiral_utils import compute_fermat_spiral_n_max
logger = bec_logger.logger
@@ -61,12 +63,12 @@ class OmnyFermatScan(ScanBase):
def __init__(
# fmt: off
self,
fovx: Annotated[float, ScanArgument(display_name="Fovx", description="FOV in the piezo plane (i.e. piezo range).", units=Units.µm, gt=0, lt=200)],
fovy: Annotated[float, ScanArgument(display_name="Fovy", description="FOV in the piezo plane (i.e. piezo range).", units=Units.µm, gt=0, lt=100)],
cenx: Annotated[float, ScanArgument(display_name="Cenx", description="Center position in x.", units=Units.µm)],
ceny: Annotated[float, ScanArgument(display_name="Ceny", description="Center position in y.", units=Units.µm)],
step: Annotated[float, ScanArgument(display_name="Step", description="Step size.", units=Units.µm)],
zshift: Annotated[float, ScanArgument(display_name="Zshift", description="Shift in z.", units=Units.µm)],
fovx: Annotated[float, ScanArgument(display_name="Fovx", description="FOV in the piezo plane (i.e. piezo range).", units=Units.µm, gt=0, lt=200, example=10)],
fovy: Annotated[float, ScanArgument(display_name="Fovy", description="FOV in the piezo plane (i.e. piezo range).", units=Units.µm, gt=0, lt=100, example=10)],
cenx: Annotated[float, ScanArgument(display_name="Cenx", description="Center position in x.", units=Units.µm, example=0)],
ceny: Annotated[float, ScanArgument(display_name="Ceny", description="Center position in y.", units=Units.µm, example=0)],
step: Annotated[float, ScanArgument(display_name="Step", description="Step size.", units=Units.µm, example=1)],
zshift: Annotated[float, ScanArgument(display_name="Zshift", description="Shift in z.", units=Units.µm, example=0)],
angle: Annotated[float | None, ScanArgument(display_name="Angle", description="Rotation angle (will rotate first)", units=Units.deg)] = None,
corridor_size: Annotated[float, ScanArgument(display_name="Corridor Size", description="Corridor size for the corridor optimization. ", units=Units.µm)] = 3,
exp_time: DefaultArgType.ExposureTime = 0,
@@ -93,9 +95,6 @@ class OmnyFermatScan(ScanBase):
Returns:
ScanReport
Examples:
>>> scans.omny_fermat_scan(10, 10, 0, 0, 1, 0, 0, exp_time=0.1, frames_per_trigger=1, readout_time=0)
"""
super().__init__(**kwargs)
self._baseline_readout_status = None
@@ -282,6 +281,7 @@ class OmnyFermatScan(ScanBase):
step: float = 1,
spiral_type: int = 0,
center: bool = False,
seed: int | None = None,
):
"""
Calculate positions for a Fermat spiral scan.
@@ -294,6 +294,9 @@ class OmnyFermatScan(ScanBase):
step(float): stepsize
spiral_type(int): 0 for traditional Fermat spiral
center(bool): whether to include the center position
seed(int | None): seed for the per-call random sub-step offset (see below).
Pass a fixed value for reproducible output (e.g. in tests); leave as None
to draw a fresh offset every call.
Returns:
positions(array): positions
@@ -305,19 +308,26 @@ class OmnyFermatScan(ScanBase):
length_axis1 = np.abs(m1_stop - m1_start)
length_axis2 = np.abs(m2_stop - m2_start)
n_max = int(length_axis1 * length_axis2 * 3.2 / step / step)
n_max = compute_fermat_spiral_n_max(length_axis1, length_axis2, step)
z_pos = self.zshift
# Randomize the sampling lattice's sub-step phase on every call (i.e. every
# projection), matching the legacy spec behaviour: this decorrelates position-tied
# reconstruction errors that would otherwise accumulate into ring artifacts across
# a tomographic series.
offset = np.random.default_rng(seed).uniform(0, step)
logger.info(f"Fermat spiral sub-step offset: {offset:.4f} um")
for ii in range(start, n_max):
radius = step * 0.57 * np.sqrt(ii)
x = radius * np.sin(ii * phi) - offset
y = radius * np.cos(ii * phi) - offset
# FOV is restructed below at check pos in range
if abs(radius * np.sin(ii * phi)) > length_axis1 / 2:
if abs(x) > length_axis1 / 2:
continue
if abs(radius * np.cos(ii * phi)) > length_axis2 / 2:
if abs(y) > length_axis2 / 2:
continue
x = radius * np.sin(ii * phi)
y = radius * np.cos(ii * phi)
positions.append([x + self.cenx, y + self.ceny, z_pos])
left_lower_corner = [
min(m1_start, m1_stop) + self.cenx,
@@ -116,6 +116,54 @@ layout can be restored unambiguously (see `ptycho_flomni.yaml`):
CLOCK1 (enabled by constant ONE) ──▶ PCAP.gate, PCAP.trig (continuous arm/capture)
```
### Possible variant: gate continuous capture by exposure-active (proposed, not implemented)
Discussed 2026-09-19, not yet implemented or tested against hardware. Today
`PCAP.gate` and `PCAP.trig` are both driven by `CLOCK1.OUT`, so PCAP
free-runs continuously at `CLOCK1`'s rate regardless of whether the detector
is actually exposing -- including idle/dead time between exposures. The
proposal is to split the two roles so they're driven by different sources:
```
CLOCK1 (enabled by constant ONE) ──▶ PCAP.trig (sample-rate clock, unchanged)
TTLIN1.VAL (detector-trigger / exposure-active) ──▶ PCAP.gate (only capture while exposing)
```
`PCAP.trig` keeps ticking at `CLOCK1`'s configured rate exactly as today;
`PCAP.gate` switches from `CLOCK1.OUT` to `TTLIN1.VAL`, so a `CLOCK1` tick
only produces a captured row while the detector is actively exposing --
idle-time ticks are simply dropped, not written.
This should be a pure PandA-layout change -- no `ptycho_flomni.yaml` or
`panda_box_omny.py` change would be needed, since it doesn't change which
BEC signal names exist or what they mean:
- `TTLIN1.VAL` is already proven usable as a `PCAP.gate` source -- it's what
Mode A already uses (`PCAP.GATE`/`PCAP.TRIG` = `TTLIN1.VAL` there).
- `frame_counter` (`COUNTER1`, clocked directly off `TTLIN1.VAL`'s falling
edge, not off `PCAP`) is unaffected -- still counts every completed
exposure and still works as the completion/grouping signal exactly as
today.
- `gate_detector_active` (`PCAP.BITS0` bit 0, also `TTLIN1.VAL`) would become
trivially always `1` on every captured row, since a row is only ever
captured while that signal is high -- harmless, since `frame_counter`
already does the actual per-exposure grouping, not the gate bit.
Motivation: at 20 kHz continuous (current, ungated behavior), the
free-running raw stream produces roughly 67-86 MB/minute (~4-5 GB/hour) of
raw payload across the 10 captured fields (see
`panda_box_free_running_data_schema.md` for the field list; the range
reflects uncertainty over whether the four `INENC` channels report as
`int32` or `float64`). Gating capture to only actual exposure windows would
cut that in proportion to the detector's duty cycle -- e.g. roughly 10x less
at a 10% duty cycle -- with no BEC-side software change required.
If this is implemented: update this section (and the "Continuous gate/arm"
wiring diagram above) to confirmed status, record which `.ini` layout
captures it, and re-verify the `PCAP.BITS0` bit-0 assignment per the caveat
under "Gate-bit input" above (bit assignment is fixed by the FPGA build, not
guaranteed stable across a layout rebuild).
## Saving / restoring layouts
Use `ophyd_devices/devices/panda_box/utility_scripts.py` to save the current
@@ -172,6 +220,99 @@ The BEC-side data schema produced by Mode B (what ends up in the HDF5 async
dataset) is documented separately in
`docs/developer/panda_box_free_running_data_schema.md`.
## Known data-quality caveat: stale `frame_counter` carryover (found + fixed 2026-09-21)
Diagnosed against a real scan (`S06313`): grouping raw `interf_st_fzp_x`/`_y`
samples by `frame_counter` (restricted to `gate_detector_active == 1`) and
averaging per group reproduces `rt_positions`'s trusted
`average_x_st_fzp`/`average_y_st_fzp` to R² > 0.9999 once a single constant
per-axis offset is removed -- an expected reference-zero difference between
the PandA/`INENC` counter and RT's own internal tracking, irrelevant to
reconstruction since it's the same constant for every point in a scan. This
also confirms the grouping/filtering method itself: naively treating every
raw row as its own position (without grouping by `frame_counter` and
filtering to `gate_detector_active == 1`) does not reproduce the true
positions and cannot be fixed by any scale/offset correction, since most raw
rows are samples taken between real exposures, not at them (see "Continuous
gate/arm" above).
One point out of 472 was a clear outlier under that method: its
`frame_counter` group had 38 extra samples (1036 vs. the usual ~998) and its
recomputed position was off by ~116 nm in Y, against ~5 nm typical.
Root cause: PandA's `COUNTER1.OUT` (the block behind `frame_counter`) is not
cleared by `Arm()`. `COUNTER1.ENABLE=PCAP.ACTIVE` only gates *counting*, not
the persisted output value, and `COUNTER1.SET` -- the block's reset-trigger
input -- is wired to the constant `ZERO` in the saved layout (see
`layouts/omny_freerun.ini`), i.e. never pulsed on arm, unlike `ENABLE` which
*is* wired to a live source (`PCAP.ACTIVE`). So the first raw samples of a new
acquisition can briefly carry a stale `frame_counter` value left over from the
previous acquisition's tail -- observed directly in `S06313`'s raw data: 36
samples reading a stale `167` before dropping to `0` and counting correctly
from there. Harmless on its own, except that stale value collided with a real
point index later in the *same* scan (point 167), so grouping by
`frame_counter` silently mixed ~38 stale rows into that one point's average.
Fix, implemented in `panda_box_omny.py` (software-only, no PandA layout
change needed): every time PCAP is armed -- including `on_pre_scan` retries,
since each `Arm()` reopens the same stale-value window -- `PandaBoxOMNY` now
drops any raw rows captured before it observes `frame_counter == 0` for the
first time in that acquisition (`_drop_stale_frame_counter_rows`, called from
`_receive_frame_data`), and drops any leftover buffered rows from an
abandoned previous attempt at the same point (`_reset_raw_stream_state`,
called from `on_stage` and from `_try_arm` right before every `Arm()`). Every
signal's row list is truncated at the same index to keep channels aligned.
Verified against `S06313`: dropping the (37, in that scan) leading stale rows
before grouping brings point 167's residual from ~116 nm down to ~0.1 nm, in
line with the rest of the scan.
A more thorough fix would rewire `COUNTER1.SET` on the PandA layout itself
(from the constant `ZERO` to `PCAP.ACTIVE`'s rising edge, matching how
`ENABLE` is already wired) so `OUT` resets at the source on every arm, rather
than relying on the client to detect and discard the stale prefix. Not
implemented here since it requires a PandA layout change to be tested against
live hardware; the software-side drop is a correct and sufficient fix in the
meantime, and remains a reasonable defense-in-depth even if the layout is
fixed later. If this is revisited: re-verify against a range of scans
(especially ones with `frame_counter` staying below a few hundred, where a
stale-value collision like point 167's is most likely) before removing the
software-side check.
This also means the "gap in an otherwise-monotonic sequence" completeness
check described in the mode-comparison table's "Data completeness
verification" row needs one caveat: a single backward step at the very start
of the raw sequence, from the previous acquisition's terminal value down to
`0`, is this expected (now-filtered) artifact, not a sign of dropped rows.
## Conditional arming (`arm_scan_allowlist`, added 2026-09-22)
Generic BEC staging (`stage_all_devices`/`pre_scan_all_devices` in
`bec_server.scan_server.scans.scan_actions`) has no concept of "this device
only matters for scan X" -- every `enabled: true` device gets `stage()`/
`pre_scan()`/`unstage()` called for *every* scan that runs (`grid_scan`,
`line_scan`, alignment moves, ...), not just the ones that actually consume
its data. `deviceTags` (e.g. `ptycho_flomni`) are never consulted by that
staging code -- they're purely a client-side grouping convenience. So
without any further gating, `omny_panda_continuous` was being `Arm()`ed and
disarmed for every single scan run during a session, even though only
`flomni_fermat_scan` reads its data. That's unnecessary hardware arm/disarm
cycling, and it directly compounds the stale-`frame_counter` bug above: more
arm cycles mean more windows where a mid-scan abort can leave a stale
counter value for the *next* acquisition to inherit (see
`AI_docs/PANDA_POSITION_VALIDATION.md` sec 7 for a real incident traced to
exactly this).
`PandaBoxOMNY` now takes an `arm_scan_allowlist: list[str] | None` constructor
kwarg (`None` by default -- arm for every scan, unchanged behavior). When
set, `on_stage()` computes `_should_arm_panda` from
`scan_parameters.scan_name` against that list; `on_pre_scan()` skips sending
`Arm()` entirely (resolves immediately) when it's `False`, and `on_complete()`
skips the wait-for-frame-count poll loop the same way. `on_stage()`'s own
readout-thread/TCP-connection startup is untouched by this -- it doesn't
touch PandA's hardware register state (`COUNTER1`), unlike `Arm()`, so it
stays out of scope. `ptycho_flomni.yaml`'s `omny_panda_continuous` sets
`arm_scan_allowlist: [flomni_fermat_scan]`.
## Switching between modes
`omny_panda` (Mode A) and `omny_panda_continuous` (Mode B) need different PandA hardware
+94
View File
@@ -0,0 +1,94 @@
# Fix needed: restore random sub-step offset in Fermat spiral scans (OMNY, flOMNI, LamNI)
**Status:** Implemented on `fixes/flomni_beamtime_2`, pending real/simulated-hardware
verification. All three generators now draw a fresh `np.random.default_rng(seed).uniform(0,
step)` offset per call and subtract it from the raw lattice `x`/`y` before the FOV crop check
(same scalar offset for both axes, matching the legacy spec). Decisions made on the open
questions below: no scan-metadata logging (a `logger.info` line is emitted instead); the
opt-out is a `seed: int | None = None` parameter (default `None` = fresh randomization);
x/y share one scalar offset. The four "preview/estimate" call sites that predict Fermat
point counts before a real scan runs (`flomni.py`/`lamni.py`'s
`_expected_fermat_position_count()`, `tomo_params.py`'s `_compute_fermat_positions_flomni`/
`_compute_fermat_positions_lamni`) now pin `seed=0` so their own repeated-call estimates stay
reproducible; their docstrings note the real scan's count can differ by a point or two near
the threshold because of the randomization. Tests updated to match (`seed=0` on the
comparison calls in `test_fermat_position_warning.py`, `test_tomo_params_widget_math.py`,
`test_lamni_tomo_params_widget_math.py`); full suite passes (725 passed).
**Applies to:** all three setups — **OMNY, flOMNI, and LamNI**. The fix must be applied consistently across all three, not just one.
## Background
Ptychographic tomography ring artifacts arise from any error that is systematically fixed relative to a common coordinate frame across all projection angles (e.g. a local reconstruction inconsistency always occurring at the same scan-grid position). Randomizing the Fermat scan grid's sub-step phase from projection to projection is a known mitigation: it decorrelates such position-tied errors so they don't accumulate into a ring during backprojection.
## What the old spec code did
In the legacy spec macros (`spec/scan_handling.mac`, `_rtscan_point_by_point`, fermat branch), a fresh random offset was drawn **on every call** (i.e. every projection) and used to shift the sampling lattice before cropping to the field of view:
```c
_fermat_offset = step*rand(100)/100 // uniform random in [0, step)
for (ind0=1; ind0<=_arb_num1; ind0++) {
_radius = step/sqrt(PI) * sqrt(ind0)
_angle = 8*PI/(1+sqrt(5))/(1+sqrt(5)) * ind0
tmp_x = radius*cos(angle)
tmp_y = radius*sin(angle)
if (tmp_x in [-(fovx/2)+offset, (fovx/2)+offset] and
tmp_y in [-(fovy/2)+offset, (fovy/2)+offset]) keep it
}
final_x = tmp_x + cenx - _fermat_offset
final_y = tmp_y + ceny - _fermat_offset
```
Net effect (worked out by algebra): the entire Fermat lattice is shifted by `-offset` before cropping to the FOV window, with `offset` redrawn uniformly from `[0, step)` on every scan. So the exact sub-step "phase" of the sampling lattice relative to the sample/FOV origin varies from projection to projection, rather than being bit-for-bit identical every time.
## Current state in BEC (the problem)
None of the three current Fermat spiral generators in `csaxs_bec` draw any random number. All three are fully deterministic given `(cenx, ceny, step, fovx, fovy, ...)`:
| Setup | File | Function |
|--------|------|----------|
| flOMNI | `csaxs_bec/csaxs_bec/scans/flomni_fermat_scan.py` | `get_flomni_fermat_spiral_pos` (staticmethod) |
| OMNY | `csaxs_bec/csaxs_bec/scans/omny_fermat_scan.py` | `get_omny_fermat_spiral_pos` (instance method) |
| LamNI | `csaxs_bec/csaxs_bec/scans/lamni_fermat_scan.py` | `get_lamni_fermat_spiral_pos` (staticmethod) |
All three share the same core loop shape:
```python
for ii in range(start, n_max):
radius = step * 0.57 * np.sqrt(ii)
x = radius * np.sin(ii * phi)
y = radius * np.cos(ii * phi)
# ... FOV/circular-FOV crop check ...
positions.append([x + cenx, y + ceny, z_pos])
```
Since `cenx`/`ceny` come from the per-angle alignment/offset computation (not randomized), every projection at a given nominal center currently samples the **exact same** lattice phase relative to the sample. This is a plausible, currently-unmitigated contributor to ring artifacts in reconstructions from all three instruments.
Note: there is an existing `random.uniform` call in `csaxs_bec/csaxs_bec/bec_ipython_client/plugins/flomni/flomni.py` (`single_point_random_shift_max`, used in `tomo_acquire_at_angle`). That is a **separate, unrelated** mechanism for single-point burst acquisitions — it does not touch the Fermat spiral generators and does not address this issue.
## What needs to happen
1. Add a per-call random sub-step offset to all three spiral generators (`get_flomni_fermat_spiral_pos`, `get_omny_fermat_spiral_pos`, `get_lamni_fermat_spiral_pos`), reproducing the intent of the old spec `_fermat_offset` (uniform draw in `[0, step)`, applied as a lattice-phase shift before FOV cropping).
2. Apply the fix **identically in spirit** across OMNY, flOMNI, and LamNI — the three functions currently share near-identical structure and should stay in sync on this behavior. Watch for the small existing differences between them (flOMNI/LamNI take `cenx`/`ceny` as static args; OMNY reads `self.cenx`/`self.ceny`; LamNI has additional stitching/shift/circular-FOV parameters that the offset must compose correctly with).
3. Decide and document (open questions below) rather than guessing silently.
## Open questions for the implementing session to resolve (and record the decision)
- **Reproducibility/logging:** should the drawn offset be logged into scan metadata (e.g. via `update_scan_info`) so a given projection's exact grid phase can be recovered/audited later, the way `_arb_num1`/positions already are?
- **Opt-out for testing/repeatability:** should there be a parameter (e.g. `randomize_offset: bool = True`, or an explicit `seed`) so existing/new unit tests and commissioning scans can request deterministic output?
- **X/Y coupling:** the original spec used the *same* scalar `_fermat_offset` for both the x- and y-shift. Decide whether to preserve that (simpler, matches historical behavior) or draw independent offsets per axis (stronger decorrelation, but diverges from the reference implementation) — pick one and note the reasoning.
- **Interaction with existing FOV/circular-FOV cropping logic**, especially in LamNI's `_lamni_check_pos_in_fov_range_and_circ_fov` and any stitching offsets — the offset must shift the lattice consistently with those checks, not just the final translation.
## Tests likely needing updates
- `csaxs_bec/csaxs_bec/tests/tests_scans/test_flomni_fermat_scan.py`
- `csaxs_bec/csaxs_bec/tests/tests_scans/test_omny_fermat_scan.py`
- `csaxs_bec/csaxs_bec/tests/tests_scans/test_lamni_fermat_scan.py`
Any test currently asserting exact/deterministic position arrays will need either a fixed seed, the opt-out parameter above, or an update to assert distributional/structural properties instead of exact values.
## Non-goals
- Do not change the underlying spiral density/index-to-radius/angle formula (`radius = step * 0.57 * sqrt(ii)`, golden-angle `phi`) — only the lattice-phase (offset) behavior is in scope.
- Do not touch `single_point_random_shift_max` / `tomo_acquire_at_angle` in `flomni.py` — unrelated mechanism, out of scope here.
+237
View File
@@ -0,0 +1,237 @@
# Plan: `fermat_asymmetry` (x/y spacing ratio) for the Flomni Fermat scan
Status: **implemented on `fixes/flomni_beamtime_2`**, pending real/simulated-hardware
verification. Followed this plan as written: scan class (`FlomniFermatScan`), the
`fermat_asymmetry` global var on `Flomni`, the queue/CLI plumbing (`_TOMO_SCAN_PARAM_NAMES`,
`scan_kwargs`, `tomo_parameters()` print/prompt, `_set_default_tomo_params()`), and the
`tomo_params` Qt widget (field + a "?" help button next to it, following the widget's
existing offsets-section help-button pattern -- `_build_fermat_asymmetry_row`/
`_show_fermat_asymmetry_help`). Tests added to `tests/tests_scans/test_flomni_fermat_scan.py`
verify the inflate/compress math directly (against an equivalent call with an inflated FOV
and `fermat_asymmetry=1.0`) rather than via an approximate spacing-ratio heuristic, which a
first attempt showed is too noisy on a Fermat spiral. Also updated flOMNI's user
documentation (`docs/user/ptychography/flomni.md`) and the OMNY port TODO
(`csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/TODO_fermat_asymmetry_port.md`) to
describe how this composes with the already-landed sub-step randomization (see
`docs/plans/fermat-scan-randomization.md`). Full test suite passes (728 passed).
## Context
The Flomni ptycho Fermat-spiral scan (`FlomniFermatScan`) currently generates
an isotropic spiral: the local point spacing is the same in x and y,
controlled by a single `step` parameter, and points are kept only if they
fall inside the requested field of view (`fovx` × `fovy`). For some samples
it's desirable to bias the spacing — e.g. because the sample or an expected
feature is elongated in one direction — so that the effective spacing in x
differs from y while the requested field of view stays exactly as specified.
This plan introduces `fermat_asymmetry` (ratio = x_spacing / y_spacing,
default `1.0` = symmetric). `>1` means x spacing is larger than y spacing (y
is denser); `<1` is the reverse. Because spacing changes while the FOV area
stays fixed, the total number of positions will change with the ratio.
Scope: the scan argument, the BEC global variable, the queuing path (client
→ queue → scan-server validation → scan execution), the Flomni CLI's
`tomo_parameters()`, and the `tomo_params` Qt widget's Flomni profile. OMNY
and LamNI are out of scope for the code change itself — OMNY is flagged
separately for a future port (see
`csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/TODO_fermat_asymmetry_port.md`);
LamNI is excluded on purpose.
## The math
Current core loop, `FlomniFermatScan.get_flomni_fermat_spiral_pos` (static
method), `csaxs_bec/scans/flomni_fermat_scan.py:410-463`:
```python
length_axis1 = np.abs(m1_stop - m1_start)
length_axis2 = np.abs(m2_stop - m2_start)
n_max = compute_fermat_spiral_n_max(length_axis1, length_axis2, step)
...
for ii in range(start, n_max):
radius = step * 0.57 * np.sqrt(ii)
if abs(radius * np.sin(ii * phi)) > length_axis1 / 2:
continue
if abs(radius * np.cos(ii * phi)) > length_axis2 / 2:
continue
x = radius * np.sin(ii * phi)
y = radius * np.cos(ii * phi)
positions.append([x + cenx, y + ceny, z_pos])
```
This is an isotropic Fermat spiral: `step` controls spacing equally in both
directions. To bias it while keeping x spacing pinned at `step` and the
requested FOV fixed:
- Generate the spiral exactly as today (unchanged x0/y0). Filter x against
the true `fovx` half-width as before. For y, generate/filter against an
**inflated** half-width `length_axis2 * fermat_asymmetry`, then **compress
the accepted y-coordinate back down** by dividing by `fermat_asymmetry`
before storing the final position.
- This keeps x spacing at `step` (untouched), while y spacing becomes
`step / fermat_asymmetry` — "keep x spacing constant, scale y spacing by
the ratio". When `fermat_asymmetry == 1.0` the math reduces exactly to
today's formula (no behavior change for existing callers/tests).
- The FOV corner points appended after the loop (lines 459-462) stay
untouched — they represent the real, requested physical FOV box and must
not be skewed by the asymmetry.
- `compute_fermat_spiral_n_max` (`csaxs_bec/scans/fermat_spiral_utils.py:8`)
must be called with the *inflated* y half-width (`length_axis2 *
fermat_asymmetry`), since that's the actual unscaled extent the loop needs
to reach before compression.
Range: `fermat_asymmetry` must be `> 0` (zero/negative makes the
y-compression nonsensical) — reject that outright via the scan argument's
`gt=0` bound (validated by BEC's `ScanInputValidator` at queue-submit time,
same mechanism as every other `Annotated[..., ScanArgument(...)]`
parameter). Additionally, clamp the practical working range to `[0.1, 10]`
with a warning, the same soft-clamp style already used for `zshift` and
`corridor_size` in this file (`flomni_fermat_scan.py:121-134`). Put this
clamp **inside `get_flomni_fermat_spiral_pos` itself** (not in `__init__`),
because that static method is also called directly — bypassing `__init__`
by the point-count preview helpers (`Flomni._expected_fermat_position_count()`
and the widget's `_compute_fermat_positions_flomni()`, see below). Clamping
at that one shared call site keeps the real scan and every preview in sync
automatically.
## Files to change
### 1. `csaxs_bec/scans/flomni_fermat_scan.py`
- Add class constants next to `MIN_POSITIONS`/`MAX_CORRIDOR_SIZE` (line
67-68): `MIN_FERMAT_ASYMMETRY = 0.1`, `MAX_FERMAT_ASYMMETRY = 10.0`.
- Add `"fermat_asymmetry"` to `gui_config["Scan Parameters"]` (line 54-63).
- Add a new `__init__` parameter (near `step`, line 77):
```python
fermat_asymmetry: Annotated[float, ScanArgument(display_name="Fermat Asymmetry", description="Ratio of x spacing to y spacing (x/y). 1 = symmetric. >1 widens x spacing / tightens y spacing; <1 the opposite. FOV is preserved; point count changes with the ratio.", gt=0, example=1)] = 1.0,
```
Store as `self.fermat_asymmetry = fermat_asymmetry` and document it in the
docstring `Args:` block.
- Pass it through in `prepare_scan()`'s call to `get_flomni_fermat_spiral_pos`
(line 150-161): add `fermat_asymmetry=self.fermat_asymmetry`.
- Update `get_flomni_fermat_spiral_pos` (line 410-463): add
`fermat_asymmetry: float = 1.0` parameter, implement the clamp (with
`logger.warning` when out of `[MIN_FERMAT_ASYMMETRY, MAX_FERMAT_ASYMMETRY]`)
and the x/y-scaling math described above, replacing the direct
`radius*sin`/`radius*cos` filter+store with the generate-against-inflated-
window / compress-on-accept logic.
### 2. `csaxs_bec/bec_ipython_client/plugins/flomni/flomni.py`
- Add a new global-var-backed property next to `tomo_shellstep`
(line 2005): `fermat_asymmetry = _GlobalVarParam(1.0)`.
- Add `"fermat_asymmetry"` to `_TOMO_SCAN_PARAM_NAMES` (line 3900-3914) so
it's snapshotted/restored by `tomo_queue_add()`/`tomo_queue_execute()`
like every other tomo scan parameter (`tomo_queue_mixin.py:625`, `899`).
No migration needed for already-queued jobs: `tomo_queue_execute()` only
sets attributes present in a job's saved `params` dict
(`tomo_queue_mixin.py:898-899`), so older queued jobs simply leave the
live `fermat_asymmetry` value untouched — the same behavior every
previously-added parameter already relies on.
- Thread it into the per-projection scan call, `scan_kwargs` dict
(line 3478-3488): add `fermat_asymmetry=self.fermat_asymmetry`, and into
the accompanying `logger.info(...)` repro string (line 3467-3472) for
consistency with the other logged kwargs.
- Pass it into the preview helper, `_expected_fermat_position_count()`
(line 3638-3646): add `fermat_asymmetry=self.fermat_asymmetry` to the
`get_flomni_fermat_spiral_pos(...)` call.
- Surface it in `tomo_parameters()`:
- print current value near line 3661-3662 (alongside step/FOV).
- add a prompt line near line 3749-3751, following the existing
`self.tomo_shellstep = self._get_val(...)` pattern.
### 3. `csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py` (Qt widget)
The widget is largely data-driven off a few module-level tables plus a
per-setup `SETUP_PROFILES["flomni"]` dict, so most of the wiring is
declarative:
- Add `"fermat_asymmetry"` to `QUEUE_PARAM_NAMES` (line 127-149, the tuple
mirroring `Flomni._TOMO_SCAN_PARAM_NAMES`) and to `DEFAULTS`
(line 151-173): `"fermat_asymmetry": 1.0`. **Do not** touch
`LAMNI_QUEUE_PARAM_NAMES` (line 184) — LamNI is explicitly excluded.
- Add a token builder to the `token_builders` dict in `_build_params_panel`
(~line 463-470), matching the existing `tomo_shellstep` entry's style:
```python
"fermat_asymmetry": lambda: self._add_double(
common_form,
"fermat_asymmetry",
"Fermat asymmetry (x/y)",
min_=0.1,
max_=10.0,
decimals=2,
),
```
- Add `"fermat_asymmetry"` to `SETUP_PROFILES["flomni"]["field_order"]`
(~line 2280-2291), right after `"tomo_shellstep"`.
- Add `"fermat_asymmetry"` to `SETUP_PROFILES["flomni"]["fermat_position_fields"]`
(line 2296) so the existing live point-count preview wiring (the loop at
line 505-508 that connects each listed field's `valueChanged` signal to
`_update_fermat_position_preview`) picks it up automatically — no new
signal-wiring code needed.
- Update `_compute_fermat_positions_flomni()` (line 2162-2189): read
`fermat_asymmetry = params.get("fermat_asymmetry", 1.0)` and pass
`fermat_asymmetry=fermat_asymmetry` into the
`FlomniFermatScan.get_flomni_fermat_spiral_pos(...)` call, so the GUI's
"Estimated Fermat scan points" preview never drifts from the real
algorithm (matches this function's own stated purpose in its docstring).
No other widget code needs to change: `_load_params()` (line 1411-1416) and
`add_edited_to_queue()`'s snapshot (line 1266) both iterate generically over
`self._profile["param_names"]`, and `_job_tooltip()` (line 2432-2458) iterates
generically over `sorted(job["params"])` — all three automatically pick up
`fermat_asymmetry` once it's in `QUEUE_PARAM_NAMES`/`DEFAULTS` and has a
widget in `self._pw`.
**Deliberately left out**: the fixed-column queue table (`_build_ui`,
`QTableWidget(0, 8)`, headers `["#", "Label", "Status", "Type",
"Projections", "Exp (s)", "Step (µm)", "Details"]`, populated around line
1522-1533) is not extended with a new column for `fermat_asymmetry`. Adding
a column means touching header setup, column-width logic, and the
alignment-index checks (`col in (1, 7)`) in the same function — more
surface area for a cosmetic addition. The value is still fully visible via
the row tooltip (`_job_tooltip`, which is generic). Revisit only if this
turns out to be commonly-varied-enough to want at-a-glance visibility.
### 4. Tests — `tests/tests_scans/test_flomni_fermat_scan.py`
Add cases alongside the existing
`test_flomni_fermat_spiral_reaches_elongated_fov_corners`:
- `fermat_asymmetry=1.0` (or omitted) reproduces exactly today's positions
(no-op / backward-compatibility check).
- For `fermat_asymmetry > 1` and `< 1`, the ratio of median nearest-neighbor
spacing along x vs. along y (via `scipy.spatial.cKDTree`, same technique
already used in `_estimate_corridor_size`) approximately matches the
requested ratio, while `max(abs(x)) <= fovx/2` and `max(abs(y)) <= fovy/2`
still hold.
- A value outside `[0.1, 10]` (e.g. `100`) is silently clamped rather than
raising, and does not change the FOV bounds.
## Verification
1. `cd /sls/x12sa/config/bec/production/csaxs_bec && python -m pytest tests/tests_scans/test_flomni_fermat_scan.py -v`
2. Sanity-check the position generator directly for a few ratios:
```python
from csaxs_bec.scans.flomni_fermat_scan import FlomniFermatScan
for r in (0.5, 1.0, 2.0):
pos = FlomniFermatScan.get_flomni_fermat_spiral_pos(-10, 10, -5, 5, step=1, fermat_asymmetry=r)
print(r, len(pos), pos[:, 0].max(), pos[:, 1].max())
```
3. In a BEC client session against the Flomni deployment: confirm
`flomni.fermat_asymmetry` reads/writes via the global var (persists
across a client restart), shows up in `flomni.tomo_parameters()`, and
that `scans.flomni_fermat_scan(..., fermat_asymmetry=2.0)` is accepted by
the queue (client-side `ScanInputValidator` and server-side
`ScanAssembler` both validate against the new `ScanArgument`).
4. Open the `tomo_params` widget for the flomni profile: confirm the new
"Fermat asymmetry (x/y)" field appears in the parameter panel in the
right position, edit it, confirm the "Estimated Fermat scan points"
preview updates live, and confirm that queuing/loading a job round-trips
the value correctly.
## Out of scope
- `OmnyFermatScan` / `OMNY` — flagged for a future port, see
`csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/TODO_fermat_asymmetry_port.md`.
- `LamniFermatScan` / `LamNI` — explicitly excluded, not planned.
+166
View File
@@ -0,0 +1,166 @@
# Plan: improve the `fsamx` PID centering in `move_samx_to_scan_region`
Status: **DRAFT — NOT FINAL, NEEDS FURTHER DISCUSSION. Not implemented.**
Written during live beamtime operation on request; no code changes made — see
the conversation this plan came from. It rests on one day of logs
(2026-09-18) and small samples (18 first-move cases, 25 later-iteration steps)
and has not been tested on the instrument. The open questions at the bottom
must be settled before any code is changed.
## Context
`RtFlomniController.move_samx_to_scan_region`
(`csaxs_bec/devices/omny/rt/rt_flomni_ophyd.py`, ~lines 181-230) centres
`fsamx` after the rt stage has moved to `cenx`. It reads the PID X voltage
(`get_pid_x()`, `g` command on the rt controller, range ±10 V) and compares it
with the expected voltage (`rtx` user parameter `rt_pid_voltage`, stored by
`feedback_enable_with_reset`).
- If `|V - expected|` is below the start threshold: `No correction of fsamx needed`.
- Otherwise loop until `|V - expected| < 1`, each iteration doing
`pid_x_correction -= (V - expected) * 0.006` and moving `fsamx` to
`fsamx_in + cenx/1000*0.7 + pid_x_correction`, then `laser_tracker_on()`.
**Trigger for this plan.** The start threshold was lowered from 2.0 V to 1.2 V
(uncommitted edit in the production working tree when this was written) with
the aim of reducing the target range below 0.05 V. Afterwards corrections rose
from about 8.5 to about 49 per hour and the readings still swung by about
±1-2 V.
**Data source.** `/var/log/bec/DeviceServer.log` and `ScanServer.log` on
`x12sa-bec-001.psi.ch`, 2026-09-18 08:00-20:19. These logs are not part of the
repo. The relevant log messages come from the `rt_flomni_ophyd` logger
(`Current PID voltage`, `Moving fsamx to …, PID portion of that …`,
`No correction of fsamx needed`, `No further correction needed`).
## Findings this plan rests on
1. **The loop gain is too small, not too large.** The measured stage response
is 13.4 µm/V (rms 0.18 V over 25 steps at constant `cenx`), while the code
uses `0.006` mm/V = 6 µm/V. Each step therefore removes only about 45 % of
the error (measured contraction ×0.57, predicted ×0.55). Going from 1.2 V to
below 0.05 V takes about 6 iterations, each followed by a laser tracker
wait. At constant `cenx` the loop does not overshoot.
2. **The first move of a sequence double counts.** It applies the `cenx`
feed-forward jump (`0.7*Δcenx` since the last move, median 35 µm) and a
correction increment `-0.006*(V-e)` computed from a reading that already
contains that `cenx` lag. The sign of V flips after the first move in 12 of
18 cases, e.g. V = -2.44 V with a +36.7 µm feed-forward jump and a +14.6 µm
correction step gives V = +1.82 V. A two-term fit suggests the true
feed-forward factor is nearer 0.8 than 0.7 (weak evidence, n = 18).
3. **Blind steps at the rail overshoot.** In four cases (12:25, 13:56, 18:42,
19:04) a scan stop was followed by a reading pinned at -10.0 V. The loop
then made three fixed +60 µm moves (`0.006 * 10 V`), the reading flipped to
+10.0 V, and the laser tracker waited 37-100 s for the target. Twice the
wait exceeded 100 polls and raised
`RtError: Failed to reach laser target position`.
4. **`pid_x_correction` is lost on reload.** After the 19:30 device reload it
restarted from 0 (first PID part -0.009), while `fsamx` itself stayed where
it was.
5. **About 1 V of scan-to-scan alternation is unexplained.** Consecutive scan
points alternate high/low with a period of about 46 s. It was present all
day, including before the threshold change (08:00-08:30, 10:00-16:00). The
`cenx` and last-`fsamx`-position model leaves a residual of about 1 V
(R² 0.67 before 19:30, 0.30 after 19:36). This is a rotation-tomography
run: every scan point is a new `angle` and `cenx` follows it.
6. **The `Expected PID voltage` log line is commented out**
(`rt_flomni_ophyd.py:196`). The expected value only appears once per
restart as `Using PID voltage from rtx user parameter`.
## The math
Notation: `V` = PID voltage, `e` = expected voltage, `s` = stage response
(mm/V), `x` = `fsamx` position relative to `fsamx_in` (mm), `c` = `cenx`.
Plant (fits the in-loop data to 0.18 V rms):
```
V - e = (x - x*(c)) / s, x*(c) = 0.7*c/1000 + drift, s ≈ 0.0134 mm/V
```
Current step: `Δx = -0.006 * (V - e)`, i.e. gain `g = 0.006` and per-step error
ratio `r = 1 - g/s ≈ 0.55`. To reach a residual `ε` from `V0` needs
`n = ln(ε/V0) / ln(r)` steps: 6 steps for 1.2 V → 0.05 V.
Proposed step (Newton, damped): `Δx = -DAMP * s * (V - e)` with
`DAMP ≈ 0.85`. With a 15 % gain error the ratio is about 0.15, so 1.5 V → 0.03 V
in two iterations. `s` may be adapted from the last two iterations
(`s_est = Δx_prev / (V_prev - V_now)`), clamped to 8-20 µm/V.
Double counting: on the first move `V - e` already contains `-(0.7*(c - c_prev)/1000)/s`,
which the feed-forward jump removes. Adding `-0.006*(V-e)` on top overshoots.
## Proposed changes (all in `move_samx_to_scan_region`)
1. **Newton step** as above instead of the fixed small gain.
2. **Remove the double counting.**
- **Option A (preferred):** compute the target from the actual position,
`x_target = x_now - DAMP*s*(V-e)`, and store
`pid_x_correction = x_target - 0.7*cenx/1000`, so the `cenx` lag is never
accumulated as drift.
- **Option B:** apply the feed-forward move first, re-read V, then correct.
Simpler, but costs about 1 s plus a tracker wait per correction.
3. **Derive `x_now` from the `fsamx` readback** instead of a running sum, so a
reload does not reset the bookkeeping.
4. **Clamp steps at the rail.** When `|V| >= 9.9`, cap the step at about 30 µm
and re-read instead of making blind ±60 µm steps.
5. **Hysteresis.** Start about 0.5 V, stop about 0.15 V (currently start 1.2 V,
stop 1.0 V, so the next `cenx` step re-triggers immediately).
6. **Logging.**
- Restore the `Expected PID voltage` line next to each `Current PID voltage`.
- Log `angle`, `cenx`, V and the `fsamx` readback per scan point.
- Log `laser_tracker_show_all()` output when `laser_tracker_wait_on_target`
times out (the tracker's beam position, target and piezo state are not
logged today).
Sketch (not tested):
```python
S, DAMP, FF, VCLIP = 0.0134, 0.85, 0.7e-3, 8.0 # mm/V, -, mm per cenx unit, V
x_now = fsamx.obj.readback.get() - fsamx_in # actual position (mm)
for _ in range(MAX_ITER):
v = self.get_pid_x() - expected_voltage
if abs(v) < V_STOP:
break
x_now -= DAMP * S * np.clip(v, -VCLIP, VCLIP)
fsamx.obj.pid_x_correction = x_now - FF * cenx # drift not explained by cenx
fsamx.obj.move(fsamx_in + x_now, wait=True)
time.sleep(0.1)
self.laser_tracker_on()
```
## What this will not fix
The unexplained ~1 V scan-to-scan alternation (finding 5) is not addressed. A
0.05 V threshold would chase it with a move on nearly every scan point, and each
move costs a laser tracker wait. The plan therefore assumes about 0.15 V as the
stop level until the alternation is understood.
## Verification (once approved)
- **Offline replay:** run the proposed step logic against the logged
(V, `cenx`, position) sequences from 2026-09-18 and compare iterations per
correction and residual V with what happened.
- **Bench check:** on the instrument, force a known offset of about 2 V and
confirm convergence to below 0.15 V in at most 3 iterations with no sign
flip; repeat with about 5 V.
- **Rail case:** stop and restart a scan mid-way and confirm no ±10 V
overshoot and no tracker timeout.
- **Rate check:** confirm corrections per hour do not rise above the
8.5/hour baseline.
- **Unit tests:** the step function should be a pure function so it can be
tested without hardware (see `tests/tests_devices`).
## Open questions for discussion
1. Is a threshold of 0.05 V realistic while the alternation is unexplained? The
plan assumes about 0.15 V for the stop level.
2. What is the true feed-forward factor: 0.7 (code) or about 0.8 (fit)?
3. Option A or B for the double counting?
4. Should the alternation be diagnosed first (angle dependence, backlash, the
`v0`/`v1` FZP feedback toggle) before any code change?
5. Is the 13.4 µm/V response stable across days? Only 2026-09-18 was analysed;
the rotated logs from 2026-09-11 to 2026-09-17 could be checked.
6. Should the already-made 1.2 V threshold change be kept, reverted, or left
until this is settled? It is currently uncommitted in the production working
tree.
+120
View File
@@ -0,0 +1,120 @@
# Plan: "Skip" button on the X-ray-eye alignment widget (flomni: leave out one angle, e.g. 90 deg)
Status: **implemented 2026-09-21** on a BEC test server (tests: `tests_bec_ipython_client/test_flomni_xray_eye_skip.py`,
`tests_bec_widgets/test_x_ray_eye_skip_button.py`); tested live in the flomni simulator (works, ok'd by Ana); **not yet tried on real hardware** (see Verification).
Written after a read-only stability study of the DAP sine fit with 4 instead of 5 angles.
Deviations from the plan below: the widget couples the Skip button to the Submit button
(`enable_skip_button(True)` only takes effect while Submit is enabled), so the client calls it
just once per run phase instead of next to every `enable_submit_button`. A refused skip re-sends
the pending step prompt. The h5 `alignment_values` dataset got a `step_k` attr and
`alignment_fit` a `skipped_angles_deg` attr. Skipping 0/180 is warned about, not forbidden.
## Context
Flat samples are edge-on at 90 deg, so their centre cannot be submitted there. The flomni X-ray-eye
alignment (`XrayEyeAlign._align_impl` in
`csaxs_bec/bec_ipython_client/plugins/flomni/x_ray_eye_align.py`) submits 5 angles
0/45/90/135/180. The GUI fit tab (`OMNY_XRayEye` in
`csaxs_bec/bec_widgets/widgets/xray_eye/x_ray_eye.py`) fits
`A*sin(f*x + shift) + intercept` through the DAP server (`LmfitService1D`, model
`SineModel + LinearModel`, `f` fixed to 1 deg, start values `SINE_FIT_DAP_PARAMETERS`), i.e. 3 free
parameters. With 4 points there is 1 degree of freedom (today 2).
Wanted: a **Skip** button on the widget. One position may be skipped as long as at least 4 values
remain for the fit; otherwise error.
## Stability study (does the fit still work with 4 points?)
**Yes.** Method: (1) linear noise propagation for several angle sets, (2) Monte Carlo through the
real `LmfitService1D` (`configure(data_x=, data_y=, parameters=SINE_FIT_DAP_PARAMETERS,
oversample=5)` then `process()`, in `bec_venv`) with the widget's exact start values, compared with
the global least-squares optimum. Truth `A*sin(rad(angle)+phase)+C` plus Gaussian noise. There is no
real alignment data on disk, so the click noise sigma is unknown (pixel = 0.44 um, hand-placed ROI
centre, probably 1-3 um); results are in units of sigma.
- ~49000 fits, **0 failed fits**.
- RMS error of the fitted correction curve over 0-180 / 0-360 deg (units of sigma):
| angles used | cond. number | 0-180 | 0-360 |
|---|---|---|---|
| 0,45,90,135,180 (today) | 3.11 | 0.71 | 1.12 |
| skip 90: 0,45,135,180 | 3.23 | 0.88 | 1.29 |
| skip 45 or 135 | 2.87 | 0.82 | 1.17 |
| skip 0 or 180 | 5.36 | 1.02 | 1.91 |
Monte Carlo at sigma = 1 um agrees (skip 90: median 0.72 sigma vs 0.61 sigma, p95 1.6 vs 1.2).
So skipping 90/45/135 is cheap (+15..25% noise); skipping an end point (0/180) is clearly worse
but still usable.
- Optimizer misses versus the global optimum only occur for a large offset |C| >~ 45 um (start
intercept C = 0 is far away): 15/4000 in that regime for both 4 and 5 points, 0/4000 for
|C| <= 20 um. A property of the start values, not of the point count. Optional hardening: seed
the intercept from the data mean (the static widget parameters cannot do that themselves).
- What is really lost with 4 points is **redundancy**: 1 dof, the residual has one fixed pattern
(0.71, -1, 1, -0.71 for 0,45,135,180), so chi-square / the plot cannot tell which click was bad,
and a mis-click at 0/180 (leverage 0.83) is mostly absorbed into phase/offset.
- Other 4-point layouts (0,60,120,180: 0.80 sigma; 0,30,150,180: 1.17 sigma). Extending beyond
180 deg is the most robust option if the flat-sample geometry and fsamroy travel allow it:
0,45,135,225 gives 0.84 / 0.93 sigma (0-180 / 0-360).
## Decision: what does Skip send to the client script?
**Reuse the existing `submit` signal with a sentinel: `omny_xray_gui.submit.set(-1)`**
(0 = idle, 1 = submit, -1 = skip). The widget writes no `xval_x_k / yval_y_k / width_*_k` for a skip.
- The client already polls exactly this one signal (`submit.get() == 1`), so there is no new polling
path and a skip is atomic (no race between "values written" and "flag set").
- No device change. A new `skip` `Signal` on `OMNYXRayAlignGUI`
(`csaxs_bec/devices/omny/xray_epics_gui.py`) would need a device-server reload; `submit` exists.
- No NaN/stale-data trick. `submit = 1` with NaN in `xval_x_k` would make every consumer
(`alignment_values`, `update_fov`, `roi_pixel_data`) responsible for noticing it, and the stale
values of an earlier run would be read if one forgets. With -1 the client never reads that step.
Rejected: separate `skip` signal (cleaner, but needs device reload), NaN payload (implicit,
error-prone).
## Implementation plan
**Widget** (`x_ray_eye.py`; shared with LamNI and omny, so it must be inert for them):
- `self.skip_button = QPushButton("Skip", ...)` under the Submit button, **disabled by default**.
Its slot does `self.dev.omny_xray_gui.submit.set(-1)` (no ROI needed, unlike `submit()`).
- New RPC `enable_skip_button(enable: bool)`: add to `USER_ACCESS` next to `enable_submit_button`
and regenerate `csaxs_bec/bec_widgets/widgets/client.py` (auto-generated by `generate_cli.py`).
LamNI/omny never call it, so the button stays disabled there.
**Client** (`flomni/x_ray_eye_align.py`):
- Loop: `submit_val = dev.omny_xray_gui.submit.get()`; act on `1` or `-1`; reset to 0 as today.
- `MIN_FIT_POINTS = 4`; track `self.skipped_steps`. A skip is legal only for real fit points (steps
1-5 **after** `_height_centered`; not k = 0 FZP centre, not the height-centering submission at
k == 1) and only while `5 - n_skipped - 1 >= MIN_FIT_POINTS`. Otherwise do nothing to the run,
`send_message("Cannot skip: at least 4 positions are needed for the fit")` and keep waiting, so
the error is immediate and not after the whole rotation series. Call `enable_skip_button(False)`
once the budget is used and `True` wherever `enable_submit_button(True)` is called for steps 1-5.
- Skip branch: no `alignment_values[k]`, store a NaN / "skipped" row in `roi_pixel_data`, **no
`update_fov(k)`** (would read stale widths), then advance like a submit: for `1 <= k < 5` rotate
to the next angle (`tomo_rotate(k * 45)`, `update_frame`, next step message); at `k == 5` finish
(`hide_crosshair`, `break`) without `update_fov`.
- `write_output()`: build angles/offsets only from non-skipped steps (e.g. `[0,45,135,180]`);
final guard `if len(angles) < MIN_FIT_POINTS: raise`. `submit_fit_array` and the fit tab are
already length-agnostic. Record the skipped angles in the h5 (`alignment_fit` attrs).
- Messages: with 4 points "1 dof: the fit cannot flag a bad click - check the plot"; extra warning
if the skipped angle is 0 or 180 (noisier fit). Step numbering "k/5" stays.
## Verification
- Mock/`test_wo_movements=True` run of the client loop with a fake `submit` sequence: (1,1,-1,1,1)
ends with 4 fit points; a second -1 is refused with a message; -1 at k == 0 or at the
height-centering submission is refused; -1 at the last step finishes cleanly.
- `write_output()` gives the 4-point `angles`/`offsets`; the fit tab plots 4 points; DAP parameters
arrive in `omny_xray_gui.fit_params_x`; `flomni.read_alignment_offset(get_data_from_gui=True)`
loads sane `tomo_alignment_fit[0]`.
- Regression: the default 5-submit run is unchanged; LamNI/omny show Skip disabled.
- Hardware: dry run at the beamline with a flat sample, skipping 90 deg.
## Open questions
- Real click noise sigma (no saved alignment data on the analysis machine).
- Should skipping 0/180 be forbidden instead of only warned about?
- Is an angle beyond 180 deg possible for flat samples (better 4-point layout)?
+6 -5
View File
@@ -1,10 +1,11 @@
# Plan: Manual exposure / auto-gain control for IDSCamera + xrayeye widget knobs
Status: **implemented on `feat/ids-camera-manual-exposure`, pending real-hardware
verification** (`IDSCamera` and the `OMNY_XRayEye` GUI widget are both used in
production during beamtimes; this change should be reviewed and verified against
real hardware outside a live beamtime -- see `csaxs_bec/device_configs/test_ids_camera_41.yaml`
for a single-camera (ID 41, color) config for that purpose -- before merging).
Status: **done — implemented and merged into `main`**, after four rounds of
real-hardware verification (see the addenda below for the bugs found and fixed
each round: startup auto-exposure/gain never actually engaging, a zero-ms
exposure floor, auto-gain being more useful as a one-time startup pulse than a
persistent toggle, and the pixel-clock slider needing to be index-based over
the camera's real discrete list instead of a ranged value).
## Context
+145
View File
@@ -0,0 +1,145 @@
# Plan: Auto-lock vertical center of the sample selection box to the FZP height (flomni X-ray eye alignment)
Status: **not implemented** (written during live beamtime operation on request; no
code changes made — see conversation this plan came from).
## Context
The flomni X-ray eye alignment procedure (`XrayEyeAlign._align_impl` in
`csaxs_bec/bec_ipython_client/plugins/flomni/x_ray_eye_align.py`) works in two phases:
1. **Step 0** — the user draws a selection box around the FZP and submits it
(`xval_x_0`/`yval_y_0`). This position is stored and a fixed crosshair
(`TargetCrosshair`, drawn via `self.gui.set_crosshair_position()` /
`show_crosshair()`) is placed on the live view at that FZP center — the
horizontal line of that crosshair marks the FZP's vertical (`y`) position.
2. **Height centering** — the user draws one more box around the sample and
submits it (still logically "step 1"). The script computes
`delta_y_mm = HEIGHT_CENTERING_SIGN * (fzp_center_y - mark_y) * pixel_calibration`
and moves `fsamy` so the sample lines up with the FZP height on camera
(`x_ray_eye_align.py:307-340`). `self._height_centered` flips to `True`
right after this move.
3. **Steps 1-5 (angles 0°/45°/90°/135°/180°)** — for each rotation angle, the
user draws a new box around the sample and submits it; only the box's `x`
center is used (`alignment_values[k]`, used later to compute per-angle
fovx offsets). The vertical (`y`) position of these boxes is not used for
anything, but the user currently still has to eyeball/re-draw it near the
right height each time.
Once `_height_centered` becomes `True`, the sample's vertical position is
physically fixed for the rest of the run — it should equal the FZP's `y`
(the same value the crosshair's horizontal line is drawn at). The goal is
for the selection box in steps 1-5 to no longer require manual vertical
placement: draw the box anywhere, and its vertical center snaps to the FZP
height automatically, staying correct even if the user drags or resizes it
further, right up until they hit submit.
This must **not** affect the box drawn during the height-centering step
itself (the one submission where `k == 1 and not self._height_centered`) —
that box's vertical position is the actual measurement the height
correction is computed from, so it must remain fully free.
## Design
Add an explicit widget-side "vertical lock" flag, toggled by the flomni
script via a new RPC method pair on `OMNY_XRayEye`, mirroring the existing
`show_crosshair()`/`hide_crosshair()` pattern (`x_ray_eye.py:238-261` for the
USER_ACCESS declarations, `x_ray_eye.py:833-874` for implementations). This
avoids adding a new EPICS/ophyd `Signal` and avoids ambiguity from
overloading the existing `step` counter (confirmed by reading
`x_ray_eye_align.py`: `step` reads `1` for *both* the height-centering box
and the first real angle-0° box, so `step` alone can't distinguish "lock"
from "don't lock").
- New widget state: `self._vertical_lock_enabled: bool = False`, initialized
alongside `self.target_crosshair` (`x_ray_eye.py:299`).
- New RPC methods on `OMNY_XRayEye` (USER_ACCESS, same section as
`show_crosshair`/`hide_crosshair`): `lock_vertical_center()` /
`unlock_vertical_center()`. They just set `self._vertical_lock_enabled`.
- New helper `_snap_roi_vertical(self, roi)`:
- No-op if `not self._vertical_lock_enabled` or the crosshair isn't
visible.
- Otherwise reads the crosshair's fixed `y` (the same value used by
`crosshair_position()`, `x_ray_eye.py:868-874`), reads the roi's current
`x` origin and height via `roi.pos()` / `roi.get_coordinates(typed=True)`,
computes `new_origin_y = fzp_y - height / 2` (ROIs are positioned by
bottom-left origin + size, not by center — confirmed via
`bec_widgets/widgets/plots/roi/image_roi.py`), and calls
`roi.set_position(current_x, new_origin_y)`. Guard against signal
re-entrancy (block signals or a re-entrancy flag) while calling
`set_position`, since this will itself be triggered from a
region-changed signal.
- Wire this helper into two places:
- `_style_new_roi` (`x_ray_eye.py:630-636`, connected to
`self.roi_manager.controller.roiAdded`) — call it right after the ROI is
styled, so a freshly-drawn box snaps immediately (`roiAdded` already
fires only after the user finishes drawing/releases the mouse, so this
naturally happens post-draw, not mid-drag).
- Each new ROI's `sigRegionChangeFinished` (pyqtgraph's inherited
"drag/resize finished" signal) — connect it in the same place `roiAdded`
is handled, so any subsequent manual drag or resize snaps back to the
correct vertical center as soon as the user releases the mouse, without
fighting the live drag. (No extra visual affordance beyond this — the
box stays draggable/resizable in both axes, it's just always corrected
back to the right vertical center on release.)
### Script-side wiring (`x_ray_eye_align.py`)
- Call `self.gui.lock_vertical_center()` right after
`self._height_centered = True` is set (line 334), before the "Step 1/5"
prompt — so every box from here through step 5 is auto-centered
vertically. The height-centering box itself (processed just before this
line) is unaffected since the lock isn't enabled yet at that point.
- Call `self.gui.unlock_vertical_center()`:
- At the start of a fresh run, alongside `self._reset_init_values()`
(`x_ray_eye_align.py:196`) or where the crosshair is implicitly reset,
so a new alignment run doesn't inherit a stale lock.
- In `align()`'s `finally` block (`x_ray_eye_align.py:172`), alongside the
existing `self.gui.hide_crosshair()` cleanup call, so the lock never
outlives a run (including on error/interruption).
LamNI and OMNY have their own near-identical `x_ray_eye_align.py` scripts
but different height/rotation procedures — out of scope here. The widget
change is inert by default (`_vertical_lock_enabled` starts `False` and is
only ever set by a script that explicitly opts in), so it's safe to leave
their scripts untouched.
## Files to change
- `csaxs_bec/bec_widgets/widgets/xray_eye/x_ray_eye.py`
- Add `self._vertical_lock_enabled` state near `self.target_crosshair`
init (~line 299).
- Add `lock_vertical_center()` / `unlock_vertical_center()` USER_ACCESS
RPC methods (declare near ~238-261, implement near ~833-874, next to the
crosshair show/hide methods).
- Add `_snap_roi_vertical(self, roi)` helper.
- In `_style_new_roi` (~630-636): call `_snap_roi_vertical(roi)` after
existing styling, and connect `roi.sigRegionChangeFinished` to
`lambda: self._snap_roi_vertical(roi)`.
- `csaxs_bec/bec_ipython_client/plugins/flomni/x_ray_eye_align.py`
- Add `self.gui.lock_vertical_center()` call after line 334
(`self._height_centered = True`).
- Add `self.gui.unlock_vertical_center()` calls at run start (near line
196) and in `align()`'s `finally` block (near line 172).
## Verification
Since this is real beamtime hardware/procedure code, verify without
disrupting operation:
1. Read back the diff carefully against the exact step sequence traced above
(k=0 FZP, k=1 first pass height-centering, k=1 second pass angle-0°,
k=2..5 angles 45°-180°) to confirm the lock is enabled exactly once,
right after `_height_centered` flips, and disabled on both run-start and
run-teardown paths.
2. If a test/offline instance of the GUI is available (`test_wo_movements`
flag on `XrayEyeAlign`, or a non-production BEC session), run through a
full alignment: confirm the FZP box and height-centering box are
completely unaffected (free vertical placement, `delta_y_mm` computed
correctly), then confirm boxes drawn for steps 1-5 immediately snap their
vertical center to the crosshair's horizontal line, and stay snapped
after manual drag/resize.
3. Confirm no infinite signal loop / visual jitter when dragging or resizing
a locked box (guard against `_snap_roi_vertical`'s own `set_position`
call re-triggering `sigRegionChangeFinished`).
4. Only test live on the actual flomni beamline once available and outside
critical beamtime, per standing practice for this codebase.
+29 -2
View File
@@ -56,6 +56,8 @@ If you see your sample already at the approximately correct height:
1. `flomni.xrayeye_alignment_start()` start the coarse alignment of the sample by measuring (clicking in the X-ray eye software) the sample position at its height and then angles of 0, 45, 90, 135, 180 degrees. The GUI will present a fit of this data, which is automatically loaded to BEC for aligning the sample.
**Flat samples** are edge-on at 90 degrees, so their center cannot be measured there: press **Skip** in the GUI instead of Submit to leave that angle out. One angle may be skipped, the fit needs at least 4 of the 5 angles; a second Skip is refused with a message. Skipping 45, 90 or 135 degrees costs little accuracy, skipping 0 or 180 degrees makes the fit noticeably noisier (a warning is printed). The FZP center and the height marking cannot be skipped. With 4 points the fit cannot flag a bad click, so check the plot. The skipped angles are stored in the alignment h5 file.
Otherwise adjust the height manually:
1. `flomni.umvr_fsamy_tracked(0.01)`, attention: unit <mm>, move the sample stage relative up (positive) or down (negative) until the sample is approximately vertically centered in xray eye screen
@@ -64,7 +66,7 @@ Otherwise adjust the height manually:
#### Fine alignment
After the xrayeyealign, a fine alignment needs to be performed using ptychography.
_To bypass the fine alignment: `flomni.feye_out`_
_To bypass the fine alignment: `flomni.feye_out()`_
1. `flomni.tomo_parameters()` Adjust the ptychographic scan parameters for performing an alignment scan. Typically FOVX = FOVX(Xrayeye)+20 mu, shell step = beamsize/2.5, number of projections and tomo mode are ignored in the alignment scans.
@@ -223,15 +225,38 @@ The basic scan function can be called by `scans.flomni_fermat_scan()` and offers
| cenx (float) | center position in x |
| ceny (float) | center position in y |
| step (float) | stepsize |
| fermat_asymmetry (float) | ratio of x spacing to y spacing (x/y). 1 = symmetric (default). x spacing stays fixed at `step`; y spacing scales by this ratio — e.g. `2.0` doubles x spacing relative to y (denser sampling in y), `0.5` does the opposite. The requested FOV is preserved, so the total point count changes with the ratio. Soft-clamped to `[0.1, 10]`. Useful for samples/features elongated in one direction. |
| zshift (float) | shift in z |
| angle (float) | rotation angle (will rotate first) |
| corridor_size (float) | corridor size for the corridor optimization. Default 3 um (auto-estimated if not provided) |
| randomize_offset (bool) | randomize the sub-step offset of the Fermat lattice on every scan. Default `True`. Set `False` for the unshifted lattice (identical positions on every call). See below. |
| exp_time (float) | exposure time per frame |
| frames_per_trigger (int) | Number of burst frames per position |
| burst_at_each_point (int) | Number of triggers and readouts at each point |
Example:
`scans.flomni_fermat_scan(fovx=20, fovy=25, cenx=0.02, ceny=0, zshift=0, angle=0, step=0.5, exp_time=0.01, frames_per_trigger=1)`
`scans.flomni_fermat_scan(fovx=10, fovy=10, cenx=0, ceny=0, step=1, fermat_asymmetry=1.0, zshift=0, angle=0, exp_time=0.1, frames_per_trigger=1, burst_at_each_point=1)`
**Effective spacing with `fermat_asymmetry`:** the x spacing always stays at `step`;
the y spacing becomes `step / fermat_asymmetry`. So with `step=1 um`:
| fermat_asymmetry | x spacing | y spacing | points (fovx=fovy=10 um example) |
| --- | --- | --- | --- |
| 0.7 | 1.0 um | ~1.43 um (sparser) | ~70 |
| 1.0 (default) | 1.0 um | 1.0 um | ~100 |
| 1.3 | 1.0 um | ~0.77 um (denser) | ~130 |
The requested FOV stays exactly as specified either way -- only the point density
(and therefore the point count and scan duration) changes.
By default the Fermat spiral's sub-step phase is randomized on every projection
(drawn fresh per scan, uniform in `[0, step)`) to decorrelate position-tied
reconstruction errors that would otherwise accumulate into ring artifacts across a
tomographic series. Pass `randomize_offset=False` to switch this off and get the
unshifted lattice, e.g. for reproducible positions when comparing scans:
`scans.flomni_fermat_scan(fovx=10, fovy=10, cenx=0, ceny=0, step=1, randomize_offset=False, ...)`.
The tomo scans (`tomo_scan`, `tomo_scan_projection`) do not pass this parameter and
therefore always randomize.
`frames_per_trigger` controls burst acquisition for detectors generally.
Separately, the OMNY PandABox is configured as one of two distinct devices in
@@ -354,6 +379,8 @@ Three modes for angular sampling are implemented and they have different optiona
For the "8 sub-tomograms" mode, `flomni.tomo_parameters()` also offers a `zero_deg_reference_at_each_subtomo` option: when enabled, an additional reference projection at exactly 0 degrees is acquired at the start of every odd (forward) sub-tomogram and once more after the final sub-tomogram, useful for tracking radiation damage across the full measurement.
`flomni.tomo_parameters()` also offers `stitch_x`/`stitch_y` (integers, default 0, up to 50 each) for a field of view larger than a single Fermat scan can cover: when nonzero, every projection angle is acquired as a `(2*stitch_x+1) × (2*stitch_y+1)` grid of separate Fermat scans instead of a single one, tiled around the normal alignment-corrected center and spaced by `fovx`/`fovy` minus `tomo_stitch_overlap` (µm, default 0.2) per step in x/y — stitching together an effective field of view of roughly `(2*stitch_x+1) × fovx` by `(2*stitch_y+1) × fovy`, with `tomo_stitch_overlap` controlling the deliberate overlap between adjacent tiles used for aligning them together during reconstruction. Stitching is forced to 0 (disabled) whenever `single_point_instead_of_fermat_scan` is enabled.
The parameters above can be used to __restart an interrupted acquisition__ manually, or - more conveniently - by running
`flomni.tomo_scan_resume()`
which reads the last recorded progress and resumes automatically at the exact point (subtomogram/angle, or projection for the golden ratio modes) the scan was interrupted at, without needing to look up the values by hand. When running from the tomo scan queuing system use `flomni.tomo_queue_execute()` instead!
+2
View File
@@ -111,6 +111,8 @@ which reads the last recorded progress and resumes automatically at the exact po
In case of eight equally spaced sub-tomograms, an individual sub-tomogram can be scanned by `lamni.sub_tomo_scan(subtomo_number, start_angle)`. If the start angle is not specified, it is computed depending on `subtomo_number`, ranging from 1 to 8.
`lamni.tomo_parameters()` also offers `lamni_stitch_x`/`lamni_stitch_y` (integers, default 0, up to 50 each) for a field of view larger than a single scan can cover: when nonzero, every projection angle is acquired as a `(2*lamni_stitch_x+1) × (2*lamni_stitch_y+1)` grid of separate Fermat scans instead of a single one, tiled around the normal alignment-corrected center and spaced by `lamni_piezo_range_x`/`lamni_piezo_range_y` minus `tomo_stitch_overlap` (µm, default 0.2) per step in x/y — stitching together an effective field of view of roughly `(2*lamni_stitch_x+1) × lamni_piezo_range_x` by `(2*lamni_stitch_y+1) × lamni_piezo_range_y`, with `tomo_stitch_overlap` controlling the deliberate overlap between adjacent tiles used for aligning them together during reconstruction. Same mechanism as flomni's `stitch_x`/`stitch_y`, just under lamni's own parameter names (see `lamni.tomo_scan_projection()`, which passes `stitch_x`/`stitch_y`/`stitch_overlap` straight through to `scans.lamni_fermat_scan()`).
#### Queueing multiple scans
Several tomo parameter sets can be queued and run sequentially on the same sample, without having to start each one by hand. Easiest is to use the tomo parameters GUI for preparing the scans. But it is also possible on the command line interface.
@@ -0,0 +1,53 @@
from unittest import mock
import pytest
import csaxs_bec.bec_ipython_client.plugins.cSAXS.filter_transmission as ft_module
from csaxs_bec.bec_ipython_client.plugins.cSAXS.filter_transmission import (
cSAXSFilterTransmission,
)
def _make_filter():
filt = cSAXSFilterTransmission()
filt.OMNYTools = mock.MagicMock()
return filt
def _make_dev(readbacks: dict):
"""Mock `dev` where each filter axis' readback.get returns the given value."""
dev = mock.MagicMock()
for axis in cSAXSFilterTransmission._AXES:
axis_obj = mock.MagicMock()
axis_obj.readback.get.return_value = readbacks[axis]
setattr(dev, axis, axis_obj)
return dev
def test_fil_trans_report_reads_positions_cached(capsys):
filt = _make_filter()
# Sit exactly on the first defined nominal position of every unit
readbacks = {}
for axis, positions in zip(filt._AXES, filt._POSITIONS_USER):
readbacks[axis] = next(p for p in positions if p is not None)
dev = _make_dev(readbacks)
with mock.patch.object(ft_module, "dev", dev, create=True):
filt._fil_trans_report(energy_kev=7.0)
for axis in filt._AXES:
getattr(dev, axis).readback.get.assert_called_once_with(cached=True)
assert "Matched filter code" in capsys.readouterr().out
def test_fil_trans_report_without_cached_value_reports_error(capsys):
filt = _make_filter()
readbacks = {axis: None for axis in filt._AXES} # nothing published to redis yet
dev = _make_dev(readbacks)
with mock.patch.object(ft_module, "dev", dev, create=True):
filt._fil_trans_report(energy_kev=7.0)
out = capsys.readouterr().out
assert f"readback unavailable for axis {filt._AXES[0]}" in out
assert "Matched filter code" not in out
@@ -0,0 +1,45 @@
from unittest import mock
from csaxs_bec.bec_ipython_client.plugins.cSAXS.slits import cSAXSSlits
def _make_slits(devices: dict):
client = mock.MagicMock()
client.device_manager.devices = devices
return cSAXSSlits(client)
def test_sl_get_position_uses_cached_read():
dev = mock.MagicMock()
dev.read.return_value = {"sl4xc": {"value": 1.25}}
slits = _make_slits({"sl4xc": dev})
assert slits._sl_get_position("sl4xc") == 1.25
dev.read.assert_called_once_with(cached=True)
def test_sl_get_position_returns_none_without_cached_value():
dev = mock.MagicMock()
dev.read.return_value = None # nothing published to redis yet
slits = _make_slits({"sl4xc": dev})
assert slits._sl_get_position("sl4xc") is None
assert slits._sl_get_position("unknown_device") is None
def test_slits_show_all_reads_every_slit_device_cached(capsys):
devices = {}
for prefix in cSAXSSlits._SLIT_LABELS:
for suffix in ("xc", "xs", "yc", "ys"):
name = f"{prefix}{suffix}"
dev = mock.MagicMock()
dev.read.return_value = {name: {"value": 0.5}}
devices[name] = dev
slits = _make_slits(devices)
slits.slits_show_all()
assert len(devices) == 24
for dev in devices.values():
dev.read.assert_called_once_with(cached=True)
assert "cSAXS Slits Overview" in capsys.readouterr().out
@@ -0,0 +1,104 @@
"""Tests for HistoryTracker's upload-size guard on eps_alarm_history.json.
A long-running/flapping EPS alarm could push the uploaded file past the
upload server's request-size limit (bare HTTP 413, no size check in
upload.php itself -- almost certainly nginx's client_max_body_size, default
1m) even with the count-based caps in eps_status_generator.py, since those
bound event/text *counts* but not individual alarm-text length, and a
previously-oversized event's texts list is never re-capped on load. See
x12sa report 2026-09-18: eps_alarm_history.json observed at 1,209,271 bytes.
"""
import json
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.eps.eps_status_generator import (
_MAX_UPLOAD_BYTES,
HistoryTracker,
_payload_size,
_shrink_to_fit,
)
def _bloated_payload(n_events=50, texts_per_event=100, text_len=250):
eps_events = [
{
"first_seen": i,
"cleared_at": None,
"count": 3,
"texts": [f"alarm text variant {i}-{j} " + "x" * text_len for j in range(texts_per_event)],
}
for i in range(n_events)
]
machine_events = [
{"pv": f"PV{i}", "label": "x", "from": "a" * 20, "to": "b" * 20, "at": i}
for i in range(n_events)
]
ring_current_trace = [{"t": i, "v": 400.0 + i * 0.01} for i in range(288)]
return {
"eps_events": eps_events,
"machine_events": machine_events,
"ring_current_trace": ring_current_trace,
}
def test_shrink_to_fit_is_noop_when_already_under_budget():
payload = _bloated_payload(n_events=2, texts_per_event=1, text_len=10)
assert _payload_size(payload) <= _MAX_UPLOAD_BYTES
shrunk = _shrink_to_fit(payload)
assert shrunk == payload
assert shrunk is not payload # returns a copy, doesn't mutate the caller's data
def test_shrink_to_fit_brings_bloated_payload_under_budget():
payload = _bloated_payload()
assert _payload_size(payload) > _MAX_UPLOAD_BYTES # reproduces the reported 1.2 MB case
shrunk = _shrink_to_fit(payload)
assert _payload_size(shrunk) <= _MAX_UPLOAD_BYTES
# trimming per-event alarm texts alone is enough here -- full event/trace
# history (the actually useful part) is preserved
assert len(shrunk["eps_events"]) == 50
assert len(shrunk["machine_events"]) == 50
assert len(shrunk["ring_current_trace"]) == 288
def test_shrink_to_fit_does_not_mutate_input():
payload = _bloated_payload()
original_texts_len = len(payload["eps_events"][0]["texts"])
_shrink_to_fit(payload)
assert len(payload["eps_events"][0]["texts"]) == original_texts_len
def test_shrink_to_fit_drops_oldest_events_when_a_tiny_budget_forces_it():
payload = _bloated_payload(n_events=10, texts_per_event=5, text_len=50)
shrunk = _shrink_to_fit(payload, max_bytes=500)
assert _payload_size(shrunk) <= 500 or len(shrunk["eps_events"]) == 1
# oldest dropped first: whatever's left is the most recent
if len(shrunk["eps_events"]) < 10:
kept_ids = [ev["first_seen"] for ev in shrunk["eps_events"]]
assert kept_ids == sorted(kept_ids)
assert kept_ids[-1] == 9
def test_history_tracker_save_writes_file_under_budget(tmp_path):
path = tmp_path / "eps_alarm_history.json"
tracker = HistoryTracker(path)
bloated = _bloated_payload()
tracker.eps_events = bloated["eps_events"]
tracker.machine_events = bloated["machine_events"]
tracker.ring_trace = bloated["ring_current_trace"]
tracker.save()
written_size = path.stat().st_size
assert written_size <= _MAX_UPLOAD_BYTES
# written file is still valid JSON with the expected top-level shape
data = json.loads(path.read_text())
assert set(data.keys()) == {"eps_events", "machine_events", "ring_current_trace"}
@@ -76,8 +76,10 @@ def test_flomni_expected_position_count_below_threshold():
count = flomni._expected_fermat_position_count()
assert count < flomni._fermat_min_positions()
# matches calling the real scan-class algorithm directly
expected = FlomniFermatScan.get_flomni_fermat_spiral_pos(-2.5, 2.5, -2.5, 2.5, step=2.0)
# matches calling the real scan-class algorithm directly. _expected_fermat_position_count()
# pins seed=0 for its own reproducibility -- match it here rather than drawing an
# independent random offset.
expected = FlomniFermatScan.get_flomni_fermat_spiral_pos(-2.5, 2.5, -2.5, 2.5, step=2.0, seed=0)
assert count == len(expected)
@@ -104,7 +106,9 @@ def test_lamni_expected_position_count_below_threshold():
count = lamni._expected_fermat_position_count()
assert count < lamni._fermat_min_positions()
# matches calling the real scan-class algorithm directly, center tile, angle 0
# matches calling the real scan-class algorithm directly, center tile, angle 0.
# _expected_fermat_position_count() pins seed=0 for its own reproducibility -- match
# it here rather than drawing an independent random offset.
expected = LamniFermatScan.get_lamni_fermat_spiral_pos(
-2.5,
2.5,
@@ -118,6 +122,7 @@ def test_lamni_expected_position_count_below_threshold():
fovx=5.0,
fovy=5.0,
fov_circular=0.0,
seed=0,
)
assert count == len(expected)
@@ -0,0 +1,81 @@
"""Tests for filters_out_of_beam() (csaxs_bec/bec_ipython_client/plugins/OMNY_shared/filter_check.py).
Regression coverage for a bug surfaced on a simulated flomni session:
tomo_scan_projection() -> filters_out_of_beam() crashed with
DeviceConfigError: Device filter_array_1_x does not exist, because the sim
config (unlike the real beamline's bl_endstation.yaml) doesn't define the
filter_array_*_x devices at all. The lookup used `getattr(dev, axis_name,
None)`, but BEC's device container raises DeviceConfigError (a plain
Exception, not AttributeError) for a missing device, so getattr's default
never applied.
"""
from unittest import mock
from bec_lib.bec_errors import DeviceConfigError
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.filter_check import filters_out_of_beam
class _FakeDeviceContainer(dict):
"""Stands in for BEC's real device-manager container: attribute access
for a present device returns it, for a missing one raises
DeviceConfigError (not AttributeError) -- see devicemanager.py's
__getattr__."""
def __getattr__(self, attr):
dev = self.get(attr)
if not dev:
raise DeviceConfigError(f"Device {attr} does not exist.")
return dev
def _axis_at(position: float):
axis = mock.MagicMock()
axis.readback.get.return_value = position
return axis
def test_filters_out_of_beam_dev_is_none():
# The usual state of the module-level `dev` idiom before a real BEC
# session has set it (`dev = builtins.__dict__.get("dev")`) -- several
# LamNI/Flomni call sites pass this straight through. Must not raise.
all_out, offending = filters_out_of_beam(None)
assert all_out is True
assert offending == []
def test_filters_out_of_beam_all_devices_missing_from_container():
# Mirrors a simulated deployment: none of the filter_array_*_x devices
# exist at all. Must not raise, and must report "all out" (nothing to
# check -> vacuously true), matching the existing axis_obj-is-None skip.
dev = _FakeDeviceContainer()
all_out, offending = filters_out_of_beam(dev)
assert all_out is True
assert offending == []
def test_filters_out_of_beam_some_devices_missing_from_container():
# A mixed deployment: filter_array_1_x exists and is out of the beam,
# the other three axes don't exist at all.
dev = _FakeDeviceContainer(filter_array_1_x=_axis_at(25.0))
all_out, offending = filters_out_of_beam(dev)
assert all_out is True
assert offending == []
def test_filters_out_of_beam_flags_axis_in_beam():
dev = _FakeDeviceContainer(
filter_array_1_x=_axis_at(25.0), # out
filter_array_2_x=_axis_at(17.6), # NOT out (out position is 25.5)
)
all_out, offending = filters_out_of_beam(dev)
assert all_out is False
assert offending == ["filter_array_2_x"]
@@ -0,0 +1,61 @@
"""Tests for Flomni._get_val()'s min_/max_ range validation.
tomo_parameters() (the CLI wizard) used to accept any value of the right
type with no range check at all, even though the Tomo Parameters GUI
enforces spinbox ranges for the same fields (tomo_params.py) -- this let the
CLI silently set e.g. manual_shift_y or fovx far outside their GUI-intended
bounds. _get_val() now optionally re-prompts until the typed value is in
range, so tomo_parameters() enforces the same limits as the GUI.
"""
import builtins
import pytest
from csaxs_bec.bec_ipython_client.plugins.flomni.flomni import Flomni
def _fake_inputs(monkeypatch, values):
it = iter(values)
monkeypatch.setattr(builtins, "input", lambda prompt="": next(it))
def test_get_val_without_bounds_accepts_anything(monkeypatch):
_fake_inputs(monkeypatch, ["12345"])
assert Flomni._get_val("x", 0.0, float) == 12345.0
def test_get_val_accepts_in_range_value_first_try(monkeypatch):
_fake_inputs(monkeypatch, ["25"])
assert Flomni._get_val("x", 0.0, float, min_=0.1, max_=220.0) == 25.0
def test_get_val_reprompts_when_above_max(monkeypatch, capsys):
_fake_inputs(monkeypatch, ["500", "50"])
result = Flomni._get_val("FOV X", 0.0, float, min_=0.1, max_=220.0)
assert result == 50.0
assert "must be <= 220.0" in capsys.readouterr().out
def test_get_val_reprompts_when_below_min(monkeypatch, capsys):
_fake_inputs(monkeypatch, ["-999", "-30"])
result = Flomni._get_val("manual_shift_y", 0.0, float, min_=-50.0, max_=50.0)
assert result == -30.0
assert "must be >= -50.0" in capsys.readouterr().out
def test_get_val_reprompts_until_valid_across_multiple_bad_attempts(monkeypatch):
_fake_inputs(monkeypatch, ["-5", "999", "30"])
assert Flomni._get_val("stitch X", 0, int, min_=0, max_=50) == 30
def test_get_val_empty_input_uses_default_and_still_checked(monkeypatch):
# empty input -> falls back to default_value, which is already in range
_fake_inputs(monkeypatch, [""])
assert Flomni._get_val("x", 25, int, min_=0, max_=50) == 25
def test_get_val_boundary_values_are_accepted(monkeypatch):
_fake_inputs(monkeypatch, ["0.1", "220.0"])
assert Flomni._get_val("a", 0.0, float, min_=0.1, max_=220.0) == 0.1
assert Flomni._get_val("b", 0.0, float, min_=0.1, max_=220.0) == 220.0
@@ -0,0 +1,219 @@
"""Tests for the "Skip" button of the flomni X-ray-eye alignment (see
docs/plans/flomni-xrayeye-skip-button.md): the GUI signals a skip with
``omny_xray_gui.submit == -1`` and XrayEyeAlign._align_impl leaves that angle out of the
fit, as long as at least 4 of the 5 angles (0/45/90/135/180 deg) remain.
The real _align_impl loop is driven with a scripted sequence of ``submit`` values; the
hardware (dev, umv, flomni, gui) is mocked, no BEC service is contacted.
"""
from unittest import mock
import numpy as np
import pytest
from csaxs_bec.bec_ipython_client.plugins.flomni.x_ray_eye_align import XrayEyeAlign
# pylint: disable=protected-access
# pylint: disable=redefined-outer-name
MODULE = "csaxs_bec.bec_ipython_client.plugins.flomni.x_ray_eye_align"
class _Signal:
def __init__(self, value=0.0):
self.value = value
def get(self):
return self.value
def set(self, value):
self.value = value
return mock.MagicMock()
class _ScriptedSubmit(_Signal):
"""submit signal: get() plays back the script (one entry per loop iteration), then
fails loudly instead of letting a broken loop spin forever."""
def __init__(self, script):
super().__init__(0)
self.script = list(script)
self.sets = []
def get(self):
if not self.script:
raise AssertionError("client loop polled submit more often than scripted")
return self.script.pop(0)
def set(self, value):
self.sets.append(value)
return super().set(value)
def _make_dev(submit_script):
"""dev mock with a real-number omny_xray_gui: step k was clicked at x = 100 + 10 k px
(pixel_calibration 0.001 mm/px -> 0.1 + 0.01 k mm)."""
gui = mock.MagicMock()
gui.submit = _ScriptedSubmit(submit_script)
gui.mvx = _Signal(0)
gui.mvy = _Signal(0)
for k in range(6):
setattr(gui, f"xval_x_{k}", _Signal(100.0 + 10 * k))
setattr(gui, f"yval_y_{k}", _Signal(50.0))
setattr(gui, f"width_x_{k}", _Signal(20.0 + k))
setattr(gui, f"width_y_{k}", _Signal(30.0 + k))
dev = mock.MagicMock()
dev.omny_xray_gui = gui
dev.cam_xeye.user_parameter.get.return_value = 0.001
dev.rtx.readback.get.return_value = 0.0
return dev
def _run(submit_script):
"""Run the whole _align_impl loop; returns (align, dev, messages, write_output data)."""
align = XrayEyeAlign(mock.MagicMock(), mock.MagicMock())
align.test_wo_movements = True
messages = []
align.send_message = messages.append
align.update_frame = mock.MagicMock()
align._save_alignment_data = mock.MagicMock()
dev = _make_dev(submit_script)
with mock.patch(f"{MODULE}.dev", dev), mock.patch(f"{MODULE}.umv"), mock.patch(
f"{MODULE}.time.sleep"
):
align._align_impl()
return align, dev, messages
def _fit_data(align):
return align.gui.submit_fit_array.call_args.args[0]
# FZP center (k=0), height marking (first submit at k=1), then the five angle submissions
# 0/45/90/135/180 deg = steps k=1..5. 0 entries are idle polls.
FZP, HEIGHT = 1, 1
def test_default_run_uses_all_five_angles():
align, _, _ = _run([FZP, HEIGHT, 1, 1, 1, 1, 1])
assert align.skipped_steps == []
assert list(_fit_data(align)[0]) == [0, 45, 90, 135, 180]
assert len(_fit_data(align)[1]) == 5
# every non-skipped angle contributed to the field of view
assert align._xray_fov_xy == [25.0, 35.0]
def test_skip_90_deg_leaves_four_points():
align, _, _ = _run([FZP, HEIGHT, 1, 1, -1, 1, 1])
assert align.skipped_steps == [3]
data = _fit_data(align)
assert list(data[0]) == [0, 45, 135, 180]
# offset = value[0] - value[k] (um): steps 1, 2, 4, 5 -> clicked at 110, 120, 140, 150 px
np.testing.assert_allclose(data[1], [-10.0, -20.0, -40.0, -50.0])
assert data.shape == (3, 4)
kwargs = align._save_alignment_data.call_args.kwargs
assert kwargs["skipped_angles"] == [90]
def test_skip_writes_no_values_and_a_nan_row_for_the_step():
align, _, _ = _run([FZP, HEIGHT, 1, 1, -1, 1, 1])
assert sorted(align.alignment_values) == [0, 1, 2, 4, 5]
rows = {int(r[0]): r for r in align.roi_pixel_data}
assert np.isnan(rows[3][1:5]).all()
assert not np.isnan(rows[4][1:5]).any()
def test_skipped_step_does_not_read_its_stale_widths_for_the_fov():
# step 5 would be the widest ROI (25 px) but is skipped; stale widths must not be read
align, _, _ = _run([FZP, HEIGHT, 1, 1, 1, 1, -1])
assert align.skipped_steps == [5]
assert align._xray_fov_xy == [24.0, 34.0]
assert list(_fit_data(align)[0]) == [0, 45, 90, 135]
def test_skip_at_last_step_finishes_cleanly():
align, dev, messages = _run([FZP, HEIGHT, 1, 1, 1, 1, -1])
assert "done..." in messages
align.gui.hide_crosshair.assert_called()
assert dev.omny_xray_gui.submit.value == 0
def test_second_skip_is_refused_and_run_continues():
# the second -1 (at k=4) is refused, the same step is then submitted normally
align, dev, messages = _run([FZP, HEIGHT, 1, 1, -1, -1, 1, 1])
assert align.skipped_steps == [3]
assert list(_fit_data(align)[0]) == [0, 45, 135, 180]
refusals = [m for m in messages if m.startswith("Cannot skip")]
assert len(refusals) == 1
assert "at least 4 positions" in refusals[0]
# the message keeps telling the user which step is pending
assert "Step 4/5" in refusals[0]
# a refused skip is reset, so it cannot leak into the next step
assert dev.omny_xray_gui.submit.value == 0
align.gui.enable_skip_button.assert_called_with(False)
def test_skip_of_fzp_center_is_refused():
align, _, messages = _run([-1, FZP, HEIGHT, 1, 1, 1, 1, 1])
assert align.skipped_steps == []
assert list(_fit_data(align)[0]) == [0, 45, 90, 135, 180]
assert "Cannot skip: the FZP center cannot be skipped. Submit center value of FZP." in messages
def test_skip_of_height_marking_is_refused():
align, _, messages = _run([FZP, -1, HEIGHT, 1, 1, 1, 1, 1])
assert align.skipped_steps == []
assert list(_fit_data(align)[0]) == [0, 45, 90, 135, 180]
assert any(m.startswith("Cannot skip: the sample height marking") for m in messages)
def test_skip_button_is_enabled_only_for_fit_steps():
align, _, _ = _run([FZP, HEIGHT, 1, -1, 1, 1, 1])
calls = [c.args[0] for c in align.gui.enable_skip_button.call_args_list]
# start of the run: off; height marked: on; after the one allowed skip the budget is used
# up: off; last step: off
assert calls == [False, True, False, False]
def test_second_skip_would_leave_three_points_write_output_guard():
align = XrayEyeAlign(mock.MagicMock(), mock.MagicMock())
align._reset_init_values()
align.alignment_values = {k: 0.1 * k for k in range(6)}
align.skipped_steps = [2, 3]
align.gui.submit_fit_array = mock.MagicMock()
align._save_alignment_data = mock.MagicMock()
with pytest.raises(RuntimeError, match="at least 4"):
align.write_output()
align._save_alignment_data.assert_not_called()
align.gui.submit_fit_array.assert_not_called()
def test_skip_end_point_prints_warning(capsys):
_run([FZP, HEIGHT, 1, 1, 1, 1, -1])
assert "skipping an end point" in capsys.readouterr().out
def test_skip_refusal_reason_table():
align = XrayEyeAlign(mock.MagicMock(), mock.MagicMock())
align._reset_init_values()
assert align._skip_refusal_reason(0) is not None
assert align._skip_refusal_reason(1) is not None # height marking
align._height_centered = True
assert align._skip_refusal_reason(1) is None # angle 0 deg
assert align._skip_refusal_reason(5) is None
align.skipped_steps = [3]
assert "at least 4" in align._skip_refusal_reason(4)
@@ -0,0 +1,93 @@
"""Regression test for a real hardware bug (2026-09-19): flomni.fovy could be
set to exactly 100.0 via the property/GUI/CLI (all capped at an inclusive
100.0), but FlomniFermatScan's own ScanArgument declares fovy as gt=0, lt=100
(strictly less than) -- so fovy=100.0 passed every check right up until the
scan itself rejected it with ScanInputValidationError on real hardware.
lamni_piezo_range_x/y (which become LamniFermatScan's fovx/fovy) had the same
class of bug at both ends: GUI/CLI min was 0.0 but LamniFermatScan requires
gt=0, and max was an inclusive 80.0 but LamniFermatScan requires lt=80.
This test introspects the actual ScanArgument bounds from the scan classes
themselves and checks our configured GUI/CLI ranges (tomo_params.py's
SETUP_PROFILES, and flomni.py/lamni.py's tomo_parameters() prompts) are
strictly inside them -- not just checked once by hand -- so this class of
"looks fine until you actually run the scan" bug can't silently return.
"""
import re
from typing import get_type_hints
import csaxs_bec.bec_ipython_client.plugins.flomni.flomni as _flomni_cli_module
import csaxs_bec.bec_ipython_client.plugins.LamNI.lamni as _lamni_cli_module
from csaxs_bec.bec_widgets.widgets.tomo_params.tomo_params import SETUP_PROFILES
from csaxs_bec.scans.flomni_fermat_scan import FlomniFermatScan
from csaxs_bec.scans.lamni_fermat_scan import LamniFermatScan
def _scan_arg_bounds(scan_cls, param_name: str):
hints = get_type_hints(scan_cls.__init__, include_extras=True)
metadata = hints[param_name].__metadata__
(scan_argument,) = [m for m in metadata if hasattr(m, "lt")]
return scan_argument.gt, scan_argument.ge, scan_argument.lt, scan_argument.le
def _assert_range_inside_bounds(our_min, our_max, gt, ge, lt, le):
if gt is not None:
assert our_min > gt, f"configured min {our_min} must be > scan's gt={gt}"
if ge is not None:
assert our_min >= ge, f"configured min {our_min} must be >= scan's ge={ge}"
if lt is not None:
assert our_max < lt, f"configured max {our_max} must be < scan's lt={lt}"
if le is not None:
assert our_max <= le, f"configured max {our_max} must be <= scan's le={le}"
def _gui_fields(profile_key: str) -> dict:
"""Flatten a profile's fov_fields + piezo_range_fields into {name: (min, max)}."""
profile = SETUP_PROFILES[profile_key]
fields = {}
for name, _label, min_, max_, *_ in profile["fov_fields"]:
fields[name] = (min_, max_)
for name, _label, min_, max_, *_ in profile.get("piezo_range_fields", []):
fields[name] = (min_, max_)
return fields
def test_flomni_gui_fovx_fovy_within_scan_bounds():
fields = _gui_fields("flomni")
our_min, our_max = fields["fovx"]
_assert_range_inside_bounds(our_min, our_max, *_scan_arg_bounds(FlomniFermatScan, "fovx"))
our_min, our_max = fields["fovy"]
_assert_range_inside_bounds(our_min, our_max, *_scan_arg_bounds(FlomniFermatScan, "fovy"))
def test_flomni_cli_fovy_bounds_within_scan_bounds():
src = open(_flomni_cli_module.__file__, encoding="utf-8").read()
m = re.search(r'"<FOV Y \(max 100\)> um".*?min_=([\d.]+), max_=([\d.]+)', src, re.DOTALL)
assert m, "could not find the fovy _get_val() call in flomni.py -- did it move?"
cli_min, cli_max = float(m.group(1)), float(m.group(2))
_assert_range_inside_bounds(cli_min, cli_max, *_scan_arg_bounds(FlomniFermatScan, "fovy"))
def test_lamni_gui_piezo_range_within_scan_bounds():
fields = _gui_fields("lamni")
for name, scan_param in (("lamni_piezo_range_x", "fovx"), ("lamni_piezo_range_y", "fovy")):
our_min, our_max = fields[name]
_assert_range_inside_bounds(
our_min, our_max, *_scan_arg_bounds(LamniFermatScan, scan_param)
)
def test_lamni_cli_piezo_range_bounds_within_scan_bounds():
src = open(_lamni_cli_module.__file__, encoding="utf-8").read()
for axis, scan_param in (("X", "fovx"), ("Y", "fovy")):
m = re.search(
rf'"<piezo range {axis} \(max 80\)> um".*?min_=([\d.]+), max_=([\d.]+)', src, re.DOTALL
)
assert m, f"could not find the piezo range {axis} _get_val() call in lamni.py -- did it move?"
cli_min, cli_max = float(m.group(1)), float(m.group(2))
_assert_range_inside_bounds(
cli_min, cli_max, *_scan_arg_bounds(LamniFermatScan, scan_param)
)
@@ -11,6 +11,7 @@ import types
import csaxs_bec.bec_ipython_client.plugins.LamNI.lamni as lamni_module
from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI, _ProgressProxy
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.tomo_queue_mixin import _TomoQueueProxy
class FakeClient:
@@ -55,6 +56,7 @@ def make_lamni(answer=True):
obj = object.__new__(LamNI)
obj.client = FakeClient()
obj._progress_proxy = _ProgressProxy(obj.client)
obj._tomo_queue_proxy = _TomoQueueProxy(obj.client)
obj.OMNYTools = FakeOMNYTools(answer=answer)
obj.reconstructor = types.SimpleNamespace(folder_name=None)
return obj
@@ -87,6 +89,8 @@ def test_prompts_and_resets_on_new_account_when_confirmed(monkeypatch):
lamni.client.set_global_var("defaults_applied_for_account", "e11111")
lamni.tomo_shellstep = 99.0
lamni.tomo_circfov = 123.0
lamni._tomo_queue_proxy.append({"label": "leftover job", "params": {}, "status": "pending"})
lamni._progress_proxy.update(projection=42, angle=90.0)
monkeypatch.setitem(builtins.__dict__, "bec", type("Bec", (), {"active_account": "e22222"})())
lamni._maybe_reset_params_on_account_change()
@@ -97,18 +101,24 @@ def test_prompts_and_resets_on_new_account_when_confirmed(monkeypatch):
assert lamni.tomo_shellstep == 1
assert lamni.tomo_circfov == 0.0
assert lamni.client.get_global_var("defaults_applied_for_account") == "e22222"
assert lamni._tomo_queue_proxy.as_list() == []
assert lamni._progress_proxy.as_dict() == _ProgressProxy._DEFAULTS
def test_records_account_without_resetting_when_declined(monkeypatch):
lamni = make_lamni(answer=False)
lamni.client.set_global_var("defaults_applied_for_account", "e11111")
lamni.tomo_shellstep = 99.0
lamni._tomo_queue_proxy.append({"label": "leftover job", "params": {}, "status": "pending"})
lamni._progress_proxy.update(projection=42, angle=90.0)
monkeypatch.setitem(builtins.__dict__, "bec", type("Bec", (), {"active_account": "e22222"})())
lamni._maybe_reset_params_on_account_change()
assert lamni.tomo_shellstep == 99.0
assert lamni.client.get_global_var("defaults_applied_for_account") == "e22222"
assert len(lamni._tomo_queue_proxy.as_list()) == 1
assert lamni._progress_proxy["projection"] == 42
def test_set_default_tomo_params_matches_getter_fallbacks():
@@ -0,0 +1,46 @@
"""Tests for LamNI._get_val()'s min_/max_ range validation.
Mirrors test_flomni_get_val_range.py -- LamNI's tomo_parameters() CLI wizard
previously had no range validation at all (unlike the GUI's spinbox
ranges); _get_val() now optionally re-prompts until the typed value is in
range, same mechanism as Flomni's.
"""
import builtins
from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI
def _fake_inputs(monkeypatch, values):
it = iter(values)
monkeypatch.setattr(builtins, "input", lambda prompt="": next(it))
def test_get_val_without_bounds_accepts_anything(monkeypatch):
_fake_inputs(monkeypatch, ["12345"])
assert LamNI._get_val("x", 0.0, float) == 12345.0
def test_get_val_accepts_in_range_value_first_try(monkeypatch):
_fake_inputs(monkeypatch, ["10"])
assert LamNI._get_val("shellstep", 0.0, float, min_=0.025, max_=20.0) == 10.0
def test_get_val_reprompts_when_above_max(monkeypatch, capsys):
_fake_inputs(monkeypatch, ["999", "50"])
result = LamNI._get_val("circfov", 0.0, float, min_=0.0, max_=500.0)
assert result == 50.0
assert "must be <= 500.0" in capsys.readouterr().out
def test_get_val_reprompts_when_below_min(monkeypatch, capsys):
_fake_inputs(monkeypatch, ["-5", "10"])
result = LamNI._get_val("piezo range x", 20.0, float, min_=0.0, max_=80.0)
assert result == 10.0
assert "must be >= 0.0" in capsys.readouterr().out
def test_get_val_boundary_values_are_accepted(monkeypatch):
_fake_inputs(monkeypatch, ["0.025", "20.0"])
assert LamNI._get_val("a", 0.0, float, min_=0.025, max_=20.0) == 0.025
assert LamNI._get_val("b", 0.0, float, min_=0.025, max_=20.0) == 20.0
@@ -164,6 +164,10 @@ def make_lamni_for_tomo_scan(
obj.special_angles = []
obj.write_pdf_report = lambda: None
obj.sub_tomo_scan = lambda subtomo_number, start_angle=None: None
# Default no-op for the unconditional 0-deg reference projection tomo_scan()
# now fires at the start of every new scan (see lamni.py) -- tests that
# care about it override this after calling make_lamni_for_tomo_scan().
obj._tomo_scan_at_angle = lambda angle, subtomo_number: None
obj._print_progress = lambda: None
obj._format_duration = lambda seconds: "0s"
obj.lamnigui_show_progress = lambda: None
@@ -301,7 +305,9 @@ def test_subtomo_starts_near_zero(subtomo_number, expected):
def test_zero_deg_reference_disabled_by_default(monkeypatch):
"""zero_deg_reference_at_each_subtomo defaults to False -- a fresh
tomo_scan() must not fire any extra 0-deg shots."""
tomo_scan() must not fire any *flag-gated* extra 0-deg shots, but the
unconditional single 0-deg reference projection at scan start (see
lamni.py) always fires regardless of the flag."""
lamni = make_lamni_for_tomo_scan(monkeypatch, 45.0, active_account="")
lamni.add_sample_database = lambda *a, **k: 0
recorded = []
@@ -312,13 +318,16 @@ def test_zero_deg_reference_disabled_by_default(monkeypatch):
lamni.tomo_scan()
assert recorded == []
assert recorded == [(0, 1)]
def test_zero_deg_reference_fires_for_odd_subtomos_and_final_shot(monkeypatch):
"""With the flag on, a fresh tomo_scan() must fire an extra angle-0 shot
before each odd sub-tomogram (1, 3, 5, 7) plus one final shot after
sub-tomogram 8 completes -- mirrors Flomni.tomo_scan()'s equivalent."""
sub-tomogram 8 completes -- mirrors Flomni.tomo_scan()'s equivalent. The
leading (0, 1) is the unconditional scan-start reference projection
(fires regardless of the flag), followed by the flag's own (0, 1) for
sub-tomogram 1 itself -- the two are independent and both fire here."""
lamni = make_lamni_for_tomo_scan(monkeypatch, 45.0, active_account="")
lamni.add_sample_database = lambda *a, **k: 0
lamni.zero_deg_reference_at_each_subtomo = True
@@ -330,7 +339,7 @@ def test_zero_deg_reference_fires_for_odd_subtomos_and_final_shot(monkeypatch):
lamni.tomo_scan()
assert recorded == [(0, 1), (0, 3), (0, 5), (0, 7), (0, 8)]
assert recorded == [(0, 1), (0, 1), (0, 3), (0, 5), (0, 7), (0, 8)]
def test_zero_deg_reference_skipped_when_resuming_mid_subtomo(monkeypatch):
@@ -137,10 +137,12 @@ def test_lamni_offset_fields_excluded_from_param_names():
def test_flomni_field_order_matches_pre_reorder_layout():
"""Regression guard: the field_order dispatch must reproduce flomni's
exact, unchanged field build order (this task only reordered lamni)."""
build order (this task only reordered lamni). Updated to include
fermat_asymmetry, added right after tomo_shellstep."""
assert SETUP_PROFILES["flomni"]["field_order"] == [
"tomo_countingtime",
"tomo_shellstep",
"fermat_asymmetry",
"fov",
"stitch",
"tomo_stitch_overlap",
@@ -182,6 +184,8 @@ def test_compute_fermat_positions_lamni_matches_scan_class_below_threshold():
count, min_positions = _compute_fermat_positions_lamni(params)
assert min_positions == LamniFermatScan.MIN_POSITIONS
assert count < min_positions
# _compute_fermat_positions_lamni() pins seed=0 for its own reproducibility -- match
# it here rather than drawing an independent random offset.
expected = LamniFermatScan.get_lamni_fermat_spiral_pos(
-2.5,
2.5,
@@ -195,6 +199,7 @@ def test_compute_fermat_positions_lamni_matches_scan_class_below_threshold():
fovx=5.0,
fovy=5.0,
fov_circular=0.0,
seed=0,
)
assert count == len(expected)
@@ -55,7 +55,9 @@ def test_compute_fermat_positions_flomni_matches_scan_class_below_threshold():
count, min_positions = _compute_fermat_positions_flomni(params)
assert min_positions == FlomniFermatScan.MIN_POSITIONS
assert count < min_positions
expected = FlomniFermatScan.get_flomni_fermat_spiral_pos(-2.5, 2.5, -2.5, 2.5, step=2.0)
# _compute_fermat_positions_flomni() pins seed=0 for its own reproducibility --
# match it here rather than drawing an independent random offset.
expected = FlomniFermatScan.get_flomni_fermat_spiral_pos(-2.5, 2.5, -2.5, 2.5, step=2.0, seed=0)
assert count == len(expected)
@@ -63,7 +65,9 @@ def test_compute_fermat_positions_flomni_matches_scan_class_above_threshold():
params = {"fovx": 100.0, "fovy": 80.0, "tomo_shellstep": 1.0}
count, min_positions = _compute_fermat_positions_flomni(params)
assert count >= min_positions
expected = FlomniFermatScan.get_flomni_fermat_spiral_pos(-50.0, 50.0, -40.0, 40.0, step=1.0)
expected = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-50.0, 50.0, -40.0, 40.0, step=1.0, seed=0
)
assert count == len(expected)
@@ -0,0 +1,214 @@
"""Tests for the hard-stop logic of z_ConsoleButtonsWidget: every configured
Galil controller gets a stop (one thread each, configured order in the
result), a device that can't be resolved or a controller that fails is
reported instead of silently skipped, and one failure never prevents the
other stops.
The stop helpers are plain methods (no Qt slots), so they are exercised on a
minimal stub instead of constructing the full widget, which needs a live BEC
client context via get_bec_shortcuts() -- the same approach as
test_x_ray_eye_vertical_lock.py. No hardware or BEC service is contacted.
"""
import threading
from unittest import mock
import pytest
from csaxs_bec.bec_widgets.widgets.console_buttons.console_buttons import z_ConsoleButtonsWidget
Widget = z_ConsoleButtonsWidget
class _StubConsole:
"""Minimal stand-in for z_ConsoleButtonsWidget, isolated to the
attributes the hard-stop helpers touch."""
_RESOLVE_ATTEMPTS = 3
_RESOLVE_DELAY_S = 0.0
_STOP_JOIN_TIMEOUT_S = 5.0
_resolve_controller = Widget._resolve_controller
_stop_controller = Widget._stop_controller
_stop_all_controllers = Widget._stop_all_controllers
_restore_positioning_mode = Widget._restore_positioning_mode
def _make_stub(devices: dict, names: list[str]):
stub = _StubConsole()
stub._hard_stop_device_names = names
stub.client = mock.Mock()
stub.client.device_manager.devices = devices
return stub
def _device(controller):
return mock.Mock(controller=controller)
def _controller(stop_reply=":"):
ctrl = mock.Mock()
ctrl.stop_all_axes.return_value = stop_reply
ctrl.socket_put_and_receive.return_value = ":"
return ctrl
# ---------------------------------------------------------------- name list
def test_collect_names_prefers_new_list_and_drops_duplicates():
assert Widget._collect_hard_stop_names(["a", "b", "a", None, ""], "x", "y") == ["a", "b"]
def test_collect_names_falls_back_to_legacy_pair():
assert Widget._collect_hard_stop_names(None, "ftransy", "foptx") == ["ftransy", "foptx"]
assert Widget._collect_hard_stop_names(None, "ftransy", None) == ["ftransy"]
assert Widget._collect_hard_stop_names(None, None, None) == []
def test_collect_names_accepts_a_single_string():
assert Widget._collect_hard_stop_names("ftransy") == ["ftransy"]
# ---------------------------------------------------------------- resolving
def test_resolve_returns_controller():
ctrl = _controller()
stub = _make_stub({"ftransy": _device(ctrl)}, ["ftransy"])
assert stub._resolve_controller("ftransy") is ctrl
def test_resolve_reports_missing_device_after_retries():
devices = mock.MagicMock()
devices.get.return_value = None
stub = _make_stub(devices, ["ftransy"])
with pytest.raises(RuntimeError, match="not found"):
stub._resolve_controller("ftransy")
assert devices.get.call_count == _StubConsole._RESOLVE_ATTEMPTS
def test_resolve_retries_after_transient_lookup_error():
ctrl = _controller()
devices = mock.MagicMock()
devices.get.side_effect = [Exception("container being reloaded"), _device(ctrl)]
stub = _make_stub(devices, ["ftransy"])
assert stub._resolve_controller("ftransy") is ctrl
# ---------------------------------------------------------------- one stop
def test_stop_controller_uses_stop_all_axes():
ctrl = _controller()
stub = _make_stub({"ftransy": _device(ctrl)}, ["ftransy"])
assert stub._stop_controller("ftransy") == (True, ":")
ctrl.stop_all_axes.assert_called_once_with()
ctrl.socket_put_and_receive.assert_not_called()
def test_stop_controller_falls_back_to_raw_command_when_not_exposed():
ctrl = mock.Mock(spec=["socket_put_and_receive"]) # no stop_all_axes attribute
ctrl.socket_put_and_receive.return_value = ":"
stub = _make_stub({"ftransy": _device(ctrl)}, ["ftransy"])
assert stub._stop_controller("ftransy") == (True, ":")
ctrl.socket_put_and_receive.assert_called_once_with("XQ#STOP,1")
def test_stop_controller_falls_back_to_raw_command_when_rpc_call_fails():
ctrl = _controller()
ctrl.stop_all_axes.side_effect = TimeoutError("rpc timeout")
stub = _make_stub({"ftransy": _device(ctrl)}, ["ftransy"])
assert stub._stop_controller("ftransy") == (True, ":")
ctrl.socket_put_and_receive.assert_called_once_with("XQ#STOP,1")
def test_stop_controller_reports_unresolved_device():
devices = mock.MagicMock()
devices.get.return_value = None
stub = _make_stub(devices, ["fsamroy"])
ok, detail = stub._stop_controller("fsamroy")
assert not ok
assert "not resolved" in detail
def test_stop_controller_reports_total_failure():
ctrl = _controller()
ctrl.stop_all_axes.side_effect = RuntimeError("boom")
ctrl.socket_put_and_receive.side_effect = RuntimeError("socket dead")
stub = _make_stub({"ftransy": _device(ctrl)}, ["ftransy"])
ok, detail = stub._stop_controller("ftransy")
assert not ok
assert "socket dead" in detail
# ---------------------------------------------------------------- all stops
def test_stop_all_controllers_stops_each_and_keeps_configured_order():
ctrls = {name: _controller() for name in ("ftransy", "foptx", "fsamroy")}
devices = {name: _device(ctrl) for name, ctrl in ctrls.items()}
stub = _make_stub(devices, ["ftransy", "foptx", "fsamroy"])
results = stub._stop_all_controllers()
assert [name for name, _, _ in results] == ["ftransy", "foptx", "fsamroy"]
assert all(ok for _, ok, _ in results)
for ctrl in ctrls.values():
ctrl.stop_all_axes.assert_called_once_with()
def test_one_unresolved_device_does_not_prevent_the_other_stops():
ctrls = {name: _controller() for name in ("ftransy", "fsamroy")}
devices = mock.MagicMock()
devices.get.side_effect = lambda name: _device(ctrls[name]) if name in ctrls else None
stub = _make_stub(devices, ["ftransy", "foptx", "fsamroy"])
results = {name: ok for name, ok, _ in stub._stop_all_controllers()}
assert results == {"ftransy": True, "foptx": False, "fsamroy": True}
ctrls["ftransy"].stop_all_axes.assert_called_once_with()
ctrls["fsamroy"].stop_all_axes.assert_called_once_with()
def test_hung_controller_does_not_delay_the_others():
release = threading.Event()
hung = _controller()
hung.stop_all_axes.side_effect = lambda: release.wait(5) and ":"
fast = {name: _controller() for name in ("foptx", "fsamroy")}
devices = {"ftransy": _device(hung), **{n: _device(c) for n, c in fast.items()}}
stub = _make_stub(devices, ["ftransy", "foptx", "fsamroy"])
stub._STOP_JOIN_TIMEOUT_S = 0.5
try:
results = {name: (ok, detail) for name, ok, detail in stub._stop_all_controllers()}
finally:
release.set()
assert results["foptx"][0] and results["fsamroy"][0]
assert not results["ftransy"][0]
assert "no answer" in results["ftransy"][1]
for ctrl in fast.values():
ctrl.stop_all_axes.assert_called_once_with()
# ---------------------------------------------------------------- restore
def test_restore_positioning_mode_success_and_failure():
ctrl = _controller()
stub = _make_stub({"ftransy": _device(ctrl)}, ["ftransy"])
assert stub._restore_positioning_mode("ftransy") == (True, "")
ctrl.hard_abort_and_restore_positioning_mode.assert_called_once_with()
ctrl.hard_abort_and_restore_positioning_mode.side_effect = RuntimeError("still in mount mode")
ok, detail = stub._restore_positioning_mode("ftransy")
assert not ok
assert "still in mount mode" in detail
@@ -0,0 +1,105 @@
"""Tests for the Skip button of OMNY_XRayEye (see docs/plans/flomni-xrayeye-skip-button.md):
it is disabled by default (LamNI/omny never enable it), only clickable while the client
allows a skip *and* the submit button is enabled, and a click signals ``submit = -1``
without needing an ROI.
The slots are exercised on a minimal stub instead of the full widget, which needs a live BEC
client context via get_bec_shortcuts() -- same approach as test_x_ray_eye_vertical_lock.py.
"""
from unittest import mock
from csaxs_bec.bec_widgets.widgets.xray_eye.x_ray_eye import OMNY_XRayEye
class _Button:
def __init__(self, enabled=False):
self._enabled = enabled
self._visible = False
def setVisible(self, visible):
self._visible = bool(visible)
def isVisible(self):
return self._visible
def setEnabled(self, enabled):
self._enabled = bool(enabled)
def isEnabled(self):
return self._enabled
class _StubXRayEye:
enable_submit_button = OMNY_XRayEye.enable_submit_button
enable_skip_button = OMNY_XRayEye.enable_skip_button
_refresh_skip_button = OMNY_XRayEye._refresh_skip_button
skip = OMNY_XRayEye.skip
def _make_stub():
stub = _StubXRayEye()
stub._skip_enabled = False
stub.submit_button = _Button(enabled=False)
stub.skip_button = _Button(enabled=False)
stub.dev = mock.Mock()
return stub
def test_skip_button_stays_disabled_unless_the_client_enables_it():
stub = _make_stub()
stub.enable_submit_button(True) # what LamNI/omny do
assert stub.submit_button.isEnabled()
assert not stub.skip_button.isEnabled()
assert not stub.skip_button.isVisible()
def test_skip_button_appears_on_the_first_enable_call():
stub = _make_stub()
stub.enable_skip_button(False) # flomni does this at the start of a run
assert stub.skip_button.isVisible()
assert not stub.skip_button.isEnabled()
def test_skip_button_follows_submit_button_once_enabled():
stub = _make_stub()
stub.enable_skip_button(True)
assert not stub.skip_button.isEnabled() # submit still disabled ("please wait ...")
stub.enable_submit_button(True)
assert stub.skip_button.isEnabled()
stub.enable_submit_button(False)
assert not stub.skip_button.isEnabled()
stub.enable_submit_button(True) # next step is ready again
assert stub.skip_button.isEnabled()
def test_skip_button_can_be_withdrawn_while_submit_stays_enabled():
stub = _make_stub()
stub.enable_skip_button(True)
stub.enable_submit_button(True)
stub.enable_skip_button(False)
assert stub.submit_button.isEnabled()
assert not stub.skip_button.isEnabled()
def test_skip_click_signals_minus_one_and_blocks_double_click():
stub = _make_stub()
stub.enable_skip_button(True)
stub.enable_submit_button(True)
stub.skip()
stub.dev.omny_xray_gui.submit.set.assert_called_once_with(-1)
assert not stub.skip_button.isEnabled()
# the client re-enables the buttons for the next step
stub.enable_submit_button(True)
assert stub.skip_button.isEnabled()
@@ -0,0 +1,102 @@
"""Tests for OMNY_XRayEye's ROI-vertical-lock feature (see
docs/plans/xrayeye-vertical-lock.md): _snap_roi_vertical() snaps a selection
box's vertical center to the crosshair's fixed y position while leaving its
x position and height/diameter untouched, but only while the lock is enabled
and the crosshair is visible.
_snap_roi_vertical is a plain (non-Qt-slot-decorated) method, so it can be
exercised directly on a minimal stub without constructing the full
OMNY_XRayEye widget (which needs a live BEC client context via
get_bec_shortcuts()).
"""
from unittest import mock
from bec_widgets.widgets.plots.roi.image_roi import CircularROI, RectangularROI
from csaxs_bec.bec_widgets.widgets.xray_eye.x_ray_eye import OMNY_XRayEye
class _StubXRayEye:
"""Minimal stand-in for OMNY_XRayEye, isolated to just the attributes
_snap_roi_vertical touches."""
_snap_roi_vertical = OMNY_XRayEye._snap_roi_vertical
def _make_stub(*, lock_enabled: bool, crosshair_visible: bool, fzp_y: float = 42.0):
stub = _StubXRayEye()
stub._vertical_lock_enabled = lock_enabled
stub.target_crosshair = mock.Mock()
stub.target_crosshair.is_visible.return_value = crosshair_visible
stub.crosshair_position = mock.Mock(return_value=(0.0, fzp_y))
return stub
def _make_rect_roi(x: float, y: float, height: float):
roi = mock.Mock(spec=RectangularROI)
roi.get_coordinates.return_value = {"height": height}
roi.pos.return_value = mock.Mock(x=lambda: x, y=lambda: y)
return roi
def test_snap_roi_vertical_snaps_to_crosshair_when_locked():
stub = _make_stub(lock_enabled=True, crosshair_visible=True, fzp_y=42.0)
roi = _make_rect_roi(x=5.0, y=0.0, height=10.0)
stub._snap_roi_vertical(roi)
# new_origin_y = fzp_y - height/2 = 42 - 5 = 37; x is untouched
roi.set_position.assert_called_once_with(5.0, 37.0)
def test_snap_roi_vertical_noop_when_lock_disabled():
stub = _make_stub(lock_enabled=False, crosshair_visible=True)
roi = _make_rect_roi(x=5.0, y=0.0, height=10.0)
stub._snap_roi_vertical(roi)
roi.set_position.assert_not_called()
def test_snap_roi_vertical_noop_when_crosshair_hidden():
stub = _make_stub(lock_enabled=True, crosshair_visible=False)
roi = _make_rect_roi(x=5.0, y=0.0, height=10.0)
stub._snap_roi_vertical(roi)
roi.set_position.assert_not_called()
def test_snap_roi_vertical_noop_when_already_snapped():
# Regression guard: sigRegionChangeFinished is connected to this method,
# so a call that finds the ROI already at the target position must not
# call set_position() again -- otherwise a re-triggered signal could loop.
stub = _make_stub(lock_enabled=True, crosshair_visible=True, fzp_y=42.0)
roi = _make_rect_roi(x=5.0, y=37.0, height=10.0) # already at target y
stub._snap_roi_vertical(roi)
roi.set_position.assert_not_called()
def test_snap_roi_vertical_preserves_x_and_height():
stub = _make_stub(lock_enabled=True, crosshair_visible=True, fzp_y=-10.0)
roi = _make_rect_roi(x=-3.5, y=100.0, height=4.0)
stub._snap_roi_vertical(roi)
# new_origin_y = -10 - 2 = -12; x (-3.5) is preserved
roi.set_position.assert_called_once_with(-3.5, -12.0)
def test_snap_roi_vertical_handles_circular_roi():
stub = _make_stub(lock_enabled=True, crosshair_visible=True, fzp_y=20.0)
roi = mock.Mock(spec=CircularROI)
roi.get_coordinates.return_value = {"diameter": 8.0}
roi.pos.return_value = mock.Mock(x=lambda: 1.0, y=lambda: 0.0)
stub._snap_roi_vertical(roi)
# new_origin_y = fzp_y - diameter/2 = 20 - 4 = 16
roi.set_position.assert_called_once_with(1.0, 16.0)
+319
View File
@@ -3,11 +3,14 @@
# pylint: skip-file
from __future__ import annotations
import threading
from unittest import mock
import numpy as np
import pytest
from ophyd import Staged
from ophyd_devices.devices.panda_box.panda_box import PandaState
from ophyd_devices.interfaces.base_classes.psi_device_base import DeviceStoppedError
from pandablocks.responses import FrameData
from csaxs_bec.devices.panda_box.panda_box import PandaBoxCSAXS
@@ -294,6 +297,322 @@ def test_panda_omny_complete_free_running_timeout(panda_omny_raw_stream):
mock_flush.assert_called_once()
def _deliver_ready_after(dev, delay: float) -> threading.Timer:
"""Simulate the PandA data thread delivering the READY event `delay` seconds from now."""
timer = threading.Timer(delay, dev._run_status_callbacks, args=(PandaState.READY,))
timer.daemon = True
timer.start()
return timer
def test_panda_omny_pre_scan_arms_when_ready(panda_omny, capsys):
dev = panda_omny
dev.pre_scan_timeout = 2
with (
mock.patch.object(dev, "_arm") as mock_arm,
mock.patch.object(dev, "_reset_panda") as mock_reset,
):
_deliver_ready_after(dev, 0.05)
status = dev.on_pre_scan()
status.wait(timeout=4)
assert status.success is True
mock_arm.assert_called_once()
mock_reset.assert_not_called()
assert "Forcing abort/reset" not in capsys.readouterr().out
def test_panda_omny_pre_scan_retries_once_then_succeeds(panda_omny, capsys):
"""No READY on the first attempt: the box is force-reset and arming retried. The retry is
printed and logged."""
dev = panda_omny
dev.pre_scan_timeout = 0.2
def _reset_then_ready():
_deliver_ready_after(dev, 0.05)
with (
mock.patch.object(dev, "_arm") as mock_arm,
mock.patch.object(dev, "_reset_panda", side_effect=_reset_then_ready) as mock_reset,
mock.patch("csaxs_bec.devices.panda_box.panda_box_omny.logger") as mock_logger,
):
status = dev.on_pre_scan()
status.wait(timeout=4)
assert status.success is True
mock_reset.assert_called_once()
mock_arm.assert_called_once()
assert any("Forcing abort/reset" in str(c) for c in mock_logger.warning.call_args_list)
assert any("armed successfully on attempt 2" in str(c) for c in mock_logger.info.call_args_list)
assert "Forcing abort/reset" in capsys.readouterr().out
def test_panda_omny_pre_scan_raises_after_retry_fails(panda_omny, capsys):
dev = panda_omny
dev.pre_scan_timeout = 0.1
with (
mock.patch.object(dev, "_arm") as mock_arm,
mock.patch.object(dev, "_reset_panda") as mock_reset,
mock.patch("csaxs_bec.devices.panda_box.panda_box_omny.logger") as mock_logger,
):
status = dev.on_pre_scan()
with pytest.raises(RuntimeError, match="could not be armed after 2 attempts"):
status.wait(timeout=4)
mock_arm.assert_not_called()
# once for the forced reset before the retry, once to leave the box clean at the end
assert mock_reset.call_count == 2
mock_logger.error.assert_called_once()
assert capsys.readouterr().out.count("Forcing abort/reset") == 1
# stale status callbacks of the failed attempts are removed again
assert dev._status_callbacks == {}
def test_panda_omny_pre_scan_stop_is_not_retried(panda_omny, capsys):
dev = panda_omny
dev.pre_scan_timeout = 0.3
with (
mock.patch.object(dev, "_arm") as mock_arm,
mock.patch.object(dev, "_reset_panda") as mock_reset,
):
status = dev.on_pre_scan()
threading.Event().wait(0.05)
dev._stop_stoppable_status_objects()
with pytest.raises(DeviceStoppedError):
status.wait(timeout=4)
# give a wrongly retrying task thread the time to show up
threading.Event().wait(0.5)
mock_arm.assert_not_called()
mock_reset.assert_not_called()
assert "Forcing abort/reset" not in capsys.readouterr().out
def test_panda_omny_raw_stream_drops_stale_leading_frame_counter_rows(panda_omny_raw_stream):
"""PandA's COUNTER1.OUT is not cleared by Arm(), so the first raw samples of a new
acquisition can briefly carry a stale frame_counter value left over from the previous
acquisition's tail (confirmed 2026-09-21 against a real scan, S06313). Rows before the
first observed frame_counter == 0 must be dropped, and every channel must be truncated
at the same row so they stay aligned."""
dev = panda_omny_raw_stream
# Two stale rows carrying the previous acquisition's terminal counter value (167), then
# the real acquisition starting at frame_counter 0.
data = np.array(
[(167.0, 1.0), (167.0, 0.0), (0.0, 1.0), (0.0, 0.0), (1.0, 1.0)],
dtype=[("COUNTER1.OUT.Value", "<f8"), ("PCAP.BITS0.Value", "<f8")],
)
with mock.patch.object(dev.data, "put") as mock_put:
dev._receive_frame_data(FrameData(data))
dev._flush_raw_stream_buffer()
out = mock_put.call_args.args[0]
assert [v for v in out["frame_counter"]["value"]] == [0.0, 0.0, 1.0]
assert [v for v in out["gate_detector_active"]["value"]] == [1, 0, 1]
def test_panda_omny_raw_stream_drops_whole_chunk_if_still_stale(panda_omny_raw_stream):
"""If a single FrameData batch is entirely stale rows (no frame_counter == 0 observed
yet), it must be dropped in full rather than partially buffered, and the next batch
must still be checked for the reset."""
dev = panda_omny_raw_stream
all_stale = np.array(
[(167.0, 0.0), (167.0, 1.0)], dtype=[("COUNTER1.OUT.Value", "<f8"), ("PCAP.BITS0.Value", "<f8")]
)
real_start = np.array(
[(0.0, 1.0), (1.0, 0.0)], dtype=[("COUNTER1.OUT.Value", "<f8"), ("PCAP.BITS0.Value", "<f8")]
)
with mock.patch.object(dev.data, "put") as mock_put:
dev._receive_frame_data(FrameData(all_stale))
assert dev._raw_stream_buffer == {}
assert dev._raw_stream_seen_reset is False
dev._receive_frame_data(FrameData(real_start))
dev._flush_raw_stream_buffer()
out = mock_put.call_args.args[0]
assert [v for v in out["frame_counter"]["value"]] == [0.0, 1.0]
def test_panda_omny_raw_stream_passes_through_once_reset_seen(panda_omny_raw_stream):
"""After the first frame_counter == 0 has been observed, later rows must never be
filtered again, even if a later point's real frame_counter happens to collide with the
earlier stale value (e.g. point 167 in a scan with more than 167 points)."""
dev = panda_omny_raw_stream
with mock.patch.object(dev.data, "put"):
dev._receive_frame_data(_make_frame_data(3, start=0)) # observes the reset at row 0
assert dev._raw_stream_seen_reset is True
dev._flush_raw_stream_buffer()
# A later chunk whose frame_counter happens to read 167 again must pass through
# untouched -- filtering only ever applies before the first reset.
collide = np.array(
[(167.0, 1.0), (168.0, 0.0)], dtype=[("COUNTER1.OUT.Value", "<f8"), ("PCAP.BITS0.Value", "<f8")]
)
with mock.patch.object(dev.data, "put") as mock_put:
dev._receive_frame_data(FrameData(collide))
dev._flush_raw_stream_buffer()
out = mock_put.call_args.args[0]
assert [v for v in out["frame_counter"]["value"]] == [167.0, 168.0]
def test_panda_omny_raw_stream_state_reset_on_stage(panda_omny_raw_stream):
"""on_stage must reopen the stale-value window (new acquisition) even if a previous
acquisition already observed its reset."""
dev = panda_omny_raw_stream
dev._raw_stream_seen_reset = True
dev._raw_stream_buffer = {"leftover": {"value": [1, 2, 3], "timestamp": 0.0}}
dev.stage()
assert dev._raw_stream_seen_reset is False
assert dev._raw_stream_buffer == {}
def test_panda_omny_pre_scan_arm_resets_raw_stream_state(panda_omny):
"""Each Arm() resets COUNTER1's counting on the PandA side, so every successful arm in
on_pre_scan -- not just the one implied by on_stage -- must reopen the stale-frame_counter
window right before Arm() is sent. This matters most for a retried arm after a forced
abort/reset, since on_stage's reset has already happened by then."""
dev = panda_omny
dev.pre_scan_timeout = 2
with (
mock.patch.object(dev, "_arm") as mock_arm,
mock.patch.object(dev, "_reset_panda"),
mock.patch.object(dev, "_reset_raw_stream_state") as mock_reset_raw,
):
_deliver_ready_after(dev, 0.05)
status = dev.on_pre_scan()
status.wait(timeout=4)
assert status.success is True
mock_arm.assert_called_once()
mock_reset_raw.assert_called_once()
def test_panda_omny_arm_scan_allowlist_defaults_to_none(panda_omny):
"""A device constructed without arm_scan_allowlist must arm for every scan (today's
behavior), both via the raw config attribute and the computed should-arm flag."""
dev = panda_omny
assert dev.arm_scan_allowlist is None
assert dev._should_arm_panda is True
def test_panda_omny_stage_should_arm_stays_true_when_allowlist_none(panda_omny):
"""Backward-compatibility regression: staging with the default allowlist (None) must keep
_should_arm_panda True regardless of the scan_name in scan_info."""
dev = panda_omny
dev.stage()
assert dev._should_arm_panda is True
def test_panda_omny_stage_should_arm_true_when_scan_name_in_allowlist(panda_omny):
dev = panda_omny
dev.arm_scan_allowlist = ["flomni_fermat_scan"]
dev.scan_info.msg.info["scan_name"] = "flomni_fermat_scan"
dev.stage()
assert dev._should_arm_panda is True
def test_panda_omny_stage_should_arm_false_when_scan_name_not_in_allowlist(panda_omny):
dev = panda_omny
dev.arm_scan_allowlist = ["flomni_fermat_scan"]
dev.scan_info.msg.info["scan_name"] = "grid_scan"
dev.stage()
assert dev._should_arm_panda is False
def test_panda_omny_should_arm_recomputed_every_stage(panda_omny):
"""The same device instance is staged for many different scan types across a session; the
should-arm decision must be recomputed fresh on every stage, not stick from a previous
scan."""
dev = panda_omny
dev.arm_scan_allowlist = ["flomni_fermat_scan"]
with mock.patch.object(dev, "_disarm"):
dev.scan_info.msg.info["scan_name"] = "grid_scan"
dev.stage()
assert dev._should_arm_panda is False
dev.unstage()
dev.scan_info.msg.info["scan_name"] = "flomni_fermat_scan"
dev.stage()
assert dev._should_arm_panda is True
def test_panda_omny_pre_scan_skips_arm_when_should_arm_false(panda_omny):
"""When arm_scan_allowlist excludes the current scan, on_pre_scan must resolve immediately
without sending Arm() or spinning up the retry task at all."""
dev = panda_omny
dev._should_arm_panda = False
with (
mock.patch.object(dev, "_arm") as mock_arm,
mock.patch.object(dev, "_arm_with_retry") as mock_arm_with_retry,
mock.patch.object(dev.task_handler, "submit_task") as mock_submit_task,
):
status = dev.on_pre_scan()
assert status.done is True
assert status.success is True
mock_arm.assert_not_called()
mock_arm_with_retry.assert_not_called()
mock_submit_task.assert_not_called()
def test_panda_omny_pre_scan_arms_when_should_arm_true_via_allowlist(panda_omny):
"""End-to-end: a scan_name matching arm_scan_allowlist still goes through the full arm/
retry machinery exactly as with the default (None) allowlist."""
dev = panda_omny
dev.pre_scan_timeout = 2
dev.arm_scan_allowlist = ["flomni_fermat_scan"]
dev.scan_info.msg.info["scan_name"] = "flomni_fermat_scan"
dev.stage()
assert dev._should_arm_panda is True
with (
mock.patch.object(dev, "_arm") as mock_arm,
mock.patch.object(dev, "_reset_panda") as mock_reset,
):
_deliver_ready_after(dev, 0.05)
status = dev.on_pre_scan()
status.wait(timeout=4)
assert status.success is True
mock_arm.assert_called_once()
mock_reset.assert_not_called()
def test_panda_omny_complete_skips_wait_loop_when_should_arm_false(panda_omny):
"""When this device was never armed for the current scan, on_complete must resolve
immediately without polling PandA or disarming it."""
dev = panda_omny
dev._should_arm_panda = False
with (
mock.patch.object(dev, "send_raw") as mock_send_raw,
mock.patch.object(dev, "_disarm") as mock_disarm,
mock.patch.object(dev.task_handler, "submit_task") as mock_submit_task,
):
status = dev.on_complete()
assert status.done is True
assert status.success is True
mock_send_raw.assert_not_called()
mock_disarm.assert_not_called()
mock_submit_task.assert_not_called()
def test_panda_omny_complete_skips_wait_loop_when_should_arm_false_raw_stream_mode(
panda_omny_raw_stream,
):
"""Same as test_panda_omny_complete_skips_wait_loop_when_should_arm_false, but on the
raw_stream_mode=True path that would otherwise dispatch to _on_complete_free_running."""
dev = panda_omny_raw_stream
dev._should_arm_panda = False
with (
mock.patch.object(dev, "send_raw") as mock_send_raw,
mock.patch.object(dev, "_disarm") as mock_disarm,
mock.patch.object(dev, "_flush_raw_stream_buffer") as mock_flush,
mock.patch.object(dev.task_handler, "submit_task") as mock_submit_task,
):
status = dev.on_complete()
assert status.done is True
assert status.success is True
mock_send_raw.assert_not_called()
mock_disarm.assert_not_called()
mock_flush.assert_not_called()
mock_submit_task.assert_not_called()
def test_panda_omny_raw_stream_signal_alias(panda_omny_raw_stream):
all_signal_names = [name for name, _ in panda_omny_raw_stream.data.signals]
assert "gate_detector_active" in all_signal_names
@@ -0,0 +1,14 @@
from csaxs_bec.scans.fermat_spiral_utils import compute_fermat_spiral_n_max
def test_n_max_unchanged_for_square_box():
# the old area-based term already dominates for a square/near-square box,
# so the corner-radius-based term must not change the result there
assert compute_fermat_spiral_n_max(50, 50, 1) == int(50 * 50 * 3.2)
def test_n_max_boosted_for_elongated_box():
# for a long, narrow box the old area-based term under-estimates the
# iterations needed to reach the far corners, so the fixed result must
# exceed it
assert compute_fermat_spiral_n_max(220, 30, 1) > int(220 * 30 * 3.2)
@@ -1,5 +1,7 @@
from unittest import mock
import numpy as np
from csaxs_bec.scans.flomni_fermat_scan import FlomniFermatScan
@@ -43,3 +45,110 @@ def test_flomni_rotation_moves_when_setpoint_matches_but_readback_far_off():
FlomniFermatScan.flomni_rotation(scan, 10.0)
scan.actions.set.assert_called_once()
def test_flomni_fermat_spiral_reaches_elongated_fov_corners():
# Regression test: for an elongated (non-square) FOV, the spiral's iteration
# cap used to be sized from FOV area alone, which under-estimated the
# iterations needed and left the loop-generated points well short of the
# long axis's outer region (only the two explicit corner points reached it).
fovx, fovy = 220.0, 30.0
pos = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-fovx / 2, fovx / 2, -fovy / 2, fovy / 2, step=1
)
loop_points = pos[:-2] # exclude the two corner points appended after the loop
assert np.max(np.abs(loop_points[:, 0])) > 0.95 * (fovx / 2)
assert len(pos) > FlomniFermatScan.MIN_POSITIONS
def test_fermat_asymmetry_default_matches_symmetric_spiral():
# fermat_asymmetry=1.0 (or omitted) must reproduce exactly today's positions --
# no-op / backward-compatibility check. Same seed on both calls isolates this
# from the (unrelated) per-call random sub-step offset.
fovx, fovy = 10.0, 10.0
default_call = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-fovx / 2, fovx / 2, -fovy / 2, fovy / 2, step=1, seed=0
)
explicit_symmetric = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-fovx / 2, fovx / 2, -fovy / 2, fovy / 2, step=1, seed=0, fermat_asymmetry=1.0
)
np.testing.assert_array_equal(default_call, explicit_symmetric)
def test_fermat_asymmetry_biases_spacing_while_preserving_fov():
# Directly verifies the documented "generate against an inflated fovy, then
# compress y back down" design: get_flomni_fermat_spiral_pos(..., fovy,
# fermat_asymmetry=ratio) must select exactly the same raw lattice points as
# get_flomni_fermat_spiral_pos(..., fovy*ratio, fermat_asymmetry=1.0) (same
# seed => same random sub-step offset), just with y compressed by `ratio`.
fovx, fovy, step = 40.0, 40.0, 1.0
for ratio in (2.0, 0.5):
asym = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-fovx / 2, fovx / 2, -fovy / 2, fovy / 2, step=step, seed=42, fermat_asymmetry=ratio
)
inflated_symmetric = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-fovx / 2, fovx / 2, -(fovy * ratio) / 2, (fovy * ratio) / 2, step=step, seed=42
)
asym_loop = asym[:-2]
inflated_loop = inflated_symmetric[:-2]
assert len(asym_loop) == len(inflated_loop)
np.testing.assert_allclose(asym_loop[:, 0], inflated_loop[:, 0])
np.testing.assert_allclose(asym_loop[:, 1] * ratio, inflated_loop[:, 1])
# FOV preserved
assert np.max(np.abs(asym_loop[:, 0])) <= fovx / 2
assert np.max(np.abs(asym_loop[:, 1])) <= fovy / 2
def test_fermat_asymmetry_out_of_range_is_clamped_not_raised():
fovx, fovy, step = 10.0, 10.0, 1.0
pos_too_large = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-fovx / 2, fovx / 2, -fovy / 2, fovy / 2, step=step, seed=0, fermat_asymmetry=100.0
)
pos_clamped = FlomniFermatScan.get_flomni_fermat_spiral_pos(
-fovx / 2,
fovx / 2,
-fovy / 2,
fovy / 2,
step=step,
seed=0,
fermat_asymmetry=FlomniFermatScan.MAX_FERMAT_ASYMMETRY,
)
np.testing.assert_array_equal(pos_too_large, pos_clamped)
loop_points = pos_too_large[:-2]
assert np.max(np.abs(loop_points[:, 0])) <= fovx / 2
assert np.max(np.abs(loop_points[:, 1])) <= fovy / 2
def test_randomize_offset_default_is_active():
# Default keeps the per-call random sub-step offset: different seeds give different
# lattices, and omitting randomize_offset equals randomize_offset=True.
fovx, fovy = 10.0, 10.0
args = (-fovx / 2, fovx / 2, -fovy / 2, fovy / 2)
a = FlomniFermatScan.get_flomni_fermat_spiral_pos(*args, step=1, seed=1)
b = FlomniFermatScan.get_flomni_fermat_spiral_pos(*args, step=1, seed=2)
c = FlomniFermatScan.get_flomni_fermat_spiral_pos(
*args, step=1, seed=1, randomize_offset=True
)
assert a.shape != b.shape or not np.allclose(a, b)
np.testing.assert_array_equal(a, c)
def test_randomize_offset_false_gives_unshifted_deterministic_lattice():
# randomize_offset=False ignores the seed and is identical from call to call.
fovx, fovy = 10.0, 10.0
args = (-fovx / 2, fovx / 2, -fovy / 2, fovy / 2)
a = FlomniFermatScan.get_flomni_fermat_spiral_pos(
*args, step=1, seed=1, randomize_offset=False
)
b = FlomniFermatScan.get_flomni_fermat_spiral_pos(
*args, step=1, seed=2, randomize_offset=False
)
d = FlomniFermatScan.get_flomni_fermat_spiral_pos(*args, step=1, randomize_offset=False)
np.testing.assert_array_equal(a, b)
np.testing.assert_array_equal(a, d)
# unshifted lattice: first loop point (ii=1) is at radius step*0.57, no offset
phi = 2 * np.pi * ((1 + np.sqrt(5)) / 2.0)
r = 0.57
np.testing.assert_allclose(a[0, :2], [r * np.sin(phi), r * np.cos(phi)])
@@ -44,3 +44,16 @@ def test_lamni_rotation_moves_when_setpoint_matches_but_not_settled():
scan.dev.lsamrot.set.assert_called_once_with(10.0)
scan.dev.lsamrot.set.return_value.wait.assert_called_once_with()
def test_lamni_fermat_spiral_reaches_elongated_fov_corners():
# Regression test: for an elongated (non-square) FOV, the spiral's iteration
# cap used to be sized from FOV area alone, which under-estimated the
# iterations needed and left coverage well short of the long axis's outer
# region. lamni's returned x/y are post-transform-shift, so assert on point
# count growth (46 -> 73 for this case) rather than raw coordinate magnitude.
fovx, fovy = 79.0, 2.0
pos = LamniFermatScan.get_lamni_fermat_spiral_pos(
-fovx / 2, fovx / 2, -fovy / 2, fovy / 2, step=1, fovx=fovx, fovy=fovy
)
assert len(pos) >= 60
@@ -1,5 +1,7 @@
from unittest import mock
import numpy as np
from csaxs_bec.scans.omny_fermat_scan import OmnyFermatScan
@@ -42,3 +44,22 @@ def test_omny_rotation_moves_when_setpoint_matches_but_readback_far_off():
OmnyFermatScan.omny_rotation(scan, 10.0)
scan.actions.set.assert_called_once()
class _PosStub:
cenx = 0.0
ceny = 0.0
zshift = 0.0
def test_omny_fermat_spiral_reaches_elongated_fov_corners():
# Regression test: for an elongated (non-square) FOV, the spiral's iteration
# cap used to be sized from FOV area alone, which under-estimated the
# iterations needed and left the loop-generated points well short of the
# long axis's outer region (only the two explicit corner points reached it).
fovx, fovy = 190.0, 20.0 # stay under omny's fovx<200 / fovy<100 argument bounds
pos = OmnyFermatScan.get_omny_fermat_spiral_pos(
_PosStub(), -fovx / 2, fovx / 2, -fovy / 2, fovy / 2, step=1
)
loop_points = pos[:-2] # exclude the two corner points appended after the loop
assert np.max(np.abs(loop_points[:, 0])) > 0.95 * (fovx / 2)