Merge branch 'main' into fix/flomni_store_ref_images
CI for csaxs_bec / test (pull_request) Successful in 3m24s
Read the Docs Deploy Trigger / trigger-rtd-webhook (push) Successful in 2s
CI for csaxs_bec / test (push) Successful in 3m27s

This commit was merged in pull request #340.
This commit is contained in:
holler committed 2026-10-26 14:09:46 +01:00
commit d0e9df21a8
265 files changed
+33471 -4282

No files matched your search

+4 -1
View File
@@ -155,7 +155,7 @@ venv.bak/
# Rope project settings
.ropeproject
# mkdocs documentation
# Zensical documentation
/site
# mypy
@@ -178,3 +178,6 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
# Local documentation tooling
.venv-docs/
+7 -20
View File
@@ -1,27 +1,14 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
version: 2
# Set the version of Python and other tools you might need
build:
os: ubuntu-24.04
tools:
python: "3.12"
jobs:
pre_install:
- pip install .
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py
# If using Sphinx, optionally build your docs in additional formats such as PDF
# formats:
# - pdf
# Optionally declare the Python requirements required to build your docs
python:
install:
- requirements: docs/requirements.txt
install:
- python -m pip install -e '.[docs]'
build:
html:
- python -m zensical build --clean
- mkdir -p "$READTHEDOCS_OUTPUT/html"
- cp -a site/. "$READTHEDOCS_OUTPUT/html/"
+62
View File
@@ -0,0 +1,62 @@
# Personal notes — mirko
Personal working notes for Claude Code sessions in this repo, for my own use across machines.
Not project-wide guidance — nothing here applies to other contributors, and nothing else in
this repo references or enforces it.
## Repo ownership boundaries
Root causes traced into BEC core (`bec_lib`/`bec_server`/`bec_ipython_client`, the `bec` repo)
or into core `bec_widgets` are not mine to fix here — those are owned by their respective core
teams. When an investigation in `csaxs_bec` turns up a bug whose fix belongs in one of those
repos, write it up (a short report or plan) instead of implementing/committing it there.
Fixes that land in `csaxs_bec` itself (beamline plugin code, widgets, device configs, scans)
are mine to implement, commit, and push directly.
## Where notes go
Never write plans, TODOs or notes to Claude's local memory / local plan files (they are
machine-local and invisible on other machines). Put them in this file, or in a markdown file in
the repo when I ask for one.
## TODO
- 2026-09-21: port "always define `tomo_alignment_fit`" from Flomni to OMNY. Flomni is done
(commit `5b688df`): `reset_tomo_alignment_fit()` writes zeros instead of deleting the global
var, and `ensure_tomo_alignment_fit()` (called from `Flomni.__init__`) creates the zero fit
only if missing, so manual tweaks like `fit[0][2] += 5.0` work. OMNY
(`omny/omny_alignment_mixin.py`) still deletes the var and never defines it at startup; nothing
in `omny/` calls the reset yet, so decide where OMNY should reset it. LamNI does not use it.
Details and steps: `csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/TODO_tomo_alignment_fit_always_defined.md`.
- 2026-09-20: `flomni_fermat_scan` hung on `pre_scan_all_devices()` after positions were
already sent to the rt controller (scans 11998, 12000 in `/var/log/bec/ScanServer.log` /
`DeviceServer.log`). Positions/PID-correction step (`prepare_setup_part_2()`) completed
fine; `kickoff()`/`start_scan()` (the actual rt-side trigger, logged as `Read N out of 106`)
was never reached because the joined `pre_scan_all_devices()` wait never returned. No error,
no timeout — DeviceServer just went silent until manually aborted.
Most likely culprit: `omny_panda_continuous` (PandaBox) — `on_pre_scan()` in
`ophyd_devices/devices/panda_box/panda_box.py:696` arms the PCAP block and returns a status
that only resolves on a `PandaState.READY`/`FRAME`/`END` event, with **no timeout**. Logs
show `Cannot emit async signal omny_panda_continuous_data ... with status aborted` spamming
throughout the hang window, consistent with the box being left in a stuck/unarmed state after
the preceding rapid interlock-triggered abort/retry cycle (11991–12001, ~11 attempts in 5
minutes).
This traces into `ophyd_devices`, not `csaxs_bec` — per the ownership note above, not mine to
fix directly here. Follow-ups to consider: (a) file/flag this with whoever owns
`ophyd_devices` (add a timeout + explicit failure path to `PandaBox.on_pre_scan`'s status
callback so a stuck arm surfaces as an error instead of hanging forever), (b) check whether
`on_unstage`/`_reset_panda()` is reliably called on scan abort so the box doesn't carry a
stale state into the next attempt.
**Status 2026-09-20 (csaxs_bec side done):** `PandaBoxOMNY.on_pre_scan`
(`csaxs_bec/devices/panda_box/panda_box_omny.py`) no longer waits for READY forever. Each
arming attempt waits `pre_scan_timeout` (default 5 s); if READY does not arrive, the box is
force-reset (`_reset_panda()`), the readout thread restarted and arming retried once (retry is
printed and logged as a warning with diagnostics). Only if the second attempt fails as well
does the status fail with a `RuntimeError`, so the scan aborts instead of hanging.
Still open for the ophyd_devices owner: the base-class race where the readout loop's
`finally: _reset_panda()` can clear `data_thread_run_event` after a fresh `on_stage()` set it
(likely root cause of the missing READY after rapid abort/retry); the new diagnostics line in
the DeviceServer log (`data_thread_run_event=False`) would confirm it.
+26
View File
@@ -134,3 +134,29 @@ scans.line_scan(dev.samx, -1, 1, dev.samy, -1, 1, steps=20, exp_time=0.5, readou
scans.sgalil_grid(start_y = , end_y = , interval_y = , start_x=, end_x=, interval_x =, exp_time=0.5, readout_time=3e-3, relative=True)
```
## Documentation
The beamline documentation is built with Zensical. See the
[editing guide](docs/developer/editing_docs.md) for authoring conventions.
From the repository root, create a documentation environment and start the preview:
```bash
python -m venv .venv-docs
source .venv-docs/bin/activate
python -m pip install -e '.[docs]'
zensical serve
```
Open the local URL printed by Zensical. Zensical reloads the preview when you edit
pages. For later sessions, activate the environment and run the same `zensical serve`
command. Stop the preview with Ctrl+C.
To build the static site in `site/`, run:
```bash
zensical build --clean
```
The `docs` extra in `pyproject.toml` adds Zensical and PyMdown Extensions to the
repository's regular dependencies.
@@ -1,5 +1,53 @@
# flomni → lamni feature gaps still open (2026-07 sweep)
> **Status update (2026-09-25 go-live audit).** The sections below are the
> original July findings; several are done since. Current state:
>
> - **Done**: §1 timing log + `scilog_last_ptycho_scans()`, §2 account-change
> reset, §4 `lfzp_in()` skip-if-in-position, §6 `tomo_alignment_scan()` (12
> angles over 360°, `write_alignment_scan_numbers()`). §7 (OSA clearance in
> `lfzp_info()`) is implemented but **inactive** until `loptz`'s
> `collision_offset` userParameter is commissioned in `ptycho_lamni.yaml`.
> - **Still open**: §3 `collect_empty_frames()` (needs a decision whether
> lamni's reconstruction uses flat fields), §5 hard-stop button in the GUI
> (see the caveat in the audit notes below), §9 status-page rows (zero-degree
> reference toggle is now shown; golden damage-check shots already were).
> - **Fixed by the September audit** (LamNI compared against the running
> flomni; tests in `tests/tests_bec_ipython_client/test_lamni_golive_fixes.py`
> and siblings): projections silently dropped on a TimeoutError alarm
> (also in `tomo_alignment_scan`); `leye_out()` opened the shutter instead of
> closing it; endless golden-ratio scan crashed on its first projection and
> any GUI failure aborted the scan; scilog writes used the removed
> `bec.logbook` API; `raise_window()` could hide the GUI; tomo_scan_projection
> now reconstructs itself like flomni (custom hooks were never reconstructed);
> PDF report ignored the tomo type; CLI range guards; special-angle
> `UnboundLocalError` (also fixed in flomni); tomo-type-3 sub-tomogram
> boundary float truncation; `lfzp_info()`/queue "move Energy" used the
> legacy `mokev` device (now `ccm_energy`); webpage showed half the
> type-1 projection count; GUI manual-shift unit label (mm, not µm) and a
> bunch-size floor of 20; x-ray-eye `align()` cleanup on any failure;
> `xrayeye_update_frame()` beam-state warning.
> - **Open questions found by the audit** (need a hardware/design decision, not
> changed): default `lopty.limits = [3.0, 3.6]` excludes the FZP-in position
> 3.724 from `ptycho_lamni.yaml`; no OSA collision guard in `losa_in()` (needs
> LamNI's own geometry constant); the hard-stop button must not blindly reuse
> flomni's `hard_abort_and_restore_positioning_mode()` (it runs `XQ#POSMODE`,
> a transfer routine LamNI's Galil program may not have; LamNI has one Galil,
> a SmarAct controller for the OSA and the RT controller);
> `_write_tomo_scan_number()` writes the literal `'lamni'` (flomni: sanitized
> sample name, 0cadd85) and `.3f` angles (flomni `.5f`); `LamniFermatScan`
> lacks flomni's `randomize_offset` switch and `MAX_CORRIDOR_SIZE` cap
> (19cfee3, 53f246a); `tomo_alignment_scan()` always runs the full eye-out
> sequence (flomni skips it when already out).
> - **Behaviour to be aware of**: flomni's `tomo_alignment_scan()` queues its
> projections for reconstruction (it goes through `tomo_scan_projection`);
> LamNI's deliberately does not (`_reconstruct=False`).
> - **Same latent issues remain in flomni** (running in production, not
> touched here): tomo-type-3 sub-tomogram boundary float truncation
> (`flomni.py` `_golden_equally_spaced`), and the queue "move Energy" device
> name `mokev` (`_TOMO_QUEUE_MOVE_DEVICES`) which is only defined in
> `device_configs/legacy/`.
Follow-up to `FLOMNI_TO_LAMNI_COMPARISON.md` (flomni's `AI_docs/`) and this
folder's `TOMO_QUEUE_PORT.md`/`TOMO_PARAMS_GUI_PORT.md` — those covered the
tomo-queue backend, the GUI, the webpage generator, and the angle-distribution
@@ -82,12 +130,13 @@ implementation plan — nothing here has been started.
## 5. No hard-stop button wired into lamni's GUI
- flomni: `flomnigui_show_cameras()` (`flomni/gui_tools.py:143-150`) wires up
a `ConsoleButtonsWidget` (`hard_stop_device_name="ftransy"`,
`extra_hard_stop_device_name="foptx"`) calling the shared, generic
`GalilController.hard_abort_and_restore_positioning_mode()` — added
specifically to replace an older blind stop-all-devices broadcast that
could crash the scan worker thread.
- flomni: `flomnigui_show_cameras()` (`flomni/gui_tools.py`) wires up
a `ConsoleButtonsWidget` (`hard_stop_device_names=["ftransy", "foptx",
"fsamroy"]`, one device per Galil controller, transfer controller first)
that sends `stop_all_axes()` to each controller and then calls the shared,
generic `GalilController.hard_abort_and_restore_positioning_mode()` on the
first — added specifically to replace an older blind stop-all-devices
broadcast that could crash the scan worker thread.
- lamni: `LamNI/gui_tools.py` has no `ConsoleButtonsWidget`/hard-stop wiring
at all.
- The original commit message explicitly notes this was "scoped to flomni
@@ -46,6 +46,8 @@ class LamniWebpageGenerator(WebpageGeneratorBase):
sample name, and measurement settings.
"""
CLIENT_NAME = "lamni"
# LamNI's tomo-queue jobs (added via TomoQueueMixin, see
# OMNY_shared/tomo_queue_mixin.py) are persisted under the same
# "tomo_queue" global var key flomni uses, so the shared "Tomo queue"
@@ -59,7 +61,11 @@ class LamniWebpageGenerator(WebpageGeneratorBase):
# FlomniWebpageGenerator's TOMO_TYPES -- confirmed against lamni.py's
# actual tomo_scan()/sub_tomo_scan() implementation, not a guess.
TOMO_TYPES = {
1: {"kind": "equally_spaced_grid", "n_subtomos": 8},
# base_angle 360: each lamni sub-tomogram sweeps the full circle
# (N = int(360 / step), LamNI._tomo_type1_actual_grid()), unlike
# flomni's 180-degree sub-tomograms -- without it the page showed
# half the real projection count (144 instead of 288 at 10 deg).
1: {"kind": "equally_spaced_grid", "n_subtomos": 8, "base_angle": 360},
2: {"kind": "golden_capped"},
3: {"kind": "golden_capped"},
}
@@ -72,6 +78,12 @@ class LamniWebpageGenerator(WebpageGeneratorBase):
("tomo_type", "Type"),
("tomo_countingtime", "Exposure (s)"),
("tomo_circfov", "Circular FOV (µm)"),
# the piezo range is the FOV of each ptycho tile actually scanned
("lamni_piezo_range_x", "Piezo range x (µm)"),
("lamni_piezo_range_y", "Piezo range y (µm)"),
("manual_shift_x", "Manual shift x (mm)"),
("manual_shift_y", "Manual shift y (mm)"),
("tomo_angle_file", "Angle file"),
("tomo_angle_stepsize", "Angle step (°)"),
("tomo_shellstep", "Shell step (µm)"),
("lamni_stitch_x", "Stitch x"),
@@ -82,6 +94,7 @@ class LamniWebpageGenerator(WebpageGeneratorBase):
("golden_ratio_bunch_size", "Golden bunch size"),
("golden_max_number_of_projections", "Golden max projections"),
("golden_projections_at_0_deg_for_damage_estimation", "0° damage-check shots"),
("zero_deg_reference_at_each_subtomo", "0° reference per sub-tomo"),
("ptycho_reconstruct_foldername", "Recon folder"),
]
@@ -18,6 +18,11 @@ import numpy as np
from bec_lib import bec_logger
from bec_lib.alarm_handler import AlarmBase
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.filter_check import (
filters_out_of_beam,
warn_and_confirm,
)
from .lamni import LamNI
logger = bec_logger.logger
@@ -56,8 +61,8 @@ class MagLamNI(LamNI):
lamni_at_each_angle(self, angle)
return
self.tomo_scan_projection(angle)
self.tomo_reconstruct()
# tomo_scan_projection() adds the projection to the reconstruction queue
self.tomo_scan_projection(angle, _internal=True)
class DataDrivenLamNI(LamNI):
@@ -87,6 +92,14 @@ class DataDrivenLamNI(LamNI):
"""
bec = builtins.__dict__.get("bec")
scans = builtins.__dict__.get("scans")
dev = builtins.__dict__.get("dev")
all_out, offending = filters_out_of_beam(dev)
if not all_out:
warning = f"Not all filters are out of the beam: {', '.join(offending)}."
if not warn_and_confirm(self, warning, interactive=True, force=False):
print("Aborting tomo scan.")
return
bec.builtin_actors.scan_interlock.trigger_setting = "restart_scan"
bec.builtin_actors.scan_interlock.enabled = True
@@ -97,9 +110,7 @@ class DataDrivenLamNI(LamNI):
content = f"Loading tomo parameters from {fname}."
logger.warning(content)
msg = bec.logbook.LogbookMessage()
msg.add_text(content).add_tag(["Data_driven_file", "BEC"])
self.client.logbook.send_logbook_message(msg)
self.write_to_scilog(content, ["Data_driven_file"])
self._update_tomo_data_from_file(fname)
self._current_special_angles = self.special_angles.copy()
@@ -163,6 +174,10 @@ class DataDrivenLamNI(LamNI):
error_caught = False
if 0 <= angle < 360.05:
print(f"Starting LamNI scan for angle {angle}")
# default for angles that aren't a special angle; without it
# num_repeats is unbound whenever special_angles is set but
# this angle isn't near the next one in the list
num_repeats = 1
while not successful:
self._start_beam_check()
if not self.special_angles:
@@ -4,6 +4,10 @@ import builtins
import datetime
import time
from bec_lib import bec_logger
logger = bec_logger.logger
if builtins.__dict__.get("bec") is not None:
bec = builtins.__dict__.get("bec")
dev = builtins.__dict__.get("dev")
@@ -41,13 +45,19 @@ class LamniGuiTools:
def lamnigui_show_gui(self):
if "lamni" in self.gui.windows:
self.lamni_window = self.gui.windows["lamni"]
self.gui.lamni.raise_window()
# raise_window() is currently buggy (bec_widgets RPCServer "raise" handling can
# detach/hide the window instead of bringing it to front) - disabled so that
# calling a lamnigui_show_* macro again on an already-open GUI is a safe no-op
# instead of risking hiding the window. Same workaround as flomni's
# flomnigui_show_gui().
# self.gui.lamni.raise_window()
else:
self.lamni_window = self.gui.new("lamni", timeout=self.GUI_RPC_TIMEOUT)
time.sleep(1)
def lamnigui_raise(self):
self.gui.lamni.raise_window()
# raise_window() disabled, see lamnigui_show_gui().
pass
def lamnigui_stop_gui(self):
self.gui.lamni.hide()
@@ -198,20 +208,27 @@ class LamniGuiTools:
# ------------------------------------------------------------------
def lamnigui_show_progress(self):
self.lamnigui_show_gui()
if self._lamnigui_is_missing("progressbar"):
self.lamnigui_remove_all_docks()
self.progressbar = self.gui.lamni.new(
"RingProgressBar", timeout=self.GUI_RPC_TIMEOUT
)
# Outer ring: overall tomo progress (manual update)
self.progressbar.add_ring().set_update("manual")
# Middle ring: current sub-tomo progress (manual update)
self.progressbar.add_ring().set_update("manual")
# Inner ring: current scan progress (driven by BEC scan events)
self.progressbar.add_ring().set_update("scan")
# The progress GUI is a display only: it must never be able to abort
# a tomogram (a closed GUI / RPC timeout raising out of here would
# propagate through @scan_repeat and fail the projection). Same
# safeguard as flomni's flomnigui_show_progress() (dea43ed).
try:
self.lamnigui_show_gui()
if self._lamnigui_is_missing("progressbar"):
self.lamnigui_remove_all_docks()
self.progressbar = self.gui.lamni.new(
"RingProgressBar", timeout=self.GUI_RPC_TIMEOUT
)
# Outer ring: overall tomo progress (manual update)
self.progressbar.add_ring().set_update("manual")
# Middle ring: current sub-tomo progress (manual update)
self.progressbar.add_ring().set_update("manual")
# Inner ring: current scan progress (driven by BEC scan events)
self.progressbar.add_ring().set_update("scan")
self._lamnigui_update_progress()
self._lamnigui_update_progress()
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Error in lamnigui_show_progress: {exc}")
def _lamnigui_update_progress(self):
"""Update the progress ring bar and centre label from the current progress state.
@@ -220,18 +237,35 @@ class LamniGuiTools:
so this reflects the live state accessible from any BEC client session via::
client.get_global_var("tomo_progress")
Never raises: called from _print_progress() inside every projection, so a
GUI failure must not fail (and, via @scan_repeat, retry) the scan itself.
"""
try:
self._lamnigui_update_progress_impl()
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Error updating the LamNI progress GUI: {exc}")
@staticmethod
def _percent(numerator, denominator) -> float:
"""numerator/denominator in percent; 0 when the denominator is not
positive -- e.g. the total of an "endless" golden-ratio scan
(golden_max_number_of_projections == 0), which has no meaningful
percentage."""
if not denominator or denominator <= 0:
return 0.0
return numerator / denominator * 100
def _lamnigui_update_progress_impl(self):
if self.progressbar is None:
return
main_progress_ring = self.progressbar.rings[0]
subtomo_progress_ring = self.progressbar.rings[1]
progress = self.progress["projection"] / self.progress["total_projections"] * 100
subtomo_progress = (
self.progress["subtomo_projection"]
/ self.progress["subtomo_total_projections"]
* 100
progress = self._percent(self.progress["projection"], self.progress["total_projections"])
subtomo_progress = self._percent(
self.progress["subtomo_projection"], self.progress["subtomo_total_projections"]
)
main_progress_ring.set_value(progress)
subtomo_progress_ring.set_value(subtomo_progress)
@@ -269,11 +303,14 @@ class LamniGuiTools:
else:
finish_display = "N/A"
total = self.progress["total_projections"]
total_display = f"{total:.0f}" if total and total > 0 else "endless"
text = (
f"Progress report:\n"
f" Tomo type: {self.progress['tomo_type']}\n"
f" Projection: {self.progress['projection']:.0f}\n"
f" Total projections expected: {self.progress['total_projections']:.1f}\n"
f" Total projections expected: {total_display}\n"
f" Angle: {self.progress['angle']:.1f}\n"
f" Current subtomo: {self.progress['subtomo']}\n"
f" Current projection within subtomo: {self.progress['subtomo_projection']}\n"
File diff suppressed because it is too large. Load diff
@@ -17,6 +17,7 @@ from __future__ import annotations
import builtins
import os
import h5py
import numpy as np
from bec_lib import bec_logger
from typeguard import typechecked
@@ -25,6 +26,27 @@ logger = bec_logger.logger
dev = builtins.__dict__.get("dev")
# Raw per-angle offsets of the most recent X-ray eye alignment, as written by
# XrayEyeAlign.write_output(): [angles_deg, x_offsets_um, y_offsets_um].
XEYE_FIT_DATA_VAR = "lamni_xeye_fit_data"
def fit_xray_eye_sine(angles_deg, values):
"""Least-squares fit of ``A * sin(radians(angle) + B) + C`` to *values*.
Same model as the XRayEye GUI's DAP fit (SineModel with the frequency
fixed to 1 cycle/360 deg, plus a constant) and the same convention as
``tomo_fit_xray_eye``. Written as ``a*sin + b*cos + C`` the model is
linear, so the fit is exact and needs no start values.
Returns:
tuple: ``(A, B, C)`` with ``A >= 0`` and ``B`` in (-pi, pi].
"""
theta = np.radians(np.asarray(angles_deg, dtype=float))
design = np.column_stack([np.sin(theta), np.cos(theta), np.ones_like(theta)])
(a, b, c), *_ = np.linalg.lstsq(design, np.asarray(values, dtype=float), rcond=None)
return float(np.hypot(a, b)), float(np.arctan2(b, a)), float(c)
class LamNIAlignmentMixin:
"""Correction infrastructure for LamNI laminography.
@@ -54,8 +76,21 @@ class LamNIAlignmentMixin:
self.corr_angle_2 = []
def reset_xray_eye_correction(self):
"""Delete the X-ray eye sinusoidal fit from the BEC global variable store."""
"""Delete the X-ray eye sinusoidal fit, and the alignment points it was
fitted to, from the BEC global variable store."""
self.client.delete_global_var("tomo_fit_xray_eye")
self.client.delete_global_var(XEYE_FIT_DATA_VAR)
def _existing_alignment(self) -> list[str]:
"""Names of the alignment results currently stored, i.e. what a new
X-ray eye alignment or an applied rotation-center calibration would
erase. Empty if there is nothing to lose."""
existing = []
if self.client.get_global_var("tomo_fit_xray_eye") is not None:
existing.append("X-ray eye alignment fit")
if self.corr_pos_x:
existing.append("fine-alignment correction")
return existing
# ------------------------------------------------------------------
# Additional lookup-table correction properties -- global-var-backed so
@@ -202,8 +237,10 @@ class LamNIAlignmentMixin:
.. important::
This method reads from the live GUI widget. If the XRayEye GUI
window has been closed since the alignment was performed the call
will fail. In that case use :meth:`read_xray_eye_correction` to
reload from the archived text files.
will fail. (The former file-based fallback
``read_xray_eye_correction()`` is disabled -- see the commented-out
code above -- so re-open the X-ray eye GUI and let the fit converge
again, or re-run the alignment.)
"""
_dev = builtins.__dict__.get("dev")
tomo_fit_xray_eye = np.zeros((2, 3))
@@ -223,9 +260,8 @@ class LamNIAlignmentMixin:
f"{type(params).__name__ if not isinstance(params, dict) else 'a dict missing keys'}). "
"This means the fit hasn't converged / DAP forwarding hasn't produced "
"a result yet, not that the correction data is missing. Wait for the "
"fit to converge in the XRayEye GUI's Fit tab, or use "
"read_xray_eye_correction() to reload from the archived text files "
"instead."
"fit to converge in the XRayEye GUI's Fit tab (keep the XRayEye "
"window open), or re-run the alignment."
)
tomo_fit_xray_eye[0][0] = params_x["SineModel_0_amplitude"]
@@ -240,6 +276,125 @@ class LamNIAlignmentMixin:
print("New alignment parameters loaded from XRayEye GUI fit:")
self._print_xeye_fit(tomo_fit_xray_eye)
def xrayeye_refit(self, exclude_angles=None, file: str | None = None, confirm: bool = True):
"""Redo the X-ray eye sinusoidal fit, optionally leaving out angles.
Use this after ``xrayeye_alignment_start()`` when one click is clearly
off the curve (sample barely visible at that angle, misclick, ...).
The clicks of the last alignment are refitted without the excluded
angles, a table of all points with their deviation from the new fit
is printed, and after confirmation the result replaces
``tomo_fit_xray_eye``. Nothing is moved and no new frame is taken.
Call it without arguments first to see the deviations of all points
from the full fit and spot the outlier. Calling it again with an
empty exclusion list restores the fit over all points.
Args:
exclude_angles: angle or list of angles in deg (of 0, 45, ...,
315) whose point is left out of the fit in both x and y.
file: refit an archived alignment instead of the last one, e.g.
~/data/raw/logs/xrayeye_alignmentvalues/xrayeye_alignmentvalues_<timestamp>.h5
confirm: ask before replacing the stored fit.
Example:
lamni.xrayeye_refit() # inspect all points
lamni.xrayeye_refit([135]) # drop the 135 deg click
lamni.xrayeye_refit([135, 270])
"""
data = self._load_xeye_fit_data(file)
angles = np.asarray(data[0], dtype=float)
offsets = {"x": np.asarray(data[1], dtype=float), "y": np.asarray(data[2], dtype=float)}
if exclude_angles is None:
exclude_angles = []
elif np.isscalar(exclude_angles):
exclude_angles = [exclude_angles]
excluded = np.zeros(len(angles), dtype=bool)
for angle in exclude_angles:
match = np.isclose(angles, float(angle) % 360)
if not match.any():
raise ValueError(
f"No alignment point at {angle} deg. "
f"Available angles: {', '.join(f'{a:g}' for a in angles)}."
)
excluded |= match
kept = ~excluded
# 3 free parameters per axis; with only 3 points any set of clicks
# fits exactly, so a bad click could no longer be noticed.
if kept.sum() < 4:
raise ValueError(
f"Only {kept.sum()} of {len(angles)} points left -- at least 4 are "
"needed for a meaningful fit."
)
fit = np.array([fit_xray_eye_sine(angles[kept], offsets[ax][kept]) for ax in ("x", "y")])
residuals = {}
for i, ax in enumerate(("x", "y")):
amp, shift, const = fit[i]
residuals[ax] = offsets[ax] - (amp * np.sin(np.radians(angles) + shift) + const)
print("X-ray eye points (offsets and deviation from the new fit, in um):")
print(" angle x_off x_dev y_off y_dev")
for j, angle in enumerate(angles):
flag = " excluded" if excluded[j] else ""
print(
f" {angle:5.0f} {offsets['x'][j]:9.2f} {residuals['x'][j]:8.2f}"
f" {offsets['y'][j]:9.2f} {residuals['y'][j]:8.2f}{flag}"
)
for ax in ("x", "y"):
rms = np.sqrt(np.mean(residuals[ax][kept] ** 2))
print(f" rms deviation of the fitted points, {ax}: {rms:.2f} um")
print("New fit:")
self._print_xeye_fit(fit)
old_fit = self.client.get_global_var("tomo_fit_xray_eye")
if old_fit is not None:
print("Currently stored fit:")
self._print_xeye_fit(old_fit)
self._show_xeye_refit_in_gui(angles[kept], offsets["x"][kept], offsets["y"][kept])
if confirm and not self.OMNYTools.yesno("Replace the stored X-ray eye fit?", "y"):
print("Stored fit left unchanged.")
return
self.client.set_global_var("tomo_fit_xray_eye", fit.tolist())
if excluded.any():
print(
"New X-ray eye fit stored (excluded: "
+ ", ".join(f"{a:g}" for a in angles[excluded])
+ " deg)."
)
else:
print("New X-ray eye fit stored (all points).")
def _load_xeye_fit_data(self, file: str | None = None):
"""Return the ``[angles, x_offsets, y_offsets]`` of the last alignment,
or of the archived alignment in *file*."""
if file is not None:
with h5py.File(os.path.expanduser(file), "r") as f:
return f["alignment_fit"][()].tolist()
data = self.client.get_global_var(XEYE_FIT_DATA_VAR)
if data is None:
raise RuntimeError(
"No X-ray eye alignment points stored -- run lamni.xrayeye_alignment_start(), "
"or pass the archived alignment as file=... (see "
"~/data/raw/logs/xrayeye_alignmentvalues/)."
)
return data
def _show_xeye_refit_in_gui(self, angles, x_offsets, y_offsets):
"""Show the points used in the refit on the XRayEye Fit tab.
Display only: DAP parameter forwarding stays off, so the GUI's own fit
of these points does not touch the stored correction.
"""
try:
self.lamnigui_show_xeyealign_fittab()
self.xeyegui.submit_fit_array(np.array([angles, x_offsets, y_offsets]))
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Could not show the refit points in the XRayEye GUI: {exc}")
@staticmethod
def _print_xeye_fit(fit):
"""Pretty-print the 2×3 X-ray eye fit array."""
@@ -314,7 +469,6 @@ class LamNIAlignmentMixin:
with open(correction_file, "r") as f:
num_elements = f.readline()
int_num_elements = int(num_elements.split(" ")[2])
print(int_num_elements)
corr_pos_x = []
corr_pos_y = []
corr_angle = []
@@ -214,23 +214,135 @@ class LamNIOpticsMixin:
return None
return param.get(var)
# slits opened by _EYE_SLIT_OPENING (relative) while the xray eye is in
_EYE_SLITS = ("sl1", "sl6")
_EYE_SLIT_OPENING = 1.0 # mm
@staticmethod
def _slit_size(name):
# sl1xs/sl1ys are EpicsMotors (user_readback), sl6xs/sl6ys are
# VirtualSlitWidth pseudo motors (readback)
device = dev[name]
if hasattr(device, "user_readback"):
return device.user_readback.get()
return device.readback.get()
def _slit_before_eye(self, slit):
return (
self.client.get_global_var(f"lamni_{slit}xs_before_eye"),
self.client.get_global_var(f"lamni_{slit}ys_before_eye"),
)
def _set_slit_before_eye(self, slit, xs, ys):
self.client.set_global_var(f"lamni_{slit}xs_before_eye", xs)
self.client.set_global_var(f"lamni_{slit}ys_before_eye", ys)
def _open_slits_for_eye(self):
for slit in self._EYE_SLITS:
xs, ys = f"{slit}xs", f"{slit}ys"
if xs not in dev or ys not in dev:
continue
self._set_slit_before_eye(slit, self._slit_size(xs), self._slit_size(ys))
scans.umv(
dev[xs], self._EYE_SLIT_OPENING, dev[ys], self._EYE_SLIT_OPENING, relative=True
)
def _restore_slits_after_eye(self):
for slit in self._EYE_SLITS:
xs, ys = f"{slit}xs", f"{slit}ys"
xs_before, ys_before = self._slit_before_eye(slit)
if xs_before is None or ys_before is None:
continue
if xs not in dev or ys not in dev:
self._set_slit_before_eye(slit, None, None)
continue
# if the slit was manually re-adjusted while the eye was in,
# offer to keep the current size instead of restoring the
# pre-eye value
tol = 0.01 # mm (10 microns)
manually_adjusted = not np.isclose(
self._slit_size(xs), xs_before + self._EYE_SLIT_OPENING, atol=tol
) or not np.isclose(self._slit_size(ys), ys_before + self._EYE_SLIT_OPENING, atol=tol)
if manually_adjusted and self.OMNYTools.yesno(
f"{xs}/{ys} appear to have been adjusted manually while the "
"eye was in. Keep the current slit size instead of "
"restoring the pre-eye value?"
):
print(f"Keeping current {slit} size, not restoring pre-eye value.")
else:
umv(dev[xs], xs_before, dev[ys], ys_before)
self._set_slit_before_eye(slit, None, None)
def leye_out(self):
self.loptics_in()
dev.fsh.fshopen()
# Close (not open) the shutter as the eye leaves the beam, like
# flomni's feye_out(). Was fshclose() originally; commit 4967474
# ("removed fsh commands") turned it into fshopen() by mistake.
dev.fsh.fshclose()
if self.csaxs is not None:
self.csaxs.fil_trans(1, print_only=False)
self._restore_slits_after_eye()
leyey_out = self._get_user_param_safe("leyey", "out")
umv(dev.leyey, leyey_out)
umv(dev.lsamrot, 0)
#umv(dev.dttrz, 5854, dev.fttrz, 2395)
# flight tube (fttrz) and detector table (dttrz) move together
if "fttrz" not in dev or "dttrz" not in dev:
if not self.OMNYTools.yesno(
"Device 'fttrz' or 'dttrz' does not exist in the current config, "
"so the flight tube and detector table can't be moved into position. "
"Continue anyway? WARNING: this could cause collisions when "
"operating on the real beamline."
):
print("Aborting leye_out(): fttrz/dttrz not moved.")
return
else:
fttrz_in = self._get_user_param_safe("leyex", "fttrz_in")
dttrz_in = self._get_user_param_safe("leyex", "dttrz_in")
umv(dev.fttrz, fttrz_in, dev.dttrz, dttrz_in)
def leye_in(self):
bec.queue.next_dataset_number += 1
umv(dev.lsamrot, 0)
#umv(dev.dttrz, 6419.677, dev.fttrz, 2959.979)
# flight tube (fttrz) and detector table (dttrz) move together
if "fttrz" not in dev or "dttrz" not in dev:
if not self.OMNYTools.yesno(
"Device 'fttrz' or 'dttrz' does not exist in the current config, "
"so the flight tube and detector table can't be moved out of the way. "
"Continue anyway? WARNING: this could cause collisions when "
"operating on the real beamline."
):
print("Aborting leye_in(): fttrz/dttrz not moved.")
return
else:
fttrz_out = self._get_user_param_safe("leyex", "fttrz_out")
dttrz_out = self._get_user_param_safe("leyex", "dttrz_out")
umv(dev.fttrz, fttrz_out, dev.dttrz, dttrz_out)
if not self.OMNYTools.yesno("Did the flight tube move out?", "y"):
return
leyex_in = self._get_user_param_safe("leyex", "in")
leyey_in = self._get_user_param_safe("leyey", "in")
current_leyex = dev.leyex.readback.get()
current_leyey = dev.leyey.readback.get()
# check if both are close enough (within 0.01)
if np.isclose(current_leyex, leyex_in, atol=0.01) and np.isclose(current_leyey, leyey_in, atol=0.01):
# both already in position -> do nothing
pass
else:
self._open_slits_for_eye()
if self.csaxs is not None:
self.csaxs.fil_trans(0.05, print_only=False)
umv(dev.leyex, leyex_in, dev.leyey, leyey_in)
self.xrayeye_update_frame()
@@ -304,9 +416,10 @@ class LamNIOpticsMixin:
)
def loptics_in(self):
"""Move in the LamNI optics (FZP + OSA)."""
"""Move in the LamNI optics (FZP + OSA + central stop)."""
self.lfzp_in()
self.losa_in()
self.lcs_in()
def _check_eye_out_and_optics_in(self, tol=0.003):
"""True if the setup is in measurement configuration: X-ray eye OUT,
@@ -353,10 +466,11 @@ class LamNIOpticsMixin:
return eye_out and optics_in and osa_in
def loptics_out(self):
"""Move out the LamNI optics."""
"""Move out the LamNI optics (central stop + OSA + FZP)."""
if "rtx" in dev and dev.rtx.enabled:
dev.rtx.controller.feedback_disable()
self.lcs_out()
self.losa_out()
loptx_out = self._get_user_param_safe("loptx", "out")
lopty_out = self._get_user_param_safe("lopty", "out")
@@ -370,10 +484,31 @@ class LamNIOpticsMixin:
print(f"Interferometer feedback re-established ({time.time() - _t0:.1f} s).")
def lcs_in(self):
pass
lcsx_in = self._get_user_param_safe("lcsx", "in")
lcsy_in = self._get_user_param_safe("lcsy", "in")
current_lcsx = dev.lcsx.readback.get()
current_lcsy = dev.lcsy.readback.get()
tol = 0.003
need_move_cs = not np.isclose(current_lcsx, lcsx_in, atol=tol) or not np.isclose(
current_lcsy, lcsy_in, atol=tol
)
if need_move_cs:
umv(dev.lcsx, lcsx_in, dev.lcsy, lcsy_in)
else:
print("Central stop is already at the IN position.")
def lcs_out(self):
umv(dev.lcsy, 3)
# only lcsy has an "out" position; lcsx stays where it is
lcsy_out = self._get_user_param_safe("lcsy", "out")
if not np.isclose(dev.lcsy.readback.get(), lcsy_out, atol=0.003):
umv(dev.lcsy, lcsy_out)
else:
print("Central stop is already at the OUT position.")
def losa_in(self):
losax_in = self._get_user_param_safe("losax", "in")
@@ -421,32 +556,55 @@ class LamNIOpticsMixin:
else:
print("OSA is already at the OUT position.")
# the 30 nm FZP sits in an additional holder that puts the chip this
# much further downstream (closer to the sample) than the 60 nm FZP.
# Calibrated 2026-10-14 for a sample on the downstream-facing side of the
# sample holder: 250 um / 30 nm FZP at 7.9 keV (focal distance 47.79 mm),
# sample 1 mm beyond the focus at loptz 80.8548
# -> 137.6 - 80.8548 - 48.79 = 7.955 mm.
_FZP_30NM_HOLDER_OFFSET = 7.955 # mm
def _fzp_sample_distance(self, loptz_val, outermost_zone_width_nm):
"""FZP-to-sample distance in mm for the given loptz position and FZP type."""
distance = -loptz_val + 85.6 + 52
if outermost_zone_width_nm == 30:
distance -= self._FZP_30NM_HOLDER_OFFSET
return distance
def lfzp_info(self, mokev_val=-1):
if mokev_val == -1:
# The energy readback device is "ccm_energy" (same as flomni's
# ffzp_info() and LamNI.write_pdf_report()); the legacy "mokev"
# device is not in the current configs, so this always bailed out.
try:
mokev_val = dev.mokev.readback.get()
mokev_val = dev.ccm_energy.read(cached=True)[dev.ccm_energy.name]["value"]
except Exception:
print(
"Device mokev does not exist. You can specify the energy in keV as an argument instead."
"Could not read the photon energy from device ccm_energy. "
"You can specify the energy in keV as an argument instead."
)
return
loptz_val = dev.loptz.read()["loptz"]["value"]
distance = -loptz_val + 85.6 + 52
print(f"The sample is in a distance of {distance:.1f} mm from the FZP.")
distance = self._fzp_sample_distance(loptz_val, 60)
distance_30nm = self._fzp_sample_distance(loptz_val, 30)
print(
f"The sample is in a distance of {distance:.1f} mm from a 60 nm FZP and "
f"{distance_30nm:.1f} mm from a 30 nm FZP (extra holder, "
f"{self._FZP_30NM_HOLDER_OFFSET:.2f} mm further downstream)."
)
print(f"At the current energy of {mokev_val:.4f} keV we have following options:\n")
wavelength = 1.2398e-9 / mokev_val
diameters = [80e-6, 100e-6, 120e-6, 150e-6, 170e-6, 200e-6, 220e-6, 250e-6]
console = Console()
table = Table(
title=f"At the current energy of {mokev_val:.4f} keV we have following options:",
box=box.SQUARE,
)
table = Table(title="Outermost zone width \033[1m60 nm\033[0m", box=box.SQUARE)
table.add_column("Diameter", justify="center")
table.add_column("Focal distance", justify="center")
table.add_column("Current beam size", justify="center")
wavelength = 1.2398e-9 / mokev_val
for diameter in diameters:
outermost_zonewidth = 60e-9
focal_distance = diameter * outermost_zonewidth / wavelength
@@ -459,9 +617,35 @@ class LamNIOpticsMixin:
f"{beam_size:.2f} microns",
)
console.print(table)
# Outermost zone width 30 nm -- diameters match flomni's 30nm FZP
# (ffzp_info(), flomni_optics_mixin.py) now that the zone plate
# mounted in LamNI is the same one mounted in Flomni.
diameters = [140e-6, 170e-6, 200e-6, 250e-6]
console = Console()
table = Table(title="Outermost zone width \033[1m30 nm\033[0m", box=box.SQUARE)
table.add_column("Diameter", justify="center")
table.add_column("Focal distance", justify="center")
table.add_column("Current beam size", justify="center")
for diameter in diameters:
outermost_zonewidth = 30e-9
focal_distance = diameter * outermost_zonewidth / wavelength
beam_size = (
-diameter / (focal_distance * 1000) * (focal_distance * 1000 - distance_30nm) * 1e6
)
table.add_row(
f"{diameter*1e6:.2f} microns",
f"{focal_distance*1000:.2f} mm",
f"{beam_size:.2f} microns",
)
console.print(table)
print(
"The numbers presented here are for a sample in the plane of the lamni sample holder.\n"
"The numbers presented here are for a sample facing the downstream side "
"of the lamni sample holder.\n"
)
collision_offset = self._get_user_param_optional("loptz", "collision_offset")
@@ -487,7 +671,7 @@ class LamNIOpticsMixin:
print(f" Current losaz {losaz_val:.1f}")
print(
" The OSA will collide with a normal sample holder at losaz "
f"\033[1m{(collision_offset - losaz_val):.1f}\033[0m"
f"\033[1m{(collision_offset - loptz_val):.1f}\033[0m"
)
print(f" Remaining space (right now): \033[1m{remaining_current:.1f}\033[0m")
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,114 @@
# TODO: interferometer signal-strength trend on the webpage
Status: **proposed, not implemented.** Discussed 2026-09-19. This is a design
note to capture the idea before it's lost -- no code changes yet.
## Goal
A new section on the flomni/lamni/omny status webpage showing the trend of
interferometer signal strength over time (a sparkline/chart, not a live
readout), so drift/degradation is visible at a glance across a session or
beamtime, not just from an interactive `show_signal_strength_interferometer()`
call in the moment.
## Sampling design
- **Very low rate on purpose**: roughly one data point per hour, or less --
explicitly *not* per-projection. A tomo scan already calls a signal-strength
check regularly (flomni: every projection, see below), but sampling that
often would produce far too many points for a trend view (tens of points
per hour per axis) for no benefit -- this is about long-term drift, not
per-projection diagnostics.
- **Timestamped, so per-axis cadence can differ freely.** Each data point is
`(timestamp, value)`; there's no requirement that every axis be sampled at
the same time or the same rate. An axis that's cheap to read opportunistically
(e.g. already fetched as a side effect of something else) can be sampled
more often than one that costs a dedicated round-trip, and the trend for
each axis is still valid independently. Staleness/eventual-consistency
between axes, or between this and the live scan, is fine -- the goal is a
general trend, not point-in-time precision.
- **Retention**: apply the lesson from `eps_alarm_history.json`'s HTTP 413
bug (see `docs/developer/...` -- fixed 2026-09-18 in `eps_status_generator.py`,
`_MAX_UPLOAD_BYTES`/`_shrink_to_fit`): cap this ring buffer by *both* count
and a hard byte budget from the start, reusing that same pattern, not just
a count cap discovered insufficient after the fact.
## Per-setup reality check (2026-09-19 survey -- see below for what's actually there)
The three setups' interferometer-read primitives are **not interchangeable**
-- this can't be one generic helper at the RT-controller layer:
| Setup | Read primitive | Axes/channels | Threshold/check helper |
|---|---|---|---|
| flomni | `RtFlomniController.read_ssi_interferometer(axis_number)` (`rt_flomni_ophyd.py:462`), generic `j{n}` socket read | 0-3 | `laser_tracker_check_signalstrength()` (`rt_flomni_ophyd.py:466`) -- reads axis 1, warns only, does not act |
| LamNI | **no equivalent single-axis reader.** `RtLamniController.show_signal_strength_interferometer()` (`rt_lamni_ophyd.py:268`) reads 3 channels (ST FZP horizontal/vertical, angle) via distinct `J2`/`J3`/`J7` commands | 3 (different meaning than flomni's 0-3) | none -- no `min_signal`/`low_signal` threshold logic exists in this file at all |
| omny | `RtOMNYController.laser_tracker_check_and_wait_for_signalstrength()` (`rt_omny_ophyd.py:270`) -- different behavior class: polls and *auto-triggers* `omny_interferometer_align_tracking()` if below `low_signal`, not just a warning. `show_signal_strength_interferometer()` (`rt_omny_ophyd.py:736`) reads 5 channels via a `self.ssi` dict from `feedback_get_status_and_ssi()` | 5 (OSA FZP Y/X, ST OSA Y/X, Angle) | yes, but actively corrective, not passive |
There is no shared `csaxs_bec`-level RT-controller base across the three
(`RtFlomniController`/`RtLamniController`/`RtOMNYController` each subclass the
generic external `ophyd_devices.utils.controller.Controller` directly). LamNI
also doesn't reuse the one shared local module that does exist
(`csaxs_bec/devices/omny/rt/rt_ophyd.py`'s `RtSignalRO`/etc., used by flomni
and omny) -- it redefines its own parallel Signal classes.
**Conclusion**: the shared piece of this feature should be the trend
storage/rendering (a common global-var shape + webpage section), not the
signal read itself. Each setup needs its own small adapter that knows its own
axis semantics and feeds the shared throttled-stash helper.
## Where the throttled read would hook in, per setup
- **flomni**: `laser_tracker_check_signalstrength()` is already called once
per projection during `tomo_scan()` (`flomni.py:3624`) -- cheapest option is
to throttle *at that call site* (check "have I stashed a data point in the
last hour?" before writing one), reusing an already-happening RT round-trip
rather than adding a new one.
- **LamNI**: confirmed (2026-09-19 survey) there is **no** scan-time
signal-strength call anywhere in the LamNI plugin -- `show_signal_strength`
(`lamni.py:724`) is a manual/interactive-only helper, never called from
`tomo_scan_projection` or `tomo_scan`. There's no existing per-projection
hook to throttle against, so this would need a separate low-rate timer
instead -- the natural fit is `WebpageGeneratorBase`'s own `_cycle()` loop
(`OMNY_shared/webpage_generator_base.py`, default 15s cadence, already runs
continuously including when idle), which already has a documented
subclass-override hook for instrument-specific data
(`_collect_setup_data()`, used today by `FlomniWebpageGenerator`/
`LamniWebpageGenerator` for temperatures/sample name/etc.) -- add the
throttle check there instead of in the scan loop.
- **omny**: blocked on a prerequisite -- **omny has no working webpage
generator at all right now.** `omny_webpage_generator.py` looks like one
(defines `WebpageGeneratorBase`, `FlomniWebpageGenerator`, a
`make_webpage_generator()`, etc.) but is confirmed dead/orphaned: nothing
in `omny.py`/`omny/__init__.py` imports it, and the real factory
(`OMNY_shared/webpage_generator_base.py` ~line 1409) explicitly falls
through to the plain base class for the omny session with a comment noting
this file is "an unwired, stale duplicate." Building a real
`OmnyWebpageGenerator` subclass is a prerequisite for this feature on omny,
not just an extra per-setup adapter -- do not revive the stale duplicate
as-is.
## Suggested shape (for whenever this is implemented)
- A global var, e.g. `interferometer_signal_trend`, holding a small
ring-buffer list of `{timestamp, axis, value}` (or per-axis sub-lists),
written via a proxy class mirroring `_ProgressProxy`/`_TomoQueueProxy`
(`OMNY_shared/tomo_queue_mixin.py`) -- the established pattern in this
codebase for cross-process/cross-thread state via
`client.get_global_var`/`set_global_var`.
- A shared rendering piece (webpage HTML/JS section, sparkline or simple line
chart) that reads this same global var shape regardless of which setup
wrote it, so the display code doesn't need to know about per-setup axis
differences -- only each setup's adapter does.
- Byte-budget-capped storage per the eps history lesson above.
## Open questions (not yet decided)
- Exact axis(es) worth trending per setup -- flomni's axis 1 is free (already
read every projection); trending flomni's other 3 axes, or any of LamNI's/
omny's channels, means new dedicated round-trips on whatever cadence is
chosen for those.
- Whether sampling should continue during idle time between scans (needs a
timer-based approach for all three setups, not just LamNI/omny) or only
while a scan is actively running (cheaper, but leaves gaps covering the
most interesting idle-drift periods).
- Exact webpage layout/placement of the new section -- not designed yet.
@@ -0,0 +1,323 @@
"""
Parser for user-supplied tomography angle files (flomni tomo_type 4).
Pure Python, no BEC imports -- shared by the flomni CLI (tomo_scan(),
tomo_parameters(), the tomo queue) and the TomoParams widget's live preview,
so both validate a file identically.
File format: plain text, one projection per line.
- column 1: angle in degrees, -2 <= angle <= 362 -- matches the physical
overtravel range of the rotation stage, not just the nominal 0-360
sweep
- column 2 (optional): sub-tomogram number, integer >= 1 (default 1)
- further, optional 'key=value' tokens overriding acquisition parameters
for this projection and (sticky) every one after it -- see below
Columns are separated by whitespace and/or commas. Everything after a '#' is a
comment; blank lines are ignored. Projections are acquired in file order.
A line that, after comment-stripping and trimming, is 'END' (case-insensitive)
marks the file as complete: no more angles will ever be appended, and
tomo_scan() may stop once it reaches that point instead of waiting for more.
A file without this marker is read as *incomplete* -- tomo_scan() waits and
polls for more lines rather than stopping at the current end of file, so an
angle file can be grown by appending lines while a scan is running. A
commented-out '# END' is an ordinary comment, not the sentinel, which lets a
marker be staged ahead of time and uncommented when the file is truly done.
Any further non-blank/non-comment content after the sentinel line is an error.
Per-line acquisition overrides: after the angle and optional subtomo number,
any number of 'key=value' tokens may override acquisition parameters that
would otherwise come from the live tomo_parameters() configuration:
- step=<um> Fermat scan step size, 0.025-20.0
- exptime=<s> exposure/counting time per frame, 0.001-100.0
- frames=<n> burst frame count, 1-100 (integer)
Each of these is *sticky*: once set on a line, it applies to that line and
every line after it, until changed again by a later 'key=value' of the same
key -- not just the one line it appears on. A field that is never set on any
line uses whatever the live tomo_shellstep/tomo_countingtime/frames_per_trigger
configuration is at the moment that projection is actually acquired. The
sub-tomogram number is NOT sticky (unchanged from today: per-line, default 1).
Example::
# angle subtomo
0 1
45 1 # still the default step/exptime/frames
90 1 step=0.5 # step=0.5 from here on...
135 1 # ...still in effect here
180 2 step=2 exptime=5 frames=3
225 2 # step/exptime/frames all still as set above
270, 2
END
"""
from __future__ import annotations
import os
import re
from dataclasses import dataclass
# Matches the physical overtravel range of the fsamroy rotation stage, not
# just the nominal 0-360 sweep (see Flomni._min_scan_angle()/_max_scan_angle()
# in flomni.py, which gate tomo_scan()'s actual acquisition to the same range).
ANGLE_MIN = -2.0
ANGLE_MAX = 362.0
# Two angles closer than this (mod 180) are reported as a 180-degree-opposite
# pair: in the parallel-beam approximation they carry the same information.
OPPOSITE_TOLERANCE_DEG = 0.05
# Per-line override ranges -- match the bounds tomo_parameters_cli() already
# enforces interactively on tomo_shellstep/tomo_countingtime/frames_per_trigger
# (flomni.py), so a value rejected here would also be rejected there.
_STEP_RANGE = (0.025, 20.0)
_EXPTIME_RANGE = (0.001, 100.0)
_FRAMES_RANGE = (1, 100)
_OVERRIDE_KEYS = ("exptime", "frames", "step")
_SPLIT_RE = re.compile(r"[\s,]+")
_END_SENTINEL = "END"
class AngleFileError(ValueError):
"""Angle file missing, unreadable, or malformed."""
@dataclass(frozen=True)
class AngleEntry:
"""One parsed projection: an angle/sub-tomogram pair, plus whichever
acquisition parameters are overridden (sticky) at this point in the file.
step/exp_time/frames_per_trigger are None when not overridden by this or
any earlier line -- callers should fall back to the live
tomo_shellstep/tomo_countingtime/frames_per_trigger configuration in that
case. Field names match the kwargs already used for these values
elsewhere in flomni.py (the file's own 'exptime' token is the only name
that differs, spelled out as exp_time here).
"""
angle: float
subtomo: int
step: float | None = None
exp_time: float | None = None
frames_per_trigger: int | None = None
@dataclass(frozen=True)
class AngleFileResult:
"""Result of parsing a tomo_type-4 angle file.
is_complete is True iff the file ends with an 'END' sentinel line, i.e.
no more angles will ever be appended to it.
"""
angles: list[AngleEntry]
is_complete: bool
def read_angle_file(path: str) -> AngleFileResult:
"""Parse an angle file into an AngleFileResult, angles in file order.
Raises:
AngleFileError: file missing/unreadable, a malformed line, an angle
outside [0, 360], a sub-tomogram number < 1, no angles at all, or
content found after the 'END' sentinel line.
"""
if not path:
raise AngleFileError("No angle file set.")
full_path = os.path.abspath(os.path.expanduser(path))
try:
with open(full_path, "r", encoding="utf-8") as f:
lines = f.readlines()
except OSError as exc:
raise AngleFileError(f"Cannot read angle file {full_path}: {exc}") from exc
entries: list[AngleEntry] = []
is_complete = False
end_lineno = None
# Sticky override state, carried forward top-to-bottom across the whole
# file -- None means "no override in effect, use the live global".
sticky_step: float | None = None
sticky_exp_time: float | None = None
sticky_frames: int | None = None
for lineno, raw in enumerate(lines, start=1):
line = raw.split("#", 1)[0].strip()
if not line:
continue
if is_complete:
raise AngleFileError(
f"{full_path}, line {lineno}: content found after the 'END' "
f"marker (line {end_lineno}) -- remove it or move it above 'END'."
)
if line.upper() == _END_SENTINEL:
is_complete = True
end_lineno = lineno
continue
fields = [f for f in _SPLIT_RE.split(line) if f]
try:
angle = float(fields[0])
except ValueError as exc:
raise AngleFileError(
f"{full_path}, line {lineno}: invalid angle {fields[0]!r}"
) from exc
if not ANGLE_MIN <= angle <= ANGLE_MAX:
raise AngleFileError(
f"{full_path}, line {lineno}: angle {angle} outside "
f"[{ANGLE_MIN:g}, {ANGLE_MAX:g}] degrees"
)
rest = fields[1:]
subtomo = 1
if rest and "=" not in rest[0]:
# positional subtomo -- must come immediately after the angle,
# before any key=value tokens
token = rest[0]
try:
subtomo = int(token)
except ValueError as exc:
raise AngleFileError(
f"{full_path}, line {lineno}: invalid sub-tomogram number {token!r}"
) from exc
if subtomo < 1:
raise AngleFileError(
f"{full_path}, line {lineno}: sub-tomogram number must be >= 1, got {subtomo}"
)
rest = rest[1:]
seen_keys: set[str] = set()
for token in rest:
if "=" not in token:
raise AngleFileError(
f"{full_path}, line {lineno}: expected 'key=value', got {token!r}"
)
key, _, value_str = token.partition("=")
key, value_str = key.strip(), value_str.strip()
if not key or not value_str:
raise AngleFileError(
f"{full_path}, line {lineno}: malformed 'key=value' token {token!r}"
)
if key not in _OVERRIDE_KEYS:
raise AngleFileError(
f"{full_path}, line {lineno}: unknown key {key!r} "
f"(expected one of {', '.join(_OVERRIDE_KEYS)})"
)
if key in seen_keys:
raise AngleFileError(
f"{full_path}, line {lineno}: duplicate key {key!r} on the same line"
)
seen_keys.add(key)
if key == "frames":
try:
frames_val = int(value_str)
except ValueError as exc:
raise AngleFileError(
f"{full_path}, line {lineno}: invalid frames value {value_str!r}"
) from exc
lo, hi = _FRAMES_RANGE
if not lo <= frames_val <= hi:
raise AngleFileError(
f"{full_path}, line {lineno}: frames={frames_val} outside [{lo}, {hi}]"
)
sticky_frames = frames_val
else:
try:
num_val = float(value_str)
except ValueError as exc:
raise AngleFileError(
f"{full_path}, line {lineno}: invalid {key} value {value_str!r}"
) from exc
lo, hi = _STEP_RANGE if key == "step" else _EXPTIME_RANGE
if not lo <= num_val <= hi:
raise AngleFileError(
f"{full_path}, line {lineno}: {key}={num_val:g} outside [{lo:g}, {hi:g}]"
)
if key == "step":
sticky_step = num_val
else:
sticky_exp_time = num_val
entries.append(
AngleEntry(
angle=angle,
subtomo=subtomo,
step=sticky_step,
exp_time=sticky_exp_time,
frames_per_trigger=sticky_frames,
)
)
if not entries:
raise AngleFileError(f"Angle file {full_path} contains no angles.")
return AngleFileResult(angles=entries, is_complete=is_complete)
def summarize_angles(entries: list[AngleEntry]) -> dict:
"""Summary of a parsed angle list for display/warnings.
Returns a dict with n, min, max, n_subtomos, n_overridden, and
opposite_pairs -- a list of (index_a, index_b, angle_a, angle_b) for
projections that are (nearly) 180 degrees apart or identical mod 180,
i.e. redundant in the parallel-beam approximation. n_overridden is the
number of entries carrying a non-default step/exp_time/frames_per_trigger.
"""
angles = [e.angle for e in entries]
subtomos = {e.subtomo for e in entries}
n_overridden = sum(
1
for e in entries
if e.step is not None or e.exp_time is not None or e.frames_per_trigger is not None
)
# Sort by angle mod 180 so near-duplicates end up adjacent (plus the
# wrap-around pair between last and first): O(n log n) instead of O(n^2).
folded = sorted((a % 180.0, i) for i, a in enumerate(angles))
opposite_pairs = []
n = len(folded)
for k in range(n):
f_a, i_a = folded[k]
for m in range(k + 1, n + k):
f_b, i_b = folded[m % n]
gap = f_b - f_a if m < n else f_b + 180.0 - f_a
if gap >= OPPOSITE_TOLERANCE_DEG:
break
pair = (min(i_a, i_b), max(i_a, i_b))
opposite_pairs.append((pair[0], pair[1], angles[pair[0]], angles[pair[1]]))
opposite_pairs = sorted(set(opposite_pairs))
return {
"n": len(angles),
"min": min(angles),
"max": max(angles),
"n_subtomos": len(subtomos),
"n_overridden": n_overridden,
"opposite_pairs": opposite_pairs,
}
def describe_summary(summary: dict) -> str:
"""One-line human-readable summary, e.g. '120 angles, 0.0-359.0 deg, 4 sub-tomograms'.
Appends ', N with per-line overrides' only when N > 0, so a file with no
step/exptime/frames overrides prints exactly as before this feature.
"""
text = (
f"{summary['n']} angles, {summary['min']:.2f}-{summary['max']:.2f} deg, "
f"{summary['n_subtomos']} sub-tomogram{'s' if summary['n_subtomos'] != 1 else ''}"
)
if summary.get("n_overridden"):
text += f", {summary['n_overridden']} with per-line overrides"
return text
def describe_opposite_pairs(summary: dict, max_listed: int = 5) -> str | None:
"""Warning text for 180-degree-opposite/duplicate pairs, or None if there are none."""
pairs = summary["opposite_pairs"]
if not pairs:
return None
listed = ", ".join(f"{a:g}/{b:g}" for _, _, a, b in pairs[:max_listed])
more = f" (+{len(pairs) - max_listed} more)" if len(pairs) > max_listed else ""
return (
f"{len(pairs)} pair(s) of angles are identical or 180 deg apart "
f"(redundant in parallel-beam geometry): {listed}{more}"
)
@@ -15,12 +15,12 @@ Two browser pages backed by one JSON file:
1. **Machine status** — ring current, lifetime, light delivery, undulator gap,
beamline status, orbit feedback, injection rate, ring vacuum.
2. **Operator messages** — the SLS control-room message board.
3. **Alarm status** — the EPS alarm banner (green when `AlarmCnt_EPS == 0`,
red with the alarm text otherwise), plus front-end alarms.
4. **Mono cryocooler** — run / alarm / inhibit / interlock / heartbeat.
5. **Beamline synoptic** — a link to the full synoptic page.
6. **Zone detail** — FE / OP / ES / EH1 cards, each a severity-colored summary
3. **Mono cryocooler** — run / alarm / inhibit / interlock / heartbeat.
4. **Beamline synoptic** — a link to the full synoptic page.
5. **Zone detail** — FE / OP / ES / EH1 cards, each a severity-colored summary
with an expandable table of every channel.
6. **Alarm status** — the EPS alarm banner (green when `AlarmCnt_EPS == 0`,
red with the alarm text otherwise), plus front-end alarms.
7. **Recent history** — the last 5 EPS + machine events.
**`eps_synoptic.html`** — a faithful replica of the caQtDM EPS expert panel
@@ -294,7 +294,7 @@ IP-whitelisted upload all apply unchanged.
- **`--verify` on a running beamline** (this was done during shutdown). Re-run
when the machine is up to confirm the FE photon-shutter PVs
(`X12SA-EPS-PH:SYSTEM`, `X12SA-FE-PH1:OPEN_EPS/CLOSE_EPS`) come back.
(`X12SA-EPS-PH:SYSTEM`) comes back (the FE-PH1 OPEN/CLOSE_EPS PVs were dropped; FE shutter state now comes from `X12SA-FE-PSH1-EMLS:STATE`).
- **Cryocooler detail.** The card shows run/alarm/inhibit/heartbeat. Full
temperature/pressure detail needs `XDS_CRYOCOOLER_OVERVIEW.ui` (a sub-template
not yet supplied).
@@ -28,8 +28,6 @@ HOT_PVS = [
("X12SA-EH1-PSYS:SH-A-OK", "sh_eh1_a_ok", "EH1 shutter A OK"),
("X12SA-OP-PSYS:ALARM-COLLECTIVE", "psys_op_alarm", "OP PSYS collective alarm"),
("X12SA-EH1-PSYS:ALARM-COLLECTIVE", "psys_eh1_alarm", "EH1 PSYS collective alarm"),
("X12SA-FE-PH1:OPEN_EPS", "fe_ph1_open", "FE photon shutter open"),
("X12SA-FE-PH1:CLOSE_EPS", "fe_ph1_closed", "FE photon shutter closed"),
("AGEBD-PARAMS:CURRENT", "ring_current", "SLS ring current"),
("AGEBD-PARAMS:LIFETIME", "lifetime", "Beam lifetime"),
("AGEOP-STATUS:LIGHT-DELIVERY", "light_delivery", "Light delivery state"),
@@ -362,6 +360,9 @@ def all_bulk_pvs():
if pv not in seen: seen.add(pv); out.append(pv)
for pv, _ in MACHINE_PVS:
if pv not in seen: seen.add(pv); out.append(pv)
for pvs in OPMSG_PVS:
for pv in pvs:
if pv not in seen: seen.add(pv); out.append(pv)
return out
def all_pvs():
@@ -107,9 +107,6 @@
<h2>Operator Messages</h2>
<div class="panel"><div id="messages"></div></div>
<h2>Alarm Status</h2>
<div id="hero" class="hero unknown"><div class="hero-title">&mdash;</div></div>
<h2>Mono Cryocooler</h2>
<div class="panel"><div class="cryo" id="cryo"></div></div>
@@ -127,6 +124,9 @@
<h2>Zone Detail</h2>
<div class="zones" id="zones"></div>
<h2>Alarm Status</h2>
<div id="hero" class="hero unknown"><div class="hero-title">&mdash;</div></div>
<h2>Recent History</h2>
<div class="panel" id="history"></div>
@@ -92,6 +92,21 @@ _HISTORY_MAX = 50 # events persisted per stream
_HISTORY_SHOW = 5 # events surfaced to the page per stream
_TRACE_MAX = 288 # ring-current samples (24 h at 5 min)
_FLAP_COALESCE_S = 60 # re-alarm within this window extends the event
_ALARM_TEXTS_MAX = 100 # distinct alarm texts kept on a single open eps_event;
# only one event is ever open at a time, so this bounds
# total history growth during a long-running flapping alarm
_MAX_UPLOAD_BYTES = 250_000 # hard ceiling on eps_alarm_history.json's serialized size,
# enforced in HistoryTracker.save(). _HISTORY_MAX/
# _ALARM_TEXTS_MAX/_TRACE_MAX bound the *shape* of the data
# but not each alarm text's length, and _load() never re-caps
# a stored event's texts list -- only new appends are capped
# -- so a sufficiently verbose/flapping EPS alarm can still
# push the file past the upload server's request-size limit
# (a bare HTTP 413, no size check in upload.php itself; almost
# certainly nginx's client_max_body_size, default 1m). This
# keeps comfortable headroom under that regardless of how the
# data grows.
_SEVERITY_NAMES = {0: "NO_ALARM", 1: "MINOR", 2: "MAJOR", 3: "INVALID"}
@@ -162,6 +177,56 @@ def _clean(value):
return value
def _payload_size(payload: dict) -> int:
return len(json.dumps(payload, default=str))
def _shrink_to_fit(payload: dict, max_bytes: int = _MAX_UPLOAD_BYTES) -> dict:
"""Return a shrunk copy of a HistoryTracker.save() payload that
serializes to at most max_bytes.
Trims progressively less-essential data -- alarm texts per event, then
the ring-current trace, then the oldest events, then (last resort, for a
single pathologically long string) individual field lengths -- stopping
as soon as the payload fits.
"""
payload = {
"eps_events": [dict(ev) for ev in payload["eps_events"]],
"machine_events": list(payload["machine_events"]),
"ring_current_trace": list(payload["ring_current_trace"]),
}
if _payload_size(payload) <= max_bytes:
return payload
for texts_cap in (20, 5, 1):
for ev in payload["eps_events"]:
texts = ev.get("texts")
if isinstance(texts, list) and len(texts) > texts_cap:
ev["texts"] = texts[-texts_cap:]
if _payload_size(payload) <= max_bytes:
return payload
while len(payload["ring_current_trace"]) > 12 and _payload_size(payload) > max_bytes:
payload["ring_current_trace"] = payload["ring_current_trace"][::2]
while len(payload["eps_events"]) > 1 and _payload_size(payload) > max_bytes:
payload["eps_events"].pop(0)
while len(payload["machine_events"]) > 1 and _payload_size(payload) > max_bytes:
payload["machine_events"].pop(0)
if _payload_size(payload) > max_bytes:
for ev in payload["eps_events"]:
if isinstance(ev.get("texts"), list):
ev["texts"] = [t[:200] for t in ev["texts"]]
for mev in payload["machine_events"]:
for key in ("from", "to"):
if isinstance(mev.get(key), str):
mev[key] = mev[key][:200]
return payload
# ---------------------------------------------------------------------------
# History tracking
# ---------------------------------------------------------------------------
@@ -207,9 +272,12 @@ class HistoryTracker:
"machine_events": self.machine_events[-_HISTORY_MAX:],
"ring_current_trace": self.ring_trace[-_TRACE_MAX:],
}
# Shrink a copy for writing/uploading; the in-memory lists above keep
# their full (count-capped) history regardless of upload-size limits.
payload = _shrink_to_fit(payload)
try:
tmp = self._path.with_suffix(".tmp")
tmp.write_text(json.dumps(payload, indent=1, default=str))
tmp.write_text(json.dumps(payload, default=str))
tmp.replace(self._path)
except Exception as exc:
logger.warning(f"HistoryTracker: cannot write {self._path}: {exc}")
@@ -235,7 +303,9 @@ class HistoryTracker:
if open_ev is not None:
open_ev["count"] = max(open_ev.get("count", 0), count)
if text and text not in open_ev.get("texts", []):
open_ev.setdefault("texts", []).append(text)
texts = open_ev.setdefault("texts", [])
texts.append(text)
del texts[:-_ALARM_TEXTS_MAX]
return
# coalesce a rapid re-alarm with identical text
if self.eps_events:
@@ -805,9 +875,15 @@ class EpsStatusGenerator:
"b_closed": hv("X12SA-OP-PSYS:SH-B-CLOSE"),
"b_ok": hv("X12SA-OP-PSYS:SH-B-OK"),
})
# The old X12SA-FE-PH1:OPEN_EPS/CLOSE_EPS PVs no longer exist; derive
# open/closed from the front-end shutter end-switch state (enum
# MOVING/OPEN/CLOSED, may arrive as index or string).
fe_state = (self._bulk_cache.get("X12SA-FE-PSH1-EMLS:STATE") or {}).get("value")
fe_state = {1: "OPEN", 2: "CLOSED"}.get(fe_state, fe_state)
fe_known = isinstance(fe_state, str) and fe_state.upper() in ("OPEN", "CLOSED", "MOVING")
shutters["FE"] = {
"a_open": hv("X12SA-FE-PH1:OPEN_EPS"),
"a_closed": hv("X12SA-FE-PH1:CLOSE_EPS"),
"a_open": (fe_state.upper() == "OPEN") if fe_known else None,
"a_closed": (fe_state.upper() == "CLOSED") if fe_known else None,
"a_ok": None,
"collective_alarm": None,
}
@@ -0,0 +1,58 @@
import time
from csaxs_bec.bec_ipython_client.plugins.cSAXS.filter_transmission import cSAXSFilterTransmission
def filters_out_of_beam(dev, tol: float = 0.1) -> tuple[bool, list[str]]:
"""Check whether all cSAXS exposure-box filters are retracted.
Reuses cSAXSFilterTransmission's own axis list/position table (index 0 of
each row is the "out" position) so there is one source of truth for what
"out" means, matching csaxs.fil_trans()/_fil_trans_report()'s own check.
Tolerates a deployment missing one or more filter axes (e.g. a simulated
session without the real beamline's filter_array_* devices), and `dev`
itself being None (the usual state of the module-level `dev =
builtins.__dict__.get("dev")` idiom used throughout this codebase before
a real BEC session has set it). BEC's device container raises
DeviceConfigError -- not AttributeError -- for a missing device, so a
plain `getattr(dev, axis_name, None)` does NOT safely skip it; wrapping
the lookup itself in try/except does, for both cases.
"""
offending = []
for axis_name, positions in zip(
cSAXSFilterTransmission._AXES, cSAXSFilterTransmission._POSITIONS_USER
):
try:
axis_obj = getattr(dev, axis_name, None)
except Exception:
axis_obj = None
out_position = positions[0]
if axis_obj is None or out_position is None:
continue
try:
rb = float(axis_obj.readback.get())
except Exception:
continue
if abs(rb - out_position) > tol:
offending.append(axis_name)
return not offending, offending
def warn_and_confirm(setup, warning: str, interactive: bool = True, force: bool = False) -> bool:
"""Eye/optics-check-style warn-and-confirm gate, reusable across plugins.
Returns True if the caller should proceed. Interactive: red warning + a
"Continue anyway?" prompt defaulting to no. Non-interactive (queued/
unattended): warns, waits 10s, and always proceeds -- matches
LamNI.tomo_scan()'s existing eye/optics gate so queued jobs never hang on
input().
"""
if force:
return True
setup.OMNYTools.printredbold(f"WARNING: {warning}")
if interactive:
return setup.OMNYTools.yesno("Continue anyway?", "n")
setup.OMNYTools.printredbold("Proceeding automatically in 10 s (unattended/queued run)...")
time.sleep(10)
return True
@@ -35,6 +35,26 @@ def umvr(*args):
return scans.umv(*args, relative=True)
def read_escape_suffix(stream, timeout: float = 0.05) -> str | None:
"""The two characters after an ESC on a non-blocking stdin (arrow keys: "[A".."[D"),
or None if they do not arrive within `timeout` (ESC alone, or a slow terminal).
A plain stream.read(2) can return fewer characters, or None, before the rest of the
sequence has arrived; unpacking that raised and ended the tweak with a traceback."""
chars = ""
deadline = time.monotonic() + timeout
while len(chars) < 2 and time.monotonic() < deadline:
try:
got = stream.read(2 - len(chars))
except (IOError, TypeError):
got = None
if got:
chars += got
else:
time.sleep(0.005)
return chars if len(chars) == 2 else None
class OMNYTools:
HEADER = "\033[95m"
@@ -145,9 +165,10 @@ class OMNYTools:
self.printgreen(message + " Automatically confirming default: no")
return False
suffix = {"y": " [Y]/n?", "n": " y/[N]?"}.get(default, " y/n?")
# No "[y]/n?" suffix here (unlike yesno()'s terminal prompt) -- the
# widget shows actual Yes/No buttons, so the text hint is redundant.
gui.clear_response()
gui.message = message + suffix
gui.message = message
while True:
response = gui.response()
if response == "yes":
@@ -164,10 +185,10 @@ class OMNYTools:
self, dev1, step1: float, dev2="none", step2: float = "0", special_command="none"
):
if dev1 not in dev.enabled_devices:
print(f"Device 1 {dev} is not in enabled devices.")
print(f"Device 1 {dev1} is not in enabled devices.")
return
if dev2 not in dev.enabled_devices and dev2 != "none":
print(f"Device 2 {dev} is not in enabled devices.")
print(f"Device 2 {dev2} is not in enabled devices.")
return
# Save the current terminal settings
fd = sys.stdin.fileno()
@@ -187,7 +208,8 @@ class OMNYTools:
print("\n\rExiting tweak mode\r")
break
elif key == "\x1b": # Escape sequences for arrow keys
next1, next2 = sys.stdin.read(2)
suffix = read_escape_suffix(sys.stdin)
next1, next2 = suffix if suffix else ("", "")
if next1 == "[":
if next2 == "A":
if dev2 != "none":
@@ -327,12 +349,13 @@ class TomoIDManager:
"""Registers a tomography measurement in the OMNY sample database
and returns its assigned tomo ID.
Non-production accounts (e.g. test accounts like "gac-x01dc") register
against the test server (TEST_OMNY_URL) instead of production, so
testing still gets a real, incrementing tomo ID -- matching the
counter the samples-folder PDF upload reads from on that same test
host -- accepting that the eaccount recorded in that test database
won't be a real e-account.
Real e-accounts (personal, e.g. "e12345") and p-accounts (proposal
accounts, e.g. "p23092") are both registered against OMNY_URL.
Non-production accounts (e.g. test accounts like "gac-x01dc") are not
registered against OMNY_URL at all -- there is no longer a separate
test server to redirect them to, and registering test runs against
the production sample database would pollute it. They get
FALLBACK_TOMO_ID instead.
Usage:
id_manager = TomoIDManager()
@@ -348,13 +371,12 @@ class TomoIDManager:
"""
OMNY_URL = "https://omny.psi.ch/samples/newmeasurement.php"
TEST_OMNY_URL = "https://omny.psi.ch/samples/newmeasurement.php"
FALLBACK_TOMO_ID = 0
@staticmethod
def _is_valid_eaccount(eaccount: str) -> bool:
"""True for real e-accounts (e.g. "e12345"), False for test accounts."""
return bool(re.fullmatch(r"e\d{5}", eaccount.strip()))
"""True for real e- or p-accounts (e.g. "e12345", "p23092"), False for test accounts."""
return bool(re.fullmatch(r"[ep]\d{5}", eaccount.strip()))
def register(
self,
@@ -368,20 +390,21 @@ class TomoIDManager:
) -> int:
"""Register a new measurement and return the assigned tomo ID.
Registers against OMNY_URL (production) for a real e-account, or
TEST_OMNY_URL (test server) otherwise. Returns FALLBACK_TOMO_ID (0)
only if the server actually can't be reached / returns an unusable
response.
Registers against OMNY_URL for a real e- or p-account. For any other
account (e.g. a test/"gac-*" account), skips registration entirely
and returns FALLBACK_TOMO_ID (0) -- there is no separate test
server to register against, and registering test runs against
production would pollute the real sample database. FALLBACK_TOMO_ID
is also returned if the server actually can't be reached / returns
an unusable response.
"""
if self._is_valid_eaccount(eaccount):
omny_url = self.OMNY_URL
else:
omny_url = self.TEST_OMNY_URL
if not self._is_valid_eaccount(eaccount):
logger.warning(
f"Account '{eaccount}' is not a valid e-account; registering "
f"against the test server ({self.TEST_OMNY_URL}) instead of "
"production -- the eaccount recorded there won't be real."
f"Account '{eaccount}' is not a valid e- or p-account; skipping OMNY "
f"registration (no separate test server to register against) "
f"and falling back to tomo ID {self.FALLBACK_TOMO_ID}."
)
return self.FALLBACK_TOMO_ID
params = {
"sample": sample_name,
@@ -407,7 +430,7 @@ class TomoIDManager:
try:
response = requests.get(
omny_url,
self.OMNY_URL,
params=params,
timeout=30,
verify=False, # accept self-signed certs
@@ -0,0 +1,321 @@
"""
Shared tomo_type 4 ("angles from file") support for the flomni and LamNI
tomography classes.
The file format and parser live in angle_file.py. This mixin holds everything
that is identical between the setups: the tomo_angle_file global-var property,
parsing/summary helpers, the per-line override plumbing and the acquisition
loop that follows (and waits for) a growing angle file.
Host-class contract:
* _TOMO_PARAMS_ERROR exception class raised on any problem
* _ANGLE_FILE_OPPOSITE_WARNING bool, report 180-deg-opposite/duplicate angles
(parallel-beam redundancy -- off for laminography)
* _tomo_scan_at_angle(angle, subtomo_number, **overrides)
* progress, OMNYTools, client, at_each_angle_hook
"""
from __future__ import annotations
import builtins
import datetime
import os
import time
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.angle_file import (
AngleEntry,
AngleFileError,
AngleFileResult,
describe_opposite_pairs,
describe_summary,
read_angle_file,
summarize_angles,
)
class TomoAngleFileMixin:
# tomo_type 4: how often to re-read tomo_angle_file while waiting for
# more lines to be appended, once the scan has caught up to its end.
_TOMO_ANGLE_FILE_POLL_INTERVAL_S = 5.0
_ANGLE_FILE_OPPOSITE_WARNING = True
# Name of the setup's legacy at-each-angle builtin, or None (ignored-overrides warning).
_AT_EACH_ANGLE_BUILTIN = "flomni_at_each_angle"
# Angle range accepted by tomo_type 4 (mechanical overtravel); the
# parser enforces the same limits.
_ANGLE_FILE_MIN = -2
_ANGLE_FILE_MAX = 362
@property
def tomo_angle_file(self):
"""Path of the angle file used by tomo_type 4 (format: see
OMNY_shared/angle_file.py). Only the path is stored -- the file is
read at tomo_scan() start, including resume and queued jobs, so an
edited file always takes effect on the next (re)start. While a scan
is running, it is also polled and re-read whenever the scan catches
up to the current end of file, so new lines appended during a scan
are picked up automatically -- unless the file ends with an 'END'
sentinel line, the scan waits indefinitely for more lines once it
reaches the end rather than stopping."""
val = self.client.get_global_var("tomo_angle_file")
if val is None:
return ""
return val
@tomo_angle_file.setter
def tomo_angle_file(self, val: str):
# Deliberately no existence/format check here: tomo_queue_execute()
# setattr()s every snapshotted param for every job, so validating here
# would make e.g. a type-1 job fail because of a stale, irrelevant
# angle file path. Validation happens where type 4 actually uses it
# (tomo_scan(), tomo_parameters(), tomo_queue_reacquire()).
val = str(val or "").strip()
if val:
val = os.path.abspath(os.path.expanduser(val))
self.client.set_global_var("tomo_angle_file", val)
def _read_tomo_angle_file(self) -> AngleFileResult:
"""Parse self.tomo_angle_file, raising _TOMO_PARAMS_ERROR on any problem."""
try:
return read_angle_file(self.tomo_angle_file)
except AngleFileError as exc:
raise self._TOMO_PARAMS_ERROR(str(exc)) from exc
@staticmethod
def _tomo_angle_subtomo_layout(entries: list[AngleEntry]):
"""subtomo_sizes (label -> count) and index_in_subtomo (per-entry
position within its own sub-tomogram), for the progress display."""
subtomo_sizes: dict[int, int] = {}
index_in_subtomo: list[int] = []
for entry in entries:
index_in_subtomo.append(subtomo_sizes.get(entry.subtomo, 0))
subtomo_sizes[entry.subtomo] = subtomo_sizes.get(entry.subtomo, 0) + 1
return subtomo_sizes, index_in_subtomo
@staticmethod
def _angle_overrides_kwargs(step=None, exp_time=None, frames_per_trigger=None) -> dict:
"""Only the non-None overrides, as kwargs to splat into
_at_each_angle()/_tomo_scan_at_angle() -- when nothing is overridden
this is {}, so the call is byte-identical to the pre-override-feature
call (no extra kwargs reach fixed-arity stubs/hooks)."""
return {
k: v
for k, v in (
("step", step),
("exp_time", exp_time),
("frames_per_trigger", frames_per_trigger),
)
if v is not None
}
@staticmethod
def _format_angle_entry(entry: AngleEntry) -> str:
"""'angle subtomo [step=.. exptime=.. frames=..]' for scilog/logging."""
text = f"{entry.angle:g} {entry.subtomo}"
if entry.step is not None:
text += f" step={entry.step:g}"
if entry.exp_time is not None:
text += f" exptime={entry.exp_time:g}"
if entry.frames_per_trigger is not None:
text += f" frames={entry.frames_per_trigger}"
return text
def _warn_if_hook_ignores_overrides(self, entries: list[AngleEntry]) -> None:
"""Print a one-time warning if a custom at_each_angle_hook/
<setup>_at_each_angle is active while entries carry step/exp_time/
frames_per_trigger overrides -- those hooks own the whole
acquisition (fixed 2-arg contract) and never see the overrides."""
hook_active = bool(self.at_each_angle_hook) or bool(
self._AT_EACH_ANGLE_BUILTIN and self._AT_EACH_ANGLE_BUILTIN in builtins.__dict__
)
if hook_active and any(
e.step is not None or e.exp_time is not None or e.frames_per_trigger is not None
for e in entries
):
self.OMNYTools.printredbold(
"Warning: an at_each_angle hook is active -- per-line step/exptime/frames "
"overrides in the angle file are ignored (the hook owns the whole acquisition)."
)
def _print_opposite_pairs_warning(self, summary) -> None:
if not self._ANGLE_FILE_OPPOSITE_WARNING:
return
warning = describe_opposite_pairs(summary)
if warning:
self.OMNYTools.printredbold(f"Warning: {warning}")
def _min_scan_angle(self) -> float:
"""Lower angle limit (inclusive) accepted by _tomo_scan_at_angle().
tomo_type 4 files may contain any angle in the mechanical overtravel
range, not just the nominal 0-360 sweep."""
if self.tomo_type == 4:
return self._ANGLE_FILE_MIN
return 0
def _max_scan_angle(self) -> float:
"""Upper angle limit (inclusive) accepted by _tomo_scan_at_angle()."""
if self.tomo_type == 4:
return self._ANGLE_FILE_MAX
return self._default_max_scan_angle()
def _default_max_scan_angle(self) -> float:
"""Upper limit for tomo types 1-3; flomni overrides (tomo_angle_range)."""
return 360
def _start_angle_file_scan(self, projection_number) -> AngleFileResult:
"""Read + report the angle file at tomo_scan() start (before any gate
or motion), so a missing/malformed file fails immediately -- and, in
the tomo queue, marks the job incomplete with this message. The file
need not be complete (end with 'END') yet -- that is the normal
starting state for a file meant to be appended to while the scan runs."""
file_angles = self._read_tomo_angle_file()
summary = summarize_angles(file_angles.angles)
print(f"Angle file {self.tomo_angle_file}: {describe_summary(summary)}")
if not file_angles.is_complete:
print(
"Angle file has no 'END' marker yet -- the scan will wait for "
"more angles to be appended once it reaches the end of the file."
)
self._print_opposite_pairs_warning(summary)
self._warn_if_hook_ignores_overrides(file_angles.angles)
if projection_number is not None and projection_number >= len(file_angles.angles):
raise self._TOMO_PARAMS_ERROR(
f"Cannot resume at projection {projection_number}: the angle file now "
f"contains only {len(file_angles.angles)} angles."
)
return file_angles
def _init_angle_file_progress(self, file_angles: AngleFileResult, projection_number) -> None:
"""Fill the progress fields that are already known from the file, so the
GUI shows the real tomo type/total (and sub-tomogram 1) during the
preparation steps and the 0-deg reference shot that precede the loop,
instead of the blank reset defaults (tomo type 0, projection 0)."""
self.progress["tomo_type"] = "Angles from file"
self.progress["total_projections"] = (
len(file_angles.angles) if file_angles.is_complete else 0
)
if projection_number is None:
self.progress["subtomo"] = 1
def _run_angle_file_loop(self, file_angles: AngleFileResult, projection_number) -> None:
"""Acquire the angles of tomo_type 4 in file order (no extra 0-deg
damage-estimation shots: sub-tomogram numbers in a user file are just
labels, put 0-deg lines in the file instead). Waits for lines to be
appended while the file has no 'END' marker."""
error = self._TOMO_PARAMS_ERROR
self.progress["tomo_type"] = "Angles from file"
angles = file_angles.angles
is_complete = file_angles.is_complete
ii = 0 if projection_number is None else projection_number
subtomo_sizes, index_in_subtomo = self._tomo_angle_subtomo_layout(angles)
self.progress["total_projections"] = len(angles) if is_complete else 0
waiting = False
while True:
if ii >= len(angles):
if is_complete:
break
# Caught up to the current end of file, which isn't marked
# complete ('END') -- wait and poll instead of stopping, so
# lines appended while this scan is running are picked up
# without needing tomo_scan_resume().
if not waiting:
print(
f"Reached the end of {self.tomo_angle_file} "
f"({len(angles)} angle(s), no 'END' marker yet) -- "
"waiting for more lines to be appended. Ctrl-C to abort."
)
waiting = True
tick_start = time.perf_counter()
try:
time.sleep(self._TOMO_ANGLE_FILE_POLL_INTERVAL_S)
except KeyboardInterrupt:
raise error(
"tomo_scan() aborted while waiting for the angle "
f"file {self.tomo_angle_file} to grow past "
f"projection {ii}. Resume with tomo_scan_resume() "
"once more angles (or an 'END' marker) have been "
"added."
) from None
# Keep the heartbeat fresh so the GUI/webpage staleness
# detectors (120s/90s windows) don't flag this scan as dead
# while it's intentionally idle -- and since that keeps the
# gap _tomo_scan_at_angle() would otherwise see small,
# account the wait as idle time here explicitly.
self.progress["heartbeat"] = datetime.datetime.now().isoformat()
self.progress["accumulated_idle_time"] = self.progress.get(
"accumulated_idle_time", 0.0
) + (time.perf_counter() - tick_start)
new_result = self._read_tomo_angle_file()
if new_result.angles[: len(angles)] != angles:
raise error(
f"{self.tomo_angle_file} changed: an already-"
f"acquired line (index < {len(angles)}) was edited "
"or removed. Fix the file (only append new lines "
"after the last acquired one) and resume with "
"tomo_scan_resume()."
)
if len(new_result.angles) > len(angles):
print(
f"Picked up {len(new_result.angles) - len(angles)} "
f"new angle(s) from {self.tomo_angle_file} "
f"(total now {len(new_result.angles)})."
)
self._warn_if_hook_ignores_overrides(new_result.angles[len(angles) :])
waiting = False
angles = new_result.angles
subtomo_sizes, index_in_subtomo = self._tomo_angle_subtomo_layout(angles)
is_complete = new_result.is_complete
self.progress["total_projections"] = len(angles) if is_complete else 0
if is_complete:
summary = summarize_angles(angles)
print(f"Angle file complete: {describe_summary(summary)}")
self._print_opposite_pairs_warning(summary)
continue
entry = angles[ii]
self.progress["subtomo"] = entry.subtomo
self.progress["projection"] = ii
self.progress["angle"] = entry.angle
self.progress["subtomo_total_projections"] = subtomo_sizes[entry.subtomo]
self.progress["subtomo_projection"] = index_in_subtomo[ii]
overrides = self._angle_overrides_kwargs(
entry.step, entry.exp_time, entry.frames_per_trigger
)
self._tomo_scan_at_angle(entry.angle, entry.subtomo, **overrides)
ii += 1
def _print_angle_file_summary(self) -> bool:
"""Print path + parsed summary (+ 180-deg-pair warning if enabled) of
tomo_angle_file. Returns False (after printing the error) if the file
can't be used, instead of raising."""
print(f"Angle file: {self.tomo_angle_file or '(not set)'}")
try:
result = read_angle_file(self.tomo_angle_file)
except AngleFileError as exc:
self.OMNYTools.printredbold(f"Angle file not usable: {exc}")
return False
summary = summarize_angles(result.angles)
print(describe_summary(summary))
if not result.is_complete:
print("No 'END' marker yet -- a scan will wait for more angles once started.")
self._print_opposite_pairs_warning(summary)
return True
def _prompt_angle_file(self) -> None:
"""Ask for tomo_angle_file until it parses, or the user presses
enter to keep the current value."""
while True:
val = input(f"Angle file ({self.tomo_angle_file}): ").strip()
if not val:
if not self._print_angle_file_summary():
print("Keeping it anyway -- the scan will refuse to start until it is fixed.")
return
try:
read_angle_file(val)
except AngleFileError as exc:
self.OMNYTools.printredbold(str(exc))
print("Please try again (enter keeps the current value).")
continue
self.tomo_angle_file = val
self._print_angle_file_summary()
return
@@ -27,10 +27,15 @@ Consumers (e.g. ``Flomni``, ``LamNI``) must:
``flomni_at_each_angle`` builtins hook, single-point acquisition -- that
it stays out of this mixin)
- implement their own ``_resolve_type1_projection()`` (dispatched
generically by ``tomo_queue_reacquire()``), mapping a flat tomo_type-1
projection number to a ``(subtomo_start, start_angle)`` pair -- the
angle math is setup-specific (see each setup's own
``_subtomo_angle_plan()``), same reasoning as ``tomo_scan()`` above
generically by ``tomo_queue_reacquire()`` for any tomo_type in
``_TOMO_TYPE_1_LIKE``), mapping a flat projection number to a
``(subtomo_start, start_angle)`` pair -- the angle math is
setup-specific (see each setup's own ``_subtomo_angle_plan()``/
``_equally_spaced_subtomo_angles()``), same reasoning as
``tomo_scan()`` above
- optionally override ``_TOMO_TYPE_1_LIKE`` (default ``frozenset({1})``)
if their setup has more than one "equally spaced sub-tomograms"
tomo_type flavor (e.g. OMNY's types 1/4/5 -- 2/4/8 sub-tomograms)
"""
from __future__ import annotations
@@ -41,11 +46,14 @@ import inspect
import json
import os
import threading
import time
import uuid
from typing import Callable
from typeguard import check_type
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.angle_file import read_angle_file
if builtins.__dict__.get("bec") is not None:
dev = builtins.__dict__.get("dev")
umv = builtins.__dict__.get("umv")
@@ -210,9 +218,26 @@ class TomoQueueMixin:
_TOMO_QUEUE_ACTIONS: dict = {}
_TOMO_QUEUE_MOVE_DEVICES: dict = {}
# tomo_type values that behave like "type 1" for tomo_queue_reacquire()'s
# purposes -- i.e. resolved via _resolve_type1_projection() rather than a
# flat projection_number. Default matches flomni/lamni (type 1 only, their
# only "equally spaced sub-tomograms" flavor); OMNY overrides this to
# {1, 4, 5} since it has three such flavors (2/4/8 sub-tomograms).
_TOMO_TYPE_1_LIKE: frozenset = frozenset({1})
# tomo_parameters() GUI window: subclasses set a unique window name (a valid
# Python identifier, e.g. "lamni_tomo_params") and their own error class.
_TOMO_PARAMS_WINDOW_NAME: str = ""
_TOMO_PARAMS_ERROR: type[Exception] = RuntimeError
_TOMO_PARAMS_GUI_TIMEOUT = 20
# (pos_x, pos_y, width, height) of the tomo parameter window when it is
# created: tall and narrow, so the whole parameter column fits on screen.
_TOMO_PARAMS_WINDOW_GEOMETRY = (50, 50, 730, 1170)
# Name of a registered at_each_angle hook (see register_at_each_angle_hook()),
# or None for the default per-projection behaviour.
at_each_angle_hook = _GlobalVarParam(None)
tomo_id = _GlobalVarParam(-1, type_=int)
def _init_tomo_queue(self) -> None:
"""Call once from ``__init__``, after ``self.client`` is set.
@@ -236,6 +261,42 @@ class TomoQueueMixin:
# since-ended session must not linger and look registered.
self._publish_at_each_angle_hooks()
# ── tomo parameter GUI window ───────────────────────────────────────────
def tomo_parameters(self) -> None:
"""Open the tomo parameter GUI (OMNY_TomoParams) in its own window.
Calling it again while the window is open does nothing. The widget
works on the same global variables as the command line version,
``tomo_parameters_cli()``, which is not used as an automatic fallback:
if the GUI cannot be opened this raises, so the problem gets fixed.
"""
name = self._TOMO_PARAMS_WINDOW_NAME
error = self._TOMO_PARAMS_ERROR
gui = getattr(self.client, "gui", None)
if gui is None:
raise error(
"No BEC GUI is available in this session, so the tomo parameter window cannot be"
" opened. Use tomo_parameters_cli() for the command line version."
)
timeout = self._TOMO_PARAMS_GUI_TIMEOUT
try:
if name in gui.windows:
window = getattr(gui, name)
if hasattr(window, "tomo_params"):
print("The tomo parameter window is already open.")
return
else:
gui.new(name, geometry=self._TOMO_PARAMS_WINDOW_GEOMETRY, timeout=timeout)
time.sleep(1)
window = getattr(gui, name)
window.new("OMNY_TomoParams", object_name="tomo_params", timeout=timeout)
except Exception as exc: # pylint: disable=broad-except
raise error(
f"Could not open the tomo parameter window ({type(exc).__name__}: {exc}). Use"
" tomo_parameters_cli() for the command line version."
) from exc
# ── at_each_angle hook registry ─────────────────────────────────────────
def register_at_each_angle_hook(self, name: str, func) -> None:
@@ -392,9 +453,8 @@ class TomoQueueMixin:
user=user,
)
# Must match whatever host TomoIDManager.OMNY_URL/TEST_OMNY_URL
# registered against, since that host's sample counter is what
# self.tomo_id came from.
# Must match whatever host TomoIDManager.OMNY_URL registered against,
# since that host's sample counter is what self.tomo_id came from.
_SAMPLES_UPLOAD_HOSTS = ("https://omny.psi.ch",)
def _upload_pdf_report_to_samples(self, pdf_path: str) -> None:
@@ -541,6 +601,11 @@ class TomoQueueMixin:
if not isinstance(value, (int, float)) or isinstance(value, bool):
raise ValueError(f"{action_name}.{name} must be a number, got {value!r}.")
lo, hi = field.get("min"), field.get("max")
device_name = field.get("device_limits_from")
if device_name is not None:
device = self.device_manager.devices.get(device_name)
if device is not None:
lo, hi = device.limits
if lo is not None and value < lo:
raise ValueError(f"{action_name}.{name}={value} is below the minimum {lo}.")
if hi is not None and value > hi:
@@ -573,23 +638,6 @@ class TomoQueueMixin:
args += [dev[name], target]
umv(*args)
def optimize_idgap(self, search_range: float = 0.5) -> None:
"""Scan idgap over +/- search_range around its current position and
move to the peak of <signal not yet decided>.
STUB -- not implemented. The peak-detection signal/device is not yet
decided (see flomni's AI_docs/TOMO_QUEUE_COMMAND_JOBS_PLAN.md section
10). This is the ONLY place that needs to change to make this a real
action: the registry entry, resume policy, and (eventually) GUI
wiring already treat it as a normal idempotent compound action, so
dropping in the real scan-and-move-to-peak logic here (or swapping
which device/signal it reads) is a self-contained change.
"""
print(
f"optimize_idgap(search_range={search_range}): not yet implemented -- "
"peak-detection signal not yet decided. No-op."
)
def _describe_tomo_queue_actions(self) -> dict:
"""Serializable description of the action registry + move allow-list.
@@ -898,13 +946,22 @@ class TomoQueueMixin:
for name, value in job["params"].items():
setattr(self, name, value)
if resume_job:
self.tomo_scan_resume()
# from_queue=True: skip tomo_scan_resume()'s
# "unfinished queue job" prompt -- this *is* the queue.
self.tomo_scan_resume(from_queue=True)
else:
# interactive=False: an unattended queued run must
# never block on input() -- see LamNI.tomo_scan()'s
# fine-alignment check.
self.tomo_scan(interactive=False)
except Exception as exc:
except (Exception, KeyboardInterrupt) as exc:
# KeyboardInterrupt is a BaseException, not an Exception -- a
# bare `except Exception` misses it, leaving the job stuck at
# status="running" with no explanation the next time the
# queue is inspected (confirmed live: a Ctrl+C landing during
# tomo_scan_resume()'s GUI-raise call produced exactly this --
# a silent "running" job and a raw traceback instead of the
# informative message below).
self._tomo_queue_proxy.update_by_id(job_id, status="incomplete")
print(f"Tomo queue job '{label}' did not complete: {exc}")
print(
@@ -969,6 +1026,35 @@ class TomoQueueMixin:
"tomo_queue_reacquire() on a tomo_type-1 job."
)
def _confirm_resume_outside_queue(self) -> bool:
"""Guard for a manual ``tomo_scan_resume()`` call.
If the queue holds an "incomplete"/"running" job (e.g. after a
Ctrl-C abort, which leaves it "running"), resuming by hand bypasses
the queue: the job stays unfinished there, so a later
``tomo_queue_resume()`` would resume it again and the jobs after it
would not run. Warn, print the hint, and ask. Returns True if the
resume should go ahead (nothing unfinished in the queue, or the
operator confirmed).
"""
unfinished = [
j for j in self._tomo_queue_proxy.as_list() if j["status"] in ("incomplete", "running")
]
if not unfinished:
return True
print("\nWARNING: the tomo queue has an unfinished job:")
for job in unfinished:
print(
f" {job['status']:>10s} {job['label']:<20s} {self._describe_tomo_job_line(job)}"
)
print(
"Use tomo_queue_resume() to resume it and continue with the rest of the queue.\n"
"If you resume manually instead, the queue is not told -- mark the job done "
"afterwards, or it will be resumed again:\n"
" self._tomo_queue_proxy.update_by_id(<job id>, status='done')"
)
return self.OMNYTools.yesno("Run tomo_scan_resume() anyway, bypassing the queue?", "n")
def tomo_queue_resume(self, start_index: int = 0) -> None:
"""Continue running the tomo queue from wherever it currently
stands -- an alias for ``tomo_queue_execute()``, added purely so
@@ -1012,10 +1098,10 @@ class TomoQueueMixin:
job_index: Position of the job to reopen, in the *current*
queue order (same indexing as ``tomo_queue_show()``/
``tomo_queue_delete()``).
projection_number: Flat 0-indexed projection to resume from,
for any tomo_type (1, 2, or 3) -- see
``_resolve_type1_projection()`` for how this maps onto a
tomo_type-1 job's (subtomo, angle).
projection_number: Flat 0-indexed projection to resume from, for
any tomo_type -- see ``_resolve_type1_projection()`` for how
this maps onto a ``_TOMO_TYPE_1_LIKE`` job's (subtomo, angle); for
a tomo_type-4 angle-file job (flOMNI/LaMNI) it is the angle-file index.
Raises:
TomoQueueError: ``job_index`` is out of range, the job at that
@@ -1049,10 +1135,22 @@ class TomoQueueMixin:
for name, value in job["params"].items():
setattr(self, name, value)
if self.tomo_type == 1:
if self.tomo_type in self._TOMO_TYPE_1_LIKE:
subtomo_start, start_angle = self._resolve_type1_projection(projection_number)
self.progress["subtomo"] = subtomo_start
self.progress["angle"] = start_angle
elif self.tomo_type == 4:
# angles from file (flomni) -- bound by the file as it reads now,
# since the resumed tomo_scan() re-reads it too.
if projection_number < 0:
raise ValueError("tomo_queue_reacquire: projection_number must be >= 0.")
n_angles = len(read_angle_file(self.tomo_angle_file).angles)
if projection_number >= n_angles:
raise ValueError(
f"projection_number must be < {n_angles} (number of angles in "
f"{self.tomo_angle_file}) for this job."
)
self.progress["projection"] = projection_number
elif self.tomo_type in (2, 3):
if projection_number < 0:
raise ValueError("tomo_queue_reacquire: projection_number must be >= 0.")
@@ -1078,6 +1176,8 @@ class TomoQueueMixin:
self.progress["estimated_finish_time"] = None
self.progress["accumulated_idle_time"] = 0.0
self.progress["heartbeat"] = None
# flOMNI: a completed scan is not resumed (tomo_scan_resume); reopened here on purpose
self.progress["tomo_completed"] = False
job_id = job["id"]
self._tomo_queue_proxy.update_by_id(job_id, status="incomplete")
@@ -404,14 +404,21 @@ class WebpageGeneratorBase:
takes. Used by the UI to compute total-projection
counts for queued jobs without hardcoding a
per-instrument formula in JS. Recognised "kind"s:
"equally_spaced_grid" (params: n_subtomos)
total = floor(180 / angle_stepsize) * n_subtomos
"equally_spaced_grid" (params: n_subtomos,
optional base_angle,
default 180)
total = floor(base_angle / angle_stepsize)
* n_subtomos
(independent of angle_range - see
Flomni._subtomo_angle_plan()'s docstring for
why: in 360-degree mode, angle_range only
splits the n_subtomos into complementary
low/high halves, it does not change the
total count)
total count). base_angle is the angular span
ONE sub-tomogram covers: 180 for flomni, 360
for lamni (which always sweeps the full
circle per sub-tomogram -- see
LamNI._tomo_type1_actual_grid()).
"golden_capped"
total = golden_max_number_of_projections
TQ_PARAM_DISPLAY -- list of (global_var_key, display_label) pairs, in
@@ -436,6 +443,10 @@ class WebpageGeneratorBase:
whole shared output dir, EPS files included.
"""
# Name of the setup's object in the BEC client namespace (e.g. "lamni"),
# used in console hints such as the set_web_password() one.
CLIENT_NAME: str | None = None
HAS_TOMO_QUEUE = True
HAS_EPS_STATUS = True
TOMO_TYPES = {
@@ -809,7 +820,8 @@ class WebpageGeneratorBase:
_BOLD_RED = "\033[1;31m"
_RESET = "\033[0m"
print(f"{_BOLD_RED}***\nWebpageGenerator: '{current_account}' has no web authentication yet.{_RESET}")
print(f"{_BOLD_RED} To set a password run: flomni.set_web_password(\"your_password\")\n***{_RESET}")
prefix = f"{self.CLIENT_NAME}." if self.CLIENT_NAME else ""
print(f"{_BOLD_RED} To set a password run: {prefix}set_web_password(\"your_password\")\n***{_RESET}")
def _clear_session_htpasswd(self) -> None:
"""Write an empty session.htpasswd locally and upload it."""
@@ -828,7 +840,7 @@ class WebpageGeneratorBase:
The username is taken from bec.active_account (e.g. 'p23092').
Example::
flomni.webpage_gen.set_web_password("my_secret_password")
flomni.set_web_password("my_secret_password")
"""
if self._set_password_url is None:
print("set_web_password: no upload URL configured — cannot reach server")
@@ -2701,13 +2713,15 @@ function calcProjections(params){{
return (gmax!=null&&gmax>0)?gmax:null;
}}
if(def.kind==='equally_spaced_grid'){{
// Always floor(180/step), never floor(angle_range/step): in 360-degree
// mode angle_range only splits the sub-tomograms into complementary
// low/high halves, it does not change the total projection count -
// see Flomni._subtomo_angle_plan()'s docstring.
// Always floor(base_angle/step), never floor(angle_range/step): in
// 360-degree mode angle_range only splits the sub-tomograms into
// complementary low/high halves, it does not change the total
// projection count - see Flomni._subtomo_angle_plan()'s docstring.
// base_angle is the span of one sub-tomogram: 180 (default, flomni) or
// 360 (lamni, which sweeps the full circle in every sub-tomogram).
const step=params.tomo_angle_stepsize;
if(step>0){{
const N=Math.floor(180/step);
const N=Math.floor((def.base_angle||180)/step);
return N*(def.n_subtomos||1);
}}
}}
@@ -757,7 +757,7 @@ class cSAXSFilterTransmission:
def _fil_trans_report(self, tol: float = 0.1, energy_kev: Optional[float] = None) -> None:
"""
Report the currently active exposure‑box filter combination.
Determines stage positions via dev.<axis>.readback.get()
Determines stage positions via dev.<axis>.readback.get(cached=True)
with a tolerance window of ±tol relative to nominal positions.
Parameters
@@ -799,7 +799,9 @@ class cSAXSFilterTransmission:
return
try:
rb = float(axis_obj.readback.get())
# Status report only: the last published readback (cached=True, from redis)
# avoids one RPC round trip per axis. A missing value raises in float() below.
rb = float(axis_obj.readback.get(cached=True))
except Exception:
print(f"ERROR: readback unavailable for axis {axis_name}")
return
@@ -47,7 +47,10 @@ class cSAXSSlits:
return None
try:
reading = device.read()
# cached=True reads the last published readback from redis. The default
# (cached=False) is one RPC round trip to the device server per device,
# i.e. 24 of them for slits_show_all().
reading = device.read(cached=True)
value = reading[device_name]["value"]
return float(value)
except Exception:
@@ -0,0 +1,419 @@
# Validating PandaBox (`omny_panda_continuous`) recovered positions against RT
**Purpose of this document:** how to check whether positions recovered from
the PandaBox's Mode B (free-running raw stream, `raw_stream_mode: true`)
capture actually match the true stage positions, using the RT controller's
own trusted per-point averages as ground truth. Read this before continuing
that testing in a new session — it captures the full methodology, the exact
conversion formula, and a bug that was found and fixed this way, so none of
it needs re-deriving from scratch.
Companion docs:
- `docs/developer/panda_box_free_running_setup.md` — the PandA layout/mode
design doc. Has the mode comparison, signal alias mapping, and (as of
2026-09-21) the "Known data-quality caveat: stale `frame_counter`
carryover" section this investigation produced.
- `docs/developer/panda_box_free_running_data_schema.md` — the HDF5 field
list for the Mode B raw stream.
- `csaxs_bec/devices/panda_box/panda_box_omny.py` — the device. See
`_drop_stale_frame_counter_rows`/`_reset_raw_stream_state` for the fix
described below.
---
## 1. The problem this validates
Mode B captures raw samples continuously at `CLOCK1`'s rate for the whole
burst window — **not** one sample per scan point. Only a subset of rows
coincide with an actual detector exposure (`gate_detector_active == 1`), and
even those need to be grouped by which point they belong to
(`frame_counter`). Treating every raw row as its own recovered position
(without grouping/filtering) produces a scatter that roughly traces the
scan's envelope but does not correspond point-for-point to the true
positions — and critically, **no scale/offset correction can fix that**,
because it's a many-to-one correspondence problem, not a coordinate error.
This was confirmed visually (see the git history around 2026-09-20/21 for the
red/blue scatter plot that started this investigation) before the grouping
method below was worked out.
## 2. Ground truth: `rt_positions`
The RT controller logs its own per-point values, already correctly averaged
and known-good (used successfully in reconstruction):
- `rt_positions_data_target_x`/`_y` — commanded (programmed) position.
- `rt_positions_data_average_x_st_fzp`/`_y` — **the trusted measured
position**, in microns. Compare recovered PandA positions against these,
not against `target_x`/`_y`.
- `rt_positions_data_stdev_x_st_fzp`/`_y` — RT's own per-point stdev, useful
as a sanity scale for how much residual noise to expect (order 0.005 µm in
the validated scan). See §9 for a raw-PandA cross-check against this value.
Don't confuse this with `rt_positions_data_average_stdeviations_x_st_fzp`/
`_y`, which is a running/cumulative mean of `stdev_x_st_fzp` up to that
point in the scan (confirmed by direct comparison, 2026-09-23), not a
per-point noise value — it's the wrong field for a per-point comparison.
One row per scan point (472 points in the validated scan, `S06313`).
## 3. Recovering positions from the Mode B raw stream
### 3.1 Raw fields (all same length, one entry per raw sample)
Under `entry/collection/devices/omny_panda_continuous/` in the master HDF5:
`omny_panda_continuous_data_<field>/value`, where `<field>` is one of
`interf_st_fzp_x`, `interf_st_fzp_y`, `cap_voltage_fzp_x`, `cap_voltage_fzp_y`,
`frame_counter`, `gate_detector_active` (plus `interf_st_rotx`/`_rotz`,
`pcap_gate_duration_value`, not needed for x/y position recovery).
### 3.2 Required grouping/filtering — do this before anything else
1. Filter to rows where `gate_detector_active == 1` (real exposure, not
inter-point flight time).
2. Group the remaining rows by **point** index. For a single-frame-per-point
scan (`frames_per_trigger == 1`), `frame_counter` directly ranges
`0..num_points-1` and *is* the point index. For a burst scan
(`frames_per_trigger > 1`), `frame_counter` increments once per
sub-frame trigger, not once per point (confirmed 2026-09-22 against real
`frames_per_trigger=3` scans, `S00395`/`S00396`/`S00398` — see §7): the
point index is `frame_counter // frames_per_trigger`, and *all* of a
point's sub-frames' gate-active rows get averaged together into that
point's one `rt_positions` value, not kept separate.
3. Average `interf_st_fzp_x`/`_y` and `cap_voltage_fzp_x`/`_y` within each
group.
Skipping this step is the #1 way to get a result that looks roughly
right-shaped but doesn't actually match — see §1.
### 3.3 Stale `frame_counter` carryover — check this if a session predates the fix
PandA's `COUNTER1.OUT` is not cleared by `Arm()`, so the first raw samples of
an acquisition can briefly carry a stale `frame_counter` value left over from
the *previous* acquisition's tail. If that stale value collides with a real
point index later in the same scan, it contaminates that point's group.
`panda_box_omny.py` fixes this client-side as of commit `3eb6e03`
(2026-09-21) — it drops rows before the first observed `frame_counter == 0`
on every arm. **Any HDF5 file captured before that fix was live on the real
device** (i.e. anything from before 2026-09-21, `S06313` included) still has
the stale rows in it and needs the same drop applied manually before
grouping:
```python
reset_idx = np.argmax(frame_counter_raw == 0.0) # first occurrence
frame_counter_raw = frame_counter_raw[reset_idx:]
# ...apply the same [reset_idx:] slice to every other raw field, in lockstep
```
For anything captured after the fix, this is unnecessary (already handled
device-side) — but it's cheap and harmless to keep doing as a defense-in-depth
check; if it ever finds `reset_idx > 0` again on a fixed device, that's a
regression worth investigating.
### 3.4 Conversion formula (from Mirko, confirmed 2026-09-21)
```python
factor_x = -0.000309078
factor_y = 0.000154539
interfx = grouped_interf_st_fzp_x * factor_x # microns
interfy = grouped_interf_st_fzp_y * factor_y # microns
capmuy = -(grouped_cap_voltage_fzp_y) * 5.0 / 20.0
capmux = (grouped_cap_voltage_fzp_x) * 5.0 / 20.0
CapX_FZP_microns = 0.9539 * capmux + 0.0009008 * capmuy
CapY_FZP_microns = -0.023140 * capmux + 1.0120 * capmuy
pos_x_um = interfx - CapX_FZP_microns # microns -- compare directly against average_x_st_fzp
pos_y_um = interfy - CapY_FZP_microns # microns -- compare directly against average_y_st_fzp
```
**Units trap:** the original formula this was transcribed from ends with
`pos_x = (interfx - CapX_FZP_microns) * 1e-6` to get meters, for whatever
consumed it downstream. `rt_positions_data_average_x_st_fzp`/`_y` are in
**microns**, so compare against `pos_x_um`/`pos_y_um` *before* that final
`* 1e-6` — don't convert one side to meters and leave the other in microns.
`factor_x`/`factor_y` were independently re-derived by least-squares fit
against `average_x_st_fzp`/`_y` from `S06313` and matched Mirko's values to 4
significant figures (fit: `-3.089108e-04`/`1.545243e-04` vs. given
`-0.000309078`/`0.000154539`) — good cross-check if a future factor is ever
in doubt.
## 4. Validating a match
```python
def r2(pred, true):
ss_res = np.sum((true - pred) ** 2)
ss_tot = np.sum((true - true.mean()) ** 2)
return 1 - ss_res / ss_tot
```
Expected result on a correctly grouped, correctly converted scan (from
`S06313`, post-fix):
- A single near-constant offset per axis between `pos_x_um`/`pos_y_um` and
`average_x_st_fzp`/`_y` (validated: X ≈ −1.26 µm, Y ≈ +0.99 µm) — this is
expected (RT's internal reference zero vs. the PandA/`INENC` counter's own
free-running zero don't coincide) and irrelevant to reconstruction, since
it's the same constant for every point in one scan. Subtract the mean
residual per axis before judging fit quality.
- After removing that offset: R² > 0.9999 on both axes, residual std ~0.25–
0.3 nm, max residual under ~1 nm across all points (post-fix; pre-fix, one
point was off by ~116 nm — see §3.3).
If R² is poor and doesn't clean up after removing a constant offset (i.e. a
best-fit affine still doesn't work — this is the actual symptom that flagged
the original bug, see §1), suspect the grouping/filtering step (§3.2) before
suspecting the conversion factors.
## 5. Practical notes
- Use a BEC production venv `python3` for any h5py-based analysis — the
system `python3` doesn't have `h5py` installed. Path is host-dependent:
`/opt/bec_deployments/production/bec_venv/bin/python3` on some hosts,
`/sls/x12sa/config/bec/production/bec_venv/bin/python3` on others (e.g.
`x12sa-bec-001`) — check both if one is missing.
- `S06313`'s files were at `/tmp/S06313/` (`S06313_master.h5` has both
`omny_panda_continuous` and `rt_positions`; `S06313_eiger_*` are detector
data, not needed for this). `/tmp` is not persistent — get a fresh scan's
files from whoever's driving the beamtime if this path is gone.
- Relevant H5 group prefix:
`entry/collection/devices/<device>/<device>_data_<field>/value` (and
`.../timestamp`, much shorter — one entry per buffer flush, not per row,
see `_flush_raw_stream_buffer_locked` in `panda_box_omny.py`).
## 6. Next steps (as of 2026-09-21)
1. **Burst acquisition** — RESOLVED 2026-09-22, see §7/§3.2: `frame_counter`
increments once per sub-frame trigger, not once per point;
`frame_counter // frames_per_trigger` is the point index, and all
sub-frames of a point get averaged together to match RT's one value per
point. Confirmed against real `frames_per_trigger=3` scans
(`S00395`/`S00396`/`S00398`), R² = 1.0000000 on both axes.
2. **Reduced data mode** — characterized empirically 2026-09-22, see §7
(`S00399`): captures only rows with `gate_detector_active == 1` (or very
close to it — 99.5% vs. ~39% of rows in normal continuous mode), cutting
total row count roughly in half to a third while keeping the same
per-point exposure sample count and full positional accuracy (R² =
1.0000000). The underlying PandA-side layout/config change that produces
this is not visible from BEC's `configuration` snapshot in the HDF5 file
(empty for both modes) — this section describes the observed effect on
the data, not the PandA-side mechanism; get that from whoever configured
it if it needs modifying.
## 7. Post-fix confirmation against live scans (2026-09-22)
Checked two scans taken after `3eb6e03` landed (both at `/tmp/S<id>/` on
`x12sa-bec-001`, files from 2026-09-22 ~12:45–12:58):
- **`S00386`** (174 points, ~348k raw rows): `frame_counter` starts at `0` on
the very first raw row and has zero backward steps across the whole raw
stream — no leading stale-value prefix at all, unlike pre-fix `S06313`.
Full grouping+conversion validation against `rt_positions`: R² =
0.99999997 (X) / 0.99999998 (Y) after removing the expected constant
per-axis offset (−1.23 µm X, +1.08 µm Y, consistent with §4's `S06313`
values); residual std 0.30 nm (X) / 0.25 nm (Y); worst single-point
residual 1.02 nm (X) / 0.51 nm (Y) — no outlier anywhere near the ~116 nm
the pre-fix bug produced. Clean confirmation of the fix.
- **`S00388`** (785 points, ~1.4M raw rows): one backward step, at raw row
36 (`frame_counter` reads `28` for rows 0–36, then drops to `0` from row
37 on) — structurally the same leading-stale-prefix signature the fix is
meant to strip, still present in data captured after the fix. However,
every one of those 37 stale rows has `gate_detector_active == 0`, so
they'd never enter a point's average regardless (§3.2 already filters to
`gate_detector_active == 1` before grouping) — this scan happened not to
be exposed to the bug's actual failure mode. Full validation still comes
out clean: R² = 0.99999999 (X) / 0.99999998 (Y), residual std 0.27 nm (X)
/ 0.26 nm (Y), max residual 0.85 nm (X) / 0.56 nm (Y).
**Root-cause investigation (updated 2026-09-22, still not fully resolved):**
Ruled out first: the leading `frame_counter` column can't be missing from
`S00388`'s first post-arm chunk the way an earlier version of this note
speculated. `pandablocks.connections.DataConnection`'s captured field set is
fixed once per TCP data connection (established once per `on_stage`, before
any `FrameData` is produced), so every `FrameData` chunk within one
acquisition carries the same columns — the "layout doesn't capture
`frame_counter`" fallback in `_drop_stale_frame_counter_rows`
(`counter_entry is None`) isn't reachable here.
What actually happened, confirmed from `ScanServer.log`/`DeviceServer.log`
for 2026-09-22: the scan immediately preceding `S00388` — scan **#387**
(786 points, `eb7cfd11-3610-4511-94a0-75f49b403e62`) — was opened at
`12:50:10` and force-**aborted** by the user only ~7 seconds later
(`12:50:16`–`17`, `action: 'abort'` in the scan queue). At `exp_time=0.1`s/point
that's on the right order to land around point ~20–30, consistent with the
leftover `frame_counter == 28` that then showed up at the start of `#388`'s
raw stream (`#388` itself opened cleanly at `12:50:39`, no arm retries
logged — matches the "one `on_stage` per scan number" mapping confirmed
against the log). This is exactly the hardware mechanism `3eb6e03`'s commit
message describes (`COUNTER1.OUT` not cleared by `Arm()`) — just triggered
by an **abort mid-scan** rather than a normal scan-to-scan transition, which
is what `S06313` (the original bug report) and `S00386` (clean, §7 above)
both were.
Why the client-side filter didn't strip it is not fully pinned down.
`PandaBoxOMNY.stop()` (inherited, not overridden) does not call
`_reset_raw_stream_state()` — only `on_stage()` (once per scan) and
`_try_arm()` (right before every `Arm()`) do, and both ran normally for
`#388` per the logs. So by the code's own logic the state should have been
freshly reset well before `#388`'s `Arm()` went out. One structural
suspect, not yet confirmed as the actual cause: `_raw_stream_buffer`/
`_raw_stream_buffer_row_count`/`_raw_stream_last_flush` are all mutated only
under `_raw_stream_buffer_lock`, but `_raw_stream_seen_reset` is set outside
that lock in `_reset_raw_stream_state()` (after the `with` block) and read
outside the lock at the top of `_drop_stale_frame_counter_rows()` — the one
piece of this acquisition's reset state that isn't synchronized the same
way as everything else it's reset alongside. Whether that gap is actually
reachable in a plain abort → restage sequence (as opposed to needing an
`on_pre_scan` retry, which `#388` didn't have) isn't shown by the logs
available here; would need instrumentation (e.g. temporarily logging
`_raw_stream_seen_reset` transitions) against a reproduced abort-then-rescan
to confirm.
Practically: harmless in both scans checked so far, since the stale rows'
`gate_detector_active` was `0` throughout — they never entered a point's
average regardless of the row-level filter. But an abort mid-scan is a
routine, user-triggered event (unlike an `on_pre_scan` retry), so this path
is worth closing rather than relying on the gate happening to be low during
the stale window. If revisited: consider (a) locking `_raw_stream_seen_reset`
reads/writes the same as the buffer fields, and (b) also calling
`_reset_raw_stream_state()` from `on_stop`/`stop()`, not only `on_stage`/
`_try_arm`, so an aborted acquisition can't leave stale state for the next
one to inherit. Re-check against a scan where the stale run's rows show
`gate_detector_active == 1` before considering this fully closed.
**Mitigation added 2026-09-22:** `omny_panda_continuous` was being armed and
disarmed for *every* scan (any `grid_scan`/`line_scan`/alignment move, not
just `flomni_fermat_scan`), since generic BEC staging has no per-scan device
opt-in and `deviceTags` aren't consulted by it — see "Conditional arming
(`arm_scan_allowlist`)" in `docs/developer/panda_box_free_running_setup.md`.
It's now only actually `Arm()`ed for `flomni_fermat_scan` via the new
`arm_scan_allowlist` device config, which directly cuts down how often this
class of bug can even be triggered (fewer arm cycles → fewer abort→restage
windows like `S00388`'s), independent of whatever the remaining root cause
above turns out to be.
## 8. Batch validation across `S00391`-`S00399` (2026-09-22)
Nine `flomni_fermat_scan` runs at `/tmp/S00<391..399>/` on `x12sa-bec-001`,
run as three groups: `S00391`-`394` (single-frame, `frames_per_trigger=1`,
`393` aborted mid-scan), `S00395`-`398` (burst, `frames_per_trigger=3`,
`397` aborted mid-scan), `S00399` (burst, reduced data mode). Note: the
running `bec-device-server` process had been up since before commit
`5590633` (`arm_scan_allowlist`) landed, so it was still running the
pre-`arm_scan_allowlist` code for this whole batch — doesn't matter here
since every scan in this batch is `flomni_fermat_scan`, which
`arm_scan_allowlist` always arms anyway, but worth knowing: this batch
doesn't demonstrate that fix, only the pre-existing stale-`frame_counter`
fix (`3eb6e03`, which does predate the process start and was active).
**Full position-recovery validation** (§3.2-3.4/§4 methodology, extended for
burst grouping per §6) against `rt_positions`, all nine scans:
| Scan | mode | points | R² x / y | residual std (nm) x / y | max abs (nm) x / y |
|---|---|---|---|---|---|
| S00391 | single | 99 | 0.9999999 / 0.9999999 | 0.94 / 0.71 | 2.64 / 1.83 |
| S00392 | single | 98 | 0.9999999 / 0.9999999 | 0.95 / 0.76 | 2.75 / 2.09 |
| S00394 | single (post-abort) | 97 | 0.9999999 / 0.9999999 | 0.93 / 0.73 | 2.28 / 2.04 |
| S00395 | burst×3 | 98 | 1.0000000 / 1.0000000 | 0.47 / 0.55 | 0.98 / 1.18 |
| S00396 | burst×3 | 101 | 1.0000000 / 1.0000000 | 0.41 / 0.55 | 1.12 / 1.17 |
| S00398 | burst×3 (post-abort) | 99 | 1.0000000 / 1.0000000 | 0.45 / 0.54 | 1.32 / 1.13 |
| S00399 | burst×3, reduced data | 100 | 1.0000000 / 1.0000000 | 0.48 / 0.50 | 1.69 / 1.14 |
No point had zero grouped samples in any scan; no residual outlier anywhere
close to the ~116 nm the pre-fix bug produced. Burst-mode R² came out even
tighter than single-frame mode (1.0000000 vs. 0.9999999), consistent with
averaging 3x more raw samples per point.
**Stale-`frame_counter` prefix — two more clean, precisely-explained
repros**, both immediately following an abort, both harmless (all stale
rows have `gate_detector_active == 0`):
- `S00393` aborted with `rt_positions` recording exactly 29 points (panda
`frame_counter` max = `29.0`, matching exactly). The very next scan,
`S00394`, starts with 37 rows reading a stale `29.0` before dropping to
`0` at row 37 — same signature as `S00388`, same stale value as `#393`'s
own last completed point.
- `S00397` (burst×3) aborted with `rt_positions` recording exactly 17
points — 17×3 = 51 sub-frame counts, and panda `frame_counter` max there
is indeed `51.0`. The next scan, `S00398`, starts with 37 rows reading a
stale `51.0` before dropping to `0` at row 37 — same signature again, and
the stale value again matches `#397`'s own last completed sub-frame count
exactly.
This nails down the trigger mechanism about as precisely as it can be nailed
down from data alone: the leftover `frame_counter` value inherited by the
next scan is *exactly* the aborted scan's last completed count (point count
for single-frame, point×frames_per_trigger for burst) — not some unrelated
stale value from further back. The remaining open question from §7 (why the
client-side filter doesn't strip these rows in the abort→restage case, given
`on_stage`/`_try_arm` both reset state normally beforehand) is unchanged by
this batch — still worth instrumenting if pursued further, though it
remains harmless in every case observed so far because the stale window
never overlaps a gated exposure.
**Reduced data mode** (`S00399`) validated cleanly at full accuracy — see
§6 for the row-count/gate-fraction characterization. No contamination, no
missing points, same R² bar as the other burst scans.
## 9. Per-point stdev cross-check against RT's control-loop stdev (`S06313`, 2026-09-23)
Besides the per-point mean position (§4), PandA's raw samples let you
cross-check RT's own per-point noise estimate
(`rt_positions_data_stdev_x_st_fzp`/`_y`, §2) against an independent
measurement — a single-frame scan has ~998 raw PandA samples landing within
each point's gate window (`CLOCK1`'s free-running rate over one exposure),
so each point's within-group sample scatter is itself a noise estimate.
### 9.1 Method
For each grouped point (§3.2, after applying §3.3's stale-prefix drop where
needed), compute `pos_x_um`/`pos_y_um` (§3.4's full formula) **per
individual raw sample**, not just on the group mean, then take the
within-group sample stdev (`ddof=1`). Compare against
`rt_positions_data_stdev_x_st_fzp`/`_y` (not
`average_stdeviations_x_st_fzp`/`_y` — see the note in §2).
### 9.2 The analog `cap_voltage_fzp_x`/`_y` term is required, not optional
Naively computing per-sample stdev from `interf_st_fzp_x`/`_y` alone
(skipping the `CapX_FZP_microns`/`CapY_FZP_microns` subtraction in §3.4)
gives a wildly wrong noise estimate:
| | X stdev (µm) | Y stdev (µm) | corr. with RT stdev (472 points) |
|---|---|---|---|
| Interferometer only, no cap term | 0.072 | 0.049 | 0.22 / 0.24 |
| Full formula (interf − cap, §3.4) | 0.0074 | 0.0063 | 0.98 / 0.99 |
| RT's own stdev | 0.0065 | 0.0058 | — |
Within a single point, `interf_st_fzp_x` and `cap_voltage_fzp_x` are
anti-correlated at **r ≈ −0.96** (checked on point index 5, n=998 raw
samples). The raw interferometer counts carry a large common-mode
noise/drift component roughly 10x bigger than the true positional noise; the
capacitive-sensor subtraction in §3.4 is what cancels it — the same
combination that's already required for the mean position (§4) turns out to
be even more load-bearing for the noise estimate. Both `FMC_IN.VAL1/VAL2`
(the capacitive analog inputs) and `INENC1`-`4.VAL` (the encoder position
inputs) are captured with `CAPTURE=Value` off the same `CLOCK1` tick (see
`docs/developer/panda_box_free_running_setup.md`), so this isn't a
sampling-rate mismatch between the two channels — the interferometer signal
genuinely is that much noisier on its own.
**Practical implication:** any future per-sample noise/variance analysis of
Mode B data must use the full interf+cap combination (§3.4), never the raw
interferometer signal alone — skipping the cap term breaks the noise
estimate far more severely than it breaks the mean position.
### 9.3 Result (`S06313`, all 472 points)
PandA's within-point stdev (full formula) runs ~8–14% higher than RT's own
stdev (X: 0.0074 vs 0.0065 µm; Y: 0.0063 vs 0.0058 µm), and the two
independent noise estimates correlate at r ≈ 0.98 (X) / 0.99 (Y)
point-by-point. The small systematic excess is expected, not a discrepancy:
RT's stdev comes from its own already-averaged control-loop samples, while
PandA's ~998 raw samples per point pick up additional high-rate noise RT's
loop doesn't report, plus whatever small residual noise the interf/cap
combination itself adds. This doesn't affect the mean-position validation in
§4 (a noise-floor cross-check, not a bias check) — it's a useful additional
confirmation that PandA's raw stream and RT's control loop are seeing
consistent underlying physics, not just consistent means.
@@ -138,10 +138,69 @@ Passed: sample storage + GUI, moves incl. live progress bars and Ctrl-C, rotatio
angle coupling, feedback enable/disable/status, tracker, fermat scans incl. progress and
abort, X-ray eye GUI with camera frames, storage bookkeeping. Pending: full transfer
chain end-to-end, tomo_scan (unblocked by the scilog fix), tomo queue executor
(crash-resume checklist), then queue-reordering implementation. LamNI/OMNY: harness-
validated; a short live smoke test (config load, a move, one scan, one OMNY transfer)
recommended before relying on them, since flOMNI testing surfaced integration issues the
harness cannot see.
(crash-resume checklist), then queue-reordering implementation. LamNI: harness-validated;
live smoke test still pending.
**OMNY (2026-08-26): live smoke test run, config load / move / transfer passed, scan
blocked by a real (pre-existing, non-sim) bug.** Fixes made along the way (separate from
this doc's sim-package scope, see the repo's git history for exact diffs):
- `bec_ipython_client/startup/post_startup.py`: `--session omny` imported `OMNY` from the
`flomni` plugin package (which doesn't export it) instead of `omny` — `ImportError` on
every OMNY session start.
- `omny/x_ray_eye_align.py`: `XrayEyeAlign.__init__` called `self.lamni.align.reset_*()`
on the OMNY instance mid-construction (`OMNY.__init__` does
`self.align = XrayEyeAlign(self.client, self)`, so `.align` doesn't exist yet) —
`AttributeError` on every `OMNY()` construction. This file is otherwise still an
unadapted copy of LamNI's `x_ray_eye_align.py` (docstrings, types, and `self.lamni`
throughout) — the same kind of OMNY-specific adaptation `LamNI`'s copy already got is
still open as future work.
- `omny/gui_tools.py`: `OMNYGuiTools.__init__` did `self.gui.windows["main"].widget`
unconditionally — `KeyError` in any session without an open BEC-widgets "main" window
(e.g. `--nogui`). Guarded to fall back to `None`, matching flomni's `GuiTools` pattern
(`self.flomni_window`, populated lazily by `flomnigui_show_gui()`).
- Two *environment* fixes, not code bugs, but required before any of the above could even
be exercised: `csaxs_bec` and `bec_widgets` were both pip-installed editable pointing at
stale pre-repo-reorganization paths (`~/bec/csaxs_bec`, `~/bec/bec_widgets`, neither
exists) instead of their current location under `~/bec/bec/...` — `bec-server` couldn't
import its own plugins, which is what "bec-server doesn't start" actually was.
- `omny_e2e_tests/sim_flomni_harness.py`: `FakeDeviceManager` had no `.connector` —
`rt_lamni_ophyd.py`'s `feedback_enable_with_reset()` gained a
`device_manager.connector.send_client_info(...)` call (commit `f736900`) the harness
mock never picked up. Added a no-op `FakeConnector`.
- `omny_e2e_tests/sim_lamni_omny_harness.py`: imported the pre-rename module path
`csaxs_bec.scans.LamNIFermatScan` (now `lamni_components.LamNIComponents`, a
`@staticmethod`) — `ModuleNotFoundError`. Both harnesses pass cleanly again (42 flOMNI +
32 LamNI/OMNY checks) after these two fixes.
Checklist result against a live local sim deployment (`simulated_omny.yaml`,
`bec --session omny`):
- **Config load**: pass — `bec.config.update_session_with_file(...)`, `OMNY(bec)`
constructs cleanly, all `simulated_omny.yaml` devices visible on `dev.*`.
- **A move**: pass — `umv(dev.osamx, ...)`/`umv(dev.osamy, ...)`, live progress bar,
correct readback.
- **One OMNY transfer**: pass — `omny.otransfer_park_slot(6)` drove `oparkz` to the
expected `-0.9` with progress reporting and no errors.
- **One scan**: **blocked**, real bug found (not fixed — needs Mirko's confirmation of
intended hardware semantics before touching it). `osamroy` (the rotation axis, axis `C`
on the port-8083 controller) has a permanent, documented `+25`° display offset applied
only on read (`OMNYGalilReadbackSignal._socket_get`, `ogalil_ophyd.py:56-60` — "we
introduce an offset of 25 to the rotation axis... that way we just do tomography from 0
to 180 degrees"). Nothing subtracts that 25 back out on the write side
(`OMNYGalilMotor.move()`, same file) or in its own success check, so
`readback == raw + 25` is compared against the raw target passed to `.move()` — every
`.move()`/`umv()` call on `osamroy` fails its success check by exactly 25, unconditionally
(confirmed: readback stays fixed at the boot value 25.0 regardless of target; `osamx`/
`osamy` on the same controller move correctly, since only this one axis has the offset).
This isn't sim-specific — `ogalil_ophyd.py` is real, unmodified device code, so the sim
is faithfully reproducing a genuine bug. It unconditionally blocks
`feedback_enable_with_reset()`'s "rotate to zero before enabling feedback" precondition
(`rt_omny_ophyd.py:526-533`), which blocks every scan needing interferometer feedback —
i.e. effectively all of them. A correct fix needs to decide, with domain input: should
`OMNYGalilMotor.move()` convert the caller's display-space target to raw before sending
it over the wire for this axis (so `.move(0)` means "go to displayed zero"), or should
callers already pass raw targets (in which case `feedback_enable_with_reset()`'s
`osamroy.obj.move(0, ...)` and its `np.fabs(readback) > 0.1` guard are the ones passing
the wrong space)? Either reading is plausible from the code alone.
## Operational notes
@@ -41,8 +41,10 @@ support on top of it:
registry `Flomni.__init__` publishes) so its action dropdown and
per-action fields always match whatever's actually in `flomni.py` — no
hardcoded mirror. Lets you add steps in order (action + its own fields
only — a device dropdown and target for `move`, a bounded spinbox for
`optimize_idgap`'s `search_range`), remove a step, reorder steps within
only — a device dropdown and target for `move`, three bounded spinboxes
for `move_foptz_with_shift`'s `foptz_target`/`manual_shift_x`/
`manual_shift_y`, the first of which ranges over `foptz`'s live device
limits rather than a static number), remove a step, reorder steps within
the job (move up/down), and set a label + idempotent flag (defaults to
"every step idempotent" ANDed, same as the CLI, until you touch the
checkbox yourself). On confirm it produces a job dict byte-for-byte
@@ -50,7 +52,8 @@ support on top of it:
2. **Command-job-aware queue table.** `TomoQueueDialog`'s table gained a
"Details" column; a command job shows `CMD` in the Type column and its
step summary + idempotency marker in Details (e.g.
`move{'positions': {'mokev': 6.2}} > optimize_idgap{'search_range': 0.5} [idem]`
`move{'positions': {'ccm_energy': 6.2}} > move_foptz_with_shift{'foptz_target':
15.0, 'manual_shift_x': 1.0, 'manual_shift_y': -2.0} [idem]`
— literally the same string `flomni.tomo_queue_show()` prints).
3. **Self-describing tomo jobs.** "Add current params to queue" now stamps
`"kind": "tomo"` and a fresh `"id"` on the job it writes, matching what
@@ -201,19 +204,22 @@ file is exactly how a regression sneaks in.
## 4. Test plan — new: command-job builder
6. Click **"Add command…"**. Confirm the dialog opens with an **action
dropdown showing `move` and `optimize_idgap`** (or whatever's currently
in `flomni.py`'s `_TOMO_QUEUE_ACTIONS` — if you've added a third action
since this was written, it should appear automatically with no GUI
change needed; that's the point of reading the live registry).
dropdown showing `move` and `move_foptz_with_shift`** (or whatever's
currently in `flomni.py`'s `_TOMO_QUEUE_ACTIONS` — if you've added a
third action since this was written, it should appear automatically with
no GUI change needed; that's the point of reading the live registry).
7. With `move` selected, confirm the field area shows a **device dropdown**
(should list `mokev`, `idgap`, `ftray` per the current allow-list) and a
**numeric target field**. Pick a device, set a target, click **"Add
step"**. Confirm a line appears in the step list above, formatted like
`move{'positions': {'mokev': 6.2}}`.
8. Switch the action dropdown to `optimize_idgap`. Confirm the field area
now shows a single **search_range** spinbox pre-filled with `0.5`,
range-limited (try typing outside 0–2, confirm the widget clamps it
rather than accepting it). Click **"Add step"**.
(should list `ccm_energy`, `idgap` per the current allow-list -- `foptz`
is deliberately absent, see point 8) and a **numeric target field**. Pick
a device, set a target, click **"Add step"**. Confirm a line appears in
the step list above, formatted like `move{'positions': {'ccm_energy': 6.2}}`.
8. Switch the action dropdown to `move_foptz_with_shift`. Confirm the field
area now shows three spinboxes: **foptz target (absolute)**,
**New manual_shift_x**, **New manual_shift_y**. Confirm
the foptz-target spinbox's range reflects `foptz`'s *live* device limits
(try typing a value past the real limit, confirm the widget clamps it
rather than accepting it -- this is the live-limit lookup, not a static
number). Fill in all three and click **"Add step"**.
9. Confirm the **idempotent checkbox auto-checked itself** after both adds
(both current actions default to idempotent=True). Uncheck it by hand,
then add another step — confirm it **stays unchecked** (your override
@@ -250,9 +256,10 @@ file is exactly how a regression sneaks in.
## 6. Test plan — mixed queue, end to end
17. Build a small realistic sequence via the GUI only: a tomo job ("Add
current params to queue"), then a command job (e.g. a `move` on
`ftray` if you're on the sim — `mokev`/`idgap` only exist on the real
front-end, see plan section 3.1), then another tomo job.
current params to queue"), then a command job (e.g. a `move_foptz_with_shift`
step if you're on the sim — `ccm_energy`/`idgap` only exist on the real
front-end, `foptz` is the only queue-movable device in the sim config),
then another tomo job.
18. From the CLI, run `flomni.tomo_queue_execute()`. Confirm: the tomo job
runs, the command job actually moves the device (check
`dev.<device>.readback.get()` before/after), and the second tomo job
@@ -405,8 +412,12 @@ GUI bug and reporting gaps for the hook feature.
42. Queue a `done` tomo job (run one, or add one via CLI with `status`
hand-edited to `done`). Select it in the queue dialog and confirm
**"Load into editor"** is enabled. Select a command job (or multiple
rows, or nothing) and confirm it's **disabled** in each case.
**"Load into editor"** is enabled. Select multiple rows (or nothing)
and confirm it's **disabled**. Select a *pending* command job and
confirm the button text instead switches to **"Edit command"**
(enabled) — see 6f below for that path; select a `running`/
`incomplete`/`done` command job and confirm it's **disabled** (text
reverts to "Load into editor").
43. Click **"Load into editor"** with nothing else in progress. Confirm the
params panel enters edit mode (Edit button hides, Submit/"Add to
queue"/Cancel appear) with every field populated from the *job's*
@@ -481,6 +492,28 @@ buttons living on two different windows.
to queue" — confirm it queues normally with **no** warning (only
fires when an edit is actually in progress).
## 6g. Test plan — new: "Edit command" (same button slot as "Load into editor")
48. Add a multi-step command job via **"Add command…"** (e.g. a `move` step
plus a `move_foptz_with_shift` step), with a custom label and the
idempotent checkbox left at its default. Select it in the queue table
and confirm the button reads **"Edit command"**.
49. Click it. Confirm `CommandJobBuilderDialog` opens titled **"Edit
command job"**, with the step list showing both steps exactly as the
queue table's Details column renders them, the label field pre-filled
with your custom label, and the idempotent checkbox matching what you
set when adding it.
50. Click **Cancel**. Confirm the queue is completely unchanged (same job,
same position, same `id` — check via `flomni.tomo_queue_show()` or
`flomni._tomo_queue_proxy.as_list()` before/after).
51. Reopen it, remove a step, add a different one, change the label, click
**OK**. Confirm the row updates **in place** (same position in the
table, same `id` via the CLI check above) with the new steps/label —
no new row is appended, no old row left behind.
52. Confirm the button is **disabled** for a `running`/`incomplete`/`done`
command job (only `pending` is editable), and also disabled while
**Sort queue…** mode is active.
## 7. Known limitations of this first iteration (by design, not bugs)
Worth knowing before you file something as broken:
@@ -501,13 +534,16 @@ Worth knowing before you file something as broken:
(`tomo_queue_add_command` with a `positions` dict of more than one
device); the GUI builder just doesn't expose it yet. Add extra `move`
steps instead if you need to move more than one device.
- **No live device-limit validation.** The plan (section 6.5) mentions
validating against `dev.<name>.limits` in addition to the schema's
min/max; this iteration only enforces the schema bounds (via the spinbox
range), not real hardware limits. The execution-time re-check in
`_queue_action_move()` is still the actual safety boundary regardless
(two-layer validation, plan section 3.3), so this is a UX gap, not a
safety gap.
- **Resolved, for fields that opt in: live device-limit validation.** A
float param schema can now declare `"device_limits_from": "<device>"`
(see `move_foptz_with_shift`'s `foptz_target`), which makes both the GUI
spinbox's range and the CLI's add-time check (`_validate_action_kwargs`)
use `dev.<name>.limits` instead of the schema's static `min`/`max`. The
plain `move` action's devices (`ccm_energy`/`idgap`) don't use this --
still just the schema bounds via the spinbox range there. The
execution-time re-check in `_queue_action_move()`/
`_queue_action_move_foptz_with_shift()` remains the actual safety
boundary regardless (two-layer validation, plan section 3.3).
- **Resolved: `Submit` is now a hard block while the beamline is busy**,
matching plan section 6.1 literally instead of the earlier soft-warn
confirmation dialog. The gap that made the soft warn tempting in the first
@@ -264,8 +264,8 @@ interruptible mid-scan for the crash-resume tests (§4, B5).
10. **✅ Add-time validation rejects bad steps, and rejects them without landing
anything in the queue.** `test_add_command_validation_errors`. Unknown action,
a device not on the `move` allow-list, an out-of-range/wrong-type
`optimize_idgap.search_range`, and an unknown kwarg are each rejected with the
expected `ValueError`; afterward the queue is confirmed still empty.
`move_foptz_with_shift.foptz_target`, and an unknown kwarg are each rejected
with the expected `ValueError`; afterward the queue is confirmed still empty.
11. **✅ Execution-time allow-list re-check is the real safety boundary.**
`test_move_execution_time_allowlist_recheck`. A job with a disallowed device
appended directly (bypassing `tomo_queue_add_command`) is refused by
@@ -278,10 +278,13 @@ interruptible mid-scan for the crash-resume tests (§4, B5).
`test_show_and_webpage_survive_command_job`. A command job (no `params` key)
renders its step summary + idempotency marker in both, instead of `KeyError`ing.
14. **✅ Back-compat + real dispatch, combined.**
`test_legacy_job_no_kind_defaults_to_tomo_and_command_moves_ftray`. A `kind`-less
`test_legacy_job_no_kind_defaults_to_tomo_and_command_moves_foptz`. A `kind`-less
legacy queue entry still runs as a tomogram; in the same mixed-queue run, a
command job actually moves a device (`ftray`, the sim-testable stand-in for
`mokev`/`idgap` — see the plan doc §3.1) via the real `move` action path.
command job actually moves `foptz` and sets new `manual_shift_x`/
`manual_shift_y` values via the real `move_foptz_with_shift` action
path (the only sim-movable path now that `ftray` is no longer in the move
allow-list; `foptz` is also the real device this action exists for, unlike
`ftray`'s old sim-only stand-in role).
15. **✅ Non-idempotent resume prompts exactly once, and respects "no".**
`test_non_idempotent_command_resume_prompts`. A command job flagged
`idempotent=False`, left `incomplete`, triggers the resume prompt (distinct from
@@ -294,13 +297,14 @@ interruptible mid-scan for the crash-resume tests (§4, B5).
16. One queued tomogram end-to-end on the real instrument, to confirm the sim's
timing/progress semantics match reality closely enough that resume lands on the
right angle.
17. `move` on `mokev`/`idgap` end-to-end — neither device is in the flomni sim
config (see the plan doc §3.1), so Step 2's `move` action itself has only been
sim-exercised via `ftray`, never against the real devices it's actually for.
17. `move` on `ccm_energy`/`idgap` end-to-end — neither device is in the flomni
sim config, so Step 2's `move` action itself has only been sim-exercised via
add-time validation (which doesn't require real movement), never a real
execute() against these two devices.
**Gate:** A–D passed on the sim (16/16 tests total). **Steps 1 and 2 are verified
enough to build Step 3 (the GUI) on.** E remains open, and blocks calling `move` on
`mokev`/`idgap` production-ready.
**Gate:** A–D passed on the sim. **Steps 1 and 2 are verified enough to build
Step 3 (the GUI) on.** E remains open, and blocks calling `move` on
`ccm_energy`/`idgap` production-ready.
### A gotcha found testing Step 2 (not a Step-2 bug)
@@ -348,11 +352,14 @@ still just its design.
- **Registry entry = 4 things:** bound method name, `idempotent` default, help string,
and a **params schema** (possibly empty).
- **Result-dependent logic lives INSIDE a named action, never across queue steps.**
E.g. "scan idgap, move to peak" is ONE action (`optimize_idgap`) whose method does
the scan, finds the peak, and moves — *not* three queue steps. The queue therefore
never has to carry runtime values between steps; `kwargs` stays plain JSON. A
compound action may take parameters (e.g. `search_range`), declared in its params
schema like any other.
E.g. a hypothetical "scan idgap, move to peak" action would be ONE action whose
method does the scan, finds the peak, and moves — *not* three queue steps. The
queue therefore never has to carry runtime values between steps; `kwargs` stays
plain JSON. The same "bundle, don't split" idea also applies to a compound action
with no runtime-dependent result but a correctness-critical pairing of writes --
`move_foptz_with_shift` (moves `foptz` + sets new `manual_shift_x`/
`manual_shift_y` values atomically, deliberately not reachable as two
separate steps) is the concrete example currently in the registry.
- **Publish the registry via a global var** (`tomo_queue_actions`) on `Flomni.__init__`
so the GUI builds its dropdowns/forms from it — no hardcoded mirror to drift (the
`QUEUE_PARAM_NAMES` mirror between `flomni.py` and `tomo_params.py` already has this
@@ -399,9 +406,10 @@ Guarded against command jobs (`job.get("kind", "tomo") == "tomo"` before touchin
## 6. Open questions for Mirko — all resolved
- **The final action set and device allow-list**, see `TOMO_QUEUE_COMMAND_JOBS_PLAN.md`
§10 — `move` + `optimize_idgap` only, `_TOMO_QUEUE_MOVE_DEVICES = {mokev, idgap}`.
One detail (`optimize_idgap`'s peak-detection signal) deferred to the
implementation session, not blocking.
§10 — originally `move` + `optimize_idgap`, `_TOMO_QUEUE_MOVE_DEVICES = {mokev,
idgap}`. Superseded since: `optimize_idgap` was removed (never commissioned, stayed
a no-op stub), `mokev` was fixed to the real device name `ccm_energy`, and a new
`move_foptz_with_shift` action was added (see §4D10/§4D14 above for its tests).
- **`_TOMO_QUEUE_PARAM_NAMES` renamed to `_TOMO_SCAN_PARAM_NAMES`** (it reads live
properties, not queue state — the old name was misleading). Done in `flomni.py`;
the `tomo_params.py` comment referencing it by name was updated too (the
@@ -198,6 +198,13 @@ cannot drift.
- **Consider**: the PSI **EPICS archiver REST interface** to sidestep CA firewall
concerns.
### Interferometer signal-strength trend (proposed, not implemented)
- See `OMNY_shared/AI_docs/TODO_interferometer_signal_trend.md` for the full
design note (2026-09-19) — a new webpage section trending interferometer
signal strength over time, sampled at ~1 point/hour or less (not
per-projection). Blocked on omny specifically needing a real
`OmnyWebpageGenerator` built first (its current file is dead/unwired code).
### Deferred
- **LamNI**: fill in `LamniWebpageGenerator` — real `tomo_type` key values in
`TOMO_TYPES`, device paths / `_collect_setup_data()`, `LamNI.png`. Add a
@@ -0,0 +1,204 @@
# How to calibrate the FZP scanner of flOMNI
Source of `local_contact_only-howto_calibrate_the_fzp_scanner.pdf`. The original text is kept unchanged; the
section "Update 2026-10-20" at the end lists what the present flOMNI loop (PREEMPT_RT, 100 µs, branch `rt_preemt`)
needs in addition (corrected 2026-10-23: "move-ahead" → **move-beyond**, see the section at the end).
```text
This is a description of how to calibrate the FZP scanner of flOMNI
Rmemeber to do all modifications for both directions x and y!
1. Align flOMNI without the FZP scanner. This implies disabling the
FZP scanner completely. To do so, in OrchestraHMI edit the parameters
of the FZP scanner as follows.
(a) Module PID_i_FZP: Set Param FAST_FZP_PID_LOOP(0) to 2. Then the FZP
will just remain
statically centered to the capacitive sensor position in closed loop.
(b) Module PID_i: Set Param FAST_FZP_PID_LOOP(0) to 0. If this is
negative, the FZP scanner will move ahead in a point-by-point scan by
that amount.
(c) Module PID_i: Set Param FAST_FZP_PID_LOOP(5) to 0. If this is
larger than zero the sample stage motion is stopped if the position is
reached by the FZP scanner (which is the case when the scanner is not
active). Zero means thus full compensation by the sample stage.
In this state the setup can be aligned, and the affine matrix for
ptychography can be found. This refers then purely to the alignment of
the interferometer and X-ray detector.
2. In the next step we perform a scan (with a small beam and small
range) such that only the FZP scanner is used and the sample stage
will not move.
Before we start ensure that the Correction Matrix for the capacitive
sensors is cleared.
To do this change the parameters of the flOMNI control
loop as follows:
(a) Module AnalogInput_6259: Set parameter MatrixCorr to 1 0 0 1
(b) Module PID_j_fzp: Set Param FAST_FZP_PID_LOOP(0) to 1 to re-activate
the FZP scanner.
(c) Module PID_i: Set Param FAST_FZP_PID_LOOP(5) to 10, so up to 10
volts output of the fast PID is accepted before starting a movement of
the sample stage.
(d) Module PID_i: Param FAST_FZP_PID_LOOP(4) 0
(e) Module PID_i: Param FAST_FZP_PID_LOOP(6) 1
with d and e it will now stabilize the sample stage position at zero
(f) Module PID_i: Param FAST_FZP_PID_LOOP(0) remains 0 (no move ahead
in a scan)
With these settings perform a scan. Use a field of view matching the
range of the FZP scanner. Watch the sample stage position in Orchestra
to ensure it does not change!
A good range should be 2.8 microns in x and 1.8 microns in y.
Reconstruct this scan with Michal's position refinement code and ask
for an update of the FZP matrix. This can then be put into the
AnalogInput_6259 module.
I guess one has to have the matrix found for the interferometer in the
reconsturction code, such that the new matrix of the FZP scanner will
adjust the coordinate system of the FZP scanner to the system of the
interferometer. Later these positions are used together and should
match in direction.
3. Back to default operating values:
(a) Module PID_i: Set Param FAST_FZP_PID_LOOP(0) to -0.5, for up to
0.5 microns move ahead in a scan.
All other parameters are already as supposed to be.
Default values for the sample stage pid, fast FZP parameter are "-
0.5,0,0,0,0.1,0.5,0"
Default values for the fzp pid, fast FZP parameter are "1,0.0001,0.1,-
4,0,0,0"
```
Naming: "move ahead" in the text above (sample PIDs `FAST_FZP_PID_LOOP(0)` < 0) is called **move-beyond** from
2026-10-23 on: the sample stage moves *beyond* the scan point towards the next one, the FZP compensates and so sits
beyond its centre during the exposure, on the side that gives it more stroke for the next step. It is not a
"pre-move" of the FZP to the next point (that is `TargetSelect` `use_final` 1, see below).
## Update 2026-10-20 (flOMNI loop on PREEMPT_RT, mpc2844)
The loop has new modules since the text above was written (flOMNI `FeedbackLoop/tuning_guide.md`,
`feedback_report.md` 5.6, 6n, 6o, 6q). Parameters are changed in the web HMI (`http://mpc2844:8080`, tab
Parameters) instead of OrchestraHMI. `i` = x or y, as above. Module names now: `PID_x`, `PID_y` (sample stage),
`PID_x_fzp`, `PID_y_fzp` (FZP scanner), `AnalogInput_6289` (was `AnalogInput_6259`).
**FAST_FZP_PID_LOOP has 7 elements** (`PID.cpp`). Sample stage PIDs: (0) move-beyond (negative, µm, point-by-point
scans only; section "Move-beyond" below), (4) FZP error threshold (µm) and (5) FZP drive limit (V) for holding the sample stage while the FZP
takes the step (5 = 0: never held), (6) 1 = hold the sample stage at 0. FZP PIDs: (0) mode (1 operation, 2 FZP
centred on its cap sensor = disabled, 3 not commissioned, 4 debug), (1)–(3) min / max / gain of the old velocity
correction (−4 before 2026-10-19, now 0: replaced by the feedforward).
**In addition to step 1 (FZP disabled):**
- `Feedforward`: `enable` 0,0 — the static FZP feedforward adds its drive also in FZP mode 2.
- `AbbeCorrection`: `enable` 0 (default until the signs are known).
- `TargetSelect_x`, `TargetSelect_y`: `use_final` 0; `OnTargetTrigger`: `WaitForSlew` 1 (defaults).
- `FreqGen`: `enable` 0 (excitation for measurements; default).
**In addition to step 2 (FZP-only calibration scan):**
- FZP range: `PID_x_fzp` / `PID_y_fzp` `Ub` / `Lb` are ±1.5 V in operation (the cap sensors are calibrated over
about ±2 V only). For the field of view above (2.8 × 1.8 µm) widen them, e.g. to ±10 V (FZP ≈ 0.13 µm/V in x,
≈ 0.05 µm/V in y). The feedforward stays off (`enable` 0,0) during this scan.
- The detector trigger (`OnTargetTrigger`) checks the relative error sample vs FZP (parameters
`ontarget_tolerance_um` / `ontarget_hold_ms`, BEC `flomni.realtime_system.trigger_*`, default 10 nm / 3 ms; until
2026-10-23 ConstEmitter lines) and waits `MinDelay_ms` (15 ms).
**Step 3, default operating values (replace the values in the text above):**
| Module | Parameter | Value |
|---|---|---|
| `PID_x`, `PID_y` | `FAST_FZP_PID_LOOP` | `0,0,0,0,0.1,0,0` (no move-beyond: −0.5 µm is larger than the ±1.5 V FZP stroke, x ≈ ±0.19 µm, y ≈ ±0.08 µm) |
| `PID_x_fzp` | `FAST_FZP_PID_LOOP`, `Ub` / `Lb` | `1,0.0001,0.1,0,0,0,0`, ±1.5 V |
| `PID_y_fzp` | `FAST_FZP_PID_LOOP`, `Ub` / `Lb` | `1,0.0005,0.02,0,0,0,0`, ±1.5 V |
| `Feedforward` | `enable`; `ff_limit`, `out_min` / `out_max` | 1,1; ±1.5 V (as the FZP `Ub` / `Lb`) |
| `OnTargetTrigger` | `WaitForSlew`, `MinDelay_ms` | 1, 15 |
| `TargetSelect_x`, `TargetSelect_y` | `use_final` | 0 |
| `AbbeCorrection` | `enable` | 0 (until the signs are known) |
| `AnalogInput_6289` | `MatrixCorr` | the new matrix from the calibration |
Check afterwards: web HMI tab **Loop check**, level 1 (it expects these values and lists live values that differ
from the `.xmp` files); save the new values as default (Parameters → Unsaved changes) or in the `.xmp`.
**After the calibration (per experiment): faster scans, exposure during the motion.** With the FZP calibrated
over a wider range the FZP can take the small steps alone (flOMNI `feedback_report.md` 6q, tested at ±7 V):
- `PID_x_fzp` / `PID_y_fzp` `Ub` / `Lb` and the `Feedforward` limits to the calibrated range (e.g. ±7 V);
- `TargetSelect_x` / `_y` `use_final` 1 (the FZP aims at the new point at once, the sample stage follows its
S-curve) and `OnTargetTrigger` `WaitForSlew` 0 (trigger on the relative error only); optionally `MinDelay_ms` 0;
- optionally move-beyond, sized to the FZP stroke (≈ half of it, see below): at ±7 V (x ≈ ±0.9 µm, y ≈ ±0.36 µm)
`PID_x` (0) e.g. −0.5, `PID_y` (0) e.g. −0.2 (the values tested in `feedback_report.md` 6q).
Measured with 101 ms exposures: 0.2–1 µm steps trigger after 4.5–5.8 ms instead of 17–32 ms; the error during the
exposure grows with the step (1 µm: ≈ 6 nm max instead of 2.5 nm), so the setting depends on the experiment.
## With the loop settings (since 2026-10-23)
The steps above are profiles of the loop settings (web HMI tab *Loop settings*, BEC `flomni.realtime_system`;
flOMNI `FeedbackLoop/loopsettings/`). The rules keep the modules consistent, so the individual parameters above do
not have to be set by hand:
| Step | Profile | BEC |
|---|---|---|
| 1, FZP disabled (alignment, affine matrix) | `fzp_off` | `flomni.realtime_system.apply_profile("fzp_off")` |
| 2, FZP-only calibration scan (clear `MatrixCorr` first: not part of the settings) | `fzp_calibration` | `apply_profile("fzp_calibration")` |
| 3, back to operation | `production` | `apply_profile("production")` |
| afterwards, fast scans with the FZP at ±9 V (scan modes, flOMNI `FeedbackLoop/scan_modes.md`) | `high_res`, `normal`, `low_res` | `apply_profile("high_res")` |
Single values: `flomni.realtime_system.set(fzp_range_V_x=7, fzp_range_V_y=7)`,
`set(move_beyond_um_x=0.3)`, `set(exposure_during_motion=1)`; refused with the reason if a rule objects
(`adjust=True` changes the dependent settings too). FZP settings only between scans. Live only until
`save_as_default()`.
## Move-beyond (`PID_x` / `PID_y` `FAST_FZP_PID_LOOP(0)` < 0), 2026-10-23
**What it does** (`PID.cpp`, unchanged since the original loop). Only in point-by-point scans (scan state 2). The
TrajectoryGenerator gives the next scan point (outputs 5 / 6, PID input 9). The sample PID then aims not at the
target T but at T + m · sign(next − T), m = min(|FAST_FZP_PID_LOOP(0)|, |next − T|): the sample stage moves *beyond*
the point towards the next one, by at most the step. The FZP loop keeps its own target T (it does not see m), so the
FZP compensates the offset: during the exposure it holds the relative position at T while sitting at +m off its
centre. At the next point the FZP then has stroke R + m in the step direction instead of R (R = stroke from the centre),
so larger steps are taken by the FZP alone. Steps smaller than |FAST_FZP_PID_LOOP(0)|: the sample stage is one point
ahead, the FZP moves between 0 and +step only. The offset is not shaped (InputSlew acts on T only); the sample stage
moves to it with its own (slow) loop, i.e. partly during the exposure, and the FZP compensates that motion.
**Is it still the same in the new loop?** Yes. The relative position is computed as before:
FZP error = Cap − (Chan − target) (mode 1), `Position sample vs FZP` = Chan − Cap (PID output 3, RT_sampler). The
sample PIDs still get the shaped target (InputSlew) + move-beyond and the raw interferometer. The FZP side now has
two additions that act on the FZP side only: `TargetSelect` (`use_final` 0 = shaped target as before, 1 = final
target) and `AbbeCorrection` (Chan corrected for the stage tilt, see below). The `Feedforward` uses the same FZP
target and Chan, so it supplies the +m offset of the FZP directly (static term m / K).
**Sizing.** During the exposure the FZP has to hold m + Abbe correction + noise / settling inside its stroke:
|m| ≤ R − reserve, as a rule ≈ R / 2. R per axis = `Ub` × K (K ≈ 0.13 µm/V x, ≈ 0.05 µm/V y). The loop settings
refuse more than 0.6 × R minus the Abbe limit (rule `move_beyond_*_fits`, constant `move_beyond_fraction`; the
tested ±7 V values are 0.56 × R):
| FZP `Ub`/`Lb` | R x / y | move-beyond x / y (≈ R/2) |
|---|---|---|
| ±1.5 V (production) | ±0.19 / ±0.08 µm | ≈ −0.1 / ≈ −0.04 µm (production: 0) |
| ±7 V (after calibration) | ±0.9 / ±0.36 µm | −0.5 / −0.2 µm (tested, 6q) |
The original default −0.5 µm in both axes dates from the ±9 V limits of the old loop (x ≈ ±1.2 µm, about R/2; in y
±9 V ≈ ±0.45 µm, i.e. about the full stroke). When the FZP saturates while holding the offset, the relative error grows and the trigger
waits.
**Abbe correction.** `AbbeCorrection` adds a new term to the measured sample x / y on the FZP side only:
x = Chan2_x + θz · offset_y_for_x, y = Chan1_y + θz · offset_x_for_y (θz = Chan3 / 13 mm). So yes: it is an error of
the sample x / y position (stage tilt about z, not seen by the x / y interferometer beams) that is corrected by the
FZP x / y, not by the sample stage (`PID_x` / `PID_y` keep the raw Chan, the rotZ piezo is too slow). The positions
recorded for the reconstruction (`Position sample vs FZP`) contain it as well. It uses FZP stroke (limited to
`correction_limit_um` 0.1 µm per axis; y at rest ≈ 2 nm, ≈ 30–37 nm during 20 µm x steps) — keep this in the reserve
of the move-beyond, in y it is a large part of the ±0.08 µm stroke. Keep it off (`enable` 0) during the FZP
calibration scan (step 2).
**Notes.**
- Hold of the sample stage, (4) / (5): input 8 of `PID_x` / `PID_y` was the FZP PID output *before* the
`Feedforward` (with the feedforward on only the remainder, not the real FZP drive). Since 2026-10-23 it is the
`Feedforward` output (FZP drive incl. feedforward), so (5) again compares the real FZP drive. No change with the
feedforward off or with (5) = 0 (production).
- On the last point of a scan the next point output is 0, so the move-beyond points towards 0 (as in the old loop).
- `TargetSelect` `use_final` 1 is a different thing: the FZP aims at the new point at once while the sample stage
follows the S-curve. Both can be combined (6q, E0).
@@ -0,0 +1,46 @@
# How to limit the FZP scanner range of flOMNI
Source of `local_contact_only-howto_limit_the_fzp_scanner_range.pdf`. The original text is kept unchanged; the
section "Update 2026-10-20" lists what the present flOMNI loop (PREEMPT_RT, 100 µs, branch `rt_preemt`) needs in
addition. "ahead movement" (element 0) is called **move-beyond** from 2026-10-23 on: the sample stage moves beyond the
scan point towards the next one, the FZP compensates and sits off-centre, which gives it stroke for the next step.
```text
to limit the fzp scanner ranges
in PID_ FAST FZP PID LOOP
0: reduce to small ahead movement range, such as -0.05
4: reduce error threshold, eg 0.01
5: inform PID that FZP range is small, such as 0.1
in PID_ _FZP
limit the analog output LB and UB
```
## Update 2026-10-20 (flOMNI loop on PREEMPT_RT, mpc2844)
The limited range is the present operating state: `PID_x_fzp` / `PID_y_fzp` `Ub` / `Lb` = ±1.5 V because the cap
sensors are calibrated over about ±2 V only (flOMNI `feedback_report.md` 6n). FZP ≈ 0.13 µm/V in x, ≈ 0.05 µm/V in y,
so ±1.5 V ≈ ±0.19 µm (x) / ±0.08 µm (y). In the sample stage PIDs (`PID_x`, `PID_y`) `FAST_FZP_PID_LOOP` is
`0,0,0,0,0.1,0,0`: no move-beyond, (5) = 0 (the sample stage always compensates fully).
Changing the FZP range now involves, in addition to the text above:
- **`Feedforward`** (static FZP feedforward): `ff_limit`, `out_min` / `out_max` to the same values as the FZP `Ub` /
`Lb` (they limit drive + feedforward).
- **`FreqGen`** `out_min` / `out_max` (channels 0 / 1) limit the FZP drive only while an excitation is running
(measurements).
- **`OnTargetTrigger`** `WaitForSlew` and **`TargetSelect_x` / `_y`** `use_final`: keep the defaults (1 / 0) with a
small range. `use_final` 1 + `WaitForSlew` 0 ("exposure during the motion") only makes sense when the FZP range
covers the scan step; steps larger than the range then still wait for the sample stage.
- Move-beyond (0) must fit into the stroke: the FZP holds it during the exposure, together with the Abbe correction
(up to `correction_limit_um` 0.1 µm when enabled) and noise. Rule ≈ half the stroke (the loop settings refuse more than 0.6 × stroke minus the Abbe limit): at ±1.5 V ≈ −0.1 µm (x),
≈ −0.04 µm (y); at ±7 V −0.5 / −0.2 µm (tested). Production: 0. Details: calibration how-to, section "Move-beyond".
- (5) compares the real FZP drive incl. the feedforward (input 8 of `PID_x` / `PID_y` = `Feedforward` output since
2026-10-23; before it was the FZP PID output without the feedforward).
- After a change: web HMI tab **Loop check** level 1 (expected settings) and, with the beamline ready, the scan test
(level 4: positioning time, stability during the exposure per step size).
**With the loop settings (since 2026-10-23):** `flomni.realtime_system.set(fzp_range_V_x=..., fzp_range_V_y=...)`
(web HMI tab *Loop settings*) sets `PID_*_fzp` `Ub` / `Lb` and the `Feedforward` limits together; a move-beyond that
no longer fits is refused, or clamped with `adjust=True` (rule `move_beyond_*_fits`). `FreqGen` limits stay
manual (measurements only).
@@ -0,0 +1,81 @@
# flOMNI real-time loop settings (beamline scientists / local contact only)
**Not for users.** `flomni.realtime_system` changes how the FZP scanner, the detector trigger and the laser tracker
work, and `save_as_default()` makes a change the default of every following loop start. A wrong combination can spoil
the data of a beamtime. The user guide only says that these settings exist, are set by the local contact and are
stored with every scan; the details are here.
Background and reference (flOMNI control repository, `OMNY/flOMNI` on gitea):
- `FeedbackLoop/loopsettings/README.md` — every setting with the loop parameters it sets and its production value,
the rules, the profiles; `FeedbackLoop/loopsettings/flomni.json` — the definition (help texts, rules, profiles)
- `FeedbackLoop/tuning_guide.md`, `FeedbackLoop/feedback_report.md` — characterization, measurements, why the values
- RT-system `docs/loop-settings.md` — design, the web HMI side, installation
- the FZP how-tos next to this file (`local_contact_only-howto_calibrate_the_fzp_scanner.md`,
`local_contact_only-howto_limit_the_fzp_scanner_range.md`) — move-beyond, FZP range, calibration steps as profiles
## What can be set
| Group | Settings |
|---|---|
| FZP scanner (only between scans) | `fzp_enable` (both axes or none), `fzp_range_V_x` / `_y`, `feedforward_enable`, `move_beyond_um_x` / `_y`, `exposure_during_motion`, `abbe_enable`, `abbe_limit_um` |
| Detector trigger | `trigger_tolerance_nm`, `trigger_hold_ms`, `trigger_min_delay_ms`, `trigger_on_move` |
| Laser tracker and thresholds | `tracking_target_y` / `_z`, `psd_threshold_low`, `interferometer_threshold`, `interferometer_threshold_rot` |
| FZP calibration (expert, only between scans) | `sample_hold_fzp_V`, `sample_hold_err_um`, `sample_hold_zero` |
`flomni.realtime_system.help("all")` explains every setting, `help("<setting>")` one setting (what it does, range,
live and saved value, the loop parameters it sets, its rules).
## Commands
- `flomni.realtime_system.show()` — all settings with live and saved value, violated rules, scan state
(`show(expert=True)` also the FZP calibration group)
- `flomni.realtime_system.set(name=value, ...)` — change; refused with the reason if a rule objects (e.g. the
feedforward must be off while the FZP loop is off); `set(..., adjust=True)` changes the dependent settings the
rule names (never the ones given)
- `flomni.realtime_system.check(name=value)` — dry run
- `flomni.realtime_system.profiles()`, `apply_profile("production" | "fzp_off" | "fzp_calibration" | "high_res" | "high_res_1v5" | "normal" | "low_res")` (scan modes 2026-10-26: flOMNI `FeedbackLoop/scan_modes.md`)
- `flomni.realtime_system.unsaved()` — what differs from the loop's parameter files
- `flomni.realtime_system.save_as_default()` — writes the live values into the loop's parameter files: the next
loop start uses them. Only for a change that should stay (a backup of each file is kept on the loop machine).
- `flomni.realtime_system.help()`, `help("all")`, `help("<setting>")`
Changes are live only until the next start of the control loop, unless saved. FZP settings can only be changed
between scans (refused while a scan runs); trigger and tracker settings any time.
### Detector trigger
In step scans the detector is triggered when the relative position sample vs FZP (interferometer minus FZP cap
sensor) stays within the tolerance in x and in y (each averaged over 0.5 ms) for the hold time.
- `flomni.realtime_system.trigger_show()` — tolerance, hold time, minimum delay, trigger after a move
- `flomni.realtime_system.trigger_set_tolerance_nm(10)` — about 30 for a fast low-resolution scan, 10 standard,
2 for the highest resolution (a few ms more per point); 0 = the former criterion (on-target flags of the FZP loops)
- `flomni.realtime_system.trigger_set_hold_ms(3)` — hold time in ms
- `flomni.realtime_system.trigger_set_after_move(True)` — also one trigger after every position change of the
sample stage (e.g. `umv(dev.rtx, ...)`, `dev.rtx.controller.move_xyz(x, y, z)`) once on target; `False` (default):
scan points only
The former `flomni.interferometer_trigger_*` were removed 2026-10-23.
### Laser tracker
`flomni.laser_parameters_show_all()`, `laser_parameters_set_targety(v)` / `_targetz(v)`,
`laser_parameters_set_intensity_threshold_laser(v)`, `laser_parameters_set_psd_intensity_threshold_tracking_low(v)`
and the arrow-key tweak `laser_parameters_tweak_tracking()` use the same settings (`tracking_target_y` / `_z`,
`interferometer_threshold`, `psd_threshold_low`).
## Stored with every scan
The baseline device `flomni_loop_settings` is read at the start of every scan: one value per setting,
`available` (0 when the loop machine could not be reached — the scan continues), the number of settings not saved
as default or inconsistent, violated error rules, the definition with its sha1, and a JSON of all settings.
`dev.flomni_loop_settings.show()` prints what a scan would store now.
## Connection
`flomni.realtime_system` sends its requests through the device `flomni_loop_settings` (device server). The device
talks to the web HMI on the loop machine (`http://mpc2844:8080`, REST `/api/settings`; port 80 as fallback) with the
API token in its device config (`ptycho_flomni.yaml`, `token:`; the line of `~/loop-data/webhmi/api_token` on
mpc2844). Without that device (e.g. a simulated config) the client talks to the web HMI directly with
`~/.flomni_webhmi_token`. The web HMI tab *Loop settings* shows and changes the same settings with the same rules.
File diff suppressed because it is too large. Load diff
@@ -9,6 +9,22 @@ class FlomniOpticsError(Exception):
pass
def compute_osa_remaining_space(foptz_val: float, fosaz_val: float) -> float:
"""Clearance (mm) between the OSA and a normal OMNY pin for a given
foptz/fosaz pair. <=0 means collision.
Module-level (not a method) so it can be imported and reused outside a
live Flomni session -- e.g. by the tomo-queue GUI (tomo_params.py),
which only has RPC device access (dev.foptz.readback.get()/
dev.fosaz.readback.get()), not a Flomni instance to call a bound method
on. FlomniOpticsMixin's own methods below are thin wrappers around this
for the two directions actually used in a live session (varying fosaz
at the current foptz, and vice versa) -- single source of truth for the
formula/constant either way.
"""
return 33 - foptz_val - fosaz_val
class FlomniOpticsMixin:
@staticmethod
def _get_user_param_safe(device, var):
@@ -21,19 +37,63 @@ class FlomniOpticsMixin:
"""Clearance (mm) between the OSA and a normal OMNY pin if fosaz were
at `fosaz_target`, at the *current* foptz reading. <=0 means collision."""
foptz_val = dev.foptz.readback.get()
return -fosaz_target + (33 - foptz_val)
return compute_osa_remaining_space(foptz_val, fosaz_target)
def _fheater_is_up(self) -> bool:
axis_id = dev.fheater._config["deviceConfig"].get("axis_Id")
low, _high = dev.fheater.controller.get_motor_limit_switch(axis_id)
return bool(low)
def _fheater_is_down(self) -> bool:
fheater_in = dev.fheater.user_parameter.get("in")
if fheater_in is None:
return False
return np.isclose(dev.fheater.readback.get(), fheater_in, atol=0.1)
def feye_out(self):
dev.fsh.fshclose()
if self.csaxs is not None:
self.csaxs.fil_trans(1, print_only=False)
# sl1xs_before = self._sl1xs_before_eye
# sl1ys_before = self._sl1ys_before_eye
# if sl1xs_before is not None and sl1ys_before is not None:
# # if the slit was manually re-adjusted while the eye was in,
# # offer to keep the current size instead of restoring the
# # pre-eye value
# current_sl1xs = dev.sl1xs.user_readback.get()
# current_sl1ys = dev.sl1ys.user_readback.get()
# expected_sl1xs = sl1xs_before + 0.3
# expected_sl1ys = sl1ys_before + 0.3
# tol = 0.01 # mm (10 microns)
# manually_adjusted = not np.isclose(
# current_sl1xs, expected_sl1xs, atol=tol
# ) or not np.isclose(current_sl1ys, expected_sl1ys, atol=tol)
# if manually_adjusted and self.OMNYTools.yesno(
# "sl1xs/sl1ys appear to have been adjusted manually while the "
# "eye was in. Keep the current slit size instead of "
# "restoring the pre-eye value?"
# ):
# print("Keeping current slit size, not restoring pre-eye value.")
# else:
# umv(dev.sl1xs, sl1xs_before, dev.sl1ys, sl1ys_before)
# self._sl1xs_before_eye = None
# self._sl1ys_before_eye = None
self.foptics_in()
self.flomnigui_show_xeyealign()
self.xrayeye_update_frame()
if self.OMNYTools.yesno("Did the direct beam on the xray eye disappear?"):
print("excellent.")
else:
print("Aborting. With visible parts of the direct beam on the xray eye, it cannot be removed.")
return
# raise, not return: callers (tomo_alignment_scan) went on measuring with the eye in
raise FlomniOpticsError(
"Aborting. With visible parts of the direct beam on the xray eye, it cannot be removed."
)
feyex_out = self._get_user_param_safe("feyex", "out")
umv(dev.feyex, feyex_out)
# move rotation stage to zero to avoid problems with wires
@@ -46,8 +106,7 @@ class FlomniOpticsMixin:
"this could cause collisions when operating on the real "
"beamline."
):
print("Aborting feye_out(): fttrx1 not moved.")
return
raise FlomniOpticsError("Aborting feye_out(): fttrx1 not moved.")
else:
fttrx_in = self._get_user_param_safe("feyex", "fttrx_in")
umv(dev.fttrx1, fttrx_in)
@@ -61,8 +120,7 @@ class FlomniOpticsMixin:
"this could cause collisions when operating on the real "
"beamline."
):
print("Aborting feye_in(): fttrx1 not moved.")
return
raise FlomniOpticsError("Aborting feye_in(): fttrx1 not moved.")
else:
fttrx_out = self._get_user_param_safe("feyex", "fttrx_out")
umv(dev.fttrx1, fttrx_out)
@@ -73,13 +131,22 @@ class FlomniOpticsMixin:
current_feyex = dev.feyex.readback.get()
current_feyey = dev.feyey.readback.get()
# check if both are close enough (within 0.01)
if np.isclose(current_feyex, feyex_in, atol=0.01) and np.isclose(current_feyey, feyey_in, atol=0.01):
# both already in position → do nothing
pass
else:
# move both axes to the desired "in" positions
umv(dev.feyex, feyex_in, dev.feyey, feyey_in)
# # check if both are close enough (within 0.01)
# if np.isclose(current_feyex, feyex_in, atol=0.01) and np.isclose(current_feyey, feyey_in, atol=0.01):
# # both already in position → do nothing
# pass
# else:
# if "sl1xs" in dev and "sl1ys" in dev:
# self._sl1xs_before_eye = dev.sl1xs.user_readback.get()
# self._sl1ys_before_eye = dev.sl1ys.user_readback.get()
# scans.umv(dev.sl1xs, 0.3, dev.sl1ys, 0.3, relative=True)
# if self.csaxs is not None:
# self.csaxs.fil_trans(0.05, print_only=False)
# # move both axes to the desired "in" positions
# umv(dev.feyex, feyex_in, dev.feyey, feyey_in)
umv(dev.feyex, feyex_in, dev.feyey, feyey_in)
def _ffzp_in(self):
foptx_in = self._get_user_param_safe("foptx", "in")
@@ -198,6 +265,13 @@ class FlomniOpticsMixin:
)
if need_move_osa:
if not (self._fheater_is_up() or self._fheater_is_down()):
raise FlomniOpticsError(
"Refusing to move OSA to its IN position: fheater is neither "
"at its 'up' (retracted) limit nor at its 'down' (heating) "
"position. An intermediate heater position risks a collision "
"with the OSA. Move the heater fully up or fully down first."
)
remaining = self._osa_remaining_space(fosaz_in)
if remaining <= 0:
foptz_val = dev.foptz.readback.get()
@@ -261,7 +335,13 @@ class FlomniOpticsMixin:
def fosa_out(self):
self.ensure_fheater_up()
if not (self._fheater_is_up() or self._fheater_is_down()):
raise FlomniOpticsError(
"Refusing to move OSA to its OUT position: fheater is neither "
"at its 'up' (retracted) limit nor at its 'down' (heating) "
"position. An intermediate heater position risks a collision "
"with the OSA. Move the heater fully up or fully down first."
)
curtain_is_triggered = dev.foptz.controller.fosaz_light_curtain_is_triggered()
if not curtain_is_triggered:
fosaz_out = self._get_user_param_safe("fosaz", "out")
@@ -340,7 +420,8 @@ class FlomniOpticsMixin:
print("\nOSA Information:")
print(f" Current fosaz {fosaz_val:.1f}")
print(f" The OSA will collide with a normal OMNY pin at fosaz \033[1m{(33-fosaz_val):.1f}\033[0m")
# collision where compute_osa_remaining_space() = 33 - foptz - fosaz reaches 0 (was 33 - fosaz)
print(f" The OSA will collide with a normal OMNY pin at fosaz \033[1m{(33-foptz_val):.1f}\033[0m")
print(f" Remaining space (right now): \033[1m{remaining_current:.1f}\033[0m")
diff = fosaz_val - fosaz_in # >0: OSA is currently more "in" than nominal -> worse
@@ -0,0 +1,273 @@
"""flomni.realtime_system: the operating settings of the flOMNI real-time loop.
BEAMLINE SCIENTISTS / LOCAL CONTACT ONLY - not for users. These settings change how the FZP scanner, the detector
trigger and the laser tracker work; a wrong combination can spoil a beamtime's data. The rules refuse combinations
that do not work, but not ones that are only unsuitable for an experiment.
The settings, the parameters they map onto and the rules between them live in the flOMNI instrument repository
(FeedbackLoop/loopsettings/flomni.json) and are applied by the web HMI on the loop machine (RT-system
webhmi/loopsettings.py). This client only talks to its REST API (/api/settings); it holds no logic of its own, so the
web page (tab "Loop settings") and BEC always behave the same way.
flomni.realtime_system.show() all settings, rules, scan state
flomni.realtime_system.set(trigger_tolerance_nm=5) change (refused if a rule objects)
flomni.realtime_system.set(fzp_enable=0, adjust=True) ... and adjust the dependent settings
flomni.realtime_system.check(fzp_range_V_x=1.0) dry run
flomni.realtime_system.apply_profile("fzp_off") a named set of settings
flomni.realtime_system.save_as_default() write the live values into the .xmp files
Changes are live only: the next start of the control loop uses the .xmp files again (save_as_default keeps them).
Connection: through the BEC device flomni_loop_settings (RPC; the API token is in its device config,
ptycho_flomni.yaml) when it exists and is enabled, else directly to the web HMI (devices/omny/webhmi_client.py: URL
http://<rtx host>:8080 with port-80 fallback, token from ~/.flomni_webhmi_token or FLOMNI_WEBHMI_TOKEN_FILE).
"""
from csaxs_bec.devices.omny.webhmi_client import WebHmiClient
from csaxs_bec.devices.omny.webhmi_client import WebHmiError as RealtimeSystemError # name kept for BEC users
STAFF_ONLY = ("flomni.realtime_system: beamline scientists / local contact only - not for users "
"(plugins/flomni/docs/local_contact_only-realtime_system.md).")
def _fmt(v):
if v is None:
return "-"
if isinstance(v, float):
return f"{v:.6g}"
return str(v)
class FlomniRealtimeSystem(WebHmiClient):
"""Requests go through the BEC device flomni_loop_settings (RPC api_get / api_post; the token is in its device
config) when it exists and is enabled; otherwise directly to the web HMI (WebHmiClient,
devices/omny/webhmi_client.py; token from token_file / ~/.flomni_webhmi_token)."""
DEVICE = "flomni_loop_settings"
def __init__(self, url=None, token_file=None, device_manager=None, timeout=5.0, token=None):
super().__init__(url=url, token=token, token_file=token_file, device_manager=device_manager, timeout=timeout)
def _device(self):
try:
dev = getattr(self._device_manager.devices, self.DEVICE)
except Exception: # noqa: BLE001 (no device manager / no such device)
return None
return dev if getattr(dev, "enabled", True) else None
def _request(self, path, body=None):
dev = self._device()
if dev is None:
return super()._request(path, body) # direct HTTP
try:
return dev.api_get(path) if body is None else dev.api_post(path, body)
except Exception as exc: # noqa: BLE001 (RPC error with the device-side message)
raise RealtimeSystemError(f"via device {self.DEVICE}: {exc}") from None
@property
def route(self):
"""'device flomni_loop_settings' or the direct URL."""
return f"device {self.DEVICE}" if self._device() is not None else self.url
# ── reading ───────────────────────────────────────────────────────────────
def read(self) -> dict:
"""The full state as the web HMI reports it (settings, rules, scan state, profiles)."""
return self._request("/api/settings")
def get(self, name: str = None):
"""Live value of one setting, or a dict of all values."""
values = {s["name"]: s["value"] for s in self.read()["settings"]}
if name is None:
return values
if name not in values:
raise RealtimeSystemError(f"unknown setting '{name}'; known: {', '.join(values)}")
return values[name]
def show(self, group: str = None, expert: bool = False):
"""Print the settings (live value, .xmp value, markers), the violated rules and the scan state."""
snap = self.read()
groups = {g["name"]: g for g in snap["groups"]}
sc = snap.get("scan") or {}
print(STAFF_ONLY)
print(f"Loop settings {snap.get('instrument', '')} ({self.route}) - "
+ (f"scan running (mode {_fmt(sc.get('mode'))})" if sc.get("active") else "no scan"))
for gname, g in groups.items():
if group and gname != group:
continue
if g.get("expert") and not expert and not group:
continue
rows = [s for s in snap["settings"] if s["group"] == gname]
if not rows:
continue
print(f"\n{g.get('title', gname)}" + (" (changed only between scans)" if g.get("interlock") else ""))
for s in rows:
info = f" ({s['info']['label']} {s['info']['value']:.4g} {s['info']['unit']})" if s.get("info") else ""
mark = (" NOT SAVED (.xmp " + _fmt(s["saved"]) + ")" if s["unsaved"] else "") + \
(" INCONSISTENT: " + "; ".join(s["issues"]) if not s["consistent"] else "")
print(f" {s['name']:30s} {_fmt(s['value']):>10s} {s.get('unit', ''):3s}{info}{mark}")
bad = [r for r in snap["rules"] if not r["ok"]]
if bad:
print()
for r in bad:
print(f"{r['severity'].upper()}: {r['message']}")
if not expert and not group and any(g.get("expert") for g in snap["groups"]):
print("\n(expert groups hidden: show(expert=True))")
print("\n" + (snap.get("note") or ""))
def profiles(self):
"""Print the named profiles and their values."""
for p in self.read()["profiles"]:
print(f"{p['name']:18s} {p.get('title', '')}")
if p.get("help"):
print(f"{'':18s} {p['help']}")
print(f"{'':18s} " + ", ".join(f"{k}={_fmt(v)}" for k, v in p["values"].items()))
def unsaved(self):
"""Print the parameters whose live value differs from the .xmp (settings only)."""
un = self._request("/api/settings/unsaved")
if not un:
print("All settings are saved: the .xmp files hold the live values.")
for u in un:
print(f" {u['module']}.{u['param']}: .xmp {u['saved']} -> live {u['live']}")
return un
# ── changing ──────────────────────────────────────────────────────────────
@staticmethod
def _print_result(r, dry=False):
head = ("Check: " if dry else "") + ("applied" if r.get("applied") else ("ok" if r["ok"] else "REFUSED"))
if r["ok"] and not r["params"]:
head = "no change"
print(head + (f" (profile {r['profile']})" if r.get("profile") else ""))
for c in r["changes"]:
print(f" {c['name']}: {_fmt(c['old'])} -> {_fmt(c['new'])}"
+ (f" (adjusted by rule {c['adjusted']})" if c.get("adjusted") else ""))
for e in r["errors"]:
print(f" ERROR: {e}")
for w in r["warnings"]:
print(f" warning: {w}")
if r.get("applied"):
print(" " + r.get("note", "Live only: save_as_default() to keep it."))
def check(self, adjust: bool = False, **changes) -> dict:
"""Dry run: what would change, which rules object. Nothing is sent to the loop."""
r = self._request("/api/settings/plan", {"changes": changes, "profile": None, "adjust": adjust})
self._print_result(r, dry=True)
return r
def set(self, adjust: bool = False, quiet: bool = False, **changes) -> dict:
"""Change settings, e.g. set(trigger_tolerance_nm=5). All parameters change in one loop step.
A rule that objects refuses the request (RealtimeSystemError); with adjust=True the settings the rule names
are changed with it (never the ones given here). Live only until save_as_default(). quiet: print nothing
(interactive tools); a refusal still raises.
"""
if not changes:
raise RealtimeSystemError("nothing to set: e.g. set(trigger_tolerance_nm=5); show() lists the names")
r = self._request("/api/settings/apply", {"changes": changes, "profile": None, "adjust": adjust})
if not quiet:
self._print_result(r)
if not r["ok"]:
raise RealtimeSystemError("; ".join(r["errors"]))
return r
def apply_profile(self, name: str, adjust: bool = False, **changes) -> dict:
"""Apply a named profile (profiles() lists them); extra keyword values override the profile's."""
r = self._request("/api/settings/apply", {"changes": changes, "profile": name, "adjust": adjust})
self._print_result(r)
if not r["ok"]:
raise RealtimeSystemError("; ".join(r["errors"]))
return r
def save_as_default(self, *names: str, confirm: bool = True) -> dict:
"""Write the live values of the settings' parameters (all, or the named settings) into the .xmp files, so the
next loop start uses them (the web HMI keeps a backup of each file)."""
un = self._request("/api/settings/unsaved")
if not un:
print("Nothing to save: the .xmp files hold the live values.")
return {"saved": []}
if not names:
print("Save as default (written into the .xmp files of the loop):")
for u in un:
print(f" {u['module']}.{u['param']}: {u['saved']} -> {u['live']}")
if confirm and input("Save? [y/N] ").strip().lower() not in ("y", "yes"):
print("not saved")
return {"saved": []}
r = self._request("/api/settings/save", {"names": list(names) or None})
print(r.get("message", ""), "; ".join(r.get("saved", [])))
return r
# ── shortcuts (detector trigger, laser tracker) ────────────────────────────
# the former flomni.interferometer_trigger_* (removed 2026-10-23)
def trigger_show(self):
"""Detector trigger (step scans): tolerance, hold time, minimum delay, trigger after every move."""
self.show(group="trigger")
def trigger_set_tolerance_nm(self, tolerance_nm: float):
"""|relative position sample vs FZP| x and y within this (nm) for the hold time. 30 coarse, 10 standard, 2 finest;
0 = the former criterion (OnTarget_legacy flags of the FZP PIDs)."""
return self.set(trigger_tolerance_nm=tolerance_nm)
def trigger_set_hold_ms(self, hold_ms: float):
"""How long the position must stay within the tolerance before the trigger (ms), typical 3."""
return self.set(trigger_hold_ms=hold_ms)
def trigger_set_after_move(self, enable: bool):
"""True: one detector trigger after every direct position change (umv rtx / rty / rtz, move_xyz) once on target;
False: triggers at step-scan points only."""
return self.set(trigger_on_move=1 if enable else 0)
def tracking_targets(self):
"""-> (target y, target z) of the laser tracker on the PSD."""
v = self.get()
return v["tracking_target_y"], v["tracking_target_z"]
def help(self, name: str = None):
"""help(): the functions; help("all"): every setting with its explanation; help(name): one setting (what it
does, unit, range, the loop parameters it sets, live and saved value), from the definition on the loop
machine (the same text as the tooltips of the web HMI tab)."""
if name is None:
print(__doc__.split("\n\nConnection:")[0])
print("\nShortcuts: trigger_show(), trigger_set_tolerance_nm(v), trigger_set_hold_ms(v),"
" trigger_set_after_move(True/False), tracking_targets(); profiles(), unsaved(), get(name)")
print('Explanations: help("all") or help("<setting>"), e.g. help("move_beyond_um_x")')
return
snap = self.read()
settings = {s["name"]: s for s in snap["settings"]}
groups = {g["name"]: g for g in snap["groups"]}
if name == "all":
print(STAFF_ONLY)
for gname, g in groups.items():
rows = [s for s in snap["settings"] if s["group"] == gname]
if not rows:
continue
print(f"\n{g.get('title', gname)}" + (" (only between scans)" if g.get("interlock") else ""))
for s in rows:
unit = f" [{s['unit']}]" if s.get("unit") else ""
print(f" {s['name']}{unit}: {s.get('title', '')}")
if s.get("help"):
print(f" {s['help']}")
return
s = settings.get(name)
if s is None:
raise RealtimeSystemError(f"unknown setting '{name}'; help('all') lists them")
g = groups.get(s["group"], {})
print(STAFF_ONLY)
print(f"{s['name']} - {s.get('title', '')} (group {g.get('title', s['group'])}"
+ (", only between scans)" if g.get("interlock") else ")"))
if s.get("help"):
print(f" {s['help']}")
rng = "0 / 1" if s.get("type") == "bool" else f"{_fmt(s.get('min'))} ... {_fmt(s.get('max'))} {s.get('unit', '')}"
print(f" range: {rng}")
print(f" live: {_fmt(s.get('value'))} saved (.xmp): {_fmt(s.get('saved'))}"
+ (" NOT SAVED" if s.get("unsaved") else "") + ("" if s.get("consistent", True) else " INCONSISTENT"))
if s.get("info"):
print(f" {s['info'].get('label', '')} {s['info']['value']:.4g} {s['info'].get('unit', '')}")
print(" sets: " + ", ".join(f"{t['module']}.{t['param']}[{t['index']}] = {t.get('write', 'v')}"
for t in s.get("targets", [])))
rules = [r for r in snap.get("rules", []) if name in r.get("settings", [])]
if rules:
print(" rules: " + ", ".join(r["name"] + ("" if r.get("ok") else " (VIOLATED)") for r in rules))
@@ -43,7 +43,6 @@ from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.webpage_generator_base imp
_safe_get,
_safe_float,
)
from csaxs_bec.devices.omny.sample_desc_codec import unpack_desc
class FlomniWebpageGenerator(WebpageGeneratorBase):
@@ -54,6 +53,8 @@ class FlomniWebpageGenerator(WebpageGeneratorBase):
Logo: flOMNI.png from the same directory as this file.
"""
CLIENT_NAME = "flomni"
# Global-var names of the tomo parameters to read live for the
# 'Current measurement' section. Kept in the same order as, and matching,
# the tomo queue's TQ_PARAM_DISPLAY keys so the two render identically.
@@ -73,6 +74,7 @@ class FlomniWebpageGenerator(WebpageGeneratorBase):
"single_point_instead_of_fermat_scan",
"ptycho_reconstruct_foldername",
"at_each_angle_hook",
"tomo_angle_file",
]
# label -> dotpath under device_manager.devices
@@ -86,7 +88,13 @@ class FlomniWebpageGenerator(WebpageGeneratorBase):
# Explicitly set (not just inherited) so flomni's page keeps rendering
# its own param names/labels even if WebpageGeneratorBase's default ever
# changes for another setup.
TQ_PARAM_DISPLAY = WebpageGeneratorBase.TQ_PARAM_DISPLAY
# tomo_angle_file: tomo_type 4 (angles from file) is flomni-only. The
# page can't read the file itself, so a type-4 job's projection count
# stays blank (not in TOMO_TYPES) -- the live progress card still shows
# the true total, which tomo_scan() sets from the parsed file.
TQ_PARAM_DISPLAY = WebpageGeneratorBase.TQ_PARAM_DISPLAY + [
("tomo_angle_file", "Angle file"),
]
def _logo_path(self):
return Path(__file__).parent / "flOMNI.png"
@@ -97,19 +105,16 @@ class FlomniWebpageGenerator(WebpageGeneratorBase):
# ── Sample name ───────────────────────────────────────────────
# slot 0 of flomni_samples is the flOMNI measurement stage itself
# (see FlomniSampleStorage.show_all()'s own "flOMNI stage:" label
# for slot 0 -- not an arbitrary tray position). The raw DESC value
# is packed as "name | owner" (sample_desc_codec.pack_desc, added
# 7e8e807) -- unpack it here too, the same way
# FlomniSampleStorage.get_sample_name_and_owner() and show_all()
# already do, so the page doesn't show the raw delimiter-joined
# string as the "sample name". Owner is appended in parentheses,
# mirroring show_all()'s own "(owner: ...)" formatting.
raw_sample = _safe_get(dm, "flomni_samples.sample_names.sample0")
if raw_sample is None:
# for slot 0 -- not an arbitrary tray position). Name and owner are
# separate IOC fields since 2026-10-17 (FLOMNI-SAMPLE:STAGE:NAME /
# :OWNER); owner is appended in parentheses, mirroring show_all()'s
# own "(owner: ...)" formatting.
name = _safe_get(dm, "flomni_samples.sample_names.sample0")
if name is None:
sample_name = "N/A"
else:
name, owner = unpack_desc(raw_sample)
sample_name = f"{name} (owner: {owner})" if owner else name
owner = _safe_get(dm, "flomni_samples.sample_owners.sample0")
sample_name = f"{name} (owner: {owner})" if owner else str(name)
# ── Temperatures ──────────────────────────────────────────────
temperatures = {
@@ -48,7 +48,12 @@ class flomniGuiTools:
def flomnigui_show_gui(self):
if "flomni" in self.gui.windows:
self.flomni_window = self.gui.windows["flomni"]
self.gui.flomni.raise_window()
# raise_window() is currently buggy (bec_widgets RPCServer "raise" handling can
# detach/hide the window instead of bringing it to front) - disabled so that
# calling a flomnigui_show_* macro again on an already-open GUI is a safe no-op
# instead of risking hiding the window. See flomnigui_raise() above for the same
# workaround.
# self.gui.flomni.raise_window()
else:
# geometry: (pos_x, pos_y, w, h)
pos_x = self._SCREEN_WIDTH - self._WINDOW_WIDTH
@@ -60,7 +65,8 @@ class flomniGuiTools:
self.gui.flomni.hide()
def flomnigui_raise(self):
self.gui.flomni.raise_window()
pass
#self.gui.flomni.raise_window()
def flomnigui_show_xeyealign(self):
self.flomnigui_show_gui()
@@ -132,20 +138,18 @@ class flomniGuiTools:
print("Cannot open camera_overview. Device does not exist.")
# Confirm/abort console, docked below the cameras. The hard-stop
# button directly calls dev.ftransy.controller
# .hard_abort_and_restore_positioning_mode() (the same Galil
# hard-stop ftransfer_abort() uses) -- see ConsoleButtonsWidget's
# docstring for why this replaced a blind stop-all-devices
# broadcast. foptx sits on a physically separate Galil
# controller (the optics stage board, not the transfer stage
# board) so it needs its own stop_all_axes() call -- also
# explained in ConsoleButtonsWidget's docstring.
# button sends a stop to each of flomni's three Galil controllers
# (one representative device per controller, in this order, the
# sample-transfer controller first): ftransy (fsamx, fsamy,
# ftransx/y/z, ftray), foptx (optics, eye, heater, tracking) and
# fsamroy (rotation). Afterwards it restores positioning mode on
# the first one (the same Galil hard-stop ftransfer_abort() uses)
# -- see ConsoleButtonsWidget's docstring.
self.console = self.gui.flomni.new(
"z_ConsoleButtonsWidget",
object_name="console",
where="bottom",
hard_stop_device_name="ftransy",
extra_hard_stop_device_name="foptx",
hard_stop_device_names=["ftransy", "foptx", "fsamroy"],
hard_stop_label="Flomni Motion Stop",
)
# set_layout_ratios uses relative weights, not pixels -- there is
@@ -261,6 +265,17 @@ class flomniGuiTools:
except Exception as e:
logger.warning(f"Error in flomnigui_show_progress: {e}")
@staticmethod
def _percent(numerator, denominator) -> float:
"""numerator/denominator in percent; 0 when the denominator is not
positive -- e.g. the total of a tomo_type 4 scan whose angle file has
no 'END' marker yet, or an endless golden-ratio scan (0 = unknown).
Without this the division raised, the error was swallowed, and the
GUI stayed frozen on the reset defaults for the whole scan."""
if not denominator or denominator <= 0:
return 0.0
return numerator / denominator * 100
def _flomnigui_update_progress(self):
"""Update the progress ring bar and center label from the current progress state.
@@ -277,11 +292,9 @@ class flomniGuiTools:
main_progress_ring = self.progressbar.rings[0]
subtomo_progress_ring = self.progressbar.rings[1]
progress = self.progress["projection"] / self.progress["total_projections"] * 100
subtomo_progress = (
self.progress["subtomo_projection"]
/ self.progress["subtomo_total_projections"]
* 100
progress = self._percent(self.progress["projection"], self.progress["total_projections"])
subtomo_progress = self._percent(
self.progress["subtomo_projection"], self.progress["subtomo_total_projections"]
)
main_progress_ring.set_value(progress)
subtomo_progress_ring.set_value(subtomo_progress)
@@ -41,6 +41,14 @@ class XrayEyeAlign:
HEIGHT_CENTERING_SIGN = -1
# Fit steps k = 1..N_FIT_STEPS are the angles 0, 45, ..., 180 deg (angle = (k - 1) *
# ANGLE_STEP_DEG). The sine fit has 3 free parameters, so at most
# N_FIT_STEPS - MIN_FIT_POINTS of them may be skipped from the GUI (submit == -1), e.g.
# the 90 deg step for a flat sample that is edge-on there.
N_FIT_STEPS = 5
ANGLE_STEP_DEG = 45
MIN_FIT_POINTS = 4
def __init__(self, client, flomni: Flomni) -> None:
self.client = client
self.flomni = flomni
@@ -95,14 +103,50 @@ class XrayEyeAlign:
# image_idx refers to alignment_images[image_idx], i.e. the last
# frame captured before that submit (shutter is closed at submit time).
self.roi_pixel_data = []
# Fit steps (k) the user skipped from the GUI; they have no alignment_values entry.
self.skipped_steps = []
def _save_alignment_data(self, file_path: str, fit_data: np.ndarray | None = None):
def _step_angle(self, k: int) -> int:
"""Rotation angle in deg of fit step k (1..N_FIT_STEPS)."""
return (k - 1) * self.ANGLE_STEP_DEG
def _step_prompt(self, k: int) -> str:
"""The instruction shown to the user while waiting for the submission of step k."""
if k == 0:
return "Submit center value of FZP."
if k == 1 and not self._height_centered:
return "Mark the sample height with the selection tool, then submit."
return f"Step {k}/{self.N_FIT_STEPS}: Submit sample center"
def _skip_refusal_reason(self, k: int) -> str | None:
"""Why step k cannot be skipped, or None if a skip is allowed.
Only real fit points can be skipped: not the FZP center (k == 0) nor the height-marking
submission (the first submission at k == 1), and only while at least MIN_FIT_POINTS
remain for the fit.
"""
if k == 0:
return "the FZP center cannot be skipped"
if k == 1 and not self._height_centered:
return "the sample height marking cannot be skipped"
if len(self.skipped_steps) + 1 > self.N_FIT_STEPS - self.MIN_FIT_POINTS:
return f"at least {self.MIN_FIT_POINTS} positions are needed for the fit"
return None
def _save_alignment_data(
self,
file_path: str,
fit_data: np.ndarray | None = None,
skipped_angles: list[int] | None = None,
):
expanded = os.path.expanduser(file_path)
os.makedirs(os.path.dirname(expanded), exist_ok=True)
with h5py.File(expanded, "w") as f:
f.create_dataset(
ds = f.create_dataset(
"alignment_values", data=np.array(list(self.alignment_values.values()))
)
# skipped steps have no entry, so say which step each value belongs to
ds.attrs["step_k"] = list(self.alignment_values.keys())
f.create_dataset("alignment_images", data=np.array(self.alignment_images))
if self.roi_pixel_data:
ds = f.create_dataset("roi_pixel_data", data=np.array(self.roi_pixel_data))
@@ -110,6 +154,7 @@ class XrayEyeAlign:
if fit_data is not None:
ds = f.create_dataset("alignment_fit", data=fit_data)
ds.attrs["rows"] = ["angles_deg", "offsets_um", "zeros"]
ds.attrs["skipped_angles_deg"] = list(skipped_angles or [])
def update_frame(self, keep_shutter_open=False):
if self.flomni._flomnigui_check_attribute_not_exists("xeyegui"):
@@ -164,6 +209,10 @@ class XrayEyeAlign:
try:
self._align_impl(keep_shutter_open)
finally:
try:
self.gui.enable_skip_button(False)
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Failed to disable XRayEye skip button: {exc}")
try:
self.gui.set_dap_params_forwarding(False)
except Exception as exc: # pylint: disable=broad-except
@@ -172,6 +221,10 @@ class XrayEyeAlign:
self.gui.hide_crosshair()
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Failed to hide XRayEye alignment crosshair: {exc}")
try:
self.gui.unlock_vertical_center()
except Exception as exc: # pylint: disable=broad-except
logger.warning(f"Failed to unlock XRayEye ROI vertical center: {exc}")
def _align_impl(self, keep_shutter_open=False):
if not keep_shutter_open:
@@ -182,6 +235,7 @@ class XrayEyeAlign:
self.alignment_images = []
self.gui.enable_submit_button(False)
self.gui.enable_skip_button(False)
# Initialize xray align device
# clear potential pending movement requests
@@ -194,6 +248,9 @@ class XrayEyeAlign:
# reset shift xy and fov params
self._reset_init_values()
# a fresh run must not inherit a stale lock from a previous (possibly
# interrupted) run -- see lock_vertical_center()/unlock_vertical_center()
self.gui.unlock_vertical_center()
# Moved here from __init__: this is the actual start of a fresh
# alignment run (matching what start_x_ray_eye_alignment() already
@@ -203,6 +260,7 @@ class XrayEyeAlign:
# there.
self.flomni.reset_correction()
self.flomni.reset_tomo_alignment_fit()
self.flomni.manual_shift_x = 0
self.flomni.manual_shift_y = 0
self.flomni.lights_off()
@@ -240,32 +298,59 @@ class XrayEyeAlign:
k = 0
while True:
if dev.omny_xray_gui.submit.get() == 1:
# submit: 0 = idle, 1 = submitted, -1 = skip this step (see the Skip button)
submit_val = dev.omny_xray_gui.submit.get()
if submit_val == -1:
refusal = self._skip_refusal_reason(k)
if refusal is not None:
# refused: the run is untouched, keep waiting at the same step
dev.omny_xray_gui.submit.set(0)
self.gui.enable_skip_button(False)
self.send_message(f"Cannot skip: {refusal}. {self._step_prompt(k)}")
print(f"Skip refused at step {k}: {refusal}.")
submit_val = 0
if submit_val in (1, -1):
skipped = submit_val == -1
self.alignment_values[k] = (
getattr(dev.omny_xray_gui, f"xval_x_{k}").get() * self.pixel_calibration
) # in mm
print(f"Clicked position {k}: x {self.alignment_values[k]}")
rtx_position = dev.rtx.readback.get() / 1000
print(f"Current rtx position {rtx_position}")
self.alignment_values[k] -= rtx_position
print(f"Corrected position {k}: x {self.alignment_values[k]}")
# reset submit channel
dev.omny_xray_gui.submit.set(0)
if skipped:
# no xval_x_k / width_*_k were written for this step, so never read them
dev.omny_xray_gui.submit.set(0)
self.skipped_steps.append(k)
_img_idx = len(self.alignment_images) - 1
self.roi_pixel_data.append([k, np.nan, np.nan, np.nan, np.nan, _img_idx])
print(f"Skipped position {k} ({self._step_angle(k)} deg)")
if self._step_angle(k) in (0, 180):
print(
"Warning: skipping an end point (0/180 deg) makes the fit noticeably"
" less stable than skipping 45/90/135 deg."
)
# only allow another skip if the fit can still spare a point
self.gui.enable_skip_button(self._skip_refusal_reason(k) is None)
else:
self.alignment_values[k] = (
getattr(dev.omny_xray_gui, f"xval_x_{k}").get() * self.pixel_calibration
) # in mm
print(f"Clicked position {k}: x {self.alignment_values[k]}")
rtx_position = dev.rtx.readback.get() / 1000
print(f"Current rtx position {rtx_position}")
self.alignment_values[k] -= rtx_position
print(f"Corrected position {k}: x {self.alignment_values[k]}")
# reset submit channel
dev.omny_xray_gui.submit.set(0)
# Raw pixel position and ROI size at submit time.
# The relevant image is the last captured frame (shutter is
# closed by the time the user clicks submit).
_raw_x = getattr(dev.omny_xray_gui, f"xval_x_{k}").get()
_raw_y = getattr(dev.omny_xray_gui, f"yval_y_{k}").get()
_raw_w = getattr(dev.omny_xray_gui, f"width_x_{k}").get()
_raw_h = getattr(dev.omny_xray_gui, f"width_y_{k}").get()
_img_idx = len(self.alignment_images) - 1
print(
f" Submit k={k}: px x={_raw_x:.1f} y={_raw_y:.1f} "
f"w={_raw_w:.1f} h={_raw_h:.1f} img={_img_idx}"
)
self.roi_pixel_data.append([k, _raw_x, _raw_y, _raw_w, _raw_h, _img_idx])
# Raw pixel position and ROI size at submit time.
# The relevant image is the last captured frame (shutter is
# closed by the time the user clicks submit).
_raw_x = getattr(dev.omny_xray_gui, f"xval_x_{k}").get()
_raw_y = getattr(dev.omny_xray_gui, f"yval_y_{k}").get()
_raw_w = getattr(dev.omny_xray_gui, f"width_x_{k}").get()
_raw_h = getattr(dev.omny_xray_gui, f"width_y_{k}").get()
_img_idx = len(self.alignment_images) - 1
print(
f" Submit k={k}: px x={_raw_x:.1f} y={_raw_y:.1f} "
f"w={_raw_w:.1f} h={_raw_h:.1f} img={_img_idx}"
)
self.roi_pixel_data.append([k, _raw_x, _raw_y, _raw_w, _raw_h, _img_idx])
# Controls whether `k` advances to the next step below. Left
# True except for the height-centering submission, which
@@ -299,7 +384,7 @@ class XrayEyeAlign:
self.gui.show_crosshair()
self.send_message(
"Submit height. Use arrows if far off."
"Mark the sample height with the selection tool, then submit. Use arrows if far off."
)
self.gui.enable_submit_button(True)
self.movement_buttons_enabled(True, True)
@@ -332,10 +417,16 @@ class XrayEyeAlign:
self.flomni.feedback_enable_with_reset()
self._height_centered = True
# From here through step 5, the sample's vertical position is
# physically fixed at the FZP height -- auto-snap every new/
# dragged/resized selection box's vertical center to the
# crosshair instead of requiring manual placement each time.
self.gui.lock_vertical_center()
self.update_frame(keep_shutter_open)
self.send_message("Step 1/5: Submit sample center")
self.gui.enable_submit_button(True)
self.gui.enable_skip_button(True)
self.movement_buttons_enabled(True, False)
advance_step = False
@@ -351,13 +442,16 @@ class XrayEyeAlign:
self.send_message(f"Step {k+1}/5: Submit sample center")
self.gui.enable_submit_button(True)
self.movement_buttons_enabled(True, False)
self.update_fov(k)
if not skipped:
self.update_fov(k)
elif k == 5: # received sample center value at samroy 270 and done
self.send_message("done...")
self.gui.enable_submit_button(False)
self.movement_buttons_enabled(False, False)
self.update_fov(k)
self.gui.enable_skip_button(False)
if not skipped:
self.update_fov(k)
self.gui.hide_crosshair()
break
@@ -422,19 +516,34 @@ class XrayEyeAlign:
timestamp = time.strftime("%Y%m%d_%H%M%S")
file_h5 = f"~/data/raw/logs/xrayeye_alignmentvalues/xrayeye_alignmentvalues_{timestamp}.h5"
fovx_offsets = np.zeros(5)
for k in range(1, 6):
steps = [k for k in range(1, self.N_FIT_STEPS + 1) if k not in self.skipped_steps]
if len(steps) < self.MIN_FIT_POINTS:
raise RuntimeError(
f"Only {len(steps)} alignment positions available, at least"
f" {self.MIN_FIT_POINTS} are needed for the fit."
)
fovx_offsets = np.zeros(len(steps))
for i, k in enumerate(steps):
fovx_offset = self.alignment_values[0] - self.alignment_values[k]
fovx_offsets[k - 1] = fovx_offset
fovx_offsets[i] = fovx_offset
print(f"Alignment number {k}, value x {fovx_offset}")
skipped_angles = [self._step_angle(k) for k in self.skipped_steps]
if skipped_angles:
print(
f"Skipped angles {skipped_angles} deg, fitting {len(steps)} positions. With 4"
" points the fit has 1 degree of freedom and cannot flag a bad click - check the"
" plot."
)
data = np.array(
[
[0, 45, 90, 135, 180], # angles
[self._step_angle(k) for k in steps], # angles
fovx_offsets * 1000, # fovx_offset values
[0, 0, 0, 0, 0],
np.zeros(len(steps)),
]
)
self._save_alignment_data(file_h5, fit_data=data)
self._save_alignment_data(file_h5, fit_data=data, skipped_angles=skipped_angles)
self.gui.submit_fit_array(data)
print(f"fit submited with {data}")
@@ -0,0 +1,304 @@
# OMNY sample-transfer machine-vision auto-confirm — design plan (2026-09-06)
Exploration only — **no code in this pass**. Written on `update-omny` so a future session/branch
can pick this up without re-deriving it. Companion docs: `OPEN_ISSUES.md` (the live-testing effort
this plan grows out of — see its "full gripper sample-transfer chain... untested" item) and
`SAMPLE_STORAGE_TEST_CHECKLIST.md`.
## Motivation
Mirko's starting question: is there a Python package for comparing a live gripper/pin camera image
against a set of reference images, robust to exposure changes and small camera-alignment drift, to
decide "is the pin sitting in the gripper correctly" — classic machine-vision, not deep learning.
Applied to BEC: could this let OMNY's sample-transfer flow (`otransfer_get_sample()` /
`otransfer_put_sample()`) auto-approve routine steps instead of stopping for a human `y/n` at every
single one, while still catching real problems (pin damage, missed grip, wrong shuttle state)?
This doc maps that idea onto OMNY's actual confirm gates and camera/GUI plumbing, and proposes a
staged plan. Follow-up feedback broadened the scope: rather than building the image-processing logic
into a single confirm/scoring device, keep it generic — a reusable camera-in/processed-image-out
interface (grayscale, edge detection, overlays, ...) that the sample-transfer confirm use case is
only the *first* consumer of; `x_ray_eye_align.py`'s alignment GUI is identified below as a second,
independent consumer of the same layer. **Goal for this session: write this plan down.
Implementation is a separate, later branch.**
## Where today's human confirmations actually live
All of these are in `omny_sample_transfer_mixin.py` and go through
`otransfer_confirm_dialog()` (line 76 — GUI `ConsoleButtonsWidget` yes/no if the camera console is
already open, else a blocking CLI `yesno()`):
1. **`_otransfer_confirm()`** (line 848) — the main per-step handshake during an actual mount/get,
driven by the Galil controller's `confirm`/`mntprgs` variables (`XQ#GRPUT`/`XQ#GRGET` macros).
Loops until the controller reports `mntprgs == 0`; each iteration is one manual "All OK?"
gate. For a *get*, the first iteration additionally hard-checks the top-pin position against a
fixed threshold (`-25.4`, line 866) and asks an extra, more alarming confirm if it's exceeded —
this is a **physical safety check**, not a generic step confirm, and should stay independent of
any vision system.
2. **`otransfer_get_sample()`** (line 934) — "confirm there is no sample in the gripper, it would
be dropped" before starting at all.
3. **`otransfer_get_sample()`** (line 971) / **`otransfer_put_sample()`** (line 812) — "Ready to
start the [un]mount process now?" right before triggering `XQ#GRGET`/`XQ#GRPUT`.
4. **`otransfer_put_sample()`** (line 836) — "Pick another sample from this shuttle? Keep shield
open?" after a successful put.
5. **`_otransfer_shuttle_align()`** (line 189) — one-off "shall I try to move the aligner to the
correct start position" recovery prompt, only on a specific bad-limit-switch state. Edge case,
not a routine per-transfer gate — out of scope for auto-confirm.
6. **`otransfer_park_loadlock_slot()`** (line 523) — CLI-only `yesno()` by design (operator is
physically at the loadlock, not watching the GUI camera console) — also out of scope here.
Gates 1–4 are the routine, repeated-every-transfer confirmations this plan targets. Every one of
them already calls **`_otransfer_save_reference_images()`** (line 885, landed in `a7429b2`) right
at the confirm point, which dumps the currently-active camera pair's `get_last_image()` frames to
an HDF5 file under `~/data/raw/logs/omny_sample_transfer_images/`, named by step + timestamp. **This
is already the reference-image corpus this plan would compare against** — it exists purely as a
post-hoc audit log today, with no comparison logic reading it back.
## Existing infrastructure to build on
- **`get_last_image() -> np.ndarray`** is already a shared, RPC-exposed contract across camera
device classes, not an `IDSCamera` idiosyncrasy: both `IDSCamera`
(`csaxs_bec/devices/ids_cameras/ids_camera.py:234`, `rpc_access` line 72) and
`AlliedVisionAravisCamera` (`csaxs_bec/devices/allied_vision_cameras/allied_vision_aravis_camera.py:216`,
`rpc_access` line 62) implement it identically, and client-side code already calls it directly
through the device-server RPC boundary — `LamNI/x_ray_eye_align.py:202,821` and
`flomni/x_ray_eye_align.py:127` do exactly this (`dev.cam_xeye.get_last_image()`) regardless of
which concrete camera class backs `cam_xeye` at a given beamline. This is the natural minimal
interface for a generic vision component to depend on — see the architecture section below.
- **`OMNYGuiTools._omnycam_active_pair`** (`gui_tools.py:88-105`) — tracks which camera pair
(`cam200`+`cam203` parking/shuttle, or `cam201`+`cam202` sample stage) is currently shown, exactly
the info needed to pick the matching reference subset.
- **Precedent for one device depending on another by configured name, resolved through the device
manager**: `DDG1.prepare_mcs_on_trigger()` (`csaxs_bec/devices/epics/delay_generator_csaxs/ddg_1.py:369-397`)
looks up a sibling device via `self.device_manager.devices.get(mcs_name, None)` at call time (not
at `__init__`, since device construction order across a YAML config isn't guaranteed) and raises
clearly if it's missing. Same pattern, generalized to a *configurable* name instead of `ddg_1.py`'s
hardcoded `"mcs"`, is the mechanism proposed below for a vision device's camera dependency.
- **`IDSCamera.push_preview_image()` / `push_smear_preview()`** (`ids_camera.py:240,269`) — an
`IDSCamera`-specific hook to publish a processed/composite array through its own
`device_preview` channel, already proven for a similar purpose in LamNI's rotation-center
calibration (`LamNI/x_ray_eye_align.py:737-797`) together with the matching UI toggle in
`bec_widgets/widgets/xray_eye/x_ray_eye.py:386-873`. Usable as a visualization *fallback* when the
configured camera happens to be an `IDSCamera`, but not something the generic vision device
should depend on — see "own diagnostic signal vs. borrowing the camera's channel" below.
- **`opencv-python`** is already a project dependency (`pyproject.toml:27`); `scikit-image` is not
yet (needed only if SSIM specifically is wanted — see below).
## Proposed architecture: a generic, camera-agnostic vision layer
Earlier drafts of this plan sketched the CV pipeline as logic living inside `IDSCamera` itself, or
inline in `omny_sample_transfer_mixin.py` calling `cv2`/`skimage` directly against whichever
`cam200`-`cam203` frame it just pulled. Mirko's feedback, in two parts: (1) keep the imaging
pipeline generic and reusable against *any* camera, structured as its own device with a camera as a
declared dependency, rather than building it into a specific camera class; (2) go further than a
single-purpose confirm/scoring device — make the underlying image manipulation (grayscale, edge
detection, and similar) a generic, reusable *interface* that can sit between a raw camera and its
consumer for other purposes too, e.g. an alternative processed/overlay view for
`x_ray_eye_align.py`'s alignment GUI, not just sample-transfer scoring. Three layers, each usable on
its own:
### Layer 0 — a plain image-processing toolkit (pure functions, no device, no BEC dependency)
Grayscale conversion, CLAHE, Sobel/Canny edge detection, ECC/ORB-based registration, contour
extraction, and overlay drawing (crosshair, contour outline, diff highlight) as plain
`numpy`/`opencv` functions — array in, array (or array + metadata, e.g. a fitted transform or a
contour) out. No devices, no BEC imports, trivially unit-testable in isolation, matching this
repo's existing preference for pure-function "widget math" test files (e.g.
`test_omny_tomo_params_widget_math.py`). This is the shared vocabulary both layers below build on —
neither the confirm/scoring use case nor a live processed-view use case should each hand-roll their
own edge-detection/registration code.
### Layer 1 — `VisionInterface`: a generic camera-in, processed-image-out device
The direct answer to "a camera interface that allows image manipulation... could be used as an
intermediate interface for x_ray_eye align, bypass the normal image, activate edge detection,
overlay graphics": a device that sits *between* a source camera and whatever is watching it,
publishing a transformed view instead of (or alongside) the raw feed.
- **Endstation-agnostic**, living outside `omny/` — e.g. `csaxs_bec/devices/vision/vision_interface.py`.
- **Same camera-dependency mechanism as before**: config-driven `camera_names: list[str]`, resolved
lazily via `self.device_manager.devices.get(name, None)` (the `ddg_1.py` pattern), requiring only
the shared `get_last_image()` contract — no dependency on which concrete camera class it's pointed
at.
- **A selectable processing mode**, built from Layer 0's toolkit — `"raw"`, `"grayscale"`,
`"edges"`, `"clahe"`, `"overlay"` (e.g. a stored reference's contour drawn over the live frame),
or a small ordered pipeline of these — settable at runtime via an RPC/`Signal` (e.g.
`set_mode(name: str)`), not hardcoded per device instance. This generalizes what
`push_smear_preview()`/the existing `smear_preview_toggle` do today (one hardcoded composite, one
fixed second channel) into "any of a small library of transforms, chosen at runtime".
- **Own `diagnostic_image` `PreviewSignal`**, decoupled from whichever camera it's pointed at (the
"own signal" option from the previous draft of this doc) — this is what makes it a genuine
in-between interface rather than a camera-specific patch: the GUI dock watches the
`VisionInterface`'s output, and swapping/reconfiguring the source camera or the active mode
doesn't change what the dock is looking at.
- **Command-line addressable from day one, independent of any GUI work.** `set_mode()` and any other
controls are plain device RPC methods — like `get_last_image()`, they're callable directly from a
BEC session (`dev.omny_vision_parking.set_mode("edges")`) the moment the device exists, with no
GUI widget required. The operator-facing question "how does someone switch modes" therefore has an
answer on day one regardless of whether/when a GUI mode-picker gets built — see the dedicated CLI
section below.
- **Two independent consumers**, justifying building this generically rather than folding it
straight into the sample-transfer use case:
- **`SampleVisionMonitor`** (Layer 2, below) uses it for registration + the diagnostic overlay it
publishes during confirm gates.
- **`x_ray_eye_align.py`**'s alignment GUI (`update_frame()`,
`flomni/x_ray_eye_align.py:114-135` — the operator watches a raw live `cam_xeye` feed and
manually places alignment points, fit later done in Matlab; OMNY's own copy is still an
unadapted LamNI leftover per `OPEN_ISSUES.md`) currently shows only the unprocessed live image.
A `VisionInterface` in front of `cam_xeye` could offer an edge-enhanced or CLAHE-normalized live
view as an operator-toggleable alternative during manual point-picking — exactly the same
exposure-invariance motivation as the sample-transfer use case, applied to a second, already-
existing manual-confirmation-style workflow. **Not proposed as a replacement for the existing
Matlab-based fit** — just a second real consumer of the same generic layer, and a concrete
illustration of why Layer 0/1 shouldn't be scoped to OMNY sample-transfer specifically.
### Layer 2 — `SampleVisionMonitor`: the sample-transfer confirm/scoring application
Everything from the previous draft of this plan, now explicitly framed as *one application built on
top of Layers 0/1*, not a separate camera-coupled implementation:
- **Runs device-server-side** (a `PSIDeviceBase` subclass, like `IDSCamera`/`VisionInterface`
itself), consistent with where camera image processing already happens in this codebase.
- **Opaque, caller-defined reference tags** — reference images and scores are grouped by a plain
string tag the caller picks (for OMNY, `f"{step_name}_{pin_position}"`); the device itself has no
notion of "pin position" or "shuttle slot".
- **Client-facing RPC surface** (sketch, names not final):
- `save_reference(tag: str) -> None` — pulls the current frame(s) (via its own or a shared
`VisionInterface`) and stores them as a labeled reference under `tag`.
- `score(tag: str) -> dict` — registers + scores the current frame(s) against every stored
reference under `tag` (see CV pipeline below, using Layer 0's registration/contour/SSIM
functions), publishes a diagnostic diff/overlay through the `VisionInterface` it uses, and
returns the best-match score(s) plus enough detail to build a confirm message.
- `omny_sample_transfer_mixin.py`'s gates (1-4 above) become *callers* of this device
(`dev.omny_vision_parking.score(...)`, `dev.omny_vision_samplestage.score(...)`, one instance
per camera pair) instead of doing any `cv2`/`skimage` work themselves — the mixin only turns the
returned score into a confirm-dialog decision (per the staged rollout below).
- **Open composition question, not resolved here**: whether `SampleVisionMonitor` *contains* a
`VisionInterface` instance internally (composition), or is itself built directly on Layer 0 with
its own separate `diagnostic_image` signal and `VisionInterface` stays a purely GUI/alignment-side
concept. Composition (reusing one `VisionInterface` per camera pair, shared by both the confirm
scoring and any live processed view of that same pair) avoids two devices both owning a signal for
the same physical camera pair — likely the cleaner shape, but worth prototyping both.
### Operator interface: command line first, a GUI mode-picker second
Per feedback: the tool an operator uses to drive this should not be gated behind new GUI widget
work — it should be addressable on the command line as its own, separately-callable interface,
independent of whether/when a GUI control for it exists.
- **This falls out of BEC's normal device-RPC model, not something to build specially.** Every
method on a device (`VisionInterface.set_mode()`, `SampleVisionMonitor.save_reference()`/`score()`)
is already callable directly from a BEC ipython session the moment the device exists —
`dev.omny_vision_parking.set_mode("edges")`, `dev.omny_vision_parking.score("put_mount_start_3")`
— the exact same mechanism `get_last_image()` already relies on for its own direct client-side
calls in `x_ray_eye_align.py`. No dedicated CLI layer needs to be designed from scratch; it's a
free consequence of this being a device.
- **Thin per-endstation wrapper methods, matching the existing convention.** Raw device names
(`omny_vision_parking`, `omny_vision_samplestage`) aren't the ergonomic surface an operator should
have to remember. `omny_sample_transfer_mixin.py` already establishes the pattern for exactly this
problem — its `otransfer_is_sample_slot_used()`/`otransfer_set_sample_slot()`/etc. block (lines
1175-1255) is a set of thin one-line wrappers around `dev.omny_samples.*`, added, per its own
comment, "so every sample-storage command tab-completes under `omny.otransfer_*`". The same shape
applies here: short `omny.ovision_*` wrappers (e.g. `ovision_set_mode(pair, mode)`,
`ovision_score(tag)`, `ovision_save_reference(tag)`) that dispatch to the right
`SampleVisionMonitor`/`VisionInterface` instance for the currently-active camera pair (using
`_omnycam_active_pair`, already tracked — see "Existing infrastructure" above), so an operator
never needs to know the underlying device names at all.
- **GUI mode-picker (the open question above) becomes strictly additive**, not a prerequisite —
the command-line interface is complete and usable the day `VisionInterface`/`SampleVisionMonitor`
land, regardless of GUI work landing later or not at all.
## Proposed CV pipeline (classical, no deep learning)
Per the original discussion this grew out of: classical machine vision is enough for a
well-constrained "is the pin geometry right" question; no training data, no model to keep in sync
with hardware changes.
1. **Registration first.** Camera mounts can drift slightly, so raw pixel comparison isn't
meaningful until the current frame is aligned to the reference. `cv2.findTransformECC` for
small translation/rotation (the expected case — a bolted-down camera); an ORB/SIFT +
`cv2.findHomography` fallback if perspective drift turns out to be larger than ECC handles.
2. **Exposure-robust representation.** Compare edges/gradients (Sobel/Canny) or CLAHE-normalized
images, not raw intensity — illumination changes should not look like a defect.
3. **Similarity/anomaly scoring**, two complementary signals:
- **Contour/silhouette match** (`cv2.findContours` + `cv2.matchShapes`) on the extracted
pin/gripper silhouette — the most directly relevant signal for "is the pin sitting right",
and close to fully lighting-invariant.
- **SSIM** (`skimage.metrics.structural_similarity`) on the registered, normalized pair as a
general structural-anomaly fallback, in case a defect doesn't show up as a clean contour
change. (Pulls in `scikit-image` as a new dependency — only if this signal proves worth its
weight; contour matching alone may be sufficient given how constrained the geometry is.)
4. **Multi-reference scoring.** Score against *every* reference image stored under the given tag
and take the best (minimum-distance) match, not a single fixed reference — natural
sample-to-sample and lighting variation shouldn't inflate the anomaly score just because one
particular reference happened to be a bad match.
5. **Escalate only if needed.** If classical contour/SSIM scoring proves insufficiently robust to
subtle real defects, `anomalib` (e.g. PatchCore) is the documented fallback — but per the
original assessment this is very likely overkill for a fixed pin/gripper geometry, and shouldn't
be built preemptively.
## What "getting rid of human confirmations" should actually mean — staged rollout
Physical hardware, real samples, real damage risk (see the top-pin threshold in gate 1 above) means
this should not be an all-or-nothing flag flip. Proposed phases for the later implementation branch:
- **Phase 0 — instrument, don't automate.** Compute and log the vision score at every existing gate
(1–4 above) via `SampleVisionMonitor.score()`; still always show the human the same confirm
prompt, but now alongside the score and the diagnostic diff/overlay visualization it publishes.
Purpose: build a labeled dataset of real operator pass/fail decisions vs. score, across real
lighting and sample variety, before trusting any threshold. No behavior change to the transfer
flow itself.
- **Phase 1 — assist, never silently reject.** A comfortably-high score can auto-answer "yes" at a
gate (with the decision logged and visibly flagged as auto-confirmed, and an easy override); a
low score never fails silently or auto-aborts — it escalates to the same confirm prompt, now with
the diff visualization made prominent, so the human's attention goes exactly where the anomaly
is. This mirrors how the existing top-pin-position check already works (a hard numeric threshold
that *escalates* the confirm text, never silently proceeds or silently aborts).
- **Phase 2 — skip the prompt on high-confidence routine steps.** Only once Phase 0/1 data shows
the scoring is reliable across real operational conditions, let specific gates (candidates: gate
2's "no sample in gripper" check, gate 4's "pick another sample" prompt — the more purely
informational ones) skip the prompt outright on a high score, behind an explicit opt-in/kill
switch. The physical safety checks (top-pin threshold, `_otransfer_check_sensor_connected()`)
stay hard-coded and independent of the vision system regardless of phase — vision augments the
judgment calls, it doesn't replace the interlocks.
## Open questions / decisions needed before implementation starts
- **Reference corpus curation.** Today's saved images are a timestamped append-only run log, not a
curated "golden" set — every run adds a new file, nothing marks one as trustworthy for
comparison. Need either a promotion step (operator marks a saved image as a reference, e.g. via
`save_reference()` above) or a separate deliberately-curated directory, distinct from the
audit-log files `_otransfer_save_reference_images()` already writes.
- **Per-camera-pair segmentation.** Parking/shuttle view (cam200+cam203) and sample-stage view
(cam201+cam202) have different backgrounds/lighting; silhouette extraction likely needs tuning
per pair, not one global set of CV parameters — one `SampleVisionMonitor` instance per pair (its
own config, own reference store) is the natural place for that tuning to live.
- **Threshold calibration.** Can't be picked a priori — needs the Phase 0 logged-score dataset
across real sessions before any auto-confirm threshold is chosen.
- **Where scores/results live.** Likely alongside the existing HDF5 reference-image files (same
per-step file, additional dataset for score + registration transform), rather than a new storage
mechanism.
- **`SampleVisionMonitor` composing a `VisionInterface` vs. owning its own signal directly** — see
the "open composition question" at the end of the architecture section; needs a short prototype of
both before committing.
- **One `SampleVisionMonitor`/`VisionInterface` config per camera pair vs. one instance whose
`camera_names` gets reconfigured at call time** — static per-pair config (e.g.
`omny_vision_parking`, `omny_vision_samplestage` as two fixed device-config entries) is simpler
and matches how OMNY's own two camera pairs are already fixed, not dynamic; leaning toward this
unless a concrete need for a dynamic pairing shows up.
- **A GUI mode-picker for `VisionInterface`** (as opposed to the command-line control, which is
covered — see "Operator interface" above). The existing `smear_preview_toggle`
(`x_ray_eye.py:386-873`) is a single on/off switch between two fixed channels; a `VisionInterface`
with several selectable modes (raw/grayscale/edges/CLAHE/overlay) would need a small mode picker
instead, not just a toggle — new GUI widget surface, not sketched yet, and not a blocker for
anything else in this plan.
- **Overlay compositing vs. the GUI's own annotations.** `x_ray_eye_align.py`'s alignment GUI
already lets the operator click to place points/markers on the live image; a `VisionInterface`
"overlay" mode drawing its own graphics (contours, diff highlights) onto the frame it republishes
must not visually conflict with or obscure those widget-drawn markers — needs checking against the
actual widget code before this mode is built, not assumed compatible.
## Non-goals for this session
No dependencies added, no code written, no new GUI elements built. This document is the handoff for
the follow-up branch.
@@ -0,0 +1,863 @@
# OMNY open issues (as of 2026-09-05)
Tracks the live-testing effort for bringing OMNY up to flomni's/BEC's current standards on
branch `update-omny`: bugs found (fixed and unfixed) and what's still untested. Written so a
future session can pick this up without re-deriving it. Companion docs:
`../../flomni/AI_docs/SIMULATED_ENDSTATIONS.md` (the sim architecture),
`../../flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md` (the flomni→lamni migration this OMNY work
is meant to mirror), and `MACHINE_VISION_AUTOCONFIRM_PLAN.md` (design-only exploration of using
classical machine vision on the sample-transfer camera feeds to reduce/replace the manual confirm
gates in `otransfer_get_sample()`/`otransfer_put_sample()` — not started, separate future branch).
## Current status
A full `omny_fermat_scan` (via `omny.tomo_scan_projection(<angle>)`) now runs end to end against
`simulated_omny.yaml` without crashing — rotation, feedback, and the scan itself all complete.
Getting there took several rounds of live-tested fixes; see the dated sections below for the
full trace of each. GUI flows (`omny.omnygui_show_progress()`, `omnygui_show_xeyealign()`,
`omnygui_show_omnycam_parking()`/`omnygui_show_omnycam_samplestage()`) also work now, ported onto
flomni's current `gui_tools.py` API.
**Now being tested**: the advanced tomo modes (golden ratio, 4/8 sub-tomogram) added earlier this
branch. `holler` (Mirko's account) was added to `_SUPERUSER_ACCOUNTS` (`omny.py`) alongside
`gac-x01dc` so `tomo_parameters()` shows the advanced options when testing from his own machine,
where `bec.active_account` is unset and the superuser check falls back to the OS login. The old
`x12sa` authorization-code prompt gating tomo types 2-5 was removed — it predates the
`_SUPERUSER_ACCOUNTS` account check and was fully redundant with it; see "Authorization code
removed" below. Tomo type 3 also gained a hard cap of 2.5 degrees on the angular step (previously
only a minimum-100-projections floor, which happened to already satisfy this but wasn't expressed
as a wear limit).
**Just landed, confirmed working live**: the tomo-queue backend (Phase 1 of 2 of porting the tomo
params GUI widget to OMNY) — `OMNY` now has a real job queue, `at_each_angle_hook` registration,
`tomo_scan_resume()`, and scan progress visible outside the CLI session (previously a plain
in-memory dict). See "Tomo-queue backend ported to OMNY" below. Live-tested by Mirko against his
own session (not a sim harness), which surfaced and fixed several more real bugs along the way —
see "scilog writes hung / migrated to bec.messaging.scilog", "tomo_scan() rejected
TomoQueueMixin's interactive= kwarg", "stale ~/Data10/ paths crashed on a fresh session", and
"tomo_parameters() now warns about too few Fermat scan points" below. Mirko confirmed: "tomo scan
and tomo queue seems to work now for omny." Also just fixed following his feedback: the progress
ring GUI only had 2 of flomni's 3 rings, and OMNY never tracked/printed measurement time or idle
time the way flomni does at the end of a tomogram -- see "Progress ring GUI + ETA/idle-time
tracking + end-of-scan timing summary" below.
**Just landed, not yet live-tested**: Phase 2 -- the `OMNY_TomoParams` GUI widget itself now
supports OMNY as a third setup (5 tomo types, not just flomni's/lamni's 3). See "TomoParamsWidget
ported to OMNY (Phase 2 of 2)" below. Mirko: "is the way to move to the widget phase" -- this is
that. No live GUI verification yet (needs a real display, same as every other GUI change this
branch) -- next step is Mirko trying it against a real OMNY session.
**Just fixed following Mirko's next round of live testing (from the test checklist)**: a
Ctrl+C during `tomo_queue_resume()`'s GUI-raise call (triggered by a frozen/unresponsive GUI
process) was escaping as a raw `KeyboardInterrupt` instead of being handled gracefully, in two
places -- see "KeyboardInterrupt during a frozen-GUI raise_window() call was not caught" below.
The underlying GUI freeze itself, and a separate "two items in the server queue... after
clearing it also did not resume" report, are not yet explained -- need more detail from Mirko
(see that section's last paragraph).
**Still open / deferred** (unchanged from before, not touched this pass):
- `x_ray_eye_align.py` is still LamNI-derived, unadapted code beyond the two call-site renames
that unblocked the GUI-open crash — see its own section below. The GUI itself opens fine now;
the alignment *logic* underneath it is still wrong for OMNY.
- The full gripper sample-transfer chain (`otransfer_put_sample()`/`otransfer_get_sample()`),
dewar/temperature methods, sample storage bookkeeping, and `omny_alignment_mixin.py`/
`omny_optics_mixin.py` beyond what scans exercise — see "Checklist" section below.
- `omny_alignment_mixin.py`/`x_ray_eye_align.py` still reference the old `~/Data10/specES1/internal/`
path for alignment-offset data (unlike the tomo-scan-path fixes below, these already tolerate a
missing path gracefully — confirmed live, no crash, just "No data available" -- so left alone for
now, but worth the same `~/data/raw/` treatment eventually for consistency).
## Standing question: OMNY should follow flOMNI, not LamNI
Mirko's reaction on seeing the first findings: OMNY should be architecturally closer to
**flOMNI**, not LamNI — the two systems are more alike (OMNY is flOMNI's predecessor/
sibling; LamNI is the more different, laminography-specific setup with e.g. no 180° mode,
different tomo-type scheduling, `tomo_circfov` instead of `fovx`/`fovy`, etc., per
`FLOMNI_TO_LAMNI_COMPARISON.md`). This makes it suspicious whenever an OMNY file reads
like LamNI, and is a general lens to apply going forward: if fixing an OMNY bug means
choosing what the "correct" shape looks like, default to whatever flOMNI already does,
not whatever LamNI does — a LamNI-shaped OMNY file is more likely a leftover copy-paste
than a deliberate design choice. See the `x_ray_eye_align.py` finding below for a concrete
example already found this way, and the `omny_fermat_scan` rotation-crash section further
down for a case where following flomni's exact pattern was the fix.
## `osamroy` rotation offset — resolved (real hardware code was correct; only the simulator was wrong)
Originally logged here as an unresolved, hardware-review-only question: whether
`OMNYGalilMotor.move()` (`devices/omny/galil/ogalil_ophyd.py`) or `feedback_enable_with_reset()`
(`rt_omny_ophyd.py`) had the wrong sign/space for the documented `+25`° read-side offset on the
rotation axis (`OMNYGalilReadbackSignal._socket_get()`), since nothing appeared to subtract it
back out on the write side. **Resolved once the real Galil DMC controller source became
available** (`controller1.dmc`, `contrID=3` handler): the real controller firmware already
applies `ntarget=ntarget-25` on write for this axis — `OMNYGalilMotor.move()` was never buggy.
Only the **simulator** (`csaxs_bec/devices/sim/sim_galil.py`) was missing this, silently passing
through raw un-offset moves. Fixed in `SimGalilState.start_move_from_newpar()`: applies the same
`-25` when the port/axis matches the OMNY rotation stage, mirroring the real controller exactly.
Also seeded `osamroy`'s `sim_initial_position: -25` in `simulated_omny.yaml` so a fresh sim
session starts at displayed 0° instead of 25° (round-trips through the same offset via
`OMNYGalilReadbackSignal`'s `+25` read-side conversion).
Separately, and unrelated to this offset: `omny_rotation()`'s own `.set()` call had a different,
real bug (targeting `osamroy.user_setpoint` instead of the root `osamroy` device), which caused a
readback-progressbar crash later in the scan. See the `omny_fermat_scan` section below — already
fixed, do not confuse with this offset issue.
## `x_ray_eye_align.py` — GUI now opens, underlying logic still wrong
The construction-time crash (`self.lamni.align.reset_correction()` called on an object
still mid-construction) is fixed (matches the fix LamNI's own copy of this file already
got — see its `__init__` for the rationale comment). The two GUI-open call sites
(`self.lamni.lamnigui_show_xeyealign()`/`lamnigui_show_xeyealign_fittab()` — LamNI-only method
names that don't exist on `OMNYGuiTools`) were also fixed, renamed to
`omnygui_show_xeyealign()`/`omnygui_show_xeyealign_fittab()`, so `omny.align.align()` now gets
past the GUI-open step without crashing.
The rest of `omny/x_ray_eye_align.py` is still an **unadapted copy of `LamNI/x_ray_eye_align.py`**,
not code actually written for OMNY:
- Docstring: `"""BEC-GUI-based X-ray eye alignment for LamNI. ... The key difference from
the FlOMNI alignment is that LamNI needs *both* x and y fits..."""` — describes LamNI
vs flOMNI, doesn't mention OMNY at all.
- Type hint: `def __init__(self, client, lamni: LamNI) -> None`, attribute `self.lamni`
used throughout (`self.lamni.align...`, `self.lamni.read_xray_eye_correction_from_gui()`
etc.) — the parameter is actually the `OMNY` instance, not `LamNI`.
- `LAMNI_ALIGNMENT_ANGLES = [k * 45 for k in range(8)]` (8 angles, full 360°, laminography-
specific) — presumably wrong for OMNY, which (per the flOMNI/LamNI comparison doc) is
the kind of full-rotation tomography flOMNI's alignment already handles differently.
- References devices/methods (`lsamrot`, `lfzp_in`, `omnyfsh`, `omny_xray_gui`) that mix LamNI
and OMNY naming conventions — needs a careful pass against OMNY's actual device inventory.
None of the alignment logic itself has been exercised live yet — only the constructor and the
GUI-open step. Given the "OMNY ≈ flOMNI, not LamNI" principle above, the right fix is almost
certainly to port OMNY's version from `flomni/x_ray_eye_align.py` (440 lines, the flOMNI-specific
implementation) the way `LamNI/x_ray_eye_align.py` was itself adapted from flOMNI's, rather than
patching the current LamNI-derived file in place. Not started.
## Checklist items not yet run
- **Passed** (live, against `simulated_omny.yaml` / `bec --session omny`): config load,
translation-axis moves (`osamx`, `osamy`), a lightweight transfer (`omny.otransfer_park_slot(6)`),
`feedback_enable_with_reset()`, a full `omny_fermat_scan` via `omny.tomo_scan_projection(<angle>)`
end to end, and the GUI flows listed under "Current status" above.
Still untested (not attempted yet, not known-broken — just unexplored):
- **Advanced tomo modes** (golden ratio, 4/8 sub-tomogram, `tomo_parameters()`'s superuser-gated
options) — about to be tested next.
- The **full gripper sample-transfer chain**: `otransfer_put_sample()` /
`otransfer_get_sample()` (the actual pin pickup/dropoff with the `confirm`/`atconfir`
handshake), as opposed to the lighter `otransfer_park_slot()` used so far.
- **X-ray eye alignment** end-to-end alignment logic (GUI now opens fine, see section above —
the actual click-through alignment procedure itself hasn't been exercised).
- **Dewar / temperatures** (`omny_dewar`, `omny_temperatures` devices) — they load and
construct (`SimOMNYDewar`/`SimOMNYTemperatures`), but no `omny.py`-level method that
reads/uses them has been called live.
- **Sample storage** (`omny_samples`) beyond it loading with seeded slots — no
`omny.py`-level storage-bookkeeping method exercised.
- `omny_alignment_mixin.py`, `omny_optics_mixin.py` beyond what a fermat scan's own setup touches.
## Known, already-documented, separate from this session
- `omny_webpage_generator.py` (2957 lines) is dead/unwired code — never instantiated by
`omny.py`, per `FLOMNI_TO_LAMNI_COMPARISON.md`'s note ("omny was left untouched...
separate future task if omny ever needs a real one"). Not touched this session either.
- **Tomo-queue backend + widget: both ported (Phase 1 and 2, this branch).** `OMNY` inherits
`TomoQueueMixin` (`OMNY_shared/tomo_queue_mixin.py`), the same shared backend flomni/lamni
use — `tomo_queue_add/delete/move/clear/show/execute/reacquire`,
`register_at_each_angle_hook`/`at_each_angle_hook`, `tomo_scan_resume()`. The
`OMNY_TomoParams` GUI widget (`bec_widgets/widgets/tomo_params/tomo_params.py`) now has a
`SETUP_PROFILES["omny"]` entry and its tomo-type selector/job-display logic supports OMNY's 5
tomo types (flomni/lamni only ever had 3). See "Tomo-queue backend ported to OMNY" and
"TomoParamsWidget ported to OMNY (Phase 2 of 2)" below for the full detail.
## Progress ring GUI + ETA/idle-time tracking + end-of-scan timing summary (fixed)
Mirko live-tested Phase 1 and asked two related questions: why does OMNY's progress ring bar show
only 2 rings where flomni shows 3, and why does OMNY not print a measurement-time/time-lost summary
at the end of a tomogram the way flomni does. Both traced back to the same root gap: nothing in
OMNY ever computed `estimated_remaining_time`/`estimated_finish_time`, and nothing ever refreshed
`heartbeat` *during* a running scan (only cleared it to `None` on completion, from the Phase-1
port) -- so `accumulated_idle_time` stayed `0.0` forever and there was no ETA data to show or
summarize. All direct ports of flomni's already-working mechanism, all sharing the same underlying
data (`self.progress`, global-var-backed since Phase 1):
- **`_tomo_scan_at_angle()`** now refreshes `progress["heartbeat"]` at the start of every angle,
and -- if the gap since the previous heartbeat exceeds a normal-cadence heuristic
(`max(60s, 5 * tomo_countingtime)`, flomni's identical formula minus its `frames_per_trigger`
factor, which OMNY has no equivalent property for) -- attributes the excess to
`accumulated_idle_time` and prints a one-line note. This is what actually makes idle-time
tracking (and therefore both features below) possible; it didn't exist in Phase 1 at all.
- **`_print_progress()`** now computes and stores `estimated_remaining_time`/
`estimated_finish_time` from elapsed time minus accumulated idle time (once past projection 9,
matching flomni's stabilization threshold), and prints them alongside the existing progress
report.
- **`tomo_scan()`** now prints an end-of-scan summary ("Total measurement time", "... excluding
detected gaps", "... lost to detected gaps") after the try/finally, and sends the same content to
scilog via `write_to_scilog()` -- direct port of flomni's block, minus the flomni-only
`self.measured_log` sample-tracking call (OMNY has no equivalent).
- **`gui_tools.py`**: `omnygui_show_progress()` now adds a third ring with `set_update("scan")`
(auto-updated from the currently running scan, no manual `.set_value()` needed -- matches
flomni's `flomnigui_show_progress()` exactly), and `_omnygui_update_progress()`'s center-label
text now includes start time / ETA / estimated finish / active-hook lines, matching flomni's
`_flomnigui_update_progress()`.
`_format_duration()` (static, `"2h 03m 15s"`-style formatting) added to `omny.py` -- previously
flomni-only. `_describe_active_hook()`/`_active_hook_source()` needed no porting: already provided
by `TomoQueueMixin` since Phase 1.
**Follow-up bug found while porting this**: Mirko reported that on macOS, the progress window
appeared to close and reopen on every single projection during a scan -- "highly annoying".
`_tomo_scan_at_angle()` was calling `self.omnygui_show_progress()` (the full show-window-and-raise-
to-front routine) once per angle, not just `_print_progress()`/`_omnygui_update_progress()` (values
and text only). flomni's own `_tomo_scan_at_angle()` never re-raises the window per angle either --
it only ever calls `_print_progress()`, which internally calls `_flomnigui_update_progress()` at
its own end. Fixed by mirroring that structure exactly: `_print_progress()` now calls
`self._omnygui_update_progress()` itself, and the separate per-angle `omnygui_show_progress()` call
was removed from `_tomo_scan_at_angle()` -- the window is still shown/raised exactly once, by
`tomo_scan()`'s own unconditional call at scan start.
**Not ported**: flomni's separate `_log_tomogram_timing()`/persistent timing-statistics-log
subsystem (writes a JSON record per tomogram to `~/data/raw/logs/timing_statistics`, backing
`scilog_last_ptycho_scans()`/a timing overview command) -- Mirko asked specifically about the
*printed* summary and the ring count, not this larger, separate logging feature. Flag if wanted.
Test coverage: 10 new tests (`test_omny_tomo_scan.py`: idle-time-from-gap, no-idle-time-within-
cadence, end-of-scan summary lines, window not re-raised per angle; `test_omny_gui_tools.py`:
three-ring creation, center-label text with/without an active hook). Full suite: 732 passed.
## cam_xeye device (simulated only, real hardware config still needed)
`omnygui_show_xeyealign()` uses the shared `OMNY_XRayEye` widget
(`csaxs_bec/bec_widgets/widgets/xray_eye/x_ray_eye.py`), which hardcodes `CAMERA = ("cam_xeye",
"image")` as a module-level constant -- the device name isn't configurable per-beamline. omny's
config had no `cam_xeye` device at all, so simply calling `omnygui_show_xeyealign()` failed with
`ValueError: Device 'cam_xeye' not found in current BEC session` as soon as the widget was
constructed. Added a `cam_xeye` entry to `simulated_omny.yaml` (hand-added, not generated --
`ptycho_omny.yaml`, the generator's source, has no such device), reusing flomni's real `cam_xeye`'s
`camera_id`/`pixel_calibration` as placeholders (`SimIDSCamera` accepts both `camera_id` and the
legacy `camera_ID` key omny's other sim cameras use). **Two follow-ups still needed before real
hardware use:**
- `ptycho_omny.yaml` needs a real `cam_xeye` entry with OMNY's actual x-ray-eye camera hardware
`camera_id` and measured `pixel_calibration` (mm/pixel) -- unknown to me, needs Mirko.
- `simulated_omny.yaml` is regenerated by `bin/generate_simulated_configs.py` from
`ptycho_omny.yaml`; since `cam_xeye` isn't in the real config yet, a regen will drop this
hand-added block until the real config also has one (matches the existing pattern for
`sim_velocity`/`sim_initial_position` hand-additions elsewhere in this file, already present
before this session).
## omny_fermat_scan rotation crashed on an arbitrary device (readback progressbar) (fixed)
`omny.tomo_scan_projection(1)` -> `omny_fermat_scan` crashed with `ValueError: '<some device>' is
not in list` in `bec_ipython_client`'s `DeviceProgressBar.set_finished()`, once a scan actually did
a readback-tracked osamroy rotation. First seen as `cam200`; changing cam200-203/cam_xeye's
`readoutPriority` to `on_request` (below) did **not** fix it -- it just changed which device the
crash landed on (`rt_positions`, confirmed by live reproduction). The `readoutPriority` change is
still correct/kept (matches flomni's convention for view-only cameras), but it was treating a
symptom, not the actual bug.
**Real root cause, confirmed via live reproduction (local BEC deployment, not just static
reading)**: `omny_fermat_scan.py`'s `omny_rotation()` called
`self.actions.set(self.dev.osamroy.user_setpoint, angle, wait=False)` -- passing the
`user_setpoint` **signal sub-component**, not the root device. `ScanActions._normalize_device_name`
(`bec_server/.../scan_actions.py:1106-1107`) normalizes a `DeviceBase` to its **dotted name** for a
sub-signal, i.e. `"osamroy.user_setpoint"`, while `add_scan_report_instruction_readback(devices=
["osamroy"], ...)` (line above) registers the plain string `"osamroy"` as requiring a response.
Since `{"osamroy.user_setpoint"} & {"osamroy"}` is empty, `_send()` never sets `response=True` on
the rotation's own instruction -- the device server never reports `"osamroy"` done, so the readback
progressbar's `ReadbackDataHandler` never sees its request finish and stays subscribed to that RID
for the rest of the scan. Later, `post_scan()`'s `complete_all_devices()` batches many unrelated
owned devices into one instruction; because the literal string `"osamroy"` *is* in that later batch
and still needs a response, `response=True` fires on the **whole batch**, and the device server
reports every device in it -- which the still-alive progressbar (only tracking `["osamroy"]`)
receives and crashes on, for whichever device happens to report first (arbitrary -- hence `cam200`
for Mirko, `rt_positions` in the live repro here).
**Confirmed this is omny-specific, not a general/flomni-shared bug** (contrary to my earlier
assumption): `flomni_fermat_scan.py`'s equivalent `flomni_rotation()` calls
`self.actions.set(self.dev.fsamroy, angle, wait=False)` -- the whole root device, not a sub-signal
-- so `_normalize_device_name` returns `"fsamroy"`, matching the required-response registration
exactly. Live-reproduced: `flomni_fermat_scan` with `cam_xeye` still `readoutPriority: async` and
enabled completed cleanly (no crash), because the rotation's own completion reports correctly and
the progressbar exits well before `complete_all_devices()` ever runs.
**Fixed** in `omny_fermat_scan.py`'s `omny_rotation()`: `self.actions.set(self.dev.osamroy, angle,
wait=False)`, matching flomni's pattern exactly. `OMNYGalilMotor.move()`
(`csaxs_bec/devices/omny/galil/ogalil_ophyd.py`) already does `self.user_setpoint.put(...)` plus
proper completion bookkeeping internally, so setting the root device is a strict superset of the
previous behavior, not a functional change to the move itself.
**Not fixed, out of scope, reported upstream as feedback (defense-in-depth, framework-level)**:
`ReadbackDataHandler.on_req_status()` and `DeviceProgressBar.set_finished()`
(`bec_ipython_client`) should guard against/ignore a device outside their tracked list rather than
crash, and `scan_actions.py`'s `_send()` should scope the `response` flag per-device rather than
per-batched-instruction -- any scan with this same signal-vs-root-device mismatch pattern would hit
the same failure mode.
**Note on the earlier `readoutPriority: on_request` change (commit `c37bfc7`)**: once this fix
landed, that change was no longer load-bearing for this crash -- the rotation's own readback
progressbar now closes out right after the move, so it's no longer still listening when
`complete_all_devices()` broadcasts later, regardless of cam200-203/cam_xeye's readout priority.
Asked Mirko whether to revert it back to `async` (original behavior, cameras' frames get
continuously logged into every scan) now that it's not strictly needed; he chose to **keep
`on_request`** -- these cameras are only ever used for live GUI viewing (`start_live_mode()`), never
read back in any scan code, so continuous per-scan logging was likely never useful. Not reverted;
this is a deliberate choice, not leftover cruft.
## laser_tracker_check_and_wait_for_signalstrength() was chatty (fixed)
Live testing of the first successful omny scan surfaced two odd things at once: a "Summary of
client messages" printout Mirko had never seen, and client-info messages ("Checking laser
tracker...") getting reprinted on a later, unrelated call (`tomo_parameters()`). Root cause of the
*reprint*, live-diagnosed: a genuine `bec_lib` bug (`QueueItem.get_client_messages()` mutates its
list while iterating -- reported upstream). But the reason it was even *noticeable* is
omny-specific: `laser_tracker_check_and_wait_for_signalstrength()`
(`csaxs_bec/devices/omny/rt/rt_omny_ophyd.py`) used `send_client_info()` for its
"Checking..."/"...completed." pair, which queues onto the scan's client-message list (and is thus
exposed to the `get_client_messages()` bug), whereas flomni's equivalent,
`laser_tracker_check_signalstrength()` (`rt_flomni_ophyd.py`), has always used plain `print()`
(server-side only, never queued, never exposed to that bug). Both mirror each other structurally
in one other way too: `RtOMNYSetpointSignal._socket_set()`/`RtFlomniSetpointSignal._socket_set()`
each run this check once per RT axis set -- so a single scan setup (rtx + rtz set together, plus,
until this fix, an extra explicit call in `omny_fermat_scan.py`'s `prepare_setup_part2()`)
triggered the check 2-3 times, each producing 2 messages on omny's chatty version vs 0 visible
output on flomni's silent one.
**Fixed**: switched `laser_tracker_check_and_wait_for_signalstrength()` to plain `print()`
throughout (matching flomni exactly -- Mirko's explicit call), and removed the now-redundant
explicit `laser_tracker_check_and_wait_for_signalstrength()` call in `prepare_setup_part2()`
(pure duplication of the per-axis checks the rtx/rtz `set()` calls already trigger). Net effect:
quieter output, one less redundant tracker-check wait per scan setup, same safety behavior.
## tomo_parameters(): authorization code removed, type-3 step cap added (fixed)
Advanced tomo modes (types 2-5) used to require entering a hardcoded string (`x12sa`) at an "Enter
authorization code" prompt, on top of already being hidden from non-superuser accounts by
`_is_superuser_account()`/`_SUPERUSER_ACCOUNTS` (added earlier this branch). Once account-based
gating existed, the code prompt was pure redundancy — Mirko pointed this out directly ("we can
remove that now that it is based on the user account") — so it's been removed from all three
blocks (types 2, 3, 4/5). Each block still prints the same "This mode causes significant wear in
OMNY" wear warning as before (now unconditional, not gated behind entering the code), reworded
slightly to explicitly address the power user.
Also added: tomo type 3 (equally spaced tomography, golden ratio starting angle) now enforces a
hard ceiling of 2.5 degrees on the angular step, in addition to just deriving the step from the
requested projection count. Previously it only enforced a minimum-100-projections floor (step
≤1.8°), which happened to already satisfy a 2.5° ceiling — but expressed the constraint as a
projection-count floor rather than a wear-motivated angular ceiling. If the requested projection
count yields a step above 2.5°, the step is clamped down to 2.5° and a message is printed; the
actual number of projections used ends up higher than what was typed. Type 2's
`golden_ratio_bunch_size` minimum was left at 100 (unchanged) — the wear-avoidance goal there is
served by the warning message, not by loosening that floor.
Test coverage: `tests/tests_bec_ipython_client/test_omny_tomo_angles.py`'s
`test_tomo_parameters_allows_superuser_to_unlock_8_subtomo_mode` updated to drop the now-removed
`"x12sa"` canned answer; new
`test_tomo_parameters_clamps_type_3_step_size_to_max_2_5_degrees` covers the clamp.
## Tomo-queue backend ported to OMNY (Phase 1 of 2)
Mirko wants the tomo params GUI widget available for OMNY before starting real tomo scans. That
widget (Phase 2, separate follow-up) depends on a working tomo-queue backend and on scan progress
being visible outside the CLI session that started it -- neither existed for OMNY before this pass.
Same shared infrastructure and pattern as the flomni -> lamni port
(`LamNI/AI_docs/TOMO_QUEUE_PORT.md`, `TOMO_PARAMS_GUI_PORT.md`), done as the same two-phase split
per Mirko's explicit choice.
**`OMNY` now inherits `TomoQueueMixin`** (`OMNY_shared/tomo_queue_mixin.py`, first in the MRO,
matching flomni/lamni), giving it `tomo_queue_add/add_command/delete/move/clear/show/execute/
resume/reacquire`, `register_at_each_angle_hook`/`unregister_at_each_angle_hook`/
`list_at_each_angle_hooks`/`at_each_angle_hook` for the first time. New class attributes:
`_TOMO_SCAN_PARAM_NAMES` (mirrors flomni's names exactly -- OMNY's properties already mirror
flomni's 1:1, no `tomo_angle_range`/`single_point_*`/`zero_deg_reference_at_each_subtomo`/
`frames_per_trigger` since none of these exist as OMNY properties), `_TOMO_QUEUE_MOVE_DEVICES`
(`ccm_energy`+`idgap`, **not** `mokev` -- OMNY has no `mokev` device in any config, real or
simulated), `_TOMO_QUEUE_ACTIONS` (same `move`/`optimize_idgap` registry as flomni/lamni,
`optimize_idgap` still a stub).
**`self.progress` is now a global-var-backed proxy, not a plain dict.** Before this pass, OMNY's
`self.progress = {}` was a plain in-memory instance dict -- invisible to any other BEC client
session, a GUI process included. New `_ProgressProxy` class (direct, byte-for-byte port of
flomni's/lamni's own, backed by `client.get_global_var("tomo_progress")`, same global-var key --
only one tomography setup is ever live per BEC session, no collision risk), with `progress` now a
`@property`. Carries the original 7 fields (`subtomo`, `subtomo_projection`,
`subtomo_total_projections`, `projection`, `total_projections`, `angle`, `tomo_type`) plus
`tomo_start_time`/`estimated_remaining_time`/`estimated_finish_time`/`heartbeat`/
`accumulated_idle_time`, matching flomni's/lamni's `_DEFAULTS` shape.
**`tomo_queue_mixin.py`'s `tomo_queue_reacquire()` generalized (small, additive, behavior-
preserving change to shared code)**: it hardcoded `if self.tomo_type == 1:` to decide when to
resolve via `_resolve_type1_projection()`. OMNY has three "equally spaced sub-tomograms" flavors
(types 1/4/5 -- 2/4/8 sub-tomograms), not just type 1. Added `_TOMO_TYPE_1_LIKE: frozenset`
(default `frozenset({1})`, so flomni/lamni are unaffected -- confirmed via their existing test
suites, unchanged and passing), which OMNY overrides to `frozenset({1, 4, 5})`.
**New OMNY methods, direct ports generalized for 3 sub-tomogram counts**: `tomo_scan_resume()`
(dispatches subtomo/angle for tomo_type in `_TOMO_TYPE_1_LIKE`, projection_number for 2/3) and
`_resolve_type1_projection()` (reuses the already-verified `_equally_spaced_subtomo_angles()` --
does not re-derive the angle grid a second way).
**Real, pre-existing bugs fixed along the way** (same shapes flomni/lamni already needed and got,
found by reading `tomo_scan()`/`write_pdf_report()` against the mixin's needs, not newly
introduced):
- `bec.active_account.decode()` in `tomo_scan()` -- `active_account` is a plain `str`, not
`bytes`; crashed on every real account. Removed (OMNY's existing `if bec.active_account != "":
... else: tomo_id = 0` branch structure kept as-is, unlike flomni's/lamni's "always attempt,
let add_sample_database() decide" pattern -- OMNY's own `add_sample_database()` has no
test-server/empty-account fallback to lean on).
- `dev.mokev.read()['mokev']['value']` in `write_pdf_report()` -- OMNY has no `mokev` device in
any config. Fixed to `dev.ccm_energy` (from `bl_optics_hutch.yaml`, always included regardless
of active endstation) with a try/except -> `"N/A"` fallback, matching lamni's identical fix.
Confirmed `ccm_energy` is *not* in the local simulated OMNY config either (`simulated_omny.yaml`/
`simulated_bl_endstation.yaml` have no such device), so live sim testing will exercise the
fallback path, not a real energy read -- still correct/safe behavior, just not the success path.
- `bec.logbook.LogbookMessage()`/`send_logbook_message()` in `write_pdf_report()` was unguarded --
wrapped in try/except, matching `write_to_scilog()`'s existing tolerance one method up.
- `write_pdf_report()`'s scilog logo/tags still said `"LamNI"`/pointed at a nonexistent
`LamNI_logo.png` (omny's plugin directory only has `OMNY.png`) -- `msg.add_file(logo_path)`
would have raised unguarded (now inside the same try/except above regardless). Fixed to
`OMNY.png`/`"OMNY"`.
- `tomo_scan()` never called `self.omnygui_show_progress()` unconditionally at scan start (only
per-angle inside `_tomo_scan_at_angle()`) and had no try/finally to clear `heartbeat` on exit --
both fixed, matching flomni's/lamni's identical pattern.
- `_at_each_angle()` now checks `self.at_each_angle_hook` (mixin-provided) first, falling through
to the pre-existing `omny_at_each_angle` builtins-global hook, then the default
`tomo_scan_projection()` + `tomo_reconstruct()` -- same 3-tier structure as flomni's, minus its
single-point branch (OMNY has none).
**Verification**: new `tests/tests_bec_ipython_client/test_omny_tomo_queue.py` (17 tests --
param-name snapshotting, queue add/show/execute/resume/reacquire, add-command move validation,
`at_each_angle_hook` dispatch/default/legacy-builtins-fallback/unregistered-name,
`tomo_scan_resume()`/`_resolve_type1_projection()`/`tomo_queue_reacquire()` correctness across all
three sub-tomogram flavors) and `test_omny_tomo_scan.py` (6 tests -- account handling, progress-GUI
auto-show, heartbeat clearing on normal completion and mid-scan exception, new-scan timing-field
resets). `write_pdf_report()`'s `ccm_energy` fallback has no dedicated unit test, matching
flomni's/lamni's own precedent (neither has one either -- verified live only, real file I/O and
device reads make it impractical to unit test in isolation). Full suite: 702 passed. Confirmed via
plain Python (no live client needed) that `OMNY`'s MRO resolves cleanly with `TomoQueueMixin`
first, no conflicts. **Not yet live-verified against a real running session** (deliberately --
the local bec-server may be in active use by Mirko's own parallel session; a full live
`tomo_scan()`/`tomo_queue_execute()` run moves real simulated hardware and consumes real scan
numbers) -- next step is Mirko pulling and testing live himself, same pattern as this whole
engagement.
## Live-testing findings after the Phase 1 port (fixed)
Mirko live-tested Phase 1 against his own session (not a sim harness) and found four more real
bugs, all fixed in a follow-up pass:
**scilog writes hung indefinitely.** `write_to_scilog()`/`write_pdf_report()` used the old
`bec.logbook`/`client.logbook.send_logbook_message()` API, which has no "is scilog even
configured" check -- on a session without scilog credentials, `send()` blocks forever instead of
failing fast. Confirmed live: a real `tomo_scan()` hung for 29+ minutes inside
`_write_subtomo_to_scilog()`. Per Mirko's direction ("Scilog Messaging should check the account.
In flomni we are doing this I believe"): flomni's actual mechanism turned out to be
`bec.messaging.scilog._enabled` (not literally an account check, but the same effect -- gates the
send behind whether scilog is actually configured/reachable). Migrated both OMNY call sites onto
the same `bec.messaging.scilog` API, matching flomni's `_scilog_write()`/`write_pdf_report()`
exactly. 3 new unit tests lock in the `_enabled` gate.
**`tomo_scan()` didn't accept `interactive=`.** `TomoQueueMixin.tomo_queue_execute()`
unconditionally calls `self.tomo_scan(interactive=False)` for unattended queued runs (the same
call convention flomni/lamni already support) -- `OMNY.tomo_scan()`'s signature never had this
parameter at all, so every `tomo_queue_execute()` run failed immediately with `TypeError: got an
unexpected keyword argument 'interactive'`. Fixed by adding `interactive: bool = True`, accepted
but currently unused -- matching flomni's own docstring reasoning exactly ("FlOMNI has no
fine-alignment gate"; OMNY doesn't either).
**Stale `~/Data10/specES1/` paths crashed on a fresh session.** Confirmed live (`FileNotFoundError`
on Mirko's Mac): `_write_tomo_scan_number()` wrote to
`~/Data10/specES1/dat-files/tomography_scannumbers.txt`, a layout that doesn't exist in the current
deployment at all -- flomni's own equivalent method already uses `~/data/raw/logs/`. Same fix
applied to `write_pdf_report()`'s PDF target (`~/Data10/documentation/` ->
`~/data/raw/documentation/`) and `tomo_reconstruct()`'s default `base_path` (`~/Data10/specES1` ->
`~/data/raw`), all three of which shared the identical stale root. Unlike flomni's own
`_write_tomo_scan_number()`/`write_pdf_report()` (which assume the target directory already
exists, and would hit this same crash on a machine where it doesn't), OMNY's versions now
explicitly `os.makedirs(..., exist_ok=True)` first -- a small hardening beyond a pure mirror,
directly motivated by this exact crash.
**`tomo_parameters()` now warns about too few Fermat scan points.** `OmnyFermatScan.prepare_scan()`
already aborted (`ScanAbortion`) when the fovx/fovy/step combination yields fewer than
`MIN_POSITIONS` (20) points -- that abort was intentional, but nothing warned ahead of time; the
scan would just silently fail once actually run. Direct port of `Flomni.tomo_parameters()`'s
identical warning line, added right after "Stitching overlap" in the printout: "Estimated Fermat
scan points per projection: N (WARNING: below the minimum of 20 -- the scan will abort when run)".
Needed a small supporting refactor: `OmnyFermatScan.get_omny_fermat_spiral_pos()` was an instance
method reading `self.cenx`/`self.ceny`/`self.zshift` (unlike flomni's already-`@staticmethod`
version), so it couldn't be called from `tomo_parameters()` without constructing a full scan
instance -- converted to a `@staticmethod` taking `cenx`/`ceny`/`zshift` as explicit keyword args
(default `0.0`, behavior-preserving), matching flomni's signature exactly. Also promoted the
hardcoded `20` in `prepare_scan()`'s abort check to a `MIN_POSITIONS = 20` class constant (mirrors
`FlomniFermatScan.MIN_POSITIONS`), so the CLI warning and the scan server's actual enforcement can
never drift out of sync. 13 new tests across `test_omny_fermat_scan.py` (scan-level:
`MIN_POSITIONS` value, static-method callability, cenx/ceny/zshift not affecting the count,
`prepare_scan()` still aborts) and `test_omny_tomo_angles.py` (CLI-level: warning appears/doesn't
appear).
## TomoParamsWidget ported to OMNY (Phase 2 of 2)
Mirko, once Phase 1 was live-confirmed working: "is the way to move to the widget phase" -- this
is Phase 2, adding OMNY as a third setup to the `OMNY_TomoParams` GUI widget
(`bec_widgets/widgets/tomo_params/tomo_params.py`). The widget was already almost entirely
`SETUP_PROFILES`-driven (two research passes plus a verification pass against the live 2476-line
file found essentially everything generalizes mechanically), with one real complication: flomni
and lamni each have exactly one "equally spaced sub-tomograms" tomo type (type 1, always 8
sub-tomograms); OMNY has three (types 1/4/5 -- 2/4/8 sub-tomograms).
**Key design decision**: replaced the module-level `TOMO_TYPES` dict (shared across all setups,
couldn't express "type 1 means 2 sub-tomograms here but 8 there") with a per-profile
`"tomo_types"` dict: `{tomo_type_int: {"label": str, "n_subtomos": int | None}}`.
`n_subtomos is not None` became the single, uniform test for "is this an equally-spaced type"
everywhere in the file (combo box, section visibility, validation, queue-table projections
display) instead of duplicating an `{1,4,5}`-shaped literal two or three times.
**Critical bug caught by a verification pass, not the initial design**: `_validate()` had
`if params.get("tomo_type") not in (1, 2, 3): return "tomo_type must be 1, 2, or 3"` -- a
hardcoded closed-enumeration check that the rest of Phase 2 would have sailed past (types 4/5
selectable, sections visible, math correct) while **silently rejecting every actual submit/queue
attempt** for an OMNY type-4/5 job. Fixed to check `self._profile["tomo_types"]` membership
instead. Also caught: three UI strings hardcoded "8" ("must be divisible into 8 equal
sub-tomograms" in the projection-preview tooltip and both confirmation dialogs) -- all three now
read the active type's real `n_subtomos`.
**New module-level OMNY math functions** (`_compute_type1_omny`/`_requested_to_stepsize_omny`),
mirroring `_compute_type1`/`_requested_to_stepsize` but parameterized by `tomo_type` (using
`{1:2, 4:4, 5:8}`, matching `OMNY._equally_spaced_subtomo_angles()`'s own mapping in omny.py
exactly) instead of a hardcoded `*8`. `_compute_fermat_positions_omny` calls
`OmnyFermatScan.get_omny_fermat_spiral_pos()`/`MIN_POSITIONS` directly, same pattern as flomni's.
`_format_projections()` (module-level, only has a job's raw `params` dict, not `self._profile`)
gained a 3-way setup duck-type: `"tomo_angle_range" in params` -> flomni; else `"fovx" in params`
-> omny (OMNY's param set is a strict subset of flomni's -- no positively-unique OMNY-only key
exists, so this negative-then-positive check is necessary, not a shortcut); else -> lamni. Once
resolved, it looks up `SETUP_PROFILES[setup]["tomo_types"]` rather than hand-picking a compute
function, avoiding a third copy of the type-count mapping.
**Confirmed needing zero changes** (verified against the live file, not assumed): `_job_tooltip()`/
`_irrelevant_params_for_type()` (OMNY's params never overlap the flomni-only keys they filter on,
so the existing `!= 1` check has no observable effect for OMNY regardless); `_build_type23_section()`
(golden-ratio-only, already type-count-agnostic); `compute_fermat_positions` call sites (fermat math
never depends on tomo_type); `_detect_setup()` (purely `discriminator_attr`-driven, `"osamroy"` for
OMNY); the busy-banner detection methods (already read the shared `tomo_progress` global var/
scan-queue-status generically, Phase 1 already made OMNY use the same key); the Qt Designer plugin
registration files (already fully generic, self-detect setup at construction).
Test coverage: new `test_omny_tomo_params_widget_math.py` (29 tests, pure-function -- no
`QApplication`/`QWidget`, same style as the existing flomni/lamni widget-math test files),
covering the new compute functions against the reference formula, `tomo_types` n_subtomos mapping,
`param_names`/`defaults` parity with `OMNY._TOMO_SCAN_PARAM_NAMES`, `_format_projections()`'s 3-way
dispatch (including a regression guard that it still correctly distinguishes flomni/lamni jobs),
`_compute_fermat_positions_omny` against the real scan class, and a standalone regression guard for
the `_validate()` fix. Full suite: 761 passed.
**Not live-tested**: needs a real display (same as every GUI change this branch) -- next step is
Mirko trying it against a real OMNY session: switching between all 5 tomo types (confirm the
equally-spaced section shows/hides correctly and the new sub-tomogram-count label updates),
submitting/queueing a type-4 or type-5 job (confirms the `_validate()` fix specifically), and
checking a queued job's tooltip/Projections column in the queue dialog.
## KeyboardInterrupt during a frozen-GUI raise_window() call was not caught (fixed)
Mirko, running from the tomo queue on his own machine: aborted a running scan with Ctrl+C
(worked correctly -- clean `ScanInterruption: User abort`, queue paused, job marked
`"incomplete"`), then called `omny.tomo_queue_resume()` twice; the second call, while
`tomo_scan_resume()` was starting up, produced a **raw Python `KeyboardInterrupt` traceback**
instead of BEC's usual graceful abort. Separately: "the gui progress and tomo parameters was
open and both a frozen on my local machine", and "i had two items in the server queue... after
clearing it also did not resume."
Root cause, read straight off the traceback: the stack bottomed out in
`self.gui.omny.raise_window()` -> `bec_widgets`' `RPCBase._run_rpc()`, blocked on
`self._msg_wait_event.wait(...)`. `tomo_scan()` calls `omnygui_show_progress()`
(-> `omnygui_show_gui()` -> `raise_window()`) unconditionally as its first line, on every call
including `tomo_scan_resume()`. When the GUI process itself is frozen/unresponsive (matching
Mirko's separate "both frozen on my local machine" report), that RPC call doesn't hang forever --
`bec_widgets`' default RPC timeout is 60s (`BECGuiClient._rpc_timeout`, `client_utils.py`), after
which it raises a clean `RPCResponseTimeoutError` -- but 60s is a long, silent wait for an
operator watching a terminal, and Mirko understandably lost patience and hit Ctrl+C himself
during that wait. `KeyboardInterrupt` is a `BaseException`, not an `Exception` subclass, so it
sailed straight past two different `except Exception` handlers that were supposed to catch
exactly this kind of trouble:
1. `omnygui_show_progress()` (`gui_tools.py`) already wrapped the whole show/raise sequence in
`try/except Exception` specifically so a GUI hiccup can never abort a scan -- but a
`KeyboardInterrupt` escaped it outright, propagating all the way up through
`tomo_scan_resume()`. **Fixed**: added a dedicated `except KeyboardInterrupt` clause that logs
a warning and continues without raising the window -- bringing the window to front is a
nicety, not something the scan itself needs.
2. `tomo_queue_execute()` (`tomo_queue_mixin.py`)'s `except Exception as exc:` around the
`tomo_scan()`/`tomo_scan_resume()` call is what marks a job `status="incomplete"` and prints
the informative "Tomo queue job '...' did not complete: ... Queue paused..." message -- the
same message Mirko saw work correctly on the *first* abort. A `KeyboardInterrupt` escaping this
handler leaves the job silently stuck at `status="running"` with no explanation printed at all,
which is exactly what "I had two items in the server queue. I do not have details" describes.
**Fixed**: widened to `except (Exception, KeyboardInterrupt) as exc:`, so any interruption --
not just a raised `Exception` -- gets the same clean status update and message.
Both fixes are narrowly scoped (they don't change how Ctrl+C is handled *during* an actual running
scan -- BEC's own deferred-pause/abort alarm flow for that is untouched) and only change what
happens when the interrupt lands during the pre-scan GUI-raise window. Regression tests:
`test_omnygui_show_progress_swallows_keyboardinterrupt_from_frozen_gui` (`gui_tools.py`'s
`raise_window()` mocked to raise `KeyboardInterrupt`, asserts `omnygui_show_progress()` doesn't
propagate it) and `test_tomo_queue_execute_marks_job_incomplete_on_keyboardinterrupt`
(`tomo_scan` mocked to raise `KeyboardInterrupt`, asserts the job ends up `"incomplete"` not
`"running"`, and that the interrupt still propagates to the caller afterward -- it's not
swallowed, just handled the same way any other mid-job exception already was).
**Follow-up, fully explained and fixed**: Mirko confirmed "i was talking becs own queue" (BEC's
own scan-server execution queue, not `tomo_queue`), and separately noticed "the gui is essentially
freezing while a scan is running... pressing queue control in the params gui, the window only
opened after the scan finished." Caught live with a stuck queue: `bec.queue.queue_storage
.describe_queue()` showed **two duplicate `device_rpc` entries stuck PENDING**, both
`device='omny_samples', func='get_sample_name_in_samplestage'` (different RIDs -- two separate
invocations), plus one `func='stop'` (routine BEC abort cleanup, unrelated), all stuck because the
queue was `PAUSED` from the earlier abort and never got serviced.
Root cause: `OMNY_TomoParams`'s poll timer (`_on_poll()` -> `_refresh_sample_name()`,
`tomo_params.py`) calls `SETUP_PROFILES["omny"]["sample_name_getter"]` on every tick.
It was `lambda w: w.dev.omny_samples.get_sample_name_in_samplestage()` -- a device *method* call,
which BEC dispatches as a `device_rpc` request through the scan queue (a real round-trip), unlike
flomni's getter (`w.dev.flomni_samples.sample_names.sample0.get()`, a direct signal read) or
lamni's (`w.client.get_global_var(...)`, a Redis read). Polling a queue-mediated call every tick
means: (a) it blocks the widget's Qt thread waiting on the round-trip, so a busy/paused queue
freezes the *entire* GUI including unrelated actions like opening the queue-control dialog, and
(b) each tick fires a *new* request without checking if the previous one is still outstanding, so
they pile up as duplicate PENDING entries whenever the queue can't service them promptly --
exactly what the "two items in the server queue" report was.
`get_sample_name_in_samplestage()` (`omny_sample_storage.py:453-455`) turned out to be a one-line
wrapper: `return str(self.sample_in_samplestage_name.get())`, around an `auto_monitor=True`
EpicsSignal that's already directly readable. **Fixed**: `sample_name_getter` now reads
`w.dev.omny_samples.sample_in_samplestage_name.get(cached=True)` directly, matching flomni's
fast-getter pattern -- no more device_rpc round-trip, so no more GUI freeze and no more duplicate
queue entries from this source. Mirko, from prior advice from BEC developers: GUI code should
prefer `cached=True` where possible -- checked `DeviceBase.get()` (`bec_lib/device.py`): without
it, `.get()` still issues a live `self._run(cached=False, ...)` round-trip to the device server;
`cached=True` reads BEC's already-locally-updated status cache instead, no round-trip at all, so
used it here too (there's existing precedent for this exact pattern in GUI code, e.g.
`sample_storage.py`'s `oparkz.readback.get(cached=True)`). Regression test:
`test_omny_sample_name_getter_reads_signal_not_rpc_method` (fake device whose
`get_sample_name_in_samplestage()` raises `AssertionError` if called, and whose fake signal
records its `cached` kwarg -- confirms both that the getter only touches the signal and that it
passes `cached=True`).
## zero_deg_reference_at_each_subtomo ported to OMNY's equally spaced types (1/4/5)
Mirko: "one thing flomni has and we could implement here as well is the repeated projection when
the stage is close to 0 anyways... for the tomo types 1,4 and 5 it could also be useful. in tomo
type 1 it would of course only mean that at the end of the measurement one additional projection
will be recorded." This flag was deliberately excluded from OMNY's `_TOMO_SCAN_PARAM_NAMES` during
the Phase 1 port as a "flomni-only concept" (flomni's version is type-1-only, tied to its fixed
8-sub-tomogram/180-vs-360-mode scheme) -- this reverses that call for OMNY's generalized 1/4/5
equally-spaced types specifically, it's not a blind re-add.
Verified against `_equally_spaced_subtomo_angles()`'s bit-reversal interleaving: odd (forward)
sub-tomograms always start near angle 0 for any `n_subtomos` (2/4/8) -- exactly flomni's 180-mode
rule (`subtomo_number % 2 == 1`), no 360-mode branch needed since OMNY only ever scans 180deg.
Sub-tomogram 1 specifically always starts at *exactly* 0.0 (phase offset 0 by construction,
regardless of tomo_type); the other odd sub-tomograms (3/5/7, types 4/5 only) start at a small but
*nonzero* phase offset -- genuinely new angles near 0, not exact duplicates.
One design question flomni's own code doesn't answer for OMNY: firing the pre-shot before
sub-tomogram 1 too would duplicate its own first projection (both exactly angle 0, back to back).
Asked Mirko directly -- answer: "it will be very helpful to start with two projections at zero
degrees" -- so the pre-shot fires before every odd sub-tomogram including 1, deliberately
producing two consecutive 0-degree shots at the very start (useful as an immediate damage/noise-
floor baseline pair), matching flomni's own ported logic exactly rather than diverging from it.
**Implementation** (`omny.py`): new `zero_deg_reference_at_each_subtomo` property (mirrors
flomni's of the same name), added to `_TOMO_SCAN_PARAM_NAMES` (comment above the tuple updated --
this is now the one flomni-adjacent concept OMNY *does* carry, unlike `tomo_angle_range`/
`single_point_*`/`frames_per_trigger`, which still don't apply). `tomo_scan()`'s type-1-like loop:
before each odd sub-tomogram (`ii % 2 == 1`), fires `self._tomo_scan_at_angle(0, ii)` -- guarded by
`start_angle is None` so a resume mid-sub-tomogram doesn't force a detour through 0 it isn't
actually passing through -- and once more after the whole loop finishes, for the final before/after
bookend shot. `tomo_parameters()` gained a matching print (current-settings summary) and a
`self.OMNYTools.yesno(...)` prompt (edit mode), gated by `self.tomo_type in self._TOMO_TYPE_1_LIKE`.
**Widget** (`tomo_params.py`): `SETUP_PROFILES["omny"]["has_zero_deg_reference"]` flipped
`False` -> `True` -- the checkbox itself (`_build_type1_section()`'s `if self._profile[
"has_zero_deg_reference"]: self._add_bool(...)`) was already fully generic, needing no new UI
code. Added `zero_deg_reference_at_each_subtomo` to `OMNY_QUEUE_PARAM_NAMES`/`OMNY_DEFAULTS`
(default `False`). **Second hardcoded-`!= 1` bug caught while wiring this up** (same class of bug
as `_validate()`'s Phase 2 fix): `_irrelevant_params_for_type()` (used by `_job_tooltip()` to hide
inapplicable fields) had `if tomo_type != 1: irrelevant |= _TYPE1_ONLY_PARAMS` -- would have hidden
this setting from every OMNY type-4/5 job's tooltip even when genuinely enabled. Fixed to
`tomo_type not in (1, 4, 5)` (safe without profile context: 4/5 are OMNY-only tomo_type values,
flomni/lamni jobs never have them).
Regression tests: `test_tomo_scan_zero_deg_reference_disabled_by_default_no_extra_shots`,
`..._type1_fires_before_subtomo1_and_after_subtomo2`, `..._type5_fires_before_every_odd_subtomo`,
`..._skipped_only_for_the_resumed_subtomo` (`test_omny_tomo_scan.py`, spy on `_tomo_scan_at_angle`
with `sub_tomo_scan` stubbed out); `test_irrelevant_params_for_type_keeps_zero_deg_reference_for_omny_types_4_and_5`
(`test_omny_tomo_params_widget_math.py`); updated the pre-existing "no flomni-only concepts" test
(`test_omny_tomo_queue.py`) and the pre-existing `has_zero_deg_reference is False` test (now
`True`), both of which would otherwise have failed against this deliberate reversal. Full suite:
770 passed.
**Deferred, needs Mirko's hardware knowledge, not implemented**: Mirko also asked for flomni's
`collect_empty_frames()` (acquires 10 flat-field/empty frames at angle 0 with the sample shifted
out of the beam by fovx/2, called once at the start of every new tomo_scan()) to be ported to
OMNY. Checked flomni's implementation (`flomni.py:2570`) -- it's built entirely on flomni's fine-
piezo/laser-tracker stack (`dev.rtx`/`rty`/`rtz`, `laser_tracker_check_signalstrength()`,
`move_samx_to_scan_region()`), none of which OMNY has. OMNY's own alignment offsets
(`omny_alignment_mixin.py`'s `get_alignment_offset()`) are a pure sinusoidal-fit computation
folded directly into `cenx`/`ceny` passed to `scans.omny_fermat_scan()` -- there's no separate
fine-stage motor to move at all, and no `scans.acquire()` (or equivalent stationary-multi-frame-
burst call) used anywhere in OMNY's plugin code to crib from. A literal port isn't possible without
knowing: (1) what OMNY-side call performs a stationary multi-frame acquisition (does one exist?),
and (2) whether "shift out of the beam" for OMNY means moving `osamx` directly (it's a real,
movable coarse stage, confirmed) or just passing a deliberately offset `cenx` into
`omny_fermat_scan()`. Not guessed at -- next step is asking Mirko for these two specifics.
## collect_empty_frames() ported to OMNY
Mirko: flomni's `collect_empty_frames()` (10 flat-field/empty-frame images at angle 0, sample
shifted out of the beam by fovx/2, taken once at the start of every new `tomo_scan()`) had no OMNY
equivalent. Working out the right port took several rounds, each correcting the last:
1. **First attempt (discarded)**: a bespoke "150 microns single-move, 300 combined" staged
rtx+osamx dance, based on Mirko's own paraphrase ("move rtx within 150 microns range, if more
range needed, move rtx to 150, then osamx 150..."). Confirmed via `AskUserQuestion` that rtx's
own reading stays put (doesn't reset) across the osamx move -- but this whole model turned out
to be an approximation of something else entirely (see below), and was never committed as final
-- superseded before being finished.
2. **Second attempt (also discarded)**: while drafting, Mirko volunteered the *actual* legacy SPEC
macro for OMNY's projection scan, which recenters `osamx` with a single position-threshold
check (0.025mm/25um), not a staged multi-move dance:
```
if (fabs(A[osamx]-(_omny_samx_in+$3/1000)) > 0.025) { ...move osamx to _omny_samx_in+$3/1000... }
```
Cross-checking `RtOMNYController.USER_ACCESS` (`rt_omny_ophyd.py`) found `"move_samx_to_scan_
region"` declared there -- exactly flomni's own mechanism's name -- but **never actually
implemented** for OMNY (only `RtFlomniController.move_samx_to_scan_region()`, flomni's real
version, exists, and it's a PID-voltage-based iterative correction Mirko confirmed OMNY has no
equivalent readback for). Implemented a new `RtOMNYController.move_samx_to_scan_region()` from
the SPEC algorithm to fill this apparent gap.
3. **Correction, before committing anything (the actual fix)**: Mirko recalled that in flomni this
recenter logic lives inside the *scan class itself* (`flomni_fermat_scan.py`), not the plugin,
and asked whether OMNY's own scan class should get the same treatment for consistency. Checking
`omny_fermat_scan.py`'s `prepare_setup_part2()` found **`self.dev.osamx.omny_osamx_to_scan_
center(self.cenx)` already there** -- a real, already-implemented, already-wired-into-every-
projection port of the exact same SPEC algorithm, just living on `OMNYGalilMotor`
(`ogalil_ophyd.py`) instead of `RtOMNYController`. The "gap" from step 2 didn't actually exist;
`move_samx_to_scan_region` (declared in `USER_ACCESS`, never implemented) was simply the wrong,
unused name to have gone looking under. **Removed** the step-2 addition entirely (replaced with
a comment pointing at the real mechanism, so nobody else goes down the same path) rather than
leave two parallel "recenter osamx" implementations that don't call each other.
**Final implementation** (`omny.py`): `collect_empty_frames()`, structurally mirroring flomni's own
(same 10-frame count, same `subtomo_number=0` logging convention, same
"shift by fovx/2 + 20um margin, direction away from the normal alignment offset" formula) but using
OMNY's own device names (`osamroy` not `fsamroy`) and reusing the *existing*
`dev.osamx.omny_osamx_to_scan_center()` for the recenter step -- no new device-driver code needed
in the end. Wired into `tomo_scan()`'s "a new tomo scan was started" branch (same placement as
flomni's), so it fires once per fresh scan across all 5 tomo types, never on a resume.
Regression tests: `test_tomo_scan_collects_empty_frames_on_new_scan_not_on_resume`,
`test_collect_empty_frames_rotates_to_zero_when_not_already_there`,
`test_collect_empty_frames_shifts_x_away_from_normal_offset_and_recenters_osamx`
(`test_omny_tomo_scan.py`) -- the last one checks the exact shifted-x value, that
`omny_osamx_to_scan_center()` is called with that same value, and that exactly one scan number
gets logged with `subtomo_number=0`. `make_omny_for_tomo_scan()`'s shared fixture gained a
`collect_empty_frames = lambda: None` stub (needed by every other `tomo_scan()` test once this
was wired in). Full suite: 773 passed.
**Not verified against real hardware** -- next step is Mirko trying a real `tomo_scan()` and
confirming: the rotation-skip-if-already-at-0 branch, the direction/magnitude of the x shift, and
that `omny_osamx_to_scan_center()`'s recenter fires (or correctly doesn't) as expected.
## zero_deg_reference_at_each_subtomo's progress fields were stale (fixed for OMNY; same bug likely in flomni, not touched)
Mirko, live-testing the just-landed feature: "the first zero degree projection does not seem to
reset the projection number. not sure if it is correctly done in the case of flomni" -- and a
pasted transcript showing the very first (pre-subtomo-1) reference shot printing "Projection: 9 /
Total projections expected: 32" (stale leftover from an earlier run), followed moments later by
subtomo 1's own real first projection correctly printing "Projection: 0 / Total: 10".
Root cause: `progress["projection"]`/`["subtomo_projection"]`/`["subtomo_total_projections"]`/
`["total_projections"]` are otherwise only ever set inside `sub_tomo_scan()`'s per-point loop. The
0-degree reference shots (both the pre-subtomo-N and the final post-loop one) call
`_tomo_scan_at_angle()` directly, bypassing `sub_tomo_scan()` entirely -- so whatever those fields
happened to hold from a previous scan (or the previous subtomo) is what gets printed, not a value
that makes sense for this shot.
**Fixed for OMNY**: both call sites in `tomo_scan()`'s type-1-like loop now explicitly set all
four fields immediately before calling `_tomo_scan_at_angle(0, ...)` -- the pre-shot is treated as
sub-tomogram `ii`'s own projection 0 (matching what `sub_tomo_scan()` is about to set for its real
first point right after), the final shot as one past the last real point of the final
sub-tomogram. Regression test:
`test_tomo_scan_zero_deg_reference_progress_fields_are_fresh_not_stale` (seeds `progress` with
stale values matching the live report, asserts both shots' snapshots are fully fresh/correct, not
just non-crashing).
**Same bug confirmed present in flomni's own code** (`flomni.py`'s equally-spaced-sub-tomograms
loop, `_tomo_scan_at_angle(0, ii)`/`_tomo_scan_at_angle(0, 8)` calls, same as OMNY's before this
fix -- flomni's `sub_tomo_scan()` is likewise the only place that sets these fields) -- this is
where OMNY's version was ported from, so it inherited the bug faithfully. **Not fixed here**:
`flomni.py` is a different, currently-in-production plugin outside this branch's scope
(`update-omny`); flagging for a separate, dedicated look rather than bundling an unreviewed
flomni-affecting change into this branch.
## Measured-status + account tracking now real EPICS for OMNY -- port to flomni later
OMNY sample storage gained a real EPICS channel for measured status and the beamtime account
(`OMNYSampleStorage`'s new `sample_status`/`sample_shuttle_{A,B,C}_status` signals, one new `ai`
record per sample position -- see `omny_samples_status.template` in `OMNY/OMNY.git` and
`OMNY/ioc-samples-and-xrayeye`). Encoding: VAL packs *both* status and account into one number --
`0` means not measured; otherwise VAL's magnitude is the 5-digit account and its sign is the
status (negative = "started", positive = "completed"), falling back to `FALLBACK_ACCOUNT` (11111)
for anything that doesn't parse as exactly 5 digits. See `OMNYSampleStorage.STATUS_NONE`'s class
comment and `_status_to_val()`/`_val_to_status()`/`_val_to_account()`/`_normalize_account()` in
`csaxs_bec/devices/omny/omny_sample_storage.py`.
This deliberately mirrors, but does not replace, FlOMNI's own measured-status tracking, which
still lives entirely in a BEC redis global var (`_MeasuredSampleLog`/`flomni_measured_samples` in
`sample_storage.py`/`flomni.py`), keyed by `pack_desc(name, owner)` rather than a fixed physical
position -- FlOMNI wasn't touched this branch (explicitly out of scope, see the sample-storage
work above). **Mirko's intent: port the same real-EPICS status+account channel to FlOMNI at some
point** -- a separate, dedicated task (its own `.template` record + `FlomniSampleStorage` changes
+ retiring/reconciling `_MeasuredSampleLog`), not bundled into this branch.
**Superseded 2026-10-18 (branch `feat/omny-sample-ioc`):** both instruments now keep one
soft-IOC record per slot (`FLOMNI-SAMPLE:<slot>:...` since 2026-10-17, `OMNY-SAMPLE:<slot>:USED/
NAME/OWNER/STATUS/MEAS_DATE` on OMNY's caproto IOC `ioc_omny.py`); the status is an enum plus a
date, the status x account packing above and `_MeasuredSampleLog` are gone, `transfer_sample()`
moves the whole record, and `tomo_scan()` sets started/completed on the STAGE record (as flOMNI).
## Reference images now saved at every manual confirm gate in otransfer_get_sample/put_sample -- flomni has no equivalent gate to save at
`_otransfer_save_reference_images()` used to be called only from inside `_otransfer_confirm()`'s
"All OK?" checkpoint (the RT-controller confirm handshake during the actual mount/unmount motion).
It's now also called at every earlier manual `otransfer_confirm_dialog()` gate in
`omny_sample_transfer_mixin.py`:
- `otransfer_get_sample()`'s "Please confirm that currently there is no sample in the gripper..."
gate (`file_suffix="get_gripper_empty_check"`)
- `otransfer_get_sample()`'s "Ready to start the unmount process now?" gate
(`file_suffix="get_unmount_start"`)
- `otransfer_put_sample()`'s "Ready to start the mount process now?" gate
(`file_suffix="put_mount_start"`)
All three already run after `_omnycam_parking()`/`_omnycam_samplestage()` has made the camera pair
live, so no extra setup was needed -- just one more call to the existing, generic
`_otransfer_save_reference_images(file_suffix=...)` helper at each site.
**Checked flOMNI for the equivalent gap, per Mirko's request: there isn't one to fix.**
`flomni.py`'s `ftransfer_get_sample()`/`ftransfer_put_sample()` have no manual confirm dialog at
all before starting the mount/unmount motion -- unlike OMNY, they just do an automated
`is_sample_in_gripper()`/`is_sample_slot_used()` check and raise `FlomniError` directly, no
`ftransfer_confirm_dialog()` gate to save an image at. FlOMNI's `save_reference_image()` is (like
OMNY's `_otransfer_save_reference_images()` used to be) only ever called from its own "All OK?
Continue?" handshake (`flomni.py` around line 1107-1112) -- so flOMNI and OMNY are consistent
with each other again after this change, just via different means (OMNY has extra manual gates
FlOMNI never had, not a missing image-save call on an existing FlOMNI gate).
If FlOMNI ever gains an interactive "confirm gripper is empty" gate of its own (closing this
long-standing behavioral difference from OMNY), it should get a `save_reference_image()` call at
the same point, for consistency with this change -- noted here so it isn't forgotten if/when that
gate gets added, not because anything is broken today.
## Environment note (not code, but will bite again if forgotten)
Both `csaxs_bec` and `bec_widgets` were pip-installed editable pointing at pre-repo-
reorganization paths (`~/bec/csaxs_bec`, `~/bec/bec_widgets`) that no longer exist — real
location is under `~/bec/bec/...` now. Fixed this session (`pip install -e` from the
correct path for both). If `bec-server` or `bec_widgets` imports start failing again with
no obvious code cause, check `pip show <pkg>` → `Editable project location` first.
@@ -0,0 +1,66 @@
# OMNY sample-storage testing checklist (as of 2026-09-04)
For live/sim testing of this branch's (`update-omny`) sample-storage work: the
`OMNY_SampleStorage` widget's OMNY support, the real owner + measured-status EPICS
channel, and the `omny.otransfer_*` CLI wrappers. Written so this can be pulled and
run from a separate session/machine. See `OPEN_ISSUES.md` for the broader OMNY
live-testing effort this is part of.
## Widget — `OMNY_SampleStorage`
- [ ] Open the widget in an OMNY session — header (stage+gripper) + fixed-O grid + 3
shuttle group boxes render, no "no known setup" banner.
- [ ] Occupied slots show names in legible text (not near-invisible — this was a
dark-theme contrast bug, fixed).
- [ ] Right-click an empty slot → **New sample…** → name + owner (both required,
empty owner rejected) → slot fills in without an error (a `STATUS_NONE`
crash here was fixed — client-side device proxies don't expose plain class
attributes, only signals/USER_ACCESS methods).
- [ ] Duplicate name (matching another occupied slot) → rejected with a clear
message; renaming a slot to its own existing name is still fine.
- [ ] Right-click an occupied slot → menu shows only **Change name…** / **Clear
(set empty)** — no "Set measured status…" (removed; status is CLI-only now).
- [ ] **Clear (set empty)** works without error.
- [ ] Each shuttle's status line: right-click → **Set parking slot…** (1-6) /
**Clear parking assignment** — shows "Slot N" / "not parked". The
"● ACTIVE" badge needs `oparkz` sitting at a real target position (2-6);
may not be easy to trigger from sim unless you drive that motor there.
- [ ] **Print** button builds a PDF (expect a "printer not available" message
unless you have a `WSLA_X12SA` CUPS queue locally — not a bug).
## CLI (`omny.otransfer_*`)
- [ ] Tab-complete `omny.otransfer_` — lists ~22 commands (`otransfer_show_all`,
`otransfer_set_sample_status`, `otransfer_get_sample_owner`, etc.).
- [ ] `omny.otransfer_help()` — prints the transfer commands **and then the full
`dev.omny_samples.help()` reference** below it, in one call.
- [ ] `omny.otransfer_show_all()` — table shows `(owner)` and
`◐ started`/`✓ completed` suffixes on occupied slots.
- [ ] `dev.omny_samples.set_sample_slot('A', 4, 'testsample', 'yourname')` then
`dev.omny_samples.get_sample_name_and_owner('A', 4)` round-trips.
- [ ] `omny.otransfer_set_sample_status('A', 1, 'started')` / `'completed'` /
`'none'` — status is settable from the CLI (by design; not from the widget).
- [ ] `omny.otransfer_park_loadlock_slot(6)` → answer **y** to "in back position"
→ should now proceed (drive to the slot) instead of raising
`OMNYTransferError` (was inverted — pre-existing bug, fixed); answer **n**
→ should correctly abort with that error.
## Transfer reference-image saving
Depends on whether your session drives the Galil `MGconfirm`/`MGmntprgs`
handshake (`_otransfer_confirm()`) — may not fire in a lightweight sim.
- [ ] Run `omny.otransfer_put_sample(...)` / `otransfer_get_sample(...)` through a
confirm checkpoint → check `~/data/raw/logs/omny_sample_transfer_images/`
for a new `.h5` file with both cameras of whichever pair was active
(parking `cam200`/`cam203` vs stage `cam201`/`cam202`).
- [ ] If the handshake doesn't fire: `omny._otransfer_save_reference_images(file_suffix='test')`
after opening a camera view (e.g. `omny.otransfer_park_slot(3)`) to set
`_omnycam_active_pair` — same effect, minus the physical move.
## X-ray eye alignment — known incomplete, not this branch's scope
The fit+image saving in `x_ray_eye_align.py` is shared code (also used by FlOMNI/LamNI)
and should already work in principle, but it has pre-existing OMNY/LamNI method-name
mismatches (e.g. `self.lfzp_in()`) that likely block the actual click-through run from
completing for OMNY. Not fixed this pass — flag if you hit it.
@@ -0,0 +1,46 @@
# TODO: OMNY still uses the old `~/Data10` layout, and its PDF report can fail on a missing folder
LamNI and flomni write their files under the new data structure (`~/data/raw/...`).
OMNY still points at the old `~/Data10/...` tree in several places and needs porting.
Not done yet, to avoid conflicting with other in-progress OMNY changes.
## 1. Port `~/Data10` paths to the new data structure
Where OMNY (`csaxs_bec/bec_ipython_client/plugins/omny/`) still uses `~/Data10`:
| File | Line | Path | LamNI/flomni equivalent |
|---|---|---|---|
| `omny.py` | 1217 | `tomo_reconstruct(base_path="~/Data10/specES1")` | `~/data/raw/logs/reconstruction_queue` |
| `omny.py` | 1236 | `~/Data10/specES1/dat-files/tomography_scannumbers.txt` | `~/data/raw/logs/tomography_scannumbers.txt` |
| `omny.py` | 1484 | `~/Data10/documentation/tomo_scan_ID_<id>.pdf` | `~/data/raw/documentation/tomo_scan_ID_<id>.pdf` |
| `omny.py` | 1489 | `xterm /work/sls/spec/local/XOMNY/bin/upload/upload_last_pon.sh &` | old spec upload script, check it still applies |
| `x_ray_eye_align.py` | 392 | `~/Data10/specES1/internal/xrayeye_alignmentvalues` | `~/data/raw/logs/xrayeye_alignmentvalues/` (timestamped HDF5) |
| `omny_alignment_mixin.py` | 62 | `~/Data10/specES1/internal/` | see the same alignment-values location |
The reconstruction queue folder in the new layout is created and written by the shared
`PtychoReconstructor.write()` (`OMNY_shared/omny_general_tools.py`), which OMNY does not use
yet. See also `TODO_tomo_reconstruct_stale_scan_list.md` (already ported to flomni and LamNI,
so switching OMNY to the shared reconstructor would cover both items).
## 2. Create the documentation folder if it does not exist
`OMNY.write_pdf_report()` (`omny.py:1439`) opens `PDFWriter(user_target)` without creating the
target folder first. `PDFWriter.close()` calls fpdf's `output(path)`
(`bec_lib/pdf_writer.py:90`), which does not create directories, so on an account without the
folder `tomo_scan()` fails with:
```
FileNotFoundError: [Errno 2] No such file or directory: '.../documentation/tomo_scan_ID_0.pdf'
```
Seen on LamNI; flomni had the identical pattern. Both are fixed by creating the folder before
the writer is opened:
```python
os.makedirs(os.path.dirname(user_target), exist_ok=True)
with PDFWriter(user_target) as file:
```
Apply the same line in `OMNY.write_pdf_report()`, and use the new-structure path from item 1.
Note the later steps in that method (the `upload_last_pon.sh` call and the logbook message)
only run if the PDF write succeeds.
@@ -0,0 +1,31 @@
# TODO: document stitch_x/stitch_y in the omny user manual
Status: **not implemented.** `docs/user/ptychography/omny.md` has zero
mentions of stitching, same gap just fixed 2026-09-19 in `flomni.md` and
`lamni.md`.
## What to add
Omny's mechanism is essentially identical to Flomni's (same parameter
names, even) — `omny.py:1263-1269`:
```python
for stitch_x in range(-self.stitch_x, self.stitch_x + 1):
for stitch_y in range(-self.stitch_y, self.stitch_y + 1):
...
cenx = sum_offset_x + stitch_x * (self.fovx - self.tomo_stitch_overlap)
ceny = sum_offset_y + stitch_y * (self.fovy - self.tomo_stitch_overlap)
```
Same paragraph shape as the ones just added to `flomni.md`/`lamni.md` should
carry over directly, substituting nothing (param names are identical):
`stitch_x`/`stitch_y` (default 0), tiled `(2*stitch_x+1) × (2*stitch_y+1)`
grid of scans per projection angle, spaced by `fovx`/`fovy` minus
`tomo_stitch_overlap` (µm, default 0.2), effective FOV roughly
`(2*stitch_x+1) × fovx` by `(2*stitch_y+1) × fovy`.
Not yet checked: omny.md's actual section structure/where the equivalent of
flomni's "### Tomography" / lamni's "### Test projections and Laminography
scan" section lives, and whether omny has anything resembling
`single_point_instead_of_fermat_scan`'s stitch-forced-to-0 interaction
(flomni does; not confirmed for omny).
@@ -0,0 +1,103 @@
# TODO: port `fermat_asymmetry` (x/y spacing ratio) to OMNY now that it's landed on Flomni
**Status: `fermat_asymmetry` is now implemented on Flomni** (branch
`fixes/flomni_beamtime_2`), not yet ported to OMNY. See
`docs/plans/flomni-fermat-asymmetry.md` for the original design (the
implementation follows it as written) and `csaxs_bec/scans/flomni_fermat_scan.py`
for the actual landed code (`get_flomni_fermat_spiral_pos`, line ~420 at
time of writing; `MIN_FERMAT_ASYMMETRY`/`MAX_FERMAT_ASYMMETRY` class
constants, line ~70).
`fermat_asymmetry` is a ratio of x spacing to y spacing; default `1.0` =
symmetric, keeps x spacing fixed at `step` and scales y spacing by the ratio
while preserving the requested FOV. Soft-clamped to `[0.1, 10]` (with a
`logger.warning` when clamped), same style as `zshift`/`corridor_size`.
**Effective spacing (verified against generated data, see
`docs/plans/flomni-fermat-asymmetry.md`'s conversation for the measurement
method):** x spacing always stays at `step`; y spacing becomes `step /
fermat_asymmetry`. E.g. with `step=1 um`, `fovx=fovy=10 um`: `fermat_asymmetry=0.7`
-> x spacing 1.0 um, y spacing ~1.43 um (sparser), ~70 points;
`fermat_asymmetry=1.3` -> x spacing 1.0 um, y spacing ~0.77 um (denser),
~130 points. Point count scales with the ratio since the FOV is fixed but
the density isn't. Same relationship should be documented for OMNY once
ported (Flomni's user doc has this table -- see
`docs/user/ptychography/flomni.md`, "Effective spacing with
`fermat_asymmetry`" -- OMNY's equivalent doc, if any, should get the same).
## Important: compose with the already-ported sub-step randomization
Before this, `fixes/flomni_beamtime_2` already ported a *different* fix to
all three setups including OMNY: a per-call random sub-step offset (see
`docs/plans/fermat-scan-randomization.md`). `OmnyFermatScan.get_omny_fermat_spiral_pos`
(`csaxs_bec/scans/omny_fermat_scan.py`) already has a `seed: int | None = None`
parameter and draws `offset = np.random.default_rng(seed).uniform(0, step)`,
subtracted from both `x` and `y` before the FOV crop check -- this landed
*before* fermat_asymmetry and is unrelated to it, but the two must compose
correctly when porting fermat_asymmetry to OMNY.
Flomni's landed implementation composes them as: `x = radius*sin(...) -
offset` (unchanged); `y_raw = radius*cos(...) - offset` (offset subtracted
at the *raw*, pre-compression scale, same as x); check `abs(y_raw) >
(length_axis2 * fermat_asymmetry) / 2` (inflated window); on accept, `y =
y_raw / fermat_asymmetry` (compression happens *after* the offset and the
check, not before). This keeps the offset's phase-shift proportional to the
local spacing on both axes (x spacing = step, y spacing = step/fermat_asymmetry,
offset/fermat_asymmetry is the resulting y-phase-shift once compressed) --
port this exact ordering (offset subtraction -> inflated check -> compression),
not just the offset or the asymmetry in isolation.
## What needs to happen
OMNY's `OmnyFermatScan.get_omny_fermat_spiral_pos`
(`csaxs_bec/scans/omny_fermat_scan.py`) uses the byte-for-byte same spiral
algorithm as Flomni's `get_flomni_fermat_spiral_pos` (`radius = step * 0.57
* sqrt(ii)`, same FOV-window filter, same offset-subtraction-before-check
shape already ported), so the same change ports directly:
1. Add `MIN_FERMAT_ASYMMETRY = 0.1` / `MAX_FERMAT_ASYMMETRY = 10.0` class
constants to `OmnyFermatScan`, mirroring Flomni's.
2. Add `fermat_asymmetry` to `OmnyFermatScan.__init__`
(`omny_fermat_scan.py:63-79`) as a new `ScanArgument`, store it, pass it
through in `prepare_scan()`'s call to `get_omny_fermat_spiral_pos`.
3. Add the clamp + inflated-window-check + y-compression logic to
`get_omny_fermat_spiral_pos` itself (same reasoning as Flomni: it's also
called directly by preview/estimate code, so clamping there keeps every
caller in sync automatically) -- apply it on top of the existing offset
logic exactly as described above, not before it.
4. Mirror the CLI-side plumbing on `OMNY`
(`csaxs_bec/bec_ipython_client/plugins/omny/omny.py`): a
`tomo_shellstep`-style property pair (OMNY hand-writes
`get_global_var`/`set_global_var` property pairs, e.g. `omny.py:671-679`,
rather than using the `_GlobalVarParam` descriptor Flomni uses), the
`scan_kwargs` dict in the per-angle scan call (~`omny.py:1282-1284`), and
`tomo_parameters()` (~`omny.py:1293-1341`).
## Difference from Flomni to account for when porting
OMNY has no tomo-queue system -- `OMNY` does not mix in `TomoQueueMixin` and
has no `_TOMO_SCAN_PARAM_NAMES` (unlike `Flomni`/`LamNI`). So there is no
queue-snapshot list to add the new parameter to on OMNY's side, and no
`tomo_params` Qt widget profile for OMNY today either (`SETUP_PROFILES` in
`tomo_params.py` only has `"flomni"` and `"lamni"` entries) -- the widget
part of the Flomni implementation (the "Fermat asymmetry (x/y)" field with
its "?" help button in `_build_fermat_asymmetry_row`/
`_show_fermat_asymmetry_help`) doesn't apply until/unless OMNY gets its own
queue + widget support. Scope the OMNY port to: the scan class parameter,
the global var, and `tomo_parameters()` display/prompt only.
## Tests
Mirror the three new cases added to `tests/tests_scans/test_flomni_fermat_scan.py`
(`test_fermat_asymmetry_default_matches_symmetric_spiral`,
`test_fermat_asymmetry_biases_spacing_while_preserving_fov`,
`test_fermat_asymmetry_out_of_range_is_clamped_not_raised`) in
`tests/tests_scans/test_omny_fermat_scan.py` -- same technique (compare
against an equivalent call with an inflated FOV and `fermat_asymmetry=1.0`,
same `seed`, to verify the inflate/compress math directly rather than via
an approximate spacing-ratio heuristic).
## Explicitly not for LamNI
This feature was scoped to Flomni (and, via this TODO, OMNY) only; do not
add it to `LamniFermatScan` / `LamNI` as part of this port.
@@ -0,0 +1,47 @@
# TODO: port `randomize_offset` (switch off the sub-step randomization) to OMNY
**Status: `randomize_offset` is implemented on Flomni only (branch `fixes/flomni_beamtime_2`,
2026-09-20), not yet ported to OMNY.**
## What landed on Flomni
The per-call random sub-step offset of the Fermat spiral (see `docs/plans/fermat-scan-randomization.md`)
had no way to be turned off from the scan: `seed` exists only as an argument of the static position
function and is not a scan argument. A fixed seed would only make the offset reproducible anyway --
the offset is always `uniform(0, step)`, so no seed gives the unshifted lattice.
Flomni now has an optional scan argument `randomize_offset: bool = True`
(`csaxs_bec/scans/flomni_fermat_scan.py`):
- `FlomniFermatScan.__init__` takes it as a `ScanArgument` (default `True`, so behaviour is
unchanged), stores `self.randomize_offset` and passes it in `prepare_scan()`.
- `get_flomni_fermat_spiral_pos(..., randomize_offset=True)` computes
`offset = np.random.default_rng(seed).uniform(0, step) if randomize_offset else 0.0`.
With `False` the offset is 0 and `seed` is irrelevant.
- Tests: `test_randomize_offset_default_is_active` and
`test_randomize_offset_false_gives_unshifted_deterministic_lattice` in
`tests/tests_scans/test_flomni_fermat_scan.py`.
- User doc: `docs/user/ptychography/flomni.md` (parameter table + paragraph on the randomization).
## What needs to happen on OMNY
`OmnyFermatScan.get_omny_fermat_spiral_pos` (`csaxs_bec/scans/omny_fermat_scan.py`, offset drawn at
~line 319) has the same `seed` argument and the same `offset = ...uniform(0, step)` line, so the
change ports directly:
1. Add `randomize_offset: Annotated[bool, ScanArgument(...)] = True` to `OmnyFermatScan.__init__`,
with docstring entry, store it and pass it through in `prepare_scan()`.
2. Add `randomize_offset: bool = True` to `get_omny_fermat_spiral_pos` and use
`0.0` instead of the random draw when it is `False`.
3. Mirror the two Flomni tests in `tests/tests_scans/test_omny_fermat_scan.py`.
4. Add the parameter to the parameter table in `docs/user/ptychography/omny.md` (~line 322-339;
the omny table has no `fermat_asymmetry` row yet either, see
`TODO_fermat_asymmetry_port.md`).
Scope notes:
- The OMNY tomo loop (`omny.py`, `scan_kwargs` in the per-angle scan call) does not need to pass it;
as on Flomni, tomo scans keep the default (randomized).
- Compose with `fermat_asymmetry` as described in `TODO_fermat_asymmetry_port.md`: `randomize_offset=False`
only sets `offset = 0.0`; the offset-subtraction -> inflated check -> compression ordering stays as is.
- LamNI has the identical offset code (`lamni_fermat_scan.py:323`) but was not asked for this switch.
@@ -0,0 +1,46 @@
# TODO: OMNY is missing two pre-scan safety checks that flomni/LamNI now have
Both items below were found while implementing a filter-out-of-beam check for
flomni and LamNI (branch `flomni_fixes_during_beamtimes`). OMNY was deliberately
left out of that work to avoid conflicting with other in-progress OMNY changes on
a separate branch — do both of these there.
## 1. Filter-out-of-beam check (new feature, not yet on OMNY at all)
flomni (`flomni.py`) and LamNI (`lamni.py`) now warn hard if any of the four cSAXS
exposure-box filters (`filter_array_1_x` … `filter_array_4_x`) are not at their "out"
position before a scan starts. The check itself is already shared/reusable:
- `csaxs_bec/bec_ipython_client/plugins/OMNY_shared/filter_check.py`
- `filters_out_of_beam(dev, tol=0.1) -> (bool, list[str])`
- `warn_and_confirm(setup, warning, interactive=True, force=False) -> bool` — red
warning + "Continue anyway?" prompt (default no) when interactive; warning + 10s
pause + always-proceed when not (for unattended/queued runs).
To wire it into OMNY, mirror what was done in `flomni.py`/`lamni.py`:
- `OMNY.tomo_scan()` (`omny.py:1005`): add a check at the top, right where
`Flomni.tomo_scan()`/`LamNI.tomo_scan()` now have theirs — note OMNY's
`tomo_scan()` currently has **no `interactive`/`force` params at all** (OMNY has no
tomo-queue, see item 2 below, so there's no unattended-run case to support yet);
just call `warn_and_confirm(self, warning, interactive=True, force=False)`.
- `OMNY.tomo_scan_projection()` (`omny.py:1244`): currently `def
tomo_scan_projection(self, angle: float)` — no `_internal` flag. Add
`_internal: bool = False`, gate the filter check on `not _internal`, and raise
`OMNYError` on decline (same rationale as flomni/LamNI: a bare `return` inside a
per-angle loop can silently drop a projection). Update its internal call sites to
pass `_internal=True`: `_at_each_angle()` (`omny.py:1166`) and the loop call at
`omny.py:893`.
## 2. Missing eye-out/optics-in check (pre-existing gap, unrelated to filters)
Discovered independently while researching the filter check: `Flomni.tomo_scan()`
and `LamNI.tomo_scan()` both hard-warn if `_check_eye_out_and_optics_in()` fails
(X-ray eye IN / optics OUT is not a valid measurement configuration). **`OMNY.tomo_scan()`
(`omny.py:1005`) has no equivalent check at all** — there's no OMNY-side
`_check_eye_out_and_optics_in()`, and nothing calls one. Worth adding the same gate,
following `flomni_optics_mixin.py`/`lamni_optics_mixin.py`'s
`_check_eye_out_and_optics_in()` implementations as the template (device names will
differ for OMNY's axes).
Not scoped/designed further here — just flagging it so it isn't lost.
@@ -0,0 +1,48 @@
# TODO: OMNY has bugs that were fixed in flomni (review 2026-10-24)
Found while reviewing `flomni.py` and its mixins (branch `fzp-move-beyond`, csaxs_bec commits
`1a5b53c`, `b939793` and the follow-up). OMNY was not changed; port these on the OMNY branch.
## 1. X-ray eye Ctrl-C handler cannot return osamx (`omny.py`, `start_x_ray_eye_alignment`)
```python
osamx_in = self._get_user_param_safe(dev.osamx, "in")
if np.isclose(osamx_in, dev.osamx.readback.get(), 0.5):
```
- `_get_user_param_safe()` looks the device up with `dev[device]` and needs the **name**
(`"osamx"`). With the device object the lookup raises (`KeyError`) inside the
`except KeyboardInterrupt` handler, so osamx is never returned to its in position and the
user sees a KeyError instead of the interrupt. flomni fix: `self._get_user_param_safe("fsamx", "in")`.
- The third positional argument of `np.isclose` is **rtol**, not atol: `0.5` means 50 % of the
readback. Use `np.isclose(a, b, rtol=0, atol=0.5)` (mm).
## 2. Special angles (`omny.py`, the per-angle loop of the tomo scan)
```python
if self._current_special_angles:
next_special_angle = self._current_special_angles[0]
if np.isclose(angle, next_special_angle, atol=0.5):
self._current_special_angles.pop(0)
num_repeats = self.special_angle_repeats
else:
num_repeats = 1
```
- `num_repeats` is unbound when special angles are set but the angle is not the next one ->
`UnboundLocalError` (flomni had the same bug, fixed earlier with `num_repeats = 1` up front).
- The angle is popped before the projection ran: a repeated attempt (timeout / beam check)
no longer sees it as special.
- `special_angle_tolerance` is never used (hard-coded 0.5); `__init__` sets it to 20 while
`set_special_angles()` defaults to 0.5; the `tolerance` docstring says "Number of repeats".
- Only the first list entry can match: a special angle that is never reached blocks all later ones.
flomni now: match any remaining special angle within `special_angle_tolerance`, pop it only after
the projection succeeded, keep the remaining list across a resume in the same session
(`Flomni._tomo_scan_at_angle`, tests `tests/tests_bec_ipython_client/test_flomni_special_angles.py`).
## Already shared (nothing to do for OMNY)
- `OMNYTools.tweak_cursor()` (OMNY_shared) no longer ends with a traceback when an arrow-key
escape sequence arrives in two parts or ESC is pressed alone (`read_escape_suffix()`), and its
"not in enabled devices" message names the device.
@@ -0,0 +1,44 @@
# TODO: OMNY is missing the scilog_last_ptycho_scans command (and its timing-log infrastructure) that flomni/LamNI have
Found while adding a step-size field to flomni's and LamNI's
`scilog_last_ptycho_scans` per-scan summary line (branch
`fixes/flomni_beamtime_2`). OMNY has no equivalent at all, not even the
older/pre-step-size version — porting it is a bigger job than the two-line
fixes just made to flomni/lamni, so it's flagged here instead of attempted
opportunistically.
## 1. scilog_last_ptycho_scans + timing-log infrastructure (new feature, not yet on OMNY at all)
flomni (`flomni.py`) and LamNI (`lamni.py`) both have a full
timing-instrumented pipeline behind `scilog_last_ptycho_scans(number_of_scans=1)`:
- `_TIMING_LOG_DIR`/`_PROJECTION_TIMING_LOG` — an append-only JSONL log of
per-projection timing records, one written per completed projection via
`_log_projection_timing(...)` (flomni.py:3180, lamni.py:966), called from
each setup's per-angle tomo-scan loop (flomni.py:2578/3502, lamni.py:1611).
- `_read_last_timing_records(number_of_scans)` (flomni.py:3265, lamni.py:1051)
— reads back the last N records from that log.
- `scilog_last_ptycho_scans(number_of_scans=1)` (flomni.py:3291, lamni.py:1077)
— prompts for a free-text comment, builds one summary line per record (scan
number(s), FOV, step size, exposure, duration), and sends it via each
setup's tolerant scilog-write helper (`Flomni._scilog_write`, flomni.py:2993;
`LamNI.write_to_scilog`, lamni.py:928).
OMNY (`omny.py`) only has the much older `write_to_scilog(content, tags=None)`
(omny.py:855) and `_write_subtomo_to_scilog(subtomo_number)` (omny.py:916) — a
per-subtomo one-liner, not a "last N scans" summary command — and no timing
log, no duration measurement, and no `scilog_last_ptycho_scans` at all.
`OMNY.tomo_scan()`/`_tomo_scan_at_angle()` (omny.py:1005/966) would be the
natural hook point for a `_log_projection_timing`-style call, mirroring where
flomni/lamni call theirs.
Porting this means: adding the timing-log write call to OMNY's per-angle
loop, then copying `_read_last_timing_records`/`scilog_last_ptycho_scans`
across (with OMNY-specific field names in place of `fovx`/`fovy` or
`lamni_piezo_range_x`/`y` — whatever OMNY's equivalent scan-size parameter is
called). Remember to include the `step {value} um` field in the per-scan
line from the start (already added to flomni's and lamni's versions, keyed
off `tomo_shellstep`) rather than needing a second follow-up fix like this
one.
Not scoped/designed further here — just flagging it so it isn't lost.
@@ -0,0 +1,62 @@
# TODO: tighten tomo-parameter ranges + enforce them in the CLI wizard
Status: **not implemented for omny.** Discussed and implemented for Flomni
and LamNI on 2026-09-19 — this note exists so the same pass isn't forgotten
for omny.
## What was done for Flomni/LamNI
Two separate things, both worth doing here too:
1. **Tightened several parameter ranges** that had grown far looser than
practically useful (GUI spinbox ranges in `tomo_params.py`, plus the
matching property/CLI bounds where they existed):
- `tomo_shellstep`: 0.001–1000.0 → **0.025–20.0 µm**
- `stitch_x`/`stitch_y`: 0–50 → **0–10** (flomni only — lamni's own
`lamni_stitch_x`/`y` were left at 0–50)
- `golden_ratio_bunch_size`: 1–10000 → **1–100**
- `tomo_stitch_overlap`: 0.0–50.0 → **0.0–10.0 µm**
- `manual_shift_y` (flomni): -1000–1000 → **-50–50 µm**
- lamni-specific: `tomo_circfov` 0.0–200.0 → **0.0–500.0 µm** (floor kept
at 0.0 — `fov_circular == 0` means "disabled", checked explicitly in
`lamni_fermat_scan.py:484`, so don't raise the floor above 0);
`lamni_piezo_range_x`/`y` 0.0–200.0 → **0.0–80.0 µm**, matching the
"(max 80)" already printed in lamni's CLI prompt text.
2. **`_get_val()` (the `tomo_parameters()` CLI wizard's input helper) now
supports `min_`/`max_`** and re-prompts until the value is in range,
instead of accepting anything of the right type with zero validation.
Both Flomni's and LamNI's `_get_val()` got this treatment (they're
separate static methods, not shared code) and every prompted field that
has a GUI-defined range got the matching `min_`/`max_` wired in.
## What's needed for omny
Omny has its own `tomo_parameters()` and `_get_val()`
(`csaxs_bec/bec_ipython_client/plugins/omny/omny.py:1293` /
`omny.py:1422`) — same bare/unvalidated pattern Flomni and LamNI both had
before this pass. Confirmed (2026-09-19) that omny's parameter names mirror
**flomni's** naming exactly, not lamni's: `fovx`/`fovy`, `manual_shift_y`,
`tomo_type`, `stitch_x`/`stitch_y`, `tomo_stitch_overlap`,
`golden_ratio_bunch_size`, `tomo_shellstep`, `tomo_countingtime` all exist
under those same names in `omny.py`. So the flomni numbers above likely
carry over directly, but this hasn't been checked against omny's own
physical setup/hardware constraints (e.g. its FOV/shellstep/piezo range may
have different practical limits than flomni's cryo ptycho setup) — don't
just copy the numbers blind.
Steps, mirroring what was done for flomni/lamni:
1. Confirmed (2026-09-19): `tomo_params.py`'s `SETUP_PROFILES` dict has only
`"flomni"` and `"lamni"` entries — **no `"omny"` profile exists**, so
omny doesn't use this shared Tomo Parameters GUI widget at all right now.
Whatever GUI (if any) omny actually uses for entering these parameters
needs to be located first; there's nothing to tighten in `tomo_params.py`
for omny today. (Separately, `omny_webpage_generator.py` was confirmed in
an earlier session to be dead/unwired code — not related to this GUI
question, just don't confuse the two when investigating.)
2. Add `min_`/`max_` support to omny's own `_get_val()` (same mechanism,
copy-pasted independently — these are three separate static methods, not
shared code, by design in this codebase).
3. Confirm with whoever runs omny scans what the actual practical ranges
should be for omny's hardware, rather than assuming flomni's numbers
apply unchanged.
@@ -0,0 +1,58 @@
# TODO: port "always define `tomo_alignment_fit`" to OMNY now that it's landed on Flomni
**Status: implemented on Flomni** (commit `5b688df`, branch
`fixes/flomni_beamtime_2`), not yet ported to OMNY.
## Problem (as seen on Flomni)
The `tomo_alignment_fit` global var was deleted by `reset_tomo_alignment_fit()`
(on stage-in and at the start of a new alignment) and never created at
startup. Manual offset tweaks from the client failed with an undefined
variable:
```python
fit = flomni.client.get_global_var("tomo_alignment_fit") # -> None
fit[0][2] += 5.0 # x offset, microns -> TypeError
```
## What Flomni does now (`flomni/flomni.py`)
- `reset_tomo_alignment_fit()` writes `np.zeros((2, 5)).tolist()` instead of
calling `delete_global_var`, so the var is always defined.
- New `ensure_tomo_alignment_fit()` sets the zero fit **only if**
`get_global_var("tomo_alignment_fit") is None`. It is called from
`Flomni.__init__`, so an existing fit survives a client restart.
- `get_alignment_offset()` is unchanged; with zeros it prints zero offsets
instead of the "Not applying any alignment offsets" message. Same result.
Fit layout (row 0 = x, row 1 = y):
```
# x amp, phase, offset, y amp, phase, offset, 3rd order amp, 3rd order phase
# 0 0 0 1 0 2 1 0 1 1 1 2 1 3 1 4
```
## What to do on OMNY
`omny/omny_alignment_mixin.py` still has the old behaviour:
- `reset_tomo_alignment_fit()` (~line 57) does
`self.client.delete_global_var("tomo_alignment_fit")`.
- Nothing defines the var at startup. `OMNY.__init__` is in `omny/omny.py`
(~line 557).
Steps:
1. Change `reset_tomo_alignment_fit()` to write `np.zeros((2, 5)).tolist()`.
2. Add `ensure_tomo_alignment_fit()` exactly as on Flomni.
3. Call it from `OMNY.__init__` (after `self.client` is set).
4. Check the callers: at the time of writing, nothing in `omny/` calls
`reset_tomo_alignment_fit()` apart from its definition, so decide where
OMNY should reset the fit (start of an alignment / sample transfer),
mirroring Flomni's `ftransfer_flomni_stage_in()` and
`XrayEyeAlign._align_impl()`.
5. `read_alignment_offset()` in the OMNY mixin builds the fit as `np.zeros((2, 5))`
and stores it with `set_global_var`, so it needs no change.
LamNI does not reference `tomo_alignment_fit` at all (checked by grep), so no
action there.
@@ -0,0 +1,68 @@
# TODO: OMNY.tomo_reconstruct() writes a stale/missing scan list
Found while fixing the same bug in `Flomni.tomo_reconstruct()`
(`csaxs_bec/bec_ipython_client/plugins/flomni/flomni.py:3114`, branch
`flomni_fixes_during_beamtimes`). Not applied to OMNY yet to avoid conflicting
with other in-progress OMNY changes on a separate branch — do it there.
## The bug
`OMNY.tomo_reconstruct()` (`csaxs_bec/bec_ipython_client/plugins/omny/omny.py:1217`)
names the reconstruction queue file from a fresh `bec.queue.next_scan_number`
(so the *filename* is always correct), but writes the file's *content* from
`self._current_scan_list`:
```python
def tomo_reconstruct(self, base_path="~/Data10/specES1"):
"""write the tomo reconstruct file for the reconstruction queue"""
bec = builtins.__dict__.get("bec")
base_path = os.path.expanduser(base_path)
ptycho_queue_path = Path(os.path.join(base_path, self.ptycho_reconstruct_foldername))
ptycho_queue_path.mkdir(parents=True, exist_ok=True)
last_scan_number = bec.queue.next_scan_number - 1
ptycho_queue_file = os.path.abspath(
os.path.join(ptycho_queue_path, f"scan_{last_scan_number:05d}.dat")
)
with open(ptycho_queue_file, "w") as queue_file:
scans = " ".join([str(scan) for scan in self._current_scan_list])
queue_file.write(f"p.scan_number {scans}\n")
queue_file.write("p.check_nextscan_started 1\n")
```
`self._current_scan_list` is only kept up to date by whichever internal method
last ran and then called `tomo_reconstruct()` itself right after (the OMNY
equivalents of flomni's `tomo_scan_projection()`/`tomo_acquire_at_angle()` —
check where `_current_scan_list` is assigned in `omny.py` for the exact call
sites). Call `tomo_reconstruct()` directly from the command line instead —
e.g. after a plain fermat scan run by hand, not through those internal
flows — and the file gets written with whatever scan list happened to be
cached from the *previous* tomo scan (or raises `AttributeError` if none ran
yet this session). Filename right, content wrong/stale.
## The fix (already applied to flomni, mirror it here)
`Flomni.tomo_reconstruct()` now falls back to just the most recently
completed scan number whenever the cached list doesn't actually match it:
```python
bec = builtins.__dict__.get("bec")
next_scan_number = bec.queue.next_scan_number
last_scan_number = next_scan_number - 1
scan_list = getattr(self, "_current_scan_list", None)
if not scan_list or scan_list[-1] != last_scan_number:
scan_list = [last_scan_number]
```
Internal callers are unaffected (their cached list's last entry always equals
`next_scan_number - 1` at the point they call `tomo_reconstruct()`), while a
direct/standalone call now correctly falls back to `[last_scan_number]`
instead of writing stale content or crashing.
Port the same `getattr(...)`/fallback logic into `OMNY.tomo_reconstruct()`
(note OMNY's version doesn't go through the shared `PtychoReconstructor`
class like flomni's does — it writes the file inline — so the fix applies
directly to the `scans = " ".join(...)` line, not to a shared `write()`
method).
Not scoped/designed further here — just flagging it so it isn't lost.
@@ -0,0 +1,25 @@
# TODO: port the "unfinished queue job" guard on `tomo_scan_resume()` to OMNY
**Status: implemented on Flomni and LamNI (2026-10), not applicable to OMNY yet** --
OMNY has no `TomoQueueMixin`, no tomo queue and no `tomo_scan_resume()`.
## What landed on Flomni/LamNI
- Problem: a Ctrl-C abort of `tomo_queue_execute()` leaves the job "running" (only
`Exception` is caught -> "incomplete"). Calling `tomo_scan_resume()` by hand then finishes
the scan but the queue still has the job unfinished, so a later `tomo_queue_resume()`
resumes it again (at the last projection) and the jobs after it never ran.
- `TomoQueueMixin._confirm_resume_outside_queue()`
(`OMNY_shared/tomo_queue_mixin.py`): if the queue has an "incomplete"/"running" job, print a
warning + hint (use `tomo_queue_resume()`, or mark the job done via
`_tomo_queue_proxy.update_by_id(id, status="done")`) and ask yes/no (default no).
- `tomo_scan_resume(from_queue=False)` in `flomni/flomni.py` and `LamNI/lamni.py` calls it
after the "nothing to resume" check; `tomo_queue_execute()` passes `from_queue=True` so the
unattended queue never prompts.
## To do for OMNY
When the tomo queue (`TomoQueueMixin`) and a `tomo_scan_resume()` are ported to OMNY, give
`tomo_scan_resume()` the same `from_queue` parameter and guard call, and make sure
`tomo_queue_execute()` calls it with `from_queue=True`.
Related, not done anywhere: catching `KeyboardInterrupt` in `tomo_queue_execute()` so an abort
marks the job "incomplete" instead of leaving it "running".
@@ -0,0 +1,38 @@
# TODO: OMNY has no end-of-scan timing summary at all, so it's also missing the new tomo_print_timing_report() commands
Found while adding `tomo_print_timing_report()` / `tomo_print_timing_report_and_scilog()`
to flomni and LamNI (branch `fixes/flomni_beamtime_2`) so the elapsed/idle
timing breakdown that normally only prints at the end of a *completed*
`tomo_scan()` can also be pulled on demand -- in particular right after
interrupting a scan with Ctrl-C, since the underlying `self.progress` data
(a BEC global var) survives the interrupt even though the old inlined
end-of-scan print block did not run.
## 1. No "Tomoscan finished" timing block on OMNY at all (pre-existing gap)
flomni (`flomni.py`) and LamNI (`lamni.py`) both print an elapsed/idle/
excl.-idle timing breakdown at the tail of `tomo_scan()`, built from
`self.progress` (`tomo_start_time`, `accumulated_idle_time`) via a shared
`_build_timing_lines()` helper (flomni.py, near line 2984; lamni.py, near
line 411). `OMNY.tomo_scan()` (omny.py:1005) has no equivalent block at
all -- it was added to flomni/lamni after the flomni-to-lamni migration and
was never back-ported to OMNY.
## 2. New on-demand commands also missing
Both setups now also expose:
- `tomo_print_timing_report()` -- prints the current timing breakdown to
console at any time (mid-scan, after an interrupt, or after completion),
no scilog write.
- `tomo_print_timing_report_and_scilog()` -- same, plus writes the report to
scilog.
If OMNY is still in active use, porting this means: adding an
`_ProgressProxy`-equivalent `tomo_start_time`/`accumulated_idle_time`
tracking (OMNY may already track some of this differently -- check before
assuming it needs the same proxy), then copying
`_build_timing_lines()`/`tomo_print_timing_report()`/
`tomo_print_timing_report_and_scilog()` across with OMNY's own
`write_to_scilog`/`_scilog_write` equivalent (omny.py:855).
Not scoped/designed further here -- just flagging it so it isn't lost.
@@ -0,0 +1,37 @@
# Note: X-ray-eye DAP sine fit (start values) -- what omny inherits, and what is still open
Status: **nothing to fix in omny code for the fit itself; recorded 2026-09-20 so it isn't rediscovered.**
## What was fixed (flomni, 2026-09-20)
The GUI fit tab (`OMNY_XRayEye` in `csaxs_bec/bec_widgets/widgets/xray_eye/x_ray_eye.py`) fits
`A*sin(f*x + shift) + intercept` through the DAP server with `f` fixed to 1 deg. The DAP server
used lmfit's `SineModel.guess()` for the start values. For flomni's 5 points over only 0..180 deg
that FFT-based guess is arbitrary and regularly left `shift` stuck at its bound (2*pi), far from
the data (plotted fit missed the submitted points; wrong `X Phase` loaded into flomni).
Fix: `SINE_FIT_DAP_PARAMETERS` (same file) overrides all five parameters (signed amplitude, start
A=30 / shift=0 within +-pi / intercept=0), so DAP skips `guess()`. 5-pt/0..180 deg: 0 failures in 300
random datasets through the real `LmfitService1D` (old parameters: 16 of 300).
## Consequences for omny
- The widget is shared by flomni, LamNI and omny, so omny's GUI fit tab already uses the new
parameters. Checked with 8 points over 0..315 deg (omny/LamNI layout): 0 of 300 failures both
with the old and the new parameters, i.e. the 8-point full-period case was never affected and
is not made worse. No omny change required.
- The fitted parameters use the same form `A*sin(radians(angle) + phase) + offset` in flomni,
LamNI and omny consumers, so the (now possibly negative) amplitude with phase in [-pi, pi] is
the same curve as before. Nothing downstream needs a sign fix.
- Cosmetic: a row of all zeros (flomni's y row) now fits to ~1e-6 noise instead of exactly 0;
the y plot autoscales that to a curved line at the 1e-6 level. Harmless (flomni overrides y
with zeros in `read_alignment_offset`).
## Open for omny
- `omny/x_ray_eye_align.py` is still a verbatim copy of LamNI's (class docstring "for LamNI",
uses `self.lamni.*` and `self.lamni.align.read_xray_eye_correction_from_gui()`), while
`omny/omny_alignment_mixin.py` only reads the Matlab fit from `ptychotomoalign_*.txt`
(`read_alignment_offset`) and has no `read_xray_eye_correction_from_gui`. So omny currently
does not consume the GUI fit at all. When omny's alignment is properly ported, decide whether
it should use the GUI fit (then this DAP fix is what it relies on) or keep the Matlab files.
@@ -0,0 +1,40 @@
# TODO: unconditional 0-degree reference projection at tomogram start
Status: **not implemented for omny.** Discussed 2026-09-19; implemented for
Flomni and LamNI the same day (see their `tomo_scan()` in `flomni.py`/
`lamni.py` — search for "Unconditional single 0-degree reference
projection").
## What was implemented elsewhere
At the very start of a new tomogram (not on resume), both Flomni and LamNI
now unconditionally take one real 0-degree projection — `self._tomo_scan_at_angle(0, 1)`
— independent of the `zero_deg_reference_at_each_subtomo` /
`golden_projections_at_0_deg_for_damage_estimation` flags. It's real data
(included in reconstruction, logged under `subtomo_number=1`), not a
reference-only/flat-field frame — this was an explicit choice: the point is
an immediate radiation-damage baseline that exists even when the periodic
damage-estimation flags are off, not a normalization frame.
Flomni additionally takes 10 flat-field/empty frames at 0° before this
(`collect_empty_frames()` — sample retracted out of the beam via `fsamx`
shifted by `fovx/2`, excluded from reconstruction). **LamNI does not get an
equivalent** — confirmed 2026-09-19 that LamNI has no way to retract the
sample out of the beam at all, so only the 0-degree projection was added
there, no flat-field step.
## What's needed for omny
1. **Confirm whether omny can retract the sample out of the beam at all**
(same open question as LamNI, not yet checked for omny specifically) —
determines whether an empty-frames/flat-field step is even possible here,
or whether omny should only get the 0-degree-projection part like LamNI.
2. Locate omny's `tomo_scan()` equivalent (or whatever its tomogram-start
entry point is) and its own `_tomo_scan_at_angle`-equivalent method, and
add the same unconditional call in its "new scan" branch, following the
same placement pattern used in `flomni.py`/`lamni.py`.
3. Decide the logged `subtomo_number` / whether omny's tomo-type numbering
matches flomni/lamni's convention closely enough to reuse `1` directly.
Not investigated yet — this note exists so the intent isn't lost, not
because the omny-side work has been scoped in detail.
@@ -0,0 +1,82 @@
# OMNY tomo-scan/tomo-queue/widget testing checklist (as of 2026-09-04)
For live/sim testing of this branch's (`update-omny`) tomo-scan work: the tomo-queue backend
(`TomoQueueMixin`, Phase 1), the progress-ring GUI + idle-time tracking + end-of-scan timing
summary, the `~/data/raw/` path fixes, and the `OMNY_TomoParams` widget's OMNY support (Phase 2).
Written so this can be pulled and run from a separate session/machine. See `OPEN_ISSUES.md` for
the broader OMNY live-testing effort this is part of.
## Widget — `OMNY_TomoParams` (never tested live, highest priority)
- [ ] Open `OMNY_TomoParams` against the OMNY session — it detects `"omny"` (not flomni/lamni),
no "no known setup" banner.
- [ ] Cycle through all 5 tomo types in the dropdown — the "Equally spaced sub-tomograms" section
shows for types 1/4/5 and hides for 2/3; the "Sub-tomograms" label reads 2/4/8 correctly for
each.
- [ ] **Submit or queue a type-4 and a type-5 job specifically** — a hardcoded validation bug
(`tomo_type not in (1, 2, 3)`) would have silently rejected these before the fix; this is
the single most important check on this list.
- [ ] Check a queued type-4/5 job's tooltip and the "Projections" column in the queue dialog show
correct values.
- [ ] Shrink FOV/step until the Fermat-position count drops low enough to trigger the orange
warning / blocked submit — compare against the CLI's `tomo_parameters()` warning for the
same values.
## Progress GUI / idle-time / timing summary (recently changed)
- [ ] Run a real scan and watch the progress window — it should **not** re-raise/refocus on every
projection anymore (was "closing and reopening" on macOS).
- [ ] Progress bar shows **3 rings** now (third one auto-tracks the live scan itself).
- [ ] After a completed `tomo_scan()`, the end-of-scan summary prints: "Total measurement time" /
"...excluding detected gaps" / "...lost to detected gaps".
- [ ] Interrupt and resume a scan (or otherwise create a gap) — confirm idle time is picked up in
that summary instead of showing 0.
## Tomo-queue backend, first real end-to-end runs
- [ ] `tomo_queue_execute()` on a real queued job, not just add/show.
- [ ] `tomo_scan_resume()` after interrupting a scan.
- [ ] `tomo_queue_reacquire()` — try it on both a type-1 job and a type-4/5 job (the
`_TOMO_TYPE_1_LIKE` generalization is new).
## Advanced tomo modes — parameters were tested, full scans weren't
- [ ] A real `omny.tomo_scan()` for type 2 (golden ratio) and type 3 (golden-start), not just
setting parameters via `tomo_parameters()`.
- [ ] A real full scan for type 4 and type 5 (4/8 sub-tomograms).
## zero_deg_reference_at_each_subtomo (new, never tested live)
- [ ] `tomo_parameters()`: enable it for tomo_type 1 -- confirm the prompt/summary text and that a
real `tomo_scan()` records two projections at angle 0 right at the start (subtomo 1's own
first projection plus the deliberate duplicate) and one more at the very end.
- [ ] Same for tomo_type 4 or 5 -- confirm extra 0-degree shots also appear before sub-tomograms
3/5/7 (not just 1), each at a distinct (nonzero) angle from 0, not a duplicate.
- [ ] Widget: the "0° reference each sub-tomo" checkbox now appears for OMNY (previously
flomni/lamni only) -- toggle it for a type-4/5 job specifically and check the queued job's
tooltip shows it (regression guard for a hardcoded `!= 1` bug caught while wiring this up).
## collect_empty_frames() (new, never tested on real hardware)
- [ ] Start a fresh `tomo_scan()` (any tomo type) -- confirm it rotates to 0 (or prints "No
rotation required" if already there), shifts x away from the normal alignment offset by
fovx/2 + 20um, and successfully acquires 10 empty frames before the real angular grid starts.
- [ ] Confirm `dev.osamx.omny_osamx_to_scan_center()`'s recenter fires when it should (osamx more
than 25um from where it needs to be for the shifted x) and is a no-op otherwise.
- [ ] Confirm this does *not* run on `tomo_scan_resume()`/a mid-scan resume -- only on a genuinely
fresh scan.
- [ ] Check `tomography_scannumbers.txt` gets one new line per empty frame, `subtomo_number=0`,
distinguishable from the real angular grid (subtomo_number 1+).
## Smaller fixes
- [ ] `tomography_scannumbers.txt` and the PDF report land under `~/data/raw/...` (not the old
`~/Data10/...`, which doesn't exist on a fresh machine).
- [ ] If your session has scilog configured: `write_to_scilog()`/the PDF report actually send
(rather than just skip with a "scilog is not enabled" warning).
## Already confirmed working live (informational, not re-testing needed)
- `omny.tomo_scan()`/`omny.tomo_queue_add()`/`omny.tomo_queue_show()` for tomo_type 1 — "tomo
scan and tomo queue seems to work now for omny."
- `tomo_parameters()`'s "too few Fermat scan points" CLI warning.
@@ -1,12 +1,16 @@
import builtins
import time
# from csaxs_bec.bec_ipython_client.plugins.cSAXS import epics_get, epics_put, fshopen, fshclose
from bec_lib import bec_logger
logger = bec_logger.logger
if builtins.__dict__.get("bec") is not None:
bec = builtins.__dict__.get("bec")
dev = builtins.__dict__.get("dev")
scans = builtins.__dict__.get("scans")
def umv(*args):
return scans.umv(*args, relative=False)
@@ -16,69 +20,158 @@ class OMNYGuiToolsError(Exception):
class OMNYGuiTools:
# Screen assumed 2560x1440. Window is right-aligned with a small margin
# from the top, matching flomni's gui_tools.py convention.
_SCREEN_WIDTH = 2560
_WINDOW_WIDTH = 1500
_WINDOW_HEIGHT = 850
_WINDOW_TOP_MARGIN = 50
def __init__(self, client):
self.gui = getattr(client, "gui", None)
self.gui_window = self.gui.windows["main"].widget
self.fig200 = None
self.fig201 = None
self.fig202 = None
self.fig203 = None
self.progressbar = None
def __init__(self):
self.omny_window = None
self.text_box = None
self.progressbar = None
self.xeyegui = None
self.idle_text_box = None
self.cam_dock_a = None
self.cam_dock_b = None
self.console = None
self._omnycam_shown_pair = None
def set_client(self, client):
self.client = client
self.gui = getattr(client, "gui", None)
def omnygui_show_gui(self):
self.gui_window.show()
if "omny" in self.gui.windows:
self.omny_window = self.gui.windows["omny"]
self.gui.omny.raise_window()
else:
# geometry: (pos_x, pos_y, w, h)
pos_x = self._SCREEN_WIDTH - self._WINDOW_WIDTH
geometry = (pos_x, self._WINDOW_TOP_MARGIN, self._WINDOW_WIDTH, self._WINDOW_HEIGHT)
self.omny_window = self.gui.new("omny", geometry=geometry)
time.sleep(1)
def omnygui_stop_gui(self):
self.gui_window.hide()
self.gui.omny.hide()
def omnygui_raise(self):
self.gui.omny.raise_window()
def _omnygui_is_missing(self, attribute_name):
widget = getattr(self, attribute_name, None)
if widget is None:
return True
if hasattr(widget, "_is_deleted") and widget._is_deleted():
return True
return False
def omnygui_show_xeyealign(self):
self.omnygui_show_gui()
if self._omnygui_is_missing("xeyegui"):
self.omnygui_remove_all_docks()
self.xeyegui = self.gui.omny.new("OMNY_XRayEye", object_name="xrayeye")
# cam_xeye is not (yet) part of omny's device config -- guard rather
# than assume it exists, unlike flomni's unconditional version.
if "cam_xeye" in dev and not dev.cam_xeye.live_mode_enabled.get():
dev.cam_xeye.live_mode_enabled.put(True)
self.xeyegui.switch_tab("alignment")
def omnygui_show_xeyealign_fittab(self):
self.omnygui_show_gui()
if self._omnygui_is_missing("xeyegui"):
self.omnygui_remove_all_docks()
self.xeyegui = self.gui.omny.new("OMNY_XRayEye", object_name="xrayeye")
self.xeyegui.switch_tab("fit")
def _omnycam_parking(self):
self.omnygui_show_omnycam_parking()
# Recorded so omny_sample_transfer_mixin.py's _otransfer_confirm()
# knows which camera pair to save a reference image from -- OMNY
# always shows exactly one pair (parking/shuttle or sample stage)
# depending on where the gripper currently is, unlike flomni's
# fixed single gripper camera.
self._omnycam_active_pair = ("cam200", "cam203")
def omnygui_show_omnycam_parking(self):
self._omnygui_show_omnycam_pair("cam200", "cam203", num_rotation_90=3)
def _omnycam_samplestage(self):
self.omnygui_show_omnycam_samplestage()
self._omnycam_active_pair = ("cam201", "cam202")
def omnygui_show_omnycam_samplestage(self):
self._omnygui_show_omnycam_pair("cam201", "cam202", num_rotation_90=0)
def _omnygui_show_omnycam_pair(self, cam_a: str, cam_b: str, num_rotation_90: int = 0):
"""Show a live camera pair, reusing the two camera docks + console
across parking<->samplestage switches instead of tearing the whole
GUI down and rebuilding it every time.
``Image.image(device=..., signal=...)`` (bec_widgets) disconnects
the previous monitor and reconnects to the new one on an existing
widget, so switching pairs is just re-pointing ``cam_dock_a``/
``cam_dock_b`` at the new devices -- the docks and console are only
(re)created here if they don't exist yet (first camera view this
session, or after a full teardown via another GUI state such as
idle/progress/xeyealign).
"""
self.omnygui_show_gui()
if self.fig200 is None:
self._omnycam_clear()
self.fig200 = self.gui_window.add_dock(name="omnycam200").add_widget("BECImageWidget")
if self._omnycam_check_device_exists(dev.cam200):
fig = self.fig200.image("cam200")
fig.set_rotation(deg_90=3)
self.fig200.lock_aspect_ratio(True)
if (
self._omnygui_is_missing("cam_dock_a")
or self._omnygui_is_missing("cam_dock_b")
or self._omnygui_is_missing("console")
):
self.omnygui_remove_all_docks()
self.cam_dock_a = self.gui.omny.new("Image")
self.cam_dock_b = self.gui.omny.new("Image")
# Hard-stop console, docked below the cameras. otransy is the
# closest omny analog to flomni's ftransy (both the transfer-Y
# axis) -- no secondary/extra stop device wired yet.
self.console = self.gui.omny.new(
"z_ConsoleButtonsWidget",
object_name="console",
where="bottom",
hard_stop_device_name="otransy",
hard_stop_label="OMNY Motion Stop",
)
self.gui.omny.set_layout_ratios(vertical=[5, 1])
outgoing_pair = self._omnycam_shown_pair
if outgoing_pair and outgoing_pair != (cam_a, cam_b):
for cam_name in outgoing_pair:
if cam_name in dev:
getattr(dev, cam_name).stop_live_mode()
for dock, cam_name in ((self.cam_dock_a, cam_a), (self.cam_dock_b, cam_b)):
if cam_name in dev:
dock.image(device=cam_name, signal="preview")
dock.num_rotation_90 = num_rotation_90
dock.lock_aspect_ratio = True
getattr(dev, cam_name).start_live_mode()
else:
print("Cannot open cam200. Device does not exist.")
self.fig203 = self.gui_window.add_dock(name="omnycam203").add_widget("BECImageWidget")
if self._omnycam_check_device_exists(dev.cam203):
fig = self.fig203.image("cam203")
fig.set_rotation(deg_90=3)
self.fig203.lock_aspect_ratio(True)
else:
print("Cannot open cam203. Device does not exist.")
try:
self.gui_window.remove_dock(name="default_figure")
except:
pass
print(f"Cannot open {cam_name}. Device does not exist.")
self._omnycam_shown_pair = (cam_a, cam_b)
def omnygui_remove_all_docks(self):
self.gui_window.clear_all()
self.fig200 = None
self.fig201 = None
self.fig202 = None
self.fig203 = None
if hasattr(self.gui, "omny"):
self.gui.omny.delete_all()
self.progressbar = None
self.text_box = None
self.xeyegui = None
self.idle_text_box = None
self.cam_dock_a = None
self.cam_dock_b = None
self.console = None
self._omnycam_shown_pair = None
def omnygui_idle(self):
self.omnygui_show_gui()
if self.idle_text_box is None:
if self._omnygui_is_missing("idle_text_box"):
self.omnygui_remove_all_docks()
self.idle_text_box = self.gui_window.add_dock(name="idle_text").add_widget("TextBox")
try:
self.gui_window.remove_dock(name="default_figure")
except:
pass
self.idle_text_box = self.gui.omny.new("TextBox")
text = (
"<pre>"
+ "██████╗ ███████╗ ██████╗ ██████╗ ███╗ ███╗███╗ ██╗██╗ ██╗\n"
@@ -91,79 +184,114 @@ class OMNYGuiTools:
)
self.idle_text_box.set_html_text(text)
def _omnycam_clear(self):
self.omnygui_remove_all_docks()
def _omnycam_check_device_exists(self, device):
try:
device
except:
return False
else:
return True
def _omnycam_samplestage(self):
self.omnygui_show_omnycam_samplestage()
def omnygui_show_omnycam_samplestage(self):
self.omnygui_show_gui()
if self.fig201 is None:
self.omnygui_remove_all_docks()
self.fig201 = self.gui_window.add_dock(name="omnycam201").add_widget("BECImageWidget")
if self._omnycam_check_device_exists(dev.cam201):
fig = self.fig201.image("cam201")
fig.set_rotation(deg_90=3)
self.fig201.lock_aspect_ratio(True)
else:
print("Cannot open cam201. Device does not exist.")
self.fig202 = self.gui_window.add_dock(name="omnycam202").add_widget("BECImageWidget")
if self._omnycam_check_device_exists(dev.cam202):
fig = self.fig202.image("cam202")
fig.set_rotation(deg_90=3)
self.fig202.lock_aspect_ratio(True)
else:
print("Cannot open cam202. Device does not exist.")
try:
self.gui_window.remove_dock(name="default_figure")
except:
pass
def omnygui_show_progress(self):
self.omnygui_show_gui()
if self.progressbar is None:
self.omnygui_remove_all_docks()
# Add a new dock with a RingProgressBar widget
self.progressbar = self.gui_window.add_dock(name="progress").add_widget(
"RingProgressBar"
try:
self.omnygui_show_gui()
if self._omnygui_is_missing("progressbar"):
self.omnygui_remove_all_docks()
self.progressbar = self.gui.omny.new("RingProgressBar")
# Two manually-updated rings (overall tomogram progress and
# subtomo progress) plus a third ring auto-updated from the
# currently running scan itself -- matches flomni's
# flomnigui_show_progress() exactly.
self.progressbar.add_ring().set_update("manual")
self.progressbar.add_ring().set_update("manual")
self.progressbar.add_ring().set_update("scan")
self._omnygui_update_progress()
except KeyboardInterrupt:
# A frozen/unresponsive GUI process makes raise_window() block on
# its RPC wait (up to the client's ~60s default timeout) -- if the
# operator loses patience and hits Ctrl+C during that wait, don't
# let it escape as a raw KeyboardInterrupt and abort the whole
# tomo_scan()/tomo_scan_resume() call: bringing the window to
# front is a nicety, not something the scan itself needs.
logger.warning(
"omnygui_show_progress() interrupted (GUI unresponsive?) -- continuing "
"without raising the window."
)
# Customize the size of the progress ring
self.progressbar.set_line_widths(20)
# Disable automatic updates and manually set the self.progressbar value
self.progressbar.enable_auto_updates(False)
# Set precision for the self.progressbar display
self.progressbar.set_precision(1) # Display self.progressbar with one decimal places
# Setting multiple rigns with different values
self.progressbar.set_number_of_bars(3)
self.progressbar.rings[2].set_update("scan")
# Set the values of the rings to 50, 75, and 25 from outer to inner ring
# self.progressbar.set_value([50, 75])
# Add a new dock with a TextBox widget
self.text_box = self.gui_window.add_dock(name="progress_text").add_widget("TextBox")
try:
self.gui_window.remove_dock(name="default_figure")
except:
pass
self._omnygui_update_progress()
except Exception as e:
logger.warning(f"Error in omnygui_show_progress: {e}")
def _omnygui_update_progress(self):
if self.progressbar is not None:
"""Update the progress ring bar and center label from the current progress state.
``self.progress`` is backed by the BEC global variable ``tomo_progress``
(see :class:`_ProgressProxy` in ``omny.py``), so this method reflects
the live state that is also accessible from other BEC client sessions via::
client.get_global_var("tomo_progress")
"""
try:
if self.progressbar is None:
logger.warning("OMNY progress GUI update skipped: progressbar is not available.")
return
main_progress_ring = self.progressbar.rings[0]
subtomo_progress_ring = self.progressbar.rings[1]
progress = self.progress["projection"] / self.progress["total_projections"] * 100
subotmo_progress = (
subtomo_progress = (
self.progress["subtomo_projection"]
/ self.progress["subtomo_total_projections"]
* 100
)
self.progressbar.set_value([progress, subotmo_progress])
main_progress_ring.set_value(progress)
subtomo_progress_ring.set_value(subtomo_progress)
text = f"Progress report:\n Tomo type: ....................... {self.progress['tomo_type']}\n Projection: ...................... {self.progress['projection']:.0f}\n Total projections expected ....... {self.progress['total_projections']}\n Angle: ........................... {self.progress['angle']}\n Current subtomo: ................. {self.progress['subtomo']}\n Current projection within subtomo: {self.progress['subtomo_projection']}\n Total projections per subtomo: ... {self.progress['subtomo_total_projections']}"
self.text_box.set_plain_text(text)
# --- format start time for display --------------------------------
start_str = self.progress.get("tomo_start_time")
if start_str is not None:
import datetime as _dt
start_display = _dt.datetime.fromisoformat(start_str).strftime("%Y-%m-%d %H:%M:%S")
else:
start_display = "N/A"
# --- format estimated remaining time ------------------------------
remaining_s = self.progress.get("estimated_remaining_time")
if remaining_s is not None and remaining_s >= 0:
remaining_s = int(remaining_s)
h, rem = divmod(remaining_s, 3600)
m, s = divmod(rem, 60)
if h > 0:
eta_display = f"{h}h {m:02d}m {s:02d}s"
elif m > 0:
eta_display = f"{m}m {s:02d}s"
else:
eta_display = f"{s}s"
else:
eta_display = "N/A"
# ------------------------------------------------------------------
# --- format estimated finish (wall-clock) time --------------------
finish_str = self.progress.get("estimated_finish_time")
if finish_str is not None:
import datetime as _dt
finish_display = _dt.datetime.fromisoformat(finish_str).strftime(
"%Y-%m-%d %H:%M:%S"
)
else:
finish_display = "N/A"
# ------------------------------------------------------------------
text = (
f"Progress report:\n"
f" Tomo type: {self.progress['tomo_type']}\n"
f" Projection: {self.progress['projection']:.0f}\n"
f" Total projections expected {self.progress['total_projections']:.1f}\n"
f" Angle: {self.progress['angle']:.1f}\n"
f" Current subtomo: {self.progress['subtomo']}\n"
f" Current projection within subtomo: {self.progress['subtomo_projection']}\n"
f" Total projections per subtomo: {int(self.progress['subtomo_total_projections'])}\n"
f" Scan started: {start_display}\n"
f" Est. remaining: {eta_display}\n"
f" Est. finish: {finish_display}"
)
# self._describe_active_hook() comes from TomoQueueMixin (this
# is a mixin, always combined with it) -- shown only when a
# hook is actually active, same as tomo_parameters()'s CLI note.
hook_description = self._describe_active_hook()
if hook_description:
text += f"\n Hook: {hook_description}"
self.progressbar.set_center_label(text)
except Exception as exc:
logger.warning(f"OMNY progress GUI update skipped: {exc}")
File diff suppressed because it is too large. Load diff
@@ -1,6 +1,7 @@
import time
import numpy as np
import os
from pathlib import Path
from rich import box
from rich.console import Console
@@ -17,8 +18,87 @@ class OMNYAlignmentError(Exception):
class OMNYAlignmentMixin:
default_correction_file = "correction_omny_202204.txt"
default_correction_file_x = "correction_omny_202204_x.txt"
# Resolved to an absolute path (mirrors Flomni's default_correction_file
# setup) -- a bare relative filename would only ever be found if bec were
# launched from this exact plugin directory, silently failing (caught by
# the FileNotFoundError handler below) everywhere else.
import csaxs_bec
_csaxs_bec_basepath = Path(csaxs_bec.__file__).parent
default_correction_file = (
_csaxs_bec_basepath
/ "bec_ipython_client"
/ "plugins"
/ "omny"
/ "correction_omny_202204.txt"
).resolve()
default_correction_file_x = (
_csaxs_bec_basepath
/ "bec_ipython_client"
/ "plugins"
/ "omny"
/ "correction_omny_202204_x.txt"
).resolve()
del csaxs_bec
# additional x/y-correction curves: global-var-backed so they survive a
# BEC client restart, same pattern as every other persistent setting in
# this class (and matching Flomni's corr_pos_y/corr_angle_y/etc.). NOTE:
# must NOT be pre-assigned in OMNY.__init__ -- doing so would overwrite
# the persisted value with [] on every restart.
@property
def corr_pos_x(self):
val = self.client.get_global_var("corr_pos_x")
return [] if val is None else val
@corr_pos_x.setter
def corr_pos_x(self, val: list):
self.client.set_global_var("corr_pos_x", val)
@property
def corr_angle_x(self):
val = self.client.get_global_var("corr_angle_x")
return [] if val is None else val
@corr_angle_x.setter
def corr_angle_x(self, val: list):
self.client.set_global_var("corr_angle_x", val)
@property
def corr_pos_y(self):
val = self.client.get_global_var("corr_pos_y")
return [] if val is None else val
@corr_pos_y.setter
def corr_pos_y(self, val: list):
self.client.set_global_var("corr_pos_y", val)
@property
def corr_angle_y(self):
val = self.client.get_global_var("corr_angle_y")
return [] if val is None else val
@corr_angle_y.setter
def corr_angle_y(self, val: list):
self.client.set_global_var("corr_angle_y", val)
@property
def corr_pos_y_2(self):
val = self.client.get_global_var("corr_pos_y_2")
return [] if val is None else val
@corr_pos_y_2.setter
def corr_pos_y_2(self, val: list):
self.client.set_global_var("corr_pos_y_2", val)
@property
def corr_angle_y_2(self):
val = self.client.get_global_var("corr_angle_y_2")
return [] if val is None else val
@corr_angle_y_2.setter
def corr_angle_y_2(self, val: list):
self.client.set_global_var("corr_angle_y_2", val)
def reset_correction(self, use_default_correction=True):
"""
@@ -215,6 +295,10 @@ class OMNYAlignmentMixin:
return self._compute_additional_correction(angle, iteration="y2")
def _compute_additional_correction(self, angle, iteration="y1"):
# corr_pos_x/corr_pos_y/corr_pos_y_2 etc. are global-var-backed
# properties that default to [] -- never AttributeError, even before
# reset_correction() has ever run (e.g. testing scan mechanics before
# any sample transfer).
if iteration == "x1":
corr_pos = self.corr_pos_x
corr_angle = self.corr_angle_x
@@ -129,16 +129,44 @@ class OMNYOpticsMixin:
if np.fabs(omny_oosa_currentx + 2) > 0.1:
raise OMNYError("oosax did not reach target position. Not moving in z.\n")
def _oosa_is_at(self, x_pos, y_pos, z_pos, tol=0.05) -> bool:
"""True if oosax/oosay/oosaz are all within tol of the given position."""
dev.oosaz.read(cached=False)
return (
np.fabs(dev.oosax.get().readback - x_pos) < tol
and np.fabs(dev.oosay.get().readback - y_pos) < tol
and np.fabs(dev.oosaz.get().readback - z_pos) < tol
)
def oosa_in(self):
# Compute the target *before* touching anything: _oosa_check_y() below
# unconditionally parks oosay at 0.9, which is not the "in" position for
# either mode, so the "already there" check must happen first or it
# would never see the real (pre-park) oosay reading.
if self.near_field == False:
mode, x_in_pos, y_in_pos, z_in_pos = (
"far-field",
self._get_user_param_safe("oosax", "far_field_in"),
self._get_user_param_safe("oosay", "far_field_in"),
self._get_user_param_safe("oosaz", "far_field_in"),
)
else:
mode, x_in_pos, y_in_pos, z_in_pos = (
"near-field",
self._get_user_param_safe("oosax", "near_field_in"),
self._get_user_param_safe("oosay", "near_field_in"),
self._get_user_param_safe("oosaz", "near_field_in"),
)
if self._oosa_is_at(x_in_pos, y_in_pos, z_in_pos):
self.OMNYTools.printgreen(f"OSA is already at the IN position ({mode}).")
return
self._oosa_check_y()
dev.oshield.read(cached=False)
omny_oshield_current = dev.oshield.get().readback
if omny_oshield_current < 15:
self._oshield_ST_close()
if self.near_field == False:
x_in_pos = self._get_user_param_safe("oosax", "far_field_in")
y_in_pos = self._get_user_param_safe("oosay", "far_field_in")
z_in_pos = self._get_user_param_safe("oosaz", "far_field_in")
print("OSA movement in far-field mode.")
dev.oosaz.read(cached=False)
omny_oosa_currentz = dev.oosaz.get().readback
@@ -154,9 +182,6 @@ class OMNYOpticsMixin:
# umv oosay 0.386015
if self.near_field == True:
x_in_pos = self._get_user_param_safe("oosax", "near_field_in")
y_in_pos = self._get_user_param_safe("oosay", "near_field_in")
z_in_pos = self._get_user_param_safe("oosaz", "near_field_in")
print("OSA movement in near-field mode.")
dev.oosaz.read(cached=False)
omny_oosa_currentz = dev.oosaz.get().readback
@@ -178,14 +203,26 @@ class OMNYOpticsMixin:
# rt_feedback_enable
def oosa_out(self):
# oosa_out() doesn't move oosay, so "already out" only depends on x/z;
# read it before _oosa_check_y() parks oosay at 0.9 below (same reason
# as in oosa_in()).
omny_oosay_current = dev.oosay.get().readback
if self.near_field == False:
mode, z_out_pos = "far-field", 6.5
else:
mode, z_out_pos = "near-field", -0.45
if self._oosa_is_at(-2, omny_oosay_current, z_out_pos):
self.OMNYTools.printgreen(f"OSA is already at the OUT position ({mode}).")
return
self._oosa_check_y()
dev.oshield.read(cached=False)
omny_oshield_current = dev.oshield.get().readback
if omny_oshield_current < 15:
self._oshield_ST_close()
omny_oosaz_current = dev.oosaz.get().readback
if self.near_field == False:
print("OSA movement in far-field mode.")
omny_oosaz_current = dev.oosaz.get().readback
if omny_oosaz_current < 6.4:
self._oosa_to_move_corridor()
dev.oosaz.limits = [6.4, 6.6]
@@ -193,6 +230,7 @@ class OMNYOpticsMixin:
umv(dev.oosax, -2)
if self.near_field == True:
print("OSA movement in near-field mode.")
omny_oosaz_current = dev.oosaz.get().readback
if omny_oosaz_current > 0:
self._oosa_to_move_corridor()
dev.oosaz.limits = [-0.4, -0.6]
@@ -217,18 +255,28 @@ class OMNYOpticsMixin:
self.OMNYTools.printgreen("OSA is out of shield.")
def ofzp_out(self):
y_out_pos = self._get_user_param_safe("ofzpy", "out")
needs_move = np.fabs(dev.ofzpy.get().readback - y_out_pos) > 0.02
if not needs_move:
self.OMNYTools.printgreen("FZP is already at the out position")
return
if "rtx" in dev and dev.rtx.enabled:
dev.rtx.controller.feedback_disable()
y_out_pos = self._get_user_param_safe("ofzpy", "out")
if np.fabs(dev.ofzpy.get().readback - y_out_pos) > 0.02:
umv(dev.ofzpy, y_out_pos)
umv(dev.ofzpy, y_out_pos)
self.OMNYTools.printgreen("FZP at out position")
def ofzp_in(self):
if "rtx" in dev and dev.rtx.enabled:
dev.rtx.controller.feedback_disable()
x_in_pos = self._get_user_param_safe("ofzpx", "in")
y_in_pos = self._get_user_param_safe("ofzpy", "in")
needs_move = (
np.fabs(dev.ofzpy.get().readback - y_in_pos) > 0.02
or np.fabs(dev.ofzpx.get().readback - x_in_pos) > 0.02
)
if not needs_move:
self.OMNYTools.printgreen("FZP is already at the in position")
return
if "rtx" in dev and dev.rtx.enabled:
dev.rtx.controller.feedback_disable()
if np.fabs(dev.ofzpy.get().readback - y_in_pos) > 0.02:
umv(dev.ofzpy, y_in_pos)
@@ -1,6 +1,8 @@
import os
import time
import builtins
import h5py
import numpy as np
from rich import box
@@ -10,8 +12,6 @@ from typeguard import typechecked
# from functools import wraps
from csaxs_bec.bec_ipython_client.plugins.cSAXS import epics_get, epics_put, fshopen, fshclose
if builtins.__dict__.get("bec") is not None:
bec = builtins.__dict__.get("bec")
@@ -26,6 +26,30 @@ class OMNYTransferError(Exception):
pass
# otransfer_park_slot()'s oparkz targets (regular gripper transfer path).
_OPARKZ_TARGET_POSITIONS = {
2: -0.9 - 29 * 3 - 42 - 0.05,
3: -0.9 - 29 * 3,
4: -0.9 - 29 * 2,
5: -0.9 - 29,
6: -0.9,
}
# otransfer_park_loadlock_slot()'s targets (_oparkz_leica_slot()) -- the
# *same* slot numbers, reached via the loadlock/leica transfer path instead
# of the regular gripper path, at a different oparkz offset (base -6, not
# -0.9). Both genuinely mean "the parking stage is at that slot", so
# _oparkz_slot_check() must recognize either -- see also
# sample_storage.py's oparkz_active_parking_slot(), which duplicates this
# same pair of tables on the widget side of the device-server/GUI boundary.
_OPARKZ_LEICA_TARGET_POSITIONS = {
2: -6 - 29 * 3 - 42 - 0.05,
3: -6 - 29 * 3,
4: -6 - 29 * 2,
5: -6 - 29,
6: -6,
}
class OMNYSampleTransferMixin:
def __init__(self) -> None:
@@ -49,6 +73,27 @@ class OMNYSampleTransferMixin:
raise ValueError(f"Device {device} has no user parameter definition for {var}.")
return param.get(var)
def otransfer_confirm_dialog(self, message: str, default: str = "none") -> bool:
"""
Yes/No confirmation for sample-transfer steps -- OMNY's counterpart
to flomni.py's ``ftransfer_confirm_dialog()``.
Uses the GUI console widget (ConsoleButtonsWidget, opened by
``_omnycam_parking()``/``_omnycam_samplestage()`` in gui_tools.py)
when it's already available, so the operator can confirm/abort
right next to the live camera view, instead of a blocking terminal
``input()`` prompt. Falls back to the regular console ``yesno()``
prompt if the cameras/console haven't been opened yet -- e.g. if a
transfer step is called directly, outside the usual
``otransfer_get_sample``/``otransfer_put_sample`` sequence that
opens them first. ``self.console`` is set by ``OMNYGuiTools``
(a sibling mixin on the composed ``OMNY`` class), not this one.
"""
if self.console is not None:
print("Using GUI console for confirmation dialog.")
return self.OMNYTools.gui_yesno(message, gui=self.console, default=default)
return self.OMNYTools.yesno(message, default)
def _otransfer_shuttle_aligner_down(self):
if not dev.oshuttlealign.get_motor_limit_switch()[1]:
dev.oshuttlealign.drive_axis_to_limit("forward")
@@ -141,7 +186,7 @@ class OMNYSampleTransferMixin:
if not dev.oshuttlealign.get_motor_limit_switch()[0]:
print("Not in correct start position for alignment.")
dev.oshuttlealign.controller.socket_put_confirmed("MO")
if self.OMNYTools.yesno(
if self.otransfer_confirm_dialog(
"Shall I try to move it to the correct start position? Do this only once!"
):
print("starting...")
@@ -371,7 +416,7 @@ class OMNYSampleTransferMixin:
def _otransfer_gripper_to_park_z(self):
if not self._otransfer_gripper_querry_status_closed():
if self.OMNYTools.yesno(
if self.otransfer_confirm_dialog(
"The gripper appears to be open, while it was expected to be closed.May it be closed now?",
"y",
):
@@ -472,7 +517,10 @@ class OMNYSampleTransferMixin:
)
# if the valve of the loadlock is not closed, ask before movement
# todo if(epics_get("XOMNY-ES1-LL1VG1:POSITION")!="CLOSED") {
if self.OMNYTools.yesno("Please confirm that the leica transfer is in back position."):
# CLI prompt on purpose, not otransfer_confirm_dialog() -- the operator
# is physically at the loadlock/leica transfer, not watching the GUI
# camera console, when confirming this.
if not self.OMNYTools.yesno("Please confirm that the leica transfer is in back position."):
raise OMNYTransferError("Leica shuttle is not in back position.")
else:
self._oparkz_leica_slot(slot_number)
@@ -511,17 +559,19 @@ class OMNYSampleTransferMixin:
return False
def _oparkz_slot_check(self):
# Checks both the regular-gripper (_OPARKZ_TARGET_POSITIONS) and
# loadlock/leica (_OPARKZ_LEICA_TARGET_POSITIONS) target tables --
# otransfer_park_loadlock_slot() parks at a different oparkz offset
# than otransfer_park_slot() for the same slot number, and both
# genuinely mean the parking stage is at that slot.
parkingposition = 10
if self._oparkz_check(-0.9):
parkingposition = 6
elif self._oparkz_check(-0.9 - 29):
parkingposition = 5
elif self._oparkz_check(-0.9 - 29 * 2):
parkingposition = 4
elif self._oparkz_check(-0.9 - 29 * 3):
parkingposition = 3
elif self._oparkz_check(-0.9 - 29 * 3 - 42 - 0.05):
parkingposition = 2
for targets in (_OPARKZ_TARGET_POSITIONS, _OPARKZ_LEICA_TARGET_POSITIONS):
for slot, target in targets.items():
if self._oparkz_check(target):
parkingposition = slot
break
if parkingposition != 10:
break
if parkingposition == 10:
print("Oparkz is not in a parking position.")
@@ -740,9 +790,13 @@ class OMNYSampleTransferMixin:
return bool(float(dev.otransy.controller.socket_put_and_receive("MGgrstat")))
def otransfer_put_sample(self, pin_position: int):
self.transfer_step = 0
self._otransfer_check_sensor_connected()
if not self._otransfer_check_free_slot_available_at_position(pin_position, 0):
raise OMNYTransferError(f"position {pin_position} is not free")
raise OMNYTransferError(
f"position {pin_position} is not free, or no shuttle is parked at the"
" active parking position"
)
# global _ogalil_encoder_steps_per_mm
if pin_position == 0:
@@ -755,21 +809,31 @@ class OMNYSampleTransferMixin:
self._otransfer_controller_enable_mount_mode()
dev.otransy.controller.socket_put_confirmed(f"mntaprch={_otransy_approach_height:.2f}")
input("Ready. Press Enter to start the mount process now...")
if not self.otransfer_confirm_dialog("Ready to start the mount process now?", "y"):
raise OMNYTransferError("Transfer process has been manually aborted.")
self._otransfer_save_reference_images(file_suffix="put_mount_start")
dev.otransy.controller.socket_put_confirmed("XQ#GRPUT")
print("The mount process started.")
self._otransfer_confirm()
self._otransfer_confirm(step_name="put")
self._otransfer_controller_disable_mount_mode()
# update the new sample status in storage
self._otransfer_check_free_slot_available_at_position(pin_position, 1)
if pin_position == 0:
# sample is now in the sample stage (scanning position): any alignment
# correction data belongs to the previous sample, discard it -- matches
# flomni's ftransfer_put_sample()/ftransfer_flomni_stage_in()
self.reset_correction()
self.reset_tomo_alignment_fit()
# this also moves the gripper to parking position
# alternatively a next sample is to be picked
# keep shuttle open and ask user which sample to pick
# but have to make sure that shield will be closed for sample pos movements beyond shuttle positions
if pin_position in range(1, 7):
if self.OMNYTools.yesno(
if self.otransfer_confirm_dialog(
"Pick another sample from this shuttle? Keep shield open?", "y"
):
print(
@@ -781,7 +845,7 @@ class OMNYSampleTransferMixin:
self._otransfer_shield_close(pin_position)
dev.omny_samples.show_all()
def _otransfer_confirm(self):
def _otransfer_confirm(self, step_name: str = ""):
mntprgs = 1
while mntprgs != 0:
@@ -800,7 +864,7 @@ class OMNYSampleTransferMixin:
toppin = self._otransfer_get_toppin_position()
print(f"TopPin position is {toppin:.3f} mm")
if self._otransfer_get_toppin_position() > -25.4:
if not self.OMNYTools.yesno(
if not self.otransfer_confirm_dialog(
"THIS IS LARGER THAN THE CURRENT THRESHOLD OF -25.4. ARE YOU SURE THE PIN IS OK AND DOES NOT HAVE A DAMAGE IN THE BOTTOM? Continue?"
):
raise OMNYTransferError("Abort by user because of top pin position.")
@@ -811,11 +875,49 @@ class OMNYSampleTransferMixin:
elif mntprgs == 0:
text = "All OK? Please confirm"
if self.OMNYTools.yesno(text, "y"):
if self.otransfer_confirm_dialog(text, "y"):
self.transfer_step += 1
self._otransfer_save_reference_images(file_suffix=f"{step_name}_{self.transfer_step}")
dev.otransy.controller.socket_put_confirmed("confirm=1")
else:
raise OMNYTransferError("The sample mount progress was aborted by user.")
def _otransfer_save_reference_images(self, file_suffix: str = "") -> None:
"""Save the currently-active camera pair's last images to one HDF5
file -- OMNY's equivalent of flomni's ``save_reference_image()``,
called at each manual confirm checkpoint (see ``_otransfer_confirm``).
Unlike flomni (one fixed camera, ``cam_flomni_gripper``), OMNY
always shows exactly one *pair* of cameras live at a time --
parking/shuttle (cam200+cam203) or sample stage (cam201+cam202) --
depending on where the gripper currently is (see
``_omnycam_parking()``/``_omnycam_samplestage()`` in gui_tools.py,
which record that pair on ``self._omnycam_active_pair``). Both
images from the active pair go into one file, keyed by camera name,
rather than flomni's single "reference_image" dataset.
IDSCamera's own ``get_last_image()`` is used directly (same as
``x_ray_eye_align.py``'s ``update_frame()`` already does for
cam_xeye), rather than flomni's BEC-preview-redis-endpoint pull --
cam200-203 are IDSCamera, not the WebcamViewer class flomni's
cameras use.
"""
pair = getattr(self, "_omnycam_active_pair", None)
if not pair:
print("No active OMNY camera pair known; skipping reference-image save.")
return
suffix = f"_{file_suffix}_" if file_suffix else "_"
timestamp = time.strftime("%Y%m%d_%H%M%S")
file = os.path.expanduser(
f"~/data/raw/logs/omny_sample_transfer_images/omny_sample_transfer_reference_image{suffix}{timestamp}.h5"
)
os.makedirs(os.path.dirname(file), exist_ok=True)
with h5py.File(file, "w") as f:
for cam_name in pair:
image = getattr(dev, cam_name).get_last_image()
if image is not None:
f.create_dataset(cam_name, data=image)
def _otransfer_check_sensor_connected(self):
sensorvoltage = float(dev.otransy.controller.socket_put_and_receive("MG@AN[8001]"))
gripper_sensorvoltagetarget = self._get_user_param_safe(
@@ -827,19 +929,24 @@ class OMNYSampleTransferMixin:
)
def otransfer_get_sample(self, pin_position: int):
self.transfer_step = 0
self._omnycam_parking()
if not self.OMNYTools.yesno(
if not self.otransfer_confirm_dialog(
"Please confirm that currently there is no sample in the gripper. It would be dropped!",
"y",
):
raise OMNYTransferError("Transfer process has been manually aborted.")
self._otransfer_save_reference_images(file_suffix="get_gripper_empty_check")
# clean abort before starting
dev.otransx.controller.socket_put_confirmed("XQ#STOP,1")
time.sleep(0.1)
self._otransfer_check_sensor_connected()
if not self._otransfer_check_sample_available_at_position(pin_position, 0):
raise OMNYTransferError(f"There is no sample available at position {pin_position}")
raise OMNYTransferError(
f"There is no sample available at position {pin_position}, or no shuttle is"
" parked at the active parking position"
)
# global _otransy_approach_height
# global _ogalil_encoder_steps_per_mm
@@ -861,20 +968,22 @@ class OMNYSampleTransferMixin:
dev.otransy.controller.socket_put_confirmed(f"getaprch={_otransy_approach_height:.2f}")
input("Ready. Press Enter to start the mount process now...")
if not self.otransfer_confirm_dialog("Ready to start the unmount process now?", "y"):
raise OMNYTransferError("Transfer process has been manually aborted.")
self._otransfer_save_reference_images(file_suffix="get_unmount_start")
dev.otransy.controller.socket_put_confirmed("XQ#GRGET")
print("The Unmount process started.")
time.sleep(1)
self._otransfer_confirm()
self._otransfer_confirm(step_name="get")
self._otransfer_controller_disable_mount_mode()
# update database
self._otransfer_check_sample_available_at_position(pin_position, 1)
self._otransfer_shield_close(pin_position)
self.otransfer_storage()
self.otransfer_show_all()
def _otransfer_stage_to_ST(self, pin_position):
self._otransfer_gripper_to_park_z()
@@ -907,14 +1016,18 @@ class OMNYSampleTransferMixin:
feedback_status = self.device_manager.devices.rtx.controller.feedback_is_running()
if feedback_status == True:
# feedback is running
if self.OMNYTools.yesno("May the feedback be disabled", "y"):
if not self.otransfer_confirm_dialog("May the feedback be disabled", "y"):
raise OMNYTransferError(
"Manual abort of sample transfer, need to disable feedback has not been confirmed."
)
self.device_manager.devices.rtx.controller.feedback_disable()
fshclose()
# dev.fsh (not the raw cSAXS_beamline.fshclose(), which does a bare
# epics.caput() straight to X12SA-ES1-TTL:OUT_01 outside BEC entirely
# -- unreachable/unmocked in a simulated session) -- same device
# omny_optics_mixin.py already uses for shutter control.
dev.fsh.fshclose()
if oeyez_current < -20:
raise OMNYTransferError(
@@ -1056,9 +1169,99 @@ class OMNYSampleTransferMixin:
# }'
def otransfer_storage(self):
def otransfer_show_all(self):
dev.omny_samples.show_all()
# Thin dev.omny_samples wrappers below, so every sample-storage command
# tab-completes under omny.otransfer_* -- same reason flomni.py exposes
# dev.flomni_samples via ftransfer_* -- instead of only being reachable
# via dev.omny_samples.* directly.
def otransfer_is_sample_slot_used(self, container: str, slot_nr: int) -> bool:
return dev.omny_samples.is_sample_slot_used(container, slot_nr)
def otransfer_is_sample_in_gripper(self) -> bool:
return dev.omny_samples.is_sample_in_gripper()
def otransfer_set_sample_slot(self, container: str, slot_nr: int, name: str, owner: str = "") -> bool:
return dev.omny_samples.set_sample_slot(container, slot_nr, name, owner)
def otransfer_unset_sample_slot(self, container: str, slot_nr: int) -> bool:
return dev.omny_samples.unset_sample_slot(container, slot_nr)
def otransfer_set_sample_in_gripper(self, name: str, owner: str = "") -> bool:
return dev.omny_samples.set_sample_in_gripper(name, owner)
def otransfer_unset_sample_in_gripper(self) -> bool:
return dev.omny_samples.unset_sample_in_gripper()
def otransfer_set_sample_in_samplestage(self, name: str, owner: str = "") -> bool:
return dev.omny_samples.set_sample_in_samplestage(name, owner)
def otransfer_unset_sample_in_samplestage(self) -> bool:
return dev.omny_samples.unset_sample_in_samplestage()
def otransfer_get_sample_name_in_samplestage(self) -> str:
return dev.omny_samples.get_sample_name_in_samplestage()
def otransfer_get_sample_name_in_gripper(self) -> str:
return dev.omny_samples.get_sample_name_in_gripper()
def otransfer_get_sample_name(self, container: str, slot_nr: int) -> str:
return dev.omny_samples.get_sample_name(container, slot_nr)
def otransfer_is_sample_in_samplestage(self) -> bool:
return dev.omny_samples.is_sample_in_samplestage()
def otransfer_set_shuttle_slot(self, container: str, slot_nr: int) -> bool:
return dev.omny_samples.set_shuttle_slot(container, slot_nr)
def otransfer_unset_shuttle_slot(self, slot_nr: int) -> bool:
return dev.omny_samples.unset_shuttle_slot(slot_nr)
def otransfer_get_shuttle_name_slot(self, slot_nr: int) -> str:
return dev.omny_samples.get_shuttle_name_slot(slot_nr)
def otransfer_is_shuttle_slot_used(self, slot_nr: int) -> bool:
return dev.omny_samples.is_shuttle_slot_used(slot_nr)
def otransfer_search_shuttle_in_slot(self, shuttle: str) -> int:
return dev.omny_samples.search_shuttle_in_slot(shuttle)
def otransfer_get_sample_owner(self, container: str, slot_nr: int) -> str:
return dev.omny_samples.get_sample_owner(container, slot_nr)
def otransfer_get_sample_name_and_owner(self, container: str, slot_nr: int) -> tuple:
return dev.omny_samples.get_sample_name_and_owner(container, slot_nr)
def otransfer_set_sample_measured(
self, container: str, slot_nr: int, status: str, meas_date: str, owner_if_empty: str = ""
):
return dev.omny_samples.set_sample_measured(container, slot_nr, status, meas_date, owner_if_empty)
def otransfer_get_sample_status(self, container: str, slot_nr: int) -> str:
return dev.omny_samples.get_sample_status(container, slot_nr)
def otransfer_get_sample_meas_date(self, container: str, slot_nr: int) -> str:
return dev.omny_samples.get_sample_meas_date(container, slot_nr)
def otransfer_get_sample_record(self, container: str, slot_nr: int) -> dict:
return dev.omny_samples.get_sample_record(container, slot_nr)
def otransfer_transfer_sample(self, from_slot, to_slot):
return dev.omny_samples.transfer_sample(from_slot, to_slot)
def otransfer_get_sample_owner_in_gripper(self) -> str:
return dev.omny_samples.get_sample_owner_in_gripper()
def otransfer_get_sample_owner_in_samplestage(self) -> str:
return dev.omny_samples.get_sample_owner_in_samplestage()
def otransfer_get_sample_status_in_gripper(self) -> str:
return dev.omny_samples.get_sample_status_in_gripper()
def otransfer_get_sample_status_in_samplestage(self) -> str:
return dev.omny_samples.get_sample_status_in_samplestage()
# def _otransfer_modify_sample(_ocontainer, _oposition, _ostatus, _oname):
# #_ocontainer is A, B, C for the pin containers
# #it can also be FZP, OSA and omny
@@ -1124,12 +1327,11 @@ class OMNYSampleTransferMixin:
# is there a sample available?
if dev.omny_samples.is_sample_slot_used(installedshuttle, _osampleposition):
if _picked_it == 1:
samplename = dev.omny_samples.get_sample_name(
installedshuttle, _osampleposition
# remove sample from slot and put to gripper -- name/owner/status/date
# carry over with it, the same physical sample
dev.omny_samples.transfer_sample(
(installedshuttle, _osampleposition), "GRIPPER"
)
# remove sample from slot and put to gripper
dev.omny_samples.unset_sample_slot(installedshuttle, _osampleposition)
dev.omny_samples.set_sample_in_gripper(samplename)
return True
else:
print("There is no sample availabe at that position.")
@@ -1217,9 +1419,7 @@ class OMNYSampleTransferMixin:
):
if dev.omny_samples.is_sample_slot_used("O", _osampleposition):
if _picked_it:
samplename = dev.omny_samples.get_sample_name("O", _osampleposition)
dev.omny_samples.unset_sample_slot("O", _osampleposition)
dev.omny_samples.set_sample_in_gripper(samplename)
dev.omny_samples.transfer_sample(("O", _osampleposition), "GRIPPER")
# FZP was picked
# if (_osampleposition == 10 || _osampleposition == 11 || _osampleposition == 100)
# _otransfer_modify_sample("omny",110,2,samplename)
@@ -1238,9 +1438,7 @@ class OMNYSampleTransferMixin:
elif _osampleposition == 0:
if dev.omny_samples.is_sample_in_samplestage():
if _picked_it:
samplename = dev.omny_samples.get_sample_name_in_samplestage()
dev.omny_samples.unset_sample_in_samplestage()
dev.omny_samples.set_sample_in_gripper(samplename)
dev.omny_samples.transfer_sample("STAGE", "GRIPPER")
# FZP was picked
# if (_osampleposition == 10 || _osampleposition == 11 || _osampleposition == 100)
# _otransfer_modify_sample("omny",110,2,samplename)
@@ -1312,10 +1510,8 @@ class OMNYSampleTransferMixin:
# is there free position available?
if not dev.omny_samples.is_sample_slot_used(installedshuttle, _osampleposition):
if _put_it:
samplename = dev.omny_samples.get_sample_name_in_gripper()
dev.omny_samples.unset_sample_in_gripper()
dev.omny_samples.set_sample_slot(
installedshuttle, _osampleposition, samplename
dev.omny_samples.transfer_sample(
"GRIPPER", (installedshuttle, _osampleposition)
)
return True
else:
@@ -1404,9 +1600,7 @@ class OMNYSampleTransferMixin:
# is there a free slot available?
if not dev.omny_samples.is_sample_slot_used("O", _osampleposition):
if _put_it:
samplename = dev.omny_samples.get_sample_name_in_gripper()
dev.omny_samples.set_sample_slot("O", _osampleposition, samplename)
dev.omny_samples.unset_sample_in_gripper()
dev.omny_samples.transfer_sample("GRIPPER", ("O", _osampleposition))
return True
else:
print("There is no free space at this position.")
@@ -1416,9 +1610,7 @@ class OMNYSampleTransferMixin:
elif _osampleposition == 0:
if not dev.omny_samples.is_sample_in_samplestage():
if _put_it:
samplename = dev.omny_samples.get_sample_name_in_gripper()
dev.omny_samples.set_sample_in_samplestage(samplename)
dev.omny_samples.unset_sample_in_gripper()
dev.omny_samples.transfer_sample("GRIPPER", "STAGE")
return True
else:
print("There is no sample in the sample stage.")
@@ -1435,4 +1627,10 @@ class OMNYSampleTransferMixin:
)
print("omny.otransfer_get_sample(position) pick with the gripper from <position>")
print("omny.otransfer_put_sample(position) put with the gripper to <position>")
print("To modify the storage information see dev.omny_samples.help()")
print("omny.otransfer_show_all() show the full sample storage overview")
print(
"The other storage commands (set/unset slots, shuttles, gripper, sample stage,"
)
print("owner, measured status) are also available as omny.otransfer_* -- see below:")
print()
dev.omny_samples.help()
@@ -57,9 +57,13 @@ class XrayEyeAlign:
# ...
# k=8 : sample at 315°
self.alignment_values: dict[int, list[float]] = {}
# Reset correction state via the existing alignment object on lamni
self.lamni.align.reset_correction()
self.lamni.align.reset_xray_eye_correction()
# Deliberately NOT calling self.lamni.align.reset_correction()/
# reset_xray_eye_correction() here: `self.lamni` (the OMNY instance) is
# still mid-construction when this runs (OMNY.__init__ does
# `self.align = XrayEyeAlign(self.client, self)`), so `self.lamni.align`
# doesn't exist yet -- this crashed every OMNY() construction with
# AttributeError. Matches LamNI's x_ray_eye_align.py, which was fixed
# the same way (see its __init__ for the full rationale).
# ------------------------------------------------------------------
# GUI shortcut
@@ -75,7 +79,7 @@ class XrayEyeAlign:
# ------------------------------------------------------------------
def _reset_init_values(self):
self.shift_xy = [0.0, 0.0] # base shift to bring sample to beam centre [µm]
self.shift_xy = [0.0, 0.0] # base shift to bring sample to beam centre [µm]
self._xray_fov_xy = [0.0, 0.0]
def tomo_rotate(self, val: float):
@@ -157,7 +161,7 @@ class XrayEyeAlign:
Compute x/y offsets, push to GUI fit tab, wait for DAP fit,
then load fit parameters into the global variable store.
"""
self.lamni.lamnigui_show_xeyealign()
self.lamni.omnygui_show_xeyealign()
self.send_message("Getting things ready. Please wait...")
self.gui.enable_submit_button(False)
@@ -184,14 +188,8 @@ class XrayEyeAlign:
k = 0
while True:
if dev.omny_xray_gui.submit.get() == 1:
val_x = (
getattr(dev.omny_xray_gui, f"xval_x_{k}").get()
* self.PIXEL_CALIBRATION
)
val_y = (
getattr(dev.omny_xray_gui, f"yval_y_{k}").get()
* self.PIXEL_CALIBRATION
)
val_x = getattr(dev.omny_xray_gui, f"xval_x_{k}").get() * self.PIXEL_CALIBRATION
val_y = getattr(dev.omny_xray_gui, f"yval_y_{k}").get() * self.PIXEL_CALIBRATION
self.alignment_values[k] = [val_x, val_y]
print(
f"Clicked position {k}: "
@@ -238,15 +236,11 @@ class XrayEyeAlign:
self.gui.enable_submit_button(False)
self.scans.lamni_move_to_scan_center(
self.shift_xy[0] / 1000,
self.shift_xy[1] / 1000,
self.get_tomo_angle(),
self.shift_xy[0] / 1000, self.shift_xy[1] / 1000, self.get_tomo_angle()
).wait()
time.sleep(1)
self.scans.lamni_move_to_scan_center(
self.shift_xy[0] / 1000,
self.shift_xy[1] / 1000,
self.get_tomo_angle(),
self.shift_xy[0] / 1000, self.shift_xy[1] / 1000, self.get_tomo_angle()
).wait()
dev.omny_xray_gui.angle.set(self.get_tomo_angle())
@@ -270,16 +264,12 @@ class XrayEyeAlign:
self._disable_rt_feedback()
self.tomo_rotate(approach_angle)
self.scans.lamni_move_to_scan_center(
self.shift_xy[0] / 1000,
self.shift_xy[1] / 1000,
self.get_tomo_angle(),
self.shift_xy[0] / 1000, self.shift_xy[1] / 1000, self.get_tomo_angle()
).wait()
self._disable_rt_feedback()
self.tomo_rotate(target_angle)
self.scans.lamni_move_to_scan_center(
self.shift_xy[0] / 1000,
self.shift_xy[1] / 1000,
self.get_tomo_angle(),
self.shift_xy[0] / 1000, self.shift_xy[1] / 1000, self.get_tomo_angle()
).wait()
dev.omny_xray_gui.angle.set(self.get_tomo_angle())
@@ -313,9 +303,7 @@ class XrayEyeAlign:
self.shift_xy[0] += _mvx
self.shift_xy[1] += _mvy
self.scans.lamni_move_to_scan_center(
self.shift_xy[0] / 1000,
self.shift_xy[1] / 1000,
self.get_tomo_angle(),
self.shift_xy[0] / 1000, self.shift_xy[1] / 1000, self.get_tomo_angle()
).wait()
print(
f"Current centre: "
@@ -346,7 +334,7 @@ class XrayEyeAlign:
self.client.set_global_var("tomo_fov_offset", self.shift_xy)
# Switch GUI to fit tab and wait for DAP to finish fitting
self.lamni.lamnigui_show_xeyealign_fittab()
self.lamni.omnygui_show_xeyealign_fittab()
print("Waiting 5 s for DAP sinusoidal fit to converge...")
time.sleep(5)
@@ -395,12 +383,8 @@ class XrayEyeAlign:
f.write("angle\thorizontal\tvertical\n")
for k in range(1, 9):
angle_deg = LAMNI_ALIGNMENT_ANGLES[k - 1]
x_off = (
self.alignment_values[1][0] - self.alignment_values[k][0]
) * 1000
y_off = (
self.alignment_values[k][1] - self.alignment_values[1][1]
) * 1000
x_off = (self.alignment_values[1][0] - self.alignment_values[k][0]) * 1000
y_off = (self.alignment_values[k][1] - self.alignment_values[1][1]) * 1000
f.write(f"{angle_deg}\t{x_off:.4f}\t{y_off:.4f}\n")
print(
f" Angle {angle_deg:3d} deg: "
@@ -424,4 +408,4 @@ class XrayEyeAlign:
# Push to XRayEye widget: feeds waveform_x (row 1) and waveform_y (row 2)
self.gui.submit_fit_array(data)
print(f"Fit data submitted with shape {data.shape}:\n{data}")
print(f"Fit data submitted with shape {data.shape}:\n{data}")
@@ -51,7 +51,7 @@ if _args.session.lower() == "lamni":
from csaxs_bec.bec_ipython_client.plugins.LamNI import LamNI
_session_name = "LamNI"
lamni = LamNI(bec)
lamni = LamNI(bec, csaxs=csaxs)
logger.success("LamNI session loaded.")
print(r"""
██████╗ ███████╗ ██████╗ ██╗ █████╗ ███╗ ███╗███╗ ██╗██╗
@@ -65,7 +65,7 @@ if _args.session.lower() == "lamni":
""")
elif _args.session.lower() == "omny":
from csaxs_bec.bec_ipython_client.plugins.flomni import OMNY
from csaxs_bec.bec_ipython_client.plugins.omny import OMNY
_session_name = "OMNY"
omny = OMNY(bec)
@@ -85,7 +85,10 @@ elif _args.session.lower() == "flomni":
from csaxs_bec.bec_ipython_client.plugins.flomni import Flomni
_session_name = "flomni"
flomni = Flomni(bec)
if "fttrx1" in dev:
flomni = Flomni(bec, csaxs=csaxs)
else:
flomni = Flomni(bec)
logger.success("flomni session loaded.")
print(r"""
██████╗ ███████╗ ██████╗ ███████╗██╗ ██████╗ ███╗ ███╗███╗ ██╗██╗
+96
View File
@@ -14,6 +14,7 @@ logger = bec_logger.logger
_Widgets = {
"DataViewer": "DataViewer",
"IDSCameraSettings": "IDSCameraSettings",
"OMNY_SampleStorage": "OMNY_SampleStorage",
"OMNY_TomoParams": "OMNY_TomoParams",
"OMNY_XRayEye": "OMNY_XRayEye",
@@ -47,6 +48,24 @@ class DataViewer(RPCBase):
"""
class IDSCameraSettings(RPCBase):
"""Pick a configured IDS camera and adjust its exposure time / pixel clock."""
_IMPORT_MODULE = "csaxs_bec.bec_widgets.widgets.ids_camera_settings.ids_camera_settings"
@rpc_call
def selected_camera():
"""
Intermediate wrapper used so that the user can optionally chain .setter(...).
"""
@rpc_call
def set_camera(self, name: "str"):
"""
Select a camera by device name, as if chosen from the dropdown.
"""
class OMNY_SampleStorage(RPCBase):
"""View and correct the FlOMNI sample-storage records."""
@@ -175,6 +194,19 @@ class OMNY_XRayEye(RPCBase):
enable(int): -1 disable else enable
"""
@rpc_timeout(20)
@rpc_call
def enable_skip_button(self, enable: "bool"):
"""
Allow/forbid skipping the current alignment angle.
The first call also shows the button (it is hidden for clients that never call this).
The button is only clickable while this is True *and* the submit button is enabled,
so it follows the client's "please wait ..." phases automatically.
Args:
enable(bool): True if the client accepts a skip for the current step
"""
@property
@rpc_call
def sample_name(self):
@@ -238,6 +270,70 @@ class OMNY_XRayEye(RPCBase):
Hide the alignment target crosshair on the image view.
"""
@rpc_timeout(20)
@rpc_call
def lock_vertical_center(self):
"""
Snap every ROI's vertical center to the crosshair's fixed y position
(see ``crosshair_position()``), immediately and again on every future
draw/drag/resize, until ``unlock_vertical_center()`` is called. Used by
``x_ray_eye_align.py`` once the sample's height has been physically
centered on the FZP, so subsequent alignment boxes no longer need manual
vertical placement. See ``_snap_roi_vertical()``.
Also settable by the operator directly via the "ROI vertical lock"
toggle in the control panel (``_on_vertical_lock_toggle_changed``) --
e.g. to temporarily release a script-engaged lock mid-run in order to
freely draw/measure an unrelated box, then re-engage it. Both paths
keep the toggle's visual state in sync with ``_vertical_lock_enabled``.
"""
@rpc_timeout(20)
@rpc_call
def unlock_vertical_center(self):
"""
Stop auto-snapping ROIs' vertical center -- see ``lock_vertical_center()``.
"""
@rpc_timeout(20)
@rpc_call
def enable_ellipse_roi(self, enable: "bool"):
"""
Show/hide the elliptical ROI tool and its angle controls.
Hidden by default; the LamNI client enables it after the FZP centre has been
submitted, for the extended/smear rotation-centre calibrations (angle locked)
and for the X-ray eye alignment (angle free). Disabling removes an existing
ellipse and releases the angle lock.
Args:
enable(bool): True to offer the ellipse tool.
"""
@rpc_timeout(20)
@rpc_call
def lock_roi_angle(self, angle: "float | None" = None):
"""
Fix the rotation of elliptical ROIs, now and for ellipses drawn later.
Args:
angle(float | None): Rotation in degrees. If None, the active ellipse's
current rotation (or, without one, the angle in the spin box) is used.
"""
@rpc_timeout(20)
@rpc_call
def unlock_roi_angle(self):
"""
Let elliptical ROIs rotate freely again -- see ``lock_roi_angle()``.
"""
@rpc_call
def roi_angle(self) -> "float | None":
"""
Rotation in degrees of the active elliptical ROI, or None if there is none.
"""
@property
@rpc_call
def crosshair_visible(self) -> "bool":
@@ -2,6 +2,8 @@ from __future__ import annotations
import os
import signal
import threading
import time
from bec_lib import bec_logger
from bec_widgets import BECWidget, SafeProperty, SafeSlot
@@ -26,11 +28,13 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
let an unrelated, un-suppressed device error kill the scan worker
thread and require a full BEC restart), then sends a real SIGINT to
the BEC IPython client process (the parent of the GUI server
process), then directly calls ``hard_stop_device_name``'s
``.controller.hard_abort_and_restore_positioning_mode()`` -- the same
Galil hard-stop (XQ#STOP,1) the CLI's ``ftransfer_abort()`` uses --
and, if configured, ``extra_hard_stop_device_name``'s
``.controller.stop_all_axes()``.
process), then sends a stop (``controller.stop_all_axes()``, i.e.
``XQ#STOP,1``) to the Galil controller of EVERY device listed in
``hard_stop_device_names`` -- one device per physical controller, in
that order (put the sample-transfer controller first) -- and finally
restores positioning mode on the FIRST listed controller via
``.controller.hard_abort_and_restore_positioning_mode()`` (the same
Galil hard-stop the CLI's ``ftransfer_abort()`` uses).
SIGINT is sent BEFORE the hard stop, not after: if a client-side loop
is actively polling this same transfer (e.g. ftransfer_get_sample's
@@ -41,41 +45,38 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
``ensure_gripper_up()``, which must not run mid-transfer. Sending
SIGINT first (an os.kill() call, effectively instant) gives the
target process's own ``except KeyboardInterrupt: ftransfer_abort();
raise FlomniError(...)`` handler a head start over the hard stop's
own multi-step sequence (which itself takes hundreds of ms due to
its internal polling/settling waits) -- without adding any
artificial delay of its own, so the actual motion stop is still
effectively immediate.
raise FlomniError(...)`` handler a head start over the hard stop.
``hard_stop_device_name`` names the device whose ``.controller`` the
hard stop is called on (e.g. ``"ftransy"`` for flomni). Without a device
*name* configured, the button is disabled rather than silently doing
nothing. A configured name that doesn't actually resolve to a device
(e.g. missing from the current session's config) is not checked ahead
of time -- it's re-resolved at click time in ``_on_abort()``, which
already logs and no-ops on failure -- so the button being enabled is
not a guarantee the device exists, only that a target was configured.
``hard_stop_device_names`` is a list of device names, one per Galil
controller (flomni: ``["ftransy", "foptx", "fsamroy"]`` -- the transfer
stages' controller, the optics controller and the rotation controller).
The legacy kwargs ``hard_stop_device_name`` / ``extra_hard_stop_device_name``
are still accepted and simply form the list ``[first, extra]``. With no
names configured, the button is disabled rather than silently doing
nothing. Names are NOT checked when the widget is built (that live check
proved unreliable, see git history), but they are resolved freshly from
the GUI client's device manager at click time, with a few retries, and
every outcome -- including "device could not be resolved" -- is logged
and shown in the widget's message label, so a controller that was not
stopped is never silent.
``extra_hard_stop_device_name`` optionally names a second device on a
physically separate Galil controller (e.g. ``"foptx"`` for flomni's
optics stage, a different board/socket port from the transfer stage
``ftransy`` sits on -- one abort press only halts axes on ``ftransy``'s
controller otherwise). Its ``.controller.stop_all_axes()`` is called
instead of ``hard_abort_and_restore_positioning_mode()``: the latter's
``#POSMODE``/``mntmod`` handling is specific to the sample-transfer/
mount program that only runs on the transfer controller, so
``stop_all_axes()`` (a plain ``XQ#STOP,1``) is the correct generic
"halt everything on this controller" primitive for any other board.
Entirely optional -- unset means only ``hard_stop_device_name`` is
stopped, as before.
The stops are emitted from one thread per controller, started in the
configured order, so a hung RPC to one controller cannot delay the
others. If the client-side proxy does not (yet) expose
``stop_all_axes`` (the device server publishes exposed methods when it
starts), the raw command ``XQ#STOP,1`` is sent through the always
exposed ``socket_put_and_receive`` instead.
"""
USER_ACCESS = ["message", "message.setter", "response", "clear_response"]
PLUGIN = True
def __init__(self, parent=None, **kwargs):
self._hard_stop_device_name = kwargs.pop("hard_stop_device_name", None)
self._extra_hard_stop_device_name = kwargs.pop("extra_hard_stop_device_name", None)
self._hard_stop_device_names = self._collect_hard_stop_names(
kwargs.pop("hard_stop_device_names", None),
kwargs.pop("hard_stop_device_name", None),
kwargs.pop("extra_hard_stop_device_name", None),
)
self._hard_stop_label = kwargs.pop("hard_stop_label", "Motion Stop")
super().__init__(parent=parent, **kwargs)
self._response = ""
@@ -87,22 +88,39 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
self._client_pid = os.getppid()
self._init_ui()
def _hard_stop_available(self) -> bool:
"""True if a hard_stop_device_name was configured for this widget.
# A fresh lookup at click time is retried a few times: a lookup can fail
# for a moment while the GUI client's device manager is being (re)filled.
_RESOLVE_ATTEMPTS = 3
_RESOLVE_DELAY_S = 0.05
# Upper bound for waiting on the per-controller stop threads.
_STOP_JOIN_TIMEOUT_S = 10.0
Deliberately does not check whether the name actually resolves to a
live, enabled device: that live check (a getattr(self.dev, name)
device-manager lookup) proved unreliable in practice -- a
confirmed-present, confirmed-enabled device could still evaluate as
unavailable here, permanently disabling the button for the widget's
lifetime with no way to recover short of recreating the dock. Since
whatever device is named is re-resolved with its own try/except in
_on_abort() anyway, a stale or wrong name still fails safely
(logged, no-op) instead of silently doing nothing -- so skipping the
redundant, unreliable pre-check here only removes a false-negative
failure mode, not real safety.
@staticmethod
def _collect_hard_stop_names(names, first=None, extra=None) -> list[str]:
"""Ordered, de-duplicated list of the devices whose controllers get a
stop. ``names`` (the ``hard_stop_device_names`` kwarg) wins; without
it the legacy pair ``[first, extra]`` is used."""
if isinstance(names, str):
names = [names]
candidates = list(names) if names else [first, extra]
result: list[str] = []
for name in candidates:
if name and name not in result:
result.append(name)
return result
def _hard_stop_available(self) -> bool:
"""True if at least one hard-stop device name was configured.
Deliberately does not check whether the names actually resolve to
live devices: that live check (a getattr(self.dev, name)
device-manager lookup at construction time) proved unreliable in
practice -- a confirmed-present, confirmed-enabled device could still
evaluate as unavailable, permanently disabling the button for the
widget's lifetime. Names are resolved at click time instead, and an
unresolvable name is reported loudly (log + message label).
"""
return bool(self._hard_stop_device_name)
return bool(self._hard_stop_device_names)
def _init_ui(self):
layout = QVBoxLayout(self)
@@ -170,6 +188,9 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
@SafeSlot()
def _on_abort(self):
if not self._hard_stop_device_names:
return # button is disabled in this case; defensive only
# 1) Coordinated abort first: a safe no-op if nothing is queued: when
# something IS queue-tracked, this registers a real stop_id so the
# device server suppresses the resulting error instead of killing
@@ -190,34 +211,113 @@ class z_ConsoleButtonsWidget(BECWidget, QWidget):
logger.warning(f"z_ConsoleButtonsWidget: sending SIGINT to client pid {self._client_pid}")
os.kill(self._client_pid, signal.SIGINT)
# 3) Hard motion stop(s) -- not delayed by anything above: sending
# SIGINT is a near-instant os.kill() call, so these still run
# effectively immediately. The two devices are independent
# controllers (see class docstring): each is attempted and
# logged on its own, so a failure on one doesn't skip the other.
if self._hard_stop_device_name:
self._stop_device_controller(
self._hard_stop_device_name, "hard_abort_and_restore_positioning_mode"
)
if self._extra_hard_stop_device_name:
self._stop_device_controller(self._extra_hard_stop_device_name, "stop_all_axes")
# 3) Stop every Galil controller, in the configured order (transfer
# controller first), then 4) restore positioning mode on the first
# one -- last, because it takes >= 0.5 s and must not delay the
# other stops. Nothing here raises: every failure is logged and
# shown in the message label.
stop_results = self._stop_all_controllers()
parts = [
f"{name}: OK" if ok else f"{name}: NOT STOPPED ({detail})"
for name, ok, detail in stop_results
]
text = "Motion stop sent -> " + " | ".join(parts)
if not all(ok for _, ok, _ in stop_results):
text += "\nCHECK THE STAGES MANUALLY!"
self.message_label.setText(text)
def _stop_device_controller(self, device_name: str, method_name: str) -> None:
"""Resolve device_name and call method_name() on its .controller,
logging and swallowing any failure -- so a missing device or a
failed stop on one controller can't prevent an already-issued stop
on another, or crash the rest of _on_abort()."""
restore_ok, restore_detail = self._restore_positioning_mode(self._hard_stop_device_names[0])
self.message_label.setText(
text + f"\nPositioning mode: {'restored' if restore_ok else 'NOT restored'}"
+ ("" if restore_ok else f" ({restore_detail})")
)
def _resolve_controller(self, device_name: str):
"""Fresh lookup of ``device_name`` in the GUI client's device manager
(not the ``self.dev`` reference cached at widget construction), with a
few retries. Returns the device's ``.controller`` RPC proxy or raises
RuntimeError with the reason."""
reason = "unknown"
for attempt in range(self._RESOLVE_ATTEMPTS):
try:
device = self.client.device_manager.devices.get(device_name)
if device is not None:
return device.controller
reason = "device not found in the GUI client's device manager"
except Exception as exc: # DeviceConfigError etc. are not AttributeErrors
reason = f"{type(exc).__name__}: {exc}"
if attempt < self._RESOLVE_ATTEMPTS - 1:
time.sleep(self._RESOLVE_DELAY_S)
raise RuntimeError(reason)
def _stop_controller(self, device_name: str) -> tuple[bool, str]:
"""Send a stop to the Galil controller behind ``device_name``.
Returns (sent, detail); never raises."""
try:
device = getattr(self.dev, device_name, None)
except Exception:
device = None
if device is None:
return
logger.warning(f"z_ConsoleButtonsWidget: hard-stopping {device_name} ({method_name})")
controller = self._resolve_controller(device_name)
except Exception as exc:
logger.error(f"z_ConsoleButtonsWidget: cannot resolve {device_name}: {exc}")
return False, f"not resolved: {exc}"
logger.warning(f"z_ConsoleButtonsWidget: stopping all axes on {device_name}'s controller")
try:
getattr(device.controller, method_name)()
except Exception:
logger.exception(f"z_ConsoleButtonsWidget: hard motion stop of {device_name} failed")
stop_all_axes = getattr(controller, "stop_all_axes", None)
if not callable(stop_all_axes):
raise AttributeError("stop_all_axes is not exposed by the device server yet")
reply = stop_all_axes()
logger.warning(f"z_ConsoleButtonsWidget: {device_name} stop_all_axes -> {reply!r}")
return True, str(reply)
except Exception as exc:
logger.warning(
f"z_ConsoleButtonsWidget: stop_all_axes on {device_name} failed ({exc!r}); "
"falling back to raw XQ#STOP,1"
)
try:
reply = controller.socket_put_and_receive("XQ#STOP,1")
logger.warning(f"z_ConsoleButtonsWidget: {device_name} raw XQ#STOP,1 -> {reply!r}")
return True, str(reply)
except Exception as exc:
logger.exception(f"z_ConsoleButtonsWidget: stopping {device_name} failed")
return False, f"stop failed: {exc}"
def _stop_all_controllers(self) -> list[tuple[str, bool, str]]:
"""Stop the controllers of all configured devices. One thread per
controller, started in the configured order, so a hung RPC to one
controller cannot delay the others. Returns (name, sent, detail) in
the configured order."""
results: dict[str, tuple[bool, str]] = {}
def worker(name: str) -> None:
results[name] = self._stop_controller(name)
threads = []
for name in self._hard_stop_device_names:
thread = threading.Thread(target=worker, args=(name,), daemon=True)
thread.start()
threads.append(thread)
deadline = time.monotonic() + self._STOP_JOIN_TIMEOUT_S
for thread in threads:
thread.join(max(0.0, deadline - time.monotonic()))
out = []
for name in self._hard_stop_device_names:
ok, detail = results.get(name, (False, f"no answer within {self._STOP_JOIN_TIMEOUT_S} s"))
out.append((name, ok, detail))
return out
def _restore_positioning_mode(self, device_name: str) -> tuple[bool, str]:
"""Wait for the transfer routine to halt and switch the controller
behind ``device_name`` back to positioning mode. Never raises."""
try:
controller = self._resolve_controller(device_name)
logger.warning(
f"z_ConsoleButtonsWidget: restoring positioning mode on {device_name}'s controller"
)
controller.hard_abort_and_restore_positioning_mode()
return True, ""
except Exception as exc:
logger.exception(
f"z_ConsoleButtonsWidget: restoring positioning mode on {device_name} failed"
)
return False, str(exc)
@SafeProperty(str)
def message(self):
@@ -6,6 +6,10 @@ from __future__ import annotations
designer_plugins = {
"DataViewer": ("csaxs_bec.bec_widgets.widgets.data_viewer.data_viewer", "DataViewer"),
"IDSCameraSettings": (
"csaxs_bec.bec_widgets.widgets.ids_camera_settings.ids_camera_settings",
"IDSCameraSettings",
),
"OMNY_SampleStorage": (
"csaxs_bec.bec_widgets.widgets.sample_storage.sample_storage",
"OMNY_SampleStorage",
@@ -22,6 +26,7 @@ designer_plugins = {
widget_icons = {
"DataViewer": "find_in_page",
"IDSCameraSettings": "photo_camera",
"OMNY_SampleStorage": "widgets",
"OMNY_TomoParams": "widgets",
"OMNY_XRayEye": "widgets",
@@ -0,0 +1,3 @@
from csaxs_bec.bec_widgets.widgets.ids_camera_settings.ids_camera_settings import IDSCameraSettings
__all__ = ["IDSCameraSettings"]
@@ -0,0 +1,511 @@
"""IDS camera settings widget for cSAXS -- pick a configured IDS camera from
a dropdown and adjust its exposure time / pixel clock.
The exposure/pixel-clock control logic mirrors OMNY_XRayEye
(bec_widgets/widgets/xray_eye/x_ray_eye.py), which implements the same
controls but hardcoded to a single camera (module-level CAMERA constant).
This widget generalizes that to whichever camera is selected, for beamlines
that configure more than one IDS camera at once (e.g. ptycho_omny.yaml's
cam200..cam203).
"""
from __future__ import annotations
import math
from bec_lib import bec_logger
from bec_lib.endpoints import MessageEndpoints
from bec_widgets import BECWidget, SafeProperty, SafeSlot
from bec_widgets.widgets.control.device_input.device_combobox.device_combobox import (
DeviceComboBox,
)
from bec_widgets.widgets.utility.toggle.toggle import ToggleSwitch
from qtpy.QtCore import Qt, QTimer
from qtpy.QtWidgets import QFrame, QGridLayout, QLabel, QSizePolicy, QSlider, QVBoxLayout, QWidget
logger = bec_logger.logger
class IDSCameraComboBox(DeviceComboBox):
"""Dropdown listing configured IDS cameras.
DeviceComboBox's device_filter only distinguishes bec_lib.device base
classes (Device/Positioner/Signal/ComputedSignal) and can't express
"exactly IDSCamera", so this overrides update_devices_from_filters() to
filter by each device's deviceClass config instead -- the same technique
device_table.py uses to filter/display device classes client-side.
To support another camera class later (once it grows the same
Kind.config exposure/pixel-clock signals this widget depends on), add
its bare class name to CAMERA_DEVICE_CLASSES -- no other change should
be required here.
"""
PLUGIN = False
RPC = False
CAMERA_DEVICE_CLASSES: tuple[str, ...] = ("IDSCamera",)
@staticmethod
def _device_class_name(device) -> str:
"""Bare class name from a device's deviceClass config, e.g.
"csaxs_bec.devices.ids_cameras.ids_camera.IDSCamera" -> "IDSCamera"."""
device_class = (getattr(device, "_config", None) or {}).get("deviceClass") or ""
return device_class.rsplit(".", 1)[-1] if device_class else ""
@SafeSlot()
def update_devices_from_filters(self):
if not self.apply_filter:
return
self.devices = sorted(
device.name
for device in self.dev.enabled_devices
if self._device_class_name(device) in self.CAMERA_DEVICE_CLASSES
)
class IDSCameraSettings(BECWidget, QWidget):
"""Pick a configured IDS camera and adjust its exposure time / pixel clock."""
ICON_NAME = "photo_camera"
PLUGIN = True
USER_ACCESS = ["selected_camera", "set_camera"]
# exposure_time_slider works in tenths of a ms internally (QSlider is
# int-only); divide by this to get ms. Same idiom as OMNY_XRayEye.
_EXPOSURE_SLIDER_SCALE = 10
def __init__(self, parent=None, **kwargs):
super().__init__(parent=parent, **kwargs)
self.get_bec_shortcuts()
self._camera_name: str | None = None
self._auto_exposure_enabled = True
# pixel_clock_slider is index-based over this list (re-fetched once
# per camera, see _init_pixel_clock_options()) rather than a raw MHz
# range -- many uEye sensors only accept a short discrete list of
# pixel clocks, not every value in [min, max] (see OMNY_XRayEye).
self._pixel_clock_options: list[int] = []
self._last_pixel_clock_mhz: int | None = None
self._queue_busy = False
self._queue_idle_timer = QTimer(self)
self._queue_idle_timer.setSingleShot(True)
self._queue_idle_timer.setInterval(800)
self._queue_idle_timer.timeout.connect(self._release_queue_busy)
self._init_ui()
self._make_connections()
self._reset_controls_to_placeholder()
self.bec_dispatcher.connect_slot(
self.on_queue_status_update, MessageEndpoints.scan_queue_status()
)
QTimer.singleShot(0, self._init_queue_status)
def _init_ui(self):
layout = QVBoxLayout(self)
camera_row = QGridLayout()
camera_row.setColumnStretch(1, 1)
self.camera_label = QLabel("Camera", parent=self)
self.camera_combo = IDSCameraComboBox(parent=self)
self.camera_combo.set_first_element_as_empty = True
camera_row.addWidget(self.camera_label, 0, 0)
camera_row.addWidget(self.camera_combo, 0, 1)
layout.addLayout(camera_row)
self.status_label = QLabel(parent=self)
layout.addWidget(self.status_label)
layout.addWidget(self._create_separator())
_right_vcenter = Qt.AlignmentFlag.AlignRight | Qt.AlignmentFlag.AlignVCenter
self.exposure_grid_widget = QWidget(parent=self)
exposure_grid = QGridLayout(self.exposure_grid_widget)
exposure_grid.setHorizontalSpacing(8)
exposure_grid.setVerticalSpacing(4)
exposure_grid.setColumnStretch(0, 1)
# No "Auto gain" control here -- see OMNY_XRayEye, which dropped it
# after HW testing found continuous auto-gain isn't useful once
# correctly exposed. IDSCamera pulses it on then off internally,
# once, at connect; there is no persistent UI for it.
self.auto_exposure_label = QLabel("Auto exposure", parent=self)
self.auto_exposure_toggle = ToggleSwitch(parent=self)
self.auto_exposure_toggle.checked = True
self.auto_exposure_toggle.enabled.connect(self.auto_exposure_enabled_changed)
exposure_grid.addWidget(self.auto_exposure_label, 0, 1, _right_vcenter)
exposure_grid.addWidget(self.auto_exposure_toggle, 0, 2, Qt.AlignmentFlag.AlignVCenter)
# QSlider only takes ints, so exposure time is tracked in tenths of a
# ms internally (_EXPOSURE_SLIDER_SCALE) for one decimal of
# resolution on the display label.
self.exposure_time_label = QLabel("Exposure time", parent=self)
self.exposure_time_slider = QSlider(Qt.Orientation.Horizontal, parent=self)
self.exposure_time_slider.setRange(1, 10000) # placeholder; reseeded per camera
self.exposure_time_value_label = QLabel("-- ms", parent=self)
self.exposure_time_value_label.setMinimumWidth(60)
self.exposure_time_slider.valueChanged.connect(self._update_exposure_time_value_label)
# sliderReleased (fires once, on mouse-up), not valueChanged (fires on
# every tick while dragging) -- don't hammer the device.
self.exposure_time_slider.sliderReleased.connect(self.exposure_time_submitted)
exposure_grid.addWidget(self.exposure_time_label, 1, 1, _right_vcenter)
exposure_grid.addWidget(self.exposure_time_slider, 1, 2, 1, 2)
exposure_grid.addWidget(self.exposure_time_value_label, 1, 4, Qt.AlignmentFlag.AlignVCenter)
# Pixel clock (MHz): lowering it raises the max achievable exposure
# time, at the cost of frame rate. Many uEye sensors only accept a
# short discrete list of pixel clocks, so this slider's *position* is
# an index into self._pixel_clock_options, not a raw MHz value.
self.pixel_clock_label = QLabel("Pixel clock", parent=self)
self.pixel_clock_slider = QSlider(Qt.Orientation.Horizontal, parent=self)
self.pixel_clock_slider.setRange(0, 0) # placeholder; see _set_pixel_clock_options()
self.pixel_clock_value_label = QLabel("-- MHz", parent=self)
self.pixel_clock_value_label.setMinimumWidth(60)
self.pixel_clock_slider.valueChanged.connect(self._update_pixel_clock_value_label)
self.pixel_clock_slider.sliderReleased.connect(self.pixel_clock_submitted)
exposure_grid.addWidget(self.pixel_clock_label, 2, 1, _right_vcenter)
exposure_grid.addWidget(self.pixel_clock_slider, 2, 2, 1, 2)
exposure_grid.addWidget(self.pixel_clock_value_label, 2, 4, Qt.AlignmentFlag.AlignVCenter)
layout.addWidget(self.exposure_grid_widget)
layout.addStretch()
self.setSizePolicy(QSizePolicy.Policy.Preferred, QSizePolicy.Policy.Fixed)
def _make_connections(self):
self.camera_combo.device_selected.connect(self._on_camera_selected)
self.camera_combo.device_reset.connect(self._on_camera_reset)
def _create_separator(self):
sep = QFrame(parent=self)
sep.setFrameShape(QFrame.Shape.HLine)
sep.setFrameShadow(QFrame.Shadow.Sunken)
sep.setLineWidth(1)
return sep
################################################################################
# Camera selection
################################################################################
@SafeSlot(str)
def _on_camera_selected(self, camera_name: str):
if not camera_name or camera_name == self._camera_name:
return
self._switch_camera(camera_name)
@SafeSlot()
def _on_camera_reset(self):
if self._camera_name is not None:
self._switch_camera(None)
@SafeProperty(str)
def selected_camera(self) -> str:
"""Name of the currently selected IDS camera, or an empty string if none."""
return self._camera_name or ""
@SafeSlot(str)
def set_camera(self, name: str):
"""Select a camera by device name, as if chosen from the dropdown."""
self.camera_combo.set_device(name)
def _switch_camera(self, camera_name: str | None):
if self._camera_name is not None:
self.bec_dispatcher.disconnect_slot(
self.getting_camera_status,
MessageEndpoints.device_read_configuration(self._camera_name),
)
self._camera_name = camera_name
self._reset_controls_to_placeholder()
if camera_name is None:
return
self.bec_dispatcher.connect_slot(
self.getting_camera_status, MessageEndpoints.device_read_configuration(camera_name)
)
# device_read_configuration is retained in redis (set_and_publish), so
# seed immediately from the last known value instead of waiting for
# the camera to spontaneously republish -- connect_slot alone only
# delivers messages published *after* it connects.
try:
msg = self.client.connector.get(
MessageEndpoints.device_read_configuration(camera_name)
)
except Exception as exc:
logger.warning(f"Failed to fetch initial config for {camera_name}: {exc}")
msg = None
if msg is not None:
self.getting_camera_status(msg.content, msg.metadata)
self._pixel_clock_options = []
self._last_pixel_clock_mhz = None
QTimer.singleShot(0, self._init_pixel_clock_options)
def _reset_controls_to_placeholder(self):
self._auto_exposure_enabled = True
self.auto_exposure_toggle.blockSignals(True)
self.auto_exposure_toggle.checked = True
self.auto_exposure_toggle.blockSignals(False)
self.exposure_time_value_label.setText("-- ms")
self._pixel_clock_options = []
self._last_pixel_clock_mhz = None
self.pixel_clock_slider.blockSignals(True)
self.pixel_clock_slider.setRange(0, 0)
self.pixel_clock_slider.blockSignals(False)
self.pixel_clock_value_label.setText("-- MHz")
if self._camera_name is None:
if self.camera_combo.devices:
self.status_label.setText("No camera selected")
else:
self.status_label.setText("No IDS cameras configured")
else:
self.status_label.setText(f"Camera: {self._camera_name}")
self._refresh_control_availability()
def _refresh_control_availability(self):
have_camera = self._camera_name is not None
enabled = have_camera and not self._queue_busy
tooltip = "Disabled while scan queue is busy." if have_camera and self._queue_busy else ""
self.auto_exposure_toggle.setEnabled(enabled)
self.auto_exposure_toggle.setToolTip(tooltip)
self.exposure_time_slider.setEnabled(enabled and not self._auto_exposure_enabled)
self.exposure_time_slider.setToolTip(tooltip)
self.pixel_clock_slider.setEnabled(enabled and bool(self._pixel_clock_options))
self.pixel_clock_slider.setToolTip(tooltip)
################################################################################
# Exposure / pixel clock
################################################################################
@SafeSlot(dict, dict)
def getting_camera_status(self, data, meta):
if self._camera_name is None:
return
signals = data.get("signals") or {}
auto_exp = signals.get(f"{self._camera_name}_auto_exposure_enabled")
if auto_exp is not None:
enabled = bool(auto_exp.get("value"))
self.auto_exposure_toggle.blockSignals(True)
self.auto_exposure_toggle.checked = enabled
self.auto_exposure_toggle.blockSignals(False)
self._auto_exposure_enabled = enabled
self._refresh_control_availability()
# No auto_gain_enabled handling here -- see the comment above the
# (omitted) toggle in _init_ui.
# Update the pixel-clock slider's position before the exposure ones
# below -- changing the pixel clock changes the exposure range, and
# both can arrive together in this same message.
pixel_clock = signals.get(f"{self._camera_name}_pixel_clock")
if pixel_clock is not None:
self._set_pixel_clock_display(int(pixel_clock.get("value")))
# Reseed the exposure slider's bounds next, so a value arriving in
# the same message is clamped against up-to-date min/max. Round the
# min UP and the max DOWN (never the other way) -- ceil (clamped to
# at least 1) keeps the bound physical instead of claiming 0 ms
# exposure is settable when it isn't.
exposure_min = signals.get(f"{self._camera_name}_exposure_time_min")
exposure_max = signals.get(f"{self._camera_name}_exposure_time_max")
if exposure_min is not None and exposure_max is not None:
lo = max(1, math.ceil(float(exposure_min.get("value")) * self._EXPOSURE_SLIDER_SCALE))
hi = math.floor(float(exposure_max.get("value")) * self._EXPOSURE_SLIDER_SCALE)
if (lo, hi) != (self.exposure_time_slider.minimum(), self.exposure_time_slider.maximum()):
self.exposure_time_slider.setRange(lo, hi)
exposure_time = signals.get(f"{self._camera_name}_exposure_time")
if exposure_time is not None:
self.exposure_time_slider.blockSignals(True)
self.exposure_time_slider.setValue(
round(float(exposure_time.get("value")) * self._EXPOSURE_SLIDER_SCALE)
)
self.exposure_time_slider.blockSignals(False)
self._update_exposure_time_value_label(self.exposure_time_slider.value())
@SafeSlot(bool)
def auto_exposure_enabled_changed(self, enabled: bool):
if self._camera_name is None or self._queue_busy:
logger.warning("Ignoring auto-exposure toggle: no camera selected or queue busy.")
return
self.auto_exposure_toggle.blockSignals(True)
self.dev.get(self._camera_name).auto_exposure_enabled.put(enabled)
self.auto_exposure_toggle.checked = enabled
self.auto_exposure_toggle.blockSignals(False)
self._auto_exposure_enabled = enabled
self._refresh_control_availability()
def _update_exposure_time_value_label(self, raw_value: int):
self.exposure_time_value_label.setText(
f"{raw_value / self._EXPOSURE_SLIDER_SCALE:.1f} ms"
)
def exposure_time_submitted(self):
if self._camera_name is None or self._queue_busy:
return
value_ms = self.exposure_time_slider.value() / self._EXPOSURE_SLIDER_SCALE
self.dev.get(self._camera_name).exposure_time.put(value_ms)
def _init_pixel_clock_options(self):
"""One-time-per-camera fetch of the actually-supported pixel clocks
(see IDSCamera.get_pixel_clock_list()'s docstring for why this can't
just be [min, max] from a Kind.config signal) -- not a polling loop,
this list is static per camera, so a single RPC call per camera
selection is the right cost/benefit trade-off."""
camera_name = self._camera_name
if camera_name is None:
return
try:
options = self.dev.get(camera_name).get_pixel_clock_list()
except Exception as exc:
logger.warning(f"Failed to fetch pixel clock options for {camera_name}: {exc}")
return
if camera_name != self._camera_name:
return # camera changed again while this RPC call was in flight
self._set_pixel_clock_options(sorted(int(v) for v in options))
def _set_pixel_clock_options(self, options: list[int]):
self._pixel_clock_options = options
if not options:
self._refresh_control_availability()
return
self.pixel_clock_slider.blockSignals(True)
self.pixel_clock_slider.setRange(0, len(options) - 1)
self.pixel_clock_slider.blockSignals(False)
if self._last_pixel_clock_mhz is not None:
# A status message already arrived before this RPC call
# returned -- apply it now that we can actually resolve it to a
# slider position.
self._set_pixel_clock_display(self._last_pixel_clock_mhz)
else:
self._update_pixel_clock_value_label(self.pixel_clock_slider.value())
self._refresh_control_availability()
def _set_pixel_clock_display(self, mhz: int):
"""Move the slider to the option nearest `mhz` (from hardware, via
getting_camera_status()) and update the value label -- never a raw
MHz value on the slider itself, only a valid index."""
self._last_pixel_clock_mhz = mhz
if not self._pixel_clock_options:
# Options not fetched yet -- _set_pixel_clock_options() will
# call back into this once they arrive.
self.pixel_clock_value_label.setText(f"{mhz} MHz")
return
index = min(
range(len(self._pixel_clock_options)),
key=lambda i: abs(self._pixel_clock_options[i] - mhz),
)
self.pixel_clock_slider.blockSignals(True)
self.pixel_clock_slider.setValue(index)
self.pixel_clock_slider.blockSignals(False)
self._update_pixel_clock_value_label(index)
def _update_pixel_clock_value_label(self, index: int):
if 0 <= index < len(self._pixel_clock_options):
self.pixel_clock_value_label.setText(f"{self._pixel_clock_options[index]} MHz")
else:
self.pixel_clock_value_label.setText("-- MHz")
def pixel_clock_submitted(self):
if self._camera_name is None or self._queue_busy:
return
index = self.pixel_clock_slider.value()
if not 0 <= index < len(self._pixel_clock_options):
return
self.dev.get(self._camera_name).pixel_clock.put(self._pixel_clock_options[index])
################################################################################
# Scan-queue guard
################################################################################
def _update_queue_busy_state(self, busy: bool):
if busy:
self._queue_idle_timer.stop()
self._set_queue_busy(True)
return
if self._queue_busy and not self._queue_idle_timer.isActive():
self._queue_idle_timer.start()
def _set_queue_busy(self, busy: bool):
if busy == self._queue_busy:
return
self._queue_busy = busy
self._refresh_control_availability()
def _release_queue_busy(self):
self._set_queue_busy(False)
def _init_queue_status(self):
try:
msg = self.client.connector.get(MessageEndpoints.scan_queue_status())
except Exception as exc:
logger.warning(f"Failed to fetch initial scan queue status for IDSCameraSettings: {exc}")
return
if msg is None:
return
self._update_queue_busy_state(self._is_queue_busy(msg.content))
@staticmethod
def _is_queue_busy(msg_content: dict) -> bool:
queues = msg_content.get("queue", {}) if isinstance(msg_content, dict) else {}
primary_queue = queues.get("primary") if isinstance(queues, dict) else None
if primary_queue is None:
return False
queue_info = getattr(primary_queue, "info", None)
if queue_info is None and isinstance(primary_queue, dict):
queue_info = primary_queue.get("info", [])
if not queue_info:
return False
idle_statuses = {"STOPPED", "COMPLETED", "IDLE"}
for item in queue_info:
status = getattr(item, "status", None)
if status is None and isinstance(item, dict):
status = item.get("status")
if str(status).upper() not in idle_statuses:
return True
return False
@SafeSlot(dict, dict)
def on_queue_status_update(self, data, meta):
_ = meta
self._update_queue_busy_state(self._is_queue_busy(data))
def cleanup(self):
self._queue_idle_timer.stop()
if self._camera_name is not None:
self.bec_dispatcher.disconnect_slot(
self.getting_camera_status,
MessageEndpoints.device_read_configuration(self._camera_name),
)
self.bec_dispatcher.disconnect_slot(
self.on_queue_status_update, MessageEndpoints.scan_queue_status()
)
super().cleanup()
if __name__ == "__main__": # pragma: no cover
import sys
from bec_widgets.utils import BECDispatcher
from bec_widgets.utils.colors import apply_theme
from qtpy.QtWidgets import QApplication
app = QApplication(sys.argv)
apply_theme("light")
dispatcher = BECDispatcher(gui_id="ids_camera_settings")
win = IDSCameraSettings()
win.resize(400, 250)
win.show()
sys.exit(app.exec_())
@@ -0,0 +1 @@
{'files': ['ids_camera_settings.py']}
@@ -0,0 +1,57 @@
# Copyright (C) 2022 The Qt Company Ltd.
# SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause
from qtpy.QtDesigner import QDesignerCustomWidgetInterface
from qtpy.QtWidgets import QWidget
from bec_widgets.utils.bec_designer import designer_material_icon
from csaxs_bec.bec_widgets.widgets.ids_camera_settings.ids_camera_settings import IDSCameraSettings
DOM_XML = """
<ui language='c++'>
<widget class='IDSCameraSettings' name='ids_camera_settings'>
</widget>
</ui>
"""
class IDSCameraSettingsPlugin(QDesignerCustomWidgetInterface): # pragma: no cover
def __init__(self):
super().__init__()
self._form_editor = None
def createWidget(self, parent):
if parent is None:
return QWidget()
t = IDSCameraSettings(parent)
return t
def domXml(self):
return DOM_XML
def group(self):
return ""
def icon(self):
return designer_material_icon(IDSCameraSettings.ICON_NAME)
def includeFile(self):
return "ids_camera_settings"
def initialize(self, form_editor):
self._form_editor = form_editor
def isContainer(self):
return False
def isInitialized(self):
return self._form_editor is not None
def name(self):
return "IDSCameraSettings"
def toolTip(self):
return "Pick a configured IDS camera and adjust its exposure time / pixel clock."
def whatsThis(self):
return self.toolTip()
@@ -0,0 +1,15 @@
def main(): # pragma: no cover
from qtpy import PYSIDE6
if not PYSIDE6:
print("PYSIDE6 is not available in the environment. Cannot patch designer.")
return
from PySide6.QtDesigner import QPyDesignerCustomWidgetCollection
from csaxs_bec.bec_widgets.widgets.ids_camera_settings.ids_camera_settings_plugin import IDSCameraSettingsPlugin
QPyDesignerCustomWidgetCollection.addCustomWidget(IDSCameraSettingsPlugin())
if __name__ == "__main__": # pragma: no cover
main()
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -1,5 +1,7 @@
from __future__ import annotations
import math
import pyqtgraph as pg
from bec_lib import bec_logger
from bec_lib.endpoints import MessageEndpoints
@@ -9,17 +11,24 @@ from bec_widgets.utils.rpc_decorator import rpc_timeout
from bec_widgets.widgets.plots.image.image import Image
from bec_widgets.widgets.plots.waveform.waveform import Waveform
from bec_widgets.widgets.plots.image.setting_widgets.image_roi_tree import ROIPropertyTree
from bec_widgets.widgets.plots.roi.image_roi import BaseROI, CircularROI, RectangularROI
from bec_widgets.widgets.plots.roi.image_roi import (
BaseROI,
CircularROI,
EllipticalROI,
RectangularROI,
)
from bec_widgets.widgets.utility.toggle.toggle import ToggleSwitch
from qtpy.QtCore import Qt, QTimer
from qtpy.QtWidgets import (
QFrame,
QGridLayout,
QHBoxLayout,
QDoubleSpinBox,
QLabel,
QLineEdit,
QPushButton,
QSizePolicy,
QSlider,
QSpinBox,
QToolButton,
QVBoxLayout,
@@ -31,6 +40,53 @@ from qtpy.QtWidgets import (
logger = bec_logger.logger
CAMERA = ("cam_xeye", "image")
# Start values / bounds for the DAP fit A*sin(f*x + shift) + intercept of the
# alignment offsets vs. angle (5 points, 0..180 deg, f fixed to 1 deg in rad).
# All five parameters are given so the DAP server does not use lmfit's
# SineModel.guess(): its FFT-based shift guess is arbitrary for 5 points and
# regularly left the fit stuck at the shift bound (2*pi) far from the data.
# A signed amplitude (SineModel defaults to A >= 0) plus a fixed start
# (A=30, shift=0, intercept=0) converged to the global least-squares optimum
# in 1500/1500 random test datasets; A >= 0 failed in a few percent.
SINE_FIT_DAP_PARAMETERS = [
{
"amplitude": {"value": 30, "vary": True, "min": -1e6, "max": 1e6},
"frequency": {"value": 0.0174533, "vary": False, "min": 0.01, "max": 0.02},
"shift": {"value": 0, "vary": True, "min": -math.pi, "max": math.pi},
},
{
"slope": {"value": 0, "vary": False, "min": 0.0, "max": 0.02},
"intercept": {"value": 0, "vary": True, "min": -1e6, "max": 1e6},
},
]
def ellipse_geometry(roi) -> dict:
"""Centre, axes and rotation of an (optionally rotated) ``EllipticalROI``.
``EllipticalROI.get_coordinates()`` ignores the rotation (the ROI rotates about
its ``pos()``, not its centre) and sorts the axes by size. Here the centre is
mapped through the ROI transform, ``width``/``height`` are the axes along the
ROI's local x/y, and ``angle`` (degrees, pyqtgraph convention, normalised to
(-90, 90]; an ellipse is symmetric under 180 deg) is the rotation of the local
x axis in data coordinates.
"""
size_x, size_y = roi.state["size"]
centre = roi.mapToParent(pg.Point(size_x / 2, size_y / 2))
return {
"center_x": centre.x(),
"center_y": centre.y(),
"width": abs(size_x),
"height": abs(size_y),
"angle": _normalize_angle(roi.angle()),
}
def _normalize_angle(angle: float) -> float:
"""Map an angle in degrees onto (-90, 90]."""
angle = (angle + 90.0) % 180.0 - 90.0
return 90.0 if math.isclose(angle, -90.0, abs_tol=1e-9) else angle
class TargetCrosshair:
"""
@@ -239,6 +295,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
"on_live_view_enabled",
"on_motors_enable",
"enable_submit_button",
"enable_skip_button",
"sample_name",
"sample_name.setter",
"enable_move_buttons",
@@ -248,6 +305,12 @@ class OMNY_XRayEye(BECWidget, QWidget):
"submit_fit_array",
"show_crosshair",
"hide_crosshair",
"lock_vertical_center",
"unlock_vertical_center",
"enable_ellipse_roi",
"lock_roi_angle",
"unlock_roi_angle",
"roi_angle",
"crosshair_visible",
"crosshair_visible.setter",
"set_crosshair_position",
@@ -269,10 +332,20 @@ class OMNY_XRayEye(BECWidget, QWidget):
PIXEL_CALIBRATION_USER_PARAM = "pixel_calibration"
PIXEL_CALIBRATION_DEFAULT = 1.0
# exposure_time_slider works in tenths of a ms internally (QSlider is
# int-only); divide by this to get ms.
_EXPOSURE_SLIDER_SCALE = 10
def __init__(self, parent=None, **kwargs):
super().__init__(parent=parent, **kwargs)
self._live_view_signal = CAMERA[1]
self._last_smear_composite = None
# pixel_clock_slider is index-based over this list (populated once,
# see _init_pixel_clock_options()) rather than a raw MHz range --
# many uEye sensors only accept a short discrete list of pixel
# clocks, not every value in [min, max] (confirmed on hardware).
self._pixel_clock_options: list[int] = []
self._last_pixel_clock_mhz: int | None = None
self._connected_motor = None
self._dap_params_forwarding_connected = False
self._queue_busy = False
@@ -284,7 +357,14 @@ class OMNY_XRayEye(BECWidget, QWidget):
self._init_ui()
self.target_crosshair = TargetCrosshair(self.image.plot_item)
# See lock_vertical_center()/unlock_vertical_center()/_snap_roi_vertical()
self._vertical_lock_enabled = False
# Elliptical ROI (LamNI centre calibration only, see enable_ellipse_roi());
# _roi_angle_locked is the fixed rotation in degrees, or None if free.
self._ellipse_enabled = False
self._roi_angle_locked = None
self._make_connections()
self.enable_ellipse_roi(False)
# Connection to redis endpoints
self.bec_dispatcher.connect_slot(
@@ -312,9 +392,23 @@ class OMNY_XRayEye(BECWidget, QWidget):
)
self.connect_motors()
self.resize(800, 600)
# No self.resize() here: this widget is normally embedded as a dock
# (see gui_tools.py's flomnigui_show_xeyealign()), where the dock
# manager (Qt Advanced Docking System) owns its geometry -- an
# explicit self-resize call fights that, and on a screen too small
# for the assumed geometry (gui_tools.py's hardcoded 2560px-wide
# assumption) corrupts the dock's layout until the operator manually
# moves/resizes the window (forcing Qt to reflow within the actual
# available space) -- confirmed live: reproduces reliably on a
# smaller screen, not on a large one, and only for this widget (the
# only one in this GUI with a self-resize call), never for the
# camera/console docks shown via flomnigui_show_cameras(). The
# standalone `python x_ray_eye.py` harness below already does its
# own win.resize(1000, 800) after construction regardless, so this
# call was never actually needed even for that use case.
QTimer.singleShot(0, self._init_queue_status)
QTimer.singleShot(0, self._init_gui_trigger)
QTimer.singleShot(0, self._init_pixel_clock_options)
def _init_ui(self):
self.root_layout = QVBoxLayout(self)
@@ -359,12 +453,13 @@ class OMNY_XRayEye(BECWidget, QWidget):
header_row.addWidget(self.live_preview_toggle, 0, Qt.AlignmentFlag.AlignVCenter)
self.control_panel_layout.addLayout(header_row)
# Shutter/camera-running (row 0) and the smear aid's own status
# switches (row 1) share one grid so the toggle columns actually
# line up regardless of each row's label text width -- two
# independent QHBoxLayouts (the previous approach) each size
# themselves from their own labels' widths, so corresponding
# switches in different rows land at different x-positions.
# Shutter/camera-running (row 0), a separator (row 1), the smear
# aid's own status switches (row 2), and the ROI-vertical-lock
# toggle (row 3) share one grid so the toggle columns actually line
# up regardless of each row's label text width -- independent
# QHBoxLayouts per row (the previous approach) each size themselves
# from their own labels' widths, so corresponding switches in
# different rows would land at different x-positions.
self.switch_grid_widget = QWidget(parent=self)
switch_grid = QGridLayout(self.switch_grid_widget)
switch_grid.setContentsMargins(0, 0, 0, 0)
@@ -392,18 +487,131 @@ class OMNY_XRayEye(BECWidget, QWidget):
self.smear_active_toggle.checked = False
self.smear_active_toggle.setEnabled(False) # read-only status, not operator-togglable
# Manual override for the ROI-vertical-lock feature (see
# lock_vertical_center()/unlock_vertical_center()): a script (e.g.
# x_ray_eye_align.py) engages the lock automatically during the
# angle-alignment steps, but an operator may still want to freely
# draw/measure an unrelated box mid-run -- this toggle lets them
# release it without aborting the run, and re-engage it after.
self.vertical_lock_label = QLabel("ROI vertical lock", parent=self)
self.vertical_lock_toggle = ToggleSwitch(parent=self)
self.vertical_lock_toggle.checked = False
self.vertical_lock_toggle.enabled.connect(self._on_vertical_lock_toggle_changed)
# Angle fix for the elliptical ROI (see lock_roi_angle()); only visible
# while the ellipse tool is enabled (enable_ellipse_roi()).
self.roi_angle_lock_label = QLabel("ROI angle lock", parent=self)
self.roi_angle_lock_toggle = ToggleSwitch(parent=self)
self.roi_angle_lock_toggle.checked = False
self.roi_angle_lock_toggle.enabled.connect(self._on_roi_angle_lock_toggle_changed)
self.roi_angle_spinbox = QDoubleSpinBox(parent=self)
self.roi_angle_spinbox.setRange(-90.0, 90.0)
self.roi_angle_spinbox.setDecimals(1)
self.roi_angle_spinbox.setSingleStep(1.0)
self.roi_angle_spinbox.setSuffix(" deg")
self.roi_angle_spinbox.setToolTip("Rotation of the elliptical ROI")
self.roi_angle_spinbox.valueChanged.connect(self._on_roi_angle_spinbox_changed)
_right_vcenter = Qt.AlignmentFlag.AlignRight | Qt.AlignmentFlag.AlignVCenter
switch_grid.addWidget(self.shutter_label, 0, 1, _right_vcenter)
switch_grid.addWidget(self.shutter_toggle, 0, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.camera_running_label, 0, 3, _right_vcenter)
switch_grid.addWidget(self.camera_running_toggle, 0, 4, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.smear_active_label, 1, 1, _right_vcenter)
switch_grid.addWidget(self.smear_active_toggle, 1, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.smear_preview_label, 1, 3, _right_vcenter)
switch_grid.addWidget(self.smear_preview_toggle, 1, 4, Qt.AlignmentFlag.AlignVCenter)
# separator: below shutter/camera-running, above the smear-integration switches
switch_grid.addWidget(self._create_separator(), 1, 0, 1, 5)
switch_grid.addWidget(self.smear_active_label, 2, 1, _right_vcenter)
switch_grid.addWidget(self.smear_active_toggle, 2, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.smear_preview_label, 2, 3, _right_vcenter)
switch_grid.addWidget(self.smear_preview_toggle, 2, 4, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.vertical_lock_label, 3, 1, _right_vcenter)
switch_grid.addWidget(self.vertical_lock_toggle, 3, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.roi_angle_lock_label, 4, 1, _right_vcenter)
switch_grid.addWidget(self.roi_angle_lock_toggle, 4, 2, Qt.AlignmentFlag.AlignVCenter)
switch_grid.addWidget(self.roi_angle_spinbox, 4, 3, 1, 2)
self.control_panel_layout.addWidget(self.switch_grid_widget)
# separator
# separator: below shutter/camera-running/smear switches, above the
# exposure/gain section
self.control_panel_layout.addWidget(self._create_separator())
# Exposure/gain section: auto-exposure and auto-gain enable toggles
# (row 0, same layout idiom as switch_grid above), plus a manual
# exposure-time slider (row 1) that's only enabled while
# auto-exposure is off. Its range is a placeholder until seeded from
# the hardware-reported (min, max) at connect time -- see
# getting_camera_status().
self.exposure_grid_widget = QWidget(parent=self)
exposure_grid = QGridLayout(self.exposure_grid_widget)
exposure_grid.setContentsMargins(0, 0, 0, 0)
exposure_grid.setHorizontalSpacing(8)
exposure_grid.setVerticalSpacing(4)
exposure_grid.setColumnStretch(0, 1)
# No "Auto gain" control here -- HW testing found continuous
# auto-gain isn't useful once correctly exposed, and takes gain out
# of manual control. IDSCamera now pulses it on then off internally,
# once, at connect (see its on_connected()); there is no persistent
# UI for it.
self.auto_exposure_label = QLabel("Auto exposure", parent=self)
self.auto_exposure_toggle = ToggleSwitch(parent=self)
self.auto_exposure_toggle.checked = True
self.auto_exposure_toggle.enabled.connect(self.auto_exposure_enabled_changed)
exposure_grid.addWidget(self.auto_exposure_label, 0, 1, _right_vcenter)
exposure_grid.addWidget(self.auto_exposure_toggle, 0, 2, Qt.AlignmentFlag.AlignVCenter)
# QSlider only takes ints, so exposure time is tracked in tenths of a
# ms internally (_EXPOSURE_SLIDER_SCALE) for one decimal of
# resolution on the display label.
self.exposure_time_label = QLabel("Exposure time", parent=self)
self.exposure_time_slider = QSlider(Qt.Orientation.Horizontal, parent=self)
# Placeholder until reseeded from hardware on connect (see
# getting_camera_status()) -- minimum is 1 (0.1 ms), never 0, since
# 0 ms exposure isn't physical.
self.exposure_time_slider.setRange(1, 10000)
self.exposure_time_slider.setEnabled(False) # auto-exposure starts enabled
self.exposure_time_value_label = QLabel("-- ms", parent=self)
self.exposure_time_value_label.setMinimumWidth(60)
self.exposure_time_slider.valueChanged.connect(self._update_exposure_time_value_label)
# sliderReleased (fires once, on mouse-up), not valueChanged (fires on
# every tick while dragging) -- the throttling choice here, same
# "don't hammer the device" concern as editingFinished was for the
# spinbox this replaces.
self.exposure_time_slider.sliderReleased.connect(self.exposure_time_submitted)
exposure_grid.addWidget(self.exposure_time_label, 1, 1, _right_vcenter)
exposure_grid.addWidget(self.exposure_time_slider, 1, 2, 1, 2)
exposure_grid.addWidget(self.exposure_time_value_label, 1, 4, Qt.AlignmentFlag.AlignVCenter)
# Pixel clock (MHz): lowering it raises the max achievable exposure
# time (at the cost of frame rate) -- exposed here since operators
# hitting the exposure slider's ceiling need this knob to go further.
# Many uEye sensors only accept a short discrete list of pixel
# clocks (confirmed on hardware -- most values in the driver's own
# reported min/max range were rejected), so this slider's *position*
# is an index into self._pixel_clock_options, not a raw MHz value --
# every position it can land on is therefore one the hardware
# actually accepts. The real list is fetched once, over RPC, in
# _init_pixel_clock_options() (it's static per camera, not something
# that needs a live subscription); disabled until then.
self.pixel_clock_label = QLabel("Pixel clock", parent=self)
self.pixel_clock_slider = QSlider(Qt.Orientation.Horizontal, parent=self)
self.pixel_clock_slider.setRange(0, 0) # placeholder; see _set_pixel_clock_options()
self.pixel_clock_slider.setEnabled(False)
self.pixel_clock_value_label = QLabel("-- MHz", parent=self)
self.pixel_clock_value_label.setMinimumWidth(60)
self.pixel_clock_slider.valueChanged.connect(self._update_pixel_clock_value_label)
self.pixel_clock_slider.sliderReleased.connect(self.pixel_clock_submitted)
exposure_grid.addWidget(self.pixel_clock_label, 2, 1, _right_vcenter)
exposure_grid.addWidget(self.pixel_clock_slider, 2, 2, 1, 2)
exposure_grid.addWidget(self.pixel_clock_value_label, 2, 4, Qt.AlignmentFlag.AlignVCenter)
self.control_panel_layout.addWidget(self.exposure_grid_widget)
# separator: below the exposure/gain section, above the alignment
# values (2D positioner + zoom)
self.control_panel_layout.addWidget(self._create_separator())
# 2D Positioner (fixed size)
@@ -430,10 +638,19 @@ class OMNY_XRayEye(BECWidget, QWidget):
self.step_size.setValue(100)
# Submit button
self.submit_button = QPushButton("Submit", parent=self)
# Skip button: leaves out the current angle of the alignment (flat samples are
# edge-on at 90 deg). Hidden and disabled until a client script opts in via
# enable_skip_button(), so LamNI/omny never see it.
self._skip_enabled = False
self.skip_button = QPushButton("Skip", parent=self)
self.skip_button.setVisible(False)
self.skip_button.setEnabled(False)
self.skip_button.setToolTip("Leave out this angle (at least 4 positions are needed)")
# Add to layout form
step_size_form.addWidget(QLabel("Step Size", parent=self), 0, 0)
step_size_form.addWidget(self.step_size, 0, 1)
step_size_form.addWidget(self.submit_button, 2, 0, 1, 2)
step_size_form.addWidget(self.skip_button, 3, 0, 1, 2)
# Add form to control panel
self.control_panel_layout.addLayout(step_size_form)
@@ -487,10 +704,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
y=[1],
label="fit-x",
dap=["SineModel", "LinearModel"],
dap_parameters=[
{"frequency": {"value": 0.0174533, "vary": False, "min": 0.01, "max": 0.02}},
{"slope": {"value": 0, "vary": False, "min": 0.0, "max": 0.02}},
],
dap_parameters=SINE_FIT_DAP_PARAMETERS,
dap_oversample=5,
)
self.waveform_y.plot(
@@ -498,10 +712,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
y=[2],
label="fit-y",
dap=["SineModel", "LinearModel"],
dap_parameters=[
{"frequency": {"value": 0.0174533, "vary": False, "min": 0.01, "max": 0.02}},
{"slope": {"value": 0, "vary": False, "min": 0.0, "max": 0.02}},
],
dap_parameters=SINE_FIT_DAP_PARAMETERS,
dap_oversample=5,
)
self.fit_x = self.waveform_x.curves[0]
@@ -528,6 +739,7 @@ class OMNY_XRayEye(BECWidget, QWidget):
lambda x: self.motor_control_2d.setProperty("step_size", x)
)
self.submit_button.clicked.connect(self.submit)
self.skip_button.clicked.connect(self.skip)
# ROIPropertyTree's compact_color only styles the line color; line width
# still needs to be forced per-ROI here.
self.roi_manager.controller.roiAdded.connect(self._style_new_roi)
@@ -536,9 +748,45 @@ class OMNY_XRayEye(BECWidget, QWidget):
def _style_new_roi(self, roi):
"""Force a thinner outline on newly drawn ROIs (color is set via compact_color)."""
roi.line_width = self.ROI_LINE_WIDTH
if isinstance(roi, EllipticalROI):
self._setup_ellipse(roi)
# Live-update the microns readout while this ROI is drawn/resized.
roi.sigRegionChanged.connect(lambda r=roi: self._update_roi_size_readout(r))
self._update_roi_size_readout(roi)
# If the vertical lock is on (see lock_vertical_center()), snap this box's
# vertical center to the crosshair immediately, and again on every future
# drag/resize.
self._snap_roi_vertical(roi)
roi.sigRegionChangeFinished.connect(lambda r=roi: self._snap_roi_vertical(r))
def _snap_roi_vertical(self, roi):
"""Snap `roi`'s vertical center to the crosshair's fixed y position, keeping
its x position and height/diameter unchanged. No-op unless the vertical lock
is enabled (``lock_vertical_center()``) and the crosshair is visible. Safe to
call from ``roi.sigRegionChangeFinished`` -- if the ROI is already at the
target position this is a no-op, so it cannot loop back into itself.
"""
if not self._vertical_lock_enabled or not self.target_crosshair.is_visible():
return
_, fzp_y = self.crosshair_position()
if isinstance(roi, EllipticalROI):
# The ellipse rotates about pos(), so move it by the centre's offset.
geo = ellipse_geometry(roi)
dy = fzp_y - geo["center_y"]
if not math.isclose(dy, 0.0, abs_tol=1e-9):
roi.setPos(roi.pos().x(), roi.pos().y() + dy)
return
if isinstance(roi, RectangularROI):
height = roi.get_coordinates(typed=True)["height"]
elif isinstance(roi, CircularROI):
height = roi.get_coordinates(typed=True)["diameter"]
else:
return
current_x = roi.pos().x()
new_origin_y = fzp_y - height / 2
if math.isclose(roi.pos().y(), new_origin_y, abs_tol=1e-9):
return
roi.set_position(current_x, new_origin_y)
def _microns_per_pixel(self):
"""Resolve microns/pixel and unit label from the camera user parameter.
@@ -584,6 +832,14 @@ class OMNY_XRayEye(BECWidget, QWidget):
# Read the raw x/y size so a non-circular (ellipse) state is reflected
# correctly; for an aspect-locked circle both values are equal.
size_x, size_y = roi.state["size"]
elif isinstance(roi, EllipticalROI):
geo = ellipse_geometry(roi)
scale, unit = self._microns_per_pixel()
self.roi_size_line_edit.setText(
f"a = {geo['width'] * scale:.1f} {unit}, b = {geo['height'] * scale:.1f} {unit}, "
f"angle = {geo['angle']:.1f} deg"
)
return
else:
self.roi_size_line_edit.setText("")
return
@@ -648,8 +904,13 @@ class OMNY_XRayEye(BECWidget, QWidget):
def enable_move_buttons(self, enabled: bool):
self.motor_control_2d.setEnabled(enabled)
def _queue_guarded_toggles(self) -> tuple[ToggleSwitch, ToggleSwitch, ToggleSwitch]:
return (self.live_preview_toggle, self.shutter_toggle, self.camera_running_toggle)
def _queue_guarded_toggles(self) -> tuple[ToggleSwitch, ...]:
return (
self.live_preview_toggle,
self.shutter_toggle,
self.camera_running_toggle,
self.auto_exposure_toggle,
)
def _set_queue_toggles_blocked(self, blocked: bool):
if blocked == self._queue_busy:
@@ -740,6 +1001,185 @@ class OMNY_XRayEye(BECWidget, QWidget):
"""Hide the alignment target crosshair on the image view."""
self.target_crosshair.set_visible(False)
@SafeSlot()
@rpc_timeout(20)
def lock_vertical_center(self):
"""Snap every ROI's vertical center to the crosshair's fixed y position
(see ``crosshair_position()``), immediately and again on every future
draw/drag/resize, until ``unlock_vertical_center()`` is called. Used by
``x_ray_eye_align.py`` once the sample's height has been physically
centered on the FZP, so subsequent alignment boxes no longer need manual
vertical placement. See ``_snap_roi_vertical()``.
Also settable by the operator directly via the "ROI vertical lock"
toggle in the control panel (``_on_vertical_lock_toggle_changed``) --
e.g. to temporarily release a script-engaged lock mid-run in order to
freely draw/measure an unrelated box, then re-engage it. Both paths
keep the toggle's visual state in sync with ``_vertical_lock_enabled``.
"""
self._vertical_lock_enabled = True
self.vertical_lock_toggle.blockSignals(True)
self.vertical_lock_toggle.checked = True
self.vertical_lock_toggle.blockSignals(False)
for roi in self.roi_manager.controller.rois:
self._snap_roi_vertical(roi)
@SafeSlot()
@rpc_timeout(20)
def unlock_vertical_center(self):
"""Stop auto-snapping ROIs' vertical center -- see ``lock_vertical_center()``."""
self._vertical_lock_enabled = False
self.vertical_lock_toggle.blockSignals(True)
self.vertical_lock_toggle.checked = False
self.vertical_lock_toggle.blockSignals(False)
@SafeSlot(bool)
def _on_vertical_lock_toggle_changed(self, enabled: bool):
"""Operator flipped the "ROI vertical lock" toggle directly -- delegate to
lock_vertical_center()/unlock_vertical_center() so both entry points
(this toggle, and script/RPC calls) stay in sync."""
if enabled:
self.lock_vertical_center()
else:
self.unlock_vertical_center()
# ------------------------------------------------------------------
# Elliptical ROI (LamNI rotation-centre calibration)
# ------------------------------------------------------------------
def _setup_ellipse(self, roi: EllipticalROI):
"""Add independent axis scale handles and the angle bookkeeping to a new ellipse.
pyqtgraph's ``EllipseROI`` only has a rotate handle and one uniform scale handle,
so the aspect ratio could not be changed after drawing.
"""
roi.addScaleHandle([0.0, 0.5], [1.0, 0.5]) # width, anchored at the opposite side
roi.addScaleHandle([0.5, 1.0], [0.5, 0.0]) # height
roi.sigRegionChanged.connect(lambda r=roi: self._on_ellipse_changed(r))
if self._roi_angle_locked is not None:
self._apply_locked_angle(roi)
self._set_rotate_handle_visible(roi, self._roi_angle_locked is None)
self._sync_angle_spinbox(roi)
@staticmethod
def _set_rotate_handle_visible(roi, visible: bool):
for handle in roi.handles:
if handle["type"] == "r":
handle["item"].setVisible(visible)
def _apply_locked_angle(self, roi):
"""Rotate `roi` about its centre to the locked angle (no-op if already there)."""
locked = self._roi_angle_locked
if locked is None:
return
if math.isclose(_normalize_angle(roi.angle()), _normalize_angle(locked), abs_tol=1e-6):
return
roi.setAngle(locked, center=(0.5, 0.5))
def _on_ellipse_changed(self, roi):
self._apply_locked_angle(roi)
self._sync_angle_spinbox(roi)
def _sync_angle_spinbox(self, roi=None):
"""Show the active ellipse's rotation in the spin box without re-triggering it."""
if roi is None:
roi = self.roi_manager.single_active_roi
if not isinstance(roi, EllipticalROI):
return
self.roi_angle_spinbox.blockSignals(True)
self.roi_angle_spinbox.setValue(ellipse_geometry(roi)["angle"])
self.roi_angle_spinbox.blockSignals(False)
@SafeSlot(bool)
@rpc_timeout(20)
def enable_ellipse_roi(self, enable: bool):
"""Show/hide the elliptical ROI tool and its angle controls.
Hidden by default; the LamNI client enables it after the FZP centre has been
submitted, for the extended/smear rotation-centre calibrations (angle locked)
and for the X-ray eye alignment (angle free). Disabling removes an existing
ellipse and releases the angle lock.
Args:
enable(bool): True to offer the ellipse tool.
"""
self._ellipse_enabled = enable
self.roi_manager.toolbar.components.get_action("roi_ellipse").action.setVisible(enable)
for widget in (self.roi_angle_lock_label, self.roi_angle_lock_toggle, self.roi_angle_spinbox):
widget.setVisible(enable)
if not enable:
self.unlock_roi_angle()
self.roi_manager.toolbar.components.get_action("roi_ellipse").action.setChecked(False)
active = self.roi_manager.single_active_roi
if isinstance(active, EllipticalROI):
self.roi_manager.controller.remove_roi(active)
self._update_roi_size_readout(None)
@SafeSlot()
@rpc_timeout(20)
def lock_roi_angle(self, angle: float | None = None):
"""Fix the rotation of elliptical ROIs, now and for ellipses drawn later.
Args:
angle(float | None): Rotation in degrees. If None, the active ellipse's
current rotation (or, without one, the angle in the spin box) is used.
"""
if angle is None:
active = self.roi_manager.single_active_roi
if isinstance(active, EllipticalROI):
angle = ellipse_geometry(active)["angle"]
else:
angle = self.roi_angle_spinbox.value()
self._roi_angle_locked = _normalize_angle(float(angle))
self.roi_angle_lock_toggle.blockSignals(True)
self.roi_angle_lock_toggle.checked = True
self.roi_angle_lock_toggle.blockSignals(False)
self.roi_angle_spinbox.blockSignals(True)
self.roi_angle_spinbox.setValue(self._roi_angle_locked)
self.roi_angle_spinbox.blockSignals(False)
for roi in self.roi_manager.controller.rois:
if isinstance(roi, EllipticalROI):
self._apply_locked_angle(roi)
self._set_rotate_handle_visible(roi, False)
self._update_roi_size_readout(roi)
@SafeSlot()
@rpc_timeout(20)
def unlock_roi_angle(self):
"""Let elliptical ROIs rotate freely again -- see ``lock_roi_angle()``."""
self._roi_angle_locked = None
self.roi_angle_lock_toggle.blockSignals(True)
self.roi_angle_lock_toggle.checked = False
self.roi_angle_lock_toggle.blockSignals(False)
for roi in self.roi_manager.controller.rois:
if isinstance(roi, EllipticalROI):
self._set_rotate_handle_visible(roi, True)
@SafeSlot(bool)
def _on_roi_angle_lock_toggle_changed(self, enabled: bool):
if enabled:
self.lock_roi_angle()
else:
self.unlock_roi_angle()
@SafeSlot(float)
def _on_roi_angle_spinbox_changed(self, angle: float):
"""Operator typed an angle: rotate the active ellipse (and the lock, if engaged)."""
if self._roi_angle_locked is not None:
self.lock_roi_angle(angle)
return
active = self.roi_manager.single_active_roi
if isinstance(active, EllipticalROI):
active.setAngle(angle, center=(0.5, 0.5))
@SafeSlot()
def roi_angle(self) -> float | None:
"""Rotation in degrees of the active elliptical ROI, or None if there is none."""
active = self.roi_manager.single_active_roi
if isinstance(active, EllipticalROI):
return ellipse_geometry(active)["angle"]
return None
@SafeProperty(bool)
def crosshair_visible(self) -> bool:
"""Whether the alignment target crosshair is currently shown."""
@@ -886,11 +1326,142 @@ class OMNY_XRayEye(BECWidget, QWidget):
@SafeSlot(dict, dict)
def getting_camera_status(self, data, meta):
print(f"msg:{data}")
live_mode_enabled = data.get("signals").get(f"{CAMERA[0]}_live_mode_enabled").get("value")
signals = data.get("signals")
live_mode_enabled = signals.get(f"{CAMERA[0]}_live_mode_enabled").get("value")
self.camera_running_toggle.blockSignals(True)
self.camera_running_toggle.checked = live_mode_enabled
self.camera_running_toggle.blockSignals(False)
auto_exp = signals.get(f"{CAMERA[0]}_auto_exposure_enabled")
if auto_exp is not None:
enabled = bool(auto_exp.get("value"))
self.auto_exposure_toggle.blockSignals(True)
self.auto_exposure_toggle.checked = enabled
self.exposure_time_slider.setEnabled(not enabled)
self.auto_exposure_toggle.blockSignals(False)
# No auto_gain_enabled handling here -- see the comment above the
# (removed) toggle in _init_ui.
# Update the pixel-clock slider's position before the exposure ones
# below -- changing the pixel clock changes the exposure range, and
# both arrive together in this same message. (No range/bounds to
# reseed here -- pixel_clock_slider's range is fixed once
# self._pixel_clock_options is populated; see _init_pixel_clock_
# options()/_set_pixel_clock_options(). pixel_clock_min/max still
# exist on the device as informational metadata, just unused here.)
pixel_clock = signals.get(f"{CAMERA[0]}_pixel_clock")
if pixel_clock is not None:
self._set_pixel_clock_display(int(pixel_clock.get("value")))
# Reseed the exposure slider's bounds next (rare -- only changes if
# the pixel clock changed), so a value arriving in the same message
# is clamped against up-to-date min/max rather than a stale range.
# Round the min UP and the max DOWN (never the other way) -- e.g. a
# hardware min of 0.04 ms rounds to 0 in tenths-of-ms units, which
# would let the slider claim 0 ms exposure is settable when it isn't;
# ceil (clamped to at least 1) keeps the bound physical instead.
exposure_min = signals.get(f"{CAMERA[0]}_exposure_time_min")
exposure_max = signals.get(f"{CAMERA[0]}_exposure_time_max")
if exposure_min is not None and exposure_max is not None:
lo = max(1, math.ceil(float(exposure_min.get("value")) * self._EXPOSURE_SLIDER_SCALE))
hi = math.floor(float(exposure_max.get("value")) * self._EXPOSURE_SLIDER_SCALE)
if (lo, hi) != (self.exposure_time_slider.minimum(), self.exposure_time_slider.maximum()):
self.exposure_time_slider.setRange(lo, hi)
exposure_time = signals.get(f"{CAMERA[0]}_exposure_time")
if exposure_time is not None:
self.exposure_time_slider.blockSignals(True)
self.exposure_time_slider.setValue(
round(float(exposure_time.get("value")) * self._EXPOSURE_SLIDER_SCALE)
)
self.exposure_time_slider.blockSignals(False)
self._update_exposure_time_value_label(self.exposure_time_slider.value())
@SafeSlot(bool)
def auto_exposure_enabled_changed(self, enabled: bool):
if self._manual_toggle_blocked_by_queue():
logger.warning("Ignoring auto-exposure toggle while scan queue is busy.")
return
self.auto_exposure_toggle.blockSignals(True)
self.dev.get(CAMERA[0]).auto_exposure_enabled.put(enabled)
self.auto_exposure_toggle.checked = enabled
self.exposure_time_slider.setEnabled(not enabled)
self.auto_exposure_toggle.blockSignals(False)
def _update_exposure_time_value_label(self, raw_value: int):
self.exposure_time_value_label.setText(
f"{raw_value / self._EXPOSURE_SLIDER_SCALE:.1f} ms"
)
def exposure_time_submitted(self):
value_ms = self.exposure_time_slider.value() / self._EXPOSURE_SLIDER_SCALE
self.dev.get(CAMERA[0]).exposure_time.put(value_ms)
def _init_pixel_clock_options(self):
"""One-time fetch of the camera's actually-supported pixel clocks
(see get_pixel_clock_list()'s docstring for why this can't just be
[min, max] from a Kind.config signal) -- not a polling loop, this
list is static per camera, so a single RPC call at widget startup is
the right cost/benefit trade-off versus adding a whole signal path
for something that never changes at runtime."""
try:
options = self.dev.get(CAMERA[0]).get_pixel_clock_list()
except Exception as exc:
logger.warning(f"Failed to fetch pixel clock options for OMNY_XRayEye: {exc}")
return
self._set_pixel_clock_options(sorted(int(v) for v in options))
def _set_pixel_clock_options(self, options: list[int]):
self._pixel_clock_options = options
if not options:
self.pixel_clock_slider.setEnabled(False)
return
self.pixel_clock_slider.blockSignals(True)
self.pixel_clock_slider.setRange(0, len(options) - 1)
self.pixel_clock_slider.blockSignals(False)
self.pixel_clock_slider.setEnabled(True)
if self._last_pixel_clock_mhz is not None:
# A status message already arrived before this RPC call
# returned -- apply it now that we can actually resolve it to a
# slider position.
self._set_pixel_clock_display(self._last_pixel_clock_mhz)
else:
# No status message yet either -- keep the label in sync with
# the slider's default position (index 0, the lowest option).
self._update_pixel_clock_value_label(self.pixel_clock_slider.value())
def _set_pixel_clock_display(self, mhz: int):
"""Move the slider to the option nearest `mhz` (from hardware, e.g.
via getting_camera_status()) and update the value label -- never a
raw MHz value on the slider itself, only a valid index."""
self._last_pixel_clock_mhz = mhz
if not self._pixel_clock_options:
# Options not fetched yet -- _set_pixel_clock_options() will
# call back into this once they arrive.
self.pixel_clock_value_label.setText(f"{mhz} MHz")
return
index = min(
range(len(self._pixel_clock_options)),
key=lambda i: abs(self._pixel_clock_options[i] - mhz),
)
self.pixel_clock_slider.blockSignals(True)
self.pixel_clock_slider.setValue(index)
self.pixel_clock_slider.blockSignals(False)
self._update_pixel_clock_value_label(index)
def _update_pixel_clock_value_label(self, index: int):
if 0 <= index < len(self._pixel_clock_options):
self.pixel_clock_value_label.setText(f"{self._pixel_clock_options[index]} MHz")
else:
self.pixel_clock_value_label.setText("-- MHz")
def pixel_clock_submitted(self):
index = self.pixel_clock_slider.value()
if not 0 <= index < len(self._pixel_clock_options):
return
self.dev.get(CAMERA[0]).pixel_clock.put(self._pixel_clock_options[index])
@SafeSlot(bool)
def opening_shutter(self, enabled: bool):
if self._manual_toggle_blocked_by_queue():
@@ -942,6 +1513,26 @@ class OMNY_XRayEye(BECWidget, QWidget):
self.submit_button.setEnabled(True)
else:
self.submit_button.setEnabled(False)
self._refresh_skip_button()
@SafeSlot(bool)
@rpc_timeout(20)
def enable_skip_button(self, enable: bool):
"""
Allow/forbid skipping the current alignment angle.
The first call also shows the button (it is hidden for clients that never call this).
The button is only clickable while this is True *and* the submit button is enabled,
so it follows the client's "please wait ..." phases automatically.
Args:
enable(bool): True if the client accepts a skip for the current step
"""
self.skip_button.setVisible(True)
self._skip_enabled = bool(enable)
self._refresh_skip_button()
def _refresh_skip_button(self):
self.skip_button.setEnabled(self._skip_enabled and self.submit_button.isEnabled())
@SafeSlot(bool)
@rpc_timeout(20)
@@ -1012,11 +1603,20 @@ class OMNY_XRayEye(BECWidget, QWidget):
if self.roi_manager.single_active_roi is None:
logger.warning("No active ROI")
return
roi_coordinates = self.roi_manager.single_active_roi.get_coordinates()
active_roi = self.roi_manager.single_active_roi
roi_angle = None
if isinstance(active_roi, EllipticalROI):
roi_coordinates = ellipse_geometry(active_roi)
roi_angle = roi_coordinates["angle"]
else:
roi_coordinates = active_roi.get_coordinates()
roi_center_x = roi_coordinates["center_x"]
roi_center_y = roi_coordinates["center_y"]
# Case of rectangular ROI
if isinstance(self.roi_manager.single_active_roi, RectangularROI):
if isinstance(active_roi, EllipticalROI):
roi_width = roi_coordinates["width"]
roi_height = roi_coordinates["height"]
elif isinstance(self.roi_manager.single_active_roi, RectangularROI):
roi_width = roi_coordinates["width"]
roi_height = roi_coordinates["height"]
elif isinstance(self.roi_manager.single_active_roi, CircularROI):
@@ -1033,10 +1633,23 @@ class OMNY_XRayEye(BECWidget, QWidget):
getattr(self.dev.omny_xray_gui, f"yval_y_{step}").set(roi_center_y)
getattr(self.dev.omny_xray_gui, f"width_x_{step}").set(roi_width)
getattr(self.dev.omny_xray_gui, f"width_y_{step}").set(roi_height)
# Rotation only matters for the ellipse; reset it for every other ROI so a
# stale angle from an earlier submit is never read for this step.
angle_signal = getattr(self.dev.omny_xray_gui, f"roi_angle_{step}", None)
if angle_signal is not None:
angle_signal.set(roi_angle if roi_angle is not None else 0.0)
self.dev.omny_xray_gui.submit.set(1)
finally:
self.submit_button.blockSignals(False)
@SafeSlot()
def skip(self):
"""Skip the current alignment angle: signal the client with submit = -1 (no ROI needed)."""
# Disable right away so a double click cannot skip the following step as well; the
# client re-enables the buttons once it is ready for the next step.
self.skip_button.setEnabled(False)
self.dev.omny_xray_gui.submit.set(-1)
def cleanup(self):
"""Cleanup connections on widget close -> disconnect slots and stop live mode of camera."""
self._queue_idle_timer.stop()
+37 -10
View File
@@ -1,26 +1,53 @@
# eiger_1_5:
# description: Eiger 1.5M in-vacuum detector
# deviceClass: csaxs_bec.devices.jungfraujoch.eiger_1_5m.Eiger1_5M
# Eiger detectors (JungfrauJoch). Only one may be enabled at a time -- the JFJ broker
# serves a single selected/initialised detector, so switching the active one here also
# requires selecting/initialising the matching detector on the JFJ broker itself.
# JungfrauJoch-side config (detector selection/init, pixel mask, azimuthal integration
# settings) is done in the Web UI at http://sls-jfjoch-001.psi.ch:8080.
# The Eiger 2 Si 500K detector server itself is reachable at 129.129.122.159.
# eiger2_si_500k:
# description: Eiger 2 Si 500K detector
# deviceClass: csaxs_bec.devices.jungfraujoch.eiger2_si_500k.Eiger2Si500K
# deviceConfig:
# detector_distance: 2200
# beam_center: [870, 1203]
# detector_distance: 1000
# beam_center: [200, 200]
# # Re-initialise the detector on staging if JFJ is in Error state (shutter must be closed)
# auto_reinitialize: true
# onFailure: raise
# enabled: True
# readoutPriority: async
# softwareTrigger: False
# deviceTags:
# - bl_detectors
# - eiger
eiger_9:
description: Eiger 9M detector
deviceClass: csaxs_bec.devices.jungfraujoch.eiger_9m.Eiger9M
eiger_1_5:
description: Eiger 1.5M in-vacuum detector
deviceClass: csaxs_bec.devices.jungfraujoch.eiger_1_5m.Eiger1_5M
deviceConfig:
detector_distance: 6975.225
beam_center: [1149.71, 1450.89]
detector_distance: 2200
beam_center: [870, 1203]
onFailure: raise
enabled: True
readoutPriority: async
softwareTrigger: False
deviceTags:
- bl_detectors
- eiger
# eiger_9:
# description: Eiger 9M detector
# deviceClass: csaxs_bec.devices.jungfraujoch.eiger_9m.Eiger9M
# deviceConfig:
# detector_distance: 6975.225
# beam_center: [1149.71, 1450.89]
# onFailure: raise
# enabled: True
# readoutPriority: async
# softwareTrigger: False
# deviceTags:
# - bl_detectors
# - eiger
# ids_cam:
# description: IDS camera for live image acquisition
+6 -1
View File
@@ -104,6 +104,7 @@ xbpm3x:
connectionTimeout: 20
userParameter:
init_position: -22.5
in_position: -0.18
# bl_smar_stage to use csaxs reference method. assign number according to axis channel
bl_smar_stage: 0
deviceTags:
@@ -129,6 +130,7 @@ xbpm3y:
connectionTimeout: 20
userParameter:
init_position: -2
in_position: -0.485
# bl_smar_stage to use csaxs reference method. assign number according to axis channel
bl_smar_stage: 1
deviceTags:
@@ -960,6 +962,7 @@ fttrx1:
deviceClass: ophyd_devices.devices.psi_motor.EpicsUserMotorVME
deviceConfig:
prefix: X12SA-ES1-FTS1:TRX1
tolerance: 0.2
onFailure: retry
enabled: true
readoutPriority: baseline
@@ -1008,6 +1011,7 @@ fttrz:
deviceClass: ophyd_devices.devices.psi_motor.EpicsUserMotorVME
deviceConfig:
prefix: X12SA-ES1-FTS1:TRZ1
tolerance: 0.2
onFailure: retry
enabled: true
readoutPriority: baseline
@@ -1092,6 +1096,7 @@ dttrz:
deviceClass: ophyd_devices.devices.psi_motor.EpicsUserMotorVME
deviceConfig:
prefix: X12SA-ES1-DETT:TRZ1
tolerance: 0.2
onFailure: retry
enabled: true
readoutPriority: baseline
@@ -1151,7 +1156,7 @@ gain_bpm_xbox2:
- bl_endstation
bpm_xbox2_slowrb:
description: BPM Xbox 2 (First Xbox in ES hutch) readback
description: BPM Xbox 2 (First Xbox in ES hutch) readback (recommended gain 1e8)
deviceClass: csaxs_bec.devices.pseudo_devices.bpm.BPM
deviceConfig:
left_top_ref: galilrioesxbox.analog_in.ch0
@@ -209,6 +209,7 @@ xbpm2x:
userParameter:
init_position: 22.5
in_position: -1.5
# in_position: 0.5 (from Excel file)
# bl_smar_stage to use csaxs reference method. assign number according to axis channel
bl_smar_stage: 0
deviceTags:
+29
View File
@@ -51,3 +51,32 @@ operator:
softwareTrigger: false
deviceTags:
- machine
injection_status:
# Top-up gating: '0' / 'Filling Storage Ring' means the storage ring is
# being refilled. readoutPriority is 'monitored' (not 'baseline') so this
# is captured continuously throughout the scan, not just read once at
# scan start - a top-up can start or stop mid-scan.
#
# Enum values of AGETI-CVME-MASTER-TMA:SR-Inj-Status-I (8 states):
# 0: Filling Storage Ring <- top-up in progress
# 1: Waiting manual trig...
# 2: Electron gun event OFF!
# 3: SR Injection ready
# 4: Inj. events OFF
# 5: Beam Dump
# 6: !Event LI-RF set on 10Hz!
# 7: !Injection guard inhibit!
description: SLS injection/top-up status
deviceClass: ophyd_devices.EpicsSignalRO
deviceConfig:
read_pv: AGETI-CVME-MASTER-TMA:SR-Inj-Status-I
auto_monitor: true
string: true
onFailure: buffer
enabled: true
readoutPriority: monitored
readOnly: true
softwareTrigger: false
deviceTags:
- machine
+6 -6
View File
@@ -16,11 +16,11 @@ endstation:
detectors:
- !include ./bl_detectors.yaml
xeye:
- !include ./xeye.yaml
# xeye:
# - !include ./xeye.yaml
ssaxs:
- !include ./ssaxs.yaml
# ssaxs:
# - !include ./ssaxs.yaml
#sastt:
# - !include ./sastt.yaml
@@ -31,8 +31,8 @@ ssaxs:
#omny:
# - !include ./ptycho_omny.yaml
#lamni:
# - !include ./ptycho_lamni.yaml
lamni:
- !include ./ptycho_lamni.yaml
# user setup:
# - !include ./user_setup.yaml
+161 -54
View File
@@ -21,8 +21,9 @@ feyex:
userParameter:
in: -16.453
out: -1
fttrx_in: 2.3
fttrx_out: -24
fttrx_in: 11.025 #-2.3
fttrx_out: -7 #-20
#fttrx_out: -27
deviceTags:
- ptycho_flomni
@@ -46,6 +47,7 @@ feyey:
in: -10.09
deviceTags:
- ptycho_flomni
fheater:
description: Heater Y
deviceClass: csaxs_bec.devices.omny.galil.fgalil_ophyd.FlomniGalilMotor
@@ -62,6 +64,10 @@ fheater:
readOnly: false
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#in: -0.5
in: -11.5
enabled: 0
deviceTags:
- ptycho_flomni
@@ -82,18 +88,24 @@ foptx:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#120 micros, 60
# in: -13.890
# out: -14
#150 micros, 60
# in: -14.191
# out: -13.831
#170 micron, 60 nm
in: -13.831
out: -13.831
#in: -13.8909
#out: -13.831
#250 micron, 30 nm, Abe structures
# in: -13.8809375
# out: -14.1809
#250 micron, 30 nm, Tomas structures
# in: -14.5490625
# out: -14.1809
in: -14.5490625
out: -14.1809
fzp_diameter: 170 # microns
fzp_outermost_zone_width: 60 # nm
detector_distance: -1 # mm, sample-to-detector; unknown for now
detector_distance: 7196 # mm, sample-to-detector; unknown for now
fzp_details: "manufacturing notes here" # free-text FZP manufacturing notes
deviceTags:
- ptycho_flomni
@@ -115,15 +127,18 @@ fopty:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#150 micron, 60
# in: 1.02
# out: 0.869
#170 micron, 60 nm
in: 0.42
out: 0.57
#in: 0.42
#out: 0.57
#250 micron, 30 nm, Abe structures
# in: 2.8299
# out: 2.8299
#250 micron, 30 nm, Tomas structures
# in: 2.8419921875
# out: 2.8419921875
in: 2.8419921875
out: 2.8419921875
deviceTags:
- ptycho_flomni
@@ -277,7 +292,7 @@ ftransy:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
sensor_voltage: -1.6
sensor_voltage: -2.2
deviceTags:
- ptycho_flomni
ftransz:
@@ -354,18 +369,24 @@ fosax:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#120micron, 60 nm, 7.9 kev
# in: 8.74251
# out: 5.1
#150micron, 60 nm, 7.9 kev
# in: 9.0393
# out: 5.1
#170 micron, 60 nm, 7.6 kev
#in: 8.7568
#out: 5.1
# in: 8.7568
# out: 5.1
#170 micron, 60 nm, 7.9 kev
in: 8.727079
out: 5.1
#in: 8.718233
#out: 5.1
#250 micron, 30 nm, Abe structures
# in: 8.7392
# out: 5.1
#250 micron, 30 nm, Tomas structures
# in: 9.420798
# out: 5.1
in: 9.400897
out: 5.1
deviceTags:
- ptycho_flomni
@@ -386,14 +407,18 @@ fosay:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#120 micron, 60 nm, 7.9 kev
# in: -0.645276
#150 micron, 60 nm, 7.9 kev
#in: -0.6422
#170 micron, 60 nm, 7.6 kev
#in: -0.0235
#170 micron, 60 nm, 7.9 kev
in: -0.04603
#in: -0.062253
#250 micron, 30 nm, Abe structures
# in: -2.3684
#250 micron, 30 nm, Tomas structures
# in: -2.383993
in: -2.401459
deviceTags:
- ptycho_flomni
fosaz:
@@ -413,18 +438,25 @@ fosaz:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
#150 micron, 60 nm
# in: 11.0998
# out: 6
#170 micron, 60 nm, 7.6 kev
#in: 8.5
#out: 6
#170 micron, 60 nm, 7.9 kev, foptz 16.9, probe size 7.5 mu
in: 13.1
out: 6
#in: 14.1
#out: 4
# micron, 30 nm, 7.9 kev, very close to the sample. make sure foptz is 32.02 or smaller //abe's fzp's
# in: 0.5
# out: -5
in: 0.4
out: -5
deviceTags:
- ptycho_flomni
############################################################
#################### flOMNI RT motors ######################
############################################################
@@ -584,6 +616,28 @@ flomni_temphum:
deviceTags:
- ptycho_flomni
# ############################################################
# ############ flOMNI real-time loop settings #################
# ############################################################
# The loop settings (FZP, trigger, tracker; flOMNI FeedbackLoop/loopsettings) stored with every scan: read from the
# web HMI on the loop machine at the start of each scan (baseline). Web HMI down: available = 0, the scan continues.
# The user commands flomni.realtime_system go through this device (RPC), so the token is needed only here.
# token: the web HMI API token (~/loop-data/webhmi/api_token on mpc2844; behind the beamline firewall, Mirko
# 2026-10-23). When empty, token_file (on the machine of the device server) is used.
flomni_loop_settings:
description: flOMNI real-time loop settings (web HMI loop settings, snapshot at scan start)
deviceClass: csaxs_bec.devices.omny.flomni_loop_settings.FlomniLoopSettings
deviceConfig:
host: mpc2844.psi.ch
token: "3CcGHMdz16G7zuC6iPHEoZPWvKDY1HkCZnTlP1msZbg"
token_file: ~/.flomni_webhmi_token
timeout: 2.0
enabled: true
onFailure: buffer
readOnly: true
readoutPriority: baseline
deviceTags:
- ptycho_flomni
# ############################################################
# ########## OMNY / flOMNI / LamNI fast shutter ##############
# REPLACED BY SIMULATED BEAMLINE DEVICE !!!!!!!!!!!!!!!!
# ############################################################
@@ -625,33 +679,86 @@ calculated_signal:
############################################################
#################### OMNY Pandabox #########################
############################################################
omny_panda:
readoutPriority: async
deviceClass: csaxs_bec.devices.panda_box.panda_box_omny.PandaBoxOMNY
deviceConfig:
host: omny-panda.psi.ch
signal_alias:
FMC_IN.VAL1.Min: cap_voltage_fzp_y_min
FMC_IN.VAL1.Max: cap_voltage_fzp_y_max
FMC_IN.VAL1.Mean: cap_voltage_fzp_y_mean
FMC_IN.VAL2.Min: cap_voltage_fzp_x_min
FMC_IN.VAL2.Max: cap_voltage_fzp_x_max
FMC_IN.VAL2.Mean: cap_voltage_fzp_x_mean
INENC1.VAL.Max: interf_st_fzp_y_max
INENC1.VAL.Mean: interf_st_fzp_y_mean
INENC1.VAL.Min: interf_st_fzp_y_min
INENC2.VAL.Max: interf_st_fzp_x_max
INENC2.VAL.Mean: interf_st_fzp_x_mean
INENC2.VAL.Min: interf_st_fzp_x_min
INENC3.VAL.Max: interf_st_rotz_max
INENC3.VAL.Mean: interf_st_rotz_mean
INENC3.VAL.Min: interf_st_rotz_min
INENC4.VAL.Max: interf_st_rotx_max
INENC4.VAL.Mean: interf_st_rotx_mean
INENC4.VAL.Min: interf_st_rotx_min
PCAP.GATE_DURATION.Value: pcap_gate_duration_value
enabled: true
readOnly: false
softwareTrigger: false
deviceTags:
- ptycho_flomni
# Mode A (legacy per-gate aggregate acquisition) and Mode B (continuous free-running
# raw-sample acquisition, see docs/developer/panda_box_free_running_setup.md) need different
# PandA hardware layouts and can't run simultaneously (see that doc's "Open question"). Only
# one of the two devices below should be active at a time, matching whichever layout is
# currently loaded on the PandA.
#
# NOTE 2026-09-15: setting enabled: false on omny_panda_continuous was not sufficient by
# itself to stop it being active in the running BEC session (both devices ended up
# connected/active simultaneously against the same physical PandA at once) - a device that
# was already connected apparently doesn't get disconnected just by disabling it in a config
# reload. Commenting the block out is the reliable way to guarantee it's not loaded; do the
# same for omny_panda when switching to Mode B, and don't rely on enabled: false alone
# without also confirming (e.g. restarting the device server) that the other device has
# actually dropped its connection.
#
# Currently: Mode B active for speed testing beyond the initial 100 Hz, Mode A commented out
# (both modes confirmed working end-to-end 2026-09-15; this swap 2026-09-15 is for Mode B
# speed testing specifically).
# omny_panda:
# readoutPriority: async
# deviceClass: csaxs_bec.devices.panda_box.panda_box_omny.PandaBoxOMNY
# deviceConfig:
# host: omny-panda.psi.ch
# signal_alias:
# FMC_IN.VAL1.Min: cap_voltage_fzp_y_min
# FMC_IN.VAL1.Max: cap_voltage_fzp_y_max
# FMC_IN.VAL1.Mean: cap_voltage_fzp_y_mean
# FMC_IN.VAL2.Min: cap_voltage_fzp_x_min
# FMC_IN.VAL2.Max: cap_voltage_fzp_x_max
# FMC_IN.VAL2.Mean: cap_voltage_fzp_x_mean
# INENC1.VAL.Max: interf_st_fzp_y_max
# INENC1.VAL.Mean: interf_st_fzp_y_mean
# INENC1.VAL.Min: interf_st_fzp_y_min
# INENC2.VAL.Max: interf_st_fzp_x_max
# INENC2.VAL.Mean: interf_st_fzp_x_mean
# INENC2.VAL.Min: interf_st_fzp_x_min
# INENC3.VAL.Max: interf_st_rotz_max
# INENC3.VAL.Mean: interf_st_rotz_mean
# INENC3.VAL.Min: interf_st_rotz_min
# INENC4.VAL.Max: interf_st_rotx_max
# INENC4.VAL.Mean: interf_st_rotx_mean
# INENC4.VAL.Min: interf_st_rotx_min
# PCAP.GATE_DURATION.Value: pcap_gate_duration_value
# enabled: true
# readOnly: false
# softwareTrigger: false
# deviceTags:
# - ptycho_flomni
# omny_panda_continuous:
# readoutPriority: async
# deviceClass: csaxs_bec.devices.panda_box.panda_box_omny.PandaBoxOMNY
# deviceConfig:
# host: omny-panda.psi.ch
# raw_stream_mode: true
# # Only actually Arm() PandA for scans that consume its data -- generic BEC staging
# # (stage_all_devices/pre_scan_all_devices) otherwise arms this device for every scan type
# # (grid_scan, line_scan, alignment moves, ...), not just the ptycho scan that reads it. See
# # docs/developer/panda_box_free_running_setup.md and
# # AI_docs/PANDA_POSITION_VALIDATION.md sec 7 (stale frame_counter exposure).
# arm_scan_allowlist:
# - flomni_fermat_scan
# signal_alias:
# FMC_IN.VAL1.Value: cap_voltage_fzp_y
# FMC_IN.VAL2.Value: cap_voltage_fzp_x
# INENC1.VAL.Value: interf_st_fzp_y
# INENC2.VAL.Value: interf_st_fzp_x
# INENC3.VAL.Value: interf_st_rotz
# INENC4.VAL.Value: interf_st_rotx
# PCAP.GATE_DURATION.Value: pcap_gate_duration_value
# # TTLIN1 (detector trigger) feeds both blocks below - confirmed 2026-09-15 against the
# # omny-panda hardware, see docs/developer/panda_box_free_running_setup.md. PCAP.BITS0
# # is a shared 32-bit word; PandaBoxOMNY masks it to bit 0 (TTLIN1.VAL) before
# # publishing gate_detector_active. COUNTER1 counts TTLIN1's falling edge, i.e. one
# # increment per completed exposure; it also doubles as the on_complete signal. Confirmed
# # working end-to-end 2026-09-15; now under speed testing beyond the initial 100 Hz.
# PCAP.BITS0.Value: gate_detector_active
# COUNTER1.OUT.Value: frame_counter
# enabled: true
# readOnly: false
# softwareTrigger: false
# deviceTags:
# - ptycho_flomni
+79 -18
View File
@@ -19,7 +19,13 @@ leyex:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: 14.117
in: 13.18
# flight tube / detector table positions used by leye_out() (*_in) and
# leye_in() (*_out)
fttrz_in: 1381.433
fttrz_out: 2000
dttrz_in: 12614.99
dttrz_out: 13263
deviceTags:
- ptycho_lamni
leyey:
@@ -39,7 +45,7 @@ leyey:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: 48.069
in: 47.94
out: 0.5
deviceTags:
- ptycho_lamni
@@ -60,13 +66,21 @@ loptx:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: -0.244
out: -0.699
in: -0.8698
#out: -0.8665
out: -0.8659
# 170 micron, 60 nm
fzp_diameter: 170 # microns
fzp_outermost_zone_width: 60 # nm
fzp_diameter: 250 # microns
fzp_outermost_zone_width: 30 # nm
detector_distance: -1 # mm, sample-to-detector; unknown for now
fzp_details: "manufacturing notes here" # free-text FZP manufacturing notes
fzp_details: "30 nm Abe Tomas 05/2026" # free-text FZP manufacturing notes
# in: -0.244
# out: -0.699
# # 170 micron, 60 nm
# fzp_diameter: 170 # microns
# fzp_outermost_zone_width: 60 # nm
# detector_distance: -1 # mm, sample-to-detector; unknown for now
# fzp_details: "manufacturing notes here" # free-text FZP manufacturing notes
deviceTags:
- ptycho_lamni
lopty:
@@ -86,8 +100,8 @@ lopty:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: 3.724
out: 3.53
in: 2.6266
out: 3.029
deviceTags:
- ptycho_lamni
loptz:
@@ -97,8 +111,8 @@ loptz:
axis_Id: D
host: mpc2680.psi.ch
limits:
- 0
- 0
- 70
- 83
port: 8081
sign: -1
enabled: true
@@ -106,6 +120,10 @@ loptz:
readOnly: false
readoutPriority: baseline
connectionTimeout: 20
userParameter:
# OSA collides with the sample holder when loptz + losaz reaches this
# value (measured 2026-10-14: loptz 77.8676, losaz 10.0029)
collision_offset: 87.87
deviceTags:
- ptycho_lamni
lsamrot:
@@ -144,7 +162,7 @@ lsamx:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
center: 8.768
center: 9.68
deviceTags:
- ptycho_lamni
lsamy:
@@ -164,7 +182,7 @@ lsamy:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
center: 10.041
center: 9.47
deviceTags:
- ptycho_lamni
@@ -189,7 +207,7 @@ losax:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: -1.442
in: -1.6437
deviceTags:
- ptycho_lamni
losay:
@@ -209,7 +227,7 @@ losay:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: -0.171
in: -0.2109
out: 3.8
deviceTags:
- ptycho_lamni
@@ -230,11 +248,54 @@ losaz:
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: -1
in: 6.8288
out: -3
deviceTags:
- ptycho_lamni
lcsx:
description: CS X
deviceClass: csaxs_bec.devices.smaract.smaract_ophyd.SmaractMotor
deviceConfig:
axis_Id: D
host: mpc2680.psi.ch
limits:
- 0
- 0
port: 8085
sign: 1
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: -1.831
deviceTags:
- ptycho_lamni
lcsy:
description: CS Y
deviceClass: csaxs_bec.devices.smaract.smaract_ophyd.SmaractMotor
deviceConfig:
axis_Id: E
host: mpc2680.psi.ch
limits:
- 0
- 0
port: 8085
sign: -1
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: 1.0795
out: 5
deviceTags:
- ptycho_lamni
############################################################
#################### flOMNI RT motors ######################
############################################################
@@ -297,7 +358,7 @@ cam_xeye:
deviceConfig:
camera_id: 15
bits_per_pixel: 24
num_rotation_90: 3
num_rotation_90: 1
transpose: false
force_monochrome: true
m_n_colormode: 1
@@ -306,7 +367,7 @@ cam_xeye:
readOnly: false
readoutPriority: async
userParameter:
pixel_calibration: 0.0009174311926605505 # mm/pixel (= 0.2 / 218)
pixel_calibration: 0.00045872 # mm/pixel (= 0.2 / 218)
deviceTags:
- ptycho_lamni
#################### GUI Signals ###########################
+4 -4
View File
@@ -12,7 +12,7 @@ cam200:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- ptycho_omny
@@ -27,7 +27,7 @@ cam201:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- ptycho_omny
@@ -42,7 +42,7 @@ cam202:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- ptycho_omny
@@ -57,7 +57,7 @@ cam203:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- ptycho_omny
@@ -24,5 +24,5 @@ fsh:
flomni:
- !include ../ptycho_flomni.yaml
machine:
- !include ../machine.yml
# machine:
# - !include ../machine.yml
@@ -104,6 +104,9 @@ fheater:
readOnly: false
readoutPriority: baseline
connectionTimeout: 20
userParameter:
in: -0.5
enabled: 1
deviceTags:
- simulated_flomni
@@ -141,7 +141,7 @@ lsamrot:
sign: 1
sim_stppermm: 50154.32099
sim_encpermm: 36000
sim_velocity: 5 # 48 #4x than hw
sim_velocity: 48 # 4x faster than hw
sim_initial_position: 0
tolerance: 1
enabled: true
@@ -15,7 +15,7 @@ cam200:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- simulated_omny
cam201:
@@ -29,7 +29,7 @@ cam201:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- simulated_omny
cam202:
@@ -43,7 +43,7 @@ cam202:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- simulated_omny
cam203:
@@ -57,7 +57,31 @@ cam203:
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
readoutPriority: on_request
deviceTags:
- simulated_omny
# Hand-added (not part of the generated ptycho_omny.yaml source -- OMNY has no cam_xeye
# entry in the real config yet). The OMNY_XRayEye widget hardcodes the device name
# "cam_xeye", so omnygui_show_xeyealign() needs one to exist even in simulation.
# camera_id/pixel_calibration are placeholders duplicated from flomni's real cam_xeye
# (simulated_flomni.yaml); see omny/AI_docs/OPEN_ISSUES.md for the real-hardware follow-up.
cam_xeye:
description: Camera OMNY Xray eye (simulated, placeholder)
deviceClass: csaxs_bec.devices.sim.sim_cameras.SimIDSCamera
deviceConfig:
camera_id: 11
bits_per_pixel: 24
num_rotation_90: 0
transpose: true
force_monochrome: true
m_n_colormode: 1
sim_noise_std: 0
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: on_request
userParameter:
pixel_calibration: 0.00044247787610619477 # mm/pixel, placeholder (flomni's value)
deviceTags:
- simulated_omny
rtx:
@@ -163,6 +187,7 @@ ofzpx:
- 0
port: 8081
sign: 1
sim_velocity: 10.0
sim_initial_position: -0.4317
enabled: true
onFailure: buffer
@@ -185,6 +210,7 @@ ofzpy:
- 0
port: 8081
sign: 1
sim_velocity: 10.0
sim_initial_position: 0.7944
enabled: true
onFailure: buffer
@@ -207,6 +233,7 @@ ofzpz:
- 0
port: 8081
sign: -1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -228,6 +255,7 @@ otransx:
- 0
port: 8081
sign: 1
sim_velocity: 100.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -249,7 +277,20 @@ otransy:
- 0
port: 8081
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
# _otransfer_check_sensor_connected() reads @AN[8001] as the gripper's
# sensor voltage and expects it within 0.5V of gripper_sensorvoltagetarget
# below (-2.3). Without an override here it falls back to the simulated
# Galil's shared default for unseeded analog channels (-8.0V, tuned for
# motor-temperature registers elsewhere, not this one), which always
# fails the check -- see sim_galil.py's _DEFAULT_ANALOG_INPUT_V.
# Key must be a quoted string, not a bare YAML int: BEC's msgpack config
# transport uses strict_map_key=True, which rejects int map keys outright
# (a bare 8001: here breaks config reload entirely) -- matches the
# existing convention in simulated_flomni.yaml's own sim_analog_inputs.
sim_analog_inputs:
"8001": -2.3
enabled: true
onFailure: buffer
readOnly: false
@@ -271,6 +312,7 @@ otransz:
- 0
port: 8081
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -292,6 +334,7 @@ osamx:
- 0
port: 8082
sign: 1
sim_velocity: 10.0
sim_initial_position: -0.1
enabled: true
onFailure: buffer
@@ -313,6 +356,7 @@ osamz:
- 0
port: 8082
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: false
onFailure: buffer
@@ -334,6 +378,7 @@ oosay:
- 0
port: 8082
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -356,6 +401,7 @@ oosax:
- 0
port: 8082
sign: -1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -378,6 +424,7 @@ oosaz:
- 0
port: 8082
sign: -1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -400,6 +447,7 @@ oparkz:
- 0
port: 8082
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -421,7 +469,16 @@ oshuttleopen:
- 0
port: 8082
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
# _otransfer_shield_open() drives this axis open with plain PRG=17000
# then PRG=37000 (omny_sample_transfer_mixin.py) -- never through the
# naxis/XQ#NEWPAR reference-axis routine that normally flips the
# limit-switch flags -- so without a position-derived trip point,
# _otransfer_is_shuttle_open() (checks the high/forward limit) never
# sees it as open. 1.0 mm is comfortably below the ~1.05 mm (54000
# step) total commanded excursion.
sim_high_limit_trip_position: 1.0
enabled: true
onFailure: buffer
readOnly: true
@@ -442,7 +499,20 @@ oshuttlealign:
- 0
port: 8082
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
# Real hardware idles with the shuttle aligner down (at the forward/high
# limit, _otransfer_shuttle_aligner_down()'s target) -- start the sim
# already there so the first alignment each session doesn't spend time
# re-driving to a limit it was already sitting at.
sim_limit_high_active: true
# _otransfer_shuttle_align()'s "routine for broken hardware" comment
# (omny_sample_transfer_mixin.py): the negative/low limit switch is
# wired close enough to the positive/high one that it also trips once
# driven positive of the forward-reference zero point, not just via
# normal referencing. Without this, the alignment routine's final
# "searchwrongendswitch" loop never finds the low switch and times out.
sim_quirky_positive_trips_low_limit: true
enabled: true
onFailure: buffer
readOnly: true
@@ -463,6 +533,7 @@ osamy:
- 0
port: 8083
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -484,6 +555,7 @@ otracky:
- 0
port: 8083
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -505,7 +577,8 @@ osamroy:
- 0
port: 8083
sign: 1
sim_initial_position: 0
sim_velocity: 5.716463414632019
sim_initial_position: -25
enabled: true
onFailure: buffer
readOnly: false
@@ -526,6 +599,7 @@ otrackz:
- 0
port: 8083
sign: -1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -547,6 +621,7 @@ oeyex:
- 0
port: 8083
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -568,6 +643,7 @@ oeyez:
- 0
port: 8083
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -589,6 +665,7 @@ oeyey:
- 0
port: 8083
sign: 1
sim_velocity: 10.0
sim_initial_position: 0
enabled: true
onFailure: buffer
@@ -652,7 +729,12 @@ oshield:
- 15.8
port: 3334
sign: 1
sim_initial_position: 0
sim_velocity: 10.0
# Real hardware idles with the sample-stage shield closed; 15.5 is
# _oshield_ST_close()'s own umv() target (omny_sample_transfer_mixin.py).
# Starting at 0 (neither open -11.88 nor closed 15.5) always failed
# _otransfer_stage_to_ST()'s "oshield_current < 15" closed check.
sim_initial_position: 15.5
enabled: true
onFailure: buffer
readOnly: false
+86
View File
@@ -0,0 +1,86 @@
# Example BEC device-config entries for the SmarGon goniometer (via smargopolo).
#
# Per-axis architecture (like the smaract motors): each SCS axis is its own top-level
# motor sharing ONE SmargopoloController (singleton per host:port), plus a thin
# `smargon` coordinator for whole-goniometer ops. All entries must use the SAME
# host/port so they share the controller.
#
# The motor NAME (the top-level key: sgx, sgchi, ...) is yours to choose. The
# `scs_axis` field binds it to the fixed smargopolo axis (SHX/SHY/SHZ/CHI/PHI/OMEGA).
#
# Referencing is a deliberate operator action: after loading, run once
# dev.smargon.reference() # (or dev.sgx.controller.reference())
# --- Coordinator (whole-goniometer ops: reference/mode/move_scs/diagnostics) ------
smargon:
description: SmarGon coordinator (referencing, mode, coordinated moves)
deviceClass: csaxs_bec.devices.smargon.smargon.SmarGon
deviceConfig:
host: tt-smargopolo # cSAXS beamline smargopolo REST API
port: 3000 # REST API port (per maintainer). NB: :8080 is the web GUI, not the API
timeout: 5.0
deviceTags: [goniometer, smargon]
enabled: true
onFailure: retry
readOnly: false
readoutPriority: baseline
softwareTrigger: false
# --- SCS axis motors --------------------------------------------------------------
# Limits from the MX-lab reference client (resources_2/Smargon.py) -- VERIFY per build.
# NOTE: SHZ is not centred on 0 (home ~18); PHI is continuous (client wraps to
# [-180,180) -- open question whether smargopolo wraps server-side).
sgx:
description: SmarGon SHX (sample X)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig: {scs_axis: SHX, host: tt-smargopolo, port: 3000, limits: [-2, 2], tolerance: 0.001}
deviceTags: [goniometer, smargon]
enabled: true
onFailure: retry
readoutPriority: baseline
sgy:
description: SmarGon SHY (sample Y)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig: {scs_axis: SHY, host: tt-smargopolo, port: 3000, limits: [-2, 2], tolerance: 0.001}
deviceTags: [goniometer, smargon]
enabled: true
onFailure: retry
readoutPriority: baseline
sgz:
description: SmarGon SHZ (sample Z)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig: {scs_axis: SHZ, host: tt-smargopolo, port: 3000, limits: [10, 22], tolerance: 0.001}
deviceTags: [goniometer, smargon]
enabled: true
onFailure: retry
readoutPriority: baseline
sgchi:
description: SmarGon CHI (goniometer tilt)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig: {scs_axis: CHI, host: tt-smargopolo, port: 3000, limits: [0, 65], tolerance: 0.01}
deviceTags: [goniometer, smargon]
enabled: true
onFailure: retry
readoutPriority: baseline
sgphi:
description: SmarGon PHI (goniometer rotation)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig: {scs_axis: PHI, host: tt-smargopolo, port: 3000, limits: [-180, 180], tolerance: 0.01}
deviceTags: [goniometer, smargon]
enabled: true
onFailure: retry
readoutPriority: baseline
# --- Optional OMEGA rotation drive (only on builds that have it) -------------------
# sgomega:
# description: SmarGon OMEGA (rotation drive)
# deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
# deviceConfig: {scs_axis: OMEGA, host: tt-smargopolo, port: 3000, tolerance: 0.01}
# deviceTags: [goniometer, smargon]
# enabled: true
# onFailure: retry
# readoutPriority: baseline
@@ -0,0 +1,124 @@
# Self-contained laptop test session for the SmarGon device (SIMULATION).
#
# Per-axis motors (dev.sgx, dev.sgy, dev.sgz, dev.sgchi, dev.sgphi, dev.sgomega) +
# a `smargon` coordinator, all sharing one simulated controller, plus a sim monitor
# so scans record data. No smargopolo server / hardware needed.
#
# Load in the client:
# bec.config.update_session_with_file(
# '/Users/menzel/BEC_development/stack/csaxs_bec-smargon/csaxs_bec/device_configs/smargon_sim_session.yaml')
# Then: dev.smargon.reference(); umv(dev.sgx, 0.5); dev.wm('sg*')
#
# NOTE: update_session_with_file REPLACES the current session. PREREQUISITE: the
# running BEC must import csaxs_bec.devices.smargon (repoint the editable install at
# the smargon worktree; see RUNNING_BEC.md).
smargon:
description: SmarGon coordinator (SIMULATION)
deviceClass: csaxs_bec.devices.smargon.smargon.SmarGon
deviceConfig: { simulation: true, gradual_motion: true }
deviceTags: [goniometer, smargon, simulation]
enabled: true
onFailure: retry
readoutPriority: baseline
sgx:
description: SmarGon SHX (sim)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig:
{
scs_axis: SHX,
simulation: true,
gradual_motion: true,
limits: [-2, 2],
tolerance: 0.001,
}
deviceTags: [goniometer, smargon, simulation]
enabled: true
onFailure: retry
readoutPriority: baseline
sgy:
description: SmarGon SHY (sim)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig:
{
scs_axis: SHY,
simulation: true,
gradual_motion: true,
limits: [-2, 2],
tolerance: 0.001,
}
deviceTags: [goniometer, smargon, simulation]
enabled: true
onFailure: retry
readoutPriority: baseline
sgz:
description: SmarGon SHZ (sim)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig:
{
scs_axis: SHZ,
simulation: true,
gradual_motion: true,
limits: [10, 22],
tolerance: 0.001,
}
deviceTags: [goniometer, smargon, simulation]
enabled: true
onFailure: retry
readoutPriority: baseline
sgchi:
description: SmarGon CHI (sim)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig:
{
scs_axis: CHI,
simulation: true,
gradual_motion: true,
limits: [0, 65],
tolerance: 0.01,
}
deviceTags: [goniometer, smargon, simulation]
enabled: true
onFailure: retry
readoutPriority: baseline
sgphi:
description: SmarGon PHI (sim)
deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
deviceConfig:
{
scs_axis: PHI,
simulation: true,
gradual_motion: true,
limits: [-180, 180],
tolerance: 0.01,
}
deviceTags: [goniometer, smargon, simulation]
enabled: true
onFailure: retry
readoutPriority: baseline
# sgomega:
# description: SmarGon OMEGA (sim)
# deviceClass: csaxs_bec.devices.smargon.positioner.SmargopoloMotor
# deviceConfig: {scs_axis: OMEGA, simulation: true, gradual_motion: true, tolerance: 0.01}
# deviceTags: [goniometer, smargon, simulation]
# enabled: true
# onFailure: retry
# readoutPriority: baseline
sim_mon:
description: Simulated 1D monitor (readout for test scans over SmarGon axes)
deviceClass: ophyd_devices.SimMonitor
deviceConfig:
sim_init:
model: "GaussianModel"
deviceTags: [monitor, simulation]
enabled: true
onFailure: retry
readoutPriority: monitored
softwareTrigger: false
@@ -0,0 +1,26 @@
# TEMPORARY test config: a single IDS color camera (camera_id 41), used to
# manually verify the exposure/auto-gain controls added in
# docs/plans/ids-camera-manual-exposure.md against real hardware, without
# reconfiguring a production beamline. Named "cam_xeye" so the OMNY_XRayEye
# widget (which hardcodes that device name) can be pointed at it directly.
#
# Delete this file once real-hardware verification of the exposure/gain
# feature is done -- it is not meant to be loaded in a production session.
cam_xeye:
description: Test IDS color camera (ID 41) for exposure/auto-gain widget verification
deviceClass: csaxs_bec.devices.ids_cameras.ids_camera.IDSCamera
deviceConfig:
camera_id: 41
bits_per_pixel: 24
num_rotation_90: 0
transpose: false
force_monochrome: false
m_n_colormode: 1
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
userParameter:
pixel_calibration: 1.0
deviceTags:
- test_config
@@ -21,6 +21,7 @@ samx:
############################################################
########### OWIS LTM80 with micro-stepping #################
############################################################
# usually controller 18/19/20; Irun 1; Step F; both switches to the left (off); Adress 0
samy:
description: Owis motor stage samx
deviceClass: ophyd_devices.devices.psi_motor.EpicsUserMotorVME
+16 -16
View File
@@ -5,7 +5,7 @@ eyex:
prefix: X12SA-ES2-ES04
motor_resolution: 0.00125
base_velocity: 0.0625
velocity: 1
velocity: 5
backlash_distance: 0.125
acceleration: 0.2
user_offset_dir: 0
@@ -25,7 +25,7 @@ eyey:
prefix: X12SA-ES2-ES05
motor_resolution: 0.00125
base_velocity: 0.0625
velocity: 1
velocity: 5
backlash_distance: 0.125
acceleration: 0.2
user_offset_dir: 0
@@ -38,19 +38,19 @@ eyey:
softwareTrigger: false
cam_xeye:
description: Camera Xray eye ID1
deviceClass: csaxs_bec.devices.ids_cameras.ids_camera.IDSCamera
deviceConfig:
camera_id: 1
bits_per_pixel: 24
num_rotation_90: 2
transpose: true
force_monochrome: true
m_n_colormode: 1
enabled: true
onFailure: buffer
readOnly: false
readoutPriority: async
# cam_xeye:
# description: Camera Xray eye ID1
# deviceClass: csaxs_bec.devices.ids_cameras.ids_camera.IDSCamera
# deviceConfig:
# camera_id: 2
# bits_per_pixel: 24
# num_rotation_90: 2
# transpose: true
# force_monochrome: true
# m_n_colormode: 1
# enabled: true
# onFailure: buffer
# readOnly: false
# readoutPriority: async
# userParameter:
# pixel_calibration: 0.00044247787610619477 # mm/pixel (= 0.05 / 113)
@@ -247,6 +247,8 @@ class AlliedVisionAravisCamera(PSIDeviceBase):
def on_destroy(self):
"""Clean up resources when the device is destroyed."""
# Stop acquisition before disconnecting so the live thread cannot keep polling.
self.stop_live_mode()
self.cam.on_disconnect()
super().on_destroy()
@@ -21,7 +21,7 @@ DELAY PAIRS:
PULSE train for the MCS card. The MCS card needs one extra pulse to forward points.
DELAY CHANNELS:
- a = t0 + 2ms (2ms delay to allow the shutter to open)
- a = t0 + DEFAULT_SHUTTER_TO_OPEN_DELAY (to allow the shutter to open)
- b = a + 1us (short pulse)
- c = t0
- d = a + exp_time * burst_count
@@ -92,6 +92,81 @@ DEFAULT_IO_CONFIG: dict[AllChannelNames, ChannelConfig] = {
DEFAULT_TRIGGER_SOURCE: TRIGGERSOURCE = TRIGGERSOURCE.SINGLE_SHOT
# NOTE How long the shutter is given to open before the first gate of a burst.
# The shutter fires on `cd` at t0 and the acquisition on `ab` is held back by this
# much, so whatever the shutter still owes at that point is lost from the FIRST
# point of every line and from nothing else.
#
# Measured 2026-08-25 from the first-point deficit in four commissioning scans,
# against the 2e-3 that was in place when they were taken:
#
# scan exp_time first/near lost
# 324 15 ms 0.655 5.18 ms
# 254 20 ms 0.732 5.36 ms
# 411 50 ms 0.881 5.93 ms
# 450 50 ms 0.879 6.04 ms
#
# The deficit is a fixed time, not a fixed fraction: it varies by 17% across a
# 3.3x range of exposure while the fraction varies by 2.9x. That is what makes it
# a shutter opening late rather than anything dose-dependent. The same deficit
# appears on the integrated scattering -- a different detector behind a different
# gate -- so the cause sits upstream of both readout chains.
#
# 2e-3 already allowed, ~5.6e-3 still lost => the shutter needs about 7.6 ms,
# rounded up to 8 ms. The spread across the four scans is 0.9 ms, so the third
# digit is not meaningful, and overshooting costs only the difference in dead
# time at the start of each line.
# To re-measure after changing this, take the first point of each line over the
# mean of its next few neighbours, averaged over all lines, and multiply the
# shortfall by exp_time. That is the exposure still being lost, in seconds.
# Compare against its own neighbours rather than the whole-line mean: a scan
# whose intensity drifts along the line otherwise reports a deficit that is not
# there.
DEFAULT_SHUTTER_TO_OPEN_DELAY = 8e-3
# Guard rail for set_shutter_to_open_delay. The delay is paid once per line, so a
# fat-fingered value would quietly stretch every line rather than fail.
MAX_SHUTTER_TO_OPEN_DELAY = 50e-3
# NOTE How long it takes a move issued by the scan server to reach the motor.
#
# cont_grid issues the line move and fires the burst a few milliseconds later, on
# the assumption that the only thing in between is the acceleration ramp. It is
# not: the move first has to cross the scan server -> device server -> EPICS ->
# controller chain, and that latency does not shrink when the scan is slow while
# the computed acc_time does. The burst therefore starts before the stage.
#
# Measured 2026-08-25 from the position readback of five commissioning scans, as
# the distance the stage was behind the trigger grid once the lag stopped growing:
#
# scan exp_time v_cmd lag latency
# 254 20 ms 0.5 mm/s 29.7 pts 0.594 s
# 450 50 ms 0.1 mm/s 11.9 pts 0.595 s
# 324 15 ms 0.667 mm/s 41.5 pts 0.623 s
# 473 100 ms 0.065 mm/s 6.3 pts 0.630 s
# 411 50 ms 0.1 mm/s 12.7 pts 0.635 s
#
# Constant to +-3.5% across a 6.7x range of exposure and a 10x range of velocity,
# on two axes, which is what makes compensating with a constant sound. In scan
# 473, sampled 74 times per line, the lag appears in the first interval and then
# holds at 6.3-6.4 points for the rest of the line while the velocity sits at
# exactly the commanded 0.065 mm/s: a start-up offset, not a velocity error.
#
# Deliberately NOT solved by polling for motion. Observing the readback costs a
# round trip of this same ~0.6 s, so it would trade a systematic offset for a
# jitter of similar size -- and a constant offset displaces every line equally,
# where a varying one shears the image line by line and cannot be undone
# afterwards. The real fix is to trigger the DDG from the motor
# (scan_type: hardware_triggered), taking the round trip out of the timing chain
# altogether; this is the stopgap until then.
#
# Set to 0 to get the previous behaviour back without redeploying.
DEFAULT_MOVE_START_LATENCY = 0.6
# Guard rail for set_move_start_latency. Paid once per line, like the shutter
# delay, so an implausible value costs scan time rather than raising.
MAX_MOVE_START_LATENCY = 5.0
# NOTE Default readout times for each channel, can be adapted as needed.
# These values are relevant to calculate proper widths of the timing signals.
# They also define a minimum exposure time that can be used as they are subtracted
@@ -103,7 +178,7 @@ DEFAULT_READOUT_TIMES = {"ab": 2e-4, "cd": 2e-4, "ef": 2e-4, "gh": 2e-4} # 0.2
# If the trigger scheme changes, adapt the values here together with the README and
# PDF `trigger_scheme_ddg1_ddg2.pdf`.
DEFAULT_REFERENCES: list[tuple[LiteralChannels, CHANNELREFERENCE]] = [
("A", CHANNELREFERENCE.T0), # T0 + 2ms delay
("A", CHANNELREFERENCE.T0), # T0 + shutter-to-open delay
("B", CHANNELREFERENCE.A),
("C", CHANNELREFERENCE.T0), # T0
("D", CHANNELREFERENCE.C),
@@ -140,7 +215,11 @@ class DDG1(PSIDeviceBase, DelayGeneratorCSAXS):
"keep_shutter_open_during_scan",
"set_trigger",
"get_shutter_to_open_delay",
"set_shutter_to_open_delay",
"get_move_start_latency",
"set_move_start_latency",
"prepare_mcs_on_trigger",
"show_all",
]
# TODO Consider using the 'fsh' device instead.
@@ -173,7 +252,8 @@ class DDG1(PSIDeviceBase, DelayGeneratorCSAXS):
super().__init__(
name=name, prefix=prefix, scan_info=scan_info, device_manager=device_manager, **kwargs
)
self._shutter_to_open_delay = 2e-3
self._shutter_to_open_delay = DEFAULT_SHUTTER_TO_OPEN_DELAY
self._move_start_latency = DEFAULT_MOVE_START_LATENCY
self.device_manager = device_manager
self._poll_thread = threading.Thread(target=self._poll_event_status, daemon=True)
self._poll_thread_run_event = threading.Event()
@@ -244,11 +324,70 @@ class DDG1(PSIDeviceBase, DelayGeneratorCSAXS):
"""Get the current delay that is set to open the shutter before the exposure time."""
return self._shutter_to_open_delay
def get_move_start_latency(self) -> float:
"""Get the delay a scan should leave between issuing a line move and triggering."""
return self._move_start_latency
def set_move_start_latency(self, latency: float) -> None:
"""Set how long a scan waits after issuing a line move before it triggers.
This is not a property of the delay generator. It is the latency of the
scan server -> device server -> EPICS -> controller path, and it lives here
because this is where cont_grid already fetches its trigger timing. Too
small and the burst starts before the stage does, so the first points of
every line pile up at the line start; too large and the stage runs past
the start of the line before the first gate fires.
Set to 0 for the behaviour that preceded this parameter.
Args:
latency (float): Delay in seconds.
Raises:
ValueError: If negative or above MAX_MOVE_START_LATENCY.
"""
latency = float(latency)
if not 0 <= latency <= MAX_MOVE_START_LATENCY:
raise ValueError(
f"Move-start latency must be between 0 and {MAX_MOVE_START_LATENCY} s,"
f" got {latency}."
)
self._move_start_latency = latency
def set_shutter_to_open_delay(self, delay: float) -> None:
"""Set how long the shutter is given to open before the first gate fires.
Exists so the value can be converged without redeploying the device server.
Too small and the first point of every line is under-exposed; too large and
every line is stretched by the difference for nothing. To measure the
result, see the note at DEFAULT_SHUTTER_TO_OPEN_DELAY.
Args:
delay (float): Delay in seconds. 0 starts the acquisition together with
the shutter trigger.
Raises:
ValueError: If the delay is negative or above MAX_SHUTTER_TO_OPEN_DELAY.
"""
delay = float(delay)
if not 0 <= delay <= MAX_SHUTTER_TO_OPEN_DELAY:
raise ValueError(
f"Shutter-to-open delay must be between 0 and"
f" {MAX_SHUTTER_TO_OPEN_DELAY} s, got {delay}."
)
self._shutter_to_open_delay = delay
def keep_shutter_open_during_scan(self, open: True) -> None:
"""
Method to configure the delay generator for keeping the shutter open during a scans.
This means that the additional delay to open the shutter needs to be removed (2e-3)
from the timing of the signals.
This means that the additional delay to open the shutter needs to be removed
from the timing of the signals: a shutter that is already open owes nothing
to the first gate.
NOTE Switching this off restores DEFAULT_SHUTTER_TO_OPEN_DELAY, and so
discards any value set with set_shutter_to_open_delay. Tune with the setter
and leave this alone, or change the default if the tuned value is the one
the beamline should keep.
Args:
open (bool): If True, the shutter will be kept open during the scan.
@@ -257,7 +396,7 @@ class DDG1(PSIDeviceBase, DelayGeneratorCSAXS):
if open is True:
self._shutter_to_open_delay = 0
else:
self._shutter_to_open_delay = 2e-3
self._shutter_to_open_delay = DEFAULT_SHUTTER_TO_OPEN_DELAY
def on_stage(self) -> None:
"""
@@ -274,8 +413,10 @@ class DDG1(PSIDeviceBase, DelayGeneratorCSAXS):
- We check if any default burst parameters need to be set, and set them if needed.
- We calculate the burst pulse width based on the exposure time and frames_per_trigger.
- We set the burst_period and the shutter signal (delay pairs cd) to be
exposure_time * frames_per_trigger + 3ms (2ms for shutter to open, 1ms to close).
- We set the delay pairs ab to be 2ms delayed (to allow the shutter to open) with a width of 1us to trigger DDG2.
exposure_time * frames_per_trigger + twice the shutter-to-open delay
(once to open, once to close).
- We set the delay pairs ab to be delayed by the shutter-to-open delay (to
allow the shutter to open) with a width of 1us to trigger DDG2.
- We set the delay pairs ef to be triggered after the shutter closes with a width of 1us to trigger the MCS card.
- Finally, we add a short sleep to ensure that the IOC and DDG HW process the values properly.
"""
@@ -333,7 +474,7 @@ class DDG1(PSIDeviceBase, DelayGeneratorCSAXS):
self.burst_period.put(total_exposure_time)
# Trigger DDG2
# a = t0 + 2ms, b = a + 1us
# a = t0 + shutter-to-open delay, b = a + 1us
# a has reference to t0, b has reference to a
# AB is delayed by the shutter opening time, and the falling edge indicates the shutter has
# fully closed, it has to be considered as the blocking signal for the next acquisition to start.
@@ -343,7 +484,8 @@ class DDG1(PSIDeviceBase, DelayGeneratorCSAXS):
# Trigger shutter
# d = c/t0 + self._shutter_to_open_delay + exp_time * burst_count + 1ms
# c has reference to t0, d has reference to c
# Shutter opens without delay at t0, closes after exp_time * burst_count + 2ms (self._shutter_to_open_delay)
# Shutter opens without delay at t0, closes after
# exp_time * burst_count + self._shutter_to_open_delay
self.set_delay_pairs(channel="cd", delay=0, width=shutter_width)
self.set_delay_pairs(
Loaded 100 of 265 files, more files were not shown because too many files have changed in this diff. Show more