From 95d46c9d491176a05d16883399af9ad631530037 Mon Sep 17 00:00:00 2001 From: x01dc Date: Wed, 15 Jul 2026 14:17:51 +0200 Subject: [PATCH 1/5] feat(flomni,lamni): port tomo-queue backend to lamni via shared mixin Extract the tomo-queue/at-each-angle-hook backend (job proxy, hook registry, command-job dispatch) out of flomni.py into a new shared TomoQueueMixin (OMNY_shared/tomo_queue_mixin.py) so it can be reused instead of duplicated, avoiding the same param-name-mirror drift already flagged for tomo_params.py. Flomni's refactor is behavior- preserving; LamNI is the new consumer, with its own param names (tomo_circfov, lamni_stitch_x/y, ...) and no 180-degree/single-point concepts, since lamni doesn't have them. Also adds LamNI.tomo_scan_resume(), required for tomo_queue_execute()'s resume-after-crash path but not previously present. Verified via a new unit test suite (test_lamni_tomo_queue.py) plus live checks against the running simulated flomni and lamni deployments, documented in LamNI/AI_docs/TOMO_QUEUE_PORT.md. Co-Authored-By: Claude Sonnet 5 --- .../plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md | 109 +++ .../bec_ipython_client/plugins/LamNI/lamni.py | 127 ++- .../plugins/OMNY_shared/__init__.py | 0 .../plugins/OMNY_shared/tomo_queue_mixin.py | 758 ++++++++++++++++++ .../AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md | 125 +++ .../plugins/flomni/flomni.py | 741 +---------------- .../test_lamni_tomo_queue.py | 206 +++++ 7 files changed, 1341 insertions(+), 725 deletions(-) create mode 100644 csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md create mode 100644 csaxs_bec/bec_ipython_client/plugins/OMNY_shared/__init__.py create mode 100644 csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py create mode 100644 csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md create mode 100644 tests/tests_bec_ipython_client/test_lamni_tomo_queue.py diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md new file mode 100644 index 0000000..9531ad5 --- /dev/null +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md @@ -0,0 +1,109 @@ +# Tomo queue / at-each-angle hook backend — ported from flomni + +First `AI_docs/` entry for LamNI. Companion to flomni's +`AI_docs/TOMO_QUEUE_*.md` and `AI_docs/TOMO_QUEUE_COMMAND_JOBS_PLAN.md`, and +to `AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md` (flomni plugin), which first +identified this backend as flomni-only with no lamni equivalent. + +**Status:** Backend ported, unit-tested against a fake client, and verified +against the live simulated deployment for both setups (§ Verification below). +GUI (`TomoParamsWidget`) is a separate, later step and still flomni-gated. + +## What changed + +The tomo-queue backend (job list, resume-on-crash, named per-projection +hooks, command-job dispatch) used to live entirely in `flomni.py`. It's now a +shared `TomoQueueMixin` in +`csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py`, and +both `Flomni` and `LamNI` inherit it — see that module's docstring for the +full contract. This was a behavior-preserving refactor of `flomni.py` (moved +code, not changed logic); `LamNI` is the new consumer. + +`LamNI` now has, for the first time: +- `tomo_queue_add()` / `tomo_queue_add_command()` / `tomo_queue_delete()` / + `tomo_queue_move()` / `tomo_queue_clear()` / `tomo_queue_show()` / + `tomo_queue_execute()` +- `register_at_each_angle_hook()` / `unregister_at_each_angle_hook()` / + `list_at_each_angle_hooks()` / `at_each_angle_hook` property +- `tomo_scan_resume()` — **new method, not previously present in `lamni.py`**. + `tomo_queue_execute()` needs it to resume a crashed/interrupted job instead + of restarting from scratch; flomni already had this, lamni didn't. + Implemented identically to flomni's version (same `tomo_type`-based + dispatch: `subtomo_start`/`start_angle` for type 1, `projection_number` for + types 2/3 — lamni's `tomo_scan()` signature already matched flomni's + exactly, so this was a direct port, no design changes needed). + +`LamNI._at_each_angle()` now checks `self.at_each_angle_hook` first (same as +flomni's), falling through to its existing default body +(`tomo_scan_projection()` + `tomo_reconstruct()`) unchanged when no hook is +set. `MagLamNI`/`DataDrivenLamNI` (`extra_tomo.py`) override `_at_each_angle` +entirely and are unaffected by this change. + +## Parameter name mapping (`_TOMO_SCAN_PARAM_NAMES`) + +Lamni's existing global-var names are kept as-is — no renaming, since that +would break any in-flight session or persisted param snapshot. Where flomni +and lamni have overlapping concepts under different names, `LamNI`'s own +tuple uses lamni's names: + +| Concept | flomni | lamni | +|---|---|---| +| Field of view | `fovx` / `fovy` | `tomo_circfov` (single value — laminography FOV is circular, not rectangular) | +| Stitching | `stitch_x` / `stitch_y` | `lamni_stitch_x` / `lamni_stitch_y` | +| Piezo range | *(no equivalent)* | `lamni_piezo_range_x` / `lamni_piezo_range_y` | +| Manual shift | `manual_shift_y` only | `manual_shift_x` **and** `manual_shift_y` | +| 180°/360° toggle | `tomo_angle_range` | *(omitted — lamni is 360°-only, no property exists)* | +| Single-point acquisition | `single_point_instead_of_fermat_scan` / `single_point_random_shift_max` | *(omitted — no code path exists in lamni)* | +| Zero-deg reference per subtomo | `zero_deg_reference_at_each_subtomo` | *(omitted — no property exists)* | + +Everything else (`tomo_countingtime`, `tomo_shellstep`, `tomo_stitch_overlap`, +`ptycho_reconstruct_foldername`, `frames_per_trigger`, `tomo_type`, +`tomo_angle_stepsize`, `golden_ratio_bunch_size`, +`golden_max_number_of_projections`, +`golden_projections_at_0_deg_for_damage_estimation`, `corridor_size`, +`at_each_angle_hook`) uses identical names on both setups. + +## Assumption flagged for Mirko + +`_TOMO_QUEUE_MOVE_DEVICES` for lamni is `{mokev, idgap}` only — the same +beamline-wide devices flomni's allow-list includes. Flomni's allow-list also +has `ftray` (its sample-transfer tray) as a "sim-testable bonus" move target; +lamni has no equivalent device wired in, so it was left out. If lamni has its +own analogous device worth adding to the move allow-list, say so and it's a +one-line addition to `LamNI._TOMO_QUEUE_MOVE_DEVICES`. + +## Explicitly deferred + +- `TomoParamsWidget` GUI changes — still flomni-gated (`_check_flomni_available()`). +- `LamniWebpageGenerator` completion — separate, unrelated gap (see + `FLOMNI_TO_LAMNI_COMPARISON.md` §4). + +## Verification + +Offline suites (`pytest tests/tests_bec_ipython_client tests/tests_scans`): +121 passed, including all 60 pre-existing flomni angle-math regression tests +(`test_flomni_tomo_angles.py`) unchanged, plus the new 10-test +`test_lamni_tomo_queue.py` covering param snapshotting, fresh vs. resumed +execution, command-job move validation/dispatch, and hook registration/ +dispatch (with and without a registered hook) against a fake client. + +Live-sim checks (both run as plain scripts against the already-running +simulated deployment, not through `pytest_bec_e2e` — that plugin isn't +installed in this venv): + +- **lamni**: constructed a real `LamNI(bec)`, exercised + `tomo_queue_add`/`tomo_queue_add_command`/`tomo_queue_show`/ + `register_at_each_angle_hook`/`tomo_queue_delete` against the live global-var + store, confirmed the queue/hook state was restored to exactly what it was + before (no residue left for the interactive `bec --session lamni` session + sharing the same redis). No `tomo_queue_execute()`/real scan attempted here. +- **flomni**: reused `omny_e2e_tests/_bootstrap.py`'s `build_flomni()` + + `_queue_helpers.py`'s `SHORT_SCAN_PARAMS`/`FAST_STEPSIZE` (the same fixtures + `omny_e2e_tests/test_tomo_queue_command_jobs.py` uses) to confirm the + refactored `Flomni` is unchanged in practice: command-job add-time + validation, the `tomo_queue_actions` global var, a real `ftray` move via + `tomo_queue_execute()`, and a full real single-point tomo scan + (`tomo_queue_add()` → `tomo_queue_execute()` → `tomo_scan()` → + `_tomo_scan_at_angle()` → `_at_each_angle()` → registered hook) — 161 + projections, hook invoked for every one, job marked done, queue/progress + restored afterward. Ran twice for reproducibility. diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py index 3063ada..7743124 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py @@ -16,6 +16,7 @@ from csaxs_bec.bec_ipython_client.plugins.omny.omny_general_tools import ( PtychoReconstructor, TomoIDManager, ) +from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.tomo_queue_mixin import TomoQueueMixin from csaxs_bec.bec_ipython_client.plugins.LamNI.gui_tools import LamniGuiTools from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni_alignment_mixin import LamNIAlignmentMixin @@ -106,7 +107,80 @@ class _ProgressProxy: return self._load() -class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools): +class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools): + # Ordered set of all global-var-backed tomo scan parameters snapshotted + # by tomo_queue_add()/restored by tomo_queue_execute(). Lamni's own + # names for overlapping flomni concepts (tomo_circfov instead of + # fovx/fovy, lamni_stitch_x/y instead of stitch_x/y), no + # tomo_angle_range or single_point_* -- lamni is laminography: 360-only, + # no 180 mode, no single-point acquisition. + _TOMO_SCAN_PARAM_NAMES = ( + "tomo_countingtime", + "tomo_shellstep", + "tomo_circfov", + "lamni_stitch_x", + "lamni_stitch_y", + "tomo_stitch_overlap", + "ptycho_reconstruct_foldername", + "manual_shift_x", + "manual_shift_y", + "lamni_piezo_range_x", + "lamni_piezo_range_y", + "frames_per_trigger", + "tomo_type", + "tomo_angle_stepsize", + "golden_ratio_bunch_size", + "golden_max_number_of_projections", + "golden_projections_at_0_deg_for_damage_estimation", + "corridor_size", + "at_each_angle_hook", + ) + + # Curated allow-list for the "move" queue action -- same beamline-wide + # devices as flomni's (mokev, idgap). Lamni has no equivalent of + # flomni's ftray (sample-transfer tray) sim-testable bonus target. + _TOMO_QUEUE_MOVE_DEVICES = { + "mokev": {"label": "Energy", "unit": "keV"}, + "idgap": {"label": "Undulator gap", "unit": "mm"}, + } + + # Named-action registry for tomo-queue command jobs -- identical to + # flomni's, since both actions (move, optimize_idgap) are beamline-wide, + # not flomni-specific hardware. See TomoQueueMixin._queue_action_move()/ + # optimize_idgap() for the implementations. + _TOMO_QUEUE_ACTIONS = { + "move": { + "func": "_queue_action_move", + "idempotent": True, # absolute moves re-run harmlessly + "help": "Move device(s) to absolute position(s).", + "params": {"positions": {"type": "device_positions"}}, + }, + "optimize_idgap": { + "func": "optimize_idgap", + "idempotent": True, + "help": "Scan idgap over a search range and move to the peak. STUB: no-op.", + "params": { + "search_range": { + "type": "float", + "default": 0.5, + "min": 0.0, + "max": 2.0, + "unit": "mm", + "label": "Search range", + } + }, + }, + } + + def _describe_tomo_job_line(self, job: dict) -> str: + """lamni's tomo_queue_show() one-line summary: circfov instead of + fov, no range= (lamni is 360-only).""" + p = job["params"] + return ( + f"type={p['tomo_type']} circfov={p['tomo_circfov']}um " + f"step={p['tomo_shellstep']}um ctime={p['tomo_countingtime']}s" + ) + def __init__(self, client): super().__init__() self.client = client @@ -142,6 +216,7 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools): # tomo_scan()); use tomo_progress_reset() to explicitly clear stale # progress without starting a new scan. self._progress_proxy = _ProgressProxy(self.client) + self._init_tomo_queue() # ------------------------------------------------------------------ # Special angles @@ -683,6 +758,19 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools): ) def _at_each_angle(self, angle: float) -> None: + hook_name = self.at_each_angle_hook + if hook_name: + hook = self._at_each_angle_hooks.get(hook_name) + if hook is None: + raise ValueError( + f"at_each_angle_hook '{hook_name}' is not registered in this " + "session. Hooks are session-only and do not survive a kernel " + f"restart -- call register_at_each_angle_hook({hook_name!r}, " + ") again, then re-run tomo_queue_execute() to resume." + ) + hook(self, angle) + return + self.tomo_scan_projection(angle) self.tomo_reconstruct() @@ -1002,6 +1090,43 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools): f" {self._format_duration(self.progress.get('accumulated_idle_time', 0.0))}" ) + def tomo_scan_resume(self) -> None: + """Resume a tomo_scan() that crashed or was interrupted, picking up + automatically from wherever ``progress`` last reported -- no need to + read the last subtomo/angle/projection off the progress printout by + hand and pass it to tomo_scan() yourself. + + Re-attempts the exact angle/projection that was in progress at the + moment of interruption, rather than skipping ahead to the next one: + from here we can't be sure whether that last point was actually + acquired before the crash, and re-acquiring one extra projection is + harmless. + + Works standalone for a normal tomo_scan(), independent of the tomo + queue. tomo_queue_execute() (see TomoQueueMixin) also calls this + internally when resuming a job that previously failed partway + through, so a failed queue job picks up mid-scan rather than + restarting from subtomo 1 / angle 0. + """ + if self.progress.get("tomo_start_time") is None: + print("No tomo scan in progress to resume -- nothing to do.") + return + + if self.tomo_type == 1: + subtomo_start = self.progress["subtomo"] + start_angle = self.progress["angle"] + if subtomo_start < 1: + print("No tomo scan in progress to resume -- nothing to do.") + return + print(f"Resuming tomo scan at subtomo {subtomo_start}, angle {start_angle:.3f} deg.") + self.tomo_scan(subtomo_start=subtomo_start, start_angle=start_angle) + elif self.tomo_type in (2, 3): + projection_number = self.progress["projection"] + print(f"Resuming tomo scan at projection {projection_number}.") + self.tomo_scan(projection_number=projection_number) + else: + raise ValueError(f"Unknown tomo_type: {self.tomo_type}.") + # ------------------------------------------------------------------ # Parameter display and interactive update # ------------------------------------------------------------------ diff --git a/csaxs_bec/bec_ipython_client/plugins/OMNY_shared/__init__.py b/csaxs_bec/bec_ipython_client/plugins/OMNY_shared/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py b/csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py new file mode 100644 index 0000000..6465b6a --- /dev/null +++ b/csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py @@ -0,0 +1,758 @@ +""" +Shared tomo-queue / at-each-angle-hook backend for cSAXS ptychographic +tomography setups. + +Extracted from flomni.py, where this logic (job bookkeeping, resume-on-crash, +named per-projection hooks, command-job dispatch) was originally written +entirely generic over "which parameters/actions this setup has" -- only the +*names* of the snapshotted params and the allow-listed command-job actions +are setup-specific. Keeping one copy here instead of one per setup avoids the +same param-name-mirror drift problem already flagged (see tomo_params.py's +QUEUE_PARAM_NAMES) for a second, backend-side pair of mirrors. + +Consumers (e.g. ``Flomni``, ``LamNI``) must: + - inherit from ``TomoQueueMixin`` + - define class attributes ``_TOMO_SCAN_PARAM_NAMES`` (tuple of property + names a ``tomo_queue_add()`` job snapshots), ``_TOMO_QUEUE_ACTIONS`` + (command-job action registry) and ``_TOMO_QUEUE_MOVE_DEVICES`` (move + allow-list) + - call ``self._init_tomo_queue()`` once from ``__init__``, after + ``self.client`` is set + - optionally override ``_describe_tomo_job_line()`` to reference their + own param names in ``tomo_queue_show()``'s one-line-per-job summary + - implement their own ``tomo_scan()``/``tomo_scan_resume()`` (dispatched + generically by ``tomo_queue_execute()``) and their own ``_at_each_angle()`` + (which should consult ``self.at_each_angle_hook``/``self._at_each_angle_hooks`` + the same way, but is setup-specific enough -- e.g. flomni's legacy + ``flomni_at_each_angle`` builtins hook, single-point acquisition -- that + it stays out of this mixin) +""" + +from __future__ import annotations + +import builtins +import datetime +import inspect +import json +import uuid + +from typeguard import check_type + +if builtins.__dict__.get("bec") is not None: + dev = builtins.__dict__.get("dev") + umv = builtins.__dict__.get("umv") + + +class TomoQueueError(Exception): + """Raised for tomo-queue/command-job errors not specific to one setup.""" + + +class _TomoQueueProxy: + """List-like proxy that persists a queue of tomo parameter snapshots as a + BEC global variable, mirroring the pattern used by ``_ProgressProxy``. + + Each entry is a plain dict: ``{"label": ..., "params": {...}, "status": + ..., "added_at": ...}`` (or, for a command job, ``"steps"`` instead of + ``"params"``), where ``params`` holds a snapshot of all global-var-backed + tomo scan parameters (see the owning class's ``_TOMO_SCAN_PARAM_NAMES``). + Stored as a single list under ``tomo_queue`` so the queue is visible from + any BEC client session via ``client.get_global_var("tomo_queue")`` and + survives a kernel restart. + """ + + _GLOBAL_VAR_KEY = "tomo_queue" + + def __init__(self, client): + self._client = client + + def _load(self) -> list: + val = self._client.get_global_var(self._GLOBAL_VAR_KEY) + if val is None: + return [] + return val + + def _save(self, jobs: list) -> None: + self._client.set_global_var(self._GLOBAL_VAR_KEY, jobs) + + def as_list(self) -> list: + """Return a plain copy of the current queue.""" + return self._load() + + def append(self, job: dict) -> int: + jobs = self._load() + jobs.append(job) + self._save(jobs) + return len(jobs) - 1 + + def pop(self, index: int) -> dict: + jobs = self._load() + job = jobs.pop(index) + self._save(jobs) + return job + + def update(self, index: int, **kwargs) -> None: + jobs = self._load() + jobs[index].update(kwargs) + self._save(jobs) + + def update_by_id(self, job_id: str, **kwargs) -> bool: + """Update the job whose ``id`` matches ``job_id``. + + Returns True if a matching job was found and updated, False otherwise + (e.g. the job was deleted concurrently). Addressing a job by its stable + id rather than its list index makes status writes safe against queue + edits made while the queue runs: a reorder, or a delete of some *other* + job, can no longer misdirect a status write to the wrong entry. + """ + jobs = self._load() + for job in jobs: + if job.get("id") == job_id: + job.update(kwargs) + self._save(jobs) + return True + return False + + def move(self, index: int, new_index: int) -> None: + """Move the job at ``index`` to ``new_index``, shifting the rest. + + Pure list mechanics -- no status checks. Callers (e.g. + ``tomo_queue_move``) are responsible for deciding whether a given + move is allowed (running-job floor, movable statuses, ...). + """ + jobs = self._load() + job = jobs.pop(index) + jobs.insert(new_index, job) + self._save(jobs) + + def ensure_ids(self) -> list: + """Assign a stable id to any job that lacks one and persist the result. + + Back-compat shim for queues written before jobs carried an ``id`` + field. Called once at the top of ``tomo_queue_execute()`` so the + pick-next executor can address every job by id. Returns the + (possibly healed) job list. + """ + jobs = self._load() + changed = False + for job in jobs: + if not job.get("id"): + job["id"] = uuid.uuid4().hex + changed = True + if changed: + self._save(jobs) + return jobs + + def clear(self) -> None: + self._save([]) + + def __len__(self) -> int: + return len(self._load()) + + def __getitem__(self, index): + return self._load()[index] + + +class _GlobalVarParam: + """Descriptor for a parameter backed by a BEC global variable. + + Consolidates the boilerplate getter/setter pairs that read from and write + to the BEC global-var store. The value persists across BEC client restarts + (instance attributes do not), and is readable from other client sessions + via ``client.get_global_var()``. + + The global-var key is taken from the attribute name the descriptor is + assigned to (see ``__set_name__``), so it always matches the property + name. + + Parameters + ---------- + default + Value returned by the getter when the global var is unset (``None``). + type_ + Optional expected type. When given, the setter validates ``val`` with + ``typeguard.check_type`` before storing. + """ + + def __init__(self, default, type_=None): + self._default = default + self._type = type_ + + def __set_name__(self, owner, name): + self._key = name + + def __get__(self, obj, objtype=None): + if obj is None: + return self + val = obj.client.get_global_var(self._key) + if val is None: + return self._default + return val + + def __set__(self, obj, val): + if self._type is not None: + check_type(val, self._type) + obj.client.set_global_var(self._key, val) + + +class TomoQueueMixin: + """Generic tomo-queue / at-each-angle-hook backend, see module docstring.""" + + # Subclasses must override all three of these. + _TOMO_SCAN_PARAM_NAMES: tuple = () + _TOMO_QUEUE_ACTIONS: dict = {} + _TOMO_QUEUE_MOVE_DEVICES: dict = {} + + # Name of a registered at_each_angle hook (see register_at_each_angle_hook()), + # or None for the default per-projection behaviour. + at_each_angle_hook = _GlobalVarParam(None) + + def _init_tomo_queue(self) -> None: + """Call once from ``__init__``, after ``self.client`` is set. + + Deliberately does not reset the tomo_queue global var itself (jobs + must survive a kernel restart, exactly like tomo_progress) -- only + the session-only hook registry and the discoverability global vars + are (re)initialized here. + """ + # Session-only (not global-var-backed): the hooks themselves are + # Python callables, which can't be persisted the way params/queue + # jobs are. Only the *name* selecting one is ever snapshotted (see + # at_each_angle_hook / register_at_each_angle_hook()). + self._at_each_angle_hooks: dict = {} + self._tomo_queue_proxy = _TomoQueueProxy(self.client) + # Published so a future GUI never needs a hardcoded mirror of the + # action registry -- see _describe_tomo_queue_actions(). + self.client.set_global_var("tomo_queue_actions", self._describe_tomo_queue_actions()) + # Reset to empty on every init -- hooks are session-only (see + # self._at_each_angle_hooks above), so a stale list published by a + # since-ended session must not linger and look registered. + self._publish_at_each_angle_hooks() + + # ── at_each_angle hook registry ───────────────────────────────────────── + + def register_at_each_angle_hook(self, name: str, func) -> None: + """Register a per-projection hook function under ``name``, so a + tomo-queue job can select it by name via the ``at_each_angle_hook`` + parameter and have it run automatically. + + ``func`` is called as ``func(self, angle)`` at every projection angle. + + Hooks are **session-only**: they live in this instance's memory, not + in a global var, so they do NOT survive a kernel restart. A queued + job that references a hook by name raises a clear error at execution + time if the name isn't registered in the session running the queue -- + re-run this registration call, then resume ``tomo_queue_execute()``. + + Example -- record a projection, insert a polarizer, record again:: + + def polarizer_modulation(setup, angle): + setup.tomo_scan_projection(angle, _internal=True) + umv(dev.polarizer, "in") + setup.tomo_scan_projection(angle, _internal=True) + umv(dev.polarizer, "out") + + setup.register_at_each_angle_hook("polarizer_mod", polarizer_modulation) + setup.at_each_angle_hook = "polarizer_mod" + setup.tomo_queue_add(label="polarizer run") + setup.at_each_angle_hook = None # back to default for the next job + """ + if not callable(func): + raise TypeError(f"register_at_each_angle_hook: {func!r} is not callable.") + self._at_each_angle_hooks[name] = func + self._publish_at_each_angle_hooks() + print(f"Registered at_each_angle hook '{name}'.") + + def unregister_at_each_angle_hook(self, name: str) -> None: + """Remove a previously registered at_each_angle hook by name.""" + if self._at_each_angle_hooks.pop(name, None) is None: + print(f"No at_each_angle hook named '{name}' is registered.") + else: + self._publish_at_each_angle_hooks() + print(f"Unregistered at_each_angle hook '{name}'.") + + def _publish_at_each_angle_hooks(self) -> None: + """Publish the names of all currently registered at_each_angle + hooks as the ``tomo_at_each_angle_hooks`` global var (a plain list + of strings) -- so a GUI (or any other client) can build a dropdown + from it directly. Unlike ``tomo_queue_actions`` this list changes at + runtime, whenever register_/unregister_at_each_angle_hook() is called. + """ + self.client.set_global_var("tomo_at_each_angle_hooks", sorted(self._at_each_angle_hooks)) + + def list_at_each_angle_hooks(self) -> list: + """Print and return the names of all at_each_angle hooks registered + in this session. The active one (``self.at_each_angle_hook``, if + any) is marked.""" + names = sorted(self._at_each_angle_hooks) + if not names: + print("No at_each_angle hooks registered.") + else: + for name in names: + marker = " (active)" if name == self.at_each_angle_hook else "" + print(f" {name}{marker}") + return names + + def _describe_active_hook(self) -> str | None: + """Human-readable description of ``self.at_each_angle_hook`` for + display in tomo_parameters()/scilog/the PDF report: the name, or + the name flagged as unregistered if it doesn't (or no longer) + resolve in this session. Returns None if no hook is set at all. + """ + hook = self.at_each_angle_hook + if not hook: + return None + if hook in self._at_each_angle_hooks: + return hook + return f"{hook} (NOT registered in this session)" + + def _active_hook_source(self) -> str | None: + """Source code of the currently active at_each_angle hook, for + inclusion in the scilog/PDF record. Returns None if there's no + active hook, it isn't registered in this session, or its source + can't be retrieved (e.g. it wasn't defined via a normal `def`/ + interactive cell). + """ + hook_func = self._at_each_angle_hooks.get(self.at_each_angle_hook) + if hook_func is None: + return None + try: + return inspect.getsource(hook_func) + except (OSError, TypeError): + return None + + # ── command-job action registry / dispatch ────────────────────────────── + + def _validate_action_kwargs(self, action_name: str, kwargs: dict) -> None: + """Add-time schema validation for a command-job step's kwargs. + + Ergonomic, not the safety boundary: catches a typo or an out-of-range + value now instead of at 2am. Each action method re-validates its own + arguments against live hardware at execution time -- that's the + actual safety boundary, since the schema and the hardware can + disagree (limits changed; a value arrived via the CLI, bypassing + this). + """ + spec = self._TOMO_QUEUE_ACTIONS.get(action_name) + if spec is None: + raise ValueError(f"Unknown queue action '{action_name}'.") + schema = spec["params"] + for key in kwargs: + if key not in schema: + raise ValueError(f"Action '{action_name}' has no parameter '{key}'.") + for name, field in schema.items(): + if name not in kwargs: + continue + value = kwargs[name] + field_type = field.get("type") + if field_type == "float": + if not isinstance(value, (int, float)) or isinstance(value, bool): + raise ValueError(f"{action_name}.{name} must be a number, got {value!r}.") + lo, hi = field.get("min"), field.get("max") + if lo is not None and value < lo: + raise ValueError(f"{action_name}.{name}={value} is below the minimum {lo}.") + if hi is not None and value > hi: + raise ValueError(f"{action_name}.{name}={value} is above the maximum {hi}.") + elif field_type == "device_positions": + if not isinstance(value, dict) or not value: + raise ValueError( + f"{action_name}.{name} must be a non-empty {{device: target}} dict." + ) + for device_name in value: + if device_name not in self._TOMO_QUEUE_MOVE_DEVICES: + raise ValueError( + f"Queue move: device '{device_name}' is not queue-movable." + ) + + def _queue_action_move(self, positions: dict) -> None: + """positions: {device_name: absolute_target, ...}. Absolute umv only. + + Re-checks the allow-list at execution time (not just add time, see + _validate_action_kwargs) -- the actual safety boundary, since a value + could have arrived via the CLI, bypassing tomo_queue_add_command's + validation. + """ + args = [] + for name, target in positions.items(): + if name not in self._TOMO_QUEUE_MOVE_DEVICES: + raise ValueError(f"Queue move: device '{name}' is not queue-movable.") + if name not in dev: + raise ValueError(f"Queue move: unknown device '{name}'.") + args += [dev[name], target] + umv(*args) + + def optimize_idgap(self, search_range: float = 0.5) -> None: + """Scan idgap over +/- search_range around its current position and + move to the peak of . + + STUB -- not implemented. The peak-detection signal/device is not yet + decided (see flomni's AI_docs/TOMO_QUEUE_COMMAND_JOBS_PLAN.md section + 10). This is the ONLY place that needs to change to make this a real + action: the registry entry, resume policy, and (eventually) GUI + wiring already treat it as a normal idempotent compound action, so + dropping in the real scan-and-move-to-peak logic here (or swapping + which device/signal it reads) is a self-contained change. + """ + print( + f"optimize_idgap(search_range={search_range}): not yet implemented -- " + "peak-detection signal not yet decided. No-op." + ) + + def _describe_tomo_queue_actions(self) -> dict: + """Serializable description of the action registry + move allow-list. + + Published as the ``tomo_queue_actions`` global var (see + _init_tomo_queue()) so the GUI (or any other client) can build its + dropdowns/forms from it directly -- no hardcoded mirror to drift. + """ + return { + "actions": { + name: { + "idempotent": spec["idempotent"], + "help": spec["help"], + "params": spec["params"], + } + for name, spec in self._TOMO_QUEUE_ACTIONS.items() + }, + "move_devices": dict(self._TOMO_QUEUE_MOVE_DEVICES), + } + + # ── job list management ────────────────────────────────────────────── + + def tomo_queue_add(self, label: str = None) -> int: + """Snapshot the currently set tomo parameters and append them as a + new job to the tomo queue (persisted, survives a kernel restart). + + Args: + label: Optional name for the job, shown by tomo_queue_show(). + Defaults to "job_". + + Returns: + The index of the newly added job. + """ + params = {name: getattr(self, name) for name in self._TOMO_SCAN_PARAM_NAMES} + index = len(self._tomo_queue_proxy) + job = { + "kind": "tomo", + "id": uuid.uuid4().hex, + "label": label or f"job_{index + 1}", + "params": params, + "status": "pending", + "added_at": datetime.datetime.now().isoformat(), + } + self._tomo_queue_proxy.append(job) + print(f"Added tomo queue job #{index} ({job['label']}).") + return index + + def tomo_queue_add_command(self, steps, label: str = None, idempotent: bool = None) -> int: + """Append a "command" job -- an ordered list of named-registry actions + that reconfigure the beamline (move a device, ...) -- to the tomo + queue. + + Unlike tomo_queue_add(), this does not snapshot the current tomo scan + parameters; a command job has no "params" key at all. + + Args: + steps: A single {"action": ..., "kwargs": {...}} dict, or a list + of them, run in sequence within this one job. + label: Optional name for the job, shown by tomo_queue_show(). + Defaults to "job_". + idempotent: Overrides the default of "every step's registry + idempotent flag, ANDed together". Set this explicitly if the + registry defaults don't capture whether this particular + sequence is safe to blindly re-run after a crash. + + Returns: + The index of the newly added job. + """ + if isinstance(steps, dict): + steps = [steps] + if not steps: + raise ValueError("tomo_queue_add_command: at least one step is required.") + + resolved_steps = [] + all_idempotent = True + for step in steps: + action_name = step.get("action") + kwargs = step.get("kwargs", {}) + self._validate_action_kwargs(action_name, kwargs) + all_idempotent = all_idempotent and self._TOMO_QUEUE_ACTIONS[action_name]["idempotent"] + resolved_steps.append({"action": action_name, "kwargs": kwargs}) + + try: + json.dumps(resolved_steps) + except (TypeError, ValueError) as exc: + raise ValueError(f"Command job steps must be JSON-serializable: {exc}") from exc + + index = len(self._tomo_queue_proxy) + job = { + "kind": "command", + "id": uuid.uuid4().hex, + "label": label or f"job_{index + 1}", + "steps": resolved_steps, + "idempotent": all_idempotent if idempotent is None else idempotent, + "status": "pending", + "added_at": datetime.datetime.now().isoformat(), + } + self._tomo_queue_proxy.append(job) + print(f"Added command queue job #{index} ({job['label']}).") + return index + + def tomo_queue_delete(self, *indices: int) -> None: + """Delete one or more jobs from the tomo queue by index. + + Accepts any number of indices, e.g. ``tomo_queue_delete(2, 5)`` to + drop several jobs in one call. All indices are resolved against the + queue as it currently stands and deleted highest-index-first, so + passing several indices in any order is safe and won't shift the + meaning of the indices still to be deleted. + """ + if not indices: + print("No index given.") + return + for index in sorted(set(indices), reverse=True): + job = self._tomo_queue_proxy.pop(index) + print(f"Deleted tomo queue job #{index} ({job['label']}).") + + def tomo_queue_move(self, index: int, new_index: int) -> None: + """Move a pending tomo queue job to a new position. + + Only ``pending`` jobs can be reordered -- a ``running``, + ``incomplete``, or ``done`` job stays put. ``incomplete`` is a floor + exactly like ``running``: tomo_queue_execute()'s pick-next loop + always resumes whichever of the two it finds first, regardless of + list order, so a pending job moved "ahead" of one in the list would + visually suggest a run order that isn't real. A move is also + refused if it would place the job at or above the position of any + currently ``running``/``incomplete`` job. + + Args: + index: Current index of the job to move. + new_index: Index to move it to (list-insert semantics: the rest + of the queue shifts to make room). + """ + jobs = self._tomo_queue_proxy.as_list() + if not 0 <= index < len(jobs): + raise ValueError(f"tomo_queue_move: index {index} out of range.") + if not 0 <= new_index < len(jobs): + raise ValueError(f"tomo_queue_move: new_index {new_index} out of range.") + job = jobs[index] + if job.get("status", "pending") != "pending": + raise ValueError( + f"tomo_queue_move: job #{index} is '{job.get('status')}' -- only " + "pending jobs can be reordered." + ) + floor_indices = [ + i for i, j in enumerate(jobs) if j.get("status") in ("running", "incomplete") + ] + if floor_indices and new_index <= max(floor_indices): + raise ValueError( + f"tomo_queue_move: can't move to position {new_index} -- job " + f"#{max(floor_indices)} is running/incomplete and is a hard floor " + "for reordering." + ) + self._tomo_queue_proxy.move(index, new_index) + print(f"Moved tomo queue job #{index} -> #{new_index} ({job['label']}).") + + def tomo_queue_clear(self) -> None: + """Empty the tomo queue.""" + self._tomo_queue_proxy.clear() + print("Tomo queue cleared.") + + def _describe_tomo_job_line(self, job: dict) -> str: + """Per-setup one-line summary of a "tomo" job's snapshotted params, + for tomo_queue_show(). Override to reference this setup's actual + param names (e.g. flomni's fovx/fovy vs lamni's tomo_circfov). + """ + p = job["params"] + return ( + f"type={p.get('tomo_type')} step={p.get('tomo_shellstep')}um " + f"ctime={p.get('tomo_countingtime')}s" + ) + + def tomo_queue_show(self) -> list: + """Print and return the current tomo queue, one line per job.""" + jobs = self._tomo_queue_proxy.as_list() + if not jobs: + print("Tomo queue is empty.") + return jobs + for i, job in enumerate(jobs): + if job.get("kind", "tomo") == "command": + steps_str = " > ".join( + f"{step['action']}{step['kwargs']}" if step["kwargs"] else step["action"] + for step in job["steps"] + ) + idem = "idem" if job.get("idempotent") else "NOT idem" + print(f"[{i}] {job['status']:>10s} {job['label']:<20s} CMD {steps_str} [{idem}]") + continue + print( + f"[{i}] {job['status']:>10s} {job['label']:<20s} {self._describe_tomo_job_line(job)}" + ) + return jobs + + def tomo_queue_execute(self, start_index: int = 0) -> None: + """Run all pending tomo queue jobs in sequence, on the current sample. + + For each job, restores its snapshotted parameters onto the live + properties (exactly as if set by hand) and then calls + ``tomo_scan()`` -- or, for a job that didn't run to completion last + time, ``tomo_scan_resume()``, so it picks back up mid-scan instead + of restarting from subtomo 1 / angle 0. Jobs already marked "done" + are skipped, so simply calling tomo_queue_execute() again resumes + from where it stopped (e.g. after fixing a hardware issue). + + Pick-next execution model + ------------------------- + The queue is re-read from its global var at the start of every job + (not snapshotted once up front), and each job is addressed by its + stable ``id``, not by its list index. This is what makes it safe to + edit the queue *while it runs*: appending a job, deleting a pending + job, or reordering the pending tail all take effect at the next job + boundary, and a reorder/delete can never misdirect a status write to + the wrong entry (an index-based loop could). Legacy jobs written + before the ``id`` field existed are healed once up front by + ensure_ids(). + + Resume-before-fresh ordering: if any job is in a resumable state, it + is always run before any fresh "pending" job, regardless of list + order. This preserves the single-resumable-job invariant -- tomo + progress lives in one shared ``progress`` global var, so starting a + fresh tomo_scan() before resuming an interrupted one would overwrite + the very state tomo_scan_resume() needs. + + A job is considered not-yet-complete (and so gets resumed rather + than restarted) if its status is "incomplete" (a Python exception + was caught and execution stopped cleanly) OR "running" (the queue + process itself died -- killed kernel, dropped connection, power + loss, ... -- before it had a chance to record anything). + + If you've already manually called ``tomo_scan_resume()`` yourself + to recover from a crash (bypassing the queue), that scan is now + actually finished even though the queue still has the job marked + "incomplete" or "running" -- mark it done yourself before calling + this again, or it will be re-run from scratch. Address the job by + its id (indices can shift if the queue is reordered): + self._tomo_queue_proxy.update_by_id(job_id, status="done") + + Command jobs (job["kind"] == "command", added by + tomo_queue_add_command()) are dispatched to _run_command_job() + instead of tomo_scan()/tomo_scan_resume() -- see its docstring for + their own resume policy. A job with no "kind" key is treated as + "tomo" (back-compat for queues persisted before command jobs + existed). + + Args: + start_index: Position in the *current* queue order below which + fresh "pending" jobs are ignored. Defaults to 0. Resumable + jobs are always run regardless of start_index (a crashed + job must be recovered wherever it sits). Jobs already marked + "done" are skipped automatically either way. + """ + self._tomo_queue_proxy.ensure_ids() + jobs = self._tomo_queue_proxy.as_list() + if not jobs: + print("Tomo queue is empty.") + return + + runnable = [ + job + for idx, job in enumerate(jobs) + if job["status"] != "done" + and (job["status"] in ("incomplete", "running") or idx >= start_index) + ] + if not runnable: + print("No pending tomo queue jobs to run.") + return + + if not self.OMNYTools.yesno( + f"Starting automatic execution of {len(runnable)} queued tomo scan(s) on" + f" sample '{self.sample_name}'. OK?", + "y", + ): + print("Aborted.") + return + + while True: + jobs = self._tomo_queue_proxy.as_list() + # Resume any in-flight/interrupted job first, before starting any + # fresh one, so the shared progress var is never clobbered. Search + # the whole list (not the start_index slice): a crashed job must be + # recovered wherever it sits. + job = next((j for j in jobs if j["status"] in ("incomplete", "running")), None) + if job is None: + job = next( + ( + j + for idx, j in enumerate(jobs) + if idx >= start_index and j["status"] == "pending" + ), + None, + ) + if job is None: + break + + job_id = job["id"] + label = job["label"] + kind = job.get("kind", "tomo") + resume_job = job["status"] in ("incomplete", "running") + + print(f"\n=== Tomo queue job: {label} ===") + + self._tomo_queue_proxy.update_by_id(job_id, status="running") + try: + if kind == "command": + self._run_command_job(job, resume_job) + else: + for name, value in job["params"].items(): + setattr(self, name, value) + if resume_job: + self.tomo_scan_resume() + else: + self.tomo_scan() + except Exception as exc: + self._tomo_queue_proxy.update_by_id(job_id, status="incomplete") + print(f"Tomo queue job '{label}' did not complete: {exc}") + print( + "Queue paused. Fix the issue and call tomo_queue_execute() " + "again to resume from this job." + ) + raise + self._tomo_queue_proxy.update_by_id(job_id, status="done") + + print("\nTomo queue finished -- all jobs done.") + + def _run_command_job(self, job: dict, resume_job: bool) -> None: + """Run (or resume) a "command" job's steps in order. + + No per-step resume cursor -- deliberately: tracking which step of a + multi-step job completed means more persisted state and more edge + cases than this warrants. Instead the job-level "idempotent" flag + decides: + + - idempotent: silently re-run every step from the top. Harmless -- + absolute moves and in/out actions re-run cleanly. + - not idempotent: prompt the operator rather than silently re-run, + which could be actively wrong for a relative/stateful step. "no" + leaves the steps un-run; the caller (tomo_queue_execute) still + marks the job "done" either way, matching the same edge case tomo + jobs have -- a job that actually finished but crashed before its + status hit "done". + """ + label = job["label"] + if resume_job: + if job.get("idempotent"): + print( + f"Command job '{label}' may have partially run before the crash " + "-- idempotent, re-running from the top." + ) + else: + if not self.OMNYTools.yesno( + f"Command job '{label}' may have partially run before the crash " + "(not every step is idempotent). Re-run it from the top?", + "n", + ): + print(f"Skipping '{label}' -- it will be marked done without re-running.") + return + for step in job["steps"]: + spec = self._TOMO_QUEUE_ACTIONS[step["action"]] + method = getattr(self, spec["func"]) + method(**step["kwargs"]) diff --git a/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md b/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md new file mode 100644 index 0000000..b66f2ca --- /dev/null +++ b/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md @@ -0,0 +1,125 @@ +# flomni → lamni feature comparison + +Comparison of the `flomni` and `LamNI` ipython-client plugins, written to scope +which flomni features (GUI, status webpage, tomo-parameter widget, angle +distribution) are worth porting to lamni. This is a comparison only — no +migration has been implemented yet. + +## 1. Scope & method + +Compared: +- `csaxs_bec/bec_ipython_client/plugins/flomni/` vs + `csaxs_bec/bec_ipython_client/plugins/LamNI/` (file-by-file, line counts, class + structure). +- The shared `csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py` + (`TomoParamsWidget`, `TomoQueueDialog`, `CommandJobBuilderDialog`) — used by + flomni today, not by lamni. +- `flomni_webpage_generator.py` / `LamNI_webpage_generator.py` and their shared + base class. + +Not covered here: device-layer differences (`csaxs_bec/devices/omny/...`), +scan classes (`csaxs_bec/scans/flomni_fermat_scan.py` vs +`csaxs_bec/scans/LamNIFermatScan.py`), or user-facing docs under +`docs/user/ptychography/`. + +## 2. Side-by-side feature table + +| Feature | flomni | LamNI | Gap | +|---|---|---|---| +| Main plugin file | `flomni.py`, 4520 lines | `lamni.py`, 1226 lines | lamni is ~3.7x smaller; several flomni concepts have no lamni counterpart at all (see below) | +| Plugin class structure | `Flomni(...)` composed from mixins defined *inline* in `flomni.py`: `FlomniInitStagesMixin`, `FlomniSampleTransferMixin`, `FlomniAlignmentMixin`, plus `_ProgressProxy`, `_TomoQueueProxy`, `_GlobalVarParam` helper classes | `LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools)` — alignment/optics split into separate files (`lamni_alignment_mixin.py`, `lamni_optics_mixin.py`); only `_ProgressProxy` carried over | lamni has no `_TomoQueueProxy` / `_GlobalVarParam` equivalents — the tomo-queue-as-job-list mechanism the GUI widget depends on doesn't exist on lamni's producer side at all, not just "unwired" | +| Optics mixin | `flomni_optics_mixin.py`, 344 lines | `lamni_optics_mixin.py`, 332 lines | comparable size, setup-specific content; not a gap, just parallel implementations | +| Alignment mixin | inline `FlomniAlignmentMixin` in `flomni.py` (from line 1259) | separate `lamni_alignment_mixin.py`, 369 lines | organizational difference only | +| X-ray eye alignment | `x_ray_eye_align.py`, 440 lines | `x_ray_eye_align.py`, 568 lines | both present, lamni's is larger — not a gap | +| Extra scan types | none | `extra_tomo.py`: `MagLamNI` (XMCD/magnetic, `sub_tomo_scan`, `rotate_slowly`, `_at_each_angle` hook), `DataDrivenLamNI` (reads per-projection params from HDF5) | lamni-only feature; flomni has no analogue and doesn't need one for this migration | +| Tomo-parameter GUI widget | Used via shared `TomoParamsWidget` (`bec_widgets/widgets/tomo_params/tomo_params.py`) | **Not usable.** `TomoParamsWidget._check_flomni_available()` probes for the `fsamroy` device (flomni's rotation stage) and shows a warning banner + disables editing if absent — lamni sessions are explicitly treated as unsupported | primary migration target, see §3 | +| Tomo queue (job list: add/execute/clear scan jobs from the GUI) | Supported — `TomoQueueDialog`, `CommandJobBuilderDialog` in `tomo_params.py`, backed by `Flomni._TomoQueueProxy` | Not supported — no producer-side equivalent | needs both GUI generalization *and* a lamni-side queue proxy; not a thin wrapper job | +| Angle-distribution / projection scheduling ("tomo_type") | 3 modes: (1) 8 equally-spaced sub-tomograms, (2) golden ratio sorted in bunches, (3) equally-spaced with golden-ratio start. Implemented via `_golden()` in `flomni.py` (~line 3122), surfaced in the widget's `TOMO_TYPES` dict | Same underlying `_golden()`-style scheduling exists in `lamni.py` (`sub_tomo_scan`, `tomo_scan`, own `_golden`/`_golden_new` helpers), but only 2 modes: equally-spaced grid with either 2 or 8 sub-tomograms, hardcoded to 360° | logic already exists on both sides; the widget just needs a lamni-specific `TOMO_TYPES` mapping instead of reusing flomni's 3-mode set | +| 180°/360° angle range | Configurable per scan via `tomo_angle_range` global var (∈ {180, 360}) | No such concept — `lamni.py` hardcodes 360° everywhere (`angle_end = start_angle + 360`, `_golden(..., maxangle=360, ...)`) | confirms user's expectation: lamni has no 180° mode and none should be exposed in a generalized widget for lamni | +| Single-point acquisition (skip Fermat scan, single point per projection) | `single_point_instead_of_fermat_scan` / `single_point_random_shift_max` global vars, dedicated widget section with enable/disable wiring | No equivalent global var or code path in `lamni.py` | confirms user's expectation: not needed for lamni, should stay hidden/omitted rather than generalized | +| Global-var parameter names (shared concepts) | `fovx` / `fovy`, `stitch_x` / `stitch_y`, `tomo_angle_stepsize`, `tomo_countingtime`, `corridor_size`, `frames_per_trigger`, `ptycho_reconstruct_foldername`, `golden_ratio_bunch_size`, `golden_max_number_of_projections` | Overlapping concepts under different names: `tomo_circfov` (single value, not x/y — laminography FOV is circular) instead of `fovx`/`fovy`; `lamni_stitch_x`/`lamni_stitch_y` instead of `stitch_x`/`stitch_y`; plus lamni-only `lamni_piezo_range_x`/`lamni_piezo_range_y` with no flomni analogue. Same names reused as-is: `tomo_angle_stepsize`, `tomo_countingtime`, `corridor_size`, `frames_per_trigger`, `ptycho_reconstruct_foldername`, `golden_ratio_bunch_size`, `golden_max_number_of_projections`, `golden_projections_at_0_deg_for_damage_estimation` | the widget's hardcoded `QUEUE_PARAM_NAMES`/`DEFAULTS` (mirroring `Flomni._TOMO_SCAN_PARAM_NAMES` verbatim) can't just be reused for lamni — needs a per-setup name mapping, not a param rename in lamni.py | +| Status webpage / webapp | Rich: `flomni_webpage_generator.py`, 3042 lines. Temperatures, sample name, tomo queue card, live progress, audio alerts, HTTP upload + local server, session auth | Stub: `LamNI_webpage_generator.py`, 104 lines. Subclasses the shared `WebpageGeneratorBase`, but `_collect_setup_data()` is commented out (returns only `{"type": "lamni"}`), `_TEMP_MAP` is empty, `HAS_TOMO_QUEUE = False`, and `TOMO_TYPES` values are marked `TODO` placeholders pending confirmation of lamni's actual `tomo_type` values | biggest line-count gap, but architecturally the easiest migration — `WebpageGeneratorBase` + `make_webpage_generator()` factory already support lamni, the stub just needs finishing (see §4) | +| Webpage generator wired into plugin startup | Yes — `flomni.py` instantiates `FlomniWebpageGenerator` in `Flomni.__init__` | **No** — `lamni.py` has zero references to `webpage_gen` or `WebpageGenerator` anywhere; the stub class exists but is never instantiated | needs one `__init__` wiring change plus finishing §4 | +| `AI_docs/` planning docs | 8 files: `Webpage.md`, `TOMO_QUEUE_COMMAND_JOBS_PLAN.md`, `TOMO_QUEUE_GUI_TESTING.md`, `TOMO_QUEUE_TESTING.md`, `E2E_TESTING_GUIDE.md`, `SIMULATED_ENDSTATIONS.md`, `BEC info.md`, `Endstation INFO flomni lamni omny.md` | none (this file is the first) | n/a — documentation only | + +## 3. Tomo-parameter-widget generalization + +`TomoParamsWidget` (`bec_widgets/widgets/tomo_params/tomo_params.py`) is +currently written as a flomni-only widget that happens to live in the shared +`bec_widgets` package, not a genuinely setup-aware widget. Concretely +flomni-hardcoded: + +- `_check_flomni_available()` (~line 220): gates the entire widget on the + presence of the `fsamroy` device. For a generalized widget this needs to + become "detect which setup is active" (flomni / lamni / eventually omny) + rather than a yes/no flomni check. +- `QUEUE_PARAM_NAMES` / `DEFAULTS` (lines 98–144): a literal copy of + `Flomni._TOMO_SCAN_PARAM_NAMES`, including flomni-only param names (`fovx`, + `fovy`, `stitch_x`, `stitch_y`) that don't exist under those names in lamni's + global vars (`tomo_circfov`, `lamni_stitch_x`, `lamni_stitch_y`). +- The 180°/360° `tomo_angle_range` dropdown (line 137, referenced ~line 428, + 568, 633, 668): must not be offered at all for lamni — lamni is 360°-only, + not "180° available but unused." +- The single-point acquisition section (`single_point_instead_of_fermat_scan`, + `single_point_random_shift_max`, lines 133–134, 379–398, 735–746): must be + omitted entirely for lamni, not just disabled. +- `TOMO_TYPES` (lines 80–84): flomni's 3-mode set doesn't match lamni's 2-mode + set (2-subtomo / 8-subtomo equally-spaced grid, per the `TODO` in + `LamNI_webpage_generator.py`'s placeholder `TOMO_TYPES` dict) — these + `tomo_type` integer values need confirming against `lamni.py` before they can + be wired into either the widget or the webpage generator. +- Tomo queue backing store: the widget's queue/job management + (`TomoQueueDialog`, `CommandJobBuilderDialog`) is backed by + `Flomni._TomoQueueProxy` / `_GlobalVarParam`, which have no lamni-side + equivalent — this is not just a widget change, it needs a producer-side + queue mechanism added to `lamni.py` first (or the queue/job-list feature + scoped out of the lamni version of the widget entirely, if lamni doesn't + need scan-job queuing the way flomni's automated multi-day runs do). + +Proposed shape (not implemented): a per-setup config object (param-name +mapping + capability flags: `has_180_mode`, `has_single_point`, +`has_tomo_queue`, `tomo_types` dict) that the widget reads at construction time +instead of the current hardcoded flomni constants, with setup detection +replacing `_check_flomni_available()`. + +## 4. Webpage generator gap + +To bring lamni's status page to parity with flomni's, `LamNI_webpage_generator.py` +needs, in order: + +1. Confirm lamni's actual `tomo_type` integer values (currently `TODO` + placeholders assuming 1→2-subtomo, 2→8-subtomo) against `lamni.py`'s real + usage. +2. Fill in `_TEMP_MAP` with lamni's actual temperature/humidity device dot-paths + (placeholder comments reference `lamni_temphum.*`, unconfirmed). +3. Implement `_collect_setup_data()` (currently fully commented out) — sample + name, FOV, exposure time, angle step, mirroring flomni's version but using + lamni's global-var names (`tomo_circfov`, `lamni_stitch_x/y`, etc.). +4. Decide on `HAS_TOMO_QUEUE` — stays `False` unless/until a lamni tomo-queue + backing store exists (§3). +5. Wire `LamniWebpageGenerator` into `lamni.py.__init__`, the way + `FlomniWebpageGenerator` is wired into `Flomni.__init__` — currently there + is no `webpage_gen` reference anywhere in `lamni.py`. + +No changes needed to `WebpageGeneratorBase` or `make_webpage_generator()` — +both already support this without modification. + +## 5. Recommendation / candidate migration order + +1. **Webpage generator** — lowest risk, highest immediate payoff. Architecture + already supports it; work is filling in a known-shape stub (§4). No GUI + widget dependency. +2. **Angle-distribution mapping for the tomo-params widget** — expose lamni's + existing 2-mode scheduling through the widget's `TOMO_TYPES`, gated by setup + detection, without yet tackling the queue/job-list machinery. +3. **Tomo-parameter widget generalization proper** (§3) — setup-detection + refactor + per-setup param-name mapping + hiding 180°/single-point for + lamni. Larger, touches the widget's core assumptions. +4. **Tomo queue / job-list feature** — only if lamni actually needs automated + multi-projection job queuing the way flomni does; requires new producer-side + plumbing in `lamni.py` (`_TomoQueueProxy`-equivalent), not just GUI changes. + Worth confirming with Mirko whether lamni's workflow needs this at all + before investing here. + +This document is the input for deciding scope/order with Mirko; no +implementation plan has been written yet for any of the above. diff --git a/csaxs_bec/bec_ipython_client/plugins/flomni/flomni.py b/csaxs_bec/bec_ipython_client/plugins/flomni/flomni.py index c17c785..796cf23 100644 --- a/csaxs_bec/bec_ipython_client/plugins/flomni/flomni.py +++ b/csaxs_bec/bec_ipython_client/plugins/flomni/flomni.py @@ -1,12 +1,10 @@ import builtins import datetime -import inspect import json import os import random import subprocess import time -import uuid from pathlib import Path import h5py @@ -16,12 +14,16 @@ from bec_lib.alarm_handler import AlarmBase from bec_lib.endpoints import MessageEndpoints from bec_lib.pdf_writer import PDFWriter from bec_lib.scan_repeat import scan_repeat -from typeguard import check_type, typechecked +from typeguard import typechecked from csaxs_bec.bec_ipython_client.plugins.cSAXS import cSAXSBeamlineChecks from csaxs_bec.bec_ipython_client.plugins.flomni.flomni_optics_mixin import FlomniOpticsMixin from csaxs_bec.bec_ipython_client.plugins.flomni.gui_tools import flomniGuiTools from csaxs_bec.bec_ipython_client.plugins.flomni.x_ray_eye_align import XrayEyeAlign +from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.tomo_queue_mixin import ( + TomoQueueMixin, + _GlobalVarParam, +) from csaxs_bec.bec_ipython_client.plugins.omny.omny_general_tools import ( OMNYTools, PtychoReconstructor, @@ -1610,155 +1612,8 @@ class _ProgressProxy: return self._load() -class _TomoQueueProxy: - """List-like proxy that persists a queue of tomo parameter snapshots as a - BEC global variable, mirroring the pattern used by :class:`_ProgressProxy`. - - Each entry is a plain dict: ``{"label": ..., "params": {...}, "status": - ..., "added_at": ...}``, where ``params`` holds a snapshot of all - global-var-backed tomo scan parameters (see - ``Flomni._TOMO_SCAN_PARAM_NAMES``). Stored as a single list under - ``tomo_queue`` so the queue is visible from any BEC client session via - ``client.get_global_var("tomo_queue")`` and survives a kernel restart. - """ - - _GLOBAL_VAR_KEY = "tomo_queue" - - def __init__(self, client): - self._client = client - - def _load(self) -> list: - val = self._client.get_global_var(self._GLOBAL_VAR_KEY) - if val is None: - return [] - return val - - def _save(self, jobs: list) -> None: - self._client.set_global_var(self._GLOBAL_VAR_KEY, jobs) - - def as_list(self) -> list: - """Return a plain copy of the current queue.""" - return self._load() - - def append(self, job: dict) -> int: - jobs = self._load() - jobs.append(job) - self._save(jobs) - return len(jobs) - 1 - - def pop(self, index: int) -> dict: - jobs = self._load() - job = jobs.pop(index) - self._save(jobs) - return job - - def update(self, index: int, **kwargs) -> None: - jobs = self._load() - jobs[index].update(kwargs) - self._save(jobs) - - def update_by_id(self, job_id: str, **kwargs) -> bool: - """Update the job whose ``id`` matches ``job_id``. - - Returns True if a matching job was found and updated, False otherwise - (e.g. the job was deleted concurrently). Addressing a job by its stable - id rather than its list index makes status writes safe against queue - edits made while the queue runs: a reorder, or a delete of some *other* - job, can no longer misdirect a status write to the wrong entry. - """ - jobs = self._load() - for job in jobs: - if job.get("id") == job_id: - job.update(kwargs) - self._save(jobs) - return True - return False - - def move(self, index: int, new_index: int) -> None: - """Move the job at ``index`` to ``new_index``, shifting the rest. - - Pure list mechanics -- no status checks. Callers (e.g. - ``Flomni.tomo_queue_move``) are responsible for deciding whether a - given move is allowed (running-job floor, movable statuses, ...). - """ - jobs = self._load() - job = jobs.pop(index) - jobs.insert(new_index, job) - self._save(jobs) - - def ensure_ids(self) -> list: - """Assign a stable id to any job that lacks one and persist the result. - - Back-compat shim for queues written before jobs carried an ``id`` field - (and for any job added by older code). Called once at the top of - tomo_queue_execute() so the pick-next executor can address every job by - id. Returns the (possibly healed) job list. - """ - jobs = self._load() - changed = False - for job in jobs: - if not job.get("id"): - job["id"] = uuid.uuid4().hex - changed = True - if changed: - self._save(jobs) - return jobs - - def clear(self) -> None: - self._save([]) - - def __len__(self) -> int: - return len(self._load()) - - def __getitem__(self, index): - return self._load()[index] - - -class _GlobalVarParam: - """Descriptor for a Flomni parameter backed by a BEC global variable. - - Consolidates the boilerplate getter/setter pairs that read from and write - to the BEC global-var store. The value persists across BEC client restarts - (instance attributes do not), and is readable from other client sessions - via ``client.get_global_var()``. - - The global-var key is taken from the attribute name the descriptor is - assigned to (see :meth:`__set_name__`), so it always matches the property - name -- exactly the invariant the hand-written pairs relied on. - - Parameters - ---------- - default - Value returned by the getter when the global var is unset (``None``). - type_ - Optional expected type. When given, the setter validates ``val`` with - :func:`typeguard.check_type` before storing, reproducing the - ``@typechecked`` guard that previously wrapped some setters (raising - ``TypeCheckError`` on a mismatch). - """ - - def __init__(self, default, type_=None): - self._default = default - self._type = type_ - - def __set_name__(self, owner, name): - self._key = name - - def __get__(self, obj, objtype=None): - if obj is None: - return self - val = obj.client.get_global_var(self._key) - if val is None: - return self._default - return val - - def __set__(self, obj, val): - if self._type is not None: - check_type(val, self._type) - obj.client.set_global_var(self._key, val) - - class Flomni( + TomoQueueMixin, FlomniInitStagesMixin, FlomniSampleTransferMixin, FlomniAlignmentMixin, @@ -1776,11 +1631,6 @@ class Flomni( self.special_angle_tolerance = 20 self._current_special_angles = [] self._beam_is_okay = True - # Session-only (not global-var-backed): the hooks themselves are - # Python callables, which can't be persisted the way params/queue - # jobs are. Only the *name* selecting one is ever snapshotted (see - # at_each_angle_hook / register_at_each_angle_hook()). - self._at_each_angle_hooks: dict = {} self._progress_proxy = _ProgressProxy(self.client) # Deliberately NOT calling reset() here: this dict is persisted via a # BEC global var specifically so it survives a kernel restart (e.g. @@ -1791,14 +1641,7 @@ class Flomni( # resets the relevant fields itself (see the "new scan" branch # below); use tomo_progress_reset() if you want to explicitly clear # stale progress without starting a new scan. - self._tomo_queue_proxy = _TomoQueueProxy(self.client) - # Published so a future GUI never needs a hardcoded mirror of the - # action registry -- see _describe_tomo_queue_actions(). - self.client.set_global_var("tomo_queue_actions", self._describe_tomo_queue_actions()) - # Reset to empty on every Flomni() -- hooks are session-only (see - # self._at_each_angle_hooks above), so a stale list published by a - # since-ended session must not linger and look registered. - self._publish_at_each_angle_hooks() + self._init_tomo_queue() from csaxs_bec.bec_ipython_client.plugins.flomni.flomni_webpage_generator import ( FlomniWebpageGenerator, ) @@ -2013,11 +1856,11 @@ class Flomni( manual_shift_y = _GlobalVarParam(0.0) - # Name of a registered at_each_angle hook (see register_at_each_angle_hook()), - # or None for the default per-projection behaviour. Snapshotted like any + # at_each_angle_hook is inherited from TomoQueueMixin: name of a + # registered at_each_angle hook (see register_at_each_angle_hook()), or + # None for the default per-projection behaviour. Snapshotted like any # other tomo scan parameter, so a tomo-queue job can carry its own hook # selection -- see _at_each_angle(). - at_each_angle_hook = _GlobalVarParam(None) @property def single_point_random_shift_max(self): @@ -3016,109 +2859,6 @@ class Flomni( self.tomo_scan_projection(angle, _internal=True) - def register_at_each_angle_hook(self, name: str, func) -> None: - """Register a per-projection hook function under ``name``, so a - tomo-queue job can select it by name via the ``at_each_angle_hook`` - parameter and have it run automatically -- unlike the older - ``flomni_at_each_angle`` builtins-based hook, which the queue can't - see (it isn't a snapshotted parameter, so a queued job has no way - to record "use this hook"). - - ``func`` is called as ``func(self, angle)`` at every projection - angle, exactly like the interactive ``flomni_at_each_angle`` hook -- - write it the same way, just give it a name and register it here - instead of assigning it to the ``flomni_at_each_angle`` builtin. - - Hooks are **session-only**: they live in this ``Flomni`` instance's - memory, not in a global var, so they do NOT survive a kernel - restart. A queued job that references a hook by name raises a clear - error at execution time if the name isn't registered in the session - running the queue -- re-run this registration call, then resume - ``tomo_queue_execute()``. - - Example -- record a projection, insert a polarizer, record again:: - - def polarizer_modulation(flomni, angle): - flomni.tomo_scan_projection(angle, _internal=True) - umv(dev.polarizer, "in") - flomni.tomo_scan_projection(angle, _internal=True) - umv(dev.polarizer, "out") - - flomni.register_at_each_angle_hook("polarizer_mod", polarizer_modulation) - flomni.at_each_angle_hook = "polarizer_mod" - flomni.tomo_queue_add(label="polarizer run") - flomni.at_each_angle_hook = None # back to default for the next job - """ - if not callable(func): - raise TypeError(f"register_at_each_angle_hook: {func!r} is not callable.") - self._at_each_angle_hooks[name] = func - self._publish_at_each_angle_hooks() - print(f"Registered at_each_angle hook '{name}'.") - - def unregister_at_each_angle_hook(self, name: str) -> None: - """Remove a previously registered at_each_angle hook by name.""" - if self._at_each_angle_hooks.pop(name, None) is None: - print(f"No at_each_angle hook named '{name}' is registered.") - else: - self._publish_at_each_angle_hooks() - print(f"Unregistered at_each_angle hook '{name}'.") - - def _publish_at_each_angle_hooks(self) -> None: - """Publish the names of all currently registered at_each_angle - hooks as the ``tomo_at_each_angle_hooks`` global var (a plain list - of strings) -- so a GUI (or any other client) can build a dropdown - from it directly, no hardcoded mirror, same reasoning as - ``tomo_queue_actions`` for the command-job registry (see - _describe_tomo_queue_actions()). Unlike that registry this list - changes at runtime, whenever register_/unregister_at_each_angle_hook() - is called -- not just once at __init__. - """ - self.client.set_global_var("tomo_at_each_angle_hooks", sorted(self._at_each_angle_hooks)) - - def list_at_each_angle_hooks(self) -> list: - """Print and return the names of all at_each_angle hooks registered - in this session. The active one (``self.at_each_angle_hook``, if - any) is marked.""" - names = sorted(self._at_each_angle_hooks) - if not names: - print("No at_each_angle hooks registered.") - else: - for name in names: - marker = " (active)" if name == self.at_each_angle_hook else "" - print(f" {name}{marker}") - return names - - def _describe_active_hook(self) -> str | None: - """Human-readable description of ``self.at_each_angle_hook`` for - display in tomo_parameters()/scilog/the PDF report: the name, or - the name flagged as unregistered if it doesn't (or no longer) - resolve in this session -- e.g. unregistered after the job that - used it was added, or set by a session that has since restarted. - Returns None if no hook is set at all. - """ - hook = self.at_each_angle_hook - if not hook: - return None - if hook in self._at_each_angle_hooks: - return hook - return f"{hook} (NOT registered in this session)" - - def _active_hook_source(self) -> str | None: - """Source code of the currently active at_each_angle hook, for - inclusion in the scilog/PDF record -- so what a tomogram actually - did is part of the permanent log, not just the hook's name. - Returns None if there's no active hook, it isn't registered in - this session, or its source can't be retrieved (e.g. it wasn't - defined via a normal `def`/interactive cell). - """ - hook_func = self._at_each_angle_hooks.get(self.at_each_angle_hook) - if hook_func is None: - return None - try: - return inspect.getsource(hook_func) - except (OSError, TypeError): - return None - def _golden(self, ii, howmany_sorted, maxangle, reverse=False): """returns the iis golden ratio angle of sorted bunches of howmany_sorted and its subtomo number""" golden = [] @@ -3940,462 +3680,15 @@ class Flomni( }, } - def tomo_queue_add(self, label: str = None) -> int: - """Snapshot the currently set tomo parameters and append them as a - new job to the tomo queue (persisted, survives a kernel restart). - - Typical usage:: - - flomni.tomo_parameters() # set up parameter set #1 - flomni.tomo_queue_add("fast overview scan") - flomni.tomo_parameters() # change to parameter set #2 - flomni.tomo_queue_add("hires scan") - flomni.tomo_queue_show() - flomni.tomo_queue_execute() # runs both, in order, on this sample - - Args: - label: Optional name for the job, shown by tomo_queue_show(). - Defaults to "job_". - - Returns: - The index of the newly added job. - """ - params = {name: getattr(self, name) for name in self._TOMO_SCAN_PARAM_NAMES} - index = len(self._tomo_queue_proxy) - job = { - "kind": "tomo", - "id": uuid.uuid4().hex, - "label": label or f"job_{index + 1}", - "params": params, - "status": "pending", - "added_at": datetime.datetime.now().isoformat(), - } - self._tomo_queue_proxy.append(job) - print(f"Added tomo queue job #{index} ({job['label']}).") - return index - - def tomo_queue_add_command(self, steps, label: str = None, idempotent: bool = None) -> int: - """Append a "command" job -- an ordered list of named-registry actions - that reconfigure the beamline (move a device, ...) -- to the tomo - queue. - - Unlike tomo_queue_add(), this does not snapshot the current tomo scan - parameters; a command job has no "params" key at all. - - Typical usage:: - - flomni.tomo_queue_add_command( - [{"action": "move", "kwargs": {"positions": {"mokev": 6.2}}}, - {"action": "optimize_idgap", "kwargs": {"search_range": 0.5}}], - label="reconfigure to 6.2 keV", - ) - - Args: - steps: A single {"action": ..., "kwargs": {...}} dict, or a list - of them, run in sequence within this one job. - label: Optional name for the job, shown by tomo_queue_show(). - Defaults to "job_". - idempotent: Overrides the default of "every step's registry - idempotent flag, ANDed together". Set this explicitly if the - registry defaults don't capture whether this particular - sequence is safe to blindly re-run after a crash. - - Returns: - The index of the newly added job. - """ - if isinstance(steps, dict): - steps = [steps] - if not steps: - raise ValueError("tomo_queue_add_command: at least one step is required.") - - resolved_steps = [] - all_idempotent = True - for step in steps: - action_name = step.get("action") - kwargs = step.get("kwargs", {}) - self._validate_action_kwargs(action_name, kwargs) - all_idempotent = all_idempotent and self._TOMO_QUEUE_ACTIONS[action_name]["idempotent"] - resolved_steps.append({"action": action_name, "kwargs": kwargs}) - - try: - json.dumps(resolved_steps) - except (TypeError, ValueError) as exc: - raise ValueError(f"Command job steps must be JSON-serializable: {exc}") from exc - - index = len(self._tomo_queue_proxy) - job = { - "kind": "command", - "id": uuid.uuid4().hex, - "label": label or f"job_{index + 1}", - "steps": resolved_steps, - "idempotent": all_idempotent if idempotent is None else idempotent, - "status": "pending", - "added_at": datetime.datetime.now().isoformat(), - } - self._tomo_queue_proxy.append(job) - print(f"Added command queue job #{index} ({job['label']}).") - return index - - def _validate_action_kwargs(self, action_name: str, kwargs: dict) -> None: - """Add-time schema validation for a command-job step's kwargs. - - Ergonomic, not the safety boundary: catches a typo or an out-of-range - value now instead of at 2am. Each action method re-validates its own - arguments against live hardware at execution time -- that's the - actual safety boundary, since the schema and the hardware can - disagree (limits changed; a value arrived via the CLI, bypassing - this). - """ - spec = self._TOMO_QUEUE_ACTIONS.get(action_name) - if spec is None: - raise ValueError(f"Unknown queue action '{action_name}'.") - schema = spec["params"] - for key in kwargs: - if key not in schema: - raise ValueError(f"Action '{action_name}' has no parameter '{key}'.") - for name, field in schema.items(): - if name not in kwargs: - continue - value = kwargs[name] - field_type = field.get("type") - if field_type == "float": - if not isinstance(value, (int, float)) or isinstance(value, bool): - raise ValueError(f"{action_name}.{name} must be a number, got {value!r}.") - lo, hi = field.get("min"), field.get("max") - if lo is not None and value < lo: - raise ValueError(f"{action_name}.{name}={value} is below the minimum {lo}.") - if hi is not None and value > hi: - raise ValueError(f"{action_name}.{name}={value} is above the maximum {hi}.") - elif field_type == "device_positions": - if not isinstance(value, dict) or not value: - raise ValueError( - f"{action_name}.{name} must be a non-empty {{device: target}} dict." - ) - for device_name in value: - if device_name not in self._TOMO_QUEUE_MOVE_DEVICES: - raise ValueError( - f"Queue move: device '{device_name}' is not queue-movable." - ) - - def _queue_action_move(self, positions: dict) -> None: - """positions: {device_name: absolute_target, ...}. Absolute umv only. - - Re-checks the allow-list at execution time (not just add time, see - _validate_action_kwargs) -- the actual safety boundary, since a value - could have arrived via the CLI, bypassing tomo_queue_add_command's - validation. - """ - args = [] - for name, target in positions.items(): - if name not in self._TOMO_QUEUE_MOVE_DEVICES: - raise ValueError(f"Queue move: device '{name}' is not queue-movable.") - if name not in dev: - raise ValueError(f"Queue move: unknown device '{name}'.") - args += [dev[name], target] - umv(*args) - - def optimize_idgap(self, search_range: float = 0.5) -> None: - """Scan idgap over +/- search_range around its current position and - move to the peak of . - - STUB -- not implemented. The peak-detection signal/device is not yet - decided (see AI_docs/TOMO_QUEUE_COMMAND_JOBS_PLAN.md section 10). - This is the ONLY place that needs to change to make this a real - action: the registry entry, resume policy, and (eventually) GUI - wiring already treat it as a normal idempotent compound action, so - dropping in the real scan-and-move-to-peak logic here (or swapping - which device/signal it reads) is a self-contained change. - """ - print( - f"optimize_idgap(search_range={search_range}): not yet implemented -- " - "peak-detection signal not yet decided. No-op." + def _describe_tomo_job_line(self, job: dict) -> str: + """flomni's tomo_queue_show() one-line summary: fov/type/step/ctime/range.""" + p = job["params"] + return ( + f"type={p['tomo_type']} fov={p['fovx']}/{p['fovy']}um " + f"step={p['tomo_shellstep']}um ctime={p['tomo_countingtime']}s " + f"range={p['tomo_angle_range']}deg" ) - def _describe_tomo_queue_actions(self) -> dict: - """Serializable description of the action registry + move allow-list. - - Published as the ``tomo_queue_actions`` global var (see __init__) so - the GUI (or any other client) can build its dropdowns/forms from it - directly -- no hardcoded mirror to drift, which is exactly the - problem the QUEUE_PARAM_NAMES mirror between this file and - tomo_params.py already has. - """ - return { - "actions": { - name: { - "idempotent": spec["idempotent"], - "help": spec["help"], - "params": spec["params"], - } - for name, spec in self._TOMO_QUEUE_ACTIONS.items() - }, - "move_devices": dict(self._TOMO_QUEUE_MOVE_DEVICES), - } - - def tomo_queue_delete(self, *indices: int) -> None: - """Delete one or more jobs from the tomo queue by index. - - Accepts any number of indices, e.g. ``flomni.tomo_queue_delete(2, 5)`` - to drop several jobs in one call. All indices are resolved against - the queue as it currently stands and deleted highest-index-first, so - passing several indices in any order is safe and won't shift the - meaning of the indices still to be deleted. - """ - if not indices: - print("No index given.") - return - for index in sorted(set(indices), reverse=True): - job = self._tomo_queue_proxy.pop(index) - print(f"Deleted tomo queue job #{index} ({job['label']}).") - - def tomo_queue_move(self, index: int, new_index: int) -> None: - """Move a pending tomo queue job to a new position. - - Only ``pending`` jobs can be reordered -- a ``running``, - ``incomplete``, or ``done`` job stays put. ``incomplete`` is a floor - exactly like ``running``: tomo_queue_execute()'s pick-next loop - always resumes whichever of the two it finds first, regardless of - list order, so a pending job moved "ahead" of one in the list would - visually suggest a run order that isn't real. A move is also - refused if it would place the job at or above the position of any - currently ``running``/``incomplete`` job. Takes effect at the next - job boundary if the queue is executing (see tomo_queue_execute()'s - pick-next model). - - Args: - index: Current index of the job to move. - new_index: Index to move it to (list-insert semantics: the rest - of the queue shifts to make room). - """ - jobs = self._tomo_queue_proxy.as_list() - if not 0 <= index < len(jobs): - raise ValueError(f"tomo_queue_move: index {index} out of range.") - if not 0 <= new_index < len(jobs): - raise ValueError(f"tomo_queue_move: new_index {new_index} out of range.") - job = jobs[index] - if job.get("status", "pending") != "pending": - raise ValueError( - f"tomo_queue_move: job #{index} is '{job.get('status')}' -- only " - "pending jobs can be reordered." - ) - floor_indices = [ - i for i, j in enumerate(jobs) if j.get("status") in ("running", "incomplete") - ] - if floor_indices and new_index <= max(floor_indices): - raise ValueError( - f"tomo_queue_move: can't move to position {new_index} -- job " - f"#{max(floor_indices)} is running/incomplete and is a hard floor " - "for reordering." - ) - self._tomo_queue_proxy.move(index, new_index) - print(f"Moved tomo queue job #{index} -> #{new_index} ({job['label']}).") - - def tomo_queue_clear(self) -> None: - """Empty the tomo queue.""" - self._tomo_queue_proxy.clear() - print("Tomo queue cleared.") - - def tomo_queue_show(self) -> list: - """Print and return the current tomo queue, one line per job.""" - jobs = self._tomo_queue_proxy.as_list() - if not jobs: - print("Tomo queue is empty.") - return jobs - for i, job in enumerate(jobs): - if job.get("kind", "tomo") == "command": - steps_str = " > ".join( - f"{step['action']}{step['kwargs']}" if step["kwargs"] else step["action"] - for step in job["steps"] - ) - idem = "idem" if job.get("idempotent") else "NOT idem" - print(f"[{i}] {job['status']:>10s} {job['label']:<20s} CMD {steps_str} [{idem}]") - continue - p = job["params"] - print( - f"[{i}] {job['status']:>10s} {job['label']:<20s} " - f"type={p['tomo_type']} fov={p['fovx']}/{p['fovy']}um " - f"step={p['tomo_shellstep']}um ctime={p['tomo_countingtime']}s " - f"range={p['tomo_angle_range']}deg" - ) - return jobs - - def tomo_queue_execute(self, start_index: int = 0) -> None: - """Run all pending tomo queue jobs in sequence, on the current sample. - - For each job, restores its snapshotted parameters onto the live - properties (exactly as if set by hand) and then calls - ``tomo_scan()`` -- or, for a job that didn't run to completion last - time, ``tomo_scan_resume()``, so it picks back up mid-scan instead - of restarting from subtomo 1 / angle 0. Jobs already marked "done" - are skipped, so simply calling tomo_queue_execute() again resumes - from where it stopped (e.g. after fixing a hardware issue). - - Pick-next execution model - ------------------------- - The queue is re-read from its global var at the start of every job - (not snapshotted once up front), and each job is addressed by its - stable ``id``, not by its list index. This is what makes it safe to - edit the queue *while it runs*: appending a job, deleting a pending - job, or reordering the pending tail all take effect at the next job - boundary, and a reorder/delete can never misdirect a status write to - the wrong entry (an index-based loop could). Legacy jobs written - before the ``id`` field existed are healed once up front by - ensure_ids(). - - Resume-before-fresh ordering: if any job is in a resumable state, it - is always run before any fresh "pending" job, regardless of list - order. This preserves the single-resumable-job invariant -- tomo - progress lives in one shared ``progress`` global var, so starting a - fresh tomo_scan() before resuming an interrupted one would overwrite - the very state tomo_scan_resume() needs. (Under the old strictly - in-order loop this couldn't happen; allowing reorder makes it - possible, hence the explicit guard.) - - A job is considered not-yet-complete (and so gets resumed rather - than restarted) if its status is "incomplete" (a Python exception - was caught and execution stopped cleanly) OR "running" (the queue - process itself died -- killed kernel, dropped connection, power - loss, ... -- before it had a chance to record anything; in that - case the status is whatever was last written, which is "running", - not "incomplete", since the except block below never got to run). - Without treating "running" as resumable too, a real crash would - cause this method to silently restart that job from scratch on the - next call instead of resuming it. - - If you've already manually called flomni.tomo_scan_resume() - yourself to recover from a crash (bypassing the queue), that scan - is now actually finished even though the queue still has the job - marked "incomplete" or "running" -- mark it done yourself before - calling this again, or it will be re-run from scratch. Address the - job by its id (indices can shift if the queue is reordered): - flomni._tomo_queue_proxy.update_by_id(job_id, status="done") - - Command jobs (job["kind"] == "command", added by - tomo_queue_add_command()) are dispatched to _run_command_job() - instead of tomo_scan()/tomo_scan_resume() -- see its docstring for - their own resume policy. A job with no "kind" key is treated as - "tomo" (back-compat for queues persisted before command jobs - existed). - - Args: - start_index: Position in the *current* queue order below which - fresh "pending" jobs are ignored. Defaults to 0. Resumable - jobs are always run regardless of start_index (a crashed - job must be recovered wherever it sits). Jobs already marked - "done" are skipped automatically either way. Note that with - reordering active this is applied to the queue's current - order at each pick. - """ - self._tomo_queue_proxy.ensure_ids() - jobs = self._tomo_queue_proxy.as_list() - if not jobs: - print("Tomo queue is empty.") - return - - runnable = [ - job - for idx, job in enumerate(jobs) - if job["status"] != "done" - and (job["status"] in ("incomplete", "running") or idx >= start_index) - ] - if not runnable: - print("No pending tomo queue jobs to run.") - return - - if not self.OMNYTools.yesno( - f"Starting automatic execution of {len(runnable)} queued tomo scan(s) on" - f" sample '{self.sample_name}'. OK?", - "y", - ): - print("Aborted.") - return - - while True: - jobs = self._tomo_queue_proxy.as_list() - # Resume any in-flight/interrupted job first, before starting any - # fresh one, so the shared progress var is never clobbered. Search - # the whole list (not the start_index slice): a crashed job must be - # recovered wherever it sits. - job = next((j for j in jobs if j["status"] in ("incomplete", "running")), None) - if job is None: - job = next( - ( - j - for idx, j in enumerate(jobs) - if idx >= start_index and j["status"] == "pending" - ), - None, - ) - if job is None: - break - - job_id = job["id"] - label = job["label"] - kind = job.get("kind", "tomo") - resume_job = job["status"] in ("incomplete", "running") - - print(f"\n=== Tomo queue job: {label} ===") - - self._tomo_queue_proxy.update_by_id(job_id, status="running") - try: - if kind == "command": - self._run_command_job(job, resume_job) - else: - for name, value in job["params"].items(): - setattr(self, name, value) - if resume_job: - self.tomo_scan_resume() - else: - self.tomo_scan() - except Exception as exc: - self._tomo_queue_proxy.update_by_id(job_id, status="incomplete") - print(f"Tomo queue job '{label}' did not complete: {exc}") - print( - "Queue paused. Fix the issue and call tomo_queue_execute() " - "again to resume from this job." - ) - raise - self._tomo_queue_proxy.update_by_id(job_id, status="done") - - print("\nTomo queue finished -- all jobs done.") - - def _run_command_job(self, job: dict, resume_job: bool) -> None: - """Run (or resume) a "command" job's steps in order. - - No per-step resume cursor -- deliberately: tracking which step of a - multi-step job completed means more persisted state and more edge - cases than this warrants. Instead the job-level "idempotent" flag - decides: - - - idempotent: silently re-run every step from the top. Harmless -- - absolute moves and in/out actions re-run cleanly. - - not idempotent: prompt the operator rather than silently re-run, - which could be actively wrong for a relative/stateful step. "no" - leaves the steps un-run; the caller (tomo_queue_execute) still - marks the job "done" either way, matching the same edge case tomo - jobs have -- a job that actually finished but crashed before its - status hit "done". - """ - label = job["label"] - if resume_job: - if job.get("idempotent"): - print( - f"Command job '{label}' may have partially run before the crash " - "-- idempotent, re-running from the top." - ) - else: - if not self.OMNYTools.yesno( - f"Command job '{label}' may have partially run before the crash " - "(not every step is idempotent). Re-run it from the top?", - "n", - ): - print(f"Skipping '{label}' -- it will be marked done without re-running.") - return - for step in job["steps"]: - spec = self._TOMO_QUEUE_ACTIONS[step["action"]] - method = getattr(self, spec["func"]) - method(**step["kwargs"]) - def rt_off(self): dev.rtx.enabled = False dev.rty.enabled = False diff --git a/tests/tests_bec_ipython_client/test_lamni_tomo_queue.py b/tests/tests_bec_ipython_client/test_lamni_tomo_queue.py new file mode 100644 index 0000000..ab43eb2 --- /dev/null +++ b/tests/tests_bec_ipython_client/test_lamni_tomo_queue.py @@ -0,0 +1,206 @@ +"""Unit tests for the tomo-queue / at-each-angle-hook backend ported to +LamNI from flomni (see TomoQueueMixin in +csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py). + +Builds a LamNI instance without running __init__'s heavy side effects +(device init, reconstructor, webpage generator, ...) -- only what the +tomo-queue backend actually needs: self.client + _init_tomo_queue(). +""" + +import types + +import csaxs_bec.bec_ipython_client.plugins.LamNI.lamni as lamni_module +import csaxs_bec.bec_ipython_client.plugins.OMNY_shared.tomo_queue_mixin as tomo_queue_mixin +from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI + + +class _FakeRtx: + user_parameter = {"large_range_scan": True} + + +class _FakeDev: + rtx = _FakeRtx() + + def __contains__(self, name): + return hasattr(self, name) + + def __getitem__(self, name): + return getattr(self, name) + + +class FakeClient: + """Minimal in-memory stand-in for BEC's global-var store.""" + + def __init__(self): + self._vars = {} + + def get_global_var(self, key): + return self._vars.get(key) + + def set_global_var(self, key, value): + self._vars[key] = value + + +class FakeOMNYTools: + def __init__(self, answer=True): + self.answer = answer + + def yesno(self, *args, **kwargs): + return self.answer + + +def make_lamni(fake_client=None): + # lamni_piezo_range_x/y's setters check dev.rtx.user_parameter -- only + # ever defined (via the `if builtins.__dict__.get("bec") is not None` + # block at module import time) inside a real BEC session. Stand in a + # fake so tomo_queue_execute()'s param-restore loop can reach these + # setters like any other snapshotted param. + lamni_module.dev = _FakeDev() + + obj = object.__new__(LamNI) + obj.client = fake_client or FakeClient() + obj.OMNYTools = FakeOMNYTools() + # ptycho_reconstruct_foldername's setter keeps this in sync; normally + # built in __init__ from a real PtychoReconstructor. + obj.reconstructor = types.SimpleNamespace(folder_name=None) + obj._init_tomo_queue() + return obj + + +def test_tomo_scan_param_names_have_no_flomni_only_concepts(): + """Lamni is laminography: 360-only, no single-point acquisition -- these + must not appear in the snapshotted param set at all.""" + names = LamNI._TOMO_SCAN_PARAM_NAMES + assert "tomo_angle_range" not in names + assert "single_point_instead_of_fermat_scan" not in names + assert "single_point_random_shift_max" not in names + # lamni's own names for overlapping flomni concepts are present instead + assert "tomo_circfov" in names + assert "lamni_stitch_x" in names + assert "lamni_stitch_y" in names + assert "fovx" not in names + assert "stitch_x" not in names + + +def test_tomo_queue_add_snapshots_all_param_names(): + lamni = make_lamni() + lamni.tomo_countingtime = 0.2 + lamni.tomo_circfov = 42.0 + + index = lamni.tomo_queue_add(label="my job") + + assert index == 0 + jobs = lamni._tomo_queue_proxy.as_list() + assert len(jobs) == 1 + job = jobs[0] + assert job["label"] == "my job" + assert job["kind"] == "tomo" + assert job["status"] == "pending" + assert set(job["params"]) == set(LamNI._TOMO_SCAN_PARAM_NAMES) + assert job["params"]["tomo_countingtime"] == 0.2 + assert job["params"]["tomo_circfov"] == 42.0 + + +def test_tomo_queue_show_uses_lamni_specific_job_line(capsys): + lamni = make_lamni() + lamni.tomo_circfov = 15.0 + lamni.tomo_queue_add(label="job1") + + lamni.tomo_queue_show() + + out = capsys.readouterr().out + assert "circfov=15.0" in out + assert "range=" not in out # lamni is 360-only, no angle-range field + + +def test_tomo_queue_execute_runs_fresh_job_then_marks_done(): + lamni = make_lamni() + calls = [] + lamni.tomo_scan = lambda *a, **k: calls.append(("scan", a, k)) + lamni.tomo_scan_resume = lambda *a, **k: calls.append(("resume", a, k)) + + lamni.tomo_queue_add(label="job1") + lamni.tomo_queue_execute() + + assert calls == [("scan", (), {})] + job = lamni._tomo_queue_proxy.as_list()[0] + assert job["status"] == "done" + + +def test_tomo_queue_execute_resumes_incomplete_job(): + lamni = make_lamni() + calls = [] + lamni.tomo_scan = lambda *a, **k: calls.append(("scan", a, k)) + lamni.tomo_scan_resume = lambda *a, **k: calls.append(("resume", a, k)) + + index = lamni.tomo_queue_add(label="job1") + lamni._tomo_queue_proxy.update(index, status="incomplete") + + lamni.tomo_queue_execute() + + assert calls == [("resume", (), {})] + job = lamni._tomo_queue_proxy.as_list()[0] + assert job["status"] == "done" + + +def test_tomo_queue_add_command_move_action(monkeypatch): + lamni = make_lamni() + fake_dev = {"mokev": "mokev-device"} + moved = [] + monkeypatch.setattr(tomo_queue_mixin, "dev", fake_dev, raising=False) + monkeypatch.setattr(tomo_queue_mixin, "umv", lambda *a: moved.append(a), raising=False) + + lamni.tomo_queue_add_command( + {"action": "move", "kwargs": {"positions": {"mokev": 6.2}}}, label="reconfigure" + ) + lamni.tomo_queue_execute() + + assert moved == [("mokev-device", 6.2)] + job = lamni._tomo_queue_proxy.as_list()[0] + assert job["kind"] == "command" + assert job["status"] == "done" + + +def test_tomo_queue_add_command_rejects_non_movable_device(): + lamni = make_lamni() + try: + lamni.tomo_queue_add_command({"action": "move", "kwargs": {"positions": {"fzp": 1.0}}}) + assert False, "expected ValueError for a non-allow-listed device" + except ValueError as exc: + assert "not queue-movable" in str(exc) + + +def test_register_at_each_angle_hook_dispatches_from_at_each_angle(): + lamni = make_lamni() + seen = [] + lamni.register_at_each_angle_hook("record", lambda self, angle: seen.append(angle)) + lamni.at_each_angle_hook = "record" + + lamni._at_each_angle(12.5) + + assert seen == [12.5] + assert lamni.client.get_global_var("tomo_at_each_angle_hooks") == ["record"] + + +def test_at_each_angle_default_path_unchanged_without_hook(): + """No hook registered -- must fall through to the exact same default + behaviour lamni had before this port: tomo_scan_projection + tomo_reconstruct.""" + lamni = make_lamni() + calls = [] + lamni.tomo_scan_projection = lambda angle: calls.append(("projection", angle)) + lamni.tomo_reconstruct = lambda: calls.append(("reconstruct",)) + + lamni._at_each_angle(30.0) + + assert calls == [("projection", 30.0), ("reconstruct",)] + + +def test_at_each_angle_raises_for_unregistered_hook_name(): + lamni = make_lamni() + lamni.at_each_angle_hook = "missing_hook" + try: + lamni._at_each_angle(0.0) + assert False, "expected ValueError for an unregistered hook name" + except ValueError as exc: + assert "missing_hook" in str(exc) + assert "not registered" in str(exc) -- 2.54.0 From bac3fa875426d7f0cccf94132d84c40bce3eb634 Mon Sep 17 00:00:00 2001 From: x01dc Date: Thu, 16 Jul 2026 13:39:08 +0200 Subject: [PATCH 2/5] feat(lamni,bec_widgets): generalize tomo-params GUI to lamni, fix two sub_tomo_scan angle bugs Generalize TomoParamsWidget to lamni via a SETUP_PROFILES mechanism (setup detection, per-setup field lists/order, a new Offsets section, a setup-agnostic "Duplicate job" queue button), fixing two latent TomoQueueDialog bugs that silently mishandled lamni jobs along the way. While verifying the GUI's projection-count preview against the CLI, found two real, pre-existing bugs in LamNI.sub_tomo_scan() unrelated to the GUI itself: a duplicate closing angle every sub-tomogram (360=0 degrees), and a phase offset computed from the raw stepsize instead of the achievable one, breaking the equally-spaced-when- combined guarantee for sub-tomogram pairs/quads/the full set. Both fixed to mirror Flomni's existing, correct equivalents. Also fills in lamni's user documentation with the queue/command-job/ at-each-angle-hook system, which it previously lacked entirely, mirroring flomni.md's coverage. Co-Authored-By: Claude Sonnet 5 --- .../LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md | 162 ++++ .../plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md | 3 +- .../bec_ipython_client/plugins/LamNI/lamni.py | 86 ++- .../widgets/tomo_params/tomo_params.py | 689 +++++++++++++----- docs/user/ptychography/lamni.md | 185 +++++ .../test_lamni_tomo_angles.py | 127 ++++ .../test_lamni_tomo_params_widget_math.py | 167 +++++ 7 files changed, 1223 insertions(+), 196 deletions(-) create mode 100644 csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md create mode 100644 tests/tests_bec_ipython_client/test_lamni_tomo_angles.py create mode 100644 tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md new file mode 100644 index 0000000..92b6ae6 --- /dev/null +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md @@ -0,0 +1,162 @@ +# Tomo-params GUI generalization + two sub_tomo_scan() angle bugs fixed + +Companion to `AI_docs/TOMO_QUEUE_PORT.md` (the backend port this GUI work +builds on) and `csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md` +(which identified `TomoParamsWidget` as flomni-gated). This session's +follow-up work: generalized the widget to also support lamni, then — while +verifying the widget's projection-count preview against the CLI — found and +fixed two real, pre-existing bugs in `LamNI.sub_tomo_scan()` that predate +this session and affected every lamni tomogram ever run with a non-trivially- +achievable `tomo_angle_stepsize`. + +**Status:** All of the below implemented, tested, and live-verified against +the running simulated lamni deployment. Not yet committed as of this doc. + +## 1. GUI: `TomoParamsWidget` generalized to lamni + +`csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py`'s `SETUP_PROFILES` +dict (one entry per setup) now drives nearly everything setup-specific: + +- **Setup detection**: `_detect_setup()` checks for `fsamroy` (flomni) or + `lsamrot` (lamni) — real discriminator devices, same pattern the CLI side + already used, just generalized from a yes/no flomni check. +- **Per-setup field lists**: `fov_fields`/`stitch_fields`/`manual_shift_fields`/ + `piezo_range_fields`/`offset_fields`, each a list of `(key, label, min, max, + decimals)` tuples the widget iterates to build/validate/read fields — + small per-concept lists, not a generic form builder. +- **`field_order`**: an explicit per-profile list of section tokens + (`_build_params_panel` is now a dispatch loop over it). flomni's order is + its unchanged, pre-existing order (verified as a no-op via a regression + test); lamni's order mirrors `lamni.tomo_parameters()`'s own print/edit + order exactly (counting time → shell step → piezo range → stitching → + stitch overlap → circular FOV → reconstruct folder → frames/trigger → + offsets → at-each-angle hook). +- **Offsets section** (lamni only): `tomo_fovx_offset`/`tomo_fovy_offset` + (from `LamNIAlignmentMixin`) plus `manual_shift_x`/`_y`, with a "?" help + button. These are alignment values, not scan parameters — shown/editable + here but **excluded from queue job snapshots** (`add_edited_to_queue()`/ + `TomoQueueDialog.add_to_queue()` now explicitly filter to + `self._profile["param_names"]`), same treatment as flomni's + `corr_pos_y`/`corr_angle_y` (also alignment-only, also not shown by this + widget). Needed dedicated accessors (`_lamni_get_tomo_fov_offset`/ + `_lamni_set_tomo_fov_offset`) because both axes are packed into one + combined global var (`tomo_fov_offset = [x_um, y_um]`, exposed in mm) — + the widget's usual direct-global-var-by-name mechanism doesn't work for + them. +- **Lamni projection math** (`_lamni_compute_type1`/`_lamni_requested_to_stepsize`): + see §2 below — simplified once the backend bug was fixed, now the same + shape as flomni's math (`N*8`, base angle 360 instead of 180). +- **`TomoQueueDialog` bugs fixed** (pre-existing, silently wrong for any + lamni job before this session, since no lamni job had ever been queued + through the GUI): `_format_projections()` always used flomni's fixed-180 + formula regardless of which setup a job belonged to; `_job_tooltip()` + iterated a hardcoded flomni-only param-name tuple, so a lamni job's + tooltip silently showed no params at all. Both now duck-type off the + job's own params dict (`"tomo_angle_range" in params` distinguishes a + flomni job from a lamni job) rather than needing to know which setup is + active. +- **New "Duplicate selected" queue button**: copies the selected job (any + status, any kind — tomo or command) to the end of the queue with a fresh + id, `status="pending"`, and `_dup` appended to the label. Setup-agnostic, + no profile-specific logic. + +## 2. Backend: two real bugs found and fixed in `LamNI.sub_tomo_scan()` + +Surfaced while checking the GUI's projection-count preview against +`lamni.tomo_parameters()`'s wizard — both predate this session and are +independent of the GUI work; they affect the CLI (and therefore every real +scan) regardless of whether the GUI is ever used. + +### 2a. Duplicate closing angle + +`sub_tomo_scan()` generated each sub-tomogram's angles with +`np.linspace(start, start+360, num=int(360/stepsize)+1, endpoint=True)`. For +a full 360° sweep, `start+360` is the *same physical angle* as `start` +(360°≡0°), so every sub-tomogram re-measured its own starting angle a +second time. Confirmed numerically: requesting 66 projections set +`tomo_angle_stepsize=43.6364`; the scan then actually ran **72** acquisitions +(9/sub-tomogram × 8), not 66 — `tomo_parameters()`'s own "Resulting in +number of projections" printout was never accurate for *any* input, since it +used plain float division with no `int()` truncation and no duplicate-point +awareness. + +Fix: new `LamNI._tomo_type1_actual_grid()` (mirrors +`Flomni._tomo_type1_actual_grid()`, base angle 360 instead of 180, no `+1` +since the closing point is a duplicate not a new measurement) is now the +single source of truth for `N`/achievable step/total, used by +`sub_tomo_scan()` (now `num=N, endpoint=False`), `tomo_parameters()`, and +`write_pdf_report()` alike — no more three independent, disagreeing +formulas. + +### 2b. Phase offset computed from the wrong step + +Separately, `sub_tomo_scan()`'s inter-sub-tomogram phase offset was +`self.tomo_angle_stepsize / 8.0 * offsets[subtomo_number]` — using the +**raw, requested** stepsize, not the achievable one (`360/N` after `int()` +truncation). These only coincide when `360/stepsize` happens to already be a +whole number. Flomni's equivalent (`_subtomo_angle_plan()`) already gets +this right and says why in its own comment: *"N/step not guaranteed to be a +whole/exact division of the configured tomo_angle_stepsize ... This +corrected step ... is used for BOTH the per-point ramp AND the +inter-sub-tomogram phase offsets."* Lamni's phase offset never got that +treatment. + +Verified numerically (stepsize=43.6364, achievable step=45): combining +sub-tomograms in pairs `(1,2)`/`(3,4)`/`(5,6)`/`(7,8)`, quads +`(1,2,3,4)`/`(5,6,7,8)`, or all 8 produced **unevenly spaced** angles with +the raw-stepsize phase (e.g. pair spacing alternating 21.8°/23.2° instead of +a uniform 22.5°) — defeating the entire point of the bit-reversal +interlacing scheme, which exists specifically to give progressively finer, +*evenly* sampled combined tomograms at each combination level. Fixed by +computing the phase from the achievable step instead, exactly like flomni. + +Also added, matching flomni's wizard UX: `tomo_parameters()` now prints +`"Note: N projections does not divide evenly into 8 equally spaced +sub-tomograms over 360 degrees; adjusted to the nearest achievable total of +M projections..."` when the requested and achievable totals differ — this +is **display only**, not a substitute for the `sub_tomo_scan()` fix: the +stored `tomo_angle_stepsize` is never rewritten to the achievable value (in +flomni either — verified by reading its wizard code), so `sub_tomo_scan()` +must keep re-deriving the achievable grid every time it runs, regardless of +which code path set `tomo_angle_stepsize` (wizard, GUI Submit, queue-job +restore, or a direct script assignment). + +## Explicitly deferred (carried over / reconfirmed) + +- `zero_deg_reference_at_each_subtomo` for lamni — not implemented anywhere + in `lamni.py` (no property, no scan logic). Mirko confirmed: separate + future task, backend first then GUI. +- Shared angle-calculation code between flomni/lamni (`sub_tomo_scan`/ + `_golden`/`tomo_scan` dispatch) — Mirko confirmed: separate follow-up, not + folded into this GUI-focused work, despite the two setups' type-1 math now + having the same *shape* (`N*8`, differing only in base angle). +- `LamniWebpageGenerator` completion — unrelated gap, still open. + +## Verification + +- New tests: `tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py` + (16 tests: profile field_order/param_names/defaults guards, offset + getter/setter round-trip including a same-axis-doesn't-clobber-the-other + check) and `test_lamni_tomo_angles.py` (48 tests: no-duplicate-angle check + per sub-tomogram/stepsize, and the pair/quad/full-8 equally-spaced + invariant per stepsize). The equally-spaced tests were sanity-checked by + temporarily reverting the phase-offset fix and confirming 9/12 of them + fail as expected, then restoring it. +- Existing pinned test `test_tomo_params_widget_math.py` (flomni's math) + untouched and still passing. +- Full offline suite (`pytest tests/tests_bec_ipython_client tests/tests_scans`): + 185 passed. +- Live-sim verification (plain scripts against the running simulated lamni + deployment, snapshotting/restoring `tomo_queue`/`tomo_progress` around + each run): + - Real `sub_tomo_scan(1)` with a coarse stepsize (170°, N=2): exactly 2 + real acquisitions (`[0.0, 180.0]`), no duplicate; `tomo_parameters()`'s + display matched (16 = 2×8) after the fix (was previously wrong for + every input). + - Real `sub_tomo_scan(1)` + `sub_tomo_scan(2)` with a non-achievable + stepsize (109.09°, N=3): combined pair sorted to + `[0, 60, 120, 180, 240, 300]`, spacing exactly 60° throughout — + confirms the phase-offset fix holds against real (not just mocked) + execution. +- No live GUI verification performed (per Mirko's instruction) — verified + manually by Mirko instead. diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md index 9531ad5..6137548 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md @@ -74,7 +74,8 @@ one-line addition to `LamNI._TOMO_QUEUE_MOVE_DEVICES`. ## Explicitly deferred -- `TomoParamsWidget` GUI changes — still flomni-gated (`_check_flomni_available()`). +- `TomoParamsWidget` GUI changes — **done, see `AI_docs/TOMO_PARAMS_GUI_PORT.md`** + (was still flomni-gated when this doc was first written). - `LamniWebpageGenerator` completion — separate, unrelated gap (see `FLOMNI_TO_LAMNI_COMPARISON.md` §4). diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py index 7743124..c095998 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py @@ -817,21 +817,48 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools # Tomo scan orchestration # ------------------------------------------------------------------ + def _tomo_type1_actual_grid(self) -> tuple[int, float, int]: + """Compute the actual (achievable) tomo_type==1 grid from the + currently stored self.tomo_angle_stepsize -- the SAME way + sub_tomo_scan() does it. Returns (N, step, total_projections): N + unique angles per sub-tomogram, achievable per-projection step, + N*8 total. + + A full 360-degree sweep's closing angle (start_angle + 360) is the + same physical angle as its start (360 deg == 0 deg), so N (not N+1) + is the right point count for one sub-tomogram -- see + sub_tomo_scan()'s linspace call, endpoint=False for the same reason. + Mirrors Flomni._tomo_type1_actual_grid() (base angle 360 instead of + 180, no "+1": flomni's own 180-degree-span scheme never had a + self-duplicating endpoint to begin with). + """ + N = int(360.0 / self.tomo_angle_stepsize) + step = 360.0 / N + return N, step, N * 8 + def sub_tomo_scan(self, subtomo_number, start_angle=None): """Perform one sub-tomogram (tomo_type 1 only).""" self._write_subtomo_to_scilog(subtomo_number) + N, achievable_step, total_projections = self._tomo_type1_actual_grid() + if start_angle is None: + # Phase offset must be a fraction of the ACHIEVABLE step (after + # int() truncation), not the raw configured tomo_angle_stepsize -- + # they only coincide when 360/tomo_angle_stepsize happens to + # already be a whole number. Using the raw value here breaks the + # interlacing scheme: sub-tomograms combined in pairs/quads/all-8 + # land on unevenly-spaced angles instead of the intended + # progressively finer, evenly-spaced grid at every combination + # level. Mirrors Flomni._subtomo_angle_plan() exactly. offsets = {1: 0, 2: 4, 3: 2, 4: 6, 5: 1, 6: 5, 7: 3, 8: 7} - start_angle = self.tomo_angle_stepsize / 8.0 * offsets[subtomo_number] + start_angle = achievable_step / 8.0 * offsets[subtomo_number] angle_end = start_angle + 360 - angles = np.linspace( - start_angle, - angle_end, - num=int(360 / self.tomo_angle_stepsize) + 1, - endpoint=True, - ) + # endpoint=False: a full 360-degree sweep must NOT re-measure both + # start_angle and start_angle+360 -- they're the same physical angle, + # unlike a [0, 180) span's two distinct endpoints. + angles = np.linspace(start_angle, angle_end, num=N, endpoint=False) if not (subtomo_number % 2): angles = np.flip(angles) @@ -840,12 +867,11 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools self.progress["tomo_type"] = "Equally spaced sub-tomograms" self.progress["subtomo"] = subtomo_number self.progress["subtomo_projection"] = np.where(angles == angle)[0][0] - self.progress["subtomo_total_projections"] = 360 / self.tomo_angle_stepsize - self.progress["projection"] = ( - (subtomo_number - 1) * self.progress["subtomo_total_projections"] - + self.progress["subtomo_projection"] - ) - self.progress["total_projections"] = 360 / self.tomo_angle_stepsize * 8 + self.progress["subtomo_total_projections"] = N + self.progress["projection"] = (subtomo_number - 1) * N + self.progress[ + "subtomo_projection" + ] + self.progress["total_projections"] = total_projections self.progress["angle"] = angle self._tomo_scan_at_angle(angle, subtomo_number) @@ -1154,8 +1180,13 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools print("") if self.tomo_type == 1: print("\x1b[1mTomo type 1:\x1b[0m 8 equally spaced sub-tomograms (360 deg)") - print(f"Angular step within sub-tomogram: {self.tomo_angle_stepsize} degrees") - print(f"Resulting in number of projections: {360/self.tomo_angle_stepsize*8}") + # _tomo_type1_actual_grid() is the same computation sub_tomo_scan() + # actually runs -- reading self.tomo_angle_stepsize/naive division + # directly here previously showed a number that didn't match what + # a real scan acquired (see the helper's docstring). + _, achievable_step, total_projections = self._tomo_type1_actual_grid() + print(f"Angular step within sub-tomogram: {achievable_step} degrees") + print(f"Resulting in number of projections: {total_projections}") elif self.tomo_type == 2: print("\x1b[1mTomo type 2:\x1b[0m Golden ratio tomography") print(f"Sorted in bunches of: {self.golden_ratio_bunch_size}") @@ -1207,11 +1238,23 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools self.tomo_type = self._get_val("Tomography type", self.tomo_type, int) if self.tomo_type == 1: - tomo_numberofprojections = self._get_val( - "Number of projections", 360 / self.tomo_angle_stepsize * 8, int - ) + _, _, current_total = self._tomo_type1_actual_grid() + tomo_numberofprojections = self._get_val("Number of projections", current_total, int) self.tomo_angle_stepsize = 360 / tomo_numberofprojections * 8 - print(f"Angular step in a subtomogram: {self.tomo_angle_stepsize}") + # Report what was ACTUALLY achieved, via the same helper + # sub_tomo_scan() itself uses -- not the raw value just typed in, + # which can silently disagree with what the scan will actually + # run (mirrors Flomni.tomo_parameters()'s equivalent note). + _, achievable_step, actual_total = self._tomo_type1_actual_grid() + if actual_total != tomo_numberofprojections: + print( + f"Note: {tomo_numberofprojections} projections does not divide evenly " + "into 8 equally spaced sub-tomograms over 360 degrees; adjusted to the " + f"nearest achievable total of {actual_total} projections to keep the " + "angular grid uniform." + ) + print(f"Angular step in a subtomogram: {achievable_step}") + print(f"Actual number of projections: {actual_total}") elif self.tomo_type == 2: while True: @@ -1280,15 +1323,16 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools piezo_range = f"{self.lamni_piezo_range_x:.2f}/{self.lamni_piezo_range_y:.2f}" stitching = f"{self.lamni_stitch_x:.2f}/{self.lamni_stitch_y:.2f}" dataset_id = str(self.client.queue.next_dataset_number) + _, _, report_total_projections = self._tomo_type1_actual_grid() content = [ f"{'Sample Name:':<{padding}}{self.sample_name:>{padding}}\n", f"{'Measurement ID:':<{padding}}{str(self.tomo_id):>{padding}}\n", f"{'Dataset ID:':<{padding}}{dataset_id:>{padding}}\n", f"{'Sample Info:':<{padding}}{'Sample Info':>{padding}}\n", f"{'e-account:':<{padding}}{str(self.client.username):>{padding}}\n", - f"{'Number of projections:':<{padding}}{int(360 / self.tomo_angle_stepsize * 8):>{padding}}\n", + f"{'Number of projections:':<{padding}}{report_total_projections:>{padding}}\n", f"{'First scan number:':<{padding}}{self.client.queue.next_scan_number:>{padding}}\n", - f"{'Last scan number approx.:':<{padding}}{self.client.queue.next_scan_number + int(360 / self.tomo_angle_stepsize * 8) + 10:>{padding}}\n", + f"{'Last scan number approx.:':<{padding}}{self.client.queue.next_scan_number + report_total_projections + 10:>{padding}}\n", f"{'Current photon energy:':<{padding}}{dev.mokev.read(cached=True)['value']:>{padding}.4f}\n", f"{'Exposure time:':<{padding}}{self.tomo_countingtime:>{padding}.2f}\n", f"{'Fermat spiral step size:':<{padding}}{self.tomo_shellstep:>{padding}.2f}\n", diff --git a/csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py b/csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py index fa6812c..7e5610c 100644 --- a/csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py +++ b/csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py @@ -1,10 +1,11 @@ """ -TomoParamsWidget – BEC widget for editing FlOMNI tomo scan parameters and -managing the tomo scan queue. +TomoParamsWidget – BEC widget for editing tomo scan parameters and managing +the tomo scan queue, for whichever cSAXS tomography setup (flomni, lamni, ...) +is active in the current BEC session -- see ``SETUP_PROFILES``. Parameters are stored as BEC global vars; this widget reads and writes them via ``self.client.get_global_var`` / ``self.client.set_global_var``, which -is the same mechanism used by the ``Flomni`` CLI object. +is the same mechanism used by the ``Flomni``/``LamNI`` CLI objects. Polling ------- @@ -32,14 +33,15 @@ projection — but Submit asks for confirmation. Queue execution --------------- -``tomo_queue_execute()`` is a blocking Flomni CLI method that cannot be -called directly from the GUI server process. This widget manages the queue +``tomo_queue_execute()`` is a blocking CLI method (see ``TomoQueueMixin``) +that cannot be called directly from the GUI server process. This widget manages the queue via direct global-var manipulation (add / delete / clear) and shows an explicit hint for executing it from the CLI. """ from __future__ import annotations +import copy import datetime import uuid from typing import Any, Optional @@ -143,6 +145,56 @@ DEFAULTS: dict[str, Any] = { "at_each_angle_hook": None, } +# Exact tuple from LamNI._TOMO_SCAN_PARAM_NAMES (source of truth in lamni.py). +# No tomo_angle_range/single_point_*/zero_deg_reference_at_each_subtomo -- +# lamni is laminography: always 360 degrees, no single-point acquisition, no +# per-subtomo zero-deg reference. lamni_piezo_range_x/y have no flomni +# equivalent; manual_shift_x is lamni-only (flomni only exposes shift_y). +LAMNI_QUEUE_PARAM_NAMES = ( + "tomo_countingtime", + "tomo_shellstep", + "tomo_circfov", + "lamni_stitch_x", + "lamni_stitch_y", + "tomo_stitch_overlap", + "ptycho_reconstruct_foldername", + "manual_shift_x", + "manual_shift_y", + "lamni_piezo_range_x", + "lamni_piezo_range_y", + "frames_per_trigger", + "tomo_type", + "tomo_angle_stepsize", + "golden_ratio_bunch_size", + "golden_max_number_of_projections", + "golden_projections_at_0_deg_for_damage_estimation", + "corridor_size", + "at_each_angle_hook", +) + +# Read directly off each LamNI property getter's None-fallback in lamni.py. +LAMNI_DEFAULTS: dict[str, Any] = { + "tomo_countingtime": 0.1, + "tomo_shellstep": 1, + "tomo_circfov": 0.0, + "lamni_stitch_x": 0, + "lamni_stitch_y": 0, + "tomo_stitch_overlap": 0.2, + "ptycho_reconstruct_foldername": "ptycho_reconstruct", + "manual_shift_x": 0.0, + "manual_shift_y": 0.0, + "lamni_piezo_range_x": 20, + "lamni_piezo_range_y": 20, + "frames_per_trigger": 1, + "tomo_type": 1, + "tomo_angle_stepsize": 10.0, + "golden_ratio_bunch_size": 20, + "golden_max_number_of_projections": 1000.0, + "golden_projections_at_0_deg_for_damage_estimation": 0, + "corridor_size": -1, + "at_each_angle_hook": None, +} + def _hline() -> QFrame: line = QFrame() @@ -156,7 +208,8 @@ def _hline() -> QFrame: class TomoParamsWidget(BECWidget, QWidget): """ - Interactive GUI for FlOMNI tomo scan parameter editing and queue management. + Interactive GUI for tomo scan parameter editing and queue management, for + whichever setup (see ``SETUP_PROFILES``) is active in this BEC session. Layout ------ @@ -196,13 +249,13 @@ class TomoParamsWidget(BECWidget, QWidget): self._pw: dict[str, QWidget] = {} # key -> input widget self._queue_dlg: Optional["TomoQueueDialog"] = None self._busy_banner: Optional[QLabel] = None - # check whether the FlOMNI setup is active before building the UI - self._flomni_available = self._check_flomni_available() + # detect which known setup (if any) is active before building the UI + self._profile = SETUP_PROFILES.get(self._detect_setup()) self._build_ui() self._poll_timer = QTimer(self) self._poll_timer.setInterval(self._POLL_INTERVAL_MS) self._poll_timer.timeout.connect(self._on_poll) - if self._flomni_available: + if self._profile is not None: self._poll_timer.start() self.refresh() # live beamline-busy banner: react to scan-queue changes as they @@ -217,16 +270,21 @@ class TomoParamsWidget(BECWidget, QWidget): # ── setup detection ────────────────────────────────────────────────────── - def _check_flomni_available(self) -> bool: + def _detect_setup(self) -> Optional[str]: """ - Return True if the FlOMNI setup is active in this BEC session. - Uses ``fsamroy`` (the FlOMNI-specific rotation stage) as the - discriminator — it is absent in OMNY and LamNI sessions. + Return the key of whichever known setup (see ``SETUP_PROFILES``) is + active in this BEC session, or None if none is detected. Each + profile's ``discriminator_attr`` is a device unique to that setup + (e.g. flomni's own rotation stage, ``fsamroy``) -- a BEC session is + always exactly one setup, never more than one at a time. """ - try: - return getattr(self.dev, "fsamroy", None) is not None - except Exception: - return False + for setup, profile in SETUP_PROFILES.items(): + try: + if getattr(self.dev, profile["discriminator_attr"], None) is not None: + return setup + except Exception: + continue + return None # ── global-var I/O ─────────────────────────────────────────────────────── @@ -247,9 +305,15 @@ class TomoParamsWidget(BECWidget, QWidget): def _load_params(self) -> dict[str, Any]: params = {} - for key in QUEUE_PARAM_NAMES: + for key in self._profile["param_names"]: val = self._gv_get(key) - params[key] = val if val is not None else DEFAULTS.get(key) + params[key] = val if val is not None else self._profile["defaults"].get(key) + # offset_fields (e.g. lamni's tomo_fovx_offset/tomo_fovy_offset) are + # NOT part of param_names (alignment values, not queue params) and + # don't map to a same-named global var -- read via the profile's + # own getter instead of _gv_get(). + for key, _label, _min, _max, _decimals, axis in self._profile["offset_fields"]: + params[key] = self._profile["offset_getter"](self.client, axis) return params def _load_queue(self) -> list[dict]: @@ -284,11 +348,14 @@ class TomoParamsWidget(BECWidget, QWidget): self._busy_banner.setVisible(False) root.addWidget(self._busy_banner) - if not self._flomni_available: + if self._profile is None: + known = ", ".join( + f"{p['label']} ({p['discriminator_attr']})" for p in SETUP_PROFILES.values() + ) lbl = QLabel( - "⚠ FlOMNI setup not detected in this BEC session.\n\n" - "This widget is only meaningful when the FlOMNI\n" - "BEC session is active (fsamroy device present)." + "⚠ No known tomo setup detected in this BEC session.\n\n" + "This widget is only meaningful when one of the following " + f"is active: {known}." ) lbl.setAlignment(Qt.AlignmentFlag.AlignCenter) lbl.setWordWrap(True) @@ -340,46 +407,86 @@ class TomoParamsWidget(BECWidget, QWidget): vbox.addLayout(type_row) vbox.addWidget(_hline()) - # common parameters + # common parameters -- built by dispatching over self._profile["field_order"] + # so build ORDER (not just which fields appear) can differ per setup; + # mirrors lamni.tomo_parameters()'s exact print/edit order for lamni. + self._lbl_single_point_hook_warning = None common_form = QFormLayout() common_form.setLabelAlignment(Qt.AlignmentFlag.AlignRight) - self._add_double( - common_form, - "tomo_countingtime", - "Counting time (s)", - min_=0.001, - max_=100.0, - decimals=3, - ) - self._add_double( - common_form, "tomo_shellstep", "Shell step (µm)", min_=0.001, max_=1000.0, decimals=3 - ) - self._add_double(common_form, "fovx", "FOV x (µm)", min_=0.1, max_=200.0, decimals=2) - self._add_double(common_form, "fovy", "FOV y (µm)", min_=0.1, max_=100.0, decimals=2) - self._add_int(common_form, "stitch_x", "Stitch x", min_=0, max_=50) - self._add_int(common_form, "stitch_y", "Stitch y", min_=0, max_=50) - self._add_double( - common_form, - "tomo_stitch_overlap", - "Stitch overlap (µm)", - min_=0.0, - max_=50.0, - decimals=3, - ) - self._add_text(common_form, "ptycho_reconstruct_foldername", "Reconstruct folder") - self._add_double( - common_form, - "manual_shift_y", - "Manual shift y (µm)", - min_=-1000.0, - max_=1000.0, - decimals=3, - ) - self._add_int(common_form, "frames_per_trigger", "Frames / trigger", min_=1, max_=100) - sp = self._add_bool(common_form, "single_point_instead_of_fermat_scan", "Single-point scan") + token_builders = { + "tomo_countingtime": lambda: self._add_double( + common_form, + "tomo_countingtime", + "Counting time (s)", + min_=0.001, + max_=100.0, + decimals=3, + ), + "tomo_shellstep": lambda: self._add_double( + common_form, + "tomo_shellstep", + "Shell step (µm)", + min_=0.001, + max_=1000.0, + decimals=3, + ), + "fov": lambda: self._build_double_group(common_form, self._profile["fov_fields"]), + "stitch": lambda: self._build_int_group(common_form, self._profile["stitch_fields"]), + "tomo_stitch_overlap": lambda: self._add_double( + common_form, + "tomo_stitch_overlap", + "Stitch overlap (µm)", + min_=0.0, + max_=50.0, + decimals=3, + ), + "ptycho_reconstruct_foldername": lambda: self._add_text( + common_form, "ptycho_reconstruct_foldername", "Reconstruct folder" + ), + "manual_shift": lambda: self._build_double_group( + common_form, self._profile["manual_shift_fields"] + ), + "piezo_range": lambda: self._build_double_group( + common_form, self._profile["piezo_range_fields"] + ), + "frames_per_trigger": lambda: self._add_int( + common_form, "frames_per_trigger", "Frames / trigger", min_=1, max_=100 + ), + "single_point": lambda: self._build_single_point_section(common_form), + "offsets": lambda: self._build_offsets_section(common_form), + "at_each_angle_hook": lambda: self._build_at_each_angle_hook_row(common_form), + } + for token in self._profile["field_order"]: + token_builders[token]() + vbox.addLayout(common_form) + + # type-1 section + self._sec_type1 = self._build_type1_section() + vbox.addWidget(self._sec_type1) + + # type-2/3 section + self._sec_type23 = self._build_type23_section() + vbox.addWidget(self._sec_type23) + + return box + + def _build_double_group( + self, form: QFormLayout, fields: list[tuple[str, str, float, float, int]] + ) -> None: + for key, label, min_, max_, decimals in fields: + self._add_double(form, key, label, min_=min_, max_=max_, decimals=decimals) + + def _build_int_group(self, form: QFormLayout, fields: list[tuple[str, str, int, int]]) -> None: + for key, label, min_, max_ in fields: + self._add_int(form, key, label, min_=min_, max_=max_) + + def _build_single_point_section(self, form: QFormLayout) -> None: + if not self._profile["has_single_point"]: + return + sp = self._add_bool(form, "single_point_instead_of_fermat_scan", "Single-point scan") sp.stateChanged.connect(self._on_single_point_changed) self._add_double( - common_form, + form, "single_point_random_shift_max", "Random shift max (µm)", min_=0.0, @@ -395,40 +502,67 @@ class TomoParamsWidget(BECWidget, QWidget): self._lbl_single_point_hook_warning.setWordWrap(True) self._lbl_single_point_hook_warning.setStyleSheet("color: #B8860B;") self._lbl_single_point_hook_warning.setVisible(False) - common_form.addRow("", self._lbl_single_point_hook_warning) + form.addRow("", self._lbl_single_point_hook_warning) + + def _build_offsets_section(self, form: QFormLayout) -> None: + """FOV-offset + manual-shift group (lamni only -- see + SETUP_PROFILES["lamni"]["offset_fields"]). tomo_fovx_offset/ + tomo_fovy_offset are alignment values (LamNIAlignmentMixin), not + scan parameters -- shown here for convenience, mirroring + lamni.tomo_parameters(), but excluded from queue job snapshots + (see add_edited_to_queue()/_load_params()). + """ + if not self._profile["offset_fields"]: + return + header_row = QHBoxLayout() + header_row.addWidget(QLabel("Offsets")) + btn_help = QPushButton("?") + btn_help.setFixedWidth(24) + btn_help.setToolTip("What do these offsets do?") + btn_help.clicked.connect(self._show_offsets_help) + header_row.addWidget(btn_help) + header_row.addStretch() + form.addRow(header_row) + for key, label, min_, max_, decimals, _axis in self._profile["offset_fields"]: + self._add_double(form, key, label, min_=min_, max_=max_, decimals=decimals) + self._build_double_group(form, self._profile["manual_shift_fields"]) + + def _show_offsets_help(self) -> None: + QMessageBox.information( + self, + "Offsets", + "FOV offset rotates to find the region of interest. The initial " + "values were determined in the xrayeye alignment step. The " + "manual shifts move the rotation center.", + ) + + def _build_at_each_angle_hook_row(self, form: QFormLayout) -> None: self._pw["at_each_angle_hook"] = QComboBox() self._pw["at_each_angle_hook"].setEnabled(False) self._pw["at_each_angle_hook"].setToolTip( "Run a registered custom function at every projection angle instead " "of the normal acquisition (e.g. move a polarizer in/out between " "exposures). Registered from the CLI with " - "flomni.register_at_each_angle_hook(name, func) -- this list refreshes " + "register_at_each_angle_hook(name, func) -- this list refreshes " "automatically while the dialog is open." ) - common_form.addRow("At-each-angle hook:", self._pw["at_each_angle_hook"]) - vbox.addLayout(common_form) - - # type-1 section - self._sec_type1 = self._build_type1_section() - vbox.addWidget(self._sec_type1) - - # type-2/3 section - self._sec_type23 = self._build_type23_section() - vbox.addWidget(self._sec_type23) - - return box + form.addRow("At-each-angle hook:", self._pw["at_each_angle_hook"]) def _build_type1_section(self) -> QGroupBox: box = QGroupBox("Type 1 – sub-tomogram settings") form = QFormLayout(box) form.setLabelAlignment(Qt.AlignmentFlag.AlignRight) - # angle range combo - self._pw["tomo_angle_range"] = QComboBox() - for v in (180, 360): - self._pw["tomo_angle_range"].addItem(f"{v}°", v) - self._pw["tomo_angle_range"].currentIndexChanged.connect(self._update_projection_preview) - form.addRow("Angle range:", self._pw["tomo_angle_range"]) + # angle range combo -- only for setups with an actual 180/360 toggle; + # lamni is laminography and always scans the full 360 degrees. + if self._profile["has_180_mode"]: + self._pw["tomo_angle_range"] = QComboBox() + for v in (180, 360): + self._pw["tomo_angle_range"].addItem(f"{v}°", v) + self._pw["tomo_angle_range"].currentIndexChanged.connect( + self._update_projection_preview + ) + form.addRow("Angle range:", self._pw["tomo_angle_range"]) # requested total projections (user entry in edit mode) self._pw["_requested_total"] = QSpinBox() @@ -444,7 +578,8 @@ class TomoParamsWidget(BECWidget, QWidget): form.addRow("Actual achievable total:", self._lbl_actual_total) form.addRow("Angular step (°):", self._lbl_achievable_step) - self._add_bool(form, "zero_deg_reference_at_each_subtomo", "0° reference each sub-tomo") + if self._profile["has_zero_deg_reference"]: + self._add_bool(form, "zero_deg_reference_at_each_subtomo", "0° reference each sub-tomo") return box def _build_type23_section(self) -> QGroupBox: @@ -567,7 +702,7 @@ class TomoParamsWidget(BECWidget, QWidget): # derive requested_total from stored tomo_angle_stepsize angle_range = int(params.get("tomo_angle_range", 180)) stepsize = float(params.get("tomo_angle_stepsize", 10.0)) - actual_total, _, _ = _compute_type1(angle_range, stepsize) + actual_total, _, _ = self._profile["compute_type1"](angle_range, stepsize) self._pw["_requested_total"].blockSignals(True) self._pw["_requested_total"].setValue(actual_total) self._pw["_requested_total"].blockSignals(False) @@ -575,9 +710,11 @@ class TomoParamsWidget(BECWidget, QWidget): self._update_projection_preview() # type-3: derive projections-per-subtomo from stored tomo_angle_stepsize - # (flomni uses 180 / tomo_angle_stepsize for type 3, hardcoded to 180) + # (base angle is 180 for flomni, 360 for lamni -- see + # self._profile["type1_base_angle"]) stepsize = float(params.get("tomo_angle_stepsize", 10.0)) - numprj_t3 = int(180.0 / stepsize) if stepsize > 0 else 18 + base_angle = self._profile["type1_base_angle"] + numprj_t3 = int(base_angle / stepsize) if stepsize > 0 else int(base_angle / 10.0) self._pw["_numprj_type3"].blockSignals(True) self._pw["_numprj_type3"].setValue(numprj_t3) self._pw["_numprj_type3"].blockSignals(False) @@ -586,9 +723,9 @@ class TomoParamsWidget(BECWidget, QWidget): def _populate_at_each_angle_hook_combo(self, current: Optional[str]) -> None: """Rebuild the at_each_angle_hook dropdown from the live - ``tomo_at_each_angle_hooks`` registry (published by - ``Flomni.register_/unregister_at_each_angle_hook()``) and select - ``current``. + ``tomo_at_each_angle_hooks`` registry (published by the active CLI + object's ``register_/unregister_at_each_angle_hook()``, see + ``TomoQueueMixin``) and select ``current``. ``current`` is added as its own entry even if it isn't in the published list -- e.g. it was registered in a different, or a @@ -632,44 +769,43 @@ class TomoParamsWidget(BECWidget, QWidget): # derive tomo_angle_stepsize from requested_total + angle_range (type 1) angle_range = int(params.get("tomo_angle_range", 180)) requested = self._pw["_requested_total"].value() - stepsize = _requested_to_stepsize(angle_range, requested) + stepsize = self._profile["requested_to_stepsize"](angle_range, requested) params["tomo_angle_stepsize"] = stepsize # type 3: override tomo_angle_stepsize from projections-per-subtomo - # (flomni uses 180 / numprj for type 3) + # (base angle is 180 for flomni, 360 for lamni) if params.get("tomo_type") == 3: numprj = self._pw["_numprj_type3"].value() if numprj > 0: - params["tomo_angle_stepsize"] = 180.0 / numprj + params["tomo_angle_stepsize"] = self._profile["type1_base_angle"] / numprj # single_point forces stitch to 0 - if params.get("single_point_instead_of_fermat_scan"): - params["stitch_x"] = 0 - params["stitch_y"] = 0 + if self._profile["has_single_point"] and params.get("single_point_instead_of_fermat_scan"): + for key, _label, _min, _max in self._profile["stitch_fields"]: + params[key] = 0 return params # ── validation ──────────────────────────────────────────────────────────── def _validate(self, params: dict[str, Any]) -> Optional[str]: - if params.get("fovx", 0) > 200: - return "fovx must be ≤ 200 µm" - if params.get("fovy", 0) > 100: - return "fovy must be ≤ 100 µm" - if not isinstance(params.get("stitch_x"), int): - return "stitch_x must be an integer" - if not isinstance(params.get("stitch_y"), int): - return "stitch_y must be an integer" + for key, label, _min, max_, _decimals in self._profile["fov_fields"]: + if params.get(key, 0) > max_: + return f"{label} must be ≤ {max_} µm" + for key, _label, _min, _max in self._profile["stitch_fields"]: + if not isinstance(params.get(key), int): + return f"{key} must be an integer" fpt = params.get("frames_per_trigger", 1) if not isinstance(fpt, int) or fpt < 1 or isinstance(fpt, bool): return "frames_per_trigger must be a positive integer" if params.get("tomo_type") not in (1, 2, 3): return "tomo_type must be 1, 2, or 3" - if params.get("tomo_angle_range") not in (180, 360): + if self._profile["has_180_mode"] and params.get("tomo_angle_range") not in (180, 360): return "tomo_angle_range must be 180 or 360" - rshift = params.get("single_point_random_shift_max", 0.0) - if not 0 <= rshift <= 10: - return "single_point_random_shift_max must be between 0 and 10 µm" + if self._profile["has_single_point"]: + rshift = params.get("single_point_random_shift_max", 0.0) + if not 0 <= rshift <= 10: + return "single_point_random_shift_max must be between 0 and 10 µm" return None # ── type visibility ─────────────────────────────────────────────────────── @@ -705,13 +841,15 @@ class TomoParamsWidget(BECWidget, QWidget): # ── type-1 projection preview ───────────────────────────────────────────── def _update_projection_preview(self) -> None: + # angle_range only exists as a widget for setups with a 180/360 + # toggle (self._profile["has_180_mode"]); for others (e.g. lamni, + # always 360) the profile's math functions ignore this value + # entirely, so any fallback here is a no-op, not a guess. angle_range_widget = self._pw.get("tomo_angle_range") - if angle_range_widget is None: - return - angle_range = int(angle_range_widget.currentData() or 180) + angle_range = int(angle_range_widget.currentData() or 180) if angle_range_widget else 180 requested = self._pw["_requested_total"].value() - stepsize = _requested_to_stepsize(angle_range, requested) - actual_total, achievable_step, _ = _compute_type1(angle_range, stepsize) + stepsize = self._profile["requested_to_stepsize"](angle_range, requested) + actual_total, achievable_step, _ = self._profile["compute_type1"](angle_range, stepsize) self._lbl_actual_total.setText(str(actual_total)) self._lbl_achievable_step.setText(f"{achievable_step:.4f}") @@ -733,12 +871,16 @@ class TomoParamsWidget(BECWidget, QWidget): self._update_type_visibility(tomo_type) def _on_single_point_changed(self, state: int) -> None: + # only ever connected when self._profile["has_single_point"] (see + # _build_params_panel), so self._profile["stitch_fields"] is always + # meaningful here. checked = bool(state) + stitch_keys = [key for key, _label, _min, _max in self._profile["stitch_fields"]] if checked: - self._pw["stitch_x"].setValue(0) - self._pw["stitch_y"].setValue(0) - self._pw["stitch_x"].setEnabled(self._edit_mode and not checked) - self._pw["stitch_y"].setEnabled(self._edit_mode and not checked) + for key in stitch_keys: + self._pw[key].setValue(0) + for key in stitch_keys: + self._pw[key].setEnabled(self._edit_mode and not checked) # random shift only applies to single-point acquisitions self._pw["single_point_random_shift_max"].setEnabled(self._edit_mode and checked) # informational, not an editable field -- shown whenever single-point @@ -815,11 +957,12 @@ class TomoParamsWidget(BECWidget, QWidget): self._refresh_sample_name() def _refresh_sample_name(self) -> None: - # Mirror the exact path used by Flomni.sample_get_name(0): - # getattr(dev.flomni_samples.sample_names, "sample0").get() + # Per-setup lookup: flomni has a device-backed sample store + # (dev.flomni_samples...); lamni has no such device and stores the + # sample name as a plain global var instead (see + # SETUP_PROFILES[...]["sample_name_getter"]). try: - name_signal = self.dev.flomni_samples.sample_names.sample0 - name = name_signal.get() + name = self._profile["sample_name_getter"](self) self._lbl_sample.setText(str(name) if name else "---") except Exception: self._lbl_sample.setText("---") @@ -830,31 +973,6 @@ class TomoParamsWidget(BECWidget, QWidget): params = self._load_params() self._populate_fields(params) - def _refresh_queue(self) -> None: - jobs = self._load_queue() - table = self._queue_table - table.setRowCount(len(jobs)) - for row, job in enumerate(jobs): - params = job.get("params", {}) - status = job.get("status", "pending") - color = STATUS_COLORS.get(status, "#888888") - - cells = [ - str(row), - job.get("label", f"job_{row}"), - status, - str(params.get("tomo_type", "?")), - _format_projections(params), - _fmt_num(params.get("tomo_countingtime")), - _fmt_num(params.get("tomo_shellstep")), - job.get("added_at", ""), - ] - for col, text in enumerate(cells): - item = QTableWidgetItem(text) - item.setTextAlignment(Qt.AlignmentFlag.AlignCenter) - item.setForeground(table.palette().text() if col != 2 else _color_from_hex(color)) - table.setItem(row, col, item) - # ── edit-mode workflow ──────────────────────────────────────────────────── def enter_edit_mode(self) -> None: @@ -916,7 +1034,7 @@ class TomoParamsWidget(BECWidget, QWidget): Uses the ``tomo_progress["heartbeat"]`` timestamp written by ``_tomo_scan_at_angle()`` at the start of every projection — this is updated whether the scan was started via the queue or directly via - ``flomni.tomo_scan()``, so it's more reliable than queue job status. + ``.tomo_scan()``, so it's more reliable than queue job status. A heartbeat fresher than 2 minutes indicates a scan in progress; older than that it's either done, or stuck long enough that editing parameters is intentional. @@ -969,7 +1087,7 @@ class TomoParamsWidget(BECWidget, QWidget): angle_range = int(params.get("tomo_angle_range", 180)) requested = self._pw["_requested_total"].value() stepsize = params["tomo_angle_stepsize"] - actual_total, _, _ = _compute_type1(angle_range, stepsize) + actual_total, _, _ = self._profile["compute_type1"](angle_range, stepsize) if actual_total != requested: reply = QMessageBox.question( self, @@ -982,11 +1100,16 @@ class TomoParamsWidget(BECWidget, QWidget): if reply != QMessageBox.StandardButton.Yes: return + offset_axes = {key: axis for key, _l, _mn, _mx, _d, axis in self._profile["offset_fields"]} ok = True for key, val in params.items(): - if key in QUEUE_PARAM_NAMES: + if key in self._profile["param_names"]: if not self._gv_set(key, val): ok = False + elif key in offset_axes: + # tomo_fovx_offset/tomo_fovy_offset don't map to a + # same-named global var -- write via the profile's setter. + self._profile["offset_setter"](self.client, offset_axes[key], val) if ok: self._edit_mode = False @@ -1020,7 +1143,7 @@ class TomoParamsWidget(BECWidget, QWidget): angle_range = int(params.get("tomo_angle_range", 180)) requested = self._pw["_requested_total"].value() stepsize = params["tomo_angle_stepsize"] - actual_total, _, _ = _compute_type1(angle_range, stepsize) + actual_total, _, _ = self._profile["compute_type1"](angle_range, stepsize) if actual_total != requested: reply = QMessageBox.question( self, @@ -1042,12 +1165,17 @@ class TomoParamsWidget(BECWidget, QWidget): jobs = self._load_queue() if not label.strip(): label = f"job_{len(jobs)}" + # Queue snapshots only ever hold this setup's real scan params -- + # not internal-only widget state (_requested_total, _numprj_type3) + # and not alignment-only offset_fields (tomo_fovx_offset/etc.), + # which aren't part of param_names for exactly this reason. + snapshot = {k: v for k, v in params.items() if k in self._profile["param_names"]} jobs.append( { "kind": "tomo", "id": uuid.uuid4().hex, "label": label.strip(), - "params": params, + "params": snapshot, "status": "pending", "added_at": datetime.datetime.now().isoformat(timespec="seconds"), } @@ -1071,13 +1199,14 @@ class TomoParamsWidget(BECWidget, QWidget): if key in skip: continue widget.setEnabled(enabled) - # stitch locked when single_point is active; random shift is the - # inverse -- only editable while single_point is active - single_point = self._pw["single_point_instead_of_fermat_scan"].isChecked() - if enabled and single_point: - self._pw["stitch_x"].setEnabled(False) - self._pw["stitch_y"].setEnabled(False) - self._pw["single_point_random_shift_max"].setEnabled(enabled and single_point) + if self._profile["has_single_point"]: + # stitch locked when single_point is active; random shift is the + # inverse -- only editable while single_point is active + single_point = self._pw["single_point_instead_of_fermat_scan"].isChecked() + if enabled and single_point: + for key, _label, _min, _max in self._profile["stitch_fields"]: + self._pw[key].setEnabled(False) + self._pw["single_point_random_shift_max"].setEnabled(enabled and single_point) # projection requested total only meaningful in edit mode self._pw["_requested_total"].setEnabled(enabled) @@ -1098,7 +1227,7 @@ class TomoParamsWidget(BECWidget, QWidget): def _get_queue_dialog(self) -> "TomoQueueDialog": if self._queue_dlg is None: - self._queue_dlg = TomoQueueDialog(self.client, parent=self) + self._queue_dlg = TomoQueueDialog(self.client, self._profile, parent=self) return self._queue_dlg # ── cleanup ─────────────────────────────────────────────────────────────── @@ -1113,9 +1242,13 @@ class TomoParamsWidget(BECWidget, QWidget): class TomoQueueDialog(QDialog): """ - Non-modal popup window for managing the FlOMNI tomo scan queue. + Non-modal popup window for managing the tomo scan queue. - Opened from the main TomoParamsWidget via the "Queue control…" button. + Opened from the main TomoParamsWidget via the "Queue control…" button, + which passes its own detected ``profile`` (see ``SETUP_PROFILES``) down + so this dialog's "Add current params to queue" reads the right param + names/defaults for whichever setup is active -- it never detects the + setup independently. Shares the BEC client with its parent for global-var access. Polls the queue every 2 s to stay in sync with CLI-driven changes. """ @@ -1142,11 +1275,12 @@ class TomoQueueDialog(QDialog): # harmless. _MOVABLE_STATUSES = ("pending",) - def __init__(self, client, parent=None): + def __init__(self, client, profile: dict, parent=None): super().__init__(parent) self._client = client + self._profile = profile self._sort_mode = False - self.setWindowTitle("FlOMNI – Tomo Queue Control") + self.setWindowTitle(f"{profile['label']} – Tomo Queue Control") self.setMinimumSize(720, 360) self._build_ui() self._poll_timer = QTimer(self) @@ -1181,9 +1315,9 @@ class TomoQueueDialog(QDialog): def _load_params(self) -> dict: params = {} - for key in QUEUE_PARAM_NAMES: + for key in self._profile["param_names"]: val = self._gv_get(key) - params[key] = val if val is not None else DEFAULTS.get(key) + params[key] = val if val is not None else self._profile["defaults"].get(key) return params # ── UI ──────────────────────────────────────────────────────────────────── @@ -1216,6 +1350,7 @@ class TomoQueueDialog(QDialog): self._table.setEditTriggers(QTableWidget.EditTrigger.NoEditTriggers) self._table.itemSelectionChanged.connect(self._update_sort_buttons) self._table.itemSelectionChanged.connect(self._update_load_button) + self._table.itemSelectionChanged.connect(self._update_duplicate_button) vbox.addWidget(self._table) btn_row = QHBoxLayout() @@ -1229,6 +1364,12 @@ class TomoQueueDialog(QDialog): ) self._btn_load.setEnabled(False) self._btn_del = QPushButton("Delete selected") + self._btn_dup = QPushButton("Duplicate selected") + self._btn_dup.setToolTip( + "Append a copy of the selected job to the end of the queue, " + "regardless of its status (pending/running/incomplete/done)." + ) + self._btn_dup.setEnabled(False) self._btn_clr = QPushButton("Clear all") self._btn_sort = QPushButton("Sort queue…") self._btn_sort.setCheckable(True) @@ -1245,6 +1386,7 @@ class TomoQueueDialog(QDialog): self._btn_add_cmd.clicked.connect(self.add_command_to_queue) self._btn_load.clicked.connect(self._on_load_into_editor) self._btn_del.clicked.connect(self._delete_selected) + self._btn_dup.clicked.connect(self._duplicate_selected) self._btn_clr.clicked.connect(self._clear_queue) self._btn_sort.toggled.connect(self._toggle_sort_mode) self._btn_move_up.clicked.connect(lambda: self._move_selected(-1)) @@ -1254,6 +1396,7 @@ class TomoQueueDialog(QDialog): btn_row.addWidget(self._btn_add_cmd) btn_row.addWidget(self._btn_load) btn_row.addWidget(self._btn_del) + btn_row.addWidget(self._btn_dup) btn_row.addWidget(self._btn_clr) btn_row.addWidget(self._btn_sort) btn_row.addWidget(self._btn_move_up) @@ -1314,6 +1457,7 @@ class TomoQueueDialog(QDialog): # queue was edited from elsewhere while a row stayed selected), so # re-check eligibility here too, not just on selection-change. self._update_load_button() + self._update_duplicate_button() # ── queue actions ───────────────────────────────────────────────────────── @@ -1389,6 +1533,12 @@ class TomoQueueDialog(QDialog): not self._sort_mode and self._selected_single_tomo_job() is not None ) + def _update_duplicate_button(self) -> None: + # Unlike Load-into-editor, duplication works for a job of any kind + # (tomo or command) and any status -- it's just a dict copy, so + # "exactly one row selected" is the only requirement. + self._btn_dup.setEnabled(not self._sort_mode and self._selected_single_row() >= 0) + def _on_load_into_editor(self) -> None: job = self._selected_single_tomo_job() if job is None: @@ -1411,7 +1561,7 @@ class TomoQueueDialog(QDialog): doesn't shift out from under the operator mid-reorder. """ self._sort_mode = checked - for btn in (self._btn_add, self._btn_add_cmd, self._btn_del, self._btn_clr): + for btn in (self._btn_add, self._btn_add_cmd, self._btn_del, self._btn_dup, self._btn_clr): btn.setEnabled(not checked) self._btn_move_up.setVisible(checked) self._btn_move_down.setVisible(checked) @@ -1423,6 +1573,7 @@ class TomoQueueDialog(QDialog): self._table.clearSelection() self._update_sort_buttons() self._update_load_button() + self._update_duplicate_button() def _selected_single_row(self) -> int: rows = sorted({idx.row() for idx in self._table.selectedIndexes()}) @@ -1552,15 +1703,37 @@ class TomoQueueDialog(QDialog): self._save_queue([]) self._refresh() + def _duplicate_selected(self) -> None: + """Append a copy of the selected job to the end of the queue. + + Works regardless of the selected job's status (pending/running/ + incomplete/done) and regardless of its kind (tomo or command) -- + it's a plain deep-copy with a fresh id/label/timestamp, so no + setup- or kind-specific logic is needed here at all. + """ + row = self._selected_single_row() + jobs = self._load_queue() + if not 0 <= row < len(jobs): + return + new_job = copy.deepcopy(jobs[row]) + new_job["id"] = uuid.uuid4().hex + new_job["status"] = "pending" + new_job["label"] = f"{jobs[row].get('label', '')}_dup" + new_job["added_at"] = datetime.datetime.now().isoformat(timespec="seconds") + jobs.append(new_job) + self._save_queue(jobs) + self._refresh() + def _show_execute_hint(self) -> None: jobs = self._load_queue() pending = sum(1 for j in jobs if j.get("status") in ("pending", "incomplete", "running")) + cli_name = self._profile["cli_hint_name"] QMessageBox.information( self, "Execute queue", f"Queue: {len(jobs)} job(s), {pending} pending/resumable.\n\n" "Execution is a blocking CLI operation — run from the BEC IPython session:\n\n" - " flomni.tomo_queue_execute()\n\n" + f" {cli_name}.tomo_queue_execute()\n\n" "Queue status updates here automatically once running.", ) @@ -1622,8 +1795,8 @@ class CommandJobBuilderDialog(QDialog): self, "No actions available", "The tomo_queue_actions registry is empty or unavailable -- " - "is a Flomni CLI session running? Command jobs can't be " - "built without it.", + "is a BEC CLI session (flomni/lamni) running? Command jobs " + "can't be built without it.", ) def _build_ui(self): @@ -1855,28 +2028,194 @@ def _compute_type1(angle_range: int, stepsize: float) -> tuple[int, float, float return actual_total, achievable_step, stepsize +def _lamni_requested_to_stepsize(angle_range: int, requested_total: int) -> float: + """Lamni sibling of _requested_to_stepsize(): inverse of + _lamni_compute_type1()'s N*8 grid -- mirrors + LamNI._tomo_type1_actual_grid()'s inverse (lamni.py) exactly. + + angle_range is accepted but ignored, for call-site symmetry with the + flomni functions above -- lamni is laminography: always 360 degrees, no + tomo_angle_range concept at all (see LamNI._TOMO_SCAN_PARAM_NAMES). + """ + if requested_total <= 0: + return 360.0 + return (360.0 / requested_total) * 8 + + +def _lamni_compute_type1(angle_range: int, stepsize: float) -> tuple[int, float, float]: + """Lamni sibling of _compute_type1(): mirrors + LamNI._tomo_type1_actual_grid() (lamni.py) exactly -- + N = int(360 / stepsize) -- unique angles per sub-tomogram + achievable_step = 360 / N + actual_total = N * 8 + + A full 360-degree sweep's closing angle is the same physical angle as + its start, so N (not N+1) is the real per-sub-tomogram count -- + sub_tomo_scan() generates it with endpoint=False for exactly this + reason. angle_range is accepted but ignored (see + _lamni_requested_to_stepsize()). + """ + if stepsize <= 0: + return 0, 0.0, 0.0 + N = int(360.0 / stepsize) + if N <= 0: + return 0, 0.0, 0.0 + achievable_step = 360.0 / N + actual_total = N * 8 + return actual_total, achievable_step, stepsize + + +def _lamni_get_tomo_fov_offset(client, axis: str) -> float: + """Mirrors LamNIAlignmentMixin.tomo_fovx_offset/tomo_fovy_offset getters + exactly: both axes are packed into one global var, ``tomo_fov_offset = + [x_um, y_um]``, exposed in mm (divided by 1000). Unlike every other + field this widget handles, there is no global var literally named + ``tomo_fovx_offset``/``tomo_fovy_offset`` -- the generic + ``_gv_get``/``_gv_set`` (direct global-var-by-name) can't be used here. + """ + val = client.get_global_var("tomo_fov_offset") + if val is None: + return 0.0 + idx = 0 if axis == "x" else 1 + return val[idx] / 1000 + + +def _lamni_set_tomo_fov_offset(client, axis: str, value_mm: float) -> None: + """Mirrors LamNIAlignmentMixin.tomo_fovx_offset/tomo_fovy_offset setters + exactly: read-modify-write the packed ``[x_um, y_um]`` list so writing + one axis never clobbers the other.""" + val_old = client.get_global_var("tomo_fov_offset") + if val_old is None: + val_old = [0.0, 0.0] + else: + val_old = list(val_old) + idx = 0 if axis == "x" else 1 + val_old[idx] = value_mm * 1000 + client.set_global_var("tomo_fov_offset", val_old) + + +# ── per-setup profiles ─────────────────────────────────────────────────────── + +SETUP_PROFILES: dict[str, dict[str, Any]] = { + "flomni": { + "label": "FlOMNI", + "discriminator_attr": "fsamroy", + "param_names": QUEUE_PARAM_NAMES, + "defaults": DEFAULTS, + "has_180_mode": True, + "has_single_point": True, + "has_zero_deg_reference": True, + "fov_fields": [ + ("fovx", "FOV x (µm)", 0.1, 200.0, 2), + ("fovy", "FOV y (µm)", 0.1, 100.0, 2), + ], + "stitch_fields": [("stitch_x", "Stitch x", 0, 50), ("stitch_y", "Stitch y", 0, 50)], + "manual_shift_fields": [("manual_shift_y", "Manual shift y (µm)", -1000.0, 1000.0, 3)], + "piezo_range_fields": [], + "offset_fields": [], + "offset_getter": None, + "offset_setter": None, + # exact current build order -- unchanged behavior for flomni + "field_order": [ + "tomo_countingtime", + "tomo_shellstep", + "fov", + "stitch", + "tomo_stitch_overlap", + "ptycho_reconstruct_foldername", + "manual_shift", + "frames_per_trigger", + "single_point", + "at_each_angle_hook", + ], + "type1_base_angle": 180.0, + "compute_type1": _compute_type1, + "requested_to_stepsize": _requested_to_stepsize, + "sample_name_getter": lambda w: w.dev.flomni_samples.sample_names.sample0.get(), + "cli_hint_name": "flomni", + }, + "lamni": { + "label": "LamNI", + "discriminator_attr": "lsamrot", + "param_names": LAMNI_QUEUE_PARAM_NAMES, + "defaults": LAMNI_DEFAULTS, + "has_180_mode": False, + "has_single_point": False, + "has_zero_deg_reference": False, + "fov_fields": [("tomo_circfov", "Circular FOV (µm)", 0.1, 200.0, 2)], + "stitch_fields": [ + ("lamni_stitch_x", "Stitch x", 0, 50), + ("lamni_stitch_y", "Stitch y", 0, 50), + ], + "manual_shift_fields": [ + ("manual_shift_x", "Manual shift x (µm)", -1000.0, 1000.0, 3), + ("manual_shift_y", "Manual shift y (µm)", -1000.0, 1000.0, 3), + ], + "piezo_range_fields": [ + ("lamni_piezo_range_x", "Piezo range x (µm)", 0.0, 200.0, 2), + ("lamni_piezo_range_y", "Piezo range y (µm)", 0.0, 200.0, 2), + ], + # tomo_fovx_offset/tomo_fovy_offset: alignment values (LamNIAlignmentMixin), + # not part of _TOMO_SCAN_PARAM_NAMES -- shown/editable here (mirroring + # lamni.tomo_parameters()) but excluded from queue job snapshots, same + # as flomni's corr_pos_y/corr_angle_y (also not shown by this widget). + "offset_fields": [ + ("tomo_fovx_offset", "FOV offset x (mm)", -1000.0, 1000.0, 4, "x"), + ("tomo_fovy_offset", "FOV offset y (mm)", -1000.0, 1000.0, 4, "y"), + ], + "offset_getter": _lamni_get_tomo_fov_offset, + "offset_setter": _lamni_set_tomo_fov_offset, + # mirrors lamni.tomo_parameters()'s exact print/edit order (lamni.py:1136-1153) + "field_order": [ + "tomo_countingtime", + "tomo_shellstep", + "piezo_range", + "stitch", + "tomo_stitch_overlap", + "fov", + "ptycho_reconstruct_foldername", + "frames_per_trigger", + "offsets", + "at_each_angle_hook", + ], + "type1_base_angle": 360.0, + "compute_type1": _lamni_compute_type1, + "requested_to_stepsize": _lamni_requested_to_stepsize, + "sample_name_getter": lambda w: w.client.get_global_var("sample_name"), + "cli_hint_name": "lamni", + }, +} + + def _format_projections(params: dict) -> str: """ Human-readable projection count for a queued job, mirroring the CLI's - per-type logic: - - type 1: int(180 / stepsize) * 8 (8 equally spaced sub-tomos, - independent of angle_range) - - type 3: int(180 / stepsize) * 8 (equally spaced, golden start) + per-type logic. A job's own ``params`` dict is that setup's own param + snapshot, so ``"tomo_angle_range" in params`` distinguishes a flomni job + (fixed-180 base, mode-independent) from a lamni job (plain 360 base, + see ``_lamni_compute_type1``/``_lamni_requested_to_stepsize``) without + this module-level helper needing to know which setup is active: + - type 1: flomni int(180/stepsize)*8, lamni int(360/stepsize)*8 + - type 3: flomni int(180/stepsize)*8, lamni int(360/stepsize)*8 - type 2: golden ratio has no fixed count -> configured max, or ∞ if unset """ try: tomo_type = int(params.get("tomo_type", 1)) stepsize = float(params.get("tomo_angle_stepsize", 0) or 0) + is_flomni_job = "tomo_angle_range" in params + base_angle = 180.0 if is_flomni_job else 360.0 + compute_type1 = _compute_type1 if is_flomni_job else _lamni_compute_type1 if tomo_type == 1: angle_range = int(params.get("tomo_angle_range", 180)) - actual_total, _, _ = _compute_type1(angle_range, stepsize) + actual_total, _, _ = compute_type1(angle_range, stepsize) return str(actual_total) if tomo_type == 3: if stepsize <= 0: return "?" - return str(int(180.0 / stepsize) * 8) + n = int(base_angle / stepsize) + return str(n * 8) if tomo_type == 2: max_prj = params.get("golden_max_number_of_projections", 0) or 0 @@ -1937,9 +2276,11 @@ def _job_tooltip(job: dict) -> str: else: lines.append("Params:") params = job.get("params", {}) - for name in QUEUE_PARAM_NAMES: - if name in params: - lines.append(f" {name}: {params[name]}") + # sorted(params) rather than a fixed module-level name tuple -- a + # job's own params dict IS whichever setup's param snapshot, so this + # works for both flomni and lamni jobs without needing to know which. + for name in sorted(params): + lines.append(f" {name}: {params[name]}") lines.append(f"Added: {job.get('added_at', '')}") return "\n".join(lines) diff --git a/docs/user/ptychography/lamni.md b/docs/user/ptychography/lamni.md index 0cf3f18..3ba29d8 100644 --- a/docs/user/ptychography/lamni.md +++ b/docs/user/ptychography/lamni.md @@ -75,6 +75,7 @@ The sample fine alignment can be obtained using ptychography. For this a short l * `lamni.tomo_fovx/y_offset=value` [mm] will shift the field of view. Perform this adjustment from projections collected at **lsamrot 0 degrees**. This shift will rotate. In contrast the manual shift will be a constant shift, identical at all angles. +(user.ptychography.lamni.laminography)= ### Laminography scan Start the laminography scan by @@ -85,6 +86,190 @@ Start the laminography scan by `lamni.tomo_reconstruct()` 3. `lamni.tomo_scan()` to start the laminography scan +During the scan, a live progress report is printed (subtomogram/projection counters and an estimated time of completion). Gaps significantly longer than a normal acquisition cycle (e.g. a beamline-down interruption, or a crash followed by a restart) are detected automatically from the time between consecutive projections and excluded from the time estimate; the total time lost to such gaps is printed once the scan finishes. + +Three angular sampling modes are implemented, same as flOMNI, but LamNI always scans the full 360 degrees — laminography's tilted sample geometry does not have the symmetry that lets flOMNI's 180-degree mode work, so there is no angular-range choice to make: + +| tomography mode | parameters and defaults | +| --- | --- | +| 8 sub-tomograms | subtomo_start=1, start_angle=None | +| Golden ratio tomography (sorted in bunches) | projection_number=None | +| Equally spaced with golden starting angle | projection_number=None | + +The parameters above can be used to __restart an interrupted acquisition__ manually, or - more conveniently - by running +`lamni.tomo_scan_resume()` +which reads the last recorded progress and resumes automatically at the exact point (subtomogram/angle, or projection for the golden ratio modes) the scan was interrupted at, without needing to look up the values by hand. When running from the tomo scan queuing system use `lamni.tomo_queue_execute()` instead! + +In case of eight equally spaced sub-tomograms, an individual sub-tomogram can be scanned by `lamni.sub_tomo_scan(subtomo_number, start_angle)`. If the start angle is not specified, it is computed depending on `subtomo_number`, ranging from 1 to 8. + +#### Queueing multiple scans + +Several tomo parameter sets can be queued and run sequentially on the same sample, without having to start each one by hand. + +| command | explanation | +| --- | --- | +| `lamni.tomo_queue_add(label=None)` | Snapshot the currently set tomo parameters and add them as a new job to the queue. Returns the job's index. | +| `lamni.tomo_queue_show()` | Print and return the current queue, with status per job. | +| `lamni.tomo_queue_delete(*indices)` | Delete one or more jobs by index. | +| `lamni.tomo_queue_clear()` | Empty the queue. | +| `lamni.tomo_queue_execute(start_index=0)` | Run all pending jobs in sequence, on the current sample. | + +The queue is persisted (it survives a BEC client restart). Each job's status is one of `pending`, `running`, `incomplete`, or `done`. A job that did not run to completion (an exception was caught, or the BEC client itself crashed mid-scan) is automatically resumed - rather than restarted - the next time `lamni.tomo_queue_execute()` is called. + +If the process running the queue dies hard enough that it never gets to write a status update at all (a kernel restart, a Ctrl-C during a blocking prompt) a job can be left stuck showing `running` with nothing actually running. `lamni.tomo_queue_delete(index)` has no status guard and works on a stuck row regardless; you can also fix the status directly with `lamni._tomo_queue_proxy.update_by_id(job_id, status="incomplete")` (to make it resumable) or `status="done"` (to skip it). The GUI's Delete/Clear normally refuse to touch a `running` row, but detect a stale one (no recent scan heartbeat) and offer to proceed anyway with an explicit confirmation instead of blocking forever. + +Example: +``` +lamni.tomo_parameters() # set up parameter set #1 +lamni.tomo_queue_add("fast overview") +lamni.tomo_parameters() # set up parameter set #2 +lamni.tomo_queue_add("hires scan") +lamni.tomo_queue_show() +lamni.tomo_queue_execute() # runs both, in order, on this sample +``` + +**GUI: reusing an earlier job's settings.** In the ☰ Queue control… dialog, select a +single tomo job (not a command job — those have no scan parameters) and click +**"Load into editor"** to pull its saved settings into the params panel's editor, +as if you'd just clicked Edit. Nothing is written yet — review or tweak the fields, +then Submit (writes live, blocked while the beamline is busy) or "Add to queue" (always +allowed) as usual. If you already had an edit in progress, it asks before discarding it. + +**GUI: two different "Add to queue" buttons.** The params panel's own **"Add to +queue"** (visible in edit mode) queues whatever you've typed, unsubmitted. The ☰ +Queue control… window's **"Add current params to queue"** is a different button on a +different window — it always queues the *live* parameters, regardless of any edit +open in the panel. If you have an unsaved edit open and click the queue window's +button instead of the panel's, it warns and names the correct one before proceeding, +since it would otherwise queue your last-submitted values, not what you just typed. + +**GUI: "Duplicate selected".** Appends an exact copy of the selected job (any status — +pending, running, incomplete, or done, and either a tomo or a command job) to the end +of the queue, labeled with `_dup` appended, status reset to `pending`. Useful for +re-running a job with the same settings without re-typing them, or as a starting point +to tweak via "Load into editor". + +#### Command jobs — reconfiguring the beamline between scans + +In addition to tomogram jobs, the same queue can hold **command jobs**: an ordered +list of beamline reconfiguration steps (move a device, ...) that run instead of a +scan. This is what lets one queue express *"tomogram A, then reconfigure, then +tomogram B"* unattended, e.g. change energy and re-peak the undulator gap between +two tomograms on the same sample. + +`lamni.tomo_queue_add_command(steps, label=None, idempotent=None)` + +- `steps`: a single `{"action": ..., "kwargs": {...}}` dict, or a list of them run in + sequence within that one job. +- `label`: optional name shown by `tomo_queue_show()`, same as for `tomo_queue_add()`. +- `idempotent`: normally inferred (safe to blindly re-run after a crash only if every + step is); override explicitly if needed. + +Only actions from a fixed, reviewed registry can be queued — not arbitrary code: + +| action | does | parameters | +| --- | --- | --- | +| `move` | Move device(s) to absolute position(s). Only devices on the allow-list below can be targeted. | `positions`: `{device: target}`, e.g. `{"mokev": 6.2}` | +| `optimize_idgap` | Scan the undulator gap over a range and move to the peak. **Not yet implemented (no-op stub).** | `search_range`: mm, default 0.5, range 0–2 | + +Devices allowed for `move`: `mokev` (energy, keV), `idgap` (undulator gap, mm) — the +same beamline-wide devices as flOMNI's queue. A `move` naming any other device is +rejected, both when the job is added and again when it actually runs. + +Example — change energy, then re-peak idgap, before the next laminogram: +``` +lamni.tomo_queue_add_command( + [{"action": "move", "kwargs": {"positions": {"mokev": 6.2}}}, + {"action": "optimize_idgap", "kwargs": {"search_range": 0.5}}], + label="reconfigure to 6.2 keV", +) +``` + +`tomo_queue_show()` lists command jobs alongside tomogram jobs, e.g.: +``` +[2] pending reconfigure to 6.2 keV CMD move{'positions': {'mokev': 6.2}} > optimize_idgap{'search_range': 0.5} [idem] +``` + +If a command job is interrupted by a crash, there is no per-step resume point — the +whole job is either safe to redo from the top or it isn't: +- If every step is idempotent (the usual case — an absolute move is harmless to + repeat), `lamni.tomo_queue_execute()` silently re-runs the whole job from the top. +- If any step is not idempotent, you are asked whether to re-run the job from the top + or mark it done as-is. + +#### Custom behavior at each projection angle + +By default, every projection in a laminography scan is a single ptychography scan at +that angle. For cases that need something more — e.g. record a projection, move a +polarizer in, record again, move it back out, at every angle — write a small Python +function and register it as an **at_each_angle hook**. Once registered, it can be +selected per tomo-queue job, so the whole modulated laminogram runs unattended along +with any other queued jobs. + +A hook is a function `func(lamni, angle)`, called once per projection angle instead +of the normal acquisition: +```python +def polarizer_modulation(lamni, angle): + lamni.tomo_scan_projection(angle) + umv(dev.polarizer, "in") + lamni.tomo_scan_projection(angle) + umv(dev.polarizer, "out") +``` + +| command | explanation | +| --- | --- | +| `lamni.register_at_each_angle_hook("name", func)` | Load a hook function under `name`. | +| `lamni.at_each_angle_hook = "name"` | Activate it — the next tomo-queue job added will use it. | +| `lamni.at_each_angle_hook = None` | Deactivate it — back to normal projections for the next job. | +| `lamni.list_at_each_angle_hooks()` | Print the names of all currently registered hooks (the active one is marked). | +| `lamni.unregister_at_each_angle_hook("name")` | Remove a hook by name. | + +Example — queue a polarizer-modulated laminogram, then a normal one: +```python +lamni.register_at_each_angle_hook("polarizer_mod", polarizer_modulation) + +lamni.tomo_parameters() # set up the scan parameters as usual +lamni.at_each_angle_hook = "polarizer_mod" # activate the hook +lamni.tomo_queue_add("polarizer run") + +lamni.at_each_angle_hook = None # back to normal -- do not skip this! +lamni.tomo_queue_add("plain follow-up scan") + +lamni.tomo_queue_execute() # runs both, hook active only for the first +``` +**Do not forget to reset `lamni.at_each_angle_hook = None`** before adding a job +that should run normally — each queued job remembers whatever `at_each_angle_hook` +was set to at the moment it was added (like every other tomo parameter), so a job +added right after a hook-using one without resetting it first will silently run with +that hook still active. + +Registered hooks are **session-only** — they live in memory and do not survive a +kernel restart. If `tomo_queue_execute()` reaches a job whose `at_each_angle_hook` +isn't registered in the current session (e.g. after restarting BEC), it stops with a +clear error rather than silently running a plain scan; re-run +`register_at_each_angle_hook()` for that hook, then call `tomo_queue_execute()` +again to resume. + +**Editing a hook after registering it:** `register_at_each_angle_hook()` stores +whichever function object you pass it at that moment — it is not a live link to the +function's name. If you edit the function's source and re-run the `def` in your +session but do **not** call `register_at_each_angle_hook()` again, the *old* version +keeps running. Always re-register after an edit. + +**GUI:** the tomo parameters panel has an "At-each-angle hook" dropdown, listing +whatever is currently registered from the CLI (`register_at_each_angle_hook()`) — +the GUI process can't register hooks itself, only select one by name for the job +you're about to submit or add to the queue. Pick "None (default)" for a normal scan. +If a job's stored hook isn't currently registered anywhere, it still shows up in the +dropdown labeled "(not registered in this session)" so it's never silently hidden. + +**Unregistering does not clear `at_each_angle_hook`.** `unregister_at_each_angle_hook()` +only removes the function from this session's registry — if `at_each_angle_hook` (or a +queued job) still names it, that reference is untouched; it just becomes an +unregistered name, flagged as such wherever it's shown (CLI, GUI dropdown). Set +`lamni.at_each_angle_hook = None` (or the GUI dropdown's "None (default)") explicitly +if you want to actually clear the selection, not just remove the hook it points to. + ### Tips and Tricks #### Reset corrections diff --git a/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py b/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py new file mode 100644 index 0000000..17c5c32 --- /dev/null +++ b/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py @@ -0,0 +1,127 @@ +"""Regression test for LamNI.sub_tomo_scan()'s actual angle generation. + +Mirrors test_flomni_tomo_angles.py's style of testing angle math directly, +without a live BEC session. Guards against two bugs fixed this session: + +1. A full 360-degree sweep's closing angle (start_angle + 360) is the same + physical angle as its start, so sub_tomo_scan() must generate exactly N + unique angles per sub-tomogram (endpoint=False), not N+1. +2. The inter-sub-tomogram phase offset must be a fraction of the ACHIEVABLE + step (post int() truncation), not the raw configured tomo_angle_stepsize + -- otherwise sub-tomograms combined in pairs/quads/all-8 land on + unevenly-spaced angles instead of the intended progressively finer, even + grid at every combination level (mirrors Flomni._subtomo_angle_plan()'s + same requirement). +""" + +import numpy as np +import pytest + +from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI, _ProgressProxy + +STEPSIZES = [10.0, 7.0, 25.0, 12.5] + + +class FakeClient: + """Minimal in-memory stand-in for BEC's global-var store.""" + + def __init__(self): + self._vars = {} + + def get_global_var(self, key): + return self._vars.get(key) + + def set_global_var(self, key, value): + self._vars[key] = value + + +def make_lamni(tomo_angle_stepsize: float) -> LamNI: + """Bare LamNI instance with only what sub_tomo_scan() needs to run its + angle-generation logic, bypassing __init__'s heavy side effects.""" + obj = object.__new__(LamNI) + obj.client = FakeClient() + obj._progress_proxy = _ProgressProxy(obj.client) + obj.tomo_angle_stepsize = tomo_angle_stepsize + obj.tomo_id = -1 + obj._write_subtomo_to_scilog = lambda subtomo_number: None + return obj + + +@pytest.mark.parametrize("stepsize", STEPSIZES) +@pytest.mark.parametrize("subtomo_number", range(1, 9)) +def test_sub_tomo_scan_generates_no_duplicate_angles(stepsize, subtomo_number): + lamni = make_lamni(stepsize) + recorded = [] + lamni._tomo_scan_at_angle = lambda angle, subtomo: recorded.append(float(angle)) + + lamni.sub_tomo_scan(subtomo_number) + + N, _achievable_step, _total = lamni._tomo_type1_actual_grid() + assert len(recorded) == N, "sub_tomo_scan() must generate exactly N angles, not N+1" + unique_mod_360 = {round(a % 360, 6) for a in recorded} + assert len(unique_mod_360) == N, "sub_tomo_scan() must not re-measure its own start angle" + + +@pytest.mark.parametrize("stepsize", STEPSIZES) +def test_sub_tomo_scan_projection_count_matches_actual_grid(stepsize): + """progress['subtomo_total_projections']/['total_projections'] must + match what actually ran, not a pre-truncation float estimate.""" + lamni = make_lamni(stepsize) + lamni._tomo_scan_at_angle = lambda angle, subtomo: None + + lamni.sub_tomo_scan(1) + + N, _achievable_step, total = lamni._tomo_type1_actual_grid() + assert lamni.progress["subtomo_total_projections"] == N + assert lamni.progress["total_projections"] == total + + +def _all_subtomo_angles(stepsize: float) -> dict[int, list[float]]: + """Run sub_tomo_scan() for all 8 sub-tomograms (real code path, not a + reimplementation) and return each one's acquired angles, in order.""" + result = {} + for n in range(1, 9): + lamni = make_lamni(stepsize) + recorded = [] + lamni._tomo_scan_at_angle = lambda angle, subtomo, _r=recorded: _r.append(float(angle)) + lamni.sub_tomo_scan(n) + result[n] = recorded + return result + + +def _assert_equally_spaced(angles, label: str) -> None: + sorted_angles = np.sort(np.asarray(angles) % 360) + diffs = np.diff(np.concatenate([sorted_angles, [sorted_angles[0] + 360]])) + assert np.allclose( + diffs, diffs[0], atol=1e-6 + ), f"{label}: angles are not equally spaced when combined -- got spacings {diffs}" + + +@pytest.mark.parametrize("stepsize", STEPSIZES) +def test_subtomo_pairs_are_equally_spaced_when_combined(stepsize): + """(1,2), (3,4), (5,6), (7,8) each combine into one evenly-spaced, + doubled-resolution grid -- this is the whole point of the phase-offset + interlacing scheme (bit-reversal table), and silently breaks if the + phase is computed from the wrong step (see module docstring).""" + angles = _all_subtomo_angles(stepsize) + for a, b in [(1, 2), (3, 4), (5, 6), (7, 8)]: + _assert_equally_spaced(angles[a] + angles[b], f"pair ({a},{b})") + + +@pytest.mark.parametrize("stepsize", STEPSIZES) +def test_subtomo_quads_are_equally_spaced_when_combined(stepsize): + """(1,2,3,4) and (5,6,7,8) each combine into one evenly-spaced, + quadrupled-resolution grid.""" + angles = _all_subtomo_angles(stepsize) + for quad in [(1, 2, 3, 4), (5, 6, 7, 8)]: + combined = sum((angles[n] for n in quad), []) + _assert_equally_spaced(combined, f"quad {quad}") + + +@pytest.mark.parametrize("stepsize", STEPSIZES) +def test_all_eight_subtomos_are_equally_spaced_when_combined(stepsize): + """The full combined set of all 8 sub-tomograms is the finest, + evenly-spaced grid -- the end goal of the interlacing scheme.""" + angles = _all_subtomo_angles(stepsize) + combined = sum((angles[n] for n in range(1, 9)), []) + _assert_equally_spaced(combined, "full (all 8)") diff --git a/tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py b/tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py new file mode 100644 index 0000000..7d4c74c --- /dev/null +++ b/tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py @@ -0,0 +1,167 @@ +"""Regression test for TomoParamsWidget's lamni-specific projection-count +math (_lamni_compute_type1/_lamni_requested_to_stepsize), mirroring +test_tomo_params_widget_math.py's style for flomni. + +Lamni's formula is NOT the same as flomni's: LamNI.sub_tomo_scan() sweeps a +plain 360 degrees per sub-tomogram (vs. flomni's fixed-180-base, +mode-independent _compute_type1/_requested_to_stepsize, which these tests +must not touch -- see that file's own docstring on why they're pinned by +name/signature) -- but is otherwise the same shape: N = int(360/stepsize) +unique angles per sub-tomogram, N*8 total. (sub_tomo_scan() used to also +acquire a duplicate closing angle per sub-tomogram -- start_angle+360 is the +same physical angle as start_angle -- fixed via endpoint=False; see +lamni.py's _tomo_type1_actual_grid().) +""" + +import pytest + +from csaxs_bec.bec_ipython_client.plugins.flomni.flomni import Flomni +from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI +from csaxs_bec.bec_widgets.widgets.tomo_params.tomo_params import ( + DEFAULTS, + LAMNI_QUEUE_PARAM_NAMES, + QUEUE_PARAM_NAMES, + SETUP_PROFILES, + _compute_type1, + _lamni_compute_type1, + _lamni_get_tomo_fov_offset, + _lamni_requested_to_stepsize, + _lamni_set_tomo_fov_offset, + _requested_to_stepsize, +) + +STEPSIZES = [10.0, 7.0, 25.0, 12.5] + + +@pytest.mark.parametrize("stepsize", STEPSIZES) +def test_lamni_compute_type1_matches_lamni_subtomo_scan(stepsize): + """Reference formula mirrors LamNI.sub_tomo_scan()'s exact (fixed) grid: + N = int(360/stepsize) unique angles per sub-tomogram, N*8 total.""" + N = int(360.0 / stepsize) + expected_total = N * 8 + expected_step = 360.0 / N + + actual_total, achievable_step, _ = _lamni_compute_type1(360, stepsize) + assert actual_total == expected_total + assert achievable_step == pytest.approx(expected_step) + + +@pytest.mark.parametrize("requested_total", [32, 64, 160, 320]) +def test_lamni_requested_to_stepsize_round_trips(requested_total): + """requested_total must already be a multiple of 8 for the round trip + to land exactly back on it -- these were chosen as N=4,8,20,40.""" + stepsize = _lamni_requested_to_stepsize(360, requested_total) + actual_total, _, _ = _lamni_compute_type1(360, stepsize) + assert actual_total == requested_total + + +def test_lamni_formula_differs_from_flomni_formula(): + """Cheap insurance against accidentally aliasing the wrong function into + a SETUP_PROFILES entry -- lamni's math must not silently match flomni's.""" + for stepsize in STEPSIZES: + flomni_total, flomni_step, _ = _compute_type1(180, stepsize) + lamni_total, lamni_step, _ = _lamni_compute_type1(360, stepsize) + assert (flomni_total, flomni_step) != (lamni_total, lamni_step) + + for requested in (24, 48, 96, 144): + assert _requested_to_stepsize(180, requested) != _lamni_requested_to_stepsize( + 360, requested + ) + + +def test_setup_profiles_param_names_match_cli_classes(): + """Guards against the known mirror-drift risk: SETUP_PROFILES' + param_names must stay in sync with each CLI class's own + _TOMO_SCAN_PARAM_NAMES (there is no shared import between bec_widgets + and the ipython-client plugins, so nothing else catches this).""" + assert set(SETUP_PROFILES["flomni"]["param_names"]) == set(Flomni._TOMO_SCAN_PARAM_NAMES) + assert set(SETUP_PROFILES["lamni"]["param_names"]) == set(LamNI._TOMO_SCAN_PARAM_NAMES) + assert set(QUEUE_PARAM_NAMES) == set(Flomni._TOMO_SCAN_PARAM_NAMES) + assert set(LAMNI_QUEUE_PARAM_NAMES) == set(LamNI._TOMO_SCAN_PARAM_NAMES) + + +def test_setup_profiles_defaults_cover_all_param_names(): + for setup, profile in SETUP_PROFILES.items(): + missing = set(profile["param_names"]) - set(profile["defaults"]) + assert not missing, f"{setup} profile is missing defaults for: {missing}" + + +class _FakeClient: + """Minimal in-memory stand-in for BEC's global-var store.""" + + def __init__(self): + self._vars = {} + + def get_global_var(self, key): + return self._vars.get(key) + + def set_global_var(self, key, value): + self._vars[key] = value + + +def test_lamni_fov_offset_round_trip_does_not_clobber_other_axis(): + """tomo_fov_offset packs both axes into one global var ([x_um, y_um]) -- + writing one axis must read-modify-write, not silently reset the other + (this is exactly the kind of bug a naive per-key implementation would + introduce).""" + client = _FakeClient() + + _lamni_set_tomo_fov_offset(client, "x", 1.5) + assert _lamni_get_tomo_fov_offset(client, "x") == pytest.approx(1.5) + assert _lamni_get_tomo_fov_offset(client, "y") == pytest.approx(0.0) + + _lamni_set_tomo_fov_offset(client, "y", -2.25) + assert _lamni_get_tomo_fov_offset(client, "x") == pytest.approx(1.5) + assert _lamni_get_tomo_fov_offset(client, "y") == pytest.approx(-2.25) + + assert client.get_global_var("tomo_fov_offset") == [1500.0, -2250.0] + + +def test_lamni_fov_offset_defaults_to_zero_when_unset(): + client = _FakeClient() + assert _lamni_get_tomo_fov_offset(client, "x") == 0.0 + assert _lamni_get_tomo_fov_offset(client, "y") == 0.0 + + +def test_lamni_offset_fields_excluded_from_param_names(): + """tomo_fovx_offset/tomo_fovy_offset are alignment values, shown in the + GUI but deliberately excluded from queue job snapshots -- must never + appear in param_names.""" + offset_keys = {key for key, *_ in SETUP_PROFILES["lamni"]["offset_fields"]} + assert offset_keys == {"tomo_fovx_offset", "tomo_fovy_offset"} + assert offset_keys.isdisjoint(SETUP_PROFILES["lamni"]["param_names"]) + assert not SETUP_PROFILES["flomni"]["offset_fields"] + + +def test_flomni_field_order_matches_pre_reorder_layout(): + """Regression guard: the field_order dispatch must reproduce flomni's + exact, unchanged field build order (this task only reordered lamni).""" + assert SETUP_PROFILES["flomni"]["field_order"] == [ + "tomo_countingtime", + "tomo_shellstep", + "fov", + "stitch", + "tomo_stitch_overlap", + "ptycho_reconstruct_foldername", + "manual_shift", + "frames_per_trigger", + "single_point", + "at_each_angle_hook", + ] + + +def test_lamni_field_order_matches_tomo_parameters_cli(): + """Mirrors lamni.tomo_parameters()'s exact print/edit order + (lamni.py:1136-1153).""" + assert SETUP_PROFILES["lamni"]["field_order"] == [ + "tomo_countingtime", + "tomo_shellstep", + "piezo_range", + "stitch", + "tomo_stitch_overlap", + "fov", + "ptycho_reconstruct_foldername", + "frames_per_trigger", + "offsets", + "at_each_angle_hook", + ] -- 2.54.0 From f3c90c5b91f4f591e8442e1fce2117d4c90560e2 Mon Sep 17 00:00:00 2001 From: x01dc Date: Thu, 16 Jul 2026 13:59:12 +0200 Subject: [PATCH 3/5] fix(lamni): fix three real tomo_scan() crashes, mirroring flomni's fixes bec.active_account.decode() crashed on every real tomo_scan() call -- active_account is a plain str, not bytes. Flomni's equivalent never had the .decode() call and already guards an empty active_account (e.g. a dev/sim session) by skipping sample-database registration instead of crashing. Same bug fixed in DataDrivenLamNI.tomo_scan() (extra_tomo.py), which had an identical inline copy. write_pdf_report() read a nonexistent "mokev" device; the real device is ccm_energy. Reads it for real now, wrapped in a broad try/except falling back to "N/A" if unavailable, rather than a hardcoded placeholder. write_pdf_report()'s logbook/scilog write crashed when scilog isn't configured -- lamni.py already had this exact tolerance pattern one method over, in write_to_scilog(); applied the same try/except here. Co-Authored-By: Claude Sonnet 5 --- .../LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md | 74 +++++++++++++++--- .../plugins/LamNI/extra_tomo.py | 24 +++--- .../bec_ipython_client/plugins/LamNI/lamni.py | 54 +++++++++---- .../test_lamni_tomo_angles.py | 77 +++++++++++++++++++ 4 files changed, 192 insertions(+), 37 deletions(-) diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md index 92b6ae6..aaf4841 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md @@ -1,16 +1,20 @@ -# Tomo-params GUI generalization + two sub_tomo_scan() angle bugs fixed +# Tomo-params GUI generalization + real lamni.py bugs found and fixed Companion to `AI_docs/TOMO_QUEUE_PORT.md` (the backend port this GUI work builds on) and `csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md` (which identified `TomoParamsWidget` as flomni-gated). This session's follow-up work: generalized the widget to also support lamni, then — while -verifying the widget's projection-count preview against the CLI — found and -fixed two real, pre-existing bugs in `LamNI.sub_tomo_scan()` that predate -this session and affected every lamni tomogram ever run with a non-trivially- -achievable `tomo_angle_stepsize`. +verifying the widget's projection-count preview against the CLI, and later +just running `lamni.tomo_scan()` for real — found and fixed five real, +pre-existing bugs in `lamni.py` that predate this session and would crash +(or silently corrupt sampling in) any real lamni tomogram: two in +`sub_tomo_scan()`'s angle math (§2) and three in `tomo_scan()`/ +`write_pdf_report()`'s "new scan" bookkeeping (§3) — all three of the +latter group already correctly handled in flomni's equivalent code. **Status:** All of the below implemented, tested, and live-verified against -the running simulated lamni deployment. Not yet committed as of this doc. +the running simulated lamni deployment. §1/§2 committed as `bac3fa8`; §3 +found afterward in the same session, not yet committed as of this doc. ## 1. GUI: `TomoParamsWidget` generalized to lamni @@ -121,6 +125,40 @@ must keep re-deriving the achievable grid every time it runs, regardless of which code path set `tomo_angle_stepsize` (wizard, GUI Submit, queue-job restore, or a direct script assignment). +## 3. `tomo_scan()`/`write_pdf_report()`: three crashes on a real scan + +Found by simply calling `lamni.tomo_scan()` for real (not just +`sub_tomo_scan()` in isolation, which §2's tests/live checks exercised) — +all three already fixed in flomni's equivalent code, so each was a direct +port of flomni's existing fix, not a new design: + +- **`bec.active_account.decode()`** (`tomo_scan()`'s new-scan branch): + `active_account` is a plain `str`, not `bytes` — `.decode()` raises + `AttributeError` on every real call. Flomni's equivalent code never had + this `.decode()` at all. Also added flomni's existing guard: an empty + `active_account` (e.g. a dev/sim session not logged into a real e-account) + now skips `add_sample_database()` and falls back to `tomo_id = 0`, instead + of crashing while trying to register a sample under `""`. Same fix + applied to `DataDrivenLamNI.tomo_scan()` (`extra_tomo.py`), which had an + identical inline copy of this bug. +- **`dev.mokev.read(...)` in `write_pdf_report()`**: `mokev` isn't a real + device name here — Mirko identified the correct one, `dev.ccm_energy` + (confirmed against `flomni_optics_mixin.py`/`eiger.py`'s own `ccm_energy` + usage elsewhere in the repo). Per Mirko's request, this isn't a flomni- + style hardcoded placeholder ("To be implemented") — it now does a real + `dev.ccm_energy.read(cached=True)[dev.ccm_energy.name]['value']` read, + wrapped in a broad `try/except` that falls back to `"N/A"` if the device + isn't configured/available in a given session, so a missing device is + tolerated rather than crashing report generation. +- **`bec.logbook.LogbookMessage()` in `write_pdf_report()`**: crashes with + `AttributeError` when scilog/logbook isn't configured (e.g. a dev/sim + session) — `lamni.py` already has exactly this tolerance pattern one + method up, in `write_to_scilog()` (broad `try/except`, warn and continue), + it just wasn't applied here. Wrapped the same logbook-message block in an + identical `try/except`, logging `"Failed to write PDF report to + scilog."` on failure — the PDF file itself is still written either way + (it happens earlier in the same function, unaffected). + ## Explicitly deferred (carried over / reconfirmed) - `zero_deg_reference_at_each_subtomo` for lamni — not implemented anywhere @@ -137,15 +175,20 @@ restore, or a direct script assignment). - New tests: `tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py` (16 tests: profile field_order/param_names/defaults guards, offset getter/setter round-trip including a same-axis-doesn't-clobber-the-other - check) and `test_lamni_tomo_angles.py` (48 tests: no-duplicate-angle check - per sub-tomogram/stepsize, and the pair/quad/full-8 equally-spaced - invariant per stepsize). The equally-spaced tests were sanity-checked by - temporarily reverting the phase-offset fix and confirming 9/12 of them - fail as expected, then restoring it. + check) and `test_lamni_tomo_angles.py` (50 tests: no-duplicate-angle check + per sub-tomogram/stepsize, the pair/quad/full-8 equally-spaced invariant + per stepsize, and two tests for §3's `tomo_scan()` account handling — one + confirming an empty `active_account` skips `add_sample_database()` + entirely with `tomo_id=0`, one confirming a real account string is passed + through as-is, not `.decode()`'d). The equally-spaced tests were + sanity-checked by temporarily reverting the phase-offset fix and + confirming 9/12 of them fail as expected; the account tests were + sanity-checked the same way against the `.decode()` bug specifically, + then both reverts restored. - Existing pinned test `test_tomo_params_widget_math.py` (flomni's math) untouched and still passing. - Full offline suite (`pytest tests/tests_bec_ipython_client tests/tests_scans`): - 185 passed. + 187 passed. - Live-sim verification (plain scripts against the running simulated lamni deployment, snapshotting/restoring `tomo_queue`/`tomo_progress` around each run): @@ -158,5 +201,12 @@ restore, or a direct script assignment). `[0, 60, 120, 180, 240, 300]`, spacing exactly 60° throughout — confirms the phase-offset fix holds against real (not just mocked) execution. + - A full real `lamni.tomo_scan()` (type 1, coarse stepsize giving N=1, + 8 total real acquisitions across all 8 sub-tomograms) ran end to end + against the sim without error after §3's fixes — `tomo_id` correctly + fell back to `0` (no active account in this session), the energy read + and PDF/logbook write no longer crashed the scan. (Required creating + `~/Data10/documentation/` locally, which didn't exist in this sandbox + but is expected to already exist on the real deployed system.) - No live GUI verification performed (per Mirko's instruction) — verified manually by Mirko instead. diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/extra_tomo.py b/csaxs_bec/bec_ipython_client/plugins/LamNI/extra_tomo.py index fbae06c..3bdafc7 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/extra_tomo.py +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/extra_tomo.py @@ -102,15 +102,21 @@ class DataDrivenLamNI(LamNI): self._current_special_angles = self.special_angles.copy() if subtomo_start == 1 and start_index is None: - self.tomo_id = self.add_sample_database( - self.sample_name, - str(datetime.date.today()), - bec.active_account.decode(), - bec.queue.next_scan_number, - "lamni", - "test additional info", - "BEC", - ) + # bec.active_account is already a plain str, not bytes -- see + # LamNI.tomo_scan()'s identical fix for why .decode() crashes and + # why an empty active_account needs its own guard. + if bec.active_account != "": + self.tomo_id = self.add_sample_database( + self.sample_name, + str(datetime.date.today()), + bec.active_account, + bec.queue.next_scan_number, + "lamni", + "test additional info", + "BEC", + ) + else: + self.tomo_id = 0 self.write_pdf_report() with scans.dataset_id_on_hold: diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py index c095998..75586d5 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py @@ -1002,15 +1002,23 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools or (self.tomo_type == 2 and projection_number is None) or (self.tomo_type == 3 and projection_number is None) ): - self.tomo_id = self.add_sample_database( - self.sample_name, - str(datetime.date.today()), - bec.active_account.decode(), - bec.queue.next_scan_number, - "lamni", - "test additional info", - "BEC", - ) + # bec.active_account is already a plain str, not bytes -- .decode() + # crashes with AttributeError. Also guard against no active + # e-account (empty string, e.g. a dev/sim session not logged into + # a real account) rather than trying to register a sample under + # one -- mirrors Flomni.tomo_scan()'s equivalent check exactly. + if bec.active_account != "": + self.tomo_id = self.add_sample_database( + self.sample_name, + str(datetime.date.today()), + bec.active_account, + bec.queue.next_scan_number, + "lamni", + "test additional info", + "BEC", + ) + else: + self.tomo_id = 0 self.write_pdf_report() self.progress["tomo_start_time"] = datetime.datetime.now().isoformat() # reset stale estimates from any previous scan, otherwise the GUI @@ -1324,6 +1332,14 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools stitching = f"{self.lamni_stitch_x:.2f}/{self.lamni_stitch_y:.2f}" dataset_id = str(self.client.queue.next_dataset_number) _, _, report_total_projections = self._tomo_type1_actual_grid() + # The energy readback device is "ccm_energy", not "mokev" -- and may + # not be configured/available in every session (e.g. some simulated + # configs). Read it defensively so a missing/misbehaving device + # doesn't crash the whole report instead of just omitting one line. + try: + energy_str = f"{dev.ccm_energy.read(cached=True)[dev.ccm_energy.name]['value']:.4f}" + except Exception: + energy_str = "N/A" content = [ f"{'Sample Name:':<{padding}}{self.sample_name:>{padding}}\n", f"{'Measurement ID:':<{padding}}{str(self.tomo_id):>{padding}}\n", @@ -1333,7 +1349,7 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools f"{'Number of projections:':<{padding}}{report_total_projections:>{padding}}\n", f"{'First scan number:':<{padding}}{self.client.queue.next_scan_number:>{padding}}\n", f"{'Last scan number approx.:':<{padding}}{self.client.queue.next_scan_number + report_total_projections + 10:>{padding}}\n", - f"{'Current photon energy:':<{padding}}{dev.mokev.read(cached=True)['value']:>{padding}.4f}\n", + f"{'Current photon energy:':<{padding}}{energy_str:>{padding}}\n", f"{'Exposure time:':<{padding}}{self.tomo_countingtime:>{padding}.2f}\n", f"{'Fermat spiral step size:':<{padding}}{self.tomo_shellstep:>{padding}.2f}\n", f"{'Piezo range (FOV sample plane):':<{padding}}{piezo_range:>{padding}}\n", @@ -1351,12 +1367,18 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools subprocess.run( "xterm /work/sls/spec/local/XOMNY/bin/upload/upload_last_pon.sh &", shell=True ) - msg = bec.logbook.LogbookMessage() - logo_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "LamNI_logo.png") - msg.add_file(logo_path).add_text("".join(content).replace("\n", "

")).add_tag( - ["BEC", "tomo_parameters", f"dataset_id_{dataset_id}", "LamNI", self.sample_name] - ) - self.client.logbook.send_logbook_message(msg) + # Same tolerance as write_to_scilog(): a session without scilog/logbook + # configured (e.g. a dev/sim session) must not crash report generation + # over the logbook upload -- the PDF itself is already written above. + try: + msg = bec.logbook.LogbookMessage() + logo_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "LamNI_logo.png") + msg.add_file(logo_path).add_text("".join(content).replace("\n", "

")).add_tag( + ["BEC", "tomo_parameters", f"dataset_id_{dataset_id}", "LamNI", self.sample_name] + ) + self.client.logbook.send_logbook_message(msg) + except Exception: + logger.warning("Failed to write PDF report to scilog.") def get_calibration_of_capstops_left_and_right(self): import time diff --git a/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py b/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py index 17c5c32..6894201 100644 --- a/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py +++ b/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py @@ -14,6 +14,9 @@ without a live BEC session. Guards against two bugs fixed this session: same requirement). """ +import builtins +import types + import numpy as np import pytest @@ -125,3 +128,77 @@ def test_all_eight_subtomos_are_equally_spaced_when_combined(stepsize): angles = _all_subtomo_angles(stepsize) combined = sum((angles[n] for n in range(1, 9)), []) _assert_equally_spaced(combined, "full (all 8)") + + +# ── tomo_scan()'s new-scan account handling ───────────────────────────────── +# +# bec.active_account used to be passed through .decode() (as if it were +# bytes) -- it's a plain str, so this raised AttributeError on every real +# tomo_scan() call. Flomni's equivalent code never had this bug (no +# .decode() call at all) and already guards an empty active_account (e.g. a +# dev/sim session not logged into a real e-account) by skipping sample- +# database registration instead of trying to register under "". Both fixed +# to match. + + +class _FakeContextManager: + def __enter__(self): + return self + + def __exit__(self, *exc_info): + return False + + +class _FakeScans: + dataset_id_on_hold = _FakeContextManager() + + +def make_lamni_for_tomo_scan(tomo_angle_stepsize: float, active_account: str) -> LamNI: + """Bare LamNI instance with tomo_scan()'s "new scan" branch reachable, + everything downstream of it stubbed out (sample database, PDF report, + per-sub-tomogram scanning) so only the account-handling logic under + test actually runs.""" + obj = make_lamni(tomo_angle_stepsize) + obj.tomo_type = 1 + obj.special_angles = [] + obj.write_pdf_report = lambda: None + obj.sub_tomo_scan = lambda subtomo_number, start_angle=None: None + obj._print_progress = lambda: None + obj._format_duration = lambda seconds: "0s" + builtins.__dict__["bec"] = types.SimpleNamespace( + active_account=active_account, queue=types.SimpleNamespace(next_scan_number=1) + ) + builtins.__dict__["scans"] = _FakeScans() + return obj + + +def test_tomo_scan_skips_sample_database_when_no_active_account(): + """Empty active_account (e.g. a dev/sim session) must not crash and must + not try to register a sample -- tomo_id falls back to 0, mirroring + Flomni.tomo_scan()'s identical guard.""" + lamni = make_lamni_for_tomo_scan(45.0, active_account="") + lamni.add_sample_database = lambda *a, **k: (_ for _ in ()).throw( + AssertionError("add_sample_database must not be called with no active account") + ) + + lamni.tomo_scan() + + assert lamni.tomo_id == 0 + + +def test_tomo_scan_registers_sample_with_plain_string_account(): + """A real active_account must be passed through as-is -- calling + .decode() on it (a plain str, not bytes) raises AttributeError.""" + lamni = make_lamni_for_tomo_scan(45.0, active_account="e12345") + recorded = {} + + def _fake_add_sample_database(samplename, date, eaccount, scan_number, setup, info, user): + recorded["eaccount"] = eaccount + return 42 + + lamni.add_sample_database = _fake_add_sample_database + + lamni.tomo_scan() + + assert recorded["eaccount"] == "e12345" + assert lamni.tomo_id == 42 -- 2.54.0 From 46cadc8894d3dd2448c0d33268dbb230148a0176 Mon Sep 17 00:00:00 2001 From: x01dc Date: Thu, 16 Jul 2026 14:21:45 +0200 Subject: [PATCH 4/5] fix(lamni): show progress GUI on scan start, clear busy heartbeat on exit Mirrors two fixes already made to Flomni.tomo_scan(): LamNI.tomo_scan() never opened the progress GUI automatically, and left a stale "busy" heartbeat for up to 120s after a scan had already finished (or crashed), since it was only ever reset at the start of the next scan. Co-Authored-By: Claude Sonnet 5 --- .../LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md | 82 ++++++-- .../bec_ipython_client/plugins/LamNI/lamni.py | 176 ++++++++++-------- .../test_lamni_tomo_angles.py | 60 ++++++ 3 files changed, 218 insertions(+), 100 deletions(-) diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md index aaf4841..b214f74 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md @@ -5,16 +5,17 @@ builds on) and `csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LA (which identified `TomoParamsWidget` as flomni-gated). This session's follow-up work: generalized the widget to also support lamni, then — while verifying the widget's projection-count preview against the CLI, and later -just running `lamni.tomo_scan()` for real — found and fixed five real, -pre-existing bugs in `lamni.py` that predate this session and would crash -(or silently corrupt sampling in) any real lamni tomogram: two in -`sub_tomo_scan()`'s angle math (§2) and three in `tomo_scan()`/ -`write_pdf_report()`'s "new scan" bookkeeping (§3) — all three of the -latter group already correctly handled in flomni's equivalent code. +just running `lamni.tomo_scan()` for real — found and fixed several real, +pre-existing bugs/gaps in `lamni.py` that predate this session: two in +`sub_tomo_scan()`'s angle math (§2), three crashes in `tomo_scan()`/ +`write_pdf_report()`'s "new scan" bookkeeping (§3), and two more UX/GUI gaps +in `tomo_scan()` found once a scan could finally complete end to end (§4) — +all of §3 and §4 already correctly handled in flomni's equivalent code, so +each was a direct port, not a new design. **Status:** All of the below implemented, tested, and live-verified against -the running simulated lamni deployment. §1/§2 committed as `bac3fa8`; §3 -found afterward in the same session, not yet committed as of this doc. +the running simulated lamni deployment. §1/§2 committed as `bac3fa8`, §3 as +`f3c90c5`; §4 not yet committed as of this doc. ## 1. GUI: `TomoParamsWidget` generalized to lamni @@ -159,6 +160,36 @@ port of flomni's existing fix, not a new design: scilog."` on failure — the PDF file itself is still written either way (it happens earlier in the same function, unaffected). +## 4. `tomo_scan()`: progress GUI not shown, stale "busy" heartbeat after success + +Two more gaps found by Mirko after §3's crashes were fixed and a real scan +could finally run start to finish — both already fixed in flomni's +equivalent code, both direct ports: + +- **Progress GUI never opened automatically.** `Flomni.tomo_scan()` calls + `self.flomnigui_show_progress()` unconditionally as its first action; + `LamNI.tomo_scan()` never had an equivalent call at all, so starting a + lamni tomogram never opened/raised the progress window the way flomni's + does. Fixed by adding `self.lamnigui_show_progress()` as the first line of + `tomo_scan()` (`lamni.py:996`), mirroring flomni exactly. +- **Stale "beamline busy" heartbeat after a scan already finished.** + `progress["heartbeat"]` (the GUI's busy-detector field, refreshed on every + real projection by `_tomo_scan_at_angle()`) was only ever reset at the + *start* of a new scan (the "new scan" branch), never on completion. A + finished (or crashed/interrupted) scan's last heartbeat timestamp lingered + until the GUI's own staleness timeout (120s) finally decided it was stale — + so the GUI kept showing "beamline busy" for up to two minutes after a + tomogram had already completed successfully. Flomni hit and fixed this + exact issue earlier in this session (commit `aebfccc`); same fix ported: + the entire `with scans.dataset_id_on_hold: ...` scan body (all three + tomo_type branches) is now wrapped in `try: ... finally: + self.progress["heartbeat"] = None` (`lamni.py:1034-1129`), so the heartbeat + is cleared on every exit path — normal completion, an exception, or an + interrupt — not just at the next scan's start. + +Both fixes are narrow and additive; no other behavior in `tomo_scan()` +changed. + ## Explicitly deferred (carried over / reconfirmed) - `zero_deg_reference_at_each_subtomo` for lamni — not implemented anywhere @@ -175,20 +206,24 @@ port of flomni's existing fix, not a new design: - New tests: `tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py` (16 tests: profile field_order/param_names/defaults guards, offset getter/setter round-trip including a same-axis-doesn't-clobber-the-other - check) and `test_lamni_tomo_angles.py` (50 tests: no-duplicate-angle check + check) and `test_lamni_tomo_angles.py` (53 tests: no-duplicate-angle check per sub-tomogram/stepsize, the pair/quad/full-8 equally-spaced invariant - per stepsize, and two tests for §3's `tomo_scan()` account handling — one - confirming an empty `active_account` skips `add_sample_database()` - entirely with `tomo_id=0`, one confirming a real account string is passed - through as-is, not `.decode()`'d). The equally-spaced tests were - sanity-checked by temporarily reverting the phase-offset fix and - confirming 9/12 of them fail as expected; the account tests were - sanity-checked the same way against the `.decode()` bug specifically, - then both reverts restored. + per stepsize, two tests for §3's `tomo_scan()` account handling, one for + §4's progress-GUI-auto-show, and two for §4's heartbeat-clearing (normal + completion and mid-scan exception — both stub `sub_tomo_scan()` to set a + busy heartbeat first, the same way the real `_tomo_scan_at_angle()` does, + so the assertion genuinely depends on the `finally`-clause and isn't + trivially satisfied by the pre-existing "new scan" branch's own earlier + reset)). The equally-spaced tests were sanity-checked by temporarily + reverting the phase-offset fix and confirming 9/12 of them fail as + expected; the account tests were sanity-checked the same way against the + `.decode()` bug specifically; the two heartbeat tests were sanity-checked + by temporarily reverting the `finally`-clause fix and confirming both fail + — all reverts restored afterward. - Existing pinned test `test_tomo_params_widget_math.py` (flomni's math) untouched and still passing. - Full offline suite (`pytest tests/tests_bec_ipython_client tests/tests_scans`): - 187 passed. + 190 passed. - Live-sim verification (plain scripts against the running simulated lamni deployment, snapshotting/restoring `tomo_queue`/`tomo_progress` around each run): @@ -208,5 +243,16 @@ port of flomni's existing fix, not a new design: and PDF/logbook write no longer crashed the scan. (Required creating `~/Data10/documentation/` locally, which didn't exist in this sandbox but is expected to already exist on the real deployed system.) + - §4: a real `lamni.tomo_scan()` against the sim (real `BECIPythonClient`, + real Redis-backed `progress`, `sub_tomo_scan()` stubbed to avoid a + multi-minute motor scan but set a busy heartbeat first, mirroring the + real per-projection behavior) confirmed `lamnigui_show_progress()` is + now called unconditionally, and `progress["heartbeat"]` round-trips to + `None` through real Redis both on normal completion and after a + mid-scan `RuntimeError`. The GUI-RPC internals inside + `lamnigui_show_progress()` itself weren't exercised here — this bare + script has no `bec.gui` attached (needs a separate GUI-client bootstrap + this repo doesn't yet have for lamni), the same limitation that would + apply to flomni's identical, already-working call in the same harness. - No live GUI verification performed (per Mirko's instruction) — verified manually by Mirko instead. diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py index 75586d5..c11a96e 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/lamni.py @@ -993,6 +993,8 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools start_angle (float, optional): Override starting angle of the first sub-tomogram. projection_number (int, optional): For tomo_types 2 and 3, resume from this index. """ + self.lamnigui_show_progress() + bec = builtins.__dict__.get("bec") scans = builtins.__dict__.get("scans") self._current_special_angles = self.special_angles.copy() @@ -1029,92 +1031,102 @@ class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools self.progress["accumulated_idle_time"] = 0.0 self.progress["heartbeat"] = None - with scans.dataset_id_on_hold: - if self.tomo_type == 1: - self.progress["tomo_type"] = "Equally spaced sub-tomograms" - for ii in range(subtomo_start, 9): - self.sub_tomo_scan(ii, start_angle=start_angle) - start_angle = None + try: + with scans.dataset_id_on_hold: + if self.tomo_type == 1: + self.progress["tomo_type"] = "Equally spaced sub-tomograms" + for ii in range(subtomo_start, 9): + self.sub_tomo_scan(ii, start_angle=start_angle) + start_angle = None - elif self.tomo_type == 2: - self.progress["tomo_type"] = "Golden ratio tomography" - previous_subtomo_number = -1 - ii = 0 if projection_number is None else projection_number - while True: - angle, subtomo_number = self._golden( - ii, self.golden_ratio_bunch_size, maxangle=360, reverse=True - ) - if previous_subtomo_number != subtomo_number: - self._write_subtomo_to_scilog(subtomo_number) - if ( - subtomo_number % 2 == 1 - and ii > 10 - and self.golden_projections_at_0_deg_for_damage_estimation == 1 - ): - self._tomo_scan_at_angle(0, subtomo_number) - previous_subtomo_number = subtomo_number - - self.progress["subtomo"] = subtomo_number - self.progress["projection"] = ii - self.progress["angle"] = angle - self.progress["subtomo_total_projections"] = self.golden_ratio_bunch_size - self.progress["subtomo_projection"] = ( - ii - (subtomo_number - 1) * self.golden_ratio_bunch_size - ) - self.progress["total_projections"] = self.golden_max_number_of_projections - - self._tomo_scan_at_angle(angle, subtomo_number) - ii += 1 - if ( - self.golden_max_number_of_projections > 0 - and ii > self.golden_max_number_of_projections - ): - print( - f"Golden ratio tomography stopped after" - f" {self.golden_max_number_of_projections} projections." + elif self.tomo_type == 2: + self.progress["tomo_type"] = "Golden ratio tomography" + previous_subtomo_number = -1 + ii = 0 if projection_number is None else projection_number + while True: + angle, subtomo_number = self._golden( + ii, self.golden_ratio_bunch_size, maxangle=360, reverse=True ) - break + if previous_subtomo_number != subtomo_number: + self._write_subtomo_to_scilog(subtomo_number) + if ( + subtomo_number % 2 == 1 + and ii > 10 + and self.golden_projections_at_0_deg_for_damage_estimation == 1 + ): + self._tomo_scan_at_angle(0, subtomo_number) + previous_subtomo_number = subtomo_number - elif self.tomo_type == 3: - self.progress["tomo_type"] = "Equally spaced, golden ratio starting angles" - previous_subtomo_number = -1 - ii = 0 if projection_number is None else projection_number - while True: - angle, subtomo_number = self._golden_equally_spaced( - ii, int(360 / self.tomo_angle_stepsize), maxangle=360, reverse=True - ) - if previous_subtomo_number != subtomo_number: - self._write_subtomo_to_scilog(subtomo_number) - if ( - subtomo_number % 2 == 1 - and ii > 10 - and self.golden_projections_at_0_deg_for_damage_estimation == 1 - ): - self._tomo_scan_at_angle(0, subtomo_number) - previous_subtomo_number = subtomo_number - - self.progress["subtomo"] = subtomo_number - self.progress["projection"] = ii - self.progress["angle"] = angle - self.progress["subtomo_total_projections"] = 360 / self.tomo_angle_stepsize - self.progress["subtomo_projection"] = ( - ii - (subtomo_number - 1) * self.progress["subtomo_total_projections"] - ) - self.progress["total_projections"] = self.golden_max_number_of_projections - - self._tomo_scan_at_angle(angle, subtomo_number) - ii += 1 - if ( - self.golden_max_number_of_projections > 0 - and ii > self.golden_max_number_of_projections - ): - print( - f"Golden ratio tomography stopped after" - f" {self.golden_max_number_of_projections} projections." + self.progress["subtomo"] = subtomo_number + self.progress["projection"] = ii + self.progress["angle"] = angle + self.progress["subtomo_total_projections"] = self.golden_ratio_bunch_size + self.progress["subtomo_projection"] = ( + ii - (subtomo_number - 1) * self.golden_ratio_bunch_size ) - break - else: - raise ValueError(f"Unknown tomo_type: {self.tomo_type}.") + self.progress["total_projections"] = self.golden_max_number_of_projections + + self._tomo_scan_at_angle(angle, subtomo_number) + ii += 1 + if ( + self.golden_max_number_of_projections > 0 + and ii > self.golden_max_number_of_projections + ): + print( + f"Golden ratio tomography stopped after" + f" {self.golden_max_number_of_projections} projections." + ) + break + + elif self.tomo_type == 3: + self.progress["tomo_type"] = "Equally spaced, golden ratio starting angles" + previous_subtomo_number = -1 + ii = 0 if projection_number is None else projection_number + while True: + angle, subtomo_number = self._golden_equally_spaced( + ii, int(360 / self.tomo_angle_stepsize), maxangle=360, reverse=True + ) + if previous_subtomo_number != subtomo_number: + self._write_subtomo_to_scilog(subtomo_number) + if ( + subtomo_number % 2 == 1 + and ii > 10 + and self.golden_projections_at_0_deg_for_damage_estimation == 1 + ): + self._tomo_scan_at_angle(0, subtomo_number) + previous_subtomo_number = subtomo_number + + self.progress["subtomo"] = subtomo_number + self.progress["projection"] = ii + self.progress["angle"] = angle + self.progress["subtomo_total_projections"] = 360 / self.tomo_angle_stepsize + self.progress["subtomo_projection"] = ( + ii - (subtomo_number - 1) * self.progress["subtomo_total_projections"] + ) + self.progress["total_projections"] = self.golden_max_number_of_projections + + self._tomo_scan_at_angle(angle, subtomo_number) + ii += 1 + if ( + self.golden_max_number_of_projections > 0 + and ii > self.golden_max_number_of_projections + ): + print( + f"Golden ratio tomography stopped after" + f" {self.golden_max_number_of_projections} projections." + ) + break + else: + raise ValueError(f"Unknown tomo_type: {self.tomo_type}.") + finally: + # Cleared on every exit path (normal completion, exception, or + # interrupt/abort), not just at the next scan's start above -- + # mirrors Flomni.tomo_scan()'s identical fix: without this, the + # GUI's busy-detectors (TomoParamsWidget's banner) only notice a + # finished scan once the heartbeat goes stale (120s), showing + # "beamline busy" for up to two minutes after a tomogram already + # completed successfully. + self.progress["heartbeat"] = None self.progress["projection"] = self.progress["total_projections"] self.progress["subtomo_projection"] = self.progress["subtomo_total_projections"] diff --git a/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py b/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py index 6894201..8a5f176 100644 --- a/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py +++ b/tests/tests_bec_ipython_client/test_lamni_tomo_angles.py @@ -165,6 +165,7 @@ def make_lamni_for_tomo_scan(tomo_angle_stepsize: float, active_account: str) -> obj.sub_tomo_scan = lambda subtomo_number, start_angle=None: None obj._print_progress = lambda: None obj._format_duration = lambda seconds: "0s" + obj.lamnigui_show_progress = lambda: None builtins.__dict__["bec"] = types.SimpleNamespace( active_account=active_account, queue=types.SimpleNamespace(next_scan_number=1) ) @@ -202,3 +203,62 @@ def test_tomo_scan_registers_sample_with_plain_string_account(): assert recorded["eaccount"] == "e12345" assert lamni.tomo_id == 42 + + +def test_tomo_scan_shows_progress_gui(): + """tomo_scan() must open/show the progress GUI unconditionally at the + start of every scan, mirroring Flomni.tomo_scan()'s + self.flomnigui_show_progress() call -- lamni.py never had this at all.""" + lamni = make_lamni_for_tomo_scan(45.0, active_account="") + lamni.add_sample_database = lambda *a, **k: 0 + calls = [] + lamni.lamnigui_show_progress = lambda: calls.append("shown") + + lamni.tomo_scan() + + assert calls == ["shown"] + + +def test_tomo_scan_clears_heartbeat_on_normal_completion(): + """The busy-detector heartbeat must be cleared as soon as the scan + finishes, not left for the next poll to time out (120s) before the GUI + stops showing "beamline busy" for an already-finished scan -- mirrors + Flomni.tomo_scan()'s identical try/finally fix. + + sub_tomo_scan() is stubbed to set a fresh heartbeat before returning, + the same way the real _tomo_scan_at_angle() does at the start of every + projection -- without that, this test would pass even without the fix, + since tomo_scan()'s own "new scan" branch already resets the heartbeat + to None *before* the scan body runs. + """ + lamni = make_lamni_for_tomo_scan(45.0, active_account="") + lamni.add_sample_database = lambda *a, **k: 0 + + def _fake_sub_tomo_scan(subtomo_number, start_angle=None): + lamni.progress["heartbeat"] = "2026-01-01T00:00:00" + + lamni.sub_tomo_scan = _fake_sub_tomo_scan + + lamni.tomo_scan() + + assert lamni.progress["heartbeat"] is None + + +def test_tomo_scan_clears_heartbeat_even_if_scan_raises(): + """The heartbeat must be cleared on every exit path, including a + mid-scan exception -- otherwise a crashed scan looks permanently "busy" + to the GUI until the 120s staleness timeout, or forever if polled more + often than that.""" + lamni = make_lamni_for_tomo_scan(45.0, active_account="") + lamni.add_sample_database = lambda *a, **k: 0 + + def _boom(subtomo_number, start_angle=None): + lamni.progress["heartbeat"] = "2026-01-01T00:00:00" + raise RuntimeError("simulated crash mid-scan") + + lamni.sub_tomo_scan = _boom + + with pytest.raises(RuntimeError): + lamni.tomo_scan() + + assert lamni.progress["heartbeat"] is None -- 2.54.0 From 6744908983daeb22901d363113f4b60dd8305f8e Mon Sep 17 00:00:00 2001 From: x01dc Date: Thu, 16 Jul 2026 14:22:45 +0200 Subject: [PATCH 5/5] docs(lamni): mark flomni-to-lamni migration status, only webpage left FLOMNI_TO_LAMNI_COMPARISON.md was written before any porting work started and read as a stale plan once the tomo-queue backend and tomo-params GUI were both done. Add a status pointer at the top so a fresh session can find what's actually left (the webpage generator) without re-deriving it. Co-Authored-By: Claude Sonnet 5 --- .../LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md | 2 +- .../AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md | 26 ++++++++++++++++++- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md index b214f74..754bb69 100644 --- a/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md +++ b/csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md @@ -15,7 +15,7 @@ each was a direct port, not a new design. **Status:** All of the below implemented, tested, and live-verified against the running simulated lamni deployment. §1/§2 committed as `bac3fa8`, §3 as -`f3c90c5`; §4 not yet committed as of this doc. +`f3c90c5`; §4 as `46cadc8`. ## 1. GUI: `TomoParamsWidget` generalized to lamni diff --git a/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md b/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md index b66f2ca..4f3aaf5 100644 --- a/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md +++ b/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md @@ -3,7 +3,31 @@ Comparison of the `flomni` and `LamNI` ipython-client plugins, written to scope which flomni features (GUI, status webpage, tomo-parameter widget, angle distribution) are worth porting to lamni. This is a comparison only — no -migration has been implemented yet. +migration had been implemented yet as of when it was originally written. + +**Status update (2026-07-16): items 2–4 of §5's recommended order are now +done.** Only **item 1, the webpage generator (§4), remains open** — the rest +of this document (originally a forward-looking comparison/plan) is now a +historical snapshot of the *starting* gap, not the current state. See: + +- `csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md` — + tomo-queue/job-list backend ported to lamni via a shared `TomoQueueMixin` + (`csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py`). +- `csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md` — + `TomoParamsWidget` generalized to lamni (setup detection, per-setup field + lists/order, lamni-only offsets section, "Duplicate selected" queue + button), plus several real `lamni.py` bugs found and fixed along the way + (two `sub_tomo_scan()` angle bugs, three `tomo_scan()`/`write_pdf_report()` + crashes, progress-GUI-auto-show + stale-busy-heartbeat). +- **Remaining**: `LamNI_webpage_generator.py` is still the stub described in + §4 below — `_collect_setup_data()` commented out, `_TEMP_MAP` empty, + `HAS_TOMO_QUEUE = False` (now incorrect — a queue backend exists, see + above), `TOMO_TYPES` placeholders, and never wired into + `lamni.py.__init__`. §4's 5-step plan is otherwise still accurate. +- Also still explicitly deferred (confirmed with Mirko, separate future + tasks): `zero_deg_reference_at_each_subtomo` for lamni (not implemented at + all yet), and sharing angle-calculation code between flomni/lamni + (currently same *shape* but not literally shared/DRY'd). ## 1. Scope & method -- 2.54.0