Feat/feat from flomni to lamni #261
@@ -0,0 +1,258 @@
|
||||
# Tomo-params GUI generalization + real lamni.py bugs found and fixed
|
||||
|
||||
Companion to `AI_docs/TOMO_QUEUE_PORT.md` (the backend port this GUI work
|
||||
builds on) and `csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md`
|
||||
(which identified `TomoParamsWidget` as flomni-gated). This session's
|
||||
follow-up work: generalized the widget to also support lamni, then — while
|
||||
verifying the widget's projection-count preview against the CLI, and later
|
||||
just running `lamni.tomo_scan()` for real — found and fixed several real,
|
||||
pre-existing bugs/gaps in `lamni.py` that predate this session: two in
|
||||
`sub_tomo_scan()`'s angle math (§2), three crashes in `tomo_scan()`/
|
||||
`write_pdf_report()`'s "new scan" bookkeeping (§3), and two more UX/GUI gaps
|
||||
in `tomo_scan()` found once a scan could finally complete end to end (§4) —
|
||||
all of §3 and §4 already correctly handled in flomni's equivalent code, so
|
||||
each was a direct port, not a new design.
|
||||
|
||||
**Status:** All of the below implemented, tested, and live-verified against
|
||||
the running simulated lamni deployment. §1/§2 committed as `bac3fa8`, §3 as
|
||||
`f3c90c5`; §4 as `46cadc8`.
|
||||
|
||||
## 1. GUI: `TomoParamsWidget` generalized to lamni
|
||||
|
||||
`csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py`'s `SETUP_PROFILES`
|
||||
dict (one entry per setup) now drives nearly everything setup-specific:
|
||||
|
||||
- **Setup detection**: `_detect_setup()` checks for `fsamroy` (flomni) or
|
||||
`lsamrot` (lamni) — real discriminator devices, same pattern the CLI side
|
||||
already used, just generalized from a yes/no flomni check.
|
||||
- **Per-setup field lists**: `fov_fields`/`stitch_fields`/`manual_shift_fields`/
|
||||
`piezo_range_fields`/`offset_fields`, each a list of `(key, label, min, max,
|
||||
decimals)` tuples the widget iterates to build/validate/read fields —
|
||||
small per-concept lists, not a generic form builder.
|
||||
- **`field_order`**: an explicit per-profile list of section tokens
|
||||
(`_build_params_panel` is now a dispatch loop over it). flomni's order is
|
||||
its unchanged, pre-existing order (verified as a no-op via a regression
|
||||
test); lamni's order mirrors `lamni.tomo_parameters()`'s own print/edit
|
||||
order exactly (counting time → shell step → piezo range → stitching →
|
||||
stitch overlap → circular FOV → reconstruct folder → frames/trigger →
|
||||
offsets → at-each-angle hook).
|
||||
- **Offsets section** (lamni only): `tomo_fovx_offset`/`tomo_fovy_offset`
|
||||
(from `LamNIAlignmentMixin`) plus `manual_shift_x`/`_y`, with a "?" help
|
||||
button. These are alignment values, not scan parameters — shown/editable
|
||||
here but **excluded from queue job snapshots** (`add_edited_to_queue()`/
|
||||
`TomoQueueDialog.add_to_queue()` now explicitly filter to
|
||||
`self._profile["param_names"]`), same treatment as flomni's
|
||||
`corr_pos_y`/`corr_angle_y` (also alignment-only, also not shown by this
|
||||
widget). Needed dedicated accessors (`_lamni_get_tomo_fov_offset`/
|
||||
`_lamni_set_tomo_fov_offset`) because both axes are packed into one
|
||||
combined global var (`tomo_fov_offset = [x_um, y_um]`, exposed in mm) —
|
||||
the widget's usual direct-global-var-by-name mechanism doesn't work for
|
||||
them.
|
||||
- **Lamni projection math** (`_lamni_compute_type1`/`_lamni_requested_to_stepsize`):
|
||||
see §2 below — simplified once the backend bug was fixed, now the same
|
||||
shape as flomni's math (`N*8`, base angle 360 instead of 180).
|
||||
- **`TomoQueueDialog` bugs fixed** (pre-existing, silently wrong for any
|
||||
lamni job before this session, since no lamni job had ever been queued
|
||||
through the GUI): `_format_projections()` always used flomni's fixed-180
|
||||
formula regardless of which setup a job belonged to; `_job_tooltip()`
|
||||
iterated a hardcoded flomni-only param-name tuple, so a lamni job's
|
||||
tooltip silently showed no params at all. Both now duck-type off the
|
||||
job's own params dict (`"tomo_angle_range" in params` distinguishes a
|
||||
flomni job from a lamni job) rather than needing to know which setup is
|
||||
active.
|
||||
- **New "Duplicate selected" queue button**: copies the selected job (any
|
||||
status, any kind — tomo or command) to the end of the queue with a fresh
|
||||
id, `status="pending"`, and `_dup` appended to the label. Setup-agnostic,
|
||||
no profile-specific logic.
|
||||
|
||||
## 2. Backend: two real bugs found and fixed in `LamNI.sub_tomo_scan()`
|
||||
|
||||
Surfaced while checking the GUI's projection-count preview against
|
||||
`lamni.tomo_parameters()`'s wizard — both predate this session and are
|
||||
independent of the GUI work; they affect the CLI (and therefore every real
|
||||
scan) regardless of whether the GUI is ever used.
|
||||
|
||||
### 2a. Duplicate closing angle
|
||||
|
||||
`sub_tomo_scan()` generated each sub-tomogram's angles with
|
||||
`np.linspace(start, start+360, num=int(360/stepsize)+1, endpoint=True)`. For
|
||||
a full 360° sweep, `start+360` is the *same physical angle* as `start`
|
||||
(360°≡0°), so every sub-tomogram re-measured its own starting angle a
|
||||
second time. Confirmed numerically: requesting 66 projections set
|
||||
`tomo_angle_stepsize=43.6364`; the scan then actually ran **72** acquisitions
|
||||
(9/sub-tomogram × 8), not 66 — `tomo_parameters()`'s own "Resulting in
|
||||
number of projections" printout was never accurate for *any* input, since it
|
||||
used plain float division with no `int()` truncation and no duplicate-point
|
||||
awareness.
|
||||
|
||||
Fix: new `LamNI._tomo_type1_actual_grid()` (mirrors
|
||||
`Flomni._tomo_type1_actual_grid()`, base angle 360 instead of 180, no `+1`
|
||||
since the closing point is a duplicate not a new measurement) is now the
|
||||
single source of truth for `N`/achievable step/total, used by
|
||||
`sub_tomo_scan()` (now `num=N, endpoint=False`), `tomo_parameters()`, and
|
||||
`write_pdf_report()` alike — no more three independent, disagreeing
|
||||
formulas.
|
||||
|
||||
### 2b. Phase offset computed from the wrong step
|
||||
|
||||
Separately, `sub_tomo_scan()`'s inter-sub-tomogram phase offset was
|
||||
`self.tomo_angle_stepsize / 8.0 * offsets[subtomo_number]` — using the
|
||||
**raw, requested** stepsize, not the achievable one (`360/N` after `int()`
|
||||
truncation). These only coincide when `360/stepsize` happens to already be a
|
||||
whole number. Flomni's equivalent (`_subtomo_angle_plan()`) already gets
|
||||
this right and says why in its own comment: *"N/step not guaranteed to be a
|
||||
whole/exact division of the configured tomo_angle_stepsize ... This
|
||||
corrected step ... is used for BOTH the per-point ramp AND the
|
||||
inter-sub-tomogram phase offsets."* Lamni's phase offset never got that
|
||||
treatment.
|
||||
|
||||
Verified numerically (stepsize=43.6364, achievable step=45): combining
|
||||
sub-tomograms in pairs `(1,2)`/`(3,4)`/`(5,6)`/`(7,8)`, quads
|
||||
`(1,2,3,4)`/`(5,6,7,8)`, or all 8 produced **unevenly spaced** angles with
|
||||
the raw-stepsize phase (e.g. pair spacing alternating 21.8°/23.2° instead of
|
||||
a uniform 22.5°) — defeating the entire point of the bit-reversal
|
||||
interlacing scheme, which exists specifically to give progressively finer,
|
||||
*evenly* sampled combined tomograms at each combination level. Fixed by
|
||||
computing the phase from the achievable step instead, exactly like flomni.
|
||||
|
||||
Also added, matching flomni's wizard UX: `tomo_parameters()` now prints
|
||||
`"Note: N projections does not divide evenly into 8 equally spaced
|
||||
sub-tomograms over 360 degrees; adjusted to the nearest achievable total of
|
||||
M projections..."` when the requested and achievable totals differ — this
|
||||
is **display only**, not a substitute for the `sub_tomo_scan()` fix: the
|
||||
stored `tomo_angle_stepsize` is never rewritten to the achievable value (in
|
||||
flomni either — verified by reading its wizard code), so `sub_tomo_scan()`
|
||||
must keep re-deriving the achievable grid every time it runs, regardless of
|
||||
which code path set `tomo_angle_stepsize` (wizard, GUI Submit, queue-job
|
||||
restore, or a direct script assignment).
|
||||
|
||||
## 3. `tomo_scan()`/`write_pdf_report()`: three crashes on a real scan
|
||||
|
||||
Found by simply calling `lamni.tomo_scan()` for real (not just
|
||||
`sub_tomo_scan()` in isolation, which §2's tests/live checks exercised) —
|
||||
all three already fixed in flomni's equivalent code, so each was a direct
|
||||
port of flomni's existing fix, not a new design:
|
||||
|
||||
- **`bec.active_account.decode()`** (`tomo_scan()`'s new-scan branch):
|
||||
`active_account` is a plain `str`, not `bytes` — `.decode()` raises
|
||||
`AttributeError` on every real call. Flomni's equivalent code never had
|
||||
this `.decode()` at all. Also added flomni's existing guard: an empty
|
||||
`active_account` (e.g. a dev/sim session not logged into a real e-account)
|
||||
now skips `add_sample_database()` and falls back to `tomo_id = 0`, instead
|
||||
of crashing while trying to register a sample under `""`. Same fix
|
||||
applied to `DataDrivenLamNI.tomo_scan()` (`extra_tomo.py`), which had an
|
||||
identical inline copy of this bug.
|
||||
- **`dev.mokev.read(...)` in `write_pdf_report()`**: `mokev` isn't a real
|
||||
device name here — Mirko identified the correct one, `dev.ccm_energy`
|
||||
(confirmed against `flomni_optics_mixin.py`/`eiger.py`'s own `ccm_energy`
|
||||
usage elsewhere in the repo). Per Mirko's request, this isn't a flomni-
|
||||
style hardcoded placeholder ("To be implemented") — it now does a real
|
||||
`dev.ccm_energy.read(cached=True)[dev.ccm_energy.name]['value']` read,
|
||||
wrapped in a broad `try/except` that falls back to `"N/A"` if the device
|
||||
isn't configured/available in a given session, so a missing device is
|
||||
tolerated rather than crashing report generation.
|
||||
- **`bec.logbook.LogbookMessage()` in `write_pdf_report()`**: crashes with
|
||||
`AttributeError` when scilog/logbook isn't configured (e.g. a dev/sim
|
||||
session) — `lamni.py` already has exactly this tolerance pattern one
|
||||
method up, in `write_to_scilog()` (broad `try/except`, warn and continue),
|
||||
it just wasn't applied here. Wrapped the same logbook-message block in an
|
||||
identical `try/except`, logging `"Failed to write PDF report to
|
||||
scilog."` on failure — the PDF file itself is still written either way
|
||||
(it happens earlier in the same function, unaffected).
|
||||
|
||||
## 4. `tomo_scan()`: progress GUI not shown, stale "busy" heartbeat after success
|
||||
|
||||
Two more gaps found by Mirko after §3's crashes were fixed and a real scan
|
||||
could finally run start to finish — both already fixed in flomni's
|
||||
equivalent code, both direct ports:
|
||||
|
||||
- **Progress GUI never opened automatically.** `Flomni.tomo_scan()` calls
|
||||
`self.flomnigui_show_progress()` unconditionally as its first action;
|
||||
`LamNI.tomo_scan()` never had an equivalent call at all, so starting a
|
||||
lamni tomogram never opened/raised the progress window the way flomni's
|
||||
does. Fixed by adding `self.lamnigui_show_progress()` as the first line of
|
||||
`tomo_scan()` (`lamni.py:996`), mirroring flomni exactly.
|
||||
- **Stale "beamline busy" heartbeat after a scan already finished.**
|
||||
`progress["heartbeat"]` (the GUI's busy-detector field, refreshed on every
|
||||
real projection by `_tomo_scan_at_angle()`) was only ever reset at the
|
||||
*start* of a new scan (the "new scan" branch), never on completion. A
|
||||
finished (or crashed/interrupted) scan's last heartbeat timestamp lingered
|
||||
until the GUI's own staleness timeout (120s) finally decided it was stale —
|
||||
so the GUI kept showing "beamline busy" for up to two minutes after a
|
||||
tomogram had already completed successfully. Flomni hit and fixed this
|
||||
exact issue earlier in this session (commit `aebfccc`); same fix ported:
|
||||
the entire `with scans.dataset_id_on_hold: ...` scan body (all three
|
||||
tomo_type branches) is now wrapped in `try: ... finally:
|
||||
self.progress["heartbeat"] = None` (`lamni.py:1034-1129`), so the heartbeat
|
||||
is cleared on every exit path — normal completion, an exception, or an
|
||||
interrupt — not just at the next scan's start.
|
||||
|
||||
Both fixes are narrow and additive; no other behavior in `tomo_scan()`
|
||||
changed.
|
||||
|
||||
## Explicitly deferred (carried over / reconfirmed)
|
||||
|
||||
- `zero_deg_reference_at_each_subtomo` for lamni — not implemented anywhere
|
||||
in `lamni.py` (no property, no scan logic). Mirko confirmed: separate
|
||||
future task, backend first then GUI.
|
||||
- Shared angle-calculation code between flomni/lamni (`sub_tomo_scan`/
|
||||
`_golden`/`tomo_scan` dispatch) — Mirko confirmed: separate follow-up, not
|
||||
folded into this GUI-focused work, despite the two setups' type-1 math now
|
||||
having the same *shape* (`N*8`, differing only in base angle).
|
||||
- `LamniWebpageGenerator` completion — unrelated gap, still open.
|
||||
|
||||
## Verification
|
||||
|
||||
- New tests: `tests/tests_bec_ipython_client/test_lamni_tomo_params_widget_math.py`
|
||||
(16 tests: profile field_order/param_names/defaults guards, offset
|
||||
getter/setter round-trip including a same-axis-doesn't-clobber-the-other
|
||||
check) and `test_lamni_tomo_angles.py` (53 tests: no-duplicate-angle check
|
||||
per sub-tomogram/stepsize, the pair/quad/full-8 equally-spaced invariant
|
||||
per stepsize, two tests for §3's `tomo_scan()` account handling, one for
|
||||
§4's progress-GUI-auto-show, and two for §4's heartbeat-clearing (normal
|
||||
completion and mid-scan exception — both stub `sub_tomo_scan()` to set a
|
||||
busy heartbeat first, the same way the real `_tomo_scan_at_angle()` does,
|
||||
so the assertion genuinely depends on the `finally`-clause and isn't
|
||||
trivially satisfied by the pre-existing "new scan" branch's own earlier
|
||||
reset)). The equally-spaced tests were sanity-checked by temporarily
|
||||
reverting the phase-offset fix and confirming 9/12 of them fail as
|
||||
expected; the account tests were sanity-checked the same way against the
|
||||
`.decode()` bug specifically; the two heartbeat tests were sanity-checked
|
||||
by temporarily reverting the `finally`-clause fix and confirming both fail
|
||||
— all reverts restored afterward.
|
||||
- Existing pinned test `test_tomo_params_widget_math.py` (flomni's math)
|
||||
untouched and still passing.
|
||||
- Full offline suite (`pytest tests/tests_bec_ipython_client tests/tests_scans`):
|
||||
190 passed.
|
||||
- Live-sim verification (plain scripts against the running simulated lamni
|
||||
deployment, snapshotting/restoring `tomo_queue`/`tomo_progress` around
|
||||
each run):
|
||||
- Real `sub_tomo_scan(1)` with a coarse stepsize (170°, N=2): exactly 2
|
||||
real acquisitions (`[0.0, 180.0]`), no duplicate; `tomo_parameters()`'s
|
||||
display matched (16 = 2×8) after the fix (was previously wrong for
|
||||
every input).
|
||||
- Real `sub_tomo_scan(1)` + `sub_tomo_scan(2)` with a non-achievable
|
||||
stepsize (109.09°, N=3): combined pair sorted to
|
||||
`[0, 60, 120, 180, 240, 300]`, spacing exactly 60° throughout —
|
||||
confirms the phase-offset fix holds against real (not just mocked)
|
||||
execution.
|
||||
- A full real `lamni.tomo_scan()` (type 1, coarse stepsize giving N=1,
|
||||
8 total real acquisitions across all 8 sub-tomograms) ran end to end
|
||||
against the sim without error after §3's fixes — `tomo_id` correctly
|
||||
fell back to `0` (no active account in this session), the energy read
|
||||
and PDF/logbook write no longer crashed the scan. (Required creating
|
||||
`~/Data10/documentation/` locally, which didn't exist in this sandbox
|
||||
but is expected to already exist on the real deployed system.)
|
||||
- §4: a real `lamni.tomo_scan()` against the sim (real `BECIPythonClient`,
|
||||
real Redis-backed `progress`, `sub_tomo_scan()` stubbed to avoid a
|
||||
multi-minute motor scan but set a busy heartbeat first, mirroring the
|
||||
real per-projection behavior) confirmed `lamnigui_show_progress()` is
|
||||
now called unconditionally, and `progress["heartbeat"]` round-trips to
|
||||
`None` through real Redis both on normal completion and after a
|
||||
mid-scan `RuntimeError`. The GUI-RPC internals inside
|
||||
`lamnigui_show_progress()` itself weren't exercised here — this bare
|
||||
script has no `bec.gui` attached (needs a separate GUI-client bootstrap
|
||||
this repo doesn't yet have for lamni), the same limitation that would
|
||||
apply to flomni's identical, already-working call in the same harness.
|
||||
- No live GUI verification performed (per Mirko's instruction) — verified
|
||||
manually by Mirko instead.
|
||||
@@ -0,0 +1,110 @@
|
||||
# Tomo queue / at-each-angle hook backend — ported from flomni
|
||||
|
||||
First `AI_docs/` entry for LamNI. Companion to flomni's
|
||||
`AI_docs/TOMO_QUEUE_*.md` and `AI_docs/TOMO_QUEUE_COMMAND_JOBS_PLAN.md`, and
|
||||
to `AI_docs/FLOMNI_TO_LAMNI_COMPARISON.md` (flomni plugin), which first
|
||||
identified this backend as flomni-only with no lamni equivalent.
|
||||
|
||||
**Status:** Backend ported, unit-tested against a fake client, and verified
|
||||
against the live simulated deployment for both setups (§ Verification below).
|
||||
GUI (`TomoParamsWidget`) is a separate, later step and still flomni-gated.
|
||||
|
||||
## What changed
|
||||
|
||||
The tomo-queue backend (job list, resume-on-crash, named per-projection
|
||||
hooks, command-job dispatch) used to live entirely in `flomni.py`. It's now a
|
||||
shared `TomoQueueMixin` in
|
||||
`csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py`, and
|
||||
both `Flomni` and `LamNI` inherit it — see that module's docstring for the
|
||||
full contract. This was a behavior-preserving refactor of `flomni.py` (moved
|
||||
code, not changed logic); `LamNI` is the new consumer.
|
||||
|
||||
`LamNI` now has, for the first time:
|
||||
- `tomo_queue_add()` / `tomo_queue_add_command()` / `tomo_queue_delete()` /
|
||||
`tomo_queue_move()` / `tomo_queue_clear()` / `tomo_queue_show()` /
|
||||
`tomo_queue_execute()`
|
||||
- `register_at_each_angle_hook()` / `unregister_at_each_angle_hook()` /
|
||||
`list_at_each_angle_hooks()` / `at_each_angle_hook` property
|
||||
- `tomo_scan_resume()` — **new method, not previously present in `lamni.py`**.
|
||||
`tomo_queue_execute()` needs it to resume a crashed/interrupted job instead
|
||||
of restarting from scratch; flomni already had this, lamni didn't.
|
||||
Implemented identically to flomni's version (same `tomo_type`-based
|
||||
dispatch: `subtomo_start`/`start_angle` for type 1, `projection_number` for
|
||||
types 2/3 — lamni's `tomo_scan()` signature already matched flomni's
|
||||
exactly, so this was a direct port, no design changes needed).
|
||||
|
||||
`LamNI._at_each_angle()` now checks `self.at_each_angle_hook` first (same as
|
||||
flomni's), falling through to its existing default body
|
||||
(`tomo_scan_projection()` + `tomo_reconstruct()`) unchanged when no hook is
|
||||
set. `MagLamNI`/`DataDrivenLamNI` (`extra_tomo.py`) override `_at_each_angle`
|
||||
entirely and are unaffected by this change.
|
||||
|
||||
## Parameter name mapping (`_TOMO_SCAN_PARAM_NAMES`)
|
||||
|
||||
Lamni's existing global-var names are kept as-is — no renaming, since that
|
||||
would break any in-flight session or persisted param snapshot. Where flomni
|
||||
and lamni have overlapping concepts under different names, `LamNI`'s own
|
||||
tuple uses lamni's names:
|
||||
|
||||
| Concept | flomni | lamni |
|
||||
|---|---|---|
|
||||
| Field of view | `fovx` / `fovy` | `tomo_circfov` (single value — laminography FOV is circular, not rectangular) |
|
||||
| Stitching | `stitch_x` / `stitch_y` | `lamni_stitch_x` / `lamni_stitch_y` |
|
||||
| Piezo range | *(no equivalent)* | `lamni_piezo_range_x` / `lamni_piezo_range_y` |
|
||||
| Manual shift | `manual_shift_y` only | `manual_shift_x` **and** `manual_shift_y` |
|
||||
| 180°/360° toggle | `tomo_angle_range` | *(omitted — lamni is 360°-only, no property exists)* |
|
||||
| Single-point acquisition | `single_point_instead_of_fermat_scan` / `single_point_random_shift_max` | *(omitted — no code path exists in lamni)* |
|
||||
| Zero-deg reference per subtomo | `zero_deg_reference_at_each_subtomo` | *(omitted — no property exists)* |
|
||||
|
||||
Everything else (`tomo_countingtime`, `tomo_shellstep`, `tomo_stitch_overlap`,
|
||||
`ptycho_reconstruct_foldername`, `frames_per_trigger`, `tomo_type`,
|
||||
`tomo_angle_stepsize`, `golden_ratio_bunch_size`,
|
||||
`golden_max_number_of_projections`,
|
||||
`golden_projections_at_0_deg_for_damage_estimation`, `corridor_size`,
|
||||
`at_each_angle_hook`) uses identical names on both setups.
|
||||
|
||||
## Assumption flagged for Mirko
|
||||
|
||||
`_TOMO_QUEUE_MOVE_DEVICES` for lamni is `{mokev, idgap}` only — the same
|
||||
beamline-wide devices flomni's allow-list includes. Flomni's allow-list also
|
||||
has `ftray` (its sample-transfer tray) as a "sim-testable bonus" move target;
|
||||
lamni has no equivalent device wired in, so it was left out. If lamni has its
|
||||
own analogous device worth adding to the move allow-list, say so and it's a
|
||||
one-line addition to `LamNI._TOMO_QUEUE_MOVE_DEVICES`.
|
||||
|
||||
## Explicitly deferred
|
||||
|
||||
- `TomoParamsWidget` GUI changes — **done, see `AI_docs/TOMO_PARAMS_GUI_PORT.md`**
|
||||
(was still flomni-gated when this doc was first written).
|
||||
- `LamniWebpageGenerator` completion — separate, unrelated gap (see
|
||||
`FLOMNI_TO_LAMNI_COMPARISON.md` §4).
|
||||
|
||||
## Verification
|
||||
|
||||
Offline suites (`pytest tests/tests_bec_ipython_client tests/tests_scans`):
|
||||
121 passed, including all 60 pre-existing flomni angle-math regression tests
|
||||
(`test_flomni_tomo_angles.py`) unchanged, plus the new 10-test
|
||||
`test_lamni_tomo_queue.py` covering param snapshotting, fresh vs. resumed
|
||||
execution, command-job move validation/dispatch, and hook registration/
|
||||
dispatch (with and without a registered hook) against a fake client.
|
||||
|
||||
Live-sim checks (both run as plain scripts against the already-running
|
||||
simulated deployment, not through `pytest_bec_e2e` — that plugin isn't
|
||||
installed in this venv):
|
||||
|
||||
- **lamni**: constructed a real `LamNI(bec)`, exercised
|
||||
`tomo_queue_add`/`tomo_queue_add_command`/`tomo_queue_show`/
|
||||
`register_at_each_angle_hook`/`tomo_queue_delete` against the live global-var
|
||||
store, confirmed the queue/hook state was restored to exactly what it was
|
||||
before (no residue left for the interactive `bec --session lamni` session
|
||||
sharing the same redis). No `tomo_queue_execute()`/real scan attempted here.
|
||||
- **flomni**: reused `omny_e2e_tests/_bootstrap.py`'s `build_flomni()` +
|
||||
`_queue_helpers.py`'s `SHORT_SCAN_PARAMS`/`FAST_STEPSIZE` (the same fixtures
|
||||
`omny_e2e_tests/test_tomo_queue_command_jobs.py` uses) to confirm the
|
||||
refactored `Flomni` is unchanged in practice: command-job add-time
|
||||
validation, the `tomo_queue_actions` global var, a real `ftray` move via
|
||||
`tomo_queue_execute()`, and a full real single-point tomo scan
|
||||
(`tomo_queue_add()` → `tomo_queue_execute()` → `tomo_scan()` →
|
||||
`_tomo_scan_at_angle()` → `_at_each_angle()` → registered hook) — 161
|
||||
projections, hook invoked for every one, job marked done, queue/progress
|
||||
restored afterward. Ran twice for reproducibility.
|
||||
@@ -102,15 +102,21 @@ class DataDrivenLamNI(LamNI):
|
||||
self._current_special_angles = self.special_angles.copy()
|
||||
|
||||
if subtomo_start == 1 and start_index is None:
|
||||
self.tomo_id = self.add_sample_database(
|
||||
self.sample_name,
|
||||
str(datetime.date.today()),
|
||||
bec.active_account.decode(),
|
||||
bec.queue.next_scan_number,
|
||||
"lamni",
|
||||
"test additional info",
|
||||
"BEC",
|
||||
)
|
||||
# bec.active_account is already a plain str, not bytes -- see
|
||||
# LamNI.tomo_scan()'s identical fix for why .decode() crashes and
|
||||
# why an empty active_account needs its own guard.
|
||||
if bec.active_account != "":
|
||||
self.tomo_id = self.add_sample_database(
|
||||
self.sample_name,
|
||||
str(datetime.date.today()),
|
||||
bec.active_account,
|
||||
bec.queue.next_scan_number,
|
||||
"lamni",
|
||||
"test additional info",
|
||||
"BEC",
|
||||
)
|
||||
else:
|
||||
self.tomo_id = 0
|
||||
self.write_pdf_report()
|
||||
|
||||
with scans.dataset_id_on_hold:
|
||||
|
||||
@@ -16,6 +16,7 @@ from csaxs_bec.bec_ipython_client.plugins.omny.omny_general_tools import (
|
||||
PtychoReconstructor,
|
||||
TomoIDManager,
|
||||
)
|
||||
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.tomo_queue_mixin import TomoQueueMixin
|
||||
from csaxs_bec.bec_ipython_client.plugins.LamNI.gui_tools import LamniGuiTools
|
||||
from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni_alignment_mixin import LamNIAlignmentMixin
|
||||
|
||||
@@ -106,7 +107,80 @@ class _ProgressProxy:
|
||||
return self._load()
|
||||
|
||||
|
||||
class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
class LamNI(TomoQueueMixin, LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
# Ordered set of all global-var-backed tomo scan parameters snapshotted
|
||||
# by tomo_queue_add()/restored by tomo_queue_execute(). Lamni's own
|
||||
# names for overlapping flomni concepts (tomo_circfov instead of
|
||||
# fovx/fovy, lamni_stitch_x/y instead of stitch_x/y), no
|
||||
# tomo_angle_range or single_point_* -- lamni is laminography: 360-only,
|
||||
# no 180 mode, no single-point acquisition.
|
||||
_TOMO_SCAN_PARAM_NAMES = (
|
||||
"tomo_countingtime",
|
||||
"tomo_shellstep",
|
||||
"tomo_circfov",
|
||||
"lamni_stitch_x",
|
||||
"lamni_stitch_y",
|
||||
"tomo_stitch_overlap",
|
||||
"ptycho_reconstruct_foldername",
|
||||
"manual_shift_x",
|
||||
"manual_shift_y",
|
||||
"lamni_piezo_range_x",
|
||||
"lamni_piezo_range_y",
|
||||
"frames_per_trigger",
|
||||
"tomo_type",
|
||||
"tomo_angle_stepsize",
|
||||
"golden_ratio_bunch_size",
|
||||
"golden_max_number_of_projections",
|
||||
"golden_projections_at_0_deg_for_damage_estimation",
|
||||
"corridor_size",
|
||||
"at_each_angle_hook",
|
||||
)
|
||||
|
||||
# Curated allow-list for the "move" queue action -- same beamline-wide
|
||||
# devices as flomni's (mokev, idgap). Lamni has no equivalent of
|
||||
# flomni's ftray (sample-transfer tray) sim-testable bonus target.
|
||||
_TOMO_QUEUE_MOVE_DEVICES = {
|
||||
"mokev": {"label": "Energy", "unit": "keV"},
|
||||
"idgap": {"label": "Undulator gap", "unit": "mm"},
|
||||
}
|
||||
|
||||
# Named-action registry for tomo-queue command jobs -- identical to
|
||||
# flomni's, since both actions (move, optimize_idgap) are beamline-wide,
|
||||
# not flomni-specific hardware. See TomoQueueMixin._queue_action_move()/
|
||||
# optimize_idgap() for the implementations.
|
||||
_TOMO_QUEUE_ACTIONS = {
|
||||
"move": {
|
||||
"func": "_queue_action_move",
|
||||
"idempotent": True, # absolute moves re-run harmlessly
|
||||
"help": "Move device(s) to absolute position(s).",
|
||||
"params": {"positions": {"type": "device_positions"}},
|
||||
},
|
||||
"optimize_idgap": {
|
||||
"func": "optimize_idgap",
|
||||
"idempotent": True,
|
||||
"help": "Scan idgap over a search range and move to the peak. STUB: no-op.",
|
||||
"params": {
|
||||
"search_range": {
|
||||
"type": "float",
|
||||
"default": 0.5,
|
||||
"min": 0.0,
|
||||
"max": 2.0,
|
||||
"unit": "mm",
|
||||
"label": "Search range",
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
def _describe_tomo_job_line(self, job: dict) -> str:
|
||||
"""lamni's tomo_queue_show() one-line summary: circfov instead of
|
||||
fov, no range= (lamni is 360-only)."""
|
||||
p = job["params"]
|
||||
return (
|
||||
f"type={p['tomo_type']} circfov={p['tomo_circfov']}um "
|
||||
f"step={p['tomo_shellstep']}um ctime={p['tomo_countingtime']}s"
|
||||
)
|
||||
|
||||
def __init__(self, client):
|
||||
super().__init__()
|
||||
self.client = client
|
||||
@@ -142,6 +216,7 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
# tomo_scan()); use tomo_progress_reset() to explicitly clear stale
|
||||
# progress without starting a new scan.
|
||||
self._progress_proxy = _ProgressProxy(self.client)
|
||||
self._init_tomo_queue()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Special angles
|
||||
@@ -683,6 +758,19 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
)
|
||||
|
||||
def _at_each_angle(self, angle: float) -> None:
|
||||
hook_name = self.at_each_angle_hook
|
||||
if hook_name:
|
||||
hook = self._at_each_angle_hooks.get(hook_name)
|
||||
if hook is None:
|
||||
raise ValueError(
|
||||
f"at_each_angle_hook '{hook_name}' is not registered in this "
|
||||
"session. Hooks are session-only and do not survive a kernel "
|
||||
f"restart -- call register_at_each_angle_hook({hook_name!r}, "
|
||||
"<func>) again, then re-run tomo_queue_execute() to resume."
|
||||
)
|
||||
hook(self, angle)
|
||||
return
|
||||
|
||||
self.tomo_scan_projection(angle)
|
||||
self.tomo_reconstruct()
|
||||
|
||||
@@ -729,21 +817,48 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
# Tomo scan orchestration
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _tomo_type1_actual_grid(self) -> tuple[int, float, int]:
|
||||
"""Compute the actual (achievable) tomo_type==1 grid from the
|
||||
currently stored self.tomo_angle_stepsize -- the SAME way
|
||||
sub_tomo_scan() does it. Returns (N, step, total_projections): N
|
||||
unique angles per sub-tomogram, achievable per-projection step,
|
||||
N*8 total.
|
||||
|
||||
A full 360-degree sweep's closing angle (start_angle + 360) is the
|
||||
same physical angle as its start (360 deg == 0 deg), so N (not N+1)
|
||||
is the right point count for one sub-tomogram -- see
|
||||
sub_tomo_scan()'s linspace call, endpoint=False for the same reason.
|
||||
Mirrors Flomni._tomo_type1_actual_grid() (base angle 360 instead of
|
||||
180, no "+1": flomni's own 180-degree-span scheme never had a
|
||||
self-duplicating endpoint to begin with).
|
||||
"""
|
||||
N = int(360.0 / self.tomo_angle_stepsize)
|
||||
step = 360.0 / N
|
||||
return N, step, N * 8
|
||||
|
||||
def sub_tomo_scan(self, subtomo_number, start_angle=None):
|
||||
"""Perform one sub-tomogram (tomo_type 1 only)."""
|
||||
self._write_subtomo_to_scilog(subtomo_number)
|
||||
|
||||
N, achievable_step, total_projections = self._tomo_type1_actual_grid()
|
||||
|
||||
if start_angle is None:
|
||||
# Phase offset must be a fraction of the ACHIEVABLE step (after
|
||||
# int() truncation), not the raw configured tomo_angle_stepsize --
|
||||
# they only coincide when 360/tomo_angle_stepsize happens to
|
||||
# already be a whole number. Using the raw value here breaks the
|
||||
# interlacing scheme: sub-tomograms combined in pairs/quads/all-8
|
||||
# land on unevenly-spaced angles instead of the intended
|
||||
# progressively finer, evenly-spaced grid at every combination
|
||||
# level. Mirrors Flomni._subtomo_angle_plan() exactly.
|
||||
offsets = {1: 0, 2: 4, 3: 2, 4: 6, 5: 1, 6: 5, 7: 3, 8: 7}
|
||||
start_angle = self.tomo_angle_stepsize / 8.0 * offsets[subtomo_number]
|
||||
start_angle = achievable_step / 8.0 * offsets[subtomo_number]
|
||||
|
||||
angle_end = start_angle + 360
|
||||
angles = np.linspace(
|
||||
start_angle,
|
||||
angle_end,
|
||||
num=int(360 / self.tomo_angle_stepsize) + 1,
|
||||
endpoint=True,
|
||||
)
|
||||
# endpoint=False: a full 360-degree sweep must NOT re-measure both
|
||||
# start_angle and start_angle+360 -- they're the same physical angle,
|
||||
# unlike a [0, 180) span's two distinct endpoints.
|
||||
angles = np.linspace(start_angle, angle_end, num=N, endpoint=False)
|
||||
|
||||
if not (subtomo_number % 2):
|
||||
angles = np.flip(angles)
|
||||
@@ -752,12 +867,11 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
self.progress["tomo_type"] = "Equally spaced sub-tomograms"
|
||||
self.progress["subtomo"] = subtomo_number
|
||||
self.progress["subtomo_projection"] = np.where(angles == angle)[0][0]
|
||||
self.progress["subtomo_total_projections"] = 360 / self.tomo_angle_stepsize
|
||||
self.progress["projection"] = (
|
||||
(subtomo_number - 1) * self.progress["subtomo_total_projections"]
|
||||
+ self.progress["subtomo_projection"]
|
||||
)
|
||||
self.progress["total_projections"] = 360 / self.tomo_angle_stepsize * 8
|
||||
self.progress["subtomo_total_projections"] = N
|
||||
self.progress["projection"] = (subtomo_number - 1) * N + self.progress[
|
||||
"subtomo_projection"
|
||||
]
|
||||
self.progress["total_projections"] = total_projections
|
||||
self.progress["angle"] = angle
|
||||
self._tomo_scan_at_angle(angle, subtomo_number)
|
||||
|
||||
@@ -879,6 +993,8 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
start_angle (float, optional): Override starting angle of the first sub-tomogram.
|
||||
projection_number (int, optional): For tomo_types 2 and 3, resume from this index.
|
||||
"""
|
||||
self.lamnigui_show_progress()
|
||||
|
||||
bec = builtins.__dict__.get("bec")
|
||||
scans = builtins.__dict__.get("scans")
|
||||
self._current_special_angles = self.special_angles.copy()
|
||||
@@ -888,15 +1004,23 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
or (self.tomo_type == 2 and projection_number is None)
|
||||
or (self.tomo_type == 3 and projection_number is None)
|
||||
):
|
||||
self.tomo_id = self.add_sample_database(
|
||||
self.sample_name,
|
||||
str(datetime.date.today()),
|
||||
bec.active_account.decode(),
|
||||
bec.queue.next_scan_number,
|
||||
"lamni",
|
||||
"test additional info",
|
||||
"BEC",
|
||||
)
|
||||
# bec.active_account is already a plain str, not bytes -- .decode()
|
||||
# crashes with AttributeError. Also guard against no active
|
||||
# e-account (empty string, e.g. a dev/sim session not logged into
|
||||
# a real account) rather than trying to register a sample under
|
||||
# one -- mirrors Flomni.tomo_scan()'s equivalent check exactly.
|
||||
if bec.active_account != "":
|
||||
self.tomo_id = self.add_sample_database(
|
||||
self.sample_name,
|
||||
str(datetime.date.today()),
|
||||
bec.active_account,
|
||||
bec.queue.next_scan_number,
|
||||
"lamni",
|
||||
"test additional info",
|
||||
"BEC",
|
||||
)
|
||||
else:
|
||||
self.tomo_id = 0
|
||||
self.write_pdf_report()
|
||||
self.progress["tomo_start_time"] = datetime.datetime.now().isoformat()
|
||||
# reset stale estimates from any previous scan, otherwise the GUI
|
||||
@@ -907,92 +1031,102 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
self.progress["accumulated_idle_time"] = 0.0
|
||||
self.progress["heartbeat"] = None
|
||||
|
||||
with scans.dataset_id_on_hold:
|
||||
if self.tomo_type == 1:
|
||||
self.progress["tomo_type"] = "Equally spaced sub-tomograms"
|
||||
for ii in range(subtomo_start, 9):
|
||||
self.sub_tomo_scan(ii, start_angle=start_angle)
|
||||
start_angle = None
|
||||
try:
|
||||
with scans.dataset_id_on_hold:
|
||||
if self.tomo_type == 1:
|
||||
self.progress["tomo_type"] = "Equally spaced sub-tomograms"
|
||||
for ii in range(subtomo_start, 9):
|
||||
self.sub_tomo_scan(ii, start_angle=start_angle)
|
||||
start_angle = None
|
||||
|
||||
elif self.tomo_type == 2:
|
||||
self.progress["tomo_type"] = "Golden ratio tomography"
|
||||
previous_subtomo_number = -1
|
||||
ii = 0 if projection_number is None else projection_number
|
||||
while True:
|
||||
angle, subtomo_number = self._golden(
|
||||
ii, self.golden_ratio_bunch_size, maxangle=360, reverse=True
|
||||
)
|
||||
if previous_subtomo_number != subtomo_number:
|
||||
self._write_subtomo_to_scilog(subtomo_number)
|
||||
if (
|
||||
subtomo_number % 2 == 1
|
||||
and ii > 10
|
||||
and self.golden_projections_at_0_deg_for_damage_estimation == 1
|
||||
):
|
||||
self._tomo_scan_at_angle(0, subtomo_number)
|
||||
previous_subtomo_number = subtomo_number
|
||||
|
||||
self.progress["subtomo"] = subtomo_number
|
||||
self.progress["projection"] = ii
|
||||
self.progress["angle"] = angle
|
||||
self.progress["subtomo_total_projections"] = self.golden_ratio_bunch_size
|
||||
self.progress["subtomo_projection"] = (
|
||||
ii - (subtomo_number - 1) * self.golden_ratio_bunch_size
|
||||
)
|
||||
self.progress["total_projections"] = self.golden_max_number_of_projections
|
||||
|
||||
self._tomo_scan_at_angle(angle, subtomo_number)
|
||||
ii += 1
|
||||
if (
|
||||
self.golden_max_number_of_projections > 0
|
||||
and ii > self.golden_max_number_of_projections
|
||||
):
|
||||
print(
|
||||
f"Golden ratio tomography stopped after"
|
||||
f" {self.golden_max_number_of_projections} projections."
|
||||
elif self.tomo_type == 2:
|
||||
self.progress["tomo_type"] = "Golden ratio tomography"
|
||||
previous_subtomo_number = -1
|
||||
ii = 0 if projection_number is None else projection_number
|
||||
while True:
|
||||
angle, subtomo_number = self._golden(
|
||||
ii, self.golden_ratio_bunch_size, maxangle=360, reverse=True
|
||||
)
|
||||
break
|
||||
if previous_subtomo_number != subtomo_number:
|
||||
self._write_subtomo_to_scilog(subtomo_number)
|
||||
if (
|
||||
subtomo_number % 2 == 1
|
||||
and ii > 10
|
||||
and self.golden_projections_at_0_deg_for_damage_estimation == 1
|
||||
):
|
||||
self._tomo_scan_at_angle(0, subtomo_number)
|
||||
previous_subtomo_number = subtomo_number
|
||||
|
||||
elif self.tomo_type == 3:
|
||||
self.progress["tomo_type"] = "Equally spaced, golden ratio starting angles"
|
||||
previous_subtomo_number = -1
|
||||
ii = 0 if projection_number is None else projection_number
|
||||
while True:
|
||||
angle, subtomo_number = self._golden_equally_spaced(
|
||||
ii, int(360 / self.tomo_angle_stepsize), maxangle=360, reverse=True
|
||||
)
|
||||
if previous_subtomo_number != subtomo_number:
|
||||
self._write_subtomo_to_scilog(subtomo_number)
|
||||
if (
|
||||
subtomo_number % 2 == 1
|
||||
and ii > 10
|
||||
and self.golden_projections_at_0_deg_for_damage_estimation == 1
|
||||
):
|
||||
self._tomo_scan_at_angle(0, subtomo_number)
|
||||
previous_subtomo_number = subtomo_number
|
||||
|
||||
self.progress["subtomo"] = subtomo_number
|
||||
self.progress["projection"] = ii
|
||||
self.progress["angle"] = angle
|
||||
self.progress["subtomo_total_projections"] = 360 / self.tomo_angle_stepsize
|
||||
self.progress["subtomo_projection"] = (
|
||||
ii - (subtomo_number - 1) * self.progress["subtomo_total_projections"]
|
||||
)
|
||||
self.progress["total_projections"] = self.golden_max_number_of_projections
|
||||
|
||||
self._tomo_scan_at_angle(angle, subtomo_number)
|
||||
ii += 1
|
||||
if (
|
||||
self.golden_max_number_of_projections > 0
|
||||
and ii > self.golden_max_number_of_projections
|
||||
):
|
||||
print(
|
||||
f"Golden ratio tomography stopped after"
|
||||
f" {self.golden_max_number_of_projections} projections."
|
||||
self.progress["subtomo"] = subtomo_number
|
||||
self.progress["projection"] = ii
|
||||
self.progress["angle"] = angle
|
||||
self.progress["subtomo_total_projections"] = self.golden_ratio_bunch_size
|
||||
self.progress["subtomo_projection"] = (
|
||||
ii - (subtomo_number - 1) * self.golden_ratio_bunch_size
|
||||
)
|
||||
break
|
||||
else:
|
||||
raise ValueError(f"Unknown tomo_type: {self.tomo_type}.")
|
||||
self.progress["total_projections"] = self.golden_max_number_of_projections
|
||||
|
||||
self._tomo_scan_at_angle(angle, subtomo_number)
|
||||
ii += 1
|
||||
if (
|
||||
self.golden_max_number_of_projections > 0
|
||||
and ii > self.golden_max_number_of_projections
|
||||
):
|
||||
print(
|
||||
f"Golden ratio tomography stopped after"
|
||||
f" {self.golden_max_number_of_projections} projections."
|
||||
)
|
||||
break
|
||||
|
||||
elif self.tomo_type == 3:
|
||||
self.progress["tomo_type"] = "Equally spaced, golden ratio starting angles"
|
||||
previous_subtomo_number = -1
|
||||
ii = 0 if projection_number is None else projection_number
|
||||
while True:
|
||||
angle, subtomo_number = self._golden_equally_spaced(
|
||||
ii, int(360 / self.tomo_angle_stepsize), maxangle=360, reverse=True
|
||||
)
|
||||
if previous_subtomo_number != subtomo_number:
|
||||
self._write_subtomo_to_scilog(subtomo_number)
|
||||
if (
|
||||
subtomo_number % 2 == 1
|
||||
and ii > 10
|
||||
and self.golden_projections_at_0_deg_for_damage_estimation == 1
|
||||
):
|
||||
self._tomo_scan_at_angle(0, subtomo_number)
|
||||
previous_subtomo_number = subtomo_number
|
||||
|
||||
self.progress["subtomo"] = subtomo_number
|
||||
self.progress["projection"] = ii
|
||||
self.progress["angle"] = angle
|
||||
self.progress["subtomo_total_projections"] = 360 / self.tomo_angle_stepsize
|
||||
self.progress["subtomo_projection"] = (
|
||||
ii - (subtomo_number - 1) * self.progress["subtomo_total_projections"]
|
||||
)
|
||||
self.progress["total_projections"] = self.golden_max_number_of_projections
|
||||
|
||||
self._tomo_scan_at_angle(angle, subtomo_number)
|
||||
ii += 1
|
||||
if (
|
||||
self.golden_max_number_of_projections > 0
|
||||
and ii > self.golden_max_number_of_projections
|
||||
):
|
||||
print(
|
||||
f"Golden ratio tomography stopped after"
|
||||
f" {self.golden_max_number_of_projections} projections."
|
||||
)
|
||||
break
|
||||
else:
|
||||
raise ValueError(f"Unknown tomo_type: {self.tomo_type}.")
|
||||
finally:
|
||||
# Cleared on every exit path (normal completion, exception, or
|
||||
# interrupt/abort), not just at the next scan's start above --
|
||||
# mirrors Flomni.tomo_scan()'s identical fix: without this, the
|
||||
# GUI's busy-detectors (TomoParamsWidget's banner) only notice a
|
||||
# finished scan once the heartbeat goes stale (120s), showing
|
||||
# "beamline busy" for up to two minutes after a tomogram already
|
||||
# completed successfully.
|
||||
self.progress["heartbeat"] = None
|
||||
|
||||
self.progress["projection"] = self.progress["total_projections"]
|
||||
self.progress["subtomo_projection"] = self.progress["subtomo_total_projections"]
|
||||
@@ -1002,6 +1136,43 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
f" {self._format_duration(self.progress.get('accumulated_idle_time', 0.0))}"
|
||||
)
|
||||
|
||||
def tomo_scan_resume(self) -> None:
|
||||
"""Resume a tomo_scan() that crashed or was interrupted, picking up
|
||||
automatically from wherever ``progress`` last reported -- no need to
|
||||
read the last subtomo/angle/projection off the progress printout by
|
||||
hand and pass it to tomo_scan() yourself.
|
||||
|
||||
Re-attempts the exact angle/projection that was in progress at the
|
||||
moment of interruption, rather than skipping ahead to the next one:
|
||||
from here we can't be sure whether that last point was actually
|
||||
acquired before the crash, and re-acquiring one extra projection is
|
||||
harmless.
|
||||
|
||||
Works standalone for a normal tomo_scan(), independent of the tomo
|
||||
queue. tomo_queue_execute() (see TomoQueueMixin) also calls this
|
||||
internally when resuming a job that previously failed partway
|
||||
through, so a failed queue job picks up mid-scan rather than
|
||||
restarting from subtomo 1 / angle 0.
|
||||
"""
|
||||
if self.progress.get("tomo_start_time") is None:
|
||||
print("No tomo scan in progress to resume -- nothing to do.")
|
||||
return
|
||||
|
||||
if self.tomo_type == 1:
|
||||
subtomo_start = self.progress["subtomo"]
|
||||
start_angle = self.progress["angle"]
|
||||
if subtomo_start < 1:
|
||||
print("No tomo scan in progress to resume -- nothing to do.")
|
||||
return
|
||||
print(f"Resuming tomo scan at subtomo {subtomo_start}, angle {start_angle:.3f} deg.")
|
||||
self.tomo_scan(subtomo_start=subtomo_start, start_angle=start_angle)
|
||||
elif self.tomo_type in (2, 3):
|
||||
projection_number = self.progress["projection"]
|
||||
print(f"Resuming tomo scan at projection {projection_number}.")
|
||||
self.tomo_scan(projection_number=projection_number)
|
||||
else:
|
||||
raise ValueError(f"Unknown tomo_type: {self.tomo_type}.")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Parameter display and interactive update
|
||||
# ------------------------------------------------------------------
|
||||
@@ -1029,8 +1200,13 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
print("")
|
||||
if self.tomo_type == 1:
|
||||
print("\x1b[1mTomo type 1:\x1b[0m 8 equally spaced sub-tomograms (360 deg)")
|
||||
print(f"Angular step within sub-tomogram: {self.tomo_angle_stepsize} degrees")
|
||||
print(f"Resulting in number of projections: {360/self.tomo_angle_stepsize*8}")
|
||||
# _tomo_type1_actual_grid() is the same computation sub_tomo_scan()
|
||||
# actually runs -- reading self.tomo_angle_stepsize/naive division
|
||||
# directly here previously showed a number that didn't match what
|
||||
# a real scan acquired (see the helper's docstring).
|
||||
_, achievable_step, total_projections = self._tomo_type1_actual_grid()
|
||||
print(f"Angular step within sub-tomogram: {achievable_step} degrees")
|
||||
print(f"Resulting in number of projections: {total_projections}")
|
||||
elif self.tomo_type == 2:
|
||||
print("\x1b[1mTomo type 2:\x1b[0m Golden ratio tomography")
|
||||
print(f"Sorted in bunches of: {self.golden_ratio_bunch_size}")
|
||||
@@ -1082,11 +1258,23 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
self.tomo_type = self._get_val("Tomography type", self.tomo_type, int)
|
||||
|
||||
if self.tomo_type == 1:
|
||||
tomo_numberofprojections = self._get_val(
|
||||
"Number of projections", 360 / self.tomo_angle_stepsize * 8, int
|
||||
)
|
||||
_, _, current_total = self._tomo_type1_actual_grid()
|
||||
tomo_numberofprojections = self._get_val("Number of projections", current_total, int)
|
||||
self.tomo_angle_stepsize = 360 / tomo_numberofprojections * 8
|
||||
print(f"Angular step in a subtomogram: {self.tomo_angle_stepsize}")
|
||||
# Report what was ACTUALLY achieved, via the same helper
|
||||
# sub_tomo_scan() itself uses -- not the raw value just typed in,
|
||||
# which can silently disagree with what the scan will actually
|
||||
# run (mirrors Flomni.tomo_parameters()'s equivalent note).
|
||||
_, achievable_step, actual_total = self._tomo_type1_actual_grid()
|
||||
if actual_total != tomo_numberofprojections:
|
||||
print(
|
||||
f"Note: {tomo_numberofprojections} projections does not divide evenly "
|
||||
"into 8 equally spaced sub-tomograms over 360 degrees; adjusted to the "
|
||||
f"nearest achievable total of {actual_total} projections to keep the "
|
||||
"angular grid uniform."
|
||||
)
|
||||
print(f"Angular step in a subtomogram: {achievable_step}")
|
||||
print(f"Actual number of projections: {actual_total}")
|
||||
|
||||
elif self.tomo_type == 2:
|
||||
while True:
|
||||
@@ -1155,16 +1343,25 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
piezo_range = f"{self.lamni_piezo_range_x:.2f}/{self.lamni_piezo_range_y:.2f}"
|
||||
stitching = f"{self.lamni_stitch_x:.2f}/{self.lamni_stitch_y:.2f}"
|
||||
dataset_id = str(self.client.queue.next_dataset_number)
|
||||
_, _, report_total_projections = self._tomo_type1_actual_grid()
|
||||
# The energy readback device is "ccm_energy", not "mokev" -- and may
|
||||
# not be configured/available in every session (e.g. some simulated
|
||||
# configs). Read it defensively so a missing/misbehaving device
|
||||
# doesn't crash the whole report instead of just omitting one line.
|
||||
try:
|
||||
energy_str = f"{dev.ccm_energy.read(cached=True)[dev.ccm_energy.name]['value']:.4f}"
|
||||
except Exception:
|
||||
energy_str = "N/A"
|
||||
content = [
|
||||
f"{'Sample Name:':<{padding}}{self.sample_name:>{padding}}\n",
|
||||
f"{'Measurement ID:':<{padding}}{str(self.tomo_id):>{padding}}\n",
|
||||
f"{'Dataset ID:':<{padding}}{dataset_id:>{padding}}\n",
|
||||
f"{'Sample Info:':<{padding}}{'Sample Info':>{padding}}\n",
|
||||
f"{'e-account:':<{padding}}{str(self.client.username):>{padding}}\n",
|
||||
f"{'Number of projections:':<{padding}}{int(360 / self.tomo_angle_stepsize * 8):>{padding}}\n",
|
||||
f"{'Number of projections:':<{padding}}{report_total_projections:>{padding}}\n",
|
||||
f"{'First scan number:':<{padding}}{self.client.queue.next_scan_number:>{padding}}\n",
|
||||
f"{'Last scan number approx.:':<{padding}}{self.client.queue.next_scan_number + int(360 / self.tomo_angle_stepsize * 8) + 10:>{padding}}\n",
|
||||
f"{'Current photon energy:':<{padding}}{dev.mokev.read(cached=True)['value']:>{padding}.4f}\n",
|
||||
f"{'Last scan number approx.:':<{padding}}{self.client.queue.next_scan_number + report_total_projections + 10:>{padding}}\n",
|
||||
f"{'Current photon energy:':<{padding}}{energy_str:>{padding}}\n",
|
||||
f"{'Exposure time:':<{padding}}{self.tomo_countingtime:>{padding}.2f}\n",
|
||||
f"{'Fermat spiral step size:':<{padding}}{self.tomo_shellstep:>{padding}.2f}\n",
|
||||
f"{'Piezo range (FOV sample plane):':<{padding}}{piezo_range:>{padding}}\n",
|
||||
@@ -1182,12 +1379,18 @@ class LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools):
|
||||
subprocess.run(
|
||||
"xterm /work/sls/spec/local/XOMNY/bin/upload/upload_last_pon.sh &", shell=True
|
||||
)
|
||||
msg = bec.logbook.LogbookMessage()
|
||||
logo_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "LamNI_logo.png")
|
||||
msg.add_file(logo_path).add_text("".join(content).replace("\n", "</p><p>")).add_tag(
|
||||
["BEC", "tomo_parameters", f"dataset_id_{dataset_id}", "LamNI", self.sample_name]
|
||||
)
|
||||
self.client.logbook.send_logbook_message(msg)
|
||||
# Same tolerance as write_to_scilog(): a session without scilog/logbook
|
||||
# configured (e.g. a dev/sim session) must not crash report generation
|
||||
# over the logbook upload -- the PDF itself is already written above.
|
||||
try:
|
||||
msg = bec.logbook.LogbookMessage()
|
||||
logo_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "LamNI_logo.png")
|
||||
msg.add_file(logo_path).add_text("".join(content).replace("\n", "</p><p>")).add_tag(
|
||||
["BEC", "tomo_parameters", f"dataset_id_{dataset_id}", "LamNI", self.sample_name]
|
||||
)
|
||||
self.client.logbook.send_logbook_message(msg)
|
||||
except Exception:
|
||||
logger.warning("Failed to write PDF report to scilog.")
|
||||
|
||||
def get_calibration_of_capstops_left_and_right(self):
|
||||
import time
|
||||
|
||||
@@ -0,0 +1,758 @@
|
||||
"""
|
||||
Shared tomo-queue / at-each-angle-hook backend for cSAXS ptychographic
|
||||
tomography setups.
|
||||
|
||||
Extracted from flomni.py, where this logic (job bookkeeping, resume-on-crash,
|
||||
named per-projection hooks, command-job dispatch) was originally written
|
||||
entirely generic over "which parameters/actions this setup has" -- only the
|
||||
*names* of the snapshotted params and the allow-listed command-job actions
|
||||
are setup-specific. Keeping one copy here instead of one per setup avoids the
|
||||
same param-name-mirror drift problem already flagged (see tomo_params.py's
|
||||
QUEUE_PARAM_NAMES) for a second, backend-side pair of mirrors.
|
||||
|
||||
Consumers (e.g. ``Flomni``, ``LamNI``) must:
|
||||
- inherit from ``TomoQueueMixin``
|
||||
- define class attributes ``_TOMO_SCAN_PARAM_NAMES`` (tuple of property
|
||||
names a ``tomo_queue_add()`` job snapshots), ``_TOMO_QUEUE_ACTIONS``
|
||||
(command-job action registry) and ``_TOMO_QUEUE_MOVE_DEVICES`` (move
|
||||
allow-list)
|
||||
- call ``self._init_tomo_queue()`` once from ``__init__``, after
|
||||
``self.client`` is set
|
||||
- optionally override ``_describe_tomo_job_line()`` to reference their
|
||||
own param names in ``tomo_queue_show()``'s one-line-per-job summary
|
||||
- implement their own ``tomo_scan()``/``tomo_scan_resume()`` (dispatched
|
||||
generically by ``tomo_queue_execute()``) and their own ``_at_each_angle()``
|
||||
(which should consult ``self.at_each_angle_hook``/``self._at_each_angle_hooks``
|
||||
the same way, but is setup-specific enough -- e.g. flomni's legacy
|
||||
``flomni_at_each_angle`` builtins hook, single-point acquisition -- that
|
||||
it stays out of this mixin)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import builtins
|
||||
import datetime
|
||||
import inspect
|
||||
import json
|
||||
import uuid
|
||||
|
||||
from typeguard import check_type
|
||||
|
||||
if builtins.__dict__.get("bec") is not None:
|
||||
dev = builtins.__dict__.get("dev")
|
||||
umv = builtins.__dict__.get("umv")
|
||||
|
||||
|
||||
class TomoQueueError(Exception):
|
||||
"""Raised for tomo-queue/command-job errors not specific to one setup."""
|
||||
|
||||
|
||||
class _TomoQueueProxy:
|
||||
"""List-like proxy that persists a queue of tomo parameter snapshots as a
|
||||
BEC global variable, mirroring the pattern used by ``_ProgressProxy``.
|
||||
|
||||
Each entry is a plain dict: ``{"label": ..., "params": {...}, "status":
|
||||
..., "added_at": ...}`` (or, for a command job, ``"steps"`` instead of
|
||||
``"params"``), where ``params`` holds a snapshot of all global-var-backed
|
||||
tomo scan parameters (see the owning class's ``_TOMO_SCAN_PARAM_NAMES``).
|
||||
Stored as a single list under ``tomo_queue`` so the queue is visible from
|
||||
any BEC client session via ``client.get_global_var("tomo_queue")`` and
|
||||
survives a kernel restart.
|
||||
"""
|
||||
|
||||
_GLOBAL_VAR_KEY = "tomo_queue"
|
||||
|
||||
def __init__(self, client):
|
||||
self._client = client
|
||||
|
||||
def _load(self) -> list:
|
||||
val = self._client.get_global_var(self._GLOBAL_VAR_KEY)
|
||||
if val is None:
|
||||
return []
|
||||
return val
|
||||
|
||||
def _save(self, jobs: list) -> None:
|
||||
self._client.set_global_var(self._GLOBAL_VAR_KEY, jobs)
|
||||
|
||||
def as_list(self) -> list:
|
||||
"""Return a plain copy of the current queue."""
|
||||
return self._load()
|
||||
|
||||
def append(self, job: dict) -> int:
|
||||
jobs = self._load()
|
||||
jobs.append(job)
|
||||
self._save(jobs)
|
||||
return len(jobs) - 1
|
||||
|
||||
def pop(self, index: int) -> dict:
|
||||
jobs = self._load()
|
||||
job = jobs.pop(index)
|
||||
self._save(jobs)
|
||||
return job
|
||||
|
||||
def update(self, index: int, **kwargs) -> None:
|
||||
jobs = self._load()
|
||||
jobs[index].update(kwargs)
|
||||
self._save(jobs)
|
||||
|
||||
def update_by_id(self, job_id: str, **kwargs) -> bool:
|
||||
"""Update the job whose ``id`` matches ``job_id``.
|
||||
|
||||
Returns True if a matching job was found and updated, False otherwise
|
||||
(e.g. the job was deleted concurrently). Addressing a job by its stable
|
||||
id rather than its list index makes status writes safe against queue
|
||||
edits made while the queue runs: a reorder, or a delete of some *other*
|
||||
job, can no longer misdirect a status write to the wrong entry.
|
||||
"""
|
||||
jobs = self._load()
|
||||
for job in jobs:
|
||||
if job.get("id") == job_id:
|
||||
job.update(kwargs)
|
||||
self._save(jobs)
|
||||
return True
|
||||
return False
|
||||
|
||||
def move(self, index: int, new_index: int) -> None:
|
||||
"""Move the job at ``index`` to ``new_index``, shifting the rest.
|
||||
|
||||
Pure list mechanics -- no status checks. Callers (e.g.
|
||||
``tomo_queue_move``) are responsible for deciding whether a given
|
||||
move is allowed (running-job floor, movable statuses, ...).
|
||||
"""
|
||||
jobs = self._load()
|
||||
job = jobs.pop(index)
|
||||
jobs.insert(new_index, job)
|
||||
self._save(jobs)
|
||||
|
||||
def ensure_ids(self) -> list:
|
||||
"""Assign a stable id to any job that lacks one and persist the result.
|
||||
|
||||
Back-compat shim for queues written before jobs carried an ``id``
|
||||
field. Called once at the top of ``tomo_queue_execute()`` so the
|
||||
pick-next executor can address every job by id. Returns the
|
||||
(possibly healed) job list.
|
||||
"""
|
||||
jobs = self._load()
|
||||
changed = False
|
||||
for job in jobs:
|
||||
if not job.get("id"):
|
||||
job["id"] = uuid.uuid4().hex
|
||||
changed = True
|
||||
if changed:
|
||||
self._save(jobs)
|
||||
return jobs
|
||||
|
||||
def clear(self) -> None:
|
||||
self._save([])
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._load())
|
||||
|
||||
def __getitem__(self, index):
|
||||
return self._load()[index]
|
||||
|
||||
|
||||
class _GlobalVarParam:
|
||||
"""Descriptor for a parameter backed by a BEC global variable.
|
||||
|
||||
Consolidates the boilerplate getter/setter pairs that read from and write
|
||||
to the BEC global-var store. The value persists across BEC client restarts
|
||||
(instance attributes do not), and is readable from other client sessions
|
||||
via ``client.get_global_var(<name>)``.
|
||||
|
||||
The global-var key is taken from the attribute name the descriptor is
|
||||
assigned to (see ``__set_name__``), so it always matches the property
|
||||
name.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
default
|
||||
Value returned by the getter when the global var is unset (``None``).
|
||||
type_
|
||||
Optional expected type. When given, the setter validates ``val`` with
|
||||
``typeguard.check_type`` before storing.
|
||||
"""
|
||||
|
||||
def __init__(self, default, type_=None):
|
||||
self._default = default
|
||||
self._type = type_
|
||||
|
||||
def __set_name__(self, owner, name):
|
||||
self._key = name
|
||||
|
||||
def __get__(self, obj, objtype=None):
|
||||
if obj is None:
|
||||
return self
|
||||
val = obj.client.get_global_var(self._key)
|
||||
if val is None:
|
||||
return self._default
|
||||
return val
|
||||
|
||||
def __set__(self, obj, val):
|
||||
if self._type is not None:
|
||||
check_type(val, self._type)
|
||||
obj.client.set_global_var(self._key, val)
|
||||
|
||||
|
||||
class TomoQueueMixin:
|
||||
"""Generic tomo-queue / at-each-angle-hook backend, see module docstring."""
|
||||
|
||||
# Subclasses must override all three of these.
|
||||
_TOMO_SCAN_PARAM_NAMES: tuple = ()
|
||||
_TOMO_QUEUE_ACTIONS: dict = {}
|
||||
_TOMO_QUEUE_MOVE_DEVICES: dict = {}
|
||||
|
||||
# Name of a registered at_each_angle hook (see register_at_each_angle_hook()),
|
||||
# or None for the default per-projection behaviour.
|
||||
at_each_angle_hook = _GlobalVarParam(None)
|
||||
|
||||
def _init_tomo_queue(self) -> None:
|
||||
"""Call once from ``__init__``, after ``self.client`` is set.
|
||||
|
||||
Deliberately does not reset the tomo_queue global var itself (jobs
|
||||
must survive a kernel restart, exactly like tomo_progress) -- only
|
||||
the session-only hook registry and the discoverability global vars
|
||||
are (re)initialized here.
|
||||
"""
|
||||
# Session-only (not global-var-backed): the hooks themselves are
|
||||
# Python callables, which can't be persisted the way params/queue
|
||||
# jobs are. Only the *name* selecting one is ever snapshotted (see
|
||||
# at_each_angle_hook / register_at_each_angle_hook()).
|
||||
self._at_each_angle_hooks: dict = {}
|
||||
self._tomo_queue_proxy = _TomoQueueProxy(self.client)
|
||||
# Published so a future GUI never needs a hardcoded mirror of the
|
||||
# action registry -- see _describe_tomo_queue_actions().
|
||||
self.client.set_global_var("tomo_queue_actions", self._describe_tomo_queue_actions())
|
||||
# Reset to empty on every init -- hooks are session-only (see
|
||||
# self._at_each_angle_hooks above), so a stale list published by a
|
||||
# since-ended session must not linger and look registered.
|
||||
self._publish_at_each_angle_hooks()
|
||||
|
||||
# ── at_each_angle hook registry ─────────────────────────────────────────
|
||||
|
||||
def register_at_each_angle_hook(self, name: str, func) -> None:
|
||||
"""Register a per-projection hook function under ``name``, so a
|
||||
tomo-queue job can select it by name via the ``at_each_angle_hook``
|
||||
parameter and have it run automatically.
|
||||
|
||||
``func`` is called as ``func(self, angle)`` at every projection angle.
|
||||
|
||||
Hooks are **session-only**: they live in this instance's memory, not
|
||||
in a global var, so they do NOT survive a kernel restart. A queued
|
||||
job that references a hook by name raises a clear error at execution
|
||||
time if the name isn't registered in the session running the queue --
|
||||
re-run this registration call, then resume ``tomo_queue_execute()``.
|
||||
|
||||
Example -- record a projection, insert a polarizer, record again::
|
||||
|
||||
def polarizer_modulation(setup, angle):
|
||||
setup.tomo_scan_projection(angle, _internal=True)
|
||||
umv(dev.polarizer, "in")
|
||||
setup.tomo_scan_projection(angle, _internal=True)
|
||||
umv(dev.polarizer, "out")
|
||||
|
||||
setup.register_at_each_angle_hook("polarizer_mod", polarizer_modulation)
|
||||
setup.at_each_angle_hook = "polarizer_mod"
|
||||
setup.tomo_queue_add(label="polarizer run")
|
||||
setup.at_each_angle_hook = None # back to default for the next job
|
||||
"""
|
||||
if not callable(func):
|
||||
raise TypeError(f"register_at_each_angle_hook: {func!r} is not callable.")
|
||||
self._at_each_angle_hooks[name] = func
|
||||
self._publish_at_each_angle_hooks()
|
||||
print(f"Registered at_each_angle hook '{name}'.")
|
||||
|
||||
def unregister_at_each_angle_hook(self, name: str) -> None:
|
||||
"""Remove a previously registered at_each_angle hook by name."""
|
||||
if self._at_each_angle_hooks.pop(name, None) is None:
|
||||
print(f"No at_each_angle hook named '{name}' is registered.")
|
||||
else:
|
||||
self._publish_at_each_angle_hooks()
|
||||
print(f"Unregistered at_each_angle hook '{name}'.")
|
||||
|
||||
def _publish_at_each_angle_hooks(self) -> None:
|
||||
"""Publish the names of all currently registered at_each_angle
|
||||
hooks as the ``tomo_at_each_angle_hooks`` global var (a plain list
|
||||
of strings) -- so a GUI (or any other client) can build a dropdown
|
||||
from it directly. Unlike ``tomo_queue_actions`` this list changes at
|
||||
runtime, whenever register_/unregister_at_each_angle_hook() is called.
|
||||
"""
|
||||
self.client.set_global_var("tomo_at_each_angle_hooks", sorted(self._at_each_angle_hooks))
|
||||
|
||||
def list_at_each_angle_hooks(self) -> list:
|
||||
"""Print and return the names of all at_each_angle hooks registered
|
||||
in this session. The active one (``self.at_each_angle_hook``, if
|
||||
any) is marked."""
|
||||
names = sorted(self._at_each_angle_hooks)
|
||||
if not names:
|
||||
print("No at_each_angle hooks registered.")
|
||||
else:
|
||||
for name in names:
|
||||
marker = " (active)" if name == self.at_each_angle_hook else ""
|
||||
print(f" {name}{marker}")
|
||||
return names
|
||||
|
||||
def _describe_active_hook(self) -> str | None:
|
||||
"""Human-readable description of ``self.at_each_angle_hook`` for
|
||||
display in tomo_parameters()/scilog/the PDF report: the name, or
|
||||
the name flagged as unregistered if it doesn't (or no longer)
|
||||
resolve in this session. Returns None if no hook is set at all.
|
||||
"""
|
||||
hook = self.at_each_angle_hook
|
||||
if not hook:
|
||||
return None
|
||||
if hook in self._at_each_angle_hooks:
|
||||
return hook
|
||||
return f"{hook} (NOT registered in this session)"
|
||||
|
||||
def _active_hook_source(self) -> str | None:
|
||||
"""Source code of the currently active at_each_angle hook, for
|
||||
inclusion in the scilog/PDF record. Returns None if there's no
|
||||
active hook, it isn't registered in this session, or its source
|
||||
can't be retrieved (e.g. it wasn't defined via a normal `def`/
|
||||
interactive cell).
|
||||
"""
|
||||
hook_func = self._at_each_angle_hooks.get(self.at_each_angle_hook)
|
||||
if hook_func is None:
|
||||
return None
|
||||
try:
|
||||
return inspect.getsource(hook_func)
|
||||
except (OSError, TypeError):
|
||||
return None
|
||||
|
||||
# ── command-job action registry / dispatch ──────────────────────────────
|
||||
|
||||
def _validate_action_kwargs(self, action_name: str, kwargs: dict) -> None:
|
||||
"""Add-time schema validation for a command-job step's kwargs.
|
||||
|
||||
Ergonomic, not the safety boundary: catches a typo or an out-of-range
|
||||
value now instead of at 2am. Each action method re-validates its own
|
||||
arguments against live hardware at execution time -- that's the
|
||||
actual safety boundary, since the schema and the hardware can
|
||||
disagree (limits changed; a value arrived via the CLI, bypassing
|
||||
this).
|
||||
"""
|
||||
spec = self._TOMO_QUEUE_ACTIONS.get(action_name)
|
||||
if spec is None:
|
||||
raise ValueError(f"Unknown queue action '{action_name}'.")
|
||||
schema = spec["params"]
|
||||
for key in kwargs:
|
||||
if key not in schema:
|
||||
raise ValueError(f"Action '{action_name}' has no parameter '{key}'.")
|
||||
for name, field in schema.items():
|
||||
if name not in kwargs:
|
||||
continue
|
||||
value = kwargs[name]
|
||||
field_type = field.get("type")
|
||||
if field_type == "float":
|
||||
if not isinstance(value, (int, float)) or isinstance(value, bool):
|
||||
raise ValueError(f"{action_name}.{name} must be a number, got {value!r}.")
|
||||
lo, hi = field.get("min"), field.get("max")
|
||||
if lo is not None and value < lo:
|
||||
raise ValueError(f"{action_name}.{name}={value} is below the minimum {lo}.")
|
||||
if hi is not None and value > hi:
|
||||
raise ValueError(f"{action_name}.{name}={value} is above the maximum {hi}.")
|
||||
elif field_type == "device_positions":
|
||||
if not isinstance(value, dict) or not value:
|
||||
raise ValueError(
|
||||
f"{action_name}.{name} must be a non-empty {{device: target}} dict."
|
||||
)
|
||||
for device_name in value:
|
||||
if device_name not in self._TOMO_QUEUE_MOVE_DEVICES:
|
||||
raise ValueError(
|
||||
f"Queue move: device '{device_name}' is not queue-movable."
|
||||
)
|
||||
|
||||
def _queue_action_move(self, positions: dict) -> None:
|
||||
"""positions: {device_name: absolute_target, ...}. Absolute umv only.
|
||||
|
||||
Re-checks the allow-list at execution time (not just add time, see
|
||||
_validate_action_kwargs) -- the actual safety boundary, since a value
|
||||
could have arrived via the CLI, bypassing tomo_queue_add_command's
|
||||
validation.
|
||||
"""
|
||||
args = []
|
||||
for name, target in positions.items():
|
||||
if name not in self._TOMO_QUEUE_MOVE_DEVICES:
|
||||
raise ValueError(f"Queue move: device '{name}' is not queue-movable.")
|
||||
if name not in dev:
|
||||
raise ValueError(f"Queue move: unknown device '{name}'.")
|
||||
args += [dev[name], target]
|
||||
umv(*args)
|
||||
|
||||
def optimize_idgap(self, search_range: float = 0.5) -> None:
|
||||
"""Scan idgap over +/- search_range around its current position and
|
||||
move to the peak of <signal not yet decided>.
|
||||
|
||||
STUB -- not implemented. The peak-detection signal/device is not yet
|
||||
decided (see flomni's AI_docs/TOMO_QUEUE_COMMAND_JOBS_PLAN.md section
|
||||
10). This is the ONLY place that needs to change to make this a real
|
||||
action: the registry entry, resume policy, and (eventually) GUI
|
||||
wiring already treat it as a normal idempotent compound action, so
|
||||
dropping in the real scan-and-move-to-peak logic here (or swapping
|
||||
which device/signal it reads) is a self-contained change.
|
||||
"""
|
||||
print(
|
||||
f"optimize_idgap(search_range={search_range}): not yet implemented -- "
|
||||
"peak-detection signal not yet decided. No-op."
|
||||
)
|
||||
|
||||
def _describe_tomo_queue_actions(self) -> dict:
|
||||
"""Serializable description of the action registry + move allow-list.
|
||||
|
||||
Published as the ``tomo_queue_actions`` global var (see
|
||||
_init_tomo_queue()) so the GUI (or any other client) can build its
|
||||
dropdowns/forms from it directly -- no hardcoded mirror to drift.
|
||||
"""
|
||||
return {
|
||||
"actions": {
|
||||
name: {
|
||||
"idempotent": spec["idempotent"],
|
||||
"help": spec["help"],
|
||||
"params": spec["params"],
|
||||
}
|
||||
for name, spec in self._TOMO_QUEUE_ACTIONS.items()
|
||||
},
|
||||
"move_devices": dict(self._TOMO_QUEUE_MOVE_DEVICES),
|
||||
}
|
||||
|
||||
# ── job list management ──────────────────────────────────────────────
|
||||
|
||||
def tomo_queue_add(self, label: str = None) -> int:
|
||||
"""Snapshot the currently set tomo parameters and append them as a
|
||||
new job to the tomo queue (persisted, survives a kernel restart).
|
||||
|
||||
Args:
|
||||
label: Optional name for the job, shown by tomo_queue_show().
|
||||
Defaults to "job_<n>".
|
||||
|
||||
Returns:
|
||||
The index of the newly added job.
|
||||
"""
|
||||
params = {name: getattr(self, name) for name in self._TOMO_SCAN_PARAM_NAMES}
|
||||
index = len(self._tomo_queue_proxy)
|
||||
job = {
|
||||
"kind": "tomo",
|
||||
"id": uuid.uuid4().hex,
|
||||
"label": label or f"job_{index + 1}",
|
||||
"params": params,
|
||||
"status": "pending",
|
||||
"added_at": datetime.datetime.now().isoformat(),
|
||||
}
|
||||
self._tomo_queue_proxy.append(job)
|
||||
print(f"Added tomo queue job #{index} ({job['label']}).")
|
||||
return index
|
||||
|
||||
def tomo_queue_add_command(self, steps, label: str = None, idempotent: bool = None) -> int:
|
||||
"""Append a "command" job -- an ordered list of named-registry actions
|
||||
that reconfigure the beamline (move a device, ...) -- to the tomo
|
||||
queue.
|
||||
|
||||
Unlike tomo_queue_add(), this does not snapshot the current tomo scan
|
||||
parameters; a command job has no "params" key at all.
|
||||
|
||||
Args:
|
||||
steps: A single {"action": ..., "kwargs": {...}} dict, or a list
|
||||
of them, run in sequence within this one job.
|
||||
label: Optional name for the job, shown by tomo_queue_show().
|
||||
Defaults to "job_<n>".
|
||||
idempotent: Overrides the default of "every step's registry
|
||||
idempotent flag, ANDed together". Set this explicitly if the
|
||||
registry defaults don't capture whether this particular
|
||||
sequence is safe to blindly re-run after a crash.
|
||||
|
||||
Returns:
|
||||
The index of the newly added job.
|
||||
"""
|
||||
if isinstance(steps, dict):
|
||||
steps = [steps]
|
||||
if not steps:
|
||||
raise ValueError("tomo_queue_add_command: at least one step is required.")
|
||||
|
||||
resolved_steps = []
|
||||
all_idempotent = True
|
||||
for step in steps:
|
||||
action_name = step.get("action")
|
||||
kwargs = step.get("kwargs", {})
|
||||
self._validate_action_kwargs(action_name, kwargs)
|
||||
all_idempotent = all_idempotent and self._TOMO_QUEUE_ACTIONS[action_name]["idempotent"]
|
||||
resolved_steps.append({"action": action_name, "kwargs": kwargs})
|
||||
|
||||
try:
|
||||
json.dumps(resolved_steps)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Command job steps must be JSON-serializable: {exc}") from exc
|
||||
|
||||
index = len(self._tomo_queue_proxy)
|
||||
job = {
|
||||
"kind": "command",
|
||||
"id": uuid.uuid4().hex,
|
||||
"label": label or f"job_{index + 1}",
|
||||
"steps": resolved_steps,
|
||||
"idempotent": all_idempotent if idempotent is None else idempotent,
|
||||
"status": "pending",
|
||||
"added_at": datetime.datetime.now().isoformat(),
|
||||
}
|
||||
self._tomo_queue_proxy.append(job)
|
||||
print(f"Added command queue job #{index} ({job['label']}).")
|
||||
return index
|
||||
|
||||
def tomo_queue_delete(self, *indices: int) -> None:
|
||||
"""Delete one or more jobs from the tomo queue by index.
|
||||
|
||||
Accepts any number of indices, e.g. ``tomo_queue_delete(2, 5)`` to
|
||||
drop several jobs in one call. All indices are resolved against the
|
||||
queue as it currently stands and deleted highest-index-first, so
|
||||
passing several indices in any order is safe and won't shift the
|
||||
meaning of the indices still to be deleted.
|
||||
"""
|
||||
if not indices:
|
||||
print("No index given.")
|
||||
return
|
||||
for index in sorted(set(indices), reverse=True):
|
||||
job = self._tomo_queue_proxy.pop(index)
|
||||
print(f"Deleted tomo queue job #{index} ({job['label']}).")
|
||||
|
||||
def tomo_queue_move(self, index: int, new_index: int) -> None:
|
||||
"""Move a pending tomo queue job to a new position.
|
||||
|
||||
Only ``pending`` jobs can be reordered -- a ``running``,
|
||||
``incomplete``, or ``done`` job stays put. ``incomplete`` is a floor
|
||||
exactly like ``running``: tomo_queue_execute()'s pick-next loop
|
||||
always resumes whichever of the two it finds first, regardless of
|
||||
list order, so a pending job moved "ahead" of one in the list would
|
||||
visually suggest a run order that isn't real. A move is also
|
||||
refused if it would place the job at or above the position of any
|
||||
currently ``running``/``incomplete`` job.
|
||||
|
||||
Args:
|
||||
index: Current index of the job to move.
|
||||
new_index: Index to move it to (list-insert semantics: the rest
|
||||
of the queue shifts to make room).
|
||||
"""
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
if not 0 <= index < len(jobs):
|
||||
raise ValueError(f"tomo_queue_move: index {index} out of range.")
|
||||
if not 0 <= new_index < len(jobs):
|
||||
raise ValueError(f"tomo_queue_move: new_index {new_index} out of range.")
|
||||
job = jobs[index]
|
||||
if job.get("status", "pending") != "pending":
|
||||
raise ValueError(
|
||||
f"tomo_queue_move: job #{index} is '{job.get('status')}' -- only "
|
||||
"pending jobs can be reordered."
|
||||
)
|
||||
floor_indices = [
|
||||
i for i, j in enumerate(jobs) if j.get("status") in ("running", "incomplete")
|
||||
]
|
||||
if floor_indices and new_index <= max(floor_indices):
|
||||
raise ValueError(
|
||||
f"tomo_queue_move: can't move to position {new_index} -- job "
|
||||
f"#{max(floor_indices)} is running/incomplete and is a hard floor "
|
||||
"for reordering."
|
||||
)
|
||||
self._tomo_queue_proxy.move(index, new_index)
|
||||
print(f"Moved tomo queue job #{index} -> #{new_index} ({job['label']}).")
|
||||
|
||||
def tomo_queue_clear(self) -> None:
|
||||
"""Empty the tomo queue."""
|
||||
self._tomo_queue_proxy.clear()
|
||||
print("Tomo queue cleared.")
|
||||
|
||||
def _describe_tomo_job_line(self, job: dict) -> str:
|
||||
"""Per-setup one-line summary of a "tomo" job's snapshotted params,
|
||||
for tomo_queue_show(). Override to reference this setup's actual
|
||||
param names (e.g. flomni's fovx/fovy vs lamni's tomo_circfov).
|
||||
"""
|
||||
p = job["params"]
|
||||
return (
|
||||
f"type={p.get('tomo_type')} step={p.get('tomo_shellstep')}um "
|
||||
f"ctime={p.get('tomo_countingtime')}s"
|
||||
)
|
||||
|
||||
def tomo_queue_show(self) -> list:
|
||||
"""Print and return the current tomo queue, one line per job."""
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
if not jobs:
|
||||
print("Tomo queue is empty.")
|
||||
return jobs
|
||||
for i, job in enumerate(jobs):
|
||||
if job.get("kind", "tomo") == "command":
|
||||
steps_str = " > ".join(
|
||||
f"{step['action']}{step['kwargs']}" if step["kwargs"] else step["action"]
|
||||
for step in job["steps"]
|
||||
)
|
||||
idem = "idem" if job.get("idempotent") else "NOT idem"
|
||||
print(f"[{i}] {job['status']:>10s} {job['label']:<20s} CMD {steps_str} [{idem}]")
|
||||
continue
|
||||
print(
|
||||
f"[{i}] {job['status']:>10s} {job['label']:<20s} {self._describe_tomo_job_line(job)}"
|
||||
)
|
||||
return jobs
|
||||
|
||||
def tomo_queue_execute(self, start_index: int = 0) -> None:
|
||||
"""Run all pending tomo queue jobs in sequence, on the current sample.
|
||||
|
||||
For each job, restores its snapshotted parameters onto the live
|
||||
properties (exactly as if set by hand) and then calls
|
||||
``tomo_scan()`` -- or, for a job that didn't run to completion last
|
||||
time, ``tomo_scan_resume()``, so it picks back up mid-scan instead
|
||||
of restarting from subtomo 1 / angle 0. Jobs already marked "done"
|
||||
are skipped, so simply calling tomo_queue_execute() again resumes
|
||||
from where it stopped (e.g. after fixing a hardware issue).
|
||||
|
||||
Pick-next execution model
|
||||
-------------------------
|
||||
The queue is re-read from its global var at the start of every job
|
||||
(not snapshotted once up front), and each job is addressed by its
|
||||
stable ``id``, not by its list index. This is what makes it safe to
|
||||
edit the queue *while it runs*: appending a job, deleting a pending
|
||||
job, or reordering the pending tail all take effect at the next job
|
||||
boundary, and a reorder/delete can never misdirect a status write to
|
||||
the wrong entry (an index-based loop could). Legacy jobs written
|
||||
before the ``id`` field existed are healed once up front by
|
||||
ensure_ids().
|
||||
|
||||
Resume-before-fresh ordering: if any job is in a resumable state, it
|
||||
is always run before any fresh "pending" job, regardless of list
|
||||
order. This preserves the single-resumable-job invariant -- tomo
|
||||
progress lives in one shared ``progress`` global var, so starting a
|
||||
fresh tomo_scan() before resuming an interrupted one would overwrite
|
||||
the very state tomo_scan_resume() needs.
|
||||
|
||||
A job is considered not-yet-complete (and so gets resumed rather
|
||||
than restarted) if its status is "incomplete" (a Python exception
|
||||
was caught and execution stopped cleanly) OR "running" (the queue
|
||||
process itself died -- killed kernel, dropped connection, power
|
||||
loss, ... -- before it had a chance to record anything).
|
||||
|
||||
If you've already manually called ``tomo_scan_resume()`` yourself
|
||||
to recover from a crash (bypassing the queue), that scan is now
|
||||
actually finished even though the queue still has the job marked
|
||||
"incomplete" or "running" -- mark it done yourself before calling
|
||||
this again, or it will be re-run from scratch. Address the job by
|
||||
its id (indices can shift if the queue is reordered):
|
||||
self._tomo_queue_proxy.update_by_id(job_id, status="done")
|
||||
|
||||
Command jobs (job["kind"] == "command", added by
|
||||
tomo_queue_add_command()) are dispatched to _run_command_job()
|
||||
instead of tomo_scan()/tomo_scan_resume() -- see its docstring for
|
||||
their own resume policy. A job with no "kind" key is treated as
|
||||
"tomo" (back-compat for queues persisted before command jobs
|
||||
existed).
|
||||
|
||||
Args:
|
||||
start_index: Position in the *current* queue order below which
|
||||
fresh "pending" jobs are ignored. Defaults to 0. Resumable
|
||||
jobs are always run regardless of start_index (a crashed
|
||||
job must be recovered wherever it sits). Jobs already marked
|
||||
"done" are skipped automatically either way.
|
||||
"""
|
||||
self._tomo_queue_proxy.ensure_ids()
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
if not jobs:
|
||||
print("Tomo queue is empty.")
|
||||
return
|
||||
|
||||
runnable = [
|
||||
job
|
||||
for idx, job in enumerate(jobs)
|
||||
if job["status"] != "done"
|
||||
and (job["status"] in ("incomplete", "running") or idx >= start_index)
|
||||
]
|
||||
if not runnable:
|
||||
print("No pending tomo queue jobs to run.")
|
||||
return
|
||||
|
||||
if not self.OMNYTools.yesno(
|
||||
f"Starting automatic execution of {len(runnable)} queued tomo scan(s) on"
|
||||
f" sample '{self.sample_name}'. OK?",
|
||||
"y",
|
||||
):
|
||||
print("Aborted.")
|
||||
return
|
||||
|
||||
while True:
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
# Resume any in-flight/interrupted job first, before starting any
|
||||
# fresh one, so the shared progress var is never clobbered. Search
|
||||
# the whole list (not the start_index slice): a crashed job must be
|
||||
# recovered wherever it sits.
|
||||
job = next((j for j in jobs if j["status"] in ("incomplete", "running")), None)
|
||||
if job is None:
|
||||
job = next(
|
||||
(
|
||||
j
|
||||
for idx, j in enumerate(jobs)
|
||||
if idx >= start_index and j["status"] == "pending"
|
||||
),
|
||||
None,
|
||||
)
|
||||
if job is None:
|
||||
break
|
||||
|
||||
job_id = job["id"]
|
||||
label = job["label"]
|
||||
kind = job.get("kind", "tomo")
|
||||
resume_job = job["status"] in ("incomplete", "running")
|
||||
|
||||
print(f"\n=== Tomo queue job: {label} ===")
|
||||
|
||||
self._tomo_queue_proxy.update_by_id(job_id, status="running")
|
||||
try:
|
||||
if kind == "command":
|
||||
self._run_command_job(job, resume_job)
|
||||
else:
|
||||
for name, value in job["params"].items():
|
||||
setattr(self, name, value)
|
||||
if resume_job:
|
||||
self.tomo_scan_resume()
|
||||
else:
|
||||
self.tomo_scan()
|
||||
except Exception as exc:
|
||||
self._tomo_queue_proxy.update_by_id(job_id, status="incomplete")
|
||||
print(f"Tomo queue job '{label}' did not complete: {exc}")
|
||||
print(
|
||||
"Queue paused. Fix the issue and call tomo_queue_execute() "
|
||||
"again to resume from this job."
|
||||
)
|
||||
raise
|
||||
self._tomo_queue_proxy.update_by_id(job_id, status="done")
|
||||
|
||||
print("\nTomo queue finished -- all jobs done.")
|
||||
|
||||
def _run_command_job(self, job: dict, resume_job: bool) -> None:
|
||||
"""Run (or resume) a "command" job's steps in order.
|
||||
|
||||
No per-step resume cursor -- deliberately: tracking which step of a
|
||||
multi-step job completed means more persisted state and more edge
|
||||
cases than this warrants. Instead the job-level "idempotent" flag
|
||||
decides:
|
||||
|
||||
- idempotent: silently re-run every step from the top. Harmless --
|
||||
absolute moves and in/out actions re-run cleanly.
|
||||
- not idempotent: prompt the operator rather than silently re-run,
|
||||
which could be actively wrong for a relative/stateful step. "no"
|
||||
leaves the steps un-run; the caller (tomo_queue_execute) still
|
||||
marks the job "done" either way, matching the same edge case tomo
|
||||
jobs have -- a job that actually finished but crashed before its
|
||||
status hit "done".
|
||||
"""
|
||||
label = job["label"]
|
||||
if resume_job:
|
||||
if job.get("idempotent"):
|
||||
print(
|
||||
f"Command job '{label}' may have partially run before the crash "
|
||||
"-- idempotent, re-running from the top."
|
||||
)
|
||||
else:
|
||||
if not self.OMNYTools.yesno(
|
||||
f"Command job '{label}' may have partially run before the crash "
|
||||
"(not every step is idempotent). Re-run it from the top?",
|
||||
"n",
|
||||
):
|
||||
print(f"Skipping '{label}' -- it will be marked done without re-running.")
|
||||
return
|
||||
for step in job["steps"]:
|
||||
spec = self._TOMO_QUEUE_ACTIONS[step["action"]]
|
||||
method = getattr(self, spec["func"])
|
||||
method(**step["kwargs"])
|
||||
@@ -0,0 +1,149 @@
|
||||
# flomni → lamni feature comparison
|
||||
|
||||
Comparison of the `flomni` and `LamNI` ipython-client plugins, written to scope
|
||||
which flomni features (GUI, status webpage, tomo-parameter widget, angle
|
||||
distribution) are worth porting to lamni. This is a comparison only — no
|
||||
migration had been implemented yet as of when it was originally written.
|
||||
|
||||
**Status update (2026-07-16): items 2–4 of §5's recommended order are now
|
||||
done.** Only **item 1, the webpage generator (§4), remains open** — the rest
|
||||
of this document (originally a forward-looking comparison/plan) is now a
|
||||
historical snapshot of the *starting* gap, not the current state. See:
|
||||
|
||||
- `csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_QUEUE_PORT.md` —
|
||||
tomo-queue/job-list backend ported to lamni via a shared `TomoQueueMixin`
|
||||
(`csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py`).
|
||||
- `csaxs_bec/bec_ipython_client/plugins/LamNI/AI_docs/TOMO_PARAMS_GUI_PORT.md` —
|
||||
`TomoParamsWidget` generalized to lamni (setup detection, per-setup field
|
||||
lists/order, lamni-only offsets section, "Duplicate selected" queue
|
||||
button), plus several real `lamni.py` bugs found and fixed along the way
|
||||
(two `sub_tomo_scan()` angle bugs, three `tomo_scan()`/`write_pdf_report()`
|
||||
crashes, progress-GUI-auto-show + stale-busy-heartbeat).
|
||||
- **Remaining**: `LamNI_webpage_generator.py` is still the stub described in
|
||||
§4 below — `_collect_setup_data()` commented out, `_TEMP_MAP` empty,
|
||||
`HAS_TOMO_QUEUE = False` (now incorrect — a queue backend exists, see
|
||||
above), `TOMO_TYPES` placeholders, and never wired into
|
||||
`lamni.py.__init__`. §4's 5-step plan is otherwise still accurate.
|
||||
- Also still explicitly deferred (confirmed with Mirko, separate future
|
||||
tasks): `zero_deg_reference_at_each_subtomo` for lamni (not implemented at
|
||||
all yet), and sharing angle-calculation code between flomni/lamni
|
||||
(currently same *shape* but not literally shared/DRY'd).
|
||||
|
||||
## 1. Scope & method
|
||||
|
||||
Compared:
|
||||
- `csaxs_bec/bec_ipython_client/plugins/flomni/` vs
|
||||
`csaxs_bec/bec_ipython_client/plugins/LamNI/` (file-by-file, line counts, class
|
||||
structure).
|
||||
- The shared `csaxs_bec/bec_widgets/widgets/tomo_params/tomo_params.py`
|
||||
(`TomoParamsWidget`, `TomoQueueDialog`, `CommandJobBuilderDialog`) — used by
|
||||
flomni today, not by lamni.
|
||||
- `flomni_webpage_generator.py` / `LamNI_webpage_generator.py` and their shared
|
||||
base class.
|
||||
|
||||
Not covered here: device-layer differences (`csaxs_bec/devices/omny/...`),
|
||||
scan classes (`csaxs_bec/scans/flomni_fermat_scan.py` vs
|
||||
`csaxs_bec/scans/LamNIFermatScan.py`), or user-facing docs under
|
||||
`docs/user/ptychography/`.
|
||||
|
||||
## 2. Side-by-side feature table
|
||||
|
||||
| Feature | flomni | LamNI | Gap |
|
||||
|---|---|---|---|
|
||||
| Main plugin file | `flomni.py`, 4520 lines | `lamni.py`, 1226 lines | lamni is ~3.7x smaller; several flomni concepts have no lamni counterpart at all (see below) |
|
||||
| Plugin class structure | `Flomni(...)` composed from mixins defined *inline* in `flomni.py`: `FlomniInitStagesMixin`, `FlomniSampleTransferMixin`, `FlomniAlignmentMixin`, plus `_ProgressProxy`, `_TomoQueueProxy`, `_GlobalVarParam` helper classes | `LamNI(LamNIAlignmentMixin, LamNIOpticsMixin, LamniGuiTools)` — alignment/optics split into separate files (`lamni_alignment_mixin.py`, `lamni_optics_mixin.py`); only `_ProgressProxy` carried over | lamni has no `_TomoQueueProxy` / `_GlobalVarParam` equivalents — the tomo-queue-as-job-list mechanism the GUI widget depends on doesn't exist on lamni's producer side at all, not just "unwired" |
|
||||
| Optics mixin | `flomni_optics_mixin.py`, 344 lines | `lamni_optics_mixin.py`, 332 lines | comparable size, setup-specific content; not a gap, just parallel implementations |
|
||||
| Alignment mixin | inline `FlomniAlignmentMixin` in `flomni.py` (from line 1259) | separate `lamni_alignment_mixin.py`, 369 lines | organizational difference only |
|
||||
| X-ray eye alignment | `x_ray_eye_align.py`, 440 lines | `x_ray_eye_align.py`, 568 lines | both present, lamni's is larger — not a gap |
|
||||
| Extra scan types | none | `extra_tomo.py`: `MagLamNI` (XMCD/magnetic, `sub_tomo_scan`, `rotate_slowly`, `_at_each_angle` hook), `DataDrivenLamNI` (reads per-projection params from HDF5) | lamni-only feature; flomni has no analogue and doesn't need one for this migration |
|
||||
| Tomo-parameter GUI widget | Used via shared `TomoParamsWidget` (`bec_widgets/widgets/tomo_params/tomo_params.py`) | **Not usable.** `TomoParamsWidget._check_flomni_available()` probes for the `fsamroy` device (flomni's rotation stage) and shows a warning banner + disables editing if absent — lamni sessions are explicitly treated as unsupported | primary migration target, see §3 |
|
||||
| Tomo queue (job list: add/execute/clear scan jobs from the GUI) | Supported — `TomoQueueDialog`, `CommandJobBuilderDialog` in `tomo_params.py`, backed by `Flomni._TomoQueueProxy` | Not supported — no producer-side equivalent | needs both GUI generalization *and* a lamni-side queue proxy; not a thin wrapper job |
|
||||
| Angle-distribution / projection scheduling ("tomo_type") | 3 modes: (1) 8 equally-spaced sub-tomograms, (2) golden ratio sorted in bunches, (3) equally-spaced with golden-ratio start. Implemented via `_golden()` in `flomni.py` (~line 3122), surfaced in the widget's `TOMO_TYPES` dict | Same underlying `_golden()`-style scheduling exists in `lamni.py` (`sub_tomo_scan`, `tomo_scan`, own `_golden`/`_golden_new` helpers), but only 2 modes: equally-spaced grid with either 2 or 8 sub-tomograms, hardcoded to 360° | logic already exists on both sides; the widget just needs a lamni-specific `TOMO_TYPES` mapping instead of reusing flomni's 3-mode set |
|
||||
| 180°/360° angle range | Configurable per scan via `tomo_angle_range` global var (∈ {180, 360}) | No such concept — `lamni.py` hardcodes 360° everywhere (`angle_end = start_angle + 360`, `_golden(..., maxangle=360, ...)`) | confirms user's expectation: lamni has no 180° mode and none should be exposed in a generalized widget for lamni |
|
||||
| Single-point acquisition (skip Fermat scan, single point per projection) | `single_point_instead_of_fermat_scan` / `single_point_random_shift_max` global vars, dedicated widget section with enable/disable wiring | No equivalent global var or code path in `lamni.py` | confirms user's expectation: not needed for lamni, should stay hidden/omitted rather than generalized |
|
||||
| Global-var parameter names (shared concepts) | `fovx` / `fovy`, `stitch_x` / `stitch_y`, `tomo_angle_stepsize`, `tomo_countingtime`, `corridor_size`, `frames_per_trigger`, `ptycho_reconstruct_foldername`, `golden_ratio_bunch_size`, `golden_max_number_of_projections` | Overlapping concepts under different names: `tomo_circfov` (single value, not x/y — laminography FOV is circular) instead of `fovx`/`fovy`; `lamni_stitch_x`/`lamni_stitch_y` instead of `stitch_x`/`stitch_y`; plus lamni-only `lamni_piezo_range_x`/`lamni_piezo_range_y` with no flomni analogue. Same names reused as-is: `tomo_angle_stepsize`, `tomo_countingtime`, `corridor_size`, `frames_per_trigger`, `ptycho_reconstruct_foldername`, `golden_ratio_bunch_size`, `golden_max_number_of_projections`, `golden_projections_at_0_deg_for_damage_estimation` | the widget's hardcoded `QUEUE_PARAM_NAMES`/`DEFAULTS` (mirroring `Flomni._TOMO_SCAN_PARAM_NAMES` verbatim) can't just be reused for lamni — needs a per-setup name mapping, not a param rename in lamni.py |
|
||||
| Status webpage / webapp | Rich: `flomni_webpage_generator.py`, 3042 lines. Temperatures, sample name, tomo queue card, live progress, audio alerts, HTTP upload + local server, session auth | Stub: `LamNI_webpage_generator.py`, 104 lines. Subclasses the shared `WebpageGeneratorBase`, but `_collect_setup_data()` is commented out (returns only `{"type": "lamni"}`), `_TEMP_MAP` is empty, `HAS_TOMO_QUEUE = False`, and `TOMO_TYPES` values are marked `TODO` placeholders pending confirmation of lamni's actual `tomo_type` values | biggest line-count gap, but architecturally the easiest migration — `WebpageGeneratorBase` + `make_webpage_generator()` factory already support lamni, the stub just needs finishing (see §4) |
|
||||
| Webpage generator wired into plugin startup | Yes — `flomni.py` instantiates `FlomniWebpageGenerator` in `Flomni.__init__` | **No** — `lamni.py` has zero references to `webpage_gen` or `WebpageGenerator` anywhere; the stub class exists but is never instantiated | needs one `__init__` wiring change plus finishing §4 |
|
||||
| `AI_docs/` planning docs | 8 files: `Webpage.md`, `TOMO_QUEUE_COMMAND_JOBS_PLAN.md`, `TOMO_QUEUE_GUI_TESTING.md`, `TOMO_QUEUE_TESTING.md`, `E2E_TESTING_GUIDE.md`, `SIMULATED_ENDSTATIONS.md`, `BEC info.md`, `Endstation INFO flomni lamni omny.md` | none (this file is the first) | n/a — documentation only |
|
||||
|
||||
## 3. Tomo-parameter-widget generalization
|
||||
|
||||
`TomoParamsWidget` (`bec_widgets/widgets/tomo_params/tomo_params.py`) is
|
||||
currently written as a flomni-only widget that happens to live in the shared
|
||||
`bec_widgets` package, not a genuinely setup-aware widget. Concretely
|
||||
flomni-hardcoded:
|
||||
|
||||
- `_check_flomni_available()` (~line 220): gates the entire widget on the
|
||||
presence of the `fsamroy` device. For a generalized widget this needs to
|
||||
become "detect which setup is active" (flomni / lamni / eventually omny)
|
||||
rather than a yes/no flomni check.
|
||||
- `QUEUE_PARAM_NAMES` / `DEFAULTS` (lines 98–144): a literal copy of
|
||||
`Flomni._TOMO_SCAN_PARAM_NAMES`, including flomni-only param names (`fovx`,
|
||||
`fovy`, `stitch_x`, `stitch_y`) that don't exist under those names in lamni's
|
||||
global vars (`tomo_circfov`, `lamni_stitch_x`, `lamni_stitch_y`).
|
||||
- The 180°/360° `tomo_angle_range` dropdown (line 137, referenced ~line 428,
|
||||
568, 633, 668): must not be offered at all for lamni — lamni is 360°-only,
|
||||
not "180° available but unused."
|
||||
- The single-point acquisition section (`single_point_instead_of_fermat_scan`,
|
||||
`single_point_random_shift_max`, lines 133–134, 379–398, 735–746): must be
|
||||
omitted entirely for lamni, not just disabled.
|
||||
- `TOMO_TYPES` (lines 80–84): flomni's 3-mode set doesn't match lamni's 2-mode
|
||||
set (2-subtomo / 8-subtomo equally-spaced grid, per the `TODO` in
|
||||
`LamNI_webpage_generator.py`'s placeholder `TOMO_TYPES` dict) — these
|
||||
`tomo_type` integer values need confirming against `lamni.py` before they can
|
||||
be wired into either the widget or the webpage generator.
|
||||
- Tomo queue backing store: the widget's queue/job management
|
||||
(`TomoQueueDialog`, `CommandJobBuilderDialog`) is backed by
|
||||
`Flomni._TomoQueueProxy` / `_GlobalVarParam`, which have no lamni-side
|
||||
equivalent — this is not just a widget change, it needs a producer-side
|
||||
queue mechanism added to `lamni.py` first (or the queue/job-list feature
|
||||
scoped out of the lamni version of the widget entirely, if lamni doesn't
|
||||
need scan-job queuing the way flomni's automated multi-day runs do).
|
||||
|
||||
Proposed shape (not implemented): a per-setup config object (param-name
|
||||
mapping + capability flags: `has_180_mode`, `has_single_point`,
|
||||
`has_tomo_queue`, `tomo_types` dict) that the widget reads at construction time
|
||||
instead of the current hardcoded flomni constants, with setup detection
|
||||
replacing `_check_flomni_available()`.
|
||||
|
||||
## 4. Webpage generator gap
|
||||
|
||||
To bring lamni's status page to parity with flomni's, `LamNI_webpage_generator.py`
|
||||
needs, in order:
|
||||
|
||||
1. Confirm lamni's actual `tomo_type` integer values (currently `TODO`
|
||||
placeholders assuming 1→2-subtomo, 2→8-subtomo) against `lamni.py`'s real
|
||||
usage.
|
||||
2. Fill in `_TEMP_MAP` with lamni's actual temperature/humidity device dot-paths
|
||||
(placeholder comments reference `lamni_temphum.*`, unconfirmed).
|
||||
3. Implement `_collect_setup_data()` (currently fully commented out) — sample
|
||||
name, FOV, exposure time, angle step, mirroring flomni's version but using
|
||||
lamni's global-var names (`tomo_circfov`, `lamni_stitch_x/y`, etc.).
|
||||
4. Decide on `HAS_TOMO_QUEUE` — stays `False` unless/until a lamni tomo-queue
|
||||
backing store exists (§3).
|
||||
5. Wire `LamniWebpageGenerator` into `lamni.py.__init__`, the way
|
||||
`FlomniWebpageGenerator` is wired into `Flomni.__init__` — currently there
|
||||
is no `webpage_gen` reference anywhere in `lamni.py`.
|
||||
|
||||
No changes needed to `WebpageGeneratorBase` or `make_webpage_generator()` —
|
||||
both already support this without modification.
|
||||
|
||||
## 5. Recommendation / candidate migration order
|
||||
|
||||
1. **Webpage generator** — lowest risk, highest immediate payoff. Architecture
|
||||
already supports it; work is filling in a known-shape stub (§4). No GUI
|
||||
widget dependency.
|
||||
2. **Angle-distribution mapping for the tomo-params widget** — expose lamni's
|
||||
existing 2-mode scheduling through the widget's `TOMO_TYPES`, gated by setup
|
||||
detection, without yet tackling the queue/job-list machinery.
|
||||
3. **Tomo-parameter widget generalization proper** (§3) — setup-detection
|
||||
refactor + per-setup param-name mapping + hiding 180°/single-point for
|
||||
lamni. Larger, touches the widget's core assumptions.
|
||||
4. **Tomo queue / job-list feature** — only if lamni actually needs automated
|
||||
multi-projection job queuing the way flomni does; requires new producer-side
|
||||
plumbing in `lamni.py` (`_TomoQueueProxy`-equivalent), not just GUI changes.
|
||||
Worth confirming with Mirko whether lamni's workflow needs this at all
|
||||
before investing here.
|
||||
|
||||
This document is the input for deciding scope/order with Mirko; no
|
||||
implementation plan has been written yet for any of the above.
|
||||
@@ -1,12 +1,10 @@
|
||||
import builtins
|
||||
import datetime
|
||||
import inspect
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import subprocess
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
import h5py
|
||||
@@ -16,12 +14,16 @@ from bec_lib.alarm_handler import AlarmBase
|
||||
from bec_lib.endpoints import MessageEndpoints
|
||||
from bec_lib.pdf_writer import PDFWriter
|
||||
from bec_lib.scan_repeat import scan_repeat
|
||||
from typeguard import check_type, typechecked
|
||||
from typeguard import typechecked
|
||||
|
||||
from csaxs_bec.bec_ipython_client.plugins.cSAXS import cSAXSBeamlineChecks
|
||||
from csaxs_bec.bec_ipython_client.plugins.flomni.flomni_optics_mixin import FlomniOpticsMixin
|
||||
from csaxs_bec.bec_ipython_client.plugins.flomni.gui_tools import flomniGuiTools
|
||||
from csaxs_bec.bec_ipython_client.plugins.flomni.x_ray_eye_align import XrayEyeAlign
|
||||
from csaxs_bec.bec_ipython_client.plugins.OMNY_shared.tomo_queue_mixin import (
|
||||
TomoQueueMixin,
|
||||
_GlobalVarParam,
|
||||
)
|
||||
from csaxs_bec.bec_ipython_client.plugins.omny.omny_general_tools import (
|
||||
OMNYTools,
|
||||
PtychoReconstructor,
|
||||
@@ -1610,155 +1612,8 @@ class _ProgressProxy:
|
||||
return self._load()
|
||||
|
||||
|
||||
class _TomoQueueProxy:
|
||||
"""List-like proxy that persists a queue of tomo parameter snapshots as a
|
||||
BEC global variable, mirroring the pattern used by :class:`_ProgressProxy`.
|
||||
|
||||
Each entry is a plain dict: ``{"label": ..., "params": {...}, "status":
|
||||
..., "added_at": ...}``, where ``params`` holds a snapshot of all
|
||||
global-var-backed tomo scan parameters (see
|
||||
``Flomni._TOMO_SCAN_PARAM_NAMES``). Stored as a single list under
|
||||
``tomo_queue`` so the queue is visible from any BEC client session via
|
||||
``client.get_global_var("tomo_queue")`` and survives a kernel restart.
|
||||
"""
|
||||
|
||||
_GLOBAL_VAR_KEY = "tomo_queue"
|
||||
|
||||
def __init__(self, client):
|
||||
self._client = client
|
||||
|
||||
def _load(self) -> list:
|
||||
val = self._client.get_global_var(self._GLOBAL_VAR_KEY)
|
||||
if val is None:
|
||||
return []
|
||||
return val
|
||||
|
||||
def _save(self, jobs: list) -> None:
|
||||
self._client.set_global_var(self._GLOBAL_VAR_KEY, jobs)
|
||||
|
||||
def as_list(self) -> list:
|
||||
"""Return a plain copy of the current queue."""
|
||||
return self._load()
|
||||
|
||||
def append(self, job: dict) -> int:
|
||||
jobs = self._load()
|
||||
jobs.append(job)
|
||||
self._save(jobs)
|
||||
return len(jobs) - 1
|
||||
|
||||
def pop(self, index: int) -> dict:
|
||||
jobs = self._load()
|
||||
job = jobs.pop(index)
|
||||
self._save(jobs)
|
||||
return job
|
||||
|
||||
def update(self, index: int, **kwargs) -> None:
|
||||
jobs = self._load()
|
||||
jobs[index].update(kwargs)
|
||||
self._save(jobs)
|
||||
|
||||
def update_by_id(self, job_id: str, **kwargs) -> bool:
|
||||
"""Update the job whose ``id`` matches ``job_id``.
|
||||
|
||||
Returns True if a matching job was found and updated, False otherwise
|
||||
(e.g. the job was deleted concurrently). Addressing a job by its stable
|
||||
id rather than its list index makes status writes safe against queue
|
||||
edits made while the queue runs: a reorder, or a delete of some *other*
|
||||
job, can no longer misdirect a status write to the wrong entry.
|
||||
"""
|
||||
jobs = self._load()
|
||||
for job in jobs:
|
||||
if job.get("id") == job_id:
|
||||
job.update(kwargs)
|
||||
self._save(jobs)
|
||||
return True
|
||||
return False
|
||||
|
||||
def move(self, index: int, new_index: int) -> None:
|
||||
"""Move the job at ``index`` to ``new_index``, shifting the rest.
|
||||
|
||||
Pure list mechanics -- no status checks. Callers (e.g.
|
||||
``Flomni.tomo_queue_move``) are responsible for deciding whether a
|
||||
given move is allowed (running-job floor, movable statuses, ...).
|
||||
"""
|
||||
jobs = self._load()
|
||||
job = jobs.pop(index)
|
||||
jobs.insert(new_index, job)
|
||||
self._save(jobs)
|
||||
|
||||
def ensure_ids(self) -> list:
|
||||
"""Assign a stable id to any job that lacks one and persist the result.
|
||||
|
||||
Back-compat shim for queues written before jobs carried an ``id`` field
|
||||
(and for any job added by older code). Called once at the top of
|
||||
tomo_queue_execute() so the pick-next executor can address every job by
|
||||
id. Returns the (possibly healed) job list.
|
||||
"""
|
||||
jobs = self._load()
|
||||
changed = False
|
||||
for job in jobs:
|
||||
if not job.get("id"):
|
||||
job["id"] = uuid.uuid4().hex
|
||||
changed = True
|
||||
if changed:
|
||||
self._save(jobs)
|
||||
return jobs
|
||||
|
||||
def clear(self) -> None:
|
||||
self._save([])
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._load())
|
||||
|
||||
def __getitem__(self, index):
|
||||
return self._load()[index]
|
||||
|
||||
|
||||
class _GlobalVarParam:
|
||||
"""Descriptor for a Flomni parameter backed by a BEC global variable.
|
||||
|
||||
Consolidates the boilerplate getter/setter pairs that read from and write
|
||||
to the BEC global-var store. The value persists across BEC client restarts
|
||||
(instance attributes do not), and is readable from other client sessions
|
||||
via ``client.get_global_var(<name>)``.
|
||||
|
||||
The global-var key is taken from the attribute name the descriptor is
|
||||
assigned to (see :meth:`__set_name__`), so it always matches the property
|
||||
name -- exactly the invariant the hand-written pairs relied on.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
default
|
||||
Value returned by the getter when the global var is unset (``None``).
|
||||
type_
|
||||
Optional expected type. When given, the setter validates ``val`` with
|
||||
:func:`typeguard.check_type` before storing, reproducing the
|
||||
``@typechecked`` guard that previously wrapped some setters (raising
|
||||
``TypeCheckError`` on a mismatch).
|
||||
"""
|
||||
|
||||
def __init__(self, default, type_=None):
|
||||
self._default = default
|
||||
self._type = type_
|
||||
|
||||
def __set_name__(self, owner, name):
|
||||
self._key = name
|
||||
|
||||
def __get__(self, obj, objtype=None):
|
||||
if obj is None:
|
||||
return self
|
||||
val = obj.client.get_global_var(self._key)
|
||||
if val is None:
|
||||
return self._default
|
||||
return val
|
||||
|
||||
def __set__(self, obj, val):
|
||||
if self._type is not None:
|
||||
check_type(val, self._type)
|
||||
obj.client.set_global_var(self._key, val)
|
||||
|
||||
|
||||
class Flomni(
|
||||
TomoQueueMixin,
|
||||
FlomniInitStagesMixin,
|
||||
FlomniSampleTransferMixin,
|
||||
FlomniAlignmentMixin,
|
||||
@@ -1776,11 +1631,6 @@ class Flomni(
|
||||
self.special_angle_tolerance = 20
|
||||
self._current_special_angles = []
|
||||
self._beam_is_okay = True
|
||||
# Session-only (not global-var-backed): the hooks themselves are
|
||||
# Python callables, which can't be persisted the way params/queue
|
||||
# jobs are. Only the *name* selecting one is ever snapshotted (see
|
||||
# at_each_angle_hook / register_at_each_angle_hook()).
|
||||
self._at_each_angle_hooks: dict = {}
|
||||
self._progress_proxy = _ProgressProxy(self.client)
|
||||
# Deliberately NOT calling reset() here: this dict is persisted via a
|
||||
# BEC global var specifically so it survives a kernel restart (e.g.
|
||||
@@ -1791,14 +1641,7 @@ class Flomni(
|
||||
# resets the relevant fields itself (see the "new scan" branch
|
||||
# below); use tomo_progress_reset() if you want to explicitly clear
|
||||
# stale progress without starting a new scan.
|
||||
self._tomo_queue_proxy = _TomoQueueProxy(self.client)
|
||||
# Published so a future GUI never needs a hardcoded mirror of the
|
||||
# action registry -- see _describe_tomo_queue_actions().
|
||||
self.client.set_global_var("tomo_queue_actions", self._describe_tomo_queue_actions())
|
||||
# Reset to empty on every Flomni() -- hooks are session-only (see
|
||||
# self._at_each_angle_hooks above), so a stale list published by a
|
||||
# since-ended session must not linger and look registered.
|
||||
self._publish_at_each_angle_hooks()
|
||||
self._init_tomo_queue()
|
||||
from csaxs_bec.bec_ipython_client.plugins.flomni.flomni_webpage_generator import (
|
||||
FlomniWebpageGenerator,
|
||||
)
|
||||
@@ -2013,11 +1856,11 @@ class Flomni(
|
||||
|
||||
manual_shift_y = _GlobalVarParam(0.0)
|
||||
|
||||
# Name of a registered at_each_angle hook (see register_at_each_angle_hook()),
|
||||
# or None for the default per-projection behaviour. Snapshotted like any
|
||||
# at_each_angle_hook is inherited from TomoQueueMixin: name of a
|
||||
# registered at_each_angle hook (see register_at_each_angle_hook()), or
|
||||
# None for the default per-projection behaviour. Snapshotted like any
|
||||
# other tomo scan parameter, so a tomo-queue job can carry its own hook
|
||||
# selection -- see _at_each_angle().
|
||||
at_each_angle_hook = _GlobalVarParam(None)
|
||||
|
||||
@property
|
||||
def single_point_random_shift_max(self):
|
||||
@@ -3016,109 +2859,6 @@ class Flomni(
|
||||
|
||||
self.tomo_scan_projection(angle, _internal=True)
|
||||
|
||||
def register_at_each_angle_hook(self, name: str, func) -> None:
|
||||
"""Register a per-projection hook function under ``name``, so a
|
||||
tomo-queue job can select it by name via the ``at_each_angle_hook``
|
||||
parameter and have it run automatically -- unlike the older
|
||||
``flomni_at_each_angle`` builtins-based hook, which the queue can't
|
||||
see (it isn't a snapshotted parameter, so a queued job has no way
|
||||
to record "use this hook").
|
||||
|
||||
``func`` is called as ``func(self, angle)`` at every projection
|
||||
angle, exactly like the interactive ``flomni_at_each_angle`` hook --
|
||||
write it the same way, just give it a name and register it here
|
||||
instead of assigning it to the ``flomni_at_each_angle`` builtin.
|
||||
|
||||
Hooks are **session-only**: they live in this ``Flomni`` instance's
|
||||
memory, not in a global var, so they do NOT survive a kernel
|
||||
restart. A queued job that references a hook by name raises a clear
|
||||
error at execution time if the name isn't registered in the session
|
||||
running the queue -- re-run this registration call, then resume
|
||||
``tomo_queue_execute()``.
|
||||
|
||||
Example -- record a projection, insert a polarizer, record again::
|
||||
|
||||
def polarizer_modulation(flomni, angle):
|
||||
flomni.tomo_scan_projection(angle, _internal=True)
|
||||
umv(dev.polarizer, "in")
|
||||
flomni.tomo_scan_projection(angle, _internal=True)
|
||||
umv(dev.polarizer, "out")
|
||||
|
||||
flomni.register_at_each_angle_hook("polarizer_mod", polarizer_modulation)
|
||||
flomni.at_each_angle_hook = "polarizer_mod"
|
||||
flomni.tomo_queue_add(label="polarizer run")
|
||||
flomni.at_each_angle_hook = None # back to default for the next job
|
||||
"""
|
||||
if not callable(func):
|
||||
raise TypeError(f"register_at_each_angle_hook: {func!r} is not callable.")
|
||||
self._at_each_angle_hooks[name] = func
|
||||
self._publish_at_each_angle_hooks()
|
||||
print(f"Registered at_each_angle hook '{name}'.")
|
||||
|
||||
def unregister_at_each_angle_hook(self, name: str) -> None:
|
||||
"""Remove a previously registered at_each_angle hook by name."""
|
||||
if self._at_each_angle_hooks.pop(name, None) is None:
|
||||
print(f"No at_each_angle hook named '{name}' is registered.")
|
||||
else:
|
||||
self._publish_at_each_angle_hooks()
|
||||
print(f"Unregistered at_each_angle hook '{name}'.")
|
||||
|
||||
def _publish_at_each_angle_hooks(self) -> None:
|
||||
"""Publish the names of all currently registered at_each_angle
|
||||
hooks as the ``tomo_at_each_angle_hooks`` global var (a plain list
|
||||
of strings) -- so a GUI (or any other client) can build a dropdown
|
||||
from it directly, no hardcoded mirror, same reasoning as
|
||||
``tomo_queue_actions`` for the command-job registry (see
|
||||
_describe_tomo_queue_actions()). Unlike that registry this list
|
||||
changes at runtime, whenever register_/unregister_at_each_angle_hook()
|
||||
is called -- not just once at __init__.
|
||||
"""
|
||||
self.client.set_global_var("tomo_at_each_angle_hooks", sorted(self._at_each_angle_hooks))
|
||||
|
||||
def list_at_each_angle_hooks(self) -> list:
|
||||
"""Print and return the names of all at_each_angle hooks registered
|
||||
in this session. The active one (``self.at_each_angle_hook``, if
|
||||
any) is marked."""
|
||||
names = sorted(self._at_each_angle_hooks)
|
||||
if not names:
|
||||
print("No at_each_angle hooks registered.")
|
||||
else:
|
||||
for name in names:
|
||||
marker = " (active)" if name == self.at_each_angle_hook else ""
|
||||
print(f" {name}{marker}")
|
||||
return names
|
||||
|
||||
def _describe_active_hook(self) -> str | None:
|
||||
"""Human-readable description of ``self.at_each_angle_hook`` for
|
||||
display in tomo_parameters()/scilog/the PDF report: the name, or
|
||||
the name flagged as unregistered if it doesn't (or no longer)
|
||||
resolve in this session -- e.g. unregistered after the job that
|
||||
used it was added, or set by a session that has since restarted.
|
||||
Returns None if no hook is set at all.
|
||||
"""
|
||||
hook = self.at_each_angle_hook
|
||||
if not hook:
|
||||
return None
|
||||
if hook in self._at_each_angle_hooks:
|
||||
return hook
|
||||
return f"{hook} (NOT registered in this session)"
|
||||
|
||||
def _active_hook_source(self) -> str | None:
|
||||
"""Source code of the currently active at_each_angle hook, for
|
||||
inclusion in the scilog/PDF record -- so what a tomogram actually
|
||||
did is part of the permanent log, not just the hook's name.
|
||||
Returns None if there's no active hook, it isn't registered in
|
||||
this session, or its source can't be retrieved (e.g. it wasn't
|
||||
defined via a normal `def`/interactive cell).
|
||||
"""
|
||||
hook_func = self._at_each_angle_hooks.get(self.at_each_angle_hook)
|
||||
if hook_func is None:
|
||||
return None
|
||||
try:
|
||||
return inspect.getsource(hook_func)
|
||||
except (OSError, TypeError):
|
||||
return None
|
||||
|
||||
def _golden(self, ii, howmany_sorted, maxangle, reverse=False):
|
||||
"""returns the iis golden ratio angle of sorted bunches of howmany_sorted and its subtomo number"""
|
||||
golden = []
|
||||
@@ -3940,462 +3680,15 @@ class Flomni(
|
||||
},
|
||||
}
|
||||
|
||||
def tomo_queue_add(self, label: str = None) -> int:
|
||||
"""Snapshot the currently set tomo parameters and append them as a
|
||||
new job to the tomo queue (persisted, survives a kernel restart).
|
||||
|
||||
Typical usage::
|
||||
|
||||
flomni.tomo_parameters() # set up parameter set #1
|
||||
flomni.tomo_queue_add("fast overview scan")
|
||||
flomni.tomo_parameters() # change to parameter set #2
|
||||
flomni.tomo_queue_add("hires scan")
|
||||
flomni.tomo_queue_show()
|
||||
flomni.tomo_queue_execute() # runs both, in order, on this sample
|
||||
|
||||
Args:
|
||||
label: Optional name for the job, shown by tomo_queue_show().
|
||||
Defaults to "job_<n>".
|
||||
|
||||
Returns:
|
||||
The index of the newly added job.
|
||||
"""
|
||||
params = {name: getattr(self, name) for name in self._TOMO_SCAN_PARAM_NAMES}
|
||||
index = len(self._tomo_queue_proxy)
|
||||
job = {
|
||||
"kind": "tomo",
|
||||
"id": uuid.uuid4().hex,
|
||||
"label": label or f"job_{index + 1}",
|
||||
"params": params,
|
||||
"status": "pending",
|
||||
"added_at": datetime.datetime.now().isoformat(),
|
||||
}
|
||||
self._tomo_queue_proxy.append(job)
|
||||
print(f"Added tomo queue job #{index} ({job['label']}).")
|
||||
return index
|
||||
|
||||
def tomo_queue_add_command(self, steps, label: str = None, idempotent: bool = None) -> int:
|
||||
"""Append a "command" job -- an ordered list of named-registry actions
|
||||
that reconfigure the beamline (move a device, ...) -- to the tomo
|
||||
queue.
|
||||
|
||||
Unlike tomo_queue_add(), this does not snapshot the current tomo scan
|
||||
parameters; a command job has no "params" key at all.
|
||||
|
||||
Typical usage::
|
||||
|
||||
flomni.tomo_queue_add_command(
|
||||
[{"action": "move", "kwargs": {"positions": {"mokev": 6.2}}},
|
||||
{"action": "optimize_idgap", "kwargs": {"search_range": 0.5}}],
|
||||
label="reconfigure to 6.2 keV",
|
||||
)
|
||||
|
||||
Args:
|
||||
steps: A single {"action": ..., "kwargs": {...}} dict, or a list
|
||||
of them, run in sequence within this one job.
|
||||
label: Optional name for the job, shown by tomo_queue_show().
|
||||
Defaults to "job_<n>".
|
||||
idempotent: Overrides the default of "every step's registry
|
||||
idempotent flag, ANDed together". Set this explicitly if the
|
||||
registry defaults don't capture whether this particular
|
||||
sequence is safe to blindly re-run after a crash.
|
||||
|
||||
Returns:
|
||||
The index of the newly added job.
|
||||
"""
|
||||
if isinstance(steps, dict):
|
||||
steps = [steps]
|
||||
if not steps:
|
||||
raise ValueError("tomo_queue_add_command: at least one step is required.")
|
||||
|
||||
resolved_steps = []
|
||||
all_idempotent = True
|
||||
for step in steps:
|
||||
action_name = step.get("action")
|
||||
kwargs = step.get("kwargs", {})
|
||||
self._validate_action_kwargs(action_name, kwargs)
|
||||
all_idempotent = all_idempotent and self._TOMO_QUEUE_ACTIONS[action_name]["idempotent"]
|
||||
resolved_steps.append({"action": action_name, "kwargs": kwargs})
|
||||
|
||||
try:
|
||||
json.dumps(resolved_steps)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Command job steps must be JSON-serializable: {exc}") from exc
|
||||
|
||||
index = len(self._tomo_queue_proxy)
|
||||
job = {
|
||||
"kind": "command",
|
||||
"id": uuid.uuid4().hex,
|
||||
"label": label or f"job_{index + 1}",
|
||||
"steps": resolved_steps,
|
||||
"idempotent": all_idempotent if idempotent is None else idempotent,
|
||||
"status": "pending",
|
||||
"added_at": datetime.datetime.now().isoformat(),
|
||||
}
|
||||
self._tomo_queue_proxy.append(job)
|
||||
print(f"Added command queue job #{index} ({job['label']}).")
|
||||
return index
|
||||
|
||||
def _validate_action_kwargs(self, action_name: str, kwargs: dict) -> None:
|
||||
"""Add-time schema validation for a command-job step's kwargs.
|
||||
|
||||
Ergonomic, not the safety boundary: catches a typo or an out-of-range
|
||||
value now instead of at 2am. Each action method re-validates its own
|
||||
arguments against live hardware at execution time -- that's the
|
||||
actual safety boundary, since the schema and the hardware can
|
||||
disagree (limits changed; a value arrived via the CLI, bypassing
|
||||
this).
|
||||
"""
|
||||
spec = self._TOMO_QUEUE_ACTIONS.get(action_name)
|
||||
if spec is None:
|
||||
raise ValueError(f"Unknown queue action '{action_name}'.")
|
||||
schema = spec["params"]
|
||||
for key in kwargs:
|
||||
if key not in schema:
|
||||
raise ValueError(f"Action '{action_name}' has no parameter '{key}'.")
|
||||
for name, field in schema.items():
|
||||
if name not in kwargs:
|
||||
continue
|
||||
value = kwargs[name]
|
||||
field_type = field.get("type")
|
||||
if field_type == "float":
|
||||
if not isinstance(value, (int, float)) or isinstance(value, bool):
|
||||
raise ValueError(f"{action_name}.{name} must be a number, got {value!r}.")
|
||||
lo, hi = field.get("min"), field.get("max")
|
||||
if lo is not None and value < lo:
|
||||
raise ValueError(f"{action_name}.{name}={value} is below the minimum {lo}.")
|
||||
if hi is not None and value > hi:
|
||||
raise ValueError(f"{action_name}.{name}={value} is above the maximum {hi}.")
|
||||
elif field_type == "device_positions":
|
||||
if not isinstance(value, dict) or not value:
|
||||
raise ValueError(
|
||||
f"{action_name}.{name} must be a non-empty {{device: target}} dict."
|
||||
)
|
||||
for device_name in value:
|
||||
if device_name not in self._TOMO_QUEUE_MOVE_DEVICES:
|
||||
raise ValueError(
|
||||
f"Queue move: device '{device_name}' is not queue-movable."
|
||||
)
|
||||
|
||||
def _queue_action_move(self, positions: dict) -> None:
|
||||
"""positions: {device_name: absolute_target, ...}. Absolute umv only.
|
||||
|
||||
Re-checks the allow-list at execution time (not just add time, see
|
||||
_validate_action_kwargs) -- the actual safety boundary, since a value
|
||||
could have arrived via the CLI, bypassing tomo_queue_add_command's
|
||||
validation.
|
||||
"""
|
||||
args = []
|
||||
for name, target in positions.items():
|
||||
if name not in self._TOMO_QUEUE_MOVE_DEVICES:
|
||||
raise ValueError(f"Queue move: device '{name}' is not queue-movable.")
|
||||
if name not in dev:
|
||||
raise ValueError(f"Queue move: unknown device '{name}'.")
|
||||
args += [dev[name], target]
|
||||
umv(*args)
|
||||
|
||||
def optimize_idgap(self, search_range: float = 0.5) -> None:
|
||||
"""Scan idgap over +/- search_range around its current position and
|
||||
move to the peak of <signal not yet decided>.
|
||||
|
||||
STUB -- not implemented. The peak-detection signal/device is not yet
|
||||
decided (see AI_docs/TOMO_QUEUE_COMMAND_JOBS_PLAN.md section 10).
|
||||
This is the ONLY place that needs to change to make this a real
|
||||
action: the registry entry, resume policy, and (eventually) GUI
|
||||
wiring already treat it as a normal idempotent compound action, so
|
||||
dropping in the real scan-and-move-to-peak logic here (or swapping
|
||||
which device/signal it reads) is a self-contained change.
|
||||
"""
|
||||
print(
|
||||
f"optimize_idgap(search_range={search_range}): not yet implemented -- "
|
||||
"peak-detection signal not yet decided. No-op."
|
||||
def _describe_tomo_job_line(self, job: dict) -> str:
|
||||
"""flomni's tomo_queue_show() one-line summary: fov/type/step/ctime/range."""
|
||||
p = job["params"]
|
||||
return (
|
||||
f"type={p['tomo_type']} fov={p['fovx']}/{p['fovy']}um "
|
||||
f"step={p['tomo_shellstep']}um ctime={p['tomo_countingtime']}s "
|
||||
f"range={p['tomo_angle_range']}deg"
|
||||
)
|
||||
|
||||
def _describe_tomo_queue_actions(self) -> dict:
|
||||
"""Serializable description of the action registry + move allow-list.
|
||||
|
||||
Published as the ``tomo_queue_actions`` global var (see __init__) so
|
||||
the GUI (or any other client) can build its dropdowns/forms from it
|
||||
directly -- no hardcoded mirror to drift, which is exactly the
|
||||
problem the QUEUE_PARAM_NAMES mirror between this file and
|
||||
tomo_params.py already has.
|
||||
"""
|
||||
return {
|
||||
"actions": {
|
||||
name: {
|
||||
"idempotent": spec["idempotent"],
|
||||
"help": spec["help"],
|
||||
"params": spec["params"],
|
||||
}
|
||||
for name, spec in self._TOMO_QUEUE_ACTIONS.items()
|
||||
},
|
||||
"move_devices": dict(self._TOMO_QUEUE_MOVE_DEVICES),
|
||||
}
|
||||
|
||||
def tomo_queue_delete(self, *indices: int) -> None:
|
||||
"""Delete one or more jobs from the tomo queue by index.
|
||||
|
||||
Accepts any number of indices, e.g. ``flomni.tomo_queue_delete(2, 5)``
|
||||
to drop several jobs in one call. All indices are resolved against
|
||||
the queue as it currently stands and deleted highest-index-first, so
|
||||
passing several indices in any order is safe and won't shift the
|
||||
meaning of the indices still to be deleted.
|
||||
"""
|
||||
if not indices:
|
||||
print("No index given.")
|
||||
return
|
||||
for index in sorted(set(indices), reverse=True):
|
||||
job = self._tomo_queue_proxy.pop(index)
|
||||
print(f"Deleted tomo queue job #{index} ({job['label']}).")
|
||||
|
||||
def tomo_queue_move(self, index: int, new_index: int) -> None:
|
||||
"""Move a pending tomo queue job to a new position.
|
||||
|
||||
Only ``pending`` jobs can be reordered -- a ``running``,
|
||||
``incomplete``, or ``done`` job stays put. ``incomplete`` is a floor
|
||||
exactly like ``running``: tomo_queue_execute()'s pick-next loop
|
||||
always resumes whichever of the two it finds first, regardless of
|
||||
list order, so a pending job moved "ahead" of one in the list would
|
||||
visually suggest a run order that isn't real. A move is also
|
||||
refused if it would place the job at or above the position of any
|
||||
currently ``running``/``incomplete`` job. Takes effect at the next
|
||||
job boundary if the queue is executing (see tomo_queue_execute()'s
|
||||
pick-next model).
|
||||
|
||||
Args:
|
||||
index: Current index of the job to move.
|
||||
new_index: Index to move it to (list-insert semantics: the rest
|
||||
of the queue shifts to make room).
|
||||
"""
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
if not 0 <= index < len(jobs):
|
||||
raise ValueError(f"tomo_queue_move: index {index} out of range.")
|
||||
if not 0 <= new_index < len(jobs):
|
||||
raise ValueError(f"tomo_queue_move: new_index {new_index} out of range.")
|
||||
job = jobs[index]
|
||||
if job.get("status", "pending") != "pending":
|
||||
raise ValueError(
|
||||
f"tomo_queue_move: job #{index} is '{job.get('status')}' -- only "
|
||||
"pending jobs can be reordered."
|
||||
)
|
||||
floor_indices = [
|
||||
i for i, j in enumerate(jobs) if j.get("status") in ("running", "incomplete")
|
||||
]
|
||||
if floor_indices and new_index <= max(floor_indices):
|
||||
raise ValueError(
|
||||
f"tomo_queue_move: can't move to position {new_index} -- job "
|
||||
f"#{max(floor_indices)} is running/incomplete and is a hard floor "
|
||||
"for reordering."
|
||||
)
|
||||
self._tomo_queue_proxy.move(index, new_index)
|
||||
print(f"Moved tomo queue job #{index} -> #{new_index} ({job['label']}).")
|
||||
|
||||
def tomo_queue_clear(self) -> None:
|
||||
"""Empty the tomo queue."""
|
||||
self._tomo_queue_proxy.clear()
|
||||
print("Tomo queue cleared.")
|
||||
|
||||
def tomo_queue_show(self) -> list:
|
||||
"""Print and return the current tomo queue, one line per job."""
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
if not jobs:
|
||||
print("Tomo queue is empty.")
|
||||
return jobs
|
||||
for i, job in enumerate(jobs):
|
||||
if job.get("kind", "tomo") == "command":
|
||||
steps_str = " > ".join(
|
||||
f"{step['action']}{step['kwargs']}" if step["kwargs"] else step["action"]
|
||||
for step in job["steps"]
|
||||
)
|
||||
idem = "idem" if job.get("idempotent") else "NOT idem"
|
||||
print(f"[{i}] {job['status']:>10s} {job['label']:<20s} CMD {steps_str} [{idem}]")
|
||||
continue
|
||||
p = job["params"]
|
||||
print(
|
||||
f"[{i}] {job['status']:>10s} {job['label']:<20s} "
|
||||
f"type={p['tomo_type']} fov={p['fovx']}/{p['fovy']}um "
|
||||
f"step={p['tomo_shellstep']}um ctime={p['tomo_countingtime']}s "
|
||||
f"range={p['tomo_angle_range']}deg"
|
||||
)
|
||||
return jobs
|
||||
|
||||
def tomo_queue_execute(self, start_index: int = 0) -> None:
|
||||
"""Run all pending tomo queue jobs in sequence, on the current sample.
|
||||
|
||||
For each job, restores its snapshotted parameters onto the live
|
||||
properties (exactly as if set by hand) and then calls
|
||||
``tomo_scan()`` -- or, for a job that didn't run to completion last
|
||||
time, ``tomo_scan_resume()``, so it picks back up mid-scan instead
|
||||
of restarting from subtomo 1 / angle 0. Jobs already marked "done"
|
||||
are skipped, so simply calling tomo_queue_execute() again resumes
|
||||
from where it stopped (e.g. after fixing a hardware issue).
|
||||
|
||||
Pick-next execution model
|
||||
-------------------------
|
||||
The queue is re-read from its global var at the start of every job
|
||||
(not snapshotted once up front), and each job is addressed by its
|
||||
stable ``id``, not by its list index. This is what makes it safe to
|
||||
edit the queue *while it runs*: appending a job, deleting a pending
|
||||
job, or reordering the pending tail all take effect at the next job
|
||||
boundary, and a reorder/delete can never misdirect a status write to
|
||||
the wrong entry (an index-based loop could). Legacy jobs written
|
||||
before the ``id`` field existed are healed once up front by
|
||||
ensure_ids().
|
||||
|
||||
Resume-before-fresh ordering: if any job is in a resumable state, it
|
||||
is always run before any fresh "pending" job, regardless of list
|
||||
order. This preserves the single-resumable-job invariant -- tomo
|
||||
progress lives in one shared ``progress`` global var, so starting a
|
||||
fresh tomo_scan() before resuming an interrupted one would overwrite
|
||||
the very state tomo_scan_resume() needs. (Under the old strictly
|
||||
in-order loop this couldn't happen; allowing reorder makes it
|
||||
possible, hence the explicit guard.)
|
||||
|
||||
A job is considered not-yet-complete (and so gets resumed rather
|
||||
than restarted) if its status is "incomplete" (a Python exception
|
||||
was caught and execution stopped cleanly) OR "running" (the queue
|
||||
process itself died -- killed kernel, dropped connection, power
|
||||
loss, ... -- before it had a chance to record anything; in that
|
||||
case the status is whatever was last written, which is "running",
|
||||
not "incomplete", since the except block below never got to run).
|
||||
Without treating "running" as resumable too, a real crash would
|
||||
cause this method to silently restart that job from scratch on the
|
||||
next call instead of resuming it.
|
||||
|
||||
If you've already manually called flomni.tomo_scan_resume()
|
||||
yourself to recover from a crash (bypassing the queue), that scan
|
||||
is now actually finished even though the queue still has the job
|
||||
marked "incomplete" or "running" -- mark it done yourself before
|
||||
calling this again, or it will be re-run from scratch. Address the
|
||||
job by its id (indices can shift if the queue is reordered):
|
||||
flomni._tomo_queue_proxy.update_by_id(job_id, status="done")
|
||||
|
||||
Command jobs (job["kind"] == "command", added by
|
||||
tomo_queue_add_command()) are dispatched to _run_command_job()
|
||||
instead of tomo_scan()/tomo_scan_resume() -- see its docstring for
|
||||
their own resume policy. A job with no "kind" key is treated as
|
||||
"tomo" (back-compat for queues persisted before command jobs
|
||||
existed).
|
||||
|
||||
Args:
|
||||
start_index: Position in the *current* queue order below which
|
||||
fresh "pending" jobs are ignored. Defaults to 0. Resumable
|
||||
jobs are always run regardless of start_index (a crashed
|
||||
job must be recovered wherever it sits). Jobs already marked
|
||||
"done" are skipped automatically either way. Note that with
|
||||
reordering active this is applied to the queue's current
|
||||
order at each pick.
|
||||
"""
|
||||
self._tomo_queue_proxy.ensure_ids()
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
if not jobs:
|
||||
print("Tomo queue is empty.")
|
||||
return
|
||||
|
||||
runnable = [
|
||||
job
|
||||
for idx, job in enumerate(jobs)
|
||||
if job["status"] != "done"
|
||||
and (job["status"] in ("incomplete", "running") or idx >= start_index)
|
||||
]
|
||||
if not runnable:
|
||||
print("No pending tomo queue jobs to run.")
|
||||
return
|
||||
|
||||
if not self.OMNYTools.yesno(
|
||||
f"Starting automatic execution of {len(runnable)} queued tomo scan(s) on"
|
||||
f" sample '{self.sample_name}'. OK?",
|
||||
"y",
|
||||
):
|
||||
print("Aborted.")
|
||||
return
|
||||
|
||||
while True:
|
||||
jobs = self._tomo_queue_proxy.as_list()
|
||||
# Resume any in-flight/interrupted job first, before starting any
|
||||
# fresh one, so the shared progress var is never clobbered. Search
|
||||
# the whole list (not the start_index slice): a crashed job must be
|
||||
# recovered wherever it sits.
|
||||
job = next((j for j in jobs if j["status"] in ("incomplete", "running")), None)
|
||||
if job is None:
|
||||
job = next(
|
||||
(
|
||||
j
|
||||
for idx, j in enumerate(jobs)
|
||||
if idx >= start_index and j["status"] == "pending"
|
||||
),
|
||||
None,
|
||||
)
|
||||
if job is None:
|
||||
break
|
||||
|
||||
job_id = job["id"]
|
||||
label = job["label"]
|
||||
kind = job.get("kind", "tomo")
|
||||
resume_job = job["status"] in ("incomplete", "running")
|
||||
|
||||
print(f"\n=== Tomo queue job: {label} ===")
|
||||
|
||||
self._tomo_queue_proxy.update_by_id(job_id, status="running")
|
||||
try:
|
||||
if kind == "command":
|
||||
self._run_command_job(job, resume_job)
|
||||
else:
|
||||
for name, value in job["params"].items():
|
||||
setattr(self, name, value)
|
||||
if resume_job:
|
||||
self.tomo_scan_resume()
|
||||
else:
|
||||
self.tomo_scan()
|
||||
except Exception as exc:
|
||||
self._tomo_queue_proxy.update_by_id(job_id, status="incomplete")
|
||||
print(f"Tomo queue job '{label}' did not complete: {exc}")
|
||||
print(
|
||||
"Queue paused. Fix the issue and call tomo_queue_execute() "
|
||||
"again to resume from this job."
|
||||
)
|
||||
raise
|
||||
self._tomo_queue_proxy.update_by_id(job_id, status="done")
|
||||
|
||||
print("\nTomo queue finished -- all jobs done.")
|
||||
|
||||
def _run_command_job(self, job: dict, resume_job: bool) -> None:
|
||||
"""Run (or resume) a "command" job's steps in order.
|
||||
|
||||
No per-step resume cursor -- deliberately: tracking which step of a
|
||||
multi-step job completed means more persisted state and more edge
|
||||
cases than this warrants. Instead the job-level "idempotent" flag
|
||||
decides:
|
||||
|
||||
- idempotent: silently re-run every step from the top. Harmless --
|
||||
absolute moves and in/out actions re-run cleanly.
|
||||
- not idempotent: prompt the operator rather than silently re-run,
|
||||
which could be actively wrong for a relative/stateful step. "no"
|
||||
leaves the steps un-run; the caller (tomo_queue_execute) still
|
||||
marks the job "done" either way, matching the same edge case tomo
|
||||
jobs have -- a job that actually finished but crashed before its
|
||||
status hit "done".
|
||||
"""
|
||||
label = job["label"]
|
||||
if resume_job:
|
||||
if job.get("idempotent"):
|
||||
print(
|
||||
f"Command job '{label}' may have partially run before the crash "
|
||||
"-- idempotent, re-running from the top."
|
||||
)
|
||||
else:
|
||||
if not self.OMNYTools.yesno(
|
||||
f"Command job '{label}' may have partially run before the crash "
|
||||
"(not every step is idempotent). Re-run it from the top?",
|
||||
"n",
|
||||
):
|
||||
print(f"Skipping '{label}' -- it will be marked done without re-running.")
|
||||
return
|
||||
for step in job["steps"]:
|
||||
spec = self._TOMO_QUEUE_ACTIONS[step["action"]]
|
||||
method = getattr(self, spec["func"])
|
||||
method(**step["kwargs"])
|
||||
|
||||
def rt_off(self):
|
||||
dev.rtx.enabled = False
|
||||
dev.rty.enabled = False
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -75,6 +75,7 @@ The sample fine alignment can be obtained using ptychography. For this a short l
|
||||
|
||||
* `lamni.tomo_fovx/y_offset=value` [mm] will shift the field of view. Perform this adjustment from projections collected at **lsamrot 0 degrees**. This shift will rotate. In contrast the manual shift will be a constant shift, identical at all angles.
|
||||
|
||||
(user.ptychography.lamni.laminography)=
|
||||
### Laminography scan
|
||||
|
||||
Start the laminography scan by
|
||||
@@ -85,6 +86,190 @@ Start the laminography scan by
|
||||
`lamni.tomo_reconstruct()`
|
||||
3. `lamni.tomo_scan()` to start the laminography scan
|
||||
|
||||
During the scan, a live progress report is printed (subtomogram/projection counters and an estimated time of completion). Gaps significantly longer than a normal acquisition cycle (e.g. a beamline-down interruption, or a crash followed by a restart) are detected automatically from the time between consecutive projections and excluded from the time estimate; the total time lost to such gaps is printed once the scan finishes.
|
||||
|
||||
Three angular sampling modes are implemented, same as flOMNI, but LamNI always scans the full 360 degrees — laminography's tilted sample geometry does not have the symmetry that lets flOMNI's 180-degree mode work, so there is no angular-range choice to make:
|
||||
|
||||
| tomography mode | parameters and defaults |
|
||||
| --- | --- |
|
||||
| 8 sub-tomograms | subtomo_start=1, start_angle=None |
|
||||
| Golden ratio tomography (sorted in bunches) | projection_number=None |
|
||||
| Equally spaced with golden starting angle | projection_number=None |
|
||||
|
||||
The parameters above can be used to __restart an interrupted acquisition__ manually, or - more conveniently - by running
|
||||
`lamni.tomo_scan_resume()`
|
||||
which reads the last recorded progress and resumes automatically at the exact point (subtomogram/angle, or projection for the golden ratio modes) the scan was interrupted at, without needing to look up the values by hand. When running from the tomo scan queuing system use `lamni.tomo_queue_execute()` instead!
|
||||
|
||||
In case of eight equally spaced sub-tomograms, an individual sub-tomogram can be scanned by `lamni.sub_tomo_scan(subtomo_number, start_angle)`. If the start angle is not specified, it is computed depending on `subtomo_number`, ranging from 1 to 8.
|
||||
|
||||
#### Queueing multiple scans
|
||||
|
||||
Several tomo parameter sets can be queued and run sequentially on the same sample, without having to start each one by hand.
|
||||
|
||||
| command | explanation |
|
||||
| --- | --- |
|
||||
| `lamni.tomo_queue_add(label=None)` | Snapshot the currently set tomo parameters and add them as a new job to the queue. Returns the job's index. |
|
||||
| `lamni.tomo_queue_show()` | Print and return the current queue, with status per job. |
|
||||
| `lamni.tomo_queue_delete(*indices)` | Delete one or more jobs by index. |
|
||||
| `lamni.tomo_queue_clear()` | Empty the queue. |
|
||||
| `lamni.tomo_queue_execute(start_index=0)` | Run all pending jobs in sequence, on the current sample. |
|
||||
|
||||
The queue is persisted (it survives a BEC client restart). Each job's status is one of `pending`, `running`, `incomplete`, or `done`. A job that did not run to completion (an exception was caught, or the BEC client itself crashed mid-scan) is automatically resumed - rather than restarted - the next time `lamni.tomo_queue_execute()` is called.
|
||||
|
||||
If the process running the queue dies hard enough that it never gets to write a status update at all (a kernel restart, a Ctrl-C during a blocking prompt) a job can be left stuck showing `running` with nothing actually running. `lamni.tomo_queue_delete(index)` has no status guard and works on a stuck row regardless; you can also fix the status directly with `lamni._tomo_queue_proxy.update_by_id(job_id, status="incomplete")` (to make it resumable) or `status="done"` (to skip it). The GUI's Delete/Clear normally refuse to touch a `running` row, but detect a stale one (no recent scan heartbeat) and offer to proceed anyway with an explicit confirmation instead of blocking forever.
|
||||
|
||||
Example:
|
||||
```
|
||||
lamni.tomo_parameters() # set up parameter set #1
|
||||
lamni.tomo_queue_add("fast overview")
|
||||
lamni.tomo_parameters() # set up parameter set #2
|
||||
lamni.tomo_queue_add("hires scan")
|
||||
lamni.tomo_queue_show()
|
||||
lamni.tomo_queue_execute() # runs both, in order, on this sample
|
||||
```
|
||||
|
||||
**GUI: reusing an earlier job's settings.** In the ☰ Queue control… dialog, select a
|
||||
single tomo job (not a command job — those have no scan parameters) and click
|
||||
**"Load into editor"** to pull its saved settings into the params panel's editor,
|
||||
as if you'd just clicked Edit. Nothing is written yet — review or tweak the fields,
|
||||
then Submit (writes live, blocked while the beamline is busy) or "Add to queue" (always
|
||||
allowed) as usual. If you already had an edit in progress, it asks before discarding it.
|
||||
|
||||
**GUI: two different "Add to queue" buttons.** The params panel's own **"Add to
|
||||
queue"** (visible in edit mode) queues whatever you've typed, unsubmitted. The ☰
|
||||
Queue control… window's **"Add current params to queue"** is a different button on a
|
||||
different window — it always queues the *live* parameters, regardless of any edit
|
||||
open in the panel. If you have an unsaved edit open and click the queue window's
|
||||
button instead of the panel's, it warns and names the correct one before proceeding,
|
||||
since it would otherwise queue your last-submitted values, not what you just typed.
|
||||
|
||||
**GUI: "Duplicate selected".** Appends an exact copy of the selected job (any status —
|
||||
pending, running, incomplete, or done, and either a tomo or a command job) to the end
|
||||
of the queue, labeled with `_dup` appended, status reset to `pending`. Useful for
|
||||
re-running a job with the same settings without re-typing them, or as a starting point
|
||||
to tweak via "Load into editor".
|
||||
|
||||
#### Command jobs — reconfiguring the beamline between scans
|
||||
|
||||
In addition to tomogram jobs, the same queue can hold **command jobs**: an ordered
|
||||
list of beamline reconfiguration steps (move a device, ...) that run instead of a
|
||||
scan. This is what lets one queue express *"tomogram A, then reconfigure, then
|
||||
tomogram B"* unattended, e.g. change energy and re-peak the undulator gap between
|
||||
two tomograms on the same sample.
|
||||
|
||||
`lamni.tomo_queue_add_command(steps, label=None, idempotent=None)`
|
||||
|
||||
- `steps`: a single `{"action": ..., "kwargs": {...}}` dict, or a list of them run in
|
||||
sequence within that one job.
|
||||
- `label`: optional name shown by `tomo_queue_show()`, same as for `tomo_queue_add()`.
|
||||
- `idempotent`: normally inferred (safe to blindly re-run after a crash only if every
|
||||
step is); override explicitly if needed.
|
||||
|
||||
Only actions from a fixed, reviewed registry can be queued — not arbitrary code:
|
||||
|
||||
| action | does | parameters |
|
||||
| --- | --- | --- |
|
||||
| `move` | Move device(s) to absolute position(s). Only devices on the allow-list below can be targeted. | `positions`: `{device: target}`, e.g. `{"mokev": 6.2}` |
|
||||
| `optimize_idgap` | Scan the undulator gap over a range and move to the peak. **Not yet implemented (no-op stub).** | `search_range`: mm, default 0.5, range 0–2 |
|
||||
|
||||
Devices allowed for `move`: `mokev` (energy, keV), `idgap` (undulator gap, mm) — the
|
||||
same beamline-wide devices as flOMNI's queue. A `move` naming any other device is
|
||||
rejected, both when the job is added and again when it actually runs.
|
||||
|
||||
Example — change energy, then re-peak idgap, before the next laminogram:
|
||||
```
|
||||
lamni.tomo_queue_add_command(
|
||||
[{"action": "move", "kwargs": {"positions": {"mokev": 6.2}}},
|
||||
{"action": "optimize_idgap", "kwargs": {"search_range": 0.5}}],
|
||||
label="reconfigure to 6.2 keV",
|
||||
)
|
||||
```
|
||||
|
||||
`tomo_queue_show()` lists command jobs alongside tomogram jobs, e.g.:
|
||||
```
|
||||
[2] pending reconfigure to 6.2 keV CMD move{'positions': {'mokev': 6.2}} > optimize_idgap{'search_range': 0.5} [idem]
|
||||
```
|
||||
|
||||
If a command job is interrupted by a crash, there is no per-step resume point — the
|
||||
whole job is either safe to redo from the top or it isn't:
|
||||
- If every step is idempotent (the usual case — an absolute move is harmless to
|
||||
repeat), `lamni.tomo_queue_execute()` silently re-runs the whole job from the top.
|
||||
- If any step is not idempotent, you are asked whether to re-run the job from the top
|
||||
or mark it done as-is.
|
||||
|
||||
#### Custom behavior at each projection angle
|
||||
|
||||
By default, every projection in a laminography scan is a single ptychography scan at
|
||||
that angle. For cases that need something more — e.g. record a projection, move a
|
||||
polarizer in, record again, move it back out, at every angle — write a small Python
|
||||
function and register it as an **at_each_angle hook**. Once registered, it can be
|
||||
selected per tomo-queue job, so the whole modulated laminogram runs unattended along
|
||||
with any other queued jobs.
|
||||
|
||||
A hook is a function `func(lamni, angle)`, called once per projection angle instead
|
||||
of the normal acquisition:
|
||||
```python
|
||||
def polarizer_modulation(lamni, angle):
|
||||
lamni.tomo_scan_projection(angle)
|
||||
umv(dev.polarizer, "in")
|
||||
lamni.tomo_scan_projection(angle)
|
||||
umv(dev.polarizer, "out")
|
||||
```
|
||||
|
||||
| command | explanation |
|
||||
| --- | --- |
|
||||
| `lamni.register_at_each_angle_hook("name", func)` | Load a hook function under `name`. |
|
||||
| `lamni.at_each_angle_hook = "name"` | Activate it — the next tomo-queue job added will use it. |
|
||||
| `lamni.at_each_angle_hook = None` | Deactivate it — back to normal projections for the next job. |
|
||||
| `lamni.list_at_each_angle_hooks()` | Print the names of all currently registered hooks (the active one is marked). |
|
||||
| `lamni.unregister_at_each_angle_hook("name")` | Remove a hook by name. |
|
||||
|
||||
Example — queue a polarizer-modulated laminogram, then a normal one:
|
||||
```python
|
||||
lamni.register_at_each_angle_hook("polarizer_mod", polarizer_modulation)
|
||||
|
||||
lamni.tomo_parameters() # set up the scan parameters as usual
|
||||
lamni.at_each_angle_hook = "polarizer_mod" # activate the hook
|
||||
lamni.tomo_queue_add("polarizer run")
|
||||
|
||||
lamni.at_each_angle_hook = None # back to normal -- do not skip this!
|
||||
lamni.tomo_queue_add("plain follow-up scan")
|
||||
|
||||
lamni.tomo_queue_execute() # runs both, hook active only for the first
|
||||
```
|
||||
**Do not forget to reset `lamni.at_each_angle_hook = None`** before adding a job
|
||||
that should run normally — each queued job remembers whatever `at_each_angle_hook`
|
||||
was set to at the moment it was added (like every other tomo parameter), so a job
|
||||
added right after a hook-using one without resetting it first will silently run with
|
||||
that hook still active.
|
||||
|
||||
Registered hooks are **session-only** — they live in memory and do not survive a
|
||||
kernel restart. If `tomo_queue_execute()` reaches a job whose `at_each_angle_hook`
|
||||
isn't registered in the current session (e.g. after restarting BEC), it stops with a
|
||||
clear error rather than silently running a plain scan; re-run
|
||||
`register_at_each_angle_hook()` for that hook, then call `tomo_queue_execute()`
|
||||
again to resume.
|
||||
|
||||
**Editing a hook after registering it:** `register_at_each_angle_hook()` stores
|
||||
whichever function object you pass it at that moment — it is not a live link to the
|
||||
function's name. If you edit the function's source and re-run the `def` in your
|
||||
session but do **not** call `register_at_each_angle_hook()` again, the *old* version
|
||||
keeps running. Always re-register after an edit.
|
||||
|
||||
**GUI:** the tomo parameters panel has an "At-each-angle hook" dropdown, listing
|
||||
whatever is currently registered from the CLI (`register_at_each_angle_hook()`) —
|
||||
the GUI process can't register hooks itself, only select one by name for the job
|
||||
you're about to submit or add to the queue. Pick "None (default)" for a normal scan.
|
||||
If a job's stored hook isn't currently registered anywhere, it still shows up in the
|
||||
dropdown labeled "(not registered in this session)" so it's never silently hidden.
|
||||
|
||||
**Unregistering does not clear `at_each_angle_hook`.** `unregister_at_each_angle_hook()`
|
||||
only removes the function from this session's registry — if `at_each_angle_hook` (or a
|
||||
queued job) still names it, that reference is untouched; it just becomes an
|
||||
unregistered name, flagged as such wherever it's shown (CLI, GUI dropdown). Set
|
||||
`lamni.at_each_angle_hook = None` (or the GUI dropdown's "None (default)") explicitly
|
||||
if you want to actually clear the selection, not just remove the hook it points to.
|
||||
|
||||
### Tips and Tricks
|
||||
|
||||
#### Reset corrections
|
||||
|
||||
@@ -0,0 +1,264 @@
|
||||
"""Regression test for LamNI.sub_tomo_scan()'s actual angle generation.
|
||||
|
||||
Mirrors test_flomni_tomo_angles.py's style of testing angle math directly,
|
||||
without a live BEC session. Guards against two bugs fixed this session:
|
||||
|
||||
1. A full 360-degree sweep's closing angle (start_angle + 360) is the same
|
||||
physical angle as its start, so sub_tomo_scan() must generate exactly N
|
||||
unique angles per sub-tomogram (endpoint=False), not N+1.
|
||||
2. The inter-sub-tomogram phase offset must be a fraction of the ACHIEVABLE
|
||||
step (post int() truncation), not the raw configured tomo_angle_stepsize
|
||||
-- otherwise sub-tomograms combined in pairs/quads/all-8 land on
|
||||
unevenly-spaced angles instead of the intended progressively finer, even
|
||||
grid at every combination level (mirrors Flomni._subtomo_angle_plan()'s
|
||||
same requirement).
|
||||
"""
|
||||
|
||||
import builtins
|
||||
import types
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI, _ProgressProxy
|
||||
|
||||
STEPSIZES = [10.0, 7.0, 25.0, 12.5]
|
||||
|
||||
|
||||
class FakeClient:
|
||||
"""Minimal in-memory stand-in for BEC's global-var store."""
|
||||
|
||||
def __init__(self):
|
||||
self._vars = {}
|
||||
|
||||
def get_global_var(self, key):
|
||||
return self._vars.get(key)
|
||||
|
||||
def set_global_var(self, key, value):
|
||||
self._vars[key] = value
|
||||
|
||||
|
||||
def make_lamni(tomo_angle_stepsize: float) -> LamNI:
|
||||
"""Bare LamNI instance with only what sub_tomo_scan() needs to run its
|
||||
angle-generation logic, bypassing __init__'s heavy side effects."""
|
||||
obj = object.__new__(LamNI)
|
||||
obj.client = FakeClient()
|
||||
obj._progress_proxy = _ProgressProxy(obj.client)
|
||||
obj.tomo_angle_stepsize = tomo_angle_stepsize
|
||||
obj.tomo_id = -1
|
||||
obj._write_subtomo_to_scilog = lambda subtomo_number: None
|
||||
return obj
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stepsize", STEPSIZES)
|
||||
@pytest.mark.parametrize("subtomo_number", range(1, 9))
|
||||
def test_sub_tomo_scan_generates_no_duplicate_angles(stepsize, subtomo_number):
|
||||
lamni = make_lamni(stepsize)
|
||||
recorded = []
|
||||
lamni._tomo_scan_at_angle = lambda angle, subtomo: recorded.append(float(angle))
|
||||
|
||||
lamni.sub_tomo_scan(subtomo_number)
|
||||
|
||||
N, _achievable_step, _total = lamni._tomo_type1_actual_grid()
|
||||
assert len(recorded) == N, "sub_tomo_scan() must generate exactly N angles, not N+1"
|
||||
unique_mod_360 = {round(a % 360, 6) for a in recorded}
|
||||
assert len(unique_mod_360) == N, "sub_tomo_scan() must not re-measure its own start angle"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stepsize", STEPSIZES)
|
||||
def test_sub_tomo_scan_projection_count_matches_actual_grid(stepsize):
|
||||
"""progress['subtomo_total_projections']/['total_projections'] must
|
||||
match what actually ran, not a pre-truncation float estimate."""
|
||||
lamni = make_lamni(stepsize)
|
||||
lamni._tomo_scan_at_angle = lambda angle, subtomo: None
|
||||
|
||||
lamni.sub_tomo_scan(1)
|
||||
|
||||
N, _achievable_step, total = lamni._tomo_type1_actual_grid()
|
||||
assert lamni.progress["subtomo_total_projections"] == N
|
||||
assert lamni.progress["total_projections"] == total
|
||||
|
||||
|
||||
def _all_subtomo_angles(stepsize: float) -> dict[int, list[float]]:
|
||||
"""Run sub_tomo_scan() for all 8 sub-tomograms (real code path, not a
|
||||
reimplementation) and return each one's acquired angles, in order."""
|
||||
result = {}
|
||||
for n in range(1, 9):
|
||||
lamni = make_lamni(stepsize)
|
||||
recorded = []
|
||||
lamni._tomo_scan_at_angle = lambda angle, subtomo, _r=recorded: _r.append(float(angle))
|
||||
lamni.sub_tomo_scan(n)
|
||||
result[n] = recorded
|
||||
return result
|
||||
|
||||
|
||||
def _assert_equally_spaced(angles, label: str) -> None:
|
||||
sorted_angles = np.sort(np.asarray(angles) % 360)
|
||||
diffs = np.diff(np.concatenate([sorted_angles, [sorted_angles[0] + 360]]))
|
||||
assert np.allclose(
|
||||
diffs, diffs[0], atol=1e-6
|
||||
), f"{label}: angles are not equally spaced when combined -- got spacings {diffs}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stepsize", STEPSIZES)
|
||||
def test_subtomo_pairs_are_equally_spaced_when_combined(stepsize):
|
||||
"""(1,2), (3,4), (5,6), (7,8) each combine into one evenly-spaced,
|
||||
doubled-resolution grid -- this is the whole point of the phase-offset
|
||||
interlacing scheme (bit-reversal table), and silently breaks if the
|
||||
phase is computed from the wrong step (see module docstring)."""
|
||||
angles = _all_subtomo_angles(stepsize)
|
||||
for a, b in [(1, 2), (3, 4), (5, 6), (7, 8)]:
|
||||
_assert_equally_spaced(angles[a] + angles[b], f"pair ({a},{b})")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stepsize", STEPSIZES)
|
||||
def test_subtomo_quads_are_equally_spaced_when_combined(stepsize):
|
||||
"""(1,2,3,4) and (5,6,7,8) each combine into one evenly-spaced,
|
||||
quadrupled-resolution grid."""
|
||||
angles = _all_subtomo_angles(stepsize)
|
||||
for quad in [(1, 2, 3, 4), (5, 6, 7, 8)]:
|
||||
combined = sum((angles[n] for n in quad), [])
|
||||
_assert_equally_spaced(combined, f"quad {quad}")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stepsize", STEPSIZES)
|
||||
def test_all_eight_subtomos_are_equally_spaced_when_combined(stepsize):
|
||||
"""The full combined set of all 8 sub-tomograms is the finest,
|
||||
evenly-spaced grid -- the end goal of the interlacing scheme."""
|
||||
angles = _all_subtomo_angles(stepsize)
|
||||
combined = sum((angles[n] for n in range(1, 9)), [])
|
||||
_assert_equally_spaced(combined, "full (all 8)")
|
||||
|
||||
|
||||
# ── tomo_scan()'s new-scan account handling ─────────────────────────────────
|
||||
#
|
||||
# bec.active_account used to be passed through .decode() (as if it were
|
||||
# bytes) -- it's a plain str, so this raised AttributeError on every real
|
||||
# tomo_scan() call. Flomni's equivalent code never had this bug (no
|
||||
# .decode() call at all) and already guards an empty active_account (e.g. a
|
||||
# dev/sim session not logged into a real e-account) by skipping sample-
|
||||
# database registration instead of trying to register under "". Both fixed
|
||||
# to match.
|
||||
|
||||
|
||||
class _FakeContextManager:
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc_info):
|
||||
return False
|
||||
|
||||
|
||||
class _FakeScans:
|
||||
dataset_id_on_hold = _FakeContextManager()
|
||||
|
||||
|
||||
def make_lamni_for_tomo_scan(tomo_angle_stepsize: float, active_account: str) -> LamNI:
|
||||
"""Bare LamNI instance with tomo_scan()'s "new scan" branch reachable,
|
||||
everything downstream of it stubbed out (sample database, PDF report,
|
||||
per-sub-tomogram scanning) so only the account-handling logic under
|
||||
test actually runs."""
|
||||
obj = make_lamni(tomo_angle_stepsize)
|
||||
obj.tomo_type = 1
|
||||
obj.special_angles = []
|
||||
obj.write_pdf_report = lambda: None
|
||||
obj.sub_tomo_scan = lambda subtomo_number, start_angle=None: None
|
||||
obj._print_progress = lambda: None
|
||||
obj._format_duration = lambda seconds: "0s"
|
||||
obj.lamnigui_show_progress = lambda: None
|
||||
builtins.__dict__["bec"] = types.SimpleNamespace(
|
||||
active_account=active_account, queue=types.SimpleNamespace(next_scan_number=1)
|
||||
)
|
||||
builtins.__dict__["scans"] = _FakeScans()
|
||||
return obj
|
||||
|
||||
|
||||
def test_tomo_scan_skips_sample_database_when_no_active_account():
|
||||
"""Empty active_account (e.g. a dev/sim session) must not crash and must
|
||||
not try to register a sample -- tomo_id falls back to 0, mirroring
|
||||
Flomni.tomo_scan()'s identical guard."""
|
||||
lamni = make_lamni_for_tomo_scan(45.0, active_account="")
|
||||
lamni.add_sample_database = lambda *a, **k: (_ for _ in ()).throw(
|
||||
AssertionError("add_sample_database must not be called with no active account")
|
||||
)
|
||||
|
||||
lamni.tomo_scan()
|
||||
|
||||
assert lamni.tomo_id == 0
|
||||
|
||||
|
||||
def test_tomo_scan_registers_sample_with_plain_string_account():
|
||||
"""A real active_account must be passed through as-is -- calling
|
||||
.decode() on it (a plain str, not bytes) raises AttributeError."""
|
||||
lamni = make_lamni_for_tomo_scan(45.0, active_account="e12345")
|
||||
recorded = {}
|
||||
|
||||
def _fake_add_sample_database(samplename, date, eaccount, scan_number, setup, info, user):
|
||||
recorded["eaccount"] = eaccount
|
||||
return 42
|
||||
|
||||
lamni.add_sample_database = _fake_add_sample_database
|
||||
|
||||
lamni.tomo_scan()
|
||||
|
||||
assert recorded["eaccount"] == "e12345"
|
||||
assert lamni.tomo_id == 42
|
||||
|
||||
|
||||
def test_tomo_scan_shows_progress_gui():
|
||||
"""tomo_scan() must open/show the progress GUI unconditionally at the
|
||||
start of every scan, mirroring Flomni.tomo_scan()'s
|
||||
self.flomnigui_show_progress() call -- lamni.py never had this at all."""
|
||||
lamni = make_lamni_for_tomo_scan(45.0, active_account="")
|
||||
lamni.add_sample_database = lambda *a, **k: 0
|
||||
calls = []
|
||||
lamni.lamnigui_show_progress = lambda: calls.append("shown")
|
||||
|
||||
lamni.tomo_scan()
|
||||
|
||||
assert calls == ["shown"]
|
||||
|
||||
|
||||
def test_tomo_scan_clears_heartbeat_on_normal_completion():
|
||||
"""The busy-detector heartbeat must be cleared as soon as the scan
|
||||
finishes, not left for the next poll to time out (120s) before the GUI
|
||||
stops showing "beamline busy" for an already-finished scan -- mirrors
|
||||
Flomni.tomo_scan()'s identical try/finally fix.
|
||||
|
||||
sub_tomo_scan() is stubbed to set a fresh heartbeat before returning,
|
||||
the same way the real _tomo_scan_at_angle() does at the start of every
|
||||
projection -- without that, this test would pass even without the fix,
|
||||
since tomo_scan()'s own "new scan" branch already resets the heartbeat
|
||||
to None *before* the scan body runs.
|
||||
"""
|
||||
lamni = make_lamni_for_tomo_scan(45.0, active_account="")
|
||||
lamni.add_sample_database = lambda *a, **k: 0
|
||||
|
||||
def _fake_sub_tomo_scan(subtomo_number, start_angle=None):
|
||||
lamni.progress["heartbeat"] = "2026-01-01T00:00:00"
|
||||
|
||||
lamni.sub_tomo_scan = _fake_sub_tomo_scan
|
||||
|
||||
lamni.tomo_scan()
|
||||
|
||||
assert lamni.progress["heartbeat"] is None
|
||||
|
||||
|
||||
def test_tomo_scan_clears_heartbeat_even_if_scan_raises():
|
||||
"""The heartbeat must be cleared on every exit path, including a
|
||||
mid-scan exception -- otherwise a crashed scan looks permanently "busy"
|
||||
to the GUI until the 120s staleness timeout, or forever if polled more
|
||||
often than that."""
|
||||
lamni = make_lamni_for_tomo_scan(45.0, active_account="")
|
||||
lamni.add_sample_database = lambda *a, **k: 0
|
||||
|
||||
def _boom(subtomo_number, start_angle=None):
|
||||
lamni.progress["heartbeat"] = "2026-01-01T00:00:00"
|
||||
raise RuntimeError("simulated crash mid-scan")
|
||||
|
||||
lamni.sub_tomo_scan = _boom
|
||||
|
||||
with pytest.raises(RuntimeError):
|
||||
lamni.tomo_scan()
|
||||
|
||||
assert lamni.progress["heartbeat"] is None
|
||||
@@ -0,0 +1,167 @@
|
||||
"""Regression test for TomoParamsWidget's lamni-specific projection-count
|
||||
math (_lamni_compute_type1/_lamni_requested_to_stepsize), mirroring
|
||||
test_tomo_params_widget_math.py's style for flomni.
|
||||
|
||||
Lamni's formula is NOT the same as flomni's: LamNI.sub_tomo_scan() sweeps a
|
||||
plain 360 degrees per sub-tomogram (vs. flomni's fixed-180-base,
|
||||
mode-independent _compute_type1/_requested_to_stepsize, which these tests
|
||||
must not touch -- see that file's own docstring on why they're pinned by
|
||||
name/signature) -- but is otherwise the same shape: N = int(360/stepsize)
|
||||
unique angles per sub-tomogram, N*8 total. (sub_tomo_scan() used to also
|
||||
acquire a duplicate closing angle per sub-tomogram -- start_angle+360 is the
|
||||
same physical angle as start_angle -- fixed via endpoint=False; see
|
||||
lamni.py's _tomo_type1_actual_grid().)
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from csaxs_bec.bec_ipython_client.plugins.flomni.flomni import Flomni
|
||||
from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI
|
||||
from csaxs_bec.bec_widgets.widgets.tomo_params.tomo_params import (
|
||||
DEFAULTS,
|
||||
LAMNI_QUEUE_PARAM_NAMES,
|
||||
QUEUE_PARAM_NAMES,
|
||||
SETUP_PROFILES,
|
||||
_compute_type1,
|
||||
_lamni_compute_type1,
|
||||
_lamni_get_tomo_fov_offset,
|
||||
_lamni_requested_to_stepsize,
|
||||
_lamni_set_tomo_fov_offset,
|
||||
_requested_to_stepsize,
|
||||
)
|
||||
|
||||
STEPSIZES = [10.0, 7.0, 25.0, 12.5]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stepsize", STEPSIZES)
|
||||
def test_lamni_compute_type1_matches_lamni_subtomo_scan(stepsize):
|
||||
"""Reference formula mirrors LamNI.sub_tomo_scan()'s exact (fixed) grid:
|
||||
N = int(360/stepsize) unique angles per sub-tomogram, N*8 total."""
|
||||
N = int(360.0 / stepsize)
|
||||
expected_total = N * 8
|
||||
expected_step = 360.0 / N
|
||||
|
||||
actual_total, achievable_step, _ = _lamni_compute_type1(360, stepsize)
|
||||
assert actual_total == expected_total
|
||||
assert achievable_step == pytest.approx(expected_step)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("requested_total", [32, 64, 160, 320])
|
||||
def test_lamni_requested_to_stepsize_round_trips(requested_total):
|
||||
"""requested_total must already be a multiple of 8 for the round trip
|
||||
to land exactly back on it -- these were chosen as N=4,8,20,40."""
|
||||
stepsize = _lamni_requested_to_stepsize(360, requested_total)
|
||||
actual_total, _, _ = _lamni_compute_type1(360, stepsize)
|
||||
assert actual_total == requested_total
|
||||
|
||||
|
||||
def test_lamni_formula_differs_from_flomni_formula():
|
||||
"""Cheap insurance against accidentally aliasing the wrong function into
|
||||
a SETUP_PROFILES entry -- lamni's math must not silently match flomni's."""
|
||||
for stepsize in STEPSIZES:
|
||||
flomni_total, flomni_step, _ = _compute_type1(180, stepsize)
|
||||
lamni_total, lamni_step, _ = _lamni_compute_type1(360, stepsize)
|
||||
assert (flomni_total, flomni_step) != (lamni_total, lamni_step)
|
||||
|
||||
for requested in (24, 48, 96, 144):
|
||||
assert _requested_to_stepsize(180, requested) != _lamni_requested_to_stepsize(
|
||||
360, requested
|
||||
)
|
||||
|
||||
|
||||
def test_setup_profiles_param_names_match_cli_classes():
|
||||
"""Guards against the known mirror-drift risk: SETUP_PROFILES'
|
||||
param_names must stay in sync with each CLI class's own
|
||||
_TOMO_SCAN_PARAM_NAMES (there is no shared import between bec_widgets
|
||||
and the ipython-client plugins, so nothing else catches this)."""
|
||||
assert set(SETUP_PROFILES["flomni"]["param_names"]) == set(Flomni._TOMO_SCAN_PARAM_NAMES)
|
||||
assert set(SETUP_PROFILES["lamni"]["param_names"]) == set(LamNI._TOMO_SCAN_PARAM_NAMES)
|
||||
assert set(QUEUE_PARAM_NAMES) == set(Flomni._TOMO_SCAN_PARAM_NAMES)
|
||||
assert set(LAMNI_QUEUE_PARAM_NAMES) == set(LamNI._TOMO_SCAN_PARAM_NAMES)
|
||||
|
||||
|
||||
def test_setup_profiles_defaults_cover_all_param_names():
|
||||
for setup, profile in SETUP_PROFILES.items():
|
||||
missing = set(profile["param_names"]) - set(profile["defaults"])
|
||||
assert not missing, f"{setup} profile is missing defaults for: {missing}"
|
||||
|
||||
|
||||
class _FakeClient:
|
||||
"""Minimal in-memory stand-in for BEC's global-var store."""
|
||||
|
||||
def __init__(self):
|
||||
self._vars = {}
|
||||
|
||||
def get_global_var(self, key):
|
||||
return self._vars.get(key)
|
||||
|
||||
def set_global_var(self, key, value):
|
||||
self._vars[key] = value
|
||||
|
||||
|
||||
def test_lamni_fov_offset_round_trip_does_not_clobber_other_axis():
|
||||
"""tomo_fov_offset packs both axes into one global var ([x_um, y_um]) --
|
||||
writing one axis must read-modify-write, not silently reset the other
|
||||
(this is exactly the kind of bug a naive per-key implementation would
|
||||
introduce)."""
|
||||
client = _FakeClient()
|
||||
|
||||
_lamni_set_tomo_fov_offset(client, "x", 1.5)
|
||||
assert _lamni_get_tomo_fov_offset(client, "x") == pytest.approx(1.5)
|
||||
assert _lamni_get_tomo_fov_offset(client, "y") == pytest.approx(0.0)
|
||||
|
||||
_lamni_set_tomo_fov_offset(client, "y", -2.25)
|
||||
assert _lamni_get_tomo_fov_offset(client, "x") == pytest.approx(1.5)
|
||||
assert _lamni_get_tomo_fov_offset(client, "y") == pytest.approx(-2.25)
|
||||
|
||||
assert client.get_global_var("tomo_fov_offset") == [1500.0, -2250.0]
|
||||
|
||||
|
||||
def test_lamni_fov_offset_defaults_to_zero_when_unset():
|
||||
client = _FakeClient()
|
||||
assert _lamni_get_tomo_fov_offset(client, "x") == 0.0
|
||||
assert _lamni_get_tomo_fov_offset(client, "y") == 0.0
|
||||
|
||||
|
||||
def test_lamni_offset_fields_excluded_from_param_names():
|
||||
"""tomo_fovx_offset/tomo_fovy_offset are alignment values, shown in the
|
||||
GUI but deliberately excluded from queue job snapshots -- must never
|
||||
appear in param_names."""
|
||||
offset_keys = {key for key, *_ in SETUP_PROFILES["lamni"]["offset_fields"]}
|
||||
assert offset_keys == {"tomo_fovx_offset", "tomo_fovy_offset"}
|
||||
assert offset_keys.isdisjoint(SETUP_PROFILES["lamni"]["param_names"])
|
||||
assert not SETUP_PROFILES["flomni"]["offset_fields"]
|
||||
|
||||
|
||||
def test_flomni_field_order_matches_pre_reorder_layout():
|
||||
"""Regression guard: the field_order dispatch must reproduce flomni's
|
||||
exact, unchanged field build order (this task only reordered lamni)."""
|
||||
assert SETUP_PROFILES["flomni"]["field_order"] == [
|
||||
"tomo_countingtime",
|
||||
"tomo_shellstep",
|
||||
"fov",
|
||||
"stitch",
|
||||
"tomo_stitch_overlap",
|
||||
"ptycho_reconstruct_foldername",
|
||||
"manual_shift",
|
||||
"frames_per_trigger",
|
||||
"single_point",
|
||||
"at_each_angle_hook",
|
||||
]
|
||||
|
||||
|
||||
def test_lamni_field_order_matches_tomo_parameters_cli():
|
||||
"""Mirrors lamni.tomo_parameters()'s exact print/edit order
|
||||
(lamni.py:1136-1153)."""
|
||||
assert SETUP_PROFILES["lamni"]["field_order"] == [
|
||||
"tomo_countingtime",
|
||||
"tomo_shellstep",
|
||||
"piezo_range",
|
||||
"stitch",
|
||||
"tomo_stitch_overlap",
|
||||
"fov",
|
||||
"ptycho_reconstruct_foldername",
|
||||
"frames_per_trigger",
|
||||
"offsets",
|
||||
"at_each_angle_hook",
|
||||
]
|
||||
@@ -0,0 +1,206 @@
|
||||
"""Unit tests for the tomo-queue / at-each-angle-hook backend ported to
|
||||
LamNI from flomni (see TomoQueueMixin in
|
||||
csaxs_bec/bec_ipython_client/plugins/OMNY_shared/tomo_queue_mixin.py).
|
||||
|
||||
Builds a LamNI instance without running __init__'s heavy side effects
|
||||
(device init, reconstructor, webpage generator, ...) -- only what the
|
||||
tomo-queue backend actually needs: self.client + _init_tomo_queue().
|
||||
"""
|
||||
|
||||
import types
|
||||
|
||||
import csaxs_bec.bec_ipython_client.plugins.LamNI.lamni as lamni_module
|
||||
import csaxs_bec.bec_ipython_client.plugins.OMNY_shared.tomo_queue_mixin as tomo_queue_mixin
|
||||
from csaxs_bec.bec_ipython_client.plugins.LamNI.lamni import LamNI
|
||||
|
||||
|
||||
class _FakeRtx:
|
||||
user_parameter = {"large_range_scan": True}
|
||||
|
||||
|
||||
class _FakeDev:
|
||||
rtx = _FakeRtx()
|
||||
|
||||
def __contains__(self, name):
|
||||
return hasattr(self, name)
|
||||
|
||||
def __getitem__(self, name):
|
||||
return getattr(self, name)
|
||||
|
||||
|
||||
class FakeClient:
|
||||
"""Minimal in-memory stand-in for BEC's global-var store."""
|
||||
|
||||
def __init__(self):
|
||||
self._vars = {}
|
||||
|
||||
def get_global_var(self, key):
|
||||
return self._vars.get(key)
|
||||
|
||||
def set_global_var(self, key, value):
|
||||
self._vars[key] = value
|
||||
|
||||
|
||||
class FakeOMNYTools:
|
||||
def __init__(self, answer=True):
|
||||
self.answer = answer
|
||||
|
||||
def yesno(self, *args, **kwargs):
|
||||
return self.answer
|
||||
|
||||
|
||||
def make_lamni(fake_client=None):
|
||||
# lamni_piezo_range_x/y's setters check dev.rtx.user_parameter -- only
|
||||
# ever defined (via the `if builtins.__dict__.get("bec") is not None`
|
||||
# block at module import time) inside a real BEC session. Stand in a
|
||||
# fake so tomo_queue_execute()'s param-restore loop can reach these
|
||||
# setters like any other snapshotted param.
|
||||
lamni_module.dev = _FakeDev()
|
||||
|
||||
obj = object.__new__(LamNI)
|
||||
obj.client = fake_client or FakeClient()
|
||||
obj.OMNYTools = FakeOMNYTools()
|
||||
# ptycho_reconstruct_foldername's setter keeps this in sync; normally
|
||||
# built in __init__ from a real PtychoReconstructor.
|
||||
obj.reconstructor = types.SimpleNamespace(folder_name=None)
|
||||
obj._init_tomo_queue()
|
||||
return obj
|
||||
|
||||
|
||||
def test_tomo_scan_param_names_have_no_flomni_only_concepts():
|
||||
"""Lamni is laminography: 360-only, no single-point acquisition -- these
|
||||
must not appear in the snapshotted param set at all."""
|
||||
names = LamNI._TOMO_SCAN_PARAM_NAMES
|
||||
assert "tomo_angle_range" not in names
|
||||
assert "single_point_instead_of_fermat_scan" not in names
|
||||
assert "single_point_random_shift_max" not in names
|
||||
# lamni's own names for overlapping flomni concepts are present instead
|
||||
assert "tomo_circfov" in names
|
||||
assert "lamni_stitch_x" in names
|
||||
assert "lamni_stitch_y" in names
|
||||
assert "fovx" not in names
|
||||
assert "stitch_x" not in names
|
||||
|
||||
|
||||
def test_tomo_queue_add_snapshots_all_param_names():
|
||||
lamni = make_lamni()
|
||||
lamni.tomo_countingtime = 0.2
|
||||
lamni.tomo_circfov = 42.0
|
||||
|
||||
index = lamni.tomo_queue_add(label="my job")
|
||||
|
||||
assert index == 0
|
||||
jobs = lamni._tomo_queue_proxy.as_list()
|
||||
assert len(jobs) == 1
|
||||
job = jobs[0]
|
||||
assert job["label"] == "my job"
|
||||
assert job["kind"] == "tomo"
|
||||
assert job["status"] == "pending"
|
||||
assert set(job["params"]) == set(LamNI._TOMO_SCAN_PARAM_NAMES)
|
||||
assert job["params"]["tomo_countingtime"] == 0.2
|
||||
assert job["params"]["tomo_circfov"] == 42.0
|
||||
|
||||
|
||||
def test_tomo_queue_show_uses_lamni_specific_job_line(capsys):
|
||||
lamni = make_lamni()
|
||||
lamni.tomo_circfov = 15.0
|
||||
lamni.tomo_queue_add(label="job1")
|
||||
|
||||
lamni.tomo_queue_show()
|
||||
|
||||
out = capsys.readouterr().out
|
||||
assert "circfov=15.0" in out
|
||||
assert "range=" not in out # lamni is 360-only, no angle-range field
|
||||
|
||||
|
||||
def test_tomo_queue_execute_runs_fresh_job_then_marks_done():
|
||||
lamni = make_lamni()
|
||||
calls = []
|
||||
lamni.tomo_scan = lambda *a, **k: calls.append(("scan", a, k))
|
||||
lamni.tomo_scan_resume = lambda *a, **k: calls.append(("resume", a, k))
|
||||
|
||||
lamni.tomo_queue_add(label="job1")
|
||||
lamni.tomo_queue_execute()
|
||||
|
||||
assert calls == [("scan", (), {})]
|
||||
job = lamni._tomo_queue_proxy.as_list()[0]
|
||||
assert job["status"] == "done"
|
||||
|
||||
|
||||
def test_tomo_queue_execute_resumes_incomplete_job():
|
||||
lamni = make_lamni()
|
||||
calls = []
|
||||
lamni.tomo_scan = lambda *a, **k: calls.append(("scan", a, k))
|
||||
lamni.tomo_scan_resume = lambda *a, **k: calls.append(("resume", a, k))
|
||||
|
||||
index = lamni.tomo_queue_add(label="job1")
|
||||
lamni._tomo_queue_proxy.update(index, status="incomplete")
|
||||
|
||||
lamni.tomo_queue_execute()
|
||||
|
||||
assert calls == [("resume", (), {})]
|
||||
job = lamni._tomo_queue_proxy.as_list()[0]
|
||||
assert job["status"] == "done"
|
||||
|
||||
|
||||
def test_tomo_queue_add_command_move_action(monkeypatch):
|
||||
lamni = make_lamni()
|
||||
fake_dev = {"mokev": "mokev-device"}
|
||||
moved = []
|
||||
monkeypatch.setattr(tomo_queue_mixin, "dev", fake_dev, raising=False)
|
||||
monkeypatch.setattr(tomo_queue_mixin, "umv", lambda *a: moved.append(a), raising=False)
|
||||
|
||||
lamni.tomo_queue_add_command(
|
||||
{"action": "move", "kwargs": {"positions": {"mokev": 6.2}}}, label="reconfigure"
|
||||
)
|
||||
lamni.tomo_queue_execute()
|
||||
|
||||
assert moved == [("mokev-device", 6.2)]
|
||||
job = lamni._tomo_queue_proxy.as_list()[0]
|
||||
assert job["kind"] == "command"
|
||||
assert job["status"] == "done"
|
||||
|
||||
|
||||
def test_tomo_queue_add_command_rejects_non_movable_device():
|
||||
lamni = make_lamni()
|
||||
try:
|
||||
lamni.tomo_queue_add_command({"action": "move", "kwargs": {"positions": {"fzp": 1.0}}})
|
||||
assert False, "expected ValueError for a non-allow-listed device"
|
||||
except ValueError as exc:
|
||||
assert "not queue-movable" in str(exc)
|
||||
|
||||
|
||||
def test_register_at_each_angle_hook_dispatches_from_at_each_angle():
|
||||
lamni = make_lamni()
|
||||
seen = []
|
||||
lamni.register_at_each_angle_hook("record", lambda self, angle: seen.append(angle))
|
||||
lamni.at_each_angle_hook = "record"
|
||||
|
||||
lamni._at_each_angle(12.5)
|
||||
|
||||
assert seen == [12.5]
|
||||
assert lamni.client.get_global_var("tomo_at_each_angle_hooks") == ["record"]
|
||||
|
||||
|
||||
def test_at_each_angle_default_path_unchanged_without_hook():
|
||||
"""No hook registered -- must fall through to the exact same default
|
||||
behaviour lamni had before this port: tomo_scan_projection + tomo_reconstruct."""
|
||||
lamni = make_lamni()
|
||||
calls = []
|
||||
lamni.tomo_scan_projection = lambda angle: calls.append(("projection", angle))
|
||||
lamni.tomo_reconstruct = lambda: calls.append(("reconstruct",))
|
||||
|
||||
lamni._at_each_angle(30.0)
|
||||
|
||||
assert calls == [("projection", 30.0), ("reconstruct",)]
|
||||
|
||||
|
||||
def test_at_each_angle_raises_for_unregistered_hook_name():
|
||||
lamni = make_lamni()
|
||||
lamni.at_each_angle_hook = "missing_hook"
|
||||
try:
|
||||
lamni._at_each_angle(0.0)
|
||||
assert False, "expected ValueError for an unregistered hook name"
|
||||
except ValueError as exc:
|
||||
assert "missing_hook" in str(exc)
|
||||
assert "not registered" in str(exc)
|
||||
Reference in New Issue
Block a user