Feat/omny sample ioc #334

Merged
holler merged 69 commits from feat/omny-sample-ioc into main 2026-10-26 14:07:56 +01:00
Member
No description provided.
holler added 65 commits 2026-10-19 13:34:59 +02:00
- post_startup.py: --session omny imported OMNY from the flomni plugin
  package instead of omny (ImportError on every start)
- omny/x_ray_eye_align.py: XrayEyeAlign.__init__ referenced self.lamni.align
  on the OMNY instance mid-construction, before OMNY.__init__ finishes
  assigning it (AttributeError on every OMNY() construction)
- omny/gui_tools.py: OMNYGuiTools.__init__ crashed with KeyError in any
  session without an open BEC-widgets "main" window (e.g. --nogui); guarded
  to fall back to None like flomni's GuiTools
- omny_e2e_tests/sim_flomni_harness.py: FakeDeviceManager had no .connector,
  breaking rt_lamni_ophyd.py's feedback_enable_with_reset() client-info call
- omny_e2e_tests/sim_lamni_omny_harness.py: fixed a stale import of the
  renamed LamNIFermatScan module
- AI_docs/SIMULATED_ENDSTATIONS.md: recorded the OMNY live-test results
- omny/AI_docs/OPEN_ISSUES.md: new, tracks what the live smoke test found,
  including the unfixed osamroy rotation-offset bug that still blocks scans

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Real hardware already applies this correctly: controller1.dmc's #NEWPAR
handler subtracts 25 deg from the target when setting osamroy's position
(contrID=3, axis=2), symmetric with the +25 the ophyd read side
(OMNYGalilReadbackSignal._socket_get) already adds back. The simulated
Galil controller (SimGalilState.start_move_from_newpar) didn't replicate
this, so a simulated osamroy.move(x) landed 25 deg away from where real
hardware would -- a simulator-fidelity bug, not a hardware or ophyd bug.

Also adds osamroy to the offline sim_lamni_omny_harness so this class of
bug is caught there going forward, and a focused unit test for the
round-trip and its scoping to port 8083 / axis 2 only.
- sim_galil.py: default unseeded analog-input channels to a voltage that
  reads back as a realistic ~30 degC motor temperature instead of the
  previous 0.0V/150 degC default (real motors run well under 60 degC).
- rt_omny_ophyd.py: feedback_enable_with_reset() now tells the user the
  current angle and warns it might take a while when osamroy has more
  than 5 deg to rotate back to zero, matching flomni's style of heads-up.
- omny_sample_transfer_mixin.py: otransfer_put_sample() now calls
  reset_correction()/reset_tomo_alignment_fit() when mounting into the
  sample stage (pin_position == 0), mirroring flomni's
  ftransfer_put_sample() -> ftransfer_flomni_stage_in() pattern. Fixes
  an AttributeError ('OMNY' object has no attribute 'corr_pos_x') hit
  when calling tomo_scan_projection() after a sample transfer.
- New shared static helper _equally_spaced_subtomo_angles(), ported from
  flomni's _subtomo_angle_plan() with the 180/360-degree-range branch
  removed (OMNY only ever scans 180 degrees). Generalizes the bit-reversal
  sub-tomogram interleaving to any power-of-two sub-tomogram count; for
  n_subtomos=2 it reduces to OMNY's original type-1 scheme.
- sub_tomo_scan() refactored onto the new helper (gains an n_subtomos
  param); tomo_type 1 (2 sub-tomograms) is otherwise unchanged.
- Adds tomo_type 4 (4 sub-tomograms) and 5 (8 sub-tomograms), gated by the
  same "wear in OMNY" authorization-code prompt already used for tomo_type
  2/3.
- Advanced tomo modes (2, 3, 4, 5) are now hidden from tomo_parameters()'s
  menu unless bec.active_account is in the new _SUPERUSER_ACCOUNTS
  allowlist; a non-superuser landing on an advanced type is forced back to
  type 1. _SUPERUSER_ACCOUNTS is left empty -- needs real account codes
  before anyone can use the advanced modes.
- As a side effect of reusing flomni's formula, also fixes a latent bug:
  the old code derived the inter-sub-tomogram phase offset from the raw
  configured tomo_angle_stepsize rather than the achievable (int-truncated)
  step, producing an unevenly-spaced combined grid whenever stepsize didn't
  divide 180 evenly.
- New tests/tests_bec_ipython_client/test_omny_tomo_angles.py: angle-count
  and no-duplicate-angle checks for n_subtomos in (2, 4, 8), a regression
  check against the old type-1 formula, an explicit test of the
  achievable-step fix, a cross-check against flomni's hardcoded N=8
  bit-reversal table, and superuser-gating tests (including the full
  tomo_parameters() interactive flow forcing a non-superuser back to
  type 1, and a superuser successfully unlocking type 5).
feat(omny): grant gac-x01dc superuser access to advanced tomo modes
CI for csaxs_bec / test (push) Successful in 2m54s
336cae3cf5
For testing tomorrow. Note: bec.active_account is a Redis-backed pgroup
value set via `bec-set-account <pgroup>`, not derived from the OS login --
testing under this account requires running `bec-set-account gac-x01dc`
(or otherwise setting the pgroup to that value) before it takes effect.
bec.active_account is a Redis-backed pgroup, only set via bec-set-account
-- a fresh dev/test session normally has it unset. _is_superuser_account()
now falls back to os.getenv("USER") in that case, matching the
account/system-user comparison convention already used in
OMNY_shared/omny_general_tools.py's _accounts_match(). Lets gac-x01dc's
superuser grant take effect without requiring bec-set-account first.
- SimOMNYDewar: patched_device()'s all-zero default read as an alarm
  state (LN2 flow 0.0 l/s trips OMNYDewar.is_flow_low()'s 3.8 l/s
  threshold). Seeded to a healthy/nominal state instead. These values
  are illustrative -- no real dewar-controller reference was available.
- SimOMNYTemperatures: seeded names/units/readings/setpoints for the 29
  real "TEMP" channels, sourced from the real OMNY_tempcontroller/
  tempcontrol.py channel config (name, EGU, ChanHeat, ALARMLIM) and
  cross-referenced against the existing _set_TEMP_default_setpoints()
  (already in this repo, confirmed to cover the same 29 channels).
  Heated channels read at their real setpoint; monitor-only channels get
  a plausible reading below their real alarm limit -- no channel now
  shows a false alarm. Channels not in use are now named "-" (matching
  tempcontrol.py's real default), so show_all() cleanly hides them
  instead of listing 19 fake all-zero rows. temperature_update_time and
  cryo_temperature_update_time are set to now() to avoid a false
  "controller communication is not running" warning.
- The 4 "CRYO" channels are NOT seeded yet -- pending the cryocon
  controller reference (still needed from Mirko).
feat(omny): seed the 3 known CryoCon channels in SimOMNYTemperatures
CI for csaxs_bec / test (push) Successful in 3m56s
5e8fda761f
Chan A "Cryo sample" 90K, Chan B "Cryo head" 82K, Chan C "Sample backup"
90K, all closed-loop (reading == setpoint). Channel D has no known real
use yet and is left unseeded.
feat(omny): skip OSA/FZP moves when already at the target position
CI for csaxs_bec / test (push) Successful in 2m58s
62b4b8d8fa
oosa_in(), oosa_out(), ofzp_in(), ofzp_out() used to unconditionally
re-move every axis (and, for FZP, unconditionally disable rt feedback)
even when already at the target -- visible as repeated no-op progress
bars on back-to-back calls. Ported the same "skip if already there"
optimization flomni's fosa_in()/ffzp_in() already have (and that lamni
already got too, see test_lamni_optics_skip_if_in_position.py).

For oosa_in()/oosa_out() the "already there" check has to run *before*
_oosa_check_y(), since that call unconditionally parks oosay at 0.9 --
never the real in/out position for either near- or far-field mode -- so
checking afterward would always miss the match. oosa_out()'s "out"
targets are hardcoded literals in the existing code (no user_parameter
for out, unlike oosa_in's far_field_in/near_field_in), so the skip
check compares against those same literals.

New tests/tests_bec_ipython_client/test_omny_optics_skip_if_in_position.py
(9 tests, mirroring the lamni test's structure): skip-vs-move for both
near/far field on oosa_in/oosa_out, and skip-vs-move-and-disable-feedback
for ofzp_in/ofzp_out.
Two real bugs found while investigating a recurring AttributeError:
'OMNY' object has no attribute 'corr_pos_x' from tomo_scan_projection().

1. default_correction_file/default_correction_file_x were bare relative
   filenames ("correction_omny_202204.txt"), unlike flomni's equivalent
   which resolves an absolute path via Path(csaxs_bec.__file__). open()
   would only ever succeed if bec were launched from this exact plugin
   directory -- silently swallowed everywhere else by the existing
   FileNotFoundError handler in reset_correction(). The real correction
   files genuinely exist on disk next to the plugin code and were very
   likely never actually being loaded.

2. corr_pos_x/corr_angle_x/corr_pos_y/corr_angle_y/corr_pos_y_2/
   corr_angle_y_2 were plain instance attributes, only ever assigned
   inside reset_correction() (itself only called from
   otransfer_put_sample() during a sample transfer) -- hence the
   AttributeError when calling tomo_scan_projection() beforehand.
   Converted to global-var-backed properties defaulting to [] when unset,
   mirroring flomni's corr_pos_y/corr_angle_y/etc. exactly. This
   supersedes a shallower getattr()-based band-aid from an earlier,
   uncommitted attempt at this fix. As a bonus, correction data now
   persists across BEC client restarts, same as flomni's.

Note flomni has no x-correction concept at all (only corr_pos_y/
corr_pos_y_2) -- omny's x-correction is its own thing, not something
copied from flomni.

17 new/updated tests in test_omny_alignment_mixin.py, including an
end-to-end check that reset_correction(use_default_correction=True) now
actually loads the real correction_omny_202204*.txt files.
omny_fermat_scan.py's corridor_size was Annotated[float] = 3, never
updated to accept None the way flomni_fermat_scan.py's already does.
But omny.py's tomo_scan_projection() calling code passes
corridor_size=None whenever omny.corridor_size (a property defaulting
to -1) hasn't been set to a positive value -- producing
ScanInputValidationError: Invalid type for scan argument 'corridor_size':
None is neither float or int.

Ported flomni's fix directly: corridor_size is now
Annotated[float | None] = None, capped at the new MAX_CORRIDOR_SIZE
(3 um) class constant if a caller passes something larger, and
prepare_scan() now calls the same _estimate_corridor_size() static
method flomni uses (median nearest-neighbor distance * 1.5, capped at
MAX_CORRIDOR_SIZE) to pick a sensible corridor size from the actual
scan positions whenever corridor_size is None, instead of crashing.

2 new tests for _estimate_corridor_size() in test_omny_fermat_scan.py.
- osamroy's sim_initial_position is now -25 (raw), so the simulated
  logical readback (raw + 25, per the controller's calibration-mark
  offset) starts at ~0 deg instead of 25 -- no rotation-to-zero needed
  just from starting a session.
- All 21 SimOMNYGalilMotor axes now have an explicit sim_velocity, set
  to 4x their previous implicit default (the _SimGalilAxis fallback of
  5.0 * 25600 = 128000 steps/s). 10.0 for the 20 translation axes
  (stppermm=51200); 5.716463414632019 for osamroy specifically, since
  its stppermm=89565.8666667 differs -- both correspond to the same
  512000 steps/s, i.e. a genuine 4x speedup for every stage.
Two bugs found by comparing omny's RT-flyer code against flomni's (per
Mirko, since flomni/lamni's fermat scans already work fine in sim).

1. Root cause of the reported scan hang (tomo_scan_projection() starts a
   scan but the progress bar freezes at 0% forever, no error):
   RtOMNYFlyer.read_positions_from_sampler() called self.get_scan_status()
   instead of self.controller.get_scan_status(). get_scan_status() only
   exists on RtOMNYController, never on RtOMNYFlyer -- so this line raised
   an AttributeError inside the background thread complete() spawns for
   readout. An uncaught exception in a thread target doesn't propagate;
   the thread just dies silently, before ever calling status.set_finished().
   scan_core()'s `while not status.done` loop then spins forever, and since
   the thread died before ever updating progress, the bar stays at 0%.
   Confirmed correct in both RtFlomniFlyer's and RtLamniFlyer's equivalents
   already; fixed to match.

2. RtOMNYSetpointSignal._socket_set() never had the laser-tracker-signal-
   strength guard RtFlomniSetpointSignal._socket_set() got in commit
   68320e1 ("check tracker signal before rt move, which is otherwise stuck
   forever") -- a real-hardware failure mode (a move issued with a weak
   interferometer signal never converges). OMNY has its own duplicate
   setpoint-signal class, so the fix never propagated. Ported using OMNY's
   own existing laser_tracker_check_and_wait_for_signalstrength() (a more
   capable, retrying/auto-readjusting check) rather than flomni's
   single-shot version, which OMNY doesn't have. Lower confidence this one
   is the actual cause of the reported *simulated* hang specifically --
   the sim doesn't model signal-dependent convergence -- but it's a real,
   confirmed gap worth closing for real-hardware safety regardless.

4 new regression tests in tests/tests_devices/test_rt_omny.py, including
one that directly confirms the fake stand-in used for bug #1's test would
have caught the original bug (raises AttributeError the same way the real
class would).
OMNYGuiTools still used pre-modernization code: a gui_window handle that
was never actually populated (self.gui.windows["main"].widget always
raised/returned None), the old two-level add_dock().add_widget() API, and
bare `if self.figN is None` checks instead of an _is_deleted()-aware
guard. omnygui_show_progress() crashed with AttributeError as soon as it
was exercised live.

Ported OMNYGuiTools onto flomni's current single-level .new() pattern:
split __init__()/set_client(), a window-reuse check keyed on
self.gui.windows, _omnygui_is_missing() for dock-reuse (verbatim port of
flomni's _is_deleted()-based guard), the current RingProgressBar API for
omnygui_show_progress(), and new omnygui_show_xeyealign()/
omnygui_show_xeyealign_fittab()/omnygui_show_cameras() methods. cam_xeye
is not part of omny's device config (unlike flomni's), so the xeyealign
live-view toggle is guarded with an existence check instead of copied
unconditionally. omnygui_show_cameras() adds a hard-stop console targeting
otransy, the closest omny analog to flomni's ftransy.

Also fixes two independently-broken call sites in x_ray_eye_align.py
(self.lamni.lamnigui_show_xeyealign[_fittab]() -> the omny-side method
names actually defined on OMNYGuiTools -- self.lamni is really the OMNY
instance) that were raising AttributeError before the GUI code was even
reached. The rest of that file's LamNI-derived content is left alone; a
full rewrite is tracked separately.

omny.py's __init__ now calls OMNYGuiTools.__init__(self) + set_client(),
matching the split-constructor pattern instead of the old single-call
OMNYGuiTools.__init__(self, self.client).

Deliberately deferred: flomni's ETA/heartbeat progress fields
(tomo_start_time, heartbeat, estimated_remaining_time, etc.) are not
ported -- omny's self.progress dict only has 7 basic fields. Noted in
omny/AI_docs/OPEN_ISSUES.md so it isn't forgotten; not a crash risk since
omny_webpage_generator.py already reads these defensively.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Live testing of the previous gui_tools.py port (49ade7c) hit two further crashes:

1. omnygui_show_xeyealign() -> ValueError: Device 'cam_xeye' not found in current
   BEC session. The OMNY_XRayEye widget (shared verbatim with flomni) hardcodes
   CAMERA = ("cam_xeye", "image") as a module constant -- it isn't configurable
   per-beamline, so omny needs its own cam_xeye device or the widget can never be
   constructed. Added cam_xeye to simulated_omny.yaml (hand-added, duplicating
   flomni's real cam_xeye's camera_id/pixel_calibration as placeholders; SimIDSCamera
   already accepts both camera_id and the legacy camera_ID key omny's other cameras
   use). ptycho_omny.yaml intentionally left untouched -- the real hardware camera ID
   and pixel calibration for OMNY's x-ray-eye camera are unknown; noted as a followup
   in OPEN_ISSUES.md, including that a future regen of simulated_omny.yaml from
   ptycho_omny.yaml will drop this hand-added block until the real config has one too.

2. omnygui_show_omnycam_parking() -> ValueError: Unknown widget type: BECImageWidget.
   That string was carried over unmodified from omny's pre-modernization code, not
   from flomni's current gui_tools.py (whose current class name is "Image"). Beyond
   the widget name, the per-camera calls inside were also stale: fig.set_rotation(
   deg_90=3) doesn't exist on the current Image/ImageBase widget at all (replaced by
   the num_rotation_90 property), and self.figN.lock_aspect_ratio(True) called a
   property as a function. Ported omnygui_show_omnycam_parking()/
   omnygui_show_omnycam_samplestage() onto flomni's current per-camera pattern
   (image(device=, signal="preview"), num_rotation_90/lock_aspect_ratio as
   properties, start_live_mode()), preserving the existing 3x90 rotation for
   cam200/cam203 (physical mount) and no rotation for cam201/cam202, unchanged from
   before.

Also removed omnygui_show_cameras() (added last pass, modeled on flomni's single
generic camera view): redundant now that both existing parking/samplestage views got
the current API, since omny -- unlike flomni -- already has two purpose-built camera
views. Its hard-stop console (z_ConsoleButtonsWidget, targeting otransy) now lives
directly in both omnygui_show_omnycam_parking() and omnygui_show_omnycam_samplestage()
instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
omny.tomo_scan_projection(1) -> omny_fermat_scan crashed with ValueError: 'cam200'
is not in list once a scan actually did a readback-tracked osamroy rotation (the
first time this branch's testing got that far).

Root cause, traced through bec_server + bec_ipython_client core code: omny_fermat_scan's
add_scan_report_instruction_readback(devices=["osamroy"]) registers osamroy as requiring
a response; post_scan()'s complete_all_devices() then batches every enabled+claimable-owned
device into one end-of-scan instruction, including cam200 (readoutPriority: async +
enabled: true default to ownership_mode: "claimable", auto-acquired for any scan unless
readoutPriority is "on_request"). The batched instruction's response flag isn't scoped
per-device, so the server publishes a device-request-status message for cam200 too, under
the same RID the osamroy-only readback progressbar is listening on -- which then crashes
on an unrecognized device name.

cam200/201/202/203 (and cam_xeye, added last pass) are view-only GUI cameras never
referenced by any scan code (confirmed via repo-wide grep). Changing their readoutPriority
to on_request excludes them from the claimable/owned auto-acquire set entirely, fixing the
crash with a config-only change -- and matches flomni's own convention for this same
category of device (cam_flomni_gripper/cam_flomni_overview are both on_request already).
Live-mode GUI streaming (start_live_mode(), used by the parking/samplestage camera views)
is unaffected since it's a dedicated background thread, not driven by readout-priority
scheduling -- flomni's own cameras already prove on_request + live-mode works together.

Applied to both simulated_omny.yaml and the real ptycho_omny.yaml per Mirko: this brings
both configs in line with flomni's convention, not just a sim-only workaround.

The underlying bug (instruction-batch-scoped response flag in scan_actions.py:_send(),
plus missing per-device filtering in ReadbackDataHandler/DeviceProgressBar) is a genuine
bec-core issue, not omny-specific -- flomni's own cam_xeye has the same latent exposure,
untouched here since it's out of this branch's scope. Recorded in omny/AI_docs/OPEN_ISSUES.md
and reported upstream separately.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Live-reproduced (local BEC deployment, real client) that the readoutPriority
fix (c37bfc7) didn't actually fix the readback-progressbar crash -- it just
moved it to a different device (rt_positions instead of cam200), confirming
the earlier diagnosis was incomplete.

Real root cause: omny_rotation() called
self.actions.set(self.dev.osamroy.user_setpoint, angle, wait=False),
passing the user_setpoint Signal sub-component instead of the root device.
ScanActions._normalize_device_name (bec_server) normalizes a sub-signal to
its dotted name ("osamroy.user_setpoint"), which never matches the plain
"osamroy" registered as requiring a response by the preceding
add_scan_report_instruction_readback(devices=["osamroy"], ...) call. So the
rotation's own completion is never reported, and the readback progressbar's
request-status listener stays subscribed for the rest of the scan -- until
post_scan()'s complete_all_devices() batches many unrelated owned devices
into one instruction later, which DOES trigger a response (since the literal
"osamroy" is also in that batch), reporting every device in it and crashing
the still-alive progressbar on whichever one happens to complete first.

Confirmed live this is omny-specific, not a flomni-shared exposure as
previously assumed: flomni_fermat_scan's equivalent flomni_rotation() calls
self.actions.set(self.dev.fsamroy, angle, wait=False) -- the whole root
device -- so its own completion reports correctly and the progressbar exits
cleanly well before complete_all_devices() runs. A live flomni_fermat_scan
run with cam_xeye still async+enabled completed without incident, disproving
the earlier "flomni has the same latent exposure" assumption.

Fixed by matching flomni's pattern exactly. OMNYGalilMotor.move()
(ogalil_ophyd.py) already does self.user_setpoint.put(...) plus proper
completion bookkeeping internally, so setting the root device is a strict
superset of the previous behavior, not a functional change to the move
itself. The readoutPriority change from the previous commit is kept (still
correct, matches flomni's camera convention) even though it wasn't the
actual fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
docs(omny): note readoutPriority: on_request was kept deliberately
CI for csaxs_bec / test (push) Successful in 1m56s
37d3197f03
Mirko confirmed keeping on_request (not reverting to the original async)
even though it turned out not to be what fixed the omny_fermat_scan crash
-- these cameras are only ever used for live GUI viewing, never read back
in scan code, so async's continuous per-scan logging was likely unused
anyway. Recorded so this doesn't read as leftover cruft later.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
fix(omny): quiet laser tracker check, drop redundant call
CI for csaxs_bec / test (push) Successful in 4m21s
e0213697bc
laser_tracker_check_and_wait_for_signalstrength() used send_client_info()
for its "Checking laser tracker..."/"...completed." messages, which queues
onto the scan's client-message list -- exposing it to a bec_lib bug
(QueueItem.get_client_messages() mutates its list while iterating, so it
skips and later re-delivers leftover messages on an unrelated call; see
OPEN_ISSUES.md, reported upstream). flomni's equivalent,
laser_tracker_check_signalstrength(), has always used plain print()
instead, which never queues and so never hits that bug. Switched omny's
version to match exactly (Mirko's call).

Both check functions run once per RT axis set (via
Rt*SetpointSignal._socket_set()), so a single scan setup (rtx+rtz set
together) already triggered 2 checks; omny_fermat_scan.py's
prepare_setup_part2() had an additional explicit third call, pure
duplication of the same check. Removed it -- one less redundant
laser-tracker wait per scan setup, no loss of the actual safety check
(rtx/rtz are still individually checked via their own set() calls).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Mirko is testing live from his own machine as OS user "holler" with no
active_account set, so the superuser fallback (os.getenv("USER")) needs
"holler" in the allow-list to see the advanced tomo_parameters() options.

OPEN_ISSUES.md had gone stale relative to everything fixed this session:
the rotation-offset section still described it as an open, hardware-review
-only question (resolved days ago once the real DMC controller source
confirmed the real hardware code was always correct -- only the simulator
was missing the offset), and the checklist still said scans were blocked
entirely (a full omny_fermat_scan now runs end to end). Rewrote both
sections to reflect current reality, added a "Current status" summary at
the top so a fresh session can orient in one read, and noted the
_SUPERUSER_ACCOUNTS change and that advanced tomo mode testing is next.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
feat(omny): drop redundant tomo auth code, cap type-3 step at 2.5 deg
CI for csaxs_bec / test (push) Successful in 1m45s
2fbaa6679f
The "Enter authorization code" (x12sa) prompt gating advanced tomo modes
(types 2-5) predates the account-based _SUPERUSER_ACCOUNTS gate added
earlier this branch, and is now fully redundant with it -- Mirko asked to
remove it. Each mode still prints the existing "significant wear" warning
unconditionally instead of only after entering the code.

Also added a hard 2.5 degree ceiling on tomo type 3's angular step (golden
ratio starting angle mode), replacing its old minimum-100-projections floor
with an explicit wear-motivated cap; if the requested projection count
implies a larger step, it's clamped down to 2.5 degrees. Type 2's
golden_ratio_bunch_size minimum stays at 100 per Mirko's call -- wear
avoidance there is handled by the warning message, not a lower floor.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Mirko flagged that flomni previously had real, painful bugs where combined
sub-tomograms weren't actually equally spaced -- given that history, verify
(and permanently regression-guard) the same property for OMNY's tomo types
1/4/5, at the specific combination granularity he described: type 4's 1+2
and 3+4 pairings individually, not just the full 1-2-3-4 combination already
covered; and type 5's progressive 1+2 / 1+2+3+4 / 1..8 doubling stages,
matching flomni's own documented interlacing design.

Verified offline against the exact production function
(_equally_spaced_subtomo_angles) across several step sizes including ones
that don't divide 180 evenly; for n_subtomos=8 its bit-reversal phase index
was also cross-checked to reproduce flomni's hardcoded, already-proven-
correct phase_eighths table exactly (already covered by the existing
test_bit_reversal_matches_flomnis_hardcoded_phase_eighths_table, unchanged).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Mirko recalled that flomni/lamni check the combined angular spacing adds up
to 180 (360 for lamni) -- found the precise existing pattern in lamni's own
test suite (test_lamni_tomo_angles.py's _assert_equally_spaced(): sort the
combined angles, append a wraparound point at first_angle + range, and
assert every gap including that seam is uniform). This is strictly stronger
than checking the total span alone -- it also catches a double-wide or
missing gap right at the seam, which is exactly the shape of bug flomni's
historical fixes (e.g. 6acaadb, "fix angular start in even subtomos")
addressed.

Ported the same wraparound-inclusive check into OMNY's tests: the existing
full-combination test now asserts it directly, and the intermediate-pairing
helper (_assert_evenly_spaced, used by the type-4 and type-5 pairwise/
progressive combination tests added previously) now includes it too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Mirko wants the tomo params GUI widget available for OMNY before running
real tomo scans. That widget (Phase 2, separate follow-up) needs a working
job queue and scan progress visible outside the CLI session that started
it -- neither existed for OMNY. OMNY now inherits TomoQueueMixin
(OMNY_shared/tomo_queue_mixin.py), the same shared backend flomni/lamni
already use, giving it tomo_queue_add/delete/move/clear/show/execute/
reacquire, at_each_angle_hook registration, and tomo_scan_resume() for the
first time.

self.progress is now a global-var-backed proxy (_ProgressProxy, byte-for-
byte port of flomni's/lamni's own), not a plain in-memory dict -- the
load-bearing change the queue mixin and the eventual widget's busy-banner
both depend on. tomo_queue_mixin.py's tomo_queue_reacquire() hardcoded
tomo_type == 1; generalized via a new _TOMO_TYPE_1_LIKE class attribute
(default frozenset({1}), flomni/lamni unaffected) since OMNY has three
"equally spaced sub-tomograms" flavors (types 1/4/5), not just one.

Along the way, fixed several real bugs in tomo_scan()/write_pdf_report()
that predate this session and would have crashed on a real account/
real energy read (bec.active_account.decode() -- active_account is str
not bytes; dev.mokev doesn't exist for OMNY, only dev.ccm_energy; an
unguarded logbook send; a stale "LamNI" logo/tag copy-paste leftover) --
all direct ports of fixes flomni/lamni already needed and got. Also added
the missing unconditional omnygui_show_progress() call and heartbeat-
clearing try/finally in tomo_scan(), matching both.

Verification: 23 new tests (test_omny_tomo_queue.py, test_omny_tomo_scan.py)
covering the mixin wiring, hook dispatch/resolution, tomo_scan_resume()/
_resolve_type1_projection()/reacquire() across all three sub-tomogram
flavors, account handling, and heartbeat clearing on both normal completion
and a mid-scan exception. Full suite: 702 passed. Not yet live-tested
against a running session (the local bec-server may be in active use by
Mirko's own parallel session) -- next step is his own pull-and-test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
write_to_scilog()/write_pdf_report() used the old bec.logbook/client.logbook.
send_logbook_message() API, which has no "is scilog even configured" check.
Confirmed live: on a session without scilog credentials (this dev deployment
has no /etc/bec/secrets/.scilog.env at all), send() blocks indefinitely
instead of failing fast -- a real tomo_scan() hung inside
_write_subtomo_to_scilog() for 29+ minutes during Phase 1 live testing.

flomni already migrated off this API to bec.messaging.scilog, which gates
every send behind scilog._enabled (published from the deployment's actual
scilog config) -- per Mirko's direction, ported the same mechanism to both
OMNY call sites, matching flomni's write_pdf_report()/_scilog_write()
exactly rather than merely adding an active_account check.

Added 3 unit tests locking in the _enabled gate (skip when disabled, send
when enabled, tolerate a send() exception).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Extends the sample-storage bookkeeping widget, previously FlOMNI-only, to
also work against dev.omny_samples: shuttles A/B/C (6 slots each), the
sparse fixed-O positions, stage, gripper, and a per-shuttle installed/active
parking status (mirrors the oparkz-driven "active parking slot" concept
from omny_sample_transfer_mixin.py). Setup detection follows the
SETUP_PROFILES/_detect_setup() convention already used by tomo_params.py.

Also fixes SimOMNYSampleStorage, which never cleared the fixed-O/shuttle-B-C
/parking slots, so they read back as the mock-PV default "0" instead of the
real empty sentinels ("-"/"none") -- SimFlomniSampleStorage already does
this correctly for the same reason.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Adds omny.otransfer_show_all() (renamed from otransfer_storage) plus thin
otransfer_* wrappers for every dev.omny_samples command, so they all
tab-complete under omny.* the same way flomni.py's ftransfer_* commands do.

Also fixes two issues found in first widget testing:
- OMNY_SampleStorage's occupied-slot text was hardcoded to a near-black
  colour, invisible against BEC's dark theme -- now inherits the
  application palette's default text colour instead.
- OMNY sample slots now require a sample owner, same as FlOMNI, packed via
  the existing sample_desc_codec (name+owner in the one raw name field).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Live-testing findings from Mirko's own session, four real bugs:

- tomo_parameters() now warns when the current fovx/fovy/tomo_shellstep
  would yield fewer than OmnyFermatScan.MIN_POSITIONS (20) Fermat scan
  points, matching Flomni.tomo_parameters()'s identical warning -- the scan
  server already aborted (ScanAbortion) on too few points, but nothing told
  the user ahead of time. Needed converting
  OmnyFermatScan.get_omny_fermat_spiral_pos() to a @staticmethod (was an
  instance method reading self.cenx/ceny/zshift) so it's callable from
  tomo_parameters() without a full scan instance, matching flomni's
  already-static version. Also promoted the hardcoded "20" in
  prepare_scan()'s abort check to a MIN_POSITIONS class constant.

- OMNY.tomo_scan() didn't accept interactive=, so every
  tomo_queue_execute() run (which always calls tomo_scan(interactive=False)
  for unattended queued runs) failed immediately with TypeError. Added the
  parameter, accepted but unused, matching flomni's own docstring reasoning
  (no fine-alignment gate on either setup).

- write_to_scilog()/write_pdf_report() used the old bec.logbook API with no
  "is scilog configured" check, so a session without scilog credentials
  hung indefinitely instead of failing fast (confirmed live: 29+ minutes).
  Migrated both to bec.messaging.scilog, gated on _enabled, matching
  flomni's actual current mechanism.

- Stale ~/Data10/specES1/ paths (predating the current ~/data/raw/ deployment
  layout) crashed _write_tomo_scan_number() with FileNotFoundError on a
  fresh session. Fixed that plus write_pdf_report()'s PDF target and
  tomo_reconstruct()'s default base_path, all three sharing the same stale
  root -- now also explicitly create their target directory first, since
  flomni's own equivalents assume it already exists.

16 new tests across test_omny_fermat_scan.py, test_omny_tomo_angles.py, and
test_omny_tomo_scan.py. Full suite: 725 passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
fix(sample_storage): enforce unique sample names in the widget
CI for csaxs_bec / test (push) Failing after 5s
52c1806fe2
Renaming or newly naming a slot now rejects a name already used by another
occupied slot in the active setup (FlOMNI or OMNY), so bookkeeping can't
end up with two same-named samples that are impossible to tell apart later
(including in the FlOMNI measured-sample log, which is keyed by name+owner).
Renaming a slot to its own existing name is still allowed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Mirko: "the flomni progress bar has three rings. here i am seing only
two... at the end of a tomogram in flomni a summary of the measurement
time and time lost is displayed. omny seems not to have that, maybe not
even measuring the time."

Both traced back to the same root gap left by the Phase-1 tomo-queue port:
nothing computed estimated_remaining_time/estimated_finish_time, and
nothing refreshed progress["heartbeat"] *during* a running scan (only
cleared it to None on completion) -- so accumulated_idle_time stayed 0.0
forever and there was no ETA data to show or summarize.

- _tomo_scan_at_angle() now refreshes the heartbeat every angle and
  attributes any gap beyond a normal-cadence heuristic to
  accumulated_idle_time (mirrors flomni's identical formula, minus its
  frames_per_trigger factor which OMNY has no property for).
- _print_progress() computes/stores/prints estimated_remaining_time and
  estimated_finish_time once the scan rate has stabilized.
- tomo_scan() prints an end-of-scan "Total measurement time"/"...excluding
  detected gaps"/"...lost to detected gaps" summary and sends it to scilog,
  direct port of flomni's block (minus its measured_log call, which OMNY
  has no equivalent of).
- gui_tools.py: omnygui_show_progress() adds flomni's third, scan-linked
  auto-updating ring; _omnygui_update_progress()'s center label now shows
  start time / ETA / estimated finish / active hook, matching flomni's
  _flomnigui_update_progress().

_describe_active_hook()/_active_hook_source() needed no porting -- already
provided by TomoQueueMixin since Phase 1. Not ported: flomni's separate
persistent timing-statistics-log subsystem (_log_tomogram_timing()) --
out of scope for what was actually asked (the printed summary + ring
count).

9 new tests (test_omny_tomo_scan.py, test_omny_gui_tools.py). Full suite:
731 passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Mirko, on macOS: the progress window "seems to close and recreate each
update, which is highly annoying" -- caused by _tomo_scan_at_angle()
calling self.omnygui_show_progress() (the full show-window-and-raise-to-
front routine) once per angle, on top of _print_progress(). flomni's own
_tomo_scan_at_angle() never does this: it only ever calls _print_progress(),
which calls self._flomnigui_update_progress() (values/text only) at its own
end -- the window is shown/raised exactly once, by tomo_scan()'s
unconditional call at scan start.

Mirrored that structure: _print_progress() now calls
self._omnygui_update_progress() itself, and the redundant per-angle
omnygui_show_progress() call was removed from _tomo_scan_at_angle().

1 new regression test. Full suite: 732 passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Gives OMNY sample storage a dedicated owner + measured-status channel
(paired with the new SAMPLE_DB IOC record in OMNY/OMNY.git, one per sample
position: VAL=status enum none/started/completed, DESC=owner), replacing
the pack_desc/DESC-packing hack the widget briefly used for OMNY owner
support. FlOMNI is untouched this pass -- deferred.

- OMNYSampleStorage: new sample_status/sample_owners Dcpt/Cpt groups;
  set_sample_slot/set_sample_in_gripper/set_sample_in_samplestage take an
  owner param and reset status on every place/rename/clear; new
  set_sample_status/get_sample_status/get_sample_owner/
  get_sample_name_and_owner (+ gripper/stage equivalents); show_all()/
  help() updated.
- SimOMNYSampleStorage: seeding clears the new owner/status signals too.
- omny_sample_transfer_mixin.py: otransfer_* wrappers for all the new
  methods, existing set_sample_slot/set_sample_in_gripper/
  set_sample_in_samplestage wrappers pass owner through.
- sample_storage.py widget: OMNY cells read/write owner+status via the new
  signals instead of packing; the "✓ measured"/"◐ started" badge now works
  for OMNY too, plus a manual "Set measured status…" context-menu action.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Mirko, once Phase 1 (tomo-queue backend) was live-confirmed working: "is
the way to move to the widget phase" -- this adds OMNY as a third setup to
the tomo params GUI widget, alongside flomni/lamni.

Key design decision: replaced the module-level TOMO_TYPES dict (shared
across setups, couldn't express "type 1 means 2 sub-tomograms here but 8
there") with a per-profile "tomo_types" dict:
{tomo_type_int: {"label": str, "n_subtomos": int | None}}. n_subtomos is
not None became the single, uniform test for "is this an equally-spaced
type" everywhere (combo box, section visibility, validation, queue-table
projections display), since OMNY has three such flavors (types 1/4/5 --
2/4/8 sub-tomograms) instead of flomni's/lamni's one (always 8).

Critical bug caught by verifying the design against the live file, not by
the initial design itself: _validate() hardcoded
`tomo_type not in (1, 2, 3)` -- would have made the rest of Phase 2 look
like it worked (types 4/5 selectable, sections visible, math correct)
while silently rejecting every actual submit/queue attempt for an OMNY
type-4/5 job. Fixed to check profile membership instead. Also fixed three
UI strings that hardcoded "8 equal sub-tomograms" to read the active
type's real count.

New _compute_type1_omny/_requested_to_stepsize_omny (parameterized by
tomo_type via {1:2,4:4,5:8}, matching
OMNY._equally_spaced_subtomo_angles()'s own mapping exactly) and
_compute_fermat_positions_omny (calls OmnyFermatScan's real static
method, same pattern as flomni's). _format_projections() gained a 3-way
setup duck-type (OMNY's param set is a strict subset of flomni's -- no
positively-unique OMNY-only key exists, so a negative-then-positive check
is necessary) that resolves into SETUP_PROFILES rather than duplicating
the type-count mapping a third time.

Confirmed needing zero changes (verified, not assumed): _job_tooltip(),
_build_type23_section(), compute_fermat_positions call sites,
_detect_setup() (osamroy as OMNY's discriminator), the busy-banner
detection methods (already generic via the shared tomo_progress global
var from Phase 1), and the Qt Designer plugin registration files.

New test_omny_tomo_params_widget_math.py (29 pure-function tests, no
QApplication/QWidget, same style as the existing flomni/lamni test
files). Full suite: 761 passed. Not live-tested (needs a real display,
same as every GUI change this branch) -- next step is Mirko trying it
against a real OMNY session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
Mirrors flomni's ftransfer_confirm()/save_reference_image() pattern, adapted
for OMNY's dual-camera-pair setup: whichever camera pair is currently shown
live (cam200+cam203 for parking/shuttle, cam201+cam202 for the sample
stage -- see _omnycam_parking()/_omnycam_samplestage() in gui_tools.py) gets
saved together into one HDF5 file at each manual confirm checkpoint in
_otransfer_confirm(), via the new _otransfer_save_reference_images().

X-ray eye alignment image/fit saving is left untouched -- it already exists
as shared code (x_ray_eye_align.py) but has known pre-existing OMNY/LamNI
naming bugs unrelated to this change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
- sample_storage.py: remove the "Set measured status…" context-menu action
  (and its now-unused write helper/constants) -- measured status is CLI-only
  (omny.otransfer_set_sample_status()/dev.omny_samples.set_sample_status()),
  not manually editable from the widget.
- omny_sample_transfer_mixin.py: otransfer_park_loadlock_slot()'s confirm
  check was inverted -- it raised on "y" (in back position) and proceeded
  on "n", exactly backwards. Pre-existing bug, found during sim testing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
- sample_storage.py: _write_slot_omny() read self._samples.STATUS_NONE, a
  plain OMNYSampleStorage class attribute -- but self._samples is the BEC
  client-side device proxy, which only exposes signals and USER_ACCESS
  methods, not arbitrary class attributes. Every widget write to an OMNY
  slot raised "'Device' object has no attribute 'STATUS_NONE'". Replaced
  with a local OMNY_STATUS_NONE constant, same pattern already used for
  EMPTY_NAME/PARKING_EMPTY_SENTINEL.
- omny_sample_transfer_mixin.py: otransfer_help() now also prints
  dev.omny_samples.help()'s full command reference (set/unset slots,
  shuttles, gripper, sample stage, owner, measured status) instead of just
  pointing at it, so omny.otransfer_help() alone is a complete reference.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
docs(omny): add sample-storage testing checklist
CI for csaxs_bec / test (push) Successful in 2m46s
6291f6196b
Covers the OMNY_SampleStorage widget, the real owner + measured-status
EPICS channel, the otransfer_* CLI wrappers, and transfer reference-image
saving -- so it can be pulled to a separate session/machine for testing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
docs(omny): add tomo-scan/queue/widget live-testing checklist
CI for csaxs_bec / test (push) Successful in 2m42s
89056e5a52
Written so Mirko can pull this and run through it from his own machine --
covers the Phase 2 widget (untested, includes the critical type-4/5
_validate() fix), the progress-GUI/idle-time/timing-summary changes, first
real end-to-end tomo-queue runs, and the advanced tomo modes' full scans
(only parameters were tested so far). Mirrors SAMPLE_STORAGE_TEST_CHECKLIST.md's
format/purpose for the concurrent sample-storage work.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
A Ctrl+C landing during raise_window()'s RPC wait (e.g. when the GUI
process is frozen/unresponsive) was escaping two `except Exception`
handlers that don't catch KeyboardInterrupt (a BaseException sibling,
not an Exception subclass): omnygui_show_progress() aborted the whole
tomo_scan()/tomo_scan_resume() call instead of just skipping the
window-raise, and tomo_queue_execute() left the job silently stuck at
status="running" instead of the usual clean "incomplete" + explanatory
message. Reproduced live by Mirko via the tomo queue.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
fix(omny): stop polling sample name via a device_rpc queue round-trip
CI for csaxs_bec / test (push) Successful in 1m51s
70696756f1
OMNY_TomoParams's poll timer called sample_name_getter every tick,
which for OMNY was dev.omny_samples.get_sample_name_in_samplestage()
-- a device method, dispatched by BEC as a device_rpc request through
the scan queue rather than a local read. With the queue busy or
paused this blocked the widget's Qt thread (GUI freeze while a scan
ran) and piled up duplicate PENDING device_rpc entries in BEC's
primary queue (confirmed live via bec.queue.queue_storage
.describe_queue()). The method was just a one-line wrapper around an
already-readable EpicsSignal, so read it directly with cached=True
instead, matching flomni's fast-getter pattern and avoiding even the
signal round-trip. Also applied cached=True to flomni's own getter,
which didn't have the device_rpc bug but is polled the same way.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
feat(omny): port zero_deg_reference_at_each_subtomo to types 1/4/5
CI for csaxs_bec / test (push) Successful in 2m35s
78f0c8fa3b
flomni already has this (type-1-only: a dedicated projection at exactly
0 degrees before every odd/forward sub-tomogram, plus once more at the
end, for tracking radiation damage over a tomogram). Deliberately
excluded from OMNY during the Phase 1 port as flomni-specific; Mirko
asked for it to be extended to OMNY's generalized equally-spaced types
(1/4/5, 2/4/8 sub-tomograms), and confirmed the pre-shot should fire
even for sub-tomogram 1 (whose own first projection already lands on 0)
to get two consecutive 0-degree shots as an immediate baseline pair.

Also fixes a second hardcoded `tomo_type != 1` bug in the widget's
_irrelevant_params_for_type() (same class as the earlier _validate()
fix) that would have hidden this setting from OMNY type-4/5 job
tooltips even when enabled.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
feat(omny): port collect_empty_frames() from flomni
CI for csaxs_bec / test (push) Successful in 2m15s
71eb9d858f
10 flat-field/empty-frame images at angle 0, sample shifted out of the
beam by fovx/2 + margin, taken once at the start of every new
tomo_scan() (not on resume) -- mirrors flomni's own collect_empty_frames()
structure and formula, using OMNY's own device names.

Took a few corrections to land on the right design: an initial staged
rtx+osamx move (based on a paraphrase of the real algorithm) was
discarded once Mirko surfaced the actual legacy SPEC macro, which does
a single position-threshold recenter instead. A second implementation
of that (RtOMNYController.move_samx_to_scan_region(), matching a name
already declared but unimplemented in USER_ACCESS) was itself
discarded on realizing OMNY already has this exact mechanism, working
and already wired into every real projection: dev.osamx
.omny_osamx_to_scan_center() (omny_fermat_scan.py's
prepare_setup_part2()). collect_empty_frames() reuses that directly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
fix(omny): reset progress fields before zero-degree reference shots
CI for csaxs_bec / test (push) Successful in 3m55s
bdb0d25f90
progress["projection"]/["subtomo_projection"]/["subtomo_total_projections"]/
["total_projections"] are otherwise only ever set inside sub_tomo_scan()'s
per-point loop. The zero_deg_reference_at_each_subtomo pre/post shots call
_tomo_scan_at_angle() directly, bypassing that -- so the printed progress
showed whatever was left over from a previous scan/subtomo instead of a
value that made sense for the shot. Confirmed live: a fresh scan's first
reference shot printed a stale "Projection: 9 / Total: 32" from an earlier
run. Same bug exists in flomni.py (this was ported from there faithfully,
bug included) -- flagged in OPEN_ISSUES.md but not touched, out of this
branch's scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLrD7sVYGLAzsQjLVJpCgt
- Widget: reject assigning a shuttle to a parking slot already held by a
  different shuttle ("Slot in use", clear it first) instead of silently
  overwriting.
- Widget: PDF report now shows only the 3 shuttles + their samples (title
  "OMNY Shuttle Samples") -- stage/gripper/fixed-O positions can't
  physically leave the vacuum chamber, so a printed report has nothing
  useful to say about them. The on-screen widget still shows/manages all
  of it.
- Widget: fixed the active-parking-slot badge never showing for a
  loadlock-driven park. Root cause: otransfer_park_loadlock_slot() drives
  oparkz to a different target-position table (base -6, "leica") than the
  regular otransfer_park_slot() (base -0.9); the widget only recognized
  the regular one. Now checks both.
- OMNYSampleStorage: measured status now packs status *and* the beamtime
  account into one VAL -- 0 = not measured, negative 5-digit = started,
  positive 5-digit = completed, falling back to FALLBACK_ACCOUNT (11111)
  for anything that doesn't parse as exactly 5 digits. No IOC schema
  change needed (same ai record). New get_sample_account()(+gripper/
  samplestage variants); set_sample_status() now takes an account param;
  show_all()/help() updated. otransfer_* CLI wrappers updated to match.
- Widget: new P-touch label-print offer, but only when clearing a
  *shuttle* slot (not fixed-O/stage/gripper -- those are bookkeeping
  corrections, not physical sample removal, since only shuttle samples
  leave the vacuum chamber). Uses a separate PTouchLabelPrinter instance
  at template 3 (not yet bench-tested against a real printer, unlike
  FlOMNI's confirmed-working template 1), same field names as FlOMNI.
- AI_docs/OPEN_ISSUES.md: noted that the measured-status+account encoding
  should be ported to FlOMNI at some point (separate task, not this
  branch's scope -- FlOMNI's own status tracking is untouched, still the
  redis-based _MeasuredSampleLog).
- New tests: encode/decode round-trips + fallback behavior for the
  status+account VAL scheme, plus oparkz leica-target coverage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
- omny_sample_transfer_mixin.py: add otransfer_confirm_dialog(), mirroring
  flomni.py's ftransfer_confirm_dialog() -- routes sample-transfer
  confirmations through the GUI console widget (already opened by
  _omnycam_parking()/_omnycam_samplestage()) when available, instead of
  always blocking on a terminal input() prompt. Falls back to the plain
  yesno() prompt if the camera view hasn't been opened yet. All 8
  self.OMNYTools.yesno() call sites in the transfer flow now go through it.
- simulated_omny.yaml: otransy's gripper-sensor check
  (_otransfer_check_sensor_connected(), @AN[8001]) always failed in sim --
  it fell back to the simulated Galil's shared default for unseeded analog
  channels (-8.0V, tuned for motor-temperature registers elsewhere), against
  an expected -2.3V +-0.5V. Added sim_analog_inputs: {8001: -2.3} to seed
  that specific channel.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
The bare YAML int key (8001:) broke config reload entirely: BEC's msgpack
config transport uses strict_map_key=True, which rejects non-string map
keys outright ("int is not allowed for map key"). Quoting it ("8001":)
matches the existing convention already used in simulated_flomni.yaml's
own sim_analog_inputs, and round-trips through msgpack correctly --
configure_axis() already does int(chan) internally, so the string key
still resolves to the same channel.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
_oshield_is_ST_open() reads a digital switch (@IN[8014]) on otransx's Galil
controller to confirm the sample-stage thermal shield reached its physical
"open" limit -- but oshield is a Smaract stage, simulated by a completely
separate simulator (sim_smaract.py), so that switch had no connection to
oshield's actual simulated position and always read a stale default.

sim_galil.py's @IN[8014] handler now computes the switch value live from
oshield's real position via SimStateRegistry (cross-simulator lookup),
instead of a disconnected default -- 1.0 within tolerance of the "open"
target (-11.88, _oshield_ST_open()'s own umv() target), 0.0 otherwise.

Also fixes oshield's simulated starting position: it started at 0 (neither
the "open" -11.88 nor "closed" 15.5), so _otransfer_stage_to_ST()'s
oshield_current < 15 closed-check always failed fresh. Now starts at 15.5,
matching the real idle/closed resting state.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
- gui_tools.py: _omnycam_parking()/_omnycam_samplestage() now share one
  _omnygui_show_omnycam_pair() helper that creates the two camera Image
  docks + console once and re-points them via Image.image(device=...)
  on later switches instead of tearing down and rebuilding the whole
  GUI every time (bec_widgets' Image.image() already supports being
  called again to swap the monitored device). Also stops live mode on
  the outgoing camera pair on a switch, and drops the dead
  _omnycam_check_device_exists no-op guard in favor of the `in dev`
  idiom already used for cam_xeye in this file.
- omny_sample_transfer_mixin.py: the two remaining blocking
  input("Ready. Press Enter...") prompts in otransfer_put_sample/
  otransfer_get_sample now go through otransfer_confirm_dialog() like
  every other transfer confirmation, so they show up in the GUI console
  instead of blocking on a terminal keypress; fixed the get_sample
  prompt's copy-pasted "mount" wording to "unmount".
- simulated_omny.yaml: otransx sim_velocity 10->100 mm/s; oshield
  gained sim_velocity: 10.0 (previously unset, defaulting to the
  Smaract sim's 1 mm/s) -- both x10 per request.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
- shutter.py (OMNYFastShutter): drop the "use dev.fsh if configured, else
  fall back to a local dummy signal" branching -- always delegate to the
  'fsh' device. Removed the now-unused MonitorSignal class and the
  dummy shutter Cpt along with it. Rewrote test_omny_shutter.py to assert
  delegation to a mocked 'fsh' device instead of exercising the removed
  dummy fallback.
- omny_sample_transfer_mixin.py: picking a sample up (get_sample, three
  source kinds: shuttle slot, fixed position, sample stage) only carried
  the sample name into set_sample_in_gripper(), silently dropping owner
  (defaulted to "") and measured status/account (reset to "none" by
  set_sample_in_gripper() itself). Same gap in reverse on put_sample.
  All six call sites (three pick, three put) now read owner/status/account
  from the source before clearing it and carry them onto the destination.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
The action replaces both name and owner and resets the slot's measured
status/account back to "none" -- "change name" undersold what it
actually does. Renamed action_change_name() -> action_change_sample()
to match, and documented the status-reset behavior in its docstring.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Requested revert of one call site from the earlier GUI-console-confirm
sweep: this confirmation is about the operator physically checking the
leica transfer's back position, not something they'd be watching the
camera GUI console for, so it stays a plain terminal yesno() prompt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
otransfer_park_loadlock_slot() (_oparkz_leica_slot()) parks oparkz at a
different offset (base -6) than otransfer_park_slot() (base -0.9) for
the same slot numbers. _oparkz_slot_check() -- used throughout the
transfer flow, including otransfer_put_sample()/otransfer_get_sample()'s
availability checks -- only recognized the regular-gripper (-0.9) table,
so after parking via the loadlock (as the GUI correctly showed "ACTIVE"
for, per the earlier oparkz_active_parking_slot() widget fix), any
transfer call still raised "Parking stage is not at an active parking
position." Now checks both tables, mirroring sample_storage.py's
oparkz_active_parking_slot() on the widget side.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
_otransfer_shuttle_align()'s own comment documents this as "a routine
for broken hardware": on real hardware, driving the shuttle aligner
(oshuttlealign, axis H) positive of its forward-reference zero also
trips the low/reverse limit switch, due to how the two switches are
wired -- not just the normal reference-axis routine. The sim had no
concept of this: plain PR/BG moves never update limit_low_active/
limit_high_active at all (those flags are only ever set by the
separate #FES/#FRM reference-axis path), so the routine's final
"searchwrongendswitch" search loop (a 22-iteration, +200-step-at-a-time
hunt for the low switch) could never find it and the whole
otransfer_put_sample()/otransfer_get_sample() flow hung until RPC
timeout, raising "The shuttle aligner did not move back."

Added a per-axis opt-in flag (quirky_positive_trips_low_limit, wired
through configure_axis()/_sim_state_kwargs()/SimOMNYGalilMotor) that
makes the simulated low limit switch position-derived (active whenever
position > 0) instead of purely flag-based, and set it only for
oshuttlealign in simulated_omny.yaml. Verified end-to-end against a
replay of the real command sequence (reference, +200 recovery nudge,
the five alignment excursions, then the search loop) -- the low switch
is now found within the 22-iteration budget, same as real hardware.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
- oshuttlealign: start the sim already at the forward/high (down)
  reference (sim_limit_high_active: true), matching real hardware's
  typical idle state, instead of re-driving to a limit it was already
  at on the first alignment each session.
- oshuttleopen: _otransfer_shield_open() drives it open with plain
  PRG=17000/PRG=37000 (never through the naxis/XQ#NEWPAR reference-axis
  routine that normally flips limit-switch flags), so
  _otransfer_is_shuttle_open()'s high-limit check could never see it as
  open -- "Did not manage to open the lid" fired even though the move
  completed. Added a general, opt-in position-derived high-limit
  mechanism to the Galil sim (high_limit_trip_position_steps, wired
  through configure_axis()/_sim_state_kwargs()/SimOMNYGalilMotor) and
  set it for oshuttleopen at 1.0 mm (comfortably below the ~1.05 mm/
  54000-step total commanded excursion).

Verified both against a standalone replay of the real command
sequences before writing tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
- sim_omny.py: SimOMNYSampleStorage now seeds shuttle A (which already
  holds the default sim_sample_1/2/3) as parked at slot 6 -- saves
  calling dev.omny_samples.set_shuttle_slot('A', 6) by hand at the
  start of every test session. (oshuttlealign already starts
  pre-referenced/down from the previous commit's sim_limit_high_active
  change -- confirmed still in place, no further change needed there.)
- omny_general_tools.py: gui_yesno() no longer appends a "[y]/n?"-style
  text hint to the message -- the ConsoleButtonsWidget already shows
  actual Yes/No buttons, so the text was redundant (only the
  terminal-only yesno() prompt still needs it).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
The "Managing sample storage" section documented dev.omny_samples.*
calls as the normal way to view/edit storage, but day-to-day usage is
now the OMNY Sample Storage GUI widget (added from BEC Widgets' widget
palette) -- documented this session's right-click actions (Change
sample.../Clear/New sample.../Set parking slot...), the measured-status
badges, and that measured status is intentionally CLI/scan-only, not
settable from the widget.

Split into three subsections: widget usage (primary), sample transfer
(still CLI -- the widget only edits the record, it doesn't drive
hardware), and a "CLI reference for sample storage" section for
scripting/GUI-unavailable cases, updated with the owner parameter and
the measured-status+account otransfer_* commands added this session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Pre-existing bug from 55aa712 (before this session): confirming "yes,
disable the feedback" raised OMNYTransferError instead of proceeding --
inverted condition, missing a `not`. Same class of bug as the
otransfer_park_loadlock_slot fix earlier this session. Audited every
other otransfer_confirm_dialog() call site in this file; this was the
only inverted one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
_otransfer_save_reference_images() was only called from _otransfer_confirm()'s
"All OK?" checkpoint. Now also called at otransfer_get_sample()'s "no sample
in gripper" and "ready to start the unmount process" gates, and
otransfer_put_sample()'s "ready to start the mount process" gate -- all
three already run after the camera pair is live, so no extra setup needed.

Checked flomni.py for the equivalent gap per request: there isn't one --
ftransfer_get_sample()/ftransfer_put_sample() have no manual confirm
dialog before starting mount/unmount at all (just an automated check +
raise), so there's no existing flomni gate missing an image-save call.
Documented in AI_docs/OPEN_ISSUES.md, including a forward-looking note in
case flomni ever gains an interactive gripper-empty confirm of its own.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Both _otransfer_check_free_slot_available_at_position() and
_otransfer_check_sample_available_at_position() distinguish "slot
occupied/empty" from "no shuttle parked at the active parking
position" internally (via print()), but only returned bool -- so the
raised OMNYTransferError always said "position N is not free"/"no
sample available at position N" even when the real cause was no
shuttle being parked at oparkz's current slot at all, which reads as
if the pin itself were the problem. Widened both messages to cover
either cause, per Mirko's simplest-fix suggestion.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Explores classical CV (registration + edge/SSIM/contour matching against
multiple reference images) to reduce the manual y/n confirms in
otransfer_get_sample()/otransfer_put_sample(), reusing the existing
reference-image capture and the push_smear_preview() GUI-overlay pattern
already proven for LamNI's rotation-center calibration. Exploration only;
implementation to follow on a separate branch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Reworks the plan per feedback: instead of building CV logic into
IDSCamera / inline in the sample-transfer mixin, propose a standalone
SampleVisionMonitor device whose only dependency is the get_last_image()
contract already shared by IDSCamera and AlliedVisionAravisCamera, wired
to its camera(s) by configured name (ddg_1.py's device_manager.devices.get()
pattern, generalized). Same class is then reusable for any endstation's
camera(s) via config alone.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Per feedback: split the design into three layers instead of one
confirm-scoring device. Layer 0 is a plain image-manipulation toolkit
(grayscale, edge detection, registration, overlays) with no device/BEC
dependency. Layer 1 is a new generic VisionInterface device -- camera in,
processed image out, mode selectable at runtime -- usable as an
in-between interface for any camera consumer, not just sample-transfer
confirms; x_ray_eye_align.py's alignment GUI is called out as a second,
independent consumer. Layer 2 (SampleVisionMonitor, the sample-transfer
confirm/scoring device from the previous pass) is now framed as one
application built on top of Layers 0/1.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Per feedback: the vision device(s) should be operable from the command
line as their own separately-addressable interface, not gated behind
GUI widget work. Notes this falls out of BEC's normal device-RPC model
for free (same mechanism get_last_image() already uses client-side),
and proposes thin omny.ovision_* wrapper methods matching the existing
otransfer_* wrapper convention around dev.omny_samples.*. Downgrades the
GUI mode-picker open question to strictly additive, not a prerequisite.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Mupip8x7Hean6E3FFvpbj
Conflicts resolved:
- omny_fermat_scan: static get_omny_fermat_spiral_pos keeps cenx/ceny/zshift
  and main's seeded random offset; prepare_scan passes the offsets again
  (update-omny had dropped them); corridor cap kept, exp_time 0.1.
- tomo_params: per-profile tomo_types with an angle_file flag (flomni/lamni
  type 4 'Angles from file'), no global TOMO_TYPES.
- sample_storage widget: _MeasuredSampleLog removed (main), shuttle status
  labels kept (update-omny).
- tomo_queue_mixin, omny.py, omny.md: both sides kept.
- test_omny_fermat_scan: main's elongated-FOV test adapted to the static
  signature; cenx test uses a fixed seed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TNtKEk4jAyjebMDQFwRHJQ
OMNY sample storage on the OMNY soft IOC records (OMNY-SAMPLE:<slot>:...)
CI for csaxs_bec / test (push) Failing after 4m40s
CI for csaxs_bec / test (pull_request) Failing after 4m35s
89ba28550c
OMNYSampleStorage follows FlomniSampleStorage: one record per slot
(USED/NAME/OWNER/STATUS/MEAS_DATE; slots A1..C6, P10..P101, STAGE, GRIPPER,
PARK1..6), served by OMNY's caproto IOC (OMNY repo preemt_rt,
OMNYtools/softioc/ioc_omny.py). Replaces the XOMNY-SAMPLE_DB_* records and the
status x account VAL packing.

- device: slot_name(), transfer_sample() (whole record moves),
  set_sample_measured(), get_sample_record()/get_sample_meas_date(); the
  (container, slot_nr) API is kept; the 0-20 slot check that rejected 32-34/
  100/101 is gone.
- transfer mixin: picks/puts use transfer_sample(); otransfer_* wrappers for
  the new methods, the account wrappers removed.
- omny.py tomo_scan(): started/completed + month on the STAGE record (as flOMNI).
- sample_storage widget: bulk-read parser and writes on the new records; label
  prints owner as account and the stored date (as flOMNI).
- tomo_params: sample name from sample_names.STAGE.
- SimOMNYSampleStorage, tests, user docs.

Checked against a local ioc_omny.py (real EPICS, localhost): long names,
status enum, transfers, parking.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TNtKEk4jAyjebMDQFwRHJQ
holler closed this pull request 2026-10-19 13:41:25 +02:00
holler reopened this pull request 2026-10-26 13:27:53 +01:00
holler added 1 commit 2026-10-26 13:46:01 +01:00
tests(omny): update three OMNY tests to the code merged from main
CI for csaxs_bec / test (pull_request) Successful in 4m34s
CI for csaxs_bec / test (push) Successful in 4m35s
7c8b2a50ad
- tomo_queue_execute() resumes with tomo_scan_resume(from_queue=True)
- tomo_id is a _GlobalVarParam: give the bare OMNY object a fake client
- _irrelevant_params_for_type(): pass the omny profile; without it the
  setup-neutral fallback treats type 4 as an angle file (flomni/lamni)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
holler added 1 commit 2026-10-26 13:51:58 +01:00
Merge branch 'main' into feat/omny-sample-ioc
CI for csaxs_bec / test (push) Successful in 4m38s
CI for csaxs_bec / test (pull_request) Failing after 4m40s
b0d9b796c7
holler added 1 commit 2026-10-26 13:56:36 +01:00
Merge branch 'main' into feat/omny-sample-ioc
CI for csaxs_bec / test (push) Failing after 3m15s
CI for csaxs_bec / test (pull_request) Failing after 3m29s
8a2ec8ee7d
holler added 1 commit 2026-10-26 14:01:12 +01:00
fix(tomo_params): pin seed=0 in the OMNY fermat position estimate, as flomni/lamni
CI for csaxs_bec / test (pull_request) Successful in 3m29s
Read the Docs Deploy Trigger / trigger-rtd-webhook (push) Successful in 3s
CI for csaxs_bec / test (push) Successful in 3m39s
96c02b3367
get_omny_fermat_spiral_pos() draws a random sub-step offset per call, so the
estimate (and its test against a second call) was not reproducible.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
holler merged commit 96c02b3367 into main 2026-10-26 14:07:56 +01:00
holler deleted branch feat/omny-sample-ioc 2026-10-26 14:07:56 +01:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bec/csaxs_bec#334