csaxs_bec/docs/SIMULATED_ENDSTATIONS.md

Simulated Endstations for cSAXS BEC (flOMNI / LamNI / OMNY)

Status: implemented and validated (2026-07). flOMNI extensively live-tested against a real BEC deployment; LamNI/OMNY validated by offline harness, live smoke test pending.

Purpose

Full software development and testing of the flOMNI/LamNI/OMNY control code without hardware and without a running instrument. The real ophyd device classes, controllers, scan plugins, GUI widgets, and IPython-client scripts run unchanged; only the wire protocols are simulated. Swapping a device config is the only difference between the simulated and the real system.

Architecture: protocol-level socket injection

All three endstations funnel every hardware interaction through socket-based controllers (ophyd_devices.utils.controller.Controller), which accept a socket_cls constructor parameter. The simulation injects SocketIO-compatible classes that implement the wire protocols as in-process state machines:

• Commands are dispatched synchronously inside put(); replies go to a queue drained by receive(). Commands that are reply-less on the real hardware must not enqueue replies (the main desync risk of naive mocks) — reply behavior follows the real server sources.
• Simulation state is shared per (host, port) via SimStateRegistry, mirroring the controller singleton. Device subclasses seed axis parameters at construction.
• The sim device classes are thin subclasses of the real ones whose only jobs are (a) defaulting socket_cls and (b) seeding sim state from sim_* deviceConfig keys.
• Cross-device choreography (e.g. flOMNI feedback_enable_with_reset touching fsamroy, fsamx and the ftrackz Galil) runs through the real Python code against each controller's own sim socket — no logic is duplicated.

Protocol fidelity is grounded in the actual hardware-side sources: fgalil.dmc, galil_micos_upr.dmc (FUPR), lamni.dmc, controller1.dmc (OMNY), flOMNI CommunicationServer.cpp, LAMNI_server.cpp, OMNY_server.cpp.

File inventory (all under csaxs_bec unless noted)

• devices/sim/sim_socket.py — SimSocketBase + SimStateRegistry
• devices/sim/sim_galil.py — shared Galil DMC state machine (motion at constant velocity, threads, referencing #FES/#FRM/#REFAX, gripper transfer routines #GRGET/#GRPUT with confirm handshake and atconfir autoconfirm, #Tracker, #MNTMODE/#POSMODE, #STOP, AB/ST, direct axis commands PA/PR/BG/SH/MO/AC/DC/SP/LD, encoder axes TP/encpermm, dialect variables via sim_variables) plus SimFlomniGalilMotor, SimFuprGalilMotor
• devices/sim/sim_smaract.py — Smaract MCS protocol + SimSmaractMotor (serves flOMNI fosa*, LamNI losa*, OMNY ocs*/oshield)
• devices/sim/sim_rt_flomni.py — flOMNI Orchestra server (feedback l0/l1/l2/l3, tracker T/Ts, scans s/sc/sd/sr, 10-field sample rows, paced by sim_point_dwell_s)
• devices/sim/sim_lamni.py — LamNI Galil dialect (swver 28, encoder axes 0–5, air-bearing interlocks) + LAMNI server (J0/J1/J5/J6, J2/J4/J7, As/Ar cap sensors, Ss/Sr sampler, V velocity, 21-field rows)
• devices/sim/sim_omny.py — OMNY Galil dialect (swver 134, mntax=4, signed mntprgs, tempab/timeab/IgNoFol, grstat/relpos/toppin, #GRSTAT/#CRESET/#SAFETY, NETIO power switch stubbed always-on) + OMNY server dialect (J2 six fields, y→1 semantics, 19-field rows) + patched-EPICS SimOMNYSampleStorage/Dewar/Temperatures
• devices/sim/sim_cameras.py — SimWebcamViewer / SimIDSCamera: real camera classes with synthetic frame source (crosshair + gaussian blob); optional per-frame noise via sim_noise_std (default 0 = static, cheap over remote desktop)
• devices/sim/sim_sample_storage.py — SimFlomniSampleStorage (patched EPICS, seedable via sim_samples, default slots 0–3 populated)
• device_configs/simulated_omny/ — all simulated configs live here: simulated_flomni.yaml (hand-maintained), simulated_lamni.yaml and simulated_omny.yaml (generated), simulated_bl_endstation.yaml (pre-existing)
• bin/generate_simulated_configs.py — regenerates the LamNI/OMNY configs from ptycho_lamni.yaml/ptycho_omny.yaml (rerun after real-config changes)
• tests/sim_flomni_harness.py, tests/sim_lamni_omny_harness.py — offline harnesses (no BEC deployment needed): 39 + 26 checks covering moves, referencing, feedback, tracker, scans through the flyers, gripper transfers, storage, cameras

Real-code fixes made along the way (separate commits)

• fgalil_ophyd.py and galil_ophyd.py (base class, affects LamNI/OMNY): removed the trailing thread-0 wait in the setpoint _socket_set — it blocked user_setpoint.put() (holding the controller lock) for the whole move, so no intermediate readback/progress was possible. Applies to the real instrument too.
• ddg_1.py: bec_server.scan_server...ScanInfo import moved under TYPE_CHECKING (typing-only runtime dependency broke against the redeployed BEC layout).
• flomni.py: all four scilog sends wrapped in try/except RuntimeError — accounts without scilog no longer crash tomo_scan/alignment/timing reports.

Delivery policy

Tarballs contain ONLY simulation-package files (devices/sim/* except simulated_beamline_devices.py, device_configs/simulated_omny/*, generator, harnesses, this doc) plus the two agreed device-layer fixes (galil_ophyd.py, fgalil_ophyd.py, ddg_1.py). Fixes to user-maintained files (flomni.py, ...) are delivered as diffs only — never as full files, since the working copies evolve during testing.

BEC integration lessons (important for future sim devices)

1. Explicit signatures required: the device server passes only deviceConfig keys that appear as named parameters in inspect.signature(dev_cls), and injects device_manager only if named. *args/**kwargs-only subclasses silently lose their entire config.
2. msgpack strict_map_key: no integer keys anywhere in deviceConfig — quote channel numbers ("1": -1.6).
3. Hidden second transports exist outside the socket seam (OMNY NETIO power-switch HTTP, camera MJPEG/pyueye) and need their own stubs.

Deliberate simulation semantics ("working hardware", no error injection)

• Axes boot referenced (real init zeroes at limits; set sim_referenced: false to exercise init procedures) at positions from in user params, clamped into limits.
• allaxref/axisref[n] are genuine writable DMC variables (forcing allaxref=0 makes moves refuse, as on hardware); recomputed by referencing routines.
• Constant-velocity motion with per-axis speeds from the DMC #INIT tables; RT scans advance at sim_point_dwell_s per position (default 20 ms) so ETA/timing logic is meaningfully testable.
• Sampled scan rows carry targets + ~2 nm gaussian noise.
• LamNI: the interferometer readback is coupled to the coarse stages — with feedback off, measured (x, y) = lamni_from_stage_coordinates(lsamx-center, lsamy-center)*1000 um (geometry constants imported from LamNIFermatScan). This makes lamni_new_scan_center_interferometer converge. Coupling is configured via sim_coarse_coupling on rtx/rty/rt_positions (generator fills it from the lsamx/lsamy config). lsamx/lsamy boot at their 'center' user parameters; speeds follow the lamni.dmc axspeed table.
• A LamNI fermat scan re-enables feedback itself (feedback_enable_without_reset in the scan plugin) — starting a scan with user-disabled feedback therefore works, on the real system as well as in the sim; it only fails on real hardware when feedback cannot engage (air on, interferometer error), an error mode deliberately not simulated.
• Referencing (#FES/#FRM/#REFAX) applies its outcome synchronously (limit switch, position 0) while only the apparent motion is time-based, so status polls never race the result.
• Transfer confirm handshakes are real (GUI + gripper-camera flow works); set atconfir=1 on the controller for unattended runs.
• #GRSTAT always reports success; NETIO always on; unknown Galil commands reply : and log [sim galil] unhandled warnings — grep the device-server log after a session to find fidelity gaps worth adding (full DMC command-reference emulation was deliberately not pursued).
• LamNI sd reply uses the detector-trigger wording; sd<cycles> internal-timing variant not modeled. sr replies are integer-formatted per LAMNI_server.

Live-test status (flOMNI, testing account, redeployed BEC)

Passed: sample storage + GUI, moves incl. live progress bars and Ctrl-C, rotation→RT angle coupling, feedback enable/disable/status, tracker, fermat scans incl. progress and abort, X-ray eye GUI with camera frames, storage bookkeeping. Pending: full transfer chain end-to-end, tomo_scan (unblocked by the scilog fix), tomo queue executor (crash-resume checklist), then queue-reordering implementation. LamNI/OMNY: harness- validated; a short live smoke test (config load, a move, one scan, one OMNY transfer) recommended before relying on them, since flOMNI testing surfaced integration issues the harness cannot see.

Operational notes

• Deploy: extract the tarball in the repo root (overwrites sim files + the three fixed real files). Remove stale device_configs/simulated_flomni/ and simulated_lamni/ folders from earlier iterations if present.
• Sim tuning knobs (deviceConfig): sim_initial_position, sim_velocity, sim_stppermm, sim_encpermm, sim_referenced, sim_analog_inputs, sim_digital_inputs/outputs, sim_variables, sim_limit_low/high_active, sim_point_dwell_s, sim_noise_std, sim_samples.
• Harnesses run standalone: python tests/sim_flomni_harness.py (repo on PYTHONPATH).