From fcd51399c696d9757292f685df865c3d330d4c03 Mon Sep 17 00:00:00 2001 From: Mirko Holler Date: Sun, 6 Sep 2026 20:14:41 -0300 Subject: [PATCH] docs(omny): fix nonexistent config-helper call; document reading diagnostic_image bec.config.add_to_session(...) doesn't exist on ConfigHelperUser -- the real method is update_session_with_file(), the only one that fits this howto's "merge a file into the current session" use. Found live-testing the howto against a running session for the first time. Also: dev.vision_test_interface.diagnostic_image.get() raises AttributeError client-side -- PreviewSignal/BECMessageSignal hardcodes rpc_access=False, so it never appears as a client-side device attribute at all. Documented the actual read path (bec.connector.get_last(MessageEndpoints.device_preview(...))), verified against tests/end-2-end/test_scans_e2e.py::test_device_preview's identical pattern for eiger/preview. Status section updated to reflect the ROI addition from the previous commit. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01C9zgK28uPoZWJFkJWV4P1G --- .../MACHINE_VISION_AUTOCONFIRM_PLAN.md | 15 ++++++----- .../AI_docs/VISION_PIPELINE_TESTING_HOWTO.md | 26 ++++++++++++++----- 2 files changed, 28 insertions(+), 13 deletions(-) diff --git a/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/MACHINE_VISION_AUTOCONFIRM_PLAN.md b/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/MACHINE_VISION_AUTOCONFIRM_PLAN.md index f6c5ee83..277672a5 100644 --- a/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/MACHINE_VISION_AUTOCONFIRM_PLAN.md +++ b/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/MACHINE_VISION_AUTOCONFIRM_PLAN.md @@ -12,11 +12,13 @@ this plan grows out of — see its "full gripper sample-transfer chain... untest (`"none"`/`"translation"`/`"euclidean"`/`"affine"`, never raises on non-convergence), contour extraction/scoring, a separate position-sensitive `centroid_distance()` signal (kept independent of the shape score on purpose — see below), multi-reference best-match, contour overlay drawing. -- **Layer 1 (`csaxs_bec/devices/vision/vision_interface.py`, `VisionInterface`) — done, tested (14 - tests).** Config-driven `camera_names`, lazy per-call resolution via - `device_manager.devices.get()`, runtime-selectable mode (`raw`/`grayscale`/`edges`/`clahe`), own - `diagnostic_image` `PreviewSignal`, `refresh()`/`get_last_processed_image()`. Not yet wired into - any device config (no YAML entry, no real camera exercised) or into a GUI dock. +- **Layer 1 (`csaxs_bec/devices/vision/vision_interface.py`, `VisionInterface`) — done, tested.** + Config-driven `camera_names`, lazy per-call resolution via `device_manager.devices.get()`, + runtime-selectable mode (`raw`/`grayscale`/`edges`/`clahe`), a per-camera region-of-interest crop + (`set_roi()`/`get_roi()`, applied centrally before any processing/publishing/scoring -- see + `docs/developer/vision_pipeline_architecture.md`), own `diagnostic_image` `PreviewSignal`, + `refresh()`/`get_last_processed_image()`. Not yet wired into any device config (no YAML entry, no + real camera exercised) or into a GUI dock. - **Layer 2 (`csaxs_bec/devices/vision/sample_vision_monitor.py`, `SampleVisionMonitor`) — done, tested (15 tests).** Per-tag, per-camera in-memory reference store + `best_reference_match()` scoring across a `VisionInterface`'s cameras, worst-camera diagnostic overlay published via @@ -25,7 +27,8 @@ this plan grows out of — see its "full gripper sample-transfer chain... untest - **Testing tools — done.** `DummyVisionCamera` (`csaxs_bec/devices/vision/dummy_vision_camera.py`, a settable, hardware-free `get_last_image()` stand-in) + `device_configs/simulated_omny/vision_test.yaml` (two dummy cameras + one `VisionInterface` + one - `SampleVisionMonitor`, loadable into any running session via `bec.config.add_to_session(...)`) + + `SampleVisionMonitor`, loadable into any running session via + `bec.config.update_session_with_file(...)`) + `AI_docs/VISION_PIPELINE_TESTING_HOWTO.md` (step-by-step walkthrough). - **Finding, corrected after further testing**: an earlier pass here claimed `register_to_reference()`'s registration was what made pure translation "invisible" to scoring, diff --git a/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/VISION_PIPELINE_TESTING_HOWTO.md b/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/VISION_PIPELINE_TESTING_HOWTO.md index 45e62915..05ad2c62 100644 --- a/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/VISION_PIPELINE_TESTING_HOWTO.md +++ b/csaxs_bec/bec_ipython_client/plugins/omny/AI_docs/VISION_PIPELINE_TESTING_HOWTO.md @@ -15,14 +15,14 @@ devices at all) and merge the extra config in, without touching or replacing any loaded: ```python -bec.config.add_to_session( +bec.config.update_session_with_file( "csaxs_bec/device_configs/simulated_omny/vision_test.yaml" ) ``` -(`bec.config` is `bec_lib.config_helper.ConfigHelperUser` -- `add_to_session()` merges devices from -a file into the *current* session; it does not touch existing devices, so it's safe to run against -whatever session is already up.) +(`bec.config` is `bec_lib.config_helper.ConfigHelperUser` -- `update_session_with_file()` merges +devices from a file into the *current* session; it does not remove existing devices, so it's safe +to run against whatever session is already up.) After it returns, `dev.vision_cam_a`, `dev.vision_cam_b`, `dev.vision_test_interface`, and `dev.vision_test_monitor` should all be available, same as any other device. @@ -39,16 +39,28 @@ dev.vision_test_interface.get_raw_frame("vision_cam_a") # same array, via the i ## 3. Try the processing modes ```python +from bec_lib.endpoints import MessageEndpoints + dev.vision_test_interface.set_mode("edges") dev.vision_test_interface.refresh("vision_cam_a") # publishes to diagnostic_image -dev.vision_test_interface.diagnostic_image.get().data.shape # (200, 200) -- 2D edge map + +data = bec.connector.get_last(MessageEndpoints.device_preview("vision_test_interface", "diagnostic_image")) +data["data"].data.shape # (200, 200) -- 2D edge map dev.vision_test_interface.set_mode("raw") ``` +`diagnostic_image` is a `PreviewSignal` (a `BECMessageSignal`), and `BECMessageSignal` defaults +`rpc_access` to `False` -- it deliberately does not show up as `dev.vision_test_interface.diagnostic_image` +at all client-side (`dev..get()` only works for RPC-accessible signals). It's published as a +`DevicePreviewMessage` on the `MessageEndpoints.device_preview(device, signal)` Redis stream instead, +same as any other camera's preview signal (see `tests/end-2-end/test_scans_e2e.py::test_device_preview` +for the same pattern against `eiger`/`preview`) -- read it with `bec.connector.get_last(...)`, not +through the device proxy. + There's no GUI dock wired up for `diagnostic_image` yet (see the plan doc's open questions) -- to -actually *see* the published frame today, either pull `.data` from `diagnostic_image.get()` and -save it (e.g. `cv2.imwrite("/tmp/check.png", ...)`) or point a `bec_widgets` `Image` widget at +actually *see* the published frame today, either pull `.data` from the message above and save it +(e.g. `cv2.imwrite("/tmp/check.png", ...)`) or point a `bec_widgets` `Image` widget at `vision_test_interface`/`diagnostic_image` manually, the same way any other device's preview signal is wired into a dock.