diff --git a/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/Webpage.md b/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/Webpage.md new file mode 100644 index 0000000..18554f4 --- /dev/null +++ b/csaxs_bec/bec_ipython_client/plugins/flomni/AI_docs/Webpage.md @@ -0,0 +1,239 @@ +# flOMNI / cSAXS status monitoring — development handoff + +Consolidated context for continuing development in Claude Code. Supersedes nothing; +read alongside `project_briefing_update5.md` and `multi_setup_webpage_architecture.md` +for the deepest detail on the last session. + +--- + +## 1. What this project is + +A modern Python-based experiment status monitoring system for the **flOMNI** +ptychographic tomography instrument on the **cSAXS beamline (X12SA)** at the +**Swiss Light Source (SLS), PSI**. It replaces a legacy bash/spec-based approach. + +Purpose: give colleagues a live view of experiment status while monitoring +remotely, via a public-facing authenticated web page with audio alerting. + +Owner: Mirko (beamline scientist/engineer, PSI). + +### Repos +- `gitea.psi.ch/OMNY/BEC` — Python / BEC side +- `gitea.psi.ch/OMNY/Webserver` — web server files + +--- + +## 2. Architecture + +``` +┌──────────────────────────────┐ ┌────────────────────────────┐ +│ PSI internal network (trusted)│ HTTPS │ omny.online (IONOS) │ +│ │ POST │ lighttpd + PHP │ +│ BEC iPython session ├────────►│ - auth_gate.php / login.php│ +│ flomni_webpage_generator.py │ │ - status.html / status.json│ +│ (polls BEC every ~15s) │ │ - IP-whitelisted upload │ +└──────────────────────────────┘ └────────────────────────────┘ + ▲ + fallback: Raspberry Pi "tamipi" + (FritzBox DynDNS) +``` + +### Producer — `flomni_webpage_generator.py` +- Background service inside a **BEC iPython session**. +- Polls BEC client objects every **~15 s**, generates `status.html` + `status.json`, + uploads via **HTTP POST** to `omny.online`. +- Plugin architecture: + - `WebpageGeneratorBase` → `FlomniWebpageGenerator` (and stub `LamniWebpageGenerator`) + - Factory `make_webpage_generator()` selects the plugin by BEC session name. +- **`stop()` / `start()` cycle is required after any edit** to regenerate and + re-upload `status.html`. + +### Web server +- **lighttpd** on IONOS Webhosting Plus (`omny.online`); Raspberry Pi 3B + ("tamipi") behind FritzBox DynDNS as fallback. +- **PHP auth**: HMAC-SHA256 signed cookies. Two htpasswd files: + - `users.htpasswd` — permanent beamline staff + - `session.htpasswd` — the currently active rotating e-account (e.g. `p23092`) +- **Upload endpoint** IP-whitelisted to the PSI subnet `129.129.122.0/24`. +- **SSL**: Let's Encrypt via **webroot** authenticator. (Standalone previously + caused *silent* renewal failures — do not go back to it.) +- **Cookie generation-binding**: a monotonic `session.gen` counter invalidates old + session cookies after an e-account rotation. Fixed staff users get sentinel + `-1`, exempting them from the generation check. + +### Monitoring +- Raspberry Pi crontab `*/5 * * * *` runs `omny_upload_monitor.sh`, checking + `status.json` age against a 300 s threshold. + +--- + +## 3. BEC internals (hard-won, do not re-derive) + +- **Scan activity**: `primary.info[0].active_request_block is not None`. + Do **not** use the queue `status` field — it is always `'RUNNING'` while BEC is alive. +- **Beamline states and scan interlock are independent.** + `bec.builtin_actors.scan_interlock.states_watched` returns **accepted** (not blocking) statuses. +- **`blocked` status** uses `primary.locks`. +- **A block interrupts the running scan**, clearing `active_request_block` + (→ `queue_has_active_scan = False`). So `_derive_status()` must return `blocked` on + `beamline_blocking OR queue_locks` **independent of active scan**; only a + fresh-heartbeat `scanning` outranks it. +- `beamline_blocking = (enabled is True) AND any(state mismatched)` — gated on + `enabled` so a disabled/unknown interlock never raises BLOCKED. +- **Tomo projection count**: + - type 1: `floor(range / stepsize) * 8` — **truncation, not rounding**; 8 sub-tomograms. + - types 2/3: `golden_max_number_of_projections`. + - Now generalised in JS as `calcProjections()` with kinds `equally_spaced_grid` + and `golden_capped`, driven by the `TOMO_TYPES` data blob. + +--- + +## 4. Multi-setup design (current) + +Capability flags live as **class attributes** on `WebpageGeneratorBase` +(not constructor args — so startup code is unchanged): + +- `HAS_TOMO_QUEUE: bool` — gates the `tomo_queue` global-var read in `_cycle()` + and the "Tomo queue" card in `_render_html()`. +- `TOMO_TYPES: dict` — declarative config, embedded as JSON into the page; drives + `calcProjections()`. + +Defaults reproduce flomni behavior exactly. BEC's own primary scan queue +(`queue_status` / `queue_locks`) and beamline states are **common — not gated**. + +**Principle**: setup-specific behavior = class-level capability flag / declarative +config with a flomni-preserving default. Never a hardcoded per-setup branch in JS. + +**Principle**: when one concept appears in two places (e.g. "blocking" on the pill +*and* the card; params in the queue *and* the instrument pane), derive both from a +single identically-defined signal or shared helper (`buildParamRows()`) so they +cannot drift. + +--- + +## 5. Current sound map (flomni) + +| Transition | Sound | +|---|---| +| `scanning → idle` (normal finish) | rising success arpeggio (C5-E5-G5), repeats every 30 s until Confirm | +| `scanning → running` / other non-blocked stop | falling `warningChime`, 30 s until Confirm | +| `scanning → blocked` | **silent**; Blocked LED pulses; `blockedChime` only after 60 min, then hourly, auto-clears | +| feed stale / 3 failed fetches | high triple pip, 30 s until Confirm feed | +| Enable/test button | 880-1100-880 | + +--- + +## 6. Known-good implementation gotchas + +- **iOS Web Audio**: must use a **synchronous** silent 1-sample `BufferSource` + started directly in the gesture handler, plus fire-and-forget `ctx.resume()` with + ~80 ms `setTimeout` offsets. Promise `.then()` chains **fail** outside WebKit's + gesture scope. +- **f-string escaping**: all JS braces must be doubled (`{{`, `}}`) in Python + f-string HTML generation. +- **lighttpd**: negative-lookahead regex for pass-through rules is unreliable — use + explicit pass-through entries. `%0` in redirect rules evaluates empty if the + parent regex didn't match — use a hardcoded hostname + `$0`. +- **`handle_error` placement**: belongs on the **server class** (`BaseServer` + subclass), not on the request handler — overriding it on the handler has no effect + for this error path. +- **Mobile viewport**: `body { min-height: 100dvh }` (with `100vh` fallback); plain + `100vh` on iOS leaves empty scroll room below the page. +- **File discipline**: editing stale base files has been a recurring, costly error. + **Always verify feature presence with `grep -c` before editing any file.** + +--- + +## 7. Security model + +- PSI internal network is **trusted**. The primary threat is the **inbound path** + from the public web server back into the internal network. +- Key risks and mitigations: + - HTTP redirect-following in `requests.post()` → SSRF. Mitigated with + `allow_redirects=False`. + - DNS rebinding on the DynDNS hostname when `verify=False`. + - Unbounded response-body reading. +- `session_query.php` / `set_password.php` form the **one** feedback loop where the + trusted side acts on public-server responses — treat with care. +- **Principle**: implement the safest correct solution, not a minimal patch. + +--- + +## 8. Open items + +### Verify on next flomni restart +- **Blocked fix**: on the next real block, confirm the pill flips to BLOCKED and + the LED pulses (this was the reported bug). +- **`_CURRENT_PARAM_KEYS`**: 7 names are proven (`fovx`, `fovy`, `stitch_x`, + `stitch_y`, `tomo_shellstep`, `tomo_countingtime`, `tomo_angle_stepsize`). + 5 are **assumptions** to verify as global vars — `tomo_type`, `tomo_angle_range`, + `frames_per_trigger`, `single_point_instead_of_fermat_scan`, + `ptycho_reconstruct_foldername`. Any that isn't a real global var is silently + skipped (no row); remap in `_CURRENT_PARAM_KEYS` if a name differs. +- **Projections parity**: the queue-card Projections number for the next job should + be identical to before (`floor(range/step) * n_subtomos`). +- **Let's Encrypt auto-renewal**: confirm it is functioning (~90 days after the last + manual renewal). + +### Producer-side, not yet implemented (from briefing 4) +- `estimated_finish_time` in `flomni.py` +- `tomo_start_scan_number` in `flomni.py` +- `set_web_password()` shortcut on the `Flomni` class in `flomni.py` + +### EPS / machine status page (in progress) +- **Starting point**: `X_X12SA_BLStatus.ui` — flat, macro-free caQtDM beamline status + panel covering undulator gap, EPS permit, vacuum, temperatures, monochromator / + mirror / slit positions, BPMs, filter states. +- **Decided architecture**: + - **Separate daemon**, decoupled from the main webpage generator. + - **CA monitors preferred over polling.** + - **Two-tier update cadence**: fast (shutters, EPS-OK, ring current) / slow + (vacuum, temperature). + - **Card/table layout first**; SVG synoptic deferred. +- **Blocked on**: ~110 unresolved PVs from `caInclude` sub-templates + (`EPS_Temp.ui`, `EPS_Gauge.ui`, `EPS_IonPump.ui`, `EPS_Valve.ui`, `EPS_BST.ui`, …) + — need the template files. +- **Next deliverables requested**: `index.html`, the JS, and the JSON schema files, + so the PV manifest and `eps_status.json` schema can be built. +- **Consider**: the PSI **EPICS archiver REST interface** to sidestep CA firewall + concerns. + +### Deferred +- **LamNI**: fill in `LamniWebpageGenerator` — real `tomo_type` key values in + `TOMO_TYPES`, device paths / `_collect_setup_data()`, `LamNI.png`. Add a + session-name branch to `make_webpage_generator()`. +- **Omny**: new subclass; `TOMO_TYPES` with a public 2-subtomo type plus a hidden + 8-subtomo type restricted to staff. The staff-only visibility mechanism is + **still to be designed** (likely a role/visibility field filtered out before the + config reaches a standard user's page). Add to the factory. +- **SVG synoptic view** for the EPS page. +- **Edge case (noted only)**: `scanning → blocked → idle` directly (block clears + exactly as the scan finishes) yields no success tone, since the transition into + idle came from `blocked`. Normal flow clears back to `scanning` first, so the + success tone fires on the real finish. Revisit only if BEC ever exposes an + explicit scan-aborted signal. + +--- + +## 9. Tools & environment + +- **Languages**: Python (BEC iPython env), PHP (lighttpd), JavaScript (frontend), Bash +- **Python libs**: `requests` (upload), `Pillow` (thumbnails), `http.server` + (local server, port 8080) +- **PHP**: `password_hash()` / `password_verify()` (bcrypt) +- **Beamline systems**: EPICS (PVs), caQtDM (`.ui` panels), BEC client + (`bec.queue`, `bec.get_global_var`, `bec.builtin_actors.scan_interlock`) +- **Infra**: lighttpd, certbot / Let's Encrypt (webroot), IONOS Webhosting Plus, + Raspberry Pi 3B, PSI Gitea + +--- + +## 10. Working style + +- Discuss architecture and confirm technical details **before** implementing code. +- Live testing happens on the actual beamline system; results are reported back + iteratively. +- Write a project briefing `.md` after major sessions to enable seamless continuation. +- Prefer clean, non-redundant UI; be decisive about **removing** elements rather than + replacing them with alternatives. +- Decoupled daemon architecture for new monitoring components.