add md describing webpage design
This commit is contained in:
x01dc
@@ -0,0 +1,239 @@
|
||||
# flOMNI / cSAXS status monitoring — development handoff
|
||||
|
||||
Consolidated context for continuing development in Claude Code. Supersedes nothing;
|
||||
read alongside `project_briefing_update5.md` and `multi_setup_webpage_architecture.md`
|
||||
for the deepest detail on the last session.
|
||||
|
||||
---
|
||||
|
||||
## 1. What this project is
|
||||
|
||||
A modern Python-based experiment status monitoring system for the **flOMNI**
|
||||
ptychographic tomography instrument on the **cSAXS beamline (X12SA)** at the
|
||||
**Swiss Light Source (SLS), PSI**. It replaces a legacy bash/spec-based approach.
|
||||
|
||||
Purpose: give colleagues a live view of experiment status while monitoring
|
||||
remotely, via a public-facing authenticated web page with audio alerting.
|
||||
|
||||
Owner: Mirko (beamline scientist/engineer, PSI).
|
||||
|
||||
### Repos
|
||||
- `gitea.psi.ch/OMNY/BEC` — Python / BEC side
|
||||
- `gitea.psi.ch/OMNY/Webserver` — web server files
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────┐ ┌────────────────────────────┐
|
||||
│ PSI internal network (trusted)│ HTTPS │ omny.online (IONOS) │
|
||||
│ │ POST │ lighttpd + PHP │
|
||||
│ BEC iPython session ├────────►│ - auth_gate.php / login.php│
|
||||
│ flomni_webpage_generator.py │ │ - status.html / status.json│
|
||||
│ (polls BEC every ~15s) │ │ - IP-whitelisted upload │
|
||||
└──────────────────────────────┘ └────────────────────────────┘
|
||||
▲
|
||||
fallback: Raspberry Pi "tamipi"
|
||||
(FritzBox DynDNS)
|
||||
```
|
||||
|
||||
### Producer — `flomni_webpage_generator.py`
|
||||
- Background service inside a **BEC iPython session**.
|
||||
- Polls BEC client objects every **~15 s**, generates `status.html` + `status.json`,
|
||||
uploads via **HTTP POST** to `omny.online`.
|
||||
- Plugin architecture:
|
||||
- `WebpageGeneratorBase` → `FlomniWebpageGenerator` (and stub `LamniWebpageGenerator`)
|
||||
- Factory `make_webpage_generator()` selects the plugin by BEC session name.
|
||||
- **`stop()` / `start()` cycle is required after any edit** to regenerate and
|
||||
re-upload `status.html`.
|
||||
|
||||
### Web server
|
||||
- **lighttpd** on IONOS Webhosting Plus (`omny.online`); Raspberry Pi 3B
|
||||
("tamipi") behind FritzBox DynDNS as fallback.
|
||||
- **PHP auth**: HMAC-SHA256 signed cookies. Two htpasswd files:
|
||||
- `users.htpasswd` — permanent beamline staff
|
||||
- `session.htpasswd` — the currently active rotating e-account (e.g. `p23092`)
|
||||
- **Upload endpoint** IP-whitelisted to the PSI subnet `129.129.122.0/24`.
|
||||
- **SSL**: Let's Encrypt via **webroot** authenticator. (Standalone previously
|
||||
caused *silent* renewal failures — do not go back to it.)
|
||||
- **Cookie generation-binding**: a monotonic `session.gen` counter invalidates old
|
||||
session cookies after an e-account rotation. Fixed staff users get sentinel
|
||||
`-1`, exempting them from the generation check.
|
||||
|
||||
### Monitoring
|
||||
- Raspberry Pi crontab `*/5 * * * *` runs `omny_upload_monitor.sh`, checking
|
||||
`status.json` age against a 300 s threshold.
|
||||
|
||||
---
|
||||
|
||||
## 3. BEC internals (hard-won, do not re-derive)
|
||||
|
||||
- **Scan activity**: `primary.info[0].active_request_block is not None`.
|
||||
Do **not** use the queue `status` field — it is always `'RUNNING'` while BEC is alive.
|
||||
- **Beamline states and scan interlock are independent.**
|
||||
`bec.builtin_actors.scan_interlock.states_watched` returns **accepted** (not blocking) statuses.
|
||||
- **`blocked` status** uses `primary.locks`.
|
||||
- **A block interrupts the running scan**, clearing `active_request_block`
|
||||
(→ `queue_has_active_scan = False`). So `_derive_status()` must return `blocked` on
|
||||
`beamline_blocking OR queue_locks` **independent of active scan**; only a
|
||||
fresh-heartbeat `scanning` outranks it.
|
||||
- `beamline_blocking = (enabled is True) AND any(state mismatched)` — gated on
|
||||
`enabled` so a disabled/unknown interlock never raises BLOCKED.
|
||||
- **Tomo projection count**:
|
||||
- type 1: `floor(range / stepsize) * 8` — **truncation, not rounding**; 8 sub-tomograms.
|
||||
- types 2/3: `golden_max_number_of_projections`.
|
||||
- Now generalised in JS as `calcProjections()` with kinds `equally_spaced_grid`
|
||||
and `golden_capped`, driven by the `TOMO_TYPES` data blob.
|
||||
|
||||
---
|
||||
|
||||
## 4. Multi-setup design (current)
|
||||
|
||||
Capability flags live as **class attributes** on `WebpageGeneratorBase`
|
||||
(not constructor args — so startup code is unchanged):
|
||||
|
||||
- `HAS_TOMO_QUEUE: bool` — gates the `tomo_queue` global-var read in `_cycle()`
|
||||
and the "Tomo queue" card in `_render_html()`.
|
||||
- `TOMO_TYPES: dict` — declarative config, embedded as JSON into the page; drives
|
||||
`calcProjections()`.
|
||||
|
||||
Defaults reproduce flomni behavior exactly. BEC's own primary scan queue
|
||||
(`queue_status` / `queue_locks`) and beamline states are **common — not gated**.
|
||||
|
||||
**Principle**: setup-specific behavior = class-level capability flag / declarative
|
||||
config with a flomni-preserving default. Never a hardcoded per-setup branch in JS.
|
||||
|
||||
**Principle**: when one concept appears in two places (e.g. "blocking" on the pill
|
||||
*and* the card; params in the queue *and* the instrument pane), derive both from a
|
||||
single identically-defined signal or shared helper (`buildParamRows()`) so they
|
||||
cannot drift.
|
||||
|
||||
---
|
||||
|
||||
## 5. Current sound map (flomni)
|
||||
|
||||
| Transition | Sound |
|
||||
|---|---|
|
||||
| `scanning → idle` (normal finish) | rising success arpeggio (C5-E5-G5), repeats every 30 s until Confirm |
|
||||
| `scanning → running` / other non-blocked stop | falling `warningChime`, 30 s until Confirm |
|
||||
| `scanning → blocked` | **silent**; Blocked LED pulses; `blockedChime` only after 60 min, then hourly, auto-clears |
|
||||
| feed stale / 3 failed fetches | high triple pip, 30 s until Confirm feed |
|
||||
| Enable/test button | 880-1100-880 |
|
||||
|
||||
---
|
||||
|
||||
## 6. Known-good implementation gotchas
|
||||
|
||||
- **iOS Web Audio**: must use a **synchronous** silent 1-sample `BufferSource`
|
||||
started directly in the gesture handler, plus fire-and-forget `ctx.resume()` with
|
||||
~80 ms `setTimeout` offsets. Promise `.then()` chains **fail** outside WebKit's
|
||||
gesture scope.
|
||||
- **f-string escaping**: all JS braces must be doubled (`{{`, `}}`) in Python
|
||||
f-string HTML generation.
|
||||
- **lighttpd**: negative-lookahead regex for pass-through rules is unreliable — use
|
||||
explicit pass-through entries. `%0` in redirect rules evaluates empty if the
|
||||
parent regex didn't match — use a hardcoded hostname + `$0`.
|
||||
- **`handle_error` placement**: belongs on the **server class** (`BaseServer`
|
||||
subclass), not on the request handler — overriding it on the handler has no effect
|
||||
for this error path.
|
||||
- **Mobile viewport**: `body { min-height: 100dvh }` (with `100vh` fallback); plain
|
||||
`100vh` on iOS leaves empty scroll room below the page.
|
||||
- **File discipline**: editing stale base files has been a recurring, costly error.
|
||||
**Always verify feature presence with `grep -c` before editing any file.**
|
||||
|
||||
---
|
||||
|
||||
## 7. Security model
|
||||
|
||||
- PSI internal network is **trusted**. The primary threat is the **inbound path**
|
||||
from the public web server back into the internal network.
|
||||
- Key risks and mitigations:
|
||||
- HTTP redirect-following in `requests.post()` → SSRF. Mitigated with
|
||||
`allow_redirects=False`.
|
||||
- DNS rebinding on the DynDNS hostname when `verify=False`.
|
||||
- Unbounded response-body reading.
|
||||
- `session_query.php` / `set_password.php` form the **one** feedback loop where the
|
||||
trusted side acts on public-server responses — treat with care.
|
||||
- **Principle**: implement the safest correct solution, not a minimal patch.
|
||||
|
||||
---
|
||||
|
||||
## 8. Open items
|
||||
|
||||
### Verify on next flomni restart
|
||||
- **Blocked fix**: on the next real block, confirm the pill flips to BLOCKED and
|
||||
the LED pulses (this was the reported bug).
|
||||
- **`_CURRENT_PARAM_KEYS`**: 7 names are proven (`fovx`, `fovy`, `stitch_x`,
|
||||
`stitch_y`, `tomo_shellstep`, `tomo_countingtime`, `tomo_angle_stepsize`).
|
||||
5 are **assumptions** to verify as global vars — `tomo_type`, `tomo_angle_range`,
|
||||
`frames_per_trigger`, `single_point_instead_of_fermat_scan`,
|
||||
`ptycho_reconstruct_foldername`. Any that isn't a real global var is silently
|
||||
skipped (no row); remap in `_CURRENT_PARAM_KEYS` if a name differs.
|
||||
- **Projections parity**: the queue-card Projections number for the next job should
|
||||
be identical to before (`floor(range/step) * n_subtomos`).
|
||||
- **Let's Encrypt auto-renewal**: confirm it is functioning (~90 days after the last
|
||||
manual renewal).
|
||||
|
||||
### Producer-side, not yet implemented (from briefing 4)
|
||||
- `estimated_finish_time` in `flomni.py`
|
||||
- `tomo_start_scan_number` in `flomni.py`
|
||||
- `set_web_password()` shortcut on the `Flomni` class in `flomni.py`
|
||||
|
||||
### EPS / machine status page (in progress)
|
||||
- **Starting point**: `X_X12SA_BLStatus.ui` — flat, macro-free caQtDM beamline status
|
||||
panel covering undulator gap, EPS permit, vacuum, temperatures, monochromator /
|
||||
mirror / slit positions, BPMs, filter states.
|
||||
- **Decided architecture**:
|
||||
- **Separate daemon**, decoupled from the main webpage generator.
|
||||
- **CA monitors preferred over polling.**
|
||||
- **Two-tier update cadence**: fast (shutters, EPS-OK, ring current) / slow
|
||||
(vacuum, temperature).
|
||||
- **Card/table layout first**; SVG synoptic deferred.
|
||||
- **Blocked on**: ~110 unresolved PVs from `caInclude` sub-templates
|
||||
(`EPS_Temp.ui`, `EPS_Gauge.ui`, `EPS_IonPump.ui`, `EPS_Valve.ui`, `EPS_BST.ui`, …)
|
||||
— need the template files.
|
||||
- **Next deliverables requested**: `index.html`, the JS, and the JSON schema files,
|
||||
so the PV manifest and `eps_status.json` schema can be built.
|
||||
- **Consider**: the PSI **EPICS archiver REST interface** to sidestep CA firewall
|
||||
concerns.
|
||||
|
||||
### Deferred
|
||||
- **LamNI**: fill in `LamniWebpageGenerator` — real `tomo_type` key values in
|
||||
`TOMO_TYPES`, device paths / `_collect_setup_data()`, `LamNI.png`. Add a
|
||||
session-name branch to `make_webpage_generator()`.
|
||||
- **Omny**: new subclass; `TOMO_TYPES` with a public 2-subtomo type plus a hidden
|
||||
8-subtomo type restricted to staff. The staff-only visibility mechanism is
|
||||
**still to be designed** (likely a role/visibility field filtered out before the
|
||||
config reaches a standard user's page). Add to the factory.
|
||||
- **SVG synoptic view** for the EPS page.
|
||||
- **Edge case (noted only)**: `scanning → blocked → idle` directly (block clears
|
||||
exactly as the scan finishes) yields no success tone, since the transition into
|
||||
idle came from `blocked`. Normal flow clears back to `scanning` first, so the
|
||||
success tone fires on the real finish. Revisit only if BEC ever exposes an
|
||||
explicit scan-aborted signal.
|
||||
|
||||
---
|
||||
|
||||
## 9. Tools & environment
|
||||
|
||||
- **Languages**: Python (BEC iPython env), PHP (lighttpd), JavaScript (frontend), Bash
|
||||
- **Python libs**: `requests` (upload), `Pillow` (thumbnails), `http.server`
|
||||
(local server, port 8080)
|
||||
- **PHP**: `password_hash()` / `password_verify()` (bcrypt)
|
||||
- **Beamline systems**: EPICS (PVs), caQtDM (`.ui` panels), BEC client
|
||||
(`bec.queue`, `bec.get_global_var`, `bec.builtin_actors.scan_interlock`)
|
||||
- **Infra**: lighttpd, certbot / Let's Encrypt (webroot), IONOS Webhosting Plus,
|
||||
Raspberry Pi 3B, PSI Gitea
|
||||
|
||||
---
|
||||
|
||||
## 10. Working style
|
||||
|
||||
- Discuss architecture and confirm technical details **before** implementing code.
|
||||
- Live testing happens on the actual beamline system; results are reported back
|
||||
iteratively.
|
||||
- Write a project briefing `.md` after major sessions to enable seamless continuation.
|
||||
- Prefer clean, non-redundant UI; be decisive about **removing** elements rather than
|
||||
replacing them with alternatives.
|
||||
- Decoupled daemon architecture for new monitoring components.
|
||||
Reference in New Issue
Block a user