Files
muSRFlow/cgi/README.md
Zaher Salman 999fe92e39 Harden web CGI against concurrent runs and oversized requests
Add server-side safeguards so heavy web workflows cannot crash Apache:

- Run-concurrency gate: whole run/series_run_candidate requests lease flock
  slots under TMPROOT/.musrflow-run-slots (MUSRFIT_MAX_RUNS=2 default,
  MUSRFIT_MAX_RUNS_PER_IP optional per-client pool); saturated servers answer
  503 server-busy and leases are released when the CGI process exits.
- Per-child resource wrapper: every external command runs with RLIMIT_AS
  (MUSRFIT_CHILD_MEM_KB, 16 GiB), optional RLIMIT_CPU, and exec nice
  (MUSRFIT_CHILD_NICE=10) so a runaway child cannot starve Apache workers.
- Retune MUSRFIT_CMD_TIMEOUT default 400 -> 360 (deploy knob).
- Size/job backstops: POST body cap before CGI->new (413), write MSR cap
  (413), and MUSRFIT_MAX_JOBS refusal of new job folders (never deletes).
- Exclude the .musrflow-run-slots dir from prune_stale_jobs and job counting.
- Document the new knobs in cgi/README.md and add CGI hardening tests.
2026-10-10 18:07:57 +02:00

261 lines
13 KiB
Markdown

# Web CGI Endpoint
This folder contains the web runner endpoint for `muSRFlow`:
- `single_musrfit.cgi`
The CGI writes files and runs the `musrfit`, `musrview`, and `msr2data`
executables on the server. Computational and text-transformation logic lives in
the shared `muSRFitlib.js`; the CGI keeps only the server-side file I/O, process
execution, and path/input security. Generated artifacts under
`MUSRFIT_TMPROOT` are exposed over HTTP through `MUSRFIT_TMP_URL`.
## Deploy
Run `sudo ./scripts/deploy-web.sh` from the project root to install the browser
assets and this endpoint in the default Apache directories.
After deployment:
1. Ensure binaries exist and are executable:
- `/usr/local/bin/musrfit`
- `/usr/local/bin/musrview`
- `/usr/local/bin/msr2data`
2. Ensure web tmp paths are writable:
- `/var/www/html/tmp`
## Configuration
The endpoint uses these environment variables when present and otherwise keeps
the production defaults shown below:
| Variable | Default | Purpose |
| --- | --- | --- |
| `MUSRFIT_WEBROOT` | `/var/www/html` | Static web root used to derive the default temporary directory |
| `MUSRFIT_TMPROOT` | `<webroot>/tmp` | Filesystem directory containing web jobs and archives |
| `MUSRFIT_TMP_URL` | `/tmp` | URL path through which the web server exposes `MUSRFIT_TMPROOT` |
| `MUSRFIT_BIN` | `/usr/local/bin/musrfit` | musrfit executable |
| `MUSRVIEW_BIN` | `/usr/local/bin/musrview` | musrview executable |
| `MSR2DATA_BIN` | `/usr/local/bin/msr2data` | msr2data executable |
| `MUSRFIT_DATA_ROOT` | `/psi.ch/group/lmu/public/data` | Root containing `<instrument>/his/<year>` data directories; returned to the browser as a read-only setting |
| `MUSRFIT_STARTUP_XML` | `/var/www/cgi-bin/musrfit_startup.xml` | Server-owned `musrfit_startup.xml` parsed to build portable RUN names; must be outside the web document root and never browser-supplied |
| `ROOTSYS` | `/usr/lib64/root` | ROOT installation used by musrfit executables |
| `MUSRFITPATH` | `/usr/local/bin` | Directory containing musrfit executables and XML files |
| `MUSRFIT_RUNTIME_HOME` | `<tmp root>` | Overrides writable `HOME` for ROOT and musrfit configuration |
| `MUSRFULLDATAPATH` | `<data root>` | Fallback search path for muSR data files referenced by MSR files |
| `MUSRFIT_CMD_TIMEOUT` | `360` | Seconds each external command (musrfit, musrview, msr2data, dump_header, tar) may run before the CGI kills its process group |
| `MUSRFIT_CHILD_MEM_KB` | `16777216` | `RLIMIT_AS` (KB) applied inside the forked child before every external command; `0` disables |
| `MUSRFIT_CHILD_CPU_SEC` | `0` | `RLIMIT_CPU` (seconds) applied inside the forked child; `0` = unlimited |
| `MUSRFIT_CHILD_NICE` | `10` | Scheduling priority the external command is launched at; `0` disables |
| `MUSRFIT_MAX_RUNS` | `2` | Maximum simultaneous `run`/`series_run_candidate` requests across all clients; extra requests get HTTP 503 `server-busy`; `0` blocks all heavy runs |
| `MUSRFIT_MAX_RUNS_PER_IP` | `0` | Per-client share of the concurrent-run pool (keyed by `REMOTE_ADDR`); `0` = one client may use the whole pool |
| `MUSRFIT_MAX_REQUEST_BYTES` | `8000000` | Request body cap checked against `CONTENT_LENGTH` before CGI parsing; larger requests get HTTP 413 `request-too-large` |
| `MUSRFIT_MAX_MSR_BYTES` | `2000000` | Cap on MSR content accepted by `action=write`; larger content gets HTTP 413 `msr-too-large` |
| `MUSRFIT_MAX_JOBS` | `0` | Maximum web job folders allowed; new job folders are refused with HTTP 500 once the ceiling is reached. Existing folders keep working and nothing is ever deleted; `0` = unlimited |
`MUSRFIT_BIN` must contain only the executable path. Fit requests invoke it as
`musrfit -t -e --timeout 300 <file.msr>`, matching the original CGI behavior.
The web server must expose `MUSRFIT_TMPROOT` at `MUSRFIT_TMP_URL`. The CGI
process also needs read access to the muSR data directories, either through the
generated MSR paths or `MUSRFULLDATAPATH`.
Within one browser page session, Fit and Plot reuse the same job folder while
the first selected run stays unchanged. Changing the first run creates a new
job folder. Each execution resets its log and generated ROOT plot files in the
reused folder before running.
The template remains `<run>_tmpl.msr` as required by `msr2data`. A multi-run
global file uses the configured prefix, for example `514+global_asy.msr`; the
temporary `514+global_tmpl.msr` produced by `msr2data` is removed after it has
been validated and copied to the runnable name.
For Apache, set `MUSRFIT_DATA_ROOT` in the CGI environment when the data mount
is somewhere other than `/psi.ch/group/lmu/public/data`, for example `/psi.ch/data`. The browser
cannot override this server path.
Each `run` request holds one Apache/CGI worker for the whole
`musrfit → msr2data → musrview → tar` chain. A fit that finishes cleanly or is
killed by `MUSRFIT_CMD_TIMEOUT` returns the worker to Apache; only an unbounded
component or a worker crash leaves it stranded. Two additional safeguards keep a
saturated server responsive under many simultaneous users:
- `MUSRFIT_MAX_RUNS` bounds how many heavy workflows run at once via flock
leases on small files under `MUSRFIT_TMPROOT/.musrflow-run-slots`. A request
that cannot obtain its lease answers HTTP 503 `server-busy` instead of queuing
in a worker. Leases are released automatically when the CGI process exits, so
a crashed worker can never leave a stale lock.
- Every external command is launched with `MUSRFIT_CHILD_MEM_KB` and
`MUSRFIT_CHILD_CPU_SEC` machinery and the `MUSRFIT_CHILD_NICE` priority, so a
runaway child consumes bounded memory, bounded CPU, or low scheduling priority
instead of starving Apache workers.
The slot directory is hidden (dot-prefixed) and is never counted as a job folder
nor pruned by the job TTL cleanup.
### Portable RUN naming and `musrfit_startup.xml`
The CGI can generate portable `RUN`/`ADDRUN` file names (relative paths in the
`<run_name_template inst="...">` form) instead of the legacy absolute
`<instrument>/his/<year>/...` paths. This is opt-in and requires the admin to
install a startup XML:
- `scripts/deploy-web.sh` installs `cgi/musrfit_startup.xml` beside the CGI at
`MUSRFIT_STARTUP_XML` (default `/var/www/cgi-bin/musrfit_startup.xml`). The
file lives outside the web document root, is owned by the server, and is
**never** read from a browser-supplied path.
- The health response reports the parsed `<data_path>` entries and per-instrument
`<run_name_template>` values under `environment.startup_xml`.
- The CGI `resolve_run_name` action is a scoped, server-sided existence check:
it only confirms a candidate file under one of the configured `<data_path>`
roots and rejects any other path. It is not a general file-existence API.
- If no startup XML is configured, or it cannot be parsed, portable naming stays
disabled and absolute RUN paths remain the default.
## API
Endpoint: `POST /cgi-bin/single_musrfit.cgi`
### `action=health`
Check the server configuration without creating or running a job. The response
reports executable paths, availability, whether each binary can load and run,
versions or loader diagnostics, temporary-directory access, and
the data-root/ROOT runtime environment values. This makes missing ROOT
libraries in the restricted CGI environment visible before a fit is attempted.
```bash
curl -s -X POST -d 'action=health' \
http://localhost/cgi-bin/single_musrfit.cgi
```
### `action=write`
Write an MSR file:
- `fn`: `/tmp/<job>/<name>.msr`
- `line0..lineN`: text lines
Returns JSON:
- `ok`, `code`, `message`, `job_name`, `work_url`, `msr_url`
### `action=series_transaction`
Stage and promote a validated batch of independent-run MSRs without executing
`musrfit`. The request contains:
- `fn`: one `/tmp/<job>/<name>.msr` path used only to identify the job folder
- `transaction`: JSON containing a restricted transaction ID and candidate list
Every candidate is limited to the same job folder and must contain its numeric
run, `fit|retry|refit` action, run-prefixed `.msr` filename, and complete MSR
text. Retry/refit candidates must also provide the exact content observed during
inventory. The server derives all stage and backup names; browser-provided
filesystem paths are never accepted from the JSON document.
The endpoint exclusively stages and reads back the entire batch before any
promotion. New targets use no-clobber publication. Replaced targets receive an
exclusive recoverable backup. A failed promotion rolls earlier targets back in
reverse order. Successful retry/refit backups are deliberately retained for the
later sequential-runner commit/rollback step.
After promotion, the server seals the batch in a private transaction manifest.
The manifest records the server-derived target and backup names, content hashes,
and the five identity fields of every canonical `RUN`/`ADDRUN` directive.
Whitespace and trailing comments written by musrfit are not identity fields. The
manifest is not supplied or editable by the browser.
### `action=series_transaction_finalize`
Commit or roll back one previously promoted independent-run transaction without
running an executable. The request contains:
- `fn`: one `/tmp/<job>/<name>.msr` path used only to identify the job folder
- `finalization`: JSON with the server transaction `id` and a `commit|rollback`
decision
Commit first verifies the complete batch. Every target must be a valid,
converged fitted MSR and must retain the `RUN`/`ADDRUN` path, beamline,
institute, and data format recorded at promotion. Only then are replacement
backups deleted. Rollback operates in
reverse order: newly created targets are removed and replacement backups are
restored. The manifest is removed only after every item is finalized, allowing
a failed partial operation to be retried safely.
This endpoint only finalizes filesystem state. It does not call `musrfit`,
`msr2data`, or `musrview` and is not yet connected to the Fit button.
### `action=series_run_candidate`
Run `musrfit` for exactly one target already protected by a server-created
series transaction. The request contains:
- `fn`: the promoted `/tmp/<job>/<run-name>.msr` target
- `transaction_id`: the transaction that owns this target
The server locks the job, verifies the private manifest, checks that the target
still has its promoted content and `RUN`/`ADDRUN` identity, and then invokes
`musrfit`. Success requires a zero exit code, unchanged run identity, and a
converged fitted MSR. The endpoint writes the target's `.out` log but does not
run `msr2data`, `musrview`, package artifacts, synchronize the GUI, or finalize
the transaction.
### `action=series_inventory`
Inspect the selected independent-run files without creating a job or running an
executable. The request contains:
- `fn`: one `/tmp/<job>/<name>.msr` path used only to identify the job folder
- `inventory`: JSON lists of run-prefixed MSR filenames and resolved data paths
MSRs are restricted to the selected job folder. Data paths must match the
configured `<data-root>/<instrument>/his/<year>/<file>` layout. Symlinks are not
followed. Existing MSRs up to 2 MB are returned as text for the shared browser/
Electron classifier. Larger or unreadable files are reported as existing but
unreadable, which blocks replacement rather than treating them as missing.
The response is a single snapshot containing `job_exists`, `msr_files`, and
`data_files`. The browser verifies that every requested entry appears exactly
once before using it.
### `action=run`
Run fitting/plot workflow on an existing MSR:
- `fn`: `/tmp/<job>/<name>.msr`
- `use_msr2data`: `0|1`
- `runs`: space-separated run numbers (for global generation)
- `template_name`: e.g. `123_tmpl.msr`
- `base_run`: first run number
- `fftv`: `y|n`
- `plot_refresh`: `1` reruns only `musrview` for an existing floating plot and
appends to its command log; it requires `plot_only=1`
The CGI runs `musrview --root` in headless graphics mode because an Apache
process has no interactive display. Generated ROOT canvas files are shown by the
JSROOT web viewer and stored with the other job artifacts.
After a successful fit, it also runs the same parameter-extraction workflow as
the original CGI and writes `<fit-name>_par.dat` with `msr2data`. The fitted MSR
continues to update the editable GUI parameter table independently of this
run-wise export.
The browser links this export to the existing musrfit plotter using
`/MusrPlot/?file=<encoded-parameter-url>`. The plotter must be served from that
same-origin path, as in the original CGI deployment. Set
`window.MusrfitPlotterPath` before running a fit if a deployment uses another
route.
Returns JSON:
- `ok`, `code`, `message`, `stderr`
- `outcome`: `missing-data` when musrfit or musrview cannot resolve RUN data
- `missing_files`: paths extracted from the executable diagnostics
- `job_name`, `work_url`, `msr_url`, `out_url`, `parameter_url`
- `root_url`, `root_urls`, `root_items`
- `command_log`, `tgz_url`
The `.out` artifact referenced by `out_url` is also the user-facing command log.
For every `msr2data`, `musrfit`, and `musrview` step it records the shell command,
combined standard output/error, and the exit status.
Missing RUN data is reported as a separate, non-failure workflow outcome. A Fit
stops after musrfit reports the missing input instead of launching musrview to
repeat the same diagnostic. The full executable output remains in the `.out` log.