Reuse the cached health probe, write the runnable MSR once, and return the fitted MSR and command log inline with the run response. Add CGI and single-run preparation regression tests, document the new response fields, and bump the patch version to 1.0.1.
300 lines
14 KiB
Markdown
300 lines
14 KiB
Markdown
# Web CGI Endpoint
|
|
|
|
This folder contains the web runner endpoint for `muSRFlow`:
|
|
|
|
- `single_musrfit.cgi`
|
|
|
|
The CGI writes files and runs the `musrfit`, `musrview`, and `msr2data`
|
|
executables on the server. Computational and text-transformation logic lives in
|
|
the shared `muSRFitlib.js`; the CGI keeps only the server-side file I/O, process
|
|
execution, and path/input security. Generated artifacts under
|
|
`MUSRFIT_TMPROOT` are exposed over HTTP through `MUSRFIT_TMP_URL`.
|
|
|
|
## Deploy
|
|
|
|
Run `sudo ./scripts/deploy-web.sh` from the project root to install the browser
|
|
assets and this endpoint in the default Apache directories.
|
|
|
|
After deployment:
|
|
|
|
1. Ensure binaries exist and are executable:
|
|
- `/usr/local/bin/musrfit`
|
|
- `/usr/local/bin/musrview`
|
|
- `/usr/local/bin/msr2data`
|
|
2. Ensure web tmp paths are writable:
|
|
- `/var/www/html/tmp`
|
|
|
|
## Configuration
|
|
|
|
The endpoint uses these environment variables when present and otherwise keeps
|
|
the production defaults shown below:
|
|
|
|
| Variable | Default | Purpose |
|
|
| --- | --- | --- |
|
|
| `MUSRFIT_WEBROOT` | `/var/www/html` | Static web root used to derive the default temporary directory |
|
|
| `MUSRFIT_TMPROOT` | `<webroot>/tmp` | Filesystem directory containing web jobs and archives |
|
|
| `MUSRFIT_TMP_URL` | `/tmp` | URL path through which the web server exposes `MUSRFIT_TMPROOT` |
|
|
| `MUSRFIT_BIN` | `/usr/local/bin/musrfit` | musrfit executable |
|
|
| `MUSRVIEW_BIN` | `/usr/local/bin/musrview` | musrview executable |
|
|
| `MSR2DATA_BIN` | `/usr/local/bin/msr2data` | msr2data executable |
|
|
| `MUSRFIT_DATA_ROOT` | `/psi.ch/group/lmu/public/data` | Root containing `<instrument>/his/<year>` data directories; returned to the browser as a read-only setting |
|
|
| `MUSRFIT_STARTUP_XML` | `/var/www/cgi-bin/musrfit_startup.xml` | Server-owned `musrfit_startup.xml` parsed to build portable RUN names; must be outside the web document root and never browser-supplied |
|
|
| `ROOTSYS` | `/usr/lib64/root` | ROOT installation used by musrfit executables |
|
|
| `MUSRFITPATH` | `/usr/local/bin` | Directory containing musrfit executables and XML files |
|
|
| `MUSRFIT_RUNTIME_HOME` | `<tmp root>` | Overrides writable `HOME` for ROOT and musrfit configuration |
|
|
| `MUSRFULLDATAPATH` | `<data root>` | Fallback search path for muSR data files referenced by MSR files |
|
|
|
|
`MUSRFIT_BIN` must contain only the executable path. Fit requests invoke it as
|
|
`musrfit -t -e --timeout 300 <file.msr>`, matching the original CGI behavior.
|
|
|
|
The web server must expose `MUSRFIT_TMPROOT` at `MUSRFIT_TMP_URL`. The CGI
|
|
process also needs read access to the muSR data directories, either through the
|
|
generated MSR paths or `MUSRFULLDATAPATH`.
|
|
|
|
Within one browser page session, Fit and Plot reuse the same job folder while
|
|
the first selected run stays unchanged. Changing the first run creates a new
|
|
job folder. Each execution resets its log and generated ROOT plot files in the
|
|
reused folder before running.
|
|
|
|
The template remains `<run>_tmpl.msr` as required by `msr2data`. A multi-run
|
|
global file uses the configured prefix, for example `514+global_asy.msr`; the
|
|
temporary `514+global_tmpl.msr` produced by `msr2data` is removed after it has
|
|
been validated and copied to the runnable name.
|
|
|
|
For Apache, set `MUSRFIT_DATA_ROOT` in the CGI environment when the data mount
|
|
is somewhere other than `/psi.ch/group/lmu/public/data`, for example `/psi.ch/data`. The browser
|
|
cannot override this server path.
|
|
|
|
### Portable RUN naming and `musrfit_startup.xml`
|
|
|
|
The CGI can generate portable `RUN`/`ADDRUN` file names (relative paths in the
|
|
`<run_name_template inst="...">` form) instead of the legacy absolute
|
|
`<instrument>/his/<year>/...` paths. This is opt-in and requires the admin to
|
|
install a startup XML:
|
|
|
|
- `scripts/deploy-web.sh` installs `cgi/musrfit_startup.xml` beside the CGI at
|
|
`MUSRFIT_STARTUP_XML` (default `/var/www/cgi-bin/musrfit_startup.xml`). The
|
|
file lives outside the web document root, is owned by the server, and is
|
|
**never** read from a browser-supplied path.
|
|
- The health response reports the parsed `<data_path>` entries and per-instrument
|
|
`<run_name_template>` values under `environment.startup_xml`.
|
|
- The CGI `resolve_run_name` action is a scoped, server-sided existence check:
|
|
it only confirms a candidate file under one of the configured `<data_path>`
|
|
roots and rejects any other path. It is not a general file-existence API.
|
|
- If no startup XML is configured, or it cannot be parsed, portable naming stays
|
|
disabled and absolute RUN paths remain the default.
|
|
|
|
## API
|
|
|
|
Endpoint: `POST /cgi-bin/single_musrfit.cgi`
|
|
|
|
### `action=health`
|
|
|
|
Check the server configuration without creating or running a job. The response
|
|
reports executable paths, availability, whether each binary can load and run,
|
|
versions or loader diagnostics, temporary-directory access, and
|
|
the data-root/ROOT runtime environment values. This makes missing ROOT
|
|
libraries in the restricted CGI environment visible before a fit is attempted.
|
|
|
|
```bash
|
|
curl -s -X POST -d 'action=health' \
|
|
http://localhost/cgi-bin/single_musrfit.cgi
|
|
```
|
|
|
|
### `action=run_file_info` and `action=download_run_file`
|
|
|
|
These read-only actions supply run data to desktop clients. They work without
|
|
musrfit executables or a writable temporary directory and do not create web
|
|
jobs. Both take the same run identity:
|
|
|
|
- `instrument`: `LEM`, `GPS`, `FLAME`, `LTF`, `Dolly`, `VMS`, `GPD`, `HAL`,
|
|
`ALC`, or `MISC` (case-insensitive; `HIFI` is an alias for `HAL`).
|
|
- `year`: a four-digit year, at least 1900.
|
|
- `run`: a positive run number of up to nine digits.
|
|
|
|
The server expands its own startup-XML templates in configured order, searching
|
|
the configured `<data_path>` roots and
|
|
`<MUSRFIT_DATA_ROOT>/<instrument>/his`. For each template it tries the exact
|
|
layout, followed by `<year>/<filename>` for the flattened public data tree.
|
|
The original filename, including historical campaign names, is preserved.
|
|
Clients cannot select a server filesystem path. File and directory symlinks
|
|
must resolve inside the configured root that is being searched.
|
|
|
|
`run_file_info` returns JSON with `ok`, `code`, `message`, `instrument`, `year`,
|
|
`run`, and these file properties:
|
|
|
|
| Property | Meaning |
|
|
| --- | --- |
|
|
| `filename` | Exact data filename, including its extension |
|
|
| `relative_path` | Destination relative to the desktop Data directory, for example `lem/his/2026/lem26_his_2615.root` |
|
|
| `format` | MSR format: `MUSR-ROOT`, `PSI-BIN`, or `PSI-MDU` |
|
|
| `size` | File length in bytes |
|
|
| `mtime` | Modification time as Unix seconds |
|
|
| `version` | Opaque token identifying the observed file metadata; not a content checksum |
|
|
|
|
```bash
|
|
curl --fail --silent --show-error \
|
|
--data 'action=run_file_info&instrument=LEM&year=2026&run=2615' \
|
|
http://musruser.psi.ch/cgi-bin/single_musrfit.cgi
|
|
```
|
|
|
|
`download_run_file` returns the binary file with `Content-Length`,
|
|
`Content-Disposition: attachment`, and `Cache-Control: no-store`. Transfers use
|
|
a fixed-size buffer, and raw files are never copied into the web temporary
|
|
directory or encoded as JSON/base64.
|
|
|
|
The client should pass `version` from the metadata response when downloading.
|
|
If the file changed between requests, the CGI returns HTTP 409 and JSON with
|
|
`reason=run-file-changed`; fetch metadata again before retrying. Omitting
|
|
`version` downloads the file found at the start of the request:
|
|
|
|
```bash
|
|
curl --fail --show-error --output lem26_his_2615.root \
|
|
--data 'action=download_run_file&instrument=LEM&year=2026&run=2615' \
|
|
http://musruser.psi.ch/cgi-bin/single_musrfit.cgi
|
|
```
|
|
|
|
Download errors use both an HTTP error status and a JSON diagnostic: 400 for
|
|
invalid identifiers, 404 with `outcome=missing-data` for missing files, 409 for
|
|
empty files or changed versions, and 503 for unconfigured templates or
|
|
unreadable files. Server paths are not included in these responses. A file
|
|
that grows during streaming is limited to its initial advertised length; if
|
|
it is truncated or a read fails, the response is incomplete and the server
|
|
logs the failure. The desktop must finish and validate a temporary download
|
|
before publishing it under the Data directory.
|
|
|
|
### `action=write`
|
|
|
|
Write an MSR file:
|
|
- `fn`: `/tmp/<job>/<name>.msr`
|
|
- `line0..lineN`: text lines
|
|
|
|
Returns JSON:
|
|
- `ok`, `code`, `message`, `job_name`, `work_url`, `msr_url`
|
|
|
|
### `action=series_transaction`
|
|
|
|
Stage and promote a validated batch of independent-run MSRs without executing
|
|
`musrfit`. The request contains:
|
|
|
|
- `fn`: one `/tmp/<job>/<name>.msr` path used only to identify the job folder
|
|
- `transaction`: JSON containing a restricted transaction ID and candidate list
|
|
|
|
Every candidate is limited to the same job folder and must contain its numeric
|
|
run, `fit|retry|refit` action, run-prefixed `.msr` filename, and complete MSR
|
|
text. Retry/refit candidates must also provide the exact content observed during
|
|
inventory. The server derives all stage and backup names; browser-provided
|
|
filesystem paths are never accepted from the JSON document.
|
|
|
|
The endpoint exclusively stages and reads back the entire batch before any
|
|
promotion. New targets use no-clobber publication. Replaced targets receive an
|
|
exclusive recoverable backup. A failed promotion rolls earlier targets back in
|
|
reverse order. Successful retry/refit backups are deliberately retained for the
|
|
later sequential-runner commit/rollback step.
|
|
|
|
After promotion, the server seals the batch in a private transaction manifest.
|
|
The manifest records the server-derived target and backup names, content hashes,
|
|
and the five identity fields of every canonical `RUN`/`ADDRUN` directive.
|
|
Whitespace and trailing comments written by musrfit are not identity fields. The
|
|
manifest is not supplied or editable by the browser.
|
|
|
|
### `action=series_transaction_finalize`
|
|
|
|
Commit or roll back one previously promoted independent-run transaction without
|
|
running an executable. The request contains:
|
|
|
|
- `fn`: one `/tmp/<job>/<name>.msr` path used only to identify the job folder
|
|
- `finalization`: JSON with the server transaction `id` and a `commit|rollback`
|
|
decision
|
|
|
|
Commit first verifies the complete batch. Every target must be a valid,
|
|
converged fitted MSR and must retain the `RUN`/`ADDRUN` path, beamline,
|
|
institute, and data format recorded at promotion. Only then are replacement
|
|
backups deleted. Rollback operates in
|
|
reverse order: newly created targets are removed and replacement backups are
|
|
restored. The manifest is removed only after every item is finalized, allowing
|
|
a failed partial operation to be retried safely.
|
|
|
|
This endpoint only finalizes filesystem state. It does not call `musrfit`,
|
|
`msr2data`, or `musrview` and is not yet connected to the Fit button.
|
|
|
|
### `action=series_run_candidate`
|
|
|
|
Run `musrfit` for exactly one target already protected by a server-created
|
|
series transaction. The request contains:
|
|
|
|
- `fn`: the promoted `/tmp/<job>/<run-name>.msr` target
|
|
- `transaction_id`: the transaction that owns this target
|
|
|
|
The server locks the job, verifies the private manifest, checks that the target
|
|
still has its promoted content and `RUN`/`ADDRUN` identity, and then invokes
|
|
`musrfit`. Success requires a zero exit code, unchanged run identity, and a
|
|
converged fitted MSR. The endpoint writes the target's `.out` log but does not
|
|
run `msr2data`, `musrview`, package artifacts, synchronize the GUI, or finalize
|
|
the transaction.
|
|
|
|
### `action=series_inventory`
|
|
|
|
Inspect the selected independent-run files without creating a job or running an
|
|
executable. The request contains:
|
|
|
|
- `fn`: one `/tmp/<job>/<name>.msr` path used only to identify the job folder
|
|
- `inventory`: JSON lists of run-prefixed MSR filenames and resolved data paths
|
|
|
|
MSRs are restricted to the selected job folder. Data paths must match the
|
|
configured `<data-root>/<instrument>/his/<year>/<file>` layout. Symlinks are not
|
|
followed. Existing MSRs up to 2 MB are returned as text for the shared browser/
|
|
Electron classifier. Larger or unreadable files are reported as existing but
|
|
unreadable, which blocks replacement rather than treating them as missing.
|
|
|
|
The response is a single snapshot containing `job_exists`, `msr_files`, and
|
|
`data_files`. The browser verifies that every requested entry appears exactly
|
|
once before using it.
|
|
|
|
### `action=run`
|
|
|
|
Run fitting/plot workflow on an existing MSR:
|
|
- `fn`: `/tmp/<job>/<name>.msr`
|
|
- `use_msr2data`: `0|1`
|
|
- `runs`: space-separated run numbers (for global generation)
|
|
- `template_name`: e.g. `123_tmpl.msr`
|
|
- `base_run`: first run number
|
|
- `fftv`: `y|n`
|
|
- `plot_refresh`: `1` reruns only `musrview` for an existing floating plot and
|
|
appends to its command log; it requires `plot_only=1`
|
|
|
|
The CGI runs `musrview --root` in headless graphics mode because an Apache
|
|
process has no interactive display. Generated ROOT canvas files are shown by the
|
|
JSROOT web viewer and stored with the other job artifacts.
|
|
After a successful fit, it also runs the same parameter-extraction workflow as
|
|
the original CGI and writes `<fit-name>_par.dat` with `msr2data`. The fitted MSR
|
|
continues to update the editable GUI parameter table independently of this
|
|
run-wise export.
|
|
|
|
The browser links this export to the existing musrfit plotter using
|
|
`/MusrPlot/?file=<encoded-parameter-url>`. The plotter must be served from that
|
|
same-origin path, as in the original CGI deployment. Set
|
|
`window.MusrfitPlotterPath` before running a fit if a deployment uses another
|
|
route.
|
|
|
|
Returns JSON:
|
|
- `ok`, `code`, `message`, `stderr`
|
|
- `outcome`: `missing-data` when musrfit or musrview cannot resolve RUN data
|
|
- `missing_files`: paths extracted from the executable diagnostics
|
|
- `job_name`, `work_url`, `msr_url`, `out_url`, `parameter_url`
|
|
- `root_url`, `root_urls`, `root_items`
|
|
- `command_log`, `msr_text`, `log_text`, `tgz_url`
|
|
|
|
`msr_text` is the fitted MSR and `log_text` is the complete command log, both
|
|
returned inline so a normal browser Fit/Plot needs no follow-up artifact GETs.
|
|
The same documents remain available at `msr_url` and `out_url` for older
|
|
clients and for jobs whose text would exceed the inline size limits.
|
|
|
|
The `.out` artifact referenced by `out_url` is also the user-facing command log.
|
|
For every `msr2data`, `musrfit`, and `musrview` step it records the shell command,
|
|
combined standard output/error, and the exit status.
|
|
|
|
Missing RUN data is reported as a separate, non-failure workflow outcome. A Fit
|
|
stops after musrfit reports the missing input instead of launching musrview to
|
|
repeat the same diagnostic. The full executable output remains in the `.out` log.
|