Files
muSRFlow/cgi/README.md
T
Zaher Salman 8fa4414319
Build Windows installer / build-windows (push) Failing after 4m57s
Build and package / build-package (push) Successful in 5m38s
Speed up single-run web fits
Reuse the cached health probe, write the runnable MSR once, and return the fitted MSR and command log inline with the run response. Add CGI and single-run preparation regression tests, document the new response fields, and bump the patch version to 1.0.1.
2026-11-09 22:27:44 +01:00

14 KiB

Web CGI Endpoint

This folder contains the web runner endpoint for muSRFlow:

  • single_musrfit.cgi

The CGI writes files and runs the musrfit, musrview, and msr2data executables on the server. Computational and text-transformation logic lives in the shared muSRFitlib.js; the CGI keeps only the server-side file I/O, process execution, and path/input security. Generated artifacts under MUSRFIT_TMPROOT are exposed over HTTP through MUSRFIT_TMP_URL.

Deploy

Run sudo ./scripts/deploy-web.sh from the project root to install the browser assets and this endpoint in the default Apache directories.

After deployment:

  1. Ensure binaries exist and are executable:
    • /usr/local/bin/musrfit
    • /usr/local/bin/musrview
    • /usr/local/bin/msr2data
  2. Ensure web tmp paths are writable:
    • /var/www/html/tmp

Configuration

The endpoint uses these environment variables when present and otherwise keeps the production defaults shown below:

Variable Default Purpose
MUSRFIT_WEBROOT /var/www/html Static web root used to derive the default temporary directory
MUSRFIT_TMPROOT <webroot>/tmp Filesystem directory containing web jobs and archives
MUSRFIT_TMP_URL /tmp URL path through which the web server exposes MUSRFIT_TMPROOT
MUSRFIT_BIN /usr/local/bin/musrfit musrfit executable
MUSRVIEW_BIN /usr/local/bin/musrview musrview executable
MSR2DATA_BIN /usr/local/bin/msr2data msr2data executable
MUSRFIT_DATA_ROOT /psi.ch/group/lmu/public/data Root containing <instrument>/his/<year> data directories; returned to the browser as a read-only setting
MUSRFIT_STARTUP_XML /var/www/cgi-bin/musrfit_startup.xml Server-owned musrfit_startup.xml parsed to build portable RUN names; must be outside the web document root and never browser-supplied
ROOTSYS /usr/lib64/root ROOT installation used by musrfit executables
MUSRFITPATH /usr/local/bin Directory containing musrfit executables and XML files
MUSRFIT_RUNTIME_HOME <tmp root> Overrides writable HOME for ROOT and musrfit configuration
MUSRFULLDATAPATH <data root> Fallback search path for muSR data files referenced by MSR files

MUSRFIT_BIN must contain only the executable path. Fit requests invoke it as musrfit -t -e --timeout 300 <file.msr>, matching the original CGI behavior.

The web server must expose MUSRFIT_TMPROOT at MUSRFIT_TMP_URL. The CGI process also needs read access to the muSR data directories, either through the generated MSR paths or MUSRFULLDATAPATH.

Within one browser page session, Fit and Plot reuse the same job folder while the first selected run stays unchanged. Changing the first run creates a new job folder. Each execution resets its log and generated ROOT plot files in the reused folder before running.

The template remains <run>_tmpl.msr as required by msr2data. A multi-run global file uses the configured prefix, for example 514+global_asy.msr; the temporary 514+global_tmpl.msr produced by msr2data is removed after it has been validated and copied to the runnable name.

For Apache, set MUSRFIT_DATA_ROOT in the CGI environment when the data mount is somewhere other than /psi.ch/group/lmu/public/data, for example /psi.ch/data. The browser cannot override this server path.

Portable RUN naming and musrfit_startup.xml

The CGI can generate portable RUN/ADDRUN file names (relative paths in the <run_name_template inst="..."> form) instead of the legacy absolute <instrument>/his/<year>/... paths. This is opt-in and requires the admin to install a startup XML:

  • scripts/deploy-web.sh installs cgi/musrfit_startup.xml beside the CGI at MUSRFIT_STARTUP_XML (default /var/www/cgi-bin/musrfit_startup.xml). The file lives outside the web document root, is owned by the server, and is never read from a browser-supplied path.
  • The health response reports the parsed <data_path> entries and per-instrument <run_name_template> values under environment.startup_xml.
  • The CGI resolve_run_name action is a scoped, server-sided existence check: it only confirms a candidate file under one of the configured <data_path> roots and rejects any other path. It is not a general file-existence API.
  • If no startup XML is configured, or it cannot be parsed, portable naming stays disabled and absolute RUN paths remain the default.

API

Endpoint: POST /cgi-bin/single_musrfit.cgi

action=health

Check the server configuration without creating or running a job. The response reports executable paths, availability, whether each binary can load and run, versions or loader diagnostics, temporary-directory access, and the data-root/ROOT runtime environment values. This makes missing ROOT libraries in the restricted CGI environment visible before a fit is attempted.

curl -s -X POST -d 'action=health' \
  http://localhost/cgi-bin/single_musrfit.cgi

action=run_file_info and action=download_run_file

These read-only actions supply run data to desktop clients. They work without musrfit executables or a writable temporary directory and do not create web jobs. Both take the same run identity:

  • instrument: LEM, GPS, FLAME, LTF, Dolly, VMS, GPD, HAL, ALC, or MISC (case-insensitive; HIFI is an alias for HAL).
  • year: a four-digit year, at least 1900.
  • run: a positive run number of up to nine digits.

The server expands its own startup-XML templates in configured order, searching the configured <data_path> roots and <MUSRFIT_DATA_ROOT>/<instrument>/his. For each template it tries the exact layout, followed by <year>/<filename> for the flattened public data tree. The original filename, including historical campaign names, is preserved. Clients cannot select a server filesystem path. File and directory symlinks must resolve inside the configured root that is being searched.

run_file_info returns JSON with ok, code, message, instrument, year, run, and these file properties:

Property Meaning
filename Exact data filename, including its extension
relative_path Destination relative to the desktop Data directory, for example lem/his/2026/lem26_his_2615.root
format MSR format: MUSR-ROOT, PSI-BIN, or PSI-MDU
size File length in bytes
mtime Modification time as Unix seconds
version Opaque token identifying the observed file metadata; not a content checksum
curl --fail --silent --show-error \
  --data 'action=run_file_info&instrument=LEM&year=2026&run=2615' \
  http://musruser.psi.ch/cgi-bin/single_musrfit.cgi

download_run_file returns the binary file with Content-Length, Content-Disposition: attachment, and Cache-Control: no-store. Transfers use a fixed-size buffer, and raw files are never copied into the web temporary directory or encoded as JSON/base64.

The client should pass version from the metadata response when downloading. If the file changed between requests, the CGI returns HTTP 409 and JSON with reason=run-file-changed; fetch metadata again before retrying. Omitting version downloads the file found at the start of the request:

curl --fail --show-error --output lem26_his_2615.root \
  --data 'action=download_run_file&instrument=LEM&year=2026&run=2615' \
  http://musruser.psi.ch/cgi-bin/single_musrfit.cgi

Download errors use both an HTTP error status and a JSON diagnostic: 400 for invalid identifiers, 404 with outcome=missing-data for missing files, 409 for empty files or changed versions, and 503 for unconfigured templates or unreadable files. Server paths are not included in these responses. A file that grows during streaming is limited to its initial advertised length; if it is truncated or a read fails, the response is incomplete and the server logs the failure. The desktop must finish and validate a temporary download before publishing it under the Data directory.

action=write

Write an MSR file:

  • fn: /tmp/<job>/<name>.msr
  • line0..lineN: text lines

Returns JSON:

  • ok, code, message, job_name, work_url, msr_url

action=series_transaction

Stage and promote a validated batch of independent-run MSRs without executing musrfit. The request contains:

  • fn: one /tmp/<job>/<name>.msr path used only to identify the job folder
  • transaction: JSON containing a restricted transaction ID and candidate list

Every candidate is limited to the same job folder and must contain its numeric run, fit|retry|refit action, run-prefixed .msr filename, and complete MSR text. Retry/refit candidates must also provide the exact content observed during inventory. The server derives all stage and backup names; browser-provided filesystem paths are never accepted from the JSON document.

The endpoint exclusively stages and reads back the entire batch before any promotion. New targets use no-clobber publication. Replaced targets receive an exclusive recoverable backup. A failed promotion rolls earlier targets back in reverse order. Successful retry/refit backups are deliberately retained for the later sequential-runner commit/rollback step.

After promotion, the server seals the batch in a private transaction manifest. The manifest records the server-derived target and backup names, content hashes, and the five identity fields of every canonical RUN/ADDRUN directive. Whitespace and trailing comments written by musrfit are not identity fields. The manifest is not supplied or editable by the browser.

action=series_transaction_finalize

Commit or roll back one previously promoted independent-run transaction without running an executable. The request contains:

  • fn: one /tmp/<job>/<name>.msr path used only to identify the job folder
  • finalization: JSON with the server transaction id and a commit|rollback decision

Commit first verifies the complete batch. Every target must be a valid, converged fitted MSR and must retain the RUN/ADDRUN path, beamline, institute, and data format recorded at promotion. Only then are replacement backups deleted. Rollback operates in reverse order: newly created targets are removed and replacement backups are restored. The manifest is removed only after every item is finalized, allowing a failed partial operation to be retried safely.

This endpoint only finalizes filesystem state. It does not call musrfit, msr2data, or musrview and is not yet connected to the Fit button.

action=series_run_candidate

Run musrfit for exactly one target already protected by a server-created series transaction. The request contains:

  • fn: the promoted /tmp/<job>/<run-name>.msr target
  • transaction_id: the transaction that owns this target

The server locks the job, verifies the private manifest, checks that the target still has its promoted content and RUN/ADDRUN identity, and then invokes musrfit. Success requires a zero exit code, unchanged run identity, and a converged fitted MSR. The endpoint writes the target's .out log but does not run msr2data, musrview, package artifacts, synchronize the GUI, or finalize the transaction.

action=series_inventory

Inspect the selected independent-run files without creating a job or running an executable. The request contains:

  • fn: one /tmp/<job>/<name>.msr path used only to identify the job folder
  • inventory: JSON lists of run-prefixed MSR filenames and resolved data paths

MSRs are restricted to the selected job folder. Data paths must match the configured <data-root>/<instrument>/his/<year>/<file> layout. Symlinks are not followed. Existing MSRs up to 2 MB are returned as text for the shared browser/ Electron classifier. Larger or unreadable files are reported as existing but unreadable, which blocks replacement rather than treating them as missing.

The response is a single snapshot containing job_exists, msr_files, and data_files. The browser verifies that every requested entry appears exactly once before using it.

action=run

Run fitting/plot workflow on an existing MSR:

  • fn: /tmp/<job>/<name>.msr
  • use_msr2data: 0|1
  • runs: space-separated run numbers (for global generation)
  • template_name: e.g. 123_tmpl.msr
  • base_run: first run number
  • fftv: y|n
  • plot_refresh: 1 reruns only musrview for an existing floating plot and appends to its command log; it requires plot_only=1

The CGI runs musrview --root in headless graphics mode because an Apache process has no interactive display. Generated ROOT canvas files are shown by the JSROOT web viewer and stored with the other job artifacts. After a successful fit, it also runs the same parameter-extraction workflow as the original CGI and writes <fit-name>_par.dat with msr2data. The fitted MSR continues to update the editable GUI parameter table independently of this run-wise export.

The browser links this export to the existing musrfit plotter using /MusrPlot/?file=<encoded-parameter-url>. The plotter must be served from that same-origin path, as in the original CGI deployment. Set window.MusrfitPlotterPath before running a fit if a deployment uses another route.

Returns JSON:

  • ok, code, message, stderr
  • outcome: missing-data when musrfit or musrview cannot resolve RUN data
  • missing_files: paths extracted from the executable diagnostics
  • job_name, work_url, msr_url, out_url, parameter_url
  • root_url, root_urls, root_items
  • command_log, msr_text, log_text, tgz_url

msr_text is the fitted MSR and log_text is the complete command log, both returned inline so a normal browser Fit/Plot needs no follow-up artifact GETs. The same documents remain available at msr_url and out_url for older clients and for jobs whose text would exceed the inline size limits.

The .out artifact referenced by out_url is also the user-facing command log. For every msr2data, musrfit, and musrview step it records the shell command, combined standard output/error, and the exit status.

Missing RUN data is reported as a separate, non-failure workflow outcome. A Fit stops after musrfit reports the missing input instead of launching musrview to repeat the same diagnostic. The full executable output remains in the .out log.