Add server-side safeguards so heavy web workflows cannot crash Apache: - Run-concurrency gate: whole run/series_run_candidate requests lease flock slots under TMPROOT/.musrflow-run-slots (MUSRFIT_MAX_RUNS=2 default, MUSRFIT_MAX_RUNS_PER_IP optional per-client pool); saturated servers answer 503 server-busy and leases are released when the CGI process exits. - Per-child resource wrapper: every external command runs with RLIMIT_AS (MUSRFIT_CHILD_MEM_KB, 16 GiB), optional RLIMIT_CPU, and exec nice (MUSRFIT_CHILD_NICE=10) so a runaway child cannot starve Apache workers. - Retune MUSRFIT_CMD_TIMEOUT default 400 -> 360 (deploy knob). - Size/job backstops: POST body cap before CGI->new (413), write MSR cap (413), and MUSRFIT_MAX_JOBS refusal of new job folders (never deletes). - Exclude the .musrflow-run-slots dir from prune_stale_jobs and job counting. - Document the new knobs in cgi/README.md and add CGI hardening tests.
13 KiB
Web CGI Endpoint
This folder contains the web runner endpoint for muSRFlow:
single_musrfit.cgi
The CGI writes files and runs the musrfit, musrview, and msr2data
executables on the server. Computational and text-transformation logic lives in
the shared muSRFitlib.js; the CGI keeps only the server-side file I/O, process
execution, and path/input security. Generated artifacts under
MUSRFIT_TMPROOT are exposed over HTTP through MUSRFIT_TMP_URL.
Deploy
Run sudo ./scripts/deploy-web.sh from the project root to install the browser
assets and this endpoint in the default Apache directories.
After deployment:
- Ensure binaries exist and are executable:
/usr/local/bin/musrfit/usr/local/bin/musrview/usr/local/bin/msr2data
- Ensure web tmp paths are writable:
/var/www/html/tmp
Configuration
The endpoint uses these environment variables when present and otherwise keeps the production defaults shown below:
| Variable | Default | Purpose |
|---|---|---|
MUSRFIT_WEBROOT |
/var/www/html |
Static web root used to derive the default temporary directory |
MUSRFIT_TMPROOT |
<webroot>/tmp |
Filesystem directory containing web jobs and archives |
MUSRFIT_TMP_URL |
/tmp |
URL path through which the web server exposes MUSRFIT_TMPROOT |
MUSRFIT_BIN |
/usr/local/bin/musrfit |
musrfit executable |
MUSRVIEW_BIN |
/usr/local/bin/musrview |
musrview executable |
MSR2DATA_BIN |
/usr/local/bin/msr2data |
msr2data executable |
MUSRFIT_DATA_ROOT |
/psi.ch/group/lmu/public/data |
Root containing <instrument>/his/<year> data directories; returned to the browser as a read-only setting |
MUSRFIT_STARTUP_XML |
/var/www/cgi-bin/musrfit_startup.xml |
Server-owned musrfit_startup.xml parsed to build portable RUN names; must be outside the web document root and never browser-supplied |
ROOTSYS |
/usr/lib64/root |
ROOT installation used by musrfit executables |
MUSRFITPATH |
/usr/local/bin |
Directory containing musrfit executables and XML files |
MUSRFIT_RUNTIME_HOME |
<tmp root> |
Overrides writable HOME for ROOT and musrfit configuration |
MUSRFULLDATAPATH |
<data root> |
Fallback search path for muSR data files referenced by MSR files |
MUSRFIT_CMD_TIMEOUT |
360 |
Seconds each external command (musrfit, musrview, msr2data, dump_header, tar) may run before the CGI kills its process group |
MUSRFIT_CHILD_MEM_KB |
16777216 |
RLIMIT_AS (KB) applied inside the forked child before every external command; 0 disables |
MUSRFIT_CHILD_CPU_SEC |
0 |
RLIMIT_CPU (seconds) applied inside the forked child; 0 = unlimited |
MUSRFIT_CHILD_NICE |
10 |
Scheduling priority the external command is launched at; 0 disables |
MUSRFIT_MAX_RUNS |
2 |
Maximum simultaneous run/series_run_candidate requests across all clients; extra requests get HTTP 503 server-busy; 0 blocks all heavy runs |
MUSRFIT_MAX_RUNS_PER_IP |
0 |
Per-client share of the concurrent-run pool (keyed by REMOTE_ADDR); 0 = one client may use the whole pool |
MUSRFIT_MAX_REQUEST_BYTES |
8000000 |
Request body cap checked against CONTENT_LENGTH before CGI parsing; larger requests get HTTP 413 request-too-large |
MUSRFIT_MAX_MSR_BYTES |
2000000 |
Cap on MSR content accepted by action=write; larger content gets HTTP 413 msr-too-large |
MUSRFIT_MAX_JOBS |
0 |
Maximum web job folders allowed; new job folders are refused with HTTP 500 once the ceiling is reached. Existing folders keep working and nothing is ever deleted; 0 = unlimited |
MUSRFIT_BIN must contain only the executable path. Fit requests invoke it as
musrfit -t -e --timeout 300 <file.msr>, matching the original CGI behavior.
The web server must expose MUSRFIT_TMPROOT at MUSRFIT_TMP_URL. The CGI
process also needs read access to the muSR data directories, either through the
generated MSR paths or MUSRFULLDATAPATH.
Within one browser page session, Fit and Plot reuse the same job folder while the first selected run stays unchanged. Changing the first run creates a new job folder. Each execution resets its log and generated ROOT plot files in the reused folder before running.
The template remains <run>_tmpl.msr as required by msr2data. A multi-run
global file uses the configured prefix, for example 514+global_asy.msr; the
temporary 514+global_tmpl.msr produced by msr2data is removed after it has
been validated and copied to the runnable name.
For Apache, set MUSRFIT_DATA_ROOT in the CGI environment when the data mount
is somewhere other than /psi.ch/group/lmu/public/data, for example /psi.ch/data. The browser
cannot override this server path.
Each run request holds one Apache/CGI worker for the whole
musrfit → msr2data → musrview → tar chain. A fit that finishes cleanly or is
killed by MUSRFIT_CMD_TIMEOUT returns the worker to Apache; only an unbounded
component or a worker crash leaves it stranded. Two additional safeguards keep a
saturated server responsive under many simultaneous users:
MUSRFIT_MAX_RUNSbounds how many heavy workflows run at once via flock leases on small files underMUSRFIT_TMPROOT/.musrflow-run-slots. A request that cannot obtain its lease answers HTTP 503server-busyinstead of queuing in a worker. Leases are released automatically when the CGI process exits, so a crashed worker can never leave a stale lock.- Every external command is launched with
MUSRFIT_CHILD_MEM_KBandMUSRFIT_CHILD_CPU_SECmachinery and theMUSRFIT_CHILD_NICEpriority, so a runaway child consumes bounded memory, bounded CPU, or low scheduling priority instead of starving Apache workers.
The slot directory is hidden (dot-prefixed) and is never counted as a job folder nor pruned by the job TTL cleanup.
Portable RUN naming and musrfit_startup.xml
The CGI can generate portable RUN/ADDRUN file names (relative paths in the
<run_name_template inst="..."> form) instead of the legacy absolute
<instrument>/his/<year>/... paths. This is opt-in and requires the admin to
install a startup XML:
scripts/deploy-web.shinstallscgi/musrfit_startup.xmlbeside the CGI atMUSRFIT_STARTUP_XML(default/var/www/cgi-bin/musrfit_startup.xml). The file lives outside the web document root, is owned by the server, and is never read from a browser-supplied path.- The health response reports the parsed
<data_path>entries and per-instrument<run_name_template>values underenvironment.startup_xml. - The CGI
resolve_run_nameaction is a scoped, server-sided existence check: it only confirms a candidate file under one of the configured<data_path>roots and rejects any other path. It is not a general file-existence API. - If no startup XML is configured, or it cannot be parsed, portable naming stays disabled and absolute RUN paths remain the default.
API
Endpoint: POST /cgi-bin/single_musrfit.cgi
action=health
Check the server configuration without creating or running a job. The response reports executable paths, availability, whether each binary can load and run, versions or loader diagnostics, temporary-directory access, and the data-root/ROOT runtime environment values. This makes missing ROOT libraries in the restricted CGI environment visible before a fit is attempted.
curl -s -X POST -d 'action=health' \
http://localhost/cgi-bin/single_musrfit.cgi
action=write
Write an MSR file:
fn:/tmp/<job>/<name>.msrline0..lineN: text lines
Returns JSON:
ok,code,message,job_name,work_url,msr_url
action=series_transaction
Stage and promote a validated batch of independent-run MSRs without executing
musrfit. The request contains:
fn: one/tmp/<job>/<name>.msrpath used only to identify the job foldertransaction: JSON containing a restricted transaction ID and candidate list
Every candidate is limited to the same job folder and must contain its numeric
run, fit|retry|refit action, run-prefixed .msr filename, and complete MSR
text. Retry/refit candidates must also provide the exact content observed during
inventory. The server derives all stage and backup names; browser-provided
filesystem paths are never accepted from the JSON document.
The endpoint exclusively stages and reads back the entire batch before any promotion. New targets use no-clobber publication. Replaced targets receive an exclusive recoverable backup. A failed promotion rolls earlier targets back in reverse order. Successful retry/refit backups are deliberately retained for the later sequential-runner commit/rollback step.
After promotion, the server seals the batch in a private transaction manifest.
The manifest records the server-derived target and backup names, content hashes,
and the five identity fields of every canonical RUN/ADDRUN directive.
Whitespace and trailing comments written by musrfit are not identity fields. The
manifest is not supplied or editable by the browser.
action=series_transaction_finalize
Commit or roll back one previously promoted independent-run transaction without running an executable. The request contains:
fn: one/tmp/<job>/<name>.msrpath used only to identify the job folderfinalization: JSON with the server transactionidand acommit|rollbackdecision
Commit first verifies the complete batch. Every target must be a valid,
converged fitted MSR and must retain the RUN/ADDRUN path, beamline,
institute, and data format recorded at promotion. Only then are replacement
backups deleted. Rollback operates in
reverse order: newly created targets are removed and replacement backups are
restored. The manifest is removed only after every item is finalized, allowing
a failed partial operation to be retried safely.
This endpoint only finalizes filesystem state. It does not call musrfit,
msr2data, or musrview and is not yet connected to the Fit button.
action=series_run_candidate
Run musrfit for exactly one target already protected by a server-created
series transaction. The request contains:
fn: the promoted/tmp/<job>/<run-name>.msrtargettransaction_id: the transaction that owns this target
The server locks the job, verifies the private manifest, checks that the target
still has its promoted content and RUN/ADDRUN identity, and then invokes
musrfit. Success requires a zero exit code, unchanged run identity, and a
converged fitted MSR. The endpoint writes the target's .out log but does not
run msr2data, musrview, package artifacts, synchronize the GUI, or finalize
the transaction.
action=series_inventory
Inspect the selected independent-run files without creating a job or running an executable. The request contains:
fn: one/tmp/<job>/<name>.msrpath used only to identify the job folderinventory: JSON lists of run-prefixed MSR filenames and resolved data paths
MSRs are restricted to the selected job folder. Data paths must match the
configured <data-root>/<instrument>/his/<year>/<file> layout. Symlinks are not
followed. Existing MSRs up to 2 MB are returned as text for the shared browser/
Electron classifier. Larger or unreadable files are reported as existing but
unreadable, which blocks replacement rather than treating them as missing.
The response is a single snapshot containing job_exists, msr_files, and
data_files. The browser verifies that every requested entry appears exactly
once before using it.
action=run
Run fitting/plot workflow on an existing MSR:
fn:/tmp/<job>/<name>.msruse_msr2data:0|1runs: space-separated run numbers (for global generation)template_name: e.g.123_tmpl.msrbase_run: first run numberfftv:y|nplot_refresh:1reruns onlymusrviewfor an existing floating plot and appends to its command log; it requiresplot_only=1
The CGI runs musrview --root in headless graphics mode because an Apache
process has no interactive display. Generated ROOT canvas files are shown by the
JSROOT web viewer and stored with the other job artifacts.
After a successful fit, it also runs the same parameter-extraction workflow as
the original CGI and writes <fit-name>_par.dat with msr2data. The fitted MSR
continues to update the editable GUI parameter table independently of this
run-wise export.
The browser links this export to the existing musrfit plotter using
/MusrPlot/?file=<encoded-parameter-url>. The plotter must be served from that
same-origin path, as in the original CGI deployment. Set
window.MusrfitPlotterPath before running a fit if a deployment uses another
route.
Returns JSON:
ok,code,message,stderroutcome:missing-datawhen musrfit or musrview cannot resolve RUN datamissing_files: paths extracted from the executable diagnosticsjob_name,work_url,msr_url,out_url,parameter_urlroot_url,root_urls,root_itemscommand_log,tgz_url
The .out artifact referenced by out_url is also the user-facing command log.
For every msr2data, musrfit, and musrview step it records the shell command,
combined standard output/error, and the exit status.
Missing RUN data is reported as a separate, non-failure workflow outcome. A Fit
stops after musrfit reports the missing input instead of launching musrview to
repeat the same diagnostic. The full executable output remains in the .out log.