Harden singleTrimSP.cgi: traversal fix, watchdog, pruner, archive lock, caps
- P0 path-traversal fix: run-name class is dot-free with \z anchors, so '.', '..', and %0A-trailing values can never escape TRIMSP_TMPROOT. The deployed path allowlist is derived from the configured root instead of the old hardcoded /tmp prefix. - Replacing external with an alarm() watchdog that kills the engine's whole process group (setpgrp + kill KILL -pid), matching muSRFlow; describe_result maps -2/-1/signal to 124/127/128+sig. - prune_stale_jobs() ages out dirs (TrimSP.cfg marker only), orphaned .tgz, and leaked per-request logs; safe in shared /tmp (symlinks skipped, remove_tree safe mode, never touches non-trimsp dirs). - Archive is serialized per run with flock (.trimsp-lock); per-point engine runs stay concurrent. - Symlink/non-dir run dirs refused; mode 0700 so results are not world-readable. read/download also reject symlinked targets. - Request caps: MAX_REQUEST_LINES 20000, MAX_REQUEST_BYTES 4 MiB -> 413. - action=health deployment self-check; env overrides TRIMSP_BIN, TMPROOT, CMD_TIMEOUT, JOB_MAXAGE (sed anchor for the binary path kept intact). - Wire contract preserved: ok/phase/message/exitCode/archiveUrl/log JSON, text read, gzip download, real HTTP statuses. No TrimSPweb.js change. - New tests/cgi-tests.sh drives the real CGI offline (18 cases: traversal, %0A, escaping round-trip, success/fail/hang engine, 124 watchdog, archive+- download, pruner, symlink refusal, 413). deploy-web.sh now verifies the sed substitution embedded the selected engine path.
This commit is contained in:
salman committed
1 parent
9d88fa5d65
commit
5a8bd70957
4 files changed
+721
-30
No files matched your search
@@ -13,4 +13,23 @@ Seed handling and input hardening (deferred from the B-list fixes)
|
||||
valEnergy must be upper-bounded from share links before the engine runs, to
|
||||
prevent an accidental or malicious "run forever / write enormous .rge" job.
|
||||
- Consider persisting the effective per-point seed back into the *.seq output
|
||||
metadata so results self-document their RNG state.
|
||||
metadata so results self-document their RNG state.
|
||||
|
||||
Deferred follow-ups from the C-list CGI hardening
|
||||
=================================================
|
||||
|
||||
- The perl "Statement unlikely to be reached" warning at the exec/_exit(127)
|
||||
child branch of run_bounded_command is intentional (matching muSRFlow) and
|
||||
does not affect `perl -c`; tests run with -X to keep output clean.
|
||||
- No global concurrency cap is implemented (by design, mirroring muSRFlow):
|
||||
concurrent per-point engine runs are allowed; archive is serialized per run
|
||||
with flock. Revisit if a runaway share link floods scan points.
|
||||
- Auth (MUCH-need, muSRFlow parity): none is implemented, keeping the read-only
|
||||
/tmp contract. If this CGI is ever exposed beyond a trusted LAN, gate it with
|
||||
httpd authentication or a shared-token header.
|
||||
- Restart integration (retch): the watchdog returns exit code 124 and the JSON
|
||||
"log" carries the description; a deployment that wants automated retries can
|
||||
key off exitCode === 124 and re-post the same .inp.
|
||||
- A future client change could surface TRIMSP_TMPROOT behind a shared
|
||||
configuration; the browser path regex is derived from the same env var, so
|
||||
both sides stay consistent.
|
||||
@@ -85,8 +85,16 @@ escaped_trimsp_binary="${escaped_trimsp_binary//|/\\|}"
|
||||
escaped_trimsp_binary="${escaped_trimsp_binary//\"/\\\"}"
|
||||
sed 's|^my \$trimsp_bin = .*|my $trimsp_bin = "'"$escaped_trimsp_binary"'";|' \
|
||||
"$project_root/singleTrimSP.cgi" > "$cgi_stage"
|
||||
perl -c "$cgi_stage" >/dev/null
|
||||
install -m 0755 "$cgi_stage" "$cgi_target"
|
||||
|
||||
# The installed CGI must embed the selected engine path exactly; a future
|
||||
# source layout change could silently break the sed substitution above.
|
||||
if ! grep -Fq 'my $trimsp_bin = "'"$trimsp_binary"'";' "$cgi_target"; then
|
||||
printf 'Deployed CGI does not embed the selected engine path %s\n' "$trimsp_binary" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# The browser assets and the CGI must be readable (and, for the CGI,
|
||||
# executable) by the web server. On the live server the web space is owned by
|
||||
# the apache user; on a developer machine the files may instead be owned by
|
||||
|
||||
+369
-29
@@ -2,25 +2,280 @@
|
||||
#
|
||||
# Copyright Zaher Salman 2023-.
|
||||
# zaher.salman@psi.ch
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# singleTrimSP.cgi
|
||||
#
|
||||
# Web backend for TRIMSP-NL: accepts a run directory + file name with the
|
||||
# submitted file content, runs the transport engine, and on request produces a
|
||||
# downloadable results archive.
|
||||
#
|
||||
# Actions:
|
||||
# (none) POST fn + rngType + lineN... write the file, then run/archive
|
||||
# read GET fn stream a run-directory file
|
||||
# download GET run stream the results archive (.tgz)
|
||||
# health GET deployment self-check (JSON)
|
||||
#
|
||||
# The browser contract is deliberately unchanged from before the hardening:
|
||||
# JSON responses keep ok/phase/message/exitCode/archiveUrl/log;
|
||||
# read/download stream text/gzip with real HTTP status codes.
|
||||
#
|
||||
# Environment overrides (all optional; nothing here requires Apache config):
|
||||
# TRIMSP_BIN - path to the trimspNL engine (the deploy script rewrites
|
||||
# the $trimsp_bin literal below; env is a fallback)
|
||||
# TRIMSP_TMPROOT - run root (default /tmp). The browser builds paths as
|
||||
# <root>/<run>/<file>, so changing this requires the
|
||||
# matching client change; the default is safe.
|
||||
# TRIMSP_CMD_TIMEOUT - per-command watchdog seconds (default 300). Must stay
|
||||
# above the engine's own internal bounds.
|
||||
# TRIMSP_JOB_MAXAGE - stale run/archive lifetime seconds (default 86400).
|
||||
|
||||
use strict;
|
||||
use warnings;
|
||||
use CGI;
|
||||
use File::Path qw(make_path remove_tree);
|
||||
use File::Temp qw(tempfile);
|
||||
use Fcntl qw(:DEFAULT :flock);
|
||||
use JSON::PP qw(encode_json);
|
||||
|
||||
# The web installer rewrites this path to the selected installed binary.
|
||||
my $trimsp_bin = "/usr/local/bin/trimspNL";
|
||||
my $workroot = "/tmp";
|
||||
|
||||
$trimsp_bin = $ENV{TRIMSP_BIN} if defined($ENV{TRIMSP_BIN}) && $ENV{TRIMSP_BIN} ne "";
|
||||
$workroot = $ENV{TRIMSP_TMPROOT} if defined($ENV{TRIMSP_TMPROOT}) && $ENV{TRIMSP_TMPROOT} ne "";
|
||||
|
||||
# Per-command watchdog. Every external command (trimspNL, tar) is bounded by an
|
||||
# alarm() watchdog in run_bounded_command and killed as a process group, so a
|
||||
# hung engine or tar cannot strand an Apache worker indefinitely.
|
||||
my $CMD_TIMEOUT = (defined($ENV{TRIMSP_CMD_TIMEOUT}) && $ENV{TRIMSP_CMD_TIMEOUT} =~ /^\d+$/)
|
||||
? int($ENV{TRIMSP_CMD_TIMEOUT}) : 300;
|
||||
|
||||
# Job/archive lifetime in seconds. prune_stale_jobs() removes trimsp run
|
||||
# directories and orphaned archives older than this, bounding disk usage from
|
||||
# long-running or abandoned browser sessions.
|
||||
my $JOB_MAXAGE = (defined($ENV{TRIMSP_JOB_MAXAGE}) && $ENV{TRIMSP_JOB_MAXAGE} =~ /^\d+$/)
|
||||
? int($ENV{TRIMSP_JOB_MAXAGE}) : 86400;
|
||||
|
||||
# Request-body caps. The browser posts one cfg, one .inp per scan point, and one
|
||||
# sequence file; these limits stay far above real decks while stopping a crafted
|
||||
# link from pushing unbounded content into the run directory.
|
||||
my $MAX_REQUEST_LINES = 20000;
|
||||
my $MAX_REQUEST_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
# Run name: alphanumerics, underscore, hyphen only (no dot, so ".", ".." can
|
||||
# never be a path component). A run-name class that admitted dots is exactly
|
||||
# the traversal hole muSRFlow carries; keep it dot-free. \z (not $) additionally
|
||||
# rejects a trailing newline from a %0A tail. The file component stays nested in
|
||||
# the same anchored pattern below, so no separate file-name regex is needed.
|
||||
my $run_name_re = qr{^[A-Za-z0-9_-]+\z};
|
||||
|
||||
# Derive the path allowlist from the configured root so a TRIMSP_TMPROOT
|
||||
# override stays consistent across every action.
|
||||
my $work_path_re = qr{^\Q$workroot\E/([A-Za-z0-9_-]+)/([A-Za-z0-9_.-]+)\z};
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
# Emit an error as a real HTTP status code with a JSON body, then exit. The
|
||||
# payload carries the original ok/phase/message fields only, so error responses
|
||||
# keep exactly the same shape the browser already understands.
|
||||
sub json_error {
|
||||
my ($status, $payload) = @_;
|
||||
print "Status: $status\r\n";
|
||||
print "Content-Type: application/json; charset=UTF-8\r\n";
|
||||
print "Cache-Control: no-store\r\n\r\n";
|
||||
print encode_json($payload);
|
||||
exit 0;
|
||||
}
|
||||
|
||||
# Shell quoting for browser-derived command components. After the path checks
|
||||
# the tokens still go through shell_quote(); only server-side literals are ever
|
||||
# interpolated unquoted.
|
||||
sub shell_quote {
|
||||
my ($s) = @_;
|
||||
$s =~ s/'/'\\''/g;
|
||||
return "'" . $s . "'";
|
||||
}
|
||||
|
||||
# Run an arbitrary shell command with an alarm() watchdog, isolating the child
|
||||
# in its own process group so a timeout can kill the whole pipeline (including
|
||||
# grandchildren) without touching this CGI process. Returns system()'s waited
|
||||
# status encoding, or -1 (could not start) / -2 (watchdog fired).
|
||||
sub run_bounded_command {
|
||||
my ($command) = @_;
|
||||
my $pid = fork();
|
||||
return -1 if !defined($pid);
|
||||
if ($pid == 0) {
|
||||
setpgrp(0, 0);
|
||||
umask(022);
|
||||
open(STDIN, "<", "/dev/null") or _exit(127);
|
||||
exec { "/bin/sh" } "/bin/sh", "-c", $command;
|
||||
_exit(127);
|
||||
}
|
||||
|
||||
my $timed_out = 0;
|
||||
local $SIG{ALRM} = sub {
|
||||
$timed_out = 1;
|
||||
kill "KILL", -$pid;
|
||||
};
|
||||
my $status = -1;
|
||||
eval {
|
||||
alarm($CMD_TIMEOUT);
|
||||
waitpid($pid, 0);
|
||||
alarm(0);
|
||||
$status = $?;
|
||||
};
|
||||
my $wait_error = $@;
|
||||
alarm(0);
|
||||
if ($timed_out) {
|
||||
# waitpid may be interrupted by SIGALRM or return after the killed child
|
||||
# is reaped. A second call is harmless in the latter case and avoids a
|
||||
# zombie in the former.
|
||||
waitpid($pid, 0);
|
||||
return -2;
|
||||
}
|
||||
if ($wait_error) {
|
||||
kill "KILL", -$pid;
|
||||
waitpid($pid, 0);
|
||||
return -1;
|
||||
}
|
||||
return $status;
|
||||
}
|
||||
|
||||
# Convert a run_bounded_command result into (exit_code, human description)
|
||||
# matching the vocabulary the browser shows in diagnostics.
|
||||
sub describe_result {
|
||||
my ($result) = @_;
|
||||
if ($result == -2) {
|
||||
return (124, "timed out after ${CMD_TIMEOUT}s");
|
||||
} elsif ($result == -1) {
|
||||
return (127, "failed to start: $!");
|
||||
} elsif ($result & 127) {
|
||||
return (128 + ($result & 127), "terminated by signal " . ($result & 127));
|
||||
}
|
||||
return ($result >> 8, "exit code " . ($result >> 8));
|
||||
}
|
||||
|
||||
# Accept or create the run directory. Refuses symlinks and non-directories so a
|
||||
# pre-created /tmp/<run> link or planted file cannot redirect our writes, and
|
||||
# uses mode 0700 so results are not world-readable in the shared temp root.
|
||||
sub ensure_safe_work_dir {
|
||||
my ($work_path) = @_;
|
||||
if (-l $work_path || (-e $work_path && !-d $work_path)) {
|
||||
return (0, "job directory is not a plain directory");
|
||||
}
|
||||
if (!-d $work_path) {
|
||||
if (!-d $workroot) {
|
||||
return (0, "run root does not exist");
|
||||
}
|
||||
my $ok = eval {
|
||||
make_path($work_path, { mode => 0700, error => \my $err });
|
||||
!@{$err};
|
||||
};
|
||||
if (!$ok) {
|
||||
return (0, "could not create job directory");
|
||||
}
|
||||
}
|
||||
return (1, "");
|
||||
}
|
||||
|
||||
# Occupy the per-run archive lock. The archive step mutates the whole run
|
||||
# directory (tar + remove_tree), so it is serialized; the per-point engine runs
|
||||
# are intentionally concurrent and are not locked. The kernel releases flock
|
||||
# when the CGI exits, so a dying worker cannot strand the lock.
|
||||
sub lock_archive {
|
||||
my ($work_path) = @_;
|
||||
my $lock_path = "$work_path/.trimsp-lock";
|
||||
my $lfh;
|
||||
if (!sysopen($lfh, $lock_path, O_RDWR | O_CREAT, 0600)) {
|
||||
return undef;
|
||||
}
|
||||
if (!flock($lfh, LOCK_EX | LOCK_NB)) {
|
||||
close($lfh);
|
||||
return undef;
|
||||
}
|
||||
return $lfh;
|
||||
}
|
||||
|
||||
# Age out abandoned trimsp jobs. Mirrors muSRFlow's pruner but stays safe in
|
||||
# the shared /tmp root: only directories that match the run-name rule AND
|
||||
# contain a TrimSP.cfg marker are candidates, plus orphaned <run>.tgz archives
|
||||
# whose run directory is already gone, plus leaked per-request log files.
|
||||
# Symlinks are skipped and remove_tree runs in safe mode, so a crafted entry is
|
||||
# never followed or deleted.
|
||||
sub prune_stale_jobs {
|
||||
return 0 if !-d $workroot;
|
||||
my $now = time();
|
||||
my $removed = 0;
|
||||
|
||||
opendir(my $dh, $workroot) or return 0;
|
||||
while (my $entry = readdir($dh)) {
|
||||
next if $entry eq "." || $entry eq "..";
|
||||
next if $entry !~ /^[A-Za-z0-9_-]{5,}$/;
|
||||
my $candidate = "$workroot/$entry";
|
||||
next if -l $candidate || !-d $candidate;
|
||||
# Only ever touch directories this backend creates.
|
||||
next if !-f "$candidate/TrimSP.cfg";
|
||||
my @stats = stat($candidate);
|
||||
next if @stats < 9;
|
||||
next if $stats[9] > $now - $JOB_MAXAGE;
|
||||
# Never delete a directory that is currently being archived.
|
||||
my $lock = "$candidate/.trimsp-lock";
|
||||
if (-e $lock) {
|
||||
sysopen(my $lfh, $lock, O_RDWR | O_CREAT, 0600) or next;
|
||||
if (!flock($lfh, LOCK_EX | LOCK_NB)) { close($lfh); next; }
|
||||
close($lfh);
|
||||
}
|
||||
remove_tree($candidate, { safe => 1, error => \my $err });
|
||||
$removed++ if !@{$err};
|
||||
}
|
||||
closedir($dh);
|
||||
|
||||
# Orphaned archives: /tmp/<run>.tgz whose run directory is already gone.
|
||||
opendir($dh, $workroot) or return $removed;
|
||||
while (my $entry = readdir($dh)) {
|
||||
next if $entry !~ /^([A-Za-z0-9_-]{5,})\.tgz\z/;
|
||||
next if -d "$workroot/$1";
|
||||
my $path = "$workroot/$entry";
|
||||
next if -l $path || !-f $path;
|
||||
my @stats = stat($path);
|
||||
next if @stats < 9 || $stats[9] > $now - $JOB_MAXAGE;
|
||||
unlink($path);
|
||||
$removed++;
|
||||
}
|
||||
closedir($dh);
|
||||
|
||||
# Leaked per-request logs from requests that died before unlink().
|
||||
opendir($dh, $workroot) or return $removed;
|
||||
while (my $entry = readdir($dh)) {
|
||||
next if $entry !~ /^trimsp-([A-Za-z0-9_-]{5,})-[A-Za-z0-9_-]{4,}\z/;
|
||||
my $path = "$workroot/$entry";
|
||||
next if -l $path || !-f $path;
|
||||
my @stats = stat($path);
|
||||
next if @stats < 9 || $stats[9] > $now - $JOB_MAXAGE;
|
||||
unlink($path);
|
||||
$removed++;
|
||||
}
|
||||
closedir($dh);
|
||||
|
||||
return $removed;
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Request dispatch
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
my $in = CGI->new;
|
||||
my $action = $in->param("action") // "run";
|
||||
|
||||
# Opportunistically age out stale jobs before serving or mutating anything.
|
||||
prune_stale_jobs();
|
||||
|
||||
if ($action eq "read") {
|
||||
my $read_filename = $in->param("fn") // "";
|
||||
if ($read_filename !~ m{^/tmp/[A-Za-z0-9_-]+/[A-Za-z0-9_.-]+$}
|
||||
|| !-f $read_filename) {
|
||||
if ($read_filename !~ $work_path_re || !-f $read_filename || -l $read_filename) {
|
||||
print "Status: 404 Not Found\r\n";
|
||||
print "Content-Type: text/plain; charset=UTF-8\r\n\r\n";
|
||||
print "Requested simulation output was not found.\n";
|
||||
@@ -44,7 +299,7 @@ if ($action eq "read") {
|
||||
if ($action eq "download") {
|
||||
my $download_run = $in->param("run") // "";
|
||||
my $archive_path = "$workroot/$download_run.tgz";
|
||||
if ($download_run !~ /^[A-Za-z0-9_-]+$/ || !-f $archive_path) {
|
||||
if ($download_run !~ $run_name_re || !-f $archive_path || -l $archive_path) {
|
||||
print "Status: 404 Not Found\r\n";
|
||||
print "Content-Type: text/plain; charset=UTF-8\r\n\r\n";
|
||||
print "Requested results archive was not found.\n";
|
||||
@@ -69,20 +324,48 @@ if ($action eq "download") {
|
||||
exit 0;
|
||||
}
|
||||
|
||||
if ($action eq "health") {
|
||||
my %probe;
|
||||
$probe{engine} = {
|
||||
exists => (-e $trimsp_bin) ? JSON::PP::true : JSON::PP::false,
|
||||
executable => (-x $trimsp_bin) ? JSON::PP::true : JSON::PP::false,
|
||||
size => (-f $trimsp_bin && !-l $trimsp_bin) ? (-s $trimsp_bin || 0) : 0,
|
||||
};
|
||||
$probe{runRoot} = {
|
||||
exists => (-d $workroot) ? JSON::PP::true : JSON::PP::false,
|
||||
writable => (-w $workroot) ? JSON::PP::true : JSON::PP::false,
|
||||
};
|
||||
$probe{settings} = {
|
||||
commandTimeoutSeconds => $CMD_TIMEOUT,
|
||||
jobMaxAgeSeconds => $JOB_MAXAGE,
|
||||
maxRequestBytes => $MAX_REQUEST_BYTES,
|
||||
maxRequestLines => $MAX_REQUEST_LINES,
|
||||
};
|
||||
print "Content-Type: application/json\r\n";
|
||||
print "Cache-Control: no-store\r\n\r\n";
|
||||
print encode_json({
|
||||
ok => JSON::PP::true,
|
||||
action => "health",
|
||||
probe => \%probe,
|
||||
});
|
||||
exit 0;
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Write + run path (default action). The browser posts fn + rngType + lineN...
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
my $requested_filename = $in->param("fn") // "";
|
||||
my $rng_type = $in->param("rngType") // "pcg32";
|
||||
|
||||
# Accept only the private run path produced by TrimSPweb.js. Restricting both
|
||||
# path components prevents CGI parameters from selecting arbitrary files.
|
||||
if ($requested_filename !~ m{^/tmp/([A-Za-z0-9_-]+)/([A-Za-z0-9_.-]+)$}) {
|
||||
print "Status: 400 Bad Request\r\n";
|
||||
print "Content-Type: application/json\r\n\r\n";
|
||||
print encode_json({
|
||||
if ($requested_filename !~ $work_path_re) {
|
||||
json_error(400, {
|
||||
ok => JSON::PP::false,
|
||||
phase => "request",
|
||||
message => "Invalid simulation filename",
|
||||
});
|
||||
exit 0;
|
||||
}
|
||||
|
||||
my ($run_name, $file_name) = ($1, $2);
|
||||
@@ -98,18 +381,36 @@ $run_base =~ s/\.inp$//;
|
||||
my @line_parameters = sort {
|
||||
($a =~ /line(\d+)/)[0] <=> ($b =~ /line(\d+)/)[0]
|
||||
} grep { /^line\d+$/ } $in->param;
|
||||
if (@line_parameters > $MAX_REQUEST_LINES) {
|
||||
json_error(413, {
|
||||
ok => JSON::PP::false,
|
||||
phase => "request",
|
||||
message => "Too many submitted lines",
|
||||
});
|
||||
}
|
||||
my $content = join("\n", map { scalar($in->param($_) // "") } @line_parameters) . "\n";
|
||||
if (length($content) > $MAX_REQUEST_BYTES) {
|
||||
json_error(413, {
|
||||
ok => JSON::PP::false,
|
||||
phase => "request",
|
||||
message => "Submitted file content is too large",
|
||||
});
|
||||
}
|
||||
|
||||
make_path($work_path, { mode => 0755 }) unless -d $work_path;
|
||||
my ($dir_ok, $dir_message) = ensure_safe_work_dir($work_path);
|
||||
if (!$dir_ok) {
|
||||
json_error(500, {
|
||||
ok => JSON::PP::false,
|
||||
phase => "request",
|
||||
message => $dir_message,
|
||||
});
|
||||
}
|
||||
open(my $output_file, ">", $filename) or do {
|
||||
print "Status: 500 Internal Server Error\r\n";
|
||||
print "Content-Type: application/json\r\n\r\n";
|
||||
print encode_json({
|
||||
json_error(500, {
|
||||
ok => JSON::PP::false,
|
||||
phase => "write",
|
||||
message => "Could not write the submitted simulation file",
|
||||
});
|
||||
exit 0;
|
||||
};
|
||||
print {$output_file} $content;
|
||||
close($output_file);
|
||||
@@ -122,6 +423,16 @@ my $log_content = "";
|
||||
my $request_logfile;
|
||||
my $ok = JSON::PP::true;
|
||||
|
||||
# Append diagnostics to the private request log before and after a command so
|
||||
# the JSON log field explains timeouts/signals instead of a bare exit code.
|
||||
sub append_log {
|
||||
my ($logfile, $text) = @_;
|
||||
return unless $logfile;
|
||||
return unless open(my $lfh, ">>", $logfile);
|
||||
print {$lfh} $text;
|
||||
close($lfh);
|
||||
}
|
||||
|
||||
if ($filename =~ /_Seq_Results\.dat$/) {
|
||||
$phase = "archive";
|
||||
unlink("$work_path/edist");
|
||||
@@ -132,16 +443,40 @@ if ($filename =~ /_Seq_Results\.dat$/) {
|
||||
my ($log_handle, $logfile) = tempfile("trimsp-$run_name-XXXXXX", DIR => $workroot, UNLINK => 0);
|
||||
close($log_handle);
|
||||
$request_logfile = $logfile;
|
||||
my $command = "tar -czf '$archive_path' -C '$work_path' . > '$logfile' 2>&1";
|
||||
system($command);
|
||||
$exit_code = $? == -1 ? 127 : $? >> 8;
|
||||
if ($exit_code == 0) {
|
||||
$archive_url = "/cgi-bin/singleTrimSP.cgi?action=download&run=$run_name";
|
||||
$message = "Created results archive";
|
||||
remove_tree($work_path);
|
||||
} else {
|
||||
|
||||
my $lock_fh = lock_archive($work_path);
|
||||
if (!$lock_fh) {
|
||||
$ok = JSON::PP::false;
|
||||
$message = "Could not create results archive";
|
||||
$message = "Another archive operation is active in this run";
|
||||
} else {
|
||||
# Replace a previous archive defensively, refusing a symlink target.
|
||||
if (-e $archive_path || -l $archive_path) {
|
||||
if (-l $archive_path || !-f $archive_path || !unlink($archive_path)) {
|
||||
$ok = JSON::PP::false;
|
||||
$message = "Could not replace the previous results archive";
|
||||
}
|
||||
}
|
||||
if ($ok) {
|
||||
my $command = "tar -czf " . shell_quote($archive_path)
|
||||
. " -C " . shell_quote($work_path)
|
||||
. " . >> " . shell_quote($logfile) . " 2>&1";
|
||||
append_log($logfile, "\n[archive] \$ $command\n");
|
||||
my $result = run_bounded_command($command);
|
||||
($exit_code, my $completion) = describe_result($result);
|
||||
append_log($logfile, "[${completion}]\n");
|
||||
if ($exit_code == 0 && -s $archive_path) {
|
||||
$archive_url = "/cgi-bin/singleTrimSP.cgi?action=download&run=$run_name";
|
||||
$message = "Created results archive";
|
||||
remove_tree($work_path);
|
||||
} else {
|
||||
# A run that produced correct physics must not be reported as a
|
||||
# failure just because archiving failed.
|
||||
$ok = JSON::PP::false;
|
||||
$message = "Could not create results archive";
|
||||
unlink($archive_path) if -f $archive_path && !-l $archive_path;
|
||||
}
|
||||
}
|
||||
close($lock_fh);
|
||||
}
|
||||
} elsif ($filename !~ /TrimSP\.cfg$/) {
|
||||
$phase = "simulation";
|
||||
@@ -151,14 +486,18 @@ if ($filename =~ /_Seq_Results\.dat$/) {
|
||||
|
||||
# All user-derived command components were restricted above. Each request
|
||||
# receives a private log file so concurrent users never share output.
|
||||
my $command = "cd '$work_path' && timeout 300 '$trimsp_bin' "
|
||||
. "'$run_base' '$rng_type' >> '$logfile' 2>&1";
|
||||
system($command);
|
||||
$exit_code = $? == -1 ? 127 : $? >> 8;
|
||||
my $command = "cd " . shell_quote($work_path) . " && "
|
||||
. shell_quote($trimsp_bin) . " "
|
||||
. shell_quote($run_base) . " "
|
||||
. shell_quote($rng_type) . " >> " . shell_quote($logfile) . " 2>&1";
|
||||
append_log($logfile, "\n[simulation] \$ $command\n");
|
||||
my $result = run_bounded_command($command);
|
||||
($exit_code, my $completion) = describe_result($result);
|
||||
append_log($logfile, "[${completion}]\n");
|
||||
$ok = JSON::PP::false if $exit_code != 0;
|
||||
$message = $exit_code == 0
|
||||
? "TRIM.SP-NL completed $run_base"
|
||||
: "TRIM.SP-NL failed while running $run_base";
|
||||
: "TRIM.SP-NL failed while running $run_base ($completion)";
|
||||
}
|
||||
|
||||
# Return at most the latest 64 KiB to the browser and immediately remove the
|
||||
@@ -172,7 +511,8 @@ if ($request_logfile && -f $request_logfile && open(my $log_handle, "<", $reques
|
||||
unlink($request_logfile);
|
||||
}
|
||||
|
||||
print "Content-Type: application/json\r\n\r\n";
|
||||
print "Content-Type: application/json; charset=UTF-8\r\n";
|
||||
print "Cache-Control: no-store\r\n\r\n";
|
||||
print encode_json({
|
||||
ok => $ok,
|
||||
phase => $phase,
|
||||
@@ -182,4 +522,4 @@ print encode_json({
|
||||
log => $log_content,
|
||||
});
|
||||
|
||||
exit 0;
|
||||
exit 0;
|
||||
@@ -0,0 +1,324 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# CGI hardening tests for singleTrimSP.cgi.
|
||||
#
|
||||
# Drives the real script directly (no web server) using REQUEST_METHOD /
|
||||
# CONTENT_LENGTH / QUERY_STRING env vars, exactly like a CGI runtime would.
|
||||
# A fake trimspNL engine provides success/fail/hang behavior so the watchdog,
|
||||
# logging, and archive paths can be exercised deterministically.
|
||||
#
|
||||
# Usage: tests/cgi-tests.sh
|
||||
# Exits non-zero on the first failing test.
|
||||
|
||||
set -u
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
CGI="$HERE/../singleTrimSP.cgi"
|
||||
ROOT_REPO="$(cd "$HERE/.." && pwd)"
|
||||
|
||||
TMP="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
ROOT="$TMP/root" # TRIMSP_TMPROOT for every request
|
||||
mkdir -p "$ROOT"
|
||||
|
||||
BIN="$TMP/bin/trimspNL" # fake engine
|
||||
mkdir -p "$TMP/bin"
|
||||
|
||||
cat > "$BIN" <<'EOF'
|
||||
#!/bin/sh
|
||||
case "${FAKE_MODE:-ok}" in
|
||||
fail)
|
||||
echo "fake engine failed as requested" 1>&2
|
||||
exit 3
|
||||
;;
|
||||
hang)
|
||||
echo "$$" > "$FAKE_PIDFILE"
|
||||
sleep 120
|
||||
exit 0
|
||||
;;
|
||||
ok)
|
||||
base="$1"
|
||||
printf '# col1 col2\n1 5\n2 10\n' > "$base.rge"
|
||||
printf 'SEQ %s line1\nSEQ %s line2\n' "$base" "$base" > "$base.seq"
|
||||
echo "fake engine completed $base"
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
EOF
|
||||
chmod +x "$BIN"
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
note() { printf '\n== %s\n' "$*"; }
|
||||
ok() { PASS=$((PASS+1)); printf ' PASS: %s\n' "$*"; }
|
||||
bad() { FAIL=$((FAIL+1)); printf ' FAIL: %s\n' "$*"; }
|
||||
|
||||
# run_request METHOD QUERY_STRING BODY_FILE -> sets RUN_HEADERS, RUN_BODY, RUN_STATUS
|
||||
run_request() {
|
||||
local method="$1" query="$2" body_file="$3"
|
||||
local in_redir="/dev/null"
|
||||
local BODY_LEN=0
|
||||
if [ "$method" = "POST" ] && [ -n "$body_file" ]; then
|
||||
BODY_LEN="$(wc -c < "$body_file")"
|
||||
in_redir="$body_file"
|
||||
fi
|
||||
local out
|
||||
out="$(REQUEST_METHOD="$method" \
|
||||
CONTENT_TYPE="application/x-www-form-urlencoded" \
|
||||
CONTENT_LENGTH="$BODY_LEN" \
|
||||
QUERY_STRING="$query" \
|
||||
TRIMSP_TMPROOT="$ROOT" \
|
||||
TRIMSP_BIN="$BIN" \
|
||||
TRIMSP_CMD_TIMEOUT="4" \
|
||||
TRIMSP_JOB_MAXAGE="86400" \
|
||||
/usr/bin/perl -X "$CGI" < "$in_redir" 2>&1)"
|
||||
# Split header block from body.
|
||||
RUN_HEADERS="$(printf '%s\n' "$out" | sed -n '1,/^\r*$/p' | head -n -1)"
|
||||
RUN_BODY="$(printf '%s\n' "$out" | sed -n '/^\r*$/,$p' | tail -n +1)"
|
||||
# Without a status line the response is implicitly 200.
|
||||
if printf '%s' "$RUN_HEADERS" | grep -q '^Status:'; then
|
||||
RUN_STATUS="$(printf '%s' "$RUN_HEADERS" | sed -n 's/^Status: *\([0-9][0-9][0-9]\).*/\1/p' | head -1)"
|
||||
else
|
||||
RUN_STATUS="200"
|
||||
fi
|
||||
}
|
||||
|
||||
body_contains() { printf '%s' "$RUN_BODY" | grep -q "$1"; }
|
||||
|
||||
# Percent-encode a value the way the browser's URLSearchParams does.
|
||||
urlencode() {
|
||||
printf '%s' "$1" | perl -pe 's/([^A-Za-z0-9._~-])/sprintf("%%%02X", ord($1))/ge'
|
||||
}
|
||||
|
||||
# The CGI derives its path allowlist from TRIMSP_TMPROOT, so test bodies must
|
||||
# reference the configured root (default /tmp in production).
|
||||
FN_ROOT="$ROOT"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "health"
|
||||
run_request GET "action=health" ""
|
||||
if body_contains '"ok":true' && body_contains '"commandTimeoutSeconds"'; then
|
||||
ok "health reports ok + settings"
|
||||
else
|
||||
bad "health response: $RUN_BODY"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "path traversal is rejected"
|
||||
TRAV="$TMP/trav.txt"
|
||||
printf 'action=run&fn=%s&line0=x\n' "$(urlencode "$FN_ROOT/../../escape/x.inp")" > "$TRAV"
|
||||
run_request POST "" "$TRAV"
|
||||
if [ "$RUN_STATUS" = "400" ] && body_contains 'Invalid simulation filename'; then
|
||||
ok "fn contains /../ component -> 400"
|
||||
else
|
||||
bad "traversal not rejected (status=$RUN_STATUS body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
printf 'action=run&fn=%s&line0=x\n' "$(urlencode "$FN_ROOT/../x.inp")" > "$TRAV"
|
||||
run_request POST "" "$TRAV"
|
||||
if [ "$RUN_STATUS" = "400" ]; then
|
||||
ok "single-level /../ in run component -> 400"
|
||||
else
|
||||
bad "single-level traversal not rejected (status=$RUN_STATUS body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
printf 'action=run&fn=%s&line0=x\n' "$(urlencode "$FN_ROOT/run12345/x.inp")%0A" > "$TRAV"
|
||||
run_request POST "" "$TRAV"
|
||||
if [ "$RUN_STATUS" = "400" ]; then
|
||||
ok "trailing newline (%0A) in fn rejected"
|
||||
else
|
||||
bad "%0A tail not rejected (status=$RUN_STATUS body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "write/read round-trip with special characters"
|
||||
RUN=round1234
|
||||
CFG="$TMP/cfgbody"
|
||||
printf 'fn=%s&line0=%s&line1=%s\n' \
|
||||
"$(urlencode "$FN_ROOT/$RUN/TrimSP.cfg")" \
|
||||
"ProjectileParameters" \
|
||||
"$(urlencode 'val=100¬e=semi+colon%,#hash')" \
|
||||
> "$CFG"
|
||||
run_request POST "" "$CFG"
|
||||
if [ "$RUN_STATUS" = "200" ] && body_contains '"phase":"save"'; then
|
||||
ok "TrimSP.cfg written (phase=save)"
|
||||
else
|
||||
bad "cfg write failed (status=$RUN_STATUS body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
run_request GET "action=read&fn=$(urlencode "$FN_ROOT/$RUN/TrimSP.cfg")" ""
|
||||
if body_contains 'val=100¬e=semi+colon%,#hash'; then
|
||||
ok "read-back matches decoded content exactly (escaped +,%,&,#,=)"
|
||||
else
|
||||
bad "read-back mismatch: $RUN_BODY"
|
||||
fi
|
||||
|
||||
# Invalid read path (symlink-leading component)
|
||||
ln -s /etc "$ROOT/$RUN/linkdir" 2>/dev/null || true
|
||||
run_request GET "action=read&fn=$(urlencode "$FN_ROOT/$RUN/linkdir")" ""
|
||||
if [ "$RUN_STATUS" = "404" ]; then
|
||||
ok "read of non-regular file -> 404"
|
||||
else
|
||||
bad "read of symlink not rejected (status=$RUN_STATUS)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "engine success"
|
||||
run_request POST "" "$TMP/cfgbody" # ensure run dir exists
|
||||
INP="$TMP/inpbody"
|
||||
printf 'fn=%s&rngType=pcg32&line0=X\n' "$(urlencode "$FN_ROOT/$RUN/${RUN}_E100.inp")" > "$INP"
|
||||
FAKE_MODE=ok run_request POST "" "$INP"
|
||||
if body_contains '"exitCode":0' && body_contains '"ok":true'; then
|
||||
ok "engine completed OK"
|
||||
else
|
||||
bad "engine success not reported (body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
# The .seq file the fake engine wrote must be readable.
|
||||
run_request GET "action=read&fn=$(urlencode "$FN_ROOT/$RUN/${RUN}_E100.seq")" ""
|
||||
if body_contains 'SEQ'; then
|
||||
ok "engine output .seq readable"
|
||||
else
|
||||
bad "engine .seq not readable (status=$RUN_STATUS body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "engine failure is reported, not silently ignored"
|
||||
FAILINP="$TMP/failbody"
|
||||
printf 'fn=%s&rngType=pcg32&line0=X\n' "$(urlencode "$FN_ROOT/$RUN/${RUN}_E200.inp")" > "$FAILINP"
|
||||
FAKE_MODE=fail run_request POST "" "$FAILINP"
|
||||
if body_contains '"exitCode":3' && body_contains '"ok":false'; then
|
||||
ok "engine failure -> exitCode 3, ok=false"
|
||||
else
|
||||
bad "engine failure misreported (body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "watchdog kills a hung engine (alarm + process-group kill)"
|
||||
HANGINP="$TMP/hangbody"
|
||||
printf 'fn=%s&rngType=pcg32&line0=X\n' "$(urlencode "$FN_ROOT/$RUN/${RUN}_E300.inp")" > "$HANGINP"
|
||||
HANG_PIDF="$TMP/pid.txt"
|
||||
FAKE_MODE=hang FAKE_PIDFILE="$HANG_PIDF" TRIMSP_CMD_TIMEOUT="2" run_request POST "" "$HANGINP"
|
||||
if ! [ -e "$HANG_PIDF" ]; then
|
||||
bad "hang engine never recorded a pid (test setup broken)"
|
||||
else
|
||||
DEAD="$(kill -0 "$(cat "$HANG_PIDF")" 2>/dev/null && echo no || echo yes)"
|
||||
if body_contains '"exitCode":124' && [ "$DEAD" = "yes" ]; then
|
||||
ok "hung engine timed out -> exitCode 124, process group reaped"
|
||||
else
|
||||
bad "watchdog did not fire correctly (exit=$RUN_BODY dead=$DEAD)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "archive + download"
|
||||
# The sequence file triggers the archive of the whole run directory.
|
||||
SEQ="$TMP/seqbody"
|
||||
printf 'fn=%s&line0=S\n' "$(urlencode "$FN_ROOT/$RUN/${RUN}_Seq_Results.dat")" > "$SEQ"
|
||||
run_request POST "" "$SEQ"
|
||||
if body_contains '"archiveUrl"' && body_contains '"phase":"archive"' && [ ! -d "$ROOT/$RUN" ]; then
|
||||
ok "archive created and run directory removed"
|
||||
else
|
||||
bad "archive flow broken (body=$RUN_BODY dir_gone=$( [ ! -d "$ROOT/$RUN" ] && echo yes || echo no))"
|
||||
fi
|
||||
|
||||
# Binary download: run through a file so NUL bytes survive.
|
||||
REQ="$TMP/dlreq"
|
||||
{
|
||||
printf 'action=download&run=%s' "$RUN"
|
||||
printf '\n'
|
||||
} > "$REQ"
|
||||
DL="$TMP/dl.out"
|
||||
REQUEST_METHOD=GET CONTENT_TYPE="application/x-www-form-urlencoded" \
|
||||
CONTENT_LENGTH=0 QUERY_STRING="action=download&run=$RUN" \
|
||||
TRIMSP_TMPROOT="$ROOT" TRIMSP_BIN="$BIN" \
|
||||
TRIMSP_CMD_TIMEOUT="4" TRIMSP_JOB_MAXAGE="86400" \
|
||||
/usr/bin/perl -X "$CGI" < /dev/null > "$DL" 2>&1
|
||||
if grep -a '^Status: 404' "$DL" >/dev/null; then
|
||||
bad "download 404 for archived run"
|
||||
elif grep -a '^Status:' "$DL" >/dev/null; then
|
||||
bad "download unexpected error ($(grep -a '^Status:' "$DL" | head -1))"
|
||||
else
|
||||
DL_BODY="$TMP/dl.body"
|
||||
perl -0777 -e '
|
||||
local $/; my $raw = <>;
|
||||
my ($body) = $raw =~ /^.*?\r?\n\r?\n(.*)\z/s;
|
||||
print STDOUT $body' "$DL" > "$DL_BODY"
|
||||
if [ "$(dd if="$DL_BODY" bs=1 count=2 2>/dev/null | od -An -tx1 | tr -d ' \n')" = "1f8b" ]; then
|
||||
ok "download returns a gzip stream"
|
||||
else
|
||||
bad "download did not return gzip (hex=$(dd if="$DL_BODY" bs=1 count=8 2>/dev/null | od -An -tx1 | tr -d ' \n'))"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "pruner removes stale run dirs and keeps fresh ones"
|
||||
STALE=stale0001
|
||||
FRESH=fresh0001
|
||||
mkdir -p "$ROOT/$STALE" "$ROOT/$FRESH"
|
||||
printf 'cfg\n' > "$ROOT/$STALE/TrimSP.cfg"
|
||||
printf 'x\n' > "$ROOT/$STALE/extra"
|
||||
printf 'cfg\n' > "$ROOT/$FRESH/TrimSP.cfg"
|
||||
# Backdate AFTER populating so the mtime that matters (dir mtime) is stale.
|
||||
touch -d "2000-01-01 00:00:00" "$ROOT/$STALE"
|
||||
run_request GET "action=health" ""
|
||||
if [ ! -d "$ROOT/$STALE" ] && [ -d "$ROOT/$FRESH" ]; then
|
||||
ok "pruner removed stale run dir, kept fresh one"
|
||||
else
|
||||
bad "pruner misbehaved (stale_gone=$( [ ! -d "$ROOT/$STALE" ] && echo yes || echo no) fresh_kept=$( [ -d "$ROOT/$FRESH" ] && echo yes || echo no))"
|
||||
fi
|
||||
|
||||
# Orphaned archive pruned.
|
||||
touch -d "2000-01-01 00:00:00" "$ROOT/orphan1234.tgz"
|
||||
run_request GET "action=health" ""
|
||||
if [ ! -e "$ROOT/orphan1234.tgz" ]; then
|
||||
ok "pruner removed orphaned archive"
|
||||
else
|
||||
bad "orphaned archive not pruned"
|
||||
fi
|
||||
|
||||
# Non-trimsp directory in the shared root is never touched.
|
||||
OTHER="$ROOT/other-user-000"
|
||||
mkdir -p "$OTHER"
|
||||
touch -d "2000-01-01 00:00:00" "$OTHER"
|
||||
run_request GET "action=health" ""
|
||||
if [ -d "$OTHER" ]; then
|
||||
ok "unrelated directory left untouched"
|
||||
else
|
||||
bad "pruner removed unrelated directory"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "symlinked run directory is refused"
|
||||
mkdir -p "$TMP/real_dir"
|
||||
ln -s "$TMP/real_dir" "$ROOT/symlnk001"
|
||||
SYM="$TMP/symbody"
|
||||
printf 'fn=%s&line0=x\n' "$(urlencode "$FN_ROOT/symlnk001/TrimSP.cfg")" > "$SYM"
|
||||
run_request POST "" "$SYM"
|
||||
if [ "$RUN_STATUS" = "500" ] && body_contains 'plain directory'; then
|
||||
ok "symlinked run directory refused"
|
||||
else
|
||||
bad "symlinked run dir accepted (status=$RUN_STATUS body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
note "request size cap"
|
||||
# The real client joins submitted lines on one body line with '&' separators.
|
||||
BIG="$TMP/bigbody"
|
||||
{
|
||||
printf 'fn=%s&rngType=pcg32&line0=z' "$(urlencode "$FN_ROOT/bigrun001/x.inp")"
|
||||
for i in $(seq 1 20001); do printf '&line%d=z' "$i"; done
|
||||
printf '\n'
|
||||
} > "$BIG"
|
||||
run_request POST "" "$BIG"
|
||||
if [ "$RUN_STATUS" = "413" ]; then
|
||||
ok "oversized request -> 413"
|
||||
else
|
||||
bad "oversized request not capped (status=$RUN_STATUS body=$RUN_BODY)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
printf '\n%d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
[ "$FAIL" -eq 0 ]
|
||||
Reference in new issue
Block a user