First cleanly working series

This commit is contained in:
salman committed 2026-08-31 16:18:48 +02:00
1 parent 2c34f86f22
commit f7a45054fd
11 files changed
+692 -704

No files matched your search

+2 -2
View File
@@ -37,7 +37,7 @@ Status legend:
| Run-number expansion (`-`, `:` syntax) | `MSR.pm::ExpandRunNumbers` | `arrayRunNumbers()` | `PARTIAL` | Common ascending single/range/step syntax is supported with stricter validation. Legacy descending `from:to:step` input swaps its endpoints; the current GUI deliberately rejects descending ranges instead. |
| Single-run/asymmetry MSR creation | `MSR.pm::CreateMSRUni` | `createMSR()` / `buildMSR()` | `PARTIAL` | Core FITPARAMETER, THEORY/FUNCTIONS, GLOBAL, RUN, COMMANDS, PLOT, and FOURIER blocks are generated. Multiple fit ranges, explicit data-good-bin ranges, and legacy title handling remain gaps. |
| Multiple fit ranges / packings | `MSR.pm::CreateMSRUni` (`Tis`, `Tfs`, `BINS` arrays) | Single Fit range row | `MISSING` | Legacy CGI accepted comma-separated range/packing sets and repeated RUN blocks accordingly. Current generation expects one fit interval and one packing value. |
| Global MSR generation | `MSR.pm::CreateMSRGLB`, `msr2data global` | `prepareMSRRunFiles()` + `buildSeriesGlobalMSR()` | `DONE` | Multi-run asymmetry merges the already-fitted per-run MSRs into the combined display MSR in JS (`muSRFitlib.js`) and Perl (`single_musrfit.cgi`). msr2data is no longer asked to build it, because it only writes its `+global` file from a template and re-fits each run in every value-populating mode. Stale output is removed and the merged MSR is validated before use, instead of relying on msr2data's unreliable exit codes. |
| Global MSR generation | `MSR.pm::CreateMSRGLB`, `msr2data global` | `prepareMSRRunFiles()` + `buildSeriesGlobalMSR()` | `DONE` | Electron and the browser use the same JavaScript merger for already-fitted independent runs. In web mode the browser fetches the HTTP-visible fitted MSRs and submits the combined display text; the CGI verifies its artifact names, size, structure, and exact RUN/ADDRUN identity before publishing it. `msr2data` collects the per-run parameter table directly and never re-fits the combined display. |
| Parameter table and bounds | `MSR.pm::PrepParamTable` | `cfgParams()`, `formatFitParameterLine()`, fitted-MSR parser | `PARTIAL` | Value/step and two-sided or one-sided bounds are generated correctly. Fitted values and bounds can be imported. The GUI does not separately preserve/display the asymmetric positive MINOS error field from a seven-field fitted parameter line. |
| Parameter-data export | `MSR.pm::ExportParams`, post-fit `msr2data` | Web/Electron runners + floating parameter viewer | `DONE` | Both runners generate `<fit>_par.dat` after single/global fitting and expose the same artifact to the integrated floating Plotly viewer; the raw table remains available in Results. |
@@ -59,7 +59,7 @@ Status legend:
| Plot-only and fit+plot execution | Legacy `plotonly` / `fitplot` commands | `muSRFitelec.js`, `muSRFitweb.js`, `cgi/single_musrfit.cgi` | `DONE` | Electron and web have separate Plot/Fit paths. Multi-run jobs assemble the combined display MSR from the per-run fitted files and normalize PLOT run indices before plotting. Fit and Plot share an action lock with visible preparation/execution phases, preventing duplicate or conflicting requests. Missing RUN data from either musrfit or musrview is reported as an amber `Missing data` outcome rather than a failed fit/plot; Fit stops before starting musrview in that case. |
| Interactive plot delivery | Legacy `musrview --png` + ImageMagick crop | Native musrview (Electron), `musrview --root` + JSROOT (web) | `DONE` | This intentionally replaces static PNG delivery. Web ROOT files support interactive zoom/resizing and FFT toggling; Electron keeps the native musrview window. |
| Spectrum ASCII extraction + gzip | Legacy `musrview --ascii` + `gzip FILENAME.dat` | No equivalent spectrum dump | `MISSING` | Job tarballs and parameter `.dat` export are different artifacts and do not replace the legacy plotted-data ASCII dump. Add only if this download remains useful to users. |
| `msr2data` integration | Legacy `$MSR2DATA` workflow | Local runner + `cgi/single_musrfit.cgi` | `DONE` | msr2data handles the post-fit parameter `.dat` export in both runners. The combined multi-run display MSR is assembled locally from the per-run fitted files instead; the merged MSR is validated explicitly before use. |
| `msr2data` integration | Legacy `$MSR2DATA` workflow | Local runner + `cgi/single_musrfit.cgi` | `DONE` | `msr2data` directly collects the fitted per-run parameter `.dat` table in both runners. Combined display construction is an independent best-effort operation, so an unsupported display model cannot disable Plot parameters. |
| Debug/command output | Legacy downloadable `.out` / debug files | `.out` command log + Results details + concise status bar | `DONE` | Current logs record command, combined output, and exit status while keeping routine live status compact. |
| Web health/runtime configuration | No direct legacy equivalent | CGI `health` action + `muSRFitweb.js` | `DONE` | Reports runtime/executable/tmp/data-root readiness and refreshes the server-owned data root before actions. |
| Dedicated fit statistic display | `musrfit.cgi::PrintOut` / fitted `STATISTIC` block | `parseMSRFitStatistic()` + status bar | `DONE` | After Fit, Electron and web parse the fitted MSR's overall `STATISTIC` line and keep chisq/NDF or maxLH/NDF visible in the status bar; `*** FIT DID NOT CONVERGE ***` is surfaced explicitly. Plot-only actions do not reinterpret an older statistic as a new result. |
+91 -514
View File
@@ -194,7 +194,7 @@ sub rollback_series_receipts {
return \@errors;
}
sub series_run_directive_signature {
sub series_run_directives {
my ($content) = @_;
my @directives = ();
foreach my $line (split(/\r?\n/, $content || "")) {
@@ -209,7 +209,12 @@ sub series_run_directive_signature {
$parts[0] = uc($parts[0]);
push(@directives, join(" ", @parts[0 .. 4]));
}
return sha256_hex(join("\n", @directives));
return \@directives;
}
sub series_run_directive_signature {
my ($content) = @_;
return sha256_hex(join("\n", @{series_run_directives($content)}));
}
sub perform_series_transaction {
@@ -855,19 +860,25 @@ sub perform_series_transaction_finalization {
};
}
# msr2data represents run-qualified parameters such as Asy_2274 as Asy_ in
# the exported data header. Remove exactly one trailing underscore from header
# fields while preserving embedded underscores and labels such as Asy_PosErr.
# Global msr2data exports represent a run-qualified parameter such as Asy_2274
# as Asy_, while ordinary per-run exports retain the first run suffix as
# Asy_2274, Asy_2274PosErr, and Asy_2274NegErr. Normalize only suffixes that
# match this collection's run list while preserving embedded underscores.
sub clean_parameter_data_header {
my ($path) = @_;
my ($path, $run_numbers) = @_;
my $content = read_text_file($path);
return 0 if $content eq "";
my %runs = map { (0 + $_) => 1 }
grep { defined($_) && /^\d+$/ } @{ref($run_numbers) eq "ARRAY" ? $run_numbers : []};
my @lines = split(/\n/, $content, -1);
foreach my $line (@lines) {
my $probe = $line;
$probe =~ s/\r$//;
next if $probe =~ /^\s*$/ || $probe =~ /^\s*#/;
$line =~ s{(\S+?)_([0-9]+)((?:_)?(?:PosErr|NegErr))?(?=\s|$)}{
exists($runs{0 + $2}) ? $1 . (defined($3) ? $3 : "") : $&
}gex;
$line =~ s/(\S+)_(?=\s|$)/$1/g;
last;
}
@@ -995,463 +1006,7 @@ sub normalize_plot_runs_line {
return $content;
}
# Merge the fitted per-run MSR files of an independent series fit into a single
# display-only global MSR. msr2data is deliberately not used for this: it only
# writes its +global file from a template and, in the modes that populate
# per-run values, re-fits each run with musrfit. The merged file keeps each
# run's already-fitted parameter values and lets every RUN block select its own
# parameters through the same mapN/alpha/norm mechanism a genuine msr2data
# global file uses.
my @SERIES_MSR_BLOCK_NAMES = qw(FITPARAMETER THEORY FUNCTIONS GLOBAL RUN COMMANDS PLOT FOURIER STATISTIC);
my %SERIES_MSR_RUN_PARAM_DIRECTIVES = map { $_ => 1 } qw(alpha beta norm backgr.fit);
my $SERIES_MSR_MAX_MAP_SLOTS = 10;
# Recognize a block header line. The FOURIER block uses a lowercase
# `plot <type>` directive that would otherwise look like a PLOT header, so a
# genuine PLOT header must carry a numeric plot id ("PLOT 2"); non-numeric
# `plot` lines stay inside FOURIER.
sub series_msr_block_header_name {
my ($trimmed) = @_;
for my $name (@SERIES_MSR_BLOCK_NAMES) {
if ($trimmed =~ /^\Q$name\E(?:\s|$)/i) {
my $candidate = uc($name);
if ($candidate eq "PLOT") {
my $arg = $trimmed;
$arg =~ s/^\Q$name\E//i;
$arg =~ s/^\s+|\s+$//g;
return "" if $arg !~ /^\d/;
}
return $candidate;
}
}
return "";
}
sub series_msr_blocks {
my ($content) = @_;
my %blocks;
my $section = "";
for my $line (split(/\r?\n/, defined($content) ? $content : "")) {
my $trimmed = $line;
$trimmed =~ s/^\s+//;
my $block = series_msr_block_header_name($trimmed);
if ($block ne "") {
my $start = $line;
$start =~ s/^\s+//;
$start =~ s/\s+$//;
$section = $block;
if (!exists($blocks{$block})) {
$blocks{$block} = { header => $start, lines => [] };
} elsif ($block eq "RUN") {
# RUN is repeatable. Preserve later detector headers inside the
# first collected RUN section so series_msr_run_blocks can
# reconstruct every histogram block.
push(@{$blocks{$block}{lines}}, $start);
}
} elsif ($section ne "") {
push(@{$blocks{$section}{lines}}, $line);
} else {
$blocks{HEADER} = { header => "", lines => [] }
if !exists($blocks{HEADER});
push(@{$blocks{HEADER}{lines}}, $line);
}
}
return \%blocks;
}
sub series_msr_parameters {
my ($lines) = @_;
my @params;
for my $line (@{$lines || []}) {
my $trimmed = $line;
$trimmed =~ s/^\s+//;
next if $trimmed eq "" || $trimmed =~ /^#/;
my ($no, $name, $rest) = $trimmed =~ /^(\d+)\s+(\S+)\s*(.*)$/;
next if !defined($no);
$rest = "" if !defined($rest);
$rest =~ s/^\s+|\s+$//g;
$rest =~ s/\s+/ /g;
push(@params, { no => 0 + $no, name => $name, rest => $rest });
}
return \@params;
}
sub series_msr_run_blocks {
my ($header, $lines) = @_;
my @blocks;
my $current;
for my $line (($header, @{$lines || []})) {
my $trimmed = $line;
$trimmed =~ s/^\s+//;
if ($trimmed =~ /^RUN\b/i) {
my $start = $line;
$start =~ s/\s+$//;
$current = { header => $start, lines => [] };
push(@blocks, $current);
} elsif (defined($current)) {
push(@{$current->{lines}}, $line);
}
}
return \@blocks;
}
sub series_msr_token_shape {
my ($lines) = @_;
return join("\n", map {
join(" ", map { /^\d+$/ ? "#" : $_ } split(/\s+/, $_))
} @{$lines || []});
}
sub format_series_merged_parameter_line {
my ($no, $name, $rest) = @_;
my $line = " $no $name";
$line .= " $rest" if defined($rest) && $rest ne "";
return $line;
}
# The FOURIER block is shared by all RUN blocks in the merged file. musrview
# refuses any plot other than POWER unless the phase line carries one reference
# per run (parR7 par9 ...), which a display file assembled from shared
# parameters cannot express. Mirror msr2data's own global output by forcing the
# plot to POWER; the fitted per-run Fourier phases are deliberately left to the
# individual run files.
sub series_fourier_for_display {
my ($lines) = @_;
my @normalized;
my $saw_plot = 0;
for my $line (@{$lines || []}) {
my $trimmed = $line;
$trimmed =~ s/^\s+//;
if ($trimmed =~ /^plot\b/i) {
$saw_plot = 1;
if ($trimmed !~ /^plot\s+POWER\b/i) {
push(@normalized, "plot POWER");
} else {
push(@normalized, $line);
}
} else {
push(@normalized, $line);
}
}
push(@normalized, "plot POWER") if !$saw_plot;
return \@normalized;
}
# Returns [$ok, $reason, $text].
sub build_series_global_msr {
my ($msr_texts, $run_numbers) = @_;
return [0, "series-merge-invalid-input", ""]
if ref($msr_texts) ne "ARRAY" || ref($run_numbers) ne "ARRAY"
|| @{$msr_texts} < 2 || @{$msr_texts} != @{$run_numbers};
my @per_run;
for my $index (0 .. $#$msr_texts) {
my $blocks = series_msr_blocks($msr_texts->[$index]);
return [0, "series-merge-source-incomplete", ""]
if !$blocks->{FITPARAMETER} || !$blocks->{THEORY} || !$blocks->{RUN};
my $params = series_msr_parameters($blocks->{FITPARAMETER}{lines});
return [0, "series-merge-parameters-incomplete", ""] if !@{$params};
my $run_blocks = series_msr_run_blocks(
$blocks->{RUN}{header}, $blocks->{RUN}{lines}
);
return [0, "series-merge-run-blocks-incomplete", ""] if !@{$run_blocks};
push(@per_run, {
run => "$run_numbers->[$index]",
params => $params,
run_blocks => $run_blocks,
theory => $blocks->{THEORY}{lines} || [],
functions => $blocks->{FUNCTIONS}{lines} || [],
global => $blocks->{GLOBAL}{lines} || [],
commands => $blocks->{COMMANDS}{lines} || [],
plot => $blocks->{PLOT}{lines} || [],
plot_header => $blocks->{PLOT}{header} || "PLOT 0",
fourier => $blocks->{FOURIER}{lines} || [],
statistic => $blocks->{STATISTIC}{lines} || []
});
}
my $num_runs = scalar(@per_run);
my $first = $per_run[0];
for my $pr (@per_run) {
return [0, "series-merge-parameter-count-mismatch", ""]
if @{$pr->{params}} != @{$first->{params}};
}
# A name present in every run is shared; present in one run is run-specific.
my %name_counts;
for my $pr (@per_run) {
$name_counts{$_->{name}}++ for @{$pr->{params}};
}
for my $name (keys %name_counts) {
return [0, "series-merge-parameter-inconsistency", ""]
if $name_counts{$name} != 1 && $name_counts{$name} != $num_runs;
}
# All runs must describe the same model: the non-numeric token shape of the
# THEORY and FUNCTIONS blocks is identical, only parameter numbers differ.
for my $pr (@per_run) {
return [0, "series-merge-theory-mismatch", ""]
if series_msr_token_shape($pr->{theory}) ne series_msr_token_shape($first->{theory});
return [0, "series-merge-functions-mismatch", ""]
if series_msr_token_shape($pr->{functions}) ne series_msr_token_shape($first->{functions});
}
# Assign merged numbers: shared parameters first, run-specific grouped per run.
my @merged_params;
my %shared_name_no;
for my $p (@{$first->{params}}) {
if ($name_counts{$p->{name}} == $num_runs
&& !exists($shared_name_no{$p->{name}})) {
$shared_name_no{$p->{name}} = scalar(@merged_params) + 1;
push(@merged_params, { name => $p->{name}, rest => $p->{rest} });
}
}
my @no_map;
for my $run_index (0 .. $num_runs - 1) {
$no_map[$run_index] = {};
for my $p (@{$per_run[$run_index]{params}}) {
if ($name_counts{$p->{name}} == $num_runs) {
$no_map[$run_index]{$p->{no}} = $shared_name_no{$p->{name}};
} else {
$no_map[$run_index]{$p->{no}} = scalar(@merged_params) + 1;
push(@merged_params, { name => $p->{name}, rest => $p->{rest} });
}
}
}
my %by_number;
for my $pr (@per_run) {
my %map;
$map{$_->{no}} = $_ for @{$pr->{params}};
$by_number{$pr} = \%map;
}
# Rewrite THEORY: existing mapN/funN stay, direct integers are renumbered,
# run-specific roles become map slots filled per RUN block below.
my %occupied;
for my $lines ($first->{theory}, $first->{functions}) {
for my $line (@{$lines || []}) {
for my $token (split(/\s+/, $line)) {
$occupied{$1}++ if $token =~ /^map(\d+)$/i;
}
}
}
my $next_slot = 1;
$next_slot++ while $occupied{$next_slot};
my (%slot_role_name, %slot_position);
my $slot_for = sub {
my ($name, $line_index, $token_index) = @_;
return $slot_role_name{$name} if exists($slot_role_name{$name});
return 0 if $next_slot > $SERIES_MSR_MAX_MAP_SLOTS;
my $slot = $next_slot++;
$slot_role_name{$name} = $slot;
$slot_position{$slot} = [ $line_index, $token_index ];
return $slot;
};
my @merged_theory;
for my $line_index (0 .. $#{$first->{theory}}) {
my @out;
my $token_index = 0;
for my $token (split(/\s+/, $first->{theory}[$line_index])) {
if ($token =~ /^\d+$/ && $token ne "0") {
my $local_no = 0 + $token;
my $p = $by_number{$first}{$local_no};
return [0, "series-merge-theory-param-missing", ""] if !$p;
if ($name_counts{$p->{name}} == $num_runs) {
push(@out, $shared_name_no{$p->{name}});
} else {
my $slot = $slot_for->($p->{name}, $line_index, $token_index);
return [0, "series-merge-map-slots-exhausted", ""] if $slot == 0;
push(@out, "map$slot");
}
} else {
push(@out, $token);
}
$token_index++;
}
push(@merged_theory, join(" ", @out));
}
# FUNCTIONS is shared, so it may only reference parameters that keep their
# number in every RUN block.
my @merged_functions;
for my $line (@{$first->{functions}}) {
my @out;
for my $token (split(/\s+/, $line)) {
if ($token =~ /^par(\d+)$/i) {
my $local_no = 0 + $1;
my $p = $by_number{$first}{$local_no};
return [0, "series-merge-functions-param-missing", ""] if !$p;
return [0, "series-merge-functions-run-specific", ""]
if $name_counts{$p->{name}} != $num_runs;
push(@out, "par" . $shared_name_no{$p->{name}});
} else {
push(@out, $token);
}
}
push(@merged_functions, join(" ", @out));
}
my $max_slot = 0;
for my $slot (keys %slot_position) { $max_slot = $slot if $slot > $max_slot; }
my $merged_map_entries = sub {
my ($run_index, $block) = @_;
my @existing_values;
for my $line (@{$block->{lines}}) {
if ($line =~ /^\s*map\b/i) {
my @tokens = split(/\s+/, $line);
@existing_values = @tokens > 1 ? @tokens[1 .. $#tokens] : ();
last;
}
}
my %entries;
my $slot_max = $max_slot;
$slot_max = scalar(@existing_values) if @existing_values > $slot_max;
for my $slot (1 .. $slot_max) {
my $value = 0;
if ($slot <= @existing_values && $existing_values[$slot - 1] =~ /^\d+$/) {
my $local_no = 0 + $existing_values[$slot - 1];
$value = ($local_no > 0 && defined($no_map[$run_index]{$local_no}))
? $no_map[$run_index]{$local_no} : $local_no;
} elsif (exists($slot_position{$slot})) {
my ($line_index, $token_index) = @{$slot_position{$slot}};
my @local_tokens = split(/\s+/, $per_run[$run_index]{theory}[$line_index] || "");
my $local_no = $local_tokens[$token_index] =~ /^\d+$/
? 0 + $local_tokens[$token_index] : 0;
$value = $no_map[$run_index]{$local_no}
if $local_no > 0 && defined($no_map[$run_index]{$local_no});
}
$entries{$slot} = $value;
}
my @values;
for my $slot (1 .. $slot_max) { push(@values, $entries{$slot}); }
pop @values while @values && $values[$#values] == 0;
return \@values;
};
my @output;
push(@output, "# Independent series display MSR for the runs: " . join(" ", @{$run_numbers}));
push(@output, "# Fitted per-run parameters collected without re-fitting; "
. "msr2data handles parameter extraction separately.");
push(@output, "###############################################################");
push(@output, "FITPARAMETER");
push(@output, "###############################################################");
push(@output, "# Common parameters for all runs");
for my $index (0 .. $#merged_params) {
next if $name_counts{$merged_params[$index]{name}} != $num_runs;
push(@output, format_series_merged_parameter_line(
$index + 1, $merged_params[$index]{name}, $merged_params[$index]{rest}
));
}
for my $run_index (0 .. $num_runs - 1) {
push(@output, "# Specific parameters for run " . $per_run[$run_index]{run});
for my $p (@{$per_run[$run_index]{params}}) {
next if $name_counts{$p->{name}} == $num_runs;
push(@output, format_series_merged_parameter_line(
$no_map[$run_index]{$p->{no}}, $p->{name}, $p->{rest}
));
}
}
push(@output, "###############################################################");
push(@output, "THEORY");
push(@output, @merged_theory);
if (@{$first->{functions}}) {
push(@output, "###############################################################");
push(@output, "FUNCTIONS");
push(@output, @merged_functions);
}
push(@output, "###############################################################");
push(@output, "GLOBAL");
push(@output, @{$first->{global}});
for my $run_index (0 .. $num_runs - 1) {
push(@output, "###############################################################");
push(@output, "# RUN blocks for run " . $per_run[$run_index]{run});
for my $block (@{$per_run[$run_index]{run_blocks}}) {
push(@output, $block->{header});
my $had_map = 0;
for my $line (@{$block->{lines}}) {
my $trimmed = $line;
$trimmed =~ s/^\s+//;
next if $trimmed eq "" || $trimmed =~ /^#/;
my ($directive) = $trimmed =~ /^(\S+)/;
if ($directive =~ /^map$/i) { $had_map = 1; next; }
if ($SERIES_MSR_RUN_PARAM_DIRECTIVES{lc($directive)}) {
my $rest = $trimmed;
$rest =~ s/^\s*\Q$directive\E\s*//;
my @renumbered;
for my $token (split(/\s+/, $rest)) {
if ($token =~ /^\d+$/) {
my $n = 0 + $token;
push(@renumbered, defined($no_map[$run_index]{$n})
? $no_map[$run_index]{$n} : $token);
} else {
push(@renumbered, $token);
}
}
push(@output, $directive . " " . join(" ", @renumbered));
} else {
push(@output, $line);
}
}
if ($had_map || $max_slot > 0) {
my $values = $merged_map_entries->($run_index, $block);
push(@output, "map " . join(" ", @{$values})) if @{$values};
}
}
}
push(@output, "###############################################################");
push(@output, "COMMANDS");
push(@output, @{$first->{commands}});
if (@{$first->{fourier}}) {
push(@output, "###############################################################");
push(@output, "FOURIER");
push(@output, @{series_fourier_for_display($first->{fourier})});
}
push(@output, "###############################################################");
push(@output, $first->{plot_header});
for my $line (@{$first->{plot}}) {
next if $line =~ /^\s*runs\s+/i;
push(@output, $line);
}
my @plot_runs = (1 .. $num_runs);
push(@output, "runs " . join(" ", @plot_runs));
# musrfit always stamps a STATISTIC block into a fitted file, and msr2data's
# global data export keys each run's convergence on that block. The merged
# display MSR is not re-fitted, so carry the per-run statistics forward as
# the ordinary aggregate musrfit would write for a global fit: summed chisq
# and NDF, reduced chisq from those sums.
my ($chisq_sum, $ndf_sum) = (0, 0);
my $statistic_runs = 0;
for my $pr (@per_run) {
for my $line (@{$pr->{statistic}}) {
if ($line =~ /chisq\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i) {
my ($run_chisq, $run_ndf) = (0 + $1, 0 + $2);
next if $run_ndf <= 0;
$chisq_sum += $run_chisq;
$ndf_sum += $run_ndf;
$statistic_runs++;
}
}
}
if ($statistic_runs) {
my ($sec, $min, $hour, $mday, $mon, $year) = localtime();
my $timestamp = sprintf("%04d-%02d-%02d %02d:%02d:%02d",
$year + 1900, $mon + 1, $mday, $hour, $min, $sec);
my $reduced = $ndf_sum > 0 ? $chisq_sum / $ndf_sum : 0;
push(@output, "###############################################################");
push(@output, "STATISTIC --- " . $timestamp);
my $formatted_chisq = sprintf("%.4f", $chisq_sum);
my $formatted_reduced = sprintf("%.6f", $reduced);
$formatted_chisq =~ s/\.?0+$//;
$formatted_reduced =~ s/\.?0+$//;
push(@output, sprintf(" chisq = %s, NDF = %d, chisq/NDF = %s",
$formatted_chisq, $ndf_sum, $formatted_reduced));
}
return [1, "series-merge-ready", join("\n", @output) . "\n"];
}
# Display-MSR parsing and merging live in muSRFitlib.js and run in the browser.
# Collect a completed independent series without running musrfit again. The
# requested non-empty data table and structurally valid combined MSR are the
@@ -1473,16 +1028,21 @@ sub perform_series_artifact_collection {
};
};
return $failure->("invalid-series-collection", "Invalid series collection request")
if !defined($collection_text) || length($collection_text) > 100_000;
if !defined($collection_text) || length($collection_text) > 5_000_000;
my $collection = eval { decode_json($collection_text) };
return $failure->("invalid-series-collection", "Invalid series collection JSON")
if $@ || ref($collection) ne "HASH";
my $runs = $collection->{runs};
my $suffix = defined($collection->{suffix}) ? "$collection->{suffix}" : "";
my $fftv = defined($collection->{fftv}) ? "$collection->{fftv}" : "n";
my $contract_version = defined($collection->{version})
&& !ref($collection->{version}) ? 0 + $collection->{version} : 0;
my $client_combined = exists($collection->{combinedMSR})
&& !ref($collection->{combinedMSR}) ? "$collection->{combinedMSR}" : undef;
return $failure->("invalid-series-collection", "Invalid series collection fields")
if ref($runs) ne "ARRAY" || @{$runs} < 2 || @{$runs} > 500
|| $suffix !~ /^_[A-Za-z0-9._+\-]+$/ || $fftv !~ /^(?:y|n)$/;
|| $suffix !~ /^_[A-Za-z0-9._+\-]+$/ || $fftv !~ /^(?:y|n)$/
|| $contract_version != 1 || !defined($client_combined);
my @run_numbers = ();
my %seen = ();
@@ -1509,15 +1069,27 @@ sub perform_series_artifact_collection {
# global-fit results that may already exist in the same HTTP job folder.
my $combined_name = $base_run . "+series" . $suffix . ".msr";
my $combined_path = "$work_path/$combined_name";
# msr2data still needs a base+global<suffix>.msr staging input. A unique
# internal suffix prevents collision with the normal `_tmpl` global path.
my $template_suffix = "_musrflow_series_" . $$ . "_" . time();
my $template_name = $base_run . "+global" . $template_suffix . ".msr";
my $template_path = "$work_path/$template_name";
my $parameter_name = $base_run . "+series" . $suffix . "_par.dat";
my $parameter_path = "$work_path/$parameter_name";
my $out_name = $base_run . "+series" . $suffix . ".out";
my $out_path = "$work_path/$out_name";
my $artifacts = $collection->{artifacts};
return $failure->("invalid-series-collection", "Invalid series collection contract")
if ref($artifacts) ne "HASH"
|| ($artifacts->{combined} || "") ne $combined_name
|| ($artifacts->{parameters} || "") ne $parameter_name
|| ($artifacts->{log} || "") ne $out_name
|| length($client_combined) > 4_000_000;
# Collection mutates the job's generated artifacts and starts external
# programs. Serialize it with fitting/promotion operations for this job.
my $lock_path = "$work_path/.musrflow-series.lock";
return $failure->("series-collection-lock-failed", "Could not open series job lock")
if !sysopen(my $lock_fh, $lock_path, O_RDWR | O_CREAT, 0600);
if (!flock($lock_fh, LOCK_EX | LOCK_NB)) {
close($lock_fh);
return $failure->("series-collection-lock-failed", "Another series operation is active");
}
foreach my $stale_path ($combined_path, $parameter_path) {
return $failure->("series-artifact-output-invalid", "A series output path is unsafe")
if -l $stale_path || (-e $stale_path && !-f $stale_path);
@@ -1534,55 +1106,59 @@ sub perform_series_artifact_collection {
my $run_argument = "[" . join(" ", @run_numbers) . "]";
my @source_texts = map { $source_texts{$_} } @run_numbers;
# Build the combined display MSR first. It becomes the canonical
# `+global<suffix>.msr` file that both musrview and the parameter export
# use, and is assembled directly from the already-fitted per-run files so
# no global re-fit ever takes place.
my $combined_url = "";
my $merged = build_series_global_msr(\@source_texts, \@run_numbers);
if (!$merged->[0]) {
push(@warnings, "Could not assemble the combined display MSR ($merged->[1])");
} elsif (!is_valid_generated_global_msr($merged->[2])) {
push(@warnings, "Assembled combined display MSR failed validation");
} elsif (write_text_file($combined_path, normalize_plot_runs_line(
$merged->[2], join(" ", @run_numbers)
))) {
$combined_url = "$TMP_URL/$job_name/$combined_name";
# Collect fitted per-run values directly. This is independent of display
# merging, so models that cannot yet be represented in one display MSR
# still produce the table consumed by Plot parameters.
my $parameter_url = "";
my $parameter_cmd = "cd " . shell_quote($work_path)
. " && " . shell_quote($MSR2DATA_BIN)
. " " . shell_quote($run_argument)
. " " . shell_quote($suffix)
. " new data -o " . shell_quote($parameter_name);
my $parameter_result = run_logged_command(
$out_path, "msr2data independent series collection", $parameter_cmd
);
if (-s $parameter_path) {
if (!clean_parameter_data_header($parameter_path, \@run_numbers)) {
push(@warnings, "Could not normalize the parameter-table header");
}
$parameter_url = "$TMP_URL/$job_name/$parameter_name";
if ($parameter_result->{raw_status} != 0) {
push(@warnings, "msr2data returned code " . $parameter_result->{exit_code}
. " but produced the parameter table");
}
} else {
push(@warnings, "Could not write the combined display MSR");
push(@warnings, "msr2data did not produce parameter data");
}
# msr2data is used only for the parameter table, exported with the same
# `new global data` form and `_tmpl` staging as the ordinary multirun
# export, producing the global .dat layout (stripped base names, one row
# per run) that the parameter plot button reads.
my $parameter_url = "";
my $combined_text = $combined_url ne "" ? read_text_file($combined_path) : "";
if ($combined_text ne "" && write_text_file($template_path, $combined_text)) {
my $parameter_cmd = "cd " . shell_quote($work_path)
. " && " . shell_quote($MSR2DATA_BIN)
. " " . shell_quote($run_argument)
. " " . shell_quote($template_suffix)
. " new global data -o " . shell_quote($parameter_name);
my $parameter_result = run_logged_command(
$out_path, "msr2data independent series collection", $parameter_cmd
);
if (-s $parameter_path) {
if (!clean_parameter_data_header($parameter_path)) {
push(@warnings, "Could not normalize the parameter-table header");
}
# Keep the non-empty file available even when header cleanup fails.
$parameter_url = "$TMP_URL/$job_name/$parameter_name";
if ($parameter_result->{raw_status} != 0) {
push(@warnings, "msr2data returned code " . $parameter_result->{exit_code}
. " but produced the parameter table");
}
# The browser carries the display MSR produced by the shared JavaScript
# merger. Verify that it retains exactly the fitted sources' RUN/ADDRUN
# identities before publishing or passing it to musrview.
my $combined_url = "";
my $combined_text = $client_combined || "";
if ($combined_text ne "") {
my @expected_directives = map {
@{series_run_directives($_)}
} @source_texts;
my @actual_directives = @{series_run_directives($combined_text)};
if (!is_valid_generated_global_msr($combined_text)) {
push(@warnings, "Submitted combined display MSR failed validation");
$combined_text = "";
} elsif (join("\n", @actual_directives) ne join("\n", @expected_directives)) {
push(@warnings, "Submitted combined display MSR changed RUN identity");
$combined_text = "";
}
}
if ($combined_text ne "") {
my $combined_stage = "$work_path/.musrflow-series-display-$$.tmp";
if (!-e $combined_stage
&& write_text_file($combined_stage, $combined_text)
&& rename($combined_stage, $combined_path)) {
$combined_url = "$TMP_URL/$job_name/$combined_name";
} else {
push(@warnings, "msr2data did not produce parameter data");
unlink($combined_stage) if -f $combined_stage && !-l $combined_stage;
push(@warnings, "Could not write the combined display MSR");
}
unlink($template_path);
} elsif ($combined_url ne "") {
push(@warnings, "Could not stage the combined MSR for parameter export");
}
my @root_urls = ();
@@ -1627,6 +1203,7 @@ sub perform_series_artifact_collection {
my $command_log = read_text_file($out_path);
my $usable = $parameter_url ne "" || $combined_url ne "";
close($lock_fh);
return {
ok => $usable ? JSON::PP::true : JSON::PP::false,
code => $usable ? 0 : 1,
+66 -89
View File
@@ -723,20 +723,31 @@ function appendFitLog(logFile, text) {
}
}
// msr2data represents a run-qualified parameter such as Asy_2274 as Asy_ in
// the exported data header. Remove exactly one trailing underscore from header
// fields so downstream plotting sees the original logical parameter name.
// Underscores inside names (for example my_parameter) and error labels such as
// Asy_PosErr are left untouched.
function cleanMusrfitParameterDataHeader(parameterPath) {
// Global msr2data exports represent a run-qualified parameter such as
// Asy_2274 as Asy_, while ordinary per-run exports retain the first run suffix
// as Asy_2274, Asy_2274PosErr, and Asy_2274NegErr. Normalize only suffixes that
// match this collection's run list so downstream plotting sees the logical
// parameter name. Embedded underscores in names remain untouched.
function cleanMusrfitParameterDataHeader(parameterPath, runNumbers = []) {
let contents = nodeFs.readFileSync(parameterPath, "utf8");
let newline = contents.includes("\r\n") ? "\r\n" : "\n";
let lines = contents.split(/\r?\n/);
let runs = new Set(runNumbers.map(function(run) {
return String(Number(run));
}).filter(function(run) {
return /^\d+$/.test(run);
}));
for (let i = 0; i < lines.length; i++) {
let probe = lines[i].trim();
if (!probe || probe.startsWith("#")) continue;
lines[i] = lines[i].replace(/(\S+)_(?=\s|$)/g, "$1");
lines[i] = lines[i].replace(/\S+/g, function(column) {
let qualified = /^(.*)_([0-9]+)((?:_)?(?:PosErr|NegErr))?$/.exec(column);
if (qualified && runs.has(String(Number(qualified[2])))) {
return qualified[1] + (qualified[3] || "");
}
return column.endsWith("_") ? column.slice(0, -1) : column;
});
break;
}
@@ -975,30 +986,17 @@ window.runSeriesMSRCandidateLocal = runSeriesMSRCandidateLocal;
// +global file from a template). A non-empty requested .dat is registered
// immediately, and any structurally valid combined MSR is safe for musrview.
async function collectIndependentSeriesArtifactsLocal(spec) {
let result = {
ok: false,
reason: "invalid-series-artifact-spec",
parameterDataPath: "",
combinedMSRPath: "",
logPath: "",
plotStarted: false,
warnings: []
};
if (!spec || spec.ok !== true || !Array.isArray(spec.runs)
|| spec.runs.length < 2 || !Array.isArray(spec.msrPaths)
|| spec.msrPaths.length !== spec.runs.length
|| !/^_[A-Za-z0-9._+\-]+$/.test(spec.fileNameSuffix || "")) {
return result;
}
let prepared = prepareIndependentSeriesCollection(spec);
let result = normalizeIndependentSeriesCollectionResult({ reason: prepared.reason });
if (!prepared.ok) return result;
let workPath = nodePath.resolve(spec.workPath);
let suffix = spec.fileNameSuffix;
let baseRun = spec.runs[0];
for (let index = 0; index < spec.runs.length; index++) {
if (!/^\d+$/.test(spec.runs[index])
|| nodePath.resolve(spec.msrPaths[index])
!== nodePath.join(workPath, spec.runs[index] + suffix + ".msr")
|| !isSafeSeriesRegularFile(spec.msrPaths[index])) {
let workPath = nodePath.resolve(prepared.workPath);
let suffix = prepared.fileNameSuffix;
let baseRun = prepared.baseRun;
for (let index = 0; index < prepared.runs.length; index++) {
if (nodePath.resolve(prepared.msrPaths[index])
!== nodePath.join(workPath, prepared.runs[index] + suffix + ".msr")
|| !isSafeSeriesRegularFile(prepared.msrPaths[index])) {
result.reason = "series-artifact-source-invalid";
return result;
}
@@ -1007,18 +1005,11 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
// Independent-series artifacts use their own namespace. A genuine global
// fit can legitimately exist beside the same per-run files and must never
// be removed or replaced by display-only post-processing.
let combinedName = baseRun + "+series" + suffix + ".msr";
let combinedName = prepared.combinedName;
let combinedPath = nodePath.join(workPath, combinedName);
// msr2data global-data mode requires a `base+global<suffix>.msr` input.
// Give the staging file a process-unique suffix so an ordinary `_tmpl`
// global-fit template is neither overwritten nor deleted.
let templateSuffix = "_musrflow_series_" + process.pid + "_"
+ Date.now().toString(36);
let templateName = baseRun + "+global" + templateSuffix + ".msr";
let templatePath = nodePath.join(workPath, templateName);
let parameterName = baseRun + "+series" + suffix + "_par.dat";
let parameterName = prepared.parameterName;
let parameterPath = nodePath.join(workPath, parameterName);
let logName = baseRun + "+series" + suffix + ".out";
let logName = prepared.logName;
let logPath = nodePath.join(workPath, logName);
result.logPath = logPath;
@@ -1035,7 +1026,7 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
"# muSRFlow independent series collection\n# Working directory: "
+ workPath + "\n", { encoding: "utf8", flag: "w" });
let runsArg = "[" + spec.runs.join(" ") + "]";
let runsArg = "[" + prepared.runs.join(" ") + "]";
let msr2data = resolveLocalCommand("msr2data");
let env = Object.assign({}, process.env);
env.PATH = ["/usr/local/bin", "/usr/bin", "/bin", env.PATH || ""].join(":");
@@ -1063,64 +1054,53 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
return code;
};
// Assemble the combined display MSR first. It becomes the canonical
// `+global<suffix>.msr` file that both musrview and the parameter export
// use, and is built directly from the already-fitted per-run files so no
// global re-fit ever takes place.
// Parameter collection is the primary independent-series result. Use
// msr2data's ordinary per-run data mode—the same operation as the
// established rplcall/msr2data workflow—so an unsupported display merge
// can never prevent Plot parameters from receiving a usable table.
let parameterCode = runMsr2data("parameter msr2data",
[runsArg, suffix, "new", "data", "-o", parameterName]);
if (isSafeSeriesRegularFile(parameterPath)
&& nodeFs.statSync(parameterPath).size > 0) {
try {
cleanMusrfitParameterDataHeader(parameterPath, spec.runs);
} catch (error) {
result.warnings.push("Could not normalize the parameter-table header.");
}
result.parameterDataPath = parameterPath;
if (parameterCode !== 0) {
result.warnings.push("msr2data returned code " + parameterCode
+ " but produced the parameter table.");
}
} else {
result.warnings.push("msr2data did not produce parameter data.");
}
// Assemble the combined display MSR independently from the already
// fitted per-run files. No global re-fit takes place.
let msrTexts = [];
for (let index = 0; index < spec.msrPaths.length; index++) {
if (!isSafeSeriesRegularFile(spec.msrPaths[index])) {
result.warnings.push("Fitted MSR " + spec.msrPaths[index]
for (let index = 0; index < prepared.msrPaths.length; index++) {
if (!isSafeSeriesRegularFile(prepared.msrPaths[index])) {
result.warnings.push("Fitted MSR " + prepared.msrPaths[index]
+ " is missing for the combined display.");
break;
}
msrTexts.push(nodeFs.readFileSync(spec.msrPaths[index], "utf8"));
msrTexts.push(nodeFs.readFileSync(prepared.msrPaths[index], "utf8"));
}
if (msrTexts.length === spec.msrPaths.length) {
let merged = buildSeriesGlobalMSR(msrTexts, spec.runs);
if (msrTexts.length === prepared.msrPaths.length) {
let merged = buildSeriesGlobalMSR(msrTexts, prepared.runs);
if (merged.ok !== true) {
result.warnings.push("Could not assemble the combined display MSR ("
+ merged.reason + ").");
} else if (!isValidGeneratedGlobalMSR(merged.text)) {
result.warnings.push("Assembled combined display MSR failed validation.");
} else {
let combinedText = normalizeMSRPlotRuns(merged.text, {}, spec.runs);
let combinedText = normalizeMSRPlotRuns(merged.text, {}, prepared.runs);
nodeFs.writeFileSync(combinedPath, combinedText, {
encoding: "utf8", flag: "wx"
});
result.combinedMSRPath = combinedPath;
// msr2data is used only for the parameter table, exported with the
// same `new global data` form and `_tmpl` staging as the ordinary
// multirun export, producing the global .dat layout (stripped base
// names, one row per run) that the parameter plot button reads.
nodeFs.copyFileSync(combinedPath, templatePath, nodeFs.constants.COPYFILE_EXCL);
try {
let parameterCode = runMsr2data("parameter msr2data",
[runsArg, templateSuffix, "new", "global", "data", "-o", parameterName]);
if (isSafeSeriesRegularFile(parameterPath)
&& nodeFs.statSync(parameterPath).size > 0) {
try {
cleanMusrfitParameterDataHeader(parameterPath);
} catch (error) {
result.warnings.push("Could not normalize the parameter-table header.");
}
// A non-empty table is usable even if header cleanup or the
// external program's status is imperfect.
result.parameterDataPath = parameterPath;
if (parameterCode !== 0) {
result.warnings.push("msr2data returned code " + parameterCode
+ " but produced the parameter table.");
}
} else {
result.warnings.push("msr2data did not produce parameter data.");
}
} finally {
if (isSafeSeriesRegularFile(templatePath)) {
nodeFs.unlinkSync(templatePath);
}
}
}
} else {
result.warnings.push("Could not read all fitted per-run MSR files"
@@ -1129,7 +1109,7 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
if (result.combinedMSRPath) {
let musrviewArgs = [];
if (spec.fftv == "y") musrviewArgs.push("-f");
if (prepared.fftv == "y") musrviewArgs.push("-f");
musrviewArgs.push(combinedName);
let viewId = window.MusrfitRunner.nextViewId++;
runLocalProcess("musrview series #" + viewId,
@@ -1137,15 +1117,12 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
{ reportStatus: false });
result.plotStarted = true;
}
result.ok = Boolean(result.parameterDataPath || result.combinedMSRPath);
result.reason = result.ok
? "series-artifacts-collected" : "series-artifacts-not-produced";
return result;
return normalizeIndependentSeriesCollectionResult(result);
} catch (error) {
result.reason = "series-artifact-collection-failed";
result.warnings.push(error && error.message
? error.message : "Series post-processing failed.");
return result;
return normalizeIndependentSeriesCollectionResult(result);
}
}
+153 -34
View File
@@ -4608,14 +4608,18 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
let nextSlot = 1;
while (occupiedSlots.has(nextSlot)) nextSlot++;
let slotRoleName = Object.create(null);
let slotPosition = Object.create(null); // slot -> { lineIndex, tokenIndex }
let slotPosition = Object.create(null); // slot -> { section, lineIndex, tokenIndex }
function slotForParam(name, lineIndex, tokenIndex) {
function slotForParam(name, section, lineIndex, tokenIndex) {
if (slotRoleName[name] !== undefined) return slotRoleName[name];
if (nextSlot > SERIES_MSR_MAX_MAP_SLOTS) return 0;
let slot = nextSlot++;
slotRoleName[name] = slot;
slotPosition[slot] = { lineIndex: lineIndex, tokenIndex: tokenIndex };
slotPosition[slot] = {
section: section,
lineIndex: lineIndex,
tokenIndex: tokenIndex
};
return slot;
}
@@ -4631,7 +4635,7 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
if (nameCounts[p.name] === numRuns) {
out.push(String(sharedNameNo[p.name]));
} else {
let slot = slotForParam(p.name, lineIndex, tokenIndex);
let slot = slotForParam(p.name, "theory", lineIndex, tokenIndex);
if (slot === 0) return fail("series-merge-map-slots-exhausted");
out.push("map" + slot);
}
@@ -4646,25 +4650,29 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
if (line && typeof line == "object" && line.ok === false) return fail(line.reason);
}
// FUNCTIONS is shared by all runs, so it may only reference parameters that
// keep the same number across every RUN block. a function that multiplies
// two run-specific parameters would otherwise force a different curve per
// run block, which the global file cannot express.
let mergedFunctions = first.functions.map(function(line) {
// FUNCTIONS can select run-specific parameters through mapN just like
// THEORY. Allocate a RUN map slot for each such parN reference; genuinely
// shared parameters keep a direct parN reference.
let mergedFunctions = first.functions.map(function(line, lineIndex) {
let out = [];
let tokenIndex = 0;
for (let token of line.split(/\s+/)) {
let parMatch = token.match(/^par(\d+)$/i);
if (parMatch) {
let localNo = parseInt(parMatch[1], 10);
let p = byNumber[0][localNo];
if (!p) return fail("series-merge-functions-param-missing");
if (nameCounts[p.name] !== numRuns) {
return fail("series-merge-functions-run-specific");
if (nameCounts[p.name] === numRuns) {
out.push("par" + sharedNameNo[p.name]);
} else {
let slot = slotForParam(p.name, "functions", lineIndex, tokenIndex);
if (slot === 0) return fail("series-merge-map-slots-exhausted");
out.push("map" + slot);
}
out.push("par" + sharedNameNo[p.name]);
} else {
out.push(token);
}
tokenIndex++;
}
return out.join(" ");
});
@@ -4693,17 +4701,24 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
let slotMax = Math.max(maxSlot, existing ? existing.length : 0);
for (let slot = 1; slot <= slotMax; slot++) {
let value = 0;
if (existing && existing[slot - 1] && /^\d+$/.test(existing[slot - 1])) {
if (existing && existing[slot - 1] && /^\d+$/.test(existing[slot - 1])
&& parseInt(existing[slot - 1], 10) > 0) {
let localNo = parseInt(existing[slot - 1], 10);
value = (localNo > 0 && noMap[runIndex][localNo] !== undefined)
? noMap[runIndex][localNo] : localNo;
} else if (slotPosition[slot]) {
let position = slotPosition[slot];
let localTokens = perRun[runIndex].theory[position.lineIndex]
? perRun[runIndex].theory[position.lineIndex].split(/\s+/)
let sourceLines = position.section === "functions"
? perRun[runIndex].functions : perRun[runIndex].theory;
let localTokens = sourceLines[position.lineIndex]
? sourceLines[position.lineIndex].split(/\s+/)
: [];
let localToken = localTokens[position.tokenIndex];
let localNo = /^\d+$/.test(localToken || "") ? parseInt(localToken, 10) : 0;
let functionPar = position.section === "functions"
? String(localToken || "").match(/^par(\d+)$/i) : null;
let localNo = functionPar
? parseInt(functionPar[1], 10)
: (/^\d+$/.test(localToken || "") ? parseInt(localToken, 10) : 0);
if (localNo > 0 && noMap[runIndex][localNo] !== undefined) {
value = noMap[runIndex][localNo];
}
@@ -4802,26 +4817,32 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
for (let i = 1; i <= numRuns; i++) plotRuns.push(String(i));
output.push("runs " + plotRuns.join(" "));
// musrfit always stamps a STATISTIC block into a fitted file, and msr2data's
// global data export keys each run's convergence on that block. The merged
// display MSR is not re-fitted, so carry the per-run statistics forward as
// the ordinary aggregate musrfit would write for a global fit: summed chisq
// and NDF, reduced chisq from those sums.
// The display MSR is not re-fitted. When every source has the same objective,
// report the sum of its independently fitted statistic and NDF. Never emit a
// partial or mixed-objective aggregate.
let statistics = [];
let statisticKind = "";
for (let pr of perRun) {
let match = pr.statistic.match(
/chisq\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i
/(chisq|maxLH)\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i
);
if (!match) continue;
let runChiSq = parseFloat(match[1]);
let runNDF = parseFloat(match[2]);
if (isNaN(runChiSq) || isNaN(runNDF) || runNDF <= 0) continue;
statistics.push({ chiSq: runChiSq, ndf: runNDF });
let kind = match[1].toLowerCase() === "maxlh" ? "maxLH" : "chisq";
let runValue = parseFloat(match[2]);
let runNDF = parseFloat(match[3]);
if (isNaN(runValue) || isNaN(runNDF) || runNDF <= 0) continue;
if (statisticKind && statisticKind !== kind) {
statistics = [];
statisticKind = "";
break;
}
statisticKind = kind;
statistics.push({ value: runValue, ndf: runNDF });
}
if (statistics.length) {
let chiSq = 0;
if (statistics.length === numRuns) {
let total = 0;
let ndf = 0;
for (let s of statistics) { chiSq += s.chiSq; ndf += s.ndf; }
for (let s of statistics) { total += s.value; ndf += s.ndf; }
let date = new Date();
let timestamp = date.getFullYear() + "-"
+ String(date.getMonth() + 1).padStart(2, "0") + "-"
@@ -4829,13 +4850,13 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
+ String(date.getHours()).padStart(2, "0") + ":"
+ String(date.getMinutes()).padStart(2, "0") + ":"
+ String(date.getSeconds()).padStart(2, "0");
let formattedChiSq = parseFloat(chiSq.toFixed(4)).toString();
let formattedTotal = parseFloat(total.toFixed(4)).toString();
let formattedNDF = Math.round(ndf).toString();
let formattedReduced = parseFloat((chiSq / ndf).toFixed(6)).toString();
let formattedReduced = parseFloat((total / ndf).toFixed(6)).toString();
output.push("###############################################################");
output.push("STATISTIC --- " + timestamp);
output.push(" chisq = " + formattedChiSq + ", NDF = " + formattedNDF
+ ", chisq/NDF = " + formattedReduced);
output.push(" " + statisticKind + " = " + formattedTotal + ", NDF = "
+ formattedNDF + ", " + statisticKind + "/NDF = " + formattedReduced);
}
return {
@@ -9469,6 +9490,104 @@ function buildIndependentSeriesArtifactSpec(plan, config = {}) {
return result;
}
// Normalize the platform-neutral part of a completed-series collection. Local
// filesystem confinement and web URL/job checks remain adapter responsibilities,
// but artifact names and the data exchanged with the CGI must be identical.
function prepareIndependentSeriesCollection(spec) {
let invalid = function(reason) {
return {
ok: false,
reason: reason || "invalid-series-artifact-spec",
workPath: "",
fileNameSuffix: "",
runs: [],
msrPaths: [],
fftv: "n",
baseRun: "",
combinedName: "",
parameterName: "",
logName: ""
};
};
if (!spec || spec.ok !== true || !Array.isArray(spec.runs)
|| spec.runs.length < 2 || !Array.isArray(spec.msrPaths)
|| spec.msrPaths.length !== spec.runs.length
|| typeof spec.workPath !== "string" || spec.workPath === ""
|| !/^_[A-Za-z0-9._+\-]+$/.test(spec.fileNameSuffix || "")) {
return invalid("invalid-series-artifact-spec");
}
let runs = spec.runs.map(function(run) { return String(run); });
if (runs.some(function(run) { return !/^\d+$/.test(run) || Number(run) <= 0; })
|| new Set(runs).size !== runs.length
|| spec.msrPaths.some(function(path) {
return typeof path !== "string" || path === "";
})) {
return invalid("series-artifact-source-invalid");
}
let baseRun = runs[0];
let suffix = spec.fileNameSuffix;
return {
ok: true,
reason: "series-collection-request-ready",
workPath: spec.workPath,
fileNameSuffix: suffix,
runs: runs,
msrPaths: spec.msrPaths.slice(),
fftv: spec.fftv == "y" ? "y" : "n",
baseRun: baseRun,
combinedName: baseRun + "+series" + suffix + ".msr",
parameterName: baseRun + "+series" + suffix + "_par.dat",
logName: baseRun + "+series" + suffix + ".out"
};
}
function buildIndependentSeriesCollectionRequest(spec, combinedMSR) {
let prepared = prepareIndependentSeriesCollection(spec);
if (!prepared.ok) return prepared;
return {
ok: true,
version: 1,
runs: prepared.runs.slice(),
suffix: prepared.fileNameSuffix,
fftv: prepared.fftv,
artifacts: {
combined: prepared.combinedName,
parameters: prepared.parameterName,
log: prepared.logName
},
combinedMSR: typeof combinedMSR === "string" ? combinedMSR : ""
};
}
function normalizeIndependentSeriesCollectionResult(value) {
value = value && typeof value === "object" ? value : {};
let result = {
ok: false,
reason: typeof value.reason === "string" && value.reason
? value.reason : "series-artifacts-not-produced",
parameterDataPath: typeof value.parameterDataPath === "string"
? value.parameterDataPath : "",
combinedMSRPath: typeof value.combinedMSRPath === "string"
? value.combinedMSRPath : "",
logPath: typeof value.logPath === "string" ? value.logPath : "",
plotPath: typeof value.plotPath === "string" ? value.plotPath : "",
plotPaths: Array.isArray(value.plotPaths)
? value.plotPaths.filter(function(path) {
return typeof path === "string" && path !== "";
}) : [],
plotStarted: value.plotStarted === true,
warnings: Array.isArray(value.warnings)
? value.warnings.filter(function(warning) {
return typeof warning === "string" && warning !== "";
}) : []
};
result.ok = Boolean(result.parameterDataPath || result.combinedMSRPath);
if (result.ok && (!value.reason || value.reason == "series-artifacts-not-produced")) {
result.reason = "series-artifacts-collected";
}
return result;
}
async function collectIndependentSeriesArtifacts(plan, config, webMode) {
let spec = buildIndependentSeriesArtifactSpec(plan, config);
if (!spec.ok) {
@@ -9503,7 +9622,7 @@ async function collectIndependentSeriesArtifacts(plan, config, webMode) {
warnings: [error && error.message ? error.message : "Series collection failed."]
};
}
artifacts = artifacts && typeof artifacts === "object" ? artifacts : {};
artifacts = normalizeIndependentSeriesCollectionResult(artifacts);
let status = {};
if (artifacts.parameterDataPath) status.parameterDataPath = artifacts.parameterDataPath;
if (artifacts.combinedMSRPath) status.resultMSR = artifacts.combinedMSRPath;
+39 -31
View File
@@ -753,40 +753,48 @@ window.runSeriesMSRCandidateWeb = runSeriesMSRCandidateWeb;
// never invokes musrfit; it only creates the parameter table, a combined display
// MSR, and the ROOT canvas used by the browser viewer.
async function collectIndependentSeriesArtifactsWeb(spec) {
let result = {
ok: false,
reason: "invalid-series-artifact-spec",
parameterDataPath: "",
combinedMSRPath: "",
logPath: "",
plotPath: "",
plotStarted: false,
warnings: []
};
if (!spec || spec.ok !== true || !Array.isArray(spec.runs)
|| spec.runs.length < 2 || !Array.isArray(spec.msrPaths)
|| spec.msrPaths.length !== spec.runs.length
|| !/^_[A-Za-z0-9._+\-]+$/.test(spec.fileNameSuffix || "")) {
return result;
}
let workUrl = String(spec.workPath || "").replace(/\/+$/, "");
for (let index = 0; index < spec.runs.length; index++) {
if (!/^\d+$/.test(spec.runs[index])
|| spec.msrPaths[index]
!== workUrl + "/" + spec.runs[index] + spec.fileNameSuffix + ".msr") {
let prepared = prepareIndependentSeriesCollection(spec);
let result = normalizeIndependentSeriesCollectionResult({ reason: prepared.reason });
if (!prepared.ok) return result;
let workUrl = prepared.workPath.replace(/\/+$/, "");
for (let index = 0; index < prepared.runs.length; index++) {
if (prepared.msrPaths[index]
!== workUrl + "/" + prepared.runs[index] + prepared.fileNameSuffix + ".msr") {
result.reason = "series-artifact-source-invalid";
return result;
}
}
// Generated job files are HTTP-visible, so the browser can perform the
// pure merge itself using the same implementation as Electron. Failure to
// build a display file does not block the server-side parameter collection.
let clientWarnings = [];
let combinedMSR = "";
let sourceTexts = await Promise.all(prepared.msrPaths.map(function(path) {
return readAsciiFile(path);
}));
if (sourceTexts.every(function(text) {
return typeof text === "string" && text !== "";
})) {
let merged = buildSeriesGlobalMSR(sourceTexts, prepared.runs);
if (merged.ok && isValidGeneratedGlobalMSR(merged.text)) {
combinedMSR = normalizeMSRPlotRuns(merged.text, {}, prepared.runs);
} else {
clientWarnings.push("Could not assemble the combined display MSR ("
+ (merged.reason || "series-merge-invalid") + ").");
}
} else {
clientWarnings.push("Could not fetch all fitted per-run MSR files"
+ " for the combined display.");
}
let collectionRequest = buildIndependentSeriesCollectionRequest(
prepared, combinedMSR
);
let request = await postToMusrfitCGI({
action: "series_collect",
fn: spec.msrPaths[0],
collection: JSON.stringify({
runs: spec.runs,
suffix: spec.fileNameSuffix,
fftv: spec.fftv == "y" ? "y" : "n"
})
fn: prepared.msrPaths[0],
collection: JSON.stringify(collectionRequest)
});
if (!request.ok || !request.payload) {
result.reason = "web-series-artifact-request-failed";
@@ -802,8 +810,8 @@ async function collectIndependentSeriesArtifactsWeb(spec) {
result.parameterDataPath = response.parameter_url || "";
result.combinedMSRPath = response.msr_url || "";
result.logPath = response.out_url || "";
result.warnings = Array.isArray(response.warnings)
? response.warnings.filter(Boolean) : [];
result.warnings = clientWarnings.concat(Array.isArray(response.warnings)
? response.warnings.filter(Boolean) : []);
if (response.command_log && typeof appendJobLog === "function") {
appendJobLog(response.command_log.trim());
}
@@ -815,13 +823,13 @@ async function collectIndependentSeriesArtifactsWeb(spec) {
if (typeof showWebRootPlotWindow === "function") {
showWebRootPlotWindow(rootUrls, response.job_name || "", {
msrPath: result.combinedMSRPath,
runs: spec.runs.join(" "),
runs: prepared.runs.join(" "),
rootItems: Array.isArray(response.root_items) ? response.root_items : []
});
result.plotStarted = true;
}
}
return result;
return normalizeIndependentSeriesCollectionResult(result);
}
window.collectIndependentSeriesArtifactsWeb = collectIndependentSeriesArtifactsWeb;
+1 -1
View File
@@ -10,7 +10,7 @@
"dist:linux": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux rpm AppImage --x64",
"dist:rpm": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux rpm --x64",
"dist:appimage": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux AppImage --x64",
"test:electron": "node tests/log-highlighting.test.js && node tests/action-state.test.js && node tests/electron-missing-data.test.js && node tests/window-layering.test.js && node tests/msr-generation.test.js && node tests/web-runner.test.js && node tests/root-plot-clean.test.js && node tests/parameter-plot.test.js && node tests/help-window.test.js && node tests/release-packaging.test.js && node tests/addrun.test.js && node tests/fit-statistic.test.js && node tests/series-fit-planner.test.js && node tests/series-fit-result.test.js && node tests/series-fit-inventory.test.js && node tests/series-fit-msr-preparation.test.js && node tests/series-global-merge.test.js && node tests/series-fit-staging.test.js && node tests/series-fit-runner.test.js && node tests/series-runner-adapters.test.js && node tests/series-fit-routing.test.js && node tests/singlehist-parity.test.js && node tests/url-parameters.test.js && node tests/shared-configuration.test.js"
"test:electron": "node tests/log-highlighting.test.js && node tests/action-state.test.js && node tests/electron-missing-data.test.js && node tests/window-layering.test.js && node tests/msr-generation.test.js && node tests/web-runner.test.js && node tests/root-plot-clean.test.js && node tests/parameter-plot.test.js && node tests/help-window.test.js && node tests/release-packaging.test.js && node tests/addrun.test.js && node tests/fit-statistic.test.js && node tests/series-fit-planner.test.js && node tests/series-fit-result.test.js && node tests/series-fit-inventory.test.js && node tests/series-fit-msr-preparation.test.js && node tests/series-artifact-contract.test.js && node tests/series-global-merge.test.js && node tests/series-fit-staging.test.js && node tests/series-fit-runner.test.js && node tests/series-runner-adapters.test.js && node tests/series-fit-routing.test.js && node tests/singlehist-parity.test.js && node tests/url-parameters.test.js && node tests/shared-configuration.test.js"
},
"repository": {
"type": "git",
+86 -17
View File
@@ -6,11 +6,41 @@ const childProcess = require("child_process");
const fs = require("fs");
const os = require("os");
const path = require("path");
const vm = require("vm");
const repoRoot = path.resolve(__dirname, "..");
const testRoot = fs.mkdtempSync(path.join(os.tmpdir(), "musrflow-cgi-series-collect-"));
const perlLibraryPath = [path.join(repoRoot, "tests", "perl"), process.env.PERL5LIB]
.filter(Boolean).join(path.delimiter);
const shared = { console: console, window: { MSREditor: {} } };
shared.global = shared;
vm.runInNewContext(fs.readFileSync(path.join(repoRoot, "muSRFitlib.js"), "utf8"), shared);
function mergedDisplayMSR(texts) {
const merged = shared.buildSeriesGlobalMSR(texts, ["514", "515"]);
assert.strictEqual(merged.ok, true);
return shared.normalizeMSRPlotRuns(merged.text, {}, ["514", "515"]);
}
function seriesCollectionBody(combinedMSR) {
return new URLSearchParams({
action: "series_collect",
fn: "/tmp/seriesjob/514_asy.msr",
collection: JSON.stringify({
ok: true,
version: 1,
runs: ["514", "515"],
suffix: "_asy",
fftv: "n",
artifacts: {
combined: "514+series_asy.msr",
parameters: "514+series_asy_par.dat",
log: "514+series_asy.out"
},
combinedMSR: combinedMSR
})
}).toString();
}
function fittedMSR(run) {
return [
@@ -73,8 +103,10 @@ try {
const tmpRoot = path.join(webRoot, "tmp");
const workPath = path.join(tmpRoot, "seriesjob");
fs.mkdirSync(workPath, { recursive: true });
fs.writeFileSync(path.join(workPath, "514_asy.msr"), fittedMSR(514));
fs.writeFileSync(path.join(workPath, "515_asy.msr"), fittedMSR(515));
const first514 = fittedMSR(514);
const first515 = fittedMSR(515);
fs.writeFileSync(path.join(workPath, "514_asy.msr"), first514);
fs.writeFileSync(path.join(workPath, "515_asy.msr"), first515);
fs.writeFileSync(path.join(workPath, "514+global_asy.msr"), "genuine global fit\n");
fs.writeFileSync(path.join(workPath, "514+global_tmpl.msr"), "genuine global template\n");
@@ -85,7 +117,7 @@ try {
fs.writeFileSync(fakeMsr2data, [
"#!/bin/sh",
"printf '%s\\n' \"$*\" >> \"$MUSRFLOW_TEST_M2D_ARGS\"",
"printf 'RUN Asy_ Asy_PosErr\\n514 0.2 0.01\\n515 0.21 0.01\\n' > 514+series_asy_par.dat",
"printf 'dataT Alpha_0514 Alpha_0514PosErr Alpha_0514NegErr my_parameter_0514 CHISQ NDF CHISQred RUN\\n5 0.2 0.01 0.01 1.5 100 95 1.0526 514\\n6 0.21 0.01 0.01 1.6 102 95 1.0737 515\\n' > 514+series_asy_par.dat",
"exit 9",
""
].join("\n"));
@@ -99,11 +131,7 @@ try {
].join("\n"));
fs.chmodSync(fakeMusrview, 0o755);
const body = new URLSearchParams({
action: "series_collect",
fn: "/tmp/seriesjob/514_asy.msr",
collection: JSON.stringify({ runs: ["514", "515"], suffix: "_asy", fftv: "n" })
}).toString();
const body = seriesCollectionBody(mergedDisplayMSR([first514, first515]));
const execution = childProcess.spawnSync(
"perl", [path.join(repoRoot, "cgi", "single_musrfit.cgi")], {
cwd: repoRoot,
@@ -137,13 +165,14 @@ try {
assert.match(response.warnings.join("\n"), /returned code 9/);
const m2dArgs = fs.readFileSync(m2dArgsPath, "utf8").trim().split(/\r?\n/);
assert.strictEqual(m2dArgs.length, 1);
assert.match(m2dArgs[0],
/^\[514 515\] _musrflow_series_\d+_\d+ new global data -o 514\+series_asy_par\.dat$/);
assert.strictEqual(m2dArgs[0],
"[514 515] _asy new data -o 514+series_asy_par.dat");
assert.strictEqual(fs.readFileSync(viewArgsPath, "utf8").trim(),
"--timeout 300 --root 514+series_asy.msr");
assert.strictEqual(fs.readFileSync(
path.join(workPath, "514+series_asy_par.dat"), "utf8"
).split(/\r?\n/)[0], "RUN Asy Asy_PosErr");
).split(/\r?\n/)[0],
"dataT Alpha AlphaPosErr AlphaNegErr my_parameter CHISQ NDF CHISQred RUN");
const combinedText = fs.readFileSync(path.join(workPath, "514+series_asy.msr"), "utf8");
assert.match(combinedText, /^runs\s+1\s+2$/m);
assert.match(combinedText, /^# Common parameters for all runs$/m);
@@ -174,20 +203,22 @@ try {
assert.match(response.command_log, /\[exit code 9\]/);
// Repeat the same server path with two detector RUN blocks per physical
// run. The web/Perl merger must preserve all four blocks just like the
// shared JavaScript implementation.
fs.writeFileSync(path.join(workPath, "514_asy.msr"), fittedSingleHistMSR(514));
fs.writeFileSync(path.join(workPath, "515_asy.msr"), fittedSingleHistMSR(515));
// run. The CGI publishes the four-block shared-JavaScript result unchanged.
const single514 = fittedSingleHistMSR(514);
const single515 = fittedSingleHistMSR(515);
fs.writeFileSync(path.join(workPath, "514_asy.msr"), single514);
fs.writeFileSync(path.join(workPath, "515_asy.msr"), single515);
const singleHistBody = seriesCollectionBody(mergedDisplayMSR([single514, single515]));
const singleHistExecution = childProcess.spawnSync(
"perl", [path.join(repoRoot, "cgi", "single_musrfit.cgi")], {
cwd: repoRoot,
encoding: "utf8",
input: body,
input: singleHistBody,
env: Object.assign({}, process.env, {
REQUEST_METHOD: "POST",
PERL5LIB: perlLibraryPath,
CONTENT_TYPE: "application/x-www-form-urlencoded",
CONTENT_LENGTH: String(Buffer.byteLength(body)),
CONTENT_LENGTH: String(Buffer.byteLength(singleHistBody)),
MUSRFIT_WEBROOT: webRoot,
MUSRFIT_TMPROOT: tmpRoot,
MUSRFIT_TMP_URL: "/tmp",
@@ -211,6 +242,44 @@ try {
assert.strictEqual((singleHistCombined.match(/^forward 21 25$/gm) || []).length, 2);
assert.strictEqual((singleHistCombined.match(/^forward 23 27$/gm) || []).length, 2);
assert.match(singleHistCombined, /^runs\s+1\s+2\s+3\s+4$/m);
// Parameter extraction is independent of display merging. A structural
// mismatch prevents the combined MSR, but the per-run msr2data table is
// still returned and can be plotted.
fs.writeFileSync(path.join(workPath, "514_asy.msr"), fittedMSR(514));
fs.writeFileSync(path.join(workPath, "515_asy.msr"),
fittedMSR(515).replace("asymmetry 1", "asymmetry 1\nsimplExpo 1"));
const dataOnlyBody = seriesCollectionBody("");
const dataOnlyExecution = childProcess.spawnSync(
"perl", [path.join(repoRoot, "cgi", "single_musrfit.cgi")], {
cwd: repoRoot,
encoding: "utf8",
input: dataOnlyBody,
env: Object.assign({}, process.env, {
REQUEST_METHOD: "POST",
PERL5LIB: perlLibraryPath,
CONTENT_TYPE: "application/x-www-form-urlencoded",
CONTENT_LENGTH: String(Buffer.byteLength(dataOnlyBody)),
MUSRFIT_WEBROOT: webRoot,
MUSRFIT_TMPROOT: tmpRoot,
MUSRFIT_TMP_URL: "/tmp",
MSR2DATA_BIN: fakeMsr2data,
MUSRVIEW_BIN: fakeMusrview,
MUSRFIT_RUNTIME_HOME: tmpRoot,
MUSRFLOW_TEST_M2D_ARGS: m2dArgsPath,
MUSRFLOW_TEST_VIEW_ARGS: viewArgsPath
})
}
);
assert.strictEqual(dataOnlyExecution.status, 0, dataOnlyExecution.stderr);
const dataOnlyResponse = JSON.parse(
dataOnlyExecution.stdout.split(/\r?\n\r?\n/).slice(1).join("\n\n")
);
assert.strictEqual(dataOnlyResponse.ok, true);
assert.strictEqual(dataOnlyResponse.parameter_url,
"/tmp/seriesjob/514+series_asy_par.dat");
assert.strictEqual(dataOnlyResponse.msr_url, "");
assert.deepStrictEqual(dataOnlyResponse.root_urls, []);
} finally {
fs.rmSync(testRoot, { recursive: true, force: true });
}
+54
View File
@@ -0,0 +1,54 @@
// Copyright (C) 2026 Zaher Salman
// SPDX-License-Identifier: GPL-3.0-or-later
const assert = require("assert");
const fs = require("fs");
const path = require("path");
const vm = require("vm");
const repoRoot = path.resolve(__dirname, "..");
const sandbox = { console: console, window: { MSREditor: {} } };
sandbox.global = sandbox;
vm.runInNewContext(fs.readFileSync(path.join(repoRoot, "muSRFitlib.js"), "utf8"), sandbox);
const spec = {
ok: true,
workPath: "/tmp/job",
fileNameSuffix: "_asy",
runs: ["514", "515"],
msrPaths: ["/tmp/job/514_asy.msr", "/tmp/job/515_asy.msr"],
fftv: "y"
};
const prepared = sandbox.prepareIndependentSeriesCollection(spec);
assert.strictEqual(prepared.ok, true);
assert.strictEqual(prepared.baseRun, "514");
assert.strictEqual(prepared.combinedName, "514+series_asy.msr");
assert.strictEqual(prepared.parameterName, "514+series_asy_par.dat");
assert.strictEqual(prepared.logName, "514+series_asy.out");
const request = sandbox.buildIndependentSeriesCollectionRequest(spec, "FITPARAMETER\n...");
assert.strictEqual(request.ok, true);
assert.strictEqual(request.version, 1);
assert.deepStrictEqual(Array.from(request.runs), ["514", "515"]);
assert.strictEqual(request.artifacts.combined, "514+series_asy.msr");
assert.strictEqual(request.combinedMSR, "FITPARAMETER\n...");
assert.strictEqual(sandbox.prepareIndependentSeriesCollection(Object.assign({}, spec, {
runs: ["514", "514"]
})).reason, "series-artifact-source-invalid");
const dataOnly = sandbox.normalizeIndependentSeriesCollectionResult({
ok: false,
reason: "series-display-merge-failed",
parameterDataPath: "/tmp/job/514+series_asy_par.dat",
warnings: ["display unavailable", "", null]
});
assert.strictEqual(dataOnly.ok, true);
assert.strictEqual(dataOnly.reason, "series-display-merge-failed");
assert.deepStrictEqual(Array.from(dataOnly.warnings), ["display unavailable"]);
const empty = sandbox.normalizeIndependentSeriesCollectionResult({ ok: true });
assert.strictEqual(empty.ok, false);
assert.strictEqual(empty.reason, "series-artifacts-not-produced");
console.log("series artifact contract tests passed");
+76 -3
View File
@@ -293,7 +293,7 @@ function testMergingErrors(lib) {
assert.strictEqual(results.source.ok, false);
assert.strictEqual(results.source.reason, "series-merge-source-incomplete");
// A function that multiplies run-specific Asy cannot be shared across runs.
// Run-specific function parameters are represented through RUN map slots.
const functionSource = function(run) {
const suffix = String(run).padStart(4, "0");
return [
@@ -320,8 +320,12 @@ function testMergingErrors(lib) {
results.functionRunSpecific = lib.buildSeriesGlobalMSR(
[functionSource(518), functionSource(519)], [518, 519]
);
assert.strictEqual(results.functionRunSpecific.ok, false);
assert.strictEqual(results.functionRunSpecific.reason, "series-merge-functions-run-specific");
assert.strictEqual(results.functionRunSpecific.ok, true);
sanctuaryCase("run-specific function parameters use map slots", function() {
assert.match(results.functionRunSpecific.text, /^fun1 = map1 \* map2$/m);
assert.match(results.functionRunSpecific.text, /^map\s+1\s+2$/m);
assert.match(results.functionRunSpecific.text, /^map\s+3\s+4$/m);
});
// But a function over a shared parameter stays valid and is renumbered.
const sharedFunctionSource = function(run) {
@@ -358,6 +362,73 @@ function testMergingErrors(lib) {
});
}
function testMaximumLikelihoodStatistic(lib) {
const source = function(run, value) {
return stagedAsymmetryMSR(run, "0.2")
.replace("chisq = 101.2, NDF = 100, chisq/NDF = 1.012000",
"maxLH = " + value + ", NDF = 100, maxLH/NDF = "
+ (Number(value) / 100).toFixed(6));
};
const result = lib.buildSeriesGlobalMSR(
[source(514, "120.5"), source(515, "121.5")], [514, 515]
);
assert.strictEqual(result.ok, true);
sanctuaryCase("maximum-likelihood statistics are aggregated", function() {
assert.match(result.text, /^ maxLH = 242, NDF = 200, maxLH\/NDF = 1\.21$/m);
assert.doesNotMatch(result.text, /^ chisq =/m);
});
const partial = lib.buildSeriesGlobalMSR(
[source(514, "120.5"), stagedAsymmetryMSR(515, "0.2")], [514, 515]
);
assert.strictEqual(partial.ok, true);
sanctuaryCase("mixed objectives do not produce a misleading aggregate", function() {
assert.doesNotMatch(partial.text, /^STATISTIC\b/m);
});
}
function testFunctionAndDetectorMapsCoexist(lib) {
const source = function(run) {
const suffix = String(run).padStart(4, "0");
return [
"FITPARAMETER",
"1 Phase_" + suffix + " 10 0.1 none",
"2 RelL_" + suffix + " 0 0 none",
"3 Field_" + suffix + " 75 0.1 none",
"4 Scale_" + suffix + " 0.0135 0 none",
"5 RelR_" + suffix + " 180 0 none",
"THEORY",
"TFieldCos fun1 fun2",
"FUNCTIONS",
"fun1 = par1 + map1",
"fun2 = par3 * par4",
"GLOBAL",
"fittype 0",
"RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT",
"map 2 0 0 0 0 0 0 0 0 0",
"forward 21",
"RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT",
"map 5 0 0 0 0 0 0 0 0 0",
"forward 23",
"COMMANDS",
"MIGRAD",
"PLOT 0",
"runs 1 2",
""
].join("\n");
};
const result = lib.buildSeriesGlobalMSR([source(514), source(515)], [514, 515]);
assert.strictEqual(result.ok, true);
sanctuaryCase("function maps fill zero slots beside detector maps", function() {
assert.match(result.text, /^fun1 = map2 \+ map1$/m);
assert.match(result.text, /^fun2 = map3 \* map4$/m);
assert.match(result.text, /^map\s+2\s+1\s+3\s+4$/m);
assert.match(result.text, /^map\s+5\s+1\s+3\s+4$/m);
assert.match(result.text, /^map\s+7\s+6\s+8\s+9$/m);
assert.match(result.text, /^map\s+10\s+6\s+8\s+9$/m);
});
}
(async function() {
const lib = loadLibrary();
testMergesTwoAsymmetryRuns(lib);
@@ -365,6 +436,8 @@ function testMergingErrors(lib) {
testPreservesEverySingleHistRunBlock(lib);
testFourierForDisplay(lib);
testMergingErrors(lib);
testMaximumLikelihoodStatistic(lib);
testFunctionAndDetectorMapsCoexist(lib);
console.log("series global merge tests passed");
})().catch(function(error) {
console.error(error);
+66 -9
View File
@@ -214,8 +214,13 @@ async function testElectronSeriesArtifactCollection() {
const fakeChildProcess = {
spawnSync: function(command, args, options) {
syncCalls.push({ command: command, args: args.slice(), cwd: options.cwd });
fs.writeFileSync(path.join(options.cwd, "514+series_asy_par.dat"),
"RUN Asy_ Asy_PosErr\n514 0.2 0.01\n515 0.21 0.01\n", "utf8");
fs.writeFileSync(path.join(options.cwd, "514+series_asy_par.dat"), [
"dataT dataTErr Alpha_0514 Alpha_0514PosErr Alpha_0514NegErr "
+ "my_parameter_0514 my_parameter_0514PosErr CHISQ NDF CHISQred RUN",
"5 0.1 0.2 0.01 0.01 1.5 0.1 100 95 1.0526 514",
"6 0.1 0.21 0.01 0.01 1.6 0.1 102 95 1.0737 515",
""
].join("\n"), "utf8");
return { status: 7, stdout: "parameter data written\n", stderr: "" };
},
spawn: function(command, args, options) {
@@ -248,13 +253,14 @@ async function testElectronSeriesArtifactCollection() {
assert.strictEqual(result.plotStarted, true);
assert.match(result.warnings.join("\n"), /returned code 7/);
assert.strictEqual(syncCalls.length, 1);
assert.match(syncCalls[0].args[1], /^_musrflow_series_\d+_[a-z0-9]+$/);
assert.deepStrictEqual(Array.from(syncCalls[0].args.slice(0, 1)), ["[514 515]"]);
assert.deepStrictEqual(Array.from(syncCalls[0].args.slice(2)),
["new", "global", "data", "-o", "514+series_asy_par.dat"]);
assert.deepStrictEqual(Array.from(syncCalls[0].args),
["[514 515]", "_asy", "new", "data", "-o", "514+series_asy_par.dat"]);
assert.deepStrictEqual(Array.from(asyncCalls[0].args), ["514+series_asy.msr"]);
assert.match(fs.readFileSync(result.parameterDataPath, "utf8"),
/^RUN Asy Asy_PosErr/m);
const parameterHeader = fs.readFileSync(result.parameterDataPath, "utf8")
.split(/\r?\n/)[0];
assert.strictEqual(parameterHeader,
"dataT dataTErr Alpha AlphaPosErr AlphaNegErr "
+ "my_parameter my_parameterPosErr CHISQ NDF CHISQred RUN");
const combinedText = fs.readFileSync(result.combinedMSRPath, "utf8");
assert.match(combinedText, /^runs\s+1\s+2$/m);
assert.match(combinedText, /^# Common parameters for all runs$/m);
@@ -274,9 +280,60 @@ async function testElectronSeriesArtifactCollection() {
}
}
async function testElectronParameterCollectionSurvivesDisplayMergeFailure() {
const tempDirectory = fs.mkdtempSync(path.join(os.tmpdir(), "musrflow-series-data-only-"));
const syncCalls = [];
const asyncCalls = [];
try {
fs.writeFileSync(path.join(tempDirectory, "514_asy.msr"),
runnableMSR(514, true), "utf8");
fs.writeFileSync(path.join(tempDirectory, "515_asy.msr"),
runnableMSR(515, true).replace("asymmetry 1", "asymmetry 1\nsimplExpo 1"),
"utf8");
const sandbox = loadElectronSandbox({
spawnSync: function(command, args, options) {
syncCalls.push(args.slice());
fs.writeFileSync(path.join(options.cwd, "514+series_asy_par.dat"),
"RUN Asy_\n514 0.2\n515 0.21\n", "utf8");
return { status: 0, stdout: "", stderr: "" };
},
spawn: function(command, args) {
asyncCalls.push(args.slice());
const child = new events.EventEmitter();
child.stdout = new events.EventEmitter();
child.stderr = new events.EventEmitter();
return child;
}
}, []);
const result = await sandbox.window.collectIndependentSeriesArtifactsLocal({
ok: true,
workPath: tempDirectory,
fileNameSuffix: "_asy",
runs: ["514", "515"],
msrPaths: [
path.join(tempDirectory, "514_asy.msr"),
path.join(tempDirectory, "515_asy.msr")
],
fftv: "n"
});
assert.strictEqual(result.ok, true);
assert.strictEqual(result.parameterDataPath,
path.join(tempDirectory, "514+series_asy_par.dat"));
assert.strictEqual(result.combinedMSRPath, "");
assert.strictEqual(result.plotStarted, false);
assert.strictEqual(syncCalls.length, 1);
assert.strictEqual(asyncCalls.length, 0);
assert.match(result.warnings.join("\n"), /series-merge-theory-mismatch/);
} finally {
fs.rmSync(tempDirectory, { recursive: true, force: true });
}
}
Promise.resolve().then(testElectronAdapterFitsOnlyOneMSR)
.then(testElectronAdapterRetainsRejectedNewMSR)
.then(testElectronSeriesArtifactCollection).then(function() {
.then(testElectronSeriesArtifactCollection)
.then(testElectronParameterCollectionSurvivesDisplayMergeFailure).then(function() {
console.log("series runner adapter tests passed");
}).catch(function(error) {
console.error(error);
+58 -4
View File
@@ -272,6 +272,10 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() {
const logs = [];
const lib = loadWebLibrary(async function(url, options) {
calls.push({ url: url, options: options });
if (!options || options.method !== "POST") {
const run = /515_asy\.msr$/.test(url) ? 515 : 514;
return createResponse(convergedWebMSR(run));
}
return createResponse(JSON.stringify({
ok: true,
code: 0,
@@ -307,13 +311,24 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() {
assert.strictEqual(result.plotPath, "/tmp/job/514+series_asy_0.root");
assert.strictEqual(result.plotStarted, true);
assert.match(result.warnings[0], /returned code 9/);
assert.strictEqual(calls.length, 1);
const body = new URLSearchParams(calls[0].options.body);
assert.strictEqual(calls.length, 3);
assert.deepStrictEqual(calls.slice(0, 2).map(function(call) { return call.url; }),
["/tmp/job/514_asy.msr", "/tmp/job/515_asy.msr"]);
const body = new URLSearchParams(calls[2].options.body);
assert.strictEqual(body.get("action"), "series_collect");
assert.strictEqual(body.get("fn"), "/tmp/job/514_asy.msr");
assert.deepStrictEqual(JSON.parse(body.get("collection")), {
runs: ["514", "515"], suffix: "_asy", fftv: "n"
const collection = JSON.parse(body.get("collection"));
assert.strictEqual(collection.version, 1);
assert.deepStrictEqual(collection.runs, ["514", "515"]);
assert.strictEqual(collection.suffix, "_asy");
assert.strictEqual(collection.fftv, "n");
assert.deepStrictEqual(collection.artifacts, {
combined: "514+series_asy.msr",
parameters: "514+series_asy_par.dat",
log: "514+series_asy.out"
});
assert.match(collection.combinedMSR, /^# Independent series display MSR/m);
assert.match(collection.combinedMSR, /^runs\s+1\s+2$/m);
assert.strictEqual(plots.length, 1);
assert.deepStrictEqual(Array.from(plots[0].urls),
["/tmp/job/514+series_asy_0.root"]);
@@ -321,6 +336,44 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() {
assert.match(logs[0], /msr2data/);
}
async function testWebParameterCollectionSurvivesClientMergeFailure() {
const calls = [];
const lib = loadWebLibrary(async function(url, options) {
calls.push({ url: url, options: options });
if (!options || options.method !== "POST") {
const run = /515_asy\.msr$/.test(url) ? 515 : 514;
let text = convergedWebMSR(run);
if (run === 515) text = text.replace("asymmetry 1", "asymmetry 1\nsimplExpo 1");
return createResponse(text);
}
return createResponse(JSON.stringify({
ok: true,
reason: "series-artifacts-collected",
parameter_url: "/tmp/job/514+series_asy_par.dat",
msr_url: "",
out_url: "/tmp/job/514+series_asy.out",
root_urls: [],
warnings: []
}));
});
const result = await lib.window.collectIndependentSeriesArtifactsWeb({
ok: true,
workPath: "/tmp/job",
fileNameSuffix: "_asy",
runs: ["514", "515"],
msrPaths: ["/tmp/job/514_asy.msr", "/tmp/job/515_asy.msr"],
fftv: "n"
});
assert.strictEqual(result.ok, true);
assert.strictEqual(result.parameterDataPath, "/tmp/job/514+series_asy_par.dat");
assert.strictEqual(result.combinedMSRPath, "");
assert.strictEqual(result.plotStarted, false);
assert.match(result.warnings.join("\n"), /series-merge-theory-mismatch/);
const body = new URLSearchParams(calls[2].options.body);
assert.strictEqual(JSON.parse(body.get("collection")).combinedMSR, "");
}
function convergedWebMSR(run) {
const suffix = String(run).padStart(4, "0");
return [
@@ -615,6 +668,7 @@ async function runTests() {
await testSeriesTransactionRejectsMixedJobDirectories();
await testWebSeriesCandidateUsesDedicatedEndpoint();
await testWebSeriesCollectionRegistersArtifactsAndPlot();
await testWebParameterCollectionSurvivesClientMergeFailure();
await testWebSeriesInventoryFeedsSharedClassifier();
await testIncompleteWebSeriesInventoryFailsClosed();
await testWebRunLoadsFittedMSRAndArtifacts();