First cleanly working series
This commit is contained in:
1 parent
2c34f86f22
commit
f7a45054fd
11 files changed
+692
-704
No files matched your search
@@ -37,7 +37,7 @@ Status legend:
|
||||
| Run-number expansion (`-`, `:` syntax) | `MSR.pm::ExpandRunNumbers` | `arrayRunNumbers()` | `PARTIAL` | Common ascending single/range/step syntax is supported with stricter validation. Legacy descending `from:to:step` input swaps its endpoints; the current GUI deliberately rejects descending ranges instead. |
|
||||
| Single-run/asymmetry MSR creation | `MSR.pm::CreateMSRUni` | `createMSR()` / `buildMSR()` | `PARTIAL` | Core FITPARAMETER, THEORY/FUNCTIONS, GLOBAL, RUN, COMMANDS, PLOT, and FOURIER blocks are generated. Multiple fit ranges, explicit data-good-bin ranges, and legacy title handling remain gaps. |
|
||||
| Multiple fit ranges / packings | `MSR.pm::CreateMSRUni` (`Tis`, `Tfs`, `BINS` arrays) | Single Fit range row | `MISSING` | Legacy CGI accepted comma-separated range/packing sets and repeated RUN blocks accordingly. Current generation expects one fit interval and one packing value. |
|
||||
| Global MSR generation | `MSR.pm::CreateMSRGLB`, `msr2data global` | `prepareMSRRunFiles()` + `buildSeriesGlobalMSR()` | `DONE` | Multi-run asymmetry merges the already-fitted per-run MSRs into the combined display MSR in JS (`muSRFitlib.js`) and Perl (`single_musrfit.cgi`). msr2data is no longer asked to build it, because it only writes its `+global` file from a template and re-fits each run in every value-populating mode. Stale output is removed and the merged MSR is validated before use, instead of relying on msr2data's unreliable exit codes. |
|
||||
| Global MSR generation | `MSR.pm::CreateMSRGLB`, `msr2data global` | `prepareMSRRunFiles()` + `buildSeriesGlobalMSR()` | `DONE` | Electron and the browser use the same JavaScript merger for already-fitted independent runs. In web mode the browser fetches the HTTP-visible fitted MSRs and submits the combined display text; the CGI verifies its artifact names, size, structure, and exact RUN/ADDRUN identity before publishing it. `msr2data` collects the per-run parameter table directly and never re-fits the combined display. |
|
||||
| Parameter table and bounds | `MSR.pm::PrepParamTable` | `cfgParams()`, `formatFitParameterLine()`, fitted-MSR parser | `PARTIAL` | Value/step and two-sided or one-sided bounds are generated correctly. Fitted values and bounds can be imported. The GUI does not separately preserve/display the asymmetric positive MINOS error field from a seven-field fitted parameter line. |
|
||||
| Parameter-data export | `MSR.pm::ExportParams`, post-fit `msr2data` | Web/Electron runners + floating parameter viewer | `DONE` | Both runners generate `<fit>_par.dat` after single/global fitting and expose the same artifact to the integrated floating Plotly viewer; the raw table remains available in Results. |
|
||||
|
||||
@@ -59,7 +59,7 @@ Status legend:
|
||||
| Plot-only and fit+plot execution | Legacy `plotonly` / `fitplot` commands | `muSRFitelec.js`, `muSRFitweb.js`, `cgi/single_musrfit.cgi` | `DONE` | Electron and web have separate Plot/Fit paths. Multi-run jobs assemble the combined display MSR from the per-run fitted files and normalize PLOT run indices before plotting. Fit and Plot share an action lock with visible preparation/execution phases, preventing duplicate or conflicting requests. Missing RUN data from either musrfit or musrview is reported as an amber `Missing data` outcome rather than a failed fit/plot; Fit stops before starting musrview in that case. |
|
||||
| Interactive plot delivery | Legacy `musrview --png` + ImageMagick crop | Native musrview (Electron), `musrview --root` + JSROOT (web) | `DONE` | This intentionally replaces static PNG delivery. Web ROOT files support interactive zoom/resizing and FFT toggling; Electron keeps the native musrview window. |
|
||||
| Spectrum ASCII extraction + gzip | Legacy `musrview --ascii` + `gzip FILENAME.dat` | No equivalent spectrum dump | `MISSING` | Job tarballs and parameter `.dat` export are different artifacts and do not replace the legacy plotted-data ASCII dump. Add only if this download remains useful to users. |
|
||||
| `msr2data` integration | Legacy `$MSR2DATA` workflow | Local runner + `cgi/single_musrfit.cgi` | `DONE` | msr2data handles the post-fit parameter `.dat` export in both runners. The combined multi-run display MSR is assembled locally from the per-run fitted files instead; the merged MSR is validated explicitly before use. |
|
||||
| `msr2data` integration | Legacy `$MSR2DATA` workflow | Local runner + `cgi/single_musrfit.cgi` | `DONE` | `msr2data` directly collects the fitted per-run parameter `.dat` table in both runners. Combined display construction is an independent best-effort operation, so an unsupported display model cannot disable Plot parameters. |
|
||||
| Debug/command output | Legacy downloadable `.out` / debug files | `.out` command log + Results details + concise status bar | `DONE` | Current logs record command, combined output, and exit status while keeping routine live status compact. |
|
||||
| Web health/runtime configuration | No direct legacy equivalent | CGI `health` action + `muSRFitweb.js` | `DONE` | Reports runtime/executable/tmp/data-root readiness and refreshes the server-owned data root before actions. |
|
||||
| Dedicated fit statistic display | `musrfit.cgi::PrintOut` / fitted `STATISTIC` block | `parseMSRFitStatistic()` + status bar | `DONE` | After Fit, Electron and web parse the fitted MSR's overall `STATISTIC` line and keep chisq/NDF or maxLH/NDF visible in the status bar; `*** FIT DID NOT CONVERGE ***` is surfaced explicitly. Plot-only actions do not reinterpret an older statistic as a new result. |
|
||||
|
||||
+91
-514
@@ -194,7 +194,7 @@ sub rollback_series_receipts {
|
||||
return \@errors;
|
||||
}
|
||||
|
||||
sub series_run_directive_signature {
|
||||
sub series_run_directives {
|
||||
my ($content) = @_;
|
||||
my @directives = ();
|
||||
foreach my $line (split(/\r?\n/, $content || "")) {
|
||||
@@ -209,7 +209,12 @@ sub series_run_directive_signature {
|
||||
$parts[0] = uc($parts[0]);
|
||||
push(@directives, join(" ", @parts[0 .. 4]));
|
||||
}
|
||||
return sha256_hex(join("\n", @directives));
|
||||
return \@directives;
|
||||
}
|
||||
|
||||
sub series_run_directive_signature {
|
||||
my ($content) = @_;
|
||||
return sha256_hex(join("\n", @{series_run_directives($content)}));
|
||||
}
|
||||
|
||||
sub perform_series_transaction {
|
||||
@@ -855,19 +860,25 @@ sub perform_series_transaction_finalization {
|
||||
};
|
||||
}
|
||||
|
||||
# msr2data represents run-qualified parameters such as Asy_2274 as Asy_ in
|
||||
# the exported data header. Remove exactly one trailing underscore from header
|
||||
# fields while preserving embedded underscores and labels such as Asy_PosErr.
|
||||
# Global msr2data exports represent a run-qualified parameter such as Asy_2274
|
||||
# as Asy_, while ordinary per-run exports retain the first run suffix as
|
||||
# Asy_2274, Asy_2274PosErr, and Asy_2274NegErr. Normalize only suffixes that
|
||||
# match this collection's run list while preserving embedded underscores.
|
||||
sub clean_parameter_data_header {
|
||||
my ($path) = @_;
|
||||
my ($path, $run_numbers) = @_;
|
||||
my $content = read_text_file($path);
|
||||
return 0 if $content eq "";
|
||||
my %runs = map { (0 + $_) => 1 }
|
||||
grep { defined($_) && /^\d+$/ } @{ref($run_numbers) eq "ARRAY" ? $run_numbers : []};
|
||||
|
||||
my @lines = split(/\n/, $content, -1);
|
||||
foreach my $line (@lines) {
|
||||
my $probe = $line;
|
||||
$probe =~ s/\r$//;
|
||||
next if $probe =~ /^\s*$/ || $probe =~ /^\s*#/;
|
||||
$line =~ s{(\S+?)_([0-9]+)((?:_)?(?:PosErr|NegErr))?(?=\s|$)}{
|
||||
exists($runs{0 + $2}) ? $1 . (defined($3) ? $3 : "") : $&
|
||||
}gex;
|
||||
$line =~ s/(\S+)_(?=\s|$)/$1/g;
|
||||
last;
|
||||
}
|
||||
@@ -995,463 +1006,7 @@ sub normalize_plot_runs_line {
|
||||
return $content;
|
||||
}
|
||||
|
||||
# Merge the fitted per-run MSR files of an independent series fit into a single
|
||||
# display-only global MSR. msr2data is deliberately not used for this: it only
|
||||
# writes its +global file from a template and, in the modes that populate
|
||||
# per-run values, re-fits each run with musrfit. The merged file keeps each
|
||||
# run's already-fitted parameter values and lets every RUN block select its own
|
||||
# parameters through the same mapN/alpha/norm mechanism a genuine msr2data
|
||||
# global file uses.
|
||||
my @SERIES_MSR_BLOCK_NAMES = qw(FITPARAMETER THEORY FUNCTIONS GLOBAL RUN COMMANDS PLOT FOURIER STATISTIC);
|
||||
my %SERIES_MSR_RUN_PARAM_DIRECTIVES = map { $_ => 1 } qw(alpha beta norm backgr.fit);
|
||||
my $SERIES_MSR_MAX_MAP_SLOTS = 10;
|
||||
|
||||
# Recognize a block header line. The FOURIER block uses a lowercase
|
||||
# `plot <type>` directive that would otherwise look like a PLOT header, so a
|
||||
# genuine PLOT header must carry a numeric plot id ("PLOT 2"); non-numeric
|
||||
# `plot` lines stay inside FOURIER.
|
||||
sub series_msr_block_header_name {
|
||||
my ($trimmed) = @_;
|
||||
for my $name (@SERIES_MSR_BLOCK_NAMES) {
|
||||
if ($trimmed =~ /^\Q$name\E(?:\s|$)/i) {
|
||||
my $candidate = uc($name);
|
||||
if ($candidate eq "PLOT") {
|
||||
my $arg = $trimmed;
|
||||
$arg =~ s/^\Q$name\E//i;
|
||||
$arg =~ s/^\s+|\s+$//g;
|
||||
return "" if $arg !~ /^\d/;
|
||||
}
|
||||
return $candidate;
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
sub series_msr_blocks {
|
||||
my ($content) = @_;
|
||||
my %blocks;
|
||||
my $section = "";
|
||||
for my $line (split(/\r?\n/, defined($content) ? $content : "")) {
|
||||
my $trimmed = $line;
|
||||
$trimmed =~ s/^\s+//;
|
||||
my $block = series_msr_block_header_name($trimmed);
|
||||
if ($block ne "") {
|
||||
my $start = $line;
|
||||
$start =~ s/^\s+//;
|
||||
$start =~ s/\s+$//;
|
||||
$section = $block;
|
||||
if (!exists($blocks{$block})) {
|
||||
$blocks{$block} = { header => $start, lines => [] };
|
||||
} elsif ($block eq "RUN") {
|
||||
# RUN is repeatable. Preserve later detector headers inside the
|
||||
# first collected RUN section so series_msr_run_blocks can
|
||||
# reconstruct every histogram block.
|
||||
push(@{$blocks{$block}{lines}}, $start);
|
||||
}
|
||||
} elsif ($section ne "") {
|
||||
push(@{$blocks{$section}{lines}}, $line);
|
||||
} else {
|
||||
$blocks{HEADER} = { header => "", lines => [] }
|
||||
if !exists($blocks{HEADER});
|
||||
push(@{$blocks{HEADER}{lines}}, $line);
|
||||
}
|
||||
}
|
||||
return \%blocks;
|
||||
}
|
||||
|
||||
sub series_msr_parameters {
|
||||
my ($lines) = @_;
|
||||
my @params;
|
||||
for my $line (@{$lines || []}) {
|
||||
my $trimmed = $line;
|
||||
$trimmed =~ s/^\s+//;
|
||||
next if $trimmed eq "" || $trimmed =~ /^#/;
|
||||
my ($no, $name, $rest) = $trimmed =~ /^(\d+)\s+(\S+)\s*(.*)$/;
|
||||
next if !defined($no);
|
||||
$rest = "" if !defined($rest);
|
||||
$rest =~ s/^\s+|\s+$//g;
|
||||
$rest =~ s/\s+/ /g;
|
||||
push(@params, { no => 0 + $no, name => $name, rest => $rest });
|
||||
}
|
||||
return \@params;
|
||||
}
|
||||
|
||||
sub series_msr_run_blocks {
|
||||
my ($header, $lines) = @_;
|
||||
my @blocks;
|
||||
my $current;
|
||||
for my $line (($header, @{$lines || []})) {
|
||||
my $trimmed = $line;
|
||||
$trimmed =~ s/^\s+//;
|
||||
if ($trimmed =~ /^RUN\b/i) {
|
||||
my $start = $line;
|
||||
$start =~ s/\s+$//;
|
||||
$current = { header => $start, lines => [] };
|
||||
push(@blocks, $current);
|
||||
} elsif (defined($current)) {
|
||||
push(@{$current->{lines}}, $line);
|
||||
}
|
||||
}
|
||||
return \@blocks;
|
||||
}
|
||||
|
||||
sub series_msr_token_shape {
|
||||
my ($lines) = @_;
|
||||
return join("\n", map {
|
||||
join(" ", map { /^\d+$/ ? "#" : $_ } split(/\s+/, $_))
|
||||
} @{$lines || []});
|
||||
}
|
||||
|
||||
sub format_series_merged_parameter_line {
|
||||
my ($no, $name, $rest) = @_;
|
||||
my $line = " $no $name";
|
||||
$line .= " $rest" if defined($rest) && $rest ne "";
|
||||
return $line;
|
||||
}
|
||||
|
||||
# The FOURIER block is shared by all RUN blocks in the merged file. musrview
|
||||
# refuses any plot other than POWER unless the phase line carries one reference
|
||||
# per run (parR7 par9 ...), which a display file assembled from shared
|
||||
# parameters cannot express. Mirror msr2data's own global output by forcing the
|
||||
# plot to POWER; the fitted per-run Fourier phases are deliberately left to the
|
||||
# individual run files.
|
||||
sub series_fourier_for_display {
|
||||
my ($lines) = @_;
|
||||
my @normalized;
|
||||
my $saw_plot = 0;
|
||||
for my $line (@{$lines || []}) {
|
||||
my $trimmed = $line;
|
||||
$trimmed =~ s/^\s+//;
|
||||
if ($trimmed =~ /^plot\b/i) {
|
||||
$saw_plot = 1;
|
||||
if ($trimmed !~ /^plot\s+POWER\b/i) {
|
||||
push(@normalized, "plot POWER");
|
||||
} else {
|
||||
push(@normalized, $line);
|
||||
}
|
||||
} else {
|
||||
push(@normalized, $line);
|
||||
}
|
||||
}
|
||||
push(@normalized, "plot POWER") if !$saw_plot;
|
||||
return \@normalized;
|
||||
}
|
||||
|
||||
# Returns [$ok, $reason, $text].
|
||||
sub build_series_global_msr {
|
||||
my ($msr_texts, $run_numbers) = @_;
|
||||
return [0, "series-merge-invalid-input", ""]
|
||||
if ref($msr_texts) ne "ARRAY" || ref($run_numbers) ne "ARRAY"
|
||||
|| @{$msr_texts} < 2 || @{$msr_texts} != @{$run_numbers};
|
||||
|
||||
my @per_run;
|
||||
for my $index (0 .. $#$msr_texts) {
|
||||
my $blocks = series_msr_blocks($msr_texts->[$index]);
|
||||
return [0, "series-merge-source-incomplete", ""]
|
||||
if !$blocks->{FITPARAMETER} || !$blocks->{THEORY} || !$blocks->{RUN};
|
||||
my $params = series_msr_parameters($blocks->{FITPARAMETER}{lines});
|
||||
return [0, "series-merge-parameters-incomplete", ""] if !@{$params};
|
||||
my $run_blocks = series_msr_run_blocks(
|
||||
$blocks->{RUN}{header}, $blocks->{RUN}{lines}
|
||||
);
|
||||
return [0, "series-merge-run-blocks-incomplete", ""] if !@{$run_blocks};
|
||||
push(@per_run, {
|
||||
run => "$run_numbers->[$index]",
|
||||
params => $params,
|
||||
run_blocks => $run_blocks,
|
||||
theory => $blocks->{THEORY}{lines} || [],
|
||||
functions => $blocks->{FUNCTIONS}{lines} || [],
|
||||
global => $blocks->{GLOBAL}{lines} || [],
|
||||
commands => $blocks->{COMMANDS}{lines} || [],
|
||||
plot => $blocks->{PLOT}{lines} || [],
|
||||
plot_header => $blocks->{PLOT}{header} || "PLOT 0",
|
||||
fourier => $blocks->{FOURIER}{lines} || [],
|
||||
statistic => $blocks->{STATISTIC}{lines} || []
|
||||
});
|
||||
}
|
||||
|
||||
my $num_runs = scalar(@per_run);
|
||||
my $first = $per_run[0];
|
||||
for my $pr (@per_run) {
|
||||
return [0, "series-merge-parameter-count-mismatch", ""]
|
||||
if @{$pr->{params}} != @{$first->{params}};
|
||||
}
|
||||
|
||||
# A name present in every run is shared; present in one run is run-specific.
|
||||
my %name_counts;
|
||||
for my $pr (@per_run) {
|
||||
$name_counts{$_->{name}}++ for @{$pr->{params}};
|
||||
}
|
||||
for my $name (keys %name_counts) {
|
||||
return [0, "series-merge-parameter-inconsistency", ""]
|
||||
if $name_counts{$name} != 1 && $name_counts{$name} != $num_runs;
|
||||
}
|
||||
|
||||
# All runs must describe the same model: the non-numeric token shape of the
|
||||
# THEORY and FUNCTIONS blocks is identical, only parameter numbers differ.
|
||||
for my $pr (@per_run) {
|
||||
return [0, "series-merge-theory-mismatch", ""]
|
||||
if series_msr_token_shape($pr->{theory}) ne series_msr_token_shape($first->{theory});
|
||||
return [0, "series-merge-functions-mismatch", ""]
|
||||
if series_msr_token_shape($pr->{functions}) ne series_msr_token_shape($first->{functions});
|
||||
}
|
||||
|
||||
# Assign merged numbers: shared parameters first, run-specific grouped per run.
|
||||
my @merged_params;
|
||||
my %shared_name_no;
|
||||
for my $p (@{$first->{params}}) {
|
||||
if ($name_counts{$p->{name}} == $num_runs
|
||||
&& !exists($shared_name_no{$p->{name}})) {
|
||||
$shared_name_no{$p->{name}} = scalar(@merged_params) + 1;
|
||||
push(@merged_params, { name => $p->{name}, rest => $p->{rest} });
|
||||
}
|
||||
}
|
||||
my @no_map;
|
||||
for my $run_index (0 .. $num_runs - 1) {
|
||||
$no_map[$run_index] = {};
|
||||
for my $p (@{$per_run[$run_index]{params}}) {
|
||||
if ($name_counts{$p->{name}} == $num_runs) {
|
||||
$no_map[$run_index]{$p->{no}} = $shared_name_no{$p->{name}};
|
||||
} else {
|
||||
$no_map[$run_index]{$p->{no}} = scalar(@merged_params) + 1;
|
||||
push(@merged_params, { name => $p->{name}, rest => $p->{rest} });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
my %by_number;
|
||||
for my $pr (@per_run) {
|
||||
my %map;
|
||||
$map{$_->{no}} = $_ for @{$pr->{params}};
|
||||
$by_number{$pr} = \%map;
|
||||
}
|
||||
|
||||
# Rewrite THEORY: existing mapN/funN stay, direct integers are renumbered,
|
||||
# run-specific roles become map slots filled per RUN block below.
|
||||
my %occupied;
|
||||
for my $lines ($first->{theory}, $first->{functions}) {
|
||||
for my $line (@{$lines || []}) {
|
||||
for my $token (split(/\s+/, $line)) {
|
||||
$occupied{$1}++ if $token =~ /^map(\d+)$/i;
|
||||
}
|
||||
}
|
||||
}
|
||||
my $next_slot = 1;
|
||||
$next_slot++ while $occupied{$next_slot};
|
||||
my (%slot_role_name, %slot_position);
|
||||
my $slot_for = sub {
|
||||
my ($name, $line_index, $token_index) = @_;
|
||||
return $slot_role_name{$name} if exists($slot_role_name{$name});
|
||||
return 0 if $next_slot > $SERIES_MSR_MAX_MAP_SLOTS;
|
||||
my $slot = $next_slot++;
|
||||
$slot_role_name{$name} = $slot;
|
||||
$slot_position{$slot} = [ $line_index, $token_index ];
|
||||
return $slot;
|
||||
};
|
||||
|
||||
my @merged_theory;
|
||||
for my $line_index (0 .. $#{$first->{theory}}) {
|
||||
my @out;
|
||||
my $token_index = 0;
|
||||
for my $token (split(/\s+/, $first->{theory}[$line_index])) {
|
||||
if ($token =~ /^\d+$/ && $token ne "0") {
|
||||
my $local_no = 0 + $token;
|
||||
my $p = $by_number{$first}{$local_no};
|
||||
return [0, "series-merge-theory-param-missing", ""] if !$p;
|
||||
if ($name_counts{$p->{name}} == $num_runs) {
|
||||
push(@out, $shared_name_no{$p->{name}});
|
||||
} else {
|
||||
my $slot = $slot_for->($p->{name}, $line_index, $token_index);
|
||||
return [0, "series-merge-map-slots-exhausted", ""] if $slot == 0;
|
||||
push(@out, "map$slot");
|
||||
}
|
||||
} else {
|
||||
push(@out, $token);
|
||||
}
|
||||
$token_index++;
|
||||
}
|
||||
push(@merged_theory, join(" ", @out));
|
||||
}
|
||||
|
||||
# FUNCTIONS is shared, so it may only reference parameters that keep their
|
||||
# number in every RUN block.
|
||||
my @merged_functions;
|
||||
for my $line (@{$first->{functions}}) {
|
||||
my @out;
|
||||
for my $token (split(/\s+/, $line)) {
|
||||
if ($token =~ /^par(\d+)$/i) {
|
||||
my $local_no = 0 + $1;
|
||||
my $p = $by_number{$first}{$local_no};
|
||||
return [0, "series-merge-functions-param-missing", ""] if !$p;
|
||||
return [0, "series-merge-functions-run-specific", ""]
|
||||
if $name_counts{$p->{name}} != $num_runs;
|
||||
push(@out, "par" . $shared_name_no{$p->{name}});
|
||||
} else {
|
||||
push(@out, $token);
|
||||
}
|
||||
}
|
||||
push(@merged_functions, join(" ", @out));
|
||||
}
|
||||
|
||||
my $max_slot = 0;
|
||||
for my $slot (keys %slot_position) { $max_slot = $slot if $slot > $max_slot; }
|
||||
|
||||
my $merged_map_entries = sub {
|
||||
my ($run_index, $block) = @_;
|
||||
my @existing_values;
|
||||
for my $line (@{$block->{lines}}) {
|
||||
if ($line =~ /^\s*map\b/i) {
|
||||
my @tokens = split(/\s+/, $line);
|
||||
@existing_values = @tokens > 1 ? @tokens[1 .. $#tokens] : ();
|
||||
last;
|
||||
}
|
||||
}
|
||||
my %entries;
|
||||
my $slot_max = $max_slot;
|
||||
$slot_max = scalar(@existing_values) if @existing_values > $slot_max;
|
||||
for my $slot (1 .. $slot_max) {
|
||||
my $value = 0;
|
||||
if ($slot <= @existing_values && $existing_values[$slot - 1] =~ /^\d+$/) {
|
||||
my $local_no = 0 + $existing_values[$slot - 1];
|
||||
$value = ($local_no > 0 && defined($no_map[$run_index]{$local_no}))
|
||||
? $no_map[$run_index]{$local_no} : $local_no;
|
||||
} elsif (exists($slot_position{$slot})) {
|
||||
my ($line_index, $token_index) = @{$slot_position{$slot}};
|
||||
my @local_tokens = split(/\s+/, $per_run[$run_index]{theory}[$line_index] || "");
|
||||
my $local_no = $local_tokens[$token_index] =~ /^\d+$/
|
||||
? 0 + $local_tokens[$token_index] : 0;
|
||||
$value = $no_map[$run_index]{$local_no}
|
||||
if $local_no > 0 && defined($no_map[$run_index]{$local_no});
|
||||
}
|
||||
$entries{$slot} = $value;
|
||||
}
|
||||
my @values;
|
||||
for my $slot (1 .. $slot_max) { push(@values, $entries{$slot}); }
|
||||
pop @values while @values && $values[$#values] == 0;
|
||||
return \@values;
|
||||
};
|
||||
|
||||
my @output;
|
||||
push(@output, "# Independent series display MSR for the runs: " . join(" ", @{$run_numbers}));
|
||||
push(@output, "# Fitted per-run parameters collected without re-fitting; "
|
||||
. "msr2data handles parameter extraction separately.");
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "FITPARAMETER");
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "# Common parameters for all runs");
|
||||
for my $index (0 .. $#merged_params) {
|
||||
next if $name_counts{$merged_params[$index]{name}} != $num_runs;
|
||||
push(@output, format_series_merged_parameter_line(
|
||||
$index + 1, $merged_params[$index]{name}, $merged_params[$index]{rest}
|
||||
));
|
||||
}
|
||||
for my $run_index (0 .. $num_runs - 1) {
|
||||
push(@output, "# Specific parameters for run " . $per_run[$run_index]{run});
|
||||
for my $p (@{$per_run[$run_index]{params}}) {
|
||||
next if $name_counts{$p->{name}} == $num_runs;
|
||||
push(@output, format_series_merged_parameter_line(
|
||||
$no_map[$run_index]{$p->{no}}, $p->{name}, $p->{rest}
|
||||
));
|
||||
}
|
||||
}
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "THEORY");
|
||||
push(@output, @merged_theory);
|
||||
if (@{$first->{functions}}) {
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "FUNCTIONS");
|
||||
push(@output, @merged_functions);
|
||||
}
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "GLOBAL");
|
||||
push(@output, @{$first->{global}});
|
||||
for my $run_index (0 .. $num_runs - 1) {
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "# RUN blocks for run " . $per_run[$run_index]{run});
|
||||
for my $block (@{$per_run[$run_index]{run_blocks}}) {
|
||||
push(@output, $block->{header});
|
||||
my $had_map = 0;
|
||||
for my $line (@{$block->{lines}}) {
|
||||
my $trimmed = $line;
|
||||
$trimmed =~ s/^\s+//;
|
||||
next if $trimmed eq "" || $trimmed =~ /^#/;
|
||||
my ($directive) = $trimmed =~ /^(\S+)/;
|
||||
if ($directive =~ /^map$/i) { $had_map = 1; next; }
|
||||
if ($SERIES_MSR_RUN_PARAM_DIRECTIVES{lc($directive)}) {
|
||||
my $rest = $trimmed;
|
||||
$rest =~ s/^\s*\Q$directive\E\s*//;
|
||||
my @renumbered;
|
||||
for my $token (split(/\s+/, $rest)) {
|
||||
if ($token =~ /^\d+$/) {
|
||||
my $n = 0 + $token;
|
||||
push(@renumbered, defined($no_map[$run_index]{$n})
|
||||
? $no_map[$run_index]{$n} : $token);
|
||||
} else {
|
||||
push(@renumbered, $token);
|
||||
}
|
||||
}
|
||||
push(@output, $directive . " " . join(" ", @renumbered));
|
||||
} else {
|
||||
push(@output, $line);
|
||||
}
|
||||
}
|
||||
if ($had_map || $max_slot > 0) {
|
||||
my $values = $merged_map_entries->($run_index, $block);
|
||||
push(@output, "map " . join(" ", @{$values})) if @{$values};
|
||||
}
|
||||
}
|
||||
}
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "COMMANDS");
|
||||
push(@output, @{$first->{commands}});
|
||||
if (@{$first->{fourier}}) {
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "FOURIER");
|
||||
push(@output, @{series_fourier_for_display($first->{fourier})});
|
||||
}
|
||||
push(@output, "###############################################################");
|
||||
push(@output, $first->{plot_header});
|
||||
for my $line (@{$first->{plot}}) {
|
||||
next if $line =~ /^\s*runs\s+/i;
|
||||
push(@output, $line);
|
||||
}
|
||||
my @plot_runs = (1 .. $num_runs);
|
||||
push(@output, "runs " . join(" ", @plot_runs));
|
||||
|
||||
# musrfit always stamps a STATISTIC block into a fitted file, and msr2data's
|
||||
# global data export keys each run's convergence on that block. The merged
|
||||
# display MSR is not re-fitted, so carry the per-run statistics forward as
|
||||
# the ordinary aggregate musrfit would write for a global fit: summed chisq
|
||||
# and NDF, reduced chisq from those sums.
|
||||
my ($chisq_sum, $ndf_sum) = (0, 0);
|
||||
my $statistic_runs = 0;
|
||||
for my $pr (@per_run) {
|
||||
for my $line (@{$pr->{statistic}}) {
|
||||
if ($line =~ /chisq\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i) {
|
||||
my ($run_chisq, $run_ndf) = (0 + $1, 0 + $2);
|
||||
next if $run_ndf <= 0;
|
||||
$chisq_sum += $run_chisq;
|
||||
$ndf_sum += $run_ndf;
|
||||
$statistic_runs++;
|
||||
}
|
||||
}
|
||||
}
|
||||
if ($statistic_runs) {
|
||||
my ($sec, $min, $hour, $mday, $mon, $year) = localtime();
|
||||
my $timestamp = sprintf("%04d-%02d-%02d %02d:%02d:%02d",
|
||||
$year + 1900, $mon + 1, $mday, $hour, $min, $sec);
|
||||
my $reduced = $ndf_sum > 0 ? $chisq_sum / $ndf_sum : 0;
|
||||
push(@output, "###############################################################");
|
||||
push(@output, "STATISTIC --- " . $timestamp);
|
||||
my $formatted_chisq = sprintf("%.4f", $chisq_sum);
|
||||
my $formatted_reduced = sprintf("%.6f", $reduced);
|
||||
$formatted_chisq =~ s/\.?0+$//;
|
||||
$formatted_reduced =~ s/\.?0+$//;
|
||||
push(@output, sprintf(" chisq = %s, NDF = %d, chisq/NDF = %s",
|
||||
$formatted_chisq, $ndf_sum, $formatted_reduced));
|
||||
}
|
||||
return [1, "series-merge-ready", join("\n", @output) . "\n"];
|
||||
}
|
||||
# Display-MSR parsing and merging live in muSRFitlib.js and run in the browser.
|
||||
|
||||
# Collect a completed independent series without running musrfit again. The
|
||||
# requested non-empty data table and structurally valid combined MSR are the
|
||||
@@ -1473,16 +1028,21 @@ sub perform_series_artifact_collection {
|
||||
};
|
||||
};
|
||||
return $failure->("invalid-series-collection", "Invalid series collection request")
|
||||
if !defined($collection_text) || length($collection_text) > 100_000;
|
||||
if !defined($collection_text) || length($collection_text) > 5_000_000;
|
||||
my $collection = eval { decode_json($collection_text) };
|
||||
return $failure->("invalid-series-collection", "Invalid series collection JSON")
|
||||
if $@ || ref($collection) ne "HASH";
|
||||
my $runs = $collection->{runs};
|
||||
my $suffix = defined($collection->{suffix}) ? "$collection->{suffix}" : "";
|
||||
my $fftv = defined($collection->{fftv}) ? "$collection->{fftv}" : "n";
|
||||
my $contract_version = defined($collection->{version})
|
||||
&& !ref($collection->{version}) ? 0 + $collection->{version} : 0;
|
||||
my $client_combined = exists($collection->{combinedMSR})
|
||||
&& !ref($collection->{combinedMSR}) ? "$collection->{combinedMSR}" : undef;
|
||||
return $failure->("invalid-series-collection", "Invalid series collection fields")
|
||||
if ref($runs) ne "ARRAY" || @{$runs} < 2 || @{$runs} > 500
|
||||
|| $suffix !~ /^_[A-Za-z0-9._+\-]+$/ || $fftv !~ /^(?:y|n)$/;
|
||||
|| $suffix !~ /^_[A-Za-z0-9._+\-]+$/ || $fftv !~ /^(?:y|n)$/
|
||||
|| $contract_version != 1 || !defined($client_combined);
|
||||
|
||||
my @run_numbers = ();
|
||||
my %seen = ();
|
||||
@@ -1509,15 +1069,27 @@ sub perform_series_artifact_collection {
|
||||
# global-fit results that may already exist in the same HTTP job folder.
|
||||
my $combined_name = $base_run . "+series" . $suffix . ".msr";
|
||||
my $combined_path = "$work_path/$combined_name";
|
||||
# msr2data still needs a base+global<suffix>.msr staging input. A unique
|
||||
# internal suffix prevents collision with the normal `_tmpl` global path.
|
||||
my $template_suffix = "_musrflow_series_" . $$ . "_" . time();
|
||||
my $template_name = $base_run . "+global" . $template_suffix . ".msr";
|
||||
my $template_path = "$work_path/$template_name";
|
||||
my $parameter_name = $base_run . "+series" . $suffix . "_par.dat";
|
||||
my $parameter_path = "$work_path/$parameter_name";
|
||||
my $out_name = $base_run . "+series" . $suffix . ".out";
|
||||
my $out_path = "$work_path/$out_name";
|
||||
my $artifacts = $collection->{artifacts};
|
||||
return $failure->("invalid-series-collection", "Invalid series collection contract")
|
||||
if ref($artifacts) ne "HASH"
|
||||
|| ($artifacts->{combined} || "") ne $combined_name
|
||||
|| ($artifacts->{parameters} || "") ne $parameter_name
|
||||
|| ($artifacts->{log} || "") ne $out_name
|
||||
|| length($client_combined) > 4_000_000;
|
||||
|
||||
# Collection mutates the job's generated artifacts and starts external
|
||||
# programs. Serialize it with fitting/promotion operations for this job.
|
||||
my $lock_path = "$work_path/.musrflow-series.lock";
|
||||
return $failure->("series-collection-lock-failed", "Could not open series job lock")
|
||||
if !sysopen(my $lock_fh, $lock_path, O_RDWR | O_CREAT, 0600);
|
||||
if (!flock($lock_fh, LOCK_EX | LOCK_NB)) {
|
||||
close($lock_fh);
|
||||
return $failure->("series-collection-lock-failed", "Another series operation is active");
|
||||
}
|
||||
foreach my $stale_path ($combined_path, $parameter_path) {
|
||||
return $failure->("series-artifact-output-invalid", "A series output path is unsafe")
|
||||
if -l $stale_path || (-e $stale_path && !-f $stale_path);
|
||||
@@ -1534,55 +1106,59 @@ sub perform_series_artifact_collection {
|
||||
my $run_argument = "[" . join(" ", @run_numbers) . "]";
|
||||
my @source_texts = map { $source_texts{$_} } @run_numbers;
|
||||
|
||||
# Build the combined display MSR first. It becomes the canonical
|
||||
# `+global<suffix>.msr` file that both musrview and the parameter export
|
||||
# use, and is assembled directly from the already-fitted per-run files so
|
||||
# no global re-fit ever takes place.
|
||||
my $combined_url = "";
|
||||
my $merged = build_series_global_msr(\@source_texts, \@run_numbers);
|
||||
if (!$merged->[0]) {
|
||||
push(@warnings, "Could not assemble the combined display MSR ($merged->[1])");
|
||||
} elsif (!is_valid_generated_global_msr($merged->[2])) {
|
||||
push(@warnings, "Assembled combined display MSR failed validation");
|
||||
} elsif (write_text_file($combined_path, normalize_plot_runs_line(
|
||||
$merged->[2], join(" ", @run_numbers)
|
||||
))) {
|
||||
$combined_url = "$TMP_URL/$job_name/$combined_name";
|
||||
# Collect fitted per-run values directly. This is independent of display
|
||||
# merging, so models that cannot yet be represented in one display MSR
|
||||
# still produce the table consumed by Plot parameters.
|
||||
my $parameter_url = "";
|
||||
my $parameter_cmd = "cd " . shell_quote($work_path)
|
||||
. " && " . shell_quote($MSR2DATA_BIN)
|
||||
. " " . shell_quote($run_argument)
|
||||
. " " . shell_quote($suffix)
|
||||
. " new data -o " . shell_quote($parameter_name);
|
||||
my $parameter_result = run_logged_command(
|
||||
$out_path, "msr2data independent series collection", $parameter_cmd
|
||||
);
|
||||
if (-s $parameter_path) {
|
||||
if (!clean_parameter_data_header($parameter_path, \@run_numbers)) {
|
||||
push(@warnings, "Could not normalize the parameter-table header");
|
||||
}
|
||||
$parameter_url = "$TMP_URL/$job_name/$parameter_name";
|
||||
if ($parameter_result->{raw_status} != 0) {
|
||||
push(@warnings, "msr2data returned code " . $parameter_result->{exit_code}
|
||||
. " but produced the parameter table");
|
||||
}
|
||||
} else {
|
||||
push(@warnings, "Could not write the combined display MSR");
|
||||
push(@warnings, "msr2data did not produce parameter data");
|
||||
}
|
||||
|
||||
# msr2data is used only for the parameter table, exported with the same
|
||||
# `new global data` form and `_tmpl` staging as the ordinary multirun
|
||||
# export, producing the global .dat layout (stripped base names, one row
|
||||
# per run) that the parameter plot button reads.
|
||||
my $parameter_url = "";
|
||||
my $combined_text = $combined_url ne "" ? read_text_file($combined_path) : "";
|
||||
if ($combined_text ne "" && write_text_file($template_path, $combined_text)) {
|
||||
my $parameter_cmd = "cd " . shell_quote($work_path)
|
||||
. " && " . shell_quote($MSR2DATA_BIN)
|
||||
. " " . shell_quote($run_argument)
|
||||
. " " . shell_quote($template_suffix)
|
||||
. " new global data -o " . shell_quote($parameter_name);
|
||||
my $parameter_result = run_logged_command(
|
||||
$out_path, "msr2data independent series collection", $parameter_cmd
|
||||
);
|
||||
if (-s $parameter_path) {
|
||||
if (!clean_parameter_data_header($parameter_path)) {
|
||||
push(@warnings, "Could not normalize the parameter-table header");
|
||||
}
|
||||
# Keep the non-empty file available even when header cleanup fails.
|
||||
$parameter_url = "$TMP_URL/$job_name/$parameter_name";
|
||||
if ($parameter_result->{raw_status} != 0) {
|
||||
push(@warnings, "msr2data returned code " . $parameter_result->{exit_code}
|
||||
. " but produced the parameter table");
|
||||
}
|
||||
# The browser carries the display MSR produced by the shared JavaScript
|
||||
# merger. Verify that it retains exactly the fitted sources' RUN/ADDRUN
|
||||
# identities before publishing or passing it to musrview.
|
||||
my $combined_url = "";
|
||||
my $combined_text = $client_combined || "";
|
||||
if ($combined_text ne "") {
|
||||
my @expected_directives = map {
|
||||
@{series_run_directives($_)}
|
||||
} @source_texts;
|
||||
my @actual_directives = @{series_run_directives($combined_text)};
|
||||
if (!is_valid_generated_global_msr($combined_text)) {
|
||||
push(@warnings, "Submitted combined display MSR failed validation");
|
||||
$combined_text = "";
|
||||
} elsif (join("\n", @actual_directives) ne join("\n", @expected_directives)) {
|
||||
push(@warnings, "Submitted combined display MSR changed RUN identity");
|
||||
$combined_text = "";
|
||||
}
|
||||
}
|
||||
if ($combined_text ne "") {
|
||||
my $combined_stage = "$work_path/.musrflow-series-display-$$.tmp";
|
||||
if (!-e $combined_stage
|
||||
&& write_text_file($combined_stage, $combined_text)
|
||||
&& rename($combined_stage, $combined_path)) {
|
||||
$combined_url = "$TMP_URL/$job_name/$combined_name";
|
||||
} else {
|
||||
push(@warnings, "msr2data did not produce parameter data");
|
||||
unlink($combined_stage) if -f $combined_stage && !-l $combined_stage;
|
||||
push(@warnings, "Could not write the combined display MSR");
|
||||
}
|
||||
unlink($template_path);
|
||||
} elsif ($combined_url ne "") {
|
||||
push(@warnings, "Could not stage the combined MSR for parameter export");
|
||||
}
|
||||
|
||||
my @root_urls = ();
|
||||
@@ -1627,6 +1203,7 @@ sub perform_series_artifact_collection {
|
||||
|
||||
my $command_log = read_text_file($out_path);
|
||||
my $usable = $parameter_url ne "" || $combined_url ne "";
|
||||
close($lock_fh);
|
||||
return {
|
||||
ok => $usable ? JSON::PP::true : JSON::PP::false,
|
||||
code => $usable ? 0 : 1,
|
||||
|
||||
+66
-89
@@ -723,20 +723,31 @@ function appendFitLog(logFile, text) {
|
||||
}
|
||||
}
|
||||
|
||||
// msr2data represents a run-qualified parameter such as Asy_2274 as Asy_ in
|
||||
// the exported data header. Remove exactly one trailing underscore from header
|
||||
// fields so downstream plotting sees the original logical parameter name.
|
||||
// Underscores inside names (for example my_parameter) and error labels such as
|
||||
// Asy_PosErr are left untouched.
|
||||
function cleanMusrfitParameterDataHeader(parameterPath) {
|
||||
// Global msr2data exports represent a run-qualified parameter such as
|
||||
// Asy_2274 as Asy_, while ordinary per-run exports retain the first run suffix
|
||||
// as Asy_2274, Asy_2274PosErr, and Asy_2274NegErr. Normalize only suffixes that
|
||||
// match this collection's run list so downstream plotting sees the logical
|
||||
// parameter name. Embedded underscores in names remain untouched.
|
||||
function cleanMusrfitParameterDataHeader(parameterPath, runNumbers = []) {
|
||||
let contents = nodeFs.readFileSync(parameterPath, "utf8");
|
||||
let newline = contents.includes("\r\n") ? "\r\n" : "\n";
|
||||
let lines = contents.split(/\r?\n/);
|
||||
let runs = new Set(runNumbers.map(function(run) {
|
||||
return String(Number(run));
|
||||
}).filter(function(run) {
|
||||
return /^\d+$/.test(run);
|
||||
}));
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
let probe = lines[i].trim();
|
||||
if (!probe || probe.startsWith("#")) continue;
|
||||
lines[i] = lines[i].replace(/(\S+)_(?=\s|$)/g, "$1");
|
||||
lines[i] = lines[i].replace(/\S+/g, function(column) {
|
||||
let qualified = /^(.*)_([0-9]+)((?:_)?(?:PosErr|NegErr))?$/.exec(column);
|
||||
if (qualified && runs.has(String(Number(qualified[2])))) {
|
||||
return qualified[1] + (qualified[3] || "");
|
||||
}
|
||||
return column.endsWith("_") ? column.slice(0, -1) : column;
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -975,30 +986,17 @@ window.runSeriesMSRCandidateLocal = runSeriesMSRCandidateLocal;
|
||||
// +global file from a template). A non-empty requested .dat is registered
|
||||
// immediately, and any structurally valid combined MSR is safe for musrview.
|
||||
async function collectIndependentSeriesArtifactsLocal(spec) {
|
||||
let result = {
|
||||
ok: false,
|
||||
reason: "invalid-series-artifact-spec",
|
||||
parameterDataPath: "",
|
||||
combinedMSRPath: "",
|
||||
logPath: "",
|
||||
plotStarted: false,
|
||||
warnings: []
|
||||
};
|
||||
if (!spec || spec.ok !== true || !Array.isArray(spec.runs)
|
||||
|| spec.runs.length < 2 || !Array.isArray(spec.msrPaths)
|
||||
|| spec.msrPaths.length !== spec.runs.length
|
||||
|| !/^_[A-Za-z0-9._+\-]+$/.test(spec.fileNameSuffix || "")) {
|
||||
return result;
|
||||
}
|
||||
let prepared = prepareIndependentSeriesCollection(spec);
|
||||
let result = normalizeIndependentSeriesCollectionResult({ reason: prepared.reason });
|
||||
if (!prepared.ok) return result;
|
||||
|
||||
let workPath = nodePath.resolve(spec.workPath);
|
||||
let suffix = spec.fileNameSuffix;
|
||||
let baseRun = spec.runs[0];
|
||||
for (let index = 0; index < spec.runs.length; index++) {
|
||||
if (!/^\d+$/.test(spec.runs[index])
|
||||
|| nodePath.resolve(spec.msrPaths[index])
|
||||
!== nodePath.join(workPath, spec.runs[index] + suffix + ".msr")
|
||||
|| !isSafeSeriesRegularFile(spec.msrPaths[index])) {
|
||||
let workPath = nodePath.resolve(prepared.workPath);
|
||||
let suffix = prepared.fileNameSuffix;
|
||||
let baseRun = prepared.baseRun;
|
||||
for (let index = 0; index < prepared.runs.length; index++) {
|
||||
if (nodePath.resolve(prepared.msrPaths[index])
|
||||
!== nodePath.join(workPath, prepared.runs[index] + suffix + ".msr")
|
||||
|| !isSafeSeriesRegularFile(prepared.msrPaths[index])) {
|
||||
result.reason = "series-artifact-source-invalid";
|
||||
return result;
|
||||
}
|
||||
@@ -1007,18 +1005,11 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
|
||||
// Independent-series artifacts use their own namespace. A genuine global
|
||||
// fit can legitimately exist beside the same per-run files and must never
|
||||
// be removed or replaced by display-only post-processing.
|
||||
let combinedName = baseRun + "+series" + suffix + ".msr";
|
||||
let combinedName = prepared.combinedName;
|
||||
let combinedPath = nodePath.join(workPath, combinedName);
|
||||
// msr2data global-data mode requires a `base+global<suffix>.msr` input.
|
||||
// Give the staging file a process-unique suffix so an ordinary `_tmpl`
|
||||
// global-fit template is neither overwritten nor deleted.
|
||||
let templateSuffix = "_musrflow_series_" + process.pid + "_"
|
||||
+ Date.now().toString(36);
|
||||
let templateName = baseRun + "+global" + templateSuffix + ".msr";
|
||||
let templatePath = nodePath.join(workPath, templateName);
|
||||
let parameterName = baseRun + "+series" + suffix + "_par.dat";
|
||||
let parameterName = prepared.parameterName;
|
||||
let parameterPath = nodePath.join(workPath, parameterName);
|
||||
let logName = baseRun + "+series" + suffix + ".out";
|
||||
let logName = prepared.logName;
|
||||
let logPath = nodePath.join(workPath, logName);
|
||||
result.logPath = logPath;
|
||||
|
||||
@@ -1035,7 +1026,7 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
|
||||
"# muSRFlow independent series collection\n# Working directory: "
|
||||
+ workPath + "\n", { encoding: "utf8", flag: "w" });
|
||||
|
||||
let runsArg = "[" + spec.runs.join(" ") + "]";
|
||||
let runsArg = "[" + prepared.runs.join(" ") + "]";
|
||||
let msr2data = resolveLocalCommand("msr2data");
|
||||
let env = Object.assign({}, process.env);
|
||||
env.PATH = ["/usr/local/bin", "/usr/bin", "/bin", env.PATH || ""].join(":");
|
||||
@@ -1063,64 +1054,53 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
|
||||
return code;
|
||||
};
|
||||
|
||||
// Assemble the combined display MSR first. It becomes the canonical
|
||||
// `+global<suffix>.msr` file that both musrview and the parameter export
|
||||
// use, and is built directly from the already-fitted per-run files so no
|
||||
// global re-fit ever takes place.
|
||||
// Parameter collection is the primary independent-series result. Use
|
||||
// msr2data's ordinary per-run data mode—the same operation as the
|
||||
// established rplcall/msr2data workflow—so an unsupported display merge
|
||||
// can never prevent Plot parameters from receiving a usable table.
|
||||
let parameterCode = runMsr2data("parameter msr2data",
|
||||
[runsArg, suffix, "new", "data", "-o", parameterName]);
|
||||
if (isSafeSeriesRegularFile(parameterPath)
|
||||
&& nodeFs.statSync(parameterPath).size > 0) {
|
||||
try {
|
||||
cleanMusrfitParameterDataHeader(parameterPath, spec.runs);
|
||||
} catch (error) {
|
||||
result.warnings.push("Could not normalize the parameter-table header.");
|
||||
}
|
||||
result.parameterDataPath = parameterPath;
|
||||
if (parameterCode !== 0) {
|
||||
result.warnings.push("msr2data returned code " + parameterCode
|
||||
+ " but produced the parameter table.");
|
||||
}
|
||||
} else {
|
||||
result.warnings.push("msr2data did not produce parameter data.");
|
||||
}
|
||||
|
||||
// Assemble the combined display MSR independently from the already
|
||||
// fitted per-run files. No global re-fit takes place.
|
||||
let msrTexts = [];
|
||||
for (let index = 0; index < spec.msrPaths.length; index++) {
|
||||
if (!isSafeSeriesRegularFile(spec.msrPaths[index])) {
|
||||
result.warnings.push("Fitted MSR " + spec.msrPaths[index]
|
||||
for (let index = 0; index < prepared.msrPaths.length; index++) {
|
||||
if (!isSafeSeriesRegularFile(prepared.msrPaths[index])) {
|
||||
result.warnings.push("Fitted MSR " + prepared.msrPaths[index]
|
||||
+ " is missing for the combined display.");
|
||||
break;
|
||||
}
|
||||
msrTexts.push(nodeFs.readFileSync(spec.msrPaths[index], "utf8"));
|
||||
msrTexts.push(nodeFs.readFileSync(prepared.msrPaths[index], "utf8"));
|
||||
}
|
||||
|
||||
if (msrTexts.length === spec.msrPaths.length) {
|
||||
let merged = buildSeriesGlobalMSR(msrTexts, spec.runs);
|
||||
if (msrTexts.length === prepared.msrPaths.length) {
|
||||
let merged = buildSeriesGlobalMSR(msrTexts, prepared.runs);
|
||||
if (merged.ok !== true) {
|
||||
result.warnings.push("Could not assemble the combined display MSR ("
|
||||
+ merged.reason + ").");
|
||||
} else if (!isValidGeneratedGlobalMSR(merged.text)) {
|
||||
result.warnings.push("Assembled combined display MSR failed validation.");
|
||||
} else {
|
||||
let combinedText = normalizeMSRPlotRuns(merged.text, {}, spec.runs);
|
||||
let combinedText = normalizeMSRPlotRuns(merged.text, {}, prepared.runs);
|
||||
nodeFs.writeFileSync(combinedPath, combinedText, {
|
||||
encoding: "utf8", flag: "wx"
|
||||
});
|
||||
result.combinedMSRPath = combinedPath;
|
||||
|
||||
// msr2data is used only for the parameter table, exported with the
|
||||
// same `new global data` form and `_tmpl` staging as the ordinary
|
||||
// multirun export, producing the global .dat layout (stripped base
|
||||
// names, one row per run) that the parameter plot button reads.
|
||||
nodeFs.copyFileSync(combinedPath, templatePath, nodeFs.constants.COPYFILE_EXCL);
|
||||
try {
|
||||
let parameterCode = runMsr2data("parameter msr2data",
|
||||
[runsArg, templateSuffix, "new", "global", "data", "-o", parameterName]);
|
||||
if (isSafeSeriesRegularFile(parameterPath)
|
||||
&& nodeFs.statSync(parameterPath).size > 0) {
|
||||
try {
|
||||
cleanMusrfitParameterDataHeader(parameterPath);
|
||||
} catch (error) {
|
||||
result.warnings.push("Could not normalize the parameter-table header.");
|
||||
}
|
||||
// A non-empty table is usable even if header cleanup or the
|
||||
// external program's status is imperfect.
|
||||
result.parameterDataPath = parameterPath;
|
||||
if (parameterCode !== 0) {
|
||||
result.warnings.push("msr2data returned code " + parameterCode
|
||||
+ " but produced the parameter table.");
|
||||
}
|
||||
} else {
|
||||
result.warnings.push("msr2data did not produce parameter data.");
|
||||
}
|
||||
} finally {
|
||||
if (isSafeSeriesRegularFile(templatePath)) {
|
||||
nodeFs.unlinkSync(templatePath);
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
result.warnings.push("Could not read all fitted per-run MSR files"
|
||||
@@ -1129,7 +1109,7 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
|
||||
|
||||
if (result.combinedMSRPath) {
|
||||
let musrviewArgs = [];
|
||||
if (spec.fftv == "y") musrviewArgs.push("-f");
|
||||
if (prepared.fftv == "y") musrviewArgs.push("-f");
|
||||
musrviewArgs.push(combinedName);
|
||||
let viewId = window.MusrfitRunner.nextViewId++;
|
||||
runLocalProcess("musrview series #" + viewId,
|
||||
@@ -1137,15 +1117,12 @@ async function collectIndependentSeriesArtifactsLocal(spec) {
|
||||
{ reportStatus: false });
|
||||
result.plotStarted = true;
|
||||
}
|
||||
result.ok = Boolean(result.parameterDataPath || result.combinedMSRPath);
|
||||
result.reason = result.ok
|
||||
? "series-artifacts-collected" : "series-artifacts-not-produced";
|
||||
return result;
|
||||
return normalizeIndependentSeriesCollectionResult(result);
|
||||
} catch (error) {
|
||||
result.reason = "series-artifact-collection-failed";
|
||||
result.warnings.push(error && error.message
|
||||
? error.message : "Series post-processing failed.");
|
||||
return result;
|
||||
return normalizeIndependentSeriesCollectionResult(result);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+153
-34
@@ -4608,14 +4608,18 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
|
||||
let nextSlot = 1;
|
||||
while (occupiedSlots.has(nextSlot)) nextSlot++;
|
||||
let slotRoleName = Object.create(null);
|
||||
let slotPosition = Object.create(null); // slot -> { lineIndex, tokenIndex }
|
||||
let slotPosition = Object.create(null); // slot -> { section, lineIndex, tokenIndex }
|
||||
|
||||
function slotForParam(name, lineIndex, tokenIndex) {
|
||||
function slotForParam(name, section, lineIndex, tokenIndex) {
|
||||
if (slotRoleName[name] !== undefined) return slotRoleName[name];
|
||||
if (nextSlot > SERIES_MSR_MAX_MAP_SLOTS) return 0;
|
||||
let slot = nextSlot++;
|
||||
slotRoleName[name] = slot;
|
||||
slotPosition[slot] = { lineIndex: lineIndex, tokenIndex: tokenIndex };
|
||||
slotPosition[slot] = {
|
||||
section: section,
|
||||
lineIndex: lineIndex,
|
||||
tokenIndex: tokenIndex
|
||||
};
|
||||
return slot;
|
||||
}
|
||||
|
||||
@@ -4631,7 +4635,7 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
|
||||
if (nameCounts[p.name] === numRuns) {
|
||||
out.push(String(sharedNameNo[p.name]));
|
||||
} else {
|
||||
let slot = slotForParam(p.name, lineIndex, tokenIndex);
|
||||
let slot = slotForParam(p.name, "theory", lineIndex, tokenIndex);
|
||||
if (slot === 0) return fail("series-merge-map-slots-exhausted");
|
||||
out.push("map" + slot);
|
||||
}
|
||||
@@ -4646,25 +4650,29 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
|
||||
if (line && typeof line == "object" && line.ok === false) return fail(line.reason);
|
||||
}
|
||||
|
||||
// FUNCTIONS is shared by all runs, so it may only reference parameters that
|
||||
// keep the same number across every RUN block. a function that multiplies
|
||||
// two run-specific parameters would otherwise force a different curve per
|
||||
// run block, which the global file cannot express.
|
||||
let mergedFunctions = first.functions.map(function(line) {
|
||||
// FUNCTIONS can select run-specific parameters through mapN just like
|
||||
// THEORY. Allocate a RUN map slot for each such parN reference; genuinely
|
||||
// shared parameters keep a direct parN reference.
|
||||
let mergedFunctions = first.functions.map(function(line, lineIndex) {
|
||||
let out = [];
|
||||
let tokenIndex = 0;
|
||||
for (let token of line.split(/\s+/)) {
|
||||
let parMatch = token.match(/^par(\d+)$/i);
|
||||
if (parMatch) {
|
||||
let localNo = parseInt(parMatch[1], 10);
|
||||
let p = byNumber[0][localNo];
|
||||
if (!p) return fail("series-merge-functions-param-missing");
|
||||
if (nameCounts[p.name] !== numRuns) {
|
||||
return fail("series-merge-functions-run-specific");
|
||||
if (nameCounts[p.name] === numRuns) {
|
||||
out.push("par" + sharedNameNo[p.name]);
|
||||
} else {
|
||||
let slot = slotForParam(p.name, "functions", lineIndex, tokenIndex);
|
||||
if (slot === 0) return fail("series-merge-map-slots-exhausted");
|
||||
out.push("map" + slot);
|
||||
}
|
||||
out.push("par" + sharedNameNo[p.name]);
|
||||
} else {
|
||||
out.push(token);
|
||||
}
|
||||
tokenIndex++;
|
||||
}
|
||||
return out.join(" ");
|
||||
});
|
||||
@@ -4693,17 +4701,24 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
|
||||
let slotMax = Math.max(maxSlot, existing ? existing.length : 0);
|
||||
for (let slot = 1; slot <= slotMax; slot++) {
|
||||
let value = 0;
|
||||
if (existing && existing[slot - 1] && /^\d+$/.test(existing[slot - 1])) {
|
||||
if (existing && existing[slot - 1] && /^\d+$/.test(existing[slot - 1])
|
||||
&& parseInt(existing[slot - 1], 10) > 0) {
|
||||
let localNo = parseInt(existing[slot - 1], 10);
|
||||
value = (localNo > 0 && noMap[runIndex][localNo] !== undefined)
|
||||
? noMap[runIndex][localNo] : localNo;
|
||||
} else if (slotPosition[slot]) {
|
||||
let position = slotPosition[slot];
|
||||
let localTokens = perRun[runIndex].theory[position.lineIndex]
|
||||
? perRun[runIndex].theory[position.lineIndex].split(/\s+/)
|
||||
let sourceLines = position.section === "functions"
|
||||
? perRun[runIndex].functions : perRun[runIndex].theory;
|
||||
let localTokens = sourceLines[position.lineIndex]
|
||||
? sourceLines[position.lineIndex].split(/\s+/)
|
||||
: [];
|
||||
let localToken = localTokens[position.tokenIndex];
|
||||
let localNo = /^\d+$/.test(localToken || "") ? parseInt(localToken, 10) : 0;
|
||||
let functionPar = position.section === "functions"
|
||||
? String(localToken || "").match(/^par(\d+)$/i) : null;
|
||||
let localNo = functionPar
|
||||
? parseInt(functionPar[1], 10)
|
||||
: (/^\d+$/.test(localToken || "") ? parseInt(localToken, 10) : 0);
|
||||
if (localNo > 0 && noMap[runIndex][localNo] !== undefined) {
|
||||
value = noMap[runIndex][localNo];
|
||||
}
|
||||
@@ -4802,26 +4817,32 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
|
||||
for (let i = 1; i <= numRuns; i++) plotRuns.push(String(i));
|
||||
output.push("runs " + plotRuns.join(" "));
|
||||
|
||||
// musrfit always stamps a STATISTIC block into a fitted file, and msr2data's
|
||||
// global data export keys each run's convergence on that block. The merged
|
||||
// display MSR is not re-fitted, so carry the per-run statistics forward as
|
||||
// the ordinary aggregate musrfit would write for a global fit: summed chisq
|
||||
// and NDF, reduced chisq from those sums.
|
||||
// The display MSR is not re-fitted. When every source has the same objective,
|
||||
// report the sum of its independently fitted statistic and NDF. Never emit a
|
||||
// partial or mixed-objective aggregate.
|
||||
let statistics = [];
|
||||
let statisticKind = "";
|
||||
for (let pr of perRun) {
|
||||
let match = pr.statistic.match(
|
||||
/chisq\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i
|
||||
/(chisq|maxLH)\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i
|
||||
);
|
||||
if (!match) continue;
|
||||
let runChiSq = parseFloat(match[1]);
|
||||
let runNDF = parseFloat(match[2]);
|
||||
if (isNaN(runChiSq) || isNaN(runNDF) || runNDF <= 0) continue;
|
||||
statistics.push({ chiSq: runChiSq, ndf: runNDF });
|
||||
let kind = match[1].toLowerCase() === "maxlh" ? "maxLH" : "chisq";
|
||||
let runValue = parseFloat(match[2]);
|
||||
let runNDF = parseFloat(match[3]);
|
||||
if (isNaN(runValue) || isNaN(runNDF) || runNDF <= 0) continue;
|
||||
if (statisticKind && statisticKind !== kind) {
|
||||
statistics = [];
|
||||
statisticKind = "";
|
||||
break;
|
||||
}
|
||||
statisticKind = kind;
|
||||
statistics.push({ value: runValue, ndf: runNDF });
|
||||
}
|
||||
if (statistics.length) {
|
||||
let chiSq = 0;
|
||||
if (statistics.length === numRuns) {
|
||||
let total = 0;
|
||||
let ndf = 0;
|
||||
for (let s of statistics) { chiSq += s.chiSq; ndf += s.ndf; }
|
||||
for (let s of statistics) { total += s.value; ndf += s.ndf; }
|
||||
let date = new Date();
|
||||
let timestamp = date.getFullYear() + "-"
|
||||
+ String(date.getMonth() + 1).padStart(2, "0") + "-"
|
||||
@@ -4829,13 +4850,13 @@ function buildSeriesGlobalMSR(msrTexts, runs) {
|
||||
+ String(date.getHours()).padStart(2, "0") + ":"
|
||||
+ String(date.getMinutes()).padStart(2, "0") + ":"
|
||||
+ String(date.getSeconds()).padStart(2, "0");
|
||||
let formattedChiSq = parseFloat(chiSq.toFixed(4)).toString();
|
||||
let formattedTotal = parseFloat(total.toFixed(4)).toString();
|
||||
let formattedNDF = Math.round(ndf).toString();
|
||||
let formattedReduced = parseFloat((chiSq / ndf).toFixed(6)).toString();
|
||||
let formattedReduced = parseFloat((total / ndf).toFixed(6)).toString();
|
||||
output.push("###############################################################");
|
||||
output.push("STATISTIC --- " + timestamp);
|
||||
output.push(" chisq = " + formattedChiSq + ", NDF = " + formattedNDF
|
||||
+ ", chisq/NDF = " + formattedReduced);
|
||||
output.push(" " + statisticKind + " = " + formattedTotal + ", NDF = "
|
||||
+ formattedNDF + ", " + statisticKind + "/NDF = " + formattedReduced);
|
||||
}
|
||||
|
||||
return {
|
||||
@@ -9469,6 +9490,104 @@ function buildIndependentSeriesArtifactSpec(plan, config = {}) {
|
||||
return result;
|
||||
}
|
||||
|
||||
// Normalize the platform-neutral part of a completed-series collection. Local
|
||||
// filesystem confinement and web URL/job checks remain adapter responsibilities,
|
||||
// but artifact names and the data exchanged with the CGI must be identical.
|
||||
function prepareIndependentSeriesCollection(spec) {
|
||||
let invalid = function(reason) {
|
||||
return {
|
||||
ok: false,
|
||||
reason: reason || "invalid-series-artifact-spec",
|
||||
workPath: "",
|
||||
fileNameSuffix: "",
|
||||
runs: [],
|
||||
msrPaths: [],
|
||||
fftv: "n",
|
||||
baseRun: "",
|
||||
combinedName: "",
|
||||
parameterName: "",
|
||||
logName: ""
|
||||
};
|
||||
};
|
||||
if (!spec || spec.ok !== true || !Array.isArray(spec.runs)
|
||||
|| spec.runs.length < 2 || !Array.isArray(spec.msrPaths)
|
||||
|| spec.msrPaths.length !== spec.runs.length
|
||||
|| typeof spec.workPath !== "string" || spec.workPath === ""
|
||||
|| !/^_[A-Za-z0-9._+\-]+$/.test(spec.fileNameSuffix || "")) {
|
||||
return invalid("invalid-series-artifact-spec");
|
||||
}
|
||||
let runs = spec.runs.map(function(run) { return String(run); });
|
||||
if (runs.some(function(run) { return !/^\d+$/.test(run) || Number(run) <= 0; })
|
||||
|| new Set(runs).size !== runs.length
|
||||
|| spec.msrPaths.some(function(path) {
|
||||
return typeof path !== "string" || path === "";
|
||||
})) {
|
||||
return invalid("series-artifact-source-invalid");
|
||||
}
|
||||
let baseRun = runs[0];
|
||||
let suffix = spec.fileNameSuffix;
|
||||
return {
|
||||
ok: true,
|
||||
reason: "series-collection-request-ready",
|
||||
workPath: spec.workPath,
|
||||
fileNameSuffix: suffix,
|
||||
runs: runs,
|
||||
msrPaths: spec.msrPaths.slice(),
|
||||
fftv: spec.fftv == "y" ? "y" : "n",
|
||||
baseRun: baseRun,
|
||||
combinedName: baseRun + "+series" + suffix + ".msr",
|
||||
parameterName: baseRun + "+series" + suffix + "_par.dat",
|
||||
logName: baseRun + "+series" + suffix + ".out"
|
||||
};
|
||||
}
|
||||
|
||||
function buildIndependentSeriesCollectionRequest(spec, combinedMSR) {
|
||||
let prepared = prepareIndependentSeriesCollection(spec);
|
||||
if (!prepared.ok) return prepared;
|
||||
return {
|
||||
ok: true,
|
||||
version: 1,
|
||||
runs: prepared.runs.slice(),
|
||||
suffix: prepared.fileNameSuffix,
|
||||
fftv: prepared.fftv,
|
||||
artifacts: {
|
||||
combined: prepared.combinedName,
|
||||
parameters: prepared.parameterName,
|
||||
log: prepared.logName
|
||||
},
|
||||
combinedMSR: typeof combinedMSR === "string" ? combinedMSR : ""
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeIndependentSeriesCollectionResult(value) {
|
||||
value = value && typeof value === "object" ? value : {};
|
||||
let result = {
|
||||
ok: false,
|
||||
reason: typeof value.reason === "string" && value.reason
|
||||
? value.reason : "series-artifacts-not-produced",
|
||||
parameterDataPath: typeof value.parameterDataPath === "string"
|
||||
? value.parameterDataPath : "",
|
||||
combinedMSRPath: typeof value.combinedMSRPath === "string"
|
||||
? value.combinedMSRPath : "",
|
||||
logPath: typeof value.logPath === "string" ? value.logPath : "",
|
||||
plotPath: typeof value.plotPath === "string" ? value.plotPath : "",
|
||||
plotPaths: Array.isArray(value.plotPaths)
|
||||
? value.plotPaths.filter(function(path) {
|
||||
return typeof path === "string" && path !== "";
|
||||
}) : [],
|
||||
plotStarted: value.plotStarted === true,
|
||||
warnings: Array.isArray(value.warnings)
|
||||
? value.warnings.filter(function(warning) {
|
||||
return typeof warning === "string" && warning !== "";
|
||||
}) : []
|
||||
};
|
||||
result.ok = Boolean(result.parameterDataPath || result.combinedMSRPath);
|
||||
if (result.ok && (!value.reason || value.reason == "series-artifacts-not-produced")) {
|
||||
result.reason = "series-artifacts-collected";
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async function collectIndependentSeriesArtifacts(plan, config, webMode) {
|
||||
let spec = buildIndependentSeriesArtifactSpec(plan, config);
|
||||
if (!spec.ok) {
|
||||
@@ -9503,7 +9622,7 @@ async function collectIndependentSeriesArtifacts(plan, config, webMode) {
|
||||
warnings: [error && error.message ? error.message : "Series collection failed."]
|
||||
};
|
||||
}
|
||||
artifacts = artifacts && typeof artifacts === "object" ? artifacts : {};
|
||||
artifacts = normalizeIndependentSeriesCollectionResult(artifacts);
|
||||
let status = {};
|
||||
if (artifacts.parameterDataPath) status.parameterDataPath = artifacts.parameterDataPath;
|
||||
if (artifacts.combinedMSRPath) status.resultMSR = artifacts.combinedMSRPath;
|
||||
|
||||
+39
-31
@@ -753,40 +753,48 @@ window.runSeriesMSRCandidateWeb = runSeriesMSRCandidateWeb;
|
||||
// never invokes musrfit; it only creates the parameter table, a combined display
|
||||
// MSR, and the ROOT canvas used by the browser viewer.
|
||||
async function collectIndependentSeriesArtifactsWeb(spec) {
|
||||
let result = {
|
||||
ok: false,
|
||||
reason: "invalid-series-artifact-spec",
|
||||
parameterDataPath: "",
|
||||
combinedMSRPath: "",
|
||||
logPath: "",
|
||||
plotPath: "",
|
||||
plotStarted: false,
|
||||
warnings: []
|
||||
};
|
||||
if (!spec || spec.ok !== true || !Array.isArray(spec.runs)
|
||||
|| spec.runs.length < 2 || !Array.isArray(spec.msrPaths)
|
||||
|| spec.msrPaths.length !== spec.runs.length
|
||||
|| !/^_[A-Za-z0-9._+\-]+$/.test(spec.fileNameSuffix || "")) {
|
||||
return result;
|
||||
}
|
||||
let workUrl = String(spec.workPath || "").replace(/\/+$/, "");
|
||||
for (let index = 0; index < spec.runs.length; index++) {
|
||||
if (!/^\d+$/.test(spec.runs[index])
|
||||
|| spec.msrPaths[index]
|
||||
!== workUrl + "/" + spec.runs[index] + spec.fileNameSuffix + ".msr") {
|
||||
let prepared = prepareIndependentSeriesCollection(spec);
|
||||
let result = normalizeIndependentSeriesCollectionResult({ reason: prepared.reason });
|
||||
if (!prepared.ok) return result;
|
||||
let workUrl = prepared.workPath.replace(/\/+$/, "");
|
||||
for (let index = 0; index < prepared.runs.length; index++) {
|
||||
if (prepared.msrPaths[index]
|
||||
!== workUrl + "/" + prepared.runs[index] + prepared.fileNameSuffix + ".msr") {
|
||||
result.reason = "series-artifact-source-invalid";
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
// Generated job files are HTTP-visible, so the browser can perform the
|
||||
// pure merge itself using the same implementation as Electron. Failure to
|
||||
// build a display file does not block the server-side parameter collection.
|
||||
let clientWarnings = [];
|
||||
let combinedMSR = "";
|
||||
let sourceTexts = await Promise.all(prepared.msrPaths.map(function(path) {
|
||||
return readAsciiFile(path);
|
||||
}));
|
||||
if (sourceTexts.every(function(text) {
|
||||
return typeof text === "string" && text !== "";
|
||||
})) {
|
||||
let merged = buildSeriesGlobalMSR(sourceTexts, prepared.runs);
|
||||
if (merged.ok && isValidGeneratedGlobalMSR(merged.text)) {
|
||||
combinedMSR = normalizeMSRPlotRuns(merged.text, {}, prepared.runs);
|
||||
} else {
|
||||
clientWarnings.push("Could not assemble the combined display MSR ("
|
||||
+ (merged.reason || "series-merge-invalid") + ").");
|
||||
}
|
||||
} else {
|
||||
clientWarnings.push("Could not fetch all fitted per-run MSR files"
|
||||
+ " for the combined display.");
|
||||
}
|
||||
let collectionRequest = buildIndependentSeriesCollectionRequest(
|
||||
prepared, combinedMSR
|
||||
);
|
||||
|
||||
let request = await postToMusrfitCGI({
|
||||
action: "series_collect",
|
||||
fn: spec.msrPaths[0],
|
||||
collection: JSON.stringify({
|
||||
runs: spec.runs,
|
||||
suffix: spec.fileNameSuffix,
|
||||
fftv: spec.fftv == "y" ? "y" : "n"
|
||||
})
|
||||
fn: prepared.msrPaths[0],
|
||||
collection: JSON.stringify(collectionRequest)
|
||||
});
|
||||
if (!request.ok || !request.payload) {
|
||||
result.reason = "web-series-artifact-request-failed";
|
||||
@@ -802,8 +810,8 @@ async function collectIndependentSeriesArtifactsWeb(spec) {
|
||||
result.parameterDataPath = response.parameter_url || "";
|
||||
result.combinedMSRPath = response.msr_url || "";
|
||||
result.logPath = response.out_url || "";
|
||||
result.warnings = Array.isArray(response.warnings)
|
||||
? response.warnings.filter(Boolean) : [];
|
||||
result.warnings = clientWarnings.concat(Array.isArray(response.warnings)
|
||||
? response.warnings.filter(Boolean) : []);
|
||||
if (response.command_log && typeof appendJobLog === "function") {
|
||||
appendJobLog(response.command_log.trim());
|
||||
}
|
||||
@@ -815,13 +823,13 @@ async function collectIndependentSeriesArtifactsWeb(spec) {
|
||||
if (typeof showWebRootPlotWindow === "function") {
|
||||
showWebRootPlotWindow(rootUrls, response.job_name || "", {
|
||||
msrPath: result.combinedMSRPath,
|
||||
runs: spec.runs.join(" "),
|
||||
runs: prepared.runs.join(" "),
|
||||
rootItems: Array.isArray(response.root_items) ? response.root_items : []
|
||||
});
|
||||
result.plotStarted = true;
|
||||
}
|
||||
}
|
||||
return result;
|
||||
return normalizeIndependentSeriesCollectionResult(result);
|
||||
}
|
||||
|
||||
window.collectIndependentSeriesArtifactsWeb = collectIndependentSeriesArtifactsWeb;
|
||||
|
||||
+1
-1
@@ -10,7 +10,7 @@
|
||||
"dist:linux": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux rpm AppImage --x64",
|
||||
"dist:rpm": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux rpm --x64",
|
||||
"dist:appimage": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux AppImage --x64",
|
||||
"test:electron": "node tests/log-highlighting.test.js && node tests/action-state.test.js && node tests/electron-missing-data.test.js && node tests/window-layering.test.js && node tests/msr-generation.test.js && node tests/web-runner.test.js && node tests/root-plot-clean.test.js && node tests/parameter-plot.test.js && node tests/help-window.test.js && node tests/release-packaging.test.js && node tests/addrun.test.js && node tests/fit-statistic.test.js && node tests/series-fit-planner.test.js && node tests/series-fit-result.test.js && node tests/series-fit-inventory.test.js && node tests/series-fit-msr-preparation.test.js && node tests/series-global-merge.test.js && node tests/series-fit-staging.test.js && node tests/series-fit-runner.test.js && node tests/series-runner-adapters.test.js && node tests/series-fit-routing.test.js && node tests/singlehist-parity.test.js && node tests/url-parameters.test.js && node tests/shared-configuration.test.js"
|
||||
"test:electron": "node tests/log-highlighting.test.js && node tests/action-state.test.js && node tests/electron-missing-data.test.js && node tests/window-layering.test.js && node tests/msr-generation.test.js && node tests/web-runner.test.js && node tests/root-plot-clean.test.js && node tests/parameter-plot.test.js && node tests/help-window.test.js && node tests/release-packaging.test.js && node tests/addrun.test.js && node tests/fit-statistic.test.js && node tests/series-fit-planner.test.js && node tests/series-fit-result.test.js && node tests/series-fit-inventory.test.js && node tests/series-fit-msr-preparation.test.js && node tests/series-artifact-contract.test.js && node tests/series-global-merge.test.js && node tests/series-fit-staging.test.js && node tests/series-fit-runner.test.js && node tests/series-runner-adapters.test.js && node tests/series-fit-routing.test.js && node tests/singlehist-parity.test.js && node tests/url-parameters.test.js && node tests/shared-configuration.test.js"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
|
||||
@@ -6,11 +6,41 @@ const childProcess = require("child_process");
|
||||
const fs = require("fs");
|
||||
const os = require("os");
|
||||
const path = require("path");
|
||||
const vm = require("vm");
|
||||
|
||||
const repoRoot = path.resolve(__dirname, "..");
|
||||
const testRoot = fs.mkdtempSync(path.join(os.tmpdir(), "musrflow-cgi-series-collect-"));
|
||||
const perlLibraryPath = [path.join(repoRoot, "tests", "perl"), process.env.PERL5LIB]
|
||||
.filter(Boolean).join(path.delimiter);
|
||||
const shared = { console: console, window: { MSREditor: {} } };
|
||||
shared.global = shared;
|
||||
vm.runInNewContext(fs.readFileSync(path.join(repoRoot, "muSRFitlib.js"), "utf8"), shared);
|
||||
|
||||
function mergedDisplayMSR(texts) {
|
||||
const merged = shared.buildSeriesGlobalMSR(texts, ["514", "515"]);
|
||||
assert.strictEqual(merged.ok, true);
|
||||
return shared.normalizeMSRPlotRuns(merged.text, {}, ["514", "515"]);
|
||||
}
|
||||
|
||||
function seriesCollectionBody(combinedMSR) {
|
||||
return new URLSearchParams({
|
||||
action: "series_collect",
|
||||
fn: "/tmp/seriesjob/514_asy.msr",
|
||||
collection: JSON.stringify({
|
||||
ok: true,
|
||||
version: 1,
|
||||
runs: ["514", "515"],
|
||||
suffix: "_asy",
|
||||
fftv: "n",
|
||||
artifacts: {
|
||||
combined: "514+series_asy.msr",
|
||||
parameters: "514+series_asy_par.dat",
|
||||
log: "514+series_asy.out"
|
||||
},
|
||||
combinedMSR: combinedMSR
|
||||
})
|
||||
}).toString();
|
||||
}
|
||||
|
||||
function fittedMSR(run) {
|
||||
return [
|
||||
@@ -73,8 +103,10 @@ try {
|
||||
const tmpRoot = path.join(webRoot, "tmp");
|
||||
const workPath = path.join(tmpRoot, "seriesjob");
|
||||
fs.mkdirSync(workPath, { recursive: true });
|
||||
fs.writeFileSync(path.join(workPath, "514_asy.msr"), fittedMSR(514));
|
||||
fs.writeFileSync(path.join(workPath, "515_asy.msr"), fittedMSR(515));
|
||||
const first514 = fittedMSR(514);
|
||||
const first515 = fittedMSR(515);
|
||||
fs.writeFileSync(path.join(workPath, "514_asy.msr"), first514);
|
||||
fs.writeFileSync(path.join(workPath, "515_asy.msr"), first515);
|
||||
fs.writeFileSync(path.join(workPath, "514+global_asy.msr"), "genuine global fit\n");
|
||||
fs.writeFileSync(path.join(workPath, "514+global_tmpl.msr"), "genuine global template\n");
|
||||
|
||||
@@ -85,7 +117,7 @@ try {
|
||||
fs.writeFileSync(fakeMsr2data, [
|
||||
"#!/bin/sh",
|
||||
"printf '%s\\n' \"$*\" >> \"$MUSRFLOW_TEST_M2D_ARGS\"",
|
||||
"printf 'RUN Asy_ Asy_PosErr\\n514 0.2 0.01\\n515 0.21 0.01\\n' > 514+series_asy_par.dat",
|
||||
"printf 'dataT Alpha_0514 Alpha_0514PosErr Alpha_0514NegErr my_parameter_0514 CHISQ NDF CHISQred RUN\\n5 0.2 0.01 0.01 1.5 100 95 1.0526 514\\n6 0.21 0.01 0.01 1.6 102 95 1.0737 515\\n' > 514+series_asy_par.dat",
|
||||
"exit 9",
|
||||
""
|
||||
].join("\n"));
|
||||
@@ -99,11 +131,7 @@ try {
|
||||
].join("\n"));
|
||||
fs.chmodSync(fakeMusrview, 0o755);
|
||||
|
||||
const body = new URLSearchParams({
|
||||
action: "series_collect",
|
||||
fn: "/tmp/seriesjob/514_asy.msr",
|
||||
collection: JSON.stringify({ runs: ["514", "515"], suffix: "_asy", fftv: "n" })
|
||||
}).toString();
|
||||
const body = seriesCollectionBody(mergedDisplayMSR([first514, first515]));
|
||||
const execution = childProcess.spawnSync(
|
||||
"perl", [path.join(repoRoot, "cgi", "single_musrfit.cgi")], {
|
||||
cwd: repoRoot,
|
||||
@@ -137,13 +165,14 @@ try {
|
||||
assert.match(response.warnings.join("\n"), /returned code 9/);
|
||||
const m2dArgs = fs.readFileSync(m2dArgsPath, "utf8").trim().split(/\r?\n/);
|
||||
assert.strictEqual(m2dArgs.length, 1);
|
||||
assert.match(m2dArgs[0],
|
||||
/^\[514 515\] _musrflow_series_\d+_\d+ new global data -o 514\+series_asy_par\.dat$/);
|
||||
assert.strictEqual(m2dArgs[0],
|
||||
"[514 515] _asy new data -o 514+series_asy_par.dat");
|
||||
assert.strictEqual(fs.readFileSync(viewArgsPath, "utf8").trim(),
|
||||
"--timeout 300 --root 514+series_asy.msr");
|
||||
assert.strictEqual(fs.readFileSync(
|
||||
path.join(workPath, "514+series_asy_par.dat"), "utf8"
|
||||
).split(/\r?\n/)[0], "RUN Asy Asy_PosErr");
|
||||
).split(/\r?\n/)[0],
|
||||
"dataT Alpha AlphaPosErr AlphaNegErr my_parameter CHISQ NDF CHISQred RUN");
|
||||
const combinedText = fs.readFileSync(path.join(workPath, "514+series_asy.msr"), "utf8");
|
||||
assert.match(combinedText, /^runs\s+1\s+2$/m);
|
||||
assert.match(combinedText, /^# Common parameters for all runs$/m);
|
||||
@@ -174,20 +203,22 @@ try {
|
||||
assert.match(response.command_log, /\[exit code 9\]/);
|
||||
|
||||
// Repeat the same server path with two detector RUN blocks per physical
|
||||
// run. The web/Perl merger must preserve all four blocks just like the
|
||||
// shared JavaScript implementation.
|
||||
fs.writeFileSync(path.join(workPath, "514_asy.msr"), fittedSingleHistMSR(514));
|
||||
fs.writeFileSync(path.join(workPath, "515_asy.msr"), fittedSingleHistMSR(515));
|
||||
// run. The CGI publishes the four-block shared-JavaScript result unchanged.
|
||||
const single514 = fittedSingleHistMSR(514);
|
||||
const single515 = fittedSingleHistMSR(515);
|
||||
fs.writeFileSync(path.join(workPath, "514_asy.msr"), single514);
|
||||
fs.writeFileSync(path.join(workPath, "515_asy.msr"), single515);
|
||||
const singleHistBody = seriesCollectionBody(mergedDisplayMSR([single514, single515]));
|
||||
const singleHistExecution = childProcess.spawnSync(
|
||||
"perl", [path.join(repoRoot, "cgi", "single_musrfit.cgi")], {
|
||||
cwd: repoRoot,
|
||||
encoding: "utf8",
|
||||
input: body,
|
||||
input: singleHistBody,
|
||||
env: Object.assign({}, process.env, {
|
||||
REQUEST_METHOD: "POST",
|
||||
PERL5LIB: perlLibraryPath,
|
||||
CONTENT_TYPE: "application/x-www-form-urlencoded",
|
||||
CONTENT_LENGTH: String(Buffer.byteLength(body)),
|
||||
CONTENT_LENGTH: String(Buffer.byteLength(singleHistBody)),
|
||||
MUSRFIT_WEBROOT: webRoot,
|
||||
MUSRFIT_TMPROOT: tmpRoot,
|
||||
MUSRFIT_TMP_URL: "/tmp",
|
||||
@@ -211,6 +242,44 @@ try {
|
||||
assert.strictEqual((singleHistCombined.match(/^forward 21 25$/gm) || []).length, 2);
|
||||
assert.strictEqual((singleHistCombined.match(/^forward 23 27$/gm) || []).length, 2);
|
||||
assert.match(singleHistCombined, /^runs\s+1\s+2\s+3\s+4$/m);
|
||||
|
||||
// Parameter extraction is independent of display merging. A structural
|
||||
// mismatch prevents the combined MSR, but the per-run msr2data table is
|
||||
// still returned and can be plotted.
|
||||
fs.writeFileSync(path.join(workPath, "514_asy.msr"), fittedMSR(514));
|
||||
fs.writeFileSync(path.join(workPath, "515_asy.msr"),
|
||||
fittedMSR(515).replace("asymmetry 1", "asymmetry 1\nsimplExpo 1"));
|
||||
const dataOnlyBody = seriesCollectionBody("");
|
||||
const dataOnlyExecution = childProcess.spawnSync(
|
||||
"perl", [path.join(repoRoot, "cgi", "single_musrfit.cgi")], {
|
||||
cwd: repoRoot,
|
||||
encoding: "utf8",
|
||||
input: dataOnlyBody,
|
||||
env: Object.assign({}, process.env, {
|
||||
REQUEST_METHOD: "POST",
|
||||
PERL5LIB: perlLibraryPath,
|
||||
CONTENT_TYPE: "application/x-www-form-urlencoded",
|
||||
CONTENT_LENGTH: String(Buffer.byteLength(dataOnlyBody)),
|
||||
MUSRFIT_WEBROOT: webRoot,
|
||||
MUSRFIT_TMPROOT: tmpRoot,
|
||||
MUSRFIT_TMP_URL: "/tmp",
|
||||
MSR2DATA_BIN: fakeMsr2data,
|
||||
MUSRVIEW_BIN: fakeMusrview,
|
||||
MUSRFIT_RUNTIME_HOME: tmpRoot,
|
||||
MUSRFLOW_TEST_M2D_ARGS: m2dArgsPath,
|
||||
MUSRFLOW_TEST_VIEW_ARGS: viewArgsPath
|
||||
})
|
||||
}
|
||||
);
|
||||
assert.strictEqual(dataOnlyExecution.status, 0, dataOnlyExecution.stderr);
|
||||
const dataOnlyResponse = JSON.parse(
|
||||
dataOnlyExecution.stdout.split(/\r?\n\r?\n/).slice(1).join("\n\n")
|
||||
);
|
||||
assert.strictEqual(dataOnlyResponse.ok, true);
|
||||
assert.strictEqual(dataOnlyResponse.parameter_url,
|
||||
"/tmp/seriesjob/514+series_asy_par.dat");
|
||||
assert.strictEqual(dataOnlyResponse.msr_url, "");
|
||||
assert.deepStrictEqual(dataOnlyResponse.root_urls, []);
|
||||
} finally {
|
||||
fs.rmSync(testRoot, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
// Copyright (C) 2026 Zaher Salman
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
const assert = require("assert");
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const vm = require("vm");
|
||||
|
||||
const repoRoot = path.resolve(__dirname, "..");
|
||||
const sandbox = { console: console, window: { MSREditor: {} } };
|
||||
sandbox.global = sandbox;
|
||||
vm.runInNewContext(fs.readFileSync(path.join(repoRoot, "muSRFitlib.js"), "utf8"), sandbox);
|
||||
|
||||
const spec = {
|
||||
ok: true,
|
||||
workPath: "/tmp/job",
|
||||
fileNameSuffix: "_asy",
|
||||
runs: ["514", "515"],
|
||||
msrPaths: ["/tmp/job/514_asy.msr", "/tmp/job/515_asy.msr"],
|
||||
fftv: "y"
|
||||
};
|
||||
const prepared = sandbox.prepareIndependentSeriesCollection(spec);
|
||||
assert.strictEqual(prepared.ok, true);
|
||||
assert.strictEqual(prepared.baseRun, "514");
|
||||
assert.strictEqual(prepared.combinedName, "514+series_asy.msr");
|
||||
assert.strictEqual(prepared.parameterName, "514+series_asy_par.dat");
|
||||
assert.strictEqual(prepared.logName, "514+series_asy.out");
|
||||
|
||||
const request = sandbox.buildIndependentSeriesCollectionRequest(spec, "FITPARAMETER\n...");
|
||||
assert.strictEqual(request.ok, true);
|
||||
assert.strictEqual(request.version, 1);
|
||||
assert.deepStrictEqual(Array.from(request.runs), ["514", "515"]);
|
||||
assert.strictEqual(request.artifacts.combined, "514+series_asy.msr");
|
||||
assert.strictEqual(request.combinedMSR, "FITPARAMETER\n...");
|
||||
|
||||
assert.strictEqual(sandbox.prepareIndependentSeriesCollection(Object.assign({}, spec, {
|
||||
runs: ["514", "514"]
|
||||
})).reason, "series-artifact-source-invalid");
|
||||
|
||||
const dataOnly = sandbox.normalizeIndependentSeriesCollectionResult({
|
||||
ok: false,
|
||||
reason: "series-display-merge-failed",
|
||||
parameterDataPath: "/tmp/job/514+series_asy_par.dat",
|
||||
warnings: ["display unavailable", "", null]
|
||||
});
|
||||
assert.strictEqual(dataOnly.ok, true);
|
||||
assert.strictEqual(dataOnly.reason, "series-display-merge-failed");
|
||||
assert.deepStrictEqual(Array.from(dataOnly.warnings), ["display unavailable"]);
|
||||
|
||||
const empty = sandbox.normalizeIndependentSeriesCollectionResult({ ok: true });
|
||||
assert.strictEqual(empty.ok, false);
|
||||
assert.strictEqual(empty.reason, "series-artifacts-not-produced");
|
||||
|
||||
console.log("series artifact contract tests passed");
|
||||
@@ -293,7 +293,7 @@ function testMergingErrors(lib) {
|
||||
assert.strictEqual(results.source.ok, false);
|
||||
assert.strictEqual(results.source.reason, "series-merge-source-incomplete");
|
||||
|
||||
// A function that multiplies run-specific Asy cannot be shared across runs.
|
||||
// Run-specific function parameters are represented through RUN map slots.
|
||||
const functionSource = function(run) {
|
||||
const suffix = String(run).padStart(4, "0");
|
||||
return [
|
||||
@@ -320,8 +320,12 @@ function testMergingErrors(lib) {
|
||||
results.functionRunSpecific = lib.buildSeriesGlobalMSR(
|
||||
[functionSource(518), functionSource(519)], [518, 519]
|
||||
);
|
||||
assert.strictEqual(results.functionRunSpecific.ok, false);
|
||||
assert.strictEqual(results.functionRunSpecific.reason, "series-merge-functions-run-specific");
|
||||
assert.strictEqual(results.functionRunSpecific.ok, true);
|
||||
sanctuaryCase("run-specific function parameters use map slots", function() {
|
||||
assert.match(results.functionRunSpecific.text, /^fun1 = map1 \* map2$/m);
|
||||
assert.match(results.functionRunSpecific.text, /^map\s+1\s+2$/m);
|
||||
assert.match(results.functionRunSpecific.text, /^map\s+3\s+4$/m);
|
||||
});
|
||||
|
||||
// But a function over a shared parameter stays valid and is renumbered.
|
||||
const sharedFunctionSource = function(run) {
|
||||
@@ -358,6 +362,73 @@ function testMergingErrors(lib) {
|
||||
});
|
||||
}
|
||||
|
||||
function testMaximumLikelihoodStatistic(lib) {
|
||||
const source = function(run, value) {
|
||||
return stagedAsymmetryMSR(run, "0.2")
|
||||
.replace("chisq = 101.2, NDF = 100, chisq/NDF = 1.012000",
|
||||
"maxLH = " + value + ", NDF = 100, maxLH/NDF = "
|
||||
+ (Number(value) / 100).toFixed(6));
|
||||
};
|
||||
const result = lib.buildSeriesGlobalMSR(
|
||||
[source(514, "120.5"), source(515, "121.5")], [514, 515]
|
||||
);
|
||||
assert.strictEqual(result.ok, true);
|
||||
sanctuaryCase("maximum-likelihood statistics are aggregated", function() {
|
||||
assert.match(result.text, /^ maxLH = 242, NDF = 200, maxLH\/NDF = 1\.21$/m);
|
||||
assert.doesNotMatch(result.text, /^ chisq =/m);
|
||||
});
|
||||
|
||||
const partial = lib.buildSeriesGlobalMSR(
|
||||
[source(514, "120.5"), stagedAsymmetryMSR(515, "0.2")], [514, 515]
|
||||
);
|
||||
assert.strictEqual(partial.ok, true);
|
||||
sanctuaryCase("mixed objectives do not produce a misleading aggregate", function() {
|
||||
assert.doesNotMatch(partial.text, /^STATISTIC\b/m);
|
||||
});
|
||||
}
|
||||
|
||||
function testFunctionAndDetectorMapsCoexist(lib) {
|
||||
const source = function(run) {
|
||||
const suffix = String(run).padStart(4, "0");
|
||||
return [
|
||||
"FITPARAMETER",
|
||||
"1 Phase_" + suffix + " 10 0.1 none",
|
||||
"2 RelL_" + suffix + " 0 0 none",
|
||||
"3 Field_" + suffix + " 75 0.1 none",
|
||||
"4 Scale_" + suffix + " 0.0135 0 none",
|
||||
"5 RelR_" + suffix + " 180 0 none",
|
||||
"THEORY",
|
||||
"TFieldCos fun1 fun2",
|
||||
"FUNCTIONS",
|
||||
"fun1 = par1 + map1",
|
||||
"fun2 = par3 * par4",
|
||||
"GLOBAL",
|
||||
"fittype 0",
|
||||
"RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT",
|
||||
"map 2 0 0 0 0 0 0 0 0 0",
|
||||
"forward 21",
|
||||
"RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT",
|
||||
"map 5 0 0 0 0 0 0 0 0 0",
|
||||
"forward 23",
|
||||
"COMMANDS",
|
||||
"MIGRAD",
|
||||
"PLOT 0",
|
||||
"runs 1 2",
|
||||
""
|
||||
].join("\n");
|
||||
};
|
||||
const result = lib.buildSeriesGlobalMSR([source(514), source(515)], [514, 515]);
|
||||
assert.strictEqual(result.ok, true);
|
||||
sanctuaryCase("function maps fill zero slots beside detector maps", function() {
|
||||
assert.match(result.text, /^fun1 = map2 \+ map1$/m);
|
||||
assert.match(result.text, /^fun2 = map3 \* map4$/m);
|
||||
assert.match(result.text, /^map\s+2\s+1\s+3\s+4$/m);
|
||||
assert.match(result.text, /^map\s+5\s+1\s+3\s+4$/m);
|
||||
assert.match(result.text, /^map\s+7\s+6\s+8\s+9$/m);
|
||||
assert.match(result.text, /^map\s+10\s+6\s+8\s+9$/m);
|
||||
});
|
||||
}
|
||||
|
||||
(async function() {
|
||||
const lib = loadLibrary();
|
||||
testMergesTwoAsymmetryRuns(lib);
|
||||
@@ -365,6 +436,8 @@ function testMergingErrors(lib) {
|
||||
testPreservesEverySingleHistRunBlock(lib);
|
||||
testFourierForDisplay(lib);
|
||||
testMergingErrors(lib);
|
||||
testMaximumLikelihoodStatistic(lib);
|
||||
testFunctionAndDetectorMapsCoexist(lib);
|
||||
console.log("series global merge tests passed");
|
||||
})().catch(function(error) {
|
||||
console.error(error);
|
||||
|
||||
@@ -214,8 +214,13 @@ async function testElectronSeriesArtifactCollection() {
|
||||
const fakeChildProcess = {
|
||||
spawnSync: function(command, args, options) {
|
||||
syncCalls.push({ command: command, args: args.slice(), cwd: options.cwd });
|
||||
fs.writeFileSync(path.join(options.cwd, "514+series_asy_par.dat"),
|
||||
"RUN Asy_ Asy_PosErr\n514 0.2 0.01\n515 0.21 0.01\n", "utf8");
|
||||
fs.writeFileSync(path.join(options.cwd, "514+series_asy_par.dat"), [
|
||||
"dataT dataTErr Alpha_0514 Alpha_0514PosErr Alpha_0514NegErr "
|
||||
+ "my_parameter_0514 my_parameter_0514PosErr CHISQ NDF CHISQred RUN",
|
||||
"5 0.1 0.2 0.01 0.01 1.5 0.1 100 95 1.0526 514",
|
||||
"6 0.1 0.21 0.01 0.01 1.6 0.1 102 95 1.0737 515",
|
||||
""
|
||||
].join("\n"), "utf8");
|
||||
return { status: 7, stdout: "parameter data written\n", stderr: "" };
|
||||
},
|
||||
spawn: function(command, args, options) {
|
||||
@@ -248,13 +253,14 @@ async function testElectronSeriesArtifactCollection() {
|
||||
assert.strictEqual(result.plotStarted, true);
|
||||
assert.match(result.warnings.join("\n"), /returned code 7/);
|
||||
assert.strictEqual(syncCalls.length, 1);
|
||||
assert.match(syncCalls[0].args[1], /^_musrflow_series_\d+_[a-z0-9]+$/);
|
||||
assert.deepStrictEqual(Array.from(syncCalls[0].args.slice(0, 1)), ["[514 515]"]);
|
||||
assert.deepStrictEqual(Array.from(syncCalls[0].args.slice(2)),
|
||||
["new", "global", "data", "-o", "514+series_asy_par.dat"]);
|
||||
assert.deepStrictEqual(Array.from(syncCalls[0].args),
|
||||
["[514 515]", "_asy", "new", "data", "-o", "514+series_asy_par.dat"]);
|
||||
assert.deepStrictEqual(Array.from(asyncCalls[0].args), ["514+series_asy.msr"]);
|
||||
assert.match(fs.readFileSync(result.parameterDataPath, "utf8"),
|
||||
/^RUN Asy Asy_PosErr/m);
|
||||
const parameterHeader = fs.readFileSync(result.parameterDataPath, "utf8")
|
||||
.split(/\r?\n/)[0];
|
||||
assert.strictEqual(parameterHeader,
|
||||
"dataT dataTErr Alpha AlphaPosErr AlphaNegErr "
|
||||
+ "my_parameter my_parameterPosErr CHISQ NDF CHISQred RUN");
|
||||
const combinedText = fs.readFileSync(result.combinedMSRPath, "utf8");
|
||||
assert.match(combinedText, /^runs\s+1\s+2$/m);
|
||||
assert.match(combinedText, /^# Common parameters for all runs$/m);
|
||||
@@ -274,9 +280,60 @@ async function testElectronSeriesArtifactCollection() {
|
||||
}
|
||||
}
|
||||
|
||||
async function testElectronParameterCollectionSurvivesDisplayMergeFailure() {
|
||||
const tempDirectory = fs.mkdtempSync(path.join(os.tmpdir(), "musrflow-series-data-only-"));
|
||||
const syncCalls = [];
|
||||
const asyncCalls = [];
|
||||
try {
|
||||
fs.writeFileSync(path.join(tempDirectory, "514_asy.msr"),
|
||||
runnableMSR(514, true), "utf8");
|
||||
fs.writeFileSync(path.join(tempDirectory, "515_asy.msr"),
|
||||
runnableMSR(515, true).replace("asymmetry 1", "asymmetry 1\nsimplExpo 1"),
|
||||
"utf8");
|
||||
const sandbox = loadElectronSandbox({
|
||||
spawnSync: function(command, args, options) {
|
||||
syncCalls.push(args.slice());
|
||||
fs.writeFileSync(path.join(options.cwd, "514+series_asy_par.dat"),
|
||||
"RUN Asy_\n514 0.2\n515 0.21\n", "utf8");
|
||||
return { status: 0, stdout: "", stderr: "" };
|
||||
},
|
||||
spawn: function(command, args) {
|
||||
asyncCalls.push(args.slice());
|
||||
const child = new events.EventEmitter();
|
||||
child.stdout = new events.EventEmitter();
|
||||
child.stderr = new events.EventEmitter();
|
||||
return child;
|
||||
}
|
||||
}, []);
|
||||
const result = await sandbox.window.collectIndependentSeriesArtifactsLocal({
|
||||
ok: true,
|
||||
workPath: tempDirectory,
|
||||
fileNameSuffix: "_asy",
|
||||
runs: ["514", "515"],
|
||||
msrPaths: [
|
||||
path.join(tempDirectory, "514_asy.msr"),
|
||||
path.join(tempDirectory, "515_asy.msr")
|
||||
],
|
||||
fftv: "n"
|
||||
});
|
||||
|
||||
assert.strictEqual(result.ok, true);
|
||||
assert.strictEqual(result.parameterDataPath,
|
||||
path.join(tempDirectory, "514+series_asy_par.dat"));
|
||||
assert.strictEqual(result.combinedMSRPath, "");
|
||||
assert.strictEqual(result.plotStarted, false);
|
||||
assert.strictEqual(syncCalls.length, 1);
|
||||
assert.strictEqual(asyncCalls.length, 0);
|
||||
assert.match(result.warnings.join("\n"), /series-merge-theory-mismatch/);
|
||||
} finally {
|
||||
fs.rmSync(tempDirectory, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
Promise.resolve().then(testElectronAdapterFitsOnlyOneMSR)
|
||||
.then(testElectronAdapterRetainsRejectedNewMSR)
|
||||
.then(testElectronSeriesArtifactCollection).then(function() {
|
||||
.then(testElectronSeriesArtifactCollection)
|
||||
.then(testElectronParameterCollectionSurvivesDisplayMergeFailure).then(function() {
|
||||
console.log("series runner adapter tests passed");
|
||||
}).catch(function(error) {
|
||||
console.error(error);
|
||||
|
||||
@@ -272,6 +272,10 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() {
|
||||
const logs = [];
|
||||
const lib = loadWebLibrary(async function(url, options) {
|
||||
calls.push({ url: url, options: options });
|
||||
if (!options || options.method !== "POST") {
|
||||
const run = /515_asy\.msr$/.test(url) ? 515 : 514;
|
||||
return createResponse(convergedWebMSR(run));
|
||||
}
|
||||
return createResponse(JSON.stringify({
|
||||
ok: true,
|
||||
code: 0,
|
||||
@@ -307,13 +311,24 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() {
|
||||
assert.strictEqual(result.plotPath, "/tmp/job/514+series_asy_0.root");
|
||||
assert.strictEqual(result.plotStarted, true);
|
||||
assert.match(result.warnings[0], /returned code 9/);
|
||||
assert.strictEqual(calls.length, 1);
|
||||
const body = new URLSearchParams(calls[0].options.body);
|
||||
assert.strictEqual(calls.length, 3);
|
||||
assert.deepStrictEqual(calls.slice(0, 2).map(function(call) { return call.url; }),
|
||||
["/tmp/job/514_asy.msr", "/tmp/job/515_asy.msr"]);
|
||||
const body = new URLSearchParams(calls[2].options.body);
|
||||
assert.strictEqual(body.get("action"), "series_collect");
|
||||
assert.strictEqual(body.get("fn"), "/tmp/job/514_asy.msr");
|
||||
assert.deepStrictEqual(JSON.parse(body.get("collection")), {
|
||||
runs: ["514", "515"], suffix: "_asy", fftv: "n"
|
||||
const collection = JSON.parse(body.get("collection"));
|
||||
assert.strictEqual(collection.version, 1);
|
||||
assert.deepStrictEqual(collection.runs, ["514", "515"]);
|
||||
assert.strictEqual(collection.suffix, "_asy");
|
||||
assert.strictEqual(collection.fftv, "n");
|
||||
assert.deepStrictEqual(collection.artifacts, {
|
||||
combined: "514+series_asy.msr",
|
||||
parameters: "514+series_asy_par.dat",
|
||||
log: "514+series_asy.out"
|
||||
});
|
||||
assert.match(collection.combinedMSR, /^# Independent series display MSR/m);
|
||||
assert.match(collection.combinedMSR, /^runs\s+1\s+2$/m);
|
||||
assert.strictEqual(plots.length, 1);
|
||||
assert.deepStrictEqual(Array.from(plots[0].urls),
|
||||
["/tmp/job/514+series_asy_0.root"]);
|
||||
@@ -321,6 +336,44 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() {
|
||||
assert.match(logs[0], /msr2data/);
|
||||
}
|
||||
|
||||
async function testWebParameterCollectionSurvivesClientMergeFailure() {
|
||||
const calls = [];
|
||||
const lib = loadWebLibrary(async function(url, options) {
|
||||
calls.push({ url: url, options: options });
|
||||
if (!options || options.method !== "POST") {
|
||||
const run = /515_asy\.msr$/.test(url) ? 515 : 514;
|
||||
let text = convergedWebMSR(run);
|
||||
if (run === 515) text = text.replace("asymmetry 1", "asymmetry 1\nsimplExpo 1");
|
||||
return createResponse(text);
|
||||
}
|
||||
return createResponse(JSON.stringify({
|
||||
ok: true,
|
||||
reason: "series-artifacts-collected",
|
||||
parameter_url: "/tmp/job/514+series_asy_par.dat",
|
||||
msr_url: "",
|
||||
out_url: "/tmp/job/514+series_asy.out",
|
||||
root_urls: [],
|
||||
warnings: []
|
||||
}));
|
||||
});
|
||||
|
||||
const result = await lib.window.collectIndependentSeriesArtifactsWeb({
|
||||
ok: true,
|
||||
workPath: "/tmp/job",
|
||||
fileNameSuffix: "_asy",
|
||||
runs: ["514", "515"],
|
||||
msrPaths: ["/tmp/job/514_asy.msr", "/tmp/job/515_asy.msr"],
|
||||
fftv: "n"
|
||||
});
|
||||
assert.strictEqual(result.ok, true);
|
||||
assert.strictEqual(result.parameterDataPath, "/tmp/job/514+series_asy_par.dat");
|
||||
assert.strictEqual(result.combinedMSRPath, "");
|
||||
assert.strictEqual(result.plotStarted, false);
|
||||
assert.match(result.warnings.join("\n"), /series-merge-theory-mismatch/);
|
||||
const body = new URLSearchParams(calls[2].options.body);
|
||||
assert.strictEqual(JSON.parse(body.get("collection")).combinedMSR, "");
|
||||
}
|
||||
|
||||
function convergedWebMSR(run) {
|
||||
const suffix = String(run).padStart(4, "0");
|
||||
return [
|
||||
@@ -615,6 +668,7 @@ async function runTests() {
|
||||
await testSeriesTransactionRejectsMixedJobDirectories();
|
||||
await testWebSeriesCandidateUsesDedicatedEndpoint();
|
||||
await testWebSeriesCollectionRegistersArtifactsAndPlot();
|
||||
await testWebParameterCollectionSurvivesClientMergeFailure();
|
||||
await testWebSeriesInventoryFeedsSharedClassifier();
|
||||
await testIncompleteWebSeriesInventoryFailsClosed();
|
||||
await testWebRunLoadsFittedMSRAndArtifacts();
|
||||
|
||||
Reference in new issue
Block a user