diff --git a/CGI_PARITY_CHECKLIST.md b/CGI_PARITY_CHECKLIST.md index 85621fb..42d9146 100644 --- a/CGI_PARITY_CHECKLIST.md +++ b/CGI_PARITY_CHECKLIST.md @@ -37,7 +37,7 @@ Status legend: | Run-number expansion (`-`, `:` syntax) | `MSR.pm::ExpandRunNumbers` | `arrayRunNumbers()` | `PARTIAL` | Common ascending single/range/step syntax is supported with stricter validation. Legacy descending `from:to:step` input swaps its endpoints; the current GUI deliberately rejects descending ranges instead. | | Single-run/asymmetry MSR creation | `MSR.pm::CreateMSRUni` | `createMSR()` / `buildMSR()` | `PARTIAL` | Core FITPARAMETER, THEORY/FUNCTIONS, GLOBAL, RUN, COMMANDS, PLOT, and FOURIER blocks are generated. Multiple fit ranges, explicit data-good-bin ranges, and legacy title handling remain gaps. | | Multiple fit ranges / packings | `MSR.pm::CreateMSRUni` (`Tis`, `Tfs`, `BINS` arrays) | Single Fit range row | `MISSING` | Legacy CGI accepted comma-separated range/packing sets and repeated RUN blocks accordingly. Current generation expects one fit interval and one packing value. | -| Global MSR generation | `MSR.pm::CreateMSRGLB`, `msr2data global` | `prepareMSRRunFiles()` + local/web `msr2data` | `DONE` | Multi-run asymmetry uses Template -> `msr2data` -> Global MSR. Runners remove stale output and validate the generated global MSR instead of relying solely on unreliable `msr2data` exit codes. | +| Global MSR generation | `MSR.pm::CreateMSRGLB`, `msr2data global` | `prepareMSRRunFiles()` + `buildSeriesGlobalMSR()` | `DONE` | Multi-run asymmetry merges the already-fitted per-run MSRs into the combined display MSR in JS (`muSRFitlib.js`) and Perl (`single_musrfit.cgi`). msr2data is no longer asked to build it, because it only writes its `+global` file from a template and re-fits each run in every value-populating mode. Stale output is removed and the merged MSR is validated before use, instead of relying on msr2data's unreliable exit codes. | | Parameter table and bounds | `MSR.pm::PrepParamTable` | `cfgParams()`, `formatFitParameterLine()`, fitted-MSR parser | `PARTIAL` | Value/step and two-sided or one-sided bounds are generated correctly. Fitted values and bounds can be imported. The GUI does not separately preserve/display the asymmetric positive MINOS error field from a seven-field fitted parameter line. | | Parameter-data export | `MSR.pm::ExportParams`, post-fit `msr2data` | Web/Electron runners + floating parameter viewer | `DONE` | Both runners generate `_par.dat` after single/global fitting and expose the same artifact to the integrated floating Plotly viewer; the raw table remains available in Results. | @@ -56,10 +56,10 @@ Status legend: | Capability | Legacy source | Current GUI source | Status | Notes | |---|---|---|---|---| -| Plot-only and fit+plot execution | Legacy `plotonly` / `fitplot` commands | `muSRFitelec.js`, `muSRFitweb.js`, `cgi/single_musrfit.cgi` | `DONE` | Electron and web have separate Plot/Fit paths. Multi-run jobs generate the global MSR first and normalize PLOT run indices before plotting. Fit and Plot share an action lock with visible preparation/execution phases, preventing duplicate or conflicting requests. Missing RUN data from either musrfit or musrview is reported as an amber `Missing data` outcome rather than a failed fit/plot; Fit stops before starting musrview in that case. | +| Plot-only and fit+plot execution | Legacy `plotonly` / `fitplot` commands | `muSRFitelec.js`, `muSRFitweb.js`, `cgi/single_musrfit.cgi` | `DONE` | Electron and web have separate Plot/Fit paths. Multi-run jobs assemble the combined display MSR from the per-run fitted files and normalize PLOT run indices before plotting. Fit and Plot share an action lock with visible preparation/execution phases, preventing duplicate or conflicting requests. Missing RUN data from either musrfit or musrview is reported as an amber `Missing data` outcome rather than a failed fit/plot; Fit stops before starting musrview in that case. | | Interactive plot delivery | Legacy `musrview --png` + ImageMagick crop | Native musrview (Electron), `musrview --root` + JSROOT (web) | `DONE` | This intentionally replaces static PNG delivery. Web ROOT files support interactive zoom/resizing and FFT toggling; Electron keeps the native musrview window. | | Spectrum ASCII extraction + gzip | Legacy `musrview --ascii` + `gzip FILENAME.dat` | No equivalent spectrum dump | `MISSING` | Job tarballs and parameter `.dat` export are different artifacts and do not replace the legacy plotted-data ASCII dump. Add only if this download remains useful to users. | -| `msr2data` integration | Legacy `$MSR2DATA` workflow | Local runner + `cgi/single_musrfit.cgi` | `DONE` | Used for global generation and web post-fit parameter extraction; generated global MSR validity is checked explicitly. | +| `msr2data` integration | Legacy `$MSR2DATA` workflow | Local runner + `cgi/single_musrfit.cgi` | `DONE` | msr2data handles the post-fit parameter `.dat` export in both runners. The combined multi-run display MSR is assembled locally from the per-run fitted files instead; the merged MSR is validated explicitly before use. | | Debug/command output | Legacy downloadable `.out` / debug files | `.out` command log + Results details + concise status bar | `DONE` | Current logs record command, combined output, and exit status while keeping routine live status compact. | | Web health/runtime configuration | No direct legacy equivalent | CGI `health` action + `muSRFitweb.js` | `DONE` | Reports runtime/executable/tmp/data-root readiness and refreshes the server-owned data root before actions. | | Dedicated fit statistic display | `musrfit.cgi::PrintOut` / fitted `STATISTIC` block | `parseMSRFitStatistic()` + status bar | `DONE` | After Fit, Electron and web parse the fitted MSR's overall `STATISTIC` line and keep chisq/NDF or maxLH/NDF visible in the status bar; `*** FIT DID NOT CONVERGE ***` is surfaced explicitly. Plot-only actions do not reinterpret an older statistic as a new result. | diff --git a/cgi/single_musrfit.cgi b/cgi/single_musrfit.cgi index 1632ad0..a601e53 100755 --- a/cgi/single_musrfit.cgi +++ b/cgi/single_musrfit.cgi @@ -995,6 +995,458 @@ sub normalize_plot_runs_line { return $content; } +# Merge the fitted per-run MSR files of an independent series fit into a single +# display-only global MSR. msr2data is deliberately not used for this: it only +# writes its +global file from a template and, in the modes that populate +# per-run values, re-fits each run with musrfit. The merged file keeps each +# run's already-fitted parameter values and lets every RUN block select its own +# parameters through the same mapN/alpha/norm mechanism a genuine msr2data +# global file uses. +my @SERIES_MSR_BLOCK_NAMES = qw(FITPARAMETER THEORY FUNCTIONS GLOBAL RUN COMMANDS PLOT FOURIER STATISTIC); +my %SERIES_MSR_RUN_PARAM_DIRECTIVES = map { $_ => 1 } qw(alpha beta norm backgr.fit); +my $SERIES_MSR_MAX_MAP_SLOTS = 10; + +# Recognize a block header line. The FOURIER block uses a lowercase +# `plot ` directive that would otherwise look like a PLOT header, so a +# genuine PLOT header must carry a numeric plot id ("PLOT 2"); non-numeric +# `plot` lines stay inside FOURIER. +sub series_msr_block_header_name { + my ($trimmed) = @_; + for my $name (@SERIES_MSR_BLOCK_NAMES) { + if ($trimmed =~ /^\Q$name\E(?:\s|$)/i) { + my $candidate = uc($name); + if ($candidate eq "PLOT") { + my $arg = $trimmed; + $arg =~ s/^\Q$name\E//i; + $arg =~ s/^\s+|\s+$//g; + return "" if $arg !~ /^\d/; + } + return $candidate; + } + } + return ""; +} + +sub series_msr_blocks { + my ($content) = @_; + my %blocks; + my $section = ""; + for my $line (split(/\r?\n/, defined($content) ? $content : "")) { + my $trimmed = $line; + $trimmed =~ s/^\s+//; + my $block = series_msr_block_header_name($trimmed); + if ($block ne "") { + my $start = $line; + $start =~ s/^\s+//; + $start =~ s/\s+$//; + $section = $block; + $blocks{$block} = { header => $start, lines => [] } + if !exists($blocks{$block}); + } elsif ($section ne "") { + push(@{$blocks{$section}{lines}}, $line); + } else { + $blocks{HEADER} = { header => "", lines => [] } + if !exists($blocks{HEADER}); + push(@{$blocks{HEADER}{lines}}, $line); + } + } + return \%blocks; +} + +sub series_msr_parameters { + my ($lines) = @_; + my @params; + for my $line (@{$lines || []}) { + my $trimmed = $line; + $trimmed =~ s/^\s+//; + next if $trimmed eq "" || $trimmed =~ /^#/; + my ($no, $name, $rest) = $trimmed =~ /^(\d+)\s+(\S+)\s*(.*)$/; + next if !defined($no); + $rest = "" if !defined($rest); + $rest =~ s/^\s+|\s+$//g; + $rest =~ s/\s+/ /g; + push(@params, { no => 0 + $no, name => $name, rest => $rest }); + } + return \@params; +} + +sub series_msr_run_blocks { + my ($header, $lines) = @_; + my @blocks; + my $current; + for my $line (($header, @{$lines || []})) { + my $trimmed = $line; + $trimmed =~ s/^\s+//; + if ($trimmed =~ /^RUN\b/i) { + my $start = $line; + $start =~ s/\s+$//; + $current = { header => $start, lines => [] }; + push(@blocks, $current); + } elsif (defined($current)) { + push(@{$current->{lines}}, $line); + } + } + return \@blocks; +} + +sub series_msr_token_shape { + my ($lines) = @_; + return join("\n", map { + join(" ", map { /^\d+$/ ? "#" : $_ } split(/\s+/, $_)) + } @{$lines || []}); +} + +sub format_series_merged_parameter_line { + my ($no, $name, $rest) = @_; + my $line = " $no $name"; + $line .= " $rest" if defined($rest) && $rest ne ""; + return $line; +} + +# The FOURIER block is shared by all RUN blocks in the merged file. musrview +# refuses any plot other than POWER unless the phase line carries one reference +# per run (parR7 par9 ...), which a display file assembled from shared +# parameters cannot express. Mirror msr2data's own global output by forcing the +# plot to POWER; the fitted per-run Fourier phases are deliberately left to the +# individual run files. +sub series_fourier_for_display { + my ($lines) = @_; + my @normalized; + my $saw_plot = 0; + for my $line (@{$lines || []}) { + my $trimmed = $line; + $trimmed =~ s/^\s+//; + if ($trimmed =~ /^plot\b/i) { + $saw_plot = 1; + if ($trimmed !~ /^plot\s+POWER\b/i) { + push(@normalized, "plot POWER"); + } else { + push(@normalized, $line); + } + } else { + push(@normalized, $line); + } + } + push(@normalized, "plot POWER") if !$saw_plot; + return \@normalized; +} + +# Returns [$ok, $reason, $text]. +sub build_series_global_msr { + my ($msr_texts, $run_numbers) = @_; + return [0, "series-merge-invalid-input", ""] + if ref($msr_texts) ne "ARRAY" || ref($run_numbers) ne "ARRAY" + || @{$msr_texts} < 2 || @{$msr_texts} != @{$run_numbers}; + + my @per_run; + for my $index (0 .. $#$msr_texts) { + my $blocks = series_msr_blocks($msr_texts->[$index]); + return [0, "series-merge-source-incomplete", ""] + if !$blocks->{FITPARAMETER} || !$blocks->{THEORY} || !$blocks->{RUN}; + my $params = series_msr_parameters($blocks->{FITPARAMETER}{lines}); + return [0, "series-merge-parameters-incomplete", ""] if !@{$params}; + my $run_blocks = series_msr_run_blocks( + $blocks->{RUN}{header}, $blocks->{RUN}{lines} + ); + return [0, "series-merge-run-blocks-incomplete", ""] if !@{$run_blocks}; + push(@per_run, { + run => "$run_numbers->[$index]", + params => $params, + run_blocks => $run_blocks, + theory => $blocks->{THEORY}{lines} || [], + functions => $blocks->{FUNCTIONS}{lines} || [], + global => $blocks->{GLOBAL}{lines} || [], + commands => $blocks->{COMMANDS}{lines} || [], + plot => $blocks->{PLOT}{lines} || [], + plot_header => $blocks->{PLOT}{header} || "PLOT 0", + fourier => $blocks->{FOURIER}{lines} || [], + statistic => $blocks->{STATISTIC}{lines} || [] + }); + } + + my $num_runs = scalar(@per_run); + my $first = $per_run[0]; + for my $pr (@per_run) { + return [0, "series-merge-parameter-count-mismatch", ""] + if @{$pr->{params}} != @{$first->{params}}; + } + + # A name present in every run is shared; present in one run is run-specific. + my %name_counts; + for my $pr (@per_run) { + $name_counts{$_->{name}}++ for @{$pr->{params}}; + } + for my $name (keys %name_counts) { + return [0, "series-merge-parameter-inconsistency", ""] + if $name_counts{$name} != 1 && $name_counts{$name} != $num_runs; + } + + # All runs must describe the same model: the non-numeric token shape of the + # THEORY and FUNCTIONS blocks is identical, only parameter numbers differ. + for my $pr (@per_run) { + return [0, "series-merge-theory-mismatch", ""] + if series_msr_token_shape($pr->{theory}) ne series_msr_token_shape($first->{theory}); + return [0, "series-merge-functions-mismatch", ""] + if series_msr_token_shape($pr->{functions}) ne series_msr_token_shape($first->{functions}); + } + + # Assign merged numbers: shared parameters first, run-specific grouped per run. + my @merged_params; + my %shared_name_no; + for my $p (@{$first->{params}}) { + if ($name_counts{$p->{name}} == $num_runs + && !exists($shared_name_no{$p->{name}})) { + $shared_name_no{$p->{name}} = scalar(@merged_params) + 1; + push(@merged_params, { name => $p->{name}, rest => $p->{rest} }); + } + } + my @no_map; + for my $run_index (0 .. $num_runs - 1) { + $no_map[$run_index] = {}; + for my $p (@{$per_run[$run_index]{params}}) { + if ($name_counts{$p->{name}} == $num_runs) { + $no_map[$run_index]{$p->{no}} = $shared_name_no{$p->{name}}; + } else { + $no_map[$run_index]{$p->{no}} = scalar(@merged_params) + 1; + push(@merged_params, { name => $p->{name}, rest => $p->{rest} }); + } + } + } + + my %by_number; + for my $pr (@per_run) { + my %map; + $map{$_->{no}} = $_ for @{$pr->{params}}; + $by_number{$pr} = \%map; + } + + # Rewrite THEORY: existing mapN/funN stay, direct integers are renumbered, + # run-specific roles become map slots filled per RUN block below. + my %occupied; + for my $lines ($first->{theory}, $first->{functions}) { + for my $line (@{$lines || []}) { + for my $token (split(/\s+/, $line)) { + $occupied{$1}++ if $token =~ /^map(\d+)$/i; + } + } + } + my $next_slot = 1; + $next_slot++ while $occupied{$next_slot}; + my (%slot_role_name, %slot_position); + my $slot_for = sub { + my ($name, $line_index, $token_index) = @_; + return $slot_role_name{$name} if exists($slot_role_name{$name}); + return 0 if $next_slot > $SERIES_MSR_MAX_MAP_SLOTS; + my $slot = $next_slot++; + $slot_role_name{$name} = $slot; + $slot_position{$slot} = [ $line_index, $token_index ]; + return $slot; + }; + + my @merged_theory; + for my $line_index (0 .. $#{$first->{theory}}) { + my @out; + my $token_index = 0; + for my $token (split(/\s+/, $first->{theory}[$line_index])) { + if ($token =~ /^\d+$/ && $token ne "0") { + my $local_no = 0 + $token; + my $p = $by_number{$first}{$local_no}; + return [0, "series-merge-theory-param-missing", ""] if !$p; + if ($name_counts{$p->{name}} == $num_runs) { + push(@out, $shared_name_no{$p->{name}}); + } else { + my $slot = $slot_for->($p->{name}, $line_index, $token_index); + return [0, "series-merge-map-slots-exhausted", ""] if $slot == 0; + push(@out, "map$slot"); + } + } else { + push(@out, $token); + } + $token_index++; + } + push(@merged_theory, join(" ", @out)); + } + + # FUNCTIONS is shared, so it may only reference parameters that keep their + # number in every RUN block. + my @merged_functions; + for my $line (@{$first->{functions}}) { + my @out; + for my $token (split(/\s+/, $line)) { + if ($token =~ /^par(\d+)$/i) { + my $local_no = 0 + $1; + my $p = $by_number{$first}{$local_no}; + return [0, "series-merge-functions-param-missing", ""] if !$p; + return [0, "series-merge-functions-run-specific", ""] + if $name_counts{$p->{name}} != $num_runs; + push(@out, "par" . $shared_name_no{$p->{name}}); + } else { + push(@out, $token); + } + } + push(@merged_functions, join(" ", @out)); + } + + my $max_slot = 0; + for my $slot (keys %slot_position) { $max_slot = $slot if $slot > $max_slot; } + + my $merged_map_entries = sub { + my ($run_index, $block) = @_; + my @existing_values; + for my $line (@{$block->{lines}}) { + if ($line =~ /^\s*map\b/i) { + my @tokens = split(/\s+/, $line); + @existing_values = @tokens > 1 ? @tokens[1 .. $#tokens] : (); + last; + } + } + my %entries; + my $slot_max = $max_slot; + $slot_max = scalar(@existing_values) if @existing_values > $slot_max; + for my $slot (1 .. $slot_max) { + my $value = 0; + if ($slot <= @existing_values && $existing_values[$slot - 1] =~ /^\d+$/) { + my $local_no = 0 + $existing_values[$slot - 1]; + $value = ($local_no > 0 && defined($no_map[$run_index]{$local_no})) + ? $no_map[$run_index]{$local_no} : $local_no; + } elsif (exists($slot_position{$slot})) { + my ($line_index, $token_index) = @{$slot_position{$slot}}; + my @local_tokens = split(/\s+/, $per_run[$run_index]{theory}[$line_index] || ""); + my $local_no = $local_tokens[$token_index] =~ /^\d+$/ + ? 0 + $local_tokens[$token_index] : 0; + $value = $no_map[$run_index]{$local_no} + if $local_no > 0 && defined($no_map[$run_index]{$local_no}); + } + $entries{$slot} = $value; + } + my @values; + for my $slot (1 .. $slot_max) { push(@values, $entries{$slot}); } + pop @values while @values && $values[$#values] == 0; + return \@values; + }; + + my @output; + push(@output, "# Independent series display MSR for the runs: " . join(" ", @{$run_numbers})); + push(@output, "# Fitted per-run parameters collected without re-fitting; " + . "msr2data handles parameter extraction separately."); + push(@output, "###############################################################"); + push(@output, "FITPARAMETER"); + push(@output, "###############################################################"); + push(@output, "# Common parameters for all runs"); + for my $index (0 .. $#merged_params) { + next if $name_counts{$merged_params[$index]{name}} != $num_runs; + push(@output, format_series_merged_parameter_line( + $index + 1, $merged_params[$index]{name}, $merged_params[$index]{rest} + )); + } + for my $run_index (0 .. $num_runs - 1) { + push(@output, "# Specific parameters for run " . $per_run[$run_index]{run}); + for my $p (@{$per_run[$run_index]{params}}) { + next if $name_counts{$p->{name}} == $num_runs; + push(@output, format_series_merged_parameter_line( + $no_map[$run_index]{$p->{no}}, $p->{name}, $p->{rest} + )); + } + } + push(@output, "###############################################################"); + push(@output, "THEORY"); + push(@output, @merged_theory); + if (@{$first->{functions}}) { + push(@output, "###############################################################"); + push(@output, "FUNCTIONS"); + push(@output, @merged_functions); + } + push(@output, "###############################################################"); + push(@output, "GLOBAL"); + push(@output, @{$first->{global}}); + for my $run_index (0 .. $num_runs - 1) { + push(@output, "###############################################################"); + push(@output, "# RUN blocks for run " . $per_run[$run_index]{run}); + for my $block (@{$per_run[$run_index]{run_blocks}}) { + push(@output, $block->{header}); + my $had_map = 0; + for my $line (@{$block->{lines}}) { + my $trimmed = $line; + $trimmed =~ s/^\s+//; + next if $trimmed eq "" || $trimmed =~ /^#/; + my ($directive) = $trimmed =~ /^(\S+)/; + if ($directive =~ /^map$/i) { $had_map = 1; next; } + if ($SERIES_MSR_RUN_PARAM_DIRECTIVES{lc($directive)}) { + my $rest = $trimmed; + $rest =~ s/^\s*\Q$directive\E\s*//; + my @renumbered; + for my $token (split(/\s+/, $rest)) { + if ($token =~ /^\d+$/) { + my $n = 0 + $token; + push(@renumbered, defined($no_map[$run_index]{$n}) + ? $no_map[$run_index]{$n} : $token); + } else { + push(@renumbered, $token); + } + } + push(@output, $directive . " " . join(" ", @renumbered)); + } else { + push(@output, $line); + } + } + if ($had_map || $max_slot > 0) { + my $values = $merged_map_entries->($run_index, $block); + push(@output, "map " . join(" ", @{$values})) if @{$values}; + } + } + } + push(@output, "###############################################################"); + push(@output, "COMMANDS"); + push(@output, @{$first->{commands}}); + if (@{$first->{fourier}}) { + push(@output, "###############################################################"); + push(@output, "FOURIER"); + push(@output, @{series_fourier_for_display($first->{fourier})}); + } + push(@output, "###############################################################"); + push(@output, $first->{plot_header}); + for my $line (@{$first->{plot}}) { + next if $line =~ /^\s*runs\s+/i; + push(@output, $line); + } + my @plot_runs = (1 .. $num_runs); + push(@output, "runs " . join(" ", @plot_runs)); + + # musrfit always stamps a STATISTIC block into a fitted file, and msr2data's + # global data export keys each run's convergence on that block. The merged + # display MSR is not re-fitted, so carry the per-run statistics forward as + # the ordinary aggregate musrfit would write for a global fit: summed chisq + # and NDF, reduced chisq from those sums. + my ($chisq_sum, $ndf_sum) = (0, 0); + my $statistic_runs = 0; + for my $pr (@per_run) { + for my $line (@{$pr->{statistic}}) { + if ($line =~ /chisq\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i) { + my ($run_chisq, $run_ndf) = (0 + $1, 0 + $2); + next if $run_ndf <= 0; + $chisq_sum += $run_chisq; + $ndf_sum += $run_ndf; + $statistic_runs++; + } + } + } + if ($statistic_runs) { + my ($sec, $min, $hour, $mday, $mon, $year) = localtime(); + my $timestamp = sprintf("%04d-%02d-%02d %02d:%02d:%02d", + $year + 1900, $mon + 1, $mday, $hour, $min, $sec); + my $reduced = $ndf_sum > 0 ? $chisq_sum / $ndf_sum : 0; + push(@output, "###############################################################"); + push(@output, "STATISTIC --- " . $timestamp); + my $formatted_chisq = sprintf("%.4f", $chisq_sum); + my $formatted_reduced = sprintf("%.6f", $reduced); + $formatted_chisq =~ s/\.?0+$//; + $formatted_reduced =~ s/\.?0+$//; + push(@output, sprintf(" chisq = %s, NDF = %d, chisq/NDF = %s", + $formatted_chisq, $ndf_sum, $formatted_reduced)); + } + return [1, "series-merge-ready", join("\n", @output) . "\n"]; +} + # Collect a completed independent series without running musrfit again. The # requested non-empty data table and structurally valid combined MSR are the # success artifacts; msr2data's process status is retained only as a warning. @@ -1028,6 +1480,7 @@ sub perform_series_artifact_collection { my @run_numbers = (); my %seen = (); + my %source_texts = (); foreach my $value (@{$runs}) { my $run = defined($value) && !ref($value) ? "$value" : ""; return $failure->("invalid-series-collection", "Invalid series run number") @@ -1037,7 +1490,7 @@ sub perform_series_artifact_collection { return $failure->("series-artifact-source-invalid", "A fitted series MSR is missing or unsafe") if !-f $source_path || -l $source_path - || !series_msr_is_converged(read_text_file($source_path)); + || !series_msr_is_converged($source_texts{$run} = read_text_file($source_path)); push(@run_numbers, $run); } return $failure->("invalid-series-collection", "Invalid series anchor") @@ -1046,15 +1499,21 @@ sub perform_series_artifact_collection { if !-x $MSR2DATA_BIN; my $base_run = $run_numbers[0]; - my $generated_name = $base_run . "+global" . $suffix . ".msr"; - my $generated_path = "$work_path/$generated_name"; - my $combined_name = $base_run . "+series" . $suffix . ".msr"; + # The series artifacts follow the canonical multirun global names so the + # parameter plot button (which expects the ordinary global export layout) + # keeps working: the combined display file is `base+global.msr` + # and the parameter table is `base+global_par.dat`. + my $combined_name = $base_run . "+global" . $suffix . ".msr"; my $combined_path = "$work_path/$combined_name"; - my $parameter_name = $base_run . "+series" . $suffix . "_par.dat"; + # msr2data's multirun export reads its input under the `_tmpl` template + # name, so the combined runnable content is staged there for the export. + my $template_name = $base_run . "+global_tmpl.msr"; + my $template_path = "$work_path/$template_name"; + my $parameter_name = $base_run . "+global" . $suffix . "_par.dat"; my $parameter_path = "$work_path/$parameter_name"; - my $out_name = $base_run . "+series" . $suffix . ".out"; + my $out_name = $base_run . "+global" . $suffix . ".out"; my $out_path = "$work_path/$out_name"; - foreach my $stale_path ($generated_path, $combined_path, $parameter_path) { + foreach my $stale_path ($combined_path, $template_path, $parameter_path) { return $failure->("series-artifact-output-invalid", "A series output path is unsafe") if -l $stale_path || (-e $stale_path && !-f $stale_path); return $failure->("series-artifact-output-remove-failed", @@ -1068,49 +1527,56 @@ sub perform_series_artifact_collection { my @warnings = (); my $run_argument = "[" . join(" ", @run_numbers) . "]"; - my $parameter_cmd = "cd " . shell_quote($work_path) - . " && " . shell_quote($MSR2DATA_BIN) - . " " . shell_quote($run_argument) - . " " . shell_quote($suffix) - . " new global data -o " . shell_quote($parameter_name); - my $parameter_result = run_logged_command( - $out_path, "msr2data independent series collection", $parameter_cmd - ); + my @source_texts = map { $source_texts{$_} } @run_numbers; - my $parameter_url = ""; - if (-s $parameter_path) { - if (!clean_parameter_data_header($parameter_path)) { - push(@warnings, "Could not normalize the parameter-table header"); - } - # Keep the non-empty file available even when header cleanup fails. - $parameter_url = "$TMP_URL/$job_name/$parameter_name"; + # Build the combined display MSR first. It becomes the canonical + # `+global.msr` file that both musrview and the parameter export + # use, and is assembled directly from the already-fitted per-run files so + # no global re-fit ever takes place. + my $combined_url = ""; + my $merged = build_series_global_msr(\@source_texts, \@run_numbers); + if (!$merged->[0]) { + push(@warnings, "Could not assemble the combined display MSR ($merged->[1])"); + } elsif (!is_valid_generated_global_msr($merged->[2])) { + push(@warnings, "Assembled combined display MSR failed validation"); + } elsif (write_text_file($combined_path, normalize_plot_runs_line( + $merged->[2], join(" ", @run_numbers) + ))) { + $combined_url = "$TMP_URL/$job_name/$combined_name"; } else { - push(@warnings, "msr2data did not produce parameter data"); + push(@warnings, "Could not write the combined display MSR"); } - my $combined_url = ""; - if (-f $generated_path && !-l $generated_path) { - my $generated_text = read_text_file($generated_path); - if (is_valid_generated_global_msr($generated_text)) { - my $combined_text = normalize_plot_runs_line( - $generated_text, join(" ", @run_numbers) - ); - if (write_text_file($combined_path, $combined_text)) { - $combined_url = "$TMP_URL/$job_name/$combined_name"; - unlink($generated_path); - } else { - push(@warnings, "Could not write the combined display MSR"); + # msr2data is used only for the parameter table, exported with the same + # `new global data` form and `_tmpl` staging as the ordinary multirun + # export, producing the global .dat layout (stripped base names, one row + # per run) that the parameter plot button reads. + my $parameter_url = ""; + my $combined_text = $combined_url ne "" ? read_text_file($combined_path) : ""; + if ($combined_text ne "" && write_text_file($template_path, $combined_text)) { + my $parameter_cmd = "cd " . shell_quote($work_path) + . " && " . shell_quote($MSR2DATA_BIN) + . " " . shell_quote($run_argument) + . " _tmpl new global data -o " . shell_quote($parameter_name); + my $parameter_result = run_logged_command( + $out_path, "msr2data independent series collection", $parameter_cmd + ); + if (-s $parameter_path) { + if (!clean_parameter_data_header($parameter_path)) { + push(@warnings, "Could not normalize the parameter-table header"); + } + # Keep the non-empty file available even when header cleanup fails. + $parameter_url = "$TMP_URL/$job_name/$parameter_name"; + if ($parameter_result->{raw_status} != 0) { + push(@warnings, "msr2data returned code " . $parameter_result->{exit_code} + . " but produced the parameter table"); } } else { - push(@warnings, "msr2data produced an invalid combined display MSR"); + push(@warnings, "msr2data did not produce parameter data"); } - } else { - push(@warnings, "msr2data did not produce a combined display MSR"); - } - if ($parameter_result->{raw_status} != 0 - && ($parameter_url ne "" || $combined_url ne "")) { - push(@warnings, "msr2data returned code " . $parameter_result->{exit_code} - . " but produced usable series artifacts"); + unlink($template_path); + } elsif ($combined_url ne "") { + push(@warnings, "Could not stage the combined MSR for parameter export"); } my @root_urls = (); diff --git a/muSRFitelec.js b/muSRFitelec.js index e22f429..4f4fd45 100644 --- a/muSRFitelec.js +++ b/muSRFitelec.js @@ -969,10 +969,11 @@ async function runSeriesMSRCandidateLocal(receipt, candidate, context = {}) { window.runSeriesMSRCandidateLocal = runSeriesMSRCandidateLocal; -// Collect independent fitted MSRs without fitting them again. msr2data's -// generated files, rather than its historically inconsistent exit status, are -// authoritative: any non-empty requested .dat is registered immediately, and -// any structurally valid combined MSR is safe to hand to musrview. +// Collect independent fitted MSRs without fitting them again. The merged +// parameter table comes from msr2data, while the combined display MSR is +// assembled here from the per-run fitted files (msr2data only writes its +// +global file from a template). A non-empty requested .dat is registered +// immediately, and any structurally valid combined MSR is safe for musrview. async function collectIndependentSeriesArtifactsLocal(spec) { let result = { ok: false, @@ -1003,18 +1004,24 @@ async function collectIndependentSeriesArtifactsLocal(spec) { } } - let generatedName = baseRun + "+global" + suffix + ".msr"; - let generatedPath = nodePath.join(workPath, generatedName); - let combinedName = baseRun + "+series" + suffix + ".msr"; + // The series artifacts follow the canonical multirun global names so the + // parameter plot button (which expects the ordinary global export layout) + // keeps working: the combined display file is `base+global.msr` and + // the parameter table is `base+global_par.dat`. + let combinedName = baseRun + "+global" + suffix + ".msr"; let combinedPath = nodePath.join(workPath, combinedName); - let parameterName = baseRun + "+series" + suffix + "_par.dat"; + // msr2data's multirun export reads its input under the `_tmpl` template + // name, so the combined runnable content is staged there for the export. + let templateName = baseRun + "+global_tmpl.msr"; + let templatePath = nodePath.join(workPath, templateName); + let parameterName = baseRun + "+global" + suffix + "_par.dat"; let parameterPath = nodePath.join(workPath, parameterName); - let logName = baseRun + "+series" + suffix + ".out"; + let logName = baseRun + "+global" + suffix + ".out"; let logPath = nodePath.join(workPath, logName); result.logPath = logPath; try { - for (let stalePath of [generatedPath, combinedPath, parameterPath]) { + for (let stalePath of [combinedPath, templatePath, parameterPath]) { if (!nodeFs.existsSync(stalePath)) continue; if (!isSafeSeriesRegularFile(stalePath)) { result.reason = "series-artifact-output-invalid"; @@ -1026,63 +1033,93 @@ async function collectIndependentSeriesArtifactsLocal(spec) { "# muSRFlow independent series collection\n# Working directory: " + workPath + "\n", { encoding: "utf8", flag: "w" }); - let runArgument = "[" + spec.runs.join(" ") + "]"; - let args = [runArgument, suffix, "new", "global", "data", "-o", parameterName]; + let runsArg = "[" + spec.runs.join(" ") + "]"; let msr2data = resolveLocalCommand("msr2data"); - let commandLine = displayCommand(msr2data, args); - appendFitLog(logPath, "\n$ " + commandLine + "\n"); - if (typeof appendJobLog === "function") { - appendJobLog("Starting independent series collection: " + commandLine); - } let env = Object.assign({}, process.env); env.PATH = ["/usr/local/bin", "/usr/bin", "/bin", env.PATH || ""].join(":"); - let commandResult = nodeChildProcess.spawnSync(msr2data, args, { - cwd: workPath, - env: env, - encoding: "utf8", - shell: false - }); - if (commandResult.stdout) { - appendProcessOutput("independent series msr2data", commandResult.stdout, logPath); - } - if (commandResult.stderr) { - appendProcessOutput("independent series msr2data", commandResult.stderr, logPath); - } - let code = typeof commandResult.status === "number" ? commandResult.status : 1; - appendFitLog(logPath, "independent series msr2data exited with code " + code + "\n"); - - if (isSafeSeriesRegularFile(parameterPath) - && nodeFs.statSync(parameterPath).size > 0) { - try { - cleanMusrfitParameterDataHeader(parameterPath); - } catch (error) { - result.warnings.push("Could not normalize the parameter-table header."); + let runMsr2data = function(label, args) { + let commandLine = displayCommand(msr2data, args); + appendFitLog(logPath, "\n$ " + commandLine + "\n"); + if (typeof appendJobLog === "function") { + appendJobLog("Starting independent series " + label + ": " + commandLine); } - // A non-empty table is usable even if header cleanup or the external - // program's status is imperfect. - result.parameterDataPath = parameterPath; - } else { - result.warnings.push("msr2data did not produce parameter data."); + let commandResult = nodeChildProcess.spawnSync(msr2data, args, { + cwd: workPath, + env: env, + encoding: "utf8", + shell: false + }); + if (commandResult.stdout) { + appendProcessOutput("independent series " + label, commandResult.stdout, logPath); + } + if (commandResult.stderr) { + appendProcessOutput("independent series " + label, commandResult.stderr, logPath); + } + let code = typeof commandResult.status === "number" ? commandResult.status : 1; + appendFitLog(logPath, "independent series " + label + + " exited with code " + code + "\n"); + return code; + }; + + // Assemble the combined display MSR first. It becomes the canonical + // `+global.msr` file that both musrview and the parameter export + // use, and is built directly from the already-fitted per-run files so no + // global re-fit ever takes place. + let msrTexts = []; + for (let index = 0; index < spec.msrPaths.length; index++) { + if (!isSafeSeriesRegularFile(spec.msrPaths[index])) { + result.warnings.push("Fitted MSR " + spec.msrPaths[index] + + " is missing for the combined display."); + break; + } + msrTexts.push(nodeFs.readFileSync(spec.msrPaths[index], "utf8")); } - if (isSafeSeriesRegularFile(generatedPath)) { - let generatedText = nodeFs.readFileSync(generatedPath, "utf8"); - if (isValidGeneratedGlobalMSR(generatedText)) { - let combinedText = normalizeMSRPlotRuns(generatedText, {}, spec.runs); + if (msrTexts.length === spec.msrPaths.length) { + let merged = buildSeriesGlobalMSR(msrTexts, spec.runs); + if (merged.ok !== true) { + result.warnings.push("Could not assemble the combined display MSR (" + + merged.reason + ")."); + } else if (!isValidGeneratedGlobalMSR(merged.text)) { + result.warnings.push("Assembled combined display MSR failed validation."); + } else { + let combinedText = normalizeMSRPlotRuns(merged.text, {}, spec.runs); nodeFs.writeFileSync(combinedPath, combinedText, { encoding: "utf8", flag: "wx" }); result.combinedMSRPath = combinedPath; - nodeFs.unlinkSync(generatedPath); - } else { - result.warnings.push("msr2data produced an invalid combined display MSR."); + + // msr2data is used only for the parameter table, exported with the + // same `new global data` form and `_tmpl` staging as the ordinary + // multirun export, producing the global .dat layout (stripped base + // names, one row per run) that the parameter plot button reads. + nodeFs.copyFileSync(combinedPath, templatePath); + let parameterCode = runMsr2data("parameter msr2data", + [runsArg, "_tmpl", "new", "global", "data", "-o", parameterName]); + if (isSafeSeriesRegularFile(parameterPath) + && nodeFs.statSync(parameterPath).size > 0) { + try { + cleanMusrfitParameterDataHeader(parameterPath); + } catch (error) { + result.warnings.push("Could not normalize the parameter-table header."); + } + // A non-empty table is usable even if header cleanup or the + // external program's status is imperfect. + result.parameterDataPath = parameterPath; + if (parameterCode !== 0) { + result.warnings.push("msr2data returned code " + parameterCode + + " but produced the parameter table."); + } + } else { + result.warnings.push("msr2data did not produce parameter data."); + } + if (isSafeSeriesRegularFile(templatePath)) { + nodeFs.unlinkSync(templatePath); + } } } else { - result.warnings.push("msr2data did not produce a combined display MSR."); - } - if (code !== 0 && (result.parameterDataPath || result.combinedMSRPath)) { - result.warnings.push("msr2data returned code " + code - + " but produced usable series artifacts."); + result.warnings.push("Could not read all fitted per-run MSR files" + + " for the combined display."); } if (result.combinedMSRPath) { diff --git a/muSRFitlib.js b/muSRFitlib.js index 8fd81b0..7dec903 100644 --- a/muSRFitlib.js +++ b/muSRFitlib.js @@ -4130,6 +4130,23 @@ function syncParametersFromMSR(msrText) { return applied; } +// Fill the parameter table from the combined `+global` MSR that musrview +// displays. Its merged FITPARAMETER already carries the fitted values for every +// run (shared parameters once, run-specific ones under their `_RRRR` names), +// so the GUI rows are matched by exact name rather than through a THEORY model +// whose map slots are not meaningful for a multi-run table. +function syncSeriesParametersToGUI(msrText) { + let params = parseMSRParameters(msrText || ""); + if (!params || params.length === 0) return 0; + if (typeof applyMSRParametersToGUI !== "function") return 0; + let applied = applyMSRParametersToGUI(params, null); + if (applied) { + updateJobStatus({ + parameterSync: "Updated from combined series MSR (" + params.length + " parameters)" + }); + } + return applied; +} // Snapshot the parameter rows that are actually present in the GUI. Keep the // names exactly as displayed (including _runnumber suffixes); the generated MSR // decides which of them apply. @@ -4318,6 +4335,514 @@ function normalizeMSRPlotRuns(msrText, all, runs) { } +// Merge the fitted per-run MSR files produced by an independent series fit +// into a single display-only global MSR. msr2data is deliberately not used for +// this: it only writes the +global file from a template and, in every mode that +// populates per-run values, re-fits each run with musrfit. The merged file here +// keeps each run's already-fitted parameter values and lets every RUN block +// select its own parameters through the same mapN/alpha/norm mechanism that a +// genuine msr2data global file uses. +// +// The per-run files are expected to be structurally identical apart from the +// run-qualified parameter names/values and the RUN data paths (this is exactly +// what the series staging produces). Shared parameters keep a single entry in +// the merged FITPARAMETER; run-specific parameters are collected per run and +// referenced from THEORY through map slots. +const SERIES_MSR_RUN_PARAM_DIRECTIVES = { alpha: 1, norm: 1, "backgr.fit": 1, beta: 1 }; +const SERIES_MSR_MAX_MAP_SLOTS = 10; + +// Recognize a block header line. The FOURIER block uses a lowercase +// `plot ` directive that would otherwise look like a PLOT header, so a +// genuine PLOT header must carry a numeric plot id ("PLOT 2"); non-numeric +// `plot` lines stay inside FOURIER. +function seriesMSRBlockHeaderName(line) { + let match = line.match(/^(FITPARAMETER|THEORY|FUNCTIONS|GLOBAL|RUN|COMMANDS|PLOT|FOURIER|STATISTIC)(?:[ \t]|$)/i); + if (!match) return null; + let name = match[1].toUpperCase(); + if (name === "PLOT") { + let arg = line.slice(match[0].length).trim(); + if (!/^\d/.test(arg)) return null; + } + return name; +} + +function splitSeriesMSRBlocks(msrText) { + let sections = []; + let current = null; + let lines = String(msrText || "").split(/\r?\n/); + for (let line of lines) { + let name = seriesMSRBlockHeaderName(line.trim()); + if (name) { + current = { + name: name, + header: line.trim(), + lines: [] + }; + sections.push(current); + } else if (current) { + current.lines.push(line); + } else { + if (!sections.length) { + current = { name: "HEADER", header: "", lines: [] }; + sections.push(current); + } + current.lines.push(line); + } + } + return sections; +} + +function parseSeriesMSRParameters(lines) { + let params = []; + for (let line of lines) { + let trimmed = String(line).trim(); + if (!trimmed || trimmed.charAt(0) === "#") continue; + let match = trimmed.match(/^(\d+)\s+(\S+)\s*(.*)$/); + if (!match) continue; + params.push({ + no: parseInt(match[1], 10), + name: match[2], + rest: match[3].trim().split(/\s+/).join(" ") + }); + } + return params; +} + +function parseSeriesMSRRunBlocks(lines) { + let blocks = []; + let current = null; + for (let line of lines) { + if (/^RUN\b/i.test(String(line).trim())) { + current = { header: line, lines: [] }; + blocks.push(current); + } else if (current) { + current.lines.push(line); + } + } + return blocks; +} + +// Detach every THEORY/FUNCTIONS integer that refers to a fit parameter. Falls +// back to carrying the current per-run text through for anything that is not a +// simple parameter reference. Returns { ok, lines } and a lookup for the merged +// numbers of each run's local parameter positions. +function mapSeriesMergedParameters(perRun, fail) { + let numRuns = perRun.length; + let first = perRun[0]; + + let nameCounts = Object.create(null); + for (let pr of perRun) { + for (let p of pr.params) nameCounts[p.name] = (nameCounts[p.name] || 0) + 1; + } + for (let name in nameCounts) { + if (nameCounts[name] !== 1 && nameCounts[name] !== numRuns) { + return fail("series-merge-parameter-inconsistency"); + } + } + + let mergedParams = []; + let sharedNameNo = Object.create(null); + for (let p of first.params) { + if (nameCounts[p.name] === numRuns && sharedNameNo[p.name] === undefined) { + sharedNameNo[p.name] = mergedParams.length + 1; + mergedParams.push({ name: p.name, rest: p.rest }); + } + } + let noMap = perRun.map(function() { return Object.create(null); }); + for (let runIndex = 0; runIndex < numRuns; runIndex++) { + for (let p of perRun[runIndex].params) { + if (nameCounts[p.name] === numRuns) { + noMap[runIndex][p.no] = sharedNameNo[p.name]; + } else { + noMap[runIndex][p.no] = mergedParams.length + 1; + mergedParams.push({ name: p.name, rest: p.rest }); + } + } + } + return { + mergedParams: mergedParams, + nameCounts: nameCounts, + noMap: noMap, + sharedNameNo: sharedNameNo + }; +} + +// The FOURIER block is shared by all RUN blocks in the merged file. musrview +// refuses any plot other than POWER unless the phase line carries one reference +// per run (parR7 par9 ...), which a display file assembled from shared +// parameters cannot express. Mirror msr2data's own global output by forcing the +// plot to POWER; the fitted per-run Fourier phases are deliberately left to the +// individual run files. +function normalizeSeriesFourierForDisplay(fourierLines) { + let normalized = []; + let sawPlot = false; + for (let line of fourierLines) { + let match = line.trim().match(/^plot\b/i); + if (match) { + sawPlot = true; + if (!/^plot\s+POWER\b/i.test(line.trim())) { + normalized.push("plot POWER"); + } else { + normalized.push(line); + } + } else { + normalized.push(line); + } + } + if (!sawPlot) normalized.push("plot POWER"); + return normalized; +} + +function buildSeriesGlobalMSR(msrTexts, runs) { + let fail = function(reason) { + return { ok: false, reason: reason, text: "" }; + }; + if (!Array.isArray(msrTexts) || msrTexts.length < 2 + || !Array.isArray(runs) || msrTexts.length !== runs.length) { + return fail("series-merge-invalid-input"); + } + + let perRun = []; + for (let index = 0; index < msrTexts.length; index++) { + let sections = splitSeriesMSRBlocks(msrTexts[index]); + let byName = {}; + for (let section of sections) { + if (!byName[section.name]) byName[section.name] = section; + } + if (!byName.FITPARAMETER || !byName.THEORY || !byName.RUN) { + return fail("series-merge-source-incomplete"); + } + let params = parseSeriesMSRParameters(byName.FITPARAMETER.lines); + if (params.length === 0) { + return fail("series-merge-parameters-incomplete"); + } + let runBlocks = parseSeriesMSRRunBlocks( + [byName.RUN.header].concat(byName.RUN.lines) + ); + if (runBlocks.length === 0) { + return fail("series-merge-run-blocks-incomplete"); + } + perRun.push({ + run: String(runs[index]), + params: params, + runBlocks: runBlocks, + theory: byName.THEORY.lines, + functions: byName.FUNCTIONS ? byName.FUNCTIONS.lines : [], + global: byName.GLOBAL ? byName.GLOBAL.lines : [], + commands: byName.COMMANDS ? byName.COMMANDS.lines : [], + plot: byName.PLOT ? byName.PLOT.lines : [], + plotHeader: byName.PLOT ? byName.PLOT.header : "PLOT 0", + fourier: byName.FOURIER ? byName.FOURIER.lines : [], + statistic: byName.STATISTIC ? byName.STATISTIC.lines.join("\n") : "" + }); + } + + let numRuns = perRun.length; + let first = perRun[0]; + for (let pr of perRun) { + if (pr.params.length !== first.params.length) { + return fail("series-merge-parameter-count-mismatch"); + } + } + + // Every run must describe the same model: the non-numeric token shape of the + // THEORY and FUNCTIONS blocks must be identical, since parameter numbers are + // the only thing allowed to differ between fits of the same model. + let tokenShape = function(lines) { + return lines.map(function(line) { + return line.split(/\s+/).map(function(token) { + return /^\d+$/.test(token) ? "#" : token; + }).join(" "); + }).join("\n"); + }; + for (let pr of perRun) { + if (tokenShape(pr.theory) !== tokenShape(first.theory)) { + return fail("series-merge-theory-mismatch"); + } + if (tokenShape(pr.functions) !== tokenShape(first.functions)) { + return fail("series-merge-functions-mismatch"); + } + } + + let numbering = mapSeriesMergedParameters(perRun, fail); + if (!numbering.ok && numbering.ok !== undefined) return numbering; + if (numbering.reason) return numbering; + let mergedParams = numbering.mergedParams; + let nameCounts = numbering.nameCounts; + let noMap = numbering.noMap; + let sharedNameNo = numbering.sharedNameNo; + + let byNumber = perRun.map(function(pr) { + let map = Object.create(null); + for (let p of pr.params) map[p.no] = p; + return map; + }); + + // Rewrite THEORY. Existing mapN/funN tokens stay verbatim; direct integer + // references to run-specific parameters become map slots so every RUN block + // can select its own fitted value. Preserve explicit zeros as-is (musrfit + // uses 0 for "no parameter" in e.g. lifetime). + let occupiedSlots = new Set(); + let gatherSlots = function(lines) { + if (lines) { + for (let line of lines) { + for (let token of line.split(/\s+/)) { + let mapMatch = token.match(/^map(\d+)$/i); + if (mapMatch) occupiedSlots.add(parseInt(mapMatch[1], 10)); + } + } + } + }; + gatherSlots(first.theory); + gatherSlots(first.functions); + let nextSlot = 1; + while (occupiedSlots.has(nextSlot)) nextSlot++; + let slotRoleName = Object.create(null); + let slotPosition = Object.create(null); // slot -> { lineIndex, tokenIndex } + + function slotForParam(name, lineIndex, tokenIndex) { + if (slotRoleName[name] !== undefined) return slotRoleName[name]; + if (nextSlot > SERIES_MSR_MAX_MAP_SLOTS) return 0; + let slot = nextSlot++; + slotRoleName[name] = slot; + slotPosition[slot] = { lineIndex: lineIndex, tokenIndex: tokenIndex }; + return slot; + } + + let mergedTheory = first.theory.map(function(line, lineIndex) { + let out = []; + let tokenIndex = 0; + let tokens = line.split(/\s+/); + for (let token of tokens) { + if (/^\d+$/.test(token) && token !== "0") { + let localNo = parseInt(token, 10); + let p = byNumber[0][localNo]; + if (!p) return fail("series-merge-theory-param-missing"); + if (nameCounts[p.name] === numRuns) { + out.push(String(sharedNameNo[p.name])); + } else { + let slot = slotForParam(p.name, lineIndex, tokenIndex); + if (slot === 0) return fail("series-merge-map-slots-exhausted"); + out.push("map" + slot); + } + } else { + out.push(token); + } + tokenIndex++; + } + return out.join(" "); + }); + for (let line of mergedTheory) { + if (line && typeof line == "object" && line.ok === false) return fail(line.reason); + } + + // FUNCTIONS is shared by all runs, so it may only reference parameters that + // keep the same number across every RUN block. a function that multiplies + // two run-specific parameters would otherwise force a different curve per + // run block, which the global file cannot express. + let mergedFunctions = first.functions.map(function(line) { + let out = []; + for (let token of line.split(/\s+/)) { + let parMatch = token.match(/^par(\d+)$/i); + if (parMatch) { + let localNo = parseInt(parMatch[1], 10); + let p = byNumber[0][localNo]; + if (!p) return fail("series-merge-functions-param-missing"); + if (nameCounts[p.name] !== numRuns) { + return fail("series-merge-functions-run-specific"); + } + out.push("par" + sharedNameNo[p.name]); + } else { + out.push(token); + } + } + return out.join(" "); + }); + for (let line of mergedFunctions) { + if (line && typeof line == "object" && line.ok === false) return fail(line.reason); + } + + // Rebuild RUN blocks. RUN headers and the non-parameter directives are kept + // verbatim; parameter-number directives are renumbered and a map line covers + // the THEORY role slots resolved above. Nested function arguments must refer + // to the same shared parameter numbering, so FUNCTIONS is emitted untouched + // apart from the parN renumbering. + let maxSlot = 0; + for (let slot in slotPosition) maxSlot = Math.max(maxSlot, parseInt(slot, 10)); + + function mergedMapEntries(runIndex, block) { + let existing = null; + let mapLineIndex = -1; + for (let i = 0; i < block.lines.length; i++) { + if (/^map\b/i.test(block.lines[i].trim())) { mapLineIndex = i; break; } + } + if (mapLineIndex >= 0) { + existing = block.lines[mapLineIndex].trim().split(/\s+/).slice(1); + } + let entries = {}; + let slotMax = Math.max(maxSlot, existing ? existing.length : 0); + for (let slot = 1; slot <= slotMax; slot++) { + let value = 0; + if (existing && existing[slot - 1] && /^\d+$/.test(existing[slot - 1])) { + let localNo = parseInt(existing[slot - 1], 10); + value = (localNo > 0 && noMap[runIndex][localNo] !== undefined) + ? noMap[runIndex][localNo] : localNo; + } else if (slotPosition[slot]) { + let position = slotPosition[slot]; + let localTokens = perRun[runIndex].theory[position.lineIndex] + ? perRun[runIndex].theory[position.lineIndex].split(/\s+/) + : []; + let localToken = localTokens[position.tokenIndex]; + let localNo = /^\d+$/.test(localToken || "") ? parseInt(localToken, 10) : 0; + if (localNo > 0 && noMap[runIndex][localNo] !== undefined) { + value = noMap[runIndex][localNo]; + } + } + entries[slot] = value; + } + let values = []; + for (let slot = 1; slot <= slotMax; slot++) values.push(String(entries[slot])); + while (values.length && values[values.length - 1] === "0") values.pop(); + return { values: values, mapLineIndex: mapLineIndex }; + } + + let output = []; + output.push("# Independent series display MSR for the runs: " + runs.join(" ")); + output.push("# Fitted per-run parameters collected without re-fitting; " + + "msr2data handles parameter extraction separately."); + output.push("###############################################################"); + output.push("FITPARAMETER"); + output.push("###############################################################"); + output.push("# Common parameters for all runs"); + for (let i = 0; i < mergedParams.length; i++) { + if (nameCounts[mergedParams[i].name] !== numRuns) continue; + output.push(formatSeriesMergedParameterLine(i + 1, mergedParams[i].name, mergedParams[i].rest)); + } + for (let runIndex = 0; runIndex < numRuns; runIndex++) { + output.push("# Specific parameters for run " + perRun[runIndex].run); + for (let p of perRun[runIndex].params) { + if (nameCounts[p.name] === numRuns) continue; + output.push(formatSeriesMergedParameterLine(noMap[runIndex][p.no], p.name, p.rest)); + } + } + output.push("###############################################################"); + output.push("THEORY"); + output.push(mergedTheory.join("\n")); + if (first.functions.length) { + output.push("###############################################################"); + output.push("FUNCTIONS"); + output.push(mergedFunctions.join("\n")); + } + output.push("###############################################################"); + output.push("GLOBAL"); + output.push(first.global.length ? first.global.join("\n") : "fittype 0"); + for (let runIndex = 0; runIndex < numRuns; runIndex++) { + output.push("###############################################################"); + output.push("# RUN blocks for run " + perRun[runIndex].run); + for (let block of perRun[runIndex].runBlocks) { + output.push(block.header); + let hadMap = false; + let hadSlot = maxSlot > 0; + for (let line of block.lines) { + let trimmed = line.trim(); + let directive = trimmed.split(/\s+/)[0]; + if (!directive || trimmed.charAt(0) === "#") { + output.push(line); + continue; + } + if (/^map$/i.test(directive)) { hadMap = true; continue; } + if (SERIES_MSR_RUN_PARAM_DIRECTIVES[directive.toLowerCase()]) { + let rest = trimmed.slice(directive.length).trim(); + let renumbered = rest.split(/\s+/).map(function(token) { + if (/^\d+$/.test(token)) { + let n = parseInt(token, 10); + return (n > 0 && noMap[runIndex][n] !== undefined) + ? String(noMap[runIndex][n]) : token; + } + return token; + }).join(" "); + output.push(directive + " " + renumbered); + } else { + output.push(line); + } + } + if (hadMap || hadSlot) { + let mapInfo = mergedMapEntries(runIndex, block); + if (mapInfo.values.length) { + output.push("map " + mapInfo.values.join(" ")); + } + } + } + } + output.push("###############################################################"); + output.push("COMMANDS"); + output.push(first.commands.length ? first.commands.join("\n") : "MINIMIZE"); + if (first.fourier.length) { + output.push("###############################################################"); + output.push("FOURIER"); + output.push(normalizeSeriesFourierForDisplay(first.fourier).join("\n")); + } + output.push("###############################################################"); + output.push(first.plotHeader); + for (let line of first.plot) { + if (/^runs\s+/i.test(line.trim())) continue; + output.push(line); + } + let plotRuns = []; + for (let i = 1; i <= numRuns; i++) plotRuns.push(String(i)); + output.push("runs " + plotRuns.join(" ")); + + // musrfit always stamps a STATISTIC block into a fitted file, and msr2data's + // global data export keys each run's convergence on that block. The merged + // display MSR is not re-fitted, so carry the per-run statistics forward as + // the ordinary aggregate musrfit would write for a global fit: summed chisq + // and NDF, reduced chisq from those sums. + let statistics = []; + for (let pr of perRun) { + let match = pr.statistic.match( + /chisq\s*=\s*([^\s,]+).*?NDF\s*=\s*([^\s,]+)/i + ); + if (!match) continue; + let runChiSq = parseFloat(match[1]); + let runNDF = parseFloat(match[2]); + if (isNaN(runChiSq) || isNaN(runNDF) || runNDF <= 0) continue; + statistics.push({ chiSq: runChiSq, ndf: runNDF }); + } + if (statistics.length) { + let chiSq = 0; + let ndf = 0; + for (let s of statistics) { chiSq += s.chiSq; ndf += s.ndf; } + let date = new Date(); + let timestamp = date.getFullYear() + "-" + + String(date.getMonth() + 1).padStart(2, "0") + "-" + + String(date.getDate()).padStart(2, "0") + " " + + String(date.getHours()).padStart(2, "0") + ":" + + String(date.getMinutes()).padStart(2, "0") + ":" + + String(date.getSeconds()).padStart(2, "0"); + let formattedChiSq = parseFloat(chiSq.toFixed(4)).toString(); + let formattedNDF = Math.round(ndf).toString(); + let formattedReduced = parseFloat((chiSq / ndf).toFixed(6)).toString(); + output.push("###############################################################"); + output.push("STATISTIC --- " + timestamp); + output.push(" chisq = " + formattedChiSq + ", NDF = " + formattedNDF + + ", chisq/NDF = " + formattedReduced); + } + + return { + ok: true, + reason: "series-merge-ready", + text: output.join("\n") + "\n" + }; +} + +function formatSeriesMergedParameterLine(no, name, rest) { + let line = " " + no + " " + name; + if (rest && rest !== "") { + line += " " + rest; + } + return line; +} + function applyFittedMSRToGUI(msrText, filePath, options = {}) { if (!msrText || typeof msrText !== "string") return msrText; @@ -8953,7 +9478,7 @@ async function collectIndependentSeriesArtifacts(plan, config, webMode) { updateJobStatus({ state: "Collecting series", message: "Collecting fitted-run parameters.", - messageDetail: "Running msr2data on the fitted and reused independent-run MSRs.", + messageDetail: "Running msr2data for the merged parameter table and assembling the combined display MSR.", noticeLevel: "info" }); appendJobLog("Collecting " + spec.runs.length @@ -9306,6 +9831,20 @@ async function runIndependentSeriesFit(execution, webMode) { let artifacts = await collectIndependentSeriesArtifacts(plan, Object.assign({}, config, { fftv: execution && execution.fftv ? execution.fftv : "n" }), webMode); + if (artifacts.combinedMSRPath && typeof readAsciiFile === "function") { + // The parameter table must reflect every run's fitted values, not just + // the last one. The combined MSR used for the musrview display already + // contains them, so reuse it instead of parsing per-run files. + let combinedText = await readAsciiFile(artifacts.combinedMSRPath); + if (combinedText && typeof combinedText === "string") { + try { + syncSeriesParametersToGUI(combinedText); + } catch (error) { + artifacts.warnings.push("Could not update the parameter table from " + + "the combined series MSR."); + } + } + } let message = sequence.reason == "series-already-complete" ? "All selected runs are already fitted." : "Independent series fit completed: " + fitted + " fitted, " diff --git a/package.json b/package.json index bafdd35..efec987 100644 --- a/package.json +++ b/package.json @@ -10,7 +10,7 @@ "dist:linux": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux rpm AppImage --x64", "dist:rpm": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux rpm --x64", "dist:appimage": "npm run test:electron && npx --yes electron-builder@26.15.3 --linux AppImage --x64", - "test:electron": "node tests/log-highlighting.test.js && node tests/action-state.test.js && node tests/electron-missing-data.test.js && node tests/window-layering.test.js && node tests/msr-generation.test.js && node tests/web-runner.test.js && node tests/root-plot-clean.test.js && node tests/parameter-plot.test.js && node tests/help-window.test.js && node tests/release-packaging.test.js && node tests/addrun.test.js && node tests/fit-statistic.test.js && node tests/series-fit-planner.test.js && node tests/series-fit-result.test.js && node tests/series-fit-inventory.test.js && node tests/series-fit-msr-preparation.test.js && node tests/series-fit-staging.test.js && node tests/series-fit-runner.test.js && node tests/series-runner-adapters.test.js && node tests/series-fit-routing.test.js && node tests/singlehist-parity.test.js && node tests/url-parameters.test.js && node tests/shared-configuration.test.js" + "test:electron": "node tests/log-highlighting.test.js && node tests/action-state.test.js && node tests/electron-missing-data.test.js && node tests/window-layering.test.js && node tests/msr-generation.test.js && node tests/web-runner.test.js && node tests/root-plot-clean.test.js && node tests/parameter-plot.test.js && node tests/help-window.test.js && node tests/release-packaging.test.js && node tests/addrun.test.js && node tests/fit-statistic.test.js && node tests/series-fit-planner.test.js && node tests/series-fit-result.test.js && node tests/series-fit-inventory.test.js && node tests/series-fit-msr-preparation.test.js && node tests/series-global-merge.test.js && node tests/series-fit-staging.test.js && node tests/series-fit-runner.test.js && node tests/series-runner-adapters.test.js && node tests/series-fit-routing.test.js && node tests/singlehist-parity.test.js && node tests/url-parameters.test.js && node tests/shared-configuration.test.js" }, "repository": { "type": "git", diff --git a/tests/cgi-series-collection.test.js b/tests/cgi-series-collection.test.js index 05cc7d6..7d04bfd 100644 --- a/tests/cgi-series-collection.test.js +++ b/tests/cgi-series-collection.test.js @@ -25,6 +25,13 @@ function fittedMSR(run) { "MIGRAD", "PLOT 2", "runs 1", + "FOURIER", + "units Gauss", + "fourier_power 12", + "dc-corrected FALSE", + "apodization STRONG", + "plot PHASE", + "phase 0.0", "STATISTIC --- 2026-08-30 12:00:00", "chisq = 101.2, NDF = 100, chisq/NDF = 1.012", "" @@ -45,10 +52,8 @@ try { const fakeMusrview = path.join(testRoot, "musrview"); fs.writeFileSync(fakeMsr2data, [ "#!/bin/sh", - "printf '%s\\n' \"$*\" > \"$MUSRFLOW_TEST_M2D_ARGS\"", - "printf 'RUN Asy_ Asy_PosErr\\n514 0.2 0.01\\n515 0.21 0.01\\n' > 514+series_asy_par.dat", - "cp 514_asy.msr 514+global_asy.msr", - "printf 'RUN /data/run515 MUE4 PSI MUSR-ROOT\\n' >> 514+global_asy.msr", + "printf '%s\\n' \"$*\" >> \"$MUSRFLOW_TEST_M2D_ARGS\"", + "printf 'RUN Asy_ Asy_PosErr\\n514 0.2 0.01\\n515 0.21 0.01\\n' > 514+global_asy_par.dat", "exit 9", "" ].join("\n")); @@ -56,7 +61,7 @@ try { fs.writeFileSync(fakeMusrview, [ "#!/bin/sh", "printf '%s\\n' \"$*\" > \"$MUSRFLOW_TEST_VIEW_ARGS\"", - ": > 514+series_asy_0.root", + ": > 514+global_asy_0.root", "exit 0", "" ].join("\n")); @@ -93,22 +98,44 @@ try { const responseText = execution.stdout.split(/\r?\n\r?\n/).slice(1).join("\n\n"); const response = JSON.parse(responseText); assert.strictEqual(response.ok, true); - assert.strictEqual(response.parameter_url, "/tmp/seriesjob/514+series_asy_par.dat"); - assert.strictEqual(response.msr_url, "/tmp/seriesjob/514+series_asy.msr"); + assert.strictEqual(response.parameter_url, "/tmp/seriesjob/514+global_asy_par.dat"); + assert.strictEqual(response.msr_url, "/tmp/seriesjob/514+global_asy.msr"); assert.deepStrictEqual(response.root_urls, - ["/tmp/seriesjob/514+series_asy_0.root"]); + ["/tmp/seriesjob/514+global_asy_0.root"]); assert.match(response.warnings.join("\n"), /returned code 9/); - assert.strictEqual(fs.readFileSync(m2dArgsPath, "utf8").trim(), - "[514 515] _asy new global data -o 514+series_asy_par.dat"); + const m2dArgs = fs.readFileSync(m2dArgsPath, "utf8").trim().split(/\r?\n/); + assert.strictEqual(m2dArgs.length, 1); + assert.strictEqual(m2dArgs[0], + "[514 515] _tmpl new global data -o 514+global_asy_par.dat"); assert.strictEqual(fs.readFileSync(viewArgsPath, "utf8").trim(), - "--timeout 300 --root 514+series_asy.msr"); + "--timeout 300 --root 514+global_asy.msr"); assert.strictEqual(fs.readFileSync( - path.join(workPath, "514+series_asy_par.dat"), "utf8" + path.join(workPath, "514+global_asy_par.dat"), "utf8" ).split(/\r?\n/)[0], "RUN Asy Asy_PosErr"); - assert.match(fs.readFileSync(path.join(workPath, "514+series_asy.msr"), "utf8"), - /^runs\s+1\s+2$/m); - assert.strictEqual(fs.existsSync(path.join(workPath, "514+global_asy.msr")), false); + const combinedText = fs.readFileSync(path.join(workPath, "514+global_asy.msr"), "utf8"); + assert.match(combinedText, /^runs\s+1\s+2$/m); + assert.match(combinedText, /^# Common parameters for all runs$/m); + assert.match(combinedText, /^# Specific parameters for run 514$/m); + assert.match(combinedText, /^ 1 Asy_514 0\.2 0\.01$/m); + assert.match(combinedText, /^ 2 Asy_515 0\.2 0\.01$/m); + assert.match(combinedText, /^asymmetry map1$/m); + const fourier = combinedText.slice(combinedText.indexOf("FOURIER")); + assert.match(fourier, /^plot\s+POWER$/m); + assert.doesNotMatch(fourier, /^plot\s+PHASE$/mi); + assert.match(fourier, /^units\s+Gauss$/m); + assert.match(fourier, /^phase\s+0\.0$/m); + assert.match(fourier, /^apodization\s+STRONG$/m); + // The merge aggregates each per-run STATISTIC block into the combined MSR so + // msr2data's `new global data` export considers every run converged. + assert.match(combinedText, /^STATISTIC --- \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}$/m); + assert.match(combinedText, /^ chisq = 202\.4, NDF = 200, chisq\/NDF = 1\.012$/m); + // The combined display MSR now carries the canonical +global name, and the + // msr2data `_tmpl` staging file is removed after the export. + assert.strictEqual(fs.existsSync(path.join(workPath, "514+global_asy.msr")), true); + assert.strictEqual(fs.existsSync(path.join(workPath, "514+global_tmpl.msr")), false); + assert.strictEqual(fs.existsSync(path.join(workPath, "514+series_asy.msr")), false); assert.match(response.command_log, /msr2data independent series collection/); + assert.doesNotMatch(response.command_log, /msr2data independent series display/); assert.match(response.command_log, /\[exit code 9\]/); } finally { fs.rmSync(testRoot, { recursive: true, force: true }); diff --git a/tests/msr-generation.test.js b/tests/msr-generation.test.js index 3f0cce5..afa3114 100644 --- a/tests/msr-generation.test.js +++ b/tests/msr-generation.test.js @@ -1902,6 +1902,63 @@ function testApplyMSRParametersUpdatesParameterTable(lib) { assert.strictEqual(elements.maxAlpha.value, "1.5"); } +function testSyncSeriesParametersFromCombinedMSR(lib) { + const elements = { + valAsy_0514: { value: "old-514" }, + errAsy_0514: { value: "old-err" }, + valAsy_0515: { value: "old-515" }, + errAsy_0515: { value: "old-err" }, + valzeta: { value: "old-zeta" }, + errzeta: { value: "old-err" } + }; + let cfgParamsCalled = 0; + const statusUpdates = []; + + lib.document = { + getElementById: function(id) { + return elements[id] || null; + } + }; + lib.cfgParams = function() { + cfgParamsCalled++; + }; + lib.appendJobLog = function() {}; + lib.updateJobStatus = function(values) { + statusUpdates.push(values); + }; + lib.All.runNumbers = "514,515"; + + const combinedText = [ + "FITPARAMETER", + " 1 zeta 0.10 0.001 none", + " 2 Asy_0514 0.213 0.002 none", + " 3 Asy_0515 0.255 0.003 none", + "THEORY", + "asymmetry map1", + "GLOBAL", + "fittype 2", + "RUN /data/run514 MUE4 PSI MUSR-ROOT", + "COMMANDS", + "MIGRAD", + "PLOT 2", + "runs 1 2", + "" + ].join("\n"); + const result = lib.syncSeriesParametersToGUI(combinedText); + + assert.strictEqual(result, 1); + assert.strictEqual(cfgParamsCalled, 1); + assert.strictEqual(lib.All.valzeta, "0.10"); + assert.strictEqual(lib.All.valAsy_0514, "0.213"); + assert.strictEqual(lib.All.valAsy_0515, "0.255"); + assert.strictEqual(elements.valAsy_0514.value, "0.213"); + assert.strictEqual(elements.valAsy_0515.value, "0.255"); + assert.strictEqual(elements.valzeta.value, "0.10"); + assert.strictEqual(statusUpdates.some(function(update) { + return /combined series MSR \(3 parameters\)/.test(update.parameterSync || ""); + }), true); +} + function testDirtyEditorOverwriteGuard(lib) { resetMSREditorDocuments(lib); let confirmMessage = ""; @@ -2666,6 +2723,7 @@ testCommonT0BgValuesAppliedToGlobalMSR(lib); testT0BgValidationRejectsIncompletePairs(lib); testT0BgUseCommonIgnoresStaleRowBackground(lib); testApplyMSRParametersUpdatesParameterTable(lib); +testSyncSeriesParametersFromCombinedMSR(lib); testDirtyEditorOverwriteGuard(lib); testLoadMSRIntoEditorClearsDirtyAndUpdatesTable(lib); testMSREditorSourceTabsKeepInactiveEdits(lib); diff --git a/tests/series-fit-routing.test.js b/tests/series-fit-routing.test.js index 43c7ba4..eb1971d 100644 --- a/tests/series-fit-routing.test.js +++ b/tests/series-fit-routing.test.js @@ -228,8 +228,8 @@ async function testMissingReferenceBootstrapsFirstRun() { artifactSpec = spec; return { ok: true, - parameterDataPath: "/work/514+series_asy_par.dat", - combinedMSRPath: "/work/514+series_asy.msr", + parameterDataPath: "/work/514+global_asy_par.dat", + combinedMSRPath: "/work/514+global_asy.msr", plotStarted: true, warnings: [] }; diff --git a/tests/series-global-merge.test.js b/tests/series-global-merge.test.js new file mode 100644 index 0000000..1e16baf --- /dev/null +++ b/tests/series-global-merge.test.js @@ -0,0 +1,317 @@ +// Copyright (C) 2026 Zaher Salman +// SPDX-License-Identifier: GPL-3.0-or-later + +const assert = require("assert"); +const fs = require("fs"); +const path = require("path"); +const vm = require("vm"); + +const repoRoot = path.resolve(__dirname, ".."); + +function loadLibrary() { + const code = fs.readFileSync(path.join(repoRoot, "muSRFitlib.js"), "utf8"); + const sandbox = { + console: console, + window: { MSREditor: {} } + }; + sandbox.global = sandbox; + vm.runInNewContext(code, sandbox, { filename: "muSRFitlib.js" }); + return sandbox; +} + +function lemToken(run) { + return "/data/lem/his/2026/lem26_his_" + String(run).padStart(4, "0"); +} + +// A realistic staged per-run asymmetry file: Alpha/Asy/Lam run-qualified, +// theorical parameter references via numbers, RUN alpha + histogram grouping. +function stagedAsymmetryMSR(run, value) { + const suffix = String(run).padStart(4, "0"); + return [ + "TITLE staged run " + run, + "FITPARAMETER", + "1 Alpha_" + suffix + " 1.2 0.01 none", + "2 Asy_" + suffix + " " + value + " 0.001 none", + "3 Lam_" + suffix + " 0.45 0.01 none", + "THEORY", + "asymmetry 2", + "simplExpo 3", + "GLOBAL", + "fittype 2", + "fit 0.02 10", + "RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT", + "alpha 1", + "forward 1 5", + "backward 3 7", + "t0 1000 1000 2000 2000", + "COMMANDS", + "MIGRAD", + "PLOT 2", + "runs 1", + "STATISTIC --- 2026-08-30 12:00:00", + "chisq = 101.2, NDF = 100, chisq/NDF = 1.012000", + "" + ].join("\n"); +} + +const sanctuaryCase = function(name, assertion) { + try { + assertion(); + console.log(" ok - " + name); + } catch (error) { + error.message = name + ": " + error.message; + throw error; + } +}; + +function testMergesTwoAsymmetryRuns(lib) { + const result = lib.buildSeriesGlobalMSR( + [stagedAsymmetryMSR(514, "0.213"), stagedAsymmetryMSR(515, "0.255")], + [514, 515] + ); + assert.strictEqual(result.ok, true); + assert.strictEqual(result.reason, "series-merge-ready"); + assert.strictEqual(lib.isValidGeneratedGlobalMSR(result.text), true); + const normalized = lib.normalizeMSRPlotRuns(result.text, {}, [514, 515]); + + sanctuaryCase("combined file is structurally valid", function() { + assert.strictEqual(lib.isValidGeneratedGlobalMSR(normalized), true); + }); + sanctuaryCase("runs select the right indices", function() { + assert.match(normalized, /^runs\s+1\s+2$/m); + }); + sanctuaryCase("shared parameters block is empty for all run-specific", function() { + const specific = normalized.split(/\r?\n/).filter(function(line) { + return /^\s*\d+\s+/.test(line); + }); + assert.deepStrictEqual(specific, [ + " 1 Alpha_0514 1.2 0.01 none", + " 2 Asy_0514 0.213 0.001 none", + " 3 Lam_0514 0.45 0.01 none", + " 4 Alpha_0515 1.2 0.01 none", + " 5 Asy_0515 0.255 0.001 none", + " 6 Lam_0515 0.45 0.01 none" + ]); + }); + sanctuaryCase("theory maps run-specific roles", function() { + assert.match(normalized, /^asymmetry map1$/m); + assert.match(normalized, /^simplExpo map2$/m); + }); + sanctuaryCase("each run block keeps alpha and map pointing at its own params", function() { + assert.match(normalized, new RegExp( + "^RUN\\s+" + lemToken(514), "m")); + assert.match(normalized, /^alpha 1$/m); + assert.match(normalized, /^map 2 3$/m); + assert.match(normalized, /^forward 1 5$/m); + assert.match(normalized, /^backward 3 7$/m); + assert.match(normalized, /^t0 1000 1000 2000 2000$/m); + assert.match(normalized, /^alpha 4$/m); + assert.match(normalized, /^map 5 6$/m); + }); + sanctuaryCase("per-run statistics are aggregated for the msr2data export", function() { + // Both staged runs report chisq=101.2, NDF=100, so the merged file must + // carry the aggregate block musrfit would write for a global fit. + assert.match(normalized, /^STATISTIC --- \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}$/m); + assert.match(normalized, /^ chisq = 202\.4, NDF = 200, chisq\/NDF = 1\.012$/m); + }); +} + +function testMergesSharedAndRunSpecificParameters(lib) { + const make = function(run) { + const suffix = String(run).padStart(4, "0"); + return [ + "FITPARAMETER", + "1 zeta 0.1 0.001 none", + "2 Asy_" + suffix + " 0.3 0.01 none", + "THEORY", + "asymmetry map1", + "zetaExp 1", + "GLOBAL", + "fittype 2", + "RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT", + "alpha 2", + "map 2", + "forward 4", + "backward 6", + "COMMANDS", + "MIGRAD", + "PLOT 2", + "runs 1", + "" + ].join("\n"); + }; + const result = lib.buildSeriesGlobalMSR([make(516), make(517)], [516, 517]); + assert.strictEqual(result.ok, true); + sanctuaryCase("shared zeta keeps one entry, runs keep their own Asy", function() { + assert.match(result.text, /^ 1 zeta 0\.1 0\.001 none$/m); + assert.match(result.text, /^ 2 Asy_0516 0\.3 0\.01 none$/m); + assert.match(result.text, /^ 3 Asy_0517 0\.3 0\.01 none$/m); + assert.strictEqual((result.text.match(/zeta/g) || []).length, 2); + }); + sanctuaryCase("theory keeps shared numbers and fills run-specific map slots", function() { + assert.match(result.text, /^zetaExp 1$/m); + assert.match(result.text, /^asymmetry map1$/m); + assert.match(result.text, /^map 2$/m); + assert.match(result.text, /^map 3$/m); + assert.match(result.text, /^alpha 2$/m); + assert.match(result.text, /^alpha 3$/m); + }); +} + +function testFourierForDisplay(lib) { + const make = function(run, plotLine) { + const suffix = String(run).padStart(4, "0"); + const lines = [ + "FITPARAMETER", + "1 Alpha_" + suffix + " 1.2 0.01 none", + "THEORY", + "asymmetry 1", + "GLOBAL", + "fittype 2", + "RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT", + "alpha 1", + "forward 1 5", + "COMMANDS", + "MIGRAD", + "PLOT 2", + "runs 1", + "FOURIER", + "units Gauss", + "fourier_power 12", + "dc-corrected FALSE", + "apodization STRONG" + ]; + if (plotLine) lines.push("plot " + plotLine); + lines.push("phase 0.0", ""); + return lines.join("\n"); + }; + + const phaseResult = lib.buildSeriesGlobalMSR( + [make(514, "PHASE"), make(515, "PHASE")], [514, 515] + ); + assert.strictEqual(phaseResult.ok, true); + sanctuaryCase("non-POWER FOURIER plot is forced to POWER in the merged file", function() { + const fourier = phaseResult.text.slice(phaseResult.text.indexOf("FOURIER")); + assert.match(fourier, /^plot\s+POWER$/m); + assert.doesNotMatch(fourier, /^plot\s+PHASE$/mi); + assert.match(fourier, /^phase\s+0\.0$/m); + assert.match(fourier, /^fourier_power\s+12$/m); + assert.strictEqual((fourier.match(/plot POWER/g) || []).length, 1); + }); + + const powerResult = lib.buildSeriesGlobalMSR( + [make(516, "POWER"), make(517, "POWER")], [516, 517] + ); + assert.strictEqual(powerResult.ok, true); + sanctuaryCase("POWER FOURIER plot passes through unchanged", function() { + assert.match(powerResult.text, /^plot\s+POWER$/m); + }); + + const noPlotResult = lib.buildSeriesGlobalMSR( + [make(518, null), make(519, null)], [518, 519] + ); + assert.strictEqual(noPlotResult.ok, true); + sanctuaryCase("missing FOURIER plot gets POWER added", function() { + assert.match(noPlotResult.text, /^plot\s+POWER$/m); + }); +} + +function testMergingErrors(lib) { + const results = {}; + + results.input = lib.buildSeriesGlobalMSR([stagedAsymmetryMSR(514, "0.2")], [514]); + assert.strictEqual(results.input.ok, false); + assert.strictEqual(results.input.reason, "series-merge-invalid-input"); + + const brokenSecond = stagedAsymmetryMSR(515, "0.3").replace("simplExpo 3\n", "simplExpo 3\nlifetime 4\n"); + results.theory = lib.buildSeriesGlobalMSR( + [stagedAsymmetryMSR(514, "0.2"), brokenSecond], [514, 515] + ); + assert.strictEqual(results.theory.ok, false); + assert.strictEqual(results.theory.reason, "series-merge-theory-mismatch"); + + const sourceWithoutRun = stagedAsymmetryMSR(515, "0.3").replace( + /^RUN .*(\n.*)*?(?=^COMMANDS)/m, "" + ); + results.source = lib.buildSeriesGlobalMSR( + [stagedAsymmetryMSR(514, "0.2"), sourceWithoutRun], [514, 515] + ); + assert.strictEqual(results.source.ok, false); + assert.strictEqual(results.source.reason, "series-merge-source-incomplete"); + + // A function that multiplies run-specific Asy cannot be shared across runs. + const functionSource = function(run) { + const suffix = String(run).padStart(4, "0"); + return [ + "FITPARAMETER", + "1 Left_" + suffix + " 0.1 0.01 none", + "2 Right_" + suffix + " 0.2 0.01 none", + "THEORY", + "asymmetry fun1", + "GLOBAL", + "fittype 2", + "RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT", + "alpha 1", + "forward 2", + "backward 4", + "FUNCTIONS", + "fun1 = par1 * par2", + "COMMANDS", + "MIGRAD", + "PLOT 2", + "runs 1", + "" + ].join("\n"); + }; + results.functionRunSpecific = lib.buildSeriesGlobalMSR( + [functionSource(518), functionSource(519)], [518, 519] + ); + assert.strictEqual(results.functionRunSpecific.ok, false); + assert.strictEqual(results.functionRunSpecific.reason, "series-merge-functions-run-specific"); + + // But a function over a shared parameter stays valid and is renumbered. + const sharedFunctionSource = function(run) { + const suffix = String(run).padStart(4, "0"); + return [ + "FITPARAMETER", + "1 zeta 0.1 0.001 none", + "2 Phase_" + suffix + " 0.5 0.01 none", + "THEORY", + "asymmetry fun1", + "GLOBAL", + "fittype 2", + "RUN " + lemToken(run) + " MUE4 PSI MUSR-ROOT", + "alpha 2", + "forward 1", + "backward 3", + "FUNCTIONS", + "fun1 = par1 * par1", + "COMMANDS", + "MIGRAD", + "PLOT 2", + "runs 1", + "" + ].join("\n"); + }; + const sharedFunctionMerge = lib.buildSeriesGlobalMSR( + [sharedFunctionSource(518), sharedFunctionSource(519)], [518, 519] + ); + assert.strictEqual(sharedFunctionMerge.ok, true); + sanctuaryCase("shared-parameter function is renumbered", function() { + assert.match(sharedFunctionMerge.text, /^fun1 = par1 \* par1$/m); + assert.match(sharedFunctionMerge.text, /^asymmetry fun1$/m); + assert.match(sharedFunctionMerge.text, /^ 1 zeta 0\.1 0\.001 none$/m); + }); +} + +(async function() { + const lib = loadLibrary(); + testMergesTwoAsymmetryRuns(lib); + testMergesSharedAndRunSpecificParameters(lib); + testFourierForDisplay(lib); + testMergingErrors(lib); + console.log("series global merge tests passed"); +})().catch(function(error) { + console.error(error); + process.exit(1); +}); \ No newline at end of file diff --git a/tests/series-runner-adapters.test.js b/tests/series-runner-adapters.test.js index 80a7c24..1b964c2 100644 --- a/tests/series-runner-adapters.test.js +++ b/tests/series-runner-adapters.test.js @@ -210,14 +210,9 @@ async function testElectronSeriesArtifactCollection() { const fakeChildProcess = { spawnSync: function(command, args, options) { syncCalls.push({ command: command, args: args.slice(), cwd: options.cwd }); - fs.writeFileSync(path.join(options.cwd, "514+global_asy.msr"), - runnableMSR(514, false).replace( - "RUN /data/run514 MUE4 PSI MUSR-ROOT", - "RUN /data/run514 MUE4 PSI MUSR-ROOT\nRUN /data/run515 MUE4 PSI MUSR-ROOT" - ), "utf8"); - fs.writeFileSync(path.join(options.cwd, "514+series_asy_par.dat"), + fs.writeFileSync(path.join(options.cwd, "514+global_asy_par.dat"), "RUN Asy_ Asy_PosErr\n514 0.2 0.01\n515 0.21 0.01\n", "utf8"); - return { status: 7, stdout: "artifacts written\n", stderr: "" }; + return { status: 7, stdout: "parameter data written\n", stderr: "" }; }, spawn: function(command, args, options) { asyncCalls.push({ command: command, args: args.slice(), cwd: options.cwd }); @@ -243,21 +238,29 @@ async function testElectronSeriesArtifactCollection() { assert.strictEqual(result.ok, true); assert.strictEqual(result.parameterDataPath, - path.join(tempDirectory, "514+series_asy_par.dat")); + path.join(tempDirectory, "514+global_asy_par.dat")); assert.strictEqual(result.combinedMSRPath, - path.join(tempDirectory, "514+series_asy.msr")); + path.join(tempDirectory, "514+global_asy.msr")); assert.strictEqual(result.plotStarted, true); assert.match(result.warnings.join("\n"), /returned code 7/); + assert.strictEqual(syncCalls.length, 1); assert.deepStrictEqual(Array.from(syncCalls[0].args), - ["[514 515]", "_asy", "new", "global", "data", "-o", - "514+series_asy_par.dat"]); - assert.deepStrictEqual(Array.from(asyncCalls[0].args), ["514+series_asy.msr"]); + ["[514 515]", "_tmpl", "new", "global", "data", "-o", "514+global_asy_par.dat"]); + assert.deepStrictEqual(Array.from(asyncCalls[0].args), ["514+global_asy.msr"]); assert.match(fs.readFileSync(result.parameterDataPath, "utf8"), /^RUN Asy Asy_PosErr/m); - assert.match(fs.readFileSync(result.combinedMSRPath, "utf8"), - /^runs\s+1\s+2$/m); - assert.strictEqual(fs.existsSync(path.join(tempDirectory, - "514+global_asy.msr")), false); + const combinedText = fs.readFileSync(result.combinedMSRPath, "utf8"); + assert.match(combinedText, /^runs\s+1\s+2$/m); + assert.match(combinedText, /^# Common parameters for all runs$/m); + assert.match(combinedText, /^# Specific parameters for run 514$/m); + assert.match(combinedText, /^ 1 Asy_0514 0\.2 0\.01$/m); + assert.match(combinedText, /^ 2 Asy_0515 0\.2 0\.01$/m); + assert.match(combinedText, /^asymmetry map1$/m); + // The combined display MSR now carries the canonical +global name, and + // the msr2data `_tmpl` staging file is removed after the export. + assert.strictEqual(fs.existsSync(path.join(tempDirectory, "514+global_asy.msr")), true); + assert.strictEqual(fs.existsSync(path.join(tempDirectory, "514+global_tmpl.msr")), false); + assert.strictEqual(fs.existsSync(path.join(tempDirectory, "514+series_asy.msr")), false); await new Promise(function(resolve) { setImmediate(resolve); }); } finally { fs.rmSync(tempDirectory, { recursive: true, force: true }); diff --git a/tests/web-runner.test.js b/tests/web-runner.test.js index ad07f69..a7d1a7f 100644 --- a/tests/web-runner.test.js +++ b/tests/web-runner.test.js @@ -277,12 +277,12 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() { code: 0, reason: "series-artifacts-collected", job_name: "job", - msr_url: "/tmp/job/514+series_asy.msr", - out_url: "/tmp/job/514+series_asy.out", - parameter_url: "/tmp/job/514+series_asy_par.dat", - root_urls: ["/tmp/job/514+series_asy_0.root"], + msr_url: "/tmp/job/514+global_asy.msr", + out_url: "/tmp/job/514+global_asy.out", + parameter_url: "/tmp/job/514+global_asy_par.dat", + root_urls: ["/tmp/job/514+global_asy_0.root"], root_items: ["fMainCanvas0;1"], - command_log: "$ msr2data [514 515] _asy new global data\n", + command_log: "$ msr2data [514 515] _tmpl new global data\n", warnings: ["msr2data returned code 9 but produced usable series artifacts"] })); }); @@ -302,9 +302,9 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() { }); assert.strictEqual(result.ok, true); - assert.strictEqual(result.parameterDataPath, "/tmp/job/514+series_asy_par.dat"); - assert.strictEqual(result.combinedMSRPath, "/tmp/job/514+series_asy.msr"); - assert.strictEqual(result.plotPath, "/tmp/job/514+series_asy_0.root"); + assert.strictEqual(result.parameterDataPath, "/tmp/job/514+global_asy_par.dat"); + assert.strictEqual(result.combinedMSRPath, "/tmp/job/514+global_asy.msr"); + assert.strictEqual(result.plotPath, "/tmp/job/514+global_asy_0.root"); assert.strictEqual(result.plotStarted, true); assert.match(result.warnings[0], /returned code 9/); assert.strictEqual(calls.length, 1); @@ -316,8 +316,8 @@ async function testWebSeriesCollectionRegistersArtifactsAndPlot() { }); assert.strictEqual(plots.length, 1); assert.deepStrictEqual(Array.from(plots[0].urls), - ["/tmp/job/514+series_asy_0.root"]); - assert.strictEqual(plots[0].options.msrPath, "/tmp/job/514+series_asy.msr"); + ["/tmp/job/514+global_asy_0.root"]); + assert.strictEqual(plots[0].options.msrPath, "/tmp/job/514+global_asy.msr"); assert.match(logs[0], /msr2data/); }