Files
Jungfraujoch/viewer/windows/JFJochProcessingJobsWindow.cpp
leonarski_fandClaude Opus 5 8db75bf8e7 grid scan: the viewer's Grid mode is settable, not only startable
Grid was a mode with no settings. It could be selected and run, but the raster it scored
was whatever the file happened to record, what counted as a crystal was fixed at the
compile-time defaults, and the settings a Grid run actually uses - the cell, spot finding,
indexing - were built inside the MX page and therefore invisible in the mode that uses them.

The unit cell, goniometer, spot-finding and indexing sections now sit outside the page
stack and are shown for MX and Grid alike, the way the azimuthal section already was.
That is not four convenient extras: a raster is scored out of the spots, indexes every
cell by default, and is normally run against a known cell on a fixed target - and the
goniometer section is where the grid geometry itself is stated. Bragg integration, scaling
and the reference dataset stay on the MX page, because a raster integrates, scales and
merges nothing. None of them is duplicated; a second control for one setting is a pair of
controls that can disagree.

The Grid page carries the analysis: the two score thresholds, the minimum cells per
crystal, the decisive single-cell score, an optional cap and whether each cell is indexed.
Every default is read from GridScanAnalysisSettings rather than written out again, so this
panel and the web one cannot drift apart on what a default is; a test pins the numbers so
that moving one is a decision rather than a side effect. No cap is spelled as an unticked
box and not as a count that happens to mean "all". The settings ride to the run on the
experiment, which now survives the panel's round trip through the reader.

The grid geometry could already be edited but not trusted. A step of zero was quietly
replaced by a default, which is the worst answer available: the map is reshaped, every
crystal is reported somewhere else on the sample, and nothing looks wrong. It is now
refused - no grid is set, the summary says which field is at fault, and the run is refused
with it. The summary also says where the grid came from, the fields are highlighted when
they are not the file's, and images that do not fill the last row are called out. The
baseline for "not the file's" is taken when a file is opened, because datasetLoaded fires
again after every edit and by then the file's grid is indistinguishable from one typed
over it.

Two defects found while verifying this. A raster is stored with a stationary omega beside
its grid, since NXmx cannot say "no rotation" - and the panel read a goniometer of any kind
as outranking the grid, so the first grid-scan run to come back carrying that placeholder
axis flipped the panel to Rotation, and the next edit dropped the raster and got the
following run refused. A goniometer that does not scan no longer outranks a grid. And a
grid whose fields cannot make one is a half-finished edit, not a dataset that stopped being
a raster, so it no longer throws the mode away under the user mid-edit.

The Grid refusal now means what it says: no grid scan in the file AND none set in the
panel. A file that records no raster - an older one, a foreign one - can be given one and
analysed, which was the point.

Verified headlessly on a stored raster: the Grid page shows the file's 13 x 15 grid and the
analysis settings at their defaults, a run finds one crystal and draws it on the map, and
the same run with the protein score threshold at 0.95 finds none - so the settings reach
the analysis rather than merely being drawn. On a rotation dataset the mode is refused,
accepted once a grid is entered, and refused again with a zero step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-08 14:46:33 +02:00

815 lines
39 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include "JFJochProcessingJobsWindow.h"
#include "JFJochCalibrationResultWindow.h"
#include "JFJochMergeStatsWindow.h"
#include "../../rugnux/RugnuxDefaults.h"
#include "../widgets/ToolbarIcons.h"
#include "../../rugnux/RugnuxCommandLine.h"
#include <QApplication>
#include <QCheckBox>
#include <QClipboard>
#include <QComboBox>
#include <QDateTime>
#include <QDialog>
#include <QDir>
#include <QFileDialog>
#include <QFont>
#include <QIcon>
#include <QDialogButtonBox>
#include <QFileInfo>
#include <QFormLayout>
#include <QHeaderView>
#include <QLabel>
#include <QLineEdit>
#include <QMessageBox>
#include <QProgressBar>
#include <QPushButton>
#include <QSpinBox>
#include <QStackedWidget>
#include <QStyle>
#include <QPainter>
#include <QPixmap>
#include <QTableWidget>
#include <QToolBar>
#include <QToolButton>
#include <QVBoxLayout>
#include <thread>
namespace {
// Status (the progress bar) is column 2 so it stays visible in the narrow dock.
enum Column { COL_NAME = 0, COL_STATUS, COL_STARTED, COL_MODE, COL_IMAGES, COL_INDEX, COL_CELL, COL_ACTIONS, COL_COUNT };
int default_threads() {
const unsigned hc = std::thread::hardware_concurrency();
return hc == 0 ? 4 : static_cast<int>(hc);
}
QTableWidgetItem *fixedItem(const QString &text) { // a non-editable cell
auto *cell = new QTableWidgetItem(text);
cell->setFlags(cell->flags() & ~Qt::ItemIsEditable);
return cell;
}
// Short name of a run mode, for the dialog title, the Mode column and the run label.
const char *mode_name(AnalysisMode mode) {
switch (mode) {
case AnalysisMode::Azint: return "AzInt";
case AnalysisMode::PowderCalibration: return "Calib";
case AnalysisMode::Grid: return "Grid";
case AnalysisMode::None: return "None";
case AnalysisMode::MXRotation:
case AnalysisMode::MXStills:
default: return "Full";
}
}
// Azimuthal sectors a calibration by rings falls back to when the panel has too few of them: with
// one sector the profile has averaged the ring over every direction and cannot locate it. The
// shared default (common/AnalysisSettings.h) - the same value, for the same reason, everywhere.
}
JFJochProcessingJobsWindow::JFJochProcessingJobsWindow(JFJochImageReadingWorker *worker, QWidget *parent)
: QWidget(parent), worker_(worker) {
setWindowTitle("Processing");
controller_ = new JFJochProcessController(this);
connect(controller_, &JFJochProcessController::phaseChanged, this, &JFJochProcessingJobsWindow::onPhase);
connect(controller_, &JFJochProcessController::progress, this, &JFJochProcessingJobsWindow::onProgress);
connect(controller_, &JFJochProcessController::finished, this, &JFJochProcessingJobsWindow::onFinished);
connect(controller_, &JFJochProcessController::failed, this, &JFJochProcessingJobsWindow::onFailed);
connect(controller_, &JFJochProcessController::liveDataset, this, &JFJochProcessingJobsWindow::liveDataset);
toolbar_ = new QToolBar("Jobs", this);
toolbar_->setMovable(false);
// "New job" is launched by the "Reanalyze dataset" hero button, not from this dock's toolbar.
toolbar_->addAction("Cancel", this, &JFJochProcessingJobsWindow::cancelJob);
toolbar_->addSeparator();
toolbar_->addAction("Show selected", this, &JFJochProcessingJobsWindow::viewResults);
table_ = new QTableWidget(0, COL_COUNT, this);
table_->setMinimumHeight(60); // let the bottom dock shrink freely
table_->setHorizontalHeaderLabels({"Name", "Status", "Started", "Mode", "Images", "Index %", "Unit cell", ""});
table_->horizontalHeader()->setStretchLastSection(false);
table_->horizontalHeader()->setSectionResizeMode(COL_CELL, QHeaderView::Stretch);
table_->horizontalHeader()->setSectionResizeMode(COL_ACTIONS, QHeaderView::Fixed);
table_->setColumnWidth(COL_ACTIONS, 56);
table_->setSelectionBehavior(QAbstractItemView::SelectRows);
table_->setSelectionMode(QAbstractItemView::SingleSelection);
// Only the Name column is editable (per-item flags); editing it renames the run's legend label.
table_->setEditTriggers(QAbstractItemView::DoubleClicked | QAbstractItemView::EditKeyPressed);
connect(table_, &QTableWidget::itemChanged, this, [this](QTableWidgetItem *item) {
if (item->column() != COL_NAME)
return;
const int row = item->row();
if (row < 0 || row >= static_cast<int>(jobs_.size()) || item->text() == jobs_[row].label)
return;
jobs_[row].label = item->text();
emit renameRun(jobs_[row].id, jobs_[row].label);
});
// Double-click a row (other than its editable Name) to show that run in the plots/image.
connect(table_, &QTableWidget::cellDoubleClicked, this, [this](int row, int col) {
if (col == COL_NAME)
return; // double-clicking Name edits the label
if (row >= 0 && row < static_cast<int>(jobs_.size()) && jobs_[row].has_result)
emit activateSnapshot(jobs_[row].id);
});
auto *message = new QLabel("Dataset re-processing is currently available only in File mode.\n\n"
"Open a stored HDF5 file to run processing jobs.", this);
message->setAlignment(Qt::AlignCenter);
message->setWordWrap(true);
stack_ = new QStackedWidget(this);
stack_->addWidget(table_); // page 0
stack_->addWidget(message); // page 1
auto *layout = new QVBoxLayout(this);
layout->setContentsMargins(0, 0, 0, 0);
layout->addWidget(toolbar_);
layout->addWidget(stack_);
}
void JFJochProcessingJobsWindow::onHttpConnectionChanged(bool connected, QString addr) {
// Re-processing reads a stored HDF5 file; it is not available for the live HTTP stream.
stack_->setCurrentIndex(connected ? 1 : 0);
toolbar_->setEnabled(!connected);
}
int JFJochProcessingJobsWindow::askJob(const ReprocessingInputs &inputs, JobSpec &spec) {
QDialog dlg(window()); // centre on the main window, not inside the processing dock
// The kind of job comes from the panel's MX / AzInt / Calib toggle, so the dialog only collects
// the run options.
const bool azint = spec.mode == AnalysisMode::Azint;
const bool calibration = spec.mode == AnalysisMode::PowderCalibration;
const bool grid = spec.mode == AnalysisMode::Grid;
dlg.setWindowTitle(calibration ? "New detector-calibration job"
: grid ? "New grid-scan job"
: azint ? "New azimuthal-integration job" : "New full-analysis job");
auto *start_image = new QSpinBox(&dlg);
start_image->setRange(0, 1'000'000'000);
start_image->setValue(0);
auto *end_image = new QSpinBox(&dlg);
end_image->setRange(0, 1'000'000'000);
end_image->setValue(0);
end_image->setSpecialValueText("end"); // 0 => to the last image
auto *threads = new QSpinBox(&dlg);
threads->setRange(1, 256);
threads->setValue(default_threads());
// Two independent outputs: the large per-image _process.h5 (updates the viewer with spots/results)
// and the small merged .mtz/.cif. Either can be turned off; the ISa/R-meas window is shown
// regardless (it needs only the in-memory merge statistics), so a stats-only run turns both off.
auto *save_h5 = new QCheckBox("Save _process.h5 (per-image results; updates viewer)", &dlg);
save_h5->setChecked(!calibration);
// A calibration's output is the .poni, not per-image results. A grid scan's IS the per-image file -
// the map and the crystals it found are read back out of it - so there it is not optional.
save_h5->setEnabled(!calibration && !grid);
auto *save_merged = new QCheckBox("Write merged .mtz/.cif", &dlg);
save_merged->setChecked(true);
save_merged->setEnabled(!azint && !calibration && !grid); // merged output only in full analysis
// Default the output next to the input file, not the viewer's working directory (the viewer starts
// wherever it was installed). This is an absolute path; Rugnux writes it via the trusted setter,
// and toNativeSeparators keeps a UNC/Windows path openable afterwards.
const QFileInfo fi(inputs.file);
const QString stem = fi.completeBaseName().remove(QStringLiteral("_master"));
const QString default_prefix = QDir::toNativeSeparators(
QStringLiteral("%1/%2_run%3").arg(fi.absolutePath(), stem).arg(job_counter_ + 1));
auto *prefix = new QLineEdit(default_prefix, &dlg);
// "Browse…" opens a file dialog to choose where the outputs go. The chosen name is used as a prefix
// (suffixes like _process.h5 / .cif are appended), so drop a trailing extension.
auto *browse = new QPushButton("Browse…", &dlg);
connect(browse, &QPushButton::clicked, &dlg, [&dlg, prefix] {
const QString sel = QFileDialog::getSaveFileName(&dlg, "Output file prefix", prefix->text(),
"All Files (*)", nullptr,
QFileDialog::DontConfirmOverwrite);
if (sel.isEmpty())
return;
const QFileInfo pf(sel);
prefix->setText(QDir::toNativeSeparators(pf.dir().filePath(pf.completeBaseName())));
});
auto *prefixRow = new QWidget(&dlg);
auto *prefixRowLayout = new QHBoxLayout(prefixRow);
prefixRowLayout->setContentsMargins(0, 0, 0, 0);
prefixRowLayout->addWidget(prefix);
prefixRowLayout->addWidget(browse);
// Rotation-vs-stills mode (rotation indexing + partiality + rot3d) is set in the settings panel by
// the goniometer rotation axis; the dialog only collects run options. Scaling applies to MX full
// analysis.
auto *scaling = new QCheckBox("Scale && merge", &dlg);
scaling->setChecked(true);
scaling->setEnabled(!azint && !calibration && !grid);
// Stills geometry-refinement two-pass: an extra first pass bundle-adjusts the shared beam/distance/
// cell from the strongest indexed frames, then the main pass re-indexes with it. It is stills-only
// and anchors on a known cell, so offer it only for a stills-with-cell run; default it on there, to
// match the rugnux CLI (a no-op for rotation / de-novo stills). Stills mode = rotation indexing off.
const bool stills_with_cell = AnalysisModeIsMX(spec.mode)
&& !inputs.experiment.GetIndexingSettings().GetRotationIndexing()
&& inputs.experiment.GetUnitCell().has_value();
auto *refine_geometry = new QCheckBox("Refine geometry (stills)", &dlg);
refine_geometry->setChecked(stills_with_cell);
refine_geometry->setEnabled(stills_with_cell);
refine_geometry->setToolTip(
"Stills only: an extra first pass bundle-adjusts the shared beam / detector distance / cell from "
"the strongest indexed frames, then re-indexes every frame with it (lifts weak/sparse-stills "
"indexing). Needs a known unit cell; a no-op for rotation data.");
auto *refine_frames = new QSpinBox(&dlg);
refine_frames->setRange(10, 100000);
refine_frames->setValue(200);
refine_frames->setEnabled(stills_with_cell);
refine_frames->setToolTip("Number of strongly-indexed frames fed to the joint bundle adjustment.");
connect(refine_geometry, &QCheckBox::toggled, refine_frames, &QWidget::setEnabled);
auto *refineRow = new QWidget(&dlg);
auto *refineRowLayout = new QHBoxLayout(refineRow);
refineRowLayout->setContentsMargins(0, 0, 0, 0);
refineRowLayout->addWidget(refine_geometry);
refineRowLayout->addWidget(new QLabel("frames:", &dlg));
refineRowLayout->addWidget(refine_frames);
refineRowLayout->addStretch();
// Rotation two-pass geometry post-refine: a first pass post-refines the detector distance / beam /
// cell + rotation axis from the whole sweep, then re-integrates at that geometry (only the refined pass
// is written, as the canonical <prefix>_* output). Rotation-only;
// default on there, to match the rugnux CLI.
const bool rotation_run = AnalysisModeIsMX(spec.mode)
&& inputs.experiment.GetIndexingSettings().GetRotationIndexing();
auto *postrefine = new QCheckBox("Post-refine geometry (rotation, two-pass)", &dlg);
postrefine->setChecked(rotation_run);
postrefine->setEnabled(rotation_run);
postrefine->setToolTip(
"Rotation only: a first pass post-refines the detector distance / beam centre / cell / rotation "
"axis from the whole sweep, then re-integrates at the refined geometry. Only the refined pass is "
"written, as the canonical <prefix> output. Default on; a no-op for stills.");
// Beam-stop shadow: a projection of a few frames shows where the stop and its holder shadow the
// detector; those pixels go into the mask (bit 9) so nothing behind them is integrated. Cheap and
// useful in every mode, so it is offered for all of them and defaulted on.
auto *beam_stop = new QCheckBox("Detect beam stop", &dlg);
beam_stop->setChecked(true);
beam_stop->setToolTip(
"Project a few frames, find where the beam stop and its holder shadow the detector, and add them "
"to the pixel mask (bit 9, cleared at the start of every run). Reflections behind the stop are "
"attenuated but not flagged, so they otherwise integrate low with a plausible sigma.");
auto *beam_stop_frames = new QSpinBox(&dlg);
beam_stop_frames->setRange(3, 100000);
beam_stop_frames->setValue(60);
beam_stop_frames->setToolTip("Frames projected to find the shadow. Fewer leaves the background too "
"sparsely counted to tell a shadow from noise.");
connect(beam_stop, &QCheckBox::toggled, beam_stop_frames, &QWidget::setEnabled);
auto *beamStopRow = new QWidget(&dlg);
auto *beamStopRowLayout = new QHBoxLayout(beamStopRow);
beamStopRowLayout->setContentsMargins(0, 0, 0, 0);
beamStopRowLayout->addWidget(beam_stop);
beamStopRowLayout->addWidget(new QLabel("images:", &dlg));
beamStopRowLayout->addWidget(beam_stop_frames);
beamStopRowLayout->addStretch();
// Beam centre, measured in the same pre-scan and after the shadow: from the symmetry of the spots
// where the sweep reaches half a turn, from the radial background where it does not, and it replaces
// the header value only where it comes out precisely enough. Off by default, as in the rugnux CLI.
auto *beam_center = new QCheckBox("Measure beam centre before indexing", &dlg);
beam_center->setChecked(false);
beam_center->setToolTip(
"Measure the direct beam from the symmetry of the spots where the sweep reaches half a turn and "
"from the radial background where it does not, keeping the header value where neither can "
"measure it.");
auto *form = new QFormLayout;
form->addRow("Start image", start_image);
form->addRow("End image", end_image);
form->addRow("Threads", threads);
form->addRow(save_h5);
form->addRow(save_merged);
form->addRow("Output prefix", prefixRow);
form->addRow(scaling);
form->addRow(refineRow);
form->addRow(postrefine);
form->addRow(beamStopRow);
form->addRow(beam_center);
// Calibration: the calibrant and method come from the panel's Calib page; state them here (with the
// azimuthal sector count the rings method depends on) so the run is not a surprise.
if (calibration) {
const bool rings = spec.calibration.method == CalibrationMethod::Rings;
const int panel_bins = inputs.experiment.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount();
QString text = QStringLiteral("%1, %2 rings, method %3")
.arg(spec.calibration.name)
.arg(spec.calibration.ring_q.size())
.arg(rings ? "rings" : "spots");
if (!spec.calibration.refine_tilt)
text += ", detector tilt held fixed";
if (rings)
text += panel_bins < 4
? QStringLiteral("\nAzimuthal bins: %1 is too few to locate a ring — using %2")
.arg(panel_bins).arg(CALIBRATION_AZIM_BINS_DEFAULT)
: QStringLiteral("\nAzimuthal bins: %1").arg(panel_bins);
text += "\nWrites <output prefix>.poni";
auto *summary = new QLabel(text, &dlg);
summary->setWordWrap(true);
form->addRow("Calibration", summary);
}
int result = 0;
auto *run = new QPushButton("Run locally", &dlg);
auto *copy = new QPushButton("Copy command", &dlg);
auto *cancel = new QPushButton("Cancel", &dlg);
run->setDefault(true);
connect(run, &QPushButton::clicked, &dlg, [&] { result = 1; dlg.accept(); });
connect(copy, &QPushButton::clicked, &dlg, [&] { result = 2; dlg.accept(); });
connect(cancel, &QPushButton::clicked, &dlg, [&] { result = 0; dlg.reject(); });
auto *buttons = new QHBoxLayout;
buttons->addStretch();
buttons->addWidget(run);
buttons->addWidget(copy);
buttons->addWidget(cancel);
auto *layout = new QVBoxLayout(&dlg);
layout->addLayout(form);
layout->addLayout(buttons);
dlg.exec();
if (result != 0) {
spec.start_image = start_image->value();
spec.end_image = end_image->value();
spec.threads = threads->value();
spec.save_h5 = save_h5->isChecked();
spec.save_merged = save_merged->isEnabled() && save_merged->isChecked();
spec.prefix = prefix->text();
spec.scaling = scaling->isEnabled() && scaling->isChecked();
spec.refine_geometry = refine_geometry->isEnabled() && refine_geometry->isChecked();
spec.refine_geometry_frames = refine_frames->value();
spec.rotation_postrefine = postrefine->isEnabled() && postrefine->isChecked();
spec.detect_beam_stop = beam_stop->isChecked();
spec.detect_beam_stop_frames = beam_stop_frames->value();
spec.estimate_beam_center = beam_center->isChecked();
}
return result;
}
ProcessConfig JFJochProcessingJobsWindow::buildConfig(const JobSpec &spec, const ReprocessingInputs &inputs) const {
ProcessConfig config;
config.mode = spec.mode;
config.nthreads = spec.threads;
config.start_image = spec.start_image;
config.end_image = spec.end_image > 0 ? spec.end_image : -1; // 0 => to the end
// Files land at output_prefix; leave it empty (write nothing, stats only) when neither output is
// wanted. The two flags then select which files are actually written there. A calibration always
// keeps the prefix - the .poni written next to it is the whole point of the run.
config.output_prefix = (spec.save_h5 || spec.save_merged || spec.mode == AnalysisMode::PowderCalibration)
? spec.prefix.toStdString() : std::string();
config.write_process_h5 = spec.save_h5;
config.write_merged = spec.save_merged;
// The unmerged MTZ is a merged-output artefact - it is what another scaling program is handed
// instead of our merge - so it follows the merged switch rather than appearing beside a run that
// only asked for the per-image _process.h5.
config.export_unmerged = spec.save_merged;
config.spot_finding = inputs.spot_finding;
if (spec.detect_beam_stop)
config.detect_beam_stop = spec.detect_beam_stop_frames;
// `rugnux --estimate-beam-center` fits the spindle's skew about the beam along with the centre
// (--no-fit-spindle is the deviation), so the dialog's single switch sets both.
config.estimate_beam_center = spec.estimate_beam_center;
config.fit_spindle = spec.estimate_beam_center;
if (spec.mode == AnalysisMode::PowderCalibration) {
config.calibration_method = spec.calibration.method;
config.calibration_refine_tilt = spec.calibration.refine_tilt;
config.calibrant_ring_q = spec.calibration.ring_q;
// Spot finding for the spots method. Indexing is off: a calibration wants the spot positions and
// nothing else, and a calibrant is a powder with no lattice to index.
config.spot_finding.enable = true;
config.spot_finding.indexing = false;
}
if (spec.mode == AnalysisMode::Grid) {
// Spot finding runs, because the per-cell resolution comes from it. Whether each cell is also
// indexed is a property of the experiment, not of rasters (GridScanAnalysisSettings), which is
// what the CLI asks too. Nothing scales or merges: a raster answers where the crystals are.
config.spot_finding.enable = true;
config.spot_finding.indexing = inputs.experiment.GetGridScanAnalysisSettings().IsIndexing();
// The rest of the raster analysis - the two score thresholds, the minimum patch, the decisive
// single cell and the crystal cap - is read straight off the experiment by AnalyzeGridScan at
// the end of the run, so it travels on the experiment handed to the job and needs nothing
// copied into the config. Only indexing is here, because it gates a per-image stage.
}
if (AnalysisModeIsMX(spec.mode)) {
// Rotation indexing follows the panel's rotation axis (= the experiment's indexing
// setting); a rotation run uses 60 first-pass images to find the lattice.
config.rotation_indexing = inputs.experiment.GetIndexingSettings().GetRotationIndexing();
config.two_pass_rotation = true;
if (config.rotation_indexing)
config.rotation_indexing_image_count = 60;
config.run_scaling = spec.scaling;
config.reference_data = inputs.reference_data; // enables CCref / reference-based scaling
// Stills geometry-refinement two-pass. The dialog only offers it for a stills-with-cell run
// (Rugnux no-ops it otherwise), so honour it whenever set; unset leaves the pass off.
if (spec.refine_geometry)
config.refine_geometry = spec.refine_geometry_frames;
// Rotation two-pass geometry post-refine (rotation-only; the dialog defaults it on for rotation).
config.rotation_postrefine_geometry = spec.rotation_postrefine;
}
return config;
}
void JFJochProcessingJobsWindow::newJob(AnalysisMode mode, CalibrationSelection calibration) {
const ReprocessingInputs inputs = worker_->GetReprocessingInputs();
if (!inputs.valid) {
QMessageBox::information(this, "Processing", "Open a file first (processing is not available for live HTTP data).");
return;
}
// A grid scan is the whole point of the mode: without one there is no map to find crystals in,
// and a run would score every image and report nothing. The file is not the only source of one -
// the settings panel states the raster a file never recorded, which is how an older or foreign
// file is analysed at all - so this refuses only when neither the file nor the panel has one.
if (mode == AnalysisMode::Grid && !inputs.experiment.GetGridScan().has_value()) {
QMessageBox::information(this, "Processing",
"No grid scan: this dataset does not describe one, and none is set in "
"the settings panel. Choose \"Grid scan\" in the Goniometer section "
"and give the step and the number of points, or pick another mode.");
return;
}
JobSpec spec;
spec.mode = mode;
spec.calibration = std::move(calibration);
const int action = askJob(inputs, spec);
if (action == 0)
return;
const ProcessConfig config = buildConfig(spec, inputs);
// The experiment carries the panel's indexing settings — including RotationIndexing set by the
// rotation axis (needed so IndexAndRefine builds a rotation indexer). On top of that it gets
// exactly the defaults `rugnux` with no options would apply, so the two front ends agree on the
// analysis policy — except where the panel states its own, which wins (the polarization factor,
// and the scaling fields below).
//
// The cell and space group come from the settings panel and nowhere else: with "Unit cell known"
// ticked they are indexed and merged with, unticked the panel clears both and the run determines
// them from the data (-C / -S versus bare `rugnux`). The panel is filled from the file when one is
// opened, so what a run will use is always the crystal shown on screen - including after a finished
// job's _process.h5 becomes the active snapshot, where a run that ended in P1 shows P1 and can be
// cleared, rather than silently pinning every later run to it.
DiffractionExperiment experiment = inputs.experiment;
const auto polarization = experiment.GetPolarizationFactor();
ApplyRugnuxExperimentDefaults(experiment);
experiment.PolarizationFactor(polarization); // the panel's setting, not the rugnux default
// Calibrating from the run-summed profile needs the profile to be binned in azimuth; the panel's
// bin count is an azimuthal-integration setting and defaults to a plain radial profile, which
// carries no information about where the ring centre is. Same fallback as the rugnux CLI.
if (spec.mode == AnalysisMode::PowderCalibration && spec.calibration.method == CalibrationMethod::Rings
&& experiment.GetAzimuthalIntegrationSettings().GetAzimuthalBinCount() < 4) {
AzimuthalIntegrationSettings azint = experiment.GetAzimuthalIntegrationSettings();
azint.AzimuthalBinCount(CALIBRATION_AZIM_BINS_DEFAULT);
experiment.ImportAzimuthalIntegrationSettings(azint);
}
if (AnalysisModeIsMX(spec.mode) && spec.scaling) {
ScalingSettings scaling = RugnuxDefaultScalingSettings(config.rotation_indexing);
// Keep what the settings dock does expose; take the rest from the shared defaults.
const auto &dock = inputs.experiment.GetScalingSettings();
scaling.MergeFriedel(dock.GetMergeFriedel());
scaling.CorrectionSurfaces(dock.GetCorrectionSurfaces());
scaling.StillsPartialityRefine(dock.GetStillsPartialityRefine());
scaling.HighResolutionLimit_A(dock.GetHighResolutionLimit_A());
scaling.LowResolutionLimit_A(dock.GetLowResolutionLimit_A());
experiment.ImportScalingSettings(scaling);
}
if (action == 2) { // copy command line
// The command line is run against the file, so it has to state what the panel changed. The
// one thing it cannot state is the rotation axis (rugnux has no flag for it) - but switching
// the axis off is `--force-still`, which RugnuxCommandLine emits from the axis the file has,
// not the one the panel cleared. Put the file's back for the command line only.
DiffractionExperiment cmdline_experiment = experiment;
if (!cmdline_experiment.GetGoniometer().has_value())
cmdline_experiment.Goniometer(inputs.file_goniometer);
const QString cmd = QString::fromStdString(
RugnuxCommandLine(config, cmdline_experiment, inputs.file.toStdString(),
spec.calibration.name.toStdString()));
QApplication::clipboard()->setText(cmd);
QMessageBox::information(this, "Command line", cmd + "\n\n(copied to clipboard)");
return;
}
if (controller_->running()) {
QMessageBox::information(this, "Processing", "A job is already running; wait for it to finish or cancel it.");
return;
}
const int run_number = ++job_counter_;
const QString label = QStringLiteral("%1 %2").arg(mode_name(spec.mode)).arg(run_number);
const QString id = QStringLiteral("run-%1").arg(run_number);
JobInfo info;
info.id = id;
info.label = label;
if (spec.save_h5)
info.snapshot_path = QString::fromStdString(config.output_prefix) + "_process.h5";
if (spec.mode == AnalysisMode::PowderCalibration) {
info.poni_path = spec.prefix + ".poni";
info.calibration_header = experiment.GetDiffractionGeometry();
calibration_experiment_ = experiment;
}
const int row = table_->rowCount();
table_->insertRow(row);
jobs_.push_back(info); // jobs_[row] must exist before setItem(COL_NAME) fires itemChanged
const QString range_text = (spec.start_image == 0 && spec.end_image == 0)
? QStringLiteral("all")
: QStringLiteral("%1–%2").arg(spec.start_image)
.arg(spec.end_image > 0 ? QString::number(spec.end_image) : QStringLiteral("end"));
const int expected = spec.end_image > spec.start_image ? spec.end_image - spec.start_image : 0;
table_->setItem(row, COL_NAME, new QTableWidgetItem(label)); // editable (default flags)
table_->setItem(row, COL_STARTED, fixedItem(QDateTime::currentDateTime().toString("HH:mm:ss")));
table_->setItem(row, COL_MODE, fixedItem(mode_name(spec.mode)));
table_->setItem(row, COL_IMAGES, fixedItem(range_text));
table_->setItem(row, COL_STATUS, fixedItem("queued"));
table_->setItem(row, COL_INDEX, fixedItem("-"));
table_->setItem(row, COL_CELL, fixedItem("-"));
addRowActions(row, id);
// A progress bar lives in the Status cell while the job runs; it shows the phase as text until
// image processing starts, then "<done> / <expected>" with the bar filling in the background.
running_bar_ = new QProgressBar(table_);
running_bar_->setAlignment(Qt::AlignCenter);
running_bar_->setRange(0, expected > 0 ? expected : 1);
running_bar_->setValue(0);
running_bar_->setFormat("queued");
table_->setCellWidget(row, COL_STATUS, running_bar_);
running_row_ = row;
job_timer_.start();
controller_->start(inputs.file, experiment, inputs.pixel_mask, config);
emit jobStarted();
emit writeStatusBar("Started processing job " + label);
}
void JFJochProcessingJobsWindow::cancelJob() {
if (controller_->running()) {
controller_->cancel();
emit writeStatusBar("Cancelling processing job…");
}
}
void JFJochProcessingJobsWindow::removeResult() {
const int row = table_->currentRow();
if (row < 0 || row >= static_cast<int>(jobs_.size()))
return;
if (jobs_[row].id == "Original")
return; // the original file is always kept
if (row == running_row_) {
QMessageBox::information(this, "Processing", "Cancel the running job before removing it.");
return;
}
emit removeRun(jobs_[row].id); // drops the snapshot from the reader
QSignalBlocker block(table_); // row removal must not look like a rename
table_->removeRow(row);
jobs_.erase(jobs_.begin() + row);
if (running_row_ > row)
--running_row_;
}
void JFJochProcessingJobsWindow::addRowActions(int row, const QString &id) {
auto *w = new QWidget(table_);
auto *l = new QHBoxLayout(w);
l->setContentsMargins(2, 0, 2, 0);
l->setSpacing(2);
auto *graph = new QToolButton(w);
graph->setIcon(ToolbarIcons::linePlot());
graph->setAutoRaise(true);
graph->setToolTip("Show results");
graph->setEnabled(false); // enabled once a merge / calibration result arrives for this run
connect(graph, &QToolButton::clicked, this, [this, id] { showStats(id); });
auto *trash = new QToolButton(w);
trash->setIcon(style()->standardIcon(QStyle::SP_TrashIcon));
trash->setAutoRaise(true);
trash->setToolTip("Remove this run");
trash->setEnabled(id != "Original"); // the original file is always kept
connect(trash, &QToolButton::clicked, this, [this, id] { removeRunById(id); });
l->addWidget(graph);
l->addWidget(trash);
table_->setCellWidget(row, COL_ACTIONS, w);
jobs_[row].graph_btn = graph;
}
void JFJochProcessingJobsWindow::showStats(const QString &id) {
for (const auto &j: jobs_) {
if (j.id == id && j.calibration.has_value() && j.calibration_header.has_value()) {
auto *win = new JFJochCalibrationResultWindow(j.label, *j.calibration, *j.calibration_header,
j.poni_path, window());
win->show();
return;
}
if (j.id == id && j.has_merge_stats) {
auto *win = new JFJochMergeStatsWindow(j.label, j.merge_stats, j.isa, j.merge_has_reference,
j.twinning, j.space_group, j.space_group_search,
j.merged_i_sigma, window());
win->show();
return;
}
}
}
void JFJochProcessingJobsWindow::removeRunById(const QString &id) {
for (int row = 0; row < static_cast<int>(jobs_.size()); ++row) {
if (jobs_[row].id != id)
continue;
if (id == "Original")
return;
if (row == running_row_) {
QMessageBox::information(this, "Processing", "Cancel the running job before removing it.");
return;
}
emit removeRun(id);
QSignalBlocker block(table_);
table_->removeRow(row);
jobs_.erase(jobs_.begin() + row);
if (running_row_ > row)
--running_row_;
return;
}
}
void JFJochProcessingJobsWindow::clearJobs() {
if (controller_->running())
controller_->cancel();
emit liveDataset(nullptr);
QSignalBlocker block(table_);
table_->setRowCount(0);
jobs_.clear();
job_counter_ = 0;
running_row_ = -1;
running_bar_ = nullptr;
addOriginalRow();
}
void JFJochProcessingJobsWindow::addOriginalRow() {
// The file's own data, listed as the first run so it can be shown like any reprocessing run.
JobInfo info;
info.id = "Original";
info.label = "Original";
info.has_result = true;
const int row = table_->rowCount();
table_->insertRow(row);
jobs_.push_back(info);
QSignalBlocker block(table_);
table_->setItem(row, COL_NAME, fixedItem("Original")); // reserved name, not editable
table_->setItem(row, COL_STATUS, fixedItem(""));
table_->setItem(row, COL_STARTED, fixedItem("—"));
table_->setItem(row, COL_MODE, fixedItem("HDF5"));
table_->setItem(row, COL_IMAGES, fixedItem("all"));
table_->setItem(row, COL_INDEX, fixedItem("-"));
table_->setItem(row, COL_CELL, fixedItem("-"));
addRowActions(row, info.id);
}
void JFJochProcessingJobsWindow::setActiveRun(QString active_id) {
QSignalBlocker block(table_);
for (int row = 0; row < static_cast<int>(jobs_.size()); row++) {
auto *item = table_->item(row, COL_NAME);
if (!item)
continue;
QFont f = item->font();
f.setBold(jobs_[row].id == active_id);
item->setFont(f);
}
}
void JFJochProcessingJobsWindow::viewResults() {
const int row = table_->currentRow();
if (row < 0 || row >= static_cast<int>(jobs_.size()))
return;
if (!jobs_[row].has_result) {
QMessageBox::information(this, "Processing", "This job has no saved results to view.");
return;
}
emit activateSnapshot(jobs_[row].id);
}
void JFJochProcessingJobsWindow::setStatus(int row, const QString &text) {
if (row >= 0 && row < table_->rowCount())
table_->item(row, COL_STATUS)->setText(text);
}
void JFJochProcessingJobsWindow::onPhase(QString phase) {
// The "Processing images" phase is shown by onProgress; other phases (first pass, scaling, ...)
// show their name as text over an empty bar.
if (!running_bar_ || phase == "Processing images")
return;
running_bar_->setRange(0, 1);
running_bar_->setValue(0);
running_bar_->setFormat(phase);
}
void JFJochProcessingJobsWindow::onProgress(quint64 done, quint64 total) {
if (!running_bar_)
return;
running_bar_->setRange(0, static_cast<int>(total));
running_bar_->setValue(static_cast<int>(done));
running_bar_->setFormat("%v / %m");
// Processing rate and ETA in the status bar.
if (job_timer_.isValid() && done > 0) {
const double secs = job_timer_.elapsed() / 1000.0;
if (secs > 0.0) {
const double hz = done / secs;
const double remaining = hz > 0.0 ? (total - done) / hz : 0.0;
emit writeStatusBar(QStringLiteral("Processing %1 Hz — ~%2 s remaining")
.arg(hz, 0, 'f', 1).arg(qRound(remaining)));
}
}
}
void JFJochProcessingJobsWindow::onFinished(ProcessResult result) {
emit liveDataset(nullptr); // the finished run takes over from the live overlay
const int row = running_row_;
running_row_ = -1;
if (row >= 0)
table_->removeCellWidget(row, COL_STATUS); // deletes running_bar_
running_bar_ = nullptr;
if (row < 0 || row >= static_cast<int>(jobs_.size()))
return;
setStatus(row, result.cancelled ? "cancelled" : "done");
if (result.indexing_rate.has_value())
table_->item(row, COL_INDEX)->setText(QStringLiteral("%1%").arg(result.indexing_rate.value() * 100.0, 0, 'f', 0));
if (result.consensus_cell.has_value()) {
const auto &c = result.consensus_cell.value();
table_->item(row, COL_CELL)->setText(
QStringLiteral("%1 %2 %3 %4 %5 %6")
.arg(c.a, 0, 'f', 1).arg(c.b, 0, 'f', 1).arg(c.c, 0, 'f', 1)
.arg(c.alpha, 0, 'f', 1).arg(c.beta, 0, 'f', 1).arg(c.gamma, 0, 'f', 1));
}
if (!result.cancelled && result.written_master_path.has_value() && !jobs_[row].snapshot_path.isEmpty()) {
jobs_[row].has_result = true;
emit registerSnapshot(jobs_[row].id, jobs_[row].label, jobs_[row].snapshot_path); // also activates it
}
// A finished calibration: write the .poni (the library leaves that to the caller, as in the CLI)
// and surface the fitted geometry, on the same auto-open-once / recall-by-icon pattern as merging.
if (!result.cancelled && result.calibration.has_value()) {
jobs_[row].calibration = result.calibration;
try {
WritePoniFile(jobs_[row].poni_path.toStdString(), calibration_experiment_,
result.calibration->geometry);
} catch (const std::exception &e) {
jobs_[row].poni_path.clear();
QMessageBox::warning(this, "Calibration", QString::fromStdString(e.what()));
}
if (jobs_[row].graph_btn)
jobs_[row].graph_btn->setEnabled(true);
showStats(jobs_[row].id);
}
// Capture merge statistics and surface the analysis window (auto-open once; recall later via the
// row's graph icon).
if (!result.cancelled && result.has_merge_statistics) {
jobs_[row].has_merge_stats = true;
jobs_[row].merge_stats = result.merge_statistics;
jobs_[row].isa = result.error_model_isa;
jobs_[row].merge_has_reference = result.has_reference;
jobs_[row].twinning = result.twinning;
jobs_[row].space_group = result.space_group;
jobs_[row].space_group_search = result.space_group_search;
jobs_[row].merged_i_sigma = result.merged_i_sigma;
if (jobs_[row].graph_btn)
jobs_[row].graph_btn->setEnabled(true);
showStats(jobs_[row].id);
}
emit writeStatusBar(result.cancelled ? "Processing cancelled" : "Processing finished");
}
void JFJochProcessingJobsWindow::onFailed(QString error) {
emit liveDataset(nullptr); // clear the live overlay
const int row = running_row_;
running_row_ = -1;
if (row >= 0)
table_->removeCellWidget(row, COL_STATUS); // deletes running_bar_
running_bar_ = nullptr;
setStatus(row, "failed");
QMessageBox::warning(this, "Processing failed", error);
}