Files
Jungfraujoch/viewer/windows/JFJochProcessingJobsWindow.h
T
leonarski_fandFilip Leonarski 19b135186d
Build Packages / Create release (push) Successful in 7s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 4m56s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 9m10s
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 13m51s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 16m56s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 20m59s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 21m25s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 21m17s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 14m14s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 25m59s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 25m48s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 15m52s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 19m33s
Build Packages / Generate python client (push) Successful in 29s
Build Packages / Build documentation (push) Successful in 1m19s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 20m49s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 25m31s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 25m24s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 26m5s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 22m19s
Build Packages / Unit tests (push) Failing after 2h28m24s
Build Packages / Validation Battery (push) Successful in 40m33s
v1.0.0-rc.175 (#85)
* Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`).
* Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection.
* Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS).
* Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead.
* Rugnux: On macOS, Bragg integration runs on the Apple GPU.
* Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand.
* Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`.
* Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit.
* Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed.
* Rugnux: The beam-stop shadow is found correctly on tilted detectors.
* Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement.
* jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps.
* jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection.
* jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images.
* Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges.

---------

Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home>
Reviewed-on: #85
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-10-16 15:34:43 +02:00

125 lines
6.4 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <QElapsedTimer>
#include <QString>
#include <QWidget>
#include <cstdint>
#include <optional>
#include <vector>
#include "../JFJochProcessController.h"
#include "../../rugnux/RugnuxDefaults.h"
#include "../JFJochImageReadingWorker.h" // ReprocessingInputs
#include "../widgets/PowderCalibrationWidget.h" // CalibrationSelection
class QTableWidget;
class QProgressBar;
class QStackedWidget;
class QToolBar;
class QToolButton;
// Makes processing a first-class GUI activity: a dockable panel with a table of processing jobs run
// on the current dataset. A job can be run locally (off the GUI thread, via JFJochProcessController)
// or its rugnux command line copied for a cluster. A finished local run is registered as a
// reader snapshot so its results become a selectable view of the dataset. Re-processing reads a
// stored HDF5 file, so the panel shows an explanatory message while connected to a live HTTP stream.
class JFJochProcessingJobsWindow : public QWidget {
Q_OBJECT
public:
explicit JFJochProcessingJobsWindow(JFJochImageReadingWorker *worker, QWidget *parent = nullptr);
signals:
void registerSnapshot(QString id, QString label, QString master_path);
void activateSnapshot(QString id);
void renameRun(QString id, QString label);
void removeRun(QString id);
void writeStatusBar(QString message, int timeout_ms = 0);
// Live per-image results while a job runs, for the dataset-info plots (nullptr clears it).
void liveDataset(std::shared_ptr<const JFJochReaderDataset> dataset);
void jobStarted(); // a reprocessing job was launched — reveal the (hidden-by-default) panel
public slots:
void onHttpConnectionChanged(bool connected, QString addr);
void clearJobs(); // reset the table on a new file (re-adds the Original row)
void setActiveRun(QString active_id); // bold the row of the run currently shown in the plots
// The panel's "Analyze dataset" action; the mode is the panel's selected page. The calibrant
// selection is only used by ProcessMode::Calibration.
void newJob(ProcessMode mode = ProcessMode::FullAnalysis, CalibrationSelection calibration = {});
private slots:
void cancelJob();
void removeResult();
void viewResults();
void onPhase(QString phase);
void onProgress(quint64 done, quint64 total);
void onFinished(ProcessResult result, QString report, std::vector<ReportCheck> checks);
void onFailed(QString error);
private:
struct JobInfo {
QString id; // stable reader-snapshot key
QString label; // editable display name (legend / table)
QString snapshot_path; // empty unless saved
bool has_result = false;
// Merge statistics captured when a scaling/merging job finished (drive the analysis window).
bool has_merge_stats = false;
MergeStatistics merge_stats;
double isa = 0.0;
bool merge_has_reference = false;
TwinningAnalysisResult twinning; // twinning test of the merged intensities
std::optional<gemmi::SpaceGroup> space_group; // final space group (searched or fixed)
std::optional<SearchSpaceGroupResult> space_group_search; // ranked candidates, when a search ran
std::vector<std::pair<float, float>> merged_i_sigma; // ISa diagnostic scatter
QString report; // the rugnux result report text (a section of the window)
std::vector<ReportCheck> checks; // the report's pathology checklist
// Detector calibration result (Calibration mode): the fit, the header geometry it is compared
// against, and where the .poni was written.
std::optional<CalibrationResult> calibration;
std::optional<DiffractionGeometry> calibration_header;
QString poni_path;
QToolButton *graph_btn = nullptr; // per-row "show statistics" button (enabled once stats exist)
};
struct JobSpec {
ProcessMode mode = ProcessMode::FullAnalysis;
int start_image = 0;
int end_image = 0; // 0 == to the end
int threads = 4;
bool save_h5 = true; // write the per-image _process.h5 (also drives the viewer snapshot)
bool save_merged = true; // write the merged .mtz/.cif
QString prefix;
bool scaling = false;
bool refine_geometry = false; // stills-only global geometry bundle-adjust (needs a known cell)
int refine_geometry_frames = RUGNUX_REFINE_GEOMETRY_FRAMES; // strong indexed frames fed to that bundle adjustment
bool rotation_postrefine = true; // rotation-only two-pass geometry post-refine (default on)
bool detect_beam_stop = true; // project frames and mask the beam-stop shadow (default on)
int detect_beam_stop_frames = RUGNUX_DETECT_BEAM_STOP_FRAMES; // frames projected to find it
CalibrationSelection calibration; // Calibration mode: calibrant rings + rings/spots method
};
// Returns 0 = cancel, 1 = run locally, 2 = copy command line; fills spec from the dialog.
int askJob(const ReprocessingInputs &inputs, JobSpec &spec);
ProcessConfig buildConfig(const JobSpec &spec, const ReprocessingInputs &inputs) const;
void setStatus(int row, const QString &text);
void addOriginalRow(); // the file's own data as the first, always-present run
void addRowActions(int row, const QString &id); // per-row graph (stats) + trash (remove) buttons
void showStats(const QString &id); // open the merge-statistics window for a run
void removeRunById(const QString &id); // remove a run from the table + reader
JFJochImageReadingWorker *worker_;
JFJochProcessController *controller_;
// The experiment a running calibration job was launched with: WritePoniFile needs the detector
// size and wavelength from it once the fit comes back.
DiffractionExperiment calibration_experiment_;
QToolBar *toolbar_;
QStackedWidget *stack_; // page 0: jobs table, page 1: HTTP-mode message
QTableWidget *table_;
QProgressBar *running_bar_ = nullptr; // lives in the running row's Status cell
QElapsedTimer job_timer_; // running job wall-clock, for rate + ETA
int running_row_ = -1;
int job_counter_ = 0;
std::vector<JobInfo> jobs_;
};