Build Packages / Create release (push) Successful in 7s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 4m56s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 9m10s
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 13m51s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 16m56s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 20m59s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 21m25s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 21m17s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 14m14s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 25m59s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 25m48s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 15m52s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 19m33s
Build Packages / Generate python client (push) Successful in 29s
Build Packages / Build documentation (push) Successful in 1m19s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 20m49s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 25m31s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 25m24s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 26m5s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 22m19s
Build Packages / Unit tests (push) Failing after 2h28m24s
Build Packages / Validation Battery (push) Successful in 40m33s
* Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
253 lines
11 KiB
C++
253 lines
11 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include "JFJochViewerFileBrowser.h"
|
|
|
|
#include <QVBoxLayout>
|
|
#include <QHBoxLayout>
|
|
#include <QLineEdit>
|
|
#include <QToolButton>
|
|
#include <QTreeView>
|
|
#include <QHeaderView>
|
|
#include <QScrollBar>
|
|
#include <QFileDialog>
|
|
#include <QSettings>
|
|
#include <QDir>
|
|
#include <QFileInfo>
|
|
#include <QRegularExpression>
|
|
#include <QShortcut>
|
|
#include <QStyle>
|
|
#include <QTimer>
|
|
#include <QApplication>
|
|
#include <QSet>
|
|
|
|
namespace {
|
|
constexpr auto kDataRootEnvVar = "JUNGFRAUJOCH_DATA_ROOT";
|
|
constexpr auto kSettingsKey = "fileBrowserRoot";
|
|
constexpr int kRefreshIntervalMs = 3000;
|
|
|
|
// The filter box narrows files only; a directory always passes. Filtering directories too would
|
|
// hide everything below one whose own name does not match - including the files that do match -
|
|
// and would hide the root's ancestors, which leaves the tree with no valid root index at all.
|
|
class FileOnlyFilterProxy : public QSortFilterProxyModel {
|
|
public:
|
|
using QSortFilterProxyModel::QSortFilterProxyModel;
|
|
|
|
protected:
|
|
bool filterAcceptsRow(int row, const QModelIndex &parent) const override {
|
|
const auto *fs = qobject_cast<const QFileSystemModel *>(sourceModel());
|
|
if (fs && fs->isDir(fs->index(row, 0, parent)))
|
|
return true;
|
|
return QSortFilterProxyModel::filterAcceptsRow(row, parent);
|
|
}
|
|
};
|
|
}
|
|
|
|
QString JFJochViewerFileBrowser::SlsDataRoot(const QString &user) {
|
|
const auto match = QRegularExpression("\\Ae(\\d{5})\\z").match(user);
|
|
if (!match.hasMatch())
|
|
return {};
|
|
return "/sls/mx/data/p" + match.captured(1) + "/raw";
|
|
}
|
|
|
|
QString JFJochViewerFileBrowser::DefaultRoot() {
|
|
// Site default (e.g. a beamline launcher setting it to /sls/mx/data/p<group>), then whatever
|
|
// root was last browsed to (so re-launching without the variable set doesn't start over from
|
|
// scratch), then - for an SLS beamline account - that proposal's raw data, then the user's home
|
|
// - never the current directory, which for an installed binary is typically the install
|
|
// location rather than anywhere the user would keep data.
|
|
const QString env_root = qEnvironmentVariable(kDataRootEnvVar);
|
|
if (!env_root.isEmpty() && QDir(env_root).exists())
|
|
return env_root;
|
|
|
|
const QString last_root = QSettings("PSI", "jfjoch_viewer").value(kSettingsKey).toString();
|
|
if (!last_root.isEmpty() && QDir(last_root).exists())
|
|
return last_root;
|
|
|
|
// USER is the portable spelling Qt exposes; it is simply empty where the beamline directory
|
|
// could not exist anyway, so everyone else silently gets home, as before.
|
|
const QString sls_root = SlsDataRoot(qEnvironmentVariable("USER"));
|
|
if (!sls_root.isEmpty() && QDir(sls_root).exists())
|
|
return sls_root;
|
|
|
|
return QDir::homePath();
|
|
}
|
|
|
|
JFJochViewerFileBrowser::JFJochViewerFileBrowser(QWidget *parent) : QWidget(parent) {
|
|
auto *layout = new QVBoxLayout(this);
|
|
// No inset: every other dock panel sits flush against its edges, and 4 px here put the panel's
|
|
// content 4 px away from the resize handle it shares with the image, which reads as a handle
|
|
// that is off-centre.
|
|
layout->setContentsMargins(0, 0, 0, 0);
|
|
|
|
auto *root_row = new QHBoxLayout();
|
|
root_edit_ = new QLineEdit(this);
|
|
root_edit_->setToolTip("Data root - type a path or use Browse…");
|
|
auto *browse_button = new QToolButton(this);
|
|
browse_button->setText("…");
|
|
browse_button->setToolTip("Browse for a data root…");
|
|
auto *refresh_button = new QToolButton(this);
|
|
refresh_button->setIcon(style()->standardIcon(QStyle::SP_BrowserReload));
|
|
refresh_button->setToolTip("Refresh the file list (F5)");
|
|
root_row->addWidget(root_edit_);
|
|
root_row->addWidget(browse_button);
|
|
root_row->addWidget(refresh_button);
|
|
layout->addLayout(root_row);
|
|
|
|
filter_edit_ = new QLineEdit(this);
|
|
filter_edit_->setPlaceholderText("Filter…");
|
|
filter_edit_->setClearButtonEnabled(true);
|
|
layout->addWidget(filter_edit_);
|
|
|
|
model_ = new QFileSystemModel(this);
|
|
model_->setFilter(QDir::AllDirs | QDir::Files | QDir::NoDotAndDotDot);
|
|
// Directories are always listed regardless of these; only *_master.h5 / *_process.h5 and
|
|
// miniCBF frames among files. The many _data_NNNNNN.h5 shards of a dataset are not meant to be
|
|
// opened directly; a .cbf is one frame per file, and opening any of them opens its whole sweep.
|
|
model_->setNameFilters({"*_master.h5", "*_process.h5", "*.cbf"});
|
|
model_->setNameFilterDisables(false); // hide non-matching files rather than just greying them out
|
|
|
|
proxy_ = new FileOnlyFilterProxy(this);
|
|
proxy_->setSourceModel(model_);
|
|
proxy_->setFilterKeyColumn(0);
|
|
proxy_->setFilterCaseSensitivity(Qt::CaseInsensitive);
|
|
// Deliberately not recursive: it only ever narrows names already listed at an expanded level,
|
|
// never forces fetching a collapsed subtree just to search inside it.
|
|
|
|
tree_ = new QTreeView(this);
|
|
tree_->setModel(proxy_);
|
|
tree_->setHeaderHidden(true);
|
|
for (int col = 1; col < model_->columnCount(); ++col)
|
|
tree_->hideColumn(col);
|
|
tree_->setSortingEnabled(false);
|
|
layout->addWidget(tree_);
|
|
|
|
connect(browse_button, &QToolButton::clicked, this, &JFJochViewerFileBrowser::onBrowseClicked);
|
|
connect(root_edit_, &QLineEdit::editingFinished, this, &JFJochViewerFileBrowser::onRootEditingFinished);
|
|
connect(filter_edit_, &QLineEdit::textChanged, proxy_, &QSortFilterProxyModel::setFilterFixedString);
|
|
connect(tree_, &QTreeView::doubleClicked, this, &JFJochViewerFileBrowser::onDoubleClicked);
|
|
connect(refresh_button, &QToolButton::clicked, this, [this] { Refresh(); });
|
|
// Window-wide, but a shortcut on a hidden widget never fires, so F5 does nothing while the panel is
|
|
// tabbed behind the Processing one.
|
|
connect(new QShortcut(QKeySequence(Qt::Key_F5), this), &QShortcut::activated, this, [this] { Refresh(); });
|
|
|
|
auto *refresh_timer = new QTimer(this);
|
|
connect(refresh_timer, &QTimer::timeout, this, [this] {
|
|
// Not while hidden, and not in the middle of a click or a scroll-bar drag.
|
|
if (isVisible() && QApplication::mouseButtons() == Qt::NoButton)
|
|
Refresh(false);
|
|
});
|
|
refresh_timer->start(kRefreshIntervalMs);
|
|
|
|
SetRoot(DefaultRoot());
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::SetRoot(const QString &path) {
|
|
QDir dir(path);
|
|
if (!dir.exists())
|
|
return;
|
|
|
|
const QString canonical = QDir::toNativeSeparators(dir.absolutePath());
|
|
root_edit_->setText(canonical);
|
|
model_->setRootPath(canonical);
|
|
tree_->setRootIndex(proxy_->mapFromSource(model_->index(canonical)));
|
|
|
|
QSettings("PSI", "jfjoch_viewer").setValue(kSettingsKey, canonical);
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::ResetToDefaultRoot() {
|
|
SetRoot(DefaultRoot());
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::CollectExpanded(const QModelIndex &parent, QStringList &dirs) const {
|
|
for (int row = 0; row < proxy_->rowCount(parent); ++row) {
|
|
const QModelIndex child = proxy_->index(row, 0, parent);
|
|
if (tree_->isExpanded(child)) {
|
|
dirs.append(model_->filePath(proxy_->mapToSource(child)));
|
|
CollectExpanded(child, dirs);
|
|
}
|
|
}
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::Refresh(bool force) {
|
|
QStringList dirs{model_->rootPath()};
|
|
CollectExpanded(tree_->rootIndex(), dirs);
|
|
const QPersistentModelIndex top = tree_->indexAt(QPoint(0, 0));
|
|
|
|
bool added = false;
|
|
for (const QString &path : dirs) {
|
|
// One stat per directory: a directory's mtime moves whenever an entry is added or removed, so an
|
|
// unchanged one - typically all but the one being written to - is not read at all.
|
|
const QDateTime mtime = QFileInfo(path).lastModified();
|
|
if (!force && listed_mtime_.value(path) == mtime)
|
|
continue;
|
|
listed_mtime_[path] = mtime;
|
|
|
|
const QModelIndex parent = model_->index(path);
|
|
QSet<QString> listed;
|
|
for (int row = 0; row < model_->rowCount(parent); ++row)
|
|
listed.insert(model_->fileName(model_->index(row, 0, parent)));
|
|
|
|
// QFileSystemModel has no call to re-read a directory it has already listed, but index() of a path
|
|
// it does not know yet adds that path as a new row - in place, so the selection and the expanded
|
|
// directories stay as they are. Same filters as the model's own; only the names it lacks are
|
|
// passed, as index() costs ~10 us a call and a CBF sweep is thousands of files. There is no
|
|
// counterpart for removal: an entry deleted by another host stays listed (a local delete is seen
|
|
// by the model's own watching).
|
|
const QDir dir(path);
|
|
const QStringList names = dir.entryList(model_->nameFilters(),
|
|
QDir::AllDirs | QDir::Files | QDir::NoDotAndDotDot,
|
|
QDir::Unsorted);
|
|
for (const QString &name : names) {
|
|
if (!listed.contains(name)) {
|
|
model_->index(dir.filePath(name));
|
|
added = true;
|
|
}
|
|
}
|
|
}
|
|
if (!added)
|
|
return;
|
|
// Rows added this way land at the end of their directory. Setting the same name filters again marks
|
|
// the model as needing a sort, which sort() then does at once instead of on the next event loop pass.
|
|
model_->setNameFilters(model_->nameFilters());
|
|
model_->sort(0);
|
|
// A row sorted in above the first visible one would push everything below it down; keep the first
|
|
// visible row where it was - unless the view is scrolled to the very top, where a new row should show.
|
|
if (top.isValid() && tree_->verticalScrollBar()->value() > 0)
|
|
tree_->scrollTo(top, QAbstractItemView::PositionAtTop);
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::FollowPath(const QString &path) {
|
|
const QString absolute = QFileInfo(path).absoluteFilePath();
|
|
// Containment by path components, not by string prefix, so /data/foo2 is not inside /data/foo.
|
|
// relativeFilePath hands back an absolute path when there is no relative route at all (a file on
|
|
// another Windows drive).
|
|
const QString relative = QDir(model_->rootPath()).relativeFilePath(absolute);
|
|
if (QDir::isAbsolutePath(relative) || relative == ".." || relative.startsWith("../"))
|
|
return;
|
|
|
|
const QModelIndex index = proxy_->mapFromSource(model_->index(absolute));
|
|
if (!index.isValid())
|
|
return;
|
|
|
|
tree_->setCurrentIndex(index);
|
|
tree_->scrollTo(index);
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::onBrowseClicked() {
|
|
const QString dir = QFileDialog::getExistingDirectory(this, "Select data root", root_edit_->text());
|
|
if (!dir.isEmpty())
|
|
SetRoot(dir);
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::onRootEditingFinished() {
|
|
SetRoot(root_edit_->text());
|
|
}
|
|
|
|
void JFJochViewerFileBrowser::onDoubleClicked(const QModelIndex &index) {
|
|
const QModelIndex source = proxy_->mapToSource(index);
|
|
if (!source.isValid() || model_->isDir(source))
|
|
return; // directories: let the tree's own expand/collapse handle the double click
|
|
emit openDataset(QDir::toNativeSeparators(model_->filePath(source)), 0, 1, false);
|
|
}
|