jfjoch_viewer: keep showing the images of a raw sweep after processing it

A processing job on a CBF, marCCD, SMV or SFRM sweep writes a _process.h5 that
holds the per-image analysis but no images: there is nothing in a directory of
frames for it to link to. The viewer showed that run by reading the file on its
own, so after processing the image area stayed empty.

JFJochHDF5Reader::ReadFile now takes the reader of the sweep the file was made
from; for a file without pixels each image is read from that sweep (frame
/entry/detector/number[i]) and the file's analysis is laid over it. The viewer
passes its open raw reader when it shows such a run. HDF5 inputs are unchanged:
their runs already link the original images.

Tests: JFJochHDF5Reader_ProcessFileOfRawSweep_PixelsFromSweep (a run over one
frame of a two-frame miniCBF sweep shows that frame's pixels under the run's
analysis) and OpenableFilePatterns (the browser/dialog wildcards). The viewer
documentation said the process file carried the images; corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQrPqZ52Tp6rzzZGxf3
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-11-20 02:41:11 +01:00
1 parent 4c322b3e39
commit 6fa1797f8e
7 files changed
+130 -18

No files matched your search

+8 -5
View File
@@ -25,12 +25,14 @@ newer on an Apple Silicon Mac** (M1 and newer; Intel Macs are not supported) and
- Opens HDF5 files written by [`jfjoch_writer`](JFJOCH_WRITER.md) (`*_master.h5`) and the
`*_process.h5` files produced by [`rugnux`](RUGNUX.md). It also opens NXmx files
written by DECTRIS detectors, though that path has had only limited testing.
- Opens PILATUS miniCBF, marCCD and SMV rotation sweeps (the formats listed under
- Opens PILATUS miniCBF, marCCD, SMV and Bruker SFRM rotation sweeps (the formats listed under
[Input formats](RUGNUX_FORMATS.md)). These store one frame per file, so naming any frame opens
the whole sweep it belongs to. A raw frame carries the images and the geometry but no analysis
the whole sweep it belongs to; frames numbered in their extension (`name.001`, `name.0001`) are
listed like any other. A raw frame carries the images and the geometry but no analysis
results, so the spot, reflection and per-image plot panels stay empty until something is computed.
A processing job on such a sweep writes the images into its `_process.h5`, and its run is shown from
that file like a run of an HDF5 dataset; *Original* goes back to the sweep.
A processing job on such a sweep writes its per-image analysis, without the images, into its
`_process.h5`; its run shows the sweep's images with that analysis over them, and *Original* goes
back to the sweep alone.
- Runs an **embedded data-processing pipeline** — the same analysis code as the rest of
Jungfraujoch — performing spot finding, indexing and integration on the displayed image, with the
result drawn over it. This interactive analysis is not written anywhere.
@@ -239,7 +241,8 @@ keyboard without them `Home` / `End` are `Fn`+`←` / `Fn`+`→` and `Page Up` /
## Opening data
- **File ▸ Open** (`Ctrl+O`) — open a local HDF5 file, or any frame of a miniCBF, marCCD or SMV sweep.
- **File ▸ Open** (`Ctrl+O`) — open a local HDF5 file, or any frame of a miniCBF, marCCD, SMV or SFRM
sweep. The *All supported* filter, and the file browser panel, list every file the viewer opens.
- **File ▸ Open HTTP** (`Ctrl+H`) — connect to a `jfjoch_broker` HTTP endpoint to follow a live
collection. The dialog defaults to host `localhost` and port `8080`; these defaults can be
overridden with the environment variables `JUNGFRAUJOCH_HTTP_HOST` and `JUNGFRAUJOCH_HTTP_PORT`.
+22 -6
View File
@@ -4,7 +4,7 @@
#include "JFJochHDF5Reader.h"
#include "../common/JFJochException.h"
void JFJochHDF5Reader::ReadFile(const std::string &filename) {
void JFJochHDF5Reader::ReadFile(const std::string &filename, JFJochReader *pixels) {
std::unique_lock ul(hdf5_mutex);
image_source_.Clear();
snapshots_.clear();
@@ -16,6 +16,7 @@ void JFJochHDF5Reader::ReadFile(const std::string &filename) {
auto open_result = metadata->Open(filename, default_experiment);
has_pixels = open_result.image_layout.format != FileWriterFormat::NoFile;
pixel_reader_ = has_pixels ? nullptr : pixels;
image_source_.Configure(std::move(open_result.image_layout));
// Original file: per-image metadata is co-located with the pixels.
metadata->UseImageSourceForMetadata(&image_source_);
@@ -44,6 +45,7 @@ uint64_t JFJochHDF5Reader::GetNumberOfImages() const {
void JFJochHDF5Reader::Close() {
std::unique_lock ul(hdf5_mutex);
image_source_.Clear();
pixel_reader_ = nullptr;
snapshots_.clear();
active_metadata_.reset();
active_snapshot_.clear();
@@ -88,20 +90,34 @@ bool JFJochHDF5Reader::LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset
std::vector<uint8_t> &buffer,
int64_t image_number,
bool update_dataset) {
std::unique_lock ul(hdf5_mutex);
(void) update_dataset;
if (!dataset || !has_pixels)
if (!dataset)
return false;
// A file without images takes its pixels from the sweep it was processed from. That reader is asked
// outside hdf5_mutex, which it may take itself.
int64_t metadata_number = image_number;
if (!has_pixels) {
if (!pixel_reader_)
return false;
if (!dataset->source_image_number.empty())
metadata_number = dataset->source_image_number.at(image_number);
auto pixel_dataset = std::const_pointer_cast<JFJochReaderDataset>(pixel_reader_->GetDataset());
if (!pixel_reader_->LoadImage_i(pixel_dataset, message, buffer, metadata_number, false))
return false;
}
std::unique_lock ul(hdf5_mutex);
if (!active_metadata_)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"Cannot load image if file not loaded");
// Pixels from the shared image source, per-image metadata from the active snapshot.
auto loc = GetImageLocation(image_number);
message.image = image_source_.ReadImageAt(buffer, loc);
if (has_pixels)
message.image = image_source_.ReadImageAt(buffer, GetImageLocation(image_number));
message.number = image_number;
active_metadata_->FillPerImage(message, image_number, dataset);
active_metadata_->FillPerImage(message, metadata_number, dataset);
return true;
}
+6 -2
View File
@@ -32,8 +32,9 @@ class JFJochHDF5Reader : public JFJochReader {
std::string active_snapshot_;
size_t number_of_images = 0;
// False for a file of per-image analysis without the images (a _process.h5 of an input it could
// not link to): its images have metadata, but no pixels to load.
// not link to): its images have metadata, and their pixels come from pixel_reader_ where one is given.
bool has_pixels = false;
JFJochReader *pixel_reader_ = nullptr;
bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
DataMessage &message,
@@ -46,7 +47,10 @@ class JFJochHDF5Reader : public JFJochReader {
public:
~JFJochHDF5Reader() override = default;
void ReadFile(const std::string &filename);
// pixels is the reader of the sweep the file was processed from, for a file that holds no images of
// its own - the _process.h5 of a CBF, marCCD, SMV or SFRM sweep: image i of the file is then frame
// /entry/detector/number[i] of that sweep. The caller keeps it open while this file is.
void ReadFile(const std::string &filename, JFJochReader *pixels = nullptr);
uint64_t GetNumberOfImages() const override;
+3 -1
View File
@@ -17,8 +17,10 @@
#include "JFJochReaderSpots.h"
class JFJochReader {
// Reads each of its inputs' frames through them (JFJochMultiReader.h).
// Read each of their inputs' frames through them (JFJochMultiReader.h), or the frames of the sweep a
// file without pixels was made from (JFJochHDF5Reader.h).
friend class JFJochMultiReader;
friend class JFJochHDF5Reader;
mutable std::mutex m;
std::shared_ptr<JFJochReaderDataset> dataset;
+86
View File
@@ -13,6 +13,7 @@
#include "../reader/JFJochSMVReader.h"
#include "../reader/JFJochXTCReader.h"
#include "../reader/MiniCBF.h"
#include "../reader/FileNamePatterns.h"
#include "../rugnux/RugnuxDefaults.h"
#include "../compression/JFJochCompressor.h"
#include "../frame_serialize/CBORStream2Serializer.h"
@@ -26,6 +27,7 @@
#include <future>
#include <iomanip>
#include <random>
#include <regex>
#include <sstream>
TEST_CASE("HDF5DataType_Sign","[HDF5]") {
@@ -5082,6 +5084,90 @@ TEST_CASE("JFJochCBFReader_GzippedSweep", "[HDF5][Full]") {
}
}
// The _process.h5 of a raw sweep holds the per-image analysis but not the images: there is nothing
// in a directory of CBFs for it to link to. Opened over the sweep's reader, it shows the sweep's
// pixels under its own analysis, image i being the sweep frame it records as /entry/detector/number[i].
TEST_CASE("JFJochHDF5Reader_ProcessFileOfRawSweep_PixelsFromSweep", "[HDF5][Full][portable]") {
RegisterHDF5Filter();
const int64_t nx = 24, ny = 16;
WriteMiniCBFSweep("cbfproc", "# Oscillation_axis OMEGA\n", nx, ny);
JFJochCBFReader sweep;
REQUIRE_NOTHROW(sweep.ReadFiles("cbfproc_0001.cbf"));
REQUIRE(sweep.GetNumberOfImages() == 2);
// A run over the second frame only, as rugnux writes one for a non-HDF5 input: no images.
{
DiffractionExperiment x(sweep.GetDataset()->experiment);
x.FilePrefix("cbfproc_run").ImagesPerTrigger(1).OverwriteExistingFiles(true)
.SetFileWriterFormat(FileWriterFormat::NXmxIntegrated);
StartMessage start_message;
x.FillMessage(start_message);
start_message.master_suffix = "process";
start_message.write_images = false;
FileWriter file_set(start_message);
ScanResultGenerator generator(x);
const std::vector<int32_t> unused(nx * ny, 0);
DataMessage message{};
message.image = CompressedImage(unused, nx, ny);
message.number = 0;
message.original_number = 1;
message.indexing_result = true;
message.bkg_estimate = 42.0;
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
generator.Add(message);
EndMessage end_message;
end_message.max_image_number = 1;
generator.FillEndMessage(end_message);
file_set.WriteHDF5(end_message);
file_set.Finalize();
}
{
JFJochHDF5Reader alone;
REQUIRE_NOTHROW(alone.ReadFile("cbfproc_run_process.h5"));
REQUIRE(alone.GetNumberOfImages() == 1);
CHECK(!alone.LoadImage(0)); // no pixels of its own
JFJochHDF5Reader run;
REQUIRE_NOTHROW(run.ReadFile("cbfproc_run_process.h5", &sweep));
REQUIRE(run.GetNumberOfImages() == 1);
const auto image = run.LoadImage(0);
REQUIRE(image);
CHECK(image->Image().size() == static_cast<size_t>(nx * ny));
CHECK(image->ImageData().number == 0);
REQUIRE(image->ImageData().bkg_estimate.has_value());
CHECK(image->ImageData().bkg_estimate.value() == Catch::Approx(42.0));
REQUIRE(image->ImageData().indexing_result.has_value());
CHECK(image->ImageData().indexing_result.value());
}
sweep.Close();
RemoveMiniCBFSweep("cbfproc");
remove("cbfproc_run_process.h5");
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
}
// The viewer lists a file when its name matches one of OPENABLE_FILE_PATTERNS (QDir wildcards,
// matched without regard to case, as QFileSystemModel does by default).
TEST_CASE("OpenableFilePatterns", "[portable]") {
auto listed = [](const std::string &name) {
for (const auto &p: OPENABLE_FILE_PATTERNS) {
std::string re;
for (char c: p)
re += (c == '*') ? std::string(".*") : (c == '.') ? std::string("\\.") : std::string(1, c);
if (std::regex_match(name, std::regex(re, std::regex::icase)))
return true;
}
return false;
};
for (const char *name: {"run_master.h5", "run_process.h5", "run_0001.cbf", "run_0001.cbf.gz",
"formal.001", "frame.0001", "frame.00001", "x_001.img", "x_001.mccd",
"x_001.osc", "x_001.smv", "x_0001.sfrm", "X_0001.CBF", "r0001.xtc",
"run.cxi"})
CHECK(listed(name));
for (const char *name: {"run_data_000001.h5", "XDS.INP", "notes.txt", "run.mtz", "file.1"})
CHECK_FALSE(listed(name));
}
// The DECTRIS "hdf5 nexus v2024.2 nxmx" format: /entry/data/data is 4D, [image, channel, y, x], a
// VDS over the data files, with the pixel mask kept per channel. Only the first channel is read.
TEST_CASE("JFJochReader_DECTRISMultichannel", "[HDF5][Full]") {
+1 -1
View File
@@ -1266,7 +1266,7 @@ void JFJochImageReadingWorker::ActivateSnapshot_i(const QString &name) {
if (id == "Original") {
file_reader = raw_reader_;
} else {
hdf5_reader.ReadFile(raw_runs_.at(id).path);
hdf5_reader.ReadFile(raw_runs_.at(id).path, raw_reader_); // analysis from the run, pixels from the sweep
raw_runs_.at(id).dataset = hdf5_reader.GetDataset();
file_reader = &hdf5_reader;
}
+4 -3
View File
@@ -100,9 +100,10 @@ private:
bool cbf_mode = false; // "a raw sweep is open", whichever of the three formats it is
// The runs of a raw sweep. A run of an HDF5 file is a snapshot - its metadata read back over the
// file's own images - but a raw sweep has no HDF5 to hang one on. Its run's _process.h5 carries
// the images as well, so showing that run means reading that file (with hdf5_reader), and showing
// "Original" means reading the sweep again (raw_reader_).
// file's own images - but a raw sweep has no HDF5 to hang one on. Its run's _process.h5 holds the
// per-image analysis without the images, so showing that run means reading that file (with
// hdf5_reader) over the sweep's pixels (raw_reader_, kept open), and showing "Original" means
// reading the sweep alone again.
struct RawRun {
std::string path;
std::shared_ptr<const JFJochReaderDataset> dataset;