Files
Jungfraujoch/process/JFJochProcess.h
T
leonarski_f e4b3064254 Scaling: reference dataset, 3D combine (-P rot3d), R-meas, profile-fit integrator
Reference dataset (a): LoadReferenceMtz adds column selection + cell/SG/resolution +
a data-vs-reference consistency check; jfjoch_process/jfjoch_scale gain
--reference-column; the viewer gets a Reference section in the MX settings dock
(worker-owned, independent of the loaded dataset) that flows into reprocessing jobs.

3D combine (-P rot3d): Combine3D weight-sums a reflection's per-frame partials into one
counting-limited full before merging (orthogonal ScalingSettings::combine_3d flag, not a
partiality model), with a de-biased Poisson variance. Crystal 2: ISa 1.7->8.4, R-meas
~67%->18.9%, intensities unchanged (CCref held).

Quality metrics (b): R-meas (Diederichs-Karplus) + redundancy columns in MergeStats; ISa
logged. jfjoch fulls 18.9% vs XDS 4.5% (same ASU/run).

Profile-fit integrator (experimental): ProfileIntegrate2D (--integrator gaussian|empirical)
is a reference-free, rot3d-compatible profile-fit extraction (the decomposed PixelRefine
intensity step). Gaussian: R-meas 18.9->14.6%, ISa ->9.5. Anisotropy/per-region add nothing
(the discriminating info is in the discarded rocking direction). See NEXTGEN_INTEGRATOR.md.
--dump-observations exports the unmerged fulls for XDS comparison.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 20:43:04 +02:00

104 lines
4.0 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <atomic>
#include <optional>
#include <string>
#include <vector>
#include "../common/DiffractionExperiment.h"
#include "../common/PixelMask.h"
#include "../common/CrystalLattice.h"
#include "../common/Reflection.h" // MergedReflection
#include "../common/JFJochMessages.h" // DataMessage
#include "../common/JFJochReceiverPlots.h" // MeanProcessingTime
#include "../image_analysis/spot_finding/SpotFindingSettings.h"
class JFJochHDF5Reader;
// Offline reprocessing of a stored Jungfraujoch HDF5 dataset, shared by jfjoch_process,
// jfjoch_azint and (later) the viewer. The full processing workflow lives here, not in the CLIs:
// setup, an optional two-pass rotation-indexing pre-pass, a parallel per-image loop (std::thread),
// an optional scaling/merging post-pass, and the _process.h5 output. The detector geometry and all
// algorithm settings are configured on the DiffractionExperiment by the caller; ProcessConfig only
// carries run control. Cancellable from any thread (e.g. SIGINT or a GUI button) via Cancel().
enum class ProcessMode {
AzimuthalIntegration, // preprocess + azimuthal integration only (jfjoch_azint)
FullAnalysis // spot finding + indexing + refinement + integration (jfjoch_process)
};
struct ProcessConfig {
ProcessMode mode = ProcessMode::FullAnalysis;
int start_image = 0;
int end_image = -1; // -1 => to the end of the dataset
int stride = 1;
int nthreads = 1;
// Output prefix for the _process.h5 (and scaled reflections). Empty => process without writing.
std::string output_prefix;
SpotFindingSettings spot_finding; // FullAnalysis spot finding
// Rotation indexing (FullAnalysis)
bool rotation_indexing = false;
bool two_pass_rotation = true;
bool reuse_rotation_spots = true;
int rotation_indexing_image_count = 30;
std::optional<CrystalLattice> forced_rotation_lattice;
// Scaling / merging (FullAnalysis). reference_data (from a reference MTZ) also drives the
// per-image live scaling path when present.
bool run_scaling = false;
int64_t scaling_iter = 3;
std::vector<MergedReflection> reference_data;
// Diagnostic: if set, the -P rot3d combine writes the unmerged fulls here (for comparison vs XDS).
std::string observation_dump_path;
};
struct ProcessResult {
bool cancelled = false;
uint64_t images_processed = 0;
double processing_time_s = 0.0;
double frame_rate_hz = 0.0;
double throughput_MBs = 0.0;
std::optional<float> indexing_rate;
std::optional<UnitCell> consensus_cell;
bool rotation_lattice_found = false;
MeanProcessingTime mean_processing_time{};
std::optional<std::string> written_master_path;
std::string merge_statistics_text; // populated when scaling/merging ran
};
// Callbacks for progress and live results. Methods may be called from worker threads, so an
// implementation must be thread-safe. The default no-ops suit the CLIs.
class JFJochProcessObserver {
public:
virtual ~JFJochProcessObserver() = default;
virtual void OnPhase(const std::string &phase) {}
virtual void OnProgress(uint64_t done, uint64_t total) {}
virtual void OnImageProcessed(const DataMessage &msg) {}
};
class JFJochProcess {
JFJochHDF5Reader &reader_;
DiffractionExperiment experiment_;
PixelMask pixel_mask_;
ProcessConfig config_;
std::atomic<bool> cancelled_{false};
public:
JFJochProcess(JFJochHDF5Reader &reader, DiffractionExperiment experiment,
PixelMask pixel_mask, ProcessConfig config);
// Runs the configured workflow to completion or until Cancel(). Throws on setup failure.
ProcessResult Run(JFJochProcessObserver *observer = nullptr);
// Request cancellation; safe to call from any thread (the worker loop checks between images).
void Cancel() { cancelled_ = true; }
bool IsCancelled() const { return cancelled_; }
};