Serial XFEL stills inputs: readers from the files themselves, module geometry and its refinement
Combines the stills reader and module work (stills-readers, stills-modgeom, stills-euxfel) onto rc178 with the owner's decisions for this release: - Inputs are files, never a directory; several files of one format are one dataset (the multi-input CLI of the previous commit), so every reader takes an explicit list: JFJochXFELReader::ReadFiles, ReadCXI, ReadPanelTable and JFJochXTCReader::ReadFiles take std::vector<std::string>; ListFiles and xtc::CollectStreams are gone, CanRead* accept a regular file only. - One HDF5 frame reader (JFJochCrystFELReader -> JFJochXFELReader) parameterised by what the file holds: per-shot groups or a 3D stack, a /metadata panel table (posx/posy/angle, pixel size, nominal camera length, pixel mask: as SACLA writes), a Cheetah CXI single panel (square frame; a tiled slab is refused with a message). At most 8 files open; every file checked against the first. Signed 32-bit decode, per-frame 3D masks, panels at their own coffset (from the EuXFEL work). - Frames stored one chunk each are read the way the NXmx reader reads its images: only the chunk's place is asked of HDF5 under its lock (H5Dget_chunk_info_by_coord), the bytes are read through HDF5ImageSource::RawFile (positional, no lock) and decoded outside it - deflate(+shuffle) by zlib, bitshuffle/LZ4 (and zstd, plain LZ4) by CompressedImage's decoders. Bitshuffle/LZ4 used to go through HDF5's filter under the global lock, one frame at a time. - LCLS-I XTC (Rayonix): wavelength from the beam record, pixel size from the Rayonix model and binning, only --detector-distance needed; damaged/compressed frames skipped, a truncated last event dropped; std::ifstream, no POSIX calls: builds with MSVC. No DIALS .expt support at all. - CrystFEL .geom: module geometry only (panel ranges, fs/ss, corners, pixel size, coffset, data path) and optionally the camera length; everything else is ignored with a note in the log. - --mask-good / --mask-bad <file>:<dataset>: a mask array in another HDF5 file, in the frames' stored layout (what a .geom's mask_file used to give). - Gain measured on the background for panel-table and CXI input that states none (--gain overrides). - Detector modules from explicit geometry only (DetectorModules: the reader's panels, an NXmx file with one NXdetector_module per module, or --module-geometry); never from the image. Spots and predictions are mapped through the modules, the image is not re-assembled. - Module refinement in the stills bundle adjustment (Ceres; per-module shift and in-plane turn, gauge on mean shift/turn/scale and on the two tilt-like patterns when the tilt is refined; adopted only where it predicts held-out frames better). Applied for a foreign multi-panel detector (panel table, .geom panels); for a JUNGFRAU measured and reported only. Written to <prefix>_modules.h5 (NXmx NXdetector_module groups), read back with --module-geometry. - A panel-table file's camera length is nominal: with a reference cell it is refined from the cell. - Broker and writer output unchanged: masters keep the single NXdetector_module group, the start message carries no module list. Tests: [CrystFELGeometry], JFJochXFELReader_*, JFJochXTCReader, [JFJochMultiReader], [CellPopulation], SaturationCeiling_*, [DetectorModules], RefineGlobalGeometry_ModulePlacement, [reindex], Sublattice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
This commit is contained in:
1 parent
6557ba87f4
commit
ebce906025
46 files changed
+3686
-936
No files matched your search
@@ -4,6 +4,7 @@
|
||||
#include "MXAnalysisWithoutFPGA.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <spdlog/spdlog.h>
|
||||
|
||||
#include "spot_finding/StrongPixelSet.h"
|
||||
@@ -205,14 +206,14 @@ void MXAnalysisWithoutFPGA::Analyze(DataMessage &output,
|
||||
int64_t image_number) {
|
||||
if (count_crowding)
|
||||
crowding_engine->CountNeighbourCrowding(predicted, npredicted);
|
||||
return bragg_engine->Run(*preprocessor_buffer, predicted, npredicted, image_number);
|
||||
return IntegrateOnImage(predicted, npredicted, image_number);
|
||||
};
|
||||
// The supercell probe integrates on the same engine, but what the engine counts is the run's
|
||||
// measurement of its own stencil, and the probe's reflections are not the run's: keep them out.
|
||||
const auto probe_integrate_fn = [this](const std::vector<Reflection> &predicted, size_t npredicted,
|
||||
int64_t image_number) {
|
||||
const BraggIntegrationCounts before = bragg_engine->Counts();
|
||||
auto ret = bragg_engine->Run(*preprocessor_buffer, predicted, npredicted, image_number);
|
||||
auto ret = IntegrateOnImage(predicted, npredicted, image_number);
|
||||
const BraggIntegrationCounts after = bragg_engine->Counts();
|
||||
probe_counts.predicted += after.predicted - before.predicted;
|
||||
probe_counts.bkg_starved += after.bkg_starved - before.bkg_starved;
|
||||
@@ -504,3 +505,31 @@ void MXAnalysisWithoutFPGA::UpdateMaskResolution(const SpotFindingSettings &sett
|
||||
spotFinder->SetResolutionMaskBits(*mask_resolution);
|
||||
adaptiveSpotFinder->SetResolutionMaskBits(*mask_resolution);
|
||||
}
|
||||
|
||||
std::vector<Reflection> MXAnalysisWithoutFPGA::IntegrateOnImage(const std::vector<Reflection> &predicted,
|
||||
size_t npredicted, int64_t image_number) {
|
||||
const auto &modules = experiment.GetModules();
|
||||
if (modules.empty())
|
||||
return bragg_engine->Run(*preprocessor_buffer, predicted, npredicted, image_number);
|
||||
|
||||
// Predictions are where the reflections fall on the detector plane; the engine reads the image, so
|
||||
// each goes to where its module put those pixels, and comes back to the plane once integrated.
|
||||
std::vector<Reflection> on_image(predicted.begin(), predicted.begin() + static_cast<int64_t>(npredicted));
|
||||
for (auto &r: on_image) {
|
||||
const auto [x, y] = modules.PlaneToImage(r.predicted_x, r.predicted_y);
|
||||
r.predicted_x = static_cast<float>(x);
|
||||
r.predicted_y = static_cast<float>(y);
|
||||
}
|
||||
auto ret = bragg_engine->Run(*preprocessor_buffer, on_image, npredicted, image_number);
|
||||
for (auto &r: ret) {
|
||||
const auto [px, py] = modules.ImageToPlane(r.predicted_x, r.predicted_y);
|
||||
r.predicted_x = static_cast<float>(px);
|
||||
r.predicted_y = static_cast<float>(py);
|
||||
if (std::isfinite(r.observed_x) && std::isfinite(r.observed_y)) {
|
||||
const auto [ox, oy] = modules.ImageToPlane(r.observed_x, r.observed_y);
|
||||
r.observed_x = static_cast<float>(ox);
|
||||
r.observed_y = static_cast<float>(oy);
|
||||
}
|
||||
}
|
||||
return ret;
|
||||
}
|
||||
Reference in new issue
Block a user