Serial XFEL stills inputs: readers from the files themselves, module geometry and its refinement
Combines the stills reader and module work (stills-readers, stills-modgeom, stills-euxfel) onto rc178 with the owner's decisions for this release: - Inputs are files, never a directory; several files of one format are one dataset (the multi-input CLI of the previous commit), so every reader takes an explicit list: JFJochXFELReader::ReadFiles, ReadCXI, ReadPanelTable and JFJochXTCReader::ReadFiles take std::vector<std::string>; ListFiles and xtc::CollectStreams are gone, CanRead* accept a regular file only. - One HDF5 frame reader (JFJochCrystFELReader -> JFJochXFELReader) parameterised by what the file holds: per-shot groups or a 3D stack, a /metadata panel table (posx/posy/angle, pixel size, nominal camera length, pixel mask: as SACLA writes), a Cheetah CXI single panel (square frame; a tiled slab is refused with a message). At most 8 files open; every file checked against the first. Signed 32-bit decode, per-frame 3D masks, panels at their own coffset (from the EuXFEL work). - Frames stored one chunk each are read the way the NXmx reader reads its images: only the chunk's place is asked of HDF5 under its lock (H5Dget_chunk_info_by_coord), the bytes are read through HDF5ImageSource::RawFile (positional, no lock) and decoded outside it - deflate(+shuffle) by zlib, bitshuffle/LZ4 (and zstd, plain LZ4) by CompressedImage's decoders. Bitshuffle/LZ4 used to go through HDF5's filter under the global lock, one frame at a time. - LCLS-I XTC (Rayonix): wavelength from the beam record, pixel size from the Rayonix model and binning, only --detector-distance needed; damaged/compressed frames skipped, a truncated last event dropped; std::ifstream, no POSIX calls: builds with MSVC. No DIALS .expt support at all. - CrystFEL .geom: module geometry only (panel ranges, fs/ss, corners, pixel size, coffset, data path) and optionally the camera length; everything else is ignored with a note in the log. - --mask-good / --mask-bad <file>:<dataset>: a mask array in another HDF5 file, in the frames' stored layout (what a .geom's mask_file used to give). - Gain measured on the background for panel-table and CXI input that states none (--gain overrides). - Detector modules from explicit geometry only (DetectorModules: the reader's panels, an NXmx file with one NXdetector_module per module, or --module-geometry); never from the image. Spots and predictions are mapped through the modules, the image is not re-assembled. - Module refinement in the stills bundle adjustment (Ceres; per-module shift and in-plane turn, gauge on mean shift/turn/scale and on the two tilt-like patterns when the tilt is refined; adopted only where it predicts held-out frames better). Applied for a foreign multi-panel detector (panel table, .geom panels); for a JUNGFRAU measured and reported only. Written to <prefix>_modules.h5 (NXmx NXdetector_module groups), read back with --module-geometry. - A panel-table file's camera length is nominal: with a reference cell it is refined from the cell. - Broker and writer output unchanged: masters keep the single NXdetector_module group, the start message carries no module list. Tests: [CrystFELGeometry], JFJochXFELReader_*, JFJochXTCReader, [JFJochMultiReader], [CellPopulation], SaturationCeiling_*, [DetectorModules], RefineGlobalGeometry_ModulePlacement, [reindex], Sublattice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
This commit is contained in:
1 parent
6557ba87f4
commit
ebce906025
46 files changed
+3686
-936
No files matched your search
@@ -6,6 +6,7 @@
|
||||
#include <set>
|
||||
|
||||
#include "HDF5MetadataSource.h"
|
||||
#include "NXmxModules.h"
|
||||
#include "spdlog/fmt/fmt.h"
|
||||
#include "../image_analysis/bragg_integration/CalcISigma.h"
|
||||
#include "../image_analysis/spot_finding/SpotUtils.h"
|
||||
@@ -28,7 +29,9 @@ static Coord McStasToInternal(const std::vector<double> &v) {
|
||||
// separated from the tilt by looking at the module alone - so it is left as it is rather than
|
||||
// approximated. Every real file examined here is exactly discrete.
|
||||
static std::optional<DetectorOrientation> ReadModuleOrientation(HDF5Object *file) {
|
||||
const std::string base = "/entry/instrument/detector/module/";
|
||||
// One module for the whole image, or - where the file states its modules one by one - the first.
|
||||
const std::string base = file->IsDataSet("/entry/instrument/detector/module/fast_pixel_direction")
|
||||
? "/entry/instrument/detector/module/" : "/entry/instrument/detector/module_0/";
|
||||
if (!file->IsDataSet(base + "fast_pixel_direction") || !file->IsDataSet(base + "slow_pixel_direction"))
|
||||
return {};
|
||||
|
||||
@@ -1092,6 +1095,9 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen
|
||||
detector.MinCountTime(std::chrono::microseconds(0));
|
||||
detector.ReadOutTime(std::chrono::nanoseconds(0));
|
||||
dataset->experiment.Detector(detector);
|
||||
// The modules, where the file states them one by one (module_0, module_1, ...).
|
||||
dataset->experiment.Modules(ReadNXmxModules(*master_file, detector.GetPixelSize_mm() / 1000.0,
|
||||
detector.GetImageOrientation()));
|
||||
|
||||
// frame_time is the period between frames, count_time the exposure within one. NXmx requires
|
||||
// neither, and a master written outside the DECTRIS toolchain often carries only count_time;
|
||||
|
||||
Reference in new issue
Block a user