leonarski_fandClaude Opus 5.5
ebce906025
Combines the stills reader and module work (stills-readers, stills-modgeom, stills-euxfel) onto rc178 with the owner's decisions for this release: - Inputs are files, never a directory; several files of one format are one dataset (the multi-input CLI of the previous commit), so every reader takes an explicit list: JFJochXFELReader::ReadFiles, ReadCXI, ReadPanelTable and JFJochXTCReader::ReadFiles take std::vector<std::string>; ListFiles and xtc::CollectStreams are gone, CanRead* accept a regular file only. - One HDF5 frame reader (JFJochCrystFELReader -> JFJochXFELReader) parameterised by what the file holds: per-shot groups or a 3D stack, a /metadata panel table (posx/posy/angle, pixel size, nominal camera length, pixel mask: as SACLA writes), a Cheetah CXI single panel (square frame; a tiled slab is refused with a message). At most 8 files open; every file checked against the first. Signed 32-bit decode, per-frame 3D masks, panels at their own coffset (from the EuXFEL work). - Frames stored one chunk each are read the way the NXmx reader reads its images: only the chunk's place is asked of HDF5 under its lock (H5Dget_chunk_info_by_coord), the bytes are read through HDF5ImageSource::RawFile (positional, no lock) and decoded outside it - deflate(+shuffle) by zlib, bitshuffle/LZ4 (and zstd, plain LZ4) by CompressedImage's decoders. Bitshuffle/LZ4 used to go through HDF5's filter under the global lock, one frame at a time. - LCLS-I XTC (Rayonix): wavelength from the beam record, pixel size from the Rayonix model and binning, only --detector-distance needed; damaged/compressed frames skipped, a truncated last event dropped; std::ifstream, no POSIX calls: builds with MSVC. No DIALS .expt support at all. - CrystFEL .geom: module geometry only (panel ranges, fs/ss, corners, pixel size, coffset, data path) and optionally the camera length; everything else is ignored with a note in the log. - --mask-good / --mask-bad <file>:<dataset>: a mask array in another HDF5 file, in the frames' stored layout (what a .geom's mask_file used to give). - Gain measured on the background for panel-table and CXI input that states none (--gain overrides). - Detector modules from explicit geometry only (DetectorModules: the reader's panels, an NXmx file with one NXdetector_module per module, or --module-geometry); never from the image. Spots and predictions are mapped through the modules, the image is not re-assembled. - Module refinement in the stills bundle adjustment (Ceres; per-module shift and in-plane turn, gauge on mean shift/turn/scale and on the two tilt-like patterns when the tilt is refined; adopted only where it predicts held-out frames better). Applied for a foreign multi-panel detector (panel table, .geom panels); for a JUNGFRAU measured and reported only. Written to <prefix>_modules.h5 (NXmx NXdetector_module groups), read back with --module-geometry. - A panel-table file's camera length is nominal: with a reference cell it is refined from the cell. - Broker and writer output unchanged: masters keep the single NXdetector_module group, the start message carries no module list. Tests: [CrystFELGeometry], JFJochXFELReader_*, JFJochXTCReader, [JFJochMultiReader], [CellPopulation], SaturationCeiling_*, [DetectorModules], RefineGlobalGeometry_ModulePlacement, [reindex], Sublattice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
64 lines
3.2 KiB
C++
64 lines
3.2 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <cstdint>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
// The LCLS-I XTC container, as far as reading camera frames out of it needs: what the LCLS data
|
|
// system (pdsdata) wrote at LCLS until 2023, and what serial-crystallography depositions from that
|
|
// era (Rayonix at MFX and XPP) are archived as.
|
|
//
|
|
// A file is a sequence of datagrams. Each is a 40-byte header - the event's clock time and timestamp
|
|
// (16 bytes, whose top bits name the transition: Configure, BeginRun, ..., L1Accept for an event),
|
|
// an environment word, and the outermost Xtc header - followed by the Xtc's payload. An Xtc header is
|
|
// five 32-bit words: damage, the source (two words), the type and version (bit 31: compressed), and
|
|
// the extent, which counts the header itself. A container (type Id_Xtc) holds further Xtcs one after
|
|
// another; anything else is a leaf holding one data object:
|
|
//
|
|
// - a camera image, Id_Frame (Camera::FrameV1): width, height, bit depth and an offset added to every
|
|
// pixel, 32 bits each, then the pixels row by row;
|
|
// - the electron beam, Id_EBeam version 7 (Bld::BldDataEBeamV7): a damage mask, then 20 doubles, the
|
|
// 18th the photon energy in eV;
|
|
// - a Rayonix configuration, Id_RayonixConfig version 2: the binning along fast and slow (a byte each)
|
|
// and, further on, the device name ("MX340-HS:...").
|
|
//
|
|
// Everything is little-endian and uncompressed. A run is written as several streams (-s00, -s01, ...)
|
|
// by different recorders, each holding a share of the events. Their order across streams is the event
|
|
// time, which this reader does not reconstruct: stills are independent, so the streams are read one
|
|
// after another.
|
|
namespace xtc {
|
|
|
|
// One camera image: where its pixels start in which file.
|
|
struct Frame {
|
|
std::string file;
|
|
uint64_t offset = 0; // of the first pixel
|
|
uint32_t width = 0;
|
|
uint32_t height = 0;
|
|
uint32_t pixel_offset = 0; // added to every pixel by the camera; taken off when read
|
|
uint64_t source = 0; // the camera, as the Xtc names it
|
|
};
|
|
|
|
// What one stream holds.
|
|
struct Index {
|
|
std::vector<Frame> frames; // every undamaged 16-bit image of every event
|
|
std::vector<double> photon_energy_eV; // of every event with an undamaged electron-beam record
|
|
std::string rayonix_model; // the Rayonix configuration's device name, if any
|
|
uint32_t rayonix_binning = 0;
|
|
size_t damaged = 0; // images left out: damaged, compressed or of another version
|
|
bool truncated = false; // the file ends inside its last datagram, which is left out
|
|
};
|
|
|
|
// True if the path is an XTC file - by its first datagram, not only its name. The "smalldata" copies
|
|
// (.smd.xtc) carry the events without their images and are not read.
|
|
bool CanRead(const std::string &path);
|
|
|
|
Index IndexFile(const std::string &file);
|
|
|
|
// Reads one image as 32-bit pixels into out (width*height values), through scratch.
|
|
void ReadFrame(const Frame &frame, int32_t *out, std::vector<uint8_t> &scratch);
|
|
|
|
} // namespace xtc
|