The .geom parser now takes the masking information the format defines:
- mask arrays: mask/mask_file/mask_good/mask_bad (mask 0) and maskN_data/maskN_file/maskN_goodbits/
maskN_badbits for N = 0..7; global keys hold for the panels after them, a panel may override. A mask
that is the same on every panel is one whole-frame mask, otherwise one per panel, applied on that
panel's pixels only.
- a relative mask_file is looked for in the working directory (as CrystFEL does), then beside the .geom.
- bad regions (bad*): a lab-plane box (min_x..max_y) or a fs/ss range on a panel (or, old syntax, on
the whole frame); mask_edge_pixels and no_index.
JFJochXFELReader already had the static pixel mask and the per-frame mask path for these; the masks
go through them unchanged. A mask whose path holds % (one per shot group) is applied frame by frame
like a 3D one. Where the geometry names mask arrays, the data file's own Cheetah masks (CXI
mask_shared and per-shot mask with Cheetah's generic bad bits, /metadata/pixelmask) are not added:
the geometry says which bits matter (e.g. an AGIPD mask_bad that includes bit 32). --mask-good /
--mask-bad still add their masks to these.
Tests: [CrystFELGeometry] (new keys, per-panel override, relative mask file), JFJochXFELReader_*
(new GeometryMasks case on a synthetic CXI + .geom), [JFJochMultiReader], JFJochXTCReader*.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
Combines the stills reader and module work (stills-readers, stills-modgeom, stills-euxfel) onto
rc178 with the owner's decisions for this release:
- Inputs are files, never a directory; several files of one format are one dataset (the multi-input
CLI of the previous commit), so every reader takes an explicit list: JFJochXFELReader::ReadFiles,
ReadCXI, ReadPanelTable and JFJochXTCReader::ReadFiles take std::vector<std::string>; ListFiles and
xtc::CollectStreams are gone, CanRead* accept a regular file only.
- One HDF5 frame reader (JFJochCrystFELReader -> JFJochXFELReader) parameterised by what the file holds:
per-shot groups or a 3D stack, a /metadata panel table (posx/posy/angle, pixel size, nominal camera
length, pixel mask: as SACLA writes), a Cheetah CXI single panel (square frame; a tiled slab is
refused with a message). At most 8 files open; every file checked against the first. Signed 32-bit
decode, per-frame 3D masks, panels at their own coffset (from the EuXFEL work).
- Frames stored one chunk each are read the way the NXmx reader reads its images: only the chunk's
place is asked of HDF5 under its lock (H5Dget_chunk_info_by_coord), the bytes are read through
HDF5ImageSource::RawFile (positional, no lock) and decoded outside it - deflate(+shuffle) by zlib,
bitshuffle/LZ4 (and zstd, plain LZ4) by CompressedImage's decoders. Bitshuffle/LZ4 used to go through
HDF5's filter under the global lock, one frame at a time.
- LCLS-I XTC (Rayonix): wavelength from the beam record, pixel size from the Rayonix model and
binning, only --detector-distance needed; damaged/compressed frames skipped, a truncated last event
dropped; std::ifstream, no POSIX calls: builds with MSVC. No DIALS .expt support at all.
- CrystFEL .geom: module geometry only (panel ranges, fs/ss, corners, pixel size, coffset, data path)
and optionally the camera length; everything else is ignored with a note in the log.
- --mask-good / --mask-bad <file>:<dataset>: a mask array in another HDF5 file, in the frames' stored
layout (what a .geom's mask_file used to give).
- Gain measured on the background for panel-table and CXI input that states none (--gain overrides).
- Detector modules from explicit geometry only (DetectorModules: the reader's panels, an NXmx file with
one NXdetector_module per module, or --module-geometry); never from the image. Spots and predictions
are mapped through the modules, the image is not re-assembled.
- Module refinement in the stills bundle adjustment (Ceres; per-module shift and in-plane turn, gauge
on mean shift/turn/scale and on the two tilt-like patterns when the tilt is refined; adopted only
where it predicts held-out frames better). Applied for a foreign multi-panel detector (panel table,
.geom panels); for a JUNGFRAU measured and reported only. Written to <prefix>_modules.h5 (NXmx
NXdetector_module groups), read back with --module-geometry.
- A panel-table file's camera length is nominal: with a reference cell it is refined from the cell.
- Broker and writer output unchanged: masters keep the single NXdetector_module group, the start
message carries no module list.
Tests: [CrystFELGeometry], JFJochXFELReader_*, JFJochXTCReader, [JFJochMultiReader], [CellPopulation],
SaturationCeiling_*, [DetectorModules], RefineGlobalGeometry_ModulePlacement, [reindex], Sublattice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
A CrystFEL .geom (current and old syntax) is parsed for panels (pixel ranges, fs/ss, corners),
clen/coffset (a number, or an HDF5 path read as mm), res, adu_per_eV/adu_per_photon, max_adu,
no_index, bad regions (panel, whole-frame or lab-plane) and the HDF5 mask (mask/mask_file,
mask_good/mask_bad). JFJochCrystFELReader reads the frames the geometry's data path names - a 3D
stack (Cheetah CXI), one 2D frame per file, or one group per shot (/%/data) - from a file or a
directory, puts the panels on one flat image at their rounded positions with gaps and bad regions
masked, and converts to photons where a gain is stated (otherwise the pre-scan measures it). One
wavelength and one camera length per run: values taken from the files are averaged over the first
file. Panel positions are used as given. Frames stored as one deflate(+shuffle) chunk are read raw
under the HDF5 lock and decoded outside it.
Replaces the MPCCD-specific reader of the previous commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leonarski_fandClaude Opus 5.5