Commit Graph
3 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 5.5 a8202789f5 DetectorModules: panel look-up by id, plane -> panel, plane bounds and a per-pixel walk; mask size checked
Stage 0 of analysing panel detectors in their stored layout. The table of modules already is the panel
table; what the consumers converted in the next stages need from it:

- ImageToPlane / PlaneToImage on a known panel, no search; panel -1 (empty table) is the identity.
- PlaneToPanel: the panel holding a plane point and its image position, nullopt in a gap (never passed
  through, unlike the existing PlaneToImage).
- PlaneBounds: the box of all panel footprints on the plane; with an empty table exactly the image
  rectangle today's prediction bounds test uses.
- ForEachPixel: f(panel, index, px, py) panel by panel, row by row, gaps not visited; with an empty
  table the plain row-major loop with plane == image.

The image preprocessors (CPU, GPU) and the Bragg engines (CPU, GPU; Metal derives from CPU) now refuse
a pixel mask whose size is not the image's, instead of reading a mask of another size out of range.
Every reader sizes the experiment's detector, and so the mask, by the image it reads, so nothing that
runs today is refused. Nothing calls the new functions yet: no output changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
2026-10-21 09:41:35 +02:00
leonarski_fandClaude Opus 5.5 b9ada7914b DetectorModules.h: include <cstddef> for size_t
Build Packages / Create release (push) Successful in 22s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 4m59s
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 15m44s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 9m6s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 21m48s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 16m39s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 25m26s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 25m57s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 15m45s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 17m54s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 17m31s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 22m33s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 26m35s
Build Packages / Generate python client (push) Successful in 23s
Build Packages / Build documentation (push) Successful in 1m15s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 29m39s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 25m18s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 19m20s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 25m50s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 19m0s
Build Packages / Unit tests (push) Successful in 1h41m33s
Build Packages / Validation Battery (push) Successful in 34m52s
GCC 12 (the Ubuntu 22.04 CI job) does not pull size_t in through <cstdint>/<vector>, so the header failed
to compile there. Checked: all 601 project sources pass -fsyntax-only with GCC 12.2 (gcc-toolset-12).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
2026-10-21 08:48:32 +02:00
leonarski_fandClaude Opus 5.5 ebce906025 Serial XFEL stills inputs: readers from the files themselves, module geometry and its refinement
Combines the stills reader and module work (stills-readers, stills-modgeom, stills-euxfel) onto
rc178 with the owner's decisions for this release:

- Inputs are files, never a directory; several files of one format are one dataset (the multi-input
  CLI of the previous commit), so every reader takes an explicit list: JFJochXFELReader::ReadFiles,
  ReadCXI, ReadPanelTable and JFJochXTCReader::ReadFiles take std::vector<std::string>; ListFiles and
  xtc::CollectStreams are gone, CanRead* accept a regular file only.
- One HDF5 frame reader (JFJochCrystFELReader -> JFJochXFELReader) parameterised by what the file holds:
  per-shot groups or a 3D stack, a /metadata panel table (posx/posy/angle, pixel size, nominal camera
  length, pixel mask: as SACLA writes), a Cheetah CXI single panel (square frame; a tiled slab is
  refused with a message). At most 8 files open; every file checked against the first. Signed 32-bit
  decode, per-frame 3D masks, panels at their own coffset (from the EuXFEL work).
- Frames stored one chunk each are read the way the NXmx reader reads its images: only the chunk's
  place is asked of HDF5 under its lock (H5Dget_chunk_info_by_coord), the bytes are read through
  HDF5ImageSource::RawFile (positional, no lock) and decoded outside it - deflate(+shuffle) by zlib,
  bitshuffle/LZ4 (and zstd, plain LZ4) by CompressedImage's decoders. Bitshuffle/LZ4 used to go through
  HDF5's filter under the global lock, one frame at a time.
- LCLS-I XTC (Rayonix): wavelength from the beam record, pixel size from the Rayonix model and
  binning, only --detector-distance needed; damaged/compressed frames skipped, a truncated last event
  dropped; std::ifstream, no POSIX calls: builds with MSVC. No DIALS .expt support at all.
- CrystFEL .geom: module geometry only (panel ranges, fs/ss, corners, pixel size, coffset, data path)
  and optionally the camera length; everything else is ignored with a note in the log.
- --mask-good / --mask-bad <file>:<dataset>: a mask array in another HDF5 file, in the frames' stored
  layout (what a .geom's mask_file used to give).
- Gain measured on the background for panel-table and CXI input that states none (--gain overrides).
- Detector modules from explicit geometry only (DetectorModules: the reader's panels, an NXmx file with
  one NXdetector_module per module, or --module-geometry); never from the image. Spots and predictions
  are mapped through the modules, the image is not re-assembled.
- Module refinement in the stills bundle adjustment (Ceres; per-module shift and in-plane turn, gauge
  on mean shift/turn/scale and on the two tilt-like patterns when the tilt is refined; adopted only
  where it predicts held-out frames better). Applied for a foreign multi-panel detector (panel table,
  .geom panels); for a JUNGFRAU measured and reported only. Written to <prefix>_modules.h5 (NXmx
  NXdetector_module groups), read back with --module-geometry.
- A panel-table file's camera length is nominal: with a reference cell it is refined from the cell.
- Broker and writer output unchanged: masters keep the single NXdetector_module group, the start
  message carries no module list.

Tests: [CrystFELGeometry], JFJochXFELReader_*, JFJochXTCReader, [JFJochMultiReader], [CellPopulation],
SaturationCeiling_*, [DetectorModules], RefineGlobalGeometry_ModulePlacement, [reindex], Sublattice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
2026-10-20 23:30:48 +02:00