e4eaeba87475e2722da23e7f88f313b1d31ffb92
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  leonarski_fandClaude Opus 5.5
|
e4eaeba874 |
AGIPD: converter to one rigid panel per module, int16 photons, mask folded in; still battery reads the converted sets
tools/battery/still/agipd_to_int16.py converts an AGIPD 1M Cheetah CXI file into the only form rugnux
supports for AGIPD: each module's eight 64-row ASICs placed 66 rows apart (the 2-pixel ASIC gaps
inserted), one 526 x 128 panel per module; the per-frame mask and mask_shared (bad bits = the
geometry's mask_bad) and the geometry's bad regions folded into the data as -32768; values
round(value / ADU per photon) as int16 (clipped to +-32767; no pixel clipped in either set);
bitshuffle/LZ4, one chunk per frame at /entry_1/data_1/data. It writes the matching CrystFEL geometry
(16 panels, each the rigid fit of its ASICs, module coffset kept) and refuses a module whose ASICs are
not on one straight line at a whole-pixel pitch within 0.5 px (worst here: 0.16 px for CXIDB 83,
0.002 px for 152).
Reader: a stored value equal to the smallest value of its signed integer type (INT16_MIN, INT32_MIN)
is no measurement. The reader decodes to float and divides by the gain, so -32768 used to come out as a
count; the CrystFEL parser takes only module geometry, so a flag_lessthan in the .geom could not say it.
Still battery: euxfel83 / euxfel152 read conv/*_int16.h5 with conv/agipd_int16.geom and --gain 1.
Full sets, -N 16, --refine-geometry=off, --model (other jobs on the machine, load 7-23):
indexed d_min CC1/2 CC_HALF_SPLIT R_split R_FREE main pass wall
CXIDB 83 original 90.9% 1.84 0.641 0.617 0.482 0.280 89.3 s (31/s) 105 s
converted 94.1% 1.84 0.686 0.644 0.470 0.277 6.5 s (429/s) 20 s
CXIDB 152 original 17.5% 5.37 0.521 0.479 0.590 0.308 131.7 s (29/s) 153 s
converted 62.8% 2.92 0.876 0.891 0.354 0.284 13.2 s (287/s) 27 s
Files 4.7 -> 1.3 GB and 5.7 -> 1.1 GB. The 152 gain is the mask: the reader took only Cheetah's
generic bad bits, while the geometry's mask_bad (0xffff) also marks three broken modules (bit 32) and
its bad region a fourth; the converter run with the generic bits instead gives 17.5% / 3.41 A again.
With geometry refinement on (16 rigid modules): 83 94.2%, 1.75 A, CC1/2 0.665, R_FREE 0.284, 49 s;
152 65.1%, 2.94 A, CC1/2 0.843, R_FREE 0.278, 27 s - mixed, the manifest keeps it off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
|
||
|
leonarski_fandClaude Opus 5.5
|
ebce906025 |
Serial XFEL stills inputs: readers from the files themselves, module geometry and its refinement
Combines the stills reader and module work (stills-readers, stills-modgeom, stills-euxfel) onto rc178 with the owner's decisions for this release: - Inputs are files, never a directory; several files of one format are one dataset (the multi-input CLI of the previous commit), so every reader takes an explicit list: JFJochXFELReader::ReadFiles, ReadCXI, ReadPanelTable and JFJochXTCReader::ReadFiles take std::vector<std::string>; ListFiles and xtc::CollectStreams are gone, CanRead* accept a regular file only. - One HDF5 frame reader (JFJochCrystFELReader -> JFJochXFELReader) parameterised by what the file holds: per-shot groups or a 3D stack, a /metadata panel table (posx/posy/angle, pixel size, nominal camera length, pixel mask: as SACLA writes), a Cheetah CXI single panel (square frame; a tiled slab is refused with a message). At most 8 files open; every file checked against the first. Signed 32-bit decode, per-frame 3D masks, panels at their own coffset (from the EuXFEL work). - Frames stored one chunk each are read the way the NXmx reader reads its images: only the chunk's place is asked of HDF5 under its lock (H5Dget_chunk_info_by_coord), the bytes are read through HDF5ImageSource::RawFile (positional, no lock) and decoded outside it - deflate(+shuffle) by zlib, bitshuffle/LZ4 (and zstd, plain LZ4) by CompressedImage's decoders. Bitshuffle/LZ4 used to go through HDF5's filter under the global lock, one frame at a time. - LCLS-I XTC (Rayonix): wavelength from the beam record, pixel size from the Rayonix model and binning, only --detector-distance needed; damaged/compressed frames skipped, a truncated last event dropped; std::ifstream, no POSIX calls: builds with MSVC. No DIALS .expt support at all. - CrystFEL .geom: module geometry only (panel ranges, fs/ss, corners, pixel size, coffset, data path) and optionally the camera length; everything else is ignored with a note in the log. - --mask-good / --mask-bad <file>:<dataset>: a mask array in another HDF5 file, in the frames' stored layout (what a .geom's mask_file used to give). - Gain measured on the background for panel-table and CXI input that states none (--gain overrides). - Detector modules from explicit geometry only (DetectorModules: the reader's panels, an NXmx file with one NXdetector_module per module, or --module-geometry); never from the image. Spots and predictions are mapped through the modules, the image is not re-assembled. - Module refinement in the stills bundle adjustment (Ceres; per-module shift and in-plane turn, gauge on mean shift/turn/scale and on the two tilt-like patterns when the tilt is refined; adopted only where it predicts held-out frames better). Applied for a foreign multi-panel detector (panel table, .geom panels); for a JUNGFRAU measured and reported only. Written to <prefix>_modules.h5 (NXmx NXdetector_module groups), read back with --module-geometry. - A panel-table file's camera length is nominal: with a reference cell it is refined from the cell. - Broker and writer output unchanged: masters keep the single NXdetector_module group, the start message carries no module list. Tests: [CrystFELGeometry], JFJochXFELReader_*, JFJochXTCReader, [JFJochMultiReader], [CellPopulation], SaturationCeiling_*, [DetectorModules], RefineGlobalGeometry_ModulePlacement, [reindex], Sublattice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu |