AGIPD: converter to one rigid panel per module, int16 photons, mask folded in; still battery reads the converted sets

tools/battery/still/agipd_to_int16.py converts an AGIPD 1M Cheetah CXI file into the only form rugnux
supports for AGIPD: each module's eight 64-row ASICs placed 66 rows apart (the 2-pixel ASIC gaps
inserted), one 526 x 128 panel per module; the per-frame mask and mask_shared (bad bits = the
geometry's mask_bad) and the geometry's bad regions folded into the data as -32768; values
round(value / ADU per photon) as int16 (clipped to +-32767; no pixel clipped in either set);
bitshuffle/LZ4, one chunk per frame at /entry_1/data_1/data. It writes the matching CrystFEL geometry
(16 panels, each the rigid fit of its ASICs, module coffset kept) and refuses a module whose ASICs are
not on one straight line at a whole-pixel pitch within 0.5 px (worst here: 0.16 px for CXIDB 83,
0.002 px for 152).

Reader: a stored value equal to the smallest value of its signed integer type (INT16_MIN, INT32_MIN)
is no measurement. The reader decodes to float and divides by the gain, so -32768 used to come out as a
count; the CrystFEL parser takes only module geometry, so a flag_lessthan in the .geom could not say it.

Still battery: euxfel83 / euxfel152 read conv/*_int16.h5 with conv/agipd_int16.geom and --gain 1.
Full sets, -N 16, --refine-geometry=off, --model (other jobs on the machine, load 7-23):

                      indexed  d_min  CC1/2  CC_HALF_SPLIT  R_split  R_FREE  main pass       wall
  CXIDB 83  original  90.9%   1.84   0.641  0.617          0.482    0.280   89.3 s (31/s)   105 s
            converted 94.1%   1.84   0.686  0.644          0.470    0.277    6.5 s (429/s)   20 s
  CXIDB 152 original  17.5%   5.37   0.521  0.479          0.590    0.308  131.7 s (29/s)   153 s
            converted 62.8%   2.92   0.876  0.891          0.354    0.284   13.2 s (287/s)   27 s

Files 4.7 -> 1.3 GB and 5.7 -> 1.1 GB. The 152 gain is the mask: the reader took only Cheetah's
generic bad bits, while the geometry's mask_bad (0xffff) also marks three broken modules (bit 32) and
its bad region a fourth; the converter run with the generic bits instead gives 17.5% / 3.41 A again.
With geometry refinement on (16 rigid modules): 83 94.2%, 1.75 A, CC1/2 0.665, R_FREE 0.284, 49 s;
152 65.1%, 2.94 A, CC1/2 0.843, R_FREE 0.278, 27 s - mixed, the manifest keeps it off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-21 09:21:33 +02:00
1 parent 43e4207aaa
commit e4eaeba874
6 files changed
+192 -10

No files matched your search

+4
View File
@@ -2,6 +2,7 @@
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#include <cstdint>
#include <filesystem>
#include <hdf5.h>
#include <string>
@@ -82,6 +83,7 @@ TEST_CASE("JFJochXFELReader_SACLA", "[portable]") {
frame[8 * 4] = -5; // second sensor (fs 0, ss 0): below zero
frame[8 * 4 + 1] = 33; // second sensor (fs 1, ss 0)
frame[9 * 4] = 44; // second sensor (fs 0, ss 1)
frame[2 * 4] = INT16_MIN; // first sensor (fs 0, ss 2): no measurement
WriteFrames(g, "data", H5T_NATIVE_INT16, frame.data(), {16, 4});
WriteScalar(g, "photon_energy_ev", 10000.0 + 10.0 * shot);
H5Gclose(g);
@@ -109,6 +111,8 @@ TEST_CASE("JFJochXFELReader_SACLA", "[portable]") {
CHECK(PixelAt(reader, 0, 21, 2) == -5);
CHECK(PixelAt(reader, 0, 21, 3) == 33);
CHECK(PixelAt(reader, 0, 20, 2) == 44);
// The smallest int16 is no measurement, not a count.
CHECK(PixelAt(reader, 0, 2, 4) == INT32_MIN);
// The file's mask holds the first sensor's (fs 1, ss 0), at column 3, row 2.
const auto &mask = reader.GetDataset()->pixel_mask->GetMask();
CHECK(mask[2 * x.GetXPixelsNum() + 3] != 0);