Jungfraujoch/docs/EXTERNAL_TEST_DATA_STILLS.md

External test data: serial crystallography (XFEL stills)

The serial datasets of the still battery (tools/battery/still_battery.py, manifest tools/battery/still_manifest.json). Like the rotation sets on External test data, they were collected by other people, on detectors and in file formats we do not produce ourselves, and published for exactly this kind of reuse. All of them come from the Coherent X-ray Imaging Data Bank (CXIDB).

None of these data were collected by us. If you use any of them, cite the dataset DOI below and the paper of the entry.

Where the values come from

• Dataset DOI: the CXIDB entry's own DOI, resolved against DataCite before it was written down.
• Space group, cell, resolution and R-free are the values deposited with the PDB entry, read from the RCSB data API. Every entry is a dark or ground state refined against observed data.
• Images used are the frames in the files we use, counted in the files themselves. Where that is only part of the entry, the table says so and gives what the entry or the deposition holds.

Datasets

Set	CXIDB, dataset DOI	Facility / beamline	Detector, format	PDB	Space group, dmin (Å)	Paper
sacla243	243, 10.11577/17003933	SACLA BL2	MPCCD, 8 panels of 1024 x 512, 50 µm; Cheetah-SACLA HDF5	31JT	P 2 21 21, 1.63	Bibi, De Vrieze et al., J. Am. Chem. Soc. (2026)
sacla237	237, 10.11577/2999572	SACLA BL2	MPCCD; Cheetah-SACLA HDF5	9S06	P 21 21 21, 2.05	Rios-Santacruz et al., Nature (2026)
sacla236	236, 10.11577/3014306	SACLA BL2	MPCCD; Cheetah-SACLA HDF5	9V3D	P 43 21 2, 1.68	Kang et al., J. Appl. Cryst. (2026)
pal86	86, 10.11577/1502065	PAL-XFEL NCI	Rayonix MX225-HS, 1440 x 1440, 156 µm; Cheetah CXI	6IG7	P 61 2 2, 1.80	Park et al., Sci. Rep. (2019)
pal90	90, 10.11577/1511590	PAL-XFEL NCI	Rayonix MX225-HS; Cheetah CXI	6IRK	I 2 2 2, 1.75	Lee et al., Sci. Rep. (2019)
lcls222	222, 10.11577/2204603	LCLS MFX	Rayonix MX340-HS, 1920 x 1920, 177 µm; LCLS-I XTC	8TDQ	P 65 2 2, 1.65	Nguyen et al., J. Am. Chem. Soc. (2023)
lcls242_s, lcls242_t	242, 10.11577/3374214	LCLS MFX	Rayonix MX340-HS; LCLS-I XTC	9O4S (small cell), 9O4T (large cell)	P 1 21 1, 1.95	Wolff et al., bioRxiv (2026)
euxfel83	83, 10.11577/1464102	European XFEL SPB/SFX	AGIPD 1M, 16 modules of 512 x 128, 200 µm; Cheetah CXI	6GTH	P 32 2 1, 1.69	Wiedorn et al., Nat. Commun. (2018)
euxfel152	152, 10.11577/1660970	European XFEL SPB/SFX	AGIPD 1M; Cheetah CXI	6U57	I 2 3, 2.80	Echelmeier et al., Nat. Commun. (2020)
alvra103, alvra103_int16	103, 10.11577/1650020	SwissFEL Alvra	JUNGFRAU 16M, 75 µm; SwissFEL HDF5, float32 keV (converted)	6S0L	C 2 2 21, 2.65	Nass et al., IUCrJ (2020)
alvra213, alvra213_int16	213, 10.11577/2282689	SwissFEL Alvra	JUNGFRAU 4M, 75 µm; SwissFEL HDF5, float32 keV (converted)	8BKH (dark state)	P 1 21 1, 1.35	Barends et al., Nature (2024)

Every set of the manifest is in the table and every set in the table is in the manifest, with the same CXIDB entry, PDB entry, space group and d_min; tools/battery/doc_check.py checks this. The beamline of pal90 is the CXIDB entry's (NCI); the PDB entry 6IRK records CSI.

How the battery runs them

Each set is one rugnux command, the way a serial user runs: the deposited cell and space group given (-C, -S), the wavelength where the files do not state it, --crystfel-geometry for the AGIPD and JUNGFRAU sets (the geometry file of the entry; the SACLA, PAL and LCLS files are placed without one), and the deposited model with --model, so that the report carries its R-free against the merge. The exact arguments are the args of each set in tools/battery/still_manifest.json; the stills section of the tutorial explains them.

The files we use

Paths are relative to the still data root of the site config (tools/battery/README.md). CXIDB serves https://cxidb.org/data/<entry>/...; most of it accepts HTTP range requests, so a broken download resumes with curl -C -.

Set	Files	Images used	Of the entry	Size
sacla243	sacla_243/raw_all/run*.h5: all 87 files of 243/Complete_dark/	52,151	all (the deposition's 52,151)	198 GB
sacla237	sacla_237/raw/: the 18 files run130637-0 to run130642-2 of 237/SACLA_raw/2022Nov/dark_ref/	10,792	subset: 18 of about 1,000 dark_ref files, about 1.8% of ~600,000 hits	48 GB
sacla236	sacla_236/raw/: the 18 files run157084-0 to run157089-2 of 236/Data/lyso01-apo-bigXtal/	16,083	all (the deposition's 16,083)	67 GB
pal86	pal_86/raw/, pal_86/raw_more/: CXIDB_86_themolysin_{2..25}_SFX_Nam.cxi (file 1 is truncated on the server and cannot be opened)	49,193	all but file 1 (24 of 25 files)	111 GB
pal90	pal_90/raw/: CXIDB_90-GI_SFX-filter-r000{1,2}-c00.cxi	11,947	all (both files)	25 GB
lcls222	lcls_222/xtc/, lcls_222/xtc_more/: mfx/mfxlu5017/xtc/e1277-r0165-{s00-c00,s00-c01,s01-c00,s01-c01,s02-c00}.xtc (run 165, the substrate-bound state of 8TDQ)	8,534	subset: run 165 of the runs 165-168 behind 8TDQ	72 GB
lcls242_*	lcls_242/xtc/, lcls_242/xtc_more/: xtc/mfxl1013721-r0256 to r0267 streams (the two crystal forms are in every run; each set keeps its own form)	~90,000	subset: runs 256-267 (44 of about 220 stream files; ~610 GB of ~3 TB)	~700 GB
euxfel83	euxfel_83/raw/: XFEL-r0243-c00.cxi, XFEL-r0278-c00.cxi, XFEL-r0280-c00.cxi (Cheetah hits, from the entry's cheetah-calib-mar-v1/hdf5/ tree on the NERSC archive); euxfel_83/ref/agipd_mar18_v10.geom from the entry	2,790	subset: 3 files; the deposited stream has 14,445 frames	4.7 GB
euxfel152	euxfel_152/raw/: runs r0108, 0110, 0112, 0114, 0115, 0116, 0118, 0123, 0125 extracted from XFEL2042-runs-cxi.tar (range requests on the tar members); euxfel_152/ref/kdo_embedded.geom, the geometry embedded in the header of the entry's KDO8PS.stream	3,801	subset: 9 runs; the deposited stream has 37,006 frames	5.7 GB
alvra103	swissfel_103/raw/: Raw_Files/2018-08-07/a2a/run{10..24}.JF06T32V01.h5; swissfel_103/ref/jf-16M.geom and the pixel mask swissfel_103/ref/run47.JF06T32V01.mask.h5 from the entry; the set reads swissfel_103/ref/jf-16M_mask.geom, a copy with mask_file pointed at that mask (see below)	~11,000	subset: 15 runs of one day; the deposition indexed 199,136 of 2,251,787 images	~750 GB
alvra213	swissfel_213/raw/march_dark_MbCO_II-March-06/data/dark/run_*.JF06T08V01.h5, from march/march_dark_MbCO_II-March-06.tar.gz (425 GB; download it to disk and extract, a streamed extraction does not survive a broken connection); swissfel_213/ref/mbco_march.geom from the JUNGFRAU geometry template of 2019-MARCH-SwissFEL-MbCO-supplement.tar.gz; the pixel mask of run 243 (data/pixel_mask/ in the same tarball) is used for all runs, through swissfel_213/ref/mbco_march_mask.geom, a copy with mask_file pointed at it (see below)	~38,000	all 24 dark runs of the March 2019 series	~430 GB

The masks of the SwissFEL geometry files. rugnux takes the pixel masks and bad regions a CrystFEL geometry file names. The SwissFEL files name their mask with an absolute path on the facility's disk, so each set reads a copy whose mask_file is relative to the copy's own directory (a relative mask file is looked for in the working directory first, as CrystFEL does, then beside the geometry file): swissfel_103/ref/jf-16M_mask.geom is jf-16M.geom with mask_file = run47.JF06T32V01.mask.h5, and swissfel_213/ref/mbco_march_mask.geom is mbco_march.geom with mask_file = ../raw/march_dark_MbCO_II-March-06/data/pixel_mask/run_000243.JF06T08V01.mask.h5. Only that line differs.

The _int16 sets. The SwissFEL files store converted JUNGFRAU data as uncompressed float32 keV, 68 MB (JF16M) or 17 MB (JF4M) a frame, which caps reading at about 74 or 295 frames/s at 5 GB/s. The _int16 sets are copies with the same layout in which /data/data holds whole photons (keV divided by the photon energy, rounded) as int16 with bitshuffle/LZ4, about 25 times smaller; the battery runs them with the same arguments as the float32 originals (rugnux measures the gain from the frames), and they merge to the same numbers. Make them with tools/battery/still/float32_to_int16.py <in.h5> <out.h5> <photon energy keV> (h5py and hdf5plugin), into swissfel_103/conv/ (4.57 keV) and swissfel_213/conv/ (12.037 keV, keeping the tarball's directory tree), named <input>_int16.h5. One MbCO dark run (run_000238) holds no frames and has no copy.