The rc184-2 receiver architecture is kept; what rc184 added since rc183 is carried into it: - JFJochReceiverRugnuxRotation: the live host budget is nine tenths of the machine's total less the image buffer and the margin, at most the machine's budget (was the budget less both); the pinned host pool is released after the end message, in MeasurementThread, not in RugnuxThread. - StillsGPUAnalysis: the escalating pre-pass is bounded by PREPASS_MAX_IMAGES (4 x PREPASS_IMAGES) as well as by half the image buffer, and frames arriving during a run do not make the next one larger than that. The other live-stills changes of rc184 (pre-pass holding until the geometry can be fitted, geometry fixed when it ends, fixed q range, no running refinement) were already in rc184-2. - IndexAndRefine, RotationIndexer, rugnux: rc184's versions, with rc184-2's receiver-owned indexer. - Tests: FPGAIntegrationTest.cpp stays removed (generator memory check is in MockAcquisitionDeviceTest, pixel-sum parity in ImagePreprocessorPixelSumTest); the FPGA live-stills tests of JFJochReceiverProcessingTest are covered by JFJochReceiverRugnuxTest (pre-pass cancel, ZeroMQ writer, writer down). - Docs: rc184's text plus rc184-2's receiver table, broker diagram, stream and test notes; NumberTextField and CHANGELOG as in rc184. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK
9.3 KiB
Tests
The unit and integration tests are written with Catch2 and
collected into a single binary, tests/jfjoch_test. Build and run it with:
make -j$(nproc) jfjoch_test
cd tests
./jfjoch_test # everything
./jfjoch_test "<test name>" # one test case
./jfjoch_test "[tag]" # by tag
jfjoch_portable_test holds the cases tagged [portable]: the results that depend on the architecture,
the compiler or the standard library (bitshuffle/LZ4, HDF5 read-back, CBF parsing, CPU spot finding, FFTW
indexing, gemmi MTZ/CIF). It is built in every configuration, the viewer- and rugnux-only ones included
(make jfjoch_portable_test), and is what the macOS and Windows CI jobs run.
The tests of the receivers, the pullers, the state machine and the writer run on a mock of the
FPGA card (acquisition_device/MockAcquisitionDevice): it computes nothing, the test gives it what
the card would write (tests/MockCardOutput.h, calculated on the CPU), and it reproduces the card's
host contract - packets, module completion, summation groups, cancel and the end of a collection.
Whether the mock gives the host what the card gives is checked in MockAcquisitionDeviceTest.cpp,
tagged [HLSvsMock]: a scenario runs on the high-level-synthesis C model of the firmware
(HLSSimulatedDevice), its output is given to the mock, and everything the host sees has to be the
same. The card's own arithmetic (conversion, gain modes, summation, spot finder, integration, ROI) is
no longer tested by jfjoch_test: the firmware is frozen.
There are also benchmark and hardware routines, each printing its own usage:
jfjoch_hdf5_testto measure HDF5 dataset writing speed (single threaded). It doubles as the generator of the HDF5 files used by the external-software tests below.jfjoch_lite_perf_testto measure the CPU/GPU ("lite") analysis path - spot finding, indexing and optional file writing.jfjoch_fpga_testto test quality/performance of FPGA card(s) and software routines. With-Hit runs the high-level-synthesis C model on the CPU, so no FPGA device is needed.
Out-of-space handling is covered separately by jfjoch_hdf5_enospc_test, run under the enospc_shim
LD_PRELOAD module that makes writes fail with ENOSPC.
In addition, tests are executed to verify that datasets written by Jungfraujoch are readable by
other MX software (see Integration with MX data processing software) -
XDS through the Jungfraujoch, Durin and Neggia plugins, and DIALS xia2.ssx - for each of the
NXmx layouts. Input files for these programs are placed in the tests/xds, tests/xds_durin,
tests/xds_neggia and tests/crystfel folders. See .gitea/workflows/build_and_test.yml for the
exact commands; the CrystFEL fixtures are run by hand rather than in the pipeline.
Judging a change to the analysis itself
The harnesses below run rugnux over stored datasets and score the result. None is part of CI -
run them when a change plausibly moves merged results, not as a reflex. The public datasets the
pipeline is exercised on, and the DOI to cite for each, are listed in
External test data.
tools/battery/battery.py- the rotation battery, the one canonical way to judge a rotation change: public PDB depositions scored against the deposition, in-house standard crystals and no-crystal controls scored against XDS, and a local-only private arm. Its manifests, the run and compare protocol and the report are described intools/battery/README.md. Each set runs once, with one command per arm; Measuring Rugnux explains the protocol, the REFMAC check and the timing benchmark, and how to take timings that compare.tools/battery/still_battery.py- the still battery: public serial (XFEL stills) datasets run with their deposition's cell and space group, each merge refined against the deposited model and reported beside the deposition's R-free, with rugnux's wall time and the input's I/O floor. See "The still battery" intools/battery/README.md.tools/battery/still/rugnux_stills_ab.py- A/B comparison of argument sets on a fixed stills subset: what one change does to the merge, shell by shell. Takes its dataset list from outside the repository.tools/battery/rugnux_anomalous.py- the anomalous-peak-height arbiter, below.
The anomalous-peak-height arbiter
A change that touches partiality - a mosaicity estimator, a rocking-curve model, a background change, anything that alters how partial reflections are weighted - cannot be judged by the statistics we normally reach for:
| statistic | why it fails for this class of change |
|---|---|
ISa, R_meas, error-model b |
one measurement, not three; dominated by the low-resolution shells; not invariant to the uniform intensity rescale a partiality change produces |
| last-shell R_meas | moves with its denominator, i.e. the wrong way by construction |
rugnux --model R-free |
tracks its own zero-information floor, which moves ~22x more than R-free itself over the same sweep |
per-shell agreement with XDS_ASCII.HKL |
XDS never divides by partiality, so "divide less" moves us toward it mechanically; measured to put the optimum ~1.4x too low |
Anomalous difference density at known scatterer sites has none of these problems. It is read in units of the map's own sigma, so a uniform intensity rescale cancels exactly, and it is referenced to the structure rather than to another program's partiality model.
tools/battery/rugnux_anomalous.py measures it: shelxc + anode -a (CCP4) on each arm's merged reflections,
against a model that is placed once and then held fixed. It reports, per dataset, the mean site
height and the off-site noise floor, and, between arms, the paired per-site change.
# compare two arms (each a directory of <id>/<id>.hkl + .mtz)
tools/battery/rugnux_anomalous.py --config <table>.json base=<dir-A> test=<dir-B>
# a parameter scan: numeric labels turn the arms into a curve with a per-dataset optimum
tools/battery/rugnux_anomalous.py --config <table>.json \
0.85='<scan>/{name}/s0p85.hkl' 1.00='<scan>/{name}/s1.hkl' 1.20='<scan>/{name}/s1p2.hkl'
An arm is a Rugnux output directory or a path template containing {name}. --place does the
one-off model placement, --write-config-template prints the config skeleton, and ANODE results
are cached under the config's workdir (a full 9-dataset x 11-arm scan takes under a minute).
The gate. A dataset counts only if its reference arm shows top peak > 1.5x the highest
off-site peak and at least 3 sites over 5 sigma. A dataset that fails is reported as
EXCLUDED, never as a zero - the difference between two noise measurements is not a measurement.
Standing dataset set (2026-08): 8 datasets from 7 crystals - 114 site-measurements across the datasets, 108 distinct sulfur sites - all judged on native sulfur signal.
| crystals | space group | photon energy | sites each |
|---|---|---|---|
| 2 | P41212 | 12.4, 16.0 keV | 18 |
| 2 (lysozyme) | P43212 | 13.0, 5.0 keV | 27 |
| 3 (4 datasets - one crystal contributes two energies) | cubic, I-centred | 13.0, 6.0, 5.0, 5.0 keV | 6 |
Report n as crystals, not datasets: two energies of one crystal are not two independent votes,
and the tool prints both counts for that reason.
Traps this tool exists to encapsulate. Every one of them has already cost a working day:
- The phasing space group comes from the config, never from the merged file. I23 and
I213 have identical systematic absences (I-centring already forces the screw
condition), so no data can separate them, and phaser's automatic space-group test only tries
the enantiomorph - which for I23 is itself. Phasing an I-centred cubic case in the I23 that
both Rugnux and XDS report gives TFZ 7-11 where the other member gives 30-50, and drops the mean
site height by a factor 3-10 - enough to make four good datasets look signal-free. Thirteen
classes of chiral space group are indistinguishable this way;
--placetries every member of the class and reports each one's LLG/TFZ. - Place the model once, from a reference arm, and reuse it unchanged. Re-phasing per arm lets the model move and contaminates the comparison. Refining the placed model against the dataset's own amplitudes is allowed (it lifts the peaks another 4-10%) as long as the same refined model is then used for every arm.
- The gate and the measurement must use the same model. Gating on one model and scoring the curve with another silently changes which datasets are in the set.
- The off-site floor skips special positions. A peak on the cell origin is a ripple of the
calculated phases, not a sample of the background; leaving it in inflates the floor by several
sigma and can turn a passing dataset into a failing one. Such peaks are reported in their own
speccolumn rather than dropped silently.
Reading the result. Judge the paired per-site change, with its standard error, pooled over
crystals. A per-dataset optimum whose arm does not beat the reference on the paired test is
flagged not significant vs ref and must not be quoted as a preference; so must one sitting on the
edge of the scanned grid (grid edge) - extend the grid instead.