Choose images_per_file from the acquisition when it is not given

The value means three different things at once. It is the unit of writer
parallelism - whole files go round-robin to the writers,
(image_number / images_per_file) % socket.size() in ZMQStream2Pusher::SendImage
and TCPStreamPusher - it multiplies writer memory linearly, since every data-file
plugin reserves per file, and it decides whether a legacy master is readable at
all, because dxtbx follows only the first data file of one.

That last point is what makes a flat default wrong. Measured with DIALS on a
2500-image rotation sweep written as legacy: split into five files it reports
2500 images and then raises IndexError beyond image 499, so it half-works
silently; in one file all 2500 read. AutoPROC does not read VDS, so legacy has to
stay the default, which leaves the file count as the only lever.

So make it optional and resolve it from the acquisition. A rotation sweep of at
most 20000 images goes into one data file - rotation datasets are small enough,
and one writer keeps up with them. A grid scan splits on whole fast-axis rows, so
a file is a meaningful piece of the grid. Stills and serial keep 1000, where the
image count far exceeds it and the parallelism and the bounded writer memory are
what matter. An explicit value is always taken literally.

GetImagesPerFile is the single place this is resolved, and it must always return
a fixed non-zero number, because everything downstream - receiver, pusher, puller,
writer - requires one. That was already true of the old 0 = "one file" spelling;
0 is now gone from the API and omitting the field says the same thing better.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-22 21:57:12 +02:00
co-authored by Claude Opus 5
parent 259b43154e
commit 1f4f77fe42
15 changed files with 133 additions and 30 deletions
+56 -1
View File
@@ -1077,4 +1077,59 @@ TEST_CASE("DiffractionExperiment_PedestalRun","[DiffractionExperiment]") {
x.Mode(DetectorMode::PedestalG2);
REQUIRE(x.IsPedestalRun());
REQUIRE(!x.IsApplyPixelMask());
}
}
TEST_CASE("DiffractionExperiment_ImagesPerFile_Defaults", "[DiffractionExperiment]") {
// images_per_file is optional in the settings, because at setup time the acquisition it should
// suit is not necessarily known. GetImagesPerFile() is the single place that resolves it, and
// everything downstream (receiver, pusher, puller, writer) requires the answer to be a fixed
// non-zero number - so every branch below must produce one.
const GoniometerAxis rotation("omega", 0.0f, 0.1f, {-1, 0, 0}, {});
const GoniometerAxis stationary("omega", 0.0f, 0.0f, {-1, 0, 0}, {});
SECTION("taken literally when given") {
DiffractionExperiment x(DetJF4M());
x.ImagesPerTrigger(5000).NumTriggers(1).Goniometer(rotation).ImagesPerFile(250);
CHECK(x.GetImagesPerFile() == 250);
}
SECTION("stills default to 1000") {
DiffractionExperiment x(DetJF4M());
x.ImagesPerTrigger(50000).NumTriggers(1);
CHECK(x.GetImagesPerFile() == DEFAULT_IMAGES_PER_FILE);
}
SECTION("a rotation sweep goes into one file") {
DiffractionExperiment x(DetJF4M());
x.ImagesPerTrigger(3600).NumTriggers(1).Goniometer(rotation);
CHECK(x.GetImagesPerFile() == 3600);
}
SECTION("a long rotation sweep is split again") {
DiffractionExperiment x(DetJF4M());
x.ImagesPerTrigger(ROTATION_SINGLE_FILE_IMAGE_LIMIT + 1).NumTriggers(1).Goniometer(rotation);
CHECK(x.GetImagesPerFile() == DEFAULT_IMAGES_PER_FILE);
}
SECTION("a goniometer that does not turn is not a rotation sweep") {
DiffractionExperiment x(DetJF4M());
x.ImagesPerTrigger(3600).NumTriggers(1).Goniometer(stationary);
CHECK(x.GetImagesPerFile() == DEFAULT_IMAGES_PER_FILE);
}
SECTION("a grid scan splits on whole fast-axis rows") {
DiffractionExperiment x(DetJF4M());
x.ImagesPerTrigger(10000).NumTriggers(1)
.GridScan(GridScanSettings(30, 10.0f, 10.0f, false, false));
const int64_t images_per_file = x.GetImagesPerFile();
CHECK(images_per_file % 30 == 0);
CHECK(images_per_file >= DEFAULT_IMAGES_PER_FILE);
CHECK(images_per_file < DEFAULT_IMAGES_PER_FILE + 30);
}
SECTION("a single self-contained file holds the whole run") {
DiffractionExperiment x(DetJF4M());
x.ImagesPerTrigger(5000).NumTriggers(1)
.SetFileWriterFormat(FileWriterFormat::NXmxIntegrated).ImagesPerFile(250);
CHECK(x.GetImagesPerFile() == 5000);
}
}