Choose images_per_file from the acquisition when it is not given
The value means three different things at once. It is the unit of writer parallelism - whole files go round-robin to the writers, (image_number / images_per_file) % socket.size() in ZMQStream2Pusher::SendImage and TCPStreamPusher - it multiplies writer memory linearly, since every data-file plugin reserves per file, and it decides whether a legacy master is readable at all, because dxtbx follows only the first data file of one. That last point is what makes a flat default wrong. Measured with DIALS on a 2500-image rotation sweep written as legacy: split into five files it reports 2500 images and then raises IndexError beyond image 499, so it half-works silently; in one file all 2500 read. AutoPROC does not read VDS, so legacy has to stay the default, which leaves the file count as the only lever. So make it optional and resolve it from the acquisition. A rotation sweep of at most 20000 images goes into one data file - rotation datasets are small enough, and one writer keeps up with them. A grid scan splits on whole fast-axis rows, so a file is a meaningful piece of the grid. Stills and serial keep 1000, where the image count far exceeds it and the parallelism and the bounded writer memory are what matter. An explicit value is always taken literally. GetImagesPerFile is the single place this is resolved, and it must always return a fixed non-zero number, because everything downstream - receiver, pusher, puller, writer - requires one. That was already true of the old 0 = "one file" spelling; 0 is now gone from the API and omitting the field says the same thing better. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -21,7 +21,6 @@ DatasetSettings::DatasetSettings() {
|
||||
ntrigger = 1;
|
||||
images_per_trigger = 1;
|
||||
compression = CompressionAlgorithm::BSHUF_LZ4;
|
||||
images_per_file = 1000;
|
||||
data_reduction_factor_serialmx = 1.0;
|
||||
write_nxmx_hdf5_master = true;
|
||||
spot_finding_enable = true;
|
||||
@@ -249,13 +248,14 @@ int64_t DatasetSettings::GetImageNumPerTrigger() const {
|
||||
return images_per_trigger;
|
||||
}
|
||||
|
||||
DatasetSettings &DatasetSettings::ImagesPerFile(int64_t input) {
|
||||
check_min("Images per file", input, 0);
|
||||
DatasetSettings &DatasetSettings::ImagesPerFile(const std::optional<int64_t> &input) {
|
||||
if (input.has_value())
|
||||
check_min("Images per file", input.value(), 1);
|
||||
images_per_file = input;
|
||||
return *this;
|
||||
}
|
||||
|
||||
int64_t DatasetSettings::GetImagesPerFile() const {
|
||||
std::optional<int64_t> DatasetSettings::GetImagesPerFile() const {
|
||||
return images_per_file;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user