readoutspeed in rx master file and other master file inconsistencies (#1245)

readout speed added to json and h5 master files.
Also fixed master file inconsistencies

Sserver binaries
- update server binaries because readoutspeed needs to be sent to receiver with rx_hostname command

API
- added const to Detector class set/getburstmode

Python
- updated python bindings (burstmode const and roi arguments)

Cmd generation
- added pragma once in Caller.in.h as Caller is included in test files

m3: num channels due to #counters < 3
* workaround for m3 for messed up num channels (client always assumes all counters enabled and adds them to num channels), fix for hdf5

g2: exptime master file inconsistency
- exptime didnt match because of round of when setting burst mode (sets to a different clk divider)
- so updating actual time for all timers (exptime, period, subexptime etc, )  in Module class, get timer values from detector when setting it and then send to receiver to write in master file

ctb image size incorrect:
-  write actual size into master file and not the reserved size (digital reduces depending on dbit list and dbit offset)
- added a calculate ctb image size free function in generalData.h that is used there as well as for the tests.


master file inconsistencies
- refactored master attributes writing using templates
-    names changed to keep it consistent between json and hdf5 master file (Version, Pixels, Exposure Times, GateDelays, Acquisition Period, etc.)
-  datatypes changed to keep it simple where possible: imageSize, dynamicRange, tengiga, quad, readnrows, analog, analogsamples, digital, digitalsamples, dbitreorder, dbitoffset, transceivermask, transeiver, transceiversamples, countermask, gates =>int
- replacing "toString" with arrays, objects etc for eg for scan, rois, etc.
- json header always written (empty dataset or empty brackets)
- hdf5 needs const char* so have to convert strings to it, but taking care that strings exist prior to push_back
- master attributes (redundant string literals->error prone

tests for master file
- suppressed deprecated functions in rapidjson warnings just for the tests
- added slsREceiverSoftware/src to allow access to receiver_defs.h to test binary/hdf5 version
- refactored acquire tests by moving all the acquire tests from individual detector type files to a single one=test-Caller-acquire.cpp
- set some default settings (loadBasicSettings) for a basic acquire at load config part for the test_simulator python scripts. so minimum number of settings for detector to be set for any acquire tests.
- added tests to test master files for json and hdf5= test-Caller-master-attributes.cpp
- added option to add '-m' markers for tests using test_simulator python script

python/scripts/generate_functions.py

@@ -13,13 +13,17 @@ import subprocess
 import argparse
 import sys
 import time
+import ctypes.util, re  # to check libclang version 
 from pathlib import Path
 from parse import system_include_paths, clang_format_version
 
 REDC = "\033[91m"
 GREENC = "\033[92m"
+YELLOWC = "\033[93m"
 ENDC = "\033[0m"
 
+def yellow(msg):
+    return f"{YELLOWC}{msg}{ENDC}"
 
 def red(msg):
     return f"{REDC}{msg}{ENDC}"
@@ -29,6 +33,25 @@ def green(msg):
     return f"{GREENC}{msg}{ENDC}"
 
 
+def check_libclang_version(required="12"):
+    # Use already-loaded libclang, or let cindex resolve it
+    lib = ctypes.CDLL(cindex.Config.library_file or ctypes.util.find_library("clang"))
+
+    # Get version string
+    lib.clang_getClangVersion.restype = ctypes.c_void_p
+    version_ptr = lib.clang_getClangVersion()
+    version_str = ctypes.cast(version_ptr, ctypes.c_char_p).value.decode()
+
+    # Parse and check version
+    match = re.search(r"version\s+(\d+)", version_str)
+    if not match or match.group(1) != required:
+        msg = red(f"libclang version {match.group(1) if match else '?'} found, but version {required} required. Bye!")
+        print(msg)
+        sys.exit(1)
+    msg = green(f"Found libclang version {required}")
+    print(msg)
+
+
 def check_clang_format_version(required_version):
     if (ver := clang_format_version()) != required_version:
         msg = red(
@@ -120,6 +143,24 @@ def time_return_lambda(node, args):
 
 
 def visit(node):
+
+    loc = node.location
+    # skip if ndoe is outside project directory
+    if loc.file and not str(loc.file).startswith(str(cargs.build_path.parent)):
+        return
+    
+    '''
+    # to see which file was causing the error (not in Detector.h, so skipping others in the above code)
+    try:
+        kind = node.kind
+    except ValueError as e:
+        loc = node.location
+        file_name = loc.file.name if loc.file else "<unknown file>"
+        msg = yellow(f"\nWarning: skipping node with unknown CursorKind id {node._kind_id} at {file_name}:{loc.line}:{loc.column}")
+        print(msg)
+        return 
+    '''
+        
     if node.kind == cindex.CursorKind.CLASS_DECL:
         if node.displayname == "Detector":
             for child in node.get_children():
@@ -161,6 +202,7 @@ if __name__ == "__main__":
     )
     cargs = parser.parse_args()
 
+    check_libclang_version("12")
     check_clang_format_version(12)
     check_for_compile_commands_json(cargs.build_path)
 

python/src/detector.cpp

@@ -932,9 +932,9 @@ void init_det(py::module &m) {
                            Detector::setRxArping,
                        py::arg(), py::arg() = Positions{});
     CppDetectorApi.def("getRxROI",
-                       (std::vector<defs::ROI> (Detector::*)(int) const) &
+                       (std::vector<defs::ROI>(Detector::*)(int) const) &
                            Detector::getRxROI,
-                       py::arg());
+                       py::arg() = -1);
     CppDetectorApi.def("setRxROI",
                        (void (Detector::*)(const std::vector<defs::ROI> &)) &
                            Detector::setRxROI,
@@ -1349,8 +1349,9 @@ void init_det(py::module &m) {
         (void (Detector::*)(const int, const std::string &, sls::Positions)) &
             Detector::setVetoFile,
         py::arg(), py::arg(), py::arg() = Positions{});
-    CppDetectorApi.def("getBurstMode",
-                       (Result<defs::burstMode>(Detector::*)(sls::Positions)) &
+    CppDetectorApi.def(
+        "getBurstMode",
+        (Result<defs::burstMode>(Detector::*)(sls::Positions) const) &
             Detector::getBurstMode,
         py::arg() = Positions{});
     CppDetectorApi.def("setBurstMode",

slsDetectorServers/slsDetectorServer/src/slsDetectorServer_funcs.c

@@ -7255,7 +7255,8 @@ int get_receiver_parameters(int file_des) {
     // dynamic range
     ret = getDynamicRange(&i32);
     if (ret == FAIL) {
-        i32 = 0;
+        sprintf(mess, "Could not get dynamic range.\n");
+        return sendError(file_des);
     }
     n += sendData(file_des, &i32, sizeof(i32), INT32);
     if (n < 0)
@@ -7434,6 +7435,20 @@ int get_receiver_parameters(int file_des) {
     u32 = 0;
 #endif
     n += sendData(file_des, &u32, sizeof(u32), INT32);
+    if (n < 0)
+        return printSocketReadError();
+
+        // readout speed
+#if !defined(CHIPTESTBOARDD) && !defined(XILINX_CHIPTESTBOARDD)
+    ret = getReadoutSpeed(&i32);
+    if (ret == FAIL) {
+        sprintf(mess, "Could not get readout speed.\n");
+        return sendError(file_des);
+    }
+#else
+    i32 = 0;
+#endif
+    n += sendData(file_des, &i32, sizeof(i32), INT32);
     if (n < 0)
         return printSocketReadError();
 

slsDetectorSoftware/generator/Caller.in.h

@@ -1,5 +1,5 @@
 // This file is used as input to generate the caller class
-
+#pragma once
 #include "CmdParser.h"
 #include "HelpDacs.h"
 #include "sls/Detector.h"

slsDetectorSoftware/include/sls/Detector.h

@@ -1449,7 +1449,7 @@ class Detector {
                      Positions pos = {});
 
     /** [Gotthard2]  */
-    Result<defs::burstMode> getBurstMode(Positions pos = {});
+    Result<defs::burstMode> getBurstMode(Positions pos = {}) const;
 
     /** [Gotthard2]  BURST_INTERNAL (default), BURST_EXTERNAL,
      * CONTINUOUS_INTERNAL, CONTINUOUS_EXTERNAL. Also changes clkdiv 2, 3, 4 */

slsDetectorSoftware/src/Caller.h

@@ -1,5 +1,5 @@
 // This file is used as input to generate the caller class
-
+#pragma once
 #include "CmdParser.h"
 #include "HelpDacs.h"
 #include "sls/Detector.h"

slsDetectorSoftware/src/Detector.cpp

@@ -1883,7 +1883,7 @@ void Detector::setVetoFile(const int chipIndex, const std::string &fname,
     pimpl->Parallel(&Module::setVetoFile, pos, chipIndex, fname);
 }
 
-Result<defs::burstMode> Detector::getBurstMode(Positions pos) {
+Result<defs::burstMode> Detector::getBurstMode(Positions pos) const {
     return pimpl->Parallel(&Module::getBurstMode, pos);
 }
 

slsDetectorSoftware/src/Module.cpp

@@ -657,9 +657,16 @@ void Module::setExptime(int gateIndex, int64_t value) {
     int64_t args[]{static_cast<int64_t>(gateIndex), value};
     sendToDetector(F_SET_EXPTIME, args, nullptr);
     if (shm()->useReceiverFlag) {
+        // get exact value due to clk
+        if (shm()->detType == MYTHEN3 && gateIndex == -1) {
+            value = getExptime(0); // m3 does not support -1
+        } else {
+            value = getExptime(gateIndex); // others only support -1
+        }
+        args[1] = value;
         sendToReceiver(F_RECEIVER_SET_EXPTIME, args, nullptr);
     }
-    if (prevVal != value) {
+    if (shm()->detType == EIGER && prevVal != value) {
         updateRateCorrection();
     }
 }
@@ -671,6 +678,7 @@ int64_t Module::getPeriod() const {
 void Module::setPeriod(int64_t value) {
     sendToDetector(F_SET_PERIOD, value, nullptr);
     if (shm()->useReceiverFlag) {
+        value = getPeriod(); // get exact value due to clk
         sendToReceiver(F_RECEIVER_SET_PERIOD, value, nullptr);
     }
 }
@@ -749,6 +757,9 @@ slsDetectorDefs::speedLevel Module::getReadoutSpeed() const {
 
 void Module::setReadoutSpeed(speedLevel value) {
     sendToDetector(F_SET_READOUT_SPEED, value, nullptr);
+    if (shm()->useReceiverFlag) {
+        sendToReceiver(F_SET_RECEIVER_READOUT_SPEED, value, nullptr);
+    }
 }
 
 int Module::getClockDivider(int clkIndex) const {
@@ -1777,6 +1788,7 @@ void Module::setSubExptime(int64_t value) {
     }
     sendToDetector(F_SET_SUB_EXPTIME, value, nullptr);
     if (shm()->useReceiverFlag) {
+        value = getSubExptime(); // get exact value due to clk
         sendToReceiver(F_RECEIVER_SET_SUB_EXPTIME, value, nullptr);
     }
     if (prevVal != value) {
@@ -1791,6 +1803,7 @@ int64_t Module::getSubDeadTime() const {
 void Module::setSubDeadTime(int64_t value) {
     sendToDetector(F_SET_SUB_DEADTIME, value, nullptr);
     if (shm()->useReceiverFlag) {
+        value = getSubDeadTime(); // get exact value due to clk
         sendToReceiver(F_RECEIVER_SET_SUB_DEADTIME, value, nullptr);
     }
 }
@@ -2288,6 +2301,8 @@ void Module::setBurstMode(slsDetectorDefs::burstMode value) {
     sendToDetector(F_SET_BURST_MODE, value, nullptr);
     if (shm()->useReceiverFlag) {
         sendToReceiver(F_SET_RECEIVER_BURST_MODE, value, nullptr);
+        // changing burst mode may change exptime due to clk change
+        setExptime(-1, getExptime(-1)); // update exact exptime in receiver
     }
 }
 
@@ -2379,6 +2394,9 @@ void Module::setGateDelay(int gateIndex, int64_t value) {
     int64_t args[]{static_cast<int64_t>(gateIndex), value};
     sendToDetector(F_SET_GATE_DELAY, args, nullptr);
     if (shm()->useReceiverFlag) {
+        // get exact value due to clk
+        args[1] =
+            getGateDelay(gateIndex == -1 ? 0 : gateIndex); // m3 doesnt allow -1
         sendToReceiver(F_SET_RECEIVER_GATE_DELAY, args, nullptr);
     }
 }

slsDetectorSoftware/tests/CMakeLists.txt

@@ -17,6 +17,8 @@ target_sources(tests PRIVATE
     ${CMAKE_CURRENT_SOURCE_DIR}/Caller/test-Caller-xilinx-chiptestboard.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/Caller/test-Caller-moench.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/Caller/test-Caller-global.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/Caller/test-Caller-acquire.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/Caller/test-Caller-master-attributes.cpp
 
     
     ${CMAKE_CURRENT_SOURCE_DIR}/test-Result.cpp
@@ -34,10 +36,17 @@ if (SLS_USE_HDF5)
     )
 endif (SLS_USE_HDF5)
 
+target_compile_options(tests
+    PRIVATE
+    # Suppress warnings for deprecated functions in rapidjson
+        -Wno-deprecated-declarations
+) 
+
 target_include_directories(tests 
     PUBLIC 
         "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../src>"
         "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../include/sls>"
+        "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../../slsReceiverSoftware/src>"
     PRIVATE
     ${SLS_INTERNAL_RAPIDJSON_DIR}
 )

slsDetectorSoftware/tests/Caller/test-Caller-acquire.cpp

@@ -0,0 +1,279 @@
+// SPDX-License-Identifier: LGPL-3.0-or-other
+// Copyright (C) 2021 Contributors to the SLS Detector Package
+#include "Caller.h"
+#include "catch.hpp"
+#include "sls/Detector.h"
+#include "sls/sls_detector_defs.h"
+#include "sls/versionAPI.h"
+#include "test-Caller-global.h"
+#include "tests/globals.h"
+
+#include <filesystem>
+#include <sstream>
+
+namespace sls {
+
+using test::GET;
+using test::PUT;
+
+TEST_CASE("jungfrau_or_moench_acquire_check_file_size",
+          "[.cmdcall][.cmdacquire]") {
+
+    Detector det;
+    Caller caller(&det);
+    auto det_type =
+        det.getDetectorType().tsquash("Inconsistent detector types to test");
+
+    if (det_type == defs::JUNGFRAU || det_type == defs::MOENCH) {
+
+        auto num_udp_interfaces = det.getNumberofUDPInterfaces().tsquash(
+            "inconsistent number of udp interfaces");
+
+        int num_frames_to_acquire = 2;
+        create_files_for_acquire(det, caller, num_frames_to_acquire);
+
+        // check file size (assuming local pc)
+        {
+            detParameters par(det_type);
+            int bytes_per_pixel = det.getDynamicRange().squash() / 8;
+            // if 2 udp interfaces, data split into half
+            size_t expected_image_size = (par.nChanX * par.nChanY * par.nChipX *
+                                          par.nChipY * bytes_per_pixel) /
+                                         num_udp_interfaces;
+            testFileInfo test_file_info;
+            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
+                                          expected_image_size);
+        }
+    }
+}
+
+TEST_CASE("eiger_acquire_check_file_size", "[.cmdcall][.cmdacquire]") {
+    Detector det;
+    Caller caller(&det);
+    auto det_type =
+        det.getDetectorType().tsquash("Inconsistent detector types to test");
+
+    if (det_type == defs::EIGER) {
+
+        int dynamic_range = det.getDynamicRange().squash();
+        if (dynamic_range != 16) {
+            throw RuntimeError(
+                "Eiger detector must have dynamic range 16 to test");
+        }
+        int num_frames_to_acquire = 2;
+        create_files_for_acquire(det, caller, num_frames_to_acquire);
+
+        // check file size (assuming local pc)
+        {
+            detParameters par(det_type);
+            // data split into half due to 2 udp interfaces per half module
+            int num_chips = (par.nChipX / 2);
+            int bytes_per_pixel = (dynamic_range / 8);
+            size_t expected_image_size =
+                par.nChanX * par.nChanY * num_chips * bytes_per_pixel;
+            testFileInfo test_file_info;
+            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
+                                          expected_image_size);
+        }
+    }
+}
+
+TEST_CASE("mythen3_acquire_check_file_size", "[.cmdcall][.cmdacquire]") {
+    Detector det;
+    Caller caller(&det);
+    auto det_type =
+        det.getDetectorType().tsquash("Inconsistent detector types to test");
+
+    if (det_type == defs::MYTHEN3) {
+
+        int dynamic_range = det.getDynamicRange().squash();
+        int counter_mask = det.getCounterMask().squash();
+        if (dynamic_range != 16 && counter_mask != 0x3) {
+            throw RuntimeError("Mythen3 detector must have dynamic range 16 "
+                               "and counter mask 0x3 to test");
+        }
+        int num_counters = __builtin_popcount(counter_mask);
+        int num_frames_to_acquire = 2;
+        create_files_for_acquire(det, caller, num_frames_to_acquire);
+
+        // check file size (assuming local pc)
+        {
+            detParameters par(det_type);
+            int bytes_per_pixel = dynamic_range / 8;
+            int num_channels_per_counter = par.nChanX / 3;
+            size_t expected_image_size = num_channels_per_counter *
+                                         num_counters * par.nChipX *
+                                         bytes_per_pixel;
+            testFileInfo test_file_info;
+            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
+                                          expected_image_size);
+        }
+    }
+}
+
+TEST_CASE("gotthard2_acquire_check_file_size", "[.cmdcall][.cmdacquire]") {
+    Detector det;
+    Caller caller(&det);
+    auto det_type =
+        det.getDetectorType().tsquash("Inconsistent detector types to test");
+
+    if (det_type == defs::GOTTHARD2) {
+
+        int num_frames_to_acquire = 2;
+        create_files_for_acquire(det, caller, num_frames_to_acquire);
+
+        // check file size (assuming local pc)
+        {
+            detParameters par(det_type);
+            int bytes_per_pixel = det.getDynamicRange().squash() / 8;
+            size_t expected_image_size =
+                par.nChanX * par.nChipX * bytes_per_pixel;
+            testFileInfo test_file_info;
+            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
+                                          expected_image_size);
+        }
+    }
+}
+
+void test_ctb_file_size_with_acquire(Detector &det, Caller &caller,
+                                     int64_t num_frames,
+                                     const testCtbAcquireInfo &test_info,
+                                     bool isXilinxCtb) {
+    create_files_for_acquire(det, caller, num_frames, test_info);
+
+    // check file size (assuming local pc)
+    uint64_t expected_image_size =
+        calculate_ctb_image_size(test_info, isXilinxCtb).first;
+    testFileInfo test_file_info;
+    REQUIRE_NOTHROW(test_acquire_binary_file_size(test_file_info, num_frames,
+                                                  expected_image_size));
+}
+
+TEST_CASE("ctb_acquire_check_file_size", "[.cmdcall][.cmdacquire]") {
+    Detector det;
+    Caller caller(&det);
+    auto det_type =
+        det.getDetectorType().tsquash("Inconsistent detector types to test");
+
+    if (det_type == defs::CHIPTESTBOARD ||
+        det_type == defs::XILINX_CHIPTESTBOARD) {
+        bool isXilinxCtb = (det_type == defs::XILINX_CHIPTESTBOARD);
+        int num_frames_to_acquire = 2;
+        // all the test cases
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
+            test_ctb_config.dbit_offset = 16;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
+            test_ctb_config.dbit_reorder = true;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
+            test_ctb_config.dbit_offset = 16;
+            test_ctb_config.dbit_reorder = true;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
+            test_ctb_config.dbit_offset = 16;
+            test_ctb_config.dbit_list.clear();
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
+            test_ctb_config.dbit_offset = 16;
+            test_ctb_config.dbit_list.clear();
+            test_ctb_config.dbit_reorder = true;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
+            test_ctb_config.dbit_offset = 16;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
+            test_ctb_config.dbit_list.clear();
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
+            test_ctb_config.dbit_offset = 16;
+            test_ctb_config.dbit_list.clear();
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
+            test_ctb_config.dbit_offset = 16;
+            test_ctb_config.dbit_list.clear();
+            test_ctb_config.dbit_reorder = true;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::TRANSCEIVER_ONLY;
+            test_ctb_config.dbit_offset = 16;
+            test_ctb_config.dbit_list.clear();
+            test_ctb_config.dbit_reorder = true;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+        {
+            testCtbAcquireInfo test_ctb_config;
+            test_ctb_config.readout_mode = defs::ANALOG_ONLY;
+            test_ctb_config.dbit_offset = 16;
+            test_ctb_config.dbit_list.clear();
+            test_ctb_config.dbit_reorder = true;
+            REQUIRE_NOTHROW(test_ctb_file_size_with_acquire(
+                det, caller, num_frames_to_acquire, test_ctb_config,
+                isXilinxCtb));
+        }
+    }
+}
+
+} // namespace sls

slsDetectorSoftware/tests/Caller/test-Caller-chiptestboard.cpp

@@ -17,119 +17,6 @@ namespace sls {
 using test::GET;
 using test::PUT;
 
-TEST_CASE("ctb_acquire_check_file_size", "[.cmdcall]") {
-    Detector det;
-    Caller caller(&det);
-    auto det_type =
-        det.getDetectorType().tsquash("Inconsistent detector types to test");
-
-    if (det_type == defs::CHIPTESTBOARD ||
-        det_type == defs::XILINX_CHIPTESTBOARD) {
-        int num_frames_to_acquire = 2;
-        // all the test cases
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
-            test_ctb_config.dbit_offset = 16;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
-            test_ctb_config.dbit_reorder = true;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
-            test_ctb_config.dbit_offset = 16;
-            test_ctb_config.dbit_reorder = true;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
-            test_ctb_config.dbit_offset = 16;
-            test_ctb_config.dbit_list.clear();
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::ANALOG_AND_DIGITAL;
-            test_ctb_config.dbit_offset = 16;
-            test_ctb_config.dbit_list.clear();
-            test_ctb_config.dbit_reorder = true;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
-            test_ctb_config.dbit_offset = 16;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
-            test_ctb_config.dbit_list.clear();
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
-            test_ctb_config.dbit_offset = 16;
-            test_ctb_config.dbit_list.clear();
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::DIGITAL_AND_TRANSCEIVER;
-            test_ctb_config.dbit_offset = 16;
-            test_ctb_config.dbit_list.clear();
-            test_ctb_config.dbit_reorder = true;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::TRANSCEIVER_ONLY;
-            test_ctb_config.dbit_offset = 16;
-            test_ctb_config.dbit_list.clear();
-            test_ctb_config.dbit_reorder = true;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-        {
-            testCtbAcquireInfo test_ctb_config;
-            test_ctb_config.readout_mode = defs::ANALOG_ONLY;
-            test_ctb_config.dbit_offset = 16;
-            test_ctb_config.dbit_list.clear();
-            test_ctb_config.dbit_reorder = true;
-            REQUIRE_NOTHROW(test_ctb_acquire_with_receiver(
-                test_ctb_config, num_frames_to_acquire, det, caller));
-        }
-    }
-}
-
 /* dacs */
 
 TEST_CASE("dacname", "[.cmdcall]") {

slsDetectorSoftware/tests/Caller/test-Caller-eiger.cpp

@@ -17,70 +17,6 @@ namespace sls {
 using test::GET;
 using test::PUT;
 
-TEST_CASE("eiger_acquire_check_file_size", "[.cmdcall]") {
-    Detector det;
-    Caller caller(&det);
-    auto det_type =
-        det.getDetectorType().tsquash("Inconsistent detector types to test");
-
-    if (det_type == defs::EIGER) {
-
-        // save previous state
-        testFileInfo prev_file_info = get_file_state(det);
-        testCommonDetAcquireInfo prev_det_config_info =
-            get_common_acquire_config_state(det);
-
-        // save previous specific det type config
-        auto exptime = det.getExptime().tsquash("inconsistent exptime to test");
-        auto n_rows =
-            det.getReadNRows().tsquash("inconsistent number of rows to test");
-        auto dynamic_range =
-            det.getDynamicRange().tsquash("inconsistent dynamic range to test");
-        REQUIRE(false ==
-                det.getTenGiga().tsquash("inconsistent 10Giga to test"));
-
-        // defaults
-        int num_frames_to_acquire = 2;
-        testFileInfo test_file_info;
-        set_file_state(det, test_file_info);
-        testCommonDetAcquireInfo det_config;
-        det_config.num_frames_to_acquire = num_frames_to_acquire;
-        set_common_acquire_config_state(det, det_config);
-
-        // set default specific det type config
-        det.setExptime(std::chrono::microseconds{200});
-        det.setReadNRows(256);
-        det.setDynamicRange(16);
-
-        // acquire
-        test_acquire_with_receiver(caller, det);
-
-        // check frames caught
-        test_frames_caught(det, num_frames_to_acquire);
-
-        // check file size (assuming local pc)
-        {
-            detParameters par(det_type);
-            // data split into half due to 2 udp interfaces per half module
-            int num_chips = (par.nChipX / 2);
-            int bytes_per_pixel = (dynamic_range / 8);
-            size_t expected_image_size =
-                par.nChanX * par.nChanY * num_chips * bytes_per_pixel;
-            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
-                                          expected_image_size);
-        }
-
-        // restore previous state
-        set_file_state(det, prev_file_info);
-        set_common_acquire_config_state(det, prev_det_config_info);
-
-        // restore previous specific det type config
-        det.setExptime(exptime);
-        det.setReadNRows(n_rows);
-        det.setDynamicRange(dynamic_range);
-    }
-}
-
 /** temperature */
 
 TEST_CASE("temp_fpgaext", "[.cmdcall]") {

slsDetectorSoftware/tests/Caller/test-Caller-global.cpp

@@ -2,6 +2,7 @@
 // Copyright (C) 2021 Contributors to the SLS Detector Package
 #include "test-Caller-global.h"
 #include "Caller.h"
+#include "GeneralData.h"
 #include "catch.hpp"
 #include "sls/Detector.h"
 #include "sls/logger.h"
@@ -148,20 +149,52 @@ void test_acquire_with_receiver(Caller &caller, const Detector &det) {
     REQUIRE_NOTHROW(caller.call("rx_stop", {}, -1, PUT));
 }
 
-testCommonDetAcquireInfo get_common_acquire_config_state(const Detector &det) {
-    return testCommonDetAcquireInfo{
-        det.getTimingMode().tsquash("Inconsistent timing mode"),
-        det.getNumberOfFrames().tsquash("Inconsistent number of frames"),
-        det.getNumberOfTriggers().tsquash("Inconsistent number of triggers"),
-        det.getPeriod().tsquash("Inconsistent period")};
-}
+void create_files_for_acquire(Detector &det, Caller &caller, int64_t num_frames,
+                              std::optional<testCtbAcquireInfo> test_info) {
 
-void set_common_acquire_config_state(
-    Detector &det, const testCommonDetAcquireInfo &det_config_info) {
-    det.setTimingMode(det_config_info.timing_mode);
-    det.setNumberOfFrames(det_config_info.num_frames_to_acquire);
-    det.setNumberOfTriggers(det_config_info.num_triggers);
-    det.setPeriod(det_config_info.period);
+    // save previous state
+    testFileInfo prev_file_info = get_file_state(det);
+    auto prev_num_frames = det.getNumberOfFrames().tsquash(
+        "Inconsistent number of frames to acquire");
+    std::optional<testCtbAcquireInfo> prev_ctb_config_info{};
+    if (test_info) {
+        prev_ctb_config_info = get_ctb_config_state(det);
+    }
+
+    // set state for acquire
+    testFileInfo test_file_info;
+    set_file_state(det, test_file_info);
+    det.setNumberOfFrames(num_frames);
+    if (test_info) {
+        set_ctb_config_state(det, *test_info);
+    }
+
+    // acquire and get num frames caught
+    test_acquire_with_receiver(caller, det);
+    auto frames_caught = det.getFramesCaught().tsquash(
+        "Inconsistent number of frames caught")[0];
+    REQUIRE(frames_caught == num_frames);
+
+    // hdf5
+#ifdef HDF5C
+    test_file_info.file_format = defs::HDF5;
+    test_file_info.file_acq_index = 0;
+    set_file_state(det, test_file_info);
+
+    // acquire and get num frames caught
+    test_acquire_with_receiver(caller, det);
+    frames_caught = det.getFramesCaught().tsquash(
+        "Inconsistent number of frames caught")[0];
+    REQUIRE(frames_caught == num_frames);
+#endif
+
+    // restore previous state
+    // file
+    set_file_state(det, prev_file_info);
+    det.setNumberOfFrames(prev_num_frames);
+    if (test_info) {
+        set_ctb_config_state(det, *prev_ctb_config_info);
+    }
 }
 
 testCtbAcquireInfo get_ctb_config_state(const Detector &det) {
@@ -211,105 +244,30 @@ void set_ctb_config_state(Detector &det,
     det.setTransceiverEnableMask(ctb_config_info.transceiver_mask);
 }
 
-uint64_t calculate_ctb_image_size(const testCtbAcquireInfo &test_info) {
-    uint64_t num_analog_bytes = 0, num_digital_bytes = 0,
-             num_transceiver_bytes = 0;
-    if (test_info.readout_mode == defs::ANALOG_ONLY ||
-        test_info.readout_mode == defs::ANALOG_AND_DIGITAL) {
-        uint32_t adc_enable_mask =
-            (test_info.ten_giga ? test_info.adc_enable_10g
-                                : test_info.adc_enable_1g);
-        int num_analog_chans = __builtin_popcount(adc_enable_mask);
-        const int num_bytes_per_sample = 2;
-        num_analog_bytes =
-            num_analog_chans * num_bytes_per_sample * test_info.num_adc_samples;
-        LOG(logDEBUG1) << "[Analog Databytes: " << num_analog_bytes << ']';
-    }
+std::pair<uint64_t, int>
+calculate_ctb_image_size(const testCtbAcquireInfo &test_info,
+                         bool isXilinxCtb) {
 
-    // digital channels
-    if (test_info.readout_mode == defs::DIGITAL_ONLY ||
-        test_info.readout_mode == defs::ANALOG_AND_DIGITAL ||
-        test_info.readout_mode == defs::DIGITAL_AND_TRANSCEIVER) {
-        int num_digital_samples = test_info.num_dbit_samples;
-        if (test_info.dbit_offset > 0) {
-            uint64_t num_digital_bytes_reserved =
-                num_digital_samples * sizeof(uint64_t);
-            num_digital_bytes_reserved -= test_info.dbit_offset;
-            num_digital_samples = num_digital_bytes_reserved / sizeof(uint64_t);
-        }
-        int num_digital_chans = test_info.dbit_list.size();
-        if (num_digital_chans == 0) {
-            num_digital_chans = 64;
-        }
-        if (!test_info.dbit_reorder) {
-            uint32_t num_bits_per_sample = num_digital_chans;
-            if (num_bits_per_sample % 8 != 0) {
-                num_bits_per_sample += (8 - (num_bits_per_sample % 8));
-            }
-            num_digital_bytes = (num_bits_per_sample / 8) * num_digital_samples;
-        } else {
-            uint32_t num_bits_per_bit = num_digital_samples;
-            if (num_bits_per_bit % 8 != 0) {
-                num_bits_per_bit += (8 - (num_bits_per_bit % 8));
-            }
-            num_digital_bytes = num_digital_chans * (num_bits_per_bit / 8);
-        }
-        LOG(logDEBUG1) << "[Digital Databytes: " << num_digital_bytes << ']';
-    }
-    // transceiver channels
-    if (test_info.readout_mode == defs::TRANSCEIVER_ONLY ||
-        test_info.readout_mode == defs::DIGITAL_AND_TRANSCEIVER) {
-        int num_transceiver_chans =
-            __builtin_popcount(test_info.transceiver_mask);
-        const int num_bytes_per_channel = 8;
-        num_transceiver_bytes = num_transceiver_chans * num_bytes_per_channel *
-                                test_info.num_trans_samples;
-        LOG(logDEBUG1) << "[Transceiver Databytes: " << num_transceiver_bytes
-                       << ']';
+    sls::CtbImageInputs inputs{};
+    inputs.nAnalogSamples = test_info.num_adc_samples;
+    inputs.adcMask = test_info.adc_enable_10g;
+    if (!isXilinxCtb && !test_info.ten_giga) {
+        inputs.adcMask = test_info.adc_enable_1g;
     }
+    inputs.nTransceiverSamples = test_info.num_trans_samples;
+    inputs.transceiverMask = test_info.transceiver_mask;
+    inputs.nDigitalSamples = test_info.num_dbit_samples;
+    inputs.dbitOffset = test_info.dbit_offset;
+    inputs.dbitList = test_info.dbit_list;
+    inputs.dbitReorder = test_info.dbit_reorder;
 
+    auto out = computeCtbImageSize(inputs);
     uint64_t image_size =
-        num_analog_bytes + num_digital_bytes + num_transceiver_bytes;
+        out.nAnalogBytes + out.nDigitalBytes + out.nTransceiverBytes;
     LOG(logDEBUG1) << "Expected image size: " << image_size;
-    return image_size;
-}
-
-void test_ctb_acquire_with_receiver(const testCtbAcquireInfo &test_info,
-                                    int64_t num_frames_to_acquire,
-                                    Detector &det, Caller &caller) {
-
-    // save previous state
-    testFileInfo prev_file_info = get_file_state(det);
-    testCommonDetAcquireInfo prev_det_config_info =
-        // overwrite exptime if not using virtual ctb server
-        get_common_acquire_config_state(det);
-    testCtbAcquireInfo prev_ctb_config_info = get_ctb_config_state(det);
-
-    // defaults
-    testFileInfo test_file_info;
-    set_file_state(det, test_file_info);
-    testCommonDetAcquireInfo det_config;
-    det_config.num_frames_to_acquire = num_frames_to_acquire;
-    set_common_acquire_config_state(det, det_config);
-
-    // set ctb config
-    set_ctb_config_state(det, test_info);
-
-    // acquire
-    REQUIRE_NOTHROW(test_acquire_with_receiver(caller, det));
-
-    // check frames caught
-    REQUIRE_NOTHROW(test_frames_caught(det, num_frames_to_acquire));
-
-    // check file size (assuming local pc)
-    uint64_t expected_image_size = calculate_ctb_image_size(test_info);
-    REQUIRE_NOTHROW(test_acquire_binary_file_size(
-        test_file_info, num_frames_to_acquire, expected_image_size));
-
-    // restore previous state
-    set_file_state(det, prev_file_info);
-    set_common_acquire_config_state(det, prev_det_config_info);
-    set_ctb_config_state(det, prev_ctb_config_info);
+    int npixelx = out.nPixelsX;
+    LOG(logDEBUG1) << "Expected number of pixels in x: " << npixelx;
+    return std::make_pair(image_size, npixelx);
 }
 
 } // namespace sls

slsDetectorSoftware/tests/Caller/test-Caller-global.h

@@ -2,12 +2,13 @@
 // Copyright (C) 2021 Contributors to the SLS Detector Package
 #pragma once
 
-class Caller;
+#include "Caller.h"
 #include "sls/Detector.h"
 #include "sls/sls_detector_defs.h"
 
 #include <chrono>
 #include <filesystem>
+#include <optional>
 #include <thread>
 
 namespace sls {
@@ -18,13 +19,14 @@ struct testFileInfo {
     bool file_write{true};
     bool file_overwrite{true};
     slsDetectorDefs::fileFormat file_format{slsDetectorDefs::BINARY};
-};
-
-struct testCommonDetAcquireInfo {
-    slsDetectorDefs::timingMode timing_mode{slsDetectorDefs::AUTO_TIMING};
-    int64_t num_frames_to_acquire{2};
-    int64_t num_triggers{1};
-    std::chrono::nanoseconds period{std::chrono::milliseconds{2}};
+    std::string getMasterFileNamePrefix() const {
+        return file_path + "/" + file_prefix + "_master_" +
+               std::to_string(file_acq_index);
+    }
+    std::string getVirtualFileName() const {
+        return file_path + "/" + file_prefix + "_virtual_" +
+               std::to_string(file_acq_index) + ".h5";
+    }
 };
 
 struct testCtbAcquireInfo {
@@ -60,16 +62,14 @@ void test_frames_caught(const Detector &det, int num_frames_to_acquire);
 
 void test_acquire_with_receiver(Caller &caller, const Detector &det);
 
-testCommonDetAcquireInfo get_common_acquire_config_state(const Detector &det);
-void set_common_acquire_config_state(
-    Detector &det, const testCommonDetAcquireInfo &det_config_info);
+void create_files_for_acquire(
+    Detector &det, Caller &caller, int64_t num_frames = 1,
+    std::optional<testCtbAcquireInfo> test_info = std::nullopt);
 
 testCtbAcquireInfo get_ctb_config_state(const Detector &det);
 void set_ctb_config_state(Detector &det,
                           const testCtbAcquireInfo &ctb_config_info);
-uint64_t calculate_ctb_image_size(const testCtbAcquireInfo &test_info);
-void test_ctb_acquire_with_receiver(const testCtbAcquireInfo &test_info,
-                                    int64_t num_frames_to_acquire,
-                                    Detector &det, Caller &caller);
+std::pair<uint64_t, int>
+calculate_ctb_image_size(const testCtbAcquireInfo &test_info, bool isXilinxCtb);
 
 } // namespace sls

slsDetectorSoftware/tests/Caller/test-Caller-gotthard2.cpp

@@ -17,69 +17,6 @@ namespace sls {
 using test::GET;
 using test::PUT;
 
-TEST_CASE("gotthard2_acquire_check_file_size", "[.cmdcall]") {
-    Detector det;
-    Caller caller(&det);
-    auto det_type =
-        det.getDetectorType().tsquash("Inconsistent detector types to test");
-
-    if (det_type == defs::GOTTHARD2) {
-
-        // save previous state
-        testFileInfo prev_file_info = get_file_state(det);
-        testCommonDetAcquireInfo prev_det_config_info =
-            get_common_acquire_config_state(det);
-
-        // save previous specific det type config
-        auto exptime = det.getExptime().tsquash("inconsistent exptime to test");
-        auto burst_mode =
-            det.getBurstMode().tsquash("inconsistent burst mode to test");
-        auto number_of_bursts = det.getNumberOfBursts().tsquash(
-            "inconsistent number of bursts to test");
-        auto burst_period =
-            det.getBurstPeriod().tsquash("inconsistent burst period to test");
-
-        // defaults
-        int num_frames_to_acquire = 2;
-        testFileInfo test_file_info;
-        set_file_state(det, test_file_info);
-        testCommonDetAcquireInfo det_config;
-        det_config.num_frames_to_acquire = num_frames_to_acquire;
-        set_common_acquire_config_state(det, det_config);
-
-        // set default specific det type config
-        det.setExptime(std::chrono::microseconds{200});
-        det.setBurstMode(defs::CONTINUOUS_EXTERNAL);
-        det.setNumberOfBursts(1);
-        det.setBurstPeriod(std::chrono::milliseconds{0});
-
-        // acquire
-        test_acquire_with_receiver(caller, det);
-
-        // check frames caught
-        test_frames_caught(det, num_frames_to_acquire);
-
-        // check file size (assuming local pc)
-        {
-            detParameters par(det_type);
-            int bytes_per_pixel = det.getDynamicRange().squash() / 8;
-            size_t expected_image_size =
-                par.nChanX * par.nChipX * bytes_per_pixel;
-            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
-                                          expected_image_size);
-        }
-        // restore previous state
-        set_file_state(det, prev_file_info);
-        set_common_acquire_config_state(det, prev_det_config_info);
-
-        // restore previous specific det type config
-        det.setExptime(exptime);
-        det.setBurstMode(burst_mode);
-        det.setNumberOfBursts(number_of_bursts);
-        det.setBurstPeriod(burst_period);
-    }
-}
-
 // time specific measurements for gotthard2
 TEST_CASE("timegotthard2", "[.cmdcall]") {
     Detector det;

slsDetectorSoftware/tests/Caller/test-Caller-jungfrau.cpp

@@ -15,65 +15,6 @@ namespace sls {
 using test::GET;
 using test::PUT;
 
-TEST_CASE("jungfrau_acquire_check_file_size", "[.cmdcall]") {
-    Detector det;
-    Caller caller(&det);
-    auto det_type =
-        det.getDetectorType().tsquash("Inconsistent detector types to test");
-
-    if (det_type == defs::JUNGFRAU) {
-
-        // save previous state
-        testFileInfo prev_file_info = get_file_state(det);
-        testCommonDetAcquireInfo prev_det_config_info =
-            get_common_acquire_config_state(det);
-
-        // save previous specific det type config
-        auto exptime = det.getExptime().tsquash("inconsistent exptime to test");
-        auto num_udp_interfaces = det.getNumberofUDPInterfaces().tsquash(
-            "inconsistent number of udp interfaces");
-        auto n_rows =
-            det.getReadNRows().tsquash("inconsistent number of rows to test");
-
-        // defaults
-        int num_frames_to_acquire = 2;
-        testFileInfo test_file_info;
-        set_file_state(det, test_file_info);
-        testCommonDetAcquireInfo det_config;
-        det_config.num_frames_to_acquire = num_frames_to_acquire;
-        set_common_acquire_config_state(det, det_config);
-
-        // set default specific det type config
-        det.setExptime(std::chrono::microseconds{200});
-        det.setReadNRows(512);
-
-        // acquire
-        test_acquire_with_receiver(caller, det);
-
-        // check frames caught
-        test_frames_caught(det, num_frames_to_acquire);
-
-        // check file size (assuming local pc)
-        {
-            detParameters par(det_type);
-            int bytes_per_pixel = det.getDynamicRange().squash() / 8;
-            // if 2 udp interfaces, data split into half
-            size_t expected_image_size = (par.nChanX * par.nChanY * par.nChipX *
-                                          par.nChipY * bytes_per_pixel) /
-                                         num_udp_interfaces;
-            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
-                                          expected_image_size);
-        }
-        // restore previous state
-        set_file_state(det, prev_file_info);
-        set_common_acquire_config_state(det, prev_det_config_info);
-
-        // restore previous specific det type config
-        det.setExptime(exptime);
-        det.setReadNRows(n_rows);
-    }
-}
-
 /* dacs */
 
 TEST_CASE("Setting and reading back Jungfrau dacs", "[.cmdcall][.dacs]") {

slsDetectorSoftware/tests/Caller/test-Caller-moench.cpp

@@ -15,66 +15,6 @@ namespace sls {
 using test::GET;
 using test::PUT;
 
-TEST_CASE("moench_acquire_check_file_size", "[.cmdcall]") {
-    Detector det;
-    Caller caller(&det);
-    auto det_type =
-        det.getDetectorType().tsquash("Inconsistent detector types to test");
-
-    if (det_type == defs::MOENCH) {
-
-        // save previous state
-        testFileInfo prev_file_info = get_file_state(det);
-        testCommonDetAcquireInfo prev_det_config_info =
-            get_common_acquire_config_state(det);
-
-        // save previous specific det type config
-        auto exptime = det.getExptime().tsquash("inconsistent exptime to test");
-        auto num_udp_interfaces = det.getNumberofUDPInterfaces().tsquash(
-            "inconsistent number of udp interfaces");
-        auto n_rows =
-            det.getReadNRows().tsquash("inconsistent number of rows to test");
-
-        // defaults
-        int num_frames_to_acquire = 2;
-        testFileInfo test_file_info;
-        set_file_state(det, test_file_info);
-        testCommonDetAcquireInfo det_config;
-        det_config.num_frames_to_acquire = num_frames_to_acquire;
-        set_common_acquire_config_state(det, det_config);
-
-        // set default specific det type config
-        det.setExptime(std::chrono::microseconds{200});
-        det.setReadNRows(400);
-
-        // acquire
-        test_acquire_with_receiver(caller, det);
-
-        // check frames caught
-        test_frames_caught(det, num_frames_to_acquire);
-
-        // check file size (assuming local pc)
-        {
-            detParameters par(det_type);
-            int bytes_per_pixel = det.getDynamicRange().squash() / 8;
-            // if 2 udp interfaces, data split into half
-            size_t expected_image_size = (par.nChanX * par.nChanY * par.nChipX *
-                                          par.nChipY * bytes_per_pixel) /
-                                         num_udp_interfaces;
-            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
-                                          expected_image_size);
-        }
-
-        // restore previous state
-        set_file_state(det, prev_file_info);
-        set_common_acquire_config_state(det, prev_det_config_info);
-
-        // restore previous specific det type config
-        det.setExptime(exptime);
-        det.setReadNRows(n_rows);
-    }
-}
-
 /* dacs */
 
 TEST_CASE("Setting and reading back moench dacs", "[.cmdcall][.dacs]") {

slsDetectorSoftware/tests/Caller/test-Caller-mythen3.cpp

@@ -17,74 +17,6 @@ namespace sls {
 using test::GET;
 using test::PUT;
 
-TEST_CASE("mythen3_acquire_check_file_size", "[.cmdcall]") {
-    Detector det;
-    Caller caller(&det);
-    auto det_type =
-        det.getDetectorType().tsquash("Inconsistent detector types to test");
-
-    if (det_type == defs::MYTHEN3) {
-
-        // save previous state
-        testFileInfo prev_file_info = get_file_state(det);
-        testCommonDetAcquireInfo prev_det_config_info =
-            get_common_acquire_config_state(det);
-
-        // save previous specific det type config
-        auto exptime =
-            det.getExptimeForAllGates().tsquash("inconsistent exptime to test");
-        auto dynamic_range =
-            det.getDynamicRange().tsquash("inconsistent dynamic range to test");
-        uint32_t counter_mask =
-            det.getCounterMask().tsquash("inconsistent counter mask to test");
-
-        // defaults
-        int num_frames_to_acquire = 2;
-        testFileInfo test_file_info;
-        set_file_state(det, test_file_info);
-        testCommonDetAcquireInfo det_config;
-        det_config.num_frames_to_acquire = num_frames_to_acquire;
-        set_common_acquire_config_state(det, det_config);
-
-        // set default specific det type config
-        det.setExptime(-1, std::chrono::microseconds{200});
-        int test_dynamic_range = 16;
-        det.setDynamicRange(test_dynamic_range);
-        int test_counter_mask = 0x3;
-        int num_counters = __builtin_popcount(test_counter_mask);
-        det.setCounterMask(test_counter_mask);
-
-        // acquire
-        test_acquire_with_receiver(caller, det);
-
-        // check frames caught
-        test_frames_caught(det, num_frames_to_acquire);
-
-        // check file size (assuming local pc)
-        {
-            detParameters par(det_type);
-            int bytes_per_pixel = test_dynamic_range / 8;
-            int num_channels_per_counter = par.nChanX / 3;
-            size_t expected_image_size = num_channels_per_counter *
-                                         num_counters * par.nChipX *
-                                         bytes_per_pixel;
-            test_acquire_binary_file_size(test_file_info, num_frames_to_acquire,
-                                          expected_image_size);
-        }
-
-        // restore previous state
-        set_file_state(det, prev_file_info);
-        set_common_acquire_config_state(det, prev_det_config_info);
-
-        // restore previous specific det type config
-        for (int iGate = 0; iGate < 3; ++iGate) {
-            det.setExptime(iGate, exptime[iGate]);
-        }
-        det.setDynamicRange(dynamic_range);
-        det.setCounterMask(counter_mask);
-    }
-}
-
 /* dacs */
 
 TEST_CASE("Setting and reading back MYTHEN3 dacs", "[.cmdcall][.dacs]") {

slsDetectorSoftware/tests/Caller/test-Caller-rx.cpp

@@ -738,43 +738,19 @@ TEST_CASE("rx_roi", "[.cmdcall]") {
         // check master file creation
         // TODO: check roi in master file
         {
-            auto prev_write = det.getFileWrite().tsquash(
-                "inconsistent file write values in test");
-            auto prev_path = det.getFilePath().tsquash(
-                "inconsistent file path values in test");
-            auto prev_format = det.getFileFormat().tsquash(
-                "inconsistent file format values in test");
-            auto prev_index = det.getAcquisitionIndex().tsquash(
-                "inconsistent file index values in test");
-            auto prev_fname = det.getFileNamePrefix().tsquash(
-                "inconsistent file name prefix values in test");
-
-            det.setFileWrite(true);
-            det.setFilePath("/tmp");
-            det.setFileNamePrefix("test");
-
-            det.setAcquisitionIndex(0);
-            det.setFileFormat(defs::BINARY);
-            REQUIRE_NOTHROW(caller.call("acquire", {}, -1, PUT));
-            std::string file_path = "/tmp/test_master_0.json";
-            REQUIRE(std::filesystem::exists(file_path) == true);
+            create_files_for_acquire(det, caller);
+            testFileInfo file_info;
+            std::string master_file_prefix =
+                file_info.getMasterFileNamePrefix();
 
+            std::string fname = master_file_prefix + ".json";
+            REQUIRE(std::filesystem::exists(fname) == true);
 #ifdef HDF5C
-            det.setAcquisitionIndex(0);
-            det.setFileFormat(defs::HDF5);
-            REQUIRE_NOTHROW(caller.call("acquire", {}, -1, PUT));
-            file_path = "/tmp/test_master_0.h5";
-            REQUIRE(std::filesystem::exists(file_path) == true);
-            file_path = "/tmp/test_virtual_0.h5";
-            REQUIRE(std::filesystem::exists(file_path) == true);
+            fname = master_file_prefix + ".h5";
+            REQUIRE(std::filesystem::exists(fname) == true);
+            fname = file_info.getVirtualFileName();
+            REQUIRE(std::filesystem::exists(fname) == true);
 #endif
-
-            det.setFileWrite(prev_write);
-            if (!prev_path.empty())
-                det.setFilePath(prev_path);
-            det.setFileFormat(prev_format);
-            det.setAcquisitionIndex(prev_index);
-            det.setFileNamePrefix(prev_fname);
         }
 
         for (int i = 0; i != det.size(); ++i) {
@@ -801,9 +777,13 @@ TEST_CASE("rx_clearroi", "[.cmdcall]") {
             REQUIRE(oss.str() == "rx_clearroi successful\n");
         }
         for (int i = 0; i != det.size(); ++i) {
+            if (prev_val.size() == 1 && prev_val[0].completeRoi()) {
+                det.clearRxROI();
+            } else {
                 det.setRxROI(prev_val);
             }
         }
+    }
 }
 
 /* File */

slsReceiverSoftware/src/ClientInterface.cpp

@@ -221,7 +221,7 @@ int ClientInterface::functionTable(){
     flist[F_GET_RECEIVER_DBIT_REORDER]      =   &ClientInterface::get_dbit_reorder;
     flist[F_SET_RECEIVER_DBIT_REORDER]      =   &ClientInterface::set_dbit_reorder;
     flist[F_RECEIVER_GET_ROI_METADATA]      =   &ClientInterface::get_roi_metadata;
-
+    flist[F_SET_RECEIVER_READOUT_SPEED]     =   &ClientInterface::set_readout_speed;
 
 	for (int i = NUM_DET_FUNCTIONS + 1; i < NUM_REC_FUNCTIONS ; i++) {
 		LOG(logDEBUG1) << "function fnum: " << i << " (" <<
@@ -411,6 +411,8 @@ int ClientInterface::setup_receiver(Interface &socket) {
             impl()->setReadoutMode(arg.roMode);
             impl()->setTenGigaADCEnableMask(arg.adc10gMask);
             impl()->setTransceiverEnableMask(arg.transceiverMask);
+        } else {
+            impl()->setReadoutSpeed(arg.readoutSpeed);
         }
         if (detType == CHIPTESTBOARD) {
             impl()->setADCEnableMask(arg.adcMask);
@@ -1851,4 +1853,33 @@ int ClientInterface::get_roi_metadata(Interface &socket) {
     return OK;
 }
 
+int ClientInterface::set_readout_speed(Interface &socket) {
+    auto value = socket.Receive<int>();
+    verifyIdle(socket);
+    switch (detType) {
+    case GOTTHARD2:
+        if (value != G2_108MHZ && value != G2_144MHZ)
+            throw RuntimeError("Invalid readout speed for GOTTHARD2: " +
+                               std::to_string(value));
+        break;
+
+    case EIGER:
+    case JUNGFRAU:
+    case MYTHEN3:
+    case MOENCH:
+        if (value < 0 || value > QUARTER_SPEED) {
+            throw RuntimeError("Invalid readout speed: " +
+                               std::to_string(value));
+        }
+        break;
+
+    default:
+        functionNotImplemented();
+    }
+
+    LOG(logDEBUG1) << "Setting readout speed to " << value;
+    impl()->setReadoutSpeed(static_cast<speedLevel>(value));
+    return socket.Send(OK);
+}
+
 } // namespace sls

slsReceiverSoftware/src/ClientInterface.h

@@ -167,6 +167,7 @@ class ClientInterface : private virtual slsDetectorDefs {
     int get_dbit_reorder(ServerInterface &socket);
     int set_dbit_reorder(ServerInterface &socket);
     int get_roi_metadata(ServerInterface &socket);
+    int set_readout_speed(ServerInterface &socket);
 
     Implementation *impl() {
         if (receiver != nullptr) {

slsReceiverSoftware/src/GeneralData.h

@@ -18,6 +18,110 @@
 
 namespace sls {
 
+struct CtbImageInputs {
+    slsDetectorDefs::readoutMode mode{slsDetectorDefs::ANALOG_ONLY};
+
+    int nAnalogSamples{};
+    uint32_t adcMask{};
+    int nTransceiverSamples{};
+    uint32_t transceiverMask{};
+    int nDigitalSamples{};
+    int dbitOffset{};
+    bool dbitReorder{};
+    std::vector<int> dbitList{};
+};
+
+struct CtbImageOutputs {
+    int nAnalogBytes{};
+    int nDigitalBytes{};
+    int nDigitalBytesReserved{}; // including dbit offset and for 64 bits
+    int nTransceiverBytes{};
+    int nPixelsX{};
+};
+
+inline CtbImageOutputs computeCtbImageSize(const CtbImageInputs &in) {
+    CtbImageOutputs out{};
+
+    constexpr int num_bytes_per_analog_channel = 2;
+    constexpr int num_bytes_per_transceiver_channel = 8;
+    constexpr int max_digital_channels = 64;
+
+    // analog channels (normal, analog/digital readout)
+    if (in.mode == slsDetectorDefs::ANALOG_ONLY ||
+        in.mode == slsDetectorDefs::ANALOG_AND_DIGITAL) {
+        int nAnalogChans = __builtin_popcount(in.adcMask);
+
+        out.nPixelsX += nAnalogChans;
+        out.nAnalogBytes =
+            nAnalogChans * num_bytes_per_analog_channel * in.nAnalogSamples;
+        LOG(logDEBUG1) << " Number of Analog Channels:" << nAnalogChans
+                       << " Databytes: " << out.nAnalogBytes;
+    }
+
+    // digital channels
+    if (in.mode == slsDetectorDefs::DIGITAL_ONLY ||
+        in.mode == slsDetectorDefs::ANALOG_AND_DIGITAL ||
+        in.mode == slsDetectorDefs::DIGITAL_AND_TRANSCEIVER) {
+
+        int nSamples = in.nDigitalSamples;
+        {
+            // allocate enought for 64 bits and dbit offset for now
+            // TODO: to be replaced in the future with the actual reserved and
+            // used
+            int32_t num_bytes_per_bit =
+                (nSamples % 8 == 0) ? (nSamples / 8) : (nSamples / 8 + 1);
+            out.nDigitalBytesReserved =
+                max_digital_channels * num_bytes_per_bit;
+            LOG(logDEBUG1) << "Number of Digital Channels:"
+                           << max_digital_channels << " Databytes reserved: "
+                           << out.nDigitalBytesReserved;
+        }
+
+        // remove offset
+        if (in.dbitOffset > 0) {
+            int nBytesReserved = out.nDigitalBytesReserved - in.dbitOffset;
+            nSamples = nBytesReserved / sizeof(uint64_t);
+        }
+        // calculate channels
+        int nChans = in.dbitList.size();
+        if (nChans == 0) {
+            nChans = max_digital_channels;
+        }
+        out.nPixelsX += nChans;
+
+        // calculate actual bytes
+        if (!in.dbitReorder) {
+            uint32_t nBitsPerSample = nChans;
+            if (nBitsPerSample % 8 != 0) {
+                nBitsPerSample += (8 - (nBitsPerSample % 8));
+            }
+            out.nDigitalBytes = (nBitsPerSample / 8) * nSamples;
+        } else {
+            uint32_t nBitsPerSignal = nSamples;
+            if (nBitsPerSignal % 8 != 0) {
+                nBitsPerSignal += (8 - (nBitsPerSignal % 8));
+            }
+            out.nDigitalBytes = nChans * (nBitsPerSignal / 8);
+        }
+        LOG(logDEBUG1) << "Number of Actual Digital Channels:" << nChans
+                       << " Databytes: " << out.nDigitalBytes;
+    }
+
+    // transceiver channels
+    if (in.mode == slsDetectorDefs::TRANSCEIVER_ONLY ||
+        in.mode == slsDetectorDefs::DIGITAL_AND_TRANSCEIVER) {
+        int nTransceiverChans = __builtin_popcount(in.transceiverMask);
+
+        out.nPixelsX += nTransceiverChans;
+        out.nTransceiverBytes = nTransceiverChans *
+                                num_bytes_per_transceiver_channel *
+                                in.nTransceiverSamples;
+        LOG(logDEBUG1) << "Number of Transceiver Channels:" << nTransceiverChans
+                       << " Databytes: " << out.nTransceiverBytes;
+    }
+    return out;
+}
+
 class GeneralData {
 
   public:
@@ -62,7 +166,8 @@ class GeneralData {
     uint32_t transceiverMask{0};
     slsDetectorDefs::frameDiscardPolicy frameDiscardMode{
         slsDetectorDefs::NO_DISCARD};
-
+    /* actual image size after ctboffset and ctbreorder */
+    uint32_t actualImageSize{0};
     GeneralData(){};
     virtual ~GeneralData(){};
 
@@ -196,6 +301,7 @@ class EigerData : public GeneralData {
         dataSize = (tengigaEnable ? 4096 : 1024);
         packetSize = headerSizeinPacket + dataSize;
         imageSize = int(nPixelsX * nPixelsY * GetPixelDepth());
+        actualImageSize = imageSize;
         packetsPerFrame = imageSize / dataSize;
         fifoDepth = (dynamicRange == 32 ? 100 : 1000);
     };
@@ -226,6 +332,7 @@ class JungfrauData : public GeneralData {
         nPixelsX = (256 * 4);
         nPixelsY = (256 * 2) / numUDPInterfaces;
         imageSize = int(nPixelsX * nPixelsY * GetPixelDepth());
+        actualImageSize = imageSize;
         packetsPerFrame = imageSize / dataSize;
         udpSocketBufferSize = (1000 * 1024 * 1024) / numUDPInterfaces;
     };
@@ -257,6 +364,7 @@ class MoenchData : public GeneralData {
         nPixelsX = (400);
         nPixelsY = (400) / numUDPInterfaces;
         imageSize = int(nPixelsX * nPixelsY * GetPixelDepth());
+        actualImageSize = imageSize;
         packetsPerFrame = imageSize / dataSize;
         udpSocketBufferSize = (1000 * 1024 * 1024) / numUDPInterfaces;
     };
@@ -308,6 +416,7 @@ class Mythen3Data : public GeneralData {
         nPixelsX = (NCHAN * ncounters); // max 1280 channels x 3 counters
         LOG(logINFO) << "nPixelsX: " << nPixelsX;
         imageSize = nPixelsX * nPixelsY * GetPixelDepth();
+        actualImageSize = imageSize;
 
         // 10g
         if (tengigaEnable) {
@@ -374,6 +483,7 @@ class Gotthard2Data : public GeneralData {
     void UpdateImageSize() {
         packetSize = headerSizeinPacket + dataSize;
         imageSize = int(nPixelsX * nPixelsY * GetPixelDepth());
+        actualImageSize = imageSize;
         packetsPerFrame = imageSize / dataSize;
         vetoPacketSize = vetoHsize + vetoDataSize;
         vetoImageSize = vetoDataSize * packetsPerFrame;
@@ -384,8 +494,6 @@ class Gotthard2Data : public GeneralData {
 class ChipTestBoardData : public GeneralData {
   private:
     const int NCHAN_DIGITAL = 64;
-    const int NUM_BYTES_PER_ANALOG_CHANNEL = 2;
-    const int NUM_BYTES_PER_TRANSCEIVER_CHANNEL = 8;
     int nAnalogBytes = 0;
     int nDigitalBytes = 0;
     int nTransceiverBytes = 0;
@@ -394,7 +502,7 @@ class ChipTestBoardData : public GeneralData {
     /** Constructor */
     ChipTestBoardData() {
         detType = slsDetectorDefs::CHIPTESTBOARD;
-        nPixelsY = 1; // number of samples
+        nPixelsY = 1;
         headerSizeinPacket = sizeof(slsDetectorDefs::sls_detector_header);
         frameIndexMask = 0xFFFFFF; // 10g
         frameIndexOffset = 8;      // 10g
@@ -404,29 +512,29 @@ class ChipTestBoardData : public GeneralData {
         standardheader = true;
         ctbDbitReorder = true;
         UpdateImageSize();
-    };
+    }
 
   public:
-    int GetNumberOfAnalogDatabytes() { return nAnalogBytes; };
+    int GetNumberOfAnalogDatabytes() { return nAnalogBytes; }
 
-    int GetNumberOfDigitalDatabytes() { return nDigitalBytes; };
+    int GetNumberOfDigitalDatabytes() { return nDigitalBytes; }
 
-    int GetNumberOfTransceiverDatabytes() { return nTransceiverBytes; };
+    int GetNumberOfTransceiverDatabytes() { return nTransceiverBytes; }
 
     void SetNumberOfAnalogSamples(int n) {
         nAnalogSamples = n;
         UpdateImageSize();
-    };
+    }
 
     void SetNumberOfDigitalSamples(int n) {
         nDigitalSamples = n;
         UpdateImageSize();
-    };
+    }
 
     void SetNumberOfTransceiverSamples(int n) {
         nTransceiverSamples = n;
         UpdateImageSize();
-    };
+    }
 
     void SetctbDbitOffset(const int value) { ctbDbitOffset = value; }
 
@@ -437,83 +545,60 @@ class ChipTestBoardData : public GeneralData {
     void SetOneGigaAdcEnableMask(int n) {
         adcEnableMaskOneGiga = n;
         UpdateImageSize();
-    };
+    }
 
     void SetTenGigaAdcEnableMask(int n) {
         adcEnableMaskTenGiga = n;
         UpdateImageSize();
-    };
+    }
 
     void SetTransceiverEnableMask(int n) {
         transceiverMask = n;
         UpdateImageSize();
-    };
+    }
 
     void SetReadoutMode(slsDetectorDefs::readoutMode r) {
         readoutType = r;
         UpdateImageSize();
-    };
+    }
 
     void SetTenGigaEnable(bool tg) {
         tengigaEnable = tg;
         UpdateImageSize();
-    };
+    }
 
   private:
     void UpdateImageSize() {
-        nAnalogBytes = 0;
-        nDigitalBytes = 0;
-        nTransceiverBytes = 0;
-        int nAnalogChans = 0, nDigitalChans = 0, nTransceiverChans = 0;
-        uint64_t digital_bytes_reserved = 0;
-
-        // analog channels (normal, analog/digital readout)
-        if (readoutType == slsDetectorDefs::ANALOG_ONLY ||
-            readoutType == slsDetectorDefs::ANALOG_AND_DIGITAL) {
-            uint32_t adcEnableMask =
-                (tengigaEnable ? adcEnableMaskTenGiga : adcEnableMaskOneGiga);
-            nAnalogChans = __builtin_popcount(adcEnableMask);
-
-            nAnalogBytes =
-                nAnalogChans * NUM_BYTES_PER_ANALOG_CHANNEL * nAnalogSamples;
-            LOG(logDEBUG1) << " Number of Analog Channels:" << nAnalogChans
-                           << " Databytes: " << nAnalogBytes;
-        }
-        // digital channels
-        if (readoutType == slsDetectorDefs::DIGITAL_ONLY ||
-            readoutType == slsDetectorDefs::ANALOG_AND_DIGITAL ||
-            readoutType == slsDetectorDefs::DIGITAL_AND_TRANSCEIVER) {
-            nDigitalChans = NCHAN_DIGITAL;
-            // allocate enough memory to support reordering of digital bits
-            uint32_t num_bytes_per_bit = (nDigitalSamples % 8 == 0)
-                                             ? nDigitalSamples / 8
-                                             : nDigitalSamples / 8 + 1;
-            digital_bytes_reserved = 64 * num_bytes_per_bit;
+        // used in calculations so cant remove now - TODO: remove later
         nDigitalBytes = sizeof(uint64_t) * nDigitalSamples;
-            LOG(logDEBUG1) << "Number of Digital Channels:" << nDigitalChans
-                           << " Databytes: " << nDigitalBytes;
-        }
-        // transceiver channels
-        if (readoutType == slsDetectorDefs::TRANSCEIVER_ONLY ||
-            readoutType == slsDetectorDefs::DIGITAL_AND_TRANSCEIVER) {
-            nTransceiverChans = __builtin_popcount(transceiverMask);
-            ;
-            nTransceiverBytes = nTransceiverChans *
-                                NUM_BYTES_PER_TRANSCEIVER_CHANNEL *
-                                nTransceiverSamples;
-            LOG(logDEBUG1) << "Number of Transceiver Channels:"
-                           << nTransceiverChans
-                           << " Databytes: " << nTransceiverBytes;
-        }
-        nPixelsX = nAnalogChans + nDigitalChans + nTransceiverChans;
+
+        // calculate image size
+        CtbImageInputs inputs{};
+        inputs.nAnalogSamples = nAnalogSamples;
+        inputs.adcMask =
+            tengigaEnable ? adcEnableMaskTenGiga : adcEnableMaskOneGiga;
+        inputs.nTransceiverSamples = nTransceiverSamples;
+        inputs.transceiverMask = transceiverMask;
+        inputs.nDigitalSamples = nDigitalSamples;
+        inputs.dbitOffset = ctbDbitOffset;
+        inputs.dbitList = ctbDbitList;
+        inputs.dbitReorder = ctbDbitReorder;
+        auto out = computeCtbImageSize(inputs);
+        nPixelsX = out.nPixelsX;
+        imageSize = out.nAnalogBytes + out.nDigitalBytesReserved +
+                    out.nTransceiverBytes;
+        // to write to file: after ctb offset and reorder
+        actualImageSize =
+            out.nAnalogBytes + out.nDigitalBytes + out.nTransceiverBytes;
+        LOG(logDEBUG1) << "Actual image size: " << actualImageSize;
+
+        // calculate network parameters
         dataSize = tengigaEnable ? 8144 : UDP_PACKET_DATA_BYTES;
         packetSize = headerSizeinPacket + dataSize;
-        imageSize = nAnalogBytes + digital_bytes_reserved + nTransceiverBytes;
         packetsPerFrame = ceil((double)imageSize / (double)dataSize);
-
         LOG(logDEBUG1) << "Total Number of Channels:" << nPixelsX
                        << " Databytes: " << imageSize;
-    };
+    }
 };
 
 class XilinxChipTestBoardData : public GeneralData {
@@ -572,11 +657,6 @@ class XilinxChipTestBoardData : public GeneralData {
 
     void SetctbDbitReorder(const bool value) { ctbDbitReorder = value; }
 
-    void SetOneGigaAdcEnableMask(int n) {
-        adcEnableMaskOneGiga = n;
-        UpdateImageSize();
-    };
-
     void SetTenGigaAdcEnableMask(int n) {
         adcEnableMaskTenGiga = n;
         UpdateImageSize();
@@ -594,53 +674,30 @@ class XilinxChipTestBoardData : public GeneralData {
 
   private:
     void UpdateImageSize() {
-        nAnalogBytes = 0;
-        nDigitalBytes = 0;
-        nTransceiverBytes = 0;
-        int nAnalogChans = 0, nDigitalChans = 0, nTransceiverChans = 0;
-        uint64_t digital_bytes_reserved = 0;
-
-        // analog channels (normal, analog/digital readout)
-        if (readoutType == slsDetectorDefs::ANALOG_ONLY ||
-            readoutType == slsDetectorDefs::ANALOG_AND_DIGITAL) {
-            uint32_t adcEnableMask = adcEnableMaskTenGiga;
-            nAnalogChans = __builtin_popcount(adcEnableMask);
-
-            nAnalogBytes =
-                nAnalogChans * NUM_BYTES_PER_ANALOG_CHANNEL * nAnalogSamples;
-            LOG(logDEBUG1) << " Number of Analog Channels:" << nAnalogChans
-                           << " Databytes: " << nAnalogBytes;
-        }
-        // digital channels
-        if (readoutType == slsDetectorDefs::DIGITAL_ONLY ||
-            readoutType == slsDetectorDefs::ANALOG_AND_DIGITAL ||
-            readoutType == slsDetectorDefs::DIGITAL_AND_TRANSCEIVER) {
-            nDigitalChans = NCHAN_DIGITAL;
-            uint32_t num_bytes_per_bit = (nDigitalSamples % 8 == 0)
-                                             ? nDigitalSamples / 8
-                                             : nDigitalSamples / 8 + 1;
-            digital_bytes_reserved = 64 * num_bytes_per_bit;
+        // used in calculations so cant remove now - TODO: remove later
         nDigitalBytes = sizeof(uint64_t) * nDigitalSamples;
-            LOG(logDEBUG1) << "Number of Digital Channels:" << nDigitalChans
-                           << " Databytes: " << nDigitalBytes;
-        }
-        // transceiver channels
-        if (readoutType == slsDetectorDefs::TRANSCEIVER_ONLY ||
-            readoutType == slsDetectorDefs::DIGITAL_AND_TRANSCEIVER) {
-            nTransceiverChans = __builtin_popcount(transceiverMask);
-            ;
-            nTransceiverBytes = nTransceiverChans *
-                                NUM_BYTES_PER_TRANSCEIVER_CHANNEL *
-                                nTransceiverSamples;
-            LOG(logDEBUG1) << "Number of Transceiver Channels:"
-                           << nTransceiverChans
-                           << " Databytes: " << nTransceiverBytes;
-        }
-        nPixelsX = nAnalogChans + nDigitalChans + nTransceiverChans;
 
-        imageSize = nAnalogBytes + digital_bytes_reserved + nTransceiverBytes;
+        // calculate image size
+        CtbImageInputs inputs{};
+        inputs.nAnalogSamples = nAnalogSamples;
+        inputs.adcMask = adcEnableMaskTenGiga;
+        inputs.nTransceiverSamples = nTransceiverSamples;
+        inputs.transceiverMask = transceiverMask;
+        inputs.nDigitalSamples = nDigitalSamples;
+        inputs.dbitOffset = ctbDbitOffset;
+        inputs.dbitList = ctbDbitList;
+        inputs.dbitReorder = ctbDbitReorder;
+        auto out = computeCtbImageSize(inputs);
+        nPixelsX = out.nPixelsX;
+        imageSize = out.nAnalogBytes + out.nDigitalBytesReserved +
+                    out.nTransceiverBytes;
+        // to write to file: after ctb offset and reorder
+        actualImageSize =
+            out.nAnalogBytes + out.nDigitalBytes + out.nTransceiverBytes;
+        LOG(logDEBUG1) << "Actual image size: " << actualImageSize;
+
+        // calculate network parameters
         packetsPerFrame = ceil((double)imageSize / (double)dataSize);
-
         LOG(logDEBUG1) << "Total Number of Channels:" << nPixelsX
                        << " Databytes: " << imageSize;
     };

slsReceiverSoftware/src/Implementation.cpp

@@ -908,7 +908,7 @@ void Implementation::StartMasterWriter() {
             masterAttributes.detType = generalData->detType;
             masterAttributes.timingMode = timingMode;
             masterAttributes.geometry = numPorts;
-            masterAttributes.imageSize = generalData->imageSize;
+            masterAttributes.imageSize = generalData->actualImageSize;
             masterAttributes.nPixels =
                 xy(generalData->nPixelsX, generalData->nPixelsY);
             masterAttributes.maxFramesPerFile = generalData->framesPerFile;
@@ -943,7 +943,7 @@ void Implementation::StartMasterWriter() {
             masterAttributes.quad = quadEnable;
             masterAttributes.readNRows = readNRows;
             masterAttributes.ratecorr = rateCorrections;
-            masterAttributes.adcmask = generalData->tengigaEnable
+            masterAttributes.adcMask = generalData->tengigaEnable
                                            ? generalData->adcEnableMaskTenGiga
                                            : generalData->adcEnableMaskOneGiga;
             masterAttributes.analog =
@@ -959,12 +959,12 @@ void Implementation::StartMasterWriter() {
                     ? 1
                     : 0;
             masterAttributes.digitalSamples = generalData->nDigitalSamples;
-            masterAttributes.dbitoffset = generalData->ctbDbitOffset;
-            masterAttributes.dbitreorder = generalData->ctbDbitReorder;
-            masterAttributes.dbitlist = 0;
+            masterAttributes.dbitOffset = generalData->ctbDbitOffset;
+            masterAttributes.dbitReorder = generalData->ctbDbitReorder;
+            masterAttributes.dbitList = 0;
 
             for (auto &i : generalData->ctbDbitList) {
-                masterAttributes.dbitlist |= (static_cast<uint64_t>(1) << i);
+                masterAttributes.dbitList |= (static_cast<uint64_t>(1) << i);
             }
             masterAttributes.transceiverSamples =
                 generalData->nTransceiverSamples;
@@ -983,6 +983,7 @@ void Implementation::StartMasterWriter() {
             masterAttributes.gateDelayArray[2] = gateDelay3;
             masterAttributes.gates = numberOfGates;
             masterAttributes.additionalJsonHeader = additionalJsonHeader;
+            masterAttributes.readoutSpeed = readoutSpeed;
 
             // create master file
             masterFileName = dataProcessor[0]->CreateMasterFile(
@@ -1781,6 +1782,15 @@ void Implementation::setTransceiverEnableMask(uint32_t mask) {
     LOG(logINFO) << "Packets per Frame: " << (generalData->packetsPerFrame);
 }
 
+slsDetectorDefs::speedLevel Implementation::getReadoutSpeed() const {
+    return readoutSpeed;
+}
+
+void Implementation::setReadoutSpeed(const slsDetectorDefs::speedLevel i) {
+    readoutSpeed = i;
+    LOG(logINFO) << "Readout Speed: " << ToString(readoutSpeed);
+}
+
 /**************************************************
  *                                                *
  *    Callbacks                                   *

slsReceiverSoftware/src/Implementation.h

@@ -256,10 +256,12 @@ class Implementation : private virtual slsDetectorDefs {
     bool getDbitReorder() const;
     /* [Ctb] */
     void setDbitReorder(const bool reorder);
-
     uint32_t getTransceiverEnableMask() const;
     /* [Ctb] */
     void setTransceiverEnableMask(const uint32_t mask);
+    speedLevel getReadoutSpeed() const;
+    /* [Eiger][Jungfrau][Moench][Mythen3][Gotthard2]*/
+    void setReadoutSpeed(const speedLevel i);
 
     /**************************************************
      *                                                *
@@ -369,6 +371,7 @@ class Implementation : private virtual slsDetectorDefs {
     int thresholdEnergyeV{-1};
     std::array<int, 3> thresholdAllEnergyeV = {{-1, -1, -1}};
     std::vector<int64_t> rateCorrections;
+    speedLevel readoutSpeed{FULL_SPEED};
 
     // callbacks
     void (*startAcquisitionCallBack)(const startCallbackHeader,

slsReceiverSoftware/src/MasterAttributes.h

@@ -4,12 +4,14 @@
 
 #include "receiver_defs.h"
 #include "sls/ToString.h"
+#include "sls/TypeTraits.h"
 #include "sls/logger.h"
 #include "sls/sls_detector_defs.h"
 
 #include <chrono>
 #include <rapidjson/prettywriter.h>
 #include <rapidjson/stringbuffer.h>
+#include <string_view>
 
 #ifdef HDF5C
 #include "H5Cpp.h"
@@ -18,6 +20,7 @@
 namespace sls {
 
 using ns = std::chrono::nanoseconds;
+using writer = rapidjson::PrettyWriter<rapidjson::StringBuffer>;
 
 class MasterAttributes {
   public:
@@ -25,7 +28,7 @@ class MasterAttributes {
     slsDetectorDefs::detectorType detType{slsDetectorDefs::GENERIC};
     slsDetectorDefs::timingMode timingMode{slsDetectorDefs::AUTO_TIMING};
     slsDetectorDefs::xy geometry{};
-    uint32_t imageSize{0};
+    int imageSize{0};
     slsDetectorDefs::xy nPixels{};
     uint32_t maxFramesPerFile{0};
     slsDetectorDefs::frameDiscardPolicy frameDiscardMode{
@@ -37,123 +40,413 @@ class MasterAttributes {
     ns period{0};
     slsDetectorDefs::burstMode burstMode{slsDetectorDefs::BURST_INTERNAL};
     int numUDPInterfaces{0};
-    uint32_t dynamicRange{0};
-    uint32_t tenGiga{0};
+    int dynamicRange{0};
+    int tenGiga{0};
     int thresholdEnergyeV{0};
     std::array<int, 3> thresholdAllEnergyeV = {{0, 0, 0}};
     ns subExptime{0};
     ns subPeriod{0};
-    uint32_t quad{0};
-    uint32_t readNRows;
+    int quad{0};
+    int readNRows;
     std::vector<int64_t> ratecorr;
-    uint32_t adcmask{0};
-    uint32_t analog{0};
-    uint32_t analogSamples{0};
-    uint32_t digital{0};
-    uint32_t digitalSamples{0};
-    uint32_t dbitreorder{1};
-    uint32_t dbitoffset{0};
-    uint64_t dbitlist{0};
-    uint32_t transceiverMask{0};
-    uint32_t transceiver{0};
-    uint32_t transceiverSamples{0};
+    uint32_t adcMask{0};
+    int analog{0};
+    int analogSamples{0};
+    int digital{0};
+    int digitalSamples{0};
+    int dbitReorder{1};
+    int dbitOffset{0};
+    uint64_t dbitList{0};
+    int transceiverMask{0};
+    int transceiver{0};
+    int transceiverSamples{0};
     std::vector<slsDetectorDefs::ROI> rois{};
-    uint32_t counterMask{0};
+    int counterMask{0};
     std::array<ns, 3> exptimeArray{};
     std::array<ns, 3> gateDelayArray{};
-    uint32_t gates;
+    int gates;
     std::map<std::string, std::string> additionalJsonHeader;
     uint64_t framesInFile{0};
+    slsDetectorDefs::speedLevel readoutSpeed{slsDetectorDefs::FULL_SPEED};
+
+    inline static const std::string_view N_DETECTOR_TYPE = "Detector Type";
+    inline static const std::string_view N_TIMING_MODE = "Timing Mode";
+    inline static const std::string_view N_GEOMETRY = "Geometry";
+    inline static const std::string_view N_IMAGE_SIZE = "Image Size";
+    inline static const std::string_view N_PIXELS = "Pixels";
+    inline static const std::string_view N_MAX_FRAMES_PER_FILE =
+        "Max Frames Per File";
+    inline static const std::string_view N_FRAME_DISCARD_POLICY =
+        "Frame Discard Policy";
+    inline static const std::string_view N_FRAME_PADDING = "Frame Padding";
+    inline static const std::string_view N_TOTAL_FRAMES = "Total Frames";
+    inline static const std::string_view N_FRAMES_IN_FILE = "Frames in File";
+    inline static const std::string_view N_EXPOSURE_TIME = "Exposure Time";
+    inline static const std::string_view N_ACQUISITION_PERIOD =
+        "Acquisition Period";
+    inline static const std::string_view N_NUM_UDP_INTERFACES =
+        "Number of UDP Interfaces";
+    inline static const std::string_view N_NUMBER_OF_ROWS = "Number of Rows";
+    inline static const std::string_view N_READOUT_SPEED = "Readout Speed";
+    inline static const std::string_view N_DYNAMIC_RANGE = "Dynamic Range";
+    inline static const std::string_view N_TEN_GIGA = "Ten Giga";
+    inline static const std::string_view N_THRESHOLD_ENERGY =
+        "Threshold Energy";
+    inline static const std::string_view N_SUB_EXPOSURE_TIME =
+        "Sub Exposure Time";
+    inline static const std::string_view N_SUB_ACQUISITION_PERIOD =
+        "Sub Acquisition Period";
+    inline static const std::string_view N_QUAD = "Quad";
+    inline static const std::string_view N_RATE_CORRECTIONS =
+        "Rate Corrections";
+    inline static const std::string_view N_COUNTER_MASK = "Counter Mask";
+    inline static const std::string_view N_EXPOSURE_TIMES = "Exposure Times";
+    inline static const std::string_view N_GATE_DELAYS = "Gate Delays";
+    inline static const std::string_view N_GATES = "Gates";
+    inline static const std::string_view N_THRESHOLD_ENERGIES =
+        "Threshold Energies";
+    inline static const std::string_view N_BURST_MODE = "Burst Mode";
+    inline static const std::string_view N_ADC_MASK = "ADC Mask";
+    inline static const std::string_view N_ANALOG = "Analog Flag";
+    inline static const std::string_view N_ANALOG_SAMPLES = "Analog Samples";
+    inline static const std::string_view N_DIGITAL = "Digital Flag";
+    inline static const std::string_view N_DIGITAL_SAMPLES = "Digital Samples";
+    inline static const std::string_view N_DBIT_REORDER = "Dbit Reorder";
+    inline static const std::string_view N_DBIT_OFFSET = "Dbit Offset";
+    inline static const std::string_view N_DBIT_BITSET = "Dbit Bitset";
+    inline static const std::string_view N_TRANSCEIVER_MASK =
+        "Transceiver Mask";
+    inline static const std::string_view N_TRANSCEIVER = "Transceiver Flag";
+    inline static const std::string_view N_TRANSCEIVER_SAMPLES =
+        "Transceiver Samples";
+    inline static const std::string_view N_VERSION = "Version";
+    inline static const std::string_view N_TIMESTAMP = "Timestamp";
+    inline static const std::string_view N_RECEIVER_ROIS = "Receiver Rois";
+    inline static const std::string_view N_SCAN_PARAMETERS = "Scan Parameters";
+    inline static const std::string_view N_ADDITIONAL_JSON_HEADER =
+        "Additional JSON Header";
 
     MasterAttributes() = default;
     ~MasterAttributes() = default;
 
-    void
-    GetBinaryAttributes(rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetBinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteHDF5Attributes(H5::H5File *fd, H5::Group *group);
 #endif
 
-    void GetCommonBinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
-    void GetFinalBinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
-    void GetBinaryRois(rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetCommonBinaryAttributes(writer *w);
+    void GetFinalBinaryAttributes(writer *w);
+
 #ifdef HDF5C
     void WriteCommonHDF5Attributes(H5::H5File *fd, H5::Group *group);
     void WriteFinalHDF5Attributes(H5::Group *group);
-    void WriteHDF5ROIs(H5::Group *group);
-    void WriteHDF5Exptime(H5::Group *group);
-    void WriteHDF5Period(H5::Group *group);
-    void WriteHDF5DynamicRange(H5::Group *group);
-    void WriteHDF5TenGiga(H5::Group *group);
-    void WriteHDF5NumUDPInterfaces(H5::Group *group);
-    void WriteHDF5ReadNRows(H5::Group *group);
-    void WriteHDF5ThresholdEnergy(H5::Group *group);
-    void WriteHDF5ThresholdEnergies(H5::Group *group);
-    void WriteHDF5SubExpTime(H5::Group *group);
-    void WriteHDF5SubPeriod(H5::Group *group);
-    void WriteHDF5SubQuad(H5::Group *group);
-    void WriteHDF5RateCorrections(H5::Group *group);
-    void WriteHDF5CounterMask(H5::Group *group);
-    void WriteHDF5ExptimeArray(H5::Group *group);
-    void WriteHDF5GateDelayArray(H5::Group *group);
-    void WriteHDF5Gates(H5::Group *group);
-    void WriteHDF5BurstMode(H5::Group *group);
-    void WriteHDF5AdcMask(H5::Group *group);
-    void WriteHDF5AnalogFlag(H5::Group *group);
-    void WriteHDF5AnalogSamples(H5::Group *group);
-    void WriteHDF5DigitalFlag(H5::Group *group);
-    void WriteHDF5DigitalSamples(H5::Group *group);
-    void WriteHDF5DbitOffset(H5::Group *group);
-    void WriteHDF5DbitList(H5::Group *group);
-    void WriteHDF5DbitReorder(H5::Group *group);
-    void WriteHDF5TransceiverMask(H5::Group *group);
-    void WriteHDF5TransceiverFlag(H5::Group *group);
-    void WriteHDF5TransceiverSamples(H5::Group *group);
 #endif
 
-    void GetJungfrauBinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetJungfrauBinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteJungfrauHDF5Attributes(H5::Group *group);
 #endif
 
-    void GetEigerBinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetEigerBinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteEigerHDF5Attributes(H5::Group *group);
 #endif
 
-    void GetMythen3BinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetMythen3BinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteMythen3HDF5Attributes(H5::Group *group);
 #endif
 
-    void GetGotthard2BinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetGotthard2BinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteGotthard2HDF5Attributes(H5::Group *group);
 #endif
 
-    void GetMoenchBinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetMoenchBinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteMoenchHDF5Attributes(H5::Group *group);
 #endif
 
-    void
-    GetCtbBinaryAttributes(rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetCtbBinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteCtbHDF5Attributes(H5::Group *group);
 #endif
 
-    void GetXilinxCtbBinaryAttributes(
-        rapidjson::PrettyWriter<rapidjson::StringBuffer> *w);
+    void GetXilinxCtbBinaryAttributes(writer *w);
 #ifdef HDF5C
     void WriteXilinxCtbHDF5Attributes(H5::Group *group);
 #endif
+
+    void WriteBinaryDetectorType(writer *w);
+#ifdef HDF5C
+    void WriteHDF5DetectorType(H5::Group *group);
+#endif
+    void WriteBinaryTimingMode(writer *w);
+#ifdef HDF5C
+    void WriteHDF5TimingMode(H5::Group *group);
+#endif
+    void WriteBinaryGeometry(writer *w);
+#ifdef HDF5C
+    void WriteHDF5Geometry(H5::Group *group);
+#endif
+    void WriteBinaryImageSize(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ImageSize(H5::Group *group);
+#endif
+    void WriteBinaryPixels(writer *w);
+#ifdef HDF5C
+    void WriteHDF5Pixels(H5::Group *group);
+#endif
+    void WriteBinaryMaxFramesPerFile(writer *w);
+#ifdef HDF5C
+    void WriteHDF5MaxFramesPerFile(H5::Group *group);
+#endif
+    void WriteBinaryFrameDiscardPolicy(writer *w);
+#ifdef HDF5C
+    void WriteHDF5FrameDiscardPolicy(H5::Group *group);
+#endif
+    void WriteBinaryFramePadding(writer *w);
+#ifdef HDF5C
+    void WriteHDF5FramePadding(H5::Group *group);
+#endif
+    void WriteBinaryTotalFrames(writer *w);
+#ifdef HDF5C
+    void WriteHDF5TotalFrames(H5::Group *group);
+#endif
+    void WriteBinaryFramesInFile(writer *w);
+#ifdef HDF5C
+    void WriteHDF5FramesInFile(H5::Group *group);
+#endif
+
+    void WriteBinaryExposureTme(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ExposureTime(H5::Group *group);
+#endif
+    void WriteBinaryAcquisitionPeriod(writer *w);
+#ifdef HDF5C
+    void WriteHDF5AcquisitionPeriod(H5::Group *group);
+#endif
+    void WriteBinaryNumberOfUDPInterfaces(writer *w);
+#ifdef HDF5C
+    void WriteHDF5NumberOfUDPInterfaces(H5::Group *group);
+#endif
+    void WriteBinaryNumberOfRows(writer *w);
+#ifdef HDF5C
+    void WriteHDF5NumberOfRows(H5::Group *group);
+#endif
+    void WriteBinaryReadoutSpeed(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ReadoutSpeed(H5::Group *group);
+#endif
+    void WriteBinaryDynamicRange(writer *w);
+#ifdef HDF5C
+    void WriteHDF5DynamicRange(H5::Group *group);
+#endif
+    void WriteBinaryTenGiga(writer *w);
+#ifdef HDF5C
+    void WriteHDF5TenGiga(H5::Group *group);
+#endif
+    void WriteBinaryThresholdEnergy(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ThresholdEnergy(H5::Group *group);
+#endif
+    void WriteBinarySubExposureTime(writer *w);
+#ifdef HDF5C
+    void WriteHDF5SubExposureTime(H5::Group *group);
+#endif
+    void WriteBinarySubAcquisitionPeriod(writer *w);
+#ifdef HDF5C
+    void WriteHDF5SubAcquisitionPeriod(H5::Group *group);
+#endif
+    void WriteBinaryQuad(writer *w);
+#ifdef HDF5C
+    void WriteHDF5Quad(H5::Group *group);
+#endif
+    void WriteBinaryRateCorrections(writer *w);
+#ifdef HDF5C
+    void WriteHDF5RateCorrections(H5::Group *group);
+#endif
+    void WriteBinaryCounterMask(writer *w);
+#ifdef HDF5C
+    void WriteHDF5CounterMask(H5::Group *group);
+#endif
+    void WriteBinaryExptimeArray(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ExptimeArray(H5::Group *group);
+#endif
+    void WriteBinaryGateDelayArray(writer *w);
+#ifdef HDF5C
+    void WriteHDF5GateDelayArray(H5::Group *group);
+#endif
+    void WriteBinaryGates(writer *w);
+#ifdef HDF5C
+    void WriteHDF5Gates(H5::Group *group);
+#endif
+    void WriteBinaryThresholdAllEnergy(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ThresholdAllEnergy(H5::Group *group);
+#endif
+    void WriteBinaryBurstMode(writer *w);
+#ifdef HDF5C
+    void WriteHDF5BurstMode(H5::Group *group);
+#endif
+    void WriteBinaryAdcMask(writer *w);
+#ifdef HDF5C
+    void WriteHDF5AdcMask(H5::Group *group);
+#endif
+    void WriteBinaryAnalogFlag(writer *w);
+#ifdef HDF5C
+    void WriteHDF5AnalogFlag(H5::Group *group);
+#endif
+    void WriteBinaryAnalogSamples(writer *w);
+#ifdef HDF5C
+    void WriteHDF5AnalogSamples(H5::Group *group);
+#endif
+    void WriteBinaryDigitalFlag(writer *w);
+#ifdef HDF5C
+    void WriteHDF5DigitalFlag(H5::Group *group);
+#endif
+    void WriteBinaryDigitalSamples(writer *w);
+#ifdef HDF5C
+    void WriteHDF5DigitalSamples(H5::Group *group);
+#endif
+    void WriteBinaryDBitReorder(writer *w);
+#ifdef HDF5C
+    void WriteHDF5DBitReorder(H5::Group *group);
+#endif
+    void WriteBinaryDBitOffset(writer *w);
+#ifdef HDF5C
+    void WriteHDF5DBitOffset(H5::Group *group);
+#endif
+    void WriteBinaryDBitBitset(writer *w);
+#ifdef HDF5C
+    void WriteHDF5DBitBitset(H5::Group *group);
+#endif
+    void WriteBinaryTransceiverMask(writer *w);
+#ifdef HDF5C
+    void WriteHDF5TransceiverMask(H5::Group *group);
+#endif
+    void WriteBinaryTransceiverFlag(writer *w);
+#ifdef HDF5C
+    void WriteHDF5TransceiverFlag(H5::Group *group);
+#endif
+    void WriteBinaryTransceiverSamples(writer *w);
+#ifdef HDF5C
+    void WriteHDF5TransceiverSamples(H5::Group *group);
+#endif
+
+    /** writes according to type */
+    template <typename T> void WriteBinaryValue(writer *w, const T &value) {
+        if constexpr (std::is_same_v<T, int>) {
+            w->Int(value);
+        } else if constexpr (std::is_same_v<T, uint64_t>) {
+            w->Uint64(value);
+        } else if constexpr (std::is_same_v<T, int64_t>) {
+            w->Int64(value);
+        } else if constexpr (std::is_same_v<T, uint32_t>) {
+            w->Uint(value);
+        } else if constexpr (std::is_same_v<T, std::string>) {
+            w->String(value.c_str());
+        } else if constexpr (is_duration<T>::value) {
+            w->String(ToString(value).c_str());
+        } else {
+            throw RuntimeError("Unsupported type for Binary write: " +
+                               std::string(typeid(T).name()));
+        }
+    }
+
+    /** For non-arrays */
+    template <typename T>
+    std::enable_if_t<(!std::is_class_v<T> || std::is_same_v<T, std::string>),
+                     void>
+    WriteBinary(writer *w, const std::string &name, const T &value) {
+        w->Key(name.c_str());
+        WriteBinaryValue(w, value);
+    }
+
+    /** For arrays */
+    template <typename T>
+    std::enable_if_t<(std::is_class_v<T> && !std::is_same_v<T, std::string>),
+                     void>
+    WriteBinary(writer *w, const std::string &name, const T &value) {
+        w->Key(name.c_str());
+        w->StartArray();
+        for (const auto &v : value) {
+            WriteBinaryValue(w, v);
+        }
+        w->EndArray();
+    }
+
+#ifdef HDF5C
+    void WriteHDF5String(H5::Group *group, const std::string &name,
+                         const std::string &value);
+    void WriteHDF5StringArray(H5::Group *group, const std::string &name,
+                              const std::vector<std::string> &value);
+
+    /** get type */
+    template <typename T> H5::PredType const *GetHDF5Type() {
+        if constexpr (std::is_same_v<T, int>) {
+            return &H5::PredType::NATIVE_INT;
+        } else if constexpr (std::is_same_v<T, uint64_t>) {
+            return &H5::PredType::STD_U64LE;
+        } else if constexpr (std::is_same_v<T, int64_t>) {
+            return &H5::PredType::STD_I64LE;
+        } else if constexpr (std::is_same_v<T, uint32_t>) {
+            return &H5::PredType::STD_U32LE;
+        } else {
+            throw RuntimeError("Unsupported type for HDF5");
+        }
+    }
+
+    /** For non-arrays */
+    template <typename T>
+    typename std::enable_if<!std::is_class<T>::value, void>::type
+    WriteHDF5Int(H5::Group *group, const std::string &name, const T &value) {
+        H5::DataSpace dataspace(H5S_SCALAR);
+        auto h5type = GetHDF5Type<T>();
+        H5::DataSet dataset = group->createDataSet(name, *h5type, dataspace);
+        dataset.write(&value, *h5type);
+    }
+
+    /** For arrays */
+    template <typename T>
+    typename std::enable_if<std::is_class<T>::value, void>::type
+    WriteHDF5Int(H5::Group *group, const std::string &name, const T &value) {
+        using ElemT = typename T::value_type;
+        auto h5type = GetHDF5Type<ElemT>();
+        hsize_t dims[1] = {value.size()};
+        H5::DataSpace dataspace(1, dims);
+        H5::DataSet dataset = group->createDataSet(name, *h5type, dataspace);
+        dataset.write(value.data(), *h5type);
+    }
+
+#endif
+    void WriteBinaryXY(writer *w, const std::string &name, const defs::xy &xy);
+#ifdef HDF5C
+    void WriteHDF5XY(H5::Group *group, const std::string &name,
+                     const defs::xy &xy);
+#endif
+    void WriteBinaryVersion(writer *w);
+#ifdef HDF5C
+    void WriteHDF5Version(H5::H5File *fd);
+#endif
+    void WriteBinaryTimestamp(writer *w);
+#ifdef HDF5C
+    void WriteHDF5Timestamp(H5::Group *group);
+#endif
+    void WriteBinaryRois(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ROIs(H5::Group *group);
+#endif
+    void WriteBinaryScanParameters(writer *w);
+#ifdef HDF5C
+    void WriteHDF5ScanParameters(H5::Group *group);
+#endif
+    void WriteBinaryJsonHeader(writer *w);
+#ifdef HDF5C
+    void WriteHDF5JsonHeader(H5::Group *group);
+#endif
+    void WriteBinaryFrameHeaderFormat(writer *w);
 };
 
 } // namespace sls

slsReceiverSoftware/src/MasterFileUtility.cpp

@@ -140,13 +140,6 @@ std::string CreateMasterHDF5File(const std::string &filePath,
         fd = make_unique<H5::H5File>(fileName.c_str(), createFlags,
                                      H5::FileCreatPropList::DEFAULT, flist);
 
-        // attributes - version
-        double dValue = HDF5_WRITER_VERSION;
-        H5::DataSpace dataspace_attr = H5::DataSpace(H5S_SCALAR);
-        H5::Attribute attribute = fd->createAttribute(
-            "version", H5::PredType::NATIVE_DOUBLE, dataspace_attr);
-        attribute.write(H5::PredType::NATIVE_DOUBLE, &dValue);
-
         // Create a group in the file
         H5::Group group1(fd->createGroup("entry"));
         H5::Group group2(group1.createGroup("data"));
@@ -230,13 +223,6 @@ std::string CreateVirtualHDF5File(
             fd = make_unique<H5::H5File>(fileName.c_str(), H5F_ACC_TRUNC,
                                          H5::FileCreatPropList::DEFAULT, fapl);
 
-        // attributes - version
-        double dValue = HDF5_WRITER_VERSION;
-        H5::DataSpace dataspace_attr = H5::DataSpace(H5S_SCALAR);
-        H5::Attribute attribute = fd->createAttribute(
-            "version", H5::PredType::NATIVE_DOUBLE, dataspace_attr);
-        attribute.write(H5::PredType::NATIVE_DOUBLE, &dValue);
-
         for (size_t iRoi = 0; iRoi != multiRoi.size(); ++iRoi) {
 
             auto currentRoi = multiRoi[iRoi];

slsSupportLib/include/sls/sls_detector_defs.h

@@ -126,6 +126,9 @@ class slsDetectorDefs {
         int y{0};
         xy() = default;
         xy(int x, int y) : x(x), y(y){};
+        constexpr bool operator==(const xy &other) const {
+            return ((x == other.x) && (y == other.y));
+        }
     } __attribute__((packed));
 #endif
 
@@ -548,7 +551,7 @@ enum streamingInterface {
               stepSize(step) {
             dacSettleTime_ns = t.count();
         }
-        bool operator==(const scanParameters &other) const {
+        constexpr bool operator==(const scanParameters &other) const {
             return ((enable == other.enable) && (dacInd == other.dacInd) &&
                     (startOffset == other.startOffset) &&
                     (stopOffset == other.stopOffset) &&
@@ -663,6 +666,7 @@ enum streamingInterface {
         scanParameters scanParams{};
         int transceiverSamples{0};
         uint32_t transceiverMask{0};
+        speedLevel readoutSpeed{FULL_SPEED};
     } __attribute__((packed));
 #endif
 

slsSupportLib/include/sls/sls_detector_funcs.h

@@ -413,6 +413,7 @@ enum detFuncs {
     F_GET_RECEIVER_DBIT_REORDER,
     F_SET_RECEIVER_DBIT_REORDER,
     F_RECEIVER_GET_ROI_METADATA,
+    F_SET_RECEIVER_READOUT_SPEED,
 
     NUM_REC_FUNCTIONS
 };
@@ -822,6 +823,7 @@ const char* getFunctionNameFromEnum(enum detFuncs func) {
     case F_GET_RECEIVER_DBIT_REORDER:       return "F_GET_RECEIVER_DBIT_REORDER";
     case F_SET_RECEIVER_DBIT_REORDER:       return "F_SET_RECEIVER_DBIT_REORDER";
     case F_RECEIVER_GET_ROI_METADATA:       return "F_RECEIVER_GET_ROI_METADATA";
+    case F_SET_RECEIVER_READOUT_SPEED:      return "F_SET_RECEIVER_READOUT_SPEED";
 
     case NUM_REC_FUNCTIONS: 				return "NUM_REC_FUNCTIONS";
 	default:								return "Unknown Function";

slsSupportLib/include/sls/versionAPI.h

@@ -3,10 +3,10 @@
 /** API versions */
 #define APILIB       "0.0.0 0x250523"
 #define APIRECEIVER  "0.0.0 0x250523"
-#define APICTB       "0.0.0 0x250523"
-#define APIGOTTHARD2 "0.0.0 0x250523"
-#define APIMOENCH    "0.0.0 0x250523"
-#define APIEIGER     "0.0.0 0x250523"
-#define APIXILINXCTB "0.0.0 0x250523"
-#define APIJUNGFRAU  "0.0.0 0x250523"
-#define APIMYTHEN3   "0.0.0 0x250523"
+#define APICTB       "0.0.0 0x250714"
+#define APIGOTTHARD2 "0.0.0 0x250714"
+#define APIMOENCH    "0.0.0 0x250714"
+#define APIEIGER     "0.0.0 0x250714"
+#define APIXILINXCTB "0.0.0 0x250714"
+#define APIJUNGFRAU  "0.0.0 0x250714"
+#define APIMYTHEN3   "0.0.0 0x250714"

slsSupportLib/src/ToString.cpp

@@ -89,6 +89,7 @@ std::string ToString(const slsDetectorDefs::rxParameters &r) {
         << "scanParams:" << ToString(r.scanParams) << std::endl
         << "transceiverSamples:" << r.transceiverSamples << std::endl
         << "transceiverMask:" << r.transceiverMask << std::endl
+        << "readoutSpeed:" << ToString(r.readoutSpeed) << std::endl
         << ']';
     return oss.str();
 }

tests/scripts/test_frame_synchronizer.py

@@ -23,6 +23,7 @@ from utils_for_test import (
     checkLogForErrors,
     startDetectorVirtualServer,
     loadConfig,
+    loadBasicSettings,
     ParseArguments
 )
 
@@ -113,6 +114,7 @@ def startTestsForAll(args, fp):
             startFrameSynchronizerPullSocket(server, fp)
             startFrameSynchronizer(args.num_mods, fp)
             d = loadConfig(name=server, rx_hostname=args.rx_hostname, settingsdir=args.settingspath, fp=fp, num_mods=args.num_mods, num_frames=args.num_frames)
+            loadBasicSettings(name=server, d=d, fp=fp)
             acquire(fp, d)
             testFramesCaught(server, d, args.num_frames)
             testZmqHeadetTypeCount(server, d, args.num_mods, args.num_frames, fp)

tests/scripts/test_roi.py

@@ -7,8 +7,9 @@ This file is used to start up simulators, receivers and test roi for every detec
 import sys, time
 import traceback
 
-from slsdet import Detector
+from slsdet import Detector, burstMode
 from slsdet.defines import DEFAULT_TCP_RX_PORTNO, DEFAULT_UDP_DST_PORTNO
+from datetime import timedelta
 
 
 from utils_for_test import (
@@ -19,6 +20,7 @@ from utils_for_test import (
     startProcessInBackground,
     startDetectorVirtualServer,
     connectToVirtualServers,
+    loadBasicSettings,
     runProcessWithLogFile
 )
 
@@ -88,6 +90,7 @@ def startTestsForAll(fp):
                 startDetectorVirtualServer(server, nmods, fp)
                 startReceiver(nmods, fp)
                 d = loadConfigForRoi(name=server, fp=fp, num_mods=nmods, num_interfaces=ninterfaces)
+                loadBasicSettings(name=server, d=d, fp=fp)
 
                 fname = ROI_TEST_FNAME + server + '.txt'
                 cmd = ['tests', 'rx_roi', '--abort', '-s']

tests/scripts/test_simulators.py

@@ -21,6 +21,7 @@ from utils_for_test import (
     runProcessWithLogFile,
     startDetectorVirtualServer,
     loadConfig,
+    loadBasicSettings,
     ParseArguments
 )
 
@@ -56,14 +57,15 @@ def startCmdTestsForAll(args, fp):
         try:
             num_mods = 2 if server == 'eiger' else 1
             fname = CMD_TEST_LOG_PREFIX_FNAME + server + '.txt'
-            cmd = ['tests', '--abort', '[.cmdcall]', '-s']
+            cmd = ['tests', '--abort', args.markers, '-s']
 
             Log(LogLevel.INFOBLUE, f'Starting Cmd Tests for {server}')
             cleanup(fp)
             startDetectorVirtualServer(name=server, num_mods=num_mods, fp=fp)
             startReceiver(num_mods, fp)
-            loadConfig(name=server, rx_hostname=args.rx_hostname, settingsdir=args.settingspath, fp=fp, num_mods=num_mods)
-            runProcessWithLogFile('Cmd Tests for ' + server, cmd, fp, fname)
+            d = loadConfig(name=server, rx_hostname=args.rx_hostname, settingsdir=args.settingspath, fp=fp, num_mods=num_mods)
+            loadBasicSettings(name=server, d=d, fp=fp)
+            runProcessWithLogFile('Cmd Tests (' + args.markers + ') for ' + server, cmd, fp, fname)
         except Exception as e:
             raise RuntimeException(f'Cmd Tests failed for {server}.') from e
 
@@ -71,7 +73,7 @@ def startCmdTestsForAll(args, fp):
 
 
 if __name__ == '__main__':
-    args = ParseArguments('Automated tests with the virtual detector servers')
+    args = ParseArguments('Automated tests with the virtual detector servers', 1, 1)
     if args.num_mods > 1:
         raise RuntimeException(f'Cannot support multiple modules at the moment (except Eiger).')
 

tests/scripts/utils_for_test.py

@@ -7,8 +7,9 @@ This file is used for common utils used for integration tests between simulators
 import sys, subprocess, time, argparse
 from enum import Enum
 from colorama import Fore, Style, init
+from datetime import timedelta
 
-from slsdet import Detector, detectorSettings
+from slsdet import Detector, detectorSettings, burstMode
 from slsdet.defines import DEFAULT_TCP_RX_PORTNO, DEFAULT_UDP_DST_PORTNO
 SERVER_START_PORTNO=1900
 
@@ -156,7 +157,7 @@ def startDetectorVirtualServer(name :str, num_mods, fp):
     for i in range(num_mods):
         port_no = SERVER_START_PORTNO + (i * 2)
         cmd = [name + 'DetectorServer_virtual', '-p', str(port_no)]
-        startProcessInBackgroundWithLogFile(cmd, fp, "/tmp/virtual_det_" + name + str(i) + ".txt")
+        startProcessInBackgroundWithLogFile(cmd, fp, "/tmp/virtual_det_" + name + "_" + str(i) + ".txt")
         match name:
             case 'jungfrau':
                 time.sleep(7)
@@ -215,13 +216,43 @@ def loadConfig(name, rx_hostname, settingsdir, fp, num_mods = 1, num_frames = 1)
             d.setThresholdEnergy(4500, detectorSettings.STANDARD)
 
         d.frames = num_frames
+      
     except Exception as e:
         raise RuntimeException(f'Could not load config for {name}. Error: {str(e)}') from e
     
     return d
 
+# for easy acquire
+def loadBasicSettings(name, d, fp):
+    Log(LogLevel.INFO, 'Loading basic settings for ' + name)
+    Log(LogLevel.INFO, 'Loading basic settings for ' + name, fp)
+    try:
+        # basic settings for easy acquire
+        if name == "jungfrau":
+            d.exptime = timedelta(microseconds = 200)
+            d.readnrows = 512
+        elif name == "moench":
+            d.exptime = timedelta(microseconds = 200)
+            d.readnrows = 400
+        elif name == "eiger":
+            d.exptime = timedelta(microseconds = 200)
+            d.readnrows = 256
+            d.dr = 16
+        elif name == "mythen3":
+            d.setExptime(-1, timedelta(microseconds = 200))
+            d.dr = 16
+            d.counters = [0, 1]
+        elif name == "gotthard2":
+            d.exptime = timedelta(microseconds = 200)
+            d.burstmode = burstMode.CONTINUOUS_EXTERNAL
+            d.bursts = 1
+            d.burstperiod = 0
+        d.period = timedelta(milliseconds = 2)
 
-def ParseArguments(description, default_num_mods=1):
+    except Exception as e:
+        raise RuntimeException(f'Could not load config for {name}. Error: {str(e)}') from e
+    
+def ParseArguments(description, default_num_mods=1, markers=0):
     parser = argparse.ArgumentParser(description)
 
     parser.add_argument('rx_hostname', nargs='?', default='localhost',
@@ -234,6 +265,9 @@ def ParseArguments(description, default_num_mods=1):
                         help='Number of frames to test with')
     parser.add_argument('-s', '--servers', nargs='*',
                         help='Detector servers to run')
+    if markers == 1:
+        parser.add_argument('-m', '--markers', nargs='?', default ='[.cmdcall]',
+                        help = 'Markers to use for cmd tests, default: [.cmdcall]')
 
     args = parser.parse_args()
 
@@ -249,11 +283,17 @@ def ParseArguments(description, default_num_mods=1):
             'xilinx_ctb'
         ]
 
-    Log(LogLevel.INFO, 'Arguments:\n' + 
-        'rx_hostname: ' + args.rx_hostname + 
-        '\nsettingspath: \'' + args.settingspath + 
-        '\nservers: \'' + ' '.join(args.servers) + 
-        '\nnum_mods: \'' + str(args.num_mods) + 
-        '\nnum_frames: \'' + str(args.num_frames) + '\'') 
+    msg = (
+        'Arguments:\n'
+        f'rx_hostname: {args.rx_hostname}\n'
+        f"settingspath: '{args.settingspath}'\n"
+        f"servers: '{' '.join(args.servers)}'\n"
+        f"num_mods: '{args.num_mods}'\n"
+        f"num_frames: '{args.num_frames}'"
+    )
+    if markers == 1:
+        msg += f"\nmarkers: '{args.markers}'"
+    Log(LogLevel.INFO, msg)
+
 
     return args