9.2.0.rc (#1232)

* Python module is now built using scikit-build-core: Dev/scikitbuild #1164
* slsdet is available on PyPI from this release onwards
* Fixed broken import in typecaster.h #1181
* Dev/automate version number #1193
* Dev/automate version part2 #1209
* 9111: added expat to host section to fix conda #1216
* fix for gotthard.py to import slsdet properly
* added slsFramesynchronizer to conda copy_lib.sh
* version of release
* update version of client
* removed cmake <=3.28 that was added in 9.1.11 (main_library in meta.yaml in conda-recipes)
* added slsFrameSynchronizer binary to conda
* added numpy dependency to toml
* added documentation for pip in installation

Detailed Commits:
* skeleton pyproject.toml
* moved compiled extension into slsdet
* WIP
* WI{
* separated the recipes
* restored comments, cleanup
* cleaned meta yaml
* added back some python versions
* conda build of main library
* fixed typo
* removed conda build pin
* added zlib
* added workflow for python lib
* patching libzmq and cleaned up cmake
* removed compiler version
* switched patch tool
* reverted to scikit-build in pyproject.toml
* added sls_detector bin
* added sync, renamed action
* update cmake<=3.28 in conda build requirements
* Fixed broken import in typecaster.h (#1181)
- Fixed the broken import _slsdet --> slsdet._slsdet caused by a previous upgrade
- added tests that exercises the conversion from python to C++ and from C++ to python
- Python unit tests now run in CI (!)
* removed 3.28 restriction on cmake in meta.yaml
* from #1216 to 9.1.1.rc that got lost in merge from develoepr, added expat to host section to fix conda build
* back with the cmake restriction
* fixing gotthard1 import
* version number automated for python build
* mistakenly set version back to 0.0.0
* updated github workflow scripts to support automatic version numbering with environment variable
* managed to load VERSION file in yaml file - simplifies things
* saving changes in git workflow failed
* got typo in github workflow
* updatet regex pattern to support postfix
* normalized version to PEP 440 specification in update_version.py
* bug did not support version 0.0.0
* added regex pattern matching to version in toml file
* version now supports . before postfix
* updates api version based on version file & converted shell script files to python
* updated all makefiles
* adresses review comments
* updated release version and the api lib version
* raise an exception if the pull socket python script had errors at startup (for eg if pyzmq was not installed)
* cmake<=3.28 not required anymore
* updated documentation for pip installation as well
* 920/add numpy  (#1226)
* added numpy dependency
* aded build specifications for python version and platform
* release notes
---------
Co-authored-by: froejdh_e <erik.frojdh@psi.ch>
Co-authored-by: Fröjd Lars Erik <froejdh_e@pcmoench03.psi.ch>
Co-authored-by: Erik Fröjdh <erik.frojdh@gmail.com>
Co-authored-by: mazzol_a <mazzol_a@pc17378.psi.ch>
Co-authored-by: AliceMazzoleni99 <alice.mazzoleni@psi.ch>

.github/workflows/cmake.yaml

@@ -1,4 +1,4 @@
-name: CMake
+name: Native CMake Build
 
 on: [push, pull_request]
 
@@ -14,7 +14,13 @@ jobs:
     runs-on: ubuntu-latest
     name: Configure and build using cmake
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: 3.12
+          cache: 'pip'
+      - run: pip install pytest numpy
+
       - uses: awalsh128/cache-apt-pkgs-action@latest
         with:
           packages: libhdf5-dev qtbase5-dev qt5-qmake libqt5svg5-dev libpng-dev libtiff-dev
@@ -27,12 +33,15 @@ jobs:
 
       - name: Build
         # Build your program with the given configuration
-        run: cmake --build ${{github.workspace}}/build -j2 --config ${{env.BUILD_TYPE}}
+        run: cmake --build ${{github.workspace}}/build -j4 --config ${{env.BUILD_TYPE}}
 
-      - name: Test
+      - name: C++ unit tests
         working-directory: ${{github.workspace}}/build
-        # Execute tests defined by the CMake configuration.
-        # See https://cmake.org/cmake/help/latest/manual/ctest.1.html for more detail
         run: ctest -C ${{env.BUILD_TYPE}} -j1
 
+      - name: Python unit tests
+        working-directory: ${{github.workspace}}/build/bin
+        run: |
+          python -m pytest ${{github.workspace}}/python/tests
+
 

.github/workflows/conda_library.yaml

@@ -0,0 +1,42 @@
+name: Build slsdetlib
+
+on: [pull_request]
+
+jobs:
+  build:
+    strategy:
+      fail-fast: false
+      matrix:
+        platform: [ubuntu-latest, ] # macos-12, windows-2019]
+        python-version: ["3.12",]
+
+    runs-on: ${{ matrix.platform }}
+
+    # The setup-miniconda action needs this to activate miniconda
+    defaults:
+      run:
+        shell: "bash -l {0}"
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Get conda
+        uses: conda-incubator/setup-miniconda@v3.0.4
+        with:
+          python-version: ${{ matrix.python-version }}
+          channels: conda-forge
+
+      - name: Prepare
+        run: conda install conda-build conda-verify pytest anaconda-client
+
+      - name: Disable upload
+        run: conda config --set anaconda_upload no
+
+      - name: Build
+        run: conda build conda-recipes/main-library --output-folder build_output
+
+      - name: Upload all Conda packages
+        uses: actions/upload-artifact@v4
+        with:
+          name: conda-packages
+          path: build_output/**  # Uploads all packages

.github/workflows/conda_python.yaml

@@ -0,0 +1,42 @@
+name: slsdet
+
+on: [pull_request]
+
+jobs:
+  build:
+    strategy:
+      fail-fast: false
+      matrix:
+        platform: [ubuntu-latest, ] # macos-12, windows-2019]
+        python-version: ["3.12",]
+
+    runs-on: ${{ matrix.platform }}
+
+    # The setup-miniconda action needs this to activate miniconda
+    defaults:
+      run:
+        shell: "bash -l {0}"
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Get conda
+        uses: conda-incubator/setup-miniconda@v3.0.4
+        with:
+          python-version: ${{ matrix.python-version }}
+          channels: conda-forge
+
+      - name: Prepare
+        run: conda install conda-build conda-verify pytest anaconda-client
+
+      - name: Disable upload
+        run: conda config --set anaconda_upload no
+
+      - name: Build
+        run: conda build conda-recipes/python-client --output-folder build_output 
+
+      - name: Upload all Conda packages
+        uses: actions/upload-artifact@v4
+        with:
+          name: conda-packages
+          path: build_output/**  # Uploads all packages

CMakeLists.txt

@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: LGPL-3.0-or-other
 # Copyright (C) 2021 Contributors to the SLS Detector Package
-cmake_minimum_required(VERSION 3.14)
+cmake_minimum_required(VERSION 3.15)
 project(slsDetectorPackage)
 
 # Read VERSION file into project version
@@ -29,20 +29,40 @@ include(FetchContent)
 option(SLS_FETCH_ZMQ_FROM_GITHUB "Fetch zmq from github" OFF)
 option(SLS_FETCH_PYBIND11_FROM_GITHUB "Fetch pybind11 from github" OFF)
 
+
+
+# Allow FetchContent_Populate to be called with a single argument
+# otherwise deprecated warning is issued
+# Note: From cmake 3.28 we can pass EXCLUDE_FROM_ALL to FetchContent_Declare
+# and avoid direct use of Populate
+if (${CMAKE_VERSION} VERSION_GREATER_EQUAL "3.30")
+    cmake_policy(SET CMP0169 OLD) 
+endif()
+
+# Patch libzmq to set minimum cmake version to 3.15 to avoid warnings
+# with newer cmake versions
+# Patch is applied in the FetchContent_Declare
+set(SLS_LIBZMQ_VERSION "4.3.4")
+
+
 if(SLS_FETCH_ZMQ_FROM_GITHUB)
     # Opt in to pull down a zmq version from github instead of
-    # using the bundled verison
+    # using the bundled version
     FetchContent_Declare(
         libzmq
         GIT_REPOSITORY https://github.com/zeromq/libzmq.git
-        GIT_TAG        v4.3.4
+        GIT_TAG        v${SLS_LIBZMQ_VERSION}
+        PATCH_COMMAND ${CMAKE_COMMAND} -E chdir <SOURCE_DIR> patch -p1 < ${CMAKE_CURRENT_SOURCE_DIR}/libs/libzmq/libzmq_cmake_version.patch
+        UPDATE_DISCONNECTED 1
     )
 else()
     # Standard behaviour use libzmq included in this repo (libs/libzmq)
     FetchContent_Declare(
         libzmq
-        URL ${CMAKE_CURRENT_SOURCE_DIR}/libs/libzmq/libzmq-4.3.4.tar.gz
+        URL ${CMAKE_CURRENT_SOURCE_DIR}/libs/libzmq/libzmq-${SLS_LIBZMQ_VERSION}.tar.gz
         URL_HASH MD5=cc20b769ac10afa352e5ed2769bb23b3
+        PATCH_COMMAND ${CMAKE_COMMAND} -E chdir <SOURCE_DIR> patch -p1 < ${CMAKE_CURRENT_SOURCE_DIR}/libs/libzmq/libzmq_cmake_version.patch
+        UPDATE_DISCONNECTED 1
     )
 endif()
 
@@ -54,6 +74,11 @@ set(ENABLE_CPACK OFF CACHE BOOL "")
 set(ENABLE_CLANG OFF CACHE BOOL "")
 set(ENABLE_CURVE OFF CACHE BOOL "")
 set(ENABLE_DRAFTS OFF CACHE BOOL "")
+set(ENABLE_PRECOMPILED OFF CACHE BOOL "")
+set(WITH_DOC OFF CACHE BOOL "")
+set(WITH_DOCS OFF CACHE BOOL "")
+
+ 
 
 # Using GetProperties and Populate to be able to exclude zmq
 # from install (not possible with FetchContent_MakeAvailable(libzmq))
@@ -191,7 +216,7 @@ endif()
 # to control options for the libraries
 if(NOT TARGET slsProjectOptions)
     add_library(slsProjectOptions INTERFACE)
-    target_compile_features(slsProjectOptions INTERFACE cxx_std_11)
+   target_compile_features(slsProjectOptions INTERFACE cxx_std_11)
 endif()
 
 if (NOT TARGET slsProjectWarnings)
@@ -329,9 +354,15 @@ if (SLS_USE_CTBGUI)
     add_subdirectory(pyctbgui)
 endif(SLS_USE_CTBGUI)
 
-configure_file( .clang-tidy  
+# Workaround for file note being copied to build directory
-    ${CMAKE_BINARY_DIR}/.clang-tidy  
+# when issuing a python -m build
-)
+# TODO! Proper fix
+if(EXISTS ".clang-tidy")
+    configure_file(.clang-tidy  
+        ${CMAKE_BINARY_DIR}/.clang-tidy  
+    )
+endif()
+
 
 if (SLS_BUILD_EXAMPLES)
     add_subdirectory(sample)

RELEASE.txt

@@ -1,15 +1,13 @@
-SLS Detector Package Minor Release 9.1.0 released on 28.03.2025
+SLS Detector Package Minor Release 9.2.0 released on 02.06.2025
-===============================================================
+==================================================================
 
-This document describes the differences between v9.1.0 and v9.0.0   
+This document describes the differences between v9.2.0 and v9.1.1
 
 
 
     CONTENTS
     --------
-    1           Changes
+    1           New or Changed Features
-        1.1     New or Changed Features
-        1.2     Resolved Features
     2           On-board Detector Server Compatibility
     3           Firmware Requirements
     4           Kernel Requirements
@@ -18,130 +16,21 @@ This document describes the differences between v9.1.0 and v9.0.0
 
 
 
-1 Changes
+1 New or Changed Features
-==========
+=========================
 
 
-
+    Python
-1.1 New or Changed Features
+    ------
-============================
 
 
-    Receiver
+    *   Python module is now built using scikit-build-core
-    --------
-
-
-    *   Frame Synchronizer (experimental)
-        Added a new binary, similar to slsMultiReceiver, to collect images from
-        several receivers and stream them out as a ZMQ  multipart message 
-        (one part for each UDP port). No reconstuction of the image. Includeds start 
-        and end ZMQ messages as well for the start and end callback parameters.
-
-
-    Documentation
-    -------------
-
-
-    *   Command line - multi module and multi detector indices
-        Help on this topic has been added to the 'Command line' topic.
-
-
-    *   Row and column index (UDP header or callback)
-        Help on how this is determined from the hostname is added to the 'UDP
-        Header' and the 'Quick Start Guide' topics. Also added to the help in '
-        hostname' command line help. Please note that this can be overwritten by
-        corresponding row and column commands.
-
-
-
-1.2 Resolved Features 
-======================
-
-
-    Firmware
-    ---------
-
-
-    *   [Jungfrau] Column select and filter resistor
-        Configuration fix for chip v1.1 for these parameters
-
-
-    Firmware &/ On-board Detector Server
-    ------------------------------------
-
-
-    *   [Jungfrau] Timing Info Decoder 
-        Only allowed for hardware v2.0 now.
-
-
-    *   [Jungfrau] Auto Comparator Disable - chip v1.0
-        Previously, this mode for chip v1.0 automatically disabled the on-chip
-        gain switching compatator after a fixed portion of the exposure time.
-        Now, one must set also the comparator disable time using 'compdisabletime'
-        just as in chip v1.1.
-
-
-    *   [Mythen3] Default period on server start up is 0 now.
-        
-
-    Client
-    -------
-
-
-    *   Command line - Multi detector index inside file
-        Multi detector index '[index]-' was ignored silently in the config/parameter
-        file since 5.0.0. Now, it will throw an exception. Please use the multi
-        detector index on the 'config' or 'parameter' command instead.
-
-
-    *   [Mythen3] patternX command autocompletes the argument to a path now.
-
-
-    Receiver
-    --------
-
-
-    *   Multiple Receiver objects in multiple threads
-        slsMultiReceiver uses child processes, but if user rewrote to use multiple
-        receiver objects in multiple threads instead, a callback mutex is now
-        implemented to handle the locking mechanism between threads for the callbacks. 
-
-
-    *   Removed potentially unsafe str().c_str() calls.
-
-
-    *   slsMultiReceiver Ctrl + C
-        Now cleans up properly upon Ctrl + C, including exiting the Arping thread.
-
     
-    *   slsMultiReceiver version
-        --version or -v now gives the version of slsMultiReceiver.
 
- 
+    *   slsdet is available on PyPI from this release onwards
-    ZMQ
-    ---
 
 
-    *   [Moench] Reduced significant print out in zmq processing using energy
+    *   Updated documentation on python module installation
-        threshold.
-
-
-    *   [Moench] Zmq dummy packet restreaming command did nothing
-        Temporary solution was to move from 'stop' to 'rx_stop' as 'stop' did not
-        go further if module is idle.
-
-
-    *   [Moench] Too many Zmq dummy packets- unclear end in acquire
-        Give time to process dummy packet before restreaming it and wait more 
-        before restreaming to reduce amoutn of zmq dummy packets to process.
-
-
-    Simulators
-    -----------
-
-
-    *   [Jungfrau][Moench] Slightly faster transmistting time by removing sleeping
-        only if there is a transmission delay
 
 
 
@@ -151,7 +40,7 @@ This document describes the differences between v9.1.0 and v9.0.0
 
     Eiger       9.0.0
     Jungfrau    9.1.0
-    Mythen3     9.1.0
+    Mythen3     9.1.1
     Gotthard2   9.0.0
     Gotthard    9.0.0
     Moench      9.0.0
@@ -326,3 +215,4 @@ This document describes the differences between v9.1.0 and v9.0.0
 
         dhanya.thattil@psi.ch
         erik.frojdh@psi.ch
+        alice.mazzoleni@psi.ch

VERSION

@@ -1 +1 @@
-9.1.0
+9.2.0

conda-recipe/build_pylib.sh

@@ -1,11 +0,0 @@
-# SPDX-License-Identifier: LGPL-3.0-or-other
-# Copyright (C) 2021 Contributors to the SLS Detector Package
-
-echo "|<-------- starting python build"
-
-cd python
-
-# copy VERSION into slsdet for installation
-cp ../VERSION slsdet/VERSION
-
-${PYTHON} setup.py install 

conda-recipe/conda_build_config.yaml

@@ -1,14 +0,0 @@
-python:
-    - 3.8
-    - 3.9
-    - 3.10
-    - 3.11 
-    - 3.12
-    - 3.13
-
-
-c_stdlib:
-  - sysroot                    # [linux]
-
-c_stdlib_version:              # [linux]
-  - 2.17                       # [linux]

conda-recipe/copy_ctbgui.sh

@@ -1,11 +0,0 @@
-# SPDX-License-Identifier: LGPL-3.0-or-other
-# Copyright (C) 2021 Contributors to the SLS Detector Package
-mkdir $PREFIX/lib
-mkdir $PREFIX/bin
-mkdir $PREFIX/include
-
-
-
-cp build/bin/ctbGui $PREFIX/bin/.
-cp build/bin/libctbRootLib.so $PREFIX/lib/.
-

conda-recipe/run_test.sh

@@ -1,3 +0,0 @@
-# SPDX-License-Identifier: LGPL-3.0-or-other
-# Copyright (C) 2021 Contributors to the SLS Detector Package
-ctest -j2

conda-recipes/main-library/build.sh

@@ -8,7 +8,7 @@ if [ ! -d "install" ]; then
     mkdir install
 fi
 cd build
-cmake .. \
+cmake .. -G Ninja \
       -DCMAKE_PREFIX_PATH=$CONDA_PREFIX \
       -DCMAKE_INSTALL_PREFIX=install \
       -DSLS_USE_TEXTCLIENT=ON \
@@ -18,7 +18,7 @@ cmake .. \
       -DSLS_USE_TESTS=ON \
       -DSLS_USE_PYTHON=OFF \
       -DCMAKE_BUILD_TYPE=Release \
-      -DSLS_USE_HDF5=OFF\
+      -DSLS_USE_HDF5=OFF \
      
 NCORES=$(getconf _NPROCESSORS_ONLN)
 echo "Building using: ${NCORES} cores"

conda-recipes/main-library/conda_build_config.yaml

@@ -0,0 +1,13 @@
+
+c_compiler:
+  - gcc                        # [linux]
+
+c_stdlib:
+  - sysroot                    # [linux]
+
+cxx_compiler:
+  - gxx                        # [linux]
+
+
+c_stdlib_version:              # [linux]
+  - 2.17                       # [linux]

conda-recipes/main-library/copy_lib.sh

@@ -4,7 +4,6 @@
 mkdir -p $PREFIX/lib
 mkdir -p $PREFIX/bin
 mkdir -p $PREFIX/include/sls
-# mkdir $PREFIX/include/slsDetectorPackage
 
 #Shared and static libraries
 cp build/install/lib/* $PREFIX/lib/
@@ -15,8 +14,10 @@ cp build/install/bin/sls_detector_acquire_zmq $PREFIX/bin/.
 cp build/install/bin/sls_detector_get $PREFIX/bin/.
 cp build/install/bin/sls_detector_put $PREFIX/bin/.
 cp build/install/bin/sls_detector_help $PREFIX/bin/.
+cp build/install/bin/sls_detector $PREFIX/bin/.
 cp build/install/bin/slsReceiver $PREFIX/bin/.
 cp build/install/bin/slsMultiReceiver $PREFIX/bin/.
+cp build/install/bin/slsFrameSynchronizer $PREFIX/bin/.
 
 
 cp build/install/include/sls/* $PREFIX/include/sls

conda-recipes/main-library/meta.yaml

@@ -1,11 +1,11 @@
+source:
+  path: ../..
+
+{% set version = load_file_regex(load_file = 'VERSION', regex_pattern = '(\d+(?:\.\d+)*(?:[\+\w\.]+))').group(1) %}
 package:
   name: sls_detector_software
-  version: {{ environ.get('GIT_DESCRIBE_TAG', '') }}
+  version: {{ version }} 
-
+  
-
-source:
-  path: ..
-
 build:
   number: 0
   binary_relocation: True
@@ -15,19 +15,21 @@ build:
 requirements:
   build:
     - {{ compiler('c') }}
-    - {{stdlib('c')}}
+    - {{ stdlib("c") }}
-    - {{compiler('cxx')}}
+    - {{ compiler('cxx') }}
-    - cmake<=3.28
+    - git
+    - cmake
     - ninja
     - qt 5.*
 
   host:
     - libstdcxx-ng
     - libgcc-ng
-    - libgl-devel
+    - libgl-devel  # [linux]
     - libtiff
     - zlib
-  
+    - expat
+
   run:
     - libstdcxx-ng
     - libgcc-ng 
@@ -40,44 +42,13 @@ outputs:
     requirements:
       build:
         - {{ compiler('c') }}
-        - {{compiler('cxx')}}
+        - {{ stdlib("c") }}
-        - libstdcxx-ng
+        - {{ compiler('cxx') }}
-        - libgcc-ng
 
       run: 
         - libstdcxx-ng
         - libgcc-ng
 
-  - name: slsdet
-    script: build_pylib.sh
-
-    requirements:
-      build:
-        - python
-        - {{ compiler('c') }}
-        - {{compiler('cxx')}}
-        - {{ pin_subpackage('slsdetlib', exact=True) }}
-        - setuptools
-        - pybind11=2.13
-
-      host:
-        - python
-        - {{ pin_subpackage('slsdetlib', exact=True) }}
-        - setuptools
-        - pybind11=2.13
-
-
-      run:
-        - libstdcxx-ng
-        - libgcc-ng
-        - python
-        - numpy
-        - {{ pin_subpackage('slsdetlib', exact=True) }}
-
-    test:
-      imports:
-        - slsdet
-
 
   - name: slsdetgui
     script: copy_gui.sh
@@ -91,7 +62,7 @@ outputs:
       run:
         - {{ pin_subpackage('slsdetlib', exact=True) }}
         - qt 5.*
-        - expat
+
 
   - name: moenchzmq
     script: copy_moench.sh
@@ -105,5 +76,3 @@ outputs:
 
       run:
         - {{ pin_subpackage('slsdetlib', exact=True) }}
-        - expat
-

conda-recipes/python-client/conda_build_config.yaml

@@ -0,0 +1,16 @@
+python:
+    - 3.11 
+    - 3.12
+    - 3.13
+
+c_compiler:
+  - gcc                        # [linux]
+
+c_stdlib:
+  - sysroot                    # [linux]
+
+cxx_compiler:
+  - gxx                        # [linux]
+
+c_stdlib_version:              # [linux]
+  - 2.17                       # [linux]

conda-recipes/python-client/meta.yaml

@@ -0,0 +1,45 @@
+source:
+  path: ../..
+
+{% set version = load_file_regex(load_file = 'VERSION', regex_pattern = '(\d+(?:\.\d+)*(?:[\+\w\.]+))').group(1) %}
+package:
+  name: slsdet
+  version: {{ version }} 
+
+build:
+  number: 0
+  script:
+    - unset CMAKE_GENERATOR && {{ PYTHON }} -m pip install . -vv  # [not win]
+
+requirements:
+  build:
+    - python {{python}}
+    - {{ compiler('c') }}
+    - {{ stdlib("c") }}
+    - {{ compiler('cxx') }}
+
+  host:
+    - cmake
+    - ninja
+    - python {{python}}
+    - pip
+    - scikit-build-core
+    - pybind11 >=2.13.0
+    - fmt
+    - zeromq
+    - nlohmann_json
+    - catch2
+
+  run:
+    - python {{python}}
+    - numpy
+
+
+test:
+  imports:
+    - slsdet
+
+
+about:
+  summary: An example project built with pybind11 and scikit-build.
+  # license_file: LICENSE

docs/src/index.rst

@@ -79,8 +79,9 @@ Welcome to slsDetectorPackage's documentation!
     :caption: Receiver
     :maxdepth: 2
 
-    receivers
     slsreceiver
+    receivers
+    
 
 .. toctree::
     :caption: Receiver Files

docs/src/installation.rst

@@ -6,18 +6,33 @@
 Installation
 ===============
 
-One can either install pre-built binaries using conda or build from source.
+.. contents::
+   :local:
+   :depth: 2
+   :backlinks: top
+
+
+Overview
+--------------
+
+The ``slsDetectorPackage`` provides core detector software implemented in C++, along with Python bindings packaged as the ``slsdet`` Python extension module. Choose the option that best fits your environment and use case.
+
+:ref:`conda pre-built binaries`: 
+Install pre-built binaries for the C++ client, receiver, GUI and the Python API (``slsdet``), simplifying setup across platforms.
+
+:ref:`pip`:
+Install only the Python extension module, either by downloading the pre-built library from PyPI or by building the extension locally from source. Available only from v9.2.0 onwards.
+
+:ref:`build from source`: 
+Compile the entire package yourself, including both the C++ core and the Python bindings, for maximum control and customization. However, make sure that you have the :doc:`dependencies <../dependencies>` installed. If installing using conda, conda will manage the dependencies. Avoid installing packages with pip and conda simultaneously.
+
+
 
-.. warning ::
-    
-    Before building from source make sure that you have the 
-    :doc:`dependencies <../dependencies>` installed. If installing using conda, conda will 
-    manage the dependencies. Avoid also installing packages with pip. 
    
+.. _conda pre-built binaries:
 
-
+1. Install pre-built binaries using conda (Recommended)
-Install binaries using conda
+--------------------------------------------------------
-----------------------------------
 
 Conda is not only useful to manage python environments but can also
 be used as a user space package manager. Dates in the tag (for eg. 2020.07.23.dev0) 
@@ -63,12 +78,29 @@ We have four different packages available:
     conda search moenchzmq
 
 
+.. _pip:
+
+2. Pip
+-------
+The Python extension module ``slsdet`` can be installed using pip. This is available from v9.2.0 onwards.
+
+.. code-block:: bash
+    
+    #Install the Python extension module from PyPI
+    pip install slsdet
+
+    # or install the python extension locally from source
+    git clone https://github.com/slsdetectorgroup/slsDetectorPackage.git --branch 9.2.0
+    cd slsDetectorPackage
+    pip install .
 
 
-Build from source
+.. _build from source:
-----------------------
 
-1. Download Source Code from github
+3. Build from source
+-------------------------
+
+3.1. Download Source Code from github
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: bash
@@ -83,12 +115,12 @@ Build from source
 
 
 
-2. Build from Source
+3.2. Build from Source
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 One can either build using cmake or use the in-built cmk.sh script.
 
-Build using CMake
+3.2.1. Build using CMake
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: bash
@@ -135,7 +167,7 @@ Example cmake options               Comment
     For v7.x.x of slsDetectorPackage and older, refer :ref:`zeromq notes for cmake option to hint library location. <zeromq for different slsDetectorPackage versions>` 
 
 
-Build using in-built cmk.sh script
+3.2.2. Build using in-built cmk.sh script
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 
@@ -185,8 +217,8 @@ Build using in-built cmk.sh script
     For v7.x.x of slsDetectorPackage and older, refer :ref:`zeromq notes for cmk script option to hint library location. <zeromq for different slsDetectorPackage versions>` 
 
 
-Build on old distributions
+3.3. Build on old distributions using conda
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 If your linux distribution doesn't come with a C++11 compiler (gcc>4.8) then 
 it's possible to install a newer gcc using conda and build the slsDetectorPackage
@@ -210,7 +242,7 @@ using this compiler
 
 
 
-Build slsDetectorGui (Qt5)
+3.4. Build slsDetectorGui (Qt5)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 1. Using pre-built binary on conda
@@ -271,7 +303,7 @@ Build slsDetectorGui (Qt5)
 
 
 
-Build this documentation
+3.5. Build this documentation
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The documentation for the slsDetectorPackage is build using a combination 
@@ -294,8 +326,8 @@ is to use conda
     make rst # rst only, saves time in case the API did not change
 
 
-Pybind and Zeromq
+4. Pybind and Zeromq
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+-------------------------
 
 .. _pybind for different slsDetectorPackage versions:
 

docs/src/receivers.rst

@@ -1,25 +1,25 @@
-Receivers
+Custom Receiver
 =================
 
-Receiver processes can be run on same or different machines as the client, receives the data from the detector (via UDP packets).
+The receiver essentially listens to UDP data packets sent out by the detector.
-When using the slsReceiver/ slsMultiReceiver, they can be further configured by the client control software (via TCP/IP) to set file name, file path, progress of acquisition etc.
+
+To know more about detector receiver setup in the config file, please check out :ref:`the detector-receiver UDP configuration in the config file<detector udp header config>` and the :ref:`detector udp format<detector udp header>`.
 
 
-To know more about detector receiver configuration, please check out :ref:`detector udp header and udp commands in the config file <detector udp header>`
+| Please note the following when using a custom receiver:
 
-Custom Receiver
+* **udp_dstmac** must be configured in the config file. This parameter is not required when using an in-built receiver.
-----------------
 
-| When using  custom receiver with our package, ensure that **udp_dstmac** is also configured in the config file. This parameter is not required when using slsReceiver.
+* Cannot use "auto" for **udp_dstip**.
 
-| Cannot use "auto" for **udp_dstip**.
+* No **rx_** commands in the config file. These commands are for configuring the slsReceiver.
 
-| Also ensure that there are no **rx_** commands in the config file. These commands are for configuring the slsReceiver.
+
+
+The main difference is the lack of **rx_** commands or file commands (eg. **f**write, **f**path) and the **udp_dstmac** is required in config file.
 
 Example of a custom receiver config file
 
-* The main difference is the lack of **rx_** commands or file commands (eg. fwrite, fpath) and the udp_dstmac is required in config file.
-
 .. code-block:: bash
 
     # detector hostname

docs/src/serverupgrade.rst

@@ -1,4 +1,5 @@
 .. _Detector Server Upgrade:
+
 Upgrade
 ========
 

docs/src/slsreceiver.rst

@@ -1,16 +1,55 @@
-slsReceiver/ slsMultiReceiver
+In-built Receiver   
 ================================
 
-| One has to start the slsReceiver before loading config file or using any receiver commands (prefix: **rx_** )
 
+
+The receiver essentially listens to UDP data packets sent out by the detector. It's main features are:
+
+- **Listening**: Receives UDP data from the detector.
+- **Writing to File**: Optionally writes received data to disk.
+- **Streaming via ZMQ**: Optionally streams out the data using ZeroMQ.
+
+Each of these operations runs asynchronously and in parallel for each UDP port.
+
+
+.. note ::   
+
+    * Can be run on the same or different machine as the client.
+    * Can be configured by the client. (set file name/ discard policy, get progress etc.)
+    * Has to be started before the client runs any receiver specific command.
+
+
+Receiver Variants
+-----------------
+There are three main receiver types. How to start them is described :ref:`below<Starting up the Receiver>`.
+
++----------------------+--------------------+-----------------------------------------+--------------------------------+
+| Receiver Type        | slsReceiver        | slsMultiReceiver                        |slsFrameSynchronizer            | 
++======================+====================+=========================================+================================+
+| Modules Supported    | 1                  | Multiple                                | Multiple                       |
++----------------------+--------------------+-----------------------------------------+--------------------------------+
+| Internal Architecture| Threads per porttt | Multiple child processes of slsReceiver | Multi-threading of slsReceiver |
++----------------------+--------------------+-----------------------------------------+--------------------------------+
+| ZMQ Streaming        | Disabled by default| Disabled by default                     | Enabled, not optional          |
++----------------------+--------------------+-----------------------------------------+--------------------------------+
+| ZMQ Synchronization  | No                 | No                                      | Yes, across ports              |
++----------------------+--------------------+-----------------------------------------+--------------------------------+
+| Image Reconstruction | No                 | No                                      | No                             |
++----------------------+--------------------+-----------------------------------------+--------------------------------+
+
+
+
+
+.. _Starting up the Receiver:
+
+Starting up the Receiver
+-------------------------
 For a Single Module
     .. code-block:: bash  
+        
+        slsReceiver # default port 1954
 
-        # default port 1954
+        slsReceiver -t2012 # custom port 2012
-        slsReceiver
-
-        # custom port 2012
-        slsReceiver -t2012
 
 
 For Multiple Modules
@@ -18,57 +57,66 @@ For Multiple Modules
 
         # each receiver (for each module) requires a unique tcp port (if all on same machine)
 
-        # using slsReceiver in multiple consoles
+        # option 1 (one for each module)
         slsReceiver
         slsReceiver -t1955
 
-        # slsMultiReceiver [starting port] [number of receivers]
+        # option 2
         slsMultiReceiver 2012 2
 
-        # slsMultiReceiver [starting port] [number of receivers] [print each frame header for debugging]
+        # option 3
-        slsMultiReceiver 2012 2 1
+        slsFrameSynchronizer 2012 2
+
 
 
 Client Commands 
 -----------------
 
-| One can remove **udp_dstmac** from the config file, as the slsReceiver fetches this from the **udp_ip**.
+* Client commands to the receiver begin with **rx_** or **f_** (file commands).
 
-| One can use "auto" for **udp_dstip** if one wants to use default ip of **rx_hostname**.
+* **rx_hostname** has to be the first command to the receiver so the client knows which receiver process to communicate with.
 
-| The first command to the receiver (**rx_** commands) should be **rx_hostname**. The following are the different ways to establish contact.
+* Can use 'auto' for **udp_dstip** if using 1GbE interface or the :ref:`virtual simulators<Virtual Detector Servers>`.
+
+
+To know more about detector receiver setup in the config file, please check out :ref:`the detector-receiver UDP configuration in the config file<detector udp header config>` and the :ref:`detector udp format<detector udp header>`.
+
+
+The following are the different ways to establish contact using **rx_hostname** command.
 
     .. code-block:: bash  
 
-        # default receiver tcp port (1954)
+        # ---single module---
+
+        # default receiver port at 1954
+        rx_hostname xxx
+       
+        # custom receiver port
+        rx_hostname xxx:1957 # option 1
+       
+        rx_tcpport 1957  # option 2
         rx_hostname xxx
 
-        # custom receiver port
-        rx_hostname xxx:1957
 
-        # custom receiver port
+        # ---multi module---
-        rx_tcpport 1954
-        rx_hostname xxx
 
-        # multi modules with custom ports
+        # using increasing tcp ports
-        rx_hostname xxx:1955+xxx:1956+
-        
-        
-        # multi modules using increasing tcp ports when using multi detector command
         rx_tcpport 1955
         rx_hostname xxx
 
-        # or specify multi modules with custom ports on same rxr pc
+        # custom ports
-        0:rx_tcpport 1954
+        rx_hostname xxx:1955+xxx:1958+ # option 1
+
+        0:rx_tcpport 1954 # option 2
         1:rx_tcpport 1955
         2:rx_tcpport 1956
         rx_hostname xxx
 
-        # multi modules with custom ports on different rxr pc
+        # custom ports on different receiver machines
         0:rx_tcpport 1954
         0:rx_hostname xxx
         1:rx_tcpport 1955
-        1:rx_hostname yyy
+        1:rx_hostname yyyrxr
 
 
 | Example commands:
@@ -91,6 +139,32 @@ Client Commands
         sls_detector_get -h rx_framescaught
 
 
+Example of a config file using in-built receiver
+
+.. code-block:: bash
+
+    # detector hostname
+    hostname bchip052+bchip053+
+
+    # udp destination port (receiver)
+    # sets increasing destination udp ports starting at 50004
+    udp_dstport 50004
+
+    # udp destination ip (receiver)
+    0:udp_dstip 10.0.1.100
+    1:udp_dstip 10.0.2.100
+
+    # udp source ip (same subnet as udp_dstip)
+    0:udp_srcip 10.0.1.184
+    1:udp_srcip 10.0.2.184
+
+    # udp destination mac - not required (picked up from udp_dstip)
+    #udp_dstmac 22:47:d5:48:ad:ef
+
+    # connects to receivers at increasing tcp port starting at 1954
+    rx_hostname mpc3434
+    # same as rx_hostname mpc3434:1954+mpc3434:1955+
+
 
 
 Performance 

docs/src/udpconfig.rst

@@ -1,4 +1,4 @@
-.. _detector udp header:
+.. _detector udp header config:
 
 
 Config file

docs/src/virtualserver.rst

@@ -1,4 +1,5 @@
 .. _Virtual Detector Servers:
+
 Simulators
 ===========
 

libs/libzmq/libzmq_cmake_version.patch

@@ -0,0 +1,18 @@
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index dd3d8eb9..c0187747 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -1,11 +1,8 @@
+ # CMake build script for ZeroMQ
+ project(ZeroMQ)
+ 
+-if(${CMAKE_SYSTEM_NAME} STREQUAL Darwin)
+-  cmake_minimum_required(VERSION 3.0.2)
+-else()
+-  cmake_minimum_required(VERSION 2.8.12)
+-endif()
++cmake_minimum_required(VERSION 3.15)
++message(STATUS "Patched cmake version")
+ 
+ include(CheckIncludeFiles)
+ include(CheckCCompilerFlag)

pyproject.toml

@@ -0,0 +1,37 @@
+[tool.scikit-build.metadata.version]
+provider = "scikit_build_core.metadata.regex"
+input = "VERSION"
+regex = '^(?P<version>\d+(?:\.\d+)*(?:[\.\+\w]+)?)$'
+result = "{version}" 
+
+[build-system]
+requires = [ "scikit-build-core>=0.10", "pybind11", "numpy",]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "slsdet"
+dynamic = ["version"]
+dependencies = [
+    "numpy", 
+]
+
+[tool.cibuildwheel]
+before-all = "uname -a"
+build = "cp{311,312,313}-manylinux_x86_64"
+
+[tool.scikit-build.build]
+verbose = true
+
+[tool.scikit-build.cmake]
+build-type = "Release"
+
+[tool.scikit-build.install]
+components = [ "python",]
+
+[tool.scikit-build.cmake.define]
+SLS_USE_RECEIVER = "OFF"
+SLS_USE_RECEIVER_BINARIES = "OFF"
+SLS_USE_TEXTCLIENT = "OFF"
+SLS_BUILD_SHARED_LIBRARIES = "OFF"
+SLS_USE_PYTHON = "ON"
+SLS_INSTALL_PYTHONEXT = "ON"

python/CMakeLists.txt

@@ -20,7 +20,7 @@ target_link_libraries(_slsdet PUBLIC
 
 
 set_target_properties(_slsdet PROPERTIES
-                        LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin
+                        LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin/slsdet
                     )
 
 #Copy Python code 
@@ -64,6 +64,10 @@ configure_file( scripts/test_virtual.py
     ${CMAKE_BINARY_DIR}/test_virtual.py 
 )
 
+configure_file(scripts/frameSynchronizerPullSocket.py 
+    ${CMAKE_BINARY_DIR}/bin/frameSynchronizerPullSocket.py COPYONLY)
+
+
 configure_file( ${CMAKE_CURRENT_SOURCE_DIR}/../VERSION  
     ${CMAKE_BINARY_DIR}/bin/slsdet/VERSION
 )
@@ -71,9 +75,18 @@ configure_file( ${CMAKE_CURRENT_SOURCE_DIR}/../VERSION
 if(SLS_INSTALL_PYTHONEXT)
     install(TARGETS _slsdet
         EXPORT "${TARGETS_EXPORT_NAME}"
-        LIBRARY DESTINATION ${CMAKE_INSTALL_PREFIX}/python
+        LIBRARY DESTINATION slsdet
+        COMPONENT python
     )
+    install(
+        FILES ${PYTHON_FILES} 
+        DESTINATION slsdet
+        COMPONENT python
+        )
+    install(
+        FILES ../VERSION 
+        DESTINATION slsdet
+        COMPONENT python
+        )
 
-    install(FILES ${PYTHON_FILES} DESTINATION ${CMAKE_INSTALL_PREFIX}/python/slsdet)
+endif()
-    install(FILES ../VERSION DESTINATION ${CMAKE_INSTALL_PREFIX}/python/slsdet)
-endif()

python/setup.py

@@ -1,77 +0,0 @@
-# SPDX-License-Identifier: LGPL-3.0-or-other
-# Copyright (C) 2021 Contributors to the SLS Detector Package
-"""
-Setup file for slsdet
-Build upon the pybind11 example found here: https://github.com/pybind/python_example
-"""
-
-import os
-import sys
-from setuptools import setup, find_packages
-from pybind11.setup_helpers import Pybind11Extension, build_ext
-
-def read_version():
-    try:
-        version_file = os.path.join(os.path.dirname(__file__), 'slsdet', 'VERSION')
-        with open(version_file, "r") as f:
-            return f.read().strip()
-    except:
-        raise RuntimeError("VERSION file not found in slsdet package from setup.py.")
-
-__version__ = read_version()
-
-
-def get_conda_path():
-    """
-    Keep this a function if we need some fancier logic later
-    """
-    print('Prefix: ', os.environ['CONDA_PREFIX'])
-    return os.environ['CONDA_PREFIX']
-
-
-#TODO migrate to CMake build or fetch files from cmake? 
-ext_modules = [
-    Pybind11Extension(
-        '_slsdet',
-        ['src/main.cpp',
-        'src/enums.cpp',
-        'src/current.cpp',
-        'src/detector.cpp',
-        'src/network.cpp',
-        'src/pattern.cpp',
-        'src/scan.cpp',
-        'src/duration.cpp',
-        'src/DurationWrapper.cpp',
-        'src/pedestal.cpp',
-        ]
-        
-        
-        ,
-        include_dirs=[
-            os.path.join(get_conda_path(), 'include'),
-
-        ],
-        libraries=['SlsDetector', 'SlsSupport', 'SlsReceiver'],
-        library_dirs=[
-            os.path.join(get_conda_path(), 'lib'),
-        ],
-        language='c++'
-    ),
-]
-
-setup(
-    name='slsdet',
-    version=__version__,
-    author='Erik Frojdh',
-    author_email='erik.frojdh@psi.ch',
-    url='https://github.com/slsdetectorgroup/slsDetectorPackage',
-    description='Detector API for SLS Detector Group detectors',
-    long_description='',
-    packages=find_packages(exclude=['contrib', 'docs', 'tests']),
-    package_data={
-        'slsdet': ['VERSION'], 
-    },  
-    ext_modules=ext_modules,
-    cmdclass={"build_ext": build_ext},
-    zip_safe=False,
-)

python/slsdet/__init__.py

@@ -14,7 +14,7 @@ from .moench import Moench
 from .pattern import Pattern, patternParameters
 from .gaincaps import Mythen3GainCapsWrapper
 
-import _slsdet
+from . import _slsdet
 xy = _slsdet.xy
 defs = _slsdet.slsDetectorDefs
 
@@ -41,3 +41,6 @@ def read_version():
     
 __version__ = read_version()
 
+
+
+

python/slsdet/ctb.py

@@ -4,7 +4,7 @@ from .detector import Detector, freeze
 from .utils import element_if_equal
 from .dacs import DetectorDacs, NamedDacs
 from .powers import DetectorPowers, NamedPowers
-import _slsdet
+from . import _slsdet
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 from .detector_property import DetectorProperty
 

python/slsdet/dacs.py

@@ -3,7 +3,7 @@
 from .detector_property import DetectorProperty
 from functools import partial
 import numpy as np
-import _slsdet
+from . import _slsdet
 from .detector import freeze
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 class Dac(DetectorProperty):

python/slsdet/detector.py

@@ -1,8 +1,8 @@
 # SPDX-License-Identifier: LGPL-3.0-or-other
 # Copyright (C) 2021 Contributors to the SLS Detector Package
-from _slsdet import CppDetectorApi
+from ._slsdet import CppDetectorApi
-from _slsdet import slsDetectorDefs
+from ._slsdet import slsDetectorDefs
-from _slsdet import IpAddr, MacAddr
+from ._slsdet import IpAddr, MacAddr
 
 runStatus = slsDetectorDefs.runStatus
 timingMode = slsDetectorDefs.timingMode
@@ -15,7 +15,7 @@ defs = slsDetectorDefs
 
 from .utils import element_if_equal, all_equal, get_set_bits, list_to_bitmask
 from .utils import Geometry, to_geo, element, reduce_time, is_iterable, hostname_list
-from _slsdet import xy
+from ._slsdet import xy
 from .gaincaps import Mythen3GainCapsWrapper
 from . import utils as ut
 from .proxy import JsonProxy, SlowAdcProxy, ClkDivProxy, MaxPhaseProxy, ClkFreqProxy, PatLoopProxy, PatNLoopProxy, PatWaitProxy, PatWaitTimeProxy 

python/slsdet/eiger.py

@@ -11,7 +11,7 @@ Created on Wed Dec  6 11:51:18 2017
 from .detector import Detector
 from .temperature import Temperature, DetectorTemperature
 from .dacs import DetectorDacs
-import _slsdet
+from . import _slsdet
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 from .detector_property import DetectorProperty
 

python/slsdet/enums.py

@@ -15,8 +15,8 @@ if dt === detectorType.EIGER:
 """
 
 
-import _slsdet
+from . import _slsdet
 for name, cls in _slsdet.slsDetectorDefs.__dict__.items():
     if isinstance(cls, type):
-        exec(f'{name} = {cls.__module__}.{cls.__qualname__}')
+        exec(f'{name} = _slsdet.{cls.__qualname__}')
 

python/slsdet/gaincaps.py

@@ -1,6 +1,6 @@
 
 
-import _slsdet
+from . import _slsdet
 gc = _slsdet.slsDetectorDefs.M3_GainCaps
 
 

python/slsdet/gotthard.py

@@ -9,7 +9,7 @@ This file contains the specialization for the Moench detector
 
 from .detector import Detector, freeze
 from .dacs import DetectorDacs
-import _slsdet
+from . import _slsdet
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 from .detector_property import DetectorProperty
 

python/slsdet/gotthard2.py

@@ -11,7 +11,7 @@ from .detector import Detector, freeze
 
 # from .adcs import Adc, DetectorAdcs
 from .dacs import DetectorDacs
-import _slsdet
+from . import _slsdet
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 from .detector_property import DetectorProperty
 

python/slsdet/jungfrau.py

@@ -11,7 +11,7 @@ from .detector import Detector, freeze
 
 # from .adcs import Adc, DetectorAdcs
 from .dacs import DetectorDacs
-import _slsdet
+from . import _slsdet
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 from .detector_property import DetectorProperty
 

python/slsdet/moench.py

@@ -9,7 +9,7 @@ This file contains the specialization for the Moench detector
 
 from .detector import Detector, freeze
 from .dacs import DetectorDacs
-import _slsdet
+from . import _slsdet
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 from .detector_property import DetectorProperty
 

python/slsdet/mythen3.py

@@ -11,7 +11,7 @@ from .detector import Detector, freeze
 
 # from .adcs import Adc, DetectorAdcs
 from .dacs import DetectorDacs
-import _slsdet
+from . import _slsdet
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 gc_enums = _slsdet.slsDetectorDefs.M3_GainCaps
 from .detector_property import DetectorProperty

python/slsdet/pattern.py

@@ -1,8 +1,8 @@
 # SPDX-License-Identifier: LGPL-3.0-or-other
 # Copyright (C) 2021 Contributors to the SLS Detector Package
-import _slsdet
+from . import _slsdet
 
-from _slsdet import Pattern
+from ._slsdet import Pattern
 
 
 class patternParameters(_slsdet.patternParameters):

python/slsdet/powers.py

@@ -3,7 +3,7 @@
 from .detector_property import DetectorProperty
 from functools import partial
 import numpy as np
-import _slsdet
+from . import _slsdet
 from .detector import freeze
 dacIndex = _slsdet.slsDetectorDefs.dacIndex
 class Power(DetectorProperty):

python/slsdet/proxy.py

@@ -3,7 +3,7 @@
 from .utils import element_if_equal
 from .enums import dacIndex
 from .defines import M3_MAX_PATTERN_LEVELS, MAX_PATTERN_LEVELS
-from _slsdet import slsDetectorDefs
+from ._slsdet import slsDetectorDefs
 detectorType = slsDetectorDefs.detectorType
 
 

python/slsdet/utils.py

@@ -6,7 +6,7 @@ but not directly used in controlling the detector
 """
 
 from collections import namedtuple
-import _slsdet  #C++ lib
+from . import _slsdet  #C++ lib
 import functools
 import datetime as dt
 import pathlib

python/src/duration.cpp

@@ -1,3 +1,4 @@
+#include <chrono>
 #include "py_headers.h"
 
 #include "DurationWrapper.h"
@@ -19,4 +20,25 @@ void init_duration(py::module &m) {
                << " count: " << self.count() << ")";
             return ss.str();
         });
+
+    m.def(
+        "test_return_DurationWrapper",
+        []() {
+            DurationWrapper t(1.3);
+            return t;
+        },
+        R"(
+          Test function to return a DurationWrapper object. Ensures that the automatic conversion in typecaster.h works.
+          )");
+
+    m.def(
+        "test_duration_to_ns",
+        [](const std::chrono::nanoseconds t) {
+            //Duration wrapper is used to be able to convert from time in python to chrono::nanoseconds
+            //return count to have something to test 
+            return t.count();
+        },
+        R"(
+          Test function convert DurationWrapper or number to chrono::ns. Ensures that the automatic conversion in typecaster.h works.
+          )"); // default value to test the default constructor
 }

python/src/typecaster.h

@@ -54,11 +54,16 @@ template <> struct type_caster<std::chrono::nanoseconds> {
                 value = duration_cast<nanoseconds>(duration<double>(PyFloat_AsDouble(src.ptr())));
                 return true;
             }
+            // If invoked with an int we assume it is nanoseconds and convert, same as in chrono.h
+            if (PyLong_Check(src.ptr())) {
+                value = duration_cast<nanoseconds>(duration<int64_t>(PyLong_AsLongLong(src.ptr())));
+                return true;
+            }
 
 
             // Lastly if we were actually called with a DurationWrapper object we get
             // the number of nanoseconds and create a std::chrono::nanoseconds from it
-            py::object py_cls = py::module::import("_slsdet").attr("DurationWrapper");
+            py::object py_cls = py::module::import("slsdet._slsdet").attr("DurationWrapper");
             if (py::isinstance(src, py_cls)){
                 sls::DurationWrapper *cls = src.cast<sls::DurationWrapper *>();
                 value = nanoseconds(cls->count());
@@ -77,7 +82,7 @@ template <> struct type_caster<std::chrono::nanoseconds> {
          * set the count from chrono::nanoseconds and return
          */
         static handle cast(std::chrono::nanoseconds src, return_value_policy /* policy */, handle /* parent */) {
-            py::object py_cls = py::module::import("_slsdet").attr("DurationWrapper");
+            py::object py_cls = py::module::import("slsdet._slsdet").attr("DurationWrapper");
             py::object* obj = new py::object;
             *obj = py_cls();
             sls::DurationWrapper *dur = obj->cast<sls::DurationWrapper *>();

python/tests/test_DurationWrapper.py

@@ -0,0 +1,58 @@
+import pytest
+
+from slsdet import DurationWrapper
+
+#import the compiled extension to use test functions for the automatic conversion
+from slsdet import _slsdet
+
+
+def test_default_construction_of_DurationWrapper():
+    """Test default construction of DurationWrapper"""
+    t = DurationWrapper()
+    assert t.count() == 0
+    assert t.total_seconds() == 0
+
+def test_construction_of_DurationWrapper():
+    """Test construction of DurationWrapper with total_seconds"""
+    t = DurationWrapper(5)
+    assert t.count() == 5e9
+    assert t.total_seconds() == 5
+
+def test_set_count_on_DurationWrapper():
+    """Test set_count on DurationWrapper"""
+    t = DurationWrapper()
+    t.set_count(10)
+    assert t.count() == 10
+    assert t.total_seconds() == 10e-9
+    t.set_count(0)
+    assert t.count() == 0
+    assert t.total_seconds() == 0
+
+
+def test_return_a_DurationWrapper_from_cpp():
+    """Test returning a DurationWrapper from C++"""
+    t = _slsdet.test_return_DurationWrapper()
+    assert t.count() == 1.3e9
+    assert t.total_seconds() == 1.3
+
+def test_call_a_cpp_function_with_a_duration_wrapper():
+    """C++ functions can accept a DurationWrapper"""
+    t = DurationWrapper(5)
+    assert _slsdet.test_duration_to_ns(t) == 5e9
+    
+def test_call_a_cpp_function_converting_number_to_DurationWrapper():
+    """int and float can be converted to std::chrono::nanoseconds"""
+    assert _slsdet.test_duration_to_ns(0) == 0
+    assert _slsdet.test_duration_to_ns(3) == 3e9
+    assert _slsdet.test_duration_to_ns(1.3) == 1.3e9
+    assert _slsdet.test_duration_to_ns(10e-9) == 10
+
+def test_call_a_cpp_function_with_datetime_timedelta():
+    """datetime.timedelta can be converted to std::chrono::nanoseconds"""
+    import datetime
+    t = datetime.timedelta(seconds=5)
+    assert _slsdet.test_duration_to_ns(t) == 5e9
+    t = datetime.timedelta(seconds=0)
+    assert _slsdet.test_duration_to_ns(t) == 0
+    t = datetime.timedelta(seconds=1.3)
+    assert _slsdet.test_duration_to_ns(t) == 1.3e9

serverBin/mythen3DetectorServerv9.1.0

@@ -1 +0,0 @@
-../slsDetectorServers/mythen3DetectorServer/bin/mythen3DetectorServerv9.1.0

serverBin/mythen3DetectorServerv9.1.1

@@ -0,0 +1 @@
+../slsDetectorServers/mythen3DetectorServer/bin/mythen3DetectorServerv9.1.1

slsDetectorServers/compileAllServers.sh

@@ -1,17 +1,20 @@
 # SPDX-License-Identifier: LGPL-3.0-or-other
 # Copyright (C) 2021 Contributors to the SLS Detector Package
 
-# empty branch = developer branch in updateAPIVersion.sh
-branch=""
 det_list=("ctbDetectorServer  
 	gotthardDetectorServer  
 	gotthard2DetectorServer 
 	jungfrauDetectorServer 
 	mythen3DetectorServer 
 	moenchDetectorServer
-	xilinx_ctbDetectorServer" 
+	xilinx_ctbDetectorServer"
 	)
-usage="\nUsage: compileAllServers.sh [server|all(opt)] [branch(opt)]. \n\tNo arguments mean all servers with 'developer' branch. \n\tNo 'branch' input means 'developer branch'"
+usage="\nUsage: compileAllServers.sh [server|all(opt)] [update_api(opt)]. \n\tNo arguments mean all servers with 'developer' branch. \n\tupdate_api if true updates the api to version in VERSION file"
+
+update_api=true
+target=version
+
+
 
 # arguments
 if [ $# -eq 0 ]; then
@@ -35,15 +38,12 @@ elif [ $# -eq 1 ] || [ $# -eq 2 ]; then
 		declare -a det=("${1}")
 		#echo "Compiling only $1"
 	fi
-	# branch
+
 	if [ $# -eq 2 ]; then
-		# arg in list
+		update_api=$2
-		if [[ $det_list == *$2* ]]; then
+		if not $update_api ; then
-			echo -e "Invalid argument 2: $2. $usage"
+			target=clean
-			return 1
 		fi
-		branch+=$2
-		#echo "with branch $branch"
 	fi
 else
 	echo -e "Too many arguments.$usage"
@@ -54,6 +54,9 @@ declare -a deterror=("OK" "OK" "OK" "OK" "OK" "OK")
 
 echo -e "list is ${det[@]}"
 
+if $update_api; then
+	echo "updating api to $(cat ../VERSION)"
+fi
 # compile each server
 idet=0
 for i in ${det[@]}
@@ -63,14 +66,13 @@ do
 	echo -e "Compiling $dir [$file]"
 	cd $dir
 	make clean
-	if make version API_BRANCH=$branch; then
+	if make $target; then
 		deterror[$idet]="OK"
 	else
 		deterror[$idet]="FAIL"
 	fi
 	mv bin/$dir bin/$file
 	git add -f bin/$file
-	cp bin/$file /tftpboot/
 	cd ..
 	echo -e "\n\n"
 	((++idet))

slsDetectorServers/compileAllServers_noAPIUpdate.sh

@@ -1,87 +0,0 @@
-# SPDX-License-Identifier: LGPL-3.0-or-other
-# Copyright (C) 2021 Contributors to the SLS Detector Package
-
-# empty branch = developer branch in updateAPIVersion.sh
-branch=""
-det_list=("ctbDetectorServer" 
-	"gotthardDetectorServer" 
-	"gotthard2DetectorServer"
-	"jungfrauDetectorServer"
-	"mythen3DetectorServer"
-	"moenchDetectorServer"
-	"xilinx_ctbDetectorServer"
-	)
-usage="\nUsage: compileAllServers.sh [server|all(opt)] [branch(opt)]. \n\tNo arguments mean all servers with 'developer' branch. \n\tNo 'branch' input means 'developer branch'"
-
-# arguments
-if [ $# -eq 0 ]; then
-	# no argument, all servers
-	declare -a det=${det_list[@]}
-	echo "Compiling all servers"
-elif [ $# -eq 1 ] || [ $# -eq 2 ]; then
-	# 'all' servers
-	if [[ $1 == "all" ]]; then
-		declare -a det=${det_list[@]}
-		echo "Compiling all servers"
-	else
-		# only one server
-		# arg not in list
-		if [[ $det_list != *$1* ]]; then
-			echo -e "Invalid argument 1: $1. $usage"
-			return -1
-		fi
-		declare -a det=("${1}")
-		#echo "Compiling only $1"
-	fi
-	# branch
-	if [ $# -eq 2 ]; then
-		# arg in list
-		if [[ $det_list == *$2* ]]; then
-			echo -e "Invalid argument 2: $2. $usage"
-			return -1
-		fi
-		branch+=$2
-		#echo "with branch $branch"
-	fi
-else
-	echo -e "Too many arguments.$usage"
-	return -1
-fi
-
-declare -a deterror=("OK" "OK" "OK" "OK" "OK" "OK")
-
-echo -e "list is ${det[@]}"
-
-# compile each server
-idet=0
-for i in ${det[@]}
-do
-	dir=$i
-	file="${i}_developer"
-	echo -e "Compiling $dir [$file]"
-	cd $dir
-	make clean
-	if make API_BRANCH=$branch; then
-		deterror[$idet]="OK"
-	else
-		deterror[$idet]="FAIL"
-	fi
-	mv bin/$dir bin/$file
-	git add -f bin/$file
-	cp bin/$file /tftpboot/
-	cd ..
-	echo -e "\n\n"
-	((++idet))
-done
-
-echo -e "Results:"
-idet=0
-for i in ${det[@]}
-do
-	printf "%s\t\t= %s\n" "$i" "${deterror[$idet]}"
-	((++idet))
-done
-	
-
-
-

slsDetectorServers/compileEigerServer.sh

@@ -3,21 +3,31 @@
 deterror="OK"
 dir="eigerDetectorServer"
 file="${dir}_developer"
-branch=""
+
+usage="\nUsage: compileAllServers.sh [update_api(opt)]. \n\t update_api if true updates the api to version in VERSION file"
+
+update_api=true
+target=version
 
 # arguments
 if [ $# -eq 1 ]; then
-	branch+=$1
+	update_api=$1
-	#echo "with branch $branch"
+	if not $update_api ; then
+		target=clean
+		
+	fi
 elif [ ! $# -eq 0 ]; then
-	echo -e "Only one optional argument allowed for branch."
+	echo -e "Only one optional argument allowed for update_api."
 	return -1
 fi
 
+if $update_api; then
+	echo "updating api to $(cat ../VERSION)"
+fi
+
 echo -e "Compiling $dir [$file]"
 cd $dir
-make clean
+if make $target; then
-if make version API_BRANCH=$branch; then
 	deterror="OK"
 else
 	deterror="FAIL"
@@ -25,7 +35,6 @@ fi
 
 mv bin/$dir bin/$file
 git add -f bin/$file
-cp bin/$file /tftpboot/
 cd ..
 echo -e "\n\n"
 printf "Result:\t\t= %s\n"  "${deterror}"

slsDetectorServers/ctbDetectorServer/Makefile

@@ -25,11 +25,10 @@ version: clean versioning $(PROGS)
 
 boot: $(OBJS) 
 
-version_branch=$(API_BRANCH)
 version_name=APICTB
 version_path=slsDetectorServers/ctbDetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 
 $(PROGS): $(OBJS)

slsDetectorServers/eigerDetectorServer/Makefile

@@ -25,11 +25,10 @@ version: clean versioning $(PROGS) #hv9m_blackfin_server
 	
 boot: $(OBJS) 
 
-version_branch=$(API_BRANCH)
 version_name=APIEIGER
 version_path=slsDetectorServers/eigerDetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 
 $(PROGS):  $(OBJS)

slsDetectorServers/gotthard2DetectorServer/Makefile

@@ -24,11 +24,10 @@ version: clean versioning $(PROGS)
 
 boot: $(OBJS) 
 
-version_branch=$(API_BRANCH)
 version_name=APIGOTTHARD2
 version_path=slsDetectorServers/gotthard2DetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 
 $(PROGS): $(OBJS)

slsDetectorServers/gotthardDetectorServer/Makefile

@@ -27,7 +27,7 @@ version_branch=$(API_BRANCH)
 version_name=APIGOTTHARD
 version_path=slsDetectorServers/gotthardDetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 	
 $(PROGS): $(OBJS) 

slsDetectorServers/jungfrauDetectorServer/Makefile

@@ -24,11 +24,10 @@ version: clean versioning $(PROGS)
 
 boot: $(OBJS) 
 
-version_branch=$(API_BRANCH)
 version_name=APIJUNGFRAU
 version_path=slsDetectorServers/jungfrauDetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 
 $(PROGS): $(OBJS)

slsDetectorServers/moenchDetectorServer/Makefile

@@ -24,11 +24,10 @@ version: clean versioning $(PROGS)
 
 boot: $(OBJS) 
 
-version_branch=$(API_BRANCH)
 version_name=APIMOENCH
 version_path=slsDetectorServers/moenchDetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 
 $(PROGS): $(OBJS)

slsDetectorServers/mythen3DetectorServer/Makefile

@@ -25,11 +25,10 @@ version: clean versioning $(PROGS)
 
 boot: $(OBJS) 
 
-version_branch=$(API_BRANCH)
 version_name=APIMYTHEN3
 version_path=slsDetectorServers/mythen3DetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 
 $(PROGS): $(OBJS)

slsDetectorServers/mythen3DetectorServer/mythen3.c

@@ -304,7 +304,7 @@ patternParameters *setChannelRegisterChip(int ichip, char *mask,
                 chanReg |= (0x1 << (3 + icounter));
             }
         }
-
+        chanReg /= 2;
         // deserialize
         if (chanReg & CHAN_REG_BAD_CHANNEL_MSK) {
             LOG(logINFOBLUE,

slsDetectorServers/xilinx_ctbDetectorServer/Makefile

@@ -36,11 +36,10 @@ version: clean versioning $(PROGS)
 
 boot: $(OBJS) 
 
-version_branch=$(API_BRANCH)
 version_name=APIXILINXCTB
 version_path=slsDetectorServers/xilinx_ctbDetectorServer
 versioning: 
-	cd ../../ && echo $(PWD) && echo `tput setaf 6; ./updateAPIVersion.sh $(version_name) $(version_path) $(version_branch);  tput sgr0;`
+	cd ../../ && echo $(PWD) && echo `tput setaf 6; python updateAPIVersion.py $(version_name) $(version_path);  tput sgr0;`
 
 
 $(PROGS): $(OBJS)

slsDetectorSoftware/tests/Caller/test-Caller-rx.cpp

@@ -445,23 +445,25 @@ TEST_CASE("rx_arping", "[.cmdcall][.rx]") {
     Detector det;
     Caller caller(&det);
     auto prev_val = det.getRxArping();
-    {
+    if (det.getDestinationUDPIP()[0].str() != "127.0.0.1") {
-        std::ostringstream oss;
+        {
-        caller.call("rx_arping", {"1"}, -1, PUT, oss);
+            std::ostringstream oss;
-        REQUIRE(oss.str() == "rx_arping 1\n");
+            caller.call("rx_arping", {"1"}, -1, PUT, oss);
-    }
+            REQUIRE(oss.str() == "rx_arping 1\n");
-    {
+        }
-        std::ostringstream oss;
+        {
-        caller.call("rx_arping", {}, -1, GET, oss);
+            std::ostringstream oss;
-        REQUIRE(oss.str() == "rx_arping 1\n");
+            caller.call("rx_arping", {}, -1, GET, oss);
-    }
+            REQUIRE(oss.str() == "rx_arping 1\n");
-    {
+        }
-        std::ostringstream oss;
+        {
-        caller.call("rx_arping", {"0"}, -1, PUT, oss);
+            std::ostringstream oss;
-        REQUIRE(oss.str() == "rx_arping 0\n");
+            caller.call("rx_arping", {"0"}, -1, PUT, oss);
-    }
+            REQUIRE(oss.str() == "rx_arping 0\n");
-    for (int i = 0; i != det.size(); ++i) {
+        }
-        det.setRxArping(prev_val[i], {i});
+        for (int i = 0; i != det.size(); ++i) {
+            det.setRxArping(prev_val[i], {i});
+        }
     }
 }
 

slsReceiverSoftware/src/FrameSynchronizerApp.cpp

@@ -107,11 +107,11 @@ void zmq_free(void *data, void *hint) { delete[] static_cast<char *>(data); }
 void print_frames(const PortFrameMap &frame_port_map) {
     LOG(sls::logDEBUG) << "Printing frames";
     for (const auto &it : frame_port_map) {
-        uint16_t udpPort = it.first;
+        const uint16_t udpPort = it.first;
         const auto &frame_map = it.second;
         LOG(sls::logDEBUG) << "UDP port: " << udpPort;
         for (const auto &frame : frame_map) {
-            uint64_t fnum = frame.first;
+            const uint64_t fnum = frame.first;
             const auto &msg_list = frame.second;
             LOG(sls::logDEBUG)
                 << "  acq index: " << fnum << '[' << msg_list.size() << ']';
@@ -130,30 +130,26 @@ std::set<uint64_t> get_valid_fnums(const PortFrameMap &port_frame_map) {
 
     // collect all unique frame numbers from all ports
     std::set<uint64_t> unique_fnums;
-    for (auto it = port_frame_map.begin(); it != port_frame_map.begin(); ++it) {
+    for (const auto &it : port_frame_map) {
-        const FrameMap &frame_map = it->second;
+        const FrameMap &frame_map = it.second;
-        for (auto frame = frame_map.begin(); frame != frame_map.end();
+        for (const auto &frame : frame_map) {
-             ++frame) {
+            unique_fnums.insert(frame.first);
-            unique_fnums.insert(frame->first);
         }
     }
 
     // collect valid frame numbers
     for (auto &fnum : unique_fnums) {
         bool is_valid = true;
-        for (auto it = port_frame_map.begin(); it != port_frame_map.end();
+        for (const auto &it : port_frame_map) {
-             ++it) {
+            const uint16_t port = it.first;
-            uint16_t port = it->first;
+            const FrameMap &frame_map = it.second;
-            const FrameMap &frame_map = it->second;
             auto frame = frame_map.find(fnum);
             // invalid: fnum missing in one port
             if (frame == frame_map.end()) {
                 LOG(sls::logDEBUG)
                     << "Fnum " << fnum << " is missing in port " << port;
-                // invalid: fnum greater than all in that port
+                auto upper_frame = frame_map.upper_bound(fnum);
-                auto last_frame = std::prev(frame_map.end());
+                if (upper_frame == frame_map.end()) {
-                auto last_fnum = last_frame->first;
-                if (fnum > last_fnum) {
                     LOG(sls::logDEBUG) << "And no larger fnum found. Fnum "
                                        << fnum << " is invalid.\n";
                     is_valid = false;
@@ -223,18 +219,26 @@ void Correlate(FrameStatus *stat) {
                 // sending all valid fnum data packets
                 for (const auto &fnum : valid_fnums) {
                     ZmqMsgList msg_list;
-                    PortFrameMap &port_frame_map = stat->frames;
+                    for (const auto &it : stat->frames) {
-                    for (auto it = port_frame_map.begin();
+                        const uint16_t port = it.first;
-                         it != port_frame_map.end(); ++it) {
+                        const FrameMap &frame_map = it.second;
-                        uint16_t port = it->first;
-                        const FrameMap &frame_map = it->second;
                         auto frame = frame_map.find(fnum);
                         if (frame != frame_map.end()) {
                             msg_list.insert(msg_list.end(),
                                             stat->frames[port][fnum].begin(),
                                             stat->frames[port][fnum].end());
-                            // clean up
+                        }
-                            for (zmq_msg_t *msg : stat->frames[port][fnum]) {
+                    }
+                    LOG(printHeadersLevel)
+                        << "Sending data packets for fnum " << fnum;
+                    zmq_send_multipart(socket, msg_list);
+                    // clean up
+                    for (const auto &it : stat->frames) {
+                        const uint16_t port = it.first;
+                        const FrameMap &frame_map = it.second;
+                        auto frame = frame_map.find(fnum);
+                        if (frame != frame_map.end()) {
+                            for (zmq_msg_t *msg : frame->second) {
                                 if (msg) {
                                     zmq_msg_close(msg);
                                     delete msg;
@@ -243,9 +247,6 @@ void Correlate(FrameStatus *stat) {
                             stat->frames[port].erase(fnum);
                         }
                     }
-                    LOG(printHeadersLevel)
-                        << "Sending data packets for fnum " << fnum;
-                    zmq_send_multipart(socket, msg_list);
                 }
             }
             // sending all end packets
@@ -259,6 +260,21 @@ void Correlate(FrameStatus *stat) {
                     }
                 }
                 stat->ends.clear();
+                // clean up old frames
+                for (auto &it : stat->frames) {
+                    FrameMap &frame_map = it.second;
+                    for (auto &frame : frame_map) {
+                        for (zmq_msg_t *msg : frame.second) {
+                            if (msg) {
+                                zmq_msg_close(msg);
+                                delete msg;
+                            }
+                        }
+                        frame.second.clear();
+                    }
+                    frame_map.clear();
+                }
+                stat->frames.clear();
             }
         }
     }

slsReceiverSoftware/src/MultiReceiverApp.cpp

@@ -150,8 +150,21 @@ void GetData(slsDetectorDefs::sls_receiver_header &header,
         // header->packetsMask.to_string().c_str(),
         ((uint8_t)(*((uint8_t *)(dataPointer)))), imageSize);
 
-    // if data is modified, eg ROI and size is reduced
+    // // example of how to use roi or modify data that is later written to file
-    imageSize = 26000;
+    // slsDetectorDefs::ROI roi{0, 10, 0, 20};
+    // int width = roi.xmax - roi.xmin;
+    // int height = roi.ymax - roi.ymin;
+    // uint8_t *destPtr = (uint8_t *)dataPointer;
+    // for (int irow = roi.ymin; irow < roi.ymax; ++irow) {
+    //     memcpy(destPtr,
+    //            ((uint8_t *)(dataPointer + irow * callbackHeader.shape.x +
+    //                         roi.xmin)),
+    //            width);
+    //     destPtr += width;
+    // }
+    // memcpy((uint8_t*)dataPointer, (uint8_t*)dataPointer
+    // // setting roi for eg. changes size
+    // imageSize = width * height;
 }
 
 /**

slsSupportLib/include/sls/StaticVector.h

@@ -113,10 +113,12 @@ template <typename T, size_t Capacity> class StaticVector {
     // auto begin() noexcept -> decltype(data_.begin()) { return data_.begin();
     // }
     const_iterator begin() const noexcept { return data_.begin(); }
-    iterator end() noexcept { return &data_[current_size]; }
+    iterator end() noexcept { return data_.begin() + current_size; }
-    const_iterator end() const noexcept { return &data_[current_size]; }
+    const_iterator end() const noexcept { return data_.begin() + current_size; }
     const_iterator cbegin() const noexcept { return data_.cbegin(); }
-    const_iterator cend() const noexcept { return &data_[current_size]; }
+    const_iterator cend() const noexcept {
+        return data_.cbegin() + current_size;
+    }
 
     void size_check(size_type s) const {
         if (s > Capacity) {

slsSupportLib/include/sls/Version.h

@@ -11,7 +11,7 @@ class Version {
   private:
     std::string version_;
     std::string date_;
-    const std::string defaultBranch_ = "developer";
+    inline static const std::string defaultVersion_[] = {"developer", "0.0.0"};
 
   public:
     explicit Version(const std::string &s);

slsSupportLib/include/sls/versionAPI.h

@@ -7,7 +7,7 @@
 #define APIEIGER     "9.0.0 0x241121"
 #define APICTB       "9.1.0 0x250204"
 #define APIXILINXCTB "9.1.0 0x250204"
-#define APIMYTHEN3   "9.1.0 0x250304"
 #define APIJUNGFRAU  "9.1.0 0x250318"
-#define APILIB       "9.1.0 0x250325"
+#define APILIB       "9.2.0 0x250527"
-#define APIRECEIVER  "9.1.0 0x250325"
+#define APIMYTHEN3   "9.1.1 0x250409"
+#define APIRECEIVER  "9.1.1 0x250513"

slsSupportLib/src/Version.cpp

@@ -21,7 +21,8 @@ Version::Version(const std::string &s) {
 }
 
 bool Version::hasSemanticVersioning() const {
-    return version_ != defaultBranch_;
+
+    return (version_ != defaultVersion_[0]) && (version_ != defaultVersion_[1]);
 }
 
 std::string Version::getVersion() const { return version_; }

slsSupportLib/tests/CMakeLists.txt

@@ -14,6 +14,7 @@ target_sources(tests PRIVATE
                 ${CMAKE_CURRENT_SOURCE_DIR}/test-TypeTraits.cpp
                 ${CMAKE_CURRENT_SOURCE_DIR}/test-UdpRxSocket.cpp
                 ${CMAKE_CURRENT_SOURCE_DIR}/test-logger.cpp
+                ${CMAKE_CURRENT_SOURCE_DIR}/test-Version.cpp
                 ${CMAKE_CURRENT_SOURCE_DIR}/test-ZmqSocket.cpp
                 )
 

slsSupportLib/tests/test-StaticVector.cpp

@@ -8,10 +8,10 @@
 #include <sstream>
 #include <vector>
 
-namespace sls {
+using sls::StaticVector;
 
 TEST_CASE("StaticVector is a container") {
-    REQUIRE(is_container<StaticVector<int, 7>>::value == true);
+    REQUIRE(sls::is_container<StaticVector<int, 7>>::value == true);
 }
 
 TEST_CASE("Comparing StaticVector containers") {
@@ -90,10 +90,17 @@ TEST_CASE("Copy construct from array") {
     REQUIRE(fcc == arr);
 }
 
+TEST_CASE("Construct from a smaller StaticVector") {
+    StaticVector<int, 3> sv{1, 2, 3};
+    StaticVector<int, 5> sv2{sv};
+    REQUIRE(sv == sv2);
+}
+
 TEST_CASE("Free function and method gives the same iterators") {
     StaticVector<int, 3> fcc{1, 2, 3};
     REQUIRE(std::begin(fcc) == fcc.begin());
 }
+
 SCENARIO("StaticVectors can be sized and resized", "[support]") {
 
     GIVEN("A default constructed container") {
@@ -246,23 +253,23 @@ SCENARIO("Sorting, removing and other manipulation of a container",
                 REQUIRE(a[3] == 90);
             }
         }
-        // WHEN("Sorting is done using free function for begin and end") {
+        WHEN("Sorting is done using free function for begin and end") {
-        //     std::sort(begin(a), end(a));
+            std::sort(std::begin(a), std::end(a));
-        //     THEN("it also works") {
+            THEN("it also works") {
-        //         REQUIRE(a[0] == 12);
+                REQUIRE(a[0] == 12);
-        //         REQUIRE(a[1] == 12);
+                REQUIRE(a[1] == 12);
-        //         REQUIRE(a[2] == 14);
+                REQUIRE(a[2] == 14);
-        //         REQUIRE(a[3] == 90);
+                REQUIRE(a[3] == 90);
-        //     }
+            }
-        // }
+        }
-        // WHEN("Erasing elements of a certain value") {
+        WHEN("Erasing elements of a certain value") {
-        //     a.erase(std::remove(begin(a), end(a), 12));
+            a.erase(std::remove(std::begin(a), std::end(a), 12));
-        //     THEN("all elements of that value are removed") {
+            THEN("all elements of that value are removed") {
-        //         REQUIRE(a.size() == 2);
+                REQUIRE(a.size() == 2);
-        //         REQUIRE(a[0] == 14);
+                REQUIRE(a[0] == 14);
-        //         REQUIRE(a[1] == 90);
+                REQUIRE(a[1] == 90);
-        //     }
+            }
-        // }
+        }
     }
 }
 
@@ -334,5 +341,3 @@ TEST_CASE("StaticVector stream") {
     oss << vec;
     REQUIRE(oss.str() == "[33, 85667, 2]");
 }
-
-} // namespace sls

slsSupportLib/tests/test-Version.cpp

@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: LGPL-3.0-or-other
+// Copyright (C) 2021 Contributors to the SLS Detector Package
+#include "catch.hpp"
+#include "sls/Version.h"
+
+namespace sls {
+
+TEST_CASE("check if version is semantic", "[.version]") {
+
+    auto [version_string, has_semantic_version] =
+        GENERATE(std::make_tuple("developer 0x250512", false),
+                 std::make_tuple("0.0.0 0x250512", false));
+
+    Version version(version_string);
+
+    CHECK(version.hasSemanticVersioning() == has_semantic_version);
+}
+
+} // namespace sls

tests/CMakeLists.txt

@@ -60,3 +60,5 @@ include(Catch)
 catch_discover_tests(tests)
 
 configure_file(scripts/test_simulators.py ${CMAKE_BINARY_DIR}/bin/test_simulators.py COPYONLY)
+configure_file(scripts/test_frame_synchronizer.py ${CMAKE_BINARY_DIR}/bin/test_frame_synchronizer.py COPYONLY)
+configure_file(scripts/utils_for_test.py ${CMAKE_BINARY_DIR}/bin/utils_for_test.py COPYONLY)

tests/scripts/test_frame_synchronizer.py

@@ -0,0 +1,141 @@
+# SPDX-License-Identifier: LGPL-3.0-or-other
+# Copyright (C) 2021 Contributors to the SLS Detector Package
+'''
+This file is used to start up simulators, frame synchronizer, pull sockets, acquire, test and kill them finally.
+'''
+
+import sys, time
+import traceback, json
+
+from slsdet import Detector
+from slsdet.defines import DEFAULT_TCP_RX_PORTNO
+
+from utils_for_test import (
+    Log,
+    LogLevel,
+    RuntimeException,
+    checkIfProcessRunning,
+    killProcess,
+    cleanup,
+    cleanSharedmemory,
+    startProcessInBackground,
+    startProcessInBackgroundWithLogFile,
+    checkLogForErrors,
+    startDetectorVirtualServer,
+    loadConfig,
+    ParseArguments
+)
+
+LOG_PREFIX_FNAME = '/tmp/slsFrameSynchronizer_test'
+MAIN_LOG_FNAME = LOG_PREFIX_FNAME + '_log.txt'
+PULL_SOCKET_PREFIX_FNAME = LOG_PREFIX_FNAME + '_pull_socket_'
+
+
+def startFrameSynchronizerPullSocket(name, fp):
+    fname = PULL_SOCKET_PREFIX_FNAME + name + '.txt'
+    cmd = ['python', '-u', 'frameSynchronizerPullSocket.py']  
+    startProcessInBackgroundWithLogFile(cmd, fp, fname)
+    time.sleep(1)
+    checkLogForErrors(fp, fname)
+    
+
+
+def startFrameSynchronizer(num_mods, fp):
+    cmd = ['slsFrameSynchronizer', str(DEFAULT_TCP_RX_PORTNO), str(num_mods)]
+    # in 10.0.0
+    #cmd = ['slsFrameSynchronizer', '-p', str(DEFAULT_TCP_RX_PORTNO), '-n', str(num_mods)]
+    startProcessInBackground(cmd, fp)
+    time.sleep(1)
+
+
+def acquire(fp, det):
+    Log(LogLevel.INFO, 'Acquiring')
+    Log(LogLevel.INFO, 'Acquiring', fp)
+    det.acquire()
+
+
+def testFramesCaught(name, det, num_frames):
+    fnum = det.rx_framescaught[0]
+    if fnum != num_frames:
+        raise RuntimeException(f"{name} caught only {fnum}. Expected {num_frames}") 
+    
+    Log(LogLevel.INFOGREEN, f'Frames caught test passed for {name}')
+    Log(LogLevel.INFOGREEN, f'Frames caught test passed for {name}', fp)
+
+
+def testZmqHeadetTypeCount(name, det, num_mods, num_frames, fp):
+
+    Log(LogLevel.INFO, f"Testing Zmq Header type count for {name}")
+    Log(LogLevel.INFO, f"Testing Zmq Header type count for {name}", fp)
+    htype_counts = {
+        "header": 0,
+        "series_end": 0,
+        "module": 0
+    }
+
+    try:
+        # get a count of each htype from file
+        pull_socket_fname = PULL_SOCKET_PREFIX_FNAME + name + '.txt'
+        with open(pull_socket_fname, 'r') as log_fp:
+            for line in log_fp:
+                line = line.strip()
+                if not line or not line.startswith('{'):
+                    continue
+                try:
+                    data = json.loads(line)
+                    htype = data.get("htype")
+                    if htype in htype_counts:
+                        htype_counts[htype] += 1
+                except json.JSONDecodeError:
+                    continue
+
+        # test if file contents matches expected counts
+        num_ports_per_module = 1 if name == "gotthard2" else det.numinterfaces
+        total_num_frame_parts = num_ports_per_module * num_mods * num_frames
+        for htype, expected_count in [("header", num_mods), ("series_end", num_mods), ("module", total_num_frame_parts)]:
+            if htype_counts[htype] != expected_count:
+                msg = f"Expected {expected_count} '{htype}' entries, found {htype_counts[htype]}"
+                raise RuntimeException(msg)
+    except Exception as e:
+        raise RuntimeException(f'Failed to get zmq header count type. Error:{str(e)}') from e
+        
+    Log(LogLevel.INFOGREEN, f"Zmq Header type count test passed for {name}")
+    Log(LogLevel.INFOGREEN, f"Zmq Header type count test passed for {name}", fp)
+
+
+def startTestsForAll(args, fp):
+    for server in args.servers:
+        try:
+            Log(LogLevel.INFOBLUE, f'Synchronizer Tests for {server}')
+            Log(LogLevel.INFOBLUE, f'Synchronizer Tests for {server}', fp)
+            cleanup(fp)
+            startDetectorVirtualServer(server, args.num_mods, fp)
+            startFrameSynchronizerPullSocket(server, fp)
+            startFrameSynchronizer(args.num_mods, fp)
+            d = loadConfig(name=server, rx_hostname=args.rx_hostname, settingsdir=args.settingspath, fp=fp, num_mods=args.num_mods, num_frames=args.num_frames)
+            acquire(fp, d)
+            testFramesCaught(server, d, args.num_frames)
+            testZmqHeadetTypeCount(server, d, args.num_mods, args.num_frames, fp)
+            Log(LogLevel.INFO, '\n')
+        except Exception as e:
+            raise RuntimeException(f'Synchronizer Tests failed') from e
+
+    Log(LogLevel.INFOGREEN, 'Passed all synchronizer tests for all detectors \n' + str(args.servers))
+  
+
+if __name__ == '__main__':
+    args = ParseArguments(description='Automated tests to test frame synchronizer', default_num_mods=2)
+
+    Log(LogLevel.INFOBLUE, '\nLog File: ' + MAIN_LOG_FNAME + '\n') 
+
+    with open(MAIN_LOG_FNAME, 'w') as fp:
+        try:
+            startTestsForAll(args, fp)
+            cleanup(fp)
+        except Exception as e:
+            with open(MAIN_LOG_FNAME, 'a') as fp_error:
+                traceback.print_exc(file=fp_error)
+            cleanup(fp)
+            Log(LogLevel.ERROR, f'Tests Failed.')
+
+

tests/scripts/test_simulators.py

@@ -4,250 +4,86 @@
 This file is used to start up simulators, receivers and run all the tests on them and finally kill the simulators and receivers.
 '''
 import argparse
-import os, sys, subprocess, time, colorama, signal
+import sys, subprocess, time, traceback
 
-from colorama import Fore
+from slsdet import Detector
-from slsdet import Detector, detectorType, detectorSettings
+from slsdet.defines import DEFAULT_TCP_RX_PORTNO
-from slsdet.defines import DEFAULT_TCP_CNTRL_PORTNO, DEFAULT_TCP_RX_PORTNO, DEFAULT_UDP_DST_PORTNO
-HALFMOD2_TCP_CNTRL_PORTNO=1955
-HALFMOD2_TCP_RX_PORTNO=1957
 
-colorama.init(autoreset=True)
+from utils_for_test import (
-
+    Log,
-class RuntimeException (Exception):
+    LogLevel,
-    def __init__ (self, message):
+    RuntimeException,
-        super().__init__(Fore.RED + message)
+    checkIfProcessRunning,
-    
+    killProcess,
-def Log(color, message):
+    cleanup,
-    print('\n' + color + message, flush=True)
+    cleanSharedmemory,
+    startProcessInBackground,
+    runProcessWithLogFile,
+    startDetectorVirtualServer,
+    loadConfig,
+    ParseArguments
+)
 
 
-def checkIfProcessRunning(processName):
+LOG_PREFIX_FNAME = '/tmp/slsDetectorPackage_virtual_test'
-    cmd = "ps -ef | grep " + processName
+MAIN_LOG_FNAME = LOG_PREFIX_FNAME + '_log.txt'
-    print(cmd)
+GENERAL_TESTS_LOG_FNAME = LOG_PREFIX_FNAME + '_results_general.txt'
-    res=subprocess.getoutput(cmd)
+CMD_TEST_LOG_PREFIX_FNAME = LOG_PREFIX_FNAME + '_results_cmd_'
-    print(res)
-    # eg. of output
-    #l_user  250506  243295  0 14:38 pts/5    00:00:00 /bin/sh -c ps -ef | grep slsReceiver
-    #l_user  250508  250506  0 14:38 pts/5    00:00:00 grep slsReceiver
-
-    print('how many')
-    cmd = "ps -ef | grep " + processName + " | wc -l"
-    print(cmd)
-    res=subprocess.getoutput(cmd)
-    print(res)
-
-    if res == '2':
-        return False
-    return True
 
 
-def killProcess(name):
+def startReceiver(num_mods, fp):
-    if checkIfProcessRunning(name):
+    if num_mods == 1:
-        Log(Fore.GREEN, 'killing ' + name)
+        cmd = ['slsReceiver']
-        p = subprocess.run(['killall', name])
-        if p.returncode != 0:
-            raise RuntimeException('killall failed for ' + name)
     else:
-        print('process not running : ' + name)
+        cmd = ['slsMultiReceiver', str(DEFAULT_TCP_RX_PORTNO), str(num_mods)]
+        # in 10.0.0
+        #cmd = ['slsMultiReceiver', '-p', str(DEFAULT_TCP_RX_PORTNO), '-n', str(num_mods)]
+    startProcessInBackground(cmd, fp)
+    time.sleep(1)
 
-
+def startGeneralTests(fp):
-def killAllStaleProcesses():
+    fname = GENERAL_TESTS_LOG_FNAME
-    killProcess('eigerDetectorServer_virtual')
+    cmd = ['tests', '--abort', '-s']
-    killProcess('jungfrauDetectorServer_virtual')
-    killProcess('mythen3DetectorServer_virtual')
-    killProcess('gotthard2DetectorServer_virtual')
-    killProcess('gotthardDetectorServer_virtual')
-    killProcess('ctbDetectorServer_virtual')
-    killProcess('moenchDetectorServer_virtual')
-    killProcess('xilinx_ctbDetectorServer_virtual')
-    killProcess('slsReceiver')
-    killProcess('slsMultiReceiver')
-    cleanSharedmemory()
-
-def cleanup(name):
-    '''
-    kill both servers, receivers and clean shared memory
-    '''
-    Log(Fore.GREEN, 'Cleaning up...')
-    killProcess(name + 'DetectorServer_virtual')
-    killProcess('slsReceiver')
-    killProcess('slsMultiReceiver')
-    cleanSharedmemory()
-
-def cleanSharedmemory():
-    Log(Fore.GREEN, 'Cleaning up shared memory...')
     try:
-        p = subprocess.run(['sls_detector_get', 'free'], stdout=fp, stderr=fp)
+        cleanup(fp)
-    except:
+        runProcessWithLogFile('General Tests', cmd, fp, fname)
-        Log(Fore.RED, 'Could not free shared memory')
+    except Exception as e:
-        raise
+        raise RuntimeException(f'General tests failed.') from e
-
-def startProcessInBackground(name):
-    try:
-        # in background and dont print output
-        p = subprocess.Popen(name.split(), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, restore_signals=False) 
-        Log(Fore.GREEN, 'Starting up ' + name + ' ...')
-    except:
-        Log(Fore.RED, 'Could not start ' + name)
-        raise
-
-def startServer(name):
-    
-    startProcessInBackground(name + 'DetectorServer_virtual')
-    # second half
-    if name == 'eiger':
-        startProcessInBackground(name + 'DetectorServer_virtual -p' + str(HALFMOD2_TCP_CNTRL_PORTNO))
-    tStartup = 6
-    Log(Fore.WHITE, 'Takes ' + str(tStartup) + ' seconds... Please be patient')
-    time.sleep(tStartup)
-
-def startReceiver(name):
-    startProcessInBackground('slsReceiver')
-    # second half
-    if name == 'eiger':
-        startProcessInBackground('slsReceiver -t' + str(HALFMOD2_TCP_RX_PORTNO))
-    time.sleep(2)
-
-def loadConfig(name, rx_hostname, settingsdir):
-    Log(Fore.GREEN, 'Loading config')
-    try:
-        d = Detector()
-        if name == 'eiger':
-            d.hostname = 'localhost:' + str(DEFAULT_TCP_CNTRL_PORTNO) + '+localhost:' + str(HALFMOD2_TCP_CNTRL_PORTNO)
-            #d.udp_dstport = {2: 50003} 
-            # will set up for every module
-            d.udp_dstport = DEFAULT_UDP_DST_PORTNO
-            d.udp_dstport2 = DEFAULT_UDP_DST_PORTNO + 1
-            d.rx_hostname = rx_hostname + ':' + str(DEFAULT_TCP_RX_PORTNO) + '+' + rx_hostname + ':' + str(HALFMOD2_TCP_RX_PORTNO)
-            d.udp_dstip = 'auto'
-            d.trimen = [4500, 5400, 6400]
-            d.settingspath = settingsdir + '/eiger/'
-            d.setThresholdEnergy(4500, detectorSettings.STANDARD)
-        else:
-            d.hostname = 'localhost'
-            d.rx_hostname = rx_hostname
-            d.udp_dstip = 'auto'
-            if d.type == detectorType.GOTTHARD:
-                d.udp_srcip = d.udp_dstip
-            else:
-                d.udp_srcip = 'auto'
-        if d.type == detectorType.JUNGFRAU or d.type == detectorType.MOENCH or d.type == detectorType.XILINX_CHIPTESTBOARD:
-            d.powerchip = 1
-        if d.type == detectorType.XILINX_CHIPTESTBOARD:
-            d.configureTransceiver()
-    except:
-        Log(Fore.RED, 'Could not load config for ' + name)
-        raise
-
-def startCmdTests(name, fp, fname):
-    Log(Fore.GREEN, 'Cmd Tests for ' + name)
-    cmd = 'tests --abort [.cmdcall] -s -o ' + fname
-    p = subprocess.run(cmd.split(), stdout=fp, stderr=fp, check=True, text=True)
-    p.check_returncode()
-
-    with open (fname, 'r') as f:
-        for line in f:
-            if "FAILED" in line:
-                msg = 'Cmd tests failed for ' + name + '!!!'
-                Log(Fore.RED, msg)
-                raise Exception(msg)
-
-    Log(Fore.GREEN, 'Cmd Tests successful for ' + name)
-
-def startGeneralTests(fp, fname):
-    Log(Fore.GREEN, 'General Tests')
-    cmd = 'tests --abort -s -o ' + fname
-    p = subprocess.run(cmd.split(), stdout=fp, stderr=fp, check=True, text=True)
-    p.check_returncode()
-
-    with open (fname, 'r') as f:
-        for line in f:
-            if "FAILED" in line:
-                msg = 'General tests failed !!!'
-                Log(Fore.RED, msg)
-                raise Exception(msg)
-
-    Log(Fore.GREEN, 'General Tests successful')
 
 
-
+def startCmdTestsForAll(args, fp):
-# parse cmd line for rx_hostname and settingspath using the argparse library
+    for server in args.servers:
-parser = argparse.ArgumentParser(description = 'automated tests with the virtual detector servers')
-parser.add_argument('rx_hostname', help = 'hostname/ip of the current machine')
-parser.add_argument('settingspath', help = 'Relative or absolut path to the settingspath')
-parser.add_argument('-s', '--servers', help='Detector servers to run', nargs='*')
-args = parser.parse_args()
-if args.rx_hostname == 'localhost':
-    raise RuntimeException('Cannot use localhost for rx_hostname for the tests (fails for rx_arping for eg.)')
-
-if args.servers is None:
-    servers = [
-        'eiger',
-        'jungfrau',
-        'mythen3',
-        'gotthard2',
-        'gotthard',
-        'ctb',
-        'moench',
-        'xilinx_ctb'
-    ]
-else:
-    servers = args.servers
-
-
-Log(Fore.WHITE, 'Arguments:\nrx_hostname: ' + args.rx_hostname + '\nsettingspath: \'' + args.settingspath + '\'') 
-
-
-# redirect to file
-prefix_fname = '/tmp/slsDetectorPackage_virtual_test'
-original_stdout = sys.stdout
-original_stderr = sys.stderr
-fname = prefix_fname + '_log.txt'
-Log(Fore.BLUE, '\nLog File: ' + fname) 
-
-with open(fname, 'w') as fp:
-
-    # general tests
-    file_results = prefix_fname + '_results_general.txt'
-    Log(Fore.BLUE, 'General tests (results: ' + file_results + ')')
-    sys.stdout = fp
-    sys.stderr = fp
-    Log(Fore.BLUE, 'General tests (results: ' + file_results + ')')
-    startGeneralTests(fp, file_results)
-
-    killAllStaleProcesses()
-
-    for server in servers:
         try:
-            # print to terminal for progress
+            num_mods = 2 if server == 'eiger' else 1
-            sys.stdout = original_stdout
+            fname = CMD_TEST_LOG_PREFIX_FNAME + server + '.txt'
-            sys.stderr = original_stderr
+            cmd = ['tests', '--abort', '[.cmdcall]', '-s']
-            file_results = prefix_fname + '_results_cmd_' + server + '.txt'
+
-            Log(Fore.BLUE, 'Cmd tests for ' + server + ' (results: ' + file_results + ')')
+            Log(LogLevel.INFOBLUE, f'Starting Cmd Tests for {server}')
-            sys.stdout = fp
+            cleanup(fp)
-            sys.stderr = fp
+            startDetectorVirtualServer(name=server, num_mods=num_mods, fp=fp)
-            Log(Fore.BLUE, 'Cmd tests for ' + server + ' (results: ' + file_results + ')')
+            startReceiver(num_mods, fp)
-            
+            loadConfig(name=server, rx_hostname=args.rx_hostname, settingsdir=args.settingspath, fp=fp, num_mods=num_mods)
-            # cmd tests for det
+            runProcessWithLogFile('Cmd Tests for ' + server, cmd, fp, fname)
-            cleanup(server)
+        except Exception as e:
-            startServer(server)
+            raise RuntimeException(f'Cmd Tests failed for {server}.') from e
-            startReceiver(server)
+
-            loadConfig(server, args.rx_hostname, args.settingspath)
+    Log(LogLevel.INFOGREEN, 'Passed all tests for all detectors \n' + str(args.servers))
-            startCmdTests(server, fp, file_results)
-            cleanup(server)
-        except:
-            Log(Fore.RED, 'Exception caught. Cleaning up.')
-            cleanup(server)
-            sys.stdout = original_stdout
-            sys.stderr = original_stderr
-            Log(Fore.RED, 'Cmd tests failed for ' + server + '!!!')
-            raise
 
 
-    Log(Fore.GREEN, 'Passed all tests for virtual detectors \n' + str(servers))
+if __name__ == '__main__':
+    args = ParseArguments('Automated tests with the virtual detector servers')
+    if args.num_mods > 1:
+        raise RuntimeException(f'Cannot support multiple modules at the moment (except Eiger).')
 
-# redirect to terminal
+    Log(LogLevel.INFOBLUE, '\nLog File: ' + MAIN_LOG_FNAME + '\n') 
-sys.stdout = original_stdout
+
-sys.stderr = original_stderr
+    with open(MAIN_LOG_FNAME, 'w') as fp:  
-Log(Fore.GREEN, 'Passed all tests for virtual detectors \n' + str(servers) + '\nYayyyy! :) ')
+        try:
+            startGeneralTests(fp)
+            startCmdTestsForAll(args, fp)
+            cleanup(fp)
+        except Exception as e:
+            with open(MAIN_LOG_FNAME, 'a') as fp_error:
+                traceback.print_exc(file=fp_error)
+            cleanup(fp)
+            Log(LogLevel.ERROR, f'Tests Failed.')

tests/scripts/utils_for_test.py

@@ -0,0 +1,265 @@
+# SPDX-License-Identifier: LGPL-3.0-or-other
+# Copyright (C) 2021 Contributors to the SLS Detector Package
+'''
+This file is used for common utils used for integration tests between simulators and receivers.
+'''
+
+import sys, subprocess, time, argparse
+from enum import Enum
+from colorama import Fore, Style, init
+
+from slsdet import Detector, detectorSettings
+from slsdet.defines import DEFAULT_TCP_RX_PORTNO, DEFAULT_UDP_DST_PORTNO
+SERVER_START_PORTNO=1900
+
+init(autoreset=True)
+
+
+class LogLevel(Enum):
+    INFO = 0
+    INFORED = 1
+    INFOGREEN = 2
+    INFOBLUE = 3
+    WARNING = 4
+    ERROR = 5
+    DEBUG = 6
+
+
+LOG_LABELS = {
+    LogLevel.WARNING: "WARNING: ",
+    LogLevel.ERROR: "ERROR: ",
+    LogLevel.DEBUG: "DEBUG: "
+}
+
+
+LOG_COLORS = {
+    LogLevel.INFO: Fore.WHITE,
+    LogLevel.INFORED: Fore.RED,
+    LogLevel.INFOGREEN: Fore.GREEN,
+    LogLevel.INFOBLUE: Fore.BLUE,
+    LogLevel.WARNING: Fore.YELLOW,
+    LogLevel.ERROR: Fore.RED,
+    LogLevel.DEBUG: Fore.CYAN
+}
+
+
+def Log(level: LogLevel, message: str, stream=sys.stdout):
+    color = LOG_COLORS.get(level, Fore.WHITE)
+    label = LOG_LABELS.get(level, "")
+    print(f"{color}{label}{message}{Style.RESET_ALL}", file=stream, flush=True)
+
+
+class RuntimeException (Exception):
+    def __init__ (self, message):
+        Log(LogLevel.ERROR, message)
+        super().__init__(message)
+
+
+def checkIfProcessRunning(processName):
+    cmd = f"pgrep -f {processName}"
+    res = subprocess.getoutput(cmd)
+    return res.strip().splitlines()
+
+
+def killProcess(name, fp):
+    pids = checkIfProcessRunning(name)
+    if pids:
+        Log(LogLevel.INFO, f"Killing '{name}' processes with PIDs: {', '.join(pids)}", fp)
+        for pid in pids:
+            try:
+                p = subprocess.run(['kill', pid])
+                if p.returncode != 0 and bool(checkIfProcessRunning(name)):
+                    raise RuntimeException(f"Could not kill {name} with pid {pid}")
+            except Exception as e:
+                raise RuntimeException(f"Failed to kill process {name} pid:{pid}. Error: {str(e)}") from e
+    #else:
+    #    Log(LogLevel.INFO, 'process not running : ' + name)
+
+
+def cleanSharedmemory(fp):
+    Log(LogLevel.INFO, 'Cleaning up shared memory', fp)
+    try:
+        p = subprocess.run(['sls_detector_get', 'free'], stdout=fp, stderr=fp)
+    except:
+        raise RuntimeException('Could not free shared memory')
+
+
+def cleanup(fp):
+    Log(LogLevel.INFO, 'Cleaning up')
+    Log(LogLevel.INFO, 'Cleaning up', fp)
+    killProcess('DetectorServer_virtual', fp)
+    killProcess('slsReceiver', fp)
+    killProcess('slsMultiReceiver', fp)
+    killProcess('slsFrameSynchronizer', fp)
+    killProcess('frameSynchronizerPullSocket', fp)
+    cleanSharedmemory(fp)
+
+
+def startProcessInBackground(cmd, fp):
+    Log(LogLevel.INFO, 'Starting up ' + ' '.join(cmd))
+    Log(LogLevel.INFO, 'Starting up ' + ' '.join(cmd), fp)
+    try:
+        p = subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, restore_signals=False) 
+    except Exception as e:
+        raise RuntimeException(f'Failed to start {cmd}:{str(e)}') from e
+
+
+def startProcessInBackgroundWithLogFile(cmd, fp, log_file_name: str):
+    Log(LogLevel.INFOBLUE, 'Starting up ' +  ' '.join(cmd) + '. Log: ' +  log_file_name)
+    Log(LogLevel.INFOBLUE, 'Starting up ' +  ' '.join(cmd) + '. Log: ' +  log_file_name, fp)
+    try:
+        with open(log_file_name, 'w') as log_fp:
+            subprocess.Popen(cmd, stdout=log_fp, stderr=log_fp, text=True)
+    except Exception as e:
+        raise RuntimeException(f'Failed to start {cmd}:{str(e)}') from e
+
+
+def checkLogForErrors(fp, log_file_path: str):
+    try:
+        with open(log_file_path, 'r') as log_file:
+            for line in log_file:
+                if 'Error' in line:
+                    Log(LogLevel.ERROR, f"Error found in log: {line.strip()}")
+                    Log(LogLevel.ERROR, f"Error found in log: {line.strip()}", fp)
+                    raise RuntimeException("Error found in log file")
+    except FileNotFoundError:
+        print(f"Log file not found: {log_file_path}")
+        raise
+    except Exception as e:
+        print(f"Exception while reading log: {e}")
+        raise
+
+
+def runProcessWithLogFile(name, cmd, fp, log_file_name):
+    Log(LogLevel.INFOBLUE, 'Running ' +  name + '. Log: ' +  log_file_name)
+    Log(LogLevel.INFOBLUE, 'Running ' +  name + '. Log: ' +  log_file_name, fp)
+    Log(LogLevel.INFOBLUE, 'Cmd: ' + ' '.join(cmd), fp)
+    try:
+        with open(log_file_name, 'w') as log_fp:
+            subprocess.run(cmd, stdout=log_fp, stderr=log_fp, check=True, text=True)
+    except subprocess.CalledProcessError as e:
+        pass    
+    except Exception as e:
+        Log(LogLevel.ERROR, f'Failed to run {name}:{str(e)}', fp)
+        raise RuntimeException(f'Failed to run {name}:{str(e)}')
+    
+    with open (log_file_name, 'r') as f:
+        for line in f:
+            if "FAILED" in line:
+                raise RuntimeException(f'{line}')
+
+    Log(LogLevel.INFOGREEN, name + ' successful!\n')
+    Log(LogLevel.INFOGREEN, name + ' successful!\n', fp)
+
+
+def startDetectorVirtualServer(name :str, num_mods, fp):
+    for i in range(num_mods):
+        port_no = SERVER_START_PORTNO + (i * 2)
+        cmd = [name + 'DetectorServer_virtual', '-p', str(port_no)]
+        if name == 'gotthard':
+            cmd += ['-m', '1']
+        startProcessInBackgroundWithLogFile(cmd, fp, "/tmp/virtual_det_" + name + str(i) + ".txt")
+        match name:
+            case 'jungfrau':
+                time.sleep(7)
+            case 'gotthard2':
+                time.sleep(5)
+            case _:
+                time.sleep(3)
+
+
+def connectToVirtualServers(name, num_mods):
+    try:
+        d = Detector()
+    except Exception as e:
+        raise RuntimeException(f'Could not create Detector object for {name}. Error: {str(e)}') from e
+
+    counts_sec = 5
+    while (counts_sec != 0):
+        try:
+            d.virtual = [num_mods, SERVER_START_PORTNO]
+            break
+        except Exception as e:
+            # stop server still not up, wait a bit longer
+            if "Cannot connect to" in str(e):
+                Log(LogLevel.WARNING, f'Still waiting for {name} virtual server to be up...{counts_sec}s left')
+                time.sleep(1)
+                counts_sec -= 1
+            else:
+                raise
+
+    return d
+
+
+def loadConfig(name, rx_hostname, settingsdir, fp, num_mods = 1, num_frames = 1):
+    Log(LogLevel.INFO, 'Loading config')
+    Log(LogLevel.INFO, 'Loading config', fp)
+    try:
+        d = connectToVirtualServers(name, num_mods)
+        d.udp_dstport = DEFAULT_UDP_DST_PORTNO
+        if name == 'eiger':
+            d.udp_dstport2 = DEFAULT_UDP_DST_PORTNO + 1
+
+        d.rx_hostname = rx_hostname
+        d.udp_dstip = 'auto'
+        if name != "eiger":
+            if name == "gotthard":
+                d.udp_srcip = d.udp_dstip
+            else:
+                d.udp_srcip = 'auto'
+
+        if name == "jungfrau" or name == "moench" or name == "xilinx_ctb":
+            d.powerchip = 1
+
+        if name == "xilinx_ctb":
+            d.configureTransceiver()
+
+        if name == "eiger":
+            d.trimen = [4500, 5400, 6400]
+            d.settingspath = settingsdir + '/eiger/'
+            d.setThresholdEnergy(4500, detectorSettings.STANDARD)
+
+        d.frames = num_frames
+    except Exception as e:
+        raise RuntimeException(f'Could not load config for {name}. Error: {str(e)}') from e
+    
+    return d
+
+
+def ParseArguments(description, default_num_mods=1):
+    parser = argparse.ArgumentParser(description)
+
+    parser.add_argument('rx_hostname', nargs='?', default='localhost',
+                        help='Hostname/IP of the current machine')
+    parser.add_argument('settingspath', nargs='?', default='../../settingsdir',
+                        help='Relative or absolute path to the settings directory')
+    parser.add_argument('-n', '--num-mods', nargs='?', default=default_num_mods, type=int,
+                        help='Number of modules to test with')
+    parser.add_argument('-f', '--num-frames', nargs='?', default=1, type=int,
+                        help='Number of frames to test with')
+    parser.add_argument('-s', '--servers', nargs='*',
+                        help='Detector servers to run')
+
+    args = parser.parse_args()
+
+    # Set default server list if not provided
+    if args.servers is None:
+        args.servers = [
+            'eiger',
+            'jungfrau',
+            'mythen3',
+            'gotthard2',
+            'gotthard',
+            'ctb',
+            'moench',
+            'xilinx_ctb'
+        ]
+
+    Log(LogLevel.INFO, 'Arguments:\n' + 
+        'rx_hostname: ' + args.rx_hostname + 
+        '\nsettingspath: \'' + args.settingspath + 
+        '\nservers: \'' + ' '.join(args.servers) + 
+        '\nnum_mods: \'' + str(args.num_mods) + 
+        '\nnum_frames: \'' + str(args.num_frames) + '\'') 
+
+    return args   

updateAPIVersion.py

@@ -0,0 +1,81 @@
+# SPDX-License-Identifier: LGPL-3.0-or-other
+# Copyright (C) 2025 Contributors to the SLS Detector Package
+"""
+Script to update API VERSION file based on the version in VERSION file.
+"""
+
+import argparse
+import sys
+import os
+import re
+import time
+from datetime import datetime
+
+
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
+
+API_FILE = SCRIPT_DIR + "/slsSupportLib/include/sls/versionAPI.h"
+
+VERSION_FILE = SCRIPT_DIR + "/VERSION"
+
+parser = argparse.ArgumentParser(description = 'updates API version')
+parser.add_argument('api_module_name', choices=["APILIB", "APIRECEIVER", "APICTB", "APIGOTTHARD2", "APIMOENCH", "APIEIGER", "APIXILINXCTB", "APIJUNGFRAU", "APIMYTHEN3"], help = 'module name to change api version options are: ["APILIB", "APIRECEIVER", "APICTB", "APIGOTTHARD2", "APIMOENCH", "APIEIGER", "APIXILINXCTB", "APIJUNGFRAU", "APIMYTHEN3"]')
+parser.add_argument('api_dir', help = 'Relative or absolute path to the module code')
+
+def update_api_file(new_api : str, api_module_name : str, api_file_name : str): 
+
+    regex_pattern = re.compile(rf'#define\s+{api_module_name}\s+')
+    with open(api_file_name, "r") as api_file:
+        lines = api_file.readlines()
+
+    with open(api_file_name, "w") as api_file:
+        for line in lines:
+            if regex_pattern.match(line):
+                api_file.write(f'#define {api_module_name} "{new_api}"\n')
+            else:
+                api_file.write(line)
+
+def get_latest_modification_date(directory : str):
+    latest_time = 0
+    latest_date = None
+
+    for root, dirs, files in os.walk(directory):
+        for file in files:
+            if file.endswith(".o"):
+                continue
+            full_path = os.path.join(root, file)
+            try:
+                mtime = os.path.getmtime(full_path)
+                if mtime > latest_time:
+                    latest_time = mtime
+            except FileNotFoundError:
+                continue
+    
+    latest_date = datetime.fromtimestamp(latest_time).strftime("%y%m%d")            
+
+    return latest_date
+
+
+def update_api_version(api_module_name : str, api_dir : str):
+    api_date = get_latest_modification_date(api_dir)
+    api_date = "0x"+str(api_date)
+
+    with open(VERSION_FILE, "r") as version_file: 
+        api_version = version_file.read().strip()
+
+    api_version = api_version + " " + api_date #not sure if we should give an argument option version_branch 
+
+    update_api_file(api_version, api_module_name, API_FILE)
+
+    print(f"updated {api_module_name} api version to: {api_version}")
+
+if __name__ == "__main__":
+
+    args = parser.parse_args() 
+
+    api_dir = SCRIPT_DIR + "/" + args.api_dir
+    
+
+    update_api_version(args.api_module_name, api_dir)
+
+

updateAPIVersion.sh

@@ -1,65 +0,0 @@
-# SPDX-License-Identifier: LGPL-3.0-or-other
-# Copyright (C) 2021 Contributors to the SLS Detector Package
-usage="\nUsage: updateAPIVersion.sh [API_NAME] [API_DIR] [API_BRANCH(opt)]."
-
-if [ $# -lt 2 ]; then
-    echo -e "Requires atleast 2 arguments. $usage"
-    return [-1]
-fi
-
-API_NAME=$1
-PACKAGE_DIR=$PWD
-API_DIR=$PACKAGE_DIR/$2
-API_FILE=$PACKAGE_DIR/slsSupportLib/include/sls/versionAPI.h
-CURR_DIR=$PWD
-
-if [ ! -d "$API_DIR" ]; then
-    echo "[API_DIR] does not exist. $usage"
-    return [-1]
-fi
-
-#go to directory
-cd $API_DIR
-
-#deleting line from file
-NUM=$(sed -n '/'$API_NAME' /=' $API_FILE)
-#echo $NUM
-
-
-if [ "$NUM" -gt 0 ]; then
-    sed -i ${NUM}d $API_FILE
-fi
-
-#find new API date
-API_DATE="find .  -printf \"%T@ %CY-%Cm-%Cd\n\"| sort -nr | cut -d' ' -f2- | egrep -v '(\.)o' | head -n 1"
-
-API_DATE=`eval $API_DATE`
-
-API_DATE=$(sed "s/-//g" <<< $API_DATE | awk '{print $1;}' ) 
-
-#extracting only date
-API_DATE=${API_DATE:2:6}
-
-#prefix of 0x
-API_DATE=${API_DATE/#/0x}
-echo "date="$API_DATE
-
-
-# API_VAL concatenates branch and date
-API_VAL=""
-# API branch is defined (3rd argument)
-if [ $# -eq 3 ]; then
-    API_BRANCH=$3
-    echo "branch="$API_BRANCH
-    API_VAL+="\"$API_BRANCH $API_DATE\""
-else
-    # API branch not defined (default is developer)
-    echo "branch=developer"
-    API_VAL+="\"developer $API_DATE\""
-fi
-
-#copy it to versionAPI.h
-echo "#define "$API_NAME $API_VAL >> $API_FILE
-
-#go back to original directory
-cd $CURR_DIR

updateClientAPIVersion.py

@@ -0,0 +1,34 @@
+# SPDX-License-Identifier: LGPL-3.0-or-other
+# Copyright (C) 2025 Contributors to the SLS Detector Package
+"""
+Script to update API VERSION for slsReceiverSoftware or slsDetectorSoftware 
+""" 
+
+import argparse
+import os
+
+from updateAPIVersion import update_api_version
+
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
+
+parser = argparse.ArgumentParser(description = 'updates API version')
+parser.add_argument('module_name', nargs="?", choices=["slsDetectorSoftware", "slsReceiverSoftware", "all"], default="all", help = 'module name to change api version options are: ["slsDetectorSoftware", "slsReceiverSoftware, "all"]')
+
+if __name__ == "__main__": 
+    args = parser.parse_args()
+
+    if args.module_name == "all": 
+        client_names = ["APILIB", "APIRECEIVER"]
+        client_directories = [SCRIPT_DIR+"/slsDetectorSoftware", SCRIPT_DIR+"/slsReceiverSoftware"]
+    elif args.module_name == "slsDetectorSoftware": 
+        client_names = ["APILIB"]
+        client_directories = [SCRIPT_DIR+"/slsDetectorSoftware"]
+    else: 
+        client_names = ["APIRECEIVER"]
+        client_directories = [SCRIPT_DIR+"/slsReceiverSoftware"]
+	
+    for client_name, client_directory in zip(client_names, client_directories): 
+        update_api_version(client_name, client_directory)
+		
+
+	

updateClientAPIVersion.sh

@@ -1,59 +0,0 @@
-# SPDX-License-Identifier: LGPL-3.0-or-other
-# Copyright (C) 2021 Contributors to the SLS Detector Package
-branch=""
-client_list=("slsDetectorSoftware" "slsReceiverSoftware")
-usage="\nUsage: updateClientAPI.sh [all|slsDetectorSoftware|slsReceiverSoftware] [branch]. \n\tNo arguments means all with 'developer' branch. \n\tNo 'branch' input means 'developer branch'"
-
-# arguments
-if [ $# -eq 0 ]; then
-	declare -a client=${client_list[@]}
-	echo "API Versioning all"
-elif [ $# -eq 1 ] || [ $# -eq 2 ]; then
-	# 'all' client
-	if [[ $1 == "all" ]]; then
-		declare -a client=${client_list[@]}
-		echo "API Versioning all"
-	else
-		# only one server
-		if [[ $client_list != *$1* ]]; then
-			echo -e "Invalid argument 1: $1. $usage"
-			return -1
-		fi
-		declare -a client=("${1}")
-		#echo "Versioning only $1"
-	fi
-	if [ $# -eq 2 ]; then
-		if [[ $client_list == *$2* ]]; then
-			echo -e "Invalid argument 2: $2. $usage"
-			return -1
-		fi
-		branch+=$2
-		#echo "with branch $branch"
-	fi
-else
-	echo -e "Too many arguments.$usage"
-	return -1
-fi
-
-#echo "list is: ${client[@]}"
-
-# versioning each client
-for i in ${client[@]}
-do
-	dir=$i
-	case $dir in
-		slsDetectorSoftware)
-			declare -a name=APILIB
-			;;
-		slsReceiverSoftware)
-			declare -a name=APIRECEIVER
-			;;
-		*)
-			echo -n "unknown client argument $i"
-			return -1
-			;;
-	esac
-	echo -e "Versioning $dir [$name]"
-	./updateAPIVersion.sh $name $dir $branch
-done
-

update_version.py

@@ -5,7 +5,12 @@ Script to update VERSION file with semantic versioning if provided as an argumen
 """
 
 import sys
-import re
+import os
+
+from packaging.version import Version, InvalidVersion
+
+
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
 
 def get_version():
 
@@ -14,23 +19,24 @@ def get_version():
         return "0.0.0"
 
     version = sys.argv[1]
-    
+
-    # Validate that the version argument matches semantic versioning format (X.Y.Z)
+    try:
-    if not re.match(r'^\d+\.\d+\.\d+$', version):
+        v = Version(version)  # normalizcheck if version follows PEP 440 specification
-        print("Error: Version argument must be in semantic versioning format (X.Y.Z)")
+        #replace -
+        return version.replace("-", ".")
+    except InvalidVersion as e:
+        print(f"Invalid version {version}. Version format must follow semantic versioning format of python PEP 440 version identification specification.")
         sys.exit(1)
     
-    return version
-
 
 def write_version_to_file(version):
-    with open("VERSION", "w") as version_file:
+    version_file_path = os.path.join(SCRIPT_DIR, "VERSION")
+    with open(version_file_path, "w") as version_file:
         version_file.write(version)
     print(f"Version {version} written to VERSION file.")
 
-
 # Main script
 if __name__ == "__main__":
 
     version = get_version()
-    write_version_to_file(version)
+    write_version_to_file(version)