* added simulator tests in github workflows * indentation error * typo * debug * Logging for debugging * added more debug lines * more debugging * debug * debug * debug * dont throw if process does not exist * debug * added absolute path to sls_detector commands * some refactoring in test scripts * added absolute path to all slsdet command * typo * ../tests/scripts/test_frame_synchronizer.py * raise exception upon failure for github workflows * removed hidden tags * some refactoring in test scripts * some refactoring * fixed CMakeLists * fixed unsuccesful merge * updated python tests using simulators * debug import error * debug module import * python -m runs module pytest as script - everything in path available * removed integartion tests * enable file write not to log files * run tests without log files * increased sleep time for udp packets * added logg level variable to cmake * added testing policies to documenattion * disabled check for num_frames for jungfrau & xilinx * set log level as cmake cached variable * disable tests for jungfrau and xilinx_ctb * check frames for HDF5 * updated Documentation of Testing * changed withdetectorsimulators to detectorintegration * replaced [.cmdcall] with [.detectorintegration] * check_file_size only disabled for jungfrau - disable for all roi tests * changed time to wait after receive to 5 ms * take into account half modules of eiger * num udp interfaces needs to be consistent across modules * suppressed warning enclosing if * config added 2 udp ports per default for moench and jungfrau * write detector output to console * allow jungfrau to tests num frames, remove unused variable (numinterfaces), add comment for future to handle traceback to know which calling function threw the files unmatched, added documentation for tests (examples for .detectoritnegration and how to disable marked tests, removed addditional argumetns to disable for test_simulator as one can just use ~, removed the check that checks for jungfrau checking number of frames at master attributes and at rx test, removed unused advanced_test_settings in test_simulator script, the num_mods check for multiple modules is removed and default num modules set to 1 for test_simulator (to be increased later), back to raising exception for killprocess * removed integration tests from cmakelists.txt and cmk.sh, modified the tests workflow command to reflect the disable argument and removed xilinx_ctb from test (fix fromdeveloper merge to be done) * filtering by actual name for disable certain tests on github workflow * minor refactor * wip * wip * changes to run on local rh9 runner instead of github workfloa * modified yml to remove some leftover from github workflow * test * fix build_dir in scripts (github workflow) and pytest dir in gitea workflow * making the local machine use python3.13 binary * pythonpath added * changes for build_Dir back * allowing ctb api tests * allowed ctb api tests and set up slsdetname envt variable for shared memory being reserved just for these tests * added rh8 workflow for local runner on gitea * remnants from rh9 local runner * remnants from rh9 local runner * conda env for all shell for local runner * allowing hdf5 to build on local runner * run all tests for both the runners * refactored fixtures a bit and merged some tests that use one session for entire server * test fail * test fix * adding github workflow to test without data file checks and without logs * documentation changes * unnecessary import in conftest * allowing the session_simulator to test for multiple modules and interfaces etc * allow test_simulator script to run for 2 modules for all modules except ctb and xilinx ctb * run upon push * removing the disable file check on github workflow * minor adjustment * testing without synch * reverting to previous * with log file * without the space * summary from file and more error extracts from file to terminal * minor * trying nlf for more details * updated with no log file to print everything to screen also for det and rxr * trying a no throw * stoi was more about indent in yaml * tries * wip * debug * number of frames inconsistent fix=>just take first one, only test xilinx * jungfrau tests without frames caught check * extend the disable file check to everywhere that creates files * specify path for test_simulator * withoutprinting == * wip * back with printing===, but not parsing file for errors anymore * lang? * wip * safe log? * wip2 * wip * dont split error as its streaming live, just raise * with log files * lang? * last resort * wip * test no det with general tests * show tests live * also include hidden integration tests * without extra summary? * revert * last resort again * tsquash on int64_t? * tsquash on int64_t? mroe print * writing to /tmp? * all tests * might be the fix? * write to file * fixed a few quiet mode no log file tests * work on any branch for github tests, work on also release candidates for gitea tests * added frame synchronizer tests to github workflow * moved tests to run_tests.yaml from cmake.yaml * documentation * disabled general tests --------- Co-authored-by: Dhanya Thattil <dhanya.thattil@psi.ch>
Dependencies
Before building from source make sure that you have the dependencies installed. If installing using conda, conda will manage the dependencies. Avoid also installing dependency packages with pip.
Documentaion
Detailed documentation including installation can be found in the software wiki.
List of releases can be found on the official site.
Firmware compatiblity can be found in firmware page
Installation
The slsDetectorPackage provides core detector software implemented in C++, along with Python bindings packaged as the slsdet Python extension module. Choose the option that best fits your environment and use case.
-
Install pre-built binaries using conda (Recommended): Install pre-built binaries for the C++ client, receiver, GUI and the Python API (
slsdet), simplifying setup across platforms. -
Pip: Install only the Python extension module, either by downloading the pre-built library from PyPI or by building the extension locally from source. Available only from v9.2.0 onwards.
-
Build from source: Compile the entire package yourself, including both the C++ core and the Python bindings, for maximum control and customization. However, make sure that you have the dependencies installed. If installing using conda, conda will manage the dependencies. Avoid installing packages with pip and conda simultaneously.
1. Install pre-built binaries using conda (Recommended)
Conda is not only useful to manage python environments but can also be used as a user space package manager. Dates in the tag (for eg. 2020.07.23.dev0) are from the developer branch. Please use released tags for stability.
We have three different packages available:
- slsdetlib Shared libraries and command line utilities
- slsdetgui GUI
- slsdet Python bindings
- moenchzmq Moench
#Add channels for dependencies and our library
conda config --add channels conda-forge
conda config --add channels slsdetectorgroup
conda config --set channel_priority strict
#create and activate an environment with our library
#replace 6.1.1 with the required tag
conda create -n myenv slsdetlib=6.1.1
conda activate myenv
#ready to use
sls_detector_get exptime
etc ...
# List available versions
# lib and binaries
conda search slsdetlib
# python
conda search slsdet
# gui
conda search slsdetgui
# moench
conda search moenchzmq
2. Pip
The Python extension module slsdet can be installed using pip. This is available from v9.2.0 onwards.
#Install the Python extension module from PyPI
pip install slsdet
# or install the python extension locally from source
git clone https://github.com/slsdetectorgroup/slsDetectorPackage.git --branch 9.2.0
cd slsDetectorPackage
pip install .
3. Build from source
3.1. Download Source Code from github
git clone https://github.com/slsdetectorgroup/slsDetectorPackage.git --branch 7.0.0
Note: For v6.x.x of slsDetectorPackage and older, refer pybind11 notes on cloning.
3.2. Build from source
One can either build using cmake or use the in-built cmk.sh script.
3.2.1. Build using CMake
# outside slsDetecorPackage folder
mkdir build && cd build
# configure & generate Makefiles using cmake
# by listing all your options (alternately use ccmake described below)
# cmake3 for some systems
cmake ../slsDetectorPackage -DCMAKE_INSTALL_PREFIX=/your/install/path
# compiled to the build/bin directory
make -j12 #or whatever number of cores you are using to build
# install headers and libs in /your/install/path directory
make install
Instead of the cmake command, one can use ccmake to get a list of options to configure and generate Makefiles at ease.
# ccmake3 for some systems
ccmake ..
# choose the options
# first press [c] - configure (unil you see [g])
# then press [g] - generate
| Example cmake options | Comment |
|---|---|
| -DSLS_USE_PYTHON=ON | Python |
| -DPython_FIND_VIRTUALENV=ONLY | Python from only the conda env |
| -DSLS_USE_GUI=ON | GUI |
| -DSLS_USE_HDF5=ON | HDF5 |
| -DSLS_USE_SIMULATOR=ON | Simulator |
Note: For v7.x.x of slsDetectorPackage and older, refer zeromq notes for cmake option to hint library location.
3.2.2. Build using in-built cmk.sh script
The binaries are generated in slsDetectorPackage/build/bin directory.
Usage: $0 [-b] [-c] [-d <HDF5 directory>] [-e] [-g] [-h] [-i]
[-j <Number of threads>] [-k <CMake command>] [-l <Install directory>]
[-m] [-n] [-p] [-r] [-s] [-t] [-u] [-z]
-[no option]: only make
-b: Builds/Rebuilds CMake files normal mode
-c: Clean
-d: HDF5 Custom Directory
-e: Debug mode
-g: Build/Rebuilds gui
-h: Builds/Rebuilds Cmake files with HDF5 package
-i: Builds tests
-j: Number of threads to compile through
-k: CMake command
-l: Install directory
-m: Manuals
-n: Manuals without compiling doxygen (only rst)
-p: Builds/Rebuilds Python API
-r: Build/Rebuilds only receiver
-s: Simulator
-t: Build/Rebuilds only text client
-u: Chip Test Gui
-z: Moench zmq processor
# display all options
./cmk.sh -?
# new build and compile in parallel (recommended basic option):
./cmk.sh -cbj5
# new build, python and compile in parallel:
./cmk.sh -cbpj5
#For rebuilding only certain sections
./cmk.sh -tg #only text client and gui
./cmk.sh -r #only receiver
Note: For v7.x.x of slsDetectorPackage and older, refer zeromq notes for cmk script option to hint library location.
3.3. Build on old distributions using conda
If your linux distribution doesn't come with a C++17 compiler (gcc>8) then it's possible to install a newer gcc using conda and build the slsDetectorPackage using this compiler
#Create an environment with the dependencies
conda create -n myenv gxx_linux-64 cmake zmq
conda activate myenv
# outside slsDetecorPackage folder
mkdir build && cd build
cmake ../slsDetectorPackage -DCMAKE_PREFIX_PATH=$CONDA_PREFIX
make -j12
Note: For v7.x.x of slsDetectorPackage and older, refer zeromq notes for dependencies for conda.
3.4. Build slsDetectorGui (Qt5)
- Using pre-built binary on conda
conda create -n myenv slsdetgui=7.0.0
conda activate myenv
- Using system installation on RHEL7
yum install qt5-qtbase-devel.x86_64
yum install qt5-qtsvg-devel.x86_64
- Using system installation on RHEL8
yum install qt5-qtbase-devel.x86_64
yum install qt5-qtsvg-devel.x86_64
yum install expat-devel.x86_64
- Using conda
#Add channels for dependencies and our library
conda config --add channels conda-forge
conda config --add channels slsdetectorgroup
conda config --set channel_priority strict
# create environment to compile
# on rhel7
conda create -n slsgui zeromq gxx_linux-64 gxx_linux-64 mesa-libgl-devel-cos6-x86_64 qt
# on fedora or newer systems
conda create -n slsgui zeromq qt
# when using conda compilers, would also need libgl, but no need for it on fedora unless maybe using it with ROOT
# activate environment
conda activate slsgui
# compile with cmake outside slsDetecorPackage folder
mkdir build && cd build
cmake ../slsDetectorPackage -DSLS_USE_GUI=ON
make -j12
# or compile with cmk.sh
cd slsDetectorPackage
./cmk.sh -cbgj9
Note: For v7.x.x of slsDetectorPackage and older, refer zeromq notes for dependencies for conda.
3.5. Build documentation from package
The documentation for the slsDetectorPackage is build using a combination of Doxygen, Sphinx and Breathe. The easiest way to install the dependencies is to use conda
conda create -n myenv python=3.12 sphinx sphinx_rtd_theme breathe doxygen numpy
# using cmake or ccmake to enable DSLS_BUILD_DOCS
# outside slsDetecorPackage folder
mkdir build && cd build
cmake ../slsDetectorPackage -DSLS_BUILD_DOCS=ON
make docs # generate API docs and build Sphinx RST
make rst # rst only, saves time in case the API did not change
4. Pybind and Zeromq
Pybind11 for Python
v8.0.0+: pybind11 is built
- by default from tar file in repo (libs/pybind/v2.1x.0.tar.gz)
- or use advanced option SLS_FETCH_PYBIND11_FROM_GITHUB [link].
- v9.0.0+: pybind11 (v2.13.6)
- v8.x.x : pybind11 (v2.11.0)
v7.x.x: pybind11 packaged into ‘libs/pybind’. No longer a submodule. No need for “recursive” or “submodule update”.
Older versions: pybind11 is a submodule. Must be cloned using “recursive” and updated when switching between versions using the following commands.
# Note: Only for v6.x.x versions and older
# clone using recursive to get pybind11 submodule
git clone --recursive https://github.com/slsdetectorgroup/slsDetectorPackage.git
# update submodule when switching between releases
cd slsDetectorPackage
git submodule update --init
Zeromq
v8.0.0+: zeromq (v4.3.4) is built
- by default from tar file in repo (libs/libzmq/libzmq-4.3.4.tar.gz)
- or use advanced option SLS_FETCH_ZMQ_FROM_GITHUB [link].
v7.x.x and older: zeromq-devel must be installed and one can hint its location using
- cmake option:’-DZeroMQ_HINT=/usr/lib64’ or
- option ‘-q’ in cmk.sh script: : ./cmk.sh -cbj5 -q /usr/lib64
- ‘zeromq’ dependency added when installing using conda
Support
dhanya.thattil@psi.ch
erik.frojdh@psi.ch
alice.mazzoleni@psi.ch