14 Commits

Author SHA1 Message Date
Bechir Braham
68dcfca74e
Feature/reactivate python bindings (#74)
major changes:
- add python bindings for all c++ features except network_io
- changes to cross compile on windows,linux and macos
- fix bugs with cluster_finder
- use Dtype in Frame instead of bitdepth
- remove boost::program_options and replace with our implementation 
- add Transforms class that applies a sequence of functions (c++ or
python functions) on a Frame.
- remove frame reorder and flip from SubFile.cpp. use Transforms instead
- Test clusterFinder and Pedestal results in comparison with
slsDetectorCalibration

---------

Co-authored-by: Bechir <bechir.brahem420@gmail.com>
Co-authored-by: Erik Fröjdh <erik.frojdh@gmail.com>
2024-07-04 11:51:48 +02:00
Bechir Braham
91a628cd6c
pedestal (#67)
* add config files for multimodule receiving

* read subfiles with unordered and missing frames

* save work debugging

* Revert "save work debugging"

This reverts commit e791992a05efd754f93a80c980d17397eb4b6045.

* Revert "read subfiles with unordered and missing frames"

This reverts commit 1177fd129d3690db92e9597ccda62598e5a44d41.

* throw when two frames have different frame numbers

* write single part RawFile (working beta)

* correct total number of frames in master file

* add new mythen file with syncd frames

* save work

* save work for receive multimodule
multimodule config results in too much packet loss. needs more debugging.

* setup Task Distributiosn/ parallelization programming model

* read frames with same frame number

* clang-tidy fixes, formatting, add tests

* added second receiver

* Synchronize between zmq streams and merge frames

* improve readability in loop

* fix failing tests

* add simple test for merge frames

* restructure files and use top-level header

* working pedestal + tests

* test_pedestal statistics

* QA test pedestal, fix clang-tidy errors

---------

Co-authored-by: Bechir <bechir.brahem420@gmail.com>
Co-authored-by: Erik Frojdh <erik.frojdh@gmail.com>
2024-05-08 12:33:51 +02:00
Bechir Braham
9637d0602f
receive multimodule (#65)
* add config files for multimodule receiving

* read subfiles with unordered and missing frames

* save work debugging

* Revert "save work debugging"

This reverts commit e791992a05efd754f93a80c980d17397eb4b6045.

* Revert "read subfiles with unordered and missing frames"

This reverts commit 1177fd129d3690db92e9597ccda62598e5a44d41.

* throw when two frames have different frame numbers

* write single part RawFile (working beta)

* correct total number of frames in master file

* add new mythen file with syncd frames

* save work

* save work for receive multimodule
multimodule config results in too much packet loss. needs more debugging.

* setup Task Distributiosn/ parallelization programming model

* read frames with same frame number

* clang-tidy fixes, formatting, add tests

* added second receiver

* Synchronize between zmq streams and merge frames

* improve readability in loop

* fix failing tests

* add simple test for merge frames

---------

Co-authored-by: Bechir <bechir.brahem420@gmail.com>
Co-authored-by: Erik Frojdh <erik.frojdh@gmail.com>
2024-05-07 11:06:37 +02:00
Bechir Braham
28d7e8c07a
read write cluster file (#60)
* Read and write cluster files (save work)

* add reading test

* use define for examples env variable and fix ci

* read and write cluster files (working)

* fix cluster CI
2024-04-16 13:14:41 +02:00
Bechir Braham
9dfd388927
use clang-tidy (#59)
* use clang-tidy for Frame.cpp

* fixes for defs.cpp

* clang-tidy 6/45

* clang-tidy for core

* clang-tidy fixes: for hpp File,FileInterface,SubFile.cpp

* ci fixes

* fix build errors

* fix clang-tidy command ci

* fix clang-tidy ci

* clang-tidy for rawfile.cpp

* clang-tidy numpy helpers

* fix ci

* clang-tidy file_io

* clang-tidy file_io and core working

* zmqheader

* clagn-tidy: network_io,file_io,core

* clang-tidy working

* format

---------

Co-authored-by: Bechir <bechir.brahem420@gmail.com>
2024-04-12 17:35:36 +02:00
Bechir Braham
260cb96f64
add documentation to code 2024-04-10 17:18:23 +02:00
Bechir Braham
0755de309e
working simple streamer 2024-04-09 17:30:42 +02:00
Bechir
a2499c96d6
restructure zmq socker interface
zmq socket can now return vector of frames. it knows end of transmission with header.data == 0
it can also send vector of frames
2024-04-09 11:14:21 +02:00
Bechir
598e9f6708 format files 2024-04-05 17:05:42 +02:00
Bechir
2f23e4610d fix warnings 2024-04-05 16:04:01 +02:00
Bechir
3c58039502 add tests for zmqheader and remove simdjson warnings 2024-04-05 15:22:31 +02:00
Bechir Braham
35d2b274f4
add simdjson as public for network_io
in ZmqHeader.hpp we are using templates that depend on simdjson. other libraries should also link simdjson
2024-04-04 16:35:46 +02:00
Bechir Braham
47cf462f3d
zmq sender and receiver examples 2024-04-04 15:51:18 +02:00
Bechir Braham
f772434072
add network_io folder 2024-04-03 16:40:00 +02:00