* added simulator tests in github workflows
* indentation error
* typo
* debug
* Logging for debugging
* added more debug lines
* more debugging
* debug
* debug
* debug
* dont throw if process does not exist
* debug
* added absolute path to sls_detector commands
* some refactoring in test scripts
* added absolute path to all slsdet command
* typo
* ../tests/scripts/test_frame_synchronizer.py
* raise exception upon failure for github workflows
* removed hidden tags
* some refactoring in test scripts
* some refactoring
* fixed CMakeLists
* fixed unsuccesful merge
* updated python tests using simulators
* debug import error
* debug module import
* python -m runs module pytest as script - everything in path available
* removed integartion tests
* enable file write not to log files
* run tests without log files
* increased sleep time for udp packets
* added logg level variable to cmake
* added testing policies to documenattion
* disabled check for num_frames for jungfrau & xilinx
* set log level as cmake cached variable
* disable tests for jungfrau and xilinx_ctb
* check frames for HDF5
* updated Documentation of Testing
* changed withdetectorsimulators to detectorintegration
* replaced [.cmdcall] with [.detectorintegration]
* check_file_size only disabled for jungfrau - disable for all roi tests
* changed time to wait after receive to 5 ms
* take into account half modules of eiger
* num udp interfaces needs to be consistent across modules
* suppressed warning enclosing if
* config added 2 udp ports per default for moench and jungfrau
* write detector output to console
* allow jungfrau to tests num frames, remove unused variable (numinterfaces), add comment for future to handle traceback to know which calling function threw the files unmatched, added documentation for tests (examples for .detectoritnegration and how to disable marked tests, removed addditional argumetns to disable for test_simulator as one can just use ~, removed the check that checks for jungfrau checking number of frames at master attributes and at rx test, removed unused advanced_test_settings in test_simulator script, the num_mods check for multiple modules is removed and default num modules set to 1 for test_simulator (to be increased later), back to raising exception for killprocess
* removed integration tests from cmakelists.txt and cmk.sh, modified the tests workflow command to reflect the disable argument and removed xilinx_ctb from test (fix fromdeveloper merge to be done)
* filtering by actual name for disable certain tests on github workflow
* minor refactor
* wip
* wip
* changes to run on local rh9 runner instead of github workfloa
* modified yml to remove some leftover from github workflow
* test
* fix build_dir in scripts (github workflow) and pytest dir in gitea workflow
* making the local machine use python3.13 binary
* pythonpath added
* changes for build_Dir back
* allowing ctb api tests
* allowed ctb api tests and set up slsdetname envt variable for shared memory being reserved just for these tests
* added rh8 workflow for local runner on gitea
* remnants from rh9 local runner
* remnants from rh9 local runner
* conda env for all shell for local runner
* allowing hdf5 to build on local runner
* run all tests for both the runners
* refactored fixtures a bit and merged some tests that use one session for entire server
* test fail
* test fix
* adding github workflow to test without data file checks and without logs
* documentation changes
* unnecessary import in conftest
* allowing the session_simulator to test for multiple modules and interfaces etc
* allow test_simulator script to run for 2 modules for all modules except ctb and xilinx ctb
* run upon push
* removing the disable file check on github workflow
* minor adjustment
* testing without synch
* reverting to previous
* with log file
* without the space
* summary from file and more error extracts from file to terminal
* minor
* trying nlf for more details
* updated with no log file to print everything to screen also for det and rxr
* trying a no throw
* stoi was more about indent in yaml
* tries
* wip
* debug
* number of frames inconsistent fix=>just take first one, only test xilinx
* jungfrau tests without frames caught check
* extend the disable file check to everywhere that creates files
* specify path for test_simulator
* withoutprinting ==
* wip
* back with printing===, but not parsing file for errors anymore
* lang?
* wip
* safe log?
* wip2
* wip
* dont split error as its streaming live, just raise
* with log files
* lang?
* last resort
* wip
* test no det with general tests
* show tests live
* also include hidden integration tests
* without extra summary?
* revert
* last resort again
* tsquash on int64_t?
* tsquash on int64_t? mroe print
* writing to /tmp?
* all tests
* might be the fix?
* write to file
* fixed a few quiet mode no log file tests
* work on any branch for github tests, work on also release candidates for gitea tests
* added frame synchronizer tests to github workflow
* moved tests to run_tests.yaml from cmake.yaml
* documentation
* disabled general tests
---------
Co-authored-by: Dhanya Thattil <dhanya.thattil@psi.ch>
readout speed added to json and h5 master files.
Also fixed master file inconsistencies
Sserver binaries
- update server binaries because readoutspeed needs to be sent to receiver with rx_hostname command
API
- added const to Detector class set/getburstmode
Python
- updated python bindings (burstmode const and roi arguments)
Cmd generation
- added pragma once in Caller.in.h as Caller is included in test files
m3: num channels due to #counters < 3
* workaround for m3 for messed up num channels (client always assumes all counters enabled and adds them to num channels), fix for hdf5
g2: exptime master file inconsistency
- exptime didnt match because of round of when setting burst mode (sets to a different clk divider)
- so updating actual time for all timers (exptime, period, subexptime etc, ) in Module class, get timer values from detector when setting it and then send to receiver to write in master file
ctb image size incorrect:
- write actual size into master file and not the reserved size (digital reduces depending on dbit list and dbit offset)
- added a calculate ctb image size free function in generalData.h that is used there as well as for the tests.
master file inconsistencies
- refactored master attributes writing using templates
- names changed to keep it consistent between json and hdf5 master file (Version, Pixels, Exposure Times, GateDelays, Acquisition Period, etc.)
- datatypes changed to keep it simple where possible: imageSize, dynamicRange, tengiga, quad, readnrows, analog, analogsamples, digital, digitalsamples, dbitreorder, dbitoffset, transceivermask, transeiver, transceiversamples, countermask, gates =>int
- replacing "toString" with arrays, objects etc for eg for scan, rois, etc.
- json header always written (empty dataset or empty brackets)
- hdf5 needs const char* so have to convert strings to it, but taking care that strings exist prior to push_back
- master attributes (redundant string literals->error prone
tests for master file
- suppressed deprecated functions in rapidjson warnings just for the tests
- added slsREceiverSoftware/src to allow access to receiver_defs.h to test binary/hdf5 version
- refactored acquire tests by moving all the acquire tests from individual detector type files to a single one=test-Caller-acquire.cpp
- set some default settings (loadBasicSettings) for a basic acquire at load config part for the test_simulator python scripts. so minimum number of settings for detector to be set for any acquire tests.
- added tests to test master files for json and hdf5= test-Caller-master-attributes.cpp
- added option to add '-m' markers for tests using test_simulator python script
* slsSupportLib done, at receiver rooting out in implementation
* removed from receiver and client
* removed everywhere except gui, python and client(commands.yaml and Detector.h)
* updated python
* fixed autocomplete to print what the issue is if there is one with ToString when running the autocomplete script to generate fixed.json. updated readme.md in generator folder
* formatting
* removed enums for dacs
* udpating autocomplete and generating commands
* removed gotthard from docs and release notes
* removed dac test
* bug from removing g1
* fixed virtual test for xilinx, was minor. so in this PR
* gui done
* binary in merge fix
* formatting and removing enums
* updated fixed and dump.json
* bash autocomplete
* updated doc on command line generation
* removing increments in dac enums for backward compatibility. Not required
* removed ROI from rxParameters (only in g1), not needed to be backward compatible
* removed the phase shift option from det server staruip
* skeleton of structure for callbacks and funcitons updated
* updated callback header structures and implemented in receiver
* fixed bugs
* minor
* formatting
* wip: draft of frame synchronizer, semaphores not done yet
* signal handler not affecting semaphore inside lambda function
* finally works with sem
* install targets cmake error fix
* removed modified callback and instead passing by reference instead of value to the oriignal receiver data callback
* reducing the number of data call backs. incoming from developer
* added json header to receiver start acquiistion call back
* WIP: of synchronisation (#969)
* WIP of synchronisation
* working so far if everything goes right
* added all information into json headers
* valid json
* allow frame synchronizer to have access to static libzmq when compiling on conda (libzeromq-devel not installed by default
* upto date with multirecieverapp for invalid arguments and help
* formatting
* remove warnings
* changes to print
* removed prints
* no need for print frames to be called
* minor
* commnet
* adding json header in start callback, imagesize in data callback and formatted
* startcallback returns an unused int (changed to exceptions and forgotten in last modification to callbacks likely)
* fixed sanitizer issues. 1 left for ctrl+C
- zmq_msg_t should be deleted, not freed. Same with the char arrays and semaphores.
* fixed sanitizer issues and made it more readable
* moving clearing old frames to new startacq just in case it has to process soem frames before the callback
* fix cherry-pick merge of fixing sanitizer thread issues but has start callbacks signature change.fixed
---------
Co-authored-by: Felix Engelmann <felix-github@nlogn.org>
* docs receiver formats rewrite
* added documentation for all the receiver files, updated release notes, udpated help in commands help for timing, fixed by throwing exception for aa dividy by 0 error caused by not freeing memory (detsize) when switching between 1d and 2d detectors, removed unnecessary 'recevier up' printout, fixed dbit list 64 bit mask error in master json file (was not 64 bit before), fixed bug in reading gotthard1 data (needs to be tested)
* generating commands help and formatting, also fix help for trimen command line
* added ctb frame format documentation, added some links to some commands, added documentation about adding expat-devel in installation for rhel8 gui, fixed some indentation issues that screwed up command line help documentation
* added ctb frame format documentation
* updated documentation about zeromq-devel for <8.0.0 versions
* changed rxr callback signatures to all include structs
* removed datamodify call back as size can be changed in the original data call back now
* bringing some parameters (set functions) to dataProcessor class for its callback (namely udpport, quad, fliprows, totalframes, jsonheader), resulting in also removing totalframes from 2 other function signatures
* updated MultiReceiverApp to reflect the new callback signatures
* enable ipv6 in zmq socket
* removed rx_zmqip API and field in gui, changed client updaterxrzip to updateclientzmqip to have the rx_hostname ip if 0.
* updated command line for rx_zmqip to give a warning.
* Replaced 'depreciated' to 'deprecated' everywhere
* switching from * to 0.0.0.0 works for rebinding zmq sockets
* fixed help in command line for rx_zmqip * to 0.0.0.0 and removed cmd in python
* remove publisher zmq socket ip also for moench post processing
* fixed tests
* publisher zmq ip macros to be reused
* period and exptime(patternwaittime level 0)
* added new regsieterdefs and updated api version and fixedpattern reg
* autogenerate commands
* formatting
* minor
* wip resetflow, readout mode, transceiver mask, transceiver enable
* acquisition, but streaming done bit and busy (exposing + read chip to fifo) not known yet from fw
* programming fpga and device tree done
* most configuration done, need to connect configuretransceiver to client
* stuck at resetting transciever timed out
* minor
* fixed virtual, added chip busyto fifo, streaming busy, set/getnext framenumber
* configuretransceiver from client, added help in client
* make formatt and command generation
* tests for xilinx ctb works
* command generation
* dacs added and tested, power not done
* power added
* added temp_fpga
* binaries in
* ctrlreg is 0 to enable chip=fixed, high dac val = min val= fixed, power regulators in weird order=fixed, device tree could be loaded with dacs before adcs=fixed
* start works
* virtual server sends
* receiver works
* tests
* python function and enum generation, commands generatorn and autocomplete, formatting, tests
* tests fail at start(transceiver not aligned)
* tests passed
* all binaries compiled
* eiger binary in
* added --nomodule cehck for xilinx
* merge fix from 7.0.2: new jungfrau fw versions, incremented binary, hdf5 and json versions
* moench: changed dac names and default values to old moench values
* moench: remove interface clk polarity at start up
* moench: default speed is half speed, default values for adc offset and adc phase for different speeds (only half speed confirmed), adc vref voltage to 2.0 like G1
* moench: connected adc pipeline to client
* moench: receiver- default frames per file is 100k and discard partial frames as default
* moench binary in
* using tostring in gui for dacs
* moved frame discard policy as a parameter to be configured with a default depending on detector
* moench: 300 degrees for adc phase in full speed
* transceiverenable, tsamples, romode for tranceiver and digital_transceiver
* 202 spec instr only for transceiver mode
* removed check for empty in trans readout and clean memory before reading from fifo
* ctb read fifo strobe for all after reading all channels, adding 1us after selecting channel, changing fw date
* updated 10gb transceiver enable
----
* added transceiver (tsamples, romode(transceiver, digital_transceiver), transceiverenable (mask)
* clean memory before reading from fifo (for analog and digital as well)
* read fifo then read strobe (also corresp fw) fixes number of reads (also for analg and digital)-> increases all pipelines by 1
* fixed bug in rearranging digital data in receiver
* fixed bug in streaming size of data after rearranging
* fixed bug in setbit, clearbit,and getbit
* status checks fifo before returning idle (transmitting if data in fifo if transceiver more enabled)
* soem matterhorn specifics that will need to be put into pattern in a month or two. this is temporary.
* NOTE: breaking api. rxParameters struct has transceiverenabel and tsamples given from det to receiver
adding rx_roi also in the zmq header for external guis to put the "yellow box".. sending full roi instead of -1, and sending for each zmq port. "(multiple yellow boxes)".
* eiger: hardwareversion, fix firmware version unable to read version scenarios, check to see if febl, febr and beb have same fw version
* feb versions can be picked up only after feb initialization
* eiger: adding mask to read/write registers. useful for setting quad parameters as they might have different values for left and right fpga registers.
** fix quad position
* fix quad flipping
* formatting
* first tries with a process intead of thread for rx_arping
* Moving delete pointer of udp socket to stopReciever,so rx_arping can only be set when udp socket is closed
* refactoring and formatting
* unused variable processId
* ignore sigchild to prevent zombie from child processes being killed
* copied jungfrau server to moench and adapted
* fixed image size and num packets
* read n rows allows 16
* commneted out configure_asic_timer at server startup. To be removed later the ASIC_CTRL_REG and storage cell options
* moench:removing the decrement (which was in jf) in read n rows to register
* removed lblsamples from gui
* gui message doesnt show if it has a '>' symbol in error msg
* minor refactoring for readability (size_t calc fifo size)
* refactoring listening udp socket code: activated and datastream dont create udp sockets anyway, rc<=- should be discarded in any case
* wip
* refactoring memory structure access
* wip: bugfix write header + data to binary
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* portRoi no roi effecto on progress
* fail at receiver progress, wip
* segfaults for char pointer in struct
* reference to header to get header and data
* refactoring
* use const defined for size of header of fifo
* updated release notes
* remove pointer in callback for sls_receiver_header pointer
* rx same name arguments in constructors
* rx: same name arguments in constructor
* rx: removing the '_' suffix in class data members
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* diff undo for clang later
* wip
* Wip
* const string&
* gui message doesnt show if it has a '>' symbol in error msg
* minor refactoring for readability (size_t calc fifo size)
* refactoring listening udp socket code: activated and datastream dont create udp sockets anyway, rc<=- should be discarded in any case
* wip
* refactoring memory structure access
* wip: bugfix write header + data to binary
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* portRoi no roi effecto on progress
* fail at receiver progress, wip
* segfaults for char pointer in struct
* reference to header to get header and data
* refactoring
* use const defined for size of header of fifo
* updated release notes
* remove pointer in callback for sls_receiver_header pointer
* rx same name arguments in constructors
* rx: same name arguments in constructor
* rx: removing the '_' suffix in class data members
* merge fix
* merge fix
* review fix refactoring
* gui message doesnt show if it has a '>' symbol in error msg
* minor refactoring for readability (size_t calc fifo size)
* refactoring listening udp socket code: activated and datastream dont create udp sockets anyway, rc<=- should be discarded in any case
* wip
* refactoring memory structure access
* wip: bugfix write header + data to binary
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* portRoi no roi effecto on progress
* fail at receiver progress, wip
* segfaults for char pointer in struct
* reference to header to get header and data
* refactoring
* use const defined for size of header of fifo
* updated release notes
* remove pointer in callback for sls_receiver_header pointer
* passing reference header for callback instead of copying it
