babic_a
2bf658d30c
Set the file size in advance on GPFS
...
When started to use the buffer on GPFS we notices an increase
of metadata access times on GPFS. To try to reduce the number
of metadata updates we set the file size at creation time.
2020-07-17 12:09:46 +02:00
babic_a
83dc003c79
Add stub for jf-live-writer
...
The goal of this writer is to provide immediate writing
without delay -> following the detector.
2020-07-17 11:37:29 +02:00
babic_a
7e8bde3526
Fix tests for detector_name extraction
2020-07-17 11:36:49 +02:00
babic_a
c7185f249b
Cleanup sf_streamer
2020-07-17 11:20:50 +02:00
babic_a
465369a322
Add pulse_id stream to streamer
...
The pulse_id stream can be used to synchronize components.
For example, the writer can listen for the current pulse_id
and write with 1 second delay to a file. This will help us
eliminate the need to wait for the run to complete to start
the data write request.
2020-07-16 11:27:09 +02:00
babic_a
773320799a
Switch analysis socket to NOBLOCK
...
Since we cannot control the rate of the producer (its
whatever the detector is currently doing) we cannot afford
to block if no clients are present.
2020-07-15 10:34:54 +02:00
babic_a
6129b711ba
Change processing socket from PUB to PUSH
...
Since processing of live data can be done at 100Hz it would
be more difficult to stream the full rate with a PUB/SUB mode.
To support distributed live analysis we move to PUSH/PULL
model, since we can support only one live processing at a time
anyway (network constraints when working with large detectors).
2020-07-15 10:29:21 +02:00
eb685aeea6
adding Alvra 16M in form of 4M; JF02-stream moved away from overloaded core
2020-07-07 11:40:40 +02:00
babic_a
1fd33a0eb9
Merge branch 'master' of https://github.com/paulscherrerinstitute/sf_daq_buffer
2020-07-01 00:05:25 +02:00
c4a737557d
adding Alvra 4p5M and 16M(jf02,jf06)
2020-07-01 00:04:30 +02:00
babic_a
48c64f06d9
Fix headings
2020-06-30 16:36:18 +02:00
babic_a
4b264e96fc
Improved installation instructions
2020-06-30 16:35:39 +02:00
babic_a
54a8258685
Add simple detector name check
2020-06-30 11:06:18 +02:00
babic_a
1bab686b12
Create general group with detector_name inside
2020-06-30 10:53:29 +02:00
babic_a
aa314fe4d3
Fix import of binary buffer reader
2020-06-30 08:49:39 +02:00
babic_a
47f24ac709
Rename folder
2020-06-29 17:48:16 +02:00
babic_a
c67323e297
Moved python scripts to sf_utils
2020-06-29 17:34:52 +02:00
babic_a
8eeacbc47a
Introduce logger instead of println
2020-06-29 16:24:06 +02:00
babic_a
682489c2d2
Improve output of verification
2020-06-29 16:21:12 +02:00
babic_a
1ad7b8f382
Remove useless printout
2020-06-29 16:02:16 +02:00
babic_a
a533e27f97
Make output of file comparison tool more info
2020-06-29 15:59:41 +02:00
babic_a
b938dbbd7c
Wrong argument
2020-06-29 15:52:20 +02:00
babic_a
c40a932b74
Bytes offset should be an int
2020-06-29 15:52:05 +02:00
babic_a
01644dc8b4
Fixed conversion problems
2020-06-29 15:49:57 +02:00
babic_a
106f7ed1fd
Fix small bugs in code
2020-06-29 15:42:57 +02:00
babic_a
24be62163b
First test version of file comparison
2020-06-29 15:38:35 +02:00
babic_a
2e0dabb8c3
Fix returning numpy array
2020-06-29 15:01:12 +02:00
babic_a
29e754796a
Reshape and change the data before returning
2020-06-29 14:50:40 +02:00
babic_a
7f2e8e7136
Fix bugs and improve printout
2020-06-29 14:34:07 +02:00
babic_a
66aeb448c3
Raw buffer reader
2020-06-29 13:49:17 +02:00
babic_a
a14b30b5ac
Calculate data offset for the image buffer
2020-06-22 11:14:41 +02:00
babic_a
f5b6b2fa84
Merge remote-tracking branch 'origin/master'
2020-06-22 11:11:43 +02:00
babic_a
b8f1ead002
Add second link for hyperslab selection
2020-06-22 08:53:03 +02:00
123f18c88d
retrieve according to specified rate_multiplicator
2020-06-19 19:52:47 +02:00
babic_a
5cc0948516
Fix tests for writer
2020-06-19 16:12:09 +02:00
babic_a
0c40d0b4bc
Extract detector name from provided path
2020-06-19 16:11:58 +02:00
babic_a
ea462fc1c9
Add missing include
2020-06-19 14:37:22 +02:00
babic_a
545f433f90
Make writer step aware
2020-06-19 14:35:56 +02:00
babic_a
99cb1d55b3
Introduce tests for different steps
2020-06-19 14:35:42 +02:00
babic_a
bd50fb010e
Change pulse id step to size_t
2020-06-19 13:42:14 +02:00
babic_a
eca5c33719
Extract metadata to function
2020-06-19 13:36:44 +02:00
babic_a
69907a3412
Add 50Hz verification of metadata
2020-06-19 11:45:46 +02:00
babic_a
04b0d76edf
Adapting tests to device name
2020-06-19 11:42:06 +02:00
babic_a
23242fbb9d
Add device name to JFWriter
2020-06-19 11:41:50 +02:00
babic_a
0ab7a97143
Fix block generation logic
2020-06-19 11:33:58 +02:00
babic_a
ae1757b0cc
Fix for creating intermediate groups
2020-06-19 10:44:40 +02:00
babic_a
e730b0cab3
Add detector name to datasets to mimic the old style
2020-06-19 10:36:38 +02:00
babic_a
ed23b58332
Iteration over the specified modules instead of all
...
Left over from the development process where we used always
the 16M.
2020-06-18 13:48:49 +02:00
babic_a
68e2193beb
Reset pulse_id to remove artifacts in final file
...
Since the buffer does not get zeroed out in each iteration,
the wrong pulse_id can be written to the final file if no
packets for a specific pulse_id was received.
2020-06-17 14:22:42 +02:00
babic_a
61f06c6df6
Start improving tests for H5 writer
2020-06-16 14:32:08 +02:00
babic_a
Dmitry Ozerov