Shorten metadata name in buffer format

This commit is contained in:
2020-09-24 09:51:02 +02:00
parent 19a60e42c8
commit 55882bd0d1
5 changed files with 29 additions and 29 deletions
+3 -3
View File
@@ -63,7 +63,7 @@ struct ModuleFrame {
#pragma pack(1)
struct BufferBinaryFormat {
const char FORMAT_MARKER = 0xBE;
ModuleFrame metadata;
ModuleFrame meta;
char data[buffer_config::MODULE_N_BYTES];
};
#pragma pack(pop)
@@ -74,7 +74,7 @@ struct BufferBinaryFormat {
Each frame is composed by:
- **FORMAT\_MARKER** (0xBE) - a control byte to determine the validity of the frame.
- **ModuleFrame** - frame metadata used in image assembly phase.
- **ModuleFrame** - frame meta used in image assembly phase.
- **Data** - assembled frame from a single module.
Frames are written one after another to a specific offset in the file. The
@@ -139,7 +139,7 @@ and the received data.
- VerifyH5DataConsistency.py checks the consistency between the H5 file and
buffer.
- BinaryBufferReader.py reads the buffer and prints metadata. The class inside
- BinaryBufferReader.py reads the buffer and prints meta. The class inside
can also be used in external scripts.
### ZMQ sending
@@ -12,10 +12,10 @@ TEST(BinaryWriter, basic_interaction)
BufferBinaryWriter writer(module_name, detector_folder);
BufferBinaryFormat frame_data;
frame_data.metadata.pulse_id = 1;
frame_data.metadata.frame_index = 2;
frame_data.metadata.daq_rec = 3;
frame_data.metadata.n_recv_packets = 4;
frame_data.meta.pulse_id = 1;
frame_data.meta.frame_index = 2;
frame_data.meta.daq_rec = 3;
frame_data.meta.n_recv_packets = 4;
writer.write(5, &frame_data);
@@ -33,11 +33,11 @@ TEST(BinaryWriter, basic_interaction)
::read(read_fd, &read_data, sizeof(BufferBinaryFormat));
ASSERT_EQ(frame_data.FORMAT_MARKER, read_data.FORMAT_MARKER);
ASSERT_EQ(frame_data.metadata.pulse_id, read_data.metadata.pulse_id);
ASSERT_EQ(frame_data.metadata.frame_index, read_data.metadata.frame_index);
ASSERT_EQ(frame_data.metadata.daq_rec, read_data.metadata.daq_rec);
ASSERT_EQ(frame_data.metadata.n_recv_packets,
read_data.metadata.n_recv_packets);
ASSERT_EQ(frame_data.meta.pulse_id, read_data.meta.pulse_id);
ASSERT_EQ(frame_data.meta.frame_index, read_data.meta.frame_index);
ASSERT_EQ(frame_data.meta.daq_rec, read_data.meta.daq_rec);
ASSERT_EQ(frame_data.meta.n_recv_packets,
read_data.meta.n_recv_packets);
}
TEST(BinaryWriter, test_format_marker)
@@ -49,10 +49,10 @@ TEST(BinaryWriter, test_format_marker)
BufferBinaryWriter writer(module_name, detector_folder);
BufferBinaryFormat frame_data;
frame_data.metadata.pulse_id = 1;
frame_data.metadata.frame_index = 2;
frame_data.metadata.daq_rec = 3;
frame_data.metadata.n_recv_packets = 4;
frame_data.meta.pulse_id = 1;
frame_data.meta.frame_index = 2;
frame_data.meta.daq_rec = 3;
frame_data.meta.n_recv_packets = 4;
writer.write(5, &frame_data);
+11 -11
View File
@@ -6,13 +6,13 @@ per detector.
It currently has 3 output streams:
- **Full data full metadata** rate stream (send all images and metadata)
- **Reduced data full metadata** rate stream (send less images, but
all metadata)
- **Full data full meta** rate stream (send all images and meta)
- **Reduced data full meta** rate stream (send less images, but
all meta)
- **Pulse_id** stream (send only the current pulse_id)
In addition to receiving and assembling images, sf-stream also calculates
additional metadata and constructs the structures needed to send data in
additional meta and constructs the structures needed to send data in
Array 1.0 protocol.
This component does not guarantee that the streams will always contain all
@@ -103,12 +103,12 @@ arrived.
We devide the ZMQ sending to 3 types of stream:
- Data processing stream. This is basically the complete stream from
the detector with all metadata and data. It can be described as full data full
metadata stream. Only 1 client at the time can be connected to this stream
the detector with all meta and data. It can be described as full data full
meta stream. Only 1 client at the time can be connected to this stream
(PUSH/PULL for load balancing).
- Live viewing stream. This is a reduced data full metadata stream. We send
metadata for all frames, but data only for subset of them (10Hz, for example).
- Live viewing stream. This is a reduced data full meta stream. We send
meta for all frames, but data only for subset of them (10Hz, for example).
Any number of clients can connect to the 10Hz stream, because we use PUB/SUB
for this socket.
@@ -137,7 +137,7 @@ We use following fields in the JSON header:
|type|string|Value: "uint16"|
|shape|Array[uint64]|Shape of the image in stream|
### Full data full metadata stream
### Full data full meta stream
This stream runs at detector frequency and uses PUSH/PULL to distribute data
to max 1 client (this client can have many processes, but it needs to be a
@@ -151,9 +151,9 @@ for purposes of online analysis. Given the large amount of data on this
stream only "pre-approved" applications that can handle the load should be
attached here.
### Reduced data full metadata stream
### Reduced data full meta stream
This streams also runs at detector frequency for JSON headers (metadata), but
This streams also runs at detector frequency for JSON headers (meta), but
it sends only part of the images in the stream. The rest of the images are
sent as empty buffers (the receiver needs to be aware of this behaviour, as
Array 1.0 alone does not define it).
+1 -1
View File
@@ -86,7 +86,7 @@ void ImageAssembler::process(
memcpy(
&(frame_meta_buffer_[meta_offset]),
&(frame.metadata),
&(frame.meta),
sizeof(ModuleFrame));
meta_offset += meta_offset_step;
+1 -1
View File
@@ -50,7 +50,7 @@ TEST(ImageAssembler, reconstruction)
for (size_t i_module=0; i_module < n_modules; i_module++) {
for (size_t i_pulse=0; i_pulse < BUFFER_BLOCK_SIZE; i_pulse++) {
auto& frame_meta = buffer_block->frame[i_pulse].metadata;
auto& frame_meta = buffer_block->frame[i_pulse].meta;
frame_meta.pulse_id = 100 + i_pulse;
frame_meta.daq_rec = 1000 + i_pulse;