Commit Graph

1318 Commits

Author SHA1 Message Date
babic_a 3b9bbdf01f Add FastQueue specialization 2020-05-19 16:11:34 +02:00
babic_a 911c44bda5 Implement buffered fast queue 2020-05-19 16:11:22 +02:00
babic_a e8445c6305 Make buffer in the ImageMetadata 2020-05-19 15:27:50 +02:00
babic_a c747370e8b Too slow, does not make sense 2020-05-19 14:50:03 +02:00
babic_a 6c4a7fd6a7 Re-enable compression for a test 2020-05-19 14:47:59 +02:00
babic_a 1b9d65a734 Remove config - in scripts 2020-05-19 14:47:18 +02:00
babic_a 91369b4b40 Remove basedir again 2020-05-19 14:32:02 +02:00
babic_a a6fcde9da2 Add base dir to writer 2020-05-19 14:28:07 +02:00
babic_a b6efc4c62d Remove compression from writer 2020-05-19 13:42:36 +02:00
babic_a 158f557309 Fixed bug in writer 2020-05-19 13:38:36 +02:00
babic_a 23be52ca45 Fix tests after removing compression 2020-05-19 13:36:25 +02:00
babic_a e49a9ef60b Remove compression from WriterZmqReceiver 2020-05-19 13:34:49 +02:00
babic_a b54327a001 Remove compression and add chunking to WriterH5Writer 2020-05-19 13:30:54 +02:00
babic_a a4a76ae672 Add new parameter for writer cache 2020-05-19 13:25:15 +02:00
babic_a a7f3091d4a Rename struct variable 2020-05-19 13:08:18 +02:00
babic_a 8c0015f203 Remove compression from replay 2020-05-19 12:32:28 +02:00
babic_a 03e2e6fda4 Rename size attribute of StreamModuleFrame 2020-05-19 12:31:25 +02:00
babic_a 3b08ceed4b Rename CompressedModuleFrame into StreamModuleFrame 2020-05-19 12:26:55 +02:00
babic_a 09331e4b4c Solve buffer reference bug in reader 2020-05-19 09:38:45 +02:00
babic_a a79215a187 Close files manually in tests 2020-05-19 09:38:34 +02:00
babic_a fb105f8b1c Fix bugs in ReplayH5Reader 2020-05-19 09:30:59 +02:00
babic_a 78c3ae0262 Add buffering to replay 2020-05-19 09:28:50 +02:00
babic_a 5483fe0ea2 sf_replay should linger forever 2020-05-18 19:26:05 +02:00
babic_a 1f14e3286a Fix missing semicolon 2020-05-18 19:11:23 +02:00
babic_a d03ae9945f Substitute for for while loop 2020-05-18 19:10:50 +02:00
babic_a f54a9add50 Dont need to increase the counter twice 2020-05-18 19:04:44 +02:00
babic_a 0c52ef0c66 Add number of module that missed the pulse 2020-05-18 18:53:18 +02:00
babic_a fb3a573181 Fix core pinning numbers 2020-05-18 18:53:04 +02:00
babic_a a6d1fe2e4e Fix err msg text in sf_writer recv thread 2020-05-18 18:29:53 +02:00
babic_a 60dd3cdd8b Zmq replay should always bind 2020-05-18 18:27:32 +02:00
babic_a 5e39f65953 Replay LINGER should be 0 to wait until the writer flushes 2020-05-18 18:12:47 +02:00
babic_a c478144e96 Output the compression ration 2020-05-18 18:07:28 +02:00
babic_a 6fd1d43304 1 core per sf_replay 2020-05-18 18:04:41 +02:00
babic_a 82e4d0e5b1 2 replay per core 2020-05-18 18:00:12 +02:00
babic_a ffc6acc6af Increase number of CPU for compression 2020-05-18 17:39:51 +02:00
babic_a 14d0159f79 Update dependencies 2020-05-18 17:25:01 +02:00
babic_a 188d50cbbf Adjust compression header in sf_writer 2020-05-18 17:19:07 +02:00
babic_a aa22236dfd Moved bitshuffle LZ4 property to writer header 2020-05-18 17:01:24 +02:00
babic_a 04477ff2cf Fixing test until it works 2020-05-18 17:00:14 +02:00
babic_a e85227d015 Add compression header size to props 2020-05-18 16:59:57 +02:00
babic_a d3c2cd651f Change order of link libraries 2020-05-18 11:49:15 +02:00
babic_a b6eef3b4a1 Improve compression testing 2020-05-18 11:48:53 +02:00
babic_a da5b0d75df Register LZ4 filter and make it mandatory 2020-05-18 11:47:51 +02:00
babic_a b83be1ff5b Expose functions for making the correct bytes header 2020-05-18 11:44:56 +02:00
babic_a ed99f76e05 Fix dataset type bug 2020-05-15 19:06:53 +02:00
babic_a 831f261be1 Add logic for setting maximum chunk size 2020-05-15 18:12:46 +02:00
babic_a a6e2166900 Added additional checks 2020-05-15 17:07:00 +02:00
babic_a 86015eaa9b Fix test according to new logic 2020-05-15 17:06:45 +02:00
babic_a 744285a147 Add basic test for writer receiver 2020-05-15 17:02:59 +02:00
babic_a d2d02c3ebc ZMQ connect instead of bind 2020-05-15 17:02:46 +02:00