Prototype multi threaded file reader (#343)
Build on RHEL9 / build (push) Successful in 2m40s
Build on RHEL8 / build (push) Successful in 3m4s
Run tests using data on local RHEL8 / build (push) Successful in 3m51s
Build on local RHEL8 / build (push) Successful in 2m46s

- Reading files from nfs shares tops out at ~1GB/s for single threaded
reads.
- MultiThreadedFileReader uses our File wrapper to read a generic file
in parallel
- Placed in aare::experimental to show that it's not production ready
This commit is contained in:
Erik Fröjdh authored and GitHub committed 2026-08-11 15:28:54 +02:00
1 parent 533ecb8a4a
commit e26db97b5c
26 files changed
+1021 -15

No files matched your search

+6
View File
@@ -92,6 +92,12 @@ Dtype::Dtype(Dtype::TypeIndex ti) : m_type(ti) {}
*/
Dtype::Dtype(std::string_view sv) {
// NumPy uses '|' for data types whose byte order is not applicable,
// notably one-byte integer types.
if (!sv.empty() && sv.front() == '|') {
sv.remove_prefix(1);
}
// Check if the file is using our native endianess
if (auto pos = sv.find_first_of("<>"); pos != std::string_view::npos) {
const auto endianess = [](const char c) {