compression: verify the bitshuffle seam for Apple Silicon; SIMD test knows NEON
The x86/ARM split asked for on macOS already exists: BitShuffleBlock.h (fbc507839) sends
the per-block transform to bitshuffle_hperf everywhere except aarch64 with NEON, where it
uses the classic bitshuffle's NEON path, and both call sites go through it. Nothing new is
needed for Apple Silicon - clang --target=arm64-apple-macos defines __aarch64__ and
__ARM_NEON exactly as aarch64 Linux does, both vendored files compile to arm64 Mach-O
objects, the seam resolves to bshuf_(un)trans_bit_elem and bshuf_using_NEON() is 1. The
header now says why the switch is a preprocessor one: a universal build compiles it once
per architecture, which a CMake-time answer cannot follow.
What had never happened is the NEON code actually running. It has now, under
qemu-aarch64 in the project's cross image: 504 blocks (elem 1/2/4/8, 8 elements up to the
128 kB block, odd multiples of 8, random and detector-like data) plus 54 whole-buffer
bitshuffle/LZ4 streams with element counts that are no multiple of 8 or of the block. The
output is byte-identical (same md5) across aarch64 NEON, aarch64 without NEON (seam falls
back to hperf's portable code), x86 hperf AVX2 via ifunc, hperf SSE2, hperf portable
fallback, and classic SSE2 / AVX2 / scalar; every implementation decodes every other.
Throughput, one thread on a loaded Zen 3, 16/32-bit, GB/s encode/decode: hperf AVX2
~9-12/~10-11, classic AVX2 ~8/~5.5-8, classic SSE2 ~4-5/~4.5-5.5, hperf portable fallback
auto-vectorised ~2.8/~5, not vectorised ~1/~1.8. Classic 128-bit SIMD beating hperf's
fallback is the x86 stand-in for the choice the seam makes on ARM; no ARM hardware was
available, so the NEON-vs-fallback ranking on a real core is still unmeasured.
The one thing that would have failed on aarch64 is the Bshuf_SSE test, which required
SSE2 outright. It now accepts NEON as well, which also makes it the check that a Mac or
DGX Spark build did not end up on the scalar path.
Not built: no CMake configure or jfjoch_test build was run on this machine (busy); the
test expression was compiled and run standalone on x86 only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 5957b15493dfd0cc8a18fb780a18c2573a6eccde)
This commit is contained in:
@@ -22,8 +22,10 @@ inline uint64_t read_be64(const void *ptr) {
|
||||
return __builtin_bswap64(ptr64[0]);
|
||||
}
|
||||
|
||||
TEST_CASE("Bshuf_SSE", "[bitshuffle]") {
|
||||
REQUIRE (bshuf_using_SSE2() == 1);
|
||||
// The classic bitshuffle runs behind the HDF5 filter everywhere, and on aarch64 it is also the one
|
||||
// BitShuffleBlock.h picks, so a build that left it on its scalar path must not go unnoticed.
|
||||
TEST_CASE("Bshuf_SIMD", "[bitshuffle]") {
|
||||
REQUIRE ((bshuf_using_SSE2() == 1 || bshuf_using_NEON() == 1));
|
||||
}
|
||||
|
||||
TEST_CASE("FrameTransformation_Raw_NoCompression" ,"") {
|
||||
|
||||
Reference in New Issue
Block a user