leonarski_fandClaude Opus 4.8
7e7a73062c
New CompressionAlgorithm that emits a standard Zstandard frame: zero/0xFF runs
become RLE_Blocks (like BSHUF_ZSTD_RLE) and literal regions become
Compressed_Blocks with per-block adaptive Huffman literals and no sequences
(Number_of_Sequences=0). Short runs are absorbed into the literal stream;
incompressible literals fall back to Raw_Blocks so the worst case stays within
ZSTD_compressBound.
The Huffman tree + bitstream are produced by zstd's own HUF_compress{1,4}X_repeat
(the same calls ZSTD_compressLiterals uses); only the frame/block/literals-section
framing is hand-written, with comments citing zstd_compression_format.md so it can
be checked clause by clause. Output decodes with stock ZSTD_decompress, so no
reader changes are needed (decode routes like BSHUF_ZSTD).
On sparse diffraction this gives ~12% smaller files than bitshuffle/LZ4 at about
the same end-to-end speed, sitting between LZ4 and full ZSTD; for maximum ratio
use BSHUF_ZSTD. Robust on any input: tests round-trip pure zeros, Poisson(10),
Mersenne-Twister noise (checked against the size bound), an extreme-sparsity mask,
and a real lyso image through stock ZSTD_decompress.
API: exposed as "bszstd_rlehuf"; regenerate the Python/TS clients (update_version.sh)
to surface the new value there.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
22 lines
925 B
C++
22 lines
925 B
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include <bitshuffle/bitshuffle.h>
|
|
|
|
#include "JFJochCompressor.h"
|
|
#include "MaxCompressedSize.h"
|
|
|
|
int64_t MaxCompressedSize(CompressionAlgorithm algorithm, int64_t pixels_number, uint16_t pixel_depth) {
|
|
switch (algorithm) {
|
|
case CompressionAlgorithm::BSHUF_LZ4:
|
|
return bshuf_compress_lz4_bound(pixels_number, pixel_depth, JFJochBitShuffleCompressor::BlockSize(algorithm, pixel_depth)) + 12;
|
|
case CompressionAlgorithm::BSHUF_ZSTD:
|
|
case CompressionAlgorithm::BSHUF_ZSTD_RLE:
|
|
case CompressionAlgorithm::BSHUF_ZSTD_RLE_HUFF:
|
|
return bshuf_compress_zstd_bound(pixels_number, pixel_depth, JFJochBitShuffleCompressor::BlockSize(algorithm, pixel_depth)) + 12;
|
|
default:
|
|
return pixels_number * pixel_depth;
|
|
}
|
|
}
|
|
|