10 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	aadba5b343	Compressor: bump bitshuffle block size 4096 -> 16384 elements ... On sparse lyso frames the larger block improves compression ratio across all bshuf algorithms (16-bit data): ZSTD 8.58 -> 9.30, LZ4 7.38 -> 7.58, RLE 6.82 -> 6.90. 16384 captures most of the gain available from even larger blocks (ZSTD tops out ~9.55 at 65536) while staying close to the cache sweet spot: the cheap codecs (LZ4, RLE) peak in throughput once a block's working set fits L1d (~4096 elem here), so very large blocks trade real throughput for diminishing ratio - and that penalty is worse on the Xeon Gold/Platinum production hosts (smaller private L2, shared-L3 contention under many parallel compression threads). The block size is stored per-dataset in the bitshuffle HDF5 filter params, so existing readers (XDS/Neggia/Durin/CrystFEL) stay compatible. Move the per-block bitshuffle scratch off the inline member array onto a lazily-sized heap vector, like tmp_space, so the block size no longer bloats every stack-allocated compressor (incl. the transient ones in CBORStream2Serializer). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-26 20:59:55 +02:00
leonarski_fandClaude Opus 4.8	c332e45a54	Compressor: make Compress size-aware, drop frames that don't fit ... JFJochBitShuffleCompressor::Compress now takes a dest_size and returns a negative value when the compressed output would not fit, instead of writing past the destination buffer. The check is lazy: before each block it verifies the remaining space still covers that block's worst case (mirrored by the new MaxCompressedBlockSize helper, consistent with MaxCompressedSize so a dest sized to MaxCompressedSize never fails). On overflow the dest content is undefined - no rescue. The receiver uses this to compress directly into the writer buffer slot and drop just the oversized frame instead of pre-reserving the full worst-case image size next to the per-image CBOR metadata. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-26 20:10:52 +02:00
leonarski_f	28d224afab	version 1.0.0-rc.25	2024-11-22 21:25:20 +01:00
leonarski_f	91fd44bff7	Improve release/versioning of Jungfraujoch repository	2024-05-15 11:29:01 +02:00
leonarski_f	f1935526a7	Generalized serializer	2023-12-11 06:49:24 +01:00
leonarski_f	16bbf54f2a	Remove open source license (for now)	2023-09-15 10:47:21 +02:00
leonarski_f	767dc8e1b3	JFJochCompressor: Use Kal's bitshuffle compressor	2023-08-04 16:03:45 +02:00
leonarski_f	9144058f0c	JFJochCompressor: Use Kal's bitshuffle compressor	2023-08-04 15:47:32 +02:00
leonarski_f	25359f0d60	CBOR: Adjust for DECTRIS stream2 compatibility: ... * date/time tag * initial tag self-described CBOR * series ID * fixes to typed array header in case of compression	2023-04-19 12:58:29 +02:00
leonarski_f	1757d42182	Initial commit ... Signed-off-by: Filip Leonarski <filip.leonarski@psi.ch>	2023-04-06 11:17:59 +02:00