mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 66 Wiki Activity
Files
403e6e7a68a33d70f17675a1264e0faca22f85d3
Jungfraujoch/tests/FrameTransformationTest.cpp
T
leonarski_f d6389e12da
Build Packages / Unit tests (push) Skipped
Details
Build Packages / build:windows:nocuda (push) Successful in 15m31s
Details
Build Packages / build:viewer-tgz:cpu (push) Successful in 5m46s
Details
Build Packages / build:viewer-tgz:cuda (push) Successful in 6m9s
Details
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m25s
Details
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m21s
Details
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m41s
Details
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m18s
Details
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m26s
Details
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m33s
Details
Build Packages / build:rpm (rocky8) (push) Successful in 10m32s
Details
Build Packages / build:rpm (rocky9) (push) Successful in 12m23s
Details
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m50s
Details
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m12s
Details
Build Packages / DIALS test (push) Successful in 12m6s
Details
Build Packages / XDS test (durin plugin) (push) Successful in 8m15s
Details
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m12s
Details
Build Packages / XDS test (neggia plugin) (push) Successful in 5m35s
Details
Build Packages / Generate python client (push) Successful in 27s
Details
Build Packages / Build documentation (push) Successful in 54s
Details
Build Packages / Create release (push) Skipped
Details
Build Packages / build:windows:cuda (push) Successful in 12m37s
Details
v1.0.0-rc.156 (#66) 
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output.
* jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling.
* Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool.
* jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection.
* jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata.
* CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-03 19:18:56 +02:00

588 lines
28 KiB
C++
Raw Blame History

// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#include <iostream>
#include <bitshuffle/bitshuffle.h>
#include "../receiver/FrameTransformation.h"
#include "../common/RawToConvertedGeometry.h"
#include "../compression/JFJochDecompress.h"
#include <random>
using namespace std::literals::chrono_literals;
inline uint32_t read_be32(const void *ptr) {
auto ptr32 = (uint32_t *) ptr;
return __builtin_bswap32(ptr32[0]);
}
inline uint64_t read_be64(const void *ptr) {
auto ptr64 = (uint64_t *) ptr;
return __builtin_bswap64(ptr64[0]);
}
TEST_CASE("Bshuf_SSE", "[bitshuffle]") {
REQUIRE (bshuf_using_SSE2() == 1);
}
TEST_CASE("FrameTransformation_Raw_NoCompression" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 1));
experiment.DataStreams(ndatastreams);
experiment.Raw();
experiment.Compression(CompressionAlgorithm::NO_COMPRESSION);
FrameTransformation transformation(experiment);
std::mt19937 g1(1587);
std::uniform_int_distribution<int16_t> dist;
std::vector<int16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
std::vector<int16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(image.GetCompressedSize() == experiment.GetByteDepthImage() * experiment.GetPixelsNum());
auto output = (int16_t *) image.GetCompressed();
uint32_t diff_0 = 0;
uint32_t diff_1 = 0;
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++) {
if (input_0[i] != output[i]) diff_0++;
if (input_1[i] != output[i + nmodules*RAW_MODULE_SIZE]) diff_1++;
}
REQUIRE(diff_0 == 0);
REQUIRE(diff_1 == 0);
}
TEST_CASE("FrameTransformation_Raw_NoCompression_bshuf_lz4" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 1));
experiment.DataStreams(ndatastreams);
experiment.Raw();
experiment.Compression(CompressionAlgorithm::BSHUF_LZ4);
FrameTransformation transformation(experiment);
std::mt19937 g1(1587);
std::uniform_int_distribution<int16_t> dist;
std::vector<int16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
std::vector<int16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(read_be64(image.GetCompressed()) == experiment.GetPixelsNum() * experiment.GetByteDepthImage());
REQUIRE(read_be32(image.GetCompressed() + 8) == JFJochBitShuffleCompressor::BlockSize(experiment.GetCompressionAlgorithm(),
experiment.GetByteDepthImage()) *
experiment.GetByteDepthImage());
std::vector<int16_t> output;
REQUIRE_NOTHROW(JFJochDecompress(output, experiment.GetCompressionAlgorithm(), image.GetCompressed(),
image.GetCompressedSize(),
experiment.GetPixelsNum()));
uint32_t diff_0 = 0;
uint32_t diff_1 = 0;
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++) {
if (input_0[i] != output[i]) diff_0++;
if (input_1[i] != output[i + nmodules*RAW_MODULE_SIZE]) diff_1++;
}
REQUIRE(diff_0 == 0);
REQUIRE(diff_1 == 0);
}
TEST_CASE("FrameTransformation_Conversion_NoGeomTransformation_NoCompression" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 1));
experiment.DataStreams(ndatastreams);
experiment.GeometryTransformation(false);
experiment.Compression(CompressionAlgorithm::NO_COMPRESSION);
FrameTransformation transformation(experiment);
std::mt19937 g1(1587);
std::uniform_int_distribution<int16_t> dist;
std::vector<int16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
std::vector<int16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(image.GetCompressedSize() == experiment.GetByteDepthImage() * experiment.GetPixelsNum());
auto output = (int16_t *) image.GetCompressed();
uint32_t diff_0 = 0;
uint32_t diff_1 = 0;
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++) {
if (input_0[i] != output[i]) diff_0++;
if (input_1[i] != output[i + nmodules*RAW_MODULE_SIZE]) diff_1++;
}
REQUIRE(diff_0 == 0);
REQUIRE(diff_1 == 0);
}
TEST_CASE("FrameTransformation_Converted_NoCompression" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 2));
experiment.DataStreams(ndatastreams);
experiment.Compression(CompressionAlgorithm::NO_COMPRESSION);
FrameTransformation transformation(experiment);
std::mt19937 g1(1687);
std::uniform_int_distribution<int16_t> dist;
std::vector<int16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
std::vector<int16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(image.GetCompressedSize() == experiment.GetByteDepthImage() * experiment.GetPixelsNum());
auto output = (int16_t *) image.GetCompressed();
REQUIRE(input_0[511*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 0]);
REQUIRE(input_0[511*1024+256]/2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 258]);
REQUIRE(input_0[256*1024+256]/4 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 2 * 1030 * 255 + 257]);
REQUIRE(input_0[311*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 0]);
REQUIRE(input_0[311*1024+256] / 2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 258]);
REQUIRE(input_0[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030]);
REQUIRE(input_0[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030 + 800 + 6]);
REQUIRE(input_1[511*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 0]);
REQUIRE(input_1[511*1024+256] /2 == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 258]);
REQUIRE(input_1[(311+2*512)*1024] == output[200 * 1030 * 2 + 0]);
REQUIRE(input_1[(311+2*512)*1024+512] / 2 == output[200 * 1030 * 2 + 256*2+3]);
REQUIRE(input_1[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030]);
REQUIRE(input_1[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030 + 800 + 6]);
}
TEST_CASE("FrameTransformation_Converted_bshuf_lz4" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 2));
experiment.DataStreams(ndatastreams);
experiment.Compression(CompressionAlgorithm::BSHUF_LZ4);
FrameTransformation transformation(experiment);
// Predictable random number generator
std::mt19937 g1(23433);
std::uniform_int_distribution<int16_t> distribution;
std::vector<int16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = distribution(g1);
std::vector<int16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = distribution(g1);
std::vector<char> output_compressed(experiment.GetMaxCompressedSize());
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(read_be64(image.GetCompressed()) == experiment.GetPixelsNum() * experiment.GetByteDepthImage());
REQUIRE(read_be32(image.GetCompressed() + 8) == JFJochBitShuffleCompressor::BlockSize(experiment.GetCompressionAlgorithm(),
experiment.GetByteDepthImage()) *
experiment.GetByteDepthImage());
std::vector<uint16_t> output;
REQUIRE_NOTHROW(JFJochDecompress(output, experiment.GetCompressionAlgorithm(), image.GetCompressed(),
image.GetCompressedSize(),
experiment.GetPixelsNum()));
REQUIRE(input_0[511*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 0]);
REQUIRE(input_0[511*1024+256]/2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 258]);
REQUIRE(input_0[256*1024+256]/4 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 2 * 1030 * 255 + 257]);
REQUIRE(input_0[311*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 0]);
REQUIRE(input_0[311*1024+256] / 2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 258]);
REQUIRE(input_0[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030]);
REQUIRE(input_0[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030 + 800 + 6]);
REQUIRE(input_1[511*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 0]);
REQUIRE(input_1[511*1024+256] /2 == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 258]);
REQUIRE(input_1[(311+2*512)*1024] == output[200 * 1030 * 2 + 0]);
REQUIRE(input_1[(311+2*512)*1024+512] / 2 == output[200 * 1030 * 2 + 256*2+3]);
REQUIRE(input_1[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030]);
REQUIRE(input_1[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030 + 800 + 6]);
}
TEST_CASE("FrameTransformation_Converted_bshuf_zstd" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 2));
experiment.DataStreams(ndatastreams);
experiment.Compression(CompressionAlgorithm::BSHUF_ZSTD);
FrameTransformation transformation(experiment);
std::mt19937 g1(1987);
std::uniform_int_distribution<int16_t> dist;
std::vector<int16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
std::vector<int16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
std::vector<char> output_compressed(experiment.GetMaxCompressedSize());
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(read_be64(image.GetCompressed()) == experiment.GetPixelsNum() * experiment.GetByteDepthImage());
REQUIRE(read_be32(image.GetCompressed() + 8) == JFJochBitShuffleCompressor::BlockSize(experiment.GetCompressionAlgorithm(),
experiment.GetByteDepthImage()) *
experiment.GetByteDepthImage());
std::vector<uint16_t> output;
REQUIRE_NOTHROW(JFJochDecompress(output, experiment.GetCompressionAlgorithm(), image.GetCompressed(),
image.GetCompressedSize(),
experiment.GetPixelsNum()));
REQUIRE(input_0[511*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 0]);
REQUIRE(input_0[511*1024+256]/2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 258]);
REQUIRE(input_0[256*1024+256]/4 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 2 * 1030 * 255 + 257]);
REQUIRE(input_0[311*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 0]);
REQUIRE(input_0[311*1024+256] / 2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 258]);
REQUIRE(input_0[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030]);
REQUIRE(input_0[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030 + 800 + 6]);
REQUIRE(input_1[511*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 0]);
REQUIRE(input_1[511*1024+256] /2 == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 258]);
REQUIRE(input_1[(311+2*512)*1024] == output[200 * 1030 * 2 + 0]);
REQUIRE(input_1[(311+2*512)*1024+512] / 2 == output[200 * 1030 * 2 + 256*2+3]);
REQUIRE(input_1[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030]);
REQUIRE(input_1[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030 + 800 + 6]);
}
TEST_CASE("FrameTransformation_Converted_bshuf_zstd_rle" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 2));
experiment.DataStreams(ndatastreams);
experiment.Compression(CompressionAlgorithm::BSHUF_ZSTD_RLE);
FrameTransformation transformation(experiment);
std::mt19937 g1(1987);
std::uniform_int_distribution<int16_t> dist;
std::vector<int16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
std::vector<int16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
std::vector<char> output_compressed(experiment.GetMaxCompressedSize());
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(read_be64(image.GetCompressed()) == experiment.GetPixelsNum() * experiment.GetByteDepthImage());
REQUIRE(read_be32(image.GetCompressed() + 8) == JFJochBitShuffleCompressor::BlockSize(experiment.GetCompressionAlgorithm(),
experiment.GetByteDepthImage()) *
experiment.GetByteDepthImage());
std::vector<uint16_t> output;
REQUIRE_NOTHROW(JFJochDecompress(output, experiment.GetCompressionAlgorithm(), image.GetCompressed(),
image.GetCompressedSize(),
experiment.GetPixelsNum()));
REQUIRE(input_0[511*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 0]);
REQUIRE(input_0[511*1024+256]/2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 258]);
REQUIRE(input_0[256*1024+256]/4 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 2 * 1030 * 255 + 257]);
REQUIRE(input_0[311*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 0]);
REQUIRE(input_0[311*1024+256] / 2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 258]);
REQUIRE(input_0[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030]);
REQUIRE(input_0[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030 + 800 + 6]);
REQUIRE(input_1[511*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 0]);
REQUIRE(input_1[511*1024+256] /2 == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 258]);
REQUIRE(input_1[(311+2*512)*1024] == output[200 * 1030 * 2 + 0]);
REQUIRE(input_1[(311+2*512)*1024+512] / 2 == output[200 * 1030 * 2 + 256*2+3]);
REQUIRE(input_1[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030]);
REQUIRE(input_1[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030 + 800 + 6]);
}
TEST_CASE("FrameTransformation_Converted_bshuf_zstd_32bit" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 2));
experiment.DataStreams(ndatastreams);
experiment.Compression(CompressionAlgorithm::BSHUF_ZSTD).BitDepthImage(32);
FrameTransformation transformation(experiment);
std::mt19937 g1(1987);
std::uniform_int_distribution<int32_t> dist;
std::vector<int32_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
input_0[311*1024+256] = INT32_MAX-1;
input_0[311*1024+255] = INT32_MIN;
std::vector<int32_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
std::vector<char> output_compressed(experiment.GetMaxCompressedSize());
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(read_be64(image.GetCompressed()) == experiment.GetPixelsNum() * experiment.GetByteDepthImage());
REQUIRE(read_be32(image.GetCompressed() + 8) == JFJochBitShuffleCompressor::BlockSize(experiment.GetCompressionAlgorithm(),
experiment.GetByteDepthImage()) *
experiment.GetByteDepthImage());
std::vector<uint32_t> output;
REQUIRE_NOTHROW(JFJochDecompress(output, experiment.GetCompressionAlgorithm(), image.GetCompressed(),
image.GetCompressedSize(),
experiment.GetPixelsNum()));
REQUIRE(input_0[511*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 0]);
REQUIRE(input_0[511*1024+256]/2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 258]);
REQUIRE(input_0[256*1024+256]/4 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 2 * 1030 * 255 + 257]);
REQUIRE(input_0[311*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 0]);
REQUIRE(input_0[311*1024+255] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 255]);
REQUIRE(input_0[311*1024+256] / 2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 258]);
REQUIRE(input_0[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030]);
REQUIRE(input_0[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030 + 800 + 6]);
REQUIRE(input_1[511*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 0]);
REQUIRE(input_1[511*1024+256] /2 == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 258]);
REQUIRE(input_1[(311+2*512)*1024] == output[200 * 1030 * 2 + 0]);
REQUIRE(input_1[(311+2*512)*1024+512] / 2 == output[200 * 1030 * 2 + 256*2+3]);
REQUIRE(input_1[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030]);
REQUIRE(input_1[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030 + 800 + 6]);
}
TEST_CASE("FrameTransformation_Converted_bshuf_zstd_8bit" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 2));
experiment.DataStreams(ndatastreams);
experiment.Compression(CompressionAlgorithm::BSHUF_ZSTD).BitDepthImage(8);
FrameTransformation transformation(experiment);
std::mt19937 g1(1987);
std::uniform_int_distribution<int8_t> dist;
std::vector<int8_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
input_0[311*1024+256] = INT8_MAX-1;
input_0[311*1024+255] = INT8_MIN;
std::vector<int8_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
std::vector<char> output_compressed(experiment.GetMaxCompressedSize());
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(read_be64(image.GetCompressed()) == experiment.GetPixelsNum() * experiment.GetByteDepthImage());
REQUIRE(read_be32(image.GetCompressed() + 8) == JFJochBitShuffleCompressor::BlockSize(experiment.GetCompressionAlgorithm(),
experiment.GetByteDepthImage()) *
experiment.GetByteDepthImage());
std::vector<int8_t> output;
REQUIRE_NOTHROW(JFJochDecompress(output, experiment.GetCompressionAlgorithm(), image.GetCompressed(),
image.GetCompressedSize(),
experiment.GetPixelsNum()));
REQUIRE(input_0[511*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 0]);
REQUIRE(input_0[511*1024+256]/2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 258]);
REQUIRE(input_0[256*1024+256]/4 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 2 * 1030 * 255 + 257]);
REQUIRE(input_0[311*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 0]);
REQUIRE(input_0[311*1024+255] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 255]);
REQUIRE(input_0[311*1024+256] / 2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 258]);
REQUIRE(input_0[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030]);
REQUIRE(input_0[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030 + 800 + 6]);
REQUIRE(input_1[511*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 0]);
REQUIRE(input_1[511*1024+256] /2 == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 258]);
REQUIRE(input_1[(311+2*512)*1024] == output[200 * 1030 * 2 + 0]);
REQUIRE(input_1[(311+2*512)*1024+512] / 2 == output[200 * 1030 * 2 + 256*2+3]);
REQUIRE(input_1[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030]);
REQUIRE(input_1[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030 + 800 + 6]);
}
TEST_CASE("FrameTransformation_Converted_bshuf_zstd_unsigned_16bit" ,"") {
const uint16_t nmodules = 4;
const uint16_t ndatastreams = 2;
DiffractionExperiment experiment(DetJF(ndatastreams * nmodules, 2));
experiment.DataStreams(ndatastreams);
experiment.Compression(CompressionAlgorithm::BSHUF_ZSTD).PixelSigned(false);
REQUIRE(!experiment.IsPixelSigned());
REQUIRE(experiment.GetByteDepthImage() == 2);
FrameTransformation transformation(experiment);
std::mt19937 g1(1987);
std::uniform_int_distribution<uint16_t> dist(0, UINT16_MAX - 2);
std::vector<uint16_t> input_0(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_0[i] = dist(g1);
input_0[311*1024+256] = UINT16_MAX - 2;
input_0[311*1024+255] = UINT16_MAX - 1;
input_0[255*1024] = UINT16_MAX - 1;
std::vector<uint16_t> input_1(nmodules*RAW_MODULE_SIZE);
for (int i = 0; i < nmodules*RAW_MODULE_SIZE; i++)
input_1[i] = dist(g1);
std::vector<char> output_compressed(experiment.GetMaxCompressedSize());
for (int i = 0; i < nmodules; i++) {
REQUIRE_NOTHROW(transformation.ProcessModule(input_0.data() + i * RAW_MODULE_SIZE, i, 0));
REQUIRE_NOTHROW(transformation.ProcessModule(input_1.data() + i * RAW_MODULE_SIZE, i, 1));
}
auto image = transformation.GetCompressedImage();
REQUIRE(read_be64(image.GetCompressed()) == experiment.GetPixelsNum() * experiment.GetByteDepthImage());
REQUIRE(read_be32(image.GetCompressed() + 8) == JFJochBitShuffleCompressor::BlockSize(experiment.GetCompressionAlgorithm(),
experiment.GetByteDepthImage()) *
experiment.GetByteDepthImage());
std::vector<uint16_t> output;
REQUIRE_NOTHROW(JFJochDecompress(output, experiment.GetCompressionAlgorithm(), image.GetCompressed(),
image.GetCompressedSize(),
experiment.GetPixelsNum()));
REQUIRE(input_0[511*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 0]);
REQUIRE(input_0[511*1024+256]/2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 258]);
REQUIRE(input_0[256*1024+256]/4 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 2 * 1030 * 255 + 257]);
REQUIRE(input_0[311*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 0]);
REQUIRE(input_0[311*1024+255] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 255]);
REQUIRE(input_0[311*1024+255] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 256]);
REQUIRE(input_0[311*1024+256] / 2 == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 200 * 1030 * 2 + 258]);
REQUIRE(input_0[255*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 257 * 1030 * 2 + 0]);
REQUIRE(input_0[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030]);
REQUIRE(input_0[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (2 * nmodules - 2) + 1030 + 800 + 6]);
REQUIRE(input_1[511*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 0]);
REQUIRE(input_1[511*1024+256] /2 == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 258]);
REQUIRE(input_1[(311+2*512)*1024] == output[200 * 1030 * 2 + 0]);
REQUIRE(input_1[(311+2*512)*1024+512] / 2 == output[200 * 1030 * 2 + 256*2+3]);
REQUIRE(input_1[(511+512)*1024] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030]);
REQUIRE(input_1[(511+512)*1024 + 800] == output[CONVERTED_MODULE_SIZE * (nmodules - 2) + 1030 + 800 + 6]);
}
Reference in New Issue View Git Blame Copy Permalink