leonarski_f
54c0100e8e
Build Packages / Unit tests (push) Successful in 1h28m28s
Build Packages / build:windows:nocuda (push) Successful in 14m45s
Build Packages / build:windows:cuda (push) Successful in 13m13s
Build Packages / build:viewer-tgz:cpu (push) Successful in 6m47s
Build Packages / build:viewer-tgz:cuda (push) Successful in 7m22s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m52s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m16s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m19s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m50s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m40s
Build Packages / build:rpm (rocky8) (push) Successful in 11m18s
Build Packages / build:rpm (rocky9) (push) Successful in 12m4s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m55s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m22s
Build Packages / DIALS test (push) Successful in 13m37s
Build Packages / XDS test (durin plugin) (push) Successful in 8m47s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m4s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m45s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Build documentation (push) Successful in 1m4s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m16s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Rebrand the offline data-processing subsystem as `rugnux` and consolidate all offline analysis into the single `rugnux` binary - `jfjoch_process` is now `rugnux`, the former `jfjoch_azint` is now `rugnux --azint-only`, and `jfjoch_scale` is now `rugnux --scale` (see the new docs/NAMING.md and docs/RUGNUX.md). Scaling and merging are on by default for rotation and stills (`--no-merge` disables them), replacing the previous opt-in `-M, --scale-merge`. * rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling. * rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics. * rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below <I/sigma> = 1. * rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group. * rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix. * jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation. * jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning. * jfjoch_broker: Drop uncollected placeholder rows (number = -1) from the scan_result REST endpoint. * jfjoch_broker: Fix the inverted per-image compression ratio reported by the Lite receiver (was compressed/uncompressed instead of uncompressed/compressed). * jfjoch_broker: Bragg integration adds a quantization-noise variance floor with a box-sum fallback, and treats the type-maximum marker as an invalid pixel for unsigned image types. * jfjoch_writer: Detect file-overwrite conflicts at start for back-channel transports, and reset the writer when end-of-collection finalisation fails. * jfjoch_viewer: Preview overlays follow the geometry (resolution/ROI arcs, true beam centre, predictions, coral secondary-lattice spots, legend), add save-as-JPEG, and fix an HTTP live-follow memory leak. * Frontend: Improved aesthetics and usability, and added in-browser pixel-mask and JUNGFRAU-pedestal visualisation. * CI: Name the Windows installer jfjoch-viewer-* instead of jfjoch-*.Reviewed-on: #67 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2731 lines
102 KiB
C++
2731 lines
102 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include <catch2/catch_all.hpp>
|
|
|
|
#include "../common/DiffractionExperiment.h"
|
|
#include "../common/ScanResultGenerator.h"
|
|
#include "../writer/FileWriter.h"
|
|
#include "../reader/JFJochHDF5Reader.h"
|
|
#include "../compression/JFJochCompressor.h"
|
|
|
|
TEST_CASE("HDF5DataType_Sign","[HDF5]") {
|
|
HDF5DataType type_u8((uint8_t)0), type_fl(0.0f), type_i32((int32_t) 0), type_u32((uint32_t) 0);
|
|
|
|
CHECK(!type_u8.IsSigned());
|
|
CHECK(type_fl.IsSigned());
|
|
CHECK(type_i32.IsSigned());
|
|
CHECK(!type_u32.IsSigned());
|
|
}
|
|
|
|
TEST_CASE("HDF5DataType_ElemSize","[HDF5]") {
|
|
HDF5DataType type_u8((uint8_t)0), type_fl(0.0f), type_i32((int32_t) 0), type_u32((uint32_t) 0);
|
|
|
|
CHECK(type_u8.GetElemSize() == 1);
|
|
CHECK(type_fl.GetElemSize() == 4);
|
|
CHECK(type_i32.GetElemSize() == 4);
|
|
CHECK(type_u32.GetElemSize() == 4);
|
|
}
|
|
|
|
TEST_CASE("HDF5DataType_ElemType","[HDF5]") {
|
|
HDF5DataType type_u8((uint8_t)0), type_fl(0.0f), type_i32((int32_t) 0), type_u32((uint32_t) 0);
|
|
|
|
CHECK(type_u8.IsInteger());
|
|
CHECK(!type_fl.IsInteger());
|
|
CHECK(type_fl.IsFloat());
|
|
CHECK(type_i32.IsInteger());
|
|
CHECK(type_u32.IsInteger());
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_MasterFile", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
|
|
x.FilePrefix("test08").ImagesPerTrigger(950).OverwriteExistingFiles(true);
|
|
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
|
.IncidentEnergy_keV(WVL_1A_IN_KEV)
|
|
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10))
|
|
.SetUnitCell(UnitCell{.a= 10, .b= 20, .c= 30, .alpha= 90, .beta= 101, .gamma = 90});
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 0;
|
|
std::unique_ptr<NXmx> master = std::make_unique<NXmx>(start_message);
|
|
master->Finalize(end_message);
|
|
master.reset();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test08_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetBeamX_pxl() == Catch::Approx(x.GetBeamX_pxl()));
|
|
CHECK(dataset->experiment.GetBeamY_pxl() == Catch::Approx(x.GetBeamY_pxl()));
|
|
CHECK(dataset->experiment.GetDetectorDistance_mm() == Catch::Approx(x.GetDetectorDistance_mm()));
|
|
CHECK(dataset->experiment.GetFrameTime() == x.GetFrameTime());
|
|
CHECK(dataset->experiment.GetFrameCountTime() == x.GetFrameCountTime());
|
|
CHECK(dataset->experiment.GetWavelength_A() == Catch::Approx(x.GetWavelength_A()));
|
|
CHECK(dataset->experiment.GetImageNum() == 0);
|
|
REQUIRE(dataset->experiment.GetUnitCell().has_value());
|
|
CHECK(dataset->experiment.GetUnitCell()->b == 20.0);
|
|
CHECK(dataset->experiment.GetUnitCell()->beta == 101.0);
|
|
CHECK(dataset->calibration_data.empty());
|
|
}
|
|
remove("test08_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_MasterFile_Calibration", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test_reader_calibration").ImagesPerTrigger(1).OverwriteExistingFiles(true);
|
|
RegisterHDF5Filter();
|
|
|
|
std::vector<uint16_t> calib_1(200*300, 10);
|
|
std::vector<int32_t> calib_2(100*400, 55);
|
|
std::vector<float> calib_f(100*400, 1234.56f);
|
|
JFJochBitShuffleCompressor compressor(CompressionAlgorithm::BSHUF_LZ4);
|
|
auto calib_1_compressed = compressor.Compress(calib_1);
|
|
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
|
|
CompressedImage calibration_01(calib_1, 200, 300);
|
|
CompressedImage calibration_02(calib_2, 100, 400);
|
|
CompressedImage calibration_f(calib_f, 100, 400);
|
|
CompressedImage calibration_01_lz4(
|
|
calib_1_compressed.data(), calib_1_compressed.size(),
|
|
200, 300, CompressedImageMode::Uint16, CompressionAlgorithm::BSHUF_LZ4
|
|
);
|
|
|
|
calibration_01.Channel("c1");
|
|
calibration_02.Channel("c2");
|
|
calibration_f.Channel("cf");
|
|
calibration_01_lz4.Channel("c1_lz4");
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 0;
|
|
std::unique_ptr<NXmx> master = std::make_unique<NXmx>(start_message);
|
|
master->WriteCalibration(calibration_01);
|
|
master->WriteCalibration(calibration_01_lz4);
|
|
master->WriteCalibration(calibration_02);
|
|
master->WriteCalibration(calibration_f);
|
|
master->Finalize(end_message);
|
|
master.reset();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_reader_calibration_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
REQUIRE(dataset->calibration_data.size() == 4);
|
|
CHECK(dataset->calibration_data[0] == "c1");
|
|
CHECK(dataset->calibration_data[1] == "c1_lz4");
|
|
CHECK(dataset->calibration_data[2] == "c2");
|
|
CHECK(dataset->calibration_data[3] == "cf");
|
|
|
|
std::vector<uint8_t> buffer;
|
|
std::vector<uint8_t> buff_2;
|
|
REQUIRE_THROWS(reader.ReadCalibration(buffer, "c3"));
|
|
CompressedImage test;
|
|
REQUIRE_NOTHROW(test = reader.ReadCalibration(buffer, "c1"));
|
|
CHECK(test.GetByteDepth() == 2);
|
|
CHECK(test.GetHeight() == 300);
|
|
CHECK(test.GetWidth() == 200);
|
|
CHECK(test.GetMode() == CompressedImageMode::Uint16);
|
|
CHECK(reinterpret_cast<const uint16_t *>(test.GetUncompressedPtr(buff_2))[76] == 10);
|
|
|
|
REQUIRE_NOTHROW(test = reader.ReadCalibration(buffer, "c1_lz4"));
|
|
CHECK(test.GetByteDepth() == 2);
|
|
CHECK(test.GetHeight() == 300);
|
|
CHECK(test.GetWidth() == 200);
|
|
CHECK(test.GetMode() == CompressedImageMode::Uint16);
|
|
CHECK(reinterpret_cast<const uint16_t *>(test.GetUncompressedPtr(buff_2))[76] == 10);
|
|
|
|
REQUIRE_NOTHROW(test = reader.ReadCalibration(buffer, "c2"));
|
|
CHECK(test.GetByteDepth() == 4);
|
|
CHECK(test.GetHeight() == 400);
|
|
CHECK(test.GetWidth() == 100);
|
|
CHECK(test.GetMode() == CompressedImageMode::Int32);
|
|
CHECK(reinterpret_cast<const int32_t *>(test.GetUncompressedPtr(buff_2))[76] == 55);
|
|
|
|
REQUIRE_NOTHROW(test = reader.ReadCalibration(buffer, "cf"));
|
|
CHECK(test.GetByteDepth() == 4);
|
|
CHECK(test.GetHeight() == 400);
|
|
CHECK(test.GetWidth() == 100);
|
|
CHECK(test.GetMode() == CompressedImageMode::Float32);
|
|
CHECK(reinterpret_cast<const float *>(test.GetUncompressedPtr(buff_2))[76] == Catch::Approx(1234.56f));
|
|
}
|
|
remove("test_reader_calibration_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_DefaultExperiment", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test_def").OverwriteExistingFiles(true);
|
|
RegisterHDF5Filter();
|
|
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 0;
|
|
std::unique_ptr<NXmx> master = std::make_unique<NXmx>(start_message);
|
|
master->Finalize(end_message);
|
|
master.reset();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
DiffractionExperiment x1;
|
|
IndexingSettings is;
|
|
is.FFT_NumVectors(1024);
|
|
x1.ImportIndexingSettings(is);
|
|
reader.Experiment(x1);
|
|
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_def_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
REQUIRE(x1.GetIndexingSettings().GetFFT_NumVectors() == 1024);
|
|
}
|
|
remove("test_def_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_PixelMask", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test16").ImagesPerTrigger(950).OverwriteExistingFiles(true);
|
|
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
|
.IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
|
|
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
|
RegisterHDF5Filter();
|
|
|
|
std::vector<uint32_t> pixel_mask(x.GetPixelsNum(), 0);
|
|
pixel_mask[5767] = 1;
|
|
pixel_mask[x.GetPixelsNum() - 1] = 4;
|
|
pixel_mask[0] = 256;
|
|
|
|
ScanResultGenerator generator(x);
|
|
|
|
std::vector<uint16_t> image(x.GetPixelsNum(), 0);
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
start_message.pixel_mask["default"] = pixel_mask;
|
|
|
|
FileWriter file_set(start_message);
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 0;
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
REQUIRE_NOTHROW(generator.Add(message));
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 1;
|
|
generator.FillEndMessage(end_message);
|
|
|
|
file_set.WriteHDF5(end_message);
|
|
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
reader.ReadFile("test16_master.h5");
|
|
auto dataset = reader.GetDataset();
|
|
REQUIRE(dataset->pixel_mask->GetMask().size() == x.GetPixelsNum());
|
|
CHECK(dataset->pixel_mask->GetMask() == pixel_mask);
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(0));
|
|
REQUIRE(reader_image);
|
|
|
|
CHECK(reader_image->Image().at(5767) == GAP_PXL_VALUE);
|
|
CHECK(reader_image->Image().at(0) == ERROR_PXL_VALUE);
|
|
CHECK(reader_image->Image().at(1) == 0);
|
|
CHECK(reader_image->Image().at(2) == 0);
|
|
CHECK(reader_image->Image().at(x.GetPixelsNum() - 1) == ERROR_PXL_VALUE);
|
|
}
|
|
remove("test16_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_ROIDefinitions", "[HDF5][Full]") {
|
|
RegisterHDF5Filter();
|
|
|
|
// ROI definitions and the bitmap live in the master file for every format.
|
|
auto format = GENERATE(FileWriterFormat::NXmxVDS, FileWriterFormat::NXmxIntegrated);
|
|
|
|
DiffractionExperiment x(DetJF(1));
|
|
x.FilePrefix("test_roi").ImagesPerTrigger(950).OverwriteExistingFiles(true)
|
|
.SetFileWriterFormat(format);
|
|
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
|
.IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
|
|
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
|
|
|
ROIDefinition defs;
|
|
defs.boxes.emplace_back("mybox", 10, 20, 30, 40);
|
|
defs.circles.emplace_back("mycircle", 100, 200, 15);
|
|
defs.azimuthal.emplace_back("mywedge", 2.0f, 4.0f, 30.0f, 90.0f);
|
|
x.ROI().SetROI(defs);
|
|
|
|
ScanResultGenerator generator(x);
|
|
std::vector<uint16_t> image(x.GetPixelsNum(), 0);
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
start_message.rois = x.ROI().ExportMetadata();
|
|
start_message.roi_map = x.ExportROIMap();
|
|
|
|
FileWriter file_set(start_message);
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 0;
|
|
for (const auto &name : {"mybox", "mycircle", "mywedge"})
|
|
message.roi[name] = ROIMessage{.sum = 100, .sum_square = 1000, .max_count = 50,
|
|
.pixels = 10, .x_weighted = 500, .y_weighted = 600};
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 1;
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_roi_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
|
|
const auto &rd = dataset->experiment.ROI().GetROIDefinition();
|
|
REQUIRE(rd.boxes.size() == 1);
|
|
REQUIRE(rd.circles.size() == 1);
|
|
REQUIRE(rd.azimuthal.size() == 1);
|
|
CHECK(rd.boxes[0].GetName() == "mybox");
|
|
CHECK(rd.boxes[0].GetXMin() == 10);
|
|
CHECK(rd.boxes[0].GetXMax() == 20);
|
|
CHECK(rd.circles[0].GetName() == "mycircle");
|
|
CHECK(rd.circles[0].GetRadius_pxl() == 15.0f);
|
|
CHECK(rd.azimuthal[0].GetName() == "mywedge");
|
|
CHECK(rd.azimuthal[0].HasPhi());
|
|
CHECK(rd.azimuthal[0].GetPhiMin_deg() == 30.0f);
|
|
|
|
// bitmap read back with the per-pixel footprint and the name->bit index
|
|
CHECK(dataset->roi_map.size() == x.GetXPixelsNumConv() * x.GetYPixelsNumConv());
|
|
CHECK(dataset->roi_bit_index.size() == 3);
|
|
CHECK(dataset->roi_bit_index.at("mybox") == 0);
|
|
|
|
// per-image ROI results surface from the master (VDS-linked for VDS format)
|
|
REQUIRE(dataset->roi.size() == 3);
|
|
auto it = std::find(dataset->roi.begin(), dataset->roi.end(), "mybox");
|
|
REQUIRE(it != dataset->roi.end());
|
|
const size_t idx = std::distance(dataset->roi.begin(), it);
|
|
CHECK(dataset->roi_sum.at(idx).at(0) == 100);
|
|
}
|
|
remove("test_roi_master.h5");
|
|
remove("test_roi_data_000001.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_Goniometer", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test17").ImagesPerTrigger(950).OverwriteExistingFiles(true);
|
|
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
|
.IncidentEnergy_keV(WVL_1A_IN_KEV).PixelSigned(false).BitDepthImage(16)
|
|
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
|
x.Goniometer(GoniometerAxis("omega", 95, 0.1f, Coord(0,-1,0),{}).ScreeningWedge(0.01f));
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
ScanResultGenerator generator(x);
|
|
std::vector<uint16_t> image(x.GetPixelsNum(), 0);
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
|
|
FileWriter file_set(start_message);
|
|
|
|
DataMessage message{};
|
|
for (int i = 0; i < 5; i++) {
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
REQUIRE_NOTHROW(generator.Add(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 5;
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
reader.ReadFile("test17_master.h5");
|
|
auto dataset = reader.GetDataset();
|
|
|
|
REQUIRE(!dataset->experiment.GetGridScan().has_value());
|
|
REQUIRE(dataset->experiment.GetGoniometer().has_value());
|
|
|
|
CHECK(dataset->experiment.GetGoniometer()->GetStart_deg() == 95.0);
|
|
CHECK(dataset->experiment.GetGoniometer()->GetIncrement_deg() == Catch::Approx(0.1f).margin(0.00001f));
|
|
CHECK(dataset->experiment.GetGoniometer()->GetWedge_deg() == Catch::Approx(0.01f).margin(0.00001f));
|
|
|
|
CHECK(dataset->experiment.GetGoniometer()->GetName() == "omega");
|
|
CHECK(dataset->experiment.GetGoniometer()->GetAxis().x == 0);
|
|
CHECK(dataset->experiment.GetGoniometer()->GetAxis().y == -1);
|
|
CHECK(dataset->experiment.GetGoniometer()->GetAxis().z == 0);
|
|
}
|
|
remove("test17_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_GridScan", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.PixelSigned(false).BitDepthImage(16).OverwriteExistingFiles(true);
|
|
x.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
|
|
|
DatasetSettings d;
|
|
d.FilePrefix("test_reader_grid_scan").ImagesPerTrigger(5);
|
|
d.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
|
.PhotonEnergy_keV(WVL_1A_IN_KEV)
|
|
.GridScan(GridScanSettings(3, -7.5, 8.0, true, true));
|
|
x.ImportDatasetSettings(d);
|
|
|
|
RegisterHDF5Filter();
|
|
ScanResultGenerator generator(x);
|
|
|
|
std::vector<uint16_t> image(x.GetPixelsNum(), 0);
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
|
|
FileWriter file_set(start_message);
|
|
|
|
DataMessage message{};
|
|
for (int i = 0; i < 5; i++) {
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 5;
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
reader.ReadFile("test_reader_grid_scan_master.h5");
|
|
auto dataset = reader.GetDataset();
|
|
|
|
|
|
REQUIRE(!dataset->experiment.GetGoniometer().has_value());
|
|
REQUIRE(dataset->experiment.GetGridScan().has_value());
|
|
|
|
CHECK(dataset->experiment.GetGridScan()->IsSnakeScan());
|
|
CHECK(dataset->experiment.GetGridScan()->IsVerticalScan());
|
|
CHECK(dataset->experiment.GetGridScan()->GetNFast() == 3);
|
|
CHECK(dataset->experiment.GetGridScan()->GetNSlow() == 2);
|
|
CHECK(dataset->experiment.GetGridScan()->GetNElem() == 6);
|
|
CHECK(dataset->experiment.GetGridScan()->GetGridStepX_um() == Catch::Approx(-7.5));
|
|
CHECK(dataset->experiment.GetGridScan()->GetGridStepY_um() == Catch::Approx(8.0));
|
|
}
|
|
remove("test_reader_grid_scan_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_DataI16", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test09").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
image[0] = INT16_MAX;
|
|
image[1] = INT16_MIN;
|
|
image[2] = 456;
|
|
image[3] = -3456;
|
|
|
|
ScanResultGenerator generator(x);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
image[5678] = i;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.indexing_result = (i % 2 == 0);
|
|
message.bkg_estimate = i * 345.6;
|
|
message.number = i;
|
|
message.profile_radius = 123.09;
|
|
generator.Add(message);
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test09_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
REQUIRE(dataset->spot_count.size() == 4);
|
|
REQUIRE(dataset->bkg_estimate.size() == 4);
|
|
REQUIRE(dataset->profile_radius.size() == 4);
|
|
REQUIRE_THROWS(reader.LoadImage(4));
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(i));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Image()[0] == SATURATED_PXL_VALUE);
|
|
CHECK(reader_image->Image()[1] == ERROR_PXL_VALUE);
|
|
CHECK(reader_image->Image()[2] == image[2]);
|
|
CHECK(reader_image->Image()[3] == image[3]);
|
|
CHECK(reader_image->Image()[5678] == i);
|
|
CHECK(dataset->indexing_result[i] == (i % 2 == 0));
|
|
CHECK(dataset->bkg_estimate[i] == Catch::Approx(i * 345.6));
|
|
CHECK(dataset->profile_radius[i] == Catch::Approx(123.09));
|
|
}
|
|
}
|
|
remove("test09_master.h5");
|
|
remove("test09_data_000001.h5");
|
|
remove("test09_data_000002.h5");
|
|
remove("test09_data_000003.h5");
|
|
remove("test09_data_000004.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_DataI16_OldMasterFormat", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test15").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
image[0] = INT16_MAX;
|
|
image[1] = INT16_MIN;
|
|
image[2] = 456;
|
|
image[3] = -3456;
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
image[5678] = i;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.indexing_result = (i % 2 == 0);
|
|
message.bkg_estimate = i * 345.6;
|
|
message.number = i;
|
|
message.profile_radius = 1.64;
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test15_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
REQUIRE(dataset->spot_count.size() == 4);
|
|
REQUIRE(dataset->bkg_estimate.size() == 4);
|
|
REQUIRE(dataset->profile_radius.size() == 4);
|
|
|
|
REQUIRE_THROWS(reader.LoadImage(4));
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(i));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Image()[0] == SATURATED_PXL_VALUE);
|
|
CHECK(reader_image->Image()[1] == ERROR_PXL_VALUE);
|
|
CHECK(reader_image->Image()[2] == image[2]);
|
|
CHECK(reader_image->Image()[3] == image[3]);
|
|
CHECK(reader_image->Image()[5678] == i);
|
|
|
|
CHECK(dataset->profile_radius[i] == Catch::Approx(1.64));
|
|
CHECK(dataset->indexing_result[i] == (i % 2 == 0));
|
|
CHECK(dataset->bkg_estimate[i] == Catch::Approx(i * 345.6));
|
|
}
|
|
}
|
|
remove("test15_master.h5");
|
|
remove("test15_data_000001.h5");
|
|
remove("test15_data_000002.h5");
|
|
remove("test15_data_000003.h5");
|
|
remove("test15_data_000004.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_DataU16", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test10").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(false)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<uint16_t> image(x.GetPixelsNum());
|
|
image[0] = UINT16_MAX;
|
|
image[1] = INT16_MAX;
|
|
image[2] = 456;
|
|
|
|
ScanResultGenerator generator(x);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
image[5678] = i;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.indexing_result = (i % 2 == 0);
|
|
message.bkg_estimate = i * 345.6;
|
|
message.number = i;
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test10_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
REQUIRE_THROWS(reader.LoadImage(4));
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(i));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Image()[0] == SATURATED_PXL_VALUE);
|
|
CHECK(reader_image->Image()[1] == INT16_MAX);
|
|
CHECK(reader_image->Image()[2] == 456);
|
|
CHECK(reader_image->Image()[5678] == i);
|
|
CHECK(dataset->indexing_result[i] == (i % 2 == 0));
|
|
CHECK(dataset->bkg_estimate[i] == Catch::Approx(i * 345.6));
|
|
}
|
|
}
|
|
remove("test10_master.h5");
|
|
remove("test10_data_000001.h5");
|
|
remove("test10_data_000002.h5");
|
|
remove("test10_data_000003.h5");
|
|
remove("test10_data_000004.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_DataI32", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test11").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(32).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int32_t> image(x.GetPixelsNum());
|
|
image[0] = INT32_MAX;
|
|
image[1] = INT32_MIN;
|
|
image[2] = 456;
|
|
|
|
ScanResultGenerator generator(x);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
image[5678] = i;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.number = i;
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test11_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
REQUIRE_THROWS(reader.LoadImage(4));
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(i));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Image()[0] == SATURATED_PXL_VALUE);
|
|
CHECK(reader_image->Image()[1] == ERROR_PXL_VALUE);
|
|
CHECK(reader_image->Image()[2] == 456);
|
|
CHECK(reader_image->Image()[5678] == i);
|
|
}
|
|
}
|
|
remove("test11_master.h5");
|
|
remove("test11_data_000001.h5");
|
|
remove("test11_data_000002.h5");
|
|
remove("test11_data_000003.h5");
|
|
remove("test11_data_000004.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_DataU32", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test12").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(32).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(false);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<uint32_t> image(x.GetPixelsNum());
|
|
image[0] = UINT32_MAX;
|
|
image[1] = static_cast<uint32_t>(INT32_MAX) + 50;
|
|
image[2] = 456;
|
|
image[3] = INT32_MAX;
|
|
image[4] = INT32_MAX - 1;
|
|
|
|
ScanResultGenerator generator(x);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
image[5678] = i;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.number = i;
|
|
|
|
generator.Add(message);
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test12_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
REQUIRE_THROWS(reader.LoadImage(4));
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(i));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Image()[0] == INT32_MAX);
|
|
CHECK(reader_image->Image()[1] == INT32_MAX);
|
|
CHECK(reader_image->Image()[2] == 456);
|
|
CHECK(reader_image->Image()[3] == INT32_MAX);
|
|
CHECK(reader_image->Image()[4] == INT32_MAX - 1);
|
|
CHECK(reader_image->Image()[5678] == i);
|
|
}
|
|
}
|
|
remove("test12_master.h5");
|
|
remove("test12_data_000001.h5");
|
|
remove("test12_data_000002.h5");
|
|
remove("test12_data_000003.h5");
|
|
remove("test12_data_000004.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_Summation", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test30").ImagesPerTrigger(3).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(3).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image_1(x.GetPixelsNum(),1);
|
|
std::vector<int16_t> image_2(x.GetPixelsNum(),2);
|
|
std::vector<int16_t> image_3(x.GetPixelsNum(),3);
|
|
image_3[0] = INT16_MAX;
|
|
image_2[1] = INT16_MIN;
|
|
|
|
ScanResultGenerator generator(x);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
std::vector<SpotToSave> spots;
|
|
|
|
DataMessage message{};
|
|
message.spots = spots;
|
|
|
|
message.image = CompressedImage(image_1, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 0;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
message.image = CompressedImage(image_2, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 1;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
message.image = CompressedImage(image_3, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 2;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test30_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 3);
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(0, 3));
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Image()[0] == SATURATED_PXL_VALUE);
|
|
CHECK(reader_image->Image()[1] == ERROR_PXL_VALUE);
|
|
CHECK(reader_image->Image()[2] == 1 + 2 +3);
|
|
CHECK(reader_image->Image()[5678] == 1 + 2 +3);
|
|
CHECK(reader_image->Image()[x.GetPixelsNum() - 1] == 1 + 2 +3);
|
|
|
|
REQUIRE_THROWS(reader.LoadImage(1, 3));
|
|
}
|
|
remove("test30_master.h5");
|
|
remove("test30_data_000001.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
|
|
TEST_CASE("JFJochReader_Summation_5", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test31").ImagesPerTrigger(5).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(5).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image_1(x.GetPixelsNum(),1);
|
|
std::vector<int16_t> image_2(x.GetPixelsNum(),2);
|
|
std::vector<int16_t> image_3(x.GetPixelsNum(),3);
|
|
std::vector<int16_t> image_4(x.GetPixelsNum(),4);
|
|
std::vector<int16_t> image_5(x.GetPixelsNum(),5);
|
|
image_3[0] = INT16_MAX;
|
|
image_2[1] = INT16_MIN;
|
|
|
|
ScanResultGenerator generator(x);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
std::vector<SpotToSave> spots;
|
|
|
|
DataMessage message{};
|
|
message.spots = spots;
|
|
|
|
message.image = CompressedImage(image_1, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 0;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
message.image = CompressedImage(image_2, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 1;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
message.image = CompressedImage(image_3, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 2;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
message.image = CompressedImage(image_4, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 3;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
message.image = CompressedImage(image_5, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = 4;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test31_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 5);
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(0, 5));
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Image()[0] == SATURATED_PXL_VALUE);
|
|
CHECK(reader_image->Image()[1] == ERROR_PXL_VALUE);
|
|
CHECK(reader_image->Image()[2] == 1 + 2 + 3 + 4 + 5);
|
|
CHECK(reader_image->Image()[5678] == 1 + 2 + 3 + 4 + 5);
|
|
CHECK(reader_image->Image()[x.GetPixelsNum() - 1] == 1 + 2 + 3 + 4 + 5);
|
|
|
|
REQUIRE_THROWS(reader.LoadImage(1, 6));
|
|
}
|
|
remove("test31_master.h5");
|
|
remove("test31_data_000001.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_Azint", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test27").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(false);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
AzimuthalIntegrationSettings azint_settings;
|
|
azint_settings.AzimuthalBinCount(4);
|
|
x.ImportAzimuthalIntegrationSettings(azint_settings);
|
|
|
|
std::vector<uint16_t> image(x.GetPixelsNum());
|
|
|
|
AzimuthalIntegrationMapping azint(x, PixelMask(x));
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
start_message.az_int_bin_to_q = azint.GetBinToQ();
|
|
start_message.az_int_bin_to_phi = azint.GetBinToPhi();
|
|
start_message.az_int_q_bin_count = azint.GetQBinCount();
|
|
start_message.az_int_phi_bin_count = azint.GetAzimuthalBinCount();
|
|
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.number = i;
|
|
message.az_int_profile = std::vector<float>(azint_settings.GetBinCount(), 57);
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test27_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(0));
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->Dataset().az_int_bin_to_q.size() == azint_settings.GetBinCount());
|
|
CHECK(reader_image->Dataset().azimuthal_bins == azint_settings.GetAzimuthalBinCount());
|
|
CHECK(reader_image->Dataset().q_bins == azint_settings.GetQBinCount());
|
|
REQUIRE(reader_image->ImageData().az_int_profile.size() == azint_settings.GetBinCount());
|
|
CHECK(reader_image->ImageData().az_int_profile[23] == 57);
|
|
CHECK(reader_image->GetAzInt1D_BinToQ().size() == azint_settings.GetQBinCount());
|
|
REQUIRE(reader_image->GetAzInt1D().size() == azint_settings.GetQBinCount());
|
|
CHECK(reader_image->GetAzInt1D()[23] == 4 * 57);
|
|
}
|
|
remove("test27_master.h5");
|
|
remove("test27_data_000001.h5");
|
|
remove("test27_data_000002.h5");
|
|
remove("test27_data_000003.h5");
|
|
remove("test27_data_000004.h5");
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_NiggliClass", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
x.FilePrefix("test95").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
LatticeMessage lm{
|
|
.centering = 'F',
|
|
.niggli_class = 1,
|
|
.crystal_system = gemmi::CrystalSystem::Cubic,
|
|
};
|
|
|
|
DataMessage message{};
|
|
message.number = 0;
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.indexing_result = true;
|
|
message.indexing_lattice = CrystalLattice(40, 50, 60, 90, 90, 90);
|
|
message.lattice_type = lm;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
message.number = 1;
|
|
message.indexing_result = false;
|
|
message.indexing_lattice = std::nullopt;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 2;
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test95_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 2);
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image, reader_image_2;
|
|
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(0));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->ImageData().indexing_result.value() == true);
|
|
REQUIRE(reader_image->ImageData().indexing_lattice);
|
|
REQUIRE(reader_image->ImageData().lattice_type);
|
|
CHECK(reader_image->ImageData().lattice_type->centering == 'F');
|
|
CHECK(reader_image->ImageData().lattice_type->niggli_class == 1);
|
|
CHECK(reader_image->ImageData().lattice_type->crystal_system == gemmi::CrystalSystem::Cubic);
|
|
|
|
REQUIRE_NOTHROW(reader_image_2 = reader.LoadImage(1));
|
|
|
|
REQUIRE(reader_image_2);
|
|
CHECK(!reader_image_2->ImageData().indexing_result.value());
|
|
REQUIRE(!reader_image_2->ImageData().indexing_lattice);
|
|
REQUIRE(!reader_image_2->ImageData().lattice_type);
|
|
}
|
|
remove("test95_master.h5");
|
|
remove("test95_data_000001.h5");
|
|
remove("test95_data_000002.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_NiggliClass_VDS", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
x.FilePrefix("test95").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
LatticeMessage lm{
|
|
.centering = 'F',
|
|
.niggli_class = 1,
|
|
.crystal_system = gemmi::CrystalSystem::Cubic,
|
|
};
|
|
|
|
DataMessage message_0{};
|
|
message_0.number = 0;
|
|
message_0.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message_0.indexing_result = true;
|
|
message_0.indexing_lattice = CrystalLattice(40, 50, 60, 90, 90, 90);
|
|
message_0.lattice_type = lm;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message_0));
|
|
|
|
DataMessage message_1{};
|
|
message_1.number = 1;
|
|
message_1.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message_1.indexing_result = false;
|
|
message_1.indexing_lattice = std::nullopt;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message_1));
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 2;
|
|
end_message.image_indexed = {true, false};
|
|
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test95_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 2);
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image, reader_image_2;
|
|
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(0));
|
|
|
|
REQUIRE(reader_image);
|
|
REQUIRE(reader_image->ImageData().indexing_result.has_value());
|
|
CHECK(reader_image->ImageData().indexing_result.value() == true);
|
|
REQUIRE(reader_image->ImageData().indexing_lattice.has_value());
|
|
REQUIRE(reader_image->ImageData().lattice_type.has_value());
|
|
CHECK(reader_image->ImageData().lattice_type->centering == 'F');
|
|
CHECK(reader_image->ImageData().lattice_type->niggli_class == 1);
|
|
CHECK(reader_image->ImageData().lattice_type->crystal_system == gemmi::CrystalSystem::Cubic);
|
|
|
|
REQUIRE_NOTHROW(reader_image_2 = reader.LoadImage(1));
|
|
|
|
REQUIRE(reader_image_2);
|
|
CHECK(!reader_image_2->ImageData().indexing_result.value());
|
|
REQUIRE(!reader_image_2->ImageData().indexing_lattice);
|
|
REQUIRE(!reader_image_2->ImageData().lattice_type);
|
|
}
|
|
remove("test95_master.h5");
|
|
remove("test95_data_000001.h5");
|
|
remove("test95_data_000002.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
|
|
TEST_CASE("JFJochReader_MissingEntries", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
x.FilePrefix("test96").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(10).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
DataMessage message{};
|
|
message.number = 0;
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.indexing_result = true;
|
|
message.indexing_lattice = CrystalLattice(40, 50, 60, 90, 90, 90);
|
|
message.spot_count_indexed = 56;
|
|
message.spot_count = 85;
|
|
message.b_factor = 123.45;
|
|
message.spots = {SpotToSave{.x = 10, .y=50, .intensity = 80}};
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
message.number = 1;
|
|
message.indexing_result = false;
|
|
message.indexing_lattice = std::nullopt;
|
|
message.spot_count_indexed = std::nullopt;
|
|
message.spot_count = 70;
|
|
message.b_factor = std::nullopt;
|
|
message.spots = {SpotToSave{.x = 10, .y=50, .intensity = 80}};
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 2;
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test96_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 2);
|
|
|
|
REQUIRE(dataset->b_factor.size() == 2);
|
|
REQUIRE(dataset->spot_count_indexed.size() == 2);
|
|
|
|
CHECK(dataset->b_factor[0] == Catch::Approx(123.45));
|
|
CHECK(std::isnan(dataset->b_factor[1]));
|
|
CHECK(dataset->spot_count_indexed[0] == 56);
|
|
CHECK(dataset->spot_count_indexed[1] == 0);
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image, reader_image_2;
|
|
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(0));
|
|
REQUIRE(reader_image);
|
|
REQUIRE(reader_image->ImageData().b_factor.has_value());
|
|
CHECK(reader_image->ImageData().b_factor.value() == Catch::Approx(123.45));
|
|
REQUIRE(reader_image->ImageData().spot_count_indexed.has_value());
|
|
CHECK(reader_image->ImageData().spot_count_indexed.value() == 56);
|
|
|
|
REQUIRE_NOTHROW(reader_image_2 = reader.LoadImage(1));
|
|
REQUIRE(reader_image_2);
|
|
CHECK(reader_image_2->ImageData().spot_count_indexed.has_value());
|
|
CHECK(reader_image_2->ImageData().spot_count_indexed.value() == 0);
|
|
}
|
|
remove("test96_master.h5");
|
|
remove("test96_data_000001.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_Spots_OldMasterFormat", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test40").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
spots.push_back(SpotToSave{
|
|
.x = 1, .y = 2, .intensity = 376,
|
|
.h = 11, .k = -3, .l = -5,
|
|
.dist_ewald_sphere = 0.1234f,
|
|
.ice_ring = true,
|
|
.indexed = true
|
|
});
|
|
spots.push_back(SpotToSave{
|
|
.x = 7, .y = -3, .intensity = 0.156f,
|
|
.ice_ring = false,
|
|
.indexed = false,
|
|
});
|
|
image[5678] = i;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.indexing_result = (i % 2 == 0);
|
|
message.number = i;
|
|
message.spot_count = 72 + i;
|
|
message.spot_count_ice_rings = 45 + 2 * i;
|
|
message.spot_count_low_res = 12 + 3 * i;
|
|
message.spot_count_indexed = 15 + 4 * i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test40_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
CHECK(dataset->spot_count[1] == 72 + 1);
|
|
CHECK(dataset->spot_count_ice_rings[2] == 45 + 2 * 2);
|
|
CHECK(dataset->spot_count_low_res[3] == 12 + 3 * 3);
|
|
CHECK(dataset->spot_count_indexed[0] == 15);
|
|
|
|
REQUIRE_THROWS(reader.LoadImage(4));
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(i));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->ImageData().spot_count == 72 + i);
|
|
CHECK(reader_image->ImageData().spot_count_ice_rings == 45 + 2 * i);
|
|
CHECK(reader_image->ImageData().spot_count_low_res == 12 + 3 * i);
|
|
CHECK(reader_image->ImageData().spot_count_indexed == 15 + 4 * i);
|
|
REQUIRE(reader_image->ImageData().spots.size() == 2);
|
|
CHECK(reader_image->ImageData().spots[0].x == 1);
|
|
CHECK(reader_image->ImageData().spots[0].y == 2);
|
|
CHECK(reader_image->ImageData().spots[0].intensity == 376);
|
|
CHECK(reader_image->ImageData().spots[0].ice_ring == true);
|
|
CHECK(reader_image->ImageData().spots[0].indexed == true);
|
|
CHECK(reader_image->ImageData().spots[0].h == 11);
|
|
CHECK(reader_image->ImageData().spots[0].k == -3);
|
|
CHECK(reader_image->ImageData().spots[0].l == -5);
|
|
CHECK(reader_image->ImageData().spots[0].dist_ewald_sphere == Catch::Approx(0.1234f));
|
|
CHECK(reader_image->ImageData().spots[1].x == 7);
|
|
CHECK(reader_image->ImageData().spots[1].y == -3);
|
|
CHECK(reader_image->ImageData().spots[1].intensity == Catch::Approx(0.156f));
|
|
CHECK(reader_image->ImageData().spots[1].ice_ring == false);
|
|
CHECK(reader_image->ImageData().spots[1].indexed == false);
|
|
}
|
|
}
|
|
remove("test40_master.h5");
|
|
remove("test40_data_000001.h5");
|
|
remove("test40_data_000002.h5");
|
|
remove("test40_data_000003.h5");
|
|
remove("test40_data_000004.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_Spots_VDS", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test41").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
std::vector<SpotToSave> spots;
|
|
spots.push_back(SpotToSave{
|
|
.x = 1, .y = 2, .intensity = 376,
|
|
.h = 11, .k = -3, .l = -5,
|
|
.dist_ewald_sphere = 0.1234f,
|
|
.ice_ring = true,
|
|
.indexed = true
|
|
});
|
|
spots.push_back(SpotToSave{
|
|
.x = 7, .y = -3, .intensity = 0.156f,
|
|
.ice_ring = false,
|
|
.indexed = false,
|
|
});
|
|
image[5678] = i;
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spots = spots;
|
|
message.indexing_result = (i % 2 == 0);
|
|
message.number = i;
|
|
message.spot_count = 72 + i;
|
|
message.spot_count_ice_rings = 45 + 2 * i;
|
|
message.spot_count_low_res = 12 + 3 * i;
|
|
message.spot_count_indexed = 15 + 4 * i;
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test41_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
CHECK(dataset->spot_count[1] == 72 + 1);
|
|
CHECK(dataset->spot_count_ice_rings[2] == 45 + 2 * 2);
|
|
CHECK(dataset->spot_count_low_res[3] == 12 + 3 * 3);
|
|
CHECK(dataset->spot_count_indexed[0] == 15);
|
|
REQUIRE_THROWS(reader.LoadImage(4));
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(i));
|
|
|
|
REQUIRE(reader_image);
|
|
CHECK(reader_image->ImageData().spot_count == 72 + i);
|
|
CHECK(reader_image->ImageData().spot_count_ice_rings == 45 + 2 * i);
|
|
CHECK(reader_image->ImageData().spot_count_low_res == 12 + 3 * i);
|
|
CHECK(reader_image->ImageData().spot_count_indexed == 15 + 4 * i);
|
|
|
|
REQUIRE(reader_image->ImageData().spots.size() == 2);
|
|
CHECK(reader_image->ImageData().spots[0].x == 1);
|
|
CHECK(reader_image->ImageData().spots[0].y == 2);
|
|
CHECK(reader_image->ImageData().spots[0].intensity == 376);
|
|
CHECK(reader_image->ImageData().spots[0].ice_ring == true);
|
|
CHECK(reader_image->ImageData().spots[0].indexed == true);
|
|
CHECK(reader_image->ImageData().spots[0].h == 11);
|
|
CHECK(reader_image->ImageData().spots[0].k == -3);
|
|
CHECK(reader_image->ImageData().spots[0].l == -5);
|
|
CHECK(reader_image->ImageData().spots[0].dist_ewald_sphere == Catch::Approx(0.1234f));
|
|
CHECK(reader_image->ImageData().spots[1].x == 7);
|
|
CHECK(reader_image->ImageData().spots[1].y == -3);
|
|
CHECK(reader_image->ImageData().spots[1].intensity == Catch::Approx(0.156f));
|
|
CHECK(reader_image->ImageData().spots[1].ice_ring == false);
|
|
CHECK(reader_image->ImageData().spots[1].indexed == false);
|
|
}
|
|
}
|
|
remove("test41_master.h5");
|
|
remove("test41_data_000001.h5");
|
|
remove("test41_data_000002.h5");
|
|
remove("test41_data_000003.h5");
|
|
remove("test41_data_000004.h5");
|
|
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_InstrumentMetadata_Sample_RingCurrent", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
// Set identifying names and ring current (mA in API; writer stores A)
|
|
x.FilePrefix("test_meta").ImagesPerTrigger(0).OverwriteExistingFiles(true);
|
|
|
|
InstrumentMetadata metadata;
|
|
metadata.InstrumentName("PXI").SourceName("SLS");
|
|
x.ImportInstrumentMetadata(metadata).SampleName("Lysozyme").RingCurrent_mA(399.5); // 0.3995 A
|
|
x.TotalFlux(1e7).AttenuatorTransmission(0.56);
|
|
x.DetectIceRings(false);
|
|
|
|
// Minimal other required fields
|
|
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
|
.IncidentEnergy_keV(WVL_1A_IN_KEV)
|
|
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
|
|
|
// also set fluorescence spectrum
|
|
x.FluorescenceSpectrum(XrayFluorescenceSpectrum({1.0f, 2.0f, 3.0f}, {5.0f, 7.0f, 6.0f}));
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 0;
|
|
|
|
std::unique_ptr<NXmx> master = std::make_unique<NXmx>(start_message);
|
|
master->Finalize(end_message);
|
|
master.reset();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_meta_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
|
|
auto meta = dataset->experiment.GetInstrumentMetadata();
|
|
CHECK(meta.GetInstrumentName() == "PXI");
|
|
CHECK(meta.GetSourceName() == "SLS");
|
|
|
|
CHECK(dataset->experiment.GetAttenuatorTransmission() == Catch::Approx(0.56));
|
|
CHECK(dataset->experiment.GetTotalFlux() == Catch::Approx(1e7));
|
|
|
|
// Sample name
|
|
CHECK(dataset->experiment.GetSampleName() == "Lysozyme");
|
|
|
|
// Ring current read back in mA; allow small fp tolerance
|
|
CHECK(dataset->experiment.GetRingCurrent_mA().has_value());
|
|
CHECK(dataset->experiment.GetRingCurrent_mA().value() == Catch::Approx(399.5));
|
|
|
|
CHECK(!dataset->experiment.IsDetectIceRings());
|
|
// Fluorescence spectrum presence and values
|
|
REQUIRE(!dataset->experiment.GetFluorescenceSpectrum().empty());
|
|
const auto& fl = dataset->experiment.GetFluorescenceSpectrum();
|
|
CHECK(fl.GetEnergy_eV().size() == 3);
|
|
CHECK(fl.GetData().size() == 3);
|
|
CHECK(fl.GetEnergy_eV()[0] == Catch::Approx(1.0f));
|
|
CHECK(fl.GetData()[1] == Catch::Approx(7.0f));
|
|
}
|
|
|
|
remove("test_meta_master.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_NXmxIntegrated", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test_reader_integrated").ImagesPerTrigger(3).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).PixelSigned(false).SetFileWriterFormat(FileWriterFormat::NXmxIntegrated);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
x.BeamX_pxl(100).BeamY_pxl(200).DetectorDistance_mm(150)
|
|
.IncidentEnergy_keV(WVL_1A_IN_KEV)
|
|
.FrameTime(std::chrono::microseconds(500), std::chrono::microseconds(10));
|
|
|
|
AzimuthalIntegrationSettings azint_settings;
|
|
azint_settings.AzimuthalBinCount(4);
|
|
x.ImportAzimuthalIntegrationSettings(azint_settings);
|
|
|
|
std::vector<uint16_t> image(x.GetPixelsNum(), 0);
|
|
image[0] = UINT16_MAX;
|
|
image[1] = 123;
|
|
image[5678] = 321;
|
|
|
|
AzimuthalIntegrationMapping azint(x, PixelMask(x));
|
|
RegisterHDF5Filter();
|
|
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
start_message.az_int_bin_to_q = azint.GetBinToQ();
|
|
start_message.az_int_bin_to_phi = azint.GetBinToPhi();
|
|
start_message.az_int_q_bin_count = azint.GetQBinCount();
|
|
start_message.az_int_phi_bin_count = azint.GetAzimuthalBinCount();
|
|
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
image[5678] = 321 + i;
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
message.image_collection_efficiency = 0.9f + 0.01f * i;
|
|
message.az_int_profile = std::vector<float>(azint_settings.GetBinCount(), static_cast<float>(50 + i));
|
|
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_reader_integrated_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
|
|
CHECK(dataset->experiment.GetImageNum() == 3);
|
|
REQUIRE(dataset->efficiency.size() == 3);
|
|
CHECK(dataset->efficiency[0] == Catch::Approx(0.90f));
|
|
CHECK(dataset->efficiency[1] == Catch::Approx(0.91f));
|
|
CHECK(dataset->efficiency[2] == Catch::Approx(0.92f));
|
|
|
|
CHECK(dataset->az_int_bin_to_q.size() == azint_settings.GetBinCount());
|
|
CHECK(dataset->azimuthal_bins == azint_settings.GetAzimuthalBinCount());
|
|
CHECK(dataset->q_bins == azint_settings.GetQBinCount());
|
|
|
|
std::shared_ptr<JFJochReaderImage> reader_image;
|
|
REQUIRE_NOTHROW(reader_image = reader.LoadImage(1));
|
|
REQUIRE(reader_image);
|
|
|
|
CHECK(reader_image->Image()[0] == SATURATED_PXL_VALUE);
|
|
CHECK(reader_image->Image()[1] == 123);
|
|
CHECK(reader_image->Image()[5678] == 322);
|
|
|
|
REQUIRE(reader_image->ImageData().image_collection_efficiency.has_value());
|
|
CHECK(reader_image->ImageData().image_collection_efficiency.value() == Catch::Approx(0.91f));
|
|
|
|
REQUIRE(reader_image->ImageData().az_int_profile.size() == azint_settings.GetBinCount());
|
|
CHECK(reader_image->ImageData().az_int_profile[0] == Catch::Approx(51.0f));
|
|
CHECK(reader_image->ImageData().az_int_profile[23] == Catch::Approx(51.0f));
|
|
}
|
|
remove("test_reader_integrated_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_GetRawImage_NXmxLegacy", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test_read_raw_image").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(2).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::BSHUF_ZSTD);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
for (int i = 0; i < image.size(); i++)
|
|
image[i] = static_cast<int16_t>((i * 7 + 33) % UINT16_MAX);
|
|
|
|
RegisterHDF5Filter();
|
|
JFJochBitShuffleCompressor compressor(CompressionAlgorithm::BSHUF_ZSTD);
|
|
auto compressed_image = compressor.Compress(image);
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(compressed_image, x.GetXPixelsNum(), x.GetYPixelsNum(),
|
|
CompressedImageMode::Int16, CompressionAlgorithm::BSHUF_ZSTD);
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_read_raw_image_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
std::shared_ptr<JFJochReaderRawImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.GetRawImage(i));
|
|
|
|
CHECK(reader_image->image.GetMode() == CompressedImageMode::Int16);
|
|
CHECK(reader_image->image.GetCompressionAlgorithm() == CompressionAlgorithm::BSHUF_ZSTD);
|
|
CHECK(reader_image->image.GetWidth() == x.GetXPixelsNum());
|
|
CHECK(reader_image->image.GetHeight() == x.GetYPixelsNum());
|
|
CHECK(reader_image->image.GetCompressedSize() == compressed_image.size());
|
|
CHECK(reader_image->image.GetCompressed() == reader_image->image_buffer.data());
|
|
REQUIRE(reader_image->image_buffer.size() == compressed_image.size());
|
|
CHECK(memcmp(reader_image->image_buffer.data(), compressed_image.data(), compressed_image.size()) == 0);
|
|
}
|
|
}
|
|
remove("test_read_raw_image_master.h5");
|
|
remove("test_read_raw_image_data_000001.h5");
|
|
remove("test_read_raw_image_data_000002.h5");
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_GetRawImage_VDS", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test_read_raw_image").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(2).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::BSHUF_ZSTD);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
for (int i = 0; i < image.size(); i++)
|
|
image[i] = static_cast<int16_t>((i * 7 + 33) % UINT16_MAX);
|
|
|
|
RegisterHDF5Filter();
|
|
JFJochBitShuffleCompressor compressor(CompressionAlgorithm::BSHUF_ZSTD);
|
|
auto compressed_image = compressor.Compress(image);
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(compressed_image, x.GetXPixelsNum(), x.GetYPixelsNum(),
|
|
CompressedImageMode::Int16, CompressionAlgorithm::BSHUF_ZSTD);
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_read_raw_image_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
std::shared_ptr<JFJochReaderRawImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.GetRawImage(i));
|
|
|
|
CHECK(reader_image->image.GetMode() == CompressedImageMode::Int16);
|
|
CHECK(reader_image->image.GetCompressionAlgorithm() == CompressionAlgorithm::BSHUF_ZSTD);
|
|
CHECK(reader_image->image.GetWidth() == x.GetXPixelsNum());
|
|
CHECK(reader_image->image.GetHeight() == x.GetYPixelsNum());
|
|
CHECK(reader_image->image.GetCompressedSize() == compressed_image.size());
|
|
CHECK(reader_image->image.GetCompressed() == reader_image->image_buffer.data());
|
|
REQUIRE(reader_image->image_buffer.size() == compressed_image.size());
|
|
CHECK(memcmp(reader_image->image_buffer.data(), compressed_image.data(), compressed_image.size()) == 0);
|
|
}
|
|
}
|
|
remove("test_read_raw_image_master.h5");
|
|
remove("test_read_raw_image_data_000001.h5");
|
|
remove("test_read_raw_image_data_000002.h5");
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_GetRawImage_Integrated", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("test_read_raw_image").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(2).SetFileWriterFormat(FileWriterFormat::NXmxIntegrated).PixelSigned(true)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
x.Compression(CompressionAlgorithm::BSHUF_ZSTD);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
for (int i = 0; i < image.size(); i++)
|
|
image[i] = static_cast<int16_t>((i * 7 + 33) % UINT16_MAX);
|
|
|
|
RegisterHDF5Filter();
|
|
JFJochBitShuffleCompressor compressor(CompressionAlgorithm::BSHUF_ZSTD);
|
|
auto compressed_image = compressor.Compress(image);
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(compressed_image, x.GetXPixelsNum(), x.GetYPixelsNum(),
|
|
CompressedImageMode::Int16, CompressionAlgorithm::BSHUF_ZSTD);
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_read_raw_image_master.h5"));
|
|
auto dataset = reader.GetDataset();
|
|
CHECK(dataset->experiment.GetImageNum() == 4);
|
|
|
|
std::shared_ptr<JFJochReaderRawImage> reader_image;
|
|
for (int i = 0; i < 4; i++) {
|
|
REQUIRE_NOTHROW(reader_image = reader.GetRawImage(i));
|
|
|
|
CHECK(reader_image->image.GetMode() == CompressedImageMode::Int16);
|
|
CHECK(reader_image->image.GetCompressionAlgorithm() == CompressionAlgorithm::BSHUF_ZSTD);
|
|
CHECK(reader_image->image.GetWidth() == x.GetXPixelsNum());
|
|
CHECK(reader_image->image.GetHeight() == x.GetYPixelsNum());
|
|
CHECK(reader_image->image.GetCompressedSize() == compressed_image.size());
|
|
CHECK(reader_image->image.GetCompressed() == reader_image->image_buffer.data());
|
|
REQUIRE(reader_image->image_buffer.size() == compressed_image.size());
|
|
CHECK(memcmp(reader_image->image_buffer.data(), compressed_image.data(), compressed_image.size()) == 0);
|
|
}
|
|
}
|
|
remove("test_read_raw_image_master.h5");
|
|
// No leftover HDF5 objects
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_HDF5DataSource_Integrated", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("source_integrated").ImagesPerTrigger(5).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).SetFileWriterFormat(FileWriterFormat::NXmxIntegrated).PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 17);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter writer(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
image[5678] = static_cast<int16_t>(100 + i);
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(writer.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
writer.WriteHDF5(end_message);
|
|
writer.Finalize();
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("source_integrated_master.h5"));
|
|
|
|
auto source = reader.GetHDF5DataSource(1, 3);
|
|
REQUIRE(source.size() == 1);
|
|
|
|
CHECK(source[0].filename == "source_integrated_master.h5");
|
|
CHECK(source[0].dataset == "/entry/data/data");
|
|
CHECK(source[0].source_first_image == 1);
|
|
CHECK(source[0].virtual_first_image == 0);
|
|
CHECK(source[0].image_count == 3);
|
|
}
|
|
|
|
remove("source_integrated_master.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_HDF5DataSource_VDS", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("source_vds_mapping").ImagesPerTrigger(5).ImagesPerFile(2).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 21);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter writer(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
image[5678] = static_cast<int16_t>(200 + i);
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(writer.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
writer.WriteHDF5(end_message);
|
|
writer.Finalize();
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("source_vds_mapping_master.h5"));
|
|
|
|
// Range crosses file boundary:
|
|
// global images 1,2,3 map to:
|
|
// data_000001 image 1
|
|
// data_000002 images 0,1
|
|
auto source = reader.GetHDF5DataSource(1, 3);
|
|
REQUIRE(source.size() == 2);
|
|
|
|
CHECK(source[0].filename == "source_vds_mapping_data_000001.h5");
|
|
CHECK(source[0].dataset == "/entry/data/data");
|
|
CHECK(source[0].source_first_image == 1);
|
|
CHECK(source[0].virtual_first_image == 0);
|
|
CHECK(source[0].image_count == 1);
|
|
|
|
CHECK(source[1].filename == "source_vds_mapping_data_000002.h5");
|
|
CHECK(source[1].dataset == "/entry/data/data");
|
|
CHECK(source[1].source_first_image == 0);
|
|
CHECK(source[1].virtual_first_image == 1);
|
|
CHECK(source[1].image_count == 2);
|
|
}
|
|
|
|
remove("source_vds_mapping_master.h5");
|
|
remove("source_vds_mapping_data_000001.h5");
|
|
remove("source_vds_mapping_data_000002.h5");
|
|
remove("source_vds_mapping_data_000003.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_HDF5DataSource_Legacy", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("source_legacy_mapping").ImagesPerTrigger(5).ImagesPerFile(2).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 31);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter writer(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
image[5678] = static_cast<int16_t>(300 + i);
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(writer.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
writer.WriteHDF5(end_message);
|
|
writer.Finalize();
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("source_legacy_mapping_master.h5"));
|
|
|
|
auto source = reader.GetHDF5DataSource(1, 3);
|
|
REQUIRE(source.size() == 2);
|
|
|
|
CHECK(source[0].filename == "source_legacy_mapping_data_000001.h5");
|
|
CHECK(source[0].dataset == "/entry/data/data");
|
|
CHECK(source[0].source_first_image == 1);
|
|
CHECK(source[0].virtual_first_image == 0);
|
|
CHECK(source[0].image_count == 1);
|
|
|
|
CHECK(source[1].filename == "source_legacy_mapping_data_000002.h5");
|
|
CHECK(source[1].dataset == "/entry/data/data");
|
|
CHECK(source[1].source_first_image == 0);
|
|
CHECK(source[1].virtual_first_image == 1);
|
|
CHECK(source[1].image_count == 2);
|
|
}
|
|
|
|
remove("source_legacy_mapping_master.h5");
|
|
remove("source_legacy_mapping_data_000001.h5");
|
|
remove("source_legacy_mapping_data_000002.h5");
|
|
remove("source_legacy_mapping_data_000003.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_ProcessingHDF5_FromVDS_MapsToDataFiles", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("proc_source_vds").ImagesPerTrigger(5).ImagesPerFile(2).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 51);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter writer(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
image[5678] = static_cast<int16_t>(500 + i);
|
|
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
REQUIRE_NOTHROW(writer.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
writer.WriteHDF5(end_message);
|
|
writer.Finalize();
|
|
}
|
|
|
|
std::vector<HDF5DataSourceMessage> source_data;
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("proc_source_vds_master.h5"));
|
|
source_data = reader.GetHDF5DataSource(1, 3);
|
|
|
|
REQUIRE(source_data.size() == 2);
|
|
CHECK(source_data[0].filename == "proc_source_vds_data_000001.h5");
|
|
CHECK(source_data[1].filename == "proc_source_vds_data_000002.h5");
|
|
}
|
|
|
|
{
|
|
DiffractionExperiment proc_x = x;
|
|
proc_x.FilePrefix("proc_from_vds")
|
|
.ImagesPerTrigger(3)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxIntegrated)
|
|
.OverwriteExistingFiles(true);
|
|
|
|
StartMessage start_message;
|
|
proc_x.FillMessage(start_message);
|
|
start_message.number_of_images = 3;
|
|
start_message.images_per_file = 3;
|
|
start_message.write_images = false;
|
|
start_message.write_master_file = true;
|
|
start_message.hdf5_source_data = source_data;
|
|
|
|
FileWriter writer(start_message);
|
|
|
|
for (int i = 0; i < 3; i++) {
|
|
DataMessage message{};
|
|
message.number = i;
|
|
message.original_number = i + 1;
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.spot_count = 200 + i;
|
|
REQUIRE_NOTHROW(writer.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 3;
|
|
writer.WriteHDF5(end_message);
|
|
writer.Finalize();
|
|
}
|
|
|
|
{
|
|
HDF5ReadOnlyFile file("proc_from_vds_master.h5");
|
|
HDF5DataSet data(file, "/entry/data/data");
|
|
HDF5Dcpl dcpl(data);
|
|
|
|
REQUIRE(dcpl.GetLayout() == HDF5DataSetLayout::VIRTUAL);
|
|
|
|
auto mappings = dcpl.GetVirtualMappings();
|
|
REQUIRE(mappings.size() == 2);
|
|
|
|
CHECK(mappings[0].filename == "proc_source_vds_data_000001.h5");
|
|
CHECK(mappings[1].filename == "proc_source_vds_data_000002.h5");
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("proc_from_vds_master.h5"));
|
|
|
|
auto img0 = reader.LoadImage(0);
|
|
REQUIRE(img0);
|
|
CHECK(img0->Image()[5678] == 501);
|
|
|
|
auto img2 = reader.LoadImage(2);
|
|
REQUIRE(img2);
|
|
CHECK(img2->Image()[5678] == 503);
|
|
}
|
|
|
|
remove("proc_source_vds_master.h5");
|
|
remove("proc_source_vds_data_000001.h5");
|
|
remove("proc_source_vds_data_000002.h5");
|
|
remove("proc_source_vds_data_000003.h5");
|
|
remove("proc_from_vds_master.h5");
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_ReadReflections_VDS", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("read_reflections_vds")
|
|
.ImagesPerTrigger(4)
|
|
.ImagesPerFile(1)
|
|
.OverwriteExistingFiles(true)
|
|
.BitDepthImage(16)
|
|
.PixelSigned(true)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 0);
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
|
|
FileWriter writer(start_message);
|
|
ScanResultGenerator scan_result(x);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
|
|
if (i == 1 || i == 3) {
|
|
message.integrated_reflections = 2;
|
|
message.reflections = {
|
|
Reflection{
|
|
.h = static_cast<int32_t>(10 + i),
|
|
.k = 20,
|
|
.l = 30,
|
|
.delta_phi_deg = 0.1f,
|
|
.predicted_x = 100.0f + static_cast<float>(i),
|
|
.predicted_y = 200.0f + static_cast<float>(i),
|
|
.d = 1.5f,
|
|
.I = 1000.0f + static_cast<float>(i),
|
|
.bkg = 10.0f,
|
|
.sigma = 2.0f,
|
|
.rlp = 1.0f,
|
|
.partiality = 0.9f,
|
|
.zeta = 0.01f
|
|
},
|
|
Reflection{
|
|
.h = static_cast<int32_t>(40 + i),
|
|
.k = 50,
|
|
.l = 60,
|
|
.delta_phi_deg = 0.2f,
|
|
.predicted_x = 300.0f + static_cast<float>(i),
|
|
.predicted_y = 400.0f + static_cast<float>(i),
|
|
.d = 2.5f,
|
|
.I = 2000.0f + static_cast<float>(i),
|
|
.bkg = 20.0f,
|
|
.sigma = 3.0f,
|
|
.rlp = 1.0f,
|
|
.partiality = 0.8f,
|
|
.zeta = 0.02f
|
|
}
|
|
};
|
|
|
|
message.mosaicity_deg = i*0.15f;
|
|
message.indexing_lattice = CrystalLattice({100,0,0}, {0,50,0}, {0,0,30});
|
|
}
|
|
|
|
REQUIRE_NOTHROW(writer.WriteHDF5(message));
|
|
scan_result.Add(message);
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
scan_result.FillEndMessage(end_message);
|
|
|
|
writer.WriteHDF5(end_message);
|
|
writer.Finalize();
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_reflections_vds_master.h5"));
|
|
|
|
auto reflections = reader.ReadReflections();
|
|
|
|
REQUIRE(reflections.size() == 4);
|
|
|
|
CHECK(reflections[0].reflections.empty());
|
|
|
|
REQUIRE(reflections[1].reflections.size() == 2);
|
|
CHECK(reflections[1].reflections[0].h == 11);
|
|
CHECK(reflections[1].reflections[0].k == 20);
|
|
CHECK(reflections[1].reflections[0].l == 30);
|
|
CHECK(reflections[1].reflections[0].I == Catch::Approx(1001.0f));
|
|
CHECK(reflections[1].reflections[0].predicted_x == Catch::Approx(101.0f));
|
|
CHECK(reflections[1].reflections[0].predicted_y == Catch::Approx(201.0f));
|
|
CHECK(reflections[1].mosaicity_deg == Catch::Approx(0.15f));
|
|
CHECK(reflections[1].latt.CalcVolume() == Catch::Approx(100*50*30));
|
|
|
|
CHECK(reflections[2].reflections.empty());
|
|
|
|
REQUIRE(reflections[3].reflections.size() == 2);
|
|
CHECK(reflections[3].reflections[0].h == 13);
|
|
CHECK(reflections[3].reflections[0].I == Catch::Approx(1003.0f));
|
|
CHECK(reflections[3].reflections[1].h == 43);
|
|
CHECK(reflections[3].reflections[1].I == Catch::Approx(2003.0f));
|
|
CHECK(reflections[3].mosaicity_deg == Catch::Approx(0.45f));
|
|
CHECK(reflections[3].latt.Vec0().x == Catch::Approx(100.0f));
|
|
CHECK(reflections[3].latt.Vec1().y == Catch::Approx(50.0f));
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_reflections_vds_master.h5"));
|
|
|
|
auto reflections = reader.ReadReflections(1, 3);
|
|
|
|
REQUIRE(reflections.size() == 3);
|
|
|
|
REQUIRE(reflections[0].reflections.size() == 2); // original image 1
|
|
CHECK(reflections[0].reflections[0].h == 11);
|
|
|
|
CHECK(reflections[1].reflections.empty()); // original image 2
|
|
|
|
REQUIRE(reflections[2].reflections.size() == 2); // original image 3
|
|
CHECK(reflections[2].reflections[0].h == 13);
|
|
}
|
|
|
|
remove("read_reflections_vds_master.h5");
|
|
remove("read_reflections_vds_data_000001.h5");
|
|
remove("read_reflections_vds_data_000002.h5");
|
|
remove("read_reflections_vds_data_000003.h5");
|
|
remove("read_reflections_vds_data_000004.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
static std::vector<SpotToSave> MakeTestSpots(int i) {
|
|
return {
|
|
SpotToSave{
|
|
.x = 1, .y = 2, .intensity = 376,
|
|
.h = 11, .k = -3, .l = -5,
|
|
.dist_ewald_sphere = 0.1234f,
|
|
.ice_ring = true,
|
|
.indexed = true
|
|
},
|
|
SpotToSave{
|
|
.x = 7, .y = static_cast<float>(-3 - i), .intensity = 0.156f,
|
|
.ice_ring = false,
|
|
.indexed = false,
|
|
}
|
|
};
|
|
}
|
|
|
|
// Assert the full field set on spots[0] and the per-image variation on
|
|
// spots[1].y, which is the only field that differs across images.
|
|
static void CheckSpotFields(const SpotToSave &s0, const SpotToSave &s1, int i) {
|
|
CHECK(s0.x == 1);
|
|
CHECK(s0.y == 2);
|
|
CHECK(s0.intensity == Catch::Approx(376));
|
|
CHECK(s0.ice_ring == true);
|
|
CHECK(s0.indexed == true);
|
|
CHECK(s0.h == 11);
|
|
CHECK(s0.k == -3);
|
|
CHECK(s0.l == -5);
|
|
CHECK(s0.dist_ewald_sphere == Catch::Approx(0.1234f));
|
|
CHECK(s0.image == i);
|
|
|
|
CHECK(s1.x == Catch::Approx(7));
|
|
CHECK(s1.y == Catch::Approx(static_cast<float>(-3 - i)));
|
|
CHECK(s1.intensity == Catch::Approx(0.156f));
|
|
CHECK(s1.ice_ring == false);
|
|
CHECK(s1.indexed == false);
|
|
CHECK(s1.image == i);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_ReadSpots_Legacy", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("read_spots_legacy")
|
|
.ImagesPerTrigger(4)
|
|
.ImagesPerFile(1)
|
|
.OverwriteExistingFiles(true)
|
|
.BitDepthImage(16)
|
|
.PixelSigned(true)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxLegacy)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 0);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
message.spots = MakeTestSpots(i);
|
|
message.spot_count = 72 + i;
|
|
message.spot_count_ice_rings = 45 + 2 * i;
|
|
message.spot_count_low_res = 12 + 3 * i;
|
|
message.spot_count_indexed = 15 + 4 * i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
// All images, one at a time
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_legacy_master.h5"));
|
|
|
|
for (int i = 0; i < 4; i++) {
|
|
std::vector<SpotToSave> spots;
|
|
REQUIRE_NOTHROW(spots = reader.ReadSpots(i));
|
|
REQUIRE(spots.size() == 2);
|
|
CheckSpotFields(spots[0], spots[1], i);
|
|
}
|
|
}
|
|
|
|
// Out-of-range must throw
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_legacy_master.h5"));
|
|
REQUIRE_THROWS(reader.ReadSpots(4));
|
|
}
|
|
|
|
remove("read_spots_legacy_master.h5");
|
|
remove("read_spots_legacy_data_000001.h5");
|
|
remove("read_spots_legacy_data_000002.h5");
|
|
remove("read_spots_legacy_data_000003.h5");
|
|
remove("read_spots_legacy_data_000004.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_ReadSpots_VDS", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("read_spots_vds")
|
|
.ImagesPerTrigger(4)
|
|
.ImagesPerFile(1)
|
|
.OverwriteExistingFiles(true)
|
|
.BitDepthImage(16)
|
|
.PixelSigned(true)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 0);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
message.spots = MakeTestSpots(i);
|
|
message.spot_count = 72 + i;
|
|
message.spot_count_ice_rings = 45 + 2 * i;
|
|
message.spot_count_low_res = 12 + 3 * i;
|
|
message.spot_count_indexed = 15 + 4 * i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
// All images, one at a time — also verifies that .image carries the
|
|
// correct global index across the virtual-to-source remapping.
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_vds_master.h5"));
|
|
|
|
for (int i = 0; i < 4; i++) {
|
|
std::vector<SpotToSave> spots;
|
|
REQUIRE_NOTHROW(spots = reader.ReadSpots(i));
|
|
REQUIRE(spots.size() == 2);
|
|
CheckSpotFields(spots[0], spots[1], i);
|
|
}
|
|
}
|
|
|
|
// Out-of-range must throw
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_vds_master.h5"));
|
|
REQUIRE_THROWS(reader.ReadSpots(4));
|
|
}
|
|
|
|
// Image with no spots returns an empty vector
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_vds_master.h5"));
|
|
|
|
// Write a separate 2-image VDS file where only image 0 has spots.
|
|
DiffractionExperiment y(DetJF(1));
|
|
y.FilePrefix("read_spots_vds_sparse")
|
|
.ImagesPerTrigger(2)
|
|
.ImagesPerFile(1)
|
|
.OverwriteExistingFiles(true)
|
|
.BitDepthImage(16)
|
|
.PixelSigned(true)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
{
|
|
StartMessage start_message;
|
|
y.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < 2; i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, y.GetXPixelsNum(), y.GetYPixelsNum());
|
|
message.number = i;
|
|
if (i == 0)
|
|
message.spots = MakeTestSpots(i);
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 2;
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
JFJochHDF5Reader sparse_reader;
|
|
REQUIRE_NOTHROW(sparse_reader.ReadFile("read_spots_vds_sparse_master.h5"));
|
|
|
|
std::vector<SpotToSave> spots_0, spots_1;
|
|
REQUIRE_NOTHROW(spots_0 = sparse_reader.ReadSpots(0));
|
|
REQUIRE_NOTHROW(spots_1 = sparse_reader.ReadSpots(1));
|
|
|
|
REQUIRE(spots_0.size() == 2);
|
|
CHECK(spots_0[0].image == 0);
|
|
CHECK(spots_1.empty());
|
|
|
|
remove("read_spots_vds_sparse_master.h5");
|
|
remove("read_spots_vds_sparse_data_000001.h5");
|
|
remove("read_spots_vds_sparse_data_000002.h5");
|
|
}
|
|
|
|
remove("read_spots_vds_master.h5");
|
|
remove("read_spots_vds_data_000001.h5");
|
|
remove("read_spots_vds_data_000002.h5");
|
|
remove("read_spots_vds_data_000003.h5");
|
|
remove("read_spots_vds_data_000004.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_ReadAllSpots_VDS", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("read_spots_vds")
|
|
.ImagesPerTrigger(20)
|
|
.ImagesPerFile(3)
|
|
.OverwriteExistingFiles(true)
|
|
.BitDepthImage(16)
|
|
.PixelSigned(true)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 0);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
message.spots = MakeTestSpots(i);
|
|
message.spot_count = 72 + i;
|
|
message.spot_count_ice_rings = 45 + 2 * i;
|
|
message.spot_count_low_res = 12 + 3 * i;
|
|
message.spot_count_indexed = 15 + 4 * i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_vds_master.h5"));
|
|
|
|
std::shared_ptr<JFJochReaderSpots> ret;
|
|
REQUIRE_NOTHROW(ret = reader.ReadAllSpots(1, 15, 2));
|
|
|
|
// 1,3,5,7,9,11,13,15
|
|
REQUIRE(ret);
|
|
REQUIRE(ret->start_image == 1);
|
|
REQUIRE(ret->stride == 2);
|
|
REQUIRE(ret->spots.size() == 8);
|
|
|
|
for (int i = 0; i < ret->spots.size(); i++) {
|
|
REQUIRE(ret->spots[i].size() == 2);
|
|
CheckSpotFields(ret->spots[i][0], ret->spots[i][1], 2 * i + 1);
|
|
}
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_vds_master.h5"));
|
|
REQUIRE_THROWS(reader.ReadAllSpots(-5,0));
|
|
REQUIRE_THROWS(reader.ReadAllSpots(5,0));
|
|
}
|
|
|
|
remove("read_spots_vds_master.h5");
|
|
remove("read_spots_vds_data_000001.h5");
|
|
remove("read_spots_vds_data_000002.h5");
|
|
remove("read_spots_vds_data_000003.h5");
|
|
remove("read_spots_vds_data_000004.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
|
|
|
|
TEST_CASE("JFJochReader_ReadSpots_Integrated", "[HDF5][Full]") {
|
|
DiffractionExperiment x(DetJF(1));
|
|
|
|
x.FilePrefix("read_spots_integrated")
|
|
.ImagesPerTrigger(4)
|
|
.OverwriteExistingFiles(true)
|
|
.BitDepthImage(16)
|
|
.PixelSigned(true)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxIntegrated)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
std::vector<int16_t> image(x.GetPixelsNum(), 0);
|
|
|
|
RegisterHDF5Filter();
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < x.GetImageNum(); i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
message.spots = MakeTestSpots(i);
|
|
message.spot_count = 72 + i;
|
|
message.spot_count_ice_rings = 45 + 2 * i;
|
|
message.spot_count_low_res = 12 + 3 * i;
|
|
message.spot_count_indexed = 15 + 4 * i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = x.GetImageNum();
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
// All images, one at a time
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_integrated_master.h5"));
|
|
|
|
for (int i = 0; i < 4; i++) {
|
|
std::vector<SpotToSave> spots;
|
|
REQUIRE_NOTHROW(spots = reader.ReadSpots(i));
|
|
REQUIRE(spots.size() == 2);
|
|
CheckSpotFields(spots[0], spots[1], i);
|
|
}
|
|
}
|
|
|
|
// Out-of-range must throw
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_integrated_master.h5"));
|
|
REQUIRE_THROWS(reader.ReadSpots(4));
|
|
}
|
|
|
|
// Image with no spots returns an empty vector
|
|
{
|
|
DiffractionExperiment y(DetJF(1));
|
|
y.FilePrefix("read_spots_integrated_sparse")
|
|
.ImagesPerTrigger(3)
|
|
.OverwriteExistingFiles(true)
|
|
.BitDepthImage(16)
|
|
.PixelSigned(true)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxIntegrated)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
{
|
|
StartMessage start_message;
|
|
y.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
|
|
for (int i = 0; i < 3; i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, y.GetXPixelsNum(), y.GetYPixelsNum());
|
|
message.number = i;
|
|
if (i == 1)
|
|
message.spots = MakeTestSpots(i);
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
}
|
|
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 3;
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("read_spots_integrated_sparse_master.h5"));
|
|
|
|
CHECK(reader.ReadSpots(0).empty());
|
|
REQUIRE(reader.ReadSpots(1).size() == 2);
|
|
CheckSpotFields(reader.ReadSpots(1)[0], reader.ReadSpots(1)[1], 1);
|
|
CHECK(reader.ReadSpots(2).empty());
|
|
|
|
remove("read_spots_integrated_sparse_master.h5");
|
|
}
|
|
|
|
remove("read_spots_integrated_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
}
|
|
|
|
TEST_CASE("JFJochReader_Snapshots", "[HDF5][Full]") {
|
|
RegisterHDF5Filter();
|
|
|
|
DiffractionExperiment x(DetJF(1));
|
|
x.FilePrefix("test_snap").ImagesPerTrigger(4).OverwriteExistingFiles(true);
|
|
x.BitDepthImage(16).ImagesPerFile(1).SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true);
|
|
x.Compression(CompressionAlgorithm::NO_COMPRESSION);
|
|
|
|
// 1. Original dataset: distinct pixels + "original" MX metadata (not indexed, bkg = 10 + i).
|
|
std::vector<int16_t> image(x.GetPixelsNum());
|
|
{
|
|
StartMessage start_message;
|
|
x.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
ScanResultGenerator generator(x);
|
|
for (int i = 0; i < 4; i++) {
|
|
image[5678] = 100 + i;
|
|
DataMessage message{};
|
|
message.image = CompressedImage(image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
message.indexing_result = false;
|
|
message.bkg_estimate = 10.0 + i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 4;
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
// 2. A reprocessing result over the same 4 images: integrated master, "reprocessed" MX
|
|
// metadata (all indexed, bkg = 99 + i) and deliberately wrong pixels that must NOT surface
|
|
// (snapshot pixels still come from the original image source).
|
|
{
|
|
DiffractionExperiment px(x);
|
|
px.FilePrefix("test_snap_proc").SetFileWriterFormat(FileWriterFormat::NXmxIntegrated);
|
|
StartMessage start_message;
|
|
px.FillMessage(start_message);
|
|
FileWriter file_set(start_message);
|
|
ScanResultGenerator generator(px);
|
|
std::vector<int16_t> proc_image(x.GetPixelsNum(), 7);
|
|
for (int i = 0; i < 4; i++) {
|
|
DataMessage message{};
|
|
message.image = CompressedImage(proc_image, x.GetXPixelsNum(), x.GetYPixelsNum());
|
|
message.number = i;
|
|
message.indexing_result = true;
|
|
message.bkg_estimate = 99.0 + i;
|
|
REQUIRE_NOTHROW(file_set.WriteHDF5(message));
|
|
generator.Add(message);
|
|
}
|
|
EndMessage end_message;
|
|
end_message.max_image_number = 4;
|
|
generator.FillEndMessage(end_message);
|
|
file_set.WriteHDF5(end_message);
|
|
file_set.Finalize();
|
|
}
|
|
|
|
{
|
|
JFJochHDF5Reader reader;
|
|
REQUIRE_NOTHROW(reader.ReadFile("test_snap_master.h5"));
|
|
REQUIRE(reader.GetNumberOfImages() == 4);
|
|
CHECK(reader.ActiveSnapshot() == "Original");
|
|
CHECK(reader.SnapshotNames() == std::vector<std::string>{"Original"});
|
|
|
|
// Dataset-level plot arrays come from the active (original) metadata source.
|
|
REQUIRE(reader.GetDataset()->bkg_estimate.size() == 4);
|
|
CHECK(reader.GetDataset()->bkg_estimate[2] == Catch::Approx(12.0));
|
|
|
|
auto orig0 = reader.LoadImage(0);
|
|
REQUIRE(orig0);
|
|
CHECK(orig0->Image()[5678] == 100);
|
|
REQUIRE(orig0->ImageData().indexing_result.has_value());
|
|
CHECK(orig0->ImageData().indexing_result.value() == false);
|
|
CHECK(orig0->ImageData().bkg_estimate.value() == Catch::Approx(10.0));
|
|
|
|
// Register the reprocessing result as a second metadata source over the same images.
|
|
REQUIRE_NOTHROW(reader.RegisterSnapshot("Reprocess", "test_snap_proc_master.h5"));
|
|
{
|
|
auto names = reader.SnapshotNames();
|
|
CHECK(std::find(names.begin(), names.end(), "Original") != names.end());
|
|
CHECK(std::find(names.begin(), names.end(), "Reprocess") != names.end());
|
|
}
|
|
|
|
REQUIRE_NOTHROW(reader.SetActiveSnapshot("Reprocess"));
|
|
CHECK(reader.ActiveSnapshot() == "Reprocess");
|
|
// Plots now come from the reprocessing master.
|
|
CHECK(reader.GetDataset()->bkg_estimate[2] == Catch::Approx(101.0));
|
|
|
|
auto repro0 = reader.LoadImage(0);
|
|
REQUIRE(repro0);
|
|
// Pixels still from the original image source, not the 7's stored in the process file.
|
|
CHECK(repro0->Image()[5678] == 100);
|
|
// Metadata from the reprocessing snapshot.
|
|
CHECK(repro0->ImageData().indexing_result.value() == true);
|
|
CHECK(repro0->ImageData().bkg_estimate.value() == Catch::Approx(99.0));
|
|
|
|
// Switch back to the original metadata.
|
|
REQUIRE_NOTHROW(reader.SetActiveSnapshot("Original"));
|
|
auto orig0b = reader.LoadImage(0);
|
|
REQUIRE(orig0b);
|
|
CHECK(orig0b->Image()[5678] == 100);
|
|
CHECK(orig0b->ImageData().indexing_result.value() == false);
|
|
|
|
REQUIRE_THROWS(reader.SetActiveSnapshot("Nonexistent"));
|
|
}
|
|
|
|
remove("test_snap_master.h5");
|
|
remove("test_snap_data_000001.h5");
|
|
remove("test_snap_data_000002.h5");
|
|
remove("test_snap_data_000003.h5");
|
|
remove("test_snap_data_000004.h5");
|
|
remove("test_snap_proc_master.h5");
|
|
|
|
REQUIRE(H5Fget_obj_count(H5F_OBJ_ALL, H5F_OBJ_ALL) == 0);
|
|
} |