Files
Jungfraujoch/tests/JFJochReceiverProcessingTest.cpp
leonarski_f 4809c1622c
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m43s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 28m46s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 32m5s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 39m44s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m7s
Build Packages / Create release (push) Successful in 8s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 22m30s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 31m54s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m29s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 23m51s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 26m7s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 29m38s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 28m6s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 29m44s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 33m22s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 34m6s
Build Packages / Unit tests (push) Successful in 1h2m47s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 35m59s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 34m8s
Build Packages / Validation Battery (push) Successful in 38m31s
v1.0.0-rc.187 (#98)
* Rugnux: Refinement several times faster, one protocol on the CPU and the GPU, parts of it on the Apple GPU; its random draws are the same on every platform.
* Rugnux: Refinement follows common practice: the free set decides only the restraint weights and the displacement model (isotropic, TLS or anisotropic, by Hamilton's test), everything else is decided on the working set; lower R-free for placed and predicted models.
* Rugnux: Restraint dictionaries via `--restraints` (or among the inputs) and more built-in ligands; a model whose space group is a subgroup of the data's is refined in its own group; reflections plus a model select `--mode refine` on their own.
* Rugnux: `-C` fixes the crystal system on the axes as typed; `-S` alone fixes the space group only, the cell is still found and reduced; a space group or cell stored in the input file is ignored, with a line in the log.
* Rugnux: More reliable multi-sweep merging; `--mode scale` merges several stills runs as one; `_plot.txt` and the report name each input; a second crystal form is reported for stills given a cell.
* jfjoch_broker (experimental): Live Rugnux on FPGA detectors (JUNGFRAU) takes the same path as a DECTRIS stream, rotation and serial alike, model step included, with analysis threads per GPU; frames waiting for the analysis can be viewed; no host stage drops a frame - a source faster than the analysis is held at the card.
* jfjoch_broker: Summation of more than 256 frames per image is split between the card and the host (`summation_mode` is then `hybrid`), and live Rugnux runs on such collections; pedestal runs take a receiver of their own.
* jfjoch_broker (experimental): `stills_downsample` analyses 1 of every N serial images (chosen by a hash; the first 2000 all; the others written without analysis; reset to 1 by `/initialize`); `max_spot_count` left out means 250 for per-image indexing and up to 1000 for live stills; the live status gives the merge before the model work, the fitted resolution and the anomalous peak height of each unexplained blob.
* jfjoch_broker: Without a `pcie` entry the broker runs on a mock of the card instead of the HLS simulation; its spot, ROI and azimuthal results are zero then.
* Web frontend: Redesigned Rugnux live page; Vite 8, TypeScript 6, Mol* 5.13 and dependency security updates.
* Tools: `jfjoch_fpga_stills_test` measures live stills on the cards with a stored dataset; `jfjoch_fpga_test` loses the HLS-simulation option.
* Other: FFTW no longer fetched or linked (the CPU FFT indexer keeps the name `FFTW`); PCIe driver now GPL-2.0-or-later and third-party notices complete; several crash fixes.

Reviewed-on: #98
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-11-19 20:49:28 +01:00

2080 lines
88 KiB
C++

// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#include <fstream>
#include "../receiver/JFJochReceiverTest.h"
#include "MockCardOutput.h"
#include "../writer/HDF5Objects.h"
#include "../receiver/JFJochReceiverService.h"
#include "../preview/JFJochTIFF.h"
#include "../image_pusher/ZMQStream2Pusher.h"
#include "../writer/StreamWriter.h"
#include "../image_pusher/NonePusher.h"
#include "../image_pusher/HDF5FilePusher.h"
#include "../image_pusher/TCPStreamPusher.h"
#include "../image_puller/TCPImagePuller.h"
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_GPUStills", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_gpu_stills").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device
AcquisitionDeviceGroup aq_devices;
auto test = std::make_unique<MockAcquisitionDevice>(0, 64);
for (int m = 0; m < experiment.GetModulesNum(); m++)
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
aq_devices.Add(std::move(test));
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// Serial data with rugnux on: the converted frames are analysed on the GPU, after the pre-pass.
service.RugnuxLive(true, nullptr, 4, 60);
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
// Integration in live stills is rugnux's own, whatever the broker's spot-finding settings say.
settings.quick_integration = false;
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// Done once everything is written, with the latency from the last image to then
const auto live = service.GetRugnuxStatus();
CHECK(live.state == RugnuxLiveState::Done);
CHECK(!live.space_group.empty());
REQUIRE(live.last_image_to_written_s);
REQUIRE(live.last_image_s);
CHECK(live.elapsed_s >= live.last_image_s.value() + live.last_image_to_written_s.value() - 1e-6);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_GPUStills_writer_down", "[JFJochReceiver]") {
// The start message goes out after the pre-pass: with no writer listening, the collection ends with
// the error, and the next one runs.
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_gpu_stills_writer_down").JungfrauConvPhotonCnt(false)
.SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
AcquisitionDeviceGroup aq_devices;
auto test = std::make_unique<MockAcquisitionDevice>(0, 64);
for (int m = 0; m < experiment.GetModulesNum(); m++)
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
aq_devices.Add(std::move(test));
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(4);
service.Indexing(experiment.GetIndexingSettings());
service.RugnuxLive(true, nullptr, 4, 60);
service.Start(experiment, pixel_mask, nullptr);
REQUIRE_THROWS_AS(service.Stop(), JFJochException);
REQUIRE(!service.GetProgress().has_value());
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_GPUStills_GeometryRefinedOnModules", "[JFJochReceiver]") {
// 60 strong frames of an 8-module detector: the pre-pass refines the geometry on them and indexes them again at
// the fit (live, the modules stay as given), so headers 2% apart in distance end at the same distance.
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const auto run = [&](float distance_mm) {
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(60).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("gpu_stills_modules").JungfrauConvPhotonCnt(false)
.DetectorDistance_mm(distance_mm).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha = 90, .beta = 90, .gamma = 90});
REQUIRE(experiment.GetModulesNum() == 8);
PixelMask pixel_mask(experiment);
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
std::vector<int16_t> image_conv(file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
const std::vector<hsize_t> start = {4, 0, 0};
const std::vector<hsize_t> size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
AcquisitionDeviceGroup aq_devices;
auto test = std::make_unique<MockAcquisitionDevice>(0, 64);
for (int m = 0; m < experiment.GetModulesNum(); m++)
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
aq_devices.Add(std::move(test));
NonePusher pusher;
// The pre-pass holds all 60 frames, one to a slot, in half of the image buffer
JFJochReceiverService service(aq_devices, logger, pusher,
2 * 60 * experiment.GetImageBufferLocationSize() / (1024 * 1024) + 1);
service.NumThreads(4);
service.Indexing(experiment.GetIndexingSettings());
service.RugnuxLive(true, nullptr, 4, 60);
service.Start(experiment, pixel_mask, nullptr);
service.Stop();
return service.GetRugnuxStatus();
};
const auto refined = [](const RugnuxLiveSnapshot &s) {
for (const auto &step: s.steps)
for (const auto &note: step.notes)
if (note.starts_with("Geometry refined from"))
return true;
return false;
};
const auto a = run(75.0f), b = run(76.5f);
REQUIRE(a.stills);
REQUIRE(b.stills);
CHECK(refined(a));
CHECK(refined(b));
REQUIRE(a.stills->distance_mm);
REQUIRE(b.stills->distance_mm);
logger.Info("Refined distance {:.3f} mm from 75 mm, {:.3f} mm from 76.5 mm", a.stills->distance_mm.value(),
b.stills->distance_mm.value());
CHECK(b.stills->distance_mm.value() == Catch::Approx(a.stills->distance_mm.value()).epsilon(0.001));
}
TEST_CASE("JFJochIntegrationTest_GPUStills_cancel_during_prepass", "[JFJochReceiver]") {
// A cancel while the frames are held for the pre-pass ends the collection.
Logger logger(Catch::getResultCapture().getCurrentTestName());
// Fewer frames arrive than the pre-pass waits for, and no final packet: the frames stay held until the cancel.
// How long the stop takes is not checked.
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(30).NumTriggers(1).UseInternalPacketGenerator(false)
.FilePrefix("gpu_stills_cancel").JungfrauConvPhotonCnt(false)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
AcquisitionDeviceGroup aq_devices;
auto test = std::make_unique<MockAcquisitionDevice>(0, 64);
for (int m = 0; m < experiment.GetModulesNum(); m++)
test->CreateJFPackets(experiment, 1, 10, m, nullptr);
aq_devices.Add(std::move(test));
NonePusher pusher;
// The pre-pass's 30 frames fit, one to a slot, in half of the image buffer
JFJochReceiverService service(aq_devices, logger, pusher,
2 * 30 * experiment.GetImageBufferLocationSize() / (1024 * 1024) + 1);
service.NumThreads(4);
service.Indexing(experiment.GetIndexingSettings());
service.RugnuxLive(true, nullptr, 4, 60);
service.Start(experiment, pixel_mask, nullptr);
std::this_thread::sleep_for(std::chrono::seconds(3));
const auto t0 = std::chrono::steady_clock::now();
service.Cancel(false);
JFJochReceiverOutput receiver_out;
REQUIRE_NOTHROW(receiver_out = service.Stop());
logger.Info("Stopped {:.1f} s after the cancel",
std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count());
CHECK(receiver_out.status.cancelled);
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_refinement", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_refinement").JungfrauConvPhotonCnt(false).SetFileWriterFormat(
FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90})
.GeomRefinementAlgorithm(GeomRefinementAlgorithmEnum::BeamCenter);
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_refinement_tetragonal", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_refinement_tetragonal").JungfrauConvPhotonCnt(false).SetFileWriterFormat(
FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 78.95, .b = 78.95, .c = 36.9, .alpha = 90, .beta = 90, .gamma = 90})
.GeomRefinementAlgorithm(GeomRefinementAlgorithmEnum::BeamCenter)
.SpaceGroupNumber(96);
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_index_no_integration", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
settings.quick_integration = true; // asked for, and still not done
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
logger.Info("Processing time {}", receiver_out.processing_time.processing);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
// The FPGA receiver indexes each image and integrates none, even when the settings ask for it:
// integration is rugnux's.
REQUIRE(receiver_out.scan_result.images.size() == experiment.GetImageNum());
for (const auto &i: receiver_out.scan_result.images) {
CHECK(i.uc.has_value());
CHECK(!i.integrated_reflections.has_value());
CHECK(!i.b_factor.has_value());
}
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_index_no_spots", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 20; // make sure there are zero spots possible
settings.photon_count_threshold = 10000;
settings.min_pix_per_spot = 5;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 0.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
logger.Info("Processing time {}", receiver_out.processing_time.processing);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_azint_2d", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_azim_2d").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
AzimuthalIntegrationSettings azim;
azim.QSpacing_recipA(0.05).QRange_recipA(0.05, 6.05).AzimuthalBinCount(16);
experiment.ImportAzimuthalIntegrationSettings(azim);
REQUIRE(azim.GetBinCount() == 6 * 20 * 16);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom,
DiffractionExperiment::DefaultDataProcessingSettings()));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
auto azim_plot = service.GetDataProcessingPlot(PlotRequest{.type = PlotType::AzInt, .binning = 1});
REQUIRE(azim_plot.GetPlots().size() == 1);
REQUIRE(azim_plot.GetPlots()[0].x.size() == 120 * 16);
REQUIRE(azim_plot.GetPlots()[0].y.size() == 120 * 16);
REQUIRE(azim_plot.GetPlots()[0].z.size() == 120 * 16);
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFT", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_fft").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
.OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFTW", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_fftw").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
.OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.IndexingAlgorithm(IndexingAlgorithmEnum::FFTW);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFT_reference", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_fft_ref").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
.OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFT_wrong_reference", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_fft_wrongref").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
.OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 55.0, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 0.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_grid", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
GridScanSettings grid_scan(3, 10.0f, 25.0f, false, true);
DatasetSettings dataset_settings;
dataset_settings.ImagesPerTrigger(7).NumTriggers(1).DetectorDistance_mm(75)
.BeamY_pxl(1136).BeamX_pxl(1090)
.PhotonEnergy_keV(12.4).GridScan(grid_scan)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
DiffractionExperiment experiment(DetJF4M());
experiment.UseInternalPacketGenerator(true).ImagesPerFile(2).ImportDatasetSettings(dataset_settings)
.FilePrefix("crystal_test_grid").JungfrauConvPhotonCnt(false)
.SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// 3x3 grid = 9 cells, but only 7 images collected; the two uncollected
// placeholder rows (grid indices 5 and 8) are dropped from scan_result.
REQUIRE(receiver_out.scan_result.images.size() == 7);
CHECK(receiver_out.scan_result.images[0].number == 0);
CHECK(receiver_out.scan_result.images[1].number == 3);
CHECK(receiver_out.scan_result.images[3].number == 1);
auto grid_scan_out = service.GetDataProcessingPlot(
PlotRequest{.type = PlotType::IndexingRate, .experimental_coord = true});
CHECK(grid_scan_out.GetUnits() == MultiLinePlotUnits::Grid_um);
REQUIRE(grid_scan_out.GetPlots().size() == 1);
REQUIRE(grid_scan_out.GetPlots()[0].z.size() == 7);
CHECK(grid_scan_out.GetPlots()[0].z[0] == 1.0f);
CHECK(grid_scan_out.GetPlots()[0].z[1] == 1.0f);
CHECK(grid_scan_out.GetPlots()[0].z[2] == 1.0f);
CHECK(grid_scan_out.GetPlots()[0].z[3] == 1.0f);
CHECK(grid_scan_out.GetPlots()[0].z[4] == 1.0f);
CHECK(grid_scan_out.GetPlots()[0].z[5] == 1.0f);
CHECK(grid_scan_out.GetPlots()[0].z[6] == 1.0f);
CHECK(grid_scan_out.GetPlots()[0].x[5] == 1.5 * 10.0f);
CHECK(grid_scan_out.GetPlots()[0].y[5] == 2.5 * 25.0f);
CHECK(grid_scan_out.GetPlots()[0].x[6] == 2.5 * 10.0f);
CHECK(grid_scan_out.GetPlots()[0].y[6] == 0.5 * 25.0f);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_32", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_32").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4).BitDepthImage(32).PixelSigned(true)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
pusher.WriterNotificationSocket("ipc://*");
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_min_pix_2", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).OverwriteExistingFiles(true)
.FilePrefix("crystal_test_min_pix_2").JungfrauConvPhotonCnt(false).SetFileWriterFormat(
FileWriterFormat::NXmxVDS)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90})
.PixelSigned(true);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {2,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 3;
settings.min_pix_per_spot = 2;
settings.max_pix_per_spot = 200;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg, jpeg_indexed;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_min_pix_2.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
REQUIRE_NOTHROW(jpeg_indexed = service.GetJPEGFromBuffer(jpeg_settings, ImageBuffer::MaxIndexedImage));
REQUIRE(!jpeg_indexed.empty());
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.indexing_rate == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("GenerateResolutionMap") {
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
std::vector<float> spot_finder_resolution_map(experiment.GetModulesNum() * RAW_MODULE_SIZE, 1.0);
for (int m = 0; m < experiment.GetModulesNum(); m++)
experiment.CalcSpotFinderResolutionMap(spot_finder_resolution_map.data() + m * RAW_MODULE_SIZE, m);
std::vector<uint32_t> spot_finder_resolution_map_int(spot_finder_resolution_map.size());
for (int i = 0; i < spot_finder_resolution_map.size(); i++)
spot_finder_resolution_map_int[i] = static_cast<uint32_t>(spot_finder_resolution_map[i] * 100);
std::vector<uint32_t> spot_finder_resolution_map_int_conv(experiment.GetPixelsNum(), 0);
RawToConvertedGeometry(experiment, spot_finder_resolution_map_int_conv.data(), spot_finder_resolution_map_int.data());
CompressedImage image(spot_finder_resolution_map_int_conv, experiment.GetXPixelsNum(),
experiment.GetYPixelsNum() );
WriteTIFFToFile("ResolutionMap.tiff", image);
}
TEST_CASE("JFJochIntegrationTest_ZMQ_ROI", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("crystal_test_roi").JungfrauConvPhotonCnt(false)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90})
.SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true).OverwriteExistingFiles(true);
PixelMask pixel_mask(experiment);
experiment.ROI().SetROI(ROIDefinition{
.boxes = {ROIBox("beam", 100, 120, 20, 30)},
.circles = {ROICircle("roi1", 500, 800, 10)},
.azimuthal = {ROIAzimuthal("roi2", 4.0, 7.0)}
});
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {5,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
uint64_t roi_value = 0;
uint64_t pixels = 0;
for (int y = 20; y <= 30; y++) {
for (int x = 100; x <= 120; x++) {
int16_t val = image_conv[experiment.GetXPixelsNum() * y + x];
if ((val != INT16_MIN) && (val != INT16_MAX)) {
pixels += 1;
roi_value += val;
}
}
}
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
auto &mask = pixel_mask.GetMaskRaw();
uint64_t sat_pixels = 0;
uint64_t err_pixels = 0;
for (int i = 0; i < image_raw_geom.size(); i++) {
if (mask[i] == 0) {
if (image_raw_geom[i] == INT16_MAX)
sat_pixels++;
else if (image_raw_geom[i] == INT16_MIN)
err_pixels++;
}
}
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom,
DiffractionExperiment::DefaultDataProcessingSettings()));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
ZMQStream2Pusher pusher({"ipc://*"});
ZMQImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
std::string jpeg;
PreviewImageSettings jpeg_settings{
.saturation_value = 10,
.jpeg_quality = 100,
.show_spots = true,
.show_roi = true
};
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
REQUIRE(!jpeg.empty());
std::ofstream f("crystal_processing_test_roi.jpeg", std::ios::binary);
f.write(jpeg.data(), jpeg.size());
auto plot = service.GetDataProcessingPlot(PlotRequest{.type = PlotType::ROISum, .binning = 1});
REQUIRE(plot.GetPlots().size() == 3);
CHECK(plot.GetPlots()[0].title == "beam");
REQUIRE(!plot.GetPlots()[0].x.empty());
CHECK(plot.GetPlots()[0].x[0] == 0);
CHECK(plot.GetPlots()[0].y[0] == roi_value);
CHECK(plot.GetPlots()[1].title == "roi1");
CHECK(plot.GetPlots()[2].title == "roi2");
CHECK(receiver_out.status.roi_beam_sum == roi_value);
CHECK(receiver_out.status.roi_beam_npixel == pixels);
REQUIRE(receiver_out.status.error_pixels.has_value());
REQUIRE(receiver_out.status.saturated_pixels.has_value());
CHECK(receiver_out.status.error_pixels.value() == err_pixels);
CHECK(receiver_out.status.saturated_pixels.value() == sat_pixels);
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_ZMQPreview", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("").JungfrauConvPhotonCnt(false)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4);
PixelMask pixel_mask(experiment);
// Setup acquisition device
AcquisitionDeviceGroup aq_devices;
aq_devices.Add(std::make_unique<MockAcquisitionDevice>(0, 64));
NonePusher pusher;
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.PreviewSocket("tcp://0.0.0.0:*", 16);
service.PreviewSocketSettings({.period = std::chrono::seconds(0)}); // Send one message
ZMQSocket sub_socket(ZMQSocketType::Sub);
sub_socket.ReceiveWaterMark(10);
sub_socket.Connect(service.GetPreviewSocketAddress());
sub_socket.SubscribeAll();
sub_socket.ReceiveTimeout(std::chrono::seconds(1)); // Ensure Subscriber is connected
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.images_collected == 5);
CHECK(receiver_out.status.images_sent == 0);
CHECK(!receiver_out.status.cancelled);
ZMQMessage msg1;
REQUIRE(sub_socket.Receive(msg1, true));
// start message
auto out = CBORStream2Deserialize(msg1.data(), msg1.size());
REQUIRE(out != nullptr);
CHECK(out->msg_type == CBORImageType::START);
REQUIRE(sub_socket.Receive(msg1, true));
out = CBORStream2Deserialize(msg1.data(), msg1.size());
REQUIRE(out != nullptr);
CHECK(out->msg_type == CBORImageType::IMAGE);
// 4 other images
REQUIRE(sub_socket.Receive(msg1, true));
REQUIRE(sub_socket.Receive(msg1, true));
REQUIRE(sub_socket.Receive(msg1, true));
REQUIRE(sub_socket.Receive(msg1, true));
// end message
REQUIRE(sub_socket.Receive(msg1, true));
REQUIRE(!sub_socket.Receive(msg1, true));
}
TEST_CASE("JFJochIntegrationTest_ZMQMetadata", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("").JungfrauConvPhotonCnt(false)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4);
PixelMask pixel_mask(experiment);
// Setup acquisition device
AcquisitionDeviceGroup aq_devices;
aq_devices.Add(std::make_unique<MockAcquisitionDevice>(0, 64));
NonePusher pusher;
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.MetadataSocket("tcp://0.0.0.0:*");
service.MetadataSocketSettings({.period = std::chrono::seconds(2500)}); // Send all metadata as one message
ZMQSocket sub_socket(ZMQSocketType::Sub);
sub_socket.ReceiveWaterMark(10);
sub_socket.Connect(service.GetMetadataSocketAddress());
sub_socket.SubscribeAll();
sub_socket.ReceiveTimeout(std::chrono::seconds(1)); // Ensure Subscriber is connected
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.images_collected == 5);
CHECK(receiver_out.status.images_sent == 0);
CHECK(!receiver_out.status.cancelled);
ZMQMessage msg1;
REQUIRE(sub_socket.Receive(msg1, true));
auto out = CBORStream2Deserialize(msg1.data(), msg1.size());
REQUIRE(out != nullptr);
CHECK(out->msg_type == CBORImageType::START);
REQUIRE(sub_socket.Receive(msg1, true));
out = CBORStream2Deserialize(msg1.data(), msg1.size());
REQUIRE(out != nullptr);
CHECK(out->msg_type == CBORImageType::METADATA);
CHECK(out->metadata);
CHECK(out->metadata->images.size() == 5);
// end message
REQUIRE(sub_socket.Receive(msg1, true));
REQUIRE(!sub_socket.Receive(msg1, true));
}
TEST_CASE("JFJochIntegrationTest_HDF5FilePusher", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("HDF5FilePusherTest").JungfrauConvPhotonCnt(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4).OverwriteExistingFiles(true);
JFCalibration calibration(experiment);
PixelMask pixel_mask(experiment);
// Setup acquisition device
AcquisitionDeviceGroup aq_devices;
aq_devices.Add(std::make_unique<MockAcquisitionDevice>(0, 64));
HDF5FilePusher pusher;
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.Start(experiment, pixel_mask, &calibration);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.images_collected == 5);
CHECK(receiver_out.status.images_sent == 5);
CHECK(!receiver_out.status.cancelled);
}
TEST_CASE("JFJochIntegrationTest_HDF5FilePusher_republish", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
const std::string repub_addr = "tcp://127.0.0.1:9132";
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("HDF5FilePusherRepublishTest").JungfrauConvPhotonCnt(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.OverwriteExistingFiles(true);
PixelMask pixel_mask(experiment);
AcquisitionDeviceGroup aq_devices;
aq_devices.Add(std::make_unique<MockAcquisitionDevice>(0, 64));
HDF5FilePusher pusher(repub_addr);
ZMQSocket pull_socket(ZMQSocketType::Pull);
pull_socket.ReceiveWaterMark(16);
pull_socket.Connect(repub_addr);
pull_socket.ReceiveTimeout(std::chrono::seconds(1));
std::this_thread::sleep_for(std::chrono::milliseconds(100));
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.images_collected == 5);
CHECK(receiver_out.status.images_sent == 5);
CHECK(!receiver_out.status.cancelled);
ZMQMessage msg;
REQUIRE(pull_socket.Receive(msg, true));
auto out = CBORStream2Deserialize(msg.data(), msg.size());
REQUIRE(out != nullptr);
CHECK(out->msg_type == CBORImageType::START);
for (int i = 0; i < experiment.GetImageNum(); i++) {
REQUIRE(pull_socket.Receive(msg, true));
out = CBORStream2Deserialize(msg.data(), msg.size());
REQUIRE(out != nullptr);
CHECK(out->msg_type == CBORImageType::IMAGE);
}
REQUIRE(pull_socket.Receive(msg, true));
out = CBORStream2Deserialize(msg.data(), msg.size());
REQUIRE(out != nullptr);
CHECK(out->msg_type == CBORImageType::END);
REQUIRE(!pull_socket.Receive(msg, true));
}
TEST_CASE("JFJochIntegrationTest_HDF5FilePusher_Raw", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF(1));
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
.FilePrefix("HDF5FilePusherTest_Raw").OverwriteExistingFiles(true);
experiment.Raw();
PixelMask pixel_mask(experiment);
// Setup acquisition device
AcquisitionDeviceGroup aq_devices;
aq_devices.Add(std::make_unique<MockAcquisitionDevice>(0, 64));
HDF5FilePusher pusher;
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.images_collected == 5);
CHECK(receiver_out.status.images_sent == 5);
CHECK(!receiver_out.status.cancelled);
}
TEST_CASE("JFJochIntegrationTest_TCP_crystal_no_processing", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_tcp_no_processing").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.enable = false;
settings.indexing = true;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
TCPImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
CHECK(!receiver_out.status.indexing_rate);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(receiver_out.writer_err.empty());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
auto ack = pusher.GetImagesWritten();
REQUIRE(ack.has_value());
CHECK(ack == experiment.GetImageNum());
}
TEST_CASE("JFJochIntegrationTest_TCP_crystal_spot_and_index", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_tcp").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
TCPImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(receiver_out.writer_err.empty());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
auto ack = pusher.GetImagesWritten();
REQUIRE(ack.has_value());
CHECK(ack == experiment.GetImageNum());
}
TEST_CASE("JFJochIntegrationTest_TCP_calibration", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2).SaveCalibration(true)
.FilePrefix("calib_integration_test").JungfrauConvPhotonCnt(true).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
JFCalibration calibration(experiment);
for (int i = 0; i < experiment.GetModulesNum(); i++) {
std::vector<int16_t> pedestal_g0(RAW_MODULE_SIZE, i);
std::vector<int16_t> pedestal_g1(RAW_MODULE_SIZE);
std::vector<int16_t> pedestal_g2(RAW_MODULE_SIZE);
for (int j = 0; j < RAW_MODULE_SIZE; j++) {
pedestal_g1[j] = j % RAW_MODULE_COLS;
pedestal_g2[j] = i * (RAW_MODULE_LINES) + j / RAW_MODULE_COLS; // line number
}
calibration.Pedestal(i,0).LoadPedestal(pedestal_g0);
calibration.Pedestal(i,1).LoadPedestal(pedestal_g1);
calibration.Pedestal(i,2).LoadPedestal(pedestal_g2);
}
// Setup acquisition device
AcquisitionDeviceGroup aq_devices;
aq_devices.Add(std::make_unique<MockAcquisitionDevice>(0, 64));
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
TCPImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
service.Start(experiment, pixel_mask, &calibration);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
}
TEST_CASE("JFJochIntegrationTest_TCP_crystal_spot_and_index_single_file", "[JFJochReceiver]") {
Logger logger(Catch::getResultCapture().getCurrentTestName());
RegisterHDF5Filter();
const uint16_t nthreads = 4;
DiffractionExperiment experiment(DetJF4M());
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
.FilePrefix("crystal_test_tcp_single_file").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxIntegrated).OverwriteExistingFiles(true)
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
PixelMask pixel_mask(experiment);
// Load example image
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
HDF5DataSet dataset(data, "/entry/data/data");
HDF5DataSpace file_space(dataset);
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
std::vector<hsize_t> start = {4,0,0};
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
dataset.ReadVector(image_conv, start, file_size);
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
logger.Info("Loaded image");
// Setup acquisition device: the card writes the image, with the strong pixels of its spot finder
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
settings.signal_to_noise_threshold = 2.5;
settings.photon_count_threshold = 5;
settings.min_pix_per_spot = 1;
settings.max_pix_per_spot = 200;
settings.high_resolution_limit = 2.0;
settings.low_resolution_limit = 50.0;
std::vector<CardImage> images;
images.push_back(Int16Image(experiment, pixel_mask, image_raw_geom, settings));
AcquisitionDeviceGroup aq_devices = MockDevices(experiment, images);
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
TCPImagePuller puller(pusher.GetAddress()[0]);
StreamWriter writer(logger, puller);
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
JFJochReceiverService service(aq_devices, logger, pusher);
service.NumThreads(nthreads);
service.Indexing(experiment.GetIndexingSettings());
// No progress value at the start of measurement
REQUIRE(!service.GetProgress().has_value());
service.SetSpotFindingSettings(settings);
service.Start(experiment, pixel_mask, nullptr);
auto receiver_out = service.Stop();
CHECK(receiver_out.efficiency == 1.0);
REQUIRE(receiver_out.status.indexing_rate);
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
CHECK(receiver_out.writer_err.empty());
CHECK(!receiver_out.status.cancelled);
// No progress value at the end of measurement
REQUIRE(!service.GetProgress().has_value());
REQUIRE_NOTHROW(writer_future.get());
auto ack = pusher.GetImagesWritten();
REQUIRE(ack.has_value());
CHECK(ack == experiment.GetImageNum());
}