leonarski_f
67dca388bd
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:cuda (push) Successful in 18m44s
Build Packages / build:viewer-tgz:cpu (push) Successful in 6m11s
Build Packages / build:viewer-tgz:cuda (push) Successful in 6m54s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m40s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m41s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m10s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m4s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m23s
Build Packages / build:rpm (rocky8) (push) Successful in 11m30s
Build Packages / build:rpm (rocky9) (push) Successful in 12m51s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m8s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m21s
Build Packages / DIALS test (push) Successful in 13m22s
Build Packages / XDS test (durin plugin) (push) Successful in 9m2s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m55s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m57s
Build Packages / Generate python client (push) Successful in 23s
Build Packages / Build documentation (push) Successful in 57s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 10m24s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell. * rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged. * rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set. * rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme. * rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free. * rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry. * Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md. * Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #70 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
1881 lines
77 KiB
C++
1881 lines
77 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include <catch2/catch_all.hpp>
|
|
|
|
#include <fstream>
|
|
|
|
#include "../receiver/JFJochReceiverTest.h"
|
|
#include "../acquisition_device/HLSSimulatedDevice.h"
|
|
#include "../writer/HDF5Objects.h"
|
|
#include "../receiver/JFJochReceiverService.h"
|
|
#include "../preview/JFJochTIFF.h"
|
|
#include "../image_pusher/ZMQStream2Pusher.h"
|
|
#include "../writer/StreamWriter.h"
|
|
#include "../image_pusher/NonePusher.h"
|
|
#include "../image_pusher/HDF5FilePusher.h"
|
|
#include "../image_pusher/TCPStreamPusher.h"
|
|
#include "../image_puller/TCPImagePuller.h"
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_refinement", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_refinement").JungfrauConvPhotonCnt(false).SetFileWriterFormat(
|
|
FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90})
|
|
.GeomRefinementAlgorithm(GeomRefinementAlgorithmEnum::BeamCenter);
|
|
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_refinement_tetragonal", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_refinement_tetragonal").JungfrauConvPhotonCnt(false).SetFileWriterFormat(
|
|
FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 78.95, .b = 78.95, .c = 36.9, .alpha = 90, .beta = 90, .gamma = 90})
|
|
.GeomRefinementAlgorithm(GeomRefinementAlgorithmEnum::BeamCenter)
|
|
.SpaceGroupNumber(96);
|
|
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_index_and_integrate", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
settings.quick_integration = true;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
logger.Info("Processing time {}", receiver_out.processing_time.processing);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
auto b_factor = service.GetDataProcessingPlot(
|
|
PlotRequest{.type = PlotType::BFactor, .binning = 1});
|
|
|
|
REQUIRE(b_factor.GetPlots().size() == 1);
|
|
REQUIRE(b_factor.GetPlots()[0].x.size() == experiment.GetImageNum());
|
|
logger.Info("B-factor {}", b_factor.GetPlots()[0].y[0]);
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_index_and_integrate_no_spots", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 20; // make sure there are zero spots possible
|
|
settings.photon_count_threshold = 10000;
|
|
settings.min_pix_per_spot = 5;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
settings.quick_integration = true;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 0.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
logger.Info("Processing time {}", receiver_out.processing_time.processing);
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_azint_2d", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_azim_2d").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
AzimuthalIntegrationSettings azim;
|
|
azim.QSpacing_recipA(0.05).QRange_recipA(0.05, 6.05).AzimuthalBinCount(16);
|
|
experiment.ImportAzimuthalIntegrationSettings(azim);
|
|
REQUIRE(azim.GetBinCount() == 6 * 20 * 16);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
auto azim_plot = service.GetDataProcessingPlot(PlotRequest{.type = PlotType::AzInt, .binning = 1});
|
|
REQUIRE(azim_plot.GetPlots().size() == 1);
|
|
|
|
REQUIRE(azim_plot.GetPlots()[0].x.size() == 120 * 16);
|
|
REQUIRE(azim_plot.GetPlots()[0].y.size() == 120 * 16);
|
|
REQUIRE(azim_plot.GetPlots()[0].z.size() == 120 * 16);
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFT", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_fft").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT);
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFTW", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_fftw").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFTW);
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFT_reference", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_fft_ref").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_FFT_wrong_reference", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_fft_wrongref").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS)
|
|
.OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.IndexingAlgorithm(IndexingAlgorithmEnum::FFT)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 55.0, .alpha =90, .beta = 90, .gamma = 90});
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 0.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_grid", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
GridScanSettings grid_scan(3, 10.0f, 25.0f, false, true);
|
|
DatasetSettings dataset_settings;
|
|
dataset_settings.ImagesPerTrigger(7).NumTriggers(1).DetectorDistance_mm(75)
|
|
.BeamY_pxl(1136).BeamX_pxl(1090)
|
|
.PhotonEnergy_keV(12.4).GridScan(grid_scan)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.UseInternalPacketGenerator(true).ImagesPerFile(2).ImportDatasetSettings(dataset_settings)
|
|
.FilePrefix("crystal_test_grid").JungfrauConvPhotonCnt(false)
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true);
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// 3x3 grid = 9 cells, but only 7 images collected; the two uncollected
|
|
// placeholder rows (grid indices 5 and 8) are dropped from scan_result.
|
|
REQUIRE(receiver_out.scan_result.images.size() == 7);
|
|
CHECK(receiver_out.scan_result.images[0].number == 0);
|
|
CHECK(receiver_out.scan_result.images[1].number == 3);
|
|
CHECK(receiver_out.scan_result.images[3].number == 1);
|
|
|
|
auto grid_scan_out = service.GetDataProcessingPlot(
|
|
PlotRequest{.type = PlotType::IndexingRate, .experimental_coord = true});
|
|
|
|
CHECK(grid_scan_out.GetUnits() == MultiLinePlotUnits::Grid_um);
|
|
REQUIRE(grid_scan_out.GetPlots().size() == 1);
|
|
REQUIRE(grid_scan_out.GetPlots()[0].z.size() == 7);
|
|
|
|
CHECK(grid_scan_out.GetPlots()[0].z[0] == 1.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].z[1] == 1.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].z[2] == 1.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].z[3] == 1.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].z[4] == 1.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].z[5] == 1.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].z[6] == 1.0f);
|
|
|
|
CHECK(grid_scan_out.GetPlots()[0].x[5] == 1.5 * 10.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].y[5] == 2.5 * 25.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].x[6] == 2.5 * 10.0f);
|
|
CHECK(grid_scan_out.GetPlots()[0].y[6] == 0.5 * 25.0f);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_32", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_32").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxLegacy).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4).BitDepthImage(32).PixelSigned(true)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
pusher.WriterNotificationSocket("ipc://*");
|
|
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_1.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_crystal_spot_and_index_min_pix_2", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).OverwriteExistingFiles(true)
|
|
.FilePrefix("crystal_test_min_pix_2").JungfrauConvPhotonCnt(false).SetFileWriterFormat(
|
|
FileWriterFormat::NXmxVDS)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90})
|
|
.PixelSigned(true);
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {2,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 3;
|
|
settings.min_pix_per_spot = 2;
|
|
settings.max_pix_per_spot = 200;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg, jpeg_indexed;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true
|
|
};
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_min_pix_2.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
REQUIRE_NOTHROW(jpeg_indexed = service.GetJPEGFromBuffer(jpeg_settings, ImageBuffer::MaxIndexedImage));
|
|
REQUIRE(!jpeg_indexed.empty());
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.indexing_rate == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("GenerateResolutionMap") {
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
|
|
.FilePrefix("crystal_test").JungfrauConvPhotonCnt(false)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
|
|
std::vector<float> spot_finder_resolution_map(experiment.GetModulesNum() * RAW_MODULE_SIZE, 1.0);
|
|
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
experiment.CalcSpotFinderResolutionMap(spot_finder_resolution_map.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
std::vector<uint32_t> spot_finder_resolution_map_int(spot_finder_resolution_map.size());
|
|
for (int i = 0; i < spot_finder_resolution_map.size(); i++)
|
|
spot_finder_resolution_map_int[i] = static_cast<uint32_t>(spot_finder_resolution_map[i] * 100);
|
|
std::vector<uint32_t> spot_finder_resolution_map_int_conv(experiment.GetPixelsNum(), 0);
|
|
|
|
RawToConvertedGeometry(experiment, spot_finder_resolution_map_int_conv.data(), spot_finder_resolution_map_int.data());
|
|
|
|
CompressedImage image(spot_finder_resolution_map_int_conv, experiment.GetXPixelsNum(),
|
|
experiment.GetYPixelsNum() );
|
|
WriteTIFFToFile("ResolutionMap.tiff", image);
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQ_ROI", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
|
|
.FilePrefix("crystal_test_roi").JungfrauConvPhotonCnt(false)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90})
|
|
.SetFileWriterFormat(FileWriterFormat::NXmxVDS).PixelSigned(true).OverwriteExistingFiles(true);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
experiment.ROI().SetROI(ROIDefinition{
|
|
.boxes = {ROIBox("beam", 100, 120, 20, 30)},
|
|
.circles = {ROICircle("roi1", 500, 800, 10)},
|
|
.azimuthal = {ROIAzimuthal("roi2", 4.0, 7.0)}
|
|
});
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {5,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
uint64_t roi_value = 0;
|
|
uint64_t pixels = 0;
|
|
for (int y = 20; y <= 30; y++) {
|
|
for (int x = 100; x <= 120; x++) {
|
|
int16_t val = image_conv[experiment.GetXPixelsNum() * y + x];
|
|
if ((val != INT16_MIN) && (val != INT16_MAX)) {
|
|
pixels += 1;
|
|
roi_value += val;
|
|
}
|
|
}
|
|
}
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
auto &mask = pixel_mask.GetMaskRaw();
|
|
|
|
uint64_t sat_pixels = 0;
|
|
uint64_t err_pixels = 0;
|
|
for (int i = 0; i < image_raw_geom.size(); i++) {
|
|
if (mask[i] == 0) {
|
|
if (image_raw_geom[i] == INT16_MAX)
|
|
sat_pixels++;
|
|
else if (image_raw_geom[i] == INT16_MIN)
|
|
err_pixels++;
|
|
}
|
|
}
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
ZMQStream2Pusher pusher({"ipc://*"});
|
|
ZMQImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
std::string jpeg;
|
|
PreviewImageSettings jpeg_settings{
|
|
.saturation_value = 10,
|
|
.jpeg_quality = 100,
|
|
.show_spots = true,
|
|
.show_roi = true
|
|
};
|
|
|
|
REQUIRE_NOTHROW(jpeg = service.GetJPEGFromBuffer(jpeg_settings));
|
|
|
|
REQUIRE(!jpeg.empty());
|
|
std::ofstream f("crystal_processing_test_roi.jpeg", std::ios::binary);
|
|
f.write(jpeg.data(), jpeg.size());
|
|
|
|
auto plot = service.GetDataProcessingPlot(PlotRequest{.type = PlotType::ROISum, .binning = 1});
|
|
|
|
REQUIRE(plot.GetPlots().size() == 3);
|
|
CHECK(plot.GetPlots()[0].title == "beam");
|
|
REQUIRE(!plot.GetPlots()[0].x.empty());
|
|
CHECK(plot.GetPlots()[0].x[0] == 0);
|
|
CHECK(plot.GetPlots()[0].y[0] == roi_value);
|
|
CHECK(plot.GetPlots()[1].title == "roi1");
|
|
CHECK(plot.GetPlots()[2].title == "roi2");
|
|
|
|
CHECK(receiver_out.status.roi_beam_sum == roi_value);
|
|
CHECK(receiver_out.status.roi_beam_npixel == pixels);
|
|
REQUIRE(receiver_out.status.error_pixels.has_value());
|
|
REQUIRE(receiver_out.status.saturated_pixels.has_value());
|
|
|
|
CHECK(receiver_out.status.error_pixels.value() == err_pixels);
|
|
CHECK(receiver_out.status.saturated_pixels.value() == sat_pixels);
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQPreview", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
|
|
.FilePrefix("").JungfrauConvPhotonCnt(false)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
aq_devices.Add(std::make_unique<HLSSimulatedDevice>(0, 64));
|
|
|
|
NonePusher pusher;
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
service.PreviewSocket("tcp://0.0.0.0:*", 16);
|
|
service.PreviewSocketSettings({.period = std::chrono::seconds(0)}); // Send one message
|
|
|
|
ZMQSocket sub_socket(ZMQSocketType::Sub);
|
|
sub_socket.ReceiveWaterMark(10);
|
|
sub_socket.Connect(service.GetPreviewSocketAddress());
|
|
sub_socket.SubscribeAll();
|
|
sub_socket.ReceiveTimeout(std::chrono::seconds(1)); // Ensure Subscriber is connected
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.images_collected == 5);
|
|
CHECK(receiver_out.status.images_sent == 0);
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
ZMQMessage msg1;
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
|
|
// start message
|
|
auto out = CBORStream2Deserialize(msg1.data(), msg1.size());
|
|
REQUIRE(out != nullptr);
|
|
CHECK(out->msg_type == CBORImageType::START);
|
|
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
out = CBORStream2Deserialize(msg1.data(), msg1.size());
|
|
REQUIRE(out != nullptr);
|
|
CHECK(out->msg_type == CBORImageType::IMAGE);
|
|
|
|
// 4 other images
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
|
|
// end message
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
REQUIRE(!sub_socket.Receive(msg1, true));
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_ZMQMetadata", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
|
|
.FilePrefix("").JungfrauConvPhotonCnt(false)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
aq_devices.Add(std::make_unique<HLSSimulatedDevice>(0, 64));
|
|
|
|
NonePusher pusher;
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
service.MetadataSocket("tcp://0.0.0.0:*");
|
|
service.MetadataSocketSettings({.period = std::chrono::seconds(2500)}); // Send all metadata as one message
|
|
|
|
ZMQSocket sub_socket(ZMQSocketType::Sub);
|
|
sub_socket.ReceiveWaterMark(10);
|
|
sub_socket.Connect(service.GetMetadataSocketAddress());
|
|
sub_socket.SubscribeAll();
|
|
sub_socket.ReceiveTimeout(std::chrono::seconds(1)); // Ensure Subscriber is connected
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.images_collected == 5);
|
|
CHECK(receiver_out.status.images_sent == 0);
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
ZMQMessage msg1;
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
|
|
auto out = CBORStream2Deserialize(msg1.data(), msg1.size());
|
|
REQUIRE(out != nullptr);
|
|
CHECK(out->msg_type == CBORImageType::START);
|
|
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
out = CBORStream2Deserialize(msg1.data(), msg1.size());
|
|
REQUIRE(out != nullptr);
|
|
CHECK(out->msg_type == CBORImageType::METADATA);
|
|
CHECK(out->metadata);
|
|
CHECK(out->metadata->images.size() == 5);
|
|
|
|
// end message
|
|
REQUIRE(sub_socket.Receive(msg1, true));
|
|
REQUIRE(!sub_socket.Receive(msg1, true));
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_HDF5FilePusher", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
|
|
.FilePrefix("HDF5FilePusherTest").JungfrauConvPhotonCnt(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4).OverwriteExistingFiles(true);
|
|
|
|
JFCalibration calibration(experiment);
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
aq_devices.Add(std::make_unique<HLSSimulatedDevice>(0, 64));
|
|
|
|
HDF5FilePusher pusher;
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
service.Start(experiment, pixel_mask, &calibration);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.images_collected == 5);
|
|
CHECK(receiver_out.status.images_sent == 5);
|
|
CHECK(!receiver_out.status.cancelled);
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_HDF5FilePusher_republish", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
const std::string repub_addr = "tcp://127.0.0.1:9132";
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
|
|
.FilePrefix("HDF5FilePusherRepublishTest").JungfrauConvPhotonCnt(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.OverwriteExistingFiles(true);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
AcquisitionDeviceGroup aq_devices;
|
|
aq_devices.Add(std::make_unique<HLSSimulatedDevice>(0, 64));
|
|
|
|
HDF5FilePusher pusher(repub_addr);
|
|
|
|
ZMQSocket pull_socket(ZMQSocketType::Pull);
|
|
pull_socket.ReceiveWaterMark(16);
|
|
pull_socket.Connect(repub_addr);
|
|
pull_socket.ReceiveTimeout(std::chrono::seconds(1));
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.images_collected == 5);
|
|
CHECK(receiver_out.status.images_sent == 5);
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
ZMQMessage msg;
|
|
|
|
REQUIRE(pull_socket.Receive(msg, true));
|
|
auto out = CBORStream2Deserialize(msg.data(), msg.size());
|
|
REQUIRE(out != nullptr);
|
|
CHECK(out->msg_type == CBORImageType::START);
|
|
|
|
for (int i = 0; i < experiment.GetImageNum(); i++) {
|
|
REQUIRE(pull_socket.Receive(msg, true));
|
|
out = CBORStream2Deserialize(msg.data(), msg.size());
|
|
REQUIRE(out != nullptr);
|
|
CHECK(out->msg_type == CBORImageType::IMAGE);
|
|
}
|
|
|
|
REQUIRE(pull_socket.Receive(msg, true));
|
|
out = CBORStream2Deserialize(msg.data(), msg.size());
|
|
REQUIRE(out != nullptr);
|
|
CHECK(out->msg_type == CBORImageType::END);
|
|
|
|
REQUIRE(!pull_socket.Receive(msg, true));
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_HDF5FilePusher_Raw", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF(1));
|
|
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true)
|
|
.FilePrefix("HDF5FilePusherTest_Raw").OverwriteExistingFiles(true);
|
|
experiment.Raw();
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
aq_devices.Add(std::make_unique<HLSSimulatedDevice>(0, 64));
|
|
|
|
HDF5FilePusher pusher;
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.images_collected == 5);
|
|
CHECK(receiver_out.status.images_sent == 5);
|
|
CHECK(!receiver_out.status.cancelled);
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_TCP_crystal_no_processing", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_tcp_no_processing").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
|
|
TCPImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.enable = false;
|
|
settings.indexing = true;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(!receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(receiver_out.writer_err.empty());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
|
|
auto ack = pusher.GetImagesWritten();
|
|
REQUIRE(ack.has_value());
|
|
CHECK(ack == experiment.GetImageNum());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_TCP_crystal_spot_and_index", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_tcp").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
|
|
|
|
TCPImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(receiver_out.writer_err.empty());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
|
|
auto ack = pusher.GetImagesWritten();
|
|
REQUIRE(ack.has_value());
|
|
CHECK(ack == experiment.GetImageNum());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_TCP_calibration", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2).SaveCalibration(true)
|
|
.FilePrefix("calib_integration_test").JungfrauConvPhotonCnt(true).SetFileWriterFormat(FileWriterFormat::NXmxVDS).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
JFCalibration calibration(experiment);
|
|
|
|
for (int i = 0; i < experiment.GetModulesNum(); i++) {
|
|
std::vector<int16_t> pedestal_g0(RAW_MODULE_SIZE, i);
|
|
std::vector<int16_t> pedestal_g1(RAW_MODULE_SIZE);
|
|
std::vector<int16_t> pedestal_g2(RAW_MODULE_SIZE);
|
|
for (int j = 0; j < RAW_MODULE_SIZE; j++) {
|
|
pedestal_g1[j] = j % RAW_MODULE_COLS;
|
|
pedestal_g2[j] = i * (RAW_MODULE_LINES) + j / RAW_MODULE_COLS; // line number
|
|
}
|
|
calibration.Pedestal(i,0).LoadPedestal(pedestal_g0);
|
|
calibration.Pedestal(i,1).LoadPedestal(pedestal_g1);
|
|
calibration.Pedestal(i,2).LoadPedestal(pedestal_g2);
|
|
}
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
|
|
|
|
TCPImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
service.Start(experiment, pixel_mask, &calibration);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
}
|
|
|
|
TEST_CASE("JFJochIntegrationTest_TCP_crystal_spot_and_index_single_file", "[JFJochReceiver]") {
|
|
Logger logger(Catch::getResultCapture().getCurrentTestName());
|
|
|
|
RegisterHDF5Filter();
|
|
|
|
const uint16_t nthreads = 4;
|
|
|
|
DiffractionExperiment experiment(DetJF4M());
|
|
experiment.ImagesPerTrigger(5).NumTriggers(1).UseInternalPacketGenerator(true).ImagesPerFile(2)
|
|
.FilePrefix("crystal_test_tcp_single_file").JungfrauConvPhotonCnt(false).SetFileWriterFormat(FileWriterFormat::NXmxIntegrated).OverwriteExistingFiles(true)
|
|
.DetectorDistance_mm(75).BeamY_pxl(1136).BeamX_pxl(1090).IncidentEnergy_keV(12.4)
|
|
.SetUnitCell(UnitCell{.a = 36.9, .b = 78.95, .c = 78.95, .alpha =90, .beta = 90, .gamma = 90});
|
|
experiment.SampleTemperature_K(123.0).RingCurrent_mA(115);
|
|
|
|
PixelMask pixel_mask(experiment);
|
|
|
|
// Load example image
|
|
HDF5ReadOnlyFile data("../../tests/test_data/compression_benchmark.h5");
|
|
HDF5DataSet dataset(data, "/entry/data/data");
|
|
HDF5DataSpace file_space(dataset);
|
|
|
|
REQUIRE(file_space.GetDimensions()[2] == experiment.GetXPixelsNum());
|
|
REQUIRE(file_space.GetDimensions()[1] == experiment.GetYPixelsNum());
|
|
std::vector<int16_t> image_conv (file_space.GetDimensions()[1] * file_space.GetDimensions()[2]);
|
|
|
|
std::vector<hsize_t> start = {4,0,0};
|
|
std::vector<hsize_t> file_size = {1, file_space.GetDimensions()[1], file_space.GetDimensions()[2]};
|
|
dataset.ReadVector(image_conv, start, file_size);
|
|
|
|
std::vector<int16_t> image_raw_geom(experiment.GetModulesNum() * RAW_MODULE_SIZE);
|
|
ConvertedToRawGeometry(experiment, image_raw_geom.data(), image_conv.data());
|
|
logger.Info("Loaded image");
|
|
|
|
// Setup acquisition device
|
|
AcquisitionDeviceGroup aq_devices;
|
|
std::unique_ptr<HLSSimulatedDevice> test = std::make_unique<HLSSimulatedDevice>(0, 64);
|
|
for (int m = 0; m < experiment.GetModulesNum(); m++)
|
|
test->SetInternalGeneratorFrame((uint16_t *) image_raw_geom.data() + m * RAW_MODULE_SIZE, m);
|
|
|
|
aq_devices.Add(std::move(test));
|
|
|
|
TCPStreamPusher pusher("tcp://127.0.0.1:*", 1);
|
|
|
|
TCPImagePuller puller(pusher.GetAddress()[0]);
|
|
StreamWriter writer(logger, puller);
|
|
auto writer_future = std::async(std::launch::async, &StreamWriter::Run, &writer);
|
|
|
|
JFJochReceiverService service(aq_devices, logger, pusher);
|
|
service.NumThreads(nthreads);
|
|
service.Indexing(experiment.GetIndexingSettings());
|
|
|
|
// No progress value at the start of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
SpotFindingSettings settings = DiffractionExperiment::DefaultDataProcessingSettings();
|
|
settings.signal_to_noise_threshold = 2.5;
|
|
settings.photon_count_threshold = 5;
|
|
settings.min_pix_per_spot = 1;
|
|
settings.max_pix_per_spot = 200;
|
|
settings.high_resolution_limit = 2.0;
|
|
settings.low_resolution_limit = 50.0;
|
|
service.SetSpotFindingSettings(settings);
|
|
|
|
service.Start(experiment, pixel_mask, nullptr);
|
|
auto receiver_out = service.Stop();
|
|
|
|
CHECK(receiver_out.efficiency == 1.0);
|
|
REQUIRE(receiver_out.status.indexing_rate);
|
|
CHECK(receiver_out.status.indexing_rate.value() == 1.0);
|
|
CHECK(receiver_out.status.images_sent == experiment.GetImageNum());
|
|
CHECK(receiver_out.writer_err.empty());
|
|
CHECK(!receiver_out.status.cancelled);
|
|
|
|
// No progress value at the end of measurement
|
|
REQUIRE(!service.GetProgress().has_value());
|
|
|
|
REQUIRE_NOTHROW(writer_future.get());
|
|
|
|
auto ack = pusher.GetImagesWritten();
|
|
REQUIRE(ack.has_value());
|
|
CHECK(ack == experiment.GetImageNum());
|
|
}
|