Build Packages / build:rpm (rocky8) (push) Successful in 24m0s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m54s
Build Packages / build:windows:cuda (push) Successful in 19m25s
Build Packages / build:viewer-tgz:cpu (push) Successful in 14m44s
Build Packages / build:viewer-tgz:cuda (push) Successful in 16m3s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 13m15s
Build Packages / build:rugnux:windows (push) Successful in 10m45s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 9m34s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 19m7s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 18m9s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 24m48s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 18m13s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 24m51s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 22m58s
Build Packages / build:rpm (rocky9) (push) Successful in 21m23s
Build Packages / Generate python client (push) Successful in 1m2s
Build Packages / Build documentation (push) Successful in 1m23s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 9m45s
Build Packages / XDS test (neggia plugin) (push) Successful in 10m19s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m10s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 22m15s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 17m37s
Build Packages / DIALS test (push) Successful in 17m16s
* `rugnux --model` adopts the model's space group as a label where the data were merged in its enantiomorph, instead of reindexing the reflections - which swapped I(+) with I(-). * `rugnux --model` warns, naming the atom, when the anomalous density at the model's atoms comes out inverted, which means the data and the model are in opposite hands. * `rugnux --model` writes an anomalous difference map (`<prefix>_anom.ccp4`) when the merge kept the Bijvoet split, and names the ten model atoms it peaks highest on as `ANOMALOUS_SITE_01`..`_10`. * `MEAN_ATOM_DENSITY_SIGMA` is read from the map by cubic rather than linear interpolation and comes out around a tenth higher; it is no longer comparable with the figure earlier versions printed. * `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name. * A model `rugnux --model` cannot use is reported as a `WARNING:` line in the results report instead of only in the log. * The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given; `REPORT_VERSION` is 4, `WARNINGS` moves to section 11 and no existing key changed. * The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=`, `WALL_TIME=` and `GPU_COUNT=` / `GPU=`. * rugnux says which GPUs it can see before it starts processing. * `rugnux --export-unmerged` also writes `<prefix>_unmerged.mtz` on a `--no-merge` run, and is ignored on a run with no output prefix instead of writing a file called `_unmerged.mtz`. * `/start` asks the writer whether the run can be written before the detector is armed, so a run whose master file already exists, or whose output directory cannot be created, is refused up front with the writer's own message. This needs the TCP image stream or the built-in HDF5 writer; the ZeroMQ stream is unchanged. * A calibration that fails goes to `Error` carrying the reason instead of `Inactive`, so `/wait_till_done` and `/wait_until_running` report it; a cancelled calibration still goes to `Inactive`. * `/wait_till_done` answers 500 with the message when a collection ended in an error. A cancelled collection and a collection that only triggered a warning still answer 200. * A pending start failure is discarded by `/cancel` and `/deactivate`, as it already was by `/start` and `/initialize`. * `/scan_result` no longer reports the previous run's images after a collection that failed to start, or after `/deactivate`. * The TCP image stream protocol version is 4. `jfjoch_writer` and `jfjoch_broker` have to be of the same release, as before. Reviewed-on: #75
79 lines
3.3 KiB
C++
79 lines
3.3 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <vector>
|
|
|
|
#include "../common/DiffractionExperiment.h"
|
|
#include "../common/JFJochMessages.h"
|
|
#include "../common/ZeroCopyReturnValue.h"
|
|
#include "../common/Logger.h"
|
|
#include "../frame_serialize/CBORStream2Serializer.h"
|
|
|
|
enum class ImagePusherType {HDF5, CBOR, TCP, ZMQ, Test, None};
|
|
|
|
struct ImagePusherStatus {
|
|
ImagePusherType pusher_type = ImagePusherType::None;
|
|
std::vector<std::string> address = {};
|
|
size_t connected_writers = 0;
|
|
std::optional<size_t> images_written = 0;
|
|
std::optional<size_t> images_write_error = 0;
|
|
std::vector<int64_t> writer_fifo_utilization;
|
|
};
|
|
|
|
struct ImagePusherQueueElement {
|
|
const uint8_t *image_data;
|
|
ZeroCopyReturnValue *z;
|
|
bool end;
|
|
};
|
|
|
|
void PrepareCBORImage(DataMessage& message,
|
|
const DiffractionExperiment &experiment,
|
|
void *image, size_t image_size);
|
|
|
|
class ImagePusher {
|
|
protected:
|
|
std::vector<uint8_t> serialization_buffer;
|
|
CBORStream2Serializer serializer;
|
|
ImagePusher();
|
|
public:
|
|
// Dry run before the detector is armed: ask the writer whether it could write this run's
|
|
// output, and throw if it could not. Only transports with a back-channel can answer, so the
|
|
// default is to say nothing - the ZeroMQ pusher is fire-and-forget and keeps failing at the
|
|
// final rename instead. Called from the same serialized control thread as StartDataCollection.
|
|
virtual void Preflight(StartMessage& message) {}
|
|
virtual void StartDataCollection(StartMessage& message) = 0;
|
|
virtual bool EndDataCollection(const EndMessage& message) = 0; // Non-blocking
|
|
virtual bool SendImage(const uint8_t *image_data, size_t image_size, int64_t image_number) = 0;
|
|
// Returns true if the image was accepted (handed to the writer / enqueued for
|
|
// sending), false if it was dropped (e.g. no TCP connection, or the enqueue
|
|
// deadline expired). Ownership is settled either way: on a false return the
|
|
// slot has already been release()d. The authoritative count of images actually
|
|
// delivered is GetImagesWritten() (ACK-based for TCP).
|
|
virtual bool SendImage(ZeroCopyReturnValue &z);
|
|
virtual bool SendCalibration(const CompressedImage& message) = 0;
|
|
virtual std::string Finalize(); // Ensure that all streams are closed, can throw exception
|
|
virtual std::string GetWriterNotificationSocketAddress() const;
|
|
virtual ~ImagePusher() = default;
|
|
virtual std::string PrintSetup() const = 0;
|
|
|
|
virtual std::vector<std::string> GetAddress() const { return {}; }
|
|
virtual std::optional<uint64_t> GetImagesWritten() const { return std::nullopt; }
|
|
virtual std::optional<uint64_t> GetImagesWriteError() const { return std::nullopt; }
|
|
virtual size_t GetConnectedWriters() const { return 0; };
|
|
virtual ImagePusherType GetType() const = 0;
|
|
virtual std::vector<int64_t> GetWriterFifoUtilization() const { return {}; }
|
|
|
|
ImagePusherStatus GetStatus() const {
|
|
return ImagePusherStatus{
|
|
GetType(),
|
|
GetAddress(),
|
|
GetConnectedWriters(),
|
|
GetImagesWritten(),
|
|
GetImagesWriteError(),
|
|
GetWriterFifoUtilization()
|
|
};
|
|
}
|
|
};
|