leonarski_fandClaude Fable 5
dc8a80f3e0
The viewer could grow to ~100 GB RAM when live-following an HTTP broker. The rc.153 images_in_flight backpressure only throttled imageLoaded; the heavy per-frame payload rides datasetLoaded, fanned out over ~10 queued cross-thread connections with no cap. In HTTPSyncDataset follow mode (entered when an operator clicks an image while following live) RefreshDatasetOnly_i emits a fresh full dataset every autoload tick with no imageLoaded, so the gate never engaged and the queued events - each pinning a full JFJochReaderDataset (full-detector PixelMask + per-image plots) - accumulated without bound. Backpressure datasetLoaded the same way as imageLoaded: a datasets_in_flight counter (cap 2), all emits routed through EmitDatasetLoaded_i, and AutoLoadTimerExpired gated on it (covers HTTPSyncDataset). The window routes the worker's datasetLoaded through a single OnDatasetReady sink that fans out synchronously via datasetReady and acks with datasetConsumed. Under load stale datasets are dropped; the next tick sends the latest. Share the pixel mask instead of deep-copying it: JFJochReaderDataset::pixel_mask is now shared_ptr<const PixelMask>, so per-frame dataset copies share the ~72 MB mask. UpdateUserMask does copy-on-write; JFJochHttpReader caches the mask by arm_date so a live refresh reuses one shared mask per acquisition. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
56 lines
2.1 KiB
C++
56 lines
2.1 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include "JFJochReader.h"
|
|
#include "../common/BrokerStatus.h"
|
|
#include "../common/ImageBuffer.h"
|
|
#include "../common/ROIDefinition.h"
|
|
|
|
class JFJochHttpReader : public JFJochReader {
|
|
mutable std::mutex http_mutex;
|
|
std::string addr;
|
|
|
|
std::optional<int64_t> last_image_buffer_counter;
|
|
bool last_op_http_sync = false;
|
|
|
|
// Cache of the (constant-per-acquisition) pixel mask, keyed on arm date, so the per-refresh
|
|
// dataset rebuild reuses one shared mask instead of reconstructing it every tick.
|
|
std::shared_ptr<const PixelMask> cached_pixel_mask;
|
|
std::string cached_pixel_mask_arm_date;
|
|
|
|
ImageBufferStatus GetImageBufferStatus() const;
|
|
|
|
bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
|
|
DataMessage& message,
|
|
std::vector<uint8_t> &buffer,
|
|
int64_t image_number,
|
|
bool update_dataset) override;
|
|
std::shared_ptr<JFJochReaderDataset> UpdateDataset_i();
|
|
std::vector<float> GetPlot_i(const std::string &plot_type, float fill_value = 0.0) const;
|
|
public:
|
|
~JFJochHttpReader() override = default;
|
|
|
|
void ReadURL(const std::string& url);
|
|
uint64_t GetNumberOfImages() const override;
|
|
void Close() override;
|
|
|
|
// Refresh dataset/plots if the image buffer changed since the last poll (returns nullptr if
|
|
// unchanged). Always writes the current number of images to num_images_out. Does not load an image.
|
|
std::shared_ptr<const JFJochReaderDataset> RefreshDatasetIfChanged(int64_t &num_images_out);
|
|
|
|
void UploadUserMask(const std::vector<uint32_t>& mask);
|
|
|
|
[[nodiscard]] ROIDefinition GetROIDefinitions() const; // GET /config/roi
|
|
void UploadROIDefinitions(const ROIDefinition &rois) const; // PUT /config/roi
|
|
|
|
BrokerStatus GetBrokerStatus() const;
|
|
|
|
std::shared_ptr<JFJochReaderRawImage> GetRawImage(int64_t image_number) override;
|
|
std::vector<SpotToSave> ReadSpots(int64_t image) const override;
|
|
};
|
|
|
|
|
|
|