leonarski_fandClaude Opus 5
4bac6d3f21
GetRawImage() allocated a fresh JFJochReaderRawImage per frame, and on a miniCBF sweep its buffer holds the whole decoded image: 72.6 MB on a 4150 x 4371 detector. That is over glibc's 32 MB mmap ceiling, so the block is mmapped and munmapped every image and the decoder faults in 17 700 untouched pages as it writes them. Measured on one 18.1 Mpx frame, decoding into a fresh buffer takes 61 ms against 16 ms into one that has been written before - the allocation costs nearly three times what the decode does, and the per-image loop pays it twice per sweep. ReadRawImage() fills a raw image the caller owns, so a worker declares one outside its loop and keeps the pages. GetRawImage() stays as the allocating wrapper for the single-image callers. The two loops that hand the image to the process-file writer keep a shared_ptr and recycle it only while the writer thread is not still reading the previous frame's pixels. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
88 lines
3.9 KiB
C++
88 lines
3.9 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include "../reader/JFJochReader.h"
|
|
#include "../common/BrokerStatus.h"
|
|
#include "../common/ImageBuffer.h"
|
|
#include "../common/ROIDefinition.h"
|
|
|
|
// libcurl's easy handle, kept opaque here so this widely-included header does NOT pull in
|
|
// <curl/curl.h>. On Windows curl.h drags in <windows.h>, whose min/max/ERROR/LoadImage macros
|
|
// would then clobber every viewer translation unit that includes this header (breaking std::min /
|
|
// std::max in gemmi, the `ERROR` enum, the LoadImage() calls, ...). CURL is `typedef void` in
|
|
// libcurl; <curl/curl.h> is included only in JFJochHttpReader.cpp.
|
|
using CURL = void;
|
|
|
|
class JFJochHttpReader : public JFJochReader {
|
|
mutable std::mutex http_mutex;
|
|
std::string addr;
|
|
|
|
// Persistent libcurl handle: created lazily on the first request (i.e. as soon as an address
|
|
// is selected) and then reused for every subsequent request so the TCP/TLS connection is kept
|
|
// alive. libcurl transparently reopens the connection if it was dropped (idle timeout,
|
|
// server-side session limit, ...). curl_mutex serialises access because a single easy handle
|
|
// must never be used from two threads at once; it is a leaf lock (nothing else is taken while
|
|
// it is held), so it cannot deadlock with http_mutex.
|
|
mutable std::mutex curl_mutex;
|
|
mutable CURL *curl_handle = nullptr;
|
|
|
|
std::optional<int64_t> last_image_buffer_counter;
|
|
bool last_op_http_sync = false;
|
|
|
|
// Cache of the (constant-per-acquisition) pixel mask, keyed on arm date, so the per-refresh
|
|
// dataset rebuild reuses one shared mask instead of reconstructing it every tick.
|
|
std::shared_ptr<const PixelMask> cached_pixel_mask;
|
|
std::string cached_pixel_mask_arm_date;
|
|
|
|
// Minimal HTTP result: whether a response arrived (transport-level success), the HTTP status
|
|
// code, and the (possibly binary) response body.
|
|
struct HttpResult {
|
|
bool ok = false;
|
|
long status = 0;
|
|
std::string body;
|
|
};
|
|
// Single libcurl request helper behind every endpoint call. method is "GET" or "PUT"; for PUT
|
|
// the body is sent with the given content_type.
|
|
HttpResult Request(const std::string &method, const std::string &path,
|
|
const std::string &body = {}, const std::string &content_type = {}) const;
|
|
|
|
// Tear down the persistent connection (on disconnect, when a new address is selected, and in
|
|
// the destructor); the next Request transparently re-opens it.
|
|
void ResetConnection() const;
|
|
|
|
ImageBufferStatus GetImageBufferStatus() const;
|
|
|
|
bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
|
|
DataMessage& message,
|
|
std::vector<uint8_t> &buffer,
|
|
int64_t image_number,
|
|
bool update_dataset) override;
|
|
std::shared_ptr<JFJochReaderDataset> UpdateDataset_i();
|
|
std::vector<float> GetPlot_i(const std::string &plot_type, float fill_value = 0.0) const;
|
|
public:
|
|
~JFJochHttpReader() override;
|
|
|
|
void ReadURL(const std::string& url);
|
|
uint64_t GetNumberOfImages() const override;
|
|
void Close() override;
|
|
|
|
// Refresh dataset/plots if the image buffer changed since the last poll (returns nullptr if
|
|
// unchanged). Always writes the current number of images to num_images_out. Does not load an image.
|
|
std::shared_ptr<const JFJochReaderDataset> RefreshDatasetIfChanged(int64_t &num_images_out);
|
|
|
|
void UploadUserMask(const std::vector<uint32_t>& mask);
|
|
|
|
[[nodiscard]] ROIDefinition GetROIDefinitions() const; // GET /config/roi
|
|
void UploadROIDefinitions(const ROIDefinition &rois) const; // PUT /config/roi
|
|
|
|
BrokerStatus GetBrokerStatus() const;
|
|
|
|
bool ReadRawImage(int64_t image_number, JFJochReaderRawImage &image) override;
|
|
std::vector<SpotToSave> ReadSpots(int64_t image) const override;
|
|
};
|
|
|
|
|
|
|