Files
Jungfraujoch/viewer/JFJochHttpReader.h
T
leonarski_fandFilip Leonarski 19b135186d
Build Packages / Create release (push) Successful in 7s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 4m56s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 9m10s
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 13m51s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 16m56s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 20m59s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 21m25s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 21m17s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 14m14s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 25m59s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 25m48s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 15m52s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 19m33s
Build Packages / Generate python client (push) Successful in 29s
Build Packages / Build documentation (push) Successful in 1m19s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 20m49s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 25m31s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 25m24s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 26m5s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 22m19s
Build Packages / Unit tests (push) Failing after 2h28m24s
Build Packages / Validation Battery (push) Successful in 40m33s
v1.0.0-rc.175 (#85)
* Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`).
* Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection.
* Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS).
* Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead.
* Rugnux: On macOS, Bragg integration runs on the Apple GPU.
* Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand.
* Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`.
* Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit.
* Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed.
* Rugnux: The beam-stop shadow is found correctly on tilted detectors.
* Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement.
* jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps.
* jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection.
* jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images.
* Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges.

---------

Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home>
Reviewed-on: #85
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-10-16 15:34:43 +02:00

106 lines
4.8 KiB
C++

// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include "../reader/JFJochReader.h"
#include "../common/BrokerStatus.h"
#include "../common/ImageBuffer.h"
#include "../common/ROIDefinition.h"
#include <stdexcept>
// cpp-httplib's client, kept opaque here so this widely-included header does NOT pull in
// <httplib.h>. On Windows httplib.h drags in <windows.h>, whose min/max/ERROR/LoadImage macros
// would then clobber every viewer translation unit that includes this header (breaking std::min /
// std::max in gemmi, the `ERROR` enum, the LoadImage() calls, ...). <httplib.h> is included only in
// JFJochHttpReader.cpp.
namespace httplib { class Client; }
// The broker refused the request with 401/403: the current dataset is protected by tokens and the
// one we hold (or its absence) does not open it. Distinct from "broker unreachable" so the viewer
// can report it quietly instead of raising an error dialog.
struct HttpUnauthorized : public std::runtime_error {
using std::runtime_error::runtime_error;
};
class JFJochHttpReader : public JFJochReader {
mutable std::mutex http_mutex;
std::string addr;
// Sent as "Authorization: Bearer <token>" with every request while non-empty.
std::string bearer_token;
// Persistent HTTP(S) client: created lazily on the first request (i.e. as soon as an address
// is selected) and then reused for every subsequent request so the TCP/TLS connection is kept
// alive. httplib transparently reopens the connection if it was dropped (idle timeout,
// server-side session limit, ...). client_mutex serialises access because one client must never
// be used from two threads at once; it is a leaf lock (nothing else is taken while it is held),
// so it cannot deadlock with http_mutex.
mutable std::mutex client_mutex;
mutable std::unique_ptr<httplib::Client> client;
// Path part of addr (e.g. "/jfjoch" behind a reverse proxy), prepended to every request path;
// the client itself is constructed from the scheme://host:port part only.
mutable std::string path_prefix;
std::optional<int64_t> last_image_buffer_counter;
bool last_op_http_sync = false;
// Cache of the (constant-per-acquisition) pixel mask, keyed on arm date, so the per-refresh
// dataset rebuild reuses one shared mask instead of reconstructing it every tick.
std::shared_ptr<const PixelMask> cached_pixel_mask;
std::string cached_pixel_mask_arm_date;
// Minimal HTTP result: whether a response arrived (transport-level success), the HTTP status
// code, and the (possibly binary) response body.
struct HttpResult {
bool ok = false;
long status = 0;
std::string body;
};
// Single HTTP request helper behind every endpoint call. method is "GET" or "PUT"; for PUT
// the body is sent with the given content_type.
HttpResult Request(const std::string &method, const std::string &path,
const std::string &body = {}, const std::string &content_type = {}) const;
// Tear down the persistent connection (on disconnect, when a new address is selected, and in
// the destructor); the next Request transparently re-opens it.
void ResetConnection() const;
ImageBufferStatus GetImageBufferStatus() const;
bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
DataMessage& message,
std::vector<uint8_t> &buffer,
int64_t image_number,
bool update_dataset) override;
std::shared_ptr<JFJochReaderDataset> UpdateDataset_i();
std::vector<float> GetPlot_i(const std::string &plot_type, float fill_value = 0.0) const;
public:
// Both defined in the .cpp, where httplib::Client is a complete type.
JFJochHttpReader();
~JFJochHttpReader() override;
void ReadURL(const std::string& url);
// The dataset token; kept across Close() so a reconnect to the same broker reuses it.
void Token(const std::string &token);
uint64_t GetNumberOfImages() const override;
void Close() override;
// Refresh dataset/plots if the image buffer changed since the last poll (returns nullptr if
// unchanged). Always writes the current number of images to num_images_out. Does not load an image.
std::shared_ptr<const JFJochReaderDataset> RefreshDatasetIfChanged(int64_t &num_images_out);
void UploadUserMask(const std::vector<uint32_t>& mask);
[[nodiscard]] ROIDefinition GetROIDefinitions() const; // GET /config/roi
void UploadROIDefinitions(const ROIDefinition &rois) const; // PUT /config/roi
BrokerStatus GetBrokerStatus() const;
bool ReadRawImage(int64_t image_number, JFJochReaderRawImage &image) override;
std::vector<SpotToSave> ReadSpots(int64_t image) const override;
};