leonarski_fandClaude Opus 5.5
f849e2d1be
The two pre-scan steps that were still CPU-bound in a GPU build now run where the projection already is. - FindBeamCenterFromBackground: the per-iteration binning pass and the two clip rounds run on the device (BeamCenterBackgroundGPU); the fit itself stays on the host. Each cell is summed in the host's order (pixel order within the host's row blocks, blocks in order), and the per-pixel cell/derivative formula is shared (BackgroundBand.h). The angles come from BackgroundAtan2 (IEEE ops only) instead of atan2f, and both translation units are compiled without FMA contraction, so host and device give the same bits: 0 of 6.5 M pixels in a different cell, identical walks on the three in-house rotation sets. With glibc/CUDA atan2f and default contraction ~30 pixels per 16 Mpx sweep changed cell and the fitted centre moved by up to 0.05 px. - ShadowFinder::GetMask: the whole mask (pooling, ring medians, components, morphology, hole fill, arm search) runs on the device from ShadowAccumulatorGPU's projection (ShadowMaskGPU), so the 360 MB projection no longer comes back; the mean projection is divided on the device too (same bits). The two small fits over rings and sectors (BlockedOutTo, HarmonicFit) are shared with the host path in ShadowFinderInternal.h. Integers, comparisons, sorts and components are exact; the polarization trig, the Poisson log and the arm-search azimuth are not, so a pixel at a threshold can differ. The one-time change against the previous CPU arithmetic (BackgroundAtan2, no contraction), measured on the myoglobin, cytochrome C and thaumatin rotation sets: ring centre moves 0.002-0.045 px (fit sigma 0.75-1.2 px), beam-centre capture 0.01-0.04 px; beam-stop mask differs on 31 / 144 / 53 pixels of 259k / 144k / 198k (25 of the myoglobin ones are GPU-vs-CPU arithmetic in the mask, the rest follow the centre); hot-pixel mask identical. Spot width, integration radii, bandwidth, beam-centre arbitration, indexing, space group, cell, resolution and the merged statistics table are identical; only the error model moves in its 4th digit. CPU build: the same centres and decisions. Timing (GPU, box at load 30-38): ring walk 0.54 -> 0.23-0.27 s, mask 1.24-1.44 -> 0.18-0.22 s, beam-centre capture walk 1.1-1.3 -> 0.31-0.35 s. Tests: ShadowFinder_DeviceMaskMatchesHost, BeamCenterFromBackground_DeviceMatchesHost (bit-exact), plus [ShadowFinder], [BeamCenter], [HotPixelFinder]. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
73 lines
3.3 KiB
C++
73 lines
3.3 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <cstdint>
|
|
#include <memory>
|
|
#include <vector>
|
|
|
|
#include "../../common/CompressedImage.h"
|
|
#include "../image_preprocessing/BSLZ4DecoderGPU.h"
|
|
#include "../indexing/CUDAMemHelpers.h"
|
|
#include "ShadowMaskGPU.h"
|
|
|
|
// The beam-stop projection accumulated on the device: only the compressed chunk crosses PCIe, and
|
|
// both the decode and the per-pixel maximum / sum / count run on the GPU. The projection comes back
|
|
// once, when the mask is read.
|
|
//
|
|
// This accumulates the RAW STORED COUNTS, with the pixel type's sentinel skipped, exactly as the
|
|
// host ShadowFinder does - NOT the preprocessed image. The two are not the same thing and the
|
|
// shadow is defined on the raw one. Sums and counts are integers, so the device result is identical
|
|
// to the host's rather than merely close.
|
|
//
|
|
// Only bitshuffle+LZ4 images are handled; anything else is left to the host path.
|
|
class ShadowAccumulatorGPU {
|
|
const size_t npixels;
|
|
std::shared_ptr<CudaStream> stream;
|
|
std::unique_ptr<BSLZ4DecoderGPU> decoder;
|
|
|
|
// Frames are decoded into a batch and folded into the projection together. The fold reads and
|
|
// writes 20 bytes per pixel whatever the batch holds, so doing it once per frame would spend
|
|
// most of the bandwidth on the accumulator rather than on the data. Four is where that stops
|
|
// mattering - the fold is then a few percent of the phase - and every frame beyond it is a
|
|
// further full frame of device memory, which costs more in cudaMalloc (device-synchronizing)
|
|
// than it saves.
|
|
static constexpr int BATCH = 4;
|
|
CudaDevicePtr<uint8_t> raw; // BATCH decoded frames, still in their stored type
|
|
CudaDevicePtr<int64_t> gpu_max;
|
|
CudaDevicePtr<int64_t> gpu_sum;
|
|
CudaDevicePtr<uint32_t> gpu_count;
|
|
size_t frame_bytes = 0; // stride of one frame within `raw`
|
|
size_t raw_capacity = 0; // bytes actually allocated for `raw`
|
|
int pending = 0; // frames decoded but not yet folded in
|
|
CompressedImageMode pending_mode = CompressedImageMode::Uint32;
|
|
uint32_t frames = 0;
|
|
int blocks = 0;
|
|
|
|
void EnsureRawCapacity(size_t bytes_per_frame);
|
|
void FoldPending();
|
|
|
|
public:
|
|
explicit ShadowAccumulatorGPU(size_t npixels);
|
|
|
|
// True when this image can be decoded and accumulated on the device.
|
|
static bool Supports(const CompressedImage &image);
|
|
|
|
// Decode and accumulate one frame. Throws if the image is not one this can take (ask Supports
|
|
// first) or if the chunk does not decode.
|
|
void Add(const CompressedImage &image);
|
|
|
|
[[nodiscard]] uint32_t GetFrameCount() const { return frames; }
|
|
|
|
// The beam-stop mask and the mean projection, made where the projection is (ShadowMaskGPU.h), so
|
|
// that it does not have to come back at all.
|
|
std::vector<uint32_t> Mask(const ShadowMaskSetup &setup, const std::vector<uint32_t> &pixel_mask);
|
|
std::vector<float> MeanProjection(const std::vector<uint32_t> &pixel_mask);
|
|
|
|
// Bring the projection back to the host, folding in whatever the last batch still holds. Cheap
|
|
// to call once; it moves 20 bytes per pixel.
|
|
void Download(std::vector<int64_t> &max_value, std::vector<int64_t> &sum_value,
|
|
std::vector<uint32_t> &valid_count);
|
|
};
|