jungfrauandClaude Opus 5
5ee0f22a61
The image loop gives every worker its own analysis engine, so a run builds ninety-six of them. Each one derived, from scratch, tables that are the same in all of them: the byte-per-pixel mask, the resolution mask, the radial kernel, and the checksum that names the shared device tables. The checksum was the worst of it, because it is part of the cache KEY and so is computed before the lookup - a hit still hashed the whole table. On a 16 Mpx detector that is the bin table, the corrections and the mask, 126 MB an engine, about twelve gigabytes over a run, to answer a question whose answer had not changed. The header said it cost nothing measurable; a profile says otherwise, and says it is worst exactly during the ramp when the machine has nothing else to do. It cannot simply be remembered against the address, which is what it exists to catch: a buffer can be freed and another allocated where it was, and the cache would then hand back a device copy of something else. So the owner of the bytes computes it instead. The azimuthal mapping writes its two tables in its constructor and never again. The pixel mask re-derives its binary form and its checksum on every path that changes the mask, and all of those paths are now private to the class. The key therefore still describes the bytes as they are at the moment of the lookup. The resolution mask was two passes over every pixel - a float comparison into a vector<bool>, then a bit-by-bit repack - in each of the ninety-six. It is one pass now, writing the packed form directly, built once for the limits asked for and handed out as a shared pointer so a worker keeps the mask it was given. The radial kernel is cached on the six numbers it is derived from. Nothing computes a different value; only who computes it changes. Byte-identical merged output on a 16 Mpx set and on a small one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011n8riB6X59oRjkrSHzNPAU
73 lines
3.4 KiB
C++
73 lines
3.4 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include "CompressedImage.h"
|
|
#include "DetectorSetup.h"
|
|
#include "DiffractionExperiment.h"
|
|
#include "../jungfrau/JFCalibration.h"
|
|
|
|
struct PixelMaskStatistics {
|
|
uint32_t user_mask;
|
|
uint32_t noisy_pixel;
|
|
uint32_t error_pixel;
|
|
uint32_t chip_gap_pixel;
|
|
uint32_t total_masked;
|
|
uint32_t module_gap_pixel;
|
|
};
|
|
|
|
class PixelMask {
|
|
std::vector<uint32_t> mask;
|
|
std::vector<uint32_t> raw_mask;
|
|
// One byte per pixel, 1 where the pixel is masked at all - the form the GPU image preprocessor
|
|
// uploads - and the checksum the shared device-table cache keys on (CudaSharedTables.h). Both are
|
|
// pure functions of `mask`, and an analysis engine is built per worker per pass, so they are
|
|
// derived here once instead of in every one of those engines.
|
|
std::vector<uint8_t> binary_mask;
|
|
uint64_t binary_mask_checksum = 0;
|
|
|
|
uint32_t LoadMask(const std::vector<uint32_t>& mask, uint8_t bit);
|
|
// Everything that follows from the mask, recomputed wherever the mask changes.
|
|
void UpdateDerived(const DiffractionExperiment& experiment);
|
|
void UpdateBinaryMask();
|
|
|
|
void CalcEdgePixels_i(const DiffractionExperiment& experiment);
|
|
|
|
public:
|
|
// NXmx bits
|
|
constexpr static const uint8_t ModuleGapPixelBit = 0;
|
|
constexpr static const uint8_t ErrorPixelBit = 1;
|
|
constexpr static const uint8_t NoisyPixelBit = 4;
|
|
constexpr static const uint8_t UserMaskedPixelBit = 8;
|
|
constexpr static const uint8_t BeamStopPixelBit = 9;
|
|
constexpr static const uint8_t ChipGapPixelBit = 31;
|
|
constexpr static const uint8_t ModuleEdgePixelBit = 30;
|
|
|
|
PixelMask();
|
|
explicit PixelMask(size_t width, size_t height);
|
|
explicit PixelMask(const DiffractionExperiment& experiment);
|
|
explicit PixelMask(const std::vector<uint32_t> &mask);
|
|
|
|
void CalcEdgePixels(const DiffractionExperiment& experiment);
|
|
void LoadUserMask(const DiffractionExperiment& experiment, const std::vector<uint32_t>& mask);
|
|
void LoadUserMask(const DiffractionExperiment& experiment, const CompressedImage& image);
|
|
void LoadBeamStopMask(const DiffractionExperiment& experiment, const std::vector<uint32_t>& mask);
|
|
// The beam-stop shadow belongs to the run that found it, not to the dataset, so a mask read back
|
|
// from a file that carries one starts clear. The user mask (bit 8) is deliberately left alone.
|
|
void ClearBeamStopMask(const DiffractionExperiment& experiment);
|
|
void LoadDECTRISBadPixelMask(const std::vector<uint32_t>& mask);
|
|
void LoadDarkBadPixelMask(const DiffractionExperiment& experiment, const std::vector<uint32_t>& mask);
|
|
void LoadDetectorBadPixelMask(const DiffractionExperiment& experiment, const JFCalibration *calib);
|
|
|
|
[[nodiscard]] const std::vector<uint32_t> &GetMaskRaw() const;
|
|
[[nodiscard]] const std::vector<uint32_t> &GetMask(const DiffractionExperiment& experiment) const;
|
|
[[nodiscard]] const std::vector<uint32_t> &GetMask() const;
|
|
[[nodiscard]] const std::vector<uint8_t> &GetBinaryMask() const;
|
|
[[nodiscard]] uint64_t GetBinaryMaskChecksum() const;
|
|
|
|
[[nodiscard]] std::vector<uint32_t> GetUserMask(const DiffractionExperiment& experiment) const;
|
|
[[nodiscard]] std::vector<uint32_t> GetUserMask() const;
|
|
[[nodiscard]] PixelMaskStatistics GetStatistics() const;
|
|
};
|