receiver: stop copying every frame back from the device on the Lite path
Build Packages / Unit tests (push) Successful in 1h1m55s
Build Packages / build:viewer-tgz:cpu (push) Successful in 8m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 9m20s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m6s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m9s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m13s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m22s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m21s
Build Packages / build:rpm (rocky8) (push) Successful in 12m0s
Build Packages / build:rpm (rocky9) (push) Successful in 13m23s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 13m18s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 13m6s
Build Packages / DIALS test (push) Successful in 13m59s
Build Packages / XDS test (durin plugin) (push) Successful in 8m4s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m40s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m1s
Build Packages / Generate python client (push) Successful in 32s
Build Packages / Build documentation (push) Successful in 1m9s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:nocuda (push) Failing after 13m23s
Build Packages / build:windows:cuda (push) Failing after 12m24s

The Lite workflow built its analysis with the fused GPU engine disabled, which is
also what decides whether the preprocessed image is copied device-to-host after
every frame. So on a machine with a GPU the online path was moving the whole image
back - 72 MB on a large detector, every frame, per worker - for a host reader that
does not exist on that path.

It was left off deliberately when the fused engine was added, to keep the online
path unchanged in that commit, and never revisited. Nothing depends on it: the FPGA
workflow uses a different analysis class, and strong-pixel values are read through a
device gather rather than from the host image.

Turning it on changes no result, and cannot: adaptive detection is unreachable
online, because the REST schema exposes no way to enable it, so the classic GPU
finder runs either way. Measured anyway, both engines on the same frames across five
datasets including very weak ones: 2400 frames, 638260 spots, not one difference -
identical lists, identical indexing rate, identical merge statistics to every
printed digit.

On a large detector with eight workers the median per-image cost falls from 94 to
59 ms and preprocessing from 21 to 6 ms; throughput rises from about 48 to 55 Hz. No
percentile regresses, which is what matters for a service - the ninetieth improves
from 128 to 74 ms and the tail with it. Spot finding gets faster too, because the
large copy no longer contends with the device gather.

Correct two statements while here. The flag's comment and the data-analysis
document both said the online receiver uses the CPU adaptive finder; online never
runs an adaptive finder at all, and the copy the flag really controls was not
mentioned. That copy would be better expressed as what it is - whether a host engine
will read the image, which the constructor already knows - rather than inferred from
which spot finder is wanted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-02 15:16:18 +02:00
co-authored by Claude Opus 5
parent ccbc366e2f
commit e7be5447d3
3 changed files with 7 additions and 4 deletions
+1 -1
View File
@@ -201,7 +201,7 @@ where $k_\mathrm{Poisson}(\mu_b,p)$ is the smallest count whose Poisson$(\mu_b)$
Because detection reads the pixel's ring, a pixel that falls outside the azimuthal-integration $q$ range has no ring and can never be strong: the integration range bounds what adaptive detection can see. Both upper limits are therefore optional and default to the detector itself — the azimuthal integration runs to the highest $q$ any pixel of the detector reaches (`--azim-max-q` unset), and spot finding is not clipped in resolution (`--spot-high-resolution` unset), for rotation data as well as stills. Setting either one narrows detection accordingly — appropriate for weak, high-background data, where the spots admitted at the detector edge are dominated by noise.
**Fused GPU engine.** The per-ring reduction the adaptive threshold needs is the *same* reduction the azimuthal integrator performs. On the GPU path the two are fused into a single image pass (`AdaptiveSpotFinderGPU`): one reduction accumulates the corrected per-ring sums for the azimuthal profile (§2) *and* the raw per-ring statistics for the threshold, after which a light kernel flags the strong pixels. One GPU pass therefore replaces both the separate azimuthal-integration pass and the host-side adaptive spot-finding pass, at a small fraction of the CPU finder's cost per frame and producing the same spot list and azimuthal profile. It is enabled by default in the offline `rugnux` path and the interactive viewer; the online receiver uses the CPU adaptive finder.
**Fused GPU engine.** The per-ring reduction the adaptive threshold needs is the *same* reduction the azimuthal integrator performs. On the GPU path the two are fused into a single image pass (`AdaptiveSpotFinderGPU`): one reduction accumulates the corrected per-ring sums for the azimuthal profile (§2) *and* the raw per-ring statistics for the threshold, after which a light kernel flags the strong pixels. One GPU pass therefore replaces both the separate azimuthal-integration pass and the host-side adaptive spot-finding pass, at a small fraction of the CPU finder's cost per frame and producing the same spot list and azimuthal profile. It is enabled by default in the offline `rugnux` path, the interactive viewer and the online receiver. Note that adaptive detection is not currently reachable online — `spot_finding_settings` in the REST API exposes no `adaptive_threshold`, so the receiver runs the fixed-threshold finder and the fused engine is held ready rather than used.
### 3.3 Resolution and ice-ring handling
+4 -2
View File
@@ -69,8 +69,10 @@ class MXAnalysisWithoutFPGA {
public:
// enable_fused_adaptive_gpu turns on the fused GPU azint+adaptive spot finder (only takes effect on
// the GPU path with adaptive detection). The rugnux offline path and the interactive viewer enable
// it by default; the online receiver leaves it off and keeps the CPU adaptive finder + separate
// azint. It only changes performance - the fused engine reproduces the CPU finder's spots.
// it by default, as does the online receiver. It only changes performance - the fused engine
// reproduces the CPU finder's spots. Note it also decides whether the preprocessed image is copied
// back to the host each frame: that copy exists only for a CPU engine to read, and while adaptive
// detection is not reachable through the REST API the copy is the flag's only effect online.
MXAnalysisWithoutFPGA(const DiffractionExperiment &experiment, const AzimuthalIntegrationMapping &integration,
const PixelMask &mask, IndexAndRefine &indexer, bool enable_fused_adaptive_gpu = false);
void Analyze(DataMessage &output, AzimuthalIntegrationProfile &profile, const SpotFindingSettings &spot_finding_settings);
+2 -1
View File
@@ -276,7 +276,8 @@ void JFJochReceiverLite::DataAnalysisThread(uint32_t id) {
measurement_started.wait();
try {
analysis = std::make_unique<MXAnalysisWithoutFPGA>(experiment, *az_int_mapping, pixel_mask, indexer);
analysis = std::make_unique<MXAnalysisWithoutFPGA>(experiment, *az_int_mapping, pixel_mask, indexer,
/*enable_fused_adaptive_gpu=*/true);
} catch (const JFJochException &e) {
Cancel(e);
return;