RotationScaleMerge::DeviceBytes: the resident ingest's peak as measured
The ingest term was 90 bytes per reflection and 64 per run. Measured with cudaMemGetInfo at each step of IngestUpload/IngestSort (relative to the free memory before the observation arrays): - the second radix sort is the peak where runs are few: the 66 B of observation arrays, by_image, perm, key, key_sorted and cub's alternate key/value buffers of a plain SortPairs - 102.3-103.0 B per reflection on 7qij (51.1 M, 52.2 M, 78.1 M reflections) and 4nwv (32.7 M); - the cut is the peak where runs are many: 98 B per reflection plus 56 per run (4nwv, 15.4 M runs: 125.8 B per reflection, 4106 MB against 3923 MB from the old formula); - the upload holds the 66 B beside the two 64 MB staging buffers (~150 MB measured). Now max(103 B per reflection + 58 B per run, 66 B per reflection + 160 MB): 2-4% above every measured peak, where the old formula was 10% under on 7qij (7170 MB against 7986 MB for the main engine) and 4.5% under on 4nwv. Its only consumer is whether the all-observation arm goes beside the main engine, which changes when that merge runs, not what it gives. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
This commit is contained in:
1 parent
5e916fe120
commit
b736c4cefb
1 file changed
+9
-5
@@ -701,11 +701,15 @@ size_t RotationScaleMerge::DeviceBytes() const {
|
||||
const size_t bytes = BYTES_PER_OBS * static_cast<size_t>(n_partials_obs) + BYTES_PER_EVENT * events
|
||||
+ BYTES_PER_HKL * rawrun_start.size();
|
||||
// The resident ingest's own peak comes before any of that, over every reflection read and every run
|
||||
// before the resolution cut: the observation arrays, the sort key and the integrated delta_phi of
|
||||
// each (66 bytes) and 24 bytes of the sort's or the cut's working arrays beside them, plus sixty-four
|
||||
// bytes per run at the cut (IngestSort).
|
||||
constexpr size_t INGEST_BYTES_PER_REFLECTION = 90, INGEST_BYTES_PER_RUN = 64;
|
||||
const size_t ingest = INGEST_BYTES_PER_REFLECTION * ingest_reflections + INGEST_BYTES_PER_RUN * ingest_runs;
|
||||
// before the resolution cut, measured with cudaMemGetInfo. Beside the observation arrays, the sort key
|
||||
// and the integrated delta_phi of each (66 bytes): the second radix sort holds the frame order, the
|
||||
// permutation, both keys and cub's alternate key and value buffers, 102.6 bytes per reflection in
|
||||
// all; the cut (IngestSort) 98 per reflection and 56 per run. The upload before them holds the 66
|
||||
// bytes and the two 64 MB staging buffers.
|
||||
constexpr size_t INGEST_BYTES_PER_REFLECTION = 103, INGEST_BYTES_PER_RUN = 58;
|
||||
constexpr size_t INGEST_ARRAY_BYTES = 66, INGEST_STAGING_BYTES = size_t(160) << 20;
|
||||
const size_t ingest = std::max(INGEST_BYTES_PER_REFLECTION * ingest_reflections + INGEST_BYTES_PER_RUN * ingest_runs,
|
||||
INGEST_ARRAY_BYTES * ingest_reflections + INGEST_STAGING_BYTES);
|
||||
return std::max(bytes + bytes / 5, ingest);
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user