Refinement: density records sent from pinned memory, and the L-BFGS direction in fused sweeps
Two byte-identical cuts to the per-evaluation host chain, which on a large model is twice the GPU's work per evaluation (34 ms against 16 ms of kernels per weight-trial slot, measured in nsys) and so bounds the refinement on one card with three slots, and the whole of it on a card with one slot. - The gather's per-atom density records (38 MB on a 242k-atom model, sent on every B-factor evaluation) live in a PinnedVector and go up by DMA; the boxes go through a pinned staging buffer of the density object. ModelDensityGPU::SetAtoms and RefinementGPUEngine::SetDensities take a pointer and a count. - RefinementMinimiser::Direction takes each pair's dot product in the same sweep that applies the previous pair's update to q: the dot products are serial chains of n additions, and the update's memory traffic now hides under them (20 -> 13 ms per call at 365k parameters). The same operations per element in the same order; the loop-2 coefficient stays written as (a[i] - b) with b on its own, because a[i] - rho * dot contracts into a fused multiply-add and changes the bits (checked bitwise in a standalone harness). - A debug line with the GPU engine's bytes and the budget. 4nwv (open arm, 242k atoms), live flags, one lock hold, loaded box: refinement 42.35 -> 38.40 s; refine.gpu.upload 8.14 -> 3.94 s. All outputs identical but provenance lines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
This commit is contained in:
1 parent
3189f71e8f
commit
b16873036c
7 files changed
+93
-40
No files matched your search
@@ -304,7 +304,7 @@ private:
|
||||
stage.reset();
|
||||
const RefinementStage upload("gpu.upload");
|
||||
if (std::find(changed.begin(), changed.end(), 1) != changed.end())
|
||||
engine_->SetDensities(atoms_);
|
||||
engine_->SetDensities(atoms_.data(), atoms_.size());
|
||||
engine_->SetPositions(pos);
|
||||
}
|
||||
|
||||
@@ -320,7 +320,7 @@ private:
|
||||
std::unique_ptr<RefinementGPUEngine> engine_;
|
||||
std::vector<const gemmi::Atom *> atom_ptrs_; // the model's atoms at the last SetAtoms()
|
||||
std::vector<float> last_b_;
|
||||
std::vector<ModelDensityAtom> atoms_;
|
||||
PinnedVector<ModelDensityAtom> atoms_; // pinned: a B rung sends them every evaluation (37 MB on a large model)
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
Reference in new issue
Block a user