Refinement minimiser: reorthogonalise the conjugate gradients' residuals
The normal-matrix solve (20 fixed preconditioned CG iterations per L-BFGS direction) was forward unstable: run in lockstep on right-hand sides 1e-14 apart, two CPU solves in double drifted apart 5-7x per iteration from about the 5th iteration and ended up to 7% apart (myoglobin: 17% of the position solves above 1e-5). That is the loss of orthogonality of the recurrence in rounding once its first Ritz values converge, and it is what set the CPU and the GPU apart: the device solve matched the host to 1e-14 until the first such call and then differed by exactly that spread. It is not small curvature: an absolute floor on the X-ray diagonal (1e-3 x median, the review's remedy) changed nothing, and blocks of up to 8 residues in the preconditioner hardly anything. Each new residual is now made M^-1-orthogonal to all the earlier ones (modified Gram-Schmidt on r and z together), on the CPU and on the card. Measured on every solve of a refinement run (myoglobin, cytochrome C): a 1e-12 change of q now moves the result by 2e-12 (max 2e-11), was median 2e-7 / max 8e-2; device vs host median 4e-14, max 1e-11, was p90 2e-3 / max 6e-2. The residual after 20 iterations is the same (mean |r20|/|r0| 0.016 vs 0.017). Costs 2 x 20 stored vectors per solve. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UqXYdHHZ4cogS1mrGUtfN7
This commit is contained in:
1 parent
a1838b023a
commit
3c59c4fff2
6 files changed
+80
-13
No files matched your search
@@ -603,7 +603,12 @@ initial inverse Hessian is the target's sparse normal matrix - the X-ray target'
|
||||
on the diagonal and the restraints' Gauss-Newton matrix in full - as REFMAC refines on its normal
|
||||
matrix (Murshudov, Vagin & Dodson 1997, above), solved by conjugate gradients: M. R. Hestenes and
|
||||
E. Stiefel, "Methods of conjugate gradients for solving linear systems" (1952), J. Res. Natl. Bur.
|
||||
Stand. 49, 409-436 [doi:10.6028/jres.049.044](https://doi.org/10.6028/jres.049.044). Each atom's
|
||||
Stand. 49, 409-436 [doi:10.6028/jres.049.044](https://doi.org/10.6028/jres.049.044), each residual
|
||||
made orthogonal again to the earlier ones against the loss of orthogonality in rounding described by
|
||||
C. C. Paige, "Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix" (1976),
|
||||
J. Inst. Maths Applics 18, 341-349 (DOI not verified here), and A. Greenbaum, "Behavior of
|
||||
slightly perturbed Lanczos and conjugate-gradient recurrences" (1989), Linear Algebra Appl. 113, 7-63
|
||||
[doi:10.1016/0024-3795(89)90285-1](https://doi.org/10.1016/0024-3795(89)90285-1). Each atom's
|
||||
diagonal is raised to hold its step within the step bound, a trust region per atom after the damping
|
||||
of K. Levenberg, "A method for the solution of certain non-linear problems in least squares" (1944),
|
||||
Q. Appl. Math. 2, 164-168 [doi:10.1090/qam/10666](https://doi.org/10.1090/qam/10666), and D. W.
|
||||
|
||||
@@ -73,6 +73,7 @@ The methods draw on, and in places reimplement, solutions from:
|
||||
- D. C. Liu & J. Nocedal, "On the limited memory BFGS method for large scale optimization", *Math. Program.* **45** (1989), 503-528 (the minimiser, §15).
|
||||
- J. Nocedal, "Updating quasi-Newton matrices with limited storage", *Math. Comp.* **35** (1980), 773-782 (the two-loop recursion of the minimiser, §15.6).
|
||||
- M. R. Hestenes & E. Stiefel, "Methods of conjugate gradients for solving linear systems", *J. Res. Natl. Bur. Stand.* **49** (1952), 409-436 (the normal equations of the minimiser, §15.6).
|
||||
- C. C. Paige, "Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix", *J. Inst. Maths Applics* **18** (1976), 341-349, and A. Greenbaum, "Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences", *Linear Algebra Appl.* **113** (1989), 7-63 (the reorthogonalised conjugate gradients of the minimiser, §15.6).
|
||||
- G. N. Murshudov, P. Skubák, A. A. Lebedev et al., "REFMAC5 for the refinement of macromolecular crystal structures", *Acta Cryst.* **D67** (2011), 355-367 (the likelihood's D from the working set and its variance from the free, and the Kullback-Leibler B restraints, §15.3, §15.5).
|
||||
- P. Roversi, E. Blanc, S. Johnson & S. M. Lea, "Tetartohedral twinning could happen to you too", *Acta Cryst.* **D68** (2012), 418-424 (twin refinement with more than two domains, and the free set closed under the twin domains, §15.12).
|
||||
- K. A. Sultana, I. Alexeev, I. Kursula, P. Mäntsälä, J. Niemi & G. Schneider, "Structure determination by multiwavelength anomalous diffraction of aclacinomycin oxidoreductase: indications of multidomain pseudomerohedral twinning", *Acta Cryst.* **D63** (2007), 149-159 (a six-domain twin of a monoclinic crystal on a hexagonal metric, §15.12).
|
||||
|
||||
@@ -320,7 +320,12 @@ leaves out.
|
||||
conjugate-gradient iterations - not to a tolerance, whose iteration count, and with it the direction, a rounding
|
||||
difference can flip - preconditioned per residue for the positions (the residue's own $J^\mathsf{T}J$ and X-ray
|
||||
diagonal, by Cholesky: the stiff bonds and angles inside a residue are solved there, what couples residues is
|
||||
left to the iterations) and by the diagonal elsewhere. The Jacobian is taken again at each point the minimiser
|
||||
left to the iterations) and by the diagonal elsewhere. Each new residual is made orthogonal again (in the
|
||||
preconditioner's inner product) to all the earlier ones: the plain recurrence loses that orthogonality in
|
||||
rounding once its first Ritz values converge (Paige; Greenbaum), and from then on a rounding difference grew
|
||||
several-fold per iteration, so that two solves of right-hand sides $10^{-14}$ apart ended up to 7% apart - which
|
||||
is what set the CPU and GPU results apart. With it the solve is a smooth function of its input, and the GPU's
|
||||
float-pair solve agrees with the CPU's to $10^{-13}$. The Jacobian is taken again at each point the minimiser
|
||||
stands on.
|
||||
- **Its memory is kept from one macro-cycle to the next.** Each weight trial starts with an empty memory.
|
||||
|
||||
@@ -790,6 +795,7 @@ What the refinement does not do yet - each a known gap, not a design choice:
|
||||
- N. Echols et al., "Automated identification of elemental ions in macromolecular crystal structures", *Acta Cryst.* **D70** (2014), 1104-1114 (a light ion flagged by its density above the waters', §15.10).
|
||||
- J. Nocedal, "Updating quasi-Newton matrices with limited storage", *Math. Comp.* **35** (1980), 773-782, and D. C. Liu & J. Nocedal, "On the limited memory BFGS method for large scale optimization", *Math. Program.* **45** (1989), 503-528 (the minimiser, §15.6).
|
||||
- M. R. Hestenes & E. Stiefel, "Methods of conjugate gradients for solving linear systems", *J. Res. Natl. Bur. Stand.* **49** (1952), 409-436 (the normal equations of the minimiser, §15.6).
|
||||
- C. C. Paige, "Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix", *J. Inst. Maths Applics* **18** (1976), 341-349, and A. Greenbaum, "Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences", *Linear Algebra Appl.* **113** (1989), 7-63 (the loss of orthogonality the minimiser's conjugate gradients are kept from, §15.6).
|
||||
- K. Levenberg, "A method for the solution of certain non-linear problems in least squares", *Q. Appl. Math.* **2** (1944), 164-168, and D. W. Marquardt, "An algorithm for least-squares estimation of nonlinear parameters", *J. Soc. Ind. Appl. Math.* **11** (1963), 431-441 (the minimiser's trust floor per atom, §15.6).
|
||||
- A. T. Brünger, M. Karplus & G. A. Petsko, "Crystallographic refinement by simulated annealing: application to crambin", *Acta Cryst.* **A45** (1989), 50-61 (the starting weight from the ratio of the gradients, §15.7).
|
||||
- P. V. Afonine et al., "Towards automated crystallographic structure refinement with phenix.refine", *Acta Cryst.* **D68** (2012), 352-367 (weights chosen among refinements by R-free within geometry bounds, and the overall B moved into the atoms, §15.3, §15.7).
|
||||
|
||||
@@ -172,6 +172,12 @@ private:
|
||||
// z with H z = b to `tolerance` of |b| (at most `max_iterations`), by conjugate gradients preconditioned with M:
|
||||
// hv = H v (product), z = M^-1 r (precondition), both symmetric positive definite. Returns the iterations taken.
|
||||
// Following Hestenes & Stiefel (1952) J. Res. Natl. Bur. Stand. 49, 409-436
|
||||
// Each residual is made M^-1-orthogonal again to all the earlier ones (modified Gram-Schmidt, r and z = M^-1 r
|
||||
// together). Without it the recurrence loses that orthogonality in rounding once the first Ritz values converge, and
|
||||
// from then on amplifies a rounding difference several times per iteration: on the refinement's normal matrix two
|
||||
// solves of right-hand sides 1e-14 apart ended up to 7% apart after 20 iterations, which set CPU and GPU apart.
|
||||
// The loss of orthogonality after Paige (1976) J. Inst. Maths Applics 18, 341-349 and Greenbaum (1989) Linear
|
||||
// Algebra Appl. 113, 7-63
|
||||
inline int ConjugateGradients(const std::function<void(const std::vector<double> &v, std::vector<double> &hv)> &product,
|
||||
const std::function<void(const std::vector<double> &r, std::vector<double> &z)> &precondition,
|
||||
const std::vector<double> &b, std::vector<double> &x, int max_iterations, double tolerance,
|
||||
@@ -183,6 +189,8 @@ inline int ConjugateGradients(const std::function<void(const std::vector<double>
|
||||
precondition(r, z);
|
||||
p = z;
|
||||
double rz = dot(r, z);
|
||||
std::vector<std::vector<double>> rs{r}, zs{z}; // the residuals so far, and M^-1 of each
|
||||
std::vector<double> rzs{rz};
|
||||
const double r0 = std::sqrt(dot(r, r));
|
||||
int k = 0;
|
||||
while (k < max_iterations) {
|
||||
@@ -199,9 +207,19 @@ inline int ConjugateGradients(const std::function<void(const std::vector<double>
|
||||
if (std::sqrt(dot(r, r)) < tolerance * r0)
|
||||
break;
|
||||
precondition(r, z);
|
||||
for (size_t i = 0; i < rs.size(); i++) {
|
||||
const double c = dot(r, zs[i]) / rzs[i];
|
||||
refine_vec::Each(n, nthreads, [&](size_t j) {
|
||||
r[j] -= c * rs[i][j];
|
||||
z[j] -= c * zs[i][j];
|
||||
});
|
||||
}
|
||||
const double rz_new = dot(r, z);
|
||||
refine_vec::Each(n, nthreads, [&](size_t j) { p[j] = z[j] + rz_new / rz * p[j]; });
|
||||
rz = rz_new;
|
||||
rs.push_back(r);
|
||||
zs.push_back(z);
|
||||
rzs.push_back(rz);
|
||||
}
|
||||
return k;
|
||||
}
|
||||
@@ -228,7 +228,7 @@ __global__ void DiagonalKernel(const int *entry_start, const int *entry, const f
|
||||
|
||||
// The conjugate gradients' scalars, kept on the card between the kernels: r M^-1 r, the step a and beta
|
||||
struct CGScalars {
|
||||
float2 rz, a, beta;
|
||||
float2 rz, a, beta, c; // c: the residual's component along an earlier one
|
||||
int done; // p H p was not positive: x stays where it is
|
||||
};
|
||||
|
||||
@@ -256,10 +256,11 @@ __global__ void DotPartial(const float2 *u, const float2 *v, int n, float2 *part
|
||||
partial[blockIdx.x] = block;
|
||||
}
|
||||
|
||||
enum class DotUse { FirstRz, PHp, Rz };
|
||||
enum class DotUse { FirstRz, PHp, Rz, Component };
|
||||
|
||||
// The partial sums' total, taken as the first r M^-1 r, as p H p (a = rz / pHp) or as the next r M^-1 r (beta)
|
||||
__global__ void DotFinish(const float2 *partial, CGScalars *s, DotUse use) {
|
||||
// The partial sums' total, taken as the first r M^-1 r, as p H p (a = rz / pHp), as the next r M^-1 r (beta), each
|
||||
// r M^-1 r kept as rz_of[j], or as r . z_j (c = r . z_j / rz_of[j])
|
||||
__global__ void DotFinish(const float2 *partial, CGScalars *s, DotUse use, float2 *rz_of, int j) {
|
||||
__shared__ float2 shared[DOT_BLOCKS];
|
||||
const float2 dot = BlockSum(shared, partial[threadIdx.x]);
|
||||
if (threadIdx.x != 0)
|
||||
@@ -267,6 +268,7 @@ __global__ void DotFinish(const float2 *partial, CGScalars *s, DotUse use) {
|
||||
switch (use) {
|
||||
case DotUse::FirstRz:
|
||||
s->rz = dot;
|
||||
rz_of[j] = dot;
|
||||
break;
|
||||
case DotUse::PHp:
|
||||
s->a = Div(s->rz, dot);
|
||||
@@ -276,6 +278,10 @@ __global__ void DotFinish(const float2 *partial, CGScalars *s, DotUse use) {
|
||||
case DotUse::Rz:
|
||||
s->beta = Div(dot, s->rz);
|
||||
s->rz = dot;
|
||||
rz_of[j] = dot;
|
||||
break;
|
||||
case DotUse::Component:
|
||||
s->c = Div(dot, rz_of[j]);
|
||||
break;
|
||||
}
|
||||
}
|
||||
@@ -289,6 +295,15 @@ __global__ void Step(const CGScalars *s, const float2 *p, const float2 *hp, floa
|
||||
r[i] = Add(r[i], Neg(Mul(s->a, hp[i])));
|
||||
}
|
||||
|
||||
// r -= c r_j, z -= c z_j
|
||||
__global__ void Project(const CGScalars *s, const float2 *rj, const float2 *zj, float2 *r, float2 *z, int n) {
|
||||
const int i = blockIdx.x * blockDim.x + threadIdx.x;
|
||||
if (i >= n)
|
||||
return;
|
||||
r[i] = Add(r[i], Neg(Mul(s->c, rj[i])));
|
||||
z[i] = Add(z[i], Neg(Mul(s->c, zj[i])));
|
||||
}
|
||||
|
||||
// p = z + beta p
|
||||
__global__ void NextDirection(const CGScalars *s, const float2 *z, float2 *p, int n) {
|
||||
const int i = blockIdx.x * blockDim.x + threadIdx.x;
|
||||
@@ -340,6 +355,8 @@ struct RestraintNormalMatrixGPU::Device {
|
||||
// the conjugate gradients: H's X-ray diagonal, M's blocks and their inverses, the vectors, the dot products' partial sums
|
||||
int elements = 0;
|
||||
CudaDevicePtr<float2> h, inverse, x, r, z, p, hp, partial;
|
||||
CudaDevicePtr<float2> rs, zs, rz_of; // the residuals of a solve so far, M^-1 of each, r M^-1 r of each
|
||||
int kept = 0; // how many of them rs and zs hold room for
|
||||
CudaDevicePtr<int> block_of, block_start, block_element, inverse_start;
|
||||
CudaDevicePtr<CGScalars> scalars;
|
||||
PinnedVector<float2> host; // v and hv as float pairs
|
||||
@@ -485,9 +502,21 @@ void RestraintNormalMatrixGPU::Solve(const std::vector<double> &b, std::vector<d
|
||||
BlockApply<<<GridSize(d.elements), THREADS, 0, d.stream>>>(d.block_of, d.block_start, d.block_element,
|
||||
d.inverse_start, d.inverse, d.r, d.z, d.elements);
|
||||
};
|
||||
auto dot = [&](const float2 *u, const float2 *v, DotUse use) {
|
||||
auto dot = [&](const float2 *u, const float2 *v, DotUse use, int j) {
|
||||
DotPartial<<<DOT_BLOCKS, THREADS, 0, d.stream>>>(u, v, n, d.partial);
|
||||
DotFinish<<<1, DOT_BLOCKS, 0, d.stream>>>(d.partial, d.scalars, use);
|
||||
DotFinish<<<1, DOT_BLOCKS, 0, d.stream>>>(d.partial, d.scalars, use, d.rz_of, j);
|
||||
};
|
||||
if (d.kept < iterations) {
|
||||
d.rs = CudaDevicePtr<float2>(static_cast<size_t>(iterations) * n);
|
||||
d.zs = CudaDevicePtr<float2>(static_cast<size_t>(iterations) * n);
|
||||
d.rz_of = CudaDevicePtr<float2>(iterations);
|
||||
d.kept = iterations;
|
||||
}
|
||||
auto keep = [&](int j) { // r and z as the j-th
|
||||
cuda_err(cudaMemcpyAsync(d.rs.get() + static_cast<size_t>(j) * n, d.r.get(), n * sizeof(float2),
|
||||
cudaMemcpyDeviceToDevice, d.stream));
|
||||
cuda_err(cudaMemcpyAsync(d.zs.get() + static_cast<size_t>(j) * n, d.z.get(), n * sizeof(float2),
|
||||
cudaMemcpyDeviceToDevice, d.stream));
|
||||
};
|
||||
for (int i = 0; i < n; i++)
|
||||
d.host[i] = Split(b[i]);
|
||||
@@ -496,18 +525,25 @@ void RestraintNormalMatrixGPU::Solve(const std::vector<double> &b, std::vector<d
|
||||
cuda_err(cudaMemsetAsync(d.scalars.get(), 0, sizeof(CGScalars), d.stream));
|
||||
precondition();
|
||||
cuda_err(cudaMemcpyAsync(d.p.get(), d.z.get(), n * sizeof(float2), cudaMemcpyDeviceToDevice, d.stream));
|
||||
dot(d.r, d.z, DotUse::FirstRz);
|
||||
dot(d.r, d.z, DotUse::FirstRz, 0);
|
||||
keep(0);
|
||||
for (int k = 0; k < iterations; k++) {
|
||||
if (d.rows > 0)
|
||||
RowProduct<<<GridSize(d.rows), THREADS, 0, d.stream>>>(d.start, d.param, d.j, per_, c0_, d.p, d.t, d.rows);
|
||||
TransposeProduct<<<GridSize(static_cast<int>(n_params_)), THREADS, 0, d.stream>>>(
|
||||
d.entry_start, d.entry, d.row_of, d.j, per_, c0_, d.t, d.h, d.p, d.hp, static_cast<int>(n_params_));
|
||||
dot(d.p, d.hp, DotUse::PHp);
|
||||
dot(d.p, d.hp, DotUse::PHp, 0);
|
||||
Step<<<GridSize(n), THREADS, 0, d.stream>>>(d.scalars, d.p, d.hp, d.x, d.r, n);
|
||||
if (k + 1 < iterations) { // the last direction would not be used
|
||||
precondition();
|
||||
dot(d.r, d.z, DotUse::Rz);
|
||||
for (int j = 0; j <= k; j++) { // M^-1-orthogonal to the earlier residuals, as ConjugateGradients
|
||||
const float2 *rj = d.rs.get() + static_cast<size_t>(j) * n, *zj = d.zs.get() + static_cast<size_t>(j) * n;
|
||||
dot(d.r, zj, DotUse::Component, j);
|
||||
Project<<<GridSize(n), THREADS, 0, d.stream>>>(d.scalars, rj, zj, d.r, d.z, n);
|
||||
}
|
||||
dot(d.r, d.z, DotUse::Rz, k + 1);
|
||||
NextDirection<<<GridSize(n), THREADS, 0, d.stream>>>(d.scalars, d.z, d.p, n);
|
||||
keep(k + 1);
|
||||
}
|
||||
}
|
||||
cuda_err(cudaGetLastError());
|
||||
|
||||
@@ -40,8 +40,9 @@ public:
|
||||
// `inverse`, in full, row by row. h and M stay until the next SetSolve().
|
||||
void SetSolve(const std::vector<double> &h, const std::vector<int> &start, const std::vector<int> &element,
|
||||
const std::vector<double> &inverse);
|
||||
// x = H^-1 b by `iterations` of them from x = 0 (fewer only where p H p is not positive), the vectors on the card
|
||||
// and every sum in float pairs, in a fixed order (RefinementNewtonGPU.cu).
|
||||
// x = H^-1 b by `iterations` of them from x = 0 (fewer only where p H p is not positive), each residual made
|
||||
// M^-1-orthogonal to the earlier ones as there; the vectors on the card and every sum in float pairs, in a fixed
|
||||
// order (RefinementNewtonGPU.cu).
|
||||
void Solve(const std::vector<double> &b, std::vector<double> &x, int iterations);
|
||||
|
||||
private:
|
||||
|
||||
Reference in new issue
Block a user