rugnux --model null: redraw a replicate the rigid body carried onto the model's own orientation
The pre-draw guard keeps a null replicate from STARTING within RigidBodyReachDeg of an orientation
equivalent to the model's (space-group rotations x {identity + twin laws}), but the analytic-Jacobian
rigid body walks further than that reach: on 9yzk one replicate drawn outside the 10.6 deg reach was
walked 23.9 deg onto an equivalent orientation (R-free 0.5578 -> 0.3917), the null SD went
0.016 -> 0.066 and MODEL_FIT flipped ACCEPTED -> REJECTED.
Now where the replicate ENDS is checked too: RefineRigidBody reports the rotation it applied, the
final orientation is that times the drawn one, and a replicate that ended within reach is drawn again
(through the same pre-draw filter) and placed again, capped at NULL_MAX_REDRAWS. The redraws come from
a generator per replicate (NULL_SEED + 1 + i), so they do not depend on the order the concurrent
replicates finish in; the count is logged once per run. The real fit's procedure is unchanged.
Validation, 12 open-arm sets against 20260928-0459_24ae26_r6-pooled: post-placement redraws on 9yzk (1)
and 5epe (1), none elsewhere. 9yzk back to ACCEPTED, +14.15 sigma, null 0.5880 +- 0.0138 (base +12.21,
0.5864 +- 0.0158), p.mtz byte-identical to base. 5epe: MODEL_FIT +41.62 -> +46.39, indexing margin
+40.94 -> +31.97 sigma, same decision. Every other set identical to the previous rigid-bc commit.
-N 1 and the default thread count give identical outputs and logs on 9yzk and 6toc.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
This commit is contained in:
1 parent
e0e3e92147
commit
4ce36935b9
6 files changed
+106
-11
No files matched your search
@@ -11,6 +11,7 @@
|
||||
* Rugnux adds beam-stop holder arms that let part of the beam through to the beam-stop mask, without changing the mask of a sweep that has none.
|
||||
* Rugnux's `--model` rigid-body refinement computes the model's density and solvent mask on all threads, with results unchanged.
|
||||
* Rugnux's `--model` rigid-body refinement computes the model's structure factors from one copy of the model and derives the translation part of its Jacobian exactly, so it runs faster; placements can differ slightly.
|
||||
* `rugnux --model` also draws a null replicate again when its rigid-body placement ends near an orientation equivalent to the model's, not only when it starts near one.
|
||||
|
||||
### 1.0.0-rc.173
|
||||
|
||||
|
||||
@@ -173,6 +173,13 @@ as fitted, or the comparison would be between a placed model and unplaced nulls
|
||||
be inflated by the placement rather than by the model. Measured on one rotation data set: the crystal's
|
||||
own model +15.0σ, an unrelated protein +1.8σ, and the correct model rigidly rotated 90° −1.0σ.
|
||||
|
||||
A replicate is a sample of the null only while it stays away from the model's own solution. A draw
|
||||
within the rigid body's reach of an orientation equivalent to the model's — under the space group's
|
||||
rotations or a twin law of the lattice — is drawn again before it is placed, and a replicate the
|
||||
placement nevertheless carries to within that reach is drawn and placed again afterwards, each
|
||||
replicate from a generator of its own so the result does not depend on the order the concurrent
|
||||
replicates finish in.
|
||||
|
||||
**R-work carries the decision, not R-free.** Nothing is refined against the working set here — the
|
||||
scale has a handful of parameters (a scale, an anisotropic $B$ and two solvent terms) and the placement six — so R-work carries no optimism, and it has an order
|
||||
of magnitude more reflections than R-free, and so that much more power to separate the two arms. R-free
|
||||
|
||||
+38
-11
@@ -641,6 +641,7 @@ ModelValidationResult ValidateAgainstModel(const std::vector<MergedReflection> &
|
||||
Fit fit;
|
||||
bool rb_applied = false;
|
||||
double rb_angle_deg = 0, rb_shift_A = 0, r_free_before_rb = 0;
|
||||
gemmi::Mat33 rb_rotation; // the committed rotation about the centroid, identity if none
|
||||
};
|
||||
auto place_and_fit = [&](ModelState &ms, const std::vector<MergedReflection> &obs_in,
|
||||
std::optional<Fit> fitted, double to_d) -> Placement {
|
||||
@@ -673,6 +674,7 @@ ModelValidationResult ValidateAgainstModel(const std::vector<MergedReflection> &
|
||||
if (commit) {
|
||||
out.rb_applied = true;
|
||||
out.rb_angle_deg = rb.angle_deg;
|
||||
out.rb_rotation = rb.rotation;
|
||||
out.rb_shift_A = rb.shift_A;
|
||||
out.r_free_before_rb = out.fit.r_free;
|
||||
out.fit = std::move(moved);
|
||||
@@ -857,20 +859,41 @@ ModelValidationResult ValidateAgainstModel(const std::vector<MergedReflection> &
|
||||
// Each replicate works on its own copy of the model and of its structure factors, and the
|
||||
// replicates share nothing else, so they run concurrently and the real model - which the
|
||||
// maps and the atom-density readout below are still taken from - is never moved.
|
||||
//
|
||||
// The check before the fit is only a filter: the rigid body can walk further than its nominal
|
||||
// reach, and a replicate it carries onto an orientation equivalent to the model's has been
|
||||
// refined into the model's own solution (measured: 24 deg, R-free 0.56 -> 0.39, which took the
|
||||
// null's spread from 0.016 to 0.066 and a fitting model below the gate). So where the
|
||||
// replicate ENDS is checked as well, and one that ended within reach is drawn again and
|
||||
// placed again - from a generator of its own, so the draws a replicate takes do not depend on
|
||||
// the order the concurrent replicates finish in.
|
||||
std::vector<double> null_r_work(NULL_REPLICATES), null_margin(NULL_REPLICATES);
|
||||
std::atomic<int> placed_redraws{0};
|
||||
auto run_replicate = [&](int i) {
|
||||
ModelState rep;
|
||||
rep.st = mdl.st;
|
||||
SetModelPositions(rep.st.models[0], as_read);
|
||||
reorient_about_centroid(rep.st.models[0], null_rotation[i]);
|
||||
compute_model_factors(rep, null_d_min);
|
||||
std::optional<Fit> identity_fit;
|
||||
if (!reindex_ops.empty()) {
|
||||
IndexingProbe probe = probe_indexing(rep, obs_null);
|
||||
null_margin[i] = probe.margin;
|
||||
identity_fit = std::move(probe.identity_fit); // replicates are placed against obs
|
||||
std::mt19937 rng(NULL_SEED + 1 + i);
|
||||
gemmi::Mat33 rot = null_rotation[i];
|
||||
for (int redraw = 0;; redraw++) {
|
||||
ModelState rep;
|
||||
rep.st = mdl.st;
|
||||
SetModelPositions(rep.st.models[0], as_read);
|
||||
reorient_about_centroid(rep.st.models[0], rot);
|
||||
compute_model_factors(rep, null_d_min);
|
||||
std::optional<Fit> identity_fit;
|
||||
if (!reindex_ops.empty()) {
|
||||
IndexingProbe probe = probe_indexing(rep, obs_null);
|
||||
null_margin[i] = probe.margin;
|
||||
identity_fit = std::move(probe.identity_fit); // replicates are placed against obs
|
||||
}
|
||||
const Placement placed = place_and_fit(rep, obs_null, std::move(identity_fit), null_d_min);
|
||||
null_r_work[i] = placed.fit.r_work;
|
||||
const gemmi::Mat33 ended = placed.rb_rotation.multiply(rot);
|
||||
if (redraw >= NULL_MAX_REDRAWS || angle_to_nearest_deg(ended, equivalent) >= reach_deg)
|
||||
break;
|
||||
++placed_redraws;
|
||||
rot = random_rotation(rng);
|
||||
for (int k = 0; k < NULL_MAX_REDRAWS && angle_to_nearest_deg(rot, equivalent) < reach_deg; k++)
|
||||
rot = random_rotation(rng);
|
||||
}
|
||||
null_r_work[i] = place_and_fit(rep, obs_null, std::move(identity_fit), null_d_min).fit.r_work;
|
||||
};
|
||||
// On threads of their own, not on ParallelFor's pool. A parallel pass reached from a pool
|
||||
// worker runs inline (ParallelFor.h), so there the scale fits and the rigid-body Jacobian of
|
||||
@@ -887,6 +910,10 @@ ModelValidationResult ValidateAgainstModel(const std::vector<MergedReflection> &
|
||||
}));
|
||||
for (std::future<void> &f : replicate_threads)
|
||||
f.get();
|
||||
if (placed_redraws > 0)
|
||||
logger.Info("Model validation: {} random placement(s) were refined to within the rigid body's reach "
|
||||
"({:.1f} deg) of an orientation equivalent to the model's, and were drawn and placed again",
|
||||
placed_redraws.load(), reach_deg);
|
||||
const auto [null_mean, null_sd] = mean_sd(null_r_work);
|
||||
result.fit_tested = true;
|
||||
result.null_replicates = NULL_REPLICATES;
|
||||
|
||||
@@ -540,6 +540,15 @@ RigidBodyRefineResult RefineRigidBody(gemmi::Model &model,
|
||||
result.b_sol = target.b_sol;
|
||||
const double aa = std::sqrt(q[0] * q[0] + q[1] * q[1] + q[2] * q[2]) / target.RmsRadius();
|
||||
result.angle_deg = aa * 180.0 / PI;
|
||||
const double axis_angle[3] = {q[0] / target.RmsRadius(), q[1] / target.RmsRadius(), q[2] / target.RmsRadius()};
|
||||
double column[3][3];
|
||||
for (int j = 0; j < 3; j++) {
|
||||
const double e[3] = {j == 0 ? 1.0 : 0.0, j == 1 ? 1.0 : 0.0, j == 2 ? 1.0 : 0.0};
|
||||
ceres::AngleAxisRotatePoint(axis_angle, e, column[j]);
|
||||
}
|
||||
result.rotation = gemmi::Mat33(column[0][0], column[1][0], column[2][0],
|
||||
column[0][1], column[1][1], column[2][1],
|
||||
column[0][2], column[1][2], column[2][2]);
|
||||
result.shift_A = std::sqrt(q[3] * q[3] + q[4] * q[4] + q[5] * q[5]);
|
||||
result.seconds = std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count();
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ class Logger;
|
||||
struct RigidBodyRefineResult {
|
||||
bool converged = false;
|
||||
double angle_deg = 0.0; // magnitude of the rotation about the model centroid
|
||||
gemmi::Mat33 rotation; // that rotation, Cartesian (identity where nothing moved)
|
||||
double shift_A = 0.0; // magnitude of the translation
|
||||
std::vector<double> zones; // the resolution ladder actually walked, coarsest first
|
||||
int evaluations = 0; // structure-factor evaluations (each one re-fits the scale)
|
||||
|
||||
@@ -1140,3 +1140,53 @@ TEST_CASE("ModelValidation_RigidBodyJacobianMatchesNumericScaleRefit", "[ModelVa
|
||||
CheckRigidBodyJacobian(cryst, far, {0, 1, 2, 3, 4, 5}, 0.975, 0.03);
|
||||
}
|
||||
}
|
||||
|
||||
// The null checks where a replicate ENDED against the orientations equivalent to the model's, from the
|
||||
// rotation the rigid body reports - which must be the rotation it applied: every atom's offset from the
|
||||
// centroid after the refinement is that rotation of its offset before.
|
||||
TEST_CASE("ModelValidation_RigidBodyReportsTheRotationItApplied", "[ModelValidation]") {
|
||||
Logger logger("ModelValidation_RigidBodyReportsTheRotationItApplied");
|
||||
const auto path = WriteTemp("rigid_body_rotation_test.pdb", ClusterPdb().c_str());
|
||||
gemmi::Structure st = gemmi::read_structure_gz(path, gemmi::CoorFormat::Detect);
|
||||
const gemmi::SpaceGroup *sg = st.find_spacegroup();
|
||||
REQUIRE(sg != nullptr);
|
||||
st.setup_cell_images();
|
||||
const auto ref = ModelReferenceIntensities(path, {}, {}, 3.0, logger);
|
||||
std::filesystem::remove(path);
|
||||
REQUIRE_FALSE(ref.empty());
|
||||
gemmi::AsuData<gemmi::ValueSigma<float>> fobs;
|
||||
fobs.unit_cell_ = st.cell;
|
||||
fobs.spacegroup_ = sg;
|
||||
for (const auto &r : ref)
|
||||
fobs.v.push_back({{{r.h, r.k, r.l}}, {std::sqrt(r.I), 1.0f}});
|
||||
fobs.ensure_sorted();
|
||||
|
||||
// Turned 3 deg about z through the centroid, so there is a rotation to take back.
|
||||
std::vector<gemmi::Position> turned = ModelPositions(st.models[0]);
|
||||
gemmi::Vec3 centre;
|
||||
for (const gemmi::Position &p : turned)
|
||||
centre += p;
|
||||
centre *= 1.0 / static_cast<double>(turned.size());
|
||||
const double a = 3.0 * PI / 180.0;
|
||||
const gemmi::Mat33 rz(std::cos(a), -std::sin(a), 0, std::sin(a), std::cos(a), 0, 0, 0, 1);
|
||||
for (gemmi::Position &p : turned)
|
||||
p = gemmi::Position(rz.multiply(gemmi::Vec3(p) - centre) + centre);
|
||||
SetModelPositions(st.models[0], turned);
|
||||
|
||||
const RigidBodyRefineResult result = RefineRigidBody(st.models[0], st.cell, *sg, fobs, 3.0, logger);
|
||||
REQUIRE(result.angle_deg > 1.0);
|
||||
const std::vector<gemmi::Position> refined = ModelPositions(st.models[0]);
|
||||
gemmi::Vec3 moved_centre;
|
||||
for (const gemmi::Position &p : refined)
|
||||
moved_centre += p;
|
||||
moved_centre *= 1.0 / static_cast<double>(refined.size());
|
||||
double worst = 0;
|
||||
for (size_t i = 0; i < refined.size(); i++) {
|
||||
const gemmi::Vec3 expected = result.rotation.multiply(gemmi::Vec3(turned[i]) - centre);
|
||||
worst = std::max(worst, (gemmi::Vec3(refined[i]) - moved_centre - expected).length());
|
||||
}
|
||||
CHECK(worst < 1e-6);
|
||||
const double trace = result.rotation[0][0] + result.rotation[1][1] + result.rotation[2][2];
|
||||
CHECK(std::acos(std::clamp((trace - 1) / 2, -1.0, 1.0)) * 180 / PI ==
|
||||
Catch::Approx(result.angle_deg).margin(1e-6));
|
||||
}
|
||||
Reference in new issue
Block a user