leonarski_fandClaude Opus 5
ae126c3d5b
`RefineGeometryIfNeeded` hands XtalOptimizer the WHOLE spot list, not the
indexed subset, and the first pass admits anything within 0.3 fractional-Miller
units of an integer - which is 11.3% of RANDOMLY placed spots, since the
admitted volume is (4/3)*pi*t^3. Every one of them then enters an unweighted L2
fit with an arbitrary rounded index. On images with many detections the
refined orientation ends up 2.3-2.8 degrees from the goniometer-consistent one
and explains 14 of its own 250 spots where the undragged orientation explains
68; mosaicity and profile radius inherit the error and integration follows.
Weight every spot by its intensity divided by the median intensity of its own
equal-count resolution shell, applied as w^2 on the squared residual with
w^2 = r/(1+r). The shell normalisation is the point: refinement needs the
high-resolution spots because they carry the cell and distance, and those are
LEGITIMATELY weaker, so a raw intensity weight would suppress exactly the
spots the fit depends on. Measured, the weight is resolution-neutral - median
exactly 0.707 in every shell, and corr(w, 1/d^2) = -0.20 / -0.11 against
-0.32 / -0.34 for the same function of un-normalised intensity.
This is a PRIOR: it is computed from the spot alone and never looks at the
current residual, so unlike a robust loss it cannot mistake a genuine spot for
an outlier while the starting geometry is still far off and leave the fit
unable to move. That failure is not hypothetical - a CauchyLoss on this same
residual, at the scale the multi-frame GeometryRefiner uses, collapsed one
crystal's indexing rate from 99.89% to 19.83% and was rejected.
It does not work by telling good spots from bad, and it does not need to. No
per-spot property separates spots that index from spots that do not: measured
AUC is 0.53 for peak pixel, 0.53 for total intensity, 0.51 for pixel count,
0.45 for peakedness, and a logistic regression on all twelve available
features with pairwise interactions reaches only 0.64. What the weight does is
halve the EFFECTIVE COUNT of every spot (mean w^2 = 0.517), and the damage
scales with the absolute count of unexplained spots in the objective - 80.6
per frame here against 36.8 for the finder that was never damaged. That is
also why an empirical `--max-spots 66` cap works while leaving the list no
purer than before: it reaches the same operating point by discarding spots.
This reaches it without discarding any, and without a tuned constant.
Rotation battery, 33 crystals, both spot finders:
finder A 29/33 -> 30/33 point groups (one crystal P222 -> P4212 = XDS,
its high-shell CC1/2 86.0 -> 98.4)
finder B 28/33 -> 29/33 point groups (one crystal I222 -> I23,
its high-shell CC1/2 14.8 -> 38.0)
No crystal lost its point group in either mode and no run failed. On the
meta-stable multi-lattice dataset the CC1/2 spread over four frame ranges
falls 19.7 -> 13.1 for finder B, and the indexing rate rises in 8 of 8
configurations. The crystal that the rejected robust loss destroyed keeps its
99.89% indexing rate exactly.
The cost, stated plainly: ISa falls by 0.2-1.7 on about five crystals (and
rises on two). Point-group correctness is worth more than that - merging in
the wrong symmetry cannot be undone from the output, whereas ISa is a quality
metric of data that remain correct - but it is a real trade and not a free win.
Off by default. The indexers pass a spot list they have already selected, so
their calls are unchanged; only the per-image refinement, which gets the raw
list, turns it on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
59 lines
2.1 KiB
C++
59 lines
2.1 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <optional>
|
|
|
|
#include "../common/GoniometerAxis.h"
|
|
#include "../common/CrystalLattice.h"
|
|
#include "../common/DiffractionGeometry.h"
|
|
#include "../common/SpotToSave.h"
|
|
#include "gemmi/symmetry.hpp"
|
|
|
|
struct XtalOptimizerData {
|
|
DiffractionGeometry geom;
|
|
CrystalLattice latt;
|
|
gemmi::CrystalSystem crystal_system = gemmi::CrystalSystem::Triclinic;
|
|
int64_t min_spots = 8;
|
|
|
|
float min_length_A = 5.0;
|
|
float max_length_A = 500.0;
|
|
float min_angle_deg = 60.0f;
|
|
float max_angle_deg = 120.0f;
|
|
|
|
bool refine_beam_center = true;
|
|
bool refine_distance_mm = false;
|
|
bool refine_detector_angles = false;
|
|
bool refine_unit_cell = true; // This refines unit cell size + angles - orientation is always refined
|
|
bool refine_rotation_axis = false;
|
|
|
|
bool index_ice_rings = true;
|
|
|
|
// Weight each spot by how strong it is for its resolution, so that low-confidence spots contribute
|
|
// without driving the fit (see SpotConfidenceWeights). Off by default: the indexers call this with a
|
|
// spot list they have already selected, it is the per-image refinement that gets the raw list.
|
|
bool weight_spots_by_confidence = false;
|
|
|
|
float max_time = 1.0;
|
|
|
|
std::optional<GoniometerAxis> axis;
|
|
|
|
// output
|
|
std::optional<double> beam_corr_x;
|
|
std::optional<double> beam_corr_y;
|
|
|
|
// For rotation only optimizer
|
|
std::optional<double> angle_corr;
|
|
std::optional<Coord> angle_axis;
|
|
};
|
|
|
|
// num_threads sets the Ceres solver thread count for the internal least-squares refine. It defaults
|
|
// to 1 because XtalOptimizer is usually called from many threads at once; raise it only when a caller
|
|
// runs a small number of refinements concurrently and wants each to use several cores.
|
|
bool XtalOptimizer(XtalOptimizerData &data, const std::vector<std::vector<SpotToSave>> &spots,
|
|
int num_threads = 1);
|
|
bool XtalOptimizerRotationOnly(XtalOptimizerData &data, const std::vector<SpotToSave> &spots, float tolerance);
|
|
|
|
|