Stills -C: never dropped; the de novo comparison is a warning on the trial sample
Owner decision: the cell given (-C) is always the cell the run uses. The switch to the de novo cell and the second, de novo geometry refinement (+20-110 s per -C stills run) are removed. What stays: the free cell angles, -C as the prior of the global fit, and module placement from a held-modules fit of the same frames. The frames' disagreement with -C is reported, never acted on, at almost no cost: - the stills geometry trial (DecideStillsGeometry, already indexing a 96-frame sample with -C) also indexes that sample de novo at the geometry it decided and warns, with both counts, where the de novo frames put more than 1/0.9 times as many spots on the lattice as the cell given; - RefineStillsGeometry warns where the frames refine the shape of the cell given (a length beyond the common scale, which trades with the distance) more than three prior widths (3%) or an angle more than 3 degrees. Docs: RUGNUX_CLI (-C), CPU_DATA_ANALYSIS_INDEXING 7.7, RUGNUX_MODEL_AND_REFERENCE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK (cherry picked from commit 9b5531fe79012cef2f236b7eb29002f469ae8947)
This commit is contained in:
leonarski_f committed
1 parent
3d905099ca
commit
778b4d58cf
6 files changed
+53
-113
No files matched your search
@@ -412,7 +412,7 @@ The extraction window around a ring is capped at half the gap to its neighbour,
|
||||
|
||||
### 7.7 Stills: a given cell and the detector modules
|
||||
|
||||
**A given cell is a prior, and a hypothesis.** On stills a cell and a distance are degenerate near the
|
||||
**A given cell is a prior.** On stills a cell and a distance are degenerate near the
|
||||
beam - a spot's radius fixes only their ratio - and are told apart only by the curvature of
|
||||
$r=D\tan2\theta$ against $\sin\theta$ at high angle. The stills geometry refinement therefore refines
|
||||
the beam centre, the distance and the cell together over the strong frames: the cell lengths and the
|
||||
@@ -426,17 +426,14 @@ from (scale, each length, each angle) and how far the distance moved. A distance
|
||||
`--detector-distance` is held, and the cell alone takes up its error. Frames are matched to the given
|
||||
cell in any setting of the same lattice, and are described on its axes.
|
||||
|
||||
With `-C` the cell given is also tested as a hypothesis against the cell the frames index to de novo.
|
||||
The frames are first fitted de novo from the file's geometry, as without `-C`, and the cell given is then fitted
|
||||
from the detector distance, centre and module placement that fit found: from a start several percent off, a fit
|
||||
can settle as far off as it started, and the de novo fit, whose cell takes up the start's error, is the nearer
|
||||
start for any cell of the sample. Each, with its refined cell and geometry, then indexes the same sample of
|
||||
frames. The cell given is kept where
|
||||
it puts at least 0.9 times as many of the sample's spots on the lattice as the de novo cell does. A cell given
|
||||
that does not describe the frames - the same lattice in a shape they do not have, which the fit cannot leave
|
||||
because the frames were indexed on it - fails this, and the run is then processed as without `-C`, with a warning. So does a minority crystal form
|
||||
given with `-C` where the frames' de novo cell describes the majority well enough to index materially more of
|
||||
the sample's spots.
|
||||
With `-C` the run always uses the cell given; the frames' own evidence is reported, never acted on. Two
|
||||
warnings say where it disagrees with them, each with both numbers. Where the geometry refinement moves the cell's
|
||||
shape (a length beyond the common scale, which trades with the distance) more than 3 % or an angle more than 3°
|
||||
from what was given. And where, at the centre and distance the stills geometry trial decides (§1.4), the trial's
|
||||
96-frame sample indexed de novo puts more than 1/0.9 times as many spots on the lattice as the cell given: the cell
|
||||
given may then be another crystal form than most of the frames, or a shape they do not have - the same lattice in a
|
||||
shape it does not have still indexes the frames, at a distance of its own, and merges far worse than the run
|
||||
without `-C`.
|
||||
|
||||
**One crystal form.** A sample can hold two crystal forms, and a known-cell indexer fits both to the
|
||||
given cell's orientation search; their refined cells then fall into two volume populations, and
|
||||
|
||||
+1
-1
@@ -264,7 +264,7 @@ For very long axes the direction grid's angular resolution binds well below 500
|
||||
| `--force-still` | Treat a rotation (goniometer) dataset as independent stills instead of rotation |
|
||||
| `-X, --indexing-algorithm <txt>` | `Auto` (default) \| `FFT` \| `cuFFT` \| `FFTW` \| `FFBIDX` \| `None`, case-insensitive. `FFT` is the de-novo FFT search, on the GPU (cuFFT) where there is one and `--no-gpu` is not given, otherwise on the CPU (FFTW); `cuFFT` and `FFTW` name one implementation each; `cuFFT` needs a GPU and is refused without one. `FFBIDX` (known cell, `-C`) runs on the GPU where there is one and otherwise on the CPU. `Auto` takes `FFBIDX` on stills with a known cell, and `FFT` otherwise; rotation data always use the FFT search |
|
||||
| `-F` | Short for `-X FFT`, kept for old command lines |
|
||||
| `-C, --unit-cell <cell>` | Reference unit cell `"a,b,c,alpha,beta,gamma"` (required by `ffbidx`). On rotation data it also widens the FFT search to cover the cell given, at both ends: the longest axis looked for is raised to reach it, and `--fft-min-unit-cell` is lowered to admit it. A candidate that is an integer supercell of the cell given (a twin's coincidence lattice, a row taken at a multiple of its length) is stepped down to it. On stills the geometry refinement takes it as a prior (1 % on the lengths, 1° on the angles the crystal system leaves free) and tests it against the cell the frames index to de novo: a cell that, refined from the geometry the de novo fit found, puts fewer than 0.9 times as many sampled spots on the lattice as the de novo cell is dropped with a warning, and the run is processed as without `-C` ([§7.7](CPU_DATA_ANALYSIS_INDEXING.md#77-stills-a-given-cell-and-the-detector-modules)) |
|
||||
| `-C, --unit-cell <cell>` | Reference unit cell `"a,b,c,alpha,beta,gamma"` (required by `ffbidx`). On rotation data it also widens the FFT search to cover the cell given, at both ends: the longest axis looked for is raised to reach it, and `--fft-min-unit-cell` is lowered to admit it. A candidate that is an integer supercell of the cell given (a twin's coincidence lattice, a row taken at a multiple of its length) is stepped down to it. On stills the run always uses it: the geometry refinement takes it as a prior (1 % on the lengths, 1° on the angles the crystal system leaves free), and the log warns - with both numbers - where the frames refine its shape more than 3 % / 3° away, or where a sample of frames indexed de novo puts materially more spots on the lattice than the cell given ([§7.7](CPU_DATA_ANALYSIS_INDEXING.md#77-stills-a-given-cell-and-the-detector-modules)) |
|
||||
| `--fft-min-unit-cell <num>` | Shortest cell axis the FFT search accepts, Å (default: 10). A candidate with a shorter axis is discarded — but on rotation data a row shorter than the floor, down to 3 Å, is recognised under the multiple of it the search reports past the floor, so a small-molecule cell is indexed without this flag. `-C` lowers the floor on its own to cover the cell given. On stills a crystal below the floor cannot be indexed unless this is lowered |
|
||||
| `--min-indexed-spots <num>` | Spots a frame must have on the lattice before it counts as indexed (default: 6, minimum 4). It sets the reported indexing rate and the count the rotation first pass ranks candidate lattices by; whether the run has a lattice at all is decided on the pooled spots instead ([§4.1](CPU_DATA_ANALYSIS_INDEXING.md#41-indexed-spot-decision-inlier-test)), and integration is not gated by it |
|
||||
| `-S, --space-group <num\|symbol>` | Space group number (`96`) or Hermann-Mauguin symbol (`P43212`) — for indexing and scaling. On serial stills whose group admits two indexings, it also lets the crystals be put on one from the data ([the indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity)) |
|
||||
|
||||
@@ -58,7 +58,7 @@ the rest walks through a run step by step.
|
||||
| `--reference-column <label>` | An MTZ column label | Which column of the reference is read | Through `-z` only |
|
||||
| `--free-flags <file>` | Merged data carrying an R-free set: an MTZ, or a structure-factor mmCIF - typically the deposited reflections of the `--model` | Its R-free set and nothing else: the model is validated and refined against it, on the model's axes, where the merge matches the file's intensities (`R_FREE_FLAGS= FILE`) | The free flags only |
|
||||
| `-S, --space-group` | A space group | Fixes it: no space-group search | **Yes**: the group is yours (a fitting model may still relabel its hand) |
|
||||
| `-C, --unit-cell` | A cell | Indexing looks for this cell (within tolerance); the FFT search range is widened to cover it; on stills the shared-geometry refinement anchors on it as a prior (without it, on the cell the frames index to), and drops it, as without `-C`, where it puts materially fewer of a sample's spots on the lattice than the cell the frames index to de novo and, with `-S` and `--model`, the per-image use of the model | **Yes**: the cell, and the axis order the files are written in |
|
||||
| `-C, --unit-cell` | A cell | Indexing looks for this cell (within tolerance); the FFT search range is widened to cover it; on stills the shared-geometry refinement anchors on it as a prior (without it, on the cell the frames index to), and warns, keeping it, where it puts materially fewer of a sample's spots on the lattice than the frames indexed de novo and, with `-S` and `--model`, the per-image use of the model | **Yes**: the cell, and the axis order the files are written in |
|
||||
| `--force-rotation-lattice` | Nine numbers: a lattice | Skips the first-pass indexing and uses this lattice | **Yes**: the lattice |
|
||||
| `--report-resolution <dmin>[,<dmax>]` | Another program's resolution range | A second statistics table at that range (`REFRES_*`) | No — report only |
|
||||
|
||||
|
||||
+2
-6
@@ -975,12 +975,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
|
||||
std::future<saturation_ceiling::Histogram> saturation;
|
||||
if (full && !experiment_.IsRotationIndexing() && !config_.live)
|
||||
saturation = std::async(std::launch::async, [&] { return SampleSaturation(start_image, images_to_process); });
|
||||
if (full && config_.refine_geometry.has_value() && !config_.detector_distance_unknown) {
|
||||
if (stills_user_cell_ && !config_.detector_distance_given && !config_.live)
|
||||
RefineStillsGeometryOrDeNovo(start_image, end_image, images_to_process, observer);
|
||||
else
|
||||
RefineStillsGeometry(start_image, end_image, images_to_process, observer);
|
||||
}
|
||||
if (full && config_.refine_geometry.has_value() && !config_.detector_distance_unknown)
|
||||
RefineStillsGeometry(start_image, end_image, images_to_process, observer);
|
||||
// The detector tilt's fits read only the frames just fitted and the geometry they left: started now, they
|
||||
// run beside the pre-scan and the geometry trial, and are used where neither moved the geometry.
|
||||
if (full)
|
||||
|
||||
@@ -1760,10 +1760,6 @@ class Rugnux {
|
||||
// cannot run. Stills only; rotation has its own two-pass.
|
||||
void RefineStillsGeometry(int start_image, int end_image, int images_to_process,
|
||||
RugnuxObserver *observer);
|
||||
// Stills with a cell given (-C): RefineStillsGeometry with the cell given, and again de novo, each from the
|
||||
// file's geometry, and the run keeps the hypothesis the frames bear out (see the definition).
|
||||
void RefineStillsGeometryOrDeNovo(int start_image, int end_image, int images_to_process,
|
||||
RugnuxObserver *observer);
|
||||
|
||||
// Stills: the beam centre and the detector distance as hypotheses, each indexed on the same sample
|
||||
// of frames and decided on what that sample shows (see DecideStillsGeometry).
|
||||
|
||||
+40
-89
@@ -699,107 +699,35 @@ void Rugnux::RefineStillsGeometry(int start_image, int end_image, int images_to_
|
||||
r.cell.a, r.cell.b, r.cell.c, r.cell.alpha, r.cell.beta, r.cell.gamma,
|
||||
r.median_residual_px, r.frames_used, r.spots_used);
|
||||
// How far the refined cell is from the one the fit was anchored on: in scale (the cube root of the volume),
|
||||
// per length and per angle, beside the distance's move. Whether a cell given (-C) describes the frames at all
|
||||
// is decided apart (RefineStillsGeometryOrDeNovo).
|
||||
// per length and per angle, beside the distance's move.
|
||||
const double scale = std::cbrt(CrystalLattice(r.cell).CalcVolume() / CrystalLattice(*cell).CalcVolume()) - 1.0;
|
||||
logger.Info("Geometry refinement: refined cell against the {} cell: scale {:+.2f}%, lengths {:+.2f}/{:+.2f}/{:+.2f}%, "
|
||||
"angles {:+.2f}/{:+.2f}/{:+.2f} deg; distance {:+.2f}%", stills_user_cell_ ? "given" : "de novo",
|
||||
100 * scale, 100 * (r.cell.a / cell->a - 1.0), 100 * (r.cell.b / cell->b - 1.0),
|
||||
100 * (r.cell.c / cell->c - 1.0), r.cell.alpha - cell->alpha, r.cell.beta - cell->beta,
|
||||
r.cell.gamma - cell->gamma, 100 * (r.distance_mm / experiment_.GetDetectorDistance_mm() - 1.0));
|
||||
// A cell given (-C) whose shape the frames refine far outside its prior (three of its widths) is worth the
|
||||
// user's attention: the cell is kept, as given, but it may not describe these crystals. The scale is left
|
||||
// out - it trades with the distance, and the distance may still be decided (DecideStillsGeometry).
|
||||
if (stills_user_cell_) {
|
||||
const double shape = 100 * std::max({std::abs(r.cell.a / cell->a / (1 + scale) - 1),
|
||||
std::abs(r.cell.b / cell->b / (1 + scale) - 1),
|
||||
std::abs(r.cell.c / cell->c / (1 + scale) - 1)});
|
||||
const double angle = std::max({std::abs(r.cell.alpha - cell->alpha), std::abs(r.cell.beta - cell->beta),
|
||||
std::abs(r.cell.gamma - cell->gamma)});
|
||||
if (shape > 3 * 100 * gr_settings.cell_length_sigma || angle > 3 * gr_settings.cell_angle_sigma_deg)
|
||||
logger.Warning("The frames refine the shape of the cell given (-C) {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} "
|
||||
"to {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f}: up to {:.1f}% in a length beyond the common "
|
||||
"scale and {:.1f} deg in an angle. The run uses the cell given; it may not describe these "
|
||||
"crystals", cell->a, cell->b, cell->c, cell->alpha, cell->beta, cell->gamma,
|
||||
r.cell.a, r.cell.b, r.cell.c, r.cell.alpha, r.cell.beta, r.cell.gamma, shape, angle);
|
||||
}
|
||||
|
||||
experiment_.BeamX_pxl(r.beam_x_px).BeamY_pxl(r.beam_y_px).DetectorDistance_mm(r.distance_mm);
|
||||
experiment_.SetUnitCell(r.cell);
|
||||
stills_geometry_frames_ = r.frames_used;
|
||||
}
|
||||
|
||||
// The cell given (-C) is a hypothesis, and the cell the frames index to de novo another. A cell of the wrong shape
|
||||
// still indexes the frames, at a distance of its own: given 4% too long with its beta 4 degrees off, a cell
|
||||
// indexed the frames of a crystal at a distance 7% too long, refined towards its own shape there, and merged to
|
||||
// CC1/2 0.4 where the run without -C merged to 0.9. The frames are first fitted de novo from the file's geometry,
|
||||
// as without -C, and the cell given is then fitted from where that put the detector: a cell fitted from a start
|
||||
// distance 5% off can settle as far off as it started (a right cell on weak frames moved 6% from a start 3% off),
|
||||
// and the de novo fit, whose cell takes up the start's error, is the nearer start for any cell of the sample. Each
|
||||
// is then indexed with its refined cell and geometry on the same sample of frames, as DecideStillsGeometry tries
|
||||
// its centres and distances. The cell given is kept where it puts as many of the sample's spots on the lattice,
|
||||
// or nearly (GIVEN_KEPT_ABOVE, as a file's centre is kept); otherwise the run goes on as it would without -C,
|
||||
// from the de novo hypothesis, and says so. Whether the two fits put the detector at the same distance does not
|
||||
// tell a minority crystal form from a cell of the wrong shape: started from the de novo fit's detector, a cell
|
||||
// 5% too long with its beta 4 degrees off settled within 0.3% of it while putting 6% as many spots on the lattice. Where nothing indexes de novo, the cell given is fitted
|
||||
// from the file's geometry, as without this test.
|
||||
void Rugnux::RefineStillsGeometryOrDeNovo(int start_image, int end_image, int images_to_process,
|
||||
RugnuxObserver *observer) {
|
||||
Logger logger("Rugnux");
|
||||
constexpr double GIVEN_KEPT_ABOVE = 0.9;
|
||||
constexpr int SAMPLE_FRAMES = 96;
|
||||
|
||||
struct State {
|
||||
DiffractionExperiment experiment;
|
||||
std::vector<ModuleShift> module_shifts;
|
||||
bool modules_placed;
|
||||
std::optional<StillsTiltBundle> tilt_bundle;
|
||||
int geometry_frames;
|
||||
};
|
||||
const auto save = [&] {
|
||||
return State{experiment_, module_shifts_, modules_placed_, stills_tilt_bundle_, stills_geometry_frames_};
|
||||
};
|
||||
const auto restore = [&](const State &st) {
|
||||
experiment_ = st.experiment;
|
||||
module_shifts_ = st.module_shifts;
|
||||
modules_placed_ = st.modules_placed;
|
||||
stills_tilt_bundle_ = st.tilt_bundle;
|
||||
stills_geometry_frames_ = st.geometry_frames;
|
||||
};
|
||||
|
||||
const State start = save();
|
||||
const UnitCell given_cell = *stills_user_cell_;
|
||||
stills_user_cell_.reset();
|
||||
experiment_.SetUnitCell(std::nullopt);
|
||||
RefineStillsGeometry(start_image, end_image, images_to_process, observer);
|
||||
const State de_novo = save();
|
||||
stills_user_cell_ = given_cell;
|
||||
if (cancelled_ || !de_novo.experiment.GetUnitCell().has_value()) {
|
||||
restore(start);
|
||||
RefineStillsGeometry(start_image, end_image, images_to_process, observer);
|
||||
return;
|
||||
}
|
||||
|
||||
// The cell given, from the detector (and the modules) the de novo fit left.
|
||||
experiment_.SetUnitCell(given_cell);
|
||||
RefineStillsGeometry(start_image, end_image, images_to_process, observer);
|
||||
const State given = save();
|
||||
if (cancelled_)
|
||||
return;
|
||||
|
||||
const std::vector<int> sample = select_equally_spaced_image_ordinals(
|
||||
images_to_process, std::min(images_to_process, SAMPLE_FRAMES));
|
||||
const std::vector<DataMessage> frames = FindStillsSampleSpots(sample, start_image);
|
||||
IndexingSettings pool_settings = experiment_.GetIndexingSettings();
|
||||
pool_settings.IndexingThreads(StillsIndexerThreads());
|
||||
IndexerThreadPool pool(pool_settings, IndexerConstruction::OnFirstUse, gpu_work_beside_);
|
||||
const auto trials = IndexStillsSample({{given.experiment, &frames}, {de_novo.experiment, &frames}}, pool);
|
||||
|
||||
const UnitCell g = given.experiment.GetUnitCell().value();
|
||||
const UnitCell d = de_novo.experiment.GetUnitCell().value();
|
||||
const double distance_g = given.experiment.GetDetectorDistance_mm();
|
||||
const double distance_d = de_novo.experiment.GetDetectorDistance_mm();
|
||||
logger.Info("Cell hypotheses: the cell given, refined to {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} at {:.2f} mm, puts "
|
||||
"{} sampled spots on the lattice; the cell indexed de novo, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} at "
|
||||
"{:.2f} mm, {}", g.a, g.b, g.c, g.alpha, g.beta, g.gamma, distance_g, trials[0].on_lattice,
|
||||
d.a, d.b, d.c, d.alpha, d.beta, d.gamma, distance_d, trials[1].on_lattice);
|
||||
if (trials[0].on_lattice >= GIVEN_KEPT_ABOVE * trials[1].on_lattice) {
|
||||
restore(given);
|
||||
return;
|
||||
}
|
||||
restore(de_novo);
|
||||
stills_user_cell_.reset();
|
||||
logger.Warning("The cell given (-C) does not describe these frames: with it the detector is at {:.1f} mm and {} "
|
||||
"sampled spots are on the lattice, with the cell they index to de novo ({:.2f} {:.2f} {:.2f} {:.2f} "
|
||||
"{:.2f} {:.2f}) it is at {:.1f} mm and {} are. The run is processed as without -C",
|
||||
distance_g, trials[0].on_lattice, d.a, d.b, d.c, d.alpha, d.beta, d.gamma, distance_d,
|
||||
trials[1].on_lattice);
|
||||
}
|
||||
|
||||
// The detector tilt of a stills run, as a hypothesis against the tilt the run has (the file's, normally none):
|
||||
// the strong frames of the geometry refinement are dealt alternately into two halves, each half is fitted with
|
||||
// the tilt held and with it refined - the beam, the distance and the cell refined in both, the modules held
|
||||
@@ -1347,6 +1275,29 @@ bool Rugnux::DecideStillsGeometry(int start_image, int images_to_process) {
|
||||
}
|
||||
}
|
||||
|
||||
// With -C: the same sample indexed de novo at the geometry decided, against the cell given there. The run uses
|
||||
// the cell given whatever this says; where the frames index materially more of their spots de novo (below
|
||||
// FILE_KEPT_ABOVE of the de novo count, the margin a file's value is kept by), the cell given may be another
|
||||
// crystal form than most of the frames, or a shape they do not have, and the log says so with both counts. A
|
||||
// cell of the wrong shape still indexes the frames, at a distance of its own, and merges far worse than the run
|
||||
// without -C. Ninety-six frames, a few seconds.
|
||||
if (stills_user_cell_) {
|
||||
DiffractionExperiment e = experiment_;
|
||||
place(e, x, y, distance);
|
||||
e.SetUnitCell(std::nullopt);
|
||||
const int64_t given = trial(x, y, distance, frames);
|
||||
const StillsTrial de_novo = IndexStillsSample({{e, &frames}}, pool).at(0);
|
||||
logger.Info("Stills cell check at {:.2f} mm: the cell given puts {} sampled spots on the lattice, the frames "
|
||||
"indexed de novo {}", distance, given, de_novo.on_lattice);
|
||||
if (given < FILE_KEPT_ABOVE * de_novo.on_lattice)
|
||||
logger.Warning("The cell given (-C) {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} puts {} of a sample's spots on the "
|
||||
"lattice at {:.1f} mm, where the frames indexed de novo put {}: the run uses the cell given, "
|
||||
"but it may be another crystal form than most of the frames, or a shape they do not have - "
|
||||
"compare with a run without -C",
|
||||
stills_user_cell_->a, stills_user_cell_->b, stills_user_cell_->c, stills_user_cell_->alpha,
|
||||
stills_user_cell_->beta, stills_user_cell_->gamma, given, distance, de_novo.on_lattice);
|
||||
}
|
||||
|
||||
logger.Info("Stills beam centre: the run uses the direct beam at ({:.2f},{:.2f}){}", x, y,
|
||||
x == file_x && y == file_y ? ", the file's" : "");
|
||||
place(experiment_, x, y, distance);
|
||||
|
||||
Reference in new issue
Block a user