Serial stills: the indexing ambiguity resolved from the data when no reference or model is given
Where the space group is given (-S) and its lattice admits alternative indexings, a stills run without -z or --model used to merge the crystals' random indexings together (and warned). The crystals are now made consistent with each other before anything is scaled (ReferenceFreeReindex): each takes the indexing in which it agrees best with all the others (Brehm & Diederichs 2014), agreement being the sum over shared reflections of the products of shell-normalised, per-crystal standardised intensities, weighted by 1/(1+(sigma/sd)^2). The sum over all pairs is (|M|^2 - const)/2 for M the running sum, so a pass over the crystals is one pass over the reflections rather than over all pairs: coordinate ascent, four seeded starts, stop when <= 1 crystal in 1000 moves, the overall relabel that moves the fewest crystals. Deterministic (fixed summation order, sequential passes). A twin law of a pseudo-merohedral cell whose axes differ by a few per cent is decided by the crystals' own refined cells, which the indexer has already used: where >= 90% of the crystals' cells fit better as indexed, those crystals are anchors and only the others are decided by the intensities. Without this, two public sets got worse (pal90 R-free 0.183 -> 0.207, lcls242_t 0.296 -> 0.350). With a reference or model the per-image choice stands; the same resolution runs as a check, reported as AMBIGUITY_DATA_DISAGREES. New report keys (section 2): INDEXING_AMBIGUITY_RESOLVED, AMBIGUITY_CRYSTALS, AMBIGUITY_REINDEXED, AMBIGUITY_SEPARATION_SIGMA, AMBIGUITY_DECIDED_FRACTION (+ AMBIGUITY_TIME_S, dev). --mode scale is not changed. Still battery, no --model (R-free, REFMAC 0-cycle against the deposited model; before -> after; with --model for comparison): euxfel83 0.321 -> 0.207 (0.204), euxfel152 0.346 -> 0.257 (0.245), alvra213 0.209 -> 0.201 (0.207), alvra213_int16 0.210 -> 0.202 (0.208), pal90 0.183 -> 0.183 (0.188), lcls242_t 0.296 -> 0.294 (0.285); controls sacla237 / lcls222 unchanged (no ambiguity, no-op). The step takes 0.2-1.6 s on 2.7k-54k crystals; synthetic 64k crystals x 200 reflections 0.5 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 80d378a1fe92364fd5b99a17f5c9ee8498412d36)
This commit is contained in:
leonarski_f committed
1 parent
2daa173859
commit
07ae590e54
15 files changed
+756
-25
No files matched your search
@@ -367,6 +367,16 @@ S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descrip
|
||||
crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995),
|
||||
Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340).
|
||||
|
||||
**Indexing ambiguity of serial data without a reference** — where a serial run's lattice is more
|
||||
symmetric than its crystals and no reference or model is given, each crystal is put into the indexing in
|
||||
which its intensities agree best with those of all the other crystals, crystal by crystal until none
|
||||
changes. The objective - the summed agreement between every pair of crystals - and the iteration that
|
||||
maximises it are taken from Brehm and Diederichs, whose method CrystFEL's `ambigator` also implements;
|
||||
rugnux measures the agreement as a product of standardised intensities rather than a correlation, which
|
||||
lets the sum over all partners be taken against one running sum instead of pair by pair.
|
||||
W. Brehm and K. Diederichs, "Breaking the indexing ambiguity in serial crystallography" (2014),
|
||||
Acta Cryst. D70, 101-109 [doi:10.1107/S1399004713025431](https://doi.org/10.1107/S1399004713025431).
|
||||
|
||||
### Physical corrections and calibration
|
||||
|
||||
**Sensor absorption at oblique incidence, and the flight path** — the angle-dependent quantum
|
||||
|
||||
@@ -87,6 +87,7 @@ The methods draw on, and in places reimplement, solutions from:
|
||||
- C. Vonrhein, C. Flensburg, P. Keller et al., "Data processing and analysis with the autoPROC toolbox", *Acta Cryst.* **D67** (2011), 293-302 (the inspiration for not trusting header geometry and for finding the beam centre from the images, §1.4).
|
||||
- P. R. Evans & G. N. Murshudov, "How good are my data and what is the resolution?", *Acta Cryst.* **D69** (2013), 1204-1214 (AIMLESS: the anisotropic deltaB as the range of the principal components, and diffraction limits from a cone about each principal direction).
|
||||
- G. Assmann, W. Brehm & K. Diederichs, "Identification of rogue datasets in serial crystallography", *J. Appl. Cryst.* **49** (2016), 1021-1028, and G. M. Assmann, M. Wang & K. Diederichs, *Acta Cryst.* **D76** (2020), 636-652 (XDSCC12: sigma-tau CC1/2, delta-CC1/2, the Fisher transformation and the rejection discipline the frame disposition follows).
|
||||
- W. Brehm & K. Diederichs, "Breaking the indexing ambiguity in serial crystallography", *Acta Cryst.* **D70** (2014), 101-109 (serial stills without a reference: each crystal put into the indexing in which it agrees best with all the others).
|
||||
- K. Diederichs & P. A. Karplus, *Nat. Struct. Biol.* **4** (1997), 269-275, and P. A. Karplus & K. Diederichs, *Science* **336** (2012), 1030-1033 (R_meas / R_pim, CC1/2 and CC\*).
|
||||
- M. S. Weiss, "Global indicators of X-ray data quality", *J. Appl. Cryst.* **34** (2001), 130-135 (R_pim).
|
||||
- IUCr Commission on Crystallographic Nomenclature, "Statistical descriptors in crystallography", *Acta Cryst.* **A45** (1989), 63-75, and *Acta Cryst.* **A51** (1995), 565-569 (uncertainty conventions).
|
||||
|
||||
@@ -471,7 +471,11 @@ A reference dataset (`--reference`, MTZ or SF-mmCIF) supplies known intensities
|
||||
|
||||
**Fix the space group and cell.** Unless overridden on the command line (`-S` for the space group, `-C` for the cell), the reference's space group is adopted and its cell is used as the soft reference cell — indexing may still drift the cell within tolerance, so a small mismatch between reference and data is absorbed rather than rejected. This applies to both stills and rotation data.
|
||||
|
||||
**Resolve the indexing (merohedral) ambiguity.** When the lattice symmetry is higher than the crystal's Laue symmetry (e.g. $P3$, $P4$, $P6$, $C2$), more than one indexing of the same lattice is geometrically valid, and the two solutions produce *different* merged intensities that a self-consistent scale cannot tell apart — only an external reference can. The candidate reindexings are the identity together with the twin-law cosets of the metric symmetry (from the unit-cell metric and the Laue group); each is scored by the intensity correlation $\mathrm{CC}_\mathrm{ref}$ of the reindexed merge against the reference, and the data are re-merged in the best-correlating indexing. The reindex is **metric-preserving** — only the $hkl$ labels change, the cell is unchanged — and it is a no-op for a holohedral crystal, which has no twin laws (the lattice and Laue symmetry coincide). For rotation data this is done once, after the space group is determined, and the whole merge is then repeated in the chosen indexing. For stills it has to be done **per image**, at integration time: each crystal is indexed independently, so a run resolves the ambiguity image by image (the image's partiality/Lorentz-corrected intensities are correlated with the reference under each candidate operator, which is scale-invariant, and the best-correlating one is adopted once and for good) — otherwise the merge would average reflections that are not symmetry mates. In neither workflow is the reference a **scale** target: both scale against their own data (§10.2), so $\mathrm{ISa}$ and the merging statistics come from the data alone and no cross-dataset systematic is imported. Because the stills choice is made at integration time, a later re-merge of stored reflections cannot repair a dataset integrated without a reference. Where there is no reference dataset but there is a **model** (`--model`), the reference intensities are computed from it instead - $|F_\mathrm{model}|^2$ from the atomic structure factors with a flat bulk-solvent contribution at the standard constants ($k_\mathrm{sol}=0.35$, $B_\mathrm{sol}=46$ Å$^2$), which are not fitted because there are no observations yet. Nothing is scaled against them; they serve only to rank the candidate indexings, and the correlation that does the ranking is scale-invariant. This needs the cell and space group up front (`-C` / `-S`, as serial indexing wants anyway); on rotation data the same job is done after the merge, in §14.5, where a merge exists to fit the model to properly.
|
||||
**Resolve the indexing (merohedral) ambiguity.** When the lattice symmetry is higher than the crystal's Laue symmetry (e.g. $P3$, $P4$, $P6$, $C2$), more than one indexing of the same lattice is geometrically valid, and the two solutions produce *different* merged intensities that a self-consistent scale cannot tell apart — only an external reference can. The candidate reindexings are the identity together with the twin-law cosets of the metric symmetry (from the unit-cell metric and the Laue group); each is scored by the intensity correlation $\mathrm{CC}_\mathrm{ref}$ of the reindexed merge against the reference, and the data are re-merged in the best-correlating indexing. The reindex is **metric-preserving** — only the $hkl$ labels change, the cell is unchanged — and it is a no-op for a holohedral crystal, which has no twin laws (the lattice and Laue symmetry coincide). For rotation data this is done once, after the space group is determined, and the whole merge is then repeated in the chosen indexing. For stills it has to be done **per image**, at integration time: each crystal is indexed independently, so a run resolves the ambiguity image by image (the image's partiality/Lorentz-corrected intensities are correlated with the reference under each candidate operator, which is scale-invariant, and the best-correlating one is adopted once and for good) — otherwise the merge would average reflections that are not symmetry mates. In neither workflow is the reference a **scale** target: both scale against their own data (§10.2), so $\mathrm{ISa}$ and the merging statistics come from the data alone and no cross-dataset systematic is imported. Because the stills choice is made at integration time, a later re-merge of stored reflections (`--mode scale`) cannot repair a dataset integrated without a reference; the data-only resolution below runs in the full pipeline only. Where there is no reference dataset but there is a **model** (`--model`), the reference intensities are computed from it instead - $|F_\mathrm{model}|^2$ from the atomic structure factors with a flat bulk-solvent contribution at the standard constants ($k_\mathrm{sol}=0.35$, $B_\mathrm{sol}=46$ Å$^2$), which are not fitted because there are no observations yet. Nothing is scaled against them; they serve only to rank the candidate indexings, and the correlation that does the ranking is scale-invariant. This needs the cell and space group up front (`-C` / `-S`, as serial indexing wants anyway); on rotation data the same job is done after the merge, in §14.5, where a merge exists to fit the model to properly.
|
||||
|
||||
**Serial stills without a reference or model.** The data cannot say which indexing is the *true* one, but they can say which crystals are in the *same* one, and that is all a merge needs. Where the space group is fixed (`-S`) and its lattice admits alternative indexings, the crystals are made consistent with each other before anything is scaled, following Brehm & Diederichs (2014). Each crystal's partiality- and Lorentz-corrected intensities are divided by the run's mean intensity in their resolution shell (20 shells of equal reciprocal volume), standardised within the crystal so that its unknown scale and offset drop out, and weighted by $1/(1+(\sigma/s)^2)$, $s$ being the crystal's spread, so that a weak or barely recorded reflection counts for little. The agreement of two crystals is the sum of the products of these values over the reflections they share, and the objective is that agreement summed over every pair of crystals, each in the indexing assigned to it. That sum is $(|M|^2 - \mathrm{const})/2$ with $M$ the per-reflection sum over all crystals, so a crystal's agreement with all the others is one dot product with $M$ less its own contribution: each crystal in turn takes the indexing in which that dot product is largest, $M$ is updated, and the passes repeat until no more than one crystal in a thousand moves. A pass costs one read of the reflections, not one per pair of crystals, which is what keeps tens of thousands of crystals to seconds. The objective only grows, so it converges; it is started four times (once from the indexing as found, three times from a seeded random assignment) and the start that ends highest is kept, and of the equally consistent answers related by an overall relabelling the one that moves the fewest crystals is taken. The order of every floating-point sum is fixed (crystals in image order, sequential passes), so the result does not depend on the thread count. The run reports how clearly the crystals separated: for each crystal the correlation with the mean of all the others in its chosen indexing and in the best other one, compared by Fisher's $z$ in units of its standard error ($\sqrt{2/(n-3)}$ for $n$ shared reflections) — the median of that margin and the share of crystals above $3\sigma$.
|
||||
|
||||
A twin law is only an ambiguity for crystals whose own lattice cannot tell the indexings apart. On a pseudo-merohedral cell whose axes differ by a few per cent, each crystal's refined cell already says which axis is which, and the indexer has used that; letting the intensities overrule it there moves crystals that were right. So for each law the share of crystals whose refined cell fits the run's cell better as indexed than after the law is measured (a tie counting half — on an exactly merohedral lattice the share is about one half). Where it is at least 0.9 the law is decided by the cells: a crystal whose cell prefers its indexing stays there, as an anchor, and only the crystals whose cell does not are decided by the intensities, against the anchors. With a reference or a model the per-image choice above stands, and the same resolution is run only to measure it: the report gives the number of crystals on which the data alone would have chosen otherwise.
|
||||
|
||||
### 10.10 Ice rings at the scale and merge stages
|
||||
|
||||
|
||||
+26
-13
@@ -31,7 +31,7 @@ the status marks no test set. It is read once, before processing starts, and use
|
||||
already on the reference's axes), `REFERENCE_CC=` and `REFERENCE_MATCHED_FRACTION=` how well the
|
||||
result matches.
|
||||
- **It resolves the indexing ambiguity** (below) — the one thing the data cannot settle for
|
||||
themselves.
|
||||
themselves: which of the equivalent indexings is the reference's own.
|
||||
- **It hands over its R-free test set**, where the file carries one, so every dataset of a campaign
|
||||
is scored on the same free reflections - in the reference's frame, and only where the merge
|
||||
matches the reference (CC at least 0.5 over at least half of the merged reflections in its
|
||||
@@ -59,8 +59,8 @@ cell is metrically more symmetric than the Laue class by accident (*pseudo-meroh
|
||||
obliquity). The alternatives are related by the crystal's **twin laws** — reindexing operators such
|
||||
as `k,h,-l`.
|
||||
|
||||
Nothing in the data breaks the tie: the merge is equally self-consistent either way, so which
|
||||
solution comes out is arbitrary. What that costs depends on the workflow:
|
||||
Nothing in the data says which solution is the *true* one: the merge is equally self-consistent
|
||||
either way, so which comes out is arbitrary. What that costs depends on the workflow:
|
||||
|
||||
- **Rotation.** The whole sweep is one lattice, so the whole dataset lands in one indexing, picked
|
||||
at random. The merge itself is sound; it may simply be the *other* solution from an earlier
|
||||
@@ -68,20 +68,31 @@ solution comes out is arbitrary. What that costs depends on the workflow:
|
||||
same model.
|
||||
- **Serial stills.** Every crystal is indexed independently, so a run mixes both indexings into one
|
||||
merge. That is not a labelling matter — reflections that are not symmetry mates get averaged
|
||||
together, and CC1/2, R<sub>meas</sub> and the anomalous signal all degrade.
|
||||
together, and CC1/2, R<sub>meas</sub> and the anomalous signal all degrade. The data *can* say
|
||||
which crystals are in the same indexing, though, and that is what a merge needs: with the space
|
||||
group given (`-S`) and no reference, Rugnux puts each crystal into the indexing in which it agrees
|
||||
best with all the others, before anything is scaled. This takes seconds for tens of thousands of
|
||||
crystals, and the report says how clearly the crystals separated (`AMBIGUITY_SEPARATION_SIGMA=`,
|
||||
`AMBIGUITY_DECIDED_FRACTION=`) and how many were moved (`AMBIGUITY_REINDEXED=`). The indexing the
|
||||
whole merge ends up in is still arbitrary against other data sets, exactly as on rotation data.
|
||||
On a pseudo-merohedral cell whose axes differ by a few per cent, each crystal's own refined cell
|
||||
already says which axis is which; there only the crystals whose cell does not are decided by the
|
||||
intensities, and the others keep their indexing.
|
||||
|
||||
Every run that merges tests for the ambiguity, and where it exists and nothing resolves it, says so
|
||||
in the log and in the report's warnings:
|
||||
Every run that merges tests for the ambiguity, and where it exists and nothing resolves it (serial
|
||||
stills whose space group was determined by the run rather than given), says so in the log and in the
|
||||
report's warnings:
|
||||
|
||||
```
|
||||
Indexing ambiguity: this cell / space group admits alternative indexing (reindex operator(s):
|
||||
-h,-k,l). Serial-stills crystals are indexed in one hand at random, and rugnux can only break this
|
||||
against an external reference. WITHOUT one the merge mixes the hands and CC1/2 is degraded - supply
|
||||
a reference MTZ (-z) or a model (--model, which needs -C and -S here) to resolve it.
|
||||
-h,-k,l). Serial-stills crystals are indexed in one hand at random, and rugnux makes them consistent
|
||||
from the data only where the space group is given: here it was determined by the run, so the merge
|
||||
mixes the hands and CC1/2 is degraded. Giving the space group (-S) resolves it from the data; a
|
||||
reference MTZ (-z) or a model (--model, which needs -C and -S here) resolves it against them.
|
||||
```
|
||||
|
||||
Where something does resolve it, the run says so instead — `Indexing ambiguity present (reindex
|
||||
operator(s): -h,-k,l); resolved against the supplied model`.
|
||||
operator(s): -h,-k,l); resolved against the supplied reference` or `...; resolved from the data`.
|
||||
|
||||
How to resolve it:
|
||||
|
||||
@@ -90,14 +101,16 @@ How to resolve it:
|
||||
| **Rotation**, a reference dataset exists | `-z reference.mtz`. Once the space group is settled, each candidate reindexing of the merged intensities is correlated with the reference and the best-correlating one is re-merged. Only the *hkl* labels change; the cell does not |
|
||||
| **Serial stills**, a reference dataset exists | `-z reference.mtz`. Resolved **per image**, at integration time, by correlating each crystal's intensities with the reference — so the merge never mixes hands in the first place |
|
||||
| **Rotation**, only a model | `--model model.pdb`. The merged data are fitted to the model in each candidate indexing and the lowest R-free wins — provided the model first beats its random-orientation null (`MODEL_FIT= ACCEPTED`) and its lead over the runner-up beats the lead a random placement of the same model takes; the written reflections are then reindexed into it, so the file, the R-factors and the maps agree. Where either bar is missed the data keep the indexing they were merged in, and the report says by how much (`MODEL_INDEXING_MARGIN_SIGMA`). It is the **data** that are relabelled, never the model: every dataset of one crystal form processed against the same model comes out in that model's indexing, so the reflection files and maps of a screening campaign compare directly. The relabelling is a proper rotation of the lattice, so the merging statistics are unchanged and I(+)/I(-) keep their hand |
|
||||
| **Serial stills**, only a model | `--model model.pdb`, with the cell and space group given (`-C` / `-S`). Structure factors are computed from the model up front and used as the reference for the per-image test, exactly as a reference MTZ would be — the ambiguity has to be broken at integration time, and a model can supply the intensities to break it with |
|
||||
| Neither | The run warns and merges in whichever indexing it found: for rotation data a usable dataset in an arbitrary frame, for stills a degraded one |
|
||||
| **Serial stills**, only a model | `--model model.pdb`, with the cell and space group given (`-C` / `-S`). Structure factors are computed from the model up front and used as the reference for the per-image test, exactly as a reference MTZ would be. The data-only resolution below is then run only as a check: `AMBIGUITY_DATA_DISAGREES=` counts the crystals on which it would have chosen otherwise |
|
||||
| **Serial stills**, neither, space group given (`-S`) | Nothing to do: the crystals are made consistent with each other from the data alone (`INDEXING_AMBIGUITY_RESOLVED= data`), in an indexing that is arbitrary against other data sets. A model or reference still settles the weakest crystals more reliably than the other crystals can, and puts the merge in its own indexing |
|
||||
| Neither | The run warns and merges in whichever indexing it found: for rotation data a usable dataset in an arbitrary frame, for stills whose space group the run determined a degraded one |
|
||||
|
||||
Two things to know about where the choice lands:
|
||||
|
||||
- **`--mode scale` cannot repair a stills run after the fact.** The per-image test happens at
|
||||
integration time, so a `_process.h5` whose images were integrated without a reference — an MTZ or a
|
||||
model — has already lost the distinction, and no re-merge brings it back. On rotation data, where
|
||||
model — has already lost the distinction, and no re-merge brings it back; the data-only resolution
|
||||
runs in the full pipeline, not in `--mode scale`. On rotation data, where
|
||||
the ambiguity is one choice for the whole dataset, `--mode scale --model model.pdb` does resolve it.
|
||||
- **The choice reaches the written reflections**, not only the R-factors and the maps: the merged
|
||||
`.mtz` / `.cif` / `.hkl` (and `_unmerged.mtz`, where it is asked for) are written in the indexing
|
||||
|
||||
@@ -39,9 +39,9 @@ the rest walks through a run step by step.
|
||||
[Serial stills](#serial-stills-the-one-place-a-model-acts-before-the-merge)). Nothing else is
|
||||
taken from it before the merge. This is deliberate: a structure of the same protein (often a cryo
|
||||
structure) usually exists, and it settles the choice reliably. Where the lattice has no ambiguity
|
||||
the model changes nothing. Without a model the result is still a valid one, only with the
|
||||
crystals' indexings mixed - unless chance happens to sort them, which is very unlikely - so its
|
||||
merged intensities average each reflection with its alternative.
|
||||
the model changes nothing. Without a model the crystals are made consistent with each other from
|
||||
the data alone (with `-S` given), which recovers most of what the model gives; the model still
|
||||
decides the weakest crystals better than the other crystals can.
|
||||
|
||||
## Every option that brings in a model or reference data
|
||||
|
||||
@@ -103,7 +103,7 @@ these letters.
|
||||
| Point group, Laue class, centring | data (space-group search) | `-S` fixes it | the reference's | — | (a) |
|
||||
| Screw axes (space group within the point group) | data | `-S` | the reference's | a different choice → `<prefix>_mr.*` only | (a) in `p.mtz`, (c) beside it |
|
||||
| Hand (enantiomorph) | not decidable: `SPACE_GROUP_ENANTIOMORPH= UNDETERMINED` | `GIVEN` | `GIVEN` | relabelled to the model's where it fits, past the anomalous veto: `ASSUMED_FROM_MODEL` | (b) |
|
||||
| Indexing ambiguity (merohedral reindex) | arbitrary on rotation (`INDEXING_AMBIGUITY_OPERATORS=`); mixed on stills (warning) | — | resolved against the reference | resolved by the rigid-body model where its lead beats its null, rotation; per image on stills with `-C`/`-S`. Never by molecular replacement | (b) |
|
||||
| Indexing ambiguity (merohedral reindex) | arbitrary on rotation (`INDEXING_AMBIGUITY_OPERATORS=`); on stills with `-S` the crystals made consistent from the data, otherwise mixed (warning) | — | resolved against the reference | resolved by the rigid-body model where its lead beats its null, rotation; per image on stills with `-C`/`-S`. Never by molecular replacement | (b) |
|
||||
| Setting (which axis is a, b, c) | standard setting | `-C`'s axis order; `-S`'s symbol | the reference's | the model's, where it fits and there is no `-z` (`SETTING_SOURCE= MODEL`) | (b) |
|
||||
| Twinning check (L-test, twin laws) | data | — | — | — | (a) |
|
||||
| Twin hypothesis `<prefix>_twin_<SG>.*` | data (when the group may have absorbed a twin law) | — | — | the model may also *ask* for this merge; written then only where the model chooses it | (a) / (c) |
|
||||
@@ -189,6 +189,12 @@ carries no information to choose with, so the crystals stay mixed, as they would
|
||||
After the merge, the model is validated as in the steps above. Without `-C` and `-S`, or on rotation
|
||||
data, the model is not used before the merge at all.
|
||||
|
||||
Without a model or reference, a stills run given `-S` settles the same question from the data: it
|
||||
cannot say which indexing is the true one, but it can say which crystals share one, and puts each
|
||||
crystal into the indexing in which it agrees best with all the others. With a model, that resolution
|
||||
still runs, as a check only: `AMBIGUITY_DATA_DISAGREES=` in the report counts the crystals on which
|
||||
the data alone would have chosen differently from the model.
|
||||
|
||||
## Where a model can touch the space group
|
||||
|
||||
Spelled out, because this is the question that matters most:
|
||||
|
||||
+10
-1
@@ -238,7 +238,16 @@ and a run that produced a textbook data set read identically for their first thr
|
||||
correct cells show, an arbitrary indexing choice (`Indexing choice`, a rotation sweep whose cell
|
||||
admits alternative indexing: one sweep is indexed consistently, so it matters only against other data;
|
||||
the operators are **`INDEXING_AMBIGUITY_OPERATORS=`** in section 2). On serial stills the same ambiguity
|
||||
mixes the hands in the merge, and is an `INDEXING_AMBIGUITY` warning. `INDEXING_RATE=` - the share of
|
||||
mixes the hands in the merge. Where the space group was given (`-S`), the crystals are made
|
||||
consistent from the data alone and section 2 says so: **`INDEXING_AMBIGUITY_RESOLVED= data`**,
|
||||
**`AMBIGUITY_CRYSTALS=`** (the crystals that took part), **`AMBIGUITY_REINDEXED=`** (of them, the ones
|
||||
moved), **`AMBIGUITY_SEPARATION_SIGMA=`** (per crystal, how far its chosen indexing beats the best other
|
||||
one in correlation with all the other crystals, in sigma; the median) and
|
||||
**`AMBIGUITY_DECIDED_FRACTION=`** (the share of crystals decided at over 3 sigma). With a reference or
|
||||
a model the choice is theirs (`INDEXING_AMBIGUITY_RESOLVED= reference`), and
|
||||
**`AMBIGUITY_DATA_DISAGREES=`** counts the crystals on which the data alone would have chosen
|
||||
otherwise. Where the space group was determined by the run instead, the mixed merge is an
|
||||
`INDEXING_AMBIGUITY` warning. `INDEXING_RATE=` - the share of
|
||||
frames the per-image indexer took - is a stills measure: a rotation run indexes the sweep as a whole, so
|
||||
there it is written only with `--developer`, and `NO_LATTICE` rests on whether the sweep was indexed.
|
||||
- **`WARNING_COUNT=`** and the **`WARNING:`** lines follow, in the same section. They are what they
|
||||
|
||||
@@ -29,6 +29,8 @@ ADD_LIBRARY(JFJochScaleMerge
|
||||
SphericalHarmonicSurface.h
|
||||
ReindexAmbiguity.cpp
|
||||
ReindexAmbiguity.h
|
||||
ReferenceFreeReindex.cpp
|
||||
ReferenceFreeReindex.h
|
||||
CrystalSetting.cpp
|
||||
CrystalSetting.h
|
||||
TranslationalNCS.cpp
|
||||
|
||||
@@ -0,0 +1,392 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#include "ReferenceFreeReindex.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <cstdint>
|
||||
#include <numbers>
|
||||
#include <random>
|
||||
#include <unordered_map>
|
||||
|
||||
#include "HKLKey.h"
|
||||
#include "ReindexAmbiguity.h"
|
||||
#include "../../common/CrystalLattice.h"
|
||||
#include "../../common/ParallelFor.h"
|
||||
|
||||
namespace {
|
||||
constexpr size_t MIN_REFLECTIONS = 20;
|
||||
constexpr int RESOLUTION_BINS = 20;
|
||||
constexpr int RESTARTS = 4;
|
||||
constexpr int MAX_SWEEPS = 100;
|
||||
constexpr double DECIDED_SIGMA = 3.0;
|
||||
// Two fits closer than this (a 0.1% difference of one axis, squared) are the same fit.
|
||||
constexpr double METRIC_TIE = 1e-6;
|
||||
|
||||
double CellDistance(const UnitCell &u, const UnitCell &ref) {
|
||||
const auto rel = [](double v, double r) { return (v - r) / r; };
|
||||
const auto ang = [](double v, double r) { return (v - r) * std::numbers::pi / 180.0; };
|
||||
return rel(u.a, ref.a) * rel(u.a, ref.a) + rel(u.b, ref.b) * rel(u.b, ref.b)
|
||||
+ rel(u.c, ref.c) * rel(u.c, ref.c) + ang(u.alpha, ref.alpha) * ang(u.alpha, ref.alpha)
|
||||
+ ang(u.beta, ref.beta) * ang(u.beta, ref.beta) + ang(u.gamma, ref.gamma) * ang(u.gamma, ref.gamma);
|
||||
}
|
||||
|
||||
double Median(std::vector<double> v) {
|
||||
if (v.empty())
|
||||
return NAN;
|
||||
std::nth_element(v.begin(), v.begin() + v.size() / 2, v.end());
|
||||
return v[v.size() / 2];
|
||||
}
|
||||
}
|
||||
|
||||
ReferenceFreeReindexResult ResolveReindexReferenceFree(std::vector<IntegrationOutcome> &outcomes,
|
||||
const DiffractionExperiment &x,
|
||||
const UnitCell &cell,
|
||||
const std::vector<gemmi::Op> &all_ops,
|
||||
size_t nthreads, bool apply) {
|
||||
ReferenceFreeReindexResult out;
|
||||
out.choice.assign(outcomes.size(), -1);
|
||||
|
||||
if (all_ops.empty())
|
||||
return out;
|
||||
const auto &s = x.GetScalingSettings();
|
||||
const gemmi::SpaceGroup &sg = x.GetSpaceGroupOrP1();
|
||||
std::vector<gemmi::Op> indexings{gemmi::Op::identity()};
|
||||
indexings.insert(indexings.end(), all_ops.begin(), all_ops.end());
|
||||
const int K = static_cast<int>(indexings.size());
|
||||
|
||||
// How much better an image's own refined cell fits the run's cell as indexed than after indexing g.
|
||||
const auto cell_preference = [&](const IntegrationOutcome &o, int g) {
|
||||
return CellDistance(ReindexLattice(o.latt, indexings[g]).GetUnitCell(), cell)
|
||||
- CellDistance(o.latt.GetUnitCell(), cell);
|
||||
};
|
||||
// Which indexings the cells decide: those the indexed images' cells prefer the identity over (a tie
|
||||
// counting half) for at least METRIC_DECIDED of them.
|
||||
std::vector<bool> decided(K, false);
|
||||
for (int g = 1; g < K; ++g) {
|
||||
double prefer = 0, n = 0;
|
||||
for (const auto &o : outcomes) {
|
||||
if (o.reflections.empty())
|
||||
continue;
|
||||
const double p = cell_preference(o, g);
|
||||
prefer += std::fabs(p) < METRIC_TIE ? 0.5 : p > 0 ? 1.0 : 0.0;
|
||||
n += 1;
|
||||
}
|
||||
out.metric_preference.push_back(n > 0 ? prefer / n : 0.5);
|
||||
decided[g] = out.metric_preference.back() >= METRIC_DECIDED;
|
||||
}
|
||||
|
||||
const auto accept = [&](const Reflection &r) {
|
||||
return !r.on_ice_ring && AcceptReflection(r, s.GetHighResolutionLimit_A(), s.GetLowResolutionLimit_A())
|
||||
&& r.partiality >= s.GetMinPartiality() && r.partiality > 0.0f
|
||||
&& std::isfinite(r.I) && std::isfinite(r.sigma) && r.sigma > 0.0f;
|
||||
};
|
||||
const auto corrected = [](const Reflection &r) {
|
||||
return static_cast<double>(r.I) * (r.prescaling_corr * r.qe_corr * r.flight_corr) / r.partiality;
|
||||
};
|
||||
|
||||
// What each image has to offer: how many reflections, how fine, and how far its indices reach in
|
||||
// any of the indexings (which sizes the index table below).
|
||||
const int n_out = static_cast<int>(outcomes.size());
|
||||
std::vector<size_t> count(n_out, 0);
|
||||
std::vector<std::array<int, 3>> reach(n_out, {0, 0, 0});
|
||||
std::vector<double> s3_max(n_out, 0.0);
|
||||
ParallelFor(n_out, nthreads, [&](int o) {
|
||||
for (const auto &r : outcomes[o].reflections) {
|
||||
if (!accept(r))
|
||||
continue;
|
||||
++count[o];
|
||||
s3_max[o] = std::max(s3_max[o], 1.0 / (static_cast<double>(r.d) * r.d * r.d));
|
||||
for (const auto &op : indexings) {
|
||||
const auto h = op.apply_to_hkl({{r.h, r.k, r.l}});
|
||||
for (int i = 0; i < 3; ++i)
|
||||
reach[o][i] = std::max(reach[o][i], std::abs(h[i]));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// The crystals: images with enough reflections, in image order.
|
||||
std::vector<int> crystal_outcome;
|
||||
std::vector<size_t> begin{0};
|
||||
std::array<int, 3> bound{0, 0, 0};
|
||||
double s3_top = 0.0;
|
||||
for (int o = 0; o < n_out; ++o) {
|
||||
if (count[o] < MIN_REFLECTIONS)
|
||||
continue;
|
||||
crystal_outcome.push_back(o);
|
||||
begin.push_back(begin.back() + count[o]);
|
||||
for (int i = 0; i < 3; ++i)
|
||||
bound[i] = std::max(bound[i], reach[o][i]);
|
||||
s3_top = std::max(s3_top, s3_max[o]);
|
||||
}
|
||||
const int N = static_cast<int>(crystal_outcome.size());
|
||||
if (N < 2)
|
||||
return out;
|
||||
|
||||
// The indexings each crystal may take. Where the cells decide an indexing, a crystal whose own cell
|
||||
// prefers the identity stays as indexed - one of the anchors the others are decided against - and
|
||||
// only a crystal whose cell does not may move.
|
||||
std::vector<uint8_t> allowed(static_cast<size_t>(N) * K, 1);
|
||||
std::vector<bool> free(N, false);
|
||||
for (int c = 0; c < N; ++c)
|
||||
for (int g = 1; g < K; ++g) {
|
||||
if (decided[g] && cell_preference(outcomes[crystal_outcome[c]], g) >= METRIC_TIE)
|
||||
allowed[c * K + g] = 0;
|
||||
free[c] = free[c] || allowed[c * K + g];
|
||||
}
|
||||
out.crystals = std::count(free.begin(), free.end(), true);
|
||||
if (out.crystals == 0)
|
||||
return out;
|
||||
const size_t nnz = begin.back();
|
||||
|
||||
// The mean intensity in shells of equal reciprocal volume over the whole run, so that a crystal's
|
||||
// intensities are compared after the fall-off with resolution, which every indexing shares, is out.
|
||||
const auto bin_of = [&](const Reflection &r) {
|
||||
const double s3 = 1.0 / (static_cast<double>(r.d) * r.d * r.d);
|
||||
return std::min(RESOLUTION_BINS - 1, static_cast<int>(RESOLUTION_BINS * s3 / s3_top));
|
||||
};
|
||||
std::vector<std::array<double, 2 * RESOLUTION_BINS>> bin_part(N);
|
||||
ParallelFor(N, nthreads, [&](int c) {
|
||||
auto &b = bin_part[c];
|
||||
b.fill(0.0);
|
||||
for (const auto &r : outcomes[crystal_outcome[c]].reflections)
|
||||
if (accept(r)) {
|
||||
b[bin_of(r)] += corrected(r);
|
||||
b[RESOLUTION_BINS + bin_of(r)] += 1.0;
|
||||
}
|
||||
});
|
||||
std::array<double, RESOLUTION_BINS> bin_mean{};
|
||||
{
|
||||
std::array<double, 2 * RESOLUTION_BINS> total{};
|
||||
for (const auto &b : bin_part) // in crystal order: the same sum at any thread count
|
||||
for (int i = 0; i < 2 * RESOLUTION_BINS; ++i)
|
||||
total[i] += b[i];
|
||||
for (int i = 0; i < RESOLUTION_BINS; ++i)
|
||||
bin_mean[i] = total[RESOLUTION_BINS + i] > 0 ? total[i] / total[RESOLUTION_BINS + i] : 1.0;
|
||||
if (std::any_of(bin_mean.begin(), bin_mean.end(), [](double m) { return !(m > 0.0); }))
|
||||
for (auto &m : bin_mean)
|
||||
m = 1.0; // a shell with a non-positive mean: compare raw intensities instead
|
||||
}
|
||||
|
||||
// Every index any indexing can produce, mapped to a dense number of its reflection in the
|
||||
// asymmetric unit (Friedel mates together: the anomalous difference would only add noise here).
|
||||
const HKLKeyGenerator key_gen(/*merge_friedel=*/true, sg);
|
||||
const int dh = 2 * bound[0] + 1, dk = 2 * bound[1] + 1, dl = 2 * bound[2] + 1;
|
||||
const auto entry = [&](int h, int k, int l) {
|
||||
return (static_cast<size_t>(h + bound[0]) * dk + (k + bound[1])) * dl + (l + bound[2]);
|
||||
};
|
||||
std::vector<uint64_t> table_key(static_cast<size_t>(dh) * dk * dl);
|
||||
ParallelFor(dh, nthreads, [&](int ih) {
|
||||
for (int k = -bound[1]; k <= bound[1]; ++k)
|
||||
for (int l = -bound[2]; l <= bound[2]; ++l)
|
||||
table_key[entry(ih - bound[0], k, l)] = key_gen(ih - bound[0], k, l).pack();
|
||||
});
|
||||
std::vector<int32_t> table(table_key.size());
|
||||
int32_t n_keys = 0;
|
||||
{
|
||||
std::unordered_map<uint64_t, int32_t> dense;
|
||||
for (size_t i = 0; i < table_key.size(); ++i) {
|
||||
const auto [it, inserted] = dense.try_emplace(table_key[i], n_keys);
|
||||
n_keys += inserted;
|
||||
table[i] = it->second;
|
||||
}
|
||||
}
|
||||
table_key = {};
|
||||
|
||||
// Each crystal's intensities, divided by the shell mean and standardised within the crystal (the
|
||||
// unknown scale and offset of an image drop out), and where each lands in every indexing. Each is
|
||||
// weighted by how much of it is signal: a standardised intensity has unit variance, of which its
|
||||
// measurement error takes (sigma / sd)^2, so a weak or barely recorded reflection counts for little.
|
||||
std::vector<float> z(nnz);
|
||||
std::vector<std::vector<int32_t>> key(K, std::vector<int32_t>(nnz));
|
||||
ParallelFor(N, nthreads, [&](int c) {
|
||||
std::vector<double> err;
|
||||
size_t j = begin[c];
|
||||
double sum = 0, sum2 = 0;
|
||||
for (const auto &r : outcomes[crystal_outcome[c]].reflections) {
|
||||
if (!accept(r))
|
||||
continue;
|
||||
const double v = corrected(r) / bin_mean[bin_of(r)];
|
||||
z[j] = static_cast<float>(v);
|
||||
err.push_back(r.sigma * (r.prescaling_corr * r.qe_corr * r.flight_corr) / r.partiality
|
||||
/ bin_mean[bin_of(r)]);
|
||||
sum += v;
|
||||
sum2 += v * v;
|
||||
for (int g = 0; g < K; ++g) {
|
||||
const auto h = indexings[g].apply_to_hkl({{r.h, r.k, r.l}});
|
||||
key[g][j] = table[entry(h[0], h[1], h[2])];
|
||||
}
|
||||
++j;
|
||||
}
|
||||
const double n = static_cast<double>(begin[c + 1] - begin[c]);
|
||||
const double mean = sum / n;
|
||||
const double sd = std::sqrt(std::max(0.0, sum2 / n - mean * mean));
|
||||
for (size_t i = begin[c]; i < begin[c + 1]; ++i) {
|
||||
const double e = err[i - begin[c]] / sd;
|
||||
z[i] = sd > 0 ? static_cast<float>((z[i] - mean) / sd / (1.0 + e * e)) : 0.0f;
|
||||
}
|
||||
});
|
||||
table = {};
|
||||
|
||||
// Following Brehm & Diederichs (2014) Acta Cryst. D70, 101-109: each crystal in turn takes the
|
||||
// indexing in which it agrees best with all the others, until none changes. Agreement is the sum
|
||||
// over all pairs of crystals of the products of their standardised intensities on common
|
||||
// reflections, which is the dot product of a crystal with the sum of all the others - so a pass over
|
||||
// the crystals costs one pass over the reflections, not one per pair of crystals. The sum of all
|
||||
// pairs only ever grows, so it ends; a few starts are made and the one that ends highest is kept.
|
||||
std::vector<double> M(n_keys);
|
||||
const auto add = [&](int c, int g, double sign) {
|
||||
for (size_t i = begin[c]; i < begin[c + 1]; ++i)
|
||||
M[key[g][i]] += sign * z[i];
|
||||
};
|
||||
const auto dot = [&](int c, int g) {
|
||||
double d = 0;
|
||||
for (size_t i = begin[c]; i < begin[c + 1]; ++i)
|
||||
d += z[i] * M[key[g][i]];
|
||||
return d;
|
||||
};
|
||||
std::vector<int> best_choice;
|
||||
double best_objective = -1.0;
|
||||
for (int restart = 0; restart < RESTARTS; ++restart) {
|
||||
std::vector<int> choice(N, 0);
|
||||
std::mt19937 rng(restart);
|
||||
if (restart > 0)
|
||||
for (int c = 0; c < N; ++c) {
|
||||
const int g = static_cast<int>(rng() % K);
|
||||
choice[c] = allowed[c * K + g] ? g : 0;
|
||||
}
|
||||
std::fill(M.begin(), M.end(), 0.0);
|
||||
for (int c = 0; c < N; ++c)
|
||||
add(c, choice[c], 1.0);
|
||||
int sweeps = 0;
|
||||
// Run until a pass moves no more than one crystal in a thousand: what still moves then are crystals
|
||||
// too weak to be decided either way.
|
||||
for (int changed = N; changed > N / 1000 && sweeps < MAX_SWEEPS; ++sweeps) {
|
||||
changed = 0;
|
||||
for (int c = 0; c < N; ++c) {
|
||||
add(c, choice[c], -1.0);
|
||||
int best = choice[c];
|
||||
double best_dot = dot(c, best);
|
||||
for (int g = 0; g < K; ++g) {
|
||||
if (g == choice[c] || !allowed[c * K + g])
|
||||
continue;
|
||||
const double d = dot(c, g);
|
||||
if (d > best_dot) {
|
||||
best_dot = d;
|
||||
best = g;
|
||||
}
|
||||
}
|
||||
changed += best != choice[c];
|
||||
choice[c] = best;
|
||||
add(c, best, 1.0);
|
||||
}
|
||||
}
|
||||
double objective = 0;
|
||||
for (double m : M)
|
||||
objective += m * m;
|
||||
if (objective > best_objective) {
|
||||
best_objective = objective;
|
||||
best_choice = choice;
|
||||
out.sweeps = sweeps;
|
||||
}
|
||||
}
|
||||
|
||||
// How clearly each crystal is decided: its CC with the mean of all the other crystals, in the chosen
|
||||
// indexing and in the best other one, and the difference of the two in sigma (Fisher's z; each CC
|
||||
// on n reflections has an error of 1/sqrt(n - 3), and the two are nearly independent).
|
||||
std::fill(M.begin(), M.end(), 0.0);
|
||||
std::vector<double> C(n_keys, 0.0);
|
||||
for (int c = 0; c < N; ++c)
|
||||
for (size_t i = begin[c]; i < begin[c + 1]; ++i) {
|
||||
M[key[best_choice[c]][i]] += z[i];
|
||||
C[key[best_choice[c]][i]] += 1.0;
|
||||
}
|
||||
std::vector<double> sigma(N, NAN), cc_chosen(N, NAN), cc_other(N, NAN);
|
||||
ParallelFor(N, nthreads, [&](int c) {
|
||||
std::vector<double> cc(K, NAN);
|
||||
std::vector<double> n_used(K, 0.0);
|
||||
for (int g = 0; g < K; ++g) {
|
||||
double sx = 0, sy = 0, sxx = 0, syy = 0, sxy = 0, n = 0;
|
||||
for (size_t i = begin[c]; i < begin[c + 1]; ++i) {
|
||||
const int32_t k = key[g][i];
|
||||
const double own = g == best_choice[c] ? 1.0 : 0.0;
|
||||
const double cnt = C[k] - own;
|
||||
if (cnt <= 0)
|
||||
continue;
|
||||
const double xv = z[i], yv = (M[k] - own * z[i]) / cnt;
|
||||
sx += xv; sy += yv; sxx += xv * xv; syy += yv * yv; sxy += xv * yv; n += 1;
|
||||
}
|
||||
const double vx = sxx - sx * sx / n, vy = syy - sy * sy / n;
|
||||
if (n > 3 && vx > 0 && vy > 0) {
|
||||
cc[g] = std::clamp((sxy - sx * sy / n) / std::sqrt(vx * vy), -0.999, 0.999);
|
||||
n_used[g] = n;
|
||||
}
|
||||
}
|
||||
const int g0 = best_choice[c];
|
||||
int other = -1;
|
||||
for (int g = 0; g < K; ++g)
|
||||
if (g != g0 && allowed[c * K + g] && std::isfinite(cc[g]) && (other < 0 || cc[g] > cc[other]))
|
||||
other = g;
|
||||
if (!std::isfinite(cc[g0]) || other < 0)
|
||||
return;
|
||||
cc_chosen[c] = cc[g0];
|
||||
cc_other[c] = cc[other];
|
||||
sigma[c] = (std::atanh(cc[g0]) - std::atanh(cc[other]))
|
||||
* std::sqrt((std::min(n_used[g0], n_used[other]) - 3.0) / 2.0);
|
||||
});
|
||||
std::vector<double> sig, ccc, cco;
|
||||
for (int c = 0; c < N; ++c)
|
||||
if (free[c] && std::isfinite(sigma[c])) {
|
||||
sig.push_back(sigma[c]);
|
||||
ccc.push_back(cc_chosen[c]);
|
||||
cco.push_back(cc_other[c]);
|
||||
}
|
||||
out.median_sigma = Median(sig);
|
||||
out.cc_chosen = Median(ccc);
|
||||
out.cc_other = Median(cco);
|
||||
if (!sig.empty())
|
||||
out.decided_fraction = static_cast<double>(std::count_if(sig.begin(), sig.end(), [](double v) {
|
||||
return v > DECIDED_SIGMA; })) / static_cast<double>(sig.size());
|
||||
|
||||
// Any consistent answer relabelled as a whole by one of the indexings is as consistent: take the
|
||||
// one that leaves the most crystals where they are - unless the cells decide an indexing, in which
|
||||
// case the crystals they anchor fix the whole. Two indexings are the same for the merge when they
|
||||
// send a general reflection to the same reflection of the asymmetric unit.
|
||||
const std::array<int, 3> general{2, 5, 11};
|
||||
const auto same = [&](const gemmi::Op::Miller &a, const gemmi::Op::Miller &b) {
|
||||
return key_gen(a[0], a[1], a[2]) == key_gen(b[0], b[1], b[2]);
|
||||
};
|
||||
const auto composed = [&](int global, int g) { // indexing g, then the overall relabel
|
||||
return indexings[global].apply_to_hkl(indexings[g].apply_to_hkl(general));
|
||||
};
|
||||
int global = 0;
|
||||
size_t global_kept = 0;
|
||||
const bool anchored = std::find(decided.begin(), decided.end(), true) != decided.end();
|
||||
for (int h = 0; h < (anchored ? 1 : K); ++h) {
|
||||
size_t kept = 0;
|
||||
for (int c = 0; c < N; ++c)
|
||||
kept += same(composed(h, best_choice[c]), general);
|
||||
if (kept > global_kept) {
|
||||
global_kept = kept;
|
||||
global = h;
|
||||
}
|
||||
}
|
||||
for (int c = 0; c < N; ++c) {
|
||||
int final_g = best_choice[c];
|
||||
for (int g = 0; g < K; ++g)
|
||||
if (same(composed(global, best_choice[c]), indexings[g].apply_to_hkl(general))) {
|
||||
final_g = g;
|
||||
break;
|
||||
}
|
||||
out.choice[crystal_outcome[c]] = final_g;
|
||||
if (final_g == 0)
|
||||
continue;
|
||||
++out.reindexed;
|
||||
if (apply) {
|
||||
auto &o = outcomes[crystal_outcome[c]];
|
||||
ReindexImage(o.reflections, o.latt, indexings[final_g]);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
||||
// SPDX-License-Identifier: GPL-3.0-only
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <cmath>
|
||||
#include <vector>
|
||||
|
||||
#include "gemmi/symmetry.hpp"
|
||||
|
||||
#include "../../common/DiffractionExperiment.h"
|
||||
#include "../IntegrationOutcome.h"
|
||||
|
||||
// Serial stills with no reference: every crystal is indexed in one of the indexings its lattice allows
|
||||
// (the identity and the twin-law operators of ReindexAmbiguityOperators), at random. Which one each
|
||||
// crystal is in is decided from the data alone, by putting each crystal into the indexing in which it
|
||||
// agrees best with all the others. The overall choice among the equivalent consistent answers is the
|
||||
// one that moves the fewest crystals.
|
||||
//
|
||||
// An operator is only an ambiguity where the crystal's own lattice cannot tell the indexings apart. Where
|
||||
// the cell is merely close to a more symmetric one (two axes a few per cent apart, say), the crystals'
|
||||
// refined cells already say which axis is which, and the indexer has used that. For such an operator a
|
||||
// crystal whose cell prefers the indexing it came in stays there, as an anchor, and only the others are
|
||||
// decided by the intensities.
|
||||
// An operator whose reindexed cell is the worse fit to the run's cell for at least this share of the
|
||||
// crystals is one the cells decide. For a true (merohedral) ambiguity the two fits are equal and the
|
||||
// share is a half.
|
||||
constexpr double METRIC_DECIDED = 0.9;
|
||||
|
||||
struct ReferenceFreeReindexResult {
|
||||
size_t crystals = 0; // crystals that took part (enough reflections to say anything, and an
|
||||
// indexing their own cell leaves open)
|
||||
size_t reindexed = 0; // of them, put into an indexing other than the one they came in
|
||||
double median_sigma = NAN; // per crystal, how far its chosen indexing beats the best other one
|
||||
// in agreement with the rest of the merge, in sigma; the median
|
||||
double decided_fraction = NAN; // share of the crystals for which that margin is over 3 sigma
|
||||
double cc_chosen = NAN; // median CC of a crystal with the merge of all the others, as chosen
|
||||
double cc_other = NAN; // ... and in the best other indexing
|
||||
std::vector<double> metric_preference; // per operator: share of crystals whose refined cell is
|
||||
// closer to `cell` as indexed than after the operator
|
||||
int sweeps = 0; // passes over the crystals until (almost) none changed (the start kept)
|
||||
std::vector<int> choice; // per outcome: -1 did not take part, 0 kept, k reindexed by ops[k - 1]
|
||||
};
|
||||
|
||||
// `cell` is the run's cell, `ops` its twin-law operators (identity excluded). With apply = false nothing is changed - the
|
||||
// result then says how many crystals the data alone would put into another indexing than they are in,
|
||||
// which, after a reference has resolved them, is the disagreement with the reference.
|
||||
ReferenceFreeReindexResult ResolveReindexReferenceFree(std::vector<IntegrationOutcome> &outcomes,
|
||||
const DiffractionExperiment &x,
|
||||
const UnitCell &cell,
|
||||
const std::vector<gemmi::Op> &ops,
|
||||
size_t nthreads, bool apply);
|
||||
@@ -276,8 +276,12 @@ void ReindexAmbiguityResolver::Resolve(std::span<Reflection> reflections, Crysta
|
||||
if (best.is_identity)
|
||||
return;
|
||||
|
||||
ReindexImage(reflections, latt, best.op);
|
||||
}
|
||||
|
||||
void ReindexImage(std::span<Reflection> reflections, CrystalLattice &latt, const gemmi::Op &op) {
|
||||
for (auto &r : reflections) {
|
||||
const gemmi::Op::Miller h = best.op.apply_to_hkl({{r.h, r.k, r.l}});
|
||||
const gemmi::Op::Miller h = op.apply_to_hkl({{r.h, r.k, r.l}});
|
||||
r.h = h[0]; r.k = h[1]; r.l = h[2];
|
||||
}
|
||||
// Everything after this - the sublattice relabel, the partiality post-refinement - predicts a spot
|
||||
@@ -285,5 +289,5 @@ void ReindexAmbiguityResolver::Resolve(std::span<Reflection> reflections, Crysta
|
||||
// axes, a frame relabelled by an operator that is only a near-symmetry of the lattice was put
|
||||
// through the sublattice relabel a second time, and its reflections came out at indices finer
|
||||
// than anything measured.
|
||||
latt = ReindexLattice(latt, best.op);
|
||||
latt = ReindexLattice(latt, op);
|
||||
}
|
||||
@@ -21,7 +21,8 @@
|
||||
// Resolving the indexing ("alternative indexing" / reindexing) ambiguity: when the lattice symmetry
|
||||
// is higher than the crystal Laue symmetry (merohedral / pseudo-merohedral), a crystal can be validly
|
||||
// indexed in several ways related by reindexing operators. The right one is picked by agreement with a
|
||||
// reference (reference-MTZ intensities or a model), so no reference-free ambigator (cosym) is needed.
|
||||
// reference (reference-MTZ intensities or a model); serial stills without one are made consistent from
|
||||
// the data alone (ReferenceFreeReindex.h).
|
||||
//
|
||||
// This is distinct from the screw-axis / enantiomorph ambiguity (P4_1 vs P4_3): that leaves the merged
|
||||
// intensities unchanged, so it is never resolved here — it is adopted from the reference/model.
|
||||
@@ -39,6 +40,10 @@ std::vector<gemmi::Op> ReindexAmbiguityOperators(const UnitCell &cell, const gem
|
||||
// R^-1 times the old ones, so h' (a*, b*, c*)' is the same reciprocal vector as h (a*, b*, c*).
|
||||
CrystalLattice ReindexLattice(const CrystalLattice &latt, const gemmi::Op &op);
|
||||
|
||||
// One image put into another indexing: its reflections relabelled by `op` and its lattice moved onto the
|
||||
// same axes, so that a spot's new index still predicts it where it was measured.
|
||||
void ReindexImage(std::span<Reflection> reflections, CrystalLattice &latt, const gemmi::Op &op);
|
||||
|
||||
std::vector<MergedReflection> ReindexReflections(const std::vector<MergedReflection> &merged,
|
||||
const gemmi::Op &op);
|
||||
|
||||
|
||||
@@ -44,6 +44,8 @@ namespace {
|
||||
// 16: FILE_DIRECT_BEAM / BACKGROUND_DIRECT_BEAM / BEAM_CENTRE_SHIFT / BEAM_CENTRE_TOLERANCE in
|
||||
// section 1 and the BEAM_CENTRE_OFF flag - the file's beam centre against the refined one;
|
||||
// the XDS GEOMETRY KEYWORDS section; OVERLOADED_* / OVERLAPPED_* and the OVERLOADS / OVERLAPS flags.
|
||||
// 17: INDEXING_AMBIGUITY_RESOLVED and the AMBIGUITY_* keys in section 2 - serial stills, the
|
||||
// indexing ambiguity resolved from the data alone (or measured against the reference's answer).
|
||||
// 8: every line that is not `KEY= value` and not blank starts with `#`, warnings included
|
||||
// (`# WARNING:`); the header gains RUGNUX_DOWNLOAD and BUILD_CXX_FLAGS, and RUGNUX_GIT is
|
||||
// the full build-time hash.
|
||||
@@ -752,6 +754,20 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
|
||||
Add(s, KeyBool("LATTICE_FOUND", result.consensus_cell.has_value()));
|
||||
if (!result.indexing_ambiguity_operators.empty())
|
||||
Add(s, KeyText("INDEXING_AMBIGUITY_OPERATORS", result.indexing_ambiguity_operators));
|
||||
// Serial stills: the indexing ambiguity from the data alone - deciding the merge where no reference
|
||||
// or model was given, otherwise measured against the reference's answer (REINDEXED is then the
|
||||
// number of crystals on which the two disagree).
|
||||
if (result.ambiguity_reference_free) {
|
||||
Add(s, KeyText("INDEXING_AMBIGUITY_RESOLVED",
|
||||
result.ambiguity_reference_free_applied ? "data" : "reference"));
|
||||
Add(s, KeyInt("AMBIGUITY_CRYSTALS", static_cast<int64_t>(result.ambiguity_crystals)));
|
||||
Add(s, KeyInt(result.ambiguity_reference_free_applied ? "AMBIGUITY_REINDEXED"
|
||||
: "AMBIGUITY_DATA_DISAGREES",
|
||||
static_cast<int64_t>(result.ambiguity_reindexed)));
|
||||
Add(s, KeyReal("AMBIGUITY_SEPARATION_SIGMA", result.ambiguity_separation_sigma, "{:.2f}"));
|
||||
Add(s, KeyReal("AMBIGUITY_DECIDED_FRACTION", result.ambiguity_decided_fraction, "{:.3f}"));
|
||||
Add(s, KeyReal("AMBIGUITY_TIME_S", result.ambiguity_time_s, "{:.3f}", /*dev=*/true));
|
||||
}
|
||||
if (result.consensus_cell.has_value())
|
||||
Add(s, KeyText("UNIT_CELL_CONSTANTS", CellString(*result.consensus_cell)));
|
||||
// Kabsch's sigma_M, the Gaussian e.s.d. of the rocking width in degrees - XDS's
|
||||
|
||||
@@ -657,6 +657,18 @@ struct ProcessResult {
|
||||
// it matters only against other data. Empty where there is none, or where it was resolved.
|
||||
std::string indexing_ambiguity_operators;
|
||||
|
||||
// Serial stills: the indexing ambiguity resolved from the data alone (ReferenceFreeReindex.h), where
|
||||
// the cell and space group admit one. Without a reference or model it decides the merge; with one, it
|
||||
// is only measured against the reference's answer (ambiguity_reference_free_applied = false, and
|
||||
// ambiguity_reindexed is then the number of crystals on which the two disagree).
|
||||
bool ambiguity_reference_free = false;
|
||||
bool ambiguity_reference_free_applied = false;
|
||||
size_t ambiguity_crystals = 0;
|
||||
size_t ambiguity_reindexed = 0;
|
||||
double ambiguity_separation_sigma = NAN;
|
||||
double ambiguity_decided_fraction = NAN;
|
||||
double ambiguity_time_s = NAN;
|
||||
|
||||
// The same analysis measured BEFORE the space group was decided, on the subgroup merge the search
|
||||
// was given. It is the only twinning number a promotion cannot contaminate: once a higher Laue
|
||||
// class is adopted, its own twinning test can say no more than "no twin law exists inside this
|
||||
|
||||
@@ -76,6 +76,7 @@
|
||||
#include "../image_analysis/scale_merge/ScaleOnTheFly.h"
|
||||
#include "../image_analysis/scale_merge/StillsPartialityRefine.h"
|
||||
#include "../image_analysis/scale_merge/SublatticeRelabel.h"
|
||||
#include "../image_analysis/scale_merge/ReferenceFreeReindex.h"
|
||||
#include "../image_analysis/WriteReflections.h"
|
||||
#include "../image_analysis/bragg_integration/CalcISigma.h"
|
||||
#include "../common/Definitions.h"
|
||||
@@ -1519,6 +1520,47 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
|
||||
relabel.frames, relabel.reflections_kept, relabel.reflections_dropped);
|
||||
}
|
||||
const auto initial_sg = experiment_.GetGemmiSpaceGroup();
|
||||
// Stills in a group whose lattice admits alternative indexings: each crystal was indexed in one of
|
||||
// them at random. With a reference or a model each image has already been put into the reference's
|
||||
// (IndexAndRefine::ScaleImage); without one the indexings are made consistent here, from the data
|
||||
// alone, before anything is scaled. With one, the same is done only to measure it against the
|
||||
// reference's answer.
|
||||
if (!rsm && initial_sg && result.consensus_cell) {
|
||||
const auto ops = ReindexAmbiguityOperators(*result.consensus_cell, *initial_sg);
|
||||
if (!ops.empty()) {
|
||||
const auto t0 = std::chrono::steady_clock::now();
|
||||
const auto rf = ResolveReindexReferenceFree(indexer->GetIntegrationOutcome(), experiment_,
|
||||
*result.consensus_cell, ops, config_.nthreads,
|
||||
/*apply=*/!ambiguity_reference);
|
||||
result.ambiguity_time_s = std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count();
|
||||
for (size_t i = 0; i < ops.size(); ++i)
|
||||
logger.Info("Indexing ambiguity {}: {:.1f}% of the crystals' own cells fit better as indexed{}",
|
||||
IndexTriplet(ops[i]), 100.0 * rf.metric_preference[i],
|
||||
rf.metric_preference[i] >= METRIC_DECIDED
|
||||
? " - the cells decide it; only crystals whose cell does not are decided by "
|
||||
"the intensities" : "");
|
||||
result.ambiguity_reference_free = true;
|
||||
result.ambiguity_reference_free_applied = !ambiguity_reference;
|
||||
result.ambiguity_crystals = rf.crystals;
|
||||
result.ambiguity_reindexed = rf.reindexed;
|
||||
result.ambiguity_separation_sigma = rf.median_sigma;
|
||||
result.ambiguity_decided_fraction = rf.decided_fraction;
|
||||
if (!ambiguity_reference)
|
||||
logger.Info("Indexing ambiguity resolved from the data: {} of {} crystals reindexed; median "
|
||||
"separation {:.1f} sigma, {:.0f}% of crystals decided at over 3 sigma (CC with the "
|
||||
"rest {:.3f} as chosen, {:.3f} otherwise); {} sweeps, {:.2f} s",
|
||||
rf.reindexed, rf.crystals, rf.median_sigma, 100.0 * rf.decided_fraction,
|
||||
rf.cc_chosen, rf.cc_other, rf.sweeps, result.ambiguity_time_s);
|
||||
else
|
||||
logger.Info("Indexing ambiguity from the data alone, against the reference's: it would put {} "
|
||||
"of {} crystals into another indexing; median separation {:.1f} sigma, {:.0f}% of "
|
||||
"crystals decided at over 3 sigma; {:.2f} s", rf.reindexed, rf.crystals,
|
||||
rf.median_sigma, 100.0 * rf.decided_fraction, result.ambiguity_time_s);
|
||||
Note(fmt::format("indexing ambiguity from the data: {} of {} crystals {}", rf.reindexed, rf.crystals,
|
||||
ambiguity_reference ? "disagree with the reference" : "reindexed"));
|
||||
ambiguity_reference = true;
|
||||
}
|
||||
}
|
||||
auto sm = scale_and_merge(initial_sg ? initial_sg->short_name() : "P1", !initial_sg.has_value(),
|
||||
/*measure_cc=*/true);
|
||||
if (guard_beside) {
|
||||
@@ -3969,13 +4011,17 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
|
||||
const std::string msg = fmt::format(
|
||||
"Indexing ambiguity: this cell / space group admits alternative indexing "
|
||||
"(reindex operator(s): {}). Serial-stills crystals are indexed in one hand at random, "
|
||||
"and rugnux can only break this against an external reference. WITHOUT one the merge "
|
||||
"mixes the hands and CC1/2 is degraded; a reference MTZ (-z) or a model (--model, "
|
||||
"which needs -C and -S here) resolves it.", laws);
|
||||
"and rugnux makes them consistent from the data only where the space group is given: "
|
||||
"here it was determined by the run, so the merge mixes the hands and CC1/2 is degraded. "
|
||||
"Giving the space group (-S) resolves it from the data; a reference MTZ (-z) or a model "
|
||||
"(--model, which needs -C and -S here) resolves it against them.", laws);
|
||||
logger.Warning("{}", msg);
|
||||
ambiguity_warning_text = " !! " + msg + "\n\n";
|
||||
stats_text << ambiguity_warning_text;
|
||||
result.warnings.push_back({PathologyCode::INDEXING_AMBIGUITY, msg});
|
||||
} else if (result.ambiguity_reference_free_applied) {
|
||||
logger.Info("Indexing ambiguity present (reindex operator(s): {}); resolved "
|
||||
"from the data.", laws);
|
||||
} else {
|
||||
logger.Info("Indexing ambiguity present (reindex operator(s): {}); resolved "
|
||||
"against the supplied reference.", laws);
|
||||
|
||||
@@ -3,7 +3,10 @@
|
||||
|
||||
#include <catch2/catch_all.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <cmath>
|
||||
#include <random>
|
||||
#include <unordered_map>
|
||||
#include <unordered_set>
|
||||
#include <vector>
|
||||
@@ -11,6 +14,7 @@
|
||||
#include "../image_analysis/scale_merge/CrystalSetting.h"
|
||||
#include "../image_analysis/scale_merge/HKLKey.h"
|
||||
#include "../image_analysis/scale_merge/ReindexAmbiguity.h"
|
||||
#include "../image_analysis/scale_merge/ReferenceFreeReindex.h"
|
||||
#include "../image_analysis/scale_merge/SublatticeRelabel.h"
|
||||
|
||||
namespace {
|
||||
@@ -306,3 +310,158 @@ TEST_CASE("Sublattice: a frame on the given cell in another setting is described
|
||||
// An unrelated cell is not put on it.
|
||||
CHECK_FALSE(CellOnReference(UnitCell{55.0f, 66.0f, 77.0f, 90.0f, 90.0f, 90.0f}, given, settings).has_value());
|
||||
}
|
||||
|
||||
TEST_CASE("Reindex: a stills set indexed in random hands is made consistent from the data alone", "[reindex]") {
|
||||
// P4 on a tetragonal lattice: two indexings, related by a twin law. Every crystal records a random
|
||||
// part of the same synthetic structure with its own scale and noise, and half of them, at random,
|
||||
// carry the other indexing.
|
||||
const UnitCell cell = Tetragonal();
|
||||
DiffractionExperiment x;
|
||||
x.SetUnitCell(cell).SpaceGroupNumber(75);
|
||||
const auto laws = ReindexAmbiguityOperators(cell, x.GetSpaceGroupOrP1());
|
||||
REQUIRE(laws.size() == 1);
|
||||
|
||||
const HKLKeyGenerator key(true, x.GetSpaceGroupOrP1());
|
||||
std::mt19937 rng(7);
|
||||
std::exponential_distribution<double> wilson(1.0);
|
||||
std::normal_distribution<double> noise(0.0, 0.3);
|
||||
std::unordered_map<uint64_t, double> structure;
|
||||
std::vector<std::array<int, 3>> hkls;
|
||||
for (int h = -12; h <= 12; ++h)
|
||||
for (int k = -12; k <= 12; ++k)
|
||||
for (int l = -6; l <= 6; ++l) {
|
||||
if (h == 0 && k == 0 && l == 0)
|
||||
continue;
|
||||
hkls.push_back({h, k, l});
|
||||
structure.try_emplace(key(h, k, l).pack(), wilson(rng));
|
||||
}
|
||||
|
||||
const int n_crystals = 300;
|
||||
std::vector<IntegrationOutcome> outcomes(n_crystals);
|
||||
std::vector<bool> flipped(n_crystals);
|
||||
for (int c = 0; c < n_crystals; ++c) {
|
||||
flipped[c] = rng() % 2;
|
||||
const double scale = 0.5 + (rng() % 1000) / 1000.0;
|
||||
outcomes[c].latt = CrystalLattice(cell);
|
||||
for (int i = 0; i < 150; ++i) {
|
||||
const auto &t = hkls[rng() % hkls.size()];
|
||||
const auto m = flipped[c] ? laws[0].apply_to_hkl({{t[0], t[1], t[2]}}) : gemmi::Op::Miller{{t[0], t[1], t[2]}};
|
||||
Reflection r{};
|
||||
r.h = m[0]; r.k = m[1]; r.l = m[2];
|
||||
r.d = 1.0f / std::sqrt((t[0] * t[0] + t[1] * t[1]) / (cell.a * cell.a) + t[2] * t[2] / (cell.c * cell.c));
|
||||
r.I = static_cast<float>(scale * structure.at(key(t[0], t[1], t[2]).pack()) * (1.0 + noise(rng)));
|
||||
r.sigma = 1.0f;
|
||||
r.partiality = 1.0f;
|
||||
r.prescaling_corr = 1.0f;
|
||||
outcomes[c].reflections.push_back(r);
|
||||
}
|
||||
}
|
||||
|
||||
const auto result = ResolveReindexReferenceFree(outcomes, x, cell, laws, 4, /*apply=*/true);
|
||||
CHECK(result.crystals == n_crystals);
|
||||
// The overall hand is free: either exactly the flipped crystals are reindexed, or exactly the others.
|
||||
int agree = 0;
|
||||
for (int c = 0; c < n_crystals; ++c)
|
||||
agree += (result.choice[c] != 0) == flipped[c];
|
||||
CHECK((agree == n_crystals || agree == 0));
|
||||
const size_t n_flipped = std::count(flipped.begin(), flipped.end(), true);
|
||||
CHECK(result.reindexed == std::min(n_flipped, n_crystals - n_flipped));
|
||||
CHECK(result.reindexed <= n_crystals / 2); // the overall hand that moves the fewest crystals
|
||||
CHECK(result.decided_fraction > 0.9);
|
||||
|
||||
// Done again on the consistent set, nothing moves.
|
||||
const auto again = ResolveReindexReferenceFree(outcomes, x, cell, laws, 4, /*apply=*/false);
|
||||
CHECK(again.reindexed == 0);
|
||||
}
|
||||
|
||||
TEST_CASE("Reindex: a holohedral crystal gives the stills resolution nothing to do", "[reindex]") {
|
||||
DiffractionExperiment x;
|
||||
x.SetUnitCell(Tetragonal()).SpaceGroupNumber(89); // P422
|
||||
std::vector<IntegrationOutcome> outcomes(3);
|
||||
const auto laws = ReindexAmbiguityOperators(Tetragonal(), x.GetSpaceGroupOrP1());
|
||||
const auto result = ResolveReindexReferenceFree(outcomes, x, Tetragonal(), laws, 1, true);
|
||||
CHECK(laws.empty());
|
||||
CHECK(result.reindexed == 0);
|
||||
CHECK(result.crystals == 0);
|
||||
}
|
||||
|
||||
TEST_CASE("Reindex: an operator the crystals' own cells decide is left to them", "[reindex]") {
|
||||
// Orthorhombic with a and b 3% apart: a twin law of the metric to within the obliquity tolerance,
|
||||
// but every crystal's refined cell says which axis is which - nothing for the data to decide.
|
||||
const UnitCell cell{78.0f, 80.4f, 37.0f, 90.0f, 90.0f, 90.0f};
|
||||
DiffractionExperiment x;
|
||||
x.SetUnitCell(cell).SpaceGroupNumber(16); // P222
|
||||
const auto laws = ReindexAmbiguityOperators(cell, x.GetSpaceGroupOrP1());
|
||||
REQUIRE_FALSE(laws.empty());
|
||||
|
||||
std::vector<IntegrationOutcome> outcomes(50);
|
||||
std::mt19937 rng(3);
|
||||
for (auto &o : outcomes) {
|
||||
o.latt = CrystalLattice(cell);
|
||||
for (int i = 0; i < 100; ++i) {
|
||||
Reflection r{};
|
||||
r.h = static_cast<int>(rng() % 15) - 7; r.k = static_cast<int>(rng() % 15) - 7; r.l = static_cast<int>(rng() % 7) + 1;
|
||||
r.d = 3.0f; r.I = static_cast<float>(rng() % 1000); r.sigma = 1.0f; r.partiality = 1.0f; r.prescaling_corr = 1.0f;
|
||||
o.reflections.push_back(r);
|
||||
}
|
||||
}
|
||||
const auto result = ResolveReindexReferenceFree(outcomes, x, cell, laws, 1, true);
|
||||
REQUIRE(result.metric_preference.size() == laws.size());
|
||||
for (double p : result.metric_preference)
|
||||
CHECK(p == 1.0);
|
||||
CHECK(result.crystals == 0);
|
||||
CHECK(result.reindexed == 0);
|
||||
}
|
||||
|
||||
TEST_CASE("Reindex: where the cells decide, the crystals whose cell disagrees are decided by the intensities",
|
||||
"[reindex]") {
|
||||
// As above, a and b 3% apart, but a few crystals were indexed on swapped axes: their own cells say so,
|
||||
// and the intensities, read against the crystals the cells anchor, put them back.
|
||||
const UnitCell cell{78.0f, 80.4f, 37.0f, 90.0f, 90.0f, 90.0f};
|
||||
DiffractionExperiment x;
|
||||
x.SetUnitCell(cell).SpaceGroupNumber(16); // P222
|
||||
const auto laws = ReindexAmbiguityOperators(cell, x.GetSpaceGroupOrP1());
|
||||
REQUIRE(laws.size() == 1);
|
||||
|
||||
const HKLKeyGenerator key(true, x.GetSpaceGroupOrP1());
|
||||
std::mt19937 rng(11);
|
||||
std::exponential_distribution<double> wilson(1.0);
|
||||
std::normal_distribution<double> noise(0.0, 0.3);
|
||||
std::unordered_map<uint64_t, double> structure;
|
||||
std::vector<std::array<int, 3>> hkls;
|
||||
for (int h = -12; h <= 12; ++h)
|
||||
for (int k = -12; k <= 12; ++k)
|
||||
for (int l = -6; l <= 6; ++l)
|
||||
if (h != 0 || k != 0 || l != 0) {
|
||||
hkls.push_back({h, k, l});
|
||||
structure.try_emplace(key(h, k, l).pack(), wilson(rng));
|
||||
}
|
||||
|
||||
const int n_crystals = 300;
|
||||
std::vector<IntegrationOutcome> outcomes(n_crystals);
|
||||
std::vector<bool> swapped(n_crystals, false);
|
||||
for (int c = 0; c < n_crystals; ++c) {
|
||||
swapped[c] = c % 12 == 5; // 25 of 300: the cells still decide the law
|
||||
outcomes[c].latt = swapped[c] ? ReindexLattice(CrystalLattice(cell), laws[0]) : CrystalLattice(cell);
|
||||
for (int i = 0; i < 150; ++i) {
|
||||
const auto &t = hkls[rng() % hkls.size()];
|
||||
const auto m = swapped[c] ? laws[0].apply_to_hkl({{t[0], t[1], t[2]}}) : gemmi::Op::Miller{{t[0], t[1], t[2]}};
|
||||
Reflection r{};
|
||||
r.h = m[0]; r.k = m[1]; r.l = m[2];
|
||||
r.d = 1.0f / std::sqrt(t[0] * t[0] / (cell.a * cell.a) + t[1] * t[1] / (cell.b * cell.b)
|
||||
+ t[2] * t[2] / (cell.c * cell.c));
|
||||
r.I = static_cast<float>(structure.at(key(t[0], t[1], t[2]).pack()) * (1.0 + noise(rng)));
|
||||
r.sigma = 0.1f;
|
||||
r.partiality = 1.0f;
|
||||
r.prescaling_corr = 1.0f;
|
||||
outcomes[c].reflections.push_back(r);
|
||||
}
|
||||
}
|
||||
|
||||
const auto result = ResolveReindexReferenceFree(outcomes, x, cell, laws, 4, /*apply=*/true);
|
||||
REQUIRE(result.metric_preference.size() == 1);
|
||||
CHECK(result.metric_preference[0] >= METRIC_DECIDED);
|
||||
CHECK(result.crystals == 25);
|
||||
for (int c = 0; c < n_crystals; ++c)
|
||||
CHECK((result.choice[c] != 0) == swapped[c]);
|
||||
}
|
||||
Reference in new issue
Block a user