# Data analysis: space group and validation (§13–§14) Part of the [CPU/GPU data-analysis reference](CPU_DATA_ANALYSIS.md); the section numbers are continuous across its four parts. ```{contents} On this page :local: :depth: 2 ``` ## 13. Space-group determination and merge-level decisions ### 13.1 Space-group determination When no space group is supplied, a POINTLESS-like search scores Laue-group symmetry (CC of $E^2(h)$ vs $E^2(Rh)$ — the intensities normalised by the mean of their own resolution shell — plus merge self-consistency) and detects screw/centering absences from the $P1$-merged intensities. Three tests weigh a promotion to higher symmetry, all aimed at the merohedral twin, whose twin law forces non-equivalent reflections together and so mimics symmetry: 1. **Merge self-consistency** ($\chi^2$ under the candidate group, relative to the confirmed subgroup). On its own this is not sufficient: it is a ratio to an error model that moves with the *amount* of data — the parent's systematic term grows as $\sigma$ shrinks with $1/\sqrt{N}$, while a twin's is already saturated — so its verdict depends on how much data the search saw. 2. **Error-model $b$** (the intensity-proportional systematic). A genuine symmetry step gains multiplicity without inflating $b$; merging a twin law's extra operator inflates it. This is a **rescue and never a veto**: a $\chi^2$-borderline promotion whose $b$ stays close to the confirmed subgroup's is confirmed on it, which is what recovers a genuine high-symmetry group on imperfectly-scaled data, but it cannot demote a promotion that passed. The veto it once carried was removed. It was guarded by "$H$ could not be computed", so it could only ever act where the one statistic that discriminates here is blind, and what it compared there was not a measurement but the error-model bisection's own ceiling — which made a space group depend on the compiler flags, the same source refusing a correct $P2_12_12_1$ in one build and not in the other. No bound repairs it: genuine promotions have since been measured up to 5.8 where the bound was calibrated at 1.62 against a twin at 2.1, so the two populations have swapped sides. The case it was meant to catch — a merohedral twin too sparse for $H$ — is answered below it by the added-operator $R$ gate, whose reference is the global best rather than the parent. 3. **Operator disagreement**, a sigma-free statistic $H=\mathrm{median}\,|I_1-I_2|/(I_1+I_2)$, formed as the ratio of the operators a promotion *adds* to the parent's own, measured on the same reflections. Normalising against the parent divides out the systematic floor that symmetry mates carry on real data, which varies by crystal and by operator; a median is used because a twin perturbs every pair whereas a badly-measured minority perturbs only the tail. Where a candidate has several parents of the same order, it is judged against the worst of them, since a rival subgroup can itself contain the twin laws. The correlation is on **resolution-normalised** intensity $E^2 = I/\langle I\rangle(\text{shell})$, normalised over exactly the reflections the correlation pairs. Both members of a symmetry pair lie at the same $|s|$, so on raw $I$ the resolution fall-off is variance shared perfectly between the two arms and appears as a positive correlation for *any* pairing at all: a shell-matched random pairing — the exact null for a metrically-allowed false operator — scores a median 0.31 across the rotation battery, and on one crystal 0.53 — above the bound the correlation is tested against. That floor varies more from crystal to crystal (spread 0.46) than the whole true/false gap is wide (0.38), so an absolute bound on the raw statistic is a different test on every crystal; and it moves with the search resolution cut, which is what made that cut a symmetry-deciding parameter. Normalised, the floor has a median of 0.015, never exceeds 0.06, and barely moves with the cut. Both the correlation stage and the absence tests need to know whether a reflection is **genuinely present**, and that question is asked of its *counting* significance, not of the merged $I/\sigma$. A merged $\sigma$ carries the error model's intensity-proportional term, $\sigma^2 = a\,\sigma_0^2 + (b\,I)^2$, so merged $I/\sigma$ saturates — at $\mathrm{ISa}\sqrt{n}$ for a reflection observed $n$ times, and at $\mathrm{ISa}$ exactly for one observed once. Above that knee it stops rising with the intensity: on the weakest search merge of the rotation battery the $I/\sigma$ of every decile of $E^2$ reads 1.58–1.65 against an $\mathrm{ISa}$ of 1.70, so reflections an order of magnitude apart in real intensity report the same number. A single constant applied there demands anywhere between 2.2 and 11.9 in counting significance depending on the crystal. The nominal cut $T$ (default 3.0) is therefore converted once, using the ISa of the merge being searched. For a reflection observed once $\sigma_\text{counting}^2 = \sigma^2 - (b\,I)^2$, so $I/\sigma_\text{counting} \ge T$ is exactly $$\frac{I}{\sigma} \;\ge\; \frac{T}{\sqrt{1 + (T/\mathrm{ISa})^2}}$$ The converted cut lies strictly below $\mathrm{ISa}$ for every $\mathrm{ISa}$, so it is always reachable by the reflection the ceiling binds hardest, and it is within 1 % of $T$ on any merge with $\mathrm{ISa} \ge 21$ — a healthy merge is left exactly where it was. Multiplicity is taken as 1 deliberately rather than estimated, for the same reason. Where the merge reports no ISa (or $b = 0$) the cut is used as it stands. The candidates are enumerated in **every setting the refined cell can host**, not only in the settings the International Tables call the reference one. A setting is a statement about direction — $P112_1$ puts its $2_1$ on $c$ where $P12_11$ puts it on $b$, and both are space group 4 — so a search restricted to reference settings can only ever put a screw or a centring on the axis the convention chose, whatever the data say. Two things bound the widened set. A candidate is offered only if the cell's own metric admits the rotations its setting names ($\alpha=\beta=90^\circ$ for a $2$-fold on $c$), and a candidate predicting exactly the absences another candidate already predicts is dropped as the same hypothesis under a second name. A non-reference setting is additionally refused when its centring class holds no reflection in this merge, since there is then nothing to confirm it with. Because the candidates of a point group all share its rotations, this cannot raise the symmetry: it decides *which axes* carry the screws and the centring, never how many operators there are. The setting found this way is the one the *decisions* are made in. The files are then written in the ITA standard setting (as XDS and POINTLESS write it: $P2_12_12$ with the pure axis on $c$ and $a_report.txt`): it gates nothing and changes no reflection, no scale and no group — but the axial-absence test (§13.1) and the $L$-test here both correct themselves against the modulation it measures. **The $L$-test partners are chosen so a pseudo-translation cannot silence it.** The test compares each reflection with a partner a fixed index step away, and a pseudo-translation biases $\langle|L|\rangle$ upwards unless the partner shares its modulation class. The ordinary step of 2 preserves the class of a half-integer translation — the pseudo-centering the statistic's authors call it robust to — but not of one of a third, which inflates $\langle|L|\rangle$ past the bound that is read as evidence *against* twinning and silently loses the twin call on a crystal that has one. Where a pseudo-translation is detected, the partner steps are restricted to those that preserve its class; where no step does, the statistic is dropped from the twin verdict **in both directions** — it can no longer indicate a twin and can no longer be read as proof that there is none — and the second moment decides alone. `L_TEST_VS_TNCS=` in the report says which of the three happened. ### 13.3 Outlier rejection Merging applies an optional per-observation median-based $N\sigma$ cut (`--reject-outliers`, default 6σ for `rot3d`, off otherwise). The same $N\sigma$ cut is fed back into the error model: after an initial $a,b$ fit the parameters are re-fit once on the reflections that survive rejection (dropping any whose squared deviation exceeds $N^2\,[a\,\sigma^2 + (b\,\langle I\rangle)^2]$), so the calibrated errors describe the reflections that actually enter the merge rather than the pre-rejection pool. The median needs three observations, so a reflection measured once or twice — typically one good observation and one artefact (a hot pixel, a zinger) after Friedel merging — is never tested by it, and an artefact that looks precise (thousands of counts, a small relative $\sigma$) can outweigh mates from weak frames and become the median itself. Every observation of the written merge is therefore also judged by **Wilson statistics**. Beyond 4 Å its $E^2 = I/(\varepsilon\,\langle I/\varepsilon\rangle_\mathrm{shell})$ is compared with a bound set by a budget of $\alpha = 0.01$ expected false rejections per dataset: with $N$ observations tested, $\ln(2N/\alpha)$ for acentrics and twice that for centrics, and the observation's lower confidence limit $I - z\sigma$ (with its own $\sigma$, and $z$ from the same budget) must exceed it, so noise in weak shells does not trigger it; a shell whose $\langle I\rangle$ is not itself established at that significance is not judged. The bound is widened by the tail scale of the dataset's own intensity distribution, measured on its well-measured observations (1 for a Wilson crystal; larger under a pseudo-translation or anisotropy). An improbable singleton is dropped; an improbable observation with company only when most of the reflection's other observations are probable and it disagrees with their mean beyond the errors — several large observations confirm each other. An observation whose signal disk lost pixels to the mask or to saturation is never counted as such a witness, since its profile estimate may be low: a strong reflection is not replaced by its clipped mate. The count is `OBSERVATIONS_REJECTED_WILSON=` in the report (included in `OBSERVATIONS_REJECTED=`), and the developer report lists each observation with its image and detector position. ### 13.4 Automatic resolution cutoff By default the reported/written high-resolution limit is trimmed where $\mathrm{CC}_{1/2}$ falls off: a logistic is fitted to $\mathrm{CC}_{1/2}(s)$, and the limit is set **one reported-shell width past** the point where the fit crosses 0.30 — deliberately "one shell too far", so weak-but-real data below the crossing are kept rather than discarded. The extension is measured over the range that is actually kept, not the full measured range, so a detector reaching far past where the crystal diffracts cannot inflate it. `--scaling-high-resolution` overrides the limit and `--resolution-cutoff off` disables it. **Ice sits out of this fit, and only this fit.** A powder ring is reproducible: past the crystal's own limit the ice is still there and still the same in both half-sets, so the two halves agree *about the ring*, and a Pearson $\mathrm{CC}_{1/2}$ cannot tell that agreement from diffraction. Shells have been measured at $\mathrm{CC}_{1/2}=0.765$ where $\langle I/\sigma\rangle$ is $-0.1$ and $R_\mathrm{meas}$ is 470 %, inside the written range because the logistic was fitted through them; against archived references, runs whose frames carry ice were written a mean twelve per cent finer than the reference where runs without ice were written four per cent finer, and every over-claim past twenty per cent has rings. The same ring drags the curve the other way where the data are good — one shell of forty falling to 0.51 because the two strongest ice lines cross it, with the shells either side at 0.99 — which is a real defect of those reflections but narrower than the shell it defames. Both signs are the same cause and both leave the fit: the reflections flagged in §3.3 are still merged, still written and still counted in the shell table (§10.10), and what changes is only that the resolution **decision** now reads the same curve that scaling, the error model and the space-group search already read. **A cut the shell table refutes is not quoted.** Two things keep the number inside what the bins show. The fitted crossing has to lie **inside the fitted bins**: on a real fall-off it sits between the last bin at the target and the first bin below it, with the extension bins to spare, and a crossing past the last fitted bin is an extrapolation of a fall-off these data never showed — the crossing is then read off the bins themselves, between the two that straddle the target. And no single resolution is quoted at all where the curve is not a fall-off: where $\mathrm{CC}_{1/2}$ never reaches the target in any shell, where the fit reads finer than the finest shell whose $\mathrm{CC}_{1/2}$ still reaches it, or where $\mathrm{CC}_{1/2}$ **climbs back over the target** after falling below it — a noise shell that climbs back is not the edge of the data. The report then says which of the three happened and points at the shell table instead. ### 13.5 Diffraction anisotropy How fast the intensity falls off with resolution can depend on direction. Rugnux measures that, reports it, and does nothing else with it: no intensity is corrected, no reflection is removed on a directional criterion, and the merged data and the written files do not depend on direction at all. **The tensor.** A deviatoric anisotropic displacement tensor is fitted to the merged intensities as $$\ln \langle I(\mathbf{s})\rangle = c(\text{shell}) - \tfrac{1}{2}\,\mathbf{s}^\mathsf{T} B\, \mathbf{s},\qquad \mathbf{s} = \text{reciprocal-space vector},\ |\mathbf{s}| = 1/d$$ (§10.6 and §14.2 write $s$ for $\sin\theta/\lambda = 1/2d$, so their $s^2 = 1/4d^2$; the two conventions give the same exponent $-(B/2)(1/d^2)$, and $B$ is the same $B$.) with one free constant per resolution shell, so every isotropic feature — the Wilson curve, an ice ring, a noise floor, a scaling error — is absorbed exactly and only the $\ell = 2$ angular part drives the tensor. For an isotropic $B$ this reduces to the ordinary Wilson plot, so $B$ here is the ordinary crystallographic ($B = 8\pi^2 U$) $B$, directly comparable with phenix.xtriage's `B_cart`, ctruncate's anisotropic $B$ eigenvalues and AIMLESS's anisotropic $\Delta B$. Only the deviatoric part is fitted: the isotropic part is degenerate with the overall scale. The tensor is constrained to the directions the Laue class allows — five free deviatoric parameters in triclinic, three in monoclinic, two in orthorhombic, one in tetragonal, trigonal and hexagonal, and **none at all in cubic**, where symmetry forces $\Delta B$ to be exactly zero. The fit is on **intensities, with no positivity cut**. Fitting amplitudes, or dropping non-positive intensities as an amplitude-based tool must, loses roughly 40% of the signal: in a direction that has died half the merged intensities are negative, so a positivity cut keeps only the positive noise excursions and flattens the fall-off exactly where the anisotropy is largest. **Two different quantities are reported, and they are not interchangeable.** $\Delta B$ (the range of the principal components) is a *rate*; the diffraction limit along each principal direction — where $\langle I/\sigma(I)\rangle$ in a 20° cone about that direction falls through 2, read by interpolation in $s^2$ over equal-count shells — is where the signal actually runs out. A crystal can have a large $\Delta B$ and almost no spread in directional limit, or the reverse. Where $\langle I/\sigma(I)\rangle$ never falls through 2 in a direction, the limit returned is the **edge of the measured data** rather than the crystal's own; such a direction is marked — with a `<` in the report, a 1 in `ANISOTROPY_D_MIN_CENSORED`, and a note in the mmCIF — so the spread is not read as a measurement when it is a lower bound. Both constants are AIMLESS's defaults (cone half-angle 20°, $\langle I/\sigma\rangle = 2$). The 2 is a per-direction diagnostic level only: the dataset-wide resolution cut (§13.4) is CC$_{1/2}$-based, and the two criteria are not interchangeable. **The resolution signature.** A genuine Debye–Waller $B$ makes the directional deficit a straight line through the origin in $s^2$. The per-shell $\ell = 2$ amplitude is therefore fitted against $s^2$ and the curve is classified: *linear* (a real $B$), *flat* (a deficit that does not follow $\exp(-\tfrac12 \mathbf{s}^\mathsf{T} B \mathbf{s})$ at all, so the fitted $\Delta B$ describes the data with the wrong functional form and may be an **under**-estimate), or *convex* (a deficit that grows faster than $s^2$, which a $B$ cannot do). The verdict is re-derived at 8 and at 16 shells, and reported as undetermined if it moves. **The verdict, and what it is measured against.** Whether an anisotropy is real is not decided against a counting-statistics error bar. Real data carry systematic error far larger than counting error, and gating on the latter reports anisotropy on datasets that have none. Instead the data set measures its own systematic error: in the tensor directions the Laue class *forbids*, the true tensor is exactly zero whatever the crystal is, so whatever is measured there is systematic. That measurement needs the unmerged observations — a merge has exact Laue symmetry by construction, and the forbidden directions are identically zero in it — so it is made on the scaled, rocking-curve-assembled observations. The counting part is subtracted, the counting error of the directions actually being tested is added back, and the result is the **floor**. What is tested against it is $\Delta B_\text{linear}$ — the part of the fall-off that actually follows $\exp(-\tfrac12\mathbf{s}^\mathsf{T}B\mathbf{s})$, clamped at zero — and **not** the headline $\Delta B$; the report names which of the two it is quoting. The ratio is banded: below 2 not established, 2–3.5 marginal, above 3.5 established, above 5 strong. The bands are calibrated against known ground truth — merging cubic crystals in proper subgroups of their own Laue class, where the true anisotropy is exactly zero — which puts the false-positive rate at 24 % at 2.0, 10 % at 3.5 and 5 % at 5.0. The report says **NOT DETECTED**, **DETECTED**, or **CANNOT DETERMINE**, and the third is a real answer rather than an evasion. It is returned when the Laue class is triclinic (no forbidden direction exists, so there is no internal measurement of the systematic error and no substitute for it), when the observed rotation range is under about 90° (a lab-fixed systematic then reaches several tensor directions instead of one), when the merged data are at the noise floor, when the scale model carried no dose term (an uncorrected dose ramp manufactures anisotropy that no significance test can see through), or when no unmerged observations were available. The smallest $\Delta B$ that could have been established on the data set is reported with the verdict; it is set by the systematic error rather than by counting, so it does **not** improve with more reflections or a longer exposure. A too-high space-group assignment is the one failure mode that is silent: real anisotropy is then pushed into the directions used to measure the systematic error, which inflates the floor and biases the answer towards reporting none. A caution says so, but only where that is actually indicated — a single free direction, no detection, **and** a forbidden-direction measurement far above its own counting noise — rather than on every tetragonal, trigonal and hexagonal data set. Everything lands in `_report.txt` section 4 (`ANISOTROPY_*` keys), in the printed statistics, and in the merged mmCIF: the eigen-decomposition of the tensor as the standard `_reflns.pdbx_aniso_B_tensor_*` items (relative to the weakest direction, since only the deviatoric part is determined), and the directional limits, the shape and the verdict under the `_reflns.jfjoch_aniso_*` local prefix. ### 13.6 Practical notes and limitations - **Bragg integration is profile-fitted by default** (per-shell Gaussian profile, Kabsch extraction; §9.3), with plain box summation available as a fallback (`--integrator boxsum`). The profiles are built per frame from that frame's strong spots, which suits fast-feedback and serial/streaming use; a profile shared across many frames (as in full offline workflows) is not currently formed. - **Space-group symmetry** beyond centering absences is not enforced during prediction/integration unless the space group is supplied and used downstream. - **Resolution masking** is controllable, and so is every stage of ice-ring handling (§3.3, §10.10). None of it runs unless the crystal is measured to have ice, because the fixed bands are a fixed cost in unique reflections whether it does or not. - **Rotation vs still modes** differ substantially in prediction and scaling: partiality is angle-driven in rotation data, while stills are predicted within an excitation-error window and get their partiality from the default-on per-crystal tilt post-refinement (§10.2) — or unit partiality with `--simple-stills`. - **Amplitudes and intensities.** The merged output carries both intensities (mmCIF `intensity_meas`, MTZ `IMEAN`/`SIGIMEAN`) and French–Wilson amplitudes (mmCIF `F_meas_au`, MTZ `F`/`SIGF`; §10.8), so a downstream program can refine against either. --- ## 14. Model-based validation: R-free against a model and electron-density maps Offline (`rugnux --model model.pdb`) the merged data can be scored against a supplied atomic model and electron-density maps computed — enough to confirm that a model fits the data and to inspect the density, not a substitute for refinement. **The structure itself is not refined**; the model is re-fractionalized into the data unit cell and then **placed as one rigid body** (§14.8), and the observed amplitudes are the French–Wilson $|F|$ from §10.8, so the R-free and the maps use exactly the same amplitudes as the written reflection file. The model, structure-factor, bulk-solvent and FFT machinery is provided by GEMMI. ### 14.1 Model structure factors The model electron density is sampled on a grid (IT92 X-ray form factors, with a Refmac-compatible Gaussian blur chosen for the grid spacing) and Fourier-transformed to structure factors $F_\mathrm{calc}(hkl)$ up to the data resolution. ### 14.2 Bulk solvent and scaling A flat bulk-solvent mask around the model is transformed to $F_\mathrm{mask}$, and the model is scaled to the observed amplitudes by an overall least-squares fit of a scale $k$, an anisotropic $B$, and the flat-solvent parameters $k_\mathrm{sol}, B_\mathrm{sol}$: $ F_\mathrm{model} = k\,e^{-\mathbf{h}^\top \mathbf{B}\,\mathbf{h}/4}\left(F_\mathrm{calc} + k_\mathrm{sol}\,e^{-B_\mathrm{sol}\,s^2}\,F_\mathrm{mask}\right),\quad s^2 = 1/4d^2. $ This is the standard, few-parameter scaling model used by refinement programs. No free-form per-resolution-shell rescale is applied: such a rescale is dataset-specific and reshapes each map's radial amplitude profile differently, which would make maps from a multi-dataset campaign no longer directly comparable. That decision has a cost, and it is paid by the R-factors rather than by the maps. $k\exp(-\mathbf{s}^\mathsf{T} B \mathbf{s})$ can only bend one way with resolution, so whatever a dataset's radial amplitude profile does that this shape cannot follow is reported as R — which makes R a reading of *this* dataset and not a quantity comparable with another reduction of the same crystal. Both uses are wanted, so both are served, separately: the maps and `R_WORK` / `R_FREE` keep the scale above, and a second reading, `R_MODEL_SHELL_SCALED`, applies one free scale per resolution shell to the *same* fit and is reported beside them, with `MODEL_RADIAL_MISFIT` saying how much that rescale had to do. Nothing is written from the second reading; it never reaches a map, a map coefficient or a decision (§14.3, and RUGNUX_REPORT). The fit sees the **working reflections only**; the parameters it returns are then applied to every reflection, free ones included, so that R-free (§14.3) is computed against them. The parameters are few, but they are fitted by minimising the very sum R is made of, and a free reflection that helped choose them is no longer held out. ### 14.3 R-work and R-free Crystallographic R-factors are reported over the work and free sets (the §10.7 flags): $ R = \frac{\sum \big|\,|F_o| - |F_\mathrm{model}|\,\big|}{\sum |F_o|}, $ with R-free the same sum restricted to the free set. Every parameter $F_\mathrm{model}$ carries — the scaling of §14.2 and the rigid-body placement of §14.8 alike — is fitted on the working set alone, so the free reflections are held out of the fit as well as out of the sum. Two choices are still *selected* with the free set rather than fitted to it, and both are single discrete decisions rather than continuous parameters: the rigid-body step is committed only if it lowers R-free (§14.8), and the alternative indexing of §14.6 is the candidate with the lowest R-free. Both are the conventional use of a test set to accept or reject a step; both leave R-free very slightly optimistic where they fire. ### 14.4 Electron-density maps Two maps are formed with the model phases $\varphi_\mathrm{model}$, both $\sigma_A$ weighted: a $2mF_o-DF_c$ map and an $mF_o-DF_c$ difference map, each inverse-Fourier-transformed to a real-space CCP4 map (`_2fofc.ccp4`, `_fofc.ccp4`). A map-coefficient MTZ (`_maps.mtz`: `FP`, `FC`, `PHIC`, `FWT`/`PHWT`, `DELFWT`/`PHDELWT`, `FOM`, `FREE`) is written alongside so the maps can be reopened or rebuilt in Coot / PyMOL. $\sigma_A$ is estimated by maximum likelihood **per resolution shell**, on the **free reflections only** — on the working set the model has been fitted to the data, so $\sigma_A$ would come out too high and the weighting would understate exactly the model error the map is meant to reveal. The shells are cut by equal reflection count, and it is the *number of shells* that is chosen from the size of the free set (about 50 free reflections to a shell, at most 20 shells), so no shell is thin by construction; a shell that still ends up with fewer than 10 free reflections takes the estimate made over the whole free set instead. Acentric and centric reflections enter with their own likelihoods (Rice and Woolfson respectively), and the epsilon factor is divided out before normalising both amplitudes to $\langle|E|^2\rangle = 1$ within the shell. The figure of merit is then $m = I_1(X)/I_0(X)$ with $X = 2\sigma_A|E_o||E_c|/(1-\sigma_A^2)$ for an acentric reflection and $m = \tanh(X/2)$ for a centric one, and $D = \sigma_A\sqrt{\Sigma_o/\Sigma_c}$ carries $F_c$ onto the observed amplitudes' scale. The $2F_o-F_c$ combination becomes $2mF_o - DF_c$ for acentric reflections and $mF_o$ for centric ones — a centric reflection's phase is either exactly right or 180° wrong, never in between — while the difference coefficient is $mF_o - DF_c$ throughout. $m$ and $D$ are estimated on each dataset separately, so two datasets of one crystal form get slightly different weights and their maps are to that extent no longer scaled identically — the same property §14.2 deliberately protects by refusing a free-form per-shell rescale. The two are not the same thing: the weighting never rescales $F_o$ and never touches the R-factors of §14.3, and the difference between two datasets' $\sigma_A$ curves is the difference in how well the model explains each of them, which is what a screening campaign is looking for. A PanDDA-style analysis consumes $2mF_o-DF_c$ maps and normalises each dataset's map against the ensemble before comparing them. The per-reflection `FOM` is written to the MTZ so the weighting can be read off and undone. ### 14.5 Does the model fit? The null it is scored against A model supplied with `--model` is a **hypothesis about the crystal**, not an instruction. It is always scaled, always placed (§14.8) and always scored, and its R-factors and maps are always reported — but before it is allowed to change anything about the reflections that are written, the data have to accept it. There are only two such things (§14.6), and where the model claims neither the question is never put: see the end of this section. **No fixed threshold on R can decide that**, and this was measured three ways. The classical acentric random-structure value is $R = 0.586$ at unit scale, but the scale here is fitted to minimise the very sum the R is made of, which pulls it to about 0.550; observed nulls on real data land at 0.599–0.615; and the value moves with the *model* — its atom count and its B-factors — as much as with the data, so a cut calibrated on one pair misjudges the next. In one measured arm an unrelated protein reached a *lower* raw R-free than the correct model did on other data. The only null that fits both the model and the data is therefore **made out of them**: the same model is re-oriented at random about its own centroid and run through the identical path — the same scaling, the same rigid-body placement, the same R — nine times, and the real fit is asked how far above the resulting distribution it sits (`MODEL_FIT_SIGMA`, accepted at 3σ). The replicates are placed as well as fitted, or the comparison would be between a placed model and unplaced nulls and the margin would be inflated by the placement rather than by the model. Measured on one rotation data set: the crystal's own model +15.0σ, an unrelated protein +1.8σ, and the correct model rigidly rotated 90° −1.0σ. A replicate is a sample of the null only while it stays away from the model's own solution. A draw within the rigid body's reach of an orientation equivalent to the model's — under the space group's rotations or a twin law of the lattice — is drawn again before it is placed, and a replicate the placement nevertheless carries to within that reach is drawn and placed again afterwards, each replicate from a generator of its own so the result does not depend on the order the concurrent replicates finish in. **R-work carries the decision, not R-free.** Nothing is refined against the working set here — the scale has a handful of parameters (a scale, an anisotropic $B$ and two solvent terms) and the placement six — so R-work carries no optimism, and it has an order of magnitude more reflections than R-free, and so that much more power to separate the two arms. R-free is still reported, and is still what the placement of §14.8 is committed on, since that *is* a fit. What the verdict gates is exactly the two decisions of §14.6 that **rewrite the data**: the space-group label and the indexing. It does not gate the R-factors, the maps or the rigid-body placement, which are statements about the model and cannot corrupt a reflection. A rejected model therefore leaves the written files byte for byte what a run with no model would have produced, and the report says so beside the R it was rejected on: a negative result, not a failed run. **The null is only built where a decision is actually pending.** A model can change exactly two things, and it does not always claim either: one already in the space group the data were merged in, on a crystal with no merohedral ambiguity, asserts no enantiomorph and prefers the data's own indexing, so there is nothing to arbitrate and nothing for a null to gate. That case reports `MODEL_FIT= NOT_TESTED` and skips the replicates. It is not an edge case — it is the isomorphous fragment-screening run, the one that has to deliver a map seconds after the last image — and it is why the ordering matters: the indexing probe is a handful of scaling fits and runs first anyway, so whether the expensive part is worth paying for is known before it starts. The count is nine, and it is the *spread* that sets it rather than the mean: the verdict is (mean − real)/sd of the sample, the relative error on an sd from $n$ draws is $1/\sqrt{2(n-1)}$, and a sample that happens to come out narrow is what turns a model that does not fit into one that appears to. Measured on the case nearest the gate — an unrelated protein at 1.83σ against a threshold of 3 — the chance of reading over the gate on a different seed is 31 % at $n=3$, 17 % at $n=5$ and 6 % at $n=9$. Nine is affordable only because the replicates run at once: the null costs the slowest of them rather than their sum, and the slowest of nine is barely above the slowest of five. The null costs one full fit and one rigid-body placement per replicate. The replicates share nothing — each is the same model under a different rotation, scored the same way — so each takes a copy of the model and of its structure factors and they run **concurrently**, on the run's own thread budget (`-N`); the real model is not touched by any of them. Measured on an idle machine, five replicates cost 2.5 s together where running them one after another costs 10 s, on a `--mode scale` run that merges in 2 s. What remains is the cost of the *slowest* replicate: how many placement evaluations an orientation needs varies by nearly half between them, so the concurrency saturates around 4×, and more threads than replicates buy nothing. The orientations are drawn up front, in order, from the fixed seed, so replicate $i$ gets the same orientation whatever order the threads run in — a σ that depended on the scheduling would not be a measurement. Where the null is skipped, `--model` costs what it did before the null existed (measured: indistinguishable, ~4 s against ~4 s, on a loaded machine where a single run scatters by a second). Where it runs, that is the price of the answer being a measurement rather than a threshold. ### 14.6 Aligning the data to the model: enantiomorph and indexing ambiguity The model fixes a definite hand and indexing, but the merged data need not share them, so before comparison the observed reflections are brought into the model's frame. - **Enantiomorph / screw.** When the data space group is the enantiomorph of the model's (e.g. data $P4_12_12$, model $P4_32_12$; or $P3_1/P3_2$), the two are **indistinguishable from merged intensities** — $|F_\mathrm{calc}|$ is invariant under the change of hand, so R-free cannot choose between them and probing would be meaningless. Where the model was accepted (§14.5), its group is therefore adopted as the **label** the reflections are written under, and the reflections themselves are left untouched. Where it was not, the label is left alone: adopting it is arithmetic on two group numbers, which a model that does not belong to this crystal can do exactly as readily as one that does, and the anomalous map of §14.7 vetoes it outright where it says the two are in opposite hands. The two groups of an enantiomorphic pair differ only in the translations of their operations: their rotations are identical, so they transform $hkl$ identically, share a reciprocal ASU, and assign the Bijvoet hands identically. The label carries no handedness, and there is nothing about it to undo. Reindexing by the change-of-hand operator — which is the inversion — would instead **swap $I(+)$ with $I(-)$**, flipping every anomalous difference on the strength of a label the space-group search itself reports as undetermined; where the model is genuinely the wrong enantiomorph for the crystal, it would manufacture agreement rather than reveal the mismatch. What does carry the hand is the indexing the data already have, from the diffraction geometry, and the anomalous differences that come with it. §14.7 is what tests them against the model. - **Indexing (merohedral) ambiguity.** When the crystal has a merohedral ambiguity (§10.9), the observed intensities *do* differ between indexings, and the right one is chosen against the best available reference. **If a reference MTZ was supplied, the data were already reindexed to agree with it** (§10.9 — by the reference-intensity correlation, at the merge stage for rotation data or per image in stills scaling), and model validation keeps that authoritative choice. **Only with a model and no reference** does validation resolve the ambiguity itself, as a fallback: the scaled model is fit to each reindexing of the data (identity plus the twin-law cosets) and the one giving the **lowest R-free** is kept — but only where its lead over the runner-up beats the lead the *same model in a random orientation* takes, since a random model also picks a winner, and, as measured, by a comparable margin. Where it does not, the data keep the indexing they were merged in. The candidate operators are enumerated from the **data's** space group, not the model's: it is the observed intensities that are being relabelled, and asking the model's group enumerates nothing at all wherever the two differ. This matters for a multi-dataset campaign — a single shared reference fixes one indexing convention for every dataset, whereas an independent per-dataset lowest-R-free choice could send borderline datasets to different conventions. A no-op either way for a holohedral crystal (no twin laws). The two decisions then reach the written output differently, because only one of them moves reflections. The **ambiguity choice is applied to the merged reflections themselves** — and to the integrated observations behind the unmerged export — which are written after this step, so the reflection file, the R-factors and the maps describe one indexing. The **change of hand changes only the space group the files are written under** (the model's enantiomorph): no reflection moves, exactly as the first bullet says, so $I(+)$ and $I(-)$ stay as measured and §14.7's hand check remains a genuine test rather than an agreement manufactured by reindexing. The ambiguity choice is reported with that margin and with the null it was judged against, since the margin alone is what cannot say whether the data decided or the two came out within noise of each other. ### 14.7 Anomalous difference map and the sites it names Where the merge kept the Bijvoet split (§10.5) — which a rotation merge does by default, whether or not the mates were averaged — an **anomalous difference map** is computed as well, with coefficients $ \big(|F(+)| - |F(-)|\big)\, e^{i(\varphi_\mathrm{model} - \pi/2)}, $ over the acentric reflections that have both hands (a centric reflection has no anomalous difference, only noise). Turning the model phase back by 90° is what makes the anomalous scattering, which is 90° out of phase with the normal scattering, add up in the real part: the map's peaks then sit on the anomalous scatterers. It is written as `_anom.ccp4`. Its hand is the one §14.6 settled: with the mates the wrong way round every peak becomes a trough, so a map of clean peaks is itself a check that the frame is right. Rather than searching the map for blobs and leaving a list of coordinates, the map is read **at the model's own atom centres** (hydrogens excluded — they scatter no anomalous signal), and the ten highest, in units of the map's r.m.s., are reported in the log and as `ANOMALOUS_SITE_01`…`ANOMALOUS_SITE_10` in the results report. Each site is therefore named — the atom, residue and chain it belongs to — which is what says *what* carries the signal, not just where it is. The reading is cubic, not linear: the map is sampled every $d_\mathrm{min}/3$, and a peak that sharp read by trilinear interpolation comes out up to a quarter low — unevenly enough to reorder the sites. This reading is the one ANODE reports. Because the map is built on the model's phases and the data's own indexing, it is also the only test of whether the two agree about the **hand** (§14.6). A model and a dataset in opposite hands turn every anomalous peak into a trough, so a map whose deepest hole at an atom is both deeper than $5\sigma$ and deeper than its highest peak says so, and the run reports it as a warning naming that atom. It is not repaired by reindexing: that would make the two agree by construction and destroy the evidence for which of the model and the data is in the wrong hand. Note that R-free cannot see this at all — a mirrored model gives R-free to four decimal places unchanged, and an exactly inverted anomalous map. The list is always ten entries long, so it is their height that carries the information: on a sulfur-SAD dataset the sulfurs fill the top of the list and are followed by a clear drop to the couple of sigma that is the map's noise, while a dataset with no anomalous signal has no such separation and lists ten unrelated atoms at noise level. A scatterer the **model does not contain** — a bound ion, a soaked heavy atom — is by construction invisible in the list, and is what the map file is for. ### 14.8 Rigid-body placement of the model Re-fractionalizing a model into the data cell puts it in the right box but not necessarily in the right place: a non-isomorphous cell squeezes the box without moving the body inside it, and the body's own position in the cell differs from crystal to crystal. Six parameters recover that — an angle-axis rotation about the model's own centroid, then a translation. Rotating about the centroid rather than the cell origin is what keeps the rotation from moving the body bodily, so the two triplets are close to independent. Parameters are carried as six lengths in ångström (the rotation vector multiplied by the model's r.m.s. radius), so a unit of each moves a typical atom by the same amount. It is **one** rigid body. A fragment-screening model arrives already solved and isomorphous, and what is being recovered is the crystal's movement, not the molecule's; splitting it into domains, or giving a bound ligand six parameters of its own, would refine against evidence these data do not separately carry — and the ligand is what the difference map is there to show, not to model away. The refinement walks a coarse-to-fine ladder, 6 Å → 4.5 Å → 3.5 Å, each zone starting from the previous one's answer. It stops at 3.5 Å because that is where rigid-body refinement is conventionally run and because the movement being recovered is a few tenths of an ångström, a tenth of that resolution; a finer zone costs $(1/d)^3$ in grid points and reflections for a placement it cannot meaningfully sharpen. Each evaluation recomputes $F_\mathrm{calc}$ and the solvent mask for the moved model **and re-fits the scale of §14.2** — otherwise the target would measure the scale as much as the placement, and the body would translate to repair a scale error instead of moving to where the density is. The minimiser is Levenberg–Marquardt on the amplitude residuals. $F_\mathrm{calc}$ is computed from **one copy** of the model, gridded without symmetrization and transformed once, and composed over the space group's operators, $F(\mathbf h) = n_\mathrm{cen}\sum_{(R,\mathbf t)} e^{2\pi i\,\mathbf h\cdot\mathbf t}\,F_1(\mathbf hR)$ — the transform of the symmetrized map, rearranged. That makes the derivative with respect to the translation exact and free ($F_1(\mathbf k)$ only picks up the phase $e^{2\pi i\,\mathbf s_\mathbf k\cdot\Delta}$). The rotation's three columns are forward differences, the step a fixed fraction of the zone's resolution, each needing only one copy gridded and transformed. The Jacobian holds the scale and the bulk-solvent mask at the evaluation's; the residuals keep both exact, and the scale re-fit is folded into the Jacobian by variable projection (Golub & Pereyra 1973, in Kaufman's 1975 form), so a step is judged on the target the evaluations actually compute. The refinement sees **only the working reflections**. The step is then committed only if it lowers **R-free**, computed at full resolution on the free set it never saw; otherwise the model is put back exactly where it was read and the maps are the ones it would have given. The whole addition costs about two seconds. Note that the origin is a gauge in some space groups — free in all three directions in $P1$, and along the unique axis in a polar group — so those components of the translation are undetermined; nothing is done about that beyond the Levenberg–Marquardt damping and the R-free gate, which between them make an undetermined direction harmless rather than unstable. Where a CUDA GPU is present, the refinement's target is evaluated on it: the same density, the same composition of $F_\mathrm{calc}$ from one copy of the model, the same bulk-solvent mask with its islands removed, the same scale fit and the same Jacobian, computed on the device, where a fit takes a fraction of a second rather than several seconds. The device is chosen once per validation - the real fit and every replicate of the null of §14.5 on the same one - and the CPU is used where there is no GPU, where the card has too little free memory for the fit's buffers, or after a CUDA failure, in which case the whole validation is run again on the CPU. The two agree to rounding and not bit for bit: the device measures distances in single precision and transforms with cuFFT instead of FFTW, and GEMMI's scale fit, whose stopping rule leaves it unconverged at about $10^{-4}$ of $|F|$, can respond to that rounding with a step of its own. A rigid-body step accepted or rejected on an R-free difference below about $5\cdot10^{-4}$ can therefore go either way between the two; every kernel is deterministic, so either one gives the same answer on every run. The rotation and translation actually taken are reported (`RIGID_BODY_ROTATION_DEG`, `RIGID_BODY_SHIFT_A`), together with the R-free before it (`R_FREE_BEFORE_RIGID_BODY`), so what the placement bought is visible. The coordinates **as placed** are written as `_model.cif`, and as `_model.pdb` where the PDB format can hold the cell (the fragment-screening tools the file feeds — PanDDA, and the pair dimple produces — take a PDB beside the MTZ), so there is a coordinate file that describes the maps: the input's chains, residues, ligands, waters, B-factors, occupancies and anisotropic $U$s, moved, in the same cell and space group as `.mtz`. Taking the frame from the written reflections rather than from the input model matters — §14.6 may have relabelled them to the model's enantiomorph, which is neither the data's original group nor, necessarily, the model's. It is written **whenever the maps are**, not only where the rigid-body step was committed: the model is re-fractionalized into the data cell and may be relabelled whatever the placement decided, so an unmoved model is still not the input file; and a model the null of §14.5 rejected is scored, placed and mapped like any other, which is precisely the case where the density is worth looking at.