leonarski_f
a39fd29f77
* `rugnux --model` reports CC(model, data) - the correlation of the merged intensities with the placed, scaled model - by resolution shell, on the same shells as CC1/2, with the reflection count and a significance for each. * `rugnux --model` fits the model's scale, anisotropic B and bulk-solvent parameters on the working reflections only, so the R-free it reports is measured against a model no free reflection helped scale. * The bulk-solvent parameters of `rugnux --model` are searched over their physically meaningful range instead of being fitted without bounds, so a model is never scaled with a solvent term that has silently switched itself off. * The rigid-body placement of `rugnux --model` uses the same bounded bulk solvent as the reported fit, so a model is no longer placed against a target carrying a solvent term with no physical meaning. * `rugnux --model` puts the model into the data's own description of the lattice before placing it, so a model whose cell is written on other axes - I-centred where the run indexed C-centred, a different unique axis, a permuted orthorhombic cell - is placed rather than scored where it was read; `MODEL_CHANGE_OF_BASIS=` and `MODEL_SETTING_AS_READ=` report it when it happens. * The rugnux results report opens with a summary - `VERDICT=` (`OK`, `WARNINGS`, `UNUSABLE`, `FAILED`), `VERDICT_TEXT=`, `PATHOLOGY_FLAGS=` with one closed-vocabulary code per condition that warned, and the `WARNING:` lines, which used to close the file - and the sections after it are renumbered 1-5 with no gaps. * `rugnux --developer` writes the full results report - the pipeline-internal keys and the long explanations the default report now leaves out - and `--finalist-ledger` adds the evidence for every space group the search considered, not only the one it adopted. * The results report warns when the merged data carry no usable signal and when too little of reciprocal space was measured inside the fitted resolution, and omits `FITTED_RESOLUTION` where the CC1/2 curve it is fitted on never falls off. * rugnux detects translational pseudo-symmetry and reports it under the `PSEUDO_TRANSLATION` flag as `TNCS_DETECTED=` and the `TNCS_*` keys - a translation the merged data are exactly invariant under is reported as `UNDECLARED_LATTICE_TRANSLATION=` under `LATTICE_TRANSLATION` instead - and a detected pseudo-translation can no longer buy a false screw axis in the space-group search or hide a twin from the L-test (`L_TEST_VS_TNCS=`). * The space-group search determines glide planes from zonal systematic absences, so a non-Sohncke space group such as P 2_1/c or Pbca is named where the run previously stopped at its Sohncke subgroup; `SOHNCKE_SPACE_GROUP=` carries the best Sohncke group beside it on every run that searched, and a centre of symmetry is never claimed. * Where the cell metric carries more rotational symmetry than the Bravais class the indexer named, the extra rotations are put to the intensities and the space-group search is asked again on the metric's own cell - adopted only where the intensities confirm the higher symmetry - so a lattice that is nearly but not exactly hexagonal, or whose reduction landed in a sub-cell, still reaches its true point group. * Systematic-absence calls rest on the evidence rather than on counts: a screw axis whose absent class the data show extinct is no longer refused because a handful of reflections in it read as present, and `SPACE_GROUP_ALTERNATIVES=` no longer drops a candidate that differs only on a zone the sweep never measured. * A reference correlation measured on too few reflections is refused instead of scored zero, so a run given a reference MTZ is no longer reindexed on an operator that mapped almost everything outside the reference's coverage. * A frame counts as indexed from 6 spots on its lattice rather than 9, so a weakly diffracting crystal whose frames cannot carry 9 is no longer refused the lattice it fits; `--min-indexed-spots` overrides it. * `-C` accepts a known cell in any equivalent description - conventional or primitive, centred or not - instead of only the reduced primitive form, so a centred cell given the way it is published no longer makes the run report that it found no lattice. * Each reflection is corrected for the sensor's quantum efficiency at the angle it meets the detector (attenuation lengths from the NIST tables, which also fixes the spot-width parallax term on CdTe) and for the attenuation of the flight path between the sample and its pixel; `--flight-path air|helium|vacuum` declares the medium - default air, since no file states it - and the report says what was assumed and what it was worth. The unmerged MTZ records the factors in new `QE` and `FLIGHT` columns beside `LP`, so raw counts are `I / LP * QE * FLIGHT`, and `_process.h5` in new optional `qe` and `flight` datasets. * Rotation geometry post-refinement fits the crystal and the detector at once, against the observed spot positions and the observed rocking angles together, so the refined distance depends far less on how wrong the file's distance was. * A coarsely sliced sweep integrates correctly: partials are joined into one rocking event by angle rather than by frame count, so two crossings of the Ewald sphere are no longer summed into one full, and at 0.5 degrees per image or coarser the per-frame geometry refinement accepts a spot whose miss the exposure's own rotation accounts for. * `rugnux --mode scale` reports the detector tilt and direct beam of the geometry it re-scaled at, instead of zeros that read as a flat detector, and no longer warns that no image was indexed on a run whose lattice came from its input file. * Every rotation run that determined a space group and merged reports what the mounting cost: `SPINDLE_LOST_UNIQUE_FRACTION=` is the fraction (0-1) of unique reflections the mounting made unmeasurable under the measured point group, also written to the master as `/entry/MX/spindleLostUniqueFraction` and what the mounting warning fires on; `SPINDLE_SYMMETRY_AXIS_ANGLE_DEG=` / `SPINDLE_SYMMETRY_AXIS_ORDER=` describe the mounting in the `--developer` report. * Stills and grid scans carry a per-image `spindle_blind_fraction` - how much of a rotation sweep's blind cone this orientation would make unrecoverable, 0.5 and above calling for a second orientation - through the CBOR stream, HDF5 (`/entry/MX/spindleBlindFraction`), the plot and scan-result APIs, and the viewer and frontend plots; an absent value means the frame could not be assessed and is not a 0. * The results report's `REPORT_VERSION` is 7. Reviewed-on: #77 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
42 KiB
Advanced rugnux
The machinery behind a run that needs more than the defaults: external reference data, model validation, re-merging without re-integration, and the full option tables.
:local:
:depth: 2
Reference data and the indexing ambiguity
What a reference MTZ does (-z)
-z reference.mtz supplies known intensities of the same crystal form — a previously merged
dataset, or F-model amplitudes computed from a structure. It is read once, before processing
starts, and used for four things:
- It fixes the space group and the unit cell the run works in, unless
-S/-Coverride them. The cell is a soft reference: indexing may still drift within tolerance, so a small mismatch between reference and data is absorbed rather than rejected. - It resolves the indexing ambiguity (below) — the one thing the data cannot settle for themselves.
- It hands over its R-free test set, where the file carries one, so every dataset of a campaign is scored on the same free reflections.
- It reports CCref, the correlation of the merged intensities against the reference,
in the statistics table. Stills only — the rotation merge never scores itself against the
reference, and its table shows
-in that column.
A reference is not a scale anchor. Both workflows scale against their own data — scaling images
against a foreign dataset injects that dataset's systematics — so -z never puts the reference's
errors into the intensities. --reference-column picks the column to read where the automatic
choice (F-model, else IMEAN/I, else FP/FOBS/F) is not the right one.
For the second of those four jobs — and only that one — an atomic model does as well: --model
computes the intensities it needs from the structure. Where a reference dataset exists, prefer it;
where only a model does, it resolves the ambiguity just the same.
The indexing ambiguity
Some crystals can be indexed in more than one way, each equally valid geometrically, and each
giving different merged intensities. This happens whenever the lattice is more symmetric than the
crystal: in P3, P4, P6, P31, C2 and their relatives (merohedral), and also where the
cell is metrically more symmetric than the Laue class by accident (pseudo-merohedral, up to 2° of
obliquity). The alternatives are related by the crystal's twin laws — reindexing operators such
as k,h,-l.
Nothing in the data breaks the tie: the merge is equally self-consistent either way, so which solution comes out is arbitrary. What that costs depends on the workflow:
- Rotation. The whole sweep is one lattice, so the whole dataset lands in one indexing, picked at random. The merge itself is sound; it may simply be the other solution from an earlier dataset of the same crystal form, and the two cannot be combined, compared or phased against the same model.
- Serial stills. Every crystal is indexed independently, so a run mixes both indexings into one merge. That is not a labelling matter — reflections that are not symmetry mates get averaged together, and CC1/2, Rmeas and the anomalous signal all degrade.
Every run that merges tests for the ambiguity, and where it exists and nothing resolves it, says so in the log and in the report's warnings:
Indexing ambiguity: this cell / space group admits alternative indexing (reindex operator(s):
-h,-k,l). Serial-stills crystals are indexed in one hand at random, and rugnux can only break this
against an external reference. WITHOUT one the merge mixes the hands and CC1/2 is degraded - supply
a reference MTZ (-z) or a model (--model, which needs -C and -S here) to resolve it.
Where something does resolve it, the run says so instead — Indexing ambiguity present (reindex operator(s): -h,-k,l); resolved against the supplied model.
How to resolve it:
| Situation | What to do |
|---|---|
| Rotation, a reference dataset exists | -z reference.mtz. Once the space group is settled, each candidate reindexing of the merged intensities is correlated with the reference and the best-correlating one is re-merged. Only the hkl labels change; the cell does not |
| Serial stills, a reference dataset exists | -z reference.mtz. Resolved per image, at integration time, by correlating each crystal's intensities with the reference — so the merge never mixes hands in the first place |
| Rotation, only a model | --model model.pdb. The merged data are fitted to the model in each candidate indexing and the lowest R-free wins — provided the model first beats its random-orientation null (MODEL_FIT= ACCEPTED) and its lead over the runner-up beats the lead a random placement of the same model takes; the written reflections are then reindexed into it, so the file, the R-factors and the maps agree. Where either bar is missed the data keep the indexing they were merged in, and the report says by how much (MODEL_INDEXING_MARGIN_SIGMA) |
| Serial stills, only a model | --model model.pdb, with the cell and space group given (-C / -S). Structure factors are computed from the model up front and used as the reference for the per-image test, exactly as a reference MTZ would be — the ambiguity has to be broken at integration time, and a model can supply the intensities to break it with |
| Neither | The run warns and merges in whichever indexing it found: for rotation data a usable dataset in an arbitrary frame, for stills a degraded one |
Two things to know about where the choice lands:
--mode scalecannot repair a stills run after the fact. The per-image test happens at integration time, so a_process.h5whose images were integrated without a reference — an MTZ or a model — has already lost the distinction, and no re-merge brings it back. On rotation data, where the ambiguity is one choice for the whole dataset,--mode scale --model model.pdbdoes resolve it.- The choice reaches the written reflections, not only the R-factors and the maps: the merged
.mtz/.cif/.hkl(and_unmerged.mtz, where it is asked for) are written in the indexing the model or the reference settled on, so the file can be refined against that model as it stands.
Two things the indexing ambiguity is not:
- Not the enantiomorph. P41212 versus P43212 (or
P31 versus P32) leaves the merged intensities unchanged, so no amount of
data can choose between them and rugnux never tries — the run reports the pair it cannot separate.
A model the data accept names it (
MODEL_FIT= ACCEPTEDin the report), and the naming is a label only: the written reflections take the model's space group, no reflection moves, and I(+)/I(-) stay exactly as measured — reindexing by the change of hand would flip every anomalous difference, so it is never done. Whether the model and the data really agree about the hand is answered afterwards by the anomalous difference map, which warns when the density at the model's atoms comes out inverted. The_process.h5, whose per-image reflections were written as they were integrated, keeps the group the run determined and is left alone. - Not twinning. The twin laws are the same operators, but twinning is a property of the crystal — two orientations diffracting at once — and is reported separately in the report's twinning section. A crystal can have an indexing ambiguity without being twinned, and usually is.
The algorithms behind both are in CPU/GPU data analysis ▸ Reference data.
Validating against a model (rugnux --model)
Given an atomic model of the same structure, --model model.pdb scales the model structure
factors to the merged amplitudes — fitting a flat bulk-solvent contribution and an overall
anisotropic B — and reports R-work / R-free and the mean 2mFo−DFc density at the atom centres.
It also writes the σA-weighted maps <prefix>_2fofc.ccp4 (2mFo−DFc) and
<prefix>_fofc.ccp4 (mFo−DFc), and the map-coefficient MTZ <prefix>_maps.mtz, next to the
merged reflections — and, where the merge kept the Bijvoet split (a rotation merge always does),
<prefix>_anom.ccp4, the anomalous difference map whose strongest sites the report names
(ANOMALOUS_SITE_01…10). The structure itself is not refined; the model is re-fractionalized into
the data cell and then placed as one rigid body, so a deposited model from a crystal that is not
quite isomorphous still sits where the density is.
Because the model moves, the input file no longer describes these maps, so the model as placed is
written as <prefix>_model.cif — the input's chains, residues, ligands, waters, B-factors,
occupancies and anisotropic Us, at the coordinates the maps were computed from, in the same unit
cell and space group as <prefix>.mtz beside it. The same coordinates are written as
<prefix>_model.pdb as well, because the fragment-screening tools this file exists to feed take a
PDB: PanDDA's per-dataset input is <name>.pdb beside <name>.mtz, and dimple produces that same
pair. (The PDB is skipped, with a log line, for a cell its fixed-width format cannot hold; the mmCIF
is unconditional.) That is the file to open with the maps, and the one
to hand to REFMAC5 or phenix.refine; its starting R-free is the R_FREE= the report quotes.
(<prefix>.cif is the merged reflections — hence the separate name.) Both are written whenever the
maps are, including for a model the data rejected: the rejection is a result, and it is exactly the
case where someone wants to look at the model in the density.
The model may be PDB or mmCIF, gzipped or not, and the format is taken from the file's own
content rather than from its name — a model downloaded as .cif, .pdb, .ent or with no useful
extension at all is read the same way. A model that cannot be read, or that has no atoms, no unit
cell or no usable space group, does not fail the run: it is logged, and the results report carries a
WARNING: Model validation did not run: … line, so a run that silently produced no R-free and no
maps cannot be mistaken for one that was never given --model.
Either way the results report carries a 5. MODEL VALIDATION section: R_WORK= / R_FREE=
with their reflection counts, the bulk-solvent and overall scale parameters, the mean 2mFo−DFc density
at the atom centres, the reindexing operators the written reflections were brought into the model's
frame with, and MAPS_PREFIX=; or MODEL_VALIDATION= NOT_PERFORMED with
MODEL_VALIDATION_REASON= when the model could not be used. A run given no --model has no such
section at all.
It is a data-quality lens, independent of the internal statistics: R-free measures the merged
intensities against external truth, where CC1/2 and Rmeas only measure them against
themselves. It also settles the two things merged intensities alone cannot, in two different ways —
though only where the data accept the model first: the same model is refitted, and re-placed, from
random orientations about its own centroid, and the real fit has to beat that null (MODEL_FIT= in
the report; §14.5).
A model that fits no better than its own random placements decides nothing, and the reflection files
are byte for byte what a run with no model would have written. The
enantiomorph is a relabelling: data merged in P41212 against a
P43212 model take the model's space group as the label they are written
under, with no reflection moved and I(+)/I(-) exactly as measured. A merohedral
indexing ambiguity — when no reference MTZ has already fixed it — is a
reindexing: the candidate with the lowest R-free is kept — where its lead over the runner-up
beats the lead a random placement of the same model takes — and applied to the written
reflections as well as to the R-factors and the maps; otherwise the data keep the indexing they
were merged in. Validation runs before the reflection
files, so the .mtz / .cif / .hkl come out in the model's frame either way — reindexed where
the ambiguity decided, and carrying the model's space group where the hand was adopted. The log names the operator in each case, and for the indexing choice gives the winning
R-free together with the runner-up.
Re-scaling and re-merging (rugnux --mode scale)
The scale mode re-scales and merges the already-integrated reflections stored in a
_process.h5 file, without re-running spot finding or integration. Use it to re-merge quickly with a
different space group, resolution limit, anomalous setting or outlier rejection. It reuses the same
-o/-N/-s/-e/-S/-A/-z/--scaling-* options as the full run, and (unlike the full pipeline) does
not run a space-group search: it merges in the space group and unit cell the file records, and -S
/ -C override them. A _process.h5 written before the group was stored carries none, and merges
in P1 unless -S says otherwise.
A reference MTZ (-z) is accepted here on stills data, where it fixes the space group and cell,
reports CCref and hands over its R-free flags; on rotation data the rotation scaler
declines it and the run stops with a message saying so. --model works here exactly as in a full
run: on rotation data, where the indexing ambiguity is one choice for
the whole dataset, it can settle the enantiomorph label and the indexing — under the same fit test
as always — and the re-merged reflections are written in what it settled. What no re-merge can
repair is a stills _process.h5 integrated without a reference: there the ambiguity was
decided per image, at integration time, and the distinction is gone from the stored reflections.
Where the full run re-seated the lattice — the space group it settled on is in a different setting
from the one each image was indexed in — the file records the change of basis as
/entry/MX/reindexMatrix, and rugnux applies it on read, so the reflections and the stored cell
describe the same frame. An older file that was affected by this cannot be repaired (the matrix is
not recoverable after the fact); such a file now stops with a message naming both cells and the
exact -S/-C override to merge it in its own setting, instead of failing inside the merge.
Command-line options
General:
| Option | Description |
|---|---|
-o, --output-prefix <txt> |
Output file prefix (default: output) |
-N, --threads <num> |
Number of worker threads (default, and for any value ≤ 0: all hardware threads). Some stages take fewer, because past a point more workers make them slower: the per-image loop of --mode mx uses at most 16 per GPU unless -N was given a positive value, and first-pass spot finding and the beam-stop pre-scan have ceilings of their own that -N does not lift. Scaling, merging and the space-group search use the full count |
-s, --start-image <num> |
First image to process (default: 0) |
-e, --end-image <num> |
Last image to process (default: all) |
-t, --stride <num> |
Process every n-th image (default: 1) |
-v, --verbose |
Verbose output |
Mode — --mode <name> (default mx):
| Value | Description |
|---|---|
mx |
Full analysis — spot finding, indexing, integration and merging |
azint |
Only azimuthal integration (no spot finding/indexing); writes <prefix>_process.h5 |
scale |
Only re-scale/merge the already-integrated reflections in the input _process.h5 (no re-integration) |
calibration |
Determine the detector geometry from powder rings; writes <prefix>.poni and <prefix>.json |
Calibration (--mode calibration):
| Option | Description |
|---|---|
--calibrant <name> |
Powder standard: lab6 | agbh | ceo2 | si | ice (default lab6, case-insensitive) |
--calibration <txt> |
How the rings are measured: rings | spots (default rings; see above). rings defaults --azim-phi-bins to 32 |
--no-refine-tilt |
Do not refine the detector tilt: hold rot1/rot2 at the header value and fit only the beam centre and the distance, for a calibration handed to a program that cannot express a tilted detector (XDS) |
Detector mask:
| Option | Description |
|---|---|
--detect-beam-stop[=N|off] |
Find the beam stop and its holder in a projection of N images and add them to the pixel mask as bit 9, so nothing shadowed by them is integrated. On by default (60 images); =off disables. Reflections behind the stop are attenuated but not flagged, so they integrate low with a plausible sigma and no existing rejection catches them |
Geometry:
| Option | Description |
|---|---|
--beam-center-check[=off] |
Measure the beam centre from the isotropy of the scattered background on every run, report how far the file's value is from it — against how right this particular geometry needs it to be — and index a second first pass at the measured centre to see whether the two centres give the same lattice. The fit reads the projection --detect-beam-stop already builds, so it costs no extra frames. On by default; =off disables. Nothing is committed on a run that indexes: the measured centre is adopted only where the file's centre indexes nothing and the measured one indexes a majority, and a disagreement between two passes that both worked is reported with both cells and left undecided |
--beam-center-search[=N|off] |
After a first pass that indexes fewer than half the validation frames, step the centre a pixel at a time out to N px along each detector axis and keep the first rung that indexes a majority. On by default (12 px); =off disables. It runs only after a pass that has already failed, and spot finding is not repeated, so a run that indexes never pays for it. Both detector directions are searched: a centre error across the spindle collapses the indexed fraction and announces itself, while one along it holds the frame count up and quietly returns an axis harmonic |
--estimate-beam-center |
Measure the direct beam before indexing, from the symmetry of the spots where the sweep reaches at least half a turn and from the radial background profile where it does not; the value in the file is kept where neither can measure it. Off by default |
--no-fit-spindle |
With the above, keep the rotation axis given in the file instead of fitting its skew about the beam |
Spot finding:
| Option | Description |
|---|---|
--spot-sigma <num> |
Noise sigma level for spot finding (default: 4.0) |
--spot-threshold <num> |
Photon-count threshold for spot finding (default: 10) |
--adaptive-spots |
Self-calibrating detection (default, stills and rotation alike): the strong-pixel threshold comes from each image's own per-resolution-ring noise instead of the fixed --spot-threshold, so one setting adapts across datasets (no per-dataset --spot-threshold/--spot-sigma tuning) |
--no-adaptive-spots |
Turn adaptive detection off and use the fixed --spot-threshold / --spot-sigma finder |
--spot-false-pixels <num> |
Adaptive-detection operating point: expected noise pixels tolerated per frame (default: 100; implies --adaptive-spots) |
--spot-high-resolution <num> |
High-resolution limit for spot finding, Å. Omitted (or 0): no resolution clipping — spot finding extends as far as the detector reaches, for rotation data as well as stills |
--spot-low-resolution <num> |
Low-resolution limit for spot finding, Å (default: 50; lower it, e.g. 24, to exclude the direct-beam halo on weak serial data; 0 removes the limit) |
--min-pix-per-spot <num> |
Minimum connected strong pixels per spot. If omitted, min-pix is chosen per image (stills indexing): the frame is indexed at min-pix 3/2/1 and the one maximising indexed-spot count × indexed fraction is kept. Give an explicit value to force a fixed min-pix instead. |
--max-spots <num> |
Maximum spots kept per image (the strongest ones) and handed to indexing. If omitted, the budget is measured on rotation data: the first pass reads how deep into an image's spot list its spots still lie on the lattice it found, and the run keeps that many (never more than 1000). Give a value to pin it. Stills always use the fixed 1000. |
--detect-ice-rings[=on|off] |
Flag ice-ring spots (de-prioritised in indexing) and exclude ice-ring reflections from scaling. Default: the master file's detect_ice_rings, or — where the file carries no such key — on for rotation and off for stills |
Azimuthal integration (the radial profile behind the per-image ice-ring score). Every q here is q = 2π/d, in Å⁻¹:
| Option | Description |
|---|---|
-q, --azim-q-spacing <num> |
Q bin spacing, 1/Å (default: 0.01; finer resolves the narrow ice rings) |
--azim-min-q <num> |
Minimum Q, 1/Å |
--azim-max-q <num> |
Maximum Q, 1/Å. Omitted: integration extends to the highest Q the detector reaches. The adaptive spot finder shares these Q bins, so this also sets how far self-calibrating detection can see |
--azim-phi-bins <num> |
Number of azimuthal (phi) bins (default: 1) |
--polarization-correction <on|off> |
Enable/disable the azimuthal polarization correction |
--solid-angle-correction <on|off> |
Enable/disable the azimuthal solid-angle correction |
Indexing:
A dataset with a rotation goniometer axis is processed as rotation data (two-pass rotation
indexing) by default; a dataset without one is processed as independent stills. --force-still
overrides the former; the -R / --single-pass-rotation / --force-rotation-lattice flags request
rotation explicitly and pick the pass or lattice.
The FFT search looks for cell axes between 10 Å (--fft-min-unit-cell) and a default longest
axis of 500 Å, which has no flag of its own — a reference cell (-C) moves both bounds to cover
the cell it names. A de-novo rotation run also tries a second first-pass hypothesis with the short
end lowered to 5 Å, so a small-molecule cell below the 10 Å floor is indexed on its true axes
rather than as a supercell of them; the standard pass's answer stands unless that evidence takes it. An axis beyond the search's reach is not refused: the run returns a
plausible shorter sub-cell or an axis harmonic and processes it happily, so a cell that comes out
at a half or a third of the expected long axis should be read as this limit, not as the crystal.
For very long axes the direction grid's angular resolution binds well below 500 Å — see the
analysis reference on FFT indexing.
| Option | Description |
|---|---|
--force-still |
Treat a rotation (goniometer) dataset as independent stills instead of rotation |
-X, --indexing-algorithm <txt> |
FFBIDX | FFT | FFTW | Auto | None |
-C, --unit-cell <cell> |
Reference unit cell "a,b,c,alpha,beta,gamma" (required by ffbidx). On rotation data it also widens the FFT search to cover the cell given, at both ends: the longest axis looked for is raised to reach it, and --fft-min-unit-cell is lowered to admit it |
--fft-min-unit-cell <num> |
Shortest cell axis the FFT search accepts, Å (default: 10). A candidate with a shorter axis is discarded — but a de-novo rotation run also runs a second first-pass hypothesis with the floor lowered to 5 Å by default, adopted when the standing cell turns out to be an integer supercell of what it finds, so a small-molecule cell is indexed without this flag. -C lowers the floor on its own to cover the cell given (and the second hypothesis then stays out of the way). On stills, where neither applies, a crystal below the floor cannot be indexed unless this is lowered |
--min-indexed-spots <num> |
Spots a frame must have on the lattice before it counts as indexed (default: 6). The rotation first pass scores candidate lattices on that count, so on a pattern with fewer spots per frame no lattice can score and the run gives up; integration itself is not gated by it |
-S, --space-group <num|symbol> |
Space group number (92) or Hermann-Mauguin symbol (P43212) — for indexing and scaling |
-r, --refine <txt> |
Geometry refinement: none | orientation | beam_and_lattice (default) | flex (try all three per image, keep whichever indexes the most spots; alias multi) |
-R, --two-pass-rotation[=num] |
Two-pass offline rotation indexing (default for goniometer data; optional first-pass image count, default 100) |
--single-pass-rotation[=num] |
Online-like single-pass rotation indexing (optional min angular range, deg) |
--force-rotation-lattice <vec> |
Force rotation lattice (9 floats, Å), skipping the first pass |
--rotation-no-postrefine |
Rotation: disable the default-on two-pass geometry post-refine (see the rotation section) |
--refine-geometry[=N|off] |
Stills: extra first pass that bundle-adjusts the shared beam/distance/cell from N strongly-indexed frames (default 200) then re-indexes; default ON for stills with a reference cell (-C / -z), =off disables |
--index-ice-rings[=on|off] |
Index on the spots flagged as sitting on an ice ring too, instead of setting them aside (default: off; no effect without --detect-ice-rings, which does the flagging) |
Indexer choice in brief: ffbidx (GPU) refines toward a known cell and is best for sparse
serial stills; fft (GPU) / fftw (CPU) index de novo and suit strong rotation data. See the
CPU/GPU data-analysis reference for the algorithms.
Scaling and merging:
| Option | Description |
|---|---|
--no-merge |
Skip scaling and merging (on by default); write only the per-image _process.h5 |
-A, --anomalous |
Anomalous mode: merge each Bijvoet hand as its own unique reflection, so multiplicity, completeness, ⟨I/σ⟩ and the error model are counted anomalously. A default rotation merge already writes I(+)/I(-) (see Reflection-file conventions); -A changes the counting basis, not whether the anomalous signal is in the file |
--scale-fulls / --no-scale-fulls |
rot3d: refit a per-frame scale on the combined fulls (XDS order, Unity model); on by default for rotation data, off for stills |
--smooth-g[=deg] |
rot3d: smooth the per-frame scale G over a degree range before the 3D combine (XDS DELPHI-like; default 5° for rotation, 0 = off) |
--no-scaling-corrections |
rot3d: disable the default-on decay + absorption + modulation correction surfaces fitted on the fulls after scale-fulls (see below) |
--relative-b[=deg] |
rot3d: fit a per-batch relative-B beyond the single decay slope over deg-degree batches, cross-validated (default 10° when bare; off otherwise) |
--simple-stills |
Stills: treat every reflection as a full (p = 1, single-pass scale/merge) — disables the default-on physical partiality post-refinement |
--no-expected-variance-merge |
Stills: disable the default expected-variance merge weighting (which rebuilds each weak observation's signal variance at the reflection mean to de-bias the inverse-variance merge); restores observed-sigma weighting |
--capture-uncertainty <num> |
rot3d: systematic sigma on under-captured fulls, ~num·(1−captured_fraction)·I (default: 1.0 for rotation, 0 otherwise) |
--min-captured-fraction <num> |
rot3d: drop a combined full whose rocking curve was captured below this fraction — edge-of-sweep truncated fulls (default: 0.7 for rotation, 0 otherwise; 0 = off) |
--scaling-high-resolution <num> |
High-resolution limit for scaling, Å — manual override (default: no limit; disables the automatic cutoff below) |
--scaling-low-resolution <num> |
Low-resolution limit for scaling and merging, Å (default: 50, which is also XDS's own default; 0 removes the limit). The beam stop suppresses air scatter well beyond the shadow it casts, so the background stays depressed across pixels the beam-stop mask leaves open — measured at 30–44% of the field value, recovering only around 50 Å. A background ring in that zone over-estimates the background, so reflections coarser than the limit mostly integrate negative. Beam-stop masking therefore does not make this redundant. A large cell (a few hundred Å) does have real reflections coarser than 50 Å, but on the geometries measured here they fall in that zone and are not usable as integrated; raise the limit only if the low-resolution shell statistics justify it |
--resolution-cutoff <txt> |
Automatic high-resolution cutoff for the written reflections and reported shells: cc-logistic | off (default: cc-logistic; ignored when --scaling-high-resolution is set) |
--resolution-cc-target <num> |
CC1/2 target defining the cc-logistic fall-off (default: 0.30) |
--resolution-shells <num> |
Number of resolution shells in the reported statistics table (default: 9). The bins are equal steps in 1/d² between the lowest- and highest-resolution reflection merged, which is XDS's rule, and 9 is XDS's count — so at the same resolution limits the two tables have the same shells and can be read row for row |
--min-partiality <num> |
Minimum partiality to accept a reflection (default: 0.02) |
--ice-min-score <num> |
Ice-presence gate: the measured per-run ice score (1 = no ice) a dataset must reach before any ice handling is applied — the flagging and the exclusion from scaling (default: 1.5; 0 = no gate). The eleven fixed hexagonal bands cover 16–26 % of the unique reflections whether or not the crystal has ice, so handling ice on a clean crystal only costs completeness |
--ice-min-spot-ratio <num> |
The second ice-presence channel: found spots on the hexagonal rings over the same q width of ice-free flanks beside them (1 = spots spread evenly). Ice in large crystallites diffracts as discrete spots and leaves the radial profile flat, so --ice-min-score alone is blind to it (default: 2.0; 0 disables this channel) |
--reject-outliers <num> |
Per-observation outlier rejection, N σ from the per-reflection median (default: 6 for rot3d, off otherwise) |
--min-image-cc <num> |
Per-image CC limit, percent (default: no limit) |
--search-min-zeta <num> |
De-novo space-group search only: also search a merge of just the observations whose Lorentz geometry |ζ| reaches this, and report both answers (default: 0.85 for rotation, 0 = single search). Reflections crossing the Ewald sphere near-tangentially are measured worst and can make a real symmetry operator look like a twin law. Where the two searches disagree, the merge of all the observations decides — as it always has for the systematic absences |
--mosaicity <num> |
Diagnostic: fix the scaling mosaicity (°) instead of using the per-image seed |
--scaling-iterations <num> |
Scaling iterations with no reference data (default: 3) |
-z, --reference-mtz <file> |
Reference MTZ of the same crystal form: fixes the space group and cell, resolves the indexing ambiguity, hands over the R-free set and reports CCref. Not a scale anchor |
--reference-column <label> |
Reference MTZ column to use (default: auto — F-model, else IMEAN/I/…) |
--model <file> |
Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see Validating against a model). It also settles the frame the reflections are written in: the enantiomorph, and the indexing ambiguity where no -z did. For serial stills given -C / -S, the model's structure factors become the per-image reference |
--write-process-h5 |
Also write the (large) _process.h5 when merging (default: only .mtz/.cif) |
--developer |
Write the full <prefix>_report.txt: the pipeline-internal keys and the long explanations the default report leaves out. Nothing is computed differently — the same report, rendered in full (see The results report) |
--finalist-ledger |
Report the full-resolution evidence for each space group the search considered, not only the one it adopted (report-only; the decision is unchanged) |
--export-unmerged |
Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On by default whenever there is an output prefix. Rotation partials are summed into one full per reflection. Intensities carry the deterministic per-reflection corrections and nothing else — Lorentz and polarization in LP, sensor efficiency in QE, flight path in FLIGHT — while the partiality is not divided out and the per-image scale is not applied. Written in --mode mx and --mode scale, and with --no-merge. See The unmerged export |
--no-export-unmerged |
Do not write <prefix>_unmerged.mtz. It is the largest file a run produces, so a run whose observations are not going to another scaling program can skip it |
--no-p1-crosscheck |
Do not write <prefix>_P1.mtz, the P1 cross-check merge described under Output files. Every rotation run that determines its own space group writes it; a run given -S writes none either way |
--export-unmerged-partials |
Write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) for the reading program to sum. Off by default, and independent of --export-unmerged |
Integration:
| Option | Description |
|---|---|
--integrator <txt> |
Spot integrator: gaussian (profile-fit, default) | empirical | boxsum (classical fallback) |
--integration-radius <r> |
Signal-box radius r1, or r1,r2,r3 (px). One value ⇒ r2=r1+2, r3=r1+4 |
--adaptive-integration-radius[=on|off] |
Set the signal radius r1 from how wide this crystal's spots actually are (default: on for rotation, off for stills). r1 is not the integration domain — that is the profile-fit grid — but it is the aperture the profile width is learned over, and a second moment over a disk of radius a saturates at a²/4, so at the shipped r1 = 4 the learned σ can never exceed 2 px and a broader spot is fitted with a profile the model cannot represent. The pre-scan reads r80, the radius holding 80 % of a spot's flux, off isolated strong spots over a fixed 14 px aperture that owes nothing to r1, fits it against 1/d and evaluates it at 5 Å; then r1 = clamp(round(2·r80), 4, 6), r2 = r1 + 2, and r3 is taken so the r2..r3 background ring keeps the area it has at the default 4,6,13. The ceiling of 6 is pattern density: r2 also drives the neighbour-ownership radius and the ring's inner edge, and past it a dense pattern starts losing reflections whose ring falls below six clean pixels. Ignored when --integration-radius is given. The widened radius applies to the final integration pass only — the two-pass geometry pre-pass keeps the radius the run started with, because the post-refinement fits the detector distance and beam to the observed reflection positions and those move with the signal disk (33 µm and 0.02 px between r1 = 4 and r1 = 6 on one crystal, enough for the second pass's de-novo lattice search to settle on a different lattice and index a fifth fewer frames). And where the widened radius leaves more than 1.1 % of the predicted reflections without a background ring — a pattern too dense for it — the final pass is re-integrated at the fixed 4 px radius, and the log says so. Over the rotation regression battery it moves 12 of 38 crystals and leaves the merged intensities of the other 26 unchanged; where it moves them, per-shell ⟨I/σ⟩ improves by up to 31 % and R_meas by up to 24 % |
--integration-stencil <k> |
Push the r2..r3 background ring out by k times the beam's radial streak bandwidth·R_px, per reflection (default 0 = the fixed circular ring). A fixed ring otherwise ends up on a streaked reflection's own tails at high resolution and measures them as background. Only the ring moves, and only radially — the r1 signal box stays a circle — and the growth is capped at 2·r3. The neighbour exclusion grows with it, so on a crowded pattern a few reflections can be left with too little background and dropped. Needs --bandwidth: on a monochromatic beam the streak is zero and this does nothing |
--background-clip <n> |
Monochromatic (rotation + still): high-side clip of the background ring at mean + n·√mean (default 4; 0 = off). The default background estimator — it rejects neighbour cores and zingers without the symmetric trim's Poisson skew bias. Broadband data always clip, at 3σ; ignored by --integrator boxsum |
--background-trim <f> |
Use the old symmetric trimmed mean for the background ring instead of the clip, 0≤f<0.5 (0.10 was the former default). Switches --background-clip off. A symmetric trim is biased low on Poisson data and adds ~5 counts to every partial, so this is for back compatibility only; 0 = plain ring mean. Rings holding more than 512 pixels fall back to the plain mean (the GPU sorts the ring in shared memory and the CPU now matches it), which the default radii never reach but wide ones do |
--background-radial[=on|off|auto] |
Correct the background ring for the curvature of the radial background (default off). Disk and ring are concentric, so a background linear in position cancels between them and only curvature survives — which on a smooth ice ring reaches +26 counts on a single reflection. auto applies it per image where that image's ice score shows a smooth powder ring, since the model is a function of radius alone: on ice made of discrete crystallite spots there is no smooth ring and the correction makes the bias worse. Ignored by --integrator boxsum (no clip pass to take the curve from) |
--integration-high-resolution <num> |
High-resolution limit for prediction and integration. Omitted (or 0) means integration extends as far as the detector reaches — which is what the predictor can place on the detector anyway, since it rejects reflections that miss it. Set a value to integrate less than the detector offers |
--max-hkl <n> |
Predict reflections with |h|,|k|,|l| ≤ n (max 511). By default this is derived per crystal from the refined cell as ceil(max(a,b,c)/d_min) + 1, which is the exact bound: the predictor keeps only |q| ≤ 1/d_min and h = a·q, so no reflection can lie outside it and no candidate inside it is wasted on a shorter axis. Set it only to override that |
--bandwidth <num> |
Relative X-ray bandwidth FWHM (e.g. 0.01 for a 1% DMM); default from file or 0 (monochromatic) |
--overlap <txt> |
What to do where two predicted reflections share signal pixels: off | reject | exclude (default exclude). A shared pixel belongs to the nearer centre; without this a crowded reflection reads high on a dense pattern. exclude drops the shared pixels from the profile fit, which renormalises itself, and keeps the reflection; reject instead drops the whole reflection when too little of its profile is cleanly its own. --integrator boxsum has no profile to renormalise, so only reject acts there |
--overlap-minpk <f> |
Least fraction of a reflection's expected profile that must be usable for it to be kept (default 0.75, XDS MINPK). Governs both the fraction that must be readable — not masked, untrusted, in a gap or overloaded — in every profile mode, and, under --overlap reject, the fraction that must be cleanly its own. Under --integrator boxsum any unreadable pixel discards the disk outright and the reject fraction goes by disk area, which cuts harder |
--flight-path <txt> |
What the diffracted beam crosses between the sample and the detector: air (default) | helium | vacuum. A reflection arriving at angle α to the detector normal flies D/cos α instead of D, so it crosses more of the medium and reads low; the factor is exp(D/L·(1/cos α − 1)), with L the attenuation length of the medium from the NIST tabulation and D and λ as stated by the file — nothing in it is fitted. Nothing in NXmx, and no field of any master file rugnux reads, describes the medium, so it cannot be auto-detected and is assumed; the report states which medium was assumed and what it was worth (FLIGHT_PATH, FLIGHT_PATH_WILSON_B). Nor can it be inferred from the implied transmission — in this corpus a confirmed helium station sits at 51 % implied air transmission and a confirmed air station at 63 %. In air the term is +1.5 % at α = 55° for 18 keV over 160 mm and several-fold below 5 keV, where stations use helium precisely because air is unusable; helium attenuates ~1/600 of air, and vacuum leaves intensities untouched |
--prediction-mosaicity <num> |
Diagnostic: fix the rocking width (deg) the prediction window opens to, leaving partiality on the per-image σ_M. The two are one number by default, so a σ_M that moves takes the integrated reflection population with it |
Geometry overrides (defaults are taken from the input file; override them to reprocess with a corrected geometry):
| Option | Description |
|---|---|
--beam-x <num> |
Beam centre X (pixel) |
--beam-y <num> |
Beam centre Y (pixel) |
--detector-distance <num> |
Detector distance (mm) |
--wavelength <num> |
Wavelength (Å) |
--rot1 <num> |
PONI detector rotation 1 (rad) |
--rot2 <num> |
PONI detector rotation 2 (rad) |
--rot3 <num> |
PONI detector rotation 3, about the beam (rad) — the one rugnux --mode calibration writes into its .poni, so a refined third rotation can be given straight back |
--detector-mirror-y |
The stored image is mirrored in Y relative to the frame the PONI angles are stated in |
--detector-quarter-turns <0-3> |
The stored image is turned by this many multiples of 90° about the beam relative to that same frame |
--polarization <num> |
Polarization factor |
--rotation-scale <k> |
Goniometer rotation scale: the stage turned k times the angle stored in the file (the commanded one). Applied to both passes, and overrides the scale rugnux fits for itself |