diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 58f73f86a..c6c9de516 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -11,6 +11,7 @@ * `rugnux` writes a batch header in the unmerged MTZ for every image the observations span, not only for the images that produced one, so a scaling program reads one run per sweep. * `rugnux` leaves an event out of the unmerged MTZ when less of its rocking curve was captured than `--min-captured-fraction`, as the merge does, since the exported rows are declared full. * `rugnux` writes `FreeR_flag` with 0 for the test set and 1 for the working set, the CCP4 convention REFMAC5 defaults to; it was the other way round. +* `rugnux` writes the merged and P1 MTZ with the reserved `HKL_base` dataset ahead of its own, so a CCP4 program reads the wavelength from the file instead of falling back to 1.54187 A. * `rugnux` marks the free set in the merged mmCIF as `_refln.status` = `f` instead of a `_refln.status_free` column that is not in the PDBx dictionary and that `cif2mtz` refused to read. * `rugnux -S` refuses or re-seats a fixed space group whose symmetry axes the indexed cell does not carry, not only one whose centring differs; `--mode scale` refuses it too. * The rugnux results report names the groups the data cannot separate, the enantiomorph state and any refused higher point group as `SPACE_GROUP_ALTERNATIVES`, `SPACE_GROUP_ENANTIOMORPH`, `SPACE_GROUP_REFUSED_POINT_GROUP` and `SPACE_GROUP_REFUSED_REASON`; `REPORT_VERSION` is 5. @@ -34,6 +35,7 @@ * `jfjoch_viewer` reads PILATUS miniCBF sweeps natively, and draws grid scan cells in the proportion of the scan steps. * The HDF5 and API documentation say how a grid scan records the angle its spindle stood at: send the goniometer axis with a step of 0. * The rugnux manual is reorganised into task pages with a run overview, and gains worked phenix / REFMAC5 / POINTLESS-AIMLESS / careless examples. +* The rugnux manual gains worked Phaser and SHELXC/D/E examples, and says how to hand each of them the space groups the merged intensities could not separate. * `jfjoch_viewer` labels the merge-statistics plot over the range the axis is drawn on, so the CC1/2 curve is no longer read against tick labels covering only part of it. ### 1.0.0-rc.165 diff --git a/docs/RUGNUX_INTEGRATION.md b/docs/RUGNUX_INTEGRATION.md index 846c6fc2d..3033d38c6 100644 --- a/docs/RUGNUX_INTEGRATION.md +++ b/docs/RUGNUX_INTEGRATION.md @@ -49,6 +49,20 @@ expect. Two properties worth knowing before using it: untouched, but the absolute scale is not meaningful. This matters only if you intend to compare magnitudes with another file; SHELXC and ANODE use ratios alone. +**MTZ** (`.mtz`, and `_P1.mtz` beside it). The CCP4 anomalous layout, with the +column types CCP4 programs dispatch on: + +``` +H K L IMEAN SIGIMEAN I(+) SIGI(+) I(-) SIGI(-) F SIGF F(+) SIGF(+) F(-) SIGF(-) FreeR_flag +H H H J Q K M K M F Q G L G L I +``` + +`F` is the French–Wilson amplitude. The header carries the determined space group, the refined cell +and the wavelength, on a dataset of its own behind the reserved `HKL_base` — so `mtzinfo` and +`mtzdmp` report the wavelength the data were collected at, and a program that takes *f′*/*f″* from +the file is given the right edge. The Bijvoet columns are present on any rotation merge, with or +without `-A`; a stills merge has no Bijvoet split and the file then stops after `F SIGF FreeR_flag`. + ## The unmerged export `_unmerged.mtz` holds every integrated observation, before scaling and merging, in the column @@ -173,13 +187,188 @@ careless mono --anomalous "BATCH,dHKL,Hobs,Kobs,Lobs,XDET,YDET,BGMEAN,BGVAR,LP,F myrun_unmerged.mtz out/myrun ``` -**Molecular replacement.** When the report names an enantiomorphic pair (`P 41 or P 43`), let -Phaser try both hands: `SGALTERNATIVE SELECT HAND`. When section 4's candidate table shows a close -runner-up, `SGALTERNATIVE SELECT ALL` searches the alternatives, and `myrun_P1.mtz` is there for -re-merging in a subgroup when the call itself is in doubt. +**Molecular replacement and experimental phasing** each get a section of their own below — +[Phaser](#molecular-replacement-with-phaser) and [SHELXC/D/E](#experimental-phasing-with-shelx). +Both are where rugnux stops and the next program starts, and both meet the one thing the merged +intensities could not decide: which of several space groups the data are in. -**`iotbx.merging_statistics myrun_unmerged.mtz`** and **SHELXC on `myrun.hkl`** need no arguments -or label choices at all. +**`iotbx.merging_statistics myrun_unmerged.mtz`** needs no arguments or label choices at all. + +## Molecular replacement with Phaser + +rugnux does not do molecular replacement, so Phaser is the next program for anyone who has a search +model. Both CCP4 and phenix ship it — `phaser` and `phenix.phaser`, the same 2.8.3 build in the +versions this was checked against — and either takes the merged `myrun.mtz` as it is written. + +**No `LABIN`, no label choices.** Phaser reads the cell, the space group and the resolution range +out of the file and picks the intensity columns itself. Where phenix stops on a merged file because +it cannot choose between two equally usable observation arrays (see above), Phaser simply announces +what it took: + +``` + Data read from mtz file: myrun.mtz + Space-Group Name (Hall Symbol): P 41 21 2 ( P 4abw 2nw) + Unit Cell: 78.06 78.06 37.70 90.00 90.00 90.00 + Column Labels Selected: IMEAN SIGIMEAN + Resolution on Mtz file: 0.99 39.03 +``` + +So the whole run is the model and the cell contents: + +``` +phaser <` should be +about 0.80 where there is no anomalous signal. Two sweeps are quoted below, both collected at 5 keV +for the sulfur signal: a cubic one that went all the way, and a tetragonal one that did not. The +cubic one, 2.5 Å at 95 % completeness and multiplicity 30, reads: + +``` + Resl. Inf. 13.02 8.01 6.03 4.93 4.22 3.71 3.33 3.04 2.80 2.60 2.43 + 108.8 91.4 63.0 64.7 70.6 61.8 45.5 34.7 23.7 12.4 5.0 + %Complete 96.2 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 99.0 72.7 + 2.58 5.06 3.97 2.92 2.36 1.68 1.50 1.33 1.48 1.38 1.79 +``` + +**SHELXD will separate space groups the merged intensities could not.** That sweep's report named a +body-centred cubic pair as indistinguishable, so SHELXC and SHELXD were run once per candidate — +same reflections, same `FIND`, only `SPAG` different. One gave `CC 37.93 / CC(weak) 14.05 / +CFOM 51.98` and the other `CC 46.76 / CC(weak) 22.61 / CFOM 69.37`. The substructure is where the +screw axis shows itself, and the second group is the right one. This is the same handover as +Phaser's arrived at from the other side, and it is worth doing whenever `SPACE_GROUP_ALTERNATIVES` +is not `NONE` — SHELXD takes seconds, and the pair of runs costs less than reprocessing anything. + +**SHELXE decides the hand**, and says so. Run it twice, `-i` inverting the substructure. `-s` is the +solvent fraction, `-h` says the substructure atoms belong to the native structure, as sulfur does, +and `-a` turns on autotracing, which is what actually makes the two hands separate. The two runs +write `sad.pdb` and `sad_i.pdb`, so they can share a directory: + +``` +shelxe sad sad_fa -h -s0.62 -m20 -a15 -q +shelxe sad sad_fa -h -s0.62 -m20 -a15 -q -i +``` + +At 63 % solvent the two hands came out at 42.93 % and 15.28 % for the autotrace CC against the +native data — pseudo-free CC 66.49 against 37.12, map contrast 0.87 against 0.44, 215 traced atoms +— which is a solved structure, from `myrun.hkl` and nothing else. Where the group is one of the 22 +that come in enantiomorphic pairs, SHELXE makes the group change itself: the inverted run prints +`** Space group converted to enantiomorph **` and writes the changed group into the `CRYST1` of its +traced model, so the answer is readable off the output file the same way it is off Phaser's. + +**A negative result, for calibration.** A tetragonal dataset at the same wavelength with the same +kind of substructure, but 87 % complete at multiplicity 20 rather than 95 % at 30, gave a plausible +SHELXD `CFOM 47.62` and then failed at the hand: 15.33 % against 15.60 % autotrace CC, map contrast +0.33 either way. That is not a discrimination and it is not a solution. Nothing about the file was +the limit — the anomalous signal SHELXC measured on it was real, `` reaching 4.2 — so the +reading is that sulfur phasing wants the completeness and the multiplicity, and a `.hkl` from a +sweep that does not have them will get this far and no further. ## Comparing the geometry with XDS diff --git a/image_analysis/WriteReflections.cpp b/image_analysis/WriteReflections.cpp index c1de19809..8332288e4 100644 --- a/image_analysis/WriteReflections.cpp +++ b/image_analysis/WriteReflections.cpp @@ -451,7 +451,11 @@ void WriteMtzReflections(const std::vector &reflections, mtz.title = "Rugnux merged reflections"; mtz.history.push_back("From Rugnux " + jfjoch_version() + ", data reduction"); - // Add dataset + // Add dataset. HKL_base first, so the data dataset is id 1: a dataset written as id 0 is the + // reserved base dataset as far as CCP4's mtzlib is concerned, and the wavelength on it is + // dropped - mtzinfo then reports the Cu K-alpha default 1.54187 A, which is what a program that + // takes f'/f'' from the file would phase on. + mtz.add_base(); // the HKL_base dataset and the H K L columns gemmi::Mtz::Dataset& ds = mtz.add_dataset("native"); ds.crystal_name = experiment.GetSampleName(); ds.wavelength = experiment.GetWavelength_A(); @@ -463,9 +467,6 @@ void WriteMtzReflections(const std::vector &reflections, bool has_anom = true; const std::vector out_rows = BuildMergedRows(reflections, experiment, has_anom); - mtz.add_column("H", 'H', dataset_id, -1, false); - mtz.add_column("K", 'H', dataset_id, -1, false); - mtz.add_column("L", 'H', dataset_id, -1, false); mtz.add_column("IMEAN", 'J', dataset_id, -1, false); mtz.add_column("SIGIMEAN", 'Q', dataset_id, -1, false); if (has_anom) {