docs: rc176 - molecular replacement as a pipeline step, Metal and multi-GPU MR, header polarization, Rugnux live tab

- Pipeline figures: the structure/MR box carries the Metal badge.
- RUGNUX_OVERVIEW: a model/molecular-replacement step (triggers, outputs, GPUs).
- GPU_DATA_ANALYSIS: multi-card model validation and MR; the Apple GPU's model density and MR
  rotation function. Install/release/viewer/benchmarking/report/--no-gpu texts no longer say Bragg
  integration is the only Metal engine.
- --polarization default in RUGNUX_ADVANCED and the tutorial follows the usage text (miniCBF/d*TREK header).
- WEB_FRONTEND and JFJOCH_BROKER: the Rugnux live tab; the scan result's rotation lattice.
- battery README: a --model-check run records the REFMAC version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-18 17:38:49 +02:00
1 parent 6a3d52ffa3
commit f3bf945cca
15 files changed
+65 -31

No files matched your search

+16 -2
View File
@@ -156,7 +156,8 @@ The model's structure factors and the rigid body run on the card where there is
placed on it or on the CPU up front against one budget, 80% of the card's total memory
([§14.8](CPU_DATA_ANALYSIS_DECISIONS.md#148-rigid-body-placement-of-the-model)); the GPU and CPU
agree to rounding (float distances, cuFFT for FFTW, the crystal's symmetry composed in reciprocal
space instead of by symmetrizing the grid).
space instead of by symmetrizing the grid). On a machine with several cards, the rigid-body engines
the first card has no room for go on the others, each against its own budget.
- **Model density** (`ModelDensityGPU.cu`) - the density of one copy of the model's atoms on a grid,
as a gather over bricks of the grid, bit-identical from run to run.
@@ -194,6 +195,10 @@ space instead of by symmetrizing the grid).
What of the search goes on the card is planned once per merge on the same budget as model
validation's, before the search starts
([§14.10](CPU_DATA_ANALYSIS_DECISIONS.md#1410-molecular-replacement-placing-a-model-the-rigid-body-cannot)).
On a machine with several cards, every other card holds a copy of the molecular transform where it
fits, the rotation and translation functions are split over the cards by orientation, and the
rigid-body engines are spread over them as in model validation; the result does not depend on how
many cards there are.
The placements are then refined, scored and checked against their null with the rigid body and the
structure factors above; the likelihood scoring runs on the host. The search is described in
@@ -202,7 +207,8 @@ structure factors above; the likelihood scoring runs on the host. The search is
## Apple GPUs (Metal)
A macOS build has no CUDA, and one engine runs on the Apple GPU instead:
A macOS build has no CUDA, and three engines run on the Apple GPU instead (none under
`rugnux --no-gpu`; where the GPU fails, the CPU takes over with a warning):
- **Bragg integration** (`BraggIntegrationEngineMetal.mm`, `.metal`) - the CPU engine with its
pixel work moved to the GPU: the neighbour mask, the ownership of shared pixels, the box
@@ -213,3 +219,11 @@ A macOS build has no CUDA, and one engine runs on the Apple GPU instead:
every push; `rugnux --no-gpu` keeps it off the GPU, and so does `--background-trim`, whose trimmed
mean has no Metal kernel.
[§9.2](CPU_DATA_ANALYSIS_INTEGRATION.md#92-box-summation-seed-and-fallback)
- **Model density** (`ModelDensityMetal.mm`, `.metal`) - the density of the model's atoms on a grid,
for model validation and molecular replacement, as a gather over bricks of the grid as under CUDA;
the same grid as the CPU's **to the bit**.
[§14.1](CPU_DATA_ANALYSIS_DECISIONS.md#141-model-structure-factors)
- **Molecular-replacement rotation function** (`MolecularReplacementMetal.mm`, `.metal`) - the
rotation function's trial orientations scored from a copy of the molecular transform; equal to the
CPU's to rounding.
[§14.10](CPU_DATA_ANALYSIS_DECISIONS.md#1410-molecular-replacement-placing-a-model-the-rigid-body-cannot)
+4 -2
View File
@@ -210,8 +210,10 @@ once with that, and the live status records the cut (`deadline` or `cancel`) and
`GET /rugnux/status` reports the live run of the last collection, kept until the next one starts: its
state, the images received, the step it is on and how far into it, the steps so far with what each
found, and once it is over the verdict, the summary, warnings and the time from the last image to
every result written. The web frontend shows it on the dashboard. It needs the dataset's bearer token
like the routes above. A collection in which rugnux finds no lattice still gets the spot count, resolution
every result written. The web frontend shows it on the dashboard and in its "Rugnux live" tab, which
also changes the settings and uploads the model. It needs the dataset's bearer token
like the routes above. `GET /result/scan` carries the rotation lattice, unit cell and Bravais symbol
rugnux found. A collection in which rugnux finds no lattice still gets the spot count, resolution
estimate and background of every image in `GET /result/scan` and the master file, marked as not indexed.
## Setting up a local test for Jungfraujoch
+2 -2
View File
@@ -178,8 +178,8 @@ they run on — a V100 needs the CUDA 12 `.tgz`. See
[Release contents ▸ GPU generations and the NVIDIA driver](RELEASE_CONTENTS.md#gpu-generations-and-the-nvidia-driver).
On a Mac there is no CUDA: the viewer runs the same pipeline on the CPU, with the FFTW indexer, like
the non-CUDA build elsewhere, except Bragg integration, which runs on the Apple GPU through Metal with
the same result as the CPU engine to the bit.
the non-CUDA build elsewhere, except Bragg integration, the model density and the molecular-replacement
rotation function, which run on the Apple GPU through Metal.
### Remote displays
+2 -2
View File
@@ -162,8 +162,8 @@ To build the viewer yourself on Windows, see
The macOS artefacts are the `jfjoch_viewer` disk image and the `rugnux` `.tgz`; as on Windows, the
rest of Jungfraujoch, including the XDS plugin, is Linux-only. Both are **without CUDA** — macOS has none, so indexing uses the FFTW indexer and the pipeline runs
on the CPU, except Bragg integration, which runs on the Apple GPU through Metal with the same result
as on the CPU to the bit — and both are built for **Apple Silicon only**.
on the CPU, except Bragg integration, the model density and the molecular-replacement rotation
function, which run on the Apple GPU through Metal — and both are built for **Apple Silicon only**.
They need **macOS 13 (Ventura) or newer**.
**Intel Macs are not supported.** No Intel (`x86_64`) code is built for macOS, and Rosetta cannot
+2 -2
View File
@@ -330,7 +330,7 @@ General:
| `-e, --end-image <num>` | Last image to process (default: all) |
| `-t, --stride <num>` | Process every *n*-th image (default: 1) |
| `-v, --verbose` | Show the whole log on the terminal in place of the steps; `<prefix>_log.txt` is written either way |
| `--no-gpu` | Run on the CPU only, as on a machine without a GPU, even where one is present: CUDA is not touched (nor, on macOS, the Apple GPU that Bragg integration otherwise runs on) and every GPU step takes its CPU path (`-X Auto` and `-X FFT` index with FFTW; `-X cuFFT` and `-X FFBIDX` are refused) |
| `--no-gpu` | Run on the CPU only, as on a machine without a GPU, even where one is present: CUDA is not touched (nor, on macOS, the Apple GPU that Bragg integration, the model density and the molecular-replacement rotation function otherwise run on) and every GPU step takes its CPU path (`-X Auto` and `-X FFT` index with FFTW; `-X cuFFT` and `-X FFBIDX` are refused) |
Mode — `--mode <name>` (default `mx`):
@@ -497,5 +497,5 @@ Geometry overrides (defaults are taken from the input file; override them to rep
| `--rot3 <num>` | PONI detector rotation 3, about the beam (rad) — the one `rugnux --mode calibration` writes into its `.poni`, so a refined third rotation can be given straight back |
| `--detector-mirror-y` | The stored image is mirrored in Y relative to the frame the PONI angles are stated in |
| `--detector-quarter-turns <0-3>` | The stored image is turned by this many multiples of 90° about the beam relative to that same frame |
| `--polarization <num>` | Degree of polarization *p* (XDS `FRACTION_OF_POLARIZATION` = (1+*p*)/2), signed: positive for a horizontally polarized source, negative for a vertically polarized one. Default 0.99, an undulator value; bending-magnet and wiggler sources are typically lower (0.8–0.95), and no format Rugnux reads states it reliably, so pass the beamline's known value here |
| `--polarization <num>` | Degree of polarization *p* (XDS `FRACTION_OF_POLARIZATION` = (1+*p*)/2), signed: positive for a horizontally polarized source, negative for a vertically polarized one. Default: what a miniCBF `# Polarization` line or a d*TREK `SOURCE_POLARZ` states (a fraction *f*, read as *p* = 2*f* − 1, so a laboratory tube's 0.5 is unpolarized), and otherwise 0.99, an undulator value; bending-magnet and wiggler sources are typically lower (0.8–0.95), and neither ADSC SMV, marCCD nor NXmx files state it, so pass the beamline's known value here |
| `--rotation-scale <k>` | Goniometer rotation scale: the stage turned `k` times the angle stored in the file (the commanded one). Applied to both passes, and overrides the scale Rugnux fits for itself |
+4 -4
View File
@@ -512,10 +512,10 @@ power, and which runs are the first after a build.
### macOS
- The macOS build has no CUDA. Bragg integration runs on the Apple GPU through Metal, with the same
result as the CPU to the bit. Everything else runs on the CPU, so a Mac timing is a CPU-bound
timing. Molecular replacement in particular takes much longer than on a CUDA machine (on an M6 on
2026-10-16, 240 of 4nwv's 431 s).
- The macOS build has no CUDA. Bragg integration, the model density and the molecular-replacement
rotation function run on the Apple GPU through Metal. Everything else runs on the CPU, so a Mac
timing is mostly a CPU-bound timing. Molecular replacement in particular takes much longer than on
a CUDA machine, as its translation function and rigid body stay on the CPU.
- Memory is unified: CPU and GPU share it, and the OS compresses memory before it swaps. A 16 GB Mac
can run the benchmark, but its biggest set comes close to the limit, so report both measures
below.
+3 -3
View File
@@ -38,7 +38,7 @@ take the archive for your architecture from the Gitea release page:
| `rugnux-<version>-linux-x86_64-cuda12.tgz` | 64-bit Intel/AMD Linux. Built on RHEL 8, so it runs on any newer Linux |
| `rugnux-<version>-linux-aarch64-cuda13.tgz` | 64-bit Arm Linux — NVIDIA GH200 and DGX Spark. Built on Ubuntu 24.04, so it needs glibc 2.39 or newer. Cross-compiled and **not yet exercised on Arm hardware** |
| `rugnux-<version>-win64-cuda13.zip` | 64-bit Windows |
| `rugnux-<version>-macos-arm64-cpu.tgz` | macOS 13 (Ventura) or newer on Apple Silicon (M1 and newer). No CUDA on macOS: CPU, with Bragg integration on the Apple GPU (Metal). Intel Macs are not supported |
| `rugnux-<version>-macos-arm64-cpu.tgz` | macOS 13 (Ventura) or newer on Apple Silicon (M1 and newer). No CUDA on macOS: CPU, with Bragg integration and parts of molecular replacement on the Apple GPU (Metal). Intel Macs are not supported |
**The archive has no top-level directory** — it unpacks straight into `bin/` and `share/`. Always
give `tar` a destination of its own, or it will scatter those into whatever directory you are in:
@@ -101,8 +101,8 @@ answer.
On a Mac (Apple Silicon), leave the two `-march` flags out — they name an x86 level; the Apple
compiler's default target is already the M1 — and use `sysctl -n hw.ncpu` for the job count. The
build has no CUDA there and needs nothing installed beyond Xcode and CMake; its Bragg integration runs
on the Apple GPU through Metal, with the same result as the CPU engine to the bit
build has no CUDA there and needs nothing installed beyond Xcode and CMake; its Bragg integration, the model
density and the molecular-replacement rotation function run on the Apple GPU through Metal
([GPU data analysis ▸ Apple GPUs](GPU_DATA_ANALYSIS.md#apple-gpus-metal)).
## Hardware
+14 -5
View File
@@ -64,12 +64,21 @@ CC1/2-based resolution cut, the anisotropy description, French–Wilson amplitud
flags ([§10](CPU_DATA_ANALYSIS_INTEGRATION.md),
[§13.3–§13.5](CPU_DATA_ANALYSIS_DECISIONS.md)).
**Model: validation and molecular replacement.** Only with a model (`--model` or `--mr`, PDB or
mmCIF). With `--model` the model is placed as a rigid body and checked against a null; where that does
not place it, or the model is a prediction, it is placed by molecular replacement instead, and `--mr`
goes to molecular replacement straight away. The search places as many copies as the cell holds, in
every space group the merge cannot tell apart and in the twin hypothesis' subgroup; the placed model
then gives R-work/R-free and σA-weighted maps. The model never changes how the data are processed
([Model and reference](RUGNUX_MODEL_AND_REFERENCE.md),
[§14](CPU_DATA_ANALYSIS_DECISIONS.md)). It runs on every GPU of the machine, and on a Mac its
model density and rotation function on the Apple GPU ([GPU data analysis](GPU_DATA_ANALYSIS.md#molecular-replacement)).
**Write.** The merged `.mtz` / `.cif` / `.hkl`, the unmerged MTZ and the
results report land next to each other ([Output files](RUGNUX_TUTORIAL.md#output-files)); with
`--model`, validation runs first - the model placed as a rigid body, or by molecular replacement
where that does not place it or the model is a prediction - and the maps and the placed model are
written too, with `<prefix>_mr.mtz` where the model chooses another group or the twin hypothesis'
subgroup ([§14](CPU_DATA_ANALYSIS_DECISIONS.md)).
results report land next to each other ([Output files](RUGNUX_TUTORIAL.md#output-files)); with a
model, the maps and the placed model are written too, and where molecular replacement chooses another
space group or the twin hypothesis' subgroup, that merge is written beside the answer as
`<prefix>_mr.mtz`/`.cif` with the model placed there, `<prefix>_mr_model.pdb`.
**Stills instead.** Serial data skip the two-pass machinery: each image is indexed independently
(with the known-cell `ffbidx` indexer where a cell is given), partiality comes from a per-crystal
+1 -1
View File
@@ -105,7 +105,7 @@ The header block above section 1 records **how the result was produced**: `RUGNU
`GPU_COUNT= 0` appears on its own, with no `GPU=` line, when nothing was visible — which is the
first thing to check when a run took far longer than expected. Rugnux prints the GPU at startup
(`Compute`), before the run, so a missing GPU can be caught while there is still time to stop.
On macOS, where only the Bragg integration uses the Apple GPU (Metal), the report has
On macOS, where the Apple GPU (Metal) is used only for Bragg integration and parts of model validation and molecular replacement, the report has
`GPU_COUNT= 0` and the startup line and the log name the Apple GPU instead.
Rates, per-image costs and progress remain on stdout only.
+3 -3
View File
@@ -224,9 +224,9 @@ What differs from a protein is handled by the run itself:
which of these happened (see [The results report](RUGNUX_REPORT.md#the-space-group-and-the-sohncke-answer-beside-it)).
Structure solution and refinement settle it.
Two settings are worth a look before running. The default polarization (`--polarization 0.99`) is an
undulator's; a laboratory source is close to unpolarized, and no file format states it, so give the
instrument's value. And the written reflections stop where CC1/2 falls off (the default
Two settings are worth a look before running. The default polarization is what a miniCBF or d*TREK header
states, and otherwise `--polarization 0.99`, an undulator's; a laboratory source is close to
unpolarized, and neither ADSC SMV, marCCD nor NXmx files state it, so give the instrument's value there. And the written reflections stop where CC1/2 falls off (the default
`--resolution-cutoff cc-logistic`, the same cut for every file, `.hkl` included); `--resolution-cutoff
off` keeps everything to the edge of the detector, for a refinement that weights the weak data itself.
+3
View File
@@ -5,6 +5,9 @@ Jungfraujoch is equipped with a React-based web frontend for a user-friendly exp
* Showing live view images from the detector
* JUNGFRAU calibration numbers
* Configuring the detector, as well as pedestal/initialization operations
* Rugnux live mode: the "Rugnux live" tab switches it on, sets its CPU threads and deadline, uploads the
model, and follows the run's steps; the dashboard shows the live run in a row of its own
(see [Rugnux live mode](JFJOCH_BROKER.md#rugnux-live-mode))
When the current dataset was started with `tokens`, the plots, the live preview and the
measurement statistics need one of them: the key button in the top bar takes it (kept in the tab's
+4 -2
View File
@@ -108,8 +108,10 @@
<text x="1180.0" y="828" text-anchor="middle" font-size="14" fill="#222222">placement or MR, σA maps</text>
<rect x="1223" y="869" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1254.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="1163" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1189.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1163" y="869" width="52" height="20" rx="4" fill="#555555"/>
<text x="1189.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Metal</text>
<rect x="1103" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1129.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1320" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#E8740C" stroke-width="3"/>
<text x="1435.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#E8740C">Report</text>
<text x="1435.0" y="828" text-anchor="middle" font-size="15" fill="#222222">MTZ, mmCIF, maps</text>

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 14 KiB

+1 -1
View File
@@ -88,7 +88,7 @@ def dataset_row(merge_p1, symmetry, merge, gpu):
box(QX[0], QY, QBW, QBH, "Scale &amp; merge (P1)", merge_p1, TEAL, gpu=gpu)
box(QX[1], QY, QBW, QBH, "Symmetry", symmetry, TEAL, tested=True, bs=14)
box(QX[2], QY, QBW, QBH, "Scale &amp; merge", merge, TEAL, gpu=gpu, tested=gpu, bs=14)
box(QX[3], QY, QBW, QBH, "Structure &amp; map", "placement or MR, σA maps", PURPLE, gpu=True, tested=True, dashed=True, bs=14)
box(QX[3], QY, QBW, QBH, "Structure &amp; map", "placement or MR, σA maps", PURPLE, gpu=True, metal=True, tested=True, dashed=True, bs=14)
box(QX[4], QY, QBW, QBH, "Report", "MTZ, mmCIF, maps", ORANGE)
for i in range(4):
path(f"M{QX[i]+QBW},{QY+52} L{QX[i+1]-2},{QY+52}")
+4 -2
View File
@@ -91,8 +91,10 @@
<text x="1180.0" y="828" text-anchor="middle" font-size="14" fill="#222222">placement or MR, σA maps</text>
<rect x="1223" y="869" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1254.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="1163" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1189.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1163" y="869" width="52" height="20" rx="4" fill="#555555"/>
<text x="1189.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Metal</text>
<rect x="1103" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1129.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1320" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#E8740C" stroke-width="3"/>
<text x="1435.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#E8740C">Report</text>
<text x="1435.0" y="828" text-anchor="middle" font-size="15" fill="#222222">MTZ, mmCIF, maps</text>

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

+2
View File
@@ -555,6 +555,8 @@ set (`refmac_reason` says why). The runner records `refmac_rfree`, `refmac_rwork
`refmac_untwinned`, `refmac_status` and `refmac_reason`. Small-molecule sets are skipped.
`model_check.py <p.mtz> <pdb id> --workdir DIR` runs it on one merge and prints every field
(setting, reindexing operators of both data sets, what the depositor's amplitudes were made from).
A `--model-check` run records the REFMAC version in its manifest (`refmac`), as does every recheck,
and the report lists it.
**Refined, shared free set (second protocol).** The first-cycle numbers above score the model as
deposited at the common resolution limit. The second protocol refines the same deposited model