Merge docs-pipeline-fig into rc175: rugnux pipeline figures for rotation and stills in the overview docs, one generator

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-15 21:11:28 +02:00
commit 044b5813ed
4 files changed
+480

No files matched your search

+4
View File
@@ -4,6 +4,8 @@ The map of a run, in the order it happens — one paragraph per stage, each link
[data-analysis reference](CPU_DATA_ANALYSIS.md) where the algorithm lives. The walk-through is a
rotation run with the defaults; stills differences are at the end.
![The rotation pipeline: pass 1 integrates half the sweep at the file's geometry, pass 2 the whole sweep at the refined one, and the better of the two goes on to the space group and the written merge](images/rugnux_pipeline.svg)
**Open the dataset.** The geometry, wavelength and goniometer come from the file
([What Rugnux reads](RUGNUX_FORMATS.md)). A goniometer axis makes it a rotation run, none makes it
serial stills — nothing is asked of the user.
@@ -74,3 +76,5 @@ subgroup ([§14](CPU_DATA_ANALYSIS_DECISIONS.md)).
orientation-tilt post-refinement rather than a rocking curve, and a merohedral indexing ambiguity
has to be broken per image, at integration time, against a reference or a model
([Advanced ▸ the indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity)).
![The stills pipeline: every image indexed on its own, the geometry refined first where the cell is known, and the crystals scaled and merged together](images/rugnux_pipeline_stills.svg)
+136
View File
@@ -0,0 +1,136 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1600" height="990" viewBox="0 0 1600 990" font-family="Helvetica, Arial, sans-serif">
<defs><marker id="arr" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto" markerUnits="userSpaceOnUse"><path d="M0,0 L10,5 L0,10 z" fill="#555"/></marker></defs>
<rect width="1600" height="990" fill="white"/>
<rect x="300" y="300" width="790" height="360" rx="18" fill="#EEF3FC" stroke="#B9CBEF" stroke-width="2"/>
<text x="360" y="332" font-size="21" font-weight="bold" fill="#2F6FDE">Per image<tspan dx="8" font-weight="normal" font-size="16" fill="#6B6B6B">(independent)</tspan></text>
<rect x="360" y="350" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="470.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Decode</text>
<text x="470.0" y="408" text-anchor="middle" font-size="15" fill="#222222">bitshuffle/LZ4, CBF</text>
<rect x="518" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="544.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="610" y="350" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="720.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Azimuthal profile</text>
<text x="720.0" y="408" text-anchor="middle" font-size="15" fill="#222222">noise, ice rings</text>
<rect x="768" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="794.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="860" y="350" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="970.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Spot finding</text>
<text x="970.0" y="408" text-anchor="middle" font-size="15" fill="#222222">self-calibrating threshold</text>
<rect x="1018" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1044.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="360" y="520" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="470.0" y="552" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Refine</text>
<text x="470.0" y="578" text-anchor="middle" font-size="15" fill="#222222">beam centre, orientation</text>
<rect x="502" y="597" width="68" height="20" rx="4" fill="#B7791F"/>
<text x="536.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">autodiff</text>
<rect x="610" y="520" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="720.0" y="552" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Predict</text>
<text x="720.0" y="578" text-anchor="middle" font-size="15" fill="#222222">Ewald crossing, partiality</text>
<rect x="768" y="597" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="794.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="860" y="520" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="970.0" y="552" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Integrate</text>
<text x="970.0" y="578" text-anchor="middle" font-size="15" fill="#222222">2D profile fit</text>
<rect x="1018" y="597" width="52" height="20" rx="4" fill="#555555"/>
<text x="1044.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Metal</text>
<rect x="958" y="597" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="984.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<path d="M580,402 L608,402" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M830,402 L858,402" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M580,572 L608,572" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M830,572 L858,572" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M950,625 L950,638 L322,638 L322,402 L358,402" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<rect x="60" y="150" width="220" height="80" rx="12" ry="12" fill="white" stroke="#222222" stroke-width="3"/>
<text x="170.0" y="182" text-anchor="middle" font-size="20" font-weight="bold" fill="#222222">Input</text>
<text x="170.0" y="208" text-anchor="middle" font-size="14" fill="#222222">live, HDF5, CBF, SMV, marCCD</text>
<rect x="560" y="120" width="240" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="680.0" y="152" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Pre-scan</text>
<text x="680.0" y="178" text-anchor="middle" font-size="15" fill="#222222">beam-stop shadow, beam centre,</text>
<text x="680.0" y="197" text-anchor="middle" font-size="15" fill="#222222">spot widths</text>
<rect x="728" y="219" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="759.0" y="234" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="668" y="219" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="694.0" y="234" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<path d="M280,190 L558,190" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="420.0" y="182" text-anchor="middle" font-size="13" fill="#6B6B6B" font-style="italic">frame sample</text>
<path d="M170.0,230 L170.0,270 L470,270 L470,298" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="334" y="262" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">every frame</text>
<path d="M680,247 L680,298" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="692" y="275" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">masks, spot widths</text>
<rect x="1200" y="350" width="250" height="105" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="1325.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Index</text>
<text x="1325.0" y="408" text-anchor="middle" font-size="15" fill="#222222">frame sample, 1D FFT</text>
<rect x="1378" y="427" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1409.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="1302" y="427" width="68" height="20" rx="4" fill="#B7791F"/>
<text x="1336.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">autodiff</text>
<rect x="1242" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1268.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1200" y="520" width="250" height="105" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="1325.0" y="552" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Post-refine</text>
<text x="1325.0" y="578" text-anchor="middle" font-size="15" fill="#222222">distance, beam, cell, axis</text>
<rect x="1378" y="597" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1409.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="1302" y="597" width="68" height="20" rx="4" fill="#B7791F"/>
<text x="1336.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">autodiff</text>
<path d="M800,180 L1325.0,180 L1325.0,348" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="1062.5" y="172" text-anchor="middle" font-size="13" fill="#6B6B6B" font-style="italic">beam centre</text>
<path d="M1080,390 L1198,390" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="1140.0" y="382" text-anchor="middle" font-size="13" fill="#6B6B6B" font-style="italic">spots</text>
<path d="M1240,455 L1240,480 L470,480 L470,518" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="1252" y="472" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">lattice</text>
<path d="M970,455 L970,478" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1080,572 L1198,572" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="1140.0" y="564" text-anchor="middle" font-size="13" fill="#6B6B6B" font-style="italic">pass 1</text>
<path d="M1450,572 L1490,572 L1490,80 L322,80 L322,400" fill="none" stroke="#555" stroke-width="2.2" stroke-dasharray="8,6" marker-end="url(#arr)"/>
<text x="900" y="72" text-anchor="middle" font-size="14" fill="#6B6B6B" font-style="italic">pass 2, repeated while it gains</text>
<text x="542" y="740" font-size="21" font-weight="bold" fill="#1A8A7D">Whole dataset</text>
<rect x="300" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="415.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Scale &amp; merge (P1)</text>
<text x="415.0" y="828" text-anchor="middle" font-size="15" fill="#222222">3D combine of partials</text>
<rect x="468" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="494.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="555" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="670.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Symmetry</text>
<text x="670.0" y="828" text-anchor="middle" font-size="14" fill="#222222">space group, twinning, tNCS</text>
<rect x="713" y="869" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="744.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="810" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="925.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Scale &amp; merge</text>
<text x="925.0" y="828" text-anchor="middle" font-size="14" fill="#222222">in space group, resolution cut</text>
<text x="925.0" y="847" text-anchor="middle" font-size="14" fill="#222222">decay, absorption, modulation</text>
<rect x="968" y="869" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="999.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="908" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="934.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1065" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#8E44AD" stroke-width="3" stroke-dasharray="9,6"/>
<text x="1180.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#8E44AD">Structure &amp; map</text>
<text x="1180.0" y="828" text-anchor="middle" font-size="14" fill="#222222">placement or MR, σA maps</text>
<rect x="1223" y="869" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1254.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="1163" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1189.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1320" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#E8740C" stroke-width="3"/>
<text x="1435.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#E8740C">Report</text>
<text x="1435.0" y="828" text-anchor="middle" font-size="15" fill="#222222">MTZ, mmCIF, maps</text>
<path d="M530,822 L553,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M785,822 L808,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1040,822 L1063,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1295,822 L1318,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1000,770 L1000,744 L1360,744 L1360,768" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="1180.0" y="736" text-anchor="middle" font-size="13" fill="#6B6B6B" font-style="italic">without a model</text>
<path d="M1020,625 L1020,710 L420,710 L420,768" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="435" y="702" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">pass 2, or pass 1 if it merges better</text>
<rect x="60" y="936" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="86.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<text x="124" y="951" font-size="15" fill="#222222">CUDA kernel when a GPU is present</text>
<rect x="390" y="936" width="52" height="20" rx="4" fill="#555555"/>
<text x="416.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Metal</text>
<text x="454" y="951" font-size="15" fill="#222222">also an Apple Metal kernel</text>
<rect x="670" y="936" width="68" height="20" rx="4" fill="#B7791F"/>
<text x="704.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">autodiff</text>
<text x="750" y="951" font-size="15" fill="#222222">derivatives by automatic differentiation</text>
<rect x="1040" y="936" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1071.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<text x="1114" y="951" font-size="15" fill="#222222">accepted only after a test against the data</text>
</svg>

After

Width:  |  Height:  |  Size: 14 KiB

+220
View File
@@ -0,0 +1,220 @@
#!/usr/bin/env python3
"""Extended Data Fig. 1 — Rugnux pipeline, per-image loop vs whole-dataset steps (minimal-text version)."""
# Two figures in one style: the rotation path (rugnux_pipeline.svg) and the stills path
# (rugnux_pipeline_stills.svg), both written next to this script.
import os
W, H = 1600, 990
BLUE, TEAL, PURPLE, ORANGE, INK = "#2F6FDE", "#1A8A7D", "#8E44AD", "#E8740C", "#222222"
PANEL_FILL, PANEL_STROKE = "#EEF3FC", "#B9CBEF"
GPU_BG, TEST_BG, METAL_BG, AD_BG = "#2F6FDE", "#1DB38B", "#555555", "#B7791F"
GREY = "#6B6B6B"
out = []
def a(s): out.append(s)
def start():
out.clear()
a(f'<svg xmlns="http://www.w3.org/2000/svg" width="{W}" height="{H}" viewBox="0 0 {W} {H}" '
f'font-family="Helvetica, Arial, sans-serif">')
a('<defs>'
'<marker id="arr" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto" markerUnits="userSpaceOnUse">'
'<path d="M0,0 L10,5 L0,10 z" fill="#555"/></marker>'
'</defs>')
a(f'<rect width="{W}" height="{H}" fill="white"/>')
def finish(name):
a('</svg>')
open(os.path.join(os.path.dirname(os.path.abspath(__file__)), name), "w").write("\n".join(out))
def badge(x, y, w, bg, text):
a(f'<rect x="{x}" y="{y}" width="{w}" height="20" rx="4" fill="{bg}"/>')
a(f'<text x="{x+w/2}" y="{y+15}" text-anchor="middle" font-size="13" font-weight="bold" fill="white">{text}</text>')
BADGES = [("tested", 62, TEST_BG), ("autodiff", 68, AD_BG), ("Metal", 52, METAL_BG), ("CUDA", 52, GPU_BG)]
def box(x, y, w, h, title, body, color, gpu=False, tested=False, metal=False, autodiff=False, dashed=False, ts=20, bs=15):
dash = ' stroke-dasharray="9,6"' if dashed else ''
a(f'<rect x="{x}" y="{y}" width="{w}" height="{h}" rx="12" ry="12" fill="white" stroke="{color}" stroke-width="3"{dash}/>')
a(f'<text x="{x+w/2}" y="{y+32}" text-anchor="middle" font-size="{ts}" font-weight="bold" fill="{color}">{title}</text>')
if body:
lines = body if isinstance(body, list) else [body]
for i, ln in enumerate(lines):
a(f'<text x="{x+w/2}" y="{y+58+i*19}" text-anchor="middle" font-size="{bs}" fill="{INK}">{ln}</text>')
bx = x + w - 10
for (text, bw, bg), on in zip(BADGES, (tested, autodiff, metal, gpu)):
if on:
bx -= bw
badge(bx, y+h-28, bw, bg, text)
bx -= 8
def path(d, dashed=False):
dash = ' stroke-dasharray="8,6"' if dashed else ''
a(f'<path d="{d}" fill="none" stroke="#555" stroke-width="2.2"{dash} marker-end="url(#arr)"/>')
def label(x, y, text, anchor="middle", size=14):
a(f'<text x="{x}" y="{y}" text-anchor="{anchor}" font-size="{size}" fill="{GREY}" font-style="italic">{text}</text>')
def panel(x0, x1, y0, y1, tx):
a(f'<rect x="{x0}" y="{y0}" width="{x1-x0}" height="{y1-y0}" rx="18" fill="{PANEL_FILL}" stroke="{PANEL_STROKE}" stroke-width="2"/>')
a(f'<text x="{tx}" y="{y0+32}" font-size="21" font-weight="bold" fill="{BLUE}">Per image<tspan dx="8" font-weight="normal" font-size="16" fill="{GREY}">(independent)</tspan></text>')
def legend():
LY = H-40
lx = 60
for text, bw, bg, desc, step in [("CUDA", 52, GPU_BG, "CUDA kernel when a GPU is present", 330),
("Metal", 52, METAL_BG, "also an Apple Metal kernel", 280),
("autodiff", 68, AD_BG, "derivatives by automatic differentiation", 370),
("tested", 62, TEST_BG, "accepted only after a test against the data", 0)]:
badge(lx, LY-14, bw, bg, text)
a(f'<text x="{lx+bw+12}" y="{LY+1}" font-size="15" fill="{INK}">{desc}</text>')
lx += step
# ---------- geometry
BW, BH = 220, 105
PY0, PY1 = 300, 660 # loop panel
R1Y, R2Y = 350, 520 # rows inside panel
LX = 322 # loop-return / pass-2 column
IX, IY, IW, IH = 60, 150, 220, 80 # input
SY = 120 # pre-scan row
QY, QBW, QBH = 770, 230, BH + 22 # whole-dataset row
QX = [300, 555, 810, 1065, 1320]
OUTY = 710
def dataset_row(merge_p1, symmetry, merge, gpu):
a(f'<text x="{QX[0]+QBW+12}" y="{QY-30}" font-size="21" font-weight="bold" fill="{TEAL}">Whole dataset</text>')
box(QX[0], QY, QBW, QBH, "Scale &amp; merge (P1)", merge_p1, TEAL, gpu=gpu)
box(QX[1], QY, QBW, QBH, "Symmetry", symmetry, TEAL, tested=True, bs=14)
box(QX[2], QY, QBW, QBH, "Scale &amp; merge", merge, TEAL, gpu=gpu, tested=gpu, bs=14)
box(QX[3], QY, QBW, QBH, "Structure &amp; map", "placement or MR, σA maps", PURPLE, gpu=True, tested=True, dashed=True, bs=14)
box(QX[4], QY, QBW, QBH, "Report", "MTZ, mmCIF, maps", ORANGE)
for i in range(4):
path(f"M{QX[i]+QBW},{QY+52} L{QX[i+1]-2},{QY+52}")
# without a model the merge goes straight to the report
path(f"M{QX[2]+QBW-40},{QY} L{QX[2]+QBW-40},{QY-26} L{QX[4]+40},{QY-26} L{QX[4]+40},{QY-2}")
label(QX[3]+QBW/2, QY-34, "without a model", size=13)
def rotation():
start()
PX0, PX1 = 300, 1090
C = [360, 610, 860] # columns inside panel
# ---------- loop panel
panel(PX0, PX1, PY0, PY1, C[0])
box(C[0], R1Y, BW, BH, "Decode", "bitshuffle/LZ4, CBF", BLUE, gpu=True)
box(C[1], R1Y, BW, BH, "Azimuthal profile", "noise, ice rings", BLUE, gpu=True)
box(C[2], R1Y, BW, BH, "Spot finding", "self-calibrating threshold", BLUE, gpu=True)
box(C[0], R2Y, BW, BH, "Refine", "beam centre, orientation", BLUE, autodiff=True)
box(C[1], R2Y, BW, BH, "Predict", "Ewald crossing, partiality", BLUE, gpu=True)
box(C[2], R2Y, BW, BH, "Integrate", "2D profile fit", BLUE, gpu=True, metal=True)
for r in (R1Y, R2Y):
path(f"M{C[0]+BW},{r+52} L{C[1]-2},{r+52}")
path(f"M{C[1]+BW},{r+52} L{C[2]-2},{r+52}")
# loop return: Integrate bottom -> panel bottom bus -> left column -> Decode left edge
path(f"M{C[2]+90},{R2Y+BH} L{C[2]+90},{PY1-22} L{LX},{PY1-22} L{LX},{R1Y+52} L{C[0]-2},{R1Y+52}")
# ---------- Input
box(IX, IY, IW, IH, "Input", "live, HDF5, CBF, SMV, marCCD", INK, bs=14)
# ---------- Pre-scan
SX = 560
box(SX, SY, 240, BH+22, "Pre-scan", ["beam-stop shadow, beam centre,", "spot widths"], TEAL, gpu=True, tested=True)
path(f"M{IX+IW},{IY+40} L{SX-2},{IY+40}")
label((IX+IW+SX)/2, IY+32, "frame sample", size=13)
path(f"M{IX+IW/2},{IY+IH} L{IX+IW/2},{IY+IH+40} L{C[0]+110},{IY+IH+40} L{C[0]+110},{PY0-2}")
label(LX+12, IY+IH+32, "every frame", anchor="start", size=13)
path(f"M{SX+120},{SY+BH+22} L{SX+120},{PY0-2}")
label(SX+132, SY+BH+22+28, "masks, spot widths", anchor="start", size=13)
# ---------- Index and Post-refine
DX, DW = 1200, 250
box(DX, R1Y, DW, BH, "Index", "frame sample, 1D FFT", TEAL, gpu=True, autodiff=True, tested=True)
box(DX, R2Y, DW, BH, "Post-refine", "distance, beam, cell, axis", TEAL, autodiff=True, tested=True)
# the beam centre the pre-scan measured is a hypothesis: the first pass indexes at both centres
path(f"M{SX+240},{SY+60} L{DX+DW/2},{SY+60} L{DX+DW/2},{R1Y-2}")
label((SX+240+DX+DW/2)/2, SY+52, "beam centre", size=13)
path(f"M{C[2]+BW},{R1Y+40} L{DX-2},{R1Y+40}")
label((C[2]+BW+DX)/2, R1Y+32, "spots", size=13)
BUSY = 480
path(f"M{DX+40},{R1Y+BH} L{DX+40},{BUSY} L{C[0]+110},{BUSY} L{C[0]+110},{R2Y-2}")
label(DX+52, BUSY-8, "lattice", anchor="start", size=13)
path(f"M{C[2]+110},{R1Y+BH} L{C[2]+110},{BUSY-2}")
path(f"M{C[2]+BW},{R2Y+52} L{DX-2},{R2Y+52}")
label((C[2]+BW+DX)/2, R2Y+44, "pass 1", size=13)
TOPY = 80
path(f"M{DX+DW},{R2Y+52} L{DX+DW+40},{R2Y+52} L{DX+DW+40},{TOPY} L{LX},{TOPY} L{LX},{R1Y+50}", dashed=True)
label(900, TOPY-8, "pass 2, repeated while it gains", size=14)
# ---------- Whole-dataset chain
dataset_row("3D combine of partials", "space group, twinning, tNCS",
["in space group, resolution cut", "decay, absorption, modulation"], gpu=True)
path(f"M{C[2]+160},{R2Y+BH} L{C[2]+160},{OUTY} L{QX[0]+120},{OUTY} L{QX[0]+120},{QY-2}")
label(QX[0]+135, OUTY-8, "pass 2, or pass 1 if it merges better", anchor="start", size=13)
legend()
finish("rugnux_pipeline.svg")
def stills():
start()
PX0, PX1 = 300, 1340
C = [360, 610, 860, 1110] # columns inside panel
# ---------- loop panel: the first row left to right, the second back right to left
panel(PX0, PX1, PY0, PY1, C[0])
box(C[0], R1Y, BW, BH, "Decode", "bitshuffle/LZ4, CBF", BLUE, gpu=True)
box(C[1], R1Y, BW, BH, "Azimuthal profile", "noise per ring", BLUE, gpu=True)
box(C[2], R1Y, BW, BH, "Spot finding", "self-calibrating threshold", BLUE, gpu=True)
box(C[3], R1Y, BW, BH, "Index", "1D FFT, or ffbidx with a cell", BLUE, gpu=True, tested=True, bs=14)
box(C[3], R2Y, BW, BH, "Refine", "beam, orientation, cell", BLUE, autodiff=True)
box(C[2], R2Y, BW, BH, "Predict", "excitation-error cutoff", BLUE, gpu=True)
box(C[1], R2Y, BW, BH, "Integrate", "2D profile fit", BLUE, gpu=True, metal=True)
for i in range(3):
path(f"M{C[i]+BW},{R1Y+52} L{C[i+1]-2},{R1Y+52}")
path(f"M{C[3]+BW/2},{R1Y+BH} L{C[3]+BW/2},{R2Y-2}")
path(f"M{C[3]},{R2Y+52} L{C[2]+BW+2},{R2Y+52}")
path(f"M{C[2]},{R2Y+52} L{C[1]+BW+2},{R2Y+52}")
# loop return: Integrate left edge -> left column -> Decode left edge
path(f"M{C[1]},{R2Y+52} L{LX},{R2Y+52} L{LX},{R1Y+52} L{C[0]-2},{R1Y+52}")
# ---------- Input, geometry refinement, pre-scan
box(IX, IY, IW, IH, "Input", "HDF5, CBF, SMV, marCCD", INK, bs=14)
GX, SX = 560, 900
box(GX, SY, 240, BH+22, "Geometry", ["with a known cell:", "beam, distance, cell"], TEAL, autodiff=True, dashed=True)
box(SX, SY, 240, BH+22, "Pre-scan", ["beam-stop shadow,", "bad pixels"], TEAL, gpu=True)
path(f"M{IX+IW},{IY+40} L{GX-2},{IY+40}")
label((IX+IW+GX)/2, IY+32, "strong frames", size=13)
path(f"M{GX+240},{IY+40} L{SX-2},{IY+40}")
path(f"M{IX+IW/2},{IY+IH} L{IX+IW/2},{IY+IH+40} L{C[0]+110},{IY+IH+40} L{C[0]+110},{PY0-2}")
label(LX+12, IY+IH+32, "every frame", anchor="start", size=13)
path(f"M{GX+120},{SY+BH+22} L{GX+120},{PY0-2}")
label(GX+132, SY+BH+22+28, "geometry", anchor="start", size=13)
path(f"M{SX+120},{SY+BH+22} L{SX+120},{PY0-2}")
label(SX+132, SY+BH+22+28, "masks", anchor="start", size=13)
# ---------- Whole-dataset chain
dataset_row(["consensus cell,", "per-image scale"], "space group, twinning, tNCS",
["in space group, resolution cut", "partiality, error model"], gpu=False)
path(f"M{C[1]+110},{R2Y+BH} L{C[1]+110},{OUTY} L{QX[0]+120},{OUTY} L{QX[0]+120},{QY-2}")
label(QX[0]+135, OUTY-8, "every crystal", anchor="start", size=13)
legend()
finish("rugnux_pipeline_stills.svg")
rotation()
stills()
print("ok")
+120
View File
@@ -0,0 +1,120 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1600" height="990" viewBox="0 0 1600 990" font-family="Helvetica, Arial, sans-serif">
<defs><marker id="arr" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto" markerUnits="userSpaceOnUse"><path d="M0,0 L10,5 L0,10 z" fill="#555"/></marker></defs>
<rect width="1600" height="990" fill="white"/>
<rect x="300" y="300" width="1040" height="360" rx="18" fill="#EEF3FC" stroke="#B9CBEF" stroke-width="2"/>
<text x="360" y="332" font-size="21" font-weight="bold" fill="#2F6FDE">Per image<tspan dx="8" font-weight="normal" font-size="16" fill="#6B6B6B">(independent)</tspan></text>
<rect x="360" y="350" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="470.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Decode</text>
<text x="470.0" y="408" text-anchor="middle" font-size="15" fill="#222222">bitshuffle/LZ4, CBF</text>
<rect x="518" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="544.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="610" y="350" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="720.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Azimuthal profile</text>
<text x="720.0" y="408" text-anchor="middle" font-size="15" fill="#222222">noise per ring</text>
<rect x="768" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="794.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="860" y="350" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="970.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Spot finding</text>
<text x="970.0" y="408" text-anchor="middle" font-size="15" fill="#222222">self-calibrating threshold</text>
<rect x="1018" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1044.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1110" y="350" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="1220.0" y="382" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Index</text>
<text x="1220.0" y="408" text-anchor="middle" font-size="14" fill="#222222">1D FFT, or ffbidx with a cell</text>
<rect x="1258" y="427" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1289.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="1198" y="427" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1224.0" y="442" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1110" y="520" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="1220.0" y="552" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Refine</text>
<text x="1220.0" y="578" text-anchor="middle" font-size="15" fill="#222222">beam, orientation, cell</text>
<rect x="1252" y="597" width="68" height="20" rx="4" fill="#B7791F"/>
<text x="1286.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">autodiff</text>
<rect x="860" y="520" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="970.0" y="552" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Predict</text>
<text x="970.0" y="578" text-anchor="middle" font-size="15" fill="#222222">excitation-error cutoff</text>
<rect x="1018" y="597" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1044.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="610" y="520" width="220" height="105" rx="12" ry="12" fill="white" stroke="#2F6FDE" stroke-width="3"/>
<text x="720.0" y="552" text-anchor="middle" font-size="20" font-weight="bold" fill="#2F6FDE">Integrate</text>
<text x="720.0" y="578" text-anchor="middle" font-size="15" fill="#222222">2D profile fit</text>
<rect x="768" y="597" width="52" height="20" rx="4" fill="#555555"/>
<text x="794.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Metal</text>
<rect x="708" y="597" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="734.0" y="612" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<path d="M580,402 L608,402" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M830,402 L858,402" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1080,402 L1108,402" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1220.0,455 L1220.0,518" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1110,572 L1082,572" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M860,572 L832,572" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M610,572 L322,572 L322,402 L358,402" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<rect x="60" y="150" width="220" height="80" rx="12" ry="12" fill="white" stroke="#222222" stroke-width="3"/>
<text x="170.0" y="182" text-anchor="middle" font-size="20" font-weight="bold" fill="#222222">Input</text>
<text x="170.0" y="208" text-anchor="middle" font-size="14" fill="#222222">HDF5, CBF, SMV, marCCD</text>
<rect x="560" y="120" width="240" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3" stroke-dasharray="9,6"/>
<text x="680.0" y="152" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Geometry</text>
<text x="680.0" y="178" text-anchor="middle" font-size="15" fill="#222222">with a known cell:</text>
<text x="680.0" y="197" text-anchor="middle" font-size="15" fill="#222222">beam, distance, cell</text>
<rect x="722" y="219" width="68" height="20" rx="4" fill="#B7791F"/>
<text x="756.0" y="234" text-anchor="middle" font-size="13" font-weight="bold" fill="white">autodiff</text>
<rect x="900" y="120" width="240" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="1020.0" y="152" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Pre-scan</text>
<text x="1020.0" y="178" text-anchor="middle" font-size="15" fill="#222222">beam-stop shadow,</text>
<text x="1020.0" y="197" text-anchor="middle" font-size="15" fill="#222222">bad pixels</text>
<rect x="1078" y="219" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1104.0" y="234" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<path d="M280,190 L558,190" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="420.0" y="182" text-anchor="middle" font-size="13" fill="#6B6B6B" font-style="italic">strong frames</text>
<path d="M800,190 L898,190" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M170.0,230 L170.0,270 L470,270 L470,298" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="334" y="262" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">every frame</text>
<path d="M680,247 L680,298" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="692" y="275" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">geometry</text>
<path d="M1020,247 L1020,298" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="1032" y="275" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">masks</text>
<text x="542" y="740" font-size="21" font-weight="bold" fill="#1A8A7D">Whole dataset</text>
<rect x="300" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="415.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Scale &amp; merge (P1)</text>
<text x="415.0" y="828" text-anchor="middle" font-size="15" fill="#222222">consensus cell,</text>
<text x="415.0" y="847" text-anchor="middle" font-size="15" fill="#222222">per-image scale</text>
<rect x="555" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="670.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Symmetry</text>
<text x="670.0" y="828" text-anchor="middle" font-size="14" fill="#222222">space group, twinning, tNCS</text>
<rect x="713" y="869" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="744.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="810" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#1A8A7D" stroke-width="3"/>
<text x="925.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#1A8A7D">Scale &amp; merge</text>
<text x="925.0" y="828" text-anchor="middle" font-size="14" fill="#222222">in space group, resolution cut</text>
<text x="925.0" y="847" text-anchor="middle" font-size="14" fill="#222222">partiality, error model</text>
<rect x="1065" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#8E44AD" stroke-width="3" stroke-dasharray="9,6"/>
<text x="1180.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#8E44AD">Structure &amp; map</text>
<text x="1180.0" y="828" text-anchor="middle" font-size="14" fill="#222222">placement or MR, σA maps</text>
<rect x="1223" y="869" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1254.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<rect x="1163" y="869" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="1189.0" y="884" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<rect x="1320" y="770" width="230" height="127" rx="12" ry="12" fill="white" stroke="#E8740C" stroke-width="3"/>
<text x="1435.0" y="802" text-anchor="middle" font-size="20" font-weight="bold" fill="#E8740C">Report</text>
<text x="1435.0" y="828" text-anchor="middle" font-size="15" fill="#222222">MTZ, mmCIF, maps</text>
<path d="M530,822 L553,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M785,822 L808,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1040,822 L1063,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1295,822 L1318,822" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<path d="M1000,770 L1000,744 L1360,744 L1360,768" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="1180.0" y="736" text-anchor="middle" font-size="13" fill="#6B6B6B" font-style="italic">without a model</text>
<path d="M720,625 L720,710 L420,710 L420,768" fill="none" stroke="#555" stroke-width="2.2" marker-end="url(#arr)"/>
<text x="435" y="702" text-anchor="start" font-size="13" fill="#6B6B6B" font-style="italic">every crystal</text>
<rect x="60" y="936" width="52" height="20" rx="4" fill="#2F6FDE"/>
<text x="86.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">CUDA</text>
<text x="124" y="951" font-size="15" fill="#222222">CUDA kernel when a GPU is present</text>
<rect x="390" y="936" width="52" height="20" rx="4" fill="#555555"/>
<text x="416.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Metal</text>
<text x="454" y="951" font-size="15" fill="#222222">also an Apple Metal kernel</text>
<rect x="670" y="936" width="68" height="20" rx="4" fill="#B7791F"/>
<text x="704.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">autodiff</text>
<text x="750" y="951" font-size="15" fill="#222222">derivatives by automatic differentiation</text>
<rect x="1040" y="936" width="62" height="20" rx="4" fill="#1DB38B"/>
<text x="1071.0" y="951" text-anchor="middle" font-size="13" font-weight="bold" fill="white">tested</text>
<text x="1114" y="951" font-size="15" fill="#222222">accepted only after a test against the data</text>
</svg>

After

Width:  |  Height:  |  Size: 12 KiB