Files
Jungfraujoch/CPU_DATA_ANALYSIS_INDEXING.html
T
2026-10-06 20:38:39 +00:00

1 line
90 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html> <html lang=en data-content_root="./"> <meta charset=utf-8 /> <meta name=viewport content="width=device-width, initial-scale=1.0" /><meta name=viewport content="width=device-width, initial-scale=1" /> <meta name=viewport content="width=device-width,initial-scale=1"> <meta http-equiv=x-ua-compatible content="ie=edge"> <meta name="lang:clipboard.copy" content="Copy to clipboard"> <meta name="lang:clipboard.copied" content="Copied to clipboard"> <meta name="lang:search.language" content=en > <meta name="lang:search.pipeline.stopwords" content=True > <meta name="lang:search.pipeline.trimmer" content=True > <meta name="lang:search.result.none" content="No matching documents"> <meta name="lang:search.result.one" content="1 matching document"> <meta name="lang:search.result.other" content="# matching documents"> <meta name="lang:search.tokenizer" content="[\s\-]+"> <link href="https://fonts.gstatic.com/" rel=preconnect crossorigin> <link href="https://fonts.googleapis.com/css?family=Roboto+Mono:400,500,700|Roboto:300,400,400i,700&display=fallback" rel=stylesheet > <style> body, input { font-family: "Roboto", "Helvetica Neue", Helvetica, Arial, sans-serif } code, kbd, pre { font-family: "Roboto Mono", "Courier New", Courier, monospace } </style> <link rel=stylesheet href="_static/stylesheets/application.css"/> <link rel=stylesheet href="_static/stylesheets/application-palette.css"/> <link rel=stylesheet href="_static/stylesheets/application-fixes.css"/> <link rel=stylesheet href="_static/fonts/material-icons.css"/> <meta name=theme-color content="#3f51b5"> <script src="_static/javascripts/modernizr.js"></script> <title>Data analysis: indexing and geometry (§4–§7) &#8212; Jungfraujoch 1.0.0-rc.174 documentation</title> <link rel=stylesheet type="text/css" href="_static/pygments.css?v=83e35b93" /> <link rel=stylesheet type="text/css" href="_static/material.css?v=79c92029" /> <script src="_static/documentation_options.js?v=3dccfb20"></script> <script src="_static/doctools.js?v=9bcbadda"></script> <script src="_static/sphinx_highlight.js?v=dc90522c"></script> <script>window.MathJax = {"tex": {"tags": "ams", "inlineMath": [["\\(", "\\)"], ["$", "$"]], "displayMath": [["\\[", "\\]"], ["$$", "$$"]], "processEscapes": true}, "options": {"processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script> <script defer=defer src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script> <link rel=icon href="_static/jfjoch.png"/> <link rel=index title=Index href=genindex.html /> <link rel=search title=Search href=search.html /> <link rel=next title="Data analysis: integration, scaling and merging (§8–§12)" href=CPU_DATA_ANALYSIS_INTEGRATION.html /> <link rel=prev title="Data analysis: from images to spots (§0–§3)" href=CPU_DATA_ANALYSIS_IMAGE.html /> <body dir=ltr data-md-color-primary=indigo data-md-color-accent=lime> <svg class=md-svg > <defs data-children-count=0 > <svg xmlns="http://www.w3.org/2000/svg" width=500 height=500 viewBox="0 0 500 500" id=__gitlab ><path fill=currentColor d="M93.667 473.347l90.684-279.097H2.983l90.684 279.097z" transform="translate(156.198 1.16)"/><path fill=currentColor d="M221.333 473.345L130.649 194.25H3.557l217.776 279.095z" transform="translate(28.531 1.16)" opacity=.7 /><path fill=currentColor d="M32 195.155L4.441 279.97a18.773 18.773 0 0 0 6.821 20.99l238.514 173.29L32 195.155z" transform="translate(.089 .256)" opacity=.5 /><path fill=currentColor d="M2.667-84.844h127.092L75.14-252.942c-2.811-8.649-15.047-8.649-17.856 0L2.667-84.844z" transform="translate(29.422 280.256)"/><path fill=currentColor d="M2.667 473.345L93.351 194.25h127.092L2.667 473.345z" transform="translate(247.198 1.16)" opacity=.7 /><path fill=currentColor d="M221.334 195.155l27.559 84.815a18.772 18.772 0 0 1-6.821 20.99L3.557 474.25l217.777-279.095z" transform="translate(246.307 .256)" opacity=.5 /><path fill=currentColor d="M130.667-84.844H3.575l54.618-168.098c2.811-8.649 15.047-8.649 17.856 0l54.618 168.098z" transform="translate(336.974 280.256)"/></svg> </defs> </svg> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer > <input class=md-toggle data-md-toggle=search type=checkbox id=__search > <label class=md-overlay data-md-component=overlay for=__drawer ></label> <a href="#CPU_DATA_ANALYSIS_INDEXING" tabindex=1 class=md-skip > Skip to content </a> <header class=md-header data-md-component=header > <nav class="md-header-nav md-grid"> <div class="md-flex navheader"> <div class="md-flex__cell md-flex__cell--shrink"> <a href=index.html title="Jungfraujoch 1.0.0-rc.174 documentation" class="md-header-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--menu md-header-nav__button" for=__drawer ></label> </div> <div class="md-flex__cell md-flex__cell--stretch"> <div class="md-flex__ellipsis md-header-nav__title" data-md-component=title > <span class=md-header-nav__topic >PSI Jungfraujoch</span> <span class=md-header-nav__topic > Data analysis: indexing and geometry (§4–§7) </span> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--search md-header-nav__button" for=__search ></label> <div class=md-search data-md-component=search role=dialog > <label class=md-search__overlay for=__search ></label> <div class=md-search__inner role=search > <form class=md-search__form action=search.html method=get name=search > <input type=text class=md-search__input name=q placeholder=""Search"" autocapitalize=off autocomplete=off spellcheck=false data-md-component=query data-md-state=active > <label class="md-icon md-search__icon" for=__search ></label> <button type=reset class="md-icon md-search__icon" data-md-component=reset tabindex=-1 > &#xE5CD; </button> </form> <div class=md-search__output > <div class=md-search__scrollwrap data-md-scrollfix> <div class=md-search-result data-md-component=result > <div class=md-search-result__meta > Type to start searching </div> <ol class=md-search-result__list ></ol> </div> </div> </div> </div> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <div class=md-header-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> </div> <script src="_static/javascripts/version_dropdown.js"></script> <script> var json_loc = ""versions.json"", target_loc = "../", text = "Versions"; $( document ).ready( add_version_dropdown(json_loc, target_loc, text)); </script> </div> </nav> </header> <div class=md-container > <nav class=md-tabs data-md-component=tabs > <div class="md-tabs__inner md-grid"> <ul class=md-tabs__list > <li class=md-tabs__item ><a href=index.html class=md-tabs__link >Jungfraujoch 1.0.0-rc.174 documentation</a> </ul> </div> </nav> <main class=md-main > <div class="md-main__inner md-grid" data-md-component=container > <div class="md-sidebar md-sidebar--primary" data-md-component=navigation > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--primary" data-md-level=0 > <label class="md-nav__title md-nav__title--site" for=__drawer > <a href=index.html title="Jungfraujoch 1.0.0-rc.174 documentation" class="md-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> <a href=index.html title="Jungfraujoch 1.0.0-rc.174 documentation">PSI Jungfraujoch</a> </label> <div class=md-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> <ul class=md-nav__list > <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >rugnux — data processing</span></span> <li class=md-nav__item > <a href=RUGNUX.html class=md-nav__link >Rugnux</a> <li class=md-nav__item > <a href=RUGNUX_OVERVIEW.html class=md-nav__link >What Rugnux does</a> <li class=md-nav__item > <a href=RUGNUX_INSTALL.html class=md-nav__link >Installing Rugnux</a> <li class=md-nav__item > <a href=RUGNUX_FORMATS.html class=md-nav__link >What Rugnux reads</a> <li class=md-nav__item > <a href=RUGNUX_TUTORIAL.html class=md-nav__link >Running Rugnux</a> <li class=md-nav__item > <a href=RUGNUX_INTEGRATION.html class=md-nav__link >Rugnux with other programs</a> <li class=md-nav__item > <a href=RUGNUX_REPORT.html class=md-nav__link >The results report</a> <li class=md-nav__item > <a href=RUGNUX_ADVANCED.html class=md-nav__link >Advanced Rugnux</a> <li class=md-nav__item > <a href=RUGNUX_CALIBRATION.html class=md-nav__link >Detector calibration from powder rings (<code class="docutils literal notranslate"><span class=pre >rugnux</span> <span class=pre >--mode</span> <span class=pre >calibration</span></code>)</a> <li class=md-nav__item > <a href=CPU_DATA_ANALYSIS.html class=md-nav__link >CPU-side crystallographic data analysis (Jungfraujoch)</a> <li class=md-nav__item > <a href=CPU_DATA_ANALYSIS_IMAGE.html class=md-nav__link >Data analysis: from images to spots (§0–§3)</a> <li class=md-nav__item > <input class="md-toggle md-nav__toggle" data-md-toggle=toc type=checkbox id=__toc > <label class="md-nav__link md-nav__link--active" for=__toc > Data analysis: indexing and geometry (§4–§7) </label> <a href="#" class="md-nav__link md-nav__link--active">Data analysis: indexing and geometry (§4–§7)</a> <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> <ul class=md-nav__list > <li class=md-nav__item > <a href="#indexing-overview" class=md-nav__link >4. Indexing overview</a> <li class=md-nav__item > <a href="#fft-indexing-unknown-unit-cell" class=md-nav__link >5. FFT indexing (unknown unit cell)</a> <li class=md-nav__item > <a href="#bravais-lattice-centering-inference-lattice-search" class=md-nav__link >6. Bravais lattice / centering inference (“lattice search”)</a> <li class=md-nav__item > <a href="#geometry-and-lattice-refinement" class=md-nav__link >7. Geometry and lattice refinement</a> </ul> <li class=md-nav__item > <a href=CPU_DATA_ANALYSIS_INTEGRATION.html class=md-nav__link >Data analysis: integration, scaling and merging (§8–§12)</a> <li class=md-nav__item > <a href=CPU_DATA_ANALYSIS_DECISIONS.html class=md-nav__link >Data analysis: space group and validation (§13–§14)</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Jungfraujoch — acquisition</span></span> <li class=md-nav__item > <a href=JFJOCH_BROKER.html class=md-nav__link >jfjoch_broker</a> <li class=md-nav__item > <a href=JFJOCH_WRITER.html class=md-nav__link >jfjoch_writer</a> <li class=md-nav__item > <a href=JFJOCH_VIEWER.html class=md-nav__link >jfjoch_viewer</a> <li class=md-nav__item > <a href=SOFTWARE_INTEGRATION.html class=md-nav__link >Integration with MX data processing software</a> <li class=md-nav__item > <a href=TOOLS.html class=md-nav__link >Tools</a> <li class=md-nav__item > <a href=DEPLOYMENT.html class=md-nav__link >Deployment</a> <li class=md-nav__item > <a href=DETECTORS.html class=md-nav__link >Supported detectors</a> <li class=md-nav__item > <a href=HARDWARE.html class=md-nav__link >Hardware requirements</a> <li class=md-nav__item > <a href=SOFTWARE.html class=md-nav__link >Software requirements</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >FPGA</span></span> <li class=md-nav__item > <a href=FPGA.html class=md-nav__link >FPGA smartNIC</a> <li class=md-nav__item > <a href=FPGA_LICENSE.html class=md-nav__link >FPGA license</a> <li class=md-nav__item > <a href=FPGA_DESIGN.html class=md-nav__link >FPGA data flow</a> <li class=md-nav__item > <a href=FPGA_NETWORK.html class=md-nav__link >FPGA network</a> <li class=md-nav__item > <a href=FPGA_PCIE_DRIVER.html class=md-nav__link >FPGA PCIe driver</a> <li class=md-nav__item > <a href=FPGA_SETTINGS.html class=md-nav__link >FPGA advanced reference</a> <li class=md-nav__item > <a href=FPGA_DATA_ANALYSIS.html class=md-nav__link >FPGA data analysis</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Reference</span></span> <li class=md-nav__item > <a href=DETECTOR_GEOMETRY.html class=md-nav__link >Detector geometry</a> <li class=md-nav__item > <a href=OPENAPI.html class=md-nav__link >OpenAPI</a> <li class=md-nav__item > <a href=OPENAPI_SPECS.html class=md-nav__link >OpenAPI specification</a> <li class=md-nav__item > <a href=PYTHON_CLIENT.html class=md-nav__link >OpenAPI Python client</a> <li class=md-nav__item > <a href=CBOR.html class=md-nav__link >CBOR messages</a> <li class=md-nav__item > <a href=HDF5.html class=md-nav__link >HDF5 / NeXus data format</a> <li class=md-nav__item > <a href=IMAGE_STREAM.html class=md-nav__link >Data streams</a> <li class=md-nav__item > <a href=PIXEL_MASK.html class=md-nav__link >Pixel mask</a> <li class=md-nav__item > <a href=WEB_FRONTEND.html class=md-nav__link >Web frontend</a> <li class=md-nav__item > <a href=TESTS.html class=md-nav__link >Tests</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Project</span></span> <li class=md-nav__item > <a href=ACKNOWLEDGEMENT.html class=md-nav__link >Acknowledgements</a> <li class=md-nav__item > <a href=EXTERNAL_TEST_DATA.html class=md-nav__link >External test data</a> <li class=md-nav__item > <a href=LICENSE.html class=md-nav__link >License</a> <li class=md-nav__item > <a href=THIRD_PARTY_NOTICES.html class=md-nav__link >Third-party software notices</a> <li class=md-nav__item > <a href=VERSIONING.html class=md-nav__link >Semantic versioning</a> <li class=md-nav__item > <a href=SECURITY.html class=md-nav__link >Security</a> <li class=md-nav__item > <a href=RELEASE_CONTENTS.html class=md-nav__link >Release contents</a> <li class=md-nav__item > <a href=REPOSITORIES.html class=md-nav__link >Linux package repositories</a> <li class=md-nav__item > <a href=NAMING.html class=md-nav__link >Naming</a> <li class=md-nav__item > <a href=CHANGELOG.html class=md-nav__link >Changelog</a> </ul> </nav> </div> </div> </div> <div class="md-sidebar md-sidebar--secondary" data-md-component=toc > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> </div> </div> </div> <div class=md-content > <article class="md-content__inner md-typeset" role=main > <section class="tex2jax_ignore mathjax_ignore" id=data-analysis-indexing-and-geometry-47 > <h1 id=cpu-data-analysis-indexing--page-root >Data analysis: indexing and geometry (§4–§7)<a class=headerlink href="#cpu-data-analysis-indexing--page-root" title="Link to this heading"></a></h1> <p>Part of the <a class="reference internal" href=CPU_DATA_ANALYSIS.html ><span class="std std-doc">CPU/GPU data-analysis reference</span></a>; the section numbers are continuous across its four parts.</p> <nav class="contents local" id=on-this-page > <p class=topic-title >On this page</p> <ul class=simple > <li><p><a class="reference internal" href="#indexing-overview" id=id1 >4. Indexing overview</a></p> <ul> <li><p><a class="reference internal" href="#indexed-spot-decision-inlier-test" id=id2 >4.1 Indexed-spot decision (inlier test)</a></p> <li><p><a class="reference internal" href="#the-spot-budget" id=id3 >4.2 The spot budget</a></p> </ul> <li><p><a class="reference internal" href="#fft-indexing-unknown-unit-cell" id=id4 >5. FFT indexing (unknown unit cell)</a></p> <ul> <li><p><a class="reference internal" href="#directional-projections-and-histograms" id=id5 >5.1 Directional projections and histograms</a></p> <li><p><a class="reference internal" href="#fft-peak-picking-and-candidate-vectors" id=id6 >5.2 FFT peak picking and candidate vectors</a></p> <li><p><a class="reference internal" href="#lattice-reduction-and-cell-candidates" id=id7 >5.3 Lattice reduction and cell candidates</a></p> <li><p><a class="reference internal" href="#robust-refinement-and-best-cell-selection" id=id8 >5.4 Robust refinement and best-cell selection</a></p> <li><p><a class="reference internal" href="#spindle-alignment-the-part-of-the-blind-cone-symmetry-cannot-repair" id=id9 >5.5 Spindle alignment: the part of the blind cone symmetry cannot repair</a></p> </ul> <li><p><a class="reference internal" href="#bravais-lattice-centering-inference-lattice-search" id=id10 >6. Bravais lattice / centering inference (“lattice search”)</a></p> <li><p><a class="reference internal" href="#geometry-and-lattice-refinement" id=id11 >7. Geometry and lattice refinement</a></p> <ul> <li><p><a class="reference internal" href="#parameterization" id=id12 >7.1 Parameterization</a></p> <li><p><a class="reference internal" href="#residuals-and-objective" id=id13 >7.2 Residuals and objective</a></p> <li><p><a class="reference internal" href="#rotation-datasets-bringing-observations-to-a-common-reference-frame" id=id14 >7.3 Rotation datasets: bringing observations to a common reference frame</a></p> <li><p><a class="reference internal" href="#multi-stage-tightening-of-inlier-tolerance" id=id15 >7.4 Multi-stage tightening of inlier tolerance</a></p> <li><p><a class="reference internal" href="#rotation-geometry-post-refinement-two-pass" id=id16 >7.5 Rotation geometry post-refinement (two-pass)</a></p> <li><p><a class="reference internal" href="#detector-geometry-from-powder-rings" id=id17 >7.6 Detector geometry from powder rings</a></p> </ul> </ul> </nav> <section id=indexing-overview > <h2 id=indexing-overview ><a class=toc-backref href="#id1" role=doc-backlink >4. Indexing overview</a><a class=headerlink href="#indexing-overview" title="Link to this heading"></a></h2> <p>Indexing maps observed reciprocal-space vectors <span class="math notranslate nohighlight">\(\mathbf{s}_i\)</span> to a lattice such that: <span class="math notranslate nohighlight">\( \mathbf{s}_i \approx h_i\mathbf{a}^* + k_i\mathbf{b}^* + l_i\mathbf{c}^*, \)</span> with integer <span class="math notranslate nohighlight">\((h_i,k_i,l_i)\)</span>.</p> <p>Jungfraujoch supports two complementary indexing strategies:</p> <ol class="arabic simple"> <li><p><strong>FFT-based indexing</strong> (Rossmann-type): does not require an a priori unit cell; suitable for unknown samples.</p> <li><p><strong>Fast-feedback indexing</strong> (TORO-like): requires an approximate unit cell; optimized for speed and feedback.</p> </ol> <p>Both feed into a common robust refinement/selection stage which maximizes the number of inliers under an indexing tolerance, and which can return <strong>more than one lattice</strong> per image (multi-lattice indexing; see §5.4).</p> <section id=indexed-spot-decision-inlier-test > <h3 id=indexed-spot-decision-inlier-test ><a class=toc-backref href="#id2" role=doc-backlink >4.1 Indexed-spot decision (inlier test)</a><a class=headerlink href="#indexed-spot-decision-inlier-test" title="Link to this heading"></a></h3> <p>Given a trial lattice with direct basis vectors <span class="math notranslate nohighlight">\(\mathbf{a},\mathbf{b},\mathbf{c}\)</span> (used here as reciprocal-space dot-test vectors), fractional indices are estimated by: <span class="math notranslate nohighlight">\( h_f = \mathbf{s}\cdot\mathbf{a},\quad k_f = \mathbf{s}\cdot\mathbf{b},\quad l_f = \mathbf{s}\cdot\mathbf{c}. \)</span> Let <span class="math notranslate nohighlight">\((h,k,l)=(\mathrm{round}(h_f),\mathrm{round}(k_f),\mathrm{round}(l_f))\)</span> and define the fractional residual: <span class="math notranslate nohighlight">\( \delta^2 = (h_f-h)^2 + (k_f-k)^2 + (l_f-l)^2. \)</span> A spot is indexed if <span class="math notranslate nohighlight">\(\delta^2 &lt; \tau^2\)</span>, where <span class="math notranslate nohighlight">\(\tau\)</span> is the configured tolerance.</p> <p>For indexed spots, the reciprocal lattice point <span class="math notranslate nohighlight">\(\mathbf{p} = h\mathbf{a}^*+k\mathbf{b}^*+l\mathbf{c}^*\)</span> is used to compute <span class="math notranslate nohighlight">\(\Delta_\mathrm{Ewald}(\mathbf{p})\)</span> (stored as a diagnostic and later used in profile-radius estimation).</p> <p>A frame is taken to be this crystal’s when at least a fraction <span class="math notranslate nohighlight">\(g = 0.20\)</span> of its in-resolution, non-ice spots index. On rotation data that decision is what admits the frame to integration, so its denominator matters: every spot handed to it that is not a reflection of this crystal argues against the frame. Where it cannot do that job — a lattice whose pooled spots fall below <span class="math notranslate nohighlight">\(g\)</span> and which fewer than half the validation frames clear — every frame is integrated instead: the frames that clear <span class="math notranslate nohighlight">\(g\)</span> are then only the upper tail of the same sparse population, not the frames the crystal was in, and admitting them alone dropped most of a small-molecule sweep and its completeness with it.</p> <p>That test decides a <strong>frame</strong>. Whether a rotation run has a lattice <strong>at all</strong> is decided on the spots instead. A frame count comes from serial crystallography, where each image is its own experiment; a rotation sweep is one crystal and one orientation matrix, its frames are not independent of each other, and what such a count mostly measures is how many spots happen to land on a frame — a sweep carrying four spots an image cannot reach a six-spot bar on three frames in four however right the lattice is. The refusal therefore compares the fraction of <em>all</em> spots in the sampled frames that the lattice explains against what the same lattice explains when each frame’s spots are put at <strong>another frame’s angle</strong>: same lattice, same spots, same detector, same refinement, with only the claim that these spots were seen at <em>these</em> angles removed. That difference is the evidence, and it carries no spots-per-frame number anywhere, so nothing has to be chosen for a crystal that diffracts weakly. Measured over a hundred datasets the permuted level never exceeds 2.6 % and the smallest true margin is seventeen points. <code class="docutils literal notranslate"><span class=pre >--min-indexed-spots</span></code> (default 6, floor 4 — four is where a lattice stops being fitted by any three spots) still sets the reported indexing rate and the count the first pass <em>ranks</em> candidate lattices by; every rescue and every arbiter still counts frames.</p> </section> <section id=the-spot-budget > <h3 id=the-spot-budget ><a class=toc-backref href="#id3" role=doc-backlink >4.2 The spot budget</a><a class=headerlink href="#the-spot-budget" title="Link to this heading"></a></h3> <p>Only the strongest <code class="docutils literal notranslate"><span class=pre >--max-spots</span></code> spots of an image are kept (<code class="docutils literal notranslate"><span class=pre >FilterSpotsByCount</span></code>), and that budget therefore sets the denominator above. Detections are not all reflections — background structure, unlisted ice and detector artefacts are found too — so a budget deeper than an image’s reflections makes the test above a measurement of the background rather than of the crystal, and a <em>larger</em> budget can integrate <em>fewer</em> images.</p> <p>rugnux measures the budget instead of fixing it. With the sweep’s lattice in hand, the first pass tallies the spots of a sample of frames by their rank in the intensity-ordered list: how many images carried a spot at that rank, and on how many of them it indexed. Weighting each indexed spot by <span class="math notranslate nohighlight">\(1-g\)</span> and each unindexed one by <span class="math notranslate nohighlight">\(-g\)</span> — the same weighting the frame test applies to the list as a whole — the running sum over ranks</p> <div class="math notranslate nohighlight"> \[ E(N) = n_\mathrm{indexed}(N) - g\, n_\mathrm{counted}(N) \]</div> <p>rises exactly while the spots at that depth lie on the lattice more often than <span class="math notranslate nohighlight">\(g\)</span>, and falls after. The budget is <span class="math notranslate nohighlight">\(\arg\max_N E(N)\)</span>. Its meaning is “as deep into the list as the image is still showing reflections of this crystal”: deeper spots cannot help the frame test and can only push a frame towards rejection. On crystals whose spot lists are reflections all the way down the maximum is at the end of the list and the budget is unchanged.</p> <p>The peak has to be one. Under the null — the spots lie on the lattice at the same rate at every depth — <span class="math notranslate nohighlight">\(E\)</span> is a driftless random walk in the counted spots, with per-spot variance <span class="math notranslate nohighlight">\(g(1-g)\)</span>, and the maximum of such a walk is positive whatever the data; an <span class="math notranslate nohighlight">\(\arg\max\)</span> taken on its own would shorten every dataset, including one with nothing to shorten. What the budget acts on is the fall from the peak to the end of the list, <span class="math notranslate nohighlight">\(E(N^*) - E(L)\)</span>, which is the maximum of the same walk read backwards; by the reflection principle its null law is <span class="math notranslate nohighlight">\(P(\mathrm{fall} &gt; z\sqrt{g(1-g)T}) = 2(1-\Phi(z))\)</span> over <span class="math notranslate nohighlight">\(T\)</span> counted spots in all, so the search over ranks is already accounted for and no further multiple-comparison correction applies. The budget is taken only where the fall clears that bar at <span class="math notranslate nohighlight">\(z = 3.29\)</span>, one false shortening in a thousand measurements; otherwise the whole list is kept.</p> </section> </section> <hr class=docutils /> <section id=fft-indexing-unknown-unit-cell > <h2 id=fft-indexing-unknown-unit-cell ><a class=toc-backref href="#id4" role=doc-backlink >5. FFT indexing (unknown unit cell)</a><a class=headerlink href="#fft-indexing-unknown-unit-cell" title="Link to this heading"></a></h2> <p>FFT indexing follows a classical approach: detect dominant periodicities by projecting reciprocal-space points onto many directions and Fourier transforming the resulting 1D histograms.</p> <section id=directional-projections-and-histograms > <h3 id=directional-projections-and-histograms ><a class=toc-backref href="#id5" role=doc-backlink >5.1 Directional projections and histograms</a><a class=headerlink href="#directional-projections-and-histograms" title="Link to this heading"></a></h3> <p>Choose a set of unit vectors <span class="math notranslate nohighlight">\(\{\mathbf{u}_d\}\)</span> on a half-sphere (a near-uniform distribution generated via a golden-angle construction). For each direction <span class="math notranslate nohighlight">\(d\)</span>, form a histogram in the scalar projection: <span class="math notranslate nohighlight">\( t_{id} = \left|\mathbf{u}_d\cdot \mathbf{s}_i\right|. \)</span></p> <p>Bin width is chosen approximately as: <span class="math notranslate nohighlight">\( \Delta t \approx \frac{1}{2 L_\mathrm{max}}, \)</span> where <span class="math notranslate nohighlight">\(L_\mathrm{max}\)</span> is the maximum expected real-space unit-cell edge (Å). The histogram extent is tied to the maximum <span class="math notranslate nohighlight">\(q\)</span> used (set by a high-resolution cutoff for indexing).</p> </section> <section id=fft-peak-picking-and-candidate-vectors > <h3 id=fft-peak-picking-and-candidate-vectors ><a class=toc-backref href="#id6" role=doc-backlink >5.2 FFT peak picking and candidate vectors</a><a class=headerlink href="#fft-peak-picking-and-candidate-vectors" title="Link to this heading"></a></h3> <p>For each direction, the FFT magnitude spectrum is computed; peaks correspond to periodicities along <span class="math notranslate nohighlight">\(\mathbf{u}_d\)</span>. Each direction yields a candidate real-space length <span class="math notranslate nohighlight">\(L\)</span> chosen <strong>not</strong> by raw magnitude but by <strong>maximum prominence above a running-mean local background</strong> (subtracting the broad low-frequency envelope that otherwise dominates on weak or pink-beam frames), subject to <span class="math notranslate nohighlight">\(L\ge L_\mathrm{min}\)</span>.</p> <p>The running-mean background window keeps a <strong>constant width</strong> and is slid inward at the ends of the spectrum rather than truncated there, so a peak within half a window of either end — which is where the longest cells sit — is judged against as much background as any other. Both window bounds stay monotonically non-decreasing in the bin index, so the GPU kernel’s running sum is still valid.</p> <p>The longest basis vector the transform can return is <code class="docutils literal notranslate"><span class=pre >fft_max_unit_cell_A</span></code>, since the histogram is sized from it and its last usable bin <em>is</em> that length; the shortest is <code class="docutils literal notranslate"><span class=pre >fft_min_unit_cell_A</span></code> (<code class="docutils literal notranslate"><span class=pre >rugnux</span> <span class=pre >--fft-min-unit-cell</span></code>, default 10 Å), below which a candidate is discarded. The defaults are unchanged (500 Å and 10 Å), but the accepted range for the maximum now reaches 1200 Å, and a reference cell given with <code class="docutils literal notranslate"><span class=pre >-C</span></code> moves <strong>both</strong> bounds on its own — up to reach a long axis, down to admit a small-molecule cell — since a cell the search cannot represent cannot be found by it.</p> <p>Candidate vectors are <span class="math notranslate nohighlight">\(\mathbf{v}_d = L_d\,\mathbf{u}_d\)</span>.</p> <p>A collinearity filter removes nearly parallel vectors (e.g. within 5°) and attempts to resolve harmonic ambiguity: shorter “fundamental” vectors may be preferred over longer harmonics if their peak magnitude is sufficiently strong relative to the dominant peak.</p> </section> <section id=lattice-reduction-and-cell-candidates > <h3 id=lattice-reduction-and-cell-candidates ><a class=toc-backref href="#id7" role=doc-backlink >5.3 Lattice reduction and cell candidates</a><a class=headerlink href="#lattice-reduction-and-cell-candidates" title="Link to this heading"></a></h3> <p>Triples of candidate vectors are combined to form candidate bases <span class="math notranslate nohighlight">\((\mathbf{A},\mathbf{B},\mathbf{C})\)</span>, each reduced to its <strong>Niggli-reduced cell</strong> (Gruber-vector reduction) before comparison, and filtered by allowed length and angle ranges. Two passes are run: a standard pass forms shortest-vector triples from the ~30 strongest filtered directions; if the best cell then indexes fewer than half the spots, a <strong>widened fallback</strong> anchors the two shortest axes and lets the third range over up to ~60 candidate vectors (deduplicated by Niggli cell), catching large, elongated or superstructure cells the first pass misses.</p> <p>A triple whose three vectors are <strong>coplanar</strong> is rejected before refinement. The length and angle filters cannot see it — any flat combination satisfies them — and a cell that flat has a metric determinant small enough for <code class="docutils literal notranslate"><span class=pre >float</span></code> to get its <em>sign</em> wrong, after which the guard against a negative argument to the square root places <span class="math notranslate nohighlight">\(\mathbf{c}\)</span> in the <span class="math notranslate nohighlight">\(\mathbf{a}\)</span>-<span class="math notranslate nohighlight">\(\mathbf{b}\)</span> plane, the reciprocal volume diverges and the solver reports a not-a-number Jacobian. The test is the volume fraction <span class="math notranslate nohighlight">\(|V|/(|\mathbf{a}||\mathbf{b}||\mathbf{c}|)\)</span>, which must reach 0.02 — about 1.1° off flat, well below the flattest genuine candidate observed and far above where <code class="docutils literal notranslate"><span class=pre >float</span></code> loses the sign — and it is applied both where triples are produced and at the optimizer’s entry points.</p> <p>A shortlist <strong>confined to one plane</strong> cannot close a cell at all, and the row it is missing is the plane normal. That is detected from the eigenvalue ratio of the shortlisted directions’ scatter matrix, and one further transform is then spent with the same direction count inside a narrow cap about the normal. More directions do not substitute for it: at the exact true direction a very long axis can still rank far below the shortlist cut, so for <strong>this</strong> rescue the obstacle is the ranking rather than the sampling, and a denser grid costs several times the device memory for the same answer.</p> <p>Sampling has a limit of its own, and it binds well before the 1200 Å the accepted range for <code class="docutils literal notranslate"><span class=pre >fft_max_unit_cell_A</span></code> admits (§5.2). A direction off a real-space axis of length <span class="math notranslate nohighlight">\(a\)</span> by an angle <span class="math notranslate nohighlight">\(\theta\)</span> smears each projected lattice plane by about <span class="math notranslate nohighlight">\(\theta/d_\mathrm{min}\)</span> in the projection, so the planes (spacing <span class="math notranslate nohighlight">\(1/a\)</span>) stay resolved only while <span class="math notranslate nohighlight">\(\theta \lesssim d_\mathrm{min}/(2a)\)</span>. The shipped grid of 16384 directions puts the nearest one within about 0.6° of any axis, which satisfies that bound only up to roughly 120–150 Å at typical indexing resolutions; a longer axis is not refused but returned as a plausible sub-cell or harmonic. Raising the maximum cell alone therefore does not extend the reach — the direction grid has to resolve the axis before the histogram can represent it.</p> </section> <section id=robust-refinement-and-best-cell-selection > <h3 id=robust-refinement-and-best-cell-selection ><a class=toc-backref href="#id8" role=doc-backlink >5.4 Robust refinement and best-cell selection</a><a class=headerlink href="#robust-refinement-and-best-cell-selection" title="Link to this heading"></a></h3> <p>Candidate bases are refined against observed spots using an iterative inlier‑focused least‑squares procedure (trimmed/contracting threshold). Candidates are then ranked:</p> <ol class="arabic simple"> <li><p>more indexed spots wins — <strong>unless</strong> two candidates index within ~10 % of each other, in which case</p> <li><p>the <strong>smaller-volume</strong> cell is preferred (when the volumes differ by more than ~5 %), avoiding a doubled supercell, then</p> <li><p>the smaller refinement score, then the spot count again.</p> </ol> <p>Selection is <strong>not limited to a single lattice</strong>: after the best cell is accepted, further lattices are added as separate crystals provided fewer than ~40 % of their indexed spots overlap an already-accepted lattice (up to two extra by default), so split or multi-lattice crystals are indexed rather than discarded.</p> <p>An optional reference unit cell (if supplied) restricts acceptance to cells within a relative distance tolerance in edge lengths (permutation-invariant).</p> </section> <section id=spindle-alignment-the-part-of-the-blind-cone-symmetry-cannot-repair > <h3 id=spindle-alignment-the-part-of-the-blind-cone-symmetry-cannot-repair ><a class=toc-backref href="#id9" role=doc-backlink >5.5 Spindle alignment: the part of the blind cone symmetry cannot repair</a><a class=headerlink href="#spindle-alignment-the-part-of-the-blind-cone-symmetry-cannot-repair" title="Link to this heading"></a></h3> <p>A sweep about the spindle <span class="math notranslate nohighlight">\(\hat{\mathbf{e}}\)</span> never brings a reciprocal point closer than <span class="math notranslate nohighlight">\(\theta_\mathrm{max}=\arcsin(\lambda/2d)\)</span> to the axis onto the Ewald sphere, so a double cone of half-angle <span class="math notranslate nohighlight">\(\theta_\mathrm{max}\)</span> is missing from every resolution shell — each shell losing its own <span class="math notranslate nohighlight">\(1-\cos\theta(d)\)</span> — however long the sweep runs. Crystal symmetry normally repairs that loss by mapping the cone onto measured territory. It fails to when a symmetry axis lies inside the cone (the cone maps onto itself) — and, for a <strong>2-fold</strong>, equally when the axis is perpendicular to the spindle, because the diad carries the cone onto its opposite lobe, which the sweep leaves just as unmeasured. Friedel never helps: the cone is double-sided. The loss is a coherent cap rather than a scatter of absences, so it costs a map more than the same percentage lost at random.</p> <p>The per-image score asks how much of that cone the frame’s own orientation makes unrecoverable. The crystal’s short lattice rows are read off the FFT row shortlist of §5.2 (a symmetry axis is always a lattice row, and usually among the short ones), and each plausible direction is scored as if it carried a lone 2-fold:</p> <div class="math notranslate nohighlight"> \[ \text{spindle blind fraction} = \frac{2}{\pi}\left(\arccos x - x\sqrt{1-x^{2}}\right), \qquad x = \min(\beta,\,90^\circ-\beta)\,/\,\theta_\mathrm{max}, \]</div> <p>where <span class="math notranslate nohighlight">\(\beta\)</span> is the direction’s miss-angle from the spindle. The <strong>fold</strong> of <span class="math notranslate nohighlight">\(\beta\)</span> about <span class="math notranslate nohighlight">\(45^\circ\)</span> is the diad geometry above: both ends of the range are the bad case, and the closed form reproduces a Monte Carlo of the true double-cone self-overlap to 0.004 at <span class="math notranslate nohighlight">\(\theta_\mathrm{max}=15^\circ\)</span> and 0.008 at <span class="math notranslate nohighlight">\(25^\circ\)</span> (past <span class="math notranslate nohighlight">\(45^\circ\)</span> it under-reports, by 0.05 at <span class="math notranslate nohighlight">\(50^\circ\)</span>). <span class="math notranslate nohighlight">\(\theta_\mathrm{max}\)</span> is taken from the <strong>geometric</strong> resolution of the setup — the detector corner at the recorded distance and wavelength — an upper bound on any sweep collected without moving the detector; the still’s own spot resolution would understate the cone on exactly the weak frames that mislead. The worst direction wins, and the directions scored are the strong in-window rows <strong>and the normals of their pairs</strong> — the normal to two lattice rows is itself a reciprocal-lattice row and a symmetry axis is parallel in both bases, so a lone 2-fold on an axis far beyond the length window (a long monoclinic unique axis) is still seen by direction: measured on a synthetic lone-diad crystal with a 300 Å unique axis, the fraction of severe mounts reported severe rises from 0.60 to 1.00 with the pair normals, at no extra engagement on that class’s harmless mounts. Nothing about the goniometer enters: the number describes the problem and leaves the remedy — a second sweep, a reorientation — to the beamline.</p> <p><strong>This is a worst-case bound under an assumption of no symmetry, not an estimate.</strong> A still cannot know the point group, so the nearest plausible row is scored as a lone 2-fold. An axis of order <span class="math notranslate nohighlight">\(\geq 3\)</span> perpendicular to the spindle in fact <strong>fully repairs</strong> the cone (measured unrepaired fraction 0.000 for orders 3, 4 and 6, against 1.000 for a diad), which a still cannot see, so the bound is deliberately pessimistic on higher-symmetry crystals — that is the intended trade, because the number exists as a <strong>trigger</strong> for beamline automation, not as a physical quantity a user interprets.</p> <p><strong>Trigger states.</strong> The stored quantity is the continuous score; automation reads it through three fixed states with nothing to tune (<code class="docutils literal notranslate"><span class=pre >SpindleTrigger</span></code> in <code class="docutils literal notranslate"><span class=pre >SpindleBlindFraction.h</span></code>): <strong>engage</strong> at score ≥ 0.5, <strong>don’t engage</strong> below, and <strong>cannot say</strong> when there is no value at all — too few spots, no shortlist, the consistency guard refused, the path never computed one. <strong>Automation must treat CANNOT SAY as ENGAGE</strong>: the error costs are asymmetric — a false negative is unrecoverable (one sweep is collected and the data stay short forever) while a false positive costs minutes of beamtime. Every transport keeps absence distinguishable from a measured zero (an absent CBOR key, a NaN in the HDF5 arrays, an absent optional after read-back). The 0.5 threshold is geometry, not tuning: the score is monotone in the folded miss-angle, so a threshold is a fold-angle gate, and 0.5 gates at <span class="math notranslate nohighlight">\(\min(\beta,90^\circ-\beta) \le 0.404\,\theta_\mathrm{max}\)</span>; engaging on any overlap at all would gate at the cone edge, whose perpendicular band alone spans <span class="math notranslate nohighlight">\(\sin\theta_\mathrm{max}\)</span> of orientation space per row (26 % at <span class="math notranslate nohighlight">\(15^\circ\)</span>) and unions over a frame’s rows to well over half of all mountings — a trigger that always fires decides nothing.</p> <p><strong>Reach and honest rates.</strong> The score needs 60 spots (calibrated per crystal — 22 independent mounts — misses triple below it); below that, a frame that still indexed answers from the winning lattice’s shortest rows, and otherwise the state is <em>cannot say</em>. Because the bound is pessimistic by design, it engages on a substantial share of harmless mountings: a single strong row’s perpendicular band alone covers ~11 % of orientation space at the severe level (<span class="math notranslate nohighlight">\(\theta_\mathrm{max}=15^\circ\)</span>), and the union over a frame’s rows and pair normals reaches roughly a quarter to three quarters of random mountings depending on cone width and row count (measured 0.74 on a generic triclinic cell at <span class="math notranslate nohighlight">\(15^\circ\)</span> via the lattice path). That is accepted: the cheap error is the extra wedge. An earlier figure of ~1 % false alarms (AUC 0.948) came from a null of five <em>decoy directions per frame</em> — it shows the estimator does not hallucinate rows near arbitrary directions, which is worth knowing, but it is <strong>not</strong> a false-alarm rate over harmless mountings, which geometry forbids to be that low.</p> <p><strong>Offline, the guessing stops.</strong> Once <code class="docutils literal notranslate"><span class=pre >rugnux</span></code> has merged a rotation run it holds the measured point group and the exact indexed orientation, and the run-level number is computed exactly instead: the group’s proper rotations are applied to the blind double cone in the crystal’s actual orientation, and the fraction of unique reflections no operator can recover is reported as <code class="docutils literal notranslate"><span class=pre >SPINDLE_LOST_UNIQUE_FRACTION</span></code> in the processing report and <code class="docutils literal notranslate"><span class=pre >/entry/MX/spindleLostUniqueFraction</span></code> in the master file (<a class="reference internal" href=RUGNUX_REPORT.html ><span class="std std-doc">the results report</span></a>). That number clears or convicts a mounting the per-image bound can only be pessimistic about: a dihedral crystal with an in-plane diad on the spindle, or any cubic crystal in any orientation, loses nothing at all.</p> </section> </section> <hr class=docutils /> <section id=bravais-lattice-centering-inference-lattice-search > <h2 id=bravais-lattice-centering-inference-lattice-search ><a class=toc-backref href="#id10" role=doc-backlink >6. Bravais lattice / centering inference (“lattice search”)</a><a class=headerlink href="#bravais-lattice-centering-inference-lattice-search" title="Link to this heading"></a></h2> <p>If the space group is supplied by the user, its lattice constraints are assumed for refinement and subsequent processing.</p> <p>If not, Jungfraujoch attempts to infer the most plausible Bravais lattice type from the metric tensor after Niggli reduction:</p> <ol class="arabic simple"> <li><p><strong>Niggli reduction</strong> is performed to obtain a reduced cell in <span class="math notranslate nohighlight">\(G^6\)</span> representation (Gruber vector).</p> <li><p>The reduced cell is compared against a list of Niggli classes corresponding to Bravais lattices and centerings.</p> <li><p>The highest-symmetry class that matches within tolerances is selected (relative metric tolerance and angular tolerance). The list is walked in order of decreasing symmetry and the first class that fits is taken, so a class that only just fits can pre-empt a lower-symmetry one that fits exactly.</p> </ol> <p>The output includes:</p> <ul class=simple > <li><p>a conventional cell,</p> <li><p>crystal system (triclinic, monoclinic, …),</p> <li><p>centering symbol (one of <span class="math notranslate nohighlight">\(P, C, I, F, R\)</span>; the <span class="math notranslate nohighlight">\(A/B\)</span> variants are not emitted here — they are handled only later as prediction absences, §8.4).</p> </ul> <p>This stage provides centering information used for systematic absences in prediction (§8.4) and for reporting.</p> <p><strong>A metric symmetry has to earn itself.</strong> The class is chosen from the <em>unrefined</em> candidate against a fixed angular tolerance (3°), so a lattice that is pseudo-symmetric to a few tenths of a degree is promoted a class too far — and the constraint then snaps a real angle to the ideal one, which throws nearly every reflection of every frame outside tolerance. Measured on a monoclinic crystal pseudo-C-orthorhombic to 0.42°, the promoted cell indexed 2 of 60 frames where its own primitive cell indexed 39: the same lattice, <span class="math notranslate nohighlight">\(\mathbf{b}_{oC}=-(\mathbf{a}+2\mathbf{c})\)</span>, at exactly twice the volume. A more accurate candidate is more likely to be promoted, not less: a run escapes the promotion only when the raw candidate misses the tolerance window.</p> <p>The lookup can also land <em>short</em>: near the Niggli type-I/type-II boundary the character is decided by the last digits of the refined cell, and the class it names caps which point groups the space-group search enumerates. The metric-symmetry re-ask of §13.1 covers this — rotations the named class has no room for are put to the intensities directly, and where the metric group exceeds the class’s holohedry the merge is reindexed into the metric cell and the search re-run there.</p> <p>The rotation first pass refines the <strong>twelve</strong> best candidate lattices rather than four. The pre-refinement indexed fraction is an unreliable ranking, so a correct cell can sit below several degenerate ones and never be refined at all.</p> <p>For rotation data the first pass therefore refines the constrained cell <em>and</em> an unconstrained (triclinic) one from the same spots — which it finds itself, over a sample spread across the sweep, rather than reading what the acquisition wrote — and settles the two on how many of a fixed set of validation frames each actually indexes. The bar is a clear majority, not a margin. An unconstrained refinement holds no cell parameter fixed, so it can only index at least as many frames as the constrained one, and on genuine symmetry it indexes a few more — a percentage margin therefore demotes real lattices (measured: a genuine <span class="math notranslate nohighlight">\(I\)</span>-centred orthorhombic to <span class="math notranslate nohighlight">\(P1\)</span>). Only a constrained cell that fails outright while its unconstrained cell works distinguishes a false promotion. The intensities settle the space group later regardless (§13).</p> <p>Two further hypotheses are weighed at the same point, both by default. Where two first-pass answers have primitive volumes in a small integer ratio (2–4×) and tie on the validation frames, the frame count has saturated — a spurious axis multiple indexes every frame its true sub-cell does — so the tie is settled at the granularity that does not saturate: <strong>which cell accounts for more of the validation frames’ spots</strong>. That comparison leans toward the smaller cell by construction (indexing is a fractional-Miller test, so multiplying an axis by <span class="math notranslate nohighlight">\(n\)</span> multiplies that axis’s residual by <span class="math notranslate nohighlight">\(n\)</span>), and only a reflection class the larger cell genuinely adds — a real superstructure’s satellite rows — can pay for the loss; the occupancy of that added class is computed and logged beside the decision, deliberately without a threshold, but the spot count is what decides. Separately, the 10 Å floor of §5.2 is applied to the FFT’s per-direction peak search, so a lattice row whose true repeat is below it is reported at its first harmonic and the cell assembled from those harmonics is an exact integer supercell of the true one; a <strong>second first-pass hypothesis with the floor lowered to 5 Å</strong> therefore also runs (rotation only, and not when a cell was given — <code class="docutils literal notranslate"><span class=pre >-C</span></code> already lowers the floor to cover it), and its answer is adopted only when it is an integer sub-cell of the standing one and ties or beats it on the validation frames. Everywhere else the standard pass’s answer stands.</p> <p><strong>A first pass that ends with no usable lattice tries a leaner, shallower one.</strong> A crystal sitting in a crystalline powder floods the pass with spots that are not its own — measured, 2112 spots a frame against the 80 a clean crystal on the same beamline gives — and the FFT then takes its cell out of the powder shells. So the pass is retried over <strong>how much of each frame it reads</strong>: the strongest 30, 80, 200 or all spots an image, each at the file’s resolution and at the resolutions a quarter and a half of this sample’s own spots lie coarser than, with the rings measured in §3.3 set aside where there are any. The rungs are decided late, on the validation-frame count — the way the rotation-axis sign already is — and one is adopted only on a win of a sixth of the frames. The same ladder is asked inside the beam-centre check of §1.4, where a centre that is wrong and a spot list that is too deep otherwise hide each other.</p> <p>The frame count cannot arbitrate an axis harmonic, though: a cell twice as long must place every spot twice as accurately to score the same, and across rungs the bias compounds, since the leanest and shallowest rung is both the one a halved axis scores best on and the one a smaller cell is easiest on. The winning rung is therefore put through the same harmonic arbiter as the hypotheses above — which of two cells accounts for more of the validation frames’ spots — against every rung that cleared the bar and whose primitive volume is a near-integer multiple of the winner’s. A winner that loses that question is a sub-multiple, and the ladder then adopts nothing rather than promoting the rival, which on the crystal this was measured on is a multiple of the true axis in its own right.</p> <p><strong>Note.</strong> In ambiguous or special cases, forcing space group to <span class="math notranslate nohighlight">\(P1\)</span> (no symmetry assumptions) is recommended.</p> </section> <hr class=docutils /> <section id=geometry-and-lattice-refinement > <h2 id=geometry-and-lattice-refinement ><a class=toc-backref href="#id11" role=doc-backlink >7. Geometry and lattice refinement</a><a class=headerlink href="#geometry-and-lattice-refinement" title="Link to this heading"></a></h2> <p>Refinement adjusts experimental geometry and crystal parameters to minimize discrepancies between observed spot reciprocal vectors and those predicted by a lattice model with integer indices.</p> <section id=parameterization > <h3 id=parameterization ><a class=toc-backref href="#id12" role=doc-backlink >7.1 Parameterization</a><a class=headerlink href="#parameterization" title="Link to this heading"></a></h3> <p>The refinement jointly optimizes, depending on mode and constraints:</p> <ul class=simple > <li><p>beam center <span class="math notranslate nohighlight">\((x_\mathrm{beam}, y_\mathrm{beam})\)</span>,</p> <li><p>detector distance <span class="math notranslate nohighlight">\(D\)</span>,</p> <li><p>detector tilt angles (two-angle model; third rotation often held at 0),</p> <li><p>rotation axis direction (for rotation datasets),</p> <li><p>crystal orientation (a global rotation),</p> <li><p>unit-cell parameters, with constraints determined by inferred crystal system.</p> </ul> <p>The detector distance is not refined against one crystal’s spots at all: the positional residual leaves it degenerate with the cell scale, so it is fitted elsewhere - by the rotation post-refinement, which frees it alongside the whole crystal and adds the distance-independent rocking-angle residual that breaks the degeneracy, and by the stills <code class="docutils literal notranslate"><span class=pre >--refine-geometry</span></code> bundle. Per image, the beam centre and the crystal orientation are refined, and the unit cell as well for stills. The first-pass rotation indexing refines the detector tilt and the rotation-axis direction too, against the spots accumulated across the sweep; everywhere else both are held fixed, because on a single crystal a tilt is absorbed almost exactly by the crystal orientation. A lighter <strong>orientation-only</strong> mode refines just the crystal orientation, for stills whose geometry is already trusted. It carries a weak small-rotation prior penalising the whole angle-axis vector (all three components, at a low weight); what it is there for is the poorly-determined out-of-plane component, which is the one the data barely constrain.</p> <p>For higher symmetries, constraints are enforced, e.g.</p> <ul class=simple > <li><p>cubic: <span class="math notranslate nohighlight">\(a=b=c,\ \alpha=\beta=\gamma=90^\circ\)</span>,</p> <li><p>tetragonal: <span class="math notranslate nohighlight">\(a=b\)</span>,</p> <li><p>hexagonal: <span class="math notranslate nohighlight">\(a=b,\ \gamma=120^\circ\)</span>,</p> <li><p>monoclinic (unique axis <span class="math notranslate nohighlight">\(b\)</span>): <span class="math notranslate nohighlight">\(\alpha=\gamma=90^\circ\)</span>, <span class="math notranslate nohighlight">\(\beta\)</span> refined.</p> </ul> </section> <section id=residuals-and-objective > <h3 id=residuals-and-objective ><a class=toc-backref href="#id13" role=doc-backlink >7.2 Residuals and objective</a><a class=headerlink href="#residuals-and-objective" title="Link to this heading"></a></h3> <p>For each indexed spot assigned integer <span class="math notranslate nohighlight">\((h,k,l)\)</span>, compute:</p> <ul class=simple > <li><p>observed reciprocal vector <span class="math notranslate nohighlight">\(\mathbf{s}_\mathrm{obs}\)</span> from its detector position and current geometry,</p> <li><p>predicted reciprocal vector <span class="math notranslate nohighlight">\(\mathbf{s}_\mathrm{pred}(h,k,l;\ \text{lattice params})\)</span>.</p> </ul> <p>Residual is: <span class="math notranslate nohighlight">\( \mathbf{r} = \mathbf{s}_\mathrm{obs} - \mathbf{s}_\mathrm{pred}. \)</span></p> <p>A non-linear least squares solver minimizes <span class="math notranslate nohighlight">\(\sum \|\mathbf{r}\|^2\)</span> over all selected inlier spots.</p> </section> <section id=rotation-datasets-bringing-observations-to-a-common-reference-frame > <h3 id=rotation-datasets-bringing-observations-to-a-common-reference-frame ><a class=toc-backref href="#id14" role=doc-backlink >7.3 Rotation datasets: bringing observations to a common reference frame</a><a class=headerlink href="#rotation-datasets-bringing-observations-to-a-common-reference-frame" title="Link to this heading"></a></h3> <p>For oscillation/rotation data, each image corresponds to a rotation angle <span class="math notranslate nohighlight">\(\phi\)</span> about an axis <span class="math notranslate nohighlight">\(\mathbf{m}_2\)</span>. Observed reciprocal vectors are rotated “back to start” so that all images are refined in a single reference crystal frame: <span class="math notranslate nohighlight">\( \mathbf{s}_\mathrm{obs,ref} = R(\phi)\,\mathbf{s}_\mathrm{obs}, \)</span> where <span class="math notranslate nohighlight">\(R(\phi)\)</span> is the rotation by <span class="math notranslate nohighlight">\(+\phi\)</span> about the goniometer axis <strong>as stored in the file</strong>. The sign is a convention and it is load-bearing: rotating the observations forward by <span class="math notranslate nohighlight">\(+\phi\)</span> means the crystal itself turns by <span class="math notranslate nohighlight">\(-\phi\)</span> about that stored axis, i.e. <span class="math notranslate nohighlight">\(R(\phi)\)</span> is the <em>inverse</em> of the crystal’s own rotation from the reference orientation to frame <span class="math notranslate nohighlight">\(\phi\)</span>. The same convention is why the unmerged-MTZ batch headers and the XDS geometry echo carry the axis <strong>negated</strong> relative to the input file (<a class="reference internal" href="RUGNUX_INTEGRATION.html#the-unmerged-export"><span class="std std-ref">Rugnux ▸ the unmerged export</span></a>) — a reimplementation that takes <span class="math notranslate nohighlight">\(R(\phi)\)</span> as the crystal rotation must use <span class="math notranslate nohighlight">\(R(-\phi)\)</span> here instead. The angle <span class="math notranslate nohighlight">\(\phi\)</span> is taken at the centre of each frame’s oscillation (the frame angle plus half the oscillation width).</p> </section> <section id=multi-stage-tightening-of-inlier-tolerance > <h3 id=multi-stage-tightening-of-inlier-tolerance ><a class=toc-backref href="#id15" role=doc-backlink >7.4 Multi-stage tightening of inlier tolerance</a><a class=headerlink href="#multi-stage-tightening-of-inlier-tolerance" title="Link to this heading"></a></h3> <p>Refinement is performed in stages with decreasing acceptance tolerance for including reflections (three stages, indexing tolerance <span class="math notranslate nohighlight">\(0.3\to0.2\to0.1\)</span>), which stabilizes convergence when starting from imperfect indexing and approximate geometry.</p> <p>The loose first stage necessarily admits some spots that are not reflections of this lattice — the fraction of <em>randomly</em> placed spots inside a fractional-Miller tolerance <span class="math notranslate nohighlight">\(t\)</span> is <span class="math notranslate nohighlight">\(\tfrac{4}{3}\pi t^3\)</span>, i.e. 11 % at <span class="math notranslate nohighlight">\(t=0.3\)</span> — and an unweighted fit lets them pull the orientation. Each residual is therefore weighted by how strong its spot is <strong>for its resolution</strong>: the frame’s spots are cut into equal-count resolution shells and each intensity is divided by its shell median, mapped to <span class="math notranslate nohighlight">\(w^2=r/(1+r)\)</span>. The shell normalisation is what makes this safe — genuine high-resolution spots are legitimately weaker and carry the cell and distance information, so an un-normalised intensity weight would suppress exactly the spots the fit needs. The weight is a property of the spot and never of the current residual, so it does not depend on how far the geometry is from convergence.</p> <p><strong>The rotation chain commits its best round, not the round it stops on.</strong> After the winning candidate is selected, the refinement is run again — solve, re-accumulate the reciprocal-space cloud under the refined geometry, solve again — up to twenty times, and the loop stops on a test of the detector-tilt step. The chain is a trajectory and its last point is not always its best one: every solve ends by fitting only the spots inside its <strong>tightest</strong> gate, so a cell with a direction the data barely constrain (which is what a free cell whose metric is near a Bravais class has) can slide along it, pulling a core of spots tighter while the periphery falls out of the fit altogether. Measured on such a chain, the spots inside the tight gate rise over seventeen rounds while the spots inside the widest gate peak at round three and fall away — and round three is the round that merges at ISa 11.0 against 6.3 and <span class="math notranslate nohighlight">\(R_\mathrm{meas}\)</span> 0.148 against 0.213. Nothing the run consulted could see it: indexed fraction, validation frames, validation spots and the tight-gate count all prefer the overfitted end.</p> <p>So every round is scored on the <strong>widest</strong> gate — the population the first pass selects on and the last pass does not fit, which makes it the one a converged solve is not optimising — and the best-scoring round is committed. Two conditions keep that from acting on noise. The score is a <em>count</em> of spots, so a lead of fewer than <span class="math notranslate nohighlight">\(\sqrt{\text{count}}\)</span> of them leaves the last round standing. And the round taken has to be the less distorted lattice as well as the better-fitting one: the lattice search (§6) is re-asked every round to <strong>measure</strong> how far the cell sits from the ideal metric of the class it matches (imposing that class is measured fatal — the snap puts almost everything outside the refinement’s own gate), and an earlier round is taken only when it matched the <em>same</em> class and sits closer to it. Same class is a precondition and not a precaution: the deviation is a fraction of whichever class’s tolerance admitted it, so two classes’ deviations are not the same quantity, and a round that matched no class reports zero, which means “nothing was asserted” rather than “undistorted”. A chain that has settled scores its rounds within a spot or two of each other and a symmetry-constrained solve holds its distortion at zero throughout, so the rule fires on neither: measured over 914 chains, an earlier round scores higher on 44 % of them and the committed round changes on 1 dataset in 54.</p> <p><strong>A tilt no mounting can have has to prove itself.</strong> The detector tilt is refined freely because on a sweep whose spots reach far enough in <span class="math notranslate nohighlight">\(2\theta\)</span> it is a measurement, and restraining it costs those crystals resolution (measured: 0.13–0.16 Å and up to a quarter of ISa on the crystals whose fitted tilt is largest). What makes it a measurement is the <em>keystone</em> — a tilted plane puts one side of the detector nearer and the other further, so the spots move by an amount that grows with their distance from the beam — and a first pass made of spots reaching a few degrees of <span class="math notranslate nohighlight">\(2\theta\)</span> sees a keystone of a pixel or two at most. To such a fit a tilt is a whole-pattern shift the beam centre imitates exactly, its size is whatever the centroids’ own systematics happen to prefer, and the value it commits then mispredicts the far corner of the detector by tens of pixels against an integration disc of a few: a first pass seeded to <span class="math notranslate nohighlight">\(2\theta = 5°\)</span> committed 2.5° and the run collapsed from 2.8 Å to 6.4 Å. For a tilt of a few tenths of a degree nothing in the spots says whether it is real — the held-out positional residual, the rocking angles and a re-fit at the header tilt were all measured unable to, at the same insignificance on crystals whose tilt is real and on the one whose tilt was the artefact — so below what a mounting can be off square by the fit is trusted as before: a detector is mounted square to the beam to a fraction of a degree, and over 211 datasets the fitted tilt left the file’s by more than 0.56° on one. A chain that has walked more than 1° from the tilt it started at has either measured nothing or found a detector the file misdescribes (that one: a <span class="math notranslate nohighlight">\(2\theta\)</span> arm swung out 12.8° that the file records as square), and at that size the spots <em>do</em> tell the two apart, because a real tilt of degrees has a keystone of tens of pixels over the spots the fit is made of and an artefact has none. So the candidate is refined again from where it started with the tilt held there — the beam centre takes the shift the tilt is equivalent to — and the two are judged as the rounds of one chain are, on the spots each indexes inside the wide gate: the walked tilt stands only when it leads by more than the count’s own noise. Held, not bounded — a box the fit lands on is the same wrong answer at a smaller size. The log says what the walked tilt would have moved the far corner by and how the two counts came out; where it was refused, the report’s <code class="docutils literal notranslate"><span class=pre >REFINED_DETECTOR_TILT</span></code> is the tilt the pass started at and <code class="docutils literal notranslate"><span class=pre >REFUSED_DETECTOR_TILT</span></code> the tilt the fit had walked to.</p> </section> <section id=rotation-geometry-post-refinement-two-pass > <h3 id=rotation-geometry-post-refinement-two-pass ><a class=toc-backref href="#id16" role=doc-backlink >7.5 Rotation geometry post-refinement (two-pass)</a><a class=headerlink href="#rotation-geometry-post-refinement-two-pass" title="Link to this heading"></a></h3> <p>The refinement above (§7.2) runs per image against that image’s spots. For rotation data an additional <strong>post-refinement</strong> (on by default; <code class="docutils literal notranslate"><span class=pre >--rotation-no-postrefine</span></code> disables it) improves the detector distance, beam centre and crystal cell/axis using <strong>all</strong> frames at once, then re-integrates:</p> <ol class=arabic > <li><p><strong>Pass 1</strong> integrates, scales and merges at the header geometry.</p> <li><p>From pass-1’s integrated reflections, the crystal and the detector are refined together over all frames (Ceres, robust loss) in <strong>one joint fit</strong>, against both residuals at once:</p> <ul class=simple > <li><p>the <strong>positional</strong> detector↔reciprocal residual at each partial’s observed spot, and</p> <li><p>a distance-independent <strong>Ewald excitation</strong> residual at each reflection’s observed rocking centroid <span class="math notranslate nohighlight">\(\phi_\mathrm{obs}\)</span>.</p> </ul> <p>Free: the crystal orientation, the unit cell (every parameter the crystal system leaves free, not one overall scale), the goniometer-axis direction, the detector distance and the beam centre. The positional residual on its own <em>is</em> degenerate with the cell scale — that is why this used to be split into a cell-scale step and a distance step — but the excitation residual does not involve the detector at all, so it fixes the absolute size of the reciprocal lattice and breaks the degeneracy inside the same problem. Splitting it instead cost accuracy twice over: pass 1 frees the whole lattice against a frozen distance, so the distortion it absorbs is <em>anisotropic</em> and no single scale can undo it; and whatever bias is left in that scale goes straight into the distance, which is only ever determined relative to the cell.</p> <p>The fit is <strong>cross-validated</strong> on a deterministic split of the <em>reflections</em> (an avalanche-mixed <span class="math notranslate nohighlight">\(hkl\)</span> hash, not a frame split and not an <span class="math notranslate nohighlight">\(h+k+l\)</span> parity, which would collide with a centering condition and leave the held-out half empty): fitted on one half, committed only if it lowers the held-out residual by more than that residual’s own noise (the standard errors of the two held-out means combined, the bar a round of the geometry walk has to clear) and its two families agree: the positional residual has to fall by more than its own noise, because the positions are the only evidence of the detector geometry a commit hands the next pass, and the excitation residual must not rise by more than its own noise, because it is the only evidence of the cell scale and the positional values outnumber it about three to one. That noise is the <em>paired</em> standard error — both geometries are read on the same held-out reflections, so what a change has to beat is the scatter of the change each reflection sees; bare signs stood here, and were a coin toss wherever a family did not move — and the move stays inside its bounds: every free cell angle within 1° and the beam centre within 15 px of the nearest centre anything already believes.</p> <p>The <strong>distance and the cell lengths are bounded one step at a time, not as a whole</strong>. One per cent was once a cap on the entire move, and as a cap it was the opposite of its job — a header is most worth correcting when it is most wrong, and a geometry genuinely several per cent out could never be reached (measured: a refused fit of 310.000 → 305.692 mm whose cell landed within 0.06 % of the deposited one). It is a <strong>trust region</strong> instead. The first solve is asked in the wide box around nominal exactly as before, so a fit that settles within one step commits unchanged; a fit that wants more is re-fitted as a <em>walk</em> of one-per-cent steps, each seeded where the last arrived and each required to lower the held-out residual, stopping where a step stops paying. A walk that uses every step it is allowed has not settled — it stopped because it ran out of steps, not because it arrived — and is refused, which is the runaway the cap stood in for, tested where it can be seen. A move of more than one step is additionally <strong>ratified by re-indexing at where it arrived</strong>: that is what separates the failure the cap was really aimed at (a second lattice, whose spots bias every cross-validation fold identically) from a wrong header, since a second lattice does not index better at the new geometry and a real distance error does.</p> <p>The geometry the run commits is re-fitted on all the reflections once the held-out half has approved it — a walk from where it arrived, one step wide; everything else from nominal in the wide box — and the bounds are asked again of that fit rather than only of the half that earned it. A move outside them leaves the geometry at nominal, as every other refusal does, and the refusal says so in the report (<code class="docutils literal notranslate"><span class=pre >POSTREFINE_REFUSED</span></code>, with what the fit wanted) rather than passing silently. Detector tilt is held fixed, being gauge-coupled to the crystal orientation on a single crystal.</p> <p><strong>Whether the data determine the distance at all is asked, not assumed.</strong> At a detector far enough away that no reflection reaches more than a few degrees of <span class="math notranslate nohighlight">\(2\theta\)</span>, a longer distance and a larger cell move every spot the same way to first order — the difference is of order <span class="math notranslate nohighlight">\(\sin^2\theta\)</span> of the spot’s own position, about 0.4 px rms per per cent of distance over a 2M detector at 820 mm against 2–3 px at the distances a crystal is usually collected at. The joint fit then finds a distance/cell pair that fits its own spot positions a little better than the header, commits it, and the pass re-integrated there finds the next pair: a walk along the degenerate direction that the realised residual never ratifies (measured on such a sweep: a header at 820 mm walked to 846 mm with the cell 3.3 % too large, the realised held-out residual flat at every round). So the same fit is asked once more with the distance <strong>held at the header</strong>, every other block as free as before — the nested hypothesis “the header distance is right” — and the two are compared on the one residual family that can tell them apart: the <strong>excitation</strong> residual. It never involves the detector, so it is blind to the distance itself; what it sees is the cell scale, and a held fit at a wrong header distance is forced into a wrong cell scale by the spot positions, which the rocking angles then refuse (measured: a header 1.4 % long leaves the held fit’s excitation residual seventeen times the free fit’s). Where freeing the distance lowers the held-out excitation residual below the held fit’s by more than that residual’s own standard error, the free fit is committed exactly as before; where it does not, the held fit is — header distance, refined beam, cell, orientation and axis — and the report says so (<code class="docutils literal notranslate"><span class=pre >POSTREFINE_DISTANCE_HELD</span></code>). The positional residual is deliberately not consulted for this: it is the family whose in-fit gain along the degenerate direction re-integration erases, and pooled with the excitation family it either drowns a decisive excitation gain in its own noise (a 54 % excitation gain read as 9 % pooled against a 9 % noise) or lends the degenerate direction a gain that is not there. Nothing is tuned here: the only input is the standard error of the residual itself, the same noise the geometry walk’s rounds have to beat. The wavelength is never refined on a single crystal for the same reason in its exact form: it scales the spot positions and the rocking angles identically to the cell, so no sweep can tell the two apart at any <span class="math notranslate nohighlight">\(2\theta\)</span>.</p> <li><p><strong>Pass 2</strong> re-indexes de novo and re-integrates at the committed geometry. Only the <strong>detector distance and beam centre</strong> carry over: the refined cell, orientation and axis are what make the distance identifiable, but pass 2 re-indexes from scratch, so they are not propagated. Where the re-index finds a <strong>different</strong> lattice — the two compared on their Niggli-reduced primitive edges, within 2 %, so a symmetric setting is never told apart from its own primitive cell — pass 1’s lattice is scored at the refined geometry as a hypothesis of its own, and integrated when it indexes more validation frames; the same lattice found again is kept as the re-index refined it. The run likewise falls back to pass 1’s lattice where the re-index indexes too few frames, and integrates it at the refined geometry — and the cell is then <strong>scaled to the distance it will be used at</strong>, since a real-space cell is measured against the distance its spots were seen at, and carrying it across a distance change otherwise scales the whole cell by the ratio of the two. The orientation is untouched.</p> <p>Pass 2 measures the post-refinement again, and where it still moves the geometry the run <strong>walks</strong>: it re-indexes and re-integrates at what the fit asks for, and repeats. A round is kept only for what it <em>realises</em>, not for what the fit predicts, and it can realise a gain in two ways, either of which has to beat its own noise: a lower held-out residual (the standard errors of the two means combined), or a larger share of the validation spots on the lattice beyond chance (the binomial noise of the two shares, z = 3.29). The residual alone misses exactly the errors that cost resolution: it is dominated by the low-resolution reflections, where a distance and the cell scale that compensates it move every spot alike, and its centroids are taken inside a disc centred on the prediction, so they follow the prediction part of the way; the high-resolution validation spots are the first to leave the lattice (measured: 0.6 % of distance read 0.74 of the residual’s noise and 70.4 % against 58.1 % of the validation spots, and cost 0.07 Å of resolution). A move of a trust-region step or more starts the walk outright. A smaller one is first tried as two indexing probes on the validation frames — at the fit’s geometry and at the one in hand, each stopping once the lattice is scored — and pays for a re-integrated round only where the fit’s geometry scores higher. The run keeps the best round it reached.</p> </ol> <p>The space group is determined <strong>after</strong> pass 2, on the geometry the run refined, and pass 1 does not search at all: a decision taken on the worse of the two passes and then carried forward is a constraint on the better one, and would have to be reconciled with what pass 2 later found. The guard that chooses which pass is written compares each pass’s <strong>first</strong> merge — <span class="math notranslate nohighlight">\(P1\)</span> on both sides, full resolution range, before the correction surfaces — which both passes produce anyway, so it never compares statistics computed in two different space groups. What it compares there is the <strong>signal each pass measured</strong>: the count of unique reflections merged at <span class="math notranslate nohighlight">\(I/\sigma \ge 2\)</span>, less the count at <span class="math notranslate nohighlight">\(I/\sigma \le -2\)</span>. An empty reflection is as likely to land in either tail, so the second count is the merge’s own measure of how much of the first is noise — which is what makes two passes on different lattices comparable: a pass on an <span class="math notranslate nohighlight">\(n\)</span>-fold supercell merges <span class="math notranslate nohighlight">\(n\)</span> times the reflections, most of them empty, and their noise alone once out-counted the crystal’s own lattice. Self-consistency cannot do this job — against an external arbiter the signal count named the more accurate geometry on 23 of 27 arm-dataset pairs where <span class="math notranslate nohighlight">\(R_\mathrm{meas}\)</span>, <span class="math notranslate nohighlight">\(CC_{1/2}\)</span> and ISa managed 13, a coin flip — and that merge’s own <span class="math notranslate nohighlight">\(CC_{1/2}\)</span> least of all, being pooled over the whole range, uncut and uncorrected, so the shells with no signal in them dominate it and they are exactly the shells a geometry move disturbs (it reads 0.13 on a crystal whose data merge at 0.995). The refined pass is sent back only where it merges more unique reflections than its cell can hold, where it measured decisively less signal (10 %, and only where it covers no more of reciprocal space either — a wider integration disk pulls weak reflections in and dilutes the strong fraction without measuring less; between two lattices the reflection count of the smaller is scaled by the integer index of one in the other), or where it lost the axial rows the systematic absences are read off. One index-time veto remains and is keyed to pass 1’s <strong>lattice</strong> rather than its group: a centred pass-1 lattice against a primitive pass-2 one. A pass sent back is pass 1 as it was judged — its whole indexing result is reused, not indexed again de novo at the header geometry, which is a hypothesis nobody had judged.</p> <p>Only pass 2 is written, as the canonical <code class="docutils literal notranslate"><span class=pre >&lt;prefix&gt;_*</span></code> output. Pass 1’s merge exists to give the guard something to judge pass 2 against, so it stops short of the parts of the merge that only fill in a file — the correction surfaces, the twinning and radiation-damage analyses, the R-free flags and the amplitudes — and writes no merged files of its own.</p> <p><strong>Goniometer rotation scale.</strong> A stage that turns further than it was commanded to leaves no trace in the file, because the stored <span class="math notranslate nohighlight">\(\omega\)</span> values <em>are</em> the commanded ones; the excess then presents as the crystal drifting, in this program and in others. The excitation residual already measures it without a new degree of freedom: it rotates by <span class="math notranslate nohighlight">\(-\phi\,\mathbf{u}\)</span> with <span class="math notranslate nohighlight">\(\mathbf{u}\)</span> an <strong>unnormalised</strong> 3-vector, so <span class="math notranslate nohighlight">\(|\mathbf{u}|\)</span> is the factor <span class="math notranslate nohighlight">\(k\)</span> by which the stage actually turned, and normalising the axis throws it away. Pass 1 fits <span class="math notranslate nohighlight">\(k\)</span> as a single parameter on its rocking events, with the crystal and the axis direction held at their committed values and the angle measured from the centre of the sweep. That fit <strong>under-reads</strong> a real error: it only sees the frames the stored angles still track, and a rate error is exactly what stops them tracking the rest. So it is not acted on directly. Between the passes, at the detector geometry pass 2 runs at, the lattice is indexed under the stored angles and under the fitted <span class="math notranslate nohighlight">\(k\)</span>, and each is scored on the validation frames spread over the whole sweep, as the share of their spots it puts on the lattice beyond what it puts there at a wrong spindle angle. The fitted <span class="math notranslate nohighlight">\(k\)</span> is adopted only where it scores higher by more than the binomial noise of the two scores (z = 3.29); the run then integrates and post-refines at it, fits <span class="math notranslate nohighlight">\(k\)</span> again on top of it, and repeats until the next <span class="math notranslate nohighlight">\(k\)</span> no longer scores better - the fixed point of the fit. Otherwise the stored angles stand. The adopted <span class="math notranslate nohighlight">\(k\)</span> drives every later pass (prediction, integration, scaling and the reported oscillation) and is reported as <code class="docutils literal notranslate"><span class=pre >GONIOMETER_ROTATION_SCALE</span></code>, with <code class="docutils literal notranslate"><span class=pre >GONIOMETER_ROTATION_SCALE_SUSPECT=</span> <span class=pre >TRUE</span></code>. <code class="docutils literal notranslate"><span class=pre >--rotation-scale</span> <span class=pre >&lt;k&gt;</span></code> asserts a calibration and skips all of this.</p> </section> <section id=detector-geometry-from-powder-rings > <h3 id=detector-geometry-from-powder-rings ><a class=toc-backref href="#id17" role=doc-backlink >7.6 Detector geometry from powder rings</a><a class=headerlink href="#detector-geometry-from-powder-rings" title="Link to this heading"></a></h3> <p>Everything above fits the geometry to <em>Bragg</em> data, where the beam centre is the weakest parameter: it is gauge-coupled to the crystal orientation, which is why §7.5 bounds it to within 15 px of a centre something already believes rather than letting the spots place it freely. A <strong>powder ring has no orientation to be coupled to</strong>. Where it falls on the detector depends on the geometry and on nothing else, which makes a calibrant — LaB₆, silver behenate, CeO₂, silicon — or even ice an independent constraint on exactly the quantity Bragg data cannot pin.</p> <p>The ring positions are matched to the observed rings and the geometry is refined (Ceres, five parameters: beam centre, distance, and the two detector tilts) so that the <span class="math notranslate nohighlight">\(|s|\)</span> predicted at each observed ring point matches the ring it belongs to. The two tilts can be held fixed (<code class="docutils literal notranslate"><span class=pre >rugnux</span> <span class=pre >--no-refine-tilt</span></code>, the viewer’s <em>Refine detector tilt</em> tick box), leaving a three-parameter fit: a tilt a downstream program cannot express is better left out of the fit than refined and then dropped, since the centre and the distance of a tilted fit have already absorbed it.</p> <p><strong>Calibrants.</strong> LaB₆, silver behenate, CeO₂ and silicon are held as unit cells and their rings enumerated from them. Ice is held as the hexagonal-ice ring positions of §3.3 instead — measured to 1.522 Å, calculated below it — because hexagonal ice is <span class="math notranslate nohighlight">\(P6_3/mmc\)</span> with oxygen on <span class="math notranslate nohighlight">\(4f\)</span> and enumerating <span class="math notranslate nohighlight">\(hkl\)</span> from its cell would emit rings the oxygen sublattice extinguishes. A calibrant is therefore a list of ring <span class="math notranslate nohighlight">\(q\)</span> values throughout, not a cell.</p> <p><strong>What a ring can and cannot determine.</strong> A ring is a conic centred on the beam, so a wrong centre makes its apparent radius oscillate once per turn, <span class="math notranslate nohighlight">\(r(\phi)=R+\delta_x\cos\phi+\delta_y\sin\phi\)</span>, with the <strong>same amplitude on every ring</strong>. A detector tilt <span class="math notranslate nohighlight">\(\beta\)</span> produces a <span class="math notranslate nohighlight">\(\cos\phi\)</span> term too — not the <span class="math notranslate nohighlight">\(\cos2\phi\)</span> one might expect — but one that grows as the ring’s radius <em>squared</em>, <span class="math notranslate nohighlight">\(r(\phi)=R+(R^2/F)(\beta_x\cos\phi+\beta_y\sin\phi)\)</span>; the true <span class="math notranslate nohighlight">\(\cos2\phi\)</span> term is <span class="math notranslate nohighlight">\(O(R^3\beta^2/F^2)\)</span>, hundredths of a pixel. The two are therefore separated by how the amplitude scales with radius, which needs <strong>at least two rings</strong> — on a single ring they are exactly degenerate. None of this uses the calibrant’s <span class="math notranslate nohighlight">\(d\)</span>-spacings, so the centre is determined without assuming anything about the standard.</p> <p>The <strong>distance</strong> is different: it follows from <span class="math notranslate nohighlight">\(r=F\tan2\theta\)</span> with <span class="math notranslate nohighlight">\(\sin\theta=\lambda/2d\)</span>, so a fractional error in the lattice constant passes straight into it, and the <span class="math notranslate nohighlight">\(\lambda\)</span>–<span class="math notranslate nohighlight">\(F\)</span> pair is separated only by the curvature of <span class="math notranslate nohighlight">\(\tan(2\arcsin(\lambda/2d))\)</span> across the rings — <span class="math notranslate nohighlight">\(\partial\ln r/\partial\ln F=1\)</span> at every ring against <span class="math notranslate nohighlight">\(\partial\ln r/\partial\ln\lambda=4\tan\theta/\sin4\theta\)</span>, which runs from about 1.05 at low angle to 1.43 at high. That lever collapses as the detector moves back and the rings crowd into small <span class="math notranslate nohighlight">\(2\theta\)</span>, so distance is a short-distance measurement and the wavelength is better calibrated by other means.</p> <p><strong>Reading the rings.</strong> The ring points come from one of two measurements, both accumulated over <strong>every processed image</strong> rather than one. The default reads the <strong>azimuthally-binned profile (§2) summed over the run</strong>: for each ring and each azimuthal sector, the radial peak is fitted against a locally interpolated background and the measured <span class="math notranslate nohighlight">\((q,\phi)\)</span> mapped back through the current geometry to the pixel it came from. The alternative pools the <strong>spot lists</strong>, which samples each arc wherever the spot finder’s threshold happens to bite. The accumulated profile is the same size however many images went into it; the pooled spot list is capped, each image contributing an equal share.</p> <p>A plain radial profile — one azimuthal sector — has averaged the ring over every direction and carries no centre at all, so the profile route requires at least four sectors and uses 32 by default. Sixteen to thirty-two are enough; beyond that the limit is the ring’s own texture, not counting statistics.</p> <p>The extraction window around a ring is capped at half the gap to its neighbour, because the background under a peak is taken from the ends of that window: hexagonal ice has a triplet of rings (1.947, 1.916 and 1.882 Å) whose neighbours sit only 0.05–0.06 Å⁻¹ apart in <span class="math notranslate nohighlight">\(q = 2\pi/d\)</span>, which a fixed window merges into a single peak. Where only one ring is in reach the two tilts are held at their input values rather than fitted, since on a single ring they are degenerate with the centre (above) and the fit would otherwise trade the centre away for them.</p> </section> </section> </section> </article> </div> </div> </main> </div> <footer class=md-footer > <div class=md-footer-nav > <nav class="md-footer-nav__inner md-grid"> <a href=CPU_DATA_ANALYSIS_IMAGE.html title="Data analysis: from images to spots (§0–§3)" class="md-flex md-footer-nav__link md-footer-nav__link--prev" rel=prev > <div class="md-flex__cell md-flex__cell--shrink"> <i class="md-icon md-icon--arrow-back md-footer-nav__button"></i> </div> <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"> <span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Previous" </span> Data analysis: from images to spots (§0–§3) </span> </div> </a> <a href=CPU_DATA_ANALYSIS_INTEGRATION.html title="Data analysis: integration, scaling and merging (§8–§12)" class="md-flex md-footer-nav__link md-footer-nav__link--next" rel=next > <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"><span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Next" </span> Data analysis: integration, scaling and merging (§8–§12) </span> </div> <div class="md-flex__cell md-flex__cell--shrink"><i class="md-icon md-icon--arrow-forward md-footer-nav__button"></i> </div> </a> </nav> </div> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-footer-copyright > <div class=md-footer-copyright__highlight > &#169; Copyright 2024, Paul Scherrer Institute. </div> Created using <a href="http://www.sphinx-doc.org/">Sphinx</a> 8.1.3. and <a href="https://github.com/bashtage/sphinx-material/">Material for Sphinx</a> </div> </div> </div> </footer> <script src="_static/javascripts/application.js"></script> <script>app.initialize({version: "1.0.4", url: {base: ".."}})</script>