mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests 1 Actions 1 Packages Projects Releases 72 Wiki Activity
Files
889e6df1d55b794058d41450b9b50ccbeb0b0ef5
Jungfraujoch/HDF5.html
T
leonarski_f 889e6df1d5 Deploy site
2026-08-28 00:42:58 +00:00

18 lines
136 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html> <html lang=en data-content_root="./"> <meta charset=utf-8 /> <meta name=viewport content="width=device-width, initial-scale=1.0" /><meta name=viewport content="width=device-width, initial-scale=1" /> <meta name=viewport content="width=device-width,initial-scale=1"> <meta http-equiv=x-ua-compatible content="ie=edge"> <meta name="lang:clipboard.copy" content="Copy to clipboard"> <meta name="lang:clipboard.copied" content="Copied to clipboard"> <meta name="lang:search.language" content=en > <meta name="lang:search.pipeline.stopwords" content=True > <meta name="lang:search.pipeline.trimmer" content=True > <meta name="lang:search.result.none" content="No matching documents"> <meta name="lang:search.result.one" content="1 matching document"> <meta name="lang:search.result.other" content="# matching documents"> <meta name="lang:search.tokenizer" content="[\s\-]+"> <link href="https://fonts.gstatic.com/" rel=preconnect crossorigin> <link href="https://fonts.googleapis.com/css?family=Roboto+Mono:400,500,700|Roboto:300,400,400i,700&display=fallback" rel=stylesheet > <style> body, input { font-family: "Roboto", "Helvetica Neue", Helvetica, Arial, sans-serif } code, kbd, pre { font-family: "Roboto Mono", "Courier New", Courier, monospace } </style> <link rel=stylesheet href="_static/stylesheets/application.css"/> <link rel=stylesheet href="_static/stylesheets/application-palette.css"/> <link rel=stylesheet href="_static/stylesheets/application-fixes.css"/> <link rel=stylesheet href="_static/fonts/material-icons.css"/> <meta name=theme-color content="#3f51b5"> <script src="_static/javascripts/modernizr.js"></script> <title>HDF5 / NeXus data format &#8212; Jungfraujoch 1.0.0-rc.165 documentation</title> <link rel=stylesheet type="text/css" href="_static/pygments.css?v=83e35b93" /> <link rel=stylesheet type="text/css" href="_static/material.css?v=79c92029" /> <script src="_static/documentation_options.js?v=fd7baf7d"></script> <script src="_static/doctools.js?v=9bcbadda"></script> <script src="_static/sphinx_highlight.js?v=dc90522c"></script> <link rel=icon href="_static/jfjoch.png"/> <link rel=index title=Index href=genindex.html /> <link rel=search title=Search href=search.html /> <link rel=next title="Data streams" href=IMAGE_STREAM.html /> <link rel=prev title="CBOR messages" href=CBOR.html /> <body dir=ltr data-md-color-primary=indigo data-md-color-accent=lime> <svg class=md-svg > <defs data-children-count=0 > <svg xmlns="http://www.w3.org/2000/svg" width=500 height=500 viewBox="0 0 500 500" id=__gitlab ><path fill=currentColor d="M93.667 473.347l90.684-279.097H2.983l90.684 279.097z" transform="translate(156.198 1.16)"/><path fill=currentColor d="M221.333 473.345L130.649 194.25H3.557l217.776 279.095z" transform="translate(28.531 1.16)" opacity=.7 /><path fill=currentColor d="M32 195.155L4.441 279.97a18.773 18.773 0 0 0 6.821 20.99l238.514 173.29L32 195.155z" transform="translate(.089 .256)" opacity=.5 /><path fill=currentColor d="M2.667-84.844h127.092L75.14-252.942c-2.811-8.649-15.047-8.649-17.856 0L2.667-84.844z" transform="translate(29.422 280.256)"/><path fill=currentColor d="M2.667 473.345L93.351 194.25h127.092L2.667 473.345z" transform="translate(247.198 1.16)" opacity=.7 /><path fill=currentColor d="M221.334 195.155l27.559 84.815a18.772 18.772 0 0 1-6.821 20.99L3.557 474.25l217.777-279.095z" transform="translate(246.307 .256)" opacity=.5 /><path fill=currentColor d="M130.667-84.844H3.575l54.618-168.098c2.811-8.649 15.047-8.649 17.856 0l54.618 168.098z" transform="translate(336.974 280.256)"/></svg> </defs> </svg> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer > <input class=md-toggle data-md-toggle=search type=checkbox id=__search > <label class=md-overlay data-md-component=overlay for=__drawer ></label> <a href="#HDF5" tabindex=1 class=md-skip > Skip to content </a> <header class=md-header data-md-component=header > <nav class="md-header-nav md-grid"> <div class="md-flex navheader"> <div class="md-flex__cell md-flex__cell--shrink"> <a href=index.html title="Jungfraujoch 1.0.0-rc.165 documentation" class="md-header-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--menu md-header-nav__button" for=__drawer ></label> </div> <div class="md-flex__cell md-flex__cell--stretch"> <div class="md-flex__ellipsis md-header-nav__title" data-md-component=title > <span class=md-header-nav__topic >PSI Jungfraujoch</span> <span class=md-header-nav__topic > HDF5 / NeXus data format </span> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--search md-header-nav__button" for=__search ></label> <div class=md-search data-md-component=search role=dialog > <label class=md-search__overlay for=__search ></label> <div class=md-search__inner role=search > <form class=md-search__form action=search.html method=get name=search > <input type=text class=md-search__input name=q placeholder=""Search"" autocapitalize=off autocomplete=off spellcheck=false data-md-component=query data-md-state=active > <label class="md-icon md-search__icon" for=__search ></label> <button type=reset class="md-icon md-search__icon" data-md-component=reset tabindex=-1 > &#xE5CD; </button> </form> <div class=md-search__output > <div class=md-search__scrollwrap data-md-scrollfix> <div class=md-search-result data-md-component=result > <div class=md-search-result__meta > Type to start searching </div> <ol class=md-search-result__list ></ol> </div> </div> </div> </div> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <div class=md-header-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> </div> <script src="_static/javascripts/version_dropdown.js"></script> <script> var json_loc = ""versions.json"", target_loc = "../", text = "Versions"; $( document ).ready( add_version_dropdown(json_loc, target_loc, text)); </script> </div> </nav> </header> <div class=md-container > <nav class=md-tabs data-md-component=tabs > <div class="md-tabs__inner md-grid"> <ul class=md-tabs__list > <li class=md-tabs__item ><a href=index.html class=md-tabs__link >Jungfraujoch 1.0.0-rc.165 documentation</a> </ul> </div> </nav> <main class=md-main > <div class="md-main__inner md-grid" data-md-component=container > <div class="md-sidebar md-sidebar--primary" data-md-component=navigation > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--primary" data-md-level=0 > <label class="md-nav__title md-nav__title--site" for=__drawer > <a href=index.html title="Jungfraujoch 1.0.0-rc.165 documentation" class="md-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> <a href=index.html title="Jungfraujoch 1.0.0-rc.165 documentation">PSI Jungfraujoch</a> </label> <div class=md-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> <ul class=md-nav__list > <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >General</span></span> <li class=md-nav__item > <a href=ACKNOWLEDGEMENT.html class=md-nav__link >Acknowledgements</a> <li class=md-nav__item > <a href=LICENSE.html class=md-nav__link >License</a> <li class=md-nav__item > <a href=THIRD_PARTY_NOTICES.html class=md-nav__link >Third-party software notices</a> <li class=md-nav__item > <a href=DETECTORS.html class=md-nav__link >Supported detectors</a> <li class=md-nav__item > <a href="DETECTORS.html#dectris-detectors" class=md-nav__link >DECTRIS detectors</a> <li class=md-nav__item > <a href=HARDWARE.html class=md-nav__link >Hardware requirements</a> <li class=md-nav__item > <a href=SOFTWARE.html class=md-nav__link >Software requirements</a> <li class=md-nav__item > <a href=VERSIONING.html class=md-nav__link >Semantic versioning</a> <li class=md-nav__item > <a href=DEPLOYMENT.html class=md-nav__link >Deployment</a> <li class=md-nav__item > <a href=SECURITY.html class=md-nav__link >Security</a> <li class=md-nav__item > <a href=RELEASE_CONTENTS.html class=md-nav__link >Release contents</a> <li class=md-nav__item > <a href=REPOSITORIES.html class=md-nav__link >Linux package repositories</a> <li class=md-nav__item > <a href=NAMING.html class=md-nav__link >Naming</a> <li class=md-nav__item > <a href=CHANGELOG.html class=md-nav__link >Changelog</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Software</span></span> <li class=md-nav__item > <a href=JFJOCH_BROKER.html class=md-nav__link >jfjoch_broker</a> <li class=md-nav__item > <a href=JFJOCH_WRITER.html class=md-nav__link >jfjoch_writer</a> <li class=md-nav__item > <a href=RUGNUX.html class=md-nav__link >rugnux</a> <li class=md-nav__item > <a href=JFJOCH_VIEWER.html class=md-nav__link >jfjoch_viewer</a> <li class=md-nav__item > <a href=SOFTWARE_INTEGRATION.html class=md-nav__link >Integration with MX data processing software</a> <li class=md-nav__item > <a href=TOOLS.html class=md-nav__link >Tools</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >FPGA</span></span> <li class=md-nav__item > <a href=FPGA.html class=md-nav__link >FPGA smartNIC</a> <li class=md-nav__item > <a href=FPGA_LICENSE.html class=md-nav__link >FPGA license</a> <li class=md-nav__item > <a href=FPGA_DESIGN.html class=md-nav__link >FPGA data flow</a> <li class=md-nav__item > <a href=FPGA_NETWORK.html class=md-nav__link >FPGA network</a> <li class=md-nav__item > <a href=FPGA_PCIE_DRIVER.html class=md-nav__link >FPGA PCIe driver</a> <li class=md-nav__item > <a href=FPGA_SETTINGS.html class=md-nav__link >FPGA advanced reference</a> <li class=md-nav__item > <a href=FPGA_DATA_ANALYSIS.html class=md-nav__link >FPGA data analysis</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Reference</span></span> <li class=md-nav__item > <a href=DETECTOR_GEOMETRY.html class=md-nav__link >Detector geometry</a> <li class=md-nav__item > <a href=CPU_DATA_ANALYSIS.html class=md-nav__link >CPU-side crystallographic data analysis (Jungfraujoch)</a> <li class=md-nav__item > <a href=OPENAPI.html class=md-nav__link >OpenAPI</a> <li class=md-nav__item > <a href=OPENAPI_SPECS.html class=md-nav__link >OpenAPI specification</a> <li class=md-nav__item > <a href=CBOR.html class=md-nav__link >CBOR messages</a> <li class=md-nav__item > <input class="md-toggle md-nav__toggle" data-md-toggle=toc type=checkbox id=__toc > <label class="md-nav__link md-nav__link--active" for=__toc > HDF5 / NeXus data format </label> <a href="#" class="md-nav__link md-nav__link--active">HDF5 / NeXus data format</a> <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> <ul class=md-nav__list > <li class=md-nav__item > <a href="#motivation-derived-metadata-and-fair-data" class=md-nav__link >1. Motivation: derived metadata and FAIR data</a> <li class=md-nav__item > <a href="#file-layout" class=md-nav__link >2. File layout</a> <li class=md-nav__item > <a href="#nxmx-standard-content" class=md-nav__link >3. NXmx-standard content</a> <li class=md-nav__item > <a href="#extensions-beyond-nxmx" class=md-nav__link >4. Extensions beyond NXmx</a> <li class=md-nav__item > <a href="#notes" class=md-nav__link >5. Notes</a> </ul> <li class=md-nav__item > <a href=IMAGE_STREAM.html class=md-nav__link >Data streams</a> <li class=md-nav__item > <a href=PIXEL_MASK.html class=md-nav__link >Pixel mask</a> <li class=md-nav__item > <a href=WEB_FRONTEND.html class=md-nav__link >Web frontend</a> <li class=md-nav__item > <a href=TESTS.html class=md-nav__link >Tests</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >OpenAPI Python client</span></span> <li class=md-nav__item > <a href="python_client/README.html" class=md-nav__link >jfjoch-client</a> <li class=md-nav__item > <a href="python_client/README.html#license-clarification" class=md-nav__link >License Clarification</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html" class=md-nav__link >jfjoch_client.DefaultApi</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#cancel-post" class=md-nav__link ><strong>cancel_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-get" class=md-nav__link ><strong>config_azim_int_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-put" class=md-nav__link ><strong>config_azim_int_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-bragg-integration-get" class=md-nav__link ><strong>config_bragg_integration_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-bragg-integration-put" class=md-nav__link ><strong>config_bragg_integration_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-get" class=md-nav__link ><strong>config_dark_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-put" class=md-nav__link ><strong>config_dark_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-get" class=md-nav__link ><strong>config_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-put" class=md-nav__link ><strong>config_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-get" class=md-nav__link ><strong>config_file_writer_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-put" class=md-nav__link ><strong>config_file_writer_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-conversion-post" class=md-nav__link ><strong>config_image_format_conversion_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-get" class=md-nav__link ><strong>config_image_format_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-put" class=md-nav__link ><strong>config_image_format_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-raw-post" class=md-nav__link ><strong>config_image_format_raw_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-get" class=md-nav__link ><strong>config_indexing_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-put" class=md-nav__link ><strong>config_indexing_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-get" class=md-nav__link ><strong>config_instrument_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-put" class=md-nav__link ><strong>config_instrument_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-put" class=md-nav__link ><strong>config_internal_generator_image_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-tiff-put" class=md-nav__link ><strong>config_internal_generator_image_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-get" class=md-nav__link ><strong>config_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-tiff-get" class=md-nav__link ><strong>config_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-get" class=md-nav__link ><strong>config_roi_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-put" class=md-nav__link ><strong>config_roi_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-get" class=md-nav__link ><strong>config_select_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-put" class=md-nav__link ><strong>config_select_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-get" class=md-nav__link ><strong>config_spot_finding_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-put" class=md-nav__link ><strong>config_spot_finding_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-get" class=md-nav__link ><strong>config_user_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-put" class=md-nav__link ><strong>config_user_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-get" class=md-nav__link ><strong>config_user_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-put" class=md-nav__link ><strong>config_user_mask_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-get" class=md-nav__link ><strong>config_zeromq_metadata_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-put" class=md-nav__link ><strong>config_zeromq_metadata_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-get" class=md-nav__link ><strong>config_zeromq_preview_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-put" class=md-nav__link ><strong>config_zeromq_preview_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#deactivate-post" class=md-nav__link ><strong>deactivate_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#detector-status-get" class=md-nav__link ><strong>detector_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#fpga-status-get" class=md-nav__link ><strong>fpga_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-clear-post" class=md-nav__link ><strong>image_buffer_clear_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-cbor-get" class=md-nav__link ><strong>image_buffer_image_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-jpeg-get" class=md-nav__link ><strong>image_buffer_image_jpeg_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-tiff-get" class=md-nav__link ><strong>image_buffer_image_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-start-cbor-get" class=md-nav__link ><strong>image_buffer_start_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-status-get" class=md-nav__link ><strong>image_buffer_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-pusher-status-get" class=md-nav__link ><strong>image_pusher_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#initialize-post" class=md-nav__link ><strong>initialize_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#pedestal-post" class=md-nav__link ><strong>pedestal_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-pedestal-tiff-get" class=md-nav__link ><strong>preview_pedestal_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-bin-get" class=md-nav__link ><strong>preview_plot_bin_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-get" class=md-nav__link ><strong>preview_plot_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#result-scan-get" class=md-nav__link ><strong>result_scan_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#start-post" class=md-nav__link ><strong>start_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-calibration-get" class=md-nav__link ><strong>statistics_calibration_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-data-collection-get" class=md-nav__link ><strong>statistics_data_collection_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-get" class=md-nav__link ><strong>statistics_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#status-get" class=md-nav__link ><strong>status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#trigger-post" class=md-nav__link ><strong>trigger_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#version-get" class=md-nav__link ><strong>version_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-till-done-post" class=md-nav__link ><strong>wait_till_done_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-until-running-post" class=md-nav__link ><strong>wait_until_running_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-event-code-get" class=md-nav__link ><strong>xfel_event_code_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-pulse-id-get" class=md-nav__link ><strong>xfel_pulse_id_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/AzimIntSettings.html" class=md-nav__link >AzimIntSettings</a> <li class=md-nav__item > <a href="python_client/docs/BraggIntegrationSettings.html" class=md-nav__link >BraggIntegrationSettings</a> <li class=md-nav__item > <a href="python_client/docs/BrokerStatus.html" class=md-nav__link >BrokerStatus</a> <li class=md-nav__item > <a href="python_client/docs/CalibrationStatisticsInner.html" class=md-nav__link >CalibrationStatisticsInner</a> <li class=md-nav__item > <a href="python_client/docs/DarkMaskSettings.html" class=md-nav__link >DarkMaskSettings</a> <li class=md-nav__item > <a href="python_client/docs/DatasetSettings.html" class=md-nav__link >DatasetSettings</a> <li class=md-nav__item > <a href="python_client/docs/DatasetSettingsSmargon.html" class=md-nav__link >DatasetSettingsSmargon</a> <li class=md-nav__item > <a href="python_client/docs/DatasetSettingsXrayFluorescenceSpectrum.html" class=md-nav__link >DatasetSettingsXrayFluorescenceSpectrum</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html" class=md-nav__link >jfjoch_client.DefaultApi</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#cancel-post" class=md-nav__link ><strong>cancel_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-get" class=md-nav__link ><strong>config_azim_int_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-put" class=md-nav__link ><strong>config_azim_int_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-bragg-integration-get" class=md-nav__link ><strong>config_bragg_integration_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-bragg-integration-put" class=md-nav__link ><strong>config_bragg_integration_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-get" class=md-nav__link ><strong>config_dark_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-put" class=md-nav__link ><strong>config_dark_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-get" class=md-nav__link ><strong>config_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-put" class=md-nav__link ><strong>config_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-get" class=md-nav__link ><strong>config_file_writer_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-put" class=md-nav__link ><strong>config_file_writer_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-conversion-post" class=md-nav__link ><strong>config_image_format_conversion_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-get" class=md-nav__link ><strong>config_image_format_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-put" class=md-nav__link ><strong>config_image_format_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-raw-post" class=md-nav__link ><strong>config_image_format_raw_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-get" class=md-nav__link ><strong>config_indexing_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-put" class=md-nav__link ><strong>config_indexing_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-get" class=md-nav__link ><strong>config_instrument_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-put" class=md-nav__link ><strong>config_instrument_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-put" class=md-nav__link ><strong>config_internal_generator_image_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-tiff-put" class=md-nav__link ><strong>config_internal_generator_image_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-get" class=md-nav__link ><strong>config_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-tiff-get" class=md-nav__link ><strong>config_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-get" class=md-nav__link ><strong>config_roi_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-put" class=md-nav__link ><strong>config_roi_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-get" class=md-nav__link ><strong>config_select_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-put" class=md-nav__link ><strong>config_select_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-get" class=md-nav__link ><strong>config_spot_finding_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-put" class=md-nav__link ><strong>config_spot_finding_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-get" class=md-nav__link ><strong>config_user_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-put" class=md-nav__link ><strong>config_user_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-get" class=md-nav__link ><strong>config_user_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-put" class=md-nav__link ><strong>config_user_mask_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-get" class=md-nav__link ><strong>config_zeromq_metadata_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-put" class=md-nav__link ><strong>config_zeromq_metadata_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-get" class=md-nav__link ><strong>config_zeromq_preview_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-put" class=md-nav__link ><strong>config_zeromq_preview_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#deactivate-post" class=md-nav__link ><strong>deactivate_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#detector-status-get" class=md-nav__link ><strong>detector_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#fpga-status-get" class=md-nav__link ><strong>fpga_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-clear-post" class=md-nav__link ><strong>image_buffer_clear_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-cbor-get" class=md-nav__link ><strong>image_buffer_image_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-jpeg-get" class=md-nav__link ><strong>image_buffer_image_jpeg_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-tiff-get" class=md-nav__link ><strong>image_buffer_image_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-start-cbor-get" class=md-nav__link ><strong>image_buffer_start_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-status-get" class=md-nav__link ><strong>image_buffer_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-pusher-status-get" class=md-nav__link ><strong>image_pusher_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#initialize-post" class=md-nav__link ><strong>initialize_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#pedestal-post" class=md-nav__link ><strong>pedestal_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-pedestal-tiff-get" class=md-nav__link ><strong>preview_pedestal_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-bin-get" class=md-nav__link ><strong>preview_plot_bin_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-get" class=md-nav__link ><strong>preview_plot_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#result-scan-get" class=md-nav__link ><strong>result_scan_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#start-post" class=md-nav__link ><strong>start_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-calibration-get" class=md-nav__link ><strong>statistics_calibration_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-data-collection-get" class=md-nav__link ><strong>statistics_data_collection_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-get" class=md-nav__link ><strong>statistics_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#status-get" class=md-nav__link ><strong>status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#trigger-post" class=md-nav__link ><strong>trigger_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#version-get" class=md-nav__link ><strong>version_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-till-done-post" class=md-nav__link ><strong>wait_till_done_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-until-running-post" class=md-nav__link ><strong>wait_until_running_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-event-code-get" class=md-nav__link ><strong>xfel_event_code_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-pulse-id-get" class=md-nav__link ><strong>xfel_pulse_id_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/Detector.html" class=md-nav__link >Detector</a> <li class=md-nav__item > <a href="python_client/docs/DetectorList.html" class=md-nav__link >DetectorList</a> <li class=md-nav__item > <a href="python_client/docs/DetectorListElement.html" class=md-nav__link >DetectorListElement</a> <li class=md-nav__item > <a href="python_client/docs/DetectorModule.html" class=md-nav__link >DetectorModule</a> <li class=md-nav__item > <a href="python_client/docs/DetectorModuleDirection.html" class=md-nav__link >DetectorModuleDirection</a> <li class=md-nav__item > <a href="python_client/docs/DetectorPowerState.html" class=md-nav__link >DetectorPowerState</a> <li class=md-nav__item > <a href="python_client/docs/DetectorSelection.html" class=md-nav__link >DetectorSelection</a> <li class=md-nav__item > <a href="python_client/docs/DetectorSettings.html" class=md-nav__link >DetectorSettings</a> <li class=md-nav__item > <a href="python_client/docs/DetectorState.html" class=md-nav__link >DetectorState</a> <li class=md-nav__item > <a href="python_client/docs/DetectorStatus.html" class=md-nav__link >DetectorStatus</a> <li class=md-nav__item > <a href="python_client/docs/DetectorTiming.html" class=md-nav__link >DetectorTiming</a> <li class=md-nav__item > <a href="python_client/docs/DetectorType.html" class=md-nav__link >DetectorType</a> <li class=md-nav__item > <a href="python_client/docs/ErrorMessage.html" class=md-nav__link >ErrorMessage</a> <li class=md-nav__item > <a href="python_client/docs/FileWriterFormat.html" class=md-nav__link >FileWriterFormat</a> <li class=md-nav__item > <a href="python_client/docs/FileWriterSettings.html" class=md-nav__link >FileWriterSettings</a> <li class=md-nav__item > <a href="python_client/docs/FpgaStatusInner.html" class=md-nav__link >FpgaStatusInner</a> <li class=md-nav__item > <a href="python_client/docs/GeomRefinementAlgorithm.html" class=md-nav__link >GeomRefinementAlgorithm</a> <li class=md-nav__item > <a href="python_client/docs/GridScan.html" class=md-nav__link >GridScan</a> <li class=md-nav__item > <a href="python_client/docs/ImageBufferStatus.html" class=md-nav__link >ImageBufferStatus</a> <li class=md-nav__item > <a href="python_client/docs/ImageFormatSettings.html" class=md-nav__link >ImageFormatSettings</a> <li class=md-nav__item > <a href="python_client/docs/ImagePusherStatus.html" class=md-nav__link >ImagePusherStatus</a> <li class=md-nav__item > <a href="python_client/docs/ImagePusherType.html" class=md-nav__link >ImagePusherType</a> <li class=md-nav__item > <a href="python_client/docs/IndexingAlgorithm.html" class=md-nav__link >IndexingAlgorithm</a> <li class=md-nav__item > <a href="python_client/docs/IndexingSettings.html" class=md-nav__link >IndexingSettings</a> <li class=md-nav__item > <a href="python_client/docs/InstrumentMetadata.html" class=md-nav__link >InstrumentMetadata</a> <li class=md-nav__item > <a href="python_client/docs/IntegrationModel.html" class=md-nav__link >IntegrationModel</a> <li class=md-nav__item > <a href="python_client/docs/JfjochSettings.html" class=md-nav__link >JfjochSettings</a> <li class=md-nav__item > <a href="python_client/docs/JfjochStatistics.html" class=md-nav__link >JfjochStatistics</a> <li class=md-nav__item > <a href="python_client/docs/MeasurementStatistics.html" class=md-nav__link >MeasurementStatistics</a> <li class=md-nav__item > <a href="python_client/docs/PcieDevicesInner.html" class=md-nav__link >PcieDevicesInner</a> <li class=md-nav__item > <a href="python_client/docs/PixelMaskStatistics.html" class=md-nav__link >PixelMaskStatistics</a> <li class=md-nav__item > <a href="python_client/docs/Plot.html" class=md-nav__link >Plot</a> <li class=md-nav__item > <a href="python_client/docs/PlotUnitX.html" class=md-nav__link >PlotUnitX</a> <li class=md-nav__item > <a href="python_client/docs/Plots.html" class=md-nav__link >Plots</a> <li class=md-nav__item > <a href="python_client/docs/RoiAzimList.html" class=md-nav__link >RoiAzimList</a> <li class=md-nav__item > <a href="python_client/docs/RoiAzimuthal.html" class=md-nav__link >RoiAzimuthal</a> <li class=md-nav__item > <a href="python_client/docs/RoiBox.html" class=md-nav__link >RoiBox</a> <li class=md-nav__item > <a href="python_client/docs/RoiBoxList.html" class=md-nav__link >RoiBoxList</a> <li class=md-nav__item > <a href="python_client/docs/RoiCircle.html" class=md-nav__link >RoiCircle</a> <li class=md-nav__item > <a href="python_client/docs/RoiCircleList.html" class=md-nav__link >RoiCircleList</a> <li class=md-nav__item > <a href="python_client/docs/RoiDefinitions.html" class=md-nav__link >RoiDefinitions</a> <li class=md-nav__item > <a href="python_client/docs/RotationAxis.html" class=md-nav__link >RotationAxis</a> <li class=md-nav__item > <a href="python_client/docs/ScanResult.html" class=md-nav__link >ScanResult</a> <li class=md-nav__item > <a href="python_client/docs/ScanResultImagesInner.html" class=md-nav__link >ScanResultImagesInner</a> <li class=md-nav__item > <a href="python_client/docs/SpotFindingSettings.html" class=md-nav__link >SpotFindingSettings</a> <li class=md-nav__item > <a href="python_client/docs/StandardDetectorGeometry.html" class=md-nav__link >StandardDetectorGeometry</a> <li class=md-nav__item > <a href="python_client/docs/TcpSettings.html" class=md-nav__link >TcpSettings</a> <li class=md-nav__item > <a href="python_client/docs/UnitCell.html" class=md-nav__link >UnitCell</a> <li class=md-nav__item > <a href="python_client/docs/ZeromqMetadataSettings.html" class=md-nav__link >ZeromqMetadataSettings</a> <li class=md-nav__item > <a href="python_client/docs/ZeromqPreviewSettings.html" class=md-nav__link >ZeromqPreviewSettings</a> <li class=md-nav__item > <a href="python_client/docs/ZeromqSettings.html" class=md-nav__link >ZeromqSettings</a> </ul> </nav> </div> </div> </div> <div class="md-sidebar md-sidebar--secondary" data-md-component=toc > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> </div> </div> </div> <div class=md-content > <article class="md-content__inner md-typeset" role=main > <section class="tex2jax_ignore mathjax_ignore" id=hdf5-nexus-data-format > <h1 id=hdf5--page-root >HDF5 / NeXus data format<a class=headerlink href="#hdf5--page-root" title="Link to this heading"></a></h1> <p>Jungfraujoch stores images and on-the-fly analysis results in HDF5 files that aim to be <a class="reference external" href="https://manual.nexusformat.org/classes/applications/NXmx.html">NXmx</a>-compliant. On top of the NXmx application definition, Jungfraujoch records a substantial amount of <em>derived</em> metadata (spot finding, indexing, integration, azimuthal integration, per-image statistics, timing). These extra entries do not exist in NXmx and are documented here so that the layout is unambiguous and reusable.</p> <p>This page documents the <strong>file layout and the data fields</strong>. The operational behaviour of the writer (running, republishing, file finalisation) is described in <a class="reference internal" href=JFJOCH_WRITER.html ><span class="std std-doc">jfjoch_writer</span></a>. The wire format that feeds the writer is described in <a class="reference internal" href=CBOR.html ><span class="std std-doc">CBOR messages</span></a>; fields below frequently correspond one-to-one to CBOR message fields, and that document is a useful companion for their meaning.</p> <nav class="contents local" id=on-this-page > <p class=topic-title >On this page</p> <ul class=simple > <li><p><a class="reference internal" href="#motivation-derived-metadata-and-fair-data" id=id1 >1. Motivation: derived metadata and FAIR data</a></p> <ul> <li><p><a class="reference internal" href="#two-complementary-layouts-per-image-spots-vs-a-reflection-table" id=id2 >Two complementary layouts: per-image spots vs. a reflection table</a></p> </ul> <li><p><a class="reference internal" href="#file-layout" id=id3 >2. File layout</a></p> <ul> <li><p><a class="reference internal" href="#reprocessing-output-prefix-process-h5" id=id4 >Reprocessing output: <code class="docutils literal notranslate"><span class=pre >&lt;prefix&gt;_process.h5</span></code></a></p> </ul> <li><p><a class="reference internal" href="#nxmx-standard-content" id=id5 >3. NXmx-standard content</a></p> <ul> <li><p><a class="reference internal" href="#entry-nxentry" id=id6 ><code class="docutils literal notranslate"><span class=pre >/entry</span></code> (NXentry)</a></p> <li><p><a class="reference internal" href="#entry-source-nxsource-entry-instrument-nxinstrument" id=id7 ><code class="docutils literal notranslate"><span class=pre >/entry/source</span></code> (NXsource), <code class="docutils literal notranslate"><span class=pre >/entry/instrument</span></code> (NXinstrument)</a></p> <li><p><a class="reference internal" href="#entry-instrument-beam-nxbeam" id=id8 ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/beam</span></code> (NXbeam)</a></p> <li><p><a class="reference internal" href="#entry-instrument-attenuator-nxattenuator" id=id9 ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/attenuator</span></code> (NXattenuator)</a></p> <li><p><a class="reference internal" href="#entry-instrument-detector-nxdetector" id=id10 ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector</span></code> (NXdetector)</a></p> <li><p><a class="reference internal" href="#entry-instrument-detector-transformations-nxtransformations" id=id11 ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/transformations</span></code> (NXtransformations)</a></p> <li><p><a class="reference internal" href="#entry-instrument-detector-module-nxdetector-module" id=id12 ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/module</span></code> (NXdetector_module)</a></p> <li><p><a class="reference internal" href="#entry-sample-nxsample" id=id13 ><code class="docutils literal notranslate"><span class=pre >/entry/sample</span></code> (NXsample)</a></p> <li><p><a class="reference internal" href="#entry-data-nxdata" id=id14 ><code class="docutils literal notranslate"><span class=pre >/entry/data</span></code> (NXdata)</a></p> </ul> <li><p><a class="reference internal" href="#extensions-beyond-nxmx" id=id15 >4. Extensions beyond NXmx</a></p> <ul> <li><p><a class="reference internal" href="#entry-mx-spot-finding-and-indexing-cxi-style" id=id16 >4.1 <code class="docutils literal notranslate"><span class=pre >/entry/MX</span></code> — spot finding and indexing (CXI-style)</a></p> <li><p><a class="reference internal" href="#entry-reflections-integrated-reflections-nxreflections" id=id17 >4.2 <code class="docutils literal notranslate"><span class=pre >/entry/reflections</span></code> — integrated reflections (NXreflections)</a></p> <li><p><a class="reference internal" href="#entry-azint-azimuthal-integration" id=id18 >4.3 <code class="docutils literal notranslate"><span class=pre >/entry/azint</span></code> — azimuthal integration</a></p> <li><p><a class="reference internal" href="#entry-roi-regions-of-interest-per-image-results" id=id19 >4.4 <code class="docutils literal notranslate"><span class=pre >/entry/roi</span></code> — regions of interest (per-image results)</a></p> <li><p><a class="reference internal" href="#entry-roi-defs-roi-definitions-master-file" id=id20 >4.4.1 <code class="docutils literal notranslate"><span class=pre >/entry/roi_defs</span></code> — ROI definitions (master file)</a></p> <li><p><a class="reference internal" href="#entry-image-per-image-pixel-statistics" id=id21 >4.5 <code class="docutils literal notranslate"><span class=pre >/entry/image</span></code> — per-image pixel statistics</a></p> <li><p><a class="reference internal" href="#entry-profiling-per-image-timing" id=id22 >4.6 <code class="docutils literal notranslate"><span class=pre >/entry/profiling</span></code> — per-image timing</a></p> <li><p><a class="reference internal" href="#entry-detector-acquisition-diagnostics-data-file" id=id23 >4.7 <code class="docutils literal notranslate"><span class=pre >/entry/detector</span></code> — acquisition diagnostics (data file)</a></p> <li><p><a class="reference internal" href="#entry-xfel-pulsed-source-metadata" id=id24 >4.8 <code class="docutils literal notranslate"><span class=pre >/entry/xfel</span></code> — pulsed-source metadata</a></p> <li><p><a class="reference internal" href="#other-collections" id=id25 >4.9 Other collections</a></p> <li><p><a class="reference internal" href="#non-standard-fields-inside-the-nxmx-detector-group" id=id26 >4.10 Non-standard fields inside the NXmx detector group</a></p> <li><p><a class="reference internal" href="#user-supplied-metadata-header-appendix-and-image-appendix" id=id27 >4.11 User-supplied metadata: <code class="docutils literal notranslate"><span class=pre >header_appendix</span></code> and <code class="docutils literal notranslate"><span class=pre >image_appendix</span></code></a></p> </ul> <li><p><a class="reference internal" href="#notes" id=id28 >5. Notes</a></p> </ul> </nav> <section id=motivation-derived-metadata-and-fair-data > <h2 id=motivation-derived-metadata-and-fair-data ><a class=toc-backref href="#id1" role=doc-backlink >1. Motivation: derived metadata and FAIR data</a><a class=headerlink href="#motivation-derived-metadata-and-fair-data" title="Link to this heading"></a></h2> <p>The goal of Jungfraujoch is not only to store high-throughput datasets efficiently, but to keep them findable, accessible, interoperable and reusable (FAIR). Jungfraujoch is used for both <strong>rotation</strong> macromolecular crystallography (single- and multi-crystal, including fine-sliced and helical scans) and <strong>serial</strong> crystallography (stills, grid scans); the same concerns apply to both:</p> <ul class=simple > <li><p><strong>Findability.</strong> Raw diffraction images carry almost no descriptive metadata about <em>content</em>. Quantities such as background level, number of diffraction spots, or indexing outcome let a user judge the quality and relevance of a dataset <em>before</em> inspecting the raw images.</p> <li><p><strong>Accessibility at scale.</strong> A single experiment can span tens to hundreds of terabytes. Standard retrieval (e.g. HTTP) makes a dataset <em>available</em> but not <em>inspectable</em> — users would otherwise have to download a large fraction of the data just to decide whether it is useful. Compact derived representations make discovery, assessment and reuse feasible.</p> </ul> <p>Because Jungfraujoch couples acquisition with real-time analysis used to <em>steer</em> experiments, transparency and reproducibility of that analysis matter. As a minimum the writer therefore preserves spot-finding and indexing results together with the filters that were applied, and it can retain an unbiased, down-sampled reference set of unfiltered images for validation and reuse.</p> <section id=two-complementary-layouts-per-image-spots-vs-a-reflection-table > <h3 id=two-complementary-layouts-per-image-spots-vs-a-reflection-table ><a class=toc-backref href="#id2" role=doc-backlink >Two complementary layouts: per-image spots vs. a reflection table</a><a class=headerlink href="#two-complementary-layouts-per-image-spots-vs-a-reflection-table" title="Link to this heading"></a></h3> <p>Jungfraujoch stores analysis products in two shapes, matching how each is accessed.</p> <p><strong>Per-image spot finding / indexing.</strong> Spot finding and indexing are inherently <em>image-centric</em> — the natural query is “give me the spots for image <em>n</em>” — and this holds for serial stills and for rotation frames alike. For these products Jungfraujoch adopts a layout similar to the <a class="reference external" href="https://www.cxidb.org">Coherent X-ray Imaging (CXI) data bank</a> (Maia, 2012) and the convention understood by <a class="reference external" href="https://www.desy.de/~twhite/crystfel/">CrystFEL</a>: spot properties (position, intensity, Miller index, …) are stored in fixed-size two-dimensional arrays indexed by image number, with each image allocated room for up to a predefined maximum number of spots. These dense arrays are addressed with ordinary HDF5 hyperslab reads, so the spots of a single image are retrieved without traversing variable-length structures. The cost is some storage overhead for unused slots (padded with sentinels), which is acceptable for the access pattern.</p> <p><strong>Integrated reflections.</strong> Integrated intensities are naturally a <em>dataset-wide</em> table, which is exactly the model of the NeXus <a class="reference external" href="https://manual.nexusformat.org/classes/base_classes/NXreflections.html">NXreflections</a> base class. This fits rotation crystallography well, and Jungfraujoch uses NXreflections for its integration results (see §4.2 below). We deliberately do <em>not</em> force spot finding/indexing into a single experiment-wide table: across the hundreds of thousands of patterns typical of serial — or fine-sliced rotation — experiments, that would require aggregating the whole experiment before the spots of one image can be read. We encourage the community to develop standardised NeXus application definitions for image-centric crystallography products that combine NeXus interoperability with the access patterns and scale of modern high-throughput experiments.</p> </section> </section> <section id=file-layout > <h2 id=file-layout ><a class=toc-backref href="#id3" role=doc-backlink >2. File layout</a><a class=headerlink href="#file-layout" title="Link to this heading"></a></h2> <p>A run is written as one <strong>master file</strong> plus, depending on the format, one or more <strong>data files</strong>:</p> <div class="highlight-default notranslate"><div class=highlight ><pre><span></span><span class=o >&lt;</span><span class=n >prefix</span><span class=o >&gt;</span><span class=n >_master</span><span class=o >.</span><span class=n >h5</span> <span class=c1 ># NXmx master file (metadata + links / virtual datasets)</span>
<span class=o >&lt;</span><span class=n >prefix</span><span class=o >&gt;</span><span class=n >_data_000001</span><span class=o >.</span><span class=n >h5</span> <span class=c1 ># data file: images + per-image analysis</span>
<span class=o >&lt;</span><span class=n >prefix</span><span class=o >&gt;</span><span class=n >_data_000002</span><span class=o >.</span><span class=n >h5</span>
<span class=o >...</span>
</pre></div> </div> <p>The master file is produced by <code class="docutils literal notranslate"><span class=pre >writer/HDF5NXmx.cpp</span></code>; data files by <code class="docutils literal notranslate"><span class=pre >writer/HDF5DataFile.cpp</span></code> and its plugins (<code class="docutils literal notranslate"><span class=pre >writer/HDF5DataFilePlugin*.cpp</span></code>). Files are written to a temporary <code class="docutils literal notranslate"><span class=pre >*.&lt;random&gt;.tmp</span></code> name and renamed on successful close.</p> <p>Three master-file variants exist (set via <code class="docutils literal notranslate"><span class=pre >file_format</span></code>):</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Format</p> <th class="head text-center"><p>Value</p> <th class=head ><p>Master ↔ data linking</p> <tr class=row-even ><td><p><strong>NXmxLegacy</strong></p> <td class=text-center ><p>1</p> <td><p>One external link in <code class="docutils literal notranslate"><span class=pre >/entry/data</span></code> per data file (<code class="docutils literal notranslate"><span class=pre >data_000001</span></code>, …). HDF5 1.8 compatible — works with Neggia/Durin XDS plugins and Albula 4.0.</p> <tr class=row-odd ><td><p><strong>NXmxVDS</strong> (default)</p> <td class=text-center ><p>2</p> <td><p>A single virtual dataset <code class="docutils literal notranslate"><span class=pre >/entry/data/data</span></code> spans all data files; spot finding, azimuthal integration and reflections are linked the same way. Requires HDF5 1.10 / Albula 4.1+.</p> <tr class=row-even ><td><p><strong>NXmxIntegrated</strong></p> <td class=text-center ><p>3</p> <td><p>No separate data files — images and all metadata live in one file. Equivalent in content to the VDS format.</p> </table> <p>In legacy/VDS mode, image-indexed analysis arrays live in the <strong>data files</strong> and are exposed in the master file through external links or virtual datasets; in integrated mode they are written directly into the single file. Throughout this document a “✓ in master” column marks entries that are visible (directly or via link/VDS) from the master file.</p> <p>Images are stored chunked (one image per chunk) and compressed with bitshuffle + LZ4 or bitshuffle + Zstd. Signed integer image datasets carry <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span></code> as the HDF5 fill value (the “masked / no-data” sentinel); unsigned ones are left at HDF5’s own fill of 0, because every unsigned code is a legitimate count. In the master’s virtual dataset the fill is the error marker for both, so a data file missing beside a VDS master reads as masked rather than as zero counts.</p> <blockquote> <div><p><strong>Signed images and the Neggia XDS plugin.</strong> Neggia dispatches on the size of the pixel in bytes and always casts to an unsigned type, consulting the signedness of the data only for the pixel mask. A <strong>signed 16-bit</strong> image is therefore read wrongly: a count of <code class="docutils literal notranslate"><span class=pre >-2</span></code> reaches XDS as <code class="docutils literal notranslate"><span class=pre >65534</span></code>, and the <code class="docutils literal notranslate"><span class=pre >-32768</span></code> error marker as <code class="docutils literal notranslate"><span class=pre >32768</span></code>. Signed 32-bit happens to degrade safely, because every negative value ends up above <code class="docutils literal notranslate"><span class=pre >INT32_MAX</span></code> and is mapped to <code class="docutils literal notranslate"><span class=pre >-1</span></code>. Use the Jungfraujoch XDS plugin, or the Global Phasing build of Durin, for signed data — see <a class="reference internal" href=SOFTWARE_INTEGRATION.html ><span class="std std-doc">Integration with MX data processing software</span></a>.</p> </div></blockquote> <section id=reprocessing-output-prefix-process-h5 > <h3 id=reprocessing-output-prefix-process-h5 ><a class=toc-backref href="#id4" role=doc-backlink >Reprocessing output: <code class="docutils literal notranslate"><span class=pre >&lt;prefix&gt;_process.h5</span></code></a><a class=headerlink href="#reprocessing-output-prefix-process-h5" title="Link to this heading"></a></h3> <p>The offline reprocessing tool <a class="reference internal" href=TOOLS.html ><span class="std std-doc"><code class="docutils literal notranslate"><span class=pre >rugnux</span></code></span></a> (<code class="docutils literal notranslate"><span class=pre >rugnux/rugnux_cli.cpp</span></code>) re-runs the full analysis pipeline (spot finding, indexing, refinement, integration, scaling) on an existing dataset and writes its results to a master file named <strong><code class="docutils literal notranslate"><span class=pre >&lt;prefix&gt;_process.h5</span></code></strong>. This file uses the <strong>integrated</strong> format, but instead of copying the images its <code class="docutils literal notranslate"><span class=pre >/entry/data/data</span></code> is a <em>virtual dataset that links back to the original image files</em> (<code class="docutils literal notranslate"><span class=pre >hdf5_source_data</span></code> → <code class="docutils literal notranslate"><span class=pre >NXmx::LinkToData_ProcessingVDS</span></code>). The result is a compact, self-describing companion file that holds <em>all</em> the derived analysis (everything in §4) plus a virtual view of the raw images — without duplicating terabytes of data.</p> <p>This is a particularly FAIR-friendly artefact: it can be shared or archived alongside (or instead of) the raw data to convey what is in a dataset and how it processed, while the <code class="docutils literal notranslate"><span class=pre >/entry/data/data</span></code> VDS still resolves to the original images when they are available. <code class="docutils literal notranslate"><span class=pre >rugnux</span></code> can also process an equally-spaced <em>subset</em> of images (start/end/stride), producing a down-sampled reference set.</p> </section> </section> <section id=nxmx-standard-content > <h2 id=nxmx-standard-content ><a class=toc-backref href="#id5" role=doc-backlink >3. NXmx-standard content</a><a class=headerlink href="#nxmx-standard-content" title="Link to this heading"></a></h2> <p>The entries below are part of, or valid base classes for, the <a class="reference external" href="https://manual.nexusformat.org/classes/applications/NXmx.html">NXmx</a> application definition. “NXmx” = listed in the application definition; “base” = a valid field of the relevant NeXus base class (<code class="docutils literal notranslate"><span class=pre >NXdetector</span></code>, <code class="docutils literal notranslate"><span class=pre >NXsample</span></code>, <code class="docutils literal notranslate"><span class=pre >NXsource</span></code>) but not in the NXmx required/recommended subset.</p> <section id=entry-nxentry > <h3 id=entry-nxentry ><a class=toc-backref href="#id6" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry</span></code> (NXentry)</a><a class=headerlink href="#entry-nxentry" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Notes</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >definition</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>value <code class="docutils literal notranslate"><span class=pre >"NXmx"</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >start_time</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>arming time</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >end_time</span></code>, <code class="docutils literal notranslate"><span class=pre >end_time_estimated</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>approximate end time</p> </table> <p>File-level HDF5 attributes <code class="docutils literal notranslate"><span class=pre >file_name</span></code>, <code class="docutils literal notranslate"><span class=pre >file_time</span></code>, <code class="docutils literal notranslate"><span class=pre >HDF5_Version</span></code> are also set.</p> </section> <section id=entry-source-nxsource-entry-instrument-nxinstrument > <h3 id=entry-source-nxsource-entry-instrument-nxinstrument ><a class=toc-backref href="#id7" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/source</span></code> (NXsource), <code class="docutils literal notranslate"><span class=pre >/entry/instrument</span></code> (NXinstrument)</a><a class=headerlink href="#entry-source-nxsource-entry-instrument-nxinstrument" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >source/name</span></code>, <code class="docutils literal notranslate"><span class=pre >source/type</span></code></p> <td class=text-center ><p>NXmx / base</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >source/current</span></code></p> <td class=text-center ><p>base</p> <td><p>A</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >instrument/name</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> </table> </section> <section id=entry-instrument-beam-nxbeam > <h3 id=entry-instrument-beam-nxbeam ><a class=toc-backref href="#id8" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/beam</span></code> (NXbeam)</a><a class=headerlink href="#entry-instrument-beam-nxbeam" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >incident_wavelength</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>angstrom</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >incident_wavelength_spread</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>angstrom (only if polychromatic)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >total_flux</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>Hz</p> </table> </section> <section id=entry-instrument-attenuator-nxattenuator > <h3 id=entry-instrument-attenuator-nxattenuator ><a class=toc-backref href="#id9" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/attenuator</span></code> (NXattenuator)</a><a class=headerlink href="#entry-instrument-attenuator-nxattenuator" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >attenuator_transmission</span></code></p> <td class=text-center ><p>NXmx</p> </table> </section> <section id=entry-instrument-detector-nxdetector > <h3 id=entry-instrument-detector-nxdetector ><a class=toc-backref href="#id10" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector</span></code> (NXdetector)</a><a class=headerlink href="#entry-instrument-detector-nxdetector" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >depends_on</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>→ <code class="docutils literal notranslate"><span class=pre >transformations/rot3</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >beam_center_x</span></code>, <code class="docutils literal notranslate"><span class=pre >beam_center_y</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>pixel (0.0 = centre of the first pixel, see <a class="reference internal" href=DETECTOR_GEOMETRY.html ><span class="std std-doc">DETECTOR_GEOMETRY</span></a>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >distance</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>m</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >count_time</span></code>, <code class="docutils literal notranslate"><span class=pre >frame_time</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>s</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >sensor_thickness</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>m</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >sensor_material</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >description</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >threshold_energy</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>eV (EIGER; written only for a single channel)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >x_pixel_size</span></code>, <code class="docutils literal notranslate"><span class=pre >y_pixel_size</span></code></p> <td class=text-center ><p>base</p> <td><p>m</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >serial_number</span></code></p> <td class=text-center ><p>base</p> <td><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>bit depth of the <strong>stored image</strong>, not of the detector electronics - see below</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >saturation_value</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>highest valid value. Read inclusively by NXmx, by the DIALS <code class="docutils literal notranslate"><span class=pre >trusted_range</span></code> and by the XDS <code class="docutils literal notranslate"><span class=pre >OVERLOAD</span></code> parameter; a saturated pixel carries the value one above it</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >underload_value</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>lowest valid value: <code class="docutils literal notranslate"><span class=pre >0</span></code> for an unsigned image, <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span> <span class=pre >+</span> <span class=pre >1</span></code> for a signed one</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >flatfield_applied</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >pixel_mask</span></code>, <code class="docutils literal notranslate"><span class=pre >pixel_mask_applied</span></code></p> <td class=text-center ><p>NXmx</p> <td><p><code class="docutils literal notranslate"><span class=pre >pixel_mask</span></code> is <code class="docutils literal notranslate"><span class=pre >[y,</span> <span class=pre >x]</span></code>, hard-linked from <code class="docutils literal notranslate"><span class=pre >detectorSpecific/pixel_mask</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >countrate_correction_applied</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >number_of_cycles</span></code></p> <td class=text-center ><p>base</p> <td><p>frame-summation factor</p> </table> <section id=why-bit-depth-readout-is-the-image-depth > <h4 id=why-bit-depth-readout-is-the-image-depth >Why <code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code> is the image depth<a class=headerlink href="#why-bit-depth-readout-is-the-image-depth" title="Link to this heading"></a></h4> <p>NXmx defines only <code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code>, “how many bits the electronics record per pixel”, and has no field for the depth of the image actually stored. The two differ whenever summation is used: the readout stays at the detector’s native width while the summed image must be wider to hold the sum.</p> <p>Jungfraujoch writes the <strong>stored image depth</strong> into <code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code> (and the identical value into the non-standard <code class="docutils literal notranslate"><span class=pre >bit_depth_image</span></code>). The electronic value is a constant of the detector and tells a data consumer nothing, whereas readers do use <code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code> as the width of the stored pixel — DIALS, for instance, derives its masking markers from it and cannot read a 32-bit image without it. Writing the electronic value there would therefore mislead exactly in the case where the two differ.</p> <p>Note that <code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code> gives the width only. The <strong>sign</strong> is carried solely by the HDF5 element type of <code class="docutils literal notranslate"><span class=pre >/entry/data/data</span></code> (and, on the wire, by <code class="docutils literal notranslate"><span class=pre >image_dtype</span></code>); there is no NXmx field for it.</p> </section> </section> <section id=entry-instrument-detector-transformations-nxtransformations > <h3 id=entry-instrument-detector-transformations-nxtransformations ><a class=toc-backref href="#id11" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/transformations</span></code> (NXtransformations)</a><a class=headerlink href="#entry-instrument-detector-transformations-nxtransformations" title="Link to this heading"></a></h3> <p>The NXtransformations <em>mechanism</em> (the <code class="docutils literal notranslate"><span class=pre >depends_on</span></code> chain, <code class="docutils literal notranslate"><span class=pre >transformation_type</span></code>, <code class="docutils literal notranslate"><span class=pre >vector</span></code>, <code class="docutils literal notranslate"><span class=pre >offset</span></code> attributes) is standard. The axis <strong>names</strong> follow the PyFAI PONI convention chosen by Jungfraujoch (see <a class="reference internal" href=DETECTOR_GEOMETRY.html ><span class="std std-doc">DETECTOR_GEOMETRY</span></a>):</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Axis</p> <th class=head ><p>Type</p> <th class=head ><p>Units</p> <th class=head ><p>Vector</p> <th class=head ><p>Depends on</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rot3</span></code></p> <td><p>rotation</p> <td><p>rad</p> <td><p><code class="docutils literal notranslate"><span class=pre >(0,</span> <span class=pre >0,</span> <span class=pre >-1)</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >.</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >rot2</span></code></p> <td><p>rotation</p> <td><p>rad</p> <td><p><code class="docutils literal notranslate"><span class=pre >(1,</span> <span class=pre >0,</span> <span class=pre >0)</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >rot3</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rot1</span></code></p> <td><p>rotation</p> <td><p>rad</p> <td><p><code class="docutils literal notranslate"><span class=pre >(0,</span> <span class=pre >-1,</span> <span class=pre >0)</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >rot2</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >translation</span></code></p> <td><p>translation</p> <td><p>m</p> <td><p>unit vector along the sample→PONI direction</p> <td><p><code class="docutils literal notranslate"><span class=pre >rot1</span></code></p> </table> <p><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/depends_on</span></code> is <code class="docutils literal notranslate"><span class=pre >translation</span></code>, and the module’s <code class="docutils literal notranslate"><span class=pre >fast_pixel_direction</span></code>, <code class="docutils literal notranslate"><span class=pre >slow_pixel_direction</span></code> and <code class="docutils literal notranslate"><span class=pre >module_offset</span></code> depend on it in turn. A chain is applied innermost-first, so reading it outwards the detector is placed at its distance and beam centre and <em>then</em> tilted about the sample — which is what makes a tilt pivot about the crystal rather than about the panel corner. The <code class="docutils literal notranslate"><span class=pre >vector</span></code> values are in NXmx’s <strong>McStas</strong> frame, which is Jungfraujoch’s internal frame with x and y negated.</p> <p>The beam centre is encoded in <code class="docutils literal notranslate"><span class=pre >translation</span></code> (its offset from the sample), not only in the informational <code class="docutils literal notranslate"><span class=pre >beam_center_x</span></code>/<code class="docutils literal notranslate"><span class=pre >beam_center_y</span></code> fields. In a <code class="docutils literal notranslate"><span class=pre >_process.h5</span></code> written by rugnux these axes carry the <strong>refined</strong> detector geometry — the refined beam centre folds into <code class="docutils literal notranslate"><span class=pre >translation</span></code> and the refined tilt into <code class="docutils literal notranslate"><span class=pre >rot1</span></code>/<code class="docutils literal notranslate"><span class=pre >rot2</span></code>/<code class="docutils literal notranslate"><span class=pre >rot3</span></code>; the broker writes the user-provided geometry unchanged.</p> </section> <section id=entry-instrument-detector-module-nxdetector-module > <h3 id=entry-instrument-detector-module-nxdetector-module ><a class=toc-backref href="#id12" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/module</span></code> (NXdetector_module)</a><a class=headerlink href="#entry-instrument-detector-module-nxdetector-module" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >data_origin</span></code>, <code class="docutils literal notranslate"><span class=pre >data_size</span></code>, <code class="docutils literal notranslate"><span class=pre >fast_pixel_direction</span></code>, <code class="docutils literal notranslate"><span class=pre >slow_pixel_direction</span></code>, <code class="docutils literal notranslate"><span class=pre >module_offset</span></code> — all NXmx (<code class="docutils literal notranslate"><span class=pre >fast/slow_pixel_direction</span></code> and <code class="docutils literal notranslate"><span class=pre >module_offset</span></code> carry transformation attributes).</p> </section> <section id=entry-sample-nxsample > <h3 id=entry-sample-nxsample ><a class=toc-backref href="#id13" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/sample</span></code> (NXsample)</a><a class=headerlink href="#entry-sample-nxsample" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units / notes</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >name</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >depends_on</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>points at the innermost axis of the sample chain, or <code class="docutils literal notranslate"><span class=pre >.</span></code> for stills</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >temperature</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>K</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >transformations/</span></code> (NXtransformations)</p> <td class=text-center ><p>NXmx</p> <td><p>the sample chain, written in mounting order; hard-linked as <code class="docutils literal notranslate"><span class=pre >/entry/sample/goniometer</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >unit_cell</span></code></p> <td class=text-center ><p>base</p> <td><p><code class="docutils literal notranslate"><span class=pre >[a,</span> <span class=pre >b,</span> <span class=pre >c,</span> <span class=pre >α,</span> <span class=pre >β,</span> <span class=pre >γ]</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >ub_matrix</span></code></p> <td class=text-center ><p>base</p> <td><p><code class="docutils literal notranslate"><span class=pre >[1,</span> <span class=pre >3,</span> <span class=pre >3]</span></code>, Angstrom⁻¹</p> </table> <p>The chain is written from the base outwards, so the innermost axis — the one <code class="docutils literal notranslate"><span class=pre >depends_on</span></code> names — is the one nearest the sample. It may hold, in that order: the grid-scan translations <code class="docutils literal notranslate"><span class=pre >grid_scan_x</span></code> and <code class="docutils literal notranslate"><span class=pre >grid_scan_y</span></code>, the spindle, and a Smargon head’s <code class="docutils literal notranslate"><span class=pre >chi</span></code> and <code class="docutils literal notranslate"><span class=pre >phi</span></code>. A grid scan and a goniometer axis are <strong>not</strong> alternatives; both can be present.</p> <p>A <strong>Smargon head position is told apart from the spindle</strong> by the <code class="docutils literal notranslate"><span class=pre >equipment_component</span></code> attribute, which is <code class="docutils literal notranslate"><span class=pre >"smargon"</span></code> on <code class="docutils literal notranslate"><span class=pre >chi</span></code> and <code class="docutils literal notranslate"><span class=pre >phi</span></code> and absent on the spindle. This is load-bearing: a spindle can itself be named <code class="docutils literal notranslate"><span class=pre >phi</span></code>, and without the attribute a reader would take a head position for the scan axis. <code class="docutils literal notranslate"><span class=pre >chi</span></code> and <code class="docutils literal notranslate"><span class=pre >phi</span></code> are written with one value per image even though neither turns, because a reader takes the image count from the innermost axis of the chain: written as scalars, a still recorded at a head position would read back as a single image however many were collected.</p> <p>For a rotation scan the goniometer axis carries, beyond the per-image angle array <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;</span></code>, the Jungfraujoch conveniences <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_end</span></code>, scalar <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_range_average</span></code> and <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_range_total</span></code>, and for helical scans <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_helical_x/_y/_z</span></code>.</p> </section> <section id=entry-data-nxdata > <h3 id=entry-data-nxdata ><a class=toc-backref href="#id14" role=doc-backlink ><code class="docutils literal notranslate"><span class=pre >/entry/data</span></code> (NXdata)</a><a class=headerlink href="#entry-data-nxdata" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >data</span></code> (3-D image stack, <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >y,</span> <span class=pre >x]</span></code>) with <code class="docutils literal notranslate"><span class=pre >image_nr_low</span></code> / <code class="docutils literal notranslate"><span class=pre >image_nr_high</span></code> attributes. In legacy mode this group instead contains one external link <code class="docutils literal notranslate"><span class=pre >data_000001</span></code>, … per data file.</p> </section> </section> <section id=extensions-beyond-nxmx > <h2 id=extensions-beyond-nxmx ><a class=toc-backref href="#id15" role=doc-backlink >4. Extensions beyond NXmx</a><a class=headerlink href="#extensions-beyond-nxmx" title="Link to this heading"></a></h2> <p>Everything in this section is <strong>outside the NXmx standard</strong>. Each group is declared with <code class="docutils literal notranslate"><span class=pre >NX_class</span> <span class=pre >=</span> <span class=pre >NXcollection</span></code> (the NeXus-sanctioned container for non-standardised content) unless noted. The per-image arrays are indexed by image number, padded to the run length and filled with a sentinel (<code class="docutils literal notranslate"><span class=pre >NaN</span></code> for floats, <code class="docutils literal notranslate"><span class=pre >-1</span></code>/<code class="docutils literal notranslate"><span class=pre >0</span></code> for integer indices) where a quantity is absent.</p> <section id=entry-mx-spot-finding-and-indexing-cxi-style > <h3 id=entry-mx-spot-finding-and-indexing-cxi-style ><a class=toc-backref href="#id16" role=doc-backlink >4.1 <code class="docutils literal notranslate"><span class=pre >/entry/MX</span></code> — spot finding and indexing (CXI-style)</a><a class=headerlink href="#entry-mx-spot-finding-and-indexing-cxi-style" title="Link to this heading"></a></h3> <p>The flagship extension. Spot (“peak”) properties are stored as fixed-size <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >max_spots]</span></code> arrays (CXI layout, recognised by CrystFEL); scalar-per-image quantities as <code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors. In legacy/VDS mode these live in the data files and are linked/virtual-stacked into the master.</p> <p><strong>Per-spot arrays <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >max_spots]</span></code>:</strong></p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <th class="head text-center"><p>Indexing only</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakXPosRaw</span></code>, <code class="docutils literal notranslate"><span class=pre >peakYPosRaw</span></code></p> <td><p>pixel</p> <td><p>spot position (raw detector frame)</p> <td class=text-center ><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakTotalIntensity</span></code></p> <td><p>photons</p> <td><p>spot intensity</p> <td class=text-center ><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakIceRingRes</span></code></p> <td><p></p> <td><p>spot lies in an ice-ring resolution band</p> <td class=text-center ><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakH</span></code>, <code class="docutils literal notranslate"><span class=pre >peakK</span></code>, <code class="docutils literal notranslate"><span class=pre >peakL</span></code></p> <td><p></p> <td><p>Miller indices of the (indexed) spot</p> <td class=text-center ><p>✓</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakDistEwaldSphere</span></code></p> <td><p>Å⁻¹</p> <td><p>distance of the spot from the Ewald sphere</p> <td class=text-center ><p>✓</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakIndexed</span></code></p> <td><p></p> <td><p>spot fits the indexing solution</p> <td class=text-center ><p>✓</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakLattice</span></code></p> <td><p></p> <td><p>lattice the spot belongs to (<code class="docutils literal notranslate"><span class=pre >-1</span></code> = unindexed)</p> <td class=text-center ><p>✓</p> </table> <p><strong>Per-image vectors <code class="docutils literal notranslate"><span class=pre >[n_images]</span></code>:</strong></p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >nPeaks</span></code></p> <td><p></p> <td><p>number of spots stored for the image (CXI)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >strongPixels</span></code></p> <td><p></p> <td><p>strong-pixel count (first spot-finding stage)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountUnfiltered</span></code></p> <td><p></p> <td><p>spots found before filtering</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountLowRes</span></code></p> <td><p></p> <td><p>low-resolution spots</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountIceRingRes</span></code></p> <td><p></p> <td><p>spots inside ice-ring bands</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountIceRingControl</span></code></p> <td><p></p> <td><p>spots in the ice-free flanks beside those bands, rescaled to their q width - the control for the count above (their ratio, pooled over the run, is the spot-based ice indicator)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountIndexed</span></code></p> <td><p></p> <td><p>spots fitting the indexing solution</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >imageIndexed</span></code></p> <td><p></p> <td><p>image was indexed (0/1)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >indexingLatticeCount</span></code></p> <td><p></p> <td><p>number of lattices found for the image</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >niggliClass</span></code></p> <td><p></p> <td><p>Niggli class of the indexed Bravais lattice (see <em>International Tables for Crystallography A</em> (2016), Vol. A, <a class="reference external" href="https://onlinelibrary.wiley.com/iucr/itc/Ac/ch3o1v0001/table3o1o3o1.pdf">Table 3.1.3.1</a>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bravaisLattice</span></code></p> <td><p></p> <td><p>Bravais lattice short code, e.g. <code class="docutils literal notranslate"><span class=pre >aP</span></code>, <code class="docutils literal notranslate"><span class=pre >mC</span></code>, <code class="docutils literal notranslate"><span class=pre >oF</span></code>, <code class="docutils literal notranslate"><span class=pre >tI</span></code>, <code class="docutils literal notranslate"><span class=pre >hP</span></code>, <code class="docutils literal notranslate"><span class=pre >hR</span></code>, <code class="docutils literal notranslate"><span class=pre >cF</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >profileRadius</span></code></p> <td><p>Å⁻¹</p> <td><p>crystal profile radius</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >mosaicity</span></code></p> <td><p>deg</p> <td><p>mosaicity estimate</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >bFactor</span></code></p> <td><p>Ų</p> <td><p>per-image B-factor estimate</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >resolutionEstimate</span></code></p> <td><p>Å</p> <td><p>resolution the merged data are predicted to reach, from this image’s spots alone</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >integratedReflections</span></code></p> <td><p></p> <td><p>number of integrated reflections</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bkgEstimate</span></code></p> <td><p>photons</p> <td><p>mean background in the 3–5 Å resolution band</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >iceRingScore</span></code></p> <td><p>ratio</p> <td><p>strongest hexagonal-ice ring intensity over the smooth radial background (1 = no ice)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >beam_corr_x</span></code>, <code class="docutils literal notranslate"><span class=pre >beam_corr_y</span></code></p> <td><p>pixel</p> <td><p>beam-center correction applied during processing</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >imageScaleFactor</span></code></p> <td><p></p> <td><p>on-the-fly per-image scale factor <em>g</em></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >imageScaleCC</span></code></p> <td><p></p> <td><p>on-the-fly scaling correlation coefficient</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >imageScaleMosaicity</span></code></p> <td><p>deg</p> <td><p>scaling-model mosaicity</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >sweepQuality</span></code></p> <td><p></p> <td><p>why this image’s stretch of the sweep was flagged — see below</p> </table> <p><strong>Per-image lattices:</strong> <code class="docutils literal notranslate"><span class=pre >latticeIndexed</span></code> <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >9]</span></code> (Å) — the real-space lattice (flattened 3×3); <code class="docutils literal notranslate"><span class=pre >latticeIndexedExtra</span></code> <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >max_extra_lattices,</span> <span class=pre >9]</span></code> (Å) — additional orientation variants.</p> <p><strong>Run-level summaries</strong> (written into the master <code class="docutils literal notranslate"><span class=pre >/entry/MX</span></code> at finalisation):</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >indexing_algorithm</span></code></p> <td><p></p> <td><p><code class="docutils literal notranslate"><span class=pre >FFBIDX</span></code> / <code class="docutils literal notranslate"><span class=pre >FFT</span> <span class=pre >(CUDA)</span></code> / <code class="docutils literal notranslate"><span class=pre >FFT</span> <span class=pre >(FFTW)</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >geom_refinement_algorithm</span></code></p> <td><p></p> <td><p>e.g. <code class="docutils literal notranslate"><span class=pre >beam_center</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rotationLatticeIndexed</span></code></p> <td><p>Å</p> <td><p>whole-run rotation-indexing lattice (<code class="docutils literal notranslate"><span class=pre >[9]</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >rotationLatticeIndexedExtra</span></code></p> <td><p>Å</p> <td><p>additional whole-run lattices (<code class="docutils literal notranslate"><span class=pre >[m,</span> <span class=pre >9]</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rotationLatticeNiggliClass</span></code></p> <td><p></p> <td><p>Niggli class of the run lattice</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >imageIndexedMean</span></code></p> <td><p></p> <td><p>mean indexing rate over the run</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bkgEstimateMean</span></code></p> <td><p>photons</p> <td><p>mean background over the run</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >iceRingScoreMean</span></code></p> <td><p>ratio</p> <td><p>mean <code class="docutils literal notranslate"><span class=pre >iceRingScore</span></code> over the run — the single “how icy was this dataset” number (1 = no ice)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >indexedLatticeCount</span></code></p> <td><p></p> <td><p>per-image lattice count summary (master). <em>Note: data files use <code class="docutils literal notranslate"><span class=pre >indexingLatticeCount</span></code>; readers accept either.</em></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >reindexMatrix</span></code></p> <td><p></p> <td><p>change of basis from the setting the per-image data are in to the setting of <code class="docutils literal notranslate"><span class=pre >/entry/sample/unit_cell</span></code> (<code class="docutils literal notranslate"><span class=pre >[9]</span></code>, <code class="docutils literal notranslate"><span class=pre >int32</span></code>, flattened 3×3, row major) — see below</p> </table> <p><strong>Reindex matrix.</strong> The per-image <code class="docutils literal notranslate"><span class=pre >h</span></code>, <code class="docutils literal notranslate"><span class=pre >k</span></code>, <code class="docutils literal notranslate"><span class=pre >l</span></code> and <code class="docutils literal notranslate"><span class=pre >latticeIndexed</span></code> are written as each image is processed, in the setting that image was <em>indexed</em> in. The space group is only settled afterwards, by the merge, and settling it can re-seat the lattice into the group’s conventional setting — so <code class="docutils literal notranslate"><span class=pre >/entry/sample/unit_cell</span></code>, <code class="docutils literal notranslate"><span class=pre >/entry/sample/space_group_number</span></code> and <code class="docutils literal notranslate"><span class=pre >rotationLatticeIndexed</span></code> can be in a different setting from the per-image data beside them. <code class="docutils literal notranslate"><span class=pre >reindexMatrix</span></code> <strong>M</strong> is the integral change of basis between the two: <code class="docutils literal notranslate"><span class=pre >hkl_cell</span> <span class=pre >=</span> <span class=pre >M</span> <span class=pre >·</span> <span class=pre >hkl_written</span></code>, and the same <strong>M</strong> takes each per-image lattice across (<code class="docutils literal notranslate"><span class=pre >latticeIndexedExtra</span></code> is not re-seated and stays as indexed). It is <strong>absent</strong> when the two settings are the same one, which is the identity — as it is on every file written before rugnux recorded it. The Jungfraujoch reader applies it, so everything it hands out is already in the cell’s setting; a third-party reader that ignores it will index the reflections in the wrong frame whenever the dataset is present. Written by the offline <code class="docutils literal notranslate"><span class=pre >rugnux</span></code> path only — the broker never re-seats a lattice — and not carried on the CBOR stream, in the same way as the other offline-only fields.</p> <p>A <code class="docutils literal notranslate"><span class=pre >--model</span></code> run can leave the merged reflection files (<code class="docutils literal notranslate"><span class=pre >.mtz</span></code>/<code class="docutils literal notranslate"><span class=pre >.cif</span></code>/<code class="docutils literal notranslate"><span class=pre >.hkl</span></code>) in a <em>different</em> frame from the <code class="docutils literal notranslate"><span class=pre >_process.h5</span></code> beside them: the model settles the alternative indexing where nothing else did, and that choice is applied to the merged reflections as they are written. It also names the enantiomorph, but that is a change of space-group label only and moves no reflection. The process file is not rewritten — its per-image reflections went to disk as they were integrated — so it keeps the space group the run itself determined and stays self-consistent with its own data. The two frames describe the same measurements; an enantiomorphic pair merges identically, having the same Laue class and the same absences.</p> <p><strong>Sweep quality.</strong> <code class="docutils literal notranslate"><span class=pre >sweepQuality</span></code> <code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> (<code class="docutils literal notranslate"><span class=pre >uint8</span></code>) says why the stretch of the sweep this image belongs to was flagged as delivering much less than the rest of the run: <strong>0</strong> means it was not, and any other value is a <strong>1-based index into <code class="docutils literal notranslate"><span class=pre >sweepQualityReasons</span></code></strong>, a string vector written beside it that carries the whole vocabulary, so the codes can be read without this source. The vocabulary is closed and stable — a code is never renamed and never reused — and currently reads <code class="docutils literal notranslate"><span class=pre >no_diffraction</span></code>, <code class="docutils literal notranslate"><span class=pre >crystal_out_of_beam</span></code>, <code class="docutils literal notranslate"><span class=pre >weak_diffraction</span></code>, <code class="docutils literal notranslate"><span class=pre >loss_of_centring</span></code>, <code class="docutils literal notranslate"><span class=pre >radiation_damage</span></code>; <a class="reference internal" href="RUGNUX.html#sweep-quality-and-the-reason-vocabulary"><span class="std std-ref">the rugnux documentation</span></a> defines what each one means. Both datasets are <strong>absent</strong> unless the sweep-quality diagnostic ran, which needs scaling and merging; their absence therefore means “not looked for”, <em>not</em> “every image clean”. Written by the offline <code class="docutils literal notranslate"><span class=pre >rugnux</span></code> path only — the broker does not merge — and not carried on the CBOR stream, in the same way as the other offline-only fields (<code class="docutils literal notranslate"><span class=pre >space_group_number</span></code>, the refined geometry). Nothing is excluded from processing on the strength of it. The condensed, dataset-wide form of the same finding is in <code class="docutils literal notranslate"><span class=pre >&lt;prefix&gt;_report.txt</span></code>.</p> <p>CrystFEL can read the spots directly with:</p> <div class="highlight-default notranslate"><div class=highlight ><pre><span></span><span class=n >peak_list</span> <span class=o >=</span> <span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >MX</span>
<span class=n >peak_list_type</span> <span class=o >=</span> <span class=n >cxi</span>
</pre></div> </div> </section> <section id=entry-reflections-integrated-reflections-nxreflections > <h3 id=entry-reflections-integrated-reflections-nxreflections ><a class=toc-backref href="#id17" role=doc-backlink >4.2 <code class="docutils literal notranslate"><span class=pre >/entry/reflections</span></code> — integrated reflections (NXreflections)</a><a class=headerlink href="#entry-reflections-integrated-reflections-nxreflections" title="Link to this heading"></a></h3> <p>Integrated reflections are stored <strong>per image</strong> as <code class="docutils literal notranslate"><span class=pre >/entry/reflections/image_NNNNNN</span></code> groups, each declared <code class="docutils literal notranslate"><span class=pre >NX_class</span> <span class=pre >=</span> <span class=pre >NXreflections</span></code>. The columns map mostly onto the standard <a class="reference external" href="https://manual.nexusformat.org/classes/base_classes/NXreflections.html">NXreflections</a> base class:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class="head text-center"><p>NXreflections</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >h</span></code>, <code class="docutils literal notranslate"><span class=pre >k</span></code>, <code class="docutils literal notranslate"><span class=pre >l</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>Miller indices</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >d</span></code></p> <td><p>Å</p> <td class=text-center ><p>standard</p> <td><p>resolution</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >int_sum</span></code></p> <td><p>photons</p> <td class=text-center ><p>standard</p> <td><p>integrated intensity (summation)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >int_err</span></code></p> <td><p>photons</p> <td class=text-center ><p>non-standard name</p> <td><p>σ of the intensity (standard equivalent: <code class="docutils literal notranslate"><span class=pre >int_sum_errors</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >background_mean</span></code></p> <td><p>photons</p> <td class=text-center ><p>standard</p> <td><p>mean background under the peak</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >background_variance</span></code></p> <td><p>photons²</p> <td class=text-center ><p>non-standard</p> <td><p>non-signal part of σ², carried to the merge. Absent in files written before it existed; the reader then recovers it from σ² − I</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >predicted_x</span></code>, <code class="docutils literal notranslate"><span class=pre >predicted_y</span></code></p> <td><p>pixel</p> <td class=text-center ><p>name standard, units differ</p> <td><p>predicted position. NXreflections <code class="docutils literal notranslate"><span class=pre >predicted_x/_y</span></code> are <em>physical</em> lengths; the pixel datasets are <code class="docutils literal notranslate"><span class=pre >predicted_px_x/_y</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >observed_x</span></code>, <code class="docutils literal notranslate"><span class=pre >observed_y</span></code></p> <td><p>pixel</p> <td class=text-center ><p>name standard, units differ</p> <td><p>observed centroid (pixels; standard pixel form is <code class="docutils literal notranslate"><span class=pre >observed_px_x/_y</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >observed_frame</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>image number of the reflection</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >lp</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>Lorentz–polarization factor (stored as <code class="docutils literal notranslate"><span class=pre >1/rlp</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >partiality</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>recorded fraction of the reflection</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >delta_phi</span></code></p> <td><p>deg</p> <td class=text-center ><p><strong>extension</strong></p> <td><p>XDS Δφ: offset from the centre of the current frame</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >zeta</span></code></p> <td><p></p> <td class=text-center ><p><strong>extension</strong></p> <td><p>Lorentz ζ factor (reciprocal-space geometry term)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image_scale_corr</span></code></p> <td><p></p> <td class=text-center ><p><strong>extension</strong></p> <td><p>per-image scale correction; <code class="docutils literal notranslate"><span class=pre >I_true</span> <span class=pre >=</span> <span class=pre >image_scale_corr</span> <span class=pre >·</span> <span class=pre >int_sum</span></code></p> </table> <p>In the master file these per-image groups are exposed through <code class="docutils literal notranslate"><span class=pre >/entry/reflections</span></code> external links (VDS/integrated formats).</p> </section> <section id=entry-azint-azimuthal-integration > <h3 id=entry-azint-azimuthal-integration ><a class=toc-backref href="#id18" role=doc-backlink >4.3 <code class="docutils literal notranslate"><span class=pre >/entry/azint</span></code> — azimuthal integration</a><a class=headerlink href="#entry-azint-azimuthal-integration" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Shape</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bin_to_q</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p>Å⁻¹</p> <td><p>q value of each bin</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >bin_to_two_theta</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p>deg</p> <td><p>2θ of each bin</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bin_to_phi</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p>deg</p> <td><p>azimuthal angle of each bin</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p></p> <td><p>per-image integrated profile (NaN for empty bins)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >image_std</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p></p> <td><p>per-bin standard deviation</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image_count</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p></p> <td><p>pixels contributing per bin</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >map</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[y,</span> <span class=pre >x]</span></code></p> <td><p></p> <td><p>pixel→bin mapping (master file only)</p> </table> </section> <section id=entry-roi-regions-of-interest-per-image-results > <h3 id=entry-roi-regions-of-interest-per-image-results ><a class=toc-backref href="#id19" role=doc-backlink >4.4 <code class="docutils literal notranslate"><span class=pre >/entry/roi</span></code> — regions of interest (per-image results)</a><a class=headerlink href="#entry-roi-regions-of-interest-per-image-results" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >/entry/roi/&lt;roi_name&gt;</span></code> has one sub-group per configured ROI, holding the <strong>per-image result vectors</strong> <code class="docutils literal notranslate"><span class=pre >[n_images]</span></code>. These are written into the data files; in VDS mode they are exposed from the master file through virtual datasets, and in integrated mode they are in the single file. (In legacy mode they remain only in the data files.)</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >max</span></code></p> <td><p>maximum pixel value in the ROI</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >sum</span></code></p> <td><p>sum of pixel values</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >sum_sq</span></code></p> <td><p>sum of squared pixel values</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >npixel</span></code></p> <td><p>number of valid pixels</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >x</span></code>, <code class="docutils literal notranslate"><span class=pre >y</span></code></p> <td><p>intensity-weighted centroid</p> </table> </section> <section id=entry-roi-defs-roi-definitions-master-file > <h3 id=entry-roi-defs-roi-definitions-master-file ><a class=toc-backref href="#id20" role=doc-backlink >4.4.1 <code class="docutils literal notranslate"><span class=pre >/entry/roi_defs</span></code> — ROI definitions (master file)</a><a class=headerlink href="#entry-roi-defs-roi-definitions-master-file" title="Link to this heading"></a></h3> <p>The <strong>dataset-wide ROI definitions</strong> (geometry, fixed for the whole acquisition) live in the master file under a <em>separate</em> <code class="docutils literal notranslate"><span class=pre >/entry/roi_defs</span></code> group — kept apart from <code class="docutils literal notranslate"><span class=pre >/entry/roi</span></code> above so that older readers, which iterate <code class="docutils literal notranslate"><span class=pre >/entry/roi</span></code>, are unaffected by these entries. One sub-group <code class="docutils literal notranslate"><span class=pre >/entry/roi_defs/&lt;roi_name&gt;</span></code> per ROI:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bit_index</span></code></p> <td><p>which bit of <code class="docutils literal notranslate"><span class=pre >roi_map</span></code> (below) marks this ROI</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >type</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >box</span></code>, <code class="docutils literal notranslate"><span class=pre >circle</span></code> or <code class="docutils literal notranslate"><span class=pre >azim</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >min_x_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >max_x_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >min_y_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >max_y_pxl</span></code></p> <td><p>box bounds (type <code class="docutils literal notranslate"><span class=pre >box</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >center_x_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >center_y_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >radius_pxl</span></code></p> <td><p>circle (type <code class="docutils literal notranslate"><span class=pre >circle</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >q_min_recipA</span></code>, <code class="docutils literal notranslate"><span class=pre >q_max_recipA</span></code></p> <td><p>Q range (type <code class="docutils literal notranslate"><span class=pre >azim</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >phi_min_deg</span></code>, <code class="docutils literal notranslate"><span class=pre >phi_max_deg</span></code></p> <td><p>azimuthal-angle sector (type <code class="docutils literal notranslate"><span class=pre >azim</span></code>, omitted for a full ring)</p> </table> <p><code class="docutils literal notranslate"><span class=pre >/entry/roi_defs/roi_map</span></code> <code class="docutils literal notranslate"><span class=pre >[y,</span> <span class=pre >x]</span></code> is a <code class="docutils literal notranslate"><span class=pre >uint16</span></code> per-pixel bitmask: bit <code class="docutils literal notranslate"><span class=pre >bit_index</span></code> is set for every pixel belonging to that ROI, so an ROI’s footprint can be recovered exactly.</p> </section> <section id=entry-image-per-image-pixel-statistics > <h3 id=entry-image-per-image-pixel-statistics ><a class=toc-backref href="#id21" role=doc-backlink >4.5 <code class="docutils literal notranslate"><span class=pre >/entry/image</span></code> — per-image pixel statistics</a><a class=headerlink href="#entry-image-per-image-pixel-statistics" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors: <code class="docutils literal notranslate"><span class=pre >max_value</span></code>, <code class="docutils literal notranslate"><span class=pre >min_value</span></code> (viable min/max, excluding error/saturated pixels), <code class="docutils literal notranslate"><span class=pre >error_pixels</span></code>, <code class="docutils literal notranslate"><span class=pre >saturated_pixels</span></code>, <code class="docutils literal notranslate"><span class=pre >pixel_sum</span></code>. Surfaced in the master file under <code class="docutils literal notranslate"><span class=pre >/entry/image</span></code>.</p> </section> <section id=entry-profiling-per-image-timing > <h3 id=entry-profiling-per-image-timing ><a class=toc-backref href="#id22" role=doc-backlink >4.6 <code class="docutils literal notranslate"><span class=pre >/entry/profiling</span></code> — per-image timing</a><a class=headerlink href="#entry-profiling-per-image-timing" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors in seconds: <code class="docutils literal notranslate"><span class=pre >spotFindingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >indexingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >integrationTime</span></code>, <code class="docutils literal notranslate"><span class=pre >refinementTime</span></code>, <code class="docutils literal notranslate"><span class=pre >processingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >braggPredictionTime</span></code>, <code class="docutils literal notranslate"><span class=pre >preprocessingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >compressionTime</span></code>, <code class="docutils literal notranslate"><span class=pre >azIntTime</span></code>, <code class="docutils literal notranslate"><span class=pre >indexAnalysisTime</span></code>, <code class="docutils literal notranslate"><span class=pre >imageScaleTime</span></code>.</p> </section> <section id=entry-detector-acquisition-diagnostics-data-file > <h3 id=entry-detector-acquisition-diagnostics-data-file ><a class=toc-backref href="#id23" role=doc-backlink >4.7 <code class="docutils literal notranslate"><span class=pre >/entry/detector</span></code> — acquisition diagnostics (data file)</a><a class=headerlink href="#entry-detector-acquisition-diagnostics-data-file" title="Link to this heading"></a></h3> <p>A convenience NXcollection in the data file (note: distinct from the standard <code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector</span></code>). In <strong>integrated</strong> format these datasets are written under <code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/detectorSpecific</span></code> instead.</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >timestamp</span></code>, <code class="docutils literal notranslate"><span class=pre >exptime</span></code></p> <td><p>per-image timestamp and exposure time</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >number</span></code></p> <td><p>image number (original number if image rejection was used)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >det_info</span></code></p> <td><p>JUNGFRAU debug field</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >storage_cell_image</span></code></p> <td><p>storage-cell number</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rcv_delay</span></code>, <code class="docutils literal notranslate"><span class=pre >rcv_free_send_buffers</span></code></p> <td><p>receiver internal diagnostics</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >packets_expected</span></code>, <code class="docutils literal notranslate"><span class=pre >packets_received</span></code></p> <td><p>UDP packets per image</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >data_collection_efficiency_image</span></code></p> <td><p>received / expected packet ratio</p> </table> </section> <section id=entry-xfel-pulsed-source-metadata > <h3 id=entry-xfel-pulsed-source-metadata ><a class=toc-backref href="#id24" role=doc-backlink >4.8 <code class="docutils literal notranslate"><span class=pre >/entry/xfel</span></code> — pulsed-source metadata</a><a class=headerlink href="#entry-xfel-pulsed-source-metadata" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors <code class="docutils literal notranslate"><span class=pre >pulseID</span></code> and <code class="docutils literal notranslate"><span class=pre >eventCode</span></code>, written for pulsed sources (e.g. SwissFEL).</p> </section> <section id=other-collections > <h3 id=other-collections ><a class=toc-backref href="#id25" role=doc-backlink >4.9 Other collections</a><a class=headerlink href="#other-collections" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Path</p> <th class=head ><p>Class</p> <th class=head ><p>Content</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/detectorSpecific</span></code></p> <td><p>NXcollection</p> <td><p>Dectris-style detector metadata + Jungfraujoch fields: <code class="docutils literal notranslate"><span class=pre >x_pixels_in_detector</span></code>, <code class="docutils literal notranslate"><span class=pre >y_pixels_in_detector</span></code>, <code class="docutils literal notranslate"><span class=pre >nimages</span></code>, <code class="docutils literal notranslate"><span class=pre >ntrigger</span></code>, <code class="docutils literal notranslate"><span class=pre >nimages_collected</span></code>, <code class="docutils literal notranslate"><span class=pre >nimages_written</span></code>, <code class="docutils literal notranslate"><span class=pre >data_collection_efficiency</span></code>, <code class="docutils literal notranslate"><span class=pre >max_receiver_delay</span></code>, <code class="docutils literal notranslate"><span class=pre >storage_cell_number</span></code>, <code class="docutils literal notranslate"><span class=pre >storage_cell_delay</span></code> [ns], <code class="docutils literal notranslate"><span class=pre >software_git_commit</span></code>, <code class="docutils literal notranslate"><span class=pre >software_git_date</span></code>, <code class="docutils literal notranslate"><span class=pre >jfjoch_release</span></code>, <code class="docutils literal notranslate"><span class=pre >jfjoch_writer_release</span></code>, <code class="docutils literal notranslate"><span class=pre >summation_mode</span></code>, <code class="docutils literal notranslate"><span class=pre >detect_ice_rings</span></code>, <code class="docutils literal notranslate"><span class=pre >gain_file_names</span></code>, <code class="docutils literal notranslate"><span class=pre >data_reduction_factor_serialmx</span></code>, <code class="docutils literal notranslate"><span class=pre >adu_histogram/</span></code>, <code class="docutils literal notranslate"><span class=pre >data_collection_efficiency_image</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/calibration</span></code></p> <td><p>NXcollection</p> <td><p>per-channel pedestal / calibration images (bitshuffle-compressed)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/instrument/fluorescence</span></code></p> <td><p>NXcollection</p> <td><p>XRF spectrum: <code class="docutils literal notranslate"><span class=pre >energy</span></code> [eV], <code class="docutils literal notranslate"><span class=pre >data</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/user</span></code></p> <td><p>NXcollection</p> <td><p>scalar values supplied under <code class="docutils literal notranslate"><span class=pre >header_appendix.hdf5</span></code></p> </table> </section> <section id=non-standard-fields-inside-the-nxmx-detector-group > <h3 id=non-standard-fields-inside-the-nxmx-detector-group ><a class=toc-backref href="#id26" role=doc-backlink >4.10 Non-standard fields inside the NXmx detector group</a><a class=headerlink href="#non-standard-fields-inside-the-nxmx-detector-group" title="Link to this heading"></a></h3> <p>A few extension scalars are written <em>inside</em> the otherwise-standard <code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector</span></code> group for compatibility with existing tooling:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >detector_distance</span></code></p> <td><p>m</p> <td><p>duplicate of <code class="docutils literal notranslate"><span class=pre >distance</span></code> (Dectris/Neggia compatibility)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >detector_number</span></code></p> <td><p></p> <td><p>detector identifier (Dectris convention)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >mirror_y</span></code> (in <code class="docutils literal notranslate"><span class=pre >detectorSpecific</span></code>)</p> <td><p></p> <td><p>whether the stored image is mirrored in Y relative to the raw readout; true is the MX convention (row 0 at the top)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >error_value</span></code></p> <td><p></p> <td><p>masked/error pixel sentinel: <code class="docutils literal notranslate"><span class=pre >UINTx_MAX</span></code> unsigned, <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span></code> signed (NXmx has no equivalent). NXmx <code class="docutils literal notranslate"><span class=pre >underload_value</span></code> is written too: <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span> <span class=pre >+</span> <span class=pre >1</span></code> for signed, <code class="docutils literal notranslate"><span class=pre >0</span></code> for unsigned</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bit_depth_image</span></code></p> <td><p></p> <td><p>stored image bit depth (DECTRIS convention, not NXmx). Equal to <code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code> where that is written, i.e. for unsigned images</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >acquisition_type</span></code></p> <td><p></p> <td><p>always <code class="docutils literal notranslate"><span class=pre >triggered</span></code> (Dectris convention)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >jungfrau_conversion_applied</span></code></p> <td><p></p> <td><p>JUNGFRAU photon/keV conversion applied</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >jungfrau_conversion_factor</span></code></p> <td><p>eV</p> <td><p>conversion factor</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >geometry_transformation_applied</span></code></p> <td><p></p> <td><p>module→full-detector geometry applied</p> </table> <p>NeXus has no concept of a fill or no-data value — it expects bad pixels to be flagged in <code class="docutils literal notranslate"><span class=pre >pixel_mask</span></code>, which Jungfraujoch also writes. The in-band sentinel above is a DECTRIS compatibility convention: SIMPLON specifies that masked pixels are flagged with <code class="docutils literal notranslate"><span class=pre >2^bit_depth_image</span> <span class=pre >-</span> <span class=pre >1</span></code>.</p> <p>For an <strong>unsigned</strong> image the sentinel and the saturation code are the same value, so a saturated pixel and a masked one cannot be told apart — the FPGA collapses both onto <code class="docutils literal notranslate"><span class=pre >UINTx_MAX</span></code>. Signed images keep them separate: <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span></code> is the marker, <code class="docutils literal notranslate"><span class=pre >INTx_MAX</span></code> is saturation.</p> <p><code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code> is written for <strong>unsigned</strong> images only. DIALS remaps the top two codes of <code class="docutils literal notranslate"><span class=pre >2^bit_depth_readout</span></code> to <code class="docutils literal notranslate"><span class=pre >-1</span></code> and <code class="docutils literal notranslate"><span class=pre >-2</span></code> whenever the field is present, regardless of the pixel type: for an unsigned image those land below <code class="docutils literal notranslate"><span class=pre >underload_value</span></code> and are correctly masked, but for a signed one they land inside the trusted range and a saturated pixel would be integrated as a count of <code class="docutils literal notranslate"><span class=pre >-2</span></code>. Signed images are read correctly without the field; unsigned 32-bit cannot be read at all without it.</p> </section> <section id=user-supplied-metadata-header-appendix-and-image-appendix > <h3 id=user-supplied-metadata-header-appendix-and-image-appendix ><a class=toc-backref href="#id27" role=doc-backlink >4.11 User-supplied metadata: <code class="docutils literal notranslate"><span class=pre >header_appendix</span></code> and <code class="docutils literal notranslate"><span class=pre >image_appendix</span></code></a><a class=headerlink href="#user-supplied-metadata-header-appendix-and-image-appendix" title="Link to this heading"></a></h3> <p>Facilities frequently need to attach metadata that Jungfraujoch does not model explicitly. Two free-form JSON fields in the <code class="docutils literal notranslate"><span class=pre >/start</span></code> request (<code class="docutils literal notranslate"><span class=pre >broker/jfjoch_api.yaml</span></code>) provide this without any schema change; both accept <em>any valid JSON</em>:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class=head ><p>Carried in</p> <th class=head ><p>Persisted to HDF5?</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >header_appendix</span></code></p> <td><p>the <strong>start</strong> message, under <code class="docutils literal notranslate"><span class=pre >user_data.user</span></code> (see <a class="reference internal" href=CBOR.html ><span class="std std-doc">CBOR</span></a>)</p> <td><p>no — except the <code class="docutils literal notranslate"><span class=pre >hdf5</span></code> sub-object (below)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image_appendix</span></code></p> <td><p><strong>every image</strong> message, as <code class="docutils literal notranslate"><span class=pre >user_data</span></code></p> <td><p>no</p> </table> <p>Both are forwarded verbatim through the ZeroMQ/CBOR stream to every downstream consumer (writer, republished analysis, viewers), so they are the recommended channel for facility- or beamline-specific provenance (proposal, operator, optics state, per-image trigger info, …) that has no dedicated API field.</p> <p><strong>Persisting selected values to HDF5.</strong> <code class="docutils literal notranslate"><span class=pre >header_appendix</span></code> is normally <em>not</em> written to the master file. As an exception, if it contains a key <code class="docutils literal notranslate"><span class=pre >hdf5</span></code> whose value is a JSON object of scalars (strings and numbers — no arrays or nested objects), the writer stores each entry under <code class="docutils literal notranslate"><span class=pre >/entry/user/&lt;key&gt;</span></code>.</p> <p>For example, a <code class="docutils literal notranslate"><span class=pre >/start</span></code> request containing:</p> <div class="highlight-json notranslate"><div class=highlight ><pre><span></span><span class=p >{</span>
<span class=w > </span><span class=nt >"header_appendix"</span><span class=p >:</span><span class=w > </span><span class=p >{</span>
<span class=w > </span><span class=nt >"proposal"</span><span class=p >:</span><span class=w > </span><span class=s2 >"p20001"</span><span class=p >,</span>
<span class=w > </span><span class=nt >"operator"</span><span class=p >:</span><span class=w > </span><span class=s2 >"jdoe"</span><span class=p >,</span>
<span class=w > </span><span class=nt >"hdf5"</span><span class=p >:</span><span class=w > </span><span class=p >{</span><span class=w > </span><span class=nt >"beamline"</span><span class=p >:</span><span class=w > </span><span class=s2 >"X06SA"</span><span class=p >,</span><span class=w > </span><span class=nt >"ring_mode"</span><span class=p >:</span><span class=w > </span><span class=s2 >"top-up"</span><span class=p >,</span><span class=w > </span><span class=nt >"attenuator_foils"</span><span class=p >:</span><span class=w > </span><span class=mi >2</span><span class=w > </span><span class=p >}</span>
<span class=w > </span><span class=p >},</span>
<span class=w > </span><span class=nt >"image_appendix"</span><span class=p >:</span><span class=w > </span><span class=p >{</span><span class=w > </span><span class=nt >"trigger_source"</span><span class=p >:</span><span class=w > </span><span class=s2 >"external"</span><span class=w > </span><span class=p >}</span>
<span class=p >}</span>
</pre></div> </div> <p>forwards the whole <code class="docutils literal notranslate"><span class=pre >header_appendix</span></code> as <code class="docutils literal notranslate"><span class=pre >user_data.user</span></code> on the start message and <code class="docutils literal notranslate"><span class=pre >{"trigger_source":</span> <span class=pre >"external"}</span></code> as <code class="docutils literal notranslate"><span class=pre >user_data</span></code> on every image message, and writes three scalars into the master file:</p> <div class="highlight-default notranslate"><div class=highlight ><pre><span></span><span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >user</span><span class=o >/</span><span class=n >beamline</span> <span class=o >=</span> <span class=s2 >"X06SA"</span>
<span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >user</span><span class=o >/</span><span class=n >ring_mode</span> <span class=o >=</span> <span class=s2 >"top-up"</span>
<span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >user</span><span class=o >/</span><span class=n >attenuator_foils</span> <span class=o >=</span> <span class=mi >2</span>
</pre></div> </div> </section> </section> <section id=notes > <h2 id=notes ><a class=toc-backref href="#id28" role=doc-backlink >5. Notes</a><a class=headerlink href="#notes" title="Link to this heading"></a></h2> <ul class=simple > <li><p><strong>Units</strong> are written as the HDF5 <code class="docutils literal notranslate"><span class=pre >units</span></code> attribute on the dataset (e.g. <code class="docutils literal notranslate"><span class=pre >m</span></code>, <code class="docutils literal notranslate"><span class=pre >eV</span></code>, <code class="docutils literal notranslate"><span class=pre >deg</span></code>, <code class="docutils literal notranslate"><span class=pre >Angstrom</span></code>, <code class="docutils literal notranslate"><span class=pre >Angstrom^-1</span></code>, <code class="docutils literal notranslate"><span class=pre >Angstrom^2</span></code>, <code class="docutils literal notranslate"><span class=pre >pixel</span></code>, <code class="docutils literal notranslate"><span class=pre >s</span></code>).</p> <li><p><strong>Sentinels.</strong> Missing per-image values are <code class="docutils literal notranslate"><span class=pre >NaN</span></code> (floats) or <code class="docutils literal notranslate"><span class=pre >-1</span></code>/<code class="docutils literal notranslate"><span class=pre >0</span></code> (integer indices); image pixels use <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span></code> / <code class="docutils literal notranslate"><span class=pre >UINTx_MAX</span></code>.</p> <li><p><strong>Master vs data file.</strong> In legacy/VDS formats the analysis arrays physically live in the data files; the master file links to them (external links in legacy, virtual datasets in VDS). In the integrated format there are no data files and everything is in one place.</p> <li><p><strong>CXI / CrystFEL.</strong> <code class="docutils literal notranslate"><span class=pre >/entry/MX</span></code> follows the CXI peak-list convention; see <a class="reference external" href="https://raw.githubusercontent.com/cxidb/CXI/master/cxi_file_format.pdf">CXI file format</a>.</p> </ul> </section> </section> </article> </div> </div> </main> </div> <footer class=md-footer > <div class=md-footer-nav > <nav class="md-footer-nav__inner md-grid"> <a href=CBOR.html title="CBOR messages" class="md-flex md-footer-nav__link md-footer-nav__link--prev" rel=prev > <div class="md-flex__cell md-flex__cell--shrink"> <i class="md-icon md-icon--arrow-back md-footer-nav__button"></i> </div> <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"> <span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Previous" </span> CBOR messages </span> </div> </a> <a href=IMAGE_STREAM.html title="Data streams" class="md-flex md-footer-nav__link md-footer-nav__link--next" rel=next > <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"><span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Next" </span> Data streams </span> </div> <div class="md-flex__cell md-flex__cell--shrink"><i class="md-icon md-icon--arrow-forward md-footer-nav__button"></i> </div> </a> </nav> </div> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-footer-copyright > <div class=md-footer-copyright__highlight > &#169; Copyright 2024, Paul Scherrer Institute. </div> Created using <a href="http://www.sphinx-doc.org/">Sphinx</a> 8.1.3. and <a href="https://github.com/bashtage/sphinx-material/">Material for Sphinx</a> </div> </div> </div> </footer> <script src="_static/javascripts/application.js"></script> <script>app.initialize({version: "1.0.4", url: {base: ".."}})</script>
Reference in New Issue View Git Blame Copy Permalink