mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 62 Wiki Activity
Files
5cbdabef27e910dafdeac9cf9aeb8e4c5d2e034b
Jungfraujoch/HDF5.html
T
leonarski_f 5cbdabef27 Deploy site
2026-06-25 21:18:12 +00:00

18 lines
111 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html> <html lang=en data-content_root="./"> <meta charset=utf-8 /> <meta name=viewport content="width=device-width, initial-scale=1.0" /><meta name=viewport content="width=device-width, initial-scale=1" /> <meta name=viewport content="width=device-width,initial-scale=1"> <meta http-equiv=x-ua-compatible content="ie=edge"> <meta name="lang:clipboard.copy" content="Copy to clipboard"> <meta name="lang:clipboard.copied" content="Copied to clipboard"> <meta name="lang:search.language" content=en > <meta name="lang:search.pipeline.stopwords" content=True > <meta name="lang:search.pipeline.trimmer" content=True > <meta name="lang:search.result.none" content="No matching documents"> <meta name="lang:search.result.one" content="1 matching document"> <meta name="lang:search.result.other" content="# matching documents"> <meta name="lang:search.tokenizer" content="[\s\-]+"> <link href="https://fonts.gstatic.com/" rel=preconnect crossorigin> <link href="https://fonts.googleapis.com/css?family=Roboto+Mono:400,500,700|Roboto:300,400,400i,700&display=fallback" rel=stylesheet > <style> body, input { font-family: "Roboto", "Helvetica Neue", Helvetica, Arial, sans-serif } code, kbd, pre { font-family: "Roboto Mono", "Courier New", Courier, monospace } </style> <link rel=stylesheet href="_static/stylesheets/application.css"/> <link rel=stylesheet href="_static/stylesheets/application-palette.css"/> <link rel=stylesheet href="_static/stylesheets/application-fixes.css"/> <link rel=stylesheet href="_static/fonts/material-icons.css"/> <meta name=theme-color content="#3f51b5"> <script src="_static/javascripts/modernizr.js"></script> <title>HDF5 / NeXus data format &#8212; Jungfraujoch 1.0.0-rc.155 documentation</title> <link rel=stylesheet type="text/css" href="_static/pygments.css?v=83e35b93" /> <link rel=stylesheet type="text/css" href="_static/material.css?v=79c92029" /> <script src="_static/documentation_options.js?v=502bdfb1"></script> <script src="_static/doctools.js?v=9bcbadda"></script> <script src="_static/sphinx_highlight.js?v=dc90522c"></script> <link rel=icon href="_static/jfjoch.png"/> <link rel=index title=Index href=genindex.html /> <link rel=search title=Search href=search.html /> <link rel=next title="Data streams" href=IMAGE_STREAM.html /> <link rel=prev title="CBOR messages" href=CBOR.html /> <body dir=ltr data-md-color-primary=indigo data-md-color-accent=lime> <svg class=md-svg > <defs data-children-count=0 > <svg xmlns="http://www.w3.org/2000/svg" width=500 height=500 viewBox="0 0 500 500" id=__gitlab ><path fill=currentColor d="M93.667 473.347l90.684-279.097H2.983l90.684 279.097z" transform="translate(156.198 1.16)"/><path fill=currentColor d="M221.333 473.345L130.649 194.25H3.557l217.776 279.095z" transform="translate(28.531 1.16)" opacity=.7 /><path fill=currentColor d="M32 195.155L4.441 279.97a18.773 18.773 0 0 0 6.821 20.99l238.514 173.29L32 195.155z" transform="translate(.089 .256)" opacity=.5 /><path fill=currentColor d="M2.667-84.844h127.092L75.14-252.942c-2.811-8.649-15.047-8.649-17.856 0L2.667-84.844z" transform="translate(29.422 280.256)"/><path fill=currentColor d="M2.667 473.345L93.351 194.25h127.092L2.667 473.345z" transform="translate(247.198 1.16)" opacity=.7 /><path fill=currentColor d="M221.334 195.155l27.559 84.815a18.772 18.772 0 0 1-6.821 20.99L3.557 474.25l217.777-279.095z" transform="translate(246.307 .256)" opacity=.5 /><path fill=currentColor d="M130.667-84.844H3.575l54.618-168.098c2.811-8.649 15.047-8.649 17.856 0l54.618 168.098z" transform="translate(336.974 280.256)"/></svg> </defs> </svg> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer > <input class=md-toggle data-md-toggle=search type=checkbox id=__search > <label class=md-overlay data-md-component=overlay for=__drawer ></label> <a href="#HDF5" tabindex=1 class=md-skip > Skip to content </a> <header class=md-header data-md-component=header > <nav class="md-header-nav md-grid"> <div class="md-flex navheader"> <div class="md-flex__cell md-flex__cell--shrink"> <a href=index.html title="Jungfraujoch 1.0.0-rc.155 documentation" class="md-header-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--menu md-header-nav__button" for=__drawer ></label> </div> <div class="md-flex__cell md-flex__cell--stretch"> <div class="md-flex__ellipsis md-header-nav__title" data-md-component=title > <span class=md-header-nav__topic >PSI Jungfraujoch</span> <span class=md-header-nav__topic > HDF5 / NeXus data format </span> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--search md-header-nav__button" for=__search ></label> <div class=md-search data-md-component=search role=dialog > <label class=md-search__overlay for=__search ></label> <div class=md-search__inner role=search > <form class=md-search__form action=search.html method=get name=search > <input type=text class=md-search__input name=q placeholder=""Search"" autocapitalize=off autocomplete=off spellcheck=false data-md-component=query data-md-state=active > <label class="md-icon md-search__icon" for=__search ></label> <button type=reset class="md-icon md-search__icon" data-md-component=reset tabindex=-1 > &#xE5CD; </button> </form> <div class=md-search__output > <div class=md-search__scrollwrap data-md-scrollfix> <div class=md-search-result data-md-component=result > <div class=md-search-result__meta > Type to start searching </div> <ol class=md-search-result__list ></ol> </div> </div> </div> </div> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <div class=md-header-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> </div> <script src="_static/javascripts/version_dropdown.js"></script> <script> var json_loc = ""versions.json"", target_loc = "../", text = "Versions"; $( document ).ready( add_version_dropdown(json_loc, target_loc, text)); </script> </div> </nav> </header> <div class=md-container > <nav class=md-tabs data-md-component=tabs > <div class="md-tabs__inner md-grid"> <ul class=md-tabs__list > <li class=md-tabs__item ><a href=index.html class=md-tabs__link >Jungfraujoch 1.0.0-rc.155 documentation</a> </ul> </div> </nav> <main class=md-main > <div class="md-main__inner md-grid" data-md-component=container > <div class="md-sidebar md-sidebar--primary" data-md-component=navigation > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--primary" data-md-level=0 > <label class="md-nav__title md-nav__title--site" for=__drawer > <a href=index.html title="Jungfraujoch 1.0.0-rc.155 documentation" class="md-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> <a href=index.html title="Jungfraujoch 1.0.0-rc.155 documentation">PSI Jungfraujoch</a> </label> <div class=md-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> <ul class=md-nav__list > <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >General</span></span> <li class=md-nav__item > <a href=ACKNOWLEDGEMENT.html class=md-nav__link >Acknowledgements</a> <li class=md-nav__item > <a href=LICENSE.html class=md-nav__link >License</a> <li class=md-nav__item > <a href=DETECTORS.html class=md-nav__link >Supported detectors</a> <li class=md-nav__item > <a href="DETECTORS.html#dectris-detectors" class=md-nav__link >DECTRIS detectors</a> <li class=md-nav__item > <a href=HARDWARE.html class=md-nav__link >Hardware requirements</a> <li class=md-nav__item > <a href=SOFTWARE.html class=md-nav__link >Software requirements</a> <li class=md-nav__item > <a href=VERSIONING.html class=md-nav__link >Semantic versioning</a> <li class=md-nav__item > <a href=DEPLOYMENT.html class=md-nav__link >Deployment</a> <li class=md-nav__item > <a href=REPOSITORIES.html class=md-nav__link >Linux package repositories</a> <li class=md-nav__item > <a href=CHANGELOG.html class=md-nav__link >Changelog</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Software</span></span> <li class=md-nav__item > <a href=JFJOCH_BROKER.html class=md-nav__link >jfjoch_broker</a> <li class=md-nav__item > <a href=JFJOCH_WRITER.html class=md-nav__link >jfjoch_writer</a> <li class=md-nav__item > <a href=JFJOCH_PROCESS.html class=md-nav__link >jfjoch_process</a> <li class=md-nav__item > <a href=JFJOCH_VIEWER.html class=md-nav__link >jfjoch_viewer</a> <li class=md-nav__item > <a href=SOFTWARE_INTEGRATION.html class=md-nav__link >Integration with MX data processing software</a> <li class=md-nav__item > <a href=TOOLS.html class=md-nav__link >Tools</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >FPGA</span></span> <li class=md-nav__item > <a href=FPGA.html class=md-nav__link >FPGA smartNIC</a> <li class=md-nav__item > <a href=FPGA_LICENSE.html class=md-nav__link >FPGA license</a> <li class=md-nav__item > <a href=FPGA_DESIGN.html class=md-nav__link >FPGA data flow</a> <li class=md-nav__item > <a href=FPGA_NETWORK.html class=md-nav__link >FPGA network</a> <li class=md-nav__item > <a href=FPGA_PCIE_DRIVER.html class=md-nav__link >FPGA PCIe driver</a> <li class=md-nav__item > <a href=FPGA_SETTINGS.html class=md-nav__link >FPGA advanced reference</a> <li class=md-nav__item > <a href=FPGA_DATA_ANALYSIS.html class=md-nav__link >FPGA data analysis</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Reference</span></span> <li class=md-nav__item > <a href=DETECTOR_GEOMETRY.html class=md-nav__link >Detector geometry</a> <li class=md-nav__item > <a href=CPU_DATA_ANALYSIS.html class=md-nav__link >CPU-side crystallographic data analysis (Jungfraujoch)</a> <li class=md-nav__item > <a href=OPENAPI.html class=md-nav__link >OpenAPI</a> <li class=md-nav__item > <a href=OPENAPI_SPECS.html class=md-nav__link >OpenAPI specification</a> <li class=md-nav__item > <a href=CBOR.html class=md-nav__link >CBOR messages</a> <li class=md-nav__item > <input class="md-toggle md-nav__toggle" data-md-toggle=toc type=checkbox id=__toc > <label class="md-nav__link md-nav__link--active" for=__toc > HDF5 / NeXus data format </label> <a href="#" class="md-nav__link md-nav__link--active">HDF5 / NeXus data format</a> <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> <ul class=md-nav__list > <li class=md-nav__item > <a href="#motivation-derived-metadata-and-fair-data" class=md-nav__link >1. Motivation: derived metadata and FAIR data</a> <li class=md-nav__item > <a href="#file-layout" class=md-nav__link >2. File layout</a> <li class=md-nav__item > <a href="#nxmx-standard-content" class=md-nav__link >3. NXmx-standard content</a> <li class=md-nav__item > <a href="#extensions-beyond-nxmx" class=md-nav__link >4. Extensions beyond NXmx</a> <li class=md-nav__item > <a href="#notes" class=md-nav__link >5. Notes</a> </ul> <li class=md-nav__item > <a href=IMAGE_STREAM.html class=md-nav__link >Data streams</a> <li class=md-nav__item > <a href=PIXEL_MASK.html class=md-nav__link >Pixel mask</a> <li class=md-nav__item > <a href=WEB_FRONTEND.html class=md-nav__link >Web frontend</a> <li class=md-nav__item > <a href=TESTS.html class=md-nav__link >Tests</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >OpenAPI Python client</span></span> <li class=md-nav__item > <a href="python_client/README.html" class=md-nav__link >jfjoch-client</a> <li class=md-nav__item > <a href="python_client/README.html#license-clarification" class=md-nav__link >License Clarification</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html" class=md-nav__link >jfjoch_client.DefaultApi</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#cancel-post" class=md-nav__link ><strong>cancel_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-get" class=md-nav__link ><strong>config_azim_int_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-put" class=md-nav__link ><strong>config_azim_int_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-get" class=md-nav__link ><strong>config_dark_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-put" class=md-nav__link ><strong>config_dark_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-get" class=md-nav__link ><strong>config_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-put" class=md-nav__link ><strong>config_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-get" class=md-nav__link ><strong>config_file_writer_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-put" class=md-nav__link ><strong>config_file_writer_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-conversion-post" class=md-nav__link ><strong>config_image_format_conversion_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-get" class=md-nav__link ><strong>config_image_format_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-put" class=md-nav__link ><strong>config_image_format_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-raw-post" class=md-nav__link ><strong>config_image_format_raw_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-get" class=md-nav__link ><strong>config_indexing_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-put" class=md-nav__link ><strong>config_indexing_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-get" class=md-nav__link ><strong>config_instrument_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-put" class=md-nav__link ><strong>config_instrument_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-put" class=md-nav__link ><strong>config_internal_generator_image_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-tiff-put" class=md-nav__link ><strong>config_internal_generator_image_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-get" class=md-nav__link ><strong>config_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-tiff-get" class=md-nav__link ><strong>config_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-get" class=md-nav__link ><strong>config_roi_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-put" class=md-nav__link ><strong>config_roi_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-get" class=md-nav__link ><strong>config_select_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-put" class=md-nav__link ><strong>config_select_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-get" class=md-nav__link ><strong>config_spot_finding_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-put" class=md-nav__link ><strong>config_spot_finding_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-get" class=md-nav__link ><strong>config_user_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-put" class=md-nav__link ><strong>config_user_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-get" class=md-nav__link ><strong>config_user_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-put" class=md-nav__link ><strong>config_user_mask_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-get" class=md-nav__link ><strong>config_zeromq_metadata_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-put" class=md-nav__link ><strong>config_zeromq_metadata_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-get" class=md-nav__link ><strong>config_zeromq_preview_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-put" class=md-nav__link ><strong>config_zeromq_preview_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#deactivate-post" class=md-nav__link ><strong>deactivate_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#detector-status-get" class=md-nav__link ><strong>detector_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#fpga-status-get" class=md-nav__link ><strong>fpga_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-clear-post" class=md-nav__link ><strong>image_buffer_clear_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-cbor-get" class=md-nav__link ><strong>image_buffer_image_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-jpeg-get" class=md-nav__link ><strong>image_buffer_image_jpeg_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-tiff-get" class=md-nav__link ><strong>image_buffer_image_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-start-cbor-get" class=md-nav__link ><strong>image_buffer_start_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-status-get" class=md-nav__link ><strong>image_buffer_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-pusher-status-get" class=md-nav__link ><strong>image_pusher_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#initialize-post" class=md-nav__link ><strong>initialize_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#pedestal-post" class=md-nav__link ><strong>pedestal_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-pedestal-tiff-get" class=md-nav__link ><strong>preview_pedestal_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-bin-get" class=md-nav__link ><strong>preview_plot_bin_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-get" class=md-nav__link ><strong>preview_plot_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#result-scan-get" class=md-nav__link ><strong>result_scan_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#start-post" class=md-nav__link ><strong>start_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-calibration-get" class=md-nav__link ><strong>statistics_calibration_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-data-collection-get" class=md-nav__link ><strong>statistics_data_collection_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-get" class=md-nav__link ><strong>statistics_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#status-get" class=md-nav__link ><strong>status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#trigger-post" class=md-nav__link ><strong>trigger_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#version-get" class=md-nav__link ><strong>version_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-till-done-post" class=md-nav__link ><strong>wait_till_done_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-until-running-post" class=md-nav__link ><strong>wait_until_running_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-event-code-get" class=md-nav__link ><strong>xfel_event_code_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-pulse-id-get" class=md-nav__link ><strong>xfel_pulse_id_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/AzimIntSettings.html" class=md-nav__link >AzimIntSettings</a> <li class=md-nav__item > <a href="python_client/docs/BrokerStatus.html" class=md-nav__link >BrokerStatus</a> <li class=md-nav__item > <a href="python_client/docs/CalibrationStatisticsInner.html" class=md-nav__link >CalibrationStatisticsInner</a> <li class=md-nav__item > <a href="python_client/docs/ColorScale.html" class=md-nav__link >ColorScale</a> <li class=md-nav__item > <a href="python_client/docs/DarkMaskSettings.html" class=md-nav__link >DarkMaskSettings</a> <li class=md-nav__item > <a href="python_client/docs/DatasetSettings.html" class=md-nav__link >DatasetSettings</a> <li class=md-nav__item > <a href="python_client/docs/DatasetSettingsUnitCell.html" class=md-nav__link >DatasetSettingsUnitCell</a> <li class=md-nav__item > <a href="python_client/docs/DatasetSettingsXrayFluorescenceSpectrum.html" class=md-nav__link >DatasetSettingsXrayFluorescenceSpectrum</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html" class=md-nav__link >jfjoch_client.DefaultApi</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#cancel-post" class=md-nav__link ><strong>cancel_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-get" class=md-nav__link ><strong>config_azim_int_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-put" class=md-nav__link ><strong>config_azim_int_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-get" class=md-nav__link ><strong>config_dark_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-put" class=md-nav__link ><strong>config_dark_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-get" class=md-nav__link ><strong>config_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-put" class=md-nav__link ><strong>config_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-get" class=md-nav__link ><strong>config_file_writer_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-put" class=md-nav__link ><strong>config_file_writer_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-conversion-post" class=md-nav__link ><strong>config_image_format_conversion_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-get" class=md-nav__link ><strong>config_image_format_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-put" class=md-nav__link ><strong>config_image_format_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-raw-post" class=md-nav__link ><strong>config_image_format_raw_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-get" class=md-nav__link ><strong>config_indexing_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-put" class=md-nav__link ><strong>config_indexing_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-get" class=md-nav__link ><strong>config_instrument_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-put" class=md-nav__link ><strong>config_instrument_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-put" class=md-nav__link ><strong>config_internal_generator_image_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-tiff-put" class=md-nav__link ><strong>config_internal_generator_image_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-get" class=md-nav__link ><strong>config_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-tiff-get" class=md-nav__link ><strong>config_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-get" class=md-nav__link ><strong>config_roi_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-put" class=md-nav__link ><strong>config_roi_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-get" class=md-nav__link ><strong>config_select_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-put" class=md-nav__link ><strong>config_select_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-get" class=md-nav__link ><strong>config_spot_finding_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-put" class=md-nav__link ><strong>config_spot_finding_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-get" class=md-nav__link ><strong>config_user_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-put" class=md-nav__link ><strong>config_user_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-get" class=md-nav__link ><strong>config_user_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-put" class=md-nav__link ><strong>config_user_mask_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-get" class=md-nav__link ><strong>config_zeromq_metadata_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-put" class=md-nav__link ><strong>config_zeromq_metadata_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-get" class=md-nav__link ><strong>config_zeromq_preview_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-put" class=md-nav__link ><strong>config_zeromq_preview_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#deactivate-post" class=md-nav__link ><strong>deactivate_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#detector-status-get" class=md-nav__link ><strong>detector_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#fpga-status-get" class=md-nav__link ><strong>fpga_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-clear-post" class=md-nav__link ><strong>image_buffer_clear_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-cbor-get" class=md-nav__link ><strong>image_buffer_image_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-jpeg-get" class=md-nav__link ><strong>image_buffer_image_jpeg_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-tiff-get" class=md-nav__link ><strong>image_buffer_image_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-start-cbor-get" class=md-nav__link ><strong>image_buffer_start_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-status-get" class=md-nav__link ><strong>image_buffer_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-pusher-status-get" class=md-nav__link ><strong>image_pusher_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#initialize-post" class=md-nav__link ><strong>initialize_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#pedestal-post" class=md-nav__link ><strong>pedestal_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-pedestal-tiff-get" class=md-nav__link ><strong>preview_pedestal_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-bin-get" class=md-nav__link ><strong>preview_plot_bin_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-get" class=md-nav__link ><strong>preview_plot_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#result-scan-get" class=md-nav__link ><strong>result_scan_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#start-post" class=md-nav__link ><strong>start_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-calibration-get" class=md-nav__link ><strong>statistics_calibration_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-data-collection-get" class=md-nav__link ><strong>statistics_data_collection_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-get" class=md-nav__link ><strong>statistics_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#status-get" class=md-nav__link ><strong>status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#trigger-post" class=md-nav__link ><strong>trigger_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#version-get" class=md-nav__link ><strong>version_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-till-done-post" class=md-nav__link ><strong>wait_till_done_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-until-running-post" class=md-nav__link ><strong>wait_until_running_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-event-code-get" class=md-nav__link ><strong>xfel_event_code_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-pulse-id-get" class=md-nav__link ><strong>xfel_pulse_id_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/Detector.html" class=md-nav__link >Detector</a> <li class=md-nav__item > <a href="python_client/docs/DetectorList.html" class=md-nav__link >DetectorList</a> <li class=md-nav__item > <a href="python_client/docs/DetectorListDetectorsInner.html" class=md-nav__link >DetectorListDetectorsInner</a> <li class=md-nav__item > <a href="python_client/docs/DetectorListElement.html" class=md-nav__link >DetectorListElement</a> <li class=md-nav__item > <a href="python_client/docs/DetectorModule.html" class=md-nav__link >DetectorModule</a> <li class=md-nav__item > <a href="python_client/docs/DetectorModuleDirection.html" class=md-nav__link >DetectorModuleDirection</a> <li class=md-nav__item > <a href="python_client/docs/DetectorPowerState.html" class=md-nav__link >DetectorPowerState</a> <li class=md-nav__item > <a href="python_client/docs/DetectorSelection.html" class=md-nav__link >DetectorSelection</a> <li class=md-nav__item > <a href="python_client/docs/DetectorSettings.html" class=md-nav__link >DetectorSettings</a> <li class=md-nav__item > <a href="python_client/docs/DetectorState.html" class=md-nav__link >DetectorState</a> <li class=md-nav__item > <a href="python_client/docs/DetectorStatus.html" class=md-nav__link >DetectorStatus</a> <li class=md-nav__item > <a href="python_client/docs/DetectorTiming.html" class=md-nav__link >DetectorTiming</a> <li class=md-nav__item > <a href="python_client/docs/DetectorType.html" class=md-nav__link >DetectorType</a> <li class=md-nav__item > <a href="python_client/docs/ErrorMessage.html" class=md-nav__link >ErrorMessage</a> <li class=md-nav__item > <a href="python_client/docs/FileWriterFormat.html" class=md-nav__link >FileWriterFormat</a> <li class=md-nav__item > <a href="python_client/docs/FileWriterSettings.html" class=md-nav__link >FileWriterSettings</a> <li class=md-nav__item > <a href="python_client/docs/FpgaStatusInner.html" class=md-nav__link >FpgaStatusInner</a> <li class=md-nav__item > <a href="python_client/docs/GeomRefinementAlgorithm.html" class=md-nav__link >GeomRefinementAlgorithm</a> <li class=md-nav__item > <a href="python_client/docs/GridPlot.html" class=md-nav__link >GridPlot</a> <li class=md-nav__item > <a href="python_client/docs/GridPlots.html" class=md-nav__link >GridPlots</a> <li class=md-nav__item > <a href="python_client/docs/GridScan.html" class=md-nav__link >GridScan</a> <li class=md-nav__item > <a href="python_client/docs/GridScanResult.html" class=md-nav__link >GridScanResult</a> <li class=md-nav__item > <a href="python_client/docs/GridScanResultImagesInner.html" class=md-nav__link >GridScanResultImagesInner</a> <li class=md-nav__item > <a href="python_client/docs/ImageBufferStatus.html" class=md-nav__link >ImageBufferStatus</a> <li class=md-nav__item > <a href="python_client/docs/ImageFormatSettings.html" class=md-nav__link >ImageFormatSettings</a> <li class=md-nav__item > <a href="python_client/docs/ImagePusherStatus.html" class=md-nav__link >ImagePusherStatus</a> <li class=md-nav__item > <a href="python_client/docs/ImagePusherType.html" class=md-nav__link >ImagePusherType</a> <li class=md-nav__item > <a href="python_client/docs/IndexingAlgorithm.html" class=md-nav__link >IndexingAlgorithm</a> <li class=md-nav__item > <a href="python_client/docs/IndexingSettings.html" class=md-nav__link >IndexingSettings</a> <li class=md-nav__item > <a href="python_client/docs/InstrumentMetadata.html" class=md-nav__link >InstrumentMetadata</a> <li class=md-nav__item > <a href="python_client/docs/JfjochBrokerApi.html" class=md-nav__link >jfjoch_client.JfjochBrokerApi</a> <li class=md-nav__item > <a href="python_client/docs/JfjochBrokerApi.html#config-zeromq-metadata-put" class=md-nav__link ><strong>config_zeromq_metadata_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/JfjochSettings.html" class=md-nav__link >JfjochSettings</a> <li class=md-nav__item > <a href="python_client/docs/JfjochSettingsSsl.html" class=md-nav__link >JfjochSettingsSsl</a> <li class=md-nav__item > <a href="python_client/docs/JfjochStatistics.html" class=md-nav__link >JfjochStatistics</a> <li class=md-nav__item > <a href="python_client/docs/MeasurementStatistics.html" class=md-nav__link >MeasurementStatistics</a> <li class=md-nav__item > <a href="python_client/docs/PcieDevicesInner.html" class=md-nav__link >PcieDevicesInner</a> <li class=md-nav__item > <a href="python_client/docs/PixelMaskStatistics.html" class=md-nav__link >PixelMaskStatistics</a> <li class=md-nav__item > <a href="python_client/docs/Plot.html" class=md-nav__link >Plot</a> <li class=md-nav__item > <a href="python_client/docs/PlotTypeEnum.html" class=md-nav__link >PlotTypeEnum</a> <li class=md-nav__item > <a href="python_client/docs/PlotUnitX.html" class=md-nav__link >PlotUnitX</a> <li class=md-nav__item > <a href="python_client/docs/Plots.html" class=md-nav__link >Plots</a> <li class=md-nav__item > <a href="python_client/docs/PreviewSettings.html" class=md-nav__link >PreviewSettings</a> <li class=md-nav__item > <a href="python_client/docs/RoiAzimList.html" class=md-nav__link >RoiAzimList</a> <li class=md-nav__item > <a href="python_client/docs/RoiAzimuthal.html" class=md-nav__link >RoiAzimuthal</a> <li class=md-nav__item > <a href="python_client/docs/RoiBox.html" class=md-nav__link >RoiBox</a> <li class=md-nav__item > <a href="python_client/docs/RoiBoxList.html" class=md-nav__link >RoiBoxList</a> <li class=md-nav__item > <a href="python_client/docs/RoiCircle.html" class=md-nav__link >RoiCircle</a> <li class=md-nav__item > <a href="python_client/docs/RoiCircleList.html" class=md-nav__link >RoiCircleList</a> <li class=md-nav__item > <a href="python_client/docs/RoiDefinitions.html" class=md-nav__link >RoiDefinitions</a> <li class=md-nav__item > <a href="python_client/docs/RotationAxis.html" class=md-nav__link >RotationAxis</a> <li class=md-nav__item > <a href="python_client/docs/ScanResult.html" class=md-nav__link >ScanResult</a> <li class=md-nav__item > <a href="python_client/docs/ScanResultImagesInner.html" class=md-nav__link >ScanResultImagesInner</a> <li class=md-nav__item > <a href="python_client/docs/SpotFindingSettings.html" class=md-nav__link >SpotFindingSettings</a> <li class=md-nav__item > <a href="python_client/docs/StandardDetectorGeometry.html" class=md-nav__link >StandardDetectorGeometry</a> <li class=md-nav__item > <a href="python_client/docs/TcpSettings.html" class=md-nav__link >TcpSettings</a> <li class=md-nav__item > <a href="python_client/docs/UnitCell.html" class=md-nav__link >UnitCell</a> <li class=md-nav__item > <a href="python_client/docs/ZeromqMetadataSettings.html" class=md-nav__link >ZeromqMetadataSettings</a> <li class=md-nav__item > <a href="python_client/docs/ZeromqPreviewSettings.html" class=md-nav__link >ZeromqPreviewSettings</a> <li class=md-nav__item > <a href="python_client/docs/ZeromqSettings.html" class=md-nav__link >ZeromqSettings</a> </ul> </nav> </div> </div> </div> <div class="md-sidebar md-sidebar--secondary" data-md-component=toc > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> </div> </div> </div> <div class=md-content > <article class="md-content__inner md-typeset" role=main > <section class="tex2jax_ignore mathjax_ignore" id=hdf5-nexus-data-format > <h1 id=hdf5--page-root >HDF5 / NeXus data format<a class=headerlink href="#hdf5--page-root" title="Link to this heading"></a></h1> <p>Jungfraujoch stores images and on-the-fly analysis results in HDF5 files that aim to be <a class="reference external" href="https://manual.nexusformat.org/classes/applications/NXmx.html">NXmx</a>-compliant. On top of the NXmx application definition, Jungfraujoch records a substantial amount of <em>derived</em> metadata (spot finding, indexing, integration, azimuthal integration, per-image statistics, timing). These extra entries do not exist in NXmx and are documented here so that the layout is unambiguous and reusable.</p> <p>This page documents the <strong>file layout and the data fields</strong>. The operational behaviour of the writer (running, republishing, file finalisation, CBF/TIFF output) is described in <a class="reference internal" href=JFJOCH_WRITER.html ><span class="std std-doc">jfjoch_writer</span></a>. The wire format that feeds the writer is described in <a class="reference internal" href=CBOR.html ><span class="std std-doc">CBOR messages</span></a>; fields below frequently correspond one-to-one to CBOR message fields, and that document is a useful companion for their meaning.</p> <section id=motivation-derived-metadata-and-fair-data > <h2 id=motivation-derived-metadata-and-fair-data >1. Motivation: derived metadata and FAIR data<a class=headerlink href="#motivation-derived-metadata-and-fair-data" title="Link to this heading"></a></h2> <p>The goal of Jungfraujoch is not only to store high-throughput datasets efficiently, but to keep them findable, accessible, interoperable and reusable (FAIR). Jungfraujoch is used for both <strong>rotation</strong> macromolecular crystallography (single- and multi-crystal, including fine-sliced and helical scans) and <strong>serial</strong> crystallography (stills, grid scans); the same concerns apply to both:</p> <ul class=simple > <li><p><strong>Findability.</strong> Raw diffraction images carry almost no descriptive metadata about <em>content</em>. Quantities such as background level, number of diffraction spots, or indexing outcome let a user judge the quality and relevance of a dataset <em>before</em> inspecting the raw images.</p> <li><p><strong>Accessibility at scale.</strong> A single experiment can span tens to hundreds of terabytes. Standard retrieval (e.g. HTTP) makes a dataset <em>available</em> but not <em>inspectable</em> — users would otherwise have to download a large fraction of the data just to decide whether it is useful. Compact derived representations make discovery, assessment and reuse feasible.</p> </ul> <p>Because Jungfraujoch couples acquisition with real-time analysis used to <em>steer</em> experiments, transparency and reproducibility of that analysis matter. As a minimum the writer therefore preserves spot-finding and indexing results together with the filters that were applied, and it can retain an unbiased, down-sampled reference set of unfiltered images for validation and reuse.</p> <section id=two-complementary-layouts-per-image-spots-vs-a-reflection-table > <h3 id=two-complementary-layouts-per-image-spots-vs-a-reflection-table >Two complementary layouts: per-image spots vs. a reflection table<a class=headerlink href="#two-complementary-layouts-per-image-spots-vs-a-reflection-table" title="Link to this heading"></a></h3> <p>Jungfraujoch stores analysis products in two shapes, matching how each is accessed.</p> <p><strong>Per-image spot finding / indexing.</strong> Spot finding and indexing are inherently <em>image-centric</em> — the natural query is “give me the spots for image <em>n</em>” — and this holds for serial stills and for rotation frames alike. For these products Jungfraujoch adopts a layout similar to the <a class="reference external" href="https://www.cxidb.org">Coherent X-ray Imaging (CXI) data bank</a> (Maia, 2012) and the convention understood by <a class="reference external" href="https://www.desy.de/~twhite/crystfel/">CrystFEL</a>: spot properties (position, intensity, Miller index, …) are stored in fixed-size two-dimensional arrays indexed by image number, with each image allocated room for up to a predefined maximum number of spots. These dense arrays are addressed with ordinary HDF5 hyperslab reads, so the spots of a single image are retrieved without traversing variable-length structures. The cost is some storage overhead for unused slots (padded with sentinels), which is acceptable for the access pattern.</p> <p><strong>Integrated reflections.</strong> Integrated intensities are naturally a <em>dataset-wide</em> table, which is exactly the model of the NeXus <a class="reference external" href="https://manual.nexusformat.org/classes/base_classes/NXreflections.html">NXreflections</a> base class. This fits rotation crystallography well, and Jungfraujoch uses NXreflections for its integration results (see §4.2 below). We deliberately do <em>not</em> force spot finding/indexing into a single experiment-wide table: across the hundreds of thousands of patterns typical of serial — or fine-sliced rotation — experiments, that would require aggregating the whole experiment before the spots of one image can be read. We encourage the community to develop standardised NeXus application definitions for image-centric crystallography products that combine NeXus interoperability with the access patterns and scale of modern high-throughput experiments.</p> </section> </section> <section id=file-layout > <h2 id=file-layout >2. File layout<a class=headerlink href="#file-layout" title="Link to this heading"></a></h2> <p>A run is written as one <strong>master file</strong> plus, depending on the format, one or more <strong>data files</strong>:</p> <div class="highlight-default notranslate"><div class=highlight ><pre><span></span><span class=o >&lt;</span><span class=n >prefix</span><span class=o >&gt;</span><span class=n >_master</span><span class=o >.</span><span class=n >h5</span> <span class=c1 ># NXmx master file (metadata + links / virtual datasets)</span>
<span class=o >&lt;</span><span class=n >prefix</span><span class=o >&gt;</span><span class=n >_data_000001</span><span class=o >.</span><span class=n >h5</span> <span class=c1 ># data file: images + per-image analysis</span>
<span class=o >&lt;</span><span class=n >prefix</span><span class=o >&gt;</span><span class=n >_data_000002</span><span class=o >.</span><span class=n >h5</span>
<span class=o >...</span>
</pre></div> </div> <p>The master file is produced by <code class="docutils literal notranslate"><span class=pre >writer/HDF5NXmx.cpp</span></code>; data files by <code class="docutils literal notranslate"><span class=pre >writer/HDF5DataFile.cpp</span></code> and its plugins (<code class="docutils literal notranslate"><span class=pre >writer/HDF5DataFilePlugin*.cpp</span></code>). Files are written to a temporary <code class="docutils literal notranslate"><span class=pre >*.&lt;random&gt;.tmp</span></code> name and renamed on successful close.</p> <p>Three master-file variants exist (set via <code class="docutils literal notranslate"><span class=pre >file_format</span></code>):</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Format</p> <th class="head text-center"><p>Value</p> <th class=head ><p>Master ↔ data linking</p> <tr class=row-even ><td><p><strong>NXmxLegacy</strong> (default)</p> <td class=text-center ><p>1</p> <td><p>One external link in <code class="docutils literal notranslate"><span class=pre >/entry/data</span></code> per data file (<code class="docutils literal notranslate"><span class=pre >data_000001</span></code>, …). HDF5 1.8 compatible — works with Neggia/Durin XDS plugins and Albula 4.0.</p> <tr class=row-odd ><td><p><strong>NXmxVDS</strong></p> <td class=text-center ><p>2</p> <td><p>A single virtual dataset <code class="docutils literal notranslate"><span class=pre >/entry/data/data</span></code> spans all data files; spot finding, azimuthal integration and reflections are linked the same way. Requires HDF5 1.10 / Albula 4.1+.</p> <tr class=row-even ><td><p><strong>NXmxIntegrated</strong></p> <td class=text-center ><p>3</p> <td><p>No separate data files — images and all metadata live in one file. Equivalent in content to the VDS format.</p> </table> <p>In legacy/VDS mode, image-indexed analysis arrays live in the <strong>data files</strong> and are exposed in the master file through external links or virtual datasets; in integrated mode they are written directly into the single file. Throughout this document a “✓ in master” column marks entries that are visible (directly or via link/VDS) from the master file.</p> <p>Images are stored chunked (one image per chunk) and compressed with bitshuffle + LZ4 or bitshuffle + Zstd; signed integer image datasets use <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span></code> as the HDF5 fill value (the “masked / no-data” sentinel), unsigned use <code class="docutils literal notranslate"><span class=pre >UINTx_MAX</span></code>.</p> <section id=reprocessing-output-prefix-process-h5 > <h3 id=reprocessing-output-prefix-process-h5 >Reprocessing output: <code class="docutils literal notranslate"><span class=pre >&lt;prefix&gt;_process.h5</span></code><a class=headerlink href="#reprocessing-output-prefix-process-h5" title="Link to this heading"></a></h3> <p>The offline reprocessing tool <a class="reference internal" href=TOOLS.html ><span class="std std-doc"><code class="docutils literal notranslate"><span class=pre >jfjoch_process</span></code></span></a> (<code class="docutils literal notranslate"><span class=pre >tools/jfjoch_process.cpp</span></code>) re-runs the full analysis pipeline (spot finding, indexing, refinement, integration, scaling) on an existing dataset and writes its results to a master file named <strong><code class="docutils literal notranslate"><span class=pre >&lt;prefix&gt;_process.h5</span></code></strong>. This file uses the <strong>integrated</strong> format, but instead of copying the images its <code class="docutils literal notranslate"><span class=pre >/entry/data/data</span></code> is a <em>virtual dataset that links back to the original image files</em> (<code class="docutils literal notranslate"><span class=pre >hdf5_source_data</span></code> → <code class="docutils literal notranslate"><span class=pre >NXmx::LinkToData_ProcessingVDS</span></code>). The result is a compact, self-describing companion file that holds <em>all</em> the derived analysis (everything in §4) plus a virtual view of the raw images — without duplicating terabytes of data.</p> <p>This is a particularly FAIR-friendly artefact: it can be shared or archived alongside (or instead of) the raw data to convey what is in a dataset and how it processed, while the <code class="docutils literal notranslate"><span class=pre >/entry/data/data</span></code> VDS still resolves to the original images when they are available. <code class="docutils literal notranslate"><span class=pre >jfjoch_process</span></code> can also process an equally-spaced <em>subset</em> of images (start/end/stride), producing a down-sampled reference set.</p> </section> </section> <section id=nxmx-standard-content > <h2 id=nxmx-standard-content >3. NXmx-standard content<a class=headerlink href="#nxmx-standard-content" title="Link to this heading"></a></h2> <p>The entries below are part of, or valid base classes for, the <a class="reference external" href="https://manual.nexusformat.org/classes/applications/NXmx.html">NXmx</a> application definition. “NXmx” = listed in the application definition; “base” = a valid field of the relevant NeXus base class (<code class="docutils literal notranslate"><span class=pre >NXdetector</span></code>, <code class="docutils literal notranslate"><span class=pre >NXsample</span></code>, <code class="docutils literal notranslate"><span class=pre >NXsource</span></code>) but not in the NXmx required/recommended subset.</p> <section id=entry-nxentry > <h3 id=entry-nxentry ><code class="docutils literal notranslate"><span class=pre >/entry</span></code> (NXentry)<a class=headerlink href="#entry-nxentry" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Notes</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >definition</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>value <code class="docutils literal notranslate"><span class=pre >"NXmx"</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >start_time</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>arming time</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >end_time</span></code>, <code class="docutils literal notranslate"><span class=pre >end_time_estimated</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>approximate end time</p> </table> <p>File-level HDF5 attributes <code class="docutils literal notranslate"><span class=pre >file_name</span></code>, <code class="docutils literal notranslate"><span class=pre >file_time</span></code>, <code class="docutils literal notranslate"><span class=pre >HDF5_Version</span></code> are also set.</p> </section> <section id=entry-source-nxsource-entry-instrument-nxinstrument > <h3 id=entry-source-nxsource-entry-instrument-nxinstrument ><code class="docutils literal notranslate"><span class=pre >/entry/source</span></code> (NXsource), <code class="docutils literal notranslate"><span class=pre >/entry/instrument</span></code> (NXinstrument)<a class=headerlink href="#entry-source-nxsource-entry-instrument-nxinstrument" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >source/name</span></code>, <code class="docutils literal notranslate"><span class=pre >source/type</span></code></p> <td class=text-center ><p>NXmx / base</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >source/current</span></code></p> <td class=text-center ><p>base</p> <td><p>A</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >instrument/name</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> </table> </section> <section id=entry-instrument-beam-nxbeam > <h3 id=entry-instrument-beam-nxbeam ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/beam</span></code> (NXbeam)<a class=headerlink href="#entry-instrument-beam-nxbeam" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >incident_wavelength</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>angstrom</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >incident_wavelength_spread</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>angstrom (only if polychromatic)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >total_flux</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>Hz</p> </table> </section> <section id=entry-instrument-attenuator-nxattenuator > <h3 id=entry-instrument-attenuator-nxattenuator ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/attenuator</span></code> (NXattenuator)<a class=headerlink href="#entry-instrument-attenuator-nxattenuator" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >attenuator_transmission</span></code></p> <td class=text-center ><p>NXmx</p> </table> </section> <section id=entry-instrument-detector-nxdetector > <h3 id=entry-instrument-detector-nxdetector ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector</span></code> (NXdetector)<a class=headerlink href="#entry-instrument-detector-nxdetector" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >depends_on</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>→ <code class="docutils literal notranslate"><span class=pre >transformations/rot3</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >beam_center_x</span></code>, <code class="docutils literal notranslate"><span class=pre >beam_center_y</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>pixel</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >distance</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>m</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >count_time</span></code>, <code class="docutils literal notranslate"><span class=pre >frame_time</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>s</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >sensor_thickness</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>m</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >sensor_material</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >description</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >threshold_energy</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>eV (EIGER; written only for a single channel)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >x_pixel_size</span></code>, <code class="docutils literal notranslate"><span class=pre >y_pixel_size</span></code></p> <td class=text-center ><p>base</p> <td><p>m</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >serial_number</span></code></p> <td class=text-center ><p>base</p> <td><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >saturation_value</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >flatfield_applied</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >pixel_mask</span></code>, <code class="docutils literal notranslate"><span class=pre >pixel_mask_applied</span></code></p> <td class=text-center ><p>NXmx</p> <td><p><code class="docutils literal notranslate"><span class=pre >pixel_mask</span></code> is <code class="docutils literal notranslate"><span class=pre >[y,</span> <span class=pre >x]</span></code>, hard-linked from <code class="docutils literal notranslate"><span class=pre >detectorSpecific/pixel_mask</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >countrate_correction_applied</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >number_of_cycles</span></code></p> <td class=text-center ><p>base</p> <td><p>frame-summation factor</p> </table> </section> <section id=entry-instrument-detector-transformations-nxtransformations > <h3 id=entry-instrument-detector-transformations-nxtransformations ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/transformations</span></code> (NXtransformations)<a class=headerlink href="#entry-instrument-detector-transformations-nxtransformations" title="Link to this heading"></a></h3> <p>The NXtransformations <em>mechanism</em> (the <code class="docutils literal notranslate"><span class=pre >depends_on</span></code> chain, <code class="docutils literal notranslate"><span class=pre >transformation_type</span></code>, <code class="docutils literal notranslate"><span class=pre >vector</span></code>, <code class="docutils literal notranslate"><span class=pre >offset</span></code> attributes) is standard. The axis <strong>names</strong> follow the PyFAI PONI convention chosen by Jungfraujoch (see <a class="reference internal" href=DETECTOR_GEOMETRY.html ><span class="std std-doc">DETECTOR_GEOMETRY</span></a>):</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Axis</p> <th class=head ><p>Type</p> <th class=head ><p>Units</p> <th class=head ><p>Depends on</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >translation</span></code></p> <td><p>translation</p> <td><p>m</p> <td><p><code class="docutils literal notranslate"><span class=pre >.</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >rot1</span></code></p> <td><p>rotation</p> <td><p>rad</p> <td><p><code class="docutils literal notranslate"><span class=pre >translation</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rot2</span></code></p> <td><p>rotation</p> <td><p>rad</p> <td><p><code class="docutils literal notranslate"><span class=pre >rot1</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >rot3</span></code></p> <td><p>rotation</p> <td><p>rad</p> <td><p><code class="docutils literal notranslate"><span class=pre >rot2</span></code></p> </table> </section> <section id=entry-instrument-detector-module-nxdetector-module > <h3 id=entry-instrument-detector-module-nxdetector-module ><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/module</span></code> (NXdetector_module)<a class=headerlink href="#entry-instrument-detector-module-nxdetector-module" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >data_origin</span></code>, <code class="docutils literal notranslate"><span class=pre >data_size</span></code>, <code class="docutils literal notranslate"><span class=pre >fast_pixel_direction</span></code>, <code class="docutils literal notranslate"><span class=pre >slow_pixel_direction</span></code>, <code class="docutils literal notranslate"><span class=pre >module_offset</span></code> — all NXmx (<code class="docutils literal notranslate"><span class=pre >fast/slow_pixel_direction</span></code> and <code class="docutils literal notranslate"><span class=pre >module_offset</span></code> carry transformation attributes).</p> </section> <section id=entry-sample-nxsample > <h3 id=entry-sample-nxsample ><code class="docutils literal notranslate"><span class=pre >/entry/sample</span></code> (NXsample)<a class=headerlink href="#entry-sample-nxsample" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class="head text-center"><p>Std</p> <th class=head ><p>Units / notes</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >name</span></code></p> <td class=text-center ><p>NXmx</p> <td><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >depends_on</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>points at the last goniometer / grid-scan axis, or <code class="docutils literal notranslate"><span class=pre >.</span></code> for stills</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >temperature</span></code></p> <td class=text-center ><p>NXmx</p> <td><p>K</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >transformations/</span></code> (NXtransformations)</p> <td class=text-center ><p>NXmx</p> <td><p>rotation axis (e.g. <code class="docutils literal notranslate"><span class=pre >omega</span></code>) or grid-scan translation; hard-linked as <code class="docutils literal notranslate"><span class=pre >/entry/sample/goniometer</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >unit_cell</span></code></p> <td class=text-center ><p>base</p> <td><p><code class="docutils literal notranslate"><span class=pre >[a,</span> <span class=pre >b,</span> <span class=pre >c,</span> <span class=pre >α,</span> <span class=pre >β,</span> <span class=pre >γ]</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >ub_matrix</span></code></p> <td class=text-center ><p>base</p> <td><p><code class="docutils literal notranslate"><span class=pre >[1,</span> <span class=pre >3,</span> <span class=pre >3]</span></code>, Angstrom⁻¹</p> </table> <p>For a rotation scan the goniometer axis is written as a per-image angle array <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;</span></code> plus <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_end</span></code>, scalar <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_range_average</span></code>, <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_range_total</span></code>, and for helical scans <code class="docutils literal notranslate"><span class=pre >&lt;axis&gt;_helical_x/_y/_z</span></code>. These extra goniometer datasets beyond the bare axis array are Jungfraujoch conveniences.</p> </section> <section id=entry-data-nxdata > <h3 id=entry-data-nxdata ><code class="docutils literal notranslate"><span class=pre >/entry/data</span></code> (NXdata)<a class=headerlink href="#entry-data-nxdata" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >data</span></code> (3-D image stack, <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >y,</span> <span class=pre >x]</span></code>) with <code class="docutils literal notranslate"><span class=pre >image_nr_low</span></code> / <code class="docutils literal notranslate"><span class=pre >image_nr_high</span></code> attributes. In legacy mode this group instead contains one external link <code class="docutils literal notranslate"><span class=pre >data_000001</span></code>, … per data file.</p> </section> </section> <section id=extensions-beyond-nxmx > <h2 id=extensions-beyond-nxmx >4. Extensions beyond NXmx<a class=headerlink href="#extensions-beyond-nxmx" title="Link to this heading"></a></h2> <p>Everything in this section is <strong>outside the NXmx standard</strong>. Each group is declared with <code class="docutils literal notranslate"><span class=pre >NX_class</span> <span class=pre >=</span> <span class=pre >NXcollection</span></code> (the NeXus-sanctioned container for non-standardised content) unless noted. The per-image arrays are indexed by image number, padded to the run length and filled with a sentinel (<code class="docutils literal notranslate"><span class=pre >NaN</span></code> for floats, <code class="docutils literal notranslate"><span class=pre >-1</span></code>/<code class="docutils literal notranslate"><span class=pre >0</span></code> for integer indices) where a quantity is absent.</p> <section id=entry-mx-spot-finding-and-indexing-cxi-style > <h3 id=entry-mx-spot-finding-and-indexing-cxi-style >4.1 <code class="docutils literal notranslate"><span class=pre >/entry/MX</span></code> — spot finding and indexing (CXI-style)<a class=headerlink href="#entry-mx-spot-finding-and-indexing-cxi-style" title="Link to this heading"></a></h3> <p>The flagship extension. Spot (“peak”) properties are stored as fixed-size <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >max_spots]</span></code> arrays (CXI layout, recognised by CrystFEL); scalar-per-image quantities as <code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors. In legacy/VDS mode these live in the data files and are linked/virtual-stacked into the master.</p> <p><strong>Per-spot arrays <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >max_spots]</span></code>:</strong></p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <th class="head text-center"><p>Indexing only</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakXPosRaw</span></code>, <code class="docutils literal notranslate"><span class=pre >peakYPosRaw</span></code></p> <td><p>pixel</p> <td><p>spot position (raw detector frame)</p> <td class=text-center ><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakTotalIntensity</span></code></p> <td><p>photons</p> <td><p>spot intensity</p> <td class=text-center ><p></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakIceRingRes</span></code></p> <td><p></p> <td><p>spot lies in an ice-ring resolution band</p> <td class=text-center ><p></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakH</span></code>, <code class="docutils literal notranslate"><span class=pre >peakK</span></code>, <code class="docutils literal notranslate"><span class=pre >peakL</span></code></p> <td><p></p> <td><p>Miller indices of the (indexed) spot</p> <td class=text-center ><p>✓</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakDistEwaldSphere</span></code></p> <td><p>Å⁻¹</p> <td><p>distance of the spot from the Ewald sphere</p> <td class=text-center ><p>✓</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakIndexed</span></code></p> <td><p></p> <td><p>spot fits the indexing solution</p> <td class=text-center ><p>✓</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakLattice</span></code></p> <td><p></p> <td><p>lattice the spot belongs to (<code class="docutils literal notranslate"><span class=pre >-1</span></code> = unindexed)</p> <td class=text-center ><p>✓</p> </table> <p><strong>Per-image vectors <code class="docutils literal notranslate"><span class=pre >[n_images]</span></code>:</strong></p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >nPeaks</span></code></p> <td><p></p> <td><p>number of spots stored for the image (CXI)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >strongPixels</span></code></p> <td><p></p> <td><p>strong-pixel count (first spot-finding stage)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountUnfiltered</span></code></p> <td><p></p> <td><p>spots found before filtering</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountLowRes</span></code></p> <td><p></p> <td><p>low-resolution spots</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountIceRingRes</span></code></p> <td><p></p> <td><p>spots inside ice-ring bands</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >peakCountIndexed</span></code></p> <td><p></p> <td><p>spots fitting the indexing solution</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >imageIndexed</span></code></p> <td><p></p> <td><p>image was indexed (0/1)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >indexingLatticeCount</span></code></p> <td><p></p> <td><p>number of lattices found for the image</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >niggliClass</span></code></p> <td><p></p> <td><p>Niggli class of the indexed Bravais lattice (see <em>International Tables for Crystallography A</em> (2016), Vol. A, <a class="reference external" href="https://onlinelibrary.wiley.com/iucr/itc/Ac/ch3o1v0001/table3o1o3o1.pdf">Table 3.1.3.1</a>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >bravaisLattice</span></code></p> <td><p></p> <td><p>Bravais lattice short code, e.g. <code class="docutils literal notranslate"><span class=pre >aP</span></code>, <code class="docutils literal notranslate"><span class=pre >mC</span></code>, <code class="docutils literal notranslate"><span class=pre >oF</span></code>, <code class="docutils literal notranslate"><span class=pre >tI</span></code>, <code class="docutils literal notranslate"><span class=pre >hP</span></code>, <code class="docutils literal notranslate"><span class=pre >hR</span></code>, <code class="docutils literal notranslate"><span class=pre >cF</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >profileRadius</span></code></p> <td><p>Å⁻¹</p> <td><p>crystal profile radius</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >mosaicity</span></code></p> <td><p>deg</p> <td><p>mosaicity estimate</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bFactor</span></code></p> <td><p>Ų</p> <td><p>per-image B-factor estimate</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >resolutionEstimate</span></code></p> <td><p>Å</p> <td><p>diffraction resolution estimate</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >integratedReflections</span></code></p> <td><p></p> <td><p>number of integrated reflections</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >bkgEstimate</span></code></p> <td><p>photons</p> <td><p>mean background in the 3–5 Å resolution band</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >beam_corr_x</span></code>, <code class="docutils literal notranslate"><span class=pre >beam_corr_y</span></code></p> <td><p>pixel</p> <td><p>beam-center correction applied during processing</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >imageScaleFactor</span></code></p> <td><p></p> <td><p>on-the-fly per-image scale factor <em>g</em></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >imageScaleCC</span></code></p> <td><p></p> <td><p>on-the-fly scaling correlation coefficient</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >imageScaleMosaicity</span></code></p> <td><p>deg</p> <td><p>scaling-model mosaicity</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >imageScaleBFactor</span></code></p> <td><p>Ų</p> <td><p>scaling-model B-factor</p> </table> <p><strong>Per-image lattices:</strong> <code class="docutils literal notranslate"><span class=pre >latticeIndexed</span></code> <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >9]</span></code> (Å) — the real-space lattice (flattened 3×3); <code class="docutils literal notranslate"><span class=pre >latticeIndexedExtra</span></code> <code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >max_extra_lattices,</span> <span class=pre >9]</span></code> (Å) — additional orientation variants.</p> <p><strong>Run-level summaries</strong> (written into the master <code class="docutils literal notranslate"><span class=pre >/entry/MX</span></code> at finalisation):</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >indexing_algorithm</span></code></p> <td><p></p> <td><p><code class="docutils literal notranslate"><span class=pre >FFBIDX</span></code> / <code class="docutils literal notranslate"><span class=pre >FFT</span> <span class=pre >(CUDA)</span></code> / <code class="docutils literal notranslate"><span class=pre >FFT</span> <span class=pre >(FFTW)</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >geom_refinement_algorithm</span></code></p> <td><p></p> <td><p>e.g. <code class="docutils literal notranslate"><span class=pre >beam_center</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rotationLatticeIndexed</span></code></p> <td><p>Å</p> <td><p>whole-run rotation-indexing lattice (<code class="docutils literal notranslate"><span class=pre >[9]</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >rotationLatticeIndexedExtra</span></code></p> <td><p>Å</p> <td><p>additional whole-run lattices (<code class="docutils literal notranslate"><span class=pre >[m,</span> <span class=pre >9]</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rotationLatticeNiggliClass</span></code></p> <td><p></p> <td><p>Niggli class of the run lattice</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >imageIndexedMean</span></code></p> <td><p></p> <td><p>mean indexing rate over the run</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bkgEstimateMean</span></code></p> <td><p>photons</p> <td><p>mean background over the run</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >indexedLatticeCount</span></code></p> <td><p></p> <td><p>per-image lattice count summary (master). <em>Note: data files use <code class="docutils literal notranslate"><span class=pre >indexingLatticeCount</span></code>; readers accept either.</em></p> </table> <p>CrystFEL can read the spots directly with:</p> <div class="highlight-default notranslate"><div class=highlight ><pre><span></span><span class=n >peak_list</span> <span class=o >=</span> <span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >MX</span>
<span class=n >peak_list_type</span> <span class=o >=</span> <span class=n >cxi</span>
</pre></div> </div> </section> <section id=entry-reflections-integrated-reflections-nxreflections > <h3 id=entry-reflections-integrated-reflections-nxreflections >4.2 <code class="docutils literal notranslate"><span class=pre >/entry/reflections</span></code> — integrated reflections (NXreflections)<a class=headerlink href="#entry-reflections-integrated-reflections-nxreflections" title="Link to this heading"></a></h3> <p>Integrated reflections are stored <strong>per image</strong> as <code class="docutils literal notranslate"><span class=pre >/entry/reflections/image_NNNNNN</span></code> groups, each declared <code class="docutils literal notranslate"><span class=pre >NX_class</span> <span class=pre >=</span> <span class=pre >NXreflections</span></code>. The columns map mostly onto the standard <a class="reference external" href="https://manual.nexusformat.org/classes/base_classes/NXreflections.html">NXreflections</a> base class:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Units</p> <th class="head text-center"><p>NXreflections</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >h</span></code>, <code class="docutils literal notranslate"><span class=pre >k</span></code>, <code class="docutils literal notranslate"><span class=pre >l</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>Miller indices</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >d</span></code></p> <td><p>Å</p> <td class=text-center ><p>standard</p> <td><p>resolution</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >int_sum</span></code></p> <td><p>photons</p> <td class=text-center ><p>standard</p> <td><p>integrated intensity (summation)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >int_err</span></code></p> <td><p>photons</p> <td class=text-center ><p>non-standard name</p> <td><p>σ of the intensity (standard equivalent: <code class="docutils literal notranslate"><span class=pre >int_sum_errors</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >background_mean</span></code></p> <td><p>photons</p> <td class=text-center ><p>standard</p> <td><p>mean background under the peak</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >predicted_x</span></code>, <code class="docutils literal notranslate"><span class=pre >predicted_y</span></code></p> <td><p>pixel</p> <td class=text-center ><p>name standard, units differ</p> <td><p>predicted position. NXreflections <code class="docutils literal notranslate"><span class=pre >predicted_x/_y</span></code> are <em>physical</em> lengths; the pixel datasets are <code class="docutils literal notranslate"><span class=pre >predicted_px_x/_y</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >observed_x</span></code>, <code class="docutils literal notranslate"><span class=pre >observed_y</span></code></p> <td><p>pixel</p> <td class=text-center ><p>name standard, units differ</p> <td><p>observed centroid (pixels; standard pixel form is <code class="docutils literal notranslate"><span class=pre >observed_px_x/_y</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >observed_frame</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>image number of the reflection</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >lp</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>Lorentz–polarization factor (stored as <code class="docutils literal notranslate"><span class=pre >1/rlp</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >partiality</span></code></p> <td><p></p> <td class=text-center ><p>standard</p> <td><p>recorded fraction of the reflection</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >delta_phi</span></code></p> <td><p>deg</p> <td class=text-center ><p><strong>extension</strong></p> <td><p>XDS Δφ: offset from the centre of the current frame</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >zeta</span></code></p> <td><p></p> <td class=text-center ><p><strong>extension</strong></p> <td><p>Lorentz ζ factor (reciprocal-space geometry term)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >image_scale_corr</span></code></p> <td><p></p> <td class=text-center ><p><strong>extension</strong></p> <td><p>per-image scale correction; <code class="docutils literal notranslate"><span class=pre >I_true</span> <span class=pre >=</span> <span class=pre >image_scale_corr</span> <span class=pre >·</span> <span class=pre >int_sum</span></code></p> </table> <p>In the master file these per-image groups are exposed through <code class="docutils literal notranslate"><span class=pre >/entry/reflections</span></code> external links (VDS/integrated formats).</p> </section> <section id=entry-azint-azimuthal-integration > <h3 id=entry-azint-azimuthal-integration >4.3 <code class="docutils literal notranslate"><span class=pre >/entry/azint</span></code> — azimuthal integration<a class=headerlink href="#entry-azint-azimuthal-integration" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Shape</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bin_to_q</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p>Å⁻¹</p> <td><p>q value of each bin</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >bin_to_two_theta</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p>deg</p> <td><p>2θ of each bin</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bin_to_phi</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p>deg</p> <td><p>azimuthal angle of each bin</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p></p> <td><p>per-image integrated profile (NaN for empty bins)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >image_std</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p></p> <td><p>per-bin standard deviation</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image_count</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[n_images,</span> <span class=pre >φ_bins,</span> <span class=pre >q_bins]</span></code></p> <td><p></p> <td><p>pixels contributing per bin</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >map</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >[y,</span> <span class=pre >x]</span></code></p> <td><p></p> <td><p>pixel→bin mapping (master file only)</p> </table> </section> <section id=entry-roi-regions-of-interest-per-image-results > <h3 id=entry-roi-regions-of-interest-per-image-results >4.4 <code class="docutils literal notranslate"><span class=pre >/entry/roi</span></code> — regions of interest (per-image results)<a class=headerlink href="#entry-roi-regions-of-interest-per-image-results" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >/entry/roi/&lt;roi_name&gt;</span></code> has one sub-group per configured ROI, holding the <strong>per-image result vectors</strong> <code class="docutils literal notranslate"><span class=pre >[n_images]</span></code>. These are written into the data files; in VDS mode they are exposed from the master file through virtual datasets, and in integrated mode they are in the single file. (In legacy mode they remain only in the data files.)</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >max</span></code></p> <td><p>maximum pixel value in the ROI</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >sum</span></code></p> <td><p>sum of pixel values</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >sum_sq</span></code></p> <td><p>sum of squared pixel values</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >npixel</span></code></p> <td><p>number of valid pixels</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >x</span></code>, <code class="docutils literal notranslate"><span class=pre >y</span></code></p> <td><p>intensity-weighted centroid</p> </table> </section> <section id=entry-roi-defs-roi-definitions-master-file > <h3 id=entry-roi-defs-roi-definitions-master-file >4.4.1 <code class="docutils literal notranslate"><span class=pre >/entry/roi_defs</span></code> — ROI definitions (master file)<a class=headerlink href="#entry-roi-defs-roi-definitions-master-file" title="Link to this heading"></a></h3> <p>The <strong>dataset-wide ROI definitions</strong> (geometry, fixed for the whole acquisition) live in the master file under a <em>separate</em> <code class="docutils literal notranslate"><span class=pre >/entry/roi_defs</span></code> group — kept apart from <code class="docutils literal notranslate"><span class=pre >/entry/roi</span></code> above so that older readers, which iterate <code class="docutils literal notranslate"><span class=pre >/entry/roi</span></code>, are unaffected by these entries. One sub-group <code class="docutils literal notranslate"><span class=pre >/entry/roi_defs/&lt;roi_name&gt;</span></code> per ROI:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >bit_index</span></code></p> <td><p>which bit of <code class="docutils literal notranslate"><span class=pre >roi_map</span></code> (below) marks this ROI</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >type</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >box</span></code>, <code class="docutils literal notranslate"><span class=pre >circle</span></code> or <code class="docutils literal notranslate"><span class=pre >azim</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >min_x_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >max_x_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >min_y_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >max_y_pxl</span></code></p> <td><p>box bounds (type <code class="docutils literal notranslate"><span class=pre >box</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >center_x_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >center_y_pxl</span></code>, <code class="docutils literal notranslate"><span class=pre >radius_pxl</span></code></p> <td><p>circle (type <code class="docutils literal notranslate"><span class=pre >circle</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >q_min_recipA</span></code>, <code class="docutils literal notranslate"><span class=pre >q_max_recipA</span></code></p> <td><p>Q range (type <code class="docutils literal notranslate"><span class=pre >azim</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >phi_min_deg</span></code>, <code class="docutils literal notranslate"><span class=pre >phi_max_deg</span></code></p> <td><p>azimuthal-angle sector (type <code class="docutils literal notranslate"><span class=pre >azim</span></code>, omitted for a full ring)</p> </table> <p><code class="docutils literal notranslate"><span class=pre >/entry/roi_defs/roi_map</span></code> <code class="docutils literal notranslate"><span class=pre >[y,</span> <span class=pre >x]</span></code> is a <code class="docutils literal notranslate"><span class=pre >uint16</span></code> per-pixel bitmask: bit <code class="docutils literal notranslate"><span class=pre >bit_index</span></code> is set for every pixel belonging to that ROI, so an ROI’s footprint can be recovered exactly.</p> </section> <section id=entry-image-per-image-pixel-statistics > <h3 id=entry-image-per-image-pixel-statistics >4.5 <code class="docutils literal notranslate"><span class=pre >/entry/image</span></code> — per-image pixel statistics<a class=headerlink href="#entry-image-per-image-pixel-statistics" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors: <code class="docutils literal notranslate"><span class=pre >max_value</span></code>, <code class="docutils literal notranslate"><span class=pre >min_value</span></code> (viable min/max, excluding error/saturated pixels), <code class="docutils literal notranslate"><span class=pre >error_pixels</span></code>, <code class="docutils literal notranslate"><span class=pre >saturated_pixels</span></code>, <code class="docutils literal notranslate"><span class=pre >pixel_sum</span></code>. Surfaced in the master file under <code class="docutils literal notranslate"><span class=pre >/entry/image</span></code>.</p> </section> <section id=entry-profiling-per-image-timing > <h3 id=entry-profiling-per-image-timing >4.6 <code class="docutils literal notranslate"><span class=pre >/entry/profiling</span></code> — per-image timing<a class=headerlink href="#entry-profiling-per-image-timing" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors in seconds: <code class="docutils literal notranslate"><span class=pre >spotFindingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >indexingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >integrationTime</span></code>, <code class="docutils literal notranslate"><span class=pre >refinementTime</span></code>, <code class="docutils literal notranslate"><span class=pre >processingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >braggPredictionTime</span></code>, <code class="docutils literal notranslate"><span class=pre >preprocessingTime</span></code>, <code class="docutils literal notranslate"><span class=pre >compressionTime</span></code>, <code class="docutils literal notranslate"><span class=pre >azIntTime</span></code>, <code class="docutils literal notranslate"><span class=pre >indexAnalysisTime</span></code>, <code class="docutils literal notranslate"><span class=pre >imageScaleTime</span></code>.</p> </section> <section id=entry-detector-acquisition-diagnostics-data-file > <h3 id=entry-detector-acquisition-diagnostics-data-file >4.7 <code class="docutils literal notranslate"><span class=pre >/entry/detector</span></code> — acquisition diagnostics (data file)<a class=headerlink href="#entry-detector-acquisition-diagnostics-data-file" title="Link to this heading"></a></h3> <p>A convenience NXcollection in the data file (note: distinct from the standard <code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector</span></code>). In <strong>integrated</strong> format these datasets are written under <code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/detectorSpecific</span></code> instead.</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Dataset</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >timestamp</span></code>, <code class="docutils literal notranslate"><span class=pre >exptime</span></code></p> <td><p>per-image timestamp and exposure time</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >number</span></code></p> <td><p>image number (original number if image rejection was used)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >det_info</span></code></p> <td><p>JUNGFRAU debug field</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >storage_cell_image</span></code></p> <td><p>storage-cell number</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >rcv_delay</span></code>, <code class="docutils literal notranslate"><span class=pre >rcv_free_send_buffers</span></code></p> <td><p>receiver internal diagnostics</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >packets_expected</span></code>, <code class="docutils literal notranslate"><span class=pre >packets_received</span></code></p> <td><p>UDP packets per image</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >data_collection_efficiency_image</span></code></p> <td><p>received / expected packet ratio</p> </table> </section> <section id=entry-xfel-pulsed-source-metadata > <h3 id=entry-xfel-pulsed-source-metadata >4.8 <code class="docutils literal notranslate"><span class=pre >/entry/xfel</span></code> — pulsed-source metadata<a class=headerlink href="#entry-xfel-pulsed-source-metadata" title="Link to this heading"></a></h3> <p><code class="docutils literal notranslate"><span class=pre >[n_images]</span></code> vectors <code class="docutils literal notranslate"><span class=pre >pulseID</span></code> and <code class="docutils literal notranslate"><span class=pre >eventCode</span></code>, written for pulsed sources (e.g. SwissFEL).</p> </section> <section id=other-collections > <h3 id=other-collections >4.9 Other collections<a class=headerlink href="#other-collections" title="Link to this heading"></a></h3> <table> <thead> <tr class=row-odd ><th class=head ><p>Path</p> <th class=head ><p>Class</p> <th class=head ><p>Content</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/detectorSpecific</span></code></p> <td><p>NXcollection</p> <td><p>Dectris-style detector metadata + Jungfraujoch fields: <code class="docutils literal notranslate"><span class=pre >x_pixels_in_detector</span></code>, <code class="docutils literal notranslate"><span class=pre >y_pixels_in_detector</span></code>, <code class="docutils literal notranslate"><span class=pre >nimages</span></code>, <code class="docutils literal notranslate"><span class=pre >ntrigger</span></code>, <code class="docutils literal notranslate"><span class=pre >nimages_collected</span></code>, <code class="docutils literal notranslate"><span class=pre >nimages_written</span></code>, <code class="docutils literal notranslate"><span class=pre >data_collection_efficiency</span></code>, <code class="docutils literal notranslate"><span class=pre >max_receiver_delay</span></code>, <code class="docutils literal notranslate"><span class=pre >storage_cell_number</span></code>, <code class="docutils literal notranslate"><span class=pre >storage_cell_delay</span></code> [ns], <code class="docutils literal notranslate"><span class=pre >software_git_commit</span></code>, <code class="docutils literal notranslate"><span class=pre >software_git_date</span></code>, <code class="docutils literal notranslate"><span class=pre >jfjoch_release</span></code>, <code class="docutils literal notranslate"><span class=pre >jfjoch_writer_release</span></code>, <code class="docutils literal notranslate"><span class=pre >summation_mode</span></code>, <code class="docutils literal notranslate"><span class=pre >detect_ice_rings</span></code>, <code class="docutils literal notranslate"><span class=pre >gain_file_names</span></code>, <code class="docutils literal notranslate"><span class=pre >data_reduction_factor_serialmx</span></code>, <code class="docutils literal notranslate"><span class=pre >adu_histogram/</span></code>, <code class="docutils literal notranslate"><span class=pre >data_collection_efficiency_image</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector/calibration</span></code></p> <td><p>NXcollection</p> <td><p>per-channel pedestal / calibration images (bitshuffle-compressed)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/instrument/fluorescence</span></code></p> <td><p>NXcollection</p> <td><p>XRF spectrum: <code class="docutils literal notranslate"><span class=pre >energy</span></code> [eV], <code class="docutils literal notranslate"><span class=pre >data</span></code></p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >/entry/user</span></code></p> <td><p>NXcollection</p> <td><p>scalar values supplied under <code class="docutils literal notranslate"><span class=pre >header_appendix.hdf5</span></code></p> </table> </section> <section id=non-standard-fields-inside-the-nxmx-detector-group > <h3 id=non-standard-fields-inside-the-nxmx-detector-group >4.10 Non-standard fields inside the NXmx detector group<a class=headerlink href="#non-standard-fields-inside-the-nxmx-detector-group" title="Link to this heading"></a></h3> <p>A few extension scalars are written <em>inside</em> the otherwise-standard <code class="docutils literal notranslate"><span class=pre >/entry/instrument/detector</span></code> group for compatibility with existing tooling:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class=head ><p>Units</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >detector_distance</span></code></p> <td><p>m</p> <td><p>duplicate of <code class="docutils literal notranslate"><span class=pre >distance</span></code> (Dectris/Neggia compatibility)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >detector_number</span></code></p> <td><p></p> <td><p>detector identifier (Dectris convention)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >error_value</span></code></p> <td><p></p> <td><p>masked/error pixel sentinel (NXmx standard would be <code class="docutils literal notranslate"><span class=pre >underload_value</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >bit_depth_image</span></code></p> <td><p></p> <td><p>stored image bit depth (NXmx standard is <code class="docutils literal notranslate"><span class=pre >bit_depth_readout</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >acquisition_type</span></code></p> <td><p></p> <td><p>always <code class="docutils literal notranslate"><span class=pre >triggered</span></code> (Dectris convention)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >jungfrau_conversion_applied</span></code></p> <td><p></p> <td><p>JUNGFRAU photon/keV conversion applied</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >jungfrau_conversion_factor</span></code></p> <td><p>eV</p> <td><p>conversion factor</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >geometry_transformation_applied</span></code></p> <td><p></p> <td><p>module→full-detector geometry applied</p> </table> </section> <section id=user-supplied-metadata-header-appendix-and-image-appendix > <h3 id=user-supplied-metadata-header-appendix-and-image-appendix >4.11 User-supplied metadata: <code class="docutils literal notranslate"><span class=pre >header_appendix</span></code> and <code class="docutils literal notranslate"><span class=pre >image_appendix</span></code><a class=headerlink href="#user-supplied-metadata-header-appendix-and-image-appendix" title="Link to this heading"></a></h3> <p>Facilities frequently need to attach metadata that Jungfraujoch does not model explicitly. Two free-form JSON fields in the <code class="docutils literal notranslate"><span class=pre >/start</span></code> request (<code class="docutils literal notranslate"><span class=pre >broker/jfjoch_api.yaml</span></code>) provide this without any schema change; both accept <em>any valid JSON</em>:</p> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class=head ><p>Carried in</p> <th class=head ><p>Persisted to HDF5?</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >header_appendix</span></code></p> <td><p>the <strong>start</strong> message, under <code class="docutils literal notranslate"><span class=pre >user_data.user</span></code> (see <a class="reference internal" href=CBOR.html ><span class="std std-doc">CBOR</span></a>)</p> <td><p>no — except the <code class="docutils literal notranslate"><span class=pre >hdf5</span></code> sub-object (below)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image_appendix</span></code></p> <td><p><strong>every image</strong> message, as <code class="docutils literal notranslate"><span class=pre >user_data</span></code></p> <td><p>no</p> </table> <p>Both are forwarded verbatim through the ZeroMQ/CBOR stream to every downstream consumer (writer, republished analysis, viewers), so they are the recommended channel for facility- or beamline-specific provenance (proposal, operator, optics state, per-image trigger info, …) that has no dedicated API field.</p> <p><strong>Persisting selected values to HDF5.</strong> <code class="docutils literal notranslate"><span class=pre >header_appendix</span></code> is normally <em>not</em> written to the master file. As an exception, if it contains a key <code class="docutils literal notranslate"><span class=pre >hdf5</span></code> whose value is a JSON object of scalars (strings and numbers — no arrays or nested objects), the writer stores each entry under <code class="docutils literal notranslate"><span class=pre >/entry/user/&lt;key&gt;</span></code>.</p> <p>For example, a <code class="docutils literal notranslate"><span class=pre >/start</span></code> request containing:</p> <div class="highlight-json notranslate"><div class=highlight ><pre><span></span><span class=p >{</span>
<span class=w > </span><span class=nt >"header_appendix"</span><span class=p >:</span><span class=w > </span><span class=p >{</span>
<span class=w > </span><span class=nt >"proposal"</span><span class=p >:</span><span class=w > </span><span class=s2 >"p20001"</span><span class=p >,</span>
<span class=w > </span><span class=nt >"operator"</span><span class=p >:</span><span class=w > </span><span class=s2 >"jdoe"</span><span class=p >,</span>
<span class=w > </span><span class=nt >"hdf5"</span><span class=p >:</span><span class=w > </span><span class=p >{</span><span class=w > </span><span class=nt >"beamline"</span><span class=p >:</span><span class=w > </span><span class=s2 >"X06SA"</span><span class=p >,</span><span class=w > </span><span class=nt >"ring_mode"</span><span class=p >:</span><span class=w > </span><span class=s2 >"top-up"</span><span class=p >,</span><span class=w > </span><span class=nt >"attenuator_foils"</span><span class=p >:</span><span class=w > </span><span class=mi >2</span><span class=w > </span><span class=p >}</span>
<span class=w > </span><span class=p >},</span>
<span class=w > </span><span class=nt >"image_appendix"</span><span class=p >:</span><span class=w > </span><span class=p >{</span><span class=w > </span><span class=nt >"trigger_source"</span><span class=p >:</span><span class=w > </span><span class=s2 >"external"</span><span class=w > </span><span class=p >}</span>
<span class=p >}</span>
</pre></div> </div> <p>forwards the whole <code class="docutils literal notranslate"><span class=pre >header_appendix</span></code> as <code class="docutils literal notranslate"><span class=pre >user_data.user</span></code> on the start message and <code class="docutils literal notranslate"><span class=pre >{"trigger_source":</span> <span class=pre >"external"}</span></code> as <code class="docutils literal notranslate"><span class=pre >user_data</span></code> on every image message, and writes three scalars into the master file:</p> <div class="highlight-default notranslate"><div class=highlight ><pre><span></span><span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >user</span><span class=o >/</span><span class=n >beamline</span> <span class=o >=</span> <span class=s2 >"X06SA"</span>
<span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >user</span><span class=o >/</span><span class=n >ring_mode</span> <span class=o >=</span> <span class=s2 >"top-up"</span>
<span class=o >/</span><span class=n >entry</span><span class=o >/</span><span class=n >user</span><span class=o >/</span><span class=n >attenuator_foils</span> <span class=o >=</span> <span class=mi >2</span>
</pre></div> </div> </section> </section> <section id=notes > <h2 id=notes >5. Notes<a class=headerlink href="#notes" title="Link to this heading"></a></h2> <ul class=simple > <li><p><strong>Units</strong> are written as the HDF5 <code class="docutils literal notranslate"><span class=pre >units</span></code> attribute on the dataset (e.g. <code class="docutils literal notranslate"><span class=pre >m</span></code>, <code class="docutils literal notranslate"><span class=pre >eV</span></code>, <code class="docutils literal notranslate"><span class=pre >deg</span></code>, <code class="docutils literal notranslate"><span class=pre >Angstrom</span></code>, <code class="docutils literal notranslate"><span class=pre >Angstrom^-1</span></code>, <code class="docutils literal notranslate"><span class=pre >Angstrom^2</span></code>, <code class="docutils literal notranslate"><span class=pre >pixel</span></code>, <code class="docutils literal notranslate"><span class=pre >s</span></code>).</p> <li><p><strong>Sentinels.</strong> Missing per-image values are <code class="docutils literal notranslate"><span class=pre >NaN</span></code> (floats) or <code class="docutils literal notranslate"><span class=pre >-1</span></code>/<code class="docutils literal notranslate"><span class=pre >0</span></code> (integer indices); image pixels use <code class="docutils literal notranslate"><span class=pre >INTx_MIN</span></code> / <code class="docutils literal notranslate"><span class=pre >UINTx_MAX</span></code>.</p> <li><p><strong>Master vs data file.</strong> In legacy/VDS formats the analysis arrays physically live in the data files; the master file links to them (external links in legacy, virtual datasets in VDS). In the integrated format there are no data files and everything is in one place.</p> <li><p><strong>CXI / CrystFEL.</strong> <code class="docutils literal notranslate"><span class=pre >/entry/MX</span></code> follows the CXI peak-list convention; see <a class="reference external" href="https://raw.githubusercontent.com/cxidb/CXI/master/cxi_file_format.pdf">CXI file format</a>.</p> </ul> </section> </section> </article> </div> </div> </main> </div> <footer class=md-footer > <div class=md-footer-nav > <nav class="md-footer-nav__inner md-grid"> <a href=CBOR.html title="CBOR messages" class="md-flex md-footer-nav__link md-footer-nav__link--prev" rel=prev > <div class="md-flex__cell md-flex__cell--shrink"> <i class="md-icon md-icon--arrow-back md-footer-nav__button"></i> </div> <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"> <span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Previous" </span> CBOR messages </span> </div> </a> <a href=IMAGE_STREAM.html title="Data streams" class="md-flex md-footer-nav__link md-footer-nav__link--next" rel=next > <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"><span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Next" </span> Data streams </span> </div> <div class="md-flex__cell md-flex__cell--shrink"><i class="md-icon md-icon--arrow-forward md-footer-nav__button"></i> </div> </a> </nav> </div> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-footer-copyright > <div class=md-footer-copyright__highlight > &#169; Copyright 2024, Paul Scherrer Institute. </div> Created using <a href="http://www.sphinx-doc.org/">Sphinx</a> 8.1.3. and <a href="https://github.com/bashtage/sphinx-material/">Material for Sphinx</a> </div> </div> </div> </footer> <script src="_static/javascripts/application.js"></script> <script>app.initialize({version: "1.0.4", url: {base: ".."}})</script>
Reference in New Issue View Git Blame Copy Permalink