mx/Jungfraujoch
1
1
Fork 0
Code Issues Pull Requests 1 Actions 4 Packages Projects Releases 52 Wiki Activity
Files
5fdda01ff45574181cd4997370adabb7a2c9a8c4
Jungfraujoch/IMAGE_STREAM.html
T
leonarski_f 5fdda01ff4 Deploy site
2026-05-01 16:49:38 +00:00

30 lines
56 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html> <html lang=en data-content_root="./"> <meta charset=utf-8 /> <meta name=viewport content="width=device-width, initial-scale=1.0" /><meta name=viewport content="width=device-width, initial-scale=1" /> <meta name=viewport content="width=device-width,initial-scale=1"> <meta http-equiv=x-ua-compatible content="ie=edge"> <meta name="lang:clipboard.copy" content="Copy to clipboard"> <meta name="lang:clipboard.copied" content="Copied to clipboard"> <meta name="lang:search.language" content=en > <meta name="lang:search.pipeline.stopwords" content=True > <meta name="lang:search.pipeline.trimmer" content=True > <meta name="lang:search.result.none" content="No matching documents"> <meta name="lang:search.result.one" content="1 matching document"> <meta name="lang:search.result.other" content="# matching documents"> <meta name="lang:search.tokenizer" content="[\s\-]+"> <link href="https://fonts.gstatic.com/" rel=preconnect crossorigin> <link href="https://fonts.googleapis.com/css?family=Roboto+Mono:400,500,700|Roboto:300,400,400i,700&display=fallback" rel=stylesheet > <style> body, input { font-family: "Roboto", "Helvetica Neue", Helvetica, Arial, sans-serif } code, kbd, pre { font-family: "Roboto Mono", "Courier New", Courier, monospace } </style> <link rel=stylesheet href="_static/stylesheets/application.css"/> <link rel=stylesheet href="_static/stylesheets/application-palette.css"/> <link rel=stylesheet href="_static/stylesheets/application-fixes.css"/> <link rel=stylesheet href="_static/fonts/material-icons.css"/> <meta name=theme-color content="#3f51b5"> <script src="_static/javascripts/modernizr.js"></script> <title>Data streams &#8212; Jungfraujoch 1.0.0-rc.144 documentation</title> <link rel=stylesheet type="text/css" href="_static/pygments.css?v=83e35b93" /> <link rel=stylesheet type="text/css" href="_static/material.css?v=79c92029" /> <script src="_static/documentation_options.js?v=909c8bec"></script> <script src="_static/doctools.js?v=9bcbadda"></script> <script src="_static/sphinx_highlight.js?v=dc90522c"></script> <link rel=icon href="_static/jfjoch.png"/> <link rel=index title=Index href=genindex.html /> <link rel=search title=Search href=search.html /> <link rel=next title="Pixel mask" href=PIXEL_MASK.html /> <link rel=prev title="CBOR messages" href=CBOR.html /> <body dir=ltr data-md-color-primary=indigo data-md-color-accent=lime> <svg class=md-svg > <defs data-children-count=0 > <svg xmlns="http://www.w3.org/2000/svg" width=500 height=500 viewBox="0 0 500 500" id=__gitlab ><path fill=currentColor d="M93.667 473.347l90.684-279.097H2.983l90.684 279.097z" transform="translate(156.198 1.16)"/><path fill=currentColor d="M221.333 473.345L130.649 194.25H3.557l217.776 279.095z" transform="translate(28.531 1.16)" opacity=.7 /><path fill=currentColor d="M32 195.155L4.441 279.97a18.773 18.773 0 0 0 6.821 20.99l238.514 173.29L32 195.155z" transform="translate(.089 .256)" opacity=.5 /><path fill=currentColor d="M2.667-84.844h127.092L75.14-252.942c-2.811-8.649-15.047-8.649-17.856 0L2.667-84.844z" transform="translate(29.422 280.256)"/><path fill=currentColor d="M2.667 473.345L93.351 194.25h127.092L2.667 473.345z" transform="translate(247.198 1.16)" opacity=.7 /><path fill=currentColor d="M221.334 195.155l27.559 84.815a18.772 18.772 0 0 1-6.821 20.99L3.557 474.25l217.777-279.095z" transform="translate(246.307 .256)" opacity=.5 /><path fill=currentColor d="M130.667-84.844H3.575l54.618-168.098c2.811-8.649 15.047-8.649 17.856 0l54.618 168.098z" transform="translate(336.974 280.256)"/></svg> </defs> </svg> <input class=md-toggle data-md-toggle=drawer type=checkbox id=__drawer > <input class=md-toggle data-md-toggle=search type=checkbox id=__search > <label class=md-overlay data-md-component=overlay for=__drawer ></label> <a href="#IMAGE_STREAM" tabindex=1 class=md-skip > Skip to content </a> <header class=md-header data-md-component=header > <nav class="md-header-nav md-grid"> <div class="md-flex navheader"> <div class="md-flex__cell md-flex__cell--shrink"> <a href=index.html title="Jungfraujoch 1.0.0-rc.144 documentation" class="md-header-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--menu md-header-nav__button" for=__drawer ></label> </div> <div class="md-flex__cell md-flex__cell--stretch"> <div class="md-flex__ellipsis md-header-nav__title" data-md-component=title > <span class=md-header-nav__topic >PSI Jungfraujoch</span> <span class=md-header-nav__topic > Data streams </span> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <label class="md-icon md-icon--search md-header-nav__button" for=__search ></label> <div class=md-search data-md-component=search role=dialog > <label class=md-search__overlay for=__search ></label> <div class=md-search__inner role=search > <form class=md-search__form action=search.html method=get name=search > <input type=text class=md-search__input name=q placeholder=""Search"" autocapitalize=off autocomplete=off spellcheck=false data-md-component=query data-md-state=active > <label class="md-icon md-search__icon" for=__search ></label> <button type=reset class="md-icon md-search__icon" data-md-component=reset tabindex=-1 > &#xE5CD; </button> </form> <div class=md-search__output > <div class=md-search__scrollwrap data-md-scrollfix> <div class=md-search-result data-md-component=result > <div class=md-search-result__meta > Type to start searching </div> <ol class=md-search-result__list ></ol> </div> </div> </div> </div> </div> </div> <div class="md-flex__cell md-flex__cell--shrink"> <div class=md-header-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> </div> <script src="_static/javascripts/version_dropdown.js"></script> <script> var json_loc = ""versions.json"", target_loc = "../", text = "Versions"; $( document ).ready( add_version_dropdown(json_loc, target_loc, text)); </script> </div> </nav> </header> <div class=md-container > <nav class=md-tabs data-md-component=tabs > <div class="md-tabs__inner md-grid"> <ul class=md-tabs__list > <li class=md-tabs__item ><a href=index.html class=md-tabs__link >Jungfraujoch 1.0.0-rc.144 documentation</a> </ul> </div> </nav> <main class=md-main > <div class="md-main__inner md-grid" data-md-component=container > <div class="md-sidebar md-sidebar--primary" data-md-component=navigation > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--primary" data-md-level=0 > <label class="md-nav__title md-nav__title--site" for=__drawer > <a href=index.html title="Jungfraujoch 1.0.0-rc.144 documentation" class="md-nav__button md-logo"> <i class=md-icon >&#xe30d</i> </a> <a href=index.html title="Jungfraujoch 1.0.0-rc.144 documentation">PSI Jungfraujoch</a> </label> <div class=md-nav__source > <a href="https://gitea.psi.ch/mx/jungfraujoch" title="Go to repository" class=md-source data-md-source=github > <div class=md-source__icon > <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 24 24" width=28 height=28 > <use xlink:href="#__gitlab" width=24 height=24 ></use> </svg> </div> <div class=md-source__repository > Jungfraujoch </div> </a> </div> <ul class=md-nav__list > <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >General</span></span> <li class=md-nav__item > <a href=ACKNOWLEDGEMENT.html class=md-nav__link >Acknowledgements</a> <li class=md-nav__item > <a href=LICENSE.html class=md-nav__link >License</a> <li class=md-nav__item > <a href=DETECTORS.html class=md-nav__link >Supported detectors</a> <li class=md-nav__item > <a href="DETECTORS.html#dectris-detectors" class=md-nav__link >DECTRIS detectors</a> <li class=md-nav__item > <a href=HARDWARE.html class=md-nav__link >Hardware requirements</a> <li class=md-nav__item > <a href=SOFTWARE.html class=md-nav__link >Software requirements</a> <li class=md-nav__item > <a href=VERSIONING.html class=md-nav__link >Semantic versioning</a> <li class=md-nav__item > <a href=DEPLOYMENT.html class=md-nav__link >Deployment</a> <li class=md-nav__item > <a href=CHANGELOG.html class=md-nav__link >Changelog</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Software</span></span> <li class=md-nav__item > <a href=JFJOCH_BROKER.html class=md-nav__link >jfjoch_broker</a> <li class=md-nav__item > <a href=JFJOCH_WRITER.html class=md-nav__link >jfjoch_writer</a> <li class=md-nav__item > <a href=SOFTWARE_INTEGRATION.html class=md-nav__link >Integration with MX data processing software</a> <li class=md-nav__item > <a href=TOOLS.html class=md-nav__link >Tools</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >FPGA</span></span> <li class=md-nav__item > <a href=FPGA.html class=md-nav__link >FPGA smartNIC</a> <li class=md-nav__item > <a href=FPGA_LICENSE.html class=md-nav__link >FPGA license</a> <li class=md-nav__item > <a href=FPGA_DESIGN.html class=md-nav__link >FPGA data flow</a> <li class=md-nav__item > <a href=FPGA_NETWORK.html class=md-nav__link >FPGA network</a> <li class=md-nav__item > <a href=FPGA_PCIE_DRIVER.html class=md-nav__link >FPGA PCIe driver</a> <li class=md-nav__item > <a href=FPGA_SETTINGS.html class=md-nav__link >FPGA advanced reference</a> <li class=md-nav__item > <a href=FPGA_DATA_ANALYSIS.html class=md-nav__link >FPGA data analysis</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >Reference</span></span> <li class=md-nav__item > <a href=DETECTOR_GEOMETRY.html class=md-nav__link >Detector geometry</a> <li class=md-nav__item > <a href=CPU_DATA_ANALYSIS.html class=md-nav__link >CPU-side crystallographic data analysis (Jungfraujoch)</a> <li class=md-nav__item > <a href=OPENAPI.html class=md-nav__link >OpenAPI</a> <li class=md-nav__item > <a href=OPENAPI_SPECS.html class=md-nav__link >OpenAPI specification</a> <li class=md-nav__item > <a href=CBOR.html class=md-nav__link >CBOR messages</a> <li class=md-nav__item > <input class="md-toggle md-nav__toggle" data-md-toggle=toc type=checkbox id=__toc > <label class="md-nav__link md-nav__link--active" for=__toc > Data streams </label> <a href="#" class="md-nav__link md-nav__link--active">Data streams</a> <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> <ul class=md-nav__list > <li class=md-nav__item > <a href="#image-stream" class=md-nav__link >Image stream</a> <li class=md-nav__item > <a href="#writer-notification-socket" class=md-nav__link >Writer notification socket</a> <li class=md-nav__item > <a href="#preview-stream" class=md-nav__link >Preview stream</a> <li class=md-nav__item > <a href="#metadata-stream" class=md-nav__link >Metadata stream</a> </ul> <li class=md-nav__item > <a href=PIXEL_MASK.html class=md-nav__link >Pixel mask</a> <li class=md-nav__item > <a href=WEB_FRONTEND.html class=md-nav__link >Web frontend</a> <li class=md-nav__item > <a href=TESTS.html class=md-nav__link >Tests</a> <li class=md-nav__item > <span class="md-nav__link caption"><span class=caption-text >OpenAPI Python client</span></span> <li class=md-nav__item > <a href="python_client/README.html" class=md-nav__link >jfjoch-client</a> <li class=md-nav__item > <a href="python_client/README.html#license-clarification" class=md-nav__link >License Clarification</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html" class=md-nav__link >jfjoch_client.DefaultApi</a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#cancel-post" class=md-nav__link ><strong>cancel_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-get" class=md-nav__link ><strong>config_azim_int_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-azim-int-put" class=md-nav__link ><strong>config_azim_int_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-get" class=md-nav__link ><strong>config_dark_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-dark-mask-put" class=md-nav__link ><strong>config_dark_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-get" class=md-nav__link ><strong>config_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-detector-put" class=md-nav__link ><strong>config_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-get" class=md-nav__link ><strong>config_file_writer_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-file-writer-put" class=md-nav__link ><strong>config_file_writer_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-conversion-post" class=md-nav__link ><strong>config_image_format_conversion_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-get" class=md-nav__link ><strong>config_image_format_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-put" class=md-nav__link ><strong>config_image_format_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-image-format-raw-post" class=md-nav__link ><strong>config_image_format_raw_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-get" class=md-nav__link ><strong>config_indexing_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-indexing-put" class=md-nav__link ><strong>config_indexing_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-get" class=md-nav__link ><strong>config_instrument_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-instrument-put" class=md-nav__link ><strong>config_instrument_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-put" class=md-nav__link ><strong>config_internal_generator_image_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-internal-generator-image-tiff-put" class=md-nav__link ><strong>config_internal_generator_image_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-get" class=md-nav__link ><strong>config_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-mask-tiff-get" class=md-nav__link ><strong>config_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-get" class=md-nav__link ><strong>config_roi_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-roi-put" class=md-nav__link ><strong>config_roi_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-get" class=md-nav__link ><strong>config_select_detector_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-select-detector-put" class=md-nav__link ><strong>config_select_detector_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-get" class=md-nav__link ><strong>config_spot_finding_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-spot-finding-put" class=md-nav__link ><strong>config_spot_finding_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-get" class=md-nav__link ><strong>config_user_mask_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-put" class=md-nav__link ><strong>config_user_mask_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-get" class=md-nav__link ><strong>config_user_mask_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-user-mask-tiff-put" class=md-nav__link ><strong>config_user_mask_tiff_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-get" class=md-nav__link ><strong>config_zeromq_metadata_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-metadata-put" class=md-nav__link ><strong>config_zeromq_metadata_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-get" class=md-nav__link ><strong>config_zeromq_preview_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#config-zeromq-preview-put" class=md-nav__link ><strong>config_zeromq_preview_put</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#deactivate-post" class=md-nav__link ><strong>deactivate_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#detector-status-get" class=md-nav__link ><strong>detector_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#fpga-status-get" class=md-nav__link ><strong>fpga_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-clear-post" class=md-nav__link ><strong>image_buffer_clear_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-cbor-get" class=md-nav__link ><strong>image_buffer_image_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-jpeg-get" class=md-nav__link ><strong>image_buffer_image_jpeg_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-image-tiff-get" class=md-nav__link ><strong>image_buffer_image_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-start-cbor-get" class=md-nav__link ><strong>image_buffer_start_cbor_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-buffer-status-get" class=md-nav__link ><strong>image_buffer_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#image-pusher-status-get" class=md-nav__link ><strong>image_pusher_status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#initialize-post" class=md-nav__link ><strong>initialize_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#pedestal-post" class=md-nav__link ><strong>pedestal_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-pedestal-tiff-get" class=md-nav__link ><strong>preview_pedestal_tiff_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-bin-get" class=md-nav__link ><strong>preview_plot_bin_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#preview-plot-get" class=md-nav__link ><strong>preview_plot_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#result-scan-get" class=md-nav__link ><strong>result_scan_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#start-post" class=md-nav__link ><strong>start_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-calibration-get" class=md-nav__link ><strong>statistics_calibration_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-data-collection-get" class=md-nav__link ><strong>statistics_data_collection_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#statistics-get" class=md-nav__link ><strong>statistics_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#status-get" class=md-nav__link ><strong>status_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#trigger-post" class=md-nav__link ><strong>trigger_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#version-get" class=md-nav__link ><strong>version_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-till-done-post" class=md-nav__link ><strong>wait_till_done_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#wait-until-running-post" class=md-nav__link ><strong>wait_until_running_post</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-event-code-get" class=md-nav__link ><strong>xfel_event_code_get</strong></a> <li class=md-nav__item > <a href="python_client/docs/DefaultApi.html#xfel-pulse-id-get" class=md-nav__link ><strong>xfel_pulse_id_get</strong></a> </ul> </nav> </div> </div> </div> <div class="md-sidebar md-sidebar--secondary" data-md-component=toc > <div class=md-sidebar__scrollwrap > <div class=md-sidebar__inner > <nav class="md-nav md-nav--secondary"> <ul class=md-nav__list data-md-scrollfix=""> </ul> </nav> </div> </div> </div> <div class=md-content > <article class="md-content__inner md-typeset" role=main > <section class="tex2jax_ignore mathjax_ignore" id=data-streams > <h1 id=image-stream--page-root >Data streams<a class=headerlink href="#image-stream--page-root" title="Link to this heading"></a></h1> <p>Jungfraujoch process (<code class="docutils literal notranslate"><span class=pre >jfjoch_broker</span></code>) operates three outputs. All three can be operated/enabled independently. These are:</p> <ul class=simple > <li><p><strong>Image</strong> - all the images including metadata (ZeroMQ PUSH socket or custom TCP/IP socket)</p> <li><p><strong>Preview</strong> - images with metadata at a reduced frame rate (PUB socket)</p> <li><p><strong>Metadata</strong> - only metadata for all the images, bundled into packages (PUB socket)</p> </ul> <section id=image-stream > <h2 id=image-stream >Image stream<a class=headerlink href="#image-stream" title="Link to this heading"></a></h2> <p>Images (with metadata) are serialized as CBOR <a class="reference internal" href="CBOR.html#image-message"><span class="std std-ref">image message</span></a>. The stream will also include CBOR <a class="reference internal" href="CBOR.html#start-message"><span class="std std-ref">start message</span></a>, <a class="reference internal" href="CBOR.html#calibration-message"><span class="std std-ref">calibration messages</span></a> and <a class="reference internal" href="CBOR.html#end-message"><span class="std std-ref">end message</span></a> with run metadata.</p> <p>If <code class="docutils literal notranslate"><span class=pre >file_prefix</span></code> is not provided for a data collection, images won’t be sent to image stream (or its HDF5/CBOR replacements).</p> <section id=splitting-image-stream > <h3 id=splitting-image-stream >Splitting image stream<a class=headerlink href="#splitting-image-stream" title="Link to this heading"></a></h3> <p>Image stream can be split into multiple sockets to increase performance, in this case images will be split according to file number to which the image belongs. All sockets will forward start and end messages. Only first socket will forward calibration messages and will be marked to write master file.</p> </section> <section id=zeromq-image-stream > <h3 id=zeromq-image-stream >ZeroMQ image stream<a class=headerlink href="#zeromq-image-stream" title="Link to this heading"></a></h3> <p>This is using PUSH ZeroMQ socket(s). It should be strictly avoided to have multiple receivers connected to one PUSH ZeroMQ socket. ZeroMQ will send the images in a round-robin basis to the receivers. In this case start and end messages will end up only with one receiver. Instead, Jungfraujoch feature of multiple sockets should be used. For ZeroMQ image stream, each writer connects to a different port.</p> <p>Behavior is as following:</p> <ul class=simple > <li><p>Start message is sent with timeout of 1s per socket. If within the time the message cannot be put in the outgoing queue or there is no connected puller, an exception is thrown — data collection is stopped with an error due to absence of a writer.</p> <li><p>Calibration message is sent to the first socket only, with timeout of 1s.</p> <li><p>Images are sent via a per-socket writer thread. If a send times out, the pusher switches to non-blocking mode for the remainder of the collection (images may be dropped).</p> <li><p>End message is sent with timeout of 1s per socket. No exception is thrown on timeout, but a transmission error is recorded.</p> </ul> <p>The format is generally interchangeable with DECTRIS Stream2 format.</p> <section id=zeromq-configuration > <h4 id=zeromq-configuration >ZeroMQ configuration<a class=headerlink href="#zeromq-configuration" title="Link to this heading"></a></h4> <p>ZeroMQ image stream is configured in the broker JSON configuration file under the <code class="docutils literal notranslate"><span class=pre >zeromq_settings</span></code> section:</p> <div class="highlight-json notranslate"><div class=highlight ><pre><span></span><span class=p >{</span>
<span class=w > </span><span class=nt >"image_socket"</span><span class=p >:</span><span class=w > </span><span class=p >[</span><span class=s2 >"tcp://192.168.0.1:9000"</span><span class=p >,</span><span class=w > </span><span class=s2 >"tcp://192.168.0.1:9001"</span><span class=p >],</span>
<span class=w > </span><span class=nt >"send_watermark"</span><span class=p >:</span><span class=w > </span><span class=mi >100</span><span class=p >,</span>
<span class=w > </span><span class=nt >"send_buffer_size"</span><span class=p >:</span><span class=w > </span><span class=mi >67108864</span><span class=p >,</span>
<span class=w > </span><span class=nt >"writer_notification_socket"</span><span class=p >:</span><span class=w > </span><span class=s2 >"tcp://192.168.0.1:*"</span>
<span class=p >}</span>
</pre></div> </div> <ul class=simple > <li><p><code class="docutils literal notranslate"><span class=pre >image_socket</span></code>: one or more PUSH socket addresses. Multiple entries split the image stream across sockets. Addresses follow ZeroMQ conventions (<code class="docutils literal notranslate"><span class=pre >tcp://</span></code>, <code class="docutils literal notranslate"><span class=pre >ipc://</span></code>). <code class="docutils literal notranslate"><span class=pre >0.0.0.0</span></code> binds on all network interfaces.</p> <li><p><code class="docutils literal notranslate"><span class=pre >send_watermark</span></code> (optional): ZeroMQ send high-water mark (number of outstanding messages per socket).</p> <li><p><code class="docutils literal notranslate"><span class=pre >send_buffer_size</span></code> (optional): OS-level send buffer size for the ZeroMQ socket.</p> <li><p><code class="docutils literal notranslate"><span class=pre >writer_notification_socket</span></code> (optional): see <a class="reference internal" href="#writer-notification-socket"><span class="xref myst">Writer notification socket</span></a> below.</p> </ul> </section> </section> <section id=tcp-ip-image-stream > <h3 id=tcp-ip-image-stream >TCP/IP image stream<a class=headerlink href="#tcp-ip-image-stream" title="Link to this heading"></a></h3> <p>This is using TCP/IP socket(s) with a fixed binary frame header followed by payload bytes. This format was introduced to Jungfraujoch as an alternative to ZeroMQ image stream. It allows two-way communication between the data collection and the writer, and is therefore more robust than ZeroMQ.</p> <p>For TCP/IP image stream, Jungfraujoch <strong>listens</strong> on a single TCP port and all writers <strong>connect</strong> to it. Connections are persistent — writers connect once and stay connected across multiple data collections. Jungfraujoch sends periodic <code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code> frames when no data collection is active to detect dead connections; writers are expected to respond with a <code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code> pong.</p> <p>Using <code class="docutils literal notranslate"><span class=pre >*</span></code> as port number (e.g. <code class="docutils literal notranslate"><span class=pre >tcp://127.0.0.1:*</span></code>) is supported — the OS assigns a free port and the actual bound address can be queried via <code class="docutils literal notranslate"><span class=pre >GetAddress()</span></code>.</p> <p>Payloads for <code class="docutils literal notranslate"><span class=pre >START</span></code>, <code class="docutils literal notranslate"><span class=pre >DATA</span></code>, <code class="docutils literal notranslate"><span class=pre >CALIBRATION</span></code> and <code class="docutils literal notranslate"><span class=pre >END</span></code> frames are CBOR messages, equivalent in content to the ZeroMQ image stream messages.<br/> <code class="docutils literal notranslate"><span class=pre >ACK</span></code>, <code class="docutils literal notranslate"><span class=pre >CANCEL</span></code>, and <code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code> are control frames (no CBOR payload).</p> <p>The data collection lifecycle on each connection follows: <code class="docutils literal notranslate"><span class=pre >START</span></code> → <code class="docutils literal notranslate"><span class=pre >CALIBRATION</span></code> (socket 0 only) → <code class="docutils literal notranslate"><span class=pre >DATA</span></code> (repeated) → <code class="docutils literal notranslate"><span class=pre >END</span></code></p> <p>If a <code class="docutils literal notranslate"><span class=pre >START</span></code> ACK fails on any connection, Jungfraujoch sends <code class="docutils literal notranslate"><span class=pre >CANCEL</span></code> to all already-started connections and rolls back.</p> <p>For each frame:</p> <ol class="arabic simple"> <li><p>Read one <code class="docutils literal notranslate"><span class=pre >TcpFrameHeader</span></code> (fixed size, 64-byte aligned).</p> <li><p>Validate <code class="docutils literal notranslate"><span class=pre >magic</span></code> (<code class="docutils literal notranslate"><span class=pre >0x4A464A54</span></code> / <code class="docutils literal notranslate"><span class=pre >"JFJT"</span></code>) and <code class="docutils literal notranslate"><span class=pre >version</span></code> (<code class="docutils literal notranslate"><span class=pre >2</span></code>).</p> <li><p>Read <code class="docutils literal notranslate"><span class=pre >payload_size</span></code> bytes (if non-zero).</p> </ol> <p>When image stream is split into multiple connections:</p> <ul class=simple > <li><p><code class="docutils literal notranslate"><span class=pre >START</span></code> and <code class="docutils literal notranslate"><span class=pre >END</span></code> are sent on all connections,</p> <li><p><code class="docutils literal notranslate"><span class=pre >CALIBRATION</span></code> is sent only on connection 0,</p> <li><p><code class="docutils literal notranslate"><span class=pre >DATA</span></code> frames are distributed by file grouping: connection index = <code class="docutils literal notranslate"><span class=pre >(image_number</span> <span class=pre >/</span> <span class=pre >images_per_file)</span> <span class=pre >%</span> <span class=pre >num_connections</span></code>.</p> </ul> <section id=tcp-ip-configuration > <h4 id=tcp-ip-configuration >TCP/IP configuration<a class=headerlink href="#tcp-ip-configuration" title="Link to this heading"></a></h4> <p>TCP/IP image stream is configured in the broker JSON configuration file under the <code class="docutils literal notranslate"><span class=pre >tcp_settings</span></code> section:</p> <div class="highlight-json notranslate"><div class=highlight ><pre><span></span><span class=p >{</span>
<span class=w > </span><span class=nt >"image_socket"</span><span class=p >:</span><span class=w > </span><span class=s2 >"tcp://192.168.0.1:9100"</span><span class=p >,</span>
<span class=w > </span><span class=nt >"nwriters"</span><span class=p >:</span><span class=w > </span><span class=mi >2</span><span class=p >,</span>
<span class=w > </span><span class=nt >"send_buffer_size"</span><span class=p >:</span><span class=w > </span><span class=mi >67108864</span>
<span class=p >}</span>
</pre></div> </div> <ul class=simple > <li><p><code class="docutils literal notranslate"><span class=pre >addr</span></code>: listen address in <code class="docutils literal notranslate"><span class=pre >tcp://&lt;IP&gt;:&lt;port&gt;</span></code> format. <code class="docutils literal notranslate"><span class=pre >0.0.0.0</span></code> binds on all interfaces. <code class="docutils literal notranslate"><span class=pre >*</span></code> as port selects a random free port.</p> <li><p><code class="docutils literal notranslate"><span class=pre >nwriters</span></code> (optional): maximum number of simultaneous writer connections accepted.</p> <li><p><code class="docutils literal notranslate"><span class=pre >send_buffer_size</span></code> (optional): OS-level <code class="docutils literal notranslate"><span class=pre >SO_SNDBUF</span></code> size for accepted connections.</p> </ul> </section> <section id=ack-handling > <h4 id=ack-handling >ACK handling<a class=headerlink href="#ack-handling" title="Link to this heading"></a></h4> <p>ACK handling is mandatory for correct operation:</p> <ul class=simple > <li><p><code class="docutils literal notranslate"><span class=pre >START</span></code> <strong>must</strong> be acknowledged (<code class="docutils literal notranslate"><span class=pre >ACK</span></code> with <code class="docutils literal notranslate"><span class=pre >ack_for=START</span></code>) on each connection within 5 seconds, otherwise collection start fails and a rollback is triggered.</p> <li><p><code class="docutils literal notranslate"><span class=pre >END</span></code> <strong>must</strong> be acknowledged (<code class="docutils literal notranslate"><span class=pre >ack_for=END</span></code>) on each connection within 10 seconds for successful completion.</p> <li><p><code class="docutils literal notranslate"><span class=pre >CANCEL</span></code> should be acknowledged during rollback paths (500ms timeout).</p> <li><p><code class="docutils literal notranslate"><span class=pre >DATA</span></code> should be acknowledged for every frame. A <code class="docutils literal notranslate"><span class=pre >DATA</span></code> ACK with <code class="docutils literal notranslate"><span class=pre >FATAL</span></code> flag set reports a downstream error (e.g. disk full) which is propagated to <code class="docutils literal notranslate"><span class=pre >jfjoch_broker</span></code> via <code class="docutils literal notranslate"><span class=pre >Finalize()</span></code>. A failed <code class="docutils literal notranslate"><span class=pre >DATA</span></code> ACK does <strong>not</strong> break the TCP connection on its own — data continues to flow.</p> <li><p><code class="docutils literal notranslate"><span class=pre >CALIBRATION</span></code> is not acknowledged at this time.</p> <li><p><code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code> frames are not acknowledged via ACK; the writer responds with a <code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code> pong frame instead.</p> </ul> </section> <section id=keepalive > <h4 id=keepalive >Keepalive<a class=headerlink href="#keepalive" title="Link to this heading"></a></h4> <p>When no data collection is active, Jungfraujoch sends <code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code> frames approximately every 5 seconds on each persistent connection. Writers should respond with a <code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code> frame (pong). OS-level TCP keepalive is also enabled (<code class="docutils literal notranslate"><span class=pre >TCP_KEEPIDLE=30s</span></code>, <code class="docutils literal notranslate"><span class=pre >TCP_KEEPINTVL=10s</span></code>, <code class="docutils literal notranslate"><span class=pre >TCP_KEEPCNT=3</span></code>) as a secondary safety net. Dead connections are automatically removed from the pool.</p> </section> <section id=zero-copy-transmission > <h4 id=zero-copy-transmission >Zero-copy transmission<a class=headerlink href="#zero-copy-transmission" title="Link to this heading"></a></h4> <p>On Linux, large payload transmission (<code class="docutils literal notranslate"><span class=pre >DATA</span></code> and <code class="docutils literal notranslate"><span class=pre >CALIBRATION</span></code> frames) can use kernel TCP zero-copy (<code class="docutils literal notranslate"><span class=pre >SO_ZEROCOPY</span></code>/<code class="docutils literal notranslate"><span class=pre >MSG_ZEROCOPY</span></code>) when available. If the kernel does not support it or the socket option fails, transmission transparently falls back to normal <code class="docutils literal notranslate"><span class=pre >send()</span></code> behavior. Zero-copy completion notifications are processed by a dedicated per-connection thread.</p> </section> <section id=frame-types > <h4 id=frame-types >Frame types<a class=headerlink href="#frame-types" title="Link to this heading"></a></h4> <table> <thead> <tr class=row-odd ><th class="head text-right"><p>Value</p> <th class=head ><p>Name</p> <th class=head ><p>Purpose</p> <tr class=row-even ><td class=text-right ><p>1</p> <td><p><code class="docutils literal notranslate"><span class=pre >START</span></code></p> <td><p>Start-of-run metadata</p> <tr class=row-odd ><td class=text-right ><p>2</p> <td><p><code class="docutils literal notranslate"><span class=pre >DATA</span></code></p> <td><p>One image payload</p> <tr class=row-even ><td class=text-right ><p>3</p> <td><p><code class="docutils literal notranslate"><span class=pre >CALIBRATION</span></code></p> <td><p>Calibration payload</p> <tr class=row-odd ><td class=text-right ><p>4</p> <td><p><code class="docutils literal notranslate"><span class=pre >END</span></code></p> <td><p>End-of-run metadata</p> <tr class=row-even ><td class=text-right ><p>5</p> <td><p><code class="docutils literal notranslate"><span class=pre >ACK</span></code></p> <td><p>Acknowledgement / error reporting</p> <tr class=row-odd ><td class=text-right ><p>6</p> <td><p><code class="docutils literal notranslate"><span class=pre >CANCEL</span></code></p> <td><p>Cancel run initialization/stream</p> <tr class=row-even ><td class=text-right ><p>7</p> <td><p><code class="docutils literal notranslate"><span class=pre >KEEPALIVE</span></code></p> <td><p>Connection liveness probe/pong</p> </table> </section> <section id=tcp-frame-header-tcpframeheader > <h4 id=tcp-frame-header-tcpframeheader >TCP frame header (<code class="docutils literal notranslate"><span class=pre >TcpFrameHeader</span></code>)<a class=headerlink href="#tcp-frame-header-tcpframeheader" title="Link to this heading"></a></h4> <table> <thead> <tr class=row-odd ><th class=head ><p>Field</p> <th class=head ><p>Type</p> <th class=head ><p>Description</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >magic</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint32_t</span></code></p> <td><p>Protocol magic (<code class="docutils literal notranslate"><span class=pre >0x4A464A54</span></code>, <code class="docutils literal notranslate"><span class=pre >"JFJT"</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >version</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint16_t</span></code></p> <td><p>Protocol version (<code class="docutils literal notranslate"><span class=pre >2</span></code>)</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >type</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint16_t</span></code></p> <td><p>Frame type (see table above)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >image_number</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint64_t</span></code></p> <td><p>Image index for <code class="docutils literal notranslate"><span class=pre >DATA</span></code> frames</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >payload_size</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint64_t</span></code></p> <td><p>Number of payload bytes after header</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >socket_number</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint32_t</span></code></p> <td><p>Connection index in split-stream mode</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >flags</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint32_t</span></code></p> <td><p>ACK flags (<code class="docutils literal notranslate"><span class=pre >OK</span></code>, <code class="docutils literal notranslate"><span class=pre >FATAL</span></code>, <code class="docutils literal notranslate"><span class=pre >HAS_ERROR_TEXT</span></code>)</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >run_number</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint64_t</span></code></p> <td><p>Run identifier</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >ack_processed_images</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint32_t</span></code></p> <td><p>In <code class="docutils literal notranslate"><span class=pre >ACK</span></code>: number of images processed by receiver</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >ack_code</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint16_t</span></code></p> <td><p>In <code class="docutils literal notranslate"><span class=pre >ACK</span></code>: error/status code</p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >ack_for</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint16_t</span></code></p> <td><p>In <code class="docutils literal notranslate"><span class=pre >ACK</span></code>: frame type being acknowledged</p> <tr class=row-odd ><td><p><code class="docutils literal notranslate"><span class=pre >ack_fifo_occupancy</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint16_t</span></code></p> <td><p>In <code class="docutils literal notranslate"><span class=pre >ACK</span></code>: occupancy of input FIFO in the <code class="docutils literal notranslate"><span class=pre >jfjoch_writer</span></code></p> <tr class=row-even ><td><p><code class="docutils literal notranslate"><span class=pre >ack_fifo_max_occupancy</span></code></p> <td><p><code class="docutils literal notranslate"><span class=pre >uint64_t</span></code></p> <td><p>In <code class="docutils literal notranslate"><span class=pre >ACK</span></code>: max occupancy of input FIFO</p> </table> <p>The header is 64-byte aligned (<code class="docutils literal notranslate"><span class=pre >alignas(64)</span></code>).</p> </section> <section id=ack-semantics > <h4 id=ack-semantics >ACK semantics<a class=headerlink href="#ack-semantics" title="Link to this heading"></a></h4> <ul class=simple > <li><p><code class="docutils literal notranslate"><span class=pre >ACK</span></code> frames use <code class="docutils literal notranslate"><span class=pre >ack_for</span></code> to indicate which frame type is acknowledged.</p> <li><p><code class="docutils literal notranslate"><span class=pre >flags</span></code>:</p> <ul> <li><p><code class="docutils literal notranslate"><span class=pre >OK</span></code> (bit 0): operation accepted/successful,</p> <li><p><code class="docutils literal notranslate"><span class=pre >FATAL</span></code> (bit 1): receiver reports unrecoverable error (primarily for <code class="docutils literal notranslate"><span class=pre >DATA</span></code>),</p> <li><p><code class="docutils literal notranslate"><span class=pre >HAS_ERROR_TEXT</span></code> (bit 2): ACK payload contains UTF-8 error text.</p> </ul> <li><p><code class="docutils literal notranslate"><span class=pre >ack_code</span></code> can be used to categorize errors:</p> </ul> <table> <thead> <tr class=row-odd ><th class="head text-right"><p>Code</p> <th class=head ><p>Name</p> <th class=head ><p>Meaning</p> <tr class=row-even ><td class=text-right ><p>0</p> <td><p><code class="docutils literal notranslate"><span class=pre >None</span></code></p> <td><p>No error</p> <tr class=row-odd ><td class=text-right ><p>1</p> <td><p><code class="docutils literal notranslate"><span class=pre >StartFailed</span></code></p> <td><p>START processing failed</p> <tr class=row-even ><td class=text-right ><p>2</p> <td><p><code class="docutils literal notranslate"><span class=pre >DataWriteFailed</span></code></p> <td><p>Image write failed</p> <tr class=row-odd ><td class=text-right ><p>3</p> <td><p><code class="docutils literal notranslate"><span class=pre >EndFailed</span></code></p> <td><p>END processing failed</p> <tr class=row-even ><td class=text-right ><p>4</p> <td><p><code class="docutils literal notranslate"><span class=pre >DiskQuotaExceeded</span></code></p> <td><p>Disk quota exceeded</p> <tr class=row-odd ><td class=text-right ><p>5</p> <td><p><code class="docutils literal notranslate"><span class=pre >NoSpaceLeft</span></code></p> <td><p>No space left on device</p> <tr class=row-even ><td class=text-right ><p>6</p> <td><p><code class="docutils literal notranslate"><span class=pre >PermissionDenied</span></code></p> <td><p>Permission denied</p> <tr class=row-odd ><td class=text-right ><p>7</p> <td><p><code class="docutils literal notranslate"><span class=pre >IoError</span></code></p> <td><p>General I/O error</p> <tr class=row-even ><td class=text-right ><p>8</p> <td><p><code class="docutils literal notranslate"><span class=pre >ProtocolError</span></code></p> <td><p>Protocol-level error</p> </table> </section> </section> <section id=image-stream-replacement > <h3 id=image-stream-replacement >Image stream replacement<a class=headerlink href="#image-stream-replacement" title="Link to this heading"></a></h3> <p>Image stream can be replaced with direct HDF5 writer and CBOR dump image pushers, or it can be disabled by selecting “None” image pusher for all the measurements.</p> </section> </section> <section id=writer-notification-socket > <h2 id=writer-notification-socket >Writer notification socket<a class=headerlink href="#writer-notification-socket" title="Link to this heading"></a></h2> <p>The writer notification socket is used <strong>only with ZeroMQ image stream</strong>. Since ZeroMQ is asynchronous, <code class="docutils literal notranslate"><span class=pre >jfjoch_broker</span></code> does not know whether messages were properly handled downstream (e.g. written to disk). The writer notification socket allows downstream code to report back.</p> <p>For TCP/IP image stream, this mechanism is not needed — ACK frames provide synchronous feedback for each control and data frame.</p> <p>To use writer notification socket, it has to be first enabled in the JSON configuration file of broker with <code class="docutils literal notranslate"><span class=pre >writer_notification_socket</span></code> entry:</p> <div class="highlight-json notranslate"><div class=highlight ><pre><span></span><span class=p >{</span>
<span class=w > </span><span class=nt >"writer_notification_socket"</span><span class=p >:</span><span class=s2 >"tcp://192.168.0.1:*"</span>
<span class=p >}</span>
</pre></div> </div> <p>Such entry will create PULL socket on <code class="docutils literal notranslate"><span class=pre >192.168.0.1</span></code> network interface listening on one, random TCP port. When data processing is started, the image stream will send CBOR <a class="reference internal" href="CBOR.html#start-message"><span class="std std-ref">start message</span></a>. This message will include information on <code class="docutils literal notranslate"><span class=pre >writer_notification_zmq_addr</span></code>, which needs to be used by downstream code. Since the start message must reference the address of <code class="docutils literal notranslate"><span class=pre >jfjoch_broker</span></code> host, notification socket should always listen on a particular network interface, and should not be configured with placeholder address <code class="docutils literal notranslate"><span class=pre >0.0.0.0</span></code>. It is, however, OK to use placeholder <code class="docutils literal notranslate"><span class=pre >:*</span></code> for network port, as it will be substituted for the one chosen by ZeroMQ.</p> <p>For every image stream socket, downstream code must send the following message to the PULL socket:</p> <div class="highlight-json notranslate"><div class=highlight ><pre><span></span><span class=p >{</span>
<span class=w > </span><span class=nt >"run_number"</span><span class=p >:</span><span class=mi >135</span><span class=p >,</span>
<span class=w > </span><span class=nt >"run_name"</span><span class=p >:</span><span class=w > </span><span class=s2 >"lysozyme_1"</span><span class=p >,</span>
<span class=w > </span><span class=nt >"socket_number"</span><span class=p >:</span><span class=w > </span><span class=mi >1</span><span class=p >,</span>
<span class=w > </span><span class=nt >"processed_images"</span><span class=p >:</span><span class=mi >250</span><span class=p >,</span>
<span class=w > </span><span class=nt >"ok"</span><span class=p >:</span><span class=w > </span><span class=kc >true</span>
<span class=p >}</span>
</pre></div> </div> <p>Here <code class="docutils literal notranslate"><span class=pre >run_number</span></code>, <code class="docutils literal notranslate"><span class=pre >run_name</span></code> and <code class="docutils literal notranslate"><span class=pre >socket_number</span></code> must match information from the start message. <code class="docutils literal notranslate"><span class=pre >ok</span></code> is boolean confirming if the writing process was OK. <code class="docutils literal notranslate"><span class=pre >processed_images</span></code> is number of images that were written/processed, this is to track how many images were ignored by non-blocking ZeroMQ procedures. If not, it is possible to include error message:</p> <div class="highlight-json notranslate"><div class=highlight ><pre><span></span><span class=p >{</span>
<span class=w > </span><span class=nt >"run_number"</span><span class=p >:</span><span class=mi >135</span><span class=p >,</span>
<span class=w > </span><span class=nt >"run_name"</span><span class=p >:</span><span class=w > </span><span class=s2 >"lysozyme_1"</span><span class=p >,</span>
<span class=w > </span><span class=nt >"socket_number"</span><span class=p >:</span><span class=w > </span><span class=mi >1</span><span class=p >,</span>
<span class=w > </span><span class=nt >"processed_images"</span><span class=p >:</span><span class=w > </span><span class=mi >0</span><span class=p >,</span>
<span class=w > </span><span class=nt >"ok"</span><span class=p >:</span><span class=w > </span><span class=kc >false</span><span class=p >,</span>
<span class=w > </span><span class=nt >"error"</span><span class=p >:</span><span class=w > </span><span class=s2 >"Permission error"</span>
<span class=p >}</span>
</pre></div> </div> <p>This way errors from the downstream code are propagated to <code class="docutils literal notranslate"><span class=pre >jfjoch_broker</span></code>.</p> <p>If writer notification socket is configured, but downstream code doesn’t send proper notification, <code class="docutils literal notranslate"><span class=pre >jfjoch_broker</span></code> will time out after 60 seconds producing an error message.</p> </section> <section id=preview-stream > <h2 id=preview-stream >Preview stream<a class=headerlink href="#preview-stream" title="Link to this heading"></a></h2> <p>Jungfraujoch can also send images (with metadata) at a reduced frame rate for preview purpose. Images are serialized as CBOR <a class="reference internal" href="CBOR.html#image-message"><span class="std std-ref">image message</span></a>. The stream will also include CBOR <a class="reference internal" href="CBOR.html#start-message"><span class="std std-ref">start message</span></a> and <a class="reference internal" href="CBOR.html#end-message"><span class="std std-ref">end message</span></a> with run metadata. Only start and image messages are sent.</p> <p>This is using PUB socket with conflate option. I.e., only the last message is kept by ZeroMQ, so if receiver cannot cope with the messages, it will always receive the last generated message (no backlog). For this reason it is also recommended to use the same option on receiver side.</p> <p>Given PUB socket properties, it is possible to connect multiple viewers to a single socket — all the viewers should receive all the images sent.</p> </section> <section id=metadata-stream > <h2 id=metadata-stream >Metadata stream<a class=headerlink href="#metadata-stream" title="Link to this heading"></a></h2> <p>Jungfraujoch can also send pure metadata for the purpose of archiving such information. Metadata are serialized as CBOR <a class="reference internal" href="CBOR.html#metadata-message"><span class="std std-ref">metadata message</span></a>. This is very similar as image message, but excludes the actual image array and spot positions. As metadata are relatively small, to avoid large number of messages, Jungfraujoch bundles metadata of many images in one message. Order of images within bundle, as well a size of the bundle, are not guaranteed. The stream will also include CBOR <a class="reference internal" href="CBOR.html#start-message"><span class="std std-ref">start message</span></a> and <a class="reference internal" href="CBOR.html#end-message"><span class="std std-ref">end message</span></a> with run metadata.</p> <p>This is using PUB socket with watermark, so there is some queuing of messages with ZeroMQ. Multiple receivers can be connected.</p> </section> </section> </article> </div> </div> </main> </div> <footer class=md-footer > <div class=md-footer-nav > <nav class="md-footer-nav__inner md-grid"> <a href=CBOR.html title="CBOR messages" class="md-flex md-footer-nav__link md-footer-nav__link--prev" rel=prev > <div class="md-flex__cell md-flex__cell--shrink"> <i class="md-icon md-icon--arrow-back md-footer-nav__button"></i> </div> <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"> <span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Previous" </span> CBOR messages </span> </div> </a> <a href=PIXEL_MASK.html title="Pixel mask" class="md-flex md-footer-nav__link md-footer-nav__link--next" rel=next > <div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title"><span class=md-flex__ellipsis > <span class=md-footer-nav__direction > "Next" </span> Pixel mask </span> </div> <div class="md-flex__cell md-flex__cell--shrink"><i class="md-icon md-icon--arrow-forward md-footer-nav__button"></i> </div> </a> </nav> </div> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class=md-footer-copyright > <div class=md-footer-copyright__highlight > &#169; Copyright 2024, Paul Scherrer Institute. </div> Created using <a href="http://www.sphinx-doc.org/">Sphinx</a> 8.1.3. and <a href="https://github.com/bashtage/sphinx-material/">Material for Sphinx</a> </div> </div> </div> </footer> <script src="_static/javascripts/application.js"></script> <script>app.initialize({version: "1.0.4", url: {base: ".."}})</script>
Reference in New Issue View Git Blame Copy Permalink