From 0a515bb08640d53a3812f0ee04ce0db8ed34f90c Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Fri, 23 Oct 2026 17:56:42 +0200 Subject: [PATCH 01/17] rc179: everything on rc178 after its merge into main (#88), except the CHANGELOG The rc178 branch continued after #88 was squash-merged; this is its content from b9ada7914 (the tree #88 squashed) to 094eee31a, as one change on main. The individual commits stay on the rc178 branch: - still battery: --cold evicts each set's input from the page cache; Alvra pixel masks; manifest fixes - AGIPD: converter to one rigid panel per module, int16 photons, mask folded in; still battery reads the converted sets - DetectorModules: panel look-up by id, plane -> panel, plane bounds and a per-pixel walk; mask size checked - tests: [PanelLayout] equivalence harness, stacked vs assembled panel layout - Dense per-pixel maps through the panel table (panel layout stage 1) - Panel id on spots and the panel table in the start message - Viewer and preview show a panel detector as it lies on the detector plane - Panel layout stage 3: per-panel spot finding (CPU+GPU, CCL) and raster algorithms on the plane raster - Panel layout stage 2: prediction and Bragg integration through the panel table - XTC reader: frames handed up as stored, streams indexed in parallel - XFEL reader: integer frames stay integer, one gain, placement by panel - EXTERNAL_TEST_DATA_STILLS.md: images used vs the entry for every set; drop "Not in the battery" - still battery: EuXFEL sets read the depositors' Cheetah CXI files directly; AGIPD converter removed - float32_to_int16.py: a non-finite pixel becomes -32768, the int16 error marker, not a zero count - CrystFEL geometry: read the pixel masks and bad regions instead of ignoring them - Panel layout stage 5: the XFEL reader hands up the stored frame and its panel table - Panel table persisted in NXmx; modulation grid and pre-scan memo key on the plane - still battery: Alvra sets take their pixel mask from the CrystFEL geometry, not --mask-good - still battery: take the deposited model as mmCIF where the RCSB has no PDB-format file - still battery: dimple gets PDB-format coordinates converted from an mmCIF-only deposition - stills fixed cost: geometry refinement and pre-scan do the same work in less wall time - stills geometry sample and trial spot finding run as many workers as the image pass - Stills integration: background ring reach measured on empty apertures, radial curvature from the whole frame - StillsPartialityRefine: the partiality width is the run's, measured on the crystals, not the prediction window's - MergeOnTheFly: capture term and multiplicative outlier band for the stills merge (both opt-in) - Stills given a cell: the written cell is the mean over the frames that are merged - Stills: the radial curvature correction decided per run on the same empty apertures - StillsPartialityRefine: measure the run width only on shells with signal - Stills pre-scan: report the paired z of the radial-correction comparison - module placement: the five concurrent fits each keep every thread - still battery: score R-free against the deposited data through the same dimple protocol - still battery: the R-free target is the lower of the deposition's dimple and published values - Stills: background clip follows the radial curve; signal disk from the lattice spacing - Stills keep an overloaded observation as a lower bound instead of dropping it - Stills: drop the lattice-spacing signal disk; the correction decision back on the plain clip - battery: R-free against the deposition without refinement only, one protocol for rotation and stills - still battery: EuXFEL sets run on a 2,000-image subset for now - stills geometry refinement: fit the detector distance and the cell together - stills geometry refinement: a distance given with --detector-distance is held in the joint fit - Metal integration engine: back to its panel-s5 kernels; Mac stills keep the old clip, no saturation count - Metal integration test: the CPU comparison runs on a rotation experiment - Stills: kept overloads go back to the drop where they are not concentrated at low resolution - stills distance search: an absolute 30-3000 mm scan, a sample found without d-limits, the fit iterated - battery results: XFEL still battery 20261022-1358 on rc178 0cf62452c - 12 sets, cold, R_free unrefined - battery results: still summary gains the resolution of our merge and of the deposition - Pre-scan masks a pixel at the saturation limit on most frames, as one holding the error value - Stills keep an overload as a lower bound only where it has pixels at the limit; run-level gate removed - XFEL reader: a per-frame mask is read the way the frames are, its chunk inflated outside the HDF5 lock - stills distance search: every centre scanned with a wider acceptance, a larger sample where nothing indexes - rugnux usage: --beam-center-check describes the absolute 30-3000 mm stills distance scan, not the old 60-160 % grid - stills: detector tilt refined in the geometry pre-pass, kept where held-out frames agree - CLAUDE.md: the changelog is written once, just before the merge into main - condensed one-liners of what users get - broker: serve the placed model and its maps of the last rugnux live run (/rugnux/model.cif, /rugnux/map.ccp4) - frontend: Mol* viewer of the placed model and its density on the Rugnux live page - writer: _model.cif from rugnux live keeps the input model's metadata, as the served model.cif does - rugnux live maps: one unit cell centred on the model; the viewer opens on all of it - Anomalous difference map: ANOM/PHANOM in _maps.mtz, /rugnux/map.ccp4?type=anom, a magenta mesh in the viewer - Mol* viewer: open with the maps drawn within 10 A of the centre of view - Mol* viewer: aaredb's structure-viewer setup, on a black stage - rugnux: the gain of a stills file that states none is read off its single-photon peak where the background resolves photons - Mol* viewer: in line with AareDB's StructureView at its current head (8ece2c2) - ACKNOWLEDGEMENT: the web frontend's Mol* viewer follows AareDB's setup (G. Gotthard, PSI) - stills tilt decision: the modules stay where the run placed them - still battery: no distance pins and no per-set tuning flags; gain from the data except where a file's baseline hides it - battery results: rc178 6f6f69 - full still battery and full rotation battery with REFMAC - Bragg integration: a CCD rotation sweep's sigmas at the measured gain, intensities in photons - BackgroundGain: the gain of a frame that resolves photons is the spacing of its photon peaks, so a baseline left in the frame no longer hides it - ShadowFinder: measure the background's azimuthal modulation instead of predicting it from the polarization - BackgroundGain: one gain estimate for rotation and stills; the still battery no longer gives EuXFEL 152 its gain - ShadowFinder: measure the modulation robustly under a wide one-sided shadow - ShadowFinderTest: a one-sided shadow is not fitted as part of the background's modulation - ShadowFinder: back to the predicted polarization factor, in the plane the background shows - MXAnalysisAfterFPGA: the Bragg engine's edge mask in the image's layout, so raw / untransformed FPGA runs start again - battery results: still battery on rc178 e54f55 replaces the earlier rc178 still tables - XFELReader: per-frame mask and photon conversion without a library call or a branch per pixel - stills geometry trial: the hypotheses a search asks are indexed side by side - stills tilt decision: the fits start beside the pre-scan and the geometry trial - XFELReader: a deflated frame is inflated and converted a piece at a time, while it is in the cache - XFELReader: a byte-shuffled frame is unshuffled a piece at a time, a whole pixel per step - GeometryRefiner: residuals carry only the blocks a fit moves; beam and distance one block - GeometryRefiner: a spot's residual differentiates its observed and predicted halves apart - docs: EXTERNAL_TEST_DATA lists every open-arm battery set, with corrected values and counts - battery: doc_check.py cross-checks the manifests against the dataset pages - stills module placement: the placement fit gets half the threads, each held-out fit an eighth - still battery: the EuXFEL sets run on all their frames, as the published still runs did - battery results: still battery on rc178 a0ce6c, cold on a quiet machine, replaces the earlier rc178 table - macOS build: explicit size_t in the XFEL reader's row bounds; parentheses in the CPU preprocessor's select Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu --- CLAUDE.md | 28 +- broker/JFJochBrokerHttp.cpp | 40 +- broker/JFJochBrokerHttp.h | 2 + broker/OpenAPIConvert.cpp | 3 + broker/gen/model/Rugnux_live_status.cpp | 30 +- broker/gen/model/Rugnux_live_status.h | 14 + broker/jfjoch_api.yaml | 81 + broker/redoc-static.html | 77 +- common/AssembleToPlane.cpp | 16 + common/AssembleToPlane.h | 68 + common/AzimuthalIntegrationMapping.cpp | 32 +- common/AzimuthalIntegrationMapping.h | 8 +- common/BraggIntegrationSettings.cpp | 11 + common/BraggIntegrationSettings.h | 9 + common/CMakeLists.txt | 5 + common/DetectorModules.cpp | 41 + common/DetectorModules.h | 47 + common/DiffractionExperiment.cpp | 11 +- common/DiffractionSpot.cpp | 16 +- common/DiffractionSpot.h | 6 +- common/JFJochMessages.h | 12 + common/PanelBoxes.cpp | 17 + common/PanelBoxes.h | 23 + common/PanelRows.h | 39 + common/ROIMap.cpp | 18 +- common/ROIMap.h | 6 +- common/SpotToSave.h | 1 + docs/ACKNOWLEDGEMENT.md | 6 + docs/CBOR.md | 4 +- docs/CPU_DATA_ANALYSIS_DECISIONS.md | 2 +- docs/CPU_DATA_ANALYSIS_IMAGE.md | 18 +- docs/EXTERNAL_TEST_DATA.md | 224 +- docs/EXTERNAL_TEST_DATA_STILLS.md | 58 +- docs/HDF5.md | 1 + docs/RUGNUX_ADVANCED.md | 2 +- docs/RUGNUX_FORMATS.md | 28 +- docs/RUGNUX_REPORT.md | 6 + docs/python_client/README.md | 2 + docs/python_client/docs/DefaultApi.md | 164 + docs/python_client/docs/RugnuxLiveStatus.md | 2 + frame_serialize/CBORStream2Deserializer.cpp | 57 + frame_serialize/CBORStream2Serializer.cpp | 34 + frontend/package-lock.json | 1141 +- frontend/package.json | 1 + .../src/client/@tanstack/react-query.gen.ts | 51 +- frontend/src/client/index.ts | 4 +- frontend/src/client/sdk.gen.ts | 31 +- frontend/src/client/types.gen.ts | 82 + frontend/src/client/zod.gen.ts | 20 + frontend/src/components/RugnuxPage.tsx | 10 + .../src/components/RugnuxStructureViewer.tsx | 248 + .../components/structure/MolstarCanvas.tsx | 107 + .../src/components/structure/cootControls.ts | 198 + .../src/components/structure/cootTrackball.ts | 55 + .../src/components/structure/createPlugin.ts | 37 + .../src/components/structure/densityCrop.ts | 249 + frontend/src/components/structure/loadMaps.ts | 109 + .../src/components/structure/loadModel.ts | 129 + image_analysis/IndexAndRefine.cpp | 7 +- image_analysis/MXAnalysisAfterFPGA.cpp | 6 +- image_analysis/MXAnalysisWithoutFPGA.cpp | 69 +- image_analysis/beam_stop/SHADOW_FINDER.md | 24 +- image_analysis/beam_stop/ShadowFinder.cpp | 198 +- image_analysis/beam_stop/ShadowFinder.h | 7 + .../beam_stop/ShadowFinderInternal.h | 18 + .../bragg_integration/BackgroundAperture.cpp | 181 + .../bragg_integration/BackgroundAperture.h | 79 + .../bragg_integration/BackgroundGain.cpp | 71 + .../bragg_integration/BackgroundGain.h | 28 + .../BraggIntegrationEngine.cpp | 70 +- .../BraggIntegrationEngine.h | 53 +- .../BraggIntegrationEngineCPU.cpp | 179 +- .../BraggIntegrationEngineCPU.h | 14 +- .../BraggIntegrationEngineGPU.cu | 168 +- .../BraggIntegrationEngineGPU.h | 5 + .../BraggIntegrationEngineMetal.metal | 55 +- .../BraggIntegrationEngineMetal.mm | 34 +- .../bragg_integration/BraggPanel.cpp | 18 + image_analysis/bragg_integration/BraggPanel.h | 16 + image_analysis/bragg_integration/BraggPassA.h | 3 + .../bragg_integration/BraggStencil.h | 49 +- .../bragg_integration/CMakeLists.txt | 6 + .../bragg_integration/RingBackground.cpp | 33 + .../bragg_integration/RingBackground.h | 20 + .../bragg_integration/SpotFootprint.cpp | 15 +- .../bragg_integration/SpotFootprint.h | 12 +- .../bragg_integration/SpotWidth.cpp | 51 +- image_analysis/bragg_integration/SpotWidth.h | 11 +- .../bragg_prediction/BraggPrediction.cpp | 9 +- .../bragg_prediction/BraggPredictionGPU.cu | 9 +- .../bragg_prediction/BraggPredictionGPU.h | 3 +- .../bragg_prediction/BraggPredictionRot.cpp | 9 +- .../bragg_prediction/BraggPredictionRotGPU.cu | 9 +- .../bragg_prediction/BraggPredictionRotGPU.h | 3 +- .../geom_refinement/BackgroundBand.h | 20 +- .../BeamCenterBackgroundGPU.cu | 33 +- .../geom_refinement/BeamCenterBackgroundGPU.h | 3 +- .../geom_refinement/BeamCenterFFT.cpp | 19 +- .../BeamCenterFromBackground.cpp | 48 +- .../geom_refinement/GeometryRefiner.cpp | 378 +- .../geom_refinement/GeometryRefiner.h | 14 +- image_analysis/hot_pixels/HotPixels.cpp | 60 +- image_analysis/hot_pixels/HotPixels.h | 9 +- image_analysis/hot_pixels/HotPixelsGPU.cu | 4 +- .../ImagePreprocessorBuffer.cpp | 14 +- .../ImagePreprocessorBuffer.h | 13 +- .../ImagePreprocessorBufferGPU.cu | 28 +- .../ImagePreprocessorCPU.cpp | 8 +- .../ImagePreprocessorGPU.cu | 3 + image_analysis/roi/ROIIntegration.cpp | 33 +- image_analysis/roi/ROIIntegration.h | 16 +- image_analysis/roi/ROIIntegrationCPU.h | 23 +- image_analysis/roi/ROIIntegrationGPU.cu | 91 +- image_analysis/roi/ROIIntegrationGPU.h | 11 +- image_analysis/scale_merge/Merge.cpp | 65 +- image_analysis/scale_merge/Merge.h | 8 +- .../scale_merge/RotationScaleMerge.cpp | 11 +- .../scale_merge/StillsPartialityRefine.cpp | 157 +- .../scale_merge/StillsPartialityRefine.h | 19 +- .../spot_finding/ImageSpotFinder.cpp | 37 +- image_analysis/spot_finding/ImageSpotFinder.h | 12 + .../spot_finding/ImageSpotFinderCPU.cpp | 194 +- .../spot_finding/ImageSpotFinderCPU.h | 1 + .../spot_finding/ImageSpotFinderGPU.cu | 107 +- .../spot_finding/ImageSpotFinderGPU.h | 4 + .../spot_finding/SpotExtractorGPU.cu | 55 +- .../spot_finding/SpotExtractorGPU.h | 6 + image_analysis/spot_finding/SpotUtils.cpp | 4 +- .../structure_refinement/ModelValidation.cpp | 23 +- preview/PreviewImage.cpp | 33 +- preview/PreviewImage.h | 7 +- reader/CrystFELGeometry.cpp | 100 +- reader/CrystFELGeometry.h | 19 +- reader/DetectorLayout.h | 14 +- reader/HDF5MetadataSource.cpp | 9 + reader/JFJochXFELReader.cpp | 606 +- reader/JFJochXFELReader.h | 66 +- reader/JFJochXTCReader.cpp | 45 +- reader/XTC.cpp | 11 +- reader/XTC.h | 5 +- receiver/CMakeLists.txt | 3 +- receiver/JFJochReceiverRugnux.cpp | 11 + receiver/RugnuxLiveStatus.cpp | 5 + receiver/RugnuxLiveStatus.h | 6 + receiver/RugnuxStructure.cpp | 103 + receiver/RugnuxStructure.h | 46 + rugnux/DiagnosticOutput.cpp | 13 +- rugnux/ModuleDiagnostics.cpp | 2 +- rugnux/ResultReport.cpp | 10 + rugnux/Rugnux.cpp | 39 +- rugnux/Rugnux.h | 72 +- rugnux/RugnuxFirstPassAccept.cpp | 7 +- rugnux/RugnuxImagePass.cpp | 43 +- rugnux/RugnuxPasses.cpp | 11 +- rugnux/RugnuxPreScan.cpp | 187 +- rugnux/RugnuxScaleMerge.cpp | 6 +- rugnux/RugnuxStills.cpp | 745 +- rugnux/rugnux_cli.cpp | 109 +- tests/BackgroundApertureTest.cpp | 118 + tests/BackgroundGainTest.cpp | 90 + tests/BandwidthEstimateTest.cpp | 2 +- tests/BeamCenterFromBackgroundTest.cpp | 39 + tests/BraggIntegrationEngineCPUTest.cpp | 254 +- tests/BraggIntegrationEngineGPUTest.cpp | 131 +- tests/BraggIntegrationEngineMetalTest.cpp | 3 + tests/BrokerHttpAuthTest.cpp | 8 +- tests/CBORTest.cpp | 81 + tests/CMakeLists.txt | 4 + tests/CrystFELGeometryTest.cpp | 104 +- tests/HotPixelFinderTest.cpp | 13 +- tests/JFJochReaderTest.cpp | 82 + tests/MergeScaleTest.cpp | 116 +- tests/ModelValidationTest.cpp | 45 + tests/ModuleRefinementTest.cpp | 163 +- tests/PanelLayoutTest.cpp | 637 + tests/ROIIntegrationGPUTest.cpp | 46 +- tests/ROIMapTest.cpp | 2 +- tests/RugnuxResultFilesTest.cpp | 37 + tests/RugnuxStructureTest.cpp | 154 + tests/RugnuxTest.cpp | 2 +- tests/ShadowFinderTest.cpp | 77 + tests/SpotTilesTest.cpp | 4 +- tests/TIFFTest.cpp | 56 + tests/XFELReaderTest.cpp | 367 +- tools/battery/README.md | 89 +- tools/battery/battery.py | 13 +- tools/battery/doc_check.py | 137 + tools/battery/model_check.py | 116 +- tools/battery/open.json | 2 +- tools/battery/paper_plots.py | 14 +- .../manifest.json | 421 + .../report.md | 40 + .../results.json | 464 + .../summary.md | 46 + .../manifest.json | 7333 ++++ .../report.md | 735 + .../results.json | 35814 ++++++++++++++++ tools/battery/still/float32_to_int16.py | 11 +- tools/battery/still_battery.py | 171 +- tools/battery/still_manifest.json | 22 +- viewer/JFJochHttpReader.cpp | 2 + viewer/JFJochImageReadingWorker.cpp | 18 +- viewer/JFJochViewerWindow.cpp | 4 +- .../image_viewer/JFJochDiffractionImage.cpp | 92 +- viewer/image_viewer/JFJochDiffractionImage.h | 9 +- viewer/image_viewer/JFJochFollowerImage.cpp | 6 +- viewer/image_viewer/JFJochFollowerImage.h | 4 +- viewer/image_viewer/JFJochImage.cpp | 17 +- viewer/image_viewer/JFJochImage.h | 4 + viewer/widgets/JFJochViewerMagnifier.cpp | 4 +- viewer/widgets/JFJochViewerMagnifier.h | 2 +- writer/HDF5DataFilePluginMX.cpp | 7 + writer/HDF5DataFilePluginMX.h | 2 + writer/HDF5NXmx.cpp | 11 + writer/RugnuxResultBlocks.cpp | 17 + writer/WriteModel.cpp | 54 +- writer/WriteModel.h | 4 + writer/WriteRugnuxFiles.cpp | 30 + 218 files changed, 55733 insertions(+), 1764 deletions(-) create mode 100644 common/AssembleToPlane.cpp create mode 100644 common/AssembleToPlane.h create mode 100644 common/PanelBoxes.cpp create mode 100644 common/PanelBoxes.h create mode 100644 common/PanelRows.h create mode 100644 frontend/src/components/RugnuxStructureViewer.tsx create mode 100644 frontend/src/components/structure/MolstarCanvas.tsx create mode 100644 frontend/src/components/structure/cootControls.ts create mode 100644 frontend/src/components/structure/cootTrackball.ts create mode 100644 frontend/src/components/structure/createPlugin.ts create mode 100644 frontend/src/components/structure/densityCrop.ts create mode 100644 frontend/src/components/structure/loadMaps.ts create mode 100644 frontend/src/components/structure/loadModel.ts create mode 100644 image_analysis/bragg_integration/BackgroundAperture.cpp create mode 100644 image_analysis/bragg_integration/BackgroundAperture.h create mode 100644 image_analysis/bragg_integration/BraggPanel.cpp create mode 100644 image_analysis/bragg_integration/BraggPanel.h create mode 100644 image_analysis/bragg_integration/RingBackground.cpp create mode 100644 image_analysis/bragg_integration/RingBackground.h create mode 100644 receiver/RugnuxStructure.cpp create mode 100644 receiver/RugnuxStructure.h create mode 100644 tests/BackgroundApertureTest.cpp create mode 100644 tests/PanelLayoutTest.cpp create mode 100644 tests/RugnuxStructureTest.cpp create mode 100644 tools/battery/doc_check.py create mode 100644 tools/battery/results/stills/20261023-1303_a0ce6c_rc178-final/manifest.json create mode 100644 tools/battery/results/stills/20261023-1303_a0ce6c_rc178-final/report.md create mode 100644 tools/battery/results/stills/20261023-1303_a0ce6c_rc178-final/results.json create mode 100644 tools/battery/results/stills/20261023-1303_a0ce6c_rc178-final/summary.md create mode 100644 tools/battery/results/workstation/20261022-2256_6f6f69_rc178-6f6f69-full-refmac/manifest.json create mode 100644 tools/battery/results/workstation/20261022-2256_6f6f69_rc178-6f6f69-full-refmac/report.md create mode 100644 tools/battery/results/workstation/20261022-2256_6f6f69_rc178-6f6f69-full-refmac/results.json diff --git a/CLAUDE.md b/CLAUDE.md index 3f430a2e5..0bb1218a2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -150,19 +150,27 @@ accident; name the files the change actually touches, and read `git status` befo ## Changelog -`docs/CHANGELOG.md` is written **for users**. It does not replace commit messages or developer -notes, so keep out of it anything that belongs there instead: +`docs/CHANGELOG.md` is written **for users**: it tells them what to expect from the new version. +It does not replace commit messages or developer notes, so keep out of it anything that belongs there +instead. -- **One line per entry**, ideally one sentence. Two only when a user genuinely needs both. -- **Say what changed** — not why, not what the investigation found, not how the change was arrived - at, not what it measured on. Rationale, measurements and history go in the commit message. +**When.** The changelog is written **once, just before the release branch is merged into main**, on +the maintainer's request, when the branch's full content is known. Never add or edit +`docs/CHANGELOG.md` as part of a feature, fix or performance change - describe the change in its +commit message instead. Written per change, every piece of work looks important enough for an entry +of its own; written at the end, it takes its true place in the release. + +**How.** +- **Condensed.** A release is a handful of one-liners. Fold connected changes into a single entry - + that is the expected outcome, not a loss of detail. +- **What users get, stated as fact.** Describe the capability or the behaviour a user will see. + Leave out the development story: what was tried, what failed, what was found along the way, how + the change was built. Rationale, measurements and history belong in the commit messages. +- **General beats specific.** "Significantly faster processing of serial data" serves a user better + than naming the function that changed, the reason it changed and the percentage it saved on one + dataset. - **No sample identities**, exactly as everywhere else (see below). -**Entries are written only on the maintainer's explicit request**, normally once per release branch -before it is merged, as a short summary of user-visible changes. Do not add or edit -`docs/CHANGELOG.md` as part of a feature, fix or performance change; describe the change in its -commit message instead. - ## Design direction Three standing preferences. They are **directional**, not absolute: they say which way to lean when a diff --git a/broker/JFJochBrokerHttp.cpp b/broker/JFJochBrokerHttp.cpp index 4853b4b0a..a6e7255a6 100644 --- a/broker/JFJochBrokerHttp.cpp +++ b/broker/JFJochBrokerHttp.cpp @@ -3,6 +3,7 @@ // Using OpenAPI licensed with Apache License 2.0 #include "JFJochBrokerHttp.h" +#include "../receiver/RugnuxStructure.h" #include @@ -155,7 +156,9 @@ static const std::set PROTECTED_PATHS{ "/preview/plot", "/preview/plot.bin", "/live/events", - "/rugnux/status" + "/rugnux/status", + "/rugnux/model.cif", + "/rugnux/map.ccp4" }; std::optional JFJochBrokerHttp::AuthorizationHeader(const httplib::Request &req) { @@ -259,6 +262,14 @@ void JFJochBrokerHttp::register_routes(httplib::Server &server) { server.Put("/config/rugnux/model", bind_req(&JFJochBrokerHttp::config_rugnux_model_put)); server.Delete("/config/rugnux/model", bind_noarg(&JFJochBrokerHttp::config_rugnux_model_delete)); server.Get("/rugnux/status", bind_noarg(&JFJochBrokerHttp::rugnux_status_get)); + server.Get("/rugnux/model.cif", bind_noarg(&JFJochBrokerHttp::rugnux_model_cif_get)); + server.Get("/rugnux/map.ccp4", [this](const httplib::Request &req, httplib::Response &res) { + try { + rugnux_map_ccp4_get(parse_query_string(req, "type"), res); + } catch (const std::exception &e) { + send_error(res, handleOperationException(e)); + } + }); server.Get("/config/bragg_integration", bind_noarg(&JFJochBrokerHttp::config_bragg_integration_get)); server.Put("/config/bragg_integration", bind_json(&JFJochBrokerHttp::config_bragg_integration_put, Bragg_integration_settings{})); server.Get("/config/instrument", bind_noarg(&JFJochBrokerHttp::config_instrument_get)); @@ -1025,6 +1036,33 @@ void JFJochBrokerHttp::rugnux_status_get(httplib::Response &response) { ProcessOutput(Convert(state_machine.GetRugnuxStatus()), response); } +void JFJochBrokerHttp::rugnux_model_cif_get(httplib::Response &response) { + const auto structure = state_machine.GetRugnuxStatus().structure; + if (!structure) { + response.status = 404; + return; + } + response.set_content(structure->ModelCif(), "chemical/x-mmcif"); +} + +void JFJochBrokerHttp::rugnux_map_ccp4_get(const std::optional &type, httplib::Response &response) { + const std::string t = type.value_or("2fofc"); + if (t != "2fofc" && t != "fofc" && t != "anom") { + response.status = 400; + response.set_content("Unknown map type " + t, "text/plain"); + return; + } + const auto structure = state_machine.GetRugnuxStatus().structure; + const RugnuxMapType map_type = t == "2fofc" ? RugnuxMapType::TwoFoFc + : t == "fofc" ? RugnuxMapType::FoFc : RugnuxMapType::Anomalous; + const std::string map = structure ? structure->Map(map_type) : std::string(); + if (map.empty()) { + response.status = 404; + return; + } + response.set_content(map, "application/octet-stream"); +} + void JFJochBrokerHttp::config_bragg_integration_get(httplib::Response &response) { ProcessOutput(Convert(state_machine.GetBraggIntegrationSettings()), response); } diff --git a/broker/JFJochBrokerHttp.h b/broker/JFJochBrokerHttp.h index 3528ab5d3..a5ff26c41 100644 --- a/broker/JFJochBrokerHttp.h +++ b/broker/JFJochBrokerHttp.h @@ -206,6 +206,8 @@ class JFJochBrokerHttp { void config_rugnux_model_put(const httplib::Request &request, httplib::Response &response); void config_rugnux_model_delete(httplib::Response &response); void rugnux_status_get(httplib::Response &response); + void rugnux_model_cif_get(httplib::Response &response); + void rugnux_map_ccp4_get(const std::optional &type, httplib::Response &response); void config_bragg_integration_get(httplib::Response &response); void config_bragg_integration_put(const org::openapitools::server::model::Bragg_integration_settings &braggIntegrationSettings, httplib::Response &response); diff --git a/broker/OpenAPIConvert.cpp b/broker/OpenAPIConvert.cpp index c97040847..53168776a 100644 --- a/broker/OpenAPIConvert.cpp +++ b/broker/OpenAPIConvert.cpp @@ -3,6 +3,7 @@ #include "../common/JFJochMath.h" #include "OpenAPIConvert.h" +#include "../receiver/RugnuxStructure.h" // From https://en.cppreference.com/w/cpp/string/byte/tolower std::string str_tolower(std::string s) { @@ -1276,6 +1277,8 @@ org::openapitools::server::model::Rugnux_live_status Convert(const RugnuxLiveSna } ret.setSummary(summary); ret.setWarnings(input.warnings); + ret.setPlacedModel(input.structure != nullptr); + ret.setAnomalousMap(input.structure != nullptr && input.structure->HasAnomalousMap()); return ret; } diff --git a/broker/gen/model/Rugnux_live_status.cpp b/broker/gen/model/Rugnux_live_status.cpp index df6c689da..b7abe100b 100644 --- a/broker/gen/model/Rugnux_live_status.cpp +++ b/broker/gen/model/Rugnux_live_status.cpp @@ -22,6 +22,8 @@ namespace org::openapitools::server::model Rugnux_live_status::Rugnux_live_status() { m_State = ""; + m_Placed_model = false; + m_Anomalous_map = false; m_Probe = ""; m_Probe_s = 0.0; m_Probe_sIsSet = false; @@ -67,7 +69,7 @@ bool Rugnux_live_status::validate(std::stringstream& msg, const std::string& pat bool success = true; const std::string _pathPrefix = pathPrefix.empty() ? "Rugnux_live_status" : pathPrefix; - + if (stepProgressIsSet()) { const float& value = m_Step_progress; @@ -161,6 +163,12 @@ bool Rugnux_live_status::operator==(const Rugnux_live_status& rhs) const (getState() == rhs.getState()) && + (isPlacedModel() == rhs.isPlacedModel()) + && + + (isAnomalousMap() == rhs.isAnomalousMap()) + && + (getProbe() == rhs.getProbe()) && @@ -227,6 +235,8 @@ void to_json(nlohmann::json& j, const Rugnux_live_status& o) { j = nlohmann::json::object(); j["state"] = o.m_State; + j["placed_model"] = o.m_Placed_model; + j["anomalous_map"] = o.m_Anomalous_map; j["probe"] = o.m_Probe; if(o.probeSIsSet()) j["probe_s"] = o.m_Probe_s; @@ -259,6 +269,8 @@ void to_json(nlohmann::json& j, const Rugnux_live_status& o) void from_json(const nlohmann::json& j, Rugnux_live_status& o) { j.at("state").get_to(o.m_State); + j.at("placed_model").get_to(o.m_Placed_model); + j.at("anomalous_map").get_to(o.m_Anomalous_map); j.at("probe").get_to(o.m_Probe); if(j.find("probe_s") != j.end()) { @@ -320,6 +332,22 @@ void Rugnux_live_status::setState(std::string const& value) { m_State = value; } +bool Rugnux_live_status::isPlacedModel() const +{ + return m_Placed_model; +} +void Rugnux_live_status::setPlacedModel(bool const value) +{ + m_Placed_model = value; +} +bool Rugnux_live_status::isAnomalousMap() const +{ + return m_Anomalous_map; +} +void Rugnux_live_status::setAnomalousMap(bool const value) +{ + m_Anomalous_map = value; +} std::string Rugnux_live_status::getProbe() const { return m_Probe; diff --git a/broker/gen/model/Rugnux_live_status.h b/broker/gen/model/Rugnux_live_status.h index e70814cee..a1d8ba05e 100644 --- a/broker/gen/model/Rugnux_live_status.h +++ b/broker/gen/model/Rugnux_live_status.h @@ -67,6 +67,16 @@ public: std::string getState() const; void setState(std::string const& value); /// + /// The run placed the model: it can be fetched from `/rugnux/model.cif` and its maps from `/rugnux/map.ccp4`. + /// + bool isPlacedModel() const; + void setPlacedModel(bool const value); + /// + /// The merge kept the Bijvoet pairs, so with the placed model there is an anomalous difference map (`/rugnux/map.ccp4?type=anom`). + /// + bool isAnomalousMap() const; + void setAnomalousMap(bool const value); + /// /// Answer of the early lattice probe on the first 60 degrees; empty until it answers /// std::string getProbe() const; @@ -178,6 +188,10 @@ public: protected: std::string m_State; + bool m_Placed_model; + + bool m_Anomalous_map; + std::string m_Probe; double m_Probe_s; diff --git a/broker/jfjoch_api.yaml b/broker/jfjoch_api.yaml index 4f45de5e6..305834ae3 100644 --- a/broker/jfjoch_api.yaml +++ b/broker/jfjoch_api.yaml @@ -2785,10 +2785,22 @@ components: - summary - warnings - probe + - placed_model + - anomalous_map properties: state: type: string enum: [Idle, Running, Done, Failed, Cancelled, Skipped] + placed_model: + type: boolean + description: | + The run placed the model: it can be fetched from `/rugnux/model.cif` and its maps from + `/rugnux/map.ccp4`. + anomalous_map: + type: boolean + description: | + The merge kept the Bijvoet pairs, so with the placed model there is an anomalous difference map + (`/rugnux/map.ccp4?type=anom`). probe: type: string description: Answer of the early lattice probe on the first 60 degrees; empty until it answers @@ -3605,6 +3617,75 @@ paths: application/json: schema: $ref: '#/components/schemas/rugnux_live_status' + /rugnux/model.cif: + get: + summary: Get the model as placed by the last rugnux live run + description: | + The model placed against the data by molecular replacement or validation, as mmCIF - the same text as the + `_model.cif` file written beside the data: the data's cell and space group, with the input model's own + metadata (entities, sequences, references, authors, software) kept. Kept as long as `/rugnux/status`. + security: + - bearerAuth: [] + responses: + "200": + description: Placed model + content: + chemical/x-mmcif: + schema: + type: string + format: binary + "401": + description: The current dataset is protected and no valid bearer token was presented. + content: + text/plain: + schema: + type: string + "404": + description: The last rugnux live run placed no model + /rugnux/map.ccp4: + get: + summary: Get an electron-density map of the last rugnux live run + description: | + The map of the placed model (`/rugnux/model.cif`) in CCP4 format (mode 2, float), made from the map + coefficients written to `_maps.mtz` (FWT/PHWT, DELFWT/PHDELWT, ANOM/PHANOM). It covers one unit cell centred on the model - along an axis where + the model reaches further than the cell, the model plus 5 A instead - sampled every d_min/3; the mean and + RMS in its header are those of the unit cell, so contour levels in sigma are the usual ones. + parameters: + - in: query + name: type + required: false + schema: + type: string + enum: [2fofc, fofc, anom] + default: 2fofc + description: | + `2fofc`: the 2mFo-DFc map; `fofc`: the mFo-DFc difference map; `anom`: the anomalous difference map, + (F(+) - F(-)) on the model phases less 90 degrees, where the merge kept the Bijvoet pairs (`anomalous_map` + in `/rugnux/status`) + security: + - bearerAuth: [] + responses: + "200": + description: Map + content: + application/octet-stream: + schema: + type: string + format: binary + "400": + description: Unknown map type + content: + text/plain: + schema: + type: string + "401": + description: The current dataset is protected and no valid bearer token was presented. + content: + text/plain: + schema: + type: string + "404": + description: The last rugnux live run placed no model or made no maps, or (`anom`) has no Bijvoet pairs /config/zeromq_preview: put: summary: Set ZeroMQ preview settings diff --git a/broker/redoc-static.html b/broker/redoc-static.html index bb3abf9d2..7e9320b0e 100644 --- a/broker/redoc-static.html +++ b/broker/redoc-static.html @@ -394,7 +394,7 @@ data-styled.g138[id="sc-dxcDKg"]{content:"eajCCh,"}/*!sc*/ -

Set ZeroMQ preview settings

Jungfraujoch can generate preview message stream on ZeroMQ SUB socket. +

http://localhost:5232/rugnux/status

Response samples

Content type
application/json
{
  • "state": "Idle",
  • "placed_model": true,
  • "anomalous_map": true,
  • "probe": "string",
  • "probe_s": 0.1,
  • "start_to_first_image_s": 0.1,
  • "message": "string",
  • "images_expected": 0,
  • "expected_collection_s": 0.1,
  • "images_received": 0,
  • "elapsed_s": 0.1,
  • "step_progress": 1,
  • "steps": [
    ],
  • "last_image_s": 0.1,
  • "cut": "string",
  • "cut_s": 0.1,
  • "last_image_to_written_s": 0.1,
  • "verdict": "string",
  • "verdict_text": "string",
  • "summary": [
    ],
  • "warnings": [
    ]
}

Get the model as placed by the last rugnux live run

The model placed against the data by molecular replacement or validation, as mmCIF - the same text as the +_model.cif file written beside the data: the data's cell and space group, with the input model's own +metadata (entities, sequences, references, authors, software) kept. Kept as long as /rugnux/status.

+
Authorizations:
bearerAuth

Responses

Get an electron-density map of the last rugnux live run

The map of the placed model (/rugnux/model.cif) in CCP4 format (mode 2, float), made from the map +coefficients written to _maps.mtz (FWT/PHWT, DELFWT/PHDELWT, ANOM/PHANOM). It covers one unit cell centred on the model - along an axis where +the model reaches further than the cell, the model plus 5 A instead - sampled every d_min/3; the mean and +RMS in its header are those of the unit cell, so contour levels in sigma are the usual ones.

+
Authorizations:
bearerAuth
query Parameters
type
string
Default: "2fofc"
Enum: "2fofc" "fofc" "anom"

2fofc: the 2mFo-DFc map; fofc: the mFo-DFc difference map; anom: the anomalous difference map, +(F(+) - F(-)) on the model phases less 90 degrees, where the merge kept the Bijvoet pairs (anomalous_map +in /rugnux/status)

+

Responses

Set ZeroMQ preview settings

Jungfraujoch can generate preview message stream on ZeroMQ SUB socket. Here settings of the socket can be adjusted. While the data structure contains also socket_address, this cannot be changed via HTTP and is ignore in PUT request. Options set with this PUT request have no effect on HTTP based preview.

@@ -880,9 +899,9 @@ Address follows ZeroMQ convention for sockets - in practice ipc://

Request samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get ZeroMQ preview settings

Responses

Request samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get ZeroMQ preview settings

Responses

Response samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Set ZeroMQ metadata settings

Jungfraujoch can generate metadata message stream on ZeroMQ PUB socket. This stream covers all images. +

http://localhost:5232/config/zeromq_preview

Response samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Set ZeroMQ metadata settings

Jungfraujoch can generate metadata message stream on ZeroMQ PUB socket. This stream covers all images. Here settings of the socket can be adjusted. While the data structure contains also socket_address, this cannot be changed via HTTP and is ignore in PUT request.

Request Body schema: application/json
enabled
required
boolean
Default: true

ZeroMQ metadata socket is enabled.

@@ -895,9 +914,9 @@ Address follows ZeroMQ convention for sockets - in practice ipc://

Request samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get ZeroMQ metadata socket settings

Responses

Request samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get ZeroMQ metadata socket settings

Responses

Response samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Set configuration for dark data collection to calculate mask

This is only possible when operating DECTRIS detectors at the moment; it will be also available for PSI EIGER at some point. +

http://localhost:5232/config/zeromq_metadata

Response samples

Content type
application/json
{
  • "enabled": true,
  • "period_ms": 1000,
  • "socket_address": "string"
}

Set configuration for dark data collection to calculate mask

This is only possible when operating DECTRIS detectors at the moment; it will be also available for PSI EIGER at some point. This can only be done when detector is Idle, Error or Inactive states.

Request Body schema: application/json
detector_threshold_keV
required
number <float> [ 2.5 .. 100 ]
Default: 3.5

Energy threshold for dark image collection

frame_time_us
required
integer <int64> [ 500 .. 100000 ]
Default: 10000

Time between frames for dark image collection

@@ -908,51 +927,51 @@ This can only be done when detector is Idle, Error or

Request samples

Content type
application/json
{
  • "detector_threshold_keV": 3.5,
  • "frame_time_us": 10000,
  • "number_of_frames": 1000,
  • "max_allowed_pixel_count": 1,
  • "max_frames_with_signal": 10
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get settings for dark data collection to calculate mask

Responses

Request samples

Content type
application/json
{
  • "detector_threshold_keV": 3.5,
  • "frame_time_us": 10000,
  • "number_of_frames": 1000,
  • "max_allowed_pixel_count": 1,
  • "max_frames_with_signal": 10
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get settings for dark data collection to calculate mask

Responses

Response samples

Content type
application/json
{
  • "detector_threshold_keV": 3.5,
  • "frame_time_us": 10000,
  • "number_of_frames": 1000,
  • "max_allowed_pixel_count": 1,
  • "max_frames_with_signal": 10
}

Get Jungfraujoch status

Status of the data acquisition

+
http://localhost:5232/config/dark_mask

Response samples

Content type
application/json
{
  • "detector_threshold_keV": 3.5,
  • "frame_time_us": 10000,
  • "number_of_frames": 1000,
  • "max_allowed_pixel_count": 1,
  • "max_frames_with_signal": 10
}

Get Jungfraujoch status

Status of the data acquisition

Responses

Response samples

Content type
application/json
{
  • "state": "Inactive",
  • "progress": 1,
  • "message": "string",
  • "message_severity": "success",
  • "gpu_count": 0,
  • "broker_version": "1.0.0-rc.128"
}

Get status of FPGA devices

Responses

Response samples

Content type
application/json
{
  • "state": "Inactive",
  • "progress": 1,
  • "message": "string",
  • "message_severity": "success",
  • "gpu_count": 0,
  • "broker_version": "1.0.0-rc.128"
}

Get status of FPGA devices

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Return XFEL pulse IDs for the current data acquisition

Return array of XFEL pulse IDs - (-1) if image not recorded

+
http://localhost:5232/fpga_status

Response samples

Content type
application/json
[
  • {
    }
]

Return XFEL pulse IDs for the current data acquisition

Return array of XFEL pulse IDs - (-1) if image not recorded

Responses

Response samples

Content type
application/json
[
  • 0
]

Return XFEL event codes for the current data acquisition

Return array of XFEL event codes

+
http://localhost:5232/xfel/pulse_id

Response samples

Content type
application/json
[
  • 0
]

Return XFEL event codes for the current data acquisition

Return array of XFEL event codes

Responses

Response samples

Content type
application/json
[
  • 0
]

Get status of image pusher

Responses

Response samples

Content type
application/json
[
  • 0
]

Get status of image pusher

Responses

Response samples

Content type
application/json
{
  • "pusher_type": "ZeroMQ",
  • "addr": [
    ],
  • "connected_writers": 0,
  • "images_written": 0,
  • "images_write_error": 0,
  • "writer_fifo_utilization": [
    ]
}

Get detector status

Status of the JUNGFRAU detector

+
http://localhost:5232/image_pusher/status

Response samples

Content type
application/json
{
  • "pusher_type": "ZeroMQ",
  • "addr": [
    ],
  • "connected_writers": 0,
  • "images_written": 0,
  • "images_write_error": 0,
  • "writer_fifo_utilization": [
    ]
}

Get detector status

Status of the JUNGFRAU detector

Responses

Response samples

Content type
application/json
{
  • "state": "Idle",
  • "powerchip": "PowerOn",
  • "server_version": "string",
  • "number_of_triggers_left": 0,
  • "fpga_temp_degC": [
    ],
  • "high_voltage_V": [
    ]
}

Get ROI definitions

Responses

Response samples

Content type
application/json
{
  • "state": "Idle",
  • "powerchip": "PowerOn",
  • "server_version": "string",
  • "number_of_triggers_left": 0,
  • "fpga_temp_degC": [
    ],
  • "high_voltage_V": [
    ]
}

Get ROI definitions

Responses

Response samples

Content type
application/json
{
  • "box": {
    },
  • "circle": {
    },
  • "azim": {
    }
}

Upload ROI definitions

Request Body schema: application/json
required
object (roi_box_list)

List of box ROIs

+
http://localhost:5232/config/roi

Response samples

Content type
application/json
{
  • "box": {
    },
  • "circle": {
    },
  • "azim": {
    }
}

Upload ROI definitions

Request Body schema: application/json
required
object (roi_box_list)

List of box ROIs

required
object (roi_circle_list)

List of circular ROIs

required
object (roi_azim_list)

List of azimuthal ROIs

Responses

Request samples

Content type
application/json
{
  • "box": {
    },
  • "circle": {
    },
  • "azim": {
    }
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get general statistics

measurement is omitted unless the request carries a valid bearer token for a protected +

http://localhost:5232/config/roi

Request samples

Content type
application/json
{
  • "box": {
    },
  • "circle": {
    },
  • "azim": {
    }
}

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get general statistics

measurement is omitted unless the request carries a valid bearer token for a protected dataset (see tokens in /start).

Responses

Response samples

Content type
application/json
{
  • "detector": {
    },
  • "detector_list": {
    },
  • "detector_settings": {
    },
  • "image_format_settings": {
    },
  • "instrument_metadata": {
    },
  • "file_writer_settings": {
    },
  • "data_processing_settings": {
    },
  • "measurement": {
    },
  • "broker": {
    },
  • "fpga": [
    ],
  • "calibration": [
    ],
  • "zeromq_preview": {
    },
  • "zeromq_metadata": {
    },
  • "dark_mask": {
    },
  • "pixel_mask": {
    },
  • "roi": {
    },
  • "az_int": {
    },
  • "buffer": {
    },
  • "indexing": {
    },
  • "bragg_integration": {
    },
  • "image_pusher": {
    }
}

Get data collection statistics

Results of the last data collection (dataset name, unit cell, ...)

+
http://localhost:5232/statistics

Response samples

Content type
application/json
{
  • "detector": {
    },
  • "detector_list": {
    },
  • "detector_settings": {
    },
  • "image_format_settings": {
    },
  • "instrument_metadata": {
    },
  • "file_writer_settings": {
    },
  • "data_processing_settings": {
    },
  • "measurement": {
    },
  • "broker": {
    },
  • "fpga": [
    ],
  • "calibration": [
    ],
  • "zeromq_preview": {
    },
  • "zeromq_metadata": {
    },
  • "dark_mask": {
    },
  • "pixel_mask": {
    },
  • "roi": {
    },
  • "az_int": {
    },
  • "buffer": {
    },
  • "indexing": {
    },
  • "bragg_integration": {
    },
  • "image_pusher": {
    }
}

Get data collection statistics

Results of the last data collection (dataset name, unit cell, ...)

Authorizations:
bearerAuth

Responses

Response samples

Content type
application/json
{
  • "file_prefix": "string",
  • "run_number": 0,
  • "experiment_group": "string",
  • "images_expected": 0,
  • "images_collected": 0,
  • "images_sent": 0,
  • "images_written": 0,
  • "images_discarded_lossy_compression": 0,
  • "max_image_number_sent": 0,
  • "collection_efficiency": 1,
  • "compression_ratio": 5.3,
  • "cancelled": true,
  • "max_receiver_delay": 0,
  • "indexing_rate": 0.1,
  • "detector_width": 0,
  • "detector_height": 0,
  • "detector_pixel_depth": 2,
  • "bkg_estimate": 0.1,
  • "unit_cell": "string",
  • "error_pixels": 0.1,
  • "saturated_pixels": 0.1,
  • "roi_beam_pixels": 0.1,
  • "roi_beam_sum": 0.1
}

Get calibration statistics

Statistics are provided for each module/storage cell separately

+
http://localhost:5232/statistics/data_collection

Response samples

Content type
application/json
{
  • "file_prefix": "string",
  • "run_number": 0,
  • "experiment_group": "string",
  • "images_expected": 0,
  • "images_collected": 0,
  • "images_sent": 0,
  • "images_written": 0,
  • "images_discarded_lossy_compression": 0,
  • "max_image_number_sent": 0,
  • "collection_efficiency": 1,
  • "compression_ratio": 5.3,
  • "cancelled": true,
  • "max_receiver_delay": 0,
  • "indexing_rate": 0.1,
  • "detector_width": 0,
  • "detector_height": 0,
  • "detector_pixel_depth": 2,
  • "bkg_estimate": 0.1,
  • "unit_cell": "string",
  • "error_pixels": 0.1,
  • "saturated_pixels": 0.1,
  • "roi_beam_pixels": 0.1,
  • "roi_beam_sum": 0.1
}

Get calibration statistics

Statistics are provided for each module/storage cell separately

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get mask of the detector (binary)

Detector must be Initialized. +

http://localhost:5232/statistics/calibration

Response samples

Content type
application/json
[
  • {
    }
]

Get mask of the detector (binary)

Detector must be Initialized. Get full pixel mask of the detector. See NXmx standard for meaning of pixel values.

Responses

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get mask of the detector (TIFF)

Should be in Idle state. +

http://localhost:5232/config/user_mask

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get mask of the detector (TIFF)

Should be in Idle state. Get full pixel mask of the detector See NXmx standard for meaning of pixel values

Responses

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get pedestal in TIFF format

query Parameters
gain_level
required
integer

Gain level (0, 1, 2)

+
http://localhost:5232/config/user_mask.tiff

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get pedestal in TIFF format

query Parameters
gain_level
required
integer

Gain level (0, 1, 2)

sc
integer

Storage cell number

Responses

Response samples

Content type
application/json
{
  • "title": "string",
  • "unit_x": "image_number",
  • "size_x": 0.1,
  • "size_y": 0.1,
  • "plot": [
    ]
}

Generate 1D plot from Jungfraujoch and send in raw binary format. +

http://localhost:5232/preview/plot

Response samples

Content type
application/json
{
  • "title": "string",
  • "unit_x": "image_number",
  • "size_x": 0.1,
  • "size_y": 0.1,
  • "plot": [
    ]
}

Generate 1D plot from Jungfraujoch and send in raw binary format. Data are provided as (32-bit) float binary array. This format doesn't transmit information about X-axis, only values, so it is of limited use for azimuthal integration.

Authorizations:
bearerAuth
query Parameters
type
required
string
Enum: "bkg_estimate" "spindle_blind_fraction" "azint" "azint_1d" "spot_count" "spot_count_low_res" "spot_count_indexed" "spot_count_ice" "indexing_rate" "indexing_lattice_count" "indexing_unit_cell_length" "indexing_unit_cell_angle" "profile_radius" "mosaicity" "b_factor" "error_pixels" "saturated_pixels" "image_collection_efficiency" "receiver_delay" "receiver_free_send_buf" "strong_pixels" "roi_sum" "roi_mean" "roi_max_count" "roi_pixels" "roi_weighted_x" "roi_weighted_y" "packets_received" "max_pixel_value" "resolution_estimate" "pixel_sum" "processing_time" "beam_center_x" "beam_center_y" "integrated_reflections" "image_scale_factor" "image_scale_cc" "compression_ratio" "ice_ring_score"

Type of requested plot

@@ -1026,13 +1045,13 @@ This format doesn't transmit information about X-axis, only values, so it i

Response samples

Content type
application/json
{
  • "file_prefix": "string",
  • "rotation_unit_cell": {
    },
  • "rotation_crystal_lattice": [
    ],
  • "rotation_bravais": "string",
  • "images": [
    ]
}

Get Start message in CBOR format

Contains metadata for a dataset (e.g., experimental geometry)

+
http://localhost:5232/result/scan

Response samples

Content type
application/json
{
  • "file_prefix": "string",
  • "rotation_unit_cell": {
    },
  • "rotation_crystal_lattice": [
    ],
  • "rotation_bravais": "string",
  • "images": [
    ]
}

Get Start message in CBOR format

Contains metadata for a dataset (e.g., experimental geometry)

Authorizations:
bearerAuth

Responses

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Event stream of the collection, for a reader following it

Server-sent events (text/event-stream) for the current collection, from the moment /start has +

http://localhost:5232/image_buffer/start.cbor

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Event stream of the collection, for a reader following it

Server-sent events (text/event-stream) for the current collection, from the moment /start has accepted it: first its whole history, then every change, until it ends or a new collection begins. Only one reader at a time. Events, each with a JSON data line:

    @@ -1060,7 +1079,7 @@ A comment line (: keepalive) every 10 s when nothing happens.

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in JPEG format using custom settings

Authorizations:
bearerAuth
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

+
http://localhost:5232/image_buffer/image.cbor

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in JPEG format using custom settings

Authorizations:
bearerAuth
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

show_user_mask
boolean
Default: false

Show user mask

show_roi
boolean
Default: false

Show ROI areas on the image

show_spots
boolean
Default: true

Show spot finding results on the image

@@ -1077,7 +1096,7 @@ A comment line (: keepalive) every 10 s when nothing happens.

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in TIFF format

Authorizations:
bearerAuth
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

+
http://localhost:5232/image_buffer/image.jpeg

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get preview image in TIFF format

Authorizations:
bearerAuth
query Parameters
id
integer <int64> >= -2
Default: -1

Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer

Responses

Responses

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get status of the image buffers

Can be run at any stage of Jungfraujoch operation, including during data collection. +

http://localhost:5232/image_buffer/clear

Response samples

Content type
application/json
{
  • "msg": "Detector in wrong state",
  • "reason": "WrongDAQState"
}

Get status of the image buffers

Can be run at any stage of Jungfraujoch operation, including during data collection. The status of the image buffer is volatile during data collection - if data collection goes for more images than available buffer slots, then image might be replaced in the buffer between calling /images and /image.cbor.

Responses

Response samples

Content type
application/json
{
  • "min_image_number": 0,
  • "max_image_number": 0,
  • "image_numbers": [
    ],
  • "total_slots": 0,
  • "available_slots": 0,
  • "in_preparation_slots": 0,
  • "in_sending_slots": 0,
  • "current_counter": 0
}

Get Jungfraujoch version of jfjoch_broker

Responses

Response samples

Content type
application/json
{
  • "min_image_number": 0,
  • "max_image_number": 0,
  • "image_numbers": [
    ],
  • "total_slots": 0,
  • "available_slots": 0,
  • "in_preparation_slots": 0,
  • "in_sending_slots": 0,
  • "current_counter": 0
}

Get Jungfraujoch version of jfjoch_broker

Responses