broker: accept adaptive_threshold on FPGA spot-finding detectors and ignore it

PUT /config/spot_finding no longer answers 400 when adaptive_threshold is
set on a JUNGFRAU or EIGER broker. Those workflows find spots on the FPGA at
its own fixed threshold, so the setting is stored and simply not used there;
the DECTRIS/SIMPLON workflow keeps using it. This matches what the config file
already did and lets one set of spot-finding settings be shared across
detectors. The API description, its generated C++ model, frontend client,
Python client docs and Redoc page, and the image-analysis doc now say the
setting is ignored rather than refused. A state-machine test checks the
setting is accepted on a JUNGFRAU configuration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UqXYdL4qBGYRiSXSGXfCLu

broker/JFJochStateMachine.cpp

@@ -790,15 +790,6 @@ void JFJochStateMachine::GetPlotRaw(std::vector<float> &v, PlotType type, const
 void JFJochStateMachine::SetSpotFindingSettings(const SpotFindingSettings &settings) {
     std::unique_lock ul(data_processing_settings_mutex);
     DiffractionExperiment::CheckDataProcessingSettings(settings);
-    // The adaptive threshold is a property of the software spot finder, which only the DECTRIS
-    // (SIMPLON) workflow runs - JUNGFRAU and EIGER find spots on the FPGA, at its own fixed
-    // threshold. Refuse it there rather than accept it and do nothing: a silently ignored detection
-    // setting is indistinguishable from one that had no effect on the data.
-    if (settings.adaptive_threshold && experiment.GetDetectorType() != DetectorType::DECTRIS)
-        throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
-                              "Adaptive spot-finding threshold is not available on this detector: spots "
-                              "are found on the FPGA, which applies its own fixed threshold");
-
     data_processing_settings = settings;
 
     services.SetSpotFindingSettings(data_processing_settings);

broker/gen/model/Spot_finding_settings.h

@@ -124,7 +124,7 @@ public:
     bool highResGapQRecipAIsSet() const;
     void unsetHigh_res_gap_Q_recipA();
     /// <summary>
-    /// Self-calibrating spot detection: replace the fixed photon_count_threshold by a per-resolution-ring threshold derived from each image&#39;s own noise, so the same setting works across datasets without per-dataset tuning. photon_count_threshold is then ignored and false_pixels_per_frame sets the operating point instead. Only available on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow). The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold, so enabling this there is rejected rather than silently ignored. 
+    /// Self-calibrating spot detection: replace the fixed photon_count_threshold by a per-resolution-ring threshold derived from each image&#39;s own noise, so the same setting works across datasets without per-dataset tuning. photon_count_threshold is then ignored and false_pixels_per_frame sets the operating point instead. Used only on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow). The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold; there the setting is accepted and ignored, so the same settings can be shared across detectors. 
     /// </summary>
     bool isAdaptiveThreshold() const;
     void setAdaptiveThreshold(bool const value);

broker/jfjoch_api.yaml

@@ -1180,9 +1180,9 @@ components:
             threshold derived from each image's own noise, so the same setting works across datasets without
             per-dataset tuning. photon_count_threshold is then ignored and false_pixels_per_frame sets the
             operating point instead.
-            Only available on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow).
-            The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold,
-            so enabling this there is rejected rather than silently ignored.
+            Used only on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow).
+            The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold;
+            there the setting is accepted and ignored, so the same settings can be shared across detectors.
         false_pixels_per_frame:
           type: number
           format: float

broker/redoc-static.html

@@ -791,9 +791,9 @@ This option should be turned OFF for small molecule datasets or for crystals wit
 threshold derived from each image's own noise, so the same setting works across datasets without
 per-dataset tuning. photon_count_threshold is then ignored and false_pixels_per_frame sets the
 operating point instead.
-Only available on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow).
-The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold,
-so enabling this there is rejected rather than silently ignored.</p>
+Used only on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow).
+The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold;
+there the setting is accepted and ignored, so the same settings can be shared across detectors.</p>
 </div></div></div></td></tr><tr class="last "><td class="sc-dLMFU sc-eldPxv hZbWKe jXkfaY" kind="field" title="false_pixels_per_frame"><span class="sc-gFqAkR eRqgPI"></span><span class="property-name">false_pixels_per_frame</span></td><td class="sc-fPXMVe bAXEAa"><div><div><span class="sc-Nxspf sc-cfxfcM cXxAXt jrGEaR"></span><span class="sc-Nxspf sc-gFAWRd cXxAXt dYzdDr">number</span><span class="sc-Nxspf sc-gFAWRd cXxAXt dYzdDr"> <!-- -->&lt;<!-- -->float<!-- -->&gt;<!-- --> </span><span> <span class="sc-Nxspf sc-fxwrCY cXxAXt eMVOei"> <!-- -->[ 1 .. 100000 ]<!-- --> </span></span></div><div><span class="sc-Nxspf cXxAXt"> <!-- -->Default:<!-- --> </span> <span class="sc-Nxspf sc-ddjGPC cXxAXt jZOCAW">100</span></div> <div><div class="sc-eeDRCY sc-eBMEME gTGgei fbXBig"><p>Operating point of the adaptive threshold: the number of noise pixels tolerated per frame.
 About 100 suits a multi-megapixel detector. Ignored unless adaptive_threshold is set.</p>
 </div></div></div></td></tr></tbody></table><div><h3 class="sc-fMMURN egoxtT">Responses</h3><div><button class="sc-kzqdkY jkjMMR" disabled=""><strong class="sc-dSIIpw bpEYcE">200<!-- --> </strong><div class="sc-eeDRCY sc-eBMEME sc-dCFHLb gTGgei kekOtA kCaLQd"><p>Everything OK</p>
@@ -1131,7 +1131,7 @@ then image might be replaced in the buffer between calling /images and /image.cb
 </div></button></div></div></div><div class="sc-iGgWBj sc-gsFSXq ibAZSi eDeKjd"><div class="sc-ejfMa-d bNUPQg"><button class="sc-EgOXT cFEcOl"><span type="get" class="sc-eZYNyq qGfV http-verb get">get</span><span class="sc-iEXKAA gUNSDu">/version</span><svg class="sc-cwHptR iVFMLx" style="margin-right:-25px" version="1.1" viewBox="0 0 24 24" x="0" xmlns="http://www.w3.org/2000/svg" y="0" aria-hidden="true"><polygon points="17.3 8.3 12 13.6 6.7 8.3 5.3 9.7 12 16.4 18.7 9.7 "></polygon></svg></button><div aria-hidden="true" class="sc-dlWCHZ kWacYr"><div class="sc-hHOBiw cndTNO"><div class="sc-eeDRCY sc-eBMEME gTGgei fbXBig"><p>Test Jungfraujoch system</p>
 </div><div tabindex="0" role="button"><div class="sc-kWtpeL gEncxT"><span>http://localhost:5232</span>/version</div></div></div></div></div></div></div></div></div><div class="sc-dExYaf gZbfaQ"></div></div></div>
       <script>
-      const __redoc_state = {"menu":{"activeItemIdx":-1},"spec":{"data":{"openapi":"3.0.3","info":{"title":"Jungfraujoch","description":"API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland).\nJungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU.\nJungfraujoch uses FPGA boards to acquire data at high data rates.\n\n# License Clarification\n\nWhile this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply**\nwhen this file is used solely to generate OpenAPI clients or when implementing applications that\ninteract with the API. Generated client code and applications using this API definition are not\nsubject to the GPL license requirements and may be distributed under terms of your choosing.\n\nThis exception is similar in spirit to the Linux Kernel's approach to userspace API headers and\nthe GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that\nuser programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not\nderivative works of the kernel and are not subject to the terms of the GPL.\n\nThis exception is intended to allow wider use of this API specification without imposing GPL\nrequirements on applications that merely interact with the API, regardless of whether they\ncommunicate through network calls or other mechanisms.\n","version":"1.0.0-rc.183","contact":{"name":"Filip Leonarski (Paul Scherrer Institute)","email":"filip.leonarski@psi.ch"},"license":{"name":"GPL-3.0","url":"https://www.gnu.org/licenses/gpl-3.0.html"}},"servers":[{"url":"http://localhost:5232","description":"Test Jungfraujoch system"}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","description":"Per-dataset secret. `/start` may carry one or more equivalent `tokens`; while the current\ndataset has any, the operations marked with this scheme require `Authorization: Bearer <token>`\nand answer 401 otherwise. A dataset started without `tokens` is not protected and the header is ignored.\n"}},"parameters":{"binning":{"in":"query","name":"binning","required":false,"schema":{"default":1,"type":"integer"},"description":"Binning of frames for the plot (0 = default binning)"},"experimental_coord":{"in":"query","name":"experimental_coord","required":false,"schema":{"type":"boolean","default":false},"description":"If measurement has goniometer axis defined, plot X-axis will represent rotation angle\nIf measurement has grid scan defined, plot X-axis and Y-axis will represent grid position, Z will be used as the final value\nFor still measurement the number is ignored\n"},"azint_unit":{"in":"query","name":"azint_unit","required":false,"schema":{"type":"string","enum":["Q_recipA","d_A","two_theta_deg"],"default":"Q_recipA"},"description":"Unit used for azim int.\n"},"fill_value":{"in":"query","name":"fill","required":false,"schema":{"type":"number","format":"float"},"description":"Fill value for elements that were missed during data collection\n"},"plot_type":{"in":"query","name":"type","required":true,"description":"Type of requested plot","schema":{"type":"string","enum":["bkg_estimate","spindle_blind_fraction","azint","azint_1d","spot_count","spot_count_low_res","spot_count_indexed","spot_count_ice","indexing_rate","indexing_lattice_count","indexing_unit_cell_length","indexing_unit_cell_angle","profile_radius","mosaicity","b_factor","error_pixels","saturated_pixels","image_collection_efficiency","receiver_delay","receiver_free_send_buf","strong_pixels","roi_sum","roi_mean","roi_max_count","roi_pixels","roi_weighted_x","roi_weighted_y","packets_received","max_pixel_value","resolution_estimate","pixel_sum","processing_time","beam_center_x","beam_center_y","integrated_reflections","image_scale_factor","image_scale_cc","compression_ratio","ice_ring_score"]}},"roi":{"in":"query","name":"roi","required":false,"schema":{"type":"string","minLength":1},"description":"Name of ROI for which plot is requested"},"image_id":{"in":"query","name":"id","required":false,"schema":{"type":"integer","format":"int64","default":-1,"minimum":-2},"description":"Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer"},"saturation":{"in":"query","name":"saturation","required":false,"schema":{"type":"number","format":"float","minimum":-32767,"maximum":32767},"description":"Saturation value to set contrast in the preview image; if not provided, then autocontrast procedure is used"},"background":{"in":"query","name":"background","required":false,"schema":{"type":"number","format":"float","default":0,"minimum":-32767,"maximum":32767},"description":"Background value to set contrast in the preview image"},"show_spots":{"in":"query","name":"show_spots","schema":{"type":"boolean","default":true},"description":"Show spot finding results on the image"},"show_predictions":{"in":"query","name":"show_predictions","schema":Line truncated
+      const __redoc_state = {"menu":{"activeItemIdx":-1},"spec":{"data":{"openapi":"3.0.3","info":{"title":"Jungfraujoch","description":"API to control Jungfraujoch developed by the Paul Scherrer Institute (Switzerland).\nJungfraujoch is a data acquisition and analysis system for pixel array detectors, primarly PSI JUNGFRAU.\nJungfraujoch uses FPGA boards to acquire data at high data rates.\n\n# License Clarification\n\nWhile this API definition is licensed under GPL-3.0, **the GPL copyleft provisions do not apply**\nwhen this file is used solely to generate OpenAPI clients or when implementing applications that\ninteract with the API. Generated client code and applications using this API definition are not\nsubject to the GPL license requirements and may be distributed under terms of your choosing.\n\nThis exception is similar in spirit to the Linux Kernel's approach to userspace API headers and\nthe GCC Runtime Library Exception. The Linux Kernel developers have explicitly stated that\nuser programs that merely use the kernel interfaces (syscalls, ioctl definitions, etc.) are not\nderivative works of the kernel and are not subject to the terms of the GPL.\n\nThis exception is intended to allow wider use of this API specification without imposing GPL\nrequirements on applications that merely interact with the API, regardless of whether they\ncommunicate through network calls or other mechanisms.\n","version":"1.0.0-rc.183","contact":{"name":"Filip Leonarski (Paul Scherrer Institute)","email":"filip.leonarski@psi.ch"},"license":{"name":"GPL-3.0","url":"https://www.gnu.org/licenses/gpl-3.0.html"}},"servers":[{"url":"http://localhost:5232","description":"Test Jungfraujoch system"}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","description":"Per-dataset secret. `/start` may carry one or more equivalent `tokens`; while the current\ndataset has any, the operations marked with this scheme require `Authorization: Bearer <token>`\nand answer 401 otherwise. A dataset started without `tokens` is not protected and the header is ignored.\n"}},"parameters":{"binning":{"in":"query","name":"binning","required":false,"schema":{"default":1,"type":"integer"},"description":"Binning of frames for the plot (0 = default binning)"},"experimental_coord":{"in":"query","name":"experimental_coord","required":false,"schema":{"type":"boolean","default":false},"description":"If measurement has goniometer axis defined, plot X-axis will represent rotation angle\nIf measurement has grid scan defined, plot X-axis and Y-axis will represent grid position, Z will be used as the final value\nFor still measurement the number is ignored\n"},"azint_unit":{"in":"query","name":"azint_unit","required":false,"schema":{"type":"string","enum":["Q_recipA","d_A","two_theta_deg"],"default":"Q_recipA"},"description":"Unit used for azim int.\n"},"fill_value":{"in":"query","name":"fill","required":false,"schema":{"type":"number","format":"float"},"description":"Fill value for elements that were missed during data collection\n"},"plot_type":{"in":"query","name":"type","required":true,"description":"Type of requested plot","schema":{"type":"string","enum":["bkg_estimate","spindle_blind_fraction","azint","azint_1d","spot_count","spot_count_low_res","spot_count_indexed","spot_count_ice","indexing_rate","indexing_lattice_count","indexing_unit_cell_length","indexing_unit_cell_angle","profile_radius","mosaicity","b_factor","error_pixels","saturated_pixels","image_collection_efficiency","receiver_delay","receiver_free_send_buf","strong_pixels","roi_sum","roi_mean","roi_max_count","roi_pixels","roi_weighted_x","roi_weighted_y","packets_received","max_pixel_value","resolution_estimate","pixel_sum","processing_time","beam_center_x","beam_center_y","integrated_reflections","image_scale_factor","image_scale_cc","compression_ratio","ice_ring_score"]}},"roi":{"in":"query","name":"roi","required":false,"schema":{"type":"string","minLength":1},"description":"Name of ROI for which plot is requested"},"image_id":{"in":"query","name":"id","required":false,"schema":{"type":"integer","format":"int64","default":-1,"minimum":-2},"description":"Image ID in the image buffer. Special values: -1 - last image in the buffer, -2: last indexed image in the buffer"},"saturation":{"in":"query","name":"saturation","required":false,"schema":{"type":"number","format":"float","minimum":-32767,"maximum":32767},"description":"Saturation value to set contrast in the preview image; if not provided, then autocontrast procedure is used"},"background":{"in":"query","name":"background","required":false,"schema":{"type":"number","format":"float","default":0,"minimum":-32767,"maximum":32767},"description":"Background value to set contrast in the preview image"},"show_spots":{"in":"query","name":"show_spots","schema":{"type":"boolean","default":true},"description":"Show spot finding results on the image"},"show_predictions":{"in":"query","name":"show_predictions","schema":Line truncated
 
       var container = document.getElementById('redoc');
       Redoc.hydrate(__redoc_state, container);

docs/CPU_DATA_ANALYSIS_IMAGE.md

@@ -591,7 +591,7 @@ Because detection reads the pixel's ring, a pixel that falls outside the azimuth
 
 **Fused GPU engine.** The per-ring reduction the adaptive threshold needs is the *same* reduction the azimuthal integrator performs. On the GPU path the two are fused into a single image pass (`AdaptiveSpotFinderGPU`): one reduction accumulates the corrected per-ring sums for the azimuthal profile (§2) *and* the raw per-ring statistics for the threshold, after which a light kernel flags the strong pixels. One GPU pass therefore replaces both the separate azimuthal-integration pass and the host-side adaptive spot-finding pass, at a small fraction of the CPU finder's cost per frame and producing the same spot list and azimuthal profile. It is enabled by default in the offline `rugnux` path, the interactive viewer and the online receiver.
 
-**Online.** `spot_finding_settings` in the REST API carries `adaptive_threshold` and `false_pixels_per_frame`, so the mode is reachable from the broker and from the web frontend as well as from `rugnux` and the viewer. It defaults to *off* online, unlike `rugnux` and the viewer, because the broker serves both workflows and only one of them can run it: spots are found in software only on the DECTRIS/SIMPLON path, while the JUNGFRAU and EIGER workflows find them on the FPGA at its own fixed threshold. Setting `adaptive_threshold` on those is refused with an error rather than accepted and ignored, so a detection setting that had no effect cannot be mistaken for one that did.
+**Online.** `spot_finding_settings` in the REST API carries `adaptive_threshold` and `false_pixels_per_frame`, so the mode is reachable from the broker and from the web frontend as well as from `rugnux` and the viewer. It defaults to *off* online, unlike `rugnux` and the viewer, because the broker serves both workflows and only one of them can run it: spots are found in software only on the DECTRIS/SIMPLON path, while the JUNGFRAU and EIGER workflows find them on the FPGA at its own fixed threshold. Setting `adaptive_threshold` on those is accepted and ignored, so the same spot-finding settings can be shared across detectors.
 
 ### 3.3 Resolution and ice-ring handling
 

docs/python_client/docs/SpotFindingSettings.md

@@ -17,7 +17,7 @@ Name | Type | Description | Notes
 **quick_integration** | **bool** | Quick integration of Bragg spots in diffraction images. If enabled it will likely reduce performance of Jungfraujoch for datasets with a very high indexing rate. (experimental feature)  | [default to False]
 **ice_ring_width_q_recip_a** | **float** | Half-width of the ice ring band in q (1/A). Matches the offline default in image_analysis/spot_finding/SpotFindingSettings.h, which was set from a measured ring FWHM of ~0.06; the two must agree or the same data gets a narrower ice band online. | [default to 0.03]
 **high_res_gap_q_recip_a** | **float** | This parameter is used to remove spurious spots at a very high resolution, that sometimes appear due to very low background close to the edge of the detector. If there is a gap in (1/d)-space between spots of at least this size, spots on the side of the gap with high resolution will be discarded. This is optional parameter. This option should be turned OFF for small molecule datasets or for crystals with very low mosaicity, when it is expected to see only few spots in any case.  | [optional] [default to 1.5]
-**adaptive_threshold** | **bool** | Self-calibrating spot detection: replace the fixed photon_count_threshold by a per-resolution-ring threshold derived from each image's own noise, so the same setting works across datasets without per-dataset tuning. photon_count_threshold is then ignored and false_pixels_per_frame sets the operating point instead. Only available on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow). The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold, so enabling this there is rejected rather than silently ignored.  | [optional] [default to False]
+**adaptive_threshold** | **bool** | Self-calibrating spot detection: replace the fixed photon_count_threshold by a per-resolution-ring threshold derived from each image's own noise, so the same setting works across datasets without per-dataset tuning. photon_count_threshold is then ignored and false_pixels_per_frame sets the operating point instead. Used only on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow). The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold; there the setting is accepted and ignored, so the same settings can be shared across detectors.  | [optional] [default to False]
 **false_pixels_per_frame** | **float** | Operating point of the adaptive threshold: the number of noise pixels tolerated per frame. About 100 suits a multi-megapixel detector. Ignored unless adaptive_threshold is set.  | [optional] [default to 100.0]
 
 ## Example

frontend/src/client/types.gen.ts

@@ -788,9 +788,9 @@ export type spot_finding_settings = {
      * threshold derived from each image's own noise, so the same setting works across datasets without
      * per-dataset tuning. photon_count_threshold is then ignored and false_pixels_per_frame sets the
      * operating point instead.
-     * Only available on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow).
-     * The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold,
-     * so enabling this there is rejected rather than silently ignored.
+     * Used only on detectors whose images are analysed in software (the DECTRIS/SIMPLON workflow).
+     * The JUNGFRAU and EIGER workflows find spots on the FPGA, which applies its own fixed threshold;
+     * there the setting is accepted and ignored, so the same settings can be shared across detectors.
      *
      */
     adaptive_threshold?: boolean;

tests/JFJochStateMachineTest.cpp

@@ -53,6 +53,23 @@ TEST_CASE("JFJochStateMachine_Initialize_DisablesRugnux") {
     CHECK(s.model_size_bytes == 0);
 }
 
+TEST_CASE("JFJochStateMachine_AdaptiveThreshold_AcceptedOnFPGADetector") {
+    Logger logger("JFJochStateMachine_AdaptiveThreshold_AcceptedOnFPGADetector");
+    JFJochServices services(logger);
+    DiffractionExperiment experiment;
+    JFJochStateMachine state_machine(experiment, services, logger);
+    state_machine.AddDetectorSetup(DetJF4M());
+    REQUIRE_NOTHROW(state_machine.Initialize());
+    REQUIRE_NOTHROW(state_machine.WaitTillMeasurementDone());
+
+    // JUNGFRAU finds spots on the FPGA, so the adaptive threshold is not used there - but it is
+    // accepted, so the same settings can be shared across detectors.
+    SpotFindingSettings settings = state_machine.GetSpotFindingSettings();
+    settings.adaptive_threshold = true;
+    REQUIRE_NOTHROW(state_machine.SetSpotFindingSettings(settings));
+    CHECK(state_machine.GetSpotFindingSettings().adaptive_threshold);
+}
+
 TEST_CASE("JFJochStateMachine_State_Pedestal") {
     Logger logger("JFJochStateMachine_State_Pedestal");
     JFJochServices services(logger);