Jungfraujoch/docs/JFJOCH_BROKER.md

jfjoch_broker

jfjoch_broker is the main service for the Jungfraujoch application. It is responsible for:

• Providing user interface via HTTP and OpenAPI
• Configuring FPGA firmware
• Building images from FPGA output and forwarding the results over ZeroMQ

External interfaces

Broker operates four external interfaces.

Image stream ZeroMQ PUSH socket with CBOR serialization is used to send images, metadata and processing results for writing or downstream processing. See details here.

Preview stream ZeroMQ PUB socket, as above but limited to subset of frames (1 image/s by default). See details here.

Metadata stream ZeroMQ PUB socket, contains metadata for all the images, with bundling. See details here.

Configuration, status and results interface HTTP/REST interface described in the OpenAPI format. Description of the API is presented in the OpenAPI specification.

Broker configuration

jfjoch_broker requires JSON configuration files. The file is described by OpenAPI structure jfjoch_settings defined in jfjoch_api.yaml file. It is recommended to go through example files in the etc/.

Example configuration (not every section is shown):

{
  "pcie": [
    {
      "blk": "/dev/jfjoch0",
      "ipv4": "10.1.1.7"
    },
    {
      "blk": "/dev/jfjoch1",
      "ipv4": "10.1.1.8"
    }
  ],
  "zeromq": {
    "send_watermark": 100,
    "send_buffer_size": 1024,
    "image_socket": [
      "tcp://1.2.3.4:5000",
      "tcp://1.2.3.4:5001"
    ],
    "writer_notification_socket": "tcp://1.3.4.6:7000"
  },
  "instrument": {
    "source_name": "Swiss Light Source",
    "source_type": "Synchrotron X-ray Source",
    "instrument_name": "X06SA",
    "pulsed_source": false,
    "electron_source": false
  },
  "detector": [
    {
      "description": "EIGER 1M",
      "serial_number": "E1M-01",
      "type": "EIGER",
      "high_voltage_V": 150,
      "udp_interface_count": 1,
      "module_sync": true,
      "sensor_thickness_um": 320,
      "calibration_file": [
        "gainMaps.bin"
      ],
      "hostname": [
        "e1m-01",
        "e1m-02"
      ],
      "readout_time_us": 3,
      "sensor_material": "Si",
      "tx_delay": [
        0,1
      ],
      "base_data_ipv4_address": "10.10.10.50",
      "standard_geometry": {
        "nmodules": 1,
        "gap_x": 8,
        "gap_y": 36,
        "modules_in_row": 1
      },
      "custom_geometry": [
        {
          "x0": 0,
          "y0": 0,
          "fast_axis": "Xp",
          "slow_axis": "Yp"
        }
      ],
      "mirror_y": true
    }
  ],
  "detector_settings": {
    "frame_time_us": 450,
    "count_time_us": 0,
    "internal_frame_generator": false,
    "internal_frame_generator_images": 1,
    "detector_trigger_delay_ns": 0,
    "timing": "auto",
    "eiger_threshold_keV": 6.0,
    "jungfrau_pedestal_g0_frames": 2000,
    "jungfrau_pedestal_g1_frames": 300,
    "jungfrau_pedestal_g2_frames": 300,
    "jungfrau_pedestal_g0_rms_limit": 100,
    "jungfrau_pedestal_min_image_count": 128,
    "jungfrau_storage_cell_count": 1,
    "jungfrau_storage_cell_delay_ns": 5000,
    "jungfrau_fixed_gain_g1": false,
    "jungfrau_use_gain_hg0": false
  },
  "azim_int": {
    "polarization_factor": -1,
    "solid_angle_corr": true,
    "high_q_recipA": 0,
    "low_q_recipA": 0,
    "q_spacing": 0
  },
  "image_format": {
    "summation": true,
    "geometry_transform": true,
    "jungfrau_conversion": true,
    "jungfrau_conversion_factor_keV": 0.001,
    "bit_depth_image": 16,
    "signed_output": true,
    "mask_module_edges": true,
    "mask_chip_edges": true
  },
  "image_buffer_MiB": 2048,
  "receiver_threads": 64,
  "frontend_directory": "/usr/share/jfjoch/frontend",
  "image_pusher": "ZeroMQ",
  "zeromq_metadata": {
    "enabled": true,
    "period_ms": 1000,
    "socket_address": "tcp://0.0.0.0:4357"
  },
  "zeromq_preview": {
    "enabled": true,
    "period_ms": 1000,
    "socket_address": "tcp://0.0.0.0:4356"
  }
}

Setting up a local test for Jungfraujoch

For development, it is possible to set up a local installation of Jungfraujoch. This will work without FPGA installed in the computer and allows testing the Jungfraujoch software layer, including ZeroMQ streaming and file writing.

The workflow simulates FPGA behavior, by running high-level synthesis code on the CPU - the performance is therefore very low, as fixed-point calculations have a large performance penalty on the CPU. In the CPU simulation mode, one can simulate using only a single FPGA device.

To run the test:

Compile Jungfraujoch with frontend

mkdir build
cd build
cmake ..
make jfjoch_broker
make frontend

Alternatively, on a RHEL8 system, you can use the RPMs generated by the automated pipeline. The jfjoch package alone is enough. In this case - it is necessary to update etc/broker_local.json file with frontend path in /usr/share/jfjoch/frontend.

Start service

Start broker:

cd build/broker
./jfjoch_broker ../../etc/broker_local.json 5232

Run tests

To run the test, a Python script is provided:

cd tests/test_data
python jfjoch_broker_test.py

The script will initialize Jungfraujoch, import test image and start data collection.

Expected result

You can observe online data analysis by opening the following web page: http://localhost:5232. Also, a dataset with images should be written in the build/broker directory.