Jungfraujoch/docs/JFJOCH_BROKER.md

jfjoch_broker

jfjoch_broker is the main service for the Jungfraujoch application. It is responsible for:

• Providing user interface via HTTP and OpenAPI
• Configuring FPGA firmware
• Building images from FPGA output and forwarding the results over ZeroMQ

External interfaces

Broker operates four external interfaces.

Image stream ZeroMQ PULL socket with CBOR serialization is used to send images, metadata and processing results for writing or downstream processing. See details here.

Preview stream ZeroMQ PUB socket, as above but limited to subset of frames (1 image/s by default). See details here.

Metadata stream ZeroMQ PUB socket, contains metadata for all the images, with bundling. See details here.

Configuration, status and results interface HTTP/REST interface described in the OpenAPI format. Description of the API is presented in the OpenAPI description.

Broker configuration

jfjoch_broker requires JSON configuration files. The file is described by OpenAPI structure jfjoch_settings defined in jfjoch_api.yaml file. It is recommended to go through example files in the etc/.

Example with all fields:

{
  "pcie": [
    {
      "blk": "/dev/jfjoch0",
      "ipv4": "10.1.1.7"
    },
    {
      "blk": "/dev/jfjoch1",
      "ipv4": "10.1.1.8"
    }
  ],
  "zeromq": {
    "send_watermark": 100,
    "send_buffer_size": 1024,
    "image_socket": [
      "tcp://1.2.3.4:5000",
      "tcp://1.2.3.4:5001"
    ],
    "writer_notification_socket": "tcp://1.3.4.6:7000"
  },
  "instrument": {
    "source_name": "Swiss Light Source",
    "source_type": "Synchrotron X-ray Source",
    "instrument_name": "X06SA",
    "pulsed_source": false,
    "electron_source": false
  },
  "detector": [
    {
      "description": "EIGER 1M",
      "serial_number": "E1M-01",
      "type": "EIGER",
      "high_voltage_V": 150,
      "udp_interface_count": 1,
      "module_sync": true,
      "sensor_thickness_um": 320,
      "calibration_file": [
        "gainMaps.bin"
      ],
      "hostname": [
        "e1m-01",
        "e1m-02"
      ],
      "readout_time_us": 3,
      "sensor_material": "Si",
      "tx_delay": [
        0,1
      ],
      "base_data_ipv4_address": "10.10.10.50",
      "standard_geometry": {
        "nmodules": 1,
        "gap_x": 8,
        "gap_y": 36,
        "modules_in_row": 1
      },
      "custom_geometry": [
        {
          "x0": 0,
          "y0": 0,
          "fast_axis": "Xp",
          "slow_axis": "Xp"
        }
      ],
      "mirror_y": true
    }
  ],
  "detector_settings": {
    "frame_time_us": 450,
    "count_time_us": 0,
    "internal_frame_generator": false,
    "internal_frame_generator_images": 1,
    "detector_trigger_delay_ns": 0,
    "timing": "auto",
    "eiger_threshold_keV": 6.0,
    "jungfrau_pedestal_g0_frames": 2000,
    "jungfrau_pedestal_g1_frames": 300,
    "jungfrau_pedestal_g2_frames": 300,
    "jungfrau_pedestal_g0_rms_limit": 100,
    "jungfrau_pedestal_min_image_count": 128,
    "jungfrau_storage_cell_count": 1,
    "jungfrau_storage_cell_delay_ns": 5000,
    "jungfrau_fixed_gain_g1": false,
    "jungfrau_use_gain_hg0": false
  },
  "azim_int": {
    "polarization_factor": -1,
    "solid_angle_corr": true,
    "high_q_recipA": 0,
    "low_q_recipA": 0,
    "q_spacing": 0
  },
  "image_format": {
    "summation": true,
    "geometry_transform": true,
    "jungfrau_conversion": true,
    "jungfrau_conversion_factor_keV": 0.001,
    "bit_depth_image": 16,
    "signed_output": true,
    "mask_module_edges": true,
    "mask_chip_edges": true
  },
  "image_buffer_MiB": 2048,
  "receiver_threads": 64,
  "frontend_directory": "/usr/share/jfjoch/frontend",
  "image_pusher": "ZeroMQ",
  "zeromq_metadata": {
    "enabled": true,
    "period_ms": 1000,
    "socket_address": "tcp://0.0.0.0:4357"
  },
  "zeromq_preview": {
    "enabled": true,
    "period_ms": 1000,
    "socket_address": "tcp://0.0.0.0:4356"
  }
}

Setting up a local test for Jungfraujoch

For development, it is possible to set up a local installation of Jungfraujoch. This will work without FPGA installed in the computer and allows to test Jungfraujoch software layer, including ZeroMQ streaming and file writing.

The workflow simulates FPGA behavior, by running high-level synthesis code on the CPU - the performance is therefore very low, as fixed-point calculations have large performance penalty on CPU. In the CPU simulation mode, one can simulate using only a single FPGA device.

To run the test:

Compile Jungfraujoch with frontend

mkdir build
cd build
cmake ..
make jfjoch
make frontend

Alternatively, for RHEL8 system, you can use RPM generated by automated pipeline. Solely jfjoch one is enough. In this case - it is necessary to update etc/broker_local.json file with frontend path in /usr/share/jfjoch/frontend.

Start service

Start broker:

cd build/broker
./jfjoch_broker ../../etc/broker_local.json 5232

Run tests

To run test a Python script is provided:

cd tests/test_data
python jfjoch_broker_test.py

The script will initialize Jungfraujoch, import test image and start data collection.

Expected result

You can observe online data analysis by opening the following web page: http://localhost:5232. Also, a dataset with images should be written in the build/broker directory.