81 lines
3.6 KiB
Markdown
81 lines
3.6 KiB
Markdown
# FPGA smartNIC
|
|
|
|
See separate document for [installation instructions](DEPLOYMENT.md).
|
|
|
|
## Hardware
|
|
Currently supported FPGA is only **Xilinx Alveo U55C**.
|
|
|
|
See AMD/Xilinx webpage for [card user guide (UG1469)](https://docs.xilinx.com/r/en-US/ug1469-alveo-u55c).
|
|
According to the user guide:
|
|
```
|
|
Alveo data center accelerator cards are designed to be installed into a data center server, where controlled air flow provides direct cooling.
|
|
```
|
|
|
|
Card needs to be placed in PCI Express (PCIe) Gen4 x8 slot, though mechanically slot has to accommodate x16 card.
|
|
There is no need to connect additional power cable, as power of the card is not exceeding 75 W load available from PCIe edge connector.
|
|
Current power estimation is about 30 W when idle and 45 W in operation. The card has built-in protection, which will cut power to the card if HBM temperature is above 120°C.
|
|
|
|
Two variants of the card are available:
|
|
* `100g` - this variant operates one port in 100 Gbit/s mode and should be used when connecting detector via a switch.
|
|
* `8x10g` - this variant operates both QSFP ports at 4x10 Gbit/s. QSFP+ (40 Gbit/s) transceivers and MTO/MTP harness cables
|
|
are necessary. It is designed for detector directly connected to the Jungfraujoch server, without switch.
|
|
|
|
See [network documentation](FPGA_NETWORK.md) for details of network.
|
|
|
|
## Building firmware
|
|
Xilinx Vivado version has to precisely match version described in [the system requirements](../README.md.
|
|
only when `vivado` and `vitis_hls` are detected in the path.
|
|
|
|
### Xilinx Vivado
|
|
The following procedures require having AMD (Xilinx) Vivado and Vitis HLS toolsets version **2022.2** installed on the machine.
|
|
Due to the nature of TCL scripts used to generate board designs Vivado version has to exactly match one provided above -
|
|
specifically newer versions of Vivado will not work.
|
|
|
|
In additional to Intellectual Property (IP) cores included in Vivado, two additional licenses are necessary:
|
|
* Non-cost license for Ultrascale+ 100G core has to be requested from AMD/Xilinx website, see [Xilinx website](https://www.xilinx.com/products/intellectual-property/cmac_usplus.html), to build `100g` design.
|
|
* Paid 10G/25G Subsystem for Ultrascale+ to build `8x10g` design.
|
|
PSI received non-cost licenses from Xilinx University Program for the latter cores. Therefore, usage of bitstreams
|
|
generated by PSI continuous integration pipeline for `8x10g` is only allowed for non-commercial use.
|
|
### HLS compilation
|
|
Make HLS routines:
|
|
```
|
|
mkdir build
|
|
cd build
|
|
cmake ..
|
|
make hls
|
|
```
|
|
|
|
### Synthesis
|
|
Create PCIe `100g` bitstream with the following command:
|
|
```
|
|
mkdir build
|
|
cd build
|
|
cmake ..
|
|
make pcie_100g
|
|
```
|
|
and `8x10g`:
|
|
```
|
|
mkdir build
|
|
cd build
|
|
cmake ..
|
|
make pcie_8x10g
|
|
```
|
|
### When Vivado is not present
|
|
|
|
During CMake execution, the following executables: `vivado` and `vitis_hls` must be present in the path.
|
|
If not, build targets will not be generated, and such or similar error message will show up:
|
|
```
|
|
$ make pcie_100g
|
|
make: *** No rule to make target 'pcie_100g'. Stop.
|
|
```
|
|
|
|
### Gitlab CI
|
|
If Gitlab CI is properly set-up, firmware will be automatically built for every commit that starts with FPGA.
|
|
Built firmware should be downloaded as MCS files.
|
|
|
|
### Frame generator
|
|
|
|
Jungfraujoch card is equipped with frame generator. It allows to simulate JUNGFRAU detector without having access to such system.
|
|
It is placed in parallel to Ethernet MAC - so it is placed before the network stack and before any processing happening on the card.
|
|
In the future a redirection will be possible to send the simulated stream through the 100G TX network link.
|
|
Frame generator is written in HLS and controlled with AXI-Lite. |