leonarski_fandClaude Fable 5
70c4d871b6
- Capitalize Rugnux as a proper noun throughout the prose; the command stays lowercase `rugnux` in code font. RUGNUX_OVERVIEW.md is retitled "What Rugnux does". - ACKNOWLEDGEMENT.md cites the raw-data repositories only: dataset counts and DOI prefixes moved out (EXTERNAL_TEST_DATA.md owns them), the ESRF data portal gains its citation (Dimper et al. 2019), and MXRDR remains name + link - it has no canonical citation paper. - RUGNUX_FORMATS.md: the CCD formats (marCCD, SMV) are supported as-is with very limited scope, and per-panel XFEL data is not read. - Fix wrong facts a reader would act on: nonexistent `make jfjoch` targets, invalid udev rules, PUSH sockets documented as PULL, swapped writer width/height, underload semantics, the transposed pixel-mask numpy example (the server checks width and height separately), the Durin/Neggia mask-bit table, FPGA threshold register addresses and the mailbox bit field, the I2C core's document number (PG090), an inverted MODEL_FIT_SIGMA formula, a self-inconsistent worked report example, and 11 cross-page anchors whose slugs carry MyST section numbers. - Unify CC1/2 spelling in prose; math notation and report keys unchanged. - Sweep grammar, typos and editing residue across the FPGA, deployment, streaming and analysis pages, including historical CHANGELOG typos. - rugnux_cli.cpp: the -S usage/error examples pair 96 with P43212; 92 names a different group. - Root THIRD_PARTY_NOTICES.md: scope the GPL-compatibility claim (CUDA EULA) and the vendored-table intro (traccc); the docs copy regenerates via update_version.sh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
39 lines
2.8 KiB
Markdown
39 lines
2.8 KiB
Markdown
# FPGA network
|
|
|
|
The U55C card is equipped with two network connectors - QSFP0 is the upper port and QSFP1 the lower port (when PCIe connector is on the bottom).
|
|
The card FPGA design is offered in two variants `100g` and `8x10g`. These have different behavior regarding the network:
|
|
|
|
`100g` — this variant operates the QSFP0 port in 100 Gbit/s mode and should be used when connecting detector via a **switch**.
|
|
QSFP28 transceivers are necessary.
|
|
|
|
`8x10g` — this variant operates both QSFP ports at 4x10 Gbit/s. QSFP+ (40 Gbit/s) transceivers and MTO/MTP harness cables
|
|
are necessary. It is designed for **detector directly connected** to the Jungfraujoch server, without switch.
|
|
|
|
## Transceivers
|
|
AMD doesn't provide transceiver compatibility matrix for Alveo U55C.
|
|
In our experience operating the card we haven't seen issues with transceivers from various providers (FS.com, Mellanox, Finisar).
|
|
We have also successfully operated card with correct direct attach cables instead of fiber optics. Given the card doesn't
|
|
support link training functionality of 100 Gbit/s ethernet, it could result in performance problems with copper cables, though we haven't
|
|
encountered such a situation.
|
|
|
|
## Switch configuration
|
|
Special care has to be taken for switch operation, given the FPGA core doesn't support auto-negotiation. It is necessary to configure switch port
|
|
to fixed speed (100 Gbit/s or 10 Gbit/s) and to disable auto-negotiation. It is also necessary to enable jumbo frames (MTU of 9000).
|
|
|
|
## Network LEDs
|
|
Each QSFP connector is equipped with green and orange LEDs. These LEDs are connected to Ethernet physical layer status port (rx_status).
|
|
LED on corresponds to having a physical connection to a switch/computer/detector on the other side of the network.
|
|
For 100 Gbit/s only green is used, for 8x10 Gbit/s green LEDs mean all ports connected, orange LEDs at least one of the ports connected.
|
|
|
|
## Network stack
|
|
Each Ethernet link has its own basic network stack. Functionality for Ethernet/ARP/IPv4/ICMP is therefore separately handled for each port.
|
|
Each link will get dedicated MAC address, and IPv4 addresses can be also assigned independently if needed.
|
|
|
|
The card will send gratuitous ARP messages every 5 seconds to keep its entry in switch MAC table.
|
|
The card will also reply to ARP requests for its IP and to ICMP ping requests sent with the card IPv4 address.
|
|
The card won't respond to broadcast ICMP pings.
|
|
|
|
Each link can be put in `direct` mode. In this case destination Ethernet MAC and IPv4 addresses are not enforced for incoming UDP packets.
|
|
This setting should be used for connecting detector modules directly to the FPGA card, so any detector module can be connected to any
|
|
10 Gbit/s link on the same card. Currently `direct` mode is turned OFF for `100g` design and ON for `8x10g` design.
|
|
This can be manually adjusted for each link. |