- Capitalize Rugnux as a proper noun throughout the prose; the command stays lowercase `rugnux` in code font. RUGNUX_OVERVIEW.md is retitled "What Rugnux does". - ACKNOWLEDGEMENT.md cites the raw-data repositories only: dataset counts and DOI prefixes moved out (EXTERNAL_TEST_DATA.md owns them), the ESRF data portal gains its citation (Dimper et al. 2019), and MXRDR remains name + link - it has no canonical citation paper. - RUGNUX_FORMATS.md: the CCD formats (marCCD, SMV) are supported as-is with very limited scope, and per-panel XFEL data is not read. - Fix wrong facts a reader would act on: nonexistent `make jfjoch` targets, invalid udev rules, PUSH sockets documented as PULL, swapped writer width/height, underload semantics, the transposed pixel-mask numpy example (the server checks width and height separately), the Durin/Neggia mask-bit table, FPGA threshold register addresses and the mailbox bit field, the I2C core's document number (PG090), an inverted MODEL_FIT_SIGMA formula, a self-inconsistent worked report example, and 11 cross-page anchors whose slugs carry MyST section numbers. - Unify CC1/2 spelling in prose; math notation and report keys unchanged. - Sweep grammar, typos and editing residue across the FPGA, deployment, streaming and analysis pages, including historical CHANGELOG typos. - rugnux_cli.cpp: the -S usage/error examples pair 96 with P43212; 92 names a different group. - Root THIRD_PARTY_NOTICES.md: scope the GPL-compatibility claim (CUDA EULA) and the vendored-table intro (traccc); the docs copy regenerates via update_version.sh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2.8 KiB
FPGA network
The U55C card is equipped with two network connectors - QSFP0 is the upper port and QSFP1 the lower port (when PCIe connector is on the bottom).
The card FPGA design is offered in two variants 100g and 8x10g. These have different behavior regarding the network:
100g — this variant operates the QSFP0 port in 100 Gbit/s mode and should be used when connecting detector via a switch.
QSFP28 transceivers are necessary.
8x10g — this variant operates both QSFP ports at 4x10 Gbit/s. QSFP+ (40 Gbit/s) transceivers and MTO/MTP harness cables
are necessary. It is designed for detector directly connected to the Jungfraujoch server, without switch.
Transceivers
AMD doesn't provide transceiver compatibility matrix for Alveo U55C. In our experience operating the card we haven't seen issues with transceivers from various providers (FS.com, Mellanox, Finisar). We have also successfully operated card with correct direct attach cables instead of fiber optics. Given the card doesn't support link training functionality of 100 Gbit/s ethernet, it could result in performance problems with copper cables, though we haven't encountered such a situation.
Switch configuration
Special care has to be taken for switch operation, given the FPGA core doesn't support auto-negotiation. It is necessary to configure switch port to fixed speed (100 Gbit/s or 10 Gbit/s) and to disable auto-negotiation. It is also necessary to enable jumbo frames (MTU of 9000).
Network LEDs
Each QSFP connector is equipped with green and orange LEDs. These LEDs are connected to Ethernet physical layer status port (rx_status). LED on corresponds to having a physical connection to a switch/computer/detector on the other side of the network. For 100 Gbit/s only green is used, for 8x10 Gbit/s green LEDs mean all ports connected, orange LEDs at least one of the ports connected.
Network stack
Each Ethernet link has its own basic network stack. Functionality for Ethernet/ARP/IPv4/ICMP is therefore separately handled for each port. Each link will get dedicated MAC address, and IPv4 addresses can be also assigned independently if needed.
The card will send gratuitous ARP messages every 5 seconds to keep its entry in switch MAC table. The card will also reply to ARP requests for its IP and to ICMP ping requests sent with the card IPv4 address. The card won't respond to broadcast ICMP pings.
Each link can be put in direct mode. In this case destination Ethernet MAC and IPv4 addresses are not enforced for incoming UDP packets.
This setting should be used for connecting detector modules directly to the FPGA card, so any detector module can be connected to any
10 Gbit/s link on the same card. Currently direct mode is turned OFF for 100g design and ON for 8x10g design.
This can be manually adjusted for each link.