DG FPGA Manual

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 1
FPGA setup for TOE/UDP25G-IP with CPU Demo
Rev2.0 7-Jun-21
1 Overview
This document describes how to setup FPGAboard and prepare the test environment for running
TOE25G-IP/UDP25G-IP demo. The user can setup two test environments for transferring
TCP/UDP data via 25Gb Ethernet connection by using TOE25G-IP/UDP25G-IP, as shown in
Figure 1-1.
Figure 1-1 Two test environments for running the demo

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 2
Firstly, it uses one FPGA board and Test PC with 25Gb Ethernet card for transferring the data.
TestPC runs test application, i.e., tcpdatatest (half-duplex test for TOE25G-IP),
tcp_client_txrx_40G (full-duplex test for TOE25G-IP), or udpdatatest (application test for
UDP25G-IP). Also, Serial console/JTAG Terminal is run on Test PC to be user interface console.
Secondly, it uses two FPGA boards which may be different board. Both boards run
TOE25G-IP/UDP25G-IP demo with assigning the different initialization mode (Client, Server, or
Fixed-MAC) for transferring data.

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 3
2 Test environment setup when using FPGA and PC
To run the demo on FPGA development board, please prepare following environment.
•FPGA development boards:
a) TOE25G-IP: VCU118 and KCU116 board
b) UDP25G-IP: KCU116 board and FB2CGHH@KU15P card
•PC with 25 Gigabit Ethernet card
•25Gb Ethernet cable:
a) VCU118 and FB2CGHH@KU15P card: SFP28 transceiver (25GBASE-SR), QSFP28
transceiver (100GBASE-SR), and MTP to 4xLC Fiber cable
b) KCU116: 25G SFP28Active Optical Cable (AOC).
•USB cable for connecting between FPGA and PC
a) VCU118 and KCU116 board: 2 micro USB cables for programming FPGA and Serial
console
b) FB2CGHH@KU15P card: 1 mini USB cable for programming FPGA and JTAGAURT
•For FB2CGHH@KU15P card, use AB18-PCIeX16 board provided by Design Gateway
with ATX power supply to be power board for the card. More details of AB18 card are
displayed on following website.
https://dgway.com/ABseries_E.html
•Test application provided by Design Gateway for running on Test PC
a) TOE25G-IP: “tcpdatatest.exe” and “tcp_client_txrx_40G.exe”
b) UDP25G-IP: “udpdatatest.exe”
•VCU118 and KCU116 board: Serial console software such as TeraTerm installed on PC.
The setting on the console is Baudrate=115,200, Data=8-bit, Non-parity, and Stop=1.
Note: JTAG Terminal in applied instead of Serial console when using FB2CGHH@KU15P
card.
•Vivado tool for programming FPGA and JTAG Terminal (when using FB2CGHH@KU15P
card), installed on PC
Note: Example hardware for running the demo is listed as follows.
[1] 25G Network Adapter: Intel XXV710-DA2
https://ark.intel.com/content/www/us/en/ark/products/95260/intel-ethernet-network-adap
ter-xxv710-da2.html
[2] a) SFP28 to QSFP28 connection (VCU118 board and FB2CGHH@KU15P card)
SFP28 Transceiver: AZS85-S28-M1
https://www.sfpcables.com/25gb-s-sfp28-sr-transceiver-850nm-up-to-100m-2866
QSFP28 Transceiver: AMQ28-SR4-M1
https://www.sfpcables.com/100gb-s-qsfp28-sr4-optical-transceiver-module-1499
MTP to 4xLC Fiber cable: OM4-MTP-8LC-1M
https://www.fs.com/products/68047.html
b) 25G SFP28 Active Optical Cable: S28-AO01 (KCU116 board)
https://www.fs.com/sg/products/68335.html
[3] PC: Motherboard Gigabyte B460M AORUS PRO, 32 GB RAM, and 64-bit Windows10 OS

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 4
Figure 2-1 TOE25G-IP/UDP25G-IP with CPU demo (FPGA <-> PC) on VCU118

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 5
Figure 2-2 TOE25G-IP/UDP25G-IP with CPU demo (FPGA <-> PC) on KCU116

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 6
Figure 2-3 TOE25G-IP/UDP25G-IP with CPU demo (FPGA <-> PC) on FB2CGHH@KU15P

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 7
The step to setup test environment by using FPGA and PC is described in more details as
follows.
1) Connect USB cables between FPGA and PC for JTAG programming and Serial
console/JTAGUART.
a) KCU116 and VCU118 board: Connect two micro USB cables
b) FB2CGHH@KU15P card: Connect one mini USB cable
2) Connect power supply to FPGA development board/FPGA accelerator card.
a) KCU116 and VCU118 board: Connect Xilinx power adapter
b) FB2CGHH@KU15P card: Connect the card to PC or AB18-PCIeX16 board by
following step
i) Confirm that two mini jumpers are inserted at J5 connector on AB18
ii) Connect ATX power supply to AB board
iii) Connect PCIe connector on FPGA board to Device Side (B-Side), as shown in
Figure 2-4
Figure 2-4 AB18-PCIeX16 connection for power supply of FB2CGHH@KU15P card

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 8
3) Connect 25Gb Ethernet cable between FPGA board and PC.
a) VCU118 and FB2CGHH@KU15P: Insert QSFP28 to SFP28 cable by using QSFP1
connector on VCU118 and plug SFP28 no.1 to 25Gb Ethernet card on Test PC, as
shown in Figure 2-5.
Figure 2-5 25Gb connection on VCU118 board and FB2CGHH@KU15P card
by QSFP28 to 4xSFP28 cable

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 9
b) KCU116: Insert 25G SFP28 AOC cable between FPGA board (on the left-most
channel) and 25Gb Ethernet card on Test PC, as shown in Figure 2-6.
Figure 2-6 25Gb connection on KCU116 board by 25G SFP28 AOC
4) Connect another side of cable with 25Gb Ethernet card of PC.
5) Power on FPGA board.
6) KCU116 board and U250 Card: Open Serial console and download configuration file with
firmware by following step.
i) Open Serial console. When connecting FPGA board to PC, many COM ports from
FPGA connection are detected and displayed on Device Manager. Select Standard
COM port.
On Serial console, use following setting: Buad rate=115,200, Data=8-bit, Non-Parity,
and Stop=1.
Figure 2-7 COM port number for Serial console

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 10
ii) Set programmable clock to 322.265625 MHz
a) VCU118 board: Set by using “VCU118 SCUI”application as shown in Figure 2-8.
Figure 2-8 Reference clock programming for VCU118
b) KCU116 board: Set by using “KCU116 –Board User Interface”application as
shown in Figure 2-9.
Figure 2-9 Reference clock programming for KCU116

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 11
iii) Download configuration file and firmware to FPGA board by using Vivado tool to
program configuration file, as shown in Figure 2-10.
Figure 2-10 Program FPGA by Vivado

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 12
7) FB2CGHH@KU15P card: Open vivado TCL shell and browse to the directory that
includes batch file, bit file, and elf file of the demo. After that, run the test by typing
following command
i) >> TOE25CPUTest_Silicom.bat/UDP25CPU_Silicom.bat
Note: This step is to download configuration file and firmware, as shown in Figure 2-11
Figure 2-11 Command script to download demo file on Vivado TCL shel
ii) >> xsdb.bat
iii) >> connect -url tcp:127.0.0.1:3121
iv) >> targets -set -filter {name =~"*Debug*"}
v) >> jtagterminal -start
vi) >> con
Note: Above steps are to connect JTAGUART module and run JTAG terminal to be user
console, as shown Figure 2-12.
Figure 2-12 Open JTAG Terminal

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 13
8) On Serial console/JTAG Terminal, welcome message is displayed.
i) Input ‘0’ to start TOE25G-IP/UDP25G-IP initialization in client mode (asking PC MAC
address by sending ARP request).
ii) Default parameter in client mode is displayed on the console.
Figure 2-13 Message after system boot-up
If Ethernet connection has the problem which makes the linked down, the error message
is displayed on the console instead of welcome message, as shown in Figure 2-14.
Figure 2-14 Error message when cable is linked down

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 14
iii) User enters ‘x’ to skip parameter setting for using default parameters to begin system
initialization, as shown in Figure 2-15. If user enters other keys, the menu for changing
parameter is displayed, similar to “Reset TCPIP/UDPIP parameters”menu. The
example when running the main menu is described in “dg_toe25gip_cpu_instruction”
or “dg_udp25gip_cpu_instruction”document.
Figure 2-15 Initialization complete
Note: Transfer performance in the demo is limited by Test PC performance in Test platform.
The best performance can be achieved when the test is run by using FPGA-to-FPGA
connection.

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 15
3 Test environment setup when using two FPGAs
Before running the test, please prepare following test environment.
•Two FPGA development boards which can be the same board or different board: KCU116,
VCU118 board, and FB2CGHH@KU15P card
•25Gb Ethernet cable:
a) VCU118 and FB2CGHH@KU15P card: SFP28 transceiver (25GBASE-SR), QSFP28
transceiver (100GBASE-SR) and MTP to 4xLC Fiber cable
b) KCU116: 25G SFP28Active Optical Cable (AOC).
•USB cable for connecting between FPGAand PC
a) KCU116 and VCU118: 2 micro USB cables for programming FPGA and Serial console
b) FB2CGHH@KU15P card: mini USB cable for programming FPA and JTAGUART
•VCU118 and KCU116 board: Serial console software such as TeraTerm, installed on PC.
The setting on the console is Baudrate=115,200, Data=8-bit, Non-parity and Stop=1.
Note: JTAG Terminal in applied instead of Serial console when using FB2CGHH@KU15P
card.
•Vivado tool for programming FPGA and JTAG Terminal (when using FB2CGHH@KU15P
card), installed on PC

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 16
Figure 3-1 TOE25G-IP/UDP25G-IP with CPU demo (FPGA<->FPGA)

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 17
The step to setup test environment by using two FPGAs is described in more details as
follows.
Follow step 1) –7) of topic 2 (Test environment setup when using FPGA and PC) to prepare
FPGA board and SFP28/QSFP28 connection for running the demo. After two FPGA boards
have been configured completely, Serial console/JTAG Terminal displays the menu to select
Client mode, Server mode, or Fixed MAC mode. The step after FPGA configuration is
described as follows.
1) Open Serial console/JTAG Terminal for board#1 and board#2 and then set the input to the
console for selecting the initialization mode. The example to initialize by Server-Client
mode is described as following steps.
i) Set ‘1’ on the console of FPGA board#1 for running Server mode.
ii) Set ‘0’ on the console of FPGA board#2 for running Client mode.
iii) Default parameters for Server or Client are displayed on the console, as shown in
Figure 3-2.
Figure 3-2 Input mode

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 18
2) Input ‘x’ to use default parameters or other keys to change parameters. The parameters of
Server mode must be set before Client mode.
When running TOE25G-IP,
i) Set parameters on Server console (board#1 console).
ii) Set parameters on Client console (board#2 console) to start IP initialization by
transferring ARP packet.
iii) After finishing initialization process, “IP initialization complete” and main menu are
displayed on Server console and Client console.
Figure 3-3 Main menu of TOE25G-IP

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 19
When running UDP25G-IP,
i) Set parameters on Server console (board#1 console). If user does not change the
default parameters, input ‘x’to skip parameter setting.
ii) For Client mode, user must change target port number (Target->FPGA) to use same
value as target port number (FPGA->Target).
iii) After finishing initialization process, “IP initialization complete” and main menu are
displayed on Server console and Client console.
Figure 3-4 Main menu of UDP25G-IP

dg_toeudp25gip_fpgasetup_xilinx.doc
7-Jun-21 Page 20
4 Revision History
Revision
Date
Description
1.0
5-Aug-20
Initial version release
1.1
15-Sep-20
Add KCU116
2.0
7-Jun-21
Add UDP25G-IP