DG FPGA Manual

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 1
FPGA setup for TOE100G-IP with CPU Demo
Rev1.1 28-Apr-21
1 Overview
This document describes how to setup FPGAboard and prepare the test environment for running
TOE100G-IP demo. The user can setup two test environments for transferring TCP data via
100Gb Ethernet connection by using TOE100G-IP, as shown in Figure 1-1.
Figure 1-1 Two test environments for running the demo

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 2
First uses one FPGA board and Test PC with 100Gb Ethernet card for transferring the data.
TestPC runs test application to transfer data with TOE100G-IP on FPGA, tcpdatatest for
half-duplex test or tcp_client_txrx_40G for full-duplex test. Also, Serial console or JTAGUART is
run on Test PC to be user interface console.
Second uses two FPGA boards which may be different board model. Both boards run
TOE100G-IP demo with assigning the different initialization mode (Client, Server, or Fixed MAC)
for transferring data.

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 3
2 Test environment setup when using FPGA and PC
To run the demo on FPGAdevelopment board, please prepare following environment.
•FPGA development boards: KCU116 board, Alveo U250 card, and FB2CGHH@KU15P
card
•PC with 100 Gigabit Ethernet card
•100Gb Ethernet cable:
a) KCU116 board: 4xSFP28 transceiver (25GBASE-SR), QSFP28 transceiver
(100GBASE-SR), and MTP to 8xLC Fiber cable
b) U250 card and FB2CGHH@KU15P card: 2xQSFP28 transceiver (100GBASE-SR) and
MPO to MPO cable.
•USB cable for connecting between FPGAand PC
a) KCU116: 2 micro USB cables for programming FPGA and Serial console
b) U250 card: 1 micro USB cable for programming FPGA and Serial console
c) FB2CGHH@KU15P card: 1 mini USB cable for programming FPGA and JTAGAURT
•For FB2CGHH@KU15P card, use AB18-PCIeX16 board provided by Design Gateway
with ATX power supply to be power board for the card. More details of AB18 card are
displayed on following website.
https://dgway.com/ABseries_E.html
•Test application: “tcpdatatest.exe” and “tcp_client_txrx_40G.exe”, provided by Design
Gateway for running on Test PC
•For KCU116 and U250 card, Serial console software such as TeraTerm installed on PC.
The setting on the console is Baudrate=115,200, Data=8-bit, Non-parity and Stop=1.
•Vivado tool for programming FPGA and JTAGUART (when using FB2CGHH@KU15P
card), installed on PC
Note: Example hardware for running the demo is listed as follows.
[1] 100G Network Adapter: NVIDIA Mellanox ConnectX-6 Ethernet Adapter Card
https://store.mellanox.com/products/nvidia-mcx614106a-ccat-connectx-6-en-adapter-ca
rd-100gbe-dual-port-qsfp56-socket-direct-2x-pcie3-0-x16-tall-brackets.html
[2] a) 4xSFP28 to QSFP28 connection
SFP28 Transceiver: AZS85-S28-M1
https://www.sfpcables.com/25gb-s-sfp28-sr-transceiver-850nm-up-to-100m-2866
QSFP28 Transceiver: AMQ28-SR4-M1
https://www.sfpcables.com/100gb-s-qsfp28-sr4-optical-transceiver-module-1499
MTP to 8xLC Fiber cable: OM4-MTP-8LC-1M
https://www.fs.com/products/68047.html
b) QSFP28 to QSFP28 connection
QSFP28 Transceiver: AMQ28-SR4-M1
https://www.sfpcables.com/100gb-s-qsfp28-sr4-optical-transceiver-module-1499
MPO to MPO cable: OM4-MPO-8MPO-1M
https://www.sfpcables.com/mpo-to-mpo-multimode-om4-50-125-m-8-core-4381
[3] PC: Motherboard Gigabyte B460M AORUS PRO, 32 GB RAM, and 64-bit Windows10 OS

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 4
Figure 2-1 TOE100G-IP with CPU demo (FPGA <-> PC) on KCU116

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 5
Figure 2-2 TOE100G-IP with CPU demo (FPGA <-> PC) on U250 card

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 6
Figure 2-3 TOE100G-IP with CPU demo (FPGA <-> PC) on FB2CGHH@KU15P

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 7
The step to setup test environment by using FPGA and PC is described in more details as
follows.
1) Connect USB cables between FPGA and PC for JTAG programming and Serial
console/JTAGUART.
a) For KCU116 board, connect two micro USB cables
b) For U250 card, connect one micro USB cable
c) For FB2CGHH@KU15P card, connect one mini USB cable
2) Connect power supply to FPGA development board/FPGA accelerator card.
a) For KCU116 board, connect Xilinx power adapter
b) For U250 card, connect the card to PC
c) For FB2CGHH@KU15P card: connect the card to PC or use AB18-PCIeX16 board by
following step.
i) Confirm that two mini jumpers are inserted at J5 connector on AB18
ii) Connect ATX power supply to AB board
iii) Connect PCIe connector on FPGA board to Device Side (B-Side), as shown in
Figure 2-4
Figure 2-4 Set up AB18-PCIeX16 Connection for FB2CGHH@KU15P card

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 8
3) Connect 100Gb Ethernet cable between FPGA board and PC.
a) For KCU116, insert four SFP28 transceivers and MTP to 8xLC Fiber cable on FPGA
board. Please check channel number of four cables to match with Figure 2-5.
Figure 2-5 100Gb connection on KCU116 board by QSFP28 to 4xSFP28 cable

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 9
b) For Alveo U250 and FB2CGHH@KU15P card, insert 100G QSFP28 Transceiver and
MPO to MPO cable by using QSFP1 connector, as shown in Figure 2-6.
Figure 2-6 100Gb connection on U250 card and FB2CGHH@KU15P card
by QSFP28 transceiver and MPO to MPO cable

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 10
4) Another side of MTP/MPO cable, insert QSFP28 transceiver to connect with 100Gb
Ethernet card on PC.
5) Power on FPGA board or PC which connects the FPGA card.
6) For KCU116 and U250 card, open Serial console and download configuration file with
firmware by following step.
i) Open Serial console. When connecting FPGA board to PC, many COM ports from
FPGA connection are detected and displayed on Device Manager.
For KCU116, select Standard COM port, as shown in the left window of Figure 2-7.
For U250, select the 2nd USB Serial port, as shown in the right window of Figure 2-7.
On Serial console, use following setting: Buad rate=115,200, Data=8-bit, Non-Parity,
and Stop = 1.
Figure 2-7 COM port number for Serial console

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 12
7) For FB2CGHH@KU15P card, open vivado TCL shell and browse to the directory that
include batch file, bit file, and elf file of TOE100G demo. After that, run the test by typing
following command.
i) >> TOE100CPUTest_Silicom.bat
Note: This step is to download configuration file and firmware, as shown in Figure 2-9.
Figure 2-9 Command script to download demo file on Vivado TCL shel
ii) >> xsdb.bat
iii) >> connect -url tcp:127.0.0.1:3121
iv) >> targets -set -filter {name =~"*Debug*"}
v) >> jtagterminal -start
vi) >> con
Note: Above steps are to connect JTAGUART module and run JTAG terminal to be user
console, as shown in Figure 2-10.
Figure 2-10 Open JTAG Terminal

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 13
8) On Serial console/JTAG Terminal, welcome message is displayed.
i) Input ‘0’ to start TOE100G-IP initialization in client mode (asking PC MAC address by
sending ARP request).
ii) Default parameter in client mode is displayed on the console.
Figure 2-11 Message after system boot-up
If Ethernet connection has the problem and the linked is down, the error message is
displayed on the console instead of welcome message, as shown in Figure 2-12.
Figure 2-12 Error message when cable is linked down
iii) User enters ‘x’ to skip parameter setting for using default parameters to begin system
initialization, as shown in Figure 2-13. If user enters other keys, the menu for changing
parameter is displayed, similar to “Reset TCPIP parameters”menu. The example
when running the main menu is described in “dg_toe100gip_cpu_instruction”
document.
Figure 2-13 Initialization complete
Note: Transfer performance in the demo is limited by Test PC performance in Test platform.
The best performance can be achieved when the test is run by using FPGA-to-FPGA
connection.

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 14
3 Test environment setup when using two FPGAs
Before running the test, please prepare following test environment.
•Two FPGA development boards which can be the same board or different board: KCU116
board, Alveo U250 accelerator card, and FB2CGHH@KU15P card
•100Gb Ethernet cable: 100Gb transceiver (BASE-SR) and fiber cable
a) KCU116 board: Use 4xSFP28 transceiver (25GBASE-SR) with 8xLC Fiber cable
b) U250 card and FB2CGHH@KU15P card: Use QSFP28 transceiver (100GBASE-SR)
with MTP/MPO Fiber cable
•USB cable for connecting between FPGAand PC
a) KCU116: 2 micro USB cables for programming FPGA and Serial console
b) U250 card: 1 micro USB cable for programming FPGA and Serial console
c) FB2CGHH@KU15P card: 1 mini USB cable for programming FPGA and JTAGAURT
•For KCU116 and U250 card, Serial console software such as TeraTerm, installed on PC.
The setting on the console is Baudrate=115,200, Data=8-bit, Non-parity, and Stop=1.
•Vivado tool for programming FPGA, installed on PC

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 15
Figure 3-1 TOE100G-IP with CPU demo (FPGA<->FPGA)

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 16
The step to setup test environment by using two FPGAs is described in more details as
follows.
Follow step 1) –7) of topic 2 (Test environment setup when using FPGA and PC) to prepare
FPGA board and SFP28/QSFP28 connection for running the demo. After two FPGA boards
have been configured completely, Serial console/JTAG Terminal displays the menu to select
Client mode, Server mode, or Fixed MAC mode. The step after FPGA configuration is
described as follows.
1) Open Serial console/JTAG Terminal for board#1 and board#2 and then set the input to the
console for selecting the initialization mode. The example to initialize by Server-Client
mode is described as following steps.
i) Set ‘1’ on Serial console of FPGA board#1 for running Server mode.
ii) Set ‘0’ on Serial console of FPGA board#2 for running Client mode.
iii) Default parameters for Server or Client are displayed on the console, as shown in
Figure 3-2.
Figure 3-2 Input mode

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 17
2) Input ‘x’ to use default parameters or other keys to change parameters. The parameters of
Server mode must be set before Client mode.
i) Set parameters on Server console.
ii) Set parameters on Client console to start IP initialization by transferringARP packet.
iii) After finishing initialization process, “IP initialization complete” and main menu are
displayed on Server console and Client console.
Figure 3-3 Main menu

dg_toe100gip_fpgasetup_xilinx.doc
28-Apr-21 Page 18
4 Revision History
Revision
Date
Description
1.0
25-Feb-21
Initial version release
1.1
28-Apr-21
Support Silicom FB2CGHH@KU15P card
Other manuals for FPGA
6
Other DG Computer Hardware manuals
Popular Computer Hardware manuals by other brands

Behringer
Behringer ULTRA-CURVEPRO DEQ2496 user manual

ICS
ICS IM3 user manual

Curtiss-Wright
Curtiss-Wright DuraCOR 310 user manual

PLX Technology
PLX Technology PEX 8604 quick start guide

Kingston Technology
Kingston Technology KTM-DC16 installation guide

D-Link
D-Link Express EtherNetwork DFE-670TXD Quick install guide

Supermicro
Supermicro AOC-SLG4-4X4T user guide

VIA Technologies
VIA Technologies AMOS-820 SKU Guide

Noctua
Noctua NF-A9 14 PWM Series user manual

Bitspower
Bitspower VGA Water Block instruction manual

Everex
Everex EV-159 Owner's manual and reference guide

NXP Semiconductors
NXP Semiconductors OM13488 user manual