DG TOE10G-IP User manual

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 1
FPGA setup TOE/UDP10G-IP with CPU Demo
Rev3.0 26-Aug-20
This document describes how to setup FPGAboard and prepare the test environment for running
TOE10G-IP or UDP10G-IP demo. The user can setup two test environments for transferring TCP
data or UDP data via 10Gb Ethernet connection by using TOE10G-IP or UDP10G-IP, as shown in
Figure 1-1.
Figure 1-1 Two test environments for running the demo
First uses one FPGAboard and Test PC with 10Gb Ethernet card for transferring the data. TestPC
runs test application, i.e. tcpdatatest (half-duplex test for TOE10G-IP), tcp_client_txrx_40G
(full-duplex test for TOE10G-IP) or udpdatatest (test application for UDP10G-IP). Also, NiosII
terminal is run on Test PC to be user interface console.
Second uses two FPGA boards which may be different board. Both boards run TOE10G-IP or
UDP10G-IP demo with assigning the different initialization mode (Client or Server) for transferring
data.

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 2
1 Test environment setup when using FPGA and PC
Before running the test, please prepare following test environment.
•FPGA development board: Arria10 SoC/Arria10 GX/Cyclone10 GX/Stratix10 GX (H-Tile)
development board
•PC with 10 Gigabit Ethernet or connecting with 10 Gigabit Ethernet card
•10Gb Ethernet cable:
a) 10 Gb SFP+ Passive Direct Attach Cable (DAC) which has 1-m or less length
b) 10 Gb SFP+ Active Optical Cable (AOC)
c) 2x10 Gb SFP+ transceivers (10G BASE-R) with optical cable (LC to LC, Multimode)
d) For Stratix10 GX board only, QSFP+ to four SFP+ cable
•micro USB cable for JTAG connection
•Test application provided by Design Gateway for running on Test PC:
TOE10G-IP: “tcpdatatest.exe” and “tcp_client_txrx_40G.exe”
UDP10G-IP: “udpdatatest.exe”
•QuartusII Programmer and NiosII command shell, installed on PC
Note: Example hardware for running the demo is listed as follows.
[1] 10G Network Adapter: Intel X520-DA2
http://www.intel.com/content/www/us/en/network-adapters/converged-network-adapters/
ethernet-x520-server-adapters-brief.html
[2] a) 10-Gigabit SFP+ AOC cable (AOC-S1S1-001)
https://www.10gtek.com/10gsfp+aoc
b) 40-Gigabit QSFP+ to 4x10-Gigabit SFP+ cable
https://www.finisar.com/active-optical-cables/fcbn510qe2cxx
[3] PC: Motherboard ASUS Z170-K, 32 GB RAM, and 64-bit Windows7 OS

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 3
Figure 1-1 TOE10G-IP/UDP10G-IP with CPU demo (FPGA<->PC) on Arria10 SoC
Note: Four LEDs are applied to show IP timeout status when the configuration file of the demo
uses 1-hour timeout TOE10G-IP/UDP10G-IP. After running for 1 hour, the IP stops the operation.
All LEDs are blinked to notify that the IPnow is timeout. User needs to reconfigure FPGAto restart
the test.

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 4
Figure 1-2 TOE10G-IP/UDP10G-IP with CPU demo (FPGA<->PC) on Arria10 GX

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 5
Figure 1-3 TOE10G-IP/UDP10G-IP with CPU demo (FPGA<->PC) on Cyclone10 GX

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 6
Figure 1-4 TOE10G-IP/UDP10G-IP with CPU demo (FPGA<->PC) on Stratix10 GX

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 7
The step to setup test environment by using FPGA and PC is described in more details as
follows.
1) Turn off power switch and connect power supply to FPGA board.
2) Connect micro USB cable from FPGA board to PC for JTAG programming and JTAG
UART.
Figure 1-5 Power connection and microUSB connection
3) Connect 10Gb Ethernet cable between FPGA board and PC.
a) For every board except Stratix10 GX board, insert 10 Gb SFP+ DAC (Length<1m),
AOC or SFP+ transceiver with LC-LC cable) between FPGA board and PC.
b) For Stratix10 GX board, insert QSFP+ to 4 SFP+ cable between FPGAboard and PC.
Use SFP+ no.1 to connect to QSFP1, connector on the right side, as shown in
Figure 1-6 10Gb Ethernet connection

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 8
4) Turn on power switch on FPGA board.
5) For Arria10 SoC board, set programmable clock to 322.265625 MHz by using “Clock
Control” application as following step.
a. Open “Clock Controller” application.
b. Select Si5338 tab (U50) and set CLK3 frequency = 322.265625 MHz.
c. Click “Set” button and wait until the application is active again.
d. Close Clock controller application.
Figure 1-7 Reference clock programming

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 9
6) Open QuartusII Programmer to program FPGA through USB-1 by following step.
a. Click “Hardware Setup…” to select USB-BlasterII[USB-1].
b. Click “Auto Detect” and select FPGA number.
c. Select Arria 10/Cyclone 10/Stratix 10 device icon.
d. Click “Change File” button, select SOF file in pop-up window and click “open” button.
e. Check “program”.
f. Click “Start” button to program FPGA.
g. Wait until Progress status is equal to 100%.
Figure 1-8 FPGA Programmer

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 10
7) Open NiosII command shell.
a. Type “nios2-terminal” to run the console.
Figure 1-9 Run NiosII terminal
b. Input ‘0’ to initialize TOE10G-IP/UDP10G-IP in client mode (asking PC MAC address
by sending ARP request).
c. Default parameter in client mode is displayed on the console.
Figure 1-10 Message after system boot-up
If Ethernet connection has the problem and the status is linked down, the error message
is displayed on the console instead of welcome message, as shown in Figure 1-11.
Figure 1-11 Error message when cable is linked down

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 11
d. User enters ‘x’ to skip parameter setting for using default parameters to begin system
initialization, as shown in Figure 1-12. If user enters other keys, the menu for changing
parameter is displayed, similar to “Reset TCPIP parameters”menu. The example
when running the main menu is described in “dg_toe10gip_cpu_instruction”or
“dg_udp10gip_cpu_instruction”document.
Figure 1-12 Initialization complete
Note: Transfer performance in the demo depends on Test PC specification in Test platform.

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 12
2 Test environment setup when using two FPGAs
Before running the test, please prepare following test environment.
•Two FPGA development boards which are the same board or different board,
Arria10 SoC/Arria10 GX/Cyclone10 GX/Stratix10 GX (H-Tile) development board
•10Gb Ethernet cable:
a) 10 Gb SFP+ Active Optical Cable (AOC)
b) 2x10 Gb SFP+ transceiver (10G BASE-R) with optical cable (LC to LC, Multimode)
c) For Stratix10 GX board, QSFP+ to four SFP+ cable
•Two micro USB cables, one cable for connecting one FPGA board to PC
•QuartusII Programmer for programming FPGA and NiosII command shell, installed on PC
Figure 2-1 TOE10G-IP/UDP10G-IP with CPU demo (FPGA<->FPGA)

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 13
The step to setup test environment by using two FPGAs is described in more details as
follows.
Follow step 1) –5) of topic 1 (Test environment setup when using FPGA and PC) to prepare
FPGA board.
Warning: For Arria10 SoC board, Clock controller for programming clock to 322.265625 MHz
could be used when only one FPGAis connected to PC. User connects one micro USB cable
to set clock on one Arria 10 SoC board at a time. If two Arria 10 SoC boards are used in the
test, user must switch micro USB cable to program clock on the 2nd board after finishing the
1st board setting. After that, two micro USB cables for connecting two FPGAboards to PC are
allowed.
1) Connect 10Gb Ethernet cable between two FPGA boards.
Figure 2-2 SFP+ transceiver connection

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 14
2) Connect micro USB cable of each FPGA board to PC. After that, PC detects two
USB-Blaster cables as USB-1 and USB-2 from two USB connections with two FPGA
boards. Follow step 6) of topic 1 (Test environment setup when using FPGA and PC) for
FPGA configuration.
Figure 2-3 Two USB-Blaster cables when connecting two FPGA boards to PC
3) Open QuartusII Programmer to program FPGA board#1 by using USB-1 connection and
then switch to program FPGA board#2 by using USB-2 connection.
Figure 2-4 Select USB-BlasterII

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 15
4) Open NiosII Command Shell.
a. Run nios2-terminal --cable 1 command for FPGA#1
b. Run nios2-terminal –cable 2 command for FPGA#2
Figure 2-5 Run NiosII terminal on two consoles
5) Set the input to the console.
a. Set ‘1’ on Serial console of FPGAboard#1 for running server mode.
b. Set ‘0’ on Serial console of FPGA board#2 for running client mode.
c. Default parameters for server or client are displayed on the console, as shown in
Figure 2-6.
Figure 2-6 Input mode

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 16
6) Input ‘x’ to use default parameters or other keys to change parameters. The parameters of
server mode must be set before client mode.
When running TOE10G-IP,
a. Set parameters on server Serial console.
b. Set parameters on client Serial console to start IP initialization by transferring ARP
packet.
c. After finishing initialization process. “IP initialization complete” and main menu are
displayed on server console and client console.
Figure 2-7 Main menu

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 17
When running UDP10G-IP,
a. For server mode, if user does not change default parameters, input ‘x’ to skip
parameter setting.
b. For client mode, user must change target port number (Target->FPGA) to use same
value as target port number (FPGA->Target).
c. After finishing initialization process. “IP initialization complete” and main menu are
displayed on server console and client console.
Figure 2-8 Main menu of UDP10G-IP

dg_toeudp10gip_fpgasetup_intel.doc
26-Aug-20 Page 18
3 Revision History
Revision
Date
Description
1.0
19-Mar-18
Initial version release
1.1
27-Mar-18
Add Part A (FPGA<->PC test)
1.2
4-Apr-18
Correct optical cable in Figures and the descriptions
1.3
5-Feb-19
Add Arria 10 GX board and change software to tcp_client_txrx_40G
1.4
31-May-19
Add timeout LED descriptions
1.5
20-Aug-19
Add Cyclone10 GX board
2.0
18-Jun-20
Remove test result on the console
3.0
26-Aug-20
TOE10G-IP and UDP10G-IP