Terasic DE5a-Net DDR4 Edition User manual

DE5a-NET DDR4
User Manual
1
www.terasic.com
March 21, 2018

DE5a-NET DDR4
User Manual
2
www.terasic.com
March 21, 2018
Contents
Chapter 1 Overview......................................................... 5
1.1 General Description..............................................................................................5
1.2 Key Features........................................................................................................6
1.3 Block Diagram......................................................................................................7
1.4 Operating Temperature - Important.....................................................................11
Chapter 2 Board Components ..................................... 12
2.1 Board Overview..................................................................................................12
2.2 Configuration, Status and Setup.........................................................................13
2.3 General User Input/Output .................................................................................16
2.4 Temperature Sensor and Fan Control.................................................................21
2.5 Power Monitor....................................................................................................23
2.6 Clock Circuit.......................................................................................................24
2.7 FLASH Memory..................................................................................................26
2.8 DDR4 SO-DIMM.................................................................................................29
2.9 QDRII+ SRAM....................................................................................................42
2.10 QSPF+ Ports....................................................................................................51
2.11 PCI Express .....................................................................................................55
2-12 RS-422 Expansion Header...............................................................................58
Chapter 3 System Builder ............................................ 60
3.1 Introduction......................................................................................................60
3.2 General Design Flow..........................................................................................62
3.3 Using System Builder.........................................................................................63

DE5a-NET DDR4
User Manual
3
www.terasic.com
March 21, 2018
Chapter 4 Flash Programming..................................... 69
4.1 CFI Flash Memory Map......................................................................................70
4.2 FPGA Configure Operation.................................................................................70
4.3 Flash Programming with Users Design...............................................................71
4.4 Restore Factory Settings....................................................................................73
Chapter 5 Peripheral Reference Design ..................... 75
5.1 Temperature Monitor: Board Protection..............................................................75
5.2 Configure Si5340A/B in RTL...............................................................................78
5.3 Nios II control for SI5340/Temperature/Power....................................................87
5.4 Fan Speed Control.............................................................................................92
Chapter 6 Memory Reference Design ......................... 96
6.1 QDRII+ SRAM Test ............................................................................................96
6.2 DDR4 SDRAM Test............................................................................................99
6.3 DDR4 SDRAM Test by Nios II ..........................................................................102
Chapter 7 Memory Reference Design ....................... 106
7.1 PCI Express System Infrastructure...................................................................106
7.2 PC PCI Express Software SDK........................................................................107
7.3 PCI Express Software Stack ............................................................................108
7.4 PCI Express LibraryAPI...................................................................................114
7.5 PCIe Reference Design - Fundamental............................................................119
7.6 PCIe Reference Design - DDR4.......................................................................127
Chapter 8 PCI Express Reference Design for Linux 134
8.1 PCI Express System Infrastructure...................................................................134

DE5a-NET DDR4
User Manual
4
www.terasic.com
March 21, 2018
8.2 PC PCI Express Software SDK........................................................................135
8.3 PCI Express Software Stack ............................................................................136
8.4 PCI Express LibraryAPI...................................................................................139
8.5 PCIe Reference Design - DDR4.......................................................................145
Chapter 9 Transceiver Verification............................ 153
9.1 Function of the Transceiver Test Code .............................................................153
9.2 Loopback Fixture..............................................................................................153
9.3 Testing..............................................................................................................156
9.4 40G Ethernet Example.....................................................................................158
9.5 10GBASE-R Ethernet Example........................................................................162
Additional Information................................................ 167
Getting Help...........................................................................................................167

DE5a-NET DDR4
User Manual
5
www.terasic.com
March 21, 2018
Chapter 1
Overview
his chapter provides an overview of the DE5a-Net DDR4 Edition Development
Board and installation guide.
1.1 General Description
The Terasic DE5a-Net DDR4 EditionArria 10 GX FPGA Development Kit provides the
ideal hardware solution for designs that demand high capacity and bandwidth memory
interfacing, ultra-low latency communication, and power efficiency. With a full-height,
3/4-length form-factor package, the DE5a-Net is designed for the most demanding
high-end applications, empowered with the top-of-the-line Altera Arria 10 GX,
delivering the best system-level integration and flexibility in the industry.
The Arria® 10 GX FPGA features integrated transceivers that transfer at a maximum
of 12.5 Gbps, allowing the DE5a-Net to be fully compliant with version 3.0 of the PCI
Express standard, as well as allowing an ultra low-latency, straight connections to four
external 40G QSFP+ modules. Not relying on an external PHY will accelerate
mainstream development of network applications enabling customers to deploy
designs for a broad range of high-speed connectivity applications. For designs that
demand high capacity and high speed for memory and storage, the DE5a-Net delivers
with two independent banks of DDR4 SO-DIMM RAM, four independent banks of
QDRII+ SRAM, high-speed parallel flash memory. The feature-set of the DE5a-Net
fully supports all high-intensity applications such as low-latency trading, cloud
computing, high-performance computing, data acquisition, network processing, and
signal processing.
T

DE5a-NET DDR4
User Manual
6
www.terasic.com
March 21, 2018
1.2 Key Features
The following hardware is implemented on the DE5a-Net board:
FPGA
Intel Arria® 10 GX FPGA (10AX115N2F45E1SG)
FPGA Configuration
On-Board USB Blaster II or JTAG header for FPGA programming
Fast passive parallel (FPPx32) configuration via MAX II CPLD and flash memory
General user input/output:
8 LEDs
4 push-buttons
2 slide switches
2 seven-segment displays
Clock System
50MHz Oscillator
Programmable clock generators Si5340A and Si5340B
One SMA connector for external clock input
One SMA connector for clock output
Memory
DDR4 SO-DIMM SDRAM
QDRII+ SRAM
FLASH

DE5a-NET DDR4
User Manual
7
www.terasic.com
March 21, 2018
Communication Ports
Four QSFP+ connectors
PCI Express (PCIe) x8 edge connector
One RS422 transceiver with RJ45 connector
System Monitor and Control
Temperature sensor
Fan control
Power monitor
Power
PCI Express 6-pin power connector, 12V DC Input
PCI Express edge connector power
Mechanical Specification
PCI Express full-height and 3/4-length
1.3 Block Diagram
Figure 1-1 shows the block diagram of the DE5a-Net board. To provide maximum
flexibility for the users, all key components are connected with the Arria 10 GX FPGA
device. Thus, users can configure the FPGAto implement any system design.

DE5a-NET DDR4
User Manual
8
www.terasic.com
March 21, 2018
Figure 1-1 Block diagram of the DE5a-Net board
Below is more detailed information regarding the blocks in Figure 1-1.
Arria 10 GX FPGA
10AX115N2F45E1SG
1,150K logic elements (LEs)
67-Mbits embedded memory
48 transceivers (12.5Gbps)
3,036 18-bit x 19-bit multipliers
1,518 Variable-precision DSP blocks
4 PCI Express hard IP blocks
768 user I/Os

DE5a-NET DDR4
User Manual
9
www.terasic.com
March 21, 2018
384 LVDS channels
32 phase locked loops (PLLs)
JTAG Header and FPGA Configuration
On-board USB Blaster II or JTAG header for use with the Quartus Prime
Programmer
MAXII CPLD 5M2210 System Controller and Fast Passive Parallel (FPP x32)
configuration
Memory Devices
32MB QDRII+ SRAM
Up to 16GB DDR4 SO-DIMM SDRAM for each DDR4 socket
256MB FLASH
General User I/O
8 user controllable LEDs
4 user push buttons
2 user slide switches
2 seven-segment displays
On-Board Clock

DE5a-NET DDR4
User Manual
10
www.terasic.com
March 21, 2018
50MHz oscillator
Programming PLL providing clock for 40G QSFP+ transceiver
Programming PLL providing clock for PCIe transceiver
Programming PLL providing clocks for DDR4 SDRAM and QDRII+ SRAM
Four QSFP+ Ports
Four QSFP+ connector (40 Gbps+)
PCI Express x8 Edge Connector
Support for PCIe x8 Gen1/2/3
Edge connector for PC motherboard with x8 or x16 PCI Express slot
Power Source
PCI Express 6-pin DC 12V power
PCI Express edge connector power
Temperature Range
FPGA: 0°C ~100°C

DE5a-NET DDR4
User Manual
11
www.terasic.com
March 21, 2018
1.4 Operating Temperature - Important
Please read the following instructions carefully to prevent damage to your DE5a-NET
board.
The operating temperature range of Arria 10 GX device on DE5a-NET-DDR4 is 0°C
~100°C. When the FPGA temperature stays over 100°C for a long time, the FPGA
could be damaged. It is therefore strongly recommended to use this board in an
environment with sufficient airflow to dissipate the heat generated. It is also
recommended to monitor the FPGA temperature continuously by adding Terasic IP
introduced in chapter 5.1 in the project. When the FPGA temperature is getting close
to 100°C, please turn off the board immediately to reduce the FPGAtemperature and
protect the FPGA device.
Please refer to the directory /Demonstrations/Board_Protection in the DE5a-NET
System CD for details about the Verilog based temperature monitor design example.

DE5a-NET DDR4
User Manual
12
www.terasic.com
March 21, 2018
Chapter 2
Board Components
his chapter introduces all the important components on the DE5a-Net.
2.1 Board Overview
Figure 2-1 is the top and bottom view of the DE5a-Net development board. It depicts the
layout of the board and indicates the location of the connectors and key components.
Users can refer to this figure for relative location of the connectors and key components.
Figure 2-1 FPGA Board (Top)
T

DE5a-NET DDR4
User Manual
13
www.terasic.com
March 21, 2018
Figure 2-2 FPGA Board (Bottom)
2.2 Configuration, Status and Setup
Configure
The FPGAboard supports two configuration methods for theArria 10 FPGA:
Configure the FPGAusing the on-board USB-Blaster II.
Flash memory configuration of the FPGA using stored images from the flash
memory on power-up.
For programming by on-board USB-Blaster II, the following procedures show how to
download a configuration bit stream into the Arria 10 GX FPGA:
Make sure that power is provided to the FPGAboard
Connect your PC to the FPGAboard using a mini-USB cable and make sure the
USB-Blaster II driver is installed on PC.
Launch Quartus Prime programmer and make sure the USB-Blaster II is
detected.
In Quartus Prime Programmer, add the configuration bit stream file (.sof), check
the associated “Program/Configure” item, and click “Start” to start FPGA

DE5a-NET DDR4
User Manual
14
www.terasic.com
March 21, 2018
programming.
Status LED
The FPGA Board development board includes board-specific status LEDs to indicate
board status. Please refer to Table 2-1 for the description of the LED indicator.
Table 2-1 Status LED
Board
Reference
LED Name
Description
D6
12-V Power
Illuminates when 12-V power is active.
D5
3.3-V Power
Illuminates when 3.3-V power is active.
D16
CONF DONE
Illuminates when the FPGA is successfully
configured. Driven by the MAX II CPLD 5M2210
System Controller.
D15
Loading
Illuminates when the MAX II CPLD 5M2210 System
Controller is actively configuring the FPGA. Driven by
the MAX II CPLD 5M2210 System Controller with the
Embedded Blaster CPLD.
D17
Error
Illuminates when the MAX II CPLD EPM2210 System
Controller fails to configure the FPGA. Driven by the
MAX II CPLD EPM2210 System Controller.
D19
PAGE
Illuminates when FPGA is configured by the factory
configuration bit stream.
Setup PCI Express Control DIP switch
The PCI Express Control DIP switch (SW5) is provided to enable or disable different
configurations of the PCIe Connector. Table 2-2 lists the switch controls and description.
Table 2-2 SW5 PCIe Control DIP Switch

DE5a-NET DDR4
User Manual
15
www.terasic.com
March 21, 2018
Board
Reference
Signal Name
Description
Default
SW5.1
PCIE_PRSNT2n_x1
On : Enable x1 presence detect
Off: Disable x1 presence detect
Off
SW5.2
PCIE_PRSNT2n_x4
On : Enable x4 presence detect
Off: Disable x4 presence detect
Off
SW5.3
PCIE_PRSNT2n_x8
On : Enable x8 presence detect
Off: Disable x8 presence detect
On
Setup Configure Mode
The position 1~3 of DIP switch SW3 are used to specify the configuration mode of the
FPGA. As currently only one mode is supported, please set all positions as shown in
Figure 2-3.
Figure 2-3 Position of DIP switch SW3 for Configure Mode
Select Flash Image for Configuration
The position 4 of DIP switch SW3 is used to specify the image for configuration of the
FPGA. Setting Position 4 of SW3 to “1” (down position) specifies the default factory image
to be loaded, as shown in Figure 2-4. Setting Position 4 of SW3 to “0” (up position)

DE5a-NET DDR4
User Manual
16
www.terasic.com
March 21, 2018
specifies the DE5a-Net to load a user-defined image, as shown in Figure 2-5.
Figure 2-4 Position of DIP switch SW3 for Image Select –Factory Image Load
Figure 2-5 Position of DIP switch SW3 for Image Select –User Image Load
2.3 General User Input/Output
This section describes the user I/O interface to the FPGA.
User Defined Push-buttons

DE5a-NET DDR4
User Manual
17
www.terasic.com
March 21, 2018
The FPGAboard includes four user defined push-buttons that allow users to interact with
the Arria 10 GX device. Each push-button provides a high logic level or a low logic level
when it is not pressed or pressed, respectively. Table 2-3 lists the board references, signal
names and their correspondingArria 10 GX device pin numbers.
Table 2-3 Push-button Pin Assignments, Schematic Signal Names, and Functions
Board
Reference
Schematic
Signal
Name
Description
I/O
Standard
Arria 10 GX
Pin Number
PB0
BUTTON0
High Logic Level when the button
is not pressed
1.2-V
PIN_AJ13
PB1
BUTTON1
1.2-V
PIN_AE13
PB2
BUTTON2
1.2-V
PIN_AV16
PB3
BUTTON3
1.2-V
PIN_AR9
User-Defined Slide Switch
There are two slide switches on the FPGAboard to provide additional FPGAinput control.
When a slide switch is in the DOWN position or the UPPER position, it provides a low logic
level or a high logic level to the Arria 10 GX FPGA, respectively, as shown in Figure 2-6.

DE5a-NET DDR4
User Manual
18
www.terasic.com
March 21, 2018
Figure 2-6 2 Slide switches
Table 2-4 lists the signal names and their correspondingArria 10 GX device pin numbers.
Table 2-4 Slide Switch Pin Assignments, Schematic Signal Names, and Functions
Board
Reference
Schematic
Signal Name
Description
I/O
Standard
Arria 10 GX
Pin Number
SW0
SW0
High logic level when SW in the
UPPER position.
1.2-V
PIN_AY28
SW1
SW1
1.2-V
PIN_AM27
User-Defined LEDs
The FPGA board consists of 8 user-controllable LEDs to allow status and debugging
signals to be driven to the LEDs from the designs loaded into the Arria 10 GX device. Each
LED is driven directly by the Arria 10 GX FPGA. The LED is turned on or off when the
associated pins are driven to a low or high logic level, respectively. A list of the pin names
on the FPGAthat are connected to the LEDs is given in Table 2-5.

DE5a-NET DDR4
User Manual
19
www.terasic.com
March 21, 2018
Table 2-5 User LEDs Pin Assignments, Schematic Signal Names, and Functions
Board
Reference
Schematic
Signal Name
Description
I/O
Standard
Arria 10 GX Pin
Number
D4
LED0
Driving a logic 0 on the I/O port
turns the LED ON.
Driving a logic 1 on the I/O port
turns the LED OFF.
1.8-V
PIN_T11
D3
LED1
1.8-V
PIN_R11
D2
LED2
1.8-V
PIN_N15
D1
LED3
1.8-V
PIN_M15
D9-1
LED_BRACKET0
1.8-V
PIN_AF10
D9-3
LED_BRACKET1
1.8-V
PIN_AF9
D9-5
LED_BRACKET2
1.8-V
PIN_Y13
D9-7
LED_BRACKET3
1.8-V
PIN_W11
7-Segment Displays
The FPGA board has two 7-segment displays. As indicated in the schematic in Figure 2-
7, the seven segments are connected to pins of theArria 10 GX FPGA. Applying a low or
high logic level to a segment will turn it on or turn it off, respectively.
Each segment in a display is identified by an index listed from 0 to 6 with the positions
given in Figure 2-8. In addition, the decimal point is identified as DP. Table 2-6 shows the
mapping of the FPGA pin assignments to the 7-segment displays.

DE5a-NET DDR4
User Manual
20
www.terasic.com
March 21, 2018
Figure 2-7 Connection between 7-segment displays and Arria 10 GX FPGA
Figure 2-8 Position and index of each segment in a 7-segment display
Table 2-6 User LEDs Pin Assignments, Schematic Signal Names, and Functions
Board
Reference
Schematic
Signal
Name
Description
I/O
Standard
Arria 10
GX Pin
Number
HEX1
HEX1_D0
User-Defined 7-Segment Display.
Driving logic 0 on the I/O port turns
the 7-segment signal ON. Driving
logic 1 on the I/O port turns the 7-
segment signal OFF.
1.2-V
PIN_AM32
HEX1
HEX1_D1
1.2-V
PIN_AN32
HEX1
HEX1_D2
1.2-V
PIN_AN31
HEX1
HEX1_D3
1.2-V
PIN_AP31
HEX1
HEX1_D4
1.2-V
PIN_BA35
HEX1
HEX1_D5
1.2-V
PIN_BD34
HEX1
HEX1_D6
1.2-V
PIN_AR31
HEX1
HEX1_DP
1.2-V
PIN_BC28
HEX0
HEX0_D0
1.2-V
PIN_AW8
HEX0
HEX0_D1
1.2-V
PIN_AY8
HEX0
HEX0_D2
1.2-V
PIN_AY9
HEX0
HEX0_D3
1.2-V
PIN_BA9
HEX0
HEX0_D4
1.2-V
PIN_BB9
HEX0
HEX0_D5
1.2-V
PIN_BD10
HEX0
HEX0_D6
1.8-V
PIN_V10
HEX0
HEX0_DP
1.8-V
PIN_AG9