Terasic TR10a-LPQ User manual

1
TR10a-LPQ User Manual www.terasic.com
December 10,
2018

2
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
CONTENTS
Chapter 1 Overview................................................................................... 4
1.1 General Description............................................................................... 4
1.2 Key Features......................................................................................... 5
1.3 Block Diagram....................................................................................... 6
Chapter 2 Board Components ................................................................. 9
2.1 Board Overview..................................................................................... 9
2.2 Configuration, Status and Setup.......................................................... 10
2.3 General User Input/Output .................................................................. 14
2.4 Temperature Sensor and Fan Control.................................................. 17
2.5 Power Monitor ..................................................................................... 18
2.6 Clock Circuit........................................................................................ 19
2.7 FLASH Memory................................................................................... 22
2.8 QDRII+ SRAM..................................................................................... 24
2.9 QSPF+ Ports....................................................................................... 35
2.10 PCI Express ...................................................................................... 37
2.11 2x5 Timing Header ............................................................................ 40
Chapter 3 Flash Programming ............................................................... 41
3.1 FPGA Configure Operation .............................................................. 41
3.2 CFI Flash Memory Map.................................................................... 42
3.3 Flash Example Designs.................................................................... 43
3.4 Flash_Programming Example .......................................................... 44
3.5 Flash_Factory Example.................................................................... 45
3.6 Flash_User Example........................................................................ 47
3.7 Flash_Tool Example......................................................................... 48
3.8 Programming Batch File................................................................... 48
3.9 Restore Factory Settings.................................................................. 49
Chapter 4 Peripheral Reference Design ................................................ 51
4.1 Configure Si5340A in RTL ................................................................... 51
4.2 Nios II control for SI5340..................................................................... 57
Chapter 5 Memory Reference Design.................................................... 60
5.1 QDRII+ SRAM Test ............................................................................. 60
5.2 QDRII+ SRAM Test by Nios II.............................................................. 63
Chapter 6 PCI Express Reference Design............................................. 67
6.1 PCI Express System Infrastructure...................................................... 67
6.2 PC PCI Express Software SDK ........................................................... 68
6.3 PCI Express Software Stack ............................................................... 69
6.4 PCIe Design - Fundamental ................................................................ 79
6.5 PCIe Design – QDRII+ ........................................................................ 86
6.6 PCIe Design: PCIe_Fundamental_x2.................................................. 97
Chapter 7 Transceiver Verification ...................................................... 105
7.1 Function of the Transceiver Test Code .............................................. 105
7.2 Loopback Fixture............................................................................... 105
7.3 Testing............................................................................................... 107
Chapter 8 TR10a-LPQ Dashboard........................................................ 109

3
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
8.1 Dashboard Connected via USB Blaster II...........................................110
8.2 Dashboard Connected via UART .......................................................116
Additional Information................................................................................ 128
Getting Help............................................................................................ 128

4
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Chapter 1
Overview
his chapter provides an overview of the TR10a-LPQ Development Board and
installation guide.
1.1 General Description
The Terasic TR10a-LPQ Arria 10 GX FPGA Development Kit provides the ideal
hardware solution for designs that demand high capacity and bandwidth memory
interfacing, ultra-low latency communication, and power efficiency. With a Low-Profile
form-factor package, the TR10a-LPQ is designed for the most demanding high-end
applications, empowered with the top-of-the-line Altera Arria 10 GX, delivering the best
system-level integration and flexibility in the industry.
The Arria® 10 GX FPGA features integrated transceivers that transfer at a maximum of
12.5 Gbps, allowing the TR10a-LPQ to be fully compliant with version 3.0 of the PCI
Express standard, as well as allowing an ultra low-latency, straight connections to two
external 40G QSFP+ modules. Not relying on an external PHY will accelerate
mainstream development of network applications enabling customers to deploy designs
for a broad range of high-speed connectivity applications. For designs that demand high
capacity and high speed for memory and storage, the TR10a-LPQ delivers with five
independent banks of QDRII+ SRAM, high-speed parallel flash memory. The feature-
set of the TR10a-LPQ fully supports all high-intensity applications such as low-latency
trading, cloud computing, high-performance computing, data acquisition, network
processing, and signal processing.
T

5
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
1.2 Key Features
The following hardware is implemented on the TR10a-LPQ board:
◼FPGA
⚫Altera Arria® 10 GX FPGA (10AX115N2F45E1SG)
◼FPGA Configuration
⚫On-Board USB Blaster II for FPGA programming
⚫Fast passive parallel (FPPx16) configuration via MAX II CPLD and flash
memory
◼General user input/output
⚫4 LEDs
⚫2 push-buttons
⚫2 dip switches
◼Clock System
⚫50MHz Oscillator
⚫Programmable clock generators Si5340A and Si53306
◼Memory
⚫QDRII+ SRAM
⚫FLASH
◼Communication Ports
⚫Two QSFP+ connectors
⚫Dual PCI Express (PCIe) x8 edge connector
⚫One 2x5 GPIO timing expansion header
◼System Monitor and Control
⚫Temperature sensor
⚫Fan control
⚫Power monitor
⚫UART to USB for board management

6
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
◼Power
⚫PCI Express 4-pin power connector, 12V DC Input
⚫PCI Express edge connector power
◼Mechanical Specification
⚫PCI Express Low-Profile x16
1.3 Block Diagram
Figure 1-1 shows the block diagram of the TR10a-LPQ board. To provide maximum
flexibility for the users, all key components are connected to the Arria 10 GX FPGA
device. Thus, users can configure the FPGA to implement any system design.
Figure 1-1 Block diagram of the TR10a-LPQ board
Below is more detailed information regarding the blocks in Figure 1-1.
◼Arria 10 GX FPGA
⚫10AX115N2F45E1SG
⚫1,150K logic elements (LEs)

7
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
⚫67-Mbits embedded memory
⚫48 transceivers (12.5Gbps)
⚫3,036 18-bit x 19-bit multipliers
⚫1,518 Variable-precision DSP blocks
⚫4 PCI Express hard IP blocks
⚫768 user I/Os
⚫384 LVDS channels
⚫32 phase locked loops (PLLs)
◼FPGA Configuration
⚫On-board USB Blaster II for use with the Quartus Prime Programmer
⚫MAX 10 CPLD System Controller and Fast Passive Parallel (FPP x16)
configuration
◼Memory devices
⚫40MB QDRII+ SRAM
⚫128MB FLASH
◼General user I/O
⚫4 user controllable LEDs
⚫2 user push buttons
⚫2 user dip switches
◼On-Board Clock
⚫50MHz oscillator
⚫Programming PLL providing clock for 40G QSFP+ transceiver
⚫Programming PLL providing clock for PCIe transceiver
⚫Programming PLL providing clocks for QDRII+ SRAM
◼Four QSFP+ ports
⚫Two QSFP+ connector (40 Gbps+)
◼Dual PCI Express x8 edge connector
⚫Support for Dual PCIe x8 Gen1/2/3
⚫Edge connector for PC motherboard with x16 PCI Express slot

8
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
◼System Monitor and Control
⚫Temperature sensor
⚫Fan control
⚫Power monitor
⚫UART to USB for board management
◼Power Source
⚫PCI Express 4-pin DC 12V power
⚫PCI Express edge connector power

9
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Chapter 2
Board Components
his chapter introduces all the important components on the TR10a-LPQ.
2.1 Board Overview
Figure 2-1 and Figure 2-2 are the top and bottom view of the TR10a-LPQ development
board. It depicts the layout of the board and indicates the location of the connectors and
key components. Users can refer to this figure for relative location of the connectors
and key components.
Figure 2-1 FPGA Board (Top)
T

10
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Figure 2-2 FPGA Board (Bottom)
2.2 Configuration, Status and Setup
◼Configure
The FPGA board supports two configuration methods for the Arria 10 FPGA:
⚫Configure the FPGA using the on-board USB-Blaster II.
⚫Flash memory configuration of the FPGA using stored images from the flash
memory on power-up.
For programming by on-board USB-Blaster II, the following procedures show how to
download a configuration bit stream into the Arria 10 GX FPGA:
⚫Make sure that power is provided to the FPGA board
⚫Connect your PC to the FPGA board using a micro-USB cable and make sure
the USB-Blaster II driver is installed on PC.
⚫Launch Quartus Prime programmer and make sure the USB-Blaster II is
detected.
⚫In Quartus Prime Programmer, add the configuration bit stream file (.sof),
check the associated “Program/Configure” item, and click “Start” to start
FPGA programming.

11
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
◼Status LED
The FPGA Board development board includes board-specific status LEDs to indicate
board status as shown in Figure 2-3. Please refer to Table 2-1 for the description of the
LED indicator.
Figure 2-3 Status LED on the TR10a-LPQ
Table 2-1 Status LED
Board
Reference
LED Name
Description
D10
12-V Power
Illuminates when 12-V power is active.
D9
3.3-V Power
Illuminates when 3.3-V power is active.
D6
LED_MAX_CONF_DONE
Illuminates when the FPGA is successfully
configured. Driven by the MAX 10 CPLD
System Controller.
D8
LED_MAX_LOAD
Illuminates when the MAX 10 CPLD System
Controller. is actively configuring the FPGA.
Driven by the MAX 10 CPLD System
Controller. with the Embedded Blaster CPLD.
D7
LED_MAX_ERROR
Illuminates when the MAX 10 CPLD System
Controller. fails to configure the FPGA. Driven

12
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
by the MAX 10 CPLD System Controller.
D5
LED_BOOT_PAGE
Illuminates when FPGA is configured by the
factory configuration bit stream.
D3
LED_PR_DONE
Illuminates when FPGA partial reconfiguration
is done
D4
LED_CvP_CONFIG_DONE
Illuminates when FPGA Configuration via
Protocol (CvP) is done
LED6
LED_FPGA_PDN
Illuminates when the temperature of the FPGA
is too high and exceeds the set value, the
FPGA power is automatically turned off.
D1
JTAG_LED
Indicates transmit or receive activity of the
JTAG chain. The LED flickers if the link is in
use and active.
D2
FAN_ALERT_n
Illuminates when the temperature of the FPGA
exceeds the set value.
◼Setup PCI Express Control DIP switch
The PCI Express Control DIP switch (S1) is provided to enable or disable different
configurations of the PCIe Connector. Table 2-2 lists the switch controls and description.
Figure 2-4 Setup PCI Express Control DIP switch

13
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Table 2-2 S1 PCIe Control DIP Switch
Board
Reference
Signal Name
Description
Default
S1.5
PCIE_PRSNT2n_x1
On : Enable x1 presence detect
Off: Disable x1 presence detect
Off
S1.6
PCIE_PRSNT2n_x4
On : Enable x4 presence detect
Off: Disable x4 presence detect
Off
S1.7
PCIE_PRSNT2n_x8
On : Enable x8 presence detect
Off: Disable x8 presence detect
Off
S1.8
PCIE_PRSNT2n_x16
On: Enable dual x8 presence
detect
Off: Disable dual x8 presence
detect
On
◼Setup Configure Mode
The position 1~3 of DIP switch S1 are used to specify the configuration mode of the
FPGA. As currently only one mode is supported, please set all positions as shown in
Figure 2-5.
Figure 2-5 Position of DIP switch S1 for Configure Mode
◼Select Flash Image for Configuration
The position 4 of DIP switch S1 is used to specify the image for configuration of the
FPGA. Setting Position 4 of S1 to “1” (down position) specifies the default factory image
to be loaded, as shown in Figure 2-6. Setting Position 4 of S1 to “0” (up position)
specifies the TR10a-LPQ to load a user-defined image, as shown in Figure 2-7.

14
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Figure 2-6 Position of DIP switch S1 for Image Select – Factory Image Load
Figure 2-7 Position of DIP switch S1 for Image Select – User Image Load
2.3 General User Input/Output
This section describes the user I/O interface to the FPGA.
◼User Defined Push-buttons
The FPGA board includes four user defined push-buttons that allow users to interact
with the Arria 10 GX device. Each push-button provides a high logic level or a low logic
level when it is not pressed or pressed, respectively. Table 2-3 lists the board references,
signal names and their corresponding Arria 10 GX device pin numbers.

15
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Figure 2-8 User Defined Buttons
Table 2-3 Push-button Pin Assignments, Schematic Signal Names, and
Functions
Board
Reference
Schematic
Signal Name
Description
I/O
Standard
Arria 10 GX
Pin Number
PB0
BUTTON0
High Logic Level when the
button is not pressed
1.8-V
PIN_AR6
PB1
BUTTON1
1.8-V
PIN_AP6
PB4
CPU_RESET_n
1.8-V
AP24
◼User-Defined Dip Switch
There are two dip switches on the FPGA board to provide additional FPGA input control.
When a dip switch is in the DOWN position or the UPPER position, it provides a high
logic level or a low logic level to the Arria 10 GX FPGA, respectively, as shown in Figure
2-9.

16
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Figure 2-9 2 Dip switches
Table 2-4 lists the signal names and their corresponding Arria 10 GX device pin
numbers.
Table 2-4 Dip Switch Pin Assignments, Schematic Signal Names, and Functions
Board
Reference
Schematic
Signal Name
Description
I/O
Standard
Arria 10 GX
Pin Number
SW0
SW0
High logic level when SW in the
UPPER position.
1.8-V
PIN_ AU35
SW1
1.8-V
PIN_AH33
◼User-Defined LEDs
The FPGA board consists of 4 user-controllable LEDs to allow status and debugging
signals to be driven to the LEDs from the designs loaded into the Arria 10 GX device.
Each LED is driven directly by the Arria 10 GX FPGA. The LED is turned on or off when
the associated pins are driven to a low or high logic level, respectively, as shown in
Figure 2-10. A list of the pin names on the FPGA that are connected to the LEDs is
given in Table 2-5.

17
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Figure 2-10 Four User LEDs
Table 2-5 User LEDs Pin Assignments, Schematic Signal Names, and Functions
Board
Reference
Schematic
Signal Name
Description
I/O
Standard
Arria 10 GX
Pin Number
LED0
LED0
Driving a logic 0 on the I/O port
turns the LED ON.
Driving a logic 1 on the I/O port
turns the LED OFF.
1.8-V
PIN_Y13
LED1
LED1
1.8-V
PIN_Y14
LED2
LED2
1.8-V
PIN_W11
LED3
LED3
1.8-V
PIN_V10
2.4 Temperature Sensor and Fan Control
The TR10a-LPQ has an automatic management system for the temperature of the
FPGA, the fan speed, and the power supply of the FPGA. As shown in Figure 2-11. The
MAX10 FPGA is the main control device on the board.
The temperature of the Arria 10 FPGA is obtained through the TMP441 temperature
sensor and the MAX10 FPGA can read it from TMP441 via I2C bus. The MAX10 FPGA
will adjust the fan speed automatically according to the temperature of the Arria 10
FPGA.
When the board power is on, the fan speed is preset to 3000 rpm. As the temperature

18
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
of the Arria10 FPGA rises, the MAX10 will increase the fan speed to decrease the
temperature of the FPGA.
In order to avoid the failure of the fan on the FPGA and stop the rotation and cause the
FPGA temperature to rise rapidly. When the FPGA temperature reaches 95 degrees
(The setting threshold value), the MAX10 will cut off the power supply to the Arria10
FPGA to avoid damage caused by excessive FPGA temperature.
Figure 2-11 The Temperature, Fan Speed and FPGA power control system
2.5 Power Monitor
The TR10a-LPQ has implemented a power monitor chip to monitor the board input
power voltage and current. Figure 2-12 shows the connection between the power
monitor chip and the Arria 10 GX FPGA. The power monitor chip monitors both shunt
voltage drops and board input power voltage allows user to monitor the total board
power consumption. Programmable calibration value, conversion times, and averaging,
combined with an internal multiplier, enable direct readouts of current in amperes and
power in watts. Table 2-6 shows the pin assignment of power monitor I2C bus.

19
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Figure 2-12 Connections between the Power Monitor chip and the Arria 10 GX
FPGA
Table 2-6 Pin Assignment of Power Monitor I2C bus
Schematic
Signal Name
Description
I/O
Standard
Arria 10 GX
Pin Number
POWER_MONITOR_I2C_SCL
Power Monitor SCL
1.8V
PIN_AT26
POWER_MONITOR_I2C_SDA
Power Monitor SDA
1.8V
PIN_AP25
POWER_MONITOR_ALERT_N
Power Monitor ALERT
1.8V
PIN_BD23
2.6 Clock Circuit
The development board includes four 50 MHz oscillators and two programmable clock
generators. Figure 2-13 shows the default frequencies of on-board all external clocks
going to the Arria 10 GX FPGA.

20
TR10a-LPQ User Manual www.terasic.com
December 10,
2018
Figure 2-13 Clock circuit of the FPGA Board
A clock buffer is used to duplicate the 50 MHz oscillator, so there are six 50MHz clocks
fed into different five FPGA banks. The two programming clock generators are low-jitter
oscillators which are used to provide special and high quality clock signals for high-
speed transceivers and high bandwidth memory. Through I2C serial interface, the clock
generator controllers in the Arria 10 GX FPGA can be used to program the Si5340A and
Si5340B to generate 40G Ethernet QSFP+ and high bandwidth memory reference
clocks respectively.
Table 2-7 lists the clock source, signal names, default frequency and their
corresponding Arria 10 GX device pin numbers.
Table 2-7 Clock Source, Signal Name, Default Frequency, Pin Assignments and
Functions
Source
Schematic
Signal Name
Default
Frequency
I/O
Standard
Arria 10 GX
Pin
Number
Application
U38
CLK_50_B2H
50.0 MHz
1.8V
PIN_AP34
User Application
CLK_50_B2I
1.8V
PIN_C30
User Application
CLK_50_B2J
1.8V
PIN_W36
User Application
CLK_50_B3C
1.8V
PIN_AK12
User Application
CLK_50_B3D
1.8V
PIN_AJ11
User Application
Y6
CLK_100_B2J
100.0MHz
1.8V
PIN_AC32
User Application
Y2
OSC_100_CLKUSR
100.0MHz
1.8V
PIN_AV26
User-supplied