Achronix Speedster7t GDDR6 User manual

www.achronix.com
Speedster7t GDDR6 User
Guide (UG091)
Speedster FPGAs

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 2
Copyrights, Trademarks and Disclaimers
Copyright © 2019 Achronix Semiconductor Corporation. All rights reserved. Achronix, Speedcore, Speedster,
and ACE are trademarks of Achronix Semiconductor Corporation in the U.S. and/or other countries All other
trademarks are the property of their respective owners. All specifications subject to change without notice.
NOTICE of DISCLAIMER: The information given in this document is believed to be accurate and reliable.
However, Achronix Semiconductor Corporation does not give any representations or warranties as to the
completeness or accuracy of such information and shall have no liability for the use of the information contained
herein. Achronix Semiconductor Corporation reserves the right to make changes to this document and the
information contained herein at any time and without notice. All Achronix trademarks, registered trademarks,
disclaimers and patents are listed at http://www.achronix.com/legal.
Achronix Semiconductor Corporation
2903 Bunker Hill Lane
Santa Clara, CA 95054
USA
Website: www.achronix.com
E-mail : [email protected]

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 3
Table of Contents
Chapter - 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
GDDR6 Subsystem Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Supported Frequency Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter - 2: GDDR6 Controller Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Controller Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Controller Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Modes of Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
By 16 Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
By 8 Clamshell Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Chapter - 3: GDDR6 PHY Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
PHY Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
PHY Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
PHY Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Command/Address Block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
DQ Block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
CA PLL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
DQ PLL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Chapter - 4: GDDR6 Clock and Reset Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Chapter - 5: GDDR6 Interface Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Connectivity to the Peripheral NoC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Connectivity Through the Beachfront . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Chapter - 6: GDDR6 Core and Interface Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Chapter - 7: GDDR6 IP Software Support in ACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 4
Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Step 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 5

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 6
Chapter - 1: Introduction
The Speedster7t FPGA device family provides multiple GDDR6 subsystems that enables the user to fully utilize
the high-bandwidth efficiency of these interfaces for critical applications such as high-performance compute and
machine learning systems.The number of GDDR6 subsystems varies with Speedster7t device. For example, the
Speedster7t1500 device provides eight GDDR6 interfaces (GDDR6 subsystems), four on the east side and four
on the west side of the FPGA. Each subsystem comprises the GDDR6 controller and PHY hard cores and
supports up to 512 Gbps; as a result, the 7t1500 offers up to 4 Tbps of total bandwidth. The GDDR6 controller
and PHY in the subsystem are implemented as hard IP blocks in the I/O ring of a Speedster7t FPGA. For
resource counts for other Speedster7t family members, refer to the (DS015).Speedster7t FPGA Datasheet
Note
The following sub-sections in this user guide pertain to the 7t1500 device with eight GDDR6
subsystems.
Features
Each GDDR6 subsystem supports the following features:
Memory Density – Supports GDDR6 devices from 8 Gb to 16 Gb, compliant with JEDEC GDDR6
SGRAM Standard JESD250.
Data Rate - Supports 12 Gbps, 14 Gbps and 16 Gbps data transfer rate per pin, delivering up to 512
Gbps per subsystem interface. As a result, the Speedster7t with eight GDDR6 subsystems can deliver a
total bandwidth of 4 Tbps for the entire device.
Memory Interface - The GDDR6 subsystem consists of two separate channels, each providing a 16-bit
interface. Hence each subsystem provides a 32-bit interface to the external memory.
Controller Configuration – Supports dual-controller configuration with an independent memory controller
for each memory channel.
System Configurable Modes – The subsystem can be configured as either ×16 mode or ×8 clamshell
mode for increased memory density applications.
Data Mask and Data Bus Inversion – Supports GDDR6 data bus inversion (DBI) and command address
bus inversion (CABI). Also, supports write double-byte mask and write single-byte mask operations.
CA and DQ format – Double data-rate command address and data bus.
ZQ Calibration - Supports multiple master/slave ZQ calibration.
AXI4 Interface – Connects to the other IP interfaces within the Speedster7t device or directly to the FPGA
fabric via an AXI4 interface with support for full or half-rate clocking. The connections utilize either a 256-
bit AXI4 interface to the network on chip (NoC), which can run up to 1 GHz, or a 512-bit AXI4 direct-to-
fabric interface, which can run up to 500 MHz.

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 7
Architecture Overview
The diagram below shows the architecture of Achronix's 7t1500 FPGA. The eight GDDR6 subsystem are
distributed four on the east and west sides each of the fabric. There are PLLs on four corners of the device that
supply the external reference clock to the GDDR6 SDRAM cores and other high-speed interfaces that connect
with the peripheral NoC over the FPGA fabric.
The GDDR6 subsystems can interface with the FPGA core in two ways:
NoC Interface – By using the network hierarchy that allows high-speed data flow between FPGA and
peripheral interfaces
Beachfront (direct-to-fabric) Interface – By using the beachfront interface that connects the memory
controller directly to the core. All the eight GDDR6 subsystems can accessed from the FPGA fabric
through the NoC. However, there are only four subsystems (namely GDDR6 1, 2, 5 and 6 from the
diagram below) that connect to the FPGA fabric directly.

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 8
Figure 1: Speedster7t1500 Architecture Overview Block Diagram
GDDR6 Subsystem Overview
The GDDR6 subsystem provides a simple interface between off-chip GDDR6 memory component and the user
logic mapped to the FPGA core. This memory subsystem comprises the PHY IP, the controller IP, clock and
reset block, APB interfaces and AXI4 interfaces to connect to the NoC and fabric. Below is a block diagram of the
GDDR6 subsystem.

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 9
Figure 2: Speedster7t GDDR6 Subsystem Block Diagram
The GDDR6 subsystem consists of the following functional blocks:
Clock and Reset – The clock and reset block receives its input clocks from the on-chip PLLs and
generates clocks to drive the GDDR6 memory controller and the PHY, with a maximum controller
frequency of 1 GHz and a PHY clock frequency of 500 MHz. The command address clock runs at 2 GHz
and the word clock (WCLK) at 8 GHz; this configuration generates data transactions at the maximum rate
of 16 Gbps. The GDDR6 memory uses a double-data rate (DDR) protocol with separate data being
latched at the rising and the falling edges of the clock. At reset, the controller performs the required
initialization of the external memory, including calibration and programming of the internal mode registers.
Controller IP – The controller IP consists of two channels, Channel0 and Channel1 and two controllers,
one for each 16-bit channel of the GDDR6 memory. This configuration enables the two memory channels
to operate completely independently. The controller IP uses the available AXI interfaces to either talk
directly to the fabric or connect to it through the NoC interface. On the other side, the controller is
connected to the GDDR6 PHY via the DFI4.0 interface.The controller has some sub-modules such as
read-modify-write, reorder and the multi-port front-end cores. The memory controller performs writes and
reads to/from the memory and are as described below:
Memory read – To perform a read, a user design signals a read request together with an address
and burst size. The controller responds with an acknowledgement before the data is available. The
controller translates such a burst of data into multiple consecutive transactions.
Memory write – To perform a write, a user design signals a write request together with an address
and burst size. When the GDDR6 memory is ready to receive the data, the controller generates a
data request sent to the PHY
AXI4 Slave Interface – The AXI4 slave interface is used in the memory subsystem to connect the
controller to the FPGA fabric. This interface has two components: the 256-bit AXI4 interface that
talks a to the Speedster7t NoC interface, and the 512-bit AXI4 interface that connects the signals
from the controller directly to the user logic in the core through the beachfront Interface.

Speedster7t GDDR6 User Guide (UG091)
www.achronix.com 10
PHY IP – The GDDR6 memory PHY enables the communication between the high-speed, high-bandwidth
off-chip GDDR6 memory and the controller. The PHY supports two channels, each with a data width of 16
bits and speeds up to 16 Gbps per pin, delivering a maximum bandwidth of up to 64 GBps.
Memory Interface – The GDDR6 PHY and the controller IP take care of all the details of the
GDDR6 memory interface, such as precharges, activates and refreshes. The controller issues
commands as closely as possible, subject to the timing requirements of the GDDR6 memory to
achieve maximum efficiency
APB Interface – The APB interface operates at 250 MHz and enables the user to configure the GDDR6
subsystem registers. The subsystem registers are configurable through the APB slave interface where the
master can be from the fabric or FPGA configuration unit (FCU) through the NoC. The FCU configures the
subsystem registers during boot-up, and the user can configure the registers from the fabric during user
mode.
Supported Frequency Table
The table below charts out the rates at which each of the interfaces in the GDDR6 subsystem operate:
Table 1: Supported Range of GDDR6 Interface Frequencies
Data Rate AXI-256 AXI-512 Controller Clock PHY Clock Memory CA clock Memory WCK
16 Gbps 1 GHz 500 MHz 1 GHz 500 MHz 2 GHz DDR – 8 GHz
QDR – 4 GHz
14 Gbps 875 MHz 437.5 MHz 875 MHz 437.5 MHz 1.75 GHz DDR – 7 GHz
QDR – 3.5 GHz
12 Gbps 750 Mhz 375 Mhz 750 Mhz 375 Mhz 1.5 GHz DDR – 6 GHz
QDR – 3 GHz