Alpha Data ADM-PCIE-9V3 User manual

ADM-PCIE-9V3
User Manual
Document Revision: 2.7
4th December 20 8

ADM-PCIE-9V3 User Manual
© 20 8 Copyright Alpha Data Parallel Systems Ltd.
All rights reserved.
This publication is protected by Copyright Law, with all rights reserved. No part of this
publication may be reproduced, in any shape or form, without prior written consent from Alpha
Data Parallel Systems Ltd.
Head Office
Address: 4 West Silvermills Lane
Edinburgh, EH3 5B , UK
Telephone: [phone]
Fax: [phone]
email: [email protected]
website: http://www.alpha-data.com
US Office
611 Corporate Circle, Suite H
Golden, CO 80401
[phone]
[phone] - toll free
[email protected]
http://www.alpha-data.com
All trademarks are the property of their respective owners.

ADM-PCIE-9V3 User Manual
Table Of Contents
Introduction ......................................................................................................................................
1.1 Key Features ................................................................................................................................. 1
1.2 Order Code .................................................................................................................................... 1
2 PCB Information .............................................................................................................................. 2
2.1 Physical Specifications .................................................................................................................. 2
2.2 Chassis Requirements ................................................................................................................... 2
2.2.1 PCI E press ............................................................................................................................... 2
2.2.2 Mechanical Requirements ......................................................................................................... 2
2.2.3 Power Requirements ................................................................................................................. 2
2.3 Thermal Performance .................................................................................................................... 3
2.4 Optional Blower ............................................................................................................................. 4
2.5 Full Height Heat Sink ..................................................................................................................... 4
3 Functional Description .................................................................................................................... 5
3.1 Overview ........................................................................................................................................ 5
3.1.1 Switches .................................................................................................................................... 6
3.1.2 LEDs .......................................................................................................................................... 7
3.2 Clocking ......................................................................................................................................... 8
3.2.1 PCIe Reference Clocks ............................................................................................................. 8
3.2.2 Fabric Clock ............................................................................................................................... 8
3.2.3 Programming Clock (EMCCLK) ................................................................................................. 9
3.2.4 QSFP28 ..................................................................................................................................... 9
3.2.5 Ultraport SlimSAS .................................................................................................................... 10
3.2.6 DDR4 SDRAM Reference Clocks ............................................................................................ 10
3.3 PCI E press ................................................................................................................................. 11
3.4 DDR4 SDRAM ............................................................................................................................. 11
3.5 QSFP28 ....................................................................................................................................... 12
3.6 OpenCAPI Ultraport SlimSAS ...................................................................................................... 13
3.7 System Monitor ............................................................................................................................ 14
3.7.1 System Monitor Status LEDs ................................................................................................... 15
3.8 USB Interface .............................................................................................................................. 16
3.9 Configuration ............................................................................................................................... 16
3.9.1 Configuration From Flash Memory .......................................................................................... 16
3.9.1.1 Building and Programming Configuration Images ............................................................... 17
3.9.2 Configuration via JTAG ............................................................................................................ 17
3.10 GPIO Connector .......................................................................................................................... 18
3.10.1 Direct Connect FPGA Signals .................................................................................................. 18
3.10.2 Timing Input ............................................................................................................................. 18
3.11 User EEPROM ............................................................................................................................. 19
Appendix A Complete Pinout Table .................................................................................................................. 2
List of Tables
Table 1 Mechanical imensions ..................................................................................................................... 2
Table 2 Available Power By Rail ..................................................................................................................... 2
Table 3 Switch Functions ................................................................................................................................ 6
Table 4 LE etails ........................................................................................................................................ 7
Table 5 PCIe Reference Clocks ..................................................................................................................... 8
Table 6 Fabric Clock ....................................................................................................................................... 9
Table 7 EMCCLK ............................................................................................................................................ 9

ADM-PCIE-9V3 User Manual
Table 8 QSFP28 Reference Clocks ................................................................................................................ 9
Table 9 QSFP28 Jitter Attenuated Reference Clocks ..................................................................................... 9
Table 10 SlimSAS Reference Clocks (OpenCAPI) ......................................................................................... 10
Table 11 Memory Reference Clocks .............................................................................................................. 10
Table 12 QSFP28 Part Numbers .................................................................................................................... 12
Table 13 Voltage, Current, and Temperature Monitors ................................................................................... 14
Table 14 Status LE efinitions ..................................................................................................................... 15
Table 15 Complete Pinout Table ..................................................................................................................... 21
List of Figures
Figure 1 A M-PCIE-9V3 Product Photo .......................................................................................................... 1
Figure 2 Thermal Performance ........................................................................................................................ 3
Figure 3 Optional Blower ................................................................................................................................. 4
Figure 4 Full Height Heat Sink ......................................................................................................................... 4
Figure 5 A M-PCIE-9V3 Block iagram ......................................................................................................... 5
Figure 6 Switches ............................................................................................................................................ 6
Figure 7 Backside LE s .................................................................................................................................. 7
Figure 8 Front Panel LE s .............................................................................................................................. 7
Figure 9 Clock Topology .................................................................................................................................. 8
Figure 10 Si5328 Block iagram ..................................................................................................................... 10
Figure 11 QSFP Locations ............................................................................................................................... 12
Figure 12 OpenCAPI Location ......................................................................................................................... 13
Figure 13 Flash Address Map .......................................................................................................................... 16
Figure 14 GPIO Connector Schematic ............................................................................................................ 18
Figure 15 GPIO Connector Location ................................................................................................................ 18

ADM-PCIE-9V3 User Manual
Introduction
The A M-PCIE-9V3 is a high-performance reconfigurable computing card intended for ata Center applications,
featuring a Xilinx Virtex UltraScale Plus FPGA.
Figure : ADM-PCIE-9V3 Product Photo
. Key Features
Key Features
• PCIe Gen1/2/3 x1/2/4/8/16 capable
• Half-length, low-profile x16 PCIe form factor
• Two banks of R4 S RAM 72 bit wide memory (ECC), 16GB (8GB per bank) default rated at 2400MT/s,
32GB option rated at 1866MT/s.
• Two QSFP28/zQSFP+ sites capable of data rates up to 28 Gbps per channel (112 Gbps per cage)
• One 8 lane Ultraport SlimSAS connector that is compliant with OpenCAPI
• Optional timing input
• Front panel and rear edge JTAG access via USB port
• FPGA configurable over USB/JTAG and SPI configuration flash
• XCVU3P-2FFVC1517I FPGA
• Voltage, current, and temperature monitoring
.2 Order Code
A M-PCIE-9V3
See http://www.alpha-data.com/pdfs/adm-pcie-9v3.pdf for complete ordering options.
Page Introduction
ad-ug- 322_v2_7.pdf

ADM-PCIE-9V3 User Manual
2 PCB Information
2. Physical Specifications
The A M-PCIE-9V3 complies with PCI Express CEM revision 3.0.
escription Measure
Total y 68.9 mm
Total x (Inc. QSFP Cages) 174 mm
Total z 17.45 mm
Weight 230 grams
Table : Mechanical Dimensions
2.2 Chassis Requirements
2.2. PCI Express
The A M-PCIE-9V3 is capable of PCIe Gen 1/2/3 with 1/2/4/8/16 lanes, using the Xilinx Integrated Block for PCI
Express.
2.2.2 Mechanical Requirements
A 16-lane physical PCIe slot is required for mechanical compatibility.
Each A M-PCIE-9V3 is shipped with a full height PCIe card bracket installed by default. A half-height bracket is
shipped along with the product and can be easily changed out with a Philips screw driver. If the application
requires a low-profile bracket and the order quantity is high, contact [email protected] to get the correct
bracket fitted before shipping.
2.2.3 Power Requirements
The PCIe Specification permits a standard low-profile, half-length PCIe card to dissipate up to 25 W of power,
drawn from the PCIe slot. The A M-PCIE-9V3 may consume more than 25 W of power for larger user FPGA
designs. Power estimation requires the use of the Xilinx XPE spreadsheet and/or a power estimator tool
available from Alpha ata. Please contact [email protected] to obtain this tool.
The power available to the rails calculated using XPE are as follows:
Voltage Source Name Current Capability
0.85 VCC_INT + VCCINT_IO + VCC_BRAM 36A
1.8 VCCAUX + VCCAUX_IO + VCC_BRAM + VCCO_1.8V 6A
3.3 VCCO_3.3V 6A
1.2 VCCO_1.2V 9A
1.8 MGTVCCAUX 1A
0.9 MGTAVCC 9A
1.2 MGTAVTT 15A
Table 2 : Available Power By Rail
Page 2 PCB Information
ad-ug- 322_v2_7.pdf

ADM-PCIE-9V3 User Manual
2.3 Thermal Performance
If the FPGA core temperature exceeds 100 degrees Celsius, the FPGA design will be cleared to prevent that
card from over-heating.
The A M-PCIE-9V3 comes with a heat sink to reduce the heat of the FPGA which is typically the hottest point on
the card. The FPGA die temperature must remain under 100 degrees Celsius. To calculate the FPGA die
temperature, take your application power and multiply by Theta JA from the table below, and add to your system
internal ambient temperature. If you are using the fan provided with the board, you will find theta JA is
approximately 1.4 degC/W for the board in still air.
The power dissipation can be estimated by using the Alpha ata power estimator in conjunction with the Xilinx
Power Estimator (XPE) downloadable at http://www.xilinx.com/products/technology/power/xpe.html. ownload
the UltraScale tool and set the evice to Virtex UltraScale+, VU3P, FFVC1516, -2, Industrial. Set the ambient
temperature to your system ambient and select User Override for the Effective theta JA and enter the figure
associated with your system LFM in the blank field. Proceed to enter all applicable design elements and
utilization in the following spreadsheet tabs. Next aquire the 9V3 power estimator from Alpha ata by contacting
[email protected]. You will then plug in the FPGA power figures along with R4 and QSFP usage to get
an estimate.
Figure 2 : Thermal Performance
Page 3PCB Information
ad-ug- 322_v2_7.pdf

ADM-PCIE-9V3 User Manual
2.4 Optional Blower
Because it is possible for generic PC chassis to not provide sufficient airflow to cool the FPGA, the
A M-PCIE-9V3 is shipped with an uninstalled blower. The blower is optional and can be easily installed with a
Philips screw driver at the discretion of the user. Ensure the opening is facing the heatsink fins. The blower
hangs off the back of the PCB outside of the PCIe card envelope. After screwing the blower into the heatsink,
plug in the small power connector into the connector in the corner of the board. It is possible to alternatively ship
a fan that fits in the adjacent PCIe card slot if required for mechanical fit, contact [email protected] for
details.
Figure 3 : Optional Blower
2.5 Full Height Heat Sink
For customers with full height chassis that would like better thermal performance, and to baffle the empty space
above the card, Alpha ata sells a full height heat sink variant. The thermal performance boost is documented in
the Thermal Performance image above. Contact [email protected] for more details.
Figure 4 : Full Height Heat Sink
Page 4 PCB Information
ad-ug- 322_v2_7.pdf

ADM-PCIE-9V3 User Manual
3 Functional Description
3. Overview
The A M-PCIE-9V3 is a versatile reconfigurable computing platform with a Virtex UltraScale VU3P FPGA, a
Gen3x16 PCIe interface, two banks of R4 both 72 bits wide (for 64 bits with 8 bits ECC), two QSFP28 cages
capable of 8x 28G or 2x 112G Serial IO of any Xilinx supported standard (Ethernet, SRIO, Infiniband, etc.), one
OpenCAPI compatible Ultraport SlimSAS connector also capable of 28G/channel, an input for a timing
synchronization input, a 12 pin header for general purpose use (clocking, control pins, debug, etc.), and a robust
system monitor.
XCVU3P-2
FFVC1517I
(0,4) (5,7) (8,11) (12,15)
x16 PCIe Gen3/4 Edge
QSFP28 C ge
(4x28 Gbps m x)
QSFP28 C ge
(4x28 Gbps m x) DDR4 B nk 1
DDR4-2400/1866, 8/16GB
DDR4 B nk 0
DDR4-2400/1866, 8/16GB
System
Monitor
MGT
MGT HPIO
HPIO
HRIO
x8 SPI
Config
x72
x72
MGTMGT MGT
HRIO
USB JTAG 0
Auxili ry IO
(gpio, timing)
MGTMGT
OpenCAPI SAS
(8x28Gbps m x)
MGT
MAC ID
EEPROM
Figure 5 : ADM-PCIE-9V3 Block Diagram
Page 5Functional Description
ad-ug- 322_v2_7.pdf

ADM-PCIE-9V3 User Manual
3. . Switches
The A M-PCIE-9V3 has a quad IP switch SW1, located on the rear side of the board. The function of each
switch in SW1 is detailed below:
Figure 6 : Switches
Switch Factory
efault Function OFF State ON State
SW1-1 OFF User
Switch 0 Pin AV27 = '1' Pin AV27 = '0'
SW1-2 OFF User
Switch 1 Pin AW27 = '1' Pin AW27 = '0'
SW1-3 OFF Service
Mode Regular Operation Firmware update service mode
SW1-4 OFF JTAG
Source JTAG to FPGA from USB JTAG to FPGA from debug header
SW2-1 ON HOST_I2
C_EN Sysmon over PCIe I2C Sysmon isolated
SW2-2 ON CAPI_VP
_EN OpenCAPI VP available OpenCAPI VP isolated
SW2-3 ON CAPI_VP
_WP CAPI VP is write protected CAPI VP is writable
SW2-4 ON Reserved Reserved Reserved
Table 3 : Switch Functions
Use IO Standard "LVCMOS18" when constraining the user switch pin.
Page 6 Functional Description
ad-ug- 322_v2_7.pdf