Renesas 89HPES32NT24AG2 Guide

1 of 21 May 5, 2011
Notes
®
Introduction
This application note provides system design guidelines for IDT’s PCI Express® 2.0 base specification
compliant System Interconnect switch device family. Information provided in this document is applicable to
the following devices: 89HPES32NT24A[B]G2, 89HPES32NT8A[B]G2, 89HPES24NT24[A]G2,
89HPES24NT6AG2,89HPES22NT16G2,89HPES16NT16G2,89HPES16NT2G2and89HPES12NT12G2.
In this document, the PES32NT24AG2 is used as the primary reference. The letters “G2” within the device
names indicate that these devices are capable of GEN2 (5.0 GT/S) serial data speeds. The
PES32NT24AG2 device offers 32 PCIe lanes divided into 24 ports. The PES24NT24G2 device offers 24
PCIe lanes divided into 24 ports, and so on.
This document also describes the following device interfaces and provides relevant board design
recommendations:
1) PCI Express Interface
2) Reference Clock (REFCLK) Circuitry
3) Reset (Fundamental Reset) Schemes
4) SMBus Interfaces
5) GPIO and JTAG pins
6) Power and Decoupling Scheme
7) Switch Partitioning
PCI Express Interface
Port Configuration
Eight of the twenty four ports of the PES32NT24AG2 are statically allocated 2 lanes with ports labeled
from 0 through 7 and the remaining 16 ports are statically allocated 1 lane with ports labeled from 8 through
23. In a default configuration, SWMODE[3:0] = 0x0, Port 0 is always the upstream port while the remaining
ports are always downstream ports. In a Multi-partition configuration, SWMODE[3:0] = 0xC, or a Multi-parti-
tionwithSerialEEPROMinitializationconfiguration,SWMODE[3:0] = 0xD, all portscomeupas unattached.
Through a Serial EEPROM or Slave SMBus interface, ports can be configured as an upstream port,
upstream port with NT function, upstream port with NT and DMA functions, NT function, NT with DMA func-
tions, or as downstream ports. All ports can operate at a maximum link width of x2 (i.e. 2 lanes) or x1 (i.e. 1
lane) and support both 2.5 GT/S (Gen1) and 5.0 GT/S (Gen2) speeds.
Per the PCIe® specification, each switch port is viewed as a virtual PCI-PCI bridge device. In the
PES32NT24AG2, PCI device numbering follows the port numbering. Port 0 corresponds to Device 0 on the
upstream bus. Port 1 corresponds to Device 1 on the PES32NT24AG2 virtual PCI bus, Port 2 to Device 2,
and so on.
Note: Unused PCIe TX and RX lanes are not required to have a termination and can be left
open.
Application Note
AN-727
By Bryan Le
IDT PCI Express®
24-Port 32-Lane Gen 2 Switch
Hardware Design Guide

2 of 21 May 5, 2011
IDT AN-727
Notes
Figure 1 Port Numbering and Device Numbering
The switch contains four stack blocks labeled Stack 0, Stack 1, Stack 2, and Stack 3. Stacks 0 and 1
have four ports each, and stacks 2 and 3 have eight ports each. This provides a total of 24 ports in the
device, labeled port 0 through port 23. Table 1 lists the ports associated with each stack.
Stacks0 and1may be configuredasfour x2 ports, twox4ports, onex8 port, and combinations inbetween.
Stacks 2 and 3 may be configured as eight x1 ports, four x2 ports, two x4 ports, one x8 port, and
combinations in between. The configuration of each stack is controlled by the Stack
Configuration(STK[3:0]CFG) registers. These registers are located in the Switch Configuration and Status
space (see Device User Manual Chapter 20). Stacks 0 and 1 support five possible configurations each.
Stacks 2 and 3 support 26 possible configurations each. Tables 3.4 through 3.7 below show the possible
configurations for each stack.
◆Each STKxCFG register controls the configuration of the corresponding stack (e.g., STK0CFG
controls the configuration of Stack 0, STK1CFG for Stack 1, etc.)
◆Stack configurations not shown in thetable are not allowed. Programming the STKxCFG register to
values not shown in the table produces undefined results.
Stack Port Associated with the Stack
Stack 0 0, 1, 2, 3
Stack 1 4, 5, 6, 7
Stack 2 8, 9, 10, 11, 12, 13, 14, 15
Stack 3 16, 17, 18, 19, 20, 21, 22, 23
Table 1 Ports in Each Stack
PES32NT24AG2
PE8T[0]
PE8R[0] RX
TX
PE9T[0]
PE9R[0]
PE23T[0]
PE23R[0]
(Port 8, Device8)
(Port 9, Device9)
(Port 23,Device 23)
RX
TX PE0T [1:0] PE0R [1:0]
PE1R [1:0]
PE1T [1:0]
PE7R [1:0]
PE7T [1:0]
(Port 0, Device 0)
(Port 1, Device 1)
(Port 7, Device 7)
RX
TX
RX
TX
RX
TX
RX
TX

3 of 21 May 5, 2011
IDT AN-727
Notes
STKCFG Field in the
STK0CFG Register Stack Configuration
Hex Binary Port 3 Port 2 Port 1 Port 0
0x0 0b00000 x8
0x1 0b00001 x4 x4
0x2 0b00010 x2 x2 x4
0x3 0b00011 x2 x2 x2 x2
0x6 0b00110 x4 x2 x2
Others Reserved
Table 2 Possible Configuration for Stack 0
STKCFG Field in the
STK1CFG Register Stack Configuration
Hex Binary Port 7Port 6 Port 5 Port 4
0x0 0b00000 x8
0x1 0b00001 x4 x4
0x2 0b00010 x2 x2 x4
0x3 0b00011 x2 x2 x2 x2
0x6 0b00110 x4 x2 x2
Others Reserved
Table 3 Possible Configuration for Stack 1
STKCFG Field in the
STK2CFG Register Stack Configuration
Hex Binary P15 P14 P13P12 P11 P10 P9 P8
0x0 0b00000 x8
0x1 0b00001 x4 x4
0x2 0b00010 x2 x2 x4
0x3 0b00011 x2 x2 x2 x2
0x6 0b00110 x4 x2 x2
0x8 0b01000 x4 x1 x1 x2
0x9 0b01001 x4 x1 x1 x1 x1
0xA 0b01010 x2 x2 x2 x1 x1
0xB 0b01011 x2 x2 x1 x1 x2
0xC 0b01100 x1 x2 x2 x4
0xD 0b01101 x1 x1 x1 x1 x4
0xE 0b01110 x2 x1 x1 x2 x2
Table 4 Possible Configuration for Stack 2 (Page 1 of 2)

4 of 21 May 5, 2011
IDT AN-727
Notes
0xF 0b01111 x1 x1 x2 x2 x2
0x10 0b10000 x2 x2 x1 x1 x1x 1
0x11 0b10001 x2 x1 x1 x1 x1 x2
0x12 0b10010 x2 x1 x1 x1 x1 x1 x1
0x13 0b10011 x1 x1 x2 x1 x1 x1 x1
0x14 0b10100 x1 x1 x1 x1 x2 x2
0x15 0b10101 x1 x1 x2 x2 x1x x1
0x16 0b10110 x1 x1 x1 x1 x2 x1 x1
0x17 0b10111 x1 x1 x1 x1 x1 x1 x2
0x18 0b11000 x4 x2 x1 x1
0x19 0b11001 x2 x1 x1 x2 x1 x1
0x1A 0b11010 x1 x1 x2 x1 x1 x2
0x1B 0b11011 x1 x1 x1 x1 x1 x1 x1 x1
0x1C 0b11100 x2 x1 x1 x4
Others Reserved
STKCFG Field in the
STK3CFG Register Stack Configuration
Hex Binary P23 P22 P21 P20 P19 P18 P17 P16
0x0 0b00000 x8
0x1 0b00001 x4 x4
0x2 0b00010 x2 x2 x4
0x3 0b00011 x2 x2 x2 x2
0x6 0b00110 x4 x2 x2
0x8 0b01000 x4 x1 x1 x2
0x9 0b01001 x4 x1 x1 x1 x1
0xA 0b01010 x2 x2 x2 x1 x1
0xB 0b01011 x2 x2 x1 x1 x2
0xC 0b01100 x1 x2 x2 x4
0xD 0b01101 x1 x1 x1 x1 x4
0xE 0b01110 x2 x1 x1 x2 x2
0xF 0b01111 x1 x1 x2 x2 x2
0x10 0b10000 x2 x2 x1 x1 x1x 1
0x11 0b10001 x2 x1 x1 x1 x1 x2
0x12 0b10010 x2 x1 x1 x1 x1 x1 x1
Table 5 Possible Configuration for Stack 3 (Page 1 of 2)
STKCFG Field in the
STK2CFG Register Stack Configuration
Table 4 Possible Configuration for Stack 2 (Page 2 of 2)

5 of 21 May 5, 2011
IDT AN-727
Notes
A stack may be configured statically using the corresponding Stack Configuration (STKxCFG) pins.
These pins are sampled by the switch as part of the boot-configuration vector during switch fundamental
reset. The STKxCFG pins determine the initial value of the STKCFG field in the corresponding STKxCFG
register. The encoding of the STKxCFG pins is identical to that of the STKCFG field shown in Tables 2
through 5.•For Stacks 0 and 1, the STKxCFG pins have 2 bits each (i.e., STK0CFG[1:0] and
STK1CFG[1:0]). These bits correspond to the two least significant bits of the STKCFG field in
the correspondingSTKxCFG register. Therefore, for these stacks, configurations 0x0 through
0x3maybeselectedstatically.OtherconfigurationsmustbeselecteddynamicallyviaEEPROM
or Slave SMBus interface.
•For Stacks 2 and 3, the STKxCFG pins have 5 bits each. Therefore, all 26 possible configura-
tionsmay be selected statically.
•Notethatforall stacksthe STKxCFG[2]pin canbeusedtoselect betweena stackconfiguration
anditsmirrorimage.Forexample,whentheSTKxCFGpinsaresetto0b00010,thestackiscon-
figuredperconfiguration0x2(portsareconfiguredasx4,x2,x2).Thesetting0b00110yieldsthe
mirror image which corresponds to stack configuration 0x6 (ports are configured as x2, x2, x4).
Lane Reversal
The PES32NT24AG2 supports automatic lane reversal outlined in the PCIe specification. This allows
trace routing flexibility to avoid crossovers and potentially reduces the number of trace vias required for
signal routing. Lane reversal must be done for both the transmitter and the receiver of a port.
Lane reversal mappings for the variousnon-trivial x8 maximum link width configurations are illustrated in
Figures 2 through 4. In the figures, PExRP[n] refers to the pin associated with lane 0 of port’x’.
0x13 0b10011 x1 x1 x2 x1 x1 x1 x1
0x14 0b10100 x1 x1 x1 x1 x2 x2
0x15 0b10101 x1 x1 x2 x2 x1x x1
0x16 0b10110 x1 x1 x1 x1 x2 x1 x1
0x17 0b10111 x1 x1 x1 x1 x1 x1 x2
0x18 0b11000 x4 x2 x1 x1
0x19 0b11001 x2 x1 x1 x2 x1 x1
0x1A 0b11010 x1 x1 x2 x1 x1 x2
0x1B 0b11011 x1 x1 x1 x1 x1 x1 x1 x1
0x1C 0b11100 x2 x1 x1 x4
Others Reserved
STKCFG Field in the
STK3CFG Register Stack Configuration
Table 5 Possible Configuration for Stack 3 (Page 2 of 2)

6 of 21 May 5, 2011
IDT AN-727
Notes
Figure 2 Lane Reversal for Highest Achievable Link Width of x2
Figure 3 Lane Reversal for Highest Achievable Link Width of x4

7 of 21 May 5, 2011
IDT AN-727
Notes
Figure 4 Lane Reversal for Highest Achievable Link Width of x8
Polarity Inversion
Each port of the PES32NT24AG2 supports automatic polarity inversion defined by the PCIe specifica-
tion. This allows trace routing flexibility to avoid crossovers and potentiallyreduces the number of trace vias
required for signal routing. Polarity inversion is a function of the receiver and not the transmitter. The trans-
mitter never inverts its data. During link training, the receiver examines symbols 6 through 15 of the TS1
and TS2 ordered sets for inversion of the PExRP[n] and PExRN[n] signals. If an inversion is detected, then

8 of 21 May 5, 2011
IDT AN-727
Notes logic for the receiving lane automatically inverts received data. Polarity inversion is a lane function and not
a link function. Therefore, it is possible for some lanes of link to be inverted and for others not to be
inverted.
Figure 5 Polarity Inversion
Tx Capacitors
The PCIe specification requires that each lane of the link is AC coupled between its corresponding
transmitter and receiver. The AC coupling capacitors for the PES32NT24AG2 are discrete components
located along each transmitter link on the PCB. These AC coupling capacitors allow the transmitter and
receiver on a link to be biased at separate voltages. Table 1 summarizes the guidelines for implementing
the PCIe AC coupling capacitors on a system board.
Parameter Implementation Guideline
AC Coupling AC coupling capacitors are required on the Tx pairs originating
from the PES32NT24AG2
Capacitor Value Between 75nF and 200nF
Capacitor Tolerance Specified minimum/maximum range must be met when capacitor
toleranceisconsideredalongwitheffectsduetotemperatureand
voltage
Capacitor Type Size 603 ceramic capacitors are acceptable, however, size 402
capacitors are strongly encouraged. The smaller the package
size, the less ESL is introduced into the topology. The same
package and capacitor size should be used for each signal in a
differential pair. Do not use capacitor packs for PCIe AC coupling.
Capacitor Pad Size To minimize parasitic impacts, pad sizes for each capacitor
should be the minimum allowed per PCB manufacturer.
Capacitor Placement AC coupling capacitors should be located at the same place
within the differential pair. They should not be staggered in dis-
tance from one trace of the differential pair to the other. Capaci-
tors should be placed as close to each other as possible to avoid
creating large uncoupled sections within the differential pair
traces. Relative location from one differential pair to another is
not important.
Capacitor Location –
Chip-to-Connector Routing Capacitors should be placed such that they are not located in the
center point of a trace route (for example, capacitors should be
placed next to the connector or 1/3 the distance between the con-
nector and the PES32NT24AG2.
Capacitor Location –
Chip-to-Chip Routing Capacitors should be located off-center within the interconnect
(for example, placing the capacitors next to the Rx pins of one
device is generally better than locating the capacitors in the mid-
point of the interconnect).
Table 6 PCIE Interface AC Coupling Capacitor Guidelines
PExT[n]
+
-
+
-
PExR[n]
RX
+
-
+
-
TX

9 of 21 May 5, 2011
IDT AN-727
Notes Routing Differential Pairs
The switch includes 50 Ohm resistor on-die terminations on both the transmit and the receive pins. No
externalterminationisrequired. Individual traces within a given differential pair (positive and negative) must
be matched in length to a tolerance of 5 mils. Length matching within a differential pair should occur on a
segment-by-segment basis, as opposed to length matching across the total distance of the overall route. In
addition, the spacing between traces of adjacent pairs must be at least 20 mils edge-to-edge to reduce
crosstalk effects.
Note that trace length matching between pairs is not required because the PCIe 2.0 specifications allow
up to 8ns of skew between differential pairs. AC coupling capacitors are associated with the Tx differential
pairs and should be located symmetrically on the top or bottom layer between the switch and the PCIe
connectors/devices.
Figure 6 AC Capacitor Placement
Every effort should be made to avoid vias on the PCIe differential pairs since they can result in a signal
loss of up to 0.25 dB. When a via is unavoidable, its pad size should be less than 25 mils, its hole size
should be less than 14 mils, and its anti-pads should be 35 mils or smaller. No extra vias should be added
over and above those needed for IC pads or a connector. Vias in a differential pair should always be at the
same relative location and placed in a symmetrical fashion along the differential pair as shown in Figure 7.
Figure 7 Via Placement
Avoid 90-degree bends or turns on traces. Wherever possible, the number of left and right bends should
be matched as closely as possible. Alternatingleft and right turns helps tominimize length skew differences
between each signal of a differential pair.
Figure 8 Acceptable Bends and Unacceptable Bends
Depending on the system topology and the maximum targeted trace length, regular FR4 material is
appropriatedielectric material. In case of a backplane type of application, higherquality, lower loss material,
such as Nelco 4000-13, may be used.
A HSPICE simulation kit for the switch can be requested by emailing [email protected].
Device Breakout Area
Within the breakout areas of thePES32NT24AG2, the PCIe signal pairs should maximize thedifferential
routing while minimizing any discontinuities or trace length skew. Length matching of the differential traces
should occur as close as possible to the pad or pin while avoiding the addition of tight bends within the
AC Cap Pads
Via Pair
Bottom Layer Top Layer
Preferred Bad

10 of 21 May 5, 2011
IDT AN-727
Notes traces. The breakout area should not exceed 250 mil in length for the PCIe interface. Within the breakout
areas, the trace routing guidelines ofthe differential pairs canbe slightly relaxed (if absolutely necessary) to
facilitate successful breakout of the signals. A width and spacing geometry of 6/4.5/6 (6 mil trace width and
4.5 mil spacing edge to edge) are used on PES32NT24AG2 evaluation board.
Figure 9 PCIe Differential Trace Breakout Bottom Layer
Figure 10 PCI Different Trace Breakout Top Layer
Serdes Reference Resistor Pins
The switch has one Serdes Reference Resistor pin per port. The 3.0K +/- 1% reference bias resistor
should be located as close to these pins as possible and should be tapped immediately to the GROUND
plane. This resistor must be isolated from anysource of noise injection. One way to achieve this is to place
the resistor on the back-side of the board, directly underneath the device. No bypass capacitors must be
placed on these pins.