IDT 89HPES24N3A User manual

®
April 2008
6024 Silver Creek Valley Road, San Jose,California 95138
Telephone: [phone] • [phone] • FAX: [phone]
Printed in U.S.A.
©2008 Integrated Device Technology,Inc.
IDT™89HPES24N3A
PCI Express® Switch
User Manual

GENERAL DISCLAIMER
Integrated Device Technology, Inc. reserves the right to make changes to its products or specifications at any time, without notice, in order to improve design or performance
and to supply the best possible product. IDT does not assume any responsibility for use of any circuitry described other than the circuitry embodied in an IDT product. The
Company makes no representations that circuitry described herein is free from patent infringement or other rights of third parties which may result from its use. No license is
granted by implication or otherwise under any patent, patentrights or other rights, of Integrated Device Technology, Inc.
CODE DISCLAIMER
Code examples provided byIDT are forillustrative purposes only andshould not be relied uponfor developing applications.Any use of the code examples below is completely
at yourown risk. IDT MAKESNO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE NONINFRINGEMENT, QUALITY, SAFETY OR SUITABILITY
OF THE CODE, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICU-
LAR PURPOSE,OR NON-INFRINGEMENT. FURTHER, IDTMAKES NO REPRESENTATIONS OR WARRANTIES ASTOTHE TRUTH, ACCURACY OR COMPLETENESS
OF ANY STATEMENTS, INFORMATION OR MATERIALS CONCERNING CODE EXAMPLES CONTAINED IN ANY IDT PUBLICATION OR PUBLIC DISCLOSURE OR
THAT IS CONTAINED ON ANY IDT INTERNET SITE. IN NO EVENT WILL IDT BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, INDIRECT, PUNITIVE OR
SPECIAL DAMAGES, HOWEVER THEY MAY ARISE, AND EVEN IF IDT HAS BEEN PREVIOUSLY ADVISED ABOUT THE POSSIBILITY OF SUCH DAMAGES. The code
examplesalsomaybe subjecttoUnitedStatesexportcontrollawsandmaybesubjecttotheexportor importlawsofothercountriesanditis yourresponsibilitytocomplywith
any applicable laws or regulations.
LIFE SUPPORT POLICY
Integrated Device Technology's products are not authorized for use as critical components in life support devices or systems unlessa specific written agreementpertaining to
such intended use is executed between the manufacturer and an officer of IDT.
1. Life support devices or systems are devices or systems which (a) are intended for surgical implant into the body or (b) support or sustain life and whose failure to perform,
when properly used in accordance with instructions for use providedin the labeling, can be reasonably expected toresult in a significant injury to the user.
2.A criticalcomponentisanycomponentsofalifesupportdeviceorsystem whosefailuretoperformcanbereasonablyexpected to cause the failureof the life support device
or system, or to affect its safety or effectiveness.
IDT, theIDT logo, andIntegrated Device Technology are trademarks or registered trademarksof IntegratedDevice Technology, Inc.
.

Notes
PES24N3A User Manual 1 April 10, 2008
®
About This Manual
Introduction
This user manual includes hardware and software information on the 89HPES24N3A, a member of
IDT’s PRECISE™ family of PCI Express® switching solutions offering the next-generation I/O interconnect
standard.
Finding Additional Information
Information not included inthis manual such as mechanicals, packagepin-outs, and electrical character-
istics can be found in thedata sheet for this device, which is available from the IDT website (www.idt.com)
as well as through your local IDT sales representative.
Content Summary
Chapter 1, “PES24N3A Device Overview,” provides a complete introduction to the performance capa-
bilities of the 89HPES24N3A. Included in this chapter is a summary of features for the device as well as a
system block diagram and pin description.
Chapter 2, “Clocking, Reset, and Initialization,” provides a description of the two differential refer-
ence clock inputs that are used internally to generate all of the clocks required by the internal switch logic
and the SerDes.
Chapter 3, “Theory of Operation,” provides basic information on the architecture and operationof the
89HPES24N3A chip.
Chapter 4, “Link Operation,” describes the operation of the link feature including polarity inversion,
link width negotiation, and lane reversal.
Chapter 4, “Switch Operation,” discusses the procedure for forwarding PCIe® TLPs between switch
ports.
Chapter 5, “General Purpose I/O,” describes how the 8 General Purpose I/O (GPIO) pins may be indi-
vidually configured as general purpose inputs, general purpose outputs, or alternate functions.
Chapter 6, “SMBus Interfaces,” describes the operation of the 2 SMBus interfaces on the PES24N3A.
Chapter 7, “Power Management,” describes the power management capability structure locatedin the
configuration space of each PCI-PCI bridge in the PES24N3A.
Chapter 8, “Hot-Plug and Hot-Swap,” describes the behavior of the hot-plug andhot-swap features in
the PES24N3A.
Chapter 9, “Configuration Registers,” discusses the base addresses, PCI configuration space, and
registers associated with the PES24N3A.
Chapter 10, “JTAG Boundary Scan,” discusses an enhanced JTAG interface, including a system logic
TAP controller, signal definitions, a test data register, an instruction register, and usage considerations.
Signal Nomenclature
To avoid confusion when dealing with a mixture of “active-low” and “active-high” signals, the terms
assertion and negation are used. The term assert or assertion is used to indicate that a signal is active or
true, independent of whether that level is representedby a high or low voltage. The term negate ornegation
is used to indicate that a signal is inactive or false.

IDT
PES24N3A User Manual 2 April 10, 2008
Notes To define the active polarity of a signal, a suffix will be used. Signals ending with an ‘N’ should be inter-
preted as being active, or asserted, when at a logic zero (low) level. All other signals (including clocks,
buses and select lines) will be interpreted as being active, or asserted when at a logic one (high) level.
To define buses, the most significant bit (MSB) will be on the left and least significant bit (LSB) will be on
the right. No leading zeros will be included.
Throughout this manual, when describing signal transitions, the following terminology is used. Rising
edge indicates a low-to-high (0 to1) transition. Falling edge indicates ahigh-to-low (1 to 0) transition. These
terms are illustrated in Figure 1.
Figure 1 Signal Transitions
Numeric Representations
To represent numerical values, either decimal, binary, or hexadecimal formats will be used. The binary
format is as follows: 0bDDD, where “D” represents either 0 or 1; the hexadecimal format is as follows:
0xDD, where “D” represents the hexadecimal digit(s); otherwise, it is decimal.
The compressed notation ABC[x|y|z]D refers to ABCxD, ABCyD, and ABCzD.
The compressed notation ABC[x:y]D refers to ABCxD, ABC(x+1)D, ABC(x+2)D,... ABCyD if x < y or to
ABCxD, ABC(x-1)D, ABC(x-2)D,... ABCyD if x > y.
Data Units
The following data unit terminology is used in this document.
In quadwords, bit 63 is always the most significant bit and bit 0 is the least significant bit. In double-
words, bit 31 is always the most significant bit and bit 0 is the least significant bit. In words, bit 15 is always
the most significant bit and bit 0 is the least significant bit. In bytes, bit 7 is always the most significant bit
and bit 0 is the least significant bit.
Term Words Bytes Bits
Byte 1/2 1 8
Word 1 2 16
Doubleword (Dword) 2 4 32
Quadword (Qword) 4 8 64
Table 1 Data Unit Terminology
1 2 3 4
high-to-low
transition low-to-high
transition
single clock cycle

IDT
PES24N3A User Manual 3 April 10, 2008
Notes The ordering of bytes within words is referred to as either “big endian” or “little endian.” Big endian
systems label byte zero as the most significant (leftmost) byte of a word. Little endian systems label byte
zero as the least significant (rightmost) byte of a word. See Figure 2.
Figure 2 Example of Byte Ordering for “Big Endian” or “Little Endian” System Definition
Register Terminology
Software in the context of this register terminology refers to modifications madeby PCIe root configura-
tion writes, writes to registers made through the slave SMBus interface, or serial EEPROM register initial-
ization. See Table 2.
Type Abbreviation Description
Hardware Initialized HWINIT Register bits are initialized by firmware or hardware mechanisms
such as pin strapping or serial EEPROM. (System firmware hard-
ware initialization is only allowed for system integrated devices.)
Bits are read-only after initialization and can only be reset (for
write-once by firmware) with reset.
Read Only and Clear RC Software can read the register/bits with this attribute. Reading the
value will automatically cause the register/bit to be reset to zero.
Writing to a RC location has no effect.
Read Clear and Write RCW Software can read the register/bits with this attribute. Reading the
value will automatically cause the register/bits to be reset to zero.
Writes cause the register/bits to be modified.
Reserved Reserved The value read from a reserved register/bit is undefined. Thus,
software must deal correctly with fields that are reserved. On
reads, softwaremust useappropriatemaskstoextractthedefined
bits and not rely on reserved bits being any particular value. On
writes, software must ensure that the values of reserved bit posi-
tions are preserved. That is, the values of reserved bit positions
must first be read, merged with the new values for other bit posi-
tions and then written back.
Read Only RO Software can only read registers/bits with this attribute. Contents
are hardwired to a constant value or are status bits that may be
set and cleared by hardware. Writing to a RO location has no
effect.
Read and Write RW Software can both read and write bitswith this attribute.
Table 2 Register Terminology (Sheet 1 of 2)
0123
bit 0bit 31
Address of Bytes within Words: Big Endian
3210
bit 0bit 31
Address of Bytes within Words: Little Endian

IDT
PES24N3A User Manual 4 April 10, 2008
Notes
Use of Hypertext
In Chapter 9, Tables 9.2 and 9.3 contain register names and page numbershighlighted in blue under the
Register Definition column. In pdf files, users can jump from this source table directly to the registers by
clicking on the register name in the source table. Each register name in the table is linked directly to the
appropriate register in the register section of the chapter. To return to the source table after having jumped
to the register section, click on the same register name (in blue) in the register section.
Reference Documents
PCI Express Base Specification, Revision 2.0, PCI Special Interest Group.
PCI Power Management Interface Specification, Revision 1.1, PCI Special Interest Group.
PCI to PCI Bridge Architecture Specification, Revision 1.2, PCI Special Interest Group.
SMBus Specification, Revision 2.0.
Revision History
February 8, 2007: Initial Publication.
May 30, 2007: In Table 1.2, added revision information for ZG silicon. Added Notes to Figure 2.5.
July 18, 2007: In Chapter 9, changed bits [10:9] in HPCFGCTL from RO to RW. In Chapter 2, changed
references to correctly state SRESET field is in BCTL register, not the SWCTL register.
April 10, 2008: In the About section, Table 2, changed SYSCNTL to SWCTL. In Chapter 9, changed
default value for VER field in PCIECAP register from 0x2 to 0x1 and changed 0x0 definition for bit EEPE in
SWPERCTL register from “time-out” to “end-to-end parity error”.
Read and Write Clear RW1C Software can read and write to registers/bits with this attribute.
However, writing a value of zero to a bit with this attribute has no
effect. A RW1C bit can onlybe set to a value of 1 by a hardware
event. To clear a RW1C bit (i.e., change its value to zero) a value
of one must be written to the location. An RW1C bit is never
cleared by hardware.
Read and Write when
Unlocked RWL Software can read the register/bits with this attribute. Writing to register/bits with this attribute will only cause the value to be modi-
fied if the REGUNLOCK bit in the SWCTL register isset. When
the REGUNLOCK bit is cleared, writes are ignored and the regis-
ter/bits are effectively read-only
Write Transient WT The zero is always read from a bit/field of this type. Writing of a
one is used to qualitythe writing of other bits/fields in the same
register.
Zero Zero A zero register or bit must be written with a value of zero and
returns a value of zero when read.
Type Abbreviation Description
Table 2 Register Terminology (Sheet 2 of 2)

Notes
PES24N3A User Manual i April 10, 2008
Table of Contents
®
About This Manual
Introduction ....................................................................................................................................1
Content Summary ..........................................................................................................................1
Signal Nomenclature .....................................................................................................................1
Numeric Representations ..............................................................................................................2
Data Units ......................................................................................................................................2
Register Terminology .....................................................................................................................3
Use of Hypertext ............................................................................................................................4
Reference Documents ...................................................................................................................4
Revision History .............................................................................................................................4
PES24N3A Device Overview
Introduction.....................................................................................................................................1-1
List of Features...............................................................................................................................1-1
System Diagrams............................................................................................................................1-3
Logic Diagram.................................................................................................................................1-4
System Identification.......................................................................................................................1-5
Vendor ID................................................................................................................................1-5
Device ID................................................................................................................................1-5
Revision ID.............................................................................................................................1-5
JTAG ID..................................................................................................................................1-5
SSID/SSVID............................................................................................................................1-5
Device Serial Number Enhanced Capability...........................................................................1-5
Pin Description................................................................................................................................1-6
Pin Characteristics..........................................................................................................................1-9
Clocking, Reset, and Initialization
Introduction.....................................................................................................................................2-1
Initialization.....................................................................................................................................2-3
Reset...............................................................................................................................................2-4
Fundamental Reset................................................................................................................2-4
Hot Reset................................................................................................................................2-6
Upstream Secondary Bus Reset............................................................................................2-7
Downstream Secondary Bus Reset........................................................................................2-7
Downstream Port Reset Outputs....................................................................................................2-8
Power Enable Controlled Reset Output..................................................................................2-8
Power Good Controlled Reset Output....................................................................................2-9
Hot Reset Controlled Reset Output........................................................................................2-9
Theory of Operation
Introduction.....................................................................................................................................3-1
Data Paths......................................................................................................................................3-2
Store-and-Forward vs. Cut-Through Switching and Latency..........................................................3-2
Switch Core.....................................................................................................................................3-3
Transaction Routing................................................................................................................3-4
Transaction Reordering..........................................................................................................3-4
Scheduling and Port Arbitration..............................................................................................3-5

IDT Table of Contents
PES24N3A User Manual ii April 10, 2008
Notes Peer-to-Peer Transactions..............................................................................................................3-8
Bus Locking....................................................................................................................................3-8
Port Interrupts...............................................................................................................................3-10
Legacy Interrupt Emulation...........................................................................................................3-10
Standard PCIe Error Detection and Handling...............................................................................3-11
Physical Layer Errors ...........................................................................................................3-11
Data Link Layer Errors..........................................................................................................3-11
Transaction Layer Errors......................................................................................................3-12
Routing Errors ......................................................................................................................3-14
Switch Specific Error Detection and Handling..............................................................................3-15
Switch Time-Outs.................................................................................................................3-16
End-to-End Parity Checking.................................................................................................3-16
TLP Processing ............................................................................................................................3-17
Link Operation
Introduction.....................................................................................................................................4-1
Polarity Inversion............................................................................................................................4-1
Link Width Negotiation....................................................................................................................4-1
Lane Reversal.................................................................................................................................4-1
Link Retraining................................................................................................................................4-4
Link Down.......................................................................................................................................4-5
Slot Power Limit Support................................................................................................................4-5
Upstream Port ........................................................................................................................4-5
Downstream Port....................................................................................................................4-5
Link States......................................................................................................................................4-5
Active State Power Management ...................................................................................................4-6
Link Status......................................................................................................................................4-7
General Purpose I/O
Introduction.....................................................................................................................................5-1
GPIO Configuration ........................................................................................................................5-1
GPIO Pin Configured as an Input...........................................................................................5-2
GPIO Pin Configured as an Output........................................................................................5-2
GPIO Pin Configured as an Alternate Function......................................................................5-2
SMBus Interfaces
Introduction.....................................................................................................................................6-1
Master SMBus Interface.................................................................................................................6-2
Initialization.............................................................................................................................6-2
Serial EEPROM......................................................................................................................6-2
I/O Expanders.........................................................................................................................6-6
Slave SMBus Interface.................................................................................................................6-11
Initialization...........................................................................................................................6-11
SMBus Transactions ............................................................................................................6-11
Power Management
Introduction.....................................................................................................................................7-1
PME Messages...............................................................................................................................7-2
Power Express Power Management Fence Protocol.....................................................................7-3
Power Budgeting Capability............................................................................................................7-3

IDT Table of Contents
PES24N3A User Manual iii April 10, 2008
Notes Hot-Plug and Hot-Swap
Introduction.....................................................................................................................................8-1
Hot-Plug I/O Expander ...........................................................................................................8-4
Hot-Plug Interrupts and Wake-up...........................................................................................8-4
Legacy System Hot-Plug Support ..........................................................................................8-4
Hot-Swap........................................................................................................................................8-6
Configuration Registers
Introduction.....................................................................................................................................9-1
Upstream Port (Port 0) ...........................................................................................................9-3
Downstream Ports (Ports 2 and 4).........................................................................................9-8
Register Definitions.......................................................................................................................9-11
Type 1 Configuration Header Registers...............................................................................9-11
PCI Express Capability Structure.........................................................................................9-21
Power Management Capability Structure.............................................................................9-33
Message Signaled Interrupt Capability Structure.................................................................9-34
Subsystem ID and Subsystem Vendor ID............................................................................9-36
Extended Configuration Space Access Registers................................................................9-36
Advanced Error Reporting (AER) Enhanced Capability.......................................................9-37
Device Serial Number Enhanced Capability.........................................................................9-43
PCI Express Virtual Channel Capability...............................................................................9-43
Power Budgeting Enhanced Capability................................................................................9-49
Switch Control and Status Registers....................................................................................9-50
Internal Switch Error Control and Status Registers..............................................................9-61
JTAG Boundary Scan
Introduction...................................................................................................................................10-1
Test Access Point.........................................................................................................................10-1
Signal Definitions..........................................................................................................................10-1
Boundary Scan Chain...................................................................................................................10-3
Test Data Register (DR)...............................................................................................................10-4
Boundary Scan Registers.....................................................................................................10-4
Instruction Register (IR)................................................................................................................10-6
EXTEST................................................................................................................................10-6
SAMPLE/PRELOAD.............................................................................................................10-7
BYPASS...............................................................................................................................10-7
CLAMP.................................................................................................................................10-7
IDCODE................................................................................................................................10-7
VALIDATE............................................................................................................................10-8
RESERVED..........................................................................................................................10-8
Usage Considerations..........................................................................................................10-8

IDT Table of Contents
PES24N3A User Manual iv April 10, 2008
Notes

Notes
PES24N3A User Manual v April 10, 2008
List of Tables
®
Table 1.1 PES24N3A Device ID..........................................................................................................1-5
Table 1.2 PES24N3A Revision ID.......................................................................................................1-5
Table 1.3 PCI Express Interface Pins..................................................................................................1-6
Table 1.4 SMBus Interface Pins..........................................................................................................1-6
Table 1.5 General Purpose I/O Pins....................................................................................................1-7
Table 1.6 System Pins.........................................................................................................................1-7
Table 1.7 Test Pins..............................................................................................................................1-8
Table 1.8 Power and Ground Pins.......................................................................................................1-8
Table 1.9 Pin Characteristics...............................................................................................................1-9
Table 2.1 Reference Clock Mode Encoding........................................................................................2-1
Table 2.2 Boot Configuration Vector Signals.......................................................................................2-3
Table 3.1 IFB Buffer Sizes...................................................................................................................3-1
Table 3.2 PES24N3A Buffer Sizes......................................................................................................3-2
Table 3.3 Bus Decoupler Queue and Insertion Buffer Size.................................................................3-2
Table 3.4 Latency................................................................................................................................3-3
Table 3.5 Switch Routing Methods......................................................................................................3-4
Table 3.6 IFB Transaction Ordering....................................................................................................3-5
Table 3.7 Downstream Port Interrupts...............................................................................................3-10
Table 3.8 PES24N3A Downstream to Upstream Port Interrupt Routing...........................................3-11
Table 3.9 Physical Layer Errors.........................................................................................................3-11
Table 3.10 Data Link Layer Errors.......................................................................................................3-12
Table 3.11 Transaction Layer Errors...................................................................................................3-12
Table 3.12 Ingress Malformed TLP Error Checks ...............................................................................3-13
Table 3.13 Egress Malformed TLP Error Checks................................................................................3-14
Table 5.1 General Purpose I/O Pin Alternate Function.......................................................................5-1
Table 5.2 GPIO Pin Configuration.......................................................................................................5-1
Table 6.1 Serial EEPROM SMBus Address........................................................................................6-2
Table 6.2 PES24N3A Compatible Serial EEPROMs...........................................................................6-3
Table 6.3 Serial EEPROM Initialization Errors ....................................................................................6-5
Table 6.4 I/O Expander Function Allocation........................................................................................6-6
Table 6.5 I/O Expander 0 Signals........................................................................................................6-9
Table 6.6 I/O Expander 2 Signals......................................................................................................6-10
Table 6.7 I/O Expander 4 Signals......................................................................................................6-10
Table 6.8 Slave SMBus Address When a Static Address is Selected...............................................6-11
Table 6.9 Slave SMBus Command Code Fields...............................................................................6-12
Table 6.10 CSR Register Read or Write Operation Byte Sequence ...................................................6-13
Table 6.11 CSR Register Read or Write CMD Field Description.........................................................6-13
Table 6.12 Serial EEPROM Read or Write Operation Byte Sequence................................................6-14
Table 6.13 Serial EEPROM Read or Write CMD Field Description.....................................................6-15
Table 7.1 PES24N3A Power Management State Transition Diagram.................................................7-2
Table 8.1 Downstream Port Hot-Plug Signals.....................................................................................8-3
Table 9.1 Base Addresses for Port Configuration Space Registers....................................................9-1
Table 9.2 Upstream Port 0 Configuration Space Registers.................................................................9-3
Table 9.3 Downstream Ports 2 and 4 Configuration Space Registers................................................9-8
Table 10.1 JTAG Pin Descriptions.......................................................................................................10-2
Table 10.2 Boundary Scan Chain........................................................................................................10-3
Table 10.3 Instructions Supported by PES24N3A’s JTAG Boundary Scan ........................................10-6
Table 10.4 System Controller Device Identification Register...............................................................10-7

IDT List of Tables
PES24N3A User Manual vi April 10, 2008
Notes

Notes
PES24N3A User Manual vii April 10, 2008
List of Figures
®
Figure 1.1 PES24N3A Architectural Block Diagram ............................................................................1-3
Figure 1.2 I/O Expansion Application ..................................................................................................1-3
Figure 1.3 PES24N3A Logic Diagram .................................................................................................1-4
Figure 2.1 Common Clock on Upstream and Downstream (option to enable or disable Spread
Spectrum Clock) ................................................................................................................2-1
Figure 2.2 Non-Common Clock on Upstream; Common Clock on Downstream (mustdisable
Spread Spectrum Clock) ....................................................................................................2-2
Figure 2.3 Common Clock on Upstream; Non-Common Clock on Downstream (must disable
Spread Spectrum Clock) ....................................................................................................2-2
Figure 2.4 Non-Common Clock on Upstream and Downstream (must disable Spread Spectrum
Clock) .................................................................................................................................2-3
Figure 2.5 Fundamental Reset in Transparent Mode with Serial EEPROM initialization ....................2-6
Figure 2.6 Power Enable Controlled Reset Output Mode Operation ..................................................2-8
Figure 2.7 Power Good Controlled Reset Output Mode Operation .....................................................2-9
Figure 3.1 Simplified Switch Core U-Bus and D-Bus Datapath ...........................................................3-3
Figure 3.2 U-Bus Arbitration ................................................................................................................3-7
Figure 4.1 Port Lane Reversal for Maximum Link Width of x2 (MAXLNKWDTH[5:0]=0x2) ................4-2
Figure 4.2 Port Lane Reversal for Maximum Link Width of x4 (MAXLNKWDTH[5:0]=0x4) ................4-3
Figure 4.3 Port Lane Reversal for Maximum Link Width of x8 (MAXLNKWDTH[5:0]=0x8) ................4-4
Figure 4.4 PES24N3A ASPM Link Sate Transitions ...........................................................................4-6
Figure 6.1 SMBus Interface Configuration Examples .........................................................................6-1
Figure 6.2 Single Double Word Initialization Sequence Format ..........................................................6-3
Figure 6.3 Sequential Double Word Initialization Sequence Format ...................................................6-4
Figure 6.4 Configuration Done Sequence Format ..............................................................................6-4
Figure 6.5 Slave SMBus Command Code Format ............................................................................6-12
Figure 6.6 CSR Register Read or Write CMD Field Format ..............................................................6-13
Figure 6.7 Serial EEPROM Read or Write CMD Field Format ..........................................................6-15
Figure 6.8 CSR Register Read Using SMBus Block Write/Read Transactions with PEC
Disabled ...........................................................................................................................6-16
Figure 6.9 Serial EEPROM Read Using SMBus Block Write/Read Transactions with PEC
Disabled ...........................................................................................................................6-16
Figure 6.10 CSR Register Write Using SMBus Block Write Transactions with PEC Disabled ...........6-16
Figure 6.11 Serial EEPROM Write Using SMBus Block Write Transactions with PEC Disabled ........6-17
Figure 6.12 Serial EEPROM Write Using SMBus Block Write Transactions with PEC Enabled ........6-17
Figure 6.13 CSR Register Read Using SMBus Read and Write Transactions with PEC Disabled ....6-18
Figure 7.1 PES24N3A Power Management State Transition Diagram ...............................................7-1
Figure 8.1 Hot-Plug on Switch Downstream Slots Application ............................................................8-1
Figure 8.2 Hot-Plug with Switch on Add-In Card Application ..............................................................8-2
Figure 8.3 Hot-Plug with Carrier Card Application ..............................................................................8-2
Figure 8.4 PES24N3A Hot-Plug Event Signalling ...............................................................................8-5
Figure 9.1 Port Configuration Space Organization .............................................................................9-2
Figure 10.1 Diagram of the JTAG Logic ..............................................................................................10-1
Figure 10.2 State Diagram of PES24N3A’s TAP Controller ................................................................10-2
Figure 10.3 Diagram of Observe-only Input Cell .................................................................................10-4
Figure 10.4 Diagram of Output Cell ....................................................................................................10-5
Figure 10.5 Diagram of Bidirectional Cell ............................................................................................10-5
Figure 10.6 Device ID Register Format ...............................................................................................10-7

IDT List of Figures
PES24N3A User Manual viii April 10, 2008
Notes

Notes
PES24N3A User Manual ix April 10, 2008
Register List
®
AERCAP - AER Capabilities (0x100)..................................................................................................... 9-37
AERCEM - AER Correctable Error Mask (0x114).................................................................................. 9-41
AERCES - AER Correctable Error Status (0x110)................................................................................. 9-41
AERCTL - AER Control (0x118)............................................................................................................. 9-42
AERHL1DW - AER Header Log 1st Doubleword (0x11C) ..................................................................... 9-42
AERHL2DW - AER Header Log 2nd Doubleword (0x120)..................................................................... 9-42
AERHL3DW - AER Header Log 3rd Doubleword (0x124)...................................................................... 9-42
AERHL4DW - AER Header Log 4th Doubleword (0x128)...................................................................... 9-43
AERUEM - AER Uncorrectable Error Mask (0x108).............................................................................. 9-38
AERUES - AER Uncorrectable Error Status (0x104)............................................................................. 9-37
AERUESV - AER Uncorrectable Error Severity (0x10C)........................................................................ 9-39
BAR0 - Base Address Register 0 (0x010).............................................................................................. 9-15
BAR1 - Base Address Register 1 (0x014).............................................................................................. 9-15
BCTRL - Bridge Control Register (0x03E).............................................................................................. 9-20
BIST - Built-in Self Test Register (0x00F).............................................................................................. 9-14
CAPPTR - Capabilities Pointer Register (0x034)................................................................................... 9-19
CCODE - Class Code Register(0x009)................................................................................................. 9-14
CLS - Cache Line Size Register (0x00C)............................................................................................... 9-14
DARBCTC - D-Bus Arbiter Current Transfer Count (0x464).................................................................. 9-60
DARBTC - D-Bus Arbiter Transfer Count (0x460).................................................................................. 9-60
DID - Device Identification Register (0x002).......................................................................................... 9-12
ECFGADDR - Extended Configuration Space Access Address (0x0F8)............................................... 9-36
ECFGDATA - Extended Configuration Space Access Data (0x0FC)..................................................... 9-37
EEPROMINTF - Serial EEPROM Interface (0x42C).............................................................................. 9-56
EROMBASE - Expansion ROM Base Address Register (0x038)........................................................... 9-19
GPECTL - General Purpose Event Control (0x450)............................................................................... 9-58
GPESTS - General Purpose Event Status (0x454)................................................................................ 9-59
GPIOCFG - General Purpose I/O Configuration (0x41C)....................................................................... 9-54
GPIOD - General Purpose I/O Data (0x420).......................................................................................... 9-54
GPIOFUNC - General Purpose I/O Control Function (0x418)................................................................ 9-54
GPR - General Purpose Register (0x40C)............................................................................................. 9-53
HDR - Header Type Register (0x00E).................................................................................................... 9-14
HPCFGCTL - Hot-Plug Configuration Control (0x408)........................................................................... 9-52
INTRLINE - Interrupt Line Register (0x03C)........................................................................................... 9-19
INTRPIN - Interrupt PIN Register (0x03D)............................................................................................. 9-19
IOBASE - I/O Base Register (0x01C)..................................................................................................... 9-16
IOBASEU - I/O Base Upper Register (0x030)........................................................................................ 9-18
IOEXPADDR0 - SMBus I/O Expander Address 0 (0x434)..................................................................... 9-58
IOEXPADDR1 - SMBus I/O Expander Address 1 (0x438)..................................................................... 9-58
IOEXPINTF - I/O Expander Interface (0x430)........................................................................................ 9-57
IOLIMIT - I/O Limit Register (0x01D)...................................................................................................... 9-16
IOLIMITU - I/O Limit Upper Register (0x032)......................................................................................... 9-19
MBASE - Memory Base Register (0x020).............................................................................................. 9-17
MLIMIT - Memory Limit Register (0x022)............................................................................................... 9-17
MSIADDR - Message Signaled Interrupt Address (0x0D4).................................................................... 9-35
MSICAP - Message Signaled Interrupt Capability and Control (0x0D0) ................................................ 9-34
MSIMDATA - Message Signaled Interrupt Message Data (0x0DC)....................................................... 9-36
MSIUADDR - Message Signaled Interrupt Upper Address (0x0D8)...................................................... 9-35
PBUSN - Primary Bus Number Register (0x018)................................................................................... 9-15

IDT Register List
PES24N3A User Manual x April 10, 2008
Notes PCICMD - PCI Command Register (0x004)............................................................................................9-12
PCIECAP - PCI Express Capability (0x040) ...........................................................................................9-21
PCIEDCAP - PCI Express Device Capabilities (0x044)..........................................................................9-21
PCIEDCAP2 - PCI Express Device Capabilities 2 (0x064).....................................................................9-31
PCIEDCTL - PCI Express Device Control (0x048)..................................................................................9-22
PCIEDCTL2 - PCI Express Device Control 2 (0x068).............................................................................9-31
PCIEDSTS - PCI Express Device Status (0x04A) ..................................................................................9-23
PCIEDSTS2 - PCI Express Device Status 2 (0x06A) .............................................................................9-31
PCIELCAP - PCI Express Link Capabilities (0x04C) ..............................................................................9-24
PCIELCAP2 - PCI Express Link Capabilities 2 (0x06C) .........................................................................9-31
PCIELCTL - PCI Express Link Control (0x050).......................................................................................9-25
PCIELCTL2 - PCI Express Link Control 2 (0x070)..................................................................................9-32
PCIELSTS - PCI Express Link Status (0x052)........................................................................................9-26
PCIELSTS2 - PCI Express Link Status 2 (0x072)...................................................................................9-32
PCIESCAP - PCI Express Slot Capabilities (0x054)...............................................................................9-27
PCIESCAP2 - PCI Express SlotCapabilities 2 (0x074)..........................................................................9-32
PCIESCTL - PCI Express Slot Control (0x058).......................................................................................9-29
PCIESCTL2 - PCI Express Slot Control 2 (0x078)..................................................................................9-32
PCIESSTS - PCI Express Slot Status (0x05A) .......................................................................................9-30
PCIESSTS2 - PCI Express Slot Status 2 (0x07A) ..................................................................................9-33
PCIEVCECAP - PCI Express VC Enhanced Capability Header (0x200)................................................9-43
PCISTS - PCI Status Register (0x006) ...................................................................................................9-13
PLTIMER - Primary Latency Timer (0x00D)............................................................................................9-14
PMBASE - Prefetchable Memory Base Register (0x024).......................................................................9-17
PMBASEU - Prefetchable Memory Base Upper Register (0x028)..........................................................9-18
PMCAP - PCI Power Management Capabilities (0x0C0)........................................................................9-33
PMCSR - PCI Power Management Control and Status (0x0C4) ............................................................9-34
PMLIMIT - Prefetchable Memory Limit Register (0x026)........................................................................9-18
PMLIMITU - Prefetchable Memory Limit Upper Register (0x02C)..........................................................9-18
PVCCAP1- Port VC Capability 1 (0x204)................................................................................................9-44
PVCCAP2 - Port VC Capability 2 (0x208)...............................................................................................9-44
PVCCTL - Port VC Control (0x20C)........................................................................................................9-45
PVCSTS - Port VC Status (0x20E) .........................................................................................................9-45
PWRBCAP - Power Budgeting Capabilities (0x280)...............................................................................9-49
PWRBD - Power Budgeting Data (0x288)...............................................................................................9-49
PWRBDSEL - Power Budgeting Data Select (0x284).............................................................................9-49
PWRBDV[0..7] - Power Budgeting Data Value [0..7] (0x300).................................................................9-50
PWRBPBC - Power Budgeting Power Budget Capability (0x28C) .........................................................9-50
RID - Revision Identification Register (0x008) ........................................................................................9-14
SBUSN - Secondary Bus Number Register (0x019)...............................................................................9-15
SECSTS - Secondary Status Register (0x01E) ......................................................................................9-16
SLTIMER - Secondary Latency Timer Register (0x01B).........................................................................9-15
SMBUSCTL - SMBus Control (0x428)....................................................................................................9-55
SMBUSSTS - SMBus Status (0x424) .....................................................................................................9-54
SNUMCAP - Serial Number Capabilities (0x180) ...................................................................................9-43
SNUMLDW - Serial Number Lower Doubleword (0x184) .......................................................................9-43
SNUMUDW - Serial Number Upper Doubleword (0x188).......................................................................9-43
SSIDSSVID - Subsystem ID and Subsystem Vendor ID (0x0F4)...........................................................9-36
SSIDSSVIDCAP - Subsystem ID and Subsystem Vendor ID Capability (0x0F0)...................................9-36
SUBUSN - Subordinate Bus Number Register (0x01A)..........................................................................9-15
SWCTL - Switch Control (0x404)............................................................................................................9-51
SWPECNT - Switch Parity Error Count (0x74C).....................................................................................9-62
SWPECTL - Switch Parity Error Control (0x740)....................................................................................9-61
SWPERCTL - Switch Parity Error Reporting Control (0x748).................................................................9-62
SWPESTS - Switch Parity Error Status (0x744) .....................................................................................9-61

IDT Register List
PES24N3A User Manual xi April 10, 2008
Notes SWSTS - Switch Status (0x400) .............................................................................................................9-50
SWTOCNT - Switch Time-Out Count (0x75C)........................................................................................9-64
SWTOCTL - Switch Time-Out Control (0x750).......................................................................................9-62
SWTORCTL - Switch Time-Out Reporting Control (0x758)....................................................................9-63
SWTOSTS - Switch Time-Out Status (0x754) ........................................................................................9-62
SWTOTSCTL - Switch Time-Out Time-Stamp Control (0x760)..............................................................9-64
SWTSCNTCTL - Switch Time-Stamp Counter Control (0x4A8) .............................................................9-61
UARBCTC - U-Bus Arbiter Current Transfer Count (0x45C) ..................................................................9-60
UARBTC - U-Bus Arbiter Transfer Count (0x458)...................................................................................9-59
VCR0CAP- VC Resource 0 Capability (0x210).......................................................................................9-45
VCR0CTL- VC Resource 0 Control (0x214)............................................................................................9-46
VCR0STS - VC Resource 0 Status (0x218)............................................................................................9-46
VCR0TBL0 - VC Resource 0 Arbitration Table Entry 0 (0x220)..............................................................9-47
VCR0TBL1 - VC Resource 0 Arbitration Table Entry 1 (0x224)..............................................................9-47
VCR0TBL2 - VC Resource 0 Arbitration Table Entry 2 (0x228)..............................................................9-48
VCR0TBL3 - VC Resource 0 Arbitration Table Entry 3 (0x22C).............................................................9-48
VID - Vendor Identification Register (0x000)...........................................................................................9-11

IDT Register List
PES24N3A User Manual xii April 10, 2008
Notes

Notes
PES24N3A User Manual 1 - 1 April 10, 2008
®
Chapter 1
PES24N3A Device Overview
Introduction
The 89HPES24N3A is a member of the IDT PRECISE™ family of PCI Express® switching solutions.
The PES24N3A is a 24-lane, 3-port peripheral chip that performs PCI Express packet switching with a
feature set optimized for high performance applications such as servers, storage, and communications/
networking. It provides connectivity and switching functions between a PCI Express upstream port and two
downstream ports and supports switching between downstream ports
Utilizing standard PCI Express interconnect, the PES24N3A provides the most efficient I/O connectivity
solution for applications requiring high throughput, low latency, and simple board layout with a minimum
number of board layers. It provides connectivity for up to 3 ports across 24 integrated serial lanes. Each
lane provides 2.5 Gbps of bandwidth in both directions and is fully compliant with PCI Express Base speci-
fication revision 1.1.
The PES24N3A is based on aflexibleand efficient layered architecture. The PCI Express layers consist
of SerDes, Physical, Data Link and Transaction layers. The PES24N3A can operate either as a store and
forward switch or a cut-through switch and is designed to switch memory and I/O transactions. It supports
eight Traffic Classes (TCs) and one Virtual Channel (VC) with sophisticated resource management to
enable efficient switching and I/O connectivity.
List of Features
High Performance PCI Express Switch
– Twenty-four 2.5 Gbps PCI Express lanes
– Three switch ports
– Upstream port configurable up to x8
– Downstream ports configurable up to x8
– Low-latency cut-through switch architecture
– Support for Max Payload Size up to 2048 bytes
– One virtual channel
– Eight traffic classes
– PCI Express Base Specification Revision 1.1 compliant
Flexible Architecture with Numerous Configuration Options
– Automatic per port link width negotiation to x8, x4, x2 or x1
– Automatic lane reversal on all ports
– Automatic polarity inversion on all lanes
– Ability to load device configuration from serial EEPROM
Legacy Support
– PCI compatible INTx emulation
– Bus locking
Highly Integrated Solution
– Requires no external components
– Incorporates on-chip internal memory for packet buffering and queueing
– Integrates twenty-four 2.5 Gbps embedded SerDes with 8B/10B encoder/decoder (no separate
transceivers needed)

IDT PES24N3A Device Overview
PES24N3A User Manual 1 - 2 April 10, 2008
Notes Reliability, Availability, and Serviceability (RAS) Features
– Supports ECRC and Advanced Error Reporting
– Internal end-to-end parity protection on all TLPs ensures data integrity even in systems that do
not implement end-to-end CRC (ECRC)
– Supports PCI Express Native Hot-Plug, Hot-Swap capable I/O
– Compatible with Hot-Plug I/O expanders used on PC and server motherboards
Power Management
– Utilizes advanced low-power design techniques to achieve low typical power consumption
– Supports PCI Power Management Interface specification (PCI-PM 1.1)
• Supports device power management states: D0, D3hot and D3cold
– Unused SerDes are disabled
Testability and Debug Features
– Built in Pseudo-Random Bit Stream (PRBS) generator
– Ability to read and write any internal register via the SMBus
– Ability to bypass link training and force any link into any mode
– Provides statistics and performance counters
Eight General Purpose Input/Output Pins
– Each pin may be individually configured as an input or output
– Each pin may be individually configured as an interrupt input
– Some pins have selectable alternate functions
Packaged in 27x27mm 420 ball BGA with 1mm ball spacing