PLX Technology PEX 8648 Quick reference guide

Using PEX 8648 SMA based
(SI) Card
White Paper
Version 1.3
July 2010
Website: www.plxtech.com
Technical Support: www.plxtech.com/support
Copyright © 2008 by PLX Technology, Inc. All Rights Reserved – Version 1.3
July 20, 2010

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 2
1 Introduction
Using a standard PC with PCIe slots, the PLX SMA based riser card allows quick connection and signal
observation into non-standard configurations (i.e. backplanes) thru use of SMA connectivity. With the PLX
SDK, the card allows programming either via I2C or PCIe in-band methods, to observe signal output,
adjust transmitter and receiver settings, check Gen 1 / Gen 2 link-up thru a channel, operate at a max x4
width, check system errors, run loopback and card to card signal integrity (SI) testing.
Figure 1. PLX SMA based Riser Card
2 Minimum Requirements
•A PC system with a x4 PCIe slot to provide power and system clock – if separate clocking
required – i.e. two PCs - PC must allow SSC clocking to be disabled.
oPCI-SIG baseboard with x4 connector for power and clock - optional substitute
oArdvaark I2C controller
•PLX SDK (version 5.22 or higher)
2.1 Additional References
•PLX PEX 8648 Quick Start Design Guide, www.plxtech.com/8648

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 3
3 Configuration
The PEX 8648 is a three station Gen 2 switch device with 48 maximum SerDes lanes. The SI RDK uses
only stations 0 and 1. The first station – Station 0 – contains both the default upstream port (Port 0 –
feeding a standard PCIe connector) and also four SMA output lanes on an additional port (Port 3 –
Downstream) – used for direct data observation and reception without the degradation of the PCIe
connector. The transmitter and receiver pairs of each lane of the port are clearly marked with lane and
polarity (however, it is not necessary to track lane polarity as this is automatically resolved by the PCIe
specification).
The PCIe connector at the top of the PCB is an alternate PCIe connection, located off of station1 of the
switch (Port 6). It is not typically needed for SI use, but if desired a second card (including another SI
RDK) can be installed and operated off the same base reference.
Below are block diagrams of the port connections (Figure 1) and clocking (Figure 2).
Figure 2. PEX 8648 SMA Card Port Configuration

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 4
Figure 3. PEX 8648 SMA Card Clocking
The default card configuration accepts the 100 MHz PCIe system clock from the bottom PCIe connector,
similar to a standard add-in card. Care should be used to ensure that any system clock used is of good
quality – reference clock jitter is a primary source of degraded signal performance.
If a clock other than the PCIe connector system clock is desired, the PCB can be modified to accept an
external, differential clock via SMA J17/18 and relocation of zero ohm resistors R133 and R134 to
location R135 and R136. (Schematic picture shown below) Any clock feeding the PCB is AC coupled on-
board and internally terminated within the chip. Signal swing should be typically 400mV, differential, pk-pk
(range 250mV min – 1600mV max). Frequency value should be 100 MHz +/- 300ppm max. The reference
clock feeding the chip is also made available for output via J24/25.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 5
Figure 4. PEX 8648 SMA Card Ref Clock Input Options
The default PCB configuration has Port 0 – the PCIe connector port – as the upstream connection. This
means it must be the port that points towards the PCIe root complex. This configuration allows a user to
directly connect the PCB into a PCIe system chassis, observe signals and begin using PLX software for
register manipulation without an I2C controller.
To move from signal observation to an adjustable PCIe link, .bin files are supplied, which demonstrate
how to convert the SMA port to the upstream link and operate as a standard PCIe link – progressing thru
link negotiation, and yet hold the desired channel tuning values. The various EEPROM .bin files contain
key SerDes registers used for:
9Upstream Port configuration
9Signal Swing
9De-emphasis
9RX equalization
9Electrical Idle threshold adjustment
With this method of operation (using EEPROM to set values), the EEPROM is programmed or adjusted
for the desired settings and the system is then reset to verify operation.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 6
4 Appendix
4.1 Obtaining a Gen 2 Compliance pattern for Observation
This can be done with I2C or In-band command (In-Band provided Port 3 is not upstream Port)
1. Connect the desired SMA TX lane to the high speed scope – the internal 50Ωtermination of the
scope is required for the circuit to operate.
2. With the PLX SDK active, open the memory mapped register set. (left side of screen)
3. Go to register 0234h bits 19:16 (Disable Port X). Set bits 19:16 to “8”. (disables Port 3)
4. Go to register 3098h (Link Status and Control). Set bit 4=1. (Enter Compliance).
Note: bits 3:0 should already have a register value of “2”
5. Go to register 0234h. Return bits 19:16 to “0”.
Terminated link is now in Gen 2 Compliance mode.
If it is desired to change the TX swing and emphasis settings, it can be done as below without resetting
the chip.
1. Go to register 0B90 (SerDes Drive Level 3) – Depress a register read. Value returned should
be the default Gen 2 value of “0E”.
2. Go to register 0BA0 (SerDes Post Cursor Emp) - Depress a register read. Value returned
should be the default Gen 2 value of “15”
3. With the Gen 2 values confirmed, you can now change these settings in accordance with the
formula outlined in the PEX 8648 Data Book for Swing and Pre-emphasis (Table 19-10 thru 19-
12) Swing setting is loosely approximated as 20log ((Drive + De-emp)/(Drive – De-emp))
As a point of note, we have observed some ‘bit flips’ when using I2C for changing these registers. To
confirm final setting, be sure to read back value after adjustment. Need to confirm against SDK SW
revisions.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 7
5 Obtaining PCIe Linkup with Alternate Transmitter Settings from
Default
This is most easily done by using the EEPROM to configure the four links of the SMA Port – Port 3.
An example .bin image is below for descriptive purposes. This bin was used to modify the configuration
and establish channel linkup along a 30” Tyco backplane, 6” of paddled card and Hz-Md connectors, and
~ 2 feet of coax cable.
In this file for Port 3,
oSerDes Drive level is set to the PEX 8648 Data Book (Table 19-10) setting of 0Ch.
oSerDes de-emphasis set to the PEX 8648 Data Book (Table 19-11) setting of 19h
oReceiver equalization left at default 0
oSignal Detection Threshold (EIDLE) – set to minimum (0) for Port 3 only
For RX equalization, Port 3 (SMA outputs) is controlled via bits 31 thru 19. By clicking on the “+” to the far
left of the actual bin file, the individual control bits are expanded. An expanded view is below, looking at
the SerDes Drive Level.
Note that in addition to the SerDes Drive level set to “0C”, the ‘Auto Load Disable bit is set.
This is required to prevent the PCIe link from overwriting the desired signal swing and emphasis settings
with the default Gen 2 values when the link is allowed to progress thru the normal PCIe linkup and Gen 2
negotiation process.
If connecting directly to a scope to observe and change the settings for the compliance pattern output,
setting this bit is not required.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 8
6 Bring Up of I2C
Following steps describe how to bring up the I2C Arrdvark controller.
1) Install 4 pin connect on HD1 header as shown below.
2) Power up the system with SI card and I2C header in place and USB port connected to system. With
SDK active, look under “Tools”. Select “Find I2C Device”
3) Locate and click “I2C Scan” button. Type “68” for device address. Next, locate “Select I2C Device
Type” and pull-down “8648AB”. Next depress “Find Devices”. You should see “+ 8648AB” appear in
lower right window after a few seconds. Next, depress “Refresh Device List” – it is okay that the
“8648” goes away on the screen.
4) To the left of the screen, below “Device Selector”, locate pull-down selector and select item “PLX(I2C)
Devices”. The PEX 8648 register and EEPROM menu are now visible on the left side of the page.
HD2
HD1
Ardvaark
Ribbon
Connector
Red Ribbon
wire

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 9
7 Using EEPROM Bin Files
7.1 How to configure upstream Port other than default Port 0?
The software needed is the PEX Device Editor included in the PLX SDK. The PLX SDK can be
downloaded from http://www.plxtech.com/products/sdk/.
7.2 Instructions
1. Plug the SI riser card in the PC with the SDK installed.
2. Launch the PEX Device Editor. You will see 8648 in {Device Selector} box.
a. Click [Tools] →[Program EEPROM]
b. Click [Browse…] button in Browse Bin File box, select .bin file *→Open
c. Click [PROGRAM EEPROM] button.
3. After confirmation of programming, shut down the PC. Remove the SI riser card.
4. Connect programmed SI riser card (#1 card) Port 3 to another SI riser card (or second device
nearer the root complex) by SMA coax cables (note cables are equal length). Connect TX to RX
and RX to TX of the two devices. Match lane numbers. For x1, use Lane 0. For x2, use Lane 0 to
Lane 0 and Lane 1 to Lane 1… and so on...
* Two .bin files are provided. “SI_Port3_Upstream.bin” and “SI_Port3_Adjust.bin”.
Shown below is the exact “SI_Port3_Upstream.bin” file that will execute this action.
Configuring the .bin file exactly as above will result in Port 3 of the card to be configured as the upstream
port, and as such, should point toward the root complex. Note the register load is done in two steps. If
editing an alternate .bin file to include a change of upstream ports, use the same two step sequence.
Below, “SI_Port3_Adjust.bin” use file to operate the link in PCIe mode, while adjusting the SerDes
settings and configuring Port 3 as upstream. Here, drive is set to the PEX 8648 Data Book value of
600mV and resultant de-emphasis is set for ~ 10dB.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 10
8 Using Two SI Cards for Link Testing
Another means of testing employs the use of two SI cards. This approach can be used in three
connectivity scenarios:
1) One root complex (PC) to control the PCIe link and one base card (i.e. PCIe compliance load
board) to provide power to the downstream card. The default clocking for this configuration is
asynchronous (hence PC clock must not set SSC active) or an out of band reference clock is fed
into the downstream card. The downstream card is configured to have the SMA port as the
Upstream. (Note: it is necessary for both cards to see at least one reset in order to initiate link-up
negotiation. If needed, manually depress card reset switch)
2) Two root complex systems – each SI card is placed in a PC and the SMA ports connected.
Typically, in this configuration, NT is employed to allow Host-to-Host connectivity. Standard
clocking is again asynchronous. (Note: it is necessary for both cards to see at least one reset in
order to initiate link-up negotiation. If needed, manually depress card reset switch)
3) One root complex, stacked connectivity – as shown below. In this configuration, coherent
clocking is maintained between cards. Consequently, the PC can have SSC enabled. If alternate
reference clock injection is desired, it is done so by changing the RefClk input configuration on
the bottom SI Card unit. Examples for this configuration are discussed.
Figure 5. Single PC connectivity

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 11
8.1 Digital Loopback (Two Cards)
•Step 1: Configuring port 3 on card #2 as the upstream port;
oPlug card #2 in MB and boot the system.
oOpen PEX Device Editor (PDE)
oClick on EEPROM editor.
oAdd register 0x1dc (default value should be 0x10200080)
oAdd register 1dc again and program the value 0x10200300 (port 3 is upstream port).
oShut down system.
•Step 2 -Install Cards
oRemove card #2 from MB.
oPlug card #1 in MB.
oPlug card #2 in slot provided on card #1. This will provide power and clock to card #2.
oConnect card #1 Port 3 to card #2 Port 3 by SMA coax cables (note cables are equal
length). Connect TX of card #1 to RX of card #2. Match Lane numbers. For x1, use Lane
0, for x2 use Lane 0 and Lane 1. (You can add backplane in the path once you have
verified that the initial setup works).
•Digital Loopback Test
oReboot system.
oLink between both cards should be up. You will see port 3 LED on the cards blink.
oLaunch PDE and using memory map put port 3 in loopback. The following registers are
programmed on card #1 using memory map.
Program registers 0x210-0x21C with any pattern.
Set 0x230[12] to enable loopback on lanes 12-15.
Check to see if loopback is established, 0x230[15] = 1.
Set 0x228[31] to enable user pattern on lanes 12-15.
Check 0x244 “SerDes quad 3 diagnostic data” for error count. This should be
zero.
Change pattern in register 0x210.
Check 0x244. You should see error count.
Reset bits 0x228[31], 0x254[19] and 0x230[12].
Reboot system
Repeat test with backplane in path. Change SerDes settings and check for error counts.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 12
8.2 Analog Loopback (Single Card)
Configuration: Upstream Port = Port 0
This test loops the SMA output from Port 3 transmitter(s) to Port 3 receiver(s).
•Analog Loopback (Cable) with UTP (on Port 3)
oEnable External Loopback by EEPROM: Write PHY Additional Status Register
254h[19:16]=8h in EEPROM. Power down system.
oWire the corresponding transmitter to receiver connections (either x1, x2 or x4)
oReboot the system.
oSet PHY User Pattern Register 210h through 21Ch with desired data pattern.
oEnable Loopback Master: Set Physical Layer Port Command Register 230h[12]. Verify
bit [15] is set.
oEnable User Test Pattern: Set Physical Layer Test Register 228h[31:28]=8h.
Now to check link status / errors;
oRead SerDes Quad 3 Diagnostic Data Register 244h. Verify User Test Pattern mode
active via bit [30]=0 (UTP checker)
oSelect desired SerDes (0 thru 3) for error checking via register 244h, bits [24:25] where 00
= SerDes channel 0.
oCheck error count for selected SerDes – bits [23:16] for UTP/PRBS error count. If link is
operating properly, value should be zero.
oIf more than one lane is connected, select alternate SerDes and check corresponding lane
errors per above.
Figure 6. Error Register

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 13
9 Using Packet Generator in Two Card Set
In this example, the PLX PEX 8648 Packet Generator and Monitor features are used to generate PCI
Express heavy traffic across the link under test.
•Step 1) Program Port 3 as upstream port on card # 2 as outlined in Loopback test.
•Step 2) Install cards as outlined in Loopback test.
•Step 3) Programming packet generator and monitor features
oUnder programs bring up “PLX GENMON”
oGo to ‘Performance Monitor’ box at top of screen. SI cards are identified by bus number.
The device nearest the root complex will have the lower bus number. Select this device
as monitor (either will work). Now depress “Open Monitor” button.
oWith Monitor screen now open, select “Port 3 Ingress” and “Port 3 Egress”. You can
optionally deselect the Port 0 monitors. Depress “Start”. Minimize screen.
oStarting Traffic Generator:
Go back to main GENMON screen. Select the PEX 8648 chip with the higher bus
number – this is the required traffic generator. Go to “Load File” and load the SI
Card supplied script “Upstream_Exerciser.PLE”. Depress “Start”.
oLooking at the Monitor screen – traffic should now be evident.
•Step 4) Check bad DLLP and bad TLP counters in 8648 on card #1 and card #2
oStart standard SDK software.
oIn either chip read Port 3 memory mapped registers for TLP/DLP error count. (Registers
x31E8h / x31ECh) These registers keep a running tab of DLLP and TLP errors on Port 3.
To clear write all “0” to each register.
•Step 5) Change SerDes settings and check for errors.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 14
10 Using External BERT (Analog Slave Loopback)
This mode allows the user to pre-configure the SI Card (Port 3) in slave analog loopback such that non-
PCIe compliant patterns (such as PRBS) can be injected via external means and then externally checked.
Figure 7. Configuration Drawing Example BERT application
Configuration of this mode can be done via EEPROM (example bin file included). Setup requires several
writes to the base register in Station 0 (Port 0) and a write to the Port 3 specific register. As a point of
clarity, note that Port 3 consists of SerDes numbers 12-15 (Port 3 Lane 0-3). These are the channels that
will be placed into loopback.
Programming Steps;
•BERT (slave analog loopback)
oRegister 0204 – set SerDes 12-15 to Mask EIDLE (Fh)
oRegister 0234 – set Port 3 bits 23:20 to Port Quiet (8h)
oRegister 0234 – set Port 3 bits 27:24 for rate (8h)
oRegister 0BE4 – set value bits 31:0 to (80024745) SerDes overrides
System Box – CLK/PWR
BERT and System Offset within +/-300ppm
Ext
CLK
Ext
CLK
For Synchronous Operation, can reconfigure
PCB for external clocking. PCB CLKOUT feeds BERT
Detector.

Using PEX 8648 SMA Based (SI) Card White Paper – Version 1.3
© 2010 by PLX Technology, Inc. All Rights Reserved 15
To see the representative programming via EEPROM, the appropriate bin file is below;
Figure 8. Example of Programming for Slave Loopback via EEPROM
Note: the above bin file is provided as part of the SI card documentation. Using the PEX SDK, this bin can
be directly programmed into the card. File name; SI_Card_BERT_Loopback.bin.
11 Revision History
Date Version Revision
September 2008 1.0 Initial Release
October 2008 1.2 Added Analog Slave Loopback for BERT operability.