N.A.T. NAT-MCH Product manual

NAT-MCH HUB-Module PCIe –Technical Reference Manual
NAT-MCH
TCA Telecom MCH Module
Technical Reference Manual V 1.2
HUB-Module PCIe HW Revision 2.3

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 2
The NAT-MCH HUB-Module PCIe has been designed by:
N.A.T. GmbH
Konrad-Zuse-Platz 9
53227 Bonn-Oberkassel
Phone: +49 / 228 / 965 864 –0
Fax: +49 / 228 / 965 864 –10
Internet: http://www.nateurope.com

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 3
Disclaimer
The following documentation, compiled by N.A.T. GmbH (henceforth called N.A.T.),
represents the current status of the product’s development. The documentation is
updated on a regular basis. Any changes which might ensue, including those necessitated
by updated specifications, are considered in the latest version of this documentation.
N.A.T. is under no obligation to notify any person, organization, or institution of such
changes or to make these changes public in any other way.
We must caution you, that this publication could include technical inaccuracies or
typographical errors.
N.A.T. offers no warranty, either expressed or implied, for the contents of this
documentation or for the product described therein, including but not limited to the
warranties of merchantability or the fitness of the product for any specific purpose.
In no event will N.A.T. be liable for any loss of data or for errors in data utilization or
processing resulting from the use of this product or the documentation. In particular,
N.A.T. will not be responsible for any direct or indirect damages (including lost profits,
lost savings, delays or interruptions in the flow of business activities, including but not
limited to, special, incidental, consequential, or other similar damages) arising out of the
use of or inability to use this product or the associated documentation, even if N.A.T. or
any authorized N.A.T. representative has been advised of the possibility of such
damages.
The use of registered names, trademarks, etc. in this publication does not imply, even in
the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations (patent laws, trade mark laws, etc.) and therefore free
for general use. In no case does N.A.T. guarantee that the information given in this
documentation is free of such third-party rights.
Neither this documentation nor any part thereof may be copied, translated, or reduced to
any electronic medium or machine form without the prior written consent from N.A.T.
GmbH.
This product (and the associated documentation) is governed by the N.A.T. General
Conditions and Terms of Delivery and Payment.
Note:
The release of the Hardware Manual is related to a certain HW board revision
given in the document title. For HW revisions earlier than the one given in the
document title please contact N.A.T. for the corresponding older Hardware
Manual release.

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 4
Table of Contents
DISCLAIMER ....................................................................................................... 3
TABLE OF CONTENTS .......................................................................................... 4
LIST OF TABLES .................................................................................................. 5
LIST OF FIGURES................................................................................................ 5
CONVENTIONS.................................................................................................... 6
1INTRODUCTION ........................................................................................... 7
2OVERVIEW ................................................................................................... 8
2.1 MAJOR FEATURES......................................................................................... 8
2.2 BLOCK DIAGRAM ......................................................................................... 9
2.3 LOCATION DIAGRAM ....................................................................................10
3BOARD FEATURES ...................................................................................... 11
3.1 PCI EXPRESS SWITCH PLX PEX8748 ...............................................................11
3.2 MICROCONTROLLER .....................................................................................13
3.3 FPGA .....................................................................................................13
3.4 MULTIPLEXING UNITS ...................................................................................13
3.5 PCIE INTERFACES .......................................................................................13
3.6 INTERFACE TO NAT-MCH BASE-MODULE ..........................................................13
3.7 INTERFACE TO NAT-MCH CLK-MODULE ............................................................13
4HARDWARE ................................................................................................ 14
4.1 LEDS......................................................................................................14
4.2 CONNECTORS ............................................................................................15
4.2.1 CON1: AMC Connector to 3rd tongue ...................................................16
4.2.2 CON2: AMC Connector to 4th tongue ...................................................18
4.2.3 CON3: Connector to NAT-MCH CLK/BASE-Module .................................20
4.2.4 JP1: JTAG programming interface .......................................................20
4.2.5 J1: Connector to optional Root-Complex..............................................21
5PROGRAMMING NOTES .............................................................................. 22
5.1 SPI INTERFACE ..........................................................................................22
5.1.1 SPI-Interface –Default mode.............................................................22
5.1.2 SPI-Interface –Update mode .............................................................22
5.2 I²C INTERFACE ..........................................................................................22
5.3 REGISTER.................................................................................................22
5.3.1 Board Identifier Register ...................................................................22
5.3.2 PCB Revision Register .......................................................................23
5.3.3 Atmel Version ..................................................................................23
6BOARD SPECIFICATION ............................................................................. 24
7INSTALLATION .......................................................................................... 25
7.1 SAFETY NOTE ............................................................................................25

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 5
7.2 INSTALLATION PREREQUISITES AND REQUIREMENTS ...............................................25
7.2.1 Requirements ..................................................................................25
7.2.2 Power supply ...................................................................................25
7.2.3 Automatic Power Up..........................................................................25
7.3 STATEMENT ON ENVIRONMENTAL PROTECTION ......................................................26
7.3.1 Compliance to RoHS Directive ............................................................26
7.3.2 Compliance to WEEE Directive............................................................26
7.3.3 Compliance to CE Directive ................................................................27
7.3.4 Product Safety .................................................................................27
7.3.5 Compliance to REACH .......................................................................27
8KNOWN BUGS / RESTRICTIONS................................................................. 28
APPENDIX A: REFERENCE DOCUMENTATION .................................................... 29
APPENDIX B: DOCUMENT’S HISTORY ............................................................... 30
List of Tables
Table 1: List of used Abbreviations .......................................................................... 6
Table 2: PCIe Switch Lane to MCH Fabric Port Mapping .............................................11
Table 3: Switch Lanes to AMC port mapping for X4-Link on a standard MicroTCA-
Backplane .....................................................................................................12
Table 4: LED State –Link Status ............................................................................14
Table 5: CON1: AMC Connector to 3rd tongue –Pin Assignment..................................16
Table 6: CON2: AMC Connector to 4th tongue –Pin Assignment..................................18
Table 7: CON3: Connector to NAT-MCH CLK/BASE-Module –Pin Assignment ...............20
Table 8: J1: Connector to optional Root-Complex –Pin Assignment ............................21
Table 9: Board Identifier Register ...........................................................................22
Table 10: PCB Revision Register .............................................................................23
Table 11: Atmel Revision Register ..........................................................................23
Table 12: NAT-MCH HUB-Module PCIe Features........................................................24
List of Figures
Figure 1: Arrangement of different NAT-MCH Modules............................................. 7
Figure 2: Figure 2: NAT-MCH HUB-Module PCIe –Block Diagram.............................. 9
Figure 3: NAT-MCH HUB-Module PCIe –Location Diagram (top) ..............................10
Figure 4: NAT-MCH HUB-Module PCIe –Location Diagram (bottom).........................10
Figure 5: NAT-MCH HUB-Module PCIe –Connectors (top) .......................................15
Figure 6: NAT-MCH HUB-Module PCIe –Connectors (bottom) .................................15

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 6
Conventions
If not otherwise specified, addresses and memory maps are written in hexadecimal
notation, identified by 0x.
The following table gives a list of the abbreviations used in this document:
Table 1: List of used Abbreviations
Abbreviation
Description
AMC
Advanced Mezzanine Card
b
bit, binary
B
byte
ColdFire
MCF5470
CPU
Central Processing Unit
CU
Cooling Unit
DMA
Direct Memory Access
E1
2.048 Mbit G.703 Interface
FLASH
Programmable ROM
FRU
Field Replaceable Unit
J1
1,544 Mbit G.703 Interface (Japan)
K
kilo (factor 400 in hex, factor 1024 in decimal)
LIU
Line Interface Unit
M
mega (factor 10,0000 in hex, factor 1,048,576 in decimal)
MCH
µTCA Carrier Hub
MHz
1,000,000 Herz
µTCA
Micro Telecommunications Computing Architecture
PCIe
PCI Express
PCI
Peripheral Component Interconnect
PM
Power Manager
RAM
Random Access Memory
ROM
Read Only Memory
SDRAM
Synchronous Dynamic RAM
SSC
Spread Spectrum Clock
T1
1,544 Mbit G.703 Interface (USA)

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 7
1Introduction
The NAT-MCH consists of a BASE-Module, which can be expanded with additional
PCBs. The BASE-Module satisfies the basic requirements of the MicroTCA Specification
for a MicroTCA Carrier Hub. The main capabilities of the BASE-Module are:
management of up to 12 AMCs, two cooling units (CUs) and one or more
power modules (PMs)
Gigabit Ethernet Hub Function for Fabric A (up to 12 AMCs) and for the
Update Fabric A to a second (redundant) NAT-MCH
To meet also the optional requirements of the MicroTCA specification, a CLK-Module and
different HUB-Modules are available. With the CLK-Module the following functions can
be enabled:
generation and distribution of synchronized clock signals for up to 12 AMCs
Through the extension of the NAT-MCH with a HUB-Module, hub functions for fabric D
to G can be enabled. With the different versions the customers have the opportunity to
choose a HUB-Module that fits best to their applications. The versions differ in:
max. number of supported AMCs ( up to 6 / up to 12)
supported protocols:
oPCI Express
oSerial Rapid IO
o10Gigabit Ethernet
The features of the individual modules are described in more detail in the corresponding
Technical Reference Manuals.
A general arrangement of the different modules of a NAT-MCH is shown in the following
figure
Figure 1: Arrangement of different NAT-MCH Modules
LED Module
Basic Module CLK Module
Hub Module
This Technical Reference Manual describes the NAT-MCH HUB-Module PCIe. In
addition to the CLK-Module it can be mounted on the NAT-MCH BASE-Module. With
the NAT-MCH HUB-Module PCIe the 3rd tongue and 4th tongue of the NAT-MCH
connector to the MicroTCA backplane is installed.

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 8
2Overview
2.1 Major Features
PLX PEX8748 PCI Express switch
connection to AMC 1-6 (via 3rd tongue) and to AMC 7-12 (via 4th tongue)
non-blocking switching at full line rate
Quality of Service (QoS)
2 virtual cannels and 8 traffic classes per port supported
Configuration of one of all 12 AMC ports as transparent and non-
transparent upstream port each
Atmel ATmega1284 microcontroller
Configuration of PCIe switch
Support of Hot-Swap functionality
Lattice MachXO2 FPGA
Support of Hot-Plug functionality for each AMC-Slot
Two PI3PCIE3412 multiplexing units
Connection to AMC 12 (via 4th tongue)
Switching between AMC 12 and optional Root Complex (double-width NAT-
MCH BASE-Module only)
PCIe x1 and x4 switching function
Connection of fabrics D to G of up to 6 AMCs (NAT-MCH HUB-Module
PCIe x24)
Connection of fabrics D to G of up to 12 AMCs (NAT-MCH HUB-Module
PCIe x48)
Connection to optional Root Complex (double-width NAT-MCH BASE-
Module only)
Clustering support –up too six clusters can be operated individually, each having
its own Root Complex
PCIe compliant Spread Spectrum Clocking

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 9
2.2 Block Diagram
The following figure shows a block diagram of the NAT-MCH HUB-Module PCIe.
Figure 2: Figure 1: NAT-MCH HUB-Module PCIe –Block Diagram
backplane connector
tongue 4 (Harting Plug)
IPMI
NAT-MCH
HUB-Module PCIe
Gen III –HW v2.3
to AMC 7-11
x4 PCIe
Micro-
controller
Connector from
Base-/CLK-Module
PCI Express
SWITCH
PLX PEX8748
backplane connector
tongue 3 (Harting Plug)
Management-I²C
fabric D to G
x4 PCIe
to AMC 1-6
(Optional)
Root Complex
PCIe-Clock from
CLK-Mezzanine
fabric D to G
MUX
fabric D to G
x4 PCIe
to AMC 12
Temp
Sensor
x4 PCIe
Only on Double-Width
MCH Base Module
Load
PROM
SPI
SPI
FPGA
Link Status
LED 1-12
HP-I²C

NAT-MCH HUB-Module PCIe –Technical Reference Manual
Version 1.2 © N.A.T. GmbH 10
2.3 Location Diagram
The following location diagram of the NAT-MCH HUB-Module PCIe shows the position
of important components.
Figure 3: NAT-MCH HUB-Module PCIe –Location Diagram (top)
A
M
C
C
O
N
N
E
C
T
O
R
3rd
tong
PCI Express
Switch
PLX PEX8748
Power
Supply
Osc.
MUX
A
M
C
C
O
N
N
E
C
T
O
R
4th
tongue
Load
PROM
Micro-
controller
Temp
Sensor
L
E
D
1
-
12
L
i
n
k
S
t
a
t
u
s
Power
Supply
Power
Supply
MUX
FPGA
Clock
Buffer
Heatsink
Figure 4: NAT-MCH HUB-Module PCIe –Location Diagram (bottom)
A
M
C
C
O
N
N
E
C
T
O
R
4th
tong
A
M
C
C
O
N
N
E
C
T
O
R
3rd
tongue
Temp
Sensor