ADLINK Technology aTCA-3710 User manual

1
aTCA-3710
40GbE AdvancedTCA
Fabric Interface Switch
User's Manual
Manual Revision: 1.01
Revision Date: March 18, 2015
Part Number: 50-1G032-1010
Advance Technologies; Automate the World.

aTCA-3710 User's Manual
2
Revision History
Rev Date Description
1.00 01/04/2014 Initial release
1.01 18/03/2015 Add hardware configuration settings

aTCA-3710 User's Guide
3
Preface
Copyright 2014-15 ADLINK Technology, Inc.
This document contains proprietary in formation protected by copyright. All rights are reserved. No part of this
manual may be reproduced by any mechanical, electronic, or other means in any form without prior written
permission of the manufacturer.
Disclaimer
The information in this document is subject to change without prior notice in order to improve reliability,
design, and function and does not represent a commitment on the part of the manufacturer. In no event will
the manufacturer be liable for direct, indirect, special, incidental, or consequential damages arising out of the
use or inability to use the product or documentation, even if advised of the possi bility of such damages.
Environmental Responsibility
ADLINK is committed to fulll its social responsibility to global environmental preservation through compliance
with the European Union's Restriction of Hazardous S ubstances (RoHS) directive and Waste Electrical and
Electronic Equipment (WEEE) directive. Environmental protection is a top priority for ADLINK. We have
enforced measures to ensure that our products, m anufacturing processes, components, and raw materials
have as little impact on the environment as possible. When products are at their end of life, our customers are
encouraged to dispose of them in acco rdance with the product disposal and/or recovery programs prescribed
by their nation or company.
Trademarks
Product names mentioned herein are used for identic ation purposes only and may be trademarks and/or
registered trademarks of their respective companies.

aTCA-3710 User's Guide
4
Table of Contents
Revision History .................................................................................................................................... ii
Preface................................................................................................................................................... iii
List of Tables......................................................................................................................................... vi
List of Figures ....................................................................................................................................... vi
1Overview........................................................................................................................................... 7
1.1 Introduction................................................................................................................................ 7
1.2 Board Overview ....................................................................................................................... 10
1.3 Functional Block Diagram ...................................................................................................... 11
1.4 Package Contents ................................................................................................................... 11
2Features Summary ........................................................................................................................ 12
2.1 Switch Board Specification .................................................................................................... 12
2.2 Placement................................................................................................................................. 16
2.3 Front panel Layout .................................................................................................................. 16
3External Interfaces ........................................................................................................................ 17
3.1 Front Panel............................................................................................................................... 17
3.2 Zone 2 Backplane Interfaces.................................................................................................. 19
4Software Component Overview ................................................................................................... 20
4.1 IPMI Subsystem....................................................................................................................... 20
4.2 LMP Subsystem....................................................................................................................... 21
4.3 Switch Subsystem................................................................................................................... 21
5IPMI Subsystem ............................................................................................................................. 23
5.1 Platform Management Overview............................................................................................ 23
5.2 Serial Over LAN (SOL) ............................................................................................................ 23
5.3 IPMI Sensors............................................................................................................................ 23
5.4 FRU Information ...................................................................................................................... 31
5.5 IPMI Commands ...................................................................................................................... 34
6LMP Subsystem............................................................................................................................. 36
6.1 Booting Image Redundancy................................................................................................... 36
6.2 LMP SDK/BSP Porting (U-Boot file system) ......................................................................... 38
6.3 Device Drivers ......................................................................................................................... 38
6.4 LMP State Machines................................................................................................................ 40
6.5 Boot Sequence ........................................................................................................................ 41
6.6 Non-volatile Memory Mapping ............................................................................................... 42
7Switch Subsystem......................................................................................................................... 43
7.1 Broadcom SDK ........................................................................................................................ 43
7.2 ADLINK Switch Software Overview....................................................................................... 44
8Getting Started............................................................................................................................... 46
8.1 Safety Requirements............................................................................................................... 46
8.2 Quick Start ............................................................................................................................... 46
9Configuring and Managing the Fabric Switch ............................................................................ 57
Appendix I. Advanced Operation ....................................................................................................... 58
Switching I/O between Front Panel and Base Switch ................................................................... 58
Switch UART Serial Port between Front Panel and IPMC ............................................................ 59
Appendix II. Software Setup/Update/Recovery ................................................................................ 60

aTCA-3710 User's Guide
5
U-Boot Installation............................................................................................................................ 60
BSP/Embedded Linux/BCM SDK Installation ................................................................................ 61
Redundant LMP Firmware & Boot...................................................................................................63
IPMI Firmware Update Procedure ................................................................................................... 64
Safety .................................................................................................................................................... 66
Consignes de Sécurité........................................................................................................................ 67
Getting Service .................................................................................................................................... 68

aTCA-3710 User's Guide
6
List of Tables
Table 1. Board Overview ............................................................................................................................ 10
Table 2. Serial Console Port Pinout ........................................................................................................... 17
Table 3. Serial Console Port Configuration Value...................................................................................... 17
Table 4. RJ-45 Management Port Pin-out .................................................................................................. 18
Table 5. LMP U-boot boot up failover actions ............................................................................................ 37
Table 6. Summary of Devices and Access................................................................................................. 39
Table 7. LMP State Descriptions ................................................................................................................ 40
Table 8. LMP State Transition Descriptions ............................................................................................... 41
Table 9. Boot Sequence State Descriptions............................................................................................... 41
Table 10. Non-volatile Memory Map........................................................................................................... 42
Table 11. Ground Header Settings (JP3) ................................................................................................... 48
Table 12. XAUI_SEL Status Settings (JP5)................................................................................................ 49
Table 13. JTAGSEL Pin Definition (JP8).................................................................................................... 49
Table 14. SPI Flash or SD Card Selection (SW1)...................................................................................... 51
Table 15. Faceplate Serial Port Selection (SW2)....................................................................................... 51
Table 16. Multi-function Switch (SW3) ....................................................................................................... 52
Table 17. Programmable Devices .............................................................................................................. 52
List of Figures
Figure 1. aTCA-3710 Block Diagram.......................................................................................................... 11
Figure 2. IPMC UART Interface.................................................................................................................. 13
Figure 3. aTCA-3710 Mechanical Layout................................................................................................... 16
Figure 4. aTCA-3710 Front Panel Layout................................................................................................... 16
Figure 5. aTCA-3710 Software Overview................................................................................................... 20
Figure 6. IPMI Function Blocks................................................................................................................... 21
Figure 7. Broadcom SDK Function Blocks ................................................................................................. 22
Figure 8. U-Boot Redundancy Flow ........................................................................................................... 37
Figure 9. LMP SDK/BSP Porting ................................................................................................................ 38
Figure 10. LMP State Machine ................................................................................................................... 40
Figure 11. aTCA-3710 Boot Sequence ...................................................................................................... 41
Figure 12. Broadcom SDK Porting ............................................................................................................. 43
Figure 13. Switch Software Blocks ............................................................................................................. 44
Figure 14. Hardware Configuration Switches and Jumpers ....................................................................... 47
Figure 15. Ground Header Schematic (JP3) .............................................................................................. 48
Figure 16. XAUI_SEL Status Header Schematic (JP5).............................................................................. 48
Figure 17. JTAGSEL Mode Header Schematic (JP8) ................................................................................ 49
Figure 18. IPMC JTAG Header Schematic (JP9)....................................................................................... 50
Figure 19. CPLD JTAG Header Schematic (JP12) ....................................................................................50
Figure 20. Switching I/O between Front Panel and Base Switch............................................................... 58

aTCA-3710 User's Guide
7
1 Overview
1.1 Introduction
The aTCA-3710 is a high performance, high bandwidth, 40G Fabric switch blade for AdvancedTCA®
platforms. This product is ideal for bandwidth intensive telecom applications such as wireless access
controller, DPI/security network, IPTV, IP multimedia subsystem, RNC/BSC, broadband access/bearer
network, data center, and LTE/4G network applications. The aTCA-3710 provides 40GbE hub-to-node
connectivity and integrated switch silicon that supports load balancing, priority queues, packet
classification, and flow control to enable strict bandwidth management for implemented applications.
To maximize switch blade cost effectiveness, two models of the aTCA-3710 40G Fabric switch blade are
available to meet different ATCA platform bandwidth requirements (aTCA-3710/320G and aTCA-
3710/640G). The aTCA-3710 40G Fabric switch incorporates the latest generation of switching
technology to provide 40GbE connectivity to node slots in 6 and 14-slot chassis as well as 40GbE I/O.
Additionally, the aTCA-3710 also provides front I/O to handle high throughput requirements of telecom
applications.
In addition to a Fabric switch, the aTCA-3710 also incorporates a Base switch to provide PICMG 3.0
Gigabit Ethernet backplane connectivity. The Base switch provides up to 13 ports of node slot
connectivity to support both 6 and 14-slot chassis in addition to 8 ports of front I/O, with 6 ports
supporting 1GbE and 2 ports supporting 10GbE. Other ports are used to provide connectivity to other
components on the board such as the Local Management Processor and IPMC, and a redundant switch
slot supports connectivity with the redundant base switch and shelf management. A powerful quad-core
Local Management Processor (LMP) executes all switch functions, blade setup and hardware platform
management functions. Dual Boot Flash Memory and dual OS Flash Memory is provided for redundancy.
The ADLINK aTCA-3710 is equipped with ADLINK networking software, which combines all the essential
features required for a fully functional switch infrastructure on ATCA platforms. This complete package
allows customers to focus on revenue-generating software development projects and provides the
flexibility to develop specific functions according to customer requirements.

aTCA-3710 User's Guide
8
The selected set of abbreviations frequently used in this document are listed below.
Term Description
ASIC Application Specific Integrated Circuit
ATCA Advanced Telecom Computing Architecture
Base Interface
ATCA Base Interface on the Zone-2 connectors to connect to the dual hubs, 4 differential
pairs per Base Channel.
Blade A printed circuit board assembly that plugs into a chassis
BSP Board Support Package
Boot-flash Flash memory where the bootloader program is stored
Bootloader Code used to boot the System before loading OS
DDR Double Data Rate DRAM memory
DFT/DFM Design For Test/Manufacturing
DIMM Dual Inline Memory Module
ECC Error Correction Control
E-keying
Standard defined by PICMG useful to verify whether the ATCA blade plugged in is compliant
to the fabric link capabilities
Fabric Interface ATCA Fabric Interface on the Zone-2 connectors to connect to the dual Switch Fabric Blades
Fabric Slot A slot supporting a link connection to/from each node slot
FRU Field Replaceable Unit
GPIO General Purpose Input/Output
Hot-swap Functions of replacing system components without shutting down the system
IPMB Intelligent Platform Management Bus
IPMB-L IPMB Local interface
IPMC Intelligent Platform Management Controller
IPMI Intelligent Platform Management Interface
JTAG Joint Test Action Group
MAC Media Access Controller
MMC Module Management Controller
NPU Network Processor Unit
OOS Out Of Service
PCI Peripheral Component Interconnect
PCIe Peripheral Component Interconnect-Express
PHY Physical Layer or Physical Layer Device
PICMG PCI Industrial Computer Manufacturers Group
RGMII Reduced Gigabit Medium Independent Interface
RTM Rear Transition Module
RXAUI Reduced XAUI, having 2 [email protected]
SDR Sensor Data Record

aTCA-3710 User's Guide
9
Term Description
SDRAM Synchronous Dynamic Random Access Memory
SFP Small Form Pluggable (1G interface)
SFP+ Small Form Pluggable Plus (10G interface)
SGMII Serial Gigabit Medium Independent Interface
SOL Serial Over LAN
QSFP+ Quad Small Form Pluggable Plus (40G interface)
UART Universal Asynchronous Receiver-Transmitter
VLP Very Low Profile, DRAM Module Form Factor Spec
XAUI 10 Gigabit (X) Attachment Unit Interface, having 4 [email protected]Gbps
XLAUI 40 Gigabit (XL) Attachment Unit Interface

aTCA-3710 User's Guide
10
1.2 Board Overview
Table 1. Board Overview
Feature Function Description
Processor Freescale QorIQ P2041(Quad Core) @ 1.2GHz
DRAM Memory 4GB DDR3 1333 Mini-RDIMM 1Rx8 VLP (512Mx8)
Boot Flash Memory PCT25VF080B 2x 8 Mbit SPI Serial Flash
LMP Subsystem
OS Flash Memory Industrial Grade SATA NANDrive 2x 8 GB (chip SSD)
Ethernet Switching BCM56842 320Gbps Bandwidth 40GbE switch fabric with QoS SupportData
Interconnection
Subsystem
Ethernet Switching BCM56846 640Gbps Bandwidth 40GbE switch fabric with QoS Support
Dual RJ-45 Conn. LAN from 2x BCM5466, supports 6 ports
Single RJ-45 Conn. LAN from 1x BCM54612, supports 1 port
Serial (UART),
console
UART#1 connects to either RJ-45 console port or to the SOL port
USB One USB 2.0 port
Reset button Recessed push button for blade reset
10G Interface from
I/O card (aDB-3710)
11x 10G Ethernet (SFP+) Fabric switch BCM56842/56846, 3x 10G Ethernet (SFP+) from
(aDB-3710) BCM8706, total 14 ports
I/O Interface
High Speed Re-
timer (aDB-3710)
Re-timer for SFP+ 10Gbs TI DS100DF410
SOL Interface PHY Transceiver BCM5221A4K architecture 10/100 control SOL port link (Smart Fusion A2F500 UART #0)
Microcontroller
Subsystem
Smart Fusion FPGA Shelf management and power management control sequence, redundant IPMB, IPMI v2.0-
SOL support
CPLD
Subsystem
Lattice LCMXFT256 Provides device multi-function GPIO control and sequencing
Base Interface Zone 2, provided by 4x BCM5466 supporting 13 ports 1000BASE-T Ethernet & 2 ports
1000BASE-T Ethernet to ShMC; PICMG 3.0 Base Interface switching – Gigabit Ethernet
(1.0Gbps)
Fabric Interface Zone 2, provided by BCM56842/BCM56846 PICMG 3.0 Rev 3.0 and PICMG 3.1 Rev 2.0
Fabric Interface switching – Option 3, 9, 9-KR (40GBASE-KR4), 3-KR (4x 10GBASE-KR)
Backplane
Connection
Power Interface Zone 1, provides -48VDC (SynQor power module spec)
Bootloader U-boot (LMP Freescale QorIQ P2041)
Operating System Open Linux OS
Development
Environment
Broadcom SDK running under Open Linux
Software
Firmware IPMC/IPMI over LAN (RMCP/IP) and Serial-over-LAN per PICMG 3.0; software license for
10G and 40G applications