Edge-Core AS7326-56X Operating instructions

EDGECORE NETWORKS CORPORATION 2018
1
EDGECORE Networks Corporation
Edgecore AS7326-56X
Programming Guide
Revision 1.0

EDGECORE NETWORKS CORPORATION 2018
2
Revision History
Revision
Date
Author
Description
1.0
9/15/2018
Jeff Catlin
Initial Draft

EDGECORE NETWORKS CORPORATION 2018
3
Contents
Revision History............................................................................................................................................... 2
Licenses........................................................................................................................................................... 7
1. Introduction ............................................................................................................................................ 9
1.1. Reference Documents............................................................................................................. 9
1.2. Acronyms and Terminology .................................................................................................... 9
2. Hardware Architecture ......................................................................................................................... 10
2.1. Overview ............................................................................................................................... 10
2.2. Block Diagram ....................................................................................................................... 12
2.2.1. Clock Tree.............................................................................................................................. 13
2.2.2. Power Tree............................................................................................................................ 14
2.2.3. Reset Tree ............................................................................................................................. 16
2.3. Front/Rear Placement........................................................................................................... 18
3. Sub-system of CPU Board ..................................................................................................................... 18
3.1. Configurations of CPU........................................................................................................... 19
3.1.1. POR of CPU............................................................................................................................ 20
3.2. 1G Interface .......................................................................................................................... 23
3.3. Software Configurations of CPU ........................................................................................... 24
3.4. Memory Mapping ................................................................................................................. 31
3.5. FLASH .................................................................................................................................... 36
3.6. RAM....................................................................................................................................... 37
3.7. PCIe ....................................................................................................................................... 38
3.8. I2C/SMBus Architecture........................................................................................................ 39
3.9. SATA ...................................................................................................................................... 41
3.10. 10G-Base-KR.......................................................................................................................... 42
3.11. BMC Module (Option)........................................................................................................... 42
3.11.1. LPC......................................................................................................................................... 43
3.11.2. PCIE ....................................................................................................................................... 43
3.11.3. SPI.......................................................................................................................................... 43

EDGECORE NETWORKS CORPORATION 2018
4
3.11.4. SMBUS................................................................................................................................... 43
3.11.5. UART...................................................................................................................................... 43
3.11.6. USB........................................................................................................................................ 44
3.11.7. GPIO ...................................................................................................................................... 44
3.12. CPLD (For CPU Board) ........................................................................................................... 46
3.13. CPLD Architecture (For CPU Board) ...................................................................................... 47
3.13.1. Power Sequence ................................................................................................................... 47
3.13.2. Reset ..................................................................................................................................... 47
3.13.3. UART...................................................................................................................................... 47
3.13.4. BMC....................................................................................................................................... 47
3.13.5. Interrupt................................................................................................................................ 48
3.13.6. SDATA.................................................................................................................................... 48
3.13.7. Thermal................................................................................................................................. 49
3.14. CPLD Register (For CPU Board) ............................................................................................. 50
3.14.1. Offset 0x00 Board Info (Read Only)...................................................................................... 50
3.14.2. Offset 0x01 CPLD version (Read Only) .................................................................................. 50
3.14.3. Offset 0x02 BIOS boot flash select (Read Only).................................................................... 50
3.14.4. Offset 0x03 System Reset-1 (Read& Write).......................................................................... 50
3.14.5. Offset 0x04 Reset - Device (Read& Write)............................................................................ 51
3.14.6. Offset 0x05 System Interrupt (Read& Write) ....................................................................... 51
3.14.7. Offset 0x06 System Error (Read& Write).............................................................................. 51
3.14.8. Offset 0x07 Thermal Status (Read& Write) .......................................................................... 52
3.15. LED ........................................................................................................................................ 52
3.16. GPIO ...................................................................................................................................... 54
4. Sub-system of MAC (BCM56873).......................................................................................................... 58
4.1. Configurations of MAC (BCM56873)..................................................................................... 59
4.2. POR of MAC (BCM56873)...................................................................................................... 60
4.3. Port Mapping ........................................................................................................................ 61
4.4. 10G/25G/40G/100G Interface .............................................................................................. 65
4.5. LED interface......................................................................................................................... 65
4.6. QSFP28.................................................................................................................................. 67

EDGECORE NETWORKS CORPORATION 2018
5
4.7. SFP28..................................................................................................................................... 67
4.8. SFP+....................................................................................................................................... 67
5. Sub-system............................................................................................................................................ 68
5.1. Management PHY (BCM54616S) .......................................................................................... 68
5.2. Configurations of MGMT PHY (BCM54616S)........................................................................ 68
5.3. POR of MGMT PHY (BCM54616S)......................................................................................... 68
5.4. I2C ......................................................................................................................................... 68
5.5. UART...................................................................................................................................... 72
5.6. USB........................................................................................................................................ 73
5.7. Interrupt................................................................................................................................ 73
5.8. JTAG ...................................................................................................................................... 74
5.9. LED Indicator......................................................................................................................... 75
5.9.1. Status LED ............................................................................................................................. 75
5.9.2. Port LED................................................................................................................................. 76
5.9.3. Management Port LED.......................................................................................................... 78
5.10. Button ................................................................................................................................... 79
5.11. Thermal system..................................................................................................................... 79
5.11.1. Temperature sensor.............................................................................................................. 79
5.11.2. Fan controller system............................................................................................................ 81
5.12. CPLD ...................................................................................................................................... 83
5.12.1. CPLD1 pin-out list.................................................................................................................. 86
5.12.2. CPLD 1 Register ..................................................................................................................... 92
5.12.3. CPLD2 pin-out list................................................................................................................ 108
5.12.4. CPLD 2 Register ................................................................................................................... 116
5.12.5. CPLD3 pin-out list................................................................................................................ 141
5.12.6. CPLD 3 Register ................................................................................................................... 146
5.12.7. Fan CPLD pin-out list........................................................................................................... 166
5.12.8. Fan board Register.............................................................................................................. 168
5.12.9. CPLD Field upgrade information......................................................................................... 176
5.13. IDT 8V89307........................................................................................................................ 177
5.13.1. Configurations of IDT 8V89307........................................................................................... 177

EDGECORE NETWORKS CORPORATION 2018
6
5.13.2. POR of IDT 8V89307............................................................................................................ 177
5.14. PSU...................................................................................................................................... 178
5.14.1. Pinout.................................................................................................................................. 178
5.14.2. Dimension ........................................................................................................................... 178
5.14.3. Efficiency............................................................................................................................. 179
5.14.4. Power Supply Management Controller (PSMC).................................................................. 179
5.14.5. Power Supply Field Replacement Unit (FRU)...................................................................... 180
5.14.6. PSMC Sensors...................................................................................................................... 180
5.14.7. LEDs of Power Supply units................................................................................................. 181

EDGECORE NETWORKS CORPORATION 2018
7
Licenses
All semiconductor devices that may be referred to in this specification, or required to manufacture
products described in this specification, will be considered referenced only, and no intellectual property
rights embodied in or covering such semiconductor devices shall be licensed as a result of this specification
or such references. Notwithstanding anything to the contrary in the CLA, the licenses set forth therein do
not apply to the intellectual property rights included in or related to the semi-conductor devices identifies
in the specification. These references include without limitation the reference to devices listed below. For
clarity, no patent claim that reads on such semiconductor devices will be considered a “Granted Claim”
under the applicable Contributor License Agreement for this specification
Component
Vendor
MFG P/N
Quantity
Remark
CPU Board-CPU
Intel
Broadwell-DE XeonD-1518
1
MAC
Broadcom
BCM56873
1
NIC
Broadcom
BCM5720
1
EMP PHY
Broadcom
BCM54616S
1
CPLD
Altera
5M1270ZF256C5N
1
For CPU system
CPLD
Altera
5M1270ZF256C5N
2
For Main board System
and QSFP28 LED
CPLD
Altera
5M2210ZF324C5N
1
For SFP28 LED
CPLD
Altera
5M1270ZF256C5N
1
For Fan board System
I2C Switch
NXP
PCA9548APW
10
Thermal Sensor
NS
LM75BD/SO8
3
USB HUB IC
SMSC
USB2514B-AEZC
1
Power Monitor
Lattice
POWR1014A
1
UART IC
TI
MAX3232CPWR
1
Transceiver
PSU (AC/DC)
3Y
YM-2651Y,B01R
2
650W AC

EDGECORE NETWORKS CORPORATION 2018
8
As of September 15, 2018, this specification is contributed under the OCP Contributor Licensing
Agreement (OCP-CLA) by the following entities:
Acton Technology Corporation, through its subsidiary Edgecore Networks Corporation
Limitations of the OCP CLA license are noted below:
No Limitations
You can review the signed copies of the OCP-CLA for this specification on the OCP
website. http://www.opencompute.org/products/specsanddesign
Usage of this specification is governed by the OCPHL permissive. You can review this license at
http://www.opencompute.org/participate/legal-documents/
Your use of this Specification may be subject to other third-party rights. THIS SPECIFICATION
IS PROVIDED "AS IS." The contributors expressly disclaim any warranties (express, implied, or
otherwise), including implied warranties of merchantability, non-infringement, fitness for a
particular purpose, or title, related to the Specification. The entire risk as to implementing or
otherwise using the Specification is assumed by the Specification implementer and user. IN NO
EVENT WILL ANY PARTY BE LIABLE TO ANY OTHER PARTY FOR LOST PROFITS OR ANY
FORM OF INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY
CHARACTER FROM ANY CAUSES OF ACTION OF ANY KIND WITH RESPECT TO THIS
SPECIFICATION OR ITS GOVERNING AGREEMENT, WHETHER BASED ON BREACH OF
CONTRACT, TORT (INCLUDING NEGLIGENCE), OR OTHERWISE, AND WHETHER OR
NOT THE OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

EDGECORE NETWORKS CORPORATION 2018
9
1. Introduction
The Switch is a 1U high and 536mm deep chassis base on Broadcom Trident3 chipset.
The physical layer will consist of 48x25G SFP28, 8x100G QSFP28 ports and 2x10G SFP+
ports. The chassis will have a nominal operating temperature range of 0 to +40 Degree C.
The CPU board is for management of the switch board , the solution is Intel BroadWell-
DE, that CPU board provide these interface: x4 PCIe2.0, SGMII, MDC/MDIO, USB2.0, and
2channel I2C connect to the switch board. There are mSATA and eUSB devices in the CPU
board and memory support DDR4 with ECC sodimm.
The following are key features of the product:
Redundant and hot-swappable PS (1+1)
Rear facing for all connections
Redundant and hot swappable fans (4+1)
1U rank mountable or desktop target 515mm
48 SFP28 25G + 8 QSFP28 40G/100G + 2 10G SFP+ ports
CPU module
➢CPU: Intel Broadwell-DE XeonD-1518
➢DDR SDRAM: 8GB x 2 2133MHz with ECC (DDR4 SO-DIMM)
➢SPI Flash (Boot): 16MB
➢USB to NAND Flash memory : 8GB SLC
➢mSATA: 32GB MLC (Reserve)
➢m.2 SATA: 64GB MLC
USB port (5V/1A)
SyncE & 1588 support (Reserve)
Environmental goal operation:
Front to Back Air Flow (0 to +45 degree C)
Back to Front Air Flow (0 to +45 degree C)
1.1. Reference Documents
Broadcom BCM56870 Data Sheet
IDT89307 WANPLL Data Sheet
544040_Broadwell_DE_EDS_vol1_544040_rev1p0.pdf
544041_Broadwell_DE_EDS_Registers_Vol2_544041_v1_0.pdf
544042_Broadwell_DE_SoC_EDS_vol3_544042_v1_0.pdf
544043_Broadwell_DE_EDS_rev_0_75.pdf
544044_Broadwell_DE_EDS_Vol5_544044_rev0_71.pdf
BCV-R-SB_SCH_20140804.pdf
CamelBackMnt_FAB-B_NCOR2_09-09-14.pdf
1.2. Acronyms and Terminology
Some of the acronyms used in this document are listed below:
CCSR Configuration, Control, and Status Register
PCB Printed Circuit Board
PCBA Printed Circuit Board Assembly
PSU Power Supply Unit
BMC Baseboard management controller.
POR Power On Reset

EDGECORE NETWORKS CORPORATION 2018
10
RU Rack Mount Unit (equivalent to 1.75 inches)
SERDES Serializer / Deserializer
SKU Stock Keeping Unit
SMI Serial Management Interface
FRU Field Replaceable Unit
GPIO General Purpose Input Output
IP Internet Protocol
NP Network Processor
CPLD Complex Programmable Logic Device
RXAUI Reduced XAUI
RU Rack Mount Unit (equivalent to 1.75 inches)
SGMII Serial GMII
CAUI 10G/25G Attachment Unit Interface
CAUI-4 40G/100G Attachment Unit Interface
SFP28 25G/10G Small Form-Factor Pluggable
QSFP28 100G/40G Small Form-Factor Pluggable
2. Hardware Architecture
This chapter describes the architecture of PCBA, cooling, power consumption, electrical,
reset, clocks, Ethernet port mapping etc..
2.1. Overview
The AS7326-56X provides 48x25G SFP28, 8x100G QSFP28 and 2x10G SFP+ ports on the
board and can support 1 x 1G ports for management and control.
Table 1 System Overview
AS7326-56X-0917-168ZZ
CPU
sub-system
CPU: Intel Broadwell-DE XeonD-1518 1.6G 4 Core
DDR SDRAM: 8GB x 2 2133MHz with ECC (DDR4 SO-DIMM)
SPI Flash (Boot): 16MB
USB to NAND Flash memory : 8GB SLC
mSATA SSD: 32GB MLC (Option)
m.2 SSD: 64GB MLC (2242 / 2280)
Management
UART RS232 console port (RJ45 or Micro-USB),
Out-band Management Ethernet port (RJ45)