Extreme Networks BlackDiamond 6804 User manual

Extreme Networks, Inc.
3585 Monroe Street
Santa Clara, California 95051
[phone]
http://www.extremenetworks.com
Advanced System
Diagnostics and
Troubleshooting Guide
Published: March 2005
Part number: 100189-00 Rev. 01

2
©2005 Extreme Networks, Inc. All rights reserved. Extreme Networks, ExtremeWare, Alpine, and BlackDiamond are
registered trademarks of Extreme Networks, Inc. in the United States and certain other jurisdictions. ExtremeWare Vista,
ExtremeWorks, ExtremeAssist, ExtremeAssist1, ExtremeAssist2, PartnerAssist, EPICenter, Extreme Standby Router
Protocol, ESRP, SmartTraps, Summit, Summit1i, Summit5i, Summit7i,Summit48i, Summit48si, SummitPx, Summit 200,
Summit Virtual Chassis, SummitLink, SummitGbX, SummitRPS and the Extreme Networks logo are trademarks of
Extreme Networks, Inc., which may be registered or pending registration in certain jurisdictions. The Extreme
Turbodrive logo is a service mark of Extreme Networks, which may be registered or pending registration in certain
jurisdictions. Specifications are subject to change without notice.
NetWare and Novell are registered trademarks of Novell, Inc. Merit is a registered trademark of Merit Network, Inc.
Solaris is a trademark of Sun Microsystems, Inc. F5, BIG/ip, and 3DNS are registered trademarks of F5 Networks, Inc.
see/IT is a trademark of F5 Networks, Inc.
“Data Fellows”, the triangle symbol, and Data Fellows product names and symbols/logos are
trademarks of Data Fellows.
F-Secure SSH is a registered trademark of Data Fellows.
All other registered trademarks, trademarks and service marks are property of their respective owners.
A
uthors:
E
ditor:
P
roduction:
S
pecial Thanks:

Advanced System Diagnostics and Troubleshooting Guide 3
Contents
Preface
Introduction 9
Terminology 9
Conventions 9
Related Publications 10
Chapter 1 Introduction
Introduction 11
Diagnostics: A Brief Historical Perspective 12
Overview of the ExtremeWare Diagnostics Suite 13
Supported Hardware 13
Applicable ExtremeWare Versions 13
Chapter 2 Hardware Architecture
Diagnostics Support 15
The BlackDiamond Systems 16
BlackDiamond 6800 Series Hardware Architecture Differences 16
The BlackDiamond Backplane 17
BlackDiamond I/O Modules 18
Management Switch Modules 19
BlackDiamond MSM Redundancy 20
Causes of MSM Failover and System Behavior 20
Alpine Systems 22
Summit Systems 23

4Advanced System Diagnostics and Troubleshooting Guide
Contents
Chapter 3 Packet Errors and Packet Error Detection
Overview 25
Definition of Terms 26
Standard Ethernet Detection for Packet Errors on the Wire 27
Extreme Networks’ Complementary Detection of Packet Errors Between Wires 27
Hardware System Detection Mechanisms 28
Software System Detection Mechanisms 29
Failure Modes 30
Transient Failures 30
Systematic Failures 30
Soft-State Failures 30
Permanent Failures 31
Responding to Failures 31
Error Messages for Fabric Checksums 31
Error Message Format 32
Fabric Checksum Error Message Logging 33
Checksum Message Examples 34
Panic/Action Error Messages 35
Panic/Action Message Example 36
Chapter 4 Software Exception Handling
Overview of Software Exception Handling Features 37
System Watchdog Behavior 37
System Software Exception Recovery Behavior 38
Redundant MSM Behavior 38
Configuring System Recovery Actions 40
Related Commands 40
Configuring System Recovery Actions 40
Usage Notes 41
Configuring Reboot Loop Protection 42
Related Commands 42
Configuring Reboot Loop Protection 42
Dumping the System Memory 44
Related Commands 44
Configuring an Automatic System Dump During System Recovery 44
Initiating a Manual System Dump 45
Chapter 5 Diagnostics
Diagnostic Test Functionality 47
How Diagnostic Tests are Run 47
How the Test Affects the Switch 48

Advanced System Diagnostics and Troubleshooting Guide 5
Contents
System Health Checks: A Diagnostics Suite 51
Diagnostic Suite Components 51
The Role of Memory Scanning and Memory Mapping 52
Modes of Operation 53
The Role of Processes to Monitor System Operation 54
Power On Self Test (POST) 55
Related Commands 55
Configuring the Boot-Up Diagnostics 55
Runtime (On-Demand) System Diagnostics 56
Normal System Diagnostics 56
Related Commands 56
Running the Normal Diagnostics on BlackDiamond Systems 57
Running the Normal Diagnostics on Alpine Systems 57
Running the Normal Diagnostics on Summit Systems 57
System Impact of Running the Normal Diagnostics 57
Extended System Diagnostics 58
Related Commands 58
Running the Extended Diagnostics on BlackDiamond Systems 58
Running the Extended Diagnostics on Alpine Systems 58
Running the Extended Diagnostics on Summit Systems 59
System Impact of Running the Extended Diagnostics 59
On Demand Packet Memory Scan 59
Related Commands 59
Running the Packet Memory Scan Diagnostics on BlackDiamond Systems 59
Running the Packet Memory Scan Diagnostics on Alpine Systems 59
Running the Packet Memory Scan Diagnostics on Summit Systems 60
Automatic Packet Memory Scan (via sys-health-check) 60
Memory Scanning and Memory Mapping Behavior 60
Limited Operation Mode 63
Effects of Running Memory Scanning on the Switch 63
Summit, Alpine, or BlackDiamond with a Single MSM 63
BlackDiamond System with Two MSMs 64
Interpreting Memory Scanning Results 65
Per-Slot Packet Memory Scan on BlackDiamond Switches 66
Related Commands 66
Configuring the Packet Memory Scan Recovery Mode 66
System Impact of Per-Slot Packet Memory Scanning 67
Network Impact of Per-Slot Packet Memory Scanning 67
System (CPU and Backplane) Health Check 69
Health Check Packet Types 69
Backplane Health Check States 69
Related Commands 69
Health Check Functionality 70
Alarm-Level Response Action 70
Auto-Recovery Response Action 70
Backplane Health Check 71

6Advanced System Diagnostics and Troubleshooting Guide
Contents
Viewing Backplane Health Check Results—show log Command 71
Viewing Backplane Health Check Diagnostic Results—show diagnostics Command 71
Analyzing the Results 76
CPU Health Check 77
Viewing CPU Health Check Results—show log Command 77
Viewing CPU Health Check Diagnostic Results—show diagnostics Command 78
Analyzing the CPU Health Check Results 78
Transceiver Diagnostics 80
Usage Guidelines 80
Related Commands 80
Configuring the Transceiver Diagnostics 80
System Impacts of the Transceiver Diagnostics 81
Network Impact of the Transceiver Diagnostics 81
Viewing Diagnostics Results 82
Example Log Messages for Transceiver Diagnostic Failures 82
Examples, show diagnostics Command 82
Example—show switch Command 84
Transceiver Diagnostic Result Analysis 84
FDB Scan 85
Usage Guidelines 85
Related Commands 86
Enabling FDB Scanning 86
Disabling FDB Scanning 86
Configuring the FDB Scan Diagnostics 86
System Impact of the FDB Scan Diagnostic 87
Network Impact of the FDB Scan Diagnostic 87
Viewing Diagnostics Results 88
Example Log Messages for FDB Scan Diagnostic Failures 88
Example FDB Scan Results from the show diagnostics Command 88
Example Output from the show switch command 89
Example Output from the show fdb remap Command 89
Chapter 6 Additional Diagnostics Tools
Temperature Logging 92
Related Commands 92
System Impacts of Temperature Logging 92
Network Impact of Temperature Logging 92
Syslog Servers 93
Related Commands 93
Enabling Logging to Remote Syslog Server Targets 93
Disabling Logging to Remote Syslog Server Targets 93
Adding a Syslog Server 94
Deleting a Remote Syslog Server 94
System Impact of the Syslog Server Facility 94
Network Impact of the Syslog Server Facility 94

Advanced System Diagnostics and Troubleshooting Guide 7
Contents
Chapter 7 Troubleshooting Guidelines
Contacting Extreme Technical Support 95
Americas TAC 95
Asia TAC 96
EMEA TAC 96
Japan TAC 96
What Information Should You Collect? 97
Analyzing Data 97
Diagnostic Troubleshooting 98
Extreme Networks’ Recommendations 99
Using Memory Scanning to Screen I/O Modules 101
Appendix A Limited Operation Mode and Minimal Operation Mode
Limited Operation Mode 103
Triggering Limited Operation Mode 104
Bringing a Switch Out of Limited Operation Mode 104
Minimal Operation Mode 104
Triggering Minimal Operation Mode 104
Bringing a Switch Out of Minimal Operation Mode 105
Appendix B Reference Documents
General Information 107
Other Documentation Resources 107
Index

8Advanced System Diagnostics and Troubleshooting Guide
Contents

Advanced System Diagnostics and Troubleshooting Guide 9
Preface
This Preface provides an overview of this guide, describes guide conventions, and lists other
publications that might be useful.
Introduction
This guide describes how to use the ExtremeWare hardware diagnostics suite to test and validate the
operating integrity of Extreme Networks switches. The tools in the diagnostic suite are used to detect,
isolate, and treat faults in a system.
This guide is intended for use by network designers, planners, and operations staff.
Terminology
When features, functionality, or operation is specific to a modular or stand-alone switch family, the
family name is used. Explanations about features and operations that are the same across all product
families simply refer to the product as the “switch.”
Conventions
Table 1 and Table 2 list conventions that are used throughout this guide.
Table 1: Notice Icons
Icon Notice Type Alerts you to...
Note Important features or instructions.
Caution Risk of personal injury, system damage, or loss of data.
Warning Risk of severe personal injury.

10 Advanced System Diagnostics and Troubleshooting Guide
Preface
Related Publications
The publications related to this one are:
•ExtremeWare Software User Guide, Software Version 6.2.2.
•ExtremeWare Software Command Reference, Software Version 6.2.2.
•ExtremeWare Error Message Decoder.
Documentation for Extreme Networks products is available on the World Wide Web at the following
location:
http://www.extremenetworks.com/
Table 2: Text Conventions
Convention Description
Screen displays This typeface indicates command syntax, or represents information as it appears on the
screen.
The words “enter”
and “type” When you see the word “enter” in this guide, you must type something, and then press the Return or Enter key. Do not press the Return or Enter key when an instruction
simply says “type.”
[Key] names Key names are written with brackets, such as [Return] or [Esc].
If you must press two or more keys simultaneously, the key names are linked with a
plus sign (+). Example:
Press [Ctrl]+[Alt]+[Del].
Letter in bold type Letters within a command that appear in bold type indicate the keyboard shortcut for a
command. When entering the command, you can use just the bolded letters instead of
the entire word.
Words in italicized type Italics emphasize a point or denote new terms at the place where they are defined in
the text.

Advanced System Diagnostics and Troubleshooting Guide 11
1Introduction
This guide describes how to use the ExtremeWare hardware diagnostics suite to test and validate the
operating integrity of Extreme Networks switches. The tools in the diagnostic suite are used to detect,
isolate, and treat faults in a system.
This chapter contains the following sections:
•Diagnostics: A Brief Historical Perspective on page 12
•Overview of the ExtremeWare Diagnostics Suite on page 13
•Supported Hardware on page 13
•Applicable ExtremeWare Versions on page 13
Introduction
The purpose of this guide is to provide information and guidelines to assist you in implementing the
diagnostic suite within ExtremeWare. The Extreme Networks diagnostic software is intended to identify
possible faulty hardware or software error conditions and—depending on how the various diagnostics
features are configured—take the appropriate preconfigured action. The action might be to enable the
switch to write informative error messages to the switch log, attempt to recover itself and continue
operating, or simply remove the suspect system component from service.
It is important to note that while each diagnostic test—by itself—is not complicated, system
configuration, as a whole, must be evaluated to ensure ongoing, expected behavior both with the switch
and across the network itself. For example, in implementing the diagnostic suite, you must take into
consideration these operational variables:
•Redundancy mechanisms implemented
•Levels of redundancy within the network
•Acceptable outage windows (scheduled and unscheduled)

12 Advanced System Diagnostics and Troubleshooting Guide
Introduction
Diagnostics: A Brief Historical Perspective
Diagnostic utility programs were created to aid in troubleshooting system problems by detecting and
reporting faults so that operators or administrators could go fix the problem. While this approach does
help, it has some key limitations:
•It is, at its base, reactive, meaning a failure must occur before the diagnostic test can be used to look
for a cause for the failure.
•It can be time consuming, because the ability to troubleshoot a failure successfully based on the
information provided by the diagnostics test depends greatly on the types of information reported
by the test and the level of detail in the information.
Because users of mission-critical networks and network applications are becoming increasingly
dependent on around-the-clock network access and highest performance levels, any downtime or
service degradation is disruptive and costly. So time lost to an unexpected failure, compounded by more
time lost while someone attempts to track down and fix the failure, has become increasingly less
acceptable.
The process of improving diagnostic tests to minimize failures and their impact is a kind of feedback
system: What you learn through the use of the diagnostics improves your understanding of hardware
failure modes; what you learn from an improved understanding of hardware failure modes improves
your understanding of the diagnostics.
The goal of the current generation of ExtremeWare diagnostics is to help users achieve the highest levels
of network availability and performance by providing a suite of diagnostic tests that moves away from
a reactive stance—wherein a problem occurs and then you attempt to determine what caused the
problem—to a proactive state—wherein the system hardware, software, and diagnostics work together
to reduce the total number of failures and downtime through:
•More accurate reporting of errors (fewer false notifications; more information about actual errors)
•Early detection of conditions that lead to a failure (so that corrective action can be taken before the
failure occurs)
•Automatic detection and correction of packet memory errors in the system’s control and data planes
Administrators will now find a greatly reduced MTTR (mean time to repair) due to fast and accurate
fault identification. Multiple modules will no longer need to be removed and tested; faulty components
will usually be identified directly. Over time, there should be a significant reduction in the number of
problems found.
NOTE
In spite of the improved ExtremeWare hardware diagnostics, some network events might still occur,
because software is incapable of detecting and preventing every kind of failure.

Overview of the ExtremeWare Diagnostics Suite
Advanced System Diagnostics and Troubleshooting Guide 13
Overview of the ExtremeWare Diagnostics Suite
The ExtremeWare diagnostic suite includes the following types of tools for use in detecting, isolating,
and treating faults in a switch. Each of these diagnostic types is summarized below, but is described in
greater detail in later sections of this guide.
•Power-on self test (POST)—A sequence of hardware tests that run automatically each time the switch
is booted, to validate basic system integrity. The POST runs in either of two modes: normal (more
thorough, but longer-running test sequence) or FastPOST (faster-running basic test sequence).
•On-demand system hardware diagnostics—Run on demand through user CLI commands; runs in
either of two modes: normal (faster-running basic test sequence) or extended (more thorough, but
longer-running test sequence).
The extended diagnostics include the packet memory scan, which checks the packet memory area of
the switch fabric for defects and maps out defective blocks. This test can be run by itself, as part of
the slot-based extended diagnostics, or can be invoked from within the system health checks.
•Switch-wide communication-path packet error health checks—An integrated diagnostic
subsystem—the system health check feature—that consists of a number of different test types,
operating proactively in the background to detect and respond to packet error problems in module
memory or on communication paths. The system health checks include the following kinds of tests:
—Backplane health checks
—CPU health checks
—Switch fabric checksum validation
—Dynamic memory scanning and memory mapping
—Transceiver diagnostics
—Forwarding database (FDB) scan
Supported Hardware
The ExtremeWare diagnostic suite applies only to Extreme Networks switch products based on the
“inferno” series chipset. Equipment based on this chipset are referred to as being “inferno” series or “i”
series products: the BlackDiamond family of core chassis switches (6804, 6808, and 6816), the Alpine
systems (3802, 3804, 3808), and the Summit “i”-series stackables (Summit1i, Summit5i, Summit7i,
Summit 48i, and Summit48Si).
Applicable ExtremeWare Versions
The information in this guide is based on the features and feature attributes found in
ExtremeWare Version 6.2.2b134 or later.

14 Advanced System Diagnostics and Troubleshooting Guide
Introduction

Advanced System Diagnostics and Troubleshooting Guide 15
2Hardware Architecture
This chapter provides a brief summary of the switch hardware features most relevant to understanding
the use of the Extreme Networks diagnostic suite.
This chapter contains the following sections:
•Diagnostics Support on page 15
•The BlackDiamond Systems on page 16
•Alpine Systems on page 22
•Summit Systems on page 23
Diagnostics Support
The ExtremeWare diagnostic suite applies only to Extreme Networks switch products based on the
“inferno” series chipset. Equipment based on this chipset are referred to as being “inferno” series or “i”
series products: the BlackDiamond family of core chassis switches (6804, 6808, and 6816), the Alpine
systems (3802, 3804, 3808), and the Summit “i”-series stackables (Summit1i, Summit5i, Summit7i,
Summit 48i, and Summit48Si).
NOTE
These switches and the switch modules use naming conventions ending with an “i” to identify them as
“inferno” series or “i” series products. For the most current list of products supporting the “i” chipset,
such as the MSM-3 and other “3”-series modules, such as the G16X3, consult your release notes.
Unless otherwise specified, a feature requiring the “i” chipset requires the use of both an “i”
chipset-based management module, such as the MSM64i, and an “i” chipset-based I/O module, such as
the G8Xi.

16 Advanced System Diagnostics and Troubleshooting Guide
Hardware Architecture
The BlackDiamond Systems
In the context of the system diagnostics, the BlackDiamond family of core chassis switches share the
same fundamental hardware architecture: a multislot modular chassis containing a passive backplane
that supports redundant load-sharing, hot-swappable switch fabric modules. On BlackDiamond
systems, each I/O module and MSM represents an individual switch containing its own switching
fabric and packet memory.
Figure 1: BlackDiamond 6800 Series architecture, general block diagram
BlackDiamond 6800 Series Hardware Architecture Differences
In the context of understanding the ExtremeWare diagnostics and their use in troubleshooting system
problems, these are the key hardware distinctions between the BlackDiamond 6816, 6808, and 6804.
•BlackDiamond 6816—Modular chassis with passive backplane; sixteen chassis slots for I/O modules;
four chassis slots for MSMs.
•BlackDiamond 6808—Modular chassis with passive backplane; eight chassis slots for I/O modules;
two chassis slots for MSMs.
•BlackDiamond 6804—Modular chassis with passive backplane; four chassis slots for I/O modules;
two chassis slots for MSMs.
Management Bus
PQM
OTP
VPST
FDB
AFQM
ASIC
(Quake)
PM
CPU
Subassembly
MAC
PHY
PHY
MAC
MAC
MAC
PHY
PHY
MAC
I/O Module MSM
PHY
PHY
MAC
PHY
PHY
PHY
PHY
PHY
PHY
NVRAM
E-NET
Console
PCMCIA
SE
ASIC
(Twister)
Data Bus Data Bus
PCI Bus
PQM
OTP
VPST
FDB
AFQM
ASIC
(Quake)
PM
SE
ASIC
(Twister)
Data Bus Data Bus
Backplane
DN_024B

The BlackDiamond Systems
Advanced System Diagnostics and Troubleshooting Guide 17
The BlackDiamond Backplane
The BlackDiamond backplane is a passive backplane, meaning that all the active components such as
CPUs, ASICs, and memory have been moved onto plug-in modules, such as the I/O modules and
MSMs.
Figure 2: BlackDiamond passive backplane architecture (BlackDiamond 6808 shown)
The BlackDiamond backplane provides inter-slot electrical connections for both network data traffic and
a separate control bus for switch fabric management. Data traffic is carried on four AUI links between
each MSM and each I/O slot on BlackDiamond 6804 and BlackDiamond 6808 systems, and on two AUI
links between each MSM and each I/O slot on BlackDiamond 6816 systems. Device management occurs
on a 32-bit PCI bus connecting MSMs and I/O modules. The number of backplane slots for I/O
modules and MSMs determines the BlackDiamond system type (6804, 6808, 6816).
The chief advantages of a passive backplane are:
•The absence of active components yields a much lower possibility of backplane failure.
•You can remove and replace system modules faster, making upgrades and repairs easier, faster, and
cheaper.
NOTE
One disadvantage of a passive backplane is that a problem on one switch module might cause other
switch modules to fail. More information on this possibility is covered in later chapters of this guide.
Switch
Module
64 Gbps
Switch
Fabric
Eight Load-Shared
Gigabit Links
Fault-Tolerant
Switch Fabric
and System
Management
A
1
2
3
8
B
64 Gbps
Switch
Fabric
Switch
Module
Switch
Module
Switch
Module
DN_001
A

18 Advanced System Diagnostics and Troubleshooting Guide
Hardware Architecture
BlackDiamond I/O Modules
Each BlackDiamond I/O module has a built-in switching fabric (see Figure 3) giving the module the
capability to switch local traffic on the same module. Traffic that is destined for other modules in the
chassis travels across the backplane to the MSMs, where it is switched and sent to its destination I/O
module.
Figure 3: BlackDiamond I/O module architecture (G8Xi 32 Gb fabric shown)
Each BlackDiamond I/O module has eight load-shared Gigabit Ethernet links to both MSMs through
the backplane. The load sharing algorithm distributes traffic across different channels through the
backplane’s Gigabit Ethernet links, providing bi-directional communication.
Each BlackDiamond I/O module is equipped with the following kinds of hardware components:
•PHY: An industry-standard ASIC responsible for physical layer (layer 1) signal, clocking, etc.
•MAC: The MAC handles the standard MAC layer functions as well as some other functions to
prepare a packet for transmission to the switch fabric or to the external network, including 802.1p
and DiffServ examination, VLAN insertion, MAC substitution, TTL decrement, 802.1p and DiffServ
replacement, etc.
Each I/O module has both external MACs and internal MACs. External MACs handle the interface to
the external ports; internal MACs handle the interface to the BlackDiamond backplane. Each MSM
provides four Gigabit Ethernet links to each I/O module.
•PBUS: The packet data bus that transfers packets between the MAC and the packet memory.
•Switch engine (distributed packet processor) ASIC (Twister) and its associated memories: packet
RAM and FDB RAM. The SE ASIC implements a high-speed, parallel data transfer bus for
transferring packets from MACs to packet memory and back.
•Address filtering and queue management ASIC (Quake) and its associated memories: OTP RAM, PQ
RAM, and VPST RAM.
When a data packet is received by the PHY, the PHY passes the packet to the MAC. The MAC handles
the layer 2 tasks, such as tagging and the MAC address, then transfers the packet across the PBUS to the
AFQM
ASIC
OTP RAM
4x GbE to
MSM-A
4x GbE to
MSM-B
PHY
PHY
MAC
MAC
MAC
MAC
PQ RAM
VPST RAM
SRAM (Packet Mem + FDB)
DN_026B
SE
ASIC
SE
ASIC
SE
ASIC
SE
ASIC
PBUS
G8Xi (32 Gb Fabric)

The BlackDiamond Systems
Advanced System Diagnostics and Troubleshooting Guide 19
packet memory for temporary storage. Based on the information in memory, such as the FDB, the
address filtering and queue management ASIC makes a forwarding decision. If the next hop is a local
port (on the same module), the packet is forwarded to the external MAC and PHY for the exit port. If
the packet is destined for another module (as either slow path traffic or fast path traffic), the packet is
transferred to the internal MAC and then on to the MSM (CPU).
All I/O modules share the management bus on the backplane, and use it to communicate to each other
and to the MSMs.
Management Switch Modules
As its name indicates, the Management Switch Fabric Module (MSM) serves a dual role in the system: it
is equipped to act as the internal switch fabric for data that is being transferred between I/O modules
in the chassis, and to handle the upper-layer processing and system management functions for the
switch. (See Figure 4.)
Figure 4: BlackDiamond MSM architecture (MSM64ishown)
An MSM is equipped with the following hardware components: CPU subsystem (dual CPUs, DRAM,
NVRAM, flash memory, and PCMCIA slot) and switch fabric subsystem (Quake ASICs, OTP, PQ, and
VPST RAM, packet memory and FDB SRAM, Twister ASICs, PBUS, and MAC ASICs).
AFQM
ASIC
256 MB
DRAM
OTP RAM
MAC
MAC
MAC
MAC
MAC
MAC
MAC
CPU
CPU
PCMCI
Slot
MAC
PQ RAM
VPST RAM
Flash
NVRAM AFQM
ASIC
OTP RAM
PQ RAM
VPST RAM
SRAM (Packet Mem + FDB)
DN_027B
SE
ASIC
SE
ASIC
SE
ASIC
SE
ASIC
SE
ASIC
SE
ASIC
SE
ASIC
SE
ASIC
MSM64i (64 Gb Fabric)
PBUS
CPU
Subsystem

20 Advanced System Diagnostics and Troubleshooting Guide
Hardware Architecture
BlackDiamond MSM Redundancy
The CPU subsystems on a pair of BlackDiamond MSMs operate in a master-slave relationship.
(See Figure 5.)
Figure 5: BlackDiamond MSM redundancy scheme
The master MSM CPU subsystem actively manages the switch and the task of switching packets in the
CPU control (or management) path. The slave MSM CPU subsystem is in standby mode, but is checked
periodically by the master MSM CPU (via EDP) to determine whether it is still available.
The master MSM also guarantees that management operations occur in a synchronized manner. For
example, if you make a configuration change and need to save it, the master MSM ensures that the
configuration is saved to both the master MSM and the slave MSM at the same time. That way, the
updated slave is ready to take over as the master if the master MSM fails.
Despite the master-slave switch management role, the switch fabrics on both MSMs actively switch core
traffic in a load-shared fashion. Load-sharing switches core traffic from different I/O modules.
All MSMs share the control (or management) bus on the backplane, and use it to communicate to each
other and to installed I/O modules to perform system health checks and status polling.
Causes of MSM Failover and System Behavior
A number of events can cause an MSM failover to occur, including:
•Software exception; system watchdog timer expiry
•Diagnostic failure (extended diagnostics, transceiver check/scan, FDB scan failure/remap)
•Hot removal of the master MSM or hard-reset of the master MSM
MSM64i (A)
MSM64i (B)
Switching
Fabric
Fault Tolerant
Switch Fabric
and
System
Management
I/O Module
I/O Module
I/O Module
I/O Module
I/O Module
Backplane
I/O Module
I/O Module
I/O Module
CPU
Sub-
system
Switching
Fabric
CPU Control Path
Data Path
CPU
Sub-
system
DN_028A