Extreme Networks BlackDiamond 6804 User manual

Extreme Networks, Inc.
3585 Monroe Street
Santa Clara, California 95051
[phone]
http://www.extremenetworks.com
Advanced System
Diagnostics and
Troubleshooting Guide
Published: March 2005
Part number: 100189-00 Rev. 01

2
©2005 Extreme Networks, Inc. All rights reserved. Extreme Networks, ExtremeWare, Alpine, and BlackDiamond are
registered trademarks of Extreme Networks, Inc. in the United States and certain other jurisdictions. ExtremeWare Vista,
ExtremeWorks, ExtremeAssist, ExtremeAssist1, ExtremeAssist2, PartnerAssist, EPICenter, Extreme Standby Router
Protocol, ESRP, SmartTraps, Summit, Summit1i, Summit5i, Summit7i,Summit48i, Summit48si, SummitPx, Summit 200,
Summit Virtual Chassis, SummitLink, SummitGbX, SummitRPS and the Extreme Networks logo are trademarks of
Extreme Networks, Inc., which may be registered or pending registration in certain jurisdictions. The Extreme
Turbodrive logo is a service mark of Extreme Networks, which may be registered or pending registration in certain
jurisdictions. Specifications are subject to change without notice.
NetWare and Novell are registered trademarks of Novell, Inc. Merit is a registered trademark of Merit Network, Inc.
Solaris is a trademark of Sun Microsystems, Inc. F5, BIG/ip, and 3DNS are registered trademarks of F5 Networks, Inc.
see/IT is a trademark of F5 Networks, Inc.
“Data Fellows”, the triangle symbol, and Data Fellows product names and symbols/logos are
trademarks of Data Fellows.
F-Secure SSH is a registered trademark of Data Fellows.
All other registered trademarks, trademarks and service marks are property of their respective owners.
A
uthors:
E
ditor:
P
roduction:
S
pecial Thanks:

Advanced System Diagnostics and Troubleshooting Guide 3
Contents
Preface
Introduction 9
Terminology 9
Conventions 9
Related Publications 10
Chapter 1 Introduction
Introduction 11
Diagnostics: A Brief Historical Perspective 12
Overview of the ExtremeWare Diagnostics Suite 13
Supported Hardware 13
Applicable ExtremeWare Versions 13
Chapter 2 Hardware Architecture
Diagnostics Support 15
The BlackDiamond Systems 16
BlackDiamond 6800 Series Hardware Architecture Differences 16
The BlackDiamond Backplane 17
BlackDiamond I/O Modules 18
Management Switch Modules 19
BlackDiamond MSM Redundancy 20
Causes of MSM Failover and System Behavior 20
Alpine Systems 22
Summit Systems 23

4Advanced System Diagnostics and Troubleshooting Guide
Contents
Chapter 3 Packet Errors and Packet Error Detection
Overview 25
Definition of Terms 26
Standard Ethernet Detection for Packet Errors on the Wire 27
Extreme Networks’ Complementary Detection of Packet Errors Between Wires 27
Hardware System Detection Mechanisms 28
Software System Detection Mechanisms 29
Failure Modes 30
Transient Failures 30
Systematic Failures 30
Soft-State Failures 30
Permanent Failures 31
Responding to Failures 31
Error Messages for Fabric Checksums 31
Error Message Format 32
Fabric Checksum Error Message Logging 33
Checksum Message Examples 34
Panic/Action Error Messages 35
Panic/Action Message Example 36
Chapter 4 Software Exception Handling
Overview of Software Exception Handling Features 37
System Watchdog Behavior 37
System Software Exception Recovery Behavior 38
Redundant MSM Behavior 38
Configuring System Recovery Actions 40
Related Commands 40
Configuring System Recovery Actions 40
Usage Notes 41
Configuring Reboot Loop Protection 42
Related Commands 42
Configuring Reboot Loop Protection 42
Dumping the System Memory 44
Related Commands 44
Configuring an Automatic System Dump During System Recovery 44
Initiating a Manual System Dump 45
Chapter 5 Diagnostics
Diagnostic Test Functionality 47
How Diagnostic Tests are Run 47
How the Test Affects the Switch 48

Advanced System Diagnostics and Troubleshooting Guide 5
Contents
System Health Checks: A Diagnostics Suite 51
Diagnostic Suite Components 51
The Role of Memory Scanning and Memory Mapping 52
Modes of Operation 53
The Role of Processes to Monitor System Operation 54
Power On Self Test (POST) 55
Related Commands 55
Configuring the Boot-Up Diagnostics 55
Runtime (On-Demand) System Diagnostics 56
Normal System Diagnostics 56
Related Commands 56
Running the Normal Diagnostics on BlackDiamond Systems 57
Running the Normal Diagnostics on Alpine Systems 57
Running the Normal Diagnostics on Summit Systems 57
System Impact of Running the Normal Diagnostics 57
Extended System Diagnostics 58
Related Commands 58
Running the Extended Diagnostics on BlackDiamond Systems 58
Running the Extended Diagnostics on Alpine Systems 58
Running the Extended Diagnostics on Summit Systems 59
System Impact of Running the Extended Diagnostics 59
On Demand Packet Memory Scan 59
Related Commands 59
Running the Packet Memory Scan Diagnostics on BlackDiamond Systems 59
Running the Packet Memory Scan Diagnostics on Alpine Systems 59
Running the Packet Memory Scan Diagnostics on Summit Systems 60
Automatic Packet Memory Scan (via sys-health-check) 60
Memory Scanning and Memory Mapping Behavior 60
Limited Operation Mode 63
Effects of Running Memory Scanning on the Switch 63
Summit, Alpine, or BlackDiamond with a Single MSM 63
BlackDiamond System with Two MSMs 64
Interpreting Memory Scanning Results 65
Per-Slot Packet Memory Scan on BlackDiamond Switches 66
Related Commands 66
Configuring the Packet Memory Scan Recovery Mode 66
System Impact of Per-Slot Packet Memory Scanning 67
Network Impact of Per-Slot Packet Memory Scanning 67
System (CPU and Backplane) Health Check 69
Health Check Packet Types 69
Backplane Health Check States 69
Related Commands 69
Health Check Functionality 70
Alarm-Level Response Action 70
Auto-Recovery Response Action 70
Backplane Health Check 71

6Advanced System Diagnostics and Troubleshooting Guide
Contents
Viewing Backplane Health Check Results—show log Command 71
Viewing Backplane Health Check Diagnostic Results—show diagnostics Command 71
Analyzing the Results 76
CPU Health Check 77
Viewing CPU Health Check Results—show log Command 77
Viewing CPU Health Check Diagnostic Results—show diagnostics Command 78
Analyzing the CPU Health Check Results 78
Transceiver Diagnostics 80
Usage Guidelines 80
Related Commands 80
Configuring the Transceiver Diagnostics 80
System Impacts of the Transceiver Diagnostics 81
Network Impact of the Transceiver Diagnostics 81
Viewing Diagnostics Results 82
Example Log Messages for Transceiver Diagnostic Failures 82
Examples, show diagnostics Command 82
Example—show switch Command 84
Transceiver Diagnostic Result Analysis 84
FDB Scan 85
Usage Guidelines 85
Related Commands 86
Enabling FDB Scanning 86
Disabling FDB Scanning 86
Configuring the FDB Scan Diagnostics 86
System Impact of the FDB Scan Diagnostic 87
Network Impact of the FDB Scan Diagnostic 87
Viewing Diagnostics Results 88
Example Log Messages for FDB Scan Diagnostic Failures 88
Example FDB Scan Results from the show diagnostics Command 88
Example Output from the show switch command 89
Example Output from the show fdb remap Command 89
Chapter 6 Additional Diagnostics Tools
Temperature Logging 92
Related Commands 92
System Impacts of Temperature Logging 92
Network Impact of Temperature Logging 92
Syslog Servers 93
Related Commands 93
Enabling Logging to Remote Syslog Server Targets 93
Disabling Logging to Remote Syslog Server Targets 93
Adding a Syslog Server 94
Deleting a Remote Syslog Server 94
System Impact of the Syslog Server Facility 94
Network Impact of the Syslog Server Facility 94

Advanced System Diagnostics and Troubleshooting Guide 7
Contents
Chapter 7 Troubleshooting Guidelines
Contacting Extreme Technical Support 95
Americas TAC 95
Asia TAC 96
EMEA TAC 96
Japan TAC 96
What Information Should You Collect? 97
Analyzing Data 97
Diagnostic Troubleshooting 98
Extreme Networks’ Recommendations 99
Using Memory Scanning to Screen I/O Modules 101
Appendix A Limited Operation Mode and Minimal Operation Mode
Limited Operation Mode 103
Triggering Limited Operation Mode 104
Bringing a Switch Out of Limited Operation Mode 104
Minimal Operation Mode 104
Triggering Minimal Operation Mode 104
Bringing a Switch Out of Minimal Operation Mode 105
Appendix B Reference Documents
General Information 107
Other Documentation Resources 107
Index

8Advanced System Diagnostics and Troubleshooting Guide
Contents

Advanced System Diagnostics and Troubleshooting Guide 9
Preface
This Preface provides an overview of this guide, describes guide conventions, and lists other
publications that might be useful.
Introduction
This guide describes how to use the ExtremeWare hardware diagnostics suite to test and validate the
operating integrity of Extreme Networks switches. The tools in the diagnostic suite are used to detect,
isolate, and treat faults in a system.
This guide is intended for use by network designers, planners, and operations staff.
Terminology
When features, functionality, or operation is specific to a modular or stand-alone switch family, the
family name is used. Explanations about features and operations that are the same across all product
families simply refer to the product as the “switch.”
Conventions
Table 1 and Table 2 list conventions that are used throughout this guide.
Table 1: Notice Icons
Icon Notice Type Alerts you to...
Note Important features or instructions.
Caution Risk of personal injury, system damage, or loss of data.
Warning Risk of severe personal injury.

10 Advanced System Diagnostics and Troubleshooting Guide
Preface
Related Publications
The publications related to this one are:
•ExtremeWare Software User Guide, Software Version 6.2.2.
•ExtremeWare Software Command Reference, Software Version 6.2.2.
•ExtremeWare Error Message Decoder.
Documentation for Extreme Networks products is available on the World Wide Web at the following
location:
http://www.extremenetworks.com/
Table 2: Text Conventions
Convention Description
Screen displays This typeface indicates command syntax, or represents information as it appears on the
screen.
The words “enter”
and “type” When you see the word “enter” in this guide, you must type something, and then press the Return or Enter key. Do not press the Return or Enter key when an instruction
simply says “type.”
[Key] names Key names are written with brackets, such as [Return] or [Esc].
If you must press two or more keys simultaneously, the key names are linked with a
plus sign (+). Example:
Press [Ctrl]+[Alt]+[Del].
Letter in bold type Letters within a command that appear in bold type indicate the keyboard shortcut for a
command. When entering the command, you can use just the bolded letters instead of
the entire word.
Words in italicized type Italics emphasize a point or denote new terms at the place where they are defined in
the text.