Extreme Networks ExtremeWare Version 7.8 User manual

Extreme Networks, Inc.
3585 Monroe Street
Santa Clara, California 95051
[phone]
http://www.extremenetworks.com
Advanced System
Diagnostics and
Troubleshooting Guide
ExtremeWare Software Version 7.8
Published: May 2008
Part number: 100279-00 Rev 01

2
©2005 Extreme Networks, Inc. All rights reserved. Extreme Networks, ExtremeWare, Alpine, and BlackDiamond are
registered trademarks of Extreme Networks, Inc. in the United States and certain other jurisdictions. ExtremeWare Vista,
ExtremeWorks, ExtremeAssist, ExtremeAssist1, ExtremeAssist2, PartnerAssist, EPICenter, Extreme Standby Router
Protocol, ESRP, SmartTraps, Summit, Summit1i, Summit5i, Summit7i,Summit48i, Summit48si, SummitPx, Summit 200,
Summit Virtual Chassis, SummitLink, SummitGbX, SummitRPS and the Extreme Networks logo are trademarks of
Extreme Networks, Inc., which may be registered or pending registration in certain jurisdictions. The Extreme
Turbodrive logo is a service mark of Extreme Networks, which may be registered or pending registration in certain
jurisdictions. Specifications are subject to change without notice.
NetWare and Novell are registered trademarks of Novell, Inc. Solaris is a trademark of Sun Microsystems, Inc. F5,
BIG/ip, and 3DNS are registered trademarks of F5 Networks, Inc. see/IT is a trademark of F5 Networks, Inc.
“Data Fellows”, the triangle symbol, and Data Fellows product names and symbols/logos are
trademarks of Data Fellows.
F-Secure SSH is a registered trademark of Data Fellows.
All other registered trademarks, trademarks and service marks are property of their respective owners.

Advanced System Diagnostics and Troubleshooting Guide 3
Contents
Preface
Introduction 9
Terminology 9
Conventions 9
Related Publications 10
Chapter 1 Introduction
Introduction 11
Diagnostics: A Brief Historical Perspective 12
Overview of the ExtremeWare Diagnostics Suite 12
Supported Hardware 13
Applicable ExtremeWare Versions 13
Chapter 2“ i ” Series Switch Hardware Architecture
Diagnostics Support 15
The BlackDiamond Systems 16
BlackDiamond 6800 Series Hardware Architecture Differences 16
The BlackDiamond Backplane 17
BlackDiamond I/O Modules 18
Management Switch Modules 19
BlackDiamond MSM Redundancy 20
Causes of MSM Failover and System Behavior 20
Alpine Systems 22
Summit “i” Series Systems 23
Chapter 3 Packet Errors and Packet Error Detection
Overview 25

4Advanced System Diagnostics and Troubleshooting Guide
Contents
Definition of Terms 26
Standard Ethernet Detection for Packet Errors on the Wire 27
Extreme Networks’ Complementary Detection of Packet Errors Between Wires 27
Hardware System Detection Mechanisms 28
Software System Detection Mechanisms 29
Failure Modes 30
Transient Failures 30
Systematic Failures 30
Soft-State Failures 30
Permanent Failures 31
Responding to Reported Failures 31
Health Check Messages 33
Alert Messages 33
Checksum Error Messages: 34
Corrective Behavior Messages 35
Chapter 4 Software Exception Handling
Overview of Software Exception Handling Features 37
System Watchdog Behavior 37
System Software Exception Recovery Behavior 38
Redundant MSM Behavior 38
Configuring System Recovery Actions 40
Related Commands 40
Configuring System Recovery Actions on “i” Series Switches 40
Configuring System Recovery Actions on “e” Series Switches 41
Usage Notes 41
Configuring Reboot Loop Protection 43
Related Commands 43
Configuring Reboot Loop Protection 43
Dumping the “i” Series Switch System Memory 45
Related Commands 45
Configuring an Automatic System Dump During System Recovery 45
Initiating a Manual System Dump 46
Example Log for a Software Exception 47
Chapter 5 Diagnostics
Diagnostic Test Functionality 49
How Diagnostic Tests are Run 49
How the Test Affects the Switch 50
System Health Checks: A Diagnostics Suite 52
Diagnostic Suite Components 52
The Role of Memory Scanning and Memory Mapping 53
Modes of Operation 54

Advanced System Diagnostics and Troubleshooting Guide 5
Contents
The Role of Processes to Monitor System Operation 55
Power On Self Test (POST) 56
Related Commands 56
Configuring the Boot-Up Diagnostics 56
Runtime (On-Demand) System Diagnostics 57
Runtime Diagnostics on “i” Series Systems 57
Related Commands 58
Running the Diagnostics on BlackDiamond Systems 58
Runtime Diagnostics on “i” Series Alpine and Summit Systems 58
System Impact of Running the Diagnostics on “i” Series Switches 59
Runtime Diagnostics on “e” Series Systems 59
Related Commands 60
Running the Diagnostics on Summit “e” Switches 60
System Impact of Running the Diagnostics on Summit “e” Series Switches 60
Automatic Packet Memory Scan (via sys-health-check) 60
Memory Scanning and Memory Mapping Behavior 61
Limited Operation Mode 64
Effects of Running Memory Scanning on “i” Series Switches 64
Summit, Alpine, or BlackDiamond with a Single MSM 64
BlackDiamond System with Two MSMs 64
Interpreting Memory Scanning Results 66
Per-Slot Packet Memory Scan on BlackDiamond Switches 67
Related Commands 67
Configuring the Packet Memory Scan Recovery Mode 67
System Impact of Per-Slot Packet Memory Scanning 68
Network Impact of Per-Slot Packet Memory Scanning 68
System (CPU and Backplane) Health Check 70
Health Check Packet Types 70
Backplane Health Check States 70
Related Commands 71
Health Check Functionality 71
Alarm-Level Response Action 71
Auto-Recovery Response Action 71
Backplane Health Check 72
Viewing Backplane Health Check Results—show log Command 72
Viewing Backplane Health Check Diagnostic Results—show diagnostics Command 73
Analyzing the Results 78
CPU Health Check 79
Viewing CPU Health Check Results—show log Command 79
Viewing CPU Health Check Diagnostic Results—show diagnostics Command 80
Analyzing the CPU Health Check Results 80
Transceiver Diagnostics 82
Usage Guidelines 82
Related Commands 82
Configuring the Transceiver Diagnostics 82

6Advanced System Diagnostics and Troubleshooting Guide
Contents
System Impacts of the Transceiver Diagnostics 83
Network Impact of the Transceiver Diagnostics 83
Viewing Diagnostics Results 84
Example Log Messages for Transceiver Diagnostic Failures 84
Examples, show diagnostics Command 85
Example—show switch Command 86
Transceiver Diagnostic Result Analysis 87
FDB Scan 88
Usage Guidelines 88
Related Commands 89
Enabling FDB Scanning 89
Disabling FDB Scanning 89
Configuring the FDB Scan Diagnostics 90
System Impact of the FDB Scan Diagnostic 90
Network Impact of the FDB Scan Diagnostic 90
Viewing Diagnostics Results 91
Example Log Messages for FDB Scan Diagnostic Failures 91
Example FDB Scan Results from the show diagnostics Command 91
Example Output from the show switch command 92
Example Output from the show fdb remap Command 92
Chapter 6 Additional Diagnostics Tools
Temperature Logging for Modular Switches 93
Related Commands 93
System Impacts of Temperature Logging 94
Network Impact of Temperature Logging 94
Syslog Servers 94
Related Commands 94
Enabling Logging to Remote Syslog Server Targets 94
Disabling Logging to Remote Syslog Server Targets 95
Adding a Syslog Server 95
Deleting a Remote Syslog Server 95
System Impact of the Syslog Server Facility 95
Network Impact of the Syslog Server Facility 96
Cable Diagnostics 96
Cable Diagnostics for the High-Density Gigabit Ethernet I/O Modules in “i” Series
Switches 96
Related Commands 96
Running Cable Diagnostics 97
Viewing and Interpreting CDM Test Data 98
Cable Diagnostics for “e” Series Switches 101
Chapter 7 Troubleshooting Guidelines
Contacting Extreme Technical Support 103
Americas TAC 103

Advanced System Diagnostics and Troubleshooting Guide 7
Contents
Asia TAC 104
EMEA TAC 104
Japan TAC 104
What Information Should You Collect? 105
Analyzing Data 105
Diagnostic Troubleshooting 106
Extreme Networks’ Recommendations 107
Using Memory Scanning to Screen I/O Modules 109
Appendix A Limited Operation Mode and Minimal Operation Mode
Limited Operation Mode 111
Triggering Limited Operation Mode 112
Bringing a Switch Out of Limited Operation Mode 112
Minimal Operation Mode 112
Triggering Minimal Operation Mode 112
Bringing a Switch Out of Minimal Operation Mode 113
Appendix B Reference Documents
General Information 115
Other Documentation Resources 115
Index
Index of Commands

8Advanced System Diagnostics and Troubleshooting Guide
Contents

Advanced System Diagnostics and Troubleshooting Guide 9
Preface
This Preface provides an overview of this guide, describes guide conventions, and lists other
publications that might be useful.
Introduction
This guide describes how to use the ExtremeWare hardware diagnostics suite to test and validate the
operating integrity of Extreme Networks switches. The tools in the diagnostic suite are used to detect,
isolate, and treat faults in a system.
This guide is intended for use by network designers, planners, and operations staff.
Terminology
When features, functionality, or operation is specific to a modular or stand-alone switch family, the
family name is used. Explanations about features and operations that are the same across all product
families simply refer to the product as the “switch.”
Conventions
Table 1 and Table 2 list conventions that are used throughout this guide.
Table 1: Notice Icons
Icon Notice Type Alerts you to...
Note Important features or instructions.
Caution Risk of personal injury, system damage, or loss of data.
Warning Risk of severe personal injury.

10 Advanced System Diagnostics and Troubleshooting Guide
Preface
Related Publications
The publications related to this one are:
•ExtremeWare Software User Guide, Software Version 7.7.
•ExtremeWare Software Command Reference, Software Version 7.7.
•ExtremeWare Error Message Decoder.
Documentation for Extreme Networks products is available on the World Wide Web at the following
location:
http://www.extremenetworks.com/services/documentation/Default.asp
Table 2: Text Conventions
Convention Description
Screen displays This typeface indicates command syntax, or represents information as it appears on the
screen.
The words “enter”
and “type” When you see the word “enter” in this guide, you must type something, and then press the Return or Enter key. Do not press the Return or Enter key when an instruction
simply says “type.”
[Key] names Key names are written with brackets, such as [Return] or [Esc].
If you must press two or more keys simultaneously, the key names are linked with a
plus sign (+). Example:
Press [Ctrl]+[Alt]+[Del].
Letter in bold type Letters within a command that appear in bold type indicate the keyboard shortcut for a
command. When entering the command, you can use just the bolded letters instead of
the entire word.
Words in italicized type Italics emphasize a point or denote new terms at the place where they are defined in
the text.