Brocade Communications Systems A7533A - Brocade 4Gb SAN Switch Base Operating instructions

53-1000853-01
12 March 2008
Fabric OS
Troubleshooting and Diagnostics Guide
Supporting Fabric OS v6.1.0

Copyright © 2008 Brocade Communications Systems, Inc. All Rights Reserved.
Brocade, Fabric OS, File Lifecycle Manager, MyView, and StorageX are registered trademarks and the Brocade B-wing symbol,
DCX, and SAN Health are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
All other brands, products, or service names are or may be trademarks or service marks of, and are used to identify, products or
services of their respective owners.
Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning
any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to
this document at any time, without notice, and assumes no responsibility for its use. This informational document describes
features that may not be currently available. Contact a Brocade sales office for information on feature and product availability.
Export of technical data contained in this document may require an export license from the United States government.
The authors and Brocade Communications Systems, Inc. shall have no liability or responsibility to any person or entity with
respect to any loss, cost, liability, or damages arising from the information contained in this book or the computer programs that
accompany it.
The product described by this document may contain “open source” software covered by the GNU General Public License or other
open source license agreements. To find-out which open source software is included in Brocade products, view the licensing
terms applicable to the open source software, and obtain a copy of the programming source code, please visit
http://www.brocade.com/support/oscd.
Brocade Communications Systems, Incorporated
Document History
Corporate Headquarters
Brocade Communications Systems, Inc.
1745 Technology Drive
San Jose, CA 95110
Tel: [phone]
Fax: [phone]
Email: [e-mail]
Asia-Pacific Headquarters
Brocade Communications Singapore Pte. Ltd.
9 Raffles Place
#59-02 Republic Plaza 1
Singapore 048619
Tel: [phone]
Fax: [phone]
Email: [e-mail]
European and Latin American Headquarters
Brocade Communications Switzerland Sàrl
Centre Swissair
Tour A - 2ème étage
29, Route de l'Aéroport
Case Postale 105
CH-1215 Genève 15
Switzerland
Tel: [phone]
Fax: [phone]
Email: emea[email protected]
Title Publication number Summary of changes Date
Fabric OS Troubleshooting and
Diagnostics Guide
53-0000853-01 First released edition. 12 March 2008

Fabric OS Troubleshooting and Diagnostics Guide iii
53-1000853-01
Contents
About This Document
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Supported hardware and software . . . . . . . . . . . . . . . . . . . . . . . . . . . x
What’s new in this document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
Document conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Text formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Notes, cautions, and warnings . . . . . . . . . . . . . . . . . . . . . . . . . . .xi
Key terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Brocade resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Other industry resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Getting technical help. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Document feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Chapter 1 Introduction to Troubleshooting
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
About troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Network time protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Most common problem areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Questions for common symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Gathering information for your switch support provider . . . . . . . . . . 5
Setting up your switch for FTP. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Capturing a supportSave. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Capturing a supportShow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Capturing output from a console . . . . . . . . . . . . . . . . . . . . . . . . . 6
Capturing command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Building a case for your switch support provider . . . . . . . . . . . . . . . . 7
Basic switch information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Detailed problem information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . . 9
Chapter 2 General Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Licensing issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

iv Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
Switch Message Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Checking fan components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Checking the switch temperature. . . . . . . . . . . . . . . . . . . . . . . . 13
Checking the power supply . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Checking the temperature, fan, and power supply . . . . . . . . . .13
Fibre Channel Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Checking for Fibre Channel connectivity problems . . . . . . . . . .14
Third party applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Chapter 3 Connections Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Port initialization and FCP auto discovery process . . . . . . . . . . . . . . 17
Link issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Connection problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Checking the logical connection . . . . . . . . . . . . . . . . . . . . . . . . . 19
Checking the name server (NS) . . . . . . . . . . . . . . . . . . . . . . . . .20
Link failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Determining a successful negotiation . . . . . . . . . . . . . . . . . . . . 22
Checking for a loop initialization failure . . . . . . . . . . . . . . . . . . .22
Checking for a point-to-point initialization failure . . . . . . . . . . .23
Correcting a port that has come up in the wrong mode . . . . . .23
Marginal links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Troubleshooting a marginal link . . . . . . . . . . . . . . . . . . . . . . . . . 24
Device login issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Pinpointing problems with device logins . . . . . . . . . . . . . . . . . .27
Media-related issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Testing a port’s external transmit and receive path . . . . . . . . . 30
Testing a switch’s internal components. . . . . . . . . . . . . . . . . . . 30
Testing components to and from the HBA . . . . . . . . . . . . . . . . .30
Segmented fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Reconciling fabric parameters individually . . . . . . . . . . . . . . . .31
Downloading a correct configuration . . . . . . . . . . . . . . . . . . . . . 32
Reconciling a domain ID conflict . . . . . . . . . . . . . . . . . . . . . . . . 32
Chapter 4 Configuration Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Configupload and download issues. . . . . . . . . . . . . . . . . . . . . . . . . .33
Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 35
Brocade configuration form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Chapter 5 FirmwareDownload Errors
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Blade troubleshooting tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
Firmware download issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

Fabric OS Troubleshooting and Diagnostics Guide v
53-1000853-01
Troubleshooting firmwareDownload . . . . . . . . . . . . . . . . . . . . . . . . . 41
Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 41
Brocade DCX error handling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
USB error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Considerations for downgrading firmware . . . . . . . . . . . . . . . . . . . .44
Preinstallation messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Blade types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Firmware versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
IP settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Port settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Chapter 6 Security Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Password issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Password recovery options . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
Protocol and certificate management issues . . . . . . . . . . . . . . . . . .58
Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 59
SNMP issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 60
FIPS issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
Chapter 7 ISL Trunking Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Link issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Buffer credit issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Chapter 8 Zone Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Overview of corrective action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Verifying a fabric merge problem . . . . . . . . . . . . . . . . . . . . . . . . 63
Segmented fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Zone conflicts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
Correcting a fabric merge problem quickly . . . . . . . . . . . . . . . . 65
Editing zone configuration members . . . . . . . . . . . . . . . . . . . . . 66
Reordering the zone member list . . . . . . . . . . . . . . . . . . . . . . . .66
Checking for Fibre Channel connectivity problems . . . . . . . . . .66
Checking for zoning problems. . . . . . . . . . . . . . . . . . . . . . . . . . .68
Gathering additional information. . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

vi Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
Chapter 9 FCIP Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
FCIP tunnel issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
FCIP links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 72
Port mirroring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Supported hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73
Port mirroring considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Port mirroring management . . . . . . . . . . . . . . . . . . . . . . . . . . . .75
FTRACE concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76
Tracing Fibre Channel information . . . . . . . . . . . . . . . . . . . . . . . 76
Chapter 10 FICON Fabric Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
FICON issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Troubleshooting FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80
General information to gather for all cases . . . . . . . . . . . . . . . . 80
Identifying ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81
Single-switch topology checklist . . . . . . . . . . . . . . . . . . . . . . . . . 82
Cascade mode topology checklist . . . . . . . . . . . . . . . . . . . . . . . 82
Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 82
Troubleshooting FICON CUP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83
Troubleshooting FICON NPIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85
Chapter 11 iSCSI Issues
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87
Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90
Chapter 12 Working With Diagnostic Features
In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
About Fabric OS diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Diagnostic information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92
Power-on self test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92
Switch status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Viewing the overall status of the switch. . . . . . . . . . . . . . . . . . .94
Displaying switch information . . . . . . . . . . . . . . . . . . . . . . . . . . .95
Displaying the uptime for a switch . . . . . . . . . . . . . . . . . . . . . . .95

Fabric OS Troubleshooting and Diagnostics Guide vii
53-1000853-01
Port information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Viewing the status of a port . . . . . . . . . . . . . . . . . . . . . . . . . . . .96
Displaying the port statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . .97
Displaying a summary of port errors for a switch . . . . . . . . . . .98
Equipment status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Displaying the status of the fans . . . . . . . . . . . . . . . . . . . . . . . .99
Displaying the status of a power supply. . . . . . . . . . . . . . . . . . .99
Displaying temperature status . . . . . . . . . . . . . . . . . . . . . . . . .100
System message log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100
Displaying the system message log, with no page breaks . . .100
Displaying the system message log one message at a time .100
Clearing the system message log . . . . . . . . . . . . . . . . . . . . . . .101
Port log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101
Viewing the port log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101
Syslogd configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102
Configuring the host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103
Configuring the switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103
Automatic trace dump transfers . . . . . . . . . . . . . . . . . . . . . . . . . . .104
Specifying a remote server . . . . . . . . . . . . . . . . . . . . . . . . . . . .105
Enabling the automatic transfer of trace dumps. . . . . . . . . . .105
Setting up periodic checking of the remote server . . . . . . . . .105
Saving comprehensive diagnostic files to the server . . . . . . .105
Diagnostic tests not supported by M-EOS 9.6.2 and FOS 6.0. . . .106
Appendix A Switch Type
Appendix B Hexidecimal
Index

viii Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01

Fabric OS Troubleshooting and Diagnostics Guide ix
53-1000853-01
About This Document
In this chapter
•How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
•Supported hardware and software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
•What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
•Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
•Additional information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
•Getting technical help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
•Document feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
How this document is organized
The document contains the following chapters:
•Chapter 1, “Introduction to Troubleshooting,” gives a brief overview of Fabric OS, explains the
Fabric OS CLI Help feature, and provides typical connection and configuration procedures.
•Chapter 2, “General Issues,” provides information on licensing, hardware, and syslog issues.
•Chapter 3, “Connections Issues,” provides information and procedures on managing
authentication and user accounts for the switch management channel.
•Chapter 4, “Configuration Issues,” provides information and procedures for configuring ACL
policies for FC port and switch binding and managing the fabric-wide consistency policy.
•Chapter 5, “FirmwareDownload Errors,” provides procedures for maintaining and backing up
your switch configurations.
•Chapter 6, “Security Issues,” provides procedures for basic password and user account
management.
•Chapter 7, “ISL Trunking Issues,” describes the concepts and provides procedures for using
administrative domains.
•Chapter 8, “Zone Issues,” provides preparations and procedures for performing firmware
downloads, as well troubleshooting information.
•Chapter 9, “FCIP Issues,” provides information and procedures specific to Brocade 48000 and
Brocade DCX models. Because these models have CP blades and port blades, they require
procedures that are not relevant to the Brocade fixed-port models.
•Chapter 10, “FICON Fabric Issues,” provides information and procedures specific to Brocade
48000 and Brocade DCX models. Because these models have CP blades and port blades, they
require procedures that are not relevant to the Brocade fixed-port models.

xFabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
•Chapter 11, “iSCSI Issues,” provides information and procedures specific to Brocade 48000
and Brocade DCX models. Because these models have CP blades that support the iSCSI
feature.
•Chapter 12, “Working With Diagnostic Features,” provides procedures for use of the Brocade
Adaptive Networking suite of tools, including Traffic Isolation, QoS Ingress Rate Limiting, and
QoS SID/DID Traffic Prioritization.
•The appendices provide special information to guide you in understanding switch output.
Supported hardware and software
In those instances in which procedures or parts of procedures documented here apply to some
switches but not to others, this guide identifies exactly which switches are supported and which are
not.
Although many different software and hardware configurations are tested and supported by
Brocade Communications Systems, Inc. for 6.1.0, documenting all possible configurations and
scenarios is beyond the scope of this document.
The following hardware platforms are supported by this release of Fabric OS:
•Brocade 200E switch
•Brocade 300 switch
•Brocade 4016 switch
•Brocade 4018 switch
•Brocade 4020 switch
•Brocade 4024 switch
•Brocade 4100 switch
•Brocade 4900 switch
•Brocade 5000 switch
•Brocade 5100 switch
•Brocade 5300 switch
•Brocade 7500 switch
•Brocade 7600 switch
•Brocade 48000 director
•Brocade DCX Backbone
What’s new in this document
This manual is a compilation of all the information originally distributed through out the Fabric OS
Administrator’s Guide chapters.

Fabric OS Troubleshooting and Diagnostics Guide xi
53-1000853-01
Document conventions
This section describes text formatting conventions and important notice formats used in this
document.
Text formatting
The narrative-text formatting conventions that are used are as follows:
bold text Identifies command names
Identifies the names of user-manipulated GUI elements
Identifies keywords and operands
Identifies text to enter at the GUI or CLI
italic text Provides emphasis
Identifies variables
Identifies paths and Internet addresses
Identifies document titles
code text Identifies CLI output
Identifies command syntax examples
For readability, command names in the narrative portions of this guide are presented in mixed
lettercase: for example, switchShow. In actual examples, command lettercase is often all
lowercase. Otherwise, this manual specifically notes those cases in which a command is case
sensitive.
Notes, cautions, and warnings
The following notices and statements are used in this manual. They are listed below in order of
increasing severity of potential hazards.
NOTE
A note provides a tip, guidance or advice, emphasizes important information, or provides a
reference to related information.
ATTENTION
An Attention statement indicates potential damage to hardware or data.
CAUTION
A Caution statement alerts you to situations that can be potentially hazardous to you.
DANGER
A Danger statement indicates conditions or situations that can be potentially lethal or extremely
hazardous to you. Safety labels are also attached directly to products to warn of these conditions
or situations.

xii Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
Key terms
For definitions specific to Brocade and Fibre Channel, see the Brocade Glossary.
For definitions of SAN-specific terms, visit the Storage Networking Industry Association online
dictionary at:
http://www.snia.org/education/dictionary
Additional information
This section lists additional Brocade and industry-specific documentation that you might find
helpful.
Brocade resources
To get up-to-the-minute information, join Brocade Connect. It’s free! Go to
http://www.brocade.com and click Brocade Connect to register at no cost for a user ID and
password.
For practical discussions about SAN design, implementation, and maintenance, you can obtain
Building SANs with Brocade Fabric Switches through:
http://www.amazon.com
For additional Brocade documentation, visit the Brocade SAN Info Center and click the Resource
Library location:
http://www.brocade.com
Release notes are available on the Brocade Connect Web site and are also bundled with the Fabric
OS firmware.
Other industry resources
•White papers, online demos, and data sheets are available through the Brocade Web site at
http://www.brocade.com/products/software.jhtml.
•Best practice guides, white papers, data sheets, and other documentation is available through
the Brocade Partner Web site.
For additional resource information, visit the Technical Committee T11 Web site. This Web site
provides interface standards for high-performance and mass storage applications for Fibre
Channel, storage management, and other applications:
http://www.t11.org
For information about the Fibre Channel industry, visit the Fibre Channel Industry Association Web
site:
http://www.fibrechannel.org

Fabric OS Troubleshooting and Diagnostics Guide xiii
53-1000853-01
Getting technical help
Contact your switch support supplier for hardware, firmware, and software support, including
product repairs and part ordering. To expedite your call, have the following information available:
1. General Information
•Switch model
•Switch operating system version
•Error numbers and messages received
•supportSave command output
•Detailed description of the problem, including the switch or fabric behavior immediately
following the problem, and specific questions
•Description of any troubleshooting steps already performed and the results
•Serial console and Telnet session logs
•syslog message logs
2. Switch Serial Number
The switch serial number and corresponding bar code are provided on the serial number label,
as illustrated below.:
The serial number label is located as follows:
•Brocade 200E—On the nonport side of the chassis.
•Brocade 4016—On the top of the switch module.
•Brocade 4018—On the top of the blade.
•Brocade 4020 and 4024—On the bottom of the switch module.
•Brocade 4100, 4900, and 7500—On the switch ID pull-out tab located inside the chassis
on the port side on the left.
•Brocade 5000—On the switch ID pull-out tab located on the bottom of the port side of the
switch
•Brocade 300, 5100, and 5300—On the switch ID pull-out tab located on the bottom of the
port side of the switch.
•Brocade 7600—On the bottom of the chassis.
•Brocade 48000—Inside the chassis next to the power supply bays.
•Brocade DCX Backbone—On the bottom right on the port side of the chassis.
3. World Wide Name (WWN)
Use the wwn command to display the switch WWN.
If you cannot use the wwn command because the switch is inoperable, you can get the WWN
from the same place as the serial number, except for the Brocade DCX. For the Brocade DCX,
access the numbers on the WWN cards by removing the Brocade logo plate at the top of the
nonport side of the chassis.
*FT00X0054E9*
FT00X0054E9

xiv Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
For the Brocade 4016, 4018, 4020, and 4024 embedded switches: Provide the license ID.
Use the licenseIdShow command to display the WWN.
Document feedback
Quality is our first concern at Brocade and we have made every effort to ensure the accuracy and
completeness of this document. However, if you find an error or an omission, or you think that a
topic needs further development, we want to hear from you. Forward your feedback to:
[email protected]
Provide the title and version number of the document and as much detail as possible about your
comment, including the topic heading and page number and your suggestions for improvement.

Fabric OS Troubleshooting and Diagnostics Guide 1
53-1000853-01
Chapter
1
Introduction to Troubleshooting
This chapter provides information on troubleshooting and the most common procedures to use to
diagnose and recover from problems.
This book is a companion guide to be used in conjunction with the Fabric OS Administrator’s Guide.
Although it provides a lot of common troubleshooting tips and techniques it does not teach
troubleshooting methodology.
In this chapter
•About troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
•Most common problem areas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
•Questions for common symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
•Gathering information for your switch support provider. . . . . . . . . . . . . . . . . 5
•Building a case for your switch support provider . . . . . . . . . . . . . . . . . . . . . . 7
About troubleshooting
Troubleshooting should begin at the center of the SAN — the fabric. Because switches are located
between the hosts and storage devices and have visibility into both sides of the storage network,
starting with them can help narrow the search path. After eliminating the possibility of a fault within
the fabric, see if the problem is on the storage side or the host side, and continue a more detailed
diagnosis from there. Using this approach can quickly pinpoint and isolate problems.
For example, if a host cannot detect a storage device, run a switch command, for example
switchShow to determine if the storage device is logically connected to the switch. If not, focus first
on the switch directly connecting to storage. Use your vendor-supplied storage diagnostic tools to
better understand why it is not visible to the switch. If the storage can be detected by the switch,
and the host still cannot detect the storage device, then there is still a problem between the host
and switch.
Network time protocol
One of the most frustrating parts of troubleshooting is trying to synchronize switch’s message logs
and portlogs with other switches in the fabric. If you do not have NTP set up on your switches, then
trying to synchronize log files to track a problem is practically impossible.

2Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
Most common problem areas
1
Most common problem areas
Table 1 identifies the most common problem areas that arise within SANs and identifies tools to
use to resolve them.
Questions for common symptoms
You first need to determine what the problem is. Some symptoms are obvious, such as the switch
rebooted without any user intervention, or more obscure, such as your storage is having
intermittent connectivity to a particular host. Whatever the symptom is, you will need to gather
information from the devices that are directly involved in the symptom.
Table 2 lists common symptoms and possible areas to check. You may notice that an intermittent
connectivity problem has lots of variables to look into, such as the type of connection between the
two devices, how the connection is behaving, and the port type involved.
TABLE 1 Common troubleshooting problems and tools
Problem area Investigate Tools
Fabric •Missing devices
•Marginal links (unstable connections)
•Incorrect zoning configurations
•Incorrect switch configurations
•Switch LEDs
•Switch commands (for example,
switchShow or nsAllShow) for
diagnostics
•Web or GUI-based monitoring and
management software tools
Storage Devices •Physical issues between switch and
devices
•Incorrect storage software
configurations
•Device LEDs
•Storage diagnostic tools
•Switch commands (for example,
switchShow or nsAllShow) for
diagnostics
Hosts •Downlevel HBA firmware
•Incorrect device driver installation
•Incorrect device driver configuration
•Host operating system diagnostic
tools
•Device driver diagnostic tools
•Switch commands (for example,
switchShow or nsAllShow) for
diagnostics
Also, make sure you use the latest HBA
firmware recommended by the switch
supplier or on the HBA supplier's web site
Storage Management
Applications
•Incorrect installation and
configuration of the storage devices
that the software references.
For example, if using a
volume-management application,
check for:
•Incorrect volume installation
•Incorrect volume configuration
•Application-specific tools and
resources

Fabric OS Troubleshooting and Diagnostics Guide 3
53-1000853-01
Questions for common symptoms 1
TABLE 2 Common symptoms
Symptom Areas to check Chapter
BadRootDev errors Firmware versions on switch Chapter 5, “FirmwareDownload Errors”
Blade is faulty Firmware or application download
Hardware
Chapter 2, “General Issues”
Chapter 5, “FirmwareDownload Errors”
Blade is stuck in the “LOADING” state Firmware or application download Chapter 5, “FirmwareDownload Errors”
Configupload or download fails FTP or SCP server or USB availability Chapter 4, “Configuration Issues”
E_Port failed to come online Correct licensing
Fabric parameters
Zoning
Chapter 2, “General Issues”
Chapter 3, “Connections Issues”
Chapter 8, “Zone Issues”
EX_Port does not form Links Chapter 3, “Connections Issues”
Fabric merge fails Fabric segmentation Chapter 2, “General Issues”
Chapter 3, “Connections Issues”
Chapter 8, “Zone Issues”
Fabric segments Licensing
Zoning
Fabric parameters
Chapter 2, “General Issues”
Chapter 3, “Connections Issues”
Chapter 8, “Zone Issues”
FCIP tunnel bounces FCIP tunnel Chapter 9, “FCIP Issues”
FCIP tunnel does not come online FCIP tunnel Chapter 9, “FCIP Issues”
FCIP tunnel does not form Licensing
Fabric parameters
Chapter 2, “General Issues”
Chapter 9, “FCIP Issues”
FCIP tunnel is sluggish FCIP tunnel Chapter 9, “FCIP Issues”
Feature is not working Licensing Chapter 2, “General Issues”
FICON switch does not talk to hosts FICON settings Chapter 10, “FICON Fabric Issues”
FirmwareDownload fails FTP or SCP server or USB availability
Firmware version compatibility
Unsupported features enabled
Chapter 5, “FirmwareDownload Errors”
Intermittent connectivity Links
Trunking
Buffer credits
FCIP tunnel
Chapter 3, “Connections Issues”
Chapter 7, “ISL Trunking Issues”
Chapter 9, “FCIP Issues”
LEDs are flashing Links Chapter 3, “Connections Issues”
LEDs are steady Links Chapter 3, “Connections Issues”
Marginal link Links Chapter 3, “Connections Issues”
No connectivity between host and storage Cables
SCSI timeout errors
SCSI retry errors
Chapter 3, “Connections Issues”
Chapter 7, “ISL Trunking Issues”
Chapter 9, “FCIP Issues”
No connectivity between switches Licensing
Fabric parameters
Segmentation
Zoning, if applicable
Chapter 2, “General Issues”
Chapter 3, “Connections Issues”
Chapter 8, “Zone Issues”
No light on LEDs Links Chapter 3, “Connections Issues”
Performance problems Links
FCIP tunnels
Chapter 3, “Connections Issues”
Chapter 9, “FCIP Issues”

4Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
Questions for common symptoms
1
SCSI retry errors Buffer credits
FCIP tunnel bandwidth
Chapter 9, “FCIP Issues”
SCSI timeout errors Links
HBA
Buffer credits
FCIP tunnel bandwidth
Chapter 3, “Connections Issues”
Chapter 7, “ISL Trunking Issues”
Chapter 9, “FCIP Issues”
Switch constantly reboots FIPS Chapter 6, “Security Issues”
Switch is unable to join fabric Security policies
Zoning
Fabric parameters
Chapter 3, “Connections Issues”
Switch reboots during configup/download Configuration file discrepancy Chapter 4, “Configuration Issues”
Syslog messages Hardware
SNMP management station
Chapter 2, “General Issues”
Chapter 6, “Security Issues”
Trunk bounces Cables are on same port group
SFPs
Trunked ports
Chapter 7, “ISL Trunking Issues”
Trunk failed to form Licensing
Cables are on same port group
SFPs
Trunked ports
Zoning
Chapter 2, “General Issues”
Chapter 3, “Connections Issues”
Chapter 7, “ISL Trunking Issues”
Chapter 8, “Zone Issues”
User forgot password Password recovery Chapter 6, “Security Issues”
User is unable to change switch settings RBAC settings
Account settings
Chapter 6, “Security Issues”
Zone configuration mismatch Effective configuration Chapter 8, “Zone Issues”
Zone content mismatch Effective configuration Chapter 8, “Zone Issues”
Zone type mismatch Effective configuration Chapter 8, “Zone Issues”
TABLE 2 Common symptoms
Symptom Areas to check Chapter

Fabric OS Troubleshooting and Diagnostics Guide 5
53-1000853-01
Gathering information for your switch support provider 1
Gathering information for your switch support provider
If you are troubleshooting a production system, you must gather data quickly. As soon as a problem
is observed, perform the following tasks (if using a dual CP system, run the commands on both
CPs). For more information about these commands and their operands, refer to the Fabric OS
Command Reference.
1. Enter the supportSave command to save RASLOG, TRACE, supportShow, core file, FFDC data,
and other support information.
NOTE
It is recommended that you use the supportFtp command to set up the supportSave environment
for automatic dump transfers using the -n and -c options; this will save you from having to enter or
know all the required FTP parameters needed to successfully execute a supportSave operation.
•Enter the supportShow command to collect information for the local CP to a remote FTP
location or using the USB memory device on supporting products. This command does not
collect RASLOG, TRACE, core files or FFDC data.
To capture the data from the supportShow command, you will need to run the command
through a Telnet or SSH utility or serial console connection.
2. Gather console output and logs.
For more details about these commands, see the Fabric OS Command Reference.
Setting up your switch for FTP
1. Connect to the switch and log in using an account assigned to the admin role.
2. Type the following command:
supportFtp -s [-h hostip][-u username][-p password][-d remotedirectory]
3. Respond to the prompts as follows:
Example of supportFTP command
switch:admin> supportftp -s
Host IP Addr[1080::8:800:200C:417A]:
User Name[njoe]:
Password[********]:
Remote Dir[support]:
Auto file transfer parameters changed
-h hostip Specifies FTP host IP address. It must be an IP address. hostip should be less
than 48 characters.
-u username Enter the user name of your account on the server; for example, “JaneDoe”.
-d remotedirectory Specifies remote directory to store trace dump files. The supportFtp command
cannot take a slash (/) as a directory name. The remote directory should be
less than 48 characters.
-p password Specifies FTP user password. If the user name is anonymous, the password is
not needed. password should be less than 48 characters.

6Fabric OS Troubleshooting and Diagnostics Guide
53-1000853-01
Gathering information for your switch support provider
1
Capturing a supportSave
1. Connect to the switch and log in using an account assigned to the admin role.
2. Type the supportSave command.
When invoked without operands, this command goes into interactive mode.The following
operands are optional:
-n Does not prompt for confirmation. This operand is optional; if omitted, you are prompted for
confirmation.
-c Uses the FTP parameters saved by the supportFtp command. This operand is optional; if
omitted, specify the FTP parameters through command line options or interactively. To display
the current FTP parameters, run supportFtp (on a dual-CP system, run supportFtp on the active
CP).
Capturing a supportShow
1. Connect to the switch through a Telnet or SSH utility or a serial console connection.
2. Log in using an account assigned to the admin role.
3. Set the Telnet or SSH utility to capture output from the screen.
Some Telnet or SSH utilities require this step to be performed prior to opening up a session.
Check with your Telnet or SSH utility vendor for instructions.
4. Type the supportShow command.
Capturing output from a console
Some information, such as boot information is only outputted directly to the console. In order to
capture this information you have to connect directly to the switch through its management
interface, either a serial cable or an RJ-45 connection.
1. Connect directly to the switch using hyperterminal.
2. Log in to the switch using an account assigned to the admin role.
3. Set the utility to capture output from the screen.
Some utilities require this step to be performed prior to opening up a session. Check with your
utility vendor for instructions.
4. Type the command or start the process to capture the required data on the console.
Capturing command output
1. Connect to the switch through a Telnet or SSH utility.
2. Log in using an account assigned to the admin role.
3. Set the Telnet or SSH utility to capture output from the screen.
Some Telnet or SSH utilities require this step to be performed prior to opening up a session.
Check with your Telnet or SSH utility vendor for instructions.
4. Type the command or start the process to capture the required data on the console.