ExtremeSwitching 3600 Series Operating instructions

Troubleshooting Ethernet Routing Switch
3600 Series
Release 6.3
9035810 Rev AA
August 2019

© 2017-2019, Extreme Networks, Inc.
All Rights Reserved.
Legal Notice
Extreme Networks, Inc. reserves the right to make changes in
specifications and other information contained in this document and
its website without prior notice. The reader should in all cases consult
representatives of Extreme Networks to determine whether any such
changes have been made.
The hardware, firmware, software or any specifications described or
referred to in this document are subject to change without notice.
Trademarks
Extreme Networks and the Extreme Networks logo are trademarks or
registered trademarks of Extreme Networks, Inc. in the United States
and/or other countries.
All other names (including any product names) mentioned in this
document are the property of their respective owners and may be
trademarks or registered trademarks of their respective companies/
owners.
For additional information on Extreme Networks trademarks, please
see: www.extremenetworks.com/company/legal/trademarks
Open Source Declarations
Some software files have been licensed under certain open source or
third-party licenses. End-user license agreements and open source
declarations can be found at: www.extremenetworks.com/support/
policies/software-licensing

Contents
Chapter 1: About this Document............................................................................................. 6
Purpose.................................................................................................................................. 6
Conventions............................................................................................................................ 6
Text Conventions............................................................................................................... 6
Documentation and Training..................................................................................................... 8
Getting Help............................................................................................................................ 9
Providing Feedback to Us...................................................................................................... 10
Chapter 2: New in this document.......................................................................................... 11
Chapter 3: Troubleshooting tools......................................................................................... 12
Chapter 4: Troubleshooting planning................................................................................... 13
Chapter 5: Troubleshooting fundamentals........................................................................... 15
Port mirroring........................................................................................................................ 15
Port mirroring limitations.................................................................................................. 15
Many to one Port mirroring............................................................................................... 16
Port mirroring commands................................................................................................. 16
Port statistics........................................................................................................................ 16
System logs.......................................................................................................................... 17
Remote logging..................................................................................................................... 17
Software Exception Log......................................................................................................... 17
Show environmental.............................................................................................................. 17
ASCII Configurator Generator................................................................................................ 18
CPU and Memory Utilization................................................................................................... 18
SNMP trap enhancements..................................................................................................... 18
SNMP Trap list web page in EDM........................................................................................... 19
Remote monitoring (RMON) (RFC1757) per port statistics, history, alarms, and events............... 19
Chapter 6: Troubleshooting Fabric Attach........................................................................... 20
Verifying FA settings.............................................................................................................. 20
Verifying FA message authentication status............................................................................. 20
Verifying FA per-port settings.................................................................................................. 21
Verifying discovered FA elements........................................................................................... 22
Chapter 7: General diagnostic tools..................................................................................... 23
CLI Command Modes............................................................................................................ 23
Chapter 8: Initial troubleshooting......................................................................................... 26
Gather information................................................................................................................. 26
Chapter 9: Emergency recovery trees.................................................................................. 28
Emergency recovery trees..................................................................................................... 28
Corruption of flash................................................................................................................. 29
Incorrect Port VLAN Identifier (PVID)...................................................................................... 30
August 2019 Troubleshooting ERS 3600 Series 3

Uplink ports not tagged to VLAN............................................................................................. 32
SNMP.................................................................................................................................. 33
SNMP recovery tree........................................................................................................ 34
Stack.................................................................................................................................... 36
Stack recovery tree.......................................................................................................... 37
Dynamic Host Configuration Protocol (DHCP) relay................................................................. 42
DHCP recovery tree........................................................................................................ 42
Agent Recovery.................................................................................................................... 43
AAUR: configuration for the units in the stack is not saved on the base unit............................... 44
Configuration for the units in the stack is not saved on the base unit recovery tree............... 44
AAUR: Both units display yes for Ready for Replacement......................................................... 46
Both units display yes for Ready for Replacement recovery tree.......................................... 46
DAUR................................................................................................................................... 48
Diagnostic image transfer does not start recovery tree....................................................... 48
Stack Forced Mode............................................................................................................... 50
You cannot access a switch at the stack IP address using ping, Telnet, SSH, Web, or EDM
recovery tree................................................................................................................... 50
Stack Health Check: Cascade Up and Cascade Down columns display LINK DOWN or
MISSING.............................................................................................................................. 51
Cascade Up and Cascade Down columns display LINK DOWN or MISSING recovery tree... 52
Stack Health Check: Cascade Up and Cascade Down columns display UP WITH ERRORS....... 54
Cascade Up and Cascade Down columns display UP WITH ERRORS recovery tree............ 54
Using the Diagnostics Menu................................................................................................... 56
Example Checking PVID of ports............................................................................................ 57
Example VLAN Interface VLAN IDs........................................................................................ 57
Tagging options..................................................................................................................... 59
Chapter 10: Troubleshooting hardware................................................................................ 60
Check power......................................................................................................................... 62
Check port............................................................................................................................ 63
Check fiber port..................................................................................................................... 65
Replace unit.......................................................................................................................... 67
Chapter 11: Troubleshooting ADAC...................................................................................... 69
IP phone is not detected........................................................................................................ 70
Correct filtering................................................................................................................ 70
Reload ADAC MAC in range table.................................................................................... 71
Reduce LLDP devices..................................................................................................... 72
Auto configuration is not applied............................................................................................. 73
Correct auto configuration................................................................................................ 74
Chapter 12: Troubleshooting authentication....................................................................... 76
EAP client authentication....................................................................................................... 77
Restore RADIUS connection............................................................................................ 78
Enable EAP on the PC..................................................................................................... 80
Apply the method............................................................................................................ 81
Contents
August 2019 Troubleshooting ERS 3600 Series 4

Enable EAP globally........................................................................................................ 81
EAP multihost repeated re-authentication issue....................................................................... 83
Match EAP-MAC-MAX to EAP users................................................................................. 83
Set EAPOL request packet............................................................................................... 84
EAP RADIUS VLAN is not being applied................................................................................. 85
Configure VLAN at RADIUS............................................................................................. 86
Configure switch.............................................................................................................. 88
Configured MAC is not authenticating..................................................................................... 89
Configure the switch........................................................................................................ 90
Non-EAP RADIUS MAC not authenticating.............................................................................. 93
Configure switch.............................................................................................................. 94
RADIUS server configuration error.................................................................................... 96
Non-EAP MHSA MAC is not authenticating............................................................................. 97
Configure switch.............................................................................................................. 98
EAP–non-EAP unexpected port shutdown............................................................................. 100
Configure switch............................................................................................................ 101
Non-EAP freeform password................................................................................................ 103
Contents
August 2019 Troubleshooting ERS 3600 Series 5

Chapter 1: About this Document
This section discusses the purpose of this document, the conventions used, ways to provide
feedback, additional help, and information regarding other Extreme Networks publications.
Purpose
This document describes common problems and error messages and the techniques to resolve
them.
Conventions
This section discusses the conventions used in this guide.
Text Conventions
The following tables list text conventions that can be used throughout this document.
Table 1: Notice Icons
Icon Alerts you to...
Important: A situation that can cause serious inconvenience.
Note: Important features or instructions.
Tip: Helpful tips and notices for using the product.
Danger: Situations that will result in severe bodily injury; up to
and including death.
Warning: Risk of severe personal injury or critical loss of data.
Caution: Risk of personal injury, system damage, or loss of
data.
August 2019 Troubleshooting ERS 3600 Series 6

Table 2: Text Conventions
Convention Description
Angle brackets ( < > ) Angle brackets ( < > ) indicate that you choose the
text to enter based on the description inside the
brackets. Do not type the brackets when you enter
the command.
If the command syntax is cfm maintenance-
domain maintenance-level <0-7> , you can
enter cfm maintenance-domain
maintenance-level 4.
Bold text Bold text indicates the GUI object name you must act
upon.
Examples:
•Click OK.
• On the Tools menu, choose Options.
Braces ( { } ) Braces ( { } ) indicate required elements in syntax
descriptions. Do not type the braces when you enter
the command.
For example, if the command syntax is ip address
{A.B.C.D}, you must enter the IP address in
dotted, decimal notation.
Brackets ( [ ] ) Brackets ( [ ] ) indicate optional elements in syntax
descriptions. Do not type the brackets when you
enter the command.
For example, if the command syntax is show clock
[detail], you can enter either show clock or
show clock detail.
Ellipses ( … ) An ellipsis ( … ) indicates that you repeat the last
element of the command as needed.
For example, if the command syntax is
ethernet/2/1 [ <parameter>
<value> ]..., you enter ethernet/2/1 and as
many parameter-value pairs as you need.
Italic Text Italics emphasize a point or denote new terms at the
place where they are defined in the text. Italics are
also used when referring to publication titles that are
not active links.
Plain Courier Text Plain Courier text indicates command names,
options, and text that you must enter. Plain Courier
text also indicates command syntax and system
output, for example, prompts and system messages.
Table continues…
Conventions
August 2019 Troubleshooting ERS 3600 Series 7

Convention Description
Examples:
•show ip route
•Error: Invalid command syntax
[Failed][2013-03-22 13:37:03.303
-04:00]
Separator ( > ) A greater than sign ( > ) shows separation in menu
paths.
For example, in the Navigation tree, expand the
Configuration > Edit folders.
Vertical Line ( | ) A vertical line ( | ) separates choices for command
keywords and arguments. Enter only one choice. Do
not type the vertical line when you enter the
command.
For example, if the command syntax is access-
policy by-mac action { allow | deny } ,
you enter either access-policy by-mac action
allow or access-policy by-mac action
deny, but not both.
Documentation and Training
To find Extreme Networks product guides, visit our documentation pages at:
Current Product Documentation www.extremenetworks.com/documentation/
Archived Documentation (for earlier
versions and legacy products)
www.extremenetworks.com/support/documentation-archives/
Release Notes www.extremenetworks.com/support/release-notes
Hardware/Software Compatibility
Matrices
https://www.extremenetworks.com/support/compatibility-matrices/
White papers, data sheets, case
studies, and other product resources
https://www.extremenetworks.com/resources/
Training
Extreme Networks offers product training courses, both online and in person, as well as specialized
certifications. For more information, visit www.extremenetworks.com/education/.
About this Document
August 2019 Troubleshooting ERS 3600 Series 8

Getting Help
If you require assistance, contact Extreme Networks using one of the following methods:
Extreme
Portal
Search the GTAC (Global Technical Assistance Center) knowledge base, manage
support cases and service contracts, download software, and obtain product
licensing, training, and certifications.
The Hub A forum for Extreme Networks customers to connect with one another, answer
questions, and share ideas and feedback. This community is monitored by Extreme
Networks employees, but is not intended to replace specific guidance from GTAC.
Call GTAC For immediate support: 1-800-998-2408 (toll-free in U.S. and Canada) or +1
408-579-2826. For the support phone number in your country, visit:
www.extremenetworks.com/support/contact
Before contacting Extreme Networks for technical support, have the following information ready:
• Your Extreme Networks service contract number and/or serial numbers for all involved Extreme
Networks products
• A description of the failure
• A description of any action(s) already taken to resolve the problem
• A description of your network environment (such as layout, cable type, other relevant
environmental information)
• Network load at the time of trouble (if known)
• The device history (for example, if you have returned the device before, or if this is a recurring
problem)
• Any related RMA (Return Material Authorization) numbers
Subscribing to Service Notifications
You can subscribe to email notifications for product and software release announcements,
Vulnerability Notices, and Service Notifications.
1. Go to www.extremenetworks.com/support/service-notification-form.
2. Complete the form with your information (all fields are required).
3. Select the products for which you would like to receive notifications.
Note:
You can modify your product selections or unsubscribe at any time.
4. Click Submit.
Getting Help
August 2019 Troubleshooting ERS 3600 Series 9

Providing Feedback to Us
Quality is our first concern at Extreme Networks, and we have made every effort to ensure the
accuracy and completeness of this document. We are always striving to improve our documentation
and help you work better, so we want to hear from you! We welcome all feedback but especially
want to know about:
• Content errors or confusing or conflicting information.
• Ideas for improvements to our documentation so you can find the information you need faster.
• Broken links or usability issues.
If you would like to provide feedback to the Extreme Networks Information Development team, you
can do so in two ways:
• Use our short online feedback form at https://www.extremenetworks.com/documentation-
feedback/.
• Email us at [email protected].
Please provide the publication title, part number, and as much detail as possible, including the topic
heading and page number if applicable, as well as your suggestions for improvement.
About this Document
August 2019 Troubleshooting ERS 3600 Series 10

Chapter 2: New in this document
There are no feature updates in this document.
August 2019 Troubleshooting ERS 3600 Series 11

Chapter 3: Troubleshooting tools
This document:
•describes the diagnostic tools and utilities available for troubleshooting using Command Line
Interface (CLI) and Enterprise Device Manager (EDM)
• guides you through some common problems to achieve a first tier solution to these situations
• advises you what information to compile prior to troubleshooting or calling Extreme Networks
for help
This documents assumes that you:
• have basic knowledge of networks, ethernet bridging, and IP routing
• are familiar with networking concepts and terminology
• have experience with Graphical User Interface (GUI)
• have basic knowledge of network topologies
Troubleshooting tools
The switch supports a range of protocols, utilities, and diagnostic tools that you can use to monitor
and analyze traffic, monitor laser operating characteristics, capture and analyze data packets, trace
data flows, view statistics, and manage event messages.
Certain protocols and tools are tailored for troubleshooting specific network topologies. Other tools
are more general in their application and can be used to diagnose and monitor ingress and egress
traffic.
August 2019 Troubleshooting ERS 3600 Series 12

Chapter 4: Troubleshooting planning
You can minimize the need for troubleshooting and to plan for doing it as effectively as possible.
1. Use the Documentation Reference for Ethernet Routing Switch 3600 Series to familiarize
yourself with the documentation set, so you know where to get information when you need it.
2. Make sure the system is properly installed and maintained so that it operates as expected.
3. Make sure you gather and keep up to date the site map, logical connections, device
configuration information, and other data that you will require if you have to troubleshoot:
• A site network map identifies where each device is physically located on your site, which
helps locate the users and applications that are affected by a problem. You can use the
map to systematically search each part of your network for problems.
• You must know how your devices are connected logically and physically with virtual local
area networks (VLAN).
• Maintain online and paper copies of your device configuration information. Ensure that
all online data is stored with your site’s regular data backup for your site. If your site has
no backup system, copy the information onto a backup medium and store the backup
offsite.
• Store passwords in a safe place. It is a good practice to keep records of your previous
passwords in case you must restore a device to a previous software version. You need to
use the old password that was valid for that version.
• A good practice is to maintain a device inventory, which list all devices and relevant
information for your network. Use this inventory to easily see the device types, IP
addresses, ports, MAC addresses, and attached devices.
• If your hubs or switches are not managed, you must keep a list of the MAC addresses
that correlate to the ports on your hubs and switches.
• Maintain a change-control system for all critical systems. Permanently store change-
control records.
• A good practice is to store the details of all key contacts, such as support contacts,
support numbers, engineer details, and telephone and fax numbers. Having this
information available during troubleshooting saves you time.
August 2019 Troubleshooting ERS 3600 Series 13

4. Understand the normal network behavior so you can be more effective at troubleshooting
problems.
• Monitor your network over a period of time sufficient to allow you to obtain statistics and
data to see patterns in the traffic flow, such as which devices are typically accessed or
when peak usage times occur.
• Use a baseline analysis as an important indicator of overall network health. A baseline
view of network traffic as it typically is during normal operation is a reference that you can
compare to network traffic data that you capture during troubleshooting. This speeds the
process of isolating network problems.
Troubleshooting planning
August 2019 Troubleshooting ERS 3600 Series 14

Chapter 5: Troubleshooting fundamentals
This section describes available troubleshooting tools and their applications.
Port mirroring
The port mirroring feature helps you to monitor and analyze network traffic. This feature supports
both ingress (incoming traffic) and egress (outgoing traffic) port mirroring. When port mirroring is
enabled, the ingress or egress packets of the mirrored (source) port are forwarded normally and a
copy of the packets is sent from the mirrored port to the mirroring (destination) port.
You can observe and analyze packet traffic at the mirroring port using a network analyzer. A copy of
the packet can be captured and analyzed. Unlike other methods that are used to analyze packet
traffic, the packet traffic is uninterrupted and packets flow normally through the mirrored port.
Port mirroring limitations
The supports port mirroring in the following three modes:
•ingress mode (XRX or ->Port X)
• egress mode (XTX or Port X ->)
• ingress and egress mode (XRX or XTX or <->Port X)
There are limitations to the egress mode. As a standalone unit or in a stack, port-mirroring mode
XTX mirrors egress traffic on the mirrored port, but does not mirror control packets generated by the
switch. The monitor port does not receive copies of the generated control packets that egress from
the mirrored port.
The following limitations apply to the ingress and egress mode:
• the same limitation on the XTX portion also applies to the ingress and egress mode
• the monitor port and the mirror port should be on the same unit in a stack.
August 2019 Troubleshooting ERS 3600 Series 15

Many to one Port mirroring
The Many to One Port Mirroring feature provides the ability of mirroring multiple ports to a single
monitor port. With this feature, you can configure a single port to capture traffic from a set of
selected ports. The captured traffic can be ingress or egress traffic.
Many to One Port Mirroring is supported in Stand-alone and stack configurations.
Many to One Port Mirroring modes
There are four modes of Many to One Port Mirroring. They are:
• ManyToOneRx. Monitors all traffic received on the mirrored ports.
• ManyToOneTx. Monitors all traffic transmitted by the mirrored ports.
• ManyToOneRxTx. Monitors all traffic received or transmitted by the mirror ports.
• XrxOrYtx. Monitors all traffic received by port X or transmitted by port Y.
Many to One Port Mirroring limitations
Following are the Many to One Port Mirroring limitations:
• Only ingress and egress port traffic mirroring is supported
• Can configure up to four mirror ports
• Mirror ports must continue to perform normal frame switching operation
• Cannot configure an MLT group as monitor port
Port mirroring commands
Use the port mirroring commands to assist in diagnostics and information gathering.
For more information, see Configuring System Monitoring on Ethernet Routing Switch 3600 Series.
Port statistics
Use port statistics commands to display information on received and transmitted packets at the
ports. The ingress and egress counts occur at the MAC layer.
For more information regarding port statistics and commands, see Configuring System Monitoring
on Ethernet Routing Switch 3600 Series.
Troubleshooting fundamentals
August 2019 Troubleshooting ERS 3600 Series 16

System logs
You can use the syslog messaging feature to manage event messages. The syslog software
communicates with a server software component called syslogd that resides on your management
workstation.
The daemon syslogd is a software component that receives and locally logs, displays, prints, or
forwards messages that originate from sources that are internal and external to the workstation. For
example, syslogd software concurrently handles messages received from applications running on
the workstation, as well as messages received from a switch running in a network accessible to the
workstation.
For more information about system logging, see Configuring System Monitoring on Ethernet Routing
Switch 3600 Series.
Remote logging
As part of configuring system logging, you can specify remote logging parameters. This involves
configuring a remote syslog address, enabling remote logging, and configuring the remote logging
level.
For more information, see Configuring System Monitoring on Ethernet Routing Switch 3600 Series.
Software Exception Log
This feature allows an administrator to see the software exceptions generated in the switching
system. The software exception log provides a method for capturing software faults in the SYSLOG
application as critical customer messages. The CLI allows you to display and clear the last software
exceptions generated in the system.
For more information, see Configuring System Monitoring on Ethernet Routing Switch 3600 Series.
Show environmental
This feature displays environmental information, such as power supply status, fan status, and switch
system temperature.
For more information, see Configuring System Monitoring on Ethernet Routing Switch 3600 Series.
System logs
August 2019 Troubleshooting ERS 3600 Series 17

ASCII Configurator Generator
Use the ASCII Configurator Generator (ACG) tool to easily modify the configuration of a switch.
ACG generates an ASCII configuration file, which reproduces the behavior of the current binary
configuration file. This function maintains backup configurations and provides a method to debug the
current configuration of a switch.
For more information about configuration files, see Configuring Systems on Ethernet Routing Switch
3600 Series.
CPU and Memory Utilization
The CPU and Memory Utilization feature provides data for CPU and memory utilization. You can
view CPU utilization information for the past 10 seconds (s), 1 minute (min), 10 minutes (min), 1
hour (hr), 24 hours (hr), or since system startup. The switch displays CPU utilization as a
percentage. With CPU utilization information you can see how the CPU was used during a specific
time interval.
The memory utilization provides information about the percentage of the dynamic memory currently
used by the system. The switch displays memory utilization in terms of the lowest percentage of
dynamic memory available since system startup.
No configuration is required for this display-only feature. For more information, see Configuring
System Monitoring on Ethernet Routing Switch 3600 Series.
SNMP trap enhancements
With SNMP management, you can configure SNMP traps to automatically generate notifications
globally, or on individual ports. These notifications can report conditions, such as an unauthorized
access attempt or changes in port operating status. All notifications are enabled on individual
interfaces by default.
The switch supports both industry-standard SNMP traps, as well as private Extreme Networks
enterprise traps. SNMP trap notification-control provides a generic mechanism for the trap
generation control that works with any trap type.
For more information, see Configuring System Monitoring on Ethernet Routing Switch 3600 Series.
Troubleshooting fundamentals
August 2019 Troubleshooting ERS 3600 Series 18

SNMP Trap list web page in EDM
You can use Enterprise Device Manager (EDM) MIB Web page to query SNMP objects on the
switch.
For more information, see Configuring System Monitoring on Ethernet Routing Switch 3600 Series.
Remote monitoring (RMON) (RFC1757) per port statistics,
history, alarms, and events
Remote Monitoring (RMON) MIB is an interface between the RMON agent on a switch and an
RMON management application, such as Enterprise Device Manager (EDM).
The RMON agent defines objects that are suitable for the management of any type of network, but
some groups are targeted for Ethernet networks in particular. The RMON agent continuously
collects statistics and monitors switch performance. This data can be viewed through CLI and EDM.
RMON has three major functions:
• creating and displaying alarms for user-defined events
• gathering cumulative statistics for Ethernet interfaces
• tracking a history of statistics for Ethernet interfaces
For more information on RMON per port statistics, history, alarms, and events, see Configuring
System Monitoring on Ethernet Routing Switch 3600 Series.
SNMP Trap list web page in EDM
August 2019 Troubleshooting ERS 3600 Series 19

Chapter 6: Troubleshooting Fabric Attach
This chapter contains details about how to troubleshoot common Fabric Attach (FA) problems you
might encounter.
Verifying FA settings
Use this procedure to verify the FA settings.
Procedure
1. Enter Privileged EXEC mode:
enable
2. Verify the FA settings:
show fa agent
Example
The following example displays output sample for the show fa agent command in FA Proxy
mode.
Switch(config)#show fa agent
Fabric Attach Service Status: Enabled
Fabric Attach Element Type: Proxy
Fabric Attach Zero Touch Status: Enabled
Fabric Attach Auto Provision Setting: Proxy
Fabric Attach Provision Mode: Legacy
Fabric Attach Client Proxy Status: Enabled
Fabric Attach Standalone Proxy Status: Disabled
Fabric Attach Agent Timeout: 240 seconds
Fabric Attach Extended Logging Status: Disabled
Fabric Attach Primary Server Id: <none>
Fabric Attach Primary Server Descr: <none>
Verifying FA message authentication status
Use this procedure to verify whether both FA Proxy and FA Server have the same authentication
settings (enabled on both, or disabled on both).
August 2019 Troubleshooting ERS 3600 Series 20