Cisco MDS 9000 Series User manual

Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of
the UNIX operating system. All rights reserved. Copyright ©1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.
All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online version for the latest version.
Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on the Cisco website at www.cisco.com/go/offices.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com
go trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any
other company. (1721R)
©2019 Cisco Systems, Inc. All rights reserved.

CONTENTS
Full Cisco Trademarks with Software License ?
Preface xv
PREFACE
Preface xv
Audience xv
Document Conventions xv
Related Documentation xvi
Communications, Services, and Additional Information xvi
New and Changed Information 1
CHAPTER 1
Change Summary 2
Interface Overview 5
CHAPTER 2
Finding Feature Information 6
Trunks and PortChannels 7
Fibre Channel Port Rate Limiting 8
Maximum NPIV Limit 9
Extended Credits 10
N Port Virtualization 11
FlexAttach 12
Configuring Interfaces 13
CHAPTER 3
Finding Feature Information 14
Feature History for Interfaces 15
Information About Interfaces 17
Interface Description 17
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
iii

Interface Modes 17
E Port 18
F Port 18
FL Port 18
NP Ports 18
TE Port 18
TF Port 19
TNP Port 19
SD Port 19
ST Port 19
Fx Port 19
Auto Mode 20
Interface States 20
Administrative States 20
Operational States 20
Reason Codes 20
Graceful Shutdown 23
Port Administrative Speeds 24
Autosensing 24
Frame Encapsulation 24
Port Beaconing 25
Bit Error Rate Thresholds 25
Disabling the Bit Error Rate Threshold 25
SFP Transmitter Types 26
Portguard 27
Port Level Portguard 27
Port Monitor Portguard 28
Port Monitor 29
Warning Threshold 33
Port Monitor Check Interval 35
Port Group Monitor 35
Interface Types 36
Management Interfaces 36
VSAN Interfaces 36
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
iv
Contents

Prerequisites for Interfaces 37
Guidelines and Limitations 38
Guidelines for Configuring Port Monitor Check Interval 38
Guidelines for VSAN Interface Configuration 40
Guidelines and Limitations for Port Beaconing 40
Default Settings 41
Configuring Interfaces 42
Configuring a Fibre Channel Interface 42
Configuring a Range of Fibre Channel Interfaces 42
Setting the Interface Administrative State 42
Shutting Down an Interface 42
Enabling Traffic Flow 43
Configuring an Interface Mode 43
Configuring the MAX NPIV Limit 44
Configuring the System Default F Port Mode 44
Configuring ISL Between Two Switches 45
Configuring the Port Administrative Speeds 46
Configuring Port Speed Group 46
Configuring the Interface Description 47
Configuring a Port Logical Type 47
Specifying a Port Owner 48
Configuring Beacon Mode 48
Configuring the Port Beacon LED 49
Configuring a Switch Port Attribute Default Value 49
Configuring the Port-Level Portguard 50
Configuring a Port Monitor 52
Enabling a Port Monitor 52
Configuring the Check Interval 53
Configuring a Port Monitor Policy 53
Activating a Port Monitor Policy 54
Configuring Port Monitor Portguard 55
Configuring Port Group Monitor 56
Enabling a Port Group Monitor 56
Configuring a Port Group Monitor Policy 56
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
v
Contents

Reverting to the Default Policy for a Specific Counter 57
Turning Off Specific Counter Monitoring 58
Activating a Port Group Monitor Policy 58
Configuring Management Interfaces 58
Configuring the Management Interface Over IPv4 58
Configuring the Management Interface Over IPv6 59
Creating VSAN Interfaces 60
Verifying Interfaces Configuration 61
Displaying Interface Information 61
Displaying the Port-Level Portguard 70
Displaying Port Monitor Status and Policies 71
Displaying Port Group Monitor Status and Policies 75
Displaying the Management Interface Configuration 76
Displaying VSAN Interface Information 76
Transmit-Wait History Graph 77
Configuring Fibre Channel Interfaces 81
CHAPTER 4
Finding Feature Information 82
Information About Fibre Channel Interfaces 83
Forward Error Correction 83
Dynamic Bandwidth Management 83
Out-of-Service Interfaces 83
Bandwidth Fairness 84
Upgrade or Downgrade Scenario 84
Guidelines and Limitations 85
Port Index Limitations 85
PortChannel Limitations 87
Configuring Fibre Channel Interfaces 91
Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode 91
Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode 91
Configuring Port Speed 92
Configuring FEC 93
Configuring Rate Mode 95
Disabling Restrictions on Oversubscription Ratios 95
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
vi
Contents

Examples 96
Enabling Restrictions on Oversubscription Ratios 98
Enabling Bandwidth Fairness 99
Disabling Bandwidth Fairness 100
Taking Interfaces out of Service 100
Releasing Shared Resources in a Port Group 102
Disabling ACL Adjacency Sharing for System Image Downgrade 102
Verifying Fibre Channel Interfaces Configuration 104
Displaying FEC Module Interfaces 104
Displaying SFP Diagnostic Information 105
Configuration Examples for Fibre Channel Interfaces 106
Configuration Example for FEC Module Interfaces 106
Configuring Interface Buffers 107
CHAPTER 5
Finding Feature Information 108
Information About Interface Buffers 109
Buffer-to-Buffer Credits 109
Performance Buffers 109
Buffer Pools 109
Buffer-to-Buffer Credit Buffers for Switching Modules 109
48-Port 32-Gbps Fibre Channel Module Buffer-to-Buffer Credit Buffers 110
48-Port 16-Gbps Fibre Channel Module Buffer-to-Buffer Credit Buffers 111
Buffer-to-Buffer Credit Buffers for Fabric Switches 112
Cisco MDS 9396S Fabric Switch Buffer-to-Buffer Credit Buffers 112
Cisco MDS 9250i and Cisco MDS 9148S Fabric Switch Buffer-to-Buffer Credit Buffers 113
Extended Buffer-to-Buffer Credits 114
Buffer-to-Buffer Credit Recovery 115
Receive Data Field Size 116
Configuring Interface Buffers 117
Configuring Buffer-to-Buffer Credits 117
Configuring Buffer-to-Buffer Credits for Virtual Links 117
Configuring Performance Buffers 118
Configuring Extended Buffer-to-Buffer Credits 118
Configuring Extended Buffer-to-Buffer Credits for Virtual Links 119
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
vii
Contents

Configuring Buffer-to-Buffer Credit Recovery 120
Configuring Receive Data Field Size 120
Configuration Examples for Interface Buffers 122
Verifying Interface Buffer Configuration 124
Troubleshooting Interface Buffer Credits 126
Congestion Detection, Avoidance, and Isolation 129
CHAPTER 6
Finding Feature Information 130
Feature History for Congestion Detection, Avoidance, and Isolation 131
Information About SAN Congestion 134
Information About SAN Congestion Caused by Slow-Drain Devices 134
Information About SAN Congestion Caused by Over Utilization 137
Information About Congestion Detection, Avoidance, and Isolation 137
Information About Congestion Detection 137
Information About Congestion Avoidance 157
Information About Congestion Isolation 158
Guidelines and Limitations for Congestion Detection, Avoidance, and Isolation 161
Guidelines and Limitations for Congestion Detection 161
Guidelines and Limitations for Congestion Avoidance 162
Guidelines and Limitations for Congestion Isolation 163
Extended Receiver Ready 163
Congestion Isolation 165
Configuring Congestion Avoidance 168
Configuring Congestion Detection 168
Configuring the Slow-Port Monitor Timeout Value for Fibre Channel 169
Configuring Slow Port Monitor for Port Monitor 170
Configuring the Transmit Average Credit-Not-Available Duration Threshold and Action in Port
Monitor 170
Configuring Other Congestion Related Port Monitor Counters 172
Configuring Congestion Avoidance 173
Configuring the Congestion Drop Timeout Value for FCoE 173
Configuring Pause Drop Timeout for FCoE 174
Configuring the Congestion Drop Timeout Value for Fibre Channel 175
Configuring the No-Credit Frame Timeout Value for Fibre Channel 175
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
viii
Contents

Displaying Credit Loss Recovery Actions 176
Configuring Congestion Isolation 177
Configuring Extended Receiver Ready 177
Configuring Congestion Isolation 179
Configuring the Port-Monitor Portguard Action for Congestion Isolation 180
Verifying Slow-Drain Device Detection and Congestion Isolation 182
Configuration Examples for Congestion Detection, Avoidance, and Isolation 183
Configuration Examples for Congestion Detection 183
Configuration Examples for Congestion Avoidance 185
Configuration Examples for Congestion Isolation 187
Verifying Congestion Detection, Avoidance, and Isolation 190
Verifying Congestion Detection and Avoidance 190
Verifying Congestion Isolation 191
Configuring Trunking 197
CHAPTER 7
Finding Feature Information 198
Information About Trunking 199
Trunking E Ports 199
Trunking F Ports 199
Key Concepts 200
Trunking Protocols 201
Trunk Modes 202
Trunk-Allowed VSAN Lists and VF_IDs 202
Guidelines and Limitations 205
General Guidelines and Limitations 205
Upgrade and Downgrade Limitations 205
Difference Between TE Ports and TF-TNP Ports 206
Trunking Misconfiguration Examples 207
Default Settings 209
Configuring Trunking 210
Enabling the Cisco Trunking and Channeling Protocols 210
Enabling the F Port Trunking and Channeling Protocol 210
Configuring Trunk Mode 210
Configuring an Allowed-Active List of VSANs 211
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
ix
Contents

Verifying Trunking Configuration 212
Configuration Example for F Port Trunking 214
Configuring PortChannels 217
CHAPTER 8
Finding Feature Information 218
Information About PortChannels 219
PortChannels Overview 219
E PortChannels 219
F and TF PortChannels 220
PortChanneling and Trunking 220
Load Balancing 221
PortChannel Modes 223
PortChannel Deletion 224
Interfaces in a PortChannel 224
Interface Addition to a PortChannel 225
Forcing an Interface Addition 226
Interface Deletion from a PortChannel 226
PortChannel Protocols 226
Channel Group Creation 227
Autocreation 228
Manually Configured Channel Groups 229
Prerequisites for PortChannels 230
Default Settings 231
Guidelines and Limitations 232
General Guidelines and Limitations 232
Generation 1 PortChannel Limitations 232
F and TF PortChannel Limitations 232
Valid and Invalid PortChannel Examples 233
Configuring PortChannels 235
Configuring PortChannels Using the Wizard Creating a PortChannel 235
Configuring the PortChannel Mode 235
Deleting PortChannels 235
Adding an Interface to a PortChannel 236
Adding a Range of Ports to a PortChannel 236
Cisco MDS 9000 Series Interfaces Configuration Guide, Release 8.x
x
Contents