Cisco Nexus B22 Guide

© 2014 Cisco | IBM. All rights reserved. Page 1
Cisco Nexus B22 Blade Fabric
Extender for IBM
Design and Deployment Guide
November 2013
Int

© 2014 Cisco | IBM. All rights reserved. Page 2
Contents
Introduction .............................................................................................................................................................. 3
Network Diagram...................................................................................................................................................... 3
Hardware Installation............................................................................................................................................... 5
Fabric Extender Management Model .....................................................................................................................6
Fabric Connectivity Options ................................................................................................................................... 7
Statically Pinned Fabric Interface Connection.......................................................................................................7
Port Channel Fabric Interface Connection ............................................................................................................9
Configuring a Fabric Port Channel ......................................................................................................................10
Virtual Port Channel Connection..........................................................................................................................13
Configuring a vPC................................................................................................................................................14
Server Network Teaming.......................................................................................................................................16
Creating Host-Side vPC for Server Links with LACP ..........................................................................................17
Fibre Channel over Ethernet.................................................................................................................................21
Configuring FCoE ................................................................................................................................................21
Configuring the Cisco Nexus 5000 Series and B22 for IBM for FCoE ................................................................22
iSCSI Configuration ...............................................................................................................................................27
Virtual Network Adapter Partitioning................................................................................................................... 30
Debug Commands .................................................................................................................................................35
show fex...............................................................................................................................................................35
show fex detail.....................................................................................................................................................36
show interface brief..............................................................................................................................................37
show interface ethernet 191/1/1 ..........................................................................................................................42
show vlan.............................................................................................................................................................43
show interface fex-fabric......................................................................................................................................44
Cisco Nexus Configurations.................................................................................................................................45
Cisco Nexus 5000 Series Switch 1 Configuration ...............................................................................................45
Cisco Nexus 5000 Series Switch 2 Configuration ...............................................................................................56
Conclusion..............................................................................................................................................................70
For More Information.............................................................................................................................................70

© 2014 Cisco | IBM. All rights reserved. Page 3
Introduction
The Cisco Nexus®B22 Blade Fabric Extender for IBM®extends the Cisco Nexus switch fabric to the server edge.
Logically, it behaves like a remote line card to a parent Cisco Nexus 5000 or 6000 Series Switch. The fabric
extender and the parent Cisco Nexus 5000 or 6000 Series Switch together form a distributed modular system.
The Cisco Nexus B22 for IBM forwards all traffic to the parent Cisco Nexus 5000 or 6000 Series Switch over eight
10 Gigabit Ethernet uplinks. Low-cost uplink connections of up to 10 meters can be made with copper Twinax
cable, and longer connections of up to 100 meters can use the Cisco®10-Gbps fabric extender transceiver (FET-
10G). Standard 10-Gbps optics such as short reach (SR), long reach (LR), and extended reach (ER) are also
supported. Downlinks to each server are 10 Gigabit Ethernet and work with all Ethernet and converged network
adapter (CNA) mezzanine cards, allowing customers a choice of Ethernet, Fibre Channel over Ethernet (FCoE),
or Small Computer System Interface over IP (iSCSI) connections. Because the Cisco Nexus B22 for IBM is a
transparent extension of a Cisco Nexus switch, traffic can be switched according to policies established by the
Cisco Nexus switch using a single point of management.
The Cisco Nexus B22 for IBM provides the following benefits:
Highly scalable, consistent server access: This distributed modular system creates a scalable server access
environment with no reliance on Spanning Tree Protocol and with consistent features and architecture between
blade and rack servers.
Simplified operations: The availability of one single point of management and policy enforcement using upstream
Cisco Nexus 5000 Series Switches eases the commissioning and decommissioning of blades through zero-touch
installation and automatic configuration of fabric extenders.
Increased business benefits: Consolidation, reduced cabling, investment protection through feature inheritance
from the parent switch, and the capability to add functions without the need for a major equipment upgrade of
server-attached infrastructure all contribute to reduced operating expenses (OpEx) and capital expenditures
(CapEx).
The Cisco Nexus B22 for IBM integrates into the I/O module slot of a third-party blade chassis, drawing both
power and cooling from the blade chassis itself.
Network Diagram
Figure 1 presents a sample network topology that can be built using the Cisco Nexus B22 for IBM, 2000 Series
Fabric Extenders, and 5000 or 6000 Series Switches. In this topology, the Cisco Nexus 5000 or 6000 Series
Switch serves as the parent switch, performing all packet switching and policy enforcement for the entire
distributed modular system. The Cisco Nexus switch also serves as the only point of management for both
configuration and monitoring within the domain, making it simple to manage blade server and rack server
connections together.

© 2014 Cisco | IBM. All rights reserved. Page 4
Figure 1: Cisco Nexus Virtual Chassis Topology
The Cisco Nexus switches, along with the Cisco Nexus 2000 Series and B22 for IBM, create a distributed
modular system that unifies the data center architecture. Within this distributed modular system, both IBM Flex
System®computing nodes and rack servers are managed identically. This approach allows the use of the same
business and technical processes and procedures.
The left-most blade chassis in Figure 1 contains dual Cisco Nexus B22 for IBM fabric extenders. Each Cisco
Nexus B22 for IBM is singlely attached to a parent Cisco Nexus 5500 platform switch, a connection mode referred
to as straight-through mode. The fabric links can be either statically pinned or put into a Port Channel. This
connection mode helps ensure that all data packets from a particular Cisco Nexus B22 for IBM enter the same
parent Cisco Nexus switch. This approach may be necessary when certain types of traffic must be restricted to
either the left or right Cisco Nexus 5500 platform switch: for instance, to maintain SAN A and SAN B separation.
Also, in this example the connections to individual computing nodes are in active-standby mode, which helps
ensure traffic flow consistency but does not make full use of the server network interface card (NIC) bandwidth.
The second IBM Flex System chassis from the left in Figure 1 improves on the first with the creation of an
Ethernet virtual Port Channel (vPC) from the computing node to the Cisco Nexus parent switch. This vPC places
the Ethernet portion of the NICs in an active-active configuration, giving increased bandwidth to each host. The
FCoE portion of the CNA is also configured as active-active but maintains SAN A and SAN B separation because
each virtual Fibre Channel (vFC) interface is bound to a particular link at the server. This configuration also
achieves high availability through redundancy, and it can withstand a failure of a Cisco Nexus 5500 platform
switch, a Cisco Nexus B22 for IBM, or any connecting cable. This topology is widely used in FCoE deployments.
The third blade chassis from the left in Figure 1 contains Cisco Nexus B22 for IBM fabric extenders that connect
to both Cisco Nexus 5500 platform switches through vPC for redundancy. In this configuration, active-active load
balancing using vPC from the blade server to the Cisco Nexus 5500 platform switch cannot be enabled. However,
the servers can still be dual-homed with active-standby or active-active transmit-load-balancing (TLB) teaming.
This topology is only for Ethernet traffic because SAN A and SAN B separation between the fabric extender and
the parent switch is necessary.

© 2014 Cisco | IBM. All rights reserved. Page 5
The fourth blade chassis from the left in Figure 1 contains Cisco Nexus B22 for IBM fabric extenders that connect
to both Cisco Nexus 5500 platform switches with enhanced vPC (EvPC) technology. This configuration allows
active-active load balancing from the fabric extenders and the computing nodes.
The last two configurations show how rack-mount servers can connect to the same Cisco Nexus parent switch
using rack-mount Cisco Nexus 2000 Series Fabric Extenders. The topology for blade servers and rack-mount
servers can be identical if desired.
Hardware Installation
Installation of the Cisco Nexus B22 for IBM in the rear of the blade server chassis is similar to the installation of
other connection blades. The layout of the blade server chassis, as well as the server types and mezzanine cards
used, determines the slots that should be populated with the Cisco Nexus B22 for IBM for 10 Gigabit Ethernet
connectivity. Tables 1 through 3 summarize the typical options for servers using dual-port 10 Gigabit Ethernet
devices.
Table 1 Mapping of Third-Party Half-Wide Server Dual-Port Mezzanine Card to I/O Module
Card
Connection Blades
LAN on motherboard (LoM) plus
mezzanine card in slot 1
I/O module bays 1 and 2
Mezzanine card in slot 2
I/O module bays 3 and 4
Table 2 Mapping of Third-Party Half-Wide Server Quad-Port Mezzanine Card to I/O Module
Card
Connection Blades
Mezzanine 1 ports 1 to 4
I/O module bays 1 and 2
Mezzanine 2 ports 1 to 4
I/O module bays 3 and 4
Table 3 Mapping of Third-Party Full-Wide Server Quad-Port Mezzanine Card to I/O Module
Card
Connection Blades
Mezzanine 1 ports 1 to 4
I/O module bays 1 and 2
Mezzanine 2 ports 1 to 4
I/O module bays 3 and 4
Mezzanine 3 ports 1 to 4
I/O module bays 1 and 2
Mezzanine 4 ports 1 to 4
I/O module bays 3 and 4
After the Cisco Nexus B22 for IBM fabric extenders are installed, the chassis management module (CMM) should
be updated to at least the minimum version shown in Table 4.
Table 4 Management Blade Minimum Firmware Versions
Blade Chassis
Server Manager Firmware
IBM PureFlex™System Model 8721HC1
DSA:9.41, IMM2:2.6, UEFI:1.31, and CMM: 2PET12E

© 2014 Cisco | IBM. All rights reserved. Page 6
No configuration is required from the chassis MMB. Only the minimum CMM firmware is required to properly
detect and enable the Cisco Nexus B22 for IBM in the blade chassis (Figure 2).
Figure 2: Cisco Nexus B22 for IBM Fabric Extenders as Seen in the CMM
Fabric Extender Management Model
The Cisco Nexus fabric extenders are managed by a parent switch through the fabric interfaces using a zero-
touch configuration model. The switch discovers the fabric extender by using a detection protocol.
After discovery, if the fabric extender has been correctly associated with the parent switch, the following
operations are performed:
1. The switch checks the software image compatibility and upgrades the fabric extender if necessary.
2. The switch and fabric extender establish in-band IP connectivity with each other. The switch assigns an IP
address in the range of loopback addresses (127.15.1.0/24) to the fabric extender to avoid conflicts with IP
addresses that might be in use on the network.
3. The switch pushes the configuration data to the fabric extender. The fabric extender does not store any
configuration locally.
4. The fabric extender updates the switch with its operating status. All fabric extender information is displayed
using the switch commands for monitoring and troubleshooting.
This management model allows fabric extender modules to be added without adding management points or
complexity. Software image and configuration management is also handled automatically, without the need for
user intervention.

© 2014 Cisco | IBM. All rights reserved. Page 7
Fabric Connectivity Options
The Cisco Nexus B22 for IBM creates a distributed, modular chassis with the Cisco Nexus parent switch after a
fabric connection has been made over standard 10-Gbps cabling. This connection can be accomplished using
any of the following types of interconnects:
Cisco passive direct-attach cables (1m, 3m, or 5m)
Cisco active direct-attach cables (7m or 10m)
Cisco standard Enhanced Small Form-Factor Pluggable (SFP+) optics (SR, LR, and ER)
Cisco Fabric Extender Transceivers (FET modules)
After the fabric links have been physically established, the logical configuration of the links must be established.
The fabric links to the Cisco Nexus B22 for IBM can use either of two connection methods:
Statically pinned fabric interface connection
Port Channel fabric interface connection
Statically Pinned Fabric Interface Connection
Static pinning is the default method of connection between the fabric extender and the Cisco Nexus parent switch.
In this mode of operation, a deterministic relationship exists between the host interfaces and the upstream parent;
up to eight fabric interfaces can be connected. These fabric interfaces are equally divided among the 16 server-
side host ports. If fewer fabric ports are allocated, more server ports are assigned to a single fabric link. The
advantage of this configuration is that the traffic path and the amount of allocated bandwidth are always known for
a particular set of servers.
Since static pinning will group host-side ports into individual fabric links, you should understand how ports are
grouped. The size of the port groups is determined by the number of host ports divided by the max link
parameter value. For example, if the max link parameter is set to 2, eight host ports would be assigned to each
link. The interfaces will be grouped in ascending order starting from interface 1. Thus, interfaces 1 to 8 will be
pinned to one fabric link, and interfaces 9 to 16 will be pinned to a different interface (Table 5).
Table 5 Interface Assignment with Two Fabric Links
Interface
Fabric Link
1, 2, 3, 4, 5, 6, and 7
Fabric link 1
8, 9, 10, 11, 12, 13, and 14
Fabric link 2

© 2014 Cisco | IBM. All rights reserved. Page 8
Table 6 summarizes the assignment with four fabric links: With the max link parameter set to 4, the interfaces are
divided into four groups.
Table 6 Interface Assignment with Four Fabric Links
Interface
Fabric Link
1, 2, 3, and 4
Fabric link 1
5, 6, 7, and 8
Fabric link 2
9, 10, and 11
Fabric link 3
12, 13, and 14
Fabric link 4
Table 7 summarizes the assignment of eight fabric links: With the max link parameter set to 8, the interfaces are
divided into eight groups.
Table 7 Interface Assignment with Eight Fabric Links
Interface
Fabric Link
1 and 2
Fabric link 1
3 and 4
Fabric link 2
5 and 6
Fabric link 3
7 and 8
Fabric link 4
9 and 10
Fabric link 5
11 and 12
Fabric link 6
13
Fabric link 7
14
Fabric link 8
Note: The assignment of the host-side ports is always based on the configured max link parameter and not the actual
physical number of fabric ports connected. Be sure to match the max link parameter with the actual number of
physical links used.

© 2014 Cisco | IBM. All rights reserved. Page 9
The relationship of host-side ports to parent switch fabric ports is static. If a fabric interface fails, all its associated
host interfaces are brought down and will remain down until the fabric interface is restored. Figure 3 shows static
port mappings.
Figure 3: Static Port Mapping Based on Max Link Parameter
Port Channel Fabric Interface Connection
The Port Channel fabric interface provides an alternative way of connecting the parent switch and the Cisco
Nexus B22 for IBM fabric extender. In this mode of operation, the physical fabric links are bundled into a single
logical channel. This approach prevents a single fabric interconnect link loss from disrupting traffic to any one
server. The total bandwidth of the logical channel is shared by all the servers, and traffic is spread across the
members through the use of a hash algorithm.
For a Layer 2 frame, the switch uses the source and destination MAC addresses.
For a Layer 3 frame, the switch uses the source and destination MAC addresses and the source and
destination IP addresses.
Since both redundancy and increased bandwidth are possible, configuration of the fabric links on a Port Channel
is the most popular connection option.

© 2014 Cisco | IBM. All rights reserved. Page 10
Figure 4 shows Port Channel designs.
Note: A fabric interface that fails in the Port Channel does not trigger a change to the host interfaces. Traffic is automatically
redistributed across the remaining links in the Port Channel fabric interface.
Figure 4: Port Channel Designs
Configuring a Fabric Port Channel
Follow these steps to configure a fabric Port Channel.
1. Log into the first parent switch and enter configuration mode:
Nexus 5000 Switch
login: admin
Password:
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2011, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
N5548-Bottom# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.

© 2014 Cisco | IBM. All rights reserved. Page 11
N5548-Bottom(config)#
2. Enable the fabric extender feature:
N5548-Bottom(config)#
N5548-Bottom(config)# feature fex
N5548-Bottom(config)#
3. Logically create the fabric extender:
N5548-Bottom(config)#
N5548-Bottom(config)# fex 191
N5548-Bottom(config-fex)#
4. Create the Port Channel, change the port mode, and associate the fabric extender with the Port
Channel:
N5548-Bottom(config-if)# interface port-channel 191
N5548-Bottom(config-if)# switchport mode fex-fabric
N5548-Bottom(config-if)# fex associate 191
N5548-Bottom(config-if)#
5. Assign the Cisco Nexus parent switch ports to the Port Channel:
N5548-Bottom(config-if)# interface ethernet 1/17
N5548-Bottom(config-if)# switchport mode fex-fabric
N5548-Bottom(config-if)# fex associate 191
N5548-Bottom(config-if)# channel-group 191
N5548-Bottom(config-if)# interface ethernet 1/18
N5548-Bottom(config-if)# switchport mode fex-fabric
N5548-Bottom(config-if)# fex associate 191
N5548-Bottom(config-if)# channel-group 191

© 2014 Cisco | IBM. All rights reserved. Page 12
6. Repeat the steps on the second Cisco Nexus 5000 Series Switch connected to the fabric extender in
interconnect bay 4:
N5548-Top# configure terminal
N5548-Top(config)# feature fex
N5548-Top(config)# fex 192
N5548-Top(config-if)# interface port-channel 192
N5548-Top(config-if)# switchport mode fex-fabric
N5548-Top(config-if)# fex associate 192
N5548-Top(config-if)# interface ethernet 1/17
N5548-Top(config-if)# switchport mode fex-fabric
N5548-Top(config-if)# fex associate 192
N5548-Top(config-if)# channel-group 192
N5548-Top(config-if)# interface ethernet 1/18
N5548-Top(config-if)# switchport mode fex-fabric
N5548-Top(config-if)# fex associate 192
N5548-Top(config-if)# channel-group 192
7. Verify that the Cisco Nexus B22 for IBM is up and running:
switch(config-if)# show fex
FEX FEX FEX FEX
Number Description State Model Serial
------------------------------------------------------------------------
192 FEX0192 Online N2K-B22IBM-P FOC1730R0XQ
A pair of fabric extenders now is configured in straight-through mode, also known as a single-attached
configuration, and each is communicating with its respective Cisco Nexus switch. The links between the two
Cisco Nexus switches and the Cisco Nexus B22 fabric extenders use Port Channels for connectitivity.

© 2014 Cisco | IBM. All rights reserved. Page 13
Virtual Port Channel Connection
vPCs allow links that are physically connected to two different Cisco Nexus switches to form a Port Channel to a
downstream device. The downstream device can be a switch, a server, or any other networking device that
supports IEEE 802.3ad Port Channels. vPC technology enables networks to be designed with multiple links for
redundancy while also allowing those links to connect to different endpoints for added resiliency (Figure 5).
More information about vPC technology can be found at
http://www.cisco.com/en/US/products/ps9670/products_implementation_design_guides_list.html.
Figure 5: Blade Server Configuration Options

© 2014 Cisco | IBM. All rights reserved. Page 14
Configuring a vPC
The high-level steps for enabling vPC are listed here. This configuration should be implemented on both switches
in parallel.
1. Enable the vPC feature.
2. Create the vPC domain.
3. Configure the peer keepalive link.
4. Configure the vPC peer link Port Channel.
5. Check the status of the vPC.
1. Enable the vPC feature:
N5548-Bottom# configure terminal
N5548-Bottom(config)# feature vpc
N5548-Top# configure terminal
N5548-Top(config)# feature vpc
2. Create the vPC domain (the domain should be unique within network):
N5548-Bottom(config)# vpc domain 5
N5548-Bottom(config)# role priority 1
N5548-Bottom(config)# system-priority 1
N5548-Top(config)# vpc domain 5
N5548-Top(config)# role priority 2
N5548-Top(config)# system-priority 1
3. Configure the peer keepalive link over the management network:
N5548-Bottom(config-vpc-domain)# peer-keepalive destination 172.25.182. 109 source
172.25.182. 108
Note:
--------:: Management VRF will be used as the default VRF ::--------
N5548-Top(config-vpc-domain)# peer-keepalive destination 172.25.182. 108 source
172.25.182. 109
Note:
--------:: Management VRF will be used as the default VRF ::--------

© 2014 Cisco | IBM. All rights reserved. Page 15
4. Configure the vPC peer link:
N5548-Bottom# interface port-channel 20
N5548-Bottom(config-if)# interface ethernet 1/9
N5548-Bottom(config-if)# channel-group 20
N5548-Bottom(config-if)# interface ethernet 1/10
N5548-Bottom(config-if)# channel-group 20
N5548-Bottom(config-if)# interface port-channel 20
N5548-Bottom(config-if)# vpc peer-link
Please note that spanning tree port type is changed to "network" port type on vPC peer-
link.
This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge
Assurance(which is enabled by default) is not disabled.
N5548-Bottom(config-if)#
N5548-Top# interface port-channel 20
N5548-Top (config-if)# interface ethernet 1/9
N5548-Top(config-if)# channel-group 20
N5548-Top(config-if)# interface ethernet 1/10
N5548-Top(config-if)# channel-group 20
N5548-Top(config-if)# interface port-channel 20
N5548-Top(config-if)# vpc peer-link
Please note that spanning tree port type is changed to "network" port type on vPC peer-
link.
This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge
Assurance(which is enabled by default) is not disabled.
N5548-Bottom(config-if)#
5. Check the vPC status:
N5548-Bottom(config-if)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 5
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : success
Type-2 consistency status : success

© 2014 Cisco | IBM. All rights reserved. Page 16
vPC role : primary
Number of vPCs configured : 0
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po20 up 1,182
N5548-Bottom(config-if)#
Now the two switches have been configured to support vPC links to other devices. These connections can be
used for upstream links to the data center core. These vPC links can be used for connections to hosts in the data
center, allowing additional bandwidth and redundant links.
Server Network Teaming
Server NIC teaming provides an additional layer of redundancy to servers. It makes it possible for multiple links to
be available, for redundancy. In the blade server environment, server network teaming typically is limited to
active-standby configurations and cannot provide active-active links, because active-active links require an
EtherChannel or Link Aggregation Control Protocol (LACP) connection to a single switch. However, because the
Cisco Nexus B22 for IBM fabric extender is an extension of the parent switch, EtherChannel or LACP connections
can be created between the blade server and the virtual chassis. Dual Cisco Nexus switches can be used with
vPC for additional switch redundancy while providing active-active links to servers, thus enabling aggregate 40-
Gbps bandwidth with dual links (Figure 6).
Figure 6: Fabric Link and Server Topologies

© 2014 Cisco | IBM. All rights reserved. Page 17
Figure 7: Enhanced vPC Configuration
Creating Host-Side vPC for Server Links with LACP
1. Enable LACP on both parent switches.
5548-Bottom(config)# feature lacp
5548-Top(config)# feature lacp
2. Define and configure the left diagram FEX 191 for enhanced vPC on the left Nexus parent
5548-Bottom(config)# fex 191
5548-Bottom(config-fex)# interface ethernet 1/21-22
5548-Bottom(config-if)# channel-group 191
5548-Bottom(config-if)# no shutdown
5548-Bottom(config-if)# interface port-channel 191
5548-Bottom(config-if)# switchport mode fex-fabric
5548-Bottom(config-if)# fex associate 191
5548-Bottom(config-if)# vpc 191
5548-Bottom(config-if)# no shutdown

© 2014 Cisco | IBM. All rights reserved. Page 18
3. Define and configure the right diagram FEX 192 for enhanced vPC on the left Nexus parent
5548-Bottom(config)# fex 192
5548-Bottom(config-fex)# interface ethernet 1/23-24
5548-Bottom(config-if)# channel-group 192
5548-Bottom(config-if)# no shutdown
5548-Bottom(config-if)# interface port-channel 192
5548-Bottom(config-if)# switchport mode fex-fabric
5548-Bottom(config-if)# fex associate 192
5548-Bottom(config-if)# vpc 192
5548-Bottom(config-if)# no shutdown
4. Define and configure the left diagram FEX 191 for enhanced vPC on the right Nexus parent
5548-Top(config)# fex 191
5548-Top(config-fex)# interface ethernet 1/21-22
5548-Top(config-if)# channel-group 191
5548-Top(config-if)# no shutdown
5548-Top(config-if)# interface port-channel 191
5548-Top(config-if)# switchport mode fex-fabric
5548-Top(config-if)# fex associate 191
5548-Top(config-if)# vpc 191
5548-Top(config-if)# no shutdown
5. Define and configure the right diagram FEX 192 for enhanced vPC on the right Nexus parent
5548-Top(config)# fex 192
5548-Top(config-fex)# interface ethernet 1/23-24
5548-Top(config-if)# channel-group 192
5548-Top(config-if)# no shutdown
5548-Top(config-if)# interface port-channel 192
5548-Top(config-if)# switchport mode fex-fabric
5548-Top(config-if)# fex associate 192
5548-Top(config-if)# vpc 192
5548-Top(config-if)# no shutdown
6. Create the port channel between the blade server and the FEX
5548-Bottom# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
5548-Bottom#(config)# interface port-channel 201
5548-Bottom#(config-if)# switchport mode trunk
5548-Bottom#(config-if)# switchport trunk native vlan 182
5548-Bottom#(config-if)# switchport trunk allowed vlan 182-184,200

© 2014 Cisco | IBM. All rights reserved. Page 19
5548-Bottom#(config-if)# no shutdown
5548-Top# configure terminal
5548-Top#(config)# interface port-channel 201
5548-Top#(config)# switchport mode trunk
5548-Top#(config)# switchport trunk native vlan 182
5548-Top#(config)# switchport trunk allowed vlan 182-182,201
5548-Top#(config)# no shutdown
7. Add the member interfaces to the vPC port channel and permit VLAN or desired VLANs for L2 Trunk
links.
Note: If a Native VLAN besides 1 is desired ensure it is correctly defined before use.
5548-Bottom(config-if)# interface ethernet 191/1/7
5548-Bottom(config-if)# switchport mode trunk
5548-Bottom(config-if)# switchport trunk native vlan 182
5548-Bottom(config-if)# switchport trunk allowed vlan 182-184,200
5548-Bottom(config-if)# channel-group 201 mode active
5548-Bottom(config-if)# no shutdown
5548-Bottom(config-if)# interface ethernet 192/1/7
5548-Bottom(config-if)# switchport mode trunk
5548-Bottom(config-if)# switchport trunk native vlan 182
5548-Bottom(config-if)# switchport trunk allowed vlan 182-184,200
5548-Bottom(config-if)# channel-group 201 mode active
5548-Bottom(config-if)# no shutdown
8. Repeat the vlan configuration on the second Nexus parent switch for the vPC port channel and permit
VLAN or desired VLANs for L2 Trunk links.
5548-Top(config-if)# interface ethernet 191/1/7
5548-Top(config-if)# switchport mode trunk
5548-Top(config-if)# switchport trunk native vlan 182
5548-Top(config-if)# switchport trunk allowed vlan 182-184,201
5548-Top(config-if)# channel-group 201 mode active
5548-Top(config-if)# no shutdown
5548-Top(config-if)# interface ethernet 192/1/7
5548-Top(config-if)# switchport mode trunk
5548-Top(config-if)# switchport trunk native vlan 182
5548-Top(config-if)# switchport trunk allowed vlan 182-184,201
5548-Top(config-if)# channel-group 201 mode active
5548-Top(config-if)# no shutdown

© 2014 Cisco | IBM. All rights reserved. Page 20
Note: With EvPC when you configure a Port Channel from the Cisco Nexus 2000 Series to the server, do not include the
vpc x configuration under the Port Channel. vPC should be assigned automatically by the Cisco NX-OS Software. For
more information, see
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5500/sw/layer2/6x/b_5500_Layer2_Config_602N12_cha
pter_0101.html.
To verify that the vPC is formed, go to one of the Cisco Nexus switches to check the status of the server Port
Channel interface. The pair of Cisco Nexus switches is in a vPC configuration, so each has a single port in the
Port Channel. A check of the status of the Port Channel on each parent switch shows that channel group 201 is in
the “P - Up in port-channel” state on each switch. A check from the OneCommand utility will show that the status
is “Active” for each link that is up in the Port Channel.
5548-Bottom# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
20 Po20(SU) Eth NONE Eth1/9(P) Eth1/10(D)
191 Po191(SU) Eth NONE Eth1/17(P)
192 Po192(SU) Eth NONE Eth1/18(P)
193 Po193(SU) Eth NONE Eth1/19(P)
194 Po194(SD) Eth NONE Eth1/20(D)
201 Po201(SU) Eth NONE Eth191/1/1(P)
202 Po202(SU) Eth NONE Eth192/1/1(P)
5548-Bottom #
N5548-Top# show port-channel summary
show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------