Dell Force10 C150 User manual

Authors
Network Enabled Solutions Team
Jason Pearce, Manjesh Siddamurthy
Networking Enterprise Technologist
Kevin Horton
Deploying the Dell Force10 MXL into a
Cisco Nexus Network Environment
A guide with basic examples for deploying the Dell Force10 MXL
modular switch into the access layer of a Cisco Nexus network

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
2
This document is for informational purposes only and may contain typographical errors and
technical inaccuracies. The content is provided as is, without express or implied warranties of any
kind.
© 2012 Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions
in typography or photography. Dell, the Dell logo, and PowerEdge are trademarks of Dell Inc. Intel and
Xeon are registered trademarks of Intel Corporation in the U.S. and other countries. Microsoft,
Windows, and Windows Server are either trademarks or registered trademarks of Microsoft Corporation
in the United States and/or other countries. Other trademarks and trade names may be used in this
document to refer to either the entities claiming the marks and names or their products. Dell disclaims
proprietary interest in the marks and names of others.
December 2012| Rev 1.1

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
3
Contents
Introduction.............................................................................................................. 5
Content Overview ..................................................................................................... 6
Document Conventions ............................................................................................... 7
Device Introduction ................................................................................................... 8
Dell Force10 MXL Deployment ...................................................................................... 12
Initial Configuration ................................................................................................. 12
Access Port Downlink Configuration.............................................................................. 16
Trunk Port Downlink Configuration ............................................................................... 18
Link Aggregation Group Configuration ........................................................................... 20
Trunk Port Uplink Configuration with MLAG at Top of Rack ................................................. 21
Trunk Port Uplink Configuration with Per-VLAN Spanning Tree ............................................. 27
Spanning Tree Protocol Failover and Failback Observations .................................................. 31
Lab Environment ..................................................................................................... 31
Nexus MSTP and MXL MSTP Observations ........................................................................ 32
Nexus MSTP and MXL MSTP –Remainder of Test Observations .............................................. 37
Nexus MSTP and MXL MSTP –Greater than 64 Instances ...................................................... 38
Nexus Rapid PVST+ and MXL PVST Observations ............................................................... 38
Nexus Rapid PVST+ and MXL PVST –Remainder of Test Observations...................................... 42
Summary –Spanning Tree Observations ......................................................................... 42
Appendices ............................................................................................................. 43
Appendix A: Referenced Network Topology and Device Configurations ................................... 43
Appendix B: Basic Terminology ................................................................................... 54

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
4
Figures
Figure 1. Dell Force10 MXL Switch .................................................................................. 5
Figure 2. Example Network Topology .............................................................................. 7
Figure 3. MXL External Interfaces ................................................................................... 8
Figure 4. FlexIO 40GbE QSFP+ Module.............................................................................. 9
Figure 5. FlexIO 10GbE SFP+ Module ............................................................................... 9
Figure 6. FlexIO 10GBASE-T Module ................................................................................ 9
Figure 1. Topology Diagram: Top of Rack Nexus 5K using vPC ............................................... 21
Figure 2. Topology Diagram: Top of Rack Nexus 5K using PVST+ ............................................ 27
Figure 3. Topology Diagram: STP Interoperability Observations Lab ....................................... 31
Figure 4. Topology Diagram: Detailed, Full Example Network............................................... 43
Tables
Table 1. Force10 MXL External Ethernet Interface Numbering ............................................. 10
Table 2. Results –Spanning Tree Observations ................................................................ 42
Table 3. Devices and Firmware Versions Used in this Document ........................................... 44

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
5
Introduction
This document is an easy-to-use guide of recommended basic deployment practices for the Dell
Force10 MXL in the access layer of a Cisco Nexus network environment. The Dell Force 10 MXL—an I/O
module for the Dell PowerEdge M1000e chassis—is a 10 and 40 gigabit Ethernet multilayer switch
targeted at deployment in data center networks. The MXL provides 32 10GbE internal ports to server
blades in an M1000e modular chassis and has multiple 10 and 40GbE external uplink and stacking
options allowing it to serve well in many applications and environments.
Dell Force10 MXL SwitchFigure 1.
The Dell Force10 MXL 10/40GbE switch brings a new level of connectivity and value to the blade server
network switch environment. With 32 1/10GbE server facing ports, any transition from previous
architectures will be a huge step forward. The 32 server facing ports provide connectivity on any of the
available fabric slots (A, B, or C) for the M-Series blade servers with 1GbE or 10GbE KR-based network
card capability. The switch offers 1/10/40GbE connectivity on the uplinks to interface with a top of
rack switch, directly to a core switch, or directly to an iSCSI storage solution.
The MXL switch is an industry first, 40GbE capable, modular, and stackable blade switch for the
PowerEdge M1000e chassis.

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
6
Content Overview
This document is broken up topically to allow the steps and concepts of MXL switch deployment to be
addressed separately in a simple manner as detailed in Document Conventions. An introduction to the
Force10 MXL hardware and its connectivity and management options is presented in Device
Introduction. Once familiarized with the MXL switch, the out-of-box experience is continued with the
first steps of MXL deployment as presented in Initial Configuration.
Link configuration is split into coverage of downlink configuration options (on the MXL, downlinks are
generally its internal ports which connect to the server blades in a deployed M1000e chassis),
configuration of a link aggregation group (LAG)—a set of ports configured to pass traffic together and
behave as one link, and coverage of uplink configuration options (on the MXL these are generally its
external ports, which connect to administratively designated ports on top-of-rack or end-of-row
switches.
Downlink configuration for the MXL is covered for the two most common use cases: for VLAN unaware
servers see Access Port Downlink Configuration and for VLAN aware servers see Trunk Port Downlink
Configuration.
LAG configuration is then covered in Link Aggregation Group Configuration. While LAGs are sometimes
configured on downlinks (dependent on application demands) they are recommended to use with
uplinks.
Uplink configuration for the MXL will generally be done in one of two ways in a Cisco Nexus network
environment: running PVST with discreet LAGs connected to separate top-of-rack switches as covered
in Trunk Port Uplink Configuration with Per-VLAN Spanning Tree and running PVST with a single LAG
connected across two top-of-rack switches that are using a Multi-chassis LAG as covered in Trunk Port
Uplink Configuration with MLAG at Top of Rack (this uses the Cisco Nexus vPC feature).
To present the above configuration options as clearly as possible in this document, they have been
integrated into a single example network. The full details of the example network, including overall
topology and the complete configurations of the participating devices, are presented in Appendix A.
A glossary of applicable networking terms and concepts can be found in Appendix B. The definitions
listed there are used throughout this document; so as unfamiliar terms are encountered in this
document, refer to Appendix B for their definitions.
Additional materials referenced in this document and useful for configuration that is more advanced or
specific than what this document covers are detailed in Appendix C.

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
7
Document Conventions
This document is intended as a quick reference guide for use during basic deployment of a Force10 MXL
switch. As such, many details, configuration options, and specific features of the MXL are left out of
this document.
Example CLI Notation:
FTOS#configure
FTOS(conf)#hostname MXL1
MXL1(conf)#exit
MXL1#
As shown above, CLI examples are listed inside indented text blocks, use a monospace font, the user
input is emboldened, and the portions of the commands that will likely need to be adjusted for local
use are emboldened and italicized. Also note that each CLI example starts in privileged execution
mode and where needed starts and ends with commands to move between modes. Following this
practice each time is not required when entering multiple examples together (commands issued in the
same mode can generally be strung together). But, it is presented in this manner to ease following
individual examples in the document.
Example Network Topology:
Example Network TopologyFigure 2.
po20 (vpc1)
po20 (vpc1)
Nexus
N5K1
Nexus
N5K2
Force10
MXL1
Force10
MXL2
po10—po10
vPC link
po40—po40
po30—po30
po20—vpc1
As shown in the figure above, the network topology diagrams in this document are kept as simple as
possible to communicate the need-to-know information for the topic on hand. Figure 2 above shows the
example network that is explored in this document. While the example network topology does not
mirror a production environment, individual parts of it do and the example topology does illustrate
those parts more clearly. A more detailed topology diagram of this example network can be found in
Appendix A.

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
8
Device Introduction
The Dell Force10 MXL is an I/O Module for the M1000e modular server chassis that is a line-speed,
multilayer, 10 and 40Gb Ethernet switch that provides 32 internal 10GbE links—enabling full
connectivity to M420 quarter-height server blades—and flexible 10 and 40Gb Ethernet options for
external connectivity. With as much as 240 Gigabits of full-duplex external connectivity, it offers a
favorable 1.33:1 over-subscription ratio of internal to external connectivity. With spanning-tree options
of RSTP, MSTP, and PVST it can be natively integrated into most layer-2 network environments and
optionally participate at layer-3 environments using OSPF or static routing. The MXL also offers Data-
Center Bridging features enabling it to act as an FCoE transit switch (FIP snooping bridge). This
document however only covers basic layer-2 deployment.
External Interfaces—Connectivity, Cabling, and Port Numbering
MXL External InterfacesFigure 3.
IO Bay 2 IO Bay 1 Fixed QSFP+ Ports
IO Bays 2 and 1 can each use one of the FlexIO modules detailed below. The fixed QSFP+ 40Gb
Ethernet ports can be connected using Dell Force10 QSFP+ transceivers or Dell Force10 twinax cabling.
Additionally there are special breakout fiber and twinax cables that allow each 40GbE port to instead
be split into four 10GbE links that will connect with compatible 10GbE fiber transceivers or SFP+ ports
respectively.
Between IO Bay 1 and the fixed QSFP+ ports is a USB type-A port that allows directly connecting a USB
storage device to the MXL for copying files directly to and from the switch. Between the fixed QSFP+
ports and the MXL’s latch hinge are the MXL’s status indicator LEDs and a serial console interface that
uses a physical USB type-A port—a required serial console cable is provided with each Force10 MXL
switch. If the serial console cable is unavailable, the console may also be accessed via the M1000e CMC
CLI “connect” command. Access to the serial console (either via the provided cable or the CMC) is
required for manual out-of-box deployment of the MXL switch. See Management Connectivity for
further details.

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
9
FlexIO 40GbE QSFP+ Module
Figure 4.
The FlexIO 40GbE QSFP+ module provides two 40Gb Ethernet ports that can be connected using Dell
Force10 QSFP+ transceivers or Dell Force10 twinax cabling. Additionally Dell offers breakout fiber and
twinax cables that enable each 40GbE port to be split into four 10GbE links that will connect with
compatible 10GbE fiber transceivers or SFP+ ports . This industry leading QSFP+ module provides the
highest throughput available for each MXL FlexIO bay.
FlexIO 10GbE SFP+ ModuleFigure 5.
The FlexIO 10GbE SFP+ module provides four 10Gb Ethernet ports that can be connected using Dell
Force10 SFP+ transceivers or Dell Force10 twinax cabling.
FlexIO 10GBASE-T ModuleFigure 6.
The FlexIO 10GBASE-T module provides four 10Gb Ethernet RJ45 ports that can be connected to other
10GBASE-T devices using standard Cat6A cabling.

Deploying the Dell Force10 MXL into a Cisco Nexus Network Environment
10
Due to the availability of FlexIO modules with varying port and link count, the external interface
numbering on the MXL can be difficult to understand. Table 1 details the external interface numbering
in various modular configurations of the MXL switch. This numbering follows the potential10GbE links
skipping numbers where available link count is less than the potential. Both the Ethernet interfaces
and IO Bays are numbered from bottom to top when the MXL switch is installed in an M1000e modular
chassis.
Force10 MXL External Ethernet Interface NumberingTable 1.
Module Type
Link Speed
QSFP+
40GbE
QSFP+ (breakout cables)
10GbE
SFP+ / 10GBASE-T
10GbE
IO Bay 2
53
56
n/a
55
54
53
49
52
52
51
51
50
50
49
49
IO Bay 1
45
48
n/a
47
46
45
41
44
44
43
43
42
42
41
41
Fixed QSFP+ Ports
37
40
n/a
39
38
37
33
36
35
34
33