Dell PowerEdge M IO Aggregator User manual

Dell PowerEdge M I/O
Aggregator
Getting Started Guide
Regulatory Model: PEMIOAggregatr

Notes, Cautions, and Warnings
NOTE: A NOTE indicates important information that helps you make better use of
your computer.
CAUTION: A CAUTION indicates potential damage to hardware or loss of data if
instructions are not followed.
WARNING: A WARNING indicates a potential for property damage, personal
injury, or death.
____________________
Information in this publication is subject to change without notice.
© 2012 Dell Inc. All rights reserved.
Reproduction of these materials in any manner whatsoever without the written permission of Dell Inc.
is strictly forbidden.
Trademarks used in this text: Dell™, the DELL logo, Dell Precision™, OptiPlex™, Latitude™,
PowerEdge™, PowerVault™, PowerConnect™, OpenManage™, EqualLogic™, KACE™,
FlexAddress™ and Vostro™ are trademarks of Dell Inc. Intel
®
, Pentium
®
, Xeon
®
, Core™ and
Celeron
®
are registered trademarks of Intel Corporation in the U.S. and other countries. AMD
®
is a
registered trademark and AMD Opteron™, AMD Phenom™, and AMD Sempron™ are trademarks
of Advanced Micro Devices, Inc. Microsoft®, Windows®, Windows Server®, MS-DOS®and
Windows Vista®are either trademarks or registered trademarks of Microsoft Corporation in the United
States and/or other countries. Red Hat Enterprise Linux®and Enterprise Linux®are registered
trademarks of Red Hat, Inc. in the United States and/or other countries. Novell®is a registered
trademark and SUSE ™ is a trademark of Novell Inc. in the United States and other countries. Oracle®
is a registered trademark of Oracle Corporation and/or its affiliates. Citrix®, Xen®, XenServer®and
XenMotion®are either registered trademarks or trademarks of Citrix Systems, Inc. in the United States
and/or other countries. VMware®, Virtual SMP®, vMotion®, vCenter®, and vSphere®are registered
trademarks or trademarks of VMWare, Inc. in the United States or other countries.
Other trademarks and trade names may be used in this publication to refer to either the entities claiming
the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and
trade names other than its own.
Regulatory Model PEMIOAggregatr
August 2012 P/N KGCK9 Rev. A00

Contents 3
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Product Description . . . . . . . . . . . . . . . . . 7
3 Hardware Overview. . . . . . . . . . . . . . . . . . 8
Internal Ports. . . . . . . . . . . . . . . . . . . . . . . . 8
Front Panel . . . . . . . . . . . . . . . . . . . . . . . . . 8
Base Module with 40GbE Ports . . . . . . . . . . . . . . 8
FlexIOTM Modules with Uplink Ports . . . . . . . . . . . 9
Port Numbering . . . . . . . . . . . . . . . . . . . . . 11
USB Console Port . . . . . . . . . . . . . . . . . . . . 12
System and Port LEDs . . . . . . . . . . . . . . . . . . 13
Installation . . . . . . . . . . . . . . . . . . . . . . . . 15
Site Preparation . . . . . . . . . . . . . . . . . . 15
Unpacking the Aggregator . . . . . . . . . . . . . . . 16
Package Contents . . . . . . . . . . . . . . . . . 16
Unpacking Steps . . . . . . . . . . . . . . . . . . 16

4Contents
4 Installing and Configuring the Aggregator 16
Installing the Aggregator in a PowerEdge Chassis . . . 18
Connecting a Console Terminal . . . . . . . . . . . . . 20
Invoking the X-Loader and U-Boot CLIs . . . . . . . . . 21
Reconfiguring Default Settings . . . . . . . . . . . . . 21
Before You Start. . . . . . . . . . . . . . . . . . . 22
Reconfiguration Using the CMC Interface . . . . . 22
Reconfiguration Using the CLI . . . . . . . . . . . 22
Aggregator Auto-Configuration . . . . . . . . . . . . . 25
DCB Support . . . . . . . . . . . . . . . . . . . . 26
FCoE Connectivity . . . . . . . . . . . . . . . . . . 26
iSCSI Operation . . . . . . . . . . . . . . . . . . . 26
Link Aggregation . . . . . . . . . . . . . . . . . . 27
Link Tracking . . . . . . . . . . . . . . . . . . . . 27
Configuring VLANs . . . . . . . . . . . . . . . . . . . . 27
Uplink LAG . . . . . . . . . . . . . . . . . . . . . 28
Server-Facing LAGs. . . . . . . . . . . . . . . . . 28
Stacking Mode . . . . . . . . . . . . . . . . . . . 28
5 Assembling a Switch Stack . . . . . . . . . . 29
Cabling the Switch Stack . . . . . . . . . . . . . . . . 29
Configuring and Bringing Up a Stack . . . . . . . . . . 31
Managing a Stack . . . . . . . . . . . . . . . . . . . . 32
Master and Member Switches . . . . . . . . . . . 32
Stack Startup . . . . . . . . . . . . . . . . . . . . 32

Contents 5
6 Next Steps . . . . . . . . . . . . . . . . . . . . . . . . 34
7 Technical Specifications . . . . . . . . . . . . . 35
Chassis Physical Design . . . . . . . . . . . . . . . . 35
Environmental Parameters . . . . . . . . . . . . . . . 35
Power Requirements . . . . . . . . . . . . . . . . . . 36
IEEE Standards . . . . . . . . . . . . . . . . . . . . . . 36

6Contents

Introduction 7
Introduction
This document provides basic information about the Dell PowerEdge M I/O
Aggregator, including how to install the switch in the Dell PowerEdge
M1000e Enclosure and perform the initial configuration.
For more detailed information about any
of the basic installation steps , refer to
the
Dell PowerEdge M1000e Enclosure Owner's Manual on the Dell Support
website at http://support.dell.com/manuals.
Product Description
The I/O Aggregator is a zero-touch, layer 2 switch blade with two fixed 40GbE
ports on the base module and support for two optional plug-in modules. The
Aggregator operates in a Dell PowerEdge M1000e chassis, which can support
up to 32 servers and six I/O Aggregator blade switches.
The Aggregator runs the Dell Force10 Operating System (FTOS) and auto-
configures as an unmanaged switch with bridging and multiplexing
functionality, which transmits traffic as follows:
• GbE/10GbE connections through the midplane to up to 32 server ports in
the chassis
• 10GbE connections through uplink ports to top-of-rack (ToR) switches
• 40GbE connections through stacking ports to other I/O Aggregators in a
switch stack
In a data center network, the Aggregator functions as a flat layer 2 pass-
through switch that interoperates with Dell and third-party ToR devices.
The switch supports
data center bridging (DCB) features, and provides
seamless connectivity for a Fiber Channel over Ethernet (FCoE) server and
iSCSI storage data traffic.
In its default operational mode, the Aggregator requires no additional
configuration; network connectivity for servers in an M1000e chassis is
enabled. By simplifying switch configuration and management, the
Aggregator reduces complexity in a data center network.

8Hardware Overview
Hardware Overview
This section contains information about device characteristics and modular
hardware configurations for the Aggregator.
Internal Ports
The Aggregator provides thirty-two 1/10-Gigabit Ethernet internal ports. The
internal ports are connected to server blades through the M1000e chassis
midplane. Each port can operate in either 1GbE or 10GbE mode.
The Aggregator also provides an internal Ethernet interface—the out-of-band
(OOB) interface—which is dedicated to switch management. Traffic on this
port is segregated from operational network traffic on the switch ports and
cannot be switched or routed to the operational network.
Front Panel
Figure 1-1 shows the Aggregator front panel:
Figure 1-1. I/O Aggregator Front Panel
Base Module with 40GbE Ports
The Aggregator provides two fixed 40-Gigabit Ethernet ports on the base
module. By default, these ports operate in 4x10GbE mode with breakout cables
and support up to eight 10GbE uplinks. You can also use the base-module ports
as 40GbE links for stacking.
Expansion Slot 1
40GbE QSFP+ Ports
USB Console Port
Expansion Slot 0
USB Storage Port Activity and Link Status LEDs

Hardware Overview 9
FlexIOTM Modules with Uplink Ports
On the Aggregator, FlexIO plug-in modules (Figure 1-2) operate only as
uplinks; stacking is not supported. The following modules are supported in
the two expansion slots:
• 4-Port 10-Gigabit Ethernet module using SFP+ optics or direct-attach
cables (1m, 3m, or 5m DAC)
4-Port 10GBASE-T module using RJ-45 connector (copper) cables
NOTE: Only one 4-Port 10GBASE-T module is supported in an Aggregator.
NOTE: The 10M speed is not supported on the 4-Port 10GBASE-T module. Only
100M, 1G, and 10G speeds are supported.
• 2-Port 40-Gigabit Ethernet QSFP+ (SR or direct attach only) module only
for 10GbE SFP+ connections using 4x10GbE breakout cables.
Figure 1-2. I/O Aggregator: FlexIO Modules
4-Port 10GbE SFP+ Module
4-Port 10GBASE-T Module (100Mb/1GbE/10GbE)
2-Port 40GbE QSFP+ Module

10 Hardware Overview
Use the FlexIO modules in an Aggregator that operates in standalone or
stacking mode:
• In standalone mode, you can install any of the supported FlexIO modules
in two expansion slots. All ports operate at 10GbE to provide increased
bandwidth of up to 160Gbps for uplinks. Stacking is not supported.
• In stacking mode, only the 40GbE ports on the base module are used for
stacking. FlexIO modules are required for uplink connections.
Usage Notes:
• After you insert any FlexIO module into an empty slot, you must reload
the Aggregator for the module to be operational.
• You can hot swap FlexIO modules of the same type without having to
reboot the Aggregator. If you insert a different type of plug-in module, you
must reload the Aggregator for the new module to be recognized.
• All uplink ports are in the same 10GbE link aggregation group (LAG).
Tips for Aggregator standalone operation:
• If you use 16 servers in an M1000e and you want a 2:1 oversubscription,
you only need to use the base module ports in 8x10GbE mode.
• If you use 32 servers in an M1000e and you want a 2:1 oversubscription,
use the base-module ports in 8x10GbE mode and either a 2-Port 40GbE
QSFP+ module operating in 8x10GbE mode or two 4-Port 10GbE SFP+
modules. Use breakout cables with the 2-Port QSFP+ module.
Tips for Aggregator stacking operation:
• To provide uplink connectivity for the stack, you must use at least one
FlexIO module in either expansion slot on any Aggregator in the stack.
• To optimize redundancy and high availability, Dell Force10 recommends
that you use at least one FlexIO module for uplinks on each Aggregator in
a stack. If you do not use a FlexIO module on each stacked Aggregator for
uplink connectivity, in the event of a stack split, the server-facing ports on
Aggregators without an uplink connection are brought down.
All FlexIO modules, transceivers, and direct attach cables are sold separately.

Hardware Overview 11
Port Numbering
When installed in a PowerEdge M1000e Enclosure, Aggregator ports are
numbered 1 to 56. Ports 1 to 32 are internal server-facing ports. Ports 33 to 56
are external ports numbered from the bottom to the top of the switch as
follows:
• The 40GbE base-module ports operate:
• In standalone 4x10GbE mode (default); ports are numbered 33 to 36
and 37 to 40.
• In stacking mode; ports are numbered 33 and 37, and cannot be used
for uplinks.
• The 2-Port 40-GbE QSFP+ module operates only in 4x10GbE mode:
• In the bottom expansion slot, ports are numbered 41 to 44 and 45 to
48.
• In the top expansion slot, ports are numbered 49 to 52 and 53 to 56.
• 4-Port 10-GbE SFP+ and 4-Port 10GBASE-T modules operate only
in
4x10GbE mode:
• In the bottom expansion slot, ports are numbered 41 to 44.
• In the top expansion slot, ports are numbered 49 to 52.
To configure a port, you must specify the slot (0-5; default: 0) and port
number (1 to 56) in the interface port-type slot/port command, where slot is
the unit number of the Aggregator displayed in the show system brief
command; for example:
FTOS(conf)# interface tengigabitethernet 0/4

12 Hardware Overview
USB Console Port
Figure 1-3. USB Ports on Front Panel
Use the lower USB console port (Figure 1-3) to manage the switch through an
RS-232 serial interface. This port provides a direct connection to the switch
and allows you to access the command-line interface (CLI) from a console
terminal connected to the port through the provided serial cable (with USB
type-A to female DB-9 connectors).
The console port supports asynchronous data of eight data bits, one stop bit,
no parity bit, and no flow control. The default baud rate is 9600 bps.
The upper USB port (Figure 1-3) functions as an external flash drive that you
can use to store configuration files, scripts, and so on.
USB Console Port
USB Storage Port

Hardware Overview 13
System and Port LEDs
The front panel of the Aggregator contains light emitting diodes (LEDs) that
provide information about the status of the switch (Figure 1-4).
Figure 1-4. System LEDs on Front Panel
Table 1-1 describes system LED conditions.
Table 1-1. System LEDs
System LED Color Meaning
Power Green Power is being supplied to the switch.
Off The switch does not have power.
Status
Blue The switch is operating normally as a standalone
switch or as a stack master.
Off The switch is not the stack master.
Amber A fault has occurred, or the switch is booting.
System Status LED
System Power LED
Standalone 4x10GbE
Uplink Ports or
40GbE Stacking Ports

14 Hardware Overview
Plug-in modules and the base module (Figure 1-1) also contains LEDs that
provide information about the link status and traffic activity on a port.
Figure 1-5. Port LEDs
Table 1-2 describes the LED status of 10GbE BASE-T, 10GbE SFP+, and
40GbE QSFP+ ports used in stacking mode on the base module.
Table 1-2. Port LED Status
Port LED Color Meaning
Link Status
Off The port is down.
Green The port is up and can transmit traffic at maximum
speed:
A QSFP+ port can transmit at 40G.
An SFP+ port can transmit at 10G.
A BASE-T port can transmit at 10G.
Yellow The port is up and is transmitting traffic at lower than
maximum speed:
A 40GbE QSFP+ port is transmitting at 10G.
A 10GbE SFP+ or 10GbE BASE-T port is
transmitting at 1G or 100M.
Activity
Off No traffic is being transmitted or received on the port.
Blinking
Green
Traffic is being transmitted or received on the port.
Link Status
Activity
Link Status
Activity
Link Status
Activity

Hardware Overview 15
Table 1-3 describes the LED status of a 40GbE QSFP+ port that is split into
four 10GbE SFP+ ports using a
4x10GbE breakout cable .
Installation
Site Preparation
Before installing the switch or switches, ensure that the chosen installation
location meets the following site requirements:
• Clearance — There is adequate front and rear clearance for operator
access. Allow clearance for cabling, power connections, and ventilation.
• Cabling — The cabling is routed to avoid sources of electrical noise such
as radio transmitters, broadcast amplifiers, power lines, and fluorescent
lighting fixtures.
• Ambient Temperature — The ambient switch operating temperature
range is 10° to 40ºC (50° to 104ºF).
NOTE: Decrease the maximum temperature by 1°C (1.8°F) per 300 m (985 ft.)
above 900 m (2955 ft.).
• Relative Humidity — The operating relative humidity is 8% to 85% (non-
condensing) with a maximum humidity gradation of 10% per hour.
Table 1-3. LED Status of 4x10GbE Port
Port LED Color Meaning
Link Status
Off All four 10GbE ports on a breakout cable are down.
Yellow At least one of the four 10GbE ports on a breakout
cable is up.
Activity
Off No traffic is being transmitted on any 10GbE port on
the breakout cable.
Blinking
Green
Traffic is being transmitted or received on at least one
of the 10GbE ports on the breakout cable.

16 Installing and Configuring the Aggregator
Unpacking the Aggregator
Package Contents
When unpacking each switch, ensure that the following items are included:
• One PowerEdge M I/O Aggregator switch blade
• One USB type A-to-DB-9 female cable
• Getting Started Guide
• Safety and Regulatory Information
• Warranty and Support Information
• Software License Agreement
Unpacking Steps
NOTE: Before unpacking the switch, inspect the container and immediately
report any evidence of damage.
1 Place the container on a clean, flat surface and cut all straps securing the
container.
2 Open the container or remove the container top.
3 Carefully remove the switch from the container and place it on a secure
and clean surface.
4 Remove all packing material.
5 Inspect the product and accessories for damage.
Installing and Configuring the
Aggregator
After you unpack the Aggregator, refer to the flow chart in Figure 1-6 for an
overview of the steps you must follow to install the blade and perform the
initial configuration.

Installing and Configuring the Aggregator 17
Figure 1-6. I/O Aggregator: Installation and Configuration Flow Chart
Insert Switch Blade
with FlexIO Modules
and
Power On
Connect Console
Load Image
from Flash to RAM
X-Loader CLI
Run Memory Tests/
Configure Switch
Settings
Reboot
Yes
No Do not press a key
(Optional) Assemble a
Switch Stack
(Optional) Reconfigure
Default Settings
Press
any key to
enter X-Loader
or U-Boot
menu?
U-Boot CLI
No Do not press a key
Yes

18 Installing and Configuring the Aggregator
Installing the Aggregator in a PowerEdge Chassis
After you unpack the Aggregator, install FlexIO modules in the base module
and slide the Aggregator into one of the open I/O module slots in the back of
a PowerEdge M1000e chassis.
NOTE: If you plug a FlexIO module into an Aggregator that is already powered
on and inserted in an M1000e chassis, you must reboot the Aggregator for the plug-
in module to be activated.
The M1000e chassis is a 10U rack-mountable blade chassis that holds:
• Server blades: Eight full-height or 16 half-height, or 32 quarter-height
blades
• Switch blades: Six I/O modules and two integrated chassis management
controllers
Server blades are installed in the front of the chassis (Figure 1-7); I/O
modules are installed in the back of the chassis (Figure 1-8).
Figure 1-7. PowerEdge M1000e: Front View with Server Blades
16 half-height
server blades

Installing and Configuring the Aggregator 19
Figure 1-8. PowerEdge M1000e: Back View with Six Aggregator Blades
After you slide the Aggregator in so that the connectors on the back of the
blade are inserted into the chassis midplane, the switch receives power from
the chassis and automatically powers on. The chassis management controller
(CMC) in the chassis validates that the switch blade is a supported I/O
module before powering it on.
When the Aggregator powers on, the Boot loader loads the image in the local
flash. The image initializes the hardware and brings the switch up in
operational mode.
Six I/O
Aggregator
blades
CMC 1
CMC 2

20 Installing and Configuring the Aggregator
Connecting a Console Terminal
After the I/O Aggregator powers on, complete all external cabling connections
and connect a terminal to the blade to configure the switch.
To monitor and configure the Aggregator using the serial console, use the
USB console port on the front panel of the switch ( Figure 1-3) to connect it
to a VT100 terminal or to a computer running VT100 terminal emulation
software. The console port is implemented as a data terminal equipment
(DTE) connector.
The following equipment is required to use the console port:
• VT100-compatible terminal or a desktop or a portable computer with a
serial port running VT100 terminal emulation software, such as Microsoft
HyperTerminal.
• A serial cable (provided) with a USB type-A connector for the console port
and DB-9 connector for the terminal.
Perform the following tasks to connect a terminal to the switch console port:
1
Connect the DB-9 connector on the serial cable to the terminal or
computer running VT100 terminal emulation software.
2 Configure the terminal emulation software as follows:
a Select the appropriate serial port (for example, COM 1) to connect to
the console.
b Set the data rate to 9600 baud.
c Set the data format to 8 data bits, 1 stop bit, and no parity.
d Set the flow control to none.
e Set the terminal emulation mode to VT100.
f Select Terminal keys for Function, Arrow, and Ctrl keys. Ensure that
the setting is for Terminal keys (not Microsoft Windows keys).
Connect the USB connector on the cable directly to the switch console port.
The console port on the Aggregator is located below the fixed 40GbE ports
(Figure 1-3).