Dell Networking S5000 User manual

Humair Ahmed
Dell Technical Marketing –Data Center Networking
August 2013
Dell Networking S5000:
Deployment of a Converged Infrastructure
with FCoE
Deployment/Configuration Guide

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
2 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
This document is for informational purposes only and may contain typographical errors. The
content is provided as is, without express or implied warranties of any kind.
© 2013 Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions
in typography or photography. Dell, the Dell logo, and PowerEdge are trademarks of Dell Inc. Intel and
Xeon are registered trademarks of Intel Corporation in the U.S. and other countries. Microsoft,
Windows, and Windows Server are either trademarks or registered trademarks of Microsoft
Corporation in the United States and/or other countries. Other trademarks and trade names may be
used in this document to refer to either the entities claiming the marks and names or their products.
Dell disclaims proprietary interest in the marks and names of others.
August 2013 | Rev 3.5

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
3 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
Contents
Overview................................................................................................................................................................. 4
A: Converged Network Solution - Dell PowerEdge Server, Dell Compellent storage array, and Dell
S5000 as NPIV Proxy Gateway...................................................................................................................... 4
B: Converged Network Solution - Dell PowerEdge Server, Dell PowerVault storage array, and Dell
S5000 as NPIV Proxy Gateway.................................................................................................................... 33
C: Using Dell S4810 or Dell MXL/IOA Blade switch as a FIP-snooping Bridge........................................ 57
D: FCoE CNA adapter configuration specifics ............................................................................................... 65
Broadcom BCM57810S................................................................................................................................. 65
Creating a NIC Team .................................................................................................................................68
Dell QLogic QLE8262.................................................................................................................................... 74
Creating a NIC Team. ................................................................................................................................ 76

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
4 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
Overview
In the “Dell Networking S5000: The Building Blocks of Unified Fabric and LAN/SAN Convergence”
whitepaper we demonstrated and explained the movement from a traditional non-converged
LAN/SAN network to a converged LAN/SAN infrastructure and how the Dell S5000 switch is an ideal
solution for this transition. In addition, we covered the many benefits of moving to a converged
infrastructure such as less maintenance and considerable cost savings. The Dell S5000 converged
switch with its unique modular design allows end users to migrate to a converged solution and
increase port count at their own pace without replacing the entire switch. This benefit is unmatched in
the industry. In this whitepaper we cover detailed Dell S5000 topology and configuration examples.
A: Converged Network Solution - Dell PowerEdge Server, Dell
Compellent storage array, and Dell S5000 as NPIV Proxy
Gateway
We will first demonstrate a non-converged setup and then add the Dell S5000 to the picture. This will
allow us to see how the connections and configuration change from a traditional non-converged
environment to a converged environment with the introduction of the Dell S5000 switch. You’ll be
surprised how easy the setup is and how the backend LAN and SAN can remain untouched.
The traditional LAN/SAN non-converged setup example is shown below in Figure 1. As you can see, a
Dell PowerEdge R720 server with a 2-port FC HBA is used to connect to two FC switches which are
then connected to a Dell Compellent storage array composed of two SC8000 controllers and one
SC220 enclosure. Each FC port is connecting to a different fabric. Windows Server 2008 R2 Enterprise
is installed on the server.
In the below setup, the LAN side will be the usual setup with an active/standby configuration up to
separate ToR Dell S4810 switches which have VLT employed up to the core Z9000 switches.
For the below diagram I’ll focus in on the SAN configuration.

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
5 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
Figure 1: Traditional LAN/SAN non-converged network
The Dell Compellent Storage Center controllers are used to support various I/O adapters including FC,
iSCSI, FCoE, and SAS. A Dell Compellent Storage Center consists of one or two controllers, FC
switches, and one or more enclosures. In the above example, two Compellent SC8000 controllers,
one Compellent SC220 enclosure, two FC switches, and one 4-port FC HBA card on each Compellent
controller is used for the SAN network. The FC switches provide robust connectivity to servers,
allowing for the use of multiple controllers and redundant transport paths.
SAS enclosures hold disks for data storage and connect to the controllers through back-end ports via
SAS cables; you can see how the SC220 enclosure and controllers are cabled together in Figure 1
above.
To keep the diagram uncluttered yet detailed, the only connections not shown are the ‘eth0’ ports on
each controller connecting to the management network and the ‘eth1’ port on each controller
connecting to the ‘eth1’ port on the other controller. The ‘eth0’ connection supports system login and
access for the software. It’s used to send emails, alerts, SNMP traps, and Phone Home data. The ‘eth1’
connection is used for dedicated Inter-Process Communication (IPC) between controllers in a dual-
controller Storage Center. There is no default gateway for ‘eth1’and it does not need to be set. See the
“CT-SC040 and SC8000 Connectivity Guide” and “Compellent Storage Center System Setup Guide” to
get started on cabling and configuring your Compellent storage array.
In this example setup, two SC8000 controllers and one SC220 disk enclosure have been cabled
together. There are two paths available from the server to the FC switches and four paths available
from each FC switch to the Compellent storage array.
Compellent SC8000 Load Balancing Policy Options:
The Compellent SC8000 controller uses Microsoft Multipath I/O (MPIO) for load balancing over ports.
Microsoft MPIO is a framework that allows administrators to configure load balancing and failover

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
6 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
processes for FC and iSCSI connected storage devices. You can configure load balancing to use up to
32 independent paths from the connected storage devices. The MPIO framework uses Device Specific
Modules (DSM) to allow path configuration. For Windows Server 2008 and above, Microsoft provides a
built-in generic Microsoft DSM (MSDSM) and it should be used. For Windows Server 2003 only, Dell
Compellent provides a DSM.
A load balance policy is used to determine which path is used to process I/O. Once the Compellent
volume has been created and mapped accordingly as will be demonstrated shortly, to see the selected
MPIO policy in Windows Server 2008 R2 Enterprise navigate to ‘Start->Administrative Tools-
>Computer Management’. On the left-hand pane navigate to ‘Computer Management->Storage->Disk
Management’ and right click the disk created on the Compellent storage array and select ‘Properties’.
Next, select the ‘Hardware’ tab, click the ‘Properties’ button at the bottom right, and select the ‘MPIO’
tab. Figure 2 below displays what you should see. Note that the default will be “Round Robin”.
The Dell Compellent SC8000 will only support ‘Round Robin’ or ‘Fail Over Only’ MPIO policies. The
other policies require Asymmetric Logical Unit Access (ALUA) support on the array and additional
support through a DSM.
Figure 2: Checking MPIO settings in Windows Server 2008 R2 Enterprise
Additionally, there are two IO connection options available with the Dell Compellent Storage Center
that allow multiple paths to be presented to the servers: ‘Legacy Ports’and ‘Virtual Ports’. You will be
asked which one you would like to use when initially setting-up the Compellent Storage Center and

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
7 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
configuring the FC IO cards. See the “Storage Center 6.2 System Setup Guide” for more information
on initial setup of the Dell Compellent Storage Center.
In legacy mode, front-end IO ports (in this case FC ports) are broken into primary and reserve ports
based on a fault domain. The reserve port is in a standby mode until a primary port fails over to the
reserve port. In terms of MPIO, this requires twice the IO ports to enable multiple paths. For
redundancy, a primary port connects to one controller, and the reserved port in that fault domain
connects to the other controller. While this is a highly robust failover solution, it requires a large
number of ports.
Dell Compellent introduced virtual ports in Storage Center 5.0. Virtual ports allow all front-end IO
ports to be virtualized. All FC ports can be used at the same time for load balancing as well as failover
to another port. Although a virtual disk can still only be written to from the controller that owns the
disk, virtual ports allow for better performance in terms of failover as the virtual connection can simply
be moved to another physical port in the same fault domain. To use virtual ports all FC switches and
HBAs must support N_Port ID Virtualization (NPIV). See the “Dell Compellent Storage Center Microsoft
Multipath IO (MPIO) Best Practices Guide” for more information on multipathing with Microsoft
Windows 2008 R2 Enterprise.
The two FC switches I am using are Brocade 6505s and the zoning configurations are below. The
WWPNs starting with ‘10’ are the FC HBA WWPNs and the other WWPNs are for the Compellent
storage array.
Figure 3: Zoning for fabric A FC switch
Figure 4: Zoning for fabric B FC switch
> zonecreate
financeServer1_p1_test,"10:00:8c:7c:ff:30:7d:28;50:00:d3:10:00:ed:b2:3d;50:00:d3:10:00:ed:b2:43;
50:00:d3:10:00:ed:b2:3b;50:00:d3:10:00:ed:b2:41"
> cfgcreate zoneCfg_test,"financeServer1_p1_test"
> cfgenable zoneCfg_test
> cfgsave
> zonecreate
financeServer1_p2_test,"10:00:8c:7c:ff:30:7d:29;50:00:d3:10:00:ed:b2:3c;50:00:d3:10:00:ed:b2:42;
50:00:d3:10:00:ed:b2:3a;50:00:d3:10:00:ed:b2:40"
> cfgcreate zoneCfg_test,"financeServer1_p2_test"
> cfgenable zoneCfg_test
> cfgsave

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
8 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
During initial configuration of the Compellent Storage Center, we created a disk pool labeled “Pool_1”
consisting of seven 300 GB drives. The total disk space is 1.64 TB; this can be seen in the screen shot
of the Storage Center System Manager GUI as shown below in Figure 5.
Figure 5: Storage Center System Manager GUI displays disk pool “Pool_1” with 1.64 TB free space
Since we have two fabrics, fabric A and fabric B, we create two fault domains. Domain 1 is already
created by default and all the FC ports are currently in domain 1. To create another domain click
‘Storage Management’ on the top left of the webpage and then select ‘System->Setup->Configure
Local Ports’. Next, click the ‘Edit Fault Domains’ button at the bottom right of the dialog box. On the
next dialog box click the ‘Create Fault Domain’ button on the lower right of the dialog box. In the
‘Name’ field type a name for the new domain. In this case, we used “Domain 2”. Make sure ‘FC’ is
selected in the ‘Type’ field and click ‘Continue’. Figure 6 below shows that we have already created the
second domain.
Figure 6: Creating an additional Fault Domain on Compellent Storage Array
Now we can navigate back to the ‘Configure Local Ports’ dialog and select the appropriate Domain to
put each port in. Each fabric should be in its own Domain; we put all ports going to fabric A in Domain
1 and all ports going to fabric B in Domain 2 as shown below.

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
9 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
Figure 7: Assigning ports on Compellent Storage to respective Fault Domains
If you get a warning that paths are not balanced, navigate to the left-hand pane, right click
‘Controllers’ and select ‘Rebalance Local Ports’.
Next, a server object needs to be created and the respective FC ports have to be selected to be used
by the server object. This can be accomplished by right clicking ‘Servers’ on the left pane and selecting
‘Create Server’. In Figure 8 below, you can see a server object named “Finance_Server” was created
that includes both of the FC ports on the FC HBA card.
Figure 8: Added Dell PowerEdge Server HBAs to ‘Server Object’ on Dell Compellent Storage Array
The next step is to enable mulipathing on Windows Server 2008 R2 Enterprise. Navigate to
‘Start->Administrative Tools->Server Manager->Features->Add Features’and select ‘Multipath I/O’.
You can see in Figure 9 below that we have already installed the ‘Multipath I/O’ feature.

Dell Networking S5000: Deployment of a Converged Infrastructure with FCoE
10 Deployment of a Converged Infrastructure with FCoE
Deployment/Configuration Guide
Figure 9: Installing Windows Server 2008 R2 Enterprise Multipath I/O feature
Now navigate to ‘Start->Control Panel->MPIO’and click the ‘Add’ button. When prompted for a
‘Device Hardware ID’, input “COMPELNTCompellent Vol” and click the ‘OK’ button. The system will
need to be restarted for the changes to take effect. Figure 10 displays the “COMPELNTCompellent Vol”
text that you should see on the MPIO Devices tab in ‘MPIO Properties’once the system is brought
back up.