HP ProLiant BL620c G7 Installation and operating instructions

HP StorageWorks Fibre Channel Over
Ethernet Application Note
Abstract
This document describes support for HP FCoE products. It provides an overview of the product features, functionality,
and configuration rules.
Part Number: AA-RWQ5J-TE
Eleventh edition: January 2011

Legal and notice information
© Copyright 2009-2011 Hewlett-Packard Development Company, L.P.
Intel® is a trademark of Intel Corporation in the U.S. and other countries.
Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation.

Description
HP offers multiple FCoE solutions, from fabric-edge solutions using FCoE CNAs and CN switches
integrated with existing FC target environments, to full FCoE end-to-end solutions using CNAs, CN
switches and FCoE storage targets.
The HP FCoE fabric-edge solution deploys FCoE technology in existing Ethernet and FC environments
with the benefit of convergence at the server and fabric edge using CNAs and CN switches. This
solution provides this benefit in addition to investment protection by allowing FCoE to be integrated
with existing HP Fibre Channel SANs.
The HP end-to-end FCoE solution for EVA storage systems provides an FCoE technology implementation
from server to storage. This solution consists of FCoE CNAs, CN switches, and FCoE storage targets
using the MPX200 Multifunction Router integrated with the EVA family of storage systems. Two FCoE
end-to-end configurations are supported:
•Direct-connect—Provides 10-GbE FCoE and optionally, iSCSI connectivity for up to two EVA storage
systems without requiring any Fibre Channel switches.
•Fabric-connect—Provides additional connectivity, allowing for up to four EVA storage systems using
a fabric connection between the MPX200 and EVAs.
The FCoE product set consists of the following:
•HP B-series and C-series CN switches—Provide a bridge function from the CN to separate Ethernet
and FC fabrics, or support end-to-end FCoE when used with CNAs and the MPX200 with FCoE
target functionality
•HP CN1000E and CN1000Q CNAs—Converge Ethernet and FC technologies in the server over
10-GbE links to converged network switches
•HP MPX200 Multifunction Router FCoE—Provides FCoE target functionality when integrated with
EVA storage systems (for more information, see the HP StorageWorks SAN Design Reference
Guide and the HP StorageWorks MPX200 Multifunction Router User Guide).
Benefits
This section describes some of the benefits of FCoE technology. The convergence of Ethernet and FC
provides the same level of connectivity as previous technologies, but requires:
•50% fewer switches in each server rack—Only two CN ToR switches, compared with four (two
network and two FC) switches per rack with separate Ethernet and FC switches
•50% fewer adapters per server
•75% fewer cable connections
HP StorageWorks Fibre Channel Over Ethernet Application Note 3

Figure 1 illustrates a rack with 16 servers using FCoE technology compared to a rack using separate
FC and Ethernet with the same level of connectivity. Similarly, Table 1 provides a comparison of cable
requirements for these same racks.
Figure 1 Rack cable comparison
.
Table 1 Rack cable requirements comparison
Converged FCoE network1
Separate FC and Ethernet networks
(2) 10-GbE cables
nl
(2) cables at 20 Gb per server
(4) 4-Gb FC cables
nl
(4) 1-GbE NIC cables
nl
(8) cables at 20 Gb per server
Total of 32 cables for 320 Gb of connectivityTotal of 128 cables for 320 Gb of connectivity
1Using copper cables for FCoE connections further reduces the per-port connection cost versus the cost of optical SFP+
nl
transceivers.
4

Converged network switches
The first generation of FCoE CN switches enable Ethernet and FC to coexist in a fabric and are
designed as edge switches or ToR switches. Edge/ToR switches are typically deployed in redundant
pairs installed at the top of a server rack (see Figure 1). By using FCoE CN switches, you reduce the
number of required switches by replacing separate Ethernet and FC switches with a converged
edge/ToR switch. From the edge/ToR switch, ISL connections to the EoR FC and EoR Ethernet/IP
switches provide access to the separate core layers of the FC fabric and Ethernet/IP network.
An FCoE blade that resides in a DC or DC04 SAN Director can also be used to integrate FCoE
solutions with existing or new FC fabrics.
The following HP FCoE CN switches and blades are available:
•HP StorageWorks 2408 FCoE Converged Network Switch
•HP StorageWorks DC SAN Director Switch 10/24 FCoE Blade
•HP C-series Nexus 5010 Converged Network Switch
•HP C-series Nexus 5020 Converged Network Switch
Converged network switch ports
HP FCoE CN switches have two types of physical ports:
•10-GbE ports
•4-Gb or 8-Gb FC ports
FC ports can be used for either ISL connections to an existing FC fabric, or connection directly to an
HP storage array.
Table 2 lists the number of ports for each HP FCoE CN switch.
Table 2 Number of ports per switch
Number of FC portsNumber of 10-GbE portsHP switch
8 (8 Gb/s)242408 FCoE Converged Network Switch
0 external ports
nl
32 (8 Gb/s) backplane ports
24DC SAN Director Switch 10/24 FCoE Blade
0 to 8 (4 Gb/s)
nl
0 to 6 (8 Gb/s)
20 to 26
C-series Nexus 5010 Converged Network
Switch
0 to 16 (4 Gb/s)
nl
0 to 12 (8 Gb/s)
40 to 52
C-series Nexus 5020 Converged Network
Switch
NOTE:
All FCoE CN switch models are by design CEE/IEEE DCB switches and are enabled for Ethernet by
default. In order to be used for FCoE they must be enabled to forward FC frames to the FC ports.
HP StorageWorks Fibre Channel Over Ethernet Application Note 5

HP does not support interoperability between different vendors' CN and FC switches:
•2408 FCoE Converged Network Switch operate with B-series FC switches only.
•C-series Nexus 5010/5020 Converged Network Switches operate with other C-series MDS FC
switches only.
NOTE:
The FCoE CN switch firmware update process will result in momentary loss of Ethernet and Fibre
Channel port connectivity. HP recommends implementing a dual path configuration with redundant
FCoE CN switches to minimize disruption during firmware updates.
HP StorageWorks 2408 FCoE Converged Network Switch
The 2408 FCoE Converged Network Switch is a rebranded version of the Brocade 8000 FCoE Switch
(Figure 2).
Figure 2 2408 FCoE Converged Network Switch
.
Features
•Next-generation Ethernet L2/L3 switch and next-generation FC switch merged into one product
•CEE support
•Full industry-standard implementation
•Supports FCoE and FCoE Initialization Protocol (FIP)
•24 10-GbE and 8 8-Gb/s FC ports
•Uses 1 RU of space
•Leverages existing Brocade fabric operating system (FOS)
•Supports link aggregation (LAG) on DCB ports
•Supports Brocade ISL trunking on FC ports
Considerations
•The port types are fixed. You cannot use 10-GbE ports for FC connections, and you cannot use
FC ports for 10-GbE connections.
•L3 routing features are not currently supported.
•10-GbE ports support virtual F_Ports only (virtual E_Ports are not supported).
6

HP StorageWorks DC SAN Director Switch 10/24 FCoE Blade
The HP DC SAN Director Switch 10/24 FCoE Blade is a rebranded version of the Brocade FCOE10-24
blade (Figure 3).
26608b
10-GbE ports
Figure 3 HP DC SAN Director Switch 10/24 FCoE Blade
.
Features
•24 10-GbE ports
•32 8-Gb/s FC ports on the backplane (DC and DC04 SAN Directors only)
•Hot pluggable
•Blade power and status LEDs
•Link status LEDs for each port
•FCoE switching
•CEE switching
•L2 Ethernet protocols STP/MSTP/RSTP, VLAN tagging, and link aggregation
•Standard Ethernet encapsulation
Considerations
•The DC SAN Director Switch 10/24 FCoE Blade has 24 FCoE ports and can be installed in either
a DC or DC04 SAN director.
•The DC SAN Director Switch 10/24 FCoE Blade enables server edge connectivity by connecting
CNAs directly to any of its 24 Ethernet ports.
•Storage can be connected to:
•Any other Fibre Channel blade in the same director
•Any Fibre Channel switch that is in the same fabric as the DC or DC04 SAN Director that
contains the DC SAN Director Switch 10/24 FCoE Blade
•The DC SAN Director Switch 10/24 FCoE Blade supports optical cabling and SFP+ transceivers
only.
•There are no licensing requirements for this blade.
HP StorageWorks Fibre Channel Over Ethernet Application Note 7

HP C-series Nexus 5010/5020 Converged Network Switches
HP resells the Cisco Nexus 5010 and 5020 Switches (Figure 4).
10-GbE ports
FC ports 2 6 6 51 a
Figure 4 C-series Nexus 5010 and 5020 Converged Network Switches
.
Features
•L2 access:
•IEEE DCB and FCoE support
IEEE DCB is a Cisco unified fabric product with additional proprietary features.
•NX-OS with combined features from Cisco IOS and Cisco MDS 9000 SAN-OS/NX-OS
•Cisco 5020 (2 RU):
•40 10-GbE ports
•Two optional expansion module slots
•Up to 52 10-GbE ports, or a combination of 40 10-GbE ports and 16 FC ports
•Cisco 5010 (1 RU):
•20 10-GbE ports
•One optional expansion module slot
•Up to 26 10-GbE ports, or a combination of 20 10-GbE ports and 8 FC ports
•Optional expansion modules:
•6-port 10-GbE expansion module
•6-port 8-Gb/s FC expansion module
•8-port 4-Gb/s FC expansion module
•4-port 4-Gb/s FC and 4-port 10-GbE expansion module
Converged network adapters
Servers with CNAs require fewer adapters per server and, therefore, 75% fewer cable connections
for network and FC attachments.
HP offers the HP CN1000E CNA (Figure 5) and the HP CN1000Q CNA (Figure 7). For information
about servers that support CNAs, see Server support on page 19.
8

Table 3 lists the supported HP CNAs.
Table 3 HP CNA part numbers
DescriptionHP model numberHP part number
Dual port converged network adapter
nl
(no cable or transceiver is included)
CN1000EAW520A
Dual port converged network adapter
nl
(no cable or transceiver is included)
CN1000QBS668A
HP CN1000E
Figure 5 shows an HP CN1000E CNA.
Figure 5 HP CN1000E CNA
.
HP StorageWorks Fibre Channel Over Ethernet Application Note 9

Figure 6 shows a diagram of the internal components of an HP CN1000E CNA.
Figure 6 HP CN1000E internal components
.
Features
The HP CN1000E CNA has the following features:
•Ships with half-height and full-height brackets
•Dual ports for redundancy
•Full 10-Gb/s bandwidth on both ports
•Each port can operate as a NIC and/or FCoE port
•2 SFP+ connectors
•Supports optical or copper cables
Considerations
•x8 PCI Express Gen2 card
•Requires 14.5 W of power
•1 GbE is not supported
10

CN1000Q
Figure 7 shows an HP CN1000Q CNA.
Figure 7 HP CN1000Q CNA
.
Features
The HP CN1000Q CNA has the following features:
•Ships with half-height and full-height brackets
•Dual ports for redundancy
•Full 10-Gb/s bandwidth on both ports
•Each port can operate as a NIC and/or FCoE port
•2 SFP+ connectors
•Supports optical or copper cables
Considerations
•x8 PCI Express Gen2 card
•Requires 19 W of power
•1 GbE is not supported
HP StorageWorks Fibre Channel Over Ethernet Application Note 11

Configuration rules
This section describes configuration rules for the HP FCoE solution. The rules are defined for the CNAs
and each of the FCoE CN switch series.
Figure 8 shows a converged SAN fabric with servers using CNAs only.
26647b
Server
with CNA
IP network
Server
with CNA
X-series
FCoE switch
FC attached
HP storage
Fabric A
X-series
FC switch
Server
with CNA
Server
with CNA
X-series
FCoE switch
FC attached
HP storage
10-GbE
Fabric B
X-series
FC switch
X-series
FCoE switch
X-series
FCoE switch
10-GbE/FCoE A/FCoE B connection
Fabric A Fibre Channel connection
Fabric B Fibre Channel connection
10-GbE connection
Figure 8 Converged SAN fabric configuration
.
12

Figure 9 shows a converged SAN fabric with servers using CNAs connected to a B-series CN switch
and a DC SAN Director Switch 10/24 FCoE Blade.
26648b
Server
with CNA
IP network
Server
with CNA
B-series
CN switch
FC attached
HP storage
Fabric A
B-series
FCoE
blade
Server
with CNA
Server
with CNA
FC attached
HP storage
10-GbE
Fabric B
B-series
CN switch
B-series
FCoE
blade
10-GbE/FCoE A/FCoE B connection
Fabric A Fibre Channel connection
Fabric B Fibre Channel connection
10-GbE connection
Figure 9 Converged SAN fabric with an FCoE blade
.
HP StorageWorks Fibre Channel Over Ethernet Application Note 13

Figure 10 shows a converged fabric that has servers with CNAs and servers with FC HBAs and NICs.
26649b
Server
with FC HBA
and NIC
IP network
Server
with FC HBA
and NIC
X-series
CN switch
FC attached
HP storage
Fabric A
X-series
FC switch
Server
with CNA
Server
with CNA
FC attached
HP storage
10-GbE
X-series
CN switch
Fabric B
X-series
FC switch
10-GbE/FCoE A/FCoE B connection
Fabric A Fibre Channel connection
Fabric B Fibre Channel connection
10-GbE connection
Figure 10 FCoE integrated with FC SAN fabric
.
14

Figure 11 shows an FCoE end-to-end direct-connect storage configuration using an MPX200
Multifunction Router and EVA storage systems. This provides 10-GbE FCoE and optionally, iSCSI
connectivity for up to two EVA storage systems without requiring any Fibre Channel switches.
26652b
Servers with CNAs
Servers with NICs
FCoE/IP
/iSCSI
MPX200
FCoE/iSCSI
target
EVA Storage Systems
Ethernet
Network
FCoE switches
FCoE/iSCSI
Converged
network iSCSI/IP
IP
GE1 GE2
IOIOI
FC1 FC2MGMT
10GbE4 10GbE3
HPStorageWorks
MPX200
10-1GbEBlade
MPX200MultifunctionRouter
GE1 GE2
IOIOI
FC1 FC2MGMT
10GbE4 10GbE3
HPStorageWorks
MPX200
10-1GbEBlade
MPX200MultifunctionRouter
10-GbE/FCoE A/FCoE B connection
iSCSI/IP connection
10-GbE connection
FCoE/iSCSI connection
Fabric A Fibre Channel connection
Fabric B Fibre Channel connection
Figure 11 FCoE end-to-end direct-connect EVA storage configuration
.
The configuration shown in Figure 12 provides additional connectivity, allowing for up to four EVA
storage systems using a fabric connection between the MPX200 and EVAs. In addition to FCoE and
iSCSI, Fibre Channel connected servers can access the same EVA storage systems through the Fibre
Channel fabric. This configuration provides the highest level of multi-protocol support with FCoE,
iSCSI, and Fibre Channel connectivity for up to four EVA storage systems.
HP StorageWorks Fibre Channel Over Ethernet Application Note 15

26653b
Servers with CNAs Servers with NICs
FCoE/IP
/iSCSI
MPX200
FCoE/iSCSI
target
FCoE/iSCSI
Fibre
Channel
EVA Storage Systems
Ethernet
Network
FCoE switches
Fabric B
Fabric A
Converged
network
iSCSI/IP
IP
GE1 GE2
IOIOI
FC1 FC2MGMT
10GbE4 10GbE3
HPStorageWorks
MPX200
10- 1 GbE Blade
MPX200Multifunction Router
GE1 GE2
IOIOI
FC1 FC2MGMT
10GbE4 10GbE3
HPStorageWorks
MPX200
10- 1 GbE Blade
MPX200Multifunction Router
10-GbE/FCoE A/FCoE B connection
iSCSI/IP connection
10-GbE connection
FCoE/iSCSI connection
Fabric A Fibre Channel connection
Fabric B Fibre Channel connection
Figure 12 FCoE end-to-end fabric-connect EVA storage configuration
.
16

CNA configuration rules
Table 4 HP CNA requirements
DescriptionItem
2
Number of CNAs per server
(maximum)
•SFP+ optical
•Copper
Cable types
For current firmware and driver support, see the SPOCK website at http://www.hp.com/storage/
spock. You must sign up for an HP Passport to enable access.
FCoE CN switch configuration rules
Table 5 DC SAN Director Switch 10/24 FCoE Blade and 2408 FCoE Converged Network Switch fabric
rules
DescriptionRule number
A 2408 FCoE Converged Network Switch or DC SAN Director Switch 10/24 Blade can be
a standalone switch or an edge switch in a Fibre Channel fabric. To attach the switch to an
existing Fibre Channel fabric as an edge switch, at least one Fibre Channel port on the FCoE
CN switch must be connected to a Fibre Channel switch in the fabric (E_Port). There cannot
be any other FCoE or 10-GbE CEE switches in the path to the Fibre Channel switch.
Similarly, a DC or DC04 SAN Director with a 10/24 FCoE Blade installed can be a
standalone switch, an edge switch, or a core switch in a Fibre Channel fabric. If it is an edge
switch or core switch, other switches in the fabric can be attached to any available FC port
(E_Port) on other FC blades in the director.
For FCoE E_Port connectivity, see the appropriate Fabric OS release notes for the minimum
and recommended Fibre Channel switch firmware versions and the supported Fibre Channel
switch models. HP recommends Brocade FOS 6.3.0b or later.
1
Server CNAs must be directly connected to the CEE ports on a 2408 FCoE Converged Network
Switch or DC SAN Director Switch 10/24 FCoE Blade.
2
Fibre Channel storage can be directly connected to the 2408 FCoE Converged Network
Switch (one of the eight FC F_Ports) or to any Fibre Channel switch that is part of the same
fabric as the 2408 FCoE Converged Network Switch.
With the DC SAN Director Switch 10/24 FCoE Blade, storage can be connected to any of
the other FC blades in the same director or to any Fibre Channel switch that is part of the
same fabric as the DC SAN Director that contains the DC SAN Director Switch 10/24 FCoE
Blade.
3
Supports a maximum of seven switch hops (eight switches) between a CNA and storage.4
Supports the following CEE media types:
•HP B-series hot pluggable, 10-GbE SFP+ Short Reach (SR) and Long Reach (LR) optical
transceivers
•HP B-series SFP+ 1 m, 3 m, or 5 m copper Twinax active cables (2408 FCoE Converged
Network Switch only)
5
Up to 4,000 VLANs, but only one FCoE VLAN is currently supported6
Only FPMA is currently supported.7
HP StorageWorks Fibre Channel Over Ethernet Application Note 17

DescriptionRule number
Supports existing B-series Fibre Channel rules and features except for the following:
•Advanced Zoning (port-based and QoS)
•Extended Fabrics
•Adaptive Networking with QoS
•Virtual Fabrics
•Integrated Fibre Channel Routing
•FICON
•Access Gateway
•Admin Domains
•TI Zones
•M-EOS Interop
•Nondisruptive firmware upgrade
8
A maximum of two DC SAN Director Switch 10/24 FCoE Blades per chassis are supported
in DC and DC04 SAN Directors.
9
The DC SAN Director Switch 10/24 FCoE Blade does not support Twinax copper cables
(SFP+).
10
DC and DC04 SAN Directors are supported for concurrent installation of the DC SAN Director
Switch 10/24 FCoE Blade with the FC8-16, FC8-32, FC8-48 or FC10-6 blades only.
11
Table 6 C-series FCoE Converged Network switch fabric rules
DescriptionRule number
A C-series FCoE CN switch can be a standalone switch or an edge switch in a Fibre Channel
fabric. To attach the switch to an existing Fibre Channel fabric as an edge switch, at least
one Fibre Channel port on the FCoE CN switch must be connected to a Fibre Channel switch
in the fabric (E_Port). There cannot be any other FCoE or 10-GbE IEEE DCB switches in the
path to the Fibre Channel switch.
For FCoE E_Port connectivity, the Fibre Channel switch minimum firmware version is NX-OS
4.1(3)N1(1). All C-series 4 Gb/s or 8 Gb/s switch models are supported when using the
minimum firmware version.
MDS switches running SAN-OS 3.xcan be in the Fibre Channel SAN but cannot be connected
directly to an FCoE switch.
1
Server CNAs must be connected directly to FCoE CN switch IEEE DCB ports (pre-FIP support
only).
2
Fibre Channel storage can be connected directly to the FCoE CN switch (F_Port) or to a Fibre
Channel switch (F_Port) in the same fabric as the FCoE CN switch. SAN boot is not supported
when storage systems are connected directly to FCoE Fibre Channel ports.
3
Supports a maximum of three switch hops (four switches) between a CNA and storage.4
Supports the following IEEE DCB media types:
•HP C-series hot pluggable, 10-GbE SFP+ Short Reach (SR) optical transceivers
•HP C-series SFP+ 3m or 5m copper Twinax passive cables
5
Supports up to 50 switches in a single physical fabric or VSAN6
Supports up to 40 domains per VSAN7
18

DescriptionRule number
Supports a maximum of 250 VLANs per switch8
Supports a maximum of 32 VSANs per switch9
Supports a maximum of 32 zones per virtual or physical F_Port (includes all VSANs)10
Supports a maximum of 500 zone sets per switch (includes all VSANs)11
Supports a maximum of 8,000 zone members per physical fabric (includes all VSANs)12
Supports a maximum of 8,000 zones per switch (includes all VSANs)13
Supports up to 52 virtual Fibre Channel interfaces per switch14
Server support
FCoE is supported with ProLiant G5, G6, and G7 servers.
For current server support, see the server QuickSpecs at http://www.hp.com/go/quickspecs/.
Storage system support
This section describes storage system support for access from CNA-based servers. These storage
systems can be attached to Fibre Channel switches in the fabric or connected to the Fibre Channel
ports of an FCoE CN switch.
SAN boot is not supported when storage systems are connected to the Fibre Channel ports of an
FCoE CN switch. SAN boot is supported for storage attached to the Fibre Channel ports in a SAN
Director that contains a 10/24 FCoE blade and based on the current support listed for Fibre Channel
switches.
The following storage systems are supported with HP CN1000E and CN1000Q CNAs:
•MSA2000fc G2 (MSA2312fc/MSA2324fc)
•MSA2000fc (MSA2012fc/MSA2212fc)—Supported on CN1000E only
•P2000 G3
•EVA4400
•EVA4400 with embedded switch1
•EVA4100/6100/8100
•EVA6400/8400
•XP10000/12000
•XP20000/24000
1The EVA4400 with the embedded switch is not supported for connectivity to C-series Fibre Channel
switches or Nexus Converged Network Switches.
For current storage system controller software support, see the SPOCK website at http://www.hp.com/
storage/spock. You must sign up for an HP Passport to enable access.
HP StorageWorks Fibre Channel Over Ethernet Application Note 19

Operating system support
For current operating system support, see the SPOCK website at http://www.hp.com/storage/spock.
You must sign up for an HP Passport to enable access.
Boot from SAN support
BFS is not supported for FC arrays attached to 2408 FCoE Converged Network Switches or C-series
Nexus 5000 Converged Network Switches at this time. BFS is supported for FC arrays attached to
standard FC switches in a fabric that includes CN switches.
DC SAN Director Switch 10/24 FCoE Blade supports BFS from the storage attached to any of the
other FC blades in the same director or to any Fibre Channel switch that is part of the same fabric as
the DC SAN Director that contains the 10/24 FCoE Blade.
This section provides information about installing BFS software.
Windows 2003 (x86, x64) and 2008 (x86, x64)
•BFS is supported on all supported HP servers.
•The information in HP StorageWorks Fibre Channel Host Bus Adapters Software Guide for Windows
(available on http://h18006.www1.hp.com/storage/networking/bootsan.html) is accurate for
Windows BFS on a CNA based server for Windows 2003 with the following exceptions:
•You must install an HP CNA driver kit (available for download at the HP website:
nl
http://www.hp.com/support/downloads
•For Windows 2003, press F6 to prompt for the driver disk
•For Windows 2008, click Load Driver when prompted for the location in which to install
Windows (see Figure 13).
Figure 13 Install Windows 2008 (x86, x64)
.
20