Brocade Communications Systems FCX Series User manual

DATA CENTER
Configuring an iSCSI Storage Area
Network Using Brocade FCX Switches
A step-by-step guide describing how to install and configure Brocade FCX
Switches in an iSCSI Storage Area Network (SAN).

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 2
CONTENTS
Introduction........................................................................................................................................................................................................................................ 3
Getting Ready ................................................................................................................................................................................................................................... 4
Installation.......................................................................................................................................................................................................................................... 4
Cabling Infrastructure.................................................................................................................................................... 4
Installation Location ...................................................................................................................................................... 4
Rack Installation ............................................................................................................................................................ 5
Power Connection.......................................................................................................................................................... 7
Cabling............................................................................................................................................................................ 8
Connecting switches in a stack ............................................................................................................................. 8
Cabling requirements for redundancy.................................................................................................................10
Basic Switch Setup and Configuration...................................................................................................................................................................................11
Powering On the System..............................................................................................................................................11
Connecting to a PC or Terminal...................................................................................................................................11
Using the CLI ................................................................................................................................................................12
Configuring IP Addresses.............................................................................................................................................12
Assigning Passwords ...................................................................................................................................................13
Attaching to the LAN ....................................................................................................................................................14
Configuring the Stack ..................................................................................................................................................15
Configuring Dynamic Link Aggregation between switches.................................................................................16
Configuring the Switch for SAN.................................................................................................................................................................................................18
Network Loop Prevention ............................................................................................................................................18
Symmetric Flow Control...............................................................................................................................................18
Jumbo Frames .............................................................................................................................................................20
Rate Limiting ................................................................................................................................................................22
Switch Configuration Dump.......................................................................................................................................................................................................22

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 3
INTRODUCTION
This document describes the steps for configuring the Brocade FCX switch in an iSCSI SAN environment. In
this document, you will find recommendations and instructions for specific commands to optimize the
switch for an iSCSI SAN network.
Internet Small Computer Systems Interface (iSCSI) is an IP-based protocol that links data storage facilities
by ensuring reliable data transfers over long distances. It allows IT staff to utilize a single underlying
infrastructure (Ethernet) to meet IT and storage networking needs. The behavior of iSCSI traffic differs from
regular IP traffic. IP traffic is generally composed of packets ranging in different sizes, requiring various
amounts of bandwidth from the network. On the other hand, iSCSI traffic encapsulates SCSI traffic (SCSI is
commonly used in data centers to connect disks and tape drives) and sends large blocks of data in bursts.
Carrying block SCSI traffic places greater demands on the network compared to regular IP traffic. In order to
run high-performance SCSI traffic over an IP infrastructure, dedicated iSCSI networks are recommended to
ensure traffic flows freely and quickly between initiators (clients) and targets (iSCSI arrays) without
competition from general IP traffic.
In order to provide reliable operation in a mission-critical iSCSI SAN environment, an ideal switch must offer
the following switch functionality:
•Non-Blocking backplane design. A switch should be able to provide the same amount of backplane
bandwidth required to support full duplex communication on ALL ports simultaneously. The Brocade
FCX, with its non-blocking line-rate architecture, is an ideal switch for such environments.
•Support for Inter-Switch Linking (ISL) or Dedicated Stacking Architecture. ISL support is required to
link all switches in a SAN infrastructure together. For non-stacking switches, the switch should support
designating one or more (through Link Aggregation Groups) ports for inter-switch links. Brocade FCX
stand-alone switches provide up to 40Gbps full-duplex bandwidth for ISL. For stacking switches, the
use of stacking ports for ISL is assumed, and each stacking switch should provide at least 20Mbps full-
duplex bandwidth. The Brocade FCX offers up to 64Gbps of stacking bandwidth.
•Support for creating Link Aggregation Groups (LAG). For non-stacking switches, the ability to bind
multiple physical ports into a single logical link for use as an ISL is required. These switches should
support creating LAGs of at least 8x 1Gbps ports or 1x 10Gbps port. In a non-stacking configuration, a
Brocade FCX offers 8x 1Gbps ports or 4x 10Gbps for LAG.
•Support for active or passive Flow Control (802.3x) on ALL ports. Switches must be able to actively
manage “pause” frames received from hosts, or they must passively pass all “pause” frames through
to the target arrays. Brocade FCX switches actively respond to “pause” frames received from hosts and
also generate “pause” frames offering a backpressure mechanism when the receiving port (such as
initiator) is not able to process data as fast as it is being sent to it.
•Support for Rapid Spanning Tree Protocol (R-STP). For SAN infrastructures consisting of more than
two non-stacking switches, R-STP must be enabled on all ports used for ISLs. All non-ISL ports should
be marked as “edge” ports or they should be set to “fast port-span.”
•Support for Jumbo Frames. Many storage implementations are able to take advantage of Jumbo
Frames. Whether or not Jumbo frames are able to provide performance increases depends on the
application and data characteristics.
•Ability to disable Unicast Storm Control. In General, iSCSI can send packets in a very “bursty” profile
that many switches misdiagnose as a viral induced packet storm. Since the SAN should be isolated
from general Ethernet traffic, the viral possibilities are non-existent. Switches need to always pass
Ethernet packets regardless of bandwidth utilization. The default behavior of a Brocade FCX is such
that iSCSI packets sent in bursts will not be diagnosed as a packet storm.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 4
•Adequate Buffer Space per switch port. Switch buffer mechanism also has an impact on the
performance of the switch. The Brocade FCX offers capabilities to optimize the buffers that will provide
the best possible performance in iSCSI SAN.
GETTING READY
Before you install and configure the Brocade FCX switch and create an iSCSI SAN, ensure you have the
following ready:
1. Download Brocade FCX switch documentation from https://kp.foundrynet.com.
a. If you are a first time visitor to the site, click the First Time Registration link to register.
b. Once logged-in, click Product Manuals, then FastIron.
2. A box cutter to unpack the device.
3. A #2 Phillips screw driver for rack installation.
4. Four #2 Phillips screws (per switch) to mount the device in the rack.
5. Acquire the IP address (or multiple addresses if configuring more than one switch), subnet mask, and
default gateway IP address from the networking group. The IP address will be assigned to the management
port on Brocade FCX switches.
6. Ethernet cable(s) with RJ-45 connecter that will connect to the management port on the Brocade FCX
switch. Only one cable is required per switch or one per stack in a Brocade FCX stack. The Ethernet cable
must be connected to a corporate or data center network that allows the administrator to manage the
switch remotely via telnet, web, or SNMP based management application.
INSTALLATION
Prior to installation, verify the contents of the shipping container. If any items are missing, contact the place
of purchase. Brocade FCX switches ship with the following:
•Brocade FCX switch
•115V AC power cable (for AC sourced devices)
•Rack mount brackets
•Warranty card
•A straight-through EIA or TIA DB-9 serial cable (F/F). The serial cable can be ordered separately from
Brocade.
NOTE: Hardware mounting screws are a customer-provided item and are not included in the rack mount kit.
Cabling Infrastructure
Ensure proper cabling is installed at the site. Refer to Brocade’s Best Practices Guide: Cabling the Data
Center (PN: GA-BP-036-01) for a summary of supported cabling types and their specifications. Brocade
recommends the use of Cat 6/Cat 6a cables to connect the Brocade FCX switch with the servers and
storage devices in an iSCSI SAN.
Installation Location
Before installing the switch, plan its location and orientation relative to other devices and equipment.
Switches can be mounted in a standard 19-inch equipment rack or on a flat surface. Be sure to follow the
guidelines below when choosing a location.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 5
The site should meet the following requirements:
•Maintain temperatures within 0 to 40οC (32 to 104οF) and humidity levels within 5% to 95%, non-
condensing.
•Allow a minimum of 3 in. of space between the sides and the back of the device and walls or other
obstructions for proper air flow.
•Allow at least 3 in. of space at the front and back of the device for the twisted-pair, fiber-optic, and
power cabling.
•Allow accessibility for installing, cabling, and maintaining the devices.
•Allow the status LEDs to be clearly visible.
•Allow for twisted-pair Ethernet cables to always be routed away from power lines, fluorescent lighting
fixtures, and other sources of electrical interference, such as radios and transmitters.
•Allow for the unit to be connected to a separate grounded power outlet that provides 110 to 240 VAC,
50 to 60 Hz, is within 2 m (6.6 feet) of each device, and is powered from an independent circuit
breaker. As with any equipment, a filter or surge suppressor is recommended.
Rack Installation
As noted in the previous section, Brocade FCX switches can be installed on a desktop or in an equipment
rack. Due to the density of devices and associated power, cooling, and safety requirements, Brocade
recommends installing the switches in a rack. Follow the step-by-step instructions in this section for proper
rack installation.
Before mounting a switch in a rack, pay particular attention to the following factors:
•Temperature. Since the temperature within a rack assembly may be higher than the ambient room
temperature, ensure that the rack-environment temperature is within the specified operating
temperature range. For details, refer to the “Operating Environment” section in the Brocade FCX
Installation Guide.
•Mechanical loading. Do not place any equipment on top of a rack-mounted unit.
•Circuit overloading. Ensure that the supply circuit to the rack assembly is not overloaded.
•Grounding. Rack-mounted equipment must be properly grounded. Particular attention should be given
to supply connections rather than direct connections to the mains.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 6
To mount a Brocade FCX in a rack:
1. Ensure you have a #2 Phillips screwdriver handy for the installation.
2. Remove the rack mount kit from the shipping carton. The kit contains two L-shaped mounting brackets and
mounting screws.
3. Attach the mounting brackets to the each side of the device using the provided screws.
4. Mount the device to the rack. Rack mount screws are not included in the rack mount kit.
5. If installing multiple switches, mount them in the rack, one below the other, and in any order.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 7
Power Connection
A secondary power supply provides backup power in case of a failure and for load-balancing when both
power supplies are operational. Load-balancing gives the power supplies a longer life span. The Brocade
FCX PSUs are hot-swappable. In addition, power for each PSU should be routed from two independent
power sources to increase availability. The same power considerations should be made for the servers and
storage arrays.
BEST PRACTICE:Brocade highly recommends that redundant power supplies be installed in the switch. An
integral component of an effective SAN is reliability. A redundant power supply connected to a separate
source of power ensures no downtime in the event of a circuit failure.
The Brocade FCX has two power receptacles at the rear of the unit. Each device ships with one power
supply installed, and a redundant power supply can be purchased and installed separately. Each power
supply has one standard power receptacle for the AC power cable, and includes AC and DC status LEDs for
easy monitoring and troubleshooting.
When only one PSU is installed, both "AC OK" and "DC OK" LEDs on the installed PSU must be green for the
Brocade FCX device to function normally. When two PSUs are installed, both "AC OK" and "DC OK" LEDs for
one of the installed PSUs must be green for the FCX device to function normally.
To install a power supply unit in the switch:
1. Remove the blank metal plate (or a previously installed PSU) from the appropriate slot by removing the two
screws with a flat-head screwdriver.
2. Before opening the package that contains the PSU, touch the bag to the switch casing to discharge any
potential static electricity. Brocade recommends using an ESD wrist strap during installation.
3. Remove the PSU from the anti-static shielded bag.
4. Holding the PSU level, guide it into the carrier rails on each side and gently push it all the way into the slot,
ensuring that it firmly engages with the connector.
5. When you are sure the PSU has properly engaged with the connector, tighten the retainer screws to secure
the PSU in the slot. When the device is powered on, the AC and DC LEDs on the PSU back panel should turn
green to confirm that the PSU is correctly installed and is supplying power.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 8
Cabling
A separate, isolated network should be configured to handle iSCSI traffic for performance and reliability of
the iSCSI SAN. Proper cabling is also necessary to provide redundancy in the iSCSI SAN environment. This
section reviews required cabling practices.
NOTE: If you only installed one switch, proceed to “Powering On the System” on page 11.
Connecting switches in a stack
NOTE: If you are creating an iSCSI SAN based on non-stackable switches, skip this section.
The Brocade FCX switch can be stacked to increase the available port count. To stack multiple Brocade FCX
648-S switches, use the dedicated 16Gbps stacking ports found on the rear of each unit, see Figure 1. To
stack multiple Brocade FCX 648-E switches, use the 10 Gbps SFP+ front-facing ports with LC-LC MM Fiber
cables, see Figure 2. Note that Brocade FCX 648-E switches must have a 4-port 10 Gbps SFP+ module
(optional) installed to operate in a stack. Up to eight Brocade FCX switches may be stacked, although a
typical iSCSI solution will not require this many ports.
In ring stack topology, an extra cable is connected between the top and bottom switches forming a “ring” or
“closed-loop.” The closed-loop cable provides a redundant path for the stack link, so if one link fails, stack
communications can be maintained. Figure 1 illustrates a ring-topology stacking configuration. In a
redundant iSCSI deployment of stacked Brocade FCX switches, the closed-loop topology should be
deployed.
BEST PRACTICE: Brocade highly recommends using the ring topology stack. This topology will ensure that all
switches have a redundant path in the stack.
Connecting Brocade FCX-S Switches
A stack can contain up to eight Brocade FCX-S switches. To connect Brocade FCX-S switches in a stack:
1. Plug one end of a stack cable into one of the top unit’s CX4 stacking ports.
2. Plug the other end of the stack cable into the next unit’s CX4 stacking ports.
3. Repeat steps 1 and 2 for each unit in the stack. Form a simple chain starting with a stacking port on the top
unit and ending at a stacking port on the bottom unit (stacking up to eight units).
4. (Optional) To form a ring stack topology, plug one end of a stack cable into the remaining stacking port on
the bottom unit and the other end into the remaining stacking port on the top unit.
One device in the stack will operate as the Active Controller, one will operate as standby Controller, with
the rest of the units operating as stack members. For information about how to configure your stack,
see the Brocade FCX Configuration Guide.
BEST PRACTICE: If you are approaching the eight-switch stack limitation, contact Brocade for
recommendations on how to effectively scale your iSCSI SAN for the future.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 9
Figure 1. Connecting Brocade FCX-S switches in a ring topology stack
Connecting Brocade FCX-E Switches
NOTE: Brocade FCX 648-E switches must have a 4-port 10 Gbps SFP+ module (optional) installed to
operate in a stack.
A stack can contain up to eight Brocade FCX-E switches. To connect Brocade FCX-E switches in a stack:
1. Plug one end of an LC-LC MM Fiber cable into one of the top unit’s SFP+ stacking ports.
2. Plug the other end of the cable into one of the stacking ports of the next unit.
3. Repeat steps 1 and 2 for each unit in the stack. Form a simple chain starting with a stacking port on the top
unit and ending at a stacking port on the bottom unit (stacking up to eight units).
4. (Optional) To form a ring stack topology, plug one end of an LC-LC MM Fiber cable into the remaining
stacking port on the bottom unit and the other end into the remaining stacking port on the top unit.
Figure 2. Connecting Brocade FCX-E switches in a ring topology stack

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 10
Cabling requirements for redundancy
Most iSCSI storage arrays have two storage controllers, with two to four Ethernet ports per controller. At
least two Ethernet ports should be cabled from each of the controllers for redundancy. Each controller is
then connected to multiple switches in the iSCSI SAN.
To build a highly available iSCSI SAN, ensure the following requirements are met:
•Contains at least two Brocade FCX switches, each with two PSUs, and each PSU uses a separate power
source.
•For Brocade FCX switches configured in a stack, create two redundant stacks. Each stack is managed
as a logical switch with one active management node. Each switch in the stack should have two PSUs,
and each PSU should be supplied by a separate power source.
•Ensure there are two separate power sources for each iSCSI storage array.
•For each iSCSI storage array, ensure there are at least two Ethernet ports on each controller with
connections to multiple switches in the stack.
•For greater redundancy, each initiator may be configured with multiple adapters or dual-port adapters
to connect to the iSCSI SAN.
Figure 3.Redundant cabling example using Brocade FCX-S stacks
In Figure 3, switches in each Brocade FCX stack are connected via link aggregation to both switches in the
opposing stack. Initiators have redundant connections to each stack, and both the active and passive iSCSI
controllers on each array are connected to each switch for full redundancy.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 11
BASIC SWITCH SETUP AND CONFIGURATION
Powering On the System
Once physical installation is complete, perform the following to power on the system:
1. Remove the power cable from the shipping carton.
2. Connect the AC power cable to the AC connector on the rear panel.
3. Insert the power cable plug into a 115V or 120V outlet.
4. Repeat steps 1-3 for redundant a power supply (if applicable), but connect it to a separate power source.
5. Repeat steps 1-4 for all switches in the SAN.
NOTES:
•To turn the system off, simply unplug the power cable or cables.
•The power outlet should be installed near the equipment, and it should be easily accessible.
•If the outlet is not rated 115/120V, stop and obtain the appropriate cable for the outlet.
Connecting to a PC or Terminal
To manage the switch, connect the console serial port on the switch to a management station, such as a PC
running a terminal emulation application (for example, HyperTerminal or Putty).
Use the serial connection to perform basic configuration tasks, including assigning an IP address and
subnet mask to the system. This information is required to manage the system using the Web management
interface, IronView Network Manager, or the CLI through Telnet.
To assign an IP address, you must have access to the Command Line Interface (CLI). The CLI is a text-based
interface that can be accessed through a direct serial connection to the device and through Telnet
connections. The CLI is described in detail in the Brocade FCX Configuration Guide.
To attach a management station to the console serial port:
1. Connect a PC or terminal to the serial port of the switch using a straight-through cable. The serial port has a
male DB-9 connector.
Figure 4. Console serial port (DB-9 DTE) pin-out

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 12
2. Launch the terminal emulation program and set the following session parameters:
Baud: 9600 bps
Data bits: 8
Parity: None
Stop bits: 1
Flow control: None
The EIA or TIA 232 serial communication port serves as a connection point for management by a PC or
SNMP workstation. Brocade devices come with a standard male DB-9 connector, shown in Figure 5.
Using the CLI
CLI commands are organized into the following levels:
•User EXEC. Lets you display information and perform basic tasks such as pings and traceroutes.
•Privileged EXEC. Lets you use the User EXEC commands plus configuration commands that do not
require saving the changes to the system-config file.
•CONFIG . Lets you make configuration changes to the device. To save the changes across reboots,
save them to the system-config file. The CONFIG level contains sub-levels for individual ports, for
VLANs, for routing protocols, and other configuration areas.
NOTE:The CLI supports command completion, so you do not need to enter the entire name of a command
or option. As long as you enter enough characters of the command or option name to avoid ambiguity with
other commands or options, the CLI understands what you are typing.
Configuring IP Addresses
At least one IP address must be configured using the serial connection to the CLI before you can manage
each switch using the other management interfaces. Brocade devices support both classical IP network
masks (Class A, B, and C subnet masks, and so on) and Classless Interdomain Routing (CIDR) network
prefix masks.
•To enter a classical network mask, enter the mask in IP address format. For example, enter
“209.157.22.99 255.255.255.0” for an IP address with a Class-C subnet mask.
•To enter a prefix number for a network mask, enter a forward slash ( / ) and the number of bits in the
mask immediately after the IP address. For example, enter “209.157.22.99/24” for an IP address that
has a network mask with 24 significant (“mask”) bits.
By default, the CLI displays network masks in classical IP address format (example: 255.255.255.0). You
can change the display to the prefix format. For details, see the Brocade FCX Configuration Guide.
NOTE:Ensure that you configure IP addresses for each initiator and target.
To configure an IP address on a device running Layer 2 software:
1. At the opening CLI prompt, enter enable.
FCX648S Switch> enable
Syntax: enable
2. Enter the following command at the Privileged EXEC level prompt (for example, FCX648S Switch#), then
press Enter. This command erases the factory test configuration if still present:

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 13
FCX648S Switch# erase startup-config
FCX648S Switch# reload
Enter ‘Yes’ to reload the switch
Syntax: erase startup-config
CAUTION: Use the erase startup-config command for new systems only. If you enter this command on a
system you have already configured, the command erases the configuration. If you accidentally do erase the
configuration on a configured system, enter the write memory command to save the running configuration
to the startup-config file.
3. After the switch reloads, access the configuration level of the CLI by entering the following command:
FCX648S Switch> enable
FCX648S Switch# config terminal
4. Configure the IP address and mask for each switch. Replace the IP address and mask in the example below
with the value provided by the networking group for each switch. Use the write memory or write mem
command to save changes.
FCX648S Switch (config)# ip address 192.168.1.252 255.255.255.0
FCX648S Switch (config)#write memory
Syntax: ip address <ip-address> <sub-net-mask>
5. Set a default gateway address for the switch. Replace the IP address and mask in the example below with
the value provided by your networking group. You do not need to assign a default gateway address for
single subnet networks.
FCX648S Switch (config)#ip default-gateway 192.168.1.1
FCX648S Switch (config)#write mem
Syntax: ip default-gateway <ip address of default gateway>
IP Command Summary:
Syntax: enable [<password>]
Syntax: configure terminal
Syntax: [no] ip address <ip-addr> <ip-mask>
(OR)
Syntax: [no] ip address <ip-addr>/<mask-bits>
Syntax: ip default-gateway <ip-addr>
Assigning Passwords
By default, the CLI is not protected by passwords. To secure CLI access, Brocade strongly recommends
assigning passwords.
1. At the opening CLI prompt, enter the following command to access the Privileged level of the EXEC mode:
FCX648S Switch > enable
2. Access the CONFIG level of the CLI by entering the following command:
FCX648S Switch # config terminal
FCX648S Switch (config)#

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 14
3. Enter the following command to set the super user password:
FCX648S Switch (config)# enable super-user-password <text>
NOTE:All passwords can be up to 32 characters long. You must set the super user password before you
can set other types of passwords. If you forget your super user password, refer to “Recovering from a lost
password” in the Brocade FCX Install Guide or Config Guide.
4. Enter the following commands to set the port configuration and read-only passwords:
FCX648S Switch (config)# enable port-config-password <text>
FCX648S Switch (config)# enable read-only-password <text>
Syntax: enable super-user-password | read-only-password | port-config-password <text>
5. Remember to save the configuration by writing changes to memory.
FCX648S Switch(config)#write mem
Write startup-config done.
Attaching to the LAN
For management purposes, the switch may be connected to the LAN using the out-of-band management
port through cables with RJ-45 connectors. No interface ports should be connected to the LAN in a
dedicated iSCSI SAN. When connecting the management port to the LAN, ensure that the port is secure to
avoid allowing access to the SAN from the LAN.
The management port is an out-of-band port that customers can use to manage their switches without
interfering with the in-band ports. The management port is widely used to download images and
configurations, for telnet sessions, for Web management, and for SNMP based management.
For stacking devices (for example, a Brocade FCX stack), each stack unit has one out-of band management
port. Only the management port on the Active Controller will actively send and receive packets. If a new
Active Controller is elected, the new Active Controller management port will become the active management
port. In this situation, the MAC address of the old Active Controller and the MAC address of the new
controller will be different.
To show the current configuration, enter the following:
FCX648S Switch (config)# show running-config
FCX648S Switch (config)# show interfaces management 1
Syntax: show running-config
Syntax: show interfaces management <num>
To display the management interface information in brief form, enter the show interfaces brief
management command:
FCX648S Switch (config)# show interfaces brief management 1
Syntax: show interfaces brief management <num>
To display management port statistics, enter the show statistics management command:
FCX648S Switch (config)# show statistics management 1
Syntax: show statistics management <num>

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 15
Configuring the Stack
NOTE: Skip this section if you are configuring an iSCSI SAN based on non-stacked switches.
If you are configuring multiple Brocade FCX switches in a stack, ensure the devices are connected per the
“Cabling” section on page 8.
When utilizing a full stack of eight units (a switch in a stack is referred to as a unit), you may need to
increase the trap hold time from the default, which is 60 seconds, to three minutes (180 seconds). This will
prevent the loss of initial boot traps.
To increase the trap hold time, use the following command:
FCX648S Switch# snmp-server enable traps hold 180
Syntax: snmp-server enable traps hold <seconds>
To configure the stack, following these steps:
1. Connect the devices using the stacking ports and stack cabling (FCX-S) or fiber cable (FCX-E) per the
“Cabling” section on page 8.
2. Power on the units.
3. Connect your console to the intended Active unit. The unit through which you run secure-setup becomes the
Active unit by default.
4. Issue the stack enable command on the intended Active unit.
Enter these commands to configure the stack on a Brocade FCX 648-S:
FCX648S Switch# config t
FCX648S Switch (config)# stack enable
FCX648S Switch (config)# write mem
Enter these commands to configure the stack on an Brocade FCX 648-E:
FCX648-E Switch# stack secure-setup
FCX648-E Switch# config t
FCX648-E Switch (config)# stack enable
FCX648-E Switch (config)# write mem
5. Enter the stack secure-setup command. This command triggers a Brocade proprietary discovery protocol
that begins the discovery process in both upstream and downstream directions. The discovery process
produces a list of upstream and downstream devices that are available to join the stack. Secure-setup can
detect up to seven units in each direction (14 total), but since the maximum number of units in a stack is
eight, you must select a maximum of seven units from both directions.
6. The Active unit automatically checks all prospective stack members to see if they are password protected. If
a unit is password protected, you will be asked to enter the password before you can add the unit. If you do
not know the password, take one of the following actions:
•Discontinue secure-setup by entering ^C
•Obtain the device password from the administrator
•Continue secure-setup for your stack. The password-protected device and all devices connected
behind it will not be included in the setup process.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 16
7. When the Active unit has finished the authentication process, you will see output that shows the suggested
assigned stack IDs for each member. You can accept these recommendations, or you can manually
configure stack IDs. Enter the show stack command to verify that all units are in the ready state.
FCX648S Switch# show stack
Syntax: show stack
Here is a sample output after issuing the show stack command:
FCX648S Switch# show stack.
alone: standalone, D: dynamic config, S: static config
ID Type Role Mac Address Pri State Comment
1 S FCX648S active 0024.3814.c140 0 local Ready
2 D FCX648S standby 0024.3814.f140 0 remote Ready
active standby
+---+ +---+
-2/1| 1 |2/2--2/1| 2 |2/2-
| +---+ +---+ |
| |
|------------------------|
Standby unit 2: protocols ready, can failover or manually switch over
Current stack management MAC is 0024.3814.c140
Note: no "stack mac" config. My MAC will change after failover.
Some configuration commands later in this document will require the user to enter unit ID for the unit in the stack
being configured. There are two ways to find the unit ID. Every Brocade FCX switch has stack indicators in front of the
switch as shown below. These indicators will help the users identify the unit ID assigned to each switch in the stack.
Another way the unit ID can be verified is by entering the show stack command as shown in step 7 above.
Configuring Dynamic Link Aggregation between switches
NOTE:Skip this section if you are configuring an iSCSI SAN without using link aggregation.
Brocade software supports the IEEE 802.3ad standard for link aggregation. This standard describes the
Link Aggregation Control Protocol (LACP), a mechanism for allowing ports on both sides of a redundant link
to form a trunk link (aggregate link), without the need for manual configuration of the ports into trunk
groups.
When you enable link aggregation on a group of Brocade ports, the Brocade ports can negotiate with the
ports at the remote ends of the links to establish trunk groups. By default, link aggregation is disabled on all
ports. To enable link aggregation on a set of ports, enter commands such as the following at the Interface
configuration level of the CLI.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 17
NOTE:If you plan to use the the 10 Gbps SFP+ front-facing ports on a Brocade FCX 648-E for link aggregation, you
must first disable stacking on those ports. Use the show stack command to verify if stacking is enabled on those
ports.
NOTE:Brocade recommends that you remove the cables from the ports you plan to enable for dynamic link
aggregation prior to configuring.
To configure link aggregation on a Brocade FCX 648-S using the front-facing 10G uplink ports, enter the
following commands on each switch involved:
FCX648S Switch(config)# interface ethernet 1/3/1
FCX648S Switch(config-if-e10000-1/3/1)# link-aggregate active
FCX648S Switch(config)# interface ethernet 1/3/2
FCX648S Switch(config-if-e10000-1/3/2)#)# link-aggregate active
Syntax: interface ethernet <stack-unit/slotnum/portnum>
link-aggregate active
To configure link aggregation on a Brocade FCX 648-E using the front-facing 10G uplink ports, enter the
following commands for each switch involved:
FCX648-E Switch(config)# interface ethernet 1/2/3
FCX648-E Switch(config-if-e10000-1/2/3)# link-aggregate active
FCX648-E Switch(config)# interface ethernet 1/2/4
FCX648-E Switch(config-if-e10000-1/2/4)#)# link-aggregate active
Syntax: interface ethernet <stack-unit/slotnum/portnum>
link-aggregate active
Figure 5. Redundant cabling example using two Brocade FCX switches, no stacking.
The Brocade FCXs in Figure 5 are connected via link aggregations on the 10G uplink ports. The active and
passive controllers on each array have a link to both switches. Initiators can also be provided redundant
connections to the iSCSI SAN using multiple NICs or a dual-port NIC.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 18
CONFIGURING THE SWITCH FOR DEDICATED ISCSI SAN
Network Loop Prevention
The Spanning Tree Protocol (STP) eliminates Layer 2 loops in networks, by selectively blocking some ports
and allowing other ports to forward traffic, based on configurable global (bridge) and local (port)
parameters. STP related features, such as RSTP and PVST, enhance the operation of standard STP, allowing
for fine-tuning of standard STP, and therefore avoiding some of its limitations. You can enable or disable
STP on a global basis (for the entire device), a port-based VLAN basis (for the individual Layer 2 broadcast
domain), or an individual port basis.
BEST PRACTICE:Brocade highly recommends that customers create iSCSI SANs based on dedicated
switches. Brocade FCX switches offer capabilities to logically partition LAN traffic from SAN traffic. However,
the mission critical nature of SANs demand that LAN traffic, which is beyond the control of sever and
storage administrators, be separated from the switch completely.
If the iSCSI SAN will be connected to another Brocade FCX for redundancy or to another device for uplink
(as per Figure 5), Brocade recommends the customers to use Rapid Spanning Tree. Rapid Spanning Tree
(802.1W) provides rapid traffic reconvergence for point-to-point links within a few milliseconds (0 – 500
milliseconds), following the failure of a bridge or bridge port. This reconvergence occurs more rapidly than
the reconvergence provided by the 802.1D Spanning Tree Protocol (STP)) or by RSTP Draft 3.
FCX648S Switch (config)#spanning-tree 802-1w
FCX648S Switch (config)#write mem
Syntax: spanning-tree 802-1w
BEST PRACTICE:For iSCSI deployments, Brocade recommends that spanning tree be disabled on all access
ports. Access ports are the switch ports that are directly connected to the server or storage ports. In the
event of a change in the network, the ports with spanning tree enabled take a few additional seconds to
reach a state of full functionality. However, since access ports are never going to be connected to Ethernet
switches or hubs, there is no need to enable spanning tree on them.
To disable spanning tree, issue the following commands per VLAN (do not disable globally):
FCX648S Switch (config)#no spanning-tree
FCX648S Switch (config)#write mem
Syntax: [no] spanning-tree
If the environment where the iSCSI SAN will be deployed is not a controlled environment (for example, there
is a possibility of someone accidentally plugging in a switch or a hub to the FCX), Brocade recommends that
customers enable fast port-span. This feature enables the ports to become functional much more quickly
compared to regular spanning tree.
FCX648S Switch (config)# fast port-span
FCX648S Switch (config)#write mem
Syntax: [no] fast port-span
To disable fast-port span on an inter-switch link, issue the following commands specific to the port:
FCX648S Switch (config)# fast port-span exclude ethernet 1/1/1 ethernet 1/1/2
FCX648S Switch (config)# write mem
Syntax: [no] fast port-span [exclude ethernet <port> [ethernet <port>] | to [<port>]]
Symmetric Flow Control and Buffer Configuration
One of the advantages of Brocade FCX switches offered in iSCSI SANs is the deployment of lossless
Ethernet. By deploying lossless Ethernet using a Brocade FCX, customers may significantly improve the
performance of their SANs. In the event of congestion, this feature relies on flow control capabilities as

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 19
defined by IEEE 802.3x to signal the sender (iSCSI initiator or target) to stop sending data for a brief period
of time. During this time, the Brocade FCX switch quickly takes care of the congestion, allowing the iSCSI
traffic to resume to full performance quickly. Since iSCSI datagram is encapsulated on TCP, this mechanism
helps avoid the slow start problem of TCP, where loss of packets causes the flow to slow down.
To enable symmetric flow control, enter the following commands:
1. Enable symmetric flow control globally or on a concerned unit (disabled by default). Do not enter dynamic
buffer allocation, buffer profiles, or DHCP QOS commands enabling symmetric flow control.
FCX648S Switch (config)#symmetric-flow-control enable
FCX648S Switch (config)#write mem
2. Enter the following commands to enable symmetric flow control in a stack:
FCX648S Switch (config)#symmetric-flow-control enable unit 1
FCX648S Switch (config)#symmetric-flow-control enable unit 2
FCX648S Switch (config)#write mem
Syntax: [no] symmetric-flow-control enable [unit <unit_ID>]
NOTE: Refer to the section on stack configuration in this document to identify the unit ID.
3. Enter the following commands to optimize buffer allocations on all ports of the switch or stack of switches.
FCX648S Switch (config)#buffer-sharing-full
FCX648S Switch (config)#write mem
Syntax: [no] buffer-sharing-full
To enable flow control on the initiators, perform the steps in the following Windows or Linux section (as
required):
Windows
Ensure you have a compatible network interface card and appropriate driver. You may need to enable flow
control on the NIC.
To verify flow control support is enabled on the NIC:
1. Open Network and Sharing Center from the Control Panel.
2. Locate the network connection and click View Status. The Local Area Connection X Status displays.
3. Click Properties.
4. Click Configure.
5. Click the Advanced tab.
6. From the Property list, select Flow Control.
7. From the Value menu, select Rx & Tx Enabled.
8. Click OK to apply and close all open boxes. You may need to reboot the server for the change to take effect.

DATA CENTER CONFIGURATION GUIDE
Configuring an iSCSI SAN Using Brocade FCX Switches 20
Figure 6. Enabling Flow Control in Windows
Linux
Ensure you have a compatible network interface card and appropriate driver. Reference your OS guide for
more information or for assistance with specific commands to enable Rx and Tx flow control.
Jumbo Frames
Ethernet traffic moves in units called frames. The maximum size of frames is called the Maximum
Transmission Unit (MTU). When a network device receives a frame larger than its MTU, the data is either
fragmented or dropped. Historically, Ethernet has a maximum frame size of 1,500 bytes, so most devices
use 1,500 as their default MTU.
Jumbo frames are Ethernet frames with more than 1,500 bytes MTU. Conventionally, jumbo frames can
carry up to 9,000 bytes MTU. Brocade FCX switches support Layer 2 jumbo frames on 100/100/1000, and
10GbE ports. Brocade devices also support jumbo frames per VLAN.
All servers/arrays connected to the iSCSI SAN should have jumbo frame support enabled due to the higher
payload demands of storage network traffic. This includes not only the switch but also the network adapters
for the initiators. Note that the switch needs to be reloaded to enable jumbo frame processing.
To enable jumbo frames on the Brocade FCX, enter the following commands:
FCX648S Switch (config)#jumbo
FCX648S Switch (config)#write mem
Syntax: jumbo