EMC AX100 Series User manual

1
If you want to connect an AX100-Series Fibre Channel storage system -
AX100SC or AX100 - through one or two switches to a Linux server, this
document is for you. You can use the procedures in this document to
install configurations with a new or existing storage system that you
want to connect to a new or existing server.
The main topics are
◆AX100 Support Website........................................................................2
◆Terminology ...........................................................................................3
◆Before You Start .....................................................................................4
◆Installing HBAs in the Server ..............................................................7
◆Installing or Updating the HBA Driver .............................................9
◆Installing PowerPath on the Server ..................................................11
◆Upgrading PowerPath on the Server................................................20
◆Installing the Navisphere Server Utility ..........................................26
◆Installing the UPS (AX100 Only).......................................................27
◆Unpacking the Storage System..........................................................28
◆Installing the Storage System.............................................................30
◆Installing Additional Disks................................................................34
◆Installing the Switches in the Cabinet ..............................................41
◆Connecting ac Power to an AX100SC Storage System...................44
◆Connecting ac Power to an AX100 Storage System........................48
◆Connecting the Storage-System and Switch Management Ports .53
◆Initializing the Storage System..........................................................56
AX100-Series
Installing a Fibre Channel Storage
System with a Switch Connection to a
Linux Server
Revision A06
July 25, 2005

2
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
AX100 Support Website
◆Connecting Switches to the Storage System and Server............... 63
◆Registering the Server with the Storage System ............................ 66
◆Configuring a New Storage System................................................. 69
◆Configuring an Existing Storage System......................................... 73
◆Preparing Virtual Disks to Receive Data ......................................... 77
◆Sending Linux Disk Information to the Storage System............... 80
◆Verifying Your Configuration ........................................................... 81
AX100 Support Website
The AX100 support website contains the storage-system release notes
and the most recent versions of documents and of the interactive
troubleshooting trees. For information on accessing the AX100
support website, refer to the support information that shipped with
your storage system.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
3
Terminology
Terminology
This document uses the terms new and existing to refer to storage
systems, servers, and host bus adapters (HBAs) as follows:
The document also uses SP to refer to a storage processor in the storage
system. An AX100SC storage system has only one SP, so it is called a
single-SP storage system. An AX100 storage system has two SPs, so it
is called a dual-SP storage system.
Existing HBA The host bus adapter (HBA) that is already installed in a server.
Existing server The server that is already configured for and connected to an
AX-Series storage system.
Existing storage system An AX-Series storage system that is already connected to a
server and configured for storage.
Host A computer that is or will be connected to an AX-Series storage
system. This computer is called either a
management host or a
server, depending on how it is or will be connected to the storage
system.
Management host A host from which you manage AX-Series storage systems. It
must be on the same LAN as the storage-system management
ports. A management host may also be server.
New HBA The HBA that was
not already installed in the server.
New server A server that is not already connected to or set up for an
AX-Series storage system.
New storage system A storage system that is
not connected to a server or configured
for storage. It is just as it was shipped to you.
Server A host that is already or will be connected through a switch to the
Fibre channel ports on a AX-Series storage systems. A server
can also be a management host.

4
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Before You Start
Before You Start
Before you start the installation procedure
❑Review the Supported Configurations in the "Technical
descriptions" section on the AX100 website to confirm that the
following are supported:
❑The server hardware that will access the storage system
❑The operating system version running on the server hardware.
❑Read the AX100 Release Notes, which are in the "Release Notes"
section on the AX100 support website.
❑Complete the configuration planning worksheets in Planning Your
AX100-Series Fibre Channel Storage-System Configuration. This
guide is in the "Plan" section on the AX100 support website and
on the AX100-Series Documentation CD.
For a new storage system installation, you need
❑An AX100SC or AX100 storage system and the power cord(s),
rails, and CDs that ship with it.
❑For an AX100 storage system (dual-SP system), an
uninterruptible power supply (UPS) kit, including the power
cords, UPS sense (serial) cable, and mounting hardware.
❑Standard ac power for each power supply in the storage system
from an independent ac source or a cabinet/rack power
distribution unit.
❑The storage-system serial number located on a label on the back
of the chassis (see below).
HW S/N XXX 000NNNNNNNN
EMC3030

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
5
Before You Start
❑The following management port network information, which the
person responsible for your network should provide:
❑Static IP address for each SP in the storage system.
❑Subnet mask for the LAN to which you will connect the
storage system.
❑Default gateway for the LAN to which you will connect the
storage system.
For any installation, you need
❑A management host on the network to which you will connect the
storage-system management ports and with a supported Internet
browser for running Navisphere® Express. This host can also be
the server. For supported hosts and browsers, refer to Supported
Configurations in the "Technical descriptions" section on the
AX100 support website.
❑A Linux® host that is or will be a server with Fibre Channel
connections to the storage system. This server must have all
required updates, such as hot fixes or patches, installed. For
supported hosts and required updates, refer to Supported
Configurations in the "Technical descriptions" section on the
AX100 support website.
❑One or more supported Emulex or QLogic Fibre Channel host bus
adapters (HBAs), which may already be installed in the server.
These adapters must have the latest supported BIOS and driver.
For information on supported HBAs, BIOS, and drivers, refer to
Supported Configurations in the "Technical descriptions" section on
the AX100 support website.
Never mix Fibre Channel HBAs from different vendors in the same
server.
❑One or two customer-installable switches, which may already be
installed for an existing storage system or server. (For information
on customer-installable switches, refer to Supported Configurations
in the "Technical descriptions" section on the AX100 support
website.)

6
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Before You Start
❑The following cables, which may already be connected for a
configuration with an existing storage system, server, or switches:
❑A CAT 5 or higher LAN cable for each SP and for each switch
if you want to use the switch management software.
❑An optical cable for each storage-processor (SP) Fibre Channel
port you will use on the storage system and for each HBA you
will use in the server. (Each storage processor has two ports.)
For cable specifications, refer to Technical Specifications in the
"Technical description" section on the AX100 support website.
❑A method for writing data to a virtual disk on the storage system
to test the path from a new HBA to the storage system. You can
download an I/O simulator (Iometer) from the following
website: http://www.iometer.org/.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
7
Installing HBAs in the Server
Installing HBAs in the Server
For the server to communicate with the storage system through the
Fibre Channel ports, the server must have one or more supported
Fibre Channel host bus adapters (HBAs). If the HBAs are already
installed, go to Installing or Updating the HBA Driver (page 9). Read
this section for all other configurations.
Before You Start
To complete this procedure, you need one or more supported QLogic
or Emulex Fibre Channel HBAs with the latest supported BIOS and
driver. For information on supported HBAs, BIOS, and drivers, refer
to Supported Configurations in the "Technical descriptions" section on
the AX100 support website. Never mix Fibre Channel HBAs from
different vendors in the same server.
Install HBAs
CAUTION
!
HBAs are very susceptible to damage caused by static discharge
and need to be handled accordingly. Before handling HBAs,
observe the following precautions:
• Store HBAs in anti-static bags.
• Use a ground (ESD) strap whenever you handle HBAs.
• Never plug or unplug HBAs with the power on. Severe
component damage can result.
1. If the server is running, shut it down as follows:
a. Shut down the server’s operating system.
b. Power down the server.
c. Unplug the server power cord from the power outlet.
2. Remove the outer covers from the server so that you can access
the PCI bus slots.
For information on how to remove the covers from the server,
refer to the installation documentation supplied with the server.

8
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Installing HBAs in the Server
3. Put on an ESD wristband, and clip its lead to bare metal on the
server’s chassis.
4. For the HBA you are adding, do the following:
a. If the server has a removable PCI card cage, remove it.
b. Locate an empty PCI bus slot.
c. Remove the filler panel from the slot and save it.
d. If your server has a low profile, replace the mounting bracket
on the HBA bracket with the low-profile mounting bracket
provided with your HBA.
e. Carefully align the HBA with the slot, and then firmly and
evenly press the HBA into the slot.
f. Secure the HBA to the slot with the screw or clip from the filler
panel.
g. If you have another HBA to install, repeat steps b through f.
h. If you removed a PCI card cage, replace it.
5. Replace the covers you removed from the server.
6. Remove the ESD wristband.
7. Plug the server power cord into the power outlet.
8. Power up the server.
You have completed the installation of the HBA.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
9
Installing or Updating the HBA Driver
Installing or Updating the HBA Driver
The server must run a supported HBA driver. For information on the
supported HBA drivers, refer to Supported Configurations in the
"Technical descriptions" section on the AX100 support website.
If the server is already running a supported HBA driver, go to
Installing PowerPath on the Server (page 11).
Read this section for all other configurations.
Before You Start To complete this procedure, you need
❑The latest version of a supported HBA driver. You can download
the latest supported version and instructions for installing it from
the HBA vendor’s website.
Emulex website
http://www.emulex.com/ts/docoem/framemc.htm
QLogic HBA website
http://www.qlogic.com/go/emc_approved
The HBA driver is also on the installation CD that ships with the HBA.
This version may not be the latest supported version.
❑Any updates, such as hot fixes or patches, to the server’s
operating system that are required for the HBA driver version
you will install.
For information on any required updates, refer to one of the
following:
•Supported Configurations in the "Technical descriptions" section
on the AX100 support website
• The HBA vendor’s website (see above)

10
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Installing or Updating the HBA Driver
Install or Update the HBA Driver
You can load the driver into the kernel statically or as a module.
1. Install any updates, such as hot fixes or patches, to the server’s
operating system that are required for the HBA driver version
you are installing.
2. After installing any updates, reboot the server.
3. Install the driver following the instructions on the HBA vendor’s
website.
4. Reboot the server when the installation program prompts you to
reboot.
Some driver updates may require you to reboot the server.
5. If the installation program did not prompt you to reboot, reboot
the server when the driver installation is complete.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
11
Installing PowerPath on the Server
Installing PowerPath on the Server
For all configurations you must install PowerPath®on each server that
you will connect to the storage system. PowerPath manages the data
paths between the server and storage system.
If PowerPath is already running on the server and you need or want
to upgrade it to a higher version, go to Install a PowerPath Patch
(page 19). If you do not need or want to upgrade it, do one of the
following:
◆For a new AX100SC (single-SP) storage system, go to Unpacking
the Storage System (page 28).
◆For a new AX100 (dual-SP) storage system, go to Installing the
UPS (AX100 Only) (page 27).
◆For an existing storage system (that is, one already connected to
another server), go to Connecting Switches to the Storage System and
Server (page 63).
Before You Start
To complete this procedure, you need to
❑Locate the PowerPath for Linux CD that shipped with the storage
system.
❑Locate your AX100 PowerPath registration number, which is on
the PowerPath License Key Card you received.
❑Save your kernel and back up your system.
❑Change the Linux kernel. Refer to Change the Kernel (page 12).
❑Change the Linux configuration files and save the kernel. Refer to
either
Modify Configuration Files on a Red Hat Server (page 13) and Save
Kernel Changes on a Red Hat Server (page 13)
or
Modify Configuration Files on a SuSE Server (page 14) and Save
Kernel Changes on a SuSE Server (page 15).

12
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Installing PowerPath on the Server
❑Verify that the /lib/modules/`uname -r`/kernel/drivers/addon
directory exists on the server. If the directory does not exist, create
it:
mkdir /lib/modules/`uname -r`/kernel/drivers/addon
chmod 755 /lib/modules/`uname -r`/kernel/drivers/addon
❑Verify that the driver module directory exists. Refer to Install
PowerPath (page 15).
❑For RHEL 3.0 and SuSE SLES 8 servers, make sure you have 256
sd and sg devices in the /dev directory.
This prerequisite is required even if the devfs file system is
mounted over /dev.
❑Ensure that the PowerPath driver’s major numbers (232-247) are
not already in use.
If the major numbers are already in use, the driver fails to install
at system boot time and issues this error:
emcp: Device or resource busy.
❑Stop using any storage-system device. For example,
storage-system devices cannot be mounted, configured as swap
devices or accessed by applications via sd, sg, or raw devices.
Change the Kernel
If you load the HBA driver as a module, install its binary file in the
appropriate location.
Red Hat Location
/lib/modules/‘uname -r‘/drivers/addon/driver_module
where
uname -r is the string returned to the console by the uname -r
command.
driver_module is the name of the HBA driver module.
SuSE Location
/lib/modules/‘uname -r‘/drivers/scsi
where uname -r is the string returned to the console by the uname -r
command.
If you have Red Hat Linux 3.0 or SuSE Linux SLES 8 SP 3 or higher, you no
longer need to build the HBA driver modules and copy them to the above
location. The pre-built kernel now includes these modules.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
13
Installing PowerPath on the Server
Modify Configuration Files on a Red Hat Server
1. Edit the /etc/modules.conf file to add an alias for each HBA.
In the sample file below, the second and third lines are aliases for
HBAs;
driver_module in these entries is the name of the HBA
driver module.
alias eth0 eepro100
alias scsi_hostadapter1
driver_module
alias scsi_hostadapter2 driver_module
2. Enable the following parameters in /etc/modules.conf:
max_scsi_luns
scsi_allow_ghost_devices
You must enable the last parameter so that you do not have to
recompile the kernel to properly connect a Linux host to the
storage system.
For example, in the following modules.conf file, the
max_scsi_luns and scsi_allow_ghost_devices
parameters are enabled:
alias parport_lowlevel parport_pc
alias scsi_hostadapter sym53c8xx
alias scsi_hostadapter1
driver_module
alias scsi_hostadapter2 driver_module
alias eth0 tlan
options scsi_mod max_scsi_luns=256 scsi_allow_ghost_devices=1
where driver_module in the third and fourth lines is the name of
the HBA driver module for accessing the storage system.
Save Kernel Changes on a Red Hat Server
1. Run ls -l /boot and look at the line with initrd -> to determine
which initrd file is used.
2. Run the uname -r to display the kernel version.
3. Enter:
/sbin/mkinitrd -f -v initrd kernel
where
initrd is the configuration identified by ls -l /boot
kernel is the kernel version displayed by uname -r

14
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Installing PowerPath on the Server
Modify Configuration Files on a SuSE Server
1. Verify that the HBA driver is always loaded after the internal SCSI
adapter driver as specified by the /etc/sysconfig/kernel file. For
example, in the following file,
driver_module (the HBA driver
module for accessing the storage system) is always loaded after
all internal SCSI drivers are loaded:
INITRD_MODULES="scsi_mod sd_mod cciss reiserfs
driver_module "
When loading HBA drivers into the ramdisk, make sure that the HBA
module name used in /etc/sysconfig/kernel is the same as that used in
the /etc/modules.conf file.
2. Enable the following parameters in /etc/modules.conf:
max_scsi_luns
scsi_allow_ghost_devices
You must enable the last parameter so that you do not have to
recompile the kernel to properly connect a Linux host to the
storage system.
For example, in the following modules.conf file, the
max_scsi_luns and scsi_allow_ghost_devices
parameters are enabled:
alias parport_lowlevel parport_pc
alias scsi_hostadapter sym53c8xx
alias scsi_hostadapter1
driver_module
alias scsi_hostadapter2 driver_module
alias eth0 tlan
options scsi_mod max_scsi_luns=256 scsi_allow_ghost_devices=1
where driver_module in the third and fourth lines is the name of
the HBA driver module for accessing the storage system.
When loading HBA drivers into the ramdisk, make sure that the HBA
module name in /etc/sysconfig/kernel is the same as that used in the
/etc/modules.conf file.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
15
Installing PowerPath on the Server
Save Kernel Changes on a SuSE Server
1. Run uname -r to display the kernel version.
2. Run ls -l /boot to determine which initrd and vmlinuz files are
used for the kernel version.
The names of both the initrd and vmlinuz files include the kernel
version.
3. Enter:
/sbin/mk_initrd -k vmlinuz -i initrd
where
vmlinuz and initrd correspond to the configuration identified by
ls -l /boot for the kernel version.
Install PowerPath
1. Log in as root and insert the PowerPath CD into the server’s CD
drive.
2. Create the directory /cdrom as the mount point for the CD:
mkdir /cdrom
3. Mount the PowerPath CD on /cdrom:
mount -o ro /dev/cdrom /cdrom
4. Change to the LINUX directory for your kernel and revision of
PowerPath that you are installing.
For example, for a 2.4 kernel and PowerPath 4.3.2:
5. cd /cdrom/LINUX/2.4/pp4.3.2Install PowerPath:
rpm -iv EMCpower.LINUX-version-build.i386.rpm
where version and build are the version number and build number
number of PowerPath on the CD.
For a Red Hat 3.0 AS/ES iA64 or EM64T system, use the following
command: rpm --prefix / EMCpower.LINUX-version-build.i386.rpm

16
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Installing PowerPath on the Server
You have completed the PowerPath installation. After running
the rpm command, certain error messages may occur. The
following acceptable warning appears when loading the
PowerPath drivers:
Warning: loading /lib/modules/‘uname
-r‘/kernel/drivers/addon/emcpower/emcp.o will taint the kernel:
no license
This message is expected and is acceptable behavior for loading a
binary module.
Initially, PowerPath attempts to load modules with version-checking
enabled. If the pre-selected module load fails, the software silently
(without issuing error messages) loads the module with
version-checking disabled.
6. If you have PowerPath 4.3.2 or lower, register your PowerPath
license on the server.
a. Enter
emcpreg -install
You see the following output:
=========== EMC PowerPath Registration ===========
Do you have a new registration key or keys to
enter? [n]
b. Enter
y
You see the following output:
Enter the registration key(s) for your product(s),
one per line, pressing Enter after each key.
After typing all keys, press Enter again.
Key (Enter if done):
c. Enter the 24-character alphanumeric sequence from the AX100
License Key Card that shipped with the PowerPath media kit,
and press Enter.
Enter the registration number exactly as written on the card that you
received with the software. To reduce common typographical errors,
the PowerPath Registration Utility accepts either uppercase or
lowercase letters, and certain numbers and letters are
interchangeable. Specifically, entering the alphabetic letters O, I, S,
and B is equivalent to entering the numbers, 0, 1, 5, and 8.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
17
Installing PowerPath on the Server
If you entered a valid registration key, you see the following
output:
1 Key(s) successfully added.
Key successfully installed:
If you entered an invalid registration key, the screen displays
an error message and prompts you to enter a valid key.
d. Press Enter.
You see the following output:
1 key(s) successfully registered.
7. Start PowerPath:
/etc/init.d/PowerPath start
Use the above command only for loading and unloading the
EMC® PowerPath module.
8. Unmount the CD:
cd /
umount /cdrom
9. Remove the PowerPath installation CD from the CD drive.

18
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Installing PowerPath on the Server
Verify All HBA Driver Extensions Are Loaded
If you loaded the HBA driver as a module (not as a static kernel
driver), use lsmod to verify that all extensions are loaded.
If you built the HBA driver as a module and modified
/etc/modules.conf to specify that the HBA driver be loaded before the
PowerPath drivers, the HBA driver name appears in the lsmod
output below the PowerPath drivers.
For example:
Module Size Used by Tainted: PF
emcphr 9624 0 (unused)
emcpmpap 105024 0 (unused)
emcpmpaa 72160 0 (unused)
emcpmpc 92064 0 (unused)
emcpmp 55840 0 (unused)
emcp 542520 0 [emcphr emcpmpap emcpmpaa emcpmpc emcpmp]
emcpsf 6820 0 [emcpmpap emcp]
.
.
.
driver_module 684380 1
sg mptscsih
driver_module sd_mod]
where driver_module is the name of the HBA driver module.
The /etc/modules.conf file should look similar to the following:
alias ppfilter emcpsf
pre-install emcpsf modprobe sd_mod
add options emcp hostid=00:C0:9F:35:4A:30
insmod_opt=-N
alias power_path emcp
add above emcp emcpmp
add above emcpmp emcpmpc
add above emcpmpc emcpmpaa
add above emcpmpaa emcpmpap
add above emcpmpap emcphr
add above emcphr emcpioc
post-install emcpioc rmmod emcpioc
add options emcp
managedclass="hpxp","ess","hitachi","hphsx"
add below emcp
driver_module
alias scsi_ppfilteradapter driver_module
post-install emcpsf modprobe
driver_module
pre-remove emcpsf rmmod emcpmpaa emcpmpc emcpmp emcp
where driver_module is the name of the HBA driver module.

AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
19
Installing PowerPath on the Server
Rebuild the Ramdisk on a SuSE Server
If the server is running SuSE Linux SLES8 and a kernel version newer
than 2.4.21-138, rebuild the ramdisk:
1. Edit the /etc/sysconfig/kernel file to ensure that the HBA driver is
always loaded after the internal SCSI adapter driver as follows:
INITRD_MODULES="scsi_mod sd_mod cciss reiserfs
driver_module "
where driver_module is the name of the HBA driver module
used to access the storage system.
For SuSE Linux, when loading HBA drivers into the ramdisk, make sure
that the HBA module name used in /etc/sysconfig/kernel is the same
as that used in the /etc/modules.conf file.
2. Create the new ramdisk:
mk_initrd -i /boot/ramdisk name -k /boot/kernel image name
Install a PowerPath Patch
Check the PowerPath information on the "Register to download
software" section on the AX100 support website for a patch to the
version of PowerPath that you just installed. If such a patch is
available, install it, as described in the readme file that accompanies
the patch.
What Next? For information on using PowerPath, refer to PowerPath in Action in
the "Technical descriptions" section on the AX100 support website or
the AX100-Series Documentation CD.
If you are connecting the server to a new AX100SC (single-SP) storage
system, go to Unpacking the Storage System (page 28).
If you are connecting the server to a new AX100 (dual-SP) storage
system, go to Installing the UPS (AX100 Only) (page 27).
If you are connecting a server to an existing storage system (that is,
one already connected to another server), go to Connecting Switches to
the Storage System and Server (page 63).

20
AX100-Series - Installing a Fibre Channel Storage System with a Switch Connection to a Linux Server
Upgrading PowerPath on the Server
Upgrading PowerPath on the Server
Your server must be running PowerPath 4.3.0 to upgrade to a higher
version of 4.3.
Before You Start
❑Check the AX100 support website for the most current
information. EMC updates release notes periodically and posts
them on this website.
❑Run powermt save to ensure that you have saved the latest
PowerPath configuration information.
❑Stop powermt display if it is running. Refer to the PowerPath
Product Guide or the powermt(1) man page for information about
powermt display.
❑Ensure that there are no PowerPath devices in use; for example,
unmount a mounted file system.
❑Make sure that you load the HBA drivers before upgrading
PowerPath. If the HBA drivers are not loaded, your custom
PowerPath settings will be lost during the upgrade process.
❑If the default major and minor numbers associated with a native
device (for example, /dev/sda) have been changed, upgrading to
PowerPath fails to preserve pseudo-to-native device mappings
and device policy/priority settings. To avoid this problem:
1. Run powermt display dev=all and redirect the output to a
file.
2. Run the /etc/init.d/PowerPath stop.
3. Delete all the native devices listed in the powermt log file
created in step 1.
4. Use the mknod to recreate the native devices with the default
major/minor numbers.
5. Delete the /etc/opt/emcpower/emcpower.conf file.
6. Run the /etc/init.d/PowerPath start.