IBM SAS RAID Series User manual

Power Systems
SAS RAID controllers for IBM i


Power Systems
SAS RAID controllers for IBM i


Note
Before using this information and the product it supports, read the information in “Notices,” on
page 57, “Safety notices” on page v, the IBM Systems Safety Notices manual, G229-9054, and the
IBM Environmental Notices and User Guide, Z125–5823.
This edition applies to IBM Power Systems™servers that contain the POWER6®processor and to all associated
models.
© Copyright International Business Machines Corporation 2009.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.

Contents
Safety notices .................................v
Chapter 1. What’s new in SAS RAID controllers for IBM i ..............1
Chapter 2. SAS RAID controllers for IBM i .....................3
Feature comparison of SAS RAID cards ..........................3
PCI-X SAS RAID card comparison ...........................3
PCIe SAS RAID card comparison ...........................6
SAS architecture ..................................8
Disk arrays ....................................9
Supported RAID levels ..............................10
RAID 5 ...................................10
RAID 6 ...................................11
System mirroring ...............................11
Disk array capacities ...............................11
RAID level summary ...............................12
Valid states for disk arrays and physical disk units ......................12
States for disk arrays ...............................12
States for disk units ...............................13
Auxiliary write cache ................................13
Auxiliary write cache adapter ............................14
Viewing link status information ...........................15
Chapter 3. Controller software .........................17
Verifying the controller software ............................17
Chapter 4. Common controller and disk array management tasks ..........19
Viewing IBM SAS disk information ...........................19
Considerations for solid-state drives ...........................19
Chapter 5. Dual storage IOA configurations ....................21
Possible disk storage IOA configurations .........................21
Dual storage IOA functions ..............................22
Dual storage IOA function attributes ...........................24
Viewing dual storage IOA attributes ...........................24
SAS cabling considerations ..............................25
Performance considerations ..............................26
Dual storage IOA access optimization ..........................26
Installing dual storage IOA configurations .........................28
Chapter 6. SAS RAID controller maintenance ...................31
Rechargeable battery maintenance ............................31
Displaying rechargeable battery information .......................31
Error state ...................................33
Forcing a rechargeable battery error ..........................34
Replacing a battery pack ...............................34
Replacing a 572B nonconcurrent maintainable battery pack ..................35
Replacing a 57B7 concurrent maintainable battery pack....................37
Replacing a 574E concurrent maintainable battery pack....................38
Replacing a 572F/575C card set concurrent maintainable battery pack ...............39
Separating the 572F/575C card set and moving the cache directory card ...............40
Viewing SAS fabric path information ...........................45
Example: Using SAS fabric path information ........................49
© Copyright IBM Corp. 2009 iii

Chapter 7. SAS address and physical location information .............55
Appendix. Notices ...............................57
Trademarks ...................................58
Electronic emission notices ..............................58
Class A Notices .................................58
Terms and conditions ................................62
iv SAS RAID controllers for IBM i

Safety notices
Safety notices may be printed throughout this guide:
vDANGER notices call attention to a situation that is potentially lethal or extremely hazardous to
people.
vCAUTION notices call attention to a situation that is potentially hazardous to people because of some
existing condition.
vAttention notices call attention to the possibility of damage to a program, device, system, or data.
World Trade safety information
Several countries require the safety information contained in product publications to be presented in their
national languages. If this requirement applies to your country, a safety information booklet is included
in the publications package shipped with the product. The booklet contains the safety information in
your national language with references to the U.S. English source. Before using a U.S. English publication
to install, operate, or service this product, you must first become familiar with the related safety
information in the booklet. You should also refer to the booklet any time you do not clearly understand
any safety information in the U.S. English publications.
German safety information
Das Produkt ist nicht für den Einsatz an Bildschirmarbeitsplätzen im Sinne§2der
Bildschirmarbeitsverordnung geeignet.
Laser safety information
IBM®servers can use I/O cards or features that are fiber-optic based and that utilize lasers or LEDs.
Laser compliance
All lasers are certified in the U.S. to conform to the requirements of DHHS 21 CFR Subchapter J for class
1 laser products. Outside the U.S., they are certified to be in compliance with IEC 60825 as a class 1 laser
product. Consult the label on each part for laser certification numbers and approval information.
CAUTION:
This product might contain one or more of the following devices: CD-ROM drive, DVD-ROM drive,
DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information:
vDo not remove the covers. Removing the covers of the laser product could result in exposure to
hazardous laser radiation. There are no serviceable parts inside the device.
vUse of the controls or adjustments or performance of procedures other than those specified herein
might result in hazardous radiation exposure.
(C026)
CAUTION:
Data processing environments can contain equipment transmitting on system links with laser modules
that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical
fiber cable or open receptacle. (C027)
CAUTION:
This product contains a Class 1M laser. Do not view directly with optical instruments. (C028)
© Copyright IBM Corp. 2009 v

CAUTION:
Some laser products contain an embedded Class 3A or Class 3B laser diode. Note the following
information: laser radiation when open. Do not stare into the beam, do not view directly with optical
instruments, and avoid direct exposure to the beam. (C030)
Power and cabling information for NEBS (Network Equipment-Building System)
GR-1089-CORE
The following comments apply to the IBM servers that have been designated as conforming to NEBS
(Network Equipment-Building System) GR-1089-CORE:
The equipment is suitable for installation in the following:
vNetwork telecommunications facilities
vLocations where the NEC (National Electrical Code) applies
The intrabuilding ports of this equipment are suitable for connection to intrabuilding or unexposed
wiring or cabling only. The intrabuilding ports of this equipment must not be metallically connected to the
interfaces that connect to the OSP (outside plant) or its wiring. These interfaces are designed for use as
intrabuilding interfaces only (Type 2 or Type 4 ports as described in GR-1089-CORE) and require isolation
from the exposed OSP cabling. The addition of primary protectors is not sufficient protection to connect
these interfaces metallically to OSP wiring.
Note: All Ethernet cables must be shielded and grounded at both ends.
The ac-powered system does not require the use of an external surge protection device (SPD).
The dc-powered system employs an isolated DC return (DC-I) design. The DC battery return terminal
shall not be connected to the chassis or frame ground.
vi SAS RAID controllers for IBM i

Chapter 1. What’s new in SAS RAID controllers for IBM i
This topic collection is new for the October 2009 release.
© Copyright IBM Corp. 2009 1

2SAS RAID controllers for IBM i

Chapter 2. SAS RAID controllers for IBM i
Find usage and maintenance information regarding controllers for the serial-attached SCSI (SAS)
Redundant Array of Independent Disks (RAID) for IBM i. Use this information with your specific system
unit and operating system documentation. General information is intended for all users of this product.
Service information is intended for a service representative trained on the system unit and the subsystem
being serviced.
The SAS RAID controllers for IBM i have the following features:
vPCI-X 266 system interface or PCI Express (PCIe) system interface.
vPhysical link speed of 3 Gbps SAS that supports transfer rates of 300 MB per second.
vSupport for SAS devices and nondisk Serial Advanced Technology Attachment (SATA) devices.
vOptimized for SAS disk configurations that use dual paths through dual expanders for redundancy
and reliability.
vController managed path redundancy and path switching for multiported SAS devices.
vEmbedded PowerPC®Reduced Instruction Set Computer (RISC) processor, hardware XOR Direct
Memory Access engine, and hardware finite field multiplier (FFM) DMA engine for RAID 6.
vSome adapters that support nonvolatile write cache.
vSupport for RAID 5 and RAID 6 disk arrays and system mirroring.
vSupports attachment of other devices such as non-RAID disks, tape, and optical devices.
vRAID disk arrays and non-RAID devices supported as a bootable device.
vAdvanced RAID features:
– Hot spares for RAID 5 and 6 disk arrays and system mirroring.
– Ability to increase the capacity of an existing RAID 5 or 6 disk array by adding disks.
– Background parity checking.
– Background data scrubbing.
– Disks formatted to 528 bytes per sector, providing cyclical redundancy checking (CRC) and logically
bad-block checking.
– Optimized hardware for RAID 5 and 6 sequential write workloads.
– Optimized skip read-and-write disk support for transaction workloads.
vSupports a maximum of 64 advanced function disks with a maximum of 255 devices.
Note: The number of all physical SAS and SATA devices plus the number of logical RAID disk arrays
must be less than 255 per controller.
Feature comparison of SAS RAID cards
Compare the main features of PCI-X and PCI Express (PCIe) SAS RAID cards for IBM i.
The tables in this section provide a breakdown of the main features of the SAS RAID PCI-X and PCIe
controller cards.
PCI-X SAS RAID card comparison
Use the table in this topic to compare the features of PCI-X SAS RAID cards for IBM i. There are also
images of adapters for you to view.
© Copyright IBM Corp. 2009 3

Table 1. PCI-X SAS RAID controller card comparison
Features 572A 572C 572F and 575C 57B8
Description PCI-X 266 Ext
Dual-x4 3 Gb SAS
adapter
PCI-X 266 planar 3
Gb SAS adapter
PCI-X 266 Ext Tri-x4 3
Gb SAS RAID
adapter
PCI-X 266 planar 3
Gb SAS RAID
adapter
Form factor Low profile 64 bit
PCI-X
Planar integrated Long 64 bit PCI-X,
double-wide card set
Planar RAID
enablement
Physical links 8 (two mini SAS 4x
connectors)
8112 (bottom 3 mini
SAS 4x connectors)
and 2 (top mini SAS
4x connector for high
availability only)
81
RAID levels
supported
RAID 53, RAID 63,
system mirroring
System mirroring RAID 5, RAID 6,
system mirroring
RAID 5, RAID 6,
system mirroring
Write cache size Up to 1.5 Gb
(compressed)
175 MB
Read cache size Up to 1.6 Gb
(compressed)
Cache battery pack
technology
Lithium ion Not applicable2
Cache battery
concurrent
maintenance
No No Yes4Not applicable2
Cache data present
LED
No No No No
Removable cache card No No No No
Auxiliary write cache
(AWC) support
No No Yes Yes
Dual storage IOA
configuration
No No Yes No
Requires dual storage
IOA configuration
No No No No
1. Some systems provide an external mini SAS 4x connector from the integrated backplane controller.
2. The controller contains battery-backed cache, but the battery power is supplied by the 57B7 controller through
the backplane connections.
3. The write performance of RAID 5 and RAID 6 might be poor on adapters that do not provide write cache.
Consider using an adapter that provides write cache when using RAID 5 or RAID 6.
4. The cache battery pack for both adapters is contained in a single battery field-replaceable Unit (FRU), which is
physically located on the 575C Auxiliary Cache card.
Adapter graphics
View the SAS RAID controllers.
4SAS RAID controllers for IBM i

Figure 1. CCIN 572A PCI-X266 External Dual-x4 3 Gb SAS adapter
Figure 2. CCIN 57B8 planar RAID enablement card
Chapter 2. SAS RAID controllers for IBM i 5

Related concepts
Chapter 5, “Dual storage IOA configurations,” on page 21
You can increase availability using a dual storage I/O adapter (IOA) configuration to connect multiple
controllers to a common set of disk expansion drawers and the included disks and disk arrays.
“Dual storage IOA functions” on page 22
Consider these factors when using dual storage I/O adapter (IOA) functions.
PCIe SAS RAID card comparison
Use the table in this topic to compare the features of a PCI Express (PCIe) SAS RAID cards for IBM i.
There are also images of adapters for you to view.
Table 2. PCIe SAS RAID controller card comparison
Features 57B7 57B3 574E
Description PCIe x1 Auxiliary Cache
adapter
PCIe x8 Ext Dual-x4 3 Gb
SAS adapter
PCIe x8 Ext Dual-x4 3 Gb
SAS RAID adapter
Form factor Planar Auxiliary Cache PCIe x8 PCIe x8
Physical links 2 8 (two mini SAS 4x
connectors)
8 (two mini SAS 4x
connectors)
RAID levels supported RAID 51, RAID 61, system
mirroring
RAID 5, RAID 6, system
mirroring
Write cache size 175 MB 380 MB
Read cache size
Cache battery pack
technology
Lithium ion Lithium ion
Cache battery concurrent
maintenance
Yes No Yes
Cache data present LED Yes No Yes
Removable cache card No No Yes
Figure 3. CCIN 572F PCI-X266 Ext Tri-x4 3 Gb SAS RAID adapter and CCIN 575C PCI-X266 Auxiliary Cache adapter
6SAS RAID controllers for IBM i

Table 2. PCIe SAS RAID controller card comparison (continued)
Features 57B7 57B3 574E
Auxiliary write cache
(AWC) support
Yes No No
Dual storage IOA
configuration
No No Yes
Requires dual storage IOA
configuration
No No Yes
1. The write performance of RAID 5 and RAID 6 might be poor on adapters that do not provide write cache.
Consider using an adapter that provides write cache when using RAID 5 or RAID 6.
Adapter graphics
View the SAS RAID controllers.
Figure 4. CCIN 57B7 Planar Auxiliary Cache
Figure 5. CCIN 57B3 PCIe x8 Ext Dual-x4 3 Gb SAS adapter
Chapter 2. SAS RAID controllers for IBM i 7

Related concepts
Chapter 5, “Dual storage IOA configurations,” on page 21
You can increase availability using a dual storage I/O adapter (IOA) configuration to connect multiple
controllers to a common set of disk expansion drawers and the included disks and disk arrays.
“Dual storage IOA functions” on page 22
Consider these factors when using dual storage I/O adapter (IOA) functions.
SAS architecture
Serial-attached SCSI (SAS) architecture defines a serial device interconnect and transport protocol that
defines the rules for information exchange between devices.
SAS is an evolution of the parallel SCSI device interface into a serial point-to-point interface. SAS
physical links (phys) are a set of four wires used as two differential signal pairs. One differential signal
transmits in one direction, while the other differential signal transmits in the opposite direction. Data can
be transmitted in both directions simultaneously. Phys are contained in SAS ports which contain one or
more phys. A port is a wide port if there are more than one phy in the port. If there is only one phy in
the port, it is a narrow port. A port is identified by a unique SAS worldwide name (also called SAS
address).
A SAS controller contains one or more SAS ports. A path is a logical point-to-point link between a SAS
initiator port in the controller and a SAS target port in the I/O device (for example a disk). A connection
is a temporary association between a controller and an I/O device through a path. A connection enables
communication to a device. The controller can communicate to the I/O device over this connection by
using either the SCSI command set or the ATA/ATAPI command set depending on the device type.
A SAS expander enables connections between a controller port and multiple I/O device ports by routing
connections between the expander ports. Only a single connection through an expander can exist at any
Figure 6. CCIN 574E PCIe x8 Ext Dual-x4 3 Gb SAS RAID adapter
8SAS RAID controllers for IBM i

given time. Using expanders creates more nodes in the path from the controller to the I/O device. If an
I/O device supports multiple ports, more than one path to the device can exist when there are expander
devices included in the path.
ASAS fabric refers to the summation of all paths between all SAS controller ports and all I/O device
ports in the SAS subsystem including cables, enclosures, and expanders.
The following example SAS subsystem shows some of the concepts described in this SAS overview. A
controller is shown with eight SAS phys. Four of those phys are connected into two different wide ports.
One connector contains four phys grouped into two ports. The connectors have no significance in SAS
other than causing a physical wire connection. The four-phy connector can contain between one and four
ports depending on the type of cabling that is used. The uppermost port in the figure shows a
controller-wide port number 6 that consists of phy numbers 6 and 7. Port 6 connects to an expander,
which attaches to one of the dual ports of the I/O devices. The dashed red line indicates a path between
the controller and an I/O device. Another path runs from the controller’s port number 4 to the other port
of the I/O device. These two paths provide two different possible connections for increased reliability by
using redundant controller ports, expanders, and I/O device ports. The SCSI Enclosure Services (SES) is a
component of each expander.
Disk arrays
Disk arrays are groups of disks that work together with a specialized array controller to take advantage of
potentially higher data transfer rates and data redundancy.
Disk arrays use RAID technology to offer data redundancy and to provide improved data transfer rates
over single large disks. If a disk failure occurs, the disk can typically be replaced without interrupting
normal system operation.
Data redundancy
The disk array controller tracks how the data is distributed across the disks. RAID 5 and RAID 6 disk
arrays provide data redundancy, ensuring that data is not lost if a disk in the array fails. If a disk failure
occurs, the disk can typically be replaced without interrupting normal system operations. System
mirroring provides data redundancy by mirroring the same data across pairs of disks.
Figure 7. Example SAS Subsystem
Chapter 2. SAS RAID controllers for IBM i 9

Supported RAID levels
The RAID level of a disk array determines how data is stored on the disk array and the level of
protection that is provided.
If a part of the RAID system fails, different RAID levels help to recover lost data in different ways. If a
single drive fails within an array, the array controller can reconstruct the data for the failed disk by using
the data stored on other hard drives within the array. This data reconstruction has little or no impact to
current system programs and users. The controller supports RAID levels 5 and 6 as well as system
mirroring. Not all controllers support all RAID levels. Each RAID level supported by the controller has its
own attributes and uses a different method of writing data. The following information provides details
for each supported RAID level.
Related concepts
“PCI-X SAS RAID card comparison” on page 3
Use the table in this topic to compare the features of PCI-X SAS RAID cards for IBM i. There are also
images of adapters for you to view.
“PCIe SAS RAID card comparison” on page 6
Use the table in this topic to compare the features of a PCI Express (PCIe) SAS RAID cards for IBM i.
There are also images of adapters for you to view.
Related information
Device parity protection concepts
RAID 5
Learn how data is written to a RAID 5 array.
RAID 5 stripes data across all disks in the array. RAID level 5 also writes array parity data. The parity
data is spread across all the disks. For a RAID 5 array of three disks, array data and parity information
are written in the following pattern:
If a disk fails in a RAID 5 array, you can continue to use the array normally. A RAID 5 array operating
with a single Failed disk is said to be operating in Degraded mode. Whenever data is read from a
Degraded disk array, the array controller recalculates the data on the Failed disk by using data and parity
blocks on the operational disks. If a second disk fails, the array will be placed in the Failed state and will
not be accessible.
Figure 8. RAID 5
10 SAS RAID controllers for IBM i

Related information
RAID 5 concepts
RAID 6
Learn how data is written to a RAID 6 array.
RAID 6 stripes data across all disks in the array. RAID level 6 also writes array ″P″and ″Q″parity data.
The P and Q parity data, is spread across all the disks. For a RAID 6 array of four disks, array data and
parity information are written in the following pattern:
If one or two disks fail in a RAID 6 array, you can continue to use the array normally. A RAID 6 array
operating with a one or two Failed disks is said to be operating in Degraded mode. Whenever data is
read from a Degraded disk array, the array controller recalculates the data on the failed disks by using
data and parity blocks on the operational disks. A RAID 6 array with a single failed disk has similar
protection to that of a RAID 5 array with no disk failures. If a third disk fails, the array will be placed in
the Failed state and will not be accessible.
Related information
RAID 6 concepts
System mirroring
Mirrored protection is beneficial if you have a multibus system or a system with a large single bus. A
greater number of disk units provides more opportunity for failure and increased recovery time.
Refer to Mirrored protection for more information.
Disk array capacities
These guidelines will help you calculate the capacity of a disk array.
The capacity of a disk array depends on the capacity of the disks used and the RAID level of the array.
To calculate the capacity of a disk array, do the following:
RAID 5
Multiply one fewer than the number of disks by the disk capacity.
Figure 9. RAID 6
Chapter 2. SAS RAID controllers for IBM i 11

RAID 6
Multiply two fewer than the number of disks by the disk capacity.
System mirroring
Multiply the number of disks by the disk capacity and divide by two.
Note: If disks of different capacities are used in the same disk array, all disks are treated as if they have
the capacity of the smallest disk.
RAID level summary
Compare RAID levels according to their capabilities.
The following information provides data redundancy, usable disk capacity, read performance, and write
performance for each RAID level.
Table 3. RAID level summary
RAID level Data redundancy
Usable disk
capacity
Read
performance
Write
performance Devices per array
RAID 5 Very good 67% - 94% Very good Good Minimum: 3
Maximum: 18
RAID 6 Excellent 50% - 89% Very good Fair to good Minimum: 4
Maximum: 18
System mirroring Excellent 50% Excellent Very good Not applicable
RAID 5
Creates array parity information so that the data can be reconstructed if a disk in the array fails.
Provides better capacity than System Mirroring but possibly lower performance.
RAID 6
Creates array ″P″and ″Q″parity information so that the data can be reconstructed if one or two
disks in the array fail. Provides better data redundancy than RAID 5 but with slightly lower
capacity and possibly lower performance. Provides better capacity than System Mirroring but
possibly lower performance.
System Mirroring
Stores data redundantly on mirrored pairs to provide maximum protection against disk failures.
Provides generally better performance than RAID 5 or 6, but has lower capacity.
Valid states for disk arrays and physical disk units
Disk arrays and physical disks have several operational states.
States for disk arrays
There are four valid states for disk arrays.
View disk arrays by completing the following steps:
1. Select Work with disk units on the Use System Service Tools (SST) menu.
2. Select Display disk configuration from the Work with Disk Configuration display.
3. Select Display device parity status on the Display Disk Configuration display.
The valid four states for IBM SAS Disk Arrays are RAID 5, RAID 6, RAID 5/Availability, and RAID
6/Availability.
12 SAS RAID controllers for IBM i