IBM Power System 5888 Operating and installation instructions

Power Systems
Service information for the PCIe
storage enclosures


Power Systems
Service information for the PCIe
storage enclosures


Note
Before using this information and the product it supports, read the information in “Safety notices” on page v, “Notices” on
page 43, the IBM Systems Safety Notices manual, G229-9054, and the IBM Environmental Notices and User Guide, Z125–5823.
This edition applies to IBM Power Systems™servers that contain the POWER7 processor and to all associated
models.
© Copyright IBM Corporation 2012, 2013.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.

Contents
Safety notices .................................v
Service information for the PCIe storage enclosures ................1
Overview of 5888 or EDR1 PCIe storage enclosure service ....................1
PCIe storage enclosure infrastructure ..........................2
PCIe storage enclosure features .............................3
Cabling the PCIe storage enclosure ............................4
Cabling the 5888 PCIe storage enclosure to an 8202-E4C or 8205-E6C system .............4
Cabling the EDR1 PCIe storage enclosure to an 8202-E4D or 8205-E6D system ............9
Cabling the 5887 disk drive enclosure to an EDR1 PCIe storage enclosure that is attached to an 8202-E4D or
8205-E6D ..................................14
Cabling the 5888 PCIe storage enclosure to an 8231-E1C or 8231-E2C system ............15
Cabling the EDR1 PCIe storage enclosure to an 8231-E1D, 8231-E2D, or 8268-E1D system ........18
Cabling the 5887 disk drive enclosure to an EDR1 PCIe storage enclosure that is attached to an 8231-E1D,
8231-E2D, or 8268-E1D .............................20
Cabling the EDR1 PCIe storage enclosure to an 8246-L2T system ................20
Cabling the 5887 disk drive enclosure to an EDR1 PCIe storage enclosure that is attached to an 8246-L2T . . 22
Cabling the EDR1 PCIe storage enclosure to an 8248-L4T, 8408-E8D, or 9109-RMD system ........23
Cabling the 5887 disk drive enclosure to an EDR1 PCIe storage enclosure that is attached to an 8248-L4T,
8408-E8D, or 9109-RMD .............................25
Cabling the EDR1 PCIe storage enclosure to an 8412-EAD, 9117-MMC, 9117-MMD, 9179-MHC, or 9179-MHD
system ....................................26
Cabling the 5887 disk drive enclosure to an EDR1 PCIe storage enclosure that is attached to an 8412-EAD,
9117-MMC, 9117-MMD, 9179-MHC, or 9179-MHD ....................31
Overview of problem determination ...........................33
LED use for the 5888 or EDR1 PCIe storage enclosure ....................34
5888 or EDR1 PCIe storage enclosure light-emitting diode (LED) locations. .............34
RAID controller location code ............................35
Determining the ERM location from the RAID controller location ...............36
Determining the ERM location from the RAID controller location when using AIX ........36
Determining the ERM location from the RAID controller location when using IBM i ........37
Determining the ERM location from the RAID controller location when using Linux ........38
Updating microcode for PCIe storage enclosure components ..................39
FRUs for the 5888 or EDR1 PCIe storage enclosure .....................40
Identifying a SAS fabric FRU for the 5888 or EDR1 PCIe storage enclosure .............40
Storage enclosure topology .............................41
Notices ...................................43
Trademarks ...................................44
Electronic emission notices ..............................44
Class A Notices .................................44
Class B Notices .................................48
Terms and conditions ................................51
© Copyright IBM Corp. 2012, 2013 iii

iv Power Systems: Service information for the PCIe storage enclosures

Safety notices
Safety notices may be printed throughout this guide:
vDANGER notices call attention to a situation that is potentially lethal or extremely hazardous to
people.
vCAUTION notices call attention to a situation that is potentially hazardous to people because of some
existing condition.
vAttention notices call attention to the possibility of damage to a program, device, system, or data.
World Trade safety information
Several countries require the safety information contained in product publications to be presented in their
national languages. If this requirement applies to your country, safety information documentation is
included in the publications package (such as in printed documentation, on DVD, or as part of the
product) shipped with the product. The documentation contains the safety information in your national
language with references to the U.S. English source. Before using a U.S. English publication to install,
operate, or service this product, you must first become familiar with the related safety information
documentation. You should also refer to the safety information documentation any time you do not
clearly understand any safety information in the U.S. English publications.
Replacement or additional copies of safety information documentation can be obtained by calling the IBM
Hotline at 1-800-300-8751.
German safety information
Das Produkt ist nicht für den Einsatz an Bildschirmarbeitsplätzen im Sinne§2der
Bildschirmarbeitsverordnung geeignet.
Laser safety information
IBM®servers can use I/O cards or features that are fiber-optic based and that utilize lasers or LEDs.
Laser compliance
IBM servers may be installed inside or outside of an IT equipment rack.
© Copyright IBM Corp. 2012, 2013 v

DANGER
When working on or around the system, observe the following precautions:
Electrical voltage and current from power, telephone, and communication cables are hazardous. To
avoid a shock hazard:
vConnect power to this unit only with the IBM provided power cord. Do not use the IBM
provided power cord for any other product.
vDo not open or service any power supply assembly.
vDo not connect or disconnect any cables or perform installation, maintenance, or reconfiguration
of this product during an electrical storm.
vThe product might be equipped with multiple power cords. To remove all hazardous voltages,
disconnect all power cords.
vConnect all power cords to a properly wired and grounded electrical outlet. Ensure that the outlet
supplies proper voltage and phase rotation according to the system rating plate.
vConnect any equipment that will be attached to this product to properly wired outlets.
vWhen possible, use one hand only to connect or disconnect signal cables.
vNever turn on any equipment when there is evidence of fire, water, or structural damage.
vDisconnect the attached power cords, telecommunications systems, networks, and modems before
you open the device covers, unless instructed otherwise in the installation and configuration
procedures.
vConnect and disconnect cables as described in the following procedures when installing, moving,
or opening covers on this product or attached devices.
To Disconnect:
1. Turn off everything (unless instructed otherwise).
2. Remove the power cords from the outlets.
3. Remove the signal cables from the connectors.
4. Remove all cables from the devices.
To Connect:
1. Turn off everything (unless instructed otherwise).
2. Attach all cables to the devices.
3. Attach the signal cables to the connectors.
4. Attach the power cords to the outlets.
5. Turn on the devices.
(D005)
DANGER
vi Power Systems: Service information for the PCIe storage enclosures

Observe the following precautions when working on or around your IT rack system:
vHeavy equipment–personal injury or equipment damage might result if mishandled.
vAlways lower the leveling pads on the rack cabinet.
vAlways install stabilizer brackets on the rack cabinet.
vTo avoid hazardous conditions due to uneven mechanical loading, always install the heaviest
devices in the bottom of the rack cabinet. Always install servers and optional devices starting
from the bottom of the rack cabinet.
vRack-mounted devices are not to be used as shelves or work spaces. Do not place objects on top
of rack-mounted devices.
vEach rack cabinet might have more than one power cord. Be sure to disconnect all power cords in
the rack cabinet when directed to disconnect power during servicing.
vConnect all devices installed in a rack cabinet to power devices installed in the same rack
cabinet. Do not plug a power cord from a device installed in one rack cabinet into a power
device installed in a different rack cabinet.
vAn electrical outlet that is not correctly wired could place hazardous voltage on the metal parts of
the system or the devices that attach to the system. It is the responsibility of the customer to
ensure that the outlet is correctly wired and grounded to prevent an electrical shock.
CAUTION
vDo not install a unit in a rack where the internal rack ambient temperatures will exceed the
manufacturer's recommended ambient temperature for all your rack-mounted devices.
vDo not install a unit in a rack where the air flow is compromised. Ensure that air flow is not
blocked or reduced on any side, front, or back of a unit used for air flow through the unit.
vConsideration should be given to the connection of the equipment to the supply circuit so that
overloading of the circuits does not compromise the supply wiring or overcurrent protection. To
provide the correct power connection to a rack, refer to the rating labels located on the
equipment in the rack to determine the total power requirement of the supply circuit.
v(For sliding drawers.) Do not pull out or install any drawer or feature if the rack stabilizer brackets
are not attached to the rack. Do not pull out more than one drawer at a time. The rack might
become unstable if you pull out more than one drawer at a time.
v(For fixed drawers.) This drawer is a fixed drawer and must not be moved for servicing unless
specified by the manufacturer. Attempting to move the drawer partially or completely out of the
rack might cause the rack to become unstable or cause the drawer to fall out of the rack.
(R001)
Safety notices vii

CAUTION:
Removing components from the upper positions in the rack cabinet improves rack stability during
relocation. Follow these general guidelines whenever you relocate a populated rack cabinet within a
room or building:
vReduce the weight of the rack cabinet by removing equipment starting at the top of the rack
cabinet. When possible, restore the rack cabinet to the configuration of the rack cabinet as you
received it. If this configuration is not known, you must observe the following precautions:
– Remove all devices in the 32U position and above.
– Ensure that the heaviest devices are installed in the bottom of the rack cabinet.
– Ensure that there are no empty U-levels between devices installed in the rack cabinet below the
32U level.
vIf the rack cabinet you are relocating is part of a suite of rack cabinets, detach the rack cabinet from
the suite.
vInspect the route that you plan to take to eliminate potential hazards.
vVerify that the route that you choose can support the weight of the loaded rack cabinet. Refer to the
documentation that comes with your rack cabinet for the weight of a loaded rack cabinet.
vVerify that all door openings are at least 760 x 230 mm (30 x 80 in.).
vEnsure that all devices, shelves, drawers, doors, and cables are secure.
vEnsure that the four leveling pads are raised to their highest position.
vEnsure that there is no stabilizer bracket installed on the rack cabinet during movement.
vDo not use a ramp inclined at more than 10 degrees.
vWhen the rack cabinet is in the new location, complete the following steps:
– Lower the four leveling pads.
– Install stabilizer brackets on the rack cabinet.
– If you removed any devices from the rack cabinet, repopulate the rack cabinet from the lowest
position to the highest position.
vIf a long-distance relocation is required, restore the rack cabinet to the configuration of the rack
cabinet as you received it. Pack the rack cabinet in the original packaging material, or equivalent.
Also lower the leveling pads to raise the casters off of the pallet and bolt the rack cabinet to the
pallet.
(R002)
(L001)
(L002)
viii Power Systems: Service information for the PCIe storage enclosures

(L003)
or
All lasers are certified in the U.S. to conform to the requirements of DHHS 21 CFR Subchapter J for class
1 laser products. Outside the U.S., they are certified to be in compliance with IEC 60825 as a class 1 laser
product. Consult the label on each part for laser certification numbers and approval information.
CAUTION:
This product might contain one or more of the following devices: CD-ROM drive, DVD-ROM drive,
DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following information:
vDo not remove the covers. Removing the covers of the laser product could result in exposure to
hazardous laser radiation. There are no serviceable parts inside the device.
vUse of the controls or adjustments or performance of procedures other than those specified herein
might result in hazardous radiation exposure.
(C026)
Safety notices ix

CAUTION:
Data processing environments can contain equipment transmitting on system links with laser modules
that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical
fiber cable or open receptacle. (C027)
CAUTION:
This product contains a Class 1M laser. Do not view directly with optical instruments. (C028)
CAUTION:
Some laser products contain an embedded Class 3A or Class 3B laser diode. Note the following
information: laser radiation when open. Do not stare into the beam, do not view directly with optical
instruments, and avoid direct exposure to the beam. (C030)
CAUTION:
The battery contains lithium. To avoid possible explosion, do not burn or charge the battery.
Do Not:
v___ Throw or immerse into water
v___ Heat to more than 100°C (212°F)
v___ Repair or disassemble
Exchange only with the IBM-approved part. Recycle or discard the battery as instructed by local
regulations. In the United States, IBM has a process for the collection of this battery. For information,
call 1-800-426-4333. Have the IBM part number for the battery unit available when you call. (C003)
Power and cabling information for NEBS (Network Equipment-Building System)
GR-1089-CORE
The following comments apply to the IBM servers that have been designated as conforming to NEBS
(Network Equipment-Building System) GR-1089-CORE:
The equipment is suitable for installation in the following:
vNetwork telecommunications facilities
vLocations where the NEC (National Electrical Code) applies
The intrabuilding ports of this equipment are suitable for connection to intrabuilding or unexposed
wiring or cabling only. The intrabuilding ports of this equipment must not be metallically connected to the
interfaces that connect to the OSP (outside plant) or its wiring. These interfaces are designed for use as
intrabuilding interfaces only (Type 2 or Type 4 ports as described in GR-1089-CORE) and require isolation
from the exposed OSP cabling. The addition of primary protectors is not sufficient protection to connect
these interfaces metallically to OSP wiring.
Note: All Ethernet cables must be shielded and grounded at both ends.
The ac-powered system does not require the use of an external surge protection device (SPD).
The dc-powered system employs an isolated DC return (DC-I) design. The DC battery return terminal
shall not be connected to the chassis or frame ground.
xPower Systems: Service information for the PCIe storage enclosures

Service information for the PCIe storage enclosures
This collection provides service information for the 5888 or EDR1 Peripheral Component Interconnect
Express (PCIe) storage enclosure.
Overview of 5888 or EDR1 PCIe storage enclosure service
Find information to help you service the 5888 or EDR1 PCIe storage enclosure.
The IBM EXP30 Ultra SSD I/O Drawer (feature code 5888 and EDR1) is a 1.8-inch solid-state drive (SSD)
optimized PCIe storage enclosure that is mounted in a 19-inch rack. It attaches to supported POWER7®
processor-based systems by using PCI Express generation 2 (PCIe2) GX++ adapters and PCIe2 cables. The
5888 or EDR1 PCIe storage enclosure includes the following components:
vRedundant power supplies
vRedundant fans
vTwo redundant, large cache PCIe2 serial-attached SCSI (SAS) redundant array of independent disks
(RAID) controllers along with redundant SAS port expanders
vUp to 30 1.8-inch dual port SAS SSDs
The 5888 and EDR1 PCIe storage enclosures contain two redundant enclosure RAID modules (ERMs),
which contain a SAS RAID controller, a SAS port expander, and a replaceable fan assembly. Two power
supplies provide an N+1 redundant feature that allows one power supply to power the entire enclosure if
the other power supply failed or is missing.
vThe two power supplies have built-in fans.
vFan speed is automatically adjusted, as needed, to ensure correct cooling temperatures of the enclosure
even when a fan fails.
The SAS RAID controllers are always configured in a dual controller (dual storage I/O adapter)
configuration. This configuration provides redundant access paths to the SAS devices and mirrored copies
of cache data and parity update footprints. The dual controller configuration also allows for increased
performance capability when multiple RAID arrays are used in an Active/Active configuration mode.
The SAS RAID controllers support RAID 0, 10, 5, and 6 and the hot-spare function.
The 5888 or EDR1 PCIe storage enclosure is a one EIA unit drawer that is mounted in a 19-inch rack. It is
designed for light path service with light-emitting diode (LED) indicators on the front and rear of the
enclosure chassis. These chassis LED indicators provide status and fault information for the following
components and conditions:
vEnclosure identification
vEnclosure power
vEnclosure fans
vERMs
vEnclosure faults
vSSD activity and SSD fault or identification
The PCIe storage enclosure is designed for front and rear service, performed in place (that is not in a
service position) except for the midplane field replaceable unit (FRU) replacement.
© Copyright IBM Corp. 2012, 2013 1

PCIe storage enclosure infrastructure
Learn how the 5888 or EDR1 PCIe storage enclosure is different from previous storage enclosures.
With previous IBM SAS storage enclosure models, connecting disk drives was accomplished by using
individual components that were externally connected to each other:
vStorage enclosures
vDisk drive controllers
vSAS cables for multiple paths to devices and dedicated adapter-to-adapter connections
Figure 1 shows an example of a dual storage I/O adapter (IOA) solution by using externally connected
components.
With the 5888 or EDR1 PCIe storage enclosures, all of the necessary components are integrated within the
storage enclosure.
Figure 2 on page 3 shows a dual storage IOA solution that is integrated within the 5888 or EDR1 PCIe
storage enclosure.
Figure 1. Dual storage IOA solution by using externally connected components
2Power Systems: Service information for the PCIe storage enclosures

The following components and functions are integrated within the 5888 or EDR1 PCIe storage enclosure:
vThe enclosure RAID module (ERM) contains the RAID adapter function and the expander function.
vThe storage enclosure uses internal wiring to connect components with multiple redundant paths,
eliminating the need for external SAS cables.
vA maximum of 30 1.5-inch solid-state drives (SSDs) may be installed in the storage enclosure.
PCIe storage enclosure features
Learn about the main features of the 5888 and EDR1 PCI Express (PCIe) storage enclosures.
Table 1. PCIe storage enclosure features
Feature code 5888 EDR1
Description IBM EXP30 Ultra solid-state drive
(SSD) I/O drawer
IBM EXP30 Ultra solid-state drive
(SSD) I/O drawer
System connection GX++ PCIe2 x8 GX++ PCIe2 x8
Systems supported 8202-E4C, 8205-E6C, 8231-E1C, or
8231-E2C
8202-E4D, 8205-E6D, 8231-E1D,
8231-E2D, 8246-L2T, 8248-L4T,
8268-E1D, 8408-E8D, 8412-EAD,
9109-RMD, 9117-MMC, 9117-MMD,
9179-MHC, and 9179-MHD
Operating systems supported1AIX®, Linux AIX, IBM i, Linux
High availability (HA) two-system
redundant array of independent
disks (RAID)
Yes Yes
HA single-system RAID Yes Yes
Requires HA RAID configuration Yes Yes
JBOD drives supported No (requires drives that are
formatted with 528 bytes/block)
No (requires drives that are
formatted with 528 bytes/block)
Internal solid-state drive (SSD) type 1.8-inch small form factor (SFF) 6 GB
Dual Port serial-attached SCSI (SAS)
SSD
1.8-inch small form factor (SFF) 6 GB
Dual Port serial-attached SCSI (SAS)
SSD
SSD drive feature codes supported ES02 ES02 (AIX and Linux), ES04 (IBM i)
Internal SSD drives Six minimum to 30 maximum Six minimum to 30 maximum
Figure 2. Dual storage IOA solution integrated within the 5888 or EDR1 PCIe storage enclosure
Service information 3

Table 1. PCIe storage enclosure features (continued)
Feature code 5888 EDR1
External hard disk drives (HDD) None 48 maximum
External SAS enclosures None Two 5887 disk drive enclosures
maximum
Drive redundant SAS paths Two paths from each enclosure RAID
module (ERM) (one path through
each SAS expander)
Two paths from each enclosure RAID
module (ERM) (one path through
each SAS expander)
Drive concurrent maintenance Yes Yes
Enclosure RAID module (ERM) PCIe2 3.1 GB cache RAID SAS
enclosure 6 GB x8
PCIe2 3.1 GB cache RAID SAS
enclosure 6 GB x8
ERM custom card identification
number (CCIN)
57C3 57C3
ERM RAID levels RAID 0, 5, 6, and 10 with hot spare
for 5, 6, and 10
RAID 0, 5, 6, and 10 with hot spare
for 5, 6, and 10
ERM write cache size 3.1 GB 3.1 GB
ERM external SAS links None Two mini SAS HD 4x connectors that
contain three physical links each
ERM concurrent maintenance No Yes
ERM fan assembly Two redundant fans per ERM Two redundant fans per ERM
ERM fan concurrent maintenance No Yes
Redundant power supplies Two Two
Power supply concurrent
maintenance
Yes Yes
1See Managing PCI adapters
Cabling the PCIe storage enclosure
Use this information as a guide for cabling your storage enclosure.
Cabling the 5888 PCIe storage enclosure to an 8202-E4C or 8205-E6C
system
Use this information as a guide for cabling your PCIe storage enclosure to an 8202-E4C or 8205-E6C
system.
Figure 3 on page 5 shows one 5888 PCIe storage enclosure connected to a single 8202-E4C or 8205-E6C
system by using one feature code (FC) EJ03 PCIe2 GX++ adapter and two PCIe cables.
4Power Systems: Service information for the PCIe storage enclosures

Figure 4 on page 6 shows one 5888 PCIe storage enclosure connected to an 8205-E6C system by using two
FC EJ03 PCIe2 GX++ adapters and two PCIe cables.
Figure 3. One 5888 PCIe storage enclosure connected to a single 8202-E4C or 8205-E6C system by using one FC
EJ03 PCIe2 GX++ adapter and two PCIe cables
Service information 5

Figure 5 shows two 5888 PCIe storage enclosures connected to a single 8205-E6C system by using two FC
EJ03 PCIe2 GX++ adapters and four PCIe cables.
Figure 6 on page 7 shows one 5888 PCIe storage enclosure connected to two 8202-E4C or 8205-E6C
systems by using two FC EJ03 PCIe2 GX++ adapters and two PCIe cables.
Figure 4. One 5888 PCIe storage enclosure connected to a single 8205-E6C system by using two FC EJ03 PCIe2
GX++ adapters and two PCIe cables
Figure 5. Two 5888 PCIe storage enclosures connected to a single 8205-E6C system by using two FC EJ03 PCIe2
GX++ adapters and four PCIe cables
6Power Systems: Service information for the PCIe storage enclosures

Figure 7 on page 8 shows two 5888 PCIe storage enclosures connected to two 8202-E4C or 8205-E6C
systems by using two FC EJ03 PCIe2 GX++ adapters and four PCIe cables.
Figure 6. One 5888 PCIe storage enclosure connected to two 8202-E4C or 8205-E6C systems by using two FC EJ03
PCIe2 GX++ adapters and two PCIe cables
Service information 7

Figure 8 on page 9 shows four 5888 PCIe storage enclosures connected to two 8205-E6C systems by using
four FC EJ03 PCIe2 GX++ adapters and eight PCIe cables.
Figure 7. Two 5888 PCIe storage enclosures connected to two 8202-E4C or 8205-E6C systems by using two FC EJ03
PCIe2 GX++ adapters and four PCIe cables
8Power Systems: Service information for the PCIe storage enclosures