Ibm 9006-22p User guide

Power Systems

Servicing the 9006-22P

IBM



Note

Before using this information and the product it supports, read the information in “Safety notices” on

page v, “Notices” on page 133, the IBM Systems Safety Notices manual, G229-9054, and the IBM

Environmental Notices and User Guide, Z125–5823.

This edition applies to IBM® Power Systems servers that contain the POWER9™ processor and to all associated models.

US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with

IBM Corp.

Contents

Safety notices........................................................................................................v

Removing and replacing parts in the 9006-22P.......................................................1

Disk drive......................................................................................................................................................2

Removing a front disk drive....................................................................................................................2

Replacing a front disk drive....................................................................................................................7

Removing a drive on module................................................................................................................12

Replacing a drive on module................................................................................................................13

Removing a rear drive...........................................................................................................................15

Replacing a rear drive...........................................................................................................................18

Disk drive backplane..................................................................................................................................22

Removing the front disk drive backplane............................................................................................ 22

Replacing the front disk drive backplane............................................................................................ 24

Removing the rear disk drive backplane..............................................................................................26

Replacing the rear disk drive backplane..............................................................................................30

Fans............................................................................................................................................................ 34

Removing a system fan........................................................................................................................ 34

Replacing a system fan........................................................................................................................ 36

Removing a system fan cable ..............................................................................................................37

Replacing a system fan cable...............................................................................................................42

Memory...................................................................................................................................................... 46

PCIe adapters............................................................................................................................................ 49

Removing a PCIe adapter.....................................................................................................................50

Replacing a PCIe adapter.....................................................................................................................57

Power supplies...........................................................................................................................................63

Removing a power supply.................................................................................................................... 63

Replacing a power supply.................................................................................................................... 65

System backplane......................................................................................................................................66

Removing the system backplane......................................................................................................... 66

Replacing the system backplane......................................................................................................... 73

System processor module......................................................................................................................... 88

Removing a system processor module................................................................................................88

Replacing a system processor module................................................................................................ 95

Time-of-day battery................................................................................................................................ 103

TPM card.................................................................................................................................................. 105

Common procedures for servicing or installing features in the 5104-22C,

9006-22C, or 9006-22P system...................................................................... 109

Before you begin servicing...................................................................................................................... 109

Identifying the 5104-22C, 9006-12P, 9006-22C, or 9006-22P system that contains the part to

replace................................................................................................................................................ 112

LEDs on the 5104-22C, 9006-22C, or 9006-22P system................................................................ 112

Identifying the 5104-22C, 9006-12P, 9006-22C, or 9006-22P that needs servicing................... 114

Preparing the system to remove and replace internal parts.................................................................. 115

Preparing the system for operation.........................................................................................................118

Starting and stopping the system........................................................................................................... 118

Starting the 5104-22C, 9006-12P, 9006-22C, or 9006-22P system .............................................118

Stopping the 5104-22C, 9006-12P, 9006-22C, or 9006-22P system............................................120

Drive commands for 5104-22C, 9006-12P, 9006-22C, or 9006-22P..................................................121

arcconf commands.............................................................................................................................121

iii

StorCLI commands.............................................................................................................................121

sas3ircu commands...........................................................................................................................123

NVMe commands............................................................................................................................... 123

Sensor readings GUI display................................................................................................................... 124

Removing and replacing covers.............................................................................................................. 125

Removing the service access cover...................................................................................................125

Installing the service access cover....................................................................................................126

Service and operating positions..............................................................................................................127

Placing an 5104-22C, 9006-22C, or 9006-22P system into the service position...........................127

Placing an 5104-22C, 9006-22C, or 9006-22P system into the operating position.......................128

Power cords for the 5104-22C, 9006-22C, or 9006-22P system......................................................... 129

Disconnecting the power cords......................................................................................................... 129

Connecting the power cords..............................................................................................................130

Notices..............................................................................................................133

Accessibility features for IBM Power Systems servers.......................................................................... 134

Privacy policy considerations ................................................................................................................. 135

Trademarks.............................................................................................................................................. 135

Electronic emission notices.....................................................................................................................136

Class A Notices...................................................................................................................................136

Class B Notices...................................................................................................................................139

Terms and conditions.............................................................................................................................. 141

iv

Safety notices

Safety notices may be printed throughout this guide:

•DANGER notices call attention to a situation that is potentially lethal or extremely hazardous to people.

•CAUTION notices call attention to a situation that is potentially hazardous to people because of some

existing condition.

•Attention notices call attention to the possibility of damage to a program, device, system, or data.

World Trade safety information

Several countries require the safety information contained in product publications to be presented in

their national languages. If this requirement applies to your country, safety information documentation is

included in the publications package (such as in printed documentation, on DVD, or as part of the product)

shipped with the product. The documentation contains the safety information in your national language

with references to the U.S. English source. Before using a U.S. English publication to install, operate, or

service this product, you must rst become familiar with the related safety information documentation.

You should also refer to the safety information documentation any time you do not clearly understand any

safety information in the U.S. English publications.

Replacement or additional copies of safety information documentation can be obtained by calling the IBM

Hotline at 1-800-300-8751.

German safety information

Das Produkt ist nicht für den Einsatz an Bildschirmarbeitsplätzen im Sinne § 2 der

Bildschirmarbeitsverordnung geeignet.

Laser safety information

IBM servers can use I/O cards or features that are ber-optic based and that utilize lasers or LEDs.

Laser compliance

IBM servers may be installed inside or outside of an IT equipment rack.

DANGER: When working on or around the system, observe the following precautions:

Electrical voltage and current from power, telephone, and communication cables are hazardous. To

avoid a shock hazard: If IBM supplied the power cord(s), connect power to this unit only with the

IBM provided power cord. Do not use the IBM provided power cord for any other product. Do not

open or service any power supply assembly. Do not connect or disconnect any cables or perform

installation, maintenance, or reconguration of this product during an electrical storm.

• The product might be equipped with multiple power cords. To remove all hazardous

voltages, disconnect all power cords. For AC power, disconnect all power cords from their AC power

source. For racks with a DC power distribution panel (PDP), disconnect the customer’s DC power source

to the PDP.

• When connecting power to the product ensure all power cables are properly connected. For racks with

AC power, connect all power cords to a properly wired and grounded electrical outlet. Ensure that the

outlet supplies proper voltage and phase rotation according to the system rating plate. For racks with a

DC power distribution panel (PDP), connect the customer’s DC power source to the PDP. Ensure that the

proper polarity is used when attaching the DC power and DC power return wiring.

• Connect any equipment that will be attached to this product to properly wired outlets.

• When possible, use one hand only to connect or disconnect signal cables.

• Never turn on any equipment when there is evidence of re, water, or structural damage.

• Do not attempt to switch on power to the machine until all possible unsafe conditions are corrected.

• When performing a machine inspection: Assume that an electrical safety hazard is present. Perform

all continuity, grounding, and power checks specied during the subsystem installation procedures to

ensure that the machine meets safety requirements. Do not attempt to switch power to the machine

until all possible unsafe conditions are corrected. Before you open the device covers, unless instructed

otherwise in the installation and conguration procedures: Disconnect the attached AC power cords,

turn off the applicable circuit breakers located in the rack power distribution panel (PDP), and

disconnect any telecommunications systems, networks, and modems.

• Connect and disconnect cables as described in the following procedures when installing, moving, or

opening covers on this product or attached devices.

To Disconnect: 1) Turn off everything (unless instructed otherwise). 2) For AC power, remove the power

cords from the outlets. 3) For racks with a DC power distribution panel (PDP), turn off the circuit

breakers located in the PDP and remove the power from the Customer's DC power source. 4) Remove

the signal cables from the connectors. 5) Remove all cables from the devices.

To Connect: 1) Turn off everything (unless instructed otherwise). 2) Attach all cables to the devices. 3)

Attach the signal cables to the connectors. 4) For AC power, attach the power cords to the outlets. 5)

For racks with a DC power distribution panel (PDP), restore the power from the Customer's DC power

source and turn on the circuit breakers located in the PDP. 6) Turn on the devices.

• Sharp edges, corners and joints may be present in and around the system. Use

care when handling equipment to avoid cuts, scrapes and pinching. (D005)

(R001 part 1 of 2):

DANGER: Observe the following precautions when working on or around your IT rack system:

• Heavy equipment–personal injury or equipment damage might result if mishandled.

• Always lower the leveling pads on the rack cabinet.

• Always install stabilizer brackets on the rack cabinet if provided, unless the earthquake option is

to be installed.

• To avoid hazardous conditions due to uneven mechanical loading, always install the heaviest

devices in the bottom of the rack cabinet. Always install servers and optional devices starting

from the bottom of the rack cabinet.

• Rack-mounted devices are not to be used as shelves or work spaces. Do not place objects on top

of rack-mounted devices. In addition, do not lean on rack mounted devices and do not use them

to stabilize your body position (for example, when working from a ladder).

• Stability hazard:

– The rack may tip over causing serious personal injury.

– Before extending the rack to the installation position, read the installation instructions.

– Do not put any load on the slide-rail mounted equipment mounted in the installation position.

– Do not leave the slide-rail mounted equipment in the installation position.

• Each rack cabinet might have more than one power cord.

– For AC powered racks, be sure to disconnect all power cords in the rack cabinet when directed

to disconnect power during servicing.

viPower Systems: Servicing the 9006-22P

– For racks with a DC power distribution panel (PDP), turn off the circuit breaker that controls

the power to the system unit(s), or disconnect the customer’s DC power source, when directed

to disconnect power during servicing.

• Connect all devices installed in a rack cabinet to power devices installed in the same rack

cabinet. Do not plug a power cord from a device installed in one rack cabinet into a power device

installed in a different rack cabinet.

• An electrical outlet that is not correctly wired could place hazardous voltage on the metal parts

of the system or the devices that attach to the system. It is the responsibility of the customer to

ensure that the outlet is correctly wired and grounded to prevent an electrical shock. (R001 part

1 of 2)

(R001 part 2 of 2):

CAUTION:

• Do not install a unit in a rack where the internal rack ambient temperatures will exceed the

manufacturer's recommended ambient temperature for all your rack-mounted devices.

• Do not install a unit in a rack where the air flow is compromised. Ensure that air flow is not

blocked or reduced on any side, front, or back of a unit used for air flow through the unit.

• Consideration should be given to the connection of the equipment to the supply circuit so that

overloading of the circuits does not compromise the supply wiring or overcurrent protection.

To provide the correct power connection to a rack, refer to the rating labels located on the

equipment in the rack to determine the total power requirement of the supply circuit.

•(For sliding drawers.) Do not pull out or install any drawer or feature if the rack stabilizer brackets

are not attached to the rack or if the rack is not bolted to the floor. Do not pull out more than one

drawer at a time. The rack might become unstable if you pull out more than one drawer at a time.

•(For xed drawers.) This drawer is a xed drawer and must not be moved for servicing unless

specied by the manufacturer. Attempting to move the drawer partially or completely out of the

rack might cause the rack to become unstable or cause the drawer to fall out of the rack. (R001

part 2 of 2)

CAUTION: Removing components from the upper positions in the rack cabinet improves rack

stability during relocation. Follow these general guidelines whenever you relocate a populated rack

cabinet within a room or building.

• Reduce the weight of the rack cabinet by removing equipment starting at the top of the rack

cabinet. When possible, restore the rack cabinet to the conguration of the rack cabinet as you

received it. If this conguration is not known, you must observe the following precautions:

– Remove all devices in the 32U position and above.

– Ensure that the heaviest devices are installed in the bottom of the rack cabinet.

Safety noticesvii

– Ensure that there are little-to-no empty U-levels between devices installed in the rack cabinet

below the 32U level, unless the received conguration specically allowed it.

• If the rack cabinet you are relocating is part of a suite of rack cabinets, detach the rack cabinet

from the suite.

• If the rack cabinet you are relocating was supplied with removable outriggers they must be

reinstalled before the cabinet is relocated.

• Inspect the route that you plan to take to eliminate potential hazards.

• Verify that the route that you choose can support the weight of the loaded rack cabinet. Refer to

the documentation that comes with your rack cabinet for the weight of a loaded rack cabinet.

• Verify that all door openings are at least 760 x 2083 mm (30 x 82 in.).

• Ensure that all devices, shelves, drawers, doors, and cables are secure.

• Ensure that the four leveling pads are raised to their highest position.

• Ensure that there is no stabilizer bracket installed on the rack cabinet during movement.

• Do not use a ramp inclined at more than 10 degrees.

• When the rack cabinet is in the new location, complete the following steps:

– Lower the four leveling pads.

– Install stabilizer brackets on the rack cabinet or in an earthquake environment bolt the rack to

the floor.

– If you removed any devices from the rack cabinet, repopulate the rack cabinet from the lowest

position to the highest position.

• If a long-distance relocation is required, restore the rack cabinet to the conguration of the rack

cabinet as you received it. Pack the rack cabinet in the original packaging material, or equivalent.

Also lower the leveling pads to raise the casters off of the pallet and bolt the rack cabinet to the

pallet.

(R002)

(L001)

DANGER: Hazardous voltage, current, or energy levels are present inside any component that has

this label attached. Do not open any cover or barrier that contains this label. (L001)

(L002)

DANGER: Rack-mounted devices are not to be used as shelves or work spaces. Do not place

objects on top of rack-mounted devices. In addition, do not lean on rack-mounted devices and do

not use them to stabilize your body position (for example, when working from a ladder). Stability

hazard:

• The rack may tip over causing serious personal injury.

• Before extending the rack to the installation position, read the installation instructions.

viiiPower Systems: Servicing the 9006-22P

• Do not put any load on the slide-rail mounted equipment mounted in the installation position.

• Do not leave the slide-rail mounted equipment in the installation position.

(L002)

(L003)

Safety noticesix

DANGER: Multiple power cords. The product might be equipped with multiple AC power cords

or multiple DC power cables. To remove all hazardous voltages, disconnect all power cords and

power cables. (L003)

(L007)

CAUTION: A hot surface nearby. (L007)

(L008)

CAUTION: Hazardous moving parts nearby. (L008)

All lasers are certied in the U.S. to conform to the requirements of DHHS 21 CFR Subchapter J for class

1 laser products. Outside the U.S., they are certied to be in compliance with IEC 60825 as a class 1 laser

product. Consult the label on each part for laser certication numbers and approval information.

CAUTION: This product might contain one or more of the following devices: CD-ROM drive, DVD-

ROM drive, DVD-RAM drive, or laser module, which are Class 1 laser products. Note the following

information:

• Do not remove the covers. Removing the covers of the laser product could result in exposure to

hazardous laser radiation. There are no serviceable parts inside the device.

• Use of the controls or adjustments or performance of procedures other than those specied

herein might result in hazardous radiation exposure.

(C026)

xPower Systems: Servicing the 9006-22P

CAUTION: Data processing environments can contain equipment transmitting on system links with

laser modules that operate at greater than Class 1 power levels. For this reason, never look into

the end of an optical ber cable or open receptacle. Although shining light into one end and looking

into the other end of a disconnected optical ber to verify the continuity of optic bers may not

injure the eye, this procedure is potentially dangerous. Therefore, verifying the continuity of optical

bers by shining light into one end and looking at the other end is not recommended. To verify

continuity of a ber optic cable, use an optical light source and power meter. (C027)

CAUTION: This product contains a Class 1M laser. Do not view directly with optical instruments.

(C028)

CAUTION: Some laser products contain an embedded Class 3A or Class 3B laser diode. Note the

following information:

• Laser radiation when open.

• Do not stare into the beam, do not view directly with optical instruments, and avoid direct

exposure to the beam. (C030)

(C030)

CAUTION: The battery contains lithium. To avoid possible explosion, do not burn or charge the

battery.

Do Not:

• Throw or immerse into water

• Heat to more than 100 degrees C (212 degrees F)

• Repair or disassemble

Exchange only with the IBM-approved part. Recycle or discard the battery as instructed by

local regulations. In the United States, IBM has a process for the collection of this battery. For

information, call 1-800-426-4333. Have the IBM part number for the battery unit available when

you call. (C003)

CAUTION: Regarding IBM provided VENDOR LIFT TOOL:

• Operation of LIFT TOOL by authorized personnel only.

• LIFT TOOL intended for use to assist, lift, install, remove units (load) up into rack elevations. It is

not to be used loaded transporting over major ramps nor as a replacement for such designated

tools like pallet jacks, walkies, fork trucks and such related relocation practices. When this is not

practicable, specially trained persons or services must be used (for instance, riggers or movers).

• Read and completely understand the contents of LIFT TOOL operator's manual before using.

Failure to read, understand, obey safety rules, and follow instructions may result in property

damage and/or personal injury. If there are questions, contact the vendor's service and support.

Local paper manual must remain with machine in provided storage sleeve area. Latest revision

manual available on vendor's web site.

• Test verify stabilizer brake function before each use. Do not over-force moving or rolling the LIFT

TOOL with stabilizer brake engaged.

• Do not raise, lower or slide platform load shelf unless stabilizer (brake pedal jack) is fully

engaged. Keep stabilizer brake engaged when not in use or motion.

• Do not move LIFT TOOL while platform is raised, except for minor positioning.

• Do not exceed rated load capacity. See LOAD CAPACITY CHART regarding maximum loads at

center versus edge of extended platform.

• Only raise load if properly centered on platform. Do not place more than 200 lb (91 kg) on edge

of sliding platform shelf also considering the load's center of mass/gravity (CoG).

• Do not corner load the platforms, tilt riser, angled unit install wedge or other such accessory

options. Secure such platforms -- riser tilt, wedge, etc options to main lift shelf or forks in all four

(4x or all other provisioned mounting) locations with provided hardware only, prior to use. Load

objects are designed to slide on/off smooth platforms without appreciable force, so take care not

Safety noticesxi

to push or lean. Keep riser tilt [adjustable angling platform] option flat at all times except for nal

minor angle adjustment when needed.

• Do not stand under overhanging load.

• Do not use on uneven surface, incline or decline (major ramps).

• Do not stack loads.

• Do not operate while under the influence of drugs or alcohol.

• Do not support ladder against LIFT TOOL (unless the specic allowance is provided for one

following qualied procedures for working at elevations with this TOOL).

• Tipping hazard. Do not push or lean against load with raised platform.

• Do not use as a personnel lifting platform or step. No riders.

• Do not stand on any part of lift. Not a step.

• Do not climb on mast.

• Do not operate a damaged or malfunctioning LIFT TOOL machine.

• Crush and pinch point hazard below platform. Only lower load in areas clear of personnel and

obstructions. Keep hands and feet clear during operation.

• No Forks. Never lift or move bare LIFT TOOL MACHINE with pallet truck, jack or fork lift.

• Mast extends higher than platform. Be aware of ceiling height, cable trays, sprinklers, lights, and

other overhead objects.

• Do not leave LIFT TOOL machine unattended with an elevated load.

• Watch and keep hands, ngers, and clothing clear when equipment is in motion.

• Turn Winch with hand power only. If winch handle cannot be cranked easily with one hand, it

is probably over-loaded. Do not continue to turn winch past top or bottom of platform travel.

Excessive unwinding will detach handle and damage cable. Always hold handle when lowering,

unwinding. Always assure self that winch is holding load before releasing winch handle.

• A winch accident could cause serious injury. Not for moving humans. Make certain clicking sound

is heard as the equipment is being raised. Be sure winch is locked in position before releasing

handle. Read instruction page before operating this winch. Never allow winch to unwind freely.

Freewheeling will cause uneven cable wrapping around winch drum, damage cable, and may

cause serious injury.

• This TOOL must be maintained correctly for IBM Service personnel to use it. IBM shall inspect

condition and verify maintenance history before operation. Personnel reserve the right not to use

TOOL if inadequate. (C048)

Power and cabling information for NEBS (Network Equipment-Building System)

GR-1089-CORE

The following comments apply to the IBM servers that have been designated as conforming to NEBS

(Network Equipment-Building System) GR-1089-CORE:

The equipment is suitable for installation in the following:

• Network telecommunications facilities

• Locations where the NEC (National Electrical Code) applies

The intrabuilding ports of this equipment are suitable for connection to intrabuilding or unexposed wiring

or cabling only. The intrabuilding ports of this equipment must not be metallically connected to the

interfaces that connect to the OSP (outside plant) or its wiring. These interfaces are designed for use as

intrabuilding interfaces only (Type 2 or Type 4 ports as described in GR-1089-CORE) and require isolation

from the exposed OSP cabling. The addition of primary protectors is not sufcient protection to connect

these interfaces metallically to OSP wiring.

Note: All Ethernet cables must be shielded and grounded at both ends.

The ac-powered system does not require the use of an external surge protection device (SPD).

xiiPower Systems: Servicing the 9006-22P

The dc-powered system employs an isolated DC return (DC-I) design. The DC battery return terminal shall

not be connected to the chassis or frame ground.

The dc-powered system is intended to be installed in a common bonding network (CBN) as described in

GR-1089-CORE.

Safety noticesxiii

xivPower Systems: Servicing the 9006-22P

Removing and replacing parts in the 9006-22P

Use these procedures to remove and replace failing parts in the IBM Power® System LC922 (9006-22P)

system.

Note: See the International Information Bulletin for Customers - Installation of IBM Machines(http://

www.ibm.com/e-business/linkweb/publications/servlet/pbi.wss). This bulletin (Publication number

SC27-6601-00) provides a list of the key IBM system installation activities and those activities that might

be billable activities.

Before you begin a replacement, complete these tasks:

1. If you are completing a replacement procedure that might put your data at risk, ensure, if possible,

that you have a current backup of your system or logical partition (including operating systems,

licensed programs, and data).

2. Review the installation or replacement procedure for the feature or part.

3. The color blue on the part indicates a touch point.

4. Ensure that you have access to a medium, flat-blade screwdriver, and a Phillips screwdriver.

5. If parts are incorrect, missing, or visibly damaged, contact the provider of the part or your next level of

support.