IBM Power775 Manual

Power775 Disk Enclosure Rail Service Procedure
1 of 57
Power775
Disk Enclosure Rails Service Procedure
Last Modified 11/03/2011

Power775 Disk Enclosure Rail Service Procedure
2 of 57
CONTENTS
1GENERAL............................................................................................................................................................. 5
1.1 RELEASE /REVISION HISTORY ...........................................................................................................................5
1.2 WHERE TO FIND THIS DOCUMENT.......................................................................................................................5
1.3 REQUIRED DOCUMENTS..................................................................................................................................... 5
1.4 RELATED DOCUMENTS ...................................................................................................................................... 5
1.5 ABBREVIATIONS ................................................................................................................................................6
2OVERVIEW .......................................................................................................................................................... 7
2.1 SAFETY NOTICES ...............................................................................................................................................7
2.2 ATTENTION: FILE SYSTEM RISK STATEMENT .................................................................................................9
2.3 CONFIRM HOW YOU GOT TO THIS POWER775 DISK ENCLOSURE RAIL SERVICE PROCEDURE............................... 9
2.4 DE RAILS DESCRIPTION.....................................................................................................................................9
2.4.1 Enclosure VPD label and Location Code................................................................................................... 12
2.5 CONCURRENCY ............................................................................................................................................... 12
2.6 DE RAIL WEIGHT ............................................................................................................................................ 12
2.7 REQUIRED SSRS AND ROLES ........................................................................................................................... 12
2.8 ESTIMATED SERVICE TIME .............................................................................................................................. 13
2.9 REQUIRED TOOLS ............................................................................................................................................ 13
2.10 PREREQUISITES FOR THIS PROCEDURE ............................................................................................................. 13
2.11 SERVICE PROCEDURE OVERVIEW .................................................................................................................... 13
3SERVICE PROCEDURE.................................................................................................................................... 14
3.1 ASSESS SHUTDOWN OF FILE SYSTEM <= CUSTOMER TASK ......................................................................... 14
3.2 SHUT DOWN THE FILE SYSTEM <= CUSTOMER TASK .................................................................................. 14
3.3 IDENTIFY THE FRAME <= SSR TASK ............................................................................................................... 15
3.4 POWER OFF THE DISK ENCLOSURE <= CUSTOMER TASK ............................................................................. 22
3.5 CONFIRM THAT 350V IS OFF TO THE DISK ENCLOSURE <= SSR TASK............................................................. 22
3.6 REMOVE FRONT DOOR AND LATCH PLATE,MOVE UEPO PANEL <= SSR TASK............................................. 24
3.7 DISCONNECT CABLES,HOSES,AND RETENTION SCREWS <= SSR TASK.......................................................... 29
3.8 ATTACH THE LIFT TOOL TO THE FRAME <= SSR TASK ................................................................................... 36
3.9 MOVE THE DISK ENCLOSURE ONTO THE LIFT TOOL <= SSR TASK.................................................................. 41
3.10 REMOVE DEFECTIVE DE RAILS <= SSR TASK .................................................................................................. 47
3.11 INSTALL NEW DE RAILS <= SSR TASK ............................................................................................................ 48
3.12 INSTALL DE BACK INTO FRAME <= SSR TASK ................................................................................................. 50
3.13 STORE ALL TOOLS.<= SSR TASK ................................................................................................................... 52
3.14 POWER ON THE DISK ENCLOSURE <= CUSTOMER TASK .............................................................................. 52
3.15 CONFIRM WITH THE CUSTOMER <= SSR TASK ................................................................................................. 53
4END OF POWER775 DISK ENCLOSURE RAILS SERVICE PROCEDURE................................................ 54
5APPENDIX A: POWER775 BPC FSP COMMAND LINE PROCEDURE ...................................................... 55
5.1 PROCEDURE TO ACCESS THE BPC FSP COMMAND LINE .................................................................................. 55
5.2 END OF APPENDIX A: POWER775 BPC FSP COMMAND LINE PROCEDURE........................................................ 57

Power775 Disk Enclosure Rail Service Procedure
3 of 57
Figure List
Figure 1: DE Rail Assembly Front & Back Views .......................................................................................................................... 9
Figure 2: Typical 12-Node, 1-DE Rack Configuration.................................................................................................................. 10
Figure 3: Typical 2-Node, 6-DE Rack Configuration.................................................................................................................... 11
Figure 4: DE Enclosure VPD Label Location ............................................................................................................................... 12
Figure 5 Disk Enclosure HEX Cage ID......................................................................................................................................... 16
Figure 6 Frame Photo showing UEPO location.............................................................................................................................. 17
Figure 7 UEPO assembly showing frame Identify Amber LED...................................................................................................... 18
Figure 8 Door Latch ..................................................................................................................................................................... 19
Figure 9 Door Latch handle shown extended................................................................................................................................. 19
Figure 10 The Front Enclosure ID LED is located on the DE DCCAs............................................................................................ 20
Figure 11 Rear Enclosure ID LED is located on the bottom right side of the DE .......................................................................... 21
Figure 12: T1 and T4 Input Power LEDs on the DE DCCAs......................................................................................................... 23
Figure 13 Removal of Front Door ................................................................................................................................................. 24
Figure 14 Door Hinge Pin............................................................................................................................................................. 24
Figure 15: Disk Enclosure and UEPO Location Info..................................................................................................................... 25
Figure 16: Frame UEPO Service Position..................................................................................................................................... 26
Figure 17: Disk Enclosure and Door Latch Plate Location Info..................................................................................................... 27
Figure 18: Door Latch Plate Screw Location ................................................................................................................................ 27
Figure 19: Door Latch Plate Removal .......................................................................................................................................... 28
Figure 20: DCCA Service Network Communications (E/NET) Cable ........................................................................................... 29
Figure 21 DE cabling.................................................................................................................................................................... 29
Figure 22: Disk Enclosure DCCA Power Cables (HPIC) .............................................................................................................. 30
Figure 23: Disk Enclosure Port Card Cabling............................................................................................................................... 31
Figure 24 Photo of Front Rail Block Captive Screw ...................................................................................................................... 32
Figure 25: Front Rail Block Screws.............................................................................................................................................. 33
Figure 26 Water Hose Connector .................................................................................................................................................. 34
Figure 27: Disk Enclosure Rear Rail Block Captive Screws.......................................................................................................... 35
Figure 28 Lift Tool attachment Bracket......................................................................................................................................... 36
Figure 29 Lift Tool Anchor Holes ................................................................................................................................................. 36
Figure 30: Lift Tool against Front Side of Frame.......................................................................................................................... 37
Figure 31 Lift Tool attached to frame............................................................................................................................................ 37
Figure 32: Front Table Stop ......................................................................................................................................................... 38
Figure 33: Lift Table Movement Path........................................................................................................................................... 38
Figure 34: Secure Cables ............................................................................................................................................................. 39
Figure 35 Power Drill Operation – Correct Technique................................................................................................................... 39
Figure 36 Power Drill Operation – Correct Settings for Use........................................................................................................... 40
Figure 37 Relocate the DCCA cables ............................................................................................................................................ 42
Figure 38 Cable Protector Sheet.................................................................................................................................................... 42
Figure 39 Fold Cable Protector Sheet............................................................................................................................................ 42
Figure 40 Cable protector sheet shown in upper left position ......................................................................................................... 43
Figure 41 Blue-purple Recessed Touch Points............................................................................................................................... 43
Figure 42 Front of DE resting on lift tool ...................................................................................................................................... 44
Figure 43 DE slid out slowly to point of latch engagement ............................................................................................................ 44
Figure 44 Disengage the right latch first........................................................................................................................................ 45
Figure 45 Side view of latching system of DE out of frame ........................................................................................................... 45
Figure 46 DE removed from frame................................................................................................................................................ 46
Figure 47 DE supported by Lift Tool and cage stop bracket........................................................................................................... 46
Figure 48: Disk Enclosure Drawer Rail Mounting Screws ....................................................................................................... 47
Figure 49: Disk Enclosure Drawer Rails....................................................................................................................................... 47
Figure 50: Rack Key Slot Locations............................................................................................................................................. 48
Figure 51: Align Mushroom Pins in Rack..................................................................................................................................... 48
Figure 52: Rack Outer Side View of Installed Rail ....................................................................................................................... 49
Figure 53: Rail Block Rail Block Interfering with Drive Carrier ................................................................................................... 51
Figure 54: Rail Block Correctly Installed ..................................................................................................................................... 51

Power775 Disk Enclosure Rail Service Procedure
4 of 57
Table List
Table 1: Release / Revision History................................................................................................................................................ 5
Table 2: Required Documents........................................................................................................................................................ 5
Table 3: Related Documents .......................................................................................................................................................... 5
Table 4: Abbreviations................................................................................................................................................................... 6

Power775 Disk Enclosure Rail Service Procedure
5 of 57
1
11
1ENERAL
1.1 Release / Revision History
File Name Date Description
“p775_de_rails.pdf” 11/03/2011 Initial Release
Table 1: Release / Revision History
1.2 Where to find this document
The current “Power775 Disk Enclosure Rail Service Procedure” document is “p775_de_rails.pdf” which is to be downloaded from:
InfoCenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm
Click “PDF files for the IBM Power 775 (9125-F2C) removing and replacing parts”
Under “Disk Enclosure”, click “Disk Enclosure Rails” to download PDF “p775_de_rails.pdf”
This is the only valid source for the latest Power775 Disk Enclosure Rails Service Procedure.
1.3 Required Documents
Document P/N Location
Safety Notices
http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7hdx/G229-9054.pdf
Doc# G229-9054 Infocenter*
Table 2: Required Documents
*Infocenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm
1.4 Related Documents
Document PN Location
GPFS Native RAID Administration and Programming Reference Document # SA23-1354 InfoCenter *
Power775 Disk Enclosure Fan Service Procedure InfoCenter *
Table 3: Related Documents
*InfoCenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm

Power775 Disk Enclosure Rail Service Procedure
6 of 57
1.5 Abbreviations
Abbreviation Definition Details
BPA Bulk Power Assembly
CEC Central Electronic Complex Also referred to as the “node”.
DCCA Distributed Conversion and Control
Assembly
The power supplies for the CEC and DE are called the
CEC DCCA and DE DCCA respectively.
DE Disk Enclosure
GPFS Global Parallel File System IBM’s file system utilizing software RAID
HDD Hard Disk Drive This also means hard drive
HPIC High Power Interface Cable DCCA power cable
LED Light Emitting Diode
LIC Licensed Internal Code
PCB Printed Circuit Board
RAID Redundant Array of Inexpensive Disks
SAS Serial Attached SCSI Protocol used for direct attached storage
SSR Systems Services Representative IBM Service personnel
SSD Solid State Drive
UEPO Unit Emergency Power Off
Table 4: Abbreviations

Power775 Disk Enclosure Rail Service Procedure
7 of 57
2
22
2OVERVIEW
This section is an overview only. Do not start the service procedure until Section 3 which contains the detailed steps.
2.1 Safety Notices
Read “Safety_Notices “ available from InfoCenter – see Section 1.3.
http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7hdx/G229-9054.pdf
The following cautions apply to all Power775 service procedures:
CAUTION:
Energy hazard present. Shorting might result in system outage and possible physical injury.
Remove all metallic jewelry before servicing. (C001)
CAUTION:
The doors and covers to the product are to be closed at all times except for service by trained
service personnel. All covers must be replaced and doors locked at the conclusion of the service
operation. (C013)
CAUTION:
Servicing of this product or unit is to be performed by trained service personnel only. (C032)
The following notices specifically pertain to this Power775 service procedure:
DANGER: Heavy equipment — personal injury or equipment damage might result if mishandled. (D006)
CAUTION:
The weight of this part or unit is more than 55 kg (121.2 lb). It takes specially trained persons, a lifting device, or
both to safely lift this part or unit. (C011)
CAUTION: Do not remove or install this unit without using the provided lift tool. (C017)
CAUTION: The water-based coolant solution may contain an additive intended to inhibit corrosion (or provide
other functions). The solution may cause irritation to the skin or eyes. Avoid direct contact with the solution. Employ
appropriate Personal Protective Equipment when performing operations involving the coolant or which may
potentially expose you to the coolant. Refer to the MSDS for more information. (C037)
CAUTION: This unit must not be left running unattended, service personnel should always be
overseeing the process. (C038)

Power775 Disk Enclosure Rail Service Procedure
8 of 57
DANGER: Hazardous voltage present. Voltages present constitute a shock hazard, which can cause
severe injury or death. (L004)
CAUTION: Sharp edges, corners, or joints nearby. (L006)
CAUTION: System or part is heavy. The label is accompanied by a specific weight range. (L009)
CAUTION: Protective eyewear is needed for the procedure. (L011)
CAUTION: Pinch hazard. (L012)
DANGER: Heavy equipment—personal injury or equipment damage might result if mishandled. (L013)
CAUTION: Chemical resistant gloves are needed for this procedure. (L014)
DANGER: Risk of electric shock due to water or a water solution which is present in this product.
Avoid working on or near energized equipment with wet hands or when spilled water is present. (L016)

Power775 Disk Enclosure Rail Service Procedure
9 of 57
2.2 ATTENTION: File System Risk Statement
The Global Parallel File System (GPFS) implementation of software RAID stripes data across all the Disk
Enclosures in the cluster. If a single Disk Enclosure is powered off while GPFS is active, the file system will go
into panic and become unavailable.
This service procedure requires powering off a Disk Enclosure and because of this requires interaction with GPFS in
order to limit impact to performance.
2.3 Confirm how you got to this Power775 Disk Enclosure Rail Service
Procedure
You should be performing this procedure if you have determined a Disk Enclosure Rail has failed and needs
replacement.
You should have downloaded this procedure from:
InfoCenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm
This is the only valid source for the latest Power775 Disk Enclosure Rail Service Procedure.
2.4 DE Rails Description
The Power775 Disk Enclosure is supported in the Rack on 2 rail assemblies, one on the left and one on the right of
the Disk Enclosure unit. Figure 1 shows a photograph of the Rail assembly. These Rail assemblies include
integrated SAS cable channels to assist in routing SAS cables from the front of the DE to the back (as shown in
Figure 1 (“D”)). Disk Enclosures can be located in up to six positions in the Rack. Figure 2 shows a typical 12-
Node, 1-DE configuration and Figure 3 shows a typical 2-Node, 6-DE configuration.
Figure 1: DE Rail Assembly Front & Back Views

Power775 Disk Enclosure Rail Service Procedure
10 of 57
Figure 2: Typical 12-Node, 1-DE Rack Configuration

Power775 Disk Enclosure Rail Service Procedure
11 of 57
Figure 3: Typical 2-Node, 6-DE Rack Configuration

Power775 Disk Enclosure Rail Service Procedure
12 of 57
2.4.1 Enclosure VPD label and Location Code
The enclosure VPD label is placed on the rear of a Disk Enclosure just above the enclosure LEDs as shown in
Figure 4.
Figure 4: DE Enclosure VPD Label Location
2.5 Concurrency
The DE Rails are NOT concurrently maintainable. This procedure requires the customer to stop all client
applications and shut down the file system.
2.6 DE Rail Weight
Each DE Rail assembly weighs about 5 lbs (2.3 kg)
2.7 Required SSRs and Roles
This service procedure requires two SSRs.
This service procedure contains steps to be performed by a customer system administrator and two IBM SSRs (System
Service Representatives).
•The customer system administrator performs the STEPs in Sections marked with “<= CUSTOMER TASK”
•The SSR performs the STEPs in Sections marked with “<= SSR TASK”

Power775 Disk Enclosure Rail Service Procedure
13 of 57
2.8 Estimated Service Time
The estimated service time for this procedure is about 1 hour.
2.9 Required Tools
Service card (shipped in the Power 775 frame)
Lift Tool Kit 74Y1087
•Ladder PN 46G5947 may be required for upper DE locations (shipped in the Power 775 Lift Tool Kit)
Hand Tool Kit P/N 74Y0988
•Torque Clutch P/N 74Y0985 (shipped in the Hand Tool Kit)
•Ratchet handle P/N 6428240 (shipped in the Hand Tool Kit)
•Extension (18”) P/N 46K2707 (shipped in the Hand Tool Kit)
•2 mm Hex Torque Driver P/N 74Y0983 (shipped in the Hand Tool Kit)
•3 mm Hex Torque Driver P/N 74Y0998 (shipped in the Hand Tool Kit)
•4mm Hex Torque Driver PN 41V1059 (shipped in the Hand Tool Kit)
•5 mm Hex tool P/N 74Y0986 (shipped in the Hand Tool Kit)
•10 mm Socket P/N 02F3582 (shipped in the Hand Tool Kit)
2.10 Prerequisites for this Procedure
In order to perform this procedure, you will need the following information:
•The location code of the DE where the Rails are to be serviced.
•The cage location of the DE where the Rails are to be serviced.
•The frame number and frame serial number of the Rack containing the DE where the Rails are to be serviced.
2.11 Service Procedure Overview
This is an overview of the tasks to be performed. Read this overview but do not perform any of the tasks yet.
3.1 ASSESS SHUTDOWN OF FILE SYSTEM <= CUSTOMER TASK ......................................................................... 14
3.2 SHUT DOWN THE FILE SYSTEM <= CUSTOMER TASK .................................................................................. 14
3.3 IDENTIFY THE FRAME <= SSR TASK ............................................................................................................... 15
3.4 POWER OFF THE DISK ENCLOSURE <= CUSTOMER TASK ............................................................................. 22
3.5 CONFIRM THAT 350V IS OFF TO THE DISK ENCLOSURE <= SSR TASK............................................................. 22
3.6 REMOVE FRONT DOOR AND LATCH PLATE,MOVE UEPO PANEL <= SSR TASK............................................. 24
3.7 DISCONNECT CABLES,HOSES,AND RETENTION SCREWS <= SSR TASK.......................................................... 29
3.8 ATTACH THE LIFT TOOL TO THE FRAME <= SSR TASK ................................................................................... 36
3.9 MOVE THE DISK ENCLOSURE ONTO THE LIFT TOOL <= SSR TASK.................................................................. 41
3.10 REMOVE DEFECTIVE DE RAILS <= SSR TASK .................................................................................................. 47
3.11 INSTALL NEW DE RAILS <= SSR TASK ............................................................................................................ 48
3.12 INSTALL DE BACK INTO FRAME <= SSR TASK ................................................................................................. 50
3.13 STORE ALL TOOLS.<= SSR TASK ................................................................................................................... 52
3.14 POWER ON THE DISK ENCLOSURE <= CUSTOMER TASK .............................................................................. 52
3.15 CONFIRM WITH THE CUSTOMER <= SSR TASK ................................................................................................. 53

Power775 Disk Enclosure Rail Service Procedure
14 of 57
3SERVICE PROCEDURE
STOP – Do not proceed unless you have read
“Safety Notices” which is available from InfoCenter; see Section 1.3.
3.1 Assess Shutdown of File System <= CUSTOMER TASK
STEP 1 The customer must assess whether all client applications can be stopped at this time and whether the
Global Parallel File System (GPFS) can be shut down at this time.
This procedure requires powering off a Disk Enclosure, which under most conditions, if the file system
is active, will result in putting the file system into panic and making it unavailable. While it is
technically possible to drain all the data from a Disk Enclosure before shutting it down and leave the file
system active for this procedure, it is assumed impractical due to the excessive time required to move all
the data.
For more details, see related document “GPFS Native RAID Administration and Programming
Reference”, which can be found in InfoCenter.
If all client applications can be stopped and if GPFS can be shut down, proceed to STEP 2.
If you cannot stop the applications and file system at this time, you must wait to perform this procedure
until a time when the file system can be shut down. The Disk Enclosure should continue to operate as
long as the Rails are not severely damaged and that all connections (water, power, comm. etc.) are still
intact.
3.2 Shut Down the File System <= CUSTOMER TASK
Steps 2 through 3 are to be performed by the Customer.
STEP 2 Stop all client applications.
STEP 3 Shut down GPFS. Wait for complete shutdown before proceeding to STEP 4.

Power775 Disk Enclosure Rail Service Procedure
15 of 57
3.3 Identify the frame <= SSR TASK
STEP 4 Use the procedure “Appendix A: Power775 BPC FSP Command Line Procedure” (included in Section 5 of this
document) to access the BPC FSP command line of the frame with the Disk Enclosure needing its rails replaced.
STEP 5 Blink the Enclosure Identify LEDs of the DE with the FRU needing replacement by using the following command
from the BPC Command Line:
This is the command syntax:
bpccmd –c 27<blank>0000<cage ID><FRU ID><LED blink state 00=off, 02=blink>
<blank> = “11”
<cage ID> = see Figure 5
<FRU ID> = “01” for the DE Identify LED.
<LED state> = “02” for this step
Type in the following, inserting the correct Cage ID DE chassis location:
bpccmd –c 27110000xx0102
Where “xx” is the Cage ID of the DE you are servicing.
This field will be 05,07,09,0B,0D, or 0F (see Figure 5).
The correct return code is 001100. If you do not receive this, contact the next level of support.
Below is an example assuming Cage ID=0f

Power775 Disk Enclosure Rail Service Procedure
16 of 57
Figure 5 Disk Enclosure HEX Cage ID
Cage ID (Hex) = 0F
Cage ID (Hex) = 0D
Cage ID (Hex) = 0B
Cage ID (Hex) = 09
Cage ID (Hex) = 07
Cage ID (Hex) = 05

Power775 Disk Enclosure Rail Service Procedure
17 of 57
STEP 6 Locate the frame. Confirm that it is the correct frame by checking that its UEPO Panel Identify LED is blinking.
This LED is located on the UEPO panel and it is marked by the “!” symbol (see Figure 6 and Figure 7).
Figure 6 Frame Photo showing UEPO location

Power775 Disk Enclosure Rail Service Procedure
18 of 57
Figure 7 UEPO assembly showing frame Identify Amber LED
STEP 7 Open the front and rear doors
DANGER: Hazardous voltage present. Voltages present constitute a shock hazard, which can cause
severe injury or death. (L004)
Locate the latch along the left side of the door; push in on the X button, the latch handle will pop out, extended out
from the latch pocket. Grasp the latch handle and swing the door to the right (see Figure 8 and Figure 9).

Power775 Disk Enclosure Rail Service Procedure
19 of 57
Figure 8 Door Latch
Figure 9 Door Latch handle shown extended

Power775 Disk Enclosure Rail Service Procedure
20 of 57
STEP 8 If it will be needed for service, bring over the Ladder PN 46G5947 (shipped in the Power 775 Lift Tool Kit).
STEP 9 Identify the Disk Enclosure
Identify the Disk Enclosure with the failed fans by looking for the blinking amber Enclosure ID LEDs on the DE
DCCAs on the front side of the Disk Enclosure (see Figure 10) OR on the right side of the rear of the Disk
Enclosure (see Figure 11).
Figure 10 The Front Enclosure ID LED is located on the DE DCCAs