IBM RS/6000 SP Setup guide

RS/6000 SP IBM
Maintenance Information Volume 2,
Maintenance Analysis Procedures
GA22-7376-01

RS/6000 SP IBM
Maintenance Information Volume 2,
Maintenance Analysis Procedures
GA22-7376-01

Note!
Before using this information and the product it supports, be sure to read the general information under “Notices” on page ix.
Second Edition (February 1999)
This is a major revision of GA22-7376-00.
This edition applies to the IBM RS/6000 SP Systems, Version 3 Release 1 of IBM Parallel System Support Programs for AIX
(5765-529), which runs on IBM RS/6000 SP, and to all subsequent releases and modifications until otherwise indicated in new edi-
tions.
Order publications through your IBM representative or the IBM branch office serving your locality. Publications are not stocked at the
address given below.
IBM welcomes your comments. A form for readers' comments may be provided at the back of this publication or you may address
your comments to the following address:
International Business Machines Corporation
Department 55JA, Mail Station P384
522 South Road
Poughkeepsie, NY 12601-5400
USA
FAX: (United States & Canada): 1+914+432-9405
FAX: (Other Countries): Your International Access Code+1+914+432-9405
IBMLink (United States customers only): IBMUSM10(MHVRCFS)
IBM Mail Exchange: USIB6TC9 at IBMMAIL
Internet e-mail: [email protected]
World Wide Web: http://www.rs60000.ibm.com
If you would like a reply, be sure to include your name, address, telephone number, or FAX number.
Make sure to include the following in your comment or note:
Title and order number of this book
Page number or topic related to your comment
When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes
appropriate without incurring any obligation to you.
 Copyright International Business Machines Corporation 1994, 1999. All rights reserved.
Note to U.S. Government Users — Documentation related to restricted rights — Use, duplication or disclosure is subject to
restrictions set forth in GSA ADP Schedule Contract with IBM Corp.

Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Safety Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
Personal ESD Requirements ........................................... x
Electronic Emissions Notices ......................................... xi
Federal Communications Commission (FCC) Statement ........................... xi
European Union (EU) Statement ......................................... xi
United Kingdom Telecommunications Safety Requirements ......................... xi
Notice to Customers .............................................. xi
Industry Canada Compliance Statement .................................... xii
For Installations in Japan: ............................................ xii
Electromagnetic Interference (EMI) Statement - Taiwan ........................... xii
Radio Protection for Germany .......................................... xii
About This Book ................................................. xv
Who Should Use This Book ............................................ xv
User's Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
How to Use This Book ............................................... xv
Getting More Information ............................................. xvi
Summary of Changes ............................................. xvii
GA22-7376-00 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
GA22-7376-01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Chapter 1. Maintenance Analysis Procedures (MAPs) ......................... 1-1
Safety Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Lockout Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Lockout Procedure for SEPBU ....................................... 1-2
Lockout/Tagout Procedure for PDU ..................................... 1-2
Lockout/Bagout Procedure for PDU ..................................... 1-3
MAP 0100: Start of Service Call ...................................... 1-4
MAP 0110: Supervisor Subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-10
MAP 0120: Supervisor Bus Problem ................................... 1-19
MAP 0130: Processor Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30
MAP 0140: Thin Node/Thin Node 2 Environment ........................... 1-50
MAP 0150: 120/160 MHz Thin Node Environment ........................... 1-60
MAP 0160: Thin Processor Node Power ................................ 1-69
MAP 0170: Thin Processor Node Control ................................ 1-74
MAP 0180: Thin Processor Node dc Short/Open ........................... 1-84
MAP 0190: Processor Node Function .................................. 1-87
MAP 0200: Processor IPL Problem ................................... 1-94
MAP 0210: Wide Processor Node Environment ............................ 1-98
MAP 0220: Wide Processor Node Power ................................ 1-105
MAP 0230: Wide Processor Node Control ............................... 1-110
MAP 0240: Wide Processor Node dc Short/Open ........................... 1-120
MAP 0250: 604 or 604e High Node Environment ........................... 1-124
MAP 0260: 604 or 604e High Node Power ............................... 1-130
MAP 0270: 604 or 604e High Node Control .............................. 1-135
MAP 0280: 604 or 604e High Node Minimum Configuration ..................... 1-145
 Copyright IBM Corp. 1994, 1999 iii

MAP 0290: 332 MHz SMP Node Environment ............................. 1-160
MAP 0300: 332 MHz SMP Node Power ................................ 1-169
MAP 0310: 332 MHz SMP Node Control ................................ 1-177
MAP 0320: 332 MHz SMP Node Minimum Configuration ....................... 1-186
MAP 0330: POWER3 SMP Thin and Wide Node Environment ................... 1-191
MAP 0340: POWER3 SMP Thin and Wide Node Power ....................... 1-200
MAP 0350: POWER3 SMP Thin and Wide Node Control ...................... 1-208
MAP 0360: POWER3 SMP Thin and Wide Node Minimum Configuration ............. 1-217
MAP 0370: Main Power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-222
MAP 0380: Power Distribution Unit (PDU) ............................... 1-225
MAP 0390: Supervisor Power Problem ................................. 1-229
MAP 0400: Supervisor Power 12 V dc Problem ............................ 1-237
MAP 0410: Supervisor Power 24 V dc Problem ............................ 1-244
MAP 0420: Main Power Switch ...................................... 1-247
MAP 0430: dc Power Loss from Bulk dc Power Supply(s) ...................... 1-250
MAP 0440: Short in 48 V dc Power Distribution ............................ 1-257
MAP 0450: Frame Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-262
MAP 0460: Scalable Electrical Power Base Unit (SEPBU) ...................... 1-265
MAP 0470: SEPBU Power Control Interface .............................. 1-280
MAP 0480: Open in 48 V dc Distribution ................................ 1-283
MAP 0490: Network Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-286
MAP 0500: Ethernet LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-288
MAP 0510: Switch Assembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-294
MAP 0520: Switch Assembly Environment ............................... 1-301
MAP 0530: Switch Assembly Power ................................... 1-313
MAP 0540: Switch Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-322
MAP 0550: Dependent Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-351
MAP 0560: Block Multiplexer Channel Adapter (BMCA) ....................... 1-353
MAP 0570: End of Call .......................................... 1-357
MAP 1520: Main Power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-359
MAP 1540: Minimum Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-360
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . X-1
iv RS/6000 SP MAPs

Figures
1-1. 8-LED Frame Supervisor Card LEDs ............................... 1-11
1-2. RS/6000 SP Supervisor Subsystem Diagram (PDU) ...................... 1-12
1-3. RS/6000 SP Supervisor Subsystem Diagram (SEPBU) .................... 1-13
1-4. High Node Supervisor Card LEDs ................................. 1-31
1-5. 332 MHz SMP Node Supervisor Card LEDs ........................... 1-32
1-6. POWER3 SMP Node Supervisor Card LEDs .......................... 1-32
1-7. Thin Node High Level Diagram .................................. 1-40
1-8. Thin Node 2 High Level Diagram ................................. 1-41
1-9. 120 and 160 MHz Thin Node High Level Diagram ....................... 1-42
1-10. Wide Node High Level Diagram .................................. 1-43
1-11. 604 or 604e High Node High Level Component Diagram ................... 1-44
1-12. 332 MHz SMP Node High Level Component Diagram ..................... 1-45
1-13. POWER3 SMP Thin and Wide Node High Level Component Diagram ............ 1-46
1-14. Thin Node Supervisor Control Cable ............................... 1-54
1-15. 120 or 160 MHz Thin Node Supervisor Control Cable ..................... 1-63
1-16. Wide Node Supervisor Control Cable ............................... 1-102
1-17. Rear View of Power Distribution Unit ............................... 1-222
1-18. Rear View of Scalable Electrical Power Base Unit ....................... 1-223
1-19. Front View of 49-Inch Frame Locations .............................. 1-223
1-20. RS/6000 SP PDU Power Diagram ................................. 1-226
1-21. Wiring Diagram of RS/6000 SP Power Distribution Unit .................... 1-228
1-22. PDU Supervisor Power + 12 V dc Diagram ........................... 1-237
1-23. PDU + 24 V dc Circuit Diagram .................................. 1-244
1-24. PDU + 24 V dc Circuit Diagram .................................. 1-247
1-25. PDU + 48 V dc Power Diagram .................................. 1-250
1-26. PDU 48 Volts Distribution Diagram ................................ 1-257
1-27. SEPBU Power Diagram ....................................... 1-267
1-28. Typical Ethernet TTP Topology .................................. 1-289
1-29. High Performance Switch (HiPS) Assembly High-Level Diagram ............... 1-296
1-30. Scalable POWERparallel Switch (SPS) Assembly High-Level Diagram ........... 1-297
1-31. HiPS Wrap Plugs .......................................... 1-299
1-32. SPS Wrap Plugs ........................................... 1-300
1-33. HiPS Switch Supervisor Control Cable .............................. 1-305
1-34. SPS Inner Chassis and Front Chassis Cables .......................... 1-306
1-35. Switch Connectivity Diagram .................................... 1-320
1-36. HiPS 2.0 Interposer ......................................... 1-320
1-37. BMCA Cabling to Single Processor Node ............................ 1-353
1-38. BMCA Cabling to Multiple Processor Nodes ........................... 1-355
 Copyright IBM Corp. 1994, 1999 v

vi RS/6000 SP MAPs

Tables
1-1. 8-LED Frame Supervisor LED Definitions ............................ 1-10
1-2. 4-LED Frame Supervisor LED Definitions ............................ 1-11
1-3. 604 or 604e High Node LED Definitions ............................. 1-30
1-4. 332 MHz SMP Node LED Definitions ............................... 1-31
1-5. POWER3 SMP Node LED Definitions .............................. 1-32
1-6. SP Processor Nodes and Their Functionally Equivalent RS/6000 Models .......... 1-33
1-7. Cable Continuity Check Points ................................... 1-78
1-8. Cable Continuity Check Points ................................... 1-114
1-9. Inner Chassis Cable Continuity .................................. 1-318
1-10. Front Chassis Cable Continuity .................................. 1-318
1-11. HiPS Error Conditions ........................................ 1-324
1-12. SPS Error Conditions ........................................ 1-327
1-13. Service Request Number (SRN) Table for Switch Adapters .................. 1-332
1-14. Proper Serial Number to Record .................................. 1-358
 Copyright IBM Corp. 1994, 1999 vii

viii RS/6000 SP MAPs

Notices
References in this publication to IBM products, programs, or services do not imply that IBM intends to
make these available in all countries in which IBM operates.
Any reference to an IBM product, program, or service is not intended to state or imply that only IBM's
product, program, or service may be used. Any functionally equivalent product, program, or service that
does not infringe any of IBM's intellectual property rights may be used instead of the IBM product,
program, or service. Evaluation and verification of operation in conjunction with other products, except
those expressly designated by IBM, are the user's responsibility.
IBM may have patents or pending patent applications covering subject matter in this document. The fur-
nishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
USA
Any pointers in this publication to non-IBM Web sites are provided for convenience only, and do not in any
manner serve as an endorsement of these web sites. IBM accepts no responsibility for the content or use
of non-IBM Web sites specifically mentioned in this publication or accessed through an IBM Web site that
is mentioned in this publication.
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United
States or other countries or both:
AIX
ESCON
IBM
IBMLink
LoadLeveler
Micro Channel
POWERparallel
POWERserver
POWERstation
RS/6000
RS/6000 Scalable POWERparallel Systems
Scalable POWERparallel Systems
SP
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the
United States and other countries.
UNIX is a registered trademark in the United States and other countries licensed exclusively through
X/Open Company Limited.
Adobe, Acrobat, Acrobat Reader, and PostScript are trademarks of Adobe Systems, Incorporated.
 Copyright IBM Corp. 1994, 1999 ix

Netscape is a registered trademark of Netscape Communications Corporation in the United States and
other countries.
Other company, product, and service names may be trademarks or service marks of others.
Safety Notices
RS/6000 SP: Maintenance Information, Volume 1 contains a list of all safety notices pertaining to SP hard-
ware maintenance from Volumes 1, 2, 3, and 4.
Translations of these notices are also provided in Volume 1.
Personal ESD Requirements
The processor uses FRUs that are known to be sensitive to electrostatic discharge (ESD). To prevent
ESD damage to FRUs or to prevent system failures, observe the following procedures:
Keep the FRU in its original static-dissipative shipping container until the FRU is ready to be installed
in the system. Move the static-dissipative container near the location where the FRU is to be installed
(within ESD wrist strap distance). If the FRU must be put down for any reason, first place it in its
static-dissipative container or place it on the static-dissipative mat.
Open only the covers that are necessary to complete the task. Any time a cover is open the service
representative and all people in the area must be ESD-safe. If power is switched on, or if removing or
exchanging any FRU, always use the ESD kit (part 93F2649).
1. Put on the ESD wrist strap.
2. Attach the ESD cord to the wrist strap.
3. Attach the ESD mat to the wrist strap, if required.
4. Attach the insulated clip to the ESD cord.
5. Attach the insulated clip to the frame holes labeled ESD. If the frame holes are not available, use
a grounding point on the frame.
xRS/6000 SP MAPs

Electronic Emissions Notices
Federal Communications Commission (FCC) Statement
This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant
to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful
interference when the equipment is operated in a commercial environment. This equipment generates,
uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruc-
tion manual, may cause harmful interference to radio communications. Operation of this equipment in a
residential area is likely to cause harmful interference, in which case the user will be required to correct
the interference at his own expense.
Properly shielded and grounded cables and connectors must be used in order to meet FCC emission
limits. IBM is not responsible for any radio or television interference caused by using other than recom-
mended cables and connectors or by unauthorized changes or modifications to this equipment. Unauthor-
ized changes or modifications could void the user's authority to operate the equipment.
This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:
(1) this device may not cause harmful interference, and (2) this device must accept any interference
received, including interference that may cause undesired operation.
European Union (EU) Statement
This product is in conformity with the protection requirements of EU Council Directive 89/336/EEC on the
approximation of the laws of the Member States relating to electromagnetic compatibility. The manufac-
turer cannot accept responsibility for any failure to satisfy the protection requirements resulting from a
non-recommended modification of the product, including the fitting of option cards supplied by third parties.
Consult with your dealer or sales representative for details on your specific hardware.
This product has been tested and found to comply with the limits for Class A Information Technology
Equipment according to CISPR 22 / European Standard EN 55022. The limits for Class A equipment were
derived for commercial and industrial environments to provide reasonable protection against interference
with licensed communication equipment.
Attention: This is a Class A product. In a domestic environment this product may cause radio interference
in which case the user may be required to take adequate measures.
United Kingdom Telecommunications Safety Requirements
Notice to Customers
This apparatus is approved under approval number NS/G/1234/J/100003 for indirect connection to public
telecommunications systems in the United Kingdom.
 Copyright IBM Corp. 1994, 1999 xi

Industry Canada Compliance Statement
This Class A digital apparatus meets the requirements of the Canadien Interference-Causing Equipment
Regulations.
Cet appareil numérique de la classe A respecte toutes les exigences du Règlement sur le matériel
brouilleur du Canada.
For Installations in Japan:
The following is a summary of the VCCI Japanese statement in the box above. This is a Class A product
based on the standard of the Voluntary Control Council for Interference by Information Technology Equip-
ment (VCCI). If this equipment is used in a domestic environment, radio disturbance may arise. When
such trouble occurs, the user may be required to take corrective actions.
Electromagnetic Interference (EMI) Statement - Taiwan
The following is a summary of the EMI Taiwan statement above.
Warning: This is a Class A product. In a domestic environment this product may cause radio interference
in which case the user will be required to take adequate measures.
Radio Protection for Germany
Dieses Gerät ist berechtigt in Übereinstimmung mit Dem deutschen EMVG vom 9.Nov.92 das
EG–Konformitätszeichen zu führen.
Der Aussteller der Konformitätserklärung ist die IBM Germany.
Dieses Gerät erfüllt die Bedingungen der EN 55022 Klasse A. Für diese von Geräten gilt folgende
Bestimmung nach dem EMVG:
Geräte dürfen an Orten, für die sie nicht ausreichend entstört sind, nur mit besonderer Genehmigung des
Bundesministers für Post und Telekommunikation oder des Bundesamtes für Post und Telekommunikation
betrieben werden. Die Genehmigung wird erteilt, wenn keine elektromagnetischen Störungen zu erwarten
sind.
xii RS/6000 SP MAPs

(Auszug aus dem EMVG vom 9.Nov.92, Para.3, Abs.4)
Hinweis
Dieses Genehmigungsverfahren ist von der Deutschen Bundespost noch nicht veröffentlicht worden.
Electronic Emissions Notices xiii

xiv RS/6000 SP MAPs

About This Book
This book is to help you, as a customer engineer:
Isolate RS/6000 SP failures (as well as 9076 SP1 failures in an SP1/SP2 environment
Perform isolation and verification tests and identify field replaceable unit (FRU) locations
This book is to be used with:
 IBM RS/6000 SP: Maintenance Information, Volume 1, Installation and Relocation (GA22-7375),
(referred to, throughout this publication, as RS/6000 SP: Maintenance Information, Volume 1 ). Volume
1 contains information to help you Identify RS/6000 SP frame types and components, perform installa-
tion and relocation procedures, and maintenance agreements and qualifications.
 IBM RS/6000 SP: Maintenance Information, Volume 3, Locations and Service Procedures
(GA22-7377), (referred to, throughout this publication, as RS/6000 SP: Maintenance Information,
Volume 3 ). Volume 3 contains information to help you perform isolation and verification tests, identify
field replaceable unit (FRU) locations, and remove and replace FRUs.
 IBM RS/6000 SP: Maintenance Information, Volume 4, Parts Catalog (GA22-7378), (referred to,
throughout this publication, as RS/6000 SP: Maintenance Information, Volume 4 ). Volume 4 contains a
list of RS/6000 SP parts and FRUs with corresponding figures that contain indexed descriptions and
part numbers.
For a list of other SP books and related information, see the bibliography in the back of RS/6000 SP:
Maintenance Information, Volume 1 .
Who Should Use This Book
This book is intended for the product-trained Customer Engineer (CE). The procedures for RS/6000 SP
and feature components described in this book represent a part of the overall support structure of the
RS/6000 SP product. This book is intended to support the RS/6000 SP product and SP1/SP2 coexistence
systems, in which SP2 code is utilized. The SP1 version of this book ( IBM 9076 Scalable POWERparallel
Systems: Maintenance Information , SY66-0299) should be used for ordering SP1 parts, or if SP1 code is
utilized.
User's Responsibilities
Before calling the IBM customer engineer, the system administrator should use the problem determination
section of the IBM Parallel System Support Programs for AIX: Diagnosis Guide , SC23-3866 for initial
problem determination. If there is nothing wrong with the customer operating procedures, customer-
supplied cables, or power source, the customer should call an IBM customer engineer.
How to Use This Book
When performing RS/6000 SP maintenance, follow Chapter 1, “Maintenance Analysis Procedures (MAPs)”
on page 1-1.
Beginning with the “MAP 0100: Start of Service Call” on page 1-4, isolate the problem to one or more of
these RS/6000 SP components:
Supervisor Subsystem
 Copyright IBM Corp. 1994, 1999 xv

Processor Node
Power Subsystem
High Voltage Transformer (World Trade)
Ethernet Local Area Networks (LANs)
High-Performance Switch
Micro Channel or PCI Adapter
If you cannot determine the failure's cause, you should request the assistance of the RS/6000 SP Field
Support Center.
Getting More Information
This book and other RS/6000 SP hardware and software documentation are available both online and in
hardcopy from:
The RS/6000 website at http://www.rs6000.ibm.com
The Resource Center on the PSSP product media
The Service Information Library (SIL)
Hardcopy and CDROM versions orderable from IBM
IBM internal use versions available on MKTTOOLS
For details, see the bibliography in RS/6000 SP: Maintenance Information, Volume 1 .
xvi RS/6000 SP MAPs

Summary of Changes
GA22-7376-00
This edition, along with GA22-7377-00 and GA22-7378-00, replaces GC23-3904-08 and makes it obsolete.
Changes found in this edition include:
Added SP-attached server (RS/6000 Machine Type 7017-S70 and S7A) node information.
The Maintenance Information manuals, Volumes 1, 2, and 3, (GC23-3903 and GC23-3904) were reor-
ganized into 4 volumes:
– IBM RS/6000 SP: Maintenance Information, Volume 1, Installation and Relocation , GA22-7375-00
– IBM RS/6000 SP: Maintenance Information, Volume 2, Maintenance Analysis Procedures ,
GA22-7376-00
– IBM RS/6000 SP: Maintenance Information, Volume 3, Locations and Service Procedures ,
GA22-7377-00
– IBM RS/6000 SP: Maintenance Information, Volume 4, Parts Catalog , GA22-7378-00
GA22-7376-01
This edition replaces GA22-7376-00 and makes it obsolete. Changes found in this edition include:
Added POWER3 SMP thin and wide node information.
 Copyright IBM Corp. 1994, 1999 xvii

xviii RS/6000 SP MAPs