Ibm Power 775 Manual

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

1 of 27

Power 775

D-Link Service Procedure

9/30/2011

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

2 of 27

CONTENTS

1GENERAL.............................................................................................................................................................4

1.1 RELEASE /REVISION HISTORY ...........................................................................................................................4

1.2 WHERE TO FIND THIS DOCUMENT.......................................................................................................................4

1.3 REQUIRED DOCUMENTS.....................................................................................................................................4

1.4 RELATED DOCUMENTS ......................................................................................................................................4

1.5 ABBREVIATIONS ................................................................................................................................................5

2OVERVIEW ..........................................................................................................................................................6

2.1 SAFETY NOTICES ............................................................................................................................................... 6

2.2 CONFIRM HOW YOU GOT TO THIS POWER 775 DLINK SERVICE PROCEDURE ......................................................7

2.3 D-LINK DESCRIPTION ........................................................................................................................................8

2.3.1 General Discussion of MTP Optical Connector Handling ............................................................................8

2.4 CONCURRENCY .................................................................................................................................................9

2.5 D-LINK WEIGHT ................................................................................................................................................9

2.6 REQUIRED SSRS AND ROLES ............................................................................................................................. 9

2.7 ESTIMATED SERVICE TIME ................................................................................................................................ 9

2.8 P7IH HAND TOOL KIT REQUIRED TOOLS ...........................................................................................................9

2.9 PREREQUISITES FOR THIS PROCEDURE ............................................................................................................. 10

2.10 OVERVIEW OF PROCEDURE.............................................................................................................................. 10

3SERVICE PROCEDURE.................................................................................................................................... 11

3.1 ASSESS IMPACT ON CLUSTER PERFORMANCE <= CUSTOMER TASK.................................................................. 11

3.2 IDENTIFY THE FRAME,CEC, AND D-PORT<= SSR TASK .................................................................................. 11

3.3 CLEAN THE INDIVIDUAL D-LINK CONNECTORS <= SSR TASK ......................................................................... 15

3.4 CLEAN THE D-LINK CONNECTORS IN THE GROUPER <= SSR TASK................................................................... 16

3.5 RETEST THE LINK <= CUSTOMER TASK............................................................................................................ 17

3.6 INSTALL WRAP PLUG <= SSR TASK ................................................................................................................ 17

3.7 RUN WRAP PLUG TEST (NWLINKDIAG)<= CUSTOMER TASK............................................................................ 17

3.8 EVALUATE WRAP PLUG TEST RESULTS <= SSR TASK ..................................................................................... 18

3.9 REPLACE AN INDIVIDUAL D-LINK OPTICAL CABLE <= SSR TASK.................................................................... 19

3.10 REPLACE A D-LINK CABLE IN D-LINK GROUPERS <= SSR TASK....................................................................... 20

3.11 RETEST THE LINK <= CUSTOMER TASK............................................................................................................ 21

3.12 CONFIRM THAT THE PROBLEM IS RESOLVED <= SSR TASK............................................................................... 21

3.13 END OF D-LINK SERVICE PROCEDURE ............................................................................................................. 21

4APPENDIX A: BACKGROUND INFORMATION ON FAILURE ISOLATION OF D-LINKS ................... 22

4.1.1 Likely Root Causes for Failure in D-Links ................................................................................................ 22

4.1.2 Failure Isolation and Repair for D-Links ................................................................................................... 22

5END OF DOCUMENT........................................................................................................................................ 24

Table List

Table 1 Release / Revision History.................................................................................................................................................. 4

Table 2 Required Documents.......................................................................................................................................................... 4

Table 3 Related Documents ............................................................................................................................................................ 4

Figure List

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

3 of 27

Figure 1 Identify LED Selection ................................................................................................................................................... 11

Figure 2 System Unit, Model F2C Selection – Activate LED......................................................................................................... 12

Figure 3 UEPO Identify LED location .......................................................................................................................................... 12

Figure 4 CEC Drawer ID – Front .................................................................................................................................................. 13

Figure 5 CEC Drawer ID - Rear.................................................................................................................................................... 13

Figure 6 FRU Serial number location............................................................................................................................................ 13

Figure 7 D-Link Port numbering and location codes...................................................................................................................... 14

Figure 8 Grasping push/pull tab Figure 9 Removing D-Link Connector.................................................................................. 15

Figure 10 Unlatching connector with push/pull tab Figure 11 Unlatching next connector in grouper ........................................ 16

Figure 12 Unlatching last connector in grouper ............................................................................................................................ 16

Figure 13 Grouper removed ........................................................................................................................................................ 16

Figure 14 Grouper installation ...................................................................................................................................................... 17

Figure 15 Individual D-Link Connector with push/pull tab on upper right..................................................................................... 19

Figure 16 Insert one connector Figure 17 Perform final actuation............................................................................................. 19

Figure 18 Top view of grouper Figure 11 Bottom view of grouper .......................................................................................... 20

Figure 19 System Unit, Model F2C Selection................................................................................................................................ 21

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

4 of 27

1GENERAL

1.1 Release / Revision History

File Name Date Description

“p775_d_link_cables.pdf” 9/30/2011 Initial Release

Table 1 Release / Revision History

1.2 Where to find this document

The current Power775 D-Link Service Procedure document is “p775_d_link_cables.pdf” which is to be downloaded from:

InfoCenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm

Click “PDF files for the IBM Power 775 (9125-F2C) removing and replacing parts”

Under “CEC Drawer”, click “D-Link Optical cables” to download PDF “p775_d_link_cables.pdf”

This is the only valid source for the latest Power775 D-Link Cables Service Procedure.pdf.

1.3 Required Documents

Document Doc Number Location

Safety Notices

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/top

ic/p7hdx/G229-9054.pdf

Doc# G229-9054 InfoCenter *

Table 2 Required Documents

*InfoCenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm

1.4 Related Documents

Document Doc # Location

Power 775 L-Link Service Procedure Infocenter*

Table 3 Related Documents

*InfoCenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

5 of 27

1.5 Abbreviations

Abbreviation Definition Details

CEC Central Electronic Complex Also referred to as the node or node drawer.

DCCA Distributed Conversion and

Control Assembly

The power supplies for the CEC and DE are called the CEC DCCA

and DE DCCA respectively.

DE Disk Enclosure

D-Link Distant Link Optical Cables that connect SuperNodes to other SuperNodes.

FRU Field Replaceable Unit

L-Link Local Link L-Links are optical (fiber) or electrical (copper) links connect all Hub

modules to all other Hub modules within the same supernode.

L-Links are distinguished as being either “LL-Links”, which are

electrical links that interconnect all Hubs within a CEC, or “LR-

Links”, which are optical links, enclosed within an OBE, that

interconnect each Hub with all 24 Hubs in the other 3 drawers of the

SuperNode.

MPO Multi-Fiber Push-on/Pull-off Type of multi-fiber optical connector

MTP Branded version of the MPO optical fiber connector

PVC PolyVinyl Chloride Jacketing material for D-Link Cables

SN SuperNode Array of 32 Octants or Nodes, in 4 CEC Drawers, that are

interconnected in an all-to-all full mesh topology using LL-Links or

LR-Links.

SSR Systems Services

Representative

IBM Service personnel

VCSEL Vertical Cavity Surface

Emitting Laser

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

6 of 27

2O ER IEW

You should be performing this procedure if you have determined a D-Link optical cable or port has poor signal quality or

has failed and may need replacement.

This procedure describes how to isolate a problem in optical data transmission over an D-Link to either (a) a problem in the

transmitting node, (b) a problem in the receiving node, or (c) a problem in the D-Link optical cable which interconnects

them, or in the associated optical connectors.

This procedure also instructs how to test, remove, clean, reinstall, and re-test the D-Link optical cables, if cleaning or

replacement is required.

This section, Section 2, is an overview. Do not start the service procedure until Section 3, which contains the detailed steps.

2.1 Safety Notices

Read “Safety_Notices “ available from InfoCenter – see Section 1.3.

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7hdx/G229-9054.pdf

The following cautions apply to all Power775 service procedures:

CAUTION:

Energy hazard present. Shorting might result in system outage and possible physical injury. Remove all metallic

jewelry before servicing. (C001)

CAUTION:

The doors and covers to the product are to be closed at all times except for service by trained service personnel. All

covers must be replaced and doors locked at the conclusion of the service operation. (C013)

CAUTION:

Servicing of this product or unit is to be performed by trained service personnel only. (C032)

The following caution specifically pertains to this Power775 service procedure.

CAUTION:

This product contains Class 1M lasers. Do not view directly with optical instruments. (C028)

DANGER: Hazardous voltage present. Voltages present constitute a shock hazard, which can cause

severe injury or death. (L004)

CAUTION: Sharp edges, corners, or joints nearby. (L006)

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

7 of 27

2.2 Confirm how you got to this Power 775 D Link Service Procedure

You should have downloaded this procedure from:

InfoCenter Website: http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7ee2/p7ee2kickoff.htm

This is the only valid source for the latest Power775 D-Link Service Procedure document.

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

8 of 27

2.3 D-Link Description

The D-Links that interconnect SuperNodes to each other consist of 24-fiber cables, with an orange PVC

jacket. Each cable is 3.8mm in diameter. Each end of the cable has an MTP-brand 24-fiber MPO connector,

with an attached push/pull tab that allows actuation of connectors in tightly-packed arrays.

The fibers in the top row of the left-hand, key-up connector of this figure are connected directly across to the

top row of fibers in the right-hand, key-down connector, and vice versa. When cables are plugged into system

adapter ports, they are always oriented with keys up on both ends, so the cable does a half-circle turn or helix

half-twist, and the top row in each adapter port is wired to the bottom row at the other end of the cable.1

Removing or inserting an individual D-Link from the CEC drawer can be done by pulling or pushing,

respectively, on its push/pull tab. Insertion can also be done by pushing on the connector’s strain relief boot.

In most system installations, D-Links are gathered into Cable Assemblies, or “wiring harness” bundles, which

consist of a reasonably large number of cables (14-150 or more), with vinyl tape or hook-and-loop tape

wrapped around them, and connector groupers used to gather together a row of up to 8 connectors. The

specific design of each assembly (number of D-Link cables, organization of connectors into groupers, etc.)

depends on the size and scale of the system, and on the number of CEC drawers that are interconnected and is

described in the system-specific System Installation Guide.

1Note that this over-and-under orientation of transmitting and receiving rows is different than many one-row connectors (e.g., RJ-

45, or QSFP), where transmitting and receiving lanes are left-and-right of each other. With a two-row connector,

Grouper

8 MTP connectors with

push/pull tabs in a grouper

Example Cable Bu

ndle Assembly

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

9 of 27

2.3.1 General Discussion of MTP Optical Connector Handling

The MTP implementation of the MPO (Multi-Fiber Push-on/Pull-off) optical connector is a robust, rugged

optical connector with a long record of service worldwide over nearly two decades. However, there are several

factors that will greatly improve the effectiveness in using this optical connector.

CLEANLINESS IS CRITICAL. Operation of an optical connector requires precise end-to-end alignment of

the two optical fibers, with little or no air gap in between. Fiber ends are pushed together with force of about 2

pounds, to minimize likelihood of air gaps. If dust or hair gets on the ends of the connectors, where the optical

fibers exit, it may block the light passing between fibers, or may push ferrules apart, causing larger gaps

between fibers. Therefore, it is important that the ends of the connectors are either kept clean, or are actively

cleaned before each connection. The following steps will help assure cleanliness, and will help ensure proper

signal integrity on all fiber lanes.

•Leave dust caps and dust covers on connectors and ports as much as possible,

•Ensure to never touch the end surfaces of the connectors, and

•Regularly use approved cleaning tools, such as the Fiber Optic Cleaning Tool shown below, preferably

before each and every connector insertion.

DON’T FORCE IT. The Groupers, Push/Pull Tabs, and MTP Connector housings are made of plastic. The

plastic is high-quality, robust, and non-brittle, but parts may break if pushed or pulled too hard. Also, the PVC

cable jacket may separate from the connector housing, if it is pulled too hard. No operation in the insertion or

removal of an individual connector should require more than about 1 kilogram (9.8 Newtons / 2 pound) of

force – roughly equivalent to turning a doorknob. Always use the blue purple colored Push/Pull tabs for doing

the final actuation of connectors, both on insertion and removal.

LISTEN & FEEL FOR THE CLICKS. MTP optical connectors operate in two steps: 1. Initial actuation,

where the connector is aligned with the LR-Port receptacle and pushed partway in, and 2. Final actuation,

where the latching fingers push the spring-loaded latching sleeve back, then ride over two latching nubs, and,

when past the latching nubs, release the spring-loaded latching sleeve so that it can slide over the latching

fingers. When this happens, there is an audible “click” as the latching sleeve hits its end of travel. If there is

not a “click”, the connector is not fully connected, the fiber ends are not pressed together, and the connector

will not pass light well. Similarly, on removal, there is another “click”, as the spring-loaded latching sleeve is

pulled back, releasing the latching fingers, and they snap over the latching nubs. On insertion of a LR-Link 8-

connector grouper, all connectors go through the initial actuation as a group. The final actuation is done

individually -- 8 push/pull tabs must be pushed separately, yielding 8 separate audible “clicks”, for the whole

grouper to be fully connected. On removal, all 8 push/pull tabs must be pulled, yielding 8 separate audible

“clicks”, before the grouper can be removed from the row of ports.

Blue Head

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

10 of 27

2.4 Concurrency

This procedure requires removal of all of the D-Link connectors in a grouper, which may include up to 8 D-

Link connectors at a time. Connectivity between supernodes is preserved by indirect links. However,

depending on network traffic patterns, D-Link connector removal may have significant impact on system-

level performance.

2.5 D-Link Weight

The Field Replaceable Unit is a single D-Link Cable. Each D-Link cable weighs 0.15 kg (0.33 lb) per 10

meters of length. The weight of each D-Cable Assembly Bundle is dependent on the number of cables in the

bundle and the length of each cable.

2.6 Required SSRs and Roles

This service procedure contains steps to be performed by a system administrator and an IBM Systems Services

Representative (SSR). The system administrator tasks and SSR tasks are separately called out in the

procedure.

2.7 Estimated Service Time

Removal of a D-Link cable requires from 1 to 15 minutes, depending on cable length and on whether or not it

is included in a cable bundle assembly. Installation of a D-Link cable requires from 1 to 20 minutes,

depending on the length and complexity of the routing path.

2.8 P7IH Hand Tool Kit Required Tools

Installation of optical connectors requires a cleaning tool (IBM PN: 74Y0685, FIBER OPTICAL CLEANING

TOOL), to clean the front surface of the optical connectors before connecting.

Testing of D-Ports requires the D-Link Wrap Plug (IBM PN: 74Y0792, WRAP PLUG, 24 POS STRAIGHT,

WITH SLEEVE, WITH BACK), to wrap a D-Port transmitter back to its own receiver. Testing of both end-

ports of a link will be faster if 2 wrap plugs are used.

Optionally, a bright flashlight (6-LED or 9-LED or equivalent) may be useful to inspect adapter ports for

debris or contamination. Also, optionally, a 5x eye loupe-style magnifier is useful for inspecting the OBE

connector front surfaces for dust contamination. For eye safety, this is absolutely not to be used for inspecting

the adapter ports in the CEC drawers while the system is turned on – the system’s laser transmitters are rated

Class 1M, Eye safe *without* magnifying optics.

2.9 Prerequisites for this Procedure

In order to perform this procedure, you will need the following information:

For each end of the faulty D-Link

1) The frame number and frame serial number of the frame connected to the faulty D-Link Cable

2) The cage location

3) The location code of the D-Link port (Note the CEC serial#)

4) The HUB and link id (from the customer running the lsnwlinkinfo command).

Power 775 D-Link Service Procedure

PN: 41U8487, EC N44172

11 of 27

2.10 Overview of Procedure

This is an overview of the tasks to be performed. Read this overview but do not perform any of the tasks yet.