IBM Storwize V7000 Operating manual

IBM Storwize V7000
Troubleshooting, Recovery, and Maintenance Guide
Version 6.1.0
GC27-2291-00


IBM Storwize V7000
Troubleshooting, Recovery, and Maintenance Guide
Version 6.1.0
GC27-2291-00


Note
Before using this information and the product it supports, read the general information in
“Notices” on page 115, the information in the “Safety and environmental notices” on page ix, as
well as the information in the IBM Environmental Notices and User Guide on the documentation CD.
This edition applies to the IBM Storwize V7000, Version 6.1.0, and to all subsequent releases and modifications until
otherwise indicated in new editions.
This edition applies to GC27-2291-00.
© Copyright IBM Corporation 2010.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.

Contents
Figures ...............v
Tables ...............vii
Safety and environmental notices . . . ix
Sound pressure .............ix
About this guide ...........xi
Who should use this guide .........xi
Emphasis ...............xi
Storwize V7000 library and related publications . . xi
How to order IBM publications .......xii
Sending your comments ..........xii
Chapter 1. Storwize V7000 hardware
components .............1
Storwize V7000 components in the front of the
enclosure ...............1
Drives ...............1
Drive indicators ............2
Enclosure end cap indicators ........3
Storwize V7000 components in the rear of the
enclosure ...............4
Power supply unit and battery for the control
enclosure ..............5
Power supply unit for the expansion enclosure . . 6
Node canister ports and indicators ......7
Expansion canister ports and indicators ....14
Chapter 2. Best practices for
troubleshooting ...........17
Record access information .........17
Follow power management procedures .....17
Set up event notifications..........18
Set up inventory reporting .........18
Back up your data ............18
Manage your spare and failed drives ......19
Resolve alerts in a timely manner .......19
Keep your software up to date ........19
Keep your records up to date ........19
Subscribe to support notifications .......19
Know your IBM warranty and maintenance
agreement details ............20
Chapter 3. Understanding the Storwize
V7000 battery operation for the control
enclosure .............21
Maintenance discharge cycles ........22
Chapter 4. Storwize V7000 user
interfaces for servicing your system . . 25
Management GUI interface .........25
When to use the management GUI .....25
Accessing the management GUI ......26
Service assistant interface..........26
When to use the service assistant ......26
Accessing the service assistant .......27
Cluster command-line interface........27
When to use the cluster CLI ........27
Accessing the cluster CLI .........28
Service command-line interface ........28
When to use the service CLI........28
Accessing the service CLI.........28
USB key and Initialization tool interface .....28
When to use the USB key.........28
Using a USB key ...........28
Using the initialization tool ........29
satask.txt commands ..........29
Chapter 5. Resolving a problem ....35
Start here: Use the management GUI recommended
actions ................35
Problem: Management IP address unknown . . . 36
Problem: Unable to connect to the management GUI 36
Problem: Unable to log in to the management GUI 36
Problem: Cannot create a cluster .......37
Problem: Unknown service address of a node
canister................37
Problem: Cannot connect to the service assistant . . 38
Problem: Node canister location error .....38
Problem: SAS cabling not valid ........39
Problem: New expansion enclosure not detected . . 39
Problem: Mirrored volume copies no longer
identical ...............39
Procedure: Resetting superuser password ....40
Procedure: Identifying which enclosure or canister
to service ...............40
Procedure: Checking the status of your system . . 41
Procedure: Getting node canister and cluster
information using the service assistant .....41
Procedure: Getting node canister and cluster
information using a USB key ........42
Procedure: Understanding the system status using
the LEDs ...............42
Procedure: Finding the status of the Ethernet
connections ..............44
Procedure: Deleting a cluster completely.....45
Procedure: Fixing node errors ........45
Procedure: Changing the service IP address of a
node.................46
Procedure: Initializing a cluster with a USB key
without using the initialization tool ......46
Procedure: Initializing a cluster using the service
assistant ...............47
Procedure: Accessing a canister using a directly
attached Ethernet cable ..........48
Procedure: Reseating a node canister ......48
Procedure: Powering off your system......49
© Copyright IBM Corp. 2010 iii
44
44
4
4
44
44
44
44
44
44
44
44
44
44
44
44
44
44
44
44
4
44
4
44
4
44
4
44

Procedure: Collecting information for support . . . 49
SAN problem determination.........50
Fibre Channel link failures .........50
Servicing storage systems..........51
Chapter 6. Recovery procedures....53
Recover cluster procedure .........53
When to run the recover cluster procedure . . . 53
Fix hardware errors...........54
Removing cluster information for nodes with
error code 550 or error code 578 using the service
assistant ..............54
Performing cluster recovery using the service
assistant ..............55
What to check after running cluster recovery . . 56
Backing up and restoring the cluster configuration 56
Backing up the cluster configuration using the
CLI................57
Restoring the cluster configuration .....59
Deleting backup configuration files using the CLI 61
Chapter 7. Event reporting ......63
Understanding events ...........63
Viewing the event log ..........63
Managing the event log .........63
Describing the fields in the event log .....64
Event notifications ............64
Power-on self-test ............65
Understanding the error codes ........65
Event codes .............65
Error codes .............68
Node error code overview ........75
Cluster code overview..........75
Error code range............76
Node errors .............76
Cluster recovery and states ........83
Chapter 8. Removing and replacing
parts ...............85
Preparing to remove and replace parts .....85
Replacing a node canister..........85
Replacing an expansion canister .......87
Replacing the SFP transceiver ........88
Replacing a power supply unit for a control
enclosure ...............90
Replacing a power supply unit for an expansion
enclosure ...............93
Replacing a battery in a power supply unit....96
Releasing the cable retention bracket ......98
Replacing a 3.5" drive assembly or blank carrier 100
Replacing a 2.5" drive assembly or blank carrier 102
Replacing an enclosure end cap .......103
Replacing a SAS cable ..........103
Replacing a control enclosure chassis .....104
Replacing an expansion enclosure chassis ....107
Replacing the support rails .........109
Storwize V7000 replaceable units .......110
Appendix. Accessibility .......113
Notices ..............115
Trademarks ..............117
Electronic emission notices .........117
Federal Communications Commission (FCC)
statement ..............117
Industry Canada compliance statement ....117
Avis de conformité à la réglementation
d'Industrie Canada ..........118
New Zealand compliance statement .....118
European Union EMC Directive conformance
statement ..............118
Germany compliance statement ......118
Japanese Voluntary Control Council for
Interference (VCCI) statement .......119
People's Republic of China Class A Electronic
Emission Statement ..........119
International Electrotechnical Commission (IEC)
statement ..............119
United Kingdom telecommunications
requirements ............119
Korean Class A Electronic Emission Statement 119
Taiwan Class A compliance statement ....120
European Contact Information........120
Taiwan Contact Information ........120
Index ...............121
iv Storwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide
44
44

Figures
1. 12 drives ..............2
2. 24 drives ..............2
3. LED indicators on a single 3.5" drive ....2
4. LED indicators on a single 2.5" drive ....3
5. 12 drives and two end caps .......4
6. Left enclosure end cap .........4
7. Rear view of a control enclosure ......5
8. Rear view of an expansion enclosure ....5
9. LEDs on the power supply units of the control
enclosure..............6
10. LEDs on the power supply units of the
expansion enclosure ..........7
11. Fibre-channel ports on the node canisters 8
12. LEDs on the fibre-channel ports ......8
13. USB ports on the node canisters......10
14. Ethernet ports on the node canisters ....11
15. SAS ports on the node canisters. .....12
16. LEDs on the node canisters .......12
17. SAS ports and LEDs in rear of expansion
enclosure .............14
18. LEDs on the expansion canisters .....15
19. Opening the handle on a canister .....86
20. Removing the canister from the enclosure 86
21. Opening the handle on a canister .....88
22. Removing the canister from the enclosure 88
23. Fibre-channel SFP transceiver ......89
24. Directions for lifting the handle on the power
supply unit ............91
25. Using the handle to remove a power supply
unit ...............92
26. Directions for lifting the handle on the power
supply unit ............94
27. Using the handle to remove a power supply
unit ...............95
28. Removing the battery from the control
enclosure power-supply unit .......97
29. Unlocking the cable retention bracket ....99
30. Pressing the lever to slide the cable retention
bracket. .............100
31. Unlocking the 3.5" drive ........101
32. Removing the 3.5" drive ........101
33. Unlocking the 2.5" drive ........102
34. Removing the 2.5" drive ........103
35. SAS cable .............104
36. Removing a rail assembly from a rack cabinet 110
© Copyright IBM Corp. 2010 v
44
44
4
44

vi Storwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide

Tables
1. IBM documentation and related websites xii
2. Drive LEDs .............3
3. LED descriptions ...........4
4. Power supply unit LEDs in the rear of the
control enclosure ...........6
5. Power supply unit LEDs in the rear of the
expansion enclosure ..........7
6. Fibre Channel port LED locations on canister 1 9
7. Fibre Channel port LED status descriptions 9
8. Ethernet port LEDs ..........11
9. SAS port LEDs on the node canister ....12
10. Node canister LEDs..........13
11. SAS port LEDs on the expansion canister 15
12. Expansion canister LEDs ........15
13. Access information for your system ....17
14. Power-supply unit LEDs ........43
15. Cluster status and fault LEDs ......44
16. Description of data fields for the event log 64
17. Notification types ..........65
18. Informational event codes........66
19. Event IDs and codes .........68
20. Message classification number range ....76
21. Replaceable units ..........110
© Copyright IBM Corp. 2010 vii
44
44

viii Storwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide

Safety and environmental notices
Review the multilingual safety notices for the IBM®Storwize®V7000 system before you install and use
the product.
Suitability for telecommunication environment: This product is not intended to connect directly or
indirectly by any means whatsoever to interfaces of public telecommunications networks.
To find the translated text for a caution or danger notice:
1. Look for the identification number at the end of each caution notice or each danger notice. In the
following examples, the numbers (C001) and (D002) are the identification numbers.
CAUTION:
A caution notice indicates the presence of a hazard that has the potential of causing moderate or
minor personal injury. (C001)
DANGER
A danger notice indicates the presence of a hazard that has the potential of causing death or
serious personal injury. (D002)
2. Locate IBM Systems Safety Notices with the user publications that were provided with the Storwize
V7000 hardware.
3. Find the matching identification number in the IBM Systems Safety Notices. Then review the topics
concerning the safety notices to ensure that you are in compliance.
4. Optionally, read the multilingual safety instructions on the Storwize V7000 website. Go to Support for
Storwize V7000 website at www.ibm.com/storage/support/storwize/v7000, click the current product
documentation link, and then click Multi-language.
Sound pressure
Attention: Depending on local conditions, the sound pressure can exceed 85 dB(A) during service
operations. In such cases, wear appropriate hearing protection.
© Copyright IBM Corp. 2010 ix
4
4
4
4

xStorwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide

About this guide
This guide describes how to service, maintain, and troubleshoot the IBM Storwize V7000.
The chapters that follow introduce you to the hardware components and to the tools that assist you in
troubleshooting and servicing the Storwize V7000, such as the management GUI, and the service
assistant.
The troubleshooting procedures can help you analyze failures that occur in a Storwize V7000 system.
With these procedures, you can isolate the components that fail.
You are also provided with step-by-step procedures to remove and replace parts.
Who should use this guide
This guide is intended for system administrators who use and diagnose problems with the Storwize
V7000.
Emphasis
Different typefaces are used in this guide to show emphasis.
The following typefaces are used to show emphasis:
Boldface Text in boldface represents menu items and command
names.
Italics Text in italics is used to emphasize a word. In command
syntax, it is used for variables for which you supply
actual values, such as a default directory or the name of
a cluster.
Monospace Text in monospace identifies the data or commands that
you type, samples of command output, examples of
program code or messages from the system, or names of
command flags, parameters, arguments, and name-value
pairs.
Storwize V7000 library and related publications
Product manuals, other publications, and websites contain information that relates to Storwize V7000.
Storwize V7000 Information Center
The IBM Storwize V7000 Information Center contains all of the information that is required to install,
configure, and manage the Storwize V7000. The information center is updated between Storwize V7000
product releases to provide the most current documentation. The information center is available at the
following website:
publib.boulder.ibm.com/infocenter/storwize/ic/index.jsp
© Copyright IBM Corp. 2010 xi

Storwize V7000 library
Unless otherwise noted, the publications in the Storwize V7000 library are available in Adobe portable
document format (PDF) from the following website:
Support for Storwize V7000 website at www.ibm.com/storage/support/storwize/v7000
IBM documentation and related websites
Table 1 lists websites that provide publications and other information about the Storwize V7000 or related
products or technologies.
Table 1. IBM documentation and related websites
Website Address
Support for Storwize V7000 (2076) Support for Storwize V7000 website at www.ibm.com/storage/
support/storwize/v7000
Support for IBM System Storage®and IBM
TotalStorage products
www.ibm.com/storage/support/
IBM Publications Center www.ibm.com/e-business/linkweb/publications/servlet/pbi.wss
IBM Redbooks®publications www.redbooks.ibm.com/
Related accessibility information
To view a PDF file, you need Adobe Acrobat Reader, which can be downloaded from the Adobe website:
www.adobe.com/support/downloads/main.html
How to order IBM publications
The IBM Publications Center is a worldwide central repository for IBM product publications and
marketing material.
The IBM Publications Center offers customized search functions to help you find the publications that
you need. Some publications are available for you to view or download at no charge. You can also order
publications. The publications center displays prices in your local currency. You can access the IBM
Publications Center through the following website:
www.ibm.com/e-business/linkweb/publications/servlet/pbi.wss
Sending your comments
Your feedback is important in helping to provide the most accurate and highest quality information.
To submit any comments about this book or any other Storwize V7000 documentation:
vGo to the feedback page on the website for the Storwize V7000 Information Center at
publib.boulder.ibm.com/infocenter/storwize/ic/index.jsp?topic=/com.ibm.storage.Storwize
V7000.console.doc/feedback.htm. There you can use the feedback page to enter and submit comments
or browse to the topic and use the feedback link in the running footer of that page to identify the topic
for which you have a comment.
vSend your comments by email to [email protected]. Include the following information for this
publication or use suitable replacements for the publication title and form number for the publication
on which you are commenting:
– Publication title: IBM Storwize V7000 Troubleshooting, Recovery, and Maintenance Guide
xii Storwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide

– Publication form number: GC27-2291-00
– Page, table, or illustration numbers that you are commenting on
– A detailed description of any information that should be changed
About this guide xiii

xiv Storwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide

Chapter 1. Storwize V7000 hardware components
A Storwize V7000 system consists of one or more machine type 2076 rack-mounted enclosures.
There are four model types. The main differences among the model types are the following items:
vThe number of drives that an enclosure can hold. Drives are located on the front of the enclosure. An
enclosure can hold up to 12 3.5-inch drives or up to 24 2.5-inch drives.
vWhether the model is a control enclosure or an expansion enclosure.
Control enclosures contain the main processing units that control the whole system. They are where
external systems, such as host application servers, other storage systems, and management
workstations are connected through the Ethernet ports or Fibre Channel ports. Control enclosures can
also be connected to expansion enclosures through the serial-attached SCSI (SAS) ports.
Expansion enclosures contain additional storage capacity. Expansion enclosures connect either to
control enclosures or to other expansion enclosures through the SAS ports.
There are two control enclosure models.
vMachine type and model 2076-112, which can hold up to 12 3.5-inch drives
vMachine type and model 2076-124, which can hold up to 24 2.5-inch drives
There are two expansion enclosure models.
vMachine type and model 2076-212, which can hold up to 12 3.5-inch drives
vMachine type and model 2076-224, which can hold up to 24 2.5-inch drives
The machine type and model (MTM) are shown on these labels that are located on the front and the rear
of each enclosure:
vThe left end cap on the front of the enclosure. The label also indicates if the enclosure is a control
enclosure or an expansion enclosure.
vThe rear of the left enclosure flange.
Note: The labels also show the enclosure serial number. You must know the serial number when you
contact IBM support.
Because of the differences between the enclosures, you must be able to distinguish between the control
enclosures and the expansion enclosures when you service the system. Be aware of the following
differences:
vThe model type that is shown on the labels.
vThe model description that is shown on the left end cap.
vThe number of ports at the rear of the enclosure. Control enclosures have Ethernet ports, Fibre Channel
ports, and USB ports. Expansion enclosures do not have any of these ports.
vThe number of LEDs on the power supplies. Control enclosure power supplies have six; expansion
enclosure power supplies have four.
Storwize V7000 components in the front of the enclosure
This topic describes the components in the front of the enclosure.
Drives
An enclosure can hold up to 12 3.5-inch drives or up to 24 2.5-inch drives.
© Copyright IBM Corp. 2010 1
4
4
4
4
4
4
4
4

The drives are located in the front of the enclosure. The 12 drives are mounted in four columns with
three rows.
The 24 drives are mounted vertically in one row.
Note: The drive slots cannot be empty. A drive assembly or blank carrier must be in each slot.
Figure 1 shows 12 drives, and Figure 2 shows 24 drives.
Drive indicators
The drives have two LED indicators each. They have no controls or connectors.
The LED color is the same for both drives. The LEDs for the 3.5-inch drives are placed vertically above
and below each other. The LEDs for the 2.5-inch drives are placed next to each other at the bottom.
svc00609
Figure 1. 12 drives
svc00608
Figure 2. 24 drives
1
2
svc00607
Figure 3. LED indicators on a single 3.5" drive
2Storwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide

1 Fault LED
2 Activity LED
Table 2 shows the status descriptions for the two LEDs.
Table 2. Drive LEDs
Name Description Color
Activity Indicates if the drive is ready or active.
vIf the LED is on, the drive is ready to be used.
vIf the LED is off, the drive is not ready.
vIf the LED is flashing, the drive is ready, and there is activity.
Green
Fault Indicates a fault or identifies a drive.
vIf the LED is on, a fault exists on the drive.
vIf the LED is off, no known fault exists on the drive.
vIf the LED is flashing, the drive is being identified. A fault might or might not exist.
Amber
Enclosure end cap indicators
This topic describes the indicators on the enclosure end cap.
Figure 5 on page 4 shows where the end caps are located on the front of an enclosure with 12 drives. The
end caps are located in the same position for an enclosure with 24 drives.
v1 Left end cap
v2 Drives
v3 Right end cap
Figure 6 on page 4 shows the indicators on the front of the enclosure end cap.
The left enclosure end caps for both enclosures are identical and contain only indicators. The left
enclosure end cap contains no controls or connectors. The right enclosure end cap for both enclosures has
12
svc00606
Figure 4. LED indicators on a single 2.5" drive
Chapter 1. Storwize V7000 hardware components 3
4
4
4
4
4
4
4
4
4
4

no controls, indicators, or connectors.
Table 3. LED descriptions
Name Description Color Symbol
Power 1 The power LED is the upper LED. When the green LED is
lit, it indicates that the main power is available to the enclosure
Green .
Fault 2 The fault LED is the middle LED. When the amber LED is
lit, it indicates that one of the enclosure components has a
hardware fault.
Amber
Identify 3 The identify LED is the lower LED. When the blue LED is
lit, it identifies the enclosure.
Blue
N/A 4 The two-character LCD display shows the enclosure ID. N/A N/A
Storwize V7000 components in the rear of the enclosure
This topic describes the hardware components in the rear of the enclosure.
svc00694
123
Figure 5. 12 drives and two end caps
2
3
1
4
svc00605
Figure 6. Left enclosure end cap
4Storwize V7000 Version 6.1.0: Troubleshooting, Recovery, and Maintenance Guide
4
4
4
4
4
4
4
4
4
44
4
4
4444
44
4
4
4
44
4
4
4
4
44
4
4
4
4444
4
4