Silicon Graphics InfiniteStorage TP9300 User manual

SGI®InfiniteStorage™ TP9300 and TP9300S
RAID User’s Guide
007-4644-003

CONTRIBUTORS
Written by Matt Hoy
Updated by Mark Schwenden
Illustrated by Kelly Begley, Matt Hoy, and Jim Ostrom
Production by Karen Jacobson
Engineering contributions by Bill Andrews, Marty Castilla, Bill Lusche, Robert Novak, Henry Ortiz, and Sammy Wilborn
COPYRIGHT
© 2004, Silicon Graphics, Inc. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein. No
permission is granted to copy, distribute, or create derivative works from the contents of this electronic documentation in any manner, in whole
or in part, without the prior written permission of Silicon Graphics, Inc.
LIMITED RIGHTS LEGEND
Use, duplication, or disclosure by the Government is subject to restrictions as set forth in the Rights in Data clause at FAR 52.227-14 and/or in
similar or successor clauses in the FAR, or in the DOD, DOE or NASA FAR Supplements. Unpublished rights reserved under the Copyright Laws
of the United States. Contractor/manufacturer is Silicon Graphics, Inc., 1500 Crittenden Lane, Mountain View, CA 94043.
TRADEMARKS AND ATTRIBUTIONS
Silicon Graphics, SGI, the SGI logo, IRIX, and Origin are registered trademarks, and Altix, CXFS, FailSafe, InfiniteStorage, Octane2, and Silicon
Graphics Fuel are trademarks, of Silicon Graphics, Inc., in the United States and/or other countries worldwide.
Brocade and Silkworm are registered trademarks of Brocade Communications Systems, Inc. QLogic is a trademark of QLogic Corporation. All
other trademarks mentioned herein are the property of their respective owners.

007-4644-003 iii
Record of Revision
Version Description
001 May 2003
Original publication
002 Engineering updates for
Serial ATA
003 September 2004
Engineering updates for 2822 controllers and single-controller cabling

007-4644-003 v
Contents
Record of Revision . . . . . . . . . . . . . . . . . . . . . . iii
Figures . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Tables . . . . . . . . . . . . . . . . . . . . . . . . . . xi
About This Guide. . . . . . . . . . . . . . . . . . . . . . . xiii
Important Information . . . . . . . . . . . . . . . . . . . . . xiv
Chapter Descriptions . . . . . . . . . . . . . . . . . . . . . . xiv
Related Publications . . . . . . . . . . . . . . . . . . . . . . xv
Conventions . . . . . . . . . . . . . . . . . . . . . . . xvii
Product Support . . . . . . . . . . . . . . . . . . . . . . xvii
Reader Comments . . . . . . . . . . . . . . . . . . . . . . xviii
1. Introduction to the SGI InfiniteStorage TP9300 and TP9300S . . . . . . . . . . 1
System Features . . . . . . . . . . . . . . . . . . . . . . . 1
TP9300S RAID Features . . . . . . . . . . . . . . . . . . . . 2
Performance Features . . . . . . . . . . . . . . . . . . . . 3
RAID Controller Features . . . . . . . . . . . . . . . . . . . 4
Availability Features . . . . . . . . . . . . . . . . . . . . . 5
Supported Platforms . . . . . . . . . . . . . . . . . . . . . 5
Adapter and Switch Compatibility . . . . . . . . . . . . . . . . . 6
System Components . . . . . . . . . . . . . . . . . . . . . . 6
Storage Enclosures . . . . . . . . . . . . . . . . . . . . . 7
TP9300 Rack . . . . . . . . . . . . . . . . . . . . . . . 9
2. Storage Enclosures . . . . . . . . . . . . . . . . . . . . . . 13
Overview . . . . . . . . . . . . . . . . . . . . . . . . . 14
Disk Drives . . . . . . . . . . . . . . . . . . . . . . . . . 18
RAID Controllers . . . . . . . . . . . . . . . . . . . . . . . 19

vi 007-4644-003
Contents
Environmental Status Modules (ESMs) . . . . . . . . . . . . . . . . . 21
Fans . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Power Supplies . . . . . . . . . . . . . . . . . . . . . . . . 23
Tray ID Switch . . . . . . . . . . . . . . . . . . . . . . . . 25
Link Rate Switch . . . . . . . . . . . . . . . . . . . . . . . 26
SFP Transceivers . . . . . . . . . . . . . . . . . . . . . . . 27
3. Operating Storage Enclosures . . . . . . . . . . . . . . . . . . . 29
Turning the Power On . . . . . . . . . . . . . . . . . . . . . . 29
Turning the Power On after an Unexpected Shutdown . . . . . . . . . . . . 31
Troubleshooting an Overtemp Condition . . . . . . . . . . . . . . . 31
Turning the Power On after an Overtemp Shutdown . . . . . . . . . . . . 32
Turning the Power On after an Emergency Shutdown . . . . . . . . . . . 33
Turning the Power Off. . . . . . . . . . . . . . . . . . . . . . 34
Monitoring Status with Software . . . . . . . . . . . . . . . . . . . 35
Checking the Indicator Lights. . . . . . . . . . . . . . . . . . . . 36
Front Indicator Lights . . . . . . . . . . . . . . . . . . . . . 36
Rear Indicator Lights . . . . . . . . . . . . . . . . . . . . . 38
Moving the Enclosure . . . . . . . . . . . . . . . . . . . . . . 42
Removing the Enclosure . . . . . . . . . . . . . . . . . . . . 42
Reinstalling the Enclosure . . . . . . . . . . . . . . . . . . . 45
4. Replacing Enclosure Components . . . . . . . . . . . . . . . . . . 47
Replacing a Failed Disk Drive. . . . . . . . . . . . . . . . . . . . 47
Replacing a Failed Fan . . . . . . . . . . . . . . . . . . . . . . 50
Replacing a Failed Power Supply Module . . . . . . . . . . . . . . . . 52
Replacing a Failed SFP Transceiver . . . . . . . . . . . . . . . . . . 54
Replacing a Failed RAID Controller Module . . . . . . . . . . . . . . . 56
Replacing a Failed Environmental Status Module . . . . . . . . . . . . . . 58
Upgrading Drives . . . . . . . . . . . . . . . . . . . . . . . 61
Adding a Drive to an Empty Slot . . . . . . . . . . . . . . . . . 61
Adding Drives of Larger Capacity . . . . . . . . . . . . . . . . . 63
Adding Expansion Enclosures . . . . . . . . . . . . . . . . . . . 65

Contents
007-4644-003 vii
5. Cabling . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Connecting the Drive Loop Cables . . . . . . . . . . . . . . . . . . 69
Connecting the System to the Hosts . . . . . . . . . . . . . . . . . . 75
Examples of Direct Host Connection Cabling . . . . . . . . . . . . . . 75
Examples of Fabric/Switch Host Connection Cabling . . . . . . . . . . . 78
Connecting Ethernet and Serial Cables for Array Management. . . . . . . . . 79
Powering On the System . . . . . . . . . . . . . . . . . . . . . 81
A. Specifications and Requirements . . . . . . . . . . . . . . . . . . 85
Rack Specifications . . . . . . . . . . . . . . . . . . . . . . 85
Dimensions . . . . . . . . . . . . . . . . . . . . . . . 87
Weight . . . . . . . . . . . . . . . . . . . . . . . . . 88
Area Requirements . . . . . . . . . . . . . . . . . . . . . 90
Wiring and Power . . . . . . . . . . . . . . . . . . . . . 91
Environmental . . . . . . . . . . . . . . . . . . . . . . 96
Enclosure Specifications . . . . . . . . . . . . . . . . . . . . . 97
Dimensions . . . . . . . . . . . . . . . . . . . . . . . 97
Weights . . . . . . . . . . . . . . . . . . . . . . . . 98
Wiring and Power . . . . . . . . . . . . . . . . . . . . . 99
Environmental . . . . . . . . . . . . . . . . . . . . . .100
Air Flow . . . . . . . . . . . . . . . . . . . . . . . .101
B. SGI Field Engineering Compliance Statements . . . . . . . . . . . . . .103
Electromagnetic Emissions . . . . . . . . . . . . . . . . . . . .103
Radio and Television Interference . . . . . . . . . . . . . . . . . .103
Product Safety . . . . . . . . . . . . . . . . . . . . . . . .104
Electrostatic Discharge . . . . . . . . . . . . . . . . . . . . .104
Shielded Cables . . . . . . . . . . . . . . . . . . . . . . .105
FCC Warning . . . . . . . . . . . . . . . . . . . . . . . .105
VDE 0871/6.78 . . . . . . . . . . . . . . . . . . . . . . . .106
European Union Statement . . . . . . . . . . . . . . . . . . . .106
International Special Committee on Radio Interference (CISPR). . . . . . . . . .107
Canadian Department of Communications Statement . . . . . . . . . . . .107
Attention . . . . . . . . . . . . . . . . . . . . . . . . .107
Japanese Class A Compliance Statement . . . . . . . . . . . . . . . .107

007-4644-003 ix
Figures
Figure 1-1 TP9300 Tower and Rackmount Storage Enclosures (Front View). . 7
Figure 1-2 TP9300 Controller and Expansion Enclosures (Rear View) . . . 8
Figure 1-3 TP9300 Rack with Enclosures Installed . . . . . . . . . 10
Figure 1-4 Rack Features . . . . . . . . . . . . . . . . 11
Figure 2-1 Storage Enclosure Components (Front View) . . . . . . . 15
Figure 2-2 Storage Enclosure Components (Rear View) . . . . . . . 17
Figure 2-3 Disk Drives and Drive Bay Numbering . . . . . . . . . 18
Figure 2-4 2882 or 2822 RAID Controller . . . . . . . . . . . . 20
Figure 2-5 Environmental Status Modules . . . . . . . . . . . 21
Figure 2-6 Fan Units . . . . . . . . . . . . . . . . . 22
Figure 2-7 Power Supplies . . . . . . . . . . . . . . . . 24
Figure 2-8 Tray ID Switch . . . . . . . . . . . . . . . . 25
Figure 2-9 Link Rate Switch . . . . . . . . . . . . . . . 26
Figure 2-10 SFP Transceiver. . . . . . . . . . . . . . . . 27
Figure 3-1 Turning the Power On and Off . . . . . . . . . . . 30
Figure 3-2 Storage Enclosure Front Indicator Lights . . . . . . . . 37
Figure 3-3 Storage Enclosure Rear Indicator Lights. . . . . . . . . 39
Figure 3-4 Removing the Side Brackets from the Enclosure . . . . . . 43
Figure 3-5 Removing the Empty Drive Enclosure . . . . . . . . . 44
Figure 4-1 Removing and Installing a Drive Module . . . . . . . . 48
Figure 4-2 Removing and Installing a Fan Module . . . . . . . . . 51
Figure 4-3 Fault Lights for Fan Module Replacement . . . . . . . . 51
Figure 4-4 Removing and Installing a Power Supply Module . . . . . . 53
Figure 4-5 Removing and Installing an SFP Transceiver . . . . . . . 55
Figure 4-6 Removing and Installing a RAID Controller Module . . . . . 57
Figure 4-7 Removing SFPs from the Environmental Status Module . . . . 59
Figure 4-8 Removing and Installing an Environmental Status Module . . . 60
Figure 4-9 Adding New Expansion Enclosures to an Existing Array . . . . 66

x007-4644-003
Figures
Figure 4-10 Single-Controller Expansion Example . . . . . . . . . 67
Figure 5-1 Single Controller With Single Connection Drive Cabling Example . 70
Figure 5-2 Drive Loop Cabling for Systems With Two Controllers and Two
Enclosures . . . . . . . . . . . . . . . . . 71
Figure 5-3 Single Controller With Multiple Drive Enclosures Cabling Example . 72
Figure 5-4 Drive Loop Cabling for Systems with Three or More Enclosures . . 73
Figure 5-5 Drive Loop Cabling for Systems with Eight Enclosures . . . . 74
Figure 5-6 Direct Host Connection with Failover . . . . . . . . . 76
Figure 5-7 Direct Host Connection without Failover . . . . . . . . 77
Figure 5-8 Connecting to Hosts via a Fibre Channel Switch . . . . . . 78
Figure 5-9 Connecting Ethernet Cables . . . . . . . . . . . . 80
Figure 5-10 Power Connections and Switches of the Enclosures . . . . . 81
Figure 5-11 PDU Cabling. . . . . . . . . . . . . . . . . 82
Figure A-1 Populated Rack . . . . . . . . . . . . . . . . 86
Figure A-2 Dimensions of the System Rack . . . . . . . . . . . 87
Figure A-3 Rack Weights . . . . . . . . . . . . . . . . 89
Figure A-4 System Area Requirements . . . . . . . . . . . . 90
Figure A-5 Power Connections to Controller and Expansion Enclosures . . . 93
Figure A-6 AC Power Connectors and Receptacles . . . . . . . . . 95
Figure A-7 Dimensions of the System Enclosure . . . . . . . . . . 97
Figure A-8 Air Flow in the System Enclosure . . . . . . . . . . 101

007-4644-003 xi
Tables
Table 3-1 Storage Enclosure Front Indicator Lights . . . . . . . . 38
Table 3-2 Storage Enclosure Rear Indicator Lights. . . . . . . . . 40
Table A-1 Rack, Crate, and Enclosure Weights . . . . . . . . . . 88
Table A-2 AC Power Requirements (Domestic and International) . . . . 92
Table A-3 Environmental Requirements . . . . . . . . . . . . 96
Table A-4 Enclosure Weights . . . . . . . . . . . . . . . 98
Table A-5 Enclosure Module Weights . . . . . . . . . . . . 98
Table A-6 Site Wiring Voltages for System Enclosures
(Single-phase Line-to-neutral) . . . . . . . . . . . 99
Table A-7 Altitude Requirements for System Enclosures . . . . . . .100
Table A-8 Environmental Requirements for System Enclosures . . . . .100

007-4644-003 xiii
About This Guide
This guide provides an overview of the SGI InfiniteStorage TP9300 and TP9300S. It
covers routine operation and replacement procedures and provides troubleshooting and
reference information for all customer-replaceable components. All references to the
TP9300 apply equally to the TP9300S unless otherwise specified.
The SGI InfiniteStorage TP9300 is a highly scalable RAID storage system designed for
continuous availability. Some of the features of the system include hot-swappable
components, easy expansion, and redundant power and cooling systems.
The InfiniteStorage TP9300S RAID storage subsystem has nearly identical features as the
InfiniteStorage TP9300 but uses Serial Advanced Technology Attachment (SATA) drives
rather than fibre channel attached (SCSI-protocol) drives.
This guide is intended for system operators and service technicians who have extensive
knowledge of Fibre Channel network technology and computer system operation,
maintenance, and repair.
Use this guide to learn about all of the following:
• The parts of your system: the rack, controller enclosures, and expansion enclosures
• Operating your system correctly
• Replacing failed components

xiv 007-4644-003
About This Guide
Important Information
Danger: Never look into the end of a fiber-optic cable to confirm that light is being
emitted (or for any other reason). Most fiber-optic laser wavelengths (1300 nmand
1550nm) are invisible to the eye and cause permanent eye damage. Shorter wavelength
lasers (for example, 780 nm) are visible and can cause significant eye damage. Use only
an optical power meter to verify light output.
Danger: Never look into the end of a fiber-optic cable on a powered device with any
type of magnifying device, such as a microscope, eye loupe, or magnifying glass. Such
activity causes a permanent burn on the retina of the eye. Optical signals cannot be
verified by looking into the fiber end.
Chapter Descriptions
This guide contains the following chapters:
• Chapter 1, “Introduction to the SGI InfiniteStorage TP9300 and TP9300S,” describes
the features and components of the system.
• Chapter 2, “Storage Enclosures,” gives a detailed overview of the system enclosure
and its components.
• Chapter 3, “Operating Storage Enclosures,” describes the functions and basic
operation of the system enclosures, including power on and troubleshooting with
the indicator lights.
• Chapter 4, “Replacing Enclosure Components,” gives detailed procedures for
replacing failed components.
• Chapter 5, “Cabling,” describes cabling between controller and expansion
enclosures and between controllers and hosts.
• Appendix A, “Specifications and Requirements,” describes component
specifications and requirements for purposes of installation and maintenance.
• Appendix B, “SGI Field Engineering Compliance Statements,” describes the
regulatory and compliance information for the system.

About This Guide
007-4644-003 xv
Related Publications
This guide is part of a document set that fully supports the installation, operation, and
service of the TP9300 and TP9300S. See the following documents for more information
about your system. If a document number ends in “X,” use the latest available version of
that document.
•SGI InfiniteStorage TP9300 and TP9300S Installation and Upgrade Guide (108-0401-00x)
This guide gives complete instructions on how to unpack, install, and configure the
SGI TP9300/TP9300S and components. It also contains upgrade information.
•SGI TPSSM Administration Guide (007-4306-00x)
This guide gives complete instructions on how to install the TPSSM software for
host and/or client operation.
•SGI TPSSM Software Concepts Guide (007-4305-00x)
This guide explains the terminology and features of the TPSSM storage
management software.
•SGI Storage Area Network Installation Instructions (108-0252-00x)
This guide is included with the Fibre Channel switch and provides information on
storage area network installation and topologies.
You can obtain SGI documentation, release notes, or man pages in the following ways:
• See the SGI Technical Publications Library at http://docs.sgi.com. Various formats
are available. This library contains the most recent and most comprehensive set of
online books, release notes, man pages, and other information.
• If it is installed on your SGI system, you can use InfoSearch, an online tool that
provides a more limited set of online books, release notes, and man pages. With an
IRIX system, select Help from the Toolchest, and then select InfoSearch. Or you can
type infosearch on a command line.
• You can also view release notes by typing either grelnotes or relnotes on a
command line.
• You can also view man pages by typing man <title> on a command line.

xvi 007-4644-003
About This Guide
SGI systems include a set of IRIX man pages, formatted in the standard UNIX “man
page” style. These are found online on the internal system disk (or CD-ROM) and are
displayed using the man command. For example, to display the man page for the
Add_disk command, type the following on a command line:
man Add_disk
Important system configuration files and commands are documented on man pages.
References in the documentation to these pages include the name of the command and
the section number in which the command is found. For example, “Add_disk(1)” refers
to the Add_disk command and indicates that it is found in section 1 of the IRIX
reference.
For additional information about displaying reference pages using the man command,
see man(1).
In addition, the apropos command locates man pages based on keywords. For example,
to display a list of man pages that describe disks, type the following on a command line:
apropos disk
For information about setting up and using apropos, see apropos(1) and
makewhatis(1M).

About This Guide
007-4644-003 xvii
Conventions
The following conventions are used throughout this document:
Product Support
SGI provides a comprehensive product support and maintenance program for its
products:
• If you are in North America, contact the Technical Assistance Center at
[phone]SGI or contact your authorized service provider.
• If you are outside North America, contact the SGI subsidiary or authorized
distributor in your country.
Convention Meaning
Command This fixed-space font denotes literal items such as commands, files,
routines, path names, signals, messages, and programming language
structures.
variable The italic typeface denotes variable entries and words or concepts
being defined. Italic typeface also is used for book titles.
user input This fixed-space font denotes literal items that the user enters in
interactive sessions. Output is shown in nonbold, fixed-space font.
[ ] Brackets enclose optional portions of a command or directive line.
... Ellipses indicate that a preceding element can be repeated.
man page(x) Man page section identifiers appear in parentheses after man page
names.
GUI element This font denotes the names of graphical user interface (GUI)
elements such as windows, screens, dialog boxes, menus, toolbars,
icons, buttons, boxes, fields, and lists.

xviii 007-4644-003
About This Guide
Reader Comments
If you have comments about the technical accuracy, content, or organization of this
document, contact SGI. Be sure to include the title and document number of the manual
with your comments. (Online, the document number is located in the front matter of the
manual. In printed manuals, the document number is located at the bottom of each
page.)
You can contact SGI in any of the following ways:
• Send e-mail to the following address:
[email protected]
• Use the Feedback option on the Technical Publications Library website:
http://docs.sgi.com
• Contact your customer service representative and ask that an incident be filed in the
SGI incident tracking system.
• Send mail to the following address:
Technical Publications
SGI
1500 Crittenden Lane, M/S 535
Mountain View, California 94043
SGI values your comments and will respond to them promptly.

007-4644-003 1
Chapter 1
1. Introduction to the SGI InfiniteStorage TP9300 and
TP9300S
The SGI InfiniteStorage TP9300 is a high-bandwidth RAID Fibre Channel storage system.
The TP9300S RAID system is a variant of the TP9300 that uses Serial Advanced
Technology Attachment (SATA) disk drives instead of Fibre Channel (SCSI-protocol)
disks. Both types of drive arrays are fibre channel attached. All references to the TP9300
apply equally to the TP9300S unless otherwise specified. See “TP9300S RAID Features”
on page 2 for a list of specific differences and restrictions regarding the use of TP9300S
disks and enclosures.
This chapter gives a brief overview of the features of the TP9300 and TP9300S and its
primary components in the following sections:
• “System Features” on page 1
• “System Components” on page 6
System Features
The features of the TP9300 storage systems are listed in the following sections:
• “TP9300S RAID Features” on page 2
• “Performance Features” on page 3
• “RAID Controller Features” on page 4
• “Availability Features” on page 5
• “Supported Platforms” on page 5
• “Adapter and Switch Compatibility” on page 6

2007-4644-003
1: Introduction to the SGI InfiniteStorage TP9300 and TP9300S
TP9300S RAID Features
The TP9300S RAID system is a variant of the TP9300 that uses Serial Advanced
Technology Attachment (SATA) disk drives instead of Fibre Channel (SCSI-protocol)
disks. Both types of drive arrays are fibre channel attached.
While the form and function of the TP9300 and TP9300S are very similar, it is important
you note the following:
• Serial ATA drives (TP9300S) are not compatible with TP9300 Fibre Channel
(SCSI-protocol) disks and should never be used as spares in TP9300 Fibre Channel
(SCSI-protocol) drive arrays.
• Fibre Channel (SCSI-protocol) drives cannot be used as spares in serial ATA
enclosures.
• The 2822 RAID controller interface in a TP9300S will be different than a 2882 RAID
controller interface. The difference can be confirmed by special marking or part
numbers on the RAID controller canister.
– The 2882 RAID controllers cannot be used as spares or replacements for 2822
RAID controllers.
– The 2822 RAID controllers cannot be used as spares or replacements for 2882
RAID controllers.
• The serial ATA (TP9300S) drive enclosure’s front bezel will always be a different
color (purple) than the Fibre Channel (SCSI-protocol) array (TP9300) enclosure.
• The serial ATA drive enclosure will have a different storage capacity than a Fibre
Channel (SCSI-protocol) array using the same number of disks.
• The expansion enclosures contain environmental status modules instead of RAID
controllers. Each environmental status module contains an environmental services
monitor board (ESM) and two slots for SFPs (small form-factor pluggables). The
ESM board is the interface between the controller enclosure and the expansion
enclosure. The ESMs used in TP9300 and TP9300S are not interchangeable.
• The drive interface (ESM) in a TP9300S controller will be slightly different than
those in the TP9300. The difference can be confirmed by special marking or part
numbers on the SATA drive interface.
• Serial ATA ESMs and Fibre Channel (SCSI-protocol) ESMs are not interchangeable.