Silicon Graphics CHALLENGE RAID User manual

CHALLENGE®RAID
Owner’s Guide
Document Number 007-2532-004

CHALLENGE®RAID Owner’s Guide
Document Number 007-2532-004
CONTRIBUTORS
Written by Carolyn Curtis
Illustrated by Cheri Brown, Dan Young, and Carolyn Curtis
Production by Heather Hermstad
Engineering contributions by Sammy Wilborn, Albert Lui, Dale Witt, Carl Strasen,
Brad Eacker, and Paul Tsien
Cover design and illustration by Rob Aguilar, Rikk Carey, Dean Hodgkinson,
Erik Lindholm, and Kay Maitz
© Copyright 1995, Silicon Graphics, Inc.— All Rights Reserved
This document contains proprietary and confidential information of Silicon
Graphics, Inc. The contents of this document may not be disclosed to third parties,
copied, or duplicated in any form, in whole or in part, without the prior written
permission of Silicon Graphics, Inc.
RESTRICTED RIGHTS LEGEND
Use, duplication, or disclosure of the technical data contained in this document by
the Government is subject to restrictions as set forth in subdivision (c) (1) (ii) of the
Rights in Technical Data and Computer Software clause at DFARS 52.227-7013
and/or in similar or successor clauses in the FAR, or in the DOD or NASA FAR
Supplement. Unpublished rights reserved under the Copyright Laws of the United
States. Contractor/manufacturer is Silicon Graphics, Inc., 2011 N. Shoreline Blvd.,
Mountain View, CA 94043-1389.
Silicon Graphics, CHALLENGE, and IRIS are registered trademarks and IRIX, XFS,
IRIS FailSafe, POWER CHALLENGE, and POWER Channel are trademarks of
Silicon Graphics, Inc. Oracle Parallel Server and OPS are trademarks of Oracle
Corporation.

iii
Contents
List of Figures vii
List of Tables ix
About This Guide xi
1. Features of the CHALLENGE RAID Storage System 1
Storage System Components 5
SCSI-2 Interface 5
CHALLENGE RAID Storage-Control Processor 6
Storage System Chassis 8
Disk Modules 10
Data Availability and Performance 11
Data Redundancy 11
Enhanced Performance: Disk Striping 12
Enhanced Performance: Storage System Caching 12
Data Reconstruction and Rebuilding After Disk Module Failure 13
RAID Levels 14
RAID-0 Group: Nonredundant Array 14
RAID-1: Mirrored Pair 15
RAID-1_0 Group: Mirrored RAID-0 Group 16
RAID-5: Individual Access Array 18
RAID Hot Spare 20
Using the CHALLENGE RAID Command Line Interface 21
2. Storage System Configurations 23
Basic Configuration 25
Dual-Interface/Dual-Processor Configuration 26
Split-Bus Configuration 28
Dual-Bus/Dual-Initiator Configuration 31

iv
Contents
3. Operating the Storage System 35
Checking CHALLENGE RAID Storage System Status 36
Using the raid5 Command 37
Getting Device Names With getagent 38
Getting General System Information 39
Getting Information About Disks 39
Getting Information About Other Components 42
Displaying the CHALLENGE RAID Unsolicited Event Log 43
Shutting Down the CHALLENGE RAID Storage System 44
Restarting the CHALLENGE RAID Storage System 45
4. Configuring Disks 49
Binding Disks Into RAID Units 49
Getting Disk Group (LUN) Information 53
Changing LUN Parameters 56
Dual Interfaces, Load Balancing, and Device Names 57
5. Maintaining Disk Modules 59
Identifying and Verifying a Failed Disk Module 59
Setting Up the Workplace for Replacing or Installing Disk Modules 62
Replacing a Disk Module 63
Ordering Replacement Disk Modules 63
Unbinding the Disk 64
Removing a Failed Disk Module 65
Installing a Replacement Disk Module 69
Updating the Disk Module Firmware 71
Installing an Add-On Disk Module Array 72
Ordering Add-On Disk Module Arrays 72
Installing Add-On Disk Modules 73
Creating Device Nodes and Binding the Disks 77

Contents
v
6. Identifying Failed System Components 79
Power Supply 80
Fan Module 80
Battery Backup Unit 81
Storage-Control Processor 81
Diagnosing SP Failure 81
Using the Auto-Reassign Capability 82
7. Caching 85
Setting Cache Parameters 86
Viewing Cache Statistics 87
Upgrading CHALLENGE RAID to Support Caching 90
Changing Unit Caching Parameters 91
A. Technical Specifications 93
B. The raid5 Command Line Interface 97
bind 99
chglun 102
clearlog 104
clearstats 104
firmware 104
getagent 105
getcache 106
getcontrol 109
getcrus 109
getdisk 110
getlog 114
getlun 116
setcache 120
unbind 121
Index 123

vii
List of Figures
Figure 1-1 CHALLENGE RAID Storage System, Deskside Version:
Front View 2
Figure 1-2 CHALLENGE RAID Rack 3
Figure 1-3 Computing SCSI Cable Length Example: Single-Host
Configuration Only 4
Figure 1-4 CHALLENGE RAID Server With One SP 6
Figure 1-5 SPs Connected to the Same CHALLENGE Chassis 7
Figure 1-6 SPs Connected to Different CHALLENGE Chassis 7
Figure 1-7 Disk Module Locations (Chassis Front View) 8
Figure 1-8 SCSI-2 Bus and Internal Buses (Front View) 9
Figure 1-9 Disk Modules and Status Lights 10
Figure 1-10 RAID-1 Mirrored Pair (Hardware Mirrored Pair) 15
Figure 1-11 Distribution of User Data in a RAID-1_0 Group 17
Figure 1-12 Distribution of User and Parity Data in a RAID-5 Group 19
Figure 1-13 Hot Spare Example 21
Figure 2-1 Dual-Interface/Dual-Processor Configuration Example 27
Figure 2-2 Split-Bus Configuration Example 30
Figure 2-3 Dual-Bus/Dual-Initiator Configuration Example 33
Figure 3-1 CHALLENGE RAID Indicator Lights 36
Figure 3-2 Disk Module Locations 40
Figure 3-3 Turning Off (On) Power (Back of CHALLENGE RAID
Chassis) 45
Figure 3-4 CHALLENGE RAID Indicator Lights 45
Figure 3-5 Unlocking the Fan Module 46
Figure 3-6 Enabling an SP’s Power 47
Figure 5-1 Disk Module Status Lights 60
Figure 5-2 Disk Module Locations 61

viii
List of Figures
Figure 5-3 Attaching the ESD Clip to the ESD Bracket on a Deskside
Storage System 66
Figure 5-4 Attaching the ESD Clip to the ESD Bracket on a Rack
Storage System 66
Figure 5-5 Pulling Out a Disk Module 67
Figure 5-6 Removing a Disk Module 68
Figure 5-7 Engaging the Disk Module Rail 69
Figure 5-8 Engaging the Disk Module Guide 70
Figure 5-9 Inserting the Replacement Disk Module 70
Figure 5-10 Marking the Label for Disk Module A0 74
Figure 5-11 Disk Drive Locations 75
Figure 5-12 Engaging the Disk Module Rail 75
Figure 5-13 Engaging the Disk Module Guide 76
Figure 5-14 Inserting a Disk Module 76
Figure 6-1 Unlocking the Fan Module 83
Figure 6-2 Opening the Fan Module 83
Figure 6-3 Disabling an SP’s Power 84
Figure B-1 Disk Module Locations 111

ix
List of Tables
Table 2-1 CHALLENGE RAID Configurations 24
Table 2-2 Error Recovery: Basic Configuration 25
Table 2-3 Error Recovery: Dual Interface/Dual-Processor
Configuration 26
Table 2-4 Error Recovery: Split-Bus Configuration 28
Table 2-5 Error Recovery: Dual-Bus/Dual-Initiator
Configuration 31
Table 3-1 Output of raid5 getagent 38
Table 3-2 Output of raid5 getdisk 41
Table 3-3 Output of raid5 getcrus 43
Table 4-1 Output of raid5 getlun 54
Table 5-1 Ordering Replacement Disk Modules 63
Table 5-2 Ordering Add-On Disk Module Sets 72
Table 6-1 Field-Replaceable Units 79
Table 7-1 Output of raid5 getcache 88
Table A-1 CHALLENGE RAID Deskside Chassis Specifications 93
Table A-2 CHALLENGE RAID Rack Specifications 94
Table B-1 raid5 Parameters 97
Table B-2 Output of raid5 getagent 105
Table B-3 Output of raid5 getcache 107
Table B-4 Output of raid5 getcrus 110
Table B-5 Output of raid5 getdisk 112
Table B-6 getlog Error Codes 114
Table B-7 Output of raid5 getlun 118

xi
About This Guide
The CHALLENGE®RAID storage system provides a compact,
high-capacity, high-availability source of disk storage for the complete
line of Silicon Graphics®CHALLENGE servers running IRIX™ 5.3 and 5.3
with XFS™: CHALLENGE S, CHALLENGE DM, CHALLENGE L, and
CHALLENGE XL. It also works with IRIX 6.1 and 6.2 on the POWER
CHALLENGE L and XL.
The CHALLENGE RAID storage system uses high-availability disk storage
in as many as 20 disk modules. For even more storage, the CHALLENGE
RAID rack storage system offers up to four RAID chassis assemblies, each
with as many as 20 disk modules. The chassis assemblies in a CHALLENGE
RAID rack can be connected to one or more SCSI buses on CHALLENGE
servers separately or in combination.
RAID levels 0, 1, 1_0 (0+1), and 5 are supported, as well as disks configured
as hot spares. In addition, a basic CHALLENGE RAID storage system
provides storage-system caching.
Structure of This Guide
This guide contains the following chapters:
• Chapter 1, “Features of the CHALLENGE RAID Storage System,”
introduces the main CHALLENGE RAID components and summarizes
RAID levels and data availability and performance features.
• Chapter 2, “Storage System Configurations,” explains CHALLENGE
RAID configurations in detail: basic, dual-interface/dual-processor,
and split-bus.
• Chapter 3, “Operating the Storage System,” describes how to check
status, identify failing components, and start and shut down the
storage system.

xii
About This Guide
• Chapter 4, “Configuring Disks,” explains how to use the command line
interface to group disks into RAID-5 groups and how to display or
change information on groups of disks.
• Chapter 5, “Maintaining Disk Modules,” explains how to replace a
failed disk module and add a disk module array.
• Chapter 6, “Identifying Failed System Components,” explains how to
get status information on failed components other than disk modules.
• Chapter 7, “Caching,” explains how to determine, set up, and change
caching parameters.
• Appendix A, “Technical Specifications,” summarizes technical
information for the CHALLENGE RAID deskside storage system.
• Appendix B, “The raid5 Command Line Interface,” lists and explains
all parameters of the raid5 command.
An index completes this guide.
Conventions
In command syntax descriptions and examples, square brackets ( [ ] )
surrounding an argument indicate an optional argument. Variable
parameters are in italics. Replace these variables with the appropriate string
or value.
In text descriptions, IRIX filenames are in italics. The names of keyboard
keys are printed in boldface typewriter font and enclosed in angle brackets,
such as <Enter> or <Esc>.
Messagesandpromptsthatappearon-screen are shown in fixed-width type.
Entries that are to be typed exactly as shown are in boldface fixed-width
type.

xiii
Compliance Statements
This section lists various domestic and international hardware compliance
statements that pertain to the system.
FCC Warning
This equipment has been tested and found compliant with the limits for a
Class A digital device, pursuant to Part 15 of the FCC rules. These limits are
designed to provide reasonable protection against harmful interference
when the equipment is operated in a commercial environment. This
equipment generates, uses, and can radiate radio frequency energy and if
not installed and used in accordance with the instruction manual, may cause
harmful interference to radio communications. Operation of this equipment
in a residential area is likely to cause harmful interference, in which case the
user will be required to correct the interference at personal expense.
VDE 0871/6.78
This equipment has been tested to and is in compliance with the Level A
limits per VDE 0871.
European Union Statement
This device complies with the European Directives listed on the
“Declaration of Conformity” which is included with each product. The CE
mark insignia displayed on the device is an indication of conformity to the
aforementioned European requirements.
International Special Committee on Radio Interference (CISPR)
This equipment has been tested to and is in compliance with the Class A
limits per CISPR publication 22.
Canadian Department of Communications Statement
This digital apparatus does not exceed the Class A limits for radio noise
emissions from digital apparatus as set out in the Radio Interference
Regulations of the Canadian Department of Communications.

xiv
About This Guide
Attention
Le present appareil numerique n’emet pas de bruits radioelectriques
depassant les limites applicables aux appareils numeriques de Classe A
prescrites dans le Reglement sur le Brouillage Radioelectrique etabli par le
Ministere des Communications du Canada.
Japanese Compliance Statement

1
Chapter 1
1. Features of the CHALLENGE RAID Storage
System
RAID (redundant array of inexpensive disks) technology provides
redundant disk resources in disk-array configurations that make the storage
system more highly available and improve reliability and performance.
RAID was first defined by D. Patterson, G. Garth, and R. Katz of the
University of California, Berkeley, in their 1987 paper, “A Case for
Redundant Arrays of Inexpensive Disks (RAID)” (University of California,
Berkeley, Report No. UCB/CSD/87/391). That paper defines various levels
of RAID.
This chapter introduces the CHALLENGE RAID disk-array storage system.
It explains:
• CHALLENGE RAID storage system components
• data availability and performance
• RAID levels
• the RAID hot spare
• using the CHALLENGE RAID command line interface
Figure 1-1 is an external view of the deskside version of the CHALLENGE
RAID storage system.

2
Chapter 1: Features of the CHALLENGE RAID Storage System
Figure 1-1 CHALLENGE RAID Storage System, Deskside Version: Front View
Note: In Figure 1-1, the front cover is removed for clarity.
Figure 1-2 is an external view of the CHALLENGE RAID rack, with the
maximum of four chassis assemblies installed. Each chassis assembly in a
CHALLENGE RAID rack corresponds to one deskside CHALLENGE RAID
chassis.

3
Figure 1-2 CHALLENGE RAID Rack

4
Chapter 1: Features of the CHALLENGE RAID Storage System
The CHALLENGE RAID storage system provides a compact, high-capacity,
high-availability source of disk storage for your CHALLENGE S, DM, L, or
XL server system. The storage system offers a large capacity of
high-availability disk storage in multiple disk modules that you can replace
when the storage system is turned on.
The CHALLENGE RAID storage system connects by a small computer
system interface (SCSI-2) differential bus to a SCSI-2 interface in a
CHALLENGE server.
Only Silicon Graphics service personnel authorized to open the computer
cabinet and replace parts should install or replace the SCSI-2 interface. You
can replace the disk modules by following instructions in Chapter 5 in this
guide.
A CHALLENGE server can support multiple CHALLENGE RAID storage
systems. The various storage system configurations, along with their
availability and performance features, are explained in Chapter 2 in this
guide.
The number of CHALLENGE RAID deskside storage systems or rack
chassis assemblies that can be connected on one SCSI bus is limited by the
recommended SCSI bus length limit of 60 feet, as diagrammed in Figure 1-3.
Figure 1-3 Computing SCSI Cable Length Example: Single-Host Configuration
Only
CHALLENGE RAID
chassis
assembly
3 ft 8 ft
20 ft
RAID
chassis
assembly
8 ft
5 ft
RAID
chassis
assembly
8 ft
5 ft
SCSI differential
terminator
Total length in this example: 57 feet
Recommended maximum length: 60 feet

Storage System Components
5
Caution: Although the SCSI bus absolute length limit is 80 feet, exceeding
60 feet on the SCSI bus is not recommended. When cable lengths exceed 60
feet, problems can occur on the SCSI mezzanine card, the IO4B board, or
both.
Storage System Components
The CHALLENGE RAID deskside storage system, or each chassis assembly
in the CHALLENGE RAID rack, consists of these components:
• one or more host SCSI-2 interfaces that are either
– native to the POWER Channel™ 2 I/O controller (IO4 board) in the
CHALLENGE server
– HIO add-on cards (mezzanine cards) on the POWER Channel 2 I/O
controller
• one or two storage-control processors (SPs)
• 5 to 20 disk modules in groups of five
• one fan module
• two or three power supplies (VSCs, or voltage semi-regulated
converters)
• one battery backup unit (BBU) for storage system caching (optional)
The CHALLENGE server holds the SCSI-2 interface(s); the CHALLENGE
RAID storage system chassis holds the other components.
SCSI-2 Interface
The SCSI-2 interface transfers data between host memory and the SCSI-2
differential bus. A cable connects the SCSI-2 interface to the SP(s) in the
storage-system cabinet, as diagrammed in Figure 1-4. The SCSI-2 interface
has an operating system-specific device name and ID number.

6
Chapter 1: Features of the CHALLENGE RAID Storage System
CHALLENGE RAID Storage-Control Processor
The storage-control processor (SP) is a printed circuit board with memory
modules that resides in the storage-system cabinet. It controls the disk
modules in the storage system through a synchronous SCSI-2 bus. An SP has
fiveinternalfast/narrowSCSIbuses,eachsupporting four disk modules, for
a total of 20 disk modules.
Figure 1-4 diagrams a CHALLENGE RAID storage system with one SP.
Figure 1-4 CHALLENGE RAID Server With One SP
For higher performance, a CHALLENGE RAID storage system can support
an additional SP. The second SP provides a second path to the storage
system, so both SPs can connect to the same host or two different hosts, as
diagrammed in Figure 1-5 and Figure 1-6. With two SPs, the storage system
can support storage system caching, whereby each SP temporarily stores
modified data in its memory and writes the data to disk at the most
expedient time.
Note that the disks are owned by the SP, and not by the host, the SCSI-2 bus,
or the storage system.
CHALLENGE RAID
CHALLENGE server
SCSI-2
interface SP A
SCSI-2 bus