nStor Corporation Ultra S2S User manual

Ultra S2S RAID Controller
with AdminiStor PC Utilities
™

nStor Corporation, Inc.
© 1997-1998 nStor Corporation, Inc., All Rights Reserved.
Second Edition
nStor Corporation, Inc. shall not be liable for any damages or for the loss of any information resulting from the
performance or use of the information contained herein. Your rights to the software are governed by the license
agreement included with any accompanying software. nStor Corporation reserves the right to periodically revise this
manual without notice.
Product features and specifications described are subject to change without notice.
This manual may not be reproduced in whole or part, without prior written permission from nStor Corporation, Inc.
nStor Corporation, Inc.
450 Technology Park
Lake Mary, Florida 32746
nStor, AdminiStor, and the nStor logo are trademarks of nStor Corporation, Inc.
Mylex, DAC960, RAIDfx, and MORE are trademarks or registered trademarks of Mylex Corporation.
Other company and product names herein may be trademarks or registered trademarks of their respective
companies.

i
Contents
Preface
v
Where to Find Answers vi
Features vii
1 Introduction 1
Ultra S2S RAID Controller Components 1
Processor 2
SCSI Bus 2
Memory and DRAM Cache 2
Controller Firmware 3
SCSI Functions 3
Dual Active Controllers 7
Drive Organization 8
Drive Groups 8
Logical Units (LUNs) 8
Drive Management 8
Cache Management 12
RAID Management 13
Supported RAID Levels 13
2 Installation
17
Supported Application 17
Quick Start 18
Setup 19

Contents
ii
Cabling Configuration 22
Simplex Configuration 23
Duplex (Dual-Active) Configuration 25
Clustering (Dual Host) Configuration 35
3 Configuration
37
Method 1 – Using AdminiStor PC Utilities 37
Installing the AdminiStor PC Utilities 38
Starting AdminiStor PC Utilities 38
Configuring the Array 40
Option 1 – Automatic Configuration 50
Option 2 - New Configuration 52
Initializing the System Drives 62
Method 2– Using the VT100 Mode 66
Connecting the Terminal 66
Configuring the Controllers Parameters 66
Configuring the Array 70
Initializing the Array 74
Relinquish Partner (Dual-Active only) 76
VT100 Menu Functions 77
4 Administering the Array
79
Overview 79
Increasing Array Capacity 80
Manual Configuration 84
Initializing the System Drives 94
Rebuilding Drives 96
Running a Consistency Check 98
Using Kill Partner and Relinquish Partner 99
Kill Partner 99
Relinquish Partner 100
5 Maintenance
101
Overview 101
Get New Drives 101
Bad Block Table 102
Rebuild BBT 102
Write Back BBT 102
Error Counts 103

Contents
iii
Change Disk State 104
Drive Size File 105
Firmware 106
Customize 106
Download 106
Save Accel File 107
Controller Information 107
A Technical Information
109
Specifications 109
Index 111

Contents
iv

v
Preface
Congratulations on the purchase of your new Ultra S2S RAID Controller from
nStor Corporation. The Ultra S2S RAID Controller brings high-performance
transfer rates and fault-tolerant RAID operations in an Ultra SCSI environment
to the nStor CR8e disk array subsystem attached to any computing platform
equipped with a standard Ultra SCSI host adapter. The Ultra S2S RAID
Controller is a intelligent caching controller that supports standard RAID levels:
0, 1, 0+1, 3, and 5 for multiple-drive arrays, and JBOD (just a bunch of drives)
for single drive control.
This generation of RAID controller allows continuous access to data in the event
of a disk drive failure. It also provides continuous access to the data in the event
of a controller failure and this capability is achieved with a dual-active controller
system which uses two Ultra S2S RAID Controllers sharing access to the same
array of disk drives. In the event of one controller failure, operations are assumed
by the surviving controller through a process called fail-over. Once the failed
controller is replaced, the new controller resumes processing array operations in a
fail-back process. During fail-over and fail-back, write cache coherency is
maintained within the disk drives.
New product enhancements include online RAID expansion (MORE
™
) which
allows the addition of one or more drives to a RAID set while the controller is
online with the host system. Also, depending upon the choice of RAID
configurations, the Ultra S2S RAID Controller appears to the operating system as
one or more logical units, allowing maximized application performance.

Preface
Where to Find Answers
vi
The programmable LUN mapping feature allows flexible specification of the LUN
to system drive mappings. Specific LUN ID (or multiple LUN IDs) can be
assigned to any system drive on each channel or use a default setting. Ownership
of a system drive can be assigned to any combination of controller/channel,
known as System Drive Affinity.
The Ultra S2S RAID Controller now supports an optional battery backup module
for maintaining memory content in case of an AC power failure. Its principle
purpose is to provide ride-through during a power glitch, however, the battery
backup module is capable of sustaining memory content for hours at a time,
depending on the type and size of installed controller memory.
Another new feature is Drive Sizing. Previously when a drive failed it was
required to be replaced with a drive of equivalent size. If a larger capacity drive
was used, its capacity was defined as that of the smallest drive in the array. With
Drive Sizing, the user can specify how much of the drive’s total capacity should
be used. This permits the user to make various size drives and make them appear
to all be the same size from the perspective of the Ultra S2S RAID Controller.
Registering Your Product
Registering your product allows nStor to continue providing the highest quality
technical support while keeping you informed about product developments.
To register your product, complete and fax the Plan Registration/Upgrade Form
found in the Product Registration kit.
Where to Find Answers
When you have questions about your nStor products, there are several places you
can look to find answers. Refer to the following:
In this guide This user’s guide provides detailed information for
installing and using the Ultra S2S RAID Controller, the
S2S Interface card (Single-ended and Differential), and
the AdminiStor PC Utilities software. The manual
assumes that the reader is already familiar with the
operating system environments where the controller
will be installed.

Preface
Features
vii
Features
Enhanced SCSI Performance
■
Ultra SCSI channels provide data transfers up to 40MB/second
■
System Drive Affinity allows assignment of drives to a particular system
controller including a duplex system
■ Tagged-command queuing to the host allows processing of up to 64
simultaneous data requests
■
User-defined performance tuning through selectable cache write policy and
variable stripe width
■
Disconnect/reconnect for SCSI bus optimization
■
Configuration stored on disk and controller NVRAM (if the controller is
replaced, disk and controller automatically reconfigure upon startup)
Resource Guide Use this manual to locate telephone numbers for
customer service and technical support, technical
support email address, instructions to obtain an RMA
number, and conditions of the limited warranty.
Feature Card User’s Guide
Use this manual for information on installing,
configuring, and using the nStor Ultra Extender and
Differential Converter cards.
CR8e User’s Guide Use this manual for procedures to install and use the
nStor CR8e disk array subsystem.
Battery Backup Module
documentation
Use this document for information on how to install
and use the nStor Ultra S2S RAID Controller – Battery
Backup Module.
AdminiStor Agent User’s
Guides
Use one of these manuals to install and use the nStor
SNMP-based network RAID subsystem management
software (NetWare, Windows NT, Solaris, and/or SGI).
On the software
diskettes
Review the README files for last minute information
about the release of the software products.
On the web site Refer to the nStor web site for the latest information on
products and product upgrades. (www.nstor.com)
On the ftp site Refer to the nStor ftp site for Adobe Acrobat versions of
all product user guides, addendums, and flyers.
(www.nstor.com)

Preface
Features
viii
Managed RAID/SCSI Disk Arrays
■
Multiple RAID level support
■
Online expansion allows drive(s) to be added with existing logical drives
■
Array configuration and management without special software or drivers
■
Support for all popular operating system environments (works
independent from the OS)
■
Connect up to 15 physical drives per drive channel
■
Drives can be grouped and managed as a single very large array, as multiple
large capacity drive groups, or as individual disk drives
■
Industry standard Fast-20/Wide SCSI-3 (Ultra Wide) interface
Flexible User Interface
■
Software utility (AdminiStor PC Utilities) provides DOS-level control
through a PC running ASPI drivers or Windows NT Pass-Through
■
Serial communications permits array control through a standard VT100 (or
equivalent) terminal
Automates RAID Functions
■
Automatic failed-drive detection
■
Supports AEMI protocols for automatic rebuild of the array using a
stand-by (hot spare) disk after a drive failure
■
Transparent drive rebuild permits automatic rebuild of failed drives during
normal operation without having to take the array off-line
■
Automatic error detection/correction of parity errors and bad blocks
■
Automatic sector re-mapping recovers defective media and corrects data
errors
■
Supports SAF-TE protocol for integrated monitoring of enclosure power
supplies, fans, and temperature
Continuous Operations
■ Each controller in a dual-active configuration monitors the status of its
partner controller
■ Failure of one controller in a dual-active configuration automatically
initiates a process whereby the surviving controller handles operations for
both controllers

1
1
Introduction
Ultra S2S RAID Controller Components
Some of the key components of the Ultra S2S RAID Controller include:
■
Intel i 960 RISC processor
■
Memory and DRAM cache
■ SCSI and I/O Functions
■
Battery Backup Module (optional)
Ultra S2S RAID Controller
Cache Memory
Module (SIMM)
(J5) Jumper
Block
Diagnostic Connector
(Factory use only)
Latch/Handle
J5

Introduction
Processor
2
Processor
The Ultra S2S RAID Controller uses an Intel i 960 RISC microprocessor which
operates at 33 MHz, has 4 KB of instruction cache and 1 KB of data cache to
manage all controller functions. These include: SCSI bus transfers, RAID
processing, configuration, data striping, error recovery, and drive rebuild.
SCSI Bus
The Ultra S2S RAID Controller uses the Symbios Logic 53C770 SCSI I/O
processor (SIOP) chip on each SCSI channel, allowing the controller to
simultaneously communicate with the host system and read or write data on
several drives. Up to twelve (12) disk drives can be connected to each SCSI
channel.
The controller supports the Fast-20/Wide SCSI-3 (Ultra Wide) standard, which is
backward compatible with earlier SCSI standards.
Memory and DRAM Cache
The Ultra S2S RAID Controller implements a scalable memory design utilizing
interleaved fast page mode, EDO, or self-refreshing DRAM.
Two SIMM locations are provided for standard, off-the-shelf, 36-bit 60ns (or
faster) DRAM SIMMs. Since interleaving is required to provide maximum
performance, an identical memory SIMM is required in each of the two SIMM
sockets. Up to 128 MB of memory can be installed on the controller.
A fast 32-bit interface between the i 960 CPU and the cache memory DRAM is
provided by the Memory Control Unit (MCU), which is implemented in discrete
programmable logic. In addition to memory control and addressing functions, the
MCU provides the device mapping and decode for the non-volatile memory
(NVRAM) and the electronically-erasable/programmable read-only memory
(flash EEPROM).

Introduction
Controller Firmware
3
Controller Firmware
The controller firmware contains the programs executed by the i 960 CPU. The
firmware resides in the on-board flash EEPROM. This memory device retains
information even after power is off and can be re-written to allow the controller
firmware to be upgraded.
In addition to the stored programs in EEPROM, the NVRAM stores data on the
current configuration of the controller and its attached disk drives. As the disk
drive configuration changes (e.g., drive fails) the NVRAM keeps a record of the
changes.
Configuration on Disk
The configuration on disk allows the Ultra S2S RAID Controller to detect certain
hardware changes when they occur and automatically reconfigure accordingly.
Automatic reconfiguration occurs after:
■
Change of a failed controller
■ Change of target IDs (relocating drives) or replacement of drives
■ Interchange of cables
■ Drive failure occurs during a power down
The configuration information is stored in the controller NVRAM and in the last
16 sectors of every working physical hard disk. This is information transparent to
the operating system.
SCSI Functions
The Ultra S2S RAID Controller’s i 960 RISC processor and SCSI I/O processors
provide an intelligent, high-performance SCSI interface and control. The
controller manages and controls the SCSI bus arbitration between the controller
and its connected devices, and all SCSI activity of the connected device.
Internally within the CR8e, the lower bay drives are connected to the controller’s
Channel 0 SCSI I/O processor (SIOP) and the upper bay drives are connected to
the controller’s Channel 1 SCSI I/O processor. Up to twelve (12) drives can be
connected to each of the SIOP channels. The upper and lower bays each contain
up to four disk drives. Expansion can be accomplished by connecting additional
CR8e subsystems through the Ultra Extender feature card. Refer to the diagrams
in the installation chapter and the Feature Card User’s Guide for more information.

Introduction
SCSI Functions
4
The controller provides one ultra wide SCSI channel (single-ended or differential)
out to the host system. This connection is accomplished via either the S2S
Single-ended Interface card or an S2S Differential Interface card. These cards will
be installed in the upper (or right) feature card slot located on the rear panel of
the CR8e subsystem.
S2S Single-Ended Interface Card
Termination Jumper
(remove to enable termination)
Serial B Connector for
Intercabinet Communications
(RJ11)
Serial A Connector
for VT-100 Terminal (RJ45)
68-Pin VHD
Connector (2 places)
OUT IN
AB
SERIAL
ADD JUMPER TO
DISABLE TERMINATION
JP1
S2S Differential Interface Card
OUT IN
AB
SERIAL
ADD JUMPER TO
DISABLE TERMINATION
JP1 CHANNEL - 0 & 1
Serial B Connector for
Intercabinet Communications
(RJ11)
Serial A Connector
for VT-100 Terminal (RJ45)
68-Pin VHD
Connector (2 places) Termination Jumper
(remove to enable termination)

Introduction
SCSI Functions
5
SCSI Bus Termination
When a SCSI-to-SCSI configuration exists, there are essentially two SCSI buses
present: the host SCSI bus and the drive SCSI bus.
Host SCSI Bus
The host SCSI bus consists of the host bus adapter, the data cable, the S2S
Interface card, and the Ultra S2S RAID Controllers’ host channel SCSI I/O
processor. SCSI bus termination is required on the host SCSI bus and occurs
at the host bus adapter and on the last S2S Interface card on the SCSI bus.
Since multiple CR8e subsystems can be daisy-chained together through this
host SCSI bus, SCSI termination will need to be enabled only on the last S2S
Interface card on the bus. A jumper located on the S2S Interface card (JP1) is
removed to enable SCSI bus termination when the card is the last device in
the SCSI bus chain (off-set one pin to act as a hold for the jumper). For
multiple cabinets disable SCSI bus termination on the interim S2S Interface
cards by installing the jumper at JP1 and terminating the last S2S Interface
card in the chain (removing jumper).
Drive SCSI Bus
The drive SCSI bus consists of the Ultra S2S RAID Controller’s disk channel
0 and 1 SCSI I/O processors, the backplane circuitry, the disk drives, and the
I/O Interface card. If there are multiple cabinets connected in a master/slave
configuration, the drive SCSI bus includes all cabinet drives, backplane
circuitry, data cables, Ultra Extender cards, and I/O Interface cards.
The Ultra S2S RAID Controller supports active termination providing a
stable quality SCSI termination signal. Drive SCSI bus termination is
provided automatically by either the Ultra Extender or I/O Interface cards
and requires no user intervention. The Ultra Extender and I/O Interface
cards are designed to sense whether they are the last device on the SCSI bus
and if so, automatically self terminate.

Introduction
SCSI Functions
6
SCSI Address Selection (SCSI ID)
Controller ID Selection
The SCSI ID of the Ultra S2S RAID Controller is set to a target address (SCSI
ID) of 0 by default on power up. This setting should be adequate for a single
cabinet configuration in stand-alone solutions. When multiple cabinets are
used the target address of each Ultra S2S RAID Controller must be set to a
unique SCSI ID that is available on the host SCSI bus. Use the “SCSI ID
Menu > Set RAID Module ID” selection located on the CR8e operator
control panel to set the target address. Refer to the CR8e User’s Guide , “Using
the CR8e” chapter – Options Menu selection.
When the SCSI ID is changed, the subsystem will require power cycling to
allow the new settings to take effect. Note that you will under normal
conditions be setting the Drive IDs as the next step, you can set the RAID
Module ID then access the menu and change the setting for SAFTE Chain ID
functions before power cycling the subsystem. This saves you from power
cycling the subsystem twice.
Drive ID Selection
Each drive on each channel of the Ultra S2S RAID Controller must be
configured for a target address (SCSI ID) that is different from all other
devices. The CR8e subsystem will automatically set the target address of each
device during the power cycle after the appropriate “SAFTE Chain ID”
selection is set on the CR8e operator control panel. (Refer to the “Using the
CR8e” chapter – SAFTE Chain ID in the nStor CR8e User’s Guide .)
Serial Communications
The Ultra S2S RAID Controller supports two serial asynchronous communication
channels. The two channels are labeled on the S2S Interface Card as Serial A and B.
The Serial A connector is an RJ45 type connector and provides the
communication link to a VT-100 (or equivalent) terminal and is used to configure
the RAID subsystem.
The Serial B connector is an RJ11 type connector and provides the communication
link between two Ultra S2S RAID Controller controllers for basic communications
in a Dual-Active configuration. Cables are included in the accessory kits.

Introduction
Dual Active Controllers
7
Dual Active Controllers
A dual-active (active-active) fault-tolerant configuration is provided when two
CR8e subsystems, each with Ultra S2S RAID Controllers are installed. This
configuration allows two separate host systems continuous concurrent access to
data. Refer to the installation chapter for diagram and setup information, and the
configuration chapter for software configuration.
Should a fault occur in one controller, the other controller detects the failure and
assumes operations for the failed controller. When the failed controller is
subsequently removed and replaced with a functioning controller, the active
functioning controller detects the presence of the new controller and relinquishes
its assumed responsibilities to the new controller. This process is known as a
fail-over and fail-back. Write cache coherency is maintained with the disk drives
during this process.
The following requirements must be met to establish a dual-active controller
environment:
■
both controllers must have the same amount of memory
■ both controllers must be at the same firmware level
■
all disk SCSI channels must be connected one-to-one
Performance
The advantage gained by the dual-active configuration is an additional margin of
fault tolerance. However, when operating in a dual-active mode and one
controller fails, the performance of the remaining single controller is somewhat
degraded from that of a one controller in a separate environment.

Introduction
Drive Organization
8
Drive Organization
The Ultra S2S RAID Controller organizes the connected drives as drive groups
and logical units .
Drive Groups
When using the Ultra S2S RAID Controller, up to eight individual disk drives can
be used together to form a pack or drive group . These physical drives make up the
array’s logical unit capacity. The controller supports up to eight drive groups. The
number of drives in a drive group determines the possible RAID levels available
when configuring the array.
A new feature of the controller is known as Drive Sizing. This solution allows the
user to define how much of the drive’s total capacity should be used. This permits
the user to make differently sized drives appear to all be the same size from the
perspective of the controller.
Logical Units (LUNs)
A logical unit (system drive) is that portion of a drive group (or a combination of
up to four drive groups) seen by the host system as a single logical device. Each
logical unit is identified by its logical unit number (LUN). The Ultra S2S RAID
Controller supports up to eight LUNs (system drives).
■
A LUN can span up to four (4) drive groups
■
A LUN can span up to a maximum of 32 drives
■A maximum of eight (8) LUNs per drive group
Use the VT100 utility or the AdminiStor PC Utilities (RAIDfx) to configure the
logical units from one drive group. Use only the AdminiStor PC Utilities to
configure logical units that span more than one drive group.
Drive Management
The Ultra S2S RAID Controller functions that monitor and control the operation
of the physical drives and logical units are instrumental to the controller’s ability
to perform RAID management and automated error recovery tasks.

Introduction
Drive Management 9
Controlling Physical Drive States
The state of a physical drive refers to a SCSI drive’s current operational status. At
any given time, a SCSI drive can be in one of several states: Ready, Online,
Standby, Rebuild, Dead, Format, or Unformatted. The operational status of a disk
drive is indicated by a three-letter status code.
The controller stores the state of the attached SCSI drives in its non-volatile
memory as well as on the disks. This information is retained even after the power
is turned off.
These drive states are defined as:
Ready (RDY) – a disk drive is operational but not currently included in a
drive group.
Online (ONL) – a disk drive is powered, has been defined as a member of a
drive group, and is operating properly.
Standby (SBY) – a disk drive is powered, able to operate properly, has not
been defined as part of any drive group, and has been defined to replace any
failed/removed ONLINE drive.
Dead (DED) – a disk drive is not present, is present but not powered on, or
fails to operate properly and is killed by the controller (whether or not it has
been defined as a member of a drive group).
Rebuild (RBL or WOL) – a disk drive is in the process of being rebuilt
a) during a RAID 1 rebuild, data is being copied from the mirrored drive
to the replacement drive or
b) during a RAID 3, RAID 5, or RAID 0+1 rebuild, data is being
regenerated via the XOR redundancy algorithm and written to the
replacement drive.
NOTE: RBL is displayed when a VT100 terminal is used and WOL for the
AdminiStor PC Utilities (RAIDfx)
Format (FMT) – displayed when a disk drive is being formatted.
Unformatted (UNF) – displayed when a disk drive needs to be formatted.

Introduction
Drive Management
10
Controlling Logical Unit States
The logical unit can be Online, Critical, or Offline. Note that the same term
“online” is used for both physical drives and logical units.
These states are defined as:
Online – a logical unit has all of its participating physical drives on-line.
Critical – a logical unit has any of its physical drives not on-line. Note that in
a RAID 0 configuration, logical units are always in a critical state.
Offline – a logical unit is either:
a) it is configured with a redundant RAID level (1, 3, 5, 0+1) and two or
more of its drives are not on-line, and/or
b) it is configured as RAID 0 or JBOD and one or more of its drives is not
on-line.
Controlling Standby Replacement Drives (Hot Spare)
The standby replacement drive (hot spare) is one of the most important features
in providing a high degree of fault-tolerance. With the standby rebuild function,
the controller performs a rebuild operation automatically when a disk drive fails
and both of the following conditions are true: a) a standby disk drive of identical
or larger capacity is found attached to the same controller, and b) all of the system
drives that are dependent on the failed disk are configured in a RAID 1, RAID 3,
RAID 5, or RAID 0+1 logical unit.
During the automatic rebuild process, system activity continues as normal.
System performance may degrade slightly, however, during a rebuild. Priority of
rebuild activity can be adjusted using the configuration utilities to adjust
performance versus rebuild time.
A standby replacement drive can be created in one of two ways: a) a disk may be
labeled a standby drive using the “Create Standby” option under the
Configuration menu using the VT100 option, or b) when the configuration is
created or changed using the software utility, any disks attached but not assigned
to a drive group will be automatically labeled as standby replacement drives.