EMC2 CX3 UltraScale Series Installation guide

CX3 Model 80 System
Hardware and Operational
Overview
July 7, 2008
This document describes the hardware, powerup and powerdown
sequences, and status indicators for CX3 model 80 systems, which are
members of the CX3 UltraScale™series of storage systems.
Major topics are:
Storage-system major components.................................................. 2
Storage processor enclosure (SPE2)................................................. 4
Disk-array enclosures (DAE3Ps)..................................................... 9
Standby power supplies (SPSs)....................................................... 16
Powerup and powerdown sequence ............................................... 17
Status lights (LEDs) and indicators ................................................. 22
1

Storage-system major components
The storage systemconsistsof:
A storage processor enclosure (SPE2) and two standby power
supplies (SPSs)
One Fibre Channel disk-array enclosure (DAE) with a minimum
of five disk drives
Optional DAEs
EMC3465
DAE3P
SPE2
SPS
Figure 1 Storage system
The high-availability features for the storage system include:
Redundant storage processors (SPs)
Standby power supplies (SPS)
Redundant power supplies
Redundant blowers
TheSPE2isahighlyavailablestorageenclosurewithredundantpower
and cooling. It is 4U high (a U is a NEMA unit; each unit is 1.75 inches)
2Hardware and Operational Overview

and includes two storage processors (SPs). Each SP has four Fibre
Channel front-end I/O ports and four back-end disk ports.
The storage system supports 4 Gb/s Fibre Channel operation from its
front-end host I/O ports through its back-end disk ports. The host I/O
front-end ports can operate at up to 4 Gb/s and the back-end ports
can operate at 2 or 4 Gb/s. The storage system senses the speed of
the incoming host I/O and sets the speed of the front-end ports to the
lowest speed it senses. The speed of each back-end port is determined
by the speed of the DAEs connected to it.
The storage systemrequiresatleastfivedisksandworksinconjunction
with one or more disk-array enclosures (DAEs) to provide terabytes of
highly availablediskstorage.ADAEisabasicdiskenclosurewithout
an SP. SPE2 systems include a 4 Gb/s point-to-point DAE3P, which
supports up to 15 Fibre Channel disks. Each DAE3P connects to the
SPE2 or another DAE with simple FC-AL serial cabling.
The storage system supports a total of 480 disks on its back-end
buses. A four-bus configuration (with maximum performance and
availability)supportsasmanyas120disksperback-endbus;atwo-bus
configurationsupportsasmanyas240disksperback-endbus.Youcan
place the diskenclosuresinthesamecabinetastheSPE,orinoneor
more separate cabinets. High-availability features are standard.
CX3 Model 80 storage systems require 200–240 V AC input power.
Hardware and Operational Overview 3

Storage processor enclosure (SPE2)
The SPE2 components include:
A sheet-metal enclosure with a midplane and front bezel
Two storage processors (SPs), each consisting of one CPU module
and two I/O modules
Two management modules, each with SPS, management, and
service connectors
Four blowers
Two power supplies
Figure 2 shows the SPE2 components. Details on each component
follow the figure. If the enclosure provides slots for two identical
components, the component in slot A is called component-name A. The
second component is called component-name B. For increased clarity, the
following figures depict the SPE2 outside of the rack cabinet. Your
SPE2 may be installed in a rackmount cabinet.
4Hardware and Operational Overview

ABC D
!!
! !
! !
+- +-
Blower modules
Power supply A Power supply B
Rear
I/O module 0, SP B I/O module 1, SP B
Management
module B
Management
module A
Filler
panel
Front
I/O module 0, SP A I/O module 1, SP A
EMC3432
SPS ASPS B
Figure 2 SPE2 outside the cabinet — front and rear views
Midplane
The midplane distributes power and signals to all the enclosure
components. The power supplies, blowers, and storage processors
(SPs) plug directly into midplane connectors.
Front bezel
The front bezel has a key lock and two latch release buttons. Pressing
the latch release buttons releases the bezel from the enclosure. The
front bezel inhibits electromagnetic interference (EMI). You can take
off the bezel to replace parts, but EMI compliance requires a properly
installed front bezel.
Storage processors (SPs)
The SP is the SPE2’s intelligent component and acts as the control
center. Each SP includes:
One CPU module with:
zDual processors
z8 GB of DDR DIMM (double data rate, dual in-line memory
module) memory
Two I/O modules, each with:
zTwo SFP (small-form factor pluggable) shielded Fibre Channel
connectors (optical) for server I/O (connect to a switch or server
HBA)
Hardware and Operational Overview 5

zTwo SFP shielded Fibre Channel connectors (copper SFP) for
disk connections (BE 0 and BE 1 on I/O module 0; BE 2 and
BE 3 on I/O module 1).
Figure 3 show the locations of the connectors on the rear of the SPs.
!!
! !
! !
+- +-
EMC3397
Rear
BE 0
BE 1
0 Fibre
1 Fibre
BE 2
BE 3
2 Fibre
3 Fibre
BE 0
BE 1
0 Fibre
1 Fibre
BE 2
BE 3
2 Fibre
3 Fibre
Figure 3 Connectors on the rear of the CX3-80 SPs
Management modules
Each SPE2 storage processor has a corresponding management module
that includes:
One serial port for connection to a standby power supply (SPS) –
micro DB9 connector
One 10/100 Ethernet LAN port for management and backup – RJ45
connector
One 10/100 Ethernet LAN port for peer service – RJ45 connector
One serial port for RS-232 connection to a service console – micro
DB9 connector
6Hardware and Operational Overview

!!
! !
! !
+- +-
EMC3368
!
+-
Power
Fault
Service
SPS
Service
Management
LAN
Figure 4 CX3–80 management module
Power supplies
A power supply is located on each side of the SPE2. Each power
supply is an auto-ranging, power-factor-corrected, multi-output,
offline converter with its own line cord and power switch. Each supply
supports the SPE2 and shares load currents with the other supply.
An SP or power supply with power-related faults does not adversely
affect the operation of any other component. If one power supply fails,
the write caching is disabled. You can replace a failed power supply
while the SPE2 is powered up.
Blowers
All four blowers cool both storage processors. If one blower fails,
the other blowers speed up to compensate. If two blowers fail, the
SPE2 powers down. You can replace a failed blower while the SPE2
is powered up.
SPE2 field-replaceable units (FRUs)
The following are field-replaceable units (FRUs) that you can replace
while the SPE2 is powered up:
CPU modules
Memory modules (DIMMs)
I/O modules
Hardware and Operational Overview 7

Management modules
Small form-factor pluggable (SFP) modules, which plug into the
Fibre Channel front-end port slots
Power supplies
Blowers
Youoryourservic
e provider can replace a failed power supply, blower,
or SFP module. A service provider must replace the other FRUs if they
fail.
8Hardware and Operational Overview

Disk-array enclosures (DAE3Ps)
DAE3P UltraPoint™(sometimes called "point-to-point") disk-array
enclosures are highly available, high-performance, high-capacity
storage-system components that use a Fibre Channel Arbitrated Loop
(FC-AL) as the interconnect interface. A disk enclosure connects to
another DAE3P or an SPE2 and is managed by storage-system software
in RAID (redundant array of independent disks) configurations.
The enclosure is only 3U (5.25 inches) high, but can include 15 hard
disk drive/carrier modules. Its modular, scalable design allows for
additional disk storage as your needs increase.
ADAE3Pinclud
es either high-performance Fibre Channel disk
modules or economical SATA (Serial Advanced Technology
Attachment, “SATA II”) disk modules. You can integrate and connect
FC and SATA enclosures within a storage system, but you cannot mix
SATA and Fibre Channel components within a DAE3P. The enclosure
operates at either 2 or 4 Gb/s bus speed (2 Gb/s components, including
disks, cannot operate on a 4 Gb/s bus)..
Simple serial cabling provides easy scalability. You can interconnect
disk enclosures to form a large disk storage system; the number and
size of buses depends on the capabilities of your storage processor. You
can place the disk enclosures in the same cabinet, or in one or more
separate cabinets. High-availability features are standard in the DAE.
The DAE3P includes the following components:
A sheet-metal enclosure with a midplane and front bezel
Two FC-AL link control cards (LCCs) to manage disk modules
As many as 15 disk modules
Two power supply/system cooling modules (referred to as
power/cooling modules)
Any unoccupied disk module slot has a filler module to maintain air
flow.
The power supply and system cooling components of the
power/cooling modules function independently of each other, but the
assemblies are packaged together into a single field-replaceable unit
(FRU).
Hardware and Operational Overview 9

The LCCs, disk modules, power supply/system cooling modules,
and filler modules are field-replaceable units (FRUs), which can be
added or replaced without hardware tools while the storage system
is powered up.
Figure 5 shows the disk enclosure components. Where the enclosure
provides slots for two identical components, the components are called
component-name Aorcomponent-name B,asshownintheillustrations.
For increased clarity, the following figures depict the DAE3P outside of the
rack or cabinet. Your DAE3Ps may arrive installed in a rackmount cabinet
along with the SPE.
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
Power LED
(green or blue)
Fault LED
(amber)
Power/cooling module B Link control card B
Fault LED
(amber)
Disk activity
LED (green)
Power/cooling module A Link control card A
EMC3437
Figure 5 DAE3P outside the cabinet — front and rear views
AsshowninF
igure 6, an enclosure address (EA) indicator is located on
each LCC. (TheEAissometimesreferredtoasanenclosureID.)Each
link control card (LCC) includes a bus (loop) identification indicator.
The storage processor initializes bus ID when the operating system
loads.
10 Hardware and Operational Overview

!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
0 1 2 3
4 5 6 7
0 1 2 3
4 5 6 7
Bus ID
Enclosure
address
#
EA selection
(press here to
change EA)
EMC3210
Figure 6 Disk enclosure bus (loop) and address indicators
The enclosure address is set at installation. Disk module IDs are
numbered left to right (looking at the front of the unit) and are
contiguous throughout a storage system: enclosure 0 contains modules
0-14; enclosure 1 contains modules 15-29; enclosure 2 includes 30-44,
and so on.
Midplane
A midplane between the disk modules and the LCC and power/cooling
modules distributes power and signals to all components in the
enclosure. LCCs, power/cooling modules, and disk drives – the
enclosure’s field-replaceable units (FRUs) – plug directly into the
midplane.
Front bezel
The front bezel has a locking latch and an electromagnetic interference
(EMI) shield. You must remove the bezel to remove and install drive
modules. EMI compliance requires a properly installed front bezel.
Link control cards (LCCs)
An LCC supports and controls one Fibre Channel bus and monitors
the DAE3P.
Hardware and Operational Overview 11

!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
EMC3226
Expansion link
active LED
Primary link
active LED
Fault LED (amber)
Power LED (green)
!
EXP PRI
EXPPRI
Figure 7 LCC connectors and status LEDs
A blue link active LED indicates a DAE3P enclosure operating at 4 Gb/s. The
link active LED(s) is green in a DAE3P operating at 2 Gb/s.
The LCCs in a DAE3P connect to other Fibre Channel devices
(processor enclosures, other DAE3Ps) with twin-axial copper cables.
The cables connect LCCs in a storage system together in a daisy-chain
(loop) topology.
Internally, each DAE3P LCC uses FC-AL protocols to emulate a loop;
it connects to the drives in its enclosure in a point-to-point fashion
through a switch. The LCC independently receives and electrically
terminates incoming FC-AL signals. For traffic from the system’s storage
processors, the LCC switch passes the input signal from the primary
port (PRI) to the drive being accessed; the switch then forwards the
drive’s output signal to the expansion port (EXP), where cables connect
it to the next DAE3P in the loop. (If the target drive is not in the LCC’s
enclosure, the switch passes the input signal directly to the EXP port.)
At the unconnected expansion port (EXP) of the last LCC, the output
signal (from the storage processor) is looped back to the input signal
source (to the storage processor). For traffic directed to the system’s
storage processors, the switch passes input signals from the expansion
port directly to the output signal destination of the primary port.
Each LCC independently monitors the environmental status
of the entire enclosure, using a microcomputer-controlled FRU
(field-replaceable unit) monitor program. The monitor communicates
12 Hardware and Operational Overview

status to the server, which polls disk enclosure status. LCC firmware
also controls the LCC port-bypass circuits and the disk-module status
LEDs.
LCCs do not communicate with or control each other.
Captive screws on the LCC lock it into place to ensure proper
connection to the midplane. You can add or replace an LCC while the
disk enclosure is powered up.
Disk modules
Each disk module consists of one disk drive in a carrier. You can
visually distinguish between module types by their different latch
and handle mechanisms and by type, capacity, and speed labels on
each module. An enclosure can include Fibre Channel or SATA disk
modules, but not both types. You can add or remove a disk module
while the DAE3P is powered up, but you should exercise special care
when removing modules while they are in use. Drive modules are
extremely sensitive electronic components.
Disk drives
The DAE3P supports Fibre Channel and SATA disks. The Fibre
Channel (FC) disks conform to FC-AL specifications and 2 or 4 Gb/s
Fibre Channel interface standards, and supports dual-port FC–AL
interconnects through the two LCCs. A DAE3P supports 2 Gb/s drives
only if the entire back-end bus that containsthedrivesisoperatingat2Gb/s.
SATA disks conform to Serial ATA II Electrical Specification 1.0 and
include dual-port SATA interconnects; a paddle card on each drive
converts the assembly to Fibre Channel operation. The disk module
slots in the enclosure accommodate 2.54 cm (1 in) by 8.75 cm (3.5 in)
disk drives.
Table 1 lists the disks that are availableforthestoragesystem.
Hardware and Operational Overview 13

Table 1 Available disks
Available GB
Listed Capacity Disk 0-4 (vault disk) Other disks
73 GB 33.6387 GB 66.6055 GB
146 GB 100.6806 GB 133.6473 GB
300 GB 235.4042 GB 26809 GB
400 GB 333.80 GB 366.76 GB
500 GB N/A 458.5629 GB
750 GB N/A 687.8560 GB
1000 GB 884.1824 GB 917.1492 GB
Notes: 500 GB, 750 GB, and 1000 GB disk are SATA disks and they cannot be used as vault disks (disks 0-4). Usable disk space differs
from the disk drive size reported by the manufacturer. A drive manufacturer’s definition of a gigabyte is 1,000,000,000 bytes and the
storage system’s "true" definition is 1024 x 1024 x 1024, or 1,073,741,824 bytes.
Drive carrier
The disk drive carriers are metal and plastic assemblies that provide
smooth, reliable contact with the enclosure slot guides and midplane
connectors. Each carrier has a handle with a latch and spring clips.
The latch holds the disk module in place to ensure proper connection
with the midplane. Disk drive activity/fault LEDs are integrated into
the carrier.
Power/cooling modules
The power/cooling modules are located above and below the LCCs.
The units integrate independent power supply and dual-blower
cooling assemblies into a single module.
Each power supply is an auto-ranging, power-factor-corrected,
multi-output, offline converter with its own line cord. Each supply
supports a fully configured DAE3P and shares load currents with the
other supply. The drives and LCCs have individual soft-start switches
that protect the disk drives and LCCs if they are installed while the
disk enclosure is powered up. A FRU (disk, LCC, or power/cooling
module) with power-related faults does not adversely affect the
operation of any other FRU.
The enclosure cooling system includes two dual-blower modules.
If one blower fails, the others will speed up to compensate. If two
14 Hardware and Operational Overview

blowers in a system (both in one power/cooling module, or one in each
module) fail, the DAE3P goes offline within two minutes.
Hardware and Operational Overview 15

Standby power supplies (SPSs)
Two 2U 2200-watt DC SPSs provide backup power for one SPE2 and
the first (enclosure 0, bus 0) DAE3P adjacent to it. The two SPSs
providehighavai
lability and allow write caching — which prevents
data loss during a power failure — to continue. A faulted or not fully
charged SPS disables write caching. Each SPS rear panel has one AC
inlet power connector with power switch, AC outlets for the SPE2 and
the first DAE3P (EA 0, bus 0) respectively, and one phone-jack type
connector for connection to a management module. Figure 8 shows
thelocationo
f the SPS connectors.
Online
Battery on
Replace battery
SPS fault
LEDs:
AC in
from
PDU
Power
switch
RJ12 to RS-232
interface on SPE
AC out to
SPE and DAE
EA0, bus 0
SPS B SPS A
EMC3365
Figure 8 2200 W SPS connectors
Aserviceprovider can replace an SPS while the storage system is
powered up.
16 Hardware and Operational Overview

Powerup and powerdown sequence
The SPE2 and DAE3P do not have power switches.
Powering up the storage system
1. Verify the following:
❑Master switch/circuit breakers for each cabinet/rack power
strip are off.
❑The two power cords for the SPE2 power supplies are plugged
intotheSPSsandthepowercordretentionbailsareinplace.
❑Serial connections between the SPE2 management modules and
the SPSs are in place.
❑Power cords for the first DAE3P (EA 0, bus 0; often called
the DAE-OS) are plugged into the SPSs and the power cord
retentionbailsareinplace.
❑The power cords for the SPSs and any other DAE3Ps are
plugged into the cabinet’s power strips.
❑ThepowerswitchesontheSPSsareintheonposition.
❑Any other devices in the cabinet are correctly installed and
ready for powerup.
2. Turn on the master switch/circuit breakers for each cabinet/rack
power strip.
In the 40U-C cabinet, master switches are on the power distribution
panels (PDPs), as shown in Figure 9.
Hardware and Operational Overview 17

Each AC circuit in the 40U-C cabinet requires a source connection that can
support a minimum of 4800 VA of single phase, 200-240 V AC input power.
For high availability, the left and right sides of the cabinet must receive
power from separate branch feed circuits. Each pair of power distribution
panels (PDP) in the 40U-C cabinet can support a maximum of 24 A AC
current draw from devices connected to its power distribution units (PDU).
Most cabinet configurations draw less than 24 A AC power, and require
only two discrete 240 V AC power sources. If the total AC current draw
of all the devices in a single cabinet exceeds 24 A, the cabinet requires two
additional 240 V power sources to support a second pair of PDPs. Use the
published technical specifications and device rating labels to determine the
current draw of each device in your cabinet and calculate the total.
18 Hardware and Operational Overview

ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
ON
I
OFF
O
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!!
!
EXP PRI
EXPPRI
#
!
EXP PRI
EXPPRI
#
A
B
!!
!
!
!
!
Power source A
Power source CPower source D
Power source B
EMC3433
Master switch
Master switch
Master switch
Master switch
Figure9 PDP master switches and power sources in the 40U cabinet
Thestoragesystemcantake10to15minutestocompleteatypical
powerup. If the storage system was installed in a cabinet at your site
(field-installed system), the first powerup will require several reboots
and can take 30 to 45 minutes. Amber warning LEDs flash during the
power on self-test (POST) and then go off. The front fault LED and the
SPS recharge LEDs commonly stay on for several minutes while the
Hardware and Operational Overview 19

SPSsarecharging.ThepowerupiscompletewhentheCPUpower
light on each SP is steady green.
If amber LEDs on the front or back of the storage system remain
on for more than 15 minutes (45 minutes for the first powerup of a
field-installed system), make sure the storage system is correctly cabled,
and then refer to the troubleshooting flowcharts for your storage
system on the CLARiiON Tools page on the EMC Powerlink website
(http://Powerlink.EMC.com). If you cannot determine any reasons for
the fault, contact your authorized service provider.
Powering down the storage system
1. Stop all I/O activity to the SPE. If the server connected to the SPE is
running the Linux or UNIX operating system, back up critical data
and then unmount the file systems.
Stopping I/O allows the SP to destage cache data, and may take
some time. The length of time depends on criteria such as the size
of the cache, the amount of data in the cache, the type of data in
the cache, and the target location on the disks, but it is typically
less than one minute. We recommend that you wait five minutes
before proceeding.
2. After five minutes, use the power switch on each SPS to turn off
power. The SPE and primary DAE3P power down within two
minutes.
!CAUTION
Never unplug the power supplies to shut down an SPE.
Bypassing the SPS in that manner prevents the storage system
from saving write cache data to the vault drives, and results
in data loss. You will lose access to data, and the storage
processor log displays an error message similar to the following:
Enclosure 0 Disk 5 0x90a (Can’t Assign - Cache Dirty)
0 0xafb40 0x14362c
Contact your service provider if this situation occurs.
20 Hardware and Operational Overview