NetApp E5600 User manual

E-Series Systems
E5600 Controller-Drive Tray and
Related Drive Trays Installation
Guide
December 2015 | 215-10564_A0
docc[email protected]om

Contents
Deciding whether to use this guide ............................................................. 6
Step 1 - Preparing to install a controller-drive tray .................................. 7
Key terms .................................................................................................................... 8
Storage array ................................................................................................... 8
Controller-drive tray ........................................................................................ 8
Controller ........................................................................................................ 8
Drive tray ......................................................................................................... 8
Environmental services module (ESM) .......................................................... 8
Gathering items ........................................................................................................... 8
Basic hardware ................................................................................................ 9
Configuration cables and connectors ............................................................ 10
Tools and other items .................................................................................... 12
Things to know - More details about cables, SFPs, and QSFPs ............................... 13
Things to know - Taking a quick glance at the hardware in a controller-drive
tray configuration ................................................................................................ 16
Step 2 - Installing and configuring the switches ...................................... 24
Things to know - Switches and routers ..................................................................... 24
Procedure - Installing and configuring switches ....................................................... 26
Step 3 - Installing the host bus adapters for the controller-drive tray
.................................................................................................................. 27
Key terms .................................................................................................................. 27
Host bus adapter (HBA) ................................................................................ 27
Host channel adapter (HCA) ......................................................................... 27
HBA host port ............................................................................................... 27
HBA host port world wide name (WWN) ..................................................... 27
HCA host port ............................................................................................... 27
HCA host port Global Unique Identifier (GUID) ......................................... 27
Network interface card (NIC) ........................................................................ 27
Things to know - Host bus adapters, host channel adapters, and Ethernet
network interface cards ........................................................................................ 27
Procedure - Installing host bus adapters .................................................................... 28
Procedure - Installing host channel adapters ............................................................. 29
Step 4 - Installing the controller-drive tray .............................................. 30
Things to know - General installation ....................................................................... 30
Procedure - Installing the E5612 or E5624 controller-drive tray .............................. 30
Procedure - Installing drives in the E5612 or E5624 controller-drive tray ............... 37
Procedure - Installing the E5660 controller-drive tray .............................................. 39
Procedure - Installing drives in the E5660 controller-drive tray ............................... 47
Procedure - Removing drives from a 60-drive tray ................................................... 49
Step 5 - Connecting the controller-drive tray to the hosts ...................... 51
Key terms .................................................................................................................. 51
Table of Contents | 3

Topology ........................................................................................................ 51
Direct topology .............................................................................................. 51
Switch or fabric topology .............................................................................. 51
Things to know - Storage array configuration specifications .................................... 51
Things to know - Host channels ................................................................................ 52
Procedure - Connecting host cables on a controller-drive tray ................................. 54
Step 6 - Installing the drive trays for the controller-drive tray
configurations ......................................................................................... 60
Things to know - General installation of drive trays ................................................. 60
Procedure - Installing the DE1600 and DE5600 drive trays ..................................... 60
Procedure - Installing drives in the DE1600 and DE5600 drive trays ...................... 67
Procedure - Installing the DE6600 drive tray ............................................................ 68
Procedure - Installing the drives in the DE6600 drive tray ....................................... 77
Things to know - Connecting the power cords .......................................................... 79
Procedure - Connecting the power cords .................................................................. 79
Step 7 - Connecting the controller-drive tray to the drive trays ............ 80
Key terms .................................................................................................................. 80
Drive channel ................................................................................................ 80
Things to know - Controller-drive tray ...................................................................... 80
Things to know - Drive trays ..................................................................................... 81
Things to know - Cabling configurations for the E5612 and E5624 controller-
drive trays ............................................................................................................ 83
Things to know - Cabling configurations for the E5660 controller-drive tray .......... 85
Procedure - Connecting the drive trays to the controller-drive tray .......................... 90
Step 8 - Connecting the Ethernet cables ................................................... 92
Key terms .................................................................................................................. 92
In-band management ..................................................................................... 92
Out-of-band management .............................................................................. 92
Things to know - Connecting Ethernet cables ........................................................... 92
Procedure - Connecting Ethernet cables ................................................................... 93
Step 9 - Connecting the power cords ........................................................ 94
Things to know - AC power cords ............................................................................. 94
Procedure - Connecting AC power cords .................................................................. 94
Step 10 - Turning on the power and checking for problems in a
controller-drive tray configuration ...................................................... 96
Procedure - Turning on the power to the storage array and checking for
problems in a controller-drive tray configuration ................................................ 96
Things to know - LEDs on the E5612 or E5624 controller-drive tray ...................... 96
Things to know - LEDs on the E5660 controller-drive tray .................................... 104
Things to know - General behavior of the LEDs on the controller-drive tray ........ 107
Things to know - Service Action Allowed LEDs .................................................... 110
Things to know - LEDs on the DE1600 and DE5600 drive trays ........................... 110
Things to know - LEDs on the DE6600 drive tray .................................................. 115
Things to know - LEDs on the DE6600 drive drawers ........................................... 119
Things to know - LEDs on the DE6600 drives ....................................................... 120
4| E5600 Controller-Drive Tray and Related Drive Trays Installation Guide

Things to know - General behavior of the LEDs on the drive trays ........................ 121
Things to know - Display sequence code definitions .............................................. 123
Things to know - Component failure identifications ............................................... 124
Things to know - Lock-down codes ........................................................................ 125
Things to know - Seven-segment display use cases ................................................ 127
Things to know - Seven-segment display for the ESMs on the drive trays ............. 129
Regulatory compliance statements ......................................................... 131
Copyright information ............................................................................. 134
Trademark information ........................................................................... 135
How to send comments about documentation and receive update
notifications .......................................................................................... 136
Table of Contents | 5

Deciding whether to use this guide
This guide is intended for hardware installers and system administrators who are installing an E5600
controller-drive tray and the related drive trays.
This guide does not provide extensive conceptual background or information about how to install and
configure the SANtricity®Storage Manager software. Refer to the appropriate SANtricity
documentation to learn how to install and configure the storage management software.
This guide is based on the following assumptions:
• You have consulted the Site Preparation Guide to plan the installation of this hardware in a
cabinet.
• You have all the necessary hardware required to install the controller-drive tray and drive trays for
the storage array.
Where to find the latest information about the product
You can find links to documentation and information about the latest version of the product at the
NetApp E-Series and EF-Series Systems Documentation Center .
6

Step 1 - Preparing to install a controller-drive tray
Use this document to install one of the following controller-drive tray models and all necessary drive
trays for your configuration:
• E5612 controller-drive tray
• E5624 controller-drive tray
• E5660 controller-drive tray
The following tables show the configuration options.
Table 1: E5612 controller-drive tray and E5624 controller-drive tray options
Configuration Options
Duplex E5612
controller-drive tray
with a host interface
card or duplex E5624
controller-drive tray
with a host interface
card
Any combination of controller-drive trays attached to DE1600 drive
trays, DE5600 drive trays, or DE6600 drive trays not to exceed a
maximum of 384 drive slots or 16 total trays in the storage array.
One of the following host interface cards (per controller):
• Two 56-Gb/s IB ports
• Four 12-Gb/s SFF-8644 mini SAS-HD ports
• Four 16-Gb/s Fibre Channel ports
• Four 10-Gb/s iSCSI ports
Three 4-GB DIMM memory for 12-GB capacity per controller.
Table 2: E5660 controller-drive tray options
Configuration Options
Duplex E5660
controller-drive tray
with a host interface
card
The following configurations are supported:
• A single E5660 controller-drive tray attached to no more than five
DE6600 drive trays, for a maximum of 360 drives in the storage
array.
• A single E5660 controller-drive tray attached to DE1600 drive
trays, DE5600 drive trays, or DE6600 drive trays not to exceed a
maximum of 384 drive slots or 16 total trays in the storage array.
One of the following host interface cards (per controller):
• Two 56-Gb/s IB ports
• Four 12-Gb/s SFF-8644 mini SAS-HD ports
• Four 16-Gb/s Fibre Channel ports
• Four 10-Gb/s iSCSI ports
Three 4-GB DIMM memory for 12-GB capacity per controller.
7

Attention: Possible hardware damage – To prevent electrostatic discharge damage to the tray,
use proper antistatic protection when handling tray components.
Attention: Possible hardware damage – To prevent potential damage to the tray and enhance
serviceability, install any 60-drive trays towards the bottom of the cabinet.
Key terms
Storage array
A collection of both physical components and logical components for storing data. Physical
components include solid state drives (SSDs), controllers, ESMs, fans, and power supplies. Logical
components include disk pools, volume groups, and volumes. These components are managed by the
storage management software.
Controller-drive tray
One tray with drives, one or two controllers, fans, and power supplies. The controller-drive tray
provides the interface between a host and a storage array.
Note: The E5600 controller-drive tray does not have a simplex option.
Controller
A circuit board and firmware that is located within a controller-drive tray. A controller manages the
input/output (I/O) between the host system and data volumes.
Drive tray
One tray with drives, two environmental services modules (ESMs), fans, and power supplies. A drive
tray does not contain controllers.
Environmental services module (ESM)
A canister in the drive tray that monitors the status of the components. An ESM also serves as the
connection point to transfer data between the drive tray and the controller.
Gathering items
Before you start installing the controller-drive tray, you must have installed the cabinet in which it
will be mounted.
Use the tables in this section to verify that you have all of the necessary items to install the controller-
drive tray.
Attention: Possible hardware damage – To prevent electrostatic discharge damage to the tray,
use proper antistatic protection when handling tray components.
8| E5600 Controller-Drive Tray and Related Drive Trays Installation Guide

Basic hardware
Table 3: Basic hardware
Item
Cabinet
• Ensure that your cabinet meets the installation site
specifications of the various storage array components.
Refer to the Site Preparation Guide for more information.
• Depending on the power supply limitations of your cabinet,
you might need to install more than one cabinet to
accommodate the different components of the storage array.
Refer to the installation guide for your cabinet for
instructions on installing the cabinet.
DE1600 drive tray with end caps that are packaged separately.
This drive tray ships with the drives installed and can be used
with all variations of the controller-drive tray.
DE5600 drive tray with end caps that are packaged separately.
This drive tray ships with the drives installed and can be used
with all variations of the controller-drive tray.
DE6600 drive tray (shown with the separately packaged
mounting rails attached). This drive tray can be used with all
variations of the controller-drive tray. The DE6600 drive tray
ships empty so you must install the drives when you install the
drive tray into the cabinet.
Step 1 - Preparing to install a controller-drive tray | 9

Item
Mounting rails and screws
The mounting rails that are available with the drive tray are
designed for an industry-standard cabinet.
InfiniBand switch (optional)
Gigabit Ethernet switch for Management (optional)
• Host with InfiniBand host channel adapters (HBAs)
(optional)
• Host with SAS HBAs (optional)
• Host with Fibre Channel HBAs (optional)
• Host with iSCSI HBAs (optional)
Configuration cables and connectors
Table 4: Configuration cables and connectors
Item
AC power cords.
The controller-drive tray and the drive trays ship with power cords for
connecting to an external power source, such as a wall plug. Your cabinet
might have special power cords that you use instead of the power cords that
ship with the controller-drive tray and the drive trays.
AC power to wall plug cables do not exist for the DE6600 drive trays as a
result of the C20 connector on the power supply.
Ethernet cable
For information about out-of-band storage array management, see the
description for “Deciding on the management method” in the SANtricity
Storage Manager Software Installation Reference .
10 | E5600 Controller-Drive Tray and Related Drive Trays Installation Guide

Item
SAS host cables for the 6-Gb/s SAS HIC.
1m-to-5m passive cables
5m-to-100m optical SAS cables
15m and 20m active cables
Host mini-SAS-HD to mini-SAS-HD cables for the 12-Gb/s SAS HIC.
1m-to-5m passive cables
5m-to-100m optical SAS cables
15m and 20m active cables
Host mini-SAS-HD to mini-SAS cables for the 12-Gb/s SAS HIC.
1m-to-5m passive cables
5m-to-100m optical SAS cables
15m and 20m active cables
1m-to-2m fan-out single x4 port to 4-single port passive cables
SAS drive tray cables
1m-to-5m passive cables for Mini SAS 8088 ports.
Both controllers must be individually connected to the drive tray.
Step 1 - Preparing to install a controller-drive tray | 11

Item
InfiniBand cables
Use these cables for InfiniBand connections to hosts with the host interface
card.
1m-to-3m passive cables used to connect the 56-Gb InfiniBand HIC to the
host.
5m-to-100m optical cables used to connect the 56-Gb InfiniBand HIC to
the host.
Optical Fibre Channel with SFP transceivers
SFP transceivers are only included with the controller-drive tray when a
Fibre Channel HIC is ordered.
2, 3, 5, 10 or 25m OM2 cables
50-to-100m OM3 cables
300m OS1 or OS2 cables
iSCSI with SFP+ transceivers
2m-to-7m twin-ax passive copper cables
Cat5 2m-to-7m RJ-45 passive cables
2m-to-7m RJ-45 passive cables
2, 3, 5, 10 or 25m OM2 cables
25-to-300m OM3 cables
DB9-to-PS2 adapter cable
This cable adapts the DB9 connector on commercially available serial
cables to the PS2 connector on the ESM for drive trays in the storage array.
This cable is used for support only. You do not need to connect it during
installation.
DB9-to-Mini-USB adapter cable
This 2-m cable adapts the DB9 connector on commercially available serial
cables to the mini-USB port on the controller.
This cable is used for support only. You do not need to connect it during
initial installation.
Tools and other items
Table 5: Tools and other items
Item
Labels
Help you to identify cable connections and lets you more easily
trace cables from one tray to another
A cart
Holds the tray and components
A mechanical lift (optional)
12 | E5600 Controller-Drive Tray and Related Drive Trays Installation Guide

Item
A Phillips screwdriver
A flat-blade screwdriver
Anti-static protection
A flashlight
Things to know - More details about cables, SFPs, and
QSFPs
The controller-drive tray supports SAS drive connections and SAS, InfiniBand, Fibre Channel, or
iSCSI host connections.
•SAS host cable connection
Step 1 - Preparing to install a controller-drive tray | 13

1 SAS cable
2 SFF-8088 connector
•SAS host mini-SAS-HD to mini-SAS-HD cable connection
•SAS host mini-SAS-HD to mini-SAS cable connection
•Mini-SAS to mini-SAS cable connection
•InfiniBand QSFP host connection with copper cables
•InfiniBand QSFP transceiver for use with optical host cables
14 | E5600 Controller-Drive Tray and Related Drive Trays Installation Guide

•SFP transceiver and drive tray fiber optic host cable for controller-drive tray connections
1 SFP transceiver
2 Fiber optic cable
•iSCSI host cable with an RJ-45 connector
1SFP transceiver
2 Fiber optic cable
•1m-to-2m fan-out single x4 port to 4-single port passive host cables
The table references the SFP transceivers that apply if you are using Fibre Channel host connections.
Step 1 - Preparing to install a controller-drive tray | 15

Table 6: Supported SFP transceivers*
Vendor Model Version/Data Rate*
Avago AFBR-57D9AMZ 8 Gb/s
AFBR-57D9AMZ-EP1 8 Gb/s
AFBR-703SDZ 10 Gb/s
AFBR-709SMZ SFP+ 10 GbE/s for iSCSI
AFBR-709SMZ-EP1 SFP+ 10 GbE/s for iSCSI
AVAGO AFBR-57F5MZ-
EP1 16 Gb/s for FC
AVAGO AFBR-57F5MZ-
NA2 16 Gb/s for FC
Finisar FTLF 8521-3 1 Gb/s
FTLF8528P2BCV-LS 8 Gb/s
FTLF8528P3BCV-LS 8 Gb/s
FTLX8571D3BCL-LS 10 Gb/s
FTLF142P3BNV 16 Gb/s for FC
FTLF8529P3BCV 16 Gb/s for FC
FTLF8529P3BCVA-LS 16 Gb/s for FC
JDSU PLRXPL-VC-SH4-23-N 8 Gb/s
PLRXPL-VC-SH4-NA 8 Gb/s
PLRXPL-SC-S43-22-N 10 Gb/s
PLRXPL-SC-S43-NAe SFP+ 10 GbE/s for iSCSI
UNIPHASE PLRXPL-
SC-S43-22N SFP+ 10 GbE/s for iSCSI
Molex 74741-0005 1 Gb/s
Opnext TRS2001EN-0001 10 Gb/s
*All SFP transceivers can connect and run at lower speeds, even though this is not optimal.
Things to know - Taking a quick glance at the hardware in a
controller-drive tray configuration
Each 12-drive or 24-drive tray in a storage array must have a minimum of two drives for proper
operation, while each 60-drive tray must have a minimum of 20 drives (4 drives in the front row of
each drawer) for proper operation.
All 60-drive trays have a limit of five solid state drives (SSDs) per drawer. If you exceed this limit for
SSDs in a particular drawer, the power source in the drawer could fail.
This section provides an overview of the hardware described in this document. For specific details
about how LEDs operate, see Step 10 - Turning on the power and checking for problems in a
controller-drive tray configuration on page 96.
• The top of the controller-drive tray is the side with labels.
16 | E5600 Controller-Drive Tray and Related Drive Trays Installation Guide

• The configuration of the host ports might appear different on your system depending on which
host interface card configuration is installed.
Refer to the Site Preparation Guide for information about the installation requirements of these
components.
Controller-drive trays
•E5612 controller-drive tray – Front view
1End Cap Locate LED
2 End Cap Service Action Required LED
3 End Cap Over-Temperature LED
4 End Cap Power LED
5 End Cap Standby Power LED
6 Drive canister
•E5624 controller-drive tray – Front view
1End Cap Standby Power LED
2 End Cap Power LED
3 End Cap Over-Temperature LED
4 End Cap Service Action Required LED
5 End Cap Locate LED
6 Drive canister
•E5660 controller-drive tray – Front view with bezel removed
Step 1 - Preparing to install a controller-drive tray | 17

1End Cap Power LED
2 End Cap Standby Power LED
3 End Cap Locate LED
4 End Cap Over-Temperature LED
5 End Cap Service Action Required LED
•Controller with a SAS HIC – Rear view
1USB connector 11 SAS Expansion Link 2 Fault LED
2 1GbE Management connector 1 12 SAS Expansion Link 2 Up LED
3 1GbE Link 1 Fault LED 13 SFF-8088 SAS connector 2 (Expansion)
4 1GbE Link 1 Up LED 14 BBU Fault
5 1GbE Link 2 Fault LED 15 BBU Charging
18 | E5600 Controller-Drive Tray and Related Drive Trays Installation Guide

6 1GbE Link 2 Up LED 16 SAA LED
7 1GbE Management connector 2 17 Fault LED
8 SAS Expansion Link 1 Fault LED 18 Cache Active
9 SAS Expansion Link 1 Up LED 19 Seven-segment display
10 SFF-8088 SAS connector 1 (Expansion) 20 Mini USB port
•Right-rear subplate with a 12-Gb SAS-3 HIC
1Host Interface Card Link 1 Fault LED 8 Host Interface Card Link 3 Active
LED
2 Host Interface Card Link 1 Active LED 9 Mini SAS-HD Host Interface Card
Channel 3
3 Mini SAS-HD Host Interface Card Channel 1 10 Mini USB port
4 Mini SAS-HD Host Interface Card Channel 2 11 Mini SAS-HD Host Interface Card
Channel 4
5 Host Interface Card Link 2 Fault LED 12 Host Interface Card Link 4 Fault LED
6 Host Interface Card Link 2 Active LED 13 Host Interface Card Link 4 Active
LED
7 Host Interface Card Link 3 Fault LED
•Right-rear subplate with a 56-Gb Infiniband HIC
Step 1 - Preparing to install a controller-drive tray | 19

1Host Interface Card QSFP Channel 1
2 Host Interface Card Logical Link 1 Active LED
3 Host Interface Card Physical Link 1 Active LED
4 Mini USB port
5 Host Interface Card Logical Link 2 Active LED
6 Host Interface Card Physical Link 2 Active LED
7 Host Interface Card QSFP Channel 2
•Right-rear subplate with a 16-Gb Fibre Channel or a 10-Gb iSCSI HIC
1SFP+ Host Interface Card Channel 1 8 Host Interface Card Link 3 Fault LED
2 Host Interface Card Link 1 Fault LED 9 Host Interface Card Link 3 Active LED
3 Host Interface Card Link 1 Active LED 10 Mini USB port
4 SFP+ Host Interface Card Channel 2 11 Host Interface Card Link 4 Fault LED
5 Host Interface Card Link 2 Fault LED 12 Host Interface Card Link 4 Active LED
20 | E5600 Controller-Drive Tray and Related Drive Trays Installation Guide