Hewlett Packard Enterprise Synergy 12Gb SAS Storage User manual

HPE Synergy 12Gb SAS Storage User Guide
Part Number: 789815-007
Published: October 2019
Edition: 8
Abstract
This document is for the person who installs, administers, and troubleshoots the HPE Synergy 12Gb SAS
Connection Module, HPE Synergy D3940 Storage Module, HPE Smart Array P542D RAID Controller, and HPE
Smart Array P416ie-m SR Gen10 Controller for the HPE Synergy 12000 Frame. Hewlett Packard Enterprise
assumes you are qualified in the servicing of computer equipment and trained in recognizing hazards in products
with hazardous energy levels.

©Copyright 2016, 2019 Hewlett Packard Enterprise Development LP
Notices
The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise
products and services are set forth in the express warranty statements accompanying such products and services. Nothing
herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical
or editorial errors or omissions contained herein.
Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying.
Consistent with FAR 12211 and 12212, Commercial Computer Software, Computer Software Documentation, and Technical
Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.
Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no
control over and is not responsible for information outside the Hewlett Packard Enterprise website.

Contents
Product overview.................................................................................................................... 7
Synergy SAS Storage features............................................................................................................................................................................................7
HPE Synergy 12Gb SAS Connection Module............................................................................................................................................................ 7
Features..........................................................................................................................................................................................................................7
Front panel and LED definitions...................................................................................................................................................................... 7
HPE Synergy D3940 Storage Module............................................................................................................................................................................8
Features..........................................................................................................................................................................................................................8
Front panel components.......................................................................................................................................................................................9
Front panel and LED definitions...................................................................................................................................................................... 9
Side panel................................................................................................................................................................................................................... 10
Device bay numbering........................................................................................................................................................................................ 11
SmartDrive hot-plug drive LED definitions............................................................................................................................................ 12
VRM............................................................................................................................................................................................................................... 12
HPE Smart Array controllers............................................................................................................................................................................................ 13
HPE Smart Array P542D RAID Controller.............................................................................................................................................. 13
Features..................................................................................................................................................................................................... 13
P542D controller ports and connectors................................................................................................................................. 14
HPE Smart Array P416ie-m Controller.....................................................................................................................................................15
Features..................................................................................................................................................................................................... 15
P416ie-m controller ports and connectors...........................................................................................................................15
Planning..................................................................................................................................17
Performance............................................................................................................................................................................................................................... 17
Load balancing..........................................................................................................................................................................................................................17
Wiring diagram..........................................................................................................................................................................................................................17
Synergy D3940 drive count and performance details...................................................................................................................................... 19
Synergy SAS storage solution planning guidelines.............................................................................................................................................20
Server profile..............................................................................................................................................................................................................................21
Local storage configuration options...........................................................................................................................................................21
Server profile mobility.........................................................................................................................................................................................22
Logical JBOD drive selection...........................................................................................................................................................................22
Deployment............................................................................................................................23
Installing Synergy SAS storage solution components.......................................................................................................................................23
HPE Synergy 12Gb SAS Connection Module installation guidelines......................................................................................23
Installing an HPE Synergy 12Gb SAS Connection Module..........................................................................................23
HPE Synergy D3940 Storage Module installation guidelines.....................................................................................................24
Installing an HPE Synergy D3940 Storage Module.........................................................................................................25
Installing I/O adapter......................................................................................................................................................................... 26
Installing drives.......................................................................................................................................................................................................26
HPE Smart Array controller installation guidelines...........................................................................................................................28
Installing Smart Array controller................................................................................................................................................. 28
Configuration............................................................................................................................................................................................................................. 30
Firmware bundle.....................................................................................................................................................................................................30
Creating a firmware bundle............................................................................................................................................................30
Logical interconnect groups............................................................................................................................................................................31
Creating logical interconnect groups........................................................................................................................................31
Enclosure group......................................................................................................................................................................................................34
3

Creating an enclosure group......................................................................................................................................................... 35
Logical enclosure................................................................................................................................................................................................... 37
Creating a logical enclosure........................................................................................................................................................... 37
Check alerts...............................................................................................................................................................................................................38
Dashboard................................................................................................................................................................................................ 38
Activity....................................................................................................................................................................................................... 41
Check for errors.....................................................................................................................................................................................42
Server profiles..........................................................................................................................................................................................................42
Creating server profiles.................................................................................................................................................................... 43
Delete or force delete a server profile......................................................................................................................................50
Inventory...................................................................................................................................................................................................................................... 51
Interconnect..............................................................................................................................................................................................................51
Viewing interconnects.......................................................................................................................................................................51
Drive enclosure........................................................................................................................................................................................................51
Viewing drive enclosures.................................................................................................................................................................51
Operations..............................................................................................................................53
Status event notifications...................................................................................................................................................................................................53
Receiving event notifications through HPE OneView GUI............................................................................................................53
Receiving event notifications through HPE OneView RESTapi................................................................................................. 54
Firmware.......................................................................................................................................................................................................................................55
Updating the firmware........................................................................................................................................................................................55
Modifying local storage........................................................................................................................................................................................................56
Expanding local storage.....................................................................................................................................................................................56
Manually managed.............................................................................................................................................................................. 56
HBA.............................................................................................................................................................................................................. 57
RAID............................................................................................................................................................................................................. 57
Editing server profile........................................................................................................................................................................................... 57
Reducing the number of frames in a logical enclosure without losing D3940 data.......................................................57
Removing a storage module without disturbing servers in a frame........................................................................................58
Moving the storage module to a dierent bay.....................................................................................................................................58
Replacing Synergy SAS storage solution components..................................................................................................................................... 59
Replacing HPE Synergy 12Gb SAS Connection Module.................................................................................................................59
Removing the HPE Synergy 12Gb SAS Connection Module......................................................................................59
Installing the replacement HPE Synergy 12Gb SAS Connection Module...........................................................59
Verifying proper operation.............................................................................................................................................................59
Replacing HPE Synergy D3940 Storage Module Enclosure........................................................................................................ 60
Powering down the HPE Synergy D3940 Storage Module.........................................................................................60
Removing the HPE Synergy D3940 Storage Module.....................................................................................................60
Installing the replacement HPE Synergy D3940 Storage Module..........................................................................61
Verifying proper operation.............................................................................................................................................................61
Replacing the I/O adapter.................................................................................................................................................................................61
Removing the failed I/O adapter.................................................................................................................................................61
Installing the replacement I/O adapter....................................................................................................................................62
Verifying proper operation.............................................................................................................................................................62
Electrostatic discharge.........................................................................................................63
Preventing electrostatic discharge............................................................................................................................................................................... 63
Grounding methods to prevent electrostatic discharge...................................................................................................................................63
Specifications.........................................................................................................................64
Environmental specifications............................................................................................................................................................................................64
HPE Synergy 12Gb SAS Connection Module specifications..........................................................................................................................64
4

HPE Synergy D3940 Storage Module specifications.........................................................................................................................................65
P542D specifications............................................................................................................................................................................................................ 65
P416ie-m specifications......................................................................................................................................................................................................66
Documentation and troubleshooting resources for HPE Synergy...............................67
Documentation and troubleshooting resources for HPE Synergy............................................................................................................. 67
HPE Synergy documentation..........................................................................................................................................................................67
HPE Synergy Configuration and Compatibility Guide....................................................................................................67
HPE Synergy Frame Link Module User Guide.....................................................................................................................67
HPE OneView User Guide and Help for HPE Synergy................................................................................................... 67
HPE OneView Global Dashboard User Guide and Help................................................................................................ 67
HPE Synergy Image Streamer User Guide............................................................................................................................ 67
HPE Synergy Image Streamer GitHub.....................................................................................................................................68
HPE Synergy Software Overview Guide................................................................................................................................. 68
HPE Synergy Firmware Update Overview.............................................................................................................................68
HPE OneView for Synergy Firmware and Driver Update Guide...............................................................................68
HPE OneView Support Matrix for HPE Synergy................................................................................................................68
HPE Synergy Image Streamer Support Matrix................................................................................................................... 68
HPE Synergy Glossary.......................................................................................................................................................................68
HPE Synergy troubleshooting resources.................................................................................................................................................68
Troubleshooting within HPE OneView....................................................................................................................................69
HPE Synergy Troubleshooting Guide...................................................................................................................................... 69
Error Message Guide for HPE ProLiant Gen10 servers and HPE Synergy........................................................69
HPE OneView Help for HPE Synergy and HPE OneView API Reference........................................................... 69
HPE Synergy QuickSpecs................................................................................................................................................................69
HPE Synergy document overview (documentation map)...............................................................................................................................69
Acronyms and abbreviations..............................................................................................71
Support and other resources.............................................................................................. 73
Accessing Hewlett Packard Enterprise Support................................................................................................................................................... 73
Accessing updates..................................................................................................................................................................................................................73
Customer self repair...............................................................................................................................................................................................................74
Remote support........................................................................................................................................................................................................................74
Warranty information............................................................................................................................................................................................................74
Regulatory information........................................................................................................................................................................................................74
Documentation feedback....................................................................................................................................................................................................75
5

Acknowledgments
Microsoft®and Windows®are either registered trademarks or trademarks of Microsoft Corporation in the United States
and/or other countries.
Java™is a US trademark of Sun Microsystems, Inc.
UNIX®is a registered trademark of The Open Group.
6

Product overview
Synergy SAS Storage features
Synergy SAS Storage supports the following types of devices that are designed for use in an HPE Synergy 12000 Frame:
•HPE Synergy 12Gb SAS Connection Module
•HPE Synergy D3940 Storage Module
•HPE Smart Array P542D RAID Controller
•HPE Smart Array P416ie-m RAID Controller
HPE Synergy D3940 Storage Module is a 40 SFF drive bay module, and the HPE Synergy 12Gb SAS Connection Module is a
connection module that allows direct attached storage for up to 10 compute modules per frame.
Synergy SAS storage is optimized for use as either a direct attached storage array or as software-defined storage using HPE
StoreVirtual VSA or similar solutions.
Storage modules connect to compute modules within a frame through the HPE Synergy 12Gb SAS Connection Module. Any
number of drive bays in a storage module can be composed with any compute module for eicient use of available drives.
Synergy SAS storage can scale up to 200 SFF drives with five storage modules in a single HPE Synergy 12000 Frame. Adding
a second I/O adapter and second SAS connection module provides a redundant path to dual-ported SAS drives inside the
storage module, ensuring high availability.
HPE Synergy 12Gb SAS Connection Module
The following components are located in the HPE Synergy 12Gb SAS Connection Module.
Features
• Hot Pluggable connection module for HPE Synergy 12000 Frame
• Dual Domain direct attached SAS architecture supported with HPE Synergy D3940 Storage Modules containing two I/O
adapters
• 12 internal blade bay ports each containing 4x 12Gb/s SAS links
• Managed with HPE Synergy Composer and HPE OneView version 30 or higher
Front panel and LED definitions
Product overview 7

Item LED Status Definition
1 UID O Normal
Solid blue Being identified
Flashing blue Firmware is being updated
2 Health O Not powered up
Green Healthy
Flashing amber Error, there is a problem with
the module
HPE Synergy D3940 Storage Module
The following components are located in the HPE Synergy D3940 Storage Module.
Features
• The HPE Synergy D3940 Storage Module holds up to 40 hot plug SFF drives in a half-height double-width storage module
designed for HPE Synergy 12000 Frames.
• More HPE Synergy D3940 Storage Modules can be added into an existing frame without powering down servers that are
already in operation.
• Support for 12G and 6G SAS and SATA HDDs and SSDs.
• Simple configuration and setup with HPE Synergy Composer.
• Easy to maintain and troubleshoot with industry standard management tools including HPE System Insight Manager.
• Optimized for use with HPE StoreVirtual VSA software to create a robust shared storage environment inside and across
Synergy Frames.
• HPE Synergy D3940 Storage Module supports mixing of dierent drive types (SAS/SATA, SSD/HDD) in a single enclosure.
IMPORTANT: SATA and certain SAS drives are single-ported. Therefore, there is no redundancy between the
compute and storage modules when using a redundant pair of HPE Synergy 12Gb SAS Connection Modules. Most
SAS drives are dual-ported. To achieve optimal availability, Hewlett Packard Enterprise recommends using dual-
ported SAS drives with the HPE Synergy D3940 Storage Module.
NOTE: A composed logical drive or logical JBOD can be created using either a SAS SSD, SAS HDD, SATA SSD, or SATA
HDD drive type.
• The modular design of the HPE Synergy D3940 Storage Module allows the drive drawer to slide out to service drives or I/O
adapters without interrupting the operation of other drives in the module.
8Product overview

Front panel components
Item Description
1 Storage module handle
2 Storage module handle release latch
3 Drive drawer handle release latch
4 Drive drawer handle
5 Product information tag
Front panel and LED definitions
The front panel and LED layout shown in the following illustration is an example. Depending on your specific model, the LEDs
might be dierent.
Product overview 9

Item LED Status Definition
1 Drawer status1Solid green Drive drawer is closed
Flashing green Drive drawer is open
Flashing amber Thermal warning, close the
drive drawer
2 UID Solid blue One or more drives are being
located
Flashing blue Firmware flash in progress
O No drives are being located
3 Health status Green Normal operation
Flashing amber Degraded condition
Flashing red Critical condition
4 Drive status (in priority order) Solid amber One or more drives failed
Flashing amber One or more drives are
predicted to fail
Flashing green One or more drives are
rebuilding or transforming
Solid green One or more drives are
members of a logical drive
O No drives are configured by a
RAID controller
5 Drawer power Green Storage module is in use, do
not remove
O Storage module is not in use,
safe to remove
1 The drive drawer must be closed within five minutes after it is opened.
Side panel
The side panel layout shown in the following illustration is an example. Your specific model might be dierent.
10 Product overview

Item Description
1 I/O adapter health LED
2 I/O adapter bay 1
3 I/O adapter bay 2
Device bay numbering
• 40 hot-pluggable small form factor drive bays
◦For proper air flow, drives must be populated from back to front. Based on the following drive numbering, begin
populating bays 33 through 40, and continue to populate back to front, finishing with bays 1 through 8.
◦Drive blanks are not required or provided.
• 2 hot-pluggable I/O adapter bays
◦The first I/O adapter must be installed in I/O adapter bay 1.
◦I/O adapter blanks are not required or provided.
◦I/O adapter health status is solid green or solid amber.
Product overview 11

SmartDrive hot-plug drive LED definitions
Item Description Status
1 Locate • Solid blue = The drive is being identified by a host
application.
• Flashing blue = The drive carrier firmware is being updated
or requires an update.
2 Activity ring LED • Rotating green = Drive activity.
•O = No drive activity.
3 Do not remove LED • Solid white = Do not remove the drive. Removing the drive
causes one or more of the logical drives to fail.
•O = Removing the drive does not cause a logical drive to
fail.
4 Drive status LED • Solid green = The drive is a member of one or more logical
drives.
• Flashing green = The drive is rebuilding or performing a
RAID migration, strip size migration, capacity expansion, or
logical drive extension, or is erasing.
• Flashing amber/green = The drive is a member of one or
more logical drives and predicts the drive will fail.
• Flashing amber = The drive is not configured and predicts
the drive will fail.
• Solid amber = The drive has failed.
•O = The drive is not configured by a RAID controller.
VRM
The VRM provides power to the HPE Synergy D3940 Storage Module drive drawer by converting the 12 V input power to the
local power required by the drives and I/O adapters.
12 Product overview

HPE Smart Array controllers
The HPE Smart Array controllers allow connection to SAS or SATA drives in the HPE Synergy D3940.
IMPORTANT: SATA and certain SAS drives are single-ported. Hewlett Packard Enterprise does not recommend SATA
or single-ported SAS drives with software-defined storage such as VMware vSAN or HPE StoreVirtual VSA. In single-
domain, a hardware fault could result in a cluster failure.
Depending on your configuration, use the following controllers:
• Gen9 compute modules—HPE Smart Array P542D RAID Controller
• Gen10 compute modules—HPE Smart Array P416ie-m SR Gen10 Controller
HPE Smart Array P542D RAID Controller
The following components are located in the HPE Smart Array P542D RAID Controller.
Features
• Storage interface (SAS/SATA)
◦1612Gb/s SAS physical links distributed equally across 2x4 internal ports and 2x4 external ports.
◦12Gb/s SAS technology delivers up to 1200 MB/s theoretical bandwidth per physical link.
◦6Gb/s SATA technology delivers up to 600 MB/s theoretical bandwidth per physical link for directly attached SATA
drives.
• Host interface uses an 8x PCIe Gen3
• Mix-and-match SAS and SATA drives to the same controller. Deploy drive technology to fit the computing environment.
Product overview 13

IMPORTANT: SATA and certain SAS drives are single-ported. Therefore, there is no redundancy between the
compute and storage modules when using a redundant pair of HPE Synergy 12Gb SAS Connection Modules. Most
SAS drives are dual-ported. To achieve optimal availability, Hewlett Packard Enterprise recommends using dual-
ported SAS drives with the HPE Synergy D3940 Storage Module.
• RAID 0, 1, 10, 5, 50, 6, 60, 1 ADM, 10 ADM
• Recovery ROM protects against ROM corruption.
• Capable of supporting legacy and UEFI boot operation. RAID or HBA mode
• HPE SSD Smart Path
• Optional HPE Smart Array SR SmartCache
• Optional HPE Secure Encryption
• PCIe Gen3 x8 link
• If there is an unexpected power outage, 2GB FBWC provides read ahead caching and write back caching with indefinite
write cache data retention.
• The following features require the controller to be manually managed
◦RAID 50 and 60
◦HPE Smart Array SR SmartCache (requires license).
◦FBWC (other than default setting)
◦Stripe Size (other than default setting)
◦Spare drive
◦Spare activation modes
◦Multiple logical drives in a single array
P542D controller ports and connectors
14 Product overview

Item Description
1 SAS connector, port 4i
2 SAS connector, port 3i
3 Fabric connector
4 PCIe connector
HPE Smart Array P416ie-m Controller
The following components are located in the HPE Smart Array P416ie-m Controller.
Features
• Host interface uses an 8x PCIe Gen3
• Up to 16 SAS/SATA lanes for internal or external drives
• RAID levels 0, 1, 5, 6, 10, 50, 60, 1 ADM, and 10 ADM
• Mixed mode RAID and HBA pass-through functionality simultaneously
• Data-at-rest encryption
• 12G SAS support
• UEFI and Legacy Boot modes
• Best RAID performance capabilities with large flash-backed write cache
• Smart Array management tools
For more information on the HPE Smart Array P416ie-m Controller security features, see the controller HPE Smart Array SR
Gen10 User Guide on the Hewlett Packard Enterprise website.
P416ie-m controller ports and connectors
Product overview 15

Item Description
1 Internal x4 Slim SAS port 4i
2 Fabric connector
3 Internal x4 Slim SAS port 3i
16 Product overview

Planning
Performance
Measuring I/O performance is done from the host operating system. Some tools that measure I/O performance include the
following:
• Windows—PerfMon
• Linux—sar command from systat.
• VMware—vSphere Client disk
Load balancing
When using a dual domain SAS topology, Smart Array has two paths that it could use to access each SAS drive. During power-
up, Smart Array selects an optimal path for each drive during power-up, and then the Smart Array distributes half of the drives
to each path. For example, odd bays use path one and even bays use path two.
Dual domain path selection optimizes performance by allowing the controller to use the additional bandwidth available in the
secondary drive path.
Wiring diagram
Before planning a frame level layout for your storage and compute modules, it is helpful to understand some basic principles of
how the frame is designed and how the connections are made between compute and storage modules. The HPE Synergy
D3940 Storage Module may be connected to any HPE Smart Array Controller-equipped compute module that is installed in
the same frame with the following configuration options:
• Redundant pairs of HPE Synergy 12Gb SAS Connection Module
• Nonredundant single HPE Synergy 12Gb SAS Connection Module
As a best practice, Hewlett Packard Enterprise recommends designing your storage fabric for full redundancy; however,
nonredundant solutions are supported with the correct configuration. In the following figure, you will see how the fabric allows
for connections to single or dual I/O adapters in each storage module, as well as routing required in nonredundant connections.
When configured for redundancy two connection modules are configured in the frame residing in interconnect module bays 1
and 4, and each storage module contains a pair of I/O adapters. This redundancy provides dual paths for each HPE Smart
Array controller to be able to connect with any storage module in the frame regardless of where in the frame the storage
module is configured. However, when there are configurations where there is either only a single connection module present or
the connection modules and storage modules are configured nonredundant, make sure that the storage modules are aligned in
the correct module bays to be serviced by the correct connection modules.
A SAS Connection module in interconnect module bay 1 will provide connection from the primary I/O adapter to any storage
module in device bays 1-6. For device bays 7-12, interconnect module bay 1 is routed to the secondary (redundant) I/O
adapter. In a nonredundant storage fabric configuration, the secondary connection would not be active. Likewise, a SAS
connection module interconnect module bay 4 will oer to connect to the primary I/O adapter of a storage module any place in
device bays 7-12, but will not access the primary I/O adapter in Bays 1-6. If redundancy is not configured if the SAS fabric, it is
required that connection modules in interconnect module 1 will only support storage modules in device bays 1-6 and
connection modules in interconnect module 4 will only support storage modules in device bays 7-12, but will not access the
primary I/O adapter in Bays 1-6. If redundancy is not configured if the SAS fabric, it is required that connection modules in
interconnect module 1 will only support storage modules in device bays 1-6 and connection modules in interconnect module 4
will only support storage modules in device bays 7-12.
This routing does not impede compute module access to a storage module regardless of where the storage module is located.
For example, a compute module in device bay 1 will connect to a storage module in device bay 11, even in a nonredundant
Planning 17

configuration, as long as the connection module is correctly placed in interconnect module 4 to align with the location of the
storage module.
Front view HPE Synergy 12000 Frame device bays
18 Planning

Back view HPE Synergy 12000 Frame interconnect module bays
Synergy D3940 drive count and performance details
The drive count and workload details in the following illustration are examples. The drive ratios listed are not fixed ratios and
are only examples of an even distribution of drives across available compute modules.
Planning 19

Item Total D3940 drive bays Average Drive Count1Maximum Performance per
Compute Module2
1 40 SFF drives 4 SFF 200K IOPS
2 80 SFF drives 10 SFF 500K IOPS
3 120 SFF drives 20 SFF 1M IOPS
4 160 SFF drives 40 SFF 1M IOPS
1 Based on full Synergy Frame.
2 Assumes that 12G SAS SSD drives are running 4 KB random read workload using a redundant configuration.
Synergy SAS storage solution planning guidelines
When planning the installation of components in a frame, the following guidelines must be observed.
• Select up to three HPE Synergy D3940 Storage Modules per frame when installing a single HPE Synergy 12Gb SAS
Connection Module.
• Select up to five HPE Synergy D3940 Storage Modules per frame when installing dual HPE Synergy 12Gb SAS Connection
Modules.
• One SAS I/O adapter is included with every D3940 Storage Module.
• For redundancy and higher performance when using dual-ported SAS drives, a second I/O adapter and connection module
must be added.
• Any compute module connecting to a HPE Synergy D3940 Storage Module must have an HPE Smart Array Controller
populated in fabric 1, which is mezzanine slot 1 or 4.
NOTE: HPE Synergy graphics Mezzanine Card and HPE Synergy 480 Gen10 PCIe Expansion Module are not supported
with Synergy SAS Storage because they must occupy Synergy 480 Gen 10 mezzanine slot 1. For GPU support, a Synergy
480 Gen10 Multi-MXM Expansion Module can be used with the Smart Array controller in mezzanine 4.
• If the HPE Synergy D3940 Storage Module is configured with two SAS I/O adapters, a second SAS connection module is
required.
20 Planning