Dell EMC ScaleIO Ready Node 14G User manual

EMC ScaleIO Ready Node 14G
Hardware Configuration and Operating System
Installation Guide - ESXi Servers
P/N 302-004-339 REV 02

Copyright ©2017 Dell Inc. or its subsidiaries All rights reserved.
Published December 2017
Dell believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS-IS.“ DELL MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND
WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED
IN THIS PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE.
Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners.
Published in the USA.
EMC Corporation
Hopkinton, Massachusetts 01748-9103
1-508-435-1000 In North America 1-866-464-7381
www.EMC.com
2Hardware Configuration and Operating System Installation Guide - ESXi Servers

5
7
Preface 9
Introduction 11
Overview.....................................................................................................12
ScaleIO component requirements............................................................... 12
ScaleIO cluster components...........................................................12
Physical server requirements......................................................... 13
Supported operating systems.........................................................13
ScaleIO Gateway server requirements........................................... 13
Prerequisites............................................................................................... 14
Connectivity...................................................................................14
Java............................................................................................... 15
Hardware and operating systems................................................... 15
ScaleIO packages........................................................................... 15
Supported Internet browsers......................................................... 15
Disk prerequisites...........................................................................15
iDRAC Service Module................................................................... 15
Supported hardware configurations............................................................ 16
Configuring the Hardware 17
Set up the BMC (iDRAC) IP address and BIOS........................................... 18
Verify the status of the system hardware, storage controller, and disks.....20
Open the KVM console................................................................................21
Updating the BIOS, firmware and settings..................................................22
Update the hardware using the iDRAC virtual console................... 23
Installing the Operating System 25
System requirements................................................................................. 26
Driver support............................................................................................ 26
Rebuild the M.2 cards as a RAID device......................................................26
Revert the AMS ESXi ISO Image to a native ESXi Image............................ 27
Map the ISO file on a ScaleIO Ready Node server...................................... 30
Prepare the servers for deployment........................................................... 30
Network Architecture and Physical Connectivity 35
Networking connectivity architecture and cabling best practice (R640,
R740XD servers)........................................................................................ 36
Network prerequisites................................................................................ 36
Port Configuration 41
Determine the 2U1N node type...................................................................42
Figures
Tables
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
CONTENTS
Hardware Configuration and Operating System Installation Guide - ESXi Servers 3

PowerEdge R640 and R740xd server NIC and BMC (iDRAC) port locations..
42
PowerEdge R640 and R740xd port designations - ESXi............................. 43
Configure the Management IP address - ESXi...............................52
Preparing Disks to add to a ScaleIO SDS 55
Adding disks to a ScaleIO SDS................................................................... 56
Verify the disk controller type.................................................................... 56
Enable PERCCLI on an ESXi server............................................................ 57
Create virtual devices with PERCCLI......................................................... 58
Verify the storage and disk controller status ................................ 58
Create virtual devices for HDD with PERCCLI...............................60
Create virtual devices for SSD with PERCCLI............................... 60
Ensure virtual disk creation with PERCCLI..................................... 61
Reference Material 65
Activate the NVIDIA GPU on the server..................................................... 66
Installing the iDRAC Service Module on an ESXi server..............................66
Install the iSM using SSH.............................................................. 66
Install the iSM using the vSphere CLI............................................ 67
DTK - Hardware Update Bootable ISO........................................................68
Dell OpenManage DRAC Tools (RACADM)....................................68
Recommended BIOS and firmware settings................................... 71
Troubleshooting the Hardware ISO................................................73
Additional resources...................................................................... 75
77
Chapter 6
Chapter 7
Index
CONTENTS
4Hardware Configuration and Operating System Installation Guide - ESXi Servers

vSphere client, with Power off option selected.......................................................... 28
Networking pane - VMkernel NICs tab....................................................................... 29
Networking pane - Port groups tab............................................................................ 29
PowerEdge R640 and R740xd PCI slots, integrated NICs and BMC port locations.....43
1
2
3
4
FIGURES
Hardware Configuration and Operating System Installation Guide - ESXi Servers 5

FIGURES
6Hardware Configuration and Operating System Installation Guide - ESXi Servers

Server physical requirements...................................................................................... 13
Supported operating systems - ScaleIO components..................................................13
ScaleIO management IP network................................................................................ 37
ScaleIO data IP network for Subnet #1.......................................................................38
ScaleIO data IP network for Subnet #2...................................................................... 38
R740xd server configuration for 25 GbE, 32xSAS, SFP28..........................................44
R740xd server configuration for 25GbE, 32xSAS, SFP28 (converged)...................... 44
R740xd server configuration for 25GbE, 12xNVMe, SFP28........................................ 44
R740xd server configuration for 25GbE, 12xNVMe, SFP28 (converged)................... 45
R740xd server configuration for 25GbE, 3xGPU (M60)/ 2xGPU (M10), SFP28 ....... 45
R740xd server configuration for 25GbE, 3xGPU (M60)/ 2xGPU (M10), SFP28
(converged)............................................................................................................... 46
R740xd server configuration for 10 GbE, 32xSAS, SFP+............................................ 46
R740xd server configuration for 10GbE, 32xSAS, SFP+ (converged).........................47
R740xd server configuration for 10GbE, 12xNVMe, SFP+...........................................47
R740xd server configuration for 10GbE, 12xNVMe, SFP+ (converged)...................... 47
R740xd server configuration for 10GbE, 3xGPU (M60)/ 2xGPU (M10), SFP+...........48
R740xd server configuration for 10GbE, 3xGPU (M60)/ 2xGPU (M10), SFP+
(converged)............................................................................................................... 48
R640 server configuration for 25GbE, 1CPU, SFP28 .................................................49
R640 server configuration for 25GbE, 1CPU, SFP28 (converged)............................. 49
R640 server configuration for 25GbE, 2CPU, SFP28 ................................................ 50
R640 server configuration for 25GbE, 2CPU, SFP28 (converged).............................50
R640 server configuration for 10GbE, 1CPU, SFP+ ....................................................51
R640 server configuration for 10GbE, 1CPU, SFP+ (converged)................................ 51
R640 server configuration for 10GbE, 2CPU, SFP+ ................................................... 51
R640 server configuration for 10GbE, 2CPU, SFP+ (converged)............................... 52
Hardware ISO configuration settings...........................................................................71
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
TABLES
Hardware Configuration and Operating System Installation Guide - ESXi Servers 7

TABLES
8Hardware Configuration and Operating System Installation Guide - ESXi Servers

Preface
As part of an effort to improve its product lines, EMC periodically releases revisions of
its software and hardware. Therefore, some functions described in this document
might not be supported by all versions of the software or hardware currently in use.
The product release notes provide the most up-to-date information on product
features.
Contact your EMC technical support professional if a product does not function
properly or does not function as described in this document.
Note
This document was accurate at publication time. Go to EMC Online Support (https://
support.emc.com) to ensure that you are using the latest version of this document.
Related documentation
The release notes for your version includes the latest information for your product.
The following EMC publication sets provide information about your ScaleIO or ScaleIO
Ready Node product:
lScaleIO software (downloadable as ScaleIO Software <version> Documentation
set)
lScaleIO Ready Node with AMS (downloadable as ScaleIO Ready Node with AMS
Documentation set)
lScaleIO Ready Node no AMS (downloadable as ScaleIO Ready Node no AMS
Documentation set)
lVxRack Node 100 Series (downloadable as VxRack Node 100 Series
Documentation set)
You can download the release notes, the document sets, and other related
documentation from EMC Online Support.
Typographical conventions
EMC uses the following type style conventions in this document:
Bold Used for names of interface elements, such as names of windows,
dialog boxes, buttons, fields, tab names, key names, and menu paths
(what the user specifically selects or clicks)
Italic Used for full titles of publications referenced in text
Monospace Used for:
lSystem code
lSystem output, such as an error message or script
lPathnames, filenames, prompts, and syntax
lCommands and options
Monospace italic Used for variables
Monospace bold Used for user input
[ ] Square brackets enclose optional values
Preface 9

| Vertical bar indicates alternate selections - the bar means “or”
{ } Braces enclose content that the user must specify, such as x or y or
z
... Ellipses indicate nonessential information omitted from the example
Where to get help
EMC support, product, and licensing information can be obtained as follows:
Product information
For documentation, release notes, software updates, or information about EMC
products, go to EMC Online Support at https://support.emc.com.
Technical support
Go to EMC Online Support and click Service Center. You will see several options
for contacting EMC Technical Support. Note that to open a service request, you
must have a valid support agreement. Contact your EMC sales representative for
details about obtaining a valid support agreement or with questions about your
account.
Your comments
Your suggestions will help us continue to improve the accuracy, organization, and
overall quality of the user publications. Send your opinions of this document to
[email protected].
Preface
10 Hardware Configuration and Operating System Installation Guide - ESXi Servers

CHAPTER 1
Introduction
This section describes the prerequisites for configuring the hardware and the
operating system in a ScaleIO Ready Node environment. It also provides a top-level
view of the tasks involved.
lOverview............................................................................................................ 12
lScaleIO component requirements.......................................................................12
lPrerequisites.......................................................................................................14
lSupported hardware configurations....................................................................16
Introduction 11

Overview
To prepare a server to be part of a ScaleIO Ready Node environment, you are required
to perform the following steps.
1. Ensure that the environment meets the prerequisites.
2. Configure the hardware, including the BIOS, BMC (iDRAC) IP address, and ISO
installation.
3. Install the operating system.
4. Prepare the physical network connections.
5. Configure network ports.
6. Configure disks.
These steps are described in full in this document.
ScaleIO component requirements
Components and servers in the ScaleIO system must meet the following requirements.
ScaleIO cluster components
List of required ScaleIO servers.
lScaleIO component servers:
n3-node cluster
– One Master MDM
– One Slave MDM
– One Tie Breaker
– Minimum of three SDSs (on the same servers as the above components, or
on three different servers)
– SDCs, up to the maximum allowed (on the same servers as the above
components, or on different servers)
n5-node cluster
– One Master MDM
– Two Slave MDMs
– Two Tie Breakers
– Minimum of three SDSs (on the same servers as the above components, or
on three different servers)
– SDCs, up to the maximum allowed (on the same servers as the above
components, or on different servers)
lScaleIO Gateway server on a separate server, or together with an MDM or SDS.
Do not install the Gateway on an SDC server or on an SDS on which RFcache will
be enabled.
Introduction
12 Hardware Configuration and Operating System Installation Guide - ESXi Servers

Physical server requirements
Table 1 Server physical requirements
Component Requirement
Port specifications lThe following TCP ports are not used by any other application, and are open in the local
firewall of the server:
nMDM: 6611 and 9011
nSDS: 7072 (for multiple SDS, ports 7073-7076)
nScaleIO Gateway (includes REST Gateway, Installation Manager, and SNMP trap
sender): 80 and 443
nLight Installation Agent (LIA): 9099
nSDBG ports: MDM 25620, SDS 25640, Multiple SDS 25641-25644 (not 25640).
These ports are used by ScaleIO internal debugging tools to extract live information
from the system for debugging purposes.
lThe following UDP port is open in the local firewall of the server:
nSNMP traps: 162
Note
You can change the default ports. For more information, see “Changing default ports” in the
ScaleIO Deployment Guide .
Supported operating systems
The list of operating systems supported by this version of ScaleIO.
For the most updated list, see the EMC Simple Support Matrix (ESSM) at https://
elabnavigator.emc.com/eln/elnhome.
Table 2 Supported operating systems - ScaleIO components
Operating system Requirement
Hypervisors lVMware ESXi OS: 6.0 U3, 6.5, managed by vCenter 6.0 or 6.5
ScaleIO Gateway server requirements
Component Requirement
Supported operating systems lWindows 2008 R2, 2012 R2, or 2016, including the Visual C++ redistributable 2010
package, 64-bit. Server Core editions are not supported.
Every server requires 2 cores and a minimum of 3 GB available RAM.
Connectivity The following TCP ports are not used by any other application, and are open in the local
firewall of the server: 80 and 443 (or 8080 and 8443).
Introduction
Physical server requirements 13

Component Requirement
You can change the default ports. For more information, see “Changing default ports” in the
user documentation.
Supported web browsers lInternet Explorer 10, or later
lFirefox, version 42, or later
lChrome, version 45, or later
Java requirements l1.8 or later, 64-bit
Other lFor a Windows Gateway, the Windows Management Instrumentation service must be
enabled on the IM server and on all Windows ScaleIO nodes.
lDo not install the Gateway on a server on which RFcache will be enabled or on which SDC
will be installed.
lThe Gateway server must have connectivity to all the nodes that are being installed. If you
are using separate networks for management and data, the server must be able to
communicate with both networks.
Prerequisites
The following prerequisites are relevant to using ScaleIO software in the ScaleIO
Ready Node environment.
Connectivity
Ensure that you have the following information at hand:
lThe IP address range, subnet, and Gateway IP addresses for the management and
data network for all servers.
lThe IP address range for ScaleIO VMs (SVMs).
lThe IP address range, subnet, and Gateway IP addresses for the BMC (iDRAC)
port for all servers.
lAll root and administrator passwords that you plan to set or use on the server and
BMC (iDRAC).
lAll root and administrator passwords that you plan to set or use on the SVM.
Note
ScaleIO Ready Node servers shipped with a factory-installed ESXi image are
configured with root as the username and the service tag number as the password.
To connect via a service PC or laptop computer using SSH/RDP:
lEnsure that the computer has a functional network port and an available IP
address, subnet, and Gateway on the management network.
For console operations (KVM access):
lEnsure that you have either a VGA tool kit to allow console connection from a
laptop computer to a server, or a computer screen and keyboard connection to the
rack.
Introduction
14 Hardware Configuration and Operating System Installation Guide - ESXi Servers

Java
These Java versions are required:
Use Required version
ScaleIO GUI Java 1.8 latest update
BMC (iDRAC) management client (Java
console)
Java 1.7 latest update
In your internet browser, ensure that pop-ups are enabled for Java console features.
Hardware and operating systems
Ensure that all hardware is assembled as described in the ScaleIO Ready Node
Hardware Installation Guide .
Unless otherwise specified in this guide, install the operating system in the server's
BOSS device and configure IP addresses.
For converged deployments on ESXi servers, add the nodes to vCenter.
ScaleIO packages
Download and extract the ScaleIO files for your environment.
Refer to the EMC Simple Support Matrix (ESSM) for supported versions. The files
can be downloaded from https://support.emc.com/products/33925_ScaleIO:
lESXi
nScaleIO 2.0.x.x Complete VMware Software Download
nScaleIO 2.0.x.x GUI for Windows Software Download
Supported Internet browsers
lGoogle Chrome
lFirefox 30 or higher
lInternet Explorer 8 or higher
Disk prerequisites
If an H730p/H740p controller card is installed on the server, install the PERCCLI disk
utility.
iDRAC Service Module
If an NVMe device is installed on the server, install the iDRAC Service Module (iSM).
For instructions, see Installing the iDRAC Service Module on an ESXi server on page
66.
Introduction
Java 15

Supported hardware configurations
The supported hardware configurations are listed in the EMC Simple Support Matrix
(ESSM) for your software version.
Download the ESSM from https://elabnavigator.emc.com/eln/elnhome. Click Simple
Support Matrices > Storage and select the desired product and version.
The following points apply, as well:
lBoot drive - All configurations have two 120 GB BOSS boot drives (configured as
RAID 1).
lSwitch requirements
One 10 GbE switch is required. Two or more 10 GbE switches may be used for high
availability and load balancing purposes. Separate (one or more) 1 GbE switches
may be used for the management network.
Before you begin connecting the switches, ensure that they are installed and
configured. Do not connect the switches to your organization's network until all
installation and configuration activities are complete.
Each node has four 10 GbE ports. Two are used for internal network
communications. The other two ports are used for VMware high availability, DRS,
and client network traffic.
The switches must have sufficient available network ports to accommodate the
following:
nData network 10 GbE switches:
– Two 10 GbE ports per node, per switch
nManagement network switches:
– One 1 GbE and one BMC (iDRAC) port per node.
lvCenter support
vCenter 6.0 and vCenter 6.5 are supported. Connecting to a vCenter enables
adding hosts to a VcDatacenter folder.
Network requirements are described later in this document.
Introduction
16 Hardware Configuration and Operating System Installation Guide - ESXi Servers

CHAPTER 2
Configuring the Hardware
This section describes how to configure the hardware, set BMC (iDRAC) IP
addresses, and map the ISO for servers in a ScaleIO Ready Node environment.
lSet up the BMC (iDRAC) IP address and BIOS...................................................18
lVerify the status of the system hardware, storage controller, and disks............ 20
lOpen the KVM console....................................................................................... 21
lUpdating the BIOS, firmware and settings......................................................... 22
Configuring the Hardware 17

Set up the BMC (iDRAC) IP address and BIOS
Set up the BMC (iDRAC) IP address and set up or validate the BIOS on the ScaleIO
Ready Node servers.
Before you begin
Ensure that you have access to, or have the details for:
lThe KVM console
lThe server BMC (iDRAC) IP address
lThe server BMC (iDRAC) subnet mask
lThe Gateway IP address
lThe VLAN ID of the BMC (iDRAC), if the VLAN is used
Note
The BMC (iDRAC) IP address may be an IPv4 or an IPv6 address.
Defaults:
lBMC (iDRAC) username - root
lBMC (iDRAC) password - The default password is Scaleio123.
lBIOS password - emcbios
During the BMC (iDRAC) IP and BIOS setup, use the following operations:
lUse the arrow keys to navigate in the BIOS screens.
lUse the + and - keys on the keyboard to change the option selection in the BIOS
screens.
lUse SPACE or ENTER keys to change the settings.
lThe ENTER key opens a list to the desired values.
Procedure
1. Open the KVM console.
For console operations (KVM access), ensure that you have either a VGA tool
kit/Crash Cart to allow physical console connection from a laptop computer to a
server, or a computer screen and keyboard connection to the rack.
2. Press F2 to access the main menu.
3. From the System Setup Main Menu screen, select the iDRAC Settings menu
option.
4. Configure network settings:
a. In the iDRAC Settings screen, select Network.
b. In the iDRAC Settings--Network pane, verify the following parameter
values:
lEnable NIC = Enabled
lNIC Selection = Dedicated
c. From the IPv4 Settings pane, configure the IPv4 parameter values for the
BMC (iDRAC) port:
Configuring the Hardware
18 Hardware Configuration and Operating System Installation Guide - ESXi Servers

lEnable IP IPv4 = Enabled
lEnable DHCP = Disabled
lStatic IP Address = Static IP address
lStatic Gateway = Gateway IP address
lStatic Subnet Mask = Subnet mask IP address
d. From the IPv6 Settings pane, configure the IPv6 parameter values for the
BMC (iDRAC) port.
e. From the IPMI Settings pane, verify the following parameter values:
lEnable IPMI Over LAN = Enabled
lChannel Privilege Level Limit = Administrator
f. If you are working in a VLAN setup, access the VLAN Configuration pane
and configure the VLAN ID parameters.
g. When the parameter set up is complete, click Back to display the iDRAC
Settings screen.
5. From the iDRAC Settings screen, click Finish, Yes, and then OK to return to
the System Setup Main Menu screen.
6. In the System Setup Main Menu screen, select the System BIOS menu.
7. In the System BIOS Settings screen, verify that the processor settings are
correct.
If the settings are incorrect, configure them as follows:
a. Select Processor Settings.
b. In the System BIOS Settings--Processor Settings pane, verify the
following parameter values:
lVirtualization Technology = Enabled
lNumber of Cores Per Processor = All
c. Click Back to return to the System BIOS Settings screen.
8. Configure boot settings,:
a. Select Boot Settings.
b. In the System Boot Settings--Boot Settings pane, verify that Boot Mode
is set to BIOS/UEFI.
c. Select the BIOS Boot Settings link.
d. In the System BIOS Settings--Boot Settings--BIOS Boot Settings pane,
verify that:
lIn the Boot Sequence list, Hard drive C: appears as the first item.
lIn the Hard-Disk Drive Sequence list, the BOSS device appears as the
first item.
e. Click Back twice to return to the System BIOS Settings screen.
9. Configure integrated devices:
a. Select Integrated Devices.
The System BIOS Settings--Integrated Devices screen appears.
Configuring the Hardware
Set up the BMC (iDRAC) IP address and BIOS 19

b. Set SR-IOV Global Enable to Enabled.
c. Verify that the Internal USB Port parameter is set to Off.
d. Click Back to return to the System BIOS Settings screen.
10. From the System BIOS Settings screen, click Finish, Yes, and then OK to
return to the System Setup Main Menu screen.
11. Select Finish to exit the BIOS and apply all settings post boot.
Results
The BMC (iDRAC) IP and server BIOS address configuration is complete.
Verify the status of the system hardware, storage
controller, and disks
Verify the status of the system hardware, storage controller, and disks on a ScaleIO
Ready Node server.
Before you begin
Ensure that you know:
lThe IP address of the BMC (iDRAC) port
lThe username and password for the BMC (iDRAC) portal (default username and
password are root and password)
Procedure
1. From a browser, go to http://
<BMC/iDRAC_IP_address> .
The DELL Console Login window is displayed.
2. Type the user name and password, then click Login.
The Dashboard displays the high-level status of all hardware devices in the
System Health pane.
In an ideal scenario, all hardware sensors should appear in green.
3. To view the event log (SEL log), on the Dashboard select Maintenance >
System Event Logs.
The System Event Log pane is displayed with color-coded severity levels.
4. Ensure that any power supply- and fan-related events in the event log are non-
repetitive. Repetitive events may be due to the intermittent nature of faults,
such as poor physical connections.
For repetitive events, it is recommended that you remove the relevant hardware
module and replace it in its socket.
5. In the navigation pane, select Storage > Overview > Controller.
A table displays the controller-related information, with the controller type in
the Name column. For example:
PERC H740P Mini (Embedded) or PERC H730 Mini (Embedded)
6. In the navigation pane, select Storage > Physical Disks.
A table displays information regarding the physical disks, and an option to
Blink/Unblink the selected disk.
Configuring the Hardware
20 Hardware Configuration and Operating System Installation Guide - ESXi Servers