Dell DX6000G User manual

Dell DX Object Storage
Platform
User’s Guide
book.book Page 1 Tuesday, March 1, 2011 12:08 PM

Notes, Cautions, and Warnings
NOTE: A NOTE indicates important information that helps you make better use of
your computer.
CAUTION: A CAUTION indicates potential damage to hardware or loss of data if
instructions are not followed.
WARNING: A WARNING indicates a potential for property damage, personal
injury, or death.
____________________
Information in this publication is subject to change without notice.
© 2010–2011 Dell Inc. All rights reserved.
Reproduction of these materials in any manner whatsoever without the written permission of Dell Inc.
is strictly forbidden.
Trademarks used in this text: Dell™, the DELL logo, PowerEdge™,and OpenManage™ are
trademarks of Dell Inc. Red Hat Enterprise Linux®and Enterprise Linux®are registered trademarks
of Red Hat, Inc. in the United States and/or other countries.
Other trademarks and trade names may be used in this publication to refer to either the entities claiming
the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and
trade names other than its own.
February 2011 Rev. A02
book.book Page 2 Tuesday, March 1, 2011 12:08 PM

Contents 3
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 7
2 DX Object Storage Platform
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Network Topologies . . . . . . . . . . . . . . . . . . . . 9
Local Cluster Configuration. . . . . . . . . . . . . 10
Subcluster Configuration . . . . . . . . . . . . . . 14
Remote Cluster Configuration . . . . . . . . . . . 16
DX Cluster File Server Configuration . . . . . . . . 18
Application Access to DX Object
Storage Platform . . . . . . . . . . . . . . . . . . . . . 18
Communicating Directly With a Local Cluster . . . 18
Communicating Directly With a
Remote Cluster . . . . . . . . . . . . . . . . . . . 19
Communicating With a Cluster Through the
Cluster File Server . . . . . . . . . . . . . . . . . 19
3 Setting Up the DX Object
Storage Platform . . . . . . . . . . . . . . . . . . . 21
Installing the Initial Cluster or Redundant
Cluster Services Node . . . . . . . . . . . . . . . . . . 21
Configuring the DX Object Storage Platform . . . . . . 21
Adding Additional DX Storage Nodes. . . . . . . . . . 22
Installing the System in a Rack . . . . . . . . . . . 23
book.book Page 3 Tuesday, March 1, 2011 12:08 PM

4Contents
Cabling the Appliance . . . . . . . . . . . . . . . 23
Turning On the DX Storage Node . . . . . . . . . . 23
Enabling Capacity in DX Cluster
Services Console . . . . . . . . . . . . . . . . . . 23
Adding Network Adapter to
DX Storage Nodes . . . . . . . . . . . . . . . . . 24
Adding Network Adapters to DX Cluster
Services Nodes . . . . . . . . . . . . . . . . . . . 24
Adding Cluster File Server Nodes . . . . . . . . . 25
4 Updating the Dell DX Object
Storage Platform Software . . . . . . . . . . . 27
Checking Software Version Compatibility . . . . . . . 28
Using the DX Object Storage Platform
Compatibility Manager . . . . . . . . . . . . . . . 28
Troubleshooting the Compatibility Manager . . . . 29
Updating the DX Cluster Services Node Software . . . 30
Updating the DX Storage Node Software . . . . . . . . 30
Receiving System Software
Update Notifications . . . . . . . . . . . . . . . . 31
Updating OpenManage Server Administrator . . . 31
Updating the DX Cluster File Server Software . . . . . 32
5 Disaster Recovery . . . . . . . . . . . . . . . . . 33
Recovery Solution Map . . . . . . . . . . . . . . . . . 33
Recovering From Disk Failures . . . . . . . . . . . . . 35
Single Disk Failure . . . . . . . . . . . . . . . . . 35
Two Disk Failure. . . . . . . . . . . . . . . . . . . 35
Detecting and Monitoring Disk Failures . . . . . . . . 35
book.book Page 4 Tuesday, March 1, 2011 12:08 PM

Contents 5
Reinstalling the Operating System . . . . . . . . . . . 36
Configuring and Managing RAID . . . . . . . . . . . . 36
Recovering the Operating System . . . . . . . . . . . . 37
Configuring the Partitions During Operating
System Recovery . . . . . . . . . . . . . . . . . . . . . 37
Reinstalling the DX Cluster Services
Node Software . . . . . . . . . . . . . . . . . . . . . . 39
Setting Up and Configuring the Network . . . . . . 39
Connecting the DX Cluster Services Node
to the Private and Public Networks . . . . . . . . . . . 40
Connecting the DX Storage Nodes to
the Private Network . . . . . . . . . . . . . . . . . . . 40
Creating a Local YUM Repository on
the DX6000 System . . . . . . . . . . . . . . . . . . . . 40
Installing Dell DX Compatibility Manager . . . . . . . 42
Installing Dell OpenManage
Server Administrator. . . . . . . . . . . . . . . . . . . 42
Installing DX Cluster Services Node Software . . . . . 43
Configuring the DX Cluster Services
Node Software . . . . . . . . . . . . . . . . . . . . . . 43
Restoring the Cluster Configurations . . . . . . . . . . 44
Bringing Up the DX Storage Cluster. . . . . . . . . . . 44
6 Additional References. . . . . . . . . . . . . . . 45
Software Documents . . . . . . . . . . . . . . . . . . 45
book.book Page 5 Tuesday, March 1, 2011 12:08 PM

6Contents
Hardware Documents . . . . . . . . . . . . . . . . . . 47
Systems Management Documents. . . . . . . . . . . . 48
book.book Page 6 Tuesday, March 1, 2011 12:08 PM

Introduction 7
1
Introduction
The Dell DX Object Storage Platform provides a simple and cost effective
pool of storage for less active or fixed content. The object storage platform
allows you to:
• Expand the capacity to meet rapidly changing needs without disrupting
services.
• Set policies for file retention and replication.
• Distribute information to multiple geographic locations.
• Integrate through an HTTP based interface.
• Integrate through a CIFS/NFS Gateway
The DX Object Storage Platform includes the following hardware and
software:
•DellDX6 a00 Cluster Services Node (where a is a variable that indicates
generation).
• Two or more Dell DX6
axy S Object Storage Nodes (where a is a variable
that indicates generation, and the variables
xy indicate the number of
drives. For example, 6012S indicates a 12-drive storage node).
•DellDX6
a 00G Cluster File Server (optional) (where a is a variable that
indicates generation and "G" indicates a general support node).
• A preinstalled operating system and Dell OpenManage software on the DX
Cluster Services Node.
• Dell DX Object Storage Platform software.
• A pre-installed operating system on the DX Cluster File Server.
• Dell DX Cluster File Server software.
Table 1-1 lists the terms used in this document to refer to various
components of the DX Object Storage Platform.
book.book Page 7 Tuesday, March 1, 2011 12:08 PM

8Introduction
Table 1-1. Terms Used in the Document
Component Description
DX Object Storage Cluster DX6a00S system and two or more DX6axyS
systems
DX Cluster Services Node,
Cluster Services Node or CSN
DX6a00 system
DX Object Storage Node, DX
Storage Node or SN
DX6axyS system
DX Content Router Software component on the DX6a00 system
DX Object Storage Software Software component on the DX6axyS system
DX Storage Cluster File Server,
DX Cluster File Server (CFS)
DX6a00G system running DX Cluster File Server
Software
book.book Page 8 Tuesday, March 1, 2011 12:08 PM

DX Object Storage Platform Overview 9
2
DX Object Storage Platform
Overview
Network Topologies
The Dell DX Object Storage Platform network consists of one or more DX
Storage Clusters connected by DX Content Router services. There are many
possible network topologies depending on the storage network goals. All
networks share the following features:
• One or two DX Cluster Services Nodes that provide services to the storage
cluster.
• Two or more DX Storage Nodes that provide data storage services.
• Ethernet switches to enable required services within a storage cluster
(private network) and access to the storage nodes from outside the cluster
(public network).
• Services such as Domain Name System (DNS) and Dynamic Host
Configuration Protocol (DHCP).
NOTE: DX Storage Cluster requires a dedicated internal network to ensure that the
storage cluster traffic is protected from unauthorized access. The dedicated
network also ensures that the external or corporate network is isolated from both
the PXE boot server and cluster multicast traffic.
All network services required by the cluster are present in the DX Cluster
Services Node
software bundle and preinstalled on the Cluster Services Node .
The DX Storage Cluster is automatically configured with defaults for all
network parameters that should be sufficient for most simple configurations.
All network parameters can be set manually to support more complex
configurations. For detailed information about setting network parameters,
see the DX Object Storage Getting Started Guide and the DX Object Storage
Advanced Administration Guide at support.dell.com/manuals.
book.book Page 9 Tuesday, March 1, 2011 12:08 PM

10 DX Object Storage Platform Overview
Network switch configuration depends on both the network topology and the
customer environment. To ensure proper functioning with default network
parameters and to maximize cluster performance, it is recommended that you
follow the best practices listed below, while configuring both the private and
public networks switches.
• Disable link aggregation configuration as the DX Object Storage Nodes
bond the system NIC ports in balanced-alb modes.
• Disable jumbo frames.
• Disable spanning tree protocol, if the switches are not being trunked.
• Disable Flow Control.
NOTE: If switches are trunked, enable spanning tree protocol and port fast on the
data intensive ports.
Contact Dell to customize the network configurations to meet the solution
needs. For more information about configuring the network for the Dell DX
Object Storage Platform, see "Network Topology" in the DX Object Storage
Advanced Administration Guide at support.dell.com/manuals.
Local Cluster Configuration
A basic Dell DX Storage Cluster consists of at least one DX Cluster Services
Node
and at least two DX Storage Nodes , along with the necessary Ethernet
infrastructure to connect them. See Figure 2-1.
Within a cluster,
DX Storage Nodes communicate with each other through
User Datagram Protocol (UDP) and IP multicast. All nodes within a cluster
must be on the same IP subnet. The
DX Cluster Services Node also
participates in UDP and multicast communication and must be located
within the same subnet.
The Layer 3 switch (IP router) used to create the storage cluster subnet must:
• Restrict UDP and multicast network to the cluster subnet
• Pass TCP (HTTP) traffic to and from the DX Cluster Services Node and
DX Storage Nodes
NOTE: You do not require a load-balancer in front of the storage cluster. The
applications communicate directly with the DX Object Storage Platform through
HTTP.
book.book Page 10 Tuesday, March 1, 2011 12:08 PM

DX Object Storage Platform Overview 11
It is recommended that you configure the network topology to enable
applications to communicate with the
DX Storage Nodes on private and
secure networks. This is the primary method to store and retrieve data from a
local DX Storage Cluster.
NOTE: If applications write to a cluster through an unsecure network, then to
prevent malicious activity, it is recommended that you allow the application to
encrypt the data before writing the data to the cluster.
Figure 2-1. Local Cluster Configuration
applications/DX6000G
public network
primary
NIC
teamed
interlink
private network
redundant connections
multi-cast
DX CSN
2 Port LOM 2 Port LOM
DX Storage
Node
DX Storage
Node
4 Port LOM
book.book Page 11 Tuesday, March 1, 2011 12:08 PM

12 DX Object Storage Platform Overview
Redundant DX Cluster Services Nodes
The Dell DX Storage Cluster supports redundant DX Cluster Services Node s.
In a redundant model, the active
DX Cluster Services Node is configured as
the primary node and the redundant
node is configured as the secondary
node. See Figure 2-2.
The primary
Cluster Services Node contains the cluster configuration files,
active networking utilities, and active content router services. A manual
failover process is required to demote the primary
DX Cluster Services Node
and promote the secondary
DX Cluster Services Node . Since there is a
combined virtual IP address for the primary
DX Cluster Services Node , a
failover to a redundant
DX Cluster Services Node is transparent to all
applications. When a failover is transparent, an IP address change is not
required. However, the application services remain unavailable until the
manual failover of the
Cluster Services Node is complete. For more
information, see the Cluster Services Installation and Configuration Guide at
support.dell.com/manuals.
book.book Page 12 Tuesday, March 1, 2011 12:08 PM

DX Object Storage Platform Overview 13
Figure 2-2. Redundant DX Cluster Services Nodes
public network
private network
redundant connections
primary
multi-cast
interlink
NIC
teamed
secondary
applications/DX6000G
DX CSN DX CSN
2 Port LOM
DX Storage
Node
2 Port LOM
DX Storage
Node
4 Port LOM 4 Port LOM
book.book Page 13 Tuesday, March 1, 2011 12:08 PM

14 DX Object Storage Platform Overview
Subcluster Configuration
A Dell DX Storage Cluster can be split into two or more subclusters. The
subclusters are physically part of the same cluster but are recognized as
logically separate entities by the DX Object Storage software that provides
local area replication.
A subcluster can be:
• A set of nodes within one physical group (such as building, floor, wing,
and so on)
• An environmental group (such as rack, network switch, and power
circuit)
• Any other defined groups
If the building or power circuit fails, other nodes have an optimal set of data
replicas. See Figure 2-3.
All
DX Storage Nodes within the DX Storage Cluster have the same network
connectivity requirements. The DX Storage Nodes must be in the same IP
subnet, must be able to participate in multicast and UDP traffic, and so on.
Clients may address
DX Storage Nodes within any subcluster and are
redirected to the proper
DX Storage Node . As a result, application awareness
of a subcluster configuration is not required.
For more information about subclusters, see "Local Area Replication" in the
DX Object Storage Advanced Administration Guide at
support.dell.com/manuals.
book.book Page 14 Tuesday, March 1, 2011 12:08 PM

DX Object Storage Platform Overview 15
Figure 2-3. Subcluster Configuration
public network
private network
redundant connections
primary
interlink
subcluster 1
subcluster 2
NIC
teamed
applications/DX6000G
multi-cast
DX CSN
2 Port LOM
DX Storage
Node
2 Port LOM
2 Port LOM 2 Port LOM
4 Port LOM
DX Storage
Node
DX Storage
Node
DX Storage
Node
book.book Page 15 Tuesday, March 1, 2011 12:08 PM

16 DX Object Storage Platform Overview
Remote Cluster Configuration
When you deploy multiple DX Storage Clusters (for example multiple sites,
offsite backup, and so on), the DX Cluster Services Node provides the ability
to replicate data between DX Storage Clusters using the DX Content Router
software.
The DX Content Router provides two services:
•Publisherservice
• Replicator service
Clusters are connected by one cluster's replicator service to another cluster's
publisher service. This link can either be one-way (for offsite backup) or
two-way (for mirrored configurations).
Each publisher can support a number of replicators with a specific set of
replication rules. The publisher does not provide same set of data objects to
each replicator.
For redundancy, more than one DX Cluster Services Node may be present in a
cluster, but only one DX Cluster Services Node can be active as a publisher at
any time.
NOTE: There can be multiple publishers per cluster, but only the active (or primary)
Cluster Services Node can be the active subscriber.
Clusters may be connected by any type of link that can handle the desired
traffic volume. This link must be secured to prevent reading cluster data
being passed between clusters. See Figure 2-4.
Cluster data is passed directly between the two
DX Cluster Services Node s. It
is not recommended to have direct communication between storage nodes in
remote clusters.
For more information about multiple cluster configurations, see the Content
Router Setup and Configuration Guide at support.dell.com/manuals.
book.book Page 16 Tuesday, March 1, 2011 12:08 PM

DX Object Storage Platform Overview 17
Figure 2-4. Multiple Cluster Configuration
multi-cast multi-cast
local cluster
remote cluster
NIC
teamed
interlink
interlink
NIC
teamed
primary primary
WAN VPN
secure network
applications/DX6000G
public network
private network
DX CSN DX CSN
4 Port
NIC
4 Port
NIC
4 Port
NIC
4 Port
NIC
2 Port LOM
DX Storage
Node
2 Port LOM 2 Port LOM 2 Port LOM
DX Storage
Node
DX Storage
Node
DX Storage
Node
redundant
connections
book.book Page 17 Tuesday, March 1, 2011 12:08 PM

18 DX Object Storage Platform Overview
DX Cluster File Server Configuration
When you deploy a DX Cluster File Server (CFS), the CFS must have direct
access to the DX Storage Cluster on the private network and the public
network. Gigabit Ethernet or higher is the recommended connection speed
between the CFS server and the DX Storage Cluster Nodes.
The CFS contains six Ethernet ports. In a single server configuration, two
Ethernet ports must be configured to the Public network and four to the DX
Storage Cluster. In a failover configuration, it may be desirable to have a
separate network dedicated to iSCSI traffic. You can configure two Ethernet
ports to the Public Network, two to the iSCSI network, and two to the DX
Storage Cluster. Depending on site requirements, these ports can be
configured for link aggregation protocol (LACP) operating in mode 4, or
adaptive load balancing (ALB) operating in mode 6. Many sites prefer ALB as
it does not require switch configuration. You may use any configuration that
is commonly used in your environment.
A common NTP time source is required for all CFS servers and the DX
Storage Cluster to provide clock synchronization. If you use Active Directory,
the NTP time source must be the same as the domain controller that is used
for handling Active Directory based credentials.
Application Access to DX Object Storage
Platform
Communicating Directly With a Local Cluster
Applications can communicate with a DX Storage Node at any time. The DX
Object Storage software automatically redirects applications to the proper
node even if the addressed node is an incorrect node to handle a request. If
both
Cluster Services Node and cluster nodes are not responding, you can
communicate with the remote cluster by addressing the remote Cluster
Storage Nodes or remote
Cluster Services Node directly. For many
configurations, it is recommended to create a single name through DNS
which maps to the DX Storage Nodes. A single name allows communication
with the DX Storage Nodes using one domain name. DNS automatically
rotates addressing through the storage nodes. For more information, see the
DX Object Storage Application Guide at support.dell.com/manuals.
book.book Page 18 Tuesday, March 1, 2011 12:08 PM

DX Object Storage Platform Overview 19
Communicating Directly With a Remote Cluster
Applications may communicate with a remote DX Storage Cluster during
local-cluster disaster recovery. For an application to communicate to a remote
DX Storage Cluster, the
DX Cluster Services Node on the remote public
network must be addressed. A proxy server on the
DX Cluster Services Node
(on the local cluster) accepts HTTP requests from the application and
forwards them to a DX Storage Node in the DX Storage Cluster. The DX
Storage Node redirects the requests if necessary, and returns the response
back to the requester.
Communicating With a Cluster Through the Cluster File Server
Applications that do not support HTTP integration with the DX Object
Storage Platform can communicate with DX Storage through the DX Storage
Cluster File Server (CFS). The CFS is an optional node in the DX Platform
ecosystem that extends the DX Platform archiving frontend enablement to
support a network file system for Windows and UNIX clients using CIFS and
NFS protocols.
book.book Page 19 Tuesday, March 1, 2011 12:08 PM

20 DX Object Storage Platform Overview
book.book Page 20 Tuesday, March 1, 2011 12:08 PM