Red Hat CLUSTER SUITE - CONFIGURING AND MANAGING A CLUSTER... User manual

Red Hat Cluster Suite
Configuring and Managing a
Cluster

Red Hat Cluster Suite: Configuring and Managing a Cluster
Copyright © 2000-2006 by Red Hat, Inc.Mission Critical Linux, Inc.K.M. Sorenson
Red Hat, Inc.
1801 Varsity Drive
Raleigh NC 27606-2072 USA
Phone: [phone]
Phone: [phone]
Fax: [phone]
PO Box 13588
Research Triangle Park NC 27709 USA
rh-cs(EN)-4-Print-RHI (2007-01-05T17:28)
For Part I Using the Red Hat Cluster Manager and Part III Appendixes, permission is granted to copy, distribute
and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later
version published by the Free Software Foundation. A copy of the license is available at
http://www.gnu.org/licenses/fdl.html. The content described in this paragraph is copyrighted by © Mission
Critical Linux, Inc. (2000), K.M. Sorenson (2000), and Red Hat, Inc. (2000-2003).
This material in Part II Configuring a Linux Virtual Server Cluster may be distributed only subject to the terms
and conditions set forth in the Open Publication License, V1.0 or later (the latest version is presently available at
http://www.opencontent.org/openpub/). Distribution of substantively modified versions of this material is
prohibited without the explicit permission of the copyright holder. Distribution of the work or derivative of the
work in any standard (paper) book form for commercial purposes is prohibited unless prior permission is
obtained from the copyright holder. The content described in this paragraph is copyrighted by © Red Hat, Inc.
(2000-2003).
Red Hat and the Red Hat "Shadow Man" logo are registered trademarks of Red Hat, Inc. in the United States
and other countries.
All other trademarks referenced herein are the property of their respective owners.
The GPG fingerprint of the [email protected] key is:
CA 20 86 86 2B D6 9D FC 65 F6 EC C4 21 91 80 CD DB 42 A6 0E

Table of Contents
Introduction........................................................................................................................ i
1. How To Use This Manual .................................................................................... i
2. Document Conventions....................................................................................... ii
3. More to Come ......................................................................................................v
3.1. Send in Your Feedback .........................................................................v
4. Activate Your Subscription ................................................................................ vi
4.1. Provide a Red Hat Login..................................................................... vi
4.2. Provide Your Subscription Number ................................................... vii
4.3. Connect Your System......................................................................... vii
I. Using the Red Hat Cluster Manager ............................................................................ i
1. Red Hat Cluster Manager Overview....................................................................1
1.1. Red Hat Cluster Manager Features .......................................................2
2. Hardware Installation and Operating System Configuration ...............................9
2.1. Choosing a Hardware Configuration ....................................................9
2.2. Cluster Hardware Components ...........................................................14
2.3. Setting Up the Nodes ..........................................................................18
2.4. Installing and Configuring Red Hat Enterprise Linux ........................22
2.5. Setting Up and Connecting the Cluster Hardware..............................27
3. Installing and Configuring Red Hat Cluster Suite Software..............................35
3.1. Software Installation and Configuration Tasks ...................................35
3.2. Overview of the Cluster Configuration Tool....................................36
3.3. Installing the Red Hat Cluster Suite Packages....................................39
3.4. Starting the Cluster Configuration Tool...........................................40
3.5. Naming The Cluster............................................................................43
3.6. Configuring Fence Devices.................................................................44
3.7. Adding and Deleting Members...........................................................49
3.8. Configuring a Failover Domain ..........................................................55
3.9. Adding Cluster Resources...................................................................60
3.10. Adding a Cluster Service to the Cluster............................................62
3.11. Propagating The Configuration File: New Cluster ...........................65
3.12. Starting the Cluster Software ............................................................65
4. Cluster Administration.......................................................................................67
4.1. Overview of the Cluster Status Tool.................................................67
4.2. Displaying Cluster and Service Status................................................68
4.3. Starting and Stopping the Cluster Software........................................71
4.4. Modifying the Cluster Configuration..................................................71
4.5. Backing Up and Restoring the Cluster Database................................72
4.6. Updating the Cluster Software............................................................74
4.7. Changing the Cluster Name ................................................................74
4.8. Disabling the Cluster Software ...........................................................74
4.9. Diagnosing and Correcting Problems in a Cluster..............................75
5. Setting Up Apache HTTP Server.......................................................................77
5.1. Apache HTTP Server Setup Overview ...............................................77

5.2. Configuring Shared Storage................................................................77
5.3. Installing and Configuring the Apache HTTP Server.........................78
II. Configuring a Linux Virtual Server Cluster ............................................................81
6. Introduction to Linux Virtual Server..................................................................83
6.1. Technology Overview .........................................................................83
6.2. Basic Configurations...........................................................................84
7. Linux Virtual Server Overview..........................................................................85
7.1. A Basic LVS Configuration ................................................................85
7.2. A Three Tiered LVS Configuration.....................................................87
7.3. LVS Scheduling Overview..................................................................89
7.4. Routing Methods.................................................................................91
7.5. Persistence and Firewall Marks ..........................................................93
7.6. LVS Cluster — A Block Diagram ......................................................94
8. Initial LVS Configuration...................................................................................97
8.1. Configuring Services on the LVS Routers ..........................................97
8.2. Setting a Password for the Piranha Configuration Tool ..................98
8.3. Starting the Piranha Configuration Tool Service.............................99
8.4. Limiting Access To the Piranha Configuration Tool .....................100
8.5. Turning on Packet Forwarding..........................................................101
8.6. Configuring Services on the Real Servers ........................................101
9. Setting Up a Red Hat Enterprise Linux LVS Cluster.......................................103
9.1. The NAT LVS Cluster .......................................................................103
9.2. Putting the Cluster Together .............................................................106
9.3. Multi-port Services and LVS Clustering...........................................107
9.4. FTP In an LVS Cluster......................................................................109
9.5. Saving Network Packet Filter Settings .............................................112
10. Configuring the LVS Routers with Piranha Configuration Tool ................115
10.1. Necessary Software.........................................................................115
10.2. Logging Into the Piranha Configuration Tool .............................115
10.3. CONTROL/MONITORING........................................................116
10.4. GLOBAL SETTINGS ..................................................................118
10.5. REDUNDANCY ............................................................................120
10.6. VIRTUAL SERVERS ...................................................................122
10.7. Synchronizing Configuration Files .................................................133
10.8. Starting the Cluster .........................................................................135
III. Appendixes...............................................................................................................137
A. Supplementary Hardware Information............................................................139
A.1. Attached Storage Requirements.......................................................139
A.2. Setting Up a Fibre Channel Interconnect.........................................139
A.3. SCSI Storage Requirements.............................................................141
B. Selectively Installing Red Hat Cluster Suite Packages ...................................147
B.1. Installing the Red Hat Cluster Suite Packages .................................147
C. Multipath-usage.txt File for Red Hat Enterprise Linux 4 Update 3 ......157

Index................................................................................................................................165
Colophon.........................................................................................................................171

Introduction
The Red Hat Cluster Suite is a collection of technologies working together to provide
data integrity and the ability to maintain application availability in the event of a failure.
Administrators can deploy enterprise cluster solutions using a combination of hardware
redundancy along with the failover and load-balancing technologies in Red Hat Cluster
Suite.
Red Hat Cluster Manager is a high-availability cluster solution specifically suited for
database applications, network file servers, and World Wide Web (Web) servers with dy-
namic content. A Red Hat Cluster Manager system features data integrity and applica-
tion availability using redundant hardware, shared disk storage, power management, robust
cluster communication, and robust application failover mechanisms.
Administrators can also deploy highly available applications services using Piranha, a load-
balancing and advanced routing cluster solution based on Linux Virtual Server (LVS) tech-
nology. Using Piranha, administrators can build highly available e-commerce sites that
feature complete data integrity and service availability, in addition to load balancing capa-
bilities. Refer to Part II Configuring a Linux Virtual Server Cluster for more information.
This guide assumes that the user has an advanced working knowledge of Red Hat Enter-
prise Linux and understands the concepts of server computing. For more information about
using Red Hat Enterprise Linux, refer to the following resources:
•Red Hat Enterprise Linux Installation Guide for information regarding installation.
•Red Hat Enterprise Linux Introduction to System Administration for introductory infor-
mation for new Red Hat Enterprise Linux system administrators.
•Red Hat Enterprise Linux System Administration Guide for more detailed information
about configuring Red Hat Enterprise Linux to suit your particular needs as a user.
•Red Hat Enterprise Linux Reference Guide provides detailed information suited for more
experienced users to reference when needed, as opposed to step-by-step instructions.
•Red Hat Enterprise Linux Security Guide details the planning and the tools involved in
creating a secured computing environment for the data center, workplace, and home.
HTML, PDF, and RPM versions of the manuals are available on the Red Hat Enterprise
Linux Documentation CD and online at:
http://www.redhat.com/docs/
1. How To Use This Manual
This manual contains information about setting up a Red Hat Cluster Manager system.
These tasks are described in the following chapters:

ii Introduction
•Chapter 2 Hardware Installation and Operating System Configuration
•Chapter 3 Installing and Configuring Red Hat Cluster Suite Software
Part II Configuring a Linux Virtual Server Cluster describes how to achieve load balancing
in an Red Hat Enterprise Linux cluster by using the Linux Virtual Server.
Appendix A Supplementary Hardware Information contains detailed configuration infor-
mation on specific hardware devices and shared storage configurations.
Appendix B Selectively Installing Red Hat Cluster Suite Packages contains information
about custom installation of Red Hat Cluster Suite and Red Hat GFS RPMs.
Appendix C Multipath-usage.txt File for Red Hat Enterprise Linux 4 Update 3 con-
tains information from the Multipath-usage.txt file. The file provides guidelines for
using dm-multipath with Red Hat Cluster Suite for Red Hat Enterprise Linux 4 Update
3.
This guide assumes you have a thorough understanding of Red Hat Enterprise Linux sys-
tem administration concepts and tasks. For detailed information on Red Hat Enterprise
Linux system administration, refer to the Red Hat Enterprise Linux System Administra-
tion Guide. For reference information on Red Hat Enterprise Linux, refer to the Red Hat
Enterprise Linux Reference Guide.
2. Document Conventions
In this manual, certain words are represented in different fonts, typefaces, sizes, and
weights. This highlighting is systematic; different words are represented in the same style
to indicate their inclusion in a specific category. The types of words that are represented
this way include the following:
command
Linux commands (and other operating system commands, when used) are represented
this way. This style should indicate to you that you can type the word or phrase on
the command line and press [Enter] to invoke a command. Sometimes a command
contains words that would be displayed in a different style on their own (such as file
names). In these cases, they are considered to be part of the command, so the entire
phrase is displayed as a command. For example:
Use the cat testfile command to view the contents of a file, named testfile,
in the current working directory.
file name
File names, directory names, paths, and RPM package names are represented this
way. This style indicates that a particular file or directory exists with that name on
your system. Examples:

Introduction iii
The .bashrc file in your home directory contains bash shell definitions and aliases
for your own use.
The /etc/fstab file contains information about different system devices and file
systems.
Install the webalizer RPM if you want to use a Web server log file analysis program.
application
This style indicates that the program is an end-user application (as opposed to system
software). For example:
Use Mozilla to browse the Web.
[key]
A key on the keyboard is shown in this style. For example:
To use [Tab] completion, type in a character and then press the [Tab] key. Your termi-
nal displays the list of files in the directory that start with that letter.
[key]-[combination]
A combination of keystrokes is represented in this way. For example:
The [Ctrl]-[Alt]-[Backspace] key combination exits your graphical session and returns
you to the graphical login screen or the console.
text found on a GUI interface
A title, word, or phrase found on a GUI interface screen or window is shown in this
style. Text shown in this style indicates that a particular GUI screen or an element on
a GUI screen (such as text associated with a checkbox or field). Example:
Select the Require Password checkbox if you would like your screensaver to require
a password before stopping.
top level of a menu on a GUI screen or window
A word in this style indicates that the word is the top level of a pulldown menu. If you
click on the word on the GUI screen, the rest of the menu should appear. For example:
Under File on a GNOME terminal, the New Tab option allows you to open multiple
shell prompts in the same window.
Instructions to type in a sequence of commands from a GUI menu look like the fol-
lowing example:
Go to Applications (the main menu on the panel) => Programming => Emacs Text
Editor to start the Emacs text editor.

iv Introduction
button on a GUI screen or window
This style indicates that the text can be found on a clickable button on a GUI screen.
For example:
Click on the Back button to return to the webpage you last viewed.
computer output
Text in this style indicates text displayed to a shell prompt such as error messages and
responses to commands. For example:
The ls command displays the contents of a directory. For example:
Desktop about.html logs paulwesterberg.png
Mail backupfiles mail reports
The output returned in response to the command (in this case, the contents of the
directory) is shown in this style.
prompt
A prompt, which is a computer’s way of signifying that it is ready for you to input
something, is shown in this style. Examples:
$
#
[stephen@maturin stephen]$
leopard login:
user input
Text that the user types, either on the command line or into a text box on a GUI screen,
is displayed in this style. In the following example, text is displayed in this style:
To boot your system into the text based installation program, you must type in the
text command at the boot: prompt.

replaceable

Text used in examples that is meant to be replaced with data provided by the user
is displayed in this style. In the following example,

version-number

is dis-
played in this style:
The directory for the kernel source is /usr/src/kernels/

version-number

/,
where

version-number

is the version and type of kernel installed on this
system.
Additionally, we use several different strategies to draw your attention to certain pieces of
information. In order of urgency, these items are marked as a note, tip, important, caution,
or warning. For example:

Introduction v
Note
Remember that Linux is case sensitive. In other words, a rose is not a ROSE is not a
rOsE.
Tip
The directory /usr/share/doc/ contains additional documentation for packages installed
on your system.
Important
If you modify the DHCP configuration file, the changes do not take effect until you restart
the DHCP daemon.
Caution
Do not perform routine tasks as root — use a regular user account unless you need to
use the root account for system administration tasks.
Warning
Be careful to remove only the necessary partitions. Removing other partitions could result
in data loss or a corrupted system environment.
3. More to Come
This manual is part of Red Hat’s growing commitment to provide useful and timely support
to Red Hat Enterprise Linux users.

vi Introduction
3.1. Send in Your Feedback
If you spot a typo, or if you have thought of a way to make this manual
better, we would love to hear from you. Please submit a report in Bugzilla
(http://bugzilla.redhat.com/bugzilla/) against the component rh-cs.
Be sure to mention the manual’s identifier:
rh-cs(EN)-4-Print-RHI (2007-01-05T17:28)
By mentioning this manual’s identifier, we know exactly which version of the guide you
have.
If you have a suggestion for improving the documentation, try to be as specific as possible.
If you have found an error, please include the section number and some of the surrounding
text so we can find it easily.
4. Activate Your Subscription
Before you can access service and software maintenance information, and the support doc-
umentation included in your subscription, you must activate your subscription by register-
ing with Red Hat. Registration includes these simple steps:
•Provide a Red Hat login
•Provide a subscription number
•Connect your system
The first time you boot your installation of Red Hat Enterprise Linux, you are prompted to
register with Red Hat using the Setup Agent. If you follow the prompts during the Setup
Agent, you can complete the registration steps and activate your subscription.
If you can not complete registration during the Setup Agent (which requires network
access), you can alternatively complete the Red Hat registration process online at
http://www.redhat.com/register/.
4.1. Provide a Red Hat Login
If you do not have an existing Red Hat login, you can create one when prompted during
the Setup Agent or online at:
https://www.redhat.com/apps/activate/newlogin.html
A Red Hat login enables your access to:
•Software updates, errata and maintenance via Red Hat Network

Introduction vii
•Red Hat technical support resources, documentation, and Knowledgebase
If you have forgotten your Red Hat login, you can search for your Red Hat login online at:
https://rhn.redhat.com/help/forgot_password.pxt
4.2. Provide Your Subscription Number
Your subscription number is located in the package that came with your order. If your
package did not include a subscription number, your subscription was activated for you
and you can skip this step.
You can provide your subscription number when prompted during the Setup Agent or by
visiting http://www.redhat.com/register/.
4.3. Connect Your System
The Red Hat Network Registration Client helps you connect your system so that you can
begin to get updates and perform systems management. There are three ways to connect:
1. During the Setup Agent — Check the Send hardware information and Send sys-
tem package list options when prompted.
2. After the Setup Agent has been completed — From Applications (the main menu
on the panel), go to System Tools, then select Red Hat Network.
3. After the Setup Agent has been completed — Enter the following command from
the command line as the root user:
•/usr/bin/up2date --register

viii Introduction

I. Using the Red Hat Cluster Manager
Clustered systems provide reliability, scalability, and availability to critical production ser-
vices. Using the Red Hat Cluster Manager, administrators can create high availability clus-
ters for filesharing, Web servers, and more. This part discusses the installation and con-
figuration of cluster systems using the recommended hardware and Red Hat Enterprise
Linux.
This section is licensed under the GNU Free Documentation License. For details refer to
the Copyright page.
Table of Contents
1. Red Hat Cluster Manager Overview............................................................................1
2. Hardware Installation and Operating System Configuration ...................................9
3. Installing and Configuring Red Hat Cluster Suite Software ...................................35
4. Cluster Administration................................................................................................67
5. Setting Up Apache HTTP Server ...............................................................................77

Chapter 1.
Red Hat Cluster Manager Overview
Red Hat Cluster Manager allows administrators to connect separate systems (called mem-
bers or nodes) together to create failover clusters that ensure application availability and
data integrity under several failure conditions. Administrators can use Red Hat Cluster
Manager with database applications, file sharing services, web servers, and more.
To set up a failover cluster, you must connect the nodes to the cluster hardware, and con-
figure the nodes into the cluster environment. The foundation of a cluster is an advanced
host membership algorithm. This algorithm ensures that the cluster maintains complete
data integrity by using the following methods of inter-node communication:
•Network connections between the cluster systems
•A Cluster Configuration System daemon (ccsd) that synchronizes configuration between
cluster nodes
To make an application and data highly available in a cluster, you must configure a cluster
service, an application that would benefit from Red Hat Cluster Manager to ensure high
availability. A cluster service is made up of cluster resources, components that can be failed
over from one node to another, such as an IP address, an application initialization script,
or a Red Hat GFS shared partition. Building a cluster using Red Hat Cluster Manager
allows transparent client access to cluster services. For example, you can provide clients
with access to highly-available database applications by building a cluster service using
Red Hat Cluster Manager to manage service availability and shared Red Hat GFS storage
partitions for the database data and end-user applications.
You can associate a cluster service with a failover domain, a subset of cluster nodes that
are eligible to run a particular cluster service. In general, any eligible, properly-configured
node can run the cluster service. However, each cluster service can run on only one cluster
node at a time in order to maintain data integrity. You can specify whether or not the nodes
in a failover domain are ordered by preference. You can also specify whether or not a
cluster service is restricted to run only on nodes of its associated failover domain. (When
associated with an unrestricted failover domain, a cluster service can be started on any
cluster node in the event no member of the failover domain is available.)
You can set up an active-active configuration in which the members run different cluster
services simultaneously, or a hot-standby configuration in which primary members run all
the cluster services, and a backup member takes over only if a primary member fails.
If a hardware or software failure occurs, the cluster automatically restarts the failed node’s
cluster services on the functional node. This cluster-service failover capability ensures that
no data is lost, and there is little disruption to users. When the failed node recovers, the
cluster can re-balance the cluster services across the nodes.

2 Chapter 1. Red Hat Cluster Manager Overview
In addition, you can cleanly stop the cluster services running on a cluster system and then
restart them on another system. This cluster-service relocation capability allows you to
maintain application and data availability when a cluster node requires maintenance.
1.1. Red Hat Cluster Manager Features
Cluster systems deployed with Red Hat Cluster Manager include the following features:
No-single-point-of-failure hardware configuration
Clusters can include a dual-controller RAID array, multiple bonded network channels,
multiple paths between cluster members and storage, and redundant uninterruptible
power supply (UPS) systems to ensure that no single failure results in application
down time or loss of data.
Note
For information about using dm-multipath with Red Hat Cluster Suite, refer
toAppendix C Multipath-usage.txt File for Red Hat Enterprise Linux 4 Update 3
Alternatively, a low-cost cluster can be set up to provide less availability than a no-
single-point-of-failure cluster. For example, you can set up a cluster with a single-
controller RAID array and only a single Ethernet channel.
Certain low-cost alternatives, such as host RAID controllers, software RAID without
cluster support, and multi-initiator parallel SCSI configurations are not compatible or
appropriate for use as shared cluster storage.
Cluster configuration and administration framework
Red Hat Cluster Manager allows you to easily configure and administer cluster ser-
vices to make resources such as applications, server daemons, and shared data highly
available. To create a cluster service, you specify the resources used in the cluster ser-
vice as well as the properties of the cluster service, such as the cluster service name,
application initialization (init) scripts, disk partitions, mount points, and the cluster
nodes on which you prefer the cluster service to run. After you add a cluster service,
the cluster management software stores the information in a cluster configuration file,
and the configuration data is aggregated to all cluster nodes using the Cluster Config-
uration System (or CCS), a daemon installed on each cluster node that allows retrieval
of changes to the XML-based /etc/cluster/cluster.conf configuration file.
Red Hat Cluster Manager provides an easy-to-use framework for database appli-
cations. For example, a database cluster service serves highly-available data to a
database application. The application running on a cluster node provides network
access to database client systems, such as Web applications. If the cluster service
fails over to another node, the application can still access the shared database data. A

Chapter 1. Red Hat Cluster Manager Overview 3
network-accessible database cluster service is usually assigned an IP address, which
is failed over along with the cluster service to maintain transparent access for clients.
The cluster-service framework can also easily extend to other applications through the
use of customized init scripts.
Cluster administration user interface
The Red Hat Cluster Suite management graphical user interface (GUI) facilitates the
administration and monitoring tasks of cluster resources such as the following: creat-
ing, starting, and stopping cluster services; relocating cluster services from one node
to another; modifying the cluster service configuration; and monitoring the cluster
nodes. The CMAN interface allows administrators to individually control the cluster
on a per-node basis.
Failover domains
By assigning a cluster service to a restricted failover domain, you can limit the nodes
that are eligible to run a cluster service in the event of a failover. (A cluster service
that is assigned to a restricted failover domain cannot be started on a cluster node that
is not included in that failover domain.) You can order the nodes in a failover domain
by preference to ensure that a particular node runs the cluster service (as long as that
node is active). If a cluster service is assigned to an unrestricted failover domain, the
cluster service starts on any available cluster node (if none of the nodes of the failover
domain are available).
Data integrity assurance
To ensure data integrity, only one node can run a cluster service and access cluster-
service data at one time. The use of power switches in the cluster hardware configura-
tion enables a node to power-cycle another node before restarting that node’s cluster
services during the failover process. This prevents any two systems from simulta-
neously accessing the same data and corrupting it. It is strongly recommended that
fence devices (hardware or software solutions that remotely power, shutdown, and re-
boot cluster nodes) are used to guarantee data integrity under all failure conditions.
Watchdog timers are an alternative used to ensure correct operation of cluster service
failover.
Ethernet channel bonding
To monitor the health of the other nodes, each node monitors the health of the remote
power switch, if any, and issues heartbeat pings over network channels. With Ethernet
channel bonding, multiple Ethernet interfaces are configured to behave as one, reduc-
ing the risk of a single-point-of-failure in the typical switched Ethernet connection
between systems.
Cluster-service failover capability
If a hardware or software failure occurs, the cluster takes the appropriate action to

4 Chapter 1. Red Hat Cluster Manager Overview
maintain application availability and data integrity. For example, if a node completely
fails, a healthy node (in the associated failover domain, if used) starts the service
or services that the failed node was running prior to failure. Cluster services already
running on the healthy node are not significantly disrupted during the failover process.
Note
For Red Hat Cluster Suite 4, node health is monitored through a cluster network
heartbeat. In previous versions of Red Hat Cluster Suite, node health was monitored
on shared disk. Shared disk is not required for node-health monitoring in Red Hat
Cluster Suite 4.
When a failed node reboots, it can rejoin the cluster and resume running the cluster
service. Depending on how the cluster services are configured, the cluster can re-
balance services among the nodes.
Manual cluster-service relocation capability
In addition to automatic cluster-service failover, a cluster allows you to cleanly stop
cluster services on one node and restart them on another node. You can perform
planned maintenance on a node system while continuing to provide application and
data availability.
Event logging facility
To ensure that problems are detected and resolved before they affect cluster-service
availability, the cluster daemons log messages by using the conventional Linux syslog
subsystem.
Application monitoring
The infrastructure in a cluster monitors the state and health of an application. In this
manner, should an application-specific failure occur, the cluster automatically restarts
the application. In response to the application failure, the application attempts to be
restarted on the node it was initially running on; failing that, it restarts on another
cluster node. You can specify which nodes are eligible to run a cluster service by
assigning a failover domain to the cluster service.
1.1.1. Red Hat Cluster Manager Subsystem Overview
Table 1-1 summarizes the GFS Software subsystems and their components.