Red Hat CLUSTER SUITE FOR ENTERPRISE LINUX 5.2 Installation guide

Red Hat Cluster Suite Overview
Red Hat Cluster Suite for
Red Hat Enterprise Linux
5.2
Cluster_Suite_Overview
ISBN: N/A
Publication date: May 2008

Red Hat Cluster Suite Overview provides an overview of Red Hat Cluster Suite for Red Hat
Enterprise Linux 5.2
Red Hat Cluster Suite Overview

Red Hat Cluster Suite Overview: Red Hat Cluster Suite for
Red Hat Enterprise Linux
Copyright ©2008 Red Hat, Inc.
Copyright ©2008 Red Hat, Inc. This material may only be distributed subject to the terms and conditions set forth in the
Open Publication License, V1.0 or later with the restrictions noted below (the latest version of the OPL is presently
available at http://www.opencontent.org/openpub/).
Distribution of substantively modified versions of this document is prohibited without the explicit permission of the
copyright holder.
Distribution of the work or derivative of the work in any standard (paper) book form for commercial purposes is
prohibited unless prior permission is obtained from the copyright holder.
Red Hat and the Red Hat "Shadow Man" logo are registered trademarks of Red Hat, Inc. in the United States and other
countries.
All other trademarks referenced herein are the property of their respective owners.
The GPG fingerprint of the [email protected] key is:
CA 20 86 86 2B D6 9D FC 65 F6 EC C4 21 91 80 CD DB 42 A6 0E
1801 Varsity Drive
Raleigh, NC 27606-2072
USA
Phone: [phone]
Phone: [phone]
Fax: [phone]
PO Box 13588
Research Triangle Park, NC 27709
USA

Red Hat Cluster Suite Overview

Introduction ..............................................................................................................vii
1. Document Conventions ................................................................................ viii
2. Feedback ......................................................................................................ix
1. Red Hat Cluster Suite Overview .............................................................................. 1
1. Cluster Basics ................................................................................................ 1
2. Red Hat Cluster Suite Introduction .................................................................. 2
3. Cluster Infrastructure ...................................................................................... 4
3.1. Cluster Management ........................................................................... 5
3.2. Lock Management ............................................................................... 6
3.3. Fencing .............................................................................................. 6
3.4. Cluster Configuration System ..............................................................11
4. High-availability Service Management ............................................................13
5. Red Hat GFS ................................................................................................16
5.1. Superior Performance and Scalability ..................................................18
5.2. Performance, Scalability, Moderate Price .............................................18
5.3. Economy and Performance .................................................................19
6. Cluster Logical Volume Manager ....................................................................20
7. Global Network Block Device .........................................................................24
8. Linux Virtual Server .......................................................................................25
8.1. Two-Tier LVS Topology ......................................................................27
8.2. Three-Tier LVS Topology ....................................................................30
8.3. Routing Methods ................................................................................32
8.4. Persistence and Firewall Marks ...........................................................35
9. Cluster Administration Tools ..........................................................................36
9.1. Conga ...............................................................................................36
9.2. Cluster Administration GUI ..................................................................39
9.3. Command Line Administration Tools ...................................................42
10. Linux Virtual Server Administration GUI ........................................................43
10.1. CONTROL/MONITORING ................................................................44
10.2. GLOBAL SETTINGS ........................................................................46
10.3. REDUNDANCY ................................................................................47
10.4. VIRTUAL SERVERS ........................................................................48
2. Red Hat Cluster Suite Component Summary ..........................................................57
1. Cluster Components .....................................................................................57
2. Man Pages ...................................................................................................62
3. Compatible Hardware ....................................................................................65
Index .......................................................................................................................67
v

vi

Introduction
This document provides a high-level overview of Red Hat Cluster Suite for Red Hat Enterprise
Linux 5 and is is organized as follows:
•Chapter 1, Red Hat Cluster Suite Overview
•Chapter 2, Red Hat Cluster Suite Component Summary
Although the information in this document is an overview, you should have advanced working
knowledge of Red Hat Enterprise Linux and understand the concepts of server computing to
gain a good comprehension of the information.
For more information about using Red Hat Enterprise Linux, refer to the following resources:
•Red Hat Enterprise Linux Installation Guide — Provides information regarding installation of
Red Hat Enterprise Linux 5.
•Red Hat Enterprise Linux Deployment Guide — Provides information regarding the
deployment, configuration and administration of Red Hat Enterprise Linux 5.
For more information about Red Hat Cluster Suite for Red Hat Enterprise Linux 5, refer to the
following resources:
•Configuring and Managing a Red Hat Cluster — Provides information about installing,
configuring and managing Red Hat Cluster components.
•LVM Administrator's Guide: Configuration and Administration — Provides a description of the
Logical Volume Manager (LVM), including information on running LVM in a clustered
environment.
•Global File System: Configuration and Administration — Provides information about installing,
configuring, and maintaining Red Hat GFS (Red Hat Global File System).
•Using Device-Mapper Multipath — Provides information about using the Device-Mapper
Multipath feature of Red Hat Enterprise Linux 5.
•Using GNBD with Global File System — Provides an overview on using Global Network Block
Device (GNBD) with Red Hat GFS.
•Linux Virtual Server Administration — Provides information on configuring high-performance
systems and services with the Linux Virtual Server (LVS).
•Red Hat Cluster Suite Release Notes — Provides information about the current release of
Red Hat Cluster Suite.
Red Hat Cluster Suite documentation and other Red Hat documents are available in HTML,
vii

PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at
http://www.redhat.com/docs/.
1. Document Conventions
Certain words in this manual are represented in different fonts, styles, and weights. This
highlighting indicates that the word is part of a specific category. The categories include the
following:
Courier font
Courier font represents commands,file names and paths, and prompts .
When shown as below, it indicates computer output:
Desktop about.html logs paulwesterberg.png
Mail backupfiles mail reports
bold Courier font
Bold Courier font represents text that you are to type, such as: service jonas start
If you have to run a command as root, the root prompt (#) precedes the command:
#gconftool-2
italic Courier font
Italic Courier font represents a variable, such as an installation directory:
install_dir/bin/
bold font
Bold font represents application programs and text found on a graphical interface.
When shown like this: OK , it indicates a button on a graphical application interface.
Additionally, the manual uses different strategies to draw your attention to pieces of information.
In order of how critical the information is to you, these items are marked as follows:
Note
A note is typically information that you need to understand the behavior of the
system.
Introduction
viii

Tip
A tip is typically an alternative way of performing a task.
Important
Important information is necessary, but possibly unexpected, such as a
configuration change that will not persist after a reboot.
Caution
A caution indicates an act that would violate your support agreement, such as
recompiling the kernel.
Warning
A warning indicates potential data loss, as may happen when tuning hardware
for maximum performance.
2. Feedback
If you spot a typo, or if you have thought of a way to make this document better, we would love
to hear from you. Please submit a report in Bugzilla (http://bugzilla.redhat.com/bugzilla/) against
the component Documentation-cluster.
Be sure to mention the document's identifier:
Cluster_Suite_Overview(EN)-5.2 (2008-06-01:T15:23)
By mentioning this document's identifier, we know exactly which version of the guide you have.
If you have a suggestion for improving the documentation, try to be as specific as possible. If
you have found an error, please include the section number and some of the surrounding text
so we can find it easily.
Feedback
ix

x

Red Hat Cluster Suite Overview
Clustered systems provide reliability, scalability, and availability to critical production services.
Using Red Hat Cluster Suite, you can create a cluster to suit your needs for performance, high
availability, load balancing, scalability, file sharing, and economy. This chapter provides an
overview of Red Hat Cluster Suite components and functions, and consists of the following
sections:
•Section 1, “Cluster Basics”
•Section 2, “Red Hat Cluster Suite Introduction”
•Section 3, “Cluster Infrastructure”
•Section 4, “High-availability Service Management”
•Section 5, “Red Hat GFS”
•Section 6, “Cluster Logical Volume Manager”
•Section 7, “Global Network Block Device”
•Section 8, “Linux Virtual Server”
•Section 9, “Cluster Administration Tools”
•Section 10, “Linux Virtual Server Administration GUI”
1. Cluster Basics
A cluster is two or more computers (called nodes or members) that work together to perform a
task. There are four major types of clusters:
• Storage
• High availability
• Load balancing
• High performance
Storage clusters provide a consistent file system image across servers in a cluster, allowing the
servers to simultaneously read and write to a single shared file system. A storage cluster
simplifies storage administration by limiting the installation and patching of applications to one
file system. Also, with a cluster-wide file system, a storage cluster eliminates the need for
redundant copies of application data and simplifies backup and disaster recovery. Red Hat
Cluster Suite provides storage clustering through Red Hat GFS.
Chapter 1.
1

High-availability clusters provide continuous availability of services by eliminating single points
of failure and by failing over services from one cluster node to another in case a node becomes
inoperative. Typically, services in a high-availability cluster read and write data (via read-write
mounted file systems). Therefore, a high-availability cluster must maintain data integrity as one
cluster node takes over control of a service from another cluster node. Node failures in a
high-availability cluster are not visible from clients outside the cluster. (High-availability clusters
are sometimes referred to as failover clusters.) Red Hat Cluster Suite provides high-availability
clustering through its High-availability Service Management component.
Load-balancing clusters dispatch network service requests to multiple cluster nodes to balance
the request load among the cluster nodes. Load balancing provides cost-effective scalability
because you can match the number of nodes according to load requirements. If a node in a
load-balancing cluster becomes inoperative, the load-balancing software detects the failure and
redirects requests to other cluster nodes. Node failures in a load-balancing cluster are not
visible from clients outside the cluster. Red Hat Cluster Suite provides load-balancing through
LVS (Linux Virtual Server).
High-performance clusters use cluster nodes to perform concurrent calculations. A
high-performance cluster allows applications to work in parallel, therefore enhancing the
performance of the applications. (High performance clusters are also referred to as
computational clusters or grid computing.)
Note
The cluster types summarized in the preceding text reflect basic configurations;
your needs might require a combination of the clusters described.
2. Red Hat Cluster Suite Introduction
Red Hat Cluster Suite (RHCS) is an integrated set of software components that can be
deployed in a variety of configurations to suit your needs for performance, high-availability, load
balancing, scalability, file sharing, and economy.
RHCS consists of the following major components (refer to Figure 1.1, “Red Hat Cluster Suite
Introduction”):
• Cluster infrastructure — Provides fundamental functions for nodes to work together as a
cluster: configuration-file management, membership management, lock management, and
fencing.
• High-availability Service Management — Provides failover of services from one cluster node
to another in case a node becomes inoperative.
• Cluster administration tools — Configuration and management tools for setting up,
configuring, and managing a Red Hat cluster. The tools are for use with the Cluster
Chapter 1. Red Hat Cluster Suite Overview
2

Infrastructure components, the High-availability and Service Management components, and
storage.
• Linux Virtual Server (LVS) — Routing software that provides IP-Load-balancing. LVS runs in
a pair of redundant servers that distributes client requests evenly to real servers that are
behind the LVS servers.
You can supplement Red Hat Cluster Suite with the following components, which are part of an
optional package (and not part of Red Hat Cluster Suite):
• Red Hat GFS (Global File System) — Provides a cluster file system for use with Red Hat
Cluster Suite. GFS allows multiple nodes to share storage at a block level as if the storage
were connected locally to each cluster node.
• Cluster Logical Volume Manager (CLVM) — Provides volume management of cluster storage.
Note
When you create or modify a CLVM volume for a clustered environment, you
must ensure that you are running the clvmd daemon. For further information,
refer to Section 6, “Cluster Logical Volume Manager”.
• Global Network Block Device (GNBD) — An ancillary component of GFS that exports
block-level storage to Ethernet. This is an economical way to make block-level storage
available to Red Hat GFS.
For a lower level summary of Red Hat Cluster Suite components and optional software, refer to
Chapter 2, Red Hat Cluster Suite Component Summary.
Red Hat Cluster Suite Introduction
3

Figure 1.1. Red Hat Cluster Suite Introduction
Note
Figure 1.1, “Red Hat Cluster Suite Introduction” includes GFS, CLVM, and
GNBD, which are components that are part of an optional package and not part
of Red Hat Cluster Suite.
3. Cluster Infrastructure
The Red Hat Cluster Suite cluster infrastructure provides the basic functions for a group of
computers (called nodes or members) to work together as a cluster. Once a cluster is formed
using the cluster infrastructure, you can use other Red Hat Cluster Suite components to suit
your clustering needs (for example, setting up a cluster for sharing files on a GFS file system or
setting up service failover). The cluster infrastructure performs the following functions:
• Cluster management
• Lock management
Chapter 1. Red Hat Cluster Suite Overview
4

• Fencing
• Cluster configuration management
3.1. Cluster Management
Cluster management manages cluster quorum and cluster membership. CMAN (an abbreviation
for cluster manager) performs cluster management in Red Hat Cluster Suite for Red Hat
Enterprise Linux 5. CMAN is a distributed cluster manager and runs in each cluster node;
cluster management is distributed across all nodes in the cluster (refer to Figure 1.2,
“CMAN/DLM Overview”).
CMAN keeps track of cluster quorum by monitoring the count of cluster nodes. If more than half
the nodes are active, the cluster has quorum. If half the nodes (or fewer) are active, the cluster
does not have quorum, and all cluster activity is stopped. Cluster quorum prevents the
occurrence of a "split-brain" condition — a condition where two instances of the same cluster
are running. A split-brain condition would allow each cluster instance to access cluster
resources without knowledge of the other cluster instance, resulting in corrupted cluster
integrity.
Quorum is determined by communication of messages among cluster nodes via Ethernet.
Optionally, quorum can be determined by a combination of communicating messages via
Ethernet and through a quorum disk. For quorum via Ethernet, quorum consists of 50 percent of
the node votes plus 1. For quorum via quorum disk, quorum consists of user-specified
conditions.
Note
By default, each node has one quorum vote. Optionally, you can configure each
node to have more than one vote.
CMAN keeps track of membership by monitoring messages from other cluster nodes. When
cluster membership changes, the cluster manager notifies the other infrastructure components,
which then take appropriate action. For example, if node A joins a cluster and mounts a GFS file
system that nodes B and C have already mounted, then an additional journal and lock
management is required for node A to use that GFS file system. If a cluster node does not
transmit a message within a prescribed amount of time, the cluster manager removes the node
from the cluster and communicates to other cluster infrastructure components that the node is
not a member. Again, other cluster infrastructure components determine what actions to take
upon notification that node is no longer a cluster member. For example, Fencing would fence
the node that is no longer a member.
Cluster Management
5

Figure 1.2. CMAN/DLM Overview
3.2. Lock Management
Lock management is a common cluster-infrastructure service that provides a mechanism for
other cluster infrastructure components to synchronize their access to shared resources. In a
Red Hat cluster, DLM (Distributed Lock Manager) is the lock manager. As implied in its name,
DLM is a distributed lock manager and runs in each cluster node; lock management is
distributed across all nodes in the cluster (refer to Figure 1.2, “CMAN/DLM Overview”). GFS and
CLVM use locks from the lock manager. GFS uses locks from the lock manager to synchronize
access to file system metadata (on shared storage). CLVM uses locks from the lock manager to
synchronize updates to LVM volumes and volume groups (also on shared storage).
3.3. Fencing
Fencing is the disconnection of a node from the cluster's shared storage. Fencing cuts off I/O
from shared storage, thus ensuring data integrity. The cluster infrastructure performs fencing
through the fence daemon, fenced.
When CMAN determines that a node has failed, it communicates to other cluster-infrastructure
components that the node has failed. fenced, when notified of the failure, fences the failed
node. Other cluster-infrastructure components determine what actions to take — that is, they
perform any recovery that needs to done. For example, DLM and GFS, when notified of a node
failure, suspend activity until they detect that fenced has completed fencing the failed node.
Upon confirmation that the failed node is fenced, DLM and GFS perform recovery. DLM
releases locks of the failed node; GFS recovers the journal of the failed node.
The fencing program determines from the cluster configuration file which fencing method to use.
Two key elements in the cluster configuration file define a fencing method: fencing agent and
Chapter 1. Red Hat Cluster Suite Overview
6

fencing device. The fencing program makes a call to a fencing agent specified in the cluster
configuration file. The fencing agent, in turn, fences the node via a fencing device. When
fencing is complete, the fencing program notifies the cluster manager.
Red Hat Cluster Suite provides a variety of fencing methods:
• Power fencing — A fencing method that uses a power controller to power off an inoperable
node.
• Fibre Channel switch fencing — A fencing method that disables the Fibre Channel port that
connects storage to an inoperable node.
• GNBD fencing — A fencing method that disables an inoperable node's access to a GNBD
server.
• Other fencing — Several other fencing methods that disable I/O or power of an inoperable
node, including IBM Bladecenters, PAP, DRAC/MC, HP ILO, IPMI, IBM RSA II, and others.
Figure 1.3, “Power Fencing Example” shows an example of power fencing. In the example, the
fencing program in node A causes the power controller to power off node D. Figure 1.4, “Fibre
Channel Switch Fencing Example” shows an example of Fibre Channel switch fencing. In the
example, the fencing program in node A causes the Fibre Channel switch to disable the port for
node D, disconnecting node D from storage.
Fencing
7

Figure 1.3. Power Fencing Example
Chapter 1. Red Hat Cluster Suite Overview
8

Figure 1.4. Fibre Channel Switch Fencing Example
Specifying a fencing method consists of editing a cluster configuration file to assign a
fencing-method name, the fencing agent, and the fencing device for each node in the cluster.
The way in which a fencing method is specified depends on if a node has either dual power
supplies or multiple paths to storage. If a node has dual power supplies, then the fencing
method for the node must specify at least two fencing devices — one fencing device for each
power supply (refer to Figure 1.5, “Fencing a Node with Dual Power Supplies”). Similarly, if a
node has multiple paths to Fibre Channel storage, then the fencing method for the node must
specify one fencing device for each path to Fibre Channel storage. For example, if a node has
two paths to Fibre Channel storage, the fencing method should specify two fencing devices —
one for each path to Fibre Channel storage (refer to Figure 1.6, “Fencing a Node with Dual Fibre
Channel Connections”).
Fencing
9

Figure 1.5. Fencing a Node with Dual Power Supplies
Chapter 1. Red Hat Cluster Suite Overview
10