Red Hat CLUSTER MANAGER - INSTALLATION AND Instruction Manual

Red Hat Cluster Manager
The Red Hat Cluster Manager Installation and
Administration Guide

ISBN: N/A
Red Hat, Inc.
1801 Varsity Drive
Raleigh, NC 27606 USA
[phone] (Voice)
[phone] (FAX)
888 733 4281 (Voice)
P.O. Box 13588
Research Triangle Park, NC 27709 USA
© 2002 Red Hat, Inc.
© 2000 Mission Critical Linux, Inc.
© 2000 K.M. Sorenson
rh-cm(EN)-1.0-Print-RHI (2002-04-17T17:16-0400)
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free
Documentation License, Version 1.1 or any later version published by the Free Software Foundation.
A copy of the license is included on the GNU Free Documentation License website.
Red Hat, Red Hat Network, the Red Hat "Shadow Man" logo, RPM, Maximum RPM, the RPM logo,
Linux Library, PowerTools, Linux Undercover, RHmember, RHmember More, Rough Cuts, Rawhide
and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc.
in the United States and other countries.
Linux is a registered trademark of Linus Torvalds.
Motif and UNIX are registered trademarks of The Open Group.
Itanium is a registered trademark of Intel Corporation.
Netscape is a registered trademark of Netscape Communications Corporation in the United States and
other countries.
Windows is a registered trademark of Microsoft Corporation.
SSH and Secure Shell are trademarks of SSH Communications Security, Inc.
FireWire is a trademark of Apple Computer Corporation.
S/390 and zSeries are trademarks of International Business Machines Corporation.
All other trademarks and copyrights referred to are the property of their respective owners.
ii

iii
Acknowledgments
The Red Hat Cluster Manager software was originally based on the open source Kimberlite
http://oss.missioncriticallinux.com/kimberlite/ cluster project which was developed by Mission
Critical Linux, Inc.
Subsequent to its inception based on Kimberlite, developers at Red Hat have made a large number
of enhancements and modifications. The following is a non-comprehensive list highlighting some of
these enhancements.
• Packaging and integration into the Red Hat installation paradigm in order to simplify the end user’s
experience.
• Addition of support for high availability NFS services.
• Addition of support for high availability Samba services.
• Addition of support for using watchdog timers as a data integrity provision
• Addition of service monitoring which will automatically restart a failed application.
• Rewrite of the service manager to facilitate additional cluster-wide operations.
• Addition of the Red Hat Cluster Manager GUI, a graphical monitoring tool.
• A set of miscellaneous bug fixes.
The Red Hat Cluster Manager software incorporates STONITH compliant power switch modules
from the Linux-HA project http://www.linux-ha.org/stonith/.

Contents
Red Hat Cluster Manager
Acknowledgments........................................................................ iii
Chapter 1 Introduction to Red Hat Cluster Manager........... 7
1.1 Cluster Overview ................................................................. 7
1.2 Cluster Features.................................................................. 9
1.3 How To Use This Manual ........................................................ 12
Chapter 2 Hardware Installation and Operating System
Configuration............................................................. 13
2.1 Choosing a Hardware Configuration ........................................... 13
2.2 Steps for Setting Up the Cluster Systems ..................................... 30
2.3 Steps for Installing and Configuring the Red Hat Linux Distribution ........ 33
2.4 Steps for Setting Up and Connecting the Cluster Hardware................. 39
Chapter 3 Cluster Software Installation and Configuration 55
3.1 Steps for Installing and Initializing the Cluster Software...................... 55
3.2 Checking the Cluster Configuration............................................. 62
3.3 Configuring syslog Event Logging .............................................. 65
3.4 Using the cluadmin Utility........................................................ 67
Chapter 4 Service Configuration and Administration......... 73
4.1 Configuring a Service ............................................................ 73
4.2 Displaying a Service Configuration............................................. 77
4.3 Disabling a Service............................................................... 79
4.4 Enabling a Service ............................................................... 79
4.5 Modifying a Service .............................................................. 80
4.6 Relocating a Service ............................................................. 80
4.7 Deleting a Service................................................................ 81
4.8 Handling Services that Fail to Start............................................. 81
iv

Chapter 5 Database Services.................................................. 83
5.1 Setting Up an Oracle Service ................................................... 83
5.2 Tuning Oracle Services .......................................................... 91
5.3 Setting Up a MySQL Service.................................................... 92
5.4 Setting Up a DB2 Service ....................................................... 96
Chapter 6 Network File Sharing Services............................. 103
6.1 Setting Up an NFS Service...................................................... 103
6.2 Setting Up a High Availability Samba Service................................. 112
Chapter 7 Apache Services..................................................... 123
7.1 Setting Up an Apache Service .................................................. 123
Chapter 8 Cluster Administration .......................................... 129
8.1 Displaying Cluster and Service Status ......................................... 129
8.2 Starting and Stopping the Cluster Software ................................... 132
8.3 Removing a Cluster Member.................................................... 132
8.4 Modifying the Cluster Configuration ............................................ 133
8.5 Backing Up and Restoring the Cluster Database ............................. 133
8.6 Modifying Cluster Event Logging ............................................... 134
8.7 Updating the Cluster Software .................................................. 135
8.8 Reloading the Cluster Database................................................ 136
8.9 Changing the Cluster Name..................................................... 136
8.10 Reinitializing the Cluster ......................................................... 136
8.11 Disabling the Cluster Software.................................................. 137
8.12 Diagnosing and Correcting Problems in a Cluster ............................ 137
Chapter 9 Configuring and using the Red Hat Cluster
Manager GUI............................................................... 145
9.1 Setting up the JRE ............................................................... 145
9.2 Configuring Cluster Monitoring Parameters ................................... 146
9.3 Enabling the Web Server ........................................................ 147
9.4 Starting the Red Hat Cluster Manager GUI.................................... 147
v

Appendix A Supplementary Hardware Information ................ 151
A.1 Setting Up Power Switches...................................................... 151
A.2 SCSI Bus Configuration Requirements ........................................ 160
A.3 SCSI Bus Termination............................................................ 161
A.4 SCSI Bus Length ................................................................. 162
A.5 SCSI Identification Numbers .................................................... 162
A.6 Host Bus Adapter Features and Configuration Requirements............... 163
A.7 Tuning the Failover Interval...................................................... 168
Appendix B Supplementary Software Information ................. 171
B.1 Cluster Communication Mechanisms .......................................... 171
B.2 Cluster Daemons................................................................. 172
B.3 Failover and Recovery Scenarios............................................... 173
B.4 Cluster Database Fields ......................................................... 177
B.5 Using Red Hat Cluster Manager with Piranha................................. 179
vi

Section 1.1:Cluster Overview 7
1 Introduction to Red Hat Cluster
Manager
The Red Hat Cluster Manager is a collection of technologies working together to provide data in-
tegrity and the ability to maintain application availability in the event of a failure. Using redundant
hardware, shared disk storage, power management, and robust cluster communication and application
failover mechanisms, a cluster can meet the needs of the enterprise market.
Specially suited for database applications, network file servers, and World Wide Web (Web) servers
with dynamic content, a cluster can also be used in conjunction with the Piranha load balancing cluster
software, based on the Linux Virtual Server (LVS) project, to deploy a highly available e-commerce
site that has complete data integrity and application availability, in addition to load balancing capabil-
ities. See Section B.5, Using
Red Hat Cluster Manager with Piranha for more information.
1.1 Cluster Overview
To set up a cluster, an administrator must connect the cluster systems (often referred to as member
systems) to the cluster hardware, and configure the systems into the cluster environment. The foun-
dation of a cluster is an advanced host membership algorithm. This algorithm ensures that the cluster
maintains complete data integrity at all times by using the following methods of inter-node commu-
nication:
•Quorum partitions on shared disk storage to hold system status
• Ethernet and serial connections between the cluster systems for heartbeat channels
To make an application and data highly available in a cluster, the administrator must configure a clus-
ter service — a discrete group of service properties and resources, such as an application and shared
disk storage. A service can be assigned an IP address to provide transparent client access to the ser-
vice. For example, an administrator can set up a cluster service that provides clients with access to
highly-available database application data.
Both cluster systems can run any service and access the service data on shared disk storage. However,
each service can run on only one cluster system at a time, in order to maintain data integrity. Adminis-
trators can set up an active-active configuration in which both cluster systems run different services,
or a hot-standby configuration in which a primary cluster system runs all the services, and a backup
cluster system takes over only if the primary system fails.

8 Chapter 1:Introduction to Red Hat Cluster Manager
Figure 1–1 Example Cluster
Figure 1–1, Example Cluster shows an example of a cluster in an active-active configuration.
If a hardware or software failure occurs, the cluster will automatically restart the failed system’s ser-
vices on the functional cluster system. This service failover capability ensures that no data is lost,
and there is little disruption to users. When the failed system recovers, the cluster can re-balance the
services across the two systems.
In addition, a cluster administrator can cleanly stop the services running on a cluster system and then
restart them on the other system. This service relocation capability enables the administrator to main-
tain application and data availability when a cluster system requires maintenance.

Section 1.2:Cluster Features 9
1.2 Cluster Features
A cluster includes the following features:
• No-single-point-of-failure hardware configuration
Clusters can include a dual-controller RAID array, multiple network and serial communication
channels, and redundant uninterruptible power supply (UPS) systems to ensure that no single fail-
ure results in application down time or loss of data.
Alternately, a low-cost cluster can be set up to provide less availability than a no-single-point-of-
failure cluster. For example, an administrator can set up a cluster with a single-controller RAID
array and only a single heartbeat channel.
Note
Certain low-cost alternatives, such as software RAID and multi-initiator
parallel SCSI, are not compatible or appropriate for use on the shared
cluster storage. Refer to Section 2.1, Choosing a Hardware Configura-
tion, for more information.
• Service configuration framework
Clusters enable an administrator to easily configure individual services to make data and appli-
cations highly available. To create a service, an administrator specifies the resources used in the
service and properties for the service, including the service name, application start and stop script,
disk partitions, mount points, and the cluster system on which an administrator prefers to run the
service. After the administrator adds a service, the cluster enters the information into the cluster
database on shared storage, where it can be accessed by both cluster systems.
The cluster provides an easy-to-use framework for database applications. For example, a database
service serves highly-available data to a database application. The application running on a cluster
system provides network access to database client systems, such as Web servers. If the service
fails over to another cluster system, the application can still access the shared database data. A
network-accessible database service is usually assigned an IP address, which is failed over along
with the service to maintain transparent access for clients.
The cluster service framework can be easily extended to other applications, as well.
• Data integrity assurance
To ensure data integrity, only one cluster system can run a service and access service data at one
time. Using power switches in the cluster configuration enable each cluster system to power-cycle
the other cluster system before restarting its services during the failover process. This prevents

10 Chapter 1:Introduction to Red Hat Cluster Manager
the two systems from simultaneously accessing the same data and corrupting it. Although not
required, it is recommended that power switches are used to guarantee data integrity under all
failure conditions. Watchdog timers are an optional variety of power control to ensure correct
operation of service failover.
• Cluster administration user interface
A user interface simplifies cluster administration and enables an administrator to easily create,
start, stop, relocate services, and monitor the cluster.
• Multiple cluster communication methods
To monitor the health of the other cluster system, each cluster system monitors the health of the
remote power switch, if any, and issues heartbeat pings over network and serial channels to mon-
itor the health of the other cluster system. In addition, each cluster system periodically writes a
timestamp and cluster state information to two quorum partitions located on shared disk storage.
System state information includes whether the system is an active cluster member. Service state
information includes whether the service is running and which cluster system is running the ser-
vice. Each cluster system checks to ensure that the other system’s status is up to date.
To ensure correct cluster operation, if a system is unable to write to both quorum partitions at
startup time, it will not be allowed to join the cluster. In addition, if a cluster system is not updating
its timestamp, and if heartbeats to the system fail, the cluster system will be removed from the
cluster.