Red Hat ENTERPRISE LINUX 5 - VIRTUAL SERVER... User manual

Red Hat Enterprise Linux 5
Configuration Example
- NFS Over GFS
Configuring NFS over GFS in a Red Hat Cluster

Configuration Example - NFS Over GFS
Red Hat Enterprise Linux 5 Configuration Example - NFS Over
GFS
Configuring NFS over GFS in a Red Hat Cluster
Edition 3
Copyright © 2009 Red Hat Inc.. This material may only be distributed subject to the terms and
conditions set forth in the Open Publication License, V1.0 or later (the latest version of the OPL is
presently available at http://www.opencontent.org/openpub/).
Red Hat and the Red Hat "Shadow Man" logo are registered trademarks of Red Hat, Inc. in the United
States and other countries.
All other trademarks referenced herein are the property of their respective owners.
1801 Varsity Drive
Raleigh, NC 27606-2072 USA
Phone: [phone]
Phone: [phone]
Fax: [phone]
PO Box 13588 Research Triangle Park, NC 27709 USA
This book describes a procedure for configuring NFS over GFS in a 5-node Red Hat Cluster using
Conga.

iii
Introduction v
1. About This Guide ............................................................................................................ v
2. Audience ........................................................................................................................ v
3. Software Versions ........................................................................................................... v
4. Related Documentation ................................................................................................... v
5. Feedback ....................................................................................................................... vi
6. Document Conventions ................................................................................................... vi
6.1. Typographic Conventions ..................................................................................... vi
6.2. Pull-quote Conventions ....................................................................................... viii
6.3. Notes and Warnings ............................................................................................ ix
1. NFS over GFS in a Red Hat Cluster 1
2. Prerequisite Configuration 3
3. Components to Configure 5
4. Configuring the Cluster Resources 7
4.1. Configuring an IP Address Resource ............................................................................. 7
4.2. Configuring a GFS Resource ........................................................................................ 7
4.3. Configuring an NFS Export Resource ............................................................................ 8
4.4. Configuring NFS Client Resources ................................................................................ 8
5. Configuring an NFS Cluster Service 11
5.1. Add a Service to the Cluster ....................................................................................... 11
5.2. Adding an IP Address Resource to an NFS Service ..................................................... 12
5.3. Adding a GFS Resource to an NFS Service ................................................................ 12
5.4. Adding an NFS Export Resource to an NFS Service .................................................... 13
5.5. Adding NFS Client Resources to an NFS Service ......................................................... 13
6. Testing the NFS Cluster Service 15
7. Troubleshooting 17
8. The Cluster Configuration File 21
9. Configuration Considerations 25
A. Revision History 27
Index 29

iv

v
Introduction
1. About This Guide
This book describes a procedure for configuring NFS over GFS in a 5-node Red Hat Cluster using
Conga.
2. Audience
This book is intended to be used by system administrators managing systems running the Linux
operating system. It requires familiarity with Red Hat Enterprise Linux 5, Red Hat Cluster Suite, GFS
file system administration, and basic knowledge of NFS administration.
3. Software Versions
Software Description
RHEL5 refers to RHEL5 and higher
GFS refers to GFS for RHEL5 and higher
Table 1. Software Versions
4. Related Documentation
For more information about using Red Hat Enterprise Linux, refer to the following resources:
•Red Hat Enterprise Linux Installation Guide — Provides information regarding installation of Red
Hat Enterprise Linux 5.
•Red Hat Enterprise Linux Deployment Guide — Provides information regarding the deployment,
configuration and administration of Red Hat Enterprise Linux 5.
For more information about Red Hat Cluster Suite for Red Hat Enterprise Linux 5, refer to the following
resources:
•Red Hat Cluster Suite Overview — Provides a high level overview of the Red Hat Cluster Suite.
•Configuring and Managing a Red Hat Cluster — Provides information about installing, configuring
and managing Red Hat Cluster components.
•LVM Administrator's Guide: Configuration and Administration — Provides a description of the
Logical Volume Manager (LVM), including information on running LVM in a clustered environment.
•Global File System: Configuration and Administration — Provides information about installing,
configuring, and maintaining Red Hat GFS (Red Hat Global File System).
•Global File System 2: Configuration and Administration — Provides information about installing,
configuring, and maintaining Red Hat GFS2 (Red Hat Global File System 2).
•Using Device-Mapper Multipath — Provides information about using the Device-Mapper Multipath
feature of Red Hat Enterprise Linux 5.

Introduction
vi
•Using GNBD with Global File System — Provides an overview on using Global Network Block
Device (GNBD) with Red Hat GFS.
•Linux Virtual Server Administration — Provides information on configuring high-performance
systems and services with the Linux Virtual Server (LVS).
•Red Hat Cluster Suite Release Notes — Provides information about the current release of Red Hat
Cluster Suite.
Red Hat Cluster Suite documentation and other Red Hat documents are available in HTML,
PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at http://
www.redhat.com/docs/.
5. Feedback
If you spot a typo, or if you have thought of a way to make this manual better, we would love to
hear from you. Please submit a report in Bugzilla (http://bugzilla.redhat.com/bugzilla/) against the
component rh-cs.
Be sure to mention the manual's identifier:
Bugzilla component: Documentation-cluster
Book identifier: Cluster_Config_Example_NFS_GFS(EN)-5 (2009-01-05T15:20)
By mentioning this manual's identifier, we know exactly which version of the guide you have.
If you have a suggestion for improving the documentation, try to be as specific as possible. If you have
found an error, please include the section number and some of the surrounding text so we can find it
easily.
6. Document Conventions
This manual uses several conventions to highlight certain words and phrases and draw attention to
specific pieces of information.
In PDF and paper editions, this manual uses typefaces drawn from the Liberation Fonts1 set. The
Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not,
alternative but equivalent typefaces are displayed. Note: Red Hat Enterprise Linux 5 and later includes
the Liberation Fonts set by default.
6.1. Typographic Conventions
Four typographic conventions are used to call attention to specific words and phrases. These
conventions, and the circumstances they apply to, are as follows.
Mono-spaced Bold
Used to highlight system input, including shell commands, file names and paths. Also used to highlight
key caps and key-combinations. For example:
1 https://fedorahosted.org/liberation-fonts/

Typographic Conventions
vii
To see the contents of the file my_next_bestselling_novel in your current
working directory, enter the cat my_next_bestselling_novel command at the
shell prompt and press Enter to execute the command.
The above includes a file name, a shell command and a key cap, all presented in Mono-spaced Bold
and all distinguishable thanks to context.
Key-combinations can be distinguished from key caps by the hyphen connecting each part of a key-
combination. For example:
Press Enter to execute the command.
Press Ctrl+Alt+F1 to switch to the first virtual terminal. Press Ctrl+Alt+F7 to
return to your X-Windows session.
The first sentence highlights the particular key cap to press. The second highlights two sets of three
key caps, each set pressed simultaneously.
If source code is discussed, class names, methods, functions, variable names and returned values
mentioned within a paragraph will be presented as above, in Mono-spaced Bold. For example:
File-related classes include filesystem for file systems, file for files, and dir for
directories. Each class has its own associated set of permissions.
Proportional Bold
This denotes words or phrases encountered on a system, including application names; dialogue
box text; labelled buttons; check-box and radio button labels; menu titles and sub-menu titles. For
example:
Choose System > Preferences > Mouse from the main menu bar to launch Mouse
Preferences. In the Buttons tab, click the Left-handed mouse check box and click
Close to switch the primary mouse button from the left to the right (making the mouse
suitable for use in the left hand).
To insert a special character into a gedit file, choose Applications > Accessories
> Character Map from the main menu bar. Next, choose Search > Find… from the
Character Map menu bar, type the name of the character in the Search field and
click Next. The character you sought will be highlighted in the Character Table.
Double-click this highlighted character to place it in the Text to copy field and then
click the Copy button. Now switch back to your document and choose Edit > Paste
from the gedit menu bar.
The above text includes application names; system-wide menu names and items; application-specific
menu names; and buttons and text found within a GUI interface, all presented in Proportional Bold and
all distinguishable by context.
Note the > shorthand used to indicate traversal through a menu and its sub-menus. This is to avoid
the difficult-to-follow 'Select Mouse from the Preferences sub-menu in the System menu of the main
menu bar' approach.
Mono-spaced Bold Italic or Proportional Bold Italic
Whether Mono-spaced Bold or Proportional Bold, the addition of Italics indicates replaceable or
variable text. Italics denotes text you do not input literally or displayed text that changes depending on
circumstance. For example:

Introduction
viii
To connect to a remote machine using ssh, type ssh [e-mail] at
a shell prompt. If the remote machine is example.com and your username on that
machine is john, type ssh [email protected].
The mount -o remount file-system command remounts the named file
system. For example, to remount the /home file system, the command is mount -o
remount /home.
To see the version of a currently installed package, use the rpm -q package
command. It will return a result as follows: package-version-release.
Note the words in bold italics above — username, domain.name, file-system, package, version and
release. Each word is a placeholder, either for text you enter when issuing a command or for text
displayed by the system.
Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and
important term. For example:
When the Apache HTTP Server accepts requests, it dispatches child processes
or threads to handle them. This group of child processes or threads is known as
a server-pool. Under Apache HTTP Server 2.0, the responsibility for creating and
maintaining these server-pools has been abstracted to a group of modules called
Multi-Processing Modules (MPMs). Unlike other modules, only one module from the
MPM group can be loaded by the Apache HTTP Server.
6.2. Pull-quote Conventions
Two, commonly multi-line, data types are set off visually from the surrounding text.
Output sent to a terminal is set in Mono-spaced Roman and presented thus:
books Desktop documentation drafts mss photos stuff svn
books_tests Desktop1 downloads images notes scripts svgs
Source-code listings are also set in Mono-spaced Roman but are presented and highlighted as
follows:
package org.jboss.book.jca.ex1;
import javax.naming.InitialContext;
public class ExClient
{
public static void main(String args[])
throws Exception
{
InitialContext iniCtx = new InitialContext();
Object ref = iniCtx.lookup("EchoBean");
EchoHome home = (EchoHome) ref;
Echo echo = home.create();

Notes and Warnings
ix
System.out.println("Created Echo");
System.out.println("Echo.echo('Hello') = " + echo.echo("Hello"));
}
}
6.3. Notes and Warnings
Finally, we use three visual styles to draw attention to information that might otherwise be overlooked.
Note
A note is a tip or shortcut or alternative approach to the task at hand. Ignoring a note
should have no negative consequences, but you might miss out on a trick that makes your
life easier.
Important
Important boxes detail things that are easily missed: configuration changes that only
apply to the current session, or services that need restarting before an update will apply.
Ignoring Important boxes won't cause data loss but may cause irritation and frustration.
Warning
A Warning should not be ignored. Ignoring warnings will most likely cause data loss.

x

Chapter 1.
1
NFS over GFS in a Red Hat Cluster
This document provides a procedure to configure an NFS service in a Red Hat cluster using the
Conga configuration tool. The configured cluster will have the following characteristics:
• There are 5 nodes in the cluster.
• The NFS service runs over a GFS file system.
• There are five NFS clients.
Note that this configuration is not a "high capacity" configuration in the sense that more than one
server is providing NFS service. In this configuration, the floating IP moves about as needed, but only
one server is active at a time.
Figure 1.1, “NFS over GFS in a 5-Node Cluster” shows the NFS over GFS configuration that this
procedure yields.
Figure 1.1. NFS over GFS in a 5-Node Cluster
In this configuration, Cluster Suite assigns which server will run the NFS service. If that server goes
down, the NFS service will automatically fail over to another server in the cluster. The client will not be
aware of any loss of service.
The NFS resource that this configuration defines will be NFS Version 3 by default. If you need to
restrict what NFS protocol your system provides to its clients, you can do this at NFS startup; this is
not part of cluster configuration.
This remainder of this document is organized as follows:

Chapter 1. NFS over GFS in a Red Hat Cluster
2
•Chapter 2, Prerequisite Configuration describes the prequisite configuration components that have
been set up before the procedure documented in this manual beings.
•Chapter 3, Components to Configure summarizes the cluster resources that this procedure
configures.
•Chapter 4, Configuring the Cluster Resources provides the procedures for configuring the cluster
resources needed for an NFS service.
•Chapter 5, Configuring an NFS Cluster Service provides the procedure for configuring an NFS
service in a Red Hat Cluster Suite.
•Chapter 6, Testing the NFS Cluster Service provides a procedure to check that the NFS service is
working and that it will continue to work as expected if one of the nodes goes down.
•Chapter 7, Troubleshooting provides some guidelines to follow when your configuration does not
behave as expected.
•Chapter 8, The Cluster Configuration File shows the cluster configuration file as it appears before
configuring the NFS service and after configuration the NFS service in a Red Hat Cluster Suite.
•Chapter 9, Configuration Considerations summarizes some general concerns to consider when
configuring an NFS service over a GFS file system in a Red Hat Cluster Suite.

Chapter 2.
3
Prerequisite Configuration
This document describes a procedure for configuring cluster resources and a cluster service. Before
beginning this procedure, you must configure a cluster with a fencing device configured for each node
in the cluster. In addition, you must configure a logical volume with a GFS filesystem that can be
shared among the nodes in the cluster.
Table 2.1, “Configuration Prerequisities” summarizes the prequisite configuration components that
have been set up before this procedure beings.
Component Name Comment
cluster nfsclust five-node cluster
cluster node clusternode1.example.com node in cluster nfsclust configured with a fencing device
of nfs-apc
cluster node clusternode2.example.com node in cluster nfsclust configured with a fencing device
of nfs-apc
cluster node clusternode3.example.com node in cluster nfsclust configured with a fencing device
of nfs-apc
cluster node clusternode4.example.com node in cluster nfsclust configured with a fencing device
of nfs-apc
cluster node clusternode5.example.com node in cluster nfsclust configured with a fencing device
of nfs-apc
LVM
volume
/dev/myvg/myvol The LVM device on which the GFS file system is created
GFS file
system
The GFS file system to export by means of NFS, built on
LVM volume /dev/myvg/myvol, mounted at /mnt/gfs,
and shared among the members of cluster nfsclust
IP address 10.15.86.96 The IP address for the NFS service
NFS Client nfsclient1.example.com System that will access the NFS service
NFS Client nfsclient2.example.com System that will access the NFS service
NFS Client nfsclient3.example.com System that will access the NFS service
NFS Client nfsclient4.example.com System that will access the NFS service
NFS Client nfsclient5.example.com System that will access the NFS service
Table 2.1. Configuration Prerequisities
No failover domain has been defined for this cluster. The cluster software determines on which node
to run the NFS service.
For information on configuring a cluster, see Configuring and Managing a Red Hat Cluster. For
information on configuring an LVM volume, see LVM Administrator's Guide: Configuration and
Administration.

4

Chapter 3.
5
Components to Configure
This procedure documents the configuration of cluster resources and an NFS cluster service that will
be named nfssvc.
Table 3.1, “Cluster Resources to Configure” summarizes the cluster resources that this procedure
configures.
Resource
Type
Resource
Name
Description
IP Address 10.15.86.96 The IP address for the NFS service
GFS mygfs The GFS file system that will be exported through the NFS service
NFS Export mynfs
NFS Client nfsclient1 The NFS client system nfsclient1.example.com
NFS Client nfsclient2 The NFS client system nfsclient2.example.com
NFS Client nfsclient3 The NFS client system nfsclient3.example.com
NFS Client nfsclient4 The NFS client system nfsclient4.example.com
NFS Client nfsclient5 The NFS client system nfsclient5.example.com
Table 3.1. Cluster Resources to Configure
The procedure for configuring these resources and the specific parameters to configure are described
in Chapter 4, Configuring the Cluster Resources.
You configure a Cluster service by adding a service and defining the composition of the service,
which consists of the resources that the service requires. In this procedure, you will create an NFS
cluster service named nfssvc. Table 3.2, “Parameters to Configure for NFS Cluster Service nfssvc”
summarizes the resource configuration of nfssvc. The names of the resources are those that you
assign when you define them, as noted in Table 3.1, “Cluster Resources to Configure”
Resource Name Comment
IP Address Resource
GFS Resource mygfs
NFS Export Resource mynfs NFS Export resource mynfs is a child of GFS resource
mygfs.
NFS Client Resource nfsclient1 NFS Client resource nfsclient1 is a child of NFS Export
resource mynfs.
NFS Client Resource nfsclient2 NFS Client resource nfsclient2 is a child of NFS Export
resource mynfs.
NFS Client Resource nfsclient3 NFS Client resource nfsclient3 is a child of NFS Export
resource mynfs.
NFS Client Resource nfsclient4 NFS Client resource nfsclient4 is a child of NFS Export
resource mynfs.
NFS Client Resource nfsclient5 NFS Client resource nfsclient5 is a child of NFS Export
resource mynfs.
Table 3.2. Parameters to Configure for NFS Cluster Service nfssvc

Chapter 3. Components to Configure
6
The procedure for configuring the NFS service is described in Chapter 5, Configuring an NFS Cluster
Service.

Chapter 4.
7
Configuring the Cluster Resources
This chapter provides the procedures for configuring the cluster resources needed for an NFS service.
For the procedure for configuring the NFS service itself, see Chapter 5, Configuring an NFS Cluster
Service.
Before you can configure an NFS service for a GFS file system in a cluster, you must add the following
resources to your cluster:
• The IP address for the NFS service, as described in Section 4.1, “Configuring an IP Address
Resource”.
• The GFS file system, as described in Section 4.2, “Configuring a GFS Resource”.
• The NFS export, as described in Section 4.3, “Configuring an NFS Export Resource”.
• The NFS clients, as described in Section 4.4, “Configuring NFS Client Resources”.
To add a resource to your cluster using Conga, perform the following procedure:
1. As an administrator of luci Select the cluster tab.
2. From the Choose a cluster to administer screen, select the cluster to which you will add
resources. In this example, that is the cluster with the name nfsclust.
3. At the menu for cluster nfsclust (below the clusters menu), click Resources. This causes the
display of menu items for resource configuration: Add a Resource and Configure a Resource.
4. Click Add a Resource. Clicking Add a Resource causes the Add a Resource page to be
displayed.
The following sections provide instructions for adding the resources you need for an NFS service.
4.1. Configuring an IP Address Resource
Use the following procedure to add the IP Address resource 10.15.86.96 to cluster nfsclust.
1. At the Add a Resource page for cluster nfsclust, click the drop-down box under Select a
Resource Type and select IP Address
2. For IP Address, enter 10.15.86.96.
3. Leave the Monitor Link checkbox selected to enable link status monitoring of the IP address
resource.
4. Click Submit. Clicking Submit displays a verification page. Verifying that you want to add this
resource displays a progress page followed by the display of Resources page, which displays the
resources that have been configured for the cluster.
4.2. Configuring a GFS Resource
Use the following procedure to add the GFS file system resource mygfs to cluster nfsclust.
1. At the Add a Resource page for cluster nfsclust, click the drop-down box under Select a
Resource Type and select GFS file system

Chapter 4. Configuring the Cluster Resources
8
2. For Name, enter mygfs.
3. For Mount point, enter /mnt/gfs. This is the path to which the GFS file system is mounted.
4. For Device, enter /dev/myvg/myvol. The is the LVM logical volume on which the GFS file
system was is created.
5. The Options field specifies the mount options for the GFS file system. For this example, we are
mounting the filesystem with the rw (read-write) option.
6. Leave the File System ID field blank. Leaving the field blank causes a file system ID to be
assigned automatically after you click Submit at the File System Resource Configuration dialog
box.
7. Leave the Force Unmount checkbox unchecked. Force Unmount kills all processes using the
mount point to free up the mount when it tries to unmount. With GFS resources, the mount point is
not unmounted at service tear-down unless this box is checked.
8. Click Submit and accept the verification screen.
4.3. Configuring an NFS Export Resource
Use the following procedure to add NFS export resource mynfs to cluster mynfs.
1. At the Add a Resource page for cluster mynfs, click the drop-down box under Select a
Resource Type and select NFS Export
2. For Name, enter mynfs.
3. Click Submit and accept the verification screen.
The NFS Eport resource that this configuration defines will be NFS Version 3 by default. If you need
to restrict what NFS protocol your system provides to its clients, you can do this at NFS startup; this is
not part of cluster configuration.
4.4. Configuring NFS Client Resources
This example procedure configures five NFS client resources for cluster mynfs. The procedure for
configuring the first two clients only is laid out explicitly.
Use the following procedure to add NFS client resource nfsclient1 to cluster mynfs.
1. At the Add a Resource page for cluster mynfs, click the drop-down box under Select a
Resource Type and select NFS client
2. For Name, enter nfsclient1.
3. For Target, enter nfsclient1.example.com. This is the first NFS client system.
4. This Options field species additional client access rights. Specify rw (read-write) in this field. For
more information, refer to the General Options section of the exports(5) man page.
5. Check the Allow Recover checkbox. This indicates that if someone removes the export from the
export list, the system will recover the export inline without taking down the NFS service.

Configuring NFS Client Resources
9
6. Click Submit and accept the verification screen.
Use the following procedure to add NFS client resource nfsclient2 to cluster mynfs.
1. At the Add a Resource page for cluster mynfs, click the drop-down box under Select a
Resource Type and select NFS client
2. For Name, enter nfsclient2.
3. For Target, enter nfsclient2.example.com. This is the second NFS client system.
4. Leave the Options field blank.
5. Check the Allow Recover checkbox.
6. Click Submit and accept the verification screen.
Use the same procedure to configure the remaining three NFS client resources, using
nfsclient3, nfsclient4, and nfsclient5 as the names of the resources and using
nfsclient3.example.com, nfsclient4.example.com, and nfsclient5.example.com as
the targets.

10