HP Data Protector User manual

HP Data Protector Software
HP Data Protector and Oracle 10g RAC configuration best practices
Table of contents
Executive summary............................................................................................................................... 3
Overview............................................................................................................................................ 3
Target audience .................................................................................................................................. 3
Review of test configuration .................................................................................................................. 3
10gRAC configurations on hp-ux ....................................................................................................... 6
Oracle Virtual IP Address.................................................................................................................. 7
Oracle Services to manage workloads................................................................................................ 7
Oracle Services definition ................................................................................................................. 7
Oracle Online Backup Configuration ................................................................................................... 10
Configuration Guidelines ................................................................................................................ 10
Use the hostname and the Instance name to configure the backup ....................................................... 11
Use a VIP and the Instance names to configure the backup ................................................................. 12
Use a VIP and a service name to configure the backup....................................................................... 12
Use a VIP and allocate the channels to the instances manually ............................................................ 13
Storing the connection string in the Recovery Catalog......................................................................... 14
Backup Configuration Recommendations .......................................................................................... 16
SAN Backup versus LAN Backup......................................................................................................... 18
Disk Agent and Media Agent are NOT allocated on the same host...................................................... 18
Disk Agent and Media Agent are allocated on the same host.............................................................. 18
File System Backup: Oracle Base and Cluster configuration .................................................................... 19
Oracle Cluster Registry (OCR) ......................................................................................................... 20
Oracle Voting Disk......................................................................................................................... 20
Oracle SPFILE ................................................................................................................................ 21
ORACLE Directories ....................................................................................................................... 21
File system Backup Recommendations............................................................................................... 21
Oracle RMAN Restore and Recovery ................................................................................................... 22
Oracle Restore Enhancements.......................................................................................................... 22
Oracle RMAN Restore script............................................................................................................ 23
Oracle Restore Configuration .......................................................................................................... 24

Restore Recommendations Summary................................................................................................. 25
Recovery Considerations................................................................................................................. 25
Summary .......................................................................................................................................... 27
Appendix A. Reference documentation................................................................................................. 28
HP................................................................................................................................................ 28
Oracle.......................................................................................................................................... 28
Tools ............................................................................................................................................ 28
Appendix B. RMAN helpful Scripts ...................................................................................................... 29
B.1 How to monitor RMAN job progress .......................................................................................... 29
B.2 How to monitor RMAN waits..................................................................................................... 29
B.3 How to monitor RMAN wait states ............................................................................................. 30
B.4 How to monitor RMAN channels current speed............................................................................ 30
Appendix C. Session Reports .............................................................................................................. 31
Session 1: Backup using ita018 and RAC1....................................................................................... 31
Session 2: Backup using ita018-vip and RAC1, RAC2, RAC3, RAC4................................................... 35
Session 3: Backup using ita018-vip and service BACKUP ................................................................... 40
Session 4: Backup using ita018-vip and service MSL5000.................................................................. 45
Session 5: Backup allocating channels manually on RAC1 and RAC2.................................................. 49
Session 6: Restore allocating channels manually on RAC1 and RAC2 .................................................. 53
Session 7: Restore and Recovery using an incremental backup ............................................................ 56
For more information.......................................................................................................................... 60

3
Executive summary
This white paper provides guidance and best practices for performing backup and restore of Oracle
10g Real Application Clusters using HP Data Protector Software.
Since the release of Oracle 9i Real Application Clusters, there has always been uncertainty about
what needs to be backed up and how to backup the RAC database. We will try to address these
questions throughout this paper.
This paper is not:
•Intended to replace the HP Software Data Protector installation and integration guides with Oracle
•A performance whitepaper on backup and recovery of Oracle Real Application Clusters
•A step-by-step installation and configuration guide
Overview
HP Data Protector Software Oracle integration enables customers to perform online and offline
backups of an Oracle database by using the Oracle Recovery Manager and acting as a Media
Management Layer. In the past, the integration with an Oracle Real Application Cluster environment
was performed using just one of the instances of the cluster.
On the other side, Oracle’s Workload Management in a Cluster environment managed the workload
distribution, providing optimal performance. With Oracle 10g, the concept of Services was
introduced. It enables to group database workloads and distribute them over specific instances.
Integrating HP Software Data Protector with Oracle 10g Real Application Clusters, helps you benefit
from using the workload management functionality by distributing the backup and restore load over
more than one instance in the cluster.
Target audience
The target audience for this whitepaper are customers, system integrators and solution architects.
Basic knowledge of Data Protector and Oracle 10g RAC is advisable to understand this white paper
fully.
Review of test configuration
The following hardware configuration was used for the testing:
•5 HP Integrity Server rx2620 running HP-UX 11.23
•1 HP StorageWorks EVA5000
•1 SAN Tape Library MSL5000 with 2 SDLT320 drives
•2x 1000 Base-T per server
The following software configuration was used for the testing:
•HP Data Protector 6.0 Cell Manager running on HP-UX
•Oracle 10g RAC version 10.2.0.3 running on 4 nodes using Oracle clusterware
•Oracle 10g version 10.2.0.3 for the Oracle Recovery Catalog (RCVCAT)
The Oracle10g RAC database (RAC) was configured using 2 ASM disk groups:
•Disk Group DG_DB (database): 4 EVA LUN (Vdisk), each with 100 GB
•Disk Group DG_FRA (Flash Recovery Area): 4 EVA LUN (Vdisk), each with 25 GB

4
The following figure shows the environment described above:
Figure 1. Hardware architecture
The database had a size of 300GB and was created using the AXWORK suite. This suite created a
new tablespace called POOL, with 66 datafiles, each 4095MB in size. The following representation
shows the distribution of the datafiles. Note that they are all of equal size:
FILE_NAME MB
-------------------------------------------------- ----------
+DG_DB/rac/datafile/tbs_pool_001.dbf 4095
+DG_DB/rac/datafile/tbs_pool_002.dbf 4095
+DG_DB/rac/datafile/tbs_pool_003.dbf 4095
+DG_DB/rac/datafile/tbs_pool_004.dbf 4095
+DG_DB/rac/datafile/tbs_pool_005.dbf 4095
+DG_DB/rac/datafile/tbs_pool_006.dbf 4095
+DG_DB/rac/datafile/tbs_pool_007.dbf 4095
+DG_DB/rac/datafile/tbs_pool_008.dbf 4095
+DG_DB/rac/datafile/tbs_pool_064.dbf 4095
+DG_DB/rac/datafile/tbs_pool_065.dbf 4095
+DG_DB/rac/datafile/tbs_pool_066.dbf 4095

5
The following screenshot shows the graphical display shown by the Oracle Enterprise Manager
(Database Console):
Figure 2. Oracle Enterprise Manager: POOL datafiles view
Two extra LUN were allocated on the shared storage for the Oracle Cluster Registry (OCR) and
Oracle voting disks.
Below, a list of CRS resources defined in our cluster and their status:
oracle@ita018[RAC1]:/opt/oracle/product/CRS/bin$ ./crsstat.sh
HA Resource Target State
----------- ------ -----
ora.RAC.BACKUP.RAC1.srv ONLINE ONLINE on ita018
ora.RAC.BACKUP.RAC2.srv ONLINE ONLINE on ita021
ora.RAC.BACKUP.RAC3.srv ONLINE ONLINE on ita022
ora.RAC.BACKUP.RAC4.srv ONLINE ONLINE on ita023
ora.RAC.BACKUP.cs ONLINE ONLINE on ita018
ora.RAC.MSL5000.RAC1.srv ONLINE ONLINE on ita018
ora.RAC.MSL5000.RAC2.srv ONLINE ONLINE on ita021
ora.RAC.MSL5000.cs ONLINE ONLINE on ita018
ora.RAC.RAC1.inst ONLINE ONLINE on ita018
ora.RAC.RAC2.inst ONLINE ONLINE on ita021
ora.RAC.RAC3.inst ONLINE ONLINE on ita022
ora.RAC.RAC4.inst ONLINE ONLINE on ita023
ora.RAC.db ONLINE ONLINE on ita021
ora.ita018.ASM1.asm ONLINE ONLINE on ita018
ora.ita018.LISTENER_ITA018.lsnr ONLINE ONLINE on ita018
ora.ita018.gsd ONLINE ONLINE on ita018
ora.ita018.ons ONLINE ONLINE on ita018
ora.ita018.vip ONLINE ONLINE on ita018
ora.ita021.ASM2.asm ONLINE ONLINE on ita021
ora.ita021.LISTENER_ITA021.lsnr ONLINE ONLINE on ita021
ora.ita021.gsd ONLINE ONLINE on ita021
ora.ita021.ons ONLINE ONLINE on ita021
ora.ita021.vip ONLINE ONLINE on ita021
ora.ita022.ASM3.asm ONLINE ONLINE on ita022
ora.ita022.LISTENER_ITA022.lsnr ONLINE ONLINE on ita022
ora.ita022.gsd ONLINE ONLINE on ita022
ora.ita022.ons ONLINE ONLINE on ita022
ora.ita022.vip ONLINE ONLINE on ita022
ora.ita023.ASM4.asm ONLINE ONLINE on ita023
ora.ita023.LISTENER_ITA023.lsnr ONLINE ONLINE on ita023
ora.ita023.gsd ONLINE ONLINE on ita023
ora.ita023.ons ONLINE ONLINE on ita023
ora.ita023.vip ONLINE ONLINE on ita023

6
10gRAC configurations on hp-ux
Starting with Oracle10g RAC, there are different configurations possible for HP-UX; essentially, the
customer needs to decide about which clustering solution will be deployed:
•Service Guard Extension for RAC + Oracle Clusterware
•Oracle Clusterware only
Based on the storage volume manager choice, there are three main alternatives:
•Oracle Automatic Storage Management (ASM)
•HP Cluster File system (CFS)
•RAW devices
Important Note: RAW devices do NOT provide a shared location for storing archive logs (archived
logs cannot be located on RAW). In this case, an extra approach has to be addressed to have all
archived logs accessible by all RAC instances. This is needed for backup and recovery purposes. Two
possible approaches are to:
•Define an archivelog destination local on each node (i.e. /arch1 on RAC1, /arch2 on RAC2) and
export it and cross mount it on every system. In case one of the nodes is down, the archivelog
directory is also not available.
•Define a shared archivelog destination for all RAC nodes on an NFS share and use a clustering
solution as HP Service Guard to guarantee its availability.
Figure 3. Oracle 10gRAC on hp-ux supported configurations
SGeRAC &
Oracle
Clusterware
10g RAC
HP-UX
Oracle
Clusterware
SLVM CVM ASM
RAW ASM RAW CFS
SGeRAC &
Oracle
Clusterware
10g RAC
HP-UX
Oracle
Clusterware
SLVM CVM ASM
RAW ASM RAW CFS

7
Oracle Virtual IP Address
Since Oracle 10g, clients use a Virtual IP Address (VIP) to connect to the database. The VIP is a static
IP which is defined in the Oracle Clusterware installation.
When a RAC node fails, its associated VIP will failover automatically to one of the remaining nodes.
The new node hosting the failed VIP will associate a new MAC address for this VIP and will make it
known outside.
When clients are not using a VIP for their connections, in case of node failure, they run in a TCP
timeout. For availability and failover purposes, we have defined a VIP on each RAC node (ita018-vip,
ita021-vip, ita022-vip and ita023-vip). We will use the VIP to configure the integration, thus
providing Failover capabilities.
Oracle Services to manage workloads
Applications using an RAC database need to have their workload managed across the entire cluster.
For this purpose, Oracle introduced the Workload Management framework. Based on the use of
Services, it presents the application using the cluster an interface for managing their load in a very
simple way.
Oracle Services allow breaking down workloads in a manageable way, based on Service Level
Agreements, High Availability needs, or business needs and route them to the optimal instances
assigned. It is recommended to group load with the same service level. Load balancing among the
available instances is provided by Oracle Net Services.
When defining a new service, we can specify on which RAC instances the service should run on:
•Preferred: service should run on this instance
•Available: service should run on this instance if a preferred fails
•Not Used: the service should not use this instance
Transparent Application Failover (TAF) enables Oracle Net to failover to a different listener. Based on
the TAF method, the services can be defined as:
•None:
•Basic: connection to failover server will be established at failover time
•Preconnect: pre-established connections are allocated to provide faster failover
Oracle Services definition
To define a new service, you can use the command line interface, srvctl, or the database
configuration assistant, dbca:
$ srvctl add service –d RAC –s BACKUP –r RAC1,RAC2,RAC3,RAC4 –P PRECONNECT
In the above example, we add a new service called BACKUP in the database RAC. With the option –
r we specified the list of preferred instances and the option –P specifies the TAF policy.
The same can be achieved using the Database Configuration Assistant (dbca) GUI:

8
Figure 4. Database Configuration Assistant: Database Service BACKUP
The following figure is a graphical representation of the BACKUP service, which expands over all
available instances:
Figure 5. Oracle Service BACKUP
We define also a service called MSL5000, allowed to run on two of the four instances; we set two
instances to Preferred and two instances to Not Used.

9
Figure 6. Database Configuration Assistant: Database Service MSL5000
For the Oracle Service MSL5000, we can see below how only instances RAC1 and RAC2 are
available for this service:
Figure 7. Oracle Service MSL5000

10
The configuration can be verified using the following command:
oracle@ita018[RAC1]:/home/oracle$ srvctl status service -d RAC -s BACKUP,MSL5000
Service BACKUP is running on instance(s) RAC1, RAC4, RAC3, RAC2
Service MSL5000 is running on instance(s) RAC1, RAC2
Oracle Online Backup Configuration
This section discusses topics such as Backup Session Failover and Backup Load Balancing.
In a large customer environment, hundreds of Oracle Databases are backed up daily. Backup
sessions have to run independently from the status of some of the RAC instances. While it is critical to
have regular backups for the availability of the data, it is also crucial to backup the archive logs on a
daily basis to avoid the archive log destination to fill up
The idea of Session Failover is to be able to have the backup session running although the configured
instance is not available. Session failover applies only to the start of the backup session and not to an
already running session.
Backup Load Balancing distributes the Backup Load over more than one RAC node. As a result, more
than one node will read the datafiles and write the data to tape. This is also valid for a Restore
Session.
Configuration Guidelines
The following is a list of the main tasks needed for the Oracle integration. It is only for reference and
does not replace the Data Protector Oracle online integration manual.
1. Install the Data Protector Oracle agent on each and every RAC Node
2. Add a root account has to the Data Protector Admin User group in the Users configuration
menu for every RAC Node and every virtual hostname.1(1)
3. Add the oracle account to the Data Protector Admin User group in the Users configuration
menu for every RAC Node and every virtual hostname.
4. Link the Oracle SBT library (located under $ORACLE_HOME/lib) with the Data Protector SBT
library on every node as oracle user. It can be done online (the link process is described in
the Data Protector Oracle integration documentation). This is step is not required in Data
Protector 6.1.
5. Verity that the Oracle SQLNET connectivity works between each node and the RMAN
recovery catalog
6. Optional: register database RAC in the recovery catalog (this step is done through the
Integration). The integration needs a database account with SYSDBA privileges. You can use
the Oracle Database built-in SYS account or you may prefer to create a new account (i.e.
dp_integration) and grant it SYSDBA (NOTE: the grant has to be done on each node of the
cluster).
7. Import the Virtual IP Address (VIP) of every node in the RAC cluster; this will enable us to
configure the virtual IP as the client when doing the Integration.
1A new enhancement has been developed, which doesn’t require the root account but only the oracle account is required. Contact your support
organization to receive the needed patch.