This example illustrates how to backup and recover a 2-node, 10gR2 RAC cluster on
Linux. Backup and recovery operations for RAC databases are similar to that of single
instance databases, however there are some key differences – the goal of this post is to
highlight the steps required for recoverying RAC databases.
Backup the Database
Use the method you prefer. I used a NOCATALOG, disk-based backup for this example,
using the scripts found here.
Simulate the Failure
Shutdown the database and simulate a failure by dropping and/or corrupting a datafile.
# export ORACLE_SID=em1
# . oraenv
# srvctl stop database -d em
# srvctl status database -d em
Instance em1 is not running on node rac1
Instance em2 is not running on node rac2
# cd /u02/oracle/ora1/em
# rm sysaux01.dbf
Verify the Failure
# srvctl start instance -d em -i em1
PRKP-1001 : Error starting instance em1 on node rac1
CRS-0215: Could not start resource ‘ora.em.em1.inst’.
You will find something similar to the following in the instance’s alert log:
Sat May 24 16:53:47 2008
Errors in file /u02/oracle/logs/em/bdump/em1_dbw0_16947.trc:
ORA-01157: cannot identify/lock data file 3 – see DBWR trace file
ORA-01110: data file 3: ‘/u02/oracle/ora1/em/sysaux01.dbf’
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3
ORA-1157 signalled during: ALTER DATABASE OPEN…
Sat May 24 16:53:49 2008
Shutting down instance (abort)
Restore the Database
First, take the database out of cluster mode via:
# sqlplus “/ as sysdba”
SQL*Plus: Release 10.2.0.3.0 – Production on Sat May 24 17:02:17 2008
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
Connected to an idle instance.
SQL> startup nomount;
ORACLE instance started.
Total System Global Area 268435456 bytes
Fixed Size 1261300 bytes
Variable Size 251658508 bytes
Database Buffers 8388608 bytes
Redo Buffers 7127040 bytes
SQL> alter system set cluster_database=false scope=spfile sid=’*’;
System altered.
SQL> shutdown immediate;
ORA-01507: database not mounted
ORACLE instance shut down.
Then restore the database via RMAN:
# rman target=/
Recovery Manager: Release 10.2.0.3.0 – Production on Sat May 24
17:04:10 2008
Copyright (c) 1982, 2005, Oracle. All rights reserved.
connected to target database (not started)
RMAN> startup mount;
Oracle instance started
database mounted
Total System Global Area 268435456 bytes
Fixed Size 1261300 bytes
Variable Size 251658508 bytes
Database Buffers 8388608 bytes
Redo Buffers 7127040 bytes
RMAN> restore database;
Starting restore at 24-MAY-08
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=154 devtype=DISK
channel ORA_DISK_1: starting datafile backupset restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
restoring datafile 00001 to /u02/oracle/ora1/em/system01.dbf
restoring datafile 00002 to /u02/oracle/ora1/em/undotbs01.dbf
restoring datafile 00003 to /u02/oracle/ora1/em/sysaux01.dbf
restoring datafile 00004 to /u02/oracle/ora1/em/users01.dbf
restoring datafile 00005 to /u02/oracle/ora1/em/example01.dbf
restoring datafile 00006 to /u02/oracle/ora1/em/undotbs02.dbf
restoring datafile 00007 to /u02/oracle/ora2/em/mgmt.dbf
restoring datafile 00008 to /u02/oracle/ora1/em/mgmt_ecm_depot1.dbf
channel ORA_DISK_1: reading from backup piece
/u02/oracle/ora3/em/backups/0ijh6j4t_1_1
channel ORA_DISK_1: restored backup piece 1
piece handle=/u02/oracle/ora3/em/backups/0ijh6j4t_1_1
tag=FHB_EM1200805241630
channel ORA_DISK_1: restore complete, elapsed time: 00:06:36
Finished restore at 24-MAY-08
Since my database is so small, I will simply restore the entire thing, however, since you
know which datafile is missing, you could simply restore it and then recover the database
as necessary.
Recover the Database
RMAN> recover database;
Starting recover at 24-MAY-08
using channel ORA_DISK_1
starting media recovery
media recovery complete, elapsed time: 00:00:03
Finished recover at 24-MAY-08
RMAN> alter database open;
database opened
RMAN-06900: WARNING: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT
row
RMAN-06901: WARNING: disabling update of the V$RMAN_STATUS and
V$RMAN_OUTPUT rows
ORACLE error from target database:
ORA-06502: PL/SQL: numeric or value error: character to number
conversion error
Afterwards, place the database back into cluster mode and startup both instances:
# sqlplus “/ as sysdba”
SQL*Plus: Release 10.2.0.3.0 – Production on Sat May 24 17:16:36 2008
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 – Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining
options
SQL> alter system set cluster_database=true scope=spfile sid=’*’;
System altered.
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
# srvctl start database -d em
[[email protected] bdump]$ srvctl status database -d em
Instance em1 is running on node rac1
Instance em2 is running on node rac2
