1. Volume 19 | Number 3 w w w . i o u g . o r g
Third Quarter 2012
F o r t h e C o m p l e t e Te ch n o l o g y & D a t a b a s e P r o f e s s i o n a l
Security
Oracle Enterprise Manager 12c
Infrastructure and Operational Security
by Janet Wakeley
Resolving Child Cursor Issues Resulting
in Mutex Waits
by Martin Klier
Application Development with Oracle
Advanced Queuing
by Jeffrey Jacobs

2. Time for Change: Migrate your Benefits of Using Oracle RAC
Scalability: Oracle Real Application Cluster provides scalability for all your
Non-RAC Database to RAC
enterprise business applications. Oracle provides a wide array of tools and
techniques for scaling, and you can use these tools to ensure seamless growth
while minimizing the investment in hardware resources. You can allow RAC
database to grow seamlessly from a small system to a big multinational
enterprise applications.
High Availability: Oracle RAC provides reliability, recoverability, continuous
operations and error detection features for a highly available data management.
If an instance fails in Oracle RAC database, the cluster detects the problems
immediately and recovery will start automatically. The remaining instances in
the server remain active and open for uses.
Maintenance: In Oracle RAC, most of the database maintenance operations
By Satishbabu Gunukula can be performed without downtime, and many other maintenance tasks can
Edited by Gary Gordhamer be done in a rolling fashion (rolling patch, rolling upgrade) so application
downtime is minimized.
O
racle Real Application Cluster was introduced in Oracle How to Convert Single Instance to RAC
9i and it is now proven technology used by thousands Oracle provides the following methods to convert a single instance database to
RAC. You can choose any method based upon your convenience:
of customers in every industry for every type of
1. DBCA
application. Oracle RAC provides the options for scaling
2. RCONFIG (from 10g R2)
applications beyond the capabilities of a single server. This
3. Enterprise Manager
allows customers to take advantage of lower cost commodity 4. Manual (Using RMAN)
hardware to reduce their total cost of ownership and provide
a scalable computing environment that supports their
application workload. Learn about different approaches to
convert a non-RAC database to RAC database.
The Oracle RAC eliminates the ability to remove the server as a single point of
failure in any database application environment. It enables the Oracle Database
to run mainstream business applications of all kinds of server pools, including Figure 2
products such as SAP, PeopleSoft and Oracle Applications, which can be either
DSS, OLTP or mixed workload. We are assuming that the source single-instance database and the target RAC
database are the same release of Oracle and are running on same platform. If
the versions are different, use Oracle Database Upgrade Assistant (DBUA) to
upgrade the database and use one of the methods to convert a single instance
database to RAC.
Before you convert your single-instance database to RAC, make sure that you
back up your single-instance database and configure your shared storage so it
is accessible from all nodes in the RAC environment cluster.
The diagram change describes the difference between single-instance storage
and RAC storage.
Figure 1
continued on page 26
3rd Qtr 2012 ■ Page 25

3. Time for Change: Migrate your Non-RAC database to RAC continued from page 25
DBCA automates the database configuration during single instance to RAC
conversion and minimizes the manual intervention of tasks. DBCA performs
the below tasks automatically:
•• Creates undo and redo logs required for RAC database
•• Configures control file attributes
•• Creates the initialization parameter entries required for a cluster-enabled
environment
•• Configures Oracle Net Services and cluster resource
•• Configures Oracle RAC database to manage using Oracle Enterprise Manager
•• Configures the RAC database to manage by using SRVCTL utility
Before you convert single instance to RAC, you need to create an image of single
Figure 3: RAC Database Storage Principles instance using DBCA and then perform the actual conversion process using DBCA.
We have different shared storage options for Oracle RAC, but Automatic Storage Refer to the Oracle documentation via the link below for information on how
Management(ASM) is the recommended method. Using ASM has many benefits, to convert single instance to RAC using DBCA.
such as online storage migration, easy management and automatic I/O tuning. http://docs.oracle.com/cd/E11882_01/install.112/e18069/
Please find the available shared storage options in Oracle RAC cvrt2rac.htm#BABBBDDB
•• ASM – Automatic Storage Management
RCONFIG
•• RAW – Raw devices (only supported for older releases) RCONFIG is a noninteractive command line utility that comes default on
•• CFS – Vendor cluster file system (e.g., OCFS, QFS, GXFs) Oracle 10g and above versions. This utility uses the information in
ConvertToRAC.xml in order to convert single instance to RAC. You need to
•• NFS – Network file system (certified NAS)
prepare input XML with convert options according to your environment.
Refer to the Oracle documentation via the link below for information on
This utility has the ability to migrate non-ASM to ASM, and it internally uses
Automatic Storage Management (ASM).
RMAN to copy the data from non-ASM to ASM. This utility uses the RMAN
http://docs.oracle.com/cd/E11882_01/server.112/e18951/toc.htm default configuration when copying data and does not allow you to modify the
configuration parameters. As a result, RMAN uses one channel to copy the
Automatic Storage Management (ASM) Best Practices database, which means a longer downtime. If your database is very large, it
We have tested several best practices in our test tab, and we have seen very might not be a good option for you.
good results. Several of these results are mentioned in Oracle documentation/ If you see any errors in the middle of the conversion when using this method,
publications. you need to perform cleanup on the converted instance and reinitialize the
•• Install ASM on a separate Oracle Home for high availability and manageability. conversion process. In some cases, we have noticed the conversion fails to add
resources to CRS. At this stage, instead of redoing the whole process, you can
•• Use multipathing over multiple HBAs to provide I/O load balancing and add the resources to CRS manually using SRVCTL.
failover capability.
In one of our tests, we are able to convert a 400GB database in around five hours
•• Remove ASM dependency on VIP. In case VIP fails, the ASM instances using RCONFIG. We are able to convert the same database in around two-and-
remains operational. a-half hours using RMAN parallelism. When using RMAN, there are many
•• For better performance, use ASM diskgroups with four or more disks, and parameters to optimize the performance and more room for improvement.
make sure these disks span several back-end disk adapters. Below are the common issues when using RCONFIG to convert single instance
•• Use EXTERNAL redundancy if you are doing the mirroring at storage level. to RAC.
•• If you are using hardware RAID, make sure the LUN stripe size is as close 1. When using ASM as shared storage, makes sure to mention listed ASM
to 1MB as possible information in ConvertToRAC.xml file. Otherwise, RCONFIG will not be
able to detect ASM storage and conversion will fail.
•• Always use diskgroups with disks that are similar in size and performance.
A diskgroup with large number of disks provides a wide distribution of 2. For the actual conversion process, you must set the value of “convert
data extents. This allows greater concurrency for I/O and reduces the verify” to “YES” in ConvertToRAC.xml file or conversion will fail.
occurrences of hotspots. 3. If you don’t mention the valid disk group names in ConvertToRAC.xml
Let’s discuss the options that you have to convert single instance to RAC. file, the data file will be created in the $ORACLE_HOME/dbs local file
system, and you will receive an error message that the data file did not
DBCA share across all nodes in the cluster.
Most DBAs are familiar with DBCA, but before using DBCA for conversion, For example: Make sure you specify the valid diskgroup names in
make sure that system hardware and operating system are supported. ConvertToRAC.xml file
Page 26 ■ 3rd Qtr 2012

4. <n:SharedStorage type=”ASM”> This method includes two steps:
<n:TargetDatabaseArea>+DGROUP1</n:TargetDatabaseArea> 1. Duplicate single instance database into RAC system as single instance
<n: TargetFlashRecoveryArea>+DGOUP2</n:TargetFlashRecoveryArea>
2. Convert single instance to RAC
</n:SharedStorage>
In this method, you will be using the RMAN DUPLICATE DATABASE command
4. The host names in “ConvertToRAC.xml” must be in the same case as actual to duplicate your single-instance database to RAC as single instance. Once the
host names ($hostname) or the conversion process will fail with the following database is duplicated on RAC node, you will need to add redo and undo for
error: “The cluster is not configured or is not running on node.” second instance, enable all cluster-related parameters and register with CRS.
5. The SourceDBHome, TargetDBHome must be specified in ConvertToRAC.xml Below are the common issues when using RMAN to convert single instance
file or the conversion will fail with the following error message: to RAC:
oracle.sysman.assistants.util.sqlEngine.SQLFatalErrorException:
Cannot get SQLEngine 1. The user must create the password file for auxiliary database or you will
receive the following error: RMAN-04006: error from auxiliary database:
…….
ORA-01031: insufficient privileges
The error is:
2. Make sure you add the auxiliary database TNS entry on target database
java.io.IOException: /home/oracle/product/10.2.0/db_1/bin/sqlplus:
(non-RAC) or you will receive the following error: RMAN-04006: error
not found
from auxiliary database: ORA-12154: TNS:could not resolve the connect
Refer to the Oracle documentation via the link below to convert single identifier specified
instance to RAC using RCONFIG.
How to Back Up Oracle RAC Databases
http://docs.oracle.com/cd/E11882_01/install.112/e18069/cvrt2rac.
htm#BABGGEGJ RMAN is a command-line and Enterprise Manager-based tool and is included
with the Oracle server. Using RMAN for backup is the Oracle-preferred method
Enterprise Manager for efficiently backing and recovering Oracle RAC databases. RMAN
automatically detects the bock corruptions, and you can also validate backups
Oracle Enterprise Manager is a GUI-based database management tool with without actually restoring.
which you can perform most of your database management and admin tasks.
If you want to convert your single instance to RAC without much user RAC database backups are the same as single-instance backups; you can run
intervention, then this method is best for you. the backup and restore from any node in the cluster. The best practice is to use
a shared location accessible by all nodes.
Refer to the Oracle documentation via the link below to convert single
instance to RAC using Enterprise Manager. If you enable the auto backup of control file features, RMAN automatically
restores the SPFILE if required for instance recovery. It can also recover the
http://docs.oracle.com/cd/E11882_01/install.112/e18069/cvrt2rac. database even if the current control file, catalog and server parameter file
htm#BABBAAEH are lost.
Manual (Using RMAN) Here are few compelling reasons to adopt RMAN for database backups:
I would recommend using RMAN if your database is a very large database •• Trouble-free backup and recovery
(VLDB) and you want to have full control over the conversion process. It has
many features and parameters, and you can optimize the performance based
•• Corrupt block detection
upon your requirement. You can use RMAN to convert a database running on •• Archive log validation and management
file system to ASM. •• Block media recovery (BMR)
If you have more than one disk group and you want to place data files across •• Easily integrates with media managers
different disk groups, then you can use the SET NEWNAME command for each
data file in the duplication script and RMAN will restore the data file accordingly. •• Backup and restore optimization
•• Backup and restore validation
•• Downtime-free backups
•• Incremental backups
•• Extensive reporting
Refer to the Oracle documentation via the link below to configure Recovery
Manager for backup.
http://docs.oracle.com/cd/E11882_01/rac.112/e16795/rman.htm
Figure 4
continued on page 28
3rd Qtr 2012 ■ Page 27

5. Time for Change: Migrate your Non-RAC database to RAC continued from page 27
Best Practices, Tips and Hints •• Prior to 11g R2, it is recommended to run the listener from the ASM home.
From 11g R2 on, the recommended method is to run the listener from the
Below are several best practices, tips and hints that have been tested at our
grid infrastructure home (by default).
lab with very good results. A number of them are mentioned in Oracle
documentation/publications. •• To avoid name resolution issues, ensure that the HOSTS files and DNS are
furnished with both VIP and public host names.
•• In a RAC environment, avoid using a shared Oracle Home. If you install
software in a shared Oracle Home, then you cannot perform a rolling •• If using NFS disks for RAC, make sure that correct mount options are used.
upgrade of patches/sets. Also, binaries will have local dependencies and The mount options are detailed in Document 359515.1 for each platform.
single point of failure. •• It is highly recommended to utilize SCAN for your 11.2 databases, as it
•• Choose ASM as shared storage for automatic I/O tuning, easy management simplifies client connectivity and eliminates the need to modify database
and online storage migration (adding, removing disks). connect strings when the cluster grows and/or shrinks.
•• Oracle ASM supports both older (must be at 10.1.0.3 or higher) and newer •• Oracle highly recommends configuring a redundant set of public network
versions of Oracle Database. interface cards (NICs) and private interconnect NICs on each cluster node
to avoid public and private networks from being the single point of failure.
•• Before you set up a RAC environment, check with the disk vendor that the Dedicated redundant switches are highly recommended for the private
number of nodes, OS version, CRS version, RAC version, network fabric interconnect (crossover cables are not supported).
and patches are certified, as some storage/san vendors may require special
certification for a certain number of nodes. •• Do not add more than a 2TB size disk to a disk group. ORA-15196 (ASM
block corruption) can occur if LUNs larger than 2TB are presented to an
•• It is highly recommended to test patches and upgrades in a test environment ASM diskgroup. As a result of the fix, ORA-15099 will be raised if a disk
first before moving the changes to production. Also, the test environment larger than 2TB is specified. This is irrespective of the presence of asmlib.
should mirror the production environment as closely as possible.
•• It is strongly advised that a production RAC instance does not share a node Conclusion
with a DEV, QA, test or TRAINING instance. These extra instances can often Oracle Real Application Cluster allows the enterprise application to grow its
introduce unexpected performance changes into a production environment. business in any direction by providing protection from hardware, software
failures and ensures continuous data access. Oracle RAC is designed for
•• Always multiplex control files, redo logs and archive logs across disk
groups for redundancy. scalability and high availability. Many customers implemented Oracle RAC
for their mission-critical applications. If you haven’t migrated your single-
•• The VIPs and SCAN VIPs must be on the same subnet as the public interface. instance database to RAC yet, now is the time. Please refer to the links below
For additional information, see the “Understanding SCAN VIP” whitepaper. for additional information:
•• Prior to 11g, it is recommended to store vote and OCR on RAW devices. •• Please check the link for Oracle Database High availability Solutions
From 11g R2, ASM is the recommended method for storing VOTE and OCR. http://www.oracle.com/technetwork/database/features/availability/
It is recommended to maintain no more than two ASM disk groups for index.html
storing VOTE and OCR.
•• Having a step-by-step plan for your RAC project implementation is invaluable.
Please check OTN Article for sample project outline: http://www.oracle.
com/technetwork/articles/haskins-rac-project-guide-099429.html
•• Oracle Clusterware portion can always be installed in a rolling upgrade
USERS GROUP CALENDAR fashion: HOWTO: Note: 338706.1
For the most updated calendar, please visit www.ioug.org •• For detailed information on Single Client Access Name (SCAN), refer
Document 887522.1 Understanding SCAN VIP white paper.
SEPTEMBER 2012 DECEMber 2012 •• For recommended Patches check Metalink Note 756671.1
September 30–October 4 December 12 •• Refer to the Oracle documentation link for Oracle Real Application Cluster
Oracle OpenWorld NYOUG Special Winter Installation: http://docs.oracle.com/cd/E11882_01/install.112/
Moscone Center, San Francisco General Meeting e24660/toc.htm
Event URL: http://www.oracle.com/ New York Oracle Users Group
openworld/index.html 8:30 p.m. – 5 p.m.
New Yorker Hotel
October 2012 481 Eighth Ave. (34th St.) ■ ■ ■ About the Author
October 17-18 New York, NY 10001 Satishbabu Gunukula has more than 13 years of experience in
East Coast Oracle Users Conference Event URL: http://www.nyoug.org Oracle and SQL Server Database technologies. He specializes in
Sheraton Imperial Hotel & high-availability solutions (Oracle RAC, Data Guard, Grid Control,
Convention Center APRIL 2013
SQL Server Cluster) and has implemented many business-critical
Raleigh/Durham, NC April 7-11
systems for Fortune 500 and 1000 companies. Gunukula is an
Event URL: www.eastcoastoracle.org COLLABORATE ‘13
Denver, Colo. Oracle-certified DBA in 8i/9i/10g and an Oracle-certified expert in 10g
Event URL: http://collaborate13. RAC. He was awarded the Oracle ACE by Oracle Corporation. He shares
ioug.org/p/cm/ld/fid=112 knowledge through his websites http://www.oracleracexpert.com
C
and http://www.sqlserver-expert.com.
Page 28 ■ 3rd Qtr 2012