Presentation from IOUG Data Integration SIG meeting talking about Oracle GoldenGate Solutions and Oracle GoldenGate Uni-Directional

1. 1
Oracle GoldenGate 12c Solutions,
Uni-Directional Configuration
and
IOUG Data Integration SIG
Bobby Curtis, Accenture Enkitec Group
Joseph deBuzna, Oracle

2. 2
Agenda
• Oracle GoldenGate 12c Solutions
• Oracle GoldenGate 12c Uni-Directional
Configuration
• IOUG Data Integration Special Interest Group
(SIG)

3. 3
Legacy/Traditional Replication and ELT Tools
Tools:
1. Export/Import (Datapump/Traditional)
2. Advanced Replication
3. Streams
4. Warehouse Builder
Replication/Transforming data has been around in some form
for a really long time.
Other Venders Tools:
1. Dell Shareplex
2. SQL Server Transactional Replication/Sync Framework

4. 4
Present/Future trends of Replication and ELT
Tools
Question:
What product can help meet these trends?
• The need to always move/transform data will continue to
increase in the future.
• Requirements for real-time data/metrics will continue to
increase.
• More data sources will be required regardless of vendor
(i.e. Oracle, IBM, Microsoft, etc…)

5. 5
Present/Future trends of Replication and ELT
Tools
Question:
What product can help meet these trends?
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6 7 8 9 10
Data (GB)
Years

6. 6
Oracle GoldenGate Solutions
Unidirectional
Query Off-loading/Reporting
Application integration
Active-Active
Bidirectional with CDR
Complex Active Meshes
Consolidation
Data Warehouse
Single Source of Truth
Mergers & Acquisitions
Distribution
Data dissemination
Cascade replication
Hub & Spoke
Centralized sharing
Data distribution
Zero downtime migrations
High Availability

7. 7
Scenario
ACME company wants to build an online reporting system
that will provide guarantee access to data for a critical
business system.
Requirements:
1. Provide real-time data for reporting
2. Map data between different schemas
3. Maintain availability of data at remote site

8. 8
Uni-Directional Solution
One way replication between databases
Used for:
1. Query offloading/Reporting
2. Near-Zero Downtime Migrations
3. Application Integration
4. High-Availability Support
ACME.* ACME_RPT.*

9. 9
What is required?
Processes to ensure replication:
1. Manager
2. Extract (capture)
3. Data Pump (extract – shipping)
4. Replicat (apply)
ACME.* ACME_RPT.*

10. 10
Manager Process
Performs:
1. Starts/Restarts processes
2. Manage port numbers (static/dynamic)
3. Trail File Management
4. Create reports for events, errors and
thresholds
The manager process is the controller process within the
Oracle GoldenGate environment.

11. 11
Extract Process
Performs:
1. Reads online redo log/archive logs
(transactional logs)
2. Writes extracted transactional data to trail files
The extract process is used to perform change data capture
from the source database.

12. 12
Data Pump Process
Performs:
1. Protects against network failures
2. Consolidate data from multiple sources
3. Distribute data to multiple targets
4. Used for file adaptors (Flat File / Java)
The data pump process is a secondary extract that is used to
help send data over a network.

13. 13
Replicat Process
Performs:
1. Applies transactions in chronological order as trail files
are processed
2. Maps and transforms data
3. Performs Conflict Detection and Resolution
The replicat process is the apply process within the Oracle
GoldenGate environment.

14. 14
Trail Files
Performs:
1. Stores transactions in chronological order
2. Created by extract
3. Can be read by extract/replicat
Oracle GoldenGate binary files used to store transactions in
chronological order for shipping and applying to the target
side.

15. 15
GLOBALS File (Optional)
• File used store global parameters that relate to the Oracle GoldenGate
as a whole.
• All parameters in file apply to all processes for the instance
• Parameters in file can be overridden by process specific parameters
• Does not need to exist before starting processes rather is read by each
process as they are started
Example:
GGSCI> edit params ./GLOBALS

16. 16
Uni-Directional Breakdown
Capture
Trail
Files Pump
Delivery
Trail
Files
Capture (extract): committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. As of
V.11.2.1, GoldenGate offers two options for capture for Oracle; Classic & Integrated Capture
Trail: stages and queues data for routing.
Pump: distributes data for routing to target(s).
Route: data is compressed, encrypted for routing to target(s).
Delivery: applies data with transaction integrity. New with
GoldenGate 12c, Integrated Delivery (replicat).
MGRMGR

17. 17
Parameter Files
Parameter files is where the Oracle GoldenGate magic is
performed.
Editing parameter files can be done two ways:
1. From GoldenGate Service Command Interface (GGSCI)
2. Edited using a text editor (vi, textpad, etc..)
Example:
GGSCI> edit params <process name>

18. 18
Manager Process Parameter File
--Parameter files will vary per environment--
PORT 15000
DYNAMICPORTLIST 15010-15035
PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 2
AUTORESTART ER *, RETRIES 6, WAITMINUTES 2, RESETMINUTES 30
LAGCRITICALSECONDS 30
LAGREPORTMINUTES 60
LAGINFOMINUTES 0

19. 19
Extract Process Parameter File
--Parameter files will vary per environment--
-- CHECKPARAMS
EXTRACT EXTI
SETENV (ORACLE_HOME="/u01/app/oracle/product/11.2.0/db_3")
SETENV (ORACLE_SID="orcl")
USERID <ggate user>, PASSWORD <pwd>
TRANLOGOPTIONS DBLOGREADER
WARNLONGTRANS 1h, CHECKINTERVAL 30m
EXTTRAIL ./dirdat/lt
TABLE ACME.*;

20. 20
Data Pump Process Parameter
File
--Parameter files will vary per environment--
--CHECKPARAMS
EXTRACT PMPI
PASSTHRU
RMTHOST 10.16.10.10, MGRPORT 15000, COMPRESS
RMTTRAIL ./dirdat/rt
TABLE ACME.*;

21. 21
Replicat Process Parameter File
--Parameter files will vary per environment--
-- CHECKPARAMS
REPLICAT REPC
SETENV (ORACLE_HOME="/u01/app/oracle/product/11.2.0/db_3")
SETENV (ORACLE_SID="orcl")
USERID <ggate user>, PASSWORD <pwd>
ASSUMETARGETDEFS
DISCARDFILE ./dirrpt/REP.dsc, append, megabytes 500
MAP ACME.*, target ACME_RPT.*;

22. 22
GLOBALS File (Optional)
--Parameter files will vary per environment—
CHECKPOINTTABLE <ggate user>.checkpointtable
GGSCHEMA <ggate user>
ENABLEMONITORING –enables the Jagent for monitoring

23. 23
Adding processes
Add Extract
GGSCI> add extract exti, tranlog, begin now
GGSCI> add exttrail ./dirdat/lt, extract exti, megabytes 500
Add Data Pump
GGSCI> add extract pmpi, exttrailsource ./dirdat/lt
GGSCI> add rmttrail ./dirdat/rt, extract pmpi, megabytes 500
Add Replicat
GGSCI> dblogin userid <gguser> password <ggpwd>
GGSCI> add replicat repc, exttrail ./dirdat/rt
*Manager is added by default with installation

24. 24
Successful Setup
Capture
Trail
Files Pump
Delivery
Trail
Files
Source: Target:
MGRMGR

25. 25
IOUG Data Integration SIG

26. 26
IOUG Data Integration SIG
• Initially discussed at Oracle Open World 2014
• Focus/Products
• Resources
• Board of Directors

27. 27
IOUG Data Integration SIG
• Initially discussed at Oracle Open World 2014
• Popularity of Oracle GoldenGate and Oracle Data
Integrator is rising
• Launch of Oracle Metadata Management

28. 28
IOUG Data Integration SIG
Focus/Products:
IOUG DI SIG will focus on Oracle’s data
integration tools
• Oracle GoldenGate
• Oracle Data Integrator
• Oracle Metadata Management
• Many others…

29. 29
IOUG Data Integration SIG
Resources:
• Community Site (ioug.org or linkedin.com)
• Regular webcasts (quarterly)
• SIG meetings at Collaborate and Oracle
Open World
• Activities promoted on social media sites
• Sharing IOUG content with community

30. 30
IOUG Data Integration SIG
Current Board of Directors:
Bobby Curtis, AEG, @dbasolved, http://dbasolved.com
Seth Miller, Collier IT, @seth_m_miller, http://sethmiller.org
Danny Bryant, AEG, @dbcapoeira, http://dbaontap.org
*There are spots opened if you would like to participate

31. 31