Presentation given at UKOUG_TECH17 in Birmingham, England

2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Oracle GoldenGate 12.3:
Conflict, Detection and Resolution
Bobby Curtis, EMBA
Senior Principal Product Manager - Core
Oracle GoldenGate

3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 3
Speaker
@dbasolved
https://dbasolved.com
bobby.curtis@oracle.com
https://www.linkedin.com/in/bobbycurtis294/
https://www.slideshare.net/BobbyCurtisMBA

4. 12/15/2017 4Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
 Oracle GoldenGate 12c Overview
 Oracle GoldenGate 12c Architecture
 Conflict Management
 Requirements
 New Feature
 Examples
 Monitoring

5. Oracle GoldenGate
Real time Performance
Extensible & Flexible
Proven & Reliable
Oracle GoldenGate provides low-impact capture, routing,
transformation, and delivery of database transactions
across homogeneous and heterogeneous environments in
real-time with no distance limitations.
Most
Databases
Data
Events
Transaction Streams
Cloud
DBs
Big
Data
Supports Databases, Big Data and NoSQL:
* The most popular enterprise integration tool in history

6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Where does GoldenGate fit?
Business
Data
Serving
Layer
Apps
Analytics
EDWs
Batch Layer
Data Streams
Social and Logs
Enterprise Data
Highly Available
Databases
Analytics
Speed Layer
Pub / Sub
REST APIs
NoSQL
Bulk Data
Raw Data
Layer
GoldenGate
Data Integrator
Stream Analytics
Event Hub Big Data Database
Metadata Management (for Data Governance)
Big Data SQL
Oracle Software can help customers Accelerate & Reduce Risk around adoption:
• Ingest Data with lower latency, greater reliability and from any database using Oracle GoldenGate

7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 7
Oracle GoldenGate Use Cases
New DB/
HW/OS/APP
Fully Active
Distributed DB
Application
Offloading
Big Data, DW
& Marts
Message Bus
& Data Grid
Global Data
Centers
Data
Streaming
GoldenGate
Databases
Applications
Big Data and
No SQL
Mainframes
Mobile
Streaming &
Messaging
Capture
Deliver
Distribute
Receive
Secure
Oracle
SQL Server
MySQL
IBM DB2 Z
IBM DB2 i
IBM DB2 LUW
HP NonStop
Informix
Sybase
Messaging
High Availability
Zero Downtime
Migration
Data Synchronization
Big Data Ingestion
Query/Report
Offloading
Stream Analytics
Data Bus
(Lambda/Kappa)

8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.12/15/2017 8Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
 Oracle GoldenGate 12c Overview
 Oracle GoldenGate 12c Architecture
 Conflict Management
 Requirements
 New Feature
 Examples
 Monitoring

9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 9
Oracle GoldenGate Architecture
Classic
Capture Trail
Files
Pump DeliveryTrail
Files
Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. As of V.11.2.1, GoldenGate
offers two options for capture for Oracle; Classic & Integrated Capture
Trail: stages and queues data for routing.
Pump: distributes data for routing to target(s).
Route: data is compressed, encrypted for routing to target(s).
Delivery: applies data with transaction integrity. New with
GoldenGate 12c, Integrated Delivery.
12.2-

10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 10
Oracle GoldenGate Architecture
Microservices
Capture Trail Files Trail Files DeliveryDistribution
Service
Network Receiver
Service
Administration
Service
Metrics
Service
ServiceManager
>HTML>>SQL >>CLI >>API
Customizable Interactive Experience
Deploy -- Embed -- Automate
Proxy/Reverse Proxy
12.3+

11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.12/15/2017 11Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
 Oracle GoldenGate 12c Overview
 Oracle GoldenGate 12c Architecture
 Conflict Management
 Requirements
 New Feature
 Examples
 Monitoring

12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 12
Conflict Management
• Most Active/Active or Multi-Master systems
will have some form of conflict
• Conflicts happen due to at least one row being
modified on more than one system

13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Conflict Management
Example - Financial Institution: Active/Active
ACCTINFO
ID (PK)
NAME
ADDRESS
CITY
BALANCE
ACCTINFO
ID (PK)
NAME
ADDRESS
CITY
BALANCE
AcctInfo Table: General details related to a person’s bank account at a
national bank.
ServiceManager ServiceManager
Distribution
Service
Receiver
Service
Distribution
Service
Receiver
Service
Admin.
Service
Admin.
Service
EXT1
EXT2
REP1
REP2

14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Conflict Management
Example - Financial Institution: Active/Active
ACCTINFO
ID (PK)
NAME
ADDRESS
CITY
BALANCE
CHANGE_TS
ACCTINFO
ID (PK)
NAME
ADDRESS
CITY
BALANCE
CHANGE_TS
ServiceManager ServiceManager
Distribution
Service
Receiver
Service
Distribution
Service
Receiver
Service
Admin.
Service
Admin.
Service
EXT1
EXT2
REP1
REP2
CHANGE_TS CHANGE_TS
• 12.2 and earlier - Tables should contain timestamp columns
• 12.3 and later - Timestamp column can be added without effecting the application or code
base - leverages invisible columns/RDBMS 12.2 kernel

15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 15
Conflict Management
Supported Conflicts
Insert Conflict ✔ ✔
Update Conflict ✔ ✔
Delete Conflict ✔ ✔
Delta Resolution ✔ ✔
Insert-Update/Delete Conflict ✖ ✔
Column Group Conflict ✖ ✔
Piecewise LOB Update ✖ ✔
Conflict Type Classic CDR Auto CDR

16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Conflict Management
Supported Data Types
Datatypes that can be compared are supported:
• NUMERIC
• DATE
• TIMESTAMP
• CHAR/NCHAR
• VARCHAR/NVARCHAR
Typically these data types are used with parameters like COMPARECOLS,
GETBEFORECOLS, and in resolution parameters using
RESOLVECONFLICT ([USEMIN] | [USEMAX])

17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Conflict Management
Before and After Images
ACCTINFO BEFORE AFTER
ID (PK) 10 10
NAME 32 32
ADDRESS 96 Smith Rd 96 Smith Rd
CITY Atlanta Atlanta
BALANCE 1500 1550
CHANGE_TS
2014-10-29
00.00.00.000000 AM
2014-10-30
01.32.05.000000 PM
• Transaction logs are needed for
recovery
• Before Images
• Deletes and Updates
• After Images
• Inserts and Updates
Note: LogDump utility is useful for reading trail files to identify before/after images

18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.12/15/2017 18Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
 Oracle GoldenGate 12c Overview
 Oracle GoldenGate 12c Architecture
 Conflict Management
 Requirements
 New Feature
 Examples
 Monitoring

19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Requirements
Environment
Full before image of each record
Transactional Data
adminclient> dblogin user <gg user> password <password>
adminclient> add trandata SFAA.ACCTINFO, allcols
Behind the scene:
SQL> ALTER TABLE SFAA.ACCTINFO ADD SUPPLEMENTAL LOG GROUP
GGS_94879(id, name, address, city, balance, change_ts)
ALWAYS;
Note: System generated log group names can be located in DBA_LOG_GROUPS

20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
• Sequences need to be unique on each system
• Use starting value and increment by
number of systems
• Applies to existing sequences and identity
columns<site_number>+3
<site_number>+3 <site_number>+3
Requirements
Environment - Sequences

21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
• LOGALLSUPCOLS
Extract captures before images for UPDATE operations
Extract captures before images of supplemental logged columns for both UPDATE
and DELETE operations
• GETBEFORECOLS
Ensures certain columns are logged
GETBEFOREUPDATES (DB2 only)
TABLE option in extract
GETBEFORECOLS (ON UPDATE ALL, ON DELETE ALL)
Example:
TABLE SFAA.ACCTINFO, GETBEFORECOLS(ON UPDATE ALL, ON DELETE ALL);
Requirements
Environment - Extract Parameter File

22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
• TRANSLOGOPTION EXCLUDETAG
• Tag supplied to transaction as it is extracted to prevent receiving replicat from
trying to send it back to source
• Classic/Integrated Extract (primary)
• Tag can be any number/letter [0-9 A-Z]
Example:
TRANSLOGOPTION EXCLUDETAG 0294
Requirements
Environment - Extract Parameter File

23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
• REPERROR
Used to control how Replicat responds to errors
Default:
REPERROR(default, abend)
For CDR:
REPERROR(default, exception)
REPERROR(default2, [ abend | discard ])
REPERROR(-1, exception)
Requirements
Environment - Replicat Parameter File

24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
• COMPARECOLS
Used by Replicat to detect and resolve update/delete conflicts
• RESOLVECONFLICT
Used by Replicat in bi-directional/multi-master to handle conflicts for DML operations
Example:
MAP SFAA.ACCTINFO, TARGET ATLAA.ACCTINFO,
COMPARECOLS (ON UPDATE ALL, ON DELETE ALL),
RESOLVECONFLICT (INSERTROWEXISTS, (DEFAULT,USEMAX(CHANGE_TS)));
Requirements
Environment - Replicat Parameter File

25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
• Exception Table[s]
• User defined
• Used for troubleshooting or handling
errors
• Referenced in REPERROR and MAP
parameters
• Best if defined through macros
Requirements
Environment - Exception Table

26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Requirements
Extract Parameter File
-- CHECKPARAMS
EXTRACT EXTAA
USERID <gg user>, PASSWORD <pwd>
TRANLOGOPTIONS DBLOGREADER
TRANLOGOPTIONS EXCLUDETAG 0294
SETENV (ORACLE_HOME="/u01/app/oracle/product/12.1.0/db12cr1")
SETENV (ORACLE_SID="oragg")
WARNLONGTRANS 10m, CHECKINTERVAL 5m
LOGALLSUPCOLS
EXTTRAIL ./dirdat/la
WILDCARDRESOLVE IMMEDIATE
TABLE SFAA.ACCTINFO, GETBEFORECOLS(ON UPDATE ALL, ON DELETE ALL);

27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
-- CHECKPARAMS
REPLICAT REPAB
SETENV (ORACLE_HOME="/u01/app/oracle/product/12.1.0/db12cr1")
SETENV (ORACLE_SID="oragg")
USERID ggate, PASSWORD <pwd>
DISCARDFILE ./dirrpt/REPAB.discard APPEND MEGABYTES 100
REPERROR (default, exception)
REPERROR (default2, abend)
ASSUMETARGETDEFS
WILDCARDRESOLVE IMMEDIATE
INCLUDE ./dirmac/exceptions.mac
MAP ATLAA.ACCTINFO, target SFAA.ACCTINFO,
COMPARECOLS (ON UPDATE ALL, ON DELETE ALL),
RESOLVECONFLICT (insertrowexists, (default, usemax(change_ts))),
RESOLVECONFLICT (updaterowexists, (default, usemin(change_ts))),
RESOLVECONFLICT (deleterowmissing, (default, discard));
MAP ATLAA.ACCTINFO, #exception_handler(ggate);
Requirements
Replicat Parameter File

28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.12/15/2017 28Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
 Oracle GoldenGate 12c Overview
 Oracle GoldenGate 12c Architecture
 Conflict Management
 Requirements
 New Feature
 Examples
 Monitoring

29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 29
New Feature
Automatic Conflict Detection & Resolution Highlights
• Invisible timestamp columns automatically created and maintained
by database kernel
• Delete tombstone log to support updates-delete conflicts (new
capability)
• Enables supplemental logging on required columns
• Allows ability to detect conflicts for LOB column changes
• No need to specify CDR/REPERROR parameters
• Last record wins (highest timestamp)
No Visible Application Changes
Requirements
• Requires source and target to be Oracle Database 12.2
• Requires Integrated Extract (IE)
• Classic Extract (CE) does not support invisible columns
• Requires Integrated Replication (IR) or Integrated Parallel
Replicat (IPR)
12.3

30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 30
New Feature
Simple to Enable
• Add parameter “MAPINVISIBLECOLUMNS” in your replicat parameter file
– Classic Replicat does not support invisible columns
• Run procedure ADD_AUTO_CDR or ADD_AUTO_CDR_COLUMN_GROUP
– Sets up Latest Timestamp CDR for a table
12.3

31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 31
New Feature
ADD_AUTO_CDR Example
SQL> connect ggadmin/***@inst1
Connected
SQL> BEGIN
dbms_goldengate_adm.add_auto_cdr(
schema_name => ′HR′,
table_name => ’EMPLOYEES’);
END;
/
PL/SQL procedure successfully completed.
12.3

32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 32
Key Name City
Enable Auto CDR
Key Name City CDRTS$ROW
Statement TS
PKey Delete TS
Statement TS
Invisible
DT$_TAB1
TAB1
TAB1
• Automatically creates an invisible timestamp
column to associate with the row
• Invisible timestamp column automatically
maintained
• Automatically enables Delete Tombstone
Log Table
• Tombstone table automatically maintained
• Automatically enables all column
supplemental logging
+
New Feature
ADD_AUTO_CDR Example
12.3

33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 33
DML driver will automatically populate all the
ROW_TS timestamp columns with the current
timestamp
Key Name City ROW_TS
1 Alice San Mateo @T2
Key Name City ROW_TS
DML driver will automatically populate the
ROW_TS timestamp columns associated
with the modified columns with the
current timestamp
Automatically Record a tombstone record in
the Delete Tombstone log. Another
timestamp is used to record when the row
was deleted.
Key Name City ROW_TS
1 Alice San Mateo @T2
Key Name City ROW_TS
1 Alice South SF @T5
Key Delete TS
1 @T12
Key Name City ROW_TS
1 Alice South SF @T5
Inserts Updates Deletes
Denotes fields automatically populated and maintained by database
New Feature
ADD_AUTO_CDR Example
12.3

34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 34
• DBA_APPLY_ERROR_MESSAGES tracks Error Resolution
• Win/Loss-Monitoring
CONFLICT_TYPE APPLIED_STATE CONFLICT_INFO
-------------------------- ------------------ ----------------------
UPDATE ROW EXISTS PARTIAL DRTS$C:L,CDRTS$ROW:W
• Custom Conflict Handlers
– Apply DML (Error) Handler with own sophisticated business logic
– PL/SQL Procedure is linked to the Apply process
Additional Features
12.3

35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.12/15/2017 35Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
 Oracle GoldenGate 12c Overview
 Oracle GoldenGate 12c Architecture
 Conflict Management
 Requirements
 New Feature
 Examples
 Monitoring

36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Examples
Insert Row Exists
MAP SFAA.ACCTINFO, TARGET ATLAA.ACCTINFO,
COMPARECOLS (ON UPDATE ALL, ON DELETE ALL),
RESOLVECONFLICT (INSERTROWEXISTS, (DEFAULT, USEMAX(CHANGE_TS)));
ACCTINFO SF ATL
ID 944 944
NAME 32 32
ADDRESS 55 5th Street 55 5th Street
CITY Symrna Symrna
BALANCE 1100 1200
CHANGE_TS
31-OCT-14
10.40.36.000000000 PM
31-OCT-14
10.40.40.000000000 PM
INSERTROWEXISTS
• Violates the unique
constraint on the target

37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Examples
Update Row Exists
MAP SFAA.ACCTINFO, TARGET ATLAA.ACCTINFO,
COMPARECOLS (ON UPDATE ALL, ON DELETE ALL),
RESOLVECONFLICT (UPDATEROWEXISTS, (DEFAULT, USEMIN(CHANGE_TS)));
ACCTINFO BEFORE AFTER
ID 944 944
NAME 32 32
ADDRESS 55 5th Street 55 5th Street
CITY Symrna Symrna
BALANCE 1150 1100
CHANGE_TS
31-OCT-14
10.40.36.000000000 PM
31-OCT-14
10.40.40.000000000 PM
UPDATEROWEXISTS
• Updated row exists on target
side
• One or more columns have a
before image different from
current value

38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Examples
Delete Row
MAP SFAA.ACCTINFO, TARGET ATLAA.ACCTINFO,
COMPARECOLS (ON UPDATE ALL, ON DELETE ALL),
RESOLVECONFLICT (DELETEROWMISSING, (DEFAULT, [ DISCARD | IGNORE ]));
DELETEROWMISSING
• Deleted row does not exist
on target
ACCTINFO SF ATL
ID 944 null
NAME 32 null
ADDRESS 55 5th Street null
CITY Symrna null
BALANCE 1100 null
CHANGE_TS
31-OCT-14
10.40.36.000000000 PM null

39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.12/15/2017 39Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
 Oracle GoldenGate 12c Overview
 Oracle GoldenGate 12c Architecture
 Conflict Management
 Requirements
 New Feature
 Examples
 Monitoring

40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Monitoring
Statistics
Replicating from ATLAA.ACCTINFO to SFAA.ACCTINFO:
*** Total statistics since 2014-11-02 00:30:43 ***
Total inserts 719.00
Total updates 93.00
Total deletes 524.00
Total discards 0.00
Total operations 1336.00
Total CDR conflicts 377.00
CDR resolutions succeeded 377.00
CDR INSERTROWEXISTS conflicts 257.00
CDR UPDATEROWEXISTS conflicts 93.00
CDR DELETEROWMISSING conflicts 27.00
adminclient> stats replicat <group name>, reportcdr

41. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Additional Information
@DBASOLVED
https://dbasolved.com
bobby.curtis@oracle.com
https://www.linkedin.com/in/bobbycurtis294/
CDR Info: https://docs.oracle.com/goldengate/c1230/gg-winux/GIORA/automatic-conflict-detection-and-
resolution2.htm#GIORA-GUID-DF68483B-1F77-4464-B5E3-BB42B360E813
Installing Oracle GoldenGate: https://docs.oracle.com/goldengate/c1230/gg-winux/GIORA/toc.htm
http://www.oracle.com/technetwork/middlewar
e/goldengate/downloads/index.html