Oracle Goldengate first acquaintance The Oracle's strategic solution for real time data integration. Oracle GoldenGate provides low-impact capture, routing, transformation, and delivery of transactional data across heterogeneous environments in real time

1. WWW.TRANSFER-SOLUTIONS.COM
SPREKER :
E-MAIL :
DATUM :
Oracle GoldenGate
Oracle's strategic solution for real time data integration
Jos van den Oord
Jos.van.den.Oord@transfer-solutions.com
13 Juni 2013 Cafe
SPEAKER
DATE

2. © COPYRIGHT TRANSFER SOLUTIONS B.V. 2
Who am I
Jos van den Oord
Working with Oracle products since 1998
Working at Transfer-Solutions since 2010
Interests
Databases
Maximum Availability Manageable Architecture (MAMA)
Right Availability
Security
Performance
Blog : joordsblog.vandenoord.eu
Email : jos.van.den.oord@transfer-solutions.com

3. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Agenda
Introduction Oracle GoldenGate
Oracle products for data movement
Technology overview GoldenGate
Oracle GoldenGate components
GoldenGate Process data flow
GoldenGate Software Command Interface -
GGSCI
Demos GoldenGate replications
Points of interest for Oracle GoldenGate
3

4. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Introduction Oracle GoldenGate
Oracle GoldenGate provides low-impact
capture, routing, transformation, and delivery of
transactional data across heterogeneous
environments in real time
GoldenGate Software Inc was founded in 1995
Oracle Corporation acquired GoldenGate
Software in September 2009
4

5. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Oracle GoldenGate Products
Heterogeneity Supports Afb Overview
5
Product / License Description
Oracle GoldenGate Includes GoldenGate Capture,
Deliver, Active Data Guard and
XStream
Oracle GoldenGate for Mainframe Includes GoldenGate Capture and
Deliver for HP NonStop and IBM DB2
on z/OS
Oracle GoldenGate Veridata Add-on capability to validate data in
replicated systems
Management Pack for GoldenGate Add-on management pack, includes
Oracle GoldenGate Director
Oracle GoldenGate Adapters Pre-packaged application content,
such as for Flat File and Java
adapters

6. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Oracle GoldenGate Supports
6
Databases O/S and Platforms
Capture:
 Oracle
 DB2
 Microsoft SQL Server
 Sybase ASE
 Ingres
 Teradata
 Enscribe
 SQL/MP
 SQL/MX
Delivery:
 All listed above
 MySQL, HP Neoview, Netezza, and
any ODBC compatible databases
 ETL products
 JMS message queues
Windows 2000, 2003, XP
Linux
Sun Solaris
HP NonStop
HP-UX
HP TRU64
HP OpenVMS
IBM AIX
IBM z/OS

7. © COPYRIGHT TRANSFER SOLUTIONS B.V.
GoldenGate Topologies
7
One-Way replication Bi-directional replication
Broadcast replication Integration replication
Cascading
replication

8. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Oracle Products for Data Movement
Comparing Methods
8
Redo in
Memory Buffer
Active Data
Guard
Disaster Recovery & Data Protection
GoldenGate
Real Time Data Integration & High Availability
Data
Integrator
Data Integration for Data Warehouse & SOA
DECREASINGLATANCY
INCREASINGTRANSFORMATION

9. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Oracle Products for Data Movement
• Oracle Disaster Recovery
• Read-only ReportingADG
• Real Time Data synchronisation
• Zero-Downtime Migrations
• Active – Active High Availability
OGG
• Any Data Warehouse
• Bulk Data Transformation
• Complex Migrations
ODI
9
Method Choice

10. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Technology Overview GoldenGate
Modular Building Blocks
Database
LogsLogsLogs
Deliver
Delivery: Applies transactional
data with guaranteed integrity
Trail Files
Deliver Capture
Trail Files
Bi-directional
Trail Files Trail Files
Trail files: Universal data format enables heterogeneity
Route: No Distance constraints
via TCP/IP.
Compression & encryption.
Capture
Database
LogsLogsLogs
Capture: Committed changes are captured ( and can be filtered)
As they occur by reading the transaction logs
LAN/WAN
TCP/IP

11. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Tools GoldenGate
GGSCI
GoldenGate Software Command Interface
DEFGEN
Needed by different source and target table
definitions
LOGDUMP
View trail file in hex or ASCII for debugging
REVERSE
Generate undo commands from the trail files
11

12. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Installatie OGG
Installation
 Create OS user
 Create OS file structure
 Oracle Software Delivery GoldenGate
 Valideer de OGG software
 Create directories
 Manager proces (MGR)
 Install Demo
12

13. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Oracle GoldenGate Components
13
Manager process
SUBDIRECTORIES
EXTRACT
DATA-PUMP
REPLICAT
TRAILFILES

14. © COPYRIGHT TRANSFER SOLUTIONS B.V.
The Subdirectories Golden Gate
• Default location for checkpoint files by Extract and
Replicatdirchk
• Default location Golden Gate trail files and extract
files by Extract procesdirdat
• Default location data definition files created by
DEFGEN utilitydirdef
• Default location for process status files, created
by process runningdirpcs
• Default location for GoldenGate parameter filesdirprm
• Default location for report files created by Extract,
Replicat and Manager processdirrpt
• Default location for SQL scriptsdirsql
• Default location for GoldenGate process
temporary files (“swap.out”)dirtmp
14

15. © COPYRIGHT TRANSFER SOLUTIONS B.V.
The Manager process - MGR
Master process for GoldenGate
Runs on source and target
Control process activities
Starting, monitoring, restarting processes
Allocating data storage
Report errors and events
Process status
STOPPED, RUNNING
15

16. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Extract – The capture process
Extract necessary data from the database
transaction logs.
Regularly checkpoint read and write position
Process status
STOPPED, STARTING, RUNNING, ABENDED
16
Capture Data
PumpTrail Files
Collector
Trail Files
Delivery
Sync

17. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Trail files
Data format Canonical
Eliminates any single point of failure and
ensures data integrity
Works together with checkpoint process
Optional file on source system
17
Capture Data
PumpTrail Files
Collector
Trail Files
Delivery
Sync

18. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Data - pump
Process for data synchronisation between local
en remote trail files.
Best practice for High Availability
Works together with the Server Collector
process. Write data on remote system to the
trail file (remote trail)
18
Capture Data
PumpTrail Files
Collector
Trail Files
Delivery
Sync

19. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Replicat – The Apply process
Reads the trail file and applies of read the trail
file apply to the database (DML or DDL)
Regularly checkpoint read and write position
Process status
STOPPED, STARTING, RUNNING, ABENDED
DDL supported
Unidirectional
Non-heterogeneous environments
19
Capture Data
PumpTrail Files
Collector
Trail Files
Delivery
Sync

20. © COPYRIGHT TRANSFER SOLUTIONS B.V.
GoldenGate Process data flow
20
MGR MGR
CKPT PARAM TRAIL
CKPT
CKPT PARAMTRAIL
EXTRACT REPLICAT
Server
Collector
Data
Pump

21. © COPYRIGHT TRANSFER SOLUTIONS B.V.
GoldenGate Software Command
Interface - GGSCI
Quick guide commands
START,STOP [extract|replicat|er*]
ADD [extract|replicat]
HELP
INFO [all|extract|replicat|name]
SEND [mgr|getlag]
STATUS [mgr|getlag]
21

22. © COPYRIGHT TRANSFER SOLUTIONS B.V. 22

23. © COPYRIGHT TRANSFER SOLUTIONS B.V. 23

24. © COPYRIGHT TRANSFER SOLUTIONS B.V. 24

25. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Demos GoldenGate replications
Initial load extract replicat
Replication schema Scott
BI-Replication Schema Scott
DDL replication SCOTT.EMP table
25

26. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Points of Interest for Oracle GoldenGate
• Creating Dedicated Users
• Encrypting Passwords
• Creating a Dedicated
Installation Directory
• Using a Checkpoint Table
• Verifying the Character Sets
• Developing Naming Standards
• Using a Data Pump
Installation
and Setup
26

27. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Points of Interest for Oracle GoldenGate
• Using GGSCI Command Shortcuts
• Using OBEY Files
• Generating Interim Statistics
• Using a Discard File
• Reporting Regularly on Process
Health
• Purging Old Trail Files Regularly
• Automatically Starting Processes
Management
and
Monitoring
27

28. © COPYRIGHT TRANSFER SOLUTIONS B.V.
Points of Interest for Oracle GoldenGate
• Running Performance Tests
• Limiting the Number of Extracts
• Using Passthru parameter for
Data Pumps
• Using Parallel Replicats
• Using the Fastest Available
Storage
• Tuning the Database
Performance
28

29. © COPYRIGHT TRANSFER SOLUTIONS B.V. 29
Q u e s t i o n s
A n s w e r s
CONSULTING | MANAGED SERVICES | EDUCATION
WWW.TRANSFER-SOLUTIONS.COM
Master Note - Oracle GoldenGate [ID 1298817.1]