Demo of Dbvisit replicate setup at SOUG Lausanne. http://screencast-o-matic.com/u/nTb6/dbvisit-repattack-setup

1. Dbvisit replicate: logical replication made easy
Infrastructure at your Service.
Dbvisit replicate:
logical replication made easy

2. Dbvisit replicate: logical replication made easy 07.11.2014
Infrastructure at your Service.
About me
Page 2
Franck Pachot
Senior consultant
Oracle Technology Leader
Mobile +41 79 963 27 22franck.pachot@dbi-services.comwww.dbi-services.com
Presentations, Articles, blogs, forums

3. Dbvisit replicate: logical replication made easy
Experts At Your Service
>35 specialists in IT infrastructure
>Certified, experienced, passionate
Based In Switzerland
>100% self-financed Swiss company
>Over CHF 4 mio. turnover
Leading In Infrastructure Services
>More than 100 customers in CH, D, & F
>Over 30 SLAs dbi FlexServicecontracted
07.11.2014
dbiservices
Who we are
Page 3

4. Dbvisit replicate: logical replication made easy
1.Why logical replication?
2.Demo (setup)
3.Configuration alternatives
4.Demo (run)
5.Why Dbvisit?
07.11.2014
Agenda
Page 4

5. Dbvisit replicate: logical replication made easy
Logical vs. Physical
07.11.2014
Why Logical replication
Page 5
REDO
DB
SQL
statements
REDO LOG
records
REDO LOG
records
Applied on tables
Applied on files
Applied on files
APPLY
Physical replication
>Redo based
>Oracle EE or Dbvisitstandby
redo
redo

6. Dbvisit replicate: logical replication made easy
Logical vs. Physical
07.11.2014
Why Logical replication
Page 6
REDO
DB
SQL
statements
REDO LOG
records
REDO LOG
records
Applied on tables
Applied on files
Applied on files
SQL
statements
Applied on tables
APPLY
trigger
mviewlog
sql
Logical replication
>Application based
>Oracle Streams

7. Dbvisit replicate: logical replication made easy
Logical vs. Physical
07.11.2014
Why Logical replication
Page 7
REDO
DB
REDO LOG
records
Applied on tables
Applied on files
Applied on files
SQL
statements
Applied on tables
sql
Logical replication
>Redo based
>Golden Gate or Dbvisitreplicate
SQL
statements
MINE
redo
REDO LOG
records
APPLY

8. Dbvisit replicate: logical replication made easy
Physical replication
>High performance, exact copy (HA)
>Same version (platform, database, application)
>No filter or transformation (same tables, columns, indexes, partitions)
>Oracle Data Guard (EE), Dbvisitstandby, Manual standby and scripts
Logical replication based on redo
>More complex (supported datatypes, conflict resolution)
>Destination can differ (version, data model, data)
>We choose what we replicate (filter, change deletes to inserts)
>Destination is Read-Write, can be 2-ways
>Oracle Golden Gate, Dbvisitreplicate, Streams
Logical vs. Physical
07.11.2014
Why Logical replication
Page 8

9. Dbvisit replicate: logical replication made easy
Offloaded query environments
>Reporting, Data Quality
Distributed databases
>local proxies replicated to master
Data warehousing and Audit
>Extract data for ETL, can be real time
Audit
>Stores all modifications (old an new values)
Migration projects
>Oracle Edition downgrade (go to Standard Edition)
>Oracle upgrades, OS platform, to the Cloud, etc.
>11g to PDB (without non_cdb_to_pdb.sql)
What are the use cases?
07.11.2014
Why logical replication
Page 9

10. Dbvisit replicate: logical replication made easy
1.Why logical replication
2.Demo (setup)
3.Configuration alternatives
4.Demo (run)
5.Why Dbvisit?
07.11.2014
Agenda
Page 10

11. Dbvisit replicate: logical replication made easy
http://www.dbvisit.com/repattack/
07.11.2014
Demo (setup)
Page 11

12. Dbvisit replicate: logical replication made easy
Two VirtualBoxVM:
>source
>Linux Centos 6.5, Oracle XE 11g with XE database
>Swingbenchschema, data and application
>target
>Linux Centos 6.5, Oracle XE 11g with XE database (empty)
Downloaded and installed dbvrepon both machines
>As root, on source and target:
>Run wizard: http://screencast-o-matic.com/u/nTb6/dbvisit-repattack-setup
Demo environment
07.11.2014
Demo (setup)
Page 12
root@source# rpm -ivhdbvisit_replicate-2.7.06-1.x86_64.rpm
oracle@source# dbvrep
Dbvrep> setup wizard

13. Dbvisit replicate: logical replication made easy
Setup wizard and initialization
07.11.2014
Demo (setup)
Page 13
Setup Wizard
•Describedatabases(ttorcl_srcand ttorcl_trg)
•Definereplicationpairs (initializationwithimpdpflashback_scn)
•Definereplicatedtables (all REPOE schema)
•Configure processes(MINE on source, APPLY on target)
Initialization
•Runall.sh (createsdbvrepschema)
•Copy scripts (seeNextsteps.txt)
•CreateDBLINK
•runapply.sh (impdp)
Start replication
•Start MINE process
•Start APPLY process
•Runstart-console.sh

14. Dbvisit replicate: logical replication made easy
1.Why logical replication
2.Demo (setup)
3.Configuration alternatives
4.Demo (run)
5.Why Dbvisit?
07.11.2014
Agenda
Page 14

15. Dbvisit replicate: logical replication made easy
Editions and Features
07.11.2014
Configuration alternatives
Page 15
LTD
•Replicatedata for <200 tables
•For reporting, datawarehouse
XTD
•up to 2000 tables. + fetcher, DDL, CLOB,RAC
•For migration (canberentedby month)
MAX
•Unlimitedtables, 2-way
•Databasedistribution
CHF 3000 to 7000
x SOCKET

16. Dbvisit replicate: logical replication made easy
MINE + APPLY
07.11.2014
Configuration alternatives
Page 16
Source server
Target server
DB
DB
REDO
SQL
PLOG
Dbvisitinternalformat
PLOG
APPLY
MINE

17. Dbvisit replicate: logical replication made easy
FETCHER + MINE + APPLY
07.11.2014
Configuration alternatives
Page 17
Source server
Target server
DB
DB
PLOG
REDO
REDO
SQL
APPLY
FETCHER
MINE

18. Dbvisit replicate: logical replication made easy
FETCHER + MINE + APPLY through SQL*Net
07.11.2014
Configuration alternatives
Page 18
Source server
Target server
Replicationserver
DB
DB
FETCHER
PLOG
REDO
REDO
MINE
APPLY
SQL

19. Dbvisit replicate: logical replication made easy
Archivelogonly MINE + APPLY
07.11.2014
Configuration alternatives
Page 19
Source server
Target server
Replicationserver
DB
DB
PLOG
REDO
MINE
APPLY
SQL
ARCHIVED
LOGS
ARC
(not real time)

20. Dbvisit replicate: logical replication made easy
Target can be outside (no connect to source)
Apply can be outside (no connect to source)
TCP/IP Connections
07.11.2014
Configuration alternatives
Page 20
Source server
Target server
FETCHER
REDO
MINE
APPLY
CONSOLE
DB
DB

21. Dbvisit replicate: logical replication made easy
dp_networklink
>The simplest: impdpnetwork_link=from target
dp_exp
>Create a dump file with expdpon source
exp
>Obsolete exp/imp (old versions)
ddl_run
>Get metadata as DDL
Other alternatives:
>RMAN duplicate, Storage snapshots, etc.
Initialization alternatives to import data
07.11.2014
Configuration alternatives
Page 21
sql

22. Dbvisit replicate: logical replication made easy
>Which changes are replicated by import?
>Which changes are replicated by MINE?
>We need to MIN complete transactions
We need to lock in Share mode to stop transactions when getting the SCN
Initialization alternatives to get SCN
07.11.2014
Configuration alternatives
Page 22
Transaction 1
Transaction 3
Transaction 5
Transaction 2
Trans. 4
Trans. 6
?
Transaction 1
Transaction 3
Trans. 5
Transaction 2
Trans. 4
Trans. 6
SCN
lock
Data transferredby initialization
Data transferredby replication
Data transferredby initialization
Data transferredby replication

23. Dbvisit replicate: logical replication made easy
We must MINE complete transactions: cannot initialize when we have current transaction -> Lock in Share mode.
one-by-one
>Locks table one by one and get SCN for each table ->import one by one
single-scn
>Locks all tables to get one SCN ->easier import
ddl-only
>no lock, no data ->starts with empty tables (for audit)
resetlogs
>Using standby or duplicate instead of import
no-lock (since 2.7)
>Get SCN for the oldest transaction, no lock
>When supplemental logging must be already set
Initialization alternatives to get SCN
07.11.2014
Configuration alternatives
Page 23

24. Dbvisit replicate: logical replication made easy
1.Why logical replication
2.Demo (setup)
3.Configuration alternatives
4.Demo (run)
5.Why Dbvisit?
07.11.2014
Agenda
Page 24

25. Dbvisit replicate: logical replication made easy
Setup wizard and initialization
07.11.2014
Demo (setup)
Page 25
Setup Wizard
•Describedatabases(ttorcl_srcand ttorcl_trg)
•Definereplicationpairs (initializationwithimpdpflashback_scn)
•Definereplicatedtables (all REPOE schema)
•Configure processes(MINE on source, APPLY on target)
Initialization
•Runall.sh (createsdbvrepschema)
•Copy scripts (seeNextsteps.txt)
•CreateDBLINK and runapply.sh (impdp)
Start replication
•Start MINE process
•Start APPLY process
•Runstart-console.sh

26. Dbvisit replicate: logical replication made easy
Issue
>Table has no primary key and has duplicates (all columns have same values)
>By default, a redo record replication must touch one and only one row
>When we delete on source, the sqlgenerated affect 2 rows
Conflict ‘too many rows’
>The first redo record received touches 2 rows
>We force do delete those 2 rows
>Then the second redo record touches no rows, we ignore it
Manual conflict resolution
07.11.2014
Additional demos
Page 26
dbvrep> list conflict
dbvrep> resolve conflict <id from list conflict> resolve as FORCE
dbvrep> list conflict
dbvrep> resolve conflict <id from list conflict> resolve as IGNORE

27. Dbvisit replicate: logical replication made easy
Conflict handlers
>For automatic resolution, we choose to add a ‘where rownum=1’ in the generated sql.
>Syntax is available from online help
Conflict happen in logical replication, especially
>With delete cascade constraints
>With triggers doing DML
>With 2-way replication
Solution: either disable on target, or set conflict handlers
Automatic conflict resolution
07.11.2014
Additional demos
Page 27
dbvrep> set_conflict_handlersfor table REPOE.DEMODUP for DELETE on TOO_MANY to sqls/$/ and rownum= 1/
dbvrep> help
dbvrep> help set_conflict_handlers

28. Dbvisit replicate: logical replication made easy
Reasons:
>Reorganization on source
>Bad conflict resolution, easier to restart then resolve
1. Stop replication
>The prepare gives the SCN for the initialization
2. Initialize the target table
3. Restart replication
Resynchronize a table
07.11.2014
Additional demos
Page 28
dbvrep> unpreparetable REPOE.DEMODUP
dbvrep> pause apply
dbvrep> prepare table REPOE.DEMODUP
SQL> truncate table REPOE.DEMODUP
SQL> insert /*+ append */ into REPOE.DEMODUP select * from REPOE.DEMODUP@ttorcl_srcas of scn<SCN given by the PREPARE>
dbvrep> resume

29. Dbvisit replicate: logical replication made easy
1.Why logical replication
2.Demo (setup)
3.Configuration alternatives
4.Demo (run)
5.Why Dbvisit?
07.11.2014
Agenda
Page 29

30. Dbvisit replicate: logical replication made easy
DbvisitReplicate vs. GoldenGatevs. Streams
07.11.2014
Why Dbvisit?
Page 30
What Dbvisitsays
Main differences we see:
>GoldenGatecurrently has more features but
>Dbvisitis per socket (CHF 2500 to 6000) Monthly rental possible
>Dbvisitoptimistic apply: mine and apply don’t wait for commit
>Dbvisitis easy to start (setup wizard)

31. Dbvisit replicate: logical replication made easy
Know if you need logical replication
>It’s not the magic solution. It’s a project.
Testing (POC) is easy
>Install a Dbvisitreplicate configuration in one hour
>test it on your data (datatypes, operations,etc.)
Logical replication is a DevOpsproject
>Operations is involved
supplemental logging, redo/archived, filesystemstorage, network
>Application is involved
define conflict rules, trigger behavior, supportddatatypes,…
Core message
07.11.2014
Why Dbvisit?
Page 31

32. Dbvisit replicate: logical replication made easy 07/11/2014
Page 32
Any questions? Please do ask.
Infrastructure at your Service.
We look forward to working with you!
dbiservices is the Dbvisitpartner in Switzerland
>We offer fixed prices for Dbvisitstandby
>A POC for Dbvisitreplicate is few days
>We have direct access to support
Repattackon 20thNov. in Nuremberg (DOAG)
>Dbvisitreplicate on your laptop