More Related Content Similar to Tungsten University: Setup & Operate Tungsten Replicator (20) More from Continuent (20) Tungsten University: Setup & Operate Tungsten Replicator2. Introducing Continuent
•
The leading provider of clustering and
replication for open source DBMS
•
Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance
scaling and data management for MySQL
• Replication - Flexible, high-performance data
movement
©Continuent 2013
2
6. Overview of Installation Process
1. Set up hosts
2. Prepare MySQL replicas
3. Download software
4. Install using tpm utility
Amazon Setup:
https://docs.continuent.com/wiki/display/TEDOC/
Preparing+EC2+Servers
©Continuent 2013
6
7. Old Way of Installing Tungsten...
• tungsten-installer - Perform initial
installation
• con!gure-service - Set up individual
replication services
• update - Change parameters on services
after installation
The tpm utility replaces all of this!
©Continuent 2013
7
8. How tpm Works
Staging copy
of files
db1
db2
check prereqs
copy code
configure topology
©Continuent 2013
db3
8
10. Commands to Check Replication
•
•
•
•
•
•
©Continuent 2013
trepctl services -- All services
trepctl status -- Overall pipeline status
trepctl status -name stages -- Stage summary
trepctl status -name stores -- Store summary
trepctl status -name tasks -- Task detail
trepctl [-service name] heartbeat -- Is it Alive?
10
12. Commands to Change Con!guration
•
•
tpm update - Alter con"guration
tpm update - Upgrade the replicator to a new
version
Hint: Don’t change replicator
configuration files directly...
©Continuent 2013
12
13. Recon!guring a Replicator
•
No restart on local host
trepctl offline
tools/tpm update ... --no-restart
trepctl configure
trepctl online
•
With automatic restart
tools/tpm update ...
©Continuent 2013
13
15. Parallel Replication of Sharded Data
Pipeline
Stage
Extract Filter
Master
DBMS
©Continuent 2013
Stage
Apply
Extract Filter
Transaction
History Log
Apply
Stage
Extract Filter
Extract Filter
Extract Filter
Parallel
Queue
15
Apply
Apply
Apply
Slave
DBMS
17. The Scene of the Crime
trepctl -host db2 status
Processing status command...
NAME
VALUE
-------...
pendingError
: Event application failed:
seqno=1957469 fragno=0 message=java.sql.SQLException:
Statement failed on slave but succeeded on master
pendingErrorCode
: NONE
pendingErrorEventId
: mysql-bin.000008:0000000461736666;0
pendingErrorSeqno
: 1957469
pendingExceptionMessage: java.sql.SQLException: Statement
failed on slave but succeeded on master
create table trouble(id int primary
key) /* ___SERVICE___ = [university] */
...
Finished status command...
©Continuent 2013
17
18. Correcting problems
1. Look at the transaction (thl list -seqno NNN)
2. Look at the replicator message log
3. Fix the problem and go online
OR
4. Skip over the problem and go online
OR
5. Reprovision the slave
©Continuent 2013
18
20. Simple Facts about Logs
•
•
•
•
•
Replicator log = THL (Transaction History Log)
One THL per replication service
2x the size of MySQL binlog
Default retention of 7 days
Take service o#ine to move or delete logs
safely
• The thl tool shows you log contents
©Continuent 2013
20
22. Resetting and Reinstalling
Reset
1. Stop replicator
2. Remove THL "les
3. Remove tungsten_<svc> databases
Reinstall
1. All of the above, plus delete /opt/continuent/*
2. Rerun installation
©Continuent 2013
22
24. What Backup Tools Does MySQL Have?
MySQL
Server
Process
mysqldump
(dataset < 10Gb)
xtrabackup
(10Gb < dataset < 1Tb
tar/rsync
(dataset < 1Tb)
DBMS Files
file system snapshot
(1Tb << dataset)
File System
©Continuent 2013
24
26. How Do I Install Fan-In Replication?
db1
Replicator
Service db1
Replicator
Service db1
db2
Service db2
Replicator
Service db2
©Continuent 2013
db3
26
27. How Do I install Multi-Master?
db1
Replicator
Service db1
Service db2
db3
Service db3
Service db1
Service db2
db2
Service db3
Service db1
Service db2
Service db3
Replicator
©Continuent 2013
27
28. Tpm Understands Topologies!
• Use --topology argument!
Fan-In
--topology=fan-in
©Continuent 2013
All-Masters
Star
--topology=all-masters --topology=star
28
29. For More Help: Tungsten Cookbook
•
•
•
•
•
Installs complex topologies automatically
Directions in cookbook/README
Parameters in cookbook/USER_VALUES.sh
Node names in COMMON_NODES.sh
Installation script for each supported
topology
http://datacharmer.blogspot.com/2013/07/tpm-multimaster-composer.html
©Continuent 2013
29
31. Tungsten Replicator is Open Source
•
Project home:
http://code.google.com/p/tungsten-replicator/
•
Log bugs, "nd builds, post in replicator discussion
group
•
Documentation:
https://docs.continuent.com/wiki/display/TEDOC/
Tungsten+Documentation+Home
©Continuent 2013
31
33. More Webinars and Talks!
•
How Gittigidiyor (a subsidiary of eBay)
Replicates Data in Real Time from MySQL to
Oracle
(6 November 2013 10am EDT/3pm GMT)
•
Come see us at Percona Live London!
(11-12 November 2013)
Send any feedback to: tu@continuent.com
©Continuent 2013
33
34. 560 S. Winchester Blvd., Suite 500
San Jose, CA 95128
Tel +1 (866) 998-3642
Fax +1 (408) 668-1009
e-mail: sales@continuent.com
Our Blogs:
http://scale-out-blog.blogspot.com
http://datacharmer.org/blog
http://www.continuent.com/news/blogs
http://"yingclusters.blogspot.com/
Continuent Web Page:
http://www.continuent.com
Tungsten Replicator 2.0:
http://code.google.com/p/tungsten-replicator
©Continuent 2012.