Android KNow more

1. Reliable MySQL Using
Replication
Issac Goldstand
Mirimar Networks
www.mirimar.net
margol@beamartyr.net

2. What is replication?
• Allows 2 or more databases to maintain state
between them
• MySQL 3.23 and up supports asynchronous
replication
• One server acts as a master for one or more
slaves
• Slaves pull data to be replicated from server’s
binary logs and execute the relevant statements
locally

3. Replication basics…
• One master supports multiple slaves
• Each node on the network is assigned a unique
identifying number
• Each slave attempts to connect to the master and
fetches data to be replicated
Master
Slave
Slave
Slave
Slaves

4. Replication basics…
• Clients perform data modification on master
server
• INSERT, SELECT, DELETE, LOAD DATA
• EXECUTE in MySQL 5.0 and above
Master SlaveClient
DATA DATA
INSERT
INTO …

5. Replication basics…
• Immediately following execution of command on master,
the command is written to the local binary log
• Additionally, the master records its unique ID (to prevent
endless loops in circular replication scenarios) and the
timestamp for use with statements which use NOW(),
etc.
Master SlaveClient
DATA
DATA
INSERT
INTO …
Binary
Log

6. Replication basics…
• If the slave is online, the command is transmitted to the slave in
parallel (well, immediately following) to being written in the local
binary log
• Otherwise, when the slave next connects it will receive a list of all
pending statements from the master server’s binary log
• The slave’s replication IO thread stores the command in the local
relay log
Master SlaveClient
DATA
DATA
INSERT
INTO …
INSERT
INTO …
Relay
Log
Replication
Thread

7. Replication basics…
• Once the data is received in the slave’s relay log, the
slave SQL thread executes the command locally,
bringing the slave up-to-date with the master
• In MySQL 3.23, the IO and SQL threads were just one
thread. In later versions this was changed to boost
performance
Master SlaveClient
DATA
DATA
INSERT
INTO …
INSERT
INTO …
Relay
Log
Replication
Thread
DATA

8. Replication Strategies
• Load balancing – single write, distributed read
Master
Slave Slave Slave
Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT) Client (SELECT)
Client (INSERT)

9. Replication Strategies
• Load balancing – single write, distributed read
• Load balancing – circular read/write
MySQL
Server
MySQL
Server
MySQL
Server
Client
INSERT
INTO…
INSERT
INTO …
Client

10. Replication Strategies
• Load balancing – single write, distributed read
• Load balancing – circular read/write
• High availability (hot failover)
Client Master
DATA
Slave
DATA
DATADATADATA

11. Replication Strategies
• Load balancing – single write, distributed read
• Load balancing – circular read/write
• High availability (hot failover)
• Snapshot backups
Master
Slave
Backup with LOCK TABLES (or single transaction)
Client
INSERT / SELECT
Client
INSERT / SELECT

12. Simple Replication Setup
• Modify my.cnf to include a unique server-id for
each node
• On master server, ensure that log-bin (binary
logging) is enabled in my.cnf
• On slave, configure login credentials on master,
either via my.cnf or CHANGE MASTER TO
statement
• Copy initial data snapshot from master to slave
• Configure initial binary log position on slave
• Start replication with SLAVE START command

13. Configure master
my.cnf
-------------
[mysqld]
server-id = 1
log-bin

14. Configure slave
my.cnf
-------------
[mysqld]
server-id = 2
master-user = someuser
master-password = secret
master-host = ip.of.master

15. Initial dataset
• Binary log provides a record of all modifications to
master database starting from a fixed point: when binary
logging was activated
• If all binary logs exist on master from initial install of
MySQL, the slave(s) can use these to bring themselves
up-to-date
• Otherwise, a snapshot of the master must be taken,
using mysqldump –master-data, to provide an initial
dataset for the slave(s)
• If only MyISAM tables are used, the LOAD DATA FROM
MASTER statement may be used on the slave(s)

16. Configure log position
MASTER mysql> SHOW MASTER STATUS;
+---------------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+---------------------------+----------+--------------+------------------+
| vmware-mirimar-bin.000002 | 79 | | |
+---------------------------+----------+--------------+------------------+
SLAVE mysql> CHANGE MASTER TO MASTER_LOG_FILE=‘vmware-
mirimar-bin.000002’, MASTER_LOG_POS=79;
SLAVE mysql> START SLAVE;

17. Using CHANGE MASTER TO
• MASTER_HOST
• MASTER_USER
• MASTER_PASSWORD
• MASTER_LOG_FILE
• MASTER_LOG_POS

18. Advanced CHANGE MASTER TO
• MASTER_PORT
• RELAY_LOG_FILE
• RELAY_LOG_POS
• MASTER_SSL
• SSL: CA/CAPATH/CERT/KEY/CIPHER

19. Confirming it works
SLAVE mysql> SHOW SLAVE STATUSG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: vmware-mirimar
Master_User: someuser
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: vmware-mirimar-bin.000002
Read_Master_Log_Pos: 79
Relay_Log_File: vmware1-mirimar-relay-bin.000002
Relay_Log_Pos: 250
Relay_Master_Log_File: vmware-mirimar-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 79
Relay_Log_Space: 250
Seconds_Behind_Master: 0

20. Permissions
• Slaves need REPLICATION SLAVE
permission on master for basic usage
• If LOAD TABLE FROM MASTER or LOAD
DATA FROM MASTER statements are
used, slave will also need SUPER and
RELOAD privileges

21. Internal Threads
• Since MySQL 4.0, replication slaves run
two threads
• IO thread continuously receives updates
from master and writes to local relay log
• SQL thread continuously executes
statements in relay log

22. IO thread isolation
• Isolating IO thread means that slave won’t
have to wait for long-executing statements
to finish executing before retrieving data
from master
• Also, slave will continue reading data from
master if a statement creates a data
conflict

23. SQL thread isolation
• SQL thread isolation allows for replication in an
environment without a continuous link between
slave and masters
• If master fails (or slave simply has no access),
the IO thread will try to reconnect endlessly
(waiting 60 seconds between attempts)
• SQL thread will continue processing relay logs
even while IO thread is unable to connect to
master

24. Master Thread
• Additionally, the master server runs the
Binlog Dump thread
• This thread is simply dedicated to
scanning the binary logs on the master
and sending updates to the connected
slave
• If this thread isn’t running, it means that
replication isn’t running – more accurately,
that no slaves are currently connected

25. Status files
• 2 status files for replication’s use
• Their use is to record the state of
replication between server shutdown and
startup
• master.info records information about the
slave’s master server
• relay-log.info records information about
the local relay logs

26. Information in master.info
• Master log file
• Read master log pos
• Master Host
• Master User
• Password (will not be shown in SHOW SLAVE
STATUS)
• Master Port
• Connect Retry
• In MySQL 4.1+, SSL options are stored if SSL is
used

27. Information in relay-log.info
• Relay log file
• Relay log pos
• Relay master-log pos
• Exec master-log pos

28. Backup master
• Master backups can be accomplished
with mysqldump
• Care must be taken to ensure the
following 2 special considerations:
1. Consistent snapshot of master date (via lock
tables for MyISAM or single transaction for
InnoDB)
2. Recording of binary log information, for use
on slaves (master-data)

29. Backup master files
• If a file-system level backup is required, care should be
taken to manually record binary log name and position
via SHOW MASTER STATUS statement.
• To ensure consistency between backup and binary log
position, the tables should be locked via FLUSH
TABLES WITH READ LOCK immediately before backup
(and SHOW MASTER STATUS)
• LEAVE THE CLIENT CONNECTED!!!
• After backup finishes, execute UNLOCK TABLES to
release the read lock

30. Backup slave
• Same idea as master file system backup
• Instead of recording position, it’s enough
to backup the master.info and relay-
log.info files
• Instead of acquiring global read lock, it’s
enough to STOP SLAVE before backup
and START SLAVE once backup finishes

31. Live demo
• Time permitting, we’ll show a short
demonstration of a simple unidirectional
replication setup

32. For more information
• MySQL documentation
• 5.0 documentation
http://mirror.mirimar.net/mysql/doc/refman/5.0/
• 4.1 documentation
http://mirror.mirimar.net/mysql/doc/refman/4.1/

33. Thank You!
For more information:
Issac Goldstand
margol@mirimar.net
http://www.beamartyr.net/
http://www.mirimar.net/