The OSSCube MySQL High Availability Tutorial

MySQL HA Tutorial

Presented by – Sonali Minocha
and Rakesh Kumar
OSSCube

Who Am I?

MySQL Cluster Tutorial © OSSCube

What is a Cluster?

MySQL Cluster Tutorial, © OSSCube

Single MySQL Server Architecture

SQL Request
MySQL Server

MySQL Client
MySQL Client
Response


MySQL Cluster Architecture
NDB API Call
SQL Request
MySQL Client

Partitioning
Replication
Message

Response

Response
SQL Node (MySQL server)


Data Nodes

Components of MySQL Cluster


Various Nodes
API Node

Data
Node

Application

Management Node


NdbCluster
Engine

NdbCluster
Engine

(Data Nodes)

MySQL Cluster Tutorial,© OSSCube

NdbCluster
Engine

Organization


Partition


Partitioning


Fragments

MySQL Cluster Tutorial© OSSCube

Node Groups


Split Brain

I have full
responsibility
of the cluster

Servers can
not see
each other

I have full
responsibility
of the cluster


Arbitration

NG 1

Network
Split
First Node to ask will continue while the other will be shut
down

More Data Nodes
NG 1
Network
Split/
Shutdown
NG 2


Arbitrator Issues


Normal Operation

Ia
Al m
iv
e

Ia
Al m
iv
e

m
I a ive
Al

Ia
A m
liv
e

Missed Heartbeats

Ia
Al m
iv
e

I am
Alive

m
I a ive
Al

Synchronous


Phase One: CommitRequest
Commit

Commit

Message

Message

Response

Response

Message

Message


Phase Two: Successful
Commit

Transaction

Transaction

Successful

Successful
Commit

Commit

Transaction

Transaction


Phase Two: Failure
(Abort Commit)

Transaction

Transaction

Failed

Successful
Rollback

Rollback

Transaction

Transaction


Initiate MySQL Cluster
•
•
•

•

•
•
•
•

# file "config.ini" - 2 data nodes and 2 SQL nodes# This file
is placed in the startup directory of ndb_mgmd (the#
management server)
# The first MySQL Server can be started from any host. The
second# can be started only on the host MySQLd_5.MySQL.com
[NDBD DEFAULT]
NoOfReplicas= 2
DataDir= /var/lib/MySQL-cluster
[NDB_MGMD]
Hostname= ndb_mgmd.MySQL.com
DataDir= /var/lib/MySQL-cluster
[NDBD]
HostName= ndbd_2.MySQL.com
[NDBD]
HostName=ndbd_3.MySQL.com
[MySQLD]
[MySQLD]
HostName= MySQLd_5.MySQL.com

Management Node


Data Nodes
• my.cnf# example additions to
my.cnf for MySQL Cluster
• [ndbd]
connectstring=ndb_mgmd.MySQL.com


Data Node Configurations
Options


SQL Nodes
• my.cnf
• # example additions to my.cnf for
MySQL Cluster
# (will work on all versions)
• # enable ndbcluster storage engine,
and provide connectstring for
management
• # server host to the default port
1186
• [mysqld]
Ndbcluster
ndbconnectstring=ndb_mgmd.MySQL.com
:1186

Config.ini Parameters
General Parameters
– Id
– Hostname
File and Directory Location Parameter
– BackupDataDir
– DataDir
– FileSystemPath


•Transaction Handling Parameters
•TransactionInactiveTimeout
•TransactionDeadlockDetectionTimeou
t

•Memory Usage Parameters
•LockPagesInMainMemory

•Utility Programs
•ndb_size.pl Cluster Tutorial, © OSSCube
MySQL

Monitoring the Startup Phases


MySQL Cluster Storage Engine
--NDB

NDB Storage Engine


NDB’s Storage Method
• In-Memory Storage
• Disk-Based Storage
– New in MySQL 5.1
– Undo Logging
– Fixed-Width Columns


Tablespace

Datafile

Tablespace
Datafile Datafile

Datafile

Log File group
Redo log Undo Log

Indexes


Creating Indexes
•

Each table in the cluster always has a hash index for a Primary Key

•

Ordered indexes can be created with ALTER TABLE ADD INDEX ....


First Node/First Fragment
•
•

Hidden field/table created that will handle the management of the
Unique Index on Name
Ordered Index (T-Tree) created for Continent field

Hidden
Field

Code_Hash
1351
1785
1943
2031
2345

Code
CHE
CHN
FIN
GOR
SKO

Name
Switzerland
China
Finland
United Kingdom
South Korea

Continent
Europe
Asia
Europe
Europe
Asia

Ordered
Index

Hidden
Table


Second Node/Second Fragment
•
•

Same as First Node/Fragment setup
Contains the remaining data

Hidden
Field

Code Hash
1231
1853
2145
2197

Code
BRA
ESP
IND
MOR

Name
Brazil
Spain
India
Morocco

Continent
South America
Europe
Asia
Africa

Ordered
Index

Hidden
Table


Unique Key Lookup
Primary Key Lookup may
UniqueKey Lookup network hop
require another

PK values

Function()

Hash

Look up row

Data node
Data node

Look up row

Data node
Data node

MySQL Server


Parallel Ordered Index
Scan

Data node
Data node

TTTree
Tree
Index
Index

rows

MySQL
Server

Data node
Data node


TTTree
Tree
Index
Index

Parallel full table scan
Data node
Data node

rows

MySQL
Server

Data node
Data node

MySQL Cluster Tutorial, OSSPAC 09
Singapore, © OSSCube

Parallel full table scan ( WHERE
condition processed in SQL
node)
SQL Query
Data node
Data node
Few Rows
Few Rows

Many Rows
rows Rows
Many

MySQL
Server

Where
Conditi
on


Data node
Data node

( WHERE condition
processed in data node)
SQL Query
Where
Conditi
Fewrows on
Rows
Few Rows

Few Rows
Few Rows

Data node
Data node

MySQL
Server
Where
Conditi
on


Data node
Data node

Storage of Indexes


Characteristics of NDB
storage Engine


Durability


Global Check Point
• A GCP occurs every few seconds, when
transactions for all nodes are synchronized
and the REDO log is flushed to disk


Local Check Point


Memory - RAM
•

Memory Allocation
Total Memory = N(data nodes)* Local Memory /
NoOfReplicas
-- or -Local Memory = Total Memory * NoOfReplicas / N(data
nodes)

•

Sizing up the
Memory

– Fixed size in memory
– Variable sized in memory
– Fixed size on disk

Memory Pages


HANDLING DATA IN MySQL
CLUSTER

Disk Data Objects


Creating the Log File Group
• Create Log File Group
CREATE LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_1.dat'
INITIAL_SIZE 16M
UNDO_BUFFER_SIZE 2M
ENGINE NDB;

• Adding Undo Files to Log File Group
ALTER LOGFILE GROUP lg_1
ADD UNDOFILE 'undo_2.dat'
INITIAL_SIZE 12M
ENGINE NDB;

Verifying UNDO Files are
Created

• INFORMATION_SCHEMA Tables
SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER,
EXTRA
FROM INFORMATION_SCHEMA.FILES
WHERE FILE_NAME = 'undo_1.dat';
SELECT LOGFILE_GROUP_NAME, LOGFILE_GROUP_NUMBER,
EXTRA
WHERE FILE_NAME = 'undo_2.dat';


Creating the Tablespace
• Create a Tablespace
CREATE TABLESPACE ts_1
ADD DATAFILE 'data_1.dat'
USE LOGFILE GROUP lg_1
INITIAL_SIZE 32M
ENGINE NDB;

• Add an Additional Data File
ALTER TABLESPACE ts_1
ADD DATAFILE 'data_2.dat'
INITIAL_SIZE 48M
ENGINE NDB;

• Verifying Data Files are Created
SELECT FILE_NAME, LOGFILE_GROUP_NAME, EXTRA
WHERE TABLESPACE_NAME = 'ts_1' AND FILE_TYPE =
'DATAFILE';

Create a Disk Data Table
• Non-Indexed Columns Stored on Disk in Tablespace
CREATE TABLE dt_1 (
memberId INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
lName VARCHAR(50) NOT NULL, fName VARCHAR(50) NOT NULL,
dob DATE NOT NULL, joined DATE NOT NULL,
INDEX(lName, fName)
) TABLESPACE ts_1 STORAGE DISK ENGINE NDB;

• Alter Existing Non-Cluster Tables
ALTER TABLE city TABLESPACE ts1 STORAGE DISK ENGINE=NDB;

• View Table Status
SHOW TABLE STATUS LIKE 'city'G


Monitoring Free Space


Dropping Cluster Tables


MySQL Cluster’s Native
Backup Tool
• Backup Data
– .ctl
– .log
– .data

Data
Node

• Abort Backup

Data
Node

Backup-1.2.ctl
Backup-1.2.log
Backup-1.0.2.Data
Backup-1.1.2.Data

Backup-1.3.ctl
Backup-1.3.log
Backup-1.0.3.Data
Backup-1.1.3.Data

Data
Data
ndb_mgm> ABORT BACKUP 6
Node
Node
Abort of backup 6 ordered
Node 2: Backup 10 started from 1 has
been aborted. Error: 1321


MySQLdump


Restoring Backups
• Recovering MySQL Cluster's Native Backup Files (1/3)

– Verify Management Node is Running
shell> ndb_mgm -e "SHOW"

– Start and Empty the Data Nodes
• Start each data node with the --initial option.


Restoring Backups


Restoring MySQLdump Files


EXPLAIN


Full Table Scan
• Without Condition Pushdown
• With Condition Pushdown


( WHERE condition
processed in SQL node)
SQL Query
Few Rows

Where
Condition

Data node
Many
rows Rows

MySQL
Server


Data node

Parallel full table scan ( WHERE
condition processed in data
node)
SQL Query

Where
Condition
Data node

Few Rows

Few Rows
rows

MySQL
Server


Where
Data
Condition node

Index Statistics


MySQL Cluster Query Cache


Data Size/Usage Issues


Query Design


MySQL Replication
MySQL Server
Slave

Application W

MySQL Server
Master

Binlog

Relaylog
IO thread

SQL thread

MySQL Server
Slave

Relaylog


Binlog

mysqld

mysqld

Update

Data node

Update

Data node

Cluster
Data node

Data node
Update

Update
Ndb API

Ndb API


NDB Injector Thread
• A thread inside MySQL server
• Responsible for injecting rows in binlog and
produces “Single connical binlog for cluster”
• not just one MySQL Server
• It contains everything written on all ndbApi
program including MySQLd connected to cluster


MySQL Cluster Replication
MySQLd(slave)
MySQLd(Master)

Replication

NdbClsuter Handler

IO thread

Binlog
Data node
Cluster
Data node

Data node
Data node

Binlog

NdbClsuter Handler
Relaylog

Apply thread
Data node

Data node

Cluster
Data node

Data node


Thank you for your time and attention
For more information, please feel free to drop in a line to
sales@osscube.com or visit http://www.osscube.com
Or call us at 1-888-9OSSCube (Toll Free for USA)
1-919-791-5472 (From outside USA)


The OSSCube MySQL High Availability Tutorial

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à The OSSCube MySQL High Availability Tutorial

Similaire à The OSSCube MySQL High Availability Tutorial (20)

Plus de OSSCube

Plus de OSSCube (20)

Dernier

Dernier (20)

The OSSCube MySQL High Availability Tutorial