NoSQL and MySQL webinar - best of both worlds

NoSQL Access to MySQL: The Best of Both Worlds
Andrew Morgan – andrew.morgan@oracle.com – www.clusterdb.com
MySQL Product Management
1 29th March 2012
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features
or functionality described for Oracle s products remains at the sole discretion of Oracle.

2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Session Agenda

•  NoSQL – What are people looking for?
•  RDBMS – What advantages do they still have?
•  How MySQL Delivers the Best of Both Worlds
–  MySQL Cluster
•  NoSQL attributes: Scale-out, performance, ease-of-use, schema
flexibility, on-line operations
•  NoSQL APIs
–  Key-Value store access to InnoDB (Memcached)


NoSQL – Why is it needed?

2.1BN USERS
8X DATA GROWTH IN 5 YRS 850M USERS •  Web applications
70+ NEW DOMAINS EVERY 60 SECONDS
demanding:
20M APPS PER DAY
40% DATA
GROWTH PER YEAR
1 TR VIDEO
PLAYBACKS
–  Development velocity:
$1TR BY 2014
Simplicity & flexibility of data
250m TWEETS
PER DAY $700BN IN 2011 model & APIs
–  Scalability & performance:
5.9BN MOBILE SUBS IN 2010 (78%
PENETRATION)
high write throughput and
1 BILLION Ios & ANDROIDAPPS
DOWNLOADED PER WEEK
370K CALL MINUTES
EVERY 60 SECONDS Key/Value access
–  Support for “Big Data”


What NoSQL must deliver

•  Massive scalability
–  No application-level sharding
•  Performance Scalability

•  High Availability/Fault Tolerance Performance

•  Ease of use HA

–  Simple operations/administration
Ease
of
use

–  Simple APIs
–  Quickly evolve application & schema


Types of NoSQL stores

Key-‐Value
Document
Graph

• Cassandra
• MongoDB
• Neo4J

• Memcached
• CouchDB
• FlockDB

• BigTable

• Hadoop

• Voldermort


Still a role for the RDBMS?
•  No best single solution fits all
•  Mix and match
NoSQL
RDBMS

Simple
access
paGerns
Complex
queries
with
joins
Scalability

Performance

Compromise
on
consistency

ACID
transacJons
HA

for
performance

Ease
of
use

Ad-‐hoc
data
format
Well
deﬁned
schemas

SQL/Joins

Simple
operaJon
Rich
set
of
tools
ACID
Transac>ons


MySQL Cluster introduction


MySQL Cluster Architecture

REST Scalability

Application Nodes
Performance

HA

Node Group 1 Node Group 2
Ease
of
use

Node 1

F1 F2

Node 3
Cluster Cluster SQL/Joins
"
Mgr F3 F4 Mgr
ACID
Transac>ons
"
Node 2

Node 4
F3 F4
F1 F2
Data
Nodes


MySQL Cluster Architecture http://clusterdb.com/u/demo

REST Scalability

Application Nodes
Performance

HA
"
Node Group 1 Node Group 2
Ease
of
use

Node 1

F1 F2

Node 3
Cluster Cluster SQL/Joins
"
Mgr F3 F4 Mgr
ACID
Transac>ons
"
Node 2

Node 4
F3 F4
F1 F2
Data
Nodes


Scale-Out: Auto-Partitioning

Table T1 Data Node 1

P1 Data Node 2

P2
P3 Data Node 3

P4
Data Node 4



F1

P1 Data Node 2

P2
P3 Data Node 3

P4
Data Node 4



F1

P1 Data Node 2

P2 F1

P3 Data Node 3

P4
Data Node 4



F1

P1 Data Node 2

P2 F3 F1

P3 Data Node 3

P4
Data Node 4



F1 F3

P1 Data Node 2

P2 F3 F1

P3 Data Node 3

P4
Data Node 4



F1 F3

P1 Node Group 1
Data Node 2

P2 F3 F1

P3 Data Node 3
F2 F4
P4
Node Group 2
Data Node 4
F4 F2



Table T1
Scalability
"
P1 Performance

P2 HA
"
Ease
of
use

P3 SQL/Joins
"
P4 ACID
Transac>ons
"


Scale-Out Reads & Writes on Commodity Hardware
SELECT Queries per Minute
1,200
1,000
800
Millions

600
400
200
0
2 4 8
Number of Data Nodes
•  8 x Commodity Intel Servers
UPDATE Queries per Minute –  2 x 6-core processors 2.93GHz
150
–  x5670 processors (24 threads)
100
–  48GB RAM
Millions

50 •  Infiniband networking
0 •  flexAsynch benchmark (NDB API)
4 8
Number of Data Nodes


JOIN Performance 70x
More
performance

Query Execution Time Seconds
100 •  33K rows over 11 tables
90
80
•  Must Analyze tables for best
70 results
60
50
mysql> ANALYZE TABLE <tab-name>;
40
30 Scalability
"
20
10 Performance
"
0
HA
"
MySQL Cluster 7.1 MySQL Cluster 7.2
Ease
of
use

SQL/Joins
"
ACID
Transac>ons
"


Creating & running your first Cluster
The traditional way (pre-MCM) – Up and running in 15 mins

Download
&
Conﬁgure
Start
processes

Extract
•  Cluster-‐wide
•  Management
Nodes

•  edelivery.oracle.com
“conﬁg.ini”
•  Data
Nodes

•  www.mysql.com
•  Per-‐mysqld
“my.cnf”
•  MySQL
Servers

•  dev.mysql.com

•  Up & running in 10-15 minutes using Quick Start guides from http://dev.mysql.com/downloads/cluster/
–  Versions for Linux, Windows & Solaris


Scalability
"
Performance
"
HA
"
Ease
of
use
"
SQL/Joins
"
ACID
Transac>ons
"


MySQL Cluster Manager
Bootstrap single host Cluster
1.  Download MCM/Cluster package from edelivery.oracle.com:
2.  Unzip
3.  Run agent, define, create & start Cluster!
$> binmcmd –bootstrap
MySQL Cluster Manager 1.1.2 started
Connect to MySQL Cluster Manager by running "D:AndrewDocumentsMySQLmcmbinmcm" -a NOVA:1862
Configuring default cluster 'mycluster'...
Starting default cluster 'mycluster'...
Cluster 'mycluster' started successfully
ndb_mgmd NOVA:1186
ndbd NOVA
ndbd NOVA
mysqld NOVA:3306
mysqld NOVA:3307
ndbapi *
Connect to the database by running "D:AndrewDocumentsMySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root


MCM: Upgrade Cluster

mysql> upgrade cluster
--package=7.1 mycluster;


MCM: On-Line Add-Node
Scale out with no loss of service

mysql> add hosts --hosts=192.168.0.35,192.168.0.36 mysite;
mysql> add package --basedir=/usr/local/mysql_7_1_9a –
hosts=192.168.0.35,192.168.0.36 7.1;
mysql> add process --
processhosts=mysqld@192.168.0.33,mysqld@192.168.0.34,ndbd@192.
168.0.35,ndbd@192.168.0.36 mycluster;
mysql> start process --added mycluster;


On-line Schema changes
On-Line Operations
•  Scale the cluster (add & remove nodes on-line)
•  Repartition tables
•  Upgrade / patch servers & OS
•  Upgrade / patch MySQL Cluster
•  Back-Up
•  Evolve the schema on-line, in real-time


Scalability
"
Performance
"
HA
"
Ease
of
use
"
SQL/Joins
"
ACID
Transac>ons
"


NoSQL Access to MySQL Cluster data

Mix & Match!
Same data accessed
simultaneously through
SQL & NoSQL interfaces


MySQL Cluster 7.1: ClusterJ/JPA
•  New Domain Object Model Persistence API (ClusterJ) :
–  Java API
–  High performance, low latency
–  Feature rich
•  JPA interface built upon this new Java layer:
–  Java Persistence API compliant
•  Implemented as an OpenJPA plugin
–  Uses ClusterJ where possible, reverts to JDBC for some
operations
–  Higher performance than JDBC
–  More natural for most Java designers
–  Easier Cluster adoption for web applications


ClusterJ
•  High Performance, Easy to Use
•  In the style of Hibernate / JPA / JDO
•  Domain Object Model DataMapper pattern
–  Data is represented as domain objects
–  Domain objects are separate from business logic
–  Domain objects are mapped to database tables
•  Built on ndbjtie
–  JNI adapter
–  integral part of MySQL Cluster
–  Straight mapping of MySQL Cluster API (a.k.a NDB API) to Java
•  Does not support relationships
–  Look at JDO / JPA for these modelling patterns


NoSQL with Memcached (MySQL Cluster 7.2)
•  Memcached is a distributed memory based hash-key/
value store with no persistence to disk
•  NoSQL, simple API, popular with developers
•  MySQL Cluster already provides scalable, in-memory
performance with NoSQL (hashed) access as well as
persistence
–  Provide the Memcached API but map to NDB API calls
•  Writes-in-place, so no need to invalidate cache
•  Simplifies architecture as caching & database integrated
into 1 tier
•  Access data from existing relational tables


Traditional Memcached Architecture
Two levels of hashing

httpd memcached

hash key
PHP/Perl ver memcached
pick ser to find data
key to
Memcache hash
friends:12389! memcached
memcache key


Cache hit

httpd

hash key
PHP/Perl se rver memcached
to find data
ke yto pick
Memcache hash
friends:12389! VALUE friends:12389 0 31rn!
101, 11009, 11150, 55881, 77798 rn!


Cache miss (1): fetch from DB

httpd

hash key
PHP/Perl ver memcached
memcache
pick ser to find data
key to
Memcache hash FOUN
D
NOT
mysql
SELECT
fri
FROM u end_id ! MySQL
ser_frien
WHERE ds !
user_id = Slave
?!


Cache miss (2): manage cache

httpd

hash key
PHP/Perl memcached
to find data
Memcache ! !
31 rn 798 rn
s:12389 5881, 77
riend , 11150, 5
set f 009
11
101,


Data change (1): Write to DB

httpd

PHP/Perl
mysql
INSERT INTO user_friends !
(user_id, friend_id) !
VALUES ( 12389, 999101);!
MySQL
Master


Data change (2): manage cache

httpd

PHP/Perl memcached
mysql rn!
s:1 2389
te f riend
dele
MySQL
Master


NoSQL with Memcached

•  Flexible: set maidenhead 0 0 3
SL6
–  Deployment options
STORED
–  Multiple Clusters
–  Simultaneous SQL Access get maidenhead
–  Can still cache in Memcached server VALUE maidenhead 0 3
–  Flat key-value store or map to multiple tables/ SL6
columns END


Schema-Free apps
•  Rapid application evolution
–  New types of data constantly
added
–  No time to get schema
extended
–  Missing skills to extend
schema
–  Initially roll out to just a few
users
–  Constantly adding to live
system


Cluster & Memcached – Schema-Free

key value
<town:maidenhead,SL6>
Application view
SQL view key value


Key
Value

town:maidenhead
SL6

generic table


Cluster & Memcached - Configured Schema

key value
Application view
SQL view prefix key value


Preﬁx
Table
Key-‐col
Val-‐col
policy
town
...
code
...

town:
map.zip
town
code
cluster
maidenhead
...
SL6
...

Config tables map.zip


Memcached with MySQL Cluster
Try it out

http://clusterdb.com/u/memcached


Memcached NoSQL Access with InnoDB
•  Memcached as a plugin of MySQL Server;
same process space, with very low latency
Application access to data
•  Memcapable: supports both memcached
SQL Memcached protocol
ascii protocol and binary protocol
mysqld •  Support multiple columns: users can map
multiple columns into “value”
memcached plugin
MySQL Server •  Optional local caching: “innodb-only”,
local cache
“cache-only”, and “caching”
innodb_memcache
(optional)
•  Batch operations for performance
Handler API InnoDB API
•  Available from labs.mysql.com. Is not GA.

InnoDB Storage Engine


Which API to use?


Scalability
"
Performance
"
HA
"
Ease
of
use
"
SQL/Joins
"
ACID
Transac>ons
"


Next Steps
•  Guide to MySQL and NoSQL - Delivering the Best of
Both Worlds
–  http://mysql.com/why-mysql/white-papers/mysql-wp-guide-to-
nosql.php
•  Evaluate MySQL Cluster 7.2
–  http://www.mysql.com/downloads/cluster/
•  Bootstrap a Cluster
–  https://edelivery.oracle.com/
•  Try Memcached API for InnoDB
–  http://labs.mysql.com/

Summary

Today’s web workloads demand more from
databases

Performance, scale-out, simples access
patterns & APIs

MySQL meets these needs while still
delivering benefits of an ACID RDBMS


MySQL Cluster 7.2 GA
Enabling
Next
GeneraJon
Web
Services

•  70x
Higher
Complex
Query
Performance:
AdapJve
Query
LocalizaJon

•  NaJve
Memcached
API

•  MySQL
5.5
Server
IntegraJon

•  MulJ-‐Threaded
Data
Node
Extensions

•  VM
Support

Enhancing
Cross
Data
Center
Scalability

•  MulJ-‐Site
Clustering

•  Simpliﬁed
AcJve
/
AcJve
ReplicaJon

Ease
of
Use

•  MySQL
Cluster
Manager
1.1.4

•  Consolidated
Privileges


NoSQL and MySQL webinar - best of both worlds

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à NoSQL and MySQL webinar - best of both worlds

Similaire à NoSQL and MySQL webinar - best of both worlds (20)

Plus de Mat Keep

Plus de Mat Keep (7)

Dernier

Dernier (20)

NoSQL and MySQL webinar - best of both worlds