Contenu connexe
Similaire à Scaling MySQl 1 to N Servers -- Los Angelese MySQL User Group Feb 2014 (20)
Scaling MySQl 1 to N Servers -- Los Angelese MySQL User Group Feb 2014
- 2. MySQL Scaling:
Drinking from the fire hose,
never being down & serving
billions
Dave Stokes
MySQL Community Manager
David.Stokes@Oracle.com @Stoker
Slideshare.net/davestokes/presentations
2Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Insert Picture Here
- 3. You want to get a bigger, badder MySQL -BUT what are the costs???
Budget
Complexity
Application
Opportunity
Sanity
3
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 5. Option 0
Run the Latest, Greatest MySQL
MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15%
--Make sure your config file matches your hardware
--ANALYSE TABLE
--Check for SLOW QUERIES, QUERIES w/o INDEXES
--TUNE QUERIES
This is the least complex, less upsetting, and usually lowest budget cost option
5
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 6. Option 0.1
Add in a Memcached Layer
Cache for quick access
●
Session Data
●
Sports Scores
●
Opens up NoSQL access
to InnoDB later
This is the second least complex, less upsetting, and usually lowest budget
cost option but does make you change your application to become 'cache
aware'.
6
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 7. What database service
Levels do you Need to
Provide?
7
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Insert PicturePicture Here
Insert Here
- 8. UpTime versus Response Time versus Access
Things to be engineered/negotiated upfront
Up Time
– How much down time can you afford?
Response Time
– How long is too long to wait?
Access
– How many do you need to serve simultaneously
8
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 9. Simple Level 1
Read/Write Splitting
9
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Insert PicturePicture Here
Insert Here
- 10. Use MySQL Replication to Split Read/Writes
10
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 11. Read/Write Splitting
Most Common Scaling Technique
●
More complex environment
●
Application Changes
●
Easier with PHP & Java
●
●
11
Low opportunity and sanity costs
Less risk of one server stopping operations
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 12. Read/Write Splitting—Easy Way
Use MySQL Utilities to copy master, set up failover
●
mysqlserverclone
●
mysqlfailover
●
●
12
When master dies, most up to date slave (GTID) promoted
automatically
Mysqlrplcheck & mysqlrplshow
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 13. Simple Level 2
Moderately Easy, Better
Availability
13
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Insert PicturePicture Here
Insert Here
- 14. Distributed Replicated
Block Device (DRBD)
High Availability
●
More complex environment
●
Proven, supported
●
●
14
Please practice fail over
and back
Easy to manage but
not HA panacea
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 15. CLOUD
High Availability, Low Cost (maybe)
●
Slightly more complex environment
●
Proven, supported
●
●
15
Best for ephemeral data, may not make upper management
comfortable (or you)
Easy to manage but
not HA panacea
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 16. Complex Level N^N
Never, Ever Down
16
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Insert PicturePicture Here
Insert Here
- 17. MySQL Cluster
For 99.999% uptime, upper bound on response time
●
●
●
17
Extremely Complex
No single point of
failure = lots of $$
hardware
Not general
purpose MySQL
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
●
Auto sharding
●
Add/Subtract nodes at will
●
●
Replicate between
geographically separated
locations
Need a smart staff
- 18. MySQL Cluster Shards Data Automatically
●
●
18
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Single or multiple data node
failures do not bring down a
cluster
Easy to add/remove data
nodes
- 21. Ask ten DBAs to define 'Big Data' and You Will
Get At Least Eleven Answers
.BIG as massive amounts
●
Define MASSIVE
BIG as lots of eyes
●
21
How many simultaneous users
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
BIG as in fast
●
Define fast
BIG as all of the other three!!!
- 22. The Easiest Way to Get Speed
●
NoSQL/Memcached to InnoDB/NBD
●
●
22
9x faster by skipping parser, optimizer
Key/Value pair access
and SQL access of same data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 23. Easiest Way to Gain Size
●
Columnar Storage Engines for MySQL
●
Not InnoDB
●
Unique features, quirks
●
●
23
Need > 26,000,000,000 rows to start seeing 'sweet spot'
Use new server, don't scrimp
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 24. Calpont
InfiniDB
●
●
●
●
●
24
Column-oriented
MySQL front end
Multi threaded
High speed data loading
Automatic partitioning
Enterprise Features
●
Partition Drop
●
UDF’s
●
Physical compression
●
MPP, multi-node scale-out
●
Distributed, Shared Nothing data cache
●
Auto Increment
●
Centralized database and operating system
monitoring
●
Notifications and Alerts
●
INSERT INTO...SELECT FROM...
●
Views on InfiniDB Tables
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 26. Hadoop-- getting data from MySQL to Hadoop
●
80% of Hadoop Installations feed from MySQl
●
Hadoop is made up of several components
●
Twisty little maze, all alike
●
Use Apache Bigtop
–
●
Tested for interoperability
Cloudera & Horton Works have great tutorials
●
●
26
Kinda 'SQL-ish'
Squoop and MySQL Hadoop Applier to move data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 28. Partitioning and Sharding /
Dividing and Conquering
●
Partitions built-in to MySQL
●
Nice management tool
–
●
28
Fast drops
Sharding
●
Rolling quarters/months
–
●
No change to application but
higher complexity, low
opportunity cost
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Much more complex
application and operations
●
Not off shelf
●
Hard to re-engineer
●
Works
- 29. Hard Part – Thinking
Insert PicturePicture Here
Insert Here
29
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
- 30. Goals – What are you trying to
accomplish
●
●
Define problem completely
Who produces, who consumes
data
●
What does end solution
look/feel like?
●
●
30
Negotiate performance rates
Plan for failure, catastrophe
Design for long life
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Platforms
●
Speed
●
Access areas
●
●
●
Security
●
Serenity
- 31. Where Else to Seek Info
●
MySQL Connect Conference
Sept. in San Francisco
●
Hadoop
●
Same time as OOW
Weekdays!
●
31
Planet.mysql.com and
forums.mysql.com
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
●
Horton Works and Cloudera
both have wonderful tutorials
Apache Bigtop project
- 32. MySQL Training
●
MySQL Marinate
●
Online Virtual Class run by
Boston MySQL Users Group
●
●
32
1 chapter/week
Free
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
●
Oracle and other vendors
●
Dozens of book
●
Online