Architecture and Design of MySQL
Powered Applications
Peter Zaitsev
May 8, 2014
About Presentation
What's in your
Toolbox ?
What Questions
you should ask ?
Common MySQL
Architecture
Patterns
2
LiveJournal Architecture 10 Years Ago
3
10 Years ago we see
Load Balancers
FrontEnd vs BackEnd Web Server Separation
MemcacheD (Caching)
Global Database for Share...
History Lesson
MySQL is quite Mature
Battle tested Architecture Patterns
Most Problems have proven solutions
5
YOUR TOOLBOX NOW
Well… certain things have changed
6
Very Powerful Hardware
• 16GB to 512GB
Lots of
Memory
• 1000 IOPS to 100.000 IOPS
• 10ms latency to 0.1ms IO Latency
Very ...
Reasonable Software
•MySQL
•Linux Kernel
•File Systems
•Etc..
Can use
this
Hardware
Pretty
Well
8
Cloud
Dynamic Scalability
Throw Away Server Mentality
Agility
Less involved Operations
9
Automation
Development Testing Deployment Operations
10
New Languages and Frameworks
More Agile
Less “system” code
More Client Side Development
Less Database Dependence
ORM
11
Multiple “Clients”
Desktop Web Browsers
Mobile Web Browsers
“Apps”
API
12
Not Only “MySQL”
• Memcache, Redis
• RabbitMQ
• Cassandra, MongoDB
• Hadoop
• Column Store Engines,
Vertica
Right
Tools
fo...
More HA Options
Mature MySQL Replication
Percona XtraDB Cluster, Galera
MySQL Cluster
Proprietary cloud HA
14
Ready solutions for Replication Management
MHA
PRM
Continuent Tungsten
15
Making Sharding Better
• Clustrix
• MemSQL
• ScaleDB
Not Quite
MySQL
• ScaleArc, ScaleBase, Tesora
• MySQL Proxy, MaxScale...
ARCHITECTURE QUESTIONS
17
Basics
Make some Right decisions Early
Your first Architecture will not be perfect
Some poor choices are very expensive to...
Consultants are like Lawyers
You can waste a lot of money with
them
Getting right advice on the right topics
will save you...
Dimensions to consider
Application Scale
High Availability
Team Experience
20
Approach to Scaling
Avoid Over-
Engineering
Give yourself
some
runway
Do Some
Capacity
Planning
21
High Availability
Real High Availability is Really Expensive
Medium Availability is getting more
affordable
People are oft...
Team Experience
Chose Solutions which your Team can Support
Often Developers know little about Databases
Right Development...
Simplicity
• Less complexity lower costs
Use fewer
components when
possible
• Monitor
• Troubleshoot
Each components
needs...
World of “Small” Applications
• Really
Majority of
Applications are
Small
• Especially with support of caching
May well ru...
Lets do Some Math
Modern
MySQL
Instance
Handle 100K
QPS
20 Queries
per User
Interaction
5K User
Interactions
Per Second
14...
PRACTICAL CHOICES
Basic Architecture Patterns
27
Baseline
Single MySQL Instance
No Caching
No HA
No Supplemental Technologies
28
Simple High Availability
HA with DBaaS (Amazon RDS)
HA with MySQL Replication
HA with Percona XtraDB Cluster
29
What do you need to Scale ?
Reads
Writes
Data
Size
30
Load Management Tricks
Move the load
In Time (Doing
Batch Processing
at Night)
In Space (Use
Slave for
Reports)
31
Queueing
Used by Most high volume architectures
Deal with Load Spikes
Increase Reliability
Scale parts of the system which...
Scaling Reads
Replication (and Read Write Splitting)
Caching (Memcache, Redis etc)
Percona XtraDB Cluster (Read Anywhere)
...
Scaling Writes
• Using Different Servers for
Different Parts of
Application
Functional
Partitioning
• Horizontal Partition...
Scaling Data Size
•Depends on the
Application and
Needs
•10TB good rule of
thumb
How Much
We Can
Practically
Store in
Sing...
How Is it Possible
“Online” Schema
Change
MySQL 5.6
Pt-online-schema-
change (Percona
Toolkit)
Fast Backup
Percona
Xtrabac...
Summary
MySQL is Mature. Solutions Available for
Many Problems
Make few choices right in the beginning
Hardware and Softwa...
38
Thank You!
Peter Zaitsev
pz@percona.com
Prochain SlideShare
Chargement dans…5
×

Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014

821 vues

Publié le

Palestra do Peter Zaitsev no Meetup MySQL-BR de maio-2014 sobre Arquiteturas e Projeto de Aplicações com MySQL

Publié dans : Technologie
0 commentaire
1 j’aime
Statistiques
Remarques
  • Soyez le premier à commenter

Aucun téléchargement
Vues
Nombre de vues
821
Sur SlideShare
0
Issues des intégrations
0
Intégrations
4
Actions
Partages
0
Téléchargements
12
Commentaires
0
J’aime
1
Intégrations 0
Aucune incorporation

Aucune remarque pour cette diapositive

Architecture and Design MySQL powered applications by Peter Zaitsev Meetup Sao Paulo May2014

  1. 1. Architecture and Design of MySQL Powered Applications Peter Zaitsev May 8, 2014
  2. 2. About Presentation What's in your Toolbox ? What Questions you should ask ? Common MySQL Architecture Patterns 2
  3. 3. LiveJournal Architecture 10 Years Ago 3
  4. 4. 10 Years ago we see Load Balancers FrontEnd vs BackEnd Web Server Separation MemcacheD (Caching) Global Database for Shared Data Read Write Splitting Sharding “Database Clusters” 4
  5. 5. History Lesson MySQL is quite Mature Battle tested Architecture Patterns Most Problems have proven solutions 5
  6. 6. YOUR TOOLBOX NOW Well… certain things have changed 6
  7. 7. Very Powerful Hardware • 16GB to 512GB Lots of Memory • 1000 IOPS to 100.000 IOPS • 10ms latency to 0.1ms IO Latency Very Fast Storage • 4 Cores to 32 Cores Many CPU Cores 7
  8. 8. Reasonable Software •MySQL •Linux Kernel •File Systems •Etc.. Can use this Hardware Pretty Well 8
  9. 9. Cloud Dynamic Scalability Throw Away Server Mentality Agility Less involved Operations 9
  10. 10. Automation Development Testing Deployment Operations 10
  11. 11. New Languages and Frameworks More Agile Less “system” code More Client Side Development Less Database Dependence ORM 11
  12. 12. Multiple “Clients” Desktop Web Browsers Mobile Web Browsers “Apps” API 12
  13. 13. Not Only “MySQL” • Memcache, Redis • RabbitMQ • Cassandra, MongoDB • Hadoop • Column Store Engines, Vertica Right Tools for Right Job! 13
  14. 14. More HA Options Mature MySQL Replication Percona XtraDB Cluster, Galera MySQL Cluster Proprietary cloud HA 14
  15. 15. Ready solutions for Replication Management MHA PRM Continuent Tungsten 15
  16. 16. Making Sharding Better • Clustrix • MemSQL • ScaleDB Not Quite MySQL • ScaleArc, ScaleBase, Tesora • MySQL Proxy, MaxScale, ProxySQL Proxy Solutions • Vitess • JetPants • MySQL Fabric OpenSource Frameworks 16
  17. 17. ARCHITECTURE QUESTIONS 17
  18. 18. Basics Make some Right decisions Early Your first Architecture will not be perfect Some poor choices are very expensive to fix 18
  19. 19. Consultants are like Lawyers You can waste a lot of money with them Getting right advice on the right topics will save you a lot of trouble in the end 19
  20. 20. Dimensions to consider Application Scale High Availability Team Experience 20
  21. 21. Approach to Scaling Avoid Over- Engineering Give yourself some runway Do Some Capacity Planning 21
  22. 22. High Availability Real High Availability is Really Expensive Medium Availability is getting more affordable People are often leading cause of downtime 22
  23. 23. Team Experience Chose Solutions which your Team can Support Often Developers know little about Databases Right Development, Operational Process Is a Key Tools to Support your Process • Percona Toolkit, Percona Cloud Tools • Systematically find and review new queries 23
  24. 24. Simplicity • Less complexity lower costs Use fewer components when possible • Monitor • Troubleshoot Each components needs experience to • “Follow the Crowd” • Unless you have a reason not to There is safety in numbers 24
  25. 25. World of “Small” Applications • Really Majority of Applications are Small • Especially with support of caching May well run on single database instance • Active intranet site for company with 200K employees • Ecommerce site supporting $10M+ in Sales Such “Small” Applications can be significant 25
  26. 26. Lets do Some Math Modern MySQL Instance Handle 100K QPS 20 Queries per User Interaction 5K User Interactions Per Second 144M User Interactions a Day (Considering 3x peak to average) 10M Daily Active users (with Medium Engagement) 26
  27. 27. PRACTICAL CHOICES Basic Architecture Patterns 27
  28. 28. Baseline Single MySQL Instance No Caching No HA No Supplemental Technologies 28
  29. 29. Simple High Availability HA with DBaaS (Amazon RDS) HA with MySQL Replication HA with Percona XtraDB Cluster 29
  30. 30. What do you need to Scale ? Reads Writes Data Size 30
  31. 31. Load Management Tricks Move the load In Time (Doing Batch Processing at Night) In Space (Use Slave for Reports) 31
  32. 32. Queueing Used by Most high volume architectures Deal with Load Spikes Increase Reliability Scale parts of the system which need to Many solutions • RabbitMQ • Redis • Gearman 32
  33. 33. Scaling Reads Replication (and Read Write Splitting) Caching (Memcache, Redis etc) Percona XtraDB Cluster (Read Anywhere) 33
  34. 34. Scaling Writes • Using Different Servers for Different Parts of Application Functional Partitioning • Horizontal Partitioning over many serversSharding 34
  35. 35. Scaling Data Size •Depends on the Application and Needs •10TB good rule of thumb How Much We Can Practically Store in Single MySQL Instance these days 35
  36. 36. How Is it Possible “Online” Schema Change MySQL 5.6 Pt-online-schema- change (Percona Toolkit) Fast Backup Percona Xtrabackup MySQL Enterprise Backup Modern Filesystem/Volume Managers Fast Network 10Gbps affordable High Performance Storage Flash/SSD Compression Innodb TokuDB 36
  37. 37. Summary MySQL is Mature. Solutions Available for Many Problems Make few choices right in the beginning Hardware and Software advances allow to go a long way with simple architectures 37
  38. 38. 38 Thank You! Peter Zaitsev pz@percona.com

×