Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Massively Scaling MySQL Using Vitess

269 vues

Publié le

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2ELwiu4.

Sugu Sougoumarane gives an overview of the salient features of Vitess, and at the end, covers some advanced features with a demo. Filmed at qconsf.com.

Sugu Sougoumarane is the Co-Founder and CTO at PlanetScale. He is the co-creator of Vitess, and has been working on it since 2010. Prior to Vitess, he worked on scalability at YouTube and was also part of PayPal in the early days. His recent interest is in distributed systems and consensus algorithms.

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Massively Scaling MySQL Using Vitess

  1. 1. Sugu Sougoumarane, Co-creator CTO, PlanetScale @ssougou ssougou.blogspot.com Vitess@QCon
  2. 2. InfoQ.com: News & Community Site • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ vitess
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
  4. 4. Database trends ● Transactional data explosion ● Move to cloud
  5. 5. Transactional data options ● Single instance RDBMS ● NoSQL ● App Sharded RDBMS ● NewSQL
  6. 6. Cloud ● Instances go down all the time ○ Good master promotion story ● Durability through replication ○ Semi-sync ● Small instances ● Discovery mechanism
  7. 7. Vitess ● Leverage MySQL ● Massive Scale ● Battle-hardened ● Runs “anywhere”
  8. 8. pxhere CC0 Vitess stats Started 2010 6,000+ stars 15,000+ commits Marquee adopters 100+ contributors 500+ Slack members
  9. 9. Key adopters 20% Migrated to Vitess 10,000+ Tablets on Kubernetes Cash App fully runs on Vitess
  10. 10. The Community In production Evaluating Quiz of Kings
  11. 11. MySQL Community Award “Application of the year 2018”
  12. 12. Growing with Vitess Vitess app server app server app server MainDB Connection pooling Deadlines Hot row protection Row count limit Blacklisting
  13. 13. Growing with Vitess Vitess app server app server app server Master Replica Replica routing Load balancing Master promotion with Orchestrator
  14. 14. Growing with Vitess Vitess app server app server app server DB2 Replica DB1 ReplicaUnified View
  15. 15. Growing with Vitess Vitess app server app server app server DB2 Replica DB1:S1 Replica DB1:S2 Replica DB1:S3 Replica Sharding agnostic
  16. 16. Growing with Vitess Vitess app server app server app server DB2 Replica DB1:S1 Replica DB1:S2 Replica DB1:S3 Replica Replica Replica Replica Replica Replica Replica Replica Replica Zone 1 Zone 2 Zone 3 Multi-zone Isolation
  17. 17. ALL YOUR DATABASE
  18. 18. Architecture shard n shard 1 vtgate app server app server app server mysqld vttablet master vtgate big data Vitess lockserver mysqld vttablet mysqld vttablet replicas mysqld vttablet big data replicas mysqld vttablet vtctld App
  19. 19. Sharding concepts VTGate ● VSchema ● Primary Vindex ● Secondary Vindex ○ Unique ○ Non-unique ● Shared Vindex ● Sequences Database ● Schema ● Primary Key ● Secondary Key ○ Unique ○ Non-unique ● Foreign Key ● Auto-increment
  20. 20. Resharding ● No downtime ● Vertical and Horizontal ● Splits and Merges ● Checks and balances
  21. 21. Demo user id name order id uid pid mname merchant name lookup unsharded product id user sharded merchant sharded name_user_idx name uid
  22. 22. Questions? ● Sign into slack channel at vitess.io ● Try bringing up a cluster on your laptop or kubernetes ● GH: http://github.com/vitessio/vitess
  23. 23. Appendix
  24. 24. Demo user id name order id uid pid mname merchant name lookup unsharded product id user sharded merchant sharded name_user_idx name uid product id order id uid pid mname product id select * from product select name, id as uid from user select * from merchant where in_keyrange( mname, ‘unicode_loose_md5’, ‘-80’)
  25. 25. DB Protection ● Connection pools ● Hot rows ● Damage control ○ Row-count limit ○ Time limit ○ Transaction time limits ● Blacklisting
  26. 26. Observability ● Stats end points ● Per-query stats ● Querylog
  27. 27. Life of a Query: VTGate Parse PlanBuilder vschema AST Engine Part 1 Plan Gateway topology vttablets SQL SQL shardmapVindexes vindex Keyspace ids Engine Part 2 shards ScatterConn health
  28. 28. Life of a Query: VTTablet Parse PlanBuilder schema AST ACL Plan SQL PKs Consolidator Pool MySQLDeadliner Transaction Watcher BinlogPlayer Filtered Replication
  29. 29. Roadmap ● Materialized views ● Real-time rollups ● VSchema-based resharding ● Full cross-shard SQL support
  30. 30. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ vitess

×