Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
MongoDB vs MySQL A DevOps point of view.Pierre Baillet <oct@fotopedia.com> @octplaneMathieu Poumeyrol <kali@fotopedia.com>
Summary«The question is, which is to be master»       Humpty Dumpty Who we are Context and constraints High availability a...
Who we areFotopedia (and not photopedia or fotolia)Created in 2006, Paris basedaround 20 people, some Apple ex-employeesPi...
What we doWebsite http://www.fotopedia.com (100% free website,become member and show us your best photos)2011 Crunchies aw...
Some statistics 150 Millions photos views 1 MySQL database 4 MongoDB ‘clusters’ (spread on 4 servers) Around 500GB of stru...
Context and Constraints«La nuit ne peut quempirer mille fois»   Roméo & Juliette 24/7 Website and web-services Continuous ...
24/7Several million of users around the world. Between 300and several thousands at once connected.Using either the website...
Overall activity on the main HTTP entry point
Continuous deploymentGit-based development flow with several activebranchesdevelopment branch deployed every wednesdayan av...
Our infrastructure Software stack Monitoring tools Hosting platform
Software StackRoR WebsiteMultiple OpenSource software used in the web stack:HAProxy, Nginx, Unicorn, mongo-resque, ...Some...
Monitoring tools Munin, Nagios Custom log feeder build around MongoDB (cf slideshare presentation: "mongodb as a log colle...
Hosting Platform: 100% AWS Instances are not highly reliable   but they are both abundant and disposable Disk is abundant ...
What we expect from our NoSQL DBMS and ourcompromise No downtime:   High availability   No migration cost Easy to deploy, ...
High Availability,Day to Day operations«Au fond de la cave, Paraît quil y a pas de sots métiers»   Le poinçonneur de lilas...
Dev’ CycleData localityData migration  Alter table  Index creationData backup/restoration
Dev’ Data Locality In MongoDB, a collection will typically replace 2 or 3 SQL tables The physical proximity, locality, ena...
Dev’ Data Migration: ALTERALTER TABLE is nightmarish  leads to various forms of model abusing strategy:    reuse of fields ...
Defensive strategy Application code aware of possible inconsistencies:   gracefully failing view layer   self-healing data...
Dev’ Data Migration: INDICES Indices creation leads to table-wide lock in MySQL. Renders part of the Cluster unavailable M...
Dev’ Backup/RestoreMongoDB ability to dump a db/collection empowersdeveloperPossible to restore part of the production dat...
Ops CycleMongoDB, small is beautifulCornerstone: the Replica SetHigh availabilityBackup and data import/exportHardware mig...
Ops, MongoDB, small and beautiful Young software, relatively compact (around 150,000 of C++ code) Builds out of the box on...
Ops, Replica SetA set of machine sharing the same dataOnly one Primary, several SecondariesAll writes go to Primary, route...
Ops, Master/Slave reloadedClient libraries are replica set aware  connect to any node(s), the configuration and current  la...
Ops, High Availability Strengths Primary step down can be triggered. Lead to election of a new Primary. a new Primary is p...
Ops, High Availability compromise Switch over will take 20 to 25 seconds   Some queries in the interval may crash   Some w...
Ops, Backup and exportsStop the secondary and do whatever you need done.Easy to backup a single collection or a whole data...
Ops, Hardware migrationOptionally possible to «preload» with a FS or block levelsnapshotAdd the brand new node to the repl...
Fit for the DevOps In the modern sense of DevOps, MongoDB provides the Agility and Ease of use required It provides workin...
Scalability«Accroche toi au pinceau, j’enlève le shell.»   Entendu @fotopedia Cloud Limitations Sharding and Replica Set P...
Cloud Limitations Virtual Hardware   Neighbors can eat all you I/O   No precise control and overview of this situation   L...
ShardingUse a business key to part your dataEach shard is typically a replica setAccess is provided via the MongoS servers...
ReadingWithout Sharding  Reading is performed on a master by default to  perserve read-your-own-writes. Can be  programmat...
Writing Writes are always performed on the Primary node, so replica set does not help. Sharding distributes the write amon...
StorageReplica Set and shards can be moved on as manyservers as needed.To get more space  scale up by migrating your Repli...
Scalable from the ground upMongoDB is scalable as soon as you need it to.No complex configuration for replicationBeautiful ...
Questions ?  Oh, and btw, we hire in Paris !
Prochain SlideShare
Chargement dans…5
×

MongoDB vs Mysql. A devops point of view

49 084 vues

Publié le

A fotopedia presentation made at the MongoDay 2012 in Paris at Xebia Office.

Talk by Pierre Baillet and Mathieu Poumeyrol.

French Article about the presentation:

http://www.touilleur-express.fr/2012/02/06/mongodb-retour-sur-experience-chez-fotopedia/

Video to come.

Publié dans : Technologie
  • http://www.youtube.com/watch?v=w5qr4sx5Vt0
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • i will like u to tell me what u like and what u dislike, tell me everything i need to know about u i will also tell you about my self in the next mail if i eventually see your reply.please if you are interested write me back at (( alicedavids72@yahoo.com ))
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Another presentation about MongoDB we made at Fotopedia:
    http://www.slideshare.net/kalizoy/mongodb-our-swiss-army-knife-database
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

MongoDB vs Mysql. A devops point of view

  1. 1. MongoDB vs MySQL A DevOps point of view.Pierre Baillet <oct@fotopedia.com> @octplaneMathieu Poumeyrol <kali@fotopedia.com>
  2. 2. Summary«The question is, which is to be master» Humpty Dumpty Who we are Context and constraints High availability and day-to-day operations Scalability
  3. 3. Who we areFotopedia (and not photopedia or fotolia)Created in 2006, Paris basedaround 20 people, some Apple ex-employeesPictures for humanity, cross-breed between flickr andwikipedia
  4. 4. What we doWebsite http://www.fotopedia.com (100% free website,become member and show us your best photos)2011 Crunchies award for Best Tablet Application
  5. 5. Some statistics 150 Millions photos views 1 MySQL database 4 MongoDB ‘clusters’ (spread on 4 servers) Around 500GB of structured data
  6. 6. Context and Constraints«La nuit ne peut quempirer mille fois» Roméo & Juliette 24/7 Website and web-services Continuous deployment Current Infrastructure What we expect from our NoSQL DBMS and our compromise
  7. 7. 24/7Several million of users around the world. Between 300and several thousands at once connected.Using either the website or one of the 7 available iOSapplications. Business CriticalWhen the website is down at the application level,everything starts to fail graduallyWe cannot stop the website completely. Ever.
  8. 8. Overall activity on the main HTTP entry point
  9. 9. Continuous deploymentGit-based development flow with several activebranchesdevelopment branch deployed every wednesdayan average of 3 minor hot-fixes every workdayagile: any developer can push its hot fixs in production,at any timeWe cannot easily schedule migrations. They should beas transparent as possible.
  10. 10. Our infrastructure Software stack Monitoring tools Hosting platform
  11. 11. Software StackRoR WebsiteMultiple OpenSource software used in the web stack:HAProxy, Nginx, Unicorn, mongo-resque, ...Some well known NoSQL tools: MySQL used to manage what was the core of our data MongoDB in production since September 2009, now managing more than 70% of our data
  12. 12. Monitoring tools Munin, Nagios Custom log feeder build around MongoDB (cf slideshare presentation: "mongodb as a log collector") MongoDB is also used to store slow transactions, exceptions and profiling traces for later inspection
  13. 13. Hosting Platform: 100% AWS Instances are not highly reliable but they are both abundant and disposable Disk is abundant and disposable too Use AWS RDS for MySQL hosting. Cheap and easy to setup but very shaky failover process (DNS based). We cannot rely too much on the hardware
  14. 14. What we expect from our NoSQL DBMS and ourcompromise No downtime: High availability No migration cost Easy to deploy, redeploy, replicate, reconfigure Quietly losing seconds of writes is preferable to weekly minutes-long maintenances periods minutes-long unscheduled downtime and manual failover in case of hardware failure
  15. 15. High Availability,Day to Day operations«Au fond de la cave, Paraît quil y a pas de sots métiers» Le poinçonneur de lilas Development environment Operations cycle Fit for the DevOps
  16. 16. Dev’ CycleData localityData migration Alter table Index creationData backup/restoration
  17. 17. Dev’ Data Locality In MongoDB, a collection will typically replace 2 or 3 SQL tables The physical proximity, locality, enables faster, simpler and more complete data retrieval from the application point of view. Less requests, more data.
  18. 18. Dev’ Data Migration: ALTERALTER TABLE is nightmarish leads to various forms of model abusing strategy: reuse of fields flag fields (binary encoded), blob fields (json/xml encoded), ...MongoDB solution: free form data storage, extensible.
  19. 19. Defensive strategy Application code aware of possible inconsistencies: gracefully failing view layer self-healing data access layer routine data checking and fixing batch
  20. 20. Dev’ Data Migration: INDICES Indices creation leads to table-wide lock in MySQL. Renders part of the Cluster unavailable MongoDB solution: Background indices creation, slows access a tiny bit, but do not lock !
  21. 21. Dev’ Backup/RestoreMongoDB ability to dump a db/collection empowersdeveloperPossible to restore part of the production datasetsimply on a development boxBackup a MongoDB by collections in S3, recover ondev’ platform in a matter of minutes
  22. 22. Ops CycleMongoDB, small is beautifulCornerstone: the Replica SetHigh availabilityBackup and data import/exportHardware migration
  23. 23. Ops, MongoDB, small and beautiful Young software, relatively compact (around 150,000 of C++ code) Builds out of the box on modern distributions Distros Package made by 10gen Drivers for most popular languages are also provided and maintained by 10gen staff. (although quality varies)
  24. 24. Ops, Replica SetA set of machine sharing the same dataOnly one Primary, several SecondariesAll writes go to Primary, routed to secondaries.Reads can be routed to primary or secondary at theapplication choiceWith the combination of AWS, Replica Set are verypowerful. MongoDB Loves the Cloud !
  25. 25. Ops, Master/Slave reloadedClient libraries are replica set aware connect to any node(s), the configuration and current layout is discoveredDatabase semantics are preservedIncredibly easy to setup Priority between nodes can be dynamically changed It’s possible to prevent a node from ever becoming master (slow-disk server used as a «hot backup»)
  26. 26. Ops, High Availability Strengths Primary step down can be triggered. Lead to election of a new Primary. a new Primary is picked when the Primary becomes unreachable clients will transparently connect to the new Primary MongoDB Arbiter ensure split brain will not happen Config. Server contains the sharding information. 1 or 3 config servers with internal failover mechanism
  27. 27. Ops, High Availability compromise Switch over will take 20 to 25 seconds Some queries in the interval may crash Some writes may reach a split primary
  28. 28. Ops, Backup and exportsStop the secondary and do whatever you need done.Easy to backup a single collection or a whole databaseAs a matter of fact, we just dumbly «mongodump»every collection of interest separately.
  29. 29. Ops, Hardware migrationOptionally possible to «preload» with a FS or block levelsnapshotAdd the brand new node to the replica setWait for synchroChange RS rules to get your new server primaryRemove the old hardware
  30. 30. Fit for the DevOps In the modern sense of DevOps, MongoDB provides the Agility and Ease of use required It provides working tools for developers And is much more confortable than MySQL in its daily usage Truly a DevOps-friendly tool.
  31. 31. Scalability«Accroche toi au pinceau, j’enlève le shell.» Entendu @fotopedia Cloud Limitations Sharding and Replica Set Performance Reading Writing Storage Scalable from the ground up
  32. 32. Cloud Limitations Virtual Hardware Neighbors can eat all you I/O No precise control and overview of this situation Largest VM cannot compare to largest Metal Hardware issue means zero-notice before instance retirement (Metal has same issue though). Need to be flexible Scaling-out is the way to scale on the Cloud
  33. 33. ShardingUse a business key to part your dataEach shard is typically a replica setAccess is provided via the MongoS serversConfiguration is stored and managed in the Configservers
  34. 34. ReadingWithout Sharding Reading is performed on a master by default to perserve read-your-own-writes. Can be programmatically allowed on a slave. To scale up reads, add Replica Set nodesWith Sharding Reading is performed in parallel across data nodes To scale up reads: ensure most queries will reach only one single shard
  35. 35. Writing Writes are always performed on the Primary node, so replica set does not help. Sharding distributes the write among the cluster
  36. 36. StorageReplica Set and shards can be moved on as manyservers as needed.To get more space scale up by migrating your Replica Set to bigger hardware scale out by sharding existing the collection
  37. 37. Scalable from the ground upMongoDB is scalable as soon as you need it to.No complex configuration for replicationBeautiful ability to handle replica set and shards out ofthe boxMongoS / Config Server / Shards allows more complexsetupCloud Friendly
  38. 38. Questions ? Oh, and btw, we hire in Paris !

×