Handling Redis failover  with ZooKeeper        Ryan LeCompte      BaRuCo Lightning Talk        September 2012
Me• Rubyist at Spreecast in San Francisco• redis_failover, method_locator,  synchronizable• http://github.com/ryanlecompte...
Typical Redis Setup
Master Dies    :-(
What happens to your data andconnected clients?•   Resque Jobs?•   Sidekiq Jobs?•   Cached pages?•   Cached stats?•   Ruby...
Scramble to manuallyreconfigure slave as master
Update all Redis clients totalk to the new master
Restart production servicesso they can see the new config
What do we really  want to do?   NOTHING
Failover should be safe    and automatic
Let’s talk aboutZooKeeper
Open source ApacheProject (originally from        Yahoo)
Provides a set of simpleprimitives to help you     build powerfuldistributed applications
Automatically handlesnetwork partitions &quorum management
Replicated, HighlyAvailable, Ordered,Battle-proven
Used by Yahoo, Netflix, Rackspace, Hadoop,     Neo4j, etc
How can I useZooKeeper in my  architecture?
Distributed     Configuration(update thread pool sizes &   other settings without requiring a server restart)
Distributed Locking (only one process should beable to do something at a time)
Priority Queues
Leader Election, Group Membership & more
Practical application:Using ZooKeeper to  build automatic  failover for Redis
redis_failoverhttp://github.com/ryanlecompte/redis_failover
Provides a smart “failover-aware” client that wrapsexisting Ruby client forRedis
Clients automaticallyconnect to ZooKeeperand register for watchnotifications
Node Managers monitorRedis servers for availabilityand automatically informclients of failover viaZooKeeper
Architecture
RedisFailover::Client                Example# create new client with list of ZooKeeper serversclient = RedisFailover::Clie...
Companies using     redis_failover• Papertrail• Backupify• MediaPiston• IDT Corporation• others
Consider incorporatingZooKeeper into your overallarchitecture.Utilize the primitives it gives you forsolving your next dis...
Don’t let your Redis serversgo unmanaged in production.Consider usingredis_failover to sleepbetter at night.
Thanks for listening!       http://github.com/ryanlecompte/redis_failover              Questions? Ask me at the beach!
Prochain SlideShare
Chargement dans... 5
×

Handling Redis failover with ZooKeeper

12,858

Published on

Published in: Technologies, Éducation
0 commentaires
38 mentions J'aime
Statistiques
Remarques
  • Soyez le premier à commenter

Aucun téléchargement
Vues
Total des vues
12,858
Sur Slideshare
0
À partir des ajouts
0
Nombre d'ajouts
3
Actions
Partages
0
Téléchargements
240
Commentaires
0
J'aime
38
Ajouts 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Handling Redis failover with ZooKeeper

    1. 1. Handling Redis failover with ZooKeeper Ryan LeCompte BaRuCo Lightning Talk September 2012
    2. 2. Me• Rubyist at Spreecast in San Francisco• redis_failover, method_locator, synchronizable• http://github.com/ryanlecompte• lecompte@gmail.com• @ryanlecompte
    3. 3. Typical Redis Setup
    4. 4. Master Dies :-(
    5. 5. What happens to your data andconnected clients?• Resque Jobs?• Sidekiq Jobs?• Cached pages?• Cached stats?• Rubygems?
    6. 6. Scramble to manuallyreconfigure slave as master
    7. 7. Update all Redis clients totalk to the new master
    8. 8. Restart production servicesso they can see the new config
    9. 9. What do we really want to do? NOTHING
    10. 10. Failover should be safe and automatic
    11. 11. Let’s talk aboutZooKeeper
    12. 12. Open source ApacheProject (originally from Yahoo)
    13. 13. Provides a set of simpleprimitives to help you build powerfuldistributed applications
    14. 14. Automatically handlesnetwork partitions &quorum management
    15. 15. Replicated, HighlyAvailable, Ordered,Battle-proven
    16. 16. Used by Yahoo, Netflix, Rackspace, Hadoop, Neo4j, etc
    17. 17. How can I useZooKeeper in my architecture?
    18. 18. Distributed Configuration(update thread pool sizes & other settings without requiring a server restart)
    19. 19. Distributed Locking (only one process should beable to do something at a time)
    20. 20. Priority Queues
    21. 21. Leader Election, Group Membership & more
    22. 22. Practical application:Using ZooKeeper to build automatic failover for Redis
    23. 23. redis_failoverhttp://github.com/ryanlecompte/redis_failover
    24. 24. Provides a smart “failover-aware” client that wrapsexisting Ruby client forRedis
    25. 25. Clients automaticallyconnect to ZooKeeperand register for watchnotifications
    26. 26. Node Managers monitorRedis servers for availabilityand automatically informclients of failover viaZooKeeper
    27. 27. Architecture
    28. 28. RedisFailover::Client Example# create new client with list of ZooKeeper serversclient = RedisFailover::Client.new(:zkservers => localhost:2181,localhost:2182,localhost:2183)# write operations automatically routed to masterclient.set(‘foo’, ‘bar’)# read operations automatically routed to a slaveclient.smembers(‘data’)
    29. 29. Companies using redis_failover• Papertrail• Backupify• MediaPiston• IDT Corporation• others
    30. 30. Consider incorporatingZooKeeper into your overallarchitecture.Utilize the primitives it gives you forsolving your next distributedprogramming challenge.
    31. 31. Don’t let your Redis serversgo unmanaged in production.Consider usingredis_failover to sleepbetter at night.
    32. 32. Thanks for listening! http://github.com/ryanlecompte/redis_failover Questions? Ask me at the beach!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×