Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
%w(map reduce).first
  A Tale About Rabbits,
    Latency and Slim
        Crontabs

Paolo Negri
thanks to: www.autoscout24....
Summary:
Map




http://www.matthiasdittrich.com/projekte/dliste/visualisations/index.html
rabbitMQ




http://www.flickr.com/photos/myxi/448253580
crontab diet




http://www.flickr.com/photos/tim_norris/2600843131/
Map Reduce
   “Programming model for
processing and generating large
          data sets”

           (Google)
Map Reduce quot;Mapquot; step

  the master node takes the
input, chops it up into smaller
sub-problems, and distributes
 ...
The problem


Invoicing our clients
Is it as simple as...

clients.map do |client|
  client.invoice
end
No!

   Because the process is:


• distributed
• concurrent
Problems:

• How many nodes?
• How many workers?
• Distribution mechanism to
 feed the workers?
What about queuing?

• the master node takes the input, chops
  it up into smaller sub-problems, and
  publishes them in a...
Here comes
• RabbitMQ is an implementation of AMQP,
  the emerging standard for high
  performance enterprise messaging
• ...
Erlang?

• Erlang is a general-purpose concurrent
  programming language designed by Ericsson
• distributed
• fault tolera...
Install it


• sudo apt-get install rabbitmq
• sudo gem install tmm1-amqp
Do it - master node
Use it - worker node
What and where

                     Master
                     (ruby)

RabbitMQ             Worker
            TCP/IP
 (...
Get for free

• Decoupling master/worker
• Workers take care of feeding themselves
• Flexible number of workers
Get for free

• RabbitMQ can be clustered
• Support of message acknowledgement
• Queues can be persisted on disk
  (at a p...
Queue

• Is an actual entity
• has a name
• can be inspected and managed
EventMachine
EventMachine
Is an implementation of Reactor Pattern


• Non blocking IO and lightweight
  concurrency
• eliminate the com...
EventMachine
EventMachine

   amqp gem is built on
EventMachine => you’re in a
  context where you can
    leverage concurrent
        ...
EM - Deferrables
EM - Deferrables

“The Deferrable pattern allows you to specify
 any number of Ruby code blocks that will be
executed at s...
EM - Deferrables
EM - Deferrables
Deferrables
without deferrables      with deferrables


     ClientStat          ClientStat
                              ...
Achieved so far

• Easy distribution of tasks
• Architecture that supports arbitrary
  number of workers (and masters)
• C...
More rabbits
Analogy with email system
Multicasting - producer
Multicasting - consumer
Multicasting

                                 Cons1
                        Queue1
Publisher
                       Queue...
Multicasting

                                      Cons1
                       msg A Queue1
Publisher
                  ...
Not only queues then
  Use messages distribution to build the
       nervous system of your app


• communication across h...
Where to start?

crontab -l

5 * * * * bin/do_the_quick_thing.rb
0 2 * * * bin/do_the_scary_thing.rb
Cron
• Simple
• Reliable
• No maintenance
• Status is not explicit
• Locking?
• Shot and forget
Queue
• Distributed easily
• Reliable
• Can be inspected
• Add/decrease workers
• Makes you think!
• Adds more complexity
On github - Projects

• eventmachine/eventmachine
• tmm1/amqp
• macournoyer/thin
• famoseagle/carrot
• celldee/bunny
• ezm...
Q&A



 ?
Thanks!

    Paolo Negri

/        hungryblank
Prochain SlideShare
Chargement dans…5
×

Publié le

Queue
• Distributed easily
• Reliable
• Can be inspected
• Add/decrease workers
• Makes you think!
• Adds more complexity

  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • very good,thanks!
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

×