SlideShare une entreprise Scribd logo
1  sur  87
Background
Processing
   with Nanite




                 By Jennifer Davis
Overview
Overview
Overview

• Nanite
Overview

• Nanite
• RabbitMQ/AMQP
Overview

• Nanite
• RabbitMQ/AMQP
• Code Example
Overview

• Nanite
• RabbitMQ/AMQP
• Code Example
• Questions
Overview
Background Processing
Background Processing
Background Processing

Q. Don’t we already do background
processing with BackgrounDRb?
Background Processing

Q. Don’t we already do background
processing with BackgrounDRb?
A. Yes, but it is not scalable for our needs.
Background Processing
Background Processing
Background Processing

Q. Why Nanite?
Background Processing

Q. Why Nanite?
A. Although other methods of background
processing exist, Nanite is more suited to
our needs.
Background Processing
Nanite
Nanite
Nanite

“A Self Assembling Cluster of Ruby Daemons”
                                - Ezra Zygmuntowicz
Nanite
Nanite
Nanite
Composed of Mappers and Agents
Nanite
         Composed of Mappers and Agents
 Thin

Mapper
Nanite
         Composed of Mappers and Agents
 Thin

Mapper




 Shell

Mapper
Nanite
         Composed of Mappers and Agents
 Thin

           A
Mapper
           M          Message
                      Queue
           Q
                     (RabbitMQ)
           P
 Shell

Mapper
Nanite
         Composed of Mappers and Agents
                                          Agent
 Thin
                                          Actor
           A
Mapper                                    Actor
           M          Message
                                          Actor
                      Queue
           Q
                     (RabbitMQ)
           P
 Shell

Mapper
Nanite
         Composed of Mappers and Agents
                                          Agent
 Thin
                                          Actor
           A
Mapper                                    Actor
           M          Message
                                          Actor
                      Queue
           Q
                     (RabbitMQ)
           P                              Agent
 Shell
                                          Actor
Mapper                                    Actor
                                          Actor
Nanite
         Composed of Mappers and Agents
                                          Agent
 Thin
                                          Actor
           A                       A
Mapper                                    Actor
           M                       M
                      Message
                                          Actor
                      Queue
           Q                       Q
                     (RabbitMQ)
           P                       P      Agent
 Shell
                                          Actor
Mapper                                    Actor
                                          Actor
Nanite
         Composed of Mappers and Agents
                                          Agent
 Thin
                                          Actor
           A                       A
Mapper                                    Actor
           M                       M
                      Message
                                          Actor
                      Queue
           Q                       Q
                     (RabbitMQ)
           P                       P      Agent
 Shell
                                          Actor
Mapper                                    Actor
                                          Actor
Nanite
Nanite
Mappers
Nanite
Mappers
• Can have one or many
Nanite
Mappers
• Can have one or many
• Run within Rails or the shell
Nanite
Mappers
• Can have one or many
• Run within Rails or the shell
• Subscribed to the Mapper Exchange
Nanite
Mappers
• Can have one or many
• Run within Rails or the shell
• Subscribed to the Mapper Exchange
• Adds/Removes agents based on agent
  availability
Nanite
Nanite
Nanite
Agents
Nanite
Agents
• Can have one or many
Nanite
Agents
• Can have one or many
• Ping the Mapper Exchange every
   @ping_time seconds
Nanite
Agents
• Can have one or many
• Ping the Mapper Exchange every
   @ping_time seconds
• A single agent can have many actor
   classes
Nanite
Nanite
Nanite
Selectors for load balancing
Nanite
Selectors for load balancing
• :least_loaded
Nanite
Selectors for load balancing
• :least_loaded
• :rr
Nanite
Selectors for load balancing
• :least_loaded
• :rr
• :random
Nanite
Selectors for load balancing
• :least_loaded
• :rr
• :random
• :all
Nanite
Nanite
                               Agent
 Thin
                               Actor
         A                 A
Mapper                         Actor
         M                 M
               Message
                               Actor
               Queue
         Q                 Q
              (RabbitMQ)
         P                 P   Agent
 Shell
                               Actor
Mapper                         Actor
                               Actor
Nanite
                               Agent
 Thin
                               Actor
         A                 A
Mapper                         Actor
         M                 M
               Message
                               Actor
               Queue
         Q                 Q
              (RabbitMQ)
         P                 P   Agent
 Shell
                               Actor
Mapper                         Actor
                               Actor
Nanite
                           Ping back status   Agent
 Thin
                                              Actor
         A                             A
Mapper                                        Actor
         M                             M
               Message
                                              Actor
               Queue
         Q                             Q
              (RabbitMQ)
         P                             P      Agent
 Shell
                                              Actor
Mapper                                        Actor
                                              Actor
Nanite
                               Agent
 Thin
                               Actor
         A                 A
Mapper                         Actor
         M                 M
               Message
                               Actor
               Queue
         Q                 Q
              (RabbitMQ)
         P                 P   Agent
 Shell
                               Actor
Mapper                         Actor
                               Actor
Nanite
Nanite
Nanite requires
Nanite
Nanite requires
• Erlang
Nanite
Nanite requires
• Erlang
• Nanite ruby gem
Nanite
Nanite requires
• Erlang
• Nanite ruby gem
• EventMachine
Nanite
Nanite requires
• Erlang
• Nanite ruby gem
• EventMachine
• AMQP
Nanite
Nanite requires
• Erlang
• Nanite ruby gem
• EventMachine
• AMQP
• RabbitMQ
Nanite
AMQP
AMQP
AMQP
Advanced Messaging Queue Protocol
AMQP
Advanced Messaging Queue Protocol
• An open Internet Protocol for Business
   Messaging
AMQP
Advanced Messaging Queue Protocol
• An open Internet Protocol for Business
   Messaging
• Provides a standard for messaging
   middleware
AMQP
RabbitMQ
RabbitMQ
RabbitMQ
• An AMQP Messaging Broker
RabbitMQ
• An AMQP Messaging Broker
• Written in Erlang
RabbitMQ
• An AMQP Messaging Broker
• Written in Erlang
• Uses the OTP (Open Telecom Platform)
  system
RabbitMQ
• An AMQP Messaging Broker
• Written in Erlang
• Uses the OTP (Open Telecom Platform)
  system
• Is stable - Achieved 99.9999999% availability
RabbitMQ
Code Example
Review
Review

• Why background processing is important
  to us
• What Nanite is and how to use it
• A little background on RabbitMQ
Review
Review

• Why background processing is important
  to us
• What Nanite is and how to use it
• A little background on RabbitMQ
Resources
Resources
•   Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/
    nanite/tree/master/
Resources
•   Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/
    nanite/tree/master/

•   George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/
    videos/nanite/
Resources
•   Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/
    nanite/tree/master/

•   George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/
    videos/nanite/

•   RabbitMQ - Open Source Enterprise Messaging at http://
    www.rabbitmq.com/
Resources
•   Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/
    nanite/tree/master/

•   George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/
    videos/nanite/

•   RabbitMQ - Open Source Enterprise Messaging at http://
    www.rabbitmq.com/

•   AMQP - Advanced Message Queuing Protocol at http://jira.amqp.org/
    confluence/display/AMQP/Advanced+Message+Queuing+Protocol
Resources
Questions?

Contenu connexe

En vedette

A Macro Analysis of India and Nepal Bilateral Trade: Retrospect and Prospects
A Macro Analysis of India and Nepal Bilateral Trade: Retrospect and ProspectsA Macro Analysis of India and Nepal Bilateral Trade: Retrospect and Prospects
A Macro Analysis of India and Nepal Bilateral Trade: Retrospect and Prospects
iosrjce
 
India's trade policy.ppt
India's trade policy.pptIndia's trade policy.ppt
India's trade policy.ppt
Shikha Gupta
 
International trade in india ppt
International trade in india pptInternational trade in india ppt
International trade in india ppt
shivujagga
 

En vedette (9)

India wid other countries
India wid other countriesIndia wid other countries
India wid other countries
 
A Macro Analysis of India and Nepal Bilateral Trade: Retrospect and Prospects
A Macro Analysis of India and Nepal Bilateral Trade: Retrospect and ProspectsA Macro Analysis of India and Nepal Bilateral Trade: Retrospect and Prospects
A Macro Analysis of India and Nepal Bilateral Trade: Retrospect and Prospects
 
India's Foreign Trade
India's Foreign TradeIndia's Foreign Trade
India's Foreign Trade
 
USA Economy, International Trade & Bilateral trade with India
USA Economy, International Trade & Bilateral trade with India USA Economy, International Trade & Bilateral trade with India
USA Economy, International Trade & Bilateral trade with India
 
Indo us relation
Indo us relationIndo us relation
Indo us relation
 
Foreign Trade of India: Critical Analysis
Foreign Trade of India: Critical AnalysisForeign Trade of India: Critical Analysis
Foreign Trade of India: Critical Analysis
 
Us India Relations
Us India RelationsUs India Relations
Us India Relations
 
India's trade policy.ppt
India's trade policy.pptIndia's trade policy.ppt
India's trade policy.ppt
 
International trade in india ppt
International trade in india pptInternational trade in india ppt
International trade in india ppt
 

Dernier

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Dernier (20)

AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 

Background Processing with Nanite

Notes de l'éditeur

  1. BackgrounDRb BackgroundJob Starling with Workling
  2. BackgrounDRb BackgroundJob Starling with Workling
  3. BackgrounDRb BackgroundJob Starling with Workling
  4. It’s highly scalable and provides failover. Workers can easily be removed and added therefore adding to the reliability and availability of the application. It relies on message queueing and provides load balancing.
  5. It’s highly scalable and provides failover. Workers can easily be removed and added therefore adding to the reliability and availability of the application. It relies on message queueing and provides load balancing.
  6. The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
  7. The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
  8. The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
  9. The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
  10. The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
  11. The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
  12. The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
  13. The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
  14. The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
  15. The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
  16. The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
  17. The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
  18. By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP
  19. By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP
  20. By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP
  21. By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP