SlideShare une entreprise Scribd logo
1  sur  60
Télécharger pour lire hors ligne
Riak Core:
                             Building Distributed
                             Applications Without
                                 Shared State

                     Commercial Users of Functional Programming
                         Baltimore, MD · October 2010

                                Rusty Klophaus (@rklophaus)
                                     Basho Technologies


Wednesday, October 6, 2010
You suddenly feel
             an uncontrollable desire to
                    learn Erlang.




  http://www.flickr.com/photos/procsilas/18014203
                                                   2
Wednesday, October 6, 2010
What is Riak Core?
                             How does it work?
                             How can you use it?




                                      3
Wednesday, October 6, 2010
Distributed, scalable, failure-tolerant.




                                 4
Wednesday, October 6, 2010
Distributed, scalable, failure-tolerant.

                             No central coordinator.
                             Easy to setup/operate.


                                        5
Wednesday, October 6, 2010
Distributed, scalable, failure-tolerant.

                               Horizontally scalable;
                             add commodity hardware
                                  to get more X.

                                        6
Wednesday, October 6, 2010
Distributed, scalable, failure-tolerant.

                                 Always available.
                             No single point of failure.
                                   Self-healing.

                                          7
Wednesday, October 6, 2010
Basho Technologies

               Riak KV
                    Distributed, scalable, failure-tolerant key/value datastore.
                    Started as a “Dynamo clone”.
                    Map/Reduce, Lightweight Data Relations, Client APIs

               Riak Search
                    Distributed, scalable, failure-tolerant full-text search engine.
                    Near Realtime, Riak KV Integration, Solr Support




                                                  8
Wednesday, October 6, 2010
Wednesday, October 6, 2010
Riak   Riak    Riak
                             KV     Core   Search




Wednesday, October 6, 2010
Riak Core is an Erlang library
                        that helps you build
                distributed, scalable, failure-tolerant
                             applications.


                                  11
Wednesday, October 6, 2010
Amazon Dynamo




Wednesday, October 6, 2010
“We Generalized the
                             Dynamo Architecture and
                             Open-Sourced the Bits.”




                                       13
Wednesday, October 6, 2010
Wait, doesn’t *Erlang* let you build
               distributed, scalable, failure-tolerant
                           applications?




                                 14
Wednesday, October 6, 2010
Erlang makes it easy to connect the
               components of your application.
                             Client
                                      Service A       Service B




                                                      Service C
                                        Queue E

                                                  Resource D




Wednesday, October 6, 2010
Riak Core helps you build a service that
               harnesses the power of many nodes.

                             Node A   Node B   Node C   Node D


                             Node E   Node F   Node G   Node H

                                      Service
                             Node I   Node J   Node K   Node L


                             Node M   Node N   Node O     ...




Wednesday, October 6, 2010
“People use languages
                                        other than Erlang?!?
                                       I find that hilarious.”
                                            -Al Gore actually said this to me.

  http://www.flickr.com/photos/scobleizer/2216445692
Wednesday, October 6, 2010
How does Riak Core work?




                                        18
Wednesday, October 6, 2010
Command   ObjectName, Payload




Wednesday, October 6, 2010
Predictable Routing




                                     20
Wednesday, October 6, 2010
Hash the Object Name
                   Command    ObjectName, Payload



                              SHA1(ObjName), Payload




                             0 to 2^160



Wednesday, October 6, 2010
A Naive Approach
                   Command                ObjectName, Payload



                                          SHA1(ObjName), Payload




                             Node A   Node B   Node C   Node D




Wednesday, October 6, 2010
A Naive Approach
                   Command                ObjectName, Payload



                                          SHA1(ObjName), Payload




                             Node A   Node B   Node C   Node D   Node E




Wednesday, October 6, 2010
"All problems in computer
                               science can be solved by
                             another level of indirection."
                                    - David Wheeler


                                           24
Wednesday, October 6, 2010
Add VNodes

               What
                    Virtual Node. Logical subdivision of the cluster.
                    Handles incoming commands, does work, replies.

               For Parallelism
                    # of VNodes = maximum concurrent requests

               For Rebalancing the Cluster
                    Smallest block that can be shifted to a new node.

               For Resiliance
                    The system restarts failed VNodes.
                                                25
Wednesday, October 6, 2010
Routing with Consistent Hash
                   Command                            ObjectName, Payload



                                                      SHA1(ObjName), Payload



                             VNode 0   VNode 1    VNode 2   VNode 3   VNode 4   VNode 5   VNode 6   VNode 7




                              Node A             Node B        Node C           Node D




Wednesday, October 6, 2010
Adding a Node
                   Command                            ObjectName, Payload



                                                      SHA1(ObjName), Payload



                             VNode 0   VNode 1    VNode 2   VNode 3   VNode 4   VNode 5   VNode 6   VNode 7




                              Node A             Node B        Node C           Node D         Node E




Wednesday, October 6, 2010
Removing a Node
                   Command                            ObjectName, Payload



                                                      SHA1(ObjName), Payload



                             VNode 0   VNode 1    VNode 2   VNode 3   VNode 4   VNode 5   VNode 6   VNode 7




                              Node A             Node B        Node C           Node D         Node E




Wednesday, October 6, 2010
The Ring




                             Hash Location




Wednesday, October 6, 2010
The Ring




                             Preflist




Wednesday, October 6, 2010
Writing Replicas (n_val)




                                   Preflist when N=3



Wednesday, October 6, 2010
Routing Around Failures




                              X
                                   Preflist when N=3
                                  and node 0 is down.


Wednesday, October 6, 2010
Location of the Routing Layer




                                          33
Wednesday, October 6, 2010
Router in the Middle

                                      Client                    Client                  Client




                                                               Router




                             VNode   VNode     VNode   VNode   VNode   VNode   VNode        VNode
                               0       1         3       4       2       5       6            7


                             Node A            Node B          Node C          Node D       Node E




Wednesday, October 6, 2010
Riak Core - Router on Each Node

                                      Client                    Client                   Client




                              Router            Router          Router          Router        Router
                             VNode   VNode     VNode   VNode   VNode   VNode   VNode         VNode
                               0       1         3       4       2       5       6             7


                             Node A            Node B          Node C          Node D        Node E




Wednesday, October 6, 2010
Eventually - Router on the Client

                                      Client                    Client                  Client

                                     Router                    Router                  Router




                             VNode   VNode     VNode   VNode   VNode   VNode   VNode        VNode
                               0       1         3       4       2       5       6            7


                             Node A            Node B          Node C          Node D       Node E




Wednesday, October 6, 2010
No Shared State


                              Router          Router          Router          Router    Router
                             VNode   VNode   VNode   VNode   VNode   VNode   VNode     VNode
                               0       1       3       4       2       5       6         7


                             Node A          Node B          Node C          Node D    Node E




Wednesday, October 6, 2010
Gossip


                               Local      Incoming
                             Ring State   Ring State




               Are rings equivalent?
               Strictly descendent?
               Or different?

Wednesday, October 6, 2010
Handoff

               When
                    Nodes is added to the system.
                    Node is removed from the system.
                    Node has temporarily failed.

               What
                    Ship the data backing a VNode from one node to
                    another.




                                            39
Wednesday, October 6, 2010
Not Mentioned
                                Vector Clocks
                                Merkle Trees
                                Bloom Filters




                                    40
Wednesday, October 6, 2010
Distinguished
                                               gentlemen
                                                   prefer
                                                   Erlang.

   http://www.flickr.com/photos/rebcal/3987226359

Wednesday, October 6, 2010
How do you use Riak Core?




                                        42
Wednesday, October 6, 2010
Two Things to Think About

               Command
                    Command = ObjectName, Payload
                    The commands/requests/operations that you will send
                    through the system.

               VNode Module
                    The callback module that will receive the commands.




                                            43
Wednesday, October 6, 2010
VNode Module
               Startup/Shutdown
                  init([Partition]) ->
                   {ok, State}

                  terminate(State) ->
                   ok

               Commands
                  handle_command(Cmd, Sender, State) ->
                   {noreply, State1} | {reply, Reply, State1}

                  handle_handoff_command(Cmd, Sender, State) ->
                   {noreply, State1} | {reply, ok, State1}

                                         44
Wednesday, October 6, 2010
VNode Module
               Handoff Coordination
                  handoff_starting(Node, State) ->
                   {Bool, State1}

                  encode_handoff_data(Data, State) ->
                   <<Binary>>.

                  handle_handoff_data(Data, Sender, State) ->
                   {reply, ok, State1}

                  handoff_finished(Node, State) ->
                   {ok, State1}

                                         45
Wednesday, October 6, 2010
Start the riak_core application

                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*

                             X_vnode   X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode      ...




                                  application:start(riak_core).




                                                           46
Wednesday, October 6, 2010
Start the riak_core application

                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*

                             X_vnode   X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode      ...



                                         Supervise vnode processes.




                                                           47
Wednesday, October 6, 2010
Start the riak_core application




                             Start, coordinate, and supervise handoff.




                                                48
Wednesday, October 6, 2010
Start the riak_core application

                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*

                             X_vnode   X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode      ...



                               Maintain cluster membership information.




                                                           49
Wednesday, October 6, 2010
Start the riak_core application

                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*

                             X_vnode    X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode      ...



                                           Monitor node liveness,
                                       broadcast to registered modules.




                                                           50
Wednesday, October 6, 2010
Start the riak_core application

                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*

                             X_vnode   X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode      ...



                               Send ring information to other nodes.
                               Reconcile different views of the cluster.
                             Rebalance cluster when nodes join or leave.



                                                           51
Wednesday, October 6, 2010
In your application...

                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*

                             X_vnode   X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode      ...



                                  Start the vnodes for your application.
                Master = {
                    riak_X_vnode_master, {
                       riak_core_vnode_master, start_link, [riak_X_vnode]
                    },
                    permanent, 5000, worker, [riak_core_vnode_master]
                },
                {ok, { {one_for_one, 5, 10}, [Master]} }.
                                                           52
Wednesday, October 6, 2010
In your application...

                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*

                             X_vnode   X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode      ...



                                    Tell riak_core that your application
                                        is ready to receive requests.

               riak_core:register_vnode_module(riak_X_vnode),
               riak_core_node_watcher:service_up(riak_X,
                                                 self())
                                                           53
Wednesday, October 6, 2010
In your application...
                                                      riak_core
                             riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*


                             X_vnode    X_vnode
                                                    riak_core_ring_*     riak_core_gossip_*
                             X_vnode       ...




                                                                                                  riak_core
                                                                         riak_core_vnode_sup   riak_core_handoff_*   riak_core_node_*


                                                                         X_vnode     X_vnode
                                                                                                riak_core_ring_*     riak_core_gossip_*
                                                                         X_vnode        ...




                                    Join to an existing node in the cluster.


               riak_core_gossip:send_ring(ClusterNode,
                                          node())
                                                                               54
Wednesday, October 6, 2010
Start Sending Commands

            # Figure out the preflist...
            {_Verb, ObjName, _Payload} = Command,
            PrefList = riak_core_apl:get_apl(ObjName,
                                             NVal,
                                             riak_X),

            # Send the command...
            riak_core_vnode_master:command(PrefList,
                                           Command,
                                           riak_X_vnode_master)




                                  55
Wednesday, October 6, 2010
Review

               Riak Core
                    Open source Erlang library for building distributed, scalable,
                    failure tolerant applications.
                    Continual improvement in the coming months.

               Riak KV
                    Key/Value datastore with map/reduce based on Riak Core.

               Riak Search
                    Full-text, near real-time search engine based on Riak Core.



                                                 56
Wednesday, October 6, 2010
At the center of the universe,
                                     beneath the bottom-most turtle,
                                          is a cluster of Erlang nodes.




    http://www.flickr.com/photos/jurvetson/469492885
Wednesday, October 6, 2010
At the center of the universe,
                             beneath the bottom-most turtle,
                                  is a cluster of Erlang nodes.
                                           With 100% uptime.




  http://www.flickr.com/photos/jurvetson/469492885
Wednesday, October 6, 2010
Thanks! Questions?

               Learn More
                    More Information: http://wiki.basho.com
                    Amazon’s Dynamo Paper

               Get the Code
                    http://hg.basho.com/riak_core

               Get in Touch
                    Twitter: @rklophaus, @basho/team
                    Email: rusty@basho.com


                                             59
Wednesday, October 6, 2010
END




Wednesday, October 6, 2010

Contenu connexe

Tendances

Introduction to Spark Datasets - Functional and relational together at last
Introduction to Spark Datasets - Functional and relational together at lastIntroduction to Spark Datasets - Functional and relational together at last
Introduction to Spark Datasets - Functional and relational together at lastHolden Karau
 
Getting the best performance with PySpark - Spark Summit West 2016
Getting the best performance with PySpark - Spark Summit West 2016Getting the best performance with PySpark - Spark Summit West 2016
Getting the best performance with PySpark - Spark Summit West 2016Holden Karau
 
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)jaxLondonConference
 
Getting started contributing to Apache Spark
Getting started contributing to Apache SparkGetting started contributing to Apache Spark
Getting started contributing to Apache SparkHolden Karau
 
Beyond Wordcount with spark datasets (and scalaing) - Nide PDX Jan 2018
Beyond Wordcount  with spark datasets (and scalaing) - Nide PDX Jan 2018Beyond Wordcount  with spark datasets (and scalaing) - Nide PDX Jan 2018
Beyond Wordcount with spark datasets (and scalaing) - Nide PDX Jan 2018Holden Karau
 
A super fast introduction to Spark and glance at BEAM
A super fast introduction to Spark and glance at BEAMA super fast introduction to Spark and glance at BEAM
A super fast introduction to Spark and glance at BEAMHolden Karau
 
Apache Spark Fundamentals Meetup Talk
Apache Spark Fundamentals Meetup TalkApache Spark Fundamentals Meetup Talk
Apache Spark Fundamentals Meetup TalkEren Avşaroğulları
 
Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...
Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...
Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...Holden Karau
 
Improving PySpark performance: Spark Performance Beyond the JVM
Improving PySpark performance: Spark Performance Beyond the JVMImproving PySpark performance: Spark Performance Beyond the JVM
Improving PySpark performance: Spark Performance Beyond the JVMHolden Karau
 
OrientDB - the 2nd generation of (Multi-Model) NoSQL - Devoxx Belgium 2015
OrientDB - the 2nd generation  of  (Multi-Model) NoSQL - Devoxx Belgium 2015OrientDB - the 2nd generation  of  (Multi-Model) NoSQL - Devoxx Belgium 2015
OrientDB - the 2nd generation of (Multi-Model) NoSQL - Devoxx Belgium 2015Luigi Dell'Aquila
 
Introduction to Spark ML Pipelines Workshop
Introduction to Spark ML Pipelines WorkshopIntroduction to Spark ML Pipelines Workshop
Introduction to Spark ML Pipelines WorkshopHolden Karau
 
TheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the RescueTheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the RescueShay Sofer
 
Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...
Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...
Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...Yuanxuan Wang
 
Beyond Shuffling and Streaming Preview - Salt Lake City Spark Meetup
Beyond Shuffling and Streaming Preview - Salt Lake City Spark MeetupBeyond Shuffling and Streaming Preview - Salt Lake City Spark Meetup
Beyond Shuffling and Streaming Preview - Salt Lake City Spark MeetupHolden Karau
 
Beyond shuffling - Strata London 2016
Beyond shuffling - Strata London 2016Beyond shuffling - Strata London 2016
Beyond shuffling - Strata London 2016Holden Karau
 
Spark ML for custom models - FOSDEM HPC 2017
Spark ML for custom models - FOSDEM HPC 2017Spark ML for custom models - FOSDEM HPC 2017
Spark ML for custom models - FOSDEM HPC 2017Holden Karau
 
Introducing Apache Spark's Data Frames and Dataset APIs workshop series
Introducing Apache Spark's Data Frames and Dataset APIs workshop seriesIntroducing Apache Spark's Data Frames and Dataset APIs workshop series
Introducing Apache Spark's Data Frames and Dataset APIs workshop seriesHolden Karau
 
Survey of Spark for Data Pre-Processing and Analytics
Survey of Spark for Data Pre-Processing and AnalyticsSurvey of Spark for Data Pre-Processing and Analytics
Survey of Spark for Data Pre-Processing and AnalyticsYannick Pouliot
 
Spark rdd vs data frame vs dataset
Spark rdd vs data frame vs datasetSpark rdd vs data frame vs dataset
Spark rdd vs data frame vs datasetAnkit Beohar
 
Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016
Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016
Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016Holden Karau
 

Tendances (20)

Introduction to Spark Datasets - Functional and relational together at last
Introduction to Spark Datasets - Functional and relational together at lastIntroduction to Spark Datasets - Functional and relational together at last
Introduction to Spark Datasets - Functional and relational together at last
 
Getting the best performance with PySpark - Spark Summit West 2016
Getting the best performance with PySpark - Spark Summit West 2016Getting the best performance with PySpark - Spark Summit West 2016
Getting the best performance with PySpark - Spark Summit West 2016
 
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
 
Getting started contributing to Apache Spark
Getting started contributing to Apache SparkGetting started contributing to Apache Spark
Getting started contributing to Apache Spark
 
Beyond Wordcount with spark datasets (and scalaing) - Nide PDX Jan 2018
Beyond Wordcount  with spark datasets (and scalaing) - Nide PDX Jan 2018Beyond Wordcount  with spark datasets (and scalaing) - Nide PDX Jan 2018
Beyond Wordcount with spark datasets (and scalaing) - Nide PDX Jan 2018
 
A super fast introduction to Spark and glance at BEAM
A super fast introduction to Spark and glance at BEAMA super fast introduction to Spark and glance at BEAM
A super fast introduction to Spark and glance at BEAM
 
Apache Spark Fundamentals Meetup Talk
Apache Spark Fundamentals Meetup TalkApache Spark Fundamentals Meetup Talk
Apache Spark Fundamentals Meetup Talk
 
Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...
Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...
Beyond Shuffling, Tips and Tricks for Scaling Apache Spark updated for Spark ...
 
Improving PySpark performance: Spark Performance Beyond the JVM
Improving PySpark performance: Spark Performance Beyond the JVMImproving PySpark performance: Spark Performance Beyond the JVM
Improving PySpark performance: Spark Performance Beyond the JVM
 
OrientDB - the 2nd generation of (Multi-Model) NoSQL - Devoxx Belgium 2015
OrientDB - the 2nd generation  of  (Multi-Model) NoSQL - Devoxx Belgium 2015OrientDB - the 2nd generation  of  (Multi-Model) NoSQL - Devoxx Belgium 2015
OrientDB - the 2nd generation of (Multi-Model) NoSQL - Devoxx Belgium 2015
 
Introduction to Spark ML Pipelines Workshop
Introduction to Spark ML Pipelines WorkshopIntroduction to Spark ML Pipelines Workshop
Introduction to Spark ML Pipelines Workshop
 
TheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the RescueTheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the Rescue
 
Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...
Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...
Dynamic Filtering: Multi-Purpose Architecture Support for Language Runtime Sy...
 
Beyond Shuffling and Streaming Preview - Salt Lake City Spark Meetup
Beyond Shuffling and Streaming Preview - Salt Lake City Spark MeetupBeyond Shuffling and Streaming Preview - Salt Lake City Spark Meetup
Beyond Shuffling and Streaming Preview - Salt Lake City Spark Meetup
 
Beyond shuffling - Strata London 2016
Beyond shuffling - Strata London 2016Beyond shuffling - Strata London 2016
Beyond shuffling - Strata London 2016
 
Spark ML for custom models - FOSDEM HPC 2017
Spark ML for custom models - FOSDEM HPC 2017Spark ML for custom models - FOSDEM HPC 2017
Spark ML for custom models - FOSDEM HPC 2017
 
Introducing Apache Spark's Data Frames and Dataset APIs workshop series
Introducing Apache Spark's Data Frames and Dataset APIs workshop seriesIntroducing Apache Spark's Data Frames and Dataset APIs workshop series
Introducing Apache Spark's Data Frames and Dataset APIs workshop series
 
Survey of Spark for Data Pre-Processing and Analytics
Survey of Spark for Data Pre-Processing and AnalyticsSurvey of Spark for Data Pre-Processing and Analytics
Survey of Spark for Data Pre-Processing and Analytics
 
Spark rdd vs data frame vs dataset
Spark rdd vs data frame vs datasetSpark rdd vs data frame vs dataset
Spark rdd vs data frame vs dataset
 
Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016
Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016
Improving PySpark Performance - Spark Beyond the JVM @ PyData DC 2016
 

En vedette

Building Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreBuilding Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreAndy Gross
 
Riak Search - Berlin Buzzwords 2010
Riak Search - Berlin Buzzwords 2010Riak Search - Berlin Buzzwords 2010
Riak Search - Berlin Buzzwords 2010Rusty Klophaus
 
Masterless Distributed Computing with Riak Core - EUC 2010
Masterless Distributed Computing with Riak Core - EUC 2010Masterless Distributed Computing with Riak Core - EUC 2010
Masterless Distributed Computing with Riak Core - EUC 2010Rusty Klophaus
 
All you didn't know about the CAP theorem
All you didn't know about the CAP theoremAll you didn't know about the CAP theorem
All you didn't know about the CAP theoremKanstantsin Hontarau
 
Riak Search - The Next Generation
Riak Search - The Next GenerationRiak Search - The Next Generation
Riak Search - The Next GenerationCaserta
 
Rolling With Riak
Rolling With RiakRolling With Riak
Rolling With RiakJohn Lynch
 
Riak - From Small to Large
Riak - From Small to LargeRiak - From Small to Large
Riak - From Small to LargeRusty Klophaus
 
Riak in Ten Minutes
Riak in Ten MinutesRiak in Ten Minutes
Riak in Ten MinutesJon Meredith
 
Riak Training Session — Surge 2011
Riak Training Session — Surge 2011Riak Training Session — Surge 2011
Riak Training Session — Surge 2011DstroyAllModels
 
NoSQL databases, the CAP theorem, and the theory of relativity
NoSQL databases, the CAP theorem, and the theory of relativityNoSQL databases, the CAP theorem, and the theory of relativity
NoSQL databases, the CAP theorem, and the theory of relativityLars Marius Garshol
 
CAP Theorem - Theory, Implications and Practices
CAP Theorem - Theory, Implications and PracticesCAP Theorem - Theory, Implications and Practices
CAP Theorem - Theory, Implications and PracticesYoav Francis
 

En vedette (14)

Building Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak CoreBuilding Distributed Systems With Riak and Riak Core
Building Distributed Systems With Riak and Riak Core
 
Riak Search - Berlin Buzzwords 2010
Riak Search - Berlin Buzzwords 2010Riak Search - Berlin Buzzwords 2010
Riak Search - Berlin Buzzwords 2010
 
Masterless Distributed Computing with Riak Core - EUC 2010
Masterless Distributed Computing with Riak Core - EUC 2010Masterless Distributed Computing with Riak Core - EUC 2010
Masterless Distributed Computing with Riak Core - EUC 2010
 
All you didn't know about the CAP theorem
All you didn't know about the CAP theoremAll you didn't know about the CAP theorem
All you didn't know about the CAP theorem
 
Riak Search 2: Yokozuna
Riak Search 2: YokozunaRiak Search 2: Yokozuna
Riak Search 2: Yokozuna
 
Riak Search - The Next Generation
Riak Search - The Next GenerationRiak Search - The Next Generation
Riak Search - The Next Generation
 
Rolling With Riak
Rolling With RiakRolling With Riak
Rolling With Riak
 
Riak - From Small to Large
Riak - From Small to LargeRiak - From Small to Large
Riak - From Small to Large
 
Riak in Ten Minutes
Riak in Ten MinutesRiak in Ten Minutes
Riak in Ten Minutes
 
Riak Training Session — Surge 2011
Riak Training Session — Surge 2011Riak Training Session — Surge 2011
Riak Training Session — Surge 2011
 
Elixir basics-2
Elixir basics-2Elixir basics-2
Elixir basics-2
 
Erlang OTP
Erlang OTPErlang OTP
Erlang OTP
 
NoSQL databases, the CAP theorem, and the theory of relativity
NoSQL databases, the CAP theorem, and the theory of relativityNoSQL databases, the CAP theorem, and the theory of relativity
NoSQL databases, the CAP theorem, and the theory of relativity
 
CAP Theorem - Theory, Implications and Practices
CAP Theorem - Theory, Implications and PracticesCAP Theorem - Theory, Implications and Practices
CAP Theorem - Theory, Implications and Practices
 

Similaire à Riak Core: Building Distributed Applications Without Shared State

Scientific Applications with Python
Scientific Applications with PythonScientific Applications with Python
Scientific Applications with PythonEnthought, Inc.
 
T-DOSE 2010 - Agile Enterprise, CLouds and Devops
T-DOSE 2010 - Agile Enterprise, CLouds and DevopsT-DOSE 2010 - Agile Enterprise, CLouds and Devops
T-DOSE 2010 - Agile Enterprise, CLouds and DevopsChef Software, Inc.
 
Lean analytics for startups - Leweb2010
Lean analytics for startups - Leweb2010Lean analytics for startups - Leweb2010
Lean analytics for startups - Leweb2010Alistair Croll
 
Open End To End Js Stack
Open End To End Js StackOpen End To End Js Stack
Open End To End Js StackSkills Matter
 
Akka scalaliftoff london_2010
Akka scalaliftoff london_2010Akka scalaliftoff london_2010
Akka scalaliftoff london_2010Skills Matter
 
Los Angeles R users group - Nov 17 2010 - Part 2
Los Angeles R users group - Nov 17 2010 - Part 2Los Angeles R users group - Nov 17 2010 - Part 2
Los Angeles R users group - Nov 17 2010 - Part 2rusersla
 
Large problems, Mostly Solved
Large problems, Mostly SolvedLarge problems, Mostly Solved
Large problems, Mostly Solvedericholscher
 
Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010Matt Aimonetti
 
Multiple Inheritance
Multiple InheritanceMultiple Inheritance
Multiple InheritanceMichal Píše
 
Distributed Social Networking
Distributed Social NetworkingDistributed Social Networking
Distributed Social NetworkingBastian Hofmann
 
Open stack swift_essex_meetup_2012_06_21_judd_maltin
Open stack swift_essex_meetup_2012_06_21_judd_maltinOpen stack swift_essex_meetup_2012_06_21_judd_maltin
Open stack swift_essex_meetup_2012_06_21_judd_maltinKamesh Pemmaraju
 
BRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQLBRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQLAndreas Jung
 
GoLightly: Building VM-based language runtimes in Go
GoLightly: Building VM-based language runtimes in GoGoLightly: Building VM-based language runtimes in Go
GoLightly: Building VM-based language runtimes in GoEleanor McHugh
 
Node.js, toy or power tool?
Node.js, toy or power tool?Node.js, toy or power tool?
Node.js, toy or power tool?Ovidiu Dimulescu
 
The Platypus Problem
The Platypus ProblemThe Platypus Problem
The Platypus ProblemJeff Eaton
 
Go! Go! Gadgets. Writing an OpenSocial Application
Go! Go! Gadgets.  Writing an OpenSocial ApplicationGo! Go! Gadgets.  Writing an OpenSocial Application
Go! Go! Gadgets. Writing an OpenSocial ApplicationMark Halvorson
 
JClouds at San Francisco Java User Group
JClouds at San Francisco Java User GroupJClouds at San Francisco Java User Group
JClouds at San Francisco Java User GroupMarakana Inc.
 

Similaire à Riak Core: Building Distributed Applications Without Shared State (20)

Scientific Applications with Python
Scientific Applications with PythonScientific Applications with Python
Scientific Applications with Python
 
T-DOSE 2010 - Agile Enterprise, CLouds and Devops
T-DOSE 2010 - Agile Enterprise, CLouds and DevopsT-DOSE 2010 - Agile Enterprise, CLouds and Devops
T-DOSE 2010 - Agile Enterprise, CLouds and Devops
 
Lean analytics for startups - Leweb2010
Lean analytics for startups - Leweb2010Lean analytics for startups - Leweb2010
Lean analytics for startups - Leweb2010
 
Open End To End Js Stack
Open End To End Js StackOpen End To End Js Stack
Open End To End Js Stack
 
Akka scalaliftoff london_2010
Akka scalaliftoff london_2010Akka scalaliftoff london_2010
Akka scalaliftoff london_2010
 
Los Angeles R users group - Nov 17 2010 - Part 2
Los Angeles R users group - Nov 17 2010 - Part 2Los Angeles R users group - Nov 17 2010 - Part 2
Los Angeles R users group - Nov 17 2010 - Part 2
 
Large problems, Mostly Solved
Large problems, Mostly SolvedLarge problems, Mostly Solved
Large problems, Mostly Solved
 
Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010
 
Multiple Inheritance
Multiple InheritanceMultiple Inheritance
Multiple Inheritance
 
Distributed Social Networking
Distributed Social NetworkingDistributed Social Networking
Distributed Social Networking
 
06 data
06 data06 data
06 data
 
Open stack swift_essex_meetup_2012_06_21_judd_maltin
Open stack swift_essex_meetup_2012_06_21_judd_maltinOpen stack swift_essex_meetup_2012_06_21_judd_maltin
Open stack swift_essex_meetup_2012_06_21_judd_maltin
 
BRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQLBRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQL
 
GoLightly: Building VM-based language runtimes in Go
GoLightly: Building VM-based language runtimes in GoGoLightly: Building VM-based language runtimes in Go
GoLightly: Building VM-based language runtimes in Go
 
XQuery Design Patterns
XQuery Design PatternsXQuery Design Patterns
XQuery Design Patterns
 
Node.js, toy or power tool?
Node.js, toy or power tool?Node.js, toy or power tool?
Node.js, toy or power tool?
 
The Platypus Problem
The Platypus ProblemThe Platypus Problem
The Platypus Problem
 
Riak and Ruby
Riak and RubyRiak and Ruby
Riak and Ruby
 
Go! Go! Gadgets. Writing an OpenSocial Application
Go! Go! Gadgets.  Writing an OpenSocial ApplicationGo! Go! Gadgets.  Writing an OpenSocial Application
Go! Go! Gadgets. Writing an OpenSocial Application
 
JClouds at San Francisco Java User Group
JClouds at San Francisco Java User GroupJClouds at San Francisco Java User Group
JClouds at San Francisco Java User Group
 

Plus de Rusty Klophaus

Everybody Polyglot! - Cross-Language RPC with Erlang
Everybody Polyglot! - Cross-Language RPC with ErlangEverybody Polyglot! - Cross-Language RPC with Erlang
Everybody Polyglot! - Cross-Language RPC with ErlangRusty Klophaus
 
Winning the Erlang Edit•Build•Test Cycle
Winning the Erlang Edit•Build•Test CycleWinning the Erlang Edit•Build•Test Cycle
Winning the Erlang Edit•Build•Test CycleRusty Klophaus
 
Querying Riak Just Got Easier - Introducing Secondary Indices
Querying Riak Just Got Easier - Introducing Secondary IndicesQuerying Riak Just Got Easier - Introducing Secondary Indices
Querying Riak Just Got Easier - Introducing Secondary IndicesRusty Klophaus
 
Riak - From Small to Large - StrangeLoop
Riak - From Small to Large - StrangeLoopRiak - From Small to Large - StrangeLoop
Riak - From Small to Large - StrangeLoopRusty Klophaus
 
Riak Search - Erlang Factory London 2010
Riak Search - Erlang Factory London 2010Riak Search - Erlang Factory London 2010
Riak Search - Erlang Factory London 2010Rusty Klophaus
 
Riak from Small to Large
Riak from Small to LargeRiak from Small to Large
Riak from Small to LargeRusty Klophaus
 
Getting Started with Riak - NoSQL Live 2010 - Boston
Getting Started with Riak - NoSQL Live 2010 - BostonGetting Started with Riak - NoSQL Live 2010 - Boston
Getting Started with Riak - NoSQL Live 2010 - BostonRusty Klophaus
 

Plus de Rusty Klophaus (7)

Everybody Polyglot! - Cross-Language RPC with Erlang
Everybody Polyglot! - Cross-Language RPC with ErlangEverybody Polyglot! - Cross-Language RPC with Erlang
Everybody Polyglot! - Cross-Language RPC with Erlang
 
Winning the Erlang Edit•Build•Test Cycle
Winning the Erlang Edit•Build•Test CycleWinning the Erlang Edit•Build•Test Cycle
Winning the Erlang Edit•Build•Test Cycle
 
Querying Riak Just Got Easier - Introducing Secondary Indices
Querying Riak Just Got Easier - Introducing Secondary IndicesQuerying Riak Just Got Easier - Introducing Secondary Indices
Querying Riak Just Got Easier - Introducing Secondary Indices
 
Riak - From Small to Large - StrangeLoop
Riak - From Small to Large - StrangeLoopRiak - From Small to Large - StrangeLoop
Riak - From Small to Large - StrangeLoop
 
Riak Search - Erlang Factory London 2010
Riak Search - Erlang Factory London 2010Riak Search - Erlang Factory London 2010
Riak Search - Erlang Factory London 2010
 
Riak from Small to Large
Riak from Small to LargeRiak from Small to Large
Riak from Small to Large
 
Getting Started with Riak - NoSQL Live 2010 - Boston
Getting Started with Riak - NoSQL Live 2010 - BostonGetting Started with Riak - NoSQL Live 2010 - Boston
Getting Started with Riak - NoSQL Live 2010 - Boston
 

Dernier

.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptxHansamali Gamage
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1DianaGray10
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud DataEric D. Schabell
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfQ4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfTejal81
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationKnoldus Inc.
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024Brian Pichman
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Muhammad Tiham Siddiqui
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTxtailishbaloch
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxNeo4j
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsDianaGray10
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 

Dernier (20)

.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfQ4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its application
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projects
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 

Riak Core: Building Distributed Applications Without Shared State

  • 1. Riak Core: Building Distributed Applications Without Shared State Commercial Users of Functional Programming Baltimore, MD · October 2010 Rusty Klophaus (@rklophaus) Basho Technologies Wednesday, October 6, 2010
  • 2. You suddenly feel an uncontrollable desire to learn Erlang. http://www.flickr.com/photos/procsilas/18014203 2 Wednesday, October 6, 2010
  • 3. What is Riak Core? How does it work? How can you use it? 3 Wednesday, October 6, 2010
  • 4. Distributed, scalable, failure-tolerant. 4 Wednesday, October 6, 2010
  • 5. Distributed, scalable, failure-tolerant. No central coordinator. Easy to setup/operate. 5 Wednesday, October 6, 2010
  • 6. Distributed, scalable, failure-tolerant. Horizontally scalable; add commodity hardware to get more X. 6 Wednesday, October 6, 2010
  • 7. Distributed, scalable, failure-tolerant. Always available. No single point of failure. Self-healing. 7 Wednesday, October 6, 2010
  • 8. Basho Technologies Riak KV Distributed, scalable, failure-tolerant key/value datastore. Started as a “Dynamo clone”. Map/Reduce, Lightweight Data Relations, Client APIs Riak Search Distributed, scalable, failure-tolerant full-text search engine. Near Realtime, Riak KV Integration, Solr Support 8 Wednesday, October 6, 2010
  • 10. Riak Riak Riak KV Core Search Wednesday, October 6, 2010
  • 11. Riak Core is an Erlang library that helps you build distributed, scalable, failure-tolerant applications. 11 Wednesday, October 6, 2010
  • 13. “We Generalized the Dynamo Architecture and Open-Sourced the Bits.” 13 Wednesday, October 6, 2010
  • 14. Wait, doesn’t *Erlang* let you build distributed, scalable, failure-tolerant applications? 14 Wednesday, October 6, 2010
  • 15. Erlang makes it easy to connect the components of your application. Client Service A Service B Service C Queue E Resource D Wednesday, October 6, 2010
  • 16. Riak Core helps you build a service that harnesses the power of many nodes. Node A Node B Node C Node D Node E Node F Node G Node H Service Node I Node J Node K Node L Node M Node N Node O ... Wednesday, October 6, 2010
  • 17. “People use languages other than Erlang?!? I find that hilarious.” -Al Gore actually said this to me. http://www.flickr.com/photos/scobleizer/2216445692 Wednesday, October 6, 2010
  • 18. How does Riak Core work? 18 Wednesday, October 6, 2010
  • 19. Command ObjectName, Payload Wednesday, October 6, 2010
  • 20. Predictable Routing 20 Wednesday, October 6, 2010
  • 21. Hash the Object Name Command ObjectName, Payload SHA1(ObjName), Payload 0 to 2^160 Wednesday, October 6, 2010
  • 22. A Naive Approach Command ObjectName, Payload SHA1(ObjName), Payload Node A Node B Node C Node D Wednesday, October 6, 2010
  • 23. A Naive Approach Command ObjectName, Payload SHA1(ObjName), Payload Node A Node B Node C Node D Node E Wednesday, October 6, 2010
  • 24. "All problems in computer science can be solved by another level of indirection." - David Wheeler 24 Wednesday, October 6, 2010
  • 25. Add VNodes What Virtual Node. Logical subdivision of the cluster. Handles incoming commands, does work, replies. For Parallelism # of VNodes = maximum concurrent requests For Rebalancing the Cluster Smallest block that can be shifted to a new node. For Resiliance The system restarts failed VNodes. 25 Wednesday, October 6, 2010
  • 26. Routing with Consistent Hash Command ObjectName, Payload SHA1(ObjName), Payload VNode 0 VNode 1 VNode 2 VNode 3 VNode 4 VNode 5 VNode 6 VNode 7 Node A Node B Node C Node D Wednesday, October 6, 2010
  • 27. Adding a Node Command ObjectName, Payload SHA1(ObjName), Payload VNode 0 VNode 1 VNode 2 VNode 3 VNode 4 VNode 5 VNode 6 VNode 7 Node A Node B Node C Node D Node E Wednesday, October 6, 2010
  • 28. Removing a Node Command ObjectName, Payload SHA1(ObjName), Payload VNode 0 VNode 1 VNode 2 VNode 3 VNode 4 VNode 5 VNode 6 VNode 7 Node A Node B Node C Node D Node E Wednesday, October 6, 2010
  • 29. The Ring Hash Location Wednesday, October 6, 2010
  • 30. The Ring Preflist Wednesday, October 6, 2010
  • 31. Writing Replicas (n_val) Preflist when N=3 Wednesday, October 6, 2010
  • 32. Routing Around Failures X Preflist when N=3 and node 0 is down. Wednesday, October 6, 2010
  • 33. Location of the Routing Layer 33 Wednesday, October 6, 2010
  • 34. Router in the Middle Client Client Client Router VNode VNode VNode VNode VNode VNode VNode VNode 0 1 3 4 2 5 6 7 Node A Node B Node C Node D Node E Wednesday, October 6, 2010
  • 35. Riak Core - Router on Each Node Client Client Client Router Router Router Router Router VNode VNode VNode VNode VNode VNode VNode VNode 0 1 3 4 2 5 6 7 Node A Node B Node C Node D Node E Wednesday, October 6, 2010
  • 36. Eventually - Router on the Client Client Client Client Router Router Router VNode VNode VNode VNode VNode VNode VNode VNode 0 1 3 4 2 5 6 7 Node A Node B Node C Node D Node E Wednesday, October 6, 2010
  • 37. No Shared State Router Router Router Router Router VNode VNode VNode VNode VNode VNode VNode VNode 0 1 3 4 2 5 6 7 Node A Node B Node C Node D Node E Wednesday, October 6, 2010
  • 38. Gossip Local Incoming Ring State Ring State Are rings equivalent? Strictly descendent? Or different? Wednesday, October 6, 2010
  • 39. Handoff When Nodes is added to the system. Node is removed from the system. Node has temporarily failed. What Ship the data backing a VNode from one node to another. 39 Wednesday, October 6, 2010
  • 40. Not Mentioned Vector Clocks Merkle Trees Bloom Filters 40 Wednesday, October 6, 2010
  • 41. Distinguished gentlemen prefer Erlang. http://www.flickr.com/photos/rebcal/3987226359 Wednesday, October 6, 2010
  • 42. How do you use Riak Core? 42 Wednesday, October 6, 2010
  • 43. Two Things to Think About Command Command = ObjectName, Payload The commands/requests/operations that you will send through the system. VNode Module The callback module that will receive the commands. 43 Wednesday, October 6, 2010
  • 44. VNode Module Startup/Shutdown init([Partition]) -> {ok, State} terminate(State) -> ok Commands handle_command(Cmd, Sender, State) -> {noreply, State1} | {reply, Reply, State1} handle_handoff_command(Cmd, Sender, State) -> {noreply, State1} | {reply, ok, State1} 44 Wednesday, October 6, 2010
  • 45. VNode Module Handoff Coordination handoff_starting(Node, State) -> {Bool, State1} encode_handoff_data(Data, State) -> <<Binary>>. handle_handoff_data(Data, Sender, State) -> {reply, ok, State1} handoff_finished(Node, State) -> {ok, State1} 45 Wednesday, October 6, 2010
  • 46. Start the riak_core application riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... application:start(riak_core). 46 Wednesday, October 6, 2010
  • 47. Start the riak_core application riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... Supervise vnode processes. 47 Wednesday, October 6, 2010
  • 48. Start the riak_core application Start, coordinate, and supervise handoff. 48 Wednesday, October 6, 2010
  • 49. Start the riak_core application riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... Maintain cluster membership information. 49 Wednesday, October 6, 2010
  • 50. Start the riak_core application riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... Monitor node liveness, broadcast to registered modules. 50 Wednesday, October 6, 2010
  • 51. Start the riak_core application riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... Send ring information to other nodes. Reconcile different views of the cluster. Rebalance cluster when nodes join or leave. 51 Wednesday, October 6, 2010
  • 52. In your application... riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... Start the vnodes for your application. Master = { riak_X_vnode_master, { riak_core_vnode_master, start_link, [riak_X_vnode] }, permanent, 5000, worker, [riak_core_vnode_master] }, {ok, { {one_for_one, 5, 10}, [Master]} }. 52 Wednesday, October 6, 2010
  • 53. In your application... riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... Tell riak_core that your application is ready to receive requests. riak_core:register_vnode_module(riak_X_vnode), riak_core_node_watcher:service_up(riak_X, self()) 53 Wednesday, October 6, 2010
  • 54. In your application... riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... riak_core riak_core_vnode_sup riak_core_handoff_* riak_core_node_* X_vnode X_vnode riak_core_ring_* riak_core_gossip_* X_vnode ... Join to an existing node in the cluster. riak_core_gossip:send_ring(ClusterNode, node()) 54 Wednesday, October 6, 2010
  • 55. Start Sending Commands # Figure out the preflist... {_Verb, ObjName, _Payload} = Command, PrefList = riak_core_apl:get_apl(ObjName, NVal, riak_X), # Send the command... riak_core_vnode_master:command(PrefList, Command, riak_X_vnode_master) 55 Wednesday, October 6, 2010
  • 56. Review Riak Core Open source Erlang library for building distributed, scalable, failure tolerant applications. Continual improvement in the coming months. Riak KV Key/Value datastore with map/reduce based on Riak Core. Riak Search Full-text, near real-time search engine based on Riak Core. 56 Wednesday, October 6, 2010
  • 57. At the center of the universe, beneath the bottom-most turtle, is a cluster of Erlang nodes. http://www.flickr.com/photos/jurvetson/469492885 Wednesday, October 6, 2010
  • 58. At the center of the universe, beneath the bottom-most turtle, is a cluster of Erlang nodes. With 100% uptime. http://www.flickr.com/photos/jurvetson/469492885 Wednesday, October 6, 2010
  • 59. Thanks! Questions? Learn More More Information: http://wiki.basho.com Amazon’s Dynamo Paper Get the Code http://hg.basho.com/riak_core Get in Touch Twitter: @rklophaus, @basho/team Email: rusty@basho.com 59 Wednesday, October 6, 2010