So, there's this "NoSQL" thing you may have heard of, and this related thing called "eventual consistency". Supposedly, they help you scale, but no one has ever explained why! Well, wonder no more! This talk will demystify NoSQL, eventual consistency, how they might help you scale, and -- most importantly -- why you should care.
We'll look closely at how Riak, a linearly-scalable, distributed and fault-tolerant NoSQL datastore, implements eventual consistency, and how you can harness it from Ruby via the slick Ripple client/ORM. When the talk is finished, you'll have the tools both to understand eventual consistency and to handle it like a pro inside your next Ruby application.
32. Harvest & Yield
CAP “For Dummies”
Harvest: how much of dataset is
reflected in a response (C)
33. Harvest & Yield
CAP “For Dummies”
Harvest: how much of dataset is
reflected in a response (C)
Yield: how likely is the datastore
to complete request (A)
36. # Do I need it?
def eventual_consistency?
@money === @uptime
end
37. # Do I need it?
def eventual_consistency?
@money === @uptime
end
# “Any sufficiently large
# system is in a constant
# state of failure.”
# --Justin Sheehy, Basho CTO
77. # gem install riak-client
require ‘riak’
# Connect to Riak on localhost
client = Riak::Client.new
# Fetch something, write it to a file
o = client[‘slide-decks’][‘RubyConfAR’]
File.open(‘slides.key’,‘wb’) do |f|
f.write o.raw_data
end
78. # Create a new key
v = client[‘venues’].new(‘RubyConfAR’)
# Set the value, JSON by default
v.data = {
name: ‘Ciudad Cultural Konex’,
address: {
street: ‘Sarmiento 3131’,
city: ‘Buenos Aires’,
zip: ‘C1196AAG’
}
}
# Write it to Riak
v.store
79. # Create a new key for an image
pic = client[‘pictures’].new(‘bue1.jpg’)
# Read the file, set the content-type
pic.raw_data = File.read(‘bue1.jpg’)
pic.content_type = ‘image/jpeg’
# Set some metadata
pic.meta[‘camera’] = ‘Nikon’
pic.indexes[‘date_int’] << Time.now.to_i
# Write it to Riak
pic.store
81. require ‘ripple’ # gem install ripple
class Person
include Ripple::Document
property :name, String, presence: true
key_on :name
many :addresses
end
class Address
include Ripple::EmbeddedDocument
property :street, String
property :city, String
property :zip, String
end
82. # Create my info
me = Person.new(
name: ‘Sean Cribbs’,
addresses: [{
street: ‘929 Market St’,
city: ‘San Francisco’,
zip: ‘94103’
}]
)
me.save! # Store as JSON under
# “people/Sean Cribbs”
me == Person.find(‘Sean Cribbs’) # true
83. That’s kind of cool.
But what about
Eventual Consistency?
87. Causal Consistency
Each read includes a token of the
current state (vector clock)
Clients send the token back with writes
88. Causal Consistency
Each read includes a token of the
current state (vector clock)
Clients send the token back with writes
Riak updates the token and detects stale
versions and conflicts by comparison
89. Causal Consistency
Each read includes a token of the
current state (vector clock)
Clients send the token back with writes
Riak updates the token and detects stale
versions and conflicts by comparison
Conflicts are exposed to your
application
105. # A cart is { product_id => quantity }
Riak::RObject.on_conflict do |obj|
if obj.bucket.name == ‘carts’
obj.content_type = ‘application/json’
obj.data = {}
obj.siblings.each do |val|
val.data.each do |product,q|
obj.data[product] ||= 0
obj.data[product] += q
end
end
obj
end
end
106. # Documents can be resolved too
Person.on_conflict(:addresses) do |vals|
vals.each do |doc|
self.addresses &= doc.addresses
end
end
109. Commutative
Replicated Data-Types
Recent area of research
Data-type defines
commutative operations
http://hal.inria.fr/inria-00397981/en/
110. Commutative
Replicated Data-Types
Recent area of research
Data-type defines
commutative operations
Value contains a state
and limited history of operations
http://hal.inria.fr/inria-00397981/en/
111. Commutative
Replicated Data-Types
Recent area of research
Data-type defines
commutative operations
Value contains a state
and limited history of operations
Roll-back, merge and replay
to resolve
http://hal.inria.fr/inria-00397981/en/
115. Conclusion
Any sufficiently large system is
inconsistent and constantly failing
116. Conclusion
Any sufficiently large system is
inconsistent and constantly failing
Riak remains available during failures,
progresses toward consistency
117. Conclusion
Any sufficiently large system is
inconsistent and constantly failing
Riak remains available during failures,
progresses toward consistency
Riak & Ripple help your application
recover from inconsistency