3. Why NoSQL?
✤ Relational databases don’t scale
✤ The relational model maps poorly to some problems
✤ Relational databases are slow
Saturday, August 21, 2010
4. Myth 1
✤ “NoSQL is for people who don’t understand {SQL,
denormalization, query tuning, ...}”
✤ Similarly: “Only users of [database X] are turning to NoSQL
databases, because X sucks.”
Saturday, August 21, 2010
5. eBay: NoSQL pioneer
✤ “BASE is diametrically opposed to ACID. Where ACID is
pessimistic and forces consistency at the end of every
operation, BASE is optimistic and accepts that the
database consistency will be in a state of flux. Although
this sounds impossible to cope with, in reality it is quite
manageable and leads to levels of scalability that cannot
be obtained with ACID.”
✤ --BASE: An Acid Alternative, Dan Pritchett, eBay
✤ See also: The eBay Architecture, Randy Shoup and Dan Pritchett
Saturday, August 21, 2010
7. Myth 2
✤ “NoSQL is nothing new because we had key/value
databases like bdb years ago.”
Saturday, August 21, 2010
8. Myth 3
✤ “Only huge web 2.0 sites like Facebook and Twitter care
about scalability.”
Saturday, August 21, 2010
9. Cassandra
✤ Relational databases don’t scale
✤ The relational model maps poorly to some problems
✤ Relational databases are slow
Saturday, August 21, 2010
16. Use cases
✤ Digital Reasoning: NLP + entity analytics
✤ OpenX: largest publisher-side ad network in the world
✤ Cloudkick: performance data & aggregation
✤ SimpleGEO: location-as-API
✤ Ooyala: video analytics and business intelligence
✤ ngmoco: massively multiplayer game worlds
Saturday, August 21, 2010
17. Cassandra
✤ Relational databases don’t scale
✤ The relational model maps poorly to some problems
✤ Relational databases are slow
Saturday, August 21, 2010
19. Cassandra memory-aware index
<key 127> <row data 0>
<key 255> <row data 1>
... ...
<row data 127>
...
<row data 255>
...
Saturday, August 21, 2010
20. Reader
Writer Memtable
Commitlog
The Log-Structured Merge-Tree,
Bigtable: A Distributed Storage System
for Structured Data
Saturday, August 21, 2010
22. Durable
✤ Write to commitlog
✤ fsync is cheap since it’s append-only
✤ Write to memtable
✤ [amortized] flush memtable to sstable
Saturday, August 21, 2010
31. Some headlines
✤ “Resyncing Broken MySQL Replication”
✤ “How To Repair MySQL Replication”
✤ “Fixing Broken MySQL Database Replication”
✤ “Replication on Linux broken after db restore”
✤ “MySQL :: Repairing broken replication”
Saturday, August 21, 2010
32. The opposite of heroes
✤ “If your software wakes someone up at 4 AM to fix it,
you’re doing it wrong.”
Saturday, August 21, 2010
33. Good architecture solves multiple
problems at once
✤ Availability in single datacenter
✤ Availablility in multiple datacenters
✤ Built-in repair rather than an afterthought
✤ or worse, having to restart replication entirely
Saturday, August 21, 2010
34. Y
Key “C”
A
W
U
F
T
L
P
Saturday, August 21, 2010
56. Cassandra “indexes” are
materialized views
SELECT * FROM tweets
WHERE user_id IN (SELECT follower FROM followers WHERE user_id = ?)
ORDER BY time_tweeted DESC
LIMIT 40
followers timeline
?
SORT
? uuid:tweet
tweets
Saturday, August 21, 2010
57. Analytics in Cassandra
✤ @afex: “Cassandra + Pig (Hadoop) is very exciting. A 7
line script to analyze data from my entire cluster
transparently, with no ETL? Yes, please”
Saturday, August 21, 2010
58. TaskTracker
JobTracker
Saturday, August 21, 2010
59. Coming in 0.7 [beta1 released 13/8]
✤ Secondary indexes and expressions
✤ Large (> 2GB) rows
✤ More control over replica placement
✤ Online schema changes
✤ Flow control
✤ Dynamic routing around slow nodes
Saturday, August 21, 2010
60. More
✤ http://wiki.apache.org/cassandra/
ArticlesAndPresentations
✤ http://wiki.apache.org/cassandra/ArchitectureInternals
Saturday, August 21, 2010
