This presentation was delivered during Softshake 2013. Learn why RDBMS are not enought and why NoSQL help developers to scale their applications and provide agility.

1. Introduc)on
to
NoSQL
with
Couchbase
Tugdual
“Tug”
Grall
Technical
Evangelist
tug@couchbase.com
Monday, October 28, 13
TwiBer:
tgrall

2. {“about”
:
“me”}
• Tugdual
“Tug”
Grall
-­‐
Couchbase
-­‐
-­‐
eXo
-­‐
-­‐
Technical
Evangelist
CTO
Oracle
• Web
-­‐
-­‐
-­‐
@tgrall
hEp://blog.grallandco.com
tgrall
• NantesJUG
co-­‐founder
• Pet
Project
:
• hEp://www.resultri.com
-­‐
• tug@couchbase.com
-­‐
-­‐
Developer/Product
Manager
Mainly
Java/SOA
• tugdual@gmail.com
Developer
in
consulAng
firms
Monday, October 28, 13

3. RDBMS
are
not
Enough?
Monday, October 28, 13

4. Growth
is
the
New
Reality
• Instagram
gained
nearly
1
million
users
overnight
when
then
expanded
to
Android
Monday, October 28, 13

5. 50
Million
Users
in
50
Days
Draw
Something
by
OMGPOP
Daily
Ac)ve
Users
(millions)
16
14
12
10
8
6
4
2
2/6
8
10
Monday, October 28, 13
12
14
16
18
20
22
24
26
28
3/1
3
5
7
9
11
13
15
17
19
21

6. How
do
you
take
this
growth?
Applica;on
Scales
Out
Just
add
more
commodity
web
servers
System
Cost
Applica;on
Performance
Web/App
Server
Tier
Users
RDBMS
Scales
Up
Get
a
bigger,
more
complex
server
System
Cost
Applica;on
Performance
Won’t
scale
beyond
this
point
Rela2onal
Database
Users
RDBMS
is
good
for
many
thing,
but
hard
to
scale
Monday, October 28, 13

7. Scaling
out
RDBMS
• Run
Many
SQL
Servers
Web/App
Server
Tier
• Data
could
be
sharded
Memcached
Tier
­ Done
by
the
applicaAon
code
• Caching
for
faster
response
)me
MySQL
Tier
Monday, October 28, 13

8. NoSQL
Technology
Scales
Out
Applica;on
Scales
Out
Just
add
more
commodity
web
servers
System
Cost
Applica;on
Performance
Web/App
Server
Tier
Users
NoSQL
Database
Scales
Out
Cost
and
performance
mirrors
app
;er
System
Cost
Applica;on
Performance
NoSQL
Distributed
Data
Store
Users
Scaling
out
fla?ens
the
cost
and
performance
curves
Monday, October 28, 13

9. A
New
Technology?
Bigtable
November
2006
Dynamo
October
2007
Cassandra
August
2008
Voldemort
February
2009
Building
new
database
to
answer
the
following
requirements
• No
schema
required
before
inser)ng
data
• No
schema
change
required
to
change
data
format
• Auto-­‐sharding
without
applica)on
par)cipa)on
• Distributed
queries
• Integrated
main
memory
caching
• Data
synchroniza)on
(
mul)-­‐datacenter)
Very
few
organiza)ons
want
to
(fewer
can)
build
and
maintain
database
so]ware
technology.
But
every
organiza)on
building
interac)ve
web
applica)ons
needs
this
technology.
Monday, October 28, 13

10. What
Is
Biggest
Data
Management
Problem
Driving
Use
of
NoSQL
in
Coming
Year?
49%
35%
29%
16%
Lack
of
flexibility/
rigid
schemas
Inability
to
scale
Performance
challenges
out
data
Source:
Couchbase
Survey,
December
2011,
n
=
1351.
Monday, October 28, 13
Cost
12%
11%
All
of
these
Other

11. NoSQL
Catalog
Cache
(memory
only)
Key-­‐Value
Memcached
Data
Structure
Document
Column
Graph
Redis
Database
(memory/disk)
Coherence
Membase
Cassandra
Neo4j
MongoDB
Monday, October 28, 13
Couchbase
HBase
InfiniteGraph

12. Opera)onal
vs.
Analy)c
Databases
Real-­‐Pme,
InteracPve
Databases
AnalyPc
Databases
NoSQL
Fast
access
to
data
Couchbase
MongoDB
Monday, October 28, 13
Get
insights
from
data
Cloudera
Cassandra
Hortonworks
Hbase
Mapr

13. Document
Database
Monday, October 28, 13

14. Rela;onal
vs
Document
Data
Model
C1
C2
C3
C4
{
JSON
JSON
}
JSON
RelaPonal
data
model
Document
data
model
Highly-­‐structured
table
organiza;on
with
rigidly-­‐defined
data
formats
and
record
structure.
Collec;on
of
complex
documents
with
arbitrary,
nested
data
formats
and
varying
“record”
format.
Monday, October 28, 13

15. Rela;onal
Approach
Monday, October 28, 13

16. Document
Database
o::1001
{
uid: “ji22jd”,
customer: “Ann”,
line_items: [
{ sku: 0321293533, qty: 2, price: 48.0 },
{ sku: 0321601912, qty: 1, price: 39.0 },
{ sku: 0131495054, qty: 1, price: 51.0 }
],
payment: {
type: “Amex”,
expiry: “04/2001”,
last5: 12345
}
}
Aggregate
Oriented
Database
hEp://marAnfowler.com/bliki/AggregateOrientedDatabase.html
Monday, October 28, 13

17. Schema
Update
:
RDBMS
Event
Info
ID
1
Title
Speaker
Speaker_id
Paris
JUG
ID
Name
rbin
rbin
Robin
Johnson
1.
Create
a
new
table
2
tgrall
QCON
UK
2.“Prepare”
the
Data
(move
to
new
tgrall drop
column)
table,
Tug
Grall
A
talk
could
be
done
by
one
or
more
speakers!
3.
Change
the
constraints
3
4
NoSQL
CLN
johnz
johnz
John
Zablo
fweigel
fweigel
Frank
Weigel
4.
Change
the
app
Devoxx
Monday, October 28, 13

18. Schema
Update
:
Document
user:rbin
{ user:rbin
{ user:rbin
type: “user”,
{
type: “user”,
name: “Robin Johnson”,
type: “user”,
name: “Robin Johnson”,
}
} name: “Robin Johnson”,
}
event:1
{ event:1
1. Change
The
ApplicaPon
{ event:1
type: “event”,
{ event:1
type: “event”, the
• Update
{ event:1
event:1
title: “Paris JUG”, Data
when
“you”
want
type: “Paris JUG”,
{
title: “event”, the
index
to
query
the
new
format
“event”,
{type: “Paris
•
speaker: “rbin” JUG”,
title: Change
type:“rbin” JUG”,
title: “event”,
} speaker: “Paris JUG”,
type: “Paris
title: “event”,
} speaker: “rbin” JUG”,
title: “rbin”
} speaker: “Paris
speaker: “rbin”
} speaker: [“rbin”,”tgrall”]
}
}
Monday, October 28, 13

19. Couchbase
Server
Monday, October 28, 13

20. Couchbase
Server
Core
Principles
Easy
Scalability
PE
RF O R M A N C E
Grow
cluster
without
applicaAon
changes,
without
downAme
with
a
single
click
Always
On
24x365
No
downAme
for
sodware
upgrades,
hardware
maintenance,
etc.
Monday, October 28, 13
Consistent
High
Performance
Consistent
sub-­‐millisecond
read
and
write
response
Ames
with
consistent
high
throughput
JSON
JSON JSO
JSON N
JSON
Flexible
Data
Model
JSON
document
model
with
no
fixed
schema.

21. Monday, October 28, 13

22. Couchbase
Architecture
8092
11211
Query
API
Memcapable
1.0
11210
Memcapable
2.0
New
Persistence
Layer
vBucket
state
and
replica;on
manager
Node
health
monitor
Rebalance
orchestrator
Global
singleton
supervisor
storage
interface
Configura;on
manager
Data
Manager
Process
monitor
Couchbase
EP
Engine
Heartbeat
Memcached
REST
management
API/Web
UI
Query
Engine
Moxi
Cluster
Manager
hRp
on
each
node
one
per
cluster
Erlang/OTP
HTTP
8091
Monday, October 28, 13
Erlang
port
mapper
4369
Distributed
Erlang
21100
-­‐
21199

23. Couchbase
Architecture
8092
11211
Query
API
Memcapable
1.0
11210
Memcapable
2.0
New
Persistence
Layer
Disk
Persistence
vBucket
state
and
replica;on
manager
Node
health
monitor
Rebalance
orchestrator
Global
singleton
supervisor
Configura;on
manager
storage
interface
Server/Cluster
Management
&
Communica2on
(Erlang)
hRp
Process
monitor
RAM
Cache,
Indexing
Couchbase
EP
Engine
&
Persistence
Management
(C
&
V8)
Heartbeat
Object-­‐level
Cache
REST
management
API/Web
UI
Query
Engine
Moxi
on
each
node
one
per
cluster
Erlang/OTP
The Unreasonable Effectiveness of C by Damien Katz
HTTP
8091
Monday, October 28, 13
Erlang
port
mapper
4369
Distributed
Erlang
21100
-­‐
21199

24. Open
Source
Project
Apache
2.0
hEps://github.com/couchbase/
hEps://github.com/couchbaselabs/
Monday, October 28, 13
Gerrit: hEp://review.couchbase.org/

25. Cluster-­‐wide
Basic
Opera)on
App
Server
1
App
Server
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
Cluster
Map
Cluster
Map
READ/WRITE/UPDATE
Server
1
Server
2
Server
3
Ac2ve
Ac2ve
Ac2ve
Doc
5
Doc
Doc
4
Doc
Doc
1
Doc
Doc
2
Doc
Doc
7
Doc
Doc
2
Doc
Doc
9
Doc
Doc
8
Doc
Doc
6
Doc
REPLICA
REPLICA
REPLICA
• Docs
distributed
evenly
across
servers
• Each
server
stores
both
ac)ve
and
replica
docs
Only
one
server
acAve
at
a
Ame
• Client
library
provides
app
with
simple
interface
to
database
• Cluster
map
provides
map
to
which
server
doc
is
on
Doc
4
Doc
Doc
6
Doc
Doc
7
Doc
Doc
1
Doc
Doc
3
Doc
Doc
9
Doc
• App
reads,
writes,
updates
docs
Doc
8
Doc
Doc
2
Doc
Doc
5
Doc
• Mul)ple
app
servers
can
access
same
document
at
same
)me
Couchbase
Server
Cluster
User
Configured
Replica
Count
=
1
Monday, October 28, 13
App
never
needs
to
know

26. Add
Nodes
to
Cluster
App
Server
1
App
Server
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
Cluster
Map
Cluster
Map
READ/WRITE/UPDATE
READ/WRITE/UPDATE
Server
1
Server
2
Server
3
Server
4
Server
5
Ac2ve
Ac2ve
Ac2ve
Ac2ve
Ac2ve
Doc
5
Doc
Doc
4
Doc
Doc
1
Doc
Doc
2
Doc
Doc
7
Doc
Doc
2
Doc
Doc
9
Doc
Doc
8
Doc
Doc
6
Doc
REPLICA
REPLICA
REPLICA
Doc
4
Doc
Doc
6
Doc
Doc
7
Doc
Doc
1
Doc
Doc
3
Doc
Doc
9
Doc
Doc
8
Doc
Doc
2
Doc
Doc
5
Doc
Couchbase
Server
Cluster
User
Configured
Replica
Count
=
1
Monday, October 28, 13
• Two
servers
added
One-­‐click
opera)on
• Docs
automa)cally
rebalanced
across
cluster
Even
distribuAon
of
docs
Minimum
doc
movement
REPLICA
REPLICA
• Cluster
map
updated
• App
database
calls
now
distributed
over
larger
number
of
servers

27. Fail
Over
Node
App
Server
1
App
Server
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
Cluster
Map
Cluster
Map
Server
1
Server
2
Server
3
Server
4
Server
5
Ac2ve
Ac2ve
Ac2ve
Ac2ve
Ac2ve
Doc
5
Doc
Doc
4
Doc
Doc
1
Doc
Doc
9
Doc
Doc
2
Doc
Doc
7
Doc
Doc
2
Doc
Doc
8
Doc
Doc
1
Doc
6
Doc
Doc
Doc
3
REPLICA
REPLICA
REPLICA
REPLICA
Doc
4
Doc
Doc
6
Doc
Doc
7
Doc
Doc
5
Doc
1
Doc
Doc
3
Doc
Doc
9
Doc
Doc
2
Couchbase
Server
Cluster
User
Configured
Replica
Count
=
1
Monday, October 28, 13
Doc
REPLICA
Doc
8
Doc
Doc
• App
servers
accessing
docs
• Requests
to
Server
3
fail
• Cluster
detects
server
failed
Promotes
replicas
of
docs
to
acAve
Updates
cluster
map
• Requests
for
docs
now
go
to
appropriate
server
• Typically
rebalance
would
follow

28. Indexing
and
Querying
APP
SERVER
1
APP
SERVER
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
CLUSTER
MAP
CLUSTER
MAP
Query
SERVER
1
SERVER
2
ACTIVE
ACTIVE
SERVER
3
ACTIVE
• Indexing
work
is
distributed
amongst
nodes
Doc
5
Doc
Doc
5
Doc
Doc
5
Doc
Doc
2
Doc
Doc
2
Doc
Doc
2
Doc
• Parallelize
the
effort
Doc
9
Doc
Doc
9
Doc
Doc
9
Doc
• Each
node
has
index
for
data
stored
on
it
REPLICA
REPLICA
REPLICA
Doc
4
Doc
Doc
4
Doc
Doc
4
Doc
Doc
1
Doc
Doc
1
Doc
Doc
1
Doc
Doc
8
Doc
Doc
8
Doc
Doc
8
Doc
COUCHBASE
SERVER
CLUSTER
Monday, October 28, 13
• Large
data
set
possible
• Queries
combine
the
results
from
required
nodes

29. Couchbase
SDK
Official
SDKs
Ruby
Python
Community
SDKs
Go
Clojure
www.couchbase.com/develop
Monday, October 28, 13

30. Full
Text
Search
• Elastic Search is good for ad-hoc queries and faceted browsing
• Our adapter is aware of changing Couchbase topology
• Indexed by Elastic Search after stored to disk in Couchbase
ElasPcSearch
Monday, October 28, 13

31. The
Complete
Mobile
Solu;on
Available
now
in
Beta
:
hfp://mobile.couchbase.com
Monday, October 28, 13

32. Couchbase
Docs
www.couchbase.com/docs/
Couchbase
Forums
www.couchbase.com/communi;es
Meetup
hbp://meetup.com/Couchbase-­‐France/
Mobile
(beta)
hbp://mobile.couchbase.com/
N1QL
-­‐
New
Query
Engine
(DP)
hbp://query.couchbase.com
Monday, October 28, 13
Contact
me
on
Twi?er
@tgrall
Contact
me
by
Email
tug@couchbase.com
IRC
#couchbase
#libcouchbase

33. Introduc)on
to
NoSQL
with
Couchbase
Tugdual
“Tug”
Grall
Technical
Evangelist
tug@couchbase.com
Monday, October 28, 13
TwiBer:
tgrall