Just a few years ago all software systems were designed to be monoliths running on a single big and powerful machine. But nowadays most companies desire to scale out instead of scaling up, because it is much easier to buy or rent a large cluster of commodity hardware then to get a single machine that is powerful enough. In the database area scaling out is realized by utilizing a combination of polyglot persistence and sharding of data. On the application level scaling out is realized by microservices. In this talk I will briefly introduce the concepts and ideas of microservices and discuss their benefits and drawbacks. Afterwards I will focus on the point of intersection of a microservice based application talking to one or many NoSQL databases. We will try and find answers to these questions: Are the differences to a monolithic application? How to scale the whole system properly? What about polyglot persistence? Is there a data-centric way to split microservices?

1. www.arangodb.com
NoSQL meets Microservices
Michael Hackstein
@mchacki

2. Michael Hackstein
‣ ArangoDB Core Team
‣ Web Frontend
‣ Graph visualisation
‣ Graph features
‣ Host of cologne.js
‣ Master’s Degree
(spec. Databases and
Information Systems)
2

3. Classical Setup
3

4. Scaling
4
LoadBalancer

5. Responsibilities
5
LoadBalancer
Application Code
Consistency
Joins

6. app
Monolith breakdown
6
server.route({
method: "GET", path: "/app",
handler: function (req, reply) {
var sum = cpuWaste();
reply(dataLookup["key" + sum]);
}
});
var cpuWaste = function() {
var sum = Math.floor(Math.random() * 10000);
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
var dataLookup = {};
for (var j = 0; j < 500000; ++j) {
dataLookup["key" + j] =
"This is test text number " + j + ".";
}

7. memService
cpuService
app
Monolith breakdown
6
server.route({
method: "GET", path: "/app",
handler: function (req, reply) {
var sum = cpuWaste();
reply(dataLookup["key" + sum]);
}
});
var cpuWaste = function() {
var sum = Math.floor(Math.random() * 10000);
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
var dataLookup = {};
for (var j = 0; j < 500000; ++j) {
dataLookup["key" + j] =
"This is test text number " + j + ".";
}

8. memService
cpuService
app
Monolith breakdown
6
server.route({
method: "GET", path: "/app",
handler: function (req, reply) {
var sum = cpuWaste();
reply(dataLookup["key" + sum]);
}
});
var cpuWaste = function() {
var sum = Math.floor(Math.random() * 10000);
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
var dataLookup = {};
for (var j = 0; j < 500000; ++j) {
dataLookup["key" + j] =
"This is test text number " + j + ".";
}
server.route({
method: "GET", path: "/mem/{id}",
handler: function (request, reply) {
reply(dataLookup["key" + request.params.id]);
}
});

9. memService
cpuService
app
Monolith breakdown
6
server.route({
method: "GET", path: "/app",
handler: function (req, reply) {
var sum = cpuWaste();
reply(dataLookup["key" + sum]);
}
});
var cpuWaste = function() {
var sum = Math.floor(Math.random() * 10000);
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
var dataLookup = {};
for (var j = 0; j < 500000; ++j) {
dataLookup["key" + j] =
"This is test text number " + j + ".";
}
server.route({
method: "GET", path: "/mem/{id}",
handler: function (request, reply) {
reply(dataLookup["key" + request.params.id]);
}
});
server.route({
method: "GET", path: "/cpu",
handler: function (request, reply) {
reply(cpuWaste());
}
});

10. memService
cpuService
app
Monolith breakdown
6
var cpuWaste = function() {
var sum = Math.floor(Math.random() * 10000);
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
var dataLookup = {};
for (var j = 0; j < 500000; ++j) {
dataLookup["key" + j] =
"This is test text number " + j + ".";
}
server.route({
method: "GET", path: "/mem/{id}",
handler: function (request, reply) {
reply(dataLookup["key" + request.params.id]);
}
});
server.route({
method: "GET", path: "/cpu",
handler: function (request, reply) {
reply(cpuWaste());
}
});
server.route({
method: "GET",
path: "/app",
handler: function (req, reply) {
request('http://127.0.0.1:9000/cpu',
function (error, response, body) {
if (!error
&& response.statusCode == 200) {
request('http://127.0.0.1:9000/mem/'
+ body,
function (error, response, body) {
if (!error
&& response.statusCode == 200) {
reply(body);
} else {
reply({error: error});
}
});
} else {
reply({error: error});
}
});
}
});

11. Introduction to NoSQL
7
NoSQL World
Documents - JSON
Graphs
Key Value
{
“type“: "pants",
“waist": 32,
“length”: 34,
“color": "blue",
“material”: “cotton"
}
{
“type“: "television",
“diagonal screen size": 46,
“hdmi inputs": 3,
“wall mountable": true,
“built-in digital tuner": true,
“dynamic contrast ratio”: “50,000:1”,
Resolution”: “1920x1080”
}
{
“type": "sweater",
“color": "blue",
“size": “M”,
“material”: “wool”,
“form”: “turtleneck"
}
{
“type": "sweater",
“color": "blue",
“size": “M”,
“material”: “wool”,
“form”: “turtleneck"
}
‣ Map value data to unique string keys (identifiers)
‣ Treat data as opaque (data has no schema)
‣ Can implement scaling and partitioning easily
‣ Focussed on m-to-n relations between entities
‣ Stores property graphs: entities and edges can have
attributes
‣ Easily query paths of variable length
‣ Normally based on key-value stores (each document still
has a unique key)
‣ Allow to save documents with logical similarity in
“collections”
‣ Treat data records as attribute-structured documents
(data is no more opaque)
‣ Often allow querying and indexing document attributes

12. Polyglot Modeling
‣ If you have structured data
➡ Use a document store
‣ If you have relations between entities and want to efficiently query
them by arbitrary steps in between
➡ Use a graph database
‣ If you manage the data structure yourself and do not need
complex queries
➡ Use a key-value store
‣ If you have structured data in graph format, or data model might
change
➡ Use a multi-model database
8
Use the right tool for the job

13. Using different databases
9
LoadBalancer

14. Responsibilities
10
LoadBalancer
Application Code
Consistency
Joins

15. Responsibilities
10
LoadBalancer
Application Code
Consistency
Joins

16. Solution 1: Writes only to one master
11

17. Solution 2: Microservices
12

18. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

19. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

20. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

21. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

22. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

23. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

24. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

25. neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})
6 network hops
+ 2 queries

26. Live Demo
Neo4J + MongoDB
14
https://github.com/mchacki/ms-meets-nosql-examples

27. Multi Model Database
‣ Can natively store several kinds of data models:
‣ Key-value pairs
‣ Documents
‣ Graphs
‣ Delivers query mechanisms for all data models
‣ ACID transactions
‣ Cross collection joins
15

28. Using a Multi-Model Database
16
LoadBalancer
Application Code
Consistency
Joins

29. Using a Multi-Model Database
16
LoadBalancer
Application Code Consistency
Joins

30. Foxx
‣ Customized REST API on top of ArangoDB
‣ Microservice framework
‣ Integrate with other microservices
‣ Reuse your Node.js code and NPM modules
‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth
‣ Operations are encapsulated in the database
‣ low network traffic, direct data access
‣ increases data privacy
/
(~(
) ) /_/
( _-----_(@ @)
( /
/|/--| V
" " " "
17

31. foxx
productService
Foxx Setup
18
RETURN GRAPH_NEIGHBORS('ecom',
@id, {direction: 'outbound',
minDepth: 2, maxDepth: 2,
neighborCollectionRestriction: 'Products',
includeData: true
})
4 network hops
+ 1 query

32. foxx
productService
Foxx Setup
18
RETURN GRAPH_NEIGHBORS('ecom',
@id, {direction: 'outbound',
minDepth: 2, maxDepth: 2,
neighborCollectionRestriction: 'Products',
includeData: true
})
4 network hops
+ 1 query

33. foxx
productService
Foxx Setup
18
RETURN GRAPH_NEIGHBORS('ecom',
@id, {direction: 'outbound',
minDepth: 2, maxDepth: 2,
neighborCollectionRestriction: 'Products',
includeData: true
})
4 network hops
+ 1 query

34. foxx
productService
Foxx Setup
18
RETURN GRAPH_NEIGHBORS('ecom',
@id, {direction: 'outbound',
minDepth: 2, maxDepth: 2,
neighborCollectionRestriction: 'Products',
includeData: true
})
4 network hops
+ 1 query

35. foxx
productService
Foxx Setup
18
RETURN GRAPH_NEIGHBORS('ecom',
@id, {direction: 'outbound',
minDepth: 2, maxDepth: 2,
neighborCollectionRestriction: 'Products',
includeData: true
})
4 network hops
+ 1 query

36. Live Demo
Foxx
19
https://github.com/mchacki/ms-meets-nosql-examples

37. ‣ open source and free (Apache 2 license)
‣ sharding & replication
‣ JavaScript throughout (V8 built into server)
‣ drivers for a wide range of languages
‣ web frontend
‣ good & complete documentation
‣ professional as well as community support
20
An overview of other features

38. Thank you
‣ Further questions?
‣ Follow me on twitter/github: @mchacki
‣ Write me a mail: mchacki@arangodb.com
‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb
‣ Visit us at our booth
‣ Free T-shirts
21