https://www.bigdataspain.org/2016/program/thu-next-generation-polyglot-architectures-using-neo4j.html https://www.youtube.com/watch?v=kVbUJHNyccc&t=18s&index=32&list=PL6O3g23-p8Tr5eqnIIPdBD_8eE5JBDBik

2. Neo Technology
Polyglot Architectures
September 2016
Stefan Kolmar
Director Field Engineering

3. Agenda
• Drivers for change in enterprises
• Requirements created for polyglot architecture
• Tradi<onal approaches
• Benefits of using graph technology
• Neo4j as na<ve GraphDB
• Architectures and Ecosystems
• Case studies
• Conclusions
Admin

4. Drivers for change in enterprises
• Data explosion
Admin

5. Drivers for change in enterprises
• Data explosion
• IoT / Industrie 4.0
Admin

6. Drivers for change in enterprises
• Data explosion
• IoT / Industrie 4.0
• Cloud Compu<ng
Admin

7. Drivers for change in enterprises
• Data explosion
• IoT / Industrie 4.0
• Cloud Compu<ng
• Need to use Connected Data Admin

8. The need of connected data in the Enterprise
Connected
Enterprise
Employees,
Customers
and Partners
Digital
Mesh
Internet of
Connected
Things
Knowledge
Network
7

9. Requirements created for polyglot architecture
• Ability to handle large scale data volumes
• High availability 24x7
• Open interfaces
• Structure data to take advantage of connec<ons
• Ability to query connected data on demand
• Flexibility to connect and query data with different schema
-> Using the technology which fits best doing the job
Admin

10. TradiBonal approaches
• Using RDBMS systems:
• En<ty rela<onship models
• Logical and physical Schema
• SQL queries
Admin

11. Database Landscape
1990
RDBMS
2000
RDBMS
(Opera<onal)
Data
Warehouse
2010
RDBMS
Hadoop
NoSQL
OLAP/DW
Today
RDBMS
Hadoop
NoSQL
OLAP/DW
GraphDB
Spark

12. Today: sBll the reality storing data

13. What is the problem with that?

14. What is the problem with that?
• Understandable for business
users?
• Agility?
• Needs effort to materialize
rela<onships
• Logical versus physical data
model?
• Op<miza<on needed to
perform queries?

15. Case Studies

16. What is the most powerful
database in the world?

19. AlternaBve Database: Graph Data Model

20. Advantages using Graph DB
Time to
market
Flexibility Opera<onal
Efficiency

21. The Whiteboard Model Is the Physical Model

22. CAR
name: “Dan”
born: May 29, 1960
twifer: “@dan”
name: “Ann”
born: Dec 5, 1995
since:
Jan 10, 2011
brand: “Volvo”
model: “V70”
Property Graph Model Components
Nodes
• The objects in the graph
• Can have name-value proper@es
• Can be labeled
RelaBonships
• Relate nodes by type and direc<on
• Can have name-value proper@es
FATHER OF
LIVES WITH
LIVES WITH
PERSON PERSON

23. RelaBonal Versus Graph Models
RelaBonal Model Graph Model
KNOWS
ANDREAS
TOBIAS
MICA
DELIA
Person Friend Person-Friend
ANDREAS
DELIA
TOBIAS
MICA

24. Cypher: Powerful and Expressive Query Language
KNOWS
Dan Ann
MATCH (:Person { name:“Dan”} ) -[:KNOWS]-> (:Person { name:“Ann”} )
LABEL PROPERTY
NODE NODE
LABEL PROPERTY

25. MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total
Express Complex Queries Easily with Cypher
Find all direct reports and how
many people they manage,
up to 3 levels down
Cypher Query
SQL Query

26. Connectedness and Size of Data Set
Response Time
0 to 2 hops
0 to 3 degrees
Thousands of connec<ons
Tens to hundreds of hops
Thousands of degrees
Billions of connec<ons
Rela<onal and
Other NoSQL
Databases
Neo4j
Neo4j is
1000x faster
Reduces minutes
to milliseconds
Benchmark real-Bme query performance
Neo4j Vs Rela<onal and other NoSQL Databases

27. Advantages using Neo4j Graph DB
Time to market Flexibility Opera<onal Efficiency
Query speed /
Performance for
connected data
Schema free
Traversing vs Joins
Cypher query
language

28. What is a NaBve Graph Database?
Applica<on
MATCH (:Person { name:“Dan”} )
-[:KNOWS]->
(:Person { name:“Ann”} )
Processing
Engine

29. Agenda
• Drivers for change in enterprises
• Requirements created for polyglot architecture
• Tradi<onal approaches
• Benefits of using graph technology
• Neo4j as na<ve GraphDB
• Architectures and Ecosystems
• Case studies
• Conclusions
Admin

30. PaTerns in Neo4j architectures
• Primary Store
• Reads + Write
• Secondary Store
• Mostly read, scheduled batch updates
• ODS / MDS
• Mostly read, scheduled batch updates + integra<on logic

31. Primary Store case: Telenor
• Iden<ty and access management problem. Need to compute
resource authoriza<on in real <me
Was taking up to 20 min for large customers -> precalcula<on & cache ->
stale data

32. Original architecture
500 requests / sec
Middleware services
Backend Backend Backend
Channel Channel Channel 42 channels
35 systems

33. New architecture
500 requests / sec
Middleware services
Backend Backend Backend
Channel Channel Channel 42 channels
35 systems

34. Did it work?
• Response <mes reduced to seconds and milliseconds
• Code maintainability improved. Access rules in Cypher as graph
paferns.
"The Neo4j graph database gives us dras@cally improved
performance and a simple language to query our connected
data" – Sebas<an Verheughe, Architect & Developer

35. Secondary Store Use Case : Shutl/eBay
Wanted to come up with quotes for delivery slots.
It took 2 seconds to come up with a quote for a single delivery slot and
they wanted to extend the service to show 20 poten<al delivery slots
Original architecture was a monolith based on rails

36. Original architecture
Requests
quote
Rails
monolith Relational

37. SoluBon
Broke applica<on up into services e.g. quo<ng, booking, feedback.
Key goal was to improve the speed of quo<ng and neo4j backed this
service

38. New architecture
Requests
quote
Quo<ng
service Relational
Booking
service
Feedback
service
Nightly job to
populate
ETL

39. Did it work?
• Quo<ng for 20 <me
windows down from
82,000 ms to 80ms
• Code complexity
much reduced
“We found Neo4j to be literally thousands of
0mes faster than our prior MySQL solu<on, with
queries that require 10-100 <mes less code.
Today, Neo4j provides eBay with func<onality
that was previously impossible.”
Volker Pacher, Senior Developer, eBay

40. Who else uses Neo4j as a data source to feed mission
criBcal applicaBons?
• World’s largest parcel service
• They no<ced that they would not have been able to deliver packages
by Christmas 2013
• Hierarchical rou<ng system (RDBMS backed)
• Replaced it with Neo4j using lateral rou<ng
• Value:
• Minimized <me
• Maximized usage of road network
• Recommenda<on engines
• Fraud detec<on / preven<on architectures

41. Neo4j
Web
App Data
Integra<on
RDBMS
(Oracle, MySQL,
DB2, HANA …)
Management Console
(E.g BI Tools such as
Tableau, Qlik, BO,
MicroStrategy etc)
Fraud
Analyst
Machine2Machine
generated ac<ons
Alert
Incoming Events
CRM System
Opera<onal
System
External Data
Sample Architecture for Fraud DetecBon

42. Sample Architecture for Internet Retail
RecommendaBons
Neo4j
Internet
Shop
Data
Integra<on
Invoicing
Shipping
etc
User
Opera<onal System
External Data
Current Profiles

43. ODS / MDS case: Gov. Agency
• The graph integrates data from mul<ple sources
• ODS: Addi<onal opera<ons on the integrated data
• MDS: Provide reference data to other opera<onal systems
• European government agencies

44. TradiBonal ODS/MDS architecture
Relational
ODS
Data
Source 1
Data
Source 2
Semi /
unstructured
data source
ETL
Opera<onal BI
Relational
MDS
EDW
Opera<onal
Systems

45. Graph based ODS/MDS architecture
ETL
Real <me connected data
analysis and explora<on
EDW
Opera<onal
Systems
Data
Source 1
Data
Source 2
Semi /
unstructured
data source

46. Common Insurance Scenario
Internal
users
Data Layer
Business
Logic
- Data Consumers are
employees or Sales agents
- Built on Mainframes
- Rich Business Logic
- During Office Hours
suppor<ng OLTP queries
- Nightly batch runs
- No real <me support
24x7
availability
Business
Logic
Suppor<ng RT
Data Layer
“Yesterday”
“Today”

47. IoT Touchpoints
Device
Management
Support Devices Provision Devices
Security /
Access Mgmt
Grant Access
Request Access
MDM
Layer
Supplier
Master
Customer
Master
Product
Master
Support
Master
…
Revenue
Sharing

48. Conclusions:
• Polyglot architectures are required to sa<sfy the needs exis<ng and coming
• Neo4j na<ve graph database provides a compete<ve edge:
• Agility
• Flexibility
• Opera<onal Efficiency
• Time to Market
• More informa<on:
• www.neo4j.com
• Stefan.Kolmar@neo4j.com

49. Neo Technology
The Graph Database Leader
September 2016
Stefan Kolmar
Director Field Engineering