Kevin Van Gundy gives and overview of the Neo4j graph platform at Washington DC Graphtour

1. The Neo4j Universe
Kevin Van Gundy
April 2018
in 44 minutes or less

2. We're going to
connect a lot of
dots…very quickly

4. Buckle Up!

5. What We'll Discuss Today:
Data Ecosystems: Where it All Fits
Neo4j, the Database: How and Why it Works
Beyond the Database: Building Graph Native Organizations
Our Long-Term Vision for a Connected Enterprise

6. The State of Data in 2018

8. What Happened?

9. Amazon, Google, and Facebook.

10. DATA

11. StructureData Storage
ON STAGE
BEHIND THE SCENE
`
"Customer Journey"

12. CEO
StructureData Storage
ON STAGE
BEHIND THE SCENE
`
"Customer Journey"

16. StructureData Storage
ON STAGE
BEHIND THE SCENE
`
"Customer Journey"

17. StructureData Storage
ON STAGE
BEHIND THE SCENE
"Customer Journey"

18. graph database
applications
services
products
raw data
data structurekey value storecolumn storedocument storerelational database ?relational database
Conway's Law
Any organization that designs a system or
application are constrained to produce a
design whose structure is a copy of the
organization's (data) communication
structure.

19. graph database
applications
services
products
raw data
data structurekey value storecolumn storedocument storerelational database
Conway's Law
Any organization that designs a system or
application are constrained to produce a
design whose structure is a copy of the
organization's (data) communication
structure.

20. graph database
applications
services
products
raw data
data structurekey value storecolumn storedocument store
Conway's Law
Any organization that designs a system or
application are constrained to produce a
design whose structure is a copy of the
organization's (data) communication
structure.

21. graph database
applications
services
products
raw data
data structurekey value storecolumn store
Conway's Law
Any organization that designs a system or
application are constrained to produce a
design whose structure is a copy of the
organization's (data) communication
structure.

22. graph database
applications
services
products
raw data
data structurekey value store
Conway's Law
Any organization that designs a system or
application are constrained to produce a
design whose structure is a copy of the
organization's (data) communication
structure.

23. graph database
applications
services
products
raw data
data structure
Conway's Law
Any organization that designs a system or
application are constrained to produce a
design whose structure is a copy of the
organization's (data) communication
structure.

24. We've Entered the Era of the Data Structure

25. On Stage
Business
Processes
Behind the Scene
Data
Structure

26. Hierarchies
On Stage
Business
Processes
Behind the Scene
Data
Structure
Traditional Supply Chain Information

27. Hierarchies
On Stage
Business
Processes
Behind the Scene
Data
Structure
Traditional Supply Chain Information

28. Hierarchies
On Stage
Business
Processes
Behind the Scene
Data
Structure
Traditional Supply Chain Information

29. On Stage
Behind the Scene
Business
Processes
Data
Structure
Linear Supply Chain InformationOrganizations

30. On Stage
Behind the Scene
Business
Processes
Data
Structure
InformationOrganizations Dynamic Supply Chain

31. On Stage
Behind the Scene
Business
Processes
Data
Structure
Organizations Dynamic Supply Chain Knowledge

32. What is Good at What?

33. Real-Time
Storage & Retrieval
RDBMS & NoSQL Databases
Store & Retrieve
What is Good at What?

34. Real-Time
Storage & Retrieval
RDBMS & NoSQL Databases
Store & Retrieve
Hadoop/Spark
Aggregates & Filters
Long-Running Queries
Aggregation & Filtering
What is Good at What?

35. Real-Time
Storage & Retrieval
RDBMS & NoSQL Databases
Store & Retrieve
Hadoop/Spark
Aggregates & Filters
Long-Running Queries
Aggregation & Filtering
Neo4j
Reveal Connections
Real-Time
Connected Insights
What is Good at What?

36. Neo4j: The Database Purpose
Built for Connected Data

37. A Brief History…

38. On Stage Behind the Scenes
Personal Computer: Mainstream Movement Toward Client-Server
SQL and RDBMs Consumes Market
Teeny Tiny RAM
Paper Rules
Spinning Patters

39. On Stage
Consumer Internet
Behind the Scenes
Distributed Systems Become Commonplace
GBs RAM
SSDs prohibitively expensive
Batch Compute
Commodity Hardware

40. On Stage
Mobile
Behind the Scenes
NoSQL and Cloud Native Deployment
Cloud / On Demand Hardware
Map Reduce Your Insights
OSS Rules the Roost

41. On Stage Behind the Scenes
The Internet of Me:
Mainstream AI
Behind the Scenes
Graphs
Abundant Cheap RAM TB+
FGPAs for Algos
Dynamic real world systems

42. The Property Graph Model

43. A way of representing data
DATA DATA

44. Relational
Database
Good for:
• Well-understood data structures that
don’t change too frequently
A way of representing data
• Known problems involving discrete
parts of the data, or minimal
connectivity
DATA

45. Graph
Database
Relational
Database
A way of representing data
Good for:
• Dynamic systems: where the data
topology is difficult to predict
• Dynamic requirements:
that evolve with the business
• Problems where the relationships in
data contribute meaning & value
Good for:
• Well-understood data structures that
don’t change too frequently
• Known problems involving discrete
parts of the data, or minimal
connectivity

46. CAR
Anatomy of The Property Graph Model
Nodes
• Can have name-value properties
• Can have Labels to classify nodes
Relationships
• Relate nodes by type and direction
• Can have name-value properties
PERSON PERSON
27

47. CAR
name: “Dan”
born: May 29, 1970
twitter: “@dan”
name: “Ann”
born: Dec 5, 1975
brand: “Volvo”
model: “V70”
Anatomy of The Property Graph Model
Nodes
• Can have name-value properties
• Can have Labels to classify nodes
Relationships
• Relate nodes by type and direction
• Can have name-value properties
PERSON PERSON
27

48. CAR
name: “Dan”
born: May 29, 1970
twitter: “@dan”
name: “Ann”
born: Dec 5, 1975
brand: “Volvo”
model: “V70”
Anatomy of The Property Graph Model
Nodes
• Can have name-value properties
• Can have Labels to classify nodes
Relationships
• Relate nodes by type and direction
• Can have name-value properties
MARRIED TO
PERSON PERSON
27

49. CAR
name: “Dan”
born: May 29, 1970
twitter: “@dan”
name: “Ann”
born: Dec 5, 1975
brand: “Volvo”
model: “V70”
Anatomy of The Property Graph Model
Nodes
• Can have name-value properties
• Can have Labels to classify nodes
Relationships
• Relate nodes by type and direction
• Can have name-value properties
MARRIED TO
LIVES WITH
PERSON PERSON
27

50. CAR
DRIVES
name: “Dan”
born: May 29, 1970
twitter: “@dan”
name: “Ann”
born: Dec 5, 1975
since:
Jan 10, 2011
brand: “Volvo”
model: “V70”
Anatomy of The Property Graph Model
Nodes
• Can have name-value properties
• Can have Labels to classify nodes
Relationships
• Relate nodes by type and direction
• Can have name-value properties
MARRIED TO
LIVES WITH
OW
NS
PERSON PERSON
27

51. The Whiteboard Model Is the Physical Model

52. The Whiteboard Model Is the Physical Model

53. The Whiteboard Model Is the Physical Model

54. Project Agility Benefits
• Easily understood
• Easily evolved
• Easy collaboration
between business and IT
The Whiteboard Model Is the Physical Model

55. On Building a Platform

56. Graph
Transactions
The Neo4j Graph Platform Vision

57. Neo4j Core Datebase
1
Index-Free Adjacency
In memory and on flash/disk
2
vs
ACID Foundation
Required for safe writes
3
Full-Stack Clustering
Causal consistency
Language, Drivers, Tooling
Developer Experience,
Graph Efficiency, Type Safety
5
Graph Engine
Cost-Based Optimizer, Graph
Statistics, Cypher Runtime, …
4

58. At Write Time:
Data is connected
as it is stored
Index-Free Adjacency:

59. At Write Time:
Data is connected
as it is stored
At Read Time:
Lightning-fast retrieval of data and relationships
via pointer chasing
Index-Free Adjacency:

60. How Fast is Fast?
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

61. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

62. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
MySQL 1,000 2,000 ms
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

63. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
MySQL 1,000 2,000 ms
Neo4j 1,000 2 ms
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

64. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
MySQL 1,000 2,000 ms
Neo4j 1,000 2 ms
Neo4j 10,000,000 2 ms
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

65. 34
Real-Time Query Performance
Neo4j Versus Rela.onal and Other NoSQL Databases
Connectedness and Size of Data Set
Response Time
0 to 2 hops
0 to 3 degrees
Thousands of connec;ons
Tens to hundreds of hops
Thousands of degrees
Billions of connec;ons
Rela;onal and
Other NoSQL
Databases
Neo4j
Neo4j is
1000x faster
Reduces minutes
to milliseconds
100s of Hops
1000s of Degrees
Billions of Connections
0 to 2 Hops
0 to 3 Degrees
Thousands of Connections
ResponseTime
t = O (1)
t=
O
(log(n))
Neo4j : Index-free adjacent traversals
JOIN:IndexScans
Connected-Data Query Performance

66. Querying the graph
Person
Location
Pattern: Persons who live in San Francisco
city: “San Francisco”
LIVES_IN

67. Querying the graph
Person
Location
city: “San Francisco”
LIVES_IN
( ) ( )
Pattern: Persons who live in San Francisco

68. city: “San Francisco”
Querying the graph
Person
Location
city: “San Francisco”
LIVES_IN
( ) ( )p loc
Pattern: Persons who live in San Francisco

69. city: “San Francisco”
Querying the graph
Person
Location
city: “San Francisco”
LIVES_IN
( )loc( )p :Person
Pattern: Persons who live in San Francisco

70. city: “San Francisco”
Querying the graph
Person
Location
city: “San Francisco”
LIVES_IN
( )loc( )p:Person
Pattern: Persons who live in San Francisco

71. Querying the graph
Person
Location
city: “San Francisco”
LIVES_IN
( )loc city: “San Francisco”:Location( )p:Person
Pattern: Persons who live in San Francisco

72. {city: “San Francisco”}
Querying the graph
Person
Location
city: “San Francisco”
LIVES_IN
( )loc :Location( )p :Person
Pattern: Persons who live in San Francisco

73. Querying the graph
Person
Location
city: “San Francisco”
LIVES_IN
( loc :Location( )p:Person ->-
Pattern: Persons who live in San Francisco
{city: “San Francisco”} )

74. Querying the graph
Person
Location
city: “San Francisco”(loc :Location( )p:Person ->- [:LIVES_IN]
Pattern: Persons who live in San Francisco
{city: “San Francisco”} )

75. {city: “San Francisco”}
Querying the graph
Person
Location
( )loc :Location( )p:Person ->- [:LIVES_IN]MATCH
RETURN p
Pattern: Persons who live in San Francisco

76. Cypher vs. SQL

77. (SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM (
SELECT manager.pid AS directReportees, 0 AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
UNION
SELECT manager.pid AS directReportees, count(manager.directly_manages) AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT manager.pid AS directReportees, count(reportee.directly_manages) AS count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT manager.pid AS directReportees, count(L2Reportees.directly_manages) AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM (
SELECT manager.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
UNION
SELECT reportee.pid AS directReportees, count(reportee.directly_manages) AS count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT depth1Reportees.pid AS directReportees,
count(depth2Reportees.directly_manages) AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM(
SELECT reportee.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT L2Reportees.pid AS directReportees, count(L2Reportees.directly_manages) AS
count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT L2Reportees.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
)

78. MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate, count(report) AS Total;

79. MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total;

80. Cypher: Key Benefits
Example HR Query in SQL
The Same Query using Cypher
MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total
Project Impact
Less time writing queries
• More time understanding the answers
• Leaving time to ask the next question
Less time debugging queries:
• More time writing the next piece of code
• Improved quality of overall code base
Code that’s easier to read:
• Faster ramp-up for new project members
• Improved maintainability & troubleshooting

81. Graph
Transactions
The Neo4j Graph Platform Vision

82. Graph
Analytics
Graph
Transactions
AI
The Neo4j Graph Platform Vision

83. Graph
Analytics
Graph
Transactions
Data Integration
AI
The Neo4j Graph Platform Vision

84. Common Integration Patterns

85. Common Integration Patterns
From Tabular Data
To Connected Data

86. Common Integration Patterns
From Disparate Silos
To Cross-Silo Connections
From Tabular Data
To Connected Data

87. Common Integration Patterns
From Disparate Silos
To Cross-Silo Connections
From Tabular Data
To Connected Data
From Data Lake Analytics to
Real-Time Operations

88. Graph
Analytics
Graph
Transactions
Data Integration
AI
The Neo4j Graph Platform Vision

89. DEVELOPERS Graph
Analytics
Data Integration
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions
The Neo4j Graph Platform Vision

90. Native Language Drivers
• Java
• .Net
• Python
• Javascript
• more to come…
• Massive Community Support (Go, Ruby R, Perl, Clojure, C/C++…)
• Partners like GraphAware (PHP Client)
Drivers and APIs

91. DEVELOPERS Graph
Analytics
Data Integration
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions
The Neo4j Graph Platform Vision

92. The Neo4j Graph Platform Vision
BUSINESS USERS
DEVELOPERS Graph
Analytics
Data Integration
Discovery & Visualization
DATA
ANALYSTS
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions

93. Graph Discovery & Visualization
Software that allows users to realize insights by interacting directly with their data
Neo4j Browser
Custom / JS Libraries
Partner Applications

94. The Neo4j Graph Platform Vision
BUSINESS USERS
DEVELOPERS Graph
Analytics
Data Integration
Discovery & Visualization
DATA
ANALYSTS
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions

95. The Neo4j Graph Platform Vision
Development &
Administration
BUSINESS USERS
DEVELOPERS
ADMINS
Graph
Analytics
Data Integration
Discovery & Visualization
DATA
ANALYSTS
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions

96. Neo4j Desktop and Browser Admin Tooling

97. Development &
Administration
BUSINESS USERS
DEVELOPERS
ADMINS
Graph
Analytics
Data Integration
Discovery & Visualization
DATA
ANALYSTS
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions
The Neo4j Graph Platform Vision

98. Development &
Administration
Analytics
Tooling
BUSINESS USERS
DEVELOPERS
ADMINS
Graph
Analytics
Data Integration
Discovery & Visualization
DATA
ANALYSTS
DATA
SCIENTISTS
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions
The Neo4j Graph Platform Vision

99. Cypher for
Apache® Spark™

100. Multiple Sources, Multiple Graphs
Relational
Graph
Graph
Subgraph
Output Graph

101. Multiple Sources, Multiple Graphs
Relational
Graph
Graph
Subgraph
Output Graph
Cypher for
Apache® Spark™

102. Finds the optimal path
or evaluates route
availability and quality
Evaluates how a
group is clustered
or partitioned
Determines the
importance of distinct
nodes in the network
Neo4j Graph Algorithm Library

103. One More Thing!

104. Development &
Administration
Analytics
Tooling
BUSINESS USERS
DEVELOPERS
ADMINS
Graph
Analytics
Data Integration
Discovery & Visualization
DATA
ANALYSTS
DATA
SCIENTISTS
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions
The Neo4j Graph Platform Vision

105. Development &
Administration
Analytics
Tooling
BUSINESS USERS
DEVELOPERS
ADMINS
Graph
Analytics
Data Integration
Discovery & Visualization
DATA
ANALYSTS
DATA
SCIENTISTS
Drivers & APIs
APPLICATIONS
AI
Graph
Transactions
The Neo4j Graph Platform Vision

106. Neo4j-Cloud: Preview
The Challenge
• Neo4j without the headache of operations
• Lower cost of entry to enterprise-ready features
• Scale cost with value
The Solution
• Neo4j as a Service on the public cloud
• Cloud-hosted database in < 5 minutes
• Automated backup, upgrades, and restores
The Preview Program for Graph Tour
• Looking for a wide variety of participants
• Join the list at http://neo4j.com/cloud

107. • Graphs are fundamentally the most ergonomic and humane
way of working with data at scale
• Ecosystems are most powerful with seamless access to
adjacent technologies
• The best technology is the one that is easiest to use
Why We Believe in Building a Graph Platform

108. Thanks!