Leveraging relations at scale with Neo4j

alberto@graphenedb.com | @albertoperdomo
Leveraging relations at
scale with Neo4j
Madrid.rb - July 2013
Alberto Perdomo
GrapheneDB

About me
๏Co-founder of Aentos
๏Ruby developer
๏GrapheneDB: Neo4j as a Service

Origin of Graphs

Leonhard Euler, 1736

Königsberg Bridge
Problem

Euler’s Technique

Königsberg Problem
Graph

The Math
G=( V, E )

Types of Graphs

Undirected Graph
A
B
C
Adam
Michael
John
Example: Facebook Friendships

Directed Graph
A
B
C
Adam
Michael
John
Example: Twitter follows

Weighted Graph
A
B
C
0.6 0.8
Adam John
Star Wars: Episode IV
Example: Movie Ratings

Labeled Graph
A C
friend_of fan_of
Adam LA Lakers
Michael
fan_pagefriend_of
B
user
user
Example: Facebook friendships
+ fan pages

Property Graph
A
B
C
rated: 0.6
directed
Type: Cast Member
Name: George Lucas
Born_At: 1944-05-14
Type: Movie
Title: Star Wars Episode IV - A New Hope
Release: 1977
Type: User
Name: Adam
Age: 34
Country: USA
wrote
Example: IMDB

Graph Databases

Graph DB: Deﬁnition
๏Uses graph as primary data structure
๏Property graph: store data as nodes,
relations and properties

Graph DBs
vs
other DBs

A graph can be modeled
with almost any technology

Mysql vs Neo4j
๏1M users
๏Friends of friends for 1K users
Depth Execution Time – MySQL Execution Time – Neo4j
2 0. 016 0. 010
3 30. 267 168
4 1, 543. 505 1. 359
5 Not Finished in 1 Hour 2. 132
http://www.neotechnology.com/how-much-faster-is-a-graph-database-really/,
http://www.manning.com/partner/

Conventional DBs
๏Index lookup to ﬁnd out adjacent nodes
๏Depends on total number of vertices
and edges in DB (global)

Graph DB: Deﬁnition
๏Any system that provides index-free
adjacency[1]
๏Linear cost to retrieve adjacent nodes:
depends on the number of local neighbours
[1] http://www.slideshare.net/slidarko/problemsolving-using-graph-traversals-searching-scoring-ranking-and-recommendation

While DB grows, cost of
local step remains the same

Modeling connected
data is natural

When use a graph?

High density of
relations

A search engine for
relations

Graph analysis
๏Recommend vertices to user x
๏Search for y given x
๏Score x given its local neighbourhood
๏Rank x relative to y

Recommendations

Social Graph
A representation of the
relationship between people and
other people

Social Graph
๏Facebook
๏Twitter
๏LinkedIn
“Since you have
many friends in
common, you might
know fellow X.”

Interest Graph
A representation of the
relationship between people
and things

Interest Graph
๏Pinterest
๏Instagram
๏Quora
๏Spotify
“A lot of people who
like x like you, also
like y, too.”

Pinterest Interest Graph
http://engineering.pinterest.com/post/55272557617/building-a-follower-model-from-scratch

e-commerce

e-commerce
Upselling

Recommendations
bought
Many users
Star Wars I DVD
bought
C
A
B
looking at
A user
Star Wars Trilogy
DVD Pack
“Customers who bought a, also bought b”

Rank x
๏Rank nodes based on their
neighbourhood/network
๏Klout, PageRank

Geospatial problems
๏Travelling Salesman
๏Route for delivery of parcels
๏Optimize route for duration, distance,
trafﬁc ﬂow, etc.
๏Must not be physical path, example:
connecting people

Recognize patterns
๏Fraud detection
๏Debt compensation systems
๏Text analysis
๏Chain of exchanges

Visualize connected
data

Your domain model
determines what you
can do

High chances your
data is a graph

The Neo4j
Graph Database

Data modeling

White board

Then Add Complexity

Process, Tips
๏Model facts as nodes
๏Use relations to model relations
between facts
๏Refactor - schema-less !

Neo4j
๏Graph DB written in Java
๏Java API + HTTP/REST + Embedded
๏Full ACID
๏Built-in indexing (or roll your own)
๏Scale: 32B nodes, 32B relations

The Cypher Query
Language

Cypher
๏Neo4j’s graph query language
๏Declarative pattern matching
๏“SQL for graphs”
๏ASCII art

Pattern matching

Basic Syntax
A B
(a) --> (b)

Basic Syntax
START a=node(*)
MATCH (a)-->(b)
RETURN a,b;

(a) --> (b)

Relations
(a) -[:ACTED_IN]-> (b)
A B
ACTED IN

Syntax
START a=node(*)
MATCH (a)-[:ACTED_IN]->(b)
RETURN a.name, b.title;

Syntax
START a=node(*)
MATCH (a)-[r:ACTED_IN]->(b)
RETURN a.name, r.roles, b.title;

Syntax
(a) --> (b) <-- (c)
A B C

Syntax
START a=node(*)
MATCH (a) -[:ACTED_IN]->(m)<-[:DIRECTED]- (d)
RETURN a.name, m.title, d.name;

Sort & Limit
START a=node(*)
MATCH (a) -[:ACTED_IN]->(m)<-[:DIRECTED]- (d)
RETURN a.name, m.title, d.name
ORDER BY(count) DESC
LIMIT 5;

Starting point: All nodes
START n=node(*)
RETURN n;

Starting point: Where
START n=node(*)
WHERE has (n.name) AND n.name = “George Lucas”
RETURN n;

Starting point: Auto
Index
START n=node:node_auto_index(name=“George Lucas”)
RETURN n;

Starting point: multiple
nodes
START lucas=node:node_auto_index(name=“George Lucas”),
ford=node:node_auto_index(name=”Harrison Ford”)
MATCH (lucas) -[:DIRECTED]-> (m) <-[:ACTED_IN]- (ford)
RETURN m.title;

Multiple relations
MATCH (a)-[:ACTED_IN|DIRECTED]->()

Constraints with
comparison
START a=node:node_auto_index(name=“Alberto Perdomo”)
MATCH (a) -[:KNOWS]-> (b)
WHERE b.born < a.born
RETURN a.name;

Contraints with
patterns
MATCH (alberto)-[:KNOWS*2]->(fof)
WHERE NOT((ferblape)-[:KNOWS]-(fof))

Variable length paths
MATCH (alberto)-[:KNOWS*2]->(fof)

Agreggation
๏count(x)
๏min(x)
๏max(x)
๏avg(x)
๏collect(x)
๏ﬁlter(x)

Updating the graph
๏Create, Set, Delete nodes
๏Create, Set, Delete relations

Neo4j: More features

Built-in Graph Algos
๏shortest path
๏allSimplePaths
๏allPaths
๏dijkstra

Extending Neo4j: Plugins
๏Provides extra API endpoints to run
external code. JAR ﬁles.
๏Neo4j-Spatial
๏Neo4j-Sparql

Neo4j from Ruby
๏ Neography
๏ Wrapper around REST API
๏ Neo4j.rb:
๏ Language binding for JRuby
๏ ActiveModel, Mixins
๏ Embedded Neo4j w/ GPL license (not only?)
๏ Other?

Neography
# Node creation:
node1 = @neo.create_node("age" => 31, "name" => "Max")
node2 = @neo.create_node("age" => 33, "name" => "Roel")
# Node properties:
@neo.set_node_properties(node1, {"weight" => 200})
# Relationships between nodes:
@neo.create_relationship("coding_buddies", node1, node2)
# Get node relationships:
@neo.get_node_relationships(node2, "in",
"coding_buddies")
# Use indexes:
@neo.add_node_to_index("people", "name", "max", node1)
@neo.get_node_index("people", "name", "max")
# Cypher queries:
@neo.execute_query("start n=node(0) return n")

Neo4j.rb ActiveModel
class User < Neo4j::Rails::Model
attr_accessor :password
attr_accessible :email, :password, :password_confirmation
after_save :encrypt_password
email_regex = /A[w+-.]+@[a-zd-.]+.[a-z]+z/i
# add an exact lucene index on the email property
property :email, index: :exact
has_one(:avatar).to(Avator)
accepts_nested_attributes_for :avatar, allow_destroy: true
end

Neo4j.rb Mixin
class Person
include Neo4j::NodeMixin
property :name, index: :exact
property :city
has_n :friends
has_one :address
end

Neo4j Licensing
๏ Community: GPL
๏ Advanced: Commercial + AGPL
๏ Monitoring
๏ Support
๏ Enterprise: Commercial + AGPL
๏ Monitoring + HA clustering + Online backups
๏ Support

Getting Started
๏www.neo4j.org/learn/try
๏www.neo4j.org/download
๏download -> unpack -> start
๏http://localhost:7474

Built in Web Admin
๏Stats
๏Console & browser
๏Indexes

Neo4j Resources
๏ Code & Issues: github.com/neo4j/neo4j
๏ Resources: www.neo4j.org/learn
๏ Mailing List: groups.google.com/forum/#!forum/neo4j
๏ Questions: stackoverﬂow.com/questions/tagged/neo4j
๏ Meetups: www.neo4j.org/participate/events/meetups
Free download:
http://graphdatabases.com/

GrapheneDB:
Neo4j as a Service

@albertoperdomo
alberto@graphenedb.com
Thanks!

Leveraging relations at scale with Neo4j

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (16)

Plus de Alberto Perdomo

Plus de Alberto Perdomo (14)

Dernier

Dernier (20)

Leveraging relations at scale with Neo4j