Database Pro Power Days 2010 - Graph data in the cloud using .NET

Photo: Large Magellanic Cloud, ESO

19./20. Oktober 2010
Nürnberg
www.databasepro-powerdays.de

sones

Graph Data in the cloud
using .NET
Achim Friedland <achim@sones.de> Database Pro Power Days , 10/20/2010 1

Photo: funky64, flickr

For 35 years information has been
well-defined data within some tables
jailed in closed database silos.


Photo: shamballah, flickr

The relational model and SQL have
become much too limited for open
linked data and cloud requirements.
== graph data



Application 1 Application 2
Applications can
not access, under-
stand and process
? ?
unknown relational
DB 1 DB 2
data easily.


Photo: Gephi, flickr

1. Graph-Databases



The Property-Graph Model

Vertex
Edge
Alice Friends Bob
ID = 1 since = 2009/09/21 ID = 2
Age = 21 reason = classmates Age = 23

Edge- Vertex-
Properties Properties




Direct linking
ID 1 without external
TYPE Person indices
REVISION 20101014… Bob
Name Alice ID = 2
Age 21 Age = 23

Boyfriend
Friends

FavColors Red, Green Carol
ID = 3
Address.Street 1 Infinite Loop Age = 20
Address.Town Cupertino
Close to Object- and
Document-Databases


using RDF-like semantics
+ Unambiguous identifiers
+ Named relations
+ Close to RDF molecules
ID ID http://test.com/vertices/1
rdf:type TYPE http://test.com/#person
sones:revId REVISION 20101014… Bob
foaf:name Name Alice ID = 2
foaf:age Age 21 Age = 23

person Boyfriend
Set<person> Friends

List<string> FavColors Red, Green Carol
ID = 3
XML Address Age = 20
gn:streeet Address.Street 1 Infinite Loop
gn:town Address.Town Cupertino


Graph-Databases in a cloud
• Vertices and edges are resources
• Access via e.g. http://test.com/vertices/[$id]
• Common CRUD operations (GET, POST, PUT…)

+ Atomicity
+ Statelessness
+ Idempotence
+ Parallelism
GraphDB

REST

Hypermedia • Representation must be „link-aware“
e.g. XML+XLINK, ATOM, RDFa…
• Representation should be self-describing


Photo: bombeador, flickr

Graph
2. The Object-relational
Impedance Mismatch


Inflexible relational schemata

• Expensive ALTER TABLE operations
• Entity-Attribute-Value Model ↔ RDF
• No semi-/unstructured data
XML, JSON, … hierarchies, graphs, … binary data
• No Multi-Attribute Values
List<String>, Set<Integer>, Set<Person>
• No simple way for versioned data



Relational Anti-Patterns:

• Relations via foreign key constraints
No explicit concept for relations
No index-free adjacency
• Querying relational data via JOINs is hard
Just storing a graph was never a challenge ;)
• No recursive JOINs
Inefficient query processing
(Except: Oracle’s “CONNECTED BY”)



SQL and Cloud-Readiness?

• No explicit scaling or partitioning within
the relational model
• No JOINs between different databases
and/or vendors
• No well interaction with state-of-the-art
web technologies
e.g. HTTP/REST, Hypermedia, Semantic Web



3. Benefits of Graph-Databases



The explicit graph data model
provides a higher level of abstraction
and a better understanding of the
domain model.



Index-free adjacency provides an
improved scalability, data-locality
and a superior graph traversal
performance.

( Independent of the size of the graph )


Photo: squacco, flickr

Consistency criteria and indices for
simple attributes up to complex
subgraph structures.


Photo: Khem, flickr

Traversing linked information, finding
shortest-paths, do semantic
partitions.


Photo: NASA, flickr

personal social item-related

Recommendation and discovery
of potentially interessting linked
information.


Photo: Birger Hoppe, flickr

Good integration into state-of-the-art
programming concepts and web
technologies.



Graph-Databases, REST and RDF
symantics are a solid foundation for
cloud services



4. Graph data in the cloud
using .NET / Mono


Photo: sones

sones GraphDB

• URL http://www.sones.de
• License AGPLv3
• Language C# 4.0
• Goals Management of linked data
• Concurrency MVCC
• Repl./Scaling p2p (alpha)
• Persistency Proprietary file system
• Cloud Connector for Microsoft Azure


Photo: sones

sones Architecture

GraphDS
REST, WebShell, C# API

GraphDB
GQL, Graph Traversals, Indices

GraphFS
Object Management, (De-)Serialization

Host File System / Microsoft Azure


Photo: sones

Shared nothing

GraphDB 1

GraphDS 1
GraphFS 1
User
it
depends…
Azure
GraphFS 2
GraphDS 2

GraphDB 2


Photo: sones

sones Property-Hypergraph

Edge

User Friend Bob
ID = 2
since = 2009/09/21
Hyperedge Alter = 23

Alice SET<User> Friends
ID = 1 Virtual -Edge
SetMaxNumber = 12
Alter = 21

Hyperedge-Properties
User Friend Carol
ID = 3
since = 2010/04/11
Alter = 20


Photo: sones

sones Property-Hypergraph

• Properties may include code as data
Think of stored procedures; C#: Func<…>, ExpressionTrees

• Allows hyperedge calculations be done among
the set of their edges
(GetMinWeight, SetMaxNumber, …)

Achim Friedland <achim@sones.de> Database Pro Power Days , 10/20/2010
NoSQL Frankfurt , 9/28/2010 27

Photo: Shayne Kaye, flickr

sones Graph Query Language

FROM User SELECT User.Friends.Friends.Name

• “SQL for graphs” providing a user-friendly DSL for ad-
hoc graph queries and graph discovery
• Functions and aggregates are type-safe and can be
extended by your own plug-ins, e.g.
• SELECT COUNT(User.Friends)
• SELECT User.Friends.Random(2)
• SELECT User.Friends.Name.Substring(2,5)



sones Graph Query Language
// sones gql example

CREATE VERTEX User
ADD ATTRIBUTES (String Name, SET<User> Friends)
INDICES (Name)
MANDATORY (Name)

INSERT INTO User VALUES (Name = "Alice", Age = 21)
INSERT INTO User VALUES (Name = "Bob", Age = 23)

LINK User(Name = ‘Alice') VIA Friends TO User(Name = ‘Bob')
LINK User(Name = ‘Bob') VIA Friends TO User(Name = ‘Alice‘)



C# API
// C# API type creation

var _Person = _GraphDB.TypeManager.
CreateVertex(„Person“).
AddString(„Name“, mandatory: true, indexed: true).
AddLoop(„Friends”, hyperEdge: true).
execute();

Type _PersonT = _GraphDB.TypeManager.
GenerateType(_Person);



C# API
// C# API vertex initialization

Person _Alice = _GraphDB.TypeManager.ActivateVertex(
_Person, new VertexUUID(1));

_Alice.Name = „Alice“;

dynamic _Alice2 = _Alice;
_Alice.Age = 21;
_Alice.bdayparty = (Action) (() => { _Alice.Age++; });

_Alice.bdayparty();



C# API
// sones C# API example

var _Friends = new GraphAttribute(„Friends“, Type: „foaf:knows“);

var _Bob = _GraphDB.TypeManager.ActivateVertex(
_Person, new VertexUUID(2));

_Alice.Link(_Friends, _Bob);
_Bob.Link(_Friends, _Alice);



C# API
// Graph Traversals

Public T TraverseVertex<T> (
IVertex myStartVertex,
TraversalOperation TraversalOperation =
TraversalOperation.BreathFirst,
Func<IVertex, IEdge, Boolean> myFollowThisEdge = null,
Func<IVertex, Boolean> myMatchEvaluator = null,
Action<IVertex> myMatchAction = null,
Func<TraversalState, Boolean> myStopEvaluator = null,
Func<IEnumerable<IVertex>, T> myWhenFinished = null)
{
// Traverse the graph
}


sones
For more information…

achim@sones.de
http://www.twitter.com/ahzf
http://www.twitter.com/graphdbs


Database Pro Power Days 2010 - Graph data in the cloud using .NET

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (12)

More from Achim Friedland

More from Achim Friedland (12)

Recently uploaded

Recently uploaded (20)

Database Pro Power Days 2010 - Graph data in the cloud using .NET